# Creating a New Pipeline
This guide walks you through the end-to-end process of creating and executing a pipeline, from naming and resource setup to viewing execution logs and previewing loaded data.
### Step 1: Define Pipeline Name and Resources
* Navigate to the **Pipelines** section in the workspace.
* Click **Create** to begin creation of a new pipeline.
* Enter a unique **Pipeline Name**.
* Assign the required **compute and storage resources**.
* Example: Select the compute cluster or container environment.
* Specify storage options for intermediate and final data to control how many system resources the pipeline can utilize. Choose the appropriate **Resource Allocation** level:
* **Low**
* **Medium**
* **High**
* **Schedule the Pipeline** (Optional)
* Enable **Schedule Pipeline** by checking the box.
* Define the schedule using:
* Cron expression (e.g., `0 0 0/1 1/1 * ? *`) or
* Frequency tabs (Minutes, Hourly, Daily, etc.)
* Specify time, frequency, and time zone.
* Click **Save** to create the pipeline shell.
### Step 2: Add Components
* In the newly created pipeline, open the **Pipeline Editor**.
* Click the **Add Component/Event** icon.
* The **Components & Events** panel opens on the right.
* Use the search bar in the **Components** tab to locate a component.
* Drag and drop the selected component onto the canvas.
* Repeat for additional components to design your workflow.
### Step 3: Configure Components
* Select a component on the canvas.
* In the configuration panel, complete the following:
* **Basic Information Tab** (opens by default)
* **Meta Information Tab** (adjacent tab)
* Click the **Validate Connection** icon.
* A success notification confirms the validation.
* Click the **Save Component in Storage** icon to persist the configuration.
### Step 4: Link Components with Events
* Go to the **Events** tab in the right panel.
* Click **Add New Event**.
* The **Create Kafka Event** dialog opens.
* Enter the required event details and click **Add Kafka Event**.
* The event is added to the Events list.
* Drag the event onto the canvas and connect it between the producer and consumer components.
* If **Auto-connect** is enabled (default), the system automatically links the event.
### Step 5: Design the Data Flow
* Arrange components and events to reflect the intended pipeline logic.
* Producers send output to events.
* Consumers receive input from events.
* Ensure all components are linked to at least one input and output event.
### Step 6: Execute the Pipeline
* Click the **Run Pipeline** icon in the toolbar.
* Confirm execution settings (environment, parallelism, etc.).
* Monitor the pipeline’s progress in the execution window.
### Step 7: Access Running Logs
1. During execution, open the **Logs** tab.
2. Review logs for:
* Connection validations
* Event triggers
* Runtime errors or warnings
3. Use logs to troubleshoot any issues.
### Step 8: Preview Loaded Data
1. After successful execution, open the **Data Preview** tab.
2. Inspect ingested or transformed data directly in the interface.
3. Validate schema, sample rows, and record counts.
### Next Steps
* Iterate on pipeline design by adding more components or refining configurations.
* Schedule the pipeline for automated runs once it is validated.
* Refer to the **Events Guide** and **Component Reference** for advanced usage patterns.
{% hint style="info" %}
**Please Note:** Click the **Delete** icon from the **Pipeline Workflow Editor** page to delete the selected pipeline. The deleted Pipeline gets removed from the list of pipelines, but it will be added to the **Trash** page.
{% endhint %}
---
# Agent Instructions: Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.
Perform an HTTP GET request on the current page URL with the `ask` query parameter:
```
GET https://docs.bdb.ai/bdb-user-documentation/platform-modules/11.0/data-engineering/data-pipelines/creating-a-new-pipeline.md?ask=
```
The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.
Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.