# Pipeline Toolbar The ***Pipeline Editor*** contains a Toolbar at the top right side of the page with various options to be applied on a Pipeline workflow. ![Pipeline Toolbar](https://972575688-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FRYq1HgffNfbnIMWPu1D5%2Fuploads%2F3kUQdwpSDTrZihiZg606%2Fimage.png?alt=media\&token=b695135a-f95c-4daa-a6a2-e2737faa7f91) This page aims to explain various functions provided on the Pipeline toolbar. ### Toggle Log Panel The Toggle Log Panel displays the ***Logs*** and ***Advanced Logs*** tabs for the Pipeline Workflows. * Navigate to the ***Pipeline Editor*** page. * Click the ***Toggle Log Panel*** icon on the Pipeline. ![](https://972575688-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FRYq1HgffNfbnIMWPu1D5%2Fuploads%2FEpGsHS4tY7vMVwmmJd7s%2Fimage.png?alt=media\&token=6beec129-f603-4e6c-9ac1-249dddf05c98) * A ***Log*** panel toggles displaying the collective component logs of the pipeline under the **Logs** tab. ![Logs tab displaying the collective logs for a Pipeline](https://972575688-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FRYq1HgffNfbnIMWPu1D5%2Fuploads%2FMeCiLnsVW4X4LaEr6BqT%2Fimage.png?alt=media\&token=50bb413f-da4e-4a13-9c63-3c039a35eeaa) * Select the ***Advanced Logs*** tab to display the status of the component containers. ![The Advanced Log displayed inside the toggle Log panel.](https://972575688-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FRYq1HgffNfbnIMWPu1D5%2Fuploads%2Fq45EkVPrCWw918Iv5CUW%2Fimage.png?alt=media\&token=0816a8a8-699c-4ee4-9e2c-bbe7b9661430) ### Object Browser: Search Component in Pipelines This feature helps the user to search a specific component across all the existing pipelines. The user can drag the required components to the pipeline editor to create a new pipeline workflow. * Click the ***Search Component in pipelines*** icon from the header panel of the Pipeline Editor. ![Search Component in Pipelines icon](https://972575688-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FRYq1HgffNfbnIMWPu1D5%2Fuploads%2FL8hqSn8DiJW2JE4Y0mcb%2Fimage.png?alt=media\&token=dbe6846c-016d-48c7-a951-6eb238ba0f52) * The ***Object Browser window*** opens displaying all the existing pipeline workflows. ![Object Browser window](https://972575688-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FRYq1HgffNfbnIMWPu1D5%2Fuploads%2FfR4GlmEhMsT3bepw4nV0%2Fimage.png?alt=media\&token=f98983c6-ee67-431b-9077-029008f574b5) * The user can search a component using the ***Search Component*** space. * The user gets prompt suggestions while searching for a component. ![](https://972575688-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FRYq1HgffNfbnIMWPu1D5%2Fuploads%2FufC2pTJuQ5E1BftvbKnY%2Fimage.png?alt=media\&token=369f9750-f148-4e9a-8648-0a7980cf9d2a) * Once the component name is entered, the pipeline workflows containing the searched pipeline get listed below. ![](https://972575688-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FRYq1HgffNfbnIMWPu1D5%2Fuploads%2FYJYIwptQ4PAhwmd0VJZ8%2Fimage.png?alt=media\&token=a3dca831-f635-4468-b207-644a358af095) * The user can click the expand/ collapse icon to expand the component panel for the selected pipeline. ![](https://972575688-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FRYq1HgffNfbnIMWPu1D5%2Fuploads%2F0eArv4u2ShIka9NNrDwP%2Fimage.png?alt=media\&token=14959cdf-9f26-4aa2-b501-ee4cd155cac5) * The user can drag a searched component from the Object Browser and drop to the ***Pipeline Editor*** ***canvass***. ![](https://972575688-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FRYq1HgffNfbnIMWPu1D5%2Fuploads%2Fv9qaZmQPywlJSiAhRRQ6%2Fimage.png?alt=media\&token=0419f456-15ad-474a-a123-3ee4163a680a) ### Toggle Event Panel The user can access the ***Toggle Event Panel*** to create a new Event. We have two options in the Toggle Event Panel: 1. Private (Event/ Kafka Topic) 2. Data Sync ![The Toggle Event Panel](https://972575688-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FRYq1HgffNfbnIMWPu1D5%2Fuploads%2FeqZXbptBqFMIg1pLlIMD%2Fimage.png?alt=media\&token=dca9c2bf-d567-4746-bcdb-510c2255b8cf) #### **Private (Event)** The user can create an Event (Kafka Topic) that can be used to connect two pipeline components. * Navigate to the toggle ***Event*** panel. * Click the ***Add New Event*** icon. ![Accessing the Add New Event icon from the Event panel](https://972575688-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FRYq1HgffNfbnIMWPu1D5%2Fuploads%2FhVN4RRlV5DdAicMo15qc%2Fimage.png?alt=media\&token=a667fa16-c2fd-4131-9932-e178cf91c5d8) * The ***New Event*** dialog box opens. * Provide the required information. * Slide the given button to enable the event mapping. * Provide a display name for the event (A default name based on the pipeline name appears for the Event). * Select the Event Duration from the drop-down menu (It can be set from 4 to 168 hours as per the given options). * Number of partitions (You can choose out of 1 to 50). * Number of outputs (You can choose out of 1-3) (The maximum number of outputs must not exceed the no. of Partition). * Enable the ***Is Failover?*** option if you wish to create a failover Event. * Click the ***Add Event*** option to save the new Event. ![](https://972575688-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FRYq1HgffNfbnIMWPu1D5%2Fuploads%2FbmPBbx5kLQXkGkOvvSnm%2Fimage.png?alt=media\&token=074eeede-ba82-4744-a74c-2bf1600d6181) * A confirmation message appears. * The new Event gets created and added to the ***Event Panel***. ![](https://972575688-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FRYq1HgffNfbnIMWPu1D5%2Fuploads%2F7i9w8p1IwsQGBFzI7TuE%2Fimage.png?alt=media\&token=8c5c56d3-ad91-48f9-ac1e-b789aef1aaf1) * Drag and drop the ***Event*** from the ***Event Panel*** to the workflow editor. ![Dragging an Event to the Workflow Editor](https://972575688-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FRYq1HgffNfbnIMWPu1D5%2Fuploads%2F32OGRoGbQsQYH3oeR0kL%2Fimage.png?alt=media\&token=5f165cdf-d3da-41de-b43f-6dca2cc538d1) * You can drag a pipeline component from the [***Component Panel.***](https://docs.bdb.ai/bdb-documentation/data-pipeline/getting-started/pipeline-workflow-editor/component-panel) * Connect the dragged component to the dragged ***Event*** to create a pipeline flow of data. ![A Pipeline workflow in process](https://972575688-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FRYq1HgffNfbnIMWPu1D5%2Fuploads%2F6o9qS8kDALf9LxIzvch0%2Fimage.png?alt=media\&token=d557122b-0599-456e-b4b9-5146c1894658) #### **DB Sync** The user can directly read the data with the reader and write to a DB Sync. * The user can add a new DB Sync from the toggle event panel to the workflow editor by clicking on ‘+’ icon. * Specify the display name and connection id and click on save. * Drag and drop the DB Sync from event panel to workflow editor. ### Global Variables The user can create a Global Variable by following the below-given steps: * Click the Global Variable icon from the User Interface. * Global Variables panel opens. * Click the ***Add New Variable*** icon from the Global Variables panel.
* The ***Add Variable*** window opens. * Insert Variable name. * Provide Value for the Variable. * Click the ***Save*** option. ![](https://972575688-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FRYq1HgffNfbnIMWPu1D5%2Fuploads%2F1qbeKNp9P8MbTjkkAyA7%2Fimage.png?alt=media\&token=d2c80ba4-11a2-4afb-84e3-e057cd864c0e) * A notification message appears. * The Global Variable gets created and added to the panel.
### Activate/Deactivate Pipeline The user can activate/deactivate the pipeline by clicking on the / icon. Activation will deploy all the components based on their respective invocation types. When the pipeline is deactivated all the components go down and will halt the process. ### Update Pipeline Clicking on the (Update) icon allows you to save the pipeline. It is recommended to update the pipeline every time you make changes in the workflow editor. On a successful update of the pipeline, you get a notification as given below: ![](https://972575688-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FRYq1HgffNfbnIMWPu1D5%2Fuploads%2F7aNrqQByMvR3SDwpXRsD%2Fimage.png?alt=media\&token=d10d974f-30c5-4ae1-851a-84767e1cebc5) {% hint style="info" %} Please Note: On any Failures the users get a notification through the below-given error message. ![](https://972575688-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FRYq1HgffNfbnIMWPu1D5%2Fuploads%2FcXWOG8GkqkRrhppA51sb%2Fimage.png?alt=media\&token=a01b55a6-fe09-4137-965e-e8fd62d0e883) {% endhint %} ### [Push Pipeline (to VCS/GIT)](https://docs.bdb.ai/bdb-documentation/version-control#pushing-a-pipeline-into-vcs) ### [Pull Pipeline from GIT](https://docs.bdb.ai/bdb-documentation/version-control#pulling-a-pipeline-from-git) ### Full Screen The ***Full Screen*** icon presents the Pipeline Editor page in the full screen. * Navigate to the ***Pipeline Workflow Editor*** page. * Click the ***Full Screen***icon from the toolbar.
* The Pipeline Workflow Editor opens in full screen and the icon changes to icon.
### [Pipeline Testing](https://docs.bdb.ai/bdb-documentation/data-pipeline/getting-started/testing-suite) ### Failure Analysis Failure analysis is a central failure mechanism. Here, the user can identify the failure reason. Failures of any pipeline stored at a particular location(collection). From there you can query your failed data in the Failure Analysis UI. It displays the failed records along with cause, event time, and pipelineId. * Navigate to the ***Pipeline Editor*** page. * Click the **Failure Analysis** icon.
* The ***Failure Analysis*** page opens. * **Search Component:** A Search bar is provided to search all components associated with that pipeline. It helps to find a specific component by inserting the name in the Search Bar. * **Component Panel:** It displays all the components associated with that pipeline. * **Filter:** By default, the selected component instance Id will be displayed in the filter field. Records will be displayed based on the instanceid of the selected component. It filters the failure data based on the applied filter. **Please Note the Filter Format of some of the field types.**
Field Value TypeFilter Format
Stringdata.data_desc:” "ignition"
Integerdata.data_id:35
Floatdata.lng:95.83467601
Booleandata.isActive:true
* **Project:** By default, the pipeline\_Id and \_id are selected from the records. If the user does not want to select and select any field then that field will be set with 0/1 (0 to exclude and 1 to include), displaying the selected column. {% hint style="info" %} *Please Note**:** data.data\_id:0, data.data\_desc:1* {% endhint %} * **Sort**: By default, records are displayed in descending order based on the “\_id” field. Users can change ascending order by choosing Ascending option. * **Limit:** By default, 10 records are displayed. Users can modify the records limit according to the requirement. The maximum limit is 1000. * **Find:** It filters/sorts/limits the records and projects the fields by clicking on the find button. * **Reset:** If the user clicks on the ***Reset*** button, then all the fields must be reset with a default value.
* **Cause:** The cause of the failure gets displayed by a click on any failed data.
### [Pipeline Monitoring](https://docs.bdb.ai/bdb-documentation/data-pipeline/getting-started/monitoring-pipeline) ### Delete Pipeline This function helps to remove a pipeline. * Navigate to the ***Pipeline Workflow Editor*** page. * Click the ***Delete*** icon.
* A dialog box opens to assure the deletion. * Select the ***YES*** option. ![](https://972575688-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FRYq1HgffNfbnIMWPu1D5%2Fuploads%2FiukvV58MoTg30IHM0Nmy%2Fimage.png?alt=media\&token=859343f2-ffed-4832-b77f-b495c7b959ca) * A notification message appears. * The selected pipeline gets removed from the ***Pipeline List***.