Manage Pipelines

You can access the pipelines created by you or shared to you from the Pipelines page. You can view, update, and manage related actions for a pipeline from this page.

The Pipelines page displays all pipelines saved by the logged-in user. This is where you can view and manage your available pipelines.

Accessing Pipeline List

Access the Pipelines List page from the Navigation Menu

• Click the Data Engineering option from the navigation menu.

• Select the Pipelines option. Click on it.

• The Pipelines List page will be launched on the right side of the screen.

Access the Pipeline List page from the Apps Menu

• Navigate to the Platform homepage.

• Open the Apps menu icon.

• Click the Data Pipeline plugin option from the displayed context menu.

• Select the Pipelines tab from the next screen.

• The Pipelines List will be launched.

Pipelines Access and Visibility

• Admin Users: Users with admin privileges have comprehensive visibility. They can view all pipelines created by any user within the tenant and have full access to the pipelines available to them.

• Non-admin Users: Standard users of the platform with non-admin access can access the pipelines they have created or those shared with them by other users under the Pipelines list.

  • The Share Pipeline action helps further control the action access available to the standard users on the pipelines accessible to them.

Access the Actions for a Pipeline

• Navigate to the Pipelines List page.

• Select a pipeline from the displayed list, and click on it.

  

• A panel opens on the right side of the screen with the various action icons.

  • The action icons may be enabled based on the access permissions of the logged-in user.

    

​Search Bar

You can search for a specific pipeline using the Search bar on the Pipelines List page. This feature enables you to quickly locate pipelines by typing in a common name or a partial name.

1. Access the Pipelines List: Navigate to the Pipelines List page. This page shows all the pipelines you have saved.

2. Locate the Search Bar: Find the Search bar at the top of the Pipelines List page.

3. Enter Search Terms: Type your search terms into the bar. For example, typing "resta" will list all pipelines containing those letters. You can also use terms like "Invoi" or "Fee" to find pipelines related to invoices or fees.

4. View Filtered Results: The list will automatically filter and display only the pipelines that match your search terms.

By following these steps, you can efficiently locate the specific pipeline you need without having to scroll through a long list.

View Pipeline Details

You can view the detailed information for any pipeline directly from the Pipelines List page.

1. Select a Pipeline: On the Pipelines List page, click on the name of the pipeline you want to view.

2. View Details: A panel will open on the right side of the screen, displaying all the relevant details for the selected pipeline.

You can see the following information inside the Pipeline Details panel:

Pipeline Details

Upon selecting a pipeline, the following details are presented, offering a comprehensive overview of its configuration and operational status:

• Pipeline Name: The unique identifier for the pipeline.

• Component Count: The total number of individual components or stages within the pipeline workflow.

• Created By: The user who initially designed and created the pipeline.

• Last Updated: The timestamp indicating the most recent modification to the pipeline's configuration.

• Last Activated: The date, time, and user who last initiated the execution of this pipeline.

• Last Deactivated: The date, time, and user who last stopped the execution of this pipeline

• Description: A user-defined explanation providing context or purpose for the pipeline. ​​

Total Component Configuration

This section outlines the compute resources provisioned for the pipeline's execution:

• Total Allocated Max CPU: The maximum CPU cores allocated for the pipeline's execution across all its components.

• Total Allocated Min CPU: The minimum guaranteed CPU cores allocated for the pipeline.

• Total Allocated Max Memory: The maximum memory (RAM) allocated for the pipeline's execution across all its components.

• Total Allocated Min Memory: The minimum guaranteed memory (RAM) allocated for the pipeline.​

Component Status: Failed and Ignored Components

This section highlights any components within the pipeline that have encountered issues or have been intentionally excluded from the workflow:

• Failed and Ignored Components: Lists the names of individual components that have either failed during execution or are currently configured to be ignored within the pipeline's defined workflow.

  

  Failed and Ignored Components section for a successful Pipeline

Failed and Ignored Components section for a pipeline encountering issues

Pin & Unpin Pipeline

The Pin/ Unpin feature helps you prioritize and quickly access specific pipelines. Located on each pipeline card in the list view, this functionality lets you efficiently manage your tasks, projects, and workflows.

• Navigate to the Pipelines List page.

• Select a Pipeline from the displayed list that you wish to pin. Click the Pin icon provided for the selected pipeline.

  

  Pinning a Pipeline
• The Pipeline gets pinned to the list, and it will be moved to the top of the list if it is the first pinned pipeline. In this case, the pinned pipeline has become the second pipeline from the top, as one pinned pipeline exists in the list.

  • The user can pin multiple pipelines to the list.

  • Pinned pipelines are easily identified as their pin icon turns violet.

  • For a pinned pipeline, the pin icon represents the Unpin action.

    
• Click the Unpin icon for a pinned pipeline that you wish to unpin from the list.

  
• The selected pipeline gets unpinned from the list, and it moves downwards in the list order.