Pipeline Component Configuration
The Pipeline Component Configuration feature allows users to configure all components from a single page, streamlining the process and reducing time and effort.
The Pipeline Component Configuration feature allows users to configure all components from a single page, streamlining the overall process. By centralizing configuration options, it eliminates the need to click on each component individually, making the setup more efficient. This not only saves time but also minimizes the effort required to manage and update pipeline components.
Accessing the Pipeline Component Configuration
Navigate to the Pipeline Workflow Editor page.
Click the Pipeline Component Configuration icon from the Pipeline Toolbar.
Pipeline Component Configuration Page
Upon opening the configuration page, all components associated with the selected pipeline are listed. At the top of the page, the following pipeline-level information is displayed:
Pipeline – Displays the name of the pipeline.
Status – Indicates the running status of the pipeline:
True: Pipeline is active.False: Pipeline is inactive.
Total Allocated Max CPU – Maximum CPU allocated across all components (in cores).
Total Allocated Min CPU – Minimum CPU allocated across all components (in cores).
Total Allocated Max Memory – Maximum memory allocated across all components (in MB).
Total Allocated Min Memory – Minimum memory allocated across all components (in MB).
Tabs in the Configuration Page
The configuration page contains two tabs:
1. Basic Info (Default Tab)
The Basic Info tab opens by default when accessing the configuration page. This section allows users to configure general settings for each component:
Invocation Type – Defines how the component is invoked during pipeline execution.
Batch Size – Specifies the size of data batches processed by the component.
Description – Provides descriptive details for the component.
Intelligent Scaling – Appears if enabled for the corresponding component, allowing dynamic scaling based on workload demands.
2. Configuration Tab
The Configuration tab allows users to manage resource allocation for pipeline components. Users can:
Allocate CPU (cores) and Memory (MB) to each component.
Define minimum instances (guaranteed pods) and maximum instances (scalable pods) for parallel execution.
This ensures that components are provisioned with sufficient resources while supporting elastic scaling based on workload requirements.
Using the Pipeline Component Configuration interface, administrators and developers can ensure consistent governance, performance optimization, and resource efficiency across all pipeline components.