Creating Pipeline

• Navigate to the Data Pipeline landing page.

• Click the Create Pipeline option provided on the top right side of the Pipeline landing page.

• The New Pipeline window opens asking for the basic information.

• Enter a name for the new Pipeline.

• Describe the Pipeline (Optional).

• Select a resource allocation option using the radio button- the given choices are:

  • Low

  • Medium

  • High

This feature is used to deploy the pipeline with high, medium, or low-end configurations according to the velocity and volume of data that the pipeline must handle. All the components saved in the pipeline are then allocated resources based on the selected Resource Allocation option depending on the component type (Spark and Docker).

Click the Save option to create the pipeline. By clicking the Save option, the user gets redirected to the pipeline workflow editor.

• A success message appears to confirm the creation of a new pipeline.

• The Pipeline Editor page opens for the newly created pipeline.

• Resource allocation can be changed anytime by clicking on top left edit icon near pipeline name.