Creating a New Pipeline

This section provides the steps involved in creating a new Pipeline flow.

Check out the below-given illustration on how to create a new pipeline.

Creating a New Pipeline

• Navigate to the Create Pipeline or Job interface.

• Click the Create option provided for Pipeline.

Creating a new pipeline

• The New Pipeline window opens asking for the basic information.

• Enter a name for the new Pipeline.

• Describe the Pipeline (Optional).

• Select a resource allocation option using the radio button- the given choices are:

  • Low

  • Medium

  • High

Please Note: This feature is used to deploy the pipeline with high, medium, or low-end configurations according to the velocity and volume of data that the pipeline must handle. All the components saved in the pipeline are then allocated resources based on the selected Resource Allocation option depending on the component type (Spark and Docker).

• Click the Save option to create the pipeline. By clicking the Save option, the user gets redirected to the pipeline workflow editor.​

• A success message appears to confirm the creation of a new pipeline.

• The Pipeline Editor page opens for the newly created pipeline.

• Resource allocation can be changed anytime by clicking on top left edit icon near pipeline name.​