Migrating a Pipeline

This section outlines the process for migrating pipelines between distinct tenant environments. Tenant migration is typically undertaken for various reasons, including organizational restructuring, compliance requirements, or the consolidation of resources. This guide outlines the steps required to ensure a seamless and successful transfer of your pipeline assets.

Pipeline Migration is the process of moving a data pipeline from one environment to another. This is typically done to move a pipeline from a development or staging environment to a production environment. Migration ensures that the pipeline's structure and configurations remain consistent across different systems.

Check out the illustrations below on how to migrate (export) a Pipeline.

Git Export/Pipeline Migration

The Import/ Git Import functionality facilitates the import of a pipeline that was previously exported (pushed for migration) from another environment.

Check out the illustration on importing a migrated pipeline to a different tenant space.

Importing a Migrated Pipeline

Please note: Push and Pull Permissions: The Push and Pull functionality is exclusively available to users with the 'Admin' role or non-admin users who have been granted access to the platform's Admin module.

Part 1: Exporting a Pipeline to Git

This process, also known as a Migration Export, pushes a pipeline from its current tenant space to a Git repository, making it available for import into a different environment.

Steps to Export

1. Log In to the Source Tenant: Sign in to your source platform account using the tenant space that contains the pipeline you want to export.

2. Access the Pipelines Module: Navigate to the Data Engineering module and open the Pipelines option.

3. Initiate the Push:

   • Find the pipeline you wish to export from the pipeline list.

   • Select the pipeline to open its right-side panel.

   • Click the Push & Pull Pipeline icon.

4. Configure the Export:

   • In the Push/Pull window, ensure the Push tab is selected (it is the default).

   • Provide a descriptive comment for this pipeline version.

   • Check the box next to Git Export (Migration).

5. Execute the Push: Click the Push button. A success message will confirm that the pipeline was successfully pushed to Git.

Part 2: Importing a Pipeline from Git

This process allows you to import a pipeline that was previously exported from a different tenant. This is the Migration Import step.

Please note: This function is only available to users with the 'Admin' role or non-admin users with specific access to the Admin module.

Steps to Import

1. Log In to the Destination Tenant: Sign in to the tenant space where you want the pipeline to reside.

2. Access the Admin Module:

   • Click on your user profile menu.

   • Select the Settings option.

   • Click the Admin icon.

3. Open Git Migration: From the Admin menu panel, select Git Migration.

4. Start the Import Process:

   • Click the Import file option.

   • On the Import Document page, click the Import button.

5. Select and Migrate the Pipeline:

   • On the Migration: Document Import page, select the New VCS option.

   • From the dropdown field, choose Pipeline.

   • Click the Pipelines option to view a list of available pipelines that have been exported.

   • Use the search bar to find the specific pipeline you need.

   • Select the pipeline from the list. The available versions will appear on the right side of the page.

   • Select the desired pipeline version by checking its corresponding box.

6. Complete the Migration: Click the Migrate option. A notification will confirm that the pipeline has been successfully migrated to your account.

• The newly migrated pipeline will now appear in your Pipelines list under the Data Engineering module.