> For the complete documentation index, see [llms.txt](https://docs.bdb.ai/data-science-lab/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.bdb.ai/data-science-lab/project/tabs-for-a-data-science-lab-project/tabs-for-pyspark-environment/notebook/notebook-list-page/export/export-to-pipeline.md).

# Export to Pipeline

A Notebook can be exported to the Data Pipeline module by using this option.

{% hint style="success" %}
*Check out the walk-through on how to export a Notebook script to the Data Pipeline module.*
{% endhint %}

{% embed url="<https://files.gitbook.com/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Figrg2b2URgwMO5YmejDu%2Fuploads%2FDistQsn2yC31aq5k07p6%2FPySpark_Export%20New%20(1).mp4?alt=media&token=3604ae9b-feef-4710-ac10-3d2b61272e45>" %}
Exporting a PySpark Notebook to the Data Pipeline module
{% endembed %}

* Navigate to the ***Notebook list***.
* Click the ***Export to Pipeline*** icon for a Notebook.     &#x20;

<figure><img src="https://1025450693-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Figrg2b2URgwMO5YmejDu%2Fuploads%2F7Px1DaeNr2SoYmVHpzp4%2Fimage.png?alt=media&amp;token=87b9883c-24a2-4636-820f-f1f74257ed33" alt=""><figcaption></figcaption></figure>

* The ***Export to Pipeline*** dialog box opens.
* Select a specific function using the checkbox.
* Click the ***Next*** option.

<figure><img src="https://1025450693-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Figrg2b2URgwMO5YmejDu%2Fuploads%2FIkm9DmIK18AFIhxLibdS%2Fimage.png?alt=media&amp;token=0f9c21a6-1f8f-444a-b93a-bb3dc868a5ef" alt=""><figcaption></figcaption></figure>

{% hint style="info" %}
*<mark style="color:green;">Please Note</mark>: The user must write a function to use **Export to Pipeline** functionality.*&#x20;
{% endhint %}

* Click the ***Export*** option from the next page that opens for the Pipeline Export.

<figure><img src="https://1025450693-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Figrg2b2URgwMO5YmejDu%2Fuploads%2FXjhQJsOcPTWs0zeHLKUh%2Fimage.png?alt=media&amp;token=c80feed4-50b8-48b6-89e4-b3811014e852" alt=""><figcaption></figcaption></figure>

* A confirmation message appears informing the completion of the action.

<figure><img src="https://1025450693-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Figrg2b2URgwMO5YmejDu%2Fuploads%2FnK5TBK1Zoh3aHC7ZDE14%2Fimage.png?alt=media&amp;token=dc29e122-c515-4ffb-98d8-06e959c44524" alt=""><figcaption><p>Notification message after Notebook gets exported.</p></figcaption></figure>

* Navigate to a ***Pipeline*** ***homepage***.
* Click the ***Create Job*** option.

<figure><img src="https://1025450693-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Figrg2b2URgwMO5YmejDu%2Fuploads%2F3hEYCt0duhiY3dt2RaPG%2Fimage.png?alt=media&amp;token=08b6f561-ff08-4718-a470-37ad64bb2f81" alt=""><figcaption></figcaption></figure>

* The ***New Job*** dialog window opens.
* Provide the required information to create a new job.
  * Enter name: Provide name for the new job.
  * Job Description: Enter the description for the new job.
  * Job Baseinfo: Select the ***PySpark Job*** option using the drop-down.
  * **Trigger By:** The PySpark Job can be triggered by another Job or PySpark Job. The PySpark Job can be triggered in two scenarios from another jobs:
    * **On Success:** Select a job from drop-down. Once the selected job is run successfully, it will trigger the PySpark Job.
    * **On Failure:** Select a job from drop-down. Once the selected job gets failed, it will trigger the PySpark Job.
  * **Is Scheduled**: Put a check mark in the given box to schedule the new Job.
  * **Spark config**: Select resource for the new Job.
* Click the ***Save*** option.

<figure><img src="https://1025450693-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Figrg2b2URgwMO5YmejDu%2Fuploads%2F5n0wT86ay7Spie3vSiwg%2Fimage.png?alt=media&amp;token=7af3ceb7-818b-4736-8557-1b5d4d6f228d" alt=""><figcaption></figcaption></figure>

* A notification message appears and the new Job gets created.
* The recently created Job appears dragged to the ***Job Editor workspace*** by default.

<figure><img src="https://1025450693-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Figrg2b2URgwMO5YmejDu%2Fuploads%2Fkbi1Gr3Qk9cmiwhMSp7I%2Fimage.png?alt=media&amp;token=832fa44a-27a7-473d-8dcb-9efe974c9f38" alt=""><figcaption><p>Job Editor Workspace</p></figcaption></figure>

* Click on the Job component to open the configuration tabs.

<figure><img src="https://1025450693-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Figrg2b2URgwMO5YmejDu%2Fuploads%2FHPEI8w0meRLI6fwvEAyI%2Fimage.png?alt=media&amp;token=94329581-2665-4a22-8eba-cdb86658dc01" alt=""><figcaption></figcaption></figure>

* Open the ***Meta Information*** tab of the ***PySpark Job*** component.
* **Project Name:** Select the same Project using the drop-down menu where the concerned Notebook has been created.
* **Script Name:** Select the script which has been exported from notebook in DS Lab module. The script written in DS Lab module should be inside a function.
* **External Library:** If any external libraries used in the script we can mention here. We can mention multiple libraries by giving comma(,) in between the names.
* **Start Function:** Select the function name in which the script has been written.
* **Script:** The Exported script appears under this space.
* **Input Data:** If any parameter has been given in the function, then the name of the parameter is provided as **Key** and value of the parameters has to be provided as **value** in this field.
* Click the ***Save component in the storage*** to use the PySpark component in a workflow inside the Data Pipeline module.

<figure><img src="https://1025450693-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Figrg2b2URgwMO5YmejDu%2Fuploads%2Feg07gpFYbiprCSXfB4e9%2Fimage.png?alt=media&amp;token=81d284b0-8bd4-4092-9865-3605843d3cac" alt=""><figcaption><p>Consuming the Exported Data Science Script to the PySpark component.</p></figcaption></figure>

{% hint style="info" %}
*<mark style="color:green;">Please Note:</mark> Refer the **Data Science Lab Quick Start Flow** page to get an overview of the **Data Science Lab** module in nutshell.* [***Click here***](https://docs.bdb.ai/data-science-lab/data-science-lab-quick-start-flow) *to get redirected to the quick start flow page.*
{% endhint %}