# Settings

This page is for the ***Advanced configuration*** of the Data Pipeline. The following list of the configurations get displayed on the ***Settings*** Page:

* Logger
* Default Configuration
* Pod Status
* Data Sync
* Job BaseInfo

<figure><img src="https://363587200-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fuq3RSHHup7fjHYaspk7y%2Fuploads%2Fe3q4SsDbmOQCtqnVrcrM%2FMicrosoftTeams-image%20(39).png?alt=media&#x26;token=578191e9-18c1-46a2-aefb-8b3676ec1d1e" alt=""><figcaption></figcaption></figure>

## Logger <a href="#logger" id="logger"></a>

This gives the info and config details of the Kafka topic that is set for logging.

<figure><img src="https://363587200-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fuq3RSHHup7fjHYaspk7y%2Fuploads%2FJnpm9EiWbMPPD7HXvtiH%2FMicrosoftTeams-image%20(40).png?alt=media&#x26;token=af2e7fa8-28d7-4ef3-ad68-839f9cd80d85" alt=""><figcaption><p>Logger Configuration</p></figcaption></figure>

## Default Configuration <a href="#default-configuration" id="default-configuration"></a>

This is for setting the **Default component configuration** for Low, Medium and, High configuration Pipelines.

{% embed url="<https://files.gitbook.com/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fuq3RSHHup7fjHYaspk7y%2Fuploads%2FK409XHTgtv6Jbjtr6J21%2Fdefault_config_AdobeExpress.mp4?alt=media&token=19435002-eb22-4f7c-9423-20e669e954ae>" %}
Default Configuration
{% endembed %}

{% hint style="info" %} <mark style="color:green;">Please Note:</mark> *The **Non-Spark** means **Docker** under the **Default Configuration** section.*
{% endhint %}

## Pod Status <a href="#pod-status" id="pod-status"></a>

The ***Pod Status*** page under the ***Settings*** option gives us monitoring capability on the health of the System Components.

{% embed url="<https://files.gitbook.com/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fuq3RSHHup7fjHYaspk7y%2Fuploads%2FsLLim2HlMhdnSYsVOgmb%2FScreen_Recording__20-Mar-23_4-52-50_PM__AdobeExpress.mp4?alt=media&token=5019b8d6-4241-40d8-ae43-637b10212977>" %}
*System Component Health*
{% endembed %}

{% hint style="info" %}
*<mark style="color:green;">Please Note:</mark>* *The system components should be up for proper functionality of the Pipeline module.*
{% endhint %}

## Data Sync <a href="#db-sync" id="db-sync"></a>

Data Sync in Settings is given to globally configure the ***Data Sync*** feature. This way the user  can enable a DB connection and use the same connection in the pipeline workflow without using any extra resources.

<figure><img src="https://363587200-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fuq3RSHHup7fjHYaspk7y%2Fuploads%2F9uRAkpp8IpHOyxg9nUer%2FMicrosoftTeams-image%20(41).png?alt=media&#x26;token=ec08d99f-674b-4a08-b673-b9bd332c4141" alt=""><figcaption><p>DB Sync page under the Pipeline Settings option</p></figcaption></figure>

* Go to the ***Settings*** page.
* Click the ***Data Sync*** option.
* The ***Data Sync*** page opens.
* Click the ***Plus*** icon.

{% hint style="info" %} <mark style="color:green;">Please Note</mark><mark style="color:green;">:</mark> *We support seven(7) drivers at present.*

* MongoDB

* Postgres

* MySQL

* MSSQL

* Oracle

* ClickHouse

* Snowflake
  {% endhint %}

* The ***New Settings*** dialog box appears.

* Specify all the required connection details

* Click the ***Save*** option.

{% hint style="warning" %}
ClickHouse Port: If you are using Clickhouse as Driver ; Please use TCP Port.
{% endhint %}

<figure><img src="https://363587200-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fuq3RSHHup7fjHYaspk7y%2Fuploads%2Fobd19DUIKeEUA8h1Nza9%2FMicrosoftTeams-image%20(49).png?alt=media&#x26;token=872b91f7-978f-4fb6-914e-b20cb9e338a8" alt=""><figcaption></figcaption></figure>

{% hint style="info" %}
*<mark style="color:green;">Please Note:</mark> The **configured Driver** from the **Settings** page provided for DB Sync **can be accessed inside a*** [***DB Sync component on the Pipeline Editor page***](https://docs.bdb.ai/data-pipeline/getting-started/homepage/creating-a-new-pipeline/connecting-components/events-kafka-and-data-sync) *while using it in a Pipeline.*&#x20;
{% endhint %}

* A notification message appears.
* Enable the action button.

<figure><img src="https://363587200-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fuq3RSHHup7fjHYaspk7y%2Fuploads%2FG9UokwK37sH2gWXtgsmt%2FMicrosoftTeams-image%20(43).png?alt=media&#x26;token=d7b2d25a-0080-4708-b82a-5b45cdff687e" alt=""><figcaption></figcaption></figure>

### Edit  Connection String

* Go to the ***Settings*** page.
* Click the ***Data Sync*****&#x20;option** - the DB Sync Settings page opens.
* Click the ***Edit*** icon.

<figure><img src="https://363587200-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fuq3RSHHup7fjHYaspk7y%2Fuploads%2F8qR7cC7Qc7LII6I3KtJA%2FMicrosoftTeams-image%20(47).png?alt=media&#x26;token=b5a678af-f899-4193-81c6-bf9021713257" alt=""><figcaption></figcaption></figure>

* The ***Edit Settings*** dialog box opens.
* Edit the connection string if required.
* Click the ***Update*** option.

![](https://363587200-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fuq3RSHHup7fjHYaspk7y%2Fuploads%2Fd8nMlMmhFG6PgGeSQd84%2FMicrosoftTeams-image%20\(42\).png?alt=media\&token=caa0c767-cc54-49af-9ddf-e045f0206368)

&#x20;

### Disconnect Connection String

* Go to the ***Settings*** page.
* Click the ***Data Sync*** option.
* The ***Data Sync Settings*** page opens.
* Click on the Disconnect button to disconnect.  &#x20;

<figure><img src="https://363587200-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fuq3RSHHup7fjHYaspk7y%2Fuploads%2FKcqiB5fQgmvtojUe7M25%2FMicrosoftTeams-image%20(45).png?alt=media&#x26;token=3feb92de-b5ce-45d4-a09c-2bd3c54a42b1" alt=""><figcaption></figcaption></figure>

* The Disconnect Setting confirmation dialog box appears.
* Click the ***DISCONNECT*** option.

<figure><img src="https://363587200-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fuq3RSHHup7fjHYaspk7y%2Fuploads%2FxdMESK91iHJze3N5sHIw%2FMicrosoftTeams-image%20(44).png?alt=media&#x26;token=8d6c625f-33af-4c3c-9242-94d71582f231" alt=""><figcaption></figcaption></figure>

* A success notification message appears on the top when DB Sync gets disconnected.

<figure><img src="https://363587200-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fuq3RSHHup7fjHYaspk7y%2Fuploads%2FTeY5j07mwJh1yZED5Xsk%2FMicrosoftTeams-image%20(46).png?alt=media&#x26;token=8a717851-9b1c-4131-a0ab-d5d3ecf9045f" alt=""><figcaption></figcaption></figure>

### Data Sync in a Pipeline

* All the Pipelines containing the DB Sync Event component get listed under the Pipeline using DB Sync section.
* The user can see the following information:

  * **Name**: Name of the pipeline where Data Sync is used.
  * **Running status**: This indicates if the pipeline is active or deactivated.
  * **Actions:** The user will be able to view the pipeline where Data Sync is used.

  &#x20;    &#x20;

  <figure><img src="https://363587200-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fuq3RSHHup7fjHYaspk7y%2Fuploads%2FF7pxj3iZOu4oQhXgXT0V%2FMicrosoftTeams-image%20(48).png?alt=media&#x26;token=f7537d61-fd80-4ef5-bc25-4635f68e279d" alt=""><figcaption></figcaption></figure>

## Job BaseInfo

There are two types of Jobs used in Pipeline module:

* Spark Job
* PySpark Job

These two types of are configured in the Job BaseInfo page under the settings menu.

{% hint style="info" %}
*<mark style="color:green;">**Please Note:**</mark>* **The Job BaseInfo has been created from the admin side, and the user is not supposed to create it in the settings menu**.
{% endhint %}

**Steps to create new Job BaseInfo:**

* Click on the Plus icon ![](https://363587200-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fuq3RSHHup7fjHYaspk7y%2Fuploads%2F6hlz75xQD1iDFdlyb3U2%2Fimage.png?alt=media\&token=81a9f0e1-a654-4cae-b36e-207af3dd4c34)on the Job BaseInfo details page as shown in the below image.

<figure><img src="https://363587200-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fuq3RSHHup7fjHYaspk7y%2Fuploads%2FsVucg0MmCJLVqUHhdeE0%2Fimage.png?alt=media&#x26;token=09d1288b-4ceb-4b4d-8f39-9d028807c28d" alt=""><figcaption></figcaption></figure>

* Once clicked, it will redirect to the page for creating a new Job BaseInfo and the user will be asked to fill details in the following Tabs:

**Basic Information Tab:**

<figure><img src="https://363587200-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fuq3RSHHup7fjHYaspk7y%2Fuploads%2FWWIjdsLASpneVPn9NdlN%2Fimage.png?alt=media&#x26;token=9a3bc0c3-ae61-4c34-bc52-51b5a83f5152" alt=""><figcaption><p>Basic information tab </p></figcaption></figure>

1. **Name:** Provide a Name for the Job BaseInfo.
2. **Deployment type:** Select the Deployment type from the drop-down.
3. **Image Name:** Enter the Image name for the Job BaseInfo.
4. **Version:** Specify the version.
5. **isExposed:** This option will be automatically filled as True once the deployment type is selected
6. **Job type:** Select the job type from the drop down.

**Ports Tab:**

<figure><img src="https://363587200-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fuq3RSHHup7fjHYaspk7y%2Fuploads%2F10k0EyqY0XgynxGBdxui%2Fimage.png?alt=media&#x26;token=55662e77-dde9-454f-9dc8-c2349976a1cc" alt=""><figcaption><p>Ports Tab</p></figcaption></figure>

* **Port Name:** Enter the Port name.
* **Port Number:** Enter the Port number.
* **Delete:** The user can delete the port details by clicking on this option.
* **Add Port:** The user can add the Port by clicking on this option.&#x20;

**Spark component information:**

<figure><img src="https://363587200-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fuq3RSHHup7fjHYaspk7y%2Fuploads%2FRR2SCDD95HZPgUn3UJN5%2Fimage.png?alt=media&#x26;token=c062f9ac-8476-409c-b79a-54953093e7d5" alt=""><figcaption></figcaption></figure>

* **Main class:** Enter the Main class for creating the Job BaseInfo.
* **Main application file:** Enter the main application file.
* **Runtime Environment Type:** Select from the drop-down. There are three options available: Scala, Python and R.&#x20;
* Now, click on **Save** option to create the Job BaseInfo.

Once the Job BaseInfo is created, the user can redirect to the List Job BaseInfo page by clicking on the List Job BaseInfo icon as shown in the below image:

<figure><img src="https://363587200-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fuq3RSHHup7fjHYaspk7y%2Fuploads%2FzepEHfRtJfSJvbANLURs%2Fimage.png?alt=media&#x26;token=b55e588f-4125-4539-8407-8c9e17b13adf" alt=""><figcaption></figcaption></figure>

**List Job BaseInfo Page:**

This Page will show the list of all created Job BaseInfo as shown in the below image:

<figure><img src="https://363587200-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fuq3RSHHup7fjHYaspk7y%2Fuploads%2FhGW1MgL1K7YvTX5E62AX%2Fimage.png?alt=media&#x26;token=a5dba00a-bee3-44d7-956d-2542d6637ae9" alt=""><figcaption></figcaption></figure>

* The ***Name*** column represents the Name of the Job BaseInfo.
* The ***Status*** column will display the status of the Job BaseInfo.&#x20;
* The ***Version*** column displays the version of the Job BaseInfo.
* The user can see the details of Job Baseinfo by clicking on View icon ![](https://363587200-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fuq3RSHHup7fjHYaspk7y%2Fuploads%2FZPQvdKmaCWLZ61nDyN4G%2Fimage.png?alt=media\&token=7f199092-3d35-4dc5-bbd0-19288114babf). Once clicked on view icon, the user can see the details of Job BaseInfo which were provide at the time of creating of that Job BaseInfo.
* The user can delete the Job BaseInfo by clicking on delete ![](https://363587200-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fuq3RSHHup7fjHYaspk7y%2Fuploads%2FdQOwGKmh6bGIfcwMpYjj%2Fimage.png?alt=media\&token=1b843221-a547-4832-b804-4805fb1657c0) icon.