# Monitoring Pipeline

The user can monitor a pipeline together with all the components associated with the same by using the ***Pipeline Monitoring*** icon. The user gets information about Pipeline components, Status, Types, Last Activated (Date and Time), Last Deactivated (Date and Time), Total Allocated and Consumed CPU%, Total allocated and consumed memory, Number of Records, and Component logs all displayed on the same page.

{% hint style="success" %}
*Go through the below-given video to get a basic idea on the pipeline monitoring functionality.*
{% endhint %}

{% embed url="<https://files.gitbook.com/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fuq3RSHHup7fjHYaspk7y%2Fuploads%2F6ARKbF4cctGnWWPZsKim%2Fpipeline_monitoring_and_data_matrices_updated_one_AdobeExpress.mp4?alt=media&token=ce4a1208-03cd-452c-ae12-5a3a45ed1bad>" %}
Displaying Pipeline Monitoring and Data Metrics&#x20;
{% endembed %}

* Navigate to the ***Pipeline List*** page.
* Click the ***Monitor*** icon.         &#x20;

<figure><img src="/files/heM5XxaltEjaClSWmsj6" alt=""><figcaption><p>Accessing the Pipeline Monitoring from the Pipeline List page.</p></figcaption></figure>

**Or**

* Navigate to the ***Pipeline Workflow Editor*** page.
* Click the ***Pipeline Monitoring*** icon on the Header panel.

<figure><img src="/files/3y5lajEDIh5MhMlWM9aT" alt=""><figcaption></figcaption></figure>

* The ***Pipeline Monitoring*** page opens displaying the details of the selected pipeline.
* The ***Monitor*** tab opens by default.     &#x20;

<figure><img src="/files/l5tMm1XGXnvJK3KajHC5" alt=""><figcaption></figcaption></figure>

* If there are multiple instances for a single component, click on the drop-down.
* Details for each instance will be displayed.&#x20;

**Monitoring page for Docker component in Real-Time**

<figure><img src="/files/F0BnOePKwaFFKARsuSS1" alt=""><figcaption><p>Docker component- Real time</p></figcaption></figure>

<figure><img src="/files/r8ABvpDOvifizda8ngdn" alt=""><figcaption><p>Docker component- Real time</p></figcaption></figure>

**Monitoring page for Docker component in Batch:**

<figure><img src="/files/OEFCP9zKqHdBEXORSfm7" alt=""><figcaption></figcaption></figure>

<figure><img src="/files/MYJhmZhIoVetU5vRNNaq" alt=""><figcaption><p>Docker component- Batch</p></figcaption></figure>

**Monitoring page for Spark Component:**

<figure><img src="/files/b4zDmT9bd9C6zViHnuAo" alt=""><figcaption><p>Monitoring details for Spark component</p></figcaption></figure>

**Monitoring page for Spark Component - Driver:**

<figure><img src="/files/EriyltsIHj6RMVhyhLLl" alt=""><figcaption><p>Spark component- Driver</p></figcaption></figure>

**Monitoring page for Spark Component- Executer:**

<figure><img src="/files/FmpMmQbUAnuHwajaaes7" alt=""><figcaption><p>Spark component- Executer</p></figcaption></figure>

* If memory allocated to the component is less than required, then it will be displayed in red color.
* Open the ***Data Metrics*** tab.
* Specify the time period by providing the from and to dates.
* Choose an interval option or select the custom interval.
* The component specific data metrics get displayed. The green color nodes indicate that the data has been loaded.​ Click on green color icon to get all the details of processed data as shown in the below images.  &#x20;

<figure><img src="/files/F941kmdjjT0zIX9p8zfC" alt=""><figcaption><p>Data metrics page</p></figcaption></figure>

<figure><img src="/files/cXH9LtRBvPJR7DzIuFpB" alt=""><figcaption></figcaption></figure>

* **Clear:** It will clear all the monitoring and data metrics logs for all the components in the pipeline.

<figure><img src="/files/fcRhn6k9JbeFw0ofPEaJ" alt=""><figcaption><p>Clear option in monitoring page</p></figcaption></figure>


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.bdb.ai/data-pipeline-2/getting-started/monitoring-pipeline.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.