> For the complete documentation index, see [llms.txt](https://docs.bdb.ai/7.6/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.bdb.ai/7.6/data-pipeline/components/consumers/kafka-consumer.md).

# Kafka Consumer

The Kafka Consumer component consumes messages from the internal \[pipeline] or external brokers.

All component configurations are classified broadly into 3 section

* [Basic ](/7.6/data-pipeline/components/component-base-configuration.md)
* Metadata
* [Resource Configuration](/7.6/data-pipeline/components/resource-configuration.md)

{% hint style="info" %}
Check out the steps provided in the demonstration to configure the Kafka Consumer component.
{% endhint %}

<figure><img src="https://4128708524-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FGDmsjfjJBNqow7Fo97cO%2Fuploads%2FBgLrAPQMJ3IP32CZsbna%2FKafka%20Consumer%20Component%20GIF.gif?alt=media&amp;token=2927c314-6cbb-4533-aeaa-589749db0eb9" alt=""><figcaption><p>Configuring the Kafka Consumer component</p></figcaption></figure>

{% hint style="info" %}
*<mark style="color:green;">Please Note</mark>: It currently supports SSL and plaintext as security types.*
{% endhint %}

<figure><img src="https://4128708524-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FGDmsjfjJBNqow7Fo97cO%2Fuploads%2FiaDIOtqwZJRUXzuB5jZ1%2Fimage.png?alt=media&amp;token=c8269b26-67ab-4c99-ba1e-69bc8225dac0" alt=""><figcaption><p>Component being used to read data from the external broker.</p></figcaption></figure>

This Component can read the data from external Brokers as well with SSL as the security type and host Aliases:

<figure><img src="https://4128708524-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FGDmsjfjJBNqow7Fo97cO%2Fuploads%2FtoYhWyiNVrr7iOcWjKRc%2Fimage.png?alt=media&amp;token=7aef7040-44b3-40d1-9599-652f25d5e2ac" alt=""><figcaption><p>Kafka consumer Meta Information tab</p></figcaption></figure>

## Steps to Configure

* Drag and drop the Kafka Consumer Component to the Workflow Editor from the Consumer component group. &#x20;
* Click on the dragged Kafka Consumer component to get the component properties tabs.

### Basic Information Tab

The Basic Information tab is the default tab to open while using a click on the dragged component. Configure the **Basic Information** tab by following the given steps: &#x20;

* Select an Invocation type from the drop-down menu to confirm the running mode of the component. Select ‘**Real-Time**’ from the drop-down menu.
* Deployment Type: It displays the deployment type for the component. This field comes pre-selected.
* Container Image Version: It displays the image version for the docker container. This field comes pre-selected.
* Failover Event: Select a failover Event from the drop-down menu.
* Batch Size (min 10): Provide the maximum number of records to be processed in one   execution cycle (Min limit for this field is 10.
* Enable Auto-Scaling: Component pod scale up automatically based on a given max instance, if component lag is more than 60%.

### Meta Information Tab

Click on the **Meta Information tab** to open the properties fields and configure the **Meta Information tab** by providing the required fields.

* **Topic Name:** Specify a topic name from where you want to consume data.
* **Start From:** It contains the following start from:
  * **Processed:** Using this option the component will consume the message from the last processed offset. if the component is getting deployed for the first time it will read from the&#x20;
  * **Beginning:** Using this option consumes live processed data and already processed data from the beginning
  * **Latest:** Using this option consumes the latest processed data.
  * **Timestamp:** Using this option consumes data between given interval times.
* **Is External:** The User can consume external topic data from the external bootstrap server by enabling **'Is External'** option. **‘Bootstrap Server’ and ‘Config’** fields will display after enabling the  **'Is External'** option.
  * **Bootstrap Server:** Enter external bootstrap details.
  * **Config:** Enter configuration details of external details.
* **Input Record Type:** It contains the following input record type:
  * **CSV:** The User can consume csv data using this option. **‘**&#x48;eaders’ and ‘Separator’ fields will display if the user select choose CSV input record type.
    * &#x20;**Header:** In this field, the user can enter column names of CSV data that consume from the Kafka topic.
    * **Separator:** In this field, the user can enter separators like comma (,) that used CSV data.
  * **JSON:** The User can consume JSON data using this option.
  * **XML:** The User can consume parquet data using this option.
  * **AVRO:** The User can consume Avro data using this option.
* **Security Type:** It contains the following security type:
  * **Plain Text:** Choose the Plain Text option if there environment without SSL.
  * **SSL:** Choose the SSL option if there environment with SSL. It will display the following fields:
    * **Trust Store Location:** Provide the trust store path.
    * **Trust Store Password:** Provide the trust store password.
    * **Key Store Location:** Provide the key store path.
    * **Key Store Password:** Provide the key store password.
    * **SSL Key Password:** Provide the SSL key password.
    * **Host Aliases:** It contains the following fields:
      * **IP:** Provide the IP.
      * **Host Names:** Provide the host names.

### Saving the Component Configuration

* After doing all the configurations click the ‘**Save Component in Storage**’ icon provided in the configuration panel to save the component.
* A notification message appears to inform about the component configuration saved.