Kafka Producer

The Kafka Producer task is used to publish data to an Apache Kafka topic. A Kafka producer is a client application responsible for sending messages to Kafka brokers. These messages are distributed to consumers based on topic, partitioning strategy, and broker configuration.

Prerequisites

• A running Kafka cluster (internal or external) accessible from the job environment.

• The target Kafka topic already created (or permissions to auto-create if enabled).

• Authentication and security details (Plain Text or SSL) for connecting to Kafka.

• Knowledge of bootstrap server addresses and broker host aliases if producing to an external cluster.

Configuring the Meta Information Tab

To configure the Kafka Producer:

1. Drag the Kafka Producer task to the Workspace.

2. Click on the task to open its configuration tabs.

3. The Meta Information tab opens by default. Configure the following fields:

Field	Description
Topic Name	The Kafka topic name where messages will be produced.
Security Type	Select the security type from the drop-down: <ul><li>Plain Text — no encryption/authentication.</li><li>SSL — encrypted connection.</li></ul>
Is External	Enable this option to produce data to an external Kafka cluster. When enabled, additional fields (Bootstrap Server, Config) become visible.
Bootstrap Server	Provide the bootstrap server connection string (e.g., broker1:9092,broker2:9092).
Config	Enter additional producer configuration details (key-value pairs).
Host Aliases	Define alternative hostnames for Kafka brokers in the cluster. Useful when brokers must be addressed by aliases.
IP	Enter the IP address corresponding to the broker hostname.
Host Names	Enter the hostnames that map to the provided IP addresses.

Save & Next Steps

• Once configuration is complete, click Save Task In Storage to persist the Kafka Producer settings.

• Run a test write to validate connectivity, topic availability, and authentication.

• Review logs to confirm successful message production.