Data Channel & Cluster Events
Last updated
Last updated
The Data Channel & Cluster Events page presents a comprehensive list of all Broker Info, Consumer Info, Topic Info, Kafka Version, and all the events used in the pipeline. It allows users to flush/delete the events.
Go through the below-given demonstration for the Data Chanel & Cluster Event page.
Navigate to the Pipeline Homepage.
Click the Data Channel & Cluster Events icon.
The Data Channel & Cluster Events page opens.
The list opens displaying the Data Channel & Cluster Events information.
The Data Channel includes the following information:
Broker Info: It will list all Kafka brokers and display the number of partitions used for each broker.
Consumer Info: It will display the number of active and rebalancing consumers.
Topic Info: It will display the number of topics.
Version information: It will display the Kafka version.
The Clustered Events page includes the following information:
On this page, all the pipelines will be listed along with the following details:
Pipeline Name: The name of the pipeline.
Number of Events: The number of Kafka events created in the selected pipeline.
Status: The running status of the pipeline, indicated by Green if active and Red otherwise.
Expand for Events: Click here to expand the selected row for a particular pipeline. This will list all Kafka events associated with the chosen pipeline along with the following information:
Name: Display the name of the Kafka event in the pipeline.
Event Name: Name of the Kafka event.
Partitions: Number of partitions in the Kafka event.
The user gets two options to apply to the listed Kafka events for the pipeline:
Flush All: This will flush all topic data in the selected pipeline.
Delete All: This will delete all topics in the selected pipeline.
Once the user clicks on the Event Name after expanding the row for the selected pipeline, the following information will be displayed on the new page for the selected Kafka Event:
This tab contains the following information for the selected Kafka Event:
Partitions: The number of partitions in the Kafka Event.
Replication Factor: Displays the replication factor of the Kafka topic. This refers to the number of copies of a Kafka topic's data maintained across different broker nodes within the Kafka cluster, ensuring high availability and fault tolerance data.
Sync Replicas: Displays the number of in-sync replicas of the Kafka topic. In-sync replicas (ISRs) are a subset of replicas fully synchronized with the leader replica for a partition. These replicas have the latest data as the leader and are capable of taking over as the leader if the current leader fails.
Segment Size: This shows the segment size of the Kafka topic. A segment is a smaller chunk of a partition log file. Segment size refers to the size of these log segments that Kafka uses to manage and store data within a partition. Kafka topics are divided into partitions, and each partition is further divided into segments.
Messages Count: Displays the number of messages in the Kafka topic.
Retention Period: Displays the retention period of the Kafka topic in hours. The retention period of a Kafka topic determines how long Kafka retains the messages in a topic before deleting them.
Additionally, this tab lists all the partition details along with their start and end offset, the number of messages in each partition, the number of replicas for each partition, and the size of the messages held by each partition.
This tab contains the following information for the selected Kafka Event:
Offset: Shows the offset number of the partition. An offset is a unique identifier assigned to each message within a partition.
Partitions: Displays the partition number where the offset belongs.
Time: It mentions the date and time of the message when it was stored at the offset.
Preview: This option helps the user to view and copy the message stored at the selected offset.
This tab shows details of consumers connected to Kafka Topic.
