Data Pipeline
  • Data Pipeline
    • About Data Pipeline
    • Design Philosophy
    • Low Code Visual Authoring
    • Real-time and Batch Orchestration
    • Event based Process Orchestration
    • ML and Data Ops
    • Distributed Compute
    • Fault Tolerant and Auto-recovery
    • Extensibility via Custom Scripting
  • Getting Started
    • Homepage
      • List Pipelines
      • Create
        • Creating a New Pipeline
          • Adding Components to Canvas
          • Connecting Components
            • Events [Kafka and Data Sync]
          • Memory and CPU Allocations
        • Creating a New Job
          • Job Editor Page
          • Task Components
            • Readers
              • HDFS Reader
              • MongoDB Reader
              • DB Reader
              • S3 Reader
              • Azure Blob Reader
              • ES Reader
              • Sandbox Reader
            • Writers
              • HDFS Writer
              • Azure Writer
              • DB Writer
              • ES Writer
              • S3 Writer
              • Sandbox Writer
              • Mongodb Writer
              • Kafka Producer
            • Transformations
          • PySpark Job
          • Python Job
      • List Jobs
      • List Components
      • Delete Orphan Pods
      • Scheduler
      • Data Channel
      • Cluster Event
      • Trash
      • Settings
    • Pipeline Workflow Editor
      • Pipeline Toolbar
        • Pipeline Overview
        • Pipeline Testing
        • Search Component in Pipelines
        • Push Pipeline (to VCS/GIT)
        • Pull Pipeline
        • Full Screen
        • Log Panel
        • Event Panel
        • Activate/Deactivate Pipeline
        • Update Pipeline
        • Failure Analysis
        • Pipeline Monitoring
        • Delete Pipeline
        • Pipeline Component Configuration
        • Pipeline Failure Alert History
      • Component Panel
      • Right-side Panel
    • Testing Suite
    • Activating Pipeline
    • Monitoring Pipeline
    • Job Monitoring
  • Components
    • Adding Components to Workflow
    • Component Architecture
    • Component Base Configuration
    • Resource Configuration
    • Intelligent Scaling
    • Connection Validation
    • Readers
      • S3 Reader
      • HDFS Reader
      • DB Reader
      • ES Reader
      • SFTP Stream Reader
      • SFTP Reader
      • Mongo DB Reader
        • MongoDB Reader Lite (PyMongo Reader)
        • MongoDB Reader
      • Azure Blob Reader
      • Azure Metadata Reader
      • ClickHouse Reader (Docker)
      • Sandbox Reader
      • Azure Blob Reader
    • Writers
      • S3 Writer
      • DB Writer
      • HDFS Writer
      • ES Writer
      • Video Writer
      • Azure Writer
      • ClickHouse Writer (Docker)
      • Sandbox Writer
      • MongoDB Writers
        • MongoDB Writer
        • MongoDB Writer Lite (PyMongo Writer)
    • Machine Learning
      • DSLab Runner
      • AutoML Runner
    • Consumers
      • SFTP Monitor
      • MQTT Consumer
      • Video Stream Consumer
      • Eventhub Subscriber
      • Twitter Scrapper
      • Mongo ChangeStream
      • Rabbit MQ Consumer
      • AWS SNS Monitor
      • Kafka Consumer
      • API Ingestion and Webhook Listener
    • Producers
      • WebSocket Producer
      • Eventhub Publisher
      • EventGrid Producer
      • RabbitMQ Producer
      • Kafka Producer
      • Synthetic Data Generator
    • Transformations
      • SQL Component
      • Dateprep Script Runner
      • File Splitter
      • Rule Splitter
      • Stored Producer Runner
      • Flatten JSON
      • Email Component
      • Pandas Query Component
      • Enrichment Component
      • Mongo Aggregation
      • Data Loss Protection
      • Data Preparation (Docker)
      • Rest Api Component
      • Schema Validator
    • Scripting
      • Script Runner
      • Python Script
        • Keeping Different Versions of the Python Script in VCS
    • Scheduler
  • Custom Components
  • Advance Configuration & Monitoring
    • Configuration
      • Default Component Configuration
      • Logger
    • Data Channel
    • Cluster Events
    • System Component Status
  • Version Control
  • Use Cases
Powered by GitBook
On this page
  • Configuring the Meta Information tab of Eventhub Publisher
  • Read using Connection String
  • Read using Principal Secret

Was this helpful?

  1. Components
  2. Producers

Eventhub Publisher

The EventHub Publisher leverages the scalability and throughput capabilities of Event Hubs to ensure efficient and reliable event transmission.

PreviousWebSocket ProducerNextEventGrid Producer

Was this helpful?

All component configurations are classified broadly into 3 section

  • ​​

  • Meta Information

  • ​​

Please follow the steps given in the walk-through to configure the Eventhub Publisher component.

Configuring the Meta Information tab of Eventhub Publisher

There are two read using methods:

  1. Connection String

  2. Principal Secret

Read using Connection String

  • Connection String: It is a string of parameters that are used to establish a connection to an Azure EventHub.

  • Consumer Group: It is a logical grouping of event consumers (subscribers) that read and process events from the same partition of an event hub.

  • EventHub Name: It refers to the specific Event Hub within the Event Hubs namespace to which data is being sent or received.

  • Checkpoint Location: It is a location in the event stream that represents the last event that has been successfully processed by the subscriber.

  • Enqueued time: It indicates the time when the event was added to the partition, which is typically the time when the event occurred or was generated.

  • Publisher namespace: It is a logical entity that is used to group related publishers and manage access control to EventHubs within the namespace.

Read using Principal Secret

  • Client ID: The ID of the Azure AD application that has been registered in the Azure portal and that will be used to authenticate the publisher. This can be found in the Azure portal under the "App registrations" section.

  • Tenant ID: The ID of the Azure AD tenant that contains the Azure AD application and service principal that will be used to authenticate the publisher.

  • Client secret: The secret value that is associated with the Azure AD application and that will be used to authenticate the publisher.

  • Consumer group: It is a logical grouping of event producer(publisher) that read and process events from the same partition of an event hub.

  • EventHub Name: It refers to the specific Event Hub within the Event Hubs namespace to which data is being sent or received.

  • Checkpoint Location: It is a location in the event stream that represents the last event that has been successfully processed by the publisher.

  • Enqueued time: It indicates the time when the event was added to the partition, which is typically the time when the event occurred or was generated.

  • Publisher namespace: It is a logical entity that is used to group related publishers and manage access control to EventHubs within the namespace.

​Basic Information​
Resource Configuration​
Configuring the EventGrid Publisher component
Read using connection string
Read using principal secret