# Components

- [Adding Components to Workflow](https://docs.bdb.ai/data-pipeline-2/components/adding-components-to-workflow.md)
- [Component Architecture](https://docs.bdb.ai/data-pipeline-2/components/component-architecture.md)
- [Component Base Configuration](https://docs.bdb.ai/data-pipeline-2/components/component-base-configuration.md): This page pays attention to describe the Basic Info tab provided for the pipeline components. This tab has to be configured for all the components.
- [Resource Configuration](https://docs.bdb.ai/data-pipeline-2/components/resource-configuration.md): There is a resource configuration tab while configuring the components.
- [Intelligent Scaling](https://docs.bdb.ai/data-pipeline-2/components/intelligent-scaling.md): A way to scale up the processing speed of components.
- [Connection Validation](https://docs.bdb.ai/data-pipeline-2/components/connection-validation.md): The Connection Validation option helps the users to validate the connection details of the db/cloud storages.
- [Readers](https://docs.bdb.ai/data-pipeline-2/components/readers.md): Readers are a group of components that can read data from different DB and cloud storages in both invocation types i.e., Real-Time and Batch.
- [S3 Reader](https://docs.bdb.ai/data-pipeline-2/components/readers/s3-reader.md)
- [HDFS Reader](https://docs.bdb.ai/data-pipeline-2/components/readers/hdfs-reader.md)
- [DB Reader](https://docs.bdb.ai/data-pipeline-2/components/readers/db-reader.md)
- [ES Reader](https://docs.bdb.ai/data-pipeline-2/components/readers/es-reader.md)
- [SFTP Stream Reader](https://docs.bdb.ai/data-pipeline-2/components/readers/sftp-stream-reader.md)
- [SFTP Reader](https://docs.bdb.ai/data-pipeline-2/components/readers/sftp-reader.md)
- [Mongo DB Reader](https://docs.bdb.ai/data-pipeline-2/components/readers/mongo-db-reader.md): Mongo DB reader component contains both the deployment-types: Spark & Docker
- [MongoDB Reader Lite (PyMongo Reader)](https://docs.bdb.ai/data-pipeline-2/components/readers/mongo-db-reader/mongodb-reader-lite-pymongo-reader.md)
- [MongoDB Reader](https://docs.bdb.ai/data-pipeline-2/components/readers/mongo-db-reader/mongodb-reader.md): This page covers configuration details for the MongoDB Reader component.
- [Azure Blob Reader](https://docs.bdb.ai/data-pipeline-2/components/readers/azure-blob-reader.md)
- [Azure Metadata Reader](https://docs.bdb.ai/data-pipeline-2/components/readers/azure-metadata-reader.md)
- [ClickHouse Reader (Docker)](https://docs.bdb.ai/data-pipeline-2/components/readers/clickhouse-reader-docker.md)
- [Sandbox Reader](https://docs.bdb.ai/data-pipeline-2/components/readers/sandbox-reader.md): A Sandbox reader is used to read and access data within a configured sandbox environment.
- [Azure Blob Reader](https://docs.bdb.ai/data-pipeline-2/components/readers/azure-blob-reader-1.md)
- [Writers](https://docs.bdb.ai/data-pipeline-2/components/writers.md): Data writers specifically focus on the final stage of the pipeline, where the processed or transformed data is written to the target destination. This section explains all the supported Data Writers.
- [S3 Writer](https://docs.bdb.ai/data-pipeline-2/components/writers/s3-writer.md)
- [DB Writer](https://docs.bdb.ai/data-pipeline-2/components/writers/db-writer.md)
- [HDFS Writer](https://docs.bdb.ai/data-pipeline-2/components/writers/hdfs-writer.md)
- [ES Writer](https://docs.bdb.ai/data-pipeline-2/components/writers/es-writer.md)
- [Video Writer](https://docs.bdb.ai/data-pipeline-2/components/writers/video-writer.md)
- [Azure Writer](https://docs.bdb.ai/data-pipeline-2/components/writers/azure-writer.md)
- [ClickHouse Writer (Docker)](https://docs.bdb.ai/data-pipeline-2/components/writers/clickhouse-writer-docker.md): Along with the Spark Driver in RDBMS Writer we have Docker writer that supports TCP port.
- [Sandbox Writer](https://docs.bdb.ai/data-pipeline-2/components/writers/sandbox-writer.md): A Sandbox writer is used to writer data within a configured sandbox environment.
- [MongoDB Writers](https://docs.bdb.ai/data-pipeline-2/components/writers/mongodb-writers.md): We have given two different writers for writing data to MongoDB. The available deployment types for the same are: Spark and Docker.
- [MongoDB Writer](https://docs.bdb.ai/data-pipeline-2/components/writers/mongodb-writers/mongodb-writer.md)
- [MongoDB Writer Lite (PyMongo Writer)](https://docs.bdb.ai/data-pipeline-2/components/writers/mongodb-writers/mongodb-writer-lite-pymongo-writer.md)
- [Machine Learning](https://docs.bdb.ai/data-pipeline-2/components/machine-learning.md): These components utilize machine learning algorithms and techniques to analyze and model the data.
- [DSLab Runner](https://docs.bdb.ai/data-pipeline-2/components/machine-learning/dslab-runner.md)
- [AutoML Runner](https://docs.bdb.ai/data-pipeline-2/components/machine-learning/automl-runner.md)
- [Consumers](https://docs.bdb.ai/data-pipeline-2/components/consumers.md): These are the real-time / Streaming component that ingests data or monitor for change in data objects from different sources to the pipeline.
- [SFTP Monitor](https://docs.bdb.ai/data-pipeline-2/components/consumers/sftp-monitor.md)
- [MQTT Consumer](https://docs.bdb.ai/data-pipeline-2/components/consumers/mqtt-consumer.md)
- [Video Stream Consumer](https://docs.bdb.ai/data-pipeline-2/components/consumers/video-stream-consumer.md)
- [Eventhub Subscriber](https://docs.bdb.ai/data-pipeline-2/components/consumers/eventhub-subscriber.md)
- [Twitter Scrapper](https://docs.bdb.ai/data-pipeline-2/components/consumers/twitter-scrapper.md)
- [Mongo ChangeStream](https://docs.bdb.ai/data-pipeline-2/components/consumers/mongo-changestream.md)
- [Rabbit MQ Consumer](https://docs.bdb.ai/data-pipeline-2/components/consumers/rabbit-mq-consumer.md)
- [AWS SNS Monitor](https://docs.bdb.ai/data-pipeline-2/components/consumers/aws-sns-monitor.md)
- [Kafka Consumer](https://docs.bdb.ai/data-pipeline-2/components/consumers/kafka-consumer.md)
- [API Ingestion and Webhook Listener](https://docs.bdb.ai/data-pipeline-2/components/consumers/api-ingestion-and-webhook-listener.md)
- [Producers](https://docs.bdb.ai/data-pipeline-2/components/producers.md): The role of data producers is to ensure a continuous flow of data into the pipeline, providing the necessary raw material for subsequent processing and analysis.
- [WebSocket Producer](https://docs.bdb.ai/data-pipeline-2/components/producers/websocket-producer.md): A WebSocket producer component is a software component that is used to send data over a WebSocket connection.
- [Eventhub Publisher](https://docs.bdb.ai/data-pipeline-2/components/producers/eventhub-publisher.md): The EventHub Publisher leverages the scalability and throughput capabilities of Event Hubs to ensure efficient and reliable event transmission.
- [EventGrid Producer](https://docs.bdb.ai/data-pipeline-2/components/producers/eventgrid-producer.md): The EventHub Publisher serves as a bridge between the transformed data within the pipeline and the Azure Event Hubs service. It ensures the efficient and reliable transmission of data.
- [RabbitMQ Producer](https://docs.bdb.ai/data-pipeline-2/components/producers/rabbitmq-producer.md): RabbitMQ producer plays a vital role in enabling reliable message-based communication and data flow within a data pipeline.
- [Kafka Producer](https://docs.bdb.ai/data-pipeline-2/components/producers/kafka-producer.md): The Kafka producer acts as a data source within the pipeline, generating and publishing messages to Kafka for subsequent processing and consumption.
- [Synthetic Data Generator](https://docs.bdb.ai/data-pipeline-2/components/producers/synthetic-data-generator.md)
- [Transformations](https://docs.bdb.ai/data-pipeline-2/components/transformations.md)
- [SQL Component](https://docs.bdb.ai/data-pipeline-2/components/transformations/sql-component.md): SQL transformer applies SQL operations to transform and manipulate data, providing flexibility and expressiveness in data transformations within a data pipeline.
- [Dateprep Script Runner](https://docs.bdb.ai/data-pipeline-2/components/transformations/dateprep-script-runner.md)
- [File Splitter](https://docs.bdb.ai/data-pipeline-2/components/transformations/file-splitter.md)
- [Rule Splitter](https://docs.bdb.ai/data-pipeline-2/components/transformations/rule-splitter.md)
- [Stored Producer Runner](https://docs.bdb.ai/data-pipeline-2/components/transformations/stored-producer-runner.md)
- [Flatten JSON](https://docs.bdb.ai/data-pipeline-2/components/transformations/flatten-json.md)
- [Email Component](https://docs.bdb.ai/data-pipeline-2/components/transformations/email-component.md)
- [Pandas Query Component](https://docs.bdb.ai/data-pipeline-2/components/transformations/pandas-query-component.md)
- [Enrichment Component](https://docs.bdb.ai/data-pipeline-2/components/transformations/enrichment-component.md)
- [Mongo Aggregation](https://docs.bdb.ai/data-pipeline-2/components/transformations/mongo-aggregation.md)
- [Data Loss Protection](https://docs.bdb.ai/data-pipeline-2/components/transformations/data-loss-protection.md)
- [Data Preparation (Docker)](https://docs.bdb.ai/data-pipeline-2/components/transformations/data-preparation-docker.md)
- [Rest Api Component](https://docs.bdb.ai/data-pipeline-2/components/transformations/rest-api-component.md)
- [Schema Validator](https://docs.bdb.ai/data-pipeline-2/components/transformations/schema-validator.md)
- [Scripting](https://docs.bdb.ai/data-pipeline-2/components/scripting.md): Data Pipeline module provides two types of scripting components to facilitate the users.
- [Script Runner](https://docs.bdb.ai/data-pipeline-2/components/scripting/script-runner.md)
- [Python Script](https://docs.bdb.ai/data-pipeline-2/components/scripting/python-script.md)
- [Keeping Different Versions of the Python Script in VCS](https://docs.bdb.ai/data-pipeline-2/components/scripting/python-script/keeping-different-versions-of-the-python-script-in-vcs.md)
- [Scheduler](https://docs.bdb.ai/data-pipeline-2/components/scheduler.md): A task can be scheduled to automatically execute at a given scheduler time.


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.bdb.ai/data-pipeline-2/components.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.