> For the complete documentation index, see [llms.txt](https://docs.bdb.ai/pre-sales/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.bdb.ai/pre-sales/manufacturing-use-case/technical-requirements/data-processing-batch-data.md).

# Data Processing (Batch Data)

* [x]  **Requirement**: Data platform should have out-of-the-box capability of processing the data (structured, semi-structured, unstructured) in batch mode. These includes data cleansing, validations, transformations, aggregation, enrichment. 

* [x] **BDB Response**: BDB's data pipeline is an event based serverless architecture, which can handle any type of data continuous or asynchronous, real-time, or batched or both. Data may be ranging from UI activities, logs, performance events, sensor data, emails, social media to organizational documents, BDB's Lambda architecture saves users from the nitty-gritty of data interaction and facilitates smooth data ingestion. User just need to specify the Invocation type that if your data is real-time/batch. BDB Data Pipeline supports basic and advanced level data transformations through in-built components and integrated Data Preparation scripts to enhance data insight discovery. 

* [x]   **Requirement:** Support incremental (Change Data Capture) and ad-hoc data processing capabilities**.**

* [x] **BDB Response**: BDB pipeline supports incremental and ad-hoc data processing.

* [x]   **Requiremen**t: Provide scheduling capabilities with respect to batch processing jobs. Also support scheduling capabilities with 3rd party scheduling tools if required. 

* [x] **BDB Response**: BDB pipeline has built-in scheduler for batch data processing.

<figure><img src="/files/nptN8pZimYXzQXyi2FFR" alt=""><figcaption></figcaption></figure>

Additional SDK’s/API’s can be exposed to 3rd party scheduling tools

* [x] **Requirement**: Support integration with Open-Source tools and technologies.

* [x] **BDB Response**: Yes, BDB Platform supports integration with Open-source tools and technologies.&#x20;

* [x] **Requirement**: Support Edge processing capabilities for large datasets hosted on On-Premises.

* [x] **BDB Response:** Yes, BDB Platform supports edge version of platform installation over edge server to process data from Edge.


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.bdb.ai/pre-sales/manufacturing-use-case/technical-requirements/data-processing-batch-data.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.