Data Pipeline

Does the BDB Platform provide R script?

Yes, there is an option for R Scripting in the BDB Data Pipeline module.

Does the BDB Platform enable to read the data from AWS S3, Desktop, Folder on server?

Yes, the BDB Data Pipeline enables to read XML and JSON data from the AWS S3, Desktop, Folder on a server.

Does the BDB Platform support Manage Memory Computing?

BDB Pipeline enables data engineers to configure the resource allocation to different components based on the load and operation. It allows the creation of multiple instances to run in parallel to complete the operation in the desired time frame. Also, the BDB Data Pipeline module has an integrated data pipeline monitoring section that showcases important performance statistics such as memory & CPU utilization, no. of records processed, allocated CPU & memory, last processed count, total no. of records processed, last processed record size, no. of instances with each component names associated with the selected pipeline. Along with visualizing the key parameters, there is a component log also available.

Does the BDB Platform support Incremental Extraction?

BDB Data Pipeline handles incremental data extraction to the BDB Data Store. The pipeline has a built-in scheduler component to run at the scheduled interval to capture the increments. It also enables capturing the changed steam.

Does the BDB Platform support Scheduler base Data?

BDB Data Pipeline has a built-in scheduler component to schedule the data refresh.

What type of performance tuning options are available for this type of implementation?

The performance tuning options offered by the BDB Platform are as follows: Data Pipeline: The users can monitor data workflows through the monitoring feature provided under the BDB Data Pipeline modules. It provides them with a variety of performance-tuning options:

• Optimizing resource utilization

• Detecting and preventing errors

• Improving the efficiency of pipelines

• Ensuring data quality

• Increasing transparency

E.g., If a user is writes MongoDB queries within the BDB Data Set and the query response time is prolonged, the user can implement performance tuning options such as aggregated views, indexing, sharding, etc. to enhance the query execution performance.