BDB Response: BDB Data pipeline module has integrated data pipeline monitoring section which showcases important performance statistics such as memory & CPU utilization, no. of records processed, allocated CPU & memory, last processed count, total no of records processed, last processed record size, no of instances with each component names associated with the selected pipeline. Along with visualizing the key parameters, there is a component log also available.
In addition to the pipeline monitoring facility, in the data pipeline you can configure failover events for each and every component that is available.
User can create exclusive Kafka topics to handle failures and then drag n drop it into the canvas and map it with the components. If there are any failures or errors within a component during data processing these will be registered in the corresponding fail over events.
