Feature List
  • 8.x
    • Team Feature
      • Data-Prep Features
      • Pipeline Features
        • What's New Window Pop-up
        • Failure Db-Sync
        • Testing for Kafka 3.1.0
        • Python 3.10 R&D
        • Job History
        • SPC (Single Page for Configuration)
        • Python Jobs
        • Failure Alerts
        • Event Channel Alerts
        • Pipeline Error handling
        • Pipeline : PySpark Component (PySpark Job)
      • Dashboard Charting
        • Widget as component
        • Knowledge Graph Chart
          • Sample Library based code
        • Word Cloud
        • Tile component
        • Sankey Chart
        • Model as API Connector
        • Dataprep recipe in Dataset selection
        • Decomposition Enhancment
      • Python Upgrade
        • Core Platform : Data Services
        • Core Platform : Data Catalog
        • Core Platform : Data Center
        • Data Science Lab
      • Sonar Code Scan automation by DevOps
      • DS Lab PySpark Project.
      • Core Platform
        • Tag Feature For Data Connector , Dataset , DataStore etc..
        • DataStore & Metadatastore Migration
        • MongoDB & ClickHouse Support For DataSheet
        • Data As API WorkBench
        • Pagination in Home , DataCenter , Dataset , DataStore etc..
        • Sharing Data Connector & Data Set with View or Edit Permission.
        • Core Monitoring & Alerting
      • Data Science Lab
        • Auto Forecasting Requirements
          • User Input
          • Forecasting Method
          • Explainability
        • DSLAB Sprint May1-2023-May12-2023
        • DS LAB Sprint Apr10-Apr21
        • Provide Static Variables for DSLAB Component In AutoML
        • Scheduler For DSLAB Scripts
        • Optimisation of Model Explainability code
    • QA
    • DevOps
Powered by GitBook
On this page
  1. 8.x
  2. Team Feature
  3. Pipeline Features

Failure Alerts

If there is a Failure in a pipeline There will be an alert indicator

Introduction: Pipelines are a critical part of any development process, allowing developers to automate the data flow processes. However, pipeline failures can occur due to a variety of reasons, such as incorrect configurations, broken dependencies, or network connectivity issues. To ensure that pipeline failures are quickly identified and addressed, it is important to have a robust alerting mechanism in place.

Alert Indicator: In the event of a pipeline failure, there should be an alert indicator on two levels: pipeline list pipeline-wise and inside pipeline workflow component-wise. The pipeline list will provide an overview of all pipelines, indicating which ones are experiencing issues. The inside pipeline workflow component-wise alert indicator will show precisely where the pipeline failed, giving developers more information about the root cause of the failure.

Failure Analysis Page: Clicking on the alert indicator should take developers to the failure analysis page. The failure analysis page will give detailed information about the failure, including a description of the issue and any relevant logs or error messages. The failure analysis page should also provide options for taking action on the alert, such as rerunning the pipeline or disabling the affected component.

Action Options: It is important to provide options for taking action on the alert, as well as an option to ignore the alert with some mandatory conditions. For example, if the failure is caused by a temporary network issue, developers may choose to ignore the alert and wait for the issue to resolve itself. However, if the failure is due to a critical bug in the code, developers may need to take immediate action to fix the issue. Therefore, it is important to provide options that enable developers to make informed decisions about how to address pipeline failures.

Conclusion: In conclusion, having a robust alerting mechanism in place is crucial for identifying and addressing pipeline failures quickly. By providin an alert indicator at two levels, failure analysis pages, and options for taking action on the alert, developers can quickly diagnose and resolve issues, minimizing downtime and ensuring the smooth functioning of their software development process.

PreviousPython JobsNextEvent Channel Alerts

Last updated 2 years ago