Job Trigger

The Job Trigger Component allows users to trigger a Python (On Demand) Job from within a pipeline. The in-event data connected to the Job Trigger component is automatically passed as a payload to the selected Python (On Demand) Job.

Key Capabilities

  • Trigger Python (On Demand) Jobs directly from a pipeline.

  • Pass event data as payload in JSON Array format.

  • Configure which job to trigger via meta-information settings.

  • Enable event-driven automation where job execution depends on pipeline outcomes.

Configuration Overview

All Job Trigger configurations are grouped into:

  • Basic Information

  • Meta Information

  • Resource Configuration

Configuring Meta Information

  1. Prepare Pipeline Data

    • Build a pipeline that generates meaningful data.

    • Ensure this data is sent to an out-event that will serve as the payload.

  2. Connect Events

    • Connect the Job Trigger Component to the event holding the payload data.

  3. Select Python (On Demand) Job

    • Open the Meta Information tab of the Job Trigger component.

    • Select the required Python (On Demand) Job from the drop-down menu.

  4. Payload Handling

    • The data from the connected event is automatically passed to the selected job.

    • The payload format:

      [
  { "field1": "value1", "field2": "value2" },
  { "field1": "value3", "field2": "value4" }
]

    • Each record from the event becomes a JSON object inside a JSON Array.

Usage Notes

  • The in-event data is always passed in JSON format.

  • Ensure that the Python (On Demand) Job is designed to accept and parse JSON arrays.

  • Use this component for event-driven execution such as:

    • Triggering data enrichment jobs when new ingestion data arrives.

    • Running analytics scripts when thresholds are met.

    • Launching report generation jobs on demand.

Example Use Cases

  • Trigger a Python job to validate incoming transaction data after ingestion.

  • Pass streaming data into an On Demand job for anomaly detection.

  • Activate a custom Python job that generates summary statistics for enriched data.

PreviousEmail ComponentNextCustom Components