Job Trigger

Job Triggers define the "when" of an operation, and schedules to automate task execution at specific intervals.

The Job Trigger Component allows users to trigger a Python (On Demand) Job from within a pipeline. The in-event data connected to the Job Trigger component is automatically passed as a payload to the selected Python (On Demand) Job.

Key Capabilities

• Trigger Python (On Demand) Jobs directly from a pipeline.

• Pass event data as payload in JSON Array format.

• Configure which job to trigger via meta-information settings.

• Enable event-driven automation where job execution depends on pipeline outcomes.

Configuration Overview

All Job Trigger configurations are grouped into:

• Basic Information

• Meta Information

• Resource Configuration

Configuring Meta Information

1. Prepare Pipeline Data

   • Build a pipeline that generates meaningful data.

   • Ensure this data is sent to an out-event that will serve as the payload.

2. Connect Events

   • Connect the Job Trigger Component to the event holding the payload data.

3. Select Python (On Demand) Job

   • Open the Meta Information tab of the Job Trigger component.

   • Select the required Python (On Demand) Job from the drop-down menu.

4. Payload Handling

   • The data from the connected event is automatically passed to the selected job.

   • The payload format:

     [
       { "field1": "value1", "field2": "value2" },
       { "field1": "value3", "field2": "value4" }
     ]

   • Each record from the event becomes a JSON object inside a JSON Array.

Usage Notes

• The in-event data is always passed in JSON format.

• Ensure that the Python (On Demand) Job is designed to accept and parse JSON arrays.

• Use this component for event-driven execution such as:

  • Triggering data enrichment jobs when new ingestion data arrives.

  • Running analytics scripts when thresholds are met.

  • Launching report generation jobs on demand.

Example Use Cases

• Trigger a Python job to validate incoming transaction data after ingestion.

• Pass streaming data into an On-Demand job for anomaly detection.

• Activate a custom Python job that generates summary statistics for enriched data.