Scheduler

A task can be scheduled to automatically execute at a given scheduler time.

Scheduler as a Component

Check out the walk-through on Scheduler to get an idea on how to configure & use it in a workflow.

Configuring & Using the Scheduler Component in the Data Pipeline Workflow

All component configurations are classified broadly into the following sections:

• Drag and drop the Scheduler component to the Workflow Editor.

• Connect it with a reader or Data Loading component (Input event).

• Click on the Scheduler component to get the configuration details.

• The Basic Information tab opens by default.

  • Select an Invocation type from the drop-down menu to confirm the running mode of the reader component. The supported invocation type is Real-Time.

  • Deployment Type: It displays the deployment type for the component. This field comes pre-selected.

  • Container Image Version: It displays the image version for the docker container. This field comes pre-selected.

  • Failover Event: Select a failover event from the drop-down menu.

  • Batch Size (min 10): Provide the maximum number of records to be processed in one execution cycle (Min limit for this field is 10).

• Open the Meta Information tab and provide the required details for the same.

• Scheduler Name: Provide a name for the Scheduler.

• Payload: Provide the Payload value in the predefined format.

• Scheduler Time: Time generated based on user selection in Cron generator.

• Cron Generator: A cron generator generates cron expression which is a string that represents a set of times at which a task will be executed. The user can set any time using the given units of time from Minutes to Year.

Please Note:

• The different values for various time units will be as given below:

  • Hours(0-23)

  • Minute(0-59)

  • Seconds(0-59)

  • Months in words(January-December)

  • Months in digit(1-12)

  • Week day(Monday-Sunday)

  • Date of months(1-31) The Date of the months for the February month is up to 29.

Cron Generator

The supported units of time for generating a Cron expression are as given below:

• MINUTES - Scheduler can be scheduled at every given scheduled time in minutes, and seconds.

• HOURLY - Scheduler can be scheduled at every given scheduled time in Hours, Minutes, and seconds.

• DAILY - Scheduler can be scheduled on daily basis.

  • (Every working day at) We Have one more option to schedule this task for every working day at given time interval.

  
• WEEKLY - Scheduler can be scheduled for every given day of the week at specified time.

• MONTHLY - Scheduler can be scheduled at every given date of a given month and specified time/given time interval.

  • During the nearest Weekday: By enabling this option the Pipeline will be scheduled during the nearest weekday.

  • We have another option where scheduler can be scheduled at first-last occurrence of day (Monday-Sunday) starting at given month at a specified time and ending at given count of months. E.g., Second Monday of every month starting March and continuing for 3 months(count).

• YEARLY - Scheduler can be scheduled for every given day of the any month at specified time which will execute only every year.

  • During the nearest Weekday: By enabling this option the Pipeline will be scheduled during the nearest weekday.

  • The user has another option where scheduler can be scheduled at first-last occurence of day (Monday-Sunday) of a given month at a specified time.

Scheduler List Page

The scheduled pipelines are listed together with the scheduler details. It displays the meta-information filled in the scheduler component for the respective pipeline. The page also contains information on how many times the pipeline has been triggered and when the next time the scheduled component will get deployed.

Check out the given walk-through on the Scheduler List Page.

Scheduler List page

• All the scheduled workflows get displayed on the Scheduler List Page of the Data Pipeline.

• The user can access the Scheduler icon from the left-side panel of the Pipeline landing page.

• By clicking on a Pipeline the right-side panel describes how many times the Pipeline has been triggered.

• The user gets redirected to the concerned pipeline by clicking the Pipeline name from the Scheduler List page.

Schduler List