Script Executer Job

The Script Executor job is designed to execute code snippets or scripts written in various programming languages such as Go, Julia, and Python. This job allows users to fetch code from their Git repository and execute it seamlessly.

Creating a Script Executer Job

• Navigate to the Data Pipeline module homepage.

• Open the pipeline homepage and click on the Create option.

• The new panel opens from right hand side. Click on Create button in Job option.

  • Enter a name for the new Job.

  • Describe the Job (Optional).

  • Job Baseinfo: Select Script Executer from the drop-down.

  • Trigger By: There are 2 options for triggering a job on success or failure of a job:

    • Success Job: On successful execution of the selected job the current job will be triggered.

    • Failure Job: On failure of the selected job the current job will be triggered.

  • Is Scheduled?

    • A job can be scheduled for a particular timestamp. Every time at the same timestamp the job will be triggered.

    • Job must be scheduled according to UTC.

  • Docker Configuration: Select a resource allocation option using the radio button. The given choices are:

    • Low

    • Medium

    • High

  • Provide the resources required to run the python Job in the limit and Request section.

    • Limit: Enter max CPU and Memory required for the Python Job.

    • Request: Enter the CPU and Memory required for the job at the start.

    • Instances: Enter the number of instances for the Python Job.

  • Alert: Please refer to the Job Alerts page to configure alerts in job.
  • Click the Save option to save the Python Job.

  • The Script Executer Job gets saved, and it will redirect the users to the Job Editor workspace.

  

  Cresting a Script Executer Job

Please go through the demonstration given-below to configure the Script Executor.

Configuring Script Executer

Configuring the Meta Information tab

Meta information for the Script Executer

• Git Config: Select an option from the drop-down.

  • Personal: If this option is selected, provide the following information:

    • Git URL: Enter the Git URL.

      • URL for Github: https://github.com
      • URL for Gitlab: https://gitlab.com
      • User Name: Enter the GIT username.

      • Token: Enter the Access token or API token for authentication and authorization when accessing the Git repository, commonly used for secure automated processes like code fetching and execution.

      • Branch: Specify the Git branch for code fetching.

      • Script Type: Select the script's language for execution from the drop down:

        • GO

        • Julia

        • Python

      • Start Script: Enter the script name (with extension) which has to be executed.

        • For example, if Python is selected as the Script type, then the script name will be in the following format: script_name.py.

        • If Golang is selected as the Script type, then the script name will be in the following format: script_name.go.

      • Start Function: Specify the initial function or method within the Start Script for execution, particularly relevant for languages like Python with reusable functions.

      • Repository: Provide the Git repository name.

      • Input Arguments: Allows users to provide input parameters or arguments needed for code execution, such as dynamic values, configuration settings, or user inputs affecting script behavior.

  • Admin: If this option is selected, then Git URL, User Name, Token & Branch fields have to be configured in the platform in order to use Script Executor.

    • In this option, the user has to provide the following fields:

      • Script Type

      • Start Script

      • Start Function

      • Repository

      • Input Arguments

Please Note: Follow the below given steps to configure GitLab/GitHub credentials in the Admin Settings in the platform:

• Navigate to Admin >> Configurations >> Version Control.

• From the first drop-down menu, select the Version.

• Choose 'DsLabs' as the module from the drop-down.

• Select either GitHub or GitLab based on the requirement for Git type.

• Enter the host for the selected Git type.

• Provide the token key associated with the Git account.

• Select a Git project.

• Choose the branch where the files are located.

• After providing all the details correctly, click on 'Test,' and if the authentication is successful, an appropriate message will appear. Subsequently, click on the 'Save' option.

• To complete the configuration, navigate to My Account >> Configuration. Enter the Git Token and Git Username, then save the changes.

Setting GitLab/GitHub credentials under My Account

Setting GitLab/GitHub credentials under My Account>>Configuration