Python Jobs

We need to deploy script as python jobs

DS Lab component : Transformation and Writer (Boiler-Plate)

We need a Tag for Python and PySpark so different jobs can be created.

The Importance of Tags: Tags are a powerful tool for organizing and managing jobs in a deployment environment. By using tags to differentiate between Python and PySpark scripts, developers can create separate jobs that are optimized for each process. This ensures that code is executed efficiently and we don't create unnecessary spark context.