The user can use the Job Monitoring feature to track a Job and its associated tasks. On this page, the user can view details such as Job Status, Last Activated (Date and Time), Last Deactivated (Date and Time), Total Allocated and Consumed CPU, and Total Allocated and Consumed Memory, all presented together on Job monitoring page.
Please go through the below given walk-through on the Job monitoring function.
Job Monitoring
Accessing Job Monitoring Page
The user can access the Job Monitoring icon on the List Jobs and Job Workflow Editor pages.
Navigate to the List Jobs page.
The JobMonitoring icon can be seen for all the listed Jobs.
Accessing Job Monitoring page from List Jobs page.
OR
Navigate to the Job Workflow Editor page.
The Job Monitoring icon is provided on the Header panel.
Accessing Job Monitoring page from Job Workflow Editor.
The Job Monitoringpage opens displaying the details of resource usage for the selected job.
Monitoring Page for a Spark Job
The below-given images displays Monitoring page for the Spark Job with details on the Spark driver and executor.
Monitoring page for Spark Job
Displaying the monitoring details of the Spark Job Driver
Monitoring page for Spark Job - Driver
Displaying the monitoring details of the Spark Job Executer
Monitoring page for Spark Job - Executer
Monitoring Page for a PySpark Job
The below-given images displays Monitoring page for the PySpark Job with details on the PySpark driver and executor.
Monitoring page for PySpark Job
Displaying the monitoring details of the PySpark Job Driver
Monitoring page for PySpark Job - Driver
Displaying the monitoring details of the PySpark Job Executer
Monitoring page for PySpark Job - Executer
Monitoring Page for a Python Job
Monitoring Page for Python Job
If Memory or Core allocated to the component is less than required, then it will be displayed in red color as shown in the below image.
Clear: It will clear all the monitoring details of the selected Job.
