Start your Data Science Experiment with DS Lab

Follow the given sequence to accomplish your experimentation with Data Science Lab

Create your project

Check out the walk-through on how to create a DSL Project.

Creating a new DSL Project

• Navigate to the Projects Page of the Data Science Lab plugin.

• Click the Create Project option to create a new project.

The Create Project option

• A form opens to provide the Project-related information.

• The next screen opens asking for the following details for a new project:

  • Project Name: Give a name to the new project.

  • Project Description: Describe the project.

  • Select Algorithms: Select Algorithms using the drop-down menu.

  • Environment: Allows users to select the environment they want to work in. Currently, the supported Python frameworks are TensorFlow and PyTorch.

    • If the users select the TensorFlow environment, they do not need to install packages like the TensorFlow and Keras explicitly in the notebook. These packages can simply be imported inside the notebook.

    • If the users select the PyTorch environment, they do not need to install packages like the Torch and Torchvision explicitly in the notebook. These packages can simply be imported inside the notebook.

    The user can select an option out of the given choices: 1. Python Tensor Flow, 2. Python PyTorch

  • Resource Allocation: This allows the users to allocate CPU/ GPU and memory to be used by the Notebook container inside a given project. The currently supported Resource Allocation options are Low, Medium, and High.

  • Idle Shutdown: Idle Shutdown: It allows the users to specify the idle time limit after which the notebook session will get disconnected, and the project will be deactivated. To use the notebook again, the project should be activated. The supported Idle Shutdown options are 30m, 1h, and 2h.

  • External Libraries: Provide the external libraries’ links required for the project.

Mandatory fields for creating a Project

• Based on the selection of the Resource Allocation field the following fields appear with pre-selected values:

  • Image Name

  • Image Version

  • Limit

  • Memory

  • Request (CPU)

  • Memory

• Select nvidia from the GPU Type field to improve the performance of the project. you select the nvidia as the GPU Type then

• Click the Save option.

Saving the Project details

• The newly created project gets saved, and it appears on the screen.

• The success of project creation is informed by a notification message.

Project gets created and listed under the Projects page

Please Note:

• The user can also open the Project list by clicking the View Project option.

• Click the View Project option.

View Project option

• The user gets redirected to the Project list.

Project List

Please Note: A project gets the Share, Edit, Delete, Activate/Deactivate actions to be applied on it after getting listed under the Project list.

Various Actions provided to a DSL Project

Pre-requisite: The DSL projects also get Push to VCS and Pull from VCS functionalities, but they only get enabled for the activated DSL projects.

Activate your Project

Look at the given walk-through to understand how a DSL Project gets activated.

Activating a DSL Project

• Navigate to the Projects page.

• Select a project from the list.

• Click the Activate option.

Use the Activate option for a Project

• A dialog window appears to confirm the Activation.

• Click the Yes option.

• The project gets activated and a notification message appears to communicate the completion of the action.

• The Activation option gets changed into the Deactivation option for the concerned project.

The Project gets activated

Create your first Notebook

Have a look at the walk-through to create a new Notebook.

Creating a new Notebook

• Click the Create Notebook option from the Notebook tab.

The Create Notebook option.

• A new Notebook gets created; the user gets a notification message informing the same.

• Click the Back icon.

The Notebook landing page

• The Notebook gets saved under the Notebook list.

The List of created Notebooks

Note:

1. Edit the Notebook name by using the ‘Edit Notebook Name’ icon.

2. The accessible datasets, models, and artifacts will list down under the Datasets, Models, and Artifacts menus (In this case, there are no datasets added to the concerned Notebook.).

3. Find/Replace menu facilitates the user to find and replace a specific text in the notebook code.

4. Add a description for the created Notebook by using the same page.

Notebook Landing page

Upload your Notebook to DS Lab

Check out the given walk-through on how to upload a Notebook.

Uploading a Notebook to DS Lab Project

The users can seamlessly upload Notebooks created using other tools and saved in their systems.

• Navigate to the landing page of an activated Project.

• Click the Upload Notebook option.

Upload Notebook option

• Specify a Notebook from the system.

• Click the Open option to upload the Notebook.

• The selected Notebook gets uploaded under the Project.

• The same gets confirmed by a notification message.

• Another notification message appears to inform the status of the Notebook (it gets saved by default).

• Click the Back icon.

Notification messages after a Notebook gets uploaded

• The uploaded Notebook gets listed on the landing page of the Project.

The uploaded Notebook gets listed under the Notebook list

Create your first DSL Model

Check-out the walk-through on how to save and load a DSL Model.

Saving and Loading a DSL Model

Once the Notebook script is executed successfully, the users can save them as a model. The saved model can be loaded into the Notebook.

Save your DSL Model

• Navigate to a Notebook.

• Write code using the following sequence:

  • Read DataFrame

  • Define test and train data

  • Create a model

• Execute the script.

Sample Script for a Data Science Model

• Get a new cell.

• Give a model name to specify the model.

• Execute the cell.

• After the code gets executed, click the Save Model notebook in a new cell.

• The saved model gets listed under the Models list.

Specify a DSL Model by giving a name

Load your DSL Model

• Click on a new cell and select the model by using the given checkbox to load it.

• The model gets loaded into a new cell.

Loading a saved DSL Mod

Predict the model output

Check out the walk-through on the Predict option for a DSL Notebook.

Predicting the output of a loaded model

You can get the predicted array from a loaded DSL model that contains a definite dataframe.

• Add a new cell.

• Click the Predict option.

• Execute the code.

• Provide the model and dataframe.

• The predicted output of the given dataframe appears as an array.

• The default comments on how to define the predicted output for a DS Lab model appears as well.

Predicting the Output of a DSL model

Save Artifacts of your model

Check out the walk-through on how to Save Artifacts.

Saving Artifacts in a Notebook

Please Note: The user can save the artifacts of a predicted model using this option.

• Add a new cell.

• Click the Save Artifacts.

• Give proper dataframe name and Name of Artifacts (with extensions - .csv/.txt/.json).

• Execute the cell.

• The Artifacts get saved.

Saved Artifacts inside a Notebook

Please Note: The saved Artifacts can be downloaded as well.

Deploy your DS Model to Pipeline

Check out the walk-through to deploy a DSL model to the Data Pipeline (Publish the Model from the Model tab).

Deploying a DSL Model to Data Pipeline/ Publishing a DSL Model

You can deploy a saved DSL model to the Data Pipeline plugin by using the Model tab.

• Navigate to the Model tab.

• Select a model from the list.

• Click the Deploy to Pipeline icon for the model.

Accessing the Deploy to Pipeline option for a saved DS model

• The Deploy to Pipeline dialog box appears to confirm the action.

• Click the Yes option.

• The selected model gets published and deployed to the Data Pipeline (It disappears from the Unpublished model list).

• A notification message appears to inform the same.

Notification message after the model gets deployed to the Data Pipeline plugin.

Please Note:

1. The published/deployed model gets listed under the Published filter.

2. The Publish option provided under the Notebook tab and the Deploy to Pipeline option provided under the Model tab perform the same task.

Filtering the model list by using the 'Published' filter option.

Publish your DS model as an API

This function gets completed in three steps:

1. Publish a Model as an API

2. Register an API Client

3. Pass the Model values in the Postman

Check-out the below given video to understand the Publish Model as an API Service functionality.

Publish a Model as an API

You can publish a DSL model as an API using the Model tab. Only the published models get this option.

• Navigate to the Model tab.

• Filter the model list by using the Published filter option.

• Select a model from the list.

• Click the Publish as API option.

Accessing the Publish as API option for a saved model

• The Update model page opens.

• Provide Max instance limit.

• Click the Save and Publish option.

Updating the model details

Please Note: Use the Save option to save the data which can be published later.

• The model gets saved and published as an API service. A notification message appears to inform the same.

Notification message after the Model gets saved and published as an API

Register an API Client

• Navigate to the Admin module.

• Click the API Client Registration option.

• The API Client Registration page opens.

• Click the New option.

• Select the Client type as internal.

• Provide the following client specific information:

  • Client Name

  • Client Email

  • App Name

  • Request Per Hour

  • Request Per Day

  • Select API Type- Select the Model as API option.

  • Select the Services Entitled -Select the published DSL model from the drop-down menu.

• Click the Save option.

Required information for an API Client Registration

• The client details get registered.

• A notification message appears to inform the same.

The API Client gets registered

Note: Once the client gets registered open the registered client details using the Edit option to get the Client id and Client Secrete key.

Get Client Id and Client Secret Key for a registered API Client

Pass the Model values in the Postman

• Navigate to the Postman.

• Add a new POST request.

• Pass the URL with model name (Only the sklearn models are supported at present).

Passing the Model in the POST request under the Postman

• Provide required parameters under the Params tab:

  a. Client Id

  b. Client Secret Key

  c. App Name

Providing the required Parameters

• Open the Body tab.

• Select the raw option.

• Provide the input dataframe.

• Click the Send option.

• The response will appear below.

• You can save the response by using the Save Response option.

Getting Response based on the Model input

Please Note: The model published as an API service can be easily consumed under various apps.

Add data sets to your Project

The Dataset tab offers a list of uploaded Datasets which can be added to a project. The user can get a list of uploaded Data Sets and Data Sandbox from the Data Center module under this tab.

The Add Datasets page offers the following Data service options to add as datasets:

1. Data Service – These are the uploaded data sets from the Data Center module.

2. Data Sandbox – This option lists all the available Data Sandbox from the Data Center module.

Add Datasets page

Adding Data Service

Check out the walk-through to on how to add a data service as a dataset.

Adding a Data service as a Dataset in the DS Lab

• Navigate to a Project-specific page and click the Dataset tab (E.g., the given image displays the Dataset tab under the Sample Project).

• Click the Add Datasets button.

Adding Datasets to a Project

• The Add Datasets page opens offering two options to choose data:

  • Data service (gets selected by default)

  • Data Sandbox

• Use the Search space to search through the displayed data service list.

• Select the required data service(s) using the checkboxes provided next to it.

• Click the Add option.

Selecting Data Sets

• The selected data service(s) gets added to the concerned project.

• A notification message appears to inform the same.

Dataset li

Adding Data Sandbox

Check-out the walk-through to understand how to add dataset using the Data Sandbox.

Adding Dataset using the Data Sandbox option

• Open the Dataset tab from a specific project.

• Click the Add Datasets option.

Adding Dataset to a project

• You get redirected to the Add Datasets page.

• Select the Data Sandbox option from the Data Service drop-down menu.

• Use the Search space to search for a specific Data Sandbox.

• Select the required Data Sandbox(es) using the checkboxes provided next to it.

• Click the Add option.

Selecting Data Sandbox

• The selected data sandbox(es) gets added to the concerned project.

• A notification message appears to inform the same.

Dataset list

VCS for Projects

Pre-requisite: Make sure that the Version control settings for the DSL plugin are configured by your administrator before you use this functionality.

Watch the walk-through videos given below to understand the Push to VCS and Pull from VCS functionality for a DSL Project.

Pushing a DSL Project to the VCS

Pulling a DSL Project from the VCS

Pushing a Project to the VCS

• Navigate to the Projects page of the DS Lab plugin.

• Select an activated project.

• Click the Push into VCS icon for the project.

Push to VCS option for a DSL Project

• The Push into Version Controlling System dialog box appears.

• Provide a Commit Message.

• Click the Push option.

Push into Version Controlling System dialog box

• The DSL Project version gets pushed into the Version Controlling System, a notification message appears to inform the same.

Notification message after a DSL Project gets pushed into VCS

Pulling a Project from the VCS

• Navigate to the Projects page of the DS Lab plugin.

• Select an activated project.

• Click the Pull from VCS icon for the project.

Pull from VCS icon for a DSL Project

• The Pull from Version Controlling System dialog box opens.

• Select the version that you wish to pull by using the checkbox.

• Click the Pull option.

Pull from Version Controlling System dialog box

• The pulled version of the selected Project gets updated in the Project list.

• A notification message appears to inform the same.

Notification message after a version of the DSL Project gets pulled

The Push to and Pull from VCS functionalities will not be enabled for a deactivated project.

VCS for Notebooks

Pre-requisite: Make sure that the Version control settings for the DSL plugin are configured by your administrator before you use this functionality.

Walk-through on the Version Control Functionality for Notebook

Pushing to and Pulling from VCS functionality for a Notebook

Pushing a Notebook to the VCS

• Navigate to the Notebook list of a Project.

• Select a Notebook.

• Click the Push into VCS icon for the Notebook.​

Push to VCS icon for Notebook

• The Push into Version Controlling System dialog box appears.

• Provide a Commit Message.

• Click the Push option.​

Push into Version Controlling System dialog box

• The Notebook version gets pushed into the Version Controlling System and the Notebook list gets updated with the latest version.

• A notification message appears to inform the success of the action.​

Notification message after the Notebook version gets updated in the list

Pulling a Notebook from the VCS

• Navigate to the Notebook list given under a Project.

• Select a Notebook.

• Click the Pull from VCS icon for the Notebook.

Pull from VCS icon for a Notebook

• The Pull from Version Controlling System dialog box opens.

• Select the version that you wish to pull by using the checkbox.

• Click the Pull option.

Pull from Version Controlling System dialog box for Notebook

• The pulled version of the selected Notebook gets updated in the Notebook list.

• A notification message appears to inform the success of the action.

Notification message after a version of the Notebook gets pulled