Azure Blob Reader

Azure Blob Reader is designed to read and access data stored in Azure Blob Storage. Azure Blob Readers typically authenticate with Azure Blob Storage using Azure Active Directory credentials or other authentication mechanisms supported by Azure.

This is a spark based component.

All component configurations are classified broadly into the following sections:

• ​​Basic Information​​

• Meta Information

• ​Resource Configuration​​

• ​Connection Validation

Please follow the demonstration to configure the component.

Configuring the Meta Information Tab fields

• Read using: There are three(3) options available under this tab:

  1. Shared Access Signature:

  2. Secret Key

  3. Principal Secret

Read using Shared Access Signature

Provide the following details:

1. Shared Access Signature: This is a URI that grants restricted access rights to Azure Storage resources.

2. Account Name: Provide the Azure account name.

3. Container: Provide the container name from where the file is located and which has to be read.

4. File type: There are four(4) types of file extensions are available under it:

   • CSV

   • JSON

   • PARQUET

   • AVRO

   • XML

5. Read Directory: Check in this box to read the specified directory.

6. Selected Column: Select the columns from the file.

Read using Secret Key Option

Provide the following details:

1. Account Key: It is used to authorize access to data in your storage account via Shared Key authorization

2. Account Name: Provide the Azure account name.

3. Container: Provide the container name from where the file is located and which has to be read.

4. File type: There are four(4) types of file extensions are available under it:

   • CSV

   • JSON

   • PARQUET

   • AVRO

   • XML

5. Read Directory: Check in this box to read the specified directory.

6. Selected Column: Select the columns from the file.

7. Header: Check in this box to get the Header row of the file.

8. Infer Schema: Check in this box to get the true data type of all the column of the file.

Read using Principal Secret

Provide the following details:

1. Client ID: The client ID is the unique Application (client) ID assigned to your app by Azure AD when the app was registered.

2. Tenant ID: It is a globally unique identifier (GUID) that is different than your organization name or domain.

3. Client Secret: The client secret is the password of the service principal.

4. Account Name: Provide the Azure account name.

5. File type: There are four(4) types of file extensions are available under it:

   • CSV

   • JSON

   • PARQUET

   • AVRO

   • XML

6. Read Directory: Check in this box to read the specified directory.

7. Selected Column: Select the columns from the file.

8. Header: Check in this box to get the Header row of the file.

9. Infer Schema: Check in this box to get the true data type of all the column of the file.

Note: The following fields will be displayed after selecting the following file types:

• CSV: The Header and Infer Schema fields get displayed with CSV as the selected File Type. Enable Header option to get the Header of the reading file and enable Infer Schema option to get true schema of the column in the CSV file.

• JSON: The Multiline and Charset fields get displayed with JSON as the selected File Type. Check-in the Multiline option if there is any multiline string in the file.

• PARQUET: No extra field gets displayed with PARQUET as the selected File Type.

• AVRO: This File Type provides two drop-down menus.

  • Compression: Select an option out of the Deflate and Snappy options.

  • Compression Level: This field appears for the Deflate compression option. It provides 0 to 9 levels via a drop-down menu.

• XML: Select this option to read XML file. If this option is selected, the following fields will get displayed:

  • Infer schema: Enable this option to get true schema of the column.

  • Path: Provide the path of the file.

  • Root Tag: Provide the root tag from the XML files.

  • Row Tags: Provide the row tags from the XML files.

  • Join Row Tags: Enable this option to join multiple row tags.