Azure Writer

Azure Writer component is designed to write or store data into Microsoft Azure's storage services, such as Azure Blob Storage. Azure Writers typically authenticate with Azure using Azure Active Directory credentials or other authentication mechanisms supported by Azure.

All component configurations are classified broadly into the following sections:

• ​​Basic Information​​

• Meta Information

• Resource Configuration​​

• ​Connection Validation​

Follow the steps given in the demonstration to configure the component.

Configuring the Azure Writer component​

Configuring Meta information of Azure writer:

• Write using: There are three(3) options available under this tab:

  • Shared Access

  • Signature Secret Key

  • Principal Secret

Write using Shared Access Signature:

Shared Access Signature

• Shared Access Signature: This is a URI that grants restricted access rights to Azure Storage resources.

• Account Name: Provide the Azure account name.

• Container: Provide the container name from where the blob is located. A container is a logical unit of storage in Azure Blob Storage that can hold blobs. It is similar to a directory or folder in a file system, and it can be used to organize and manage blobs.

• Blob Name: Enter the Blob name. A blob is a type of object storage that is used to store unstructured data, such as text or binary data, like images or videos.

• File Format: There are four(4) types of file extensions are available under it, select the file format in which the data has to be written:

  • CSV

  • JSON

  • PARQUET

  • AVRO

• Save Mode: Select the Save mode from the drop down.

  • Append

  • Overwrite

• Schema File Name: Upload spark schema file in JSON format.

Write using Secret Key Option:

Write using Secret Key

• Client ID: Provide Azure Client ID. The client ID is the unique Application (client) ID assigned to your app by Azure AD when the app was registered.

• Tenant ID: Provide the Azure Tenant ID. Tenant ID (also known as Directory ID) is a unique identifier that is assigned to an Azure AD tenant, which represents an organization or a developer account. It is used to identify the organization or developer account that the application is associated with.

• Client Secret: Enter the Azure Client Secret. Client Secret (also known as Application Secret or App Secret) is a secure password or key that is used to authenticate an application to Azure AD.

• Account Name: Provide the Azure account name.

• Container: Provide the container name from where the blob is located. A container is a logical unit of storage in Azure Blob Storage that can hold blobs. It is similar to a directory or folder in a file system, and it can be used to organize and manage blobs.

• Blob Name: Enter the Blob name. A blob is a type of object storage that is used to store unstructured data, such as text or binary data, like images or videos.

• File Format: There are four(4) types of file extensions are available under it:

  • CSV

  • JSON

  • PARQUET

  • AVRO

• Schema File Name: Upload spark schema file in JSON format.

• Save Mode: Select the Save mode from the drop down.

  • Append

  • Overwrite

Write using Principal Secret:

Write using Principal Secret

• Account Key: Enter the azure account key. In Azure, an account key is a security credential that is used to authenticate access to storage resources, such as blobs, files, queues, or tables, in an Azure storage account.

• Account Name: Provide the Azure account name.

• Container: Provide the container name from where the blob is located. A container is a logical unit of storage in Azure Blob Storage that can hold blobs. It is similar to a directory or folder in a file system, and it can be used to organize and manage blobs.

• Blob Name: Enter the Blob name. A blob is a type of object storage that is used to store unstructured data, such as text or binary data, like images or videos.

• File Format: There are four(4) types of file extensions are available under it:

  • CSV

  • JSON

  • PARQUET

  • AVRO

• Schema File Name: Upload spark schema file in JSON format.

• Save Mode: Select the Save mode from the drop down.

  • Append

  • Overwrite

• Column Filter: There is also a section for the selected columns in the Meta Information tab if the user can select some specific columns from the table to read data instead of selecting a complete table so this can be achieved by using the Column Filter section. Select the columns which you want to read and if you want to change the name of the column, then put that name in the alias name section otherwise keep the alias name the same as of column name and then select a Column Type from the drop-down menu.

  • Use Download Data and Upload File options to select the desired columns.

    • Upload File: The user can upload the existing system files (CSV, JSON) using the Upload File icon (file size must be less than 2 MB).

    • Download Data: Users can download the schema structure in JSON format by using the Download Data icon.