SQL Component

The SQL Component also provides an option of using aggregation functions on the complete streaming data processed by the component. The user can use SQL transformations on Spark data frames with the help of this component.

All component configurations are classified broadly into the following sections:

• ​Basic ​

• Metadata

• ​Resource Configuration​

Follow the given steps in the demonstration to configure the SQL transformation component.

Configuring the SQL Transformation component

Please Note: The schema file that can be uploaded here is a JSON spark schema.

Follow the below mentioned steps to configure SQL Component.

1. Query Type: There are two options available under this field:

   • Batch Query: When this option is selected, then there is no need to upload schema file.

   • Aggregate Query: When this option is selected, it is mandatory to upload the spark schema file in JSON format of the in-event data.

2. Schema File name: Upload the spark schema file in JSON format when Aggregate query is selected in the query type field.

3. Table name: Provide the table name

4. Query: Write a SQL query in this field.

5. Selected Columns: select the column name from the table, provide the Alias name and the desired data type for that column.