Enrichment Component
Enrich your data from the master table/collection with few simple steps.
Last updated
Enrich your data from the master table/collection with few simple steps.
Last updated
This component helps users to enrich the incoming data from an in-Event by querying lookup table in the RDBMS and MongoDB reader components.
All component configurations are classified broadly into the following sections:
Metadata
Please follow the demonstration to configure the Enrichment component.
Please Note: The supported drivers are as mentioned below:
MySql
MS-SQL
MongoDB
PostgreSQL
Oracle
ClickHouse
Drag and drop the Enrichment Component to the Workflow Editor.
Create two Events and drag them to the Workspace.
Connect the input event and the output event (The data in the input event can come from any Ingestion, Reader, or shared events).
Click the Enrichment Component to get the component properties tabs.
The Basic Information tab opens by default.
Select an Invocation type from the drop-down menu to confirm the running mode of the reader component. Select either ‘Real-Time’ or ‘Batch’ from the drop-down menu.
Deployment Type: It displays the deployment type for the component. This field comes pre-selected.
Container Image Version: It displays the image version for the docker container. This field comes pre-selected.
Failover Event: Select a failover Event from the drop-down menu.
Batch Size: Provide the maximum number of records that you want to be processed in one execution cycle.
Open the ‘Meta Information’ tab and fill in all the connection-specific details for the Enrichment Component.
Driver (*): Select Database type (MYSQL, MSSQL, Oracle, Postgres, MongoDB)
Port (*): Host server port number
Host IP Address (*): IP Address
Username (*): Username for Authentication
Password (*): Password for Authentication
Database Name (*): Provide the Database name
Table Name: Provide the table name to read the data
Query: Insert an SQL query
Limit: Set a limit for the number of records
Connection Type: This option will show if the user selects MongoDB from the Driver field. A User can configure the MongoDB driver via two connection types (Standard or SRV) that are explained below:
Standard - Port field does appear with the Standard connection type option.
SRV - Port field does not appear with the SRV connection type option.
Conditions: Select conditions type (Remove or Blank option).
The users can select some specific columns from the table to read data instead of selecting a complete table; this can be achieved via the ‘Selected Columns’ section. Select the columns which you want to read and if you want to change the name of the column, then put that name in the alias name section otherwise keep the alias name the same as of column name and then select a Column Type from the drop-down menu.
or
Use the ‘Download Data’ and ‘Upload File’ options to select the desired columns.
Upload File: The user can upload the existing system files (CSV, JSON) using the ‘Upload File’ icon (file size must be less than 2 MB).
Download Data (Schema): Users can download the schema structure in JSON format by using the ‘Download Data’ icon.
After doing all the configurations click the ‘Save Component in Storage’ icon provided in the reader configuration panel to save the component.
A notification message appears to inform about the component configuration success.
Query | Valid/Invalid |
SELECT department_name,location_id FROM departments WHERE department_id = @data.department_id | Valid |
Asterisk (*) is not valid in the select statement SELECT * FROM departments WHERE department_id = @data.department_id | Invalid |
Please Note:
Remove [works like an inner join]– This option completely removes the row If any row of incoming data is not matching with query condition of the lookup table. Only passes the matching records.
Blank – This option sets blank for the cell of a row if complete incoming data from the in-event and matching rows from the lookup table.
"@data." is used to refer to the data coming from the previous event. For e.g.: If the data coming from the previous event is department_id and the column in the master table is dept_id then in the where clause we write as where dept_id=@data.department_id.
