Enrichment Component
Enrich your data from the master table/collection with few simple steps.
All component configurations are classified broadly into 3 section
Metadata
Please follow the demonstration to configure the component.
Currently, we support 5 drivers:
MySql
Ms-SQL
MongoDB
Postgres
Oracle
This component helps users to enrich the incoming data from in-event by querying lookup table in RDBMS and MongoDB.
Steps to configure the component:
i) Drag and Drop the Enrichment Component to the Workflow Editor.
iii) Create two Events and drag them to the Workspace.
iv) Connect the input event and the output event (The data in the input event can come from any Ingestion, Reader, or shared events).
v) Click the Enrichment Component to get the component properties tabs.
vi) The Basic Information tab opens by default.
a. Select an Invocation type from the drop-down menu to confirm the running mode of the
reader component. Select ‘Real-Time’ or ‘Batch’ from the drop-down menu.
b. Deployment Type: It displays the deployment type for the component. This field comes pre-selected.
c. Container Image Version: It displays the image version for the docker container. This field comes pre-selected.
d. Failover Event: Select a failover Event from the drop-down menu.
e. Batch Size: Provide the maximum number of records that you want to be processed in one execution cycle.
vii) Open the ‘Meta Information’ tab and fill in all the connection-specific details for the Enrichment Component.
1. Driver (*): Select Database type (MYSQL, MSSQL, Oracle, Postgres, MongoDB)
2. Port (*): Host server port number
3. Host IP Address (*): IP Address
4. Username (*): Username for Authentication.
5. Password (*): Password for Authentication.
6. Database Name (*): Provide the Database name.
7. Table Name: Provide the table name to read the data.
8. Query: Insert an SQL query.
9. Limit: Set a limit for the number of records.
10. Connection Type: This option will show if the user selects MongoDB from the Driver field. A User can configure the MongoDB driver via two connection types (Standard or SRV) that are explained below:
Standard - Port field does appear with the Standard connection type option.
SRV - Port field does not appear with the SRV connection type option.
11. Conditions: Select conditions type (Remove or Blank option).
viii) Selected Columns: The users can select some specific columns from the table to read data instead of selecting a complete table; this can be achieved via the ‘Selected Columns’ section. Select the columns which you want to read and if you want to change the name of the column, then put that name in the alias name section otherwise keep the alias name the same as of column name and then select a Column Type from the drop-down menu.
or
Use ‘Download Data’ and ‘Upload File’ options to select the desired columns.
1. Upload File: The user can upload the existing system files (CSV, JSON) using the ‘Upload File’ icon (file size must be less than 2 MB).
2. Download Data (Schema): Users can download the schema structure in JSON format by using the ‘Download Data’ icon.
ix) After doing all the configurations click the ‘Save Component in Storage’ icon provided in the reader configuration panel to save the component.
x) A notification message appears to inform about the component configuration success.
Query Example:
Query | Valid/Invalid |
SELECT department_name,location_id FROM departments WHERE department_id = @data.department_id | Valid |
Asterisk (*) is not valid in the select statement SELECT * FROM departments WHERE department_id = @data.department_id | Invalid |
Remove [works like an inner join]– This option completely removes the row If any row of incoming data is not matching with query condition of the lookup table. Only passes the matching records.
Blank – This option sets blank for the cell of a row if complete incoming data from the in-event and matching rows from the lookup table.
"@data." is used to refer to the data coming from the previous event. For e.g.: If the data coming from the previous event is department_id and the column in the master table is dept_id then in the where clause we write as where dept_id=@data.department_id.
Last updated
