DB Reader

The DB reader is a spark-based reader which gives you capability to read data from multiple database sources.

All component configurations are classified broadly into the following sections:

• ​Basic ​

• Metadata

• ​Resource Configuration​

• ​Connection Validation​

Please follow the steps given in the demonstration to configure the RDBMS Reader component.

DB Reader Configuration in a Pipeline Workflow

Drivers Available:

• MySQL

• Oracle

• PostgreSQL

• MS-SQL

• ClickHouse

• Snowflake

Please Note:

• The ClickHouse driver in the Spark components will use HTTP Port and not the TCP port.

• In the case of data from multiple tables (join queries), one can write the join query directly without specifying multiple tables, as only one among table and query fields is required.

• Fetch Size: Provide the maximum number of records to be processed in one execution cycle.

• Create Partition: This is used for performance enhancement. It's going to create the sequence of indexing.

  • Auto Increment: The number of partitions will be incremented automatically.

  • Index: The number of partitions will be incremented based on the specified Partition column.

• Partition Column: specify a unique column name, whose value is a number.​

Check out the given demonstration of the DB Reader component with the Snowflake driver.