DB Reader

The DB reader is a spark-based reader which gives you capability to read data from multiple database sources.

All component configurations are classified broadly into the following sections:

• Basic

• Metadata

• Resource Configuration

• Connection Validation

Please follow the demonstration to configure the component.

RDBMS Writer

Drivers Available:

• MySQL

• Oracle

• PostgreSQL

• MS-SQL

• ClickHouse

Note: The ClickHouse driver in the Spark components will use HTTP Port and not the TCP port.

In the case of data from multiple tables (join queries), one can write the join query directly without specifying multiple tables, as only one among table and query fields is required.

Fetch Size: Provide the maximum number of records to be processed in one execution cycle.

Create Partition: This is used for performance enhancement. It's going to create the sequence of indexing.

1. Auto Increment: The number of partitions will be incremented automatically.

2. Index: The number of partitions will be incremented based on the specified Partition column.

Partition Column: specify a unique column name, whose value is a number .