SFTP Reader

The SFTP Reader component is designed to read and access files stored on an SFTP server. It supports authentication through username and password or SSH key pair (PEM/PPK), providing secure access to files for downstream processing.

Configuration Sections

The SFTP Reader component configurations are organized into the following sections:

• Basic Information

• Meta Information

• Resource Configuration

• Connection Validation

Meta Information Configuration

Parameter	Description	Example	Required
Host	Hostname or IP address of the SFTP server.	sftp.example.com	Yes
Port	Port number of the SFTP server.	22	Yes
Username	Username for authentication.	datauser	Yes
Authentication	Authentication type. Options: Password or PEM/PPK File.	Password	Yes
Password	Password for authentication (if Password is selected).	********	Conditional
PEM/PPK File	SSH key file for authentication (if PEM/PPK is selected). Must be uploaded via UI.	id_rsa.pem	Conditional
Reader Path	Path of the file or directory on the SFTP server.	/data/sales/transactions.csv	Yes
Channel	Streaming channel type. Only SFTP is supported.	SFTP	Yes
Dynamic Header	Automatically detects the header row in a file and adjusts column names and count.	true	Optional
Column Filter	Select specific columns to read. Optionally provide alias names and column types.	id AS customer_id, name STRING	Optional

File Handling Options

• Upload File: Upload local system files (CSV, JSON) to quickly test or configure schema.

• Download Data (Schema): Download the inferred schema structure in JSON format for review or reuse.

Example Configurations

Example 1: Reading a CSV File with Dynamic Header

Host: sftp.example.com
Port: 22
Username: datauser
Authentication: Password
Password: ********
Reader Path: /data/invoices/invoice_data.csv
Channel: SFTP
Dynamic Header: true

This configuration retrieves CSV data and automatically adjusts column headers based on the file’s first row.

Example 2: Using SSH Key Authentication

Host: sftp.example.com
Port: 22
Username: datauser
Authentication: PEM/PPK File
PEM/PPK File: id_rsa.ppk
Reader Path: /data/hr/employees.json
Channel: SFTP
Column Filter: id AS employee_id, name STRING, hire_date DATE

This configuration authenticates using a private key file (id_rsa.ppk) and reads specific columns with aliases.

Notes

• Use Dynamic Header when working with files that may have variable header rows.

• Column Filter allows selective data reading and schema enforcement to improve performance.

• Authentication via PEM/PPK File is recommended for production setups as it is more secure than passwords.