SFTP Monitor

The SFTP (Secure File Transfer Protocol) Monitor is used to monitor and manage file transfers over SFTP servers. It continuously watches a designated folder on the server, detects new files, and manages their movement to a defined location.

The component is often used to:

• Monitor inbound file deliveries from external systems.

• Copy and organize files for downstream ingestion.

• Provide metadata (file name, size, timestamps) to pipeline events for further processing.

Configuration Sections

All component configurations are classified into the following sections:

• Basic Information

• Meta Information

• Resource Configuration

• Connection Validation

Using the SFTP Monitor in a Workflow

1. Drag and drop the SFTP Monitor component from the Consumer section of system components into the Workflow Editor.

2. Open the component properties to access configuration tabs.

3. Configure the Basic Information and Meta Information fields.

4. Save the component to activate monitoring.

Basic Information Tab

The Basic Information tab is the default configuration screen.

Field	Description	Required
Invocation Type	Select execution mode. Currently, only Real-Time is supported.	Yes
Deployment Type	Displays the deployment type of the component (pre-selected).	Yes
Container Image Version	Displays the Docker image version used (pre-selected).	Yes
Failover Event	Select a failover event to handle retries or errors.	Optional
Batch Size	Maximum records processed in one execution cycle (minimum: 10).	Yes

Meta Information Tab

The Meta Information tab defines server connection details, authentication, and monitored folder paths.

Field	Description	Required
Host	SFTP server hostname or IP address.	Yes
Username	Username for authentication.	Conditional
Port	SFTP server port number (default: 22).	Yes
Authentication	Select authentication type: Password or PEM/PPK File.	Yes
Password	Password for authentication (used if Password method is selected).	Conditional
PEM/PPK File	Upload a private key file (used if PEM/PPK method is selected).	Conditional
Directory Path	Path of the monitored folder (e.g., /home/monitor).	Yes
Copy Directory Path	Destination path where monitored files are copied (e.g., /home/monitor_copy).	Yes
Channel	Select a channel option. Supported value: SFTP.	Yes

Note:

• Do not use nested directory structures in Directory Path or Copy Directory Path. Example of unsupported paths:

  • home/monitor/data

  • home/monitor/data/copy_data

• Only one SFTP Monitor can read and move a file. If multiple monitors watch the same directory, only one will act on the file.

Saving the Configuration

1. Click Save Component (Storage icon).

2. A notification confirms that the component has been saved successfully.

Behavior Notes

• The SFTP Monitor requires an output event to send file details.

• The component copies each monitored file into the defined copy path for further reading.

• Metadata sent to the output event includes:

  • File name

  • File size

  • Last modified time

  • Ingestion time

Example Workflow

1. Configure an SFTP Monitor to watch /home/monitor for incoming CSV files.

2. Set the Copy Directory Path to /home/monitor_copy.

3. Output metadata is sent to an event, which triggers an SFTP Reader to process the copied files.

4. Downstream transformations process the ingested CSV data into the pipeline.