MongoDB Reader Lite (PyMongo Reader)

The MongoDB Reader Lite component reads data from a specified MongoDB database collection. It supports direct filtering using Mongo Query Language (MQL), allowing queries to run directly on the MongoDB server before data is pushed to the output event.

Configuration Sections

The MongoDB Reader Lite component configurations are organized into the following sections:

  • Basic Information

  • Meta Information

  • Resource Configuration

  • Connection Validation

Accessing the Component Properties

  1. Drag and drop the MongoDB Reader Lite component into the Workflow Editor.

  2. Click the component to open its configuration panel.

  3. Configure properties across the Basic Information, Meta Information, and other tabs.

Basic Information Tab

Parameter
Description
Example
Required

Invocation Type

Defines how the reader runs. Options: Real-Time or Batch.

Batch

Yes

Deployment Type

Displays the deployment type (pre-selected).

Kubernetes

No

Container Image Version

Displays the container image version (pre-selected).

v1.2.3

No

Failover Event

Select a failover event from the drop-down.

retry_event

Optional

Batch Size

Maximum number of records processed per cycle. Minimum value: 10.

1000

Yes

Grace Period

(Batch only) Time (in seconds) for components to shut down gracefully.

30

Optional

Meta Information Tab

The Meta Information tab defines MongoDB connection details. The available fields vary depending on the Connection Type selected.

General Parameters

Parameter
Description
Example
Required

Connection Type

Options: Standard, SRV, or Connection String.

Standard

Yes

Host IP Address

MongoDB host IP address (Standard only).

192.168.1.100

Yes

Port Number

MongoDB port (Standard only).

27017

Yes

Username

MongoDB username.

mongo_user

Yes

Password

Password for the MongoDB user.

********

Yes

Database Name

Target database name.

analytics_db

Yes

Collection Name

Target collection name.

customer_orders

Yes

Fetch Size

Number of documents returned in each batch.

1000

Optional

Additional Parameters

Extra MongoDB connection options (key-value pairs).

replicaSet=rs0&authSource=admin

Optional

Enable SSL

Enable SSL for secure MongoDB connections. Requires certificates.

true

Optional

Certificate Folder

Folder containing uploaded .pem and .key files for SSL connections.

mongo_certs

Conditional

Connection String

Full MongoDB URI (Connection String type only).

mongodb+srv://cluster0…

Yes (if type=Connection String)

Query

Mongo Query Language (MQL) query. Supports find() and aggregate().

See examples below

Optional

Example Queries

Find Documents with Conditions

db.customer_orders.find({ "$or": [ { "AmountSpent": "10255" }, { "Age": "Old" } ] })

Aggregation with Match

db.sales.aggregate([{ "$match": { "Goal1Adjective": "High" } }])

SSL Configuration

When Enable SSL is checked:

  • Upload required certificates via the Admin Settings → Certificate Upload page:

    • Certificate file (.pem)

    • Key file (.key)

  • Select the Certificate Folder in the component configuration.

Saving and Activating the Component

  1. After configuration, click the Save Component in Storage icon.

    • A confirmation message appears.

  2. Click Update Pipeline to save pipeline changes.

    • A confirmation message appears.

  3. Click Activate Pipeline.

    • Confirm in the dialog by clicking YES.

    • A confirmation message appears when the pipeline is active.

Logs and Monitoring

  • Click the Toggle Log Panel icon to view logs.

  • The Log Panel provides:

    • Logs Tab: General component logs.

    • Advanced Logs Tab: Pod-level logs from Kubernetes.

Tabs for Configured Components

A configured MongoDB Reader Lite component displays additional tabs:

  • Configuration Tab: Current settings.

  • Logs Tab: Execution logs.

  • Pod Logs Tab: Pod-level system logs.

  • Summary Tab: Description of the component and its configuration.

Example Configuration: Standard Connection

Connection Type: Standard
Host IP Address: 192.168.1.100
Port Number: 27017
Username: mongo_user
Password: ********
Database Name: analytics_db
Collection Name: customer_orders
Fetch Size: 1000
Enable SSL: false
Query: db.customer_orders.find({ "status": "active" })