Documentation
  • Core Platform
    • About Data Platform
      • Why Data Platform
      • Design Philosophy
      • Plugin Architecture
    • Getting Started
      • Sign in
        • Resetting Password
        • Force Login
      • Homepage
        • Data Catalog Search
        • AI Search
        • Apps Menu
        • Notification
        • Help Menu
        • User Profile
        • Search Option
        • Documents Folders
          • Document Options
            • Accessing Document Options
            • Options Assigned to a Folder
            • Options Assigned to a Linked URL
            • Options Assigned to a Story
            • Options for a Published Dashboard
        • Sorting Documents
        • Filter
      • Signing Out
    • Platform Administration
      • Accessing the Admin Module
      • Admin Panel Options
        • Document Management
        • Configurations
          • Geo Spatial
          • Data Science Servers
          • Bulk User Creation
          • Custom Field Settings
          • Data Connectors
          • API Connectors Configurations
          • Encryption
          • Form Settings
          • Data Sheet Settings
          • Data Lake Settings
          • Data Catalog Settings
          • Open ID Settings
          • Version Control
          • New Version Control
            • Versioning
            • Migration
          • Pipeline Settings
          • Keycloak Settings
          • Email Server
          • Password
          • Sandbox Settings
          • Secret Management
          • DS Lab Settings
          • Data Store Settings
        • Authentication
          • AD Configuration
          • CA PPM Configuration
          • AWS Cognito Configuration
        • Audit Trail
          • Log Status
          • Audit Log Table
        • Language Mapping
          • Languages
          • Mapping Table
        • Migration
          • SFTP Settings
          • Document Migration
          • DSW Migration
        • GIT Migration
          • Migrating a Dashboard
          • Migrating an API Service
          • Migrating a Pipeline
        • Session Manager
        • Schedule Monitor
        • Server Monitor
        • License
        • API Client Registration
    • Users & Roles
      • User Security Page
      • Creating a new User
        • Restrict Data Access
      • Creating a new User Group
        • Assigning Custom Fields to Group Users
      • Various User Roles
        • Viewer Role
        • Admin Role
        • Non-admin User Roles
      • User Status
  • Data Center
    • Homepage
    • Data Virtualization
    • Data Connectors
      • Creating a Data Connector
      • Data Connector List
        • Edit Data Connectors
        • Create Option
        • Reconnecting to a Data Connector
        • Sharing a Data Connector
        • Delete a Data Connector
      • Supported Data Connectors
        • Database Connectors
          • MySQL
          • MSSQL
          • Oracle
          • ClickHouse
          • Arango DB
          • Hive
          • Cassandra
          • MongoDB
          • MongoDB for BI
          • PostgreSQL
          • Snowflake
        • File Data Connector
        • API Connectors
          • API Connector
          • Amazon
          • App Store
          • Bing Ads
          • Dropbox
          • FTP Server
          • Facebook
          • Facebook Ads
          • Firebase DB
          • Fitbit
          • Flipkart
          • Google Adwords
          • Google Analytics
          • Google Big Query
          • Google Forms
          • Google Sheet
          • HubSpot
          • Jira
          • Lead Squared
          • Linkedin
          • Linkedin Ads
          • MS Dynamics
          • Mailchimp
          • QuickBooks
          • SalesForce
          • ServiceNow
          • Twitter
          • Twitter Ads
          • Yelp
          • YouTube
          • ZOHO Books
        • Others
          • MS Sql Olap
          • Data Store
          • OData
          • Spark SQL
          • AWS Redshift
          • SAP Hana
    • Data Sets
      • Creating a New Data Set using RDBMS Connector
      • Creating a Data Set using Arango DB Connector
      • Creating a Data Set using an API Connector
      • Creating a New FTP Data Set
      • Data set list page
        • View Options: Data Sets List Page
        • Data Set List: Options
    • Data Stores
      • Creating a New Data Store
        • Data Store using an RDBMS Connector
        • Data Store using a Flat File Data Connector
        • Data Store using an API Data Connector
      • Adding Synonyms to a Datastore
      • Data Stores List
    • Data Store Meta Data
      • Sharing a Meta Data Store
      • Deleting a Meta Data Store
    • Data Sheets
      • Creating a Data Sheet
      • Publishing a Data Sheet
        • Entering Data
        • Viewing Data
        • Deleting a Row
      • Editing a Data Sheet
      • Removing a Data Sheet
    • Data Catalog
    • Data Sandbox
    • Data as API
  • Data Science Lab
    • What is Data Science Lab?
      • Design Philosophy
      • What is a DSL Project?
    • Getting Started
      • Accessing the DS Lab Module
    • Start your Data Science Experiment with DS Lab
    • Project
      • Creating a Project
      • Keep Multiple Versions of a Project
      • Sharing a Project
      • Editing a Project
      • Activating a Project
      • Deactivating a Project
      • Deleting a Project
    • Various Tabs to work with
      • Notebook
        • Ways to Access Notebook
          • Creating a Notebook
          • Uploading a Notebook
        • Notebook Page
          • Notebook Cells
            • Using a Code Cell
            • Using a Markdown Cell
          • Modifying a Notebook
          • Notebook Task Bar
          • Notebook Operations
            • Turn your Script into a Model
              • Registering a Model
              • Filtering a Model
            • Predict
            • Save Artifacts
            • Find and Replace
            • Secrets
            • Algorithms
            • Transforms
        • Notebook List Page
          • Export to Pipeline
          • Notebook Version Control
          • Sharing a Notebook
          • Editing a Notebook
      • Dataset
        • Adding Data Sets
        • Adding Data Sandbox
      • Utility
      • Model
        • Register a Model
        • Unregister a Model
        • Register Model as an API Service
          • Register a Model as an API
          • Register an API Client
          • Pass Model values in Postman
  • Data Pipeline
    • About Data Pipeline
      • Design Philosophy
      • Event based Process Orchestration
      • Low Code Visual Authoring
      • Real-time and Batch Orchestration
      • ML and Data Ops
      • Distributed Compute
      • Fault Tolerant and Auto-recovery
      • Extensibility via Custom Scripting
    • Getting Started
      • Homepage
        • List Pipeline
        • Creating Pipeline
          • Adding Components to Canvas
          • Connecting Components
          • Events [Kafka and DB Sync]
          • Memory and CPU allocations
        • List Components
        • Delete Orphan Pods
        • Scheduler
        • Data Channel
        • Settings
      • Pipeline Workflow Editor
        • Pipeline Toolbar
        • Component Panel
        • Right-side Panel
      • Testing Suite
      • Activating Pipeline
      • Monitoring Pipeline
    • Components
      • Adding component to worflow
      • Component Architecture
      • Component Base Configuration
      • Resource Configuration
      • Readers
        • S3 Reader
        • HDFS Reader
        • RDBMS Reader
        • Elastic Search Reader
        • SFTP Stream Reader
        • SFTP Reader
        • Mongo DB Reader
          • Docker Reader
          • Spark
        • Azure Blob Reader
        • Azure Metadata Reader
      • Writers
        • S3 Writer
        • RDBMS Writer
        • HDFS Writer
        • ES Writer
        • Mongo Writers
          • Mongo Writer (Spark)
          • Mongo Writer(Docker)
        • Azure Writer
        • Azure Cosmos DB Writer
        • Sandbox Writer
      • AI/ML
        • R Model Runner
        • Python Model Runner
        • DSL Model & Script Runner
      • Consumers
        • SFTP Monitor
        • MQTT Consumer
        • Eventhub Subscriber
        • Twitter Scrapper
        • API Ingestion and Webhook Listener
        • Mongo Change Stream
        • Rabbit MQ Consumer
        • AWS SNS Monitor
        • Kafka Consumer
      • Producers
        • WebSocket Producer
        • Eventhub Publisher
        • EventGrid Producer
        • Rabbit MQ Producer
      • Transformation
        • SQL Component
        • Dataprep Script Runner
        • File Splitter
        • Rule Splitter
        • Stored Procedure Runner
        • Flatten JSON
        • Email Component
        • Pandas Query Component
        • Enrichment Component
        • Mongo Aggregation
        • Data Loss Protection
      • Scripting
        • Script Runner
        • Python Script
      • Scheduler
    • Custom Components
    • Advance configuration & Monitoring
      • Configuration
        • Kafka Configuration
        • Logger Setting
        • Default Component Configuration
      • Data Channel
      • System component Status
    • Version Control
    • Use Cases
  • Data Visualization
    • Self Service
      • Getting Started
        • What is Story?
        • Creating a new Story
          • Accessing the Story Module
          • Creating a new View
            • Creating and Updating Instance
            • View Design
            • Search
      • Design
        • Guided Tour
        • Dimension Profiling
        • Data Store Merge at View Level
        • Measure Summary
        • Series Properties
        • Formula Field Editor
          • Creating a Formula
            • Record Level Option
            • Summary Level Option
          • Creating a Range
        • Order by and Limit
        • Adding a Slicer
      • Chart Gallery
        • Mixed chart
        • Area chart
        • Bar chart
        • Bubble chart
        • Column Stack chart
        • Line chart
        • Pie chart
        • Scattered Plot chart
        • TreeMap chart
        • Circumplex chart
        • Pareto chart
        • Semi Gauge
        • KPI Tile
        • KPI Tile: Comparative Tile
        • KPI Tile: Sparkline
        • Map
        • Data Grid
        • Metric Summary
        • R Server Visual
        • Dissolution chart
      • Storyboard
        • Shared Views
        • Export
        • Alert Center
        • Change Theme
        • Data Store Information
        • Options Assigned to a View
          • Analyse
            • Timeline Play
          • Edit
          • Remove
        • Data Interactions/ Data Drills
          • Drill Into
          • Drill Through
      • Applying Filters
        • View Specific Filter
          • Dimension-based View Filter
          • Measure-based View Filter
          • Date-based View Filter
        • Global Filter
          • Exclude from the Global Filter
          • Saving a Global Filter
          • Custom View Filter
          • Like and Not Like Filter Operations
      • Actions
        • Interactions
    • Governed Dashboards
      • About Dashboard Designer
        • What is Dashboard Designer?
        • Why is it used?
      • Getting Started
        • Accessing the Designer Module
        • Overview of Designer Module
          • Homepage
            • Guided Tour
            • Left Menu Panel
              • New
              • Manage
              • Open Dashboard
              • Preferences
              • Save As
              • Help
              • Exit
          • Dashboard Canvas Page
            • Right side Panel
              • Connection Page
              • Chart Container
              • Manage Window
              • Script Window
              • Guided Tour
            • Canvas Properties
            • Context Menu Properties
      • Create New Workspace
        • Workspace Creation: Complete Flow
        • Creating a Workspace
        • Renaming a Workspace
        • Deleting a Workspace
      • Create New Dashboard
        • Dashboard Creation: Complete Flow
        • Adding a New Dashboard
        • Create Connection
        • Drag and Drop Charts
        • Associate the Dataset
        • Preview the Dashboard
        • Save the Dashboard
      • Managing Options for a Dashboard
        • Open Dashboard in Designer
        • Publish a Dashboard
        • Share a Dashboard
        • Dashboard Version Control
        • Action Menu
          • Preview Dashboard
          • Export to Local Disk
          • Rename
          • Delete
          • Moving a Dashboard
      • Connecting to a Data Source
        • Accessing the Data Connectors
        • Establishing a Data Connection
          • CSV Connection
          • Excel Connection
          • Data Service Connection
          • Data Science Service
          • Data store Connection
          • Data Sheet
          • WebSocket Connection
          • Merged Connection
      • Charts Gallery
        • Charts
          • Area Chart
          • Bar Chart
          • Bubble Chart
          • Circumplex Chart
          • Column Chart
          • Funnel Chart
          • Histogram Chart
          • Inverted Funnel
          • KPI Tile
          • Line Chart
          • Map Chart
          • Mito Plot
          • Mixed Chart
          • Pie Chart
          • Project Timeline
          • Pyramid Chart
          • Spark Line
          • Scatter Plot
          • Spider Chart
          • Waterfall Chart
        • Grids
          • Data Grid
          • Paging Grid
          • Data Sheet
          • Scorecard
          • Pivot Grid
        • Filters
          • Checkbox
          • Combobox
          • Hierarchical Combobox
          • List
          • Radio Button
        • Advanced Charts
          • Box Plot
          • Candle Stick
          • Chevron
          • Data Search
          • Decision Tree
          • Group Bar
          • Group Column
          • Heat Map
          • Text Analyzer
          • Time Series
          • Tree Map
          • Trellis
          • Word Cloud
        • Other Charts
          • Box
          • Bullet
          • Date Picker
          • Export
          • Filter Saver
          • Gauge
          • Graphics
          • Guided Tour
          • H-Slider
          • Image
          • Info Button
          • Label
          • Legend
          • Progress Pie
          • Semi Gauge
          • Stepper
          • SVG Image
          • Text Box
          • Trend
          • Url Button
          • V-Slider
        • Custom Charts
        • Common Chart Properties
          • Background
          • Title & Sub-title
          • X & Y Axis Properties
          • Legend Properties
          • Formatter
          • Axis Setup
          • Export Options
      • Dashboard Objects
        • Manage Dashboard Components
        • Dashboard Objects Properties
      • Configuration
        • Filtering the Data
        • Drill Through
        • Indicators in Charts
        • Tooltip(Default & Custom)
        • Data Label
        • Geo Mapping
        • Language Mapping
        • Legend Mapping
        • Alerts in Grids
      • Script Help Section
        • Navigate to Script Help page
        • How scripts are used
Powered by GitBook
On this page
  • Seamless handling of Data and ML Ops
  • OIL & GAS part-1
  • OIL & GAS Part-2

Was this helpful?

  1. Data Pipeline

Use Cases

Refer to the below given YouTube videos to understand some actual scenarios on how the Data Pipeline functions.

PreviousVersion ControlNextData Visualization

Last updated 2 years ago

Was this helpful?

Seamless handling of Data and ML Ops

OIL & GAS part-1

OIL & GAS Part-2

Seamless Data Ops and ML Ops using BDB Platform
Oil & Gas part1
Oil & Gas part2