Intelligent Scaling

A way to scale up the processing speed of components.

A feature to scale your component to the max number of instances to reduce the data processing lag automatically. This feature detects the need to scale up the components in case of higher data traffic.

Please Note: This feature is available both in Spark & Docker components. This feature will only work with the real-time as invocation type.

Intelligent Scaling as a part of the ClickHouse Writer

Please Note: Components will scale up if there is a lag of more than 60% and if the lag goes less than 10% component pods will automatically scale down. This lag percentage is configurable.