Streaming processing addresses a large family of applications for which traditional processing methods and disk-based storages, like databases or file systems, fall short. Such applications are pushing the limits of traditional data processing infrastructures.
Streaming support allows to query into rolling windows of coming data, and enable users to answer such questions as “What are the 10 most popular products over last 2 hours?”, or “What is the average product price in a certain category for the past day?”.
Diagram below illustrates how events flow through the rolling windows:
Another most common use case for stream processing is the ability to control and properly pipeline distributed events workflow. As events are coming into the system at high rates, the processing of events is split into multiple stages and each stage has to be properly routed within a cluster for processing.
Get Apache 2.0 licensed In-Memory Data Fabric now:Latest Download Documentation