How-To for Real-Time Alerting, Analytics, and Reporting at Scale with Apache Kafka and Apache Ignite

This talk demonstrates how to implement integrating Apache^® Kafka with Apache^® Ignite^TM in practice,  explains the architectural reasoning, the benefits of real-time integration, and shares common usage patterns. The presenters build a streaming data pipeline using nothing but their bare hands, Apache Ignite, Kafka Connect, and KSQL.

Are you tasked to build a system or upgrade an existing architecture to a solution capable of handling unbound streams of data, do real-time alerting, storing always growing terabytes and petabytes of data and, finally, act on the data within milliseconds SLAs. Sounds like a challenging but rather a common task? Yes! Should you as an engineer or architect build it up off the ground? Unlikely!

What if we tell you that by integrating Apache Kafka with Apache Ignite you’ll solve all of the requirements faster and easier. A battle-tested recipe is simple - take Kafka Connect and have your data stream through Kafka pipelines, add a pinch of KSQL to act on the streams with SQL in real-time with zero delays, rinse and flush the preprocessed data in Ignite as in-memory databases and get further insights by analyzing your hot and cold datasets.

Watch the video.