The virtual Ignite Summit on June 14 attracted hundreds of Igniters from around the globe excited to learn and share their experiences. (Video of all sessions is available on-demand here.) The event, organized by GridGain and the Apache Ignite community, was preceded the day before by free technical training sessions on Apache Ignite essentials, compute grid, and Apache Ignite with Spring Boot and Spring Data Development.
“The Summit was all about developers talking to developers with no marketing or management-level presentations,” said Kseniya Romanova, GridGain community relations director. “They shared the amazing things they’re doing in different industries that impact millions of users and consumers every day around the world.”
Nikita Ivanov, founder of GridGain and author of Apache Ignite, provided the summit keynote with a presentation on the future of Apache Ignite that stressed the need for a simplified user experience dating back to when he started the project in 2007. He stressed that although there are many complex enterprise-class features, Ignite and HPC in general need not be overly complex. Mass adoption, in his opinion, requires a “Back to the Roots Movement” to make Ignite easier for everyone to use, install, configure, and train. Greater simplicity is a big driver of the development of Apache Ignite 3.0, the next major version.
GridGain’s effort to introduce greater simplicity is also reflected in the creation of GridGain Nebula, a SaaS offering in the cloud that dramatically simplifies the use of Ignite.
Google-like Search Plus Kubernetes and Service Mesh Deployment Benefits
Software engineers Suman Das and Ankita Sawant represented Crux Intelligence, developer of an AI-powered informatics tool that discovers patterns and insights in enterprise data automatically and is user-friendly enough to be used by business decision makers. They explained how they deployed Apache Ignite GridGain to provide a Google-like search experience after not being able to identify any caching technology that could provide this functionality. They also described how the cluster was deployed in Kubernetes along with the Istio service mesh that allows monitoring of services to make sure they are reliable and secure without having to make any changes to code.
Creating a Credit Card Processor with Apache Ignite and Kafka
Daniel Rossos, IBM Software Engineering Intern, explained how his team used Kafka and Apache Ignite hosted on an IBM OpenShift cluster to create a real-time, credit card transaction processor. The solution uses Kafka for processing and mapping the credit card data and GridGain Kafka Connect for synсing and sourcing data between Apache Ignite and Kafka.
The solution uses Apache Ignite not only as a database but also to run an in-memory continuous query to flag any fraudulent transactions. Flagged transactions are placed into a separate cache and, through the use of a sync connector, are transferred back into Kafka within a “fraud” topic. This enables the event-processing power of Kafka to work in concert with the in-memory computing and querying speed of Apache Ignite. Rossos described how the team created the architecture, wrote the Apache Ignite Helm Chart code, and configured the continuous query function.
Lightning-fast Flight Searches on Expedia
Bhanu Choudhary and Rohit Goel, software engineers from global travel reservations platform Expedia Group, shared their journey from a Cassandra caching architecture that drove an impressive 3 second response for airline flight queries to an Ignite architecture that drove the same queries down to an earth-shattering 150 MS. That's a 95% improvement in every query, multiplied by millions of queries every day on the Expedia platform.
10X Faster Analytics
From risk and investment specialist Cardano Risk Management, analytics lead Ali Ferda Arikan described how Cardano developers built a scalable, high-performance, quantitative analytics solution using Ignite. The solution analyzes pricing and risk management of financial instruments. It’s highly scalable and provides data access 10 times faster than before by removing data access bottlenecks and hardware limitations with Ignite. The team expects to speed up performance even further by breaking down computations, cache intermediate and further objects, and speed up calculations.
Innovations in Apache Ignite 3
Valentin Kulichenko, Apache Ignite PMC member, talked about the next major version, Apache Ignite 3, and the main features developers are attempting to improve. The goals of Ignite 3 include modernizing the platform through replacement of legacy replication and concurrency protocols and data formats and making the platform more adaptable for cloud deployments. Another goal, to echo Nikita Ivanov, is to simplify Ignite installation, upgrade, configuration, and schema management and to replace the legacy APIs with a single set for both client and server.
Aleksandr Polovtsev, Senior Software Engineer at GridGain and an Apache Ignite Committer, talked about the ability to specify multiple storage engines in Apache Ignite 3. While in-memory and persistent data storage based on a B+ tree structure is a standard approach in many DBMSs, some workloads require a different underlying data structure to optimize an application’s performance. Apache Ignite 3 addresses this issue by providing an alternative data storage option based on a Log-Structured Merge (LSM) tree and introducing the concept of storage engines, which allow developers to specify storage type on a per-table basis to provide optimal performance for both read-heavy and write-heavy workloads.
Konstantin Orlov, Lead Software Engineer at GridGain and an Apache Ignite Committer, is helping to develop a new query engine in Ignite 3 powered by Apache Calcite, which was first introduced as an experimental feature in Apache Ignite 2.13. It improves upon the reliance by distributed SQL of the H2 in-memory database, which has limitations due to its non-distributed essence. Apache Calcite enables execution of queries of arbitrary complexity, transaction support, and automated data collocation.
Aleksandr Pakhomov, Software Engineer at Unison Technologies, presented improvements to the Ignite CLI tool with a demo on bash autocompletion, scripting support, the SQL interface, and advanced completions in SQL REPL.
Data Management in the Internet of Things with Ignite
IoT Evangelist Piotr Wykowski of Transition Technologies talked about how Apache Ignite handles connection to tens of thousands or hundreds of thousands of IoT devices. As an example, he used PTC ThingWorx, an enterprise application development platform for the IoT. Ignite’s distributed cache helps keep data in ThingWorx synchronized in clustered environments. Distributed computation provides data insights for diverse IoT use cases, including remote asset monitoring, remote and predictive maintenance, and the optimization of equipment effectiveness. He goes into greater detail in this blog.
Migration Tool Development Overview
The Apache Ignite Migration Tool that allows migration from one database schema to another between Ignite clusters was the topic of a talk by Senior Software Engineer Edward Baranetsky of Alledium. Developed by Alledium, the open source tool is meant to improve upon existing migration tools such as Liquidbase and Flyway that have limited API solutions, no support for native Java objects or Java patches, and other limitations on Ignite snapshots and backups.
Building an Apache Ignite Compute Grid Application
Bringing distributed computing to the cloud with the Ignite Compute Grid API, that brings computations from clients to servers, was the subject of a talk by Stan Luyanov, Technical Director at GridGain. Traditional client-server architectures rely on sending data from databases to where the client code is. The Apache Ignite Compute Grid enables running code on the server side, sending your code to the data and reducing network overhead. The ability to run code on the server side is the distinguishing feature of the Apache Ignite toolset. But distributed computing can be complicated. Compute Grid makes it easier for developers by automating provisioning, code deployment, monitoring, and management of Apache Ignite clusters. Stan described how to build a Compute Grid application — from the code developed on the local computer to a production-ready cluster running in the cloud — using the GridGain Nebula.
Exposing Legacy Mainframe Data
At many enterprise companies, mainframe systems still support core transactions and batch processing. Timothy Anderson, Software Engineer at M&T Bank, discussed how to overcome the challenges of exposing critical mainframe data to the cloud and other distributed data platforms through implementation of IBM’s Z Digital Integration Hub built on the GridGain for z/OS platform, an enterprise-grade in-memory computing platform based on Apache Ignite and optimized for use on z/OS mainframes. Now, cloud integrations are easily set up, data is continuously updated in real time, consuming applications no longer concern themselves with data structures of the system of record that used to cause delays, everything is stored in one location and accessible through standard queries.
From RDBMS to Multi-site Global Architecture with Ignite
Senior Principal Architect Harish Negalaguli of Motorola Solutions talked about how the company used Apache Ignite for their global, multi-site deployment architecture for mission-critical communications. Using GridGain Ultimate Edition, Motorola has deployed an active/active server architecture with dynamic scaling, flexible memory caching based on separate storage and caching layers, and on-the-fly SQL schedule upgrades. The multi-site architecture features graceful handling of IP communication failures between data centers, deployment automation, native cloud platform support for technologies like Dockers and Kubernetes, and multi data center active/active replication.
All of the Ignite Summit 2022 sessions are available for viewing here.