Starting the Apache Ignite Streaming Demo

Denis Magda

Former Head of Developer Relations at GridGain
Apache Ignite Committer and PMC Member

The Apache Ignite Streaming Demo, window="_blank" starts a simple application that simulates the trading activities of a stock exchange. The application receives executed market deals in real time from the PubNub Market Orders data stream, window="_blank" and forwards the deals to a locally running cluster. The demo is designed for Docker environments, enabling you to deploy the application in minutes on your personal laptop. With this first step of the tutorial, you deploy an Ignite cluster, connect the cluster with the GridGain Nebula, window="_blank", and launch the application that streams market orders.

What You Need

Docker 19 or later, or Docker Desktop Community 2.3 or later
Docker Compose 1.25.5 or later
Java Developer Kit 8 or later
Apache Maven 3.3 or later
Git 2.24 or later
Your favorite IDE, such as IntelliJ IDEA or Eclipse

Download the Streaming Demo

First, you need to download the demo project to your local laptop:

Download and unzip the source repository or use Git to clone the repository:

git clone https://github.com/GridGain-Demos/ignite-streaming-monitoring-demo.git

Go into the project root directory:
```
cd ignite-streaming-monitoring-demo
```

Easy, right? Let’s move forward.

Start the Apache Ignite Cluster

Start a two-node Apache Ignite cluster, with each node running as a separate container:

docker-compose -f docker/ignite-cluster.yaml up -d --scale ignite-server-node=2

The cluster nodes will persist all the records on disk. For caching, the nodes are allowed to use no more than 150MB of memory. The on-disk records are stored in the ignite-streaming-monitoring-demo\work\db folder that is automatically created when you start the cluster. Check ignite-streaming-monitoring-demo\config\ignite-config-node.xml and ignite-streaming-monitoring-demo\docker\ignite-cluster.yaml files for additional configuration settings.

Register the Cluster With GridGain Nebula

Through the rest of the tutorial, you use GridGain Nebula as your primary management and monitoring tool for Ignite clusters. You create an account with GridGain Nebula and interconnect your locally running Ignite cluster with the tool.

First, generate a new permanent token that you use to register the cluster with Nebula (the default token expires in 5 minutes after the cluster startup time):

Connect to the container of the first cluster node:
```
docker exec -it docker-ignite-server-node-1 bash
```
Go into the /opt/gridgain/bin/ folder of the container:
```
cd /opt/gridgain/bin/
```
Generate and copy a new token to register the cluster with the tool:
```
./management.sh --token
```
Quit the node’s container:
```
exit
```

Second, sign up with Nebula and interconnect it with the cluster:

Go to https://portal.gridgain.com, window="_blank".
Create an account, window="_blank" to sign in into Nebula.
Register the cluster, window="_blank" with Nebula by using the previously generated token.
Activate, window="_blank" the cluster.

Now, you should see a dashboard similar to the following:

If You Prefer Deploying GridGain Control Center

It’s also possible to install GridGain Control Center locally and use it throughout the tutorial. Make sure that you remain in the root directory of the demo project and deploy the tool locally:

Deploy Control Center as a container:

docker-compose -f docker/control-center.yaml up -d

Connect to the container of the first cluster node:
```
docker exec -it docker-ignite-server-node-1 bash
```
Go into the /opt/gridgain/bin/ folder of the container:
```
cd /opt/gridgain/bin/
```

Instruct the cluster to connect to the local Control Center container:

./management.sh --uri http://localhost:8443,http://control-center-frontend:8443

Generate and a new token to register the cluster with the tool:
```
./management.sh --token
```
Quit the node’s container:
```
exit
```
Open the http://localhost:8443, window="_blank" address in the browser, create an account and register the cluster using the token as it was shown for the hosted version of Control Center above.

Launch Market Orders Application

In this final step of this part of the tutorial, you launch the application that streams executed market orders into the Ignite cluster. Make sure that you have an Internet connection so that the application can connect to the PubNub Market Order data stream, window="_blank".

Now, you compile, build, and start the application:

Use Maven to generate an application executable file from the source code of the project:
```
mvn clean package
```

Dockerize the application:

docker build -f docker/StreamingAppDockerfile -t ignite-streaming-app .

Deploy the application in Docker:

docker-compose -f docker/ignite-streaming-app.yaml up -d

Finally, you need to confirm that the application is, in fact, streaming data into the cluster by querying the records from the Nebula SQL screen:

Get the last 10 executed trades (as indicated in the example screenshot):
```
SELECT * FROM Trade ORDER BY order_date DESC LIMIT 10;
```

Get the most popular stocks among the buyers:

SELECT symbol, SUM(bid_price) as total FROM Trade GROUP BY(symbol) ORDER BY total DESC;

Get a list of the buyers that completed the most transactions:

SELECT buyer.id, first_name, last_name, sum(bid_price) as total FROM Trade
JOIN Buyer ON trade.buyer_id = buyer.id
GROUP BY (buyer.id) ORDER BY total DESC;

What’s Next

You set up a Nebula dashboard that displays storage-related metrics:

Part 2: Monitoring Storage Usage

© 2024 GridGain Systems, Inc. All Rights Reserved. Privacy Policy | Legal Notices. GridGain® is a registered trademark of GridGain Systems, Inc.
Apache, Apache Ignite, the Apache feather and the Apache Ignite logo are either registered trademarks or trademarks of The Apache Software Foundation.

Last updated on Oct 09, 2024