GridGain Developers Hub

Manual Install on Amazon EC2

This page is a step by step guide to running a GridGain cluster on Amazon EC2. Through this guide you will learn how to launch two EC2 instances each running a GridGain node and configure them to join with each other to form a cluster. You can extend this guide to any number of nodes.

We will use two t2.micro instances, GridGain Community Edition, and Java 8.

Prerequisites

Considerations

Choose the right type of instances that suit your needs. Consider the following points:

  • RAM: If you are going to use GridGain as an in-memory storage, make sure to select an instance with enough RAM to fit all your data.

  • Disk space: If you are launching a cluster with a persistent storage, provide enough size to accommodate all your data when configuring the instance’s volume.

    You may want to uncheck the Delete on Termination option for your storage.

  • Networking: GridGain nodes discover and communicate with each other by TCP/IP. A number of ports must be open for this communication. See Configuring Security Group.

Visit our Capacity Planning page for information about ways to estimate hardware requirements for your use case.

Launching Amazon EC2 Instances

Log into EC2 Management Console and configure two EC2 instances of your choice. In this instruction, we use two t2.micro instances.

Configuring Security Group

When configuring the instances, you will be asked to specify a security group. The security group must allow connection to the following ports:

Protocol Port Description

TCP

47500-47600

Discovery ports.

TCP

47100-47200

Communication ports.

TCP

10800

Optional. Thin client port.

TCP

8080

Optional. For REST API requests.

TCP

11211

For control.sh calls. This port should be opened only for cluster member nodes, from which a user might need to call control.sh.

These are the default ports that GridGain nodes use for discovery and communication purposes. If you want to use values other than the default ones, open them.

This is what our security group should look like:

aws security group

Starting Instances

Launch the two instances.

ec2 instances

Each instance has two IP addresses: public address and private address. The private IP address is the address of the instance within the cloud. The public address is the address that is available from the Internet. We will use the private IP addresses to configure the discovery mechanism of the nodes. Use the public IP addresses when connecting to the instances via SSH.

Setting Up the Environment

Connect to the instance via ssh:

$ ssh -i privatekey.pem ec2-user2@54.175.137.126

If java is not already installed, install it using the package manager of the instance.

$ sudo yum install java-1.8.0-openjdk.x86_64

Upload a GridGain distribution package into the instance. Run the following command from you local machine:

$ scp -i privatekey.pem gridgain-community-8.9.4.zip scp://ec2-user@54.175.137.126

Login to the instance again and unpack the package:

$ unzip gridgain-community-8.9.4.zip

If you are going to connect to the cluster via REST API or GridGain Nebula, enable the 'ignite-rest-http' module:

$ cp -r gridgain-community-8.9.4/libs/optional/ignite-rest-http/ gridgain-community-8.9.4/libs/

Repeat the above steps for the second instance. Now we are ready to configure the cluster nodes.

Configuring Discovery

Cluster nodes launched in different EC2 instances must be able to connect to each other. This is achieved by configuring the discovery mechanisms on each node. There are two ways you can do that:

Discovering Nodes by TCP/IP

To configure discovery by TCP/IP, specify the private IP addresses of each instance in the node’s configuration file.

aws-static-ip-finder.xml:
<beans xmlns="http://www.springframework.org/schema/beans"
    xmlns:util="http://www.springframework.org/schema/util"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="         http://www.springframework.org/schema/beans
    http://www.springframework.org/schema/beans/spring-beans.xsd
    http://www.springframework.org/schema/util
    http://www.springframework.org/schema/util/spring-util.xsd">
    <bean class="org.apache.ignite.configuration.IgniteConfiguration">

        <!-- other properties -->

        <!-- Discovery configuration -->
        <property name="discoverySpi">
            <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
                <property name="ipFinder">
                    <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">
                        <property name="addresses">
                            <list>
                                <value>172.31.28.36</value>
                                <value>172.31.23.105</value>
                            </list>
                        </property>
                    </bean>
                </property>
            </bean>
        </property>
    </bean>
</beans>

Connect to each instance via ssh and start a node as follows:

$ ./gridgain-community-8.9.4/bin/ignite.sh aws-static-ip-finder.xml

After starting the second node, you should see the following message in the console:

aws successful launch

server=2 means that the nodes were able to connect to each other and form a cluster.

Automatic Discovery Using Amazon S3

You can use the Amazon S3 IP Finder to configure automatic discovery of nodes.

Enable the 'ignite-aws' module:

$ cp -r gridgain-community-8.9.4/libs/optional/ignite-aws/ gridgain-community-8.9.4/libs/

Go to the Amazon S3 Management Console and create a simple bucket with default settings. Our bucket is named 'gg-ip-finder-bucket':

aws s3 bucket

Provide the name of the bucket in the node’s configuration file as follows.

aws-s3-ip-finder.xml:
<beans xmlns="http://www.springframework.org/schema/beans"
    xmlns:util="http://www.springframework.org/schema/util"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="         http://www.springframework.org/schema/beans
    http://www.springframework.org/schema/beans/spring-beans.xsd
    http://www.springframework.org/schema/util
    http://www.springframework.org/schema/util/spring-util.xsd">
    <bean class="org.apache.ignite.configuration.IgniteConfiguration">
        <!-- other properties -->
        <property name="discoverySpi">
            <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
                <property name="ipFinder">
                    <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.s3.TcpDiscoveryS3IpFinder">
                        <property name="awsCredentials" ref="aws.creds"/>
                        <property name="bucketName" value="gg-ip-finder-bucket"/>
                    </bean>
                </property>
            </bean>
        </property>
    </bean>
    <!-- AWS credentials. Provide your access key ID and secret access key. -->
    <bean class="com.amazonaws.auth.BasicAWSCredentials" id="aws.creds">
        <constructor-arg value="YOUR_ACCESS_KEY_ID"/>
        <constructor-arg value="YOUR_SECRET_ACCESS_KEY"/>
    </bean>
</beans>

Replace the YOUR_ACCESS_KEY_ID and YOUR_SECRET_ACCESS_KEY values with your access key and secret access key. Refer to the Where’s My Secret Access Key? page for details.

Use the following configuration to start nodes:

$ ./gridgain-community-8.9.4/bin/ignite.sh aws-s3-ip-finder.xml

After starting the second node, you should see the following message in the console:

aws successful launch

server=2 means that the nodes were able to connect to each other and form a cluster.

Connecting to the Cluster

You can connect to the cluster using various methods, including thin clients, REST API, JDBC/ODBC. For each method, you need to open a specific port on the instance. For example, for the REST API the port is 8080, for JDBC and thin clients it’s 10800, etc.

Connecting a Client Node

A client node is a full-featured node in that it supports all APIs available to a server node, but it does not host cache data. Just like a regular server node, the client node uses the discovery and communication mechanisms to join the cluster. If you want to run client nodes in AWS, then use the same discovery configuration as for the server nodes. If you want to connect a client node from your local machine (or any on-premise server), make sure that the discovery and communication ports are opened on the machine and that you can connect to them from the EC2 instances. Check both the security group of the instances and the firewall configuration on your local machine.

For a client node to join the cluster from your local machine, perform the following steps:

  1. Add an address resolver to the configuration of all nodes running in AWS.

    Because AWS instances are running behind a NAT, you have to map the private IP address of each instance to its public IP address in the node configuration. To do this, add an address resolver to IgniteConfiguration, as shown in the code snippet below:

    <bean class="org.apache.ignite.configuration.IgniteConfiguration">
    <property name="addressResolver">
        <bean class="org.apache.ignite.configuration.BasicAddressResolver">
            <constructor-arg>
                <map>
                    <entry key="172.31.59.27" value="3.93.186.198"/>
                </map>
            </constructor-arg>
        </bean>
    </property>

    <!-- other properties -->

    <!-- Discovery configuration -->
</bean>

    In this example, 172.31.59.27 is the private IP address of the instance, and 3.93.186.198 is its public IP address.

  2. The discovery configuration of the client node must contain the IP address of at least one remote node. If the network configuration settings are correct, the node will be able to connect to all remote nodes.

    Your local node configuration might look as follows:

    <bean class="org.apache.ignite.configuration.IgniteConfiguration" id="ignite.cfg">
    <property name="clientMode" value="true"/>

    <!-- Discovery configuration -->
    <property name="discoverySpi">
        <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
            <property name="ipFinder">
                <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">
                    <property name="addresses">
                        <list>
                            <value>3.93.186.198</value>
                        </list>
                    </property>
                </bean>
            </property>
        </bean>
    </property>
</bean>

    If your local machine is also behind a NAT, add an address resolver to its configuration.

  3. Start the server nodes first (the nodes running in AWS), and then start your local node. You should see the following message in the console on your local machine.

    client to aws

    client=1 indicates that the client node connected successfully.

Connecting Using the REST API

If you enabled the 'ignite-rest-http' module and opened port 8080, you can connect to the cluster as follows:

$ curl http://<instance_public_IP>:8080/ignite?cmd=version
{"successStatus":0,"error":null,"sessionToken":null,"response":"8.9.4"}

Connecting with a Thin Client

Let’s create a simple application that connects to our cluster with a java thin client. You can use other supported thin clients.

The default port for client connection is 10800. You need to tell the thin client the public address of one of your instances and this port. Make sure to add the 'ignite-core' dependency to your application.

ClientConfiguration cfg = new ClientConfiguration().setAddresses("54.175.137.126:10800");
IgniteClient client = Ignition.startClient(cfg);

ClientCache<Integer, String> cache = client.getOrCreateCache("test_cache");

cache.put(1, "first test value");

System.out.println(cache.get(1));

client.close();

This simple piece of code creates a cache in the cluster and puts one key-value pair into it.

© 2024 GridGain Systems, Inc. All Rights Reserved. Privacy Policy | Legal Notices. GridGain® is a registered trademark of GridGain Systems, Inc.
Apache, Apache Ignite, the Apache feather and the Apache Ignite logo are either registered trademarks or trademarks of The Apache Software Foundation.

Last updated on Apr 25, 2024