GridGain Developers Hub

Storage Profiles and Engines

GridGain 9 features a modern and highly configurable storage. With it, you can easily choose where and how your data will be stored. This topic covers the storage engines and profiles. Distribution zones determine how data is organized and are covered in the separate topic.

The diagram below depicts the relationship between tables, distribution zones, storage profiles and storage engines:

storage

What is a Storage Engine?

Storage engine is a GridGain node entity responsible for storing data on disk or in volatile memory in a particular format. It defines:

  • The binary format of stored data;

  • Configuration properties for specific data formats.

GridGain is developed with the option to support different storage engines that can be used interchangeably, depending on the type of load you expect for your database. Currently, GridGain supports:

  • Persistent Apache Ignite page memory (B+ tree);

  • Persistent RocksDb (LSM tree);

  • Volatile (in-memory) Apache Ignite page memory (B+ tree).

What is a Storage Profile?

Storage profile is the GridGain node entity that defines a Storage Engine and its configuration parameters. A Distribution Zone must be configured to use a set of declared Storage Profiles, which can be used to parameterize tables created in this Zone with different Storage Engines. Storage profiles define:

  • What storage engine is used to store data;

  • Configuration values for a particular Storage Engine’s configuration properties.

You can declare any number of storage profiles on a node.

Default Storage Profile

GridGain creates a default storage profile that uses the persistent Apache Ignite storage engine (aipersist) to store data. Unless otherwise specified, distribution zones will use this storage profile to store data. To check the currently available profiles on the node, use the following command:

node config show storage.profiles

Configuring Storage Engines

Storage engines can only be defined once in the configuration, and all storage engines start with their respective default configuration. For more information about default configuration of storage engines, see the Storage Configuration section.

To change storage engine configuration, use the CLI tool. You can use the node config show storage.engines command to check the current storage engine configuration. You can change configuration by specifying the new parameters in the node config update command, for example:

node config show storage.engines
node config update storage.engines.aipersist.checkpoint.frequency = 16000

After the configuration is updated, make sure to restart the node.

Creating and Using Storage Profiles

By default, only the default storage profile is created, however a node can have any number of storage profiles on it. To create a new profile, pass the profile configuration to the storage.profiles parameter:

node config update storage.profiles:{rocksProfile{engine:rocksdb,size:10000}}

After the configuration is updated, make sure to restart the node. The created storage profile will be available for use by a distribution zone after the restart.

Defining Tables With Storage Profiles

After you have defined your storage profiles and distribution zones, you can create tables in it by using SQL or from code. Both zone and storage profile cannot be changed after the table has been created.

For example, here is how you create a simple table:

CREATE TABLE exampleTable (key INT PRIMARY KEY, value VARCHAR) WITH PRIMARY_ZONE='ExampleZone', STORAGE_PROFILE='profile1'

In this case, the exampleTable table will be using the storage engine with the parameters specified in the profile1 storage profile. If the node does not have the profile1, the table will not be stored on it. Each node may have different configuration for profile1, and data will be stored according to local configuration.