Cluster Configuration Parameters

GridGain 9 cluster configuration is shared across the whole cluster. Regardless of which node you apply the configuration on, it will be propagated to all nodes in the cluster.

In GridGain 9, you can create and maintain configuration in either HOCON or JSON. The configuration file has a single root "node," called ignite. All configuration sections are children, grandchildren, etc., of that node.

Checking Cluster Configuration

To get cluster configuration, use the CLI tool.

Start the CLI tool and connect to any node in the cluster.
Run the cluster config show command.

The CLI tool will print the full cluster configuration. If you only need a part of the configuration, you can narrow down the search by providing the properties you need as the command argument, for example:

cluster config show ignite.transaction

Changing Cluster Configuration

Cluster configuration is changed from the CLI tool. To change the configuration:

Start the CLI tool and connect to any node in the cluster.
Run the cluster config update command and provide the updated configuration as the command argument, for example:
```
cluster config update ignite.metaStorage.idleSyncTimeInterval=600
```

The updated configuration will automatically be applied across the cluster,

Exporting Cluster Configuration

If you need to export cluster configuration to file, use the following command:

bin/gridgain9 cluster config show > cluster-config.txt

Configuration Parameters

Encryption Configuration

{
  "ignite" : {
    "encryption" : {
      "activeProvider" : "",
      "enabled" : false,
      "providers" : [ ]
    }
  }
}

Property	Default	Description	Changeable	Requires Restart	Acceptable Values
activeProvider	""	The name of the currently active encryption provider.	Yes	No	A name from the provider map
enabled	false	Whether the encryption is enabled.	Yes	No	true, false
providers		A named list (map) of the available encryption providers.	Yes	No	A valid provider map

Property

Default

Description

Changeable

Requires Restart

Acceptable Values

activeProvider

The name of the currently active encryption provider.

Yes

A name from the provider map

enabled

false

Whether the encryption is enabled.

Yes

true, false

providers

A named list (map) of the available encryption providers.

Yes

A valid provider map

Event Log Configuration

{
  "ignite" : {
    "eventlog" : {
      "channels" : [ ],
      "sinks" : [ ]
    }
  }
}

Property	Default	Description	Changeable	Requires Restart	Acceptable Values
channels		A named list of event log channels.	Yes	No	Valid channels
sinks		A named list of event log sinks.	Yes	No	Valid sinks

Property

Default

Description

Changeable

Requires Restart

Acceptable Values

channels

A named list of event log channels.

Yes

Valid channels

sinks

A named list of event log sinks.

Yes

Valid sinks

Garbage Collection Configuration

{
  "ignite" : {
    "gc" : {
      "batchSize" : 5,
      "lowWatermark" : {
        "dataAvailabilityTime" : 600000,
        "updateInterval" : 300000
      },
      "threads" : 16
    }
  }
}

Property	Default	Description	Changeable	Requires Restart	Acceptable Values
batchSize	5	The number of entries to be removed by the garbage collection batch for each partition	Yes	No	0 - inf
lowWatermark.dataAvailabilityTime	600000	The duration the outdated versions are available for, in milliseconds.	Yes	No	1000 - inf
lowWatermark.updateInterval	300000	The interval of the low watermark updates.	Yes	No	0 - inf
threads	Runtime.getRuntime().availableProcessors()	The number of threads used by the garbage collector.	Yes	Yes	1 - inf

Property

Default

Description

Changeable

Requires Restart

Acceptable Values

batchSize

The number of entries to be removed by the garbage collection batch for each partition

Yes

0 - inf

lowWatermark.dataAvailabilityTime

600000

The duration the outdated versions are available for, in milliseconds.

Yes

1000 - inf

lowWatermark.updateInterval

300000

The interval of the low watermark updates.

Yes

0 - inf

threads

Runtime.getRuntime().availableProcessors()

The number of threads used by the garbage collector.

Yes

1 - inf

License Configuration

{
  "ignite" : {
    "license" : {
        "content" : "\n    features=[],\n    id=\"2bd47bd9-c289-458e-a014-27558baf8874\",\n    infos{\n      companyName=GridGain,\n      companyWebsite=\"gridgain.com\",\n      contactEmail=\"Contact Email\",\n      contactName=\"Contact Name\",\n      contractEndDate=\"2042-12-31\",\n      contractStartDate=\"2007-01-01\",\n      licenseNote=\"GridGain Systems, Internal Development Only\"\n    },\n    limits{\n      expireDate=\"2042-12-31\",\n      maxComputers=0,\n      maxCores=0,\n      maxHeapSize=0,\n      maxHostRamSize=0,\n      maxNodes=0,\n      maxOffHeapSize=0,\n      startDate=\"2007-01-01\"\n    }\n  ",
        "signature" : "********"
    }
  }
}

Property	Default	Description	Changeable	Requires Restart	Acceptable Values
content	""	License content; a HOCON string.	Yes	No	A HOCON string with valid constraints
signature	""	Signature for the license content.	Yes	No	A valid HEX string

Property

Default

Description

Changeable

Requires Restart

Acceptable Values

content

License content; a HOCON string.

Yes

A HOCON string with valid constraints

signature

Signature for the license content.

Yes

A valid HEX string

Metastorage Configuration

{
  "ignite" : {
    "metaStorage" : {
      "idleSyncTimeInterval" : 500
    }
  }
}

Property	Default	Description	Changeable	Requires Restart	Acceptable Values
idleSyncTimeInterval	500	Period (in milliseconds) used to determine how often to issue time sync commands when Metastorage is idle (no Writes are issued). Should not exceed schemaSync.delayDuration. The optimal value is schemaSync.delayDuration / 2.	Yes	No (becomes effective on Metastorage leader reelection)	1 - inf

Property

Default

Description

Changeable

Requires Restart

Acceptable Values

idleSyncTimeInterval

500

Period (in milliseconds) used to determine how often to issue time sync commands when Metastorage is idle (no Writes are issued). Should not exceed schemaSync.delayDuration. The optimal value is schemaSync.delayDuration / 2.

Yes

No (becomes effective on Metastorage leader reelection)

1 - inf

Metrics Configuration

{
  "ignite" : {
    "metrics" : {
      "exporters" : [ ]
    }
  }
}

Property	Default	Description	Changeable	Requires Restart	Acceptable Values
exporters		The list of metric exporters currently used.	Yes	No	Valid exporters

Property

Default

Description

Changeable

Requires Restart

Acceptable Values

exporters

The list of metric exporters currently used.

Yes

Valid exporters

Replication Configuration

{
  "ignite" : {
    "replication" : {
      "idleSafeTimePropagationDuration" : 1000,
      "leaseAgreementAcceptanceTimeLimit" : 120000,
      "leaseExpirationInterval" : 5000,
      "rpcTimeout" : 60000
    }
  }
}

Property	Default	Description	Changeable	Requires Restart	Acceptable Values
idleSafeTimePropagationDuration	1000	Interval between Partition Safe Time updates.	No	N/A	1 - inf
leaseAgreementAcceptanceTimeLimit	120000	The maximum duration of an election for a new partition leaseholder, in milliseconds.	Yes	N/A	5000 - inf
leaseExpirationInterval	5000	The duration of a single lease.	Yes	N/A	2000 - 120000
rpcTimeout	60000	Replication request processing timeout.	Yes	No	0 - inf

Property

Default

Description

Changeable

Requires Restart

Acceptable Values

idleSafeTimePropagationDuration

1000

Interval between Partition Safe Time updates.

N/A

1 - inf

leaseAgreementAcceptanceTimeLimit

120000

The maximum duration of an election for a new partition leaseholder, in milliseconds.

Yes

N/A

5000 - inf

leaseExpirationInterval

5000

The duration of a single lease.

Yes

N/A

2000 - 120000

rpcTimeout

60000

Replication request processing timeout.

Yes

0 - inf

Schema Sync Configuration

{
  "ignite" : {
    "schemaSync" : {
      "delayDuration" : 100,
      "maxClockSkew" : 500
    }
  }
}

Property	Default	Description	Changeable	Requires Restart	Acceptable Values
delayDuration	100	The delay after which a schema update becomes active. Should exceed the typical time to deliver a schema update to all cluster nodes, otherwise delays in handling operations are possible. Should not be less than metaStorage.idleSyncTimeInterval. The optimal value is metaStorage.idleSyncTimeInterval * 2.	No	N/A	1 - inf
maxClockSkew	500	Maximum physical clock skew (ms) tolerated by the cluster. If the difference between physical clocks of two nodes in the cluster exceeds this value, the cluster might demonstrate abnormal behavior.	No	N/A	0 - inf

Property

Default

Description

Changeable

Requires Restart

Acceptable Values

delayDuration

100

The delay after which a schema update becomes active. Should exceed the typical time to deliver a schema update to all cluster nodes, otherwise delays in handling operations are possible. Should not be less than metaStorage.idleSyncTimeInterval. The optimal value is metaStorage.idleSyncTimeInterval * 2.

N/A

1 - inf

maxClockSkew

500

Maximum physical clock skew (ms) tolerated by the cluster. If the difference between physical clocks of two nodes in the cluster exceeds this value, the cluster might demonstrate abnormal behavior.

N/A

0 - inf

Security Configuration

{
  "ignite" : {
    "security" : {
      "authentication" : {
        "providers" : [ {
          "name" : "default",
          "type" : "basic",
          "users" : [ {
            "password" : "********",
            "roles" : [ "system" ],
            "username" : "ignite",
            "displayName" : "ignite"
          }]
        } ]
      },
      "authorization" : {
        "roles" : [ {
          "name" : "system",
          "privileges" : [ ],
          "displayName" : "system"
        }]
      },
      "enabled" : false,
      "jwt" : {
        "keyTtl" : 1209600000,
        "ttl" : 28800000
      }
    }
  }
}

Property	Default	Description	Changeable	Requires Restart	Acceptable Values
Authentication parameters
providers.name	default	The name of the authentication provider.	Yes	No	A valid string
providers.type	basic	The authentication provider type.	Yes	No	basic, ldap
providers.users		The list of users registered with the specific provider.
providers.users.displayName	ignite	Case sensitive user name.	No	N/A	A valid username
providers.users.password	**	User password.	Yes	No	A valid password
providers.users.passwordEncoding	PLAIN	User password’s encoding.	Yes	No	PLAIN, BCRYPT
providers.users.roles	system	The list of roles assigned to the user.	Yes	No	A valid role
providers.users.username	ignite	Case-insensitive user name.	Yes	No	A valid user name
Authorization parameters
roles.displayName	system	Case-sensitive role name.	Yes	No	A valid role name
roles.name	system	Case-insensitive role name.	Yes	No	A valid role name
roles.privileges		A list of privileges available to users with the role.
roles.privileges.action	""	The action the privilege relates to.	Yes	No	A valid action
roles.privileges.name	""	A privilege name on privilege list .	No	N/A	A valid name
roles.privileges.on	""	The name of the object the privilege applies to.	Yes	No	A valid object name or empty
Security parameters
enabled	false	Whether authentication is enabled.	Yes	No	true, false
JWT parameters
jwt.keyTtl	1209600000	TTL for JWT security tokens.	Yes	No	A valid LONG value
jwt.ttl	28800000	The frequency of private key updates used to issue JWT tokens.	Yes	No	A valid LONG value

Property

Default

Description

Changeable

Requires Restart

Acceptable Values

Authentication parameters

providers.name

default

The name of the authentication provider.

Yes

A valid string

providers.type

basic

The authentication provider type.

Yes

basic, ldap

providers.users

The list of users registered with the specific provider.

providers.users.displayName

ignite

Case sensitive user name.

N/A

A valid username

providers.users.password

User password.

Yes

A valid password

providers.users.passwordEncoding

PLAIN

User password’s encoding.

Yes

PLAIN, BCRYPT

providers.users.roles

system

The list of roles assigned to the user.

Yes

A valid role

providers.users.username

ignite

Case-insensitive user name.

Yes

A valid user name

Authorization parameters

roles.displayName

system

Case-sensitive role name.

Yes

A valid role name

roles.name

system

Case-insensitive role name.

Yes

A valid role name

roles.privileges

A list of privileges available to users with the role.

roles.privileges.action

The action the privilege relates to.

Yes

A valid action

roles.privileges.name

A privilege name on privilege list .

N/A

A valid name

roles.privileges.on

The name of the object the privilege applies to.

Yes

A valid object name or empty

Security parameters

enabled

false

Whether authentication is enabled.

Yes

true, false

JWT parameters

jwt.keyTtl

1209600000

TTL for JWT security tokens.

Yes

A valid LONG value

jwt.ttl

28800000

The frequency of private key updates used to issue JWT tokens.

Yes

A valid LONG value

Snapshots Configuration

{
  "ignite" : {
    "snapshot" : {
          "snapshotTombstonesTtlMinutes" : 1440,
          "snapshotsPath" : "snapshots"
    }
  }
}

Property Default Description Changeable Requires Restart Acceptable Values

Property	Default	Description	Changeable	Requires Restart	Acceptable Values
snapshotTombstonesTtlMinutes	1440	TTL for snapshot tombstones in minutes.	Yes	No	Non-negative LONG value
snapshotsPath	snapshots	The path where the snapshots are saved on the node. By default, `{GRIDGAIN_HOME}/work/snapshots`. If an absolute path is specified, a subfolder with the node name is used: `/absolute-path` → `/absolute-path/node-name` Relative paths are calculated from the node’s working directory: `relative-path` → `{GRIDGAIN_HOME}/work/relative-path`	Yes	No	A valid path or a blank

snapshotTombstonesTtlMinutes

1440

TTL for snapshot tombstones in minutes.

Yes

Non-negative LONG value

snapshotsPath

snapshots

The path where the snapshots are saved on the node. By default, {GRIDGAIN_HOME}/work/snapshots.

If an absolute path is specified, a subfolder with the node name is used: /absolute-path → /absolute-path/node-name
Relative paths are calculated from the node’s working directory: relative-path → {GRIDGAIN_HOME}/work/relative-path

Yes

A valid path or a blank

SQL Configuration

{
  "ignite" : {
    "sql" : {
      "planner" : {
        "estimatedNumberOfQueries" : 1024,
        "maxPlanningTime" : 15000
      },
      "statementMemoryQuota" : "100%"
    }
  }
}

Property	Default	Description	Changeable	Requires Restart	Acceptable Values
planner.estimatedNumberOfQueries	1024	The estimated number of unique queries that are planned to be executed in the cluster in a certain period of time. Used to optimize internal caches and processes. Optional.	Yes	Yes	0 - Integer.MAX_VALUE
planner.maxPlanningTime	15000	Query planning timeout in milliseconds. Plan optimization process stops when the timeout is reached. "0" means no timeout.	Yes	Yes	0 - Long.MAX_VALUE
statementMemoryQuota	100%	The amount of memory that can be used by a single SQL statement. A number with a dimension identifier: % - percentage of the node’s heap memory k - Kb m - Mb g - Gb "0" with any of he dimension identifiers turns the memory tracking off.	Yes	No	0-100% 0-9223372036854775807k/m/g

Property

Default

Description

Changeable

Requires Restart

Acceptable Values

planner.estimatedNumberOfQueries

1024

The estimated number of unique queries that are planned to be executed in the cluster in a certain period of time. Used to optimize internal caches and processes. Optional.

Yes

0 - Integer.MAX_VALUE

planner.maxPlanningTime

15000

Query planning timeout in milliseconds. Plan optimization process stops when the timeout is reached. "0" means no timeout.

Yes

0 - Long.MAX_VALUE

statementMemoryQuota

100%

The amount of memory that can be used by a single SQL statement. A number with a dimension identifier:

% - percentage of the node’s heap memory
k - Kb
m - Mb
g - Gb

"0" with any of he dimension identifiers turns the memory tracking off.

Yes

0-100%
0-9223372036854775807k/m/g

Storage Update Configuration

{
  "ignite" : {
    "storageUpdate" : {
      "batchByteLength" : 8192
    }
  }
}

Property	Default	Description	Changeable	Requires Restart	Acceptable Values
batchByteLength	8192	Batch length (in bytes) to be written into physical storage. Used to limit the size of an atomical Write.	Yes	No	1 - Integer.MAX_VALUE

Property

Default

Description

Changeable

Requires Restart

Acceptable Values

batchByteLength

8192

Batch length (in bytes) to be written into physical storage. Used to limit the size of an atomical Write.

Yes

1 - Integer.MAX_VALUE

Transactions Configuration

{
  "ignite" : {
    "transaction" : {
      "abandonedCheckTs" : 5000,
      "attemptsObtainLock" : 3,
      "implicitTransactionTimeout" : 3000,
      "rpcTimeout" : 60000,
      "txnResourceTtl" : 30000
    }
  }
}

Property Default Description Changeable Requires Restart Acceptable Values

Property	Default	Description	Changeable	Requires Restart	Acceptable Values
abandonedCheckTs	5000	The amount of time after which the transaction is considered abandoned.	Yes	Yes	0 - inf
attemptsObtainLock	3	The number of attempts to obtain a lock. After this number of attempts, a lock conflict exception is thrown on the transaction.	Yes	Yes	0 - inf
implicitTransactionTimeout	3000	Timeout for implicit transactions. Used for key-value table APIs invoked with `null` instead of an explicit transaction instance. Must be greater than `rpcTimeout`.	Yes	Yes	0 - inf
rpcTimeout	6000	Timeout for the transaction operation. Used for table API operations.	Yes	Yes	0 - inf
txnResourceTtl	30000	Time to live for transaction resources (transaction states, transaction nodes, open cursors, etc.). Should be higher than the lag to spread the actual transaction information across clusters. Otherwise, some transaction operations might take more time to find out the state from the persistence storage). Must be greater than `rpcTimeout`.	Yes	Yes	0 - inf

abandonedCheckTs

5000

The amount of time after which the transaction is considered abandoned.

Yes

0 - inf

attemptsObtainLock

The number of attempts to obtain a lock. After this number of attempts, a lock conflict exception is thrown on the transaction.

Yes

0 - inf

implicitTransactionTimeout

3000

Timeout for implicit transactions. Used for key-value table APIs invoked with null instead of an explicit transaction instance. Must be greater than rpcTimeout.

Yes

0 - inf

rpcTimeout

6000

Timeout for the transaction operation. Used for table API operations.

Yes

0 - inf

txnResourceTtl

30000

Time to live for transaction resources (transaction states, transaction nodes, open cursors, etc.). Should be higher than the lag to spread the actual transaction information across clusters. Otherwise, some transaction operations might take more time to find out the state from the persistence storage). Must be greater than rpcTimeout.

Yes

0 - inf

System Configuration

This section describes internal properties, which are used by a number of GridGain components. Although you can edit these properties in the same way you edit all others - using the node config update CLI command - we suggest that you discuss the proposed changes with the GridGain support team. The properties can apply to the cluster as a whole - see below - or to a specific node.

{
  "ignite" : {
    "system" : {
      "cmgPath" : "",
      "metastoragePath" : "",
      "partitionsBasePath" : "",
      "partitionsLogPath" : "",
      "properties":[]
    }
  }
}

Property Default Description Changeable Requires Restart Acceptable Values

Property	Default	Changeable	Requires Restart	Acceptable Values
system.cmgPath	The path the cluster management group information is stored to. By default, data is stored in `{GRIDGAIN_HOME}/work/cmg`.	Yes	Yes	Valid absolute path.
system.metastoragePath	The path the cluster meta information is stored to. By default, data is stored in `{GRIDGAIN_HOME}/work/metastorage`.	Yes	Yes	Valid absolute path.
system.partitionsBasePath	The path data partitions are saved to. By default, partitions are stored in `{GRIDGAIN_HOME}/work/partitions`.	Yes	Yes	Valid absolute path.
system.partitionsLogPath	The path RAFT log the partitions are stored at. By default, this log is stored in `{system.partitionsBasePath}/log`.	Yes	Yes	Valid absolute path.
system.properties	System properties used by the GridGain components.	Yes	Yes	An array of properties.

system.cmgPath

The path the cluster management group information is stored to. By default, data is stored in {GRIDGAIN_HOME}/work/cmg.

Yes

Valid absolute path.

system.metastoragePath

The path the cluster meta information is stored to. By default, data is stored in {GRIDGAIN_HOME}/work/metastorage.

Yes

Valid absolute path.

system.partitionsBasePath

The path data partitions are saved to. By default, partitions are stored in {GRIDGAIN_HOME}/work/partitions.

Yes

Valid absolute path.

system.partitionsLogPath

The path RAFT log the partitions are stored at. By default, this log is stored in {system.partitionsBasePath}/log.

Yes

Valid absolute path.

system.properties

System properties used by the GridGain components.

Yes

An array of properties.

© 2024 GridGain Systems, Inc. All Rights Reserved. Privacy Policy | Legal Notices. GridGain® is a registered trademark of GridGain Systems, Inc.
Apache, Apache Ignite, the Apache feather and the Apache Ignite logo are either registered trademarks or trademarks of The Apache Software Foundation.

Last updated on Nov 21, 2024