Node Configuration Parameters
Node configuration is individual for each node and is not shared across the whole cluster.
In GridGain 9, you can create and maintain configuration in either HOCON or JSON. The configuration file has a single root "node," called ignite
. All configuration sections are children, grandchildren, etc., of that node.
Checking Node Configuration
To get node configuration, use the CLI tool.
-
Start the CLI tool and connect to the node.
-
Run the
node config show
command.
The CLI tool will print the full node configuration. If you only need a part of the configuration, you can narrow down the search by providing the properties you need as the command argument, for example:
node config show ignite.clientConnector
Changing Node Configuration
Node configuration is changed from the CLI tool. To change the configuration:
-
Start the CLI tool and connect to the node. This becomes the "default" node for subsequent CL commands.
-
To update the default node’s configuration, run the
node config update
command and provide the update as the command argument, for example:node config update ignite.clientConnector.connectTimeout=10000
-
To update the configuration of a node other than the default one, run the
node config update
command with the target node explicitly specified. For example, for node namednode1
:node config update -n node1 ignite.nodeAttributes.nodeAttributes={clientConnector.attribute="10900"}
-
Restart the node to apply the configuration changes.
Exporting Node Configuration
If you need to export node configuration to a HOCON-formatted file, use the following command:
bin/gridgain9 node config show > node-config.conf
Configuration Parameters
Client Connector Configuration
See the Clients section for information on configuring client connector.
Compute Configuration
{
"ignite" : {
"compute" : {
"queueMaxSize" : 2147483647,
"statesLifetimeMillis" : 60000,
"threadPoolSize" : 10,
"threadPoolStopTimeoutMillis" : 10000
}
}
}
Property | Default | Description | Changeable | Requires Restart | Acceptable Values |
---|---|---|---|---|---|
queueMaxSize |
2147483647 |
Maximum number of compute tasks in queue. |
Yes |
Yes |
1 - Integer.MAX_VALUE |
statesLifetimeMillis |
60000 |
The lifetime of job states after the job finishes, in milliseconds. |
Yes |
Yes |
0 - inf |
threadPoolSize |
10 |
The number of threads available to compute jobs. |
Yes |
Yes |
1 - Integer.MAX_VALUE |
threadPoolStopTimeoutMillis |
10000 |
Job thread pool stop timeout, in milliseconds. |
Yes |
Yes |
1 - inf |
Critical Workers Configuration
{
"ignite" : {
"criticalWorkers" : {
"livenessCheckInterval" : 200,
"maxAllowedLag" : 500,
"nettyThreadsHeartbeatInterval" : 100
}
}
}
Property | Default | Description | Changeable | Requires Restart | Acceptable Values |
---|---|---|---|---|---|
livenessCheckInterval |
200 |
Interval between liveness checks (ms) performed by the critical worker infrastructure. |
Yes |
Yes |
1 - inf (not greater than half of maxAllowedLag) |
maxAllowedLag |
500 |
Maximum allowed delay from the last heartbeat to the current time (ms). If exceeded, the critical worker is considered to be blocked. |
Yes |
No |
1 - inf (should be at least twice livenessCheckInterval) |
nettyThreadsHeartbeatInterval |
100 |
Interval between heartbeats used to update the Netty threads' heartbeat timestamps (ms). |
Yes |
Yes |
1 - inf |
Code Deployment Configuration
{
"ignite" : {
"deployment" : {
"deploymentLocation" : "deployment"
}
}
}
Property | Default | Description | Changeable | Requires Restart | Acceptable Values |
---|---|---|---|---|---|
deploymentLocation |
deployment |
Relative path to folder in the working directory. All deployment units content will be stored there. |
Yes |
No |
A valid path |
Eviction Configuration
{
"ignite" : {
"eviction" : {
"checkFrequency" : 600000,
}
}
}
Property | Default | Description | Changeable | Requires Restart | Acceptable Values |
---|---|---|---|---|---|
checkInterval |
60000 |
How often the data is checked for eviction threshold, in milliseconds. |
Yes |
Yes |
1 - inf |
Expiration Configuration
{
"ignite" : {
"expiration" : {
"batchSize" : 1000,
"checkFrequency" : 600000,
"parallelismLevel" : 1
}
}
}
Property | Default | Description | Changeable | Requires Restart | Acceptable Values |
---|---|---|---|---|---|
batchSize |
1000 |
The number of items that can be expired at once. |
Yes |
Yes |
1 - inf |
checkInterval |
600000 |
How often the data is checked for expiration, in milliseconds. |
Yes |
Yes |
1 - inf |
parallelismLevel |
1 |
The number of threads used for data expiry. |
Yes |
Yes |
1 - Integer.MAX_VALUE |
Network Configuration
{
"ignite" : {
"network" : {
"fileTransfer" : {
"chunkSize" : 1048576,
"maxConcurrentRequests" : 4,
"responseTimeout" : 10000,
"threadPoolSize" : 8
},
"inbound" : {
"soBacklog" : 128,
"soKeepAlive" : true,
"soLinger" : 0,
"soReuseAddr" : true,
"tcpNoDelay" : true
},
"listenAddress" : "",
"membership" : {
"failurePingInterval" : 1000,
"membershipSyncInterval" : 30000,
"scaleCube" : {
"failurePingRequestMembers" : 3,
"gossipInterval" : 200,
"gossipRepeatMult" : 3,
"membershipSuspicionMultiplier" : 5,
"metadataTimeout" : 3000
}
},
"nodeFinder" : {
"netClusterNodes" : [ "localhost:3344" ],
"type" : "STATIC"
},
"outbound" : {
"soKeepAlive" : true,
"soLinger" : 0,
"tcpNoDelay" : true
},
"port" : 3344,
"shutdownQuietPeriod" : 0,
"shutdownTimeout" : 15000,
"ssl" : {
"ciphers" : "",
"clientAuth" : "none",
"enabled" : false,
"keyStore" : {
"password" : "********",
"path" : "",
"type" : "PKCS12"
},
"trustStore" : {
"password" : "********",
"path" : "",
"type" : "PKCS12"
}
}
}
}
}
Property | Default | Description | Changeable | Requires Restart | Acceptable Values |
---|---|---|---|---|---|
fileTransfer |
File transfer configuration. |
||||
fileTransfer.chunkSize |
1048576 |
Chunk size in bytes. |
Yes |
Yes |
1 - 1 - Integer.MAX_VALUE |
fileTransfer.maxConcurrentRequests |
4 |
Maximum number of concurrent requests. |
Yes |
Yes |
1 - Integer.MAX_VALUE |
fileTransfer.responseTimeout |
10000 |
Node response timeout during file transfer. |
Yes |
Yes |
0 - inf |
fileTransfer.threadPoolSize |
8 |
File sender thread pool size. |
Yes |
Yes |
1 - Integer.MAX_VALUE |
inbound |
Server socket configuration. See TCP documentation and socket documentation for more information. |
||||
inbound.soBacklog |
128 |
The size of the backlog. |
Yes |
Yes |
0 - Integer.MAX_VALUE |
inbound.soKeepAlive |
true |
Defines if the keep-alive packets are allowed. |
Yes |
Yes |
true, false |
inbound.soLinger |
0 |
Defines how long the closed socket should linger. |
Yes |
Yes |
0-65535 |
inbound.soReuseAddr |
true |
Defines if the address can be reused. |
Yes |
Yes |
true, false |
inbound.tcpNoDelay |
true |
Defines if the TCP no delay option is used. |
Yes |
Yes |
true, false |
listenAddress |
Address (IP or hostname) to listen on. Listens on all interfaces if empty. |
Yes |
Yes |
A valid address |
|
membership |
Node membership configuration. |
||||
membership.failurePingInterval |
1000 |
Failure detector ping interval. |
Yes |
Yes |
0 - inf |
membership.membershipSyncInterval |
30000 |
Periodic membership data synchronization interval. |
Yes |
Yes |
0 - inf |
membership.scaleCube |
ScaleCube-specific configuration. |
||||
scaleCube.failurePingRequestMembers |
3 |
Number of members that are randomly selected by a cluster node for an indirect ping request. |
Yes |
Yes |
1 - inf |
scaleCube.gossipInterval |
200 |
Gossip spreading interval. |
Yes |
Yes |
1 - inf |
scaleCube.gossipRepeatMult |
3 |
Gossip repeat multiplier. |
Yes |
Yes |
1 - inf |
scaleCube.membershipSuspicionMultiplier |
5 |
The multiplier that is used to calculate the timeout after which the node is considered dead. |
Yes |
Yes |
1 - inf |
scaleCube.metadataTimeout |
3000 |
The timeout on metadata update operation, in milliseconds. |
Yes |
Yes |
1 - inf |
nodeFinder |
Configuration for how the node finds other nodes in the cluster. |
||||
nodeFinder.netClusterNodes |
Addresses of all nodes in the cluster in the host:port format. |
Yes |
Yes |
Addresses in a valid format |
|
nodeFinder.type |
STATIC |
Node finder type. |
Yes |
Yes |
STATIC is the only currently supported type |
outbound |
Outbound request configuration. |
||||
outbound.soKeepAlive |
true |
Defines if the keep-alive packets are allowed. |
Yes |
Yes |
true, false |
outbound.soLinger |
0 |
Defines how long the closed socket should linger. |
Yes |
Yes |
0-65535 |
outbound.tcpNoDelay |
true |
Defines if the TCP no delay option is used. |
Yes |
Yes |
true, false |
port |
3344 |
Node port. |
Yes |
Yes |
A valid port number |
shutdownQuietPeriod |
0 |
The period during node shutdown when GridGain ensures that no tasks are submitted for the before the node shuts itself down. If a task is submitted during this period, it is guaranteed to be accepted. |
Yes |
No |
0 - inf |
shutdownTimeout |
15000 |
The maximum amount of time until the node is shut down regardless of if new network messages were submitted during shutdownQuietPeriod. |
Yes |
No |
0 - inf |
ssl.ciphers |
"" |
List of ciphers to enable, comma-separated. Empty for automatic cipher selection. |
Yes |
Yes |
TLS_AES_256_GCM_SHA384, etc. (standard cipher ids) |
ssl.clientAuth |
Whether the SSL client authentication is enabled and whether it is mandatory. |
Yes |
Yes |
non, optional, require |
|
ssl.enabled |
false |
Defines if SSL is enabled for the node. |
Yes |
Yes |
true, false |
ssl.keyStore |
SSL keystore configuration. |
||||
keyStore.password |
** |
Keystore password. |
Yes |
Yes |
A valid password |
keyStore.path |
Path to the keystore. |
Yes |
Yes |
A valid path |
|
keyStore.type |
PKCS12 |
Keystore type. |
Yes |
Yes |
PKCS12, JKS |
ssl.trustStore |
SSL trustsore configuration. |
||||
trustStore.password |
** |
Truststore password. |
Yes |
Yes |
A valid password |
trustStore.path |
Path to the truststore. |
Yes |
Yes |
A valid path |
|
trustStore.type |
PKCS12 |
Truststore type. |
Yes |
Yes |
PKCS12, JKS |
Node Attributes
{
"ignite" : {
"nodeAttributes" : {
"nodeAttributes" : [ ]
}
}
}
Property | Default | Description | Changeable | Requires Restart | Acceptable Values |
---|---|---|---|---|---|
nodeAttributes |
A list of node attributes used for dynamically distributing data only to those nodes that have the specified attribute values. |
Yes |
Yes |
A JSON-formatted list |
RAFT Configuration
{
"ignite" : {
"raft" : {
"fsync" : true,
"logPath" : "",
"logStripesCount" : 4,
"logYieldStrategy" : false,
"responseTimeout" : 3000,
"retryDelay" : 200,
"retryTimeout" : 10000,
"rpcInstallSnapshotTimeout" : 300000,
"stripes" : 10,
"volatileRaft" : {
"logStorage" : {
"name" : "unlimited"
}
}
}
}
}
Property | Default | Description | Changeable | Requires Restart | Acceptable Values |
---|---|---|---|---|---|
fsync |
true |
Defines if fsync will be used to transfer data in the cluster. |
Yes |
Yes |
true, false |
logPath |
Directory where the RAFT log is stored. |
Yes |
Yes |
A valid path |
|
logStripesCount |
4 |
Amount of stripes in disruptors of log manager |
Yes |
Yes |
1 - inf |
logYieldStrategy |
false |
If true, the non-blocking strategy is used in the Disruptor of log manager. |
Yes |
Yes |
true, false |
responseTimeout |
3000 |
Period for which the RAFT client will try to receive a response from a remote peer. |
Yes |
No |
0 - inf |
retryDelay |
200 |
Delay between re-sends of a failed request by the RAFT client. |
Yes |
No |
0 - inf |
retryTimeout |
10000 |
Period for which the RAFT client will try to receive a successful response from a remote peer. |
Yes |
No |
0 - inf |
installSnapshotTimeout |
300000 |
The maximum period allowed for transferring a RAFT snapshot to a recipient and installing it. |
Yes |
Yes |
1 - inf |
volatileRaft.logStorageBudget.name |
unlimited |
The name of the log storage budget used by the node. |
Yes |
No, but the new values are only applied to new partitions |
unlimited, entry-count |
REST Configuration
{
"ignite" : {
"rest" : {
"dualProtocol" : false,
"httpToHttpsRedirection" : false,
"port" : 10300,
"ssl" : {
"ciphers" : "",
"clientAuth" : "none",
"enabled" : false,
"keyStore" : {
"password" : "********",
"path" : "",
"type" : "PKCS12"
},
"port" : 10400,
"trustStore" : {
"password" : "********",
"path" : "",
"type" : "PKCS12"
}
}
}
}
}
Property | Default | Description | Changeable | Requires Restart | Acceptable Values |
---|---|---|---|---|---|
dualProtocol |
false |
Defines if both HTTP and HTTPS protocols are used by the endpoint. |
Yes |
Yes |
true, false |
httpToHttpsRedirection |
false |
Defines if requests to HTTP endpoint will be redirected to HTTPS. |
Yes |
Yes |
true, false |
port |
10300 |
The port of the node’s REST endpoint. |
Yes |
Yes |
A valid port |
ssl.ciphers |
Explicitly set node SSL cipher. |
Yes |
Yes |
||
ssl.clientAuth |
Client authorization used by the node, if any. |
Yes |
Yes |
non, optional, require |
|
ssl.enabled |
false |
Defines if SSL is enabled for the node. |
Yes |
Yes |
true, false |
ssl.keyStore |
SSL keystore configuration. |
||||
keyStore.password |
** |
Keystore password. |
Yes |
Yes |
A valid password |
keyStore.path |
Path to the keystore. |
Yes |
Yes |
A valid path |
|
keyStore.type |
PKCS12 |
Keystore type. |
Yes |
Yes |
PKCS12, JKS |
ssl.port |
10400 |
Port used for SSL connections. |
Yes |
Yes |
A valid port |
ssl.trustStore |
SSL trustsore configuration. |
||||
trustStore.password |
** |
Truststore password. |
Yes |
Yes |
A valid password |
trustStore.path |
Path to the truststore. |
Yes |
Yes |
A valid path |
|
trustStore.type |
PKCS12 |
Truststore type. |
Yes |
Yes |
PKCS12, JKS |
Snapshots Configuration
{
"ignite" : {
"snapshot" : {
"threadPoolSize" : 20
}
}
}
Property | Default | Description | Changeable | Requires Restart | Acceptable Values |
---|---|---|---|---|---|
threadPoolSize |
20 |
Number of threads used by GridGain for IO operations when creating or restoring snapshots. |
Yes |
Yes |
1 - Integer.MAX_VALUE |
SQL Configuration
{
"ignite" : {
"sql" : {
"execution" : {
"threadCount" : 4
},
"nodeMemoryQuota" : "60%",
"planner" : {
"threadCount" : 4
}
}
}
}
Property | Default | Description | Changeable | Requires Restart | Acceptable Values |
---|---|---|---|---|---|
execution.threadCount |
4 |
Number of threads for query execution. |
Yes |
Yes |
1 - Integer.MAX_VALUE |
nodeMemoryQuota |
60% |
Node-wide limit for memory to be used for SQL queries. A number with a dimension identifier:
"0" with any of he dimension identifiers turns the memory tracking off. |
Yes |
No |
|
planner.threadCount |
4 |
Number of threads for query planning. |
Yes |
Yes |
1 - Integer.MAX_VALUE |
Storage Configuration
GridGain Persistence is designed to provide a quick and responsive persistent storage. When using the persistent storage, GridGain stores all the data on disk, and loads as much data as it can into RAM for processing. When persistence is enabled, GridGain stores each partition in a separate file on disk. In addition to data partitions, GridGain stores indexes and metadata.
Each GridGain storage engine can have several storage profiles.
Checkpointing is the process of copying dirty pages from RAM to partition files on disk. A dirty page is a page that was updated in RAM but was not written to the respective partition file. After a checkpoint is created, all changes are persisted to disk and will be available if the node crashes and is restarted. Checkpointing is designed to ensure durability of data and recovery in case of a node failure. This process helps you utilize disk space frugally by keeping pages in the most up-to-date state on disk.
{
"ignite" : {
"storage" : {
"engines" : {
"aimem" : {
"pageSize" : 16384
},
"aipersist" : {
"checkpoint" : {
"checkpointDelayMillis" : 200,
"checkpointThreads" : 4,
"compactionThreads" : 4,
"frequency" : 180000,
"frequencyDeviation" : 40,
"logReadLockThresholdTimeout" : 0,
"readLockTimeout" : 10000,
"useAsyncFileIoFactory" : true
},
"pageSize" : 16384
},
"rocksdb" : {
"flushDelayMillis" : 100
},
"columnar" : {
"compressingConfiguration" : {
"enableLz4Compression" : true
"enableOrochCompression" : true
}
"memtableConfiguration" : {
"memtableBufferCapacity" : 1048576
"memtableBufferPoolCapacity" : 2147483648
"memtableMaxEntries" : 262144
}
"mergeTreeConfiguration" : {
"mergeTreeColumnBlockSizeHint" : 8192
"mergeTreeFanout" : 4
"mergeTreeFirstLevelSize" : 262144
"mergeTreeSegmentKeyIndexGranularity" : 512
}
"threadPoolConfiguration" : {
"threadPoolThreadCount" : 10
}
}
},
"profiles" : [ {
"engine" : "aipersist",
"name" : "default",
"replacementMode" : "CLOCK",
"size" : 268435456
},
{
"engine" : "aimem",
"name" : "default_aimem",
"emptyPagesPoolSize" : 100,
"eviction" : {
"batchSize" : 200,
"interval" : 60000,
"lwmThreshold" : 1000,
"lwmUpdateInterval" : 60000,
"mode" : "DISABLED",
"threshold" : "90%"
},
"initSize" : 268435456,
"maxSize" : 268435456
},
{
"engine" : "rocksdb",
"name" : "default_rocksdb",
"size" : 268435456,
"writeBufferSize" : 67108864
} ]
}
}
}
Property | Default | Description | Changeable | Requires Restart | Acceptable Values |
---|---|---|---|---|---|
engines.aimem |
Aimem configuration. |
||||
aimem.pageSize |
16384 |
The size of pages in the storage, in bytes. |
Yes |
Yes |
1024-16384 |
engines.aipersist |
Aipersist configuration. |
||||
aipersist.checkpoint.checkpointDelayMillis |
200 |
Delay before staring a checkpoint after receiving the command. |
Yes |
No |
0 - inf |
aipersist.checkpoint.checkpointThreads |
4 |
Number of CPU threads dedicated to checkpointing. |
Yes |
Yes |
1 - inf |
aipersist.checkpoint.compactionThreads |
4 |
Number of CPU threads dedicated to data compaction. |
Yes |
Yes |
1 - inf |
aipersist.checkpoint.interval |
180000 |
Interval between checkpoints in milliseconds. |
Yes |
No |
0 - inf |
aipersist.checkpoint.intervalDeviation |
40 |
Jitter that will be added or subtracted from time period till next scheduled checkpoint (percentage). |
Yes |
No |
0-100 |
aipersist.checkpoint.logReadLockThresholdTimeout |
0 |
Threshold for logging long read locks, in milliseconds. |
Yes |
Yes |
0 - inf |
aipersist.checkpoint.readLockTimeout |
10000 |
Timeout for checkpoint read lock acquisition, in milliseconds. |
Yes |
Yes |
0 - inf |
aipersist.checkpoint.useAsyncFileIoFactory |
true |
If GridGain uses asynchronous file I/O operations provider. |
Yes |
Yes |
true, false |
aipersist.pageSize |
16384 |
The size of pages in the storage, in bytes. |
No |
N/A |
1024-16384 |
engines.rocksdb |
Rocksdb configuration. |
||||
rocksdb.flushDelayMillis |
100 |
Delay before executing a flush triggered by RAFT. |
Yes |
Refreshed on engine registration |
0 - inf |
profiles |
The list of available storage profiles. |
||||
engine |
The storage engine. |
No |
N/A |
aimem, aipersist, rocksdb |
|
name |
User-defined profile name. |
No |
N/A |
A valid name |
|
replacementMode |
CLOCK |
Sets the page replacement algorithm. |
Yes |
Yes |
CLOCK, RANDOM_LRU, SEGMENTED_LRU |
size |
256Mb |
Memory (RAM) region size. |
Yes |
Yes |
Min 256Mb, max defined by the addressable memory limit of the OS |
aipersist.size |
268435456 |
Memory (offheap) region size. |
Yes |
Yes |
Min 268435456, max defined by the addressable memory limit of the OS |
aipersist.replacementMode |
CLOCK |
Sets the page replacement algorithm. |
Yes |
Yes |
CLOCK, RANDOM_LRU, SEGMENTED_LRU |
aimem.initSize |
268435456 |
Initial memory region size in bytes, when the used memory size exceeds this value, new chunks of memory will be allocated. |
Yes |
Yes |
Min 256Mb, max defined by the addressable memory limit of the OS |
aimem.maxSize |
268435456 |
Maximum memory region size in bytes. |
Yes |
Yes |
Min 256Mb, max defined by the addressable memory limit of the OS |
aimem.eviction.mode |
DISABLED |
Eviction mode. |
Yes |
No |
DISABLED, HISTORY_ONLY, RANDOM |
aimem.eviction.threshold |
90% |
Threshold for eviction initiation. A number with a dimension identifier:
For instance, "90%" means that the page memory starts eviction only after 90% of the data region is occupied. |
Yes |
No |
|
aimem.eviction.lwmUpdateInterval |
60000 |
Frequency of the low watermark update in milliseconds. |
Yes |
No |
1 - inf |
aimem.eviction.interval |
60000 |
Interval between the data eviction iterations. |
Yes |
No |
1 - inf |
aimem.eviction.lwmThreshold |
1000 |
If the low watermark is less than evictionLwmThreshold from the current timestamp, the row eviction is triggered. |
Yes |
No |
0 - inf |
aimem.eviction.batchSize |
60000 |
Eviction batch size in rows. |
Yes |
No |
1 - inf |
rocksdb.size |
268435456 |
Size of the rocksdb offheap cache. |
Yes |
Yes |
Min 0, max defined by the addressable memory limit of the OS |
rocksdb.writeBufferSize |
67108864 |
Size of rocksdb write buffer. |
Yes |
Yes |
Min 1, max defined by the addressable memory limit of the OS |
System Configuration
This section describes internal properties, which are used by a number of GridGain components. Although you can edit these properties in the same way you edit all others - using the node config update
CLI command - we suggest that you discuss the proposed changes with the GridGain support team. The properties can apply to a specific node - see below - or to the cluster as a whole.
{
"ignite" : {
"system" : {
"cmgPath" : "",
"metastoragePath" : "",
"partitionsBasePath" : "",
"partitionsLogPath" : "",
"properties":[]
}
}
}
Property | Default | Description | Changeable | Requires Restart | Acceptable Values |
---|---|---|---|---|---|
system.cmgPath |
The path the cluster management group information is stored to. Only applicable if the node is part of CMG. By default, data is stored in |
Yes |
Yes |
Valid absolute path. |
|
system.metastoragePath |
The path the cluster meta information is stored to. Only applicable if the node is part of the metastorage group. By default, data is stored in |
Yes |
Yes |
Valid absolute path. |
|
system.partitionsBasePath |
The path data partitions are saved to on the node. By default, partitions are stored in |
Yes |
Yes |
Valid absolute path. |
|
system.partitionsLogPath |
The path RAFT log the partitions are stored at. By default, this log is stored in |
Yes |
Yes |
Valid absolute path. |
|
system.properties |
System properties used by the GridGain components. |
Yes |
Yes |
An array of properties. |
© 2024 GridGain Systems, Inc. All Rights Reserved. Privacy Policy | Legal Notices. GridGain® is a registered trademark of GridGain Systems, Inc.
Apache, Apache Ignite, the Apache feather and the Apache Ignite logo are either registered trademarks or trademarks of The Apache Software Foundation.