GridGain Developers Hub

Node Configuration Parameters

Node configuration is individual for each node and is not shared across the whole cluster.

In GridGain 9, you can create and maintain configuration in either HOCON or JSON. The configuration file has a single root "node," called ignite. All configuration sections are children, grandchildren, etc., of that node.

Checking Node Configuration

To get node configuration, use the CLI tool.

  • Start the CLI tool and connect to the node.

  • Run the node config show command.

The CLI tool will print the full node configuration. If you only need a part of the configuration, you can narrow down the search by providing the properties you need as the command argument, for example:

node config show ignite.clientConnector

Changing Node Configuration

Node configuration is changed from the CLI tool. To change the configuration:

  • Start the CLI tool and connect to the node. This becomes the "default" node for subsequent CL commands.

  • To update the default node’s configuration, run the node config update command and provide the update as the command argument, for example:

    node config update ignite.clientConnector.connectTimeout=10000
  • To update the configuration of a node other than the default one, run the node config update command with the target node explicitly specified. For example, for node named node1:

    node config update -n node1 ignite.nodeAttributes.nodeAttributes={clientConnector.attribute="10900"}
  • Restart the node to apply the configuration changes.

Exporting Node Configuration

If you need to export node configuration to a HOCON-formatted file, use the following command:

bin/gridgain9 node config show > node-config.conf

Configuration Parameters

Client Connector Configuration

See the Clients section for information on configuring client connector.

Compute Configuration

{
  "ignite" : {
    "compute" : {
      "queueMaxSize" : 2147483647,
      "statesLifetimeMillis" : 60000,
      "threadPoolSize" : 10,
      "threadPoolStopTimeoutMillis" : 10000
    }
  }
}
Property Default Description Changeable Requires Restart Acceptable Values

queueMaxSize

2147483647

Maximum number of compute tasks in queue.

Yes

Yes

1 - Integer.MAX_VALUE

statesLifetimeMillis

60000

The lifetime of job states after the job finishes, in milliseconds.

Yes

Yes

0 - inf

threadPoolSize

10

The number of threads available to compute jobs.

Yes

Yes

1 - Integer.MAX_VALUE

threadPoolStopTimeoutMillis

10000

Job thread pool stop timeout, in milliseconds.

Yes

Yes

1 - inf

Critical Workers Configuration

{
  "ignite" : {
    "criticalWorkers" : {
      "livenessCheckInterval" : 200,
      "maxAllowedLag" : 500,
      "nettyThreadsHeartbeatInterval" : 100
    }
  }
}
Property Default Description Changeable Requires Restart Acceptable Values

livenessCheckInterval

200

Interval between liveness checks (ms) performed by the critical worker infrastructure.

Yes

Yes

1 - inf (not greater than half of maxAllowedLag)

maxAllowedLag

500

Maximum allowed delay from the last heartbeat to the current time (ms). If exceeded, the critical worker is considered to be blocked.

Yes

No

1 - inf (should be at least twice livenessCheckInterval)

nettyThreadsHeartbeatInterval

100

Interval between heartbeats used to update the Netty threads' heartbeat timestamps (ms).

Yes

Yes

1 - inf

Code Deployment Configuration

{
  "ignite" : {
    "deployment" : {
      "deploymentLocation" : "deployment"
    }
  }
}
Property Default Description Changeable Requires Restart Acceptable Values

deploymentLocation

deployment

Relative path to folder in the working directory. All deployment units content will be stored there.

Yes

No

A valid path

Eviction Configuration

{
  "ignite" : {
    "eviction" : {
      "checkFrequency" : 600000,
    }
  }
}
Property Default Description Changeable Requires Restart Acceptable Values

checkInterval

60000

How often the data is checked for eviction threshold, in milliseconds.

Yes

Yes

1 - inf

Expiration Configuration

{
  "ignite" : {
    "expiration" : {
      "batchSize" : 1000,
      "checkFrequency" : 600000,
      "parallelismLevel" : 1
    }
  }
}
Property Default Description Changeable Requires Restart Acceptable Values

batchSize

1000

The number of items that can be expired at once.

Yes

Yes

1 - inf

checkInterval

600000

How often the data is checked for expiration, in milliseconds.

Yes

Yes

1 - inf

parallelismLevel

1

The number of threads used for data expiry.

Yes

Yes

1 - 1 - Integer.MAX_VALUE

Network Configuration

{
  "ignite" : {
    "network" : {
      "fileTransfer" : {
        "chunkSize" : 1048576,
        "maxConcurrentRequests" : 4,
        "responseTimeout" : 10000,
        "threadPoolSize" : 8
      },
      "inbound" : {
        "soBacklog" : 128,
        "soKeepAlive" : true,
        "soLinger" : 0,
        "soReuseAddr" : true,
        "tcpNoDelay" : true
      },
      "listenAddress" : "",
      "membership" : {
        "failurePingInterval" : 1000,
        "membershipSyncInterval" : 30000,
        "scaleCube" : {
          "failurePingRequestMembers" : 3,
          "gossipInterval" : 200,
          "gossipRepeatMult" : 3,
          "membershipSuspicionMultiplier" : 5,
          "metadataTimeout" : 3000
        }
      },
      "nodeFinder" : {
        "netClusterNodes" : [ "localhost:3344" ],
        "type" : "STATIC"
      },
      "outbound" : {
        "soKeepAlive" : true,
        "soLinger" : 0,
        "tcpNoDelay" : true
      },
      "port" : 3344,
      "shutdownQuietPeriod" : 0,
      "shutdownTimeout" : 15000,
      "ssl" : {
        "ciphers" : "",
        "clientAuth" : "none",
        "enabled" : false,
        "keyStore" : {
          "password" : "********",
          "path" : "",
          "type" : "PKCS12"
        },
        "trustStore" : {
          "password" : "********",
          "path" : "",
          "type" : "PKCS12"
        }
      }
    }
  }
}
Property Default Description Changeable Requires Restart Acceptable Values

fileTransfer

File transfer configuration.

fileTransfer.chunkSize

1048576

Chunk size in bytes.

Yes

Yes

1 - 1 - Integer.MAX_VALUE

fileTransfer.maxConcurrentRequests

4

Maximum number of concurrent requests.

Yes

Yes

1 - Integer.MAX_VALUE

fileTransfer.responseTimeout

10000

Node response timeout during file transfer.

Yes

Yes

0 - inf

fileTransfer.threadPoolSize

8

File sender thread pool size.

Yes

Yes

1 - Integer.MAX_VALUE

inbound

Server socket configuration. See TCP documentation and socket documentation for more information.

inbound.soBacklog

128

The size of the backlog.

Yes

Yes

0 - Integer.MAX_VALUE

inbound.soKeepAlive

true

Defines if the keep-alive packets are allowed.

Yes

Yes

true, false

inbound.soLinger

0

Defines how long the closed socket should linger.

Yes

Yes

0-65535

inbound.soReuseAddr

true

Defines if the address can be reused.

Yes

Yes

true, false

inbound.tcpNoDelay

true

Defines if the TCP no delay option is used.

Yes

Yes

true, false

listenAddress

Address (IP or hostname) to listen on. Listens on all interfaces if empty.

Yes

Yes

A valid address

membership

Node membership configuration.

membership.failurePingInterval

1000

Failure detector ping interval.

Yes

Yes

0 - inf

membership.membershipSyncInterval

30000

Periodic membership data synchronization interval.

Yes

Yes

0 - inf

membership.scaleCube

ScaleCube-specific configuration.

scaleCube.failurePingRequestMembers

3

Number of members that are randomly selected by a cluster node for an indirect ping request.

Yes

Yes

1 - inf

scaleCube.gossipInterval

200

Gossip spreading interval.

Yes

Yes

1 - inf

scaleCube.gossipRepeatMult

3

Gossip repeat multiplier.

Yes

Yes

1 - inf

scaleCube.membershipSuspicionMultiplier

5

The multiplier that is used to calculate the timeout after which the node is considered dead.

Yes

Yes

1 - inf

scaleCube.metadataTimeout

3000

The timeout on metadata update operation, in milliseconds.

Yes

Yes

1 - inf

nodeFinder

Configuration for how the node finds other nodes in the cluster.

nodeFinder.netClusterNodes

Addresses of all nodes in the cluster in the host:port format.

Yes

Yes

Addresses in a valid format

nodeFinder.type

STATIC

Node finder type.

Yes

Yes

STATIC is the only currently supported type

outbound

Outbound request configuration.

outbound.soKeepAlive

true

Defines if the keep-alive packets are allowed.

Yes

Yes

true, false

outbound.soLinger

0

Defines how long the closed socket should linger.

Yes

Yes

0-65535

outbound.tcpNoDelay

true

Defines if the TCP no delay option is used.

Yes

Yes

true, false

port

3344

Node port.

Yes

Yes

A valid port number

shutdownQuietPeriod

0

The period during node shutdown when GridGain ensures that no tasks are submitted for the before the node shuts itself down. If a task is submitted during this period, it is guaranteed to be accepted.

Yes

No

0 - inf

shutdownTimeout

15000

The maximum amount of time until the node is shut down regardless of if new network messages were submitted during shutdownQuietPeriod.

Yes

No

0 - inf

ssl.ciphers

""

List of ciphers to enable, comma-separated. Empty for automatic cipher selection.

Yes

Yes

TLS_AES_256_GCM_SHA384, etc. (standard cipher ids)

ssl.clientAuth

Whether the SSL client authentication is enabled and whether it is mandatory.

Yes

Yes

non, optional, require

ssl.enabled

false

Defines if SSL is enabled for the node.

Yes

Yes

true, false

ssl.keyStore

SSL keystore configuration.

keyStore.password

**

Keystore password.

Yes

Yes

A valid password

keyStore.path

Path to the keystore.

Yes

Yes

A valid path

keyStore.type

PKCS12

Keystore type.

Yes

Yes

PKCS12, JKS

ssl.trustStore

SSL trustsore configuration.

trustStore.password

**

Truststore password.

Yes

Yes

A valid password

trustStore.path

Path to the truststore.

Yes

Yes

A valid path

trustStore.type

PKCS12

Truststore type.

Yes

Yes

PKCS12, JKS

Node Attributes

{
  "ignite" : {
    "nodeAttributes" : {
      "nodeAttributes" : [ ]
    }
  }
}
Property Default Description Changeable Requires Restart Acceptable Values

nodeAttributes

A list of node attributes used for dynamically distributing data only to those nodes that have the specified attribute values.

Yes

Yes

A JSON-formatted list

RAFT Configuration

{
  "ignite" : {
    "raft" : {
      "fsync" : true,
      "logPath" : "",
      "logStripesCount" : 4,
      "logYieldStrategy" : false,
      "responseTimeout" : 3000,
      "retryDelay" : 200,
      "retryTimeout" : 10000,
      "rpcInstallSnapshotTimeout" : 300000,
      "stripes" : 10,
      "volatileRaft" : {
        "logStorage" : {
          "name" : "unlimited"
        }
      }
    }
  }
}
Property Default Description Changeable Requires Restart Acceptable Values

fsync

true

Defines if fsync will be used to transfer data in the cluster.

Yes

Yes

true, false

logPath

Directory where the RAFT log is stored.

Yes

Yes

A valid path

logStripesCount

4

Amount of stripes in disruptors of log manager

Yes

Yes

1 - inf

logYieldStrategy

false

If true, the non-blocking strategy is used in the Disruptor of log manager.

Yes

Yes

true, false

responseTimeout

3000

Period for which the RAFT client will try to receive a response from a remote peer.

Yes

No

0 - inf

retryDelay

200

Delay between re-sends of a failed request by the RAFT client.

Yes

No

0 - inf

retryTimeout

10000

Period for which the RAFT client will try to receive a successful response from a remote peer.

Yes

No

0 - inf

installSnapshotTimeout

300000

The maximum period allowed for transferring a RAFT snapshot to a recipient and installing it.

Yes

Yes

1 - inf

volatileRaft.logStorageBudget.name

unlimited

The name of the log storage budget used by the node.

Yes

No, but the new values are only applied to new partitions

unlimited, entry-count

REST Configuration

{
  "ignite" : {
    "rest" : {
      "dualProtocol" : false,
      "httpToHttpsRedirection" : false,
      "port" : 10300,
      "ssl" : {
        "ciphers" : "",
        "clientAuth" : "none",
        "enabled" : false,
        "keyStore" : {
          "password" : "********",
          "path" : "",
          "type" : "PKCS12"
        },
        "port" : 10400,
        "trustStore" : {
          "password" : "********",
          "path" : "",
          "type" : "PKCS12"
        }
      }
    }
  }
}
Property Default Description Changeable Requires Restart Acceptable Values

dualProtocol

false

Defines if both HTTP and HTTPS protocols are used by the endpoint.

Yes

Yes

true, false

httpToHttpsRedirection

false

Defines if requests to HTTP endpoint will be redirected to HTTPS.

Yes

Yes

true, false

port

10300

The port of the node’s REST endpoint.

Yes

Yes

A valid port

ssl.ciphers

Explicitly set node SSL cipher.

Yes

Yes

See acceptable values

ssl.clientAuth

Client authorization used by the node, if any.

Yes

Yes

non, optional, require

ssl.enabled

false

Defines if SSL is enabled for the node.

Yes

Yes

true, false

ssl.keyStore

SSL keystore configuration.

keyStore.password

**

Keystore password.

Yes

Yes

A valid password

keyStore.path

Path to the keystore.

Yes

Yes

A valid path

keyStore.type

PKCS12

Keystore type.

Yes

Yes

PKCS12, JKS

ssl.port

10400

Port used for SSL connections.

Yes

Yes

A valid port

ssl.trustStore

SSL trustsore configuration.

trustStore.password

**

Truststore password.

Yes

Yes

A valid password

trustStore.path

Path to the truststore.

Yes

Yes

A valid path

trustStore.type

PKCS12

Truststore type.

Yes

Yes

PKCS12, JKS

SQL Configuration

{
  "ignite" : {
    "sql" : {
      "execution" : {
        "threadCount" : 4
      },
      "nodeMemoryQuota" : "60%",
      "planner" : {
        "threadCount" : 4
      }
    }
  }
}
Property Default Description Changeable Requires Restart Acceptable Values

execution.threadCount

4

Number of threads for query execution.

Yes

Yes

1 - Integer.MAX_VALUE

nodeMemoryQuota

60%

Node-wide limit for memory to be used for SQL queries. A number with a dimension identifier:

  • % - percentage of the node’s heap memory

  • k - Kb

  • m - Mb

  • g - Gb

"0" with any of he dimension identifiers turns the memory tracking off.

Yes

No

  • 0-100%

  • 0-9223372036854775807k/m/g

planner.threadCount

4

Number of threads for query planning.

Yes

Yes

1 - Integer.MAX_VALUE

Storage Configuration

GridGain Persistence is designed to provide a quick and responsive persistent storage. When using the persistent storage, GridGain stores all the data on disk, and loads as much data as it can into RAM for processing. When persistence is enabled, GridGain stores each partition in a separate file on disk. In addition to data partitions, GridGain stores indexes and metadata.

Each GridGain storage engine can have several storage profiles.

Checkpointing is the process of copying dirty pages from RAM to partition files on disk. A dirty page is a page that was updated in RAM but was not written to the respective partition file. After a checkpoint is created, all changes are persisted to disk and will be available if the node crashes and is restarted. Checkpointing is designed to ensure durability of data and recovery in case of a node failure. This process helps you utilize disk space frugally by keeping pages in the most up-to-date state on disk.

{
  "ignite" : {
    "storage" : {
      "engines" : {
        "aimem" : {
          "pageSize" : 16384
        },
        "aipersist" : {
          "checkpoint" : {
            "checkpointDelayMillis" : 200,
            "checkpointThreads" : 4,
            "compactionThreads" : 4,
            "frequency" : 180000,
            "frequencyDeviation" : 40,
            "logReadLockThresholdTimeout" : 0,
            "readLockTimeout" : 10000,
            "useAsyncFileIoFactory" : true
          },
          "pageSize" : 16384
        },
        "rocksdb" : {
          "flushDelayMillis" : 100
        },
        "columnar" : {
            "compressingConfiguration" : {
                "enableLz4Compression" : true
                "enableOrochCompression" : true
            }
            "memtableConfiguration" : {
                "memtableBufferCapacity" : 1048576
                "memtableBufferPoolCapacity" : 2147483648
                "memtableMaxEntries" : 262144
            }
            "mergeTreeConfiguration" : {
                "mergeTreeColumnBlockSizeHint" : 8192
                "mergeTreeFanout" : 4
                "mergeTreeFirstLevelSize" : 262144
                "mergeTreeSegmentKeyIndexGranularity" : 512
            }
            "threadPoolConfiguration" : {
                "threadPoolThreadCount" : 10
            }
        }
      },
      "profiles" : [ {
        "engine" : "aipersist",
        "name" : "default",
        "replacementMode" : "CLOCK",
        "size" : 268435456
      },
      {
        "engine" : "aimem",
        "name" : "default_aimem",
        "emptyPagesPoolSize" : 100,
        "eviction" : {
          "batchSize" : 200,
          "interval" : 60000,
          "lwmThreshold" : 1000,
          "lwmUpdateInterval" : 60000,
          "mode" : "DISABLED",
          "threshold" : "90%"
        },
        "initSize" : 268435456,
        "maxSize" : 268435456
      },
      {
        "engine" : "rocksdb",
        "name" : "default_rocksdb",
        "size" : 268435456,
        "writeBufferSize" : 67108864
      } ]
    }
  }
}
Property Default Description Changeable Requires Restart Acceptable Values

engines.aimem

Aimem configuration.

aimem.pageSize

16384

The size of pages in the storage, in bytes.

Yes

Yes

1024-16384

engines.aipersist

Aipersist configuration.

aipersist.checkpoint.checkpointDelayMillis

200

Delay before staring a checkpoint after receiving the command.

Yes

No

0 - inf

aipersist.checkpoint.checkpointThreads

4

Number of CPU threads dedicated to checkpointing.

Yes

Yes

1 - inf

aipersist.checkpoint.compactionThreads

4

Number of CPU threads dedicated to data compaction.

Yes

Yes

1 - inf

aipersist.checkpoint.interval

180000

Interval between checkpoints in milliseconds.

Yes

No

0 - inf

aipersist.checkpoint.intervalDeviation

40

Jitter that will be added or subtracted from time period till next scheduled checkpoint (percentage).

Yes

No

0-100

aipersist.checkpoint.logReadLockThresholdTimeout

0

Threshold for logging long read locks, in milliseconds.

Yes

Yes

0 - inf

aipersist.checkpoint.readLockTimeout

10000

Timeout for checkpoint read lock acquisition, in milliseconds.

Yes

Yes

0 - inf

aipersist.checkpoint.useAsyncFileIoFactory

true

If GridGain uses asynchronous file I/O operations provider.

Yes

Yes

true, false

aipersist.pageSize

16384

The size of pages in the storage, in bytes.

No

N/A

1024-16384

engines.rocksdb

Rocksdb configuration.

rocksdb.flushDelayMillis

100

Delay before executing a flush triggered by RAFT.

Yes

Refreshed on engine registration

0 - inf

profiles

The list of available storage profiles.

engine

The storage engine.

No

N/A

aimem, aipersist, rocksdb

name

User-defined profile name.

No

N/A

A valid name

replacementMode

CLOCK

Sets the page replacement algorithm.

Yes

Yes

CLOCK, RANDOM_LRU, SEGMENTED_LRU

size

256Mb

Memory (RAM) region size.

Yes

Yes

Min 256Mb, max defined by the addressable memory limit of the OS

aipersist.size

268435456

Memory (offheap) region size.

Yes

Yes

Min 268435456, max defined by the addressable memory limit of the OS

aipersist.replacementMode

CLOCK

Sets the page replacement algorithm.

Yes

Yes

CLOCK, RANDOM_LRU, SEGMENTED_LRU

aimem.initSize

268435456

Initial memory region size in bytes, when the used memory size exceeds this value, new chunks of memory will be allocated.

Yes

Yes

Min 256Mb, max defined by the addressable memory limit of the OS

aimem.maxSize

268435456

Maximum memory region size in bytes.

Yes

Yes

Min 256Mb, max defined by the addressable memory limit of the OS

aimem.eviction.mode

DISABLED

Eviction mode.

Yes

No

DISABLED, HISTORY_ONLY, RANDOM

aimem.eviction.threshold

90%

Threshold for eviction initiation. A number with a dimension identifier:

  • % - percentage of aimem.maxSize

  • k - Kb

  • m - Mb

  • g - Gb

For instance, "90%" means that the page memory starts eviction only after 90% of the data region is occupied.

Yes

No

  • 0-100%

  • 0-9223372036854775807k/m/g

aimem.eviction.lwmUpdateInterval

60000

Frequency of the low watermark update in milliseconds.

Yes

No

1 - inf

aimem.eviction.interval

60000

Interval between the data eviction iterations.

Yes

No

1 - inf

aimem.eviction.lwmThreshold

1000

If the low watermark is less than evictionLwmThreshold from the current timestamp, the row eviction is triggered.

Yes

No

0 - inf

aimem.eviction.batchSize

60000

Eviction batch size in rows.

Yes

No

1 - inf

rocksdb.size

268435456

Size of the rocksdb offheap cache.

Yes

Yes

Min 0, max defined by the addressable memory limit of the OS

rocksdb.writeBufferSize

67108864

Size of rocksdb write buffer.

Yes

Yes

Min 1, max defined by the addressable memory limit of the OS

System Configuration

This section describes internal properties, which are used by a number of GridGain components. Although you can edit these properties in the same way you edit all others - using the node config update CLI command - we suggest that you discuss the proposed changes with the GridGain support team. The properties can apply to a specific node - see below - or to the cluster as a whole.

{
  "ignite" : {
    "system" : {
      "cmgPath" : "",
      "metastoragePath" : "",
      "partitionsBasePath" : "",
      "partitionsLogPath" : "",
      "properties":[]
    }
  }
}
Property Default Description Changeable Requires Restart Acceptable Values

system.cmgPath

The path the cluster management group information is stored to. Only applicable if the node is part of CMG. By default, data is stored in {GRIDGAIN_HOME}/work/cmg. It is recommended to only change this path on an empty node.

Yes

Yes

Valid absolute path.

system.metastoragePath

The path the cluster meta information is stored to. Only applicable if the node is part of the metastorage group. By default, data is stored in {GRIDGAIN_HOME}/work/metastorage. It is recommended to only change this path on an empty node.

Yes

Yes

Valid absolute path.

system.partitionsBasePath

The path data partitions are saved to on the node. By default, partitions are stored in {GRIDGAIN_HOME}/work/partitions. It is recommended to only change this path on an empty node.

Yes

Yes

Valid absolute path.

system.partitionsLogPath

The path RAFT log the partitions are stored at. By default, this log is stored in {system.partitionsBasePath}/log. It is recommended to only change this path on an empty node.

Yes

Yes

Valid absolute path.

system.properties

System properties used by the GridGain components.

Yes

Yes

An array of properties.