GridGain Developers Hub

Control Script

GridGain provides a command line script — control.sh|bat — that you can use to monitor and control your clusters. The script is located under the /bin/ folder of the installation directory.

You can define the control.sh|bat log directory via an environment variable:

$ CONTROL_JVM_OPTS=-Djava.util.logging.config.file=<PATH_TO_CONFIG> ./control.sh
$ CONTROL_JVM_OPTS=-Djava.util.logging.config.file=<PATH_TO_CONFIG> ./control.bat

Commands

The control script supports the following commands (in alphabetical order):

Script Syntax

The control script has the following syntax:

control.sh <connection parameters> <command> <arguments>
control.bat <connection parameters> <command> <arguments>

  • <connection parameters> – parameters the script uses to connect to a cluster node. These parameters are required for the commands that are executed on the cluster nodes. If no connection parameters are provided, the control script tries to connect to a node running on localhost (localhost:11211).

  • <command> – one of the commands listed above.

  • <arguments> – command-specific arguments.

Connection Parameters

Parameter Description Default Value

--host HOST_OR_IP

The host name or IP address of the node.

localhost

--port PORT

The port to connect to. The port must be opened on the host specified in the --host property.

11211

--user USER

The user name.

--password PASSWORD

The user password.

--ping-interval PING_INTERVAL

The ping interval.

5000

--ping-timeout PING_TIMEOUT

The ping response timeout.

30000

--ssl-protocol PROTOCOL1, PROTOCOL2…​

A list of SSL protocols to try when connecting to the cluster. Supported protocols.

TLS

--ssl-cipher-suites CIPHER1,CIPHER2…​

A list of SSL ciphers. Supported ciphers.

--ssl-key-algorithm ALG

The SSL key algorithm.

SunX509

--keystore-type KEYSTORE_TYPE

The keystore type.

JKS

--keystore KEYSTORE_PATH

The path to the keystore. Specify a keystore to enable SSL for the control script.

--keystore-password KEYSTORE_PWD

The password to the keystore.

--truststore-type TRUSTSTORE_TYPE

The type of the truststore.

JKS

--truststore TRUSTSTORE_PATH

The path to the truststore.

--truststore-password TRUSTSTORE_PWD

The trustore password.

Activation, Deactivation, and Topology Management

You can use the control script to activate or deactivate your cluster, and manage the Baseline Topology.

set-state

Use this command to set a cluster’s state:

control.sh --set-state <state> [--yes]
control.bat --set-state <state> [--yes]

The command arguments are as follows.

Argument Description Values

state

The state to put the cluster in.

  • ACTIVE - sets the baseline topology of the cluster to the set of nodes available at the moment of activation. Activation is required only if you use native persistence.

  • INACTIVE - deactivates the cluster.

  • ACTIVE_READ_ONLY - sets cluster to read only mode: it will be active but cache updates will be denied.

--yes

Optional; if used, fast-forwards the command by automatically answering "yes" to all of the command’s prompts.

state

To get the state of a cluster (activated or not), use the following syntax:

control.sh --state
control.bat --state

baseline

To get a list of nodes registered in the baseline topology, run the following command:

control.sh --baseline
control.bat --baseline

The output contains the current topology version, the list of consistent IDs of the nodes included in the baseline topology, and the list of nodes that joined the cluster but were not added to the baseline topology.

Command [BASELINE] started
Arguments: --baseline
--------------------------------------------------------------------------------
Cluster state: active
Current topology version: 3

Current topology version: 3 (Coordinator: ConsistentId=dd3d3959-4fd6-4dc2-8199-bee213b34ff1, Order=1)

Baseline nodes:
    ConsistentId=7d79a1b5-cbbd-4ab5-9665-e8af0454f178, State=ONLINE, Order=2
    ConsistentId=dd3d3959-4fd6-4dc2-8199-bee213b34ff1, State=ONLINE, Order=1
--------------------------------------------------------------------------------
Number of baseline nodes: 2

Other nodes:
    ConsistentId=30e16660-49f8-4225-9122-c1b684723e97, Order=3
Number of other nodes: 1
Command [BASELINE] finished with code: 0
Control utility has completed execution at: 2019-12-24T16:53:08.392865
Execution time: 333 ms

baseline add

To add a node (or multiple nodes) to the baseline topology, run the following command.

control.sh --baseline add <consistentId1,consistentId2,...> [--yes]
control.bat --baseline add <consistentId1,consistentId2,...> [--yes]

The command arguments are as follows.

Argument Description

Id1,Id2,…​

A comma-separated list of consistent IDs of the nodes to add.

--yes

Optional; if used, fast-forwards the command by automatically answering "yes" to all of the command’s prompts.

baseline remove

To remove a node from the baseline topology, use the following command.

control.sh --baseline remove <consistentId1,consistentId2,...> [--yes]
control.bat --baseline remove <consistentId1,consistentId2,...> [--yes]

The command arguments are as follows.

Argument Description

Id1,Id2,…​

A comma-separated list of consistent IDs of the nodes to remove.

--yes

Optional; if used, fast-forwards the command by automatically answering "yes" to all of the command’s prompts.

baseline set

You can set baseline topology by providing a list of nodes (consistent IDs).

The command syntax is as follows:

control.sh --baseline set <consistentId1,consistentId2,...> [--yes]
control.bat --baseline set <consistentId1,consistentId2,...> [--yes]

The command arguments are as follows.

Argument Description

Id1,Id2,…​

A comma-separated list of consistent IDs of the nodes for setting baseline topology.

--yes

Optional; if used, it fast-forwards the command by automatically answering "yes" to all of the command’s prompts.

baseline version

To restore a specific version of the baseline topology, use the following command:

control.sh --baseline version <topologyVersion> [--yes]
control.bat --baseline version <topologyVersion> [--yes]

The command arguments are as follows.

Argument Description

topologyVersion

The version of the baseline topology to restore.

--yes

Optional; if used, fast-forwards the command by automatically answering "yes" to all of the command’s prompts.

baseline autoadjust enable

Baseline topology autoadjustment is an automatic update of baseline topology after the topology has been stable for a specific amount of time.

For in-memory clusters, autoadjustment is enabled by default with the timeout set to 0. It means that baseline topology changes immediately after server nodes join or leave the cluster.

For clusters with persistence, the automatic baseline adjustment is disabled by default.

To enable autoadjust, use the following command:

control.sh --baseline auto_adjust enable timeout <value>
control.bat --baseline auto_adjust enable timeout <value>

The command arguments are as follows.

Argument Description Default Value

timeout

The autoadjust timeout, in milliseconds. The baseline is set to the current topology when a given number of milliseconds has passed after the last JOIN/LEFT/FAIL event. Every new JOIN/LEFT/FAIL event restarts the timeout countdown.

0

baseline autoadjust disable

Baseline topology autoadjustment is an automatic update of baseline topology after the topology has been stable for a specific amount of time.

For clusters with persistence, the automatic baseline adjustment is disabled by default.

For in-memory clusters, autoadjustment is enabled by default with the timeout set to 0. It means that baseline topology changes immediately after server nodes join or leave the cluster.

To disable baseline autoadjustment, use the following command:

control.sh --baseline auto_adjust disable
control.bat --baseline auto_adjust disable

Transaction Management

The control script allows you to get information about the transactions that are executed in the cluster, as well as to cancel specific transactions.

tx --info

The following command returns a list of transactions that meet the filter conditions (or all transactions if no filter is defined):

control.sh --tx --info <transaction filter>
control.bat --tx --info <transaction filter>

The transaction filter parameters are as follows.

Parameter Description

--xid XID

The transaction ID.

--min-duration SECONDS

The minimum number of seconds a transaction has been executing.

--min-size SIZE

The minimum transaction size.

--label LABEL

The user label for transactions (you can use a regular expression).

--servers|--clients

Limits the scope of the operation to either the server or client nodes.

--nodes nodeId1,nodeId2…​

The list of consistent IDs of the nodes to get transactions from.

--limit NUMBER

Limits the number of transactions to the given value.

--order DURATION|SIZE|START_TIME

The parameter to sort the output by.

tx --kill

To cancel transactions, use the following command:

control.sh --tx <transaction filter> --kill
control.bat --tx <transaction filter> --kill

The transaction filter parameters are as follows.

Parameter Description

--xid XID

The transaction ID.

--min-duration SECONDS

The minimum number of seconds a transaction has been executing.

--min-size SIZE

The minimum transaction size.

--label LABEL

The user label for transactions (you can use a regular expression).

--servers|--clients

Limits the scope of the operation to either the server or client nodes.

--nodes nodeId1,nodeId2…​

The list of consistent IDs of the nodes to get transactions from.

--limit NUMBER

Limits the number of transactions to the given value.

--order DURATION|SIZE|START_TIME

The parameter to sort the output by.

The command affects the following transactions:

  • ACTIVE

  • PREPARING

  • PREPARED

For example, to cancel the transactions that have been running for more than 100 seconds, execute the following command:

control.sh --tx --min-duration 100 --kill
control.bat --tx --min-duration 100 --kill

cache contention

Use this command to detects situations where multiple transactions are in contention to create a lock for the same key. The command is useful if you have long-running or hanging transactions.

The command syntax is as follows:

control.sh --cache contention <arguments>
control.bat --cache contention <arguments>

The command arguments are as follows:

Argument Description

min queue size

The minimum number of transactions to wait for a specific key for the contention to be detected.

node id

Optional; the Id of the node to query for contentions

max lines

Optional; the number of lines to be printed in the output.

Example:

# Reports all keys that are a point of contention for at least 5 transactions on all cluster nodes.
control.sh --cache contention 5

# Reports all keys that are a point of contention for at least 5 transactions on a specific server node.
control.sh --cache contention 5 f2ea-5f56-11e8-9c2d-fa7a
# Reports all keys that are a point of contention for at least 5 transactions on all cluster nodes.
control.bat --cache contention 5

# Reports all keys that are a point of contention for at least 5 transactions on a specific server node.
control.bat --cache contention 5 f2ea-5f56-11e8-9c2d-fa7a

If contended keys are detected, the command dumps extensive information including the keys, transactions, and nodes where the contention took place.

Example:

[node=TcpDiscoveryNode [id=d9620450-eefa-4ab6-a821-644098f00001, addrs=[127.0.0.1], sockAddrs=[/127.0.0.1:47501], discPort=47501, order=2, intOrder=2, lastExchangeTime=1527169443913, loc=false, ver=2.5.0#20180518-sha1:02c9b2de, isClient=false]]

// No contention on node d9620450-eefa-4ab6-a821-644098f00001.

[node=TcpDiscoveryNode [id=03379796-df31-4dbd-80e5-09cef5000000, addrs=[127.0.0.1], sockAddrs=[/127.0.0.1:47500], discPort=47500, order=1, intOrder=1, lastExchangeTime=1527169443913, loc=false, ver=2.5.0#20180518-sha1:02c9b2de, isClient=false]]
    TxEntry [cacheId=1544803905, key=KeyCacheObjectImpl [part=0, val=0, hasValBytes=false], queue=10, op=CREATE, val=UserCacheObjectImpl [val=0, hasValBytes=false], tx=GridNearTxLocal[xid=e9754629361-00000000-0843-9f61-0000-000000000001, xidVersion=GridCacheVersion [topVer=138649441, order=1527169439646, nodeOrder=1], concurrency=PESSIMISTIC, isolation=REPEATABLE_READ, state=ACTIVE, invalidate=false, rollbackOnly=false, nodeId=03379796-df31-4dbd-80e5-09cef5000000, timeout=0, duration=1247], other=[]]
    TxEntry [cacheId=1544803905, key=KeyCacheObjectImpl [part=0, val=0, hasValBytes=false], queue=10, op=READ, val=null, tx=GridNearTxLocal[xid=8a754629361-00000000-0843-9f61-0000-000000000001, xidVersion=GridCacheVersion [topVer=138649441, order=1527169439656, nodeOrder=1], concurrency=PESSIMISTIC, isolation=REPEATABLE_READ, state=ACTIVE, invalidate=false, rollbackOnly=false, nodeId=03379796-df31-4dbd-80e5-09cef5000000, timeout=0, duration=1175], other=[]]
    TxEntry [cacheId=1544803905, key=KeyCacheObjectImpl [part=0, val=0, hasValBytes=false], queue=10, op=READ, val=null, tx=GridNearTxLocal[xid=6a754629361-00000000-0843-9f61-0000-000000000001, xidVersion=GridCacheVersion [topVer=138649441, order=1527169439654, nodeOrder=1], concurrency=PESSIMISTIC, isolation=REPEATABLE_READ, state=ACTIVE, invalidate=false, rollbackOnly=false, nodeId=03379796-df31-4dbd-80e5-09cef5000000, timeout=0, duration=1175], other=[]]
    TxEntry [cacheId=1544803905, key=KeyCacheObjectImpl [part=0, val=0, hasValBytes=false], queue=10, op=READ, val=null, tx=GridNearTxLocal[xid=7a754629361-00000000-0843-9f61-0000-000000000001, xidVersion=GridCacheVersion [topVer=138649441, order=1527169439655, nodeOrder=1], concurrency=PESSIMISTIC, isolation=REPEATABLE_READ, state=ACTIVE, invalidate=false, rollbackOnly=false, nodeId=03379796-df31-4dbd-80e5-09cef5000000, timeout=0, duration=1175], other=[]]
    TxEntry [cacheId=1544803905, key=KeyCacheObjectImpl [part=0, val=0, hasValBytes=false], queue=10, op=READ, val=null, tx=GridNearTxLocal[xid=4a754629361-00000000-0843-9f61-0000-000000000001, xidVersion=GridCacheVersion [topVer=138649441, order=1527169439652, nodeOrder=1], concurrency=PESSIMISTIC, isolation=REPEATABLE_READ, state=ACTIVE, invalidate=false, rollbackOnly=false, nodeId=03379796-df31-4dbd-80e5-09cef5000000, timeout=0, duration=1175], other=[]]

// Node 03379796-df31-4dbd-80e5-09cef5000000 is place for contention on key KeyCacheObjectImpl [part=0, val=0, hasValBytes=false].

Cache Management

cache list

This command is used for cache monitoring. It retrieves a list of deployed caches, their affinity/distribution parameters, and their distribution within cache groups. There is also a command option for viewing existing atomic sequences.

Use the following command syntax:

control.sh --cache list <arguments>
control.bat --cache list <arguments>

The command arguments are as follows.

Argument Description

.

With this argument, the command lists all caches.

account-.*

With this argument, the command lists caches whose names start with "account-".

. --groups

With this argument, the command displays info about cache group distribution for all caches.

account-.* --groups

With this argument, the command displays info about cache group distribution for the caches whose names start with "account-".

. --seq

With this argument, the command displays info about all atomic sequences.

account-.* --groups

With this argument, the command displays info about the atomic sequences whose names start with "counter-".

cache reset_lost_partitions

This command resets lost partitions in the specified caches. Refer to Partition Loss Policy for details.

Use the following command syntax:

control.sh --cache reset_lost_partitions <cacheName1,cacheName2,...> | --all
control.bat --cache reset_lost_partitions <cacheName1,cacheName2,...> | --all

The command arguments are as follows.

Argument Description

cacheName1,cacheName2,…​

A comma-separated list of the caches to be affected by the command.

--all

Makes the command check all caches for lost partition, print a list of caches with lost partitions (if any are found), and reset lost partitions in all the caches it has found.

cache idle_verify

This command verifies counters and hash sums of primary and backup partitions for the specified caches or scache groups on an idle cluster and prints out the differences, if any.

Use the following command syntax:

control.sh --cache idle_verify [--dump] [--skip-zeros] [--check-crc] [--exclude-caches <cacheName1,...,cacheNameN>] [--cache-filter ALL|USER|SYSTEM|PERSISTENT|NOT_PERSISTENT] [cacheName1,...,cacheNameN]
control.bat --cache idle_verify [--dump] [--skip-zeros] [--check-crc] [--exclude-caches <cacheName1,...,cacheNameN>] [--cache-filter ALL|USER|SYSTEM|PERSISTENT|NOT_PERSISTENT] [cacheName1,...,cacheNameN]

The command arguments are as follows.

Argument Description

--dump

Writes the command response to a local file on each node (in addition to returning response in the terminal window).

--skip-zeros

Skips zero-sized partitions.

--check-crc

Checks the CRC-sum of pages stored on disk before verifying partition data consistency between primary and backup nodes.

--exclude-caches

Excludes caches provided as a comma-separated list.

--cache-filter

Limits the caches affected by the command to only USER caches, only user PERSISTENT caches, only user NOT_PERSISTENT caches, only SYSTEM caches, or ALL of the above.

cacheName1,…​,cacheNameN

A comma-separated list of the caches to be affected by the command.

cache partition_reconciliation

Partition reconciliation is a process of consistency checking, with the goal to verify the internal data consistency invariants and fix the inconsistent entries. The main difference between idle_verify and partition_reconciliation is that the latter one can work under the load.

If the topology is changed while the script is running, or if the task execution fails, the command is automatically cancelled.

control.sh --cache partition_reconciliation <cache1,cache2,cache3...>
control.bat --cache partition_reconciliation <cache1,cache2,cache3...>

The command has the following optional arguments.

Argument Description Default Value

cache1,cache2,…​

A comma-separated list of the caches to be affected by the command. If caches are not specified, the command is executed on all caches.

--repair {option}

If specified, fixes all inconsistent data. You can choose the repair algorithm for the keys where the valid key is not obvious.

The following values can be used:

  • PRINT_ONLY - fixes only those conflicts where the key is present on all nodes. Conflicts with missing keys are printed in the report.

  • LATEST - picks the latest value. If any partition value is NULL, selects the latest value from the existing NOT NULL partitions. This may result in restoring a removed key.

  • PRIMARY - picks a value from the primary partition. Beware: this option may result in key removal.

  • MAJORITY - picks the most common value, or one of the the most common values randomly (if there is no consensus). If the missing value is a majority, the key is removed.

  • REMOVE - removes a key if a conflict cannot be resolved without a user algorithm.

PRINT_ONLY

--fast-check

Checks and repairs only those partitions that did not pass validation during the last partition map exchange. If not specified, all partitions are checked and repaired.

--parallelism

The maximum number of threads that can be involved in the reconciliation activities. If not specified, the number of the node’s cores is used.

--batch-size

The number of keys to retrieve within one job.

1000

--include-sensitive

Includes sensitive information in the printout: keys and values.

false

--recheck-attempts

The number mount of recheck attempts for the potentially inconsistent keys. The recommended value is between 1 and 5.

2

--recheck-delay

Recheck delay in seconds.

5

cache partition_reconciliation_cancel

Use this command to safely stop the partition reconciliation process. All changes done before the cancellation are preserved.

control.sh --cache  partition_reconciliation_cancel
control.bat --cache  partition_reconciliation_cancel

cache destroy

Use this command to destroy caches.

The syntax is as follows:

 control.sh --cache destroy --caches <cache1,...,cacheN>|--destroy-all-caches
 control.bat --cache destroy --caches <cache1,...,cacheN>|--destroy-all-caches

The command uses one of the following two arguments:

Argument Description

--caches

A comma-separated list of names of the caches to be destroyed.

--destroy-all-caches

Destroys all user-created caches.

cache clear

Use this command to clear caches.

The syntax is as follows:

 control.sh --cache clear --caches <cache1,...,cacheN>
 control.bat --cache clear --caches <cache1,...,cacheN>

The command uses the following argument:

Argument Description

--caches

A comma-separated list of names of the caches to be cleared.

Tracing Configuration Management

The control.sh|bat script includes a set of commands that enable you to manage tracing configurations.

tracing-configuration

Use this command to print tracing configuration for all scopes and labels.

The syntax is as follows:

 control.sh --tracing-configuration
 control.bat --tracing-configuration

tracing-configuration get_all

Use this command to print tracing configuration for all labels and, optionally, the specified --scope.

The syntax is as follows:

 control.sh --tracing-configuration get_all [--scope <scope>]
 control.bat --tracing-configuration get_all [--scope <scope>]

The command uses the following argument:

Argument Description

--scope

The scope to print the tracing configuration for: DISCOVERY, EXCHANGE, COMMUNICATION, TX, CACHE_API_WRITE, CACHE_API_READ, or SQL.

tracing-configuration get

Use this command to print tracing configuration for specified --scope and --label.

The syntax is as follows:

 control.sh --tracing-configuration get --scope <scope> [--label<label>]
 control.bat --tracing-configuration get --scope <scope> [--label<label>]

The command uses the following arguments:

Argument Description

--scope

The scope to print the tracing configuration for: DISCOVERY, EXCHANGE, COMMUNICATION, TX, CACHE_API_WRITE, CACHE_API_READ, or SQL.

--label

The label to print the tracing configuration for.

tracing-configuration reset_all

Use this command to reset tracing configurations to default values, then print the reset configurations, optionally for the specified --scope.

The syntax is as follows:

 control.sh --tracing-configuration reset_all [--scope <scope>]
 control.bat --tracing-configuration reset_all [--scope <scope>]

The command uses the following argument:

Argument Description

--scope

If specified, the command applies the configurations only to the specified scope, which can be DISCOVERY, EXCHANGE, COMMUNICATION, TX, CACHE_API_WRITE, CACHE_API_READ, or SQL.

tracing-configuration reset

Use this command to reset tracing configurations to default values, then print the reset configurations. If both --scope and --label are specified, removes the current configuration. If only --scope is specified, resets the current configuration to the default.

The syntax is as follows:

 control.sh --tracing-configuration reset --scope <scope> [--label <label>]
 control.bat --tracing-configuration reset --scope <scope> [--label <label>]

The command uses the following arguments:

Argument Description

--scope

The command applies to configurations only for the specified scope, which can be DISCOVERY, EXCHANGE, COMMUNICATION, TX, CACHE_API_WRITE, CACHE_API_READ, or SQL.

--label

If specified, the command applies to configurations for the specified label.

tracing-configuration set

Use this command to set a new tracing configuration, then print it.

The syntax is as follows:

 control.sh --tracing-configuration set --scope <scope> [--label <label>] [--sampling-rate <samplingRate>] [--included-scopes <scope, ...>]
 control.bat --tracing-configuration set --scope <scope> [--label <label>] [--sampling-rate <samplingRate>] [--included-scopes <scope, ...>]

The command uses the following arguments:

Argument Description

--scope

The command applies to configurations only for the specified scope, which can be DISCOVERY, EXCHANGE, COMMUNICATION, TX, CACHE_API_WRITE, CACHE_API_READ, or SQL.

--label

If specified, the command applies to configurations for the specified label.

--sampling-rate

A decimal value between 0 and 1, where 0 means "never" and 1 means "always." Reflects the probability of sampling a specific trace.

--included-scopes

A comma-separated list of scopes that defines the sub-traces to be included in a given trace. In other words, if a child’s span scope equals the parent’s scope, or if it belongs to the set of scopes included in the parent’s span, then the specified child span is attached to the current trace. The scopes can be DISCOVERY, EXCHANGE, COMMUNICATION, TX, CACHE_API_WRITE, CACHE_API_READ, or SQL

Consistency Checks

The control.sh|bat script includes a set of commands that enable you to verify the internal data consistency.

These commands can be used for:

  • Debugging and troubleshooting, especially during active development

  • Checking for data inconsistencies when there is a suspicion that a query - such as an SQL query - returns an incomplete or wrong result set

  • Regular cluster health monitoring

cache idle_verify

This command compares the hash of the primary partition with that of the backup partitions and reports differences, if any. Said differences might result from node failure or incorrect shutdown during an update operation.

The command syntax is as follows:

control.sh --cache idle_verify [<cache1,cache2,cache3...>]
control.bat --cache idle_verify [<cache1,cache2,cache3...>]

The command has the following optional arguments.

Argument Description

cache1,cache2,…​

A comma-separated list of the caches to be affected by the command.

A list of diverging partitions is printed out, as follows:

idle_verify check has finished, found 2 conflict partitions.

Conflict partition: PartitionKey [grpId=1544803905, grpName=default, partId=5]
Partition instances: [PartitionHashRecord [isPrimary=true, partHash=97506054, updateCntr=3, size=3, consistentId=bltTest1], PartitionHashRecord [isPrimary=false, partHash=65957380, updateCntr=3, size=2, consistentId=bltTest0]]
Conflict partition: PartitionKey [grpId=1544803905, grpName=default, partId=6]
Partition instances: [PartitionHashRecord [isPrimary=true, partHash=97595430, updateCntr=3, size=3, consistentId=bltTest1], PartitionHashRecord [isPrimary=false, partHash=66016964, updateCntr=3, size=2, consistentId=bltTest0]]

cache check_index_inline_sizes

This command verifies that index inline size is the same on all cluster nodes. Every entry in the SQL index has a constant size, which is calculated during the index creation. Having different sizes on different nodes in a cluster may lead to performance issues.

The command syntax is as follows:

control.sh --cache check_index_inline_sizes
control.bat --cache check_index_inline_sizes

The command returns a response in the command line.

cache validate_indexes

This command validates the indexes of the specified caches on the specified cluster nodes.

It verifies that:

  • All key-value entries referenced from the primary index are reachable from the secondary SQL indexes.

  • All the key-value entries referenced from the primary index are reachable as defined.

  • All the key-value entries referenced from the secondary SQL indexes are reachable from the primary index.

The command syntax is as follows:

control.sh --cache validate_indexes <optional arguments>
control.bat --cache validate_indexes <optional arguments>

The command can use one or both of the following optional arguments:

Argument Description

cache1,…​,cacheN

A comma-separated list of names of the caches to be validated.

node ID; e.g., f2ea-5f56-11e8-9c2d-fa7a

ID of the cluster node on which the caches should be validated.

If indexes refer to non-existing entries, or if some entries are not indexed, errors appear in the command output. Example:

PartitionKey [grpId=-528791027, grpName=persons-cache-vi, partId=0] ValidateIndexesPartitionResult [updateCntr=313, size=313, isPrimary=true, consistentId=bltTest0]
IndexValidationIssue [key=0, cacheName=persons-cache-vi, idxName=_key_PK], class org.apache.ignite.IgniteCheckedException: Key is present in CacheDataTree, but can't be found in SQL index.
IndexValidationIssue [key=0, cacheName=persons-cache-vi, idxName=PERSON_ORGID_ASC_IDX], class org.apache.ignite.IgniteCheckedException: Key is present in CacheDataTree, but can't be found in SQL index.
validate_indexes has finished with errors (listed above).

cache distribution

This command prints information about partition distribution.

The command syntax is as follows:

control.sh --cache distribution nodeId|null [cacheName1,...,cacheNameN] [--user-attributes attrName1,...,attrNameN]
control.bat --cache distribution nodeId|null [cacheName1,...,cacheNameN] [--user-attributes attrName1,...,attrNameN]

The command can use one or both of the following optional arguments:

Argument Description

node ID; e.g., f2ea-5f56-11e8-9c2d-fa7a

ID of the cluster node whose cache distribution must be queried. Alternatively, "null", which means "all nodes."

cacheName1,…​,cacheNameN

A comma-separated list of names of the caches whose distribution must be queried. Providing no cache names means "all caches."

--user-attributes attrName1,…​,attrNameN

A comma-separated list of the names of node attributes to be included in the command response. User attributes are configurable.

Cluster Properties

The control.sh|bat script enables you to use the SQL statistics functionality.

property list

Use this command to get the full list of the available cluster properties.

The command syntax is as follows:

control.sh --property list
control.bat  --property list

property set

Use this command to set property values.

The command syntax is as follows:

control.sh --property set --name <property name> --val <property value>
control.bat --property set --name <property name> --val <property value>

For example, you to enable SQL statistics in a cluster:

control.sh --property set --name 'statistics.usage.state' --val 'ON'
control.bat  --property set --name 'statistics.usage.state' --val 'ON'

property get

Use this command to get a property value.

The command syntax is as follows:

control.sh --property get --name <name>
control.bat --property get --name <name>

checkpoint

When a user uploads a large volume of data to the cluster, to be sure the data is delivered successfully, you can trigger a manual checkpoint on the cluster.

Use the following command to trigger a checkpoint on all server nodes simultaneously and wait until the checkpoint completes:

control.sh --checkpoint
control.bat --checkpoint

In a large distributed database, triggering multiple checkpoints at the same time can create a spike in traffic and, consequently, negatively affect the cluster performance. To prevent this, you can set the checkpoint.deviation property to offset checkpoints' time, so that they happen within specified interval of checkpoint.frequency. For example, if your checkpoints trigger every 100 seconds, and you set checkpoint.deviation at 10%, your triggers will happen every 95-105 seconds.

The example below offsets the checkpoints by 10% to reduce traffic issues:

control.sh --property set --name checkpoint.deviation --val 10
control.sh --property set --name checkpoint.deviation --val 10

Cluster Diagnostics

The control.sh|bat script provides diagnostics for your cluster.

diagnostic

Use this command to get diagnostic information for your cluster (page locks or connectivity):

control.sh --diagnostic pageLocks|connectivity dump|dump_log [--path <absolute path>] [--all]|[--nodes <nodeId1,...,nodeIdN>]
control.bat --diagnostic pageLocks|connectivity dump|dump_log [--path <absolute path>] [--all]|[--nodes <nodeId1,...,nodeIdN>]

The command arguments are as follows:

Argument Description

pageLocks

Determines what pages are currently locked and outputs the page lock information to a file/log.

connectivity

Detects inter-node communication issues and outputs the corresponding information to log/file.

dump

Dumps the page lock information to a file.

dump_log

Dumps the page lock information to the log.

--path <path>

Optional. If used, provides an absolute path to the folder to write the output to. If not used, the data is saved in the work folder.

--all

Gets information on all nodes.

--nodes

A comma-separated list of IDs of the nodes to get information on.

Example pageLock output:

Thread=[name=main, id=1], state=RUNNABLE
Locked pages = [2[0000000000000002](r=1|w=0)]
Locked pages log: name=main time=(1673270864117, 2023-01-09 15:27:44.117)
L=1 -> Read lock pageId=2, structureId=null [pageIdHex=0000000000000002, partId=0, pageIdx=2, flags=00000000]

Example connectivity output:

There is no connectivity between the following nodes:
SOURCE-NODE-ID                        SOURCE-CONSISTENT-ID     SOURCE-NODE-TYPE  DESTINATION-NODE-ID                   DESTINATION_CONSISTENT_ID  DESTINATION-NODE-TYPE
ea0b75f5-8c31-4934-b076-edb911800001  gridCommandHandlerTest1  SERVER            3db4a4bb-f758-458c-8569-1ed3e3500003  gridCommandHandlerTest3    SERVER
99ac27d8-4e5e-43cd-9acc-1320e6500002  gridCommandHandlerTest2  SERVER            3db4a4bb-f758-458c-8569-1ed3e3500003  gridCommandHandlerTest3    SERVER

Cluster Encryption

You can use the control.sh|bat script to manage cluster encryption parameters. For more information about encryption key, see the Transparent Data Encryption page.

encryption get_master_key_name

Use the this command to get the master key that is used in the cluster encryption:

control.sh --encryption get_master_key_name
control.bat --encryption get_master_key_name

encryption change_master_key

Use the this command to set the master key that is used in the cluster encryption:

control.sh --encryption change_master_key <newMasterKeyName>
control.bat --encryption change_master_key <newMasterKeyName>

encryption change_cache_key

Use this command to change cache-level encryption keys:

control.sh --encryption change_cache_key <cacheGroupName>
control.bat --encryption change_cache_key <cacheGroupName>

encryption cache_key_ids

Use this command to list IDs of the cache group encryption keys:

control.sh --encryption cache_key_ids <cacheGroupName>
control.bat --encryption cache_key_ids <cacheGroupName>

encryption reencryption_status

Use this command to monitor cluster reencryption:

control.sh --encryption reencryption_status <cacheGroupName>
control.bat --encryption reencryption_status cacheGroupName

encryption suspend_reencryption

Use this command to pause reencryption of your cluster:


control.sh --encryption suspend_reencryption <cacheGroupName>
control.bat --encryption suspend_reencryption <cacheGroupName>

encryption resume_reencryption

When re-encryption is initiated on the cluster, you can monitor it with the reencryption_status subcommand.

Use this command to resume reencryption of your cluster that you had previously suspended (see encryption suspend_reencryption):

control.sh --encryption resume_reencryption <cacheGroupName>

control.bat --encryption resume_reencryption <cacheGroupName>

encryption reencryption_rate_limit

Encryption requires a large amount of system resources. One way to address this is to suspend reencryption to allow the system to handle the workload - see encryption suspend_reencryption. Alternatively, you can use this command to limit the reencryption rate.

control.sh --encryption reencryption_rate_limit <limit in MB/sec>
control.bat --encryption reencryption_rate_limit <limit in MB/sec>

Kill Queries

kill sql

This command stops the specific sql query.

control.sh --kill SQL <queryId>
control.bat --kill SQL <queryId>

Use the SQL_QUERIES system view to get the IDs of currently running queries.

kill continuous

This command stops the specific continuous query.

control.sh --kill CONTINUOUS <NodeId> <routineId>
control.bat --kill CONTINUOUS <NodeId> <routineId>

Use the CONTINUOUS_QUERIES system view to get the origin node ID and routine ID for your queries.

Rolling Upgrades

You can manage rolling upgrades using the control.sh|bat script.

rolling-upgrade start

Use this command to enable rolling upgrades:

control.sh --rolling-upgrade start [--yes]
control.bat --rolling-upgrade start [--yes]

The command arguments are as follows.

Argument Description

--yes

Optional; if used, fast-forwards the command by automatically answering "yes" to all of the command’s prompts.

rolling-upgrade finish

Use this command to disable rolling upgrades:


control.sh --rolling-upgrade finish [--yes]

control.bat --rolling-upgrade finish [--yes]

The command arguments are as follows.

Argument Description

--yes

Optional; if used, fast-forwards the command by automatically answering "yes" to all of the command’s prompts.

rolling-upgrade force

Use this command to force a rolling upgrade on your cluster.

control.sh --rolling-upgrade force [--yes]
control.bat --rolling-upgrade force [--yes]

The command arguments are as follows.

Argument Description

--yes

Optional; if used, fast-forwards the command by automatically answering "yes" to all of the command’s prompts.

rolling-upgrade status

Use this command to check the current status of the rolling upgrade on your cluster:

control.sh --rolling-upgrade status
control.bat --rolling-upgrade status

Cluster IDs and Tags

You can set cluster ID and tag values using the control.sh|bat script.

change-id

Use this command to change the cluster ID of in-memory clusters.

control.sh --change-id <newIdValue> [--yes]
control.bat --change-id <newIdValue> [--yes]

The command arguments are as follows.

Argument Description

newIdValue

The new ID for the cluster.

--yes

Optional; if used, fast-forwards the command by automatically answering "yes" to all of the command’s prompts.

change-tag

Use this command to change the cluster tag of in-memory clusters.

control.sh --change-tag <newTagValue> [--yes]
control.bat --change-tag <newTagValue> [--yes]

The command arguments are as follows.

Argument Description

newTagValue

The new tag for the cluster.

--yes

Optional; if used, fast-forwards the command by automatically answering "yes" to all of the command’s prompts.

Data Center Replication

You can use the control.sh|bat script to track data center replication.

dr state

Use this command to get the current state of data center replication:

With the --verbose option, you can get extended status information.

control.sh --dr state [--verbose]
control.bat --dr state [--verbose]

The command arguments are as follows.

Argument Description

--verbose

Optional; if used, prints extended status information.

dr topology

Use this command to print the cluster topology with the Data Center Replication details:

control.sh --dr topology [--sender-hubs] [--receiver-hubs] [--data-nodes] [--other-nodes]
control.bat --dr topology [--sender-hubs] [--receiver-hubs] [--data-nodes] [--other-nodes]

The command arguments are as follows.

Argument Description

--sender-hubs

Optional; if used, displays information about sender nodes.

--receiver-hubs

Optional; if used, displays information about receiver nodes.

--data-nodes

Optional; if used, displays information about data nodes in the cluster.

--other-nodes

Optional; if used, displays information about nodes that are not currently involved in the DR.

dr pause

Use this command to pause data center replication on all sender nodes, or on a specific sender node.

control.sh --dr pause <remoteDataCenterId> [--yes]
control.bat --dr pause <remoteDataCenterId> [--yes]

The command arguments are as follows.

Argument Description

remoteDataCenterId

The ID of the remote data center.

--yes

Optional; if used, fast-forwards the command by automatically answering "yes" to all of the command’s prompts.

dr resume

Use this command to resume data center replication on all sender nodes, or on a specific sender node.

control.sh --dr resume <remoteDataCenterId> [--yes]
control.bat --dr resume <remoteDataCenterId> [--yes]

The command arguments are as follows.

Argument Description

remoteDataCenterId

The ID of the remote data center.

--yes

Optional; if used, fast-forwards the command by automatically answering "yes" to all of the command’s prompts.

dr full-state-transfer start

Use this command to execute a full state transfer on all caches in a cluster if the caches in the master cluster already have data:

control.sh --dr full-state-transfer start [--snapshot <snapshotId>]  [--caches <cache1, ...>] [--sender-group <group1, ...>] [--data-centers <dcId, ...>] [--sync] [--yes]
control.bat --dr full-state-transfer start [--snapshot <snapshotId>]  [--caches <cache1, ...>] [--sender-group <group1, ...>] [--data-centers <dcId, ...>] [--sync] [--yes]

The command arguments are as follows.

Argument Description

--snapshot

Optional; the ID of the snapshot to perform the state transfer on.

--caches

Optional; the list of caches to transfer.

--sender-group

Optional; the group of the sender caches. Possible values: <groupName>, ALL, DEFAULT, NONE.

--data-centers

Optional; IDs of the data centers involved in the transfer.

--sync

Optional; if used, executes the full state transfer in a synchronous way. Otherwise, the full state transfer is executed asynchronously.

--yes

Optional; if used, it fast-forwards the command by automatically answering "yes" to all of the command’s prompts.

dr check-partition-counters

Use this command to check whether all partition counters are in order:

control.sh --dr check-partition-counters [--caches <cache1,...>]
control.bat --dr check-partition-counters [--caches cache1,cache2]

The command argument is as follows.

Argument Description

--caches

Optional; the list of caches to check the partition counters for.

dr rebuild-partition-tree

Use this command to schedule/run the maintenance task for rebuilding DR trees:

control.sh --dr rebuild-partition-tree [--caches <cacheName1,...,cacheNameN>] [--groups <groupName1,...,groupNameN>] [--yes]
control.bat --dr rebuild-partition-tree [--caches <cacheName1,...,cacheNameN>] [--groups <groupName1,...,groupNameN>] [--yes]

The command arguments are as follows.

Argument Description

--caches

A comma-separated list of caches to be affected by the command.

--groups

A comma-separated list of cache groups to be affected by the command.

--yes

If used, fast-forwards the command by automatically answering "yes" to all of the command’s prompts.

dr repair-partition-counters

Use this command to repair partition counters during data center replication:

control.sh --dr repair-partition-counters [--caches <cache1,...>]
control.bat --dr repair-partition-counters [--caches cache1,cache2]

The command argument is as follows.

Argument Description

--caches

Optional; the list of caches to check the partition counters for.

dr cleanup-partition-tree

Use this command to schedule and run a maintenance task to clean up DR trees:

control.sh --dr cleanup-partition-tree [--caches <cacheName1,...,cacheNameN] [--groups groupName1,...,groupNameN] [--yes]
control.bat --dr cleanup-partition-tree [--caches <cacheName1,...,cacheNameN] [--groups groupName1,...,groupNameN] [--yes]

The command arguments are as follows.

Argument Description

--caches

Optional; the list of caches to to clean up.

--groups

Optional; the list of cache groups to clean up.

--yes

If used, fast-forwards the command by automatically answering "yes" to all of the command’s prompts.

checkpointing force

Use this command to dump all data to persistent storage (for example, to make sure that all data is saved before deleting the source data):

control.sh --checkpointing force
control.bat --checkpointing force

dr full-state-transfer cancel

Use this command to cancel an active full state transfer at any time:

control.sh --dr full-state-transfer cancel <fullStateTransferUID> [--yes]
control.bat --dr full-state-transfer cancel <fullStateTransferUID> [--yes]

The command arguments are as follows.

Argument Description

fullStateTransferUID

The ID of the fill state transfer to cancel.

--yes

Optional; if used, fast-forwards the command by automatically answering "yes" to all of the command’s prompts.

dr full-state-transfer list

Use this command to get a list of all full state transfers currently in progress.

control.sh --dr full-state-transfer list
control.bat --dr full-state-transfer list

dr node

Use this command to get information about a node’s status during data center replication:

control.sh --dr node <nodeId> [--config] [--metrics] [--clear-store] [--yes]
control.bat --dr node <nodeId> [--config] [--metrics] [--clear-store] [--yes]

The command arguments are as follows.

Argument Description

--config

Optional; displays node configuration.

--metrics

Optional; displays node metrics.

--clear-store

Optional; cleans store after the command execution.

--yes

Optional; if used, fast-forwards the command by automatically answering "yes" to all of the command’s prompts.

dr cache

Use this command to get specific cache details related to data center replication:

control.sh --dr cache <regExp> [--config] [--metrics] [--cache-filter ALL|SENDING|RECEIVING|PAUSED|ERROR] [--sender-group <groupName>|ALL|DEFAULT|NONE] [--action stop|start|full-state-transfer] [--yes]
control.bat --dr cache <regExp> [--config] [--metrics] [--cache-filter ALL|SENDING|RECEIVING|PAUSED|ERROR] [--sender-group <groupName>|ALL|DEFAULT|NONE] [--action stop|start|full-state-transfer] [--yes]

The command arguments are as follows.

Argument Description

--config

Optional; displays cache configuration.

--metrics

Optional; displays node metrics.

--cache-filter

Optional; the cache filter. Possible values: ALL, SENDING, RECEIVING, PAUSED, ERROR.

--sender-group

Optional: the group of the sender caches. Possible values: <groupName>, ALL, DEFAULT, NONE.

--action

Optional; the action to perform. Possible values: stop, start, full-state-transfer.

--yes

Optional; if used, fast-forwards the command by automatically answering "yes" to all of the command’s prompts.

Shutdown Policies

shutdown-policy

Use the shutdown-policy command to set the node shutdown policy to:

  • GRACEFUL - to have nodes manage their ongoing tasks before they shut down

  • IMMEDIATE - to have nodes shut down immediately

The command syntax is as follows:

control.sh --shutdown-policy [IMMEDIATE|GRACEFUL]
control.bat --shutdown-policy [IMMEDIATE|GRACEFUL]

Warmup Configuration

warm-up

Use the warm-up command to disable cache warmup on the cluster:

control.sh --warm-up --stop
control.bat --warm-up --stop

Persistence Configuration

You can use the control.sh|bat script to define the way the data files and caches are treated.

persistence

Use this command to print information about the potentially corrupted caches on a local node:

control.sh --persistence
control.sh --persistence

persistence clean

Use this command to clean persistent caches.

control.sh --persistence clean [corrupted]|[all]|[caches <cache1,cache2,...,cacheN>]
control.bat --persistence clean [corrupted]|[all]|[caches <cache1,cache2,...,cacheN>]

The command arguments are as follows.

Argument Description

corrupted

Optional; cleans corrupted caches.

all

Optional; cleans all caches.

caches

Optional; cleans caches included in a comma-delimited list.

persistence backup

Use this command to back up caches:

control.sh --persistence backup [corrupted]|[all]|[caches <cache1,cache2,...,cacheN>]
control.bat --persistence backup [corrupted]|[all]|[caches <cache1,cache2,...,cacheN>]

The command arguments are as follows.

Argument Description

corrupted

Optional; backs up corrupted caches.

all

Optional; backs up all caches.

caches

Optional; backs up caches included in a comma-delimited list.

Defragmentation

When persistance is enabled on the disk, you need to perform data defragmentation on that disk.

defragmentation schedule

Use this command to schedule the disk defragmentation:

control.sh --defragmentation schedule --nodes <consistentId0,...,consistentIdN,> [--caches <cache1,cache2,...,cacheN>]
control.bat --defragmentation schedule --nodes <consistentId0,...,consistentIdN,> [--caches <cache1,cache2,...,cacheN>]

The command arguments are as follows.

Argument Description

--nodes

A comma-separate list of nodes to undergo defragmentation.

--caches

Optional; a comma-separated list pf caches to undergo defragmentation.

defragmentation cancel

Use this command to cancel a scheduled or active defragmentation if it interferes with other operations:

control.sh  --defragmentation cancel
control.bat --defragmentation cancel

defragmentation status

Use this command to check the status the current defragmentation process.

control.sh  --defragmentation status
control.bat --defragmentation status

Binary Type Management

meta list

Use this command to list meta information:

control.sh --meta list
control.bat --meta list

meta details

Use this command to print information about the specified binary type:

  • typeId=<ID>

  • typeName=<name>

  • fields=<fields_count>

  • schemas=<schemas_count>

  • isEnum=<bool>

The command syntax is as follows:

control.sh --meta details --typeId <ID>|--typeName <name>
control.bat --meta details --typeId <ID>|--typeName <name>

One of the following command arguments must be used.

Argument Description

--typeId

The ID of the binary type.

--typeName

The name of the binary type.

Following is an example of the command output:

typeId=0x1FBFBC0C (532659212)
typeName=TypeName1
Fields:
name=fld3, type=long[], fieldId=0x2FFF95 (3145621)
name=fld2, type=double, fieldId=0x2FFF94 (3145620)
name=fld1, type=Date, fieldId=0x2FFF93 (3145619)
name=fld0, type=int, fieldId=0x2FFF92 (3145618)
Schemas:
schemaId=0x6C5CC179 (1818018169), fields=[fld0]
schemaId=0x70E46431 (1894016049), fields=[fld0, fld1, fld2, fld3]

meta remove

Use this command to remove metadata for the specified type form the cluster and save the removed metadata to a file.

The command syntax is as follows:

control.sh --meta remove --typeId <ID>| --typeName <name> [--out <file_name>]
control.bat --meta remove --typeId <ID>| --typeName <name> [--out <file_name>]

The command arguments are as follows.

Argument Description

--typeId

The ID of the binary type.

--typeName

The name of the binary type.

--out

The name of the file to save the removed metadata to. If not specified, the default <typeId>.bin is used.

meta update

Use this command to updates cluster metadata from a specified file.

The command syntax is as follows:

control.sh --meta update --in <file_name>
control.bat --meta update --in <file_name>

The command arguments are as follows.

Argument Description

--in

The name of the file to use as a source of the metadata.

Index Rebuild

You can use the control.sh|bat script to manage the process of rebuilding indexes for specified caches or cache groups.

cache indexes_force_rebuild

Use this command to schedule an index rebuild:

 control.sh --cache indexes_force_rebuild --node-id <nodeId1,...,nodeIdN> --cache-names <cacheName[index1,...indexN],...,cacheName3[index1] --group-names <groupName1,...groupNameN>
 control.bat -cache indexes_force_rebuild --node-id <nodeId1,...,nodeIdN> --cache-names <cacheName[index1,...indexN],...,cacheName3[index1] --group-names <groupName1,...groupNameN>

The command arguments are as follows:

Argument Description

--node-id

A comma-separated list of nodes to rebuild indexes on. If not specified, rebuild is scheduled on all nodes.

--cache-names

A comma-separated list of cache names, optionally with indexes. If indexes are not specified, all indexes of the cache will be scheduled for the rebuild operation. Can be used in addition to cache group names.

--group-names

A comma-separated list of cache group names. Can be used in addition to cache names.

© 2024 GridGain Systems, Inc. All Rights Reserved. Privacy Policy | Legal Notices. GridGain® is a registered trademark of GridGain Systems, Inc.
Apache, Apache Ignite, the Apache feather and the Apache Ignite logo are either registered trademarks or trademarks of The Apache Software Foundation.

Last updated on Dec 04, 2024