GridGain vs. Hazelcast Benchmarks

GridGain vs Hazelcast Benchmarks

GridGain® Professional Edition 1.9.1 and Hazelcast® 3.8 (Open Source) were benchmarked on AWS EC2 using Yardstick configurations. Results for throughput, latency, and percentile probes measuring various metrics during benchmark execution were collected.

GridGain vs. Hazelcast Performance Summary

Deadlock Free Transactions: 53% – 65% higher operations/sec throughput
Transactional Operations: 30% – 57% higher operations/sec throughput
Atomic Operations: Put Benchmark: Up to 21% higher operations/sec throughput
Put and Get Benchmark: Depending on configuration, either GridGain or Hazelcast performed slightly better
SQL Query: 13% – 40% higher operations/sec throughput

GridGain outperformed Hazelcast on all atomic and transactional cache operations and SQL-based cache queries except the Atomic Put and Get benchmark, which provided mixed results. The testing results are compiled below to help you make an informed decision about the best in-memory solution for your company.

AWS EC2 Configuration
Hardware r4.2xlarge
CPU 8
RAM 61 Gib
OS Ubuntu 16.04
Yardstick Configuration
Nodes 1 Client, 4 Servers
Threads 64
Backup 1, Synchronous
JDK 1.8.0_121-b13

Deadlock Free Transactions (Optimistic, Serializable)

Hazelcast does not have support for deadlock-free transactions, so we are comparing Ignite deadlock-free transactions to Hazelcast pessimistic transactions.

Put Benchmarks

Throughput, operations/sec
Bigger number represents better throughput
# of keys Ignite Hazelcast Delta
1 58,295 27,315 53%
2 26,611 11,246 58%
6 14,632 5,697 61%
10 11,384 3,925 65%
Latency, milliseconds
Smaller number represents better latency
Ignite Hazelcast
1.09 2.34
2.40 5.70
4.38 11.27
5.62 16.35

Graphs: 1 key, 2 keys, 6 keys, 10 keys

Put and Get Benchmarks

Throughput, operations/sec
Bigger number represents better throughput
# of keys Ignite Hazelcast Delta
1 22,873 11,310 51%
2 20,223 9,954 51%
6 11,856 4,811 59%
10 9,432 3,320 65%
Latency, milliseconds
Smaller number represents better latency
Ignite Hazelcast
2.80 5.68
3.16 6.43
5.40 13.32
6.79 17.67

Graphs: 1 key, 2 keys, 6 keys, 10 keys

Transactional Operations

Put Benchmarks

Throughput, operations/sec
Bigger number represents better throughput
# of keys Ignite Hazelcast Delta
1 38,840 27,315 30%
2 23,115 11,246 51%
6 10,561 5,697 46%
10 7,413 3,925 47%
Latency, milliseconds
Smaller number represents better latency
Ignite Hazelcast
1.64 2.34
2.77 5.70
6.06 11.27
8.64 16.35

Graphs: 1 key, 2 keys, 6 keys, 10 keys

Pessimistic Put and Get Benchmarks

Throughput, operations/sec
Bigger number represents better throughput
# of keys Ignite Hazelcast Delta
1 17,780 11,310 36%
2 18,829 9,954 47%
6 9,495 4,811 49%
10 6,886 3,320 51%
Latency, milliseconds
Smaller number represents better latency
Ignite Hazelcast
3.60 5.68
3.40 6.43
6.74 13.32
9.31 19.30

Graphs: 1 key, 2 keys, 6 keys, 10 keys

Optimistic Put and Get Benchmarks

Throughput, operations/sec
Bigger number represents better throughput
# of keys Ignite Hazelcast Delta
1 39,879 20,418 49%
2 18,237 9,253 49%
6 8,896 4,069 54%
10 6,367 2,720 57%
Latency, milliseconds
Smaller number represents better latency
Ignite Hazelcast
1.60 3.13
3.51 6.92
7.19 15.77
10.06 23.57

Graphs: 1 key, 2 keys, 6 keys, 10 keys

Atomic Operations

Put Benchmarks

Throughput, operations/sec
Bigger number represents better throughput
# of keys Ignite Hazelcast Delta
1 76,082 74,043 2%
2 58,958 57,999 1%
6 37,027 32,718 11%
10 29,523 23,089 21%
Latency, milliseconds
Smaller number represents better latency
Ignite Hazelcast
0.84 0.86
1.08 1.10
1.72 1.95
2.17 2.77

Graphs: 1 key, 2 keys, 6 keys, 10 keys

Atomic Put and Get Benchmarks

Throughput, operations/sec
Bigger number represents better throughput
# of keys Ignite Hazelcast Delta
1 48,191 46,780 3%
2 35,023 37,255 -6%
6 21,946 22,130 0%
10 18,033 15,931 11%
Latency, milliseconds
Smaller number represents better latency
Ignite Hazelcast
1.32 1.37
1.82 1.72
2.91 2.89
3.55 4.02

Graphs: 1 key, 2 keys, 6 keys, 10 keys

SQL Query Benchmarks

Throughput, operations/sec
Bigger number represents better throughput
Benchmark Ignite Hazelcast Delta
SQL Query 65,657 39,121 40%
SQL Query Put 56,120 48,760 13%
Latency, milliseconds
Smaller number represents better latency
Ignite Hazelcast
0.97 1.63
1.14 1.31

Graphs: SQL query, SQL query put