Monitoring with JMX
You can monitor Hazelcast members with JMX. When you enable JMX, each member registers MBeans for its distributed data structures and its internal services. Any JMX client, such as JConsole, can read the attributes of these MBeans and call their operations.
Hazelcast also exposes its metrics over JMX. The metrics MBeans do not need the hazelcast.jmx property. The jmx element of the metrics configuration controls them, and it is enabled by default.
Enable JMX
To monitor a member with JMX, enable the JMX agent of the member's JVM, and then enable JMX in Hazelcast.
Start the JVM with the following system properties to enable the JMX agent:
| System property | Effect |
|---|---|
-Dcom. | Enables the JMX agent |
-Dcom. | Optional. Accepts remote connections on the given port. |
-Dcom. | Optional. Turns off password authentication of JMX connections. |
Then set the hazelcast.jmx property to true. The default is false.
- XML
- YAML
- Java
- Spring
- System Property
<hazelcast>
...
<properties>
<property name="hazelcast.jmx">true</property>
</properties>
...
</hazelcast>
hazelcast:
properties:
hazelcast.jmx: true
config.setProperty("hazelcast.jmx", "true");
<hz:properties>
<hz:property name="hazelcast.jmx">true</hz:property>
</hz:properties>
-Dhazelcast.jmx=true
MBean names
Each Hazelcast MBean has an object name in the com.hazelcast domain. The object name has the following keys:
| Key | Value |
|---|---|
instance | The name of the Hazelcast instance |
type | The type of the MBean, such as IMap or Hazelcast |
name | The name of the data structure or service |
For example, the following object name identifies the MBean of the map trial on the instance _hzInstance_1_dev:
ObjectName mapMBeanName = new ObjectName("com.hazelcast:instance=_hzInstance_1_dev,type=IMap,name=trial");
If a value contains any of the characters :, ", ,, =, *, or ?, or a line break, Hazelcast quotes it with ObjectName.quote(). For example, the MBean of a map named orders:eu has the object name com.hazelcast:instance=_hzInstance_1_dev,type=IMap,name="orders:eu".
Connect to the JMX agent
To connect to a member, use a JMX client such as JConsole or VisualVM with the VisualVM-MBeans plugin. You can also connect from your own Java code.
To connect from Java code, first create a JMXServiceURL with the host of the member and the port that you set in com.sun.management.jmxremote.port:
int port = 1099;
String hostname = InetAddress.getLocalHost().getHostName();
JMXServiceURL url = new JMXServiceURL("service:jmx:rmi://" + hostname + ":" + port + "/jndi/rmi://" + hostname + ":" + port + "/jmxrmi");
Then connect to the URL, and get an MBeanServerConnection from the connection:
JMXConnector jmxc = JMXConnectorFactory.connect(url, null);
MBeanServerConnection mbsc = jmxc.getMBeanServerConnection();
Use the MBeanServerConnection to read attributes and call operations. The following example reads the name of a map and the number of its entries that the member owns:
ObjectName mapMBeanName = new ObjectName("com.hazelcast:instance=_hzInstance_1_dev,type=IMap,name=trial");
System.out.println("Total entries on map " + mbsc.getAttribute(mapMBeanName, "name") + " : "
+ mbsc.getAttribute(mapMBeanName, "localOwnedEntryCount"));
MBeans of a member
The following sections list the MBeans that a member registers. Each section lists the attributes and operations of one MBean. Operation names end with parentheses. Attributes whose names start with local describe only the member that you connect to, not the whole cluster. For the metrics behind many of these attributes, see List of Hazelcast metrics.
Instance (HazelcastInstance)
The HazelcastInstance MBean describes the member and its cluster. It has the following attributes and operations:
| Attribute or operation | Description |
|---|---|
name | Name of the instance |
version | Hazelcast version |
build | Hazelcast build |
config | Configuration of the member, as a string |
config | Source of the configuration |
cluster | Name of the cluster |
port | Network port |
cluster | Cluster-wide time |
member | Number of members in the cluster |
Members | List of members |
running | Whether the member is running |
shutdown() | Shuts down the member |
Node (HazelcastInstance.Node)
The HazelcastInstance.Node MBean has the following attributes:
| Attribute | Description |
|---|---|
address | Address of the member |
master | Address of the master member of the cluster |
Partition service (HazelcastInstance.PartitionServiceMBean)
The HazelcastInstance.PartitionServiceMBean MBean has the following attributes:
| Attribute | Description |
|---|---|
partition | Number of partitions |
active | Number of active partitions |
is | Whether the cluster is in a safe state |
is | Whether the member is safe to shut down |
Logging service (HazelcastInstance.LoggingService)
The HazelcastInstance.LoggingService MBean has the following attribute and operations:
| Attribute or operation | Description |
|---|---|
level | Log level that is set on the member |
set | Sets the log level of the member |
reset | Resets the log level of the member |
License (HazelcastInstance.LicenseInfo) Enterprise EditionThis feature requires Hazelcast Enterprise, a licensed edition of the Hazelcast Platform.
The HazelcastInstance.LicenseInfo MBean holds the details of the member's license. For its attributes, see Getting license details with the JMX API.
Internal executors (HazelcastInstance.ManagedExecutorService)
A member registers one HazelcastInstance.ManagedExecutorService MBean for each of the following internal executors. The name key of the object name is the name of the executor:
hz:systemhz:asynchz:scheduledhz:clienthz:client-queryhz:client-blocking-taskshz:queryhz:iohz:offloadable
Each executor MBean has the following attributes:
| Attribute | Description |
|---|---|
name | Name of the executor |
queue | Number of tasks in the work queue |
pool | Number of threads in the pool |
maximum | Maximum number of threads in the pool |
remaining | Remaining capacity of the work queue |
is | Whether the executor is shut down |
is | Whether the executor is terminated |
completed | Number of tasks that the executor has completed |
WAN publisher (WanPublisher) Enterprise EditionThis feature requires Hazelcast Enterprise, a licensed edition of the Hazelcast Platform.
A member registers one WanPublisher MBean for each WAN replication publisher. The name key of the object name is the name of the WAN replication and the ID of the publisher, separated by a period. The MBean has the following attribute:
| Attribute | Description |
|---|---|
state | State of the publisher: REPLICATING, PAUSED, or STOPPED |
Map (IMap)
A member registers an IMap MBean for each map. It has the following attributes and operations:
| Attribute or operation | Description |
|---|---|
name | Name of the map |
size | Number of entries in the map |
config | Configuration of the map |
local | Number of entries that the member owns |
local | Memory cost of the owned entries, in bytes |
local | Number of backup entries that the member holds |
local | Memory cost of the backup entries, in bytes |
local | Number of backups of each entry |
local | Time the map was created on the member |
local | Time an owned entry was last read |
local | Time an owned entry was last updated |
local | Number of reads of owned entries |
local | Number of owned keys that are locked |
local | Number of owned entries that are updated but not yet written to the map store |
local | Number of put operations |
local | Number of set operations |
local | Number of get operations |
local | Number of remove operations |
local | Total latency of put operations |
local | Total latency of set operations |
local | Total latency of get operations |
local | Total latency of remove operations |
local | Maximum latency of a put operation |
local | Maximum latency of a set operation |
local | Maximum latency of a get operation |
local | Maximum latency of a remove operation |
local | Number of events received |
local | Number of other operations |
local | Total number of operations |
local | Heap cost of the map, including its Near Cache |
clear() | Removes all entries from the map |
To get the average latency of an operation type, divide its total latency by its operation count.
MultiMap (MultiMap)
A member registers a MultiMap MBean for each MultiMap. It has the following attributes and operations:
| Attribute or operation | Description |
|---|---|
name | Name of the Multi |
size | Number of entries in the Multi |
config | Configuration of the Multi |
local | Number of entries that the member owns |
local | Memory cost of the owned entries, in bytes |
local | Number of backup entries that the member holds |
local | Memory cost of the backup entries, in bytes |
local | Number of backups of each entry |
local | Time the Multi was created on the member |
local | Time an owned entry was last read |
local | Time an owned entry was last updated |
local | Number of reads of owned entries |
local | Number of owned keys that are locked |
local | Number of owned entries that are dirty |
local | Number of put operations |
local | Number of get operations |
local | Number of remove operations |
local | Total latency of put operations |
local | Total latency of get operations |
local | Total latency of remove operations |
local | Maximum latency of a put operation |
local | Maximum latency of a get operation |
local | Maximum latency of a remove operation |
local | Number of events received |
local | Number of other operations |
local | Total number of operations |
clear() | Removes all entries from the Multi |
Replicated map (ReplicatedMap)
A member registers a ReplicatedMap MBean for each replicated map. It has the following attributes and operations:
| Attribute or operation | Description |
|---|---|
name | Name of the replicated map |
size | Number of entries in the replicated map |
config | Configuration of the replicated map |
local | Number of entries that the member owns |
local | Time the replicated map was created on the member |
local | Time an owned entry was last read |
local | Time an owned entry was last updated |
local | Number of reads of owned entries |
local | Number of put operations |
local | Number of get operations |
local | Number of remove operations |
local | Total latency of put operations |
local | Total latency of get operations |
local | Total latency of remove operations |
local | Maximum latency of a put operation |
local | Maximum latency of a get operation |
local | Maximum latency of a remove operation |
local | Number of events received |
local | Number of other operations |
local | Total number of operations |
clear() | Removes all entries from the replicated map |
Queue (IQueue)
A member registers an IQueue MBean for each queue. It has the following attributes and operations:
| Attribute or operation | Description |
|---|---|
name | Name of the queue |
config | Configuration of the queue |
partition | Partition key of the queue |
local | Number of items that the member owns |
local | Number of backup items that the member holds |
local | Minimum age of the items |
local | Maximum age of the items |
local | Average age of the items |
local | Number of offer, put, and add operations |
local | Number of rejected offers |
local | Number of poll, take, and remove operations |
local | Number of poll operations that returned null |
local | Number of other operations |
local | Number of event operations |
clear() | Removes all items from the queue |
List (IList)
A member registers an IList MBean for each list. It has the following attribute and operation:
| Attribute or operation | Description |
|---|---|
name | Name of the list |
clear() | Removes all items from the list |
Set (ISet)
A member registers an ISet MBean for each set. It has the following attributes and operation:
| Attribute or operation | Description |
|---|---|
name | Name of the set |
partition | Partition key of the set |
clear() | Removes all items from the set |
Topic (ITopic)
A member registers an ITopic MBean for each topic. It has the following attributes:
| Attribute | Description |
|---|---|
name | Name of the topic |
config | Configuration of the topic |
local | Time the topic was created on the member |
local | Number of messages published on the member |
local | Number of messages received on the member |
Reliable topic (ReliableTopic)
A member registers a ReliableTopic MBean for each reliable topic. It has the following attributes:
| Attribute | Description |
|---|---|
name | Name of the reliable topic |
config | Configuration of the reliable topic |
local | Time the reliable topic was created on the member |
local | Number of messages published on the member |
local | Number of messages received on the member |
Executor service (IExecutorService)
A member registers an IExecutorService MBean for each executor service. It has the following attributes:
| Attribute | Description |
|---|---|
name | Name of the executor service |
local | Time the executor service was created on the member |
local | Number of pending tasks |
local | Number of started tasks |
local | Number of completed tasks |
local | Number of cancelled tasks |
local | Total start latency of the started tasks |
local | Total execution time of the completed tasks |