Amazon Managed Streaming for Apache Kafka (Amazon MSK) is a fully managed service that enables you to build and run applications that use Apache Kafka to process streaming data.
Amazon MSK provides the control-plane operations, such as those for creating, updating, and deleting clusters.
- You can use Apache Kafka data-plane operations, such as those for producing and consuming data.
- It runs open-source versions of Apache Kafka.
This means existing applications, tooling, and plugins from partners and the Apache Kafka community are supported without requiring changes to application code.
Use the AWS public cloud integration to discover and collect metrics against the AWS service.
External reference
Monitoring an Amazon MSK Cluster
Setup
To set up the AWS integration and discover the AWS service, go to AWS Integration Discovery Profile and select MSK Cluster.
Event support
CloudTrail event support
- Not Supported
CloudWatch alarm support
- Supported
- Configurable in OpsRamp AWS Integration Discovery Profile.
Supported metrics
| OpsRamp Metric | Metric Display Name | Unit | Aggregation Type |
|---|---|---|---|
| aws_kafka_CpuIdle Percentage of CPU idle time. | CpuIdle | Percent | Average |
| aws_kafka_CpuSystem Percentage of CPU in kernel space. | CpuSystem | Percent | Average |
| aws_kafka_CpuUser Percentage of CPU in user space. | CpuUser | Percent | Average |
| aws_kafka_KafkaAppLogsDiskUsed Percentage of disk space used for application logs. | KafkaAppLogsDiskUsed | Percent | Average |
| aws_kafka_KafkaDataLogsDiskUsed Percentage of disk space used for data logs. | KafkaDataLogsDiskUsed | Percent | Average |
| aws_kafka_MemoryBuffered Size in bytes of buffered memory for the broker. | MemoryBuffered | Bytes | Sum |
| aws_kafka_MemoryCached Size in bytes of cached memory for the broker. | MemoryCached | Bytes | Sum |
| aws_kafka_MemoryFree Size in bytes of memory that is free and available for the broker. | MemoryFree | Bytes | Sum |
| aws_kafka_MemoryUsed Size in bytes of memory that is in use for the broker. | MemoryUsed | Bytes | Sum |
| aws_kafka_NetworkRxDropped Number of dropped receive packages. | NetworkRxDropped | Count | Sum |
| aws_kafka_NetworkRxErrors Number of network receive errors for the broker. | NetworkRxErrors | Count | Sum |
| aws_kafka_NetworkRxPackets Number of packets received by the broker. | NetworkRxPackets | Count | Sum |
| aws_kafka_NetworkTxDropped Number of dropped transmit packages. | NetworkTxDropped | Count | Sum |
| aws_kafka_NetworkTxErrors Number of network transmit errors for the broker. | NetworkTxErrors | Count | Sum |
| aws_kafka_NetworkTxPackets Number of packets transmitted by the broker. | NetworkTxPackets | Count | Sum |
| aws_kafka_RootDiskUsed Percentage of the root disk used by the broker. | RootDiskUsed | Percent | Average |
| aws_kafka_SwapFree Size in bytes of swap memory that is available for the broker. | SwapFree | Bytes | Sum |
| aws_kafka_SwapUsed Size in bytes of swap memory that is in use for the broker. | SwapUsed | Bytes | Sum |
| aws_kafka_ZooKeeperRequestLatencyMsMean Mean latency in milliseconds for ZooKeeper requests from broker. | ZooKeeperRequestLatencyMsMean | Milliseconds | Average |
| aws_kafka_ZooKeeperSessionState Connection status of brokers ZooKeeper session. | ZooKeeperSessionState | Count | Average |