Amazon Kinesis Data Streams is used to collect and process large streams of data records in real-time.
You can create data-processing applications, known as Kinesis Data Streams applications. A typical Kinesis Data Streams application reads data from a data stream as data records. These applications can use the Kinesis Client Library and run on Amazon EC2 instances.
You can:
- Send the processed records to dashboards.
- Use the record to generate alerts.
- Dynamically change pricing and advertising strategies.
- Send data to a variety of other AWS services.
Use the AWS public cloud integration to discover and collect metrics against the AWS service.
External reference
What Is Amazon Kinesis Data Streams?
Setup
To set up the AWS integration and discover the AWS services (Kinesis and Shard), go to AWS Integration Discovery Profile and select Kinesis.
Event support
CloudTrail event support
- Not Supported
CloudWatch alarm support
- Supported
- Configurable in OpsRamp AWS Integration Discovery Profile.
Supported metrics
Standard metrics
| OpsRamp Metric | AWS Metric | Metric Display Name | Unit | Aggregation Type | Description |
|---|---|---|---|---|---|
| aws_kinesis_GetRecords_Bytes | GetRecords.Bytes | Get Records Bytes | Bytes | Average | The number of bytes retrieved from the Kinesis stream, measured over the specified time period. |
| aws_kinesis_GetRecords_IteratorAgeMilliseconds | GetRecords.IteratorAgeMilliseconds | Get Records Iterator Age Milliseconds | Milliseconds | Average | The age of the last record in all GetRecords calls made against a Kinesis stream, measured over the specified time period. |
| aws_kinesis_GetRecords_Latency | GetRecords.Latency | Get Records Latency | Milliseconds | Average | The time taken per GetRecords operation, measured over the specified time period. |
| aws_kinesis_GetRecords_Records | GetRecords.Records | Get Records Records | Count | Average | The number of records retrieved from the shard, measured over the specified time period. |
| aws_kinesis_GetRecords_Success | GetRecords.Success | Get Records Success | Count | Average | The number of successful GetRecords operations per stream, measured over the specified time period. |
| aws_kinesis_IncomingBytes | IncomingBytes | Incoming Bytes | Bytes | Average | The number of bytes successfully put to the Kinesis stream over the specified time period. |
| aws_kinesis_IncomingRecords | IncomingRecords | Incoming Records | Count | Average | The number of records successfully put to the Kinesis stream over the specified time period. This metric includes record counts from PutRecord and PutRecords operations. |
| aws_kinesis_PutRecord_Bytes | PutRecord.Bytes | Put Record Bytes | Bytes | Average | The number of bytes put to the Kinesis stream using the PutRecord operation over the specified time period. |
| aws_kinesis_PutRecord_Latency | PutRecord.Latency | Put Record Latency | Milliseconds | Average | The time taken per PutRecord operation, measured over the specified time period. |
| aws_kinesis_PutRecord_Success | PutRecord.Success | Put Record Success | Count | Average | The number of successful PutRecord operations per Kinesis stream, measured over the specified time period. |
| aws_kinesis_PutRecords_Bytes | PutRecords.Bytes | Put Records Bytes | Bytes | Average | The number of bytes put to the Kinesis stream using the PutRecords operation over the specified time period. |
| aws_kinesis_PutRecords_Latency | PutRecords.Latency | Put Records Latency | Milliseconds | Average | The time taken per PutRecords operation, measured over the specified time period. |
| aws_kinesis_PutRecords_Success | PutRecords.Success | Put Records Success | Count | Average | The number of PutRecords operations where at least one record succeeded, per Kinesis stream, measured over the specified time period. |
| aws_kinesis_PutRecords_TotalRecords | PutRecords.TotalRecords | Put Records Total Records | Count | Average | The total number of records sent in a PutRecords operation per Kinesis data stream, measured over the specified time period. |
| aws_kinesis_PutRecords_SuccessfulRecords | PutRecords.SuccessfulRecords | Put Records Successful Records | Count | Average | The number of successful records in a PutRecords operation per Kinesis data stream, measured over the specified time period. |
| aws_kinesis_PutRecords_FailedRecords | PutRecords.FailedRecords | Put Records Failed Records | Count | Average | The number of records rejected due to internal failures in a PutRecords operation per Kinesis data stream, measured over the specified time period. |
| aws_kinesis_PutRecords_ThrottledRecords | PutRecords.ThrottledRecords | Put Records Throttled Records | Count | Average | The number of records rejected due to throttling in a PutRecords operation per Kinesis data stream, measured over the specified time period. |
| aws_kinesis_ReadProvisionedThroughputExceeded | ReadProvisionedThroughputExceeded | Read Provisioned Throughput Exceeded | Count | Average | The number of GetRecords calls throttled for the stream over the specified time period. |
| aws_kinesis_SubscribeToShard_RateExceeded | SubscribeToShard.RateExceeded | Subscribe To Shard Rate Exceeded | None | Average | This metric is emitted when a new subscription attempt fails because there already is an active subscription by the same consumer or if you exceed the number of calls per second allowed for this operation. |
| aws_kinesis_SubscribeToShard_Success | SubscribeToShard.Success | Subscribe To Shard Success | None | Average | This metric records whether the SubscribeToShard subscription was successfully established. The subscription only lives for at most 5 minutes. Therefore, this metric is emitted at least once every 5 minutes. |
| aws_kinesis_SubscribeToShardEvent_Bytes | SubscribeToShardEvent.Bytes | Subscribe To Shard Event Bytes | Bytes | Average | The number of bytes received from the shard, measured over the specified time period. |
| aws_kinesis_SubscribeToShardEvent_MillisBehindLatest | SubscribeToShardEvent.MillisBehindLatest | Subscribe To Shard Event Millis Behind Latest | Milliseconds | Average | The number of milliseconds the read records are from the tip of the stream, indicating how far behind current time the consumer is. |
| aws_kinesis_SubscribeToShardEvent_Records | SubscribeToShardEvent.Records | Subscribe To Shard Event Records | Count | Average | The number of records received from the shard, measured over the specified time period. |
| aws_kinesis_SubscribeToShardEvent_Success | SubscribeToShardEvent.Success | Subscribe To Shard Event Success | Count | Average | This metric is emitted every time an event is published successfully. It is only emitted when there''s an active subscription. |
| aws_kinesis_WriteProvisionedThroughputExceeded | WriteProvisionedThroughputExceeded | Write Provisioned Throughput Exceeded | Count | Average | The number of records rejected due to throttling for the stream over the specified time period. |
Shard metrics
| OpsRamp Metric | AWS Metric | Metric Display Name | Unit | Aggregation Type | Description |
|---|---|---|---|---|---|
| aws_kinesis_IncomingBytes | IncomingBytes | Incoming Bytes | Bytes | Average | The number of bytes successfully put to the shard over the specified time period. This metric includes bytes from PutRecord and PutRecords operations. |
| aws_kinesis_IncomingRecords | IncomingRecords | Incoming Records | Count | Average | The number of records successfully put to the shard over the specified time period. |
| aws_kinesis_IteratorAgeMilliseconds | IteratorAgeMilliseconds | Iterator Age Milliseconds | Milliseconds | Average | The age of the last record in all GetRecords calls made against a shard, measured over the specified time period. |
| aws_kinesis_OutgoingBytes | OutgoingBytes | Outgoing Bytes | Bytes | Average | The number of bytes retrieved from the shard, measured over the specified time period. |
| aws_kinesis_OutgoingRecords | OutgoingRecords | Outgoing Records | Count | Average | The number of records retrieved from the shard, measured over the specified time period. |
| aws_kinesis_ReadProvisionedThroughputExceeded | ReadProvisionedThroughputExceeded | Read Provisioned Throughput Exceeded | Count | Average | The number of GetRecords calls throttled for the shard over the specified time period. |
| aws_kinesis_WriteProvisionedThroughputExceeded | WriteProvisionedThroughputExceeded | Write Provisioned Throughput Exceeded | Count | Average | The number of records rejected due to throttling for the shard over the specified time period. |