Introduction

Amazon Kinesis Data Streams is used to collect and process large streams of data records in real time.

You can create data-processing applications, known as Kinesis Data Streams applications. A typical Kinesis Data Streams application reads data from a data stream as data records. These applications can use the Kinesis Client Library and run on Amazon EC2 instances.

You can:

  • Send the processed records to dashboards.
  • Use the record to generate alerts.
  • Dynamically change pricing and advertising strategies.
  • Send data to a variety of other AWS services.

Setup

To set up the OpsRamp AWS integration and discover the AWS services (Kinesis and Shard), go to AWS Integration Discovery Profile and select Kinesis.

Metrics

Standard metrics

OpsRamp MetricMetric Display NameUnitAggregation TypeDescription
aws_kinesis_PutRecord_BytesPutRecord.BytesBytesAverageNumber of bytes put to the Amazon Kinesis stream using PutRecord over the specified time period.
aws_kinesis_PutRecord_LatencyPutRecord.LatencymsAverageTime taken per PutRecord operation, measured over the specified time period.
aws_kinesis_PutRecord_SuccessPutRecord.SuccessCountAverageNumber of successful PutRecord operations per Amazon Kinesis stream, measured over the specified time period.
aws_kinesis_PutRecords_BytesPutRecords.BytesBytesAverageNumber of bytes put to the Amazon Kinesis stream using PutRecords over the specified time period.
aws_kinesis_PutRecords_LatencyPutRecords.LatencymsAverageTime taken per PutRecords operation, measured over the specified time period.
aws_kinesis_PutRecords_RecordsPutRecords.RecordsCountAverageNumber of successful records in a PutRecords operation per Amazon Kinesis stream, measured over the specified time period.
aws_kinesis_PutRecords_SuccessPutRecords.SuccessCountAverageNumber of successful PutRecords operations per Amazon Kinesis stream, measured over the specified time period.
aws_kinesis_IncomingBytesIncomingBytesBytesAverageNumber of bytes successfully put to the Amazon Kinesis stream over the specified time period. This metric includes bytes from PutRecord and PutRecords.
aws_kinesis_IncomingRecordsIncomingRecordsCountAverageNumber of records sucessfully put to the Amazon Kinesis stream over the specified time period. This metric includes record counts from PutRecord and PutRecords.
aws_kinesis_GetRecords_BytesGetRecords.BytesBytesAverageNumber of bytes retrieved from the stream, measured over the specified time period.
aws_kinesis_GetRecords_IteratorAgeMillisecondsGetRecords.IteratorAgeMillisecondsmsAverageAge of the last record in all GetRecords calls made against a stream, measured over the specified time period. Age is the difference between the current time and when the last record of a GetRecords call was written to the stream. Maximum statistic represents the oldest record that has been consumed over the specified time period, while Minimum represents the newest record.
aws_kinesis_GetRecords_LatencyGetRecords.LatencymsAverageTime taken per GetRecords operation, measured over the specified time period.
aws_kinesis_GetRecords_SuccessGetRecords.SuccessCountAverageNumber of successful GetRecords operations per stream, measured over the specified time period.
aws_kinesis_GetRecords_RecordsGet Records RecordsCountAverageNumber of records retrieved from the shard, measured over the specified time period.
aws_kinesis_ReadProvisionedThroughputExceededRead Provisioned Throughput ExceededCountAverageNumber of GetRecords calls throttled for the stream over the specified time period.
aws_kinesis_WriteProvisionedThroughputExceededWrite Provisioned Throughput ExceededCountAverageNumber of records rejected due to throttling for the stream over the specified time period.

Shard metrics

OpsRamp MetricMetric Display NameUnitAggregation TypeDescription
aws_kinesis_IncomingBytesIncomingBytesBytesAverageNumber of bytes successfully put to the Amazon Kinesis stream over the specified time period. This metric includes bytes from PutRecord and PutRecords.
aws_kinesis_IncomingRecordsIncomingRecordsCountAverageNumber of records sucessfully put to the Amazon Kinesis stream over the specified time period. This metric includes record counts from PutRecord and PutRecords.
aws_kinesis_ReadProvisionedThroughputExceededRead Provisioned Throughput ExceededCountAverageNumber of GetRecords calls throttled for the stream over the specified time period.
aws_kinesis_WriteProvisionedThroughputExceededWrite Provisioned Throughput ExceededCountAverageNumber of records rejected due to throttling for the stream over the specified time period.
aws_kinesis_IteratorAgeMillisecondsIterator Age MillisecondsMillisecondsAverageThe age of the last record in all GetRecords calls made against a shard, measured over the specified time period.
aws_kinesis_OutgoingBytesOutgoing BytesBytesAverageNumber of bytes retrieved from the shard, measured over the specified time period.
aws_kinesis_OutgoingRecordsOutgoing RecordsCountAverageNumber of records retrieved from the shard, measured over the specified time period.

Event support

CloudTrail event support

  • Not Supported

CloudWatch alarm support

  • Supported
  • Configurable in OpsRamp AWS Integration Discovery Profile.

External reference