Amazon Kinesis Data Firehose is a fully managed service for delivering real-time streaming data to destinations such as Amazon Simple Storage Service (Amazon S3), Amazon Redshift, Amazon Elasticsearch Service (Amazon ES), and Splunk. Kinesis Data Firehose is part of the Kinesis streaming data platform, including Kinesis Data Streams, Kinesis Video Streams, and Amazon Kinesis Data Analytics. With Kinesis Data Firehose, you do not need to write applications or manage resources. You configure your data producers to send data to Kinesis Data Firehose, and it automatically delivers the data to the destination that you specified. You can also configure Kinesis Data Firehose to transform your data before delivering it.

Use the OpsRamp AWS public cloud integration to discover and collect metrics against the AWS service.

Setup

To set up the OpsRamp AWS integration and discover the AWS service, go to AWS Integration Discovery Profile and select Firehose.

Metrics

OpsRamp MetricMetric Display NameUnitAggregation Type
aws_firehose_PutRecord_Bytes

Number of bytes put to the Amazon Kinesis stream using PutRecord over the specified time period.
PutRecord.BytesBytesAverage
aws_firehose_PutRecord_Latency

Time taken per PutRecord operation, measured over the specified time period.
PutRecord.LatencymsAverage
aws_firehose_IncomingBytes

Number of bytes successfully put to the Amazon Kinesis stream over the specified time period. This metric includes bytes from PutRecord and PutRecords.
IncomingBytesBytesAverage
aws_firehose_IncomingRecords

Number of records sucessfully put to the Amazon Kinesis stream over the specified time period. This metric includes record counts from PutRecord and PutRecords.
IncomingRecordsCountAverage
aws_firehose_DeliveryToElasticsearch_Bytes

Number of bytes indexed to Amazon ES over the specified time period.
DeliveryToElasticsearch.Bytes.firehoseBytesAverage
aws_firehose_DeliveryToElasticsearch_Records

Number of records indexed to Amazon ES over the specified time period.
DeliveryToElasticsearch.Records.firehoseCountAverage
aws_firehose_DeliveryToElasticsearch_Success

Sum of the successfully indexed records over the sum of records that were attempted.
DeliveryToElasticsearch.Success.firehoseCountAverage
aws_firehose_DeliveryToRedshift_Bytes

Number of bytes copied to Amazon Redshift over the specified time period.
DeliveryToRedshift.Bytes.firehoseBytesAverage
aws_firehose_DeliveryToRedshift_Records

Number of records copied to Amazon Redshift over the specified time period.
DeliveryToRedshift.Records.firehoseCountAverage
aws_firehose_DeliveryToRedshift_Success

Sum of successful Amazon Redshift COPY commands over the sum of all Amazon Redshift COPY commands.
DeliveryToRedshift.Success.firehoseCountAverage
aws_firehose_DeliveryToRedshift_Success

Number of bytes delivered to Amazon S3 over the specified time period.
DeliveryToS3.Bytes.firehoseBytesAverage
aws_firehose_DeliveryToS3_DataFreshness

Age (from getting into Kinesis Firehose to now) of the oldest record in Kinesis Firehose. Any record older than this age has been delivered to the S3 bucket.
DeliveryToS3.DataFreshness.firehoseSecondsAverage
aws_firehose_DeliveryToS3_Records

Number of records delivered to Amazon S3 over the specified time period.
DeliveryToS3.Records.firehoseCountAverage
aws_firehose_DeliveryToS3_Success

Sum of successful Amazon S3 put commands over the sum of all Amazon S3 put commands.
DeliveryToS3.Success.firehoseCountAverage
aws_firehose_DescribeDeliveryStream_Latency

Time taken per DescribeDeliveryStream operation, measured over the specified time period.
DescribeDeliveryStream.Latency.firehoseMillisecondsAverage
aws_firehose_DescribeDeliveryStream_Requests

Total number of DescribeDeliveryStream requests.
DescribeDeliveryStream.Requests.firehoseCountAverage
aws_firehose_ListDeliveryStreams_Latency

Time taken per ListDeliveryStream operation, measured over the specified time period.
ListDeliveryStreams.Latency.firehoseMillisecondsAverage
aws_firehose_ListDeliveryStreams_Requests

Total number of ListFirehose requests.
ListDeliveryStreams.Requests.firehoseCountAverage
aws_firehose_PutRecord_Requests

Total number of PutRecord requests, which is equal to total number of records from PutRecord operations.
PutRecord.Requests.firehoseCountAverage
aws_firehose_PutRecordBatch_Bytes

Number of bytes put to the Kinesis Firehose delivery stream using PutRecordBatch over the specified time period.
PutRecordBatch.Bytes.firehoseBytesAverage
aws_firehose_PutRecordBatch_Latency

Time taken per PutRecordBatch operation, measured over the specified time period.
PutRecordBatch.Latency.firehoseMillisecondsAverage
aws_firehose_PutRecordBatch_Records

Total number of records from PutRecordBatch operations.
PutRecordBatch.Records.firehoseCountAverage
aws_firehose_PutRecordBatch_Requests

Total number of PutRecordBatch requests.
PutRecordBatch.Requests.firehoseCountAverage
aws_firehose_UpdateDeliveryStream_Requests

Total number of UpdateDeliveryStream requests.
UpdateDeliveryStream.Requests.firehoseCountAverage
aws_firehose_UpdateDeliveryStream_Latency

Time taken per UpdateDeliveryStream operation, measured over the specified time period.
UpdateDeliveryStream.Latency.firehoseMillisecondsAverage
aws_firehose_BackupToS3_Bytes

Number of bytes delivered to Amazon S3 for backup over the specified time period.
BackupToS3 BytesBytesAverage
aws_firehose_BackupToS3_DataFreshness

Age (from getting into Kinesis Data Firehose to now) of the oldest record in Kinesis Data Firehose. Any record older than this age has been delivered to the Amazon S3 bucket for backup. Kinesis Data Firehose emits this metric when data transformation is enabled for Amazon S3 or Amazon Redshift destinations.
BackupToS3 DataFreshnessSecondsAverage
aws_firehose_BackupToS3_Records

Number of records delivered to Amazon S3 for backup over the specified time period.
BackupToS3 RecordsCountAverage
aws_firehose_BytesPerSecondLimit

The current maximum number of bytes per second that a delivery stream can ingest before throttling.
Bytes Per Second LimitBytesAverage
aws_firehose_DataReadFromKinesisStream_Bytes

When the data source is a Kinesis data stream, this metric indicates the number of bytes read from that data stream.
Data Read From Kinesis Stream BytesBytesAverage
aws_firehose_DataReadFromKinesisStream_Records

When the data source is a Kinesis data stream, this metric indicates the number of records read from that data stream.
Data Read From Kinesis Stream RecordsCountAverage
aws_firehose_DeliveryToElasticsearch_DataFreshness

Age (from getting into Kinesis Data Firehose to now) of the oldest record in Kinesis Data Firehose.
Delivery To Elasticsearch Data FreshnessSecondsAverage
aws_firehose_DeliveryToSplunk_Bytes

Number of bytes delivered to Splunk over the specified time period.
Delivery To Splunk BytesBytesAverage
aws_firehose_DeliveryToSplunk_DataAckLatency

The approximate duration it takes to receive an acknowledgement from Splunk after Kinesis Data Firehose sends it data.
Delivery To Splunk Data Ack LatencySecondsAverage
aws_firehose_DeliveryToSplunk_DataFreshness

Age (from getting into Kinesis Data Firehose to now) of the oldest record in Kinesis Data Firehose. Any record older than this age has been delivered to Splunk.
Delivery To Splunk Data FreshnessSecondsAverage
aws_firehose_KinesisMillisBehindLatest

When the data source is a Kinesis data stream, this metric indicates the number of milliseconds that the last read record is behind the newest record in the Kinesis data stream.
Kinesis Millis Behind LatestMillisecondAverage
aws_firehose_RecordsPerSecondLimit

Current maximum number of records per second that a delivery stream can ingest before throttling.
Records Per Second LimitBytesAverage
aws_firehose_ThrottledRecords

Number of records that were throttled because data ingestion exceeded one of the delivery stream limits.
Throttled RecordsCountAverage
aws_firehose_IncomingPutRequests

Number of PutRecord and PutRecordBatch requests over the specified period of time.
Incoming Put RequestsCountAverage
aws_firehose_PutRequestsPerSecondLimit

Maximum number of put requests per second that a delivery stream can handle before throttling.
Put Requests Per Second LimitCountAverage
aws_firehose_ThrottledDescribeStream

Total number of times the DescribeStream operation is throttled when the data source is a Kinesis data stream.
Throttled Describe StreamCountAverage
aws_firehose_ThrottledGetRecords

Total number of times the GetRecords operation is throttled when the data source is a Kinesis data stream.
Throttled Get RecordsCountAverage
aws_firehose_ThrottledGetShardIterator

Total number of times the GetShardIterator operation is throttled when the data source is a Kinesis data stream.
Throttled Get Shard IteratorCountAverage
aws_firehose_KMSKeyAccessDenied

Number of times the service encounters a KMSAccessDeniedException for the delivery stream.
KMS Key Access DeniedCountAverage
aws_firehose_KMSKeyDisabled

Number of times the service encounters a KMSDisabledException for the delivery stream.
KMS Key DisabledCountAverage
aws_firehose_KMSKeyInvalidState

Number of times the service encounters a KMSInvalidStateException for the delivery stream.
KMS Key Invalid StateCountAverage
aws_firehose_KMSKeyNotFound

Number of times the service encounters a KMSNotFoundException for the delivery stream.
KMS Key Not FoundCountAverage

Event support

CloudTrail event support

  • Supported (CreateDeliveryStream, DeleteDeliveryStream)
  • Configurable in OpsRamp AWS Integration Discovery Profile.

CloudWatch alarm support

  • Supported
  • Configurable in OpsRamp AWS Integration Discovery Profile.

External reference