Amazon Kinesis Data Firehose is a fully managed service for delivering real-time streaming data to destinations such as Amazon Simple Storage Service (Amazon S3), Amazon Redshift, Amazon Elasticsearch Service (Amazon ES), and Splunk. Kinesis Data Firehose is part of the Kinesis streaming data platform, including Kinesis Data Streams, Kinesis Video Streams, and Amazon Kinesis Data Analytics. With Kinesis Data Firehose, you do not need to write applications or manage resources. You configure your data producers to send data to Kinesis Data Firehose, and it automatically delivers the data to the destination that you specified. You can also configure Kinesis Data Firehose to transform your data before delivering it.
Use the AWS public cloud integration to discover and collect metrics against the AWS service.
External reference
What Is Amazon Kinesis Data Firehose?
Setup
To set up the AWS integration and discover the AWS service, go to AWS Integration Discovery Profile and select Firehose.
Event support
CloudTrail event support
- Supported (CreateDeliveryStream, DeleteDeliveryStream)
- Configurable in OpsRamp AWS Integration Discovery Profile.
CloudWatch alarm support
- Supported
- Configurable in OpsRamp AWS Integration Discovery Profile.
Supported metrics
| OpsRamp Metric | AWS Metric | Metric Display Name | Unit | Aggregation Type |
|---|---|---|---|---|
| aws_firehose_PutRecord_Bytes Number of bytes put to the Amazon Kinesis stream using PutRecord over the specified time period. | PutRecord.Bytes | PutRecord.Bytes | Bytes | Sum |
| aws_firehose_PutRecord_Latency Time taken per PutRecord operation, measured over the specified time period. | PutRecord.Latency | PutRecord.Latency | ms | Average |
| aws_firehose_IncomingBytes Number of bytes successfully put to the Amazon Kinesis stream over the specified time period. This metric includes bytes from PutRecord and PutRecords. | IncomingBytes | IncomingBytes | Bytes | Sum |
| aws_firehose_IncomingRecords Number of records successfully put to the Amazon Kinesis stream over the specified time period. This metric includes record counts from PutRecord and PutRecords. | IncomingRecords | IncomingRecords | Count | Sum |
| aws_firehose_DeliveryToElasticsearch_Bytes Number of bytes indexed to Amazon ES over the specified time period. | DeliveryToElasticsearch.Bytes | DeliveryToElasticsearch.Bytes.firehose | Bytes | Sum |
| aws_firehose_DeliveryToElasticsearch_Records Number of records indexed to Amazon ES over the specified time period. | DeliveryToElasticsearch.Records | DeliveryToElasticsearch.Records.firehose | Count | Sum |
| aws_firehose_DeliveryToElasticsearch_Success Sum of the successfully indexed records over the sum of records that were attempted. | DeliveryToElasticsearch.Success | DeliveryToElasticsearch.Success.firehose | Count | Sum |
| aws_firehose_DeliveryToRedshift_Bytes Number of bytes copied to Amazon Redshift over the specified time period. | DeliveryToRedshift.Bytes | DeliveryToRedshift.Bytes.firehose | Bytes | Sum |
| aws_firehose_DeliveryToRedshift_Records Number of records copied to Amazon Redshift over the specified time period. | DeliveryToRedshift.Records | DeliveryToRedshift.Records.firehose | Count | Sum |
| aws_firehose_DeliveryToRedshift_Success Sum of successful Amazon Redshift COPY commands over the sum of all Amazon Redshift COPY commands. | DeliveryToRedshift.Success | DeliveryToRedshift.Success.firehose | Count | Sum |
| aws_firehose_DeliveryToS3_DataFreshness Age (from getting into Kinesis Firehose to now) of the oldest record in Kinesis Firehose. Any record older than this age is delivered to the S3 bucket. | DeliveryToS3.DataFreshness | DeliveryToS3.DataFreshness.firehose | Seconds | Average |
| aws_firehose_DeliveryToS3_Records Number of records delivered to Amazon S3 over the specified time period. | DeliveryToS3.Records | DeliveryToS3.Records.firehose | Count | Sum |
| aws_firehose_DeliveryToS3_Success Sum of successful Amazon S3 put commands over the sum of all Amazon S3 put commands. | DeliveryToS3.Success | DeliveryToS3.Success.firehose | Count | Sum |
| aws_firehose_DescribeDeliveryStream_Latency Time taken per DescribeDeliveryStream operation, measured over the specified time period. | DescribeDeliveryStream.Latency | DescribeDeliveryStream.Latency.firehose | Milliseconds | Average |
| aws_firehose_DescribeDeliveryStream_Requests Total number of DescribeDeliveryStream requests. | DescribeDeliveryStream.Requests | DescribeDeliveryStream.Requests.firehose | Count | Average |
| aws_firehose_ListDeliveryStreams_Latency Time taken per ListDeliveryStream operation, measured over the specified time period. | ListDeliveryStreams.Latency | ListDeliveryStreams.Latency.firehose | Milliseconds | Average |
| aws_firehose_ListDeliveryStreams_Requests Total number of ListFirehose requests. | ListDeliveryStreams.Requests | ListDeliveryStreams.Requests.firehose | Count | Average |
| aws_firehose_PutRecord_Requests Total number of PutRecord requests, which is equal to total number of records from PutRecord operations. | PutRecord.Requests | PutRecord.Requests.firehose | Count | Sum |
| aws_firehose_PutRecordBatch_Bytes Number of bytes put to the Kinesis Firehose delivery stream using PutRecordBatch over the specified time period. | PutRecordBatch.Bytes | PutRecordBatch.Bytes.firehose | Bytes | Sum |
| aws_firehose_PutRecordBatch_Latency Time taken per PutRecordBatch operation, measured over the specified time period. | PutRecordBatch.Latency | PutRecordBatch.Latency.firehose | Milliseconds | Average |
| aws_firehose_PutRecordBatch_Records Total number of records from PutRecordBatch operations. | PutRecordBatch.Records | PutRecordBatch.Records.firehose | Count | Sum |
| aws_firehose_PutRecordBatch_Requests Total number of PutRecordBatch requests. | PutRecordBatch.Requests | PutRecordBatch.Requests.firehose | Count | Sum |
| aws_firehose_UpdateDeliveryStream_Requests Total number of UpdateDeliveryStream requests. | UpdateDeliveryStream.Requests | UpdateDeliveryStream.Requests.firehose | Count | Sum |
| aws_firehose_UpdateDeliveryStream_Latency Time taken per UpdateDeliveryStream operation, measured over the specified time period. | UpdateDeliveryStream.Latency | UpdateDeliveryStream.Latency.firehose | Milliseconds | Average |
| aws_firehose_BackupToS3_Bytes Number of bytes delivered to Amazon S3 for backup over the specified time period. | BackupToS3.Bytes | BackupToS3 Bytes | Bytes | Sum |
| aws_firehose_BackupToS3_DataFreshness Age (from getting into Kinesis Data Firehose to now) of the oldest record in Kinesis Data Firehose. Any record older than this age is delivered to the Amazon S3 bucket for backup. Kinesis Data Firehose emits this metric when data transformation is enabled for Amazon S3 or Amazon Redshift destinations. | BackupToS3.DataFreshness | BackupToS3 DataFreshness | Seconds | Average |
| aws_firehose_BackupToS3_Records Number of records delivered to Amazon S3 for backup over the specified time period. | BackupToS3.Records | BackupToS3 Records | Count | Sum |
| aws_firehose_BytesPerSecondLimit The current maximum number of bytes per second that a delivery stream can ingest before throttling. | BytesPerSecondLimit | Bytes Per Second Limit | Bytes | Average |
| aws_firehose_DataReadFromKinesisStream_Bytes When the data source is a Kinesis data stream, this metric indicates the number of bytes read from that data stream. | DataReadFromKinesisStream.Bytes | Data Read From Kinesis Stream Bytes | Bytes | Sum |
| aws_firehose_DataReadFromKinesisStream_Records When the data source is a Kinesis data stream, this metric indicates the number of records read from that data stream. | DataReadFromKinesisStream.Records | Data Read From Kinesis Stream Records | Count | Sum |
| aws_firehose_DeliveryToElasticsearch_DataFreshness Age (from getting into Kinesis Data Firehose to now) of the oldest record in Kinesis Data Firehose. | DeliveryToElasticsearch.DataFreshness | Delivery To Elasticsearch Data Freshness | Seconds | Average |
| aws_firehose_DeliveryToSplunk.Records Number of records delivered to Splunk over the specified time period. | DeliveryToSplunk.Records | Delivery To Splunk Records | Count | Sum |
| aws_firehose_DeliveryToSplunk_Bytes Number of bytes delivered to Splunk over the specified time period. | DeliveryToSplunk.Bytes | Delivery To Splunk Bytes | Bytes | Sum |
| aws_firehose_DeliveryToSplunk_DataAckLatency The approximate duration it takes to receive an acknowledgement from Splunk after Kinesis Data Firehose sends it data. | DeliveryToSplunk.DataAckLatency | Delivery To Splunk Data Ack Latency | Seconds | Average |
| aws_firehose_DeliveryToSplunk_DataFreshness Age (from getting into Kinesis Data Firehose to now) of the oldest record in Kinesis Data Firehose. Any record older than this age is delivered to Splunk. | DeliveryToSplunk.DataFreshness | Delivery To Splunk Data Freshness | Seconds | Average |
| aws_firehose_KinesisMillisBehindLatest When the data source is a Kinesis data stream, this metric indicates the number of milliseconds that the last read record is behind the newest record in the Kinesis data stream. | KinesisMillisBehindLatest | Kinesis Millis Behind Latest | Millisecond | Average |
| aws_firehose_RecordsPerSecondLimit Current maximum number of records per second that a delivery stream can ingest before throttling. | RecordsPerSecondLimit | Records Per Second Limit | Bytes | Average |
| aws_firehose_ThrottledRecords Number of records that were throttled because data ingestion exceeded one of the delivery stream limits. | ThrottledRecords | Throttled Records | Count | Average |
| aws_firehose_IncomingPutRequests Number of PutRecord and PutRecordBatch requests over the specified period of time. | IncomingPutRequests | Incoming Put Requests | Count | Average |
| aws_firehose_PutRequestsPerSecondLimit Maximum number of put requests per second that a delivery stream can handle before throttling. | PutRequestsPerSecondLimit | Put Requests Per Second Limit | Count | Average |
| aws_firehose_ThrottledDescribeStream Total number of times the DescribeStream operation is throttled when the data source is a Kinesis data stream. | ThrottledDescribeStream | Throttled Describe Stream | Count | Average |
| aws_firehose_ThrottledGetRecords Total number of times the GetRecords operation is throttled when the data source is a Kinesis data stream. | ThrottledGetRecords | Throttled Get Records | Count | Average |
| aws_firehose_ThrottledGetShardIterator Total number of times the GetShardIterator operation is throttled when the data source is a Kinesis data stream. | ThrottledGetShardIterator | Throttled Get Shard Iterator | Count | Average |
| aws_firehose_KMSKeyAccessDenied Number of times the service encounters a KMSAccessDeniedException for the delivery stream. | KMSKeyAccessDenied | KMS Key Access Denied | Count | Average |
| aws_firehose_KMSKeyDisabled Number of times the service encounters a KMSDisabledException for the delivery stream. | KMSKeyDisabled | KMS Key Disabled | Count | Average |
| aws_firehose_KMSKeyInvalidState Number of times the service encounters a KMSInvalidStateException for the delivery stream. | KMSKeyInvalidState | KMS Key Invalid State | Count | Average |
| aws_firehose_KMSKeyNotFound Number of times the service encounters a KMSNotFoundException for the delivery stream. | KMSKeyNotFound | KMS Key Not Found | Count | Average |