Introduction
Amazon Bedrock is a fully managed AWS service that provides access to foundation models (FMs) from leading AI providers through a unified API, along with capabilities for building and deploying generative AI applications.
OpsRamp discovers and monitors Amazon Bedrock resources, including foundation models, custom models, provisioned throughput, agents, knowledge bases, guardrails, flows, and prompts. It also collects Amazon CloudWatch metrics related to model, agent, and guardrail invocations, providing visibility into resource health and usage.
Use the AWS Public Cloud integration to discover Amazon Bedrock resources and collect associated performance and operational metrics.
Setup
To set up the AWS integration and discover Bedrock resources, do the following:
- Create an AWS Integration if not available in your installed integrations.
- Create or edit a discovery profile.
- Select Bedrock resource types in the AWS Integration Discovery Profile.
- Save the discovery profile.
- Scan to discover resources at any time independent of the predefined schedule.
Metrics are collected from Amazon CloudWatch under the AWS/Bedrock, AWS/Bedrock/Agents, and AWS/Bedrock/Guardrails namespaces.
Ensure the integration IAM role grants:
bedrock:List*bedrock:Get*bedrock-agent:List*bedrock-agent:Get*cloudwatch:GetMetricData
Supported resource types
| Resource Type (Native Type) | Resource Tag | Description |
|---|---|---|
| AWS Bedrock Foundation Model | AWS_BEDROCK_MODEL | Managed foundation models available in Amazon Bedrock. |
| AWS Bedrock Custom Model | AWS_BEDROCK_CUSTOM_MODEL | Customer fine-tuned and customized models. |
| AWS Bedrock Provisioned Throughput | AWS_BEDROCK_PROVISIONED_THROUGHPUT | Provisioned model throughput allocations. |
| AWS Bedrock Agent | AWS_BEDROCK_AGENT | Bedrock agents that orchestrate FMs, data, and actions. |
| AWS Bedrock Agent Alias | AWS_BEDROCK_AGENT_ALIAS | Named, versioned pointers to a Bedrock agent. |
| AWS Bedrock Knowledge Base | AWS_BEDROCK_KNOWLEDGE_BASE | Knowledge bases used for retrieval-augmented generation. |
| AWS Bedrock Data Source | AWS_BEDROCK_DATA_SOURCE | Data sources ingested into a knowledge base. |
| AWS Bedrock Guardrail | AWS_BEDROCK_GUARDRAIL | Safeguards that filter harmful content and PII. |
| AWS Bedrock Flow | AWS_BEDROCK_FLOW | Bedrock flows that chain prompts, agents, and knowledge bases. |
| AWS Bedrock Flow Alias | AWS_BEDROCK_FLOW_ALIAS | Named, versioned pointers to a Bedrock flow. |
| AWS Bedrock Prompt | AWS_BEDROCK_PROMPT | Managed prompts stored in Prompt Management. |
Event support
CloudTrail event support
- Not supported for Bedrock resource-specific mapping.
- Monitoring is based on Amazon CloudWatch metrics and periodic discovery polling.
CloudWatch alarm support
- Not supported for AWS Bedrock resources.
Supported metrics
Foundation Model, Custom Model, and Provisioned Throughput
Namespace: AWS/Bedrock
| OpsRamp Metric | AWS Metric | Metric Display Name | Unit | Aggregation Type | Description |
|---|---|---|---|---|---|
| aws_bedrock_Invocations | Invocations | Invocations | Count | Sum | Number of model invocation requests. |
| aws_bedrock_InvocationLatency | InvocationLatency | Invocation Latency | Milliseconds | Average | Latency of model invocations. |
| aws_bedrock_InputTokenCount | InputTokenCount | Input Token Count | Count | Sum | Number of input tokens processed. |
| aws_bedrock_OutputTokenCount | OutputTokenCount | Output Token Count | Count | Sum | Number of output tokens generated. |
| aws_bedrock_InvocationClientErrors | InvocationClientErrors | Invocation Client Errors | Count | Sum | Number of client-side invocation errors. |
| aws_bedrock_InvocationServerErrors | InvocationServerErrors | Invocation Server Errors | Count | Sum | Number of server-side invocation errors. |
| aws_bedrock_InvocationThrottles | InvocationThrottles | Invocation Throttles | Count | Sum | Number of throttled invocations. |
Agent Alias
Namespace: AWS/Bedrock/Agents
| OpsRamp Metric | AWS Metric | Metric Display Name | Unit | Aggregation Type | Description |
|---|---|---|---|---|---|
| aws_bedrock_agents_InvocationCount | InvocationCount | Invocation Count | Count | Sum | Number of agent invocations. |
| aws_bedrock_agents_TotalTime | TotalTime | Total Time | Milliseconds | Average | Total execution time for agent invocations. |
| aws_bedrock_agents_TTFT | TTFT | TTFT | Milliseconds | Average | Time to first token. |
| aws_bedrock_agents_InputTokenCount | InputTokenCount | Input Token Count | Count | Sum | Number of input tokens processed. |
| aws_bedrock_agents_OutputTokenCount | OutputTokenCount | Output Token Count | Count | Sum | Number of output tokens generated. |
| aws_bedrock_agents_InvocationThrottles | InvocationThrottles | Invocation Throttles | Count | Sum | Number of throttled agent invocations. |
| aws_bedrock_agents_InvocationClientErrors | InvocationClientErrors | Invocation Client Errors | Count | Sum | Number of client-side invocation errors. |
| aws_bedrock_agents_InvocationServerErrors | InvocationServerErrors | Invocation Server Errors | Count | Sum | Number of server-side invocation errors. |
Guardrail
Namespace: AWS/Bedrock/Guardrails
| OpsRamp Metric | AWS Metric | Metric Display Name | Unit | Aggregation Type | Description |
|---|---|---|---|---|---|
| aws_bedrock_guardrails_Invocations | Invocations | Invocations | Count | Sum | Number of guardrail invocations. |
| aws_bedrock_guardrails_InvocationLatency | InvocationLatency | Invocation Latency | Milliseconds | Average | Latency of guardrail invocations. |
| aws_bedrock_guardrails_InvocationsIntervened | InvocationsIntervened | Invocations Intervened | Count | Sum | Number of invocations where guardrails intervened. |
| aws_bedrock_guardrails_TotalFindings | TotalFindings | Total Findings | Count | Sum | Total number of findings detected by guardrails. |
| aws_bedrock_guardrails_TextUnitCount | TextUnitCount | Text Unit Count | Count | Sum | Number of text units processed by guardrails. |
| aws_bedrock_guardrails_InvocationClientErrors | InvocationClientErrors | Invocation Client Errors | Count | Sum | Number of client-side invocation errors. |
| aws_bedrock_guardrails_InvocationServerErrors | InvocationServerErrors | Invocation Server Errors | Count | Sum | Number of server-side invocation errors. |
| aws_bedrock_guardrails_InvocationThrottles | InvocationThrottles | Invocation Throttles | Count | Sum | Number of throttled invocations. |