Overview
Amazon SageMaker is a fully managed machine learning service. With Amazon SageMaker, data scientists and developers can quickly and easily build and train machine learning models, and directly deploy them into a production-ready hosted environment.
SageMaker provides:
- An integrated Jupyter authoring notebook instance for easy access to your data sources for exploration and analysis, so you do not have to manage servers.
- A common machine learning algorithms that are optimized to run efficiently against extremely large data in a distributed environment.
With native support for bring-your-own-algorithms and frameworks, Amazon SageMaker offers flexible distributed training options that adjust to your specific workflows. Deploy a model into a secure and scalable environment by launching it with a single click from the Amazon SageMaker console. Training and hosting are billed by minutes of usage, with no minimum fees and no upfront commitments.
Amazon SageMaker Ground Truth
High-quality training datasets by using workers including machine learning to create labeled datasets.
See GroundTruth metrics.
Amazon SageMaker Training
An Amazon SageMaker training job is an iterative process that teaches a model to make predictions by presenting examples from a training dataset. Typically, a training algorithm computes several metrics, such as training error and prediction accuracy. These metrics help diagnose whether the model is learning well and will generalize well for making predictions on unseen data. The training algorithm writes the values of these metrics to logs, which Amazon SageMaker monitors and sends to Amazon CloudWatch in real-time.
See Training metrics.
Amazon SageMaker Endpoint
Creates an endpoint using the endpoint configuration specified in the request. Amazon SageMaker uses the endpoint to provision resources and deploy models.
Amazon SageMaker Transform Job
Use batch transform when you need to do the following:
- Preprocess datasets to remove noise or bias that interferes with training or inference from your dataset.
- Get inferences from large datasets.
- Run inference when you do not need a persistent endpoint.
- Associate input records with inferences to help interpretation results.
External reference
Setup
To set up the integration:
- Select SageMaker GroundTruth in AWS Integration Discovery Profile to discover AWS SageMaker GroundTruth.
- Select SageMaker Training in AWS Integration Discovery Profile to discover AWS SageMaker Training Job.
- Select SageMaker EndPoint in AWS Integration Discovery Profile to discover AWS SageMaker Endpoint.
- Select SageMaker Transform Job in AWS Integration Discovery Profile to discover AWS SageMaker Transform Job.
Event support
CloudTrail event support
- Supported (Sagemaker GroundTruth, Training, Endpoint, Transform Job)
- Configurable in OpsRamp AWS Integration Discovery Profile.
CloudWatch alarm support
- Not Supported
Supported metrics
GroundTruth metrics
| OpsRamp Metric | AWS Metric | Metric Display Name | Unit | Aggregation Type | Description |
|---|---|---|---|---|---|
| aws_sagemaker_labelingjobs_ActiveWorkers | ActiveWorkers | Active Workers | None | Sum | A single active worker on a private work team submitted, released, or declined a task. Use Sum to get the total number of active workers |
| aws_sagemaker_labelingjobs_DatasetObjectsAutoAnnotated | DatasetObjectsAutoAnnotated | Dataset Objects Auto Annotated | None | Maximum | The number of dataset objects auto-annotated in a labeling job. Only emitted when automated labeling is enabled. Use Max to view labeling job progress |
| aws_sagemaker_labelingjobs_DatasetObjectsHumanAnnotated | DatasetObjectsHumanAnnotated | Dataset Objects Human Annotated | None | Maximum | The number of dataset objects annotated by a human in a labeling job. Use Max to view labeling job progress |
| aws_sagemaker_labelingjobs_DatasetObjectsLabelingFailed | DatasetObjectsLabelingFailed | Dataset Objects Labeling Failed | None | Maximum | The number of dataset objects that failed labeling in a labeling job. Use Max to view labeling job progress |
| aws_sagemaker_labelingjobs_JobsFailed | JobsFailed | Jobs Failed | None | Sum | A single labeling job failed. Use Sum to get the total number of failed labeling jobs |
| aws_sagemaker_labelingjobs_JobsSucceeded | JobsSucceeded | Jobs Succeeded | None | Sum | A single labeling job succeeded. Use Sum to get the total number of succeeded labeling jobs |
| aws_sagemaker_labelingjobs_JobsStopped | JobsStopped | Jobs Stopped | None | Sum | A single labeling job was stopped. Use Sum to get the total number of stopped labeling jobs |
| aws_sagemaker_labelingjobs_TasksAccepted | TasksAccepted | Tasks Accepted | None | Sum | A single task was accepted by a worker. Use Sum for total accepted tasks |
| aws_sagemaker_labelingjobs_TasksDeclined | TasksDeclined | Tasks Declined | None | Sum | A single task was declined by a worker. Use Sum for total declined tasks |
| aws_sagemaker_labelingjobs_TasksReturned | TasksReturned | Tasks Returned | None | Sum | A single task was returned. Use Sum for total returned tasks |
| aws_sagemaker_labelingjobs_TasksSubmitted | TasksSubmitted | Tasks Submitted | None | Sum | A single task was submitted/completed by a private worker. Use Sum for total submitted tasks |
| aws_sagemaker_labelingjobs_TimeSpent | TimeSpent | Time Spent | Seconds | Sum | Time spent on a task completed by a private worker. Does not include time when a worker paused or took a break |
| aws_sagemaker_labelingjobs_TotalDatasetObjectsLabeled | TotalDatasetObjectsLabeled | Total Dataset Objects Labeled | None | Maximum | The number of dataset objects labeled successfully in a labeling job. Use Max to view labeling job progress |
Training metrics
| OpsRamp Metric | AWS Metric | Metric Display Name | Unit | Aggregation Type | Description |
|---|---|---|---|---|---|
| aws_sagemaker_trainingjobs_CPUUtilization | CPUUtilization | CPU Utilization | Percent | Average | The sum of each individual CPU core utilization. Each core ranges 0-100. For processing jobs: CPU utilization of the processing container |
| aws_sagemaker_trainingjobs_MemoryUtilization | MemoryUtilization | Memory Utilization | Percent | Average | The percentage of memory used by the containers on an instance. Range 0%-100% |
| aws_sagemaker_trainingjobs_GPUUtilization | GPUUtilization | GPU Utilization | Percent | Average | The percentage of GPU units used by the containers on an instance. Range 0-100 multiplied by the number of GPUs |
| aws_sagemaker_trainingjobs_GPUMemoryUtilization | GPUMemoryUtilization | GPU Memory Utilization | Percent | Average | The percentage of GPU memory used by the containers on an instance. Range 0-100 multiplied by number of GPUs |
| aws_sagemaker_trainingjobs_DiskUtilization | DiskUtilization | Disk Utilization | Percent | Average | The percentage of disk space used by the containers on an instance. Range 0%-100% |