Batch export helps you incorporate OpsRamp-generated data about your environment into your overall data warehousing and data mining efforts.
As an administrator, you can do Snapshot and Incremental batch export for each client in the organization to Amazon AWS S3 and Microsoft Azure Blob Storage. Additionally, on-demand and recurring export schedules can be used to export data. You can do batch export for the following types of data:
- Ticket data
- Alert data
- Metric data
- Inventory data
- Usage data (This is introduced as Beta in 9.0.0)
Data is exported in JSON format. If applicable when exporting, a list of failed exports is provided.
Types of batch export
The following types of batch export are provided:
- Snapshot: Available for Inventory data.
- Incremental: Available for Alerts, Tickets, and Metrics data.
The snapshot batch export is generally larger in size than the incremental batch export. You can configure and send the types of batch export data to AWS S3 bucket or Azure Blob storage.
Snapshot batch export
Snapshot batch export is implemented only for Inventory data. This data export provides a snapshot view of the data at the instance of export generation into the Amazon S3 bucket folder or Azure Blob storage.
You can schedule data using one of the following methods:
- On-demand - Export provides a snapshot of the data at the time of the generation of the export.
- Recurring - Export provides snapshots of the data at the chosen time.
Incremental batch export
The incremental batch export sends previous three-month data as the first batch export. Every subsequent export is the incremental export of the updated records of the data. You can schedule the report at regular intervals. When you request incremental data on demand, the export generates a snapshot of the previous three months.
The following data types are supported for incremental export of client data:
Metric data export works every one hour by default and cannot be customized. You can achieve Metric data using the supported metric APIs.
Only a Partner is allowed to do batch exports. A partner user can export different types of data corresponding to each client in the organization. When a partner uses API to create a specific export, the new export is created for all clients who have not created the same export already through the user interface. You can also schedule batch exports using the Batch Export API.
Example: A partner creates inventory batch export that applies to client users A, B, and C within the organization. If the Inventory batch export schedule for client A already exists, API data export schedules are only created for clients B and C.
Admin wants a systematic export of client data and wants a retained copy of the data in his environment
In an organization, the admin runs API queries every 24 hours to get all the updates for inventory, alerts, tickets, metric data, and user data of the clients. The admin wants to automatically export all client data regularly and wants to maintain a copy of all the client data in their environment.
Solution: Data export using the On-Demand schedule.
You want a periodic dump of metric data from their operations monitoring product
You have a data warehouse that aggregates data from multiple tools to analyze and mine data. You want a periodic dump of metric data from their operations monitoring product. This data helps in the analysis of decision-making processes.
Example: To determine if more CPU or memory devices are required to be provisioned for the cloud resource.
Solution: Incremental data export using on-demand and on a recurring schedule.
Streaming export provides streaming of live event data to your target location (Example, AWS S3 bucket) using Amazon EventBridge or to a third-party tool using Webhook.
To confirm the occurrence of the data from OpsRamp, verify the target location. Using Streaming Export, the live alert data is streamed to the destination using Export Integration.
To direct the event stream to AWS Firehose, integrate OspRamp to EventBridge.
The Streaming Export feature streaming of live data to different third-party tools using AWS EventBridge using the Export Integration and Create Streaming Export. The stream data is managed with the edit and delete options and the exported data is viewable at the target locations.
- This feature is an optional module that is not enabled by default. For details, contact a OpsRamp Technical Account Manager.
- Stream Export is accessable only with the Stream Export add-on.
Next Step: Managing Stream Export