Overview

The Batch export integration allows you to collect and automate the export of data, such as metrics, alerts, inventory, audit, to external storage systems like Amazon S3 or Azure Blob Storage. It enables data exports in JSON format, ensuring seamless compatibility with analytics platforms, reporting tools, and other IT systems.

In the event of an export failure, a list of failed exports is provided, ensuring data integrity and facilitating troubleshooting.

Prerequisites

The following prerequisites must be met before installing the Batch Export integration:

Permissions

The following permissions are required to install Batch Export integration:

CategoryPermission TypePermission Value
Install Batch Export integrationIntegrationManage
View Batch Export integrationIntegrationView
Delete Batch Export integrationIntegrationManage
Edit the configurationIntegrationEdit

You can export the following category types for the clients:

Category TypeDescription
AlertsEach client alert list will be exported into cloud storage.
AuditEach client or all clients audit data will be exported into cloud storage.
IntegrationsEach client / partner installed integration configuration data will be exported into cloud storage.
InventoryEach client-level inventory data will be exported into cloud storage.
MetricsEach client-level metric data will be exported into cloud storage.
TenantsUser can configure the export at partner level, and it displays partner details and all client details under the partner.
TicketsEach client-level ticket data will be exported into cloud storage.
UsageUsage collection data (metering usage data for billing purposes) will be exported at partner level into cloud storage.

The following batch export schedule types for different data types are available:

Schedule TypeApplicable Data Type(s)Description
SnapshotInventory, Integrations, and TenantsThis data export provides a snapshot view of the data at the instance of export generation into the Amazon S3 bucket folder or Azure Blob storage.
IncrementalAlerts, Ticket, Metric, Audit and UsageThe incremental batch export sends data for the past three months as the first batch export. Subsequent exports are incremental exports of updated records. You can schedule recurring or on-demand data export:
  • On-demand - Export provides a snapshot of the data at the time of the generation of the export.
  • Recurring - Export provides snapshots of the data at the chosen time.
When you request incremental data on-demand, the export generates a snapshot of the past three months.

Metric data export occurs every hour by default and cannot be customized. You can get metric data using the metric APIs.

How to install Batch Export integration?

  1. Select the Batch Export Add-on
  2. Install Batch Export integration

Select the Batch Export Add-On

  1. Navigate to Setup → Account → Partners.
  2. Search for the partner and click on partner name.
  3. Click SETTINGS in the upper-right corner of the screen.
  4. In ACCOUNT SETTINGS screen, click the ADD-ONS tab.
  5. Select Batch Exports in the ADD-ONS section.


  6. Click SAVE.

Install Batch Export integration

Here is how to install a batch export integration for the Alerts category type.

  • To install the integration for other types of export categories, refer Export Categories document for more details.

The OpsRamp Batch Export integration allows you to systematically export alert data to external storage solutions like Amazon S3 or Azure Blob Storage. Alerts play a crucial role in IT operations monitoring, helping you detect, diagnose, and resolve incidents in real-time.

The procedure is similar for other category types with minor configuration changes.

  1. Navigate to Setup → Account. The Account Details screen is displayed.

  2. Select the Integrations tile. The Installed Integrations screen is displayed, with all the installed applications.

  3. If you do not have any installed applications, you will be navigated to the Available Integrations screen. The Available Integrations screen displays all the available applications along with the newly created application with the version.
    Note: Search for Batch Export application using the search option available. Alternatively, search for Exports from All Categories option and select it.

  4. Click +ADD in Batch Export tile. The ADD BATCH EXPORT screen is displayed.


  5. Enter the following basic details:

    Mandatory fields
    GENERAL DETAILS
    Field NameField TypeDescription
    NameStringUnique name of the export.
    Category TypeDropdownType of data to export: Alerts.
    ClientDropdownSelect the client for whom you want to export the data.
    Export toDropdownSelect the integration: AWS S3 or Blob Storage.
    If the integration is not available, you can create one:
    1. Click anywhere in the dropdown and click +ADD.
      The ADD INTEGRATION window is displayed.


    2. Select an Integration type from the dropdown and enter the required information in the fields.
    • For AWS S3:
      1. Name: Name of the Integration.
      2. Access Key ID: Unique identifier for the Amazon S3 bucket.
      3. Secret access key: Key and secret generated from the portal.
      4. Confirm Secret access key: Re-enter the secret access key.
      5. Bucket Name: Name of the AWS S3 bucket for the export data.
      6. Base URI: Data location in the AWS S3 bucket.
        Example: https://s3.regionName.amazonaws.com.
      7. Click ADD. The integration is added.
    • For Blob Storage:
      1. Name: Enter the integration name.
      2. Storage account name: Azure Blob account name.
      3. Secret access key: Access key generated from the portal.
      4. Confirm Secret access key: Reenter the secret access key.
      5. Container name: Name of the Azure container for the export data.
      6. Base URI: Data location in the container.
        Example: https://portal.azure.com
      7. Click ADD. The integration is added.
    Failure Export NotificationCheckboxIf you enable this option, you will receive a notification if the export fails.

    SCHEDULE: Specify the frequency of data generation.

    Field NameField TypeDescription
    ON DEMANDSelectionGenerates exports when a request is created or rerun.
    DAILYSelectionGenerates export data daily at the specified time.
    WEEKLYSelectionGenerates export data weekly on the specified day and time.
    MONTHLYSelectionGenerates export data monthly on the specified day and time.

  6. Click FINISH. The integration is installed.

View Added Integrations

The integrations (AWS S3 or Blob Storage) added in the Export to field above are available under Installed Integrations:

AWS S3:

Azure Blob Storage:

  1. Search for AWS S3 or Blob Storage in the Installed Integrations screen and click on the respective tile.
  2. Search for the integration using the search option.
  3. Click on the integration name to view the details.
  • To install the integration for other types of export categories,, refer Export Categories document for more details.

View installed Batch Export integration

You can view the Batch Export installed integration details under
Setup → Account →Integrations → Batch Export.


The following table describes the columns:

Column NameDescription
NameName of the integration.
Added OnDate and time details of the added account. Information about the user who added the account is also displayed.
Updated OnDate and time details of the modified account. Information about the user who modified the account is also displayed.
Last ExportStatus of the last export: Queued, Completed, Failed.

View Batch Exports on AWS S3

You can view the generated batch exports in an AWS S3 bucket in the corresponding folders.
Example: The alerts export is stored in the alerts folder:


AWS stores the export files in S3 folders in JSON format. The export file name has the following encoding:

export file encoding

  • (a) schedule of batch export, recurring or on-demand
  • (b) batch export type
  • (c) unique client ID
  • (d) schedule starting timestamp
  • (e) schedule ending timestamp
  • (f) recurring export serial number

View Batch Export on Azure Blob

You can view the generated data exports in the Azure Blob container in the corresponding folders:

View Batch Export in Azure Blob

Azure Blob stores the export files in Azure Blob containers in JSON format. The export file name has the encoding as shown in the following figure:

Export Name Details
  • (A) schedule of batch export, recurring or on-demand
  • (B) batch export type
  • (C) unique client ID
  • (D) schedule starting timestamp
  • (E) schedule ending timestamp
  • (F) recurring export serial number