Setup Streaming Export

Overview

To direct the event stream to AWS Firehose, integrate OspRamp with EventBridge.

The Streaming Export feature allows streaming of live data to various third-party tools using AWS EventBridge through Export Integration and create Streaming Export. The stream data is managed with the edit and delete options and the exported data is viewable at the target locations.

Notes:

• The integration is available only at partner level.
• Streaming Export is accessible only with Stream Exports add-on is enabled.
• Streaming export only supports Alerts category data.

How to Create a Streaming Export?

Follow these steps to create a Streaming export:

1. Select the Stream Exports Add-on
2. Install an Export Integration
3. Configure a Streaming Export

Step 1 - Select the Stream Exports Add-on

Follow these steps to enable the Stream Exports Add-on:

1. Navigate to Setup > Accounts > Partners.
2. Search for the partner and click on partner name.
3. Click Edit.
4. In EDIT PARTNER screen, click Add Ons tab.
5. Select Stream Exports.

   

6. Click Save.

Step 2 - Install an Export Integration

Integrate third-party tools based on preferences. After successfully integrating with the console, create a stream export to route the live data to the target location in AWS EventBridge or to a third-party tool using a Webhook.

See AWS EventBridge Integration or Webhook Integration for more information.

Step 3 - Configure a Streaming Export

Follow these steps to configure a streaming export:

1. Click Setup > Account.

2. Select the Integrations tile.

3. The Installed Integrations screen is displayed, with all the installed applications.

4. If you do not have any installed applications, you will be navigated to the Available Integrations page. The Available Integrations page displays all the available applications along with the newly created application with the version.
   Note: Search for Streaming Export application using the search option available. Alternatively, search for Exports from All Categories option and select it.

5. Click +ADD in Streaming Export tile.

   

6. Enter the following information:

   

   GENERAL DETAILS

   Field Name	Field Type	Description
   Name	String	Unique name of the export.
   Category Type	Dropdown	Type of data for export. Select Alerts or Logs. Based on the Category Type selected, the input fields change. For Category Type as Alerts: clients: Select All or select_clients. If you select All, the data will be exported to all clients under the partner. If you select select_clients, the data will be exported to the selected clients. Select clients from the Select Clients dropdown.
   Export to	Dropdown	Specify the integration: AWS EventBridge or Webhook. If the integrations are not available, you can create them: Click +ADD in the Export to dropdown. The ADD INTEGRATION window is displayed. Select Integration type from the dropdown. For AWS EventBridge Name: Enter the integration name. Access Key: Unique Identifier to access the AWS EventBridge. Secret key: Key generated from the AWS portal. Confirm Secret key: Secret generated from the AWS portal. Region name: Name of the cloud storage location. Event Bus Name: Event bus that receives the events from OpsRamp and AWS services. Event Source: Event pattern or text specified in the Rules section in EventBridge. Use this to filter the incoming events. Example: { "Source": [ "pattern" ] }. Detail Type: Parameter to do more filtering in EventBridge. Click ADD. For more information, see AWS Supported Services. For Webhook Under CONFIGURATION: Name: Unique name for the integration. Webhook URL: Provide the client (INBOUND) webhook URL. Example: https://uat.opsramp.net/integrations/alertsWebhook/{tenantId}/alerts?vtoken={vtoken} HEADERS: Select values from Name and Value dropdowns. Name: Select a Header name. Default header name is Content-Type. Value: Select a Header value. Default header value is application/json. Click +ADD to add more headers. OpsRamp supports only JSON content-type. Payload: Enter a payload. Give a space and enter $ to get a list of tokens. Note: The following tokens do not work during the ingestion of raw alerts: `repeatCount`, `alertId`, `technology`, `healedTime`. To validate the payload: Click VALIDATE PAYLOAD. Hover over the icon for more information. Click ADD. For more information, see Webhook Integration.
   Send raw events	Checkbox	Select to send raw events to the destination.
   FILTER CRITERIA:	Select	Use the smart filter criteria to filter specific alerts. You can filter alert streaming by resource type, metric, state, alert type, etc. Click +QUERY and build OpsQL query to filter alerts. Example: resource.type = "VMware" This will filter alert streaming by resource type.

   Field Name	Field Type	Description
   Category Type	Dropdown	Type of data for export. Select Logs. For Category Type as Logs: clients: Select All or select_clients. If you select All, the data will be exported to all clients under the partner. If you select select_clients, the data will be exported to the selected clients. Select clients from the Select Clients dropdown.
   Http Endpoint	String	Enter the HTTP Endpoint. Note: You need to register for Splunk Cloud to get the endpoint details. Once registered, you will receive the necessary information you need to start sending data to Splunk Cloud. The email might provide a sample HTTP endpoint like the one below: https://{prd-p-j2qyn.splunkcloud.com}:8080/services/collector/raw
   Type	Dropdown	Select Splunk.
   Use SSL	Checkbox	Select the option for enhanced security.
   AUTH HEADER	Checkbox	Auth Header Key: Authorization. Auth Header Value: Provide the Token Value prefixed with the string "Splunk". Note: You can generate a new token from Settings > Data Inputs > HTTP Event Collector from Splunk cloud.

7. Click FINISH to apply the export.

View Streaming Export Data

View Streaming Export Data on AWS S3

Check the corresponding folder in the AWS S3 bucket to view the generated streaming export. Example: Alerts stream data is stored in the Alerts folder.