An alert escalation policy can be created to support operational requirements. For example, a policy can be created to escalate critical network issues to a network engineer.
Alert escalation policies are created by:
- Defining the name and scope.
- Selecting the resources where the alert matches this policy.
- Defining alert conditions.
- Defining the escalation rules.
Both OpsQ View and OpsQ Manage permissions are required to access the Alert Escalation Policy.
A training file is required to suppress specific alerts or to snooze specific alerts. The training file must include examples of alerts that need to be suppressed. See Alert Managament Training File for more information.
Step 1: Define name and scope
To define the name and scope:
- From All Clients, select the client.
- Go to Setup > Alert Management > Alert Escalation.
- Click the Add button and provide Name and Description.
- Select the required mode from the Mode drop-down list.
- Choose the organization whose users from the organization who will receive the escalations from this policy.
For example: If you choose a partner organization, only partner users can receive alert escalations.
- Select Next: Select Resources.
Step 2: Select resources
Select the resources where alerts match a policy:
- Select resources for the escalation policy.
- Resources can be selected from one or more clients.
- Add up to 25 resources.
- To escalate alerts for users of a specific client, add only resources from that client.
- (Optional) Filter the resources by:
- Resource Name
- Resource Type
- Service Group
- Device Group
- Select Next: Define Alert Conditions.
When choosing a parent service group or parent resource group, select all of the resources in its child group that form the escalation policy.
Step 3: Define alert and resource conditions
Filter the type of alerts which occur on the selected resources.
If no conditions are defined in this section, all alerts on the selected resources will match this policy.
To define the alert conditions:
- Add conditions for an alert based on the alert properties listed.
- Click Add to add multiple conditions.
- Choose if Any or All conditions should be applied to filter the alerts.
- Click Next: Define Escalation Rules.
The alert property Client: Service Name is applicable only to the partner scope policy. As an example, to escalate alerts of a service named Windows Maintenance Support, add the condition Client: Service Name:
Step 4: Define escalation rules
Escalation rules are actions to be taken when an alert is not acknowledged within a desired time period.
List of escalation rules along with the description:
|Escalate directly as needed||Select the users who should be directly contacted on an on-demand basis. Escalating directly is a manual process. No automatic actions are taken.|
|Escalate alert automatically until it is acknowledged, closed, suppressed, or ticketed||Escalate an alert as notification until an action is taken. No further level of escalation can be added since the alert escalation will end as soon as the incident is created.|
|Escalate alert automatically until it is acknowledged, closed, or suppressed||Escalate an alert as notification even if an incident is created.|
- Alert Elapsed Timeline: Configure the time interval to escalate an alert after it is generated
Select Immediately on the timeline to escalate an alert as soon as the alert is generated. If the alert does not have correlation or first response policies, the algorithm immediately moves the alert to Alert Escalation. If the alert does have correlation and first response policies, the algorithm starts the escalation process and finishes after the correlation is complete.
Select Wait Minutes/Hours on the timeline to escalate after the alert has elapsed and the policy conditions have been met.
- Escalate as Notification: Select to send continuous notifications until an action
(acknowledge, suppress, close or create Incident) is taken on the alert.
For example, if an alert arrives at 10:00 AM, a notification is sent to the user. If the user does not acknowledge the alert, another notification is sent after 5 minutes (10:05 AM). Notifications are continuously sent every 5 minutes. When the user acknowledges the alert at 10:20 AM, no further notifications are sent.
- Escalate as Incident: Select to escalate the alert as an incident. The escalation stops immediately and no more escalations are sent.
Escalate as notification
- Select Escalate As Notifications to send periodic notifications to users to make sure that an alert is acknowledged.
- Send Alert Notifications to: Configure the users who be sent the notifications.
To select the users, click Select Users.
- Notification Priority: An escalation notification carries a priority to determine which channel for delivery of notifications.
As an example, policy P1 is configured to send Normal priority notifications for all the matching alerts to user A. User A can specify that all normal notifications are received using email (and not using SMS or Voice).
- Send Notifications:
- Repeat Notification Frequency: Configure repeat notification frequency for selected users.
The repeat notification is sent to the user as Set Repeat Frequency even if an alert is escalated
to a higher level. The user will no longer receive notifications after a certain number of repeated notifications.
- The default repeat frequency is 15 minutes and the number of notifications is 2.
- The minimum number of notifications is 2 and maximum is 10.
- Alert State Transition: Alerts can be escalated based on the alert state transition which allows notifications on selected alert state change. For example, user can escalate alerts as a notification when alert state changes from warning to critical. The notification is a one-time notification sent at the time of the state transition.
- Repeat Notification Frequency: Configure repeat notification frequency for selected users. The repeat notification is sent to the user as Set Repeat Frequency even if an alert is escalated to a higher level. The user will no longer receive notifications after a certain number of repeated notifications.
- Add Escalation: Adding escalation levels will escalate the alert to the next level if it is not acknowledged at a prior level.
- A policy can have multiple escalation notifications.
- When a level 2 escalation is added to notify users, level 1 and level 2 users receive repeated notification according to the repeat notification frequency.
Escalate as incident
Select Escalate As Incident to automatically create an incident from an alert and assign it to the desired user.
For New Incidents: Configure the properties of the incident that is created when an alert’s condition matches this policy.
- A new incident is created for an alert if there is no open incident existing for the alert.
- The incident property tokens available in the auto incident form can be used to customize the subject and description of the incident.
- Enable Continous Learning to automatically escalate incidents to appropriate groups, priority or category using machine learning.
For Created Incidents: If an open incident (an incident in any state other than Closed) is available for the alert, the incident is updated instead of creating a new incident.
Update Incident: Configure how to update existing incident of an escalated alert. The incident gets updated every time an escalated alert repeats with a state change.
Update incident with latest alert description when alert state change: The latest alert description is appended to the conversation of the incident. No change to the status of the incident is made based on escalated alert’s state.
- Resolve incident when an alert heals: The latest alert description is appended to the conversation of the incident. The incident status is changed to Resolved when an escalated alert’s state changes to OK.
- Update incident priority based on these rules: The priority of an incident is updated according to the rule configured for the alert severity change.
Notify: Configure which updates to an incident should be notified to users.
- For every alert update: Send an incident notification when any update of an escalated alert is appended to the incident.
- Only when the alert state changed: Send incident notification only when escalated alert heals and the update is appended to the incident.
Review - alert escalation policy
A summary of all sections of the escalate alert policy is available for review and editing: