Overview

In addition to pod logs, the OpsRamp Kubernetes 2.0 Agent collects node-level logs from every node in your cluster through the same opsramp-logs-user-config ConfigMap used for pod logs.

Node logs are configured independently from pod logs. They use a different structure, a different severity ordering, and different defaults so settings you apply to the pod logs do not carry over to node logs, and vice versa.

Prerequisites

  • Log Management is enabled for your client.
  • The OpsRamp Kubernetes 2.0 Agent is installed and running.

Default configuration

By default, the agent collects syslogs from all nodes in the cluster, using this configuration (already present in opsramp-logs-user-config after installation):

node:
  - logSource: syslog
    enable: true
    configs:
      - mount_path: ["/var/log/syslog", "/var/log/messages"]
        multiline:
          line_start_pattern: '^\s*(?P<timestamp>\w*\s*\d*\s*\d*:\d*:\d*)\s*(?P<host>[^\s]*)'
        parser_type: "regex"
        parser_expression: '^\s*(?P<timestamp>\w*\s*\d*\s*\d*:\d*:\d*)\s*(?P<host>[^\s]*)\s*(?P<syslog_tag>[^:]*):\s*(?P<message>.*)$'
        timestamp_layout_type: "strptime"
        timestamp_layout: "%b %e %H:%M:%S"
        log_level: "Unspecified"

To disable syslog collection entirely, set enable to false under node:

kubectl edit configmap opsramp-logs-user-config -n <agent-installed-namespace>

node:
  - logSource: syslog
    enable: false

Field reference

All fields under node are user-configurable.

FieldDefaultDescription
logSourcesyslogThe designated source name used to identify collected logs.
enabletrueSet to false to disable syslog collection.
configsRequiredAn array of configuration objects that define the log parsing behavior.
mount_path["/var/log/syslog", "/var/log/messages"]Specifies the syslog file paths to monitor. Update this value if your syslog files are stored in a different location.
multiline → line_start_patternSee default aboveGroups related log lines into a single log entry using a start pattern. Specify exactly one of line_start_pattern or line_end_pattern.
parser_typeregexSpecifies the parser type. Supported values are regex and json.
parser_expressionSee default aboveDefines the expression used to parse each log entry into structured fields.
timestamp_layout_typestrptimeSpecifies the timestamp format type. Supported values are strptime, gotime, and epoch.
timestamp_layout"%b %e %H:%M:%S"Defines the exact timestamp format used to parse timestamps from log entries.

By default, all fields extracted by parser_expression are added as log attributes.

Adjusting the parser for your OS version

The default parser_expression is tuned for Ubuntu 22.04. If your nodes run a different OS version, you may need to adjust it.

For Ubuntu 24.0x, replace the default expression with:

parser_expression: '^(?P<timestamp>\d{4}\W*\d{2}\W*\d{2}\w\d{2}:\d{2}:\d{2}.\d*\+\d{2}:\d{2})\s(?P<host>\w*\W\w*)\s(?P<syslog_tag>\w*\W\d*\W):(?P<message>.*)'

Or alternatively:

parser_expression: '^\s*(?P<timestamp>\S+)\s+(?P<host>\S+)\s+(?P<syslog_tag>\S+):\s*(?P<message>.*)$'

If neither expression matches your logs, inspect your node’s actual syslog format and write a custom regex to match it.

Adding custom configurations

You can add multiple entries under configs. For example, to collect from an additional log path with different parsing rules, alongside the default:

kubectl edit configmap opsramp-logs-user-config -n <agent-installed-namespace>

node:
  - logSource: syslog
    enable: true
    configs:
      - mount_path: ["/var/log/syslog", "/var/log/messages"]
        multiline:
          line_start_pattern: '^\s*(?P<timestamp>\w*\s*\d*\s*\d*:\d*:\d*)\s*(?P<host>[^\s]*)'
        parser_type: "regex"
        parser_expression: '^\s*(?P<timestamp>\w*\s*\d*\s*\d*:\d*:\d*)\s*(?P<host>[^\s]*)\s*(?P<syslog_tag>[^:]*):\s*(?P<message>.*)$'
        timestamp_layout_type: "strptime"
        timestamp_layout: "%b %e %H:%M:%S"
      - mount_path: ["/opt/log/syslog/0.log"]
        multiline:
          line_end_pattern: '^\s*(?P<timestamp>\w*\s*\d*\s*\d*:\d*:\d*)\s*(?P<host>[^\s]*)'
        parser_type: "json"
        parser_expression: '^\s*(?P<timestamp>\w*\s*\d*\s*\d*:\d*:\d*)\s*(?P<host>[^\s]*)\s*(?P<syslog_tag>[^:]*):\s*(?P<message>.*)$'
        timestamp_layout_type: "strptime"
        timestamp_layout: "%b %e %H:%M:%S"
        log_level: "Unspecified"

Collecting logs from a custom path

By default, the agent has access to the /var/log hostpath. If your log files live somewhere else, you need to explicitly grant the agent access to that path via Helm:

helm upgrade <release-name> <agent-chart> --version <helm-version> --reuse-values --set agent.worker.mountHostPath="{/var/log,/var/mnt}"

Note: /var/log is required for pod log collection and must always be included. Add any custom paths after it, separated by commas — in this example, /var/mnt is the custom path. Make sure --version matches your currently installed Helm chart version.

Filtering syslogs by severity

Unlike pod logs, syslog severity uses a simpler, 6-tier ordering:

Fatal > Error > Warn > Info > Debug > Trace > Unspecified/Unknown

This differs from pod log severity, which has 11 levels including Emergency, Alert, Critical, and Notice. Don’t assume the two behave identically, see Log-Level Filtering for Pod Logs for the pod log equivalent.

By default, the agent evaluates severity for every syslog entry and sends all of them to the platform. To drop logs below a certain severity, set log_level under the relevant configs entry:

kubectl edit configmap opsramp-logs-user-config -n <agent-installed-namespace>

node:
  - logSource: syslog
    enable: true
    configs:
      - mount_path: ["/var/log/syslog", "/var/log/messages"]
        multiline:
          line_start_pattern: '^\s*(?P<timestamp>\w*\s*\d*\s*\d*:\d*:\d*)\s*(?P<host>[^\s]*)'
        parser_type: "regex"
        parser_expression: '^\s*(?P<timestamp>\w*\s*\d*\s*\d*:\d*:\d*)\s*(?P<host>[^\s]*)\s*(?P<syslog_tag>[^:]*):\s*(?P<message>.*)$'
        timestamp_layout_type: "strptime"
        timestamp_layout: "%b %e %H:%M:%S"
        log_level: "warn"

This example keeps logs at Warn or higher (Warn, Error, Fatal) and drops everything below (Info, Debug, Trace, Unknown).

Note: log_level accepts only string values and is case-insensitive. Valid values are: Fatal, Error, Warn, Info, Debug, Trace.

Performance tuning

The same two performance settings available for pod logs also apply to node logs, configured per configs entry.

Controlling how far back logs are read (max_time)

Sets how far back the agent looks when reading log files. Default is 2 hours — files modified within that window are processed, keeping overhead low.

node:
  - logSource: syslog
    enable: true
    configs:
      - mount_path: ["/var/log/syslog", "/var/log/messages"]
        multiline:
          line_start_pattern: '^\s*(?P<timestamp>\w*\s*\d*\s*\d*:\d*:\d*)\s*(?P<host>[^\s]*)'
        parser_type: "regex"
        parser_expression: '^\s*(?P<timestamp>\w*\s*\d*\s*\d*:\d*:\d*)\s*(?P<host>[^\s]*)\s*(?P<syslog_tag>[^:]*):\s*(?P<message>.*)$'
        timestamp_layout_type: "strptime"
        timestamp_layout: "%b %e %H:%M:%S"
        max_time: 5h

Adjusting the read buffer size (initial_buffer_size)

Sets the starting size of the buffer used to read log headers and content, growing automatically as needed. Default is 16 KB (16384 bytes), increase this if you’re ingesting unusually large log files.

node:
  - logSource: syslog
    enable: true
    configs:
      - mount_path: ["/var/log/syslog", "/var/log/messages"]
        multiline:
          line_start_pattern: '^\s*(?P<timestamp>\w*\s*\d*\s*\d*:\d*:\d*)\s*(?P<host>[^\s]*)'
        parser_type: "regex"
        parser_expression: '^\s*(?P<timestamp>\w*\s*\d*\s*\d*:\d*:\d*)\s*(?P<host>[^\s]*)\s*(?P<syslog_tag>[^:]*):\s*(?P<message>.*)$'
        timestamp_layout_type: "strptime"
        timestamp_layout: "%b %e %H:%M:%S"
        initial_buffer_size: 20480

Format note: This value must be specified in bytes. Multiply your target size in KB by 1024, for example, 20 KB = 20 * 1024 = 20480.