Kubernetes

Describes how to setup an integration to discover and monitor Kubernetes resources.

Leave Feedback

Introduction

Kubernetes is a container-orchestration system for automating deployment, scaling, and management of containerized applications. Kubernetes integration monitors the health and performance of your Kubernetes cluster.

Kubernetes configuration

Configuration involves:

  1. Installing kube-state metrics.
  2. Performing additional configurations.

Prerequisites

The prerequisites for Kubernetes configuration include:

  1. Installing kube-state metrics.
  2. Performing additional configurations depending on the environment:
    • For AKS, installing kube-dns patch.
    • For GKE, enabling RBAC.
    • For On-Prem, patch.

Step 1: Install kube-state metrics

To see if kube-state-metrics is installed in the cluster, run the following command on the controller node(s):

kubectl get svc --all-namespaces | grep kube-state-metrics | grep -v grep

The following is a sample output that shows kube-state-metrics is already installed in the cluster:

kube-system kube-state-metrics ClusterIP 10.96.186.34 <none> 8080/TCP,8081/TCP 19d

To install kube-state metrics, do the following on the Kubernetes controller node(s):

  1. Clone the Kubernetes kube-state-metrics Github repo.
  2. Run kubectl apply -f kube-state-metrics/kubernetes/.
git clone https://github.com/kubernetes/kube-state-metrics.git
kubectl apply -f kube-state-metrics/kubernetes/

Step 2: Perform additional configurations

Additional configurations depend on the Kubernetes environment:

  • Azure AKS
  • Google GKE
  • On-premise

Azure AKS: Install kube-dns patch

For an Azure AKS environment, a patch must be applied. This patch is a DNS Service patch for the Azure (AKS) environment.

  • By default, the sidecar container is disabled.
  • The patch is required to export kube-dns metrics.

To install the kube-dns patch for the Azure (AKS) environment:

  1. Copy the provided Kube dns patch script to kube-dns-metrics-patch.yaml.
  2. Execute the command on the controller nodes to apply the patch:
kubectl patch deployment -n kube-system kube-dns-v20 --patch "$(cat kube-dns-metrics-patch.yaml)"
Sample Kube-dns patch script

The following is the Kube dns patch script that you save as kube-dns-metrics-patch.yaml.

spec:
  template:
    spec:
      containers:
      - name: kubedns
        env:
        - name: PROMETHEUS_PORT
          value: "10055"
      - name: sidecar
        image: k8s.gcr.io/k8s-dns-sidecar-amd64:1.14.10
        livenessProbe:
          httpGet:
            path: /metrics
            port: 10054
            scheme: HTTP
          initialDelaySeconds: 60
          timeoutSeconds: 5
          successThreshold: 1
          failureThreshold: 5
        args:
        - --v=2
        - --logtostderr
        - --probe=kubedns,127.0.0.1:10053,kubernetes.default.svc.cluster.local
        - --probe=dnsmasq,127.0.0.1:53,kubernetes.default.svc.cluster.local
        ports:
        - containerPort: 10054
          name: metrics
          protocol: TCP
        resources:
          requests:
            memory: 20Mi
            cpu: 10m

Execute the following command in the controller nodes to apply a patch:

kubectl patch deployment -n kube-system kube-dns-v20 --patch "$(cat kube-dns-metrics-patch.yaml)"

Google GKE: Enable RBAC

For a Google GKE environment, permissions must be granted to create roles in Kubernetes.

To grant permission to create roles, execute the following command:

kubectl create clusterrolebinding cluster-admin-binding
    --clusterrole cluster-admin
    --user $(gcloud config get-value account)

This command binds the clusterrole and cluster-admin to the current user.

On-Premise: Apply patches

If patching is required on on-premise nodes, do the following:

  1. Create a user account in all nodes of cluster in order to collect packages information and install patches.
  2. Execute the following scripts on the Kubernetes nodes. The script depends on the operating system used to create the cluster.
Sample Ubuntu script

The following is an example Ubuntu script:

Configure()
{
            mkdir /opt/opsramp/k8s/patch/{tmp,log} -p  > /dev/null 2>&1
            useradd opskube -s /bin/bash -d /opt/opsramp/k8s/patch/  > /dev/null 2>&1
            usermod -p '$6$dBsN2u5SuC.Niy.C$HxPpbXRZcaQpHui8D3QZshhdJz57xhU1roE12U4KEmlyCNiBNzcGRbrNI7.DREwsf18JUAMT27/VaZmr34Bul.' opskube > /dev/null 2>&1
            if [ -d /etc/sudoers.d ]
            then
                echo "opskube ALL=(ALL) NOPASSWD: /opt/opsramp/k8s/patch/" > /etc/sudoers.d/opskube
                chmod 0440 /etc/sudoers.d/opskube
            else
                sed -i '$ a opskube ALL=(ALL) NOPASSWD: /opt/opsramp/k8s/patch/' /etc/sudoers > /dev/null 2>&1
            fi

            dpkg -s python-apt | grep Status | grep installed
            STATUS1=$?
            if [ $STATUS1 -eq 0 ]
            then
                echo "python-apt already installed! No changes required!"
            else
                apt-get update > /dev/null 2>&1
                apt-get -y install python-apt > /dev/null 2>&1
            fi  

}
Sample CentOS script

The following is an example CentOS script:

Configure()
{
            mkdir /opt/opsramp/k8s/patch/{tmp,log} -p  > /dev/null 2>&1
            useradd opskube -s /bin/bash -d /opt/opsramp/k8s/patch/  > /dev/null 2>&1
            usermod -p '$6$dBsN2u5SuC.Niy.C$HxPpbXRZcaQpHui8D3QZshhdJz57xhU1roE12U4KEmlyCNiBNzcGRbrNI7.DREwsf18JUAMT27/VaZmr34Bul.' opskube > /dev/null 2>&1
            if [ -d /etc/sudoers.d ]
            then
                echo "opskube ALL=(ALL) NOPASSWD: /opt/opsramp/k8s/patch/" > /etc/sudoers.d/opskube
                chmod 0440 /etc/sudoers.d/opskube
            else
                sed -i '$ a opskube ALL=(ALL) NOPASSWD: /opt/opsramp/k8s/patch/' /etc/sudoers > /dev/null 2>&1
            fi

            rpm -qa | grep rpm-python
            STATUS1=$?
            if [ $STATUS1 -eq 0 ]
            then
                echo "rpm-python already installed! No changes required!"
            else
                yum  -y install rpm-python > /dev/null 2>&1
            fi

}

OpsRamp configuration

Configuration involves:

  1. Configuring the integration.
  2. Deploying the agent.

Step 1: Configuring the integration

To integrate:

  1. From All Clients, select the client.
  2. Go to Setup > Integrations > Integrations.
  3. From Available Integrations, select Compute > Kubernetes and click Install.
  4. Provide the following:
    • Name for the integration.
    • Deployment type: On prem or cloud
  5. Click Install.
Kubernetes Integration

Kubernetes Integration

Step 2: Deploy the agent

To deploy the agent on the Kubernetes nodes:

  1. Copy the YAML content and paste to a new file in kube-controller (Example file name: opsramp-agent-kubernetes.yaml)
  2. Execute the command kubectl apply -f opsramp-agent-kubernetes.yaml in kube-controller.
Kubernetes Infrastructure

Kubernetes Infrastructure

What to do next

After a discovery profile is created, perform the following:

  • View the integration, go to Infrastructure > Resources.
  • Assign monitoring templates to the resource.
  • Validate that the resource was successfully added to OpsRamp.

External references