Kubernetes

Describes how to setup an integration to discover and monitor Kubernetes resources.

Leave Feedback

Introduction

Kubernetes is a container-orchestration system for automating deployment, scaling and management of containerized applications.

Prerequisites

The prerequisites include:

  1. Installing kube-state metrics.
  2. Performing additional configurations depending on the environment.
    • For an AKS environment, installing kube-dns patch.
    • For GKE environments, enabling RBAC.
    • For On-Prem environment, patching

Step 1: Install kube-state metrics

To see if kube-state-metrics is installed in the cluster, run the following command in on the controller node(s):

kubectl get svc --all-namespaces | grep kube-state-metrics | grep -v grep

The following is a sample output which indicates kube-state-metrics is already installed in the cluster:

kube-system kube-state-metrics ClusterIP 10.96.186.34 <none> 8080/TCP,8081/TCP 19d

To install kube-state metrics, do the following on the Kubernetes controller node(s):

  1. Clone the Kubernetes kube-state-metrics Github repo.
  2. Run kubectl apply -f kube-state-metrics/kubernetes/.
git clone https://github.com/kubernetes/kube-state-metrics.git
kubectl apply -f kube-state-metrics/kubernetes/

Step 2: Perform additional configurations

Additional configurations depend on the Kubernetes environment:

  • Azure AKS
  • Google GKE
  • On-premise

Azure AKS: Install kube-dns patch

For an Azure AKS environment, a patch must be applied. This patch is a DNS Service patch for the Azure (AKS) environment.

  • By default, the sidecar container is disabled.
  • The patch is required to export kube-dns metrics.

To install the kube-dns patch for the Azure (AKS) environment:

  1. Copy the provided Kube dns patch script to kube-dns-metrics-patch.yaml.
  2. Execute the command on the controller nodes to apply the patch:
kubectl patch deployment -n kube-system kube-dns-v20 --patch "$(cat kube-dns-metrics-patch.yaml)"
Sample Kube-dns patch script

The following is the Kube dns patch script that you save as kube-dns-metrics-patch.yaml.

        spec:
          template:
            spec:
              containers:
              - name: kubedns
                env:
                - name: PROMETHEUS\_PORT
                  value: "10055"
              - name: sidecar
                image: k8s.gcr.io/k8s-dns-sidecar-amd64:1.14.10
                livenessProbe:
                  httpGet:
                    path: /metrics
                    port: 10054
                    scheme: HTTP
                  initialDelaySeconds: 60
                  timeoutSeconds: 5
                  successThreshold: 1
                  failureThreshold: 5
                args:
                - --v=2
                - --logtostderr
                - --probe=kubedns,127.0.0.1:10053,kubernetes.default.svc.cluster.local
                - --probe=dnsmasq,127.0.0.1:53,kubernetes.default.svc.cluster.local
                ports:
                - containerPort: 10054
                  name: metrics
                  protocol: TCP
                resources:
                  requests:
                    memory: 20Mi
                    cpu: 10m

Google GKE: Enable RBAC

For a Google GKE environment, permissions must be granted to create roles in Kubernetes.

To grant permission to create roles, run the following command:

kubectl create clusterrolebinding cluster-admin-binding 
    --clusterrole cluster-admin 
    --user $(gcloud config get-value account)

This command binds the clusterrole and cluster-admin to the current user.

On-Premise: Apply patches

If patching is required on on-premise nodes, do the following:

  1. Create a user account in all nodes of cluster in order to collect packages information and install patches.
  2. Execute the following scripts on the Kubernetes nodes. The script depends on the operating system used to create the cluster.
Sample Ubuntu script

The following is an example Ubuntu script:

Configure()
{
            mkdir /opt/opsramp/k8s/patch/{tmp,log} -p  > /dev/null 2>&1
            useradd opskube -s /bin/bash -d /opt/opsramp/k8s/patch/  > /dev/null 2>&1
            usermod -p '$6$dBsN2u5SuC.Niy.C$HxPpbXRZcaQpHui8D3QZshhdJz57xhU1roE12U4KEmlyCNiBNzcGRbrNI7.DREwsf18JUAMT27/VaZmr34Bul.' opskube > /dev/null 2>&1
            if \[ -d /etc/sudoers.d \]
            then 
                echo "opskube ALL=(ALL) NOPASSWD: /opt/opsramp/k8s/patch/" > /etc/sudoers.d/opskube
                chmod 0440 /etc/sudoers.d/opskube
            else
                sed -i '$ a opskube ALL=(ALL) NOPASSWD: /opt/opsramp/k8s/patch/' /etc/sudoers > /dev/null 2>&1
            fi
        
            dpkg -s python-apt | grep Status | grep installed
            STATUS1=$?
            if [ $STATUS1 -eq 0 ]
            then
                echo "python-apt already installed! No changes required!"
            else
                apt-get update > /dev/null 2>&1
                apt-get -y install python-apt > /dev/null 2>&1
            fi  
            
}
Sample CentOS script

The following is an example CentOS script:

Configure()
{
            mkdir /opt/opsramp/k8s/patch/{tmp,log} -p  > /dev/null 2>&1
            useradd opskube -s /bin/bash -d /opt/opsramp/k8s/patch/  > /dev/null 2>&1
            usermod -p '$6$dBsN2u5SuC.Niy.C$HxPpbXRZcaQpHui8D3QZshhdJz57xhU1roE12U4KEmlyCNiBNzcGRbrNI7.DREwsf18JUAMT27/VaZmr34Bul.' opskube > /dev/null 2>&1
            if \[ -d /etc/sudoers.d \]
            then 
                echo "opskube ALL=(ALL) NOPASSWD: /opt/opsramp/k8s/patch/" > /etc/sudoers.d/opskube
                chmod 0440 /etc/sudoers.d/opskube
            else
                sed -i '$ a opskube ALL=(ALL) NOPASSWD: /opt/opsramp/k8s/patch/' /etc/sudoers > /dev/null 2>&1
            fi
            
            rpm -qa | grep rpm-python
            STATUS1=$?
            if [ $STATUS1 -eq 0 ]
            then
                echo "rpm-python already installed! No changes required!"
            else
                yum  -y install rpm-python > /dev/null 2>&1
            fi

}

OpsRamp configuration

Configuration involves:

  1. Configuring the integration.
  2. Deploying the agent.

Step 1: Configuring the integration

To integrate:

  1. From All Clients, select the client.
  2. Go to Setup > Integrations > Integrations.
  3. From Available Integrations, select Compute > Kubernetes and click Install.
  4. Provide the following:
    • Name for the integration.
    • Deployment type: On prem or cloud
  5. Click Install.
Kubernetes Integration

Kubernetes Integration

Step 2: Deploy the agent

To deploy the agent on the Kubernetes nodes:

  1. Copy the YAML content and paste to a new file in kube-controller (Example file name: opsramp-agent-kubernetes.yaml)
  2. Execute the command kubectl apply -f opsramp-agent-kubernetes.yaml in kube-controller.
Kubernetes Infrastructure

Kubernetes Infrastructure

What to do next

After a discovery profile is created, you can do the following:

  • View the integration: Go to Infrastructure > Resources.
  • Assign monitoring templates to the resource.
  • Validate that the resource was successfully added to OpsRamp.

External references