Limited Availability NoticeGateway clustering is not a generally available feature at this time. This feature is visible within your OpsRamp account only if you are participating in OpsRamp’s limited availability program. This feature will be generally available in future releases. Please contact OpsRamp Support for additional information.
A gateway cluster is a set of virtual machines running gateway software that function as a single logical machine. The gateway cluster provides:
- High availability against the failure of a node in the cluster.
- High availability against the failure of a physical server on which nodes run.
- Flexible horizontal scaling of nodes to manage more IT assets.
How a gateway cluster works
A gateway cluster is a set of virtual machines (VMs), which run applications that discover and monitor your environment. Gateway nodes run on physical servers that run a hypervisor, typically, with other VMs unrelated to the gateway. Nodes use a shared NFS storage volume to persist states shared among gateway nodes.
Each node also runs a lightweight Kubernetes - MicroK8s distribution. Kubernetes enables gateway nodes to work as a single, logical machine, which automatically schedules gateway applications between nodes. If a node fails or the host on which the node runs fails, or both a node and a host fail restarts applications on a different node, the logical node restarts the applications. The following figure illustrates how a gateway cluster works:
Gateway clusters can be deployed in several configurations, depending on availability and horizontal scaling goals. The following figures illustrate three design points:
To deploy a gateway cluster, make sure your environment meets these requirements:
|Nodes||Size||4 CPU cores, 8 GB RAM, and 40 GB Disk|
|Nodes||IP addresses||Static IP address allocation requirement:|
|Nodes||Hostname||Each node should have a unique hostname. IP address and hostname should be added in **/etc/hosts** file and all nodes should be pingable with IP address and hostname.|
|Nodes||Network access||All VMs should have outbound internet access to `*.opsramp.com` for connectivity, `k8s.gcr.io` and Google Artifact registry `us-docker.pkg.dev` to download gateway applications.|
|Nodes||Number of nodes||Minimum of three nodes for high availability.|
|Hosts||OS||VMWare vSphere ESXi v6.0 or later versions.|
|Hosts||Number of hosts||Minimum of three hosts, running one gateway node on each host for high availability.|
|Storage||Type||NFS storage volume, with read/write access from all nodes.|
Set up a multi-node cluster
To set up a gateway cluster:
- Spin up nodes.
- Install MicroK8s
- Join nodes to the cluster
- Deploy gateway services on nodes.
- Register the Cluster Gateway.
The following description assumes a 3-node gateway cluster with each node running on a separate host.
See the troubleshooting section if you encounter problems.
Spin up nodes
The gateway is available as an Open Virtual Machine Appliance (OVA).
Download the OVA.
Spin up identical VMs on separate hosts, from the OVA, one VM for each node of the cluster.
Log in to each VM with the default credentials provided to you.
- Make sure to change the password.
- You now have a VM for each node of the cluster.
Set the new unique hostname on each node
> sudo hostnamectl set-hostname <name>
Add the IP address and hostname of each node in /etc/hosts file
> sudo nano /etc/hosts
Run the following commands in all the nodes
> cd /var/lib/node-manager/ > sudo python main.py --install
Check the MicroK8s status in all the nodes, it should be in the running state.
> sudo microk8s status
Join nodes to the cluster
With three gateway nodes running, join them to create a cluster.
Select one of the nodes as your first node in the cluster, called node 1. You do not need to take an action to add the first node to the cluster.
Join node 2
To add the second node, called node 2, to the cluster. Go to the node 1 and run
> microk8s add-node
This returns joining instructions:
> microk8s join ip-172-31-20-243:25000/DDOkUupkmaBezNnMheTBqFYHLWINGDbf
If the node you are adding is not reachable through the default interface, use:
> microk8s join 10.1.84.0:25000/DDOkUupkmaBezNnMheTBqFYHLWINGDbf > microk8s join 10.22.254.77:25000/DDOkUupkmaBezNnMheTBqFYHLWINGDbf
Copy the above command and run it on node 2.
> microk8s join ip-172-31-20-243:25000/DDOkUupkmaBezNnMheTBqFYHLWINGDbf
Wait for the process to complete on node 2.
Join node 3
To join the third node, called node 3, to the cluster, repeat the same steps for joining node 2.
To check that the node are successfully added, run:
> microk8s kubectl get nodes
Deploy gateway services on nodes
With the cluster set up, deploy gateway applications to the cluster.
Run the following commands on any node to enable DNS
> microk8s enable dns
Install Storage, MetalLB and Gateway manager
Run the following commands on any of the nodes to install Persistent storage, MetalLB, and Gateway manager services.
- Create a folder to download the files
> mkdir /var/lib/node-manager/charts > cd /var/lib/node-manager/charts
- Pull the required services
> helm chart pull us-docker.pkg.dev/opsramp-registry/gateway-cluster-charts/gateway-extras:1.0.0 > helm chart pull us-docker.pkg.dev/opsramp-registry/gateway-cluster-charts/kubernetes-dashboard:0.9.6 > helm chart pull us-docker.pkg.dev/opsramp-registry/gateway-cluster-charts/gateway-manager:0.9.8
- Export the requried services
> helm chart export us-docker.pkg.dev/opsramp-registry/gateway-cluster-charts/gateway-extras:1.0.0 > helm chart export us-docker.pkg.dev/opsramp-registry/gateway-cluster-charts/kubernetes-dashboard:0.9.6 > helm chart export us-docker.pkg.dev/opsramp-registry/gateway-cluster-charts/gateway-manager:0.9.8
- Configure the Storage type, NFS IP Address, Path, MetalLB IP
> nano /var/lib/node-manager/charts/gateway-extras/values.yaml
- Provide the storage type as nfs, nfs IP Address and path and MetalLB IP as shown below
storageType: &storageType nfs nfsServerIp: &nfsServerIp 172.25.251.89 nfsServerPath: &nfsServerPath /srv/nfsdemo metallbIp: &metallbIp 172.25.252.100/32 #Use ip/32 format
- Install Storage, MetalLB and Gateway manager
> helm install gateway-extras /var/lib/node-manager/charts/gateway-extras > helm install kubernetes-dashboard /var/lib/node-manager/charts/kubernetes-dashboard --namespace kubernetes-dashboard --create-namespace --debug > helm install gateway-manager /var/lib/node-manager/charts/gateway-manager -f gateway-manager/cluster.yaml --set secrets.defaultPassword=<Pass@1234> --debug
- Log in to the Gateway manager to register the gateway.
- Open a web browser
Note: IP assigned to MetalLB should be used.
Register Cluster Gateway
Log in to Gateway Manager to register the cluster:
- Go to Setup > Management Profiles.
- Create a new management profile and copy the activation token.
- Enter the activation token into the Gateway Manager.
Wait for the cluster registration to complete. You should see the status of the management profile turn to connected in the UI.
Migrate from a classic gateway
You can migrate an existing non-clustered, classic gateway to a cluster gateway without re-onboarding the resources managed by the classic gateway. To migrate from classic to clustered gateway:
- Create a new gateway cluster. Follow the steps to set up a multi-node cluster.
- Select the management profile associated with the classic gateway.
- De-register the classic gateway from the management profile.
- Register the new gateway cluster with the same management profile.
Existing configurations in the classic gateway are automatically migrated to the gateway cluster. Discovery and monitoring resume automatically after a few minutes.
Upgrading gateway firmware
To upgrade the gateway firmware:
- Swap out the existing VM.
- Deploy a new VM with the latest OVA.
- Attach the new VM to the same management profile.
Can I roll back to the classic gateway from a gateway cluster?
- Yes, you can roll back from a gateway cluster to the classic gateway without re-onboarding your managed resources and without loss of monitoring data.
- De-register the gateway cluster from its management profile and register a classic gateway in its place.
How do I validate the multi-node cluster gateway setup?
Validate that your cluster is set up correctly by:
- Running network discovery on a few managed resources.
- Applying ping monitors on the resources.
How do I validate that the cluster can successfully recover from a node failure?
- Select one of the nodes.
- Power the node off to simulate node failure. The cluster recovers from the failure by restarting monitoring applications running on the failed node on the remaining nodes.
- Verify that monitors resume monitoring by observing the metric graphs in the UI.
If you can not ping the other nodes with the hostname, add the IP address and hostname of other nodes in the /etc/hosts file. See the MicroK8s documentation for tips on troubleshooting cluster setup-related issues.