The NameNode is the centerpiece of an HDFS file system. It keeps the directory tree of all files in the file system, and tracks where across the cluster the file data is kept. It does not store the data of these files itself.

Prerequisite

For Virtual Machines, install the Linux Agent.

Configuring the credentials

Configure the credentials in the directory /opt/opsramp/agent/conf/app.d/creds.yaml

hdfs-namenode:
- name: hdfs-namenode
  user: <username>
  pwd: <Password>
  encoding-type: plain
  labels:
    key1: val1
    key2: val2

Configure the application

Virtual machine

Configure the application in the directory /opt/opsramp/agent/conf/app/discovery/auto-detection.yaml

- name: hdfs-namenode
  instance-checks:
    process-check:
      - namenode
    port-check:
      - 50070
      - 9870
    uri-check:
      - "jmx?qry=Hadoop:service=NameNode,name=FSNamesystem*"
      - "jmx?qry=Hadoop:service=NameNode,name=NameNodeInfo"

Docker environment

Configure the application in the directory /opt/opsramp/agent/conf/app/discovery/auto-container-detection.yaml

- name: hdfs-namenode
  container-checks:
    image-check:
      - namenode
    port-check:
      - 50070
      - 9870

Kubernetes environment

Configure the application in config.yaml

- name: hdfs-namenode
  container-checks:
    image-check:
      - namenode
    port-check:
      - 50070
      - 9870

Validate

Go to Resources under the Infrastructure tab to check if your resources are onboarded and the metrics are collected.

Metrics

OpsRamp MetricMetric Display NameUnit
hdfs_namenode_capacity_total

Total disk capacity
Capacity Totalbytes
hdfs_namenode_capacity_used

Disk usage
Capacity Usedbytes
hdfs_namenode_capacity_used_percent

Disk usage
Capacity Used Percentpercent
hdfs_namenode_capacity_remaining

Remaining disk space left
Capacity Remainingbytes
hdfs_namenode_capacity_remaining_percent

Remaining disk space left
Capacity Remaining Percentpercent
hdfs_namenode_total_load

Total load on the file system
Total Load
hdfs_namenode_fs_lock_queue_length

Lock queue length
Fs Lock Queue Length
hdfs_namenode_blocks_total

Total number of blocks
Blocks TotalBlocks
hdfs_namenode_max_objects

Maximum number of files that HDFS supports
Max Objects
hdfs_namenode_files_total

Total number of files
Files TotalFiles
hdfs_namenode_pending_replication_blocks

Number of blocks pending replication
Pending Replication BlocksBlocks
hdfs_namenode_under_replicated_blocks

Number of under replicated blocks
Under Replicated BlocksBlocks
hdfs_namenode_scheduled_replication_blocks

Number of blocks scheduled for replication
Scheduled Replication BlocksBlocks
hdfs_namenode_pending_deletion_blocks

Number of pending deletion blocks
Pending Deletion BlocksBlocks
hdfs_namenode_num_live_data_nodes

Total number of live data nodes
Num Live Data NodesNodes
hdfs_namenode_num_dead_data_nodes

Total number of dead data nodes
Num Dead Data NodesNodes
hdfs_namenode_num_decom_live_data_nodes

Number of decommissioning live data nodes
Num Decom Live Data NodesNodes
hdfs_namenode_num_decom_dead_data_nodes

Number of decommissioning dead data nodes
Num Decom Dead Data NodesNodes
hdfs_namenode_volume_failures_total

Total volume failures
Volume Failures Total
hdfs_namenode_estimated_capacity_lost_total

Estimated capacity lost
Estimated Capacity Lost Totalbytes
hdfs_namenode_num_decommissioning_data_nodes

Number of decommissioning data nodes
Num Decommissioning Data NodesNodes
hdfs_namenode_num_stale_data_nodes

Number of stale data nodes
Num Stale Data NodesNodes
hdfs_namenode_num_stale_storages

Number of stale storages
Num Stale Storages
hdfs_namenode_missing_blocks

Number of missing blocks
Missing BlocksBlocks
hdfs_namenode_corrupt_blocks

Number of corrupt blocks
Corrupt BlocksBlocks
hdfs_namenode_num_failed_data_nodes

Total number of failed data nodes
Num Failed Data Nodes
hdfs_namenode_nondfs_used_percent

Total space used by NonDfs
Nondfs Used PercentPercent