hadoop cluster monitoring tool-ganglia (YUM approach)

Introduction to Ganglia Architecture:
Ganglia is a scalable distributed monitoring system designed for HPC (High Performance Computing) cluster. It can monitor and display various status information of nodes in the cluster. It collects CPU, memory, disk utilization, I/O load, network traffic and other data from the daemon gmond running on each node, and then aggregates them under the daemon gmetad, using rrdt. Ool stores data, and then presents historical data in a curve way through PHP pages. Through the curve, it is easy to see the working status of each node, which plays an important role in rationally adjusting and allocating system resources and improving the overall performance of the system.

Ganglia monitors three components: gmond, gmetad and ganlia-web (web interface).
1:gmond: It is a daemon process that needs to be installed on each host. It is responsible for interacting with the operating system to get the target data that needs to be paid attention to. It runs on every node that needs to be monitored and collects monitoring statistics.
2: gmetad: Also a daemon, cluster data collection node, installed on the central node, using Hadoop's Name Node as the cluster data collection node. The main function is to integrate all information. Check all gmond s regularly, pull data, and store their metrics in the RRD(Round Robin Database Storage Format, data equal to placed in the database) storage engine. He can query multiple clusters and aggregate metrics. He is also used to generate a web front-end user interface.
3: ganglia-web: Visualization tool to display various indicators collected by ganglia. Installed on a machine running gmetad, read RRD files (gweb is a PHP program, because it needs to interact with the RD database created by the poller, so gweb is usually installed on the same physical hardware as gmetad).

Ganglia is divided into server and client, where gmetad is server and gmond is client. There is only one server, and all monitored servers install clients.

Installing Ganglia in yum mode is much simpler than installing Ganglia in make mode, because installing in yum mode does not need to download, compile and install dependency packages by itself.

Install Ganglia's cluster environment:

linux version:

[root@cloud0 hadoop]# lsb_release -a
LSB Version:    :core-4.1-amd64:core-4.1-noarch:cxx-4.1-amd64:cxx-4.1-noarch:desktop-4.1-amd64:desktop-4.1-noarch:languages-4.1-amd64:languages-4.1-noarch:printing-4.1-amd64:printing-4.1-noarch
Distributor ID: CentOS
Description:    CentOS Linux release 7.3.1611 (Core) 
Release:        7.3.1611
Codename:       Core

Commands cannot be used:

[root@cloud0 hadoop]# lsb_release -a
bash: lsb_release: No command found...

Installation command

[root@cloud0 hadoop]# yum install lsb

hadoop version:

[root@cloud0 hadoop]# hadoop version 
Hadoop 2.7.2
Subversion https://git-wip-us.apache.org/repos/asf/hadoop.git -r b165c4fe8a74265c792ce23f546c64604acf0e41
Compiled by jenkins on 2016-01-26T00:08Z
Compiled with protoc 2.5.0
From source with checksum d0fda26633fa762bff87ec759ebe689c
This command was run using /usr/local/hadoop/hadoop-2.7.2/share/hadoop/common/hadoop-common-2.7.2.jar

Test Cluster: There are two network cards, Intranet and External Network, to ensure that the network cards of External Network are started.
Network Card Start Command: ifup Network Card Name
Ensure that all machines are connected to the Internet

(If the explanation is incorrect, please discuss it in the comments section. Thank you ~)

Installation scenario:
Server 1 (master): Install gmond, gmetad, and web
Server 2 (slave1): Install only gmond
Server 3 (slave2): Install only gmond
Server n (slaven): Install only gmond

The situation of the experimental cluster

Installation process:
First, install EPEL for each machine: it is a software source of yum, which contains a lot of software that is not available in the basic source, without installing, you will not find the package.
Mode 1:
wget http://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
rpm -vih epel-release-latest-7.noarch.rpm
Mode 2:
rpm -Uvh http://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
Mode 3
yum install epel-release

Step 1: Linux opens and installs EPEL YUM source

Machine cloud 0

[root@cloud0 hadoop]# wget http://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
[root@cloud0 hadoop]# rpm  -vih epel-release-latest-7.noarch.rpm 

Machine cloud 2

[root@cloud2 hadoop]# wget http://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
[root@cloud2 hadoop]# rpm  -vih epel-release-latest-7.noarch.rpm 

Machine cloud 3

[root@cloud3 hadoop]# wget http://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
[root@cloud3 hadoop]# rpm  -vih epel-release-latest-7.noarch.rpm 

Machine cloud 4

[root@cloud4 hadoop]# wget http://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
[root@cloud4 hadoop]# rpm  -vih epel-release-latest-7.noarch.rpm 

Check that the epel source installation is successful:
Success is achieved with epel/x86_64 under the source identifier

[root@cloud0 ~]# yum repolist 

Step 2: Install Ganglia and operate under root authority

a: Machine cloud 0

[root@cloud0 ~]# yum -y install ganglia-gmetad 
[root@cloud0 ~]# yum -y install ganglia-web

b: Machine cloud 0, machine cloud 2, machine cloud 3, machine cloud 4

[root@cloud0 ~]# yum -y install ganglia-gmond 
[root@cloud2 ~]# yum -y install ganglia-gmond
[root@cloud3 ~]# yum -y install ganglia-gmond
[root@cloud4 ~]# yum -y install ganglia-gmond

Step 3: Configuration

Document description:
The gmetad.conf configuration monitors which machine files.
gmond.conf configures monitored machine files

a: File operations on machine cloud0

[root@cloud0 ~]# vim /etc/ganglia/gmetad.conf
# data_source "my cluster" localhost
data_source "MyCluster_TEST" cloud0 cloud2 cloud3 cloud4
[root@cloud0 ~]# vim /etc/httpd/conf.d/ganglia.conf
  7 <Location /ganglia>
  9   Order deny,allow 
 10   # Deny from all commentary line
 11   Allow from all 
 13   # Require local
 14   # Require ip
 15   # Require host example.org
 16 </Location>

Apache configuration file httpd.conf
Amend as follows:

[root@cloud0 ~]# vim /etc/httpd/conf/httpd.conf
    102 <Directory />
    103     #AllowOverride none
    104     #Require all denied
    106    Options FollowSymLinks
    107    AllowOverride None
    108    Order deny,allow
    109    allow from all
    111 </Directory>

Note: Without modification, when viewing graphical interface with web pages, errors will be reported: 403 has no access privileges

b: File operations on machine cloud0, machine cloud2, machine cloud3, machine cloud4

[root@cloud0 ~]# vi /etc/ganglia/gmond.conf
cluster {
  name = "MyCluster_TEST"
  owner = "unspecified"
  latlong = "unspecified"
  url = "unspecified"

Note: The name at this time should be the same as in the data_source configuration in the file / etc/ganglia/gmetad.conf

Step 4: Start the service and set up boot start

a: Machine cloud 0: Start service gmetad gmond Apache

[root@cloud0 ~]# service gmetad start  
[root@cloud0 ~]# service gmond  start  
[root@cloud0 ~]# service httpd  start  

Check to see if the startup is successful

[root@cloud0 ~]# service gmetad status
...active (running) ...
[root@cloud0 ~]# service gmond status
...active (running)...
[root@cloud0 ~]# service httpd status
... active (running) ...

Set up boot start

[root@cloud0 ~]# chkconfig gmetad on
[root@cloud0 ~]# chkconfig gmond on
[root@cloud0 ~]# systemctl enable httpd.service  

b: Machine cloud 2, machine cloud 3, machine cloud 4

[root@cloud2 ~]# service gmond  start 
[root@cloud3 ~]# service gmond  start 
[root@cloud4 ~]# service gmond  start 

[root@cloud2 ~]# chkconfig gmond on
[root@cloud3 ~]# chkconfig gmond on
[root@cloud4 ~]# chkconfig gmond on

Finally, access through the website: http://service_ip/ganglia

Links for reference:
Ganglia CentOS yum installation configuration
httpd service 403 solution
httpd service boot-up
Introduction to Environment Construction and Deployment
Another way to install: compile source files, but I don't work.
View the linux version
Look at the linux version: an inexhaustible solution for the lsb_release-a command
Whether Network Card Starts
Installation and unloading of rpm package
Installation of epel source

Tags: EPEL Hadoop RPM yum

Posted on Sat, 08 Jun 2019 11:31:06 -0700 by Norman Graham