Python monitors Linux systems

1. Monitor Linux using open source libraries

In this section, we will introduce an open source project, psutil, that is widely used in Python ecology.We will then use psutil to refactor the monitor we wrote in the previous section.In addition, the process management capabilities provided by psutil are briefly described.

1. Introduction to psutil

psutil = process and system utilities

  • Psutil is an open source, cross-platform library that provides convenient functions to get information about the operating system, such as CPU, memory, disk, network, and so on.In addition, psutil can be used for process management, including determining whether a process exists, getting a list of processes, getting process details, and so on.Psutil also provides a number of command line tools, including ps, top, lsof, netstat, ifconfig, who, df, kill, free, nice, ionice, iostat, iotop, uptime, pidof, tty, taskset, pmap.
  • psutil is a cross-platform library that supports Linux, Windows, OSX, FreeBSD, OpenBSD, NetBSD, Sun Solaris, AIX and other operating systems.psutil also supports 32-bit and 64-bit system architectures and supports all Python versions between Python 2.6 and Python 3.x.
  • Psutil has many advantages, such as easy to use, powerful, cross-platform, etc. It is widely used in open source projects, such as glances, osquery on Facebook, grr on Google, etc.Psutil is not only widely used in open source projects for Python language development, but also ported to other programming languages, such as gopsutil for Go language, cpslib for C language, rust-psutil for Rust language, posixpsutil for Ruby language, etc.

pip installation psutil

Psutil is a third-party open source project, so it needs to be installed before it can be used.If Anaconda is installed, psutil is already available.Otherwise, you need to install through pip from the command line:

[root@localhost ~]# pip3 install psutil
Collecting psutil
  Downloading psutil-5.7.0.tar.gz (449 kB)
     |████████████████████████████████| 449 kB 4.6 kB/s 
Installing collected packages: psutil
    Running setup.py install for psutil ... done
Successfully installed psutil-5.7.0
[root@python scripts]# ipython            #Open ipython

psutil contains exceptions, classes, function and constants, where function functions are used to get information about the system, such as CPU, disk, memory, network, and so on.Classes are used to implement process management functions.

2. Functions provided by psutil

According to the function of the function, it can be divided into CPU, disk, memory, network and so on. The function provided by psutil is described in the following general aspects.In this section, we will also learn how to use psutil to simplify programs that use shell scripts to obtain monitoring information and to obtain different dimensions such as CPU, memory, disk, and network.

(1)CPU

Functional functions related to CPU are as follows:

function describe
psutil.cpu_count() cpu_count(,[logical]): Returns the number of logical CPUs by default, and the number of physical CPUs when the logical parameter is set to False.
psutil.cpu_percent() cpu_percent(,[percpu],[interval]): Returns the utilization of the CPU, showing the utilization of all physical cores when percpu is True, and the average utilization of interval execution time when interval is not zero
psutil.cpu_times() cpu_times(,[percpu]): The time spent returning the CPU as a named tuple, and percpu=True indicates the time spent getting each CPU
psutil.cpu_times_percent() cpu_times_percent(,[percpu]): Functions are roughly the same as cpu_times, and when you look at the literal meaning, this function returns a time-consuming ratio.
psutil.cpu_stats() cpu_stats() returns CPU statistics as named tuples, including context switches, interrupts, soft interrupts, and number of system calls.
psutil.cpu_freq() cpu_freq([percpu]): Returns the cpu frequency

1)cpu_count

Returns the number of logical CPUs by default, and the number of physical CPUs when the logical parameter is set to False.

In [1]: import psutil                                            

In [2]: psutil.cpu_count()                    #View Number of Processor Cores
Out[2]: 1

In [3]: psutil.cpu_count(logical=False)       #View the number of processor cores currently in use                   
Out[3]: 1

2)cpu_percent

Returns the utilization of the CPU, showing the utilization of all physical cores when the percpu is True, and the average utilization of interval execution time when the interval is not zero.

In [5]: psutil.cpu_percent()                    #View each kernel usage 
Out[5]: 0.2

In [6]: psutil.cpu_percent(percpu=True)                          
Out[6]: [0.5]

In [7]: psutil.cpu_percent(percpu=True,interval=2)               
Out[7]: [0.0]

3)cpu_times

Returns the time spent on the CPU as a named tuple, and percpu=True indicates the time spent getting each CPU.

In [8]: psutil.cpu_times()                                       
Out[8]: scputimes(user=11.1, nice=0.0, system=14.05, idle=3252.64, iowait=0.98, irq=0.0, softirq=0.18, steal=0.0, guest=0.0, guest_nice=0.0)

In [9]: psutil.cpu_times_percent()                               
Out[9]: scputimes(user=0.2, nice=0.0, system=0.1, idle=99.6, iowait=0.0, irq=0.0, softirq=0.0, steal=0.0, guest=0.0, guest_nice=0.0)

4)cpu_stats

Returns CPU statistics as named tuples, including context switching, interrupts, soft interrupts, and number of system calls.

In [10]: psutil.cpu_stats()                                      
Out[10]: scpustats(ctx_switches=403960, interrupts=214204, soft_interrupts=217258, syscalls=0)

5)cpu_freq

Returns the cpu frequency.

In [11]: psutil.cpu_freq()                                       
Out[11]: scpufreq(current=1799.453, min=0.0, max=0.0)

(2) Memory

Memory-related functions are as follows:

1)virtual_memory

Returns memory usage as named tuples, including total memory, available memory, memory utilization, buffer s, cache s, and so on.Fields are returned in bytes, except for memory utilization.

In [1]: import psutil                                                              

In [12]: psutil.virtual_memory()                                 
Out[12]: svmem(total=1023934464, available=383610880, percent=62.5, used=481861632, free=67833856, active=436379648, inactive=340885504, buffers=2166784, cached=472072192, shared=7598080, slab=79253504)

Unit Conversion

#/usr/bin/python
#-*- conding:utf-8 _*_

import psutil

def bytes2human(n):
     symbols = ('K','M','G','T','P','E','Z','Y')
     prefix = {}
     for i,s in enumerate(symbols):
         prefix[s] = 1 << (i + 1) * 10
         print(prefix[s])
     print('============')
     for s in reversed(symbols):
         if n >= prefix[s]:
             value = float(n) / prefix[s]
             return '%.1f%s' % (value,s)
             # return '{0}.1f{1}'.format(value, s)
     return "%sB" % n
     # return "{0}B" .format(n)

print("Total memory:"+bytes2human(psutil.virtual_memory().total))
The results are as follows:
[root@python scripts]# python3 monitor_mem.py 
1024
1048576
1073741824
1099511627776
1125899906842624
1152921504606846976
1180591620717411303424
1208925819614629174706176
============
//Total memory: 976.5M
Analyze

2)swap_memory

Returns swap/memory usage as named tuples, including page swap in and out.

In [13]: psutil.swap_memory()                                    
Out[13]: sswap(total=2147479552, used=0, free=2147479552, percent=0.0, sin=0, sout=0)

(3) Disk

Disk-related functions are as follows:

function describe
psutil.disk_io_counters() disk_io_counters([perdisk]): Returns disk IO statistics (summary) as named tuples, including the number of reads and writes, the number of bytes read and written, and so on.When perdisk has a value of True, the statistics for a single disk are listed separately (dictionary: key is the disk name, value is the namedtuple of the statistics).
psutil.disk_partitions() disk_partitions([all=False]): Returns all mounted disks as named tuples, including disk name, mount point, file system type, and so on.Returns mount information containing special file systems such as / proc when all equals True
psutil.disk_usage() disk_usage(path): Returns the usage of the disk on which path is located as a named tuple, including disk capacity, used disk capacity, disk space utilization, and so on.

1)psutil.disk_io_counters

Returns disk IO statistics (summary) as named tuples, including the number of times read and write, the number of bytes read and write, and so on.When perdisk has a value of True, the statistics for a single disk are listed separately (dictionary: key is the disk name, value is the namedtuple of the statistics).The disk_io_counters function eliminates the hassle of parsing/proc/diskstats files.

In [1]: import psutil                                                              

In [2]: psutil.disk_io_counters()                              
Out[2]: sdiskio(read_count=86913, write_count=46560, read_bytes=5038501376, write_bytes=408987648, read_time=77974, write_time=79557, read_merged_count=5933, write_merged_count=35916, busy_time=42153)

In [3]: psutil.disk_io_counters(perdisk=Tru
Out[3]: 
{'sda': sdiskio(read_count=41472, write_count=5340, read_bytes=2524417024, write_bytes=205662720, read_time=38302, write_time=4484, read_merged_count=5933, write_merged_count=35916, busy_time=21074),
 'sda1': sdiskio(read_count=1854, write_count=4, read_bytes=6441472, write_bytes=2097152, read_time=370, write_time=35, read_merged_count=0, write_merged_count=0, busy_time=396),
 'sda2': sdiskio(read_count=39587, write_count=5337, read_bytes=2516263424, write_bytes=203570688, read_time=37925, write_time=4449, read_merged_count=5933, write_merged_count=35916, busy_time=20675),
 'sr0': sdiskio(read_count=0, write_count=0, read_bytes=0, write_bytes=0, read_time=0, write_time=0, read_merged_count=0, write_merged_count=0, busy_time=0),
 'dm-0': sdiskio(read_count=38566, write_count=5197, read_bytes=2483773952, write_bytes=55885312, read_time=37685, write_time=3546, read_merged_count=0, write_merged_count=0, busy_time=19410),
 'dm-1': sdiskio(read_count=6875, write_count=36059, read_bytes=30310400, write_bytes=147697664, read_time=1987, write_time=71537, read_merged_count=0, write_merged_count=0, busy_time=1673)}

2)psutil.disk_partitions

Returns all mounted disks as named tuples, including disk name, mount point, file system type, and so on.When all equals True, returns mount information containing special file systems such as / proc.

In [4]: psutil.disk_partitions()      #View mount point information                    
Out[4]: 
[sdiskpart(device='/dev/mapper/centos-root', mountpoint='/', fstype='xfs', opts='rw,seclabel,relatime,attr2,inode64,noquota'),
 sdiskpart(device='/dev/sda1', mountpoint='/boot', fstype='xfs', opts='rw,seclabel,relatime,attr2,inode64,noquota')]

In [5]: [device for device in psutil.disk_partitions() if device.mountpoint == '/']
Out[5]: [sdiskpart(device='/dev/mapper/centos-root', mountpoint='/', fstype='xfs', opts='rw,seclabel,relatime,attr2,inode64,noquota')]

In [6]: def get_disk_via_mountpoint(point):      #Create a function
    ...:     disk = [item for item in psutil.disk_partitions() if item.mountpoint == point] 
    ...:     return disk[0].device 
//No output                           

In [7]: get_disk_via_mountpoint('/')   #Call get_disk_via_mountpoint to see the mount point of'/'
Out[7]: '/dev/mapper/cl-root'

In [8]: get_disk_via_mountpoint('/boot') #Call get_disk_via_mountpoint to see the mount point of'/boot'
Out[8]: '/dev/sda1'

3)psutil.disk_usage

Returns the usage of the disk where path is located as a named tuple, including disk capacity, used disk capacity, disk space utilization, and so on.

In [9]: psutil.disk_usage('/')       
Out[9]: sdiskusage(total=18238930944, used=6775488512, free=11463442432, percent=37.1)

In [10]: psutil.disk_usage('/').percent
Out[10]: 37.2

In [11]: type(psutil.disk_usage('/').percent)
Out[11]: float

(4) Network

The network-related functions are as follows:

function details
psutil.net_io_counter([pernic]) Returns network io statistics for each network card in the current system in the form of named tuples, including the number of bytes sent and received, the number of packets sent and received, errors, and packet deletions.When pernic is True, statistics for all network cards are listed.
psutil.net_connections([kind]) Returns the details (namedtuple) of each network connection as a list.Named tuples contain information such as fd, family, type, laddr, raddr, status, pid, etc.kind represents the type of connection filtered and supports the following values: (inet by default)
psutil.net_if_addrs() Returns network card configuration information as a dictionary, including IP and mac addresses, subnet masks, and broadcast addresses.
psutil.net_if_stats() Returns details of the network card, including whether to start, communication type, transmission speed, and mtu.
psutil.users() Returns information about the currently logged on user, including user name, logon time, terminal, and host information, as named tuples
psutil.boot_time() Returns the system start time as a time stamp

1)psutil.net_io_counter

Returns network IO statistics for each network card in the current system in the form of named tuples, including the number of bytes sent and received, the number of packets sent and received, errors, and packet deletions.When pernic is True, statistics for all network cards are listed.Using the net_io_counter function performs the same functions as parsing/proc/net/dev file contents yourself.

In [1]: import psutil                

In [2]: psutil.net_io_counters()     
Out[2]: snetio(bytes_sent=720405, bytes_recv=3661606, packets_sent=5520, packets_recv=14886, errin=0, errout=0, dropin=0, dropout=0)

In [3]: psutil.net_io_counters(pernic=True) 
Out[3]: 
{'ens37': snetio(bytes_sent=724145, bytes_recv=3365944, packets_sent=5538, packets_recv=10017, errin=0, errout=0, dropin=0, dropout=0),
 'lo': snetio(bytes_sent=0, bytes_recv=0, packets_sent=0, packets_recv=0, errin=0, errout=0, dropin=0, dropout=0),
 'virbr0-nic': snetio(bytes_sent=0, bytes_recv=0, packets_sent=0, packets_recv=0, errin=0, errout=0, dropin=0, dropout=0),
 'virbr0': snetio(bytes_sent=0, bytes_recv=0, packets_sent=0, packets_recv=0, errin=0, errout=0, dropin=0, dropout=0),
 'ens33': snetio(bytes_sent=0, bytes_recv=298202, packets_sent=0, packets_recv=4899, errin=0, errout=0, dropin=0, dropout=0)}

2)net_connections

Returns a list of the details of each network connection (namedtuple), which can be used to view the status of the network connection, count the number of connections, and the number of network connections in a specific state.

In [4]: psutil.net_connections()      
Out[4]: 
[sconn(fd=6, family=<AddressFamily.AF_INET6: 10>, type=<SocketKind.SOCK_STREAM: 1>, laddr=addr(ip='::', port=111), raddr=(), status='LISTEN', pid=6558),
 sconn(fd=7, family=<AddressFamily.AF_INET6: 10>, type=<SocketKind.SOCK_DGRAM: 2>, laddr=addr(ip='::', port=111), raddr=(), status='NONE', pid=6558),
 sconn(fd=8, family=<AddressFamily.AF_INET6: 10>, type=<SocketKind.SOCK_STREAM: 1>, laddr=addr(ip='::1', port=6010), raddr=(), status='LISTEN', pid=9047),
 sconn(fd=6, family=<AddressFamily.AF_INET: 2>, type=<SocketKind.SOCK_STREAM: 1>,
......

In [5]: conns = psutil.net_connections()

In [6]: len([conn for conn in conns if conn.status == 'TIME_WAIT'])   
Out[6]: 0

3)net_if_addrs

Returns network card configuration information as a dictionary, including IP and mac addresses, subnet masks, and broadcast addresses.

In [7]: psutil.net_if_addrs()                                                     Out[7]: 
{'lo': [snicaddr(family=<AddressFamily.AF_INET: 2>, address='127.0.0.1', netmask='255.0.0.0', broadcast=None, ptp=None),
  snicaddr(family=<AddressFamily.AF_INET6: 10>, address='::1', netmask='ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff', broadcast=None, ptp=None),
  snicaddr(family=<AddressFamily.AF_PACKET: 17>, address='00:00:00:00:00:00', netmask=None, broadcast=None, ptp=None)],
 'ens37': [snicaddr(family=<AddressFamily.AF_INET: 2>, address='192.168.1.131', netmask='255.255.255.255', broadcast='192.168.1.131', ptp=None),
  snicaddr(family=<AddressFamily.AF_INET6: 10>, address='240e:82:e03:7342:4378:7be3:558c:fc88', netmask='ffff:ffff:ffff:ffff::', broadcast=None, ptp=None)
......

4)psutil.net_if_stats

Returns details of the network card, including whether to start, communication type, transmission speed, and mtu.

In [8]: psutil.net_if_stats()        
Out[8]: 
{'ens37': snicstats(isup=True, duplex=<NicDuplex.NIC_DUPLEX_FULL: 2>, speed=1000, mtu=1500),
 'lo': snicstats(isup=True, duplex=<NicDuplex.NIC_DUPLEX_UNKNOWN: 0>, speed=0, mtu=65536),
 'virbr0-nic': snicstats(isup=False, duplex=<NicDuplex.NIC_DUPLEX_FULL: 2>, speed=10, mtu=1500),
 'virbr0': snicstats(isup=True, duplex=<NicDuplex.NIC_DUPLEX_UNKNOWN: 0>, speed=0, mtu=1500),
 'ens33': snicstats(isup=True, duplex=<NicDuplex.NIC_DUPLEX_FULL: 2>, speed=1000, mtu=1500)}

(5) Other

1)users

Returns information about the currently logged on user, including user name, logon time, terminal, and host information, in a named tuple manner.

In [9]: psutil.users()              
Out[9]: 
[suser(name='root', terminal=':0', host='localhost', started=1582366080.0, pid=7991),
 suser(name='root', terminal='pts/0', host='localhost', started=1582366208.0, pid=8927),
 suser(name='root', terminal='pts/1', host='192.168.1.4', started=1582370816.0, pid=10099),
 suser(name='root', terminal='pts/3', host='192.168.1.4', started=1582369408.0, pid=9787)]

2)boot_time

Returns the start-up time of the system as a time stamp.

In [9]: import datetime

In [10]: psutil.boot_time()    
Out[10]: 1582527367.0

In [11]: datetime.datetime.fromtimestamp(psutil.boot_time()).strftime('%Y-%m-%d %H:%M:%S')                                  
Out[11]: '2020-02-24 14:56:07'

3. Comprehensive case: use psutil to implement monitoring program

# coding=utf-8
# !/usr/bin/python

import psutil
import datetime

def bytes2human(n):
    '''Method of Memory Unit Conversion'''
    symbols = ('K', 'M', 'G', 'T', 'P', 'E', 'Z', 'Y')
    prefix = {}
    for i, s in enumerate(symbols):
        prefix[s] = 1 << (i + 1) * 10
    for s in reversed(symbols):
        if n >= prefix[s]:
            value = float(n) / prefix[s]
            return '%.1f%s' % (value, s)
    return "%sB" % n

def get_cpu_info():
    '''Obtain CPU Usage'''
    cpu_count = psutil.cpu_count()
    cpu_percent = psutil.cpu_percent(interval=1)
    return dict(cpu_count=cpu_count, cpu_percent=cpu_percent)

def get_memory_info():
    '''Get memory information'''
    virtual_mem = psutil.virtual_memory()
    mem_total = bytes2human(virtual_mem.total)
    mem_percent = virtual_mem.percent
    mem_free = bytes2human(virtual_mem.free + virtual_mem.buffers + virtual_mem.cached)
    mem_used = bytes2human(virtual_mem.total * mem_percent / 100)
    return dict(mem_total=mem_total, mem_percent=mem_percent, mem_free=mem_free, mem_used=mem_used)

def get_disk_info():
    '''Get Disk Information'''
    disk_usage = psutil.disk_usage('/')
    disk_total = bytes2human(disk_usage.total)
    disk_percent = disk_usage.percent
    disk_free = bytes2human(disk_usage.free)
    disk_used = bytes2human(disk_usage.used)
    return dict(disk_total=disk_total, disk_percent=disk_percent, disk_free=disk_free, disk_used=disk_used)

def get_boot_info():
    '''Get Startup Time'''
    boot_time = datetime.datetime.fromtimestamp(psutil.boot_time()).strftime("%Y-%m-%d %H:%M:%S")
    return dict(boot_time=boot_time)

def collect_monitor_data():
    '''Centralized monitoring of hardware information'''
    data = {}
    data.update(get_boot_info())
    data.update(get_cpu_info())
    data.update(get_memory_info())
    data.update(get_disk_info())
    print(data)
    return data

collect_monitor_data()
The results are as follows
[root@python scripts]# python3 monitor_psutil.py
{'boot_time': '2020-05-06 16:23:37', 'cpu_count': 1, 'cpu_percent': 0.0, 'mem_total': '976.5M', 'mem_percent': 71.4, 'mem_free': '448.0M', 'mem_used': '697.2M', 'disk_total': '17.0G', 'disk_percent': 30.4, 'disk_free': '11.8G', 'disk_used': '5.2G'}

4. psutil process management

psutil also provides functionality as a process management function, including getting a list of processes, determining whether they exist, and class encapsulation for process management.

function details
psutil.Process() Encapsulate a process and use this class of methods to get detailed information about the process or to signal it.
psutil.pids() Return the currently running process as a list
psutil.pid_exists(1) Determine if a pid given to a point exists
psutil.process_iter() Iterates over currently running processes, returning the Process object for each process

1) Process class

Encapsulate a process and use this class of methods to get detailed information about the process or to signal it.

In [1]: import psutil                 

In [2]: init_process = psutil.Process()

In [3]: init_process.cmdline()      
Out[3]: ['/usr/local/python38/bin/python3.8', '/usr/local/python38/bin/ipython']

The Process class contains many ways to get detailed information about a process.Here are some of the more common methods:

Name: Get the name of the process
 cmdline: Get command line parameters to start the process
 create_time: Gets the creation time of the process (timestamp format)
num_fds: Number of files opened by the process
 num_threads: Number of child processes of a process
 is_running: Determines if the process is running
 send_signal: Sends a signal to a process, similar to os.kill, etc.
kill: Send a SIGKILL signal to end the process
 terminate: Send a SIGTEAM signal to end the process

2)pids

Returns the currently running process as a list.

In [1]: import psutil                 

In [2]: init_process = psutil.Process()

In [3]: init_process.cmdline()       
Out[3]: ['/usr/local/python38/bin/python3.8', '/usr/local/python38/bin/ipython']

In [4]: psutil.pids()[:5]            
Out[4]: [1, 2, 3, 5, 7]

3)pid_exists

Determine if the pid given to the point exists.

In [5]: psutil.pid_exists(1)          
Out[5]: True

In [6]: psutil.pid_exists(10245)                              
Out[6]: False

4)process_iter

Iterates over currently running processes, returning a Process object for each process, and pids returns a list of processes.

Tags: Python network IPython Linux

Posted on Fri, 08 May 2020 09:25:10 -0700 by erikjan