Shell programming and editing tool awk

This article mainly writes some shell script editing tool awk use.

Summary

awk is a powerful editing tool, which reads the input text line by line, searches according to the specified matching mode, and formats and outputs or filters the qualified content.

Awk tends to divide a row into multiple fields before processing, and by default, the field separator is space or tab. The results of awk execution can be printed and displayed through the function of print.

You can use the logical operators & &, and, | |, or,! To represent non; you can also use simple mathematical operations, such as +, -, *, /,%, ^, to represent addition, subtraction, multiplication, division, remainder and power respectively.

Command common usage

  • There are two formats for commands:
awk [options] 'mode or condition {edit instruction}' file 1 file 2...
awk -f script file 1 file 2...
  • Common built-in variables

FS: Specifies the field separator for each line of text. The default is space or tab stop.
NF: the number of fields in the current row.
NR: line number (ordinal number) of the currently processed line.
$0: the entire line content of the currently processed line.
$n: the nth field (column n) of the current processing row.
RS: data records are separated. The default value is \ n, i.e. one record per row.
FILENAME: the name of the file being processed.

Example

Output text by line

  • Output everything
[root@localhost ~]# awk '{print}' /etc/passwd
root:x:0:0:root:/root:/bin/bash
bin:x:1:1:bin:/bin:/sbin/nologin
daemon:x:2:2:daemon:/sbin:/sbin/nologin
adm:x:3:4:adm:/var/adm:/sbin/nologin
lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin
sync:x:5:0:sync:/sbin:/bin/sync
shutdown:x:6:0:shutdown:/sbin:/sbin/shutdown
halt:x:7:0:halt:/sbin:/sbin/halt
mail:x:8:12:mail:/var/spool/mail:/sbin/nologin
operator:x:11:0:operator:/root:/sbin/nologin
games:x:12:100:games:/usr/games:/sbin/nologin
ftp:x:14:50:FTP User:/var/ftp:/sbin/nologin
nobody:x:99:99:Nobody:/:/sbin/nologin
systemd-network:x:192:192:systemd Network Management:/:/sbin/nologin
dbus:x:81:81:System message bus:/:/sbin/nologin
polkitd:x:999:997:User for polkitd:/:/sbin/nologin
postfix:x:89:89::/var/spool/postfix:/sbin/nologin
sshd:x:74:74:Privilege-separated SSH:/var/empty/sshd:/sbin/nologin
chrony:x:998:996::/var/lib/chrony:/sbin/nologin
[root@localhost ~]# awk '{print $0}' /etc/passwd
root:x:0:0:root:/root:/bin/bash
bin:x:1:1:bin:/bin:/sbin/nologin
daemon:x:2:2:daemon:/sbin:/sbin/nologin
adm:x:3:4:adm:/var/adm:/sbin/nologin
lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin
sync:x:5:0:sync:/sbin:/bin/sync
shutdown:x:6:0:shutdown:/sbin:/sbin/shutdown
halt:x:7:0:halt:/sbin:/sbin/halt
mail:x:8:12:mail:/var/spool/mail:/sbin/nologin
operator:x:11:0:operator:/root:/sbin/nologin
games:x:12:100:games:/usr/games:/sbin/nologin
ftp:x:14:50:FTP User:/var/ftp:/sbin/nologin
nobody:x:99:99:Nobody:/:/sbin/nologin
systemd-network:x:192:192:systemd Network Management:/:/sbin/nologin
dbus:x:81:81:System message bus:/:/sbin/nologin
polkitd:x:999:997:User for polkitd:/:/sbin/nologin
postfix:x:89:89::/var/spool/postfix:/sbin/nologin
sshd:x:74:74:Privilege-separated SSH:/var/empty/sshd:/sbin/nologin
chrony:x:998:996::/var/lib/chrony:/sbin/nologin
  • Output lines 1-3
[root@localhost ~]# awk 'NR==1,NR==3{print}' /etc/passwd
root:x:0:0:root:/root:/bin/bash
bin:x:1:1:bin:/bin:/sbin/nologin
daemon:x:2:2:daemon:/sbin:/sbin/nologin
[root@localhost ~]# awk '(NR>=1)&&(NR<=3){print}' /etc/passwd
root:x:0:0:root:/root:/bin/bash
bin:x:1:1:bin:/bin:/sbin/nologin
daemon:x:2:2:daemon:/sbin:/sbin/nologin
[root@localhost ~]# awk 'NR>=1&&NR<=3{print}' /etc/passwd
root:x:0:0:root:/root:/bin/bash
bin:x:1:1:bin:/bin:/sbin/nologin
daemon:x:2:2:daemon:/sbin:/sbin/nologin
  • Output line 1, line 3
[root@localhost ~]# awk '(NR==1)||(NR==3){print}' /etc/passwd
root:x:0:0:root:/root:/bin/bash
daemon:x:2:2:daemon:/sbin:/sbin/nologin
[root@localhost ~]# awk 'NR==1||NR==3{print}' /etc/passwd
root:x:0:0:root:/root:/bin/bash
daemon:x:2:2:daemon:/sbin:/sbin/nologin
  • Output all odd rows
[root@localhost ~]# awk '(NR%2)==1{print}' /etc/passwd
root:x:0:0:root:/root:/bin/bash
daemon:x:2:2:daemon:/sbin:/sbin/nologin
lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin
shutdown:x:6:0:shutdown:/sbin:/sbin/shutdown
mail:x:8:12:mail:/var/spool/mail:/sbin/nologin
games:x:12:100:games:/usr/games:/sbin/nologin
nobody:x:99:99:Nobody:/:/sbin/nologin
dbus:x:81:81:System message bus:/:/sbin/nologin
postfix:x:89:89::/var/spool/postfix:/sbin/nologin
chrony:x:998:996::/var/lib/chrony:/sbin/nologin
  • Output all even rows
[root@localhost ~]# awk '(NR%2)==0{print}' /etc/passwd
bin:x:1:1:bin:/bin:/sbin/nologin
adm:x:3:4:adm:/var/adm:/sbin/nologin
sync:x:5:0:sync:/sbin:/bin/sync
halt:x:7:0:halt:/sbin:/sbin/halt
operator:x:11:0:operator:/root:/sbin/nologin
ftp:x:14:50:FTP User:/var/ftp:/sbin/nologin
systemd-network:x:192:192:systemd Network Management:/:/sbin/nologin
polkitd:x:999:997:User for polkitd:/:/sbin/nologin
sshd:x:74:74:Privilege-separated SSH:/var/empty/sshd:/sbin/nologin
  • Output lines starting with root
[root@localhost ~]# awk '/^root/{print}' /etc/passwd
root:x:0:0:root:/root:/bin/bash
  • Output lines ending in / bin/bash
[root@localhost ~]# awk '/\/bin\/bash$/{print}' /etc/passwd
root:x:0:0:root:/root:/bin/bash
  • Count the number of lines ending in / bin/bash
[root@localhost ~]# awk 'BEGIN{x=0};/\/bin\/bash$/{x++};END{print x}' /etc/passwd
1
[root@localhost ~]# grep -c '/bin/bash$' /etc/passwd
1
[root@localhost ~]# grep '/bin/bash$' /etc/passwd | wc -l
1
  • Count the number of text paragraphs separated by
[root@localhost ~]# awk 'BEGIN{RS=":"};END{print NR}' /etc/passwd
115

Output text by field

  • Output the third field in each row
[root@localhost ~]# awk -F":" '{print $3}' /etc/passwd
0
1
2
3
4
5
6
7
8
11
12
14
99
192
81
999
89
74
998
  • Output the first and third fields in each row
[root@localhost ~]# awk -F":" '{print $1,$3}' /etc/passwd
root 0
bin 1
daemon 2
adm 3
lp 4
sync 5
shutdown 6
halt 7
mail 8
operator 11
games 12
ftp 14
nobody 99
systemd-network 192
dbus 81
polkitd 999
postfix 89
sshd 74
chrony 998
  • Output user record with UID 0
[root@localhost ~]# awk -F":" '$3=="0"{print}' /etc/passwd
root:x:0:0:root:/root:/bin/bash
[root@localhost ~]# awk 'BEGIN{FS=":"};$3=="0"{print}' /etc/passwd
root:x:0:0:root:/root:/bin/bash
  • Output field 7 the first field of the row containing / bin/bash
[root@localhost ~]# awk -F":" '$7~"/bin/bash"{print $1}' /etc/passwd
root
  • Output the first, third and seventh fields in the row with the first field containing sshd and seven fields
[root@localhost ~]# awk -F":" '($1~"sshd")&&(NF==7){print $1,$3,$7}' /etc/passwd
sshd 74 /sbin/nologin
  • Output the seventh field is neither all lines of / bin/bash nor / sbin/nologin
[root@localhost ~]# awk -F":" '($7!="/bin/bash")&&($7!="/sbin/nologin"){print}' /etc/passwd
sync:x:5:0:sync:/sbin:/bin/sync
shutdown:x:6:0:shutdown:/sbin:/sbin/shutdown
halt:x:7:0:halt:/sbin:/sbin/halt

Pipe, double quotation mark to call shell command

  • Using wc-l to count the number of users using bash
[root@localhost ~]# awk -F":" '/bash$/{print | "wc -l"}' /etc/passwd
1
  • Call the w command and count the number of online users
[root@localhost ~]# awk 'BEGIN{while("w" | getline)n++;{print n-2}}'
1
  • Call hostname and output the current hostname
[root@localhost ~]# awk 'BEGIN{"hostname" | getline;print $0}'
localhost

Tags: Linux ftp network DBus ssh

Posted on Tue, 15 Oct 2019 12:45:47 -0700 by bettyatolive