JGsoft RegexBuddy

awk

1. Common usage of awk

Usually, the command format used by awk is as follows, where single quotation marks plus braces "{}" are used to set the processing actions for data. Awk can process the target file directly, or it can process the target file by reading the script "-f". By default, the field separator is a space or tab key. The result of awk execution can be printed and displayed by print function. In the process of using awk command, we can use the logical operator "&", "and", "or", "and", "or", "and"." It can also carry out simple mathematical operations, such as +, -,*, /,%, ^ for addition, subtraction, multiplication, division, redundancy and multiplication, respectively.

awk option'mode or condition {editing instructions}'file 1 file 2 // filters and outputs the contents of file character conditions

awk -f script file file 1 file 2 / / from the script to edit instructions, filter and output content

2 Built-in variables

FS: Specifies a field separator for each line of text, defaulting to a space or tab.
NF: Number of fields in rows currently processed.
NR: The line number (ordinal number) of the row being processed.
$0: The entire line of the row being processed.
$n: The nth field (column n) of the current processing row.
FILENAME: The name of the file being processed.
RS: Data records are separated by default to n, which means one record per action.

3 usage examples

1) Output text by line

awk '{print}' test.txt //Output of all content, equivalent to cat test.txt

awk '{print $0}' test.txt //Output of all content, equivalent to cat test.txt

awk 'NR==1,NR==3{print}' test.txt //Output lines 1-3
awk '(NR>=1)&&(NR<=3){print}' test.txt //Output lines 1-3
awk 'NR==1||NR==3{print}' test.txt //Output lines 1 and 3
awk '(NR%2)==1{print}' test.txt //Output the contents of all odd lines
awk '(NR%2)==0{print}' test.txt //Output the contents of all even lines
awk '/^root/{print}' /etc/passwd //Output lines starting with root

awk '/nologin$/{print}' /etc/passwd //Output lines ending with nologin

awk 'BEGIN {x=0} ; /\/bin\/bash$/{x++};END {print x}' /etc/passwd
//Statistically, the number of rows ending in / bin/bash is equivalent to grep - C "/ bin/bash $"/ etc / passwd awk

'BEGIN{RS=""};END{print NR}' /etc/squid/squid.conf
//Number of text paragraphs separated by blank lines

2) Output text by field

awk '{print $3}' test.txt //Output the third field in each row (separated by spaces or tabs)
awk '{print $1,$3}' test.txt //Output the first and third fields in each row
awk -F ":" '$2==""{print}' /etc/shadow //shadow record of user whose password is empty

awk 'BEGIN {FS=":"}; $2==""{print}' /etc/shadow
//shadow record of user whose password is empty

awk -F ":" '$7~"/bash"{print $1}' /etc/passwd
//The output is colon-separated and the first field of the row containing / bash in the seventh field

awk '($1~"nfs")&&(NF==8){print $1,$2}' /etc/services
//The output contains eight fields and the first field contains the first and second fields of rows with nfs

awk -F ":" '($7!="/bin/bash")&&($7!="/sbin/nologin"){print}'/etc/passwd
//Output field 7 is neither / bin/bash nor all rows of / sbin/nologin

3) Call Shell commands through pipes and double quotation marks

awk -F: '/bash$/{print | "wc -l"}' /etc/passwd
//Calling the wc-l command counts the number of users using bash, which is equivalent to grep-c "bash$"/etc/passwd

awk 'BEGIN {while ("w" | getline) n++ ; {print n-2}}'
//Call the w command and use it to count the number of online users

awk 'BEGIN { "hostname" | getline ; print $0}'
//Call hostname and output the current hostname

sort

Sort is a tool for sorting the contents of files in units of behavior, and it can also be sorted according to different data types. For example, data and characters play different cards. The syntax of the sort command is "sort [option] parameter", and the commonly used options include the following.

-f: ignoring case and case;
-b: Ignore the space in front of each line;
-M: Sort by month;
-n: Sort by number;
-r: Reverse sort;
-u: Equal to uniq, indicating that the same data only shows one row;
-t: Specify the separator, default using the [Tab] key to separate;
-o < Output File >: Transfer the sorted results to the specified file;
-k: Specifies the sorting region.


uniq

Uniq tools are commonly used in Linux systems in conjunction with sort commands to report or ignore duplicate lines in files. The specific command grammar format is: uniq [option] parameter. Common options include the following.

- c: counting;
- d: Only duplicate rows are displayed;

- u: Shows only rows that appear once;

Take the following text as an example

Tags: Linux shell

Posted on Thu, 10 Oct 2019 10:17:51 -0700 by Chris 96 WS6