Ceci est une ancienne révision du document !
Table des matières
Version : 2022.01
Updated: 2022/10/03 05:11
Topic 1 - Capacity Planning
Contents
- Topic 1 - Capacity Planning
- Contents
- LAB #1 - Measuring System Resource Usage
- 1.1 - free
- 1.2 - uptime ou w
- 1.3 - iostat
- 1.4 - hdparm
- 1.5 - vmstat
- 1.6 - mpstat
- 1.7 - sar
- LAB #2 - Viewing Open Files and Running Processes
- 2.1 - Presentation
- 2.2 - Process Types
- 2.3 - Process Commands
- The ps Command
- The pstree Command
- The top Command
- The lsof Command
LAB#1 - Interpreting Information in /proc
The information found in files in the /proc filesystem can be interpreted using the following commands:
- free,
- uptime et w,
- iostat,
- hdparm
- vmstat,
- mpstat,
- sar.
1.1 - The free Command
The free command shows total, used, free, shared, buffered, cached and swapped memory:
[root@centos8 ~]# free -m
total used free shared buff/cache available
Mem: 3735 1135 1818 14 782 2363
Swap: 3279 0 3279
In the above example, you can see:
- 3735 MB of total physical memory,
- 1135 MB of used physical memory and 1818 MB of free physical memory,
- 3279 MB of swap space with 0MB being used.
The command line switches of this command are:
[root@centos8 ~]# free --help
Usage:
free [options]
Options:
-b, --bytes show output in bytes
--kilo show output in kilobytes
--mega show output in megabytes
--giga show output in gigabytes
--tera show output in terabytes
--peta show output in petabytes
-k, --kibi show output in kibibytes
-m, --mebi show output in mebibytes
-g, --gibi show output in gibibytes
--tebi show output in tebibytes
--pebi show output in pebibytes
-h, --human show human-readable output
--si use powers of 1000 not 1024
-l, --lohi show detailed low and high memory statistics
-t, --total show total for RAM + swap
-s N, --seconds N repeat printing every N seconds
-c N, --count N repeat printing N times, then exit
-w, --wide wide output
--help display this help and exit
-V, --version output version information and exit
For more details see free(1).
1.2 - The uptime and w Commands
Each of these commands show the load average over the past 1, 5 and 15 minutes:
[root@centos8 ~]# uptime 04:39:03 up 1 day, 2:34, 1 user, load average: 0.00, 0.00, 0.00 [root@centos8 ~]# w 04:39:04 up 1 day, 2:34, 1 user, load average: 0.00, 0.00, 0.00 USER TTY FROM LOGIN@ IDLE JCPU PCPU WHAT trainee pts/0 10.0.2.1 03:58 0.00s 0.11s 0.02s sshd: trainee [priv]
The load average indicates the number of processes being executed and waiting to be executed for the period concerned.
If the load average of a single-core system was 3.48 4.00 3.85 this would indicate a bottleneck since, on average:
- 2.48 processes would have been waiting to be executed over the last minute,
- 3.00 processes would have been waiting to be executed over the last 5 minutes,
- 2.85 processes would have been waiting to be executed over the last 15 minutes,
The command line switches of these commands are:
[root@centos8 ~]# uptime --help
Usage:
uptime [options]
Options:
-p, --pretty show uptime in pretty format
-h, --help display this help and exit
-s, --since system up since
-V, --version output version information and exit
For more details see uptime(1).
[root@centos8 ~]# w --help
Usage:
w [options]
Options:
-h, --no-header do not print header
-u, --no-current ignore current process username
-s, --short short format
-f, --from show remote hostname field
-o, --old-style old style output
-i, --ip-addr display IP address instead of hostname (if possible)
--help display this help and exit
-V, --version output version information and exit
For more details see w(1).
1.3 - The iostat Command
The iostat command show disk, terminal and streamer statistics:
[root@centos8 ~]# iostat
bash: iostat: command not found...
Install package 'sysstat' to provide command 'iostat'? [N/y] y
* Waiting in queue...
The following packages have to be installed:
lm_sensors-libs-3.4.0-22.20180522git70f7e08.el8.x86_64 Lm_sensors core libraries
sysstat-11.7.3-5.el8.x86_64 Collection of performance monitoring tools for Linux
Proceed with changes? [N/y] y
* Waiting in queue...
* Waiting for authentication...
* Waiting in queue...
* Downloading packages...
* Requesting data...
* Testing changes...
* Installing packages...
Linux 4.18.0-305.3.1.el8.x86_64 (centos8.ittraining.loc) 29/06/21 _x86_64_(8 CPU)
avg-cpu: %user %nice %system %iowait %steal %idle
0.03 0.00 0.03 0.01 0.00 99.93
Device tps kB_read/s kB_wrtn/s kB_read kB_wrtn
sda 0.28 7.67 1.49 735338 142510
sdb 0.00 0.03 0.00 2995 0
scd0 0.00 0.00 0.00 2 0
dm-0 0.26 7.25 1.55 694786 148837
dm-1 0.00 0.02 0.00 2220 0
[root@centos8 ~]# iostat -d -x Linux 4.18.0-305.3.1.el8.x86_64 (centos8.ittraining.loc) 30/06/21 _x86_64_ (8 CPU) Device r/s w/s rkB/s wkB/s rrqm/s wrqm/s %rrqm %wrqm r_await w_await aqu-sz rareq-sz wareq-sz svctm %util sda 0.20 0.16 11.67 1.81 0.00 0.03 0.48 17.45 6.79 69.99 0.01 58.28 11.33 9.13 0.33 sdb 0.00 0.00 0.02 0.00 0.00 0.00 0.00 0.00 0.13 0.00 0.00 8.81 0.00 0.51 0.00 scd0 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.20 0.00 0.00 0.20 0.00 0.90 0.00 dm-0 0.12 0.19 3.98 1.90 0.00 0.00 0.00 0.00 4.18 79.07 0.02 32.88 10.07 10.34 0.32 dm-1 0.00 0.00 0.01 0.00 0.00 0.00 0.00 0.00 0.76 0.00 0.00 21.35 0.00 0.68 0.00
The command line switches of this command are:
[root@centos8 ~]# iostat --help
Usage: iostat [ options ] [ <interval> [ <count> ] ]
Options are:
[ -c ] [ -d ] [ -h ] [ -k | -m ] [ -N ] [ -s ] [ -t ] [ -V ] [ -x ] [ -y ] [ -z ]
[ -j { ID | LABEL | PATH | UUID | ... } ] [ --human ] [ -o JSON ]
[ [ -H ] -g <group_name> ] [ -p [ <device> [,...] | ALL ] ]
[ <device> [...] | ALL ]
1.4 - The hdparm Command
The hdparm command measures disk reads:
[root@centos8 ~]# hdparm -t /dev/sda /dev/sda: Timing buffered disk reads: 1410 MB in 3.00 seconds = 469.98 MB/sec
1.5 - La Commande vmstat
The vmstat commands shows memory, pagination and processor statistics:
[root@centos8 ~]# vmstat 1 10 procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu----- r b swpd free buff cache si so bi bo in cs us sy id wa st 0 0 0 1765216 2256 866336 0 0 3 0 9 12 0 0 100 0 0 0 0 0 1765136 2256 866336 0 0 0 0 57 80 0 0 100 0 0 0 0 0 1765136 2256 866376 0 0 0 0 54 77 0 0 100 0 0 0 0 0 1765136 2256 866376 0 0 0 0 66 100 0 0 100 0 0 0 0 0 1765136 2256 866376 0 0 0 0 103 125 0 0 100 0 0 0 0 0 1765108 2256 866376 0 0 0 0 64 86 0 0 100 0 0 0 0 0 1765108 2256 866376 0 0 0 0 62 88 0 0 100 0 0 0 0 0 1765108 2256 866376 0 0 0 0 68 97 0 0 100 0 0 0 0 0 1765108 2256 866376 0 0 0 0 60 88 0 0 100 0 0 0 0 0 1765108 2256 866376 0 0 0 0 177 251 0 0 100 0 0
The command line switches of this command are:
[root@centos8 ~]# vmstat --help Usage: vmstat [options] [delay [count]] Options: -a, --active active/inactive memory -f, --forks number of forks since boot -m, --slabs slabinfo -n, --one-header do not redisplay header -s, --stats event counter statistics -d, --disk disk statistics -D, --disk-sum summarize disk statistics -p, --partition <dev> partition specific statistics -S, --unit <char> define display unit -w, --wide wide output -t, --timestamp show timestamp -h, --help display this help and exit -V, --version output version information and exit For more details see vmstat(8).
Important : By default vmstat shows statistics from boot until current time.
1.6 - The mpstat Command
La commande mpstat affiche des statistiques détaillées sur le CPU :
[root@centos8 ~]# mpstat Linux 4.18.0-305.3.1.el8.x86_64 (centos8.ittraining.loc) 29/06/21 _x86_64_(8 CPU) 04:53:22 CPU %usr %nice %sys %iowait %irq %soft %steal %guest %gnice %idle 04:53:22 all 0.03 0.00 0.01 0.02 0.01 0.01 0.00 0.00 0.00 99.92
If there are several CPU's in the system, statistics can be viewed by core and as an average:
[root@centos8 ~]# mpstat -P ALL Linux 4.18.0-305.3.1.el8.x86_64 (centos8.ittraining.loc) 29/06/21 _x86_64_(8 CPU) 04:54:28 CPU %usr %nice %sys %iowait %irq %soft %steal %guest %gnice %idle 04:54:28 all 0.03 0.00 0.01 0.02 0.01 0.01 0.00 0.00 0.00 99.92 04:54:28 0 0.03 0.00 0.01 0.00 0.01 0.00 0.00 0.00 0.00 99.94 04:54:28 1 0.02 0.00 0.02 0.03 0.02 0.00 0.00 0.00 0.00 99.91 04:54:28 2 0.02 0.00 0.01 0.01 0.01 0.04 0.00 0.00 0.00 99.90 04:54:28 3 0.01 0.00 0.01 0.02 0.00 0.00 0.00 0.00 0.00 99.95 04:54:28 4 0.05 0.00 0.02 0.03 0.01 0.00 0.00 0.00 0.00 99.88 04:54:28 5 0.03 0.01 0.01 0.02 0.01 0.00 0.00 0.00 0.00 99.92 04:54:28 6 0.02 0.00 0.02 0.01 0.01 0.00 0.00 0.00 0.00 99.95 04:54:28 7 0.02 0.00 0.01 0.01 0.01 0.00 0.00 0.00 0.00 99.94
Finally, mpstat is capable of showing statistics over time. In the following example you can see 5 data sets, one taken every 2 seconds:
[root@centos8 ~]# mpstat -P ALL 2 5 Linux 4.18.0-305.3.1.el8.x86_64 (centos8.ittraining.loc) 29/06/21 _x86_64_ (8 CPU) 04:55:11 CPU %usr %nice %sys %iowait %irq %soft %steal %guest %gnice %idle 04:55:13 all 0.00 0.00 0.00 0.00 0.00 0.00 0.06 0.00 0.00 99.94 04:55:13 0 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 100.00 04:55:13 1 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 100.00 04:55:13 2 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 100.00 04:55:13 3 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 100.00 04:55:13 4 0.00 0.00 0.00 0.00 0.00 0.00 0.50 0.00 0.00 99.50 04:55:13 5 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 100.00 04:55:13 6 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 100.00 04:55:13 7 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 100.00 04:55:13 CPU %usr %nice %sys %iowait %irq %soft %steal %guest %gnice %idle 04:55:15 all 0.00 0.00 0.00 0.00 0.06 0.00 0.00 0.00 0.00 99.94 04:55:15 0 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 100.00 04:55:15 1 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 100.00 04:55:15 2 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 100.00 04:55:15 3 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 100.00 04:55:15 4 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 100.00 04:55:15 5 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 100.00 04:55:15 6 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 100.00 04:55:15 7 0.00 0.00 0.00 0.00 0.50 0.00 0.00 0.00 0.00 99.50 04:55:15 CPU %usr %nice %sys %iowait %irq %soft %steal %guest %gnice %idle 04:55:17 all 0.00 0.00 0.00 0.00 0.06 0.00 0.00 0.00 0.00 99.94 04:55:17 0 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 100.00 04:55:17 1 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 100.00 04:55:17 2 0.00 0.00 0.00 0.00 0.50 0.00 0.00 0.00 0.00 99.50 04:55:17 3 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 100.00 04:55:17 4 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 100.00 04:55:17 5 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 100.00 04:55:17 6 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 100.00 04:55:17 7 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 100.00 04:55:17 CPU %usr %nice %sys %iowait %irq %soft %steal %guest %gnice %idle 04:55:19 all 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 100.00 04:55:19 0 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 100.00 04:55:19 1 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 100.00 04:55:19 2 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 100.00 04:55:19 3 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 100.00 04:55:19 4 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 100.00 04:55:19 5 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 100.00 04:55:19 6 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 100.00 04:55:19 7 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 100.00 04:55:19 CPU %usr %nice %sys %iowait %irq %soft %steal %guest %gnice %idle 04:55:21 all 0.06 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 99.94 04:55:21 0 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 100.00 04:55:21 1 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 100.00 04:55:21 2 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 100.00 04:55:21 3 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 100.00 04:55:21 4 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 100.00 04:55:21 5 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 100.00 04:55:21 6 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 100.00 04:55:21 7 0.50 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 99.50 Average: CPU %usr %nice %sys %iowait %irq %soft %steal %guest %gnice %idle Average: all 0.01 0.00 0.00 0.00 0.02 0.00 0.01 0.00 0.00 99.95 Average: 0 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 100.00 Average: 1 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 100.00 Average: 2 0.00 0.00 0.00 0.00 0.10 0.00 0.00 0.00 0.00 99.90 Average: 3 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 100.00 Average: 4 0.00 0.00 0.00 0.00 0.00 0.00 0.10 0.00 0.00 99.90 Average: 5 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 100.00 Average: 6 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 100.00 Average: 7 0.10 0.00 0.00 0.00 0.10 0.00 0.00 0.00 0.00 99.80
The command line switches of this command are:
[root@centos8 ~]# mpstat --help
Usage: mpstat [ options ] [ <interval> [ <count> ] ]
Options are:
[ -A ] [ -n ] [ -u ] [ -V ] [ -I { SUM | CPU | SCPU | ALL } ]
[ -N { <node_list> | ALL } ] [ -o JSON ] [ -P { <cpu_list> | ALL } ]
1.7 - The sar Command
The sar command can survey all system resources dependant upon the switch that is used. Several important switches are:
| Switch | Description |
|---|---|
| -u | CPU usage in % |
| -q | Number of processes in the process queue |
| -r | Memory usage |
| -w | Swap usage |
| -p | Pagination usage |
| -b | Buffer usage |
| -d | Disk usage |
The /usr/lib64/sa/sadc command is used to collect data:
[root@centos8 ~]# ls /usr/lib64/sa sa1 sa2 sadc
The /usr/lib64/sa/sa1 script calls the /usr/lib/sa/sadc command and can use two switches:
| Switch | Description |
|---|---|
| -t | Interval |
| -n | Count |
The /usr/lib64/sa/sa2 script also creates a log at /var/log/sa/sar<dd>, where <dd> is the day of the month.
[root@centos8 ~]# ls /var/log/sa/ sa29 s ar29
Using CentOS / RHEL 8, the interval between collects is configured using systemd timers de systemd and not cron as was previously the case:
[root@centos8 ~]# cat /usr/lib/systemd/system/sysstat-collect.timer # /usr/lib/systemd/system/sysstat-collect.timer # (C) 2014 Tomasz Torcz <tomek@pipebreaker.pl> # # sysstat-11.7.3 systemd unit file: # Activates activity collector every 10 minutes [Unit] Description=Run system activity accounting tool every 10 minutes [Timer] OnCalendar=*:00/10 [Install] WantedBy=sysstat.service
The OnCalendar value indicates a collect every 10 minutes.
To change this value, you need to create an override file in /etc/systemd/system/ by using the systemctl edit command. You should never edit directly files in /usr/lib/systemd/system :
[root@centos8 ~]# systemctl edit sysstat-collect.timer [root@centos8 ~]# cat /etc/systemd/system/sysstat-collect.timer.d/override.conf [Unit] Description=Run system activity accounting tool every 2 minutes Interpreting Information in /proc [Timer] OnCalendar= OnCalendar=*:00/2 AccuracySec=0
Important : Note the line OnCalendar= which is required to override the default value.
Now check if the configuration has been applied:
[root@centos8 ~]# systemctl status sysstat-collect.timer
● sysstat-collect.timer - Run system activity accounting tool every 2 minutes
Loaded: loaded (/usr/lib/systemd/system/sysstat-collect.timer; enabled; vendor preset: disabled)
Drop-In: /etc/systemd/system/sysstat-collect.timer.d
└─override.conf
Active: active (waiting) since Tue 2021-06-29 06:16:04 EDT; 3h 2min ago
Trigger: Tue 2021-06-29 09:20:00 EDT; 1min 19s left
Jun 29 06:16:04 centos8.ittraining.loc systemd[1]: Started Run system activity accounting tool every 10 minutes.
[root@centos8 ~]# journalctl -g sysstat-collect.service -- Logs begin at Mon 2021-06-28 02:04:10 EDT, end at Tue 2021-06-29 09:18:00 EDT. -- Jun 29 06:20:33 centos8.ittraining.loc systemd[1]: sysstat-collect.service: Succeeded. Jun 29 06:26:29 centos8.ittraining.loc systemd[1]: sysstat-collect.service: Succeeded. Jun 29 06:30:33 centos8.ittraining.loc systemd[1]: sysstat-collect.service: Succeeded. Jun 29 06:40:33 centos8.ittraining.loc systemd[1]: sysstat-collect.service: Succeeded. Jun 29 06:50:33 centos8.ittraining.loc systemd[1]: sysstat-collect.service: Succeeded. Jun 29 07:00:33 centos8.ittraining.loc systemd[1]: sysstat-collect.service: Succeeded. Jun 29 07:10:33 centos8.ittraining.loc systemd[1]: sysstat-collect.service: Succeeded. Jun 29 07:20:33 centos8.ittraining.loc systemd[1]: sysstat-collect.service: Succeeded. Jun 29 07:30:33 centos8.ittraining.loc systemd[1]: sysstat-collect.service: Succeeded. Jun 29 07:40:33 centos8.ittraining.loc systemd[1]: sysstat-collect.service: Succeeded. Jun 29 07:50:33 centos8.ittraining.loc systemd[1]: sysstat-collect.service: Succeeded. Jun 29 07:53:56 centos8.ittraining.loc systemd[1]: sysstat-collect.service: Succeeded. Jun 29 07:54:00 centos8.ittraining.loc systemd[1]: sysstat-collect.service: Succeeded. Jun 29 07:56:00 centos8.ittraining.loc systemd[1]: sysstat-collect.service: Succeeded. Jun 29 07:58:00 centos8.ittraining.loc systemd[1]: sysstat-collect.service: Succeeded. Jun 29 08:00:00 centos8.ittraining.loc systemd[1]: sysstat-collect.service: Succeeded. ...
Execute the sar command:
[root@centos8 ~]# sar Linux 4.18.0-305.3.1.el8.x86_64 (centos8.ittraining.loc) 29/06/21 _x86_64_ (8 CPU) 06:16:04 LINUX RESTART (8 CPU) 06:20:33 CPU %user %nice %system %iowait %steal %idle 06:26:29 all 0.03 0.00 0.03 0.00 0.00 99.94 06:30:33 all 0.02 0.00 0.03 0.00 0.00 99.95 06:40:33 all 0.02 0.00 0.03 0.00 0.00 99.94 06:50:33 all 0.02 0.00 0.02 0.00 0.00 99.95 07:00:33 all 0.02 0.00 0.02 0.00 0.00 99.95 07:10:33 all 0.02 0.00 0.02 0.00 0.00 99.95 07:20:33 all 0.02 0.00 0.03 0.00 0.00 99.95 07:30:33 all 0.02 0.01 0.03 0.00 0.00 99.94 07:40:33 all 0.03 0.00 0.04 0.00 0.00 99.93 07:50:33 all 0.03 0.00 0.03 0.00 0.00 99.94 07:53:56 all 0.08 0.00 0.06 0.00 0.00 99.86 07:54:00 all 0.09 0.00 0.06 0.00 0.00 99.85 Average: all 0.03 0.00 0.03 0.00 0.00 99.94 07:55:44 LINUX RESTART (8 CPU) 07:56:00 CPU %user %nice %system %iowait %steal %idle 07:58:00 all 0.03 0.00 0.03 0.00 0.00 99.94 08:00:00 all 0.02 0.00 0.04 0.00 0.00 99.94 08:02:00 all 0.02 0.00 0.03 0.00 0.00 99.94 08:04:00 all 0.02 0.00 0.03 0.00 0.00 99.95 08:06:00 all 0.02 0.00 0.03 0.00 0.00 99.95 08:08:00 all 0.02 0.00 0.04 0.00 0.00 99.94 08:10:00 all 0.02 0.00 0.03 0.00 0.00 99.95 08:12:00 all 0.03 0.00 0.03 0.00 0.00 99.95 08:14:00 all 0.02 0.00 0.03 0.01 0.00 99.94 08:16:00 all 0.02 0.00 0.03 0.00 0.00 99.95 08:18:00 all 0.02 0.00 0.03 0.00 0.00 99.95 08:20:00 all 0.02 0.00 0.04 0.00 0.00 99.94 08:22:00 all 0.02 0.00 0.03 0.00 0.00 99.95 08:24:00 all 0.02 0.00 0.02 0.00 0.00 99.95 08:26:00 all 0.02 0.00 0.03 0.00 0.00 99.95 08:28:00 all 0.02 0.00 0.04 0.00 0.00 99.94 08:30:00 all 0.02 0.05 0.05 0.00 0.00 99.87 08:32:00 all 0.02 0.00 0.04 0.00 0.00 99.94 08:34:00 all 0.02 0.00 0.04 0.00 0.00 99.94 08:36:00 all 0.03 0.00 0.04 0.00 0.00 99.94 08:38:00 all 0.02 0.00 0.04 0.00 0.00 99.94 08:40:00 all 0.02 0.00 0.04 0.00 0.00 99.94 08:42:00 all 0.02 0.00 0.03 0.00 0.00 99.94 08:44:00 all 0.03 0.00 0.03 0.00 0.00 99.94 08:46:00 all 0.02 0.00 0.03 0.00 0.00 99.94 08:48:00 all 0.03 0.00 0.03 0.00 0.00 99.95 08:50:00 all 0.02 0.00 0.04 0.00 0.00 99.94 08:52:00 all 0.02 0.00 0.06 0.00 0.00 99.92 08:54:00 all 0.02 0.00 0.03 0.00 0.00 99.95 08:56:00 all 0.02 0.00 0.04 0.00 0.00 99.94 08:58:00 all 0.02 0.00 0.02 0.00 0.00 99.96 09:00:00 all 0.07 0.00 0.05 0.00 0.00 99.88 09:02:00 all 0.02 0.00 0.04 0.00 0.00 99.94 09:04:00 all 0.02 0.00 0.03 0.00 0.00 99.95 09:06:00 all 0.02 0.00 0.04 0.00 0.00 99.94 09:08:00 all 0.02 0.00 0.04 0.00 0.00 99.94 09:10:00 all 0.02 0.00 0.03 0.00 0.00 99.95 09:12:00 all 0.02 0.00 0.03 0.00 0.00 99.94 09:12:00 CPU %user %nice %system %iowait %steal %idle 09:14:00 all 0.02 0.00 0.03 0.00 0.00 99.95 09:16:00 all 0.02 0.00 0.06 0.00 0.00 99.92 09:18:00 all 0.03 0.00 0.03 0.00 0.00 99.95 09:20:00 all 0.02 0.00 0.03 0.00 0.00 99.94 Average: all 0.02 0.00 0.03 0.00 0.00 99.94
CPU Stats
Use the -u switch:
[root@centos8 ~]# sar -u 5 3 Linux 4.18.0-305.3.1.el8.x86_64 (centos8.ittraining.loc) 29/06/21 _x86_64_ (8 CPU) 09:22:52 CPU %user %nice %system %iowait %steal %idle 09:22:57 all 0.03 0.00 0.03 0.00 0.00 99.95 09:23:02 all 0.03 0.00 0.03 0.00 0.00 99.95 09:23:07 all 0.00 0.00 0.03 0.00 0.00 99.97 Average: all 0.02 0.00 0.03 0.00 0.00 99.96
More information can be shown by using the ALL argument:
[root@centos8 ~]# sar -u ALL 5 3 Linux 4.18.0-305.3.1.el8.x86_64 (centos8.ittraining.loc) 29/06/21 _x86_64_(8 CPU) 01:49:14 CPU %usr %nice %sys %iowait %steal %irq %soft %guest %gnice %idle 01:49:19 all 0.03 0.00 0.00 0.00 0.00 0.03 0.00 0.00 0.00 99.95 01:49:24 all 0.03 0.00 0.03 0.00 0.00 0.10 0.05 0.00 0.00 99.80 01:49:29 all 0.00 0.00 0.00 0.25 0.00 0.10 0.05 0.00 0.00 99.60 Average: all 0.02 0.00 0.01 0.08 0.00 0.08 0.03 0.00 0.00 99.78
To see the statistics from a specific core, use the -P switch:
[root@centos8 ~]# sar -u -P 1 5 3 Linux 4.18.0-305.3.1.el8.x86_64 (centos8.ittraining.loc) 29/06/21 _x86_64_ (8 CPU) 01:51:52 CPU %user %nice %system %iowait %steal %idle 01:51:57 1 0.00 0.00 0.00 0.00 0.00 100.00 01:52:02 1 0.20 0.00 0.00 0.00 0.00 99.80 01:52:07 1 0.00 0.00 0.00 0.00 0.00 100.00 Average: 1 0.07 0.00 0.00 0.00 0.00 99.93 [root@centos8 ~]# sar -u -P 5 5 3 Linux 4.18.0-305.3.1.el8.x86_64 (centos8.ittraining.loc) 29/06/21 _x86_64_ (8 CPU) 01:52:16 CPU %user %nice %system %iowait %steal %idle 01:52:21 5 0.00 0.00 0.00 0.00 0.00 100.00 01:52:26 5 0.00 0.00 0.00 0.00 0.00 100.00 01:52:31 5 0.00 0.00 0.00 0.00 0.00 100.00 Average: 5 0.00 0.00 0.00 0.00 0.00 100.00
Memory and Swap Statistics
Use the -r switch to see memory statistics:
[root@centos8 ~]# sar -r 5 3 Linux 4.18.0-305.3.1.el8.x86_64 (centos8.ittraining.loc) 30/06/21 _x86_64_(8 CPU) 07:33:32 kbmemfree kbavail kbmemused %memused kbbuffers kbcached kbcommit %commit kbactive kbinact kbdirty 07:33:37 1647240 2297232 2177792 56.94 3356 827396 5096432 70.94 359072 1486368 0 07:33:42 1647232 2297224 2177800 56.94 3356 827396 5095788 70.93 359072 1486300 0 07:33:47 1647232 2297224 2177800 56.94 3356 827396 5095788 70.93 359072 1486376 0 Average: 1647235 2297227 2177797 56.94 3356 827396 5096003 70.94 359072 1486348 0
Use the -S switch to see swap statistics:
[root@centos8 ~]# sar -S 5 3 Linux 4.18.0-305.3.1.el8.x86_64 (centos8.ittraining.loc) 30/06/21 _x86_64_(8 CPU) 07:31:58 kbswpfree kbswpused %swpused kbswpcad %swpcad 07:32:03 3358716 0 0.00 0 0.00 07:32:08 3358716 0 0.00 0 0.00 07:32:13 3358716 0 0.00 0 0.00 Average: 3358716 0 0.00 0 0.00
I/O Statistics
Use the -b switch:
[root@centos8 ~]# sar -b 5 3 Linux 4.18.0-305.3.1.el8.x86_64 (centos8.ittraining.loc) 29/06/21 _x86_64_ (8 CPU) 09:24:49 tps rtps wtps bread/s bwrtn/s 09:24:54 0.00 0.00 0.00 0.00 0.00 09:24:59 1.20 0.00 1.20 0.00 20.20 09:25:04 0.00 0.00 0.00 0.00 0.00 Average: 0.40 0.00 0.40 0.00 6.73
Disk I/O Statistics
Use the -d switch:
[root@centos8 ~]# sar -d 5 3 Linux 4.18.0-305.3.1.el8.x86_64 (centos8.ittraining.loc) 29/06/21 _x86_64_ (8 CPU) 09:25:45 DEV tps rkB/s wkB/s areq-sz aqu-sz await svctm %util 09:25:50 dev8-0 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 09:25:50 dev8-16 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 09:25:50 dev11-0 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 09:25:50 dev253-0 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 09:25:50 dev253-1 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 09:25:50 DEV tps rkB/s wkB/s areq-sz aqu-sz await svctm %util 09:25:55 dev8-0 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 09:25:55 dev8-16 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 09:25:55 dev11-0 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 09:25:55 dev253-0 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 09:25:55 dev253-1 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 09:25:55 DEV tps rkB/s wkB/s areq-sz aqu-sz await svctm %util 09:26:00 dev8-0 0.60 0.00 0.30 0.50 0.01 13.00 13.00 0.78 09:26:00 dev8-16 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 09:26:00 dev11-0 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 09:26:00 dev253-0 0.60 0.00 0.50 0.83 0.01 12.67 13.00 0.78 09:26:00 dev253-1 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 Average: DEV tps rkB/s wkB/s areq-sz aqu-sz await svctm %util Average: dev8-0 0.20 0.00 0.10 0.50 0.00 13.00 13.00 0.26 Average: dev8-16 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 Average: dev11-0 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 Average: dev253-0 0.20 0.00 0.17 0.83 0.00 12.67 13.00 0.26 Average: dev253-1 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
The DEV column identifies the disks by their major/minor numbers. To see the names of the disks add the, -p switch:
[root@centos8 ~]# sar -p -d 5 3 Linux 4.18.0-305.3.1.el8.x86_64 (centos8.ittraining.loc) 30/06/21 _x86_64_ (8 CPU) 07:48:32 DEV tps rkB/s wkB/s areq-sz aqu-sz await svctm %util 07:48:37 sda 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 07:48:37 sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 07:48:37 sr0 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 07:48:37 cl_centos8-root 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 07:48:37 cl_centos8-swap 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 07:48:37 DEV tps rkB/s wkB/s areq-sz aqu-sz await svctm %util 07:48:42 sda 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 07:48:42 sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 07:48:42 sr0 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 07:48:42 cl_centos8-root 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 07:48:42 cl_centos8-swap 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 07:48:42 DEV tps rkB/s wkB/s areq-sz aqu-sz await svctm %util 07:48:47 sda 0.40 0.00 0.40 1.00 0.02 56.00 56.50 2.26 07:48:47 sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 07:48:47 sr0 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 07:48:47 cl_centos8-root 0.40 0.00 0.80 2.00 0.02 56.00 56.50 2.26 07:48:47 cl_centos8-swap 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 Average: DEV tps rkB/s wkB/s areq-sz aqu-sz await svctm %util Average: sda 0.13 0.00 0.13 1.00 0.01 56.00 56.50 0.75 Average: sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 Average: sr0 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 Average: cl_centos8-root 0.13 0.00 0.27 2.00 0.01 56.00 56.50 0.75 Average: cl_centos8-swap 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
The command line switches of this command are:
[root@centos8 ~]# sar --help
Usage: sar [ options ] [ <interval> [ <count> ] ]
Main options and reports (report name between square brackets):
-B Paging statistics [A_PAGE]
-b I/O and transfer rate statistics [A_IO]
-d Block devices statistics [A_DISK]
-F [ MOUNT ]
Filesystems statistics [A_FS]
-H Hugepages utilization statistics [A_HUGE]
-I { <int_list> | SUM | ALL }
Interrupts statistics [A_IRQ]
-m { <keyword> [,...] | ALL }
Power management statistics [A_PWR_...]
Keywords are:
CPU CPU instantaneous clock frequency
FAN Fans speed
FREQ CPU average clock frequency
IN Voltage inputs
TEMP Devices temperature
USB USB devices plugged into the system
-n { <keyword> [,...] | ALL }
Network statistics [A_NET_...]
Keywords are:
DEV Network interfaces
EDEV Network interfaces (errors)
NFS NFS client
NFSD NFS server
SOCK Sockets (v4)
IP IP traffic (v4)
EIP IP traffic (v4) (errors)
ICMP ICMP traffic (v4)
EICMP ICMP traffic (v4) (errors)
TCP TCP traffic (v4)
ETCP TCP traffic (v4) (errors)
UDP UDP traffic (v4)
SOCK6 Sockets (v6)
IP6 IP traffic (v6)
EIP6 IP traffic (v6) (errors)
ICMP6 ICMP traffic (v6)
EICMP6 ICMP traffic (v6) (errors)
UDP6 UDP traffic (v6)
FC Fibre channel HBAs
SOFT Software-based network processing
-q Queue length and load average statistics [A_QUEUE]
-r [ ALL ]
Memory utilization statistics [A_MEMORY]
-S Swap space utilization statistics [A_MEMORY]
-u [ ALL ]
CPU utilization statistics [A_CPU]
-v Kernel tables statistics [A_KTABLES]
-W Swapping statistics [A_SWAP]
-w Task creation and system switching statistics [A_PCSW]
-y TTY devices statistics [A_SERIAL]
LAB #2 - Viewing Open Files and Running Processes
2.1 - Presentation
A process is a binary file that is loaded into memory and executed. When the file is loaded it needs the operating system to supply it with information such that it can execute correctly. Collectively, this information is refered to as the process environment and includes:
- A unique process ID (PID),
- The Parent PID (PPID),
- A User ID (UID),
- A Groupe ID (GID),
- Processing time,
- The process priority,
- The current working directory,
- A list of open files.
This information is stored in /proc:
[root@centos8 ~]# cd /proc; ls -d [0-9]* 1 1113 1158 1464 1590 1601 163 166 171 20 24 28 303 31 34 38 41 45 461 465 469 473 478 481 487 497 507 52 528 531 599 649 69 765 793 800 827 840 853 875 902 10 1125 12 1465 1594 162 1634 167 18 21 25 29 305 310 35 39 42 457 462 466 47 475 479 482 489 498 508 521 529 532 6 655 7 766 795 801 828 841 854 880 905 11 1126 13 15 16 1627 164 168 19 22 26 3 306 311 36 4 43 459 463 467 470 476 48 484 49 5 51 526 53 533 60 662 741 779 799 823 834 847 866 884 910 1111 1135 14 1585 1600 1628 165 17 2 23 27 30 309 32 37 40 44 46 464 468 472 477 480 485 496 50 518 527 530 534 632 666 751 791 8 825 837 850 874 9
Each directory has as a name the PID of the process it refers to. Looking into one of the directories, you can see the process environment information:
[root@centos8 proc]# cd 1 ; ls -l total 0 dr-xr-xr-x. 2 root root 0 Jun 3 09:01 attr -rw-r--r--. 1 root root 0 Jun 3 09:02 autogroup -r--------. 1 root root 0 Jun 3 09:02 auxv -r--r--r--. 1 root root 0 Jun 3 09:01 cgroup --w-------. 1 root root 0 Jun 3 09:02 clear_refs -r--r--r--. 1 root root 0 Jun 3 09:01 cmdline -rw-r--r--. 1 root root 0 Jun 3 09:01 comm -rw-r--r--. 1 root root 0 Jun 3 09:02 coredump_filter -r--r--r--. 1 root root 0 Jun 3 09:02 cpu_resctrl_groups -r--r--r--. 1 root root 0 Jun 3 09:02 cpuset lrwxrwxrwx. 1 root root 0 Jun 3 09:02 cwd -> / -r--------. 1 root root 0 Jun 3 09:01 environ lrwxrwxrwx. 1 root root 0 Jun 3 09:01 exe -> /usr/lib/systemd/systemd dr-x------. 2 root root 0 Jun 3 09:01 fd dr-x------. 2 root root 0 Jun 3 09:01 fdinfo -rw-r--r--. 1 root root 0 Jun 3 09:01 gid_map -r--------. 1 root root 0 Jun 3 09:02 io -r--r--r--. 1 root root 0 Jun 3 09:02 limits -rw-r--r--. 1 root root 0 Jun 3 09:01 loginuid dr-x------. 2 root root 0 Jun 3 09:02 map_files -r--r--r--. 1 root root 0 Jun 3 09:01 maps -rw-------. 1 root root 0 Jun 3 09:02 mem -r--r--r--. 1 root root 0 Jun 3 09:01 mountinfo -r--r--r--. 1 root root 0 Jun 3 09:01 mounts -r--------. 1 root root 0 Jun 3 09:02 mountstats dr-xr-xr-x. 6 root root 0 Jun 3 09:01 net dr-x--x--x. 2 root root 0 Jun 3 09:02 ns -r--r--r--. 1 root root 0 Jun 3 09:02 numa_maps -rw-r--r--. 1 root root 0 Jun 3 09:02 oom_adj -r--r--r--. 1 root root 0 Jun 3 09:02 oom_score -rw-r--r--. 1 root root 0 Jun 3 09:02 oom_score_adj -r--------. 1 root root 0 Jun 3 09:02 pagemap -r--------. 1 root root 0 Jun 3 09:02 patch_state -r--------. 1 root root 0 Jun 3 09:02 personality -rw-r--r--. 1 root root 0 Jun 3 09:02 projid_map lrwxrwxrwx. 1 root root 0 Jun 3 09:01 root -> / -rw-r--r--. 1 root root 0 Jun 3 09:01 sched -r--r--r--. 1 root root 0 Jun 3 09:02 schedstat -r--r--r--. 1 root root 0 Jun 3 09:01 sessionid -rw-r--r--. 1 root root 0 Jun 3 09:01 setgroups -r--r--r--. 1 root root 0 Jun 3 09:02 smaps -r--r--r--. 1 root root 0 Jun 3 09:02 smaps_rollup -r--------. 1 root root 0 Jun 3 09:02 stack -r--r--r--. 1 root root 0 Jun 3 09:01 stat -r--r--r--. 1 root root 0 Jun 3 09:02 statm -r--r--r--. 1 root root 0 Jun 3 09:01 status -r--------. 1 root root 0 Jun 3 09:02 syscall dr-xr-xr-x. 3 root root 0 Jun 3 09:01 task -r--r--r--. 1 root root 0 Jun 3 09:02 timers -rw-rw-rw-. 1 root root 0 Jun 3 09:02 timerslack_ns -rw-r--r--. 1 root root 0 Jun 3 09:01 uid_map -r--r--r--. 1 root root 0 Jun 3 09:02 wchan
Important - Note that the content of the files is of little or of no direct use to a System Administrator.
2.2 - Process Types
There are three types of processes:
- interactive - processes generated by typing a command in a terminal,
- batch - processes generated by the system itself,
- daemon - processes that do not have a parent terminal.
A process can be in any one of 9 process states:
- user mode - the process is executing in user mode,
- kernel mode- the process is executing in kernel mode,
- sleeping - the process is sleeping,
- swap - the process is sleeping in swap,
- new - the process is new,
- waiting – the process is waiting for a ressource other than the processor,
- runnable – the process has all the ressources it requires except the processor itself,
- elected – the process is in the processor,
- zombie – the process has terminated and is waiting to be killed by the system.
2.3 - Process Commands
The ps Command
The output from this command shows the processes attached to the current terminal:
[root@centos8 1]# cd ~
[root@centos8 ~]# ps
PID TTY TIME CMD
1627 pts/0 00:00:00 su
1634 pts/0 00:00:00 bash
1690 pts/0 00:00:00 ps
You can get more details by using the -l switch:
[root@centos8 ~]# ps -l F S UID PID PPID C PRI NI ADDR SZ WCHAN TTY TIME CMD 4 S 0 1627 1601 0 80 0 - 48967 - pts/0 00:00:00 su 4 S 0 1634 1627 0 80 0 - 6911 - pts/0 00:00:00 bash 0 R 0 1698 1634 0 80 0 - 11360 - pts/0 00:00:00 ps
This output shows some usefull information:
| F | Process flag. The value of 4 means the process is using root privileges. |
|---|---|
| S | The process state - S (sleeping), R (In run queue), Z (zombie), N (low priority), D (uninterruptible sleep), T (Traced) |
| UID | User ID of the user who has stated the process |
| PID | Process ID |
| PPID | Parent PID |
| C | Prority factor |
| PRI | Process priority |
| NI | Process nice value |
| ADDR | Memory address |
| SZ | Virtual memory usage |
| WCHAN | Name of the kernel function in which the process is asleep |
| TTY | Name of the terminal in which the process was started |
| TIME | Processing time |
| CMD | The command that generated the process |
To see the process table, use the l and x switches:
[root@centos8 ~]# ps lx | more F UID PID PPID PRI NI VSZ RSS WCHAN STAT TTY TIME COMMAND 4 0 1 0 20 0 245540 14252 do_epo Ss ? 0:01 /usr/lib/systemd/systemd --switched-root --system --deserialize 17 1 0 2 0 20 0 0 0 - S ? 0:00 [kthreadd] 1 0 3 2 0 -20 0 0 - I< ? 0:00 [rcu_gp] 1 0 4 2 0 -20 0 0 - I< ? 0:00 [rcu_par_gp] 1 0 5 2 20 0 0 0 - I ? 0:00 [kworker/0:0-events] 1 0 6 2 0 -20 0 0 - I< ? 0:00 [kworker/0:0H-kblockd] 1 0 7 2 20 0 0 0 - I ? 0:00 [kworker/0:1-ata_sff] 1 0 8 2 20 0 0 0 - I ? 0:00 [kworker/u8:0-events_unbound] 1 0 9 2 0 -20 0 0 - I< ? 0:00 [mm_percpu_wq] 1 0 10 2 20 0 0 0 - S ? 0:00 [ksoftirqd/0] 1 0 11 2 20 0 0 0 - I ? 0:00 [rcu_sched] 1 0 12 2 -100 - 0 0 - S ? 0:00 [migration/0] 5 0 13 2 -100 - 0 0 - S ? 0:00 [watchdog/0] 1 0 14 2 20 0 0 0 - S ? 0:00 [cpuhp/0] 1 0 15 2 20 0 0 0 - S ? 0:00 [cpuhp/1] 5 0 16 2 -100 - 0 0 - S ? 0:00 [watchdog/1] 1 0 17 2 -100 - 0 0 - S ? 0:00 [migration/1] 1 0 18 2 20 0 0 0 - S ? 0:00 [ksoftirqd/1] 1 0 19 2 20 0 0 0 - I ? 0:00 [kworker/1:0-memcg_kmem_cache] 1 0 20 2 0 -20 0 0 - I< ? 0:00 [kworker/1:0H] 1 0 21 2 20 0 0 0 - S ? 0:00 [cpuhp/2] 5 0 22 2 -100 - 0 0 - S ? 0:00 [watchdog/2] --More--
On note dans cette sortie certaines informations supplémentaires :
| VSZ | The same thing as SZ in the previous example |
|---|---|
| RSS | Memory in KB used by the process |
| STAT | The same thing as S in the previous example |
Using the a, u and x switches you obtain the following output:
[root@centos8 ~]# ps aux | more USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND root 1 0.6 0.3 245540 14252 ? Ss 09:01 0:01 /usr/lib/systemd/systemd --switched-root --system --deserialize 17 root 2 0.0 0.0 0 0 ? S 09:01 0:00 [kthreadd] root 3 0.0 0.0 0 0 ? I< 09:01 0:00 [rcu_gp] root 4 0.0 0.0 0 0 ? I< 09:01 0:00 [rcu_par_gp] root 5 0.0 0.0 0 0 ? I 09:01 0:00 [kworker/0:0-events] root 6 0.0 0.0 0 0 ? I< 09:01 0:00 [kworker/0:0H-kblockd] root 7 0.1 0.0 0 0 ? I 09:01 0:00 [kworker/0:1-ata_sff] root 8 0.0 0.0 0 0 ? I 09:01 0:00 [kworker/u8:0-events_unbound] root 9 0.0 0.0 0 0 ? I< 09:01 0:00 [mm_percpu_wq] root 10 0.0 0.0 0 0 ? S 09:01 0:00 [ksoftirqd/0] root 11 0.0 0.0 0 0 ? I 09:01 0:00 [rcu_sched] root 12 0.0 0.0 0 0 ? S 09:01 0:00 [migration/0] root 13 0.0 0.0 0 0 ? S 09:01 0:00 [watchdog/0] root 14 0.0 0.0 0 0 ? S 09:01 0:00 [cpuhp/0] root 15 0.0 0.0 0 0 ? S 09:01 0:00 [cpuhp/1] root 16 0.0 0.0 0 0 ? S 09:01 0:00 [watchdog/1] root 17 0.0 0.0 0 0 ? S 09:01 0:00 [migration/1] root 18 0.0 0.0 0 0 ? S 09:01 0:00 [ksoftirqd/1] root 19 0.0 0.0 0 0 ? I 09:01 0:00 [kworker/1:0-memcg_kmem_cache] root 20 0.0 0.0 0 0 ? I< 09:01 0:00 [kworker/1:0H] root 21 0.0 0.0 0 0 ? S 09:01 0:00 [cpuhp/2] root 22 0.0 0.0 0 0 ? S 09:01 0:00 [watchdog/2] --More--
This output provides further usefull information:
| USER | The user associated with the process |
|---|---|
| %CPU | % of the processor ressources used by the process |
| %MEM | % of the memory ressources used by the process |
Command Line Switches
The command line switches for the ps command are :
[root@centos8 ~]# ps --help all
Usage:
ps [options]
Basic options:
-A, -e all processes
-a all with tty, except session leaders
a all with tty, including other users
-d all except session leaders
-N, --deselect negate selection
r only running processes
T all processes on this terminal
x processes without controlling ttys
Selection by list:
-C <command> command name
-G, --Group <GID> real group id or name
-g, --group <group> session or effective group name
-p, p, --pid <PID> process id
--ppid <PID> parent process id
-q, q, --quick-pid <PID>
process id (quick mode)
-s, --sid <session> session id
-t, t, --tty <tty> terminal
-u, U, --user <UID> effective user id or name
-U, --User <UID> real user id or name
The selection options take as their argument either:
a comma-separated list e.g. '-u root,nobody' or
a blank-separated list e.g. '-p 123 4567'
Output formats:
-F extra full
-f full-format, including command lines
f, --forest ascii art process tree
-H show process hierarchy
-j jobs format
j BSD job control format
-l long format
l BSD long format
-M, Z add security data (for SELinux)
-O <format> preloaded with default columns
O <format> as -O, with BSD personality
-o, o, --format <format>
user-defined format
s signal format
u user-oriented format
v virtual memory format
X register format
-y do not show flags, show rss vs. addr (used with -l)
--context display security context (for SELinux)
--headers repeat header lines, one per page
--no-headers do not print header at all
--cols, --columns, --width <num>
set screen width
--rows, --lines <num>
set screen height
Show threads:
H as if they were processes
-L possibly with LWP and NLWP columns
-m, m after processes
-T possibly with SPID column
Miscellaneous options:
-c show scheduling class with -l option
c show true command name
e show the environment after command
k, --sort specify sort order as: [+|-]key[,[+|-]key[,...]]
L show format specifiers
n display numeric uid and wchan
S, --cumulative include some dead child process data
-y do not show flags, show rss (only with -l)
-V, V, --version display version information and exit
-w, w unlimited output width
--help <simple|list|output|threads|misc|all>
display help and exit
For more details see ps(1).
The pstree Command
This command shows the processes as a tree:
[root@centos8 ~]# pstree
systemd─┬─NetworkManager───2*[{NetworkManager}]
├─agetty
├─atd
├─auditd─┬─sedispatch
│ └─2*[{auditd}]
├─chronyd
├─crond
├─dbus-daemon───{dbus-daemon}
├─dnsmasq───dnsmasq
├─firewalld───{firewalld}
├─gssproxy───5*[{gssproxy}]
├─irqbalance───{irqbalance}
├─ksmtuned───sleep
├─lsmd
├─mcelog
├─polkitd───7*[{polkitd}]
├─rngd───4*[{rngd}]
├─rpcbind
├─rsyslogd───2*[{rsyslogd}]
├─smartd
├─sshd───sshd───sshd───bash───su───bash───pstree
├─sssd─┬─sssd_be
│ └─sssd_nss
├─systemd───(sd-pam)
├─systemd-journal
├─systemd-logind
├─systemd-machine
├─systemd-udevd
└─tuned───3*[{tuned}]
Command Line Switches
The command line switches for the pstree command are :
[root@centos8 ~]# pstree --help
pstree: unrecognized option '--help'
Usage: pstree [-acglpsStuZ] [ -h | -H PID ] [ -n | -N type ]
[ -A | -G | -U ] [ PID | USER ]
pstree -V
Display a tree of processes.
-a, --arguments show command line arguments
-A, --ascii use ASCII line drawing characters
-c, --compact don't compact identical subtrees
-h, --highlight-all highlight current process and its ancestors
-H PID,
--highlight-pid=PID highlight this process and its ancestors
-g, --show-pgids show process group ids; implies -c
-G, --vt100 use VT100 line drawing characters
-l, --long don't truncate long lines
-n, --numeric-sort sort output by PID
-N type,
--ns-sort=type sort by namespace type (cgroup, ipc, mnt, net, pid,
user, uts)
-p, --show-pids show PIDs; implies -c
-s, --show-parents show parents of the selected process
-S, --ns-changes show namespace transitions
-t, --thread-names show full thread names
-T, --hide-threads hide threads, show only processes
-u, --uid-changes show uid transitions
-U, --unicode use UTF-8 (Unicode) line drawing characters
-V, --version display version information
-Z, --security-context
show SELinux security contexts
PID start at this PID; default is 1 (init)
USER show only trees rooted at processes of this user
The top Command
top shows a continuous real time list of running processes:
[root@centos8 ~]# top
top - 09:10:02 up 8 min, 1 user, load average: 0.05, 0.09, 0.08
Tasks: 144 total, 1 running, 143 sleeping, 0 stopped, 0 zombie
%Cpu(s): 0.1 us, 0.2 sy, 0.0 ni, 99.6 id, 0.0 wa, 0.1 hi, 0.1 si, 0.0 st
MiB Mem : 3737.7 total, 3249.1 free, 231.4 used, 257.2 buff/cache
MiB Swap: 2000.0 total, 2000.0 free, 0.0 used. 3280.0 avail Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
1 root 20 0 245540 14260 9308 S 0.2 0.4 0:01.64 systemd
827 polkitd 20 0 1764748 25376 17348 S 0.2 0.7 0:00.12 polkitd
834 dbus 20 0 64684 5476 4480 S 0.2 0.1 0:00.20 dbus-daemon
1789 root 20 0 65420 4464 3764 R 0.2 0.1 0:00.03 top
2 root 20 0 0 0 0 S 0.0 0.0 0:00.00 kthreadd
3 root 0 -20 0 0 0 I 0.0 0.0 0:00.00 rcu_gp
4 root 0 -20 0 0 0 I 0.0 0.0 0:00.00 rcu_par_gp
6 root 0 -20 0 0 0 I 0.0 0.0 0:00.00 kworker/0:0H-kblockd
7 root 20 0 0 0 0 I 0.0 0.0 0:00.47 kworker/0:1-mm_percpu_wq
9 root 0 -20 0 0 0 I 0.0 0.0 0:00.00 mm_percpu_wq
10 root 20 0 0 0 0 S 0.0 0.0 0:00.00 ksoftirqd/0
11 root 20 0 0 0 0 I 0.0 0.0 0:00.09 rcu_sched
12 root rt 0 0 0 0 S 0.0 0.0 0:00.00 migration/0
13 root rt 0 0 0 0 S 0.0 0.0 0:00.00 watchdog/0
14 root 20 0 0 0 0 S 0.0 0.0 0:00.00 cpuhp/0
15 root 20 0 0 0 0 S 0.0 0.0 0:00.00 cpuhp/1
16 root rt 0 0 0 0 S 0.0 0.0 0:00.00 watchdog/1
17 root rt 0 0 0 0 S 0.0 0.0 0:00.00 migration/1
18 root 20 0 0 0 0 S 0.0 0.0 0:00.00 ksoftirqd/1
20 root 0 -20 0 0 0 I 0.0 0.0 0:00.00 kworker/1:0H
21 root 20 0 0 0 0 S 0.0 0.0 0:00.00 cpuhp/2
22 root rt 0 0 0 0 S 0.0 0.0 0:00.00 watchdog/2
23 root rt 0 0 0 0 S 0.0 0.0 0:00.00 migration/2
24 root 20 0 0 0 0 S 0.0 0.0 0:00.00 ksoftirqd/2
26 root 0 -20 0 0 0 I 0.0 0.0 0:00.00 kworker/2:0H-kblockd
27 root 20 0 0 0 0 S 0.0 0.0 0:00.00 cpuhp/3
28 root rt 0 0 0 0 S 0.0 0.0 0:00.00 watchdog/3
29 root rt 0 0 0 0 S 0.0 0.0 0:00.00 migration/3
30 root 20 0 0 0 0 S 0.0 0.0 0:00.00 ksoftirqd/3
32 root 0 -20 0 0 0 I 0.0 0.0 0:00.00 kworker/3:0H-kblockd
35 root 20 0 0 0 0 S 0.0 0.0 0:00.00 kdevtmpfs
36 root 0 -20 0 0 0 I 0.0 0.0 0:00.00 netns
37 root 20 0 0 0 0 S 0.0 0.0 0:00.00 kauditd
38 root 20 0 0 0 0 S 0.0 0.0 0:00.00 khungtaskd
39 root 20 0 0 0 0 S 0.0 0.0 0:00.00 oom_reaper
40 root 0 -20 0 0 0 I 0.0 0.0 0:00.00 writeback
41 root 20 0 0 0 0 S 0.0 0.0 0:00.00 kcompactd0
42 root 25 5 0 0 0 S 0.0 0.0 0:00.00 ksmd
43 root 39 19 0 0 0 S 0.0 0.0 0:00.00 khugepaged
44 root 0 -20 0 0 0 I 0.0 0.0 0:00.00 crypto
45 root 0 -20 0 0 0 I 0.0 0.0 0:00.00 kintegrityd
46 root 0 -20 0 0 0 I 0.0 0.0 0:00.00 kblockd
47 root 0 -20 0 0 0 I 0.0 0.0 0:00.00 blkcg_punt_bio
48 root 0 -20 0 0 0 I 0.0 0.0 0:00.00 tpm_dev_wq
49 root 0 -20 0 0 0 I 0.0 0.0 0:00.00 md
50 root 0 -20 0 0 0 I 0.0 0.0 0:00.00 edac-poller
51 root rt 0 0 0 0 S 0.0 0.0 0:00.00 watchdogd
53 root 0 -20 0 0 0 I 0.0 0.0 0:00.00 pm_wq
69 root 20 0 0 0 0 S 0.0 0.0 0:00.00 kswapd0
162 root 0 -20 0 0 0 I 0.0 0.0 0:00.00 kthrotld
163 root 0 -20 0 0 0 I 0.0 0.0 0:00.00 acpi_thermal_pm
...
To see top's help, use the h key:
Help for Interactive Commands - procps-ng 3.3.15
Window 1:Def: Cumulative mode Off. System: Delay 3.0 secs; Secure mode Off.
Z,B,E,e Global: 'Z' colors; 'B' bold; 'E'/'e' summary/task memory scale
l,t,m Toggle Summary: 'l' load avg; 't' task/cpu stats; 'm' memory info
0,1,2,3,I Toggle: '0' zeros; '1/2/3' cpus or numa node views; 'I' Irix mode
f,F,X Fields: 'f'/'F' add/remove/order/sort; 'X' increase fixed-width
L,&,<,> . Locate: 'L'/'&' find/again; Move sort column: '<'/'>' left/right
R,H,V,J . Toggle: 'R' Sort; 'H' Threads; 'V' Forest view; 'J' Num justify
c,i,S,j . Toggle: 'c' Cmd name/line; 'i' Idle; 'S' Time; 'j' Str justify
x,y . Toggle highlights: 'x' sort field; 'y' running tasks
z,b . Toggle: 'z' color/mono; 'b' bold/reverse (only if 'x' or 'y')
u,U,o,O . Filter by: 'u'/'U' effective/any user; 'o'/'O' other criteria
n,#,^O . Set: 'n'/'#' max tasks displayed; Show: Ctrl+'O' other filter(s)
C,... . Toggle scroll coordinates msg for: up,down,left,right,home,end
k,r Manipulate tasks: 'k' kill; 'r' renice
d or s Set update interval
W,Y Write configuration file 'W'; Inspect other output 'Y'
q Quit
( commands shown with '.' require a visible task display window )
Press 'h' or '?' for help with Windows,
Type 'q' or <Esc> to continue
Important - To return to the previous display,use q or escape.
When launched top's refresh rate is 3 seconds. To change this to 1 second, use the s key:
[root@centos8 ~]# top ... top - 09:11:24 up 10 min, 1 user, load average: 0.01, 0.07, 0.07 Tasks: 144 total, 1 running, 143 sleeping, 0 stopped, 0 zombie %Cpu(s): 0.0 us, 0.1 sy, 0.0 ni, 99.7 id, 0.0 wa, 0.2 hi, 0.0 si, 0.0 st MiB Mem : 3737.7 total, 3248.8 free, 231.7 used, 257.2 buff/cache MiB Swap: 2000.0 total, 2000.0 free, 0.0 used. 3279.7 avail Mem Change delay from 3.0 to 1 ...
To sort the list by memory usage, use the M key:
[root@centos8 ~]# top
...
top - 09:12:07 up 10 min, 1 user, load average: 0.00, 0.05, 0.06
Tasks: 146 total, 1 running, 145 sleeping, 0 stopped, 0 zombie
%Cpu(s): 0.1 us, 0.4 sy, 0.7 ni, 97.6 id, 0.2 wa, 0.9 hi, 0.1 si, 0.0 st
MiB Mem : 3737.7 total, 3192.8 free, 233.5 used, 311.4 buff/cache
MiB Swap: 2000.0 total, 2000.0 free, 0.0 used. 3277.3 avail Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
874 root 20 0 290088 40696 17960 S 0.0 1.1 0:00.54 firewalld
875 root 20 0 224868 40472 38904 S 0.0 1.1 0:00.08 sssd_nss
902 root 20 0 426324 33684 16668 S 0.0 0.9 0:00.25 tuned
827 polkitd 20 0 1764748 25376 17348 S 0.0 0.7 0:00.12 polkitd
884 root 20 0 391768 18828 16220 S 0.1 0.5 0:00.11 NetworkManager
866 root 20 0 221424 15016 12284 S 0.0 0.4 0:00.09 sssd_be
840 root 20 0 214800 14292 12260 S 0.0 0.4 0:00.03 sssd
1 root 20 0 245540 14260 9308 S 0.1 0.4 0:01.66 systemd
1627 root 20 0 195868 11896 10336 S 0.0 0.3 0:00.02 su
666 root 20 0 110416 11336 8292 S 0.0 0.3 0:00.30 systemd-udevd
1585 root 20 0 163700 10636 9340 S 0.0 0.3 0:00.01 sshd
1590 trainee 20 0 94128 9872 8240 S 0.0 0.3 0:00.05 systemd
880 root 20 0 96712 9780 7568 S 0.0 0.3 0:00.25 systemd-logind
632 root 20 0 94036 9064 8016 S 0.0 0.2 0:00.28 systemd-journal
837 root 20 0 83656 7004 6108 S 0.0 0.2 0:00.19 systemd-machine
905 root 20 0 92288 6996 6100 S 0.0 0.2 0:00.01 sshd
847 rngd 20 0 381308 6500 5656 S 0.0 0.2 0:15.38 rngd
828 root 20 0 50640 5900 4600 S 0.0 0.2 0:00.03 smartd
1600 trainee 20 0 163700 5584 4284 S 0.0 0.1 0:00.04 sshd
1601 trainee 20 0 28312 5564 3664 S 0.0 0.1 0:00.02 bash
834 dbus 20 0 64684 5476 4480 S 0.1 0.1 0:00.22 dbus-daemon
1634 root 20 0 27644 5428 3408 S 0.0 0.1 0:00.06 bash
791 rpc 20 0 67140 5396 4672 S 0.0 0.1 0:00.01 rpcbind
1594 trainee 20 0 175840 5228 40 S 0.0 0.1 0:00.00 (sd-pam)
1113 root 20 0 209436 5036 3516 S 0.0 0.1 0:00.09 rsyslogd
823 root 20 0 124908 4644 4080 S 0.0 0.1 0:00.03 irqbalance
1803 root 20 0 65420 4508 3808 R 0.0 0.1 0:00.03 top
...
To see the zombie and waiting processes, use the i key:
[root@centos8 ~]# top
...
top - 09:13:01 up 11 min, 1 user, load average: 0.00, 0.04, 0.06
Tasks: 145 total, 2 running, 143 sleeping, 0 stopped, 0 zombie
%Cpu(s): 0.1 us, 0.1 sy, 0.0 ni, 99.8 id, 0.0 wa, 0.1 hi, 0.0 si, 0.0 st
MiB Mem : 3737.7 total, 3192.5 free, 233.8 used, 311.4 buff/cache
MiB Swap: 2000.0 total, 2000.0 free, 0.0 used. 3276.9 avail Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
827 polkitd 20 0 1764748 25376 17348 S 0.1 0.7 0:00.13 polkitd
1 root 20 0 245540 14260 9308 S 0.1 0.4 0:01.67 systemd
823 root 20 0 124908 4644 4080 S 0.1 0.1 0:00.04 irqbalance
1803 root 20 0 65420 4508 3808 R 0.2 0.1 0:00.08 top
To leave top, use the q key.
Command Line Switches
The command line switches for the top command are :
[root@centos8 ~]# top --help top: inappropriate '-help' Usage: top -hv | -bcEHiOSs1 -d secs -n max -u|U user -p pid(s) -o field -w [cols]
The lsof Command
This command's output shows information about open files:
[trainee@centos8 tmp]$ su -
Password: fenestros
[root@centos8 ~]# lsof | more
COMMAND PID TID TASKCMD USER FD TYPE DEVICE
SIZE/OFF NODE NAME
systemd 1 root cwd DIR 8,3
224 128 /
systemd 1 root rtd DIR 8,3
224 128 /
systemd 1 root txt REG 8,3
1609248 8811152 /usr/lib/systemd/systemd
systemd 1 root mem REG 8,3
2191808 126296 /usr/lib64/libm-2.28.so
systemd 1 root mem REG 8,3
628744 126019 /usr/lib64/libudev.so.1.6.11
systemd 1 root mem REG 8,3
969832 151279 /usr/lib64/libsepol.so.1
systemd 1 root mem REG 8,3
1805368 179753 /usr/lib64/libunistring.so.2.1.0
systemd 1 root mem REG 8,3
303944 131440 /usr/lib64/libpcap.so.1.9.1
systemd 1 root mem REG 8,3
145984 163438 /usr/lib64/libgpg-error.so.0.24.2
systemd 1 root mem REG 8,3
71528 194381 /usr/lib64/libjson-c.so.4.0.0
systemd 1 root mem REG 8,3
--More--
Command Line Switches
To do : Use the –help option of the lsof command to view the command line switches.
LAB # 3 - Measuring Network Usage and System Availability
3.1 - The netstat Command
netstat -i
To see networking statistics, use the netstat command:
[root@centos8 ~]# netstat -i Kernel Interface table Iface MTU RX-OK RX-ERR RX-DRP RX-OVR TX-OK TX-ERR TX-DRP TX-OVR Flg ens18 1500 476056 0 0 0 363562 0 0 0 BMRU lo 65536 10936 0 0 0 10936 0 0 0 LRU virbr0 1500 0 0 0 0 0 0 0 0 BMU
Command Line Switches
The command line switches of this command are:
[root@centos8 ~]# netstat --help
usage: netstat [-vWeenNcCF] [<Af>] -r netstat {-V|--version|-h|--help}
netstat [-vWnNcaeol] [<Socket> ...]
netstat { [-vWeenNac] -I[<Iface>] | [-veenNac] -i | [-cnNe] -M | -s [-6tuw] } [delay]
-r, --route display routing table
-I, --interfaces=<Iface> display interface table for <Iface>
-i, --interfaces display interface table
-g, --groups display multicast group memberships
-s, --statistics display networking statistics (like SNMP)
-M, --masquerade display masqueraded connections
-v, --verbose be verbose
-W, --wide don't truncate IP addresses
-n, --numeric don't resolve names
--numeric-hosts don't resolve host names
--numeric-ports don't resolve port names
--numeric-users don't resolve user names
-N, --symbolic resolve hardware names
-e, --extend display other/more information
-p, --programs display PID/Program name for sockets
-o, --timers display timers
-c, --continuous continuous listing
-l, --listening display listening server sockets
-a, --all display all sockets (default: connected)
-F, --fib display Forwarding Information Base (default)
-C, --cache display routing cache instead of FIB
-Z, --context display SELinux security context for sockets
<Socket>={-t|--tcp} {-u|--udp} {-U|--udplite} {-S|--sctp} {-w|--raw}
{-x|--unix} --ax25 --ipx --netrom
<AF>=Use '-6|-4' or '-A <af>' or '--<af>'; default: inet
List of possible address families (which support routing):
inet (DARPA Internet) inet6 (IPv6) ax25 (AMPR AX.25)
netrom (AMPR NET/ROM) ipx (Novell IPX) ddp (Appletalk DDP)
x25 (CCITT X.25)
3.2 - The w Command
This command outputs the same data as the uptime command on the first line and then complements this information with the details of each user connected to the system, including what each user is currently doing. This is the replacement under Linux for the Unix command whodo:
[root@centos8 ~]# w 03:55:16 up 15:50, 1 user, load average: 0.01, 0.00, 0.00 USER TTY FROM LOGIN@ IDLE JCPU PCPU WHAT trainee pts/0 10.0.2.2 02:21 4.00s 0.16s 0.01s sshd: trainee [priv]
The JCPU time is the time used by all processes attached to the tty. It does not include past background jobs, but does include currently running background jobs.
The PCPU time is the time used by the current process, named in the what field.
Command Line Switches
To do : Use the –help option of the w command to view the command line switches.
3.3 - The uptime Command
This command prints to standard output the current time, the length of time the system has been up, the number of users on the system and the average number of jobs in the run queue over the last 1, 5 and 15 minutes:
[root@centos8 ~]# uptime 03:55:13 up 15:50, 1 user, load average: 0.01, 0.00, 0.00
Command Line Switches
The switches associated with this command are:
To do : Use the –help option of the uptime command to view the command line switches.
