Measuring and monitoring system performances through sysstat

>Being an administrator of a corporation to manage production box is an daunting task.One has to be very much aware what is going on into the box/servers by looking into it through some tools.One important package I am talking about is called “sysstats” ,which has so many important tool to disclose all the information needed by an administrator.

I do not issue any guarantee that this will work for you.

So this article I am using Arch Linux . As it doesn’t come with base installation so I have to get it(sysstat) separately.

bhaskar@bhaskar-laptop_07:05:26_Tue Aug 31:~> sudo pacman -S sysstat
warning: sysstat-9.0.6.1-1 is up to date — reinstalling
resolving dependencies…
looking for inter-conflicts…

Targets (1): sysstat-9.0.6.1-1

Total Download Size: 0.00 MB
Total Installed Size: 1.14 MB

Proceed with installation? [Y/n]

here I said N or no ,because I have already installed it long time back.You see that below:

bhaskar@bhaskar-laptop_07:08:31_Tue Aug 31:~> sudo pacman -Qi sysstat
Name : sysstat
Version : 9.0.6.1-1
URL : http://pagesperso-orange.fr/sebastien.godard/
Licenses : GPL
Groups : None
Provides : None
Depends On : glibc
Optional Deps : tk: to use isag
gnuplot: to use isag
Required By : None
Conflicts With : None
Replaces : None
Installed Size : 1168.00 K
Packager : Sergej Pupykin
Architecture : i686
Build Date : Mon 01 Mar 2010 03:51:14 AM IST
Install Date : Tue 02 Mar 2010 10:10:31 PM IST
Install Reason : Explicitly installed
Install Script : No
Description : A collection of performance monitoring tools

Here are the files installed by the package in the system..

bhaskar@bhaskar-laptop_07:10:35_Tue Aug 31:~> sudo pacman -Ql sysstat
sysstat /etc/
sysstat /etc/cron.daily/
sysstat /etc/cron.daily/sysstat
sysstat /etc/cron.hourly/
sysstat /etc/cron.hourly/sysstat
sysstat /etc/rc.d/
sysstat /etc/rc.d/sysstat
sysstat /etc/sysconfig/
sysstat /etc/sysconfig/sysstat
sysstat /etc/sysconfig/sysstat.ioconf
sysstat /etc/sysstat/
sysstat /etc/sysstat/sysstat
sysstat /usr/
sysstat /usr/bin/
sysstat /usr/bin/iostat
sysstat /usr/bin/isag
sysstat /usr/bin/mpstat
sysstat /usr/bin/pidstat
sysstat /usr/bin/sadf
sysstat /usr/bin/sar
sysstat /usr/lib/
sysstat /usr/lib/sa/
sysstat /usr/lib/sa/sa1
sysstat /usr/lib/sa/sa2
sysstat /usr/lib/sa/sadc
sysstat /usr/share/
sysstat /usr/share/doc/
sysstat /usr/share/doc/sysstat-9.0.6.1/
sysstat /usr/share/doc/sysstat-9.0.6.1/CHANGES
sysstat /usr/share/doc/sysstat-9.0.6.1/COPYING
sysstat /usr/share/doc/sysstat-9.0.6.1/CREDITS
sysstat /usr/share/doc/sysstat-9.0.6.1/FAQ
sysstat /usr/share/doc/sysstat-9.0.6.1/README
sysstat /usr/share/doc/sysstat-9.0.6.1/sysstat-9.0.6.1.lsm
sysstat /usr/share/locale/
sysstat /usr/share/locale/af/
sysstat /usr/share/locale/af/LC_MESSAGES/
sysstat /usr/share/locale/af/LC_MESSAGES/sysstat.mo
sysstat /usr/share/locale/da/
sysstat /usr/share/locale/da/LC_MESSAGES/
sysstat /usr/share/locale/da/LC_MESSAGES/sysstat.mo
sysstat /usr/share/locale/de/
sysstat /usr/share/locale/de/LC_MESSAGES/
sysstat /usr/share/locale/de/LC_MESSAGES/sysstat.mo
sysstat /usr/share/locale/es/
sysstat /usr/share/locale/es/LC_MESSAGES/
sysstat /usr/share/locale/es/LC_MESSAGES/sysstat.mo
sysstat /usr/share/locale/fi/
sysstat /usr/share/locale/fi/LC_MESSAGES/
sysstat /usr/share/locale/fi/LC_MESSAGES/sysstat.mo
sysstat /usr/share/locale/fr/
sysstat /usr/share/locale/fr/LC_MESSAGES/
sysstat /usr/share/locale/fr/LC_MESSAGES/sysstat.mo
sysstat /usr/share/locale/id/
sysstat /usr/share/locale/id/LC_MESSAGES/
sysstat /usr/share/locale/id/LC_MESSAGES/sysstat.mo
sysstat /usr/share/locale/it/
sysstat /usr/share/locale/it/LC_MESSAGES/
sysstat /usr/share/locale/it/LC_MESSAGES/sysstat.mo
sysstat /usr/share/locale/ja/
sysstat /usr/share/locale/ja/LC_MESSAGES/
sysstat /usr/share/locale/ja/LC_MESSAGES/sysstat.mo
sysstat /usr/share/locale/ky/
sysstat /usr/share/locale/ky/LC_MESSAGES/
sysstat /usr/share/locale/ky/LC_MESSAGES/sysstat.mo
sysstat /usr/share/locale/lv/
sysstat /usr/share/locale/lv/LC_MESSAGES/
sysstat /usr/share/locale/lv/LC_MESSAGES/sysstat.mo
sysstat /usr/share/locale/mt/
sysstat /usr/share/locale/mt/LC_MESSAGES/
sysstat /usr/share/locale/mt/LC_MESSAGES/sysstat.mo
sysstat /usr/share/locale/nb/
sysstat /usr/share/locale/nb/LC_MESSAGES/
sysstat /usr/share/locale/nb/LC_MESSAGES/sysstat.mo
sysstat /usr/share/locale/nl/
sysstat /usr/share/locale/nl/LC_MESSAGES/
sysstat /usr/share/locale/nl/LC_MESSAGES/sysstat.mo
sysstat /usr/share/locale/nn/
sysstat /usr/share/locale/nn/LC_MESSAGES/
sysstat /usr/share/locale/nn/LC_MESSAGES/sysstat.mo
sysstat /usr/share/locale/pl/
sysstat /usr/share/locale/pl/LC_MESSAGES/
sysstat /usr/share/locale/pl/LC_MESSAGES/sysstat.mo
sysstat /usr/share/locale/pt/
sysstat /usr/share/locale/pt/LC_MESSAGES/
sysstat /usr/share/locale/pt/LC_MESSAGES/sysstat.mo
sysstat /usr/share/locale/pt_BR/
sysstat /usr/share/locale/pt_BR/LC_MESSAGES/
sysstat /usr/share/locale/pt_BR/LC_MESSAGES/sysstat.mo
sysstat /usr/share/locale/ro/
sysstat /usr/share/locale/ro/LC_MESSAGES/
sysstat /usr/share/locale/ro/LC_MESSAGES/sysstat.mo
sysstat /usr/share/locale/ru/
sysstat /usr/share/locale/ru/LC_MESSAGES/
sysstat /usr/share/locale/ru/LC_MESSAGES/sysstat.mo
sysstat /usr/share/locale/sk/
sysstat /usr/share/locale/sk/LC_MESSAGES/
sysstat /usr/share/locale/sk/LC_MESSAGES/sysstat.mo
sysstat /usr/share/locale/sv/
sysstat /usr/share/locale/sv/LC_MESSAGES/
sysstat /usr/share/locale/sv/LC_MESSAGES/sysstat.mo
sysstat /usr/share/locale/vi/
sysstat /usr/share/locale/vi/LC_MESSAGES/
sysstat /usr/share/locale/vi/LC_MESSAGES/sysstat.mo
sysstat /usr/share/locale/zh_CN/
sysstat /usr/share/locale/zh_CN/LC_MESSAGES/
sysstat /usr/share/locale/zh_CN/LC_MESSAGES/sysstat.mo
sysstat /usr/share/locale/zh_TW/
sysstat /usr/share/locale/zh_TW/LC_MESSAGES/
sysstat /usr/share/locale/zh_TW/LC_MESSAGES/sysstat.mo
sysstat /usr/share/man/
sysstat /usr/share/man/man1/
sysstat /usr/share/man/man1/iostat.1.gz
sysstat /usr/share/man/man1/isag.1.gz
sysstat /usr/share/man/man1/mpstat.1.gz
sysstat /usr/share/man/man1/pidstat.1.gz
sysstat /usr/share/man/man1/sadf.1.gz
sysstat /usr/share/man/man1/sar.1.gz
sysstat /usr/share/man/man8/
sysstat /usr/share/man/man8/sa1.8.gz
sysstat /usr/share/man/man8/sa2.8.gz
sysstat /usr/share/man/man8/sadc.8.gz
sysstat /var/
sysstat /var/log/
sysstat /var/log/sa/

Now it puts a crontab entry to run daily on the installed system..although you can control it according your choice..

bhaskar@bhaskar-laptop_07:13:35_Tue Aug 31:/etc/cron.daily> cat sysstat
#!/bin/sh
# Generate a daily summary of process accounting. Since this will probably
# get kicked off in the morning, it would probably be better to run against
# the previous days data.
/usr/lib/sa/sa2 -A &

This package come with so many binary and all of them are very useful tool. I will explain all of them one by one.First tool is callled sar..and it will output like this..

bhaskar@bhaskar-laptop_07:15:59_Tue Aug 31:~> sudo sar
Password:
Linux 2.6.34-ARCH (bhaskar-laptop) 08/31/2010 _i686_ (2 CPU)

06:27:36 AM LINUX RESTART

06:28:02 AM CPU %user %nice %system %iowait %steal %idle
06:38:02 AM all 8.57 0.00 1.80 6.59 0.00 83.04
06:48:02 AM all 22.57 0.00 5.11 5.40 0.00 66.92
06:58:02 AM all 16.56 0.00 5.58 3.81 0.00 74.05
07:08:02 AM all 7.46 0.00 2.79 4.52 0.00 85.23
Average: all 13.79 0.00 3.82 5.08 0.00 77.31

Now bit of explanation is required for the fields it shows which I enumerated below:

Sar is system activity reporter.

%user and %nice refer to your software programs, such as MySQL or Apache.
%system refers to the kernel’s internal workings.
%iowait is time spent waiting for Input/Output, such as a disk read or write. Finally, since the kernel accounts for 100% of the runnable time it can schedule, any unused time goes into %idle.

It come along with another two binary relates sar is called sa1 and sa2.What does this fellows do to sar??

The sa1 script logs sar output into sysstat’s binary log file format, and sa2 reports it back in human readable format.Clear!

And the reports it provide kept in a dir called /var/log/sa and file with date attached to it…

bhaskar@bhaskar-laptop_07:25:28_Tue Aug 31:/var/log/sa> ls
sa23 sa24 sa25 sa30 sa31 sar24 sar30

“-W” this flag to sar shows the swap related activity on the system

bhaskar@bhaskar-laptop_07:31:09_Tue Aug 31:~> sudo sar -W
Password:
Linux 2.6.34-ARCH (bhaskar-laptop) 08/31/2010 _i686_ (2 CPU)

06:27:36 AM LINUX RESTART

06:28:02 AM pswpin/s pswpout/s
06:38:02 AM 0.00 0.00
06:48:02 AM 0.00 0.00
06:58:02 AM 0.03 1.67
07:08:02 AM 0.01 3.20
07:18:02 AM 0.13 4.02
07:28:01 AM 0.12 0.36
Average: 0.05 1.54

-r” option to sar show memory related thig from the system:

bhaskar@bhaskar-laptop_07:37:50_Tue Aug 31:~> sudo sar -r
Linux 2.6.34-ARCH (bhaskar-laptop) 08/31/2010 _i686_ (2 CPU)

06:27:36 AM LINUX RESTART

06:28:02 AM kbmemfree kbmemused %memused kbbuffers kbcached kbcommit %commit
06:38:02 AM 204468 815896 79.96 31784 399596 1412880 43.21
06:48:02 AM 62120 958244 93.91 61984 407388 1566932 47.93
06:58:02 AM 40344 980020 96.05 93368 348428 1622948 49.64
07:08:02 AM 27240 993124 97.33 126952 312652 1576108 48.21
07:18:02 AM 72528 947836 92.89 96988 334604 1454800 44.50
07:28:01 AM 58732 961632 94.24 95268 344540 1486996 45.48
Average: 77572 942792 92.40 84391 357868 1520111 46.49

-b” option shows some paging statistics :

bhaskar@bhaskar-laptop_07:41:19_Tue Aug 31:~> sudo sar -b
Linux 2.6.34-ARCH (bhaskar-laptop) 08/31/2010 _i686_ (2 CPU)

06:27:36 AM LINUX RESTART

06:28:02 AM tps rtps wtps bread/s bwrtn/s
06:38:02 AM 25.49 19.80 5.69 951.22 108.95
06:48:02 AM 35.10 20.35 14.76 259.20 337.45
06:58:02 AM 24.73 16.87 7.86 159.71 230.64
07:08:02 AM 53.55 42.64 10.91 356.85 273.27
07:18:02 AM 61.84 53.96 7.88 477.19 293.56
07:28:01 AM 5.12 1.15 3.96 49.08 74.69
07:38:01 AM 5.75 2.34 3.41 90.68 61.61
Average: 30.23 22.45 7.78 334.87 197.18

Anyway you can fuse your terminal reports into the excel sheet to manage thing as the origianl page is shown the way how to do it.Kndly visit the origianl pacakge author page to see the options.

Ok now lets talk about another binary come with package called “pidstat” What it does??

pidstat command is used to monitor processes and threads currently being managed by the Linux kernel. It can also monitor the children of those processes and threads.

on my system it shows the thing going underneath:

bhaskar@bhaskar-laptop_07:41:22_Tue Aug 31:~> sudo pidstat -d 2
Password:
Linux 2.6.34-ARCH (bhaskar-laptop) 08/31/2010 _i686_ (2 CPU)

07:46:56 AM PID kB_rd/s kB_wr/s kB_ccwr/s Command
07:46:58 AM 990 0.00 11.82 0.00 kjournald
07:46:58 AM 3948 0.00 1.97 1.97 plugin-containe

07:46:58 AM PID kB_rd/s kB_wr/s kB_ccwr/s Command

07:47:00 AM PID kB_rd/s kB_wr/s kB_ccwr/s Command
07:47:02 AM 990 0.00 6.00 0.00 kjournald
07:47:02 AM 3948 0.00 2.00 2.00 plugin-containe

07:47:02 AM PID kB_rd/s kB_wr/s kB_ccwr/s Command
07:47:04 AM 3835 0.00 2.00 2.00 firefox

07:47:04 AM PID kB_rd/s kB_wr/s kB_ccwr/s Command

07:47:06 AM PID kB_rd/s kB_wr/s kB_ccwr/s Command
07:47:08 AM 990 0.00 8.00 0.00 kjournald
07:47:08 AM 3948 0.00 2.00 2.00 plugin-containe

07:47:08 AM PID kB_rd/s kB_wr/s kB_ccwr/s Command

07:47:10 AM PID kB_rd/s kB_wr/s kB_ccwr/s Command
07:47:12 AM 990 0.00 12.00 0.00 kjournald
07:47:12 AM 3948 0.00 2.00 2.00 plugin-containe

-d” option provide I/O statistics.

Now get the memory utilisation stat through “-r” flag of this binary.

bhaskar@bhaskar-laptop_07:50:18_Tue Aug 31:~> sudo pidstat -r
Linux 2.6.34-ARCH (bhaskar-laptop) 08/31/2010 _i686_ (2 CPU)

07:51:41 AM PID minflt/s majflt/s VSZ RSS %MEM Command
07:51:41 AM 1 0.45 0.00 1752 620 0.06 init
07:51:41 AM 1023 0.54 0.00 2148 948 0.09 udevd
07:51:41 AM 2772 0.02 0.00 5080 428 0.04 syslog-ng
07:51:41 AM 2773 0.09 0.00 5396 1620 0.16 syslog-ng
07:51:41 AM 2804 0.03 0.00 3420 700 0.07 ntpd
07:51:41 AM 2805 0.25 0.00 8232 3848 0.38 named
07:51:41 AM 2824 0.04 0.00 2372 760 0.07 xinetd
07:51:41 AM 2834 0.23 0.00 18672 8492 0.83 httpd
07:51:41 AM 2874 0.05 0.00 18724 6760 0.66 httpd
07:51:41 AM 2875 0.15 0.00 18688 7660 0.75 httpd
07:51:41 AM 2876 0.15 0.00 18688 7660 0.75 httpd
07:51:41 AM 2877 0.14 0.00 18824 7844 0.77 httpd
07:51:41 AM 2878 0.15 0.00 18688 7660 0.75 httpd
07:51:41 AM 2879 0.15 0.00 18688 7660 0.75 httpd
07:51:41 AM 2917 0.13 0.00 8468 1888 0.19 master
07:51:41 AM 2931 0.12 0.00 8540 1760 0.17 pickup
07:51:41 AM 2932 0.12 0.00 8592 1776 0.17 qmgr
07:51:41 AM 2936 0.04 0.00 1800 628 0.06 crond
07:51:41 AM 2946 0.16 0.00 2788 1216 0.12 dbus-daemon
07:51:41 AM 2954 0.60 0.00 15052 3608 0.35 hald
07:51:41 AM 2955 0.19 0.00 3516 1172 0.11 hald-runner
07:51:41 AM 2984 0.08 0.00 3580 992 0.10 hald-addon-inpu
07:51:41 AM 2986 0.08 0.00 3580 988 0.10 hald-addon-rfki
07:51:41 AM 2987 0.08 0.00 3580 984 0.10 hald-addon-leds
07:51:41 AM 2996 0.07 0.00 3576 992 0.10 hald-addon-gene
07:51:41 AM 2998 0.08 0.00 3580 1000 0.10 hald-addon-stor
07:51:41 AM 3008 0.08 0.00 3244 1012 0.10 hald-addon-acpi
07:51:41 AM 3023 0.03 0.00 6216 688 0.07 rpcbind
07:51:41 AM 3027 0.02 0.00 3148 308 0.03 famd
07:51:41 AM 3036 0.02 0.00 1960 356 0.03 gpm
07:51:41 AM 3044 0.24 0.00 13912 2152 0.21 gdm-binary
07:51:41 AM 3060 3.17 0.04 143100 19616 1.92 dropbox
07:51:41 AM 3072 0.04 0.00 1752 536 0.05 agetty
07:51:41 AM 3073 0.04 0.00 1752 540 0.05 agetty
07:51:41 AM 3074 0.04 0.00 1752 540 0.05 agetty
07:51:41 AM 3075 0.04 0.00 1752 536 0.05 agetty
07:51:41 AM 3076 0.04 0.00 1752 532 0.05 agetty
07:51:41 AM 3077 0.04 0.00 1752 536 0.05 agetty
07:51:41 AM 3132 0.25 0.00 17028 2804 0.27 gdm-simple-slav
07:51:41 AM 3155 162.21 0.01 82076 35784 3.51 Xorg
07:51:41 AM 3194 0.28 0.00 18224 2236 0.22 console-kit-dae
07:51:41 AM 3195 0.03 0.00 3528 644 0.06 ntpd
07:51:41 AM 3272 0.03 0.00 3172 332 0.03 dbus-launch
07:51:41 AM 3277 0.24 0.00 5448 2540 0.25 upowerd
07:51:41 AM 3331 0.42 0.00 17664 5400 0.53 polkit-gnome-au
07:51:41 AM 3335 0.25 0.00 5640 2672 0.26 polkitd
07:51:41 AM 3336 0.16 0.00 14888 2064 0.20 gdm-session-wor
07:51:41 AM 3349 0.11 0.00 22176 1432 0.14 gnome-keyring-d
07:51:41 AM 3367 0.73 0.00 25064 5380 0.53 gnome-session
07:51:41 AM 3385 0.03 0.00 3172 332 0.03 dbus-launch
07:51:41 AM 3386 0.09 0.00 2616 1116 0.11 dbus-daemon
07:51:41 AM 3388 0.02 0.00 3540 224 0.02 ssh-agent
07:51:41 AM 3391 0.18 0.00 6776 2580 0.25 gconfd-2
07:51:41 AM 3396 0.69 0.01 22052 7000 0.69 gnome-settings-
07:51:41 AM 3401 0.13 0.00 6352 1708 0.17 gvfsd
07:51:41 AM 3404 0.79 0.00 55332 12088 1.18 metacity
07:51:41 AM 3405 0.90 0.01 42768 11040 1.08 gnome-panel
07:51:41 AM 3407 0.21 0.00 7932 2764 0.27 gvfs-gdu-volume
07:51:41 AM 3409 0.25 0.00 13512 2556 0.25 udisks-daemon
07:51:41 AM 3410 0.03 0.00 5000 512 0.05 udisks-daemon
07:51:41 AM 3425 0.11 0.00 38324 1492 0.15 gvfs-fuse-daemo
07:51:41 AM 3429 1.24 0.02 53156 8800 0.86 nautilus
07:51:41 AM 3431 0.25 0.00 41896 2352 0.23 bonobo-activati
07:51:41 AM 3442 27.99 0.00 20928 6888 0.68 multiload-apple
07:51:41 AM 3445 0.44 0.01 21036 7000 0.69 battstat-applet
07:51:41 AM 3446 0.67 0.00 50716 11744 1.15 gweather-applet
07:51:41 AM 3448 0.74 0.01 41420 11764 1.15 clock-applet
07:51:41 AM 3450 0.70 0.00 41736 11532 1.13 wnck-applet
07:51:41 AM 3451 0.42 0.00 20500 6644 0.65 notification-ar
07:51:41 AM 3452 0.29 0.00 16120 4172 0.41 polkit-gnome-au
07:51:41 AM 3454 0.35 0.00 17840 5308 0.52 gdu-notificatio
07:51:41 AM 3480 0.08 0.00 17236 1460 0.14 gnome-screensav
07:51:41 AM 3502 0.17 0.00 6856 2396 0.23 gvfsd-trash
07:51:41 AM 3509 0.18 0.00 4828 1888 0.19 system-tools-ba
07:51:41 AM 3516 0.13 0.00 6484 1740 0.17 gvfsd-burn
07:51:41 AM 3522 0.76 0.00 12724 10152 0.99 SystemToolsBack
07:51:41 AM 3528 0.76 0.01 47896 12500 1.23 gnome-terminal
07:51:41 AM 3529 0.05 0.00 1796 580 0.06 gnome-pty-helpe
07:51:41 AM 3531 0.15 0.00 4924 1884 0.18 bash
07:51:41 AM 3538 0.07 0.00 4412 844 0.08 screen
07:51:41 AM 3539 0.13 0.00 4676 1372 0.13 screen
07:51:41 AM 3540 0.16 0.00 4924 1892 0.19 bash
07:51:41 AM 3546 0.14 0.00 4000 900 0.09 su
07:51:41 AM 3547 0.20 0.00 4900 1876 0.18 bash
07:51:41 AM 3577 0.04 0.00 2144 844 0.08 udevd
07:51:41 AM 3578 0.03 0.00 2144 864 0.08 udevd
07:51:41 AM 3611 0.16 0.00 4924 1872 0.18 bash
07:51:41 AM 3617 0.26 0.01 4884 1796 0.18 bash
07:51:41 AM 3628 0.16 0.00 4924 1876 0.18 bash
07:51:41 AM 3635 0.21 0.01 5136 1164 0.11 bash
07:51:41 AM 3655 0.21 0.00 4924 1920 0.19 bash
07:51:41 AM 3674 0.14 0.00 4000 904 0.09 su
07:51:41 AM 3675 0.16 0.00 4900 1868 0.18 bash
07:51:41 AM 3684 0.27 0.01 5552 2740 0.27 bash
07:51:41 AM 3701 1.87 0.00 4924 1956 0.19 bash
07:51:41 AM 3708 0.13 0.00 4604 1372 0.13 thunderbird
07:51:41 AM 3721 0.10 0.00 4736 1420 0.14 run-mozilla.sh
07:51:41 AM 3725 8.07 0.08 272920 81716 8.01 thunderbird-bin
07:51:41 AM 3835 234.27 0.12 749812 283212 27.76 firefox
07:51:41 AM 3848 0.01 0.00 4608 448 0.04 firefox_cpu_lim
07:51:41 AM 3849 9.11 0.00 1916 588 0.06 cpulimit
07:51:41 AM 3948 9.40 0.03 112288 36312 3.56 plugin-containe
07:51:41 AM 3956 0.13 0.00 18688 7656 0.75 httpd
07:51:41 AM 3965 0.70 0.00 41128 10936 1.07 notification-da
07:51:41 AM 5130 0.13 0.00 1780 636 0.06 sadc
07:51:41 AM 5859 0.19 0.00 3656 812 0.08 pidstat

Now lets talk about two very important tool which will provide different way to view things. Those are called “sadc” and “sadf“.I will cover one after another below.

SADC:
It is system activity and data collector daemon.Even w can use it manually too!.Sadc command intened to run behind the sar command.Actually it will write the binary format of the statistics it collect day by day and put into a dir i.e /var/log/sadd,where dd stands for the particular day.As the man page said it can only provide local activity,means runs on the same host it installed.

I am putting here few example stright out of the manual page for clear understanding.Here we go:

/usr/lib/sa/sadc 1 10 /tmp/datafile
Write 10 records of one second intervals to the /tmp/datafile binary file.

/usr/lib/sa/sadc -C Backup_Start /tmp/datafile
Insert the comment Backup_Start into the file /tmp/datafile.

So move onto the next tool called sadf.

SADF:
This tool actually dispaly the collected data by sar in different format.Which is wonderful..because you can fuse your data to various places to get lot many information.It will essentially provide XML,CVS format data .

once again I am putting example stright out of the manual page for easy understanding.Here we go:

sadf -d /var/log/sa/sa21 — -r -n DEV
Extract memory, swap space and network statistics from system activity file ‘sa21’, and display them in a format that can be ingested by a
database.

sadf -p -P 1
Extract CPU statistics for processor 1 (the second processor) from current daily data file, and display them in a format that can easily be
handled by a pattern processing command.

Hope this will help.

Cheers!
Bhaskar

Filesystem internals from user space

Aha!this is a topic which excite almost everybody related to computer field.Reason is very much inevitable,because it is the base on which lot many thing depends on the computer system.In this article I am going show you some very well known and used tools to get some internals about the filesystem specifically ext2/3/4 .So first when we create a device,means partitons ,those partitions are raw.So to keep data on it you need to have a filesystem on it to hold your data.right? for that first thing we do create a files system on that newly created device.It can be done through many utility software come along with the util-linux-ng package.Few names are very common i.e fdisk,gparted,sfdisk,cfdisk.

A few words about journaling:

Journaling file systems use a journal to buffer changes to the file system (which is also used in crash recovery) but can use different strategies for when and what is journaled. Three of the most common strategies are writeback, ordered, and data.

Writeback mode, only the metadata is journaled, and the data blocks are written directly to their location on the disk. This preserves the file system structure and avoids corruption, but data corruption can occur (for example, if the system crashes after the metadata is journaled but before the data block is written). To solve this problem, you can use ordered mode.

Ordered mode is metadata journaling only but writes the data before journaling the metadata. In this way, data and file system are guaranteed consistent after a recovery. Finally, data journaling can also be supported.

Data mode, both metadata and data are journaled. This mode offers the greatest protection against file system corruption and data loss but can suffer from performance degradation, as all data is written twice (first to the journal, then to the disk).

The journal commit policy can also differ in the various approaches. For example, is the journal committed when it nears full, or through a timeout?

I am not going to show you the device creation here as I assume you have already created and make filesystem on it( by running mkfs on it),check out the mkfs man page for different options and variants of it.Like for creating ext3 filesystem you might use mkfs.ext3…..

bhaskar@bhaskar-laptop_12:01:44_Mon Aug 30:~> whereis mkfs
mkfs: /sbin/mkfs.ext4 /sbin/mkfs.minix /sbin/mkfs.ext3 /sbin/mkfs.ext4dev /sbin/mkfs.cramfs /sbin/mkfs.bfs /sbin/mkfs /sbin/mkfs.ext2

See how many variant it has.Ok lets get the information of ext3 filesystem mounted on device.Here we go:

We have to have a package called e2fsprogs installed in our system ..although it basically installed by defaults.This package hold all the tools needed to check and spit out the information about the filesystem informations.

Superblock:

What is it? Nothing but the holding the metadata about the partition and reside in the very first block in the every partition.Now it hold below information within it:

File system like ext2. ext3 etc. Superblock contents the information about file system like –
* File system type
* Size
* Status
* Information about other metadata

So you can understand how important it is.Now all the electronic device is prone to failure as the filesystem too and superblock is prone getting corrupted every so very often.

– You can’t able to mount the filesystem, it will refuse to mount
– Filesystem gets hang
– Sometimes though you are able to mount that filesystem, but strange behavior occures.

So here comes the basic thing if you can’t mount the partition then how can you work on??right.Fortunately superblock spread in different location on the disk,means has got the copy of fisrt superblock somewhere else.How do you able to find those block??Here is the way to find the alternative superblock:

bhaskar@bhaskar-laptop_12:05:31_Mon Aug 30:~> sudo dumpe2fs /dev/sda3 | grep superblock
[sudo] password for bhaskar:
dumpe2fs 1.41.3 (12-Oct-2008)
Primary superblock at 0, Group descriptors at 1-1
Backup superblock at 32768, Group descriptors at 32769-32769
Backup superblock at 98304, Group descriptors at 98305-98305
Backup superblock at 163840, Group descriptors at 163841-163841
Backup superblock at 229376, Group descriptors at 229377-229377
Backup superblock at 294912, Group descriptors at 294913-294913
Backup superblock at 819200, Group descriptors at 819201-819201
Backup superblock at 884736, Group descriptors at 884737-884737
Backup superblock at 1605632, Group descriptors at 1605633-1605633
Backup superblock at 2654208, Group descriptors at 2654209-2654209

Yes we use a tool called dumpe2fs ,which will come with the package I mentioned earlier.So the superblock is kept in different places.In the event of superblock corruption you might copy other superblock(backup location) into the main portion.Let me show you how you can do that:

bhaskar@bhaskar-laptop_12:26:58_Mon Aug 30:~> sudo /e2fsck -f -b  32768 /dev/sda3

Now a bit of explanation.The “-b” option is to provide the alternative superblock to replace the corrupted one.And the device mentioned holding the corrupted superblock,in this case it is /dev/sda3.Clear??

Ok lets move on ,I want know about the filesystem metadata of a particular device how do I do that??Here is way to do it:

bhaskar@bhaskar-laptop_12:31:20_Mon Aug 30:~> sudo tune2fs -l /dev/sda3
tune2fs 1.41.3 (12-Oct-2008)
Filesystem volume name:   <none>
Last mounted on:          <not available>
Filesystem UUID:          3cccbf0e-0354-43b4-b89a-ceee1fcadb31
Filesystem magic number:  0xEF53
Filesystem revision #:    1 (dynamic)
Filesystem features:      has_journal resize_inode dir_index filetype needs_recovery sparse_super large_file
Filesystem flags:         signed_directory_hash
Default mount options:    (none)
Filesystem state:         clean
Errors behavior:          Continue
Filesystem OS type:       Linux
Inode count:              1710240
Block count:              3417828
Reserved block count:     170891
Free blocks:              2352985
Free inodes:              1562077
First block:              0
Block size:               4096
Fragment size:            4096
Reserved GDT blocks:      834
Blocks per group:         32768
Fragments per group:      32768
Inodes per group:         16288
Inode blocks per group:   509
Filesystem created:       Thu Feb  4 15:19:47 2010
Last mount time:          Mon Aug 30 16:35:08 2010
Last write time:          Mon Aug 30 16:35:08 2010
Mount count:              14
Maximum mount count:      26
Last checked:             Sun Jul 25 22:57:56 2010
Check interval:           15552000 (6 months)
Next check after:         Fri Jan 21 22:57:56 2011
Reserved blocks uid:      0 (user root)
Reserved blocks gid:      0 (group root)
First inode:              11
Inode size:               128
Journal inode:            8
First orphan inode:       1010177
Default directory hash:   tea
Directory Hash Seed:      76073fa9-2f5a-4246-926a-b384bae24c6a
Journal backup:           inode blocks

As you can see in the above output so many internals are revealed!! kindly go through it so you can get a grasp of it or might understand what that option for.

I want to add acl(Access Control List) on the filesystem as it doesn’t have at this moment( refer to above output).ACL is beast,it add security permission in the file system.Please do not intermingle with SELinux permission with it.It’s an added layer or extra protection or preciseness you filesystem might get.Those who sits on RHEL/CentOS this ACL thing can naturally come to them as it glue with this distribution.

Now once go through the manual of tune2fs and find -o option and pass the device which holds the filesystem to add acl. So here we go:

bhaskar@bhaskar-laptop_12:40:53_Mon Aug 30:~> sudo tune2fs -o acl /dev/sda3
tune2fs 1.41.3 (12-Oct-2008)
bhaskar@bhaskar-laptop_12:41:50_Mon Aug 30:~> sudo tune2fs -l /dev/sda3 | grep acl
Default mount options:    acl

You can see I have added the acl option to the specific device filesystem.Best possible way to find that it work just to remount it.Then create a file system and check .And yes I did reboot.

Now I am going to create a file under my home dir and check acl permission for that file.Right..here we go:

bhaskar@bhaskar-laptop_12:57:30_Mon Aug 30:~> touch aclcheck
bhaskar@bhaskar-laptop_13:12:39_Mon Aug 30:~> ls -al aclcheck
-rw-r–r– 1 bhaskar bhaskar 0 2010-08-30 13:12 aclcheck
bhaskar@bhaskar-laptop_13:12:47_Mon Aug 30:~> getfacl aclcheck
# file: aclcheck
# owner: bhaskar
# group: bhaskar
user::rw-
group::r–
other::r–

As yon can see the output of getfacl…on debian you need to get acl package through aptitude. Then only you get this userspace tools.

This kind of security system greatly help sharing system resources with the outside world.You can set the acl permission through a binary called setfacl .

bhaskar@bhaskar-laptop_13:21:54_Mon Aug 30:~> setfacl -m u:bhaskar:rw aclcheck

bhaskar@bhaskar-laptop_13:22:16_Mon Aug 30:~> getfacl aclcheck
# file: aclcheck
# owner: bhaskar
# group: bhaskar
user::rw-
user:bhaskar:rw-
group::r–
mask::rw-
other::r–

So what I did? I have used sefacl command to set the file permission . Let me put across few example stright out of the manual:

Granting an additional user read access
setfacl -m u:lisa:r file

Revoking write access from all groups and all named users (using the effective rights mask)
setfacl -m m::rx file

Removing a named group entry from a file’s ACL
setfacl -x g:staff file

Copying the ACL of one file to another
getfacl file1 | setfacl –set-file=- file2

Copying the access ACL into the Default ACL
getfacl –access dir | setfacl -d -M- dir

Cool …right?

Now you need to which filesystem have been built with the kernel through a /proc virtual filesystem entry like this:

bhaskar@bhaskar-laptop_13:36:23_Mon Aug 30:~> sudo cat /proc/filesystems
nodev   sysfs
nodev   rootfs
nodev   bdev
nodev   proc
nodev   cgroup
nodev   cpuset
nodev   debugfs
nodev   securityfs
nodev   sockfs
nodev   pipefs
nodev   anon_inodefs
nodev   tmpfs
nodev   inotifyfs
nodev   devpts
nodev   ramfs
nodev   hugetlbfs
nodev   mqueue
nodev   usbfs
ext3
nodev   rpc_pipefs
nodev   nfsd

If you want add more filesystm file then you need to fuse that filesystem entry by rebuilding the kernel .And your entry should be enlisted by the /proc/filesystem file.

You can get a actually mounted system through a file called /etc/mtab or /proc/mounts

bhaskar@bhaskar-laptop_13:36:32_Mon Aug 30:~> sudo cat /etc/mtab
/dev/sda3 / ext3 rw,errors=remount-ro 0 0
tmpfs /lib/init/rw tmpfs rw,nosuid,mode=0755 0 0
proc /proc proc rw,noexec,nosuid,nodev 0 0
sysfs /sys sysfs rw,noexec,nosuid,nodev 0 0
procbususb /proc/bus/usb usbfs rw 0 0
udev /dev tmpfs rw,mode=0755 0 0
tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0
devpts /dev/pts devpts rw,noexec,nosuid,gid=5,mode=620 0 0
/dev/mapper/bhaskarlaptop-data /lvm ext3 rw 0 0
nfsd /proc/fs/nfsd nfsd rw 0 0

OR

bhaskar@bhaskar-laptop_13:39:44_Mon Aug 30:~> sudo cat /proc/mounts
rootfs / rootfs rw 0 0
none /sys sysfs rw,nosuid,nodev,noexec 0 0
none /proc proc rw,nosuid,nodev,noexec 0 0
udev /dev tmpfs rw,size=10240k,mode=755 0 0
/dev/sda3 / ext3 rw,errors=remount-ro,acl,data=ordered 0 0
tmpfs /lib/init/rw tmpfs rw,nosuid,mode=755 0 0
usbfs /proc/bus/usb usbfs rw,nosuid,nodev,noexec 0 0
tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0
devpts /dev/pts devpts rw,nosuid,noexec,gid=5,mode=620 0 0
/dev/mapper/bhaskarlaptop-data /lvm ext3 rw,errors=continue,data=ordered 0 0
nfsd /proc/fs/nfsd nfsd rw 0 0

For more enthusiastic readers I  would recommend to look into the debugfs manual page,as it provide plethora of option to examine the filesystem.I believe that I have covered regarding LVM filesystem in my other post.

Hope this will help.

Cheers!

Bhaskar

Important system tools

In this article I am going to take you to different level of system information.To know more about internals and methods . I will touch few thing which is essential to operate on such system.

So enough talks lets get our hands dirty with some methods.Here we go:

Chroot:
This is a tool used for the multihomed system.What did I mean by saying “multihomed” system?? a bit of explanation to those who are not so used to it.It essentially means that a system( server,desktop,workstation,laptop) has more then one OS installed in different partitions(Not a virtualized one..what we have these days).

So accessing those partition from single OS would a difficult without having this tool at your disposal.Let’s invoke it..but before that I should give little bit background about the system where we run this.A system running different flavour of GNU/Linux distro and has exclusive partition for each OS.

As I said earlier that I am running six different GNU/Linux distro ..out of four of them are in different partitions excluding swap,which has it;s own partition.

So everytime get into any of the OS and chroot to other OS too.How? I have written a small bash script for that ..which basically consists of chroot commands and mounts..pretty ordinary stuff. So the scripts look like this:


1 #!/bin/bash
2
3 echo " All my slices are going to be mounted . Please wait......"
4
5 #This is for Gentoo
6 /bin/mount /dev/sda1 /Gentoo
7 /bin/mount -t proc none /Gentoo/proc
8 /bin/mount -t sysfs none /Gentoo/sys
9 /bin/mount -o bind /dev /Gentoo/dev
10
11 echo " Done....mounted Gentoo"
12
13 #Arch
14
15 /bin/mount /dev/sda5 /Arch
16 /bin/mount -t proc none /Arch/proc
17 /bin/mount -t sysfs none /Arch/sys
18 /bin/mount -o bind /dev /Arch/dev
19
20 echo "Done..... mounted Arch"
21
22 #This is for Fedora
23
24 /bin/mount /dev/sda6 /Fedora/boot
25 /bin/mount -t ext4 /dev/sda7 /Fedora
26 /bin/mount -t proc none /Fedora/proc
27 /bin/mount -t sysfs none /Fedora/sys
28 /bin/mount -o bind /dev /Fedora/dev
29
30 echo "Done....mounted Fedora"
31 echo
32
33 echo " Now please run screen and chroot to that slice..."

Now bit of explanation of this script and background.First and foremost thing I have four directory created on root(/) before I run this script.And I name them according to the distributions name I am going to mount on it.So there will be no confusion.

Like my root(/) dir on Debian Lenny looks like this(as you can see in the script code that Debian mount is missing):


bhaskar@bhaskar-laptop_09:37:50_Sun Aug 22:/> ls
Arch boot dev Fedora home initrd.img lib lvm mnt proc sbin srv tmp var vmlinuz.old
bin cdrom etc Gentoo initrd initrd.img.old lost+found media opt root selinux sys usr vmlinuz

So you can put it into rc.local to execute it once the boot script finish..you can get readily mounted partitions.Now I am in the process to automate the chrooting process trough screen(a software through which you can get multiple virtual terminals…which is a very essential software..which I will cover next).But for the time being I will show you how you can get into each OS with a proper chroot ed environment.

We have to do something like below on the terminal:

root@bhaskar-laptop_09:43:02_Sun Aug 22:/ # chroot /Arch/ /bin/bash

As the script already mounted that partition so I just use the chroot binary to get into that specified directory,where we mounted that partition specific to that OS and get a bash shell.Now verify that you are in correct OS like this:

root@bhaskar-laptop_09:46:42_Sun Aug 22:/etc # pacman -Ss vnstat
community/vnstat 1.10-5 [installed]
A console-based network traffic monitor

Now hadn’t been I am in proper intended os the package manager won’t run.Yes people might say it is not the best way to know..surely the effective way to know.Right?

Screen:
It is an indispensable tool in the production environment with serveral administrator working on the same servers simeltenously. Ok..everytime you call this software on the terminal like below:


root@bhaskar-laptop_09:59:57_Sun Aug 22:/etc # screen

what it will do is it will create a virtual terminal and put you to a shell..probably in a bash shell.Now you can create more shell and move on those window of shell with the below command:

CTRL-a c ---------> all the screen command start with CTRL-a then a specific letter to do a specific job,here c will create a new widow.

Likewise if you want set the title of a particular terminal with the below keypress:

CTRL-a t------------> t here signifies the title

Now the global configuaration file for screen is placed in /etc as screenrc. My personal screenrc is below;

root@bhaskar-laptop_10:08:06_Sun Aug 22:/etc # cat screenrc
# this is the global screenrc file. Handle with care.

termcapinfo xterm* G0:is=\E[?4l\E>:ti@:te@
termcapinfo linux me=\E[m:AX
#termcapinfo xterm* ti@:te@
startup_message off
vbell off
autodetach on
altscreen on
shelltitle “$ |bash”
#defscrollback 10000
defutf8 on
nonblock on
hardstatus alwayslastline
hardstatus string ‘%{gk}[ %{G}%H %{g}][%= %{wk}%?%-Lw%?%{=b kR}(%{W}%n*%f %t%?(%u)%?%{=b kR})%{= kw}%?%+Lw%?%?%= %{g}][%{Y}%l%{g}]%{=b C}[ %m/%d %c ]%{W}’
It will look like the below image:

So if you enlarge the image you can see I am running screen with two chrooted OS terminal on it. It’s on Debian Lenny .
Now how do you determine that the terminal is running screen or under the jurisdiction ? Here is the way to find it:

First method:

bhaskar@bhaskar-laptop_10:26:35_Sun Aug 22:~> env | grep screen
TERM=screen

Second way:

bhaskar@bhaskar-laptop_10:26:52_Sun Aug 22:~> ps -ef | grep screen
bhaskar 4033 1 0 07:14 ? 00:00:11 gnome-screensaver
bhaskar 4112 4097 0 07:14 pts/0 00:00:00 screen

Screen has a huge manual page and plethora of wonderful options.I am going to provide you something very useful and frequently used straight out of the screen manual.


-d -r Reattach a session and if necessary detach it first.

-d -R Reattach a session and if necessary detach or even create it first.

-d -RR Reattach a session and if necessary detach or create it. Use the first session if more than one session is available.

-D -r Reattach a session. If necessary detach and logout remotely first.

-D -R Attach here and now. In detail this means: If a session is running, then reattach. If necessary detach and logout remotely first. If it
was not running create it and notify the user. This is the author’s favorite.

-D -RR Attach here and now. Whatever that means, just do it.

This options should be called with CTRL-a prefixed as I said earlier.Now how to determine any screen session left :

bhaskar@bhaskar-laptop_10:31:16_Sun Aug 22:~> screen -list
There is a screen on:
4113.pts-0.bhaskar-laptop (08/22/2010 07:14:36 AM) (Attached)
1 Socket in /var/run/screen/S-bhaskar.

Kindly go through the screen manual.It is very essential to know few tricks to work with.

Strace:

It is a wonderful tool for debugging problemetic software or at least throw you some light on the problem ..so you can take action on that.Say one file day you see that one of your favourite progam is not running and throwing error.After few search in the internet and post some queries in those related forums ..you got not the answer you need.So self help is the best way one help himself/herself on GNU/Linux box.


bhaskar@bhaskar-laptop_10:47:33_Sun Aug 22:~> sudo /usr/bin/strace ls
[sudo] password for bhaskar:
execve("/bin/ls", ["ls"], [/* 15 vars */]) = 0
brk(0) = 0x805f000
access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory)
mmap2(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7765000
access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory)
open("/etc/ld.so.cache", O_RDONLY) = 3
fstat64(3, {st_mode=S_IFREG|0644, st_size=63757, ...}) = 0
mmap2(NULL, 63757, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb7755000
close(3) = 0
access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory)
open("/lib/i686/cmov/librt.so.1", O_RDONLY) = 3
read(3, "\177ELF\1\1\1\3\3\1`\31004\240"..., 512) = 512
fstat64(3, {st_mode=S_IFREG|0644, st_size=30624, ...}) = 0
mmap2(NULL, 33360, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb774c000
mmap2(0xb7753000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x6) = 0xb7753000
close(3) = 0
access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory)
open("/lib/libselinux.so.1", O_RDONLY) = 3
read(3, "\177ELF\1\1\1\3\3\1\260R-B4 6, base_addr:0xb75d1700, limit:1048575, seg_32bit:1, contents:0, read_exec_only:0, limit_in_pages:1, seg_not_present:0, useable:1}) = 0
mprotect(0xb7745000, 4096, PROT_READ) = 0
munmap(0xb7755000, 63757) = 0
set_tid_address(0xb75d1748) = 18659
set_robust_list(0xb75d1750, 0xc) = 0
futex(0xbfffec80, FUTEX_WAKE_PRIVATE, 1) = 0
rt_sigaction(SIGRTMIN, {0xb75db2e0, [], SA_SIGINFO}, NULL, 8) = 0
rt_sigaction(SIGRT_1, {0xb75db720, [], SA_RESTART|SA_SIGINFO}, NULL, 8) = 0
rt_sigprocmask(SIG_UNBLOCK, [RTMIN RT_1], NULL, 8) = 0
getrlimit(RLIMIT_STACK, {rlim_cur=8192*1024, rlim_max=RLIM_INFINITY}) = 0
uname({sys="Linux", node="bhaskar-laptop", ...}) = 0
brk(0) = 0x805f000
brk(0x8080000) = 0x8080000
open("/etc/selinux/config", O_RDONLY|O_LARGEFILE) = 3
fstat64(3, {st_mode=S_IFREG|0644, st_size=578, ...}) = 0
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7764000
read(3, "# This file controls the state of"..., 4096) = 578
read(3, ""..., 4096) = 0
close(3) = 0
munmap(0xb7764000, 4096) = 0
statfs64("/selinux", 84, {f_type="EXT2_SUPER_MAGIC", f_bsize=4096, f_blocks=3364153, f_bfree=2355023, f_bavail=2184132, f_files=1710240, f_ffree=1562276, f_fsid={-529443196, -2053595620}, f_namelen=255, f_frsize=4096}) = 0
open("/proc/mounts", O_RDONLY|O_LARGEFILE) = 3
fstat64(3, {st_mode=S_IFREG|0444, st_size=0, ...}) = 0
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7764000
read(3, "rootfs / rootfs rw 0 0\nnone /sys "..., 1024) = 849
read(3, ""..., 1024) = 0
close(3) = 0
munmap(0xb7764000, 4096) = 0
open("/usr/lib/locale/locale-archive", O_RDONLY|O_LARGEFILE) = 3
fstat64(3, {st_mode=S_IFREG|0644, st_size=1282816, ...}) = 0
mmap2(NULL, 1282816, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb7497000
close(3) = 0
ioctl(1, SNDCTL_TMR_TIMEBASE or TCGETS, {B38400 opost isig icanon echo ...}) = 0
ioctl(1, TIOCGWINSZ, {ws_row=40, ws_col=157, ws_xpixel=0, ws_ypixel=0}) = 0
open(".", O_RDONLY|O_NONBLOCK|O_LARGEFILE|O_DIRECTORY|O_CLOEXEC) = 3
fstat64(3, {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
fcntl64(3, F_GETFD) = 0x1 (flags FD_CLOEXEC)
getdents64(3, /* 67 entries */, 4096) = 2248
getdents64(3, /* 0 entries */, 4096) = 0
close(3) = 0
open("/usr/lib/gconv/gconv-modules.cache", O_RDONLY) = 3
fstat64(3, {st_mode=S_IFREG|0644, st_size=25700, ...}) = 0
mmap2(NULL, 25700, PROT_READ, MAP_SHARED, 3, 0) = 0xb775e000
close(3) = 0
futex(0xb7747a6c, FUTEX_WAKE_PRIVATE, 2147483647) = 0
fstat64(1, {st_mode=S_IFCHR|0620, st_rdev=makedev(136, 4), ...}) = 0
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb775d000
write(1, "Desktop Documents febe\t ff"..., 99Desktop Documents febe ff_database_optimize LKM21_1024.png lsap_tux.png thunderbird
) = 99
write(1, "disk-mount Downloads ff_cpu_lim"..., 133disk-mount Downloads ff_cpu_lim find_inode_of_filesystem lsap_tux2.png SiteDelta
) = 133
close(1) = 0
munmap(0xb775d000, 4096) = 0
close(2) = 0
exit_group(0) = ?

As you can see from the above output that a simple run on a binary of “ls” can spit out lot of internals about that program.

For instance my thunderbird is not working on Gentoo ..so I go ahead with strace to find out why?


root@bhaskar-laptop_10:50:34_Sun Aug 22:/ # strace usr/bin/thunderbird
execve("usr/bin/thunderbird", ["usr/bin/thunderbird"], [/* 22 vars */]) = 0
brk(0) = 0x8108000
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7761000
access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory)
open("/etc/ld.so.cache", O_RDONLY) = 3
fstat64(3, {st_mode=S_IFREG|0644, st_size=141430, ...}) = 0
mmap2(NULL, 141430, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb773e000
close(3) = 0
open("/lib/libncurses.so.5", O_RDONLY) = 3
read(3, "\177ELF\1\1\1\3\3\1\320\303004"..., 512) = 512
fstat64(3, {st_mode=S_IFREG|0755, st_size=273784, ...}) = 0
mmap2(NULL, 273956, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb76fb000
mmap2(0xb773b000, 12288, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x40) = 0xb773b000
close(3) = 0
open("/lib/libdl.so.2", O_RDONLY) = 3
read(3, "\177ELF\1\1\1\3\3\1000\n004"..., 512) = 512
fstat64(3, {st_mode=S_IFREG|0755, st_size=9604, ...}) = 0
mmap2(NULL, 12408, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb76f7000
mmap2(0xb76f9000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1) = 0xb76f9000
close(3) = 0
open("/lib/libc.so.6", O_RDONLY) = 3
read(3, "\177ELF\1\1\1\3\3\1\20m\1004"..., 512) = 512
fstat64(3, {st_mode=S_IFREG|0755, st_size=1339676, ...}) = 0
mmap2(NULL, 1349928, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb75ad000
mprotect(0xb76f0000, 4096, PROT_NONE) = 0
mmap2(0xb76f1000, 12288, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x143) = 0xb76f1000
mmap2(0xb76f4000, 10536, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xb76f4000
close(3) = 0
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb75ac000
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb75ab000
set_thread_area({entry_number:-1 -> 6, base_addr:0xb75acb20, limit:1048575, seg_32bit:1, contents:0, read_exec_only:0, limit_in_pages:1, seg_not_present:0, useable:1}) = 0
mprotect(0xb76f1000, 8192, PROT_READ) = 0
mprotect(0xb76f9000, 4096, PROT_READ) = 0
mprotect(0xb773b000, 8192, PROT_READ) = 0
mprotect(0x80fd000, 4096, PROT_READ) = 0
mprotect(0xb7780000, 4096, PROT_READ) = 0
munmap(0xb773e000, 141430) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
open("/dev/tty", O_RDWR|O_NONBLOCK|O_LARGEFILE) = 3
close(3) = 0
brk(0) = 0x8108000
brk(0x8129000) = 0x8129000
open("/usr/lib/locale/locale-archive", O_RDONLY|O_LARGEFILE) = 3
fstat64(3, {st_mode=S_IFREG|0644, st_size=1772320, ...}) = 0
mmap2(NULL, 1772320, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb73fa000
close(3) = 0
getuid32() = 0
getgid32() = 0
geteuid32() = 0
getegid32() = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
time(NULL) = 1282454459
open("/proc/meminfo", O_RDONLY) = 3
fstat64(3, {st_mode=S_IFREG|0444, st_size=0, ...}) = 0
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7760000
read(3, "MemTotal: 1027648 kB\nMemFre"..., 1024) = 872
close(3) = 0
munmap(0xb7760000, 4096) = 0
rt_sigaction(SIGCHLD, {SIG_DFL, [], 0}, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGCHLD, {SIG_DFL, [], 0}, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGINT, {SIG_DFL, [], 0}, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGINT, {SIG_DFL, [], 0}, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGQUIT, {SIG_DFL, [], 0}, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGQUIT, {SIG_DFL, [], 0}, {SIG_DFL, [], 0}, 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
rt_sigaction(SIGQUIT, {SIG_IGN, [], 0}, {SIG_DFL, [], 0}, 8) = 0
uname({sys="Linux", node="bhaskar-laptop", ...}) = 0
stat64("/", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
stat64(".", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
getpid() = 18927
open("/usr/lib/gconv/gconv-modules.cache", O_RDONLY) = 3
fstat64(3, {st_mode=S_IFREG|0644, st_size=26048, ...}) = 0
mmap2(NULL, 26048, PROT_READ, MAP_SHARED, 3, 0) = 0xb775a000
close(3) = 0
getppid() = 18926
gettimeofday({1282454459, 509035}, NULL) = 0
getpgrp() = 18926
rt_sigaction(SIGCHLD, {0x8082db0, [], 0}, {SIG_DFL, [], 0}, 8) = 0
getrlimit(RLIMIT_NPROC, {rlim_cur=RLIM_INFINITY, rlim_max=RLIM_INFINITY}) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
open("usr/bin/thunderbird", O_RDONLY|O_LARGEFILE) = 3
ioctl(3, SNDCTL_TMR_TIMEBASE or TCGETS, 0xbfffdd28) = -1 ENOTTY (Inappropriate ioctl for device)
_llseek(3, 0, [0], SEEK_CUR) = 0
read(3, "#!/bin/sh\n#\n# ***** BEGIN LICENS"..., 80) = 80
_llseek(3, 0, [0], SEEK_SET) = 0
getrlimit(RLIMIT_NOFILE, {rlim_cur=1024, rlim_max=1024}) = 0
fcntl64(255, F_GETFD) = -1 EBADF (Bad file descriptor)
dup2(3, 255) = 255
close(3) = 0
fcntl64(255, F_SETFD, FD_CLOEXEC) = 0
fcntl64(255, F_GETFL) = 0x8000 (flags O_RDONLY|O_LARGEFILE)
fstat64(255, {st_mode=S_IFREG|0755, st_size=3943, ...}) = 0
_llseek(255, 0, [0], SEEK_CUR) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
read(255, "#!/bin/sh\n#\n# ***** BEGIN LICENS"..., 3943) = 3943
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
pipe([3, 4]) = 0
rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
rt_sigprocmask(SIG_BLOCK, [INT CHLD], [], 8) = 0
_llseek(255, -1795, [2148], SEEK_CUR) = 0
clone(child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0xb75acb88) = 18928
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
rt_sigaction(SIGCHLD, {0x8082db0, [], 0}, {0x8082db0, [], 0}, 8) = 0
close(4) = 0
read(3, "usr/bin\n", 128) = 8
--- SIGCHLD (Child exited) @ 0 (0) ---
waitpid(-1, [{WIFEXITED(s) && WEXITSTATUS(s) == 0}], WNOHANG) = 18928
waitpid(-1, 0xbfffd6b8, WNOHANG) = -1 ECHILD (No child processes)
sigreturn() = ? (mask now [])
read(3, "", 128) = 0
close(3) = 0
rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
rt_sigaction(SIGINT, {0x80831f0, [], 0}, {SIG_DFL, [], 0}, 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
rt_sigaction(SIGINT, {SIG_DFL, [], 0}, {0x80831f0, [], 0}, 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
read(255, "progbase=`basename \"$progname\"`\n"..., 3943) = 1795
pipe([3, 4]) = 0
rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
rt_sigprocmask(SIG_BLOCK, [INT CHLD], [], 8) = 0
_llseek(255, -1763, [2180], SEEK_CUR) = 0
clone(child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0xb75acb88) = 18929
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
rt_sigaction(SIGCHLD, {0x8082db0, [], 0}, {0x8082db0, [], 0}, 8) = 0
close(4) = 0
read(3, "thunderbird\n", 128) = 12
--- SIGCHLD (Child exited) @ 0 (0) ---
waitpid(-1, [{WIFEXITED(s) && WEXITSTATUS(s) == 0}], WNOHANG) = 18929
waitpid(-1, 0xbfffd6b8, WNOHANG) = -1 ECHILD (No child processes)
sigreturn() = ? (mask now [])
read(3, "", 128) = 0
close(3) = 0
rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
rt_sigaction(SIGINT, {0x80831f0, [], 0}, {SIG_DFL, [], 0}, 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
rt_sigaction(SIGINT, {SIG_DFL, [], 0}, {0x80831f0, [], 0}, 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
read(255, "run_moz=\"$curdir/run-mozilla.sh\""..., 3943) = 1763
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
stat64("usr/bin/run-mozilla.sh", 0xbfffda2c) = -1 ENOENT (No such file or directory)
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
pipe([3, 4]) = 0
rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
rt_sigprocmask(SIG_BLOCK, [INT CHLD], [], 8) = 0
_llseek(255, -1119, [2824], SEEK_CUR) = 0
clone(child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0xb75acb88) = 18930
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
rt_sigaction(SIGCHLD, {0x8082db0, [], 0}, {0x8082db0, [], 0}, 8) = 0
close(4) = 0
read(3, "/\n", 128) = 2
--- SIGCHLD (Child exited) @ 0 (0) ---
waitpid(-1, [{WIFEXITED(s) && WEXITSTATUS(s) == 0}], WNOHANG) = 18930
waitpid(-1, 0xbfffd478, WNOHANG) = -1 ECHILD (No child processes)
sigreturn() = ? (mask now [])
read(3, "", 128) = 0
close(3) = 0
rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
rt_sigaction(SIGINT, {0x80831f0, [], 0}, {SIG_DFL, [], 0}, 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
rt_sigaction(SIGINT, {SIG_DFL, [], 0}, {0x80831f0, [], 0}, 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
lstat64("usr/bin/thunderbird", {st_mode=S_IFLNK|0777, st_size=40, ...}) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
pipe([3, 4]) = 0
rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
rt_sigprocmask(SIG_BLOCK, [INT CHLD], [], 8) = 0
clone(child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0xb75acb88) = 18931
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
--- SIGCHLD (Child exited) @ 0 (0) ---
waitpid(-1, [{WIFEXITED(s) && WEXITSTATUS(s) == 0}], WNOHANG) = 18931
waitpid(-1, 0xbfffcc28, WNOHANG) = -1 ECHILD (No child processes)
sigreturn() = ? (mask now [])
rt_sigaction(SIGCHLD, {0x8082db0, [], 0}, {0x8082db0, [], 0}, 8) = 0
close(4) = 0
read(3, "thunderbird\n", 128) = 12
read(3, "", 128) = 0
close(3) = 0
rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
rt_sigaction(SIGINT, {0x80831f0, [], 0}, {SIG_DFL, [], 0}, 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
rt_sigaction(SIGINT, {SIG_DFL, [], 0}, {0x80831f0, [], 0}, 8) = 0
pipe([3, 4]) = 0
rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
rt_sigprocmask(SIG_BLOCK, [INT CHLD], [], 8) = 0
clone(child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0xb75acb88) = 18932
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
rt_sigaction(SIGCHLD, {0x8082db0, [], 0}, {0x8082db0, [], 0}, 8) = 0
--- SIGCHLD (Child exited) @ 0 (0) ---
waitpid(-1, [{WIFEXITED(s) && WEXITSTATUS(s) == 0}], WNOHANG) = 18932
waitpid(-1, 0xbfffcc88, WNOHANG) = -1 ECHILD (No child processes)
sigreturn() = ? (mask now [])
close(4) = 0
read(3, "usr/bin\n", 128) = 8
read(3, "", 128) = 0
close(3) = 0
rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
rt_sigaction(SIGINT, {0x80831f0, [], 0}, {SIG_DFL, [], 0}, 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
rt_sigaction(SIGINT, {SIG_DFL, [], 0}, {0x80831f0, [], 0}, 8) = 0
stat64("/usr", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
stat64("/usr/bin", {st_mode=S_IFDIR|0755, st_size=69632, ...}) = 0
chdir("/usr/bin") = 0
pipe([3, 4]) = 0
rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
rt_sigprocmask(SIG_BLOCK, [INT CHLD], [], 8) = 0
clone(child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0xb75acb88) = 18933
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
rt_sigaction(SIGCHLD, {0x8082db0, [], 0}, {0x8082db0, [], 0}, 8) = 0
close(4) = 0
read(3, "/usr/bin\n", 128) = 9
--- SIGCHLD (Child exited) @ 0 (0) ---
waitpid(-1, [{WIFEXITED(s) && WEXITSTATUS(s) == 0}], WNOHANG) = 18933
waitpid(-1, 0xbfffcfa8, WNOHANG) = -1 ECHILD (No child processes)
sigreturn() = ? (mask now [])
read(3, "", 128) = 0
close(3) = 0
rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
rt_sigaction(SIGINT, {0x80831f0, [], 0}, {SIG_DFL, [], 0}, 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
rt_sigaction(SIGINT, {SIG_DFL, [], 0}, {0x80831f0, [], 0}, 8) = 0
stat64("/usr", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
stat64("/usr/bin", {st_mode=S_IFDIR|0755, st_size=69632, ...}) = 0
chdir("/usr/bin") = 0
pipe([3, 4]) = 0
rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
rt_sigprocmask(SIG_BLOCK, [INT CHLD], [], 8) = 0
clone(child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0xb75acb88) = 18934
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
rt_sigaction(SIGCHLD, {0x8082db0, [], 0}, {0x8082db0, [], 0}, 8) = 0
close(4) = 0
read(3, "/usr/lib/mozilla-thunderbird/thu"..., 128) = 41
read(3, "", 128) = 0
--- SIGCHLD (Child exited) @ 0 (0) ---
waitpid(-1, [{WIFEXITED(s) && WEXITSTATUS(s) == 0}], WNOHANG) = 18934
waitpid(-1, 0xbfffcf78, WNOHANG) = -1 ECHILD (No child processes)
sigreturn() = ? (mask now [])
close(3) = 0
rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
rt_sigaction(SIGINT, {0x80831f0, [], 0}, {SIG_DFL, [], 0}, 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
rt_sigaction(SIGINT, {SIG_DFL, [], 0}, {0x80831f0, [], 0}, 8) = 0
pipe([3, 4]) = 0
rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
rt_sigprocmask(SIG_BLOCK, [INT CHLD], [], 8) = 0
clone(child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0xb75acb88) = 18937
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
rt_sigaction(SIGCHLD, {0x8082db0, [], 0}, {0x8082db0, [], 0}, 8) = 0
--- SIGCHLD (Child exited) @ 0 (0) ---
waitpid(-1, [{WIFEXITED(s) && WEXITSTATUS(s) == 0}], WNOHANG) = 18937
waitpid(-1, 0xbfffcdd8, WNOHANG) = -1 ECHILD (No child processes)
sigreturn() = ? (mask now [])
close(4) = 0
read(3, "thunderbird\n", 128) = 12
read(3, "", 128) = 0
close(3) = 0
rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
rt_sigaction(SIGINT, {0x80831f0, [], 0}, {SIG_DFL, [], 0}, 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
rt_sigaction(SIGINT, {SIG_DFL, [], 0}, {0x80831f0, [], 0}, 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
stat64("/usr/lib/mozilla-thunderbird/thunderbird", {st_mode=S_IFREG|0755, st_size=3943, ...}) = 0
geteuid32() = 0
getegid32() = 0
getuid32() = 0
getgid32() = 0
access("/usr/lib/mozilla-thunderbird/thunderbird", X_OK) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
pipe([3, 4]) = 0
rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
rt_sigprocmask(SIG_BLOCK, [INT CHLD], [], 8) = 0
clone(child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0xb75acb88) = 18938
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
--- SIGCHLD (Child exited) @ 0 (0) ---
waitpid(-1, [{WIFEXITED(s) && WEXITSTATUS(s) == 0}], WNOHANG) = 18938
waitpid(-1, 0xbfffd018, WNOHANG) = -1 ECHILD (No child processes)
sigreturn() = ? (mask now [])
rt_sigaction(SIGCHLD, {0x8082db0, [], 0}, {0x8082db0, [], 0}, 8) = 0
close(4) = 0
read(3, "/usr/lib/mozilla-thunderbird\n", 128) = 29
read(3, "", 128) = 0
close(3) = 0
rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
rt_sigaction(SIGINT, {0x80831f0, [], 0}, {SIG_DFL, [], 0}, 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
rt_sigaction(SIGINT, {SIG_DFL, [], 0}, {0x80831f0, [], 0}, 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
stat64("/usr/lib/mozilla-thunderbird/run-mozilla.sh", {st_mode=S_IFREG|0755, st_size=10452, ...}) = 0
geteuid32() = 0
getegid32() = 0
getuid32() = 0
getgid32() = 0
access("/usr/lib/mozilla-thunderbird/run-mozilla.sh", X_OK) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
stat64("/usr", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
stat64("/usr/lib", {st_mode=S_IFDIR|0755, st_size=139264, ...}) = 0
stat64("/usr/lib/mozilla-thunderbird", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
chdir("/usr/lib/mozilla-thunderbird") = 0
pipe([3, 4]) = 0
rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
rt_sigprocmask(SIG_BLOCK, [INT CHLD], [], 8) = 0
clone(child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0xb75acb88) = 18939
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
--- SIGCHLD (Child exited) @ 0 (0) ---
waitpid(-1, [{WIFEXITED(s) && WEXITSTATUS(s) == 0}], WNOHANG) = 18939
waitpid(-1, 0xbfffce08, WNOHANG) = -1 ECHILD (No child processes)
sigreturn() = ? (mask now [])
rt_sigaction(SIGCHLD, {0x8082db0, [], 0}, {0x8082db0, [], 0}, 8) = 0
close(4) = 0
read(3, "/usr/lib/mozilla-thunderbird\n", 128) = 29
read(3, "", 128) = 0
close(3) = 0
rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
rt_sigaction(SIGINT, {0x80831f0, [], 0}, {SIG_DFL, [], 0}, 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
rt_sigaction(SIGINT, {SIG_DFL, [], 0}, {0x80831f0, [], 0}, 8) = 0
chdir("/") = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
read(255, "if [ $found = 0 ]; then\n # Chec"..., 3943) = 1119
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
rt_sigprocmask(SIG_BLOCK, [INT CHLD], [], 8) = 0
rt_sigprocmask(SIG_BLOCK, [CHLD], [INT CHLD], 8) = 0
rt_sigprocmask(SIG_SETMASK, [INT CHLD], NULL, 8) = 0
_llseek(255, -35, [3908], SEEK_CUR) = 0
clone(child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0xb75acb88) = 18940
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
rt_sigaction(SIGINT, {0x80831f0, [], 0}, {SIG_DFL, [], 0}, 8) = 0
waitpid(-1, No protocol specified
No protocol specified
No protocol specified
No protocol specified
Error: cannot open display: :0.0
[{WIFEXITED(s) && WEXITSTATUS(s) == 1}], 0) = 18940
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
--- SIGCHLD (Child exited) @ 0 (0) ---
waitpid(-1, 0xbfffd8c8, WNOHANG) = -1 ECHILD (No child processes)
sigreturn() = ? (mask now [])
rt_sigaction(SIGINT, {SIG_DFL, [], 0}, {0x80831f0, [], 0}, 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
read(255, "exitcode=$?\n\nexit $exitcode\n# EO"..., 3943) = 35
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
exit_group(1) = ?

Now if you want investigate the outputed message later then you can call strace with -o parameter with a filename so instead outputing in the terminal it will dump that code in the specified file.Like this:

root@bhaskar-laptop_10:57:34_Sun Aug 22:/ # strace -o thunderbird-error.txt usr/bin/thunderbird

So you can examine that code later from that file thunderbird-error.txt. It will give you lot clues regarding that errors.By the way strace has two cousin named ltrace and ptrace. As you might guess it right name signifies are:
ltrace——–> library trace
ptrace———> process trace.

Pmap:

I am going to share a tool called pmap(process map).What it does??It will basically report about selected process or processes.Say I want to keep track how swiftfox(a variant of firefox) using memory in my system.

bhaskar@bhaskar-laptop_11:22:25_Sun Aug 22:~> ps -ef | grep swiftfox
bhaskar   4593     1  0 07:17 ?        00:00:00 /bin/sh /opt/swiftfox/swiftfox

So got the swiftfox pid .Now we will use that with pmap.

bhaskar@bhaskar-laptop_11:22:31_Sun Aug 22:~> sudo /usr/bin/pmap 4593
4593:   /bin/sh /opt/swiftfox/swiftfox
08047000    668K r-x–  /bin/bash
080ee000     20K rw—  /bin/bash
080f3000     68K rw—    [ anon ]
424d5000    188K r-x–  /lib/libncurses.so.5.7
42504000     12K rw—  /lib/libncurses.so.5.7
b74f0000   1256K r—-  /usr/lib/locale/locale-archive
b762a000      4K rw—    [ anon ]
b762b000   1364K r-x–  /lib/i686/cmov/libc-2.7.so
b7780000      4K r—-  /lib/i686/cmov/libc-2.7.so
b7781000      8K rw—  /lib/i686/cmov/libc-2.7.so
b7783000     16K rw—    [ anon ]
b7787000      8K r-x–  /lib/i686/cmov/libdl-2.7.so
b7789000      8K rw—  /lib/i686/cmov/libdl-2.7.so
b7794000     28K r–s-  /usr/lib/gconv/gconv-modules.cache
b779b000      8K rw—    [ anon ]
b779d000      4K r-x–    [ anon ]
b779e000    104K r-x–  /lib/ld-2.7.so
b77b8000      8K rw—  /lib/ld-2.7.so
bffea000     84K rw—    [ stack ]
total     3860K

Now will get it device wise output with -d flag:

bhaskar@bhaskar-laptop_11:23:52_Sun Aug 22:~> sudo /usr/bin/pmap  -d 4593
4593:   /bin/sh /opt/swiftfox/swiftfox
Address   Kbytes Mode  Offset           Device    Mapping
08047000     668 r-x– 0000000000000000 008:00003 bash
080ee000      20 rw— 00000000000a7000 008:00003 bash
080f3000      68 rw— 00000000080f3000 000:00000   [ anon ]
424d5000     188 r-x– 0000000000000000 008:00003 libncurses.so.5.7
42504000      12 rw— 000000000002f000 008:00003 libncurses.so.5.7
b74f0000    1256 r—- 0000000000000000 008:00003 locale-archive
b762a000       4 rw— 00000000b762a000 000:00000   [ anon ]
b762b000    1364 r-x– 0000000000000000 008:00003 libc-2.7.so
b7780000       4 r—- 0000000000155000 008:00003 libc-2.7.so
b7781000       8 rw— 0000000000156000 008:00003 libc-2.7.so
b7783000      16 rw— 00000000b7783000 000:00000   [ anon ]
b7787000       8 r-x– 0000000000000000 008:00003 libdl-2.7.so
b7789000       8 rw— 0000000000001000 008:00003 libdl-2.7.so
b7794000      28 r–s- 0000000000000000 008:00003 gconv-modules.cache
b779b000       8 rw— 00000000b779b000 000:00000   [ anon ]
b779d000       4 r-x– 00000000b779d000 000:00000   [ anon ]
b779e000     104 r-x– 0000000000000000 008:00003 ld-2.7.so
b77b8000       8 rw— 000000000001a000 008:00003 ld-2.7.so
bffea000      84 rw— 00000000bffeb000 000:00000   [ stack ]
mapped: 3860K    writeable/private: 236K    shared: 28K

Ok..cool now we will get it extended format with -x flag:

bhaskar@bhaskar-laptop_11:25:52_Sun Aug 22:~> sudo /usr/bin/pmap  -x 4593
4593:   /bin/sh /opt/swiftfox/swiftfox
Address   Kbytes     RSS    Anon  Locked Mode   Mapping
08047000     668       –       –       – r-x–  bash
080ee000      20       –       –       – rw—  bash
080f3000      68       –       –       – rw—    [ anon ]
424d5000     188       –       –       – r-x–  libncurses.so.5.7
42504000      12       –       –       – rw—  libncurses.so.5.7
b74f0000    1256       –       –       – r—-  locale-archive
b762a000       4       –       –       – rw—    [ anon ]
b762b000    1364       –       –       – r-x–  libc-2.7.so
b7780000       4       –       –       – r—-  libc-2.7.so
b7781000       8       –       –       – rw—  libc-2.7.so
b7783000      16       –       –       – rw—    [ anon ]
b7787000       8       –       –       – r-x–  libdl-2.7.so
b7789000       8       –       –       – rw—  libdl-2.7.so
b7794000      28       –       –       – r–s-  gconv-modules.cache
b779b000       8       –       –       – rw—    [ anon ]
b779d000       4       –       –       – r-x–    [ anon ]
b779e000     104       –       –       – r-x–  ld-2.7.so
b77b8000       8       –       –       – rw—  ld-2.7.so
bffea000      84       –       –       – rw—    [ stack ]
——– ——- ——- ——- ——-
total kB    3860       –       –       –

IPCS:

Last but not the least this fellow is called “Inter-process Communications“.So the name suggest a lot.If anyone has little bit of programming experience would probably heard of this.But we are not concerned here for that as try to give you a glimpse of it utility to find out little bit system information it can reveal.What it does is communicate between the processes created by the system or the users.

This tool is besically consists of four different things, those are:

  • Pipes – Provides a way for processes to communicate with each another by exchanging messages. Named pipes provide a way for processes running on different computer systems to communicate over the network.
  • Shared Memory – Processes can exchange values in the shared memory. One process will create a portion of memory which other process can access.
  • Message Queue – It is a structured and ordered list of memory segments where processes store or retrieve data.
  • Semaphores – Provides a synchronizing mechanism for processes that are accessing the same resource. No data is passed with a semaphore; it simply coordinates access to shared resources.

Now let me throw out few example of this tool..so you get a better idea how to extract out information.Here we go:

bhaskar@bhaskar-laptop_06:47:26_Mon Aug 23:~> sudo /usr/bin/ipcs

—— Shared Memory Segments ——–
key        shmid      owner      perms      bytes      nattch     status
0x00000000 98304      bhaskar    600        393216     2          dest
0x00000000 131073     bhaskar    600        393216     2          dest
0x00000000 163842     bhaskar    600        393216     2          dest
0x00000000 196611     bhaskar    600        393216     2          dest
0x00000000 229380     bhaskar    600        393216     2          dest
0x00000000 262149     bhaskar    600        393216     2          dest
0x00000000 294918     bhaskar    600        393216     2          dest
0x00000000 327687     bhaskar    600        393216     2          dest
0x00000000 360456     bhaskar    600        393216     2          dest
0x00000000 393225     bhaskar    600        393216     2          dest
0x00000000 491530     bhaskar    600        393216     2          dest
0x00000000 950283     bhaskar    600        3276800    2          dest
0x00000000 557068     bhaskar    600        393216     2          dest
0x00000000 655373     bhaskar    600        936000     2          dest
0x00000000 688142     bhaskar    600        4          2          dest
0xcbc384f8 720911     bhaskar    600        64528      1
0x00000000 753680     bhaskar    600        393216     2          dest

—— Semaphore Arrays ——–
key        semid      owner      perms      nsems
0x00000000 0          http       600        1
0x00000000 32769      http       600        1
0x00000000 65538      http       600        1
0xcbc384f8 163843     bhaskar    600        1

—— Message Queues ——–
key        msqid      owner      perms      used-bytes   messages

So I got everything at once.As the output of the command is pretty readable. Let me see how my system is limited to resources:

bhaskar@bhaskar-laptop_06:49:18_Mon Aug 23:~> sudo /usr/bin/ipcs -l

—— Shared Memory Limits ——–
max number of segments = 4096
max seg size (kbytes) = 32768
max total shared memory (kbytes) = 8388608
min seg size (bytes) = 1

—— Semaphore Limits ——–
max number of arrays = 128
max semaphores per array = 250
max semaphores system wide = 32000
max ops per semop call = 32
semaphore max value = 32767

—— Messages Limits ——–
max queues system wide = 1736
max size of message (bytes) = 8192
default max size of queue (bytes) = 16384

Let me enlist the uid and gid through ipcs:

bhaskar@bhaskar-laptop_06:50:03_Mon Aug 23:~> sudo /usr/bin/ipcs -c

—— Shared Memory Segment Creators/Owners ——–
shmid      perms      cuid       cgid       uid        gid
98304      600        bhaskar    users      bhaskar    users
131073     600        bhaskar    users      bhaskar    users
163842     600        bhaskar    users      bhaskar    users
196611     600        bhaskar    users      bhaskar    users
229380     600        bhaskar    users      bhaskar    users
262149     600        bhaskar    users      bhaskar    users
294918     600        bhaskar    users      bhaskar    users
327687     600        bhaskar    users      bhaskar    users
360456     600        bhaskar    users      bhaskar    users
393225     600        bhaskar    users      bhaskar    users
491530     600        bhaskar    users      bhaskar    users
1146891    600        bhaskar    users      bhaskar    users
557068     600        bhaskar    users      bhaskar    users
655373     600        bhaskar    users      bhaskar    users
688142     600        bhaskar    users      bhaskar    users
720911     600        bhaskar    users      bhaskar    users
753680     600        bhaskar    users      bhaskar    users

—— Semaphore Arrays Creators/Owners ——–
semid      perms      cuid       cgid       uid        gid
0          600        root       root       http       http
32769      600        root       root       http       http
65538      600        root       root       http       http
163843     600        bhaskar    users      bhaskar    users

—— Message Queues Creators/Owners ——–
msqid      perms      cuid       cgid       uid        gid

Let me get status of current usage of ipcs in my system:

bhaskar@bhaskar-laptop_06:52:52_Mon Aug 23:~> sudo /usr/bin/ipcs -u

—— Shared Memory Status ——–
segments allocated 17
pages allocated 2294
pages resident  1912
pages swapped   0
Swap performance: 0 attempts     0 successes

—— Semaphore Status ——–
used arrays = 4
allocated semaphores = 4

—— Messages Status ——–
allocated queues = 0
used headers = 0
used space = 0 bytes

Hope this will help.

Cheers!
Bhaskar

Few tricks and info about sudo

>Working in a multi-admin environment ;where more then one administrator controlling servers,as often the case with most of the big corporates.Then you need a mechanism to deal with that which not allowed each other to overlap their work and keep track who is firing what.Sudo is that kind tool ,which is quite indispensable in the multi-admin production environment.

I do not issue any guarantee that this will work for you.

Most of the GNU/Linux distribution come with sudo..if not then please download it through by it(OSes) package manager. It should be in the repository of that distribution.

Once installed a configuration file related to it placed at /etc named sudoers . So you need edit it according to your requirement to get thing going with this tool.

Tool for to edit that file is called “visudo” ..which nothing but a vi/vim editor with a lock..means when someone editing others won’t allow to do anything in it.Clear?? right.

You need to called it like this:

root@bhaskar-laptop_08:37:05_Thu Aug 19:/home/bhaskar # visudo

and the file /etc/sudoers should open in it,but with a temporary location and place with a lock.

Ok..now few internals entry need to visit for the sake of clarity of it’s function.So here we go:

Suppose we want to allow sudo with some specific host with specific users on it to allow use of sudo.Did I confuse you with the last statement??not worry …I will explain it in details..read on:

The careful reader will note that there was a bit of a change here. The line used to read jim ALL=(ALL) ALL?, but now there’s only one ALL left. Reading the man page can easily leave you quite confused as to what those three ALL??s meant. ALL refers to machines- the assumption is that this is a network wide sudoers file. In the case of this machine (lnxserve) we could do this:

jim lnxserve= /bin/kill, /usr/sbin/jim/

Now let me explain that a host/machine name called “lnxserve” has a user called “jim” and heis entitled to run those two command right side of the “=” .

So what was the (ALL)? for? Well, here;s a clue:

jim lnxserve=(paul,linda) /bin/kill, /usr/sbin/jim/

Yes this line bring another twist into the previous line.Here it says.. a machine called “lnxserv” with a user called “jim” who will be able to run command as paul and linda with specified command mentioned.

That says that jim can (using sudo -u ) run commands as paul or linda. Yes it sometimes necessary to do it because of various reason in the production environment.I not going into that details ,because that might take another whole article to talk about.

This is perfect for giving jim the power to kill paul or linda’s processes without giving him anything else. There is one thing we need to add though: if we just left it like this, jim is forced to use sudo -u paul or sudo -u linda every time. We can add a default runas_default:

Defaults:jim timestamp_timeout=-1, env_delete+=BOOP, runas_default=linda

So jim can easily run command as linda by default.I am going to put some line straight out of the man page for clarity:

To get a file listing of an unreadable directory:

$ sudo ls /usr/local/protected

To list the home directory of user yaz on a machine where the file system holding ~yaz is not exported as root:

$ sudo -u yaz ls ~yaz

To edit the index.html file as user www:

$ sudo -u www vi ~www/htdocs/index.html

To view system logs only accessible to root and users in the adm group:

$ sudo -g adm view /var/log/syslog

To run an editor as jim with a different primary group:

$ sudo -u jim -g audio vi ~jim/sound.txt

To shutdown a machine:

$ sudo shutdown -r +15 “quick reboot”

To make a usage listing of the directories in the /home partition. Note that this runs the commands in a sub-shell to make the cd and file redirection
work.

$ sudo sh -c “cd /home ; du -s * | sort -rn > USAGE”

Hope this will help.

Cheers!
Bhaskar

Extracting information out of your hard disk

In this article I am going to show you the way to get some information out of your hard disk.Sometimes it really necessary to know information about that to measure about the system health.

The tool I am going to use for this purpose is called “hdparm”. If it is not come with your distribution then get it from here . Right.

Say you have SATA drive at your disposal and you want to know the internal matrix of the drive ;then you can do like below:

bhaskar@bhaskar-laptop_08:19:54_Tue Aug 17:~> sudo /sbin/hdparm -i /dev/sda
Password:

/dev/sda:

Model=ST9160821AS, FwRev=3.BHE, SerialNo=5MA8QHLW
Config={ HardSect NotMFM HdSw>15uSec Fixed DTR>10Mbs RotSpdTol>.5% }
RawCHS=16383/16/63, TrkSize=0, SectSize=0, ECCbytes=4
BuffType=unknown, BuffSize=8192kB, MaxMultSect=16, MultSect=16
CurCHS=16383/16/63, CurSects=16514064, LBA=yes, LBAsects=312581808
IORDY=on/off, tPIO={min:120,w/IORDY:120}, tDMA={min:120,rec:120}
PIO modes:  pio0 pio1 pio2 pio3 pio4
DMA modes:  mdma0 mdma1 mdma2
UDMA modes: udma0 udma1 udma2 udma3 udma4 *udma5
AdvancedPM=yes: unknown setting WriteCache=enabled
Drive conforms to: Unspecified:  ATA/ATAPI-1,2,3,4,5,6,7

* signifies the current active mode

OR less verbose way like this:

bhaskar@bhaskar-laptop_08:32:40_Tue Aug 17:~> sudo /sbin/hdparm /dev/sda

/dev/sda:
multcount     = 16 (on)
IO_support    =  1 (32-bit)
readonly      =  0 (off)
readahead     = 256 (on)
geometry      = 19457/255/63, sectors = 312581808, start = 0

Now need to know the speed of your hard drive for caching buffer..so do this:

bhaskar@bhaskar-laptop_08:34:34_Tue Aug 17:~> sudo /sbin/hdparm -T /dev/sda

/dev/sda:
Timing cached reads:   1902 MB in  2.00 seconds = 951.03 MB/sec

Ok.. you can get the information about disk buffer read timing like this:

bhaskar@bhaskar-laptop_08:35:43_Tue Aug 17:~> sudo /sbin/hdparm -t /dev/sda

/dev/sda:
Timing buffered disk reads:  128 MB in  3.02 seconds =  42.45 MB/sec

So this software can take plethora of option to set/get feature related to SATA/IDE  drive .Kindly read the manual page before applying anything on the hard drive otherwise you might run into a catastrophe .So use it with a caution.

Hope this will help.

Cheers!

Bhaskar

How to properly run fsck on (/) root or other partitions including LVM

>As it is an important issue to deal with low level thing in the server archtecture. Being an GNU/Linux administrator/NOC/Ops one has to have the clear cut understanding what they are doing.Because handling the production box require lot of common sense and in depth knowlegde about the platform/OS.

So without much ado lets play with it or let me show you the simple tricks.

I do not issue any guarantee that this will work for you.

So the first question come into the mind why the hell you need to check the filesystem?? Specially the root(/) part of it…sound pretty dull and boring…huh..please don’t ignore this.You know ignorance is a sin…so do not commit it.

Now filesystem can be corrupted in various ways..few common ways are :

1) Not properly shutdown the server(although most of the cases journaling will do the healing)

2) Sudden power cut left your system down with lot of processing going on

3)Somebody has done something special(bad sense) to corrupt the data on that particular partition.

It is a bad idea and not recommended to run fsck(yes,this is the inbuilt tool you need to use)the mounted partition or drive.So don’t do that.

Now, running fsck on other partition like /home,/var,/usr …

First and foremost thing to be done is get into a single user mode..how do you do that?

ok once you type init 1 at the terminal prompt you will be taken to the singe user mode.From there simply unmount the partions as show below:

root@bhaskar-laptop_08:16:36_Mon Aug 16:/home/bhaskar # init 1 —> this will bring to the single user mode

root@bhaskar-laptop_08:16:36_Mon Aug 16:/home/bhaskar # umount /dev/sda2 —> assuming this partion hold the /home section.

Now run the fsck:


root@bhaskar-laptop_08:18:02_Mon Aug 16:/home/bhaskar # fsck -yfv /dev/sda2

Ok..let me explain the flags or switch I passed with the fsck .

y——> it will try to detect and fix any filesystem related corruption without manual intervention.

f———–> this will force check even the system check says it’s clean.

v——–> It will provide you the verbose explanation what that comming going through on the terminal screen.

Now a major problem in our hand. That we find out that root(/)partition of the filesystem gor corrupted due to some reasons.So we need to fix that issue to get back the system as soon as possible on the track.

For this kind of problem..it significant that on a mounted system you just cannot run fsck…as I said earlier..becauase it will corrupt the data on it.So we need a installation cd/dvd for our rescue. The first cd/dvd will do the job for us or get a systemrescuecd to do that.

Once you boot with one of those cd/dvd and put the below text at the command prompt it presents:

#linux rescue nomount

Now once you fire that one you are on the prompt so you can begin work on that.First we need to do is fire a mknod command.Now ask me why need to do that???

Because we had passed the option nomount in the last section so it will not parse any file system or it will not initialize any filesystem or create any device to operate on.If you try to run fsck now it will fail.

So to run correctly the fsck to on a filesystem we need to create device file for that.For that we need to run mknod.But to use mknod we need to know the Major number and Minor number of the device.Lets get those number…wait before that I need to tell you few thing about what Major number and Minor number of a device and how it signifies.

What is Major Number and Minor number??

Traditionally, the major number identifies the driver associated with the device. For example, /dev/null and /dev/zero are both managed by driver 1, whereas virtual consoles and serial terminals are managed by driver 4; similarly, both vcs1 and vcsa1 devices are managed by driver 7. Modern Linux kernels allow multiple drivers to share major numbers, but most devices that you will see are still organized on the one-major-one-driver principle.

The minor number is used by the kernel to determine exactly which device is being referred to. Depending on how your driver is written, you can either get a direct pointer to your device from the kernel, or you can use the minor number yourself as an index into a local array of devices. Either way, the kernel itself knows almost nothing about minor numbers beyond the fact that they refer to devices implemented by your driver.

So it’s clear?? right.lets move on we need to find out the major number and minor number of the device to run mknod:

root@bhaskar-laptop_08:42:30_Mon Aug 16:/home/bhaskar # ls -al /dev/sda
brw-rw—- 1 root disk 8, 0 Aug 16 07:15 /dev/sda

See it will look like this…as 4the and 5th column holds the major number and minor number.Now create the device file:

#mknod /dev/sda b 8 0

It will create the device file.Once it’s done you are safe to run fsck on that particular partition holding your root(/) filesystem.

#fsck -yfv /dev/YourRootPartition(sda,hda,….)

Now lets have some fun with LVM.

We need few tools to manipulate that kind of partition which will provide the lvm package within the os or in come inbuilt with other rescue cd.

We need to find out physical disk,volume group and logical partition ..where we are going to run fsck..right?

pvscan :Physical scanning of particular disk

root@bhaskar-laptop_08:44:06_Mon Aug 16:/home/bhaskar # pvscan
PV /dev/sda8 VG bhaskarlaptop lvm2 [46.15 GiB / 21.15 GiB free]
Total: 1 [46.15 GiB] / in use: 1 [46.15 GiB] / in no VG: 0 [0 ]

vgscan :Volume group scanning

root@bhaskar-laptop_08:50:24_Mon Aug 16:/home/bhaskar # vgscan
Reading all physical volumes. This may take a while…
Found volume group “bhaskarlaptop” using metadata type lvm2

lvscan :Logical volume scanning

root@bhaskar-laptop_08:52:00_Mon Aug 16:/home/bhaskar # lvscan
ACTIVE ‘/dev/bhaskarlaptop/data’ [25.00 GiB] inherit

Now it is not activates then you need to activate the specific logical volume like this:

#lvchange -ay “yourLogicalVolume”

The final step:

Run the fsck on logical volume:

#fsck -yfv /YourLogicalVolume

Hope this will help.

Cheers!
Bhaskar

How to update and upgrade different GNU/Linux distros

As the title suggest and I am going to show you how you could do it yourself..provided you run those distro and manage them. Anyway knowing them will not harm you a bit.

For this article I choose Arch Linux , Fedora , Gentoo , Debian distribution.So step by step I will show you the procedure.

First distribution is Arch Linux:

Because it is a “rolling release” model( Gentoo too!!) so you don’t have to get diffrent thing for every new update made by the developers attached to it’s development.Updating is the same as upgrading as a whole system wide approch. Upgrading to the individual package is very much possible.

Here we go:

bhaskar@bhaskar-laptop_08:36:21_Mon Aug 09:~> sudo pacman -Syu
Password:
:: Synchronizing package databases…
core is up to date
extra is up to date
community is up to date
archlinuxfr is up to date
:: Starting full system upgrade…
there is nothing to do

heck!! why doesn’t it spit out something more?? Because I have the habit of keep my OS updated as much as possible. So I had recently done the update and the OS is very much up to the mark.Now as you can see “pacman” is pacage manager binary to control thing. It consult a file called pacman.conf in the /etc/ directory to work with.Right! now you can see in the output that core,extra,community and archlinuxfr are the repositories defined in the file called pacman.conf. You can add many more if you want. Find out the details for that in the web site.

Second one is Gentoo:

Gentoo has same kind model like Arch i.e rolling release model. But the procedure is quite different.So here we go to show you the way:

root@bhaskar-laptop_09:20:37_Mon Aug 09:/ # sudo eix-sync

This “eix-sync” is wrapper around “emerge –sync” and has the ability to show what has changed since last time.I will not show you the output here because it is quite long.Behind scene it consult with the gentoo repositories.Once it fetched all the updated package from that repositories and   sync with the remote host,which contain those package.Now you need sync your system with the remote or update or upgrade your system with the latest thing available in the repositories.How do you that? here is the procedure to do that:

root@bhaskar-laptop_09:24:22_Mon Aug 09:/ # emerge –ask –verbose –newuse –deep –update world

These are the packages that would be merged, in order:

Calculating dependencies… done!

Total: 0 packages, Size of downloads: 0 kB

Nothing to merge; would you like to auto-clean packages? [Yes/No] y
>>> Auto-cleaning packages…

>>> No outdated packages were found on your system.

Now one more time I am really sorry that I am writing this post little different time. As I have already merge with the world a days back ..so the out put suggest that it has nothing to do.If that were not done …then you can see the listing of packge need to be merge enlisted there and you have select yes for those packages to get into your system.Now a bit of explanation to those flag I passed in the command line.”–ask” this flag will ask you go ahed or not..like I press yes in previous prompt… right? ok next one “ –verbose” is will spit out as much information as possiable to give you an idea what it would be. Third one “–newuse” ok this flag signifies what it means. That is use newer way of using the particular flag which might have been set beforehand or will be set now by default. Forth one “–deep” it signifies that it is going to verify and satisfy all the dependency require to build and installed those packages. Fifth one ” –update” pretty simple hah! you thing right..it will update the existing packages what have already been installed in the system.But how does it find it?? Yes that is why the “world” parameter is there. it is a file reside in the path /var/lib/portage/world. So it will compare with that file and make the changes implemented in your system.

we are not yet done! yup if you are on Gentoo you need to take care of lot of thing by yourself…but believe me it’s will be enjoyable.

Now fire this:

root@bhaskar-laptop_09:41:04_Mon Aug 09:~ # revdep-rebuild
* Configuring search environment for revdep-rebuild

* Checking reverse dependencies
* Packages containing binaries and libraries broken by a package update
* will be emerged.

* Collecting system binaries and libraries

…. output snipped.

This fellow( revdep-rebuild) is healer. Because if the previous update made your system in inconsistent state and libraris are missing ..then this come to your rescue.It was invoked implicitly in the last step ..but one should do it onece more to make sure everything in place.Otherwise you might encounter that lot of program not working as expected.What it does?? it besically scan the system for missing link and broken links then allot those required files to the appropriate package.Then do a ” –oneshot” means it will not keep things recorded in the “world” file.Whatever you installed through emerge it will keep a record in the world fire,so when you update the system it will compare that file with the remote system for version check.Once it run you can find the sanity for your system.Now your system is uptodate and latest as the remote repositoris are.

Third one is Debian:

Aha! completely free OS.Updating is not same as upgrading the system.

Here are the steps:

root@bhaskar-laptop_08:38:24_Mon Aug 09:/ # aptitude update

What it does is get the compare with the repositories with system reserved sate.I f found changed then it will get it and apply it on the system.It will ask your consent before doing that.

Next to upgrade the system :

root@bhaskar-laptop_09:55:53_Mon Aug 09:/ # aptitude safe-upgrade

that is the way to upgrade from the present system to the future system.Like I have updated form Etch to Lenny(lot more steps required).

root@bhaskar-laptop_10:02:36_Mon Aug 09:/ # aptitude dist-upgrade

I think those stated commnd are good enough to get you going on GNU/Linux Debian.

Fourth one is Fedora:

So updating the system is quite diffrent from upgrading the system.First look in how to update.

root@bhaskar-laptop_10:04:03_Mon Aug 09:/ # yum update

It will do the trick for your. What it does that looking for the repo mentioned in the repos.d directory in the /etc/ and get things from there.You can enable thing you want .It will ask your consent to go ahead onece it fetch the packge to install or update.The good thing about “yum” is it will autometically resolve dependencies,which up2date failed to do.Once the system is up to date then fire the following:

root@bhaskar-laptop_10:09:08_Mon Aug 09:/ # yum upgrade

Which will essentially do the upgrade of your system as it enlist all the packges going to changed and instlled and upgraded.

Hope this will help.

Cheers!

Bhaskar