Friday, August 5, 2011

Easy Understanding - Linux Perfomance Tuning 1


Introducing IO Monitoring

Disk IO subsystems are the slowest part of any Linux system. This is due mainly to their distance from the CPU and the fact that disks require the physics to work (rotation and seek). If the time taken to access disk as opposed to memory was converted into minutes and seconds, it is the difference between 7 days and 7 minutes. As a result, it is essential that the Linux kernel minimizes the amount of IO it generates on a disk. The following subsections describe the different ways the kernel processes data IO from disk to memory and back.

Reading and Writing Data Memory Pages

The Linux kernel breaks disk IO into pages. The default page size on most Linux systems is 4K. It reads and writes disk blocks in and out of memory in 4K page sizes. You can check the page size of your system by using the time command in verbose mode and searching for the page size:

# /usr/bin/time -v date
<snip>
Page size (bytes): 4096
<snip>

Major and Minor Page Faults

Sounds confusing to most of the beginners when they read the term “Faults”. In Actual, “Fault” is a command Kernel operation to manage the memory and perform the IO operations. Below is the explanation:

Linux, like most UNIX systems, uses a virtual memory layer that maps into physical address space. This mapping is "on demand" in the sense that when a process starts, the kernel only maps that which is required. When an application starts, the kernel searches the CPU caches and then physical memory. If the data does not exist in either, the kernel issues a major page fault (MPF). A MPF is a request to the disk subsystem to retrieve pages off disk and buffer them in RAM. Once memory pages are mapped into the buffer cache, the kernel will attempt to use these pages resulting in a minor page fault (MnPF). A MnPF saves the kernel time by reusing a page in memory as opposed to placing it back on the disk.

In the following example, the time command is used to demonstrate how many MPF and MnPF occurred when an application started. The first time the application starts, there are many MPFs:

# /usr/bin/time -v evolution
<snip>
Major (requiring I/O) page faults: 163
Minor (reclaiming a frame) page faults: 5918
<snip>

The second time evolution starts, the kernel does not issue any MPFs because the application is in memory already:

# /usr/bin/time -v evolution
<snip>
Major (requiring I/O) page faults: 0
Minor (reclaiming a frame) page faults: 5581
<snip>

The File Buffer Cache

The file buffer cache is used by the kernel to minimize MPFs and maximize MnPFs. As a system generates IO over time, this buffer cache will continue to grow as the system will leave these pages in memory until memory gets low and the kernel needs to "free" some of these pages for other uses. The end result is that many system administrators see low amounts of free memory and become concerned when in reality, the system is just making good use of its caches.

The following output is taken from the /proc/meminfo file:

# cat /proc/meminfo
MemTotal: 2075672 kB
MemFree: 52528 kB
Buffers: 24596 kB
Cached: 1766844 kB
<snip>

The system has a total of 2 GB (MemTotal) of RAM available on it. There is currently 52 MB of RAM "free" (MemFree), 24 MB RAM that is allocated to disk write operations (Buffers), and 1.7 GB of pages read from disk in RAM (Cached). The kernel is using these via the MnPF mechanism as opposed to pulling all of these pages in from disk. It is impossible to tell from these statistics whether or not the system is under distress as we only have part of the picture.

Types of Memory Pages

There are 3 types of memory pages in the Linux kernel. These pages are described below:
  • Read Pages These are pages of data read in via disk (MPF) that are read only and backed on disk. These pages exist in the Buffer Cache and include static files, binaries, and libraries that do not change. The Kernel will continue to page these into memory as it needs them. If memory becomes short, the kernel will "steal" these pages and put them back on the free list causing an application to have to MPF to bring them back in.

  • Dirty Pages These are pages of data that have been modified by the kernel while in memory. These pages need to be synced back to disk at some point using the pdflush daemon. In the event of a memory shortage, kswapd (along with pdflush) will write these pages to disk in order to make more room in memory.


  • Anonymous Pages These are pages of data that do belong to a process, but do not have any file or backing store associated with them. They can't be synchronized back to disk. In the event of a memory shortage, kswapd writes these to the swap device as temporary storage until more RAM is free ("swapping" pages).

Writing Data Pages Back to Disk

Applications themselves may choose to write dirty pages back to disk immediately using the fsync() or sync() system calls. These system calls issue a direct request to the IO scheduler. If an application does not invoke these system calls, the pdflush kernel daemon runs at periodic intervals and writes pages back to disk.

# ps -ef | grep pdflush
root 186 6 0 18:04 ? 00:00:00 [pdflush]

Easy Understanding - Mount and Un-Mount USB

Three easy steps to mount the usb:
1. Connect the usb drive. Create a folder in /mnt folder ( [root@localhost:/]#mkdir /mnt/usbdrive )
2. Execute the command ([root@localhost:/]#mount /dev/sdX /mnt/usbdrive, where sdX is the USB drive file name inside /dev folder). Verify the usb drive mapping by navigating the file /etc/mtab or /mnt/usbdrive.
3. Unmount the usb drive ([root@localhost:/]#umount /mnt/usbdrive).





Easy Understanding - Mount and Un-Mount CDROM

Attaching any filesystem to the root filesystem hierarchy of linux is basically called mounting the filesystem. Here in this blog, i will tell you how to mount the cdrom filesystem on the linux file system hierarchy. Find the easy steps:

1. Insert the CD/DVD into the physical drive
2. Login to the linux system with root privileges (Otherwise execute the command - sudo `mount command`)
3. Create a directory under /mnt or /media 
    [root@localhost:/]# mkdir /mnt/cdrom
4. Mount the cdrom with the following commands
    [root@localhost:/]# mount -t iso9660 /dev/cdrom /mnt/cdrom
5. Navigate to the folder /mnt/cdrom to verify that the drive is mounted.

One can also check the file /etc/mtab to see the disk in mounted. Now, for unmounting.....its fairly easy. Run the following command.
6. Unmount the cdrom
[root@localhost:/]#umount /mnt/cdrom

Verify it from the file /etc/mtab






Inside /proc ( Illusionary Filesystem)

The
/proc filesystem
The /proc filesystem contains a illusionary filesystem. It does not exist on a disk. Instead, the kernel creates  it in memory. It is used to provide information about the system (originally about processes, hence the name). Some of the more important files and directories

/proc/1
A directory with information about process number 1. Each process has a directory below /proc
with the name being its process identification number.

/proc/cpuinfo
Information about the processor, such as its type, make, model, and performance.

/proc/devices
List of device drivers configured into the currently running kernel.

/proc/filesystems
Filesystems configured into the kernel.

/proc/meminfo
Information about memory usage, both physical and swap.

/proc/stat
Various statistics about the system, such as the number of page faults since the system was booted.

/proc/uptime
The time the system has been up.

/proc/version
The kernel version.

Inside /etc ( Plain text configuration file)

The /etc directory
The /etc maintains a lot of files. Some of them are described below. For others, you should determine which program they belong to and read the manual page for that program. Many networking configuration files are in /etc as well

/etc/rc or /etc/rc.d or /etc/rc?.d
Scripts or directories of scripts to run at startup or when changing the run level. See Section 2.3.1 for further information.

/etc/passwd
The user database, with fields giving the username, real name, home directory, and other information about each user. The format is documented in the passwd manual page.

/etc/shadow
/etc/shadow is an encrypted file the holds user passwords.

/etc/fstab
Lists the filesystems mounted automatically at startup by the mount −a command (in /etc/rc or equivalent startup file). Under Linux, also contains information about swap areas used automatically by swapon −a.

/etc/group
Similar to /etc/passwd, but describes groups instead of users.

/etc/inittab
Configuration file for init.

/etc/issue
Output by getty before the login prompt. Usually contains a short description or welcoming message to the system. The contents are up to the system administrator.

/etc/motd
The message of the day, automatically output after a successful login. Contents are up to the system administrator. Often used for getting information to every user, such as warnings about planned downtimes.

/etc/mtab
List of currently mounted filesystems. Initially set up by the bootup scripts, and updated automatically by the mount command. Used when a list of mounted filesystems is needed, e.g., by the df command.

etc/login.defs
Configuration file for the login command.

/etc/profile, /etc/bash.rc, /etc/csh.cshrc
Files executed at login or startup time by the Bourne, BASH , or C shells. These allow the system
administrator to set global defaults for all users. Users can also create individual copies of these in their home directory to personalize their environment. See the manual pages for the respective shells.

/etc/shells
Lists trusted shells. The chsh command allows users to change their login shell only to shells listed in this file. ftpd, is the server process that provides FTP services for a machine, will check that the user's shell is listed in /etc/shells and will not let people log in unless the shell is listed there.

The root filesystem

The root filesystem
The root filesystem should generally be small, since it contains very critical files and a small, infrequently modified filesystem has a better chance of not getting corrupted. A corrupted root filesystem will generally mean that the system becomes unbootable except with special measures (e.g., from a floppy), so you don't want to risk it. The root directory generally doesn't contain any files, except perhaps on older systems where the standard boot image for the system, usually called /vmlinuz was kept there. (Most distributions have moved those files the the /boot directory. Otherwise, all files are kept in subdirectories under the root filesystem:

/bin
Commands needed during bootup that might be used by normal users (probably after bootup).

/sbin
Like /bin, but the commands are not intended for normal users, although they may use them if
necessary and allowed. /sbin is not usually in the default path of normal users, but will be in root's default path.

/etc
Configuration files specific to the machine.

/root
The home directory for user root. This is usually not accessible to other users on the system

/lib
Shared libraries needed by the programs on the root filesystem.

/lib/modules
Loadable kernel modules, especially those that are needed to boot the system when recovering from disasters (e.g., network and filesystem drivers).

/dev
Device files. These are special files that help the user interface with the various devices on the system.

/tmp
Temporary files. As the name suggests, programs running often store temporary files in here.

/boot
Files used by the bootstrap loader, e.g., LILO or GRUB. Kernel images are often kept here instead of in the root directory. If there are many kernel images, the directory can easily grow rather big, and it might be better to keep it in a separate filesystem. Another reason would be to make sure the kernel images are within the first 1024 cylinders of an IDE disk. This 1024 cylinder limit is no longer true in most cases. With modern BIOSes and later versions of LILO (the LInux LOader) the 1024 cylinder limit can be passed with logical block addressing (LBA).

/mnt
Mount point for temporary mounts by the system administrator. Programs aren't supposed to mount on /mnt automatically. /mnt might be divided into subdirectories (e.g., /mnt/dosa might be the floppy drive using an MS−DOS filesystem, and /mnt/exta might be the same with an ext2 filesystem). /proc, /usr, /var, /home. Mount points for the other filesystems. Although /proc does not reside on any disk in reality it is still mentioned here. See the section about /proc later in the chapter.

The Linux File System - 1

How many of you do really understand what is file is all about?. Or why we even need a file system. Well, thats another topic which is out of scope for the current blog. But I must recommed you to visit the website `http://www.pathname.com/fhs/` to get the clean understanding on the file system hierarchy and the standard implementations along with the deviations.

This blog is a loose attemt to through light on the FHS and the Linux directory tree structure standards.
Such a standard has the advantage that it will be easier to write or port software for Linux, and to administer Linux machines, since everything should be in standardized places.


/usr filesystem contains all commands, libraries, manual pages, and other unchanging files/usr should be specific for any given machine, nor/usr). It can make administration easier (only the master /usr needs to be changed when updating an application, not each machine separately) to have /usr network mounted. Even if the filesystem is on a local disk, it could be mounted read−only, to lessen the chance of filesystem corruption during a crash.

The
needed during normal operation. 
/usr filesystem contains all commands, libraries, manual pages, and other unchanging files No files in should they be modified during normal use. This allows the files to be shared over the network, which can be cost−effective since it saves disk space (there can easily be hundreds of megabytes, increasingly multiple gigabytes in
The

The /home filesystem contains the users' home directories, i.e., all the real data on the system.
Separating home directories to their own directory tree or filesystem makes backups easier; the other
parts often do not have to be backed up, or at least not as often as they seldom change. A big /home
might have to be broken across several filesystems, which requires adding an extra naming level
below /home, for example /home/students and /home/staff.
/var filesystem contains files that change, such as spool directories (for mail, news, printers,etc), log files, formatted manual pages, and temporary files. Traditionally everything in been somewhere below /usr , but that made it impossible to mount /usr read−only.

The root filesystem is specific for each machine (it is generally stored on a local disk, although it
could be a ramdisk or network drive as well) and contains the files that are necessary for booting the
system up, and to bring it up to such a state that the other filesystems may be mounted. The contents
of the root filesystem will therefore be sufficient for the single user state. It will also contain tools for
fixing a broken system, and for recovering lost files from backups.