Easy Understanding - DNS Server Configuration 11

I found this article very helpful in creating DNS infra for my client. Please follow the following Videos. I will also release the sample config file for the DNS BIND config in the next post.

Setting Up and Configuring a DNS Server in Ubuntu 10.10 Maverick Meerkat - Part 1


Setting Up and Configuring a DNS Server in Ubuntu 10.10 Maverick Meerkat - Part 2


Setting Up and Configuring a DNS Server in Ubuntu 10.10 Maverick Meerkat - Part 3


Setting Up and Configuring a DNS Server in Ubuntu 10.10 Maverick Meerkat - Part 4


Please note - This tutorial doesn't tells you anything about the master and slave DNS setup. It will be discussed in the next posts.

Easy Understanding - Ubuntu Run Levels

Well Guys...As stated...I have been working to make a complete document on Ubuntu HA-NLB cluster setup and publish my work on Internet...Please expect some delay in that due to some conflict with the HA-NLB software(Ultramonkey) and Ubuntu.

Recently I thought of giving you the details of how to control and change the run-levels in Ubuntu. Please read the below article.

Linux run levels are numbered 0 through 6. Run levels stop at six for practical and historical reasons, but it is entirely possible to have more if desired.

The following table summarizes the User Linux run levels:

* 0 System Halt
* 1 Single user
* 2 Full multi-user mode (Default)
* 3-5 Same as 2
* 6 System Reboot

Run level 0 is the system halt condition. Nearly all modern X86 computers will power off automatically when run level 0 is reached. Older X86 computers, and various different architectures will remain powered on and display a message referring to the halt condition.

Run Level 1 is known as ‘single user’ mode. A more apt description would be ‘rescue’, or ‘trouble-shooting’ mode. In run level 1, no daemons (services) are started. Hopefully single user mode will allow you to fix whatever made the transition to rescue mode necessary.http://www.blogger.com/post-create.g?blogID=7413783897596092588

(You can boot into single user mode typically by using your boot loader, lilo or grub, to add the word ‘single’ to the end of the kernel command line).

Run levels 2 through 5 are full multi-user mode and are the same in a default User Linux (Debian) system. It is a common practice in other Linux distributions to use run level 3 for a text console login and run level 5 for a graphical login.Note, run level 3 is for CLI and run level 5 is for GUI(Gueei)

Run level 6 is used to signal system reboot. This is just like run level 0 except a reboot is issued at the end of the sequence instead of a power off.

Run Levels Location
/etc/rc0.d Run level 0
/etc/rc1.d Run level 1
/etc/rc2.d Run level 2
/etc/rc3.d Run level 3
/etc/rc4.d Run level 4
/etc/rc5.d Run level 5
/etc/rc6.d Run level 6

Change run levels in Debian and Ubuntu Linux

Before we change run levels it might help to find out which run level is current. Use the ‘runlevel’ command to tell you two things: The last run level, and the current run level. Here is the command and the output shown together due to the sparsity of the output:

[root@localhost~:]# runlevel
N 2

The ‘N’ stands for none, meaning there has been no run level change since powering up.The primary command used to change run levels is ‘telinit’.

[root@localhost~:]# telinit 3
[root@localhost~:]# runlevel
N 3

telinit takes one argument on the command line. As always, see the man page for full details. Normally the argument will be one of: 0,1,2,3,4,5,6, or the letter ‘S’. As you may have guessed, the numbers correspond to the run level you wish to move to. Using the ‘S’, for single-user, is the same as the number 1, but don’t do it; the ‘S’ runlevel is intended for use by the UserLinux (Debian)system.

Things does not stop here. We keep on looking for the file where we can change the default run-level..do we??. So here it is::
[root@localhost~:]# nano /etc/init/rc-sysinit.conf

Now reboot the system and the new default run level is 3.

Cheers :)

Ultra Monkey - Coming Soon...

Linux HA Cluster - Steps Approach

Easy Understanding - Bind multiple IP to a single NIC

Here we go....aaa..aaa. But why we even need this....:)

That's not the topic here. I am going to explain how to bind multiple ip to a single nic.

Suppose we have nic eth0 and we bind the IP Address as:

No we want to bind the second IP with this NIC. We do it as:

To verify if the binding works fine....kindly execute the following command:
[root@localhost~:]#ip addr sh eth0

Your output is:

Easy Understanding - Rename the network device

Today I needed to change / rename network interface name ‘eth1′ to ‘eth0′, after replacing broken network card. I searched for the file ‘/etc/iftab’, and it was not there.

The solution lies in the file ‘/etc/udev/rules.d/70-persistent-net.rules’:
# PCI device 0x1022:0x2000 (pcnet32)
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="00:0b:49:81:02:13", ATTR{type}=="1", KERNEL=="eth*", NAME="eth1"

Changing NAME=”eth1″ to NAME=”eth0″ did the trick.

Easy Understanding - DHCP Server Configuration

Wow!!! One of my all time favorite server services. I simply love it.

DHCP Server in easy steps:
Step 1: Download the binaries and the isc dhcp package from the ubuntu universe.
[root@localhost~:]#apt-get install dhcp3-server
This will install the dhcp server as a service in the ubuntu server. Once installed, you can verify the dhcp service as:
[root@localhost~:]#/etc/init.d/isc-dhcp-server

Step 2: Edit the file dhcpd.conf.
[root@localhost~:]#vi /etc/dhcp/dhcpd.conf

This image is self explanatory. Apart from this, there are some other information which needs to be configured. Please find the snap.

Are we done.....oops I bind specify which ethernet card will server the dhcp server request to the client in a multi-homed nic environment. Here it is:

Edit the file /etc/default/isc-dhcp-server

Step 3: Restart the dhcp server daemon.
[root@localhost~:]#/etc/init.d/isc-dhcp-server restart

Cheers

Security Series - Password Aging!!!

Did I say "password aging"? Yes I did. There is a counter which reads from a config file...what??? About the password aging information. The next question which strikes the mind is - which file?, what info?, does that file really exist???.

Here's your answers:
Characteristics of a Strong Password:
--> Should have minimum character defined
--> Should be a combination of characters(including special characters) and digits
--> Should have expiration date.

You can view the password aging information of a particular user by executing the following command.

#chage -l username

The password expire information can be set with the following command.
#chage username

for example:
sudo chage -E 01/31/2011 -m 5 -M 90 -I 30 -W 14 username

which means - date (-E) to 01/31/2008, minimum password age (-m) of 5 days, maximum password age (-M) of 90 days, inactivity period (-I) of 5 days after password expiration, and a warning time period (-W) of 14 days before password expiration.

Cheers

Security Series - Password Password Password!!!

A strong password policy is one of the most important aspects of your security posture. Many successful security breaches involve simple brute force and dictionary attacks against weak passwords. If you intend to offer any form of remote access involving your local password system, make sure you adequately address minimum password complexity requirements, maximum password lifetimes, and frequent audits of your authentication systems.

Minimum Password Lenght:
By default, Ubuntu requires a minimum password length of 6 characters, as well as some basic entropy checks. These values are controlled in the file /etc/pam.d/common-password, which is outlined below.

password [success=2 default=ignore] pam_unix.so obscure sha512

If you would like to adjust the minimum length to 8 characters, change the appropriate variable to min=8. The modification is outlined below.

password [success=2 default=ignore] pam_unix.so obscure sha512 min=8

Security Series - Where's the "root"?

In Ubuntu, root user account is locked by default. This is done to harden the security features. You can always enable it by using the following command:
#passwd root

Set the password of the user, logout and again login with root user.

Excuse me!!!....where the admin account list in ubuntu.

:) You can find the list of the sudoers in the file /etc/sudoers.
I used to edited it whenever I need some new users (ssh/ftp) to have sudo rights.

Easy Understanding - Changing hostname in Linux(Ubuntu)

Hey Guys!!!....this is a common practice when you are testing some applications like bind to configure the DNS Server on your system. Let me provide you some simple steps to change the hostname of the running ubuntu and relative linux distribution.

[root@localhost~:]#hostname new_name
This will set the hostname as new_name. But this gets washed out as soon as the system reboots. So we need to find a permanent fix.... :)

Edit the following files to hardcode the hostname.
[root@localhost~:]#nano /etc/hostname
[root@localhost~:]#nano /etc/hosts

Cheers

Easy Understanding - DNS Server Configuration

DNS Server, nothing but a distributed light weight database which performs name resouleion as per the RFC1034 RFC.

Please find the easy steps to install and configure the DNS Server on the Ubuntu System
Step 1:
Install Ubuntu dapper, or use your WORKING installation.
[root@localhost~:]#apt-get install bind9

Install dnsutils (which includes "dig")
[root@localhost~:]#apt-get install dnsutils

Step 2:
Confguration of the DNS Server - There a many ways to configure BIND9. Some of the most common configurations are a
Caching nameserver
Primary master(pdns) and
Secondary master(sdns)


Caching Server - When configured as a caching nameserver BIND9 will find the answer to name queries and remember the answer when the domain is queried again.

Primary Master Server - As a primary master server BIND9 reads the data for a zone from a file on it's host and is authoritative for that zone.

Secondary Master Server - In a secondary master configuration BIND9 gets the zone data from another nameserver authoritative for the zone.

Insight - An overview of the bind server.
The DNS configuration files are stored in the /etc/bind directory. The important files in this directory are:
/etc/bind/named.conf - The primary configuration file
[root@localhost~:]#cat /etc/bind/named.conf

The include line specifies the filename which contains the DNS options. The directory line in the /etc/bind/named.conf.options file tells DNS where to look for files. All files BIND uses will be relative to this directory.
/etc/bind/db.root - the root nameservers in the world

Caching Nameserver
The default configuration is setup to act as a caching server. All that is required is simply adding the IP Addresses of your ISP's DNS servers. Simply uncomment and edit the following in /etc/bind/named.conf.options:

Now restart the DNS server, to enable the new configuration. From a terminal prompt:
[root@localhost~:]#/etc/init.d/bind9 restart

Primary Master
In this section BIND9 will be configured as the Primary Master for the domain example.com. Simply replace example.com with your FQDN (Fully Qualified Domain Name).

Forward Zone File
To add a DNS zone to BIND9, turning BIND9 into a Primary Master server, the first step is to edit /etc/bind/named.conf.local:

zone "server.local" {
type master;
file "/etc/bind/db.server.local";
};

Now use an existing zone file as a template to create the /etc/bind/db.server.local file:

[root@localhost~:]#cp /etc/bind/db.local /etc/bind/db.server.local

Edit the new zone file /etc/bind/db.server.local change localhost. to the FQDN of your server, leaving the additional "." at the end. Change 127.0.0.1 to the nameserver's IP Address and root.localhost to a valid email address, but with a "." instead of the usual "@" symbol, again leaving the "." at the end.

Also, create an A record for ns.example.com. The name server in this example:

;
; BIND data file for local loopback interface
;
$TTL 604800
@ IN SOA ns.server.local. root.server.local. (
1 ; Serial
604800 ; Refresh
86400 ; Retry
2419200 ; Expire
604800 ) ; Negative Cache TTL
;
@ IN NS ns.server.local.
@ IN A 192.168.1.10
ns IN A 192.168.1.10


Reverse Zone File
Now that the zone is setup and resolving names to IP Adresses a Reverse zone is also required. A Reverse zone allows DNS to resolve an address to a name.

Edit /etc/bind/named.conf.local and add the following:

zone "1.168.192.in-addr.arpa" {
type master;
notify no;
file "/etc/bind/db.192";
};


Now create the /etc/bind/db.192 file:

[root@localhost~:]#cp /etc/bind/db.127 /etc/bind/db.192
Next edit /etc/bind/db.192 changing the basically the same options as /etc/bind/db.server.local:

;
; BIND reverse data file for local loopback interface
;
$TTL 604800
@ IN SOA ns.server.local. root.server.local. (
2 ; Serial
604800 ; Refresh
86400 ; Retry
2419200 ; Expire
604800 ) ; Negative Cache TTL
;
@ IN NS ns.
10 IN PTR ns.server.local.



Restart the dns server
[root@localhost~:]#/etc/init.d/bind9 restart

Secondary Master
Once a Primary Master has been configured a Secondary Master is needed in order to maintain the availability of the domain should the Primary become unavailable.

First, on the Primary Master server, the zone transfer needs to be allowed. Add the allow-transfer option to the example Forward and Reverse zone definitions in /etc/bind/named.conf.local:

zone "server.local" {
type master;
file "/etc/bind/db.server.local";
allow-transfer { 192.168.1.11; };
};

zone "1.168.192.in-addr.arpa" {
type master;
notify no;
file "/etc/bind/db.192";
allow-transfer { 192.168.1.11; };
};


Next, on the Secondary Master, install the bind9 package the same way as on the Primary. Then edit the /etc/bind/named.conf.local and add the following declarations for the Forward and Reverse zones:

zone "server.local" {
type slave;
file "db.server.local";
masters { 192.168.1.10; };
};

zone "1.168.192.in-addr.arpa" {
type slave;
file "db.192";
masters { 192.168.1.10; };
};

Restart BIND9 on the Secondary Master:

[root@localhost~:]#/etc/init.d/bind9 restart


Done.... :)

Cheers..

vsftp - important steps

Login to the Server with root privileges and execute the command:
[roo@localhost~:]#apt-get install vsftd

This will install the ftp server and start the ftp daemon on your server. Execute the command to check the ftp daemon status

[roo@localhost~:]#/etc/init.d/vsftpd status # Check the ftp daemon status
[roo@localhost~:]#/etc/init.d/vsftpd start # Start the ftp daemon
[roo@localhost~:]#/etc/init.d/vsftpd stop # Stop the ftp daemon

Edit the vsftpd.conf file to configure
[roo@localhost~:]#nano /etc/vsftpd.conf # Edit the ftp configuration file

Uncomment the following line:
Consider changing anonymous_enable=YES to anonymous_enable=NO #unable/disable anons

write_enable=YES # un-comment this
local_enable=YES # un-comment this
chroot_local_user=YES # un-comment this

save and exit. Start the ftp daemon;
[roo@localhost~:]#/etc/init.d/vsftpd start

Please note: You can create the ftp user (which are just like normal user) with the following condition.
1. Add /bin/false shell to the /etc/shells file.
2. Create the user with shell as /bin/false (verify in /etc/passwd)
3. If you want the user not to have the access to the ftp resource, edit the /etc/ftpuser and put the user in this list.

The Very Secure FTP Server - vsftp

File Transfer Protocol (FTP) is a TCP protocol for uploading and downloading files between computers. FTP works on a client/server model. The server component is called an FTP daemon. It continuously listens for FTP requests from remote clients. When a request is received, it manages the login and sets up the connection. For the duration of the session it executes any of commands sent by the FTP client.

Access to an FTP server can be managed in two ways:

1. Anonymous
2. Authenticated

In the Anonymous mode, remote clients can access the FTP server by using the default user account called "anonymous" or "ftp" and sending an email address as the password. In the Authenticated mode a user must have an account and a password. User access to the FTP server directories and files is dependent on the permissions defined for the account used at login. As a general rule, the FTP daemon will hide the root directory of the FTP server and change it to the FTP Home directory. This hides the rest of the file system from remote sessions.

vsftpd - FTP Server Installation

vsftpd is an FTP daemon available in Ubuntu. It is easy to install, set up, and maintain. To install vsftpd you can run the following command:
[root@localhost~:]#apt-get install vsftpd

Anonymous FTP Configuration

By default vsftpd is configured to only allow anonymous download. During installation a ftp user is created with a home directory of /home/ftp. This is the default FTP directory.

If you wish to change this location, to /srv/ftp for example, simply create a directory in another location and change the ftp user's home directory:
[root@localhost~:]#mkdir /srv/ftp
[root@localhost~:]#usermod -d /srv/ftp ftp

User Authenticated FTP Configuration
To configure vsftpd to authenticate system users and allow them to upload files edit /etc/vsftpd.conf:

local_enable=YES
write_enable=YES

Securing FTP

There are options in /etc/vsftpd.conf to help make vsftpd more secure. For example users can be limited to their home directories by uncommenting:
chroot_local_user=YES

You can also limit a specific list of users to just their home directories:

chroot_list_enable=YES
chroot_list_file=/etc/vsftpd.chroot_list

After uncommenting the above options, create a /etc/vsftpd.chroot_list containing a list of users one per line. Then restart vsftpd:

[root@localhost~:]#/etc/init.d/vsftpd restart

Also, the /etc/ftpusers file is a list of users that are disallowed FTP access. The default list includes root, daemon, nobody, etc. To disable FTP access for additional users simply add them to the list.

Issssshhhhh....I mistakenly deleted my interface file

Guys..it command that when we work with root privilege, we tend to make mistake. And the most common mistake is deleting the files. I encountered a situation where I was configuring my DNS Server, and mistakenly I deleted the interface file. I thought, I am gone. Since it was a mission critical server.But quickly I realized, let me create one manually and see if it works. And guess what.....It really worked. I never though, I am so witty :).

[root@localhost~:]#nano /etc/networking/interfaces

Add the following line:

iface eth0 inet dhcp

and exit the editor saving the file.

[root@localhost~:]#/etc/init.d/networking restart

Easy Understanding - Setting the Static Network Configuration

It is not intuitively obvious how to assign Ubuntu 10.04 Lucid Lynx a a static IP address from the command line. However, much of Linux administration involves the editing of text files, and assigning a static IP address is no different. You’ll need to edit the following file:

[root@localhost~:]#/etc/network/interfaces

Initially, the file only contains information about your local loopback address:

auto lo
iface lo inet loopback

To assign a static IP address, you’ll need to make some changes to this file.

Let’s say you want to assign a static IP of 192.168.1.2 to your eth0 network connection (the first Ethernet adapter on your system; if you only have one, it will be eth0), with a subnet mask of 255.255.255.0 and a local gateway of 192.168.1.1. First, make a backup copy of the interfaces file:

[root@localhost~:]#cp /etc/network/interfaces /etc/network/interfaces.bkp

This will make a backup copy in your home directory in case something goes amiss during the editing process. Next, fire up a text editor:

[root@localhost~:]#nano /etc/network/interfaces

Once the file is open, add the following lines:

iface eth0 inet static
address 192.168.1.2
netmask 255.255.252.0
gateway 192.168.1.1

Once you’ve added these lines, save the interfaces file to disk, and exit your text editor. If you want to add a static DNS server, you’ll need to edit the /etc/resolv.conf file with this command:

[root@localhost~:]#nano /etc/resolv.conf

To set a static DNS server with the address of 192.168.1.10, add this line to the file:

nameserver 192.168.1.10

Save the file, and exit your text editor.

You’ll then to need have your system load the new IP configuration. You can do that by rebooting, but if that takes too long, you can use this command to force Ubuntu to re-read the configuration files:
[root@localhost~:]#ifup eth0

Your system will then have a static IP address.

Hey Dude !!! Look into the CPU utilization of the Linux box

Bottleneck....aaahhh. Its a pain for the system administrators.

Whenever a Linux system CPU is occupied by a process, it is unavailable for processing other requests. Rest of pending requests must wait till CPU is free. This becomes a bottleneck in the system. Following command will help you to identify CPU utilization, so that you can troubleshoot CPU related performance problems.
Finding CPU utilization is one of the important tasks. Linux comes with various utilities to report CPU utilization. With these commands, you will be able to find out:
* CPU utilization
* Display the utilization of each CPU individually (SMP cpu)
* Find out your system's average CPU utilization since the last reboot etc
* Determine which process is eating the CPU(s)
We can make use of the tools like mpstat and iostat to more accurately find the issues related to the cpu overload. But before we make use of these utilities, we need to download it from the repos. Kind following the following steps in order to download it.

[root@localhost~:]#apt-get update
[root@localhost~:]#apt-get install sysstat

This will download the utils and put in the default folder (here I presumed that it’s a ubuntu system).

What next…Let’s use the command:
[root@localhost~:]#mpstat
Linux 2.4.27-1-686-smp (test) 16/11/05

09:05:11 CPU %user %nice %system %iowait %irq %soft %idle intr/s

09:05:11 all 0.01 0.00 0.03 0.00 0.00 0.00 99.96 106.03

Commands To Remember - "dmesg"

One of the most widely used command to retrieve the information from the kernel buffer. I will illustrate this with an example - I added a network card, but when I am running the command to check the NIC config, i found that the NIC is not listed there. Find the snap:
[root@localhost~:]#ifconfig

Surprised...I were. Where is my NIC card. Its not listed. I checked the configuration of the system...but of no use. I was not able to make up where's the error. Then I ran the command:
[root@localhost~:]#ifconfig -a

I found that the network card is listed as `eth1`. Then i realized, due to some issue, I renamed my card from eth0 to eth1. I check the file /etc/network/interface and found that the name of the interface is eth1.

Again, I executed the command,
[root@localhost~:]#dmesg | grep -i network

And heres the output:

Commands To Remember - "lshw"

The `lshw` is among the most important command in the linux system. This command is comparatively better than `ioscan` command of HP-UX as its comes with lot many switches and the output can be formatted in numerous ways.

For example, when we run the command
[root@localhost~:]#lshw -short

It system in the hierarchical way.Out put is similar to `ioscan`

To check the network devices the the various parameters:
[root@localhost~:]#lshw -C network

[root@localhost~:]#lshw -C disk

Logout Users in Linux

Yet another newbie question that suggests people love to kill and show their power to rest of the world ;)

There is a package called procps. It includes various useful (nifty) utilities. One of such utility is skill which is responsible to send a signal to users and process such as:

Halt user terminal
Kill user and logout

The procps package contains utilities to browse the /proc filesystem, which is not a real file system but a way for the kernel to provide information about the status of entries in its process table. Procps includes ps, free, skill, pkill, pgrep, snice, tload, top, uptime, vmstat, w, watch and pdwx commands.

Task: How To Halt/Stop a User Called vivek
Open a command-line terminal (select Applications > Accessories > Terminal), and then type the following commands. First, switch to the root user by typing su - and entering the root password, when prompted (you can also use sudo if configured). Type the skill command as follows:

#ps -eaf | grep username

This will help you find the PID of the username. Next

#kill -s 9 PID

This will kill the interactive session of the user.

Easy Understanding - Linux Performance Tuning 11

Certain conditions occur on a system that create IO bottlenecks. These conditions may be identified by using a standard set of system monitoring tools. These tools include top, vmstat, iostat, and sar. There are some similarities between the output of these commands, but for the most part, each offers a unique set of output that provides a different aspect on performance. The following subsections describe conditions that cause IO bottlenecks.

Condition 1: CPU Wait on IO Too Much IO at Once

In an ideal environment, a CPU splits  a percentage of its time between user (65%), kernel (30%) and idle (5%). If IO starts to cause the bottleneck on the system, a new condition “Wait on IO (WIO)” appears in the CPU performance statistics. A WIO condition occurs when a CPU is completely idle because all the runnable processes are waiting on IO. This meant that all the applications are in a sleep state because they are waiting for requests to complete in the IO subsystems.

The vmstat command provides WIO statistics in the last 4 fields of output under the “cpu” header.

# vmstat 1

procs ----memory---------- ---swap-- -----io---- --system-- ----cpu----

r b swpd free  buff cache   si so bi    bo  in   cs   us sy id wa

3 2 0    55452 9236 1739020 0  0  9352  0   2580 8771 20 24 0  57

2 3 0    53888 9232 1740836 0  0  14860 0   2642 8954 23 25 0  52

2 2 0    51856 9212 1742928 0  0  12688 0   2636 8487 23 25 0  52

These last 4 columns provide percentages of CPU utilization for user (us), kernel (sys), idle (id), and WIO (wa). In the previous output, the CPU averages 50% idle waiting on IO requests to complete. This means that there is 50% of the processor that is usable for executing applications, but no applications can execute because the kernel is waiting on IO requests to complete. You can observe this in the blocked threads column (b).

It is also worth noting that the major cause of the IO bottleneck is disk reads due to the large amount of disk blocks read into memory (bi). There is no data being written out to disk as the blocks out (bo) column has a zero value. From this output alone, it appears that the system is processing a large IO request.

FIELD DESCRIPTIONS

Procs

r: The number of processes waiting for run time.

b: The number of processes in uninterruptable sleep.

w: The number of processes swapped out but otherwise runnable.

This field is calculated, but Linux never desperation swaps.

Memory

swpd: the amount of virtual memory used (kB).

free: the amount of idle memory (kB).

buff: the amount of memory used as buffers (kB).

Swap

si: Amount of memory swapped in from disk (kB/s).

so: Amount of memory swapped to disk (kB/s).

IO

bi: Blocks sent to a block device (blocks/s).

bo: Blocks received from a block device (blocks/s).

System

in: The number of interrupts per second, including the clock.

cs: The number of context switches per second.

CPU

These are percentages of total CPU time.

us: user time

sy: system time

id: idle time

The top tool can provide enough insight to make an educated guess. Start the top command with a delay of 1 second:

# top -d 1

Once top is running, sort the output by faults (MPF and MnPF) by typing "F" to bring up the sort menu and "u" to sort by faults.

Easy Understanding - Linux Perfomance Tuning 1

Introducing IO Monitoring

Disk IO subsystems are the slowest part of any Linux system. This is due mainly to their distance from the CPU and the fact that disks require the physics to work (rotation and seek). If the time taken to access disk as opposed to memory was converted into minutes and seconds, it is the difference between 7 days and 7 minutes. As a result, it is essential that the Linux kernel minimizes the amount of IO it generates on a disk. The following subsections describe the different ways the kernel processes data IO from disk to memory and back.

Reading and Writing Data Memory Pages

The Linux kernel breaks disk IO into pages. The default page size on most Linux systems is 4K. It reads and writes disk blocks in and out of memory in 4K page sizes. You can check the page size of your system by using the time command in verbose mode and searching for the page size:

# /usr/bin/time -v date

<snip>

Page size (bytes): 4096

<snip>

Major and Minor Page Faults

Sounds confusing to most of the beginners when they read the term “Faults”. In Actual, “Fault” is a command Kernel operation to manage the memory and perform the IO operations. Below is the explanation:

Linux, like most UNIX systems, uses a virtual memory layer that maps into physical address space. This mapping is "on demand" in the sense that when a process starts, the kernel only maps that which is required. When an application starts, the kernel searches the CPU caches and then physical memory. If the data does not exist in either, the kernel issues a major page fault (MPF). A MPF is a request to the disk subsystem to retrieve pages off disk and buffer them in RAM. Once memory pages are mapped into the buffer cache, the kernel will attempt to use these pages resulting in a minor page fault (MnPF). A MnPF saves the kernel time by reusing a page in memory as opposed to placing it back on the disk.

In the following example, the time command is used to demonstrate how many MPF and MnPF occurred when an application started. The first time the application starts, there are many MPFs:

# /usr/bin/time -v evolution

<snip>

Major (requiring I/O) page faults: 163

Minor (reclaiming a frame) page faults: 5918

<snip>

The second time evolution starts, the kernel does not issue any MPFs because the application is in memory already:

# /usr/bin/time -v evolution

<snip>

Major (requiring I/O) page faults: 0

Minor (reclaiming a frame) page faults: 5581

<snip>

The File Buffer Cache

The file buffer cache is used by the kernel to minimize MPFs and maximize MnPFs. As a system generates IO over time, this buffer cache will continue to grow as the system will leave these pages in memory until memory gets low and the kernel needs to "free" some of these pages for other uses. The end result is that many system administrators see low amounts of free memory and become concerned when in reality, the system is just making good use of its caches.

The following output is taken from the /proc/meminfo file:

# cat /proc/meminfo

MemTotal: 2075672 kB

MemFree: 52528 kB

Buffers: 24596 kB

Cached: 1766844 kB

<snip>

The system has a total of 2 GB (MemTotal) of RAM available on it. There is currently 52 MB of RAM "free" (MemFree), 24 MB RAM that is allocated to disk write operations (Buffers), and 1.7 GB of pages read from disk in RAM (Cached). The kernel is using these via the MnPF mechanism as opposed to pulling all of these pages in from disk. It is impossible to tell from these statistics whether or not the system is under distress as we only have part of the picture.

Types of Memory Pages

There are 3 types of memory pages in the Linux kernel. These pages are described below:

• Read Pages These are pages of data read in via disk (MPF) that are read only and backed on disk. These pages exist in the Buffer Cache and include static files, binaries, and libraries that do not change. The Kernel will continue to page these into memory as it needs them. If memory becomes short, the kernel will "steal" these pages and put them back on the free list causing an application to have to MPF to bring them back in.

• Dirty Pages These are pages of data that have been modified by the kernel while in memory. These pages need to be synced back to disk at some point using the pdflush daemon. In the event of a memory shortage, kswapd (along with pdflush) will write these pages to disk in order to make more room in memory.

• Anonymous Pages These are pages of data that do belong to a process, but do not have any file or backing store associated with them. They can't be synchronized back to disk. In the event of a memory shortage, kswapd writes these to the swap device as temporary storage until more RAM is free ("swapping" pages).

Writing Data Pages Back to Disk

Applications themselves may choose to write dirty pages back to disk immediately using the fsync() or sync() system calls. These system calls issue a direct request to the IO scheduler. If an application does not invoke these system calls, the pdflush kernel daemon runs at periodic intervals and writes pages back to disk.

# ps -ef | grep pdflush

root 186 6 0 18:04 ? 00:00:00 [pdflush]

Easy Understanding - Mount and Un-Mount USB

Three easy steps to mount the usb:
1. Connect the usb drive. Create a folder in /mnt folder ( [root@localhost:/]#mkdir /mnt/usbdrive )
2. Execute the command ([root@localhost:/]#mount /dev/sdX /mnt/usbdrive, where sdX is the USB drive file name inside /dev folder). Verify the usb drive mapping by navigating the file /etc/mtab or /mnt/usbdrive.
3. Unmount the usb drive ([root@localhost:/]#umount /mnt/usbdrive).

Easy Understanding - Mount and Un-Mount CDROM

Attaching any filesystem to the root filesystem hierarchy of linux is basically called mounting the filesystem. Here in this blog, i will tell you how to mount the cdrom filesystem on the linux file system hierarchy. Find the easy steps:

1. Insert the CD/DVD into the physical drive
2. Login to the linux system with root privileges (Otherwise execute the command - sudo `mount command`)
3. Create a directory under /mnt or /media 
    [root@localhost:/]# mkdir /mnt/cdrom
4. Mount the cdrom with the following commands
    [root@localhost:/]# mount -t iso9660 /dev/cdrom /mnt/cdrom
5. Navigate to the folder /mnt/cdrom to verify that the drive is mounted.

One can also check the file /etc/mtab to see the disk in mounted. Now, for unmounting.....its fairly easy. Run the following command.
6. Unmount the cdrom
[root@localhost:/]#umount /mnt/cdrom

Verify it from the file /etc/mtab

Inside /proc ( Illusionary Filesystem)

The
/proc filesystem

The /proc filesystem contains a illusionary filesystem. It does not exist on a disk. Instead, the kernel creates  it in memory. It is used to provide information about the system (originally about processes, hence the name). Some of the more important files and directories

/proc/1

A directory with information about process number 1. Each process has a directory below /proc

with the name being its process identification number.

/proc/cpuinfo

Information about the processor, such as its type, make, model, and performance.

/proc/devices

List of device drivers configured into the currently running kernel.

/proc/filesystems

Filesystems configured into the kernel.

/proc/meminfo

Information about memory usage, both physical and swap.

/proc/stat

Various statistics about the system, such as the number of page faults since the system was booted.

/proc/uptime

The time the system has been up.

/proc/version

The kernel version.

Inside /etc ( Plain text configuration file)

The /etc directory

The /etc maintains a lot of files. Some of them are described below. For others, you should determine which program they belong to and read the manual page for that program. Many networking configuration files are in /etc as well

/etc/rc or /etc/rc.d or /etc/rc?.d

Scripts or directories of scripts to run at startup or when changing the run level. See Section 2.3.1 for further information.

/etc/passwd

The user database, with fields giving the username, real name, home directory, and other information about each user. The format is documented in the passwd manual page.

/etc/shadow

/etc/shadow is an encrypted file the holds user passwords.

/etc/fstab

Lists the filesystems mounted automatically at startup by the mount −a command (in /etc/rc or equivalent startup file). Under Linux, also contains information about swap areas used automatically by swapon −a.

/etc/group

Similar to /etc/passwd, but describes groups instead of users.

/etc/inittab

Configuration file for init.

/etc/issue

Output by getty before the login prompt. Usually contains a short description or welcoming message to the system. The contents are up to the system administrator.

/etc/motd

The message of the day, automatically output after a successful login. Contents are up to the system administrator. Often used for getting information to every user, such as warnings about planned downtimes.

/etc/mtab

List of currently mounted filesystems. Initially set up by the bootup scripts, and updated automatically by the mount command. Used when a list of mounted filesystems is needed, e.g., by the df command.

etc/login.defs

Configuration file for the login command.

/etc/profile, /etc/bash.rc, /etc/csh.cshrc

Files executed at login or startup time by the Bourne, BASH , or C shells. These allow the system

administrator to set global defaults for all users. Users can also create individual copies of these in their home directory to personalize their environment. See the manual pages for the respective shells.

/etc/shells

Lists trusted shells. The chsh command allows users to change their login shell only to shells listed in this file. ftpd, is the server process that provides FTP services for a machine, will check that the user's shell is listed in /etc/shells and will not let people log in unless the shell is listed there.

The root filesystem

The root filesystem

The root filesystem should generally be small, since it contains very critical files and a small, infrequently modified filesystem has a better chance of not getting corrupted. A corrupted root filesystem will generally mean that the system becomes unbootable except with special measures (e.g., from a floppy), so you don't want to risk it. The root directory generally doesn't contain any files, except perhaps on older systems where the standard boot image for the system, usually called /vmlinuz was kept there. (Most distributions have moved those files the the /boot directory. Otherwise, all files are kept in subdirectories under the root filesystem:

/bin

Commands needed during bootup that might be used by normal users (probably after bootup).

/sbin

Like /bin, but the commands are not intended for normal users, although they may use them if

necessary and allowed. /sbin is not usually in the default path of normal users, but will be in root's default path.

/etc

Configuration files specific to the machine.

/root

The home directory for user root. This is usually not accessible to other users on the system

/lib

Shared libraries needed by the programs on the root filesystem.

/lib/modules

Loadable kernel modules, especially those that are needed to boot the system when recovering from disasters (e.g., network and filesystem drivers).

/dev

Device files. These are special files that help the user interface with the various devices on the system.

/tmp

Temporary files. As the name suggests, programs running often store temporary files in here.

/boot

Files used by the bootstrap loader, e.g., LILO or GRUB. Kernel images are often kept here instead of in the root directory. If there are many kernel images, the directory can easily grow rather big, and it might be better to keep it in a separate filesystem. Another reason would be to make sure the kernel images are within the first 1024 cylinders of an IDE disk. This 1024 cylinder limit is no longer true in most cases. With modern BIOSes and later versions of LILO (the LInux LOader) the 1024 cylinder limit can be passed with logical block addressing (LBA).

/mnt

Mount point for temporary mounts by the system administrator. Programs aren't supposed to mount on /mnt automatically. /mnt might be divided into subdirectories (e.g., /mnt/dosa might be the floppy drive using an MS−DOS filesystem, and /mnt/exta might be the same with an ext2 filesystem). /proc, /usr, /var, /home. Mount points for the other filesystems. Although /proc does not reside on any disk in reality it is still mentioned here. See the section about /proc later in the chapter.

The Linux File System - 1

How many of you do really understand what is file is all about?. Or why we even need a file system. Well, thats another topic which is out of scope for the current blog. But I must recommed you to visit the website `http://www.pathname.com/fhs/` to get the clean understanding on the file system hierarchy and the standard implementations along with the deviations.

This blog is a loose attemt to through light on the FHS and the Linux directory tree structure standards.

Such a standard has the advantage that it will be easier to write or port software for Linux, and to administer Linux machines, since everything should be in standardized places.

/usr filesystem contains all commands, libraries, manual pages, and other unchanging files/usr should be specific for any given machine, nor/usr). It can make administration easier (only the master /usr needs to be changed when updating an application, not each machine separately) to have /usr network mounted. Even if the filesystem is on a local disk, it could be mounted read−only, to lessen the chance of filesystem corruption during a crash.

The

needed during normal operation. 

/usr filesystem contains all commands, libraries, manual pages, and other unchanging files No files in should they be modified during normal use. This allows the files to be shared over the network, which can be cost−effective since it saves disk space (there can easily be hundreds of megabytes, increasingly multiple gigabytes in

The

The /home filesystem contains the users' home directories, i.e., all the real data on the system.
Separating home directories to their own directory tree or filesystem makes backups easier; the other
parts often do not have to be backed up, or at least not as often as they seldom change. A big /home
might have to be broken across several filesystems, which requires adding an extra naming level
below /home, for example /home/students and /home/staff.

/var filesystem contains files that change, such as spool directories (for mail, news, printers,etc), log files, formatted manual pages, and temporary files. Traditionally everything in been somewhere below /usr , but that made it impossible to mount /usr read−only.

The root filesystem is specific for each machine (it is generally stored on a local disk, although it

could be a ramdisk or network drive as well) and contains the files that are necessary for booting the

system up, and to bring it up to such a state that the other filesystems may be mounted. The contents

of the root filesystem will therefore be sufficient for the single user state. It will also contain tools for

fixing a broken system, and for recovering lost files from backups.