How e-mail works? .. even a caveman can understand, yes even a caveman

SMTP (Port 25): handles outgoing mail

POP3/IMAP (Port 110/143): handles incoming mail

Let's have scenario: John Doe at hotmail.com sends email to Laura Johnson at gmail.com. John Doe uses Outlook to send an email. Laura uses web interface for gmail.com to access the email from John. 

(Outlook, web interface are the front end for the users. Those are just the GUI interface)

John uses Outlook--> sends email from Hotmail.com account to Laura Johnson at Gmail.com account--> Laura checks email at www.gmail.com

1. Outlook connects to SMTP server at hotmail.com using Port 25

John writes an email to Laura

From: John.Doe@hotmail.com

To: Laura.Johnson@gmail.com

Subject: Hi Laura

Body: Hi Laura, How are you? What's your plan for this weekend? Let's go to the state fair at Dallas.

2. SMTP server at hotmail.com reads "TO" field and breaks it into two parts e.g Laura.Johnson@gmail.com is broken into "Laura.Johnson" and "gmail.com"

3. SMTP server at hotmail.com queries to DNS server; Who is SMTP server at gmail.com?

DNS server looks into MX records and replies back with one or more SMTP servers at gmail.com

4. SMTP server at hotmail.com connects to one of the SMTP server at gmail.com and handles the message. SENDMAIL and POSTFIX are the most popular MTA (Mail Transfer Agent) to deliver mail from one SMTP server to another another SMTP server. My preference is POSTFIX as it's free, easy, secure, customizable and popular too.

5. SMTP server at gmail.com recognizes that "Laura.Johnson" is the user at gmail.com and it handover the message to gmail POP3/IMAP server which puts the message in the Laura.Johnson's INBOX.

6. Laura logs in to www.gmail.com and checks her email.

It's that easy. Remember, we didn't talk about handling the junk emails.

It goes little bit complex for handling the junk emails. Please google about that if you want to know more...

Extra Treat:

SMTP commands:

HELO

EHLO

MAIL FROM

RCPT TO

DATA

RESET

QUIT

HELP

VRFY

EXPN

VERB

POP3 commands:

USER

PASS

QUIT

LIST

RETR

DELE

TOP

How can I setup VLAN on my Windows XP or Workstation?

If you have Intel NIC card, you are lucky. Intel PROset utility/driver allows you to add/remove VLAN on your workstation.

You could download Intel PROset from http://downloadcenter.intel.com/
Go to Downloads and Drivers and search for Intel PROset and choose your operating system

Install the downloaded executable file
(While installing it will prompt if you want to use MMC for the management, select YES)

It will also ask if you want to remove old drivers and settings, you can say NO

Once it is installed, Right Click "My Computer" --> Manage --> Device Manager --> Network Adapters --> Right Click "Inter(R) XXXX whatever it shows " --> Properties --> VLAN and add your desired VLAN ID

Try to access other devices on the same VLAN. It should work. If not contact your network administrator.

Other TIPS:
1. Start --> Run --> winmsd

if Processor properties has x86, your OS is 32 bit
if Processor properties has ia64, your OS is 64 bit

2. Device manager will show you the vendor of your NIC card. It could be Realtek, INTEL, Broadcom. The above procedure for VLAN will only work for INTEL NIC.

Remote Desktop in console mode

If you are using Windows XP RDP client, for RDP in console mode

mstsc /console

If you are using Windows Vista or above, it doesn't use /console session, it uses /admin session

mstsc /admin

Rename multiple files in Linux

Lets' change all files with extension ‘.html’ to ‘.php’.

Rename all *.html files in one folder

rename

#rename .html .php *.html



Do the same operation recursively in a directory tree

find . -name "*.html" -exec rename .html .php {} \;

Those double quotes around *.info are important



If you want to be expert go to following link and don't forget to read all the comments :)
http://www.thegeekstuff.com/2009/06/how-to-rename-files-in-group/

Basic tutorial on Logrotate

Log files in Linux usually reside at /var/log... It keeps on growing so log management is essential. Log management is usually achieved using logrotate. Logrotate is managed by cronjobs in Linux.

For logrotate, you can configure /etc/logrotate.conf or create the individual configuration files for each application or each log file in /etc/logrotate.d

step 1:

Let's say, I have VOIP application 'asterisk' running on my system. Asterisk generates various log files under /var/log/asterisk directory. I would create astlog under /etc/logrotate.d to manage the log files.

#cd /etc/logrotate.d
#vi astlog
/var/log/asterisk/full /var/log/asterisk/messages /var/log/asterisk/debug /var/log/asterisk/*.log {
nocompress
daily
rotate 5
missingok
copytruncate
}

Here we listed all the log files to be managed and provided the attributes of the log management. Don't compress the log file, rotate the log file daily, max number of log rotation 5 ( i.e logfilexxx.1, logfilexxx.2, .... , logfilexxx.5). It only keeps 5 log files. With copytruncate option, the original log file is truncated in place after creating a copy, instead of moving the old log file and optionally creating a new one. It is useful when some program cannot be told its logfile and thus might continue writing(apending) to the previous log file.

[you can use #stat < filename > or # ls -l < filename > to check the inode number
copytruncate helps the log file to preserver it's inode(unique file number) ]

If you don't want to use copytruncate option, then you have to tell the program that log file has been recreated (with new INODE number). For example, in my case I could have done

#cd /etc/logrotate.d
#vi astlog
/var/log/asterisk/full /var/log/asterisk/messages /var/log/asterisk/debug /var/log/asterisk/*.log {
nocompress
daily
rotate 5
missingok
create

        postrotate
             /usr/sbin/asterisk -rx 'logger reload' > /dev/null 2> /dev/null
 endscript

}

Here, we are telling our program 'asterisk' to reload logger as new log file has been created after log rotation.

step 2:

By default, Logrotate is scheduled daily. You can find 'logrotate' under /etc/cron.daily

Let's look at /etc/crontab

# less /etc/crontab
SHELL=/bin/sh
PATH=/usr/bin:/usr/sbin:/sbin:/bin:/usr/lib/news/bin
MAILTO=root
#
# check scripts in cron.hourly, cron.daily, cron.weekly, and cron.monthly
#
-*/15 * * * * root test -x /usr/lib/cron/run-crons && /usr/lib/cron/run-crons >/dev/null 2>&1


The time to execute the scripts is managed by crontab. /usr/lib/cron/run-crons script controls the cron.hourly, cron.daily, cron.weekly and cron.monthly. run_crons runs every 15 minutes and ensures that cron jobs are taken care of.

In SLES, if you need to change the default daily time , you can go to YAST --> System --> /etc/sysconfig editor --> System --> Cron --> DAILY_TIME and change the time.

Let's say, I want logrotation to be done at OFF hours (10:30 pm) to avoid the possible load on the server, then, I can change DAILY_TIME to 22:30


Tips:

To run the logrotate manually, you can try
#logrotate astlog


For more information:

http://www.linuxtopia.org/online_books/suse_linux_guides/SLES10/suse_enterprise_linux_server_installation_admin/sec_suse_pakete.html

http://www.thegeekstuff.com/2010/07/logrotate-examples/

CAT3 vs CAT4 vs CAT5 vs CAT5e vs CAT6

CAT3- contains 4 pairs of wires and can carry up to 10Mbps with a possible bandwidth of 16MHz. (UTP)

CAT4- contains 4 pairs of wires and can carry up to 16Mbps with a possible bandwidth of 20MHz. (UTP)

CAT5- contains 4 pairs of wires and cancarry up to 100Mbps with a possible bandwidth of100MHz. However there is more than one variety. There is also FastCAT which has a possible bandwidth of up to 350MHz. (UTP)

CAT5e- a higher grade of CAT5 that contains high-quality copper, offers a higher twist ratio (to eliminate cross-talk) and can support a bandwidth of 200MHz (2x that of REGULAR CAT5) (UTP)

CAT6- a twisted pair cable that contains 4 wire pairs each wrapped in foil insulation (to eliminate cross-talk). Additional foil covers the bundle of wires (to eliminate cross-talk further, and shield it from other interference ie. EMF from flourescent lights) and a fire-resistant plastic sheath covers the second foil layer. CAT6 supports at least six times the throughput of regular CAT5, but it is also 3-4x the price of CAT5. (STP)

UTP=Unshielded Twisted Pair
STP=Shielded Twisted Pair

Make copy batch file in windows

If you need to make a copy of your local drive to the backup drive scheduled everyday, you will need to know couple of things. [I am talking on Windows environment ;) ]

1. Download the robocopy. It is one of the microsoft product. It stands for robust copy

This is well built GUI tool. However to understand the various options available, you can check this
http://ss64.com/nt/robocopy.html

2. Install robocopy. You can write the simple script to make the copy process. [ You can use any kind of editor like notepad, wordpad, notepad++ to write the script]

robocopy source_folder destination_folder [file(s)_to_copy] [options]

#
# Script file name: C:\mycopy.cmd
#
@echo "Copying files and folders"
robocopy "C:\myLocalFile" "F:\myBackup" /E

@echo "Copy process completed"

/E  option allows to Copy Subfolders, including Empty Subfolders


3. Finally you can schedule the command file(batch file)
created by you to run in the specific time. 
Go to Control Panel --> Scheduled Tasks -->
Right click and create a new schedule task -->
Give the name to it --->
Right click on the scheduled task created and go to the properties -->
specify the path of the script file in the run section under Task Tab
(in my case it would be C:\mycopy.cmd ) -->
Now go to the Schedule Tab and specify the schedule you wanted

Hey you are done.

Etherpad installation on SLES or CentOS

Etherpad is online real-time collaborative document editing tool.

Requirements:

• Java 1.6
• Scala 2.7
• MySQL Server 5.1
• mysql-connector-java
• etherpad (If you need already cooked etherpad, feel free to ask me. I will email you)

Steps to be followed:

1. Create username 'etherpad' with password 'password'

#useradd -r -m etherpad
#passwd etherpad

2. Install MySQL and configure MySQL

--> You can use YAST for SLES and YUM for CentOS

#service mysql start --> in SLES
#service mysqld start --> in CentOS
#chkconfig mysql on --> to enable MySQL on next reboot


3. Once you download Scala 2.7 , extract the tar file to the /usr/local/scala directory.

#mkdir /usr/local/scala
#cd /usr/local/scala
#tar xvzf scala-2.7.7.final.tgz

4. Once you download mysql-connector-java, extract the tar file to the /usr/local/mysqlconnector

#cd /usr/local
#tar xvzf mysqlconnector.tar.gz

5. Install JAVA 1.6

#sh jdk-6u20-linux-x64-rpm.bin

Use YAST or YUM to install "fastjar"

6. Create myENV file

#touch myENV
#vi myENV
export JAVA_HOME=/usr/java/jdk1.6.0_20/
export JAVA=$JAVA_HOME/bin/java
export SCALA_HOME=/usr/local/scala/scala-2.7.7.final/
export SCALA=$SCALA_HOME/bin/scala
export PATH=$JAVA_HOME/bin:$SCALA_HOME/bin:$PATH
export MYSQL_CONNECTOR_JAR=/usr/local/mysqlconnector/mysql-connector-java-5.1.12-bin.jar

7. /etc/profile hosts the environment variables

#cat myENV >> /etc/profile

Logout and log back in.

8. Create database called "etherpad" and grant privilege to user "etherpad"

#mysql
>create database etherpad;
>grant all privileges on etherpad.* to 'etherpad'@'localhost' identified by 'password';
>exit;


9. Extract the content of etherpad in /etherpad directory.

#cd /etherpad/trunk/etherpad
#cd /bin
#vi rebuildjar.sh
We need to modify rebuildjar.sh as it fails to find the location of 'Infrastructure' directory

Change
source ../infrastructure/bin/compilecache.sh
to
source ../../infrastructure/bin/compilecache.sh

Change
cd ../infrastructure
to
cd ../../infrastructure


Also modify some other files

etherpad/trunk/etherpad# cp etc/etherpad.localdev-default.properties bin/data/etherpad.local.properties

etherpad/trunk/etherpad# cat bin/data/etherpad.local.properties
ajstdlibHome = ../../infrastructure/framework-src/modules
appjetHome = ./data/appjet
devMode = false
etherpad.adminPass = password
etherpad.fakeProduction = false
etherpad.isProduction = true
etherpad.SQL_JDBC_DRIVER = com.mysql.jdbc.Driver
etherpad.SQL_JDBC_URL = jdbc:mysql://localhost:3306/etherpad
etherpad.SQL_PASSWORD = password
etherpad.SQL_USERNAME = etherpad
listen = 127.0.0.1:9000
logDir = ./data/logs
modulePath = ../src
transportPrefix = /comet
transportUseWildcardSubdomains = true
useVirtualFileRoot = ../src

Also need to modify 3 other files

# vi etherpad\trunk\etherpad\src\etherpad\globals.js

var SUPERDOMAINS = {
'localbox.info': true,
'localhost': true,
'127.0.0.1': true
};

#vi etherpad\trunk\etherpad\src\etherpad\pro\pro_utils.js

var fromDomain = '127.0.0.1';

#vi etherpad\trunk\etherpad\src\main.js

var newurl = "http://127.0.0.1"+request.path;

10. Now we are almost done.
[it is very important that you are inside bin directory of etherpad to run the script files, otherwise you will get error: file location not found or something wrong]
etherpad/trunk/etherpad# cd bin
etherpad/trunk/etherpad/bin# ./rebuiljar.sh
etherpad/trunk/etherpad/bin#./run-local.sh

12. Then go to http://127.0.0.1:9000, it will prompt you to create new pad

13. To customize the home page, you can edit the file
# vi etherpad\trunk\etherpad\src\templates\main\home.ejs


14. If you replace 127.0.0.1 with your server IP address in all the files mentioned above, you can access Etherpad using http://your-ip-address:9000/


Links:
http://code.google.com/p/etherpad/wiki/Instructions
http://pauleira.com/13/installing-etherpad/
http://karteek.selfdabba.com/post/301488675/etherpad-opensourced

CVS and CVSNT in glance

1. Create Repository on the CVSNT server.

2. Launch wincvs and navigate to Admin --> Preferences --> specify CVS HOME [it is any directory that might be needed for wincvs configuration, however you will not need it if you are beginner]. If you have installed WinMerge or File Compare Tool (powerful), you can select that as your External Diff program. If you have Notepad++ installed, you can select that as your default editor.

3. Import module into the Repository (it means importing the Local project directory into the Repository)
a> Before importing module, we need to browse the local directory from wincvs [ see the browse section at the top in toolbar section]
b> On left side, you will see your local project directory which you just browsed.
c> Right click the folder and select Import Module option
d> Ignore the files which you can't edit [ e.g. .rpm, .bak, .pub and many more]
e> Specify the CVSROOT and repository location
f> You might want to tell wincvs to create CVS hierarchy so that it can create versions of files.

4. You can start editing files [Remember: When you are editing files, you are editing in your local project folder, no in the repository]. Once you are done, you can commit the changes. Commiting the changes means to make the change on the Repository too. Once the change is done on repository, you can checkout the module [ your project ]. Checkout means pulling out the project from the repository.

[local project directory] ------import--> [Repository] -----export--> [checkout project directory]

Change UUID of virtual drive

I made a copy of virtual drive (by copy and pasting .vdi file). Then, I tried to mount the .vdi file in the virtual machine. It came up with some error: A hard disk with UUID {bla..bla..bla......} is already registered. So, I was unable to mount the drive.

Now, what is the solution?
Ans: Change the UUID of the virtual/image drive.

How can I change the UUID of the virutal drive?
Ans: You have to run the VBoxManage command

[Before running command, make sure that environment variable is set to the location of the virtual box commands. To set environment variable: Go to "My Computer" --> Right click --> Properties --> Advanced --> Environment Variables --> Add the location of Virtual Box to the PATH variable; In my case it is : PATH .......;C:\Program Files\Sun\VirtualBox ]

C:\ > VBoxManage internalcommands setvdiuuid disk2.vdi

[disk2.vdi is the name of my copied .vdi file]

Now, try to mount the disk2.vdi in Virtual Machine.
Have a good one!


On my second day, I found issue with SLES (Suse Linux). It keeps on complaining "VB is waiting for /dev/disks/by-id/scsi-xxxxx-part2 to appear".
After some research on blogs, found that GRUB in SLES looks for UUID of disk to boot.

I set the copied/cloned harddisk as second harddisk (primary slave) for the original SLES VM. Then I startet the original SLES VM.
In the running SLES VM I made a new directory and mounted it to the root directory of the cloned harddisk.
Then I startet (as root) the Yast partition manager and noticed the UUID of the cloned harddisk. (It is made from the UUID shown with vboxmanage list hdds, but it is not the same).
Then I edited (as root) the /boot/grub/menu.lst and the /etc/fstab : I changed the UUID everywhere the old one was (do not use capital letters instead of small letters).
Now shutdown (after unmounting the cloned root directory)
Then I disabled the cloned harddisk as second harddisk in the original VM and set it as first harddisk in the cloned VM.
Starting the cloned VM now works.


For Network cards to work you have to do

1. rename the ifcfg-eth-id- to ifcfg-eth0 (in /etc/sysconfig/network)
2. remove the rules file (/etc/udev/rules.d/30-
3. edit /etc/sysconfig/network/config and change FORCE_PERSISTENT_NAMES to "no".

Install VMware-tools

It might be challenging to install VMware-tools if we don't know what exactly we are doing.
You need to read all the instructions that is provided by VMware workstation or Virtual Center or ESX server.
For now, I am going to give the instruction for Linux systems.
After installing Linux as Virtual Machine on the top of VMware workstation/ESX Server or VSphere (whatever), you will see that VMware-tools is not installed.

Steps to be followed:

1) Choose "Install VMware tools" from VM menu

2) You will recognize some change on "cd-rom" icon located at Left corner of VM. It just like someone has inserted the CD of VMware tools in your VM.

3) Now you need to mount the CD-ROM so that you can access the contents.
#mount /dev/cdrom /mnt/cdrom
If you are lucky, above command will work without any issue. If not you might have to specify some mount options like
# mount -o ro /dev/cdrom /mnt/cdrom
(which mean mount as read-only option)
Some systems requires you to specify the file-systems type. FYI, CD-ROM uses iso9660 file-system
# mount -t iso9660 -o ro /dev/cdrom /mnt/cdrom

4)After this, go to /mnt/cdrom directory. You will find VMware-toools over there. You can install from RPM or TAR file. It all depends on you, however I recommend tar file.

# cd /mnt/cdrom
# rpm -ivh [VMware-tools......rpm]

or
# cd /mnt/cdrom
#tar -xvzf [VMware-tools.......tar] /root/vmware
(this will un-archive the "tar" file under /root/vmware directory
# cd /root/vmware
#./vmware-install.pl

Some slick Unix/Linux commands... much more powerful and useful

KILL

usage:

Kill process by Process ID

• kill -9 [PID] --> Forcefully Kill
• kill -15 [PID] --> Gracefully Kill
• kill %[JobID] --> JobID can be checked by command #jobs

Kill process by Process Name

• killall [proces name]

PID/Process name can be found using commands like

• ps -aux
• ps -elf
• netstat -anp
• lsof -w -n -i [tcp/udp:port number]

System Tuning Commands:

#mpstat -A ALL --> CPUs status

#top --> Press 'I' to toggle between Irix Mode (Irix Mode display resource usage mulitplied by total number of CPUs) ; Press "1" to see all the CPUs

# sar

Networking Commands

#netstat -anp --> Where 'p' flag shows which program is making connection

[ Hey, now I know which program is making a connection, can I use kill command to kill by process name or PID? Answer: Ofcourse, that's the reason I am writing this article ]

Now you know how to terminate the particular remote connection in the server. There is also another technique using #lsof.

# lsof -w -n -i tcp:22

This command will display all the SSH (tcp:22) connections. So, you can see who has intruded in your system using SSH. This will also give process ID (PID) or process name, which you can terminate using KILL command.

[ use man page for each command in details]

#ifstat

#ethtool [ethernet interface name] --> this command can used to see if the interface is physically down or not

e.g # ethtool eth1

#netstat -i

#netstat -s

To get the BIOS information

#dmidecode

To list the hardware

#hwinfo

or

#lshw

[Note: lshw is not the linux project. If you want to use lshw, you have to download and install the RPM for that command http://ezix.org/project/wiki/HardwareLiSter ]

Bash Script to collect system information:

This is one of the simple script file to pull basic and informative information about your server:

#********************

#ServerReport.sh

# Created by DShah

# Please use it at your own risk

#********************

#!/bin/bash
miniDivider(){

echo "*******************************************************************";
}

serverInfo() {

uname -a
cat /etc/*release

}

diskInfo() {

# fdisk -l
df -h
iostat

}

memoryInfo() {

free -m
vmstat

}

cpuInfo() {

mpstat
mpstat -P ALL

}

overallInfo() {

top -b -n2


}

networkInfo() {
netstat -s
miniDivider

sleep 2
netstat -s
}

dividerLine(){

echo "########################################################################";

echo "########################################################################";

}

fileName="serverReporting.txt"

rm $fileName
touch $fileName

echo "ServerInfo Reporting"
serverInfo >> $fileName
dividerLine >> $fileName

echo "DiskInfo Reporting"
diskInfo >> $fileName
dividerLine >> $fileName

echo "MemoryInfo Reporting"
memoryInfo >> $fileName
dividerLine >> $fileName

echo "CPUInfo Reporting"
cpuInfo >> $fileName
dividerLine >> $fileName

echo "Network Info Reporting"
networkInfo >> $fileName
dividerLine >> $fileName

echo "Top command reporting";
overallInfo >> $fileName