Unix/Linux Utilities : Useful Commands

The following post contains the basic information of some of the most useful UNIX/LINUX commands. Unix/Linux comes in many flavours and the following may not work in some. There also be many different syntax/ways to use them, please go through manual for detail. Without any further delay lets START.

1) man

displays reference manual page for a command.

$man command1 //will display the manual page for command1, e.g man cp, man man

2) whoami

prints the current user name associated with the current effective userid.

who : full list of users currently logged in

3) cd

Change Directory.

$cd test          //will go to test dir if exist

$cd ../..           //go back 2 directories

$cd                  //typing only cd will move you into home directory

4) mkdir

Make Directories

$mkdir testdir                                 //will make a directory testdir

$mkdir -v testdir                          // -v will print a message for each directory created

$mkdir -p testdir1/testdir2      // will make parent directories as needed

5) pwd

print working directory

$ pwd

/home/dj/linuxtest

6) ls

list the contents of directory

ls [options] [names]

If name is left blank, it will list all of the files in the directory.

Options: (Some)

-l Displays the long format listing

-t Displays the newest files first

-r Displays the files in reverse order

-a Displays all files, even hidden

etc.

$ ls -l

total 12

drwxrwxr-x 2 dj dj 4096 Aug 31 11:56 testdir

drwxrwxr-x 3 dj dj 4096 Aug 31 12:03 testdir1

-rw-rw-r– 1 dj dj 12 Sep 7 16:42 testfile1.txt

Lets check the output:

lsResultAnalysis

Field 1: 1st Character is Field Type

– normal file
d directory

s socket file
l link file
Next 9 charaters – File permission

Each 3 characters refers to the read, write, execute permissions for user, group and others. Here we have read write permission for user and group and only read permission for others.

Field 2 : Number of links

Field 3 : Owner

Field 4 : Group

Field 5 : Size

Field 6 : Last modified date and time

Field 7 : File name/Dir name

7) mv

move or rename file/dir

mv myfile mynewfilename     //rename a file

mv myfile testdir/myfile         //move file to the tesrdir (in current)

mv myfile testdir/                    //move file in the testdir

Some options :
-f force overwriting exsisting files
-n no overriding

8) cp

copy files/dir

$cp testfile1 testdir/testfile2     //copies testfile1 to testfile2. If not there, it creates it

$cp testfile1 testfile2 testdir     //copies testfile1 to testdir/testfile1 and testfile2 to testdir/testdile2

Some options:

-r copy directories recursively

-p preserve the modification date, time

9) rm

remove files/directories

$rm testfile1        //will remove FILE named testfile

rm testdir          //will throw error, to remove dir you need to use -r

Some options:

-r remove directories and their contents recursively

-f force, never promp

rm -rf is probably the most dangerous command, which will delete all files. If run on root, it will delete everything. Many filesystem have protections to handle above statement on root. Linus admistration might use some alias command to modifiy so as to check, if you are sure.

10) find

locate files/dir

Eg.

find where [options] what

$find . test*                         //find all files beginning with test in the current directory

$find . -name testfile1     // here we use the criteria -name to tell to perform a name search

$find . -mtime 1                //find all the files modified exact 1 day

$find . -mtime -1               //find all the files modified less than 1 day

$find . -perm 777              //find all files that holds a 777 permission

$find . -iname testfile1    //case insesitive search result

11) grep

search a file which contains a particular word or particular pattern. g/re/p (globally search a regular expression and print)

$grep “test” testfile1                          //searches given string in file

$grep -i “test: testfile1                       //case insensitive search

$cat testfile | grep “test”                    //searches test on the result

$some command output | grep “test”  // searches on the result

12) ps

displays active process. Usually runs with -ef, where -e selects every process and -f chooses full output format

$ps -ef | grep java           //shows all java processes running

13) kill

kill a process.

$kill prcessid

$kill -s signalName prcessid

$kill -9 processid

14) tail

displays the end of the text file. Very useful to check logs of your application continuously.

$tail testfile1           //display last 10 (default) lines

$tail -100f testfile1   //displays last 100 lines of the files , -f meaning follow ,i.e monitor the growth of the file

15) head

Displays the top of the file.

$head testfile1           //display first 10 (default) lines

16) top

display and update information about the top cpu processes.

$top    //displays process sorted by cpu usage

topCmdDefaultResult

press 1 to display process from all CPU

topCmdBothCpuResult

press M to sort ouptup based on memory

press P to sort based on cpu utilization

To kill a process without exiting from top command press lowercase “k”. Enter the PID of process to quit.

By default screen refresh interval is 3.0 seconds, same can be change pressing ‘d‘ option and entering the new time.

etc…

Q to quit the top output
To display a particular user output :

$top -u dj            //where dj is your user

17) cat

The cat command reads one or more files and prints them to standard output

$cat testfile1                                             //display contents of testfile1

$cat testfile1 testfile2 >> testfile3     //creates a new file testfile3 containing contents of testfile2 and tesfile1

18) chmod

change pemission for files/dir.

Some symbol representation of different types of role and permission:

u        user

g        group

o       others

a       all,same as ugo

r       read permission

w     write permission

x      execute permission

+      adds the specific modes

–       removes the specific modes
$chmod a+r file        //read is added for all
$chmod a-x file        //execute permission is removed for all
$chmod a+rw file    //change the permissions of the file file to read and write for all.
$chmod 777 file       //change the permissions of the file file to read, write, and execute for all.

19) gzip

reduces the size of named files, replacing them with files of the same name extended by.gz

#compresses the file testfile1.txt and replace it with testfile1.txt.gz

$gzip testfile1.txt

#If you want to keep your original file use -c

$gzip -c testfile1.txt > testfile1.txt.gz

Other options:

-v gives information about the compression

-r recursively compresses al files under the specific directories

#To unzip : use gunzip

$gunzip testfile1.txt.gz

20) tar

tape archiving, usually used to combine few files into a single file for easy distribution.

$tar -cvf test.tar testfile1.txt testfile2.txt

Creates test.tar with the files in it. -c creates, -v verbose which reports all the files that were added, -f use the argumet as the name of the file, if you dont use it, tar will use the device indicated by the TAPE environment variable.

$tar -xvf test.tar //seperate the files

Compressing:

$tar -cvzf test.tar.gz testfile1.txt testfile2.txt

#-z options tells the tar to zip the archive as it is created

#To unzip :

$tar -xvzf test.tar.gz

21) touch

Create new and empty files. Also used to change the timestamps on existing files and directories

$touch testfile3.txt        //will create new empty file

$touch -a testfile3.txt     //change the access time of the file

$touch -m testfile3.txt     //change the modification time of the file

$touch -t [[CC]YY]MMDDhhmm[.SS] filename

CC – first two digit of year; YY – last two digits of year; MM – month; DD – day; hh – hour; mm – minutes; ss – seconds

$touch -a -m -t 201309062126.03 testfile2.txt

22) env

list all environmental variables.

PATH: Where Linux looks for programs

It defines the order of directories which the computer will search for programs to execute.

$ env | grep “PATH” or $ echo $PATH

23) which

identifies location of the executables.

$which java //tells java location

24) ssh

The ssh command allows you to securely connect to a remote machine. It was a replacementfor the old telnet command, which sent your password in the clear.

$ssh -i key.pem hostname //key.pem is your key to connect

25) scp

Copy files to/from remote machines.

# Copy hello.txt from local computer to remote home directory

$ scp -i key.pem hello.txt hostname:~/

# Copy hello.txt from local to remote home directory, renaming it foo.txt

$ scp -i key.pem hello.txt hostname:~/foo.txt

# Copying ~/foo.txt from remote to current local directory

$ scp -i key.pem hostname:~/foo.txt .

# Copying ~/foo.txt from remote to local directory dir1, renaming it newtestfile.txt

$ scp -i key.pem hostname:~/foo.txt dir1/newtestfile.txt

26) ln

links

In Linux each file is assigned unique properties (i.e.; permissions, creation date, file type) when created. The two properties critical to understanding links are inode number and file name.

An inode number is assinged to the file data stored on the disk.The file name is created in the directory structure and assigned to the inode number.The inode number (the file contents) can have any number of file names hard linked to it. The ls -­l command will report the number of hard linked file names to the coressponding inode number in second field of output (the field after permissions). The ls command will not tell you what other file names are hard linked to the inode number.

When a file name is deleted from the directory tree, the file name’s connection to the inode number is severed. If the file name is the last “link” to the inode number, thereby reducing the number of inode (file content) “links” to file names to 0, then the inode is detatched from the actual data and returned to the pool of free inode numbers. The data remains on the disk, but unaccessable by traditional means now that it no longer has a corresponding inode number. The disk space is free to be overwritten. This process is called unlinking.

There are two types of links available in Linux — Soft Link and Hard Link.

A hard link allows two (or more) file names to point to the same inode number. Any change made to either file name is reflected in file data. If you delete either file name, the other will remain (basically, you are unlinking the file name from the inode number and removing the file name from the directory tree). Hard links cannot be used to link directories (unless the ­d,­F option is used by super user). They cannot be used to link across different file systems. Hard links are restricted to the current mounted disk volume boundary.

Symbolic links link one (or more) files to a target file. The file name is actually connected to the target file name and not the inode number of the data (like a hard link). All changes made to the links are reflected in the target file, not the link. The only exception is if you delete the link, the target file will not be altered. If you delete the target file the link(s) will no longer be useful since the (their) target is gone.

Symbolic links will work on directories. Think of symbolic links as short­cuts. You can create a symbolic to a file which does not exist (you cannot do this with a hard link). Symlinks can cut across volume boundaries. Permission bits are not used on symlinks. The symlink retains the permission of the file it is linked to.

Example:

We have a directory testdir haivng 2 files and one directory viz. testfile1.txt, testfile2.txt, testdir1. Lets create a hard link file inside testdir1 named hardlinkfile.txt.

$ln ../testfile1.txt hardlinkfile.txt    //inside testdir1

Lets create a soft link file inside testdir1 named softlinkfile.txt

$ln -s ../testfile2.txt softlinkfile.txt

Doing an ls -Rl on testdir

testdir$ ls -Rl
.:
total 4
drwxrwxr-x 2 dj dj 4096 Sep  9 02:00 testdir1
-rw-rw-r– 2 dj dj    0 Sep  9 01:57 testfile1.txt
-rw-rw-r– 1 dj dj    0 Sep  9 02:00 testfile2.txt

./testdir1:
total 0
-rw-rw-r– 2 dj dj  0 Sep  9 01:57 hardlinkfile.txt
lrwxrwxrwx 1 dj dj 16 Sep  9 02:00 softlinkfile.txt -> ../testfile2.txt

Note that everyone is given permission on softlinkfile.txt compare to hardlinkfile.txt which retains the original permission. Also softlinkfile.txt has a pointer to the main file.

27) ldd

Linux systems have two types of executable programs:

Statically linked executables contain all the library functions that they need to execute; all library functions are linked into the executable. They are complete programs that do not depend on external libraries to run. One advantage of statically linked programs is that they work without your needing to install prerequisites.

Dynamically linked executables are much smaller programs; they are incomplete in the sense that they require functions from externalshared libraries in order to run. Besides being smaller, dynamic linking permits a package to specify prerequisite libraries without needing to include the libraries in the package. The use of dynamic linking alsoallows many running programs to share one copy of a library rather than occupying memory with many copies of the same code. For these reasons, most programs today use dynamic linking.

ldd (List Dynamic Dependencies) prints the shared libraries required by each program or shared library specified on the command line.

This is very helpful in debugging application for dependencies.

28) alias

Alias allows you to have a small more familiar command or name to execute a long string

alias mywebapplogs ‘cd path/path’

Now you can diretly visit your path using your custom name. It may also be used to change the behaviour of existing commands, for eg to protect using rm -rf command etc.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: