Unix/Linux command line concepts

Jan 24, 2012 Juche 101Artem Nagornyi

Chapter 1

ls, mkdir, cd, pwd

1.1 Listing files and directories

$ ls The ls command (lowercase L and lowercase S) lists the contents of your current working directory.

...

$ ls -a Lists all files in the current directory including those whose names begin with a dot (.) which are considered as "hidden".

1.2 Making directories

$ mkdir dirname This will create a new sub-directory in the current directory.

1.3 Changing to a different directory

$ cd mydir

$ cd /local/usr/bin

Changes the current working directory to mydir directory.

You can also use full path of the directory.

1.4 The directories . and ..

$ cd .

$ cd ..

$ cd ./mydir/innerdir$ ls ./mydir/innerdir$ cd ../../anotherdir$ ls ../../anotherdir

In UNIX, (.) means the current directory, so typing this command means stay where you are (the current directory).

(..) means the parent of the current directory, so typing this command will take you one directory up the hierarchy. Feel free to use (.) and (..) symbols when changing and listing directories.

1.5 Pathnames

$ pwd Prints path of the current directory (working directory).

1.6 Home directory

~

$ ls ~/mydir

This is the symbol of your home directory. Each user of the Unix system has its own username and own home directory under /home.For example these are home dirs: /home/artem, /home/john Will list the contents of mydir sub-directory of your home directory, no matter where you currently are in the file system.

Chapter 2

cp, mv, rm, rmdir, cat, less, head, tail, grep, wc

2.1 Copying files

$ cp ./myfile /home/artem

$ cp myfile /home/artem/mf2 $ cp /usr/local/myfile .

Copies myfile file from the current directory to /home/artem directory.

Copies myfile file from the current directory to /home/artem directory, renaming it to mf2. Copies /usr/local/myfile to the current directory.

...

$ ln -s /usr/local/ff/firefox /usr/bin/firefox This command will make a symbolic link /usr/bin/firefox to the file /usr/local/ff/firefoxSymbolic links have l symbol in the beginning of 'ls -l' output string.

$ ln /usr/local/ff/firefox /usr/bin/firefox This will make a hard link. The difference between a symbolic link and a hard link is that a hard link to a file is indistinguishable from the original directory entry; just consider it as a file alias. Hard links may not normally refer to directories.

...

$ ln myfile hlink

$ rm myfile

$ cat hlink

This experiment proves that a hard link is just another name for a file. Even after deleting original file it still exists because we haven't deleted the hard link. Simply there is really no such thing as "hard link", we just create another name for a file.

2.2 Moving files

$ mv ./myfile /home/artem

$ mv myfile /home/artem/mf2 $ mv /usr/local/myfile .

Moves myfile file from the current directory to /home/artem directory.

Moves myfile file from the current directory to /home/artem directory, renaming it to mf2. Moves /usr/local/myfile to the current directory.

2.3 Removing files and directories

$ rm myfile

$ rm /usr/local/myfile

$ rm -R mydir

$ rmdir mydir

Removes myfile file in the current directory.

Removes /usr/local/mydir file.

Removes mydir sub-directory in the current directory.

Removes mydir sub-directory in the current directory.

2.4 Displaying the contents of a file on the screen$ clear

$ cat myfile

$ less myfile

$ head myfile

$ head -5 myfile

$ tail -5 myfile

Will clear screen.

Will display the content of a file on the screen.

Will display the content of a file page-by-page.

Will display the first 10 lines of myfile on the screen.

Will display the first 5 lines.

Will display the last 5 lines.

2.5 Searching the contents of a file

$ less myfile This will display the contents of myfile page-be-page. Then, still in less, type a forward slash [/] followed by the word to search. less finds and highlights the keyword. Type [n] to search for the next occurrence of the word.

...

$ grep Science myfile

$ grep 'spinning top' myfile

$ grep -i Science myfile

This will print each line of myfile containing the word Science (it is case-sensitive).

To search for a phrase or pattern, you must enclose it in single quotes. Key -i will ignore upper/lower case in the search results.

...

Some of the other options of grep are:

-v display those lines that do NOT match -n precede each matching line with the line number -c print only the total count of matched lines

...

$ wc -w myfile

$ wc -l myfile

Will return the number of words in myfile.

Will return the number of lines in myfile.

Chapter 3

>, >>, <, |, sort, who

3.1 Redirection

$ cat Type cat without specifing a file to read. Then type a few words on the keyboard and press the [Return] key. Finally hold the [Ctrl] key down and press [d] (written as ^D for short) to end the input. It reads the standard input (the keyboard), and on receiving the 'end of file' (^D), copies it to the standard output (the screen).

...

$ cat > myfile Type something, then press [Ctrl-d] to end the input. The output will be redirected to myfile.

If the file already contains something, it will be overwritten.

...

$ cat >> myfile Type something, then press [Ctrl-d] to end the input. The output will be redirected and appended to myfile.

If the file already contains something, it will be appended.

...

$ cat myfile1 myfile2 > file3 This will join (concatenate) myfile1 and myfile2 into a new file called file3. What this is doing is reading the contents of myfile1 and myfile2 in turn, then outputting the text to the file file3.

3.3 Redirecting the input

$ sort Enter this command. Then type in the names of some animals. Press [Return] after each one.

dogcatbirdape[Ctrl-d]

The output will be:apebirdcatdog

...

$ sort < file1 > file2 Input redirection is <

In this command we use both input and output redirection.The unsorted list will be taken from file1 and already sorted list will be redirected to file2.

3.4 Pipes

$ who > usernames$ sort < usernames

$ who | sort

who command returns the list of all users currently logged in the system. This is a method to get a sorted list of names by using a temporary file usernames. This way we can avoid temporary file creation. Here we connect the output of the who command directly to the input of the sort command. This is exactly what pipes do. Pipe symbol is |

...

$ who | wc -l

$ cat myfile | grep science

And this is the way to find out how many users are logged in. We are using a pipe between who and wc commands.

This displays the line of myfile that contains 'science' string. We are using pipe between cat and grep commands.

Chapter 4

*, ?, man, whatis, apropos

4.1 Wildcards

$ ls list*

$ ls *list

The character * is called a wildcard, and will match against none or more character(s) in a file (or directory) name. This will list all files in the current directory starting with list... This will list all files in the current directory ending with ...list

...

$ ls ?ouse

The character ? will match exactly one character. So ?ouse will match files like house and mouse, but not grouse.

4.2 Unix filename conventions

Unix-legitimate filenames are any combination of these three classes of characters: 1. Upper and lower case letters: A - Z and a - z (national

characters are also supported in Unicode and other encodings)

2. Numbers 0 - 9 3. Periods, underscores, hyphens . _ - Some other characters can be also supported, but they are not recommended to use.

4.3 Getting help

$ man wc

$ whatis wc

This will display the manual page for wc command.

Gives a one-line description of the command, but omits any information about options etc.

...

$ apropos -s "1" copy If you are not sure about the exact name of the command, this will give you the list of commands with keyword in their manual page header. -s key defines section of Unix manual:

1. General commands 2. System calls 3. Library functions4. Special files 5. File formats and conventions 6. Games and screensavers 7. Miscellanea8. System administration commands and

daemons

Chapter 5

ls -lag, chmod, command &, bg, jobs, fg, kill, ps

5.1 File system access rights

$ ls -l● The left group of 3 gives the file

permissions for the user that owns the file (or directory) (artem in the above example).

● The middle group of 3 gives the permissions for the group of people to whom the file (or directory) belongs (softserve in the above example);

● The rightmost group of 3 gives the permissions for all others.

The symbol in the beginning of the string indicates whether this is a file, directory or a link:'d' indicates a directory, '-' indicates a file, 'l' indicates a symbolic link.

The permissions r, w, x (read, write, execute) have slightly different meanings depending on whether they refer to a simple file or to a directory.

-rw-rw-r-- 1 artem softserve 83 Feb 3 1995 myfile

Group of the owner

Owner of the file

Modification date

Filename

File permissions

Number of subdirectories (1

for a file)

Size

...

Access rights on files● r indicates read permission (or otherwise), that is, the presence or absence of

permission to read and copy the file ● w indicates write permission (or otherwise), that is, the permission (or otherwise) to

change a file ● x indicates execution permission (or otherwise), that is, the permission to execute a file,

where appropriate Access rights on directories

● r allows users to list files in the directory; ● w means that users may delete files from the directory or move files into it; ● x means the right to access files in the directory. This implies that you may read files in

the directory provided you have read permission on the individual files. So, in order to read a file, you must have execute permission on the directory containing that file, and hence on any directory containing that directory as a subdirectory, and so on, up the tree. Also file can be written if its permissions allow Write, but it can only be deleted if its directory's permissions allow Write.

5.2 Changing access rights

Access classes:u (user)g (group)o (other)a (all: u, g and o)

Operators:

+ (add access)- (remove access)= (set exact access)

Examples: chmod a+r myfile add permission for everyone to read a file (or just: chmod +r myfile)

chmod go-rw myfile remove read and write permission for group and other users chmod a-w+x myfileremove write permission and add execute for all users chmod go=r myfileexplicitly state that group and other users' access is set only to read

...

The other way to use the chmod command is the absolute form. In this case, you specify a set of three numbers that together determine all the access classes and types. Rather than being able to change only particular attributes, you must specify the entire state of the file's permissions.

The three numbers are specified in the order: user (or owner), group, other. Each number is the sum of values that specify: read (4), write (2), and execute (1) access, with 0 (zero) meaning no access. For example, if you wanted to give yourself read, write, and execute permissions on myfile; give users in your group read and execute permissions; and give others only execute permission, the appropriate number would be calculated as (4+2+1)(4+0+1)(0+0+1) for the three digits 751. You would then enter the command as:

chmod 751 myfile

5.2.1 Changing owner of the file

$ sudo chown artem myfile

$ sudo chown -hR artem myfolder

Change the owner of myfile to artem.We are using sudo before chown to temporarily give the current user administrative permissions (you will need to enter the root user password).

Change the owner of myfolder folder to artem including all nested sub-directories and files recursively.

5.2.2 Changing file timestamps

$ touch -d "2005-02-03 14:04:25" myfile

$ touch -md "2005-02-03 14:04:25" myfile $ touch -ad "2005-02-03 14:04:25" myfile $ touch myfile

This command will set both modification and access date/time for the file or directory.

Set only modification date/time for the file or directory.

Set only access date/time for the file or directory. Will create a new empty file 'myfile' if it doesn't exist.

5.3 Processes and jobs

$ ps auxUSER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND...

This command lists all currently running processes in the system.Output columns (this is Linux, output on other Unixes may slightly differ):

USER = user owning the processPID = process ID of the process%CPU = it is the CPU time used divided by the time the process has been running%MEM = ratio of the process’s resident memory size to the physical memoryVSZ = virtual memory usage of entire process (including swapped memory)RSS = resident set size, the non-swapped physical memory that a task has usedTTY = controlling tty (terminal)STAT = multi-character process state (running, sleeping, zombie, etc.)START = starting time or date of the processTIME = cumulative CPU timeCOMMAND = command with all its arguments

...

$ ps aux | more

$ ps aux | grep pname_or_id

This will allow listing long list of processes page-by-page.

This is the way to search for a given process name or process id in the list of processes. Only the lines containing the process name or id will be displayed.

...

$ sleep 5

$ sleep 5 &

This will wait 5 seconds and then return the command line prompt.

This will run the sleep command in background and return the command line prompt immediately, allowing you do run other programs while waiting for that one to finish.

...

$ sleep 15 You can suspend the process running in the foreground by typing ^Z, i.e.hold down the [Ctrl] key and type [z]. Then to put it in the background, type 'bg' and [Enter].

5.4 Listing suspended and background processes$ jobs

$ fg %2

$ fg %1

When a process is running, backgrounded or stopped, it will be entered onto a list along with a job number.The output of this command will be such as this:

[1] Stopped sleep 1000[2] Running vim[3] Running matlab

This will foreground the process number 2. This will resume and foreground the stopped process number 1.

5.5 Killing/signalling a process

$ sleep 5[Ctrl+c]

$ kill pid

$ kill -9 pid

$ kill n

[Ctrl+c] combination will kill the foreground process.

This will kill the process using its process id (you can get it from the output of ps command).

This will forcibly kill the process even if it hanged (or just stopped).

This will kill the job with number n from background.

...

$ sleep 15[Ctrl+z] $ kill -stop pid

$ kill -cont pid $ pkill processname

$ killall processname

This will stop (temporarily suspend) the process. This will stop (temporarily suspend) the process.

This will resume the stopped process.

To kill all processes with the name processname.

To kill all processes with the name processname.

Chapter 6

df, du, gzip, zcat, file, diff, find, history

6.1 Other useful Unix commands

$ df -h $ df -h .

$ du -ahc mydir

This will show the amount of used/available space on all mounted filesystems.

This will show the amount of used/available space only on the current filesystem.

This will show the disk usage of each subdirectory and file of mydir directory, and mydir directory itself.

...

$ gzip myfile

$ gunzip myfile

$ zcat myfile.gz

$ zcat myfile.gz | less

This will compress myfile using Gzip compressing tool. The original file will be deleted.

This will uncompress myfile.

This will read gzipped files without needing to uncompress them first. And now you can view the gzipped file page-by-page.

...

$ file *

$ diff file1 myfile2

Classifies the named files in the current directory according to the type of data they contain, for example ASCII text, pictures, compressed data, directory, etc. Compares the contents of two files and displays the differences. Lines beginning with a < denotes file1, while lines beginning with a > denotes file2.

...

$ find . -name "*.txt" -print $ find . -size +1M -ls

Searches for all files with the extension .txt, starting at the current directory (.) and working through all sub-directories, then printing the name of the file to the screen (simple output). To find files over 1Mb in size, and display the result as a long listing (similar to ls command output).

...

$ history | less

$ set history=100

This will give an ordered list of all the commands that you have entered. Piping the output to less command allows both forward and backward scrolling of the list (more command only allows forward scrolling).

This way you can change the size of the history buffer (set command changes runtime parameters).

Chapter 7

export, printenv, unset, .bashrc, source, ssh, mount, reboot,

shutdown, crontab

6.2 Environment variables

$ export MYVAR=myvalue

$ printenv

$ printenv | grep MYPAR

$ unset MYVAR

Adds a new environment variable MYVAR with value value myvalue (export command works for Debian/Ubuntu Linux).

Prints all environment variables.

Displays the value of MYVAR environment variable.

Unsets (deletes) environment variable MYVAR.

...

$ export $PATH:/mydir This way we can add new directories in the end of PATH environment variable (all directories are divided by : symbol).

...

$ vi ~/.bashrc

$ source ~/.bashrc

This way we can add environment variables on permanent basis. Just insert export MYVAR=myvalue in the end of file opened in VI. This variable will be loaded automatically at shell start.

Force reload of environment variables from ~/.bashrc file.

Note: This is for Bash, if you use a different shell, you should use another file.

6.3 Remote shell

$ ssh user@host

$ exit

This way we can connect to another Unix machine that has OpenSSH server running and port 22 opened. Upon connect you will be asked to enter a password for user. host parameter can be a hostname or IP address.

You can leave the remote shell by entering exit command.

6.4 Mounting filesystems

$ mkdir mydir$ sudo mount -t vfat /dev/sdc1 mydir

This way we can create a mount point and mount FAT32 filesystem to this mount point (only root user can do this).

$ umount mydir

And now we have unmounted the filesystem, the directory will be empty.Noticed /dev/sdc1 ? This is a device file for the filesystem. If it exists, than the filesystem is physically present, but not mounted until we execute the mount command.Other fs types exist (-t option): ext3, ext4, reiserfs, ntfs, etc.

...

$ mkdir alpha $ sudo mount -t smbfs //alpha.softservecom.com/install alpha -o username=yourusername,password=yourpassword This way we can mount remote SMB network filesystem, providing credentials for authentication.

If you want the filesystem to be mounted automatically, then you need to edit /etc/fstab file that has its own format (see the man page for details).

6.5 Shutdown and Reboot

$ sudo reboot

$ sudo shutdown -h now $ sudo shutdown -h 18:45 "Server is going down for maintenance"

Reboot the system immediately. Shutdown the system immediately.

Shutdown the system at 18:45.

6.6 Scheduling

$ crontab -eThis command opens crontab file where you can schedule commands execution. (use sudo if you need the command to be executed with root permissions) The format of a line is:minute (0-59), hour (0-23, 0 = midnight), day (1-31), month (1-12), weekday (0-6, 0 = Sunday), command Example:01 04 1 1 1 /usr/bin/somedirectory/somecommand

The above example will run /usr/bin/somedirectory/somecommandat 4:01am on January 1st plus every Monday in January.

...

An asterisk (*) can be used so that every instance (every hour, every weekday, every month, etc.) of a time period is used. Example:01 04 * * * /usr/bin/somedirectory/somecommand This command will run /usr/bin/somedirectory/somecommand at 4:01am on every day of every month.

Comma-separated values can be used to run more than one instanceof a particular command within a time period. Dash-separated values can be used to run a command continuously. Example: 01,31 04,05 1-15 1,6 * /usr/bin/somedirectory/somecommand The above example will run /usr/bin/somedirectory/somecommand at 01 and 31 past the hours of 4:00am and 5:00am on the 1st through the 15th of every January and June.

...

Usage: "@reboot /path/to/executable" will execute /path/to/executable when the system starts.

string meaning@reboot Run once, at startup.

@yearly Run once a year, "0 0 1 1 *".

@annually (same as @yearly)

@monthly Run once a month, "0 0 1 * *".

@weekly Run once a week, "0 0 * * 0".

@daily Run once a day, "0 0 * * *". @midnight (same as @daily)

@hourly Run once an hour, "0 * * * *".

The end.