Upload
mithun-aditya
View
225
Download
0
Embed Size (px)
Citation preview
8/8/2019 Unix Essentials V2.0
1/45
13 December 2010 TCS Internal
Unix EssentialsCourse @ ILP, TCS Limited
8/8/2019 Unix Essentials V2.0
2/45
13 December 2010TCS Internal 2
Course Objective
After the course you should
Have a fundamental understanding of the Unix architecture
Be able to use Unix as a platform for application development
8/8/2019 Unix Essentials V2.0
3/45
13 December 2010TCS Internal 3
History of unix
Ken thompson of AT&T ,BELL laboratories originally designed unix os in late 1960swhich evolved from timesharing OS called multics.
AT & T licenses source code for low cost
Originally written in assembly language but with the development of c programming
language in 1973,it was then written in c. Two versions of unix emerged are AT&T Unix
and BSD Unix
8/8/2019 Unix Essentials V2.0
4/45
8/8/2019 Unix Essentials V2.0
5/45
13 December 2010TCS Internal 5
Unix Principles
Multiuser and Multitasking
Everything is a file
Small, single purpose programs
Configuration data stored in text
Ability to chain programs together to perform complex tasks
8/8/2019 Unix Essentials V2.0
6/45
13 December 2010TCS Internal 6
UNIX Architecture
Shell and kernel together make UNIX system work.
H/W
user
Shells
interact
with users
KERNEL
interacts
with H/W
Database
Packages
Other Application
& System Software
Compilers
Utilities orcommands
user
user
user
user
8/8/2019 Unix Essentials V2.0
7/45
13 December 2010TCS Internal 7
The UNIX Kernel
Kernel is a collection of programs mostly written in C, that runs directly on the hardware -so parts of kernel must be customized to each systems hardware features.
Kernel is directly loaded into memory when the system is booted. Low level jobs are itsbasic services:
System Initialization
Process/Memory/File/I-O Management
Programming Interface
Communication Facilities
8/8/2019 Unix Essentials V2.0
8/45
13 December 2010TCS Internal 8
The UNIX Process
A process in UNIX is a program in execution with definite life-time and well-definedhierarchy.
User Space - Process execution on user data structure that represents an applicationand is manipulated by its own functions (user mode execution).
Kernel Space - Process execution on systems data structures that represent theresources (memory, files, devices) and supported by the kernel routines. Dependingupon resource requirement and availability, processs states are controlled by executingthe kernel routines accessed through system calls (system mode execution).
8/8/2019 Unix Essentials V2.0
9/45
13 December 2010TCS Internal 9
Environment around the UNIX Process When a process is created, UNIX opens 3 streams stdin/stdout/stderrfor basic communication with
respect to the process control terminal. In addition, it knows the current working directory forperforming file I/O.
Each login shellmaintains a description of the environment as a table of pointers to strings.
A global shell environment pointer called environ is maintained by UNIX kernel and it can be used bya process to access its own table
The shell does not directly use this table, but it creates a child process and calls exec() system call toexecute a command program that uses the table inherited from the shell parent.
TERM=vt100USER=ashokLOGNAME=ashokHOME=/usr/ashokPATH=.:/bin:/usr/ahok/bin
extern
char
**environ
A UNIX process
8/8/2019 Unix Essentials V2.0
10/45
13 December 2010TCS Internal 10
Environmental Variables (EVs)
Children inherit the entire execution environment from the parent. Some examples of theenvironment variables are the USER, LOGNAME, HOME, PATH, PS1, PS2, TERM MAIL,etc.
The HOMEVariable
It specifies an associated directory with every user in a UNIX system. If the HOMEvariable for the user Sita contains/usr/sita/stores, every time Sita logs in, she is taken tothe directory stores.
The variable HOME is referenced the same way:
$ echo ${HOME}
8/8/2019 Unix Essentials V2.0
11/45
13 December 2010TCS Internal 11
Environmental Variables (EVs) ...
The PATHVariablecontains a list of all full path-names (separated by a colon) of directories that are to besearched for an executable program. For example, the command$PATH=.:/usr/bin:/bin specifies directories to be searched for any executable fileor a command file (current directory, /usr/bin and /bin, in that order).
The PS1 VariableThe system prompt may be changed by setting the value of this variable to the desiredprompt:$ PS1=Hello>Hello> #can be changed only at the UNIX command line, not within a shell script.
The PS2Variable: prompt string for continued command line (default > ).
8/8/2019 Unix Essentials V2.0
12/45
13 December 2010TCS Internal 12
Environmental Variables (EVs) ...
The LOGNAMEVariablecontains users login name. Its contents cannot be changed by the user, but can bedisplayed:echo ${LOGNAME}
The TERMVariableNames the kind of terminal you are using; setting it helps to manage your screen moreeffectively, say,$ TERM=vt100
The PWD VariableThe current working directory can be displayed:echo ${PWD}In fact the whole environment table can be displayed.
$IFS: String of characters which are used as word separators in command line ( space,tab, newline chars).
8/8/2019 Unix Essentials V2.0
13/45
13 December 2010TCS Internal 13
Environmental Variables (EVs) ...
The MAIL Variable
Names the standard file where your mail is kept Variables for AbbreviationIt is useful if frequently referred names are abbreviated: d=/very/long/directory/name
The .profile FileSome of above variables like HOME and LOGNAME are set automatically each time theuser logs in. The others, however, have to be set. The .profile is used for this purpose asit is executed as soon the user logs in. A sample .profile file would look like:PATH=.:/bin:/usr/binexport HOME PATH PS1 MAIL
8/8/2019 Unix Essentials V2.0
14/45
13 December 2010TCS Internal 14
Any Questions ?
8/8/2019 Unix Essentials V2.0
15/45
13 December 2010TCS Internal 15
Utilities and Applications
UNIX utilities or commands are collection of about 200 programs (ed, sed,awk, vi, grep, make, link, debug, etc.) that service day-to-day processingrequirements. They are invoked through the shell, which is itself anotherutility.
More than a 1000 UNIX-based application programs like DBMS, word
processors, language processors, accounting software, etc. are available fromindependent vendors
8/8/2019 Unix Essentials V2.0
16/45
13 December 2010TCS Internal 16
The UNIX Shell
It is the interpreter of UNIX to decipher and execute commands - user interface tothe kernel for isolating the user from the knowledge of kernel functions.
Maintains interactive dialogue with the user and is capable of Input/OutputRedirection. Can do background processing so that time-consuming, non-interactivetasks can proceed side by side.
Makes it possible to connect a couple of commands with one command taking inputfrom another (pipes). The commands that can be connected in this way are called
filters, because they filter or manipulate data in different ways. Pipes and filters formthe central feature of the UNIX philosophy. Shell variables can control the behavior of shell as well as other programs by storing
data in them.
8/8/2019 Unix Essentials V2.0
17/45
13 December 2010TCS Internal 17
The UNIX Shell ...
Shell includes programming language features which can be used to build shell scripts forperforming complex operations.
Shell scripts are frequently used sequence of shell commands stored in a fi le; file-name can beused later to execute the stored sequence.
Types of Shells - Bourne Shell - original command interpreter developed at AT&T by Stephen R.Bourne; fastest official shell distributed with UNIX systems (executable filename sh)
C Shell - developed by William Joy and others at UCB; gets its name from C due to syntaxresemblance of its programming language (executable file name csh)
Korn Shell - developed by David Korn, combines best features of both shells, not popular(executable file name ksh)
Restricted Shell - restricted version of Bourne shell, typically used for guest logins and in secureinstallations (executable file name).
8/8/2019 Unix Essentials V2.0
18/45
13 December 2010TCS Internal 18
Shell variables
Data in shell scripts and environment settings stored in variables
Conventionally all upper-case
Shell variables exist only in current shell instance
Environment variables passed to subshells
Shell variables can be exported into environment
8/8/2019 Unix Essentials V2.0
19/45
13 December 2010TCS Internal 19
Shell features
Command-Line expansion/ substitution
Command Completion
Command-Line Editing
Aliases
Shell Functions Command Recall
History search
Job Control
8/8/2019 Unix Essentials V2.0
20/45
13 December 2010TCS Internal 20
Getting HelpGetting Help
Check the manual pages!
man man - help on the man commandman who - help with the who command
info ls
ls --help
8/8/2019 Unix Essentials V2.0
21/45
13 December 2010TCS Internal 21
Some Shell Commands
Command Function pwd print working directory cd change directory mkdir make directory rmdir remove directory ls list contents of directory cat display contents of files cp copy one file to another rm remove files mv change name of a file/directory
or move file from one directory to another date display system date who display logged-in users names echo display string; used for prompt ln create another link to a file
8/8/2019 Unix Essentials V2.0
22/45
13 December 2010TCS Internal 22
The UNIX File System (UFS)
It is a hierarchical collection of 3 types of files: ordinary, directory and special files (device, pipe, fifo,socket).
A UNIX file is featureless because it is simply an array of bytes. Dominant file type in UNIX is the text file. System related files are also stored in text form. Separate device can be added by creating a file for it.
Root is the supremo and is represented by the /. Every subdirectory must have a parent. File names can be up to 14 characters long; can contain both upper and lower case alphabets, digits,
a dot, hyphen (-), underscore (_) anywhere; should not have a blank or tab; are case-sensitive. Path names are a sequence of directory names separated by /. They are used to access files. Absolute pathname - file location is determined with respect to the root. Relative pathname - file location is determined with respect to the current directory.
8/8/2019 Unix Essentials V2.0
23/45
13 December 2010TCS Internal 23
The UNIX File System
/bin
/dev/etc
/mnt
bin shetcgamesdictincludelib
mansbinsharetmp
root ( / )
vi/awk
cat/cutchmod
date/sort
echo
ed/sed
grep/tarkill/pwd
ln/ping
mv/ps/ls
rm/rmdir
Though the UFS looks hierarchical, it is actually a directed acyclic
graph because files can be shared.
audiobeep
cpu
fd0 to
fd7
ram
ttyxx
aliaseshosts
init.d
mknod
motdnews
passwd
uucp
/usr
/home/tmp
/var/sbin/proc
lost+foundcrashftplibmail
opttmpspool
cron
faxnewsmailmqueueuucp
/lib ntpuser1to 4
lost+foundntpusr
8/8/2019 Unix Essentials V2.0
24/45
13 December 2010TCS Internal 24
Important Directories in Linux File System
/home It holds users home directories. In other UNIX systems, this can be /usr directory./bin It holds many of the basic Linux programs; bin stands forbinaries, files that are executable./usr It holds many user-oriented directories:
bin It holds user-oriented Linux programs.sbin It holds system administration files.spool It has several subdirectories:
. mail holds mail files
. spool holds files to be printed
. uucp holds files copied between Linux machines.
docs various documents including useful Linux infoman man pages accessed by typing the man games the fun stuff!
/sbin It holds system files that are usually run automatically./etc It and its subdirectories hold many of Linux config files./dev It holds device files.All info sent to /dev/null is thrown into trash. Your terminal is one of the /dev/tty
files.
8/8/2019 Unix Essentials V2.0
25/45
13 December 2010TCS Internal 25
File Access Permissions (FAP)
FAP refer to permissions associated with a file with respect to the following:
File Owner, Group Owner, Other Users.
Read r Allows displaying /copying
Write w Allows editing/deleting
Execute x Allows execution of a file
The protection on a file is referred to as its file mode on UNIX systems. The long version ofls command also displays FAP in addition to user and group ownership. The first characterindicates the file type, a hyphen for a plain file, and a dfor a directory; rest of the bytesmay be rwxrwxrwx or a - for r/w/x if that particular permission does not apply.
8/8/2019 Unix Essentials V2.0
26/45
13 December 2010TCS Internal 26
File Access Permissions (FAP) ...
Meaning of 0123456789 FAP bytes of a directory:
No read does not allow list or remove directory.
No write does not allow copy files to directory, remove files from directory, rename files indirectory, make a subdirectory, remove a subdirectory, move files to/from directory.
No execute does not allow display from within directory, change to directory,display a file in dir, copy a file to/from dir.
Byte 0 is b (block), c (character) or p (device) for special files.
8/8/2019 Unix Essentials V2.0
27/45
13 December 2010TCS Internal 27
Any Questions ?
8/8/2019 Unix Essentials V2.0
28/45
13 December 2010TCS Internal 28
Filters and Pipes
Filter is a program that takes its input from standard input file, processes it, and sends its output to
standard output file (simple filters: head, tail, cut, paste, sort, uniq, etc; advanced filters: grep, pg, wc, tr,awk). The grep filter(global search for regular expressions) searches files for a particular pattern. The format of
grep commandis:grep [options] reg-expr filenamesOptions:-n prints each line matching the pattern with line no.-c prints only the count of lines that match the pattern.-v prints out all those lines that do not match the pattern.Specifying grep regular expressions
\c Backslash turns off any special meaning of character c.^ Pattern following it must occur at the beginning of line.$ Pattern preceding it must occur at the end of line.. Matches any single character[ ] Matches any one of chars in ; ranges like a-z are legal[^] Matches any single char not in ; ranges are legal.r1r2 Concatenation of regular expressions: r1 followed by r2
8/8/2019 Unix Essentials V2.0
29/45
13 December 2010TCS Internal 29
Filters and Pipes ...
pgdisplays a large file, a screen-full at a time. wccounts number of lines, words, characters tris used to translate one set of characters to another; does not accept a filename as an
argument but only the redirected file input. cut(vertical file slit) - extracts only the desired columns from the output of certain
commands.paste (vertical file paste) - what you cutwith previous cmd can be pasted back with thepaste cmd. But while catpastes more than one file horizontally,paste does it vertically.
pipes (|)combine filters and commands such that standard output of one can be sent asstandard input to another.
Advantage of pipes: UNIX tools (commands) can be combined rather than be rewritten.
8/8/2019 Unix Essentials V2.0
30/45
13 December 2010TCS Internal 30
Filters and Pipes - awk
Pattern Scanning and Processing Language
$ awk program filenames whereprogram is:
pattern { action }
pattern { action }
# By Aho, Weinberger, Kernighan
awkreads the input in the filenames one line at a time. Each line is compared with each
pattern in order; for eachpattern that matches the line, the corresponding action isperformed.Awkdoes not alter its input files.
Patterns can be regular expressions exactly as in egrep, or C-like complicated conditions.
8/8/2019 Unix Essentials V2.0
31/45
13 December 2010TCS Internal 31
awk ...
Example 1
$ awk /regular expression/ { print } filenames
$ awk /regular expression/ filenames Both print every line that matches the regular expression
Example 2
$ awk { print } filenames
does exactly what catdoes, albeit more slowly.
Example 3
$ who | awk { print $1, $5 }prints names of logged-in users and time of login.Awksplits each input line automaticallyinto fields strings) separated by blanks or tab.
8/8/2019 Unix Essentials V2.0
32/45
13 December 2010TCS Internal 32
sed
Stream editor
Reads a file or stream of data, writes out the data, performing search and replaceinstructions
Uses regular expressions in search string
8/8/2019 Unix Essentials V2.0
33/45
13 December 2010TCS Internal 33
Using sed
Sed addresses
sed s/TCS/Tata Consultancy Services Ltd/g company.txt
Sed 1,50s/TCS/Tata Consultancy Services Ltd/g company.txt
Sed s/[Tt]cs/TCS/g company.txt
Multiple sed instructions
8/8/2019 Unix Essentials V2.0
34/45
13 December 2010TCS Internal 34
Any Questions ?
8/8/2019 Unix Essentials V2.0
35/45
13 December 2010TCS Internal 35
The viEditor
It is a visual editor used to enter and edit text files
Invoking viwith/without filename puts it in command mode:
$ vi []
vicommand mode syntax:
[]
viworks in three different modes:
y Input - where any key is entered as texty Command - where keys are used as commands to act on text
y ex- where ex commands can be entered in last line to act on text
8/8/2019 Unix Essentials V2.0
36/45
13 December 2010TCS Internal 36
Relationship between 3 Modes
Command
Mode
Input
Mode
exMode
8/8/2019 Unix Essentials V2.0
37/45
13 December 2010TCS Internal 37
Input Mode Commands
Command Actiona Appends text to right of cursorA Appends text at end of linei Inserts text to left of cursoro Inserts blank line below + inserts textO Inserts blank line above + inserts textrx Replace current character with charx
Rtext Replaces character with texts Replaces character under cursor with
many charactersS Replaces entire line20i- followed by or enters
20 hyphens (-) in one line
8/8/2019 Unix Essentials V2.0
38/45
13 December 2010TCS Internal 38
Save and Exit from vi
Command Action
ZZ saves file only ifvistarted with a filenameSave and Exit Commands ofexMode
:w save file and remain in edit mode
:wq save file and quit edit mode
:w write buffer to filename
:q quit editing mode when no changes
are made
:q! quit editing mode but after
abandoning changes
:x save file and quit editing mode
8/8/2019 Unix Essentials V2.0
39/45
13 December 2010TCS Internal 39
Cursor Movement Commands
Command Actionh or backspace Moves Cursor Leftlor spacebar Moves Cursor Right
j Moves Cursor Downk Moves Cursor UpnG Go to line numbern$ Moves cursor to end of line
w Moves cursor to next wordb Moves cursor back to previous worde Moves cursor forward to end of word
Note: W, B, E perform functions similar to w, b, e except that punctuation isskipped)
8/8/2019 Unix Essentials V2.0
40/45
13 December 2010TCS Internal 40
Paging and Scrolling Commands
Command Action
^d Scroll down half screen^u Scroll up half screen^for Full Page forward (12 lines by^b or Full Page backward default)^l Redraw Screen[[ Scroll to beginning of document
]] Scroll to end of document{ Scroll to previous paragraph
} Scroll to next paragraph(or) Scroll to previous or next sentence
8/8/2019 Unix Essentials V2.0
41/45
13 December 2010TCS Internal 41
Search and Repeat-Search Commands
Command Action/pat Search forward for patternpat?pat Search backward for patternpatn Repeat search in previous search
direction (no repeat factor)N same as n but in opposite directionfxortx Move cursor forward to or before the
first occurrence of characterxin current lineFxorTx Same as above but move backward
(repeat factor works with all f/F and t/T); or, Repeat last character search in same
or opposite direction (by f/F/t/T) onlyin current line (repeat factor works)
8/8/2019 Unix Essentials V2.0
42/45
13 December 2010TCS Internal 42
Deletion and Modification Commands
Command Action
dw Delete Current Worddd Delete Current Lined$orD Delete from cursor to end of linecw Change Current Wordcc Change Current Linec$orC Change from cursor to end of line
x Delete character under cursorX Delete character before cursor
J Join current, next lines (also nJ).(dot) Repeat last editing instructionu Undo single last changeU Restore all changes to line since
cursor moved to it
8/8/2019 Unix Essentials V2.0
43/45
13 December 2010TCS Internal 43
Commands to Move or Copy Lines
[][n]ddCut (delete) n lines starting from current line
[][n]yyCopy n lines starting from current line
[]pPut yanked text after current cursor position
[]PPut yanked text before current cursor position
Note: Named-buffer is useful for copying an area from one file into another. Open
some files simultaneously by vi. Mark an area in one file by ddoryy; move toanother file (by :next) to paste (byp orP) the named area; then say :rewind toreturn to the parent file.
8/8/2019 Unix Essentials V2.0
44/45
13 December 2010TCS Internal 44
Customizing vi
vican be customized as per users requirements using ex-mode commands.exCommands Action:set all Display all setoptions; those pre-fixed with no are not operative:set autoindent(aifor short)
Extremely useful to programmers forindentation of lines
:set number(nu for short)Displays all lines with numbers which are not preserved on
saving file:set nonumber(nonu for short) - Reverses number setting:set tabstop=6(ts for short)
Changes default tab setting (8 spaces):set ignorecase (icfor short)
ignores case while pattern matching
8/8/2019 Unix Essentials V2.0
45/45
13 December 2010TCS Internal 45
Customizing vi ...
:set showmatch (sm for short)
Helps locate matching brackets when ) or } is entered in input mode; beeps when no match found to alertfor correction
:set autowrite (awfor short)
Writes current file automatically whenever switching files with :n and escaping to shell with :sh
:set showmode - Displays a message when vigoes to input mode
:next(n for short) Moves to next file opened in vi
:rewind Comes back to parent file
All sets can be stored in .exrcfile used by vifor its startup instructions.
Equivalently an environment variable, EXINIT can be assigned to store the settings:
$EXINIT=set nu tabstop=6 ignorecase
It is better to make this assignment in the .profile so that it is available for all sessions.