Upload
others
View
6
Download
0
Embed Size (px)
Citation preview
EECS 470 Lab 5Linux Shell Scripting
Department of Electrical Engineering and Computer ScienceCollege of EngineeringUniversity of Michigan
Friday, 7th February, 2014
(University of Michigan) Lab 5: Scripting Friday, 7th February, 2014 1 / 40
Overview
Administrivia
UNIXFilesUtilitiesConnecting Utilities
Bourne Again ShellVariablesFlow ControlFunctionsGlobbing
Scripting
Assignment
(University of Michigan) Lab 5: Scripting Friday, 7th February, 2014 2 / 40
Administrivia
Administrivia
HomeworkI Homework 3 is due Wednesday, 12th February at the beginning of
lecture
Projects
I Project 3 is due Sunday, 9th February at 9:00PMI If you haven’t, you need to get started now
We are available to answer questions on anything here. Office hours can befound in the course google calendar.
(University of Michigan) Lab 5: Scripting Friday, 7th February, 2014 3 / 40
UNIX
UNIX
What is UNIX?I Mainframe operating systemI Written by Dennis Ritchie, Ken Thompson and Rob Pike (among
others) at Bell LabsI The basis for many modern operating systems, e.g. Linux, BSD, Mac
OSX
History of UNIX
I Written at Bell Labs in 1969I First version of BSD is installed in 1974I Last Bell Labs UNIX (Version 7) is published in 1979I The GNU Project is started by Richard Stallman in 1984I Linux Torvalds writes a monolithic kernel operating system in 1991
(University of Michigan) Lab 5: Scripting Friday, 7th February, 2014 4 / 40
UNIX
UNIX Philosophy
“This is the Unix philosophy: Write programs that do one thing and do itwell. Write programs to work together. Write programs to handle textstreams, because that is a universal interface.”
– Douglas McIlroy
“Everything is a file descriptor or a process.”– Linus Torvalds
(University of Michigan) Lab 5: Scripting Friday, 7th February, 2014 5 / 40
UNIX Files
Files
What is a file?I Anything referenced through a filesystemI Anything with a file descriptor
(University of Michigan) Lab 5: Scripting Friday, 7th February, 2014 6 / 40
UNIX Files
Types of Files
I Regular – Anything not in one of the following categoriesI Directory – Can contain other files and directories (read up on inodes
sometime)I Symbolic Link – A pointer to another fileI Pipe – Covered in a few slidesI Socket – Covered in EECS 482/489
(University of Michigan) Lab 5: Scripting Friday, 7th February, 2014 7 / 40
UNIX Files
Permissions
r w x - - - - - -User Group Other
Table: UNIX permissions are representedwith one bit for each permission in eachcategory, e.g. 700.
Three PermissionsI ReadI WriteI Execute
Three Categories
I UserI GroupI Other
(University of Michigan) Lab 5: Scripting Friday, 7th February, 2014 8 / 40
UNIX Utilities
Utilities
What is a utility?
I A program used to process text streams/filesI Called from some command line/shell
Why do I care?
I Utilities form the basis of “Linux skills”I Useful for automationI Necessary for today’s lab
(University of Michigan) Lab 5: Scripting Friday, 7th February, 2014 9 / 40
UNIX Utilities
Navigation Utilities
pwd
I Description: print working directory – where you areI Synopsis: pwd [OPTIONS]
ls
I Description: list directory contentsI Synopsis: ls [OPTIONS]... [FILE] ...
cd
I Description: change directoryI Synopsis: cd [OPTIONS] [PATH]
(University of Michigan) Lab 5: Scripting Friday, 7th February, 2014 10 / 40
UNIX Utilities
File Utilities
cp
I Description: copy filesI Synopsis: cp [OPTONS]... SOURCE DEST
mv
I Description: move filesI Synopsis: mv [OPTONS]... SOURCE DEST
rm
I Description: remove filesI Synopsis: rm [OPTONS]... FILE...
(University of Michigan) Lab 5: Scripting Friday, 7th February, 2014 11 / 40
UNIX Utilities
diff
Description
I Shows the line-by-line differences between filesI Good for checking if your output is correct
Synopsys
I diff [OPTIONS] FILES
Examples
I diff -uy ../project3_correct/writeback.out writeback.outI vimdiff ../project3_correct/writeback.out writeback.out
(University of Michigan) Lab 5: Scripting Friday, 7th February, 2014 12 / 40
UNIX Utilities
Graphical Diff
Alternatives to diffParsing output of diff is hard, so it might be useful to use some kind of“graphical diff” tool, like vimdiff, which opens up two files side-by-side inVim showing their differences. Try it and you’ll see how much easier toparse this.
(University of Michigan) Lab 5: Scripting Friday, 7th February, 2014 13 / 40
UNIX Utilities
grep
Description
I Print lines matching a pattern
Synopsys
I grep [OPTIONS] PATTERN [FILE]
Examples
I grep ’@@@’ program.outI ps -axfuw | grep "$USER" | grep "vcs"
(University of Michigan) Lab 5: Scripting Friday, 7th February, 2014 14 / 40
UNIX Utilities
Regular Expressions
I Really powerful/usefulI Complicated, and beyond the scope of this presentationI Read up on them
I at WikipediaI in the grep manual
(University of Michigan) Lab 5: Scripting Friday, 7th February, 2014 15 / 40
UNIX Utilities
man
Description
I An interface to the on-line reference manuals (pager)
Synopsys
I man PAGE
Examples
I man grepI man diff
(University of Michigan) Lab 5: Scripting Friday, 7th February, 2014 16 / 40
UNIX Utilities
Pager
DefinitionI A program which allows browsing of large text files by breaking them
into screen-sized chunks.
Examples
I lessI man
(University of Michigan) Lab 5: Scripting Friday, 7th February, 2014 17 / 40
UNIX Utilities
Other Utilities
These will be useful. . .
I cutI touchI teeI xargs
I tailI columnI findI less
These are harder, but even more useful. . .
I sedI awkI patchI vi(m)
I fmtI tmux
(University of Michigan) Lab 5: Scripting Friday, 7th February, 2014 18 / 40
UNIX Connecting Utilities
Program Features
Methods of CommunicationI Standard Text Streams
I stdinI stdoutI stderr
I Return value/code
(University of Michigan) Lab 5: Scripting Friday, 7th February, 2014 19 / 40
UNIX Connecting Utilities
Return Codes
What is a return code?I The integer value a program returns (e.g. return(0);)I Conventionally, returning zero indicates success, non-zero failureI Specific values other than zero mean different things for different
programs
(University of Michigan) Lab 5: Scripting Friday, 7th February, 2014 20 / 40
UNIX Connecting Utilities
Standard Text Streams
stdin
I The default input to a programI From a keyboard by default
stdout
I The default output of a programI To a display by default
stderr
I The default error output of a programI To a display by default, requires additional effort to save
(University of Michigan) Lab 5: Scripting Friday, 7th February, 2014 21 / 40
UNIX Connecting Utilities
Connecting Utilities
Why do we want to connect utilities?
I Combination jobs without intermediate filesI e.g. take the diff of two different grep operations (what you need to
do for today’s lab)
How can we connect utilities?I PipesI Redirection
(University of Michigan) Lab 5: Scripting Friday, 7th February, 2014 22 / 40
UNIX Connecting Utilities
Pipe
What is a pipe?
I Connects the stdout of one program to the stdin of anotherI Does not connect stderr
How do I use one?I Call a program on one side of the | and then call another on the other
sideI e.g. dmesg | less
(University of Michigan) Lab 5: Scripting Friday, 7th February, 2014 23 / 40
UNIX Connecting Utilities
Pipe: xargs
ProblemWhat if we want to pipe to utility that uses arguments instead of input?
Solution: xargsxargs splits input into individual items and calls the program that is itsargument once for each input.
(University of Michigan) Lab 5: Scripting Friday, 7th February, 2014 24 / 40
UNIX Connecting Utilities
Redirection
What is redirection?I Allows for modification of the standard text streams
I stdin is 0I stdout is 1I stderr is 2
I Several types:0< Use a file instead of the keyboard for stdini> Use a file instead of the terminal for the stream i
i>> Like i>, but append to a file instead of overwritingi>&j Put stream i into the same place as stream j
Advanced Bash Scripting Guide: I/O Redirection
(University of Michigan) Lab 5: Scripting Friday, 7th February, 2014 25 / 40
UNIX Connecting Utilities
Redirection by Example
Example
I ./vs-asm < test_progs/evens.s > program.mem
Example
I make | tee 2>&1 build.out
Example
I ./test 1>&2 2>&3
(University of Michigan) Lab 5: Scripting Friday, 7th February, 2014 26 / 40
Bourne Again Shell
Shell
What is a shell?I Before we had graphical environments, we had text shellsI Basically, an interpreter for commands, executing programs and saving
informationI Possibly a Read-Execute-Print-Loop (REPL)
(University of Michigan) Lab 5: Scripting Friday, 7th February, 2014 27 / 40
Bourne Again Shell
Bourne Again Shell
What is BASH?I Stands for the Bourne Again ShellI Created in 1989 by Brian FoxI Default shell in most Linux distributions and Mac OSX
Why BASH?
I What I learned firstI Default in CAEN Redhat, finally
(University of Michigan) Lab 5: Scripting Friday, 7th February, 2014 28 / 40
Bourne Again Shell
Warning
WarningEverything after this slide will be specific to BASH. Other shells behavesimilarly, but not identically. If you want to use something else (e.g. ZSH,TCSH, etc.), please find other resources.
(University of Michigan) Lab 5: Scripting Friday, 7th February, 2014 29 / 40
Bourne Again Shell Variables
Variables
BASH VariablesI Store dataI Contain text, for the most partI No type system
Syntax
I Assignment/DeclarationI variable=value
I ReferencingI $variableI ${variable}
(University of Michigan) Lab 5: Scripting Friday, 7th February, 2014 30 / 40
Bourne Again Shell Variables
Variable Scope
Scope
I Variables exist inside the shell they’re inI Unless exported
e.g. export EDITOR=vimI This is very important in scripts, particularly the shell startup scripts
(.bashrc, .bash_profile)
(University of Michigan) Lab 5: Scripting Friday, 7th February, 2014 31 / 40
Bourne Again Shell Variables
Special Variables
$# The number of command line arguments$0 The name of the script/function called$1 The first argument to the script/function$? The return code of the last program run in this shell
$USER The current user$HOME The user’s home directory
(University of Michigan) Lab 5: Scripting Friday, 7th February, 2014 32 / 40
Bourne Again Shell Flow Control
Flow Control
if/else
if [ var -eq "string" ];then
command1elif [ var -eq "string2" ];then
command2else
command3fi
(University of Michigan) Lab 5: Scripting Friday, 7th February, 2014 33 / 40
Bourne Again Shell Flow Control
Flow Control
case
case "$var" inval)
command;;val2)
command;;( * )
default;;
esac
(University of Michigan) Lab 5: Scripting Friday, 7th February, 2014 34 / 40
Bourne Again Shell Flow Control
Flow Control
for
for file in ./*; docommand $filecommand2
done
for (( a=1; a <= LIMIT; a++ )) docommandcommand2
done
(University of Michigan) Lab 5: Scripting Friday, 7th February, 2014 35 / 40
Bourne Again Shell Flow Control
Conditionals
Testing
I Testing happens in [ ]I String tests are different than arithmetic testsI Generally use -lt, -gt, -le, -ge, -eq, -neI Tests for files are special, e.g. [ -x simv ] makes sure that the
binary simv has execution permissions
(University of Michigan) Lab 5: Scripting Friday, 7th February, 2014 36 / 40
Bourne Again Shell Functions
Functions
FunctionsI Packages of commandsI Arguments are referenced by positionI Useful for packaging up commonly reused bits
Syntax
function (){
commands}
(University of Michigan) Lab 5: Scripting Friday, 7th February, 2014 37 / 40
Bourne Again Shell Globbing
File Globbing
What is globbing?
I File name wildcarding in the shellI Expansion is done, and then passed to the command to be executedI e.g. test_progs/*.s
What should I know globbing?
I Superior to parsing ls output in everywayI Can get more complicated, see this page of the BASH manual.
(University of Michigan) Lab 5: Scripting Friday, 7th February, 2014 38 / 40
Scripting
Scripting
1. Write a series of shell commands into a text file2. On the first line of the file, specify an interpreter
e.g. #!/bin/bash3. Name it something appropriate
e.g. test.sh4. Add execute permissions
e.g. chmod +x test.sh
5. Run the scripte.g. $ ./test.sh
(University of Michigan) Lab 5: Scripting Friday, 7th February, 2014 39 / 40
Assignment
Lab Assignment
I Assignment is posted to the course website as Lab 5 Assignment.I If you get stuck. . .
I Ask a neighbor, quietlyI Put yourself in the help queue
I When you finish the assignment, sign up in the help queue and markthat you would like to be checked off.
I If you are unable to finish today, the assignment needs to be checkedoff by a GSI in office hours before the next lab session.
(University of Michigan) Lab 5: Scripting Friday, 7th February, 2014 40 / 40