Introduction to Shell Scripts

A script is a group of commandsstored in a file

The file should be executable: chmod +x myScript

The 1st line in refers to the shell ie: /bin/bash

A “Hello World” Script

#!/bin/bash#File: hello.shecho “Hello World

#running a script:hello.sh

Receiving Arguments#!/bin/bash#args begin at $0 and go to $9echo $3echo $2echo $1echo $0#above $9 use the form ${nnnn}echo ${10} ${7}

(We may talk about the shift cmd)

Passing Arguments

The shell automatically expands the command line with files

myScript file1 ~king/Y* *.c *.txt day[1-9].inf

Missing arguments are assumed to be NULL

Special Variables

#!/bin/bashecho “Name of this script: “ $0echo “Number of args: “ $#echo “All the args as a single string: $* ” #does not include $0

echo “All the args as a list of strings: $*” #does not include $0

echo “Process ID: “ $$echo “Parent Process ID: “ $PPIDecho “Exit status of the last command: $?”echo

Environment Variables

By default, all variables are local to the script or shell in which they occur.

To make a variable global – export it

export PATHexport USERID=”FRED27”

What's the Difference?When a program or a subshell is executed, copies of all

global variables are passed to the subshell.

Local variables are not passed

#!/bin/bash#File vegas.shx=10export y=11other.sh

#!/bin/bash#File other.shecho “x is not defined: $x y is defined: $y”

Unless exported, what's done in vegas.sh, stays in vegas.sh

Special Environment VariablesPROMPTS

PS1 This the command line promptPS2 The command line continuation prompt

PS3 Prompt for the select statementPS4 Command Trace prompt

PS1=”Hello> “PS2= “Continue...>”Hello > Enter your next command \Continue...> continue your command

(we'll cover PS3 and PS4 later)

Data TypesThere are only 2 datatypes for environment variables: string and integer.

You can declare them using the typeset builtin, but its not req'd at a beginner stage. Shell variables are not strongly typed. If a value looks like an integer you can treat it like an integer or a like a string, depending on your needs.

typeset -i myInteger

Manipulating Integers IThere are 4 ways to do a calculation.* The first uses

double round brackets. Any variable inside the brackets does not require a $ to dereference the value, however to use the result of an expression you need to put a $ in front of the brackets.

y=3(( x = 2 * y + 7 ))

z=$(( 2 * x ))echo $((++x))

*The reason is historical. In the 35+ years of Unix there have been various shells with different capabilities. The bash shell includes these for reasons of backward compatibility with its most popular predecessors.

Manipulating Integers II

The 2nd syntax uses the let statement

let x=7let y=++xlet z=x*ylet k=z+5

echo “x: $x y: $y z: $z k: $kx: 8 y: 8 z: 64 k: 69

Note: The variable on the right side does not use a $ to dereference it

You can use C's Operators

Arithmetic

+ - * / ++ - - (both pre and post forms)%= += -= *= /= %=

( )

Relational (these return 1 or 0)

> < == != >= <=

Logical

&& || !

But Not These Operators

? : (the triadic operator) & (address of operator) * (value pointed to by) sizeof

Bit Manipulation

<< >> & | ^ ~

Manipulating Integers III

The 3rd syntax, which is probably the least powerful, uses a utility known as expr. Each operator and operand is treated as an arg to the utility, and must therefore be separated with a whitespace character. Arithmetic operators are limited to: + - * / and %. Comparison operators are allowed, but <, >, ( and ) must be quoted or escape quoted so that they are not interpreted by the shell.

expr 5 + $xexpr 3 '<=' 2 + 7 y=`expr $x + $z`

More Complicated CalulationsFor more difficult calculations, or for using real numbers use bc (basic calculator) and a here file. The result returned to the shell is treated as a string, not a number!

x=`bc <<EOF5+2^.5+ $yEOF`

Here the ^ is used for exponentiation and takes the square root of the number. bc is more powerful in that it handles both integers and reals, temporary variables and extended precision arithmetic (hundreds of digits long.

Manipulating Strings (I)

Assignment is easy:

x=Helloy=”Hello World”z=$x

Concatenation is also easy:

echo $x$yecho $x” George, “$y

String Operators

In any environment you need to:Copy a string x1=$x2

Concatenate strings x1=$x2$x3 Retrieve a substring x2=${x1:2:5}get the length of a string echo ${#x1}

Manipulating Strings II

Bash has 3 other interesting features

1. The expr utility can handle strings including the ability to find the position of a

string in a substring 2. The :- := and :? operators can be used to assign default values or actions to strings if they are undefined 3. A subset of regular expression pattern matching and substitution can be achieved using the %, #,

%% and ## operators

For more info, see the Advanced Bash Reference by Meyer Cooper (posted on my web site)

The 4 types of quote● Double Quotes: allows quoting of whitepaces “Hello $USER”

● Single Quotes – Prevents expansion of metacharacters 'Hello $125.00'

● Back Quote – evaluates the commands inside list=`who | cut -c1-8`

● BackSlash – quotes a single character or a hex or octal value \t

Input – the read cmd

read lineread value1 value2 value3read a b c < myFile

The values read in are from stdin. Each value read in is separated by a whitespace character on the input line and is assigned to the next variable in the list - except for the last variable – its value is the whole rest of the line.

read options

read -p “Enter a value: “ myValueread -s -p “S means silent. Passwd?” passread -d. -p “Multiline input. End with . “ sentenceread -t 3 -p “3 seconds to self destruct... ” xxxread -n 1 -p “Enter a single character response: “

Special BASH feature: <<<

Here's one that's new to me. You can use triple < to send a variable to a command as if it were a file or a 1 line “here” file.

x=”The rain in Spain”wc <<< $x1 4 18

this is the same as:echo $x | wc

Output

Most Unix commands send output to stdout and stderr (ie: the screen)

Sometimes we want to hide this, either by redirecting to a file or /dev/null

cat -n myFilels -l > dirListcat Feb March 2>/dev/null

Secret Slide

If we've redirected stdout and stderr, we can still direct output to the screen by directing output to /dev/tty

echo “My Message” > /dev/tty

echo

The echo command is for printing text and the value of variables

echo “Hello World: $USER”echo -n “-n suppresses a new line

printf

printf fmt value1 value2 value3 ..

printf uses many of the same format codes used by c: types: %x %o %d %f %lf %e %g %smodifiers: + - 0 # *

width and precision: ie: %12.5lf

Simple Program – Add 2 Numbers

#!/bin/bash#File: add.sh#Description: Prompts and adds two numbersread -p “Enter 1st number: “ number1read -p “Enter a 2nd number: “ number2result=`bc <<EOF $number1 + $number2EOF`echo "The sum is: $result"

echo "Using expr: `expr $number1 + $number2`"

Adding Dialog Boxes

Linux dialog box functions for use in scripts:

dialog presents a text only version.

kdialog only works in X-Windows and presents a gui

gdialog will present a graphic version in X, but will downgrade to text only if X is not detected

(Class discussion: merits and other alternatives)

Some exampleskdialog --yesno “Should we continue” 2>/dev/null

echo $?

kdialog --password "Enter password " 10 50 2>/dev/null

dialog --title Greeting - - infobox “Hello World” 3 20

gdialog --title "My List" --checklist "Items" 10 30 3 \ a Steak on b Strawberries off c Milk off \

d Rutabage off f Aspirin on \ g Icecream on

Results from dialogs

Status codes can be found in $?. 0 means success, 1 means No, 2 means cancel

dialog sends output to stderr, kdialog and gdialog sent output to stdout

dialog is text only and currently has the most features. gdialog automatically uses the X gui when available, but create a text box if not. kdialog currently has

more types of dialogs than gdialog but less than dialog. It also generates some (non-important) error codes

which must be discarded into /dev/null.

Secret Slide

Here's a trick when trying out commands:

PS1='`echo “$0> “`'

The status of the last command will appear in the command prompt

v

A shell script can do many of the things a C program can do – and if it can't we can write a utility in C or another language to do it.

Shell scripts are interpreted and slow

But they are quick to write and can easilyglue different commands together.

(Why is that?)

while loopswhile [ condition ]do cmd1 ... cmd2 ... ... done do and done are treated as if the are separate statements. Either they appear on separate lines or with a semicolon:

while [condition]; do; stmt; stmt; done

until loops

until [ condition ]do cmd1 ... cmd 2 ... ...done

until [ condition ] is identical towhile [ ! condition ]

Sample while loop

#Loop until EOF (or read fails)while read linedo echo “Value input: “ $line usleep 40000done

for x in listfor x in 1 2 4 9 25 36 49doecho The value of x is $xdone

for x in “$@”do echo cmd line argument $i is $x (( i += 1 ))done

Note: for x in “$*” only iterates once

More examples: for x in a list

for variable in hello 75 good-bye -1do echo $variabledone

names=”Joe Helen Kris Sam”for name in $namesdo echo $name done

Counting with for

for ((i=0; i<10;i++))do printf “%4d \n\n“ $idone

If Statement

if conditionthen cmd1 ... cmd2 ...fi

if example

if grep -sq Merlin Yankee.txtthen echo We found Merlin in the textfi

The grep command acts as a condition-s silence. Suppress error messages-q quiet. Suppress found textWhat's left? $? - whether or not grep succeeds

The if statementif conditionthen #do something #for several lineselif condtion2then #do something elseelse #until we find the keyword fifi

then, elif and else must appear as separate statements, much like do and done

Conditions

Any Unix command, C program or Shell Script can be used as a condition

if cd ~/public/htmlthen echo “I've switched to public_html”else echo “Error code: $?” echo “You are missing a public_html directory”fi

ConditionsOf course, if we find an error we should try to fix it

if `cd ~/public_html`then echo “I've switched to public_html”

elif `gdialog --title “Assistance” --yesno “Missing public_html - create?” 5 20`

then mkdir ~/public_html else echo “Sorry - can't continue....” exit 1

fi

Conditions with files

if [ -e “$1” ]thenecho “File $1 exists”

else echo File $1 does not existfi

We test a condition by using [ ]. Using a single set of [ ] will also work. Leave spaces around each operator.

Other File ConditionsFlag True if

-c fileName the file is a a character device, ie a terminal-d fileName the file is a directory-e fileName the file exists -f fileName file is a regular file-G fileName file is owned by your group-L filename file is a symbolic link-N fileName new content (modified time > access time)-O fileName file owned by you -p fileName file is a pipe-r filename the script can read the file-s fileName the file exists and the size > 0-w fileName the script can write to the file-x fileName the script can execute the file

Comparing Two Files

if [ $f1 -ot $f2 ]then echo file $f1 is older than $f2fi

if [ $f1 -nt $f2 ] ;then echo file $f1 is newer than $f2fi

Other File Comparisons

if [ $f1 -ef $f2 ]; then echo $f1 and $f2 are the same file echo The inode value is the same echo They reside on the same file systemfi

if [

Numeric Conditions

if [[ $i -le 10 ]] #i is <= 10then echo $i is smallfi

Other operators: -eq -ne -lt -gt -ge

Combining Multiple Conditions

&& Short Circuit and

|| Short Circuit or

grep Merlin Yankee.txt && grep bucket jargon.txt

(who | grep smith) || echo Smith not logged on

(gcc x.c -o x && x ) || vim x.c

caseecho -n “Chose a, b or c”read choicecase $choice in a) echo You chose a ;; b) echo You chose b ;; [cC]) echo you chose c ;; *) echo you chose none of the above echo “Try again?”esac #yeah, case spelled backwards

Note: Choice is matched against a shell regular expression.

select PS3=”Select an Entree (1-2)select name in "Red Fish" "Blue Fish"do case $name in "Red Fish") echo You chose Snapper break ;; "Blue Fish") echo You chose Tuna break ;; *) echo Try again sleep 2 clear esacdoneecho A fine choice sir $name $REPLY

Exitting a script

exit errCode

Standard Exit Codes

0 Success1 General Error2 Misuse of shell builtin

126 Permission error/can't execute127 Can't execute command130 Terminated by ctrl-C

128+n Terminated by signal n

Writing functionsA function is like a shell script within a shell script.

function f1(){ echo $0 $1 $2 $3 $4 return }

f1 one two three

$0 remains the name of the script file, but $1, $2 and $3 are the values one, two and three.

functions just do stuff – they don't return a value

...unless its through stdout

function max(){if (( $1 > $2 )); then echo $1else echo $2fi}

y=`max 3 5`max y 4echo $y

Tracing Execution

echo $LINENObash -x myScriptcallerset -o functracePS4='`echo “$? Trace: $LINENO”`'

Further Study

Linux Shell Scripting with Bash, Ken O. Burtch, Sams, 2004

Advanced Bash Shell Scripting, Mendel Cooper (online - see my web site)

Further ideas: Cover the exec cmd, ansi control codes

Part II

100 Simple Scripts(In Progress – 67 ideas so far)

Maximum: 7 linesIdeas should be understandable to 2nd

Semester(no advanced admin functions/can reproduce what an existing function

does )

1. Produce a Directory Structure

#!/bin/bashmkdir $1 mkdir $1/Reports $1/Finances $1/logfilescd $1/logFilestouch errors access echo “Created logfile on `date`”>>accesscd $1/Financestouch income overhead capital expenses

2. Randomly select a joke for a .signature file

#!/bin/bashnLines=`wc -l jokeFile`let selectedLine=$RANDOM%nLines+1head -$selectedLine jokeFile | tail -1 \ > .signature

File Management Ideas (17)● Find the oldest (or newest) file in a directory

● List all the directories

● Change the extension of all files of one type to a different type (ie: .txt to .text)

● Copy all files newer than a given date to an archive directory

● Copy all files where content has been changed to an archive directory

● Create a menu of all executable files

● List the top N largest files (with the idea of removing them)

● Convert a batch of image (or sound) files from one type to another

● List all the files I can read (or write or execute) in the current directory

● List the number of files of each type

● Check for the existence of a list of programs (or files) – report on the results

● CheckQuota, identify which disk(s) are nearly full and which users are using too much storage

● GetRange – retrieve a range of lines from a file, ie: lines 10-40

● Sum the total file sizes of a group of files

● Count how many files belong to each group

● Insert the last modification date into a set of files

● Trash – instead of removing files, move them to a trash directory. Trash w/o any entries empties the trash

More File Management Ideas (2)

● List all symbolic links that don't point to actual files

● Look for files in a directory that are exact copies of each other (and remove extra copies)

Terminal Management Ideas(4)

● Write a message to all terminals owned by a particular user

● MsgAll: Allow others to write to any of my current terminals

● Log who is logged on and repeat every 5 minutes

● Who am I allowed to write a message to amongst the users that are logged in

Process Management Ideas (5)

● killAll – kill all my processes except the current shell

● Which processes have the most CPU time

● Count the number of processes owned by each user

● Teeprocess: A version of tee that sends output to a series of processes

● PortMap – check which ports are active on the current system. (Note: this can trigger a security alert)

User Management Ideas (5)

● Nusers: how many people are logged on

● NoPlan – identify which users in a group do not have a .project or a .plan file

● Send a message to a user at their terminal. If they are not logged on, send them an email message

● Who's been logged on in the last n days

● List the last n times I've been logged on (or a specified user)

Web Ideas (8)

● Generate a simple web page

● Generate a web page listing current system statistics

● Convert name/value pairs to environment variables

● Create a response to a user's form

● Log access to a web page

● Summarize access to a web page

● Have a C program generate a web page

● Retrieve a group of web pages from one or more machines

Mail Management Ideas (6)

● Check if I have new unread email

● How many unread emails do I have

● Summarize how many emails do I have from a given user

● Count the emails for a given day

● Count the emails for a given subject

● Generate an email to a list of users

ssh Ideas (4)

● Identify IP addresses are in use on a subnet

● Copy a directory to a group of machines● Collect a file from a group of machines

and consolidate the results● Throw up a display on N different

terminals

Small Database Ideas (3)(mostly these are in the text)

● Add an entry into a rolodex (no duplicates)

● Remove an entry from a rolodex● Extract entries from a given town or

areacode

Text Handling Ideas (7)

● Explode a string into individual characters● Produce a rainbow coloured prompt● Do a hex dump of a string● Produce an ascii table● Envgrep – find an environment variable

that contains a given string● Write a script that does what strings

does● Produce a scrolling banner

Uncategorized (4)

● MyFinger - command similar to finger● Printing out powers of 2● Creating a history (trace) of directories

that you've been in● A version of cd that puts you back to any

specified previous directory