Shell Script &Perl Programming

Shell Programming

• A shell script is a text file that contains Linux (UNIX) commands, which you enter using any standard editor.

• You can then execute the commands in the file by using the filename as an argument to any dot (.) command.

Our first shell script$ vi helloscript

i# this is a simple scriptecho “hello world”<esc>:w:q$ ./helloscripthello world$

Perl Variables

• The primary distinction in Perl is not between data types (e.g. string, integer, real) but rather between “singular” and “plural” data.• Singular (scalar): strings and numbers• Plural (array): lists of strings or numbers

• Variables do not need to be predefined

Variable example

$welcome = “Hello world \n”; #set a variableprint $welcome #print the variable

Notice that - we don’t predefine the variable- the $ tells Perl that this is a scalar variable

Perl variable typesType Character example Is a name for…Scalar $ $centsAn individual value

(number or string)Array @ @largeA list of values,

keyed by numberHash % %interest A group of values,

keyed by stringSubroutine & &calcval A “callable” piece

of Perl codeTypeglob* *struck Everything named

struck

Another Variables Example

$ greeting=“hello”$ echo $greetinghello$ echo greetinggreeting$

The set command will let you view all of your currently defined variables.

Some interesting variables

• Use the set command to see some of your defined variables (e.g. after you assign the value to “greeting” in previous slide).

• Also check out some other important variables (type “set | more”)• PATH• PS1

Starting to customize your settings

• In your home directory (“/home/<username>) create a file called ‘.bash_profile’

• Insert into this two lines:• PATH=/usr/local/sbin:/usr/bin:$PATH:$HOME/bin:.• PS1=‘> ‘

• The first tells the system where to look for programs (commands) to execute, the second sets your prompt• Default probably PS1=‘\s-\v\$ ‘, use set to see

• \s tells it to print what shell you are using• \v tells it to print what version• $ is the prompt you are telling it to use

Another variable example> school=“UTSA”> whereat=“you are at $school”> echo $whereatyou are at UTSA>

What would happen if

> echo whereatwhereat>

Variables and scripts

> vi greetvarecho Please enter a greeting: read greetingecho “The greeting you entered was $greeting” #note optional quote> greetvarPlease enter a greeting:howdyThe greeting you entered was howdy>

Arguments to scripts

> vi greetargsecho The first argument of $0 is: $1echo the second argument is: $2echo the third: $3echo and finally: $4> greetargs num1 number2 N3 “simply N4”The first argument of ./greetargs is: num1the second argument is: number2the third: N3and finally: simply N4>

greetargs num1 num2 num3The first argument of ./greetargs is: num1the second argument is: num2the third: num3and finally:>

Some simple math

> vi add1let “num = $1” # quotes importantlet “num = num + 1”echo $num> add1 23>

Some loops

> vi loop3again = 1while let “again <= 3 “ # you need the let do echo $again howdy let “again = again + 1 “ done> loop31 howdy2 howdy3 howdy>

Conditional statements> vi mylsif [ $1 = a ] # watch spaces, need after [ and before ] then ls -al # lists all files including hidden else ls -l # lists files in long format fi> myls a<lists files in long format including hidden>> myls x<list files in long format>> myls./myls: [: =: unary operator expected<lists files in long format>>

Reading from a file

> vi printlistread namewhile [ $name != end ] do echo $name read name done> printlist < listLine1Line2Line3> printlist list

#will have to ^c

> vi listLine1Line2Line3end>

Script for adding users

#this is a script to create a bunch of accountsread grpidread namelet "userid = grpid + 1“while [ "$name" != end ] do echo $name $userid $grpid adduser -u $userid -g $grpid -d /home/$name -p "" -r $name mkdir /home/$name chown $name /home/$name let "userid = userid + 1" read name done

THE FILE500AbarshayJbellAbelvisend

Perl

• Practical Extraction and Report Language• Designed to provide convenience of shell scripts and

power and flexibility of a programming language.• Perl programs are interpreted and executed directly like shell

scripts.• Similar in many respects to the C programming language

• Freeware, available from Free Software Foundation (comes with Linux)

Our first Perl Program$ vi sample#!/usr/bin/perl$inputline = <STDIN>; # input textprint ("$inputline"); # output the text$ ./sampletest linetest line$

- # symbol indicates comment, except #! in first line which indicates the location of the program interpreter.- notice each line ends with a semi-colon- variable treated in a similar manner as in shell scripts

Quotes and Escape Sequences

#!/usr/bin/perl$x = "a string";$y = "This is $x"; # becomes "This is a string”$z = 'This is $x'; # remains 'This is $x’print ("$x\n");print ("$y\n");print ("$z\n");print ('$z\n');print ("\a\LFREDDIE \Umary\n");print ("a quote \"in a string\"\n");

$ ./samplea stringThis is a stringThis is $x$z\nfreddie MARYa quote "in a string”$

Some more escape sequences\a Bell (beep)\b backspace\e escape\E cancel effect of \L, \U, \Q\L all following letters are lowercase\n newline\Q do not look for special pattern characters\r carriage return\t tab\U all following letters are uppercase\v vertical tab

Performing Math$ vi math#!/usr/bin/perl$a = 15; print ("$a\n");$a = 4 + 5.1; print ("$a\n");$a = 17 - 6.2; print ("$a\n");$a = 2.1 * 6; print ("$a\n");$a = 48 / 1.5; print ("$a\n");$a = 2 ** 3; print ("$a\n");$a = 21 % 5; print ("$a\n");$a = - $a; print ("$a\n");$

$ ./math159.110.812.63281-1$

Performing Comparisons11.0 < 16 # less than16 > 11 # greater than15 == 15 # equals11.0 <= 16 # less than or equal to16 >= 11 # greater than or equal to15 != 14 # not equal to$a || $b # logical OR; true if either is non-zero$a && $b # logical AND; true only if both are non-zero! $a # logical NOT; true if $a is zero4 <==> 1 # returns 1; 4>13 <==> 3.0 # returns 0; they are equal1 <==> 4.0 # returns -1; 1<4

Performing Comparisons - strings“aaa” lt “bbb” # less than“bbb” gt “aaa” # greater than“aaa” eq “aaa” # equals“aaa” le “bbb” # less than or equal to“bbb” ge “aaa” # greater than or equal to“aaa” ne “bbb” # not equal to

“aaa” cmp “bbb” # similar to <==>, returns 1; “aaa” < “bbb”“aaa” cmp “aaa” # returns 0“bbb” cmp “aaa” # returns -1; “bbb” > “aaa”

What would be returned for“40” lt “8”

Some assignment operators$a = 9; # most common assignment operator$a += 1; # equivalent to $a = $a + 1;$a -= 1; # same as $a = $a - 1;++$a # same as $a += 1;$a++ # same as $a += 1;--$a # same as $a -= 1;$a-- # same as $a -= 1;

be careful on the last four of these$a = 7;$b = ++$a; # $a and $b are both set to 8$b = $a++; # $a is now 8, but $b is 7

Some cool things to do with strings

The . operator joins the second operand to the first operand:$a = “be” . “witched”; # $a is now “bewitched”

The x operator makes n copies of a string:$a = “t” x 5; # $a is now “ttttt”

The =~ operator matches patterns and can substitute with s:$x =~ /jkl/ # returns true if “jkl” is in $x$val =~ s/abc/def/; #replaces abc with def

The * character matches 0 or more, ? Matches zero or 1 copy/jk*l/ # matches jl, jkl, jkkl, jkkkl, …/jk?l/ # matches jl or jkl only

Lists and ArraysA list is a collection of values enclosed in parentheses.

( 1, 5.3, “hello”, 2) # contains four values() # an empty list

Perl allows you to store lists in array variables@array = (1, 2, 3); # @ character specifies var as an array@x = (11, “my string”, 27.44);@x = $y; # @x is now a list of one element

# the value of $y

Obtaining the length of a list@x = (“string1”, “string2”, “string3”);$y = @a; # $y contains the LENGTH of @a

More about lists and arrays

Array Slices@x = (1, 2, 3);@y = @x[0,1]; # @y gets slice of @x containing 2

# values (1, 2). NOTE, position 0@x = (1, 2, 3);@x[4, 5] = (10, 20); # @x now has 6 elements, the

# fourth being a null string

@x = (10, 20, 30);$y = @x[1]; # $y now has the value 20

@x = (10, “hello”);$y = “Say @x[1] to your friends”;

More fun with lists and arrays

Using the built-in function sort()@x = (“this”, “is”, “a”, “test”);@x = sort (@x); # @x is now (“a”, “is”, “test”, “this”)

Note that sort() is an ALPHABETIC sort, thus@a = (100, 50, 2);@a = sort (@a); # @a is really (“100”, “2”, “50”)

The function reverse() reverse the order@a = (“backwards”, “is”, “array”, “this”);@a = reverse (@a); # @a is now

# (“this”, “array”, “is”, “backwards”)

Some final list and array functions

To create a single string from a list or array variable use join()@x = join(“ “, “this”, “is”, “a”, “sentence”);

The first element contains the character to “glue” the rest together,@x is thus “this is a sentence”

You’d get the same results with@a = (“this”, “is”, “a”);@x = join (“ “, @a, “sentence”);

To undo the effects of join(), use split()@y = “words::separated::by::colons”;@x = split(/::/, $y); # @x is now

# (“words”, “separated”, “by”, “colons”)

Using Command-line arguments

$ vi printfirstarg#!/usr/bin/perlPrint(“the first argument is $ARGV[0]\n”);$ printfirstarg 1 2 3The first argument is 1$

Note that $ARGV[0], the first element of the $ARGV arrayvariable, does NOT contain the name of the program. Thisis a difference between Perl and C.

Controlling program flow

if ($x == 14) {print(“\$x is 14\n”);

}

Two way branching

if ($x == 14) {print(“\$x is 14\n”);

} else {

print(“\$x is not 14\n”); }

FilesTo open a file, use the function open();

open(MYFILE, “/home/gbwhite/testfile”);

The second argument is the name of the file you want to open.You can supply the full pathname or just the filename itself.

By default, Perl assumes you will open a file for reading. If youwant to open one for writing, put a > character in front of your filename:

open(MYFILE, “>/home/gbwhite/writefile”);

If you want to append instead of overwriting, use >>

More about handling files

The open() function returns true if it succeeded, false otherwise.

You can use the || (logical OR) function to test it:

open(MYFILE, “/home/gbwhite/testfile”) ||die(“unable to open /home/gbwhite/testfile\n”);

This works because the right side of a logical OR only isexecuted if the left side is false.

More about filesTo read from a file, enclose the name in angle brackets:

$line = <MYFILE>;

This statement reads a line of input.

To write to a file, print MYFILE (“this is a test\n”, “this too\n”);

Closing a file:close(MYFILE);

SubroutinesSubroutines can appear anywhere in the program but by convention generally go at the end.

#!/usr/bin/perl$thecount = 0;&getwords;while ($words[0] ne “”) { # stop when line is empty

for ($index = 0; $words[$index] ne “”; $index += 1) {$thecount +=1 if $words[$index] eq “the”;

#above line really if($words[$index] eq “the”) {$thecount +=1;} }

&getwords; }print (“Total number of occurrences of the: $thecount\n”);

sub getwords{$inputline = <>;@words = split(/\s+/, $inputline);

}

Reading from and Writing to Files

• To access a file on a UNIX file system from within your Perl program, you must perform the following steps:• Your program must open the file. This tells the system

that your Perl program wants to access the file.• The program can either read from or write to the file,

depending on how you have opened the file.• The program can close the file. This tells the system

that your program no longer needs access to the file.

Looping Constructs

$ vi loops#!/usr/bin/perl$x = 1;while ($x <= 5) { print ("\$x is now $x\n"); ++$x; }until ($x <= 0) { print ("and \$x is now $x\n"); --$x; }$

$ ./loops$x is now 1$x is now 2$x is now 3$x is now 4$x is now 5and $x is now 6and $x is now 5and $x is now 4and $x is now 3and $x is now 2and $x is now 1$

Arrays• To assign a value (or list of values) to an

array, simply group them together.@weekdays = (“Mon”, “Tue”, “Wed”, “Thu”, “Fri”);

• Arrays are zero-based like C. You could also have accomplished the above by:

@weekdays[0] = “Mon”; @weekdays[1] = “Tue”;@weekdays[2] = “Wed”; @weekdays[3] = “Thu”;@weekdays[4] = “Fri”;

• Another example@home[0] = “couch”; @home[1] = “chair”;@home[2] = “table”; @home[3] = “stove”;

Hashes• An unordered set of scalars, accessed by

some string value associated with each scalar%longday = (“Mon”, “Monday”, “Tue”, “Tuesday”, “Wed”, “Wednesday”, “Thu”, “Thursday”, “Fri”, “Friday”);

• Somewhat confusing to look at so Perl provides another way to initialize

%longday =(“Mon” => “Monday”, “Tue” => “Tuesday”, “Wed” => “Wednesday”, “Thu” => “Thursday”, “Fri” => “Friday”);

Arrays –vs– Hashes

Let’s do a little Perl Review• Write a Perl program that will read in a set

of scores from a file and calculate and print the average score.

• The file is of the format:

Fred 23Mary 24Terry 25

The code#!/usr/bin/perl

open(GRADES, "grades") or die "Can't open grades\n";$total = 0;$scores = 0;while ($line = <GRADES>) { ($student, $grade) = split(" ", $line); $total += $grade; $scores++; }close(GRADES);$average = $total / $scores;print "Average: $average\n";

STDIN and STDOUT

#!/usr/bin/perlprint STDOUT “Enter a number: “;$number = <STDIN>;print STDOUT “The number is $number.\n”;

$ ./printnumEnter a number: 4The number is 4.$

Angle Operator <>

• Encloses the file handle that you want to read lines from.

• The empty angle operator, <>, will read lines from all the files specified on the command line.• If no files were specified on the command line

then <> reads from STDIN.

Chop

• Remember the extra line at the end of our output for the previous example. This happens because the line-reading operation does not automatically remove the newline from the input line.

• The chop command will indiscriminately remove (and return) the last character of the string.

chop example

#!/usr/bin/perlprint STDOUT “Enter a number: “;$number = <STDIN>;chop($number)print STDOUT “The number is $number.\n”;

$ ./printnumEnter a number: 4The number is 4.$

Executing UNIX commands from Perl

• The backtick( `) character is the command input operator. e.g.• $info = `finger root`;

• Another way is to use system• system(“finger root”);

• These two are not exactly the same as the first example places the result in variable $info.

pop and push• push (array, value-list) adds element(s) to the end

of the array.• pop (array) removes the last element from the end

of an array.

#!/usr/bin/perl@stack = (“top”, “middle”, “bottom”);$var = pop(@stack);print “var = $var\n”;push(@stack, “next”);print “stack = @stack\n”);

$ ./pop_push_examplevar = bottomstack = top middle next$

Some String Operations

• substr(str, start-pos, length) returns a substring of the specified string, beginning at the start position for the length specified.

• substr(str, start-pos, length) = string replace the specified section of the string with the assigned string.

• length(str) find the length of a string• index(str, pattern) find the position of a pattern in

a string

String operations example#!/usr/bin/perl$var = “this is a long string”;$sub = substr($var, 0, 4);print “sub=$sub\n”;substr($var, 10, 4) = “short”;print “var=$var\n”;$lngth = length($var);print “length=$lngth\n”;$loc = index($var, “short”);print “location=$loc\n”;

$ ./str_operations_examplesub=thisvar=this is a short string;length=22location=10$