CISC 220, Fall 2016, Topic 2: More Shell Skills 1

Sub-topics:

• simple shell scripts (no ifs or loops yet)● sub-shells● quoting• shell variables• aliases

• bash initialization files• I/O redirection & pipes• text file formats

Topic 2: More Shell Skills

CISC 220, Fall 2016, Topic 2: More Shell Skills 2

Reading For This Topic

● Reading: Custom Course Notes ● sections 4.2 through 4.6

CISC 220, Fall 2016, Topic 2: More Shell Skills 3

Useful command for demonstrations: echoPrints all its arguments

Normal rules: • spaces separate words in commands• some characters have special meanings to bash

Sometimes useful to quote strings or characters.Three quoting mechanisms:

'string' : every character in the string taken literally"string" : most characters taken literally\c : character taken literally

Sub-Topic 1: Quoting

CISC 220, Fall 2016, Topic 2: More Shell Skills 4

A few special characters are interpreted by double quotes but not single quotes

1. ! : history expansion

2. $: shell variables (next sub-topic)

3. back quotes ( )̀ – next week

Difference Between Single & Double Quotes

CISC 220, Fall 2016, Topic 2: More Shell Skills 5

shell variables: a way to store information in bashTo give a value to a variable:

today=MondayIf no "today" variables exists, creates it.Sets value to the string "Monday"

To refer to value of variable: use $.Example:

echo $today

Sub-Topic 2: Shell Variables

CISC 220, Fall 2016, Topic 2: More Shell Skills 6

1. Use as variables in shell scripts

2. Store settings for shell and other programs.

To see all variables with values:set

Uses of Shell Variables

CISC 220, Fall 2016, Topic 2: More Shell Skills 7

Awkward situation: ------$ object=computer------$ echo I have two $objectsI have two------$ Why?

Fix:------$ echo I have two ${object}sI have two computers

Referring to Shell Variables: More

CISC 220, Fall 2016, Topic 2: More Shell Skills 8

The $ character retains its special meaning inside double quotes.echo "Today is $today"

Not inside single quotes.

Shell Variables and Quotes

CISC 220, Fall 2016, Topic 2: More Shell Skills 9

Changes & new variables only last for life of current shell.To make a permanent change: use an initialization file (soon!)

Shell Variables Are Temporary

CISC 220, Fall 2016, Topic 2: More Shell Skills 10

$PATH Contains list of directories separated by ": "s.When you type

lsBash looks for a command named "ls ".

Searching order:1. Is it a built-in shell command?2. In each directory in $PATH: is there a program called "ls "?

PATH

CISC 220, Fall 2016, Topic 2: More Shell Skills 11

$PS1 is your bash prompt.Can contain literal characters plus some special ones:

\d: the current date\h: the name of the host machine\j: number of jobs you have running\s: the name of the shell \u: name of user\w: current working directory\!: history number of this command

PS1

CISC 220, Fall 2016, Topic 2: More Shell Skills 12

A shell is a program.Linux automatically runs the bash program when you open

a terminal. You can run it too.One reason: experiment with settings.

Command to start a sub-shell: bashExit back to starting shell: exitUse Shell variable $SHLVL to see if you're in a sub-shell

A sub-shell "inherits" many of your settings, includingcurrent folder, umask

Does not automatically inherit variables or aliases

export command: says variable should be inherited by sub-shellsexport myVar

Sub-Topic 3: Sub-Shells

CISC 220, Fall 2016, Topic 2: More Shell Skills 13

shell script = file containing bash commands

Comments in scripts: # means rest of line is a comment

Special variable names:$0 : name of command$# : number of arguments$1,$2 ,...: the individual arguments$* : all the arguments in one string, separated by spaces

Sub-Topic 4: Simple Shell Scripts

CISC 220, Fall 2016, Topic 2: More Shell Skills 14

First step: must be executable (chmod)

Type command name alone:bash looks in your PATH directories

Type command name with directory:no lookup needed

Executing a Shell Script

CISC 220, Fall 2016, Topic 2: More Shell Skills 15

Shell command for changing permissions of a file or files: chmod

chmod <users>+<permissions> <filenames>chmod <users>-<permissions> <filenames>chmod <users>=<permissions> <filenames>

Users can be:● u (user – the person running chmod)● g (the current group of the user)● o (other people)● or a combination

Permissions can be:● r (read)● w (write)● x (execute)● or a combination

Digression #1: changing file permissions

Examples:chmod u+x myprogramchmod g-w somefilechmod o=rx otherfilechmod go= hiddenfilechmod ug+wr draft

CISC 220, Fall 2016, Topic 2: More Shell Skills 16

"umask" = "user mask" – default file permissionsSpecifies permissions for new files you createA property of your bash session – reset when you log off & back in.

umask command: lets you view or change your umaskumask with no args: displays in octal formatumask -S : displays in symbolic format

Format is like chmod:umask u+x file1umask g=rx file2umask o-w file3

Digression #2: default file permissions

CISC 220, Fall 2016, Topic 2: More Shell Skills 17

When you type the name of a script as a command:executed in a sub-shell.

Consequence: can't use to create aliases or set variables, umask, etc.

If you want script to affect current shell:source myscript

Executes commands from myscript in the current shell.

Complication: scripts & sub-shells

CISC 220, Fall 2016, Topic 2: More Shell Skills 18

#!/bin/bash

At beginning of script, specifies this is a script to be run with bash.Why bother? Syntax of some commands different for every shell.Now your script will work for people using other shells.

Not required for 220 assignments, but generally a good idea.

Special Initial Comment

Alias = easy way to create synonym for a commandprovides shorthand for something you do often

alias ll="ls –l"

Also a way to turn on certain options always.Example from standard .bashrc file:

alias rm="rm -i"

To execute original command (without alias): quote the name"rm" file1 file2

To cancel an alias:unalias rm

To see all your aliases:alias (no arguments)

Sub-Topic 5: Aliases

CISC 220, Fall 2016, Topic 2: More Shell Skills 20

In your home directory (probably!):.bash_profile : executed when you start a login shell.bashrc : executed when you start up any other shell

Use these files for your own settings (variables, etc.)

Advice: • before you change, make a copy!• don't remove standard files from path

Sub-Topic 6: Bash Initialization Files

/cas/course/cisc220 contains sample files(do ls -a to make sure you see them!)

CISC 220, Fall 2016, Topic 2: More Shell Skills 21

Three standard "streams" associated with any command or program:

0: standard input (default is keyboard)1: standard output (default is screen)2: error output (for error messages, default is screen)

Sub-Topic 7: I/O Redirection & Pipes

I/O redirection can change settings for these three screens

CISC 220, Fall 2016, Topic 2: More Shell Skills 22

Command to use as example: sort .

Reads standard input and sorts to standard output.demo.... (use ^d to end input)

Command For Examples

CISC 220, Fall 2016, Topic 2: More Shell Skills 23

command 0< file

Reads input from file , instead of keyboard.

Shorter form:command < file

Example:sort < numbers.txt

Redirecting Standard Input

CISC 220, Fall 2016, Topic 2: More Shell Skills 24

command 1> file

Sends standard output of command to file , instead of screen.

Shorter form:command > file

Example:sort > file

Example of redirecting standard input and output:sort < input > output

Redirecting Standard Output

CISC 220, Fall 2016, Topic 2: More Shell Skills 25

Bash may object if you redirect to an existing fileDepends on “noclobber” option.

To view current options: set -o

To set option:set -o noclobber

To turn off option:set +o noclobber

Safer: don't count on noclobber. Use one of these instead:command >| file : if file already exists, overwrite

command >> file : if file already exists, append

Output Redirection: Existing Files

CISC 220, Fall 2016, Topic 2: More Shell Skills 26

command 2>filenameexample:

ls *.c 2>errs

Redirecting Standard Error

CISC 220, Fall 2016, Topic 2: More Shell Skills 27

send to /dev/null

Example: error messages you want to throw away:

ls *.c 2>/dev/null

Getting Rid of Output You Don't Want

CISC 220, Fall 2016, Topic 2: More Shell Skills 28

To 2 separate files:cmd 1>goodOutput 2>errOutput

or:cmd >goodOutput 2>errOutput

To the same file:cmd 1>outFile 2>&1

or:cmd 2>outFile 1>&2

Redirecting Both Output Files

CISC 220, Fall 2016, Topic 2: More Shell Skills 29

cmd1 | cmd2

Sends standard output of cmd1 to

standard input of cmd2

Examples:command | lesscommand | sortcommand | sort > outputcommandA 2>&1 | commandB cmd1 | cmd2 | cmd3

Pipes

CISC 220, Fall 2016, Topic 2: More Shell Skills 30

Basic text file format: ASCII or Unicode charactersProblem: Different OSs, different character(s) for end of line!

Unix/Linux: '\n' (newline)Windows/Dos: '\r' + '\n' (carriage return + newline)Mac: varies with version & program

Most programs can deal with any of these formats.Others are confused by some formats.

Sub-Topic 8: Text File Formats

CISC 220, Fall 2016, Topic 2: More Shell Skills 31

Historical Note

CISC 220, Fall 2016, Topic 2: More Shell Skills 32

Best practice: Create & test your program on CASLab Linux.Do all your editing on Linux.Submit your program to OnQ from a Linux computer

(not possible with putty; use the lab or other Linux GUI)

Second best: Create & test your program on Linux.ftp to your own computer (Windows or Mac)submit to Moodle immediately

Dangerous: Create & test your program on Linux. ftp to your own computermake a couple of “trivial” editssubmit to Moodle

Safety For Marked Assignments