Unix Linux Administration II

Class 8: Scripting loops. Introduction to sendmail. Reading and printing data.

Agenda discuss Homework. Unit 1: Scripting loops. Unit 2: Introduction to sendmail. Unit 3: Reading and printing data.

Homework review

DNS configs slave and master updates

Configuring views.

Scripting – file management script.

Intermediate certificate, new chained www certificate.

Review: conditionalsExit status, 0 = success, !0 = fail.if test "$user" == “<value>”

you can also just use [][ "$user" == “<value>” ]File tests, such as does the file exist.[ -e /etc/nsswitch.conf ]logical operators-a -o || &&You can use parentheses to alter the order of evaluations.if cmd; then do; else do; fiif [ "$HOME" ]; then echo "Found home!"; else echo "shucks we are homeless!"; fi

Review: PKI

Private keys, Public certificates and CSRpublic CAChain of Trust

Chain certificatesPKI setup

private key, csrsigned cert.

sign other requests (CSR).

Class 8, Unit 1

What we are going to cover: Scripting and loops

What you should leave this session with: Basics to creating loops within your scripts. How to enable debug in your scripts.

Loops.Loops are blocks of code that run until

complete (they can be infinite loops)

The first example is the for loop.

for f in value1 value2 value3

do

cmd

done

For loops - body.for letter in a b c

do

echo “found: $letter”

done.

The “Body” is the content between “do” and “done”.

When the script is executed the value for “letter” is assigned to the first value provided after “in” and then the body of the loop is executed. When complete the second value is assigned to the variable $letter and the process is repeated.

? What happens if you enclose a b c in quotes?

for loops cont.

You can leverage the shells ability for filename substitution in loops. The shell provides for filename substitution in the list provided to the body of the loop.for f in [1-3].txt

do

echo $f

done.

Just as in the other examples, echo is executed 3 times in this example

for loops cont.you can also read in file values and feed those to

the for loop.cat filelist.txt

1.txt

2.txt

3.Txt

for files in $(cat filelist.txt) ; do echo $files; done

or

for files in $(cat filelist.txt) ; do cat $files; done

*example of command substitutions.

Using $* in loops

$* = all arguments

echo “Number of arguments passed in $#“

for variables in $*

do

echo "$variables"

done

Replacing $* with $@You know that $* returns all the values provided at

the command line. However if you use $@ this is actually a comma

separated list of values

for f in “$@”do

echo $fdone

*Best practice to place double quotes around $@

while loopsAnother looping function is "while".

while cmd

do

cmd

done

“cmd” is executed and its exit status is tested. if the exit status is zero the commands between do and done are competed otherwise the script exits with a non zero status code

while script

Similar to saying “while true do”sample “while” script counting to 10

num=1

while [ "$num" -le 10 ]

do

echo $num

num=$(( num+1 ))

done

until

until - the inverse of while, meaning it will run so long as the return code is not 0, or not successful.

Similar to the while blocks, commands between the do and done functions may never be executed if the initial command returns a successful response (zero).

Useful when checking for a status change

until cont.# if NOT successful enter the body

until ps -ef | grep -i "named“ | grep –v grep > /dev/null

do

echo "bind is not running"

sleep 5

done

echo "bind is running“

Break out!Sometimes in a logic loop you want to break

out based on user input such as the user asking to quit. Enter “break”

while truedoread cmdif [ "$cmd" = "quit" ] then break else echo "$cmd"fidone

Continue on…The opposite of break is to continue. Sometimes you

want the loop to simply leave the current loop and continue working through the script. This is where you might use continuefor filedo

if [ ! –e “$file” ]then echo “file not found”continuefi

process rest of file/datadone

Sending the process to background

You can background a process using the & after the done statement. Just as we have done at the command line.

for file in data[1-4]

do

run $file

done &

redirection

I/O redirection on a loop can be obtained using the < or > based on your need.

Write to file:

for i in 1 2 3 4

do

echo $i

done > data.out

Sleep and background

sleep n - where n is a numeric value. Sleep will pause the system for the time specified on the command line.

You can run programs in the background using ampersand "&"

script &

output from this command will tell you the process associated with your process.

Use fg to foreground a background process.

optionsYou can define options in your scripts using syntax

similar to this:if [ "$1" = "-a" ]then option=TRUE

shiftelse option=FALSEfiecho "value for option is: $option"

getoptsThe previous example is fine for simple

options but if you want more flexibility it can become tedious to script. However getopts is available for this purpose.

getopts works within a loop and examines each argument to determine if it is an option based on the existence or absence – before the value.

getopts

The syntax of the getopts command is:  getopts optstring option

opstring – is the list of options expected from the command line. option - value used to iterate over the command line options provided.

getopts cont.You can stack your options or pass them

individually. Meaning –abc or –a –b -c

If your option needs an argument add “:”

getopts a:bc name

Now a valid command line looks like:

script.sh –a braeburn –b –c

script.sh –a braeburn

script.sh –b –c

getopts cont.OPTARG used when an option requires an

argument, e.g. –a braeburn

OPTIND is a special variable used by getops which is set to 1 by default and is updated each time getopts complete a loop.

If you reset $OPTIND to 1 at the end of the loop it is possible to use getops again in the same script.

Impact of “:”When an option character not contained in optstring is found, or an option found does not have the required option-argument:

If optstring does NOT begin with a : (colon)

1.Option will be set to a ?

2.OPTARG. will be unset

3.A diagnostic message WILL be written to standard error.

Impact of “:”Alternatively if optstring DOES begin with a : (colon)

1.option will be set to a ? character for an unknown option or to a : (colon) character for a missing required option.

2.OPTARG. will be set to the option character found.

3.no output will be written to standard error.

getopts samplewhile getopts ":ab:c" option; do case $option in a) echo received -a ;; b) echo received -b with $OPTARG ;; c) echo received -c ;; :) echo "option -$OPTARG needs and an ARG" ;; *) echo "invalid option -$OPTARG" ;; esac done

Review: loops and breaksFor loops:for f in a b c; do echo "found: $f"; donefor f in $(cat filelist.txt); do echo $f; donefor f in $(cat filelist.txt); do cat $f; done

$* vs $@, $@ provides a comma separated listUntil and While:while loops, if the exit status is zero the loop is entered.until, if the exit status is NOT zero the loop is entered.Break and continue are used to manipulate the loop behavior.

Review: Options and GETOPTS

Passing options to your script manually.if [ "$1" = "-a" ]then option=TRUE

shift

GETOPTS is a built-in shell function. GETOPTS loops through arguments looking for a “-” before any arguments and determines if it is a valid option.

If arguments are required with the options then you simple add a “:” after the option in your script the GETOPTS will require one.

In class lab 8a

Lab notes for this session can be found here: http://www.ulcert.uw.edu -> Class Content -> InClass labs ->

Class 8, Unit 2

What we are going to cover: Sendmail

What you should leave this session with: DNS mail configuration Basic Sendmail message flow and

configuration.

DNS and mailIn order for mail to routed to your server there

must be a valid MX or mail server record in the DNS domain.

MX records are another type of Resource Record (RR) just as Name Servers are of type NS.

Once we add MX records we should have at least four RR types defined in our domain zone files.

Just as CNAMES and NS RR always need to eventually point to A records, so do MX records.

DNS and mail cont.Mail servers have priority ratings which are

different from other DNS records. The values are somewhat arbitrary but tend to run from 10 to 90

The lower value the higher the priority.

If you have two mail servers one set to 10 and the other to 20 mail will be routed to the lower value unless it is unavailable.

If both had the same value it would be a round robin configuration.

Sample DNS MX configuration

books.ulcert.uw.edu MX 10 mail.books.ulcert.uw.edu

mail.books.ulcert.uw.edu CNAME ns1.books.ulcert.uw.edu

-----------------------------------------------------------------------

Or

----------------------------------------------------------------------- MX 10 mail

mail CNAME ns1

How mail servers work.A client generates a message using one of

many mail clients. This client will either include a built-in SMTP client or it will hand it off to /usr/sbin/sendmail interface. This client then opens a session on port 25 with the SMTP server and begins to send SMTP commands:

HELO, MAIL FROM, RCPT TO, DATA

The message is completed with dot . on a single line. And the message is delivered.

Sendmail historyWritten by Eric Allman who was working and

studying at UC Berkley. The first version was called delivermail and shipped with BSD 4.0 and 4.1.

Sendmail came about as a result of move from NCP (Network Control Protocol) to TCP.

Also namespaces changed from a flat design to a hierarchical namespace (think DNS).

Sendmail first shipped with BSD 4.1c which happened to be the first tcp based version of BSD.

Sendmail successAs Allman has been quoted saying “sendmail

is complex because the world is complex. It is dynamic because the world is dynamic”.

Sendmail strives to accommodate all types of messages. This inclusive goal means rather than denying or rejecting messages that lack the correct header or syntax sendmail tries to compensate for them.

The low cost entry along with a high delivery percentage many consider the primary reason sendmail is so popular today.

Sendmail version info Postfix is the default MTA but sendmail is simple

to install sudo yum install sendmail sendmail-cf

YUM will install sendmail 8.14.x

We can switch between sendmail and postfix using /usr/sbin/alternatives and or enabling services using /sbin/chkconfig

Current stable sendmail version available from sendmail.org is *8.14.8

* now purchased by Proofpoint

Email and Sendmail

There are three primary roles to consider when reviewing mail:MUA – message user agent, examples?MTA – message transfer agent, delivers mail

and transports mail between machines, examples?

MSA – Mail submission agent, capable of altering mail messages such as confirming hostnames are fully qualified, examples?

What are Sendmail, postfix and Exchange?

Basic parts of SendmailThe basic parts to Sendmail The configuration file

/etc/mail/sendmail.cf A queue directory

/var/spool/mqueue Aliases

Sendmail can and will redirect mail destined for one account to another based on defined aliases.

Addresses and Rules Sendmail is based primarily on rules. rules are used to rewrite (modify) mail

addresses, to detect errors in addressing and to select mail delivery agents.

rules are used to detect and reject errors, such as mail with no username

rules examine the address of each envelope recipient and select the appropriate delivery agent.

Rule Sets a sequence of rules are grouped together into rule

sets, each set is similar to a subroutine a rule set is declared with the S command rule sets are numbered or named rule sets such as 0, 3, 4 and 5 are internally

defined by Sendmail     0 resolve mail delivery agent    3 preprocess all addresses    4 post process all address    5 rewrite un-aliased local user

The three parts to a message

All messages have three primary components Header Body Envelope

The Header

Most header lines start with a word followed by a colon.

Received: Date: From: To:Each word indicates the expected value.Not all headers are required.

The BodyThe body of a message consists of

everything following the first blank line

To: user@domain

Subject: Test message, blank line next!

The body start here. Message content here.

Is the subject line required?

The Envelope Because of the diverse recipients, the sendmail

program uses the concept of an envelope. Content that describes the sender or recipient but

is not part of the header is considered envelope information.

Envelope data is used to tell remote machines that mail is ready from a given user. Before sendmail sends the data to a remote MTA it will

send just the envelope-sender address and recipient list to the remote MTA. If ANY of the recipients are accepted the message is sent over otherwise it is not.

Aliases file functions Aliasing is the process of converting one address

into another address. Convert root to mailer-daemon Convert name to list as in mailing list

Sample conversions Bob Barker bbarker geeks allman, schmidt, wall, joy Nobody /dev/null app |/usr/local/bin/myapp

When mail is bounced (returned because it could not be delivered), it is always sent from MAILER-DAEMON. That alias is needed because users might reply to bounced mail without it, replies to bounced mail would themselves bounce.

Queue Management Messages can be temporarily

undeliverable for a variety of reasons. As a result sendmail will queue up messages that are delayed.

These messages are stored in the QueueDirectory which is defined in the sendmail.cf file

Local delivery Sendmail will deliver messages to local

user, meaning a user with a mailbox on the host where sendmail is running.

Local mail is appended to a users mailbox file.The local file is often ~/mbox

Remote delivery Of course sendmail will also deliver mail to

other machines. This happens when sendmail determines the user is not local.

By default Sendmail only supports TCP/IP enabled networks though other options are available (uucp, mfax)

Sendmail modesUsually sendmail runs in Daemon mode –bd, listening for mail but it can be run in:

Test mode –btJust resolve addresses

Verify mode –bvDon’t collect or deliver mail

Mail sender –bmJust send mail

Many others possible, verbose –v…

Sendmail MacrosSendmail macros allow you to reference text

symbolically within the config file. This means you can centrally define values.

Some macros are defined by Sendmail for you such as $u, $h

enter the following to see some of the macros used by sendmail

/usr/lib/sendmail –C/etc/mail/sendmail.cf -bt -d0

*ctrl-+d to exit and no space between –C and /etc…

Sendmail options cont.Sendmail options are defined in sendmail.cf.

Options are declared with an O O QueueDirectory=/var/spool/mqueueOther sample variables are: Timeout

Timeout.queuewarn=4h Timeout.queuereturn=5d

DeliveryMode Background most common

TempFileMode DefaultUser LogLevel

Review:Default MTA in CentOS 6.x is postfix.Installing sendmail provides two MTA options.Mail delivery requires DNS support. MX records are defined

in DNS similar to how we setup NS records.

Three primary roles for mail include: MUA MTA MSA

primary sendmail configuration file /etc/mail/sendmail.cf. This file is not typically edited directly.

Review: Mail is store in the Queue directory before/until

delivered Aliases allow mail to be redirected between

accounts or services as required. sendmail is based on rules and rulesets.

messages are processed by these rulesets before being accepted or denied.

The three primary parts of a message are: header: received, date

body: everything after the first blank line.

envelope: meta data about the message

In class lab 8b

Lab notes for this session can be found here: http://www.ulcert.uw.edu -> Class Content -> InClass labs ->

Class 8, Unit 3

What we are going to cover: Reading and printing data

What you should leave this session with: How to read data in at the cmd line How to format data for printing

Reading in dataTo read in data use

read variable

eg.

read userinput

echo $userinput

Or for multiple variables

read value1 value2 value3

Read cont.If more arguments are entered than variables

the last variable will store the overflow.echo -n "enter names: "

read name names

echo "you entered \$name $name"

echo "then you entered \"$names\" to be stored in \$names “

enter names: TOM SAM JOHN BILL

you entered $name TOM

then you entered "SAM JOHN BILL" to be stored in $names

Read exit code. Read always returns an exit status of zero

unless the end of file condition is detected from input. This usually means Ctrl+d

Knowing this we can use a while loop to read in data at the command line.

while read num1 num2

do

echo $(( $num1 + $num2 ))

done

User input, yes/noUsing the read function and if/then statements we can check for user acceptance.

echo -n "enter yes/no"

read answer

if [ "$answer" = yes ]; then

echo "you agree!"

elif [ "$answer" = no ] ; then

echo "you disagree"

else

echo "I did not understand your answer"

fi

Using $$ for uniqueness

The value for $$ is set to the process id for a given process.

Each process ID on Unix or Linux system is unique for that host. So using this value you can create objects that are very unlikely to conflict with other files on the same system.

grep $USER /etc/passwd >> /tmp/userinfo.$$.tmp

printf: print formatted outputSyntax is

printf “format” arg1 arg2

e.g.

printf “this is a number: %d\n” 10

printf scans the input, sees %d substitutes the first variable with an argument 10

printf conversions.

printf “octal for %d is %o\n” 20 20

Format characters that are NOT preceded by a percent sign are written to stdout.

octal for 20 is 24

Characters that ARE preceded by a percent sign are called “conversion specifications” and will be converted based on the display command.

printf cont.Common printf conversion characters

d integer

c single character

s literal characters

b literal strings with backslash escape char

% percent sign

printf output samples.printf "string contains backslash: %s\n" "test\string"

string contains backslash: test\string

printf "string %s and character %c\n" hello A

string hello and character A

printf “print just the first character: %c\n” QAZW

print just the first character: Q

printf general format

%[flags][width][.precision]typeOnly the % and type are required the others

are modifiers.Flags include- Left justify+ precedes integers with -/+# printf precedes hex integers with 0x or 0X

printf formattingPrintf is typically used to format output. Printf

can align output, set columns and justify content as required.

printf "%+d\n%+d\n%+d\n" 10 -10 20

+10

-10

+20

printf "%-20s%-20s\n" Firstname Lastname

Firstname Lastname

Reviewread variable

echo $variable

read var0 var1 var2

read exit code zero or true unless end of file detected

while true

or

while read input; do ...

read answer

if [ "$answer" = X ] ; then

using process id for file names: file.txt.$$

printf used for formatting output.

printf "%-20s%-20s\n" ColumnA ColumnB

Homework

homework for this week posted later tonight.