Lesson 8• Flow of data

• redirects

• Input and output

• pipe

• tee

Work Flow

Input and output redirection

bash shell, and the other shells employed by Linux allow the user to specify what should happen to the input and output of programs that run.

shell creates new processes as follows:

1. shell reads the commands to be executed

2. shell opens the files which will be needed (open(2) for files, pipe(2) for streams)

3. shell creates the new processes which will be executed (fork(2) each process)

4. Each new process is connected to the appropriate plumbing (dup2(2) each redirection)

5. Control of processes is passed to programs the user asked for (execve(2) each

command)

shell processes input and output

Standard Streams

Input and output

Normal Flow of Data:

Output Redirects

Simple output redirection. Creates/overwrites file:

Output Redirects

stderr output redirection. Creates/overwrites file.

Output Redirects

Combined out & err redirection. Creates/overwrites files.File names must be different!

Output Redirects

Combined out & err redirection. Creates/overwrites files.Only one file name, used for both output streams

Redirecting Standard Output

Standard output redirection (>, >>)

Standard output is the terminal, by default. The following redirections are possible:

[jack@foo jack]$ uptime5:09pm up 8:26, 6 users, load average: 0.22, 0.20, 0.20

No redirection

Redirection to a file (“>” means “to”). If the file already exists, it is overwritten.

[jack@foo jack]$ uptime > Uptime-file[jack@foo jack]$ ls -l Uptime-file-rw-rw-r-- 1 jack jack 62 Apr 8 17:09 Uptime-file

[jack@foo jack]$ cat Uptime-file5:09pm up 8:26, 6 users, load average: 0.13, 0.18, 0.19

Redirection appending to a file (“>>” means “to the end of”)

[jack@foo jack]$ uptime >> Uptime-file[jack@foo jack]$ cat Uptime-file5:09pm up 8:26, 6 users, load average: 0.13, 0.18, 0.195:10pm up 8:27, 6 users, load average: 0.24, 0.21, 0.20

Redirecting to a program. Here the output of uptime is being piped as the input to wc.

jack@foo jack]$ uptime | wc1 10 62

Standard error redirection (2>, 2>>, 2>&1)

[jack@foo jack]$ tar cf sysconfig.tar /etc/sysconfigtar: Removing leading `/' from member namestar: /etc/sysconfig/rhn/systemid: Cannot open: Permission deniedtar: /etc/sysconfig/rhn/up2date-keyring.gpg: Cannot open: Permissiondeniedtar: /etc/sysconfig/iptables: Cannot open: Permission deniedtar: /etc/sysconfig/static-routes: Cannot open: Permission deniedtar: Error exit delayed from previous errors

Output sent to stderr:•Error messages (command 2>/dev/null)•Warning messages (command 2>> warning.log)•Debugging and progress indicators (command 2>&1 | less )

If stderr is not redirected, error output is displayed on the terminal.

Standard error redirection (2>, 2>>, 2>&1)

The notation 2>&1 means “stderr(2) to copy of stdout(1)”. The following all send standard error output to the same destination as standard output:

• command >file 2>&1• command 2>file 1>&2• command &> file• command >& file (this is not the preferred method)• command 2>&1 | anothercommand

discard the output and the errors from the commands, sending them to /dev/null.

[jack@foo jack]$ tar vcf pam.tar /etc/sysconfig >/dev/null 2>&1[jack@foo jack]$ nosuchcommandorfilename 2>/dev/null 1>&2[jack@foo jack]$ ls -la /root 2>/dev/null 1>&2[jack@foo jack]$ updatedb >& /dev/null &[jack@foo jack]$ killall -HUP inetd sendmail &> /dev/null[jack@foo jack]$ killall -HUP inetd sendmail &> /dev/null

commands send standard error output to the same destination standard output (a program).

[jack@foo jack]$ bash -x /etc/init.d/network restart 2>&1 | less[jack@foo jack]$ tar -c / 2>&1 | less[jack@foo jack]$ strace mailq 2>&1 | less

Output Redirects Summary

> filecapture stdout to file; overwrites; > is equivalent to 1>

2> filecapture stderr to file; overwrites

> file 2> file2capture stdout to file; capture stderr to file2; overwrites

>> filecapture stdout to file; appends; >> is equivalent to 1>>

2>> filecapture stderr to file; appends

>> file 2>> file2capture stdout to file; capture stderr to file2; appends

> file 2>&1capture stdout to file; capture stderr to file; overwrites

>> file 2>&1capture stdout to file; capture stderr to file; appends

Input Redirects

Simple input redirection

Input Redirects

• Input redirection isn’t common anymore - now cmds can handle their own file I/O

• Input and output redirection can be combined:cat < who.all > cat.who.allcat < who.all 2> cat.who.all.errcat < who.all > cat.who.all.all 2>&1

Redirecting Standard Input

Standard input redirection (<, <<EOF, |)

[jack@foo jack]$ wcHello. I typed this line. I typed and typed. I wondered, as I typed, how many words I was typing. Tiswondrous that as I typed, bash piped.<Ctrl+D>4 28 143

Standard input is terminal by default. Following redirections are possible:

Read input from the terminal:

Read input from file with <. standard input anonymous: wc does not know file name which it is reading from.

[jack@foo jack]$ wc < /etc/passwd46 62 2010

Read input from a program using the pipe redirection. Here cat is piping its output to wc.

[jack@foo jack]$ cat /etc/passwd | wc46 62 2010

Read input from script or cmdline. Operator << is traditionally followed by EOF, but any letters can be used.When a line is found which consists only of EOF, that's the end of the input. This is called a “here document”.

[jack@foo jack]$ wc << EOF> Hello> There are> Four lines> I think> EOF4 7 35

Pipes

The output of who is piped into the input of wc -lThis produces a count of the current user sessions

Piping Output to Next Command

Pipes

Pipeline – Ability to join input and output of processes together.

To set up a pipe use pipe symbol | between cmds that generates output. Generally, data piped is text, but its possible to join far too many processes together.

Pipes can be chained as needed, and can also be combined with redirection:who | fgrep bob | wc -l > bob.sessions

It’s even possible to intermix pipes and redirection! Just keep streams straight:who 2> who.errors | fgrep bob 2>&1 | wc -l

The tee Command

tee – divert a copy to a file

tee is like a teepiece for pipes (or T piece). tee takes one argument, a filename, and will feed all input from stdin to file, simultaneously feeding output to stdout.

tee forks its input stream, sending one copy to file on disk, and another copy to stdout

By default tee is like redirection > and overwrites contents in the files specified.

To apend use tee -a.