Sed - important SED commands and help

7/29/2019 Sed - important SED commands and help

1/21

sedA Stream Editor

Rod Lovett

3.1


2/21

IntroductionA non-interactive editor designed to be useful in three areas:, I "1) To edit files too large for comfortable interactive editing.2) To edit any size file when the sequence of editing commandsis too complicated to be comfortably typed in interactive mode.3) To perform multiple 'global' editing functions efficiently in

one pass through the input (which may be a sequence of files).Since only a few lines of input reside in core at one time, and notemporary files are used, the effective 'Sizeof the file(s) that canbe edited is limited only by the requirement that the input and outputfit simultaneously into available secondary storage.Complicated editing scripts can be created separately and given to'sed' as a command file.The principal loss of functionality compared to an interactive editorare 'lack of relative addressing (so that it is not possible to gobackwards in a file) because of the line-at-a-time operation and lackof immediate verification that a command has done what was intended.Further, only a limited amount of 'internal memory' is available andhence it is hard to remember text from one line to the next.Sed is a direct descendant of 'ed'. The most striking familyresemblance between the two editors is in the class of patterns(regular expressions) they recognize.

3.2


3/21

.By default, 'sed' copies the standard input to the standard output,perhaps performing one or more editing commands on each line beforewriting it to the output.Pictorially we have:

Input Fileline n

line 4line 3line 2

line 1

command 1

command 2" .

command 3 1 Standard Output

3 . 3


4/21

Each line in turn is passed successively over each command. Thusline 1 is passed successively to command1, command2 andcommand3.After command1 has operated on line 1 the (possibly) modified line 1is passed to command2.After passing through all the commands the (possibly) modified lineis sent to Standard Output.Thus the commands are applied one at a time; the input to eachcommand is the output from the previous command.

(This behaviour may be modified by flags on the command line - seeshortly).The default linear order of application of editing commands can bechanged by the flow-of-control-commands, 't'and 'b' (see later).However, even in this case the input line to each command is theoutput line of the previous command ..Pattern matching

The range over which an attempt is made to match patterns is calledthe "pattern space". Ordinarily the pattern space is one line ofinput text.Examples are developed throughout the text. Many of them are basedon the following lines in file 'text':

In Xanadu did Kubla k /fJ 0t,V\A stately pleasure dome decree;Where Alph, the sacred river, ranThrough caverns measureless to manDown to a sunless sea.

Line 1Line 2Line 3Line 4Line 5

3.4


5/21

UsageSed can be called in many different ways:1. sed 'ed command' filenamets) " l,. t:,,"'j ~ ~,\-- c::..o~ ~'- _L..tb-v e, ~:::~ \..\ ~.J I e : -

I~sed ' / e/ d' t extor \l~1NL V\. oV V ' \o .a - .. . - -

,J ;sed ' 2q' t ext'\ '()\V",J ;

2. sed 'list of commands' filename(s)sed ' / e/ d\s/ I n/ Down i n/ ' t ext

Or the editor commands on the command line can be introduced bythe '-e' flag:' .

3. sed -e command1 - e command2 -e ... filename(s)~ sed - e / e/ d - e I r s/ I n/ Down i n/ " t ext.'-~'- ')' kz~oA~ ::!:-__-""--".""

whereany command containing a white space must be put in quotes.

4. sed -f edcommands text> newtextwhere 'edcommands' contains the editor commands,e.g. 'edcommands' contains:/ e / ds/ I n/ Down i n/

3.5


6/21

Usage-"

5. sed -f edcommands newtext~ (;'-I'-~~S~ b~1c~d (?,-J-~r

< ? '6. sed@ command textor,sed -n 'list ofcommands' text r , , '

sed~- Th: / an/ p t ext ,where only the lines matching the pattern are printed.

Addres.ses ;_ ~. 'tot . 'Ii' The general form of an editing command is:

, [address 1,address2][functionJ [ arguments], I

One or both addresses may be omitted; the format of addresses isgiven shortly. Any number of blanks lor tabs may separate theaddresses from the function. , .'The function must be present. " . '

',,' "The arqumentsmay.be required or optional; aceordlnq to whichfunction is used. "" . . ' ,Blanks and spaces at the beginning of lines are ignored.The addresses select the lines for editing. Addresses may be eitherline numbers or context addresses ( regular expressions ).

3.6


7/21

AddressesThe application of a group of commands can be controlled by oneaddress (or address pair) by grouping the commands with curly braces(see later)

A line number is a decimal integer. As each line is read from theinput, a line-number counter is incremented; a line-number addressmatches (selects) the input line when the internal counter equals theaddress line number. The counter runs cumulatively through multipleinput files; it is not reset when a new input file is opened.As a special case, the character '$'matches the last line of thelast input file:

A context address is a pattern (regular expression) enclosed inforward slashes. The regular expressions recognized by 'sed' aregiven in Appendix A.If a command has no addresses, it is applied to every line in theinput.If a command has one address, it is applied to all lines whichmatch that address.If a command has two addresses, it is applied to the first linewhich matches the first address, and to all subsequent lines until(and including) the first subsequent line which matches the secondaddress. Then an attempt is made on subsequent lines to againmatch the first address, and the process is repeated.

3.7


8/21

The Delete and Quit ommandsNow let us work through some examples:Initially we will look at 'delete'.

sed ' / an/ d ' t extHere Lines 2 and 5 reach the Standard Output.The 'd' function deletes from the file (i.e, does not write to StandardOutput) all those lines matched by its address(es). Italso has the sideeffect that no further commands (if present) are attempted on thecorpse of a deleted line; as soon as the 'd' function is executed,a new line is read from the input, and the list of editing commandsis re-started from the beginning on the ne_\Vjn~. .' A.J g_.~ ~ 'J~ ~~\h....-v- ,,"-) ~~ Lo'-\. \- ...,.:."..~, oke) ."'''' C '\ : ~~ (\~~~ (~sed ' / di d/ , / an/ d' t extHere Lines 4 and 5 reach Standard Output.To delete a blank line use:

I

sed ' / A$/ d' t extNow we will look at 'quit'.

sed ' 2q' t extThe output is Line 1 and Line 2. All lines up to and includingthe matching line number are read in and output to Standard Output.

who I sed ' l Oq'is equivalent to:

who I head , .

3.8


9/21

T;heSubstitute CommandSubstitute ,s, is probably the most widely used command in 'sed' andwe will look at many examples.

sed ' s / t o/ by/ ' t extOutputs Lines 1 -3 unchanged andThrough caverns measureless by manDown by a sunless sea. Line 4Line 5The first occurrence of 'to' (in this case the only occurrence) onall lines is substituted by 'by'.

'---:-7sed ' s/ a/ z~ t ext

Notice the 'g'.The output is:

InXZnZdu did KublZ KhZnA stltely pleZsure dome decree;Where Alph, the sZcred river, rZnThrough cZverns meZsureless to mZnDown to Z sunless seZ.

Line 1Line 2Line 3Line 4Line 5Here, all occurrences of 'a'on every line substituted by 'Z'.

sed ,'/In/, / Where/ , s / a/ Z/ ' t extThe output is:

InXZnadu did Kubla KhanA stZtely pleasure dome decree;Where Alph, the sZcred river, ranLines 4 and 5 unchanged.

3.9

Line 1Line 2Line 3


10/21

-';:.:J 1 '0 '\k i t .. .! ~ (0,,', ".r'\r;9r "_'/""il, . . , . . . v , , - ' F" ',- , .,.., J ...r - .,~ i)c--c ~_,,,\ crI.c-" ~

sed '/In/, IWhere/s/a/z/fV textgives an output of:

In XZnZdu did KublZ KhZnA stZtely pleZsure dome decree;Where Alph, the sZcred river, rZnLine 1Line 2Line 3 .,

Lines 4 and 5 unchanged.

sed 'S/AII' textindents all lines by one tab stop (even blank lines). A bettermethod is: ~~ C~V'(A..o,;.Ii:;-(.jj : - . - . ' ,'> d. - '" .sed 'I . Is/(' /" texn t ' ", I 1;- " ......'vfr~ ~ ~~ ~


17/21

Passing Shell Arguments to sed, . ~ , , .Let us write a shell script which outputs all_files.that are youngerthan' file 'file'. . 'Remembering that '1s -1t'gives us the files in reversechronological order we write script 'newer' as follows:. . ,

# Usage newe r f i l e namei f ( $#ar gv != 1) t henecho Us age : $0 f i l e nameexi t 1e ndi ' f 'I s - I t I s ed 'I' $1 '/q'ex i t 0

Notice that $1 is not inside the 'sed' command.

The Grouping of CommandsCommands may be grouped using the curly braces thus:

s ed - n < t e x t '1;1 {\ w t emp\q\},

will quit after the matching line has been written to "temp".

3.17


18/21

The Branch CommandComparatively rarely used. Consider the following command:

sed - n 7f edcmdO t extwhere 'edcmd'contains:

/ [IW] / {p '( ~\


19/21

The Test and Branch

A more sophisticated use of the branch instruction is the 'testand branch'.The 't'function tests whether any successful substitutions havebeen made on the current input line; if so, it branches to thespecified label; if not, it does nothing. Consider:

sed - n -f edcmd2 t extwhere 'edcmd2' contains:

/ [ I W /

outputs:InXAnadu did Kubla KhanInXAnAdu did Kubla KhanInXAnAdu did KublA KhanIn XAnAdu did KublA KhAn

Line 1Line 1Line 1Line 1The Next functionCopies the current input line to Standard Ouput and brings the nextinput line into the pattern space. Consider:

sed ' n\d' t extwhich produces the following output.

In Xanadu did Kubla KhanWhere Alph, the sacred river, ranLJ OWn. to a sunless sea.,

Line 1Line 3Line 5

3.19


20/21

Hold and Get functions'sed' has a limited 'memory' capability; the 'hold' and 'get'functions used for setting aside input for later use.The 'h' function copies the contents of the pattern space into a'hold area' destroying the previous contents of the 'hold area.The 'g' function copies the contents of the 'hold area' into thepattern space destroying the previous contents of the pattern space.The 'x' function interchanges the contents of the pattern space andhold area.Consider: sed -f edcmd3 t extwhere 'edcmd3' contains:

/ "A/ h/ D/ gproduces the following output:

InXanadu did Kubla KhanA stately pleasure dome decree;Where Alph, the sacred river, ranThrough caverns measureless to manA stately pleasure dome decree;

3.20


21/21

The regular expressions recognised by 'sed' are as follows:1. An ordinary character (not one of those discussed below) is aregular expression, and matches that character.

2. A circumflex, '1\', at the beginning of a regular expressionmatches the null character at the beginning of a line.3. A dollar-sign, '$', at the end of a regular expression matchesthe null character at the end of a line.4. The character, '\n', matches an embedded newline character, butnot the newline at the end of the pattern space.

5. A period, '.', matches any character except the terminating newline, 'at the end of the pattern space.6. A regular expression followed by the character ,*, matches anynumber (including zero) of adjacent occurrences of the regularexpression it follows.7. A string of characters in square brackets, '[]', matches anycharacter in the string, and no others. If, however, the firstcharacter is a circumflex, '1\', the regular expression matchesany charcter except the characters in the string and the terminalnewline of the pattern space.8. A concatenation of regular expressions is a regular expressionwhich matches the concatenation of strings matched by thecomponents of the regular expression.

To use one of the special characters (1\ $ . * [ ] \ I ) as itself,precede the special character by a backslash, ,\'.

Documents

Sed - important SED commands and help