1SED(1P) POSIX Programmer's Manual SED(1P)
2
6 This manual page is part of the POSIX Programmer's Manual. The Linux
7 implementation of this interface may differ (consult the corresponding
8 Linux manual page for details of Linux behavior), or the interface may
9 not be implemented on Linux.
10
12 sed — stream editor
13
15 sed [-n] script [file...]
16 sed [-n] -e script [-e script]... [-f script_file]... [file...]
18 sed [-n] [-e script]... -f script_file [-f script_file]... [file...]
20
22 The sed utility is a stream editor that shall read one or more text
23 files, make editing changes according to a script of editing commands,
24 and write the results to standard output. The script shall be obtained
25 from either the script operand string or a combination of the option-
26 arguments from the -e script and -f script_file options.
27
29 The sed utility shall conform to the Base Definitions volume of
30 POSIX.1‐2017, Section 12.2, Utility Syntax Guidelines, except that the
31 order of presentation of the -e and -f options is significant.
32 The following options shall be supported:
34 -e script Add the editing commands specified by the script option-argu‐
36 ment to the end of the script of editing commands.
37 -f script_file
39 Add the editing commands in the file script_file to the end
40 of the script of editing commands.
41 -n Suppress the default output (in which each line, after it is
43 examined for editing, is written to standard output). Only
44 lines explicitly selected for output are written.
45 If any -e or -f options are specified, the script of editing commands
47 shall initially be empty. The commands specified by each -e or -f
48 option shall be added to the script in the order specified. When each
49 addition is made, if the previous addition (if any) was from a -e
50 option, a <newline> shall be inserted before the new addition. The
51 resulting script shall have the same properties as the script operand,
52 described in the OPERANDS section.
53
55 The following operands shall be supported:
56 file A pathname of a file whose contents are read and edited. If
58 multiple file operands are specified, the named files shall
59 be read in the order specified and the concatenation shall be
60 edited. If no file operands are specified, the standard input
61 shall be used.
62 script A string to be used as the script of editing commands. The
64 application shall not present a script that violates the
65 restrictions of a text file except that the final character
66 need not be a <newline>.
67
69 The standard input shall be used if no file operands are specified, and
70 shall be used if a file operand is '-' and the implementation treats
71 the '-' as meaning standard input. Otherwise, the standard input shall
72 not be used. See the INPUT FILES section.
73
75 The input files shall be text files. The script_files named by the -f
76 option shall consist of editing commands.
77
79 The following environment variables shall affect the execution of sed:
80 LANG Provide a default value for the internationalization vari‐
82 ables that are unset or null. (See the Base Definitions vol‐
83 ume of POSIX.1‐2017, Section 8.2, Internationalization Vari‐
84 ables for the precedence of internationalization variables
85 used to determine the values of locale categories.)
86 LC_ALL If set to a non-empty string value, override the values of
88 all the other internationalization variables.
89 LC_COLLATE
91 Determine the locale for the behavior of ranges, equivalence
92 classes, and multi-character collating elements within regu‐
93 lar expressions.
94 LC_CTYPE Determine the locale for the interpretation of sequences of
96 bytes of text data as characters (for example, single-byte as
97 opposed to multi-byte characters in arguments and input
98 files), and the behavior of character classes within regular
99 expressions.
100 LC_MESSAGES
102 Determine the locale that should be used to affect the format
103 and contents of diagnostic messages written to standard
104 error.
105 NLSPATH Determine the location of message catalogs for the processing
107 of LC_MESSAGES.
108
110 Default.
111
113 The input files shall be written to standard output, with the editing
114 commands specified in the script applied. If the -n option is speci‐
115 fied, only those input lines selected by the script shall be written to
116 standard output.
117
119 The standard error shall be used only for diagnostic and warning mes‐
120 sages.
121
123 The output files shall be text files whose formats are dependent on the
124 editing commands given.
125
127 The script shall consist of editing commands of the following form:
128 [address[,address]]function
131 where function represents a single-character command verb from the list
133 in Editing Commands in sed, followed by any applicable arguments.
134 The command can be preceded by <blank> characters and/or <semicolon>
136 characters. The function can be preceded by <blank> characters. These
137 optional characters shall have no effect.
138 In default operation, sed cyclically shall append a line of input, less
140 its terminating <newline> character, into the pattern space. Reading
141 from input shall be skipped if a <newline> was in the pattern space
142 prior to a D command ending the previous cycle. The sed utility shall
143 then apply in sequence all commands whose addresses select that pattern
144 space, until a command starts the next cycle or quits. If no commands
145 explicitly started a new cycle, then at the end of the script the pat‐
146 tern space shall be copied to standard output (except when -n is speci‐
147 fied) and the pattern space shall be deleted. Whenever the pattern
148 space is written to standard output or a named file, sed shall immedi‐
149 ately follow it with a <newline>.
150 Some of the editing commands use a hold space to save all or part of
152 the pattern space for subsequent retrieval. The pattern and hold spaces
153 shall each be able to hold at least 8192 bytes.
154 Addresses in sed
156 An address is either a decimal number that counts input lines cumula‐
157 tively across files, a '$' character that addresses the last line of
158 input, or a context address (which consists of a BRE, as described in
159 Regular Expressions in sed, preceded and followed by a delimiter, usu‐
160 ally a <slash>).
161 An editing command with no addresses shall select every pattern space.
163 An editing command with one address shall select each pattern space
165 that matches the address.
166 An editing command with two addresses shall select the inclusive range
168 from the first pattern space that matches the first address through the
169 next pattern space that matches the second. (If the second address is a
170 number less than or equal to the line number first selected, only one
171 line shall be selected.) Starting at the first line following the
172 selected range, sed shall look again for the first address. Thereafter,
173 the process shall be repeated. Omitting either or both of the address
174 components in the following form produces undefined results:
175 [address[,address]]
178 Regular Expressions in sed
180 The sed utility shall support the BREs described in the Base Defini‐
181 tions volume of POSIX.1‐2017, Section 9.3, Basic Regular Expressions,
182 with the following additions:
183 * In a context address, the construction "\cBREc", where c is any
185 character other than <backslash> or <newline>, shall be identical
186 to "/BRE/". If the character designated by c appears following a
187 <backslash>, then it shall be considered to be that literal charac‐
188 ter, which shall not terminate the BRE. For example, in the context
189 address "\xabc\xdefx", the second x stands for itself, so that the
190 BRE is "abcxdef".
191 * The escape sequence '\n' shall match a <newline> embedded in the
193 pattern space. A literal <newline> shall not be used in the BRE of
194 a context address or in the substitute function.
195 * If an RE is empty (that is, no pattern is specified) sed shall
197 behave as if the last RE used in the last command applied (either
198 as an address or as part of a substitute command) was specified.
199 Editing Commands in sed
201 In the following list of editing commands, the maximum number of per‐
202 missible addresses for each function is indicated by [0addr], [1addr],
203 or [2addr], representing zero, one, or two addresses.
204 The argument text shall consist of one or more lines. Each embedded
206 <newline> in the text shall be preceded by a <backslash>. Other <back‐
207 slash> characters in text shall be removed, and the following character
208 shall be treated literally.
209 The r and w command verbs, and the w flag to the s command, take an
211 rfile (or wfile) parameter, separated from the command verb letter or
212 flag by one or more <blank> characters; implementations may allow zero
213 separation as an extension.
214 The argument rfile or the argument wfile shall terminate the editing
216 command. Each wfile shall be created before processing begins. Imple‐
217 mentations shall support at least ten wfile arguments in the script;
218 the actual number (greater than or equal to 10) that is supported by
219 the implementation is unspecified. The use of the wfile parameter shall
220 cause that file to be initially created, if it does not exist, or shall
221 replace the contents of an existing file.
222 The b, r, s, t, w, y, and : command verbs shall accept additional argu‐
224 ments. The following synopses indicate which arguments shall be sepa‐
225 rated from the command verbs by a single <space>.
226 The a and r commands schedule text for later output. The text specified
228 for the a command, and the contents of the file specified for the r
229 command, shall be written to standard output just before the next
230 attempt to fetch a line of input when executing the N or n commands, or
231 when reaching the end of the script. If written when reaching the end
232 of the script, and the -n option was not specified, the text shall be
233 written after copying the pattern space to standard output. The con‐
234 tents of the file specified for the r command shall be as of the time
235 the output is written, not the time the r command is applied. The text
236 shall be output in the order in which the a and r commands were applied
237 to the input.
238 Editing commands other than {...}, a, b, c, i, r, t, w, :, and # can be
240 followed by a <semicolon>, optional <blank> characters, and another
241 editing command. However, when an s editing command is used with the w
242 flag, following it with another command in this manner produces unde‐
243 fined results.
244 A function can be preceded by a '!' character, in which case the func‐
246 tion shall be applied if the addresses do not select the pattern space.
247 Zero or more <blank> characters shall be accepted before the '!' char‐
248 acter. It is unspecified whether <blank> characters can follow the '!'
249 character, and conforming applications shall not follow the '!' char‐
250 acter with <blank> characters.
251 If a label argument (to a b, t, or : command) contains characters out‐
253 side of the portable filename character set, or if a label is longer
254 than 8 bytes, the behavior is unspecified. The implementation shall
255 support label arguments recognized as unique up to at least 8 bytes;
256 the actual length (greater than or equal to 8) supported by the imple‐
257 mentation is unspecified. It is unspecified whether exceeding the maxi‐
258 mum supported label length causes an error or a silent truncation.
259 [2addr] {editing command
261 editing command
263 ...
265 } Execute a list of sed editing commands only when the pattern
267 space is selected. The list of sed editing commands shall be
268 surrounded by braces. The braces can be preceded or followed
269 by <blank> characters. The <right-brace> shall be preceded by
270 a <newline> or <semicolon> (before any optional <blank> char‐
271 acters preceding the <right-brace>).
272 Each command in the list of commands shall be terminated by a
274 <newline> character, or by a <semicolon> character if permit‐
275 ted when the command is used outside the braces. The editing
276 commands can be preceded by <blank> characters, but shall not
277 be followed by <blank> characters.
278 [1addr]a\
280 text Write text to standard output as described previously.
282 [2addr]b [label]
284 Branch to the : command verb bearing the label argument. If
285 label is not specified, branch to the end of the script.
286 [2addr]c\
288 text Delete the pattern space. With a 0 or 1 address or at the end
290 of a 2-address range, place text on the output and start the
291 next cycle.
292 [2addr]d Delete the pattern space and start the next cycle.
294 [2addr]D If the pattern space contains no <newline>, delete the pat‐
296 tern space and start a normal new cycle as if the d command
297 was issued. Otherwise, delete the initial segment of the pat‐
298 tern space through the first <newline>, and start the next
299 cycle with the resultant pattern space and without reading
300 any new input.
301 [2addr]g Replace the contents of the pattern space by the contents of
303 the hold space.
304 [2addr]G Append to the pattern space a <newline> followed by the con‐
306 tents of the hold space.
307 [2addr]h Replace the contents of the hold space with the contents of
309 the pattern space.
310 [2addr]H Append to the hold space a <newline> followed by the contents
312 of the pattern space.
313 [1addr]i\
315 text Write text to standard output.
317 [2addr]l (The letter ell.) Write the pattern space to standard output
319 in a visually unambiguous form. The characters listed in the
320 Base Definitions volume of POSIX.1‐2017, Table 5-1, Escape
321 Sequences and Associated Actions ('\\', '\a', '\b', '\f',
322 '\r', '\t', '\v') shall be written as the corresponding
323 escape sequence; the '\n' in that table is not applicable.
324 Non-printable characters not in that table shall be written
325 as one three-digit octal number (with a preceding <back‐
326 slash>) for each byte in the character (most significant byte
327 first).
328 Long lines shall be folded, with the point of folding indi‐
330 cated by writing a <backslash> followed by a <newline>; the
331 length at which folding occurs is unspecified, but should be
332 appropriate for the output device. The end of each line shall
333 be marked with a '$'.
334 [2addr]n Write the pattern space to standard output if the default
336 output has not been suppressed, and replace the pattern space
337 with the next line of input, less its terminating <newline>.
338 If no next line of input is available, the n command verb
340 shall branch to the end of the script and quit without start‐
341 ing a new cycle.
342 [2addr]N Append the next line of input, less its terminating <new‐
344 line>, to the pattern space, using an embedded <newline> to
345 separate the appended material from the original material.
346 Note that the current line number changes.
347 If no next line of input is available, the N command verb
349 shall branch to the end of the script and quit without start‐
350 ing a new cycle or copying the pattern space to standard out‐
351 put.
352 [2addr]p Write the pattern space to standard output.
354 [2addr]P Write the pattern space, up to the first <newline>, to stan‐
356 dard output.
357 [1addr]q Branch to the end of the script and quit without starting a
359 new cycle.
360 [1addr]r rfile
362 Copy the contents of rfile to standard output as described
363 previously. If rfile does not exist or cannot be read, it
364 shall be treated as if it were an empty file, causing no
365 error condition.
366 [2addr]s/BRE/replacement/flags
368 Substitute the replacement string for instances of the BRE in
369 the pattern space. Any character other than <backslash> or
370 <newline> can be used instead of a <slash> to delimit the BRE
371 and the replacement. Within the BRE and the replacement, the
372 BRE delimiter itself can be used as a literal character if it
373 is preceded by a <backslash>.
374 The replacement string shall be scanned from beginning to
376 end. An <ampersand> ('&') appearing in the replacement shall
377 be replaced by the string matching the BRE. The special mean‐
378 ing of '&' in this context can be suppressed by preceding it
379 by a <backslash>. The characters "\n", where n is a digit,
380 shall be replaced by the text matched by the corresponding
381 back-reference expression. If the corresponding back-refer‐
382 ence expression does not match, then the characters "\n"
383 shall be replaced by the empty string. The special meaning of
384 "\n" where n is a digit in this context, can be suppressed by
385 preceding it by a <backslash>. For each other <backslash>
386 encountered, the following character shall lose its special
387 meaning (if any).
388 A line can be split by substituting a <newline> into it. The
390 application shall escape the <newline> in the replacement by
391 preceding it by a <backslash>.
392 The meaning of an unescaped <backslash> immediately followed
394 by any character other than '&', <backslash>, a digit, <new‐
395 line>, or the delimiter character used for this command, is
396 unspecified.
397 A substitution shall be considered to have been performed
399 even if the replacement string is identical to the string
400 that it replaces. Any <backslash> used to alter the default
401 meaning of a subsequent character shall be discarded from the
402 BRE or the replacement before evaluating the BRE or using the
403 replacement.
404 The value of flags shall be zero or more of:
406 n Substitute for the nth occurrence only of the BRE
408 found within the pattern space.
409 g Globally substitute for all non-overlapping
411 instances of the BRE rather than just the first
412 one. If both g and n are specified, the results are
413 unspecified.
414 p Write the pattern space to standard output if a
416 replacement was made.
417 w wfile Write. Append the pattern space to wfile if a
419 replacement was made. A conforming application
420 shall precede the wfile argument with one or more
421 <blank> characters. If the w flag is not the last
422 flag value given in a concatenation of multiple
423 flag values, the results are undefined.
424 [2addr]t [label]
426 Test. Branch to the : command verb bearing the label if any
427 substitutions have been made since the most recent reading of
428 an input line or execution of a t. If label is not speci‐
429 fied, branch to the end of the script.
430 [2addr]w wfile
432 Append (write) the pattern space to wfile.
433 [2addr]x Exchange the contents of the pattern and hold spaces.
435 [2addr]y/string1/string2/
437 Replace all occurrences of characters in string1 with the
438 corresponding characters in string2. If a <backslash> fol‐
439 lowed by an 'n' appear in string1 or string2, the two charac‐
440 ters shall be handled as a single <newline>. If the number
441 of characters in string1 and string2 are not equal, or if any
442 of the characters in string1 appear more than once, the
443 results are undefined. Any character other than <backslash>
444 or <newline> can be used instead of <slash> to delimit the
445 strings. If the delimiter is not 'n', within string1 and
446 string2, the delimiter itself can be used as a literal char‐
447 acter if it is preceded by a <backslash>. If a <backslash>
448 character is immediately followed by a <backslash> character
449 in string1 or string2, the two <backslash> characters shall
450 be counted as a single literal <backslash> character. The
451 meaning of a <backslash> followed by any character that is
452 not 'n', a <backslash>, or the delimiter character is unde‐
453 fined.
454 [0addr]:label
456 Do nothing. This command bears a label to which the b and t
457 commands branch.
458 [1addr]= Write the following to standard output:
460 "%d\n", <current line number>
463 [0addr] Ignore this empty command.
465 [0addr]# Ignore the '#' and the remainder of the line (treat them as a
467 comment), with the single exception that if the first two
468 characters in the script are "#n", the default output shall
469 be suppressed; this shall be the equivalent of specifying -n
470 on the command line.
471
473 The following exit values shall be returned:
474 0 Successful completion.
476 >0 An error occurred.
478
480 Default.
481 The following sections are informative.
483
485 Regular expressions match entire strings, not just individual lines,
486 but a <newline> is matched by '\n' in a sed RE; a <newline> is not
487 allowed by the general definition of regular expression in
488 POSIX.1‐2008. Also note that '\n' cannot be used to match a <newline>
489 at the end of an arbitrary input line; <newline> characters appear in
490 the pattern space as a result of the N editing command.
491 When using sed to process pathnames, it is recommended that LC_ALL, or
493 at least LC_CTYPE and LC_COLLATE, are set to POSIX or C in the environ‐
494 ment, since pathnames can contain byte sequences that do not form valid
495 characters in some locales, in which case the utility's behavior would
496 be undefined. In the POSIX locale each byte is a valid single-byte
497 character, and therefore this problem is avoided.
498
500 This sed script simulates the BSD cat -s command, squeezing excess
501 empty lines from standard input.
502 sed -n '
505 # Write non-empty lines.
506 /./ {
507 p
508 d
509 }
510 # Write a single empty line, then look for more empty lines.
511 /^$/ p
512 # Get next line, discard the held <newline> (empty line),
513 # and look for more empty lines.
514 :Empty
515 /^$/ {
516 N
517 s/.//
518 b Empty
519 }
520 # Write the non-empty line before going back to search
521 # for the first in a set of empty lines.
522 p
523 '
524 The following sed command is a much simpler method of squeezing empty
526 lines, although it is not quite the same as cat -s since it removes any
527 initial empty lines:
528 sed -n '/./,/^$/p'
531
533 This volume of POSIX.1‐2017 requires implementations to support at
534 least ten distinct wfiles, matching historical practice on many imple‐
535 mentations. Implementations are encouraged to support more, but con‐
536 forming applications should not exceed this limit.
537 The exit status codes specified here are different from those in System
539 V. System V returns 2 for garbled sed commands, but returns zero with
540 its usage message or if the input file could not be opened. The stan‐
541 dard developers considered this to be a bug.
542 The manner in which the l command writes non-printable characters was
544 changed to avoid the historical backspace-overstrike method, and other
545 requirements to achieve unambiguous output were added. See the RATIO‐
546 NALE for ed for details of the format chosen, which is the same as that
547 chosen for sed.
548 This volume of POSIX.1‐2017 requires implementations to provide pattern
550 and hold spaces of at least 8192 bytes, larger than the 4000 bytes spa‐
551 ces used by some historical implementations, but less than the 20480
552 bytes limit used in an early proposal. Implementations are encouraged
553 to allocate dynamically larger pattern and hold spaces as needed.
554 The requirements for acceptance of <blank> and <space> characters in
556 command lines has been made more explicit than in early proposals to
557 describe clearly the historical practice and to remove confusion about
558 the phrase ``protect initial blanks [sic] and tabs from the stripping
559 that is done on every script line'' that appears in much of the histor‐
560 ical documentation of the sed utility description of text. (Not all
561 implementations are known to have stripped <blank> characters from text
562 lines, although they all have allowed leading <blank> characters pre‐
563 ceding the address on a command line.)
564 The treatment of '#' comments differs from the SVID which only allows a
566 comment as the first line of the script, but matches BSD-derived imple‐
567 mentations. The comment character is treated as a command, and it has
568 the same properties in terms of being accepted with leading <blank>
569 characters; the BSD implementation has historically supported this.
570 Early proposals required that a script_file have at least one non-com‐
572 ment line. Some historical implementations have behaved in unexpected
573 ways if this were not the case. The standard developers considered that
574 this was incorrect behavior and that application developers should not
575 have to avoid this feature. A correct implementation of this volume of
576 POSIX.1‐2017 shall permit script_files that consist only of comment
577 lines.
578 Early proposals indicated that if -e and -f options were intermixed,
580 all -e options were processed before any -f options. This has been
581 changed to process them in the order presented because it matches his‐
582 torical practice and is more intuitive.
583 The treatment of the p flag to the s command differs between System V
585 and BSD-based systems when the default output is suppressed. In the two
586 examples:
587 echo a | sed 's/a/A/p'
590 echo a | sed -n 's/a/A/p'
591 this volume of POSIX.1‐2017, BSD, System V documentation, and the SVID
593 indicate that the first example should write two lines with A, whereas
594 the second should write one. Some System V systems write the A only
595 once in both examples because the p flag is ignored if the -n option is
596 not specified.
597 This is a case of a diametrical difference between systems that could
599 not be reconciled through the compromise of declaring the behavior to
600 be unspecified. The SVID/BSD/System V documentation behavior was
601 adopted for this volume of POSIX.1‐2017 because:
602 * No known documentation for any historic system describes the inter‐
604 action between the p flag and the -n option.
605 * The selected behavior is more correct as there is no technical jus‐
607 tification for any interaction between the p flag and the -n
608 option. A relationship between -n and the p flag might imply that
609 they are only used together, but this ignores valid scripts that
610 interrupt the cyclical nature of the processing through the use of
611 the D, d, q, or branching commands. Such scripts rely on the p suf‐
612 fix to write the pattern space because they do not make use of the
613 default output at the ``bottom'' of the script.
614 * Because the -n option makes the p flag unnecessary, any interaction
616 would only be useful if sed scripts were written to run both with
617 and without the -n option. This is believed to be unlikely. It is
618 even more unlikely that programmers have coded the p flag expecting
619 it to be unnecessary. Because the interaction was not documented,
620 the likelihood of a programmer discovering the interaction and
621 depending on it is further decreased.
622 * Finally, scripts that break under the specified behavior produce
624 too much output instead of too little, which is easier to diagnose
625 and correct.
626 The form of the substitute command that uses the n suffix was limited
628 to the first 512 matches in an early proposal. This limit has been
629 removed because there is no reason an editor processing lines of
630 {LINE_MAX} length should have this restriction. The command s/a/A/2047
631 should be able to substitute the 2047th occurrence of a on a line.
632 The b, t, and : commands are documented to ignore leading white space,
634 but no mention is made of trailing white space. Historical implementa‐
635 tions of sed assigned different locations to the labels 'x' and "x ".
636 This is not useful, and leads to subtle programming errors, but it is
637 historical practice, and changing it could theoretically break working
638 scripts. Implementors are encouraged to provide warning messages about
639 labels that are never referenced by a b or t command, jumps to labels
640 that do not exist, and label arguments that are subject to truncation.
641 Earlier versions of this standard allowed for implementations with
643 bytes other than eight bits, but this has been modified in this ver‐
644 sion.
645
647 None.
648
650 awk, ed, grep
651 The Base Definitions volume of POSIX.1‐2017, Table 5-1, Escape
653 Sequences and Associated Actions, Chapter 8, Environment Variables,
654 Section 9.3, Basic Regular Expressions, Section 12.2, Utility Syntax
655 Guidelines
656
658 Portions of this text are reprinted and reproduced in electronic form
659 from IEEE Std 1003.1-2017, Standard for Information Technology -- Por‐
660 table Operating System Interface (POSIX), The Open Group Base Specifi‐
661 cations Issue 7, 2018 Edition, Copyright (C) 2018 by the Institute of
662 Electrical and Electronics Engineers, Inc and The Open Group. In the
663 event of any discrepancy between this version and the original IEEE and
664 The Open Group Standard, the original IEEE and The Open Group Standard
665 is the referee document. The original Standard can be obtained online
666 at http://www.opengroup.org/unix/online.html .
667 Any typographical or formatting errors that appear in this page are
669 most likely to have been introduced during the conversion of the source
670 files to man page format. To report such errors, see https://www.ker‐
671 nel.org/doc/man-pages/reporting_bugs.html .
672IEEE/The Open Group 2017 SED(1P)