1AWK(P) POSIX Programmer's Manual AWK(P)
2
6 awk - pattern scanning and processing language
7
9 awk [-F ERE][-v assignment] ... program [argument ...]
10 awk [-F ERE] -f progfile ... [-v assignment] ...[argument ...]
12
15 The awk utility shall execute programs written in the awk programming
16 language, which is specialized for textual data manipulation. An awk
17 program is a sequence of patterns and corresponding actions. When input
18 is read that matches a pattern, the action associated with that pattern
19 is carried out.
20 Input shall be interpreted as a sequence of records. By default, a
22 record is a line, less its terminating <newline>, but this can be
23 changed by using the RS built-in variable. Each record of input shall
24 be matched in turn against each pattern in the program. For each pat‐
25 tern matched, the associated action shall be executed.
26 The awk utility shall interpret each input record as a sequence of
28 fields where, by default, a field is a string of non- <blank>s. This
29 default white-space field delimiter can be changed by using the FS
30 built-in variable or -F ERE. The awk utility shall denote the first
31 field in a record $1, the second $2, and so on. The symbol $0 shall
32 refer to the entire record; setting any other field causes the re-eval‐
33 uation of $0. Assigning to $0 shall reset the values of all other
34 fields and the NF built-in variable.
35
37 The awk utility shall conform to the Base Definitions volume of
38 IEEE Std 1003.1-2001, Section 12.2, Utility Syntax Guidelines.
39 The following options shall be supported:
41 -F ERE
43 Define the input field separator to be the extended regular
44 expression ERE, before any input is read; see Regular Expres‐
45 sions .
46 -f progfile
48 Specify the pathname of the file progfile containing an awk pro‐
49 gram. If multiple instances of this option are specified, the
50 concatenation of the files specified as progfile in the order
51 specified shall be the awk program. The awk program can alterna‐
52 tively be specified in the command line as a single argument.
53 -v assignment
55 The application shall ensure that the assignment argument is in
56 the same form as an assignment operand. The specified variable
57 assignment shall occur prior to executing the awk program,
58 including the actions associated with BEGIN patterns (if any).
59 Multiple occurrences of this option can be specified.
60
63 The following operands shall be supported:
64 program
66 If no -f option is specified, the first operand to awk shall be
67 the text of the awk program. The application shall supply the
68 program operand as a single argument to awk. If the text does
69 not end in a <newline>, awk shall interpret the text as if it
70 did.
71 argument
73 Either of the following two types of argument can be intermixed:
74 file
76 A pathname of a file that contains the input to be read, which
77 is matched against the set of patterns in the program. If no
78 file operands are specified, or if a file operand is '-' , the
79 standard input shall be used.
80 assignment
82 An operand that begins with an underscore or alphabetic charac‐
83 ter from the portable character set (see the table in the Base
84 Definitions volume of IEEE Std 1003.1-2001, Section 6.1, Porta‐
85 ble Character Set), followed by a sequence of underscores, dig‐
86 its, and alphabetics from the portable character set, followed
87 by the '=' character, shall specify a variable assignment rather
88 than a pathname. The characters before the '=' represent the
89 name of an awk variable; if that name is an awk reserved word
90 (see Grammar ) the behavior is undefined. The characters follow‐
91 ing the equal sign shall be interpreted as if they appeared in
92 the awk program preceded and followed by a double-quote ( ' )'
93 character, as a STRING token (see Grammar ), except that if the
94 last character is an unescaped backslash, it shall be inter‐
95 preted as a literal backslash rather than as the first character
96 of the sequence "\"" . The variable shall be assigned the value
97 of that STRING token and, if appropriate, shall be considered a
98 numeric string (see Expressions in awk ), the variable shall
99 also be assigned its numeric value. Each such variable assign‐
100 ment shall occur just prior to the processing of the following
101 file, if any. Thus, an assignment before the first file argument
102 shall be executed after the BEGIN actions (if any), while an
103 assignment after the last file argument shall occur before the
104 END actions (if any). If there are no file arguments, assign‐
105 ments shall be executed before processing the standard input.
106
110 The standard input shall be used only if no file operands are speci‐
111 fied, or if a file operand is '-' ; see the INPUT FILES section. If the
112 awk program contains no actions and no patterns, but is otherwise a
113 valid awk program, standard input and any file operands shall not be
114 read and awk shall exit with a return status of zero.
115
117 Input files to the awk program from any of the following sources shall
118 be text files:
119 * Any file operands or their equivalents, achieved by modifying the
121 awk variables ARGV and ARGC
122 * Standard input in the absence of any file operands
124 * Arguments to the getline function
126 Whether the variable RS is set to a value other than a <newline> or
128 not, for these files, implementations shall support records terminated
129 with the specified separator up to {LINE_MAX} bytes and may support
130 longer records.
131 If -f progfile is specified, the application shall ensure that the
133 files named by each of the progfile option-arguments are text files and
134 their concatenation, in the same order as they appear in the arguments,
135 is an awk program.
136
138 The following environment variables shall affect the execution of awk:
139 LANG Provide a default value for the internationalization variables
141 that are unset or null. (See the Base Definitions volume of
142 IEEE Std 1003.1-2001, Section 8.2, Internationalization Vari‐
143 ables for the precedence of internationalization variables used
144 to determine the values of locale categories.)
145 LC_ALL If set to a non-empty string value, override the values of all
147 the other internationalization variables.
148 LC_COLLATE
150 Determine the locale for the behavior of ranges, equivalence
151 classes, and multi-character collating elements within regular
152 expressions and in comparisons of string values.
153 LC_CTYPE
155 Determine the locale for the interpretation of sequences of
156 bytes of text data as characters (for example, single-byte as
157 opposed to multi-byte characters in arguments and input files),
158 the behavior of character classes within regular expressions,
159 the identification of characters as letters, and the mapping of
160 uppercase and lowercase characters for the toupper and tolower
161 functions.
162 LC_MESSAGES
164 Determine the locale that should be used to affect the format
165 and contents of diagnostic messages written to standard error.
166 LC_NUMERIC
168 Determine the radix character used when interpreting numeric
169 input, performing conversions between numeric and string values,
170 and formatting numeric output. Regardless of locale, the period
171 character (the decimal-point character of the POSIX locale) is
172 the decimal-point character recognized in processing awk pro‐
173 grams (including assignments in command line arguments).
174 NLSPATH
176 Determine the location of message catalogs for the processing of
177 LC_MESSAGES .
178 PATH Determine the search path when looking for commands executed by
180 system(expr), or input and output pipes; see the Base Defini‐
181 tions volume of IEEE Std 1003.1-2001, Chapter 8, Environment
182 Variables.
183 In addition, all environment variables shall be visible via the awk
186 variable ENVIRON.
187
189 Default.
190
192 The nature of the output files depends on the awk program.
193
195 The standard error shall be used only for diagnostic messages.
196
198 The nature of the output files depends on the awk program.
199
201 Overall Program Structure
202 An awk program is composed of pairs of the form:
203 pattern { action }
206 Either the pattern or the action (including the enclosing brace charac‐
208 ters) can be omitted.
209 A missing pattern shall match any record of input, and a missing action
211 shall be equivalent to:
212 { print }
215 Execution of the awk program shall start by first executing the actions
217 associated with all BEGIN patterns in the order they occur in the pro‐
218 gram. Then each file operand (or standard input if no files were speci‐
219 fied) shall be processed in turn by reading data from the file until a
220 record separator is seen ( <newline> by default). Before the first ref‐
221 erence to a field in the record is evaluated, the record shall be split
222 into fields, according to the rules in Regular Expressions , using the
223 value of FS that was current at the time the record was read. Each pat‐
224 tern in the program then shall be evaluated in the order of occurrence,
225 and the action associated with each pattern that matches the current
226 record executed. The action for a matching pattern shall be executed
227 before evaluating subsequent patterns. Finally, the actions associated
228 with all END patterns shall be executed in the order they occur in the
229 program.
230 Expressions in awk
232 Expressions describe computations used in patterns and actions. In the
233 following table, valid expression operations are given in groups from
234 highest precedence first to lowest precedence last, with equal-prece‐
235 dence operators grouped between horizontal lines. In expression evalua‐
236 tion, where the grammar is formally ambiguous, higher precedence opera‐
237 tors shall be evaluated before lower precedence operators. In this ta‐
238 ble expr, expr1, expr2, and expr3 represent any expression, while
239 lvalue represents any entity that can be assigned to (that is, on the
240 left side of an assignment operator). The precise syntax of expressions
241 is given in Grammar .
242 Table: Expressions in Decreasing Precedence in awk
244 Syntax Name Type of Result Associativity
246 ( expr ) Grouping Type of expr N/A
247 $expr Field reference String N/A
248 ++ lvalue Pre-increment Numeric N/A
249 -- lvalue Pre-decrement Numeric N/A
250 lvalue ++ Post-increment Numeric N/A
251 lvalue -- Post-decrement Numeric N/A
252 expr ^ expr Exponentiation Numeric Right
253 ! expr Logical not Numeric N/A
254 + expr Unary plus Numeric N/A
255 - expr Unary minus Numeric N/A
256 expr * expr Multiplication Numeric Left
257 expr / expr Division Numeric Left
258 expr % expr Modulus Numeric Left
259 expr + expr Addition Numeric Left
260 expr - expr Subtraction Numeric Left
261 expr expr String concatenation String Left
262 expr < expr Less than Numeric None
263 expr <= expr Less than or equal to Numeric None
264 expr != expr Not equal to Numeric None
265 expr == expr Equal to Numeric None
266 expr > expr Greater than Numeric None
267 expr >= expr Greater than or equal to Numeric None
268 expr ~ expr ERE match Numeric None
269 expr !~ expr ERE non-match Numeric None
270 expr in array Array membership Numeric Left
271 ( index ) in array Multi-dimension array Numeric Left
272 membership
274 expr && expr Logical AND Numeric Left
275 expr || expr Logical OR Numeric Left
276 expr1 ? expr2 : expr3 Conditional expression Type of selected Right
277 expr2 or expr3
278 lvalue ^= expr Exponentiation assignment Numeric Right
279 lvalue %= expr Modulus assignment Numeric Right
280 lvalue *= expr Multiplication assignment Numeric Right
281 lvalue /= expr Division assignment Numeric Right
282 lvalue += expr Addition assignment Numeric Right
283 lvalue -= expr Subtraction assignment Numeric Right
284 lvalue = expr Assignment Type of expr Right
285 Each expression shall have either a string value, a numeric value, or
287 both. Except as stated for specific contexts, the value of an expres‐
288 sion shall be implicitly converted to the type needed for the context
289 in which it is used. A string value shall be converted to a numeric
290 value by the equivalent of the following calls to functions defined by
291 the ISO C standard:
292 setlocale(LC_NUMERIC, "");
295 numeric_value = atof(string_value);
296 A numeric value that is exactly equal to the value of an integer (see
298 Concepts Derived from the ISO C Standard ) shall be converted to a
299 string by the equivalent of a call to the sprintf function (see String
300 Functions ) with the string "%d" as the fmt argument and the numeric
301 value being converted as the first and only expr argument. Any other
302 numeric value shall be converted to a string by the equivalent of a
303 call to the sprintf function with the value of the variable CONVFMT as
304 the fmt argument and the numeric value being converted as the first and
305 only expr argument. The result of the conversion is unspecified if the
306 value of CONVFMT is not a floating-point format specification. This
307 volume of IEEE Std 1003.1-2001 specifies no explicit conversions
308 between numbers and strings. An application can force an expression to
309 be treated as a number by adding zero to it, or can force it to be
310 treated as a string by concatenating the null string ( "" ) to it.
311 A string value shall be considered a numeric string if it comes from
313 one of the following:
314 1. Field variables
316 2. Input from the getline() function
318 3. FILENAME
320 4. ARGV array elements
322 5. ENVIRON array elements
324 6. Array elements created by the split() function
326 7. A command line variable assignment
328 8. Variable assignment from another numeric string variable
330 and after all the following conversions have been applied, the result‐
332 ing string would lexically be recognized as a NUMBER token as described
333 by the lexical conventions in Grammar :
334 * All leading and trailing <blank>s are discarded.
336 * If the first non- <blank> is '+' or '-' , it is discarded.
338 * Changing each occurrence of the decimal point character from the
340 current locale to a period.
341 If a '-' character is ignored in the preceding description, the numeric
343 value of the numeric string shall be the negation of the numeric value
344 of the recognized NUMBER token. Otherwise, the numeric value of the
345 numeric string shall be the numeric value of the recognized NUMBER
346 token. Whether or not a string is a numeric string shall be relevant
347 only in contexts where that term is used in this section.
348 When an expression is used in a Boolean context, if it has a numeric
350 value, a value of zero shall be treated as false and any other value
351 shall be treated as true. Otherwise, a string value of the null string
352 shall be treated as false and any other value shall be treated as true.
353 A Boolean context shall be one of the following:
354 * The first subexpression of a conditional expression
356 * An expression operated on by logical NOT, logical AND, or logical OR
358 * The second expression of a for statement
360 * The expression of an if statement
362 * The expression of the while clause in either a while or do... while
364 statement
365 * An expression used as a pattern (as in Overall Program Structure)
367 All arithmetic shall follow the semantics of floating-point arithmetic
369 as specified by the ISO C standard (see Concepts Derived from the ISO C
370 Standard ).
371 The value of the expression:
373 expr1 ^ expr2
376 shall be equivalent to the value returned by the ISO C standard func‐
378 tion call:
379 pow(expr1, expr2)
382 The expression:
384 lvalue ^= expr
387 shall be equivalent to the ISO C standard expression:
389 lvalue = pow(lvalue, expr)
392 except that lvalue shall be evaluated only once. The value of the
394 expression:
395 expr1 % expr2
398 shall be equivalent to the value returned by the ISO C standard func‐
400 tion call:
401 fmod(expr1, expr2)
404 The expression:
406 lvalue %= expr
409 shall be equivalent to the ISO C standard expression:
411 lvalue = fmod(lvalue, expr)
414 except that lvalue shall be evaluated only once.
416 Variables and fields shall be set by the assignment statement:
418 lvalue = expression
421 and the type of expression shall determine the resulting variable type.
423 The assignment includes the arithmetic assignments ( "+=" , "-=" , "*="
424 , "/=" , "%=" , "^=" , "++" , "--" ) all of which shall produce a
425 numeric result. The left-hand side of an assignment and the target of
426 increment and decrement operators can be one of a variable, an array
427 with index, or a field selector.
428 The awk language supplies arrays that are used for storing numbers or
430 strings. Arrays need not be declared. They shall initially be empty,
431 and their sizes shall change dynamically. The subscripts, or element
432 identifiers, are strings, providing a type of associative array capa‐
433 bility. An array name followed by a subscript within square brackets
434 can be used as an lvalue and thus as an expression, as described in the
435 grammar; see Grammar . Unsubscripted array names can be used in only
436 the following contexts:
437 * A parameter in a function definition or function call
439 * The NAME token following any use of the keyword in as specified in
441 the grammar (see Grammar ); if the name used in this context is not
442 an array name, the behavior is undefined
443 A valid array index shall consist of one or more comma-separated
445 expressions, similar to the way in which multi-dimensional arrays are
446 indexed in some programming languages. Because awk arrays are really
447 one-dimensional, such a comma-separated list shall be converted to a
448 single string by concatenating the string values of the separate
449 expressions, each separated from the other by the value of the SUBSEP
450 variable. Thus, the following two index operations shall be equiva‐
451 lent:
452 var[expr1, expr2, ... exprn]
455 var[expr1 SUBSEP expr2 SUBSEP ... SUBSEP exprn]
458 The application shall ensure that a multi-dimensioned index used with
460 the in operator is parenthesized. The in operator, which tests for the
461 existence of a particular array element, shall not cause that element
462 to exist. Any other reference to a nonexistent array element shall
463 automatically create it.
464 Comparisons (with the '<' , "<=" , "!=" , "==" , '>' , and ">=" opera‐
466 tors) shall be made numerically if both operands are numeric, if one is
467 numeric and the other has a string value that is a numeric string, or
468 if one is numeric and the other has the uninitialized value. Otherwise,
469 operands shall be converted to strings as required and a string compar‐
470 ison shall be made using the locale-specific collation sequence. The
471 value of the comparison expression shall be 1 if the relation is true,
472 or 0 if the relation is false.
473 Variables and Special Variables
475 Variables can be used in an awk program by referencing them. With the
476 exception of function parameters (see User-Defined Functions ), they
477 are not explicitly declared. Function parameter names shall be local to
478 the function; all other variable names shall be global. The same name
479 shall not be used as both a function parameter name and as the name of
480 a function or a special awk variable. The same name shall not be used
481 both as a variable name with global scope and as the name of a func‐
482 tion. The same name shall not be used within the same scope both as a
483 scalar variable and as an array. Uninitialized variables, including
484 scalar variables, array elements, and field variables, shall have an
485 uninitialized value. An uninitialized value shall have both a numeric
486 value of zero and a string value of the empty string. Evaluation of
487 variables with an uninitialized value, to either string or numeric,
488 shall be determined by the context in which they are used.
489 Field variables shall be designated by a '$' followed by a number or
491 numerical expression. The effect of the field number expression evalu‐
492 ating to anything other than a non-negative integer is unspecified;
493 uninitialized variables or string values need not be converted to
494 numeric values in this context. New field variables can be created by
495 assigning a value to them. References to nonexistent fields (that is,
496 fields after $NF), shall evaluate to the uninitialized value. Such ref‐
497 erences shall not create new fields. However, assigning to a nonexis‐
498 tent field (for example, $(NF+2)=5) shall increase the value of NF;
499 create any intervening fields with the uninitialized value; and cause
500 the value of $0 to be recomputed, with the fields being separated by
501 the value of OFS. Each field variable shall have a string value or an
502 uninitialized value when created. Field variables shall have the
503 uninitialized value when created from $0 using FS and the variable does
504 not contain any characters. If appropriate, the field variable shall be
505 considered a numeric string (see Expressions in awk ).
506 Implementations shall support the following other special variables
508 that are set by awk:
509 ARGC The number of elements in the ARGV array.
511 ARGV An array of command line arguments, excluding options and the
513 program argument, numbered from zero to ARGC-1.
514 The arguments in ARGV can be modified or added to; ARGC can be altered.
516 As each input file ends, awk shall treat the next non-null element of
517 ARGV, up to the current value of ARGC-1, inclusive, as the name of the
518 next input file. Thus, setting an element of ARGV to null means that it
519 shall not be treated as an input file. The name '-' indicates the stan‐
520 dard input. If an argument matches the format of an assignment operand,
521 this argument shall be treated as an assignment rather than a file
522 argument.
523 CONVFMT
525 The printf format for converting numbers to strings (except for
526 output statements, where OFMT is used); "%.6g" by default.
527 ENVIRON
529 An array representing the value of the environment, as described
530 in the exec functions defined in the System Interfaces volume of
531 IEEE Std 1003.1-2001. The indices of the array shall be strings
532 consisting of the names of the environment variables, and the
533 value of each array element shall be a string consisting of the
534 value of that variable. If appropriate, the environment variable
535 shall be considered a numeric string (see Expressions in awk );
536 the array element shall also have its numeric value.
537 In all cases where the behavior of awk is affected by environment vari‐
539 ables (including the environment of any commands that awk executes via
540 the system function or via pipeline redirections with the print state‐
541 ment, the printf statement, or the getline function), the environment
542 used shall be the environment at the time awk began executing; it is
543 implementation-defined whether any modification of ENVIRON affects this
544 environment.
545 FILENAME
547 A pathname of the current input file. Inside a BEGIN action the
548 value is undefined. Inside an END action the value shall be the
549 name of the last input file processed.
550 FNR The ordinal number of the current record in the current file.
552 Inside a BEGIN action the value shall be zero. Inside an END
553 action the value shall be the number of the last record pro‐
554 cessed in the last file processed.
555 FS Input field separator regular expression; a <space> by default.
557 NF The number of fields in the current record. Inside a BEGIN
559 action, the use of NF is undefined unless a getline function
560 without a var argument is executed previously. Inside an END
561 action, NF shall retain the value it had for the last record
562 read, unless a subsequent, redirected, getline function without
563 a var argument is performed prior to entering the END action.
564 NR The ordinal number of the current record from the start of
566 input. Inside a BEGIN action the value shall be zero. Inside an
567 END action the value shall be the number of the last record pro‐
568 cessed.
569 OFMT The printf format for converting numbers to strings in output
571 statements (see Output Statements ); "%.6g" by default. The
572 result of the conversion is unspecified if the value of OFMT is
573 not a floating-point format specification.
574 OFS The print statement output field separation; <space> by default.
576 ORS The print statement output record separator; a <newline> by
578 default.
579 RLENGTH
581 The length of the string matched by the match function.
582 RS The first character of the string value of RS shall be the input
584 record separator; a <newline> by default. If RS contains more
585 than one character, the results are unspecified. If RS is null,
586 then records are separated by sequences consisting of a <new‐
587 line> plus one or more blank lines, leading or trailing blank
588 lines shall not result in empty records at the beginning or end
589 of the input, and a <newline> shall always be a field separator,
590 no matter what the value of FS is.
591 RSTART The starting position of the string matched by the match func‐
593 tion, numbering from 1. This shall always be equivalent to the
594 return value of the match function.
595 SUBSEP The subscript separator string for multi-dimensional arrays; the
597 default value is implementation-defined.
598 Regular Expressions
601 The awk utility shall make use of the extended regular expression nota‐
602 tion (see the Base Definitions volume of IEEE Std 1003.1-2001, Section
603 9.4, Extended Regular Expressions) except that it shall allow the use
604 of C-language conventions for escaping special characters within the
605 EREs, as specified in the table in the Base Definitions volume of
606 IEEE Std 1003.1-2001, Chapter 5, File Format Notation ( '\\' , '\a' ,
607 '\b' , '\f' , '\n' , '\r' , '\t' , '\v' ) and the following table;
608 these escape sequences shall be recognized both inside and outside
609 bracket expressions. Note that records need not be separated by <new‐
610 line>s and string constants can contain <newline>s, so even the "\n"
611 sequence is valid in awk EREs. Using a slash character within an ERE
612 requires the escaping shown in the following table.
613 Table: Escape Sequences in awk
615 Escape
617 Sequence Description Meaning
619 \" Backslash quotation-mark Quotation-mark character
620 \/ Backslash slash Slash character
621 \ddd A backslash character followed The character whose encoding
622 by the longest sequence of is represented by the one,
623 one, two, or three octal-digit two, or three-digit octal
624 characters [4m(01234567). If all integer. Multi-byte characters
625 of the digits are 0 (that is, require multiple, concatenated
626 representation of the NUL escape sequences of this type,
627 character), the behavior is including the leading '\' for
628 undefined. each byte.
629 \c A backslash character followed Undefined
630 by any character not described
631 in this table or in the table
632 in the Base Definitions volume
633 of IEEE Std 1003.1-2001, Chap‐
634 ter 5, File Format Notation (
635 '\\' , '\a' , '\b' , '\f' ,
636 '\n' , '\r' , '\t' , '\v' ).
637 A regular expression can be matched against a specific field or string
639 by using one of the two regular expression matching operators, '~' and
640 "!~" . These operators shall interpret their right-hand operand as a
641 regular expression and their left-hand operand as a string. If the reg‐
642 ular expression matches the string, the '~' expression shall evaluate
643 to a value of 1, and the "!~" expression shall evaluate to a value of
644 0. (The regular expression matching operation is as defined by the term
645 matched in the Base Definitions volume of IEEE Std 1003.1-2001, Section
646 9.1, Regular Expression Definitions, where a match occurs on any part
647 of the string unless the regular expression is limited with the circum‐
648 flex or dollar sign special characters.) If the regular expression does
649 not match the string, the '~' expression shall evaluate to a value of
650 0, and the "!~" expression shall evaluate to a value of 1. If the
651 right-hand operand is any expression other than the lexical token ERE,
652 the string value of the expression shall be interpreted as an extended
653 regular expression, including the escape conventions described above.
654 Note that these same escape conventions shall also be applied in deter‐
655 mining the value of a string literal (the lexical token STRING), and
656 thus shall be applied a second time when a string literal is used in
657 this context.
658 When an ERE token appears as an expression in any context other than as
660 the right-hand of the '~' or "!~" operator or as one of the built-in
661 function arguments described below, the value of the resulting expres‐
662 sion shall be the equivalent of:
663 $0 ~ /ere/
666 The ere argument to the gsub, match, sub functions, and the fs argument
668 to the split function (see String Functions ) shall be interpreted as
669 extended regular expressions. These can be either ERE tokens or arbi‐
670 trary expressions, and shall be interpreted in the same manner as the
671 right-hand side of the '~' or "!~" operator.
672 An extended regular expression can be used to separate fields by using
674 the -F ERE option or by assigning a string containing the expression to
675 the built-in variable FS. The default value of the FS variable shall be
676 a single <space>. The following describes FS behavior:
677 1. If FS is a null string, the behavior is unspecified.
679 2. If FS is a single character:
681 a. If FS is <space>, skip leading and trailing <blank>s; fields
683 shall be delimited by sets of one or more <blank>s.
684 b. Otherwise, if FS is any other character c, fields shall be
686 delimited by each single occurrence of c.
687 3. Otherwise, the string value of FS shall be considered to be an
689 extended regular expression. Each occurrence of a sequence matching
690 the extended regular expression shall delimit fields.
691 Except for the '~' and "!~" operators, and in the gsub, match, split,
693 and sub built-in functions, ERE matching shall be based on input
694 records; that is, record separator characters (the first character of
695 the value of the variable RS, <newline> by default) cannot be embedded
696 in the expression, and no expression shall match the record separator
697 character. If the record separator is not <newline>, <newline>s embed‐
698 ded in the expression can be matched. For the '~' and "!~" operators,
699 and in those four built-in functions, ERE matching shall be based on
700 text strings; that is, any character (including <newline> and the
701 record separator) can be embedded in the pattern, and an appropriate
702 pattern shall match any character. However, in all awk ERE matching,
703 the use of one or more NUL characters in the pattern, input record, or
704 text string produces undefined results.
705 Patterns
707 A pattern is any valid expression, a range specified by two expressions
708 separated by a comma, or one of the two special patterns BEGIN or END.
709 Special Patterns
711 The awk utility shall recognize two special patterns, BEGIN and END.
712 Each BEGIN pattern shall be matched once and its associated action exe‐
713 cuted before the first record of input is read (except possibly by use
714 of the getline function-see Input/Output and General Functions - in a
715 prior BEGIN action) and before command line assignment is done. Each
716 END pattern shall be matched once and its associated action executed
717 after the last record of input has been read. These two patterns shall
718 have associated actions.
719 BEGIN and END shall not combine with other patterns. Multiple BEGIN and
721 END patterns shall be allowed. The actions associated with the BEGIN
722 patterns shall be executed in the order specified in the program, as
723 are the END actions. An END pattern can precede a BEGIN pattern in a
724 program.
725 If an awk program consists of only actions with the pattern BEGIN, and
727 the BEGIN action contains no getline function, awk shall exit without
728 reading its input when the last statement in the last BEGIN action is
729 executed. If an awk program consists of only actions with the pattern
730 END or only actions with the patterns BEGIN and END, the input shall be
731 read before the statements in the END actions are executed.
732 Expression Patterns
734 An expression pattern shall be evaluated as if it were an expression in
735 a Boolean context. If the result is true, the pattern shall be consid‐
736 ered to match, and the associated action (if any) shall be executed. If
737 the result is false, the action shall not be executed.
738 Pattern Ranges
740 A pattern range consists of two expressions separated by a comma; in
741 this case, the action shall be performed for all records between a
742 match of the first expression and the following match of the second
743 expression, inclusive. At this point, the pattern range can be repeated
744 starting at input records subsequent to the end of the matched range.
745 Actions
747 An action is a sequence of statements as shown in the grammar in Gram‐
748 mar . Any single statement can be replaced by a statement list enclosed
749 in braces. The application shall ensure that statements in a statement
750 list are separated by <newline>s or semicolons. Statements in a state‐
751 ment list shall be executed sequentially in the order that they appear.
752 The expression acting as the conditional in an if statement shall be
754 evaluated and if it is non-zero or non-null, the following statement
755 shall be executed; otherwise, if else is present, the statement follow‐
756 ing the else shall be executed.
757 The if, while, do... while, for, break, and continue statements are
759 based on the ISO C standard (see Concepts Derived from the ISO C Stan‐
760 dard ), except that the Boolean expressions shall be treated as
761 described in Expressions in awk , and except in the case of:
762 for (variable in array)
765 which shall iterate, assigning each index of array to variable in an
767 unspecified order. The results of adding new elements to array within
768 such a for loop are undefined. If a break or continue statement occurs
769 outside of a loop, the behavior is undefined.
770 The delete statement shall remove an individual array element. Thus,
772 the following code deletes an entire array:
773 for (index in array)
776 delete array[index]
777 The next statement shall cause all further processing of the current
779 input record to be abandoned. The behavior is undefined if a next
780 statement appears or is invoked in a BEGIN or END action.
781 The exit statement shall invoke all END actions in the order in which
783 they occur in the program source and then terminate the program without
784 reading further input. An exit statement inside an END action shall
785 terminate the program without further execution of END actions. If an
786 expression is specified in an exit statement, its numeric value shall
787 be the exit status of awk, unless subsequent errors are encountered or
788 a subsequent exit statement with an expression is executed.
789 Output Statements
791 Both print and printf statements shall write to standard output by
792 default. The output shall be written to the location specified by out‐
793 put_redirection if one is supplied, as follows:
794 > expression>> expression| expression
797 In all cases, the expression shall be evaluated to produce a string
799 that is used as a pathname into which to write (for '>' or ">>" ) or as
800 a command to be executed (for '|' ). Using the first two forms, if the
801 file of that name is not currently open, it shall be opened, creating
802 it if necessary and using the first form, truncating the file. The out‐
803 put then shall be appended to the file. As long as the file remains
804 open, subsequent calls in which expression evaluates to the same string
805 value shall simply append output to the file. The file remains open
806 until the close function (see Input/Output and General Functions ) is
807 called with an expression that evaluates to the same string value.
808 The third form shall write output onto a stream piped to the input of a
810 command. The stream shall be created if no stream is currently open
811 with the value of expression as its command name. The stream created
812 shall be equivalent to one created by a call to the popen() function
813 defined in the System Interfaces volume of IEEE Std 1003.1-2001 with
814 the value of expression as the command argument and a value of w as the
815 mode argument. As long as the stream remains open, subsequent calls in
816 which expression evaluates to the same string value shall write output
817 to the existing stream. The stream shall remain open until the close
818 function (see Input/Output and General Functions ) is called with an
819 expression that evaluates to the same string value. At that time, the
820 stream shall be closed as if by a call to the pclose() function defined
821 in the System Interfaces volume of IEEE Std 1003.1-2001.
822 As described in detail by the grammar in Grammar , these output state‐
824 ments shall take a comma-separated list of expressions referred to in
825 the grammar by the non-terminal symbols expr_list, print_expr_list, or
826 print_expr_list_opt. This list is referred to here as the expression
827 list, and each member is referred to as an expression argument.
828 The print statement shall write the value of each expression argument
830 onto the indicated output stream separated by the current output field
831 separator (see variable OFS above), and terminated by the output record
832 separator (see variable ORS above). All expression arguments shall be
833 taken as strings, being converted if necessary; this conversion shall
834 be as described in Expressions in awk , with the exception that the
835 printf format in OFMT shall be used instead of the value in CONVFMT. An
836 empty expression list shall stand for the whole input record ($0).
837 The printf statement shall produce output based on a notation similar
839 to the File Format Notation used to describe file formats in this vol‐
840 ume of IEEE Std 1003.1-2001 (see the Base Definitions volume of
841 IEEE Std 1003.1-2001, Chapter 5, File Format Notation). Output shall
842 be produced as specified with the first expression argument as the
843 string format and subsequent expression arguments as the strings arg1
844 to argn, inclusive, with the following exceptions:
845 1. The format shall be an actual character string rather than a graph‐
847 ical representation. Therefore, it cannot contain empty character
848 positions. The <space> in the format string, in any context other
849 than a flag of a conversion specification, shall be treated as an
850 ordinary character that is copied to the output.
851 2. If the character set contains a ' ' character and that character
853 appears in the format string, it shall be treated as an ordinary
854 character that is copied to the output.
855 3. The escape sequences beginning with a backslash character shall be
857 treated as sequences of ordinary characters that are copied to the
858 output. Note that these same sequences shall be interpreted lexi‐
859 cally by awk when they appear in literal strings, but they shall
860 not be treated specially by the printf statement.
861 4. A field width or precision can be specified as the '*' character
863 instead of a digit string. In this case the next argument from the
864 expression list shall be fetched and its numeric value taken as the
865 field width or precision.
866 5. The implementation shall not precede or follow output from the d or
868 u conversion specifier characters with <blank>s not specified by
869 the format string.
870 6. The implementation shall not precede output from the o conversion
872 specifier character with leading zeros not specified by the format
873 string.
874 7. For the c conversion specifier character: if the argument has a
876 numeric value, the character whose encoding is that value shall be
877 output. If the value is zero or is not the encoding of any charac‐
878 ter in the character set, the behavior is undefined. If the argu‐
879 ment does not have a numeric value, the first character of the
880 string value shall be output; if the string does not contain any
881 characters, the behavior is undefined.
882 8. For each conversion specification that consumes an argument, the
884 next expression argument shall be evaluated. With the exception of
885 the c conversion specifier character, the value shall be converted
886 (according to the rules specified in Expressions in awk ) to the
887 appropriate type for the conversion specification.
888 9. If there are insufficient expression arguments to satisfy all the
890 conversion specifications in the format string, the behavior is
891 undefined.
892 10. If any character sequence in the format string begins with a '%'
894 character, but does not form a valid conversion specification, the
895 behavior is unspecified.
896 Both print and printf can output at least {LINE_MAX} bytes.
898 Functions
900 The awk language has a variety of built-in functions: arithmetic,
901 string, input/output, and general.
902 Arithmetic Functions
904 The arithmetic functions, except for int, shall be based on the ISO C
905 standard (see Concepts Derived from the ISO C Standard ). The behavior
906 is undefined in cases where the ISO C standard specifies that an error
907 be returned or that the behavior is undefined. Although the grammar
908 (see Grammar ) permits built-in functions to appear with no arguments
909 or parentheses, unless the argument or parentheses are indicated as
910 optional in the following list (by displaying them within the "[]"
911 brackets), such use is undefined.
912 atan2(y,x)
914 Return arctangent of y/x in radians in the range [-pi,pi].
915 cos(x) Return cosine of x, where x is in radians.
917 sin(x) Return sine of x, where x is in radians.
919 exp(x) Return the exponential function of x.
921 log(x) Return the natural logarithm of x.
923 sqrt(x)
925 Return the square root of x.
926 int(x) Return the argument truncated to an integer. Truncation shall be
928 toward 0 when x>0.
929 rand() Return a random number n, such that 0<=n<1.
931 srand([expr])
933 Set the seed value for rand to expr or use the time of day if
934 expr is omitted. The previous seed value shall be returned.
935 String Functions
938 The string functions in the following list shall be supported. Although
939 the grammar (see Grammar ) permits built-in functions to appear with no
940 arguments or parentheses, unless the argument or parentheses are indi‐
941 cated as optional in the following list (by displaying them within the
942 "[]" brackets), such use is undefined.
943 gsub(ere, repl[, in])
945 Behave like sub (see below), except that it shall replace all
946 occurrences of the regular expression (like the ed utility
947 global substitute) in $0 or in the in argument, when specified.
948 index(s, t)
950 Return the position, in characters, numbering from 1, in string
951 s where string t first occurs, or zero if it does not occur at
952 all.
953 length[([s])]
955 Return the length, in characters, of its argument taken as a
956 string, or of the whole record, $0, if there is no argument.
957 match(s, ere)
959 Return the position, in characters, numbering from 1, in string
960 s where the extended regular expression ere occurs, or zero if
961 it does not occur at all. RSTART shall be set to the starting
962 position (which is the same as the returned value), zero if no
963 match is found; RLENGTH shall be set to the length of the
964 matched string, -1 if no match is found.
965 split(s, a[, fs ])
967 Split the string s into array elements a[1], a[2], ..., a[n],
968 and return n. All elements of the array shall be deleted before
969 the split is performed. The separation shall be done with the
970 ERE fs or with the field separator FS if fs is not given. Each
971 array element shall have a string value when created and, if
972 appropriate, the array element shall be considered a numeric
973 string (see Expressions in awk ). The effect of a null string as
974 the value of fs is unspecified.
975 sprintf(fmt, expr, expr, ...)
977 Format the expressions according to the printf format given by
978 fmt and return the resulting string.
979 sub(ere, repl[, in ])
981 Substitute the string repl in place of the first instance of the
982 extended regular expression ERE in string in and return the num‐
983 ber of substitutions. An ampersand ( '&' ) appearing in the
984 string repl shall be replaced by the string from in that matches
985 the ERE. An ampersand preceded with a backslash ( '\' ) shall be
986 interpreted as the literal ampersand character. An occurrence of
987 two consecutive backslashes shall be interpreted as just a sin‐
988 gle literal backslash character. Any other occurrence of a back‐
989 slash (for example, preceding any other character) shall be
990 treated as a literal backslash character. Note that if repl is a
991 string literal (the lexical token STRING; see Grammar ), the
992 handling of the ampersand character occurs after any lexical
993 processing, including any lexical backslash escape sequence pro‐
994 cessing. If in is specified and it is not an lvalue (see Expres‐
995 sions in awk ), the behavior is undefined. If in is omitted, awk
996 shall use the current record ($0) in its place.
997 substr(s, m[, n ])
999 Return the at most n-character substring of s that begins at
1000 position m, numbering from 1. If n is omitted, or if n specifies
1001 more characters than are left in the string, the length of the
1002 substring shall be limited by the length of the string s.
1003 tolower(s)
1005 Return a string based on the string s. Each character in s that
1006 is an uppercase letter specified to have a tolower mapping by
1007 the LC_CTYPE category of the current locale shall be replaced in
1008 the returned string by the lowercase letter specified by the
1009 mapping. Other characters in s shall be unchanged in the
1010 returned string.
1011 toupper(s)
1013 Return a string based on the string s. Each character in s that
1014 is a lowercase letter specified to have a toupper mapping by the
1015 LC_CTYPE category of the current locale is replaced in the
1016 returned string by the uppercase letter specified by the map‐
1017 ping. Other characters in s are unchanged in the returned
1018 string.
1019 All of the preceding functions that take ERE as a parameter expect a
1022 pattern or a string valued expression that is a regular expression as
1023 defined in Regular Expressions .
1024 Input/Output and General Functions
1026 The input/output and general functions are:
1027 close(expression)
1029 Close the file or pipe opened by a print or printf statement or
1030 a call to getline with the same string-valued expression. The
1031 limit on the number of open expression arguments is implementa‐
1032 tion-defined. If the close was successful, the function shall
1033 return zero; otherwise, it shall return non-zero.
1034 expression | getline [var]
1036 Read a record of input from a stream piped from the output of a
1037 command. The stream shall be created if no stream is currently
1038 open with the value of expression as its command name. The
1039 stream created shall be equivalent to one created by a call to
1040 the popen() function with the value of expression as the command
1041 argument and a value of r as the mode argument. As long as the
1042 stream remains open, subsequent calls in which expression evalu‐
1043 ates to the same string value shall read subsequent records from
1044 the stream. The stream shall remain open until the close func‐
1045 tion is called with an expression that evaluates to the same
1046 string value. At that time, the stream shall be closed as if by
1047 a call to the pclose() function. If var is omitted, $0 and NF
1048 shall be set; otherwise, var shall be set and, if appropriate,
1049 it shall be considered a numeric string (see Expressions in awk
1050 ).
1051 The getline operator can form ambiguous constructs when there are
1053 unparenthesized operators (including concatenate) to the left of the
1054 '|' (to the beginning of the expression containing getline). In the
1055 context of the '$' operator, '|' shall behave as if it had a lower
1056 precedence than '$' . The result of evaluating other operators is
1057 unspecified, and conforming applications shall parenthesize properly
1058 all such usages.
1059 getline
1061 Set $0 to the next input record from the current input file.
1062 This form of getline shall set the NF, NR, and FNR variables.
1063 getline var
1065 Set variable var to the next input record from the current input
1066 file and, if appropriate, var shall be considered a numeric
1067 string (see Expressions in awk ). This form of getline shall set
1068 the FNR and NR variables.
1069 getline [var] < expression
1071 Read the next record of input from a named file. The expression
1072 shall be evaluated to produce a string that is used as a path‐
1073 name. If the file of that name is not currently open, it shall
1074 be opened. As long as the stream remains open, subsequent calls
1075 in which expression evaluates to the same string value shall
1076 read subsequent records from the file. The file shall remain
1077 open until the close function is called with an expression that
1078 evaluates to the same string value. If var is omitted, $0 and NF
1079 shall be set; otherwise, var shall be set and, if appropriate,
1080 it shall be considered a numeric string (see Expressions in awk
1081 ).
1082 The getline operator can form ambiguous constructs when there are
1084 unparenthesized binary operators (including concatenate) to the right
1085 of the '<' (up to the end of the expression containing the getline).
1086 The result of evaluating such a construct is unspecified, and conform‐
1087 ing applications shall parenthesize properly all such usages.
1088 system(expression)
1090 Execute the command given by expression in a manner equivalent
1091 to the system() function defined in the System Interfaces volume
1092 of IEEE Std 1003.1-2001 and return the exit status of the com‐
1093 mand.
1094 All forms of getline shall return 1 for successful input, zero for end-
1097 of-file, and -1 for an error.
1098 Where strings are used as the name of a file or pipeline, the applica‐
1100 tion shall ensure that the strings are textually identical. The termi‐
1101 nology "same string value" implies that "equivalent strings", even
1102 those that differ only by <space>s, represent different files.
1103 User-Defined Functions
1105 The awk language also provides user-defined functions. Such functions
1106 can be defined as:
1107 function name([parameter, ...]) { statements }
1110 A function can be referred to anywhere in an awk program; in particu‐
1112 lar, its use can precede its definition. The scope of a function is
1113 global.
1114 Function parameters, if present, can be either scalars or arrays; the
1116 behavior is undefined if an array name is passed as a parameter that
1117 the function uses as a scalar, or if a scalar expression is passed as a
1118 parameter that the function uses as an array. Function parameters shall
1119 be passed by value if scalar and by reference if array name.
1120 The number of parameters in the function definition need not match the
1122 number of parameters in the function call. Excess formal parameters can
1123 be used as local variables. If fewer arguments are supplied in a func‐
1124 tion call than are in the function definition, the extra parameters
1125 that are used in the function body as scalars shall evaluate to the
1126 uninitialized value until they are otherwise initialized, and the extra
1127 parameters that are used in the function body as arrays shall be
1128 treated as uninitialized arrays where each element evaluates to the
1129 uninitialized value until otherwise initialized.
1130 When invoking a function, no white space can be placed between the
1132 function name and the opening parenthesis. Function calls can be nested
1133 and recursive calls can be made upon functions. Upon return from any
1134 nested or recursive function call, the values of all of the calling
1135 function's parameters shall be unchanged, except for array parameters
1136 passed by reference. The return statement can be used to return a
1137 value. If a return statement appears outside of a function definition,
1138 the behavior is undefined.
1139 In the function definition, <newline>s shall be optional before the
1141 opening brace and after the closing brace. Function definitions can
1142 appear anywhere in the program where a pattern-action pair is allowed.
1143 Grammar
1145 The grammar in this section and the lexical conventions in the follow‐
1146 ing section shall together describe the syntax for awk programs. The
1147 general conventions for this style of grammar are described in Grammar
1148 Conventions . A valid program can be represented as the non-terminal
1149 symbol program in the grammar. This formal syntax shall take precedence
1150 over the preceding text syntax description.
1151 %token NAME NUMBER STRING ERE
1154 %token FUNC_NAME /* Name followed by '(' without white space. */
1155 /* Keywords */
1158 %token Begin End
1159 /* 'BEGIN' 'END' */
1160 %token Break Continue Delete Do Else
1163 /* 'break' 'continue' 'delete' 'do' 'else' */
1164 %token Exit For Function If In
1167 /* 'exit' 'for' 'function' 'if' 'in' */
1168 %token Next Print Printf Return While
1171 /* 'next' 'print' 'printf' 'return' 'while' */
1172 /* Reserved function names */
1175 %token BUILTIN_FUNC_NAME
1176 /* One token for the following:
1177 * atan2 cos sin exp log sqrt int rand srand
1178 * gsub index length match split sprintf sub
1179 * substr tolower toupper close system
1180 */
1181 %token GETLINE
1182 /* Syntactically different from other built-ins. */
1183 /* Two-character tokens. */
1186 %token ADD_ASSIGN SUB_ASSIGN MUL_ASSIGN DIV_ASSIGN MOD_ASSIGN POW_ASSIGN
1187 /* '+=' '-=' '*=' '/=' '%=' '^=' */
1188 %token OR AND NO_MATCH EQ LE GE NE INCR DECR APPEND
1191 /* '||' '&&' '!~' '==' '<=' '>=' '!=' '++' '--' '>>' */
1192 /* One-character tokens. */
1195 %token '{' '}' '(' ')' '[' ']' ',' ';' NEWLINE
1196 %token '+' '-' '*' '%' '^' '!' '>' '<' '|' '?' ':' '~' '$' '='
1197 %start program
1200 %%
1201 program : item_list
1204 | actionless_item_list
1205 ;
1206 item_list : newline_opt
1209 | actionless_item_list item terminator
1210 | item_list item terminator
1211 | item_list action terminator
1212 ;
1213 actionless_item_list : item_list pattern terminator
1216 | actionless_item_list pattern terminator
1217 ;
1218 item : pattern action
1221 | Function NAME '(' param_list_opt ')'
1222 newline_opt action
1223 | Function FUNC_NAME '(' param_list_opt ')'
1224 newline_opt action
1225 ;
1226 param_list_opt : /* empty */
1229 | param_list
1230 ;
1231 param_list : NAME
1234 | param_list ',' NAME
1235 ;
1236 pattern : Begin
1239 | End
1240 | expr
1241 | expr ',' newline_opt expr
1242 ;
1243 action : '{' newline_opt '}'
1246 | '{' newline_opt terminated_statement_list '}'
1247 | '{' newline_opt unterminated_statement_list '}'
1248 ;
1249 terminator : terminator ';'
1252 | terminator NEWLINE
1253 | ';'
1254 | NEWLINE
1255 ;
1256 terminated_statement_list : terminated_statement
1259 | terminated_statement_list terminated_statement
1260 ;
1261 unterminated_statement_list : unterminated_statement
1264 | terminated_statement_list unterminated_statement
1265 ;
1266 terminated_statement : action newline_opt
1269 | If '(' expr ')' newline_opt terminated_statement
1270 | If '(' expr ')' newline_opt terminated_statement
1271 Else newline_opt terminated_statement
1272 | While '(' expr ')' newline_opt terminated_statement
1273 | For '(' simple_statement_opt ';'
1274 expr_opt ';' simple_statement_opt ')' newline_opt
1275 terminated_statement
1276 | For '(' NAME In NAME ')' newline_opt
1277 terminated_statement
1278 | ';' newline_opt
1279 | terminatable_statement NEWLINE newline_opt
1280 | terminatable_statement ';' newline_opt
1281 ;
1282 unterminated_statement : terminatable_statement
1285 | If '(' expr ')' newline_opt unterminated_statement
1286 | If '(' expr ')' newline_opt terminated_statement
1287 Else newline_opt unterminated_statement
1288 | While '(' expr ')' newline_opt unterminated_statement
1289 | For '(' simple_statement_opt ';'
1290 expr_opt ';' simple_statement_opt ')' newline_opt
1291 unterminated_statement
1292 | For '(' NAME In NAME ')' newline_opt
1293 unterminated_statement
1294 ;
1295 terminatable_statement : simple_statement
1298 | Break
1299 | Continue
1300 | Next
1301 | Exit expr_opt
1302 | Return expr_opt
1303 | Do newline_opt terminated_statement While '(' expr ')'
1304 ;
1305 simple_statement_opt : /* empty */
1308 | simple_statement
1309 ;
1310 simple_statement : Delete NAME '[' expr_list ']'
1313 | expr
1314 | print_statement
1315 ;
1316 print_statement : simple_print_statement
1319 | simple_print_statement output_redirection
1320 ;
1321 simple_print_statement : Print print_expr_list_opt
1324 | Print '(' multiple_expr_list ')'
1325 | Printf print_expr_list
1326 | Printf '(' multiple_expr_list ')'
1327 ;
1328 output_redirection : '>' expr
1331 | APPEND expr
1332 | '|' expr
1333 ;
1334 expr_list_opt : /* empty */
1337 | expr_list
1338 ;
1339 expr_list : expr
1342 | multiple_expr_list
1343 ;
1344 multiple_expr_list : expr ',' newline_opt expr
1347 | multiple_expr_list ',' newline_opt expr
1348 ;
1349 expr_opt : /* empty */
1352 | expr
1353 ;
1354 expr : unary_expr
1357 | non_unary_expr
1358 ;
1359 unary_expr : '+' expr
1362 | '-' expr
1363 | unary_expr '^' expr
1364 | unary_expr '*' expr
1365 | unary_expr '/' expr
1366 | unary_expr '%' expr
1367 | unary_expr '+' expr
1368 | unary_expr '-' expr
1369 | unary_expr non_unary_expr
1370 | unary_expr '<' expr
1371 | unary_expr LE expr
1372 | unary_expr NE expr
1373 | unary_expr EQ expr
1374 | unary_expr '>' expr
1375 | unary_expr GE expr
1376 | unary_expr '~' expr
1377 | unary_expr NO_MATCH expr
1378 | unary_expr In NAME
1379 | unary_expr AND newline_opt expr
1380 | unary_expr OR newline_opt expr
1381 | unary_expr '?' expr ':' expr
1382 | unary_input_function
1383 ;
1384 non_unary_expr : '(' expr ')'
1387 | '!' expr
1388 | non_unary_expr '^' expr
1389 | non_unary_expr '*' expr
1390 | non_unary_expr '/' expr
1391 | non_unary_expr '%' expr
1392 | non_unary_expr '+' expr
1393 | non_unary_expr '-' expr
1394 | non_unary_expr non_unary_expr
1395 | non_unary_expr '<' expr
1396 | non_unary_expr LE expr
1397 | non_unary_expr NE expr
1398 | non_unary_expr EQ expr
1399 | non_unary_expr '>' expr
1400 | non_unary_expr GE expr
1401 | non_unary_expr '~' expr
1402 | non_unary_expr NO_MATCH expr
1403 | non_unary_expr In NAME
1404 | '(' multiple_expr_list ')' In NAME
1405 | non_unary_expr AND newline_opt expr
1406 | non_unary_expr OR newline_opt expr
1407 | non_unary_expr '?' expr ':' expr
1408 | NUMBER
1409 | STRING
1410 | lvalue
1411 | ERE
1412 | lvalue INCR
1413 | lvalue DECR
1414 | INCR lvalue
1415 | DECR lvalue
1416 | lvalue POW_ASSIGN expr
1417 | lvalue MOD_ASSIGN expr
1418 | lvalue MUL_ASSIGN expr
1419 | lvalue DIV_ASSIGN expr
1420 | lvalue ADD_ASSIGN expr
1421 | lvalue SUB_ASSIGN expr
1422 | lvalue '=' expr
1423 | FUNC_NAME '(' expr_list_opt ')'
1424 /* no white space allowed before '(' */
1425 | BUILTIN_FUNC_NAME '(' expr_list_opt ')'
1426 | BUILTIN_FUNC_NAME
1427 | non_unary_input_function
1428 ;
1429 print_expr_list_opt : /* empty */
1432 | print_expr_list
1433 ;
1434 print_expr_list : print_expr
1437 | print_expr_list ',' newline_opt print_expr
1438 ;
1439 print_expr : unary_print_expr
1442 | non_unary_print_expr
1443 ;
1444 unary_print_expr : '+' print_expr
1447 | '-' print_expr
1448 | unary_print_expr '^' print_expr
1449 | unary_print_expr '*' print_expr
1450 | unary_print_expr '/' print_expr
1451 | unary_print_expr '%' print_expr
1452 | unary_print_expr '+' print_expr
1453 | unary_print_expr '-' print_expr
1454 | unary_print_expr non_unary_print_expr
1455 | unary_print_expr '~' print_expr
1456 | unary_print_expr NO_MATCH print_expr
1457 | unary_print_expr In NAME
1458 | unary_print_expr AND newline_opt print_expr
1459 | unary_print_expr OR newline_opt print_expr
1460 | unary_print_expr '?' print_expr ':' print_expr
1461 ;
1462 non_unary_print_expr : '(' expr ')'
1465 | '!' print_expr
1466 | non_unary_print_expr '^' print_expr
1467 | non_unary_print_expr '*' print_expr
1468 | non_unary_print_expr '/' print_expr
1469 | non_unary_print_expr '%' print_expr
1470 | non_unary_print_expr '+' print_expr
1471 | non_unary_print_expr '-' print_expr
1472 | non_unary_print_expr non_unary_print_expr
1473 | non_unary_print_expr '~' print_expr
1474 | non_unary_print_expr NO_MATCH print_expr
1475 | non_unary_print_expr In NAME
1476 | '(' multiple_expr_list ')' In NAME
1477 | non_unary_print_expr AND newline_opt print_expr
1478 | non_unary_print_expr OR newline_opt print_expr
1479 | non_unary_print_expr '?' print_expr ':' print_expr
1480 | NUMBER
1481 | STRING
1482 | lvalue
1483 | ERE
1484 | lvalue INCR
1485 | lvalue DECR
1486 | INCR lvalue
1487 | DECR lvalue
1488 | lvalue POW_ASSIGN print_expr
1489 | lvalue MOD_ASSIGN print_expr
1490 | lvalue MUL_ASSIGN print_expr
1491 | lvalue DIV_ASSIGN print_expr
1492 | lvalue ADD_ASSIGN print_expr
1493 | lvalue SUB_ASSIGN print_expr
1494 | lvalue '=' print_expr
1495 | FUNC_NAME '(' expr_list_opt ')'
1496 /* no white space allowed before '(' */
1497 | BUILTIN_FUNC_NAME '(' expr_list_opt ')'
1498 | BUILTIN_FUNC_NAME
1499 ;
1500 lvalue : NAME
1503 | NAME '[' expr_list ']'
1504 | '$' expr
1505 ;
1506 non_unary_input_function : simple_get
1509 | simple_get '<' expr
1510 | non_unary_expr '|' simple_get
1511 ;
1512 unary_input_function : unary_expr '|' simple_get
1515 ;
1516 simple_get : GETLINE
1519 | GETLINE lvalue
1520 ;
1521 newline_opt : /* empty */
1524 | newline_opt NEWLINE
1525 ;
1526 This grammar has several ambiguities that shall be resolved as follows:
1528 * Operator precedence and associativity shall be as described in
1530 Expressions in Decreasing Precedence in awk .
1531 * In case of ambiguity, an else shall be associated with the most
1533 immediately preceding if that would satisfy the grammar.
1534 * In some contexts, a slash ( '/' ) that is used to surround an ERE
1536 could also be the division operator. This shall be resolved in such
1537 a way that wherever the division operator could appear, a slash is
1538 assumed to be the division operator. (There is no unary division
1539 operator.)
1540 One convention that might not be obvious from the formal grammar is
1542 where <newline>s are acceptable. There are several obvious placements
1543 such as terminating a statement, and a backslash can be used to escape
1544 <newline>s between any lexical tokens. In addition, <newline>s without
1545 backslashes can follow a comma, an open brace, logical AND operator (
1546 "&&" ), logical OR operator ( "||" ), the do keyword, the else keyword,
1547 and the closing parenthesis of an if, for, or while statement. For
1548 example:
1549 { print $1,
1552 $2 }
1553 Lexical Conventions
1555 The lexical conventions for awk programs, with respect to the preceding
1556 grammar, shall be as follows:
1557 1. Except as noted, awk shall recognize the longest possible token or
1559 delimiter beginning at a given point.
1560 2. A comment shall consist of any characters beginning with the number
1562 sign character and terminated by, but excluding the next occurrence
1563 of, a <newline>. Comments shall have no effect, except to delimit
1564 lexical tokens.
1565 3. The <newline> shall be recognized as the token NEWLINE.
1567 4. A backslash character immediately followed by a <newline> shall
1569 have no effect.
1570 5. The token STRING shall represent a string constant. A string con‐
1572 stant shall begin with the character ' .' Within a string constant,
1573 a backslash character shall be considered to begin an escape
1574 sequence as specified in the table in the Base Definitions volume
1575 of IEEE Std 1003.1-2001, Chapter 5, File Format Notation ( '\\' ,
1576 '\a' , '\b' , '\f' , '\n' , '\r' , '\t' , '\v' ). In addition, the
1577 escape sequences in Expressions in Decreasing Precedence in awk
1578 shall be recognized. A <newline> shall not occur within a string
1579 constant. A string constant shall be terminated by the first
1580 unescaped occurrence of the character '' after the one that begins
1581 the string constant. The value of the string shall be the sequence
1582 of all unescaped characters and values of escape sequences between,
1583 but not including, the two delimiting '' characters.
1584 6. The token ERE represents an extended regular expression constant.
1586 An ERE constant shall begin with the slash character. Within an
1587 ERE constant, a backslash character shall be considered to begin an
1588 escape sequence as specified in the table in the Base Definitions
1589 volume of IEEE Std 1003.1-2001, Chapter 5, File Format Notation. In
1590 addition, the escape sequences in Expressions in Decreasing Prece‐
1591 dence in awk shall be recognized. The application shall ensure that
1592 a <newline> does not occur within an ERE constant. An ERE constant
1593 shall be terminated by the first unescaped occurrence of the slash
1594 character after the one that begins the ERE constant. The extended
1595 regular expression represented by the ERE constant shall be the
1596 sequence of all unescaped characters and values of escape sequences
1597 between, but not including, the two delimiting slash characters.
1598 7. A <blank> shall have no effect, except to delimit lexical tokens or
1600 within STRING or ERE tokens.
1601 8. The token NUMBER shall represent a numeric constant. Its form and
1603 numeric value shall be equivalent to either of the tokens floating-
1604 constant or integer-constant as specified by the ISO C standard,
1605 with the following exceptions:
1606 a. An integer constant cannot begin with 0x or include the hexa‐
1608 decimal digits 'a' , 'b' , 'c' , 'd' , 'e' , 'f' , 'A' , 'B' ,
1609 'C' , 'D' , 'E' , or 'F' .
1610 b. The value of an integer constant beginning with 0 shall be
1612 taken in decimal rather than octal.
1613 c. An integer constant cannot include a suffix ( 'u' , 'U' , 'l' ,
1615 or 'L' ).
1616 d. A floating constant cannot include a suffix ( 'f' , 'F' , 'l' ,
1618 or 'L' ).
1619 If the value is too large or too small to be representable (see Con‐
1621 cepts Derived from the ISO C Standard ), the behavior is undefined.
1622 9. A sequence of underscores, digits, and alphabetics from the porta‐
1624 ble character set (see the Base Definitions volume of
1625 IEEE Std 1003.1-2001, Section 6.1, Portable Character Set), begin‐
1626 ning with an underscore or alphabetic, shall be considered a word.
1627 10. The following words are keywords that shall be recognized as indi‐
1629 vidual tokens; the name of the token is the same as the keyword:
1630
1635 11. The following words are names of built-in functions and shall be
1638 recognized as the token BUILTIN_FUNC_NAME:
1639
1645 The above-listed keywords and names of built-in functions are consid‐
1648 ered reserved words.
1649 12. The token NAME shall consist of a word that is not a keyword or a
1651 name of a built-in function and is not followed immediately (with‐
1652 out any delimiters) by the '(' character.
1653 13. The token FUNC_NAME shall consist of a word that is not a keyword
1655 or a name of a built-in function, followed immediately (without any
1656 delimiters) by the '(' character. The '(' character shall not be
1657 included as part of the token.
1658 14. The following two-character sequences shall be recognized as the
1660 named tokens:
1661 Token Name Sequence Token Name Sequence
1663 ADD_ASSIGN += NO_MATCH !~
1664 SUB_ASSIGN -= EQ ==
1665 MUL_ASSIGN *= LE <=
1666 DIV_ASSIGN /= GE >=
1667 MOD_ASSIGN %= NE !=
1668 POW_ASSIGN ^= INCR ++
1669 OR || DECR --
1670 AND && APPEND >>
1671 15. The following single characters shall be recognized as tokens whose
1673 names are the character:
1674 <newline> { } ( ) [ ] , ; + - * % ^ ! > < | ? : ~ $ =
1677 There is a lexical ambiguity between the token ERE and the tokens '/'
1679 and DIV_ASSIGN. When an input sequence begins with a slash character in
1680 any syntactic context where the token '/' or DIV_ASSIGN could appear as
1681 the next token in a valid program, the longer of those two tokens that
1682 can be recognized shall be recognized. In any other syntactic context
1683 where the token ERE could appear as the next token in a valid program,
1684 the token ERE shall be recognized.
1685
1687 The following exit values shall be returned:
1688 0 All input files were processed successfully.
1690 >0 An error occurred.
1692 The exit status can be altered within the program by using an exit
1695 expression.
1696
1698 If any file operand is specified and the named file cannot be accessed,
1699 awk shall write a diagnostic message to standard error and terminate
1700 without any further action.
1701 If the program specified by either the program operand or a progfile
1703 operand is not a valid awk program (as specified in the EXTENDED
1704 DESCRIPTION section), the behavior is undefined.
1705 The following sections are informative.
1707
1709 The index, length, match, and substr functions should not be confused
1710 with similar functions in the ISO C standard; the awk versions deal
1711 with characters, while the ISO C standard deals with bytes.
1712 Because the concatenation operation is represented by adjacent expres‐
1714 sions rather than an explicit operator, it is often necessary to use
1715 parentheses to enforce the proper evaluation precedence.
1716
1718 The awk program specified in the command line is most easily specified
1719 within single-quotes (for example, programs commonly contain characters
1720 that are special to the shell, including double-quotes. In the cases
1721 where an awk program contains single-quote characters, it is usually
1722 easiest to specify most of the program as strings within single-quotes
1723 concatenated by the shell with quoted single-quote characters. For
1724 example:
1725 awk '/'\''/ { print "quote:", $0 }'
1728 prints all lines from the standard input containing a single-quote
1730 character, prefixed with quote:.
1731 The following are examples of simple awk programs:
1733 1. Write to the standard output all input lines for which field 3 is
1735 greater than 5:
1736 $3 > 5
1739 2. Write every tenth line:
1741 (NR % 10) == 0
1744 3. Write any line with a substring matching the regular expression:
1746 /(G|D)(2[0-9][[:alpha:]]*)/
1749 4. Print any line with a substring containing a 'G' or 'D' , followed
1751 by a sequence of digits and characters. This example uses charac‐
1752 ter classes digit and alpha to match language-independent digit and
1753 alphabetic characters respectively:
1754 /(G|D)([[:digit:][:alpha:]]*)/
1757 5. Write any line in which the second field matches the regular
1759 expression and the fourth field does not:
1760 $2 ~ /xyz/ && $4 !~ /xyz/
1763 6. Write any line in which the second field contains a backslash:
1765 $2 ~ /\\/
1768 7. Write any line in which the second field contains a backslash. Note
1770 that backslash escapes are interpreted twice; once in lexical pro‐
1771 cessing of the string and once in processing the regular expres‐
1772 sion:
1773 $2 ~ "\\\\"
1776 8. Write the second to the last and the last field in each line. Sepa‐
1778 rate the fields by a colon:
1779 {OFS=":";print $(NF-1), $NF}
1782 9. Write the line number and number of fields in each line. The three
1784 strings representing the line number, the colon, and the number of
1785 fields are concatenated and that string is written to standard out‐
1786 put:
1787 {print NR ":" NF}
1790 10. Write lines longer than 72 characters:
1792 length($0) > 72
1795 11. Write the first two fields in opposite order separated by OFS:
1797 { print $2, $1 }
1800 12. Same, with input fields separated by a comma or <space>s and
1802 <tab>s, or both:
1803 BEGIN { FS = ",[ \t]*|[ \t]+" }
1806 { print $2, $1 }
1807 13. Add up the first column, print sum, and average:
1809 {s += $1 }
1812 END {print "sum is ", s, " average is", s/NR}
1813 14. Write fields in reverse order, one per line (many lines out for
1815 each line in):
1816 { for (i = NF; i > 0; --i) print $i }
1819 15. Write all lines between occurrences of the strings start and stop:
1821 /start/, /stop/
1824 16. Write all lines whose first field is different from the previous
1826 one:
1827 $1 != prev { print; prev = $1 }
1830 17. Simulate echo:
1832 BEGIN {
1835 for (i = 1; i < ARGC; ++i)
1836 printf("%s%s", ARGV[i], i==ARGC-1?"\n":" ")
1837 }
1838 18. Write the path prefixes contained in the PATH environment variable,
1840 one per line:
1841 BEGIN {
1844 n = split (ENVIRON["PATH"], path, ":")
1845 for (i = 1; i <= n; ++i)
1846 print path[i]
1847 }
1848 19. If there is a file named input containing page headers of the form:
1850 Page #
1853 and a file named program that contains:
1855 /Page/ { $2 = n++; }
1858 { print }
1859 then the command line:
1861 awk -f program n=5 input
1864 prints the file input, filling in page numbers starting at 5.
1866
1868 This description is based on the new awk, "nawk", (see the referenced
1869 The AWK Programming Language), which introduced a number of new fea‐
1870 tures to the historical awk:
1871 1. New keywords: delete, do, function, return
1873 2. New built-in functions: atan2, close, cos, gsub, match, rand, sin,
1875 srand, sub, system
1876 3. New predefined variables: FNR, ARGC, ARGV, RSTART, RLENGTH, SUBSEP
1878 4. New expression operators: ?, :, ,, ^
1880 5. The FS variable and the third argument to split, now treated as
1882 extended regular expressions.
1883 6. The operator precedence, changed to more closely match the C lan‐
1885 guage. Two examples of code that operate differently are:
1886 while ( n /= 10 > 1) ...
1889 if (!"wk" ~ /bwk/) ...
1890 Several features have been added based on newer implementations of awk:
1892 * Multiple instances of -f progfile are permitted.
1894 * The new option -v assignment.
1896 * The new predefined variable ENVIRON.
1898 * New built-in functions toupper and tolower.
1900 * More formatting capabilities are added to printf to match the ISO C
1902 standard.
1903 The overall awk syntax has always been based on the C language, with a
1905 few features from the shell command language and other sources. Because
1906 of this, it is not completely compatible with any other language, which
1907 has caused confusion for some users. It is not the intent of the stan‐
1908 dard developers to address such issues. A few relatively minor changes
1909 toward making the language more compatible with the ISO C standard were
1910 made; most of these changes are based on similar changes in recent
1911 implementations, as described above. There remain several C-language
1912 conventions that are not in awk. One of the notable ones is the comma
1913 operator, which is commonly used to specify multiple expressions in the
1914 C language for statement. Also, there are various places where awk is
1915 more restrictive than the C language regarding the type of expression
1916 that can be used in a given context. These limitations are due to the
1917 different features that the awk language does provide.
1918 Regular expressions in awk have been extended somewhat from historical
1920 implementations to make them a pure superset of extended regular
1921 expressions, as defined by IEEE Std 1003.1-2001 (see the Base Defini‐
1922 tions volume of IEEE Std 1003.1-2001, Section 9.4, Extended Regular
1923 Expressions). The main extensions are internationalization features
1924 and interval expressions. Historical implementations of awk have long
1925 supported backslash escape sequences as an extension to extended regu‐
1926 lar expressions, and this extension has been retained despite inconsis‐
1927 tency with other utilities. The number of escape sequences recognized
1928 in both extended regular expressions and strings has varied (generally
1929 increasing with time) among implementations. The set specified by
1930 IEEE Std 1003.1-2001 includes most sequences known to be supported by
1931 popular implementations and by the ISO C standard. One sequence that is
1932 not supported is hexadecimal value escapes beginning with '\x' . This
1933 would allow values expressed in more than 9 bits to be used within awk
1934 as in the ISO C standard. However, because this syntax has a non-deter‐
1935 ministic length, it does not permit the subsequent character to be a
1936 hexadecimal digit. This limitation can be dealt with in the C language
1937 by the use of lexical string concatenation. In the awk language, con‐
1938 catenation could also be a solution for strings, but not for extended
1939 regular expressions (either lexical ERE tokens or strings used dynami‐
1940 cally as regular expressions). Because of this limitation, the feature
1941 has not been added to IEEE Std 1003.1-2001.
1942 When a string variable is used in a context where an extended regular
1944 expression normally appears (where the lexical token ERE is used in the
1945 grammar) the string does not contain the literal slashes.
1946 Some versions of awk allow the form:
1948 func name(args, ... ) { statements }
1951 This has been deprecated by the authors of the language, who asked that
1953 it not be specified.
1954 Historical implementations of awk produce an error if a next statement
1956 is executed in a BEGIN action, and cause awk to terminate if a next
1957 statement is executed in an END action. This behavior has not been doc‐
1958 umented, and it was not believed that it was necessary to standardize
1959 it.
1960 The specification of conversions between string and numeric values is
1962 much more detailed than in the documentation of historical implementa‐
1963 tions or in the referenced The AWK Programming Language. Although most
1964 of the behavior is designed to be intuitive, the details are necessary
1965 to ensure compatible behavior from different implementations. This is
1966 especially important in relational expressions since the types of the
1967 operands determine whether a string or numeric comparison is performed.
1968 From the perspective of an application writer, it is usually sufficient
1969 to expect intuitive behavior and to force conversions (by adding zero
1970 or concatenating a null string) when the type of an expression does not
1971 obviously match what is needed. The intent has been to specify histori‐
1972 cal practice in almost all cases. The one exception is that, in histor‐
1973 ical implementations, variables and constants maintain both string and
1974 numeric values after their original value is converted by any use. This
1975 means that referencing a variable or constant can have unexpected side
1976 effects. For example, with historical implementations the following
1977 program:
1978 {
1981 a = "+2"
1982 b = 2
1983 if (NR % 2)
1984 c = a + b
1985 if (a == b)
1986 print "numeric comparison"
1987 else
1988 print "string comparison"
1989 }
1990 would perform a numeric comparison (and output numeric comparison) for
1992 each odd-numbered line, but perform a string comparison (and output
1993 string comparison) for each even-numbered line. IEEE Std 1003.1-2001
1994 ensures that comparisons will be numeric if necessary. With historical
1995 implementations, the following program:
1996 BEGIN {
1999 OFMT = "%e"
2000 print 3.14
2001 OFMT = "%f"
2002 print 3.14
2003 }
2004 would output "3.140000e+00" twice, because in the second print state‐
2006 ment the constant "3.14" would have a string value from the previous
2007 conversion. IEEE Std 1003.1-2001 requires that the output of the second
2008 print statement be "3.140000" . The behavior of historical implementa‐
2009 tions was seen as too unintuitive and unpredictable.
2010 It was pointed out that with the rules contained in early drafts, the
2012 following script would print nothing:
2013 BEGIN {
2016 y[1.5] = 1
2017 OFMT = "%e"
2018 print y[1.5]
2019 }
2020 Therefore, a new variable, CONVFMT, was introduced. The OFMT variable
2022 is now restricted to affecting output conversions of numbers to strings
2023 and CONVFMT is used for internal conversions, such as comparisons or
2024 array indexing. The default value is the same as that for OFMT, so
2025 unless a program changes CONVFMT (which no historical program would
2026 do), it will receive the historical behavior associated with internal
2027 string conversions.
2028 The POSIX awk lexical and syntactic conventions are specified more for‐
2030 mally than in other sources. Again the intent has been to specify his‐
2031 torical practice. One convention that may not be obvious from the for‐
2032 mal grammar as in other verbal descriptions is where <newline>s are
2033 acceptable. There are several obvious placements such as terminating a
2034 statement, and a backslash can be used to escape <newline>s between any
2035 lexical tokens. In addition, <newline>s without backslashes can follow
2036 a comma, an open brace, a logical AND operator ( "&&" ), a logical OR
2037 operator ( "||" ), the do keyword, the else keyword, and the closing
2038 parenthesis of an if, for, or while statement. For example:
2039 { print $1,
2042 $2 }
2043 The requirement that awk add a trailing <newline> to the program argu‐
2045 ment text is to simplify the grammar, making it match a text file in
2046 form. There is no way for an application or test suite to determine
2047 whether a literal <newline> is added or whether awk simply acts as if
2048 it did.
2049 IEEE Std 1003.1-2001 requires several changes from historical implemen‐
2051 tations in order to support internationalization. Probably the most
2052 subtle of these is the use of the decimal-point character, defined by
2053 the LC_NUMERIC category of the locale, in representations of floating-
2054 point numbers. This locale-specific character is used in recognizing
2055 numeric input, in converting between strings and numeric values, and in
2056 formatting output. However, regardless of locale, the period character
2057 (the decimal-point character of the POSIX locale) is the decimal-point
2058 character recognized in processing awk programs (including assignments
2059 in command line arguments). This is essentially the same convention as
2060 the one used in the ISO C standard. The difference is that the C lan‐
2061 guage includes the setlocale() function, which permits an application
2062 to modify its locale. Because of this capability, a C application
2063 begins executing with its locale set to the C locale, and only executes
2064 in the environment-specified locale after an explicit call to setlo‐
2065 cale(). However, adding such an elaborate new feature to the awk lan‐
2066 guage was seen as inappropriate for IEEE Std 1003.1-2001. It is possi‐
2067 ble to execute an awk program explicitly in any desired locale by set‐
2068 ting the environment in the shell.
2069 The undefined behavior resulting from NULs in extended regular expres‐
2071 sions allows future extensions for the GNU gawk program to process
2072 binary data.
2073 The behavior in the case of invalid awk programs (including lexical,
2075 syntactic, and semantic errors) is undefined because it was considered
2076 overly limiting on implementations to specify. In most cases such
2077 errors can be expected to produce a diagnostic and a non-zero exit sta‐
2078 tus. However, some implementations may choose to extend the language in
2079 ways that make use of certain invalid constructs. Other invalid con‐
2080 structs might be deemed worthy of a warning, but otherwise cause some
2081 reasonable behavior. Still other constructs may be very difficult to
2082 detect in some implementations. Also, different implementations might
2083 detect a given error during an initial parsing of the program (before
2084 reading any input files) while others might detect it when executing
2085 the program after reading some input. Implementors should be aware that
2086 diagnosing errors as early as possible and producing useful diagnostics
2087 can ease debugging of applications, and thus make an implementation
2088 more usable.
2089 The unspecified behavior from using multi-character RS values is to
2091 allow possible future extensions based on extended regular expressions
2092 used for record separators. Historical implementations take the first
2093 character of the string and ignore the others.
2094 Unspecified behavior when split( string, array, <null>) is used is to
2096 allow a proposed future extension that would split up a string into an
2097 array of individual characters.
2098 In the context of the getline function, equally good arguments for dif‐
2100 ferent precedences of the | and < operators can be made. Historical
2101 practice has been that:
2102 getline < "a" "b"
2105 is parsed as:
2107 ( getline < "a" ) "b"
2110 although many would argue that the intent was that the file ab should
2112 be read. However:
2113 getline < "x" + 1
2116 parses as:
2118 getline < ( "x" + 1 )
2121 Similar problems occur with the | version of getline, particularly in
2123 combination with $. For example:
2124 $"echo hi" | getline
2127 (This situation is particularly problematic when used in a print state‐
2129 ment, where the |getline part might be a redirection of the print.)
2130 Since in most cases such constructs are not (or at least should not) be
2132 used (because they have a natural ambiguity for which there is no con‐
2133 ventional parsing), the meaning of these constructs has been made
2134 explicitly unspecified. (The effect is that a conforming application
2135 that runs into the problem must parenthesize to resolve the ambiguity.)
2136 There appeared to be few if any actual uses of such constructs.
2137 Grammars can be written that would cause an error under these circum‐
2139 stances. Where backwards-compatibility is not a large consideration,
2140 implementors may wish to use such grammars.
2141 Some historical implementations have allowed some built-in functions to
2143 be called without an argument list, the result being a default argument
2144 list chosen in some "reasonable" way. Use of length as a synonym for
2145 length($0) is the only one of these forms that is thought to be widely
2146 known or widely used; this particular form is documented in various
2147 places (for example, most historical awk reference pages, although not
2148 in the referenced The AWK Programming Language) as legitimate practice.
2149 With this exception, default argument lists have always been undocu‐
2150 mented and vaguely defined, and it is not at all clear how (or if) they
2151 should be generalized to user-defined functions. They add no useful
2152 functionality and preclude possible future extensions that might need
2153 to name functions without calling them. Not standardizing them seems
2154 the simplest course. The standard developers considered that length
2155 merited special treatment, however, since it has been documented in the
2156 past and sees possibly substantial use in historical programs. Accord‐
2157 ingly, this usage has been made legitimate, but Issue 5 removed the
2158 obsolescent marking for XSI-conforming implementations and many other‐
2159 wise conforming applications depend on this feature.
2160 In sub and gsub, if repl is a string literal (the lexical token
2162 STRING), then two consecutive backslash characters should be used in
2163 the string to ensure a single backslash will precede the ampersand when
2164 the resultant string is passed to the function. (For example, to spec‐
2165 ify one literal ampersand in the replacement string, use gsub( ERE,
2166 "\\&" ).)
2167 Historically the only special character in the repl argument of sub and
2169 gsub string functions was the ampersand ( '&' ) character and preceding
2170 it with the backslash character was used to turn off its special mean‐
2171 ing.
2172 The description in the ISO POSIX-2:1993 standard introduced behavior
2174 such that the backslash character was another special character and it
2175 was unspecified whether there were any other special characters. This
2176 description introduced several portability problems, some of which are
2177 described below, and so it has been replaced with the more historical
2178 description. Some of the problems include:
2179 * Historically, to create the replacement string, a script could use
2181 gsub( ERE, "\\&" ), but with the ISO POSIX-2:1993 standard wording,
2182 it was necessary to use gsub( ERE, "\\\\&" ). Backslash characters
2183 are doubled here because all string literals are subject to lexical
2184 analysis, which would reduce each pair of backslash characters to a
2185 single backslash before being passed to gsub.
2186 * Since it was unspecified what the special characters were, for por‐
2188 table scripts to guarantee that characters are printed literally,
2189 each character had to be preceded with a backslash. (For example, a
2190 portable script had to use gsub( ERE, "\\h\\i" ) to produce a
2191 replacement string of "hi" .)
2192 The description for comparisons in the ISO POSIX-2:1993 standard did
2194 not properly describe historical practice because of the way numeric
2195 strings are compared as numbers. The current rules cause the following
2196 code:
2197 if (0 == "000")
2200 print "strange, but true"
2201 else
2202 print "not true"
2203 to do a numeric comparison, causing the if to succeed. It should be
2205 intuitively obvious that this is incorrect behavior, and indeed, no
2206 historical implementation of awk actually behaves this way.
2207 To fix this problem, the definition of numeric string was enhanced to
2209 include only those values obtained from specific circumstances (mostly
2210 external sources) where it is not possible to determine unambiguously
2211 whether the value is intended to be a string or a numeric.
2212 Variables that are assigned to a numeric string shall also be treated
2214 as a numeric string. (For example, the notion of a numeric string can
2215 be propagated across assignments.) In comparisons, all variables having
2216 the uninitialized value are to be treated as a numeric operand evaluat‐
2217 ing to the numeric value zero.
2218 Uninitialized variables include all types of variables including
2220 scalars, array elements, and fields. The definition of an uninitialized
2221 value in Variables and Special Variables is necessary to describe the
2222 value placed on uninitialized variables and on fields that are valid
2223 (for example, < $NF) but have no characters in them and to describe how
2224 these variables are to be used in comparisons. A valid field, such as
2225 $1, that has no characters in it can be obtained from an input line of
2226 "\t\t" when FS= '\t' . Historically, the comparison ( $1<10) was done
2227 numerically after evaluating $1 to the value zero.
2228 The phrase "... also shall have the numeric value of the numeric
2230 string" was removed from several sections of the ISO POSIX-2:1993 stan‐
2231 dard because is specifies an unnecessary implementation detail. It is
2232 not necessary for IEEE Std 1003.1-2001 to specify that these objects be
2233 assigned two different values. It is only necessary to specify that
2234 these objects may evaluate to two different values depending on con‐
2235 text.
2236 The description of numeric string processing is based on the behavior
2238 of the atof() function in the ISO C standard. While it is not a
2239 requirement for an implementation to use this function, many historical
2240 implementations of awk do. In the ISO C standard, floating-point con‐
2241 stants use a period as a decimal point character for the language
2242 itself, independent of the current locale, but the atof() function and
2243 the associated strtod() function use the decimal point character of the
2244 current locale when converting strings to numeric values. Similarly in
2245 awk, floating-point constants in an awk script use a period independent
2246 of the locale, but input strings use the decimal point character of the
2247 locale.
2248
2250 None.
2251
2253 Grammar Conventions , grep , lex , sed , the System Interfaces volume
2254 of IEEE Std 1003.1-2001, atof(), exec, popen(), setlocale(), strtod()
2255
2257 Portions of this text are reprinted and reproduced in electronic form
2258 from IEEE Std 1003.1, 2003 Edition, Standard for Information Technology
2259 -- Portable Operating System Interface (POSIX), The Open Group Base
2260 Specifications Issue 6, Copyright (C) 2001-2003 by the Institute of
2261 Electrical and Electronics Engineers, Inc and The Open Group. In the
2262 event of any discrepancy between this version and the original IEEE and
2263 The Open Group Standard, the original IEEE and The Open Group Standard
2264 is the referee document. The original Standard can be obtained online
2265 at http://www.opengroup.org/unix/online.html .
2266IEEE/The Open Group 2003 AWK(P)