strtok(3)

1strtok(3)                  Library Functions Manual                  strtok(3)
2
3
4

NAME

6       strtok, strtok_r - extract tokens from strings
7

LIBRARY

9       Standard C library (libc, -lc)
10

SYNOPSIS

12       #include <string.h>
13
14       char *strtok(char *restrict str, const char *restrict delim);
15       char *strtok_r(char *restrict str, const char *restrict delim,
16                      char **restrict saveptr);
17
18   Feature Test Macro Requirements for glibc (see feature_test_macros(7)):
19
20       strtok_r():
21           _POSIX_C_SOURCE
22               || /* glibc <= 2.19: */ _BSD_SOURCE || _SVID_SOURCE
23

DESCRIPTION

25       The  strtok()  function breaks a string into a sequence of zero or more
26       nonempty tokens.  On the first call  to  strtok(),  the  string  to  be
27       parsed should be specified in str.  In each subsequent call that should
28       parse the same string, str must be NULL.
29
30       The delim argument specifies a set of bytes that delimit the tokens  in
31       the  parsed  string.  The caller may specify different strings in delim
32       in successive calls that parse the same string.
33
34       Each call to strtok() returns a pointer  to  a  null-terminated  string
35       containing the next token.  This string does not include the delimiting
36       byte.  If no more tokens are found, strtok() returns NULL.
37
38       A sequence of calls to strtok() that operate on the same  string  main‐
39       tains a pointer that determines the point from which to start searching
40       for the next token.  The first call to strtok() sets  this  pointer  to
41       point  to the first byte of the string.  The start of the next token is
42       determined by scanning forward for the next nondelimiter byte  in  str.
43       If  such  a  byte is found, it is taken as the start of the next token.
44       If no such byte is found, then there are no more tokens,  and  strtok()
45       returns NULL.  (A string that is empty or that contains only delimiters
46       will thus cause strtok() to return NULL on the first call.)
47
48       The end of each token is found by scanning  forward  until  either  the
49       next  delimiter byte is found or until the terminating null byte ('\0')
50       is encountered.  If a delimiter byte is found, it is overwritten with a
51       null  byte to terminate the current token, and strtok() saves a pointer
52       to the following byte; that pointer will be used as the starting  point
53       when  searching  for  the next token.  In this case, strtok() returns a
54       pointer to the start of the found token.
55
56       From the above description, it follows that a sequence of two  or  more
57       contiguous  delimiter  bytes in the parsed string is considered to be a
58       single delimiter, and that delimiter bytes at the start or end  of  the
59       string  are  ignored.  Put another way: the tokens returned by strtok()
60       are always nonempty strings.   Thus,  for  example,  given  the  string
61       "aaa;;bbb,",  successive  calls  to strtok() that specify the delimiter
62       string ";," would return the strings "aaa" and "bbb", and then  a  null
63       pointer.
64
65       The  strtok_r()  function  is  a  reentrant  version  of strtok().  The
66       saveptr argument is a pointer to a char * variable that is used  inter‐
67       nally  by  strtok_r()  in  order to maintain context between successive
68       calls that parse the same string.
69
70       On the first call to strtok_r(), str should point to the string  to  be
71       parsed,  and the value of *saveptr is ignored (but see NOTES).  In sub‐
72       sequent calls, str should be NULL, and saveptr (and the buffer that  it
73       points to) should be unchanged since the previous call.
74
75       Different  strings  may be parsed concurrently using sequences of calls
76       to strtok_r() that specify different saveptr arguments.
77

RETURN VALUE

79       The strtok() and strtok_r() functions return a pointer to the next  to‐
80       ken, or NULL if there are no more tokens.
81

ATTRIBUTES

83       For  an  explanation  of  the  terms  used  in  this  section,  see at‐
84       tributes(7).
85
86       ┌──────────────────────────────┬───────────────┬───────────────────────┐
87       │Interface                     │ Attribute     │ Value                 │
88       ├──────────────────────────────┼───────────────┼───────────────────────┤
89       │strtok()                      │ Thread safety │ MT-Unsafe race:strtok │
90       ├──────────────────────────────┼───────────────┼───────────────────────┤
91       │strtok_r()                    │ Thread safety │ MT-Safe               │
92       └──────────────────────────────┴───────────────┴───────────────────────┘
93

VERSIONS

95       On some implementations, *saveptr is required to be NULL on  the  first
96       call to strtok_r() that is being used to parse str.
97

STANDARDS

99       strtok()
100              C11, POSIX.1-2008.
101
102       strtok_r()
103              POSIX.1-2008.
104

HISTORY

106       strtok()
107              POSIX.1-2001, C89, SVr4, 4.3BSD.
108
109       strtok_r()
110              POSIX.1-2001.
111

BUGS

113       Be cautious when using these functions.  If you do use them, note that:
114
115       •  These functions modify their first argument.
116
117       •  These functions cannot be used on constant strings.
118
119       •  The identity of the delimiting byte is lost.
120
121       •  The  strtok()  function  uses a static buffer while parsing, so it's
122          not thread safe.  Use strtok_r() if this matters to you.
123

EXAMPLES

125       The program below uses nested loops that employ strtok_r() to  break  a
126       string  into  a  two-level hierarchy of tokens.  The first command-line
127       argument specifies the string to be parsed.  The second argument speci‐
128       fies the delimiter byte(s) to be used to separate that string into "ma‐
129       jor" tokens.  The third argument specifies the delimiter byte(s) to  be
130       used to separate the "major" tokens into subtokens.
131
132       An example of the output produced by this program is the following:
133
134           $ ./a.out 'a/bbb///cc;xxx:yyy:' ':;' '/'
135           1: a/bbb///cc
136                    --> a
137                    --> bbb
138                    --> cc
139           2: xxx
140                    --> xxx
141           3: yyy
142                    --> yyy
143
144   Program source
145
146       #include <stdio.h>
147       #include <stdlib.h>
148       #include <string.h>
149
150       int
151       main(int argc, char *argv[])
152       {
153           char *str1, *str2, *token, *subtoken;
154           char *saveptr1, *saveptr2;
155           int j;
156
157           if (argc != 4) {
158               fprintf(stderr, "Usage: %s string delim subdelim\n",
159                       argv[0]);
160               exit(EXIT_FAILURE);
161           }
162
163           for (j = 1, str1 = argv[1]; ; j++, str1 = NULL) {
164               token = strtok_r(str1, argv[2], &saveptr1);
165               if (token == NULL)
166                   break;
167               printf("%d: %s\n", j, token);
168
169               for (str2 = token; ; str2 = NULL) {
170                   subtoken = strtok_r(str2, argv[3], &saveptr2);
171                   if (subtoken == NULL)
172                       break;
173                   printf("\t --> %s\n", subtoken);
174               }
175           }
176
177           exit(EXIT_SUCCESS);
178       }
179
180       Another   example  program  using  strtok()  can  be  found  in  getad‐
181       drinfo_a(3).
182