1PIPE(7) Linux Programmer's Manual PIPE(7)
2
6 pipe - overview of pipes and FIFOs
7
9 Pipes and FIFOs (also known as named pipes) provide a unidirectional
10 interprocess communication channel. A pipe has a read end and a write
11 end. Data written to the write end of a pipe can be read from the read
12 end of the pipe.
13 A pipe is created using pipe(2), which creates a new pipe and returns
15 two file descriptors, one referring to the read end of the pipe, the
16 other referring to the write end. Pipes can be used to create a commu‐
17 nication channel between related processes; see pipe(2) for an example.
18 A FIFO (short for First In First Out) has a name within the filesystem
20 (created using mkfifo(3)), and is opened using open(2). Any process
21 may open a FIFO, assuming the file permissions allow it. The read end
22 is opened using the O_RDONLY flag; the write end is opened using the
23 O_WRONLY flag. See fifo(7) for further details. Note: although FIFOs
24 have a pathname in the filesystem, I/O on FIFOs does not involve opera‐
25 tions on the underlying device (if there is one).
26 I/O on pipes and FIFOs
28 The only difference between pipes and FIFOs is the manner in which they
29 are created and opened. Once these tasks have been accomplished, I/O
30 on pipes and FIFOs has exactly the same semantics.
31 If a process attempts to read from an empty pipe, then read(2) will
33 block until data is available. If a process attempts to write to a
34 full pipe (see below), then write(2) blocks until sufficient data has
35 been read from the pipe to allow the write to complete. Nonblocking
36 I/O is possible by using the fcntl(2) F_SETFL operation to enable the
37 O_NONBLOCK open file status flag.
38 The communication channel provided by a pipe is a byte stream: there is
40 no concept of message boundaries.
41 If all file descriptors referring to the write end of a pipe have been
43 closed, then an attempt to read(2) from the pipe will see end-of-file
44 (read(2) will return 0). If all file descriptors referring to the read
45 end of a pipe have been closed, then a write(2) will cause a SIGPIPE
46 signal to be generated for the calling process. If the calling process
47 is ignoring this signal, then write(2) fails with the error EPIPE. An
48 application that uses pipe(2) and fork(2) should use suitable close(2)
49 calls to close unnecessary duplicate file descriptors; this ensures
50 that end-of-file and SIGPIPE/EPIPE are delivered when appropriate.
51 It is not possible to apply lseek(2) to a pipe.
53 Pipe capacity
55 A pipe has a limited capacity. If the pipe is full, then a write(2)
56 will block or fail, depending on whether the O_NONBLOCK flag is set
57 (see below). Different implementations have different limits for the
58 pipe capacity. Applications should not rely on a particular capacity:
59 an application should be designed so that a reading process consumes
60 data as soon as it is available, so that a writing process does not re‐
61 main blocked.
62 In Linux versions before 2.6.11, the capacity of a pipe was the same as
64 the system page size (e.g., 4096 bytes on i386). Since Linux 2.6.11,
65 the pipe capacity is 16 pages (i.e., 65,536 bytes in a system with a
66 page size of 4096 bytes). Since Linux 2.6.35, the default pipe capac‐
67 ity is 16 pages, but the capacity can be queried and set using the fc‐
68 ntl(2) F_GETPIPE_SZ and F_SETPIPE_SZ operations. See fcntl(2) for more
69 information.
70 The following ioctl(2) operation, which can be applied to a file de‐
72 scriptor that refers to either end of a pipe, places a count of the
73 number of unread bytes in the pipe in the int buffer pointed to by the
74 final argument of the call:
75 ioctl(fd, FIONREAD, &nbytes);
77 The FIONREAD operation is not specified in any standard, but is pro‐
79 vided on many implementations.
80 /proc files
82 On Linux, the following files control how much memory can be used for
83 pipes:
84 /proc/sys/fs/pipe-max-pages (only in Linux 2.6.34)
86 An upper limit, in pages, on the capacity that an unprivileged
87 user (one without the CAP_SYS_RESOURCE capability) can set for a
88 pipe.
89 The default value for this limit is 16 times the default pipe
91 capacity (see above); the lower limit is two pages.
92 This interface was removed in Linux 2.6.35, in favor of
94 /proc/sys/fs/pipe-max-size.
95 /proc/sys/fs/pipe-max-size (since Linux 2.6.35)
97 The maximum size (in bytes) of individual pipes that can be set
98 by users without the CAP_SYS_RESOURCE capability. The value as‐
99 signed to this file may be rounded upward, to reflect the value
100 actually employed for a convenient implementation. To determine
101 the rounded-up value, display the contents of this file after
102 assigning a value to it.
103 The default value for this file is 1048576 (1 MiB). The minimum
105 value that can be assigned to this file is the system page size.
106 Attempts to set a limit less than the page size cause write(2)
107 to fail with the error EINVAL.
108 Since Linux 4.9, the value on this file also acts as a ceiling
110 on the default capacity of a new pipe or newly opened FIFO.
111 /proc/sys/fs/pipe-user-pages-hard (since Linux 4.5)
113 The hard limit on the total size (in pages) of all pipes created
114 or set by a single unprivileged user (i.e., one with neither the
115 CAP_SYS_RESOURCE nor the CAP_SYS_ADMIN capability). So long as
116 the total number of pages allocated to pipe buffers for this
117 user is at this limit, attempts to create new pipes will be de‐
118 nied, and attempts to increase a pipe's capacity will be denied.
119 When the value of this limit is zero (which is the default), no
121 hard limit is applied.
122 /proc/sys/fs/pipe-user-pages-soft (since Linux 4.5)
124 The soft limit on the total size (in pages) of all pipes created
125 or set by a single unprivileged user (i.e., one with neither the
126 CAP_SYS_RESOURCE nor the CAP_SYS_ADMIN capability). So long as
127 the total number of pages allocated to pipe buffers for this
128 user is at this limit, individual pipes created by a user will
129 be limited to one page, and attempts to increase a pipe's capac‐
130 ity will be denied.
131 When the value of this limit is zero, no soft limit is applied.
133 The default value for this file is 16384, which permits creating
134 up to 1024 pipes with the default capacity.
135 Before Linux 4.9, some bugs affected the handling of the
137 pipe-user-pages-soft and pipe-user-pages-hard limits; see BUGS.
138 PIPE_BUF
140 POSIX.1 says that writes of less than PIPE_BUF bytes must be atomic:
141 the output data is written to the pipe as a contiguous sequence.
142 Writes of more than PIPE_BUF bytes may be nonatomic: the kernel may in‐
143 terleave the data with data written by other processes. POSIX.1 re‐
144 quires PIPE_BUF to be at least 512 bytes. (On Linux, PIPE_BUF is 4096
145 bytes.) The precise semantics depend on whether the file descriptor is
146 nonblocking (O_NONBLOCK), whether there are multiple writers to the
147 pipe, and on n, the number of bytes to be written:
148 O_NONBLOCK disabled, n <= PIPE_BUF
150 All n bytes are written atomically; write(2) may block if there
151 is not room for n bytes to be written immediately
152 O_NONBLOCK enabled, n <= PIPE_BUF
154 If there is room to write n bytes to the pipe, then write(2)
155 succeeds immediately, writing all n bytes; otherwise write(2)
156 fails, with errno set to EAGAIN.
157 O_NONBLOCK disabled, n > PIPE_BUF
159 The write is nonatomic: the data given to write(2) may be inter‐
160 leaved with write(2)s by other process; the write(2) blocks un‐
161 til n bytes have been written.
162 O_NONBLOCK enabled, n > PIPE_BUF
164 If the pipe is full, then write(2) fails, with errno set to EA‐
165 GAIN. Otherwise, from 1 to n bytes may be written (i.e., a
166 "partial write" may occur; the caller should check the return
167 value from write(2) to see how many bytes were actually writ‐
168 ten), and these bytes may be interleaved with writes by other
169 processes.
170 Open file status flags
172 The only open file status flags that can be meaningfully applied to a
173 pipe or FIFO are O_NONBLOCK and O_ASYNC.
174 Setting the O_ASYNC flag for the read end of a pipe causes a signal
176 (SIGIO by default) to be generated when new input becomes available on
177 the pipe. The target for delivery of signals must be set using the fc‐
178 ntl(2) F_SETOWN command. On Linux, O_ASYNC is supported for pipes and
179 FIFOs only since kernel 2.6.
180 Portability notes
182 On some systems (but not Linux), pipes are bidirectional: data can be
183 transmitted in both directions between the pipe ends. POSIX.1 requires
184 only unidirectional pipes. Portable applications should avoid reliance
185 on bidirectional pipe semantics.
186 BUGS
188 Before Linux 4.9, some bugs affected the handling of the
189 pipe-user-pages-soft and pipe-user-pages-hard limits when using the fc‐
190 ntl(2) F_SETPIPE_SZ operation to change a pipe's capacity:
191 (1) When increasing the pipe capacity, the checks against the soft and
193 hard limits were made against existing consumption, and excluded
194 the memory required for the increased pipe capacity. The new in‐
195 crease in pipe capacity could then push the total memory used by
196 the user for pipes (possibly far) over a limit. (This could also
197 trigger the problem described next.)
198 Starting with Linux 4.9, the limit checking includes the memory
200 required for the new pipe capacity.
201 (2) The limit checks were performed even when the new pipe capacity
203 was less than the existing pipe capacity. This could lead to
204 problems if a user set a large pipe capacity, and then the limits
205 were lowered, with the result that the user could no longer de‐
206 crease the pipe capacity.
207 Starting with Linux 4.9, checks against the limits are performed
209 only when increasing a pipe's capacity; an unprivileged user can
210 always decrease a pipe's capacity.
211 (3) The accounting and checking against the limits were done as fol‐
213 lows:
214 (a) Test whether the user has exceeded the limit.
216 (b) Make the new pipe buffer allocation.
217 (c) Account new allocation against the limits.
218 This was racey. Multiple processes could pass point (a) simulta‐
220 neously, and then allocate pipe buffers that were accounted for
221 only in step (c), with the result that the user's pipe buffer al‐
222 location could be pushed over the limit.
223 Starting with Linux 4.9, the accounting step is performed before
225 doing the allocation, and the operation fails if the limit would
226 be exceeded.
227 Before Linux 4.9, bugs similar to points (1) and (3) could also occur
229 when the kernel allocated memory for a new pipe buffer; that is, when
230 calling pipe(2) and when opening a previously unopened FIFO.
231
233 mkfifo(1), dup(2), fcntl(2), open(2), pipe(2), poll(2), select(2),
234 socketpair(2), splice(2), stat(2), tee(2), vmsplice(2), mkfifo(3),
235 epoll(7), fifo(7)
236
238 This page is part of release 5.13 of the Linux man-pages project. A
239 description of the project, information about reporting bugs, and the
240 latest version of this page, can be found at
241 https://www.kernel.org/doc/man-pages/.
242Linux 2021-08-27 PIPE(7)