1AIO(3) User Contributed Perl Documentation AIO(3)
2
6 IO::AIO - Asynchronous/Advanced Input/Output
7
9 use IO::AIO;
10 aio_open "/etc/passwd", IO::AIO::O_RDONLY, 0, sub {
12 my $fh = shift
13 or die "/etc/passwd: $!";
14 ...
15 };
16 aio_unlink "/tmp/file", sub { };
18 aio_read $fh, 30000, 1024, $buffer, 0, sub {
20 $_[0] > 0 or die "read error: $!";
21 };
22 # version 2+ has request and group objects
24 use IO::AIO 2;
25 aioreq_pri 4; # give next request a very high priority
27 my $req = aio_unlink "/tmp/file", sub { };
28 $req->cancel; # cancel request if still in queue
29 my $grp = aio_group sub { print "all stats done\n" };
31 add $grp aio_stat "..." for ...;
32
34 This module implements asynchronous I/O using whatever means your
35 operating system supports. It is implemented as an interface to
36 "libeio" (<http://software.schmorp.de/pkg/libeio.html>).
37 Asynchronous means that operations that can normally block your program
39 (e.g. reading from disk) will be done asynchronously: the operation
40 will still block, but you can do something else in the meantime. This
41 is extremely useful for programs that need to stay interactive even
42 when doing heavy I/O (GUI programs, high performance network servers
43 etc.), but can also be used to easily do operations in parallel that
44 are normally done sequentially, e.g. stat'ing many files, which is much
45 faster on a RAID volume or over NFS when you do a number of stat
46 operations concurrently.
47 While most of this works on all types of file descriptors (for example
49 sockets), using these functions on file descriptors that support
50 nonblocking operation (again, sockets, pipes etc.) is very inefficient.
51 Use an event loop for that (such as the EV module): IO::AIO will
52 naturally fit into such an event loop itself.
53 In this version, a number of threads are started that execute your
55 requests and signal their completion. You don't need thread support in
56 perl, and the threads created by this module will not be visible to
57 perl. In the future, this module might make use of the native aio
58 functions available on many operating systems. However, they are often
59 not well-supported or restricted (GNU/Linux doesn't allow them on
60 normal files currently, for example), and they would only support
61 aio_read and aio_write, so the remaining functionality would have to be
62 implemented using threads anyway.
63 In addition to asynchronous I/O, this module also exports some rather
65 arcane interfaces, such as "madvise" or linux's "splice" system call,
66 which is why the "A" in "AIO" can also mean advanced.
67 Although the module will work in the presence of other (Perl-) threads,
69 it is currently not reentrant in any way, so use appropriate locking
70 yourself, always call "poll_cb" from within the same thread, or never
71 call "poll_cb" (or other "aio_" functions) recursively.
72 EXAMPLE
74 This is a simple example that uses the EV module and loads /etc/passwd
75 asynchronously:
76 use EV;
78 use IO::AIO;
79 # register the IO::AIO callback with EV
81 my $aio_w = EV::io IO::AIO::poll_fileno, EV::READ, \&IO::AIO::poll_cb;
82 # queue the request to open /etc/passwd
84 aio_open "/etc/passwd", IO::AIO::O_RDONLY, 0, sub {
85 my $fh = shift
86 or die "error while opening: $!";
87 # stat'ing filehandles is generally non-blocking
89 my $size = -s $fh;
90 # queue a request to read the file
92 my $contents;
93 aio_read $fh, 0, $size, $contents, 0, sub {
94 $_[0] == $size
95 or die "short read: $!";
96 close $fh;
98 # file contents now in $contents
100 print $contents;
101 # exit event loop and program
103 EV::break;
104 };
105 };
106 # possibly queue up other requests, or open GUI windows,
108 # check for sockets etc. etc.
109 # process events as long as there are some:
111 EV::run;
112
114 Every "aio_*" function creates a request. which is a C data structure
115 not directly visible to Perl.
116 If called in non-void context, every request function returns a Perl
118 object representing the request. In void context, nothing is returned,
119 which saves a bit of memory.
120 The perl object is a fairly standard ref-to-hash object. The hash
122 contents are not used by IO::AIO so you are free to store anything you
123 like in it.
124 During their existance, aio requests travel through the following
126 states, in order:
127 ready
129 Immediately after a request is created it is put into the ready
130 state, waiting for a thread to execute it.
131 execute
133 A thread has accepted the request for processing and is currently
134 executing it (e.g. blocking in read).
135 pending
137 The request has been executed and is waiting for result processing.
138 While request submission and execution is fully asynchronous,
140 result processing is not and relies on the perl interpreter calling
141 "poll_cb" (or another function with the same effect).
142 result
144 The request results are processed synchronously by "poll_cb".
145 The "poll_cb" function will process all outstanding aio requests by
147 calling their callbacks, freeing memory associated with them and
148 managing any groups they are contained in.
149 done
151 Request has reached the end of its lifetime and holds no resources
152 anymore (except possibly for the Perl object, but its connection to
153 the actual aio request is severed and calling its methods will
154 either do nothing or result in a runtime error).
155
157 QUICK OVERVIEW
158 This section simply lists the prototypes most of the functions for
159 quick reference. See the following sections for function-by-function
160 documentation.
161 aio_wd $pathname, $callback->($wd)
163 aio_open $pathname, $flags, $mode, $callback->($fh)
164 aio_close $fh, $callback->($status)
165 aio_seek $fh,$offset,$whence, $callback->($offs)
166 aio_read $fh,$offset,$length, $data,$dataoffset, $callback->($retval)
167 aio_write $fh,$offset,$length, $data,$dataoffset, $callback->($retval)
168 aio_sendfile $out_fh, $in_fh, $in_offset, $length, $callback->($retval)
169 aio_readahead $fh,$offset,$length, $callback->($retval)
170 aio_stat $fh_or_path, $callback->($status)
171 aio_lstat $fh, $callback->($status)
172 aio_statvfs $fh_or_path, $callback->($statvfs)
173 aio_utime $fh_or_path, $atime, $mtime, $callback->($status)
174 aio_chown $fh_or_path, $uid, $gid, $callback->($status)
175 aio_chmod $fh_or_path, $mode, $callback->($status)
176 aio_truncate $fh_or_path, $offset, $callback->($status)
177 aio_allocate $fh, $mode, $offset, $len, $callback->($status)
178 aio_fiemap $fh, $start, $length, $flags, $count, $cb->(\@extents)
179 aio_unlink $pathname, $callback->($status)
180 aio_mknod $pathname, $mode, $dev, $callback->($status)
181 aio_link $srcpath, $dstpath, $callback->($status)
182 aio_symlink $srcpath, $dstpath, $callback->($status)
183 aio_readlink $pathname, $callback->($link)
184 aio_realpath $pathname, $callback->($path)
185 aio_rename $srcpath, $dstpath, $callback->($status)
186 aio_rename2 $srcpath, $dstpath, $flags, $callback->($status)
187 aio_mkdir $pathname, $mode, $callback->($status)
188 aio_rmdir $pathname, $callback->($status)
189 aio_readdir $pathname, $callback->($entries)
190 aio_readdirx $pathname, $flags, $callback->($entries, $flags)
191 IO::AIO::READDIR_DENTS IO::AIO::READDIR_DIRS_FIRST
192 IO::AIO::READDIR_STAT_ORDER IO::AIO::READDIR_FOUND_UNKNOWN
193 aio_scandir $pathname, $maxreq, $callback->($dirs, $nondirs)
194 aio_load $pathname, $data, $callback->($status)
195 aio_copy $srcpath, $dstpath, $callback->($status)
196 aio_move $srcpath, $dstpath, $callback->($status)
197 aio_rmtree $pathname, $callback->($status)
198 aio_fcntl $fh, $cmd, $arg, $callback->($status)
199 aio_ioctl $fh, $request, $buf, $callback->($status)
200 aio_sync $callback->($status)
201 aio_syncfs $fh, $callback->($status)
202 aio_fsync $fh, $callback->($status)
203 aio_fdatasync $fh, $callback->($status)
204 aio_sync_file_range $fh, $offset, $nbytes, $flags, $callback->($status)
205 aio_pathsync $pathname, $callback->($status)
206 aio_msync $scalar, $offset = 0, $length = undef, flags = MS_SYNC, $callback->($status)
207 aio_mtouch $scalar, $offset = 0, $length = undef, flags = 0, $callback->($status)
208 aio_mlock $scalar, $offset = 0, $length = undef, $callback->($status)
209 aio_mlockall $flags, $callback->($status)
210 aio_group $callback->(...)
211 aio_nop $callback->()
212 $prev_pri = aioreq_pri [$pri]
214 aioreq_nice $pri_adjust
215 IO::AIO::poll_wait
217 IO::AIO::poll_cb
218 IO::AIO::poll
219 IO::AIO::flush
220 IO::AIO::max_poll_reqs $nreqs
221 IO::AIO::max_poll_time $seconds
222 IO::AIO::min_parallel $nthreads
223 IO::AIO::max_parallel $nthreads
224 IO::AIO::max_idle $nthreads
225 IO::AIO::idle_timeout $seconds
226 IO::AIO::max_outstanding $maxreqs
227 IO::AIO::nreqs
228 IO::AIO::nready
229 IO::AIO::npending
230 IO::AIO::reinit
231 $nfd = IO::AIO::get_fdlimit
233 IO::AIO::min_fdlimit $nfd
234 IO::AIO::sendfile $ofh, $ifh, $offset, $count
236 IO::AIO::fadvise $fh, $offset, $len, $advice
237 IO::AIO::mmap $scalar, $length, $prot, $flags[, $fh[, $offset]]
239 IO::AIO::munmap $scalar
240 IO::AIO::mremap $scalar, $new_length, $flags[, $new_address]
241 IO::AIO::madvise $scalar, $offset, $length, $advice
242 IO::AIO::mprotect $scalar, $offset, $length, $protect
243 IO::AIO::munlock $scalar, $offset = 0, $length = undef
244 IO::AIO::munlockall
245 # stat extensions
247 $counter = IO::AIO::st_gen
248 $seconds = IO::AIO::st_atime, IO::AIO::st_mtime, IO::AIO::st_ctime, IO::AIO::st_btime
249 ($atime, $mtime, $ctime, $btime, ...) = IO::AIO::st_xtime
250 $nanoseconds = IO::AIO::st_atimensec, IO::AIO::st_mtimensec, IO::AIO::st_ctimensec, IO::AIO::st_btimensec
251 $seconds = IO::AIO::st_btimesec
252 ($atime, $mtime, $ctime, $btime, ...) = IO::AIO::st_xtimensec
253 # very much unportable syscalls
255 IO::AIO::accept4 $r_fh, $sockaddr, $sockaddr_len, $flags
256 IO::AIO::splice $r_fh, $r_off, $w_fh, $w_off, $length, $flags
257 IO::AIO::tee $r_fh, $w_fh, $length, $flags
258 $actual_size = IO::AIO::pipesize $r_fh[, $new_size]
259 ($rfh, $wfh) = IO::AIO::pipe2 [$flags]
260 $fh = IO::AIO::memfd_create $pathname[, $flags]
261 $fh = IO::AIO::eventfd [$initval, [$flags]]
262 $fh = IO::AIO::timerfd_create $clockid[, $flags]
263 ($cur_interval, $cur_value) = IO::AIO::timerfd_settime $fh, $flags, $new_interval, $nbw_value
264 ($cur_interval, $cur_value) = IO::AIO::timerfd_gettime $fh
265 API NOTES
267 All the "aio_*" calls are more or less thin wrappers around the syscall
268 with the same name (sans "aio_"). The arguments are similar or
269 identical, and they all accept an additional (and optional) $callback
270 argument which must be a code reference. This code reference will be
271 called after the syscall has been executed in an asynchronous fashion.
272 The results of the request will be passed as arguments to the callback
273 (and, if an error occured, in $!) - for most requests the syscall
274 return code (e.g. most syscalls return "-1" on error, unlike perl,
275 which usually delivers "false").
276 Some requests (such as "aio_readdir") pass the actual results and
278 communicate failures by passing "undef".
279 All functions expecting a filehandle keep a copy of the filehandle
281 internally until the request has finished.
282 All functions return request objects of type IO::AIO::REQ that allow
284 further manipulation of those requests while they are in-flight.
285 The pathnames you pass to these routines should be absolute. The reason
287 for this is that at the time the request is being executed, the current
288 working directory could have changed. Alternatively, you can make sure
289 that you never change the current working directory anywhere in the
290 program and then use relative paths. You can also take advantage of
291 IO::AIOs working directory abstraction, that lets you specify paths
292 relative to some previously-opened "working directory object" - see the
293 description of the "IO::AIO::WD" class later in this document.
294 To encode pathnames as octets, either make sure you either: a) always
296 pass in filenames you got from outside (command line, readdir etc.)
297 without tinkering, b) are in your native filesystem encoding, c) use
298 the Encode module and encode your pathnames to the locale (or other)
299 encoding in effect in the user environment, d) use
300 Glib::filename_from_unicode on unicode filenames or e) use something
301 else to ensure your scalar has the correct contents.
302 This works, btw. independent of the internal UTF-8 bit, which IO::AIO
304 handles correctly whether it is set or not.
305 AIO REQUEST FUNCTIONS
307 $prev_pri = aioreq_pri [$pri]
308 Returns the priority value that would be used for the next request
309 and, if $pri is given, sets the priority for the next aio request.
310 The default priority is 0, the minimum and maximum priorities are
312 "-4" and 4, respectively. Requests with higher priority will be
313 serviced first.
314 The priority will be reset to 0 after each call to one of the
316 "aio_*" functions.
317 Example: open a file with low priority, then read something from it
319 with higher priority so the read request is serviced before other
320 low priority open requests (potentially spamming the cache):
321 aioreq_pri -3;
323 aio_open ..., sub {
324 return unless $_[0];
325 aioreq_pri -2;
327 aio_read $_[0], ..., sub {
328 ...
329 };
330 };
331 aioreq_nice $pri_adjust
333 Similar to "aioreq_pri", but subtracts the given value from the
334 current priority, so the effect is cumulative.
335 aio_open $pathname, $flags, $mode, $callback->($fh)
337 Asynchronously open or create a file and call the callback with a
338 newly created filehandle for the file (or "undef" in case of an
339 error).
340 The pathname passed to "aio_open" must be absolute. See API NOTES,
342 above, for an explanation.
343 The $flags argument is a bitmask. See the "Fcntl" module for a
345 list. They are the same as used by "sysopen".
346 Likewise, $mode specifies the mode of the newly created file, if it
348 didn't exist and "O_CREAT" has been given, just like perl's
349 "sysopen", except that it is mandatory (i.e. use 0 if you don't
350 create new files, and 0666 or 0777 if you do). Note that the $mode
351 will be modified by the umask in effect then the request is being
352 executed, so better never change the umask.
353 Example:
355 aio_open "/etc/passwd", IO::AIO::O_RDONLY, 0, sub {
357 if ($_[0]) {
358 print "open successful, fh is $_[0]\n";
359 ...
360 } else {
361 die "open failed: $!\n";
362 }
363 };
364 In addition to all the common open modes/flags ("O_RDONLY",
366 "O_WRONLY", "O_RDWR", "O_CREAT", "O_TRUNC", "O_EXCL" and
367 "O_APPEND"), the following POSIX and non-POSIX constants are
368 available (missing ones on your system are, as usual, 0):
369 "O_ASYNC", "O_DIRECT", "O_NOATIME", "O_CLOEXEC", "O_NOCTTY",
371 "O_NOFOLLOW", "O_NONBLOCK", "O_EXEC", "O_SEARCH", "O_DIRECTORY",
372 "O_DSYNC", "O_RSYNC", "O_SYNC", "O_PATH", "O_TMPFILE", "O_TTY_INIT"
373 and "O_ACCMODE".
374 aio_close $fh, $callback->($status)
376 Asynchronously close a file and call the callback with the result
377 code.
378 Unfortunately, you can't do this to perl. Perl insists very
380 strongly on closing the file descriptor associated with the
381 filehandle itself.
382 Therefore, "aio_close" will not close the filehandle - instead it
384 will use dup2 to overwrite the file descriptor with the write-end
385 of a pipe (the pipe fd will be created on demand and will be
386 cached).
387 Or in other words: the file descriptor will be closed, but it will
389 not be free for reuse until the perl filehandle is closed.
390 aio_seek $fh, $offset, $whence, $callback->($offs)
392 Seeks the filehandle to the new $offset, similarly to perl's
393 "sysseek". The $whence can use the traditional values (0 for
394 "IO::AIO::SEEK_SET", 1 for "IO::AIO::SEEK_CUR" or 2 for
395 "IO::AIO::SEEK_END").
396 The resulting absolute offset will be passed to the callback, or
398 "-1" in case of an error.
399 In theory, the $whence constants could be different than the
401 corresponding values from Fcntl, but perl guarantees they are the
402 same, so don't panic.
403 As a GNU/Linux (and maybe Solaris) extension, also the constants
405 "IO::AIO::SEEK_DATA" and "IO::AIO::SEEK_HOLE" are available, if
406 they could be found. No guarantees about suitability for use in
407 "aio_seek" or Perl's "sysseek" can be made though, although I would
408 naively assume they "just work".
409 aio_read $fh,$offset,$length, $data,$dataoffset, $callback->($retval)
411 aio_write $fh,$offset,$length, $data,$dataoffset, $callback->($retval)
412 Reads or writes $length bytes from or to the specified $fh and
413 $offset into the scalar given by $data and offset $dataoffset and
414 calls the callback with the actual number of bytes transferred (or
415 -1 on error, just like the syscall).
416 "aio_read" will, like "sysread", shrink or grow the $data scalar to
418 offset plus the actual number of bytes read.
419 If $offset is undefined, then the current file descriptor offset
421 will be used (and updated), otherwise the file descriptor offset
422 will not be changed by these calls.
423 If $length is undefined in "aio_write", use the remaining length of
425 $data.
426 If $dataoffset is less than zero, it will be counted from the end
428 of $data.
429 The $data scalar MUST NOT be modified in any way while the request
431 is outstanding. Modifying it can result in segfaults or World War
432 III (if the necessary/optional hardware is installed).
433 Example: Read 15 bytes at offset 7 into scalar $buffer, starting at
435 offset 0 within the scalar:
436 aio_read $fh, 7, 15, $buffer, 0, sub {
438 $_[0] > 0 or die "read error: $!";
439 print "read $_[0] bytes: <$buffer>\n";
440 };
441 aio_sendfile $out_fh, $in_fh, $in_offset, $length, $callback->($retval)
443 Tries to copy $length bytes from $in_fh to $out_fh. It starts
444 reading at byte offset $in_offset, and starts writing at the
445 current file offset of $out_fh. Because of that, it is not safe to
446 issue more than one "aio_sendfile" per $out_fh, as they will
447 interfere with each other. The same $in_fh works fine though, as
448 this function does not move or use the file offset of $in_fh.
449 Please note that "aio_sendfile" can read more bytes from $in_fh
451 than are written, and there is no way to find out how many more
452 bytes have been read from "aio_sendfile" alone, as "aio_sendfile"
453 only provides the number of bytes written to $out_fh. Only if the
454 result value equals $length one can assume that $length bytes have
455 been read.
456 Unlike with other "aio_" functions, it makes a lot of sense to use
458 "aio_sendfile" on non-blocking sockets, as long as one end
459 (typically the $in_fh) is a file - the file I/O will then be
460 asynchronous, while the socket I/O will be non-blocking. Note,
461 however, that you can run into a trap where "aio_sendfile" reads
462 some data with readahead, then fails to write all data, and when
463 the socket is ready the next time, the data in the cache is already
464 lost, forcing "aio_sendfile" to again hit the disk. Explicit
465 "aio_read" + "aio_write" let's you better control resource usage.
466 This call tries to make use of a native "sendfile"-like syscall to
468 provide zero-copy operation. For this to work, $out_fh should refer
469 to a socket, and $in_fh should refer to an mmap'able file.
470 If a native sendfile cannot be found or it fails with "ENOSYS",
472 "EINVAL", "ENOTSUP", "EOPNOTSUPP", "EAFNOSUPPORT", "EPROTOTYPE" or
473 "ENOTSOCK", it will be emulated, so you can call "aio_sendfile" on
474 any type of filehandle regardless of the limitations of the
475 operating system.
476 As native sendfile syscalls (as practically any non-POSIX interface
478 hacked together in a hurry to improve benchmark numbers) tend to be
479 rather buggy on many systems, this implementation tries to work
480 around some known bugs in Linux and FreeBSD kernels (probably
481 others, too), but that might fail, so you really really should
482 check the return value of "aio_sendfile" - fewer bytes than
483 expected might have been transferred.
484 aio_readahead $fh,$offset,$length, $callback->($retval)
486 "aio_readahead" populates the page cache with data from a file so
487 that subsequent reads from that file will not block on disk I/O.
488 The $offset argument specifies the starting point from which data
489 is to be read and $length specifies the number of bytes to be read.
490 I/O is performed in whole pages, so that offset is effectively
491 rounded down to a page boundary and bytes are read up to the next
492 page boundary greater than or equal to (off-set+length).
493 "aio_readahead" does not read beyond the end of the file. The
494 current file offset of the file is left unchanged.
495 If that syscall doesn't exist (likely if your kernel isn't Linux)
497 it will be emulated by simply reading the data, which would have a
498 similar effect.
499 aio_stat $fh_or_path, $callback->($status)
501 aio_lstat $fh, $callback->($status)
502 Works almost exactly like perl's "stat" or "lstat" in void context.
503 The callback will be called after the stat and the results will be
504 available using "stat _" or "-s _" and other tests (with the
505 exception of "-B" and "-T").
506 The pathname passed to "aio_stat" must be absolute. See API NOTES,
508 above, for an explanation.
509 Currently, the stats are always 64-bit-stats, i.e. instead of
511 returning an error when stat'ing a large file, the results will be
512 silently truncated unless perl itself is compiled with large file
513 support.
514 To help interpret the mode and dev/rdev stat values, IO::AIO offers
516 the following constants and functions (if not implemented, the
517 constants will be 0 and the functions will either "croak" or fall
518 back on traditional behaviour).
519 "S_IFMT", "S_IFIFO", "S_IFCHR", "S_IFBLK", "S_IFLNK", "S_IFREG",
521 "S_IFDIR", "S_IFWHT", "S_IFSOCK", "IO::AIO::major $dev_t",
522 "IO::AIO::minor $dev_t", "IO::AIO::makedev $major, $minor".
523 To access higher resolution stat timestamps, see "SUBSECOND STAT
525 TIME ACCESS".
526 Example: Print the length of /etc/passwd:
528 aio_stat "/etc/passwd", sub {
530 $_[0] and die "stat failed: $!";
531 print "size is ", -s _, "\n";
532 };
533 aio_statvfs $fh_or_path, $callback->($statvfs)
535 Works like the POSIX "statvfs" or "fstatvfs" syscalls, depending on
536 whether a file handle or path was passed.
537 On success, the callback is passed a hash reference with the
539 following members: "bsize", "frsize", "blocks", "bfree", "bavail",
540 "files", "ffree", "favail", "fsid", "flag" and "namemax". On
541 failure, "undef" is passed.
542 The following POSIX IO::AIO::ST_* constants are defined:
544 "ST_RDONLY" and "ST_NOSUID".
545 The following non-POSIX IO::AIO::ST_* flag masks are defined to
547 their correct value when available, or to 0 on systems that do not
548 support them: "ST_NODEV", "ST_NOEXEC", "ST_SYNCHRONOUS",
549 "ST_MANDLOCK", "ST_WRITE", "ST_APPEND", "ST_IMMUTABLE",
550 "ST_NOATIME", "ST_NODIRATIME" and "ST_RELATIME".
551 Example: stat "/wd" and dump out the data if successful.
553 aio_statvfs "/wd", sub {
555 my $f = $_[0]
556 or die "statvfs: $!";
557 use Data::Dumper;
559 say Dumper $f;
560 };
561 # result:
563 {
564 bsize => 1024,
565 bfree => 4333064312,
566 blocks => 10253828096,
567 files => 2050765568,
568 flag => 4096,
569 favail => 2042092649,
570 bavail => 4333064312,
571 ffree => 2042092649,
572 namemax => 255,
573 frsize => 1024,
574 fsid => 1810
575 }
576 aio_utime $fh_or_path, $atime, $mtime, $callback->($status)
578 Works like perl's "utime" function (including the special case of
579 $atime and $mtime being undef). Fractional times are supported if
580 the underlying syscalls support them.
581 When called with a pathname, uses utimensat(2) or utimes(2) if
583 available, otherwise utime(2). If called on a file descriptor, uses
584 futimens(2) or futimes(2) if available, otherwise returns ENOSYS,
585 so this is not portable.
586 Examples:
588 # set atime and mtime to current time (basically touch(1)):
590 aio_utime "path", undef, undef;
591 # set atime to current time and mtime to beginning of the epoch:
592 aio_utime "path", time, undef; # undef==0
593 aio_chown $fh_or_path, $uid, $gid, $callback->($status)
595 Works like perl's "chown" function, except that "undef" for either
596 $uid or $gid is being interpreted as "do not change" (but -1 can
597 also be used).
598 Examples:
600 # same as "chown root path" in the shell:
602 aio_chown "path", 0, -1;
603 # same as above:
604 aio_chown "path", 0, undef;
605 aio_truncate $fh_or_path, $offset, $callback->($status)
607 Works like truncate(2) or ftruncate(2).
608 aio_allocate $fh, $mode, $offset, $len, $callback->($status)
610 Allocates or frees disk space according to the $mode argument. See
611 the linux "fallocate" documentation for details.
612 $mode is usually 0 or "IO::AIO::FALLOC_FL_KEEP_SIZE" to allocate
614 space, or "IO::AIO::FALLOC_FL_PUNCH_HOLE |
615 IO::AIO::FALLOC_FL_KEEP_SIZE", to deallocate a file range.
616 IO::AIO also supports "FALLOC_FL_COLLAPSE_RANGE", to remove a range
618 (without leaving a hole), "FALLOC_FL_ZERO_RANGE", to zero a range,
619 "FALLOC_FL_INSERT_RANGE" to insert a range and
620 "FALLOC_FL_UNSHARE_RANGE" to unshare shared blocks (see your
621 fallocate(2) manpage).
622 The file system block size used by "fallocate" is presumably the
624 "f_bsize" returned by "statvfs", but different filesystems and
625 filetypes can dictate other limitations.
626 If "fallocate" isn't available or cannot be emulated (currently no
628 emulation will be attempted), passes "-1" and sets $! to "ENOSYS".
629 aio_chmod $fh_or_path, $mode, $callback->($status)
631 Works like perl's "chmod" function.
632 aio_unlink $pathname, $callback->($status)
634 Asynchronously unlink (delete) a file and call the callback with
635 the result code.
636 aio_mknod $pathname, $mode, $dev, $callback->($status)
638 [EXPERIMENTAL]
639 Asynchronously create a device node (or fifo). See mknod(2).
641 The only (POSIX-) portable way of calling this function is:
643 aio_mknod $pathname, IO::AIO::S_IFIFO | $mode, 0, sub { ...
645 See "aio_stat" for info about some potentially helpful extra
647 constants and functions.
648 aio_link $srcpath, $dstpath, $callback->($status)
650 Asynchronously create a new link to the existing object at $srcpath
651 at the path $dstpath and call the callback with the result code.
652 aio_symlink $srcpath, $dstpath, $callback->($status)
654 Asynchronously create a new symbolic link to the existing object at
655 $srcpath at the path $dstpath and call the callback with the result
656 code.
657 aio_readlink $pathname, $callback->($link)
659 Asynchronously read the symlink specified by $path and pass it to
660 the callback. If an error occurs, nothing or undef gets passed to
661 the callback.
662 aio_realpath $pathname, $callback->($path)
664 Asynchronously make the path absolute and resolve any symlinks in
665 $path. The resulting path only consists of directories (same as
666 Cwd::realpath).
667 This request can be used to get the absolute path of the current
669 working directory by passing it a path of . (a single dot).
670 aio_rename $srcpath, $dstpath, $callback->($status)
672 Asynchronously rename the object at $srcpath to $dstpath, just as
673 rename(2) and call the callback with the result code.
674 On systems that support the AIO::WD working directory abstraction
676 natively, the case "[$wd, "."]" as $srcpath is specialcased -
677 instead of failing, "rename" is called on the absolute path of $wd.
678 aio_rename2 $srcpath, $dstpath, $flags, $callback->($status)
680 Basically a version of "aio_rename" with an additional $flags
681 argument. Calling this with "$flags=0" is the same as calling
682 "aio_rename".
683 Non-zero flags are currently only supported on GNU/Linux systems
685 that support renameat2. Other systems fail with "ENOSYS" in this
686 case.
687 The following constants are available (missing ones are, as usual
689 0), see renameat2(2) for details:
690 "IO::AIO::RENAME_NOREPLACE", "IO::AIO::RENAME_EXCHANGE" and
692 "IO::AIO::RENAME_WHITEOUT".
693 aio_mkdir $pathname, $mode, $callback->($status)
695 Asynchronously mkdir (create) a directory and call the callback
696 with the result code. $mode will be modified by the umask at the
697 time the request is executed, so do not change your umask.
698 aio_rmdir $pathname, $callback->($status)
700 Asynchronously rmdir (delete) a directory and call the callback
701 with the result code.
702 On systems that support the AIO::WD working directory abstraction
704 natively, the case "[$wd, "."]" is specialcased - instead of
705 failing, "rmdir" is called on the absolute path of $wd.
706 aio_readdir $pathname, $callback->($entries)
708 Unlike the POSIX call of the same name, "aio_readdir" reads an
709 entire directory (i.e. opendir + readdir + closedir). The entries
710 will not be sorted, and will NOT include the "." and ".." entries.
711 The callback is passed a single argument which is either "undef" or
713 an array-ref with the filenames.
714 aio_readdirx $pathname, $flags, $callback->($entries, $flags)
716 Quite similar to "aio_readdir", but the $flags argument allows one
717 to tune behaviour and output format. In case of an error, $entries
718 will be "undef".
719 The flags are a combination of the following constants, ORed
721 together (the flags will also be passed to the callback, possibly
722 modified):
723 IO::AIO::READDIR_DENTS
725 Normally the callback gets an arrayref consisting of names only
726 (as with "aio_readdir"). If this flag is set, then the callback
727 gets an arrayref with "[$name, $type, $inode]" arrayrefs, each
728 describing a single directory entry in more detail:
729 $name is the name of the entry.
731 $type is one of the "IO::AIO::DT_xxx" constants:
733 "IO::AIO::DT_UNKNOWN", "IO::AIO::DT_FIFO", "IO::AIO::DT_CHR",
735 "IO::AIO::DT_DIR", "IO::AIO::DT_BLK", "IO::AIO::DT_REG",
736 "IO::AIO::DT_LNK", "IO::AIO::DT_SOCK", "IO::AIO::DT_WHT".
737 "IO::AIO::DT_UNKNOWN" means just that: readdir does not know.
739 If you need to know, you have to run stat yourself. Also, for
740 speed/memory reasons, the $type scalars are read-only: you must
741 not modify them.
742 $inode is the inode number (which might not be exact on systems
744 with 64 bit inode numbers and 32 bit perls). This field has
745 unspecified content on systems that do not deliver the inode
746 information.
747 IO::AIO::READDIR_DIRS_FIRST
749 When this flag is set, then the names will be returned in an
750 order where likely directories come first, in optimal stat
751 order. This is useful when you need to quickly find
752 directories, or you want to find all directories while avoiding
753 to stat() each entry.
754 If the system returns type information in readdir, then this is
756 used to find directories directly. Otherwise, likely
757 directories are names beginning with ".", or otherwise names
758 with no dots, of which names with short names are tried first.
759 IO::AIO::READDIR_STAT_ORDER
761 When this flag is set, then the names will be returned in an
762 order suitable for stat()'ing each one. That is, when you plan
763 to stat() most or all files in the given directory, then the
764 returned order will likely be faster.
765 If both this flag and "IO::AIO::READDIR_DIRS_FIRST" are
767 specified, then the likely dirs come first, resulting in a less
768 optimal stat order for stat'ing all entries, but likely a more
769 optimal order for finding subdirectories.
770 IO::AIO::READDIR_FOUND_UNKNOWN
772 This flag should not be set when calling "aio_readdirx".
773 Instead, it is being set by "aio_readdirx", when any of the
774 $type's found were "IO::AIO::DT_UNKNOWN". The absence of this
775 flag therefore indicates that all $type's are known, which can
776 be used to speed up some algorithms.
777 aio_slurp $pathname, $offset, $length, $data, $callback->($status)
779 Opens, reads and closes the given file. The data is put into $data,
780 which is resized as required.
781 If $offset is negative, then it is counted from the end of the
783 file.
784 If $length is zero, then the remaining length of the file is used.
786 Also, in this case, the same limitations to modifying $data apply
787 as when IO::AIO::mmap is used, i.e. it must only be modified in-
788 place with "substr". If the size of the file is known, specifying a
789 non-zero $length results in a performance advantage.
790 This request is similar to the older "aio_load" request, but since
792 it is a single request, it might be more efficient to use.
793 Example: load /etc/passwd into $passwd.
795 my $passwd;
797 aio_slurp "/etc/passwd", 0, 0, $passwd, sub {
798 $_[0] >= 0
799 or die "/etc/passwd: $!\n";
800 printf "/etc/passwd is %d bytes long, and contains:\n", length $passwd;
802 print $passwd;
803 };
804 IO::AIO::flush;
805 aio_load $pathname, $data, $callback->($status)
807 This is a composite request that tries to fully load the given file
808 into memory. Status is the same as with aio_read.
809 Using "aio_slurp" might be more efficient, as it is a single
811 request.
812 aio_copy $srcpath, $dstpath, $callback->($status)
814 Try to copy the file (directories not supported as either source or
815 destination) from $srcpath to $dstpath and call the callback with a
816 status of 0 (ok) or "-1" (error, see $!).
817 Existing destination files will be truncated.
819 This is a composite request that creates the destination file with
821 mode 0200 and copies the contents of the source file into it using
822 "aio_sendfile", followed by restoring atime, mtime, access mode and
823 uid/gid, in that order.
824 If an error occurs, the partial destination file will be unlinked,
826 if possible, except when setting atime, mtime, access mode and
827 uid/gid, where errors are being ignored.
828 aio_move $srcpath, $dstpath, $callback->($status)
830 Try to move the file (directories not supported as either source or
831 destination) from $srcpath to $dstpath and call the callback with a
832 status of 0 (ok) or "-1" (error, see $!).
833 This is a composite request that tries to rename(2) the file first;
835 if rename fails with "EXDEV", it copies the file with "aio_copy"
836 and, if that is successful, unlinks the $srcpath.
837 aio_scandir $pathname, $maxreq, $callback->($dirs, $nondirs)
839 Scans a directory (similar to "aio_readdir") but additionally tries
840 to efficiently separate the entries of directory $path into two
841 sets of names, directories you can recurse into (directories), and
842 ones you cannot recurse into (everything else, including symlinks
843 to directories).
844 "aio_scandir" is a composite request that generates many sub
846 requests. $maxreq specifies the maximum number of outstanding aio
847 requests that this function generates. If it is "<= 0", then a
848 suitable default will be chosen (currently 4).
849 On error, the callback is called without arguments, otherwise it
851 receives two array-refs with path-relative entry names.
852 Example:
854 aio_scandir $dir, 0, sub {
856 my ($dirs, $nondirs) = @_;
857 print "real directories: @$dirs\n";
858 print "everything else: @$nondirs\n";
859 };
860 Implementation notes.
862 The "aio_readdir" cannot be avoided, but "stat()"'ing every entry
864 can.
865 If readdir returns file type information, then this is used
867 directly to find directories.
868 Otherwise, after reading the directory, the modification time, size
870 etc. of the directory before and after the readdir is checked, and
871 if they match (and isn't the current time), the link count will be
872 used to decide how many entries are directories (if >= 2).
873 Otherwise, no knowledge of the number of subdirectories will be
874 assumed.
875 Then entries will be sorted into likely directories a non-initial
877 dot currently) and likely non-directories (see "aio_readdirx").
878 Then every entry plus an appended "/." will be "stat"'ed, likely
879 directories first, in order of their inode numbers. If that
880 succeeds, it assumes that the entry is a directory or a symlink to
881 directory (which will be checked separately). This is often faster
882 than stat'ing the entry itself because filesystems might detect the
883 type of the entry without reading the inode data (e.g. ext2fs
884 filetype feature), even on systems that cannot return the filetype
885 information on readdir.
886 If the known number of directories (link count - 2) has been
888 reached, the rest of the entries is assumed to be non-directories.
889 This only works with certainty on POSIX (= UNIX) filesystems, which
891 fortunately are the vast majority of filesystems around.
892 It will also likely work on non-POSIX filesystems with reduced
894 efficiency as those tend to return 0 or 1 as link counts, which
895 disables the directory counting heuristic.
896 aio_rmtree $pathname, $callback->($status)
898 Delete a directory tree starting (and including) $path, return the
899 status of the final "rmdir" only. This is a composite request that
900 uses "aio_scandir" to recurse into and rmdir directories, and
901 unlink everything else.
902 aio_fcntl $fh, $cmd, $arg, $callback->($status)
904 aio_ioctl $fh, $request, $buf, $callback->($status)
905 These work just like the "fcntl" and "ioctl" built-in functions,
906 except they execute asynchronously and pass the return value to the
907 callback.
908 Both calls can be used for a lot of things, some of which make more
910 sense to run asynchronously in their own thread, while some others
911 make less sense. For example, calls that block waiting for external
912 events, such as locking, will also lock down an I/O thread while it
913 is waiting, which can deadlock the whole I/O system. At the same
914 time, there might be no alternative to using a thread to wait.
915 So in general, you should only use these calls for things that do
917 (filesystem) I/O, not for things that wait for other events
918 (network, other processes), although if you are careful and know
919 what you are doing, you still can.
920 The following constants are available and can be used for normal
922 "ioctl" and "fcntl" as well (missing ones are, as usual 0):
923 "F_DUPFD_CLOEXEC",
925 "F_OFD_GETLK", "F_OFD_SETLK", "F_OFD_GETLKW",
927 "FIFREEZE", "FITHAW", "FITRIM", "FICLONE", "FICLONERANGE",
929 "FIDEDUPERANGE".
930 "F_ADD_SEALS", "F_GET_SEALS", "F_SEAL_SEAL", "F_SEAL_SHRINK",
932 "F_SEAL_GROW" and "F_SEAL_WRITE".
933 "FS_IOC_GETFLAGS", "FS_IOC_SETFLAGS", "FS_IOC_GETVERSION",
935 "FS_IOC_SETVERSION", "FS_IOC_FIEMAP".
936 "FS_IOC_FSGETXATTR", "FS_IOC_FSSETXATTR",
938 "FS_IOC_SET_ENCRYPTION_POLICY", "FS_IOC_GET_ENCRYPTION_PWSALT",
939 "FS_IOC_GET_ENCRYPTION_POLICY", "FS_KEY_DESCRIPTOR_SIZE".
940 "FS_SECRM_FL", "FS_UNRM_FL", "FS_COMPR_FL", "FS_SYNC_FL",
942 "FS_IMMUTABLE_FL", "FS_APPEND_FL", "FS_NODUMP_FL", "FS_NOATIME_FL",
943 "FS_DIRTY_FL", "FS_COMPRBLK_FL", "FS_NOCOMP_FL", "FS_ENCRYPT_FL",
944 "FS_BTREE_FL", "FS_INDEX_FL", "FS_JOURNAL_DATA_FL", "FS_NOTAIL_FL",
945 "FS_DIRSYNC_FL", "FS_TOPDIR_FL", "FS_FL_USER_MODIFIABLE".
946 "FS_XFLAG_REALTIME", "FS_XFLAG_PREALLOC", "FS_XFLAG_IMMUTABLE",
948 "FS_XFLAG_APPEND", "FS_XFLAG_SYNC", "FS_XFLAG_NOATIME",
949 "FS_XFLAG_NODUMP", "FS_XFLAG_RTINHERIT", "FS_XFLAG_PROJINHERIT",
950 "FS_XFLAG_NOSYMLINKS", "FS_XFLAG_EXTSIZE", "FS_XFLAG_EXTSZINHERIT",
951 "FS_XFLAG_NODEFRAG", "FS_XFLAG_FILESTREAM", "FS_XFLAG_DAX",
952 "FS_XFLAG_HASATTR",
953 aio_sync $callback->($status)
955 Asynchronously call sync and call the callback when finished.
956 aio_fsync $fh, $callback->($status)
958 Asynchronously call fsync on the given filehandle and call the
959 callback with the fsync result code.
960 aio_fdatasync $fh, $callback->($status)
962 Asynchronously call fdatasync on the given filehandle and call the
963 callback with the fdatasync result code.
964 If this call isn't available because your OS lacks it or it
966 couldn't be detected, it will be emulated by calling "fsync"
967 instead.
968 aio_syncfs $fh, $callback->($status)
970 Asynchronously call the syncfs syscall to sync the filesystem
971 associated to the given filehandle and call the callback with the
972 syncfs result code. If syncfs is not available, calls sync(), but
973 returns "-1" and sets errno to "ENOSYS" nevertheless.
974 aio_sync_file_range $fh, $offset, $nbytes, $flags, $callback->($status)
976 Sync the data portion of the file specified by $offset and $length
977 to disk (but NOT the metadata), by calling the Linux-specific
978 sync_file_range call. If sync_file_range is not available or it
979 returns ENOSYS, then fdatasync or fsync is being substituted.
980 $flags can be a combination of
982 "IO::AIO::SYNC_FILE_RANGE_WAIT_BEFORE",
983 "IO::AIO::SYNC_FILE_RANGE_WRITE" and
984 "IO::AIO::SYNC_FILE_RANGE_WAIT_AFTER": refer to the sync_file_range
985 manpage for details.
986 aio_pathsync $pathname, $callback->($status)
988 This request tries to open, fsync and close the given path. This is
989 a composite request intended to sync directories after directory
990 operations (E.g. rename). This might not work on all operating
991 systems or have any specific effect, but usually it makes sure that
992 directory changes get written to disc. It works for anything that
993 can be opened for read-only, not just directories.
994 Future versions of this function might fall back to other methods
996 when "fsync" on the directory fails (such as calling "sync").
997 Passes 0 when everything went ok, and "-1" on error.
999 aio_msync $scalar, $offset = 0, $length = undef, flags = MS_SYNC,
1001 $callback->($status)
1002 This is a rather advanced IO::AIO call, which only works on
1003 mmap(2)ed scalars (see the "IO::AIO::mmap" function, although it
1004 also works on data scalars managed by the Sys::Mmap or Mmap
1005 modules, note that the scalar must only be modified in-place while
1006 an aio operation is pending on it).
1007 It calls the "msync" function of your OS, if available, with the
1009 memory area starting at $offset in the string and ending $length
1010 bytes later. If $length is negative, counts from the end, and if
1011 $length is "undef", then it goes till the end of the string. The
1012 flags can be either "IO::AIO::MS_ASYNC" or "IO::AIO::MS_SYNC", plus
1013 an optional "IO::AIO::MS_INVALIDATE".
1014 aio_mtouch $scalar, $offset = 0, $length = undef, flags = 0,
1016 $callback->($status)
1017 This is a rather advanced IO::AIO call, which works best on
1018 mmap(2)ed scalars.
1019 It touches (reads or writes) all memory pages in the specified
1021 range inside the scalar. All caveats and parameters are the same as
1022 for "aio_msync", above, except for flags, which must be either 0
1023 (which reads all pages and ensures they are instantiated) or
1024 "IO::AIO::MT_MODIFY", which modifies the memory pages (by reading
1025 and writing an octet from it, which dirties the page).
1026 aio_mlock $scalar, $offset = 0, $length = undef, $callback->($status)
1028 This is a rather advanced IO::AIO call, which works best on
1029 mmap(2)ed scalars.
1030 It reads in all the pages of the underlying storage into memory (if
1032 any) and locks them, so they are not getting swapped/paged out or
1033 removed.
1034 If $length is undefined, then the scalar will be locked till the
1036 end.
1037 On systems that do not implement "mlock", this function returns
1039 "-1" and sets errno to "ENOSYS".
1040 Note that the corresponding "munlock" is synchronous and is
1042 documented under "MISCELLANEOUS FUNCTIONS".
1043 Example: open a file, mmap and mlock it - both will be undone when
1045 $data gets destroyed.
1046 open my $fh, "<", $path or die "$path: $!";
1048 my $data;
1049 IO::AIO::mmap $data, -s $fh, IO::AIO::PROT_READ, IO::AIO::MAP_SHARED, $fh;
1050 aio_mlock $data; # mlock in background
1051 aio_mlockall $flags, $callback->($status)
1053 Calls the "mlockall" function with the given $flags (a combination
1054 of "IO::AIO::MCL_CURRENT", "IO::AIO::MCL_FUTURE" and
1055 "IO::AIO::MCL_ONFAULT").
1056 On systems that do not implement "mlockall", this function returns
1058 "-1" and sets errno to "ENOSYS". Similarly, flag combinations not
1059 supported by the system result in a return value of "-1" with errno
1060 being set to "EINVAL".
1061 Note that the corresponding "munlockall" is synchronous and is
1063 documented under "MISCELLANEOUS FUNCTIONS".
1064 Example: asynchronously lock all current and future pages into
1066 memory.
1067 aio_mlockall IO::AIO::MCL_FUTURE;
1069 aio_fiemap $fh, $start, $length, $flags, $count, $cb->(\@extents)
1071 Queries the extents of the given file (by calling the Linux
1072 "FIEMAP" ioctl, see <http://cvs.schmorp.de/IO-AIO/doc/fiemap.txt>
1073 for details). If the ioctl is not available on your OS, then this
1074 request will fail with "ENOSYS".
1075 $start is the starting offset to query extents for, $length is the
1077 size of the range to query - if it is "undef", then the whole file
1078 will be queried.
1079 $flags is a combination of flags ("IO::AIO::FIEMAP_FLAG_SYNC" or
1081 "IO::AIO::FIEMAP_FLAG_XATTR" - "IO::AIO::FIEMAP_FLAGS_COMPAT" is
1082 also exported), and is normally 0 or "IO::AIO::FIEMAP_FLAG_SYNC" to
1083 query the data portion.
1084 $count is the maximum number of extent records to return. If it is
1086 "undef", then IO::AIO queries all extents of the range. As a very
1087 special case, if it is 0, then the callback receives the number of
1088 extents instead of the extents themselves (which is unreliable, see
1089 below).
1090 If an error occurs, the callback receives no arguments. The special
1092 "errno" value "IO::AIO::EBADR" is available to test for flag
1093 errors.
1094 Otherwise, the callback receives an array reference with extent
1096 structures. Each extent structure is an array reference itself,
1097 with the following members:
1098 [$logical, $physical, $length, $flags]
1100 Flags is any combination of the following flag values (typically
1102 either 0 or "IO::AIO::FIEMAP_EXTENT_LAST" (1)):
1103 "IO::AIO::FIEMAP_EXTENT_LAST", "IO::AIO::FIEMAP_EXTENT_UNKNOWN",
1105 "IO::AIO::FIEMAP_EXTENT_DELALLOC",
1106 "IO::AIO::FIEMAP_EXTENT_ENCODED",
1107 "IO::AIO::FIEMAP_EXTENT_DATA_ENCRYPTED",
1108 "IO::AIO::FIEMAP_EXTENT_NOT_ALIGNED",
1109 "IO::AIO::FIEMAP_EXTENT_DATA_INLINE",
1110 "IO::AIO::FIEMAP_EXTENT_DATA_TAIL",
1111 "IO::AIO::FIEMAP_EXTENT_UNWRITTEN", "IO::AIO::FIEMAP_EXTENT_MERGED"
1112 or "IO::AIO::FIEMAP_EXTENT_SHARED".
1113 At the time of this writing (Linux 3.2), this request is unreliable
1115 unless $count is "undef", as the kernel has all sorts of bugs
1116 preventing it to return all extents of a range for files with a
1117 large number of extents. The code (only) works around all these
1118 issues if $count is "undef".
1119 aio_group $callback->(...)
1121 This is a very special aio request: Instead of doing something, it
1122 is a container for other aio requests, which is useful if you want
1123 to bundle many requests into a single, composite, request with a
1124 definite callback and the ability to cancel the whole request with
1125 its subrequests.
1126 Returns an object of class IO::AIO::GRP. See its documentation
1128 below for more info.
1129 Example:
1131 my $grp = aio_group sub {
1133 print "all stats done\n";
1134 };
1135 add $grp
1137 (aio_stat ...),
1138 (aio_stat ...),
1139 ...;
1140 aio_nop $callback->()
1142 This is a special request - it does nothing in itself and is only
1143 used for side effects, such as when you want to add a dummy request
1144 to a group so that finishing the requests in the group depends on
1145 executing the given code.
1146 While this request does nothing, it still goes through the
1148 execution phase and still requires a worker thread. Thus, the
1149 callback will not be executed immediately but only after other
1150 requests in the queue have entered their execution phase. This can
1151 be used to measure request latency.
1152 IO::AIO::aio_busy $fractional_seconds, $callback->() *NOT EXPORTED*
1154 Mainly used for debugging and benchmarking, this aio request puts
1155 one of the request workers to sleep for the given time.
1156 While it is theoretically handy to have simple I/O scheduling
1158 requests like sleep and file handle readable/writable, the overhead
1159 this creates is immense (it blocks a thread for a long time) so do
1160 not use this function except to put your application under
1161 artificial I/O pressure.
1162 IO::AIO::WD - multiple working directories
1164 Your process only has one current working directory, which is used by
1165 all threads. This makes it hard to use relative paths (some other
1166 component could call "chdir" at any time, and it is hard to control
1167 when the path will be used by IO::AIO).
1168 One solution for this is to always use absolute paths. This usually
1170 works, but can be quite slow (the kernel has to walk the whole path on
1171 every access), and can also be a hassle to implement.
1172 Newer POSIX systems have a number of functions (openat, fdopendir,
1174 futimensat and so on) that make it possible to specify working
1175 directories per operation.
1176 For portability, and because the clowns who "designed", or shall I
1178 write, perpetrated this new interface were obviously half-drunk, this
1179 abstraction cannot be perfect, though.
1180 IO::AIO allows you to convert directory paths into a so-called
1182 IO::AIO::WD object. This object stores the canonicalised, absolute
1183 version of the path, and on systems that allow it, also a directory
1184 file descriptor.
1185 Everywhere where a pathname is accepted by IO::AIO (e.g. in "aio_stat"
1187 or "aio_unlink"), one can specify an array reference with an
1188 IO::AIO::WD object and a pathname instead (or the IO::AIO::WD object
1189 alone, which gets interpreted as "[$wd, "."]"). If the pathname is
1190 absolute, the IO::AIO::WD object is ignored, otherwise the pathname is
1191 resolved relative to that IO::AIO::WD object.
1192 For example, to get a wd object for /etc and then stat passwd inside,
1194 you would write:
1195 aio_wd "/etc", sub {
1197 my $etcdir = shift;
1198 # although $etcdir can be undef on error, there is generally no reason
1200 # to check for errors here, as aio_stat will fail with ENOENT
1201 # when $etcdir is undef.
1202 aio_stat [$etcdir, "passwd"], sub {
1204 # yay
1205 };
1206 };
1207 The fact that "aio_wd" is a request and not a normal function shows
1209 that creating an IO::AIO::WD object is itself a potentially blocking
1210 operation, which is why it is done asynchronously.
1211 To stat the directory obtained with "aio_wd" above, one could write
1213 either of the following three request calls:
1214 aio_lstat "/etc" , sub { ... # pathname as normal string
1216 aio_lstat [$wd, "."], sub { ... # "." relative to $wd (i.e. $wd itself)
1217 aio_lstat $wd , sub { ... # shorthand for the previous
1218 As with normal pathnames, IO::AIO keeps a copy of the working directory
1220 object and the pathname string, so you could write the following
1221 without causing any issues due to $path getting reused:
1222 my $path = [$wd, undef];
1224 for my $name (qw(abc def ghi)) {
1226 $path->[1] = $name;
1227 aio_stat $path, sub {
1228 # ...
1229 };
1230 }
1231 There are some caveats: when directories get renamed (or deleted), the
1233 pathname string doesn't change, so will point to the new directory (or
1234 nowhere at all), while the directory fd, if available on the system,
1235 will still point to the original directory. Most functions accepting a
1236 pathname will use the directory fd on newer systems, and the string on
1237 older systems. Some functions (such as "aio_realpath") will always rely
1238 on the string form of the pathname.
1239 So this functionality is mainly useful to get some protection against
1241 "chdir", to easily get an absolute path out of a relative path for
1242 future reference, and to speed up doing many operations in the same
1243 directory (e.g. when stat'ing all files in a directory).
1244 The following functions implement this working directory abstraction:
1246 aio_wd $pathname, $callback->($wd)
1248 Asynchonously canonicalise the given pathname and convert it to an
1249 IO::AIO::WD object representing it. If possible and supported on
1250 the system, also open a directory fd to speed up pathname
1251 resolution relative to this working directory.
1252 If something goes wrong, then "undef" is passwd to the callback
1254 instead of a working directory object and $! is set appropriately.
1255 Since passing "undef" as working directory component of a pathname
1256 fails the request with "ENOENT", there is often no need for error
1257 checking in the "aio_wd" callback, as future requests using the
1258 value will fail in the expected way.
1259 IO::AIO::CWD
1261 This is a compile time constant (object) that represents the
1262 process current working directory.
1263 Specifying this object as working directory object for a pathname
1265 is as if the pathname would be specified directly, without a
1266 directory object. For example, these calls are functionally
1267 identical:
1268 aio_stat "somefile", sub { ... };
1270 aio_stat [IO::AIO::CWD, "somefile"], sub { ... };
1271 To recover the path associated with an IO::AIO::WD object, you can use
1273 "aio_realpath":
1274 aio_realpath $wd, sub {
1276 warn "path is $_[0]\n";
1277 };
1278 Currently, "aio_statvfs" always, and "aio_rename" and "aio_rmdir"
1280 sometimes, fall back to using an absolue path.
1281 IO::AIO::REQ CLASS
1283 All non-aggregate "aio_*" functions return an object of this class when
1284 called in non-void context.
1285 cancel $req
1287 Cancels the request, if possible. Has the effect of skipping
1288 execution when entering the execute state and skipping calling the
1289 callback when entering the the result state, but will leave the
1290 request otherwise untouched (with the exception of readdir). That
1291 means that requests that currently execute will not be stopped and
1292 resources held by the request will not be freed prematurely.
1293 cb $req $callback->(...)
1295 Replace (or simply set) the callback registered to the request.
1296 IO::AIO::GRP CLASS
1298 This class is a subclass of IO::AIO::REQ, so all its methods apply to
1299 objects of this class, too.
1300 A IO::AIO::GRP object is a special request that can contain multiple
1302 other aio requests.
1303 You create one by calling the "aio_group" constructing function with a
1305 callback that will be called when all contained requests have entered
1306 the "done" state:
1307 my $grp = aio_group sub {
1309 print "all requests are done\n";
1310 };
1311 You add requests by calling the "add" method with one or more
1313 "IO::AIO::REQ" objects:
1314 $grp->add (aio_unlink "...");
1316 add $grp aio_stat "...", sub {
1318 $_[0] or return $grp->result ("error");
1319 # add another request dynamically, if first succeeded
1321 add $grp aio_open "...", sub {
1322 $grp->result ("ok");
1323 };
1324 };
1325 This makes it very easy to create composite requests (see the source of
1327 "aio_move" for an application) that work and feel like simple requests.
1328 • The IO::AIO::GRP objects will be cleaned up during calls to
1330 "IO::AIO::poll_cb", just like any other request.
1331 • They can be canceled like any other request. Canceling will cancel
1333 not only the request itself, but also all requests it contains.
1334 • They can also can also be added to other IO::AIO::GRP objects.
1336 • You must not add requests to a group from within the group callback
1338 (or any later time).
1339 Their lifetime, simplified, looks like this: when they are empty, they
1341 will finish very quickly. If they contain only requests that are in the
1342 "done" state, they will also finish. Otherwise they will continue to
1343 exist.
1344 That means after creating a group you have some time to add requests
1346 (precisely before the callback has been invoked, which is only done
1347 within the "poll_cb"). And in the callbacks of those requests, you can
1348 add further requests to the group. And only when all those requests
1349 have finished will the the group itself finish.
1350 add $grp ...
1352 $grp->add (...)
1353 Add one or more requests to the group. Any type of IO::AIO::REQ can
1354 be added, including other groups, as long as you do not create
1355 circular dependencies.
1356 Returns all its arguments.
1358 $grp->cancel_subs
1360 Cancel all subrequests and clears any feeder, but not the group
1361 request itself. Useful when you queued a lot of events but got a
1362 result early.
1363 The group request will finish normally (you cannot add requests to
1365 the group).
1366 $grp->result (...)
1368 Set the result value(s) that will be passed to the group callback
1369 when all subrequests have finished and set the groups errno to the
1370 current value of errno (just like calling "errno" without an error
1371 number). By default, no argument will be passed and errno is zero.
1372 $grp->errno ([$errno])
1374 Sets the group errno value to $errno, or the current value of errno
1375 when the argument is missing.
1376 Every aio request has an associated errno value that is restored
1378 when the callback is invoked. This method lets you change this
1379 value from its default (0).
1380 Calling "result" will also set errno, so make sure you either set
1382 $! before the call to "result", or call c<errno> after it.
1383 feed $grp $callback->($grp)
1385 Sets a feeder/generator on this group: every group can have an
1386 attached generator that generates requests if idle. The idea behind
1387 this is that, although you could just queue as many requests as you
1388 want in a group, this might starve other requests for a potentially
1389 long time. For example, "aio_scandir" might generate hundreds of
1390 thousands of "aio_stat" requests, delaying any later requests for a
1391 long time.
1392 To avoid this, and allow incremental generation of requests, you
1394 can instead a group and set a feeder on it that generates those
1395 requests. The feed callback will be called whenever there are few
1396 enough (see "limit", below) requests active in the group itself and
1397 is expected to queue more requests.
1398 The feed callback can queue as many requests as it likes (i.e.
1400 "add" does not impose any limits).
1401 If the feed does not queue more requests when called, it will be
1403 automatically removed from the group.
1404 If the feed limit is 0 when this method is called, it will be set
1406 to 2 automatically.
1407 Example:
1409 # stat all files in @files, but only ever use four aio requests concurrently:
1411 my $grp = aio_group sub { print "finished\n" };
1413 limit $grp 4;
1414 feed $grp sub {
1415 my $file = pop @files
1416 or return;
1417 add $grp aio_stat $file, sub { ... };
1419 };
1420 limit $grp $num
1422 Sets the feeder limit for the group: The feeder will be called
1423 whenever the group contains less than this many requests.
1424 Setting the limit to 0 will pause the feeding process.
1426 The default value for the limit is 0, but note that setting a
1428 feeder automatically bumps it up to 2.
1429 SUPPORT FUNCTIONS
1431 EVENT PROCESSING AND EVENT LOOP INTEGRATION
1432 $fileno = IO::AIO::poll_fileno
1434 Return the request result pipe file descriptor. This filehandle
1435 must be polled for reading by some mechanism outside this module
1436 (e.g. EV, Glib, select and so on, see below or the SYNOPSIS). If
1437 the pipe becomes readable you have to call "poll_cb" to check the
1438 results.
1439 See "poll_cb" for an example.
1441 IO::AIO::poll_cb
1443 Process some requests that have reached the result phase (i.e. they
1444 have been executed but the results are not yet reported). You have
1445 to call this "regularly" to finish outstanding requests.
1446 Returns 0 if all events could be processed (or there were no events
1448 to process), or "-1" if it returned earlier for whatever reason.
1449 Returns immediately when no events are outstanding. The amount of
1450 events processed depends on the settings of
1451 "IO::AIO::max_poll_req", "IO::AIO::max_poll_time" and
1452 "IO::AIO::max_outstanding".
1453 If not all requests were processed for whatever reason, the poll
1455 file descriptor will still be ready when "poll_cb" returns, so
1456 normally you don't have to do anything special to have it called
1457 later.
1458 Apart from calling "IO::AIO::poll_cb" when the event filehandle
1460 becomes ready, it can be beneficial to call this function from
1461 loops which submit a lot of requests, to make sure the results get
1462 processed when they become available and not just when the loop is
1463 finished and the event loop takes over again. This function returns
1464 very fast when there are no outstanding requests.
1465 Example: Install an Event watcher that automatically calls
1467 IO::AIO::poll_cb with high priority (more examples can be found in
1468 the SYNOPSIS section, at the top of this document):
1469 Event->io (fd => IO::AIO::poll_fileno,
1471 poll => 'r', async => 1,
1472 cb => \&IO::AIO::poll_cb);
1473 IO::AIO::poll_wait
1475 Wait until either at least one request is in the result phase or no
1476 requests are outstanding anymore.
1477 This is useful if you want to synchronously wait for some requests
1479 to become ready, without actually handling them.
1480 See "nreqs" for an example.
1482 IO::AIO::poll
1484 Waits until some requests have been handled.
1485 Returns the number of requests processed, but is otherwise strictly
1487 equivalent to:
1488 IO::AIO::poll_wait, IO::AIO::poll_cb
1490 IO::AIO::flush
1492 Wait till all outstanding AIO requests have been handled.
1493 Strictly equivalent to:
1495 IO::AIO::poll_wait, IO::AIO::poll_cb
1497 while IO::AIO::nreqs;
1498 This function can be useful at program aborts, to make sure
1500 outstanding I/O has been done ("IO::AIO" uses an "END" block which
1501 already calls this function on normal exits), or when you are
1502 merely using "IO::AIO" for its more advanced functions, rather than
1503 for async I/O, e.g.:
1504 my ($dirs, $nondirs);
1506 IO::AIO::aio_scandir "/tmp", 0, sub { ($dirs, $nondirs) = @_ };
1507 IO::AIO::flush;
1508 # $dirs, $nondirs are now set
1509 IO::AIO::max_poll_reqs $nreqs
1511 IO::AIO::max_poll_time $seconds
1512 These set the maximum number of requests (default 0, meaning
1513 infinity) that are being processed by "IO::AIO::poll_cb" in one
1514 call, respectively the maximum amount of time (default 0, meaning
1515 infinity) spent in "IO::AIO::poll_cb" to process requests (more
1516 correctly the mininum amount of time "poll_cb" is allowed to use).
1517 Setting "max_poll_time" to a non-zero value creates an overhead of
1519 one syscall per request processed, which is not normally a problem
1520 unless your callbacks are really really fast or your OS is really
1521 really slow (I am not mentioning Solaris here). Using
1522 "max_poll_reqs" incurs no overhead.
1523 Setting these is useful if you want to ensure some level of
1525 interactiveness when perl is not fast enough to process all
1526 requests in time.
1527 For interactive programs, values such as 0.01 to 0.1 should be
1529 fine.
1530 Example: Install an Event watcher that automatically calls
1532 IO::AIO::poll_cb with low priority, to ensure that other parts of
1533 the program get the CPU sometimes even under high AIO load.
1534 # try not to spend much more than 0.1s in poll_cb
1536 IO::AIO::max_poll_time 0.1;
1537 # use a low priority so other tasks have priority
1539 Event->io (fd => IO::AIO::poll_fileno,
1540 poll => 'r', nice => 1,
1541 cb => &IO::AIO::poll_cb);
1542 CONTROLLING THE NUMBER OF THREADS
1544 IO::AIO::min_parallel $nthreads
1546 Set the minimum number of AIO threads to $nthreads. The current
1547 default is 8, which means eight asynchronous operations can execute
1548 concurrently at any one time (the number of outstanding requests,
1549 however, is unlimited).
1550 IO::AIO starts threads only on demand, when an AIO request is
1552 queued and no free thread exists. Please note that queueing up a
1553 hundred requests can create demand for a hundred threads, even if
1554 it turns out that everything is in the cache and could have been
1555 processed faster by a single thread.
1556 It is recommended to keep the number of threads relatively low, as
1558 some Linux kernel versions will scale negatively with the number of
1559 threads (higher parallelity => MUCH higher latency). With current
1560 Linux 2.6 versions, 4-32 threads should be fine.
1561 Under most circumstances you don't need to call this function, as
1563 the module selects a default that is suitable for low to moderate
1564 load.
1565 IO::AIO::max_parallel $nthreads
1567 Sets the maximum number of AIO threads to $nthreads. If more than
1568 the specified number of threads are currently running, this
1569 function kills them. This function blocks until the limit is
1570 reached.
1571 While $nthreads are zero, aio requests get queued but not executed
1573 until the number of threads has been increased again.
1574 This module automatically runs "max_parallel 0" at program end, to
1576 ensure that all threads are killed and that there are no
1577 outstanding requests.
1578 Under normal circumstances you don't need to call this function.
1580 IO::AIO::max_idle $nthreads
1582 Limit the number of threads (default: 4) that are allowed to idle
1583 (i.e., threads that did not get a request to process within the
1584 idle timeout (default: 10 seconds). That means if a thread becomes
1585 idle while $nthreads other threads are also idle, it will free its
1586 resources and exit.
1587 This is useful when you allow a large number of threads (e.g. 100
1589 or 1000) to allow for extremely high load situations, but want to
1590 free resources under normal circumstances (1000 threads can easily
1591 consume 30MB of RAM).
1592 The default is probably ok in most situations, especially if thread
1594 creation is fast. If thread creation is very slow on your system
1595 you might want to use larger values.
1596 IO::AIO::idle_timeout $seconds
1598 Sets the minimum idle timeout (default 10) after which worker
1599 threads are allowed to exit. SEe "IO::AIO::max_idle".
1600 IO::AIO::max_outstanding $maxreqs
1602 Sets the maximum number of outstanding requests to $nreqs. If you
1603 do queue up more than this number of requests, the next call to
1604 "IO::AIO::poll_cb" (and other functions calling "poll_cb", such as
1605 "IO::AIO::flush" or "IO::AIO::poll") will block until the limit is
1606 no longer exceeded.
1607 In other words, this setting does not enforce a queue limit, but
1609 can be used to make poll functions block if the limit is exceeded.
1610 This is a very bad function to use in interactive programs because
1612 it blocks, and a bad way to reduce concurrency because it is
1613 inexact: Better use an "aio_group" together with a feed callback.
1614 Its main use is in scripts without an event loop - when you want to
1616 stat a lot of files, you can write something like this:
1617 IO::AIO::max_outstanding 32;
1619 for my $path (...) {
1621 aio_stat $path , ...;
1622 IO::AIO::poll_cb;
1623 }
1624 IO::AIO::flush;
1626 The call to "poll_cb" inside the loop will normally return
1628 instantly, but as soon as more thna 32 reqeusts are in-flight, it
1629 will block until some requests have been handled. This keeps the
1630 loop from pushing a large number of "aio_stat" requests onto the
1631 queue.
1632 The default value for "max_outstanding" is very large, so there is
1634 no practical limit on the number of outstanding requests.
1635 STATISTICAL INFORMATION
1637 IO::AIO::nreqs
1639 Returns the number of requests currently in the ready, execute or
1640 pending states (i.e. for which their callback has not been invoked
1641 yet).
1642 Example: wait till there are no outstanding requests anymore:
1644 IO::AIO::poll_wait, IO::AIO::poll_cb
1646 while IO::AIO::nreqs;
1647 IO::AIO::nready
1649 Returns the number of requests currently in the ready state (not
1650 yet executed).
1651 IO::AIO::npending
1653 Returns the number of requests currently in the pending state
1654 (executed, but not yet processed by poll_cb).
1655 SUBSECOND STAT TIME ACCESS
1657 Both "aio_stat"/"aio_lstat" and perl's "stat"/"lstat" functions can
1659 generally find access/modification and change times with subsecond time
1660 accuracy of the system supports it, but perl's built-in functions only
1661 return the integer part.
1662 The following functions return the timestamps of the most recent stat
1664 with subsecond precision on most systems and work both after
1665 "aio_stat"/"aio_lstat" and perl's "stat"/"lstat" calls. Their return
1666 value is only meaningful after a successful "stat"/"lstat" call, or
1667 during/after a successful "aio_stat"/"aio_lstat" callback.
1668 This is similar to the Time::HiRes "stat" functions, but can return
1670 full resolution without rounding and work with standard perl "stat",
1671 alleviating the need to call the special "Time::HiRes" functions, which
1672 do not act like their perl counterparts.
1673 On operating systems or file systems where subsecond time resolution is
1675 not supported or could not be detected, a fractional part of 0 is
1676 returned, so it is always safe to call these functions.
1677 $seconds = IO::AIO::st_atime, IO::AIO::st_mtime, IO::AIO::st_ctime,
1679 IO::AIO::st_btime
1680 Return the access, modication, change or birth time, respectively,
1681 including fractional part. Due to the limited precision of floating
1682 point, the accuracy on most platforms is only a bit better than
1683 milliseconds for times around now - see the nsec function family,
1684 below, for full accuracy.
1685 File birth time is only available when the OS and perl support it
1687 (on FreeBSD and NetBSD at the time of this writing, although
1688 support is adaptive, so if your OS/perl gains support, IO::AIO can
1689 take advantage of it). On systems where it isn't available, 0 is
1690 currently returned, but this might change to "undef" in a future
1691 version.
1692 ($atime, $mtime, $ctime, $btime, ...) = IO::AIO::st_xtime
1694 Returns access, modification, change and birth time all in one go,
1695 and maybe more times in the future version.
1696 $nanoseconds = IO::AIO::st_atimensec, IO::AIO::st_mtimensec,
1698 IO::AIO::st_ctimensec, IO::AIO::st_btimensec
1699 Return the fractional access, modifcation, change or birth time, in
1700 nanoseconds, as an integer in the range 0 to 999999999.
1701 Note that no accessors are provided for access, modification and
1703 change times - you need to get those from "stat _" if required
1704 ("int IO::AIO::st_atime" and so on will not generally give you the
1705 correct value).
1706 $seconds = IO::AIO::st_btimesec
1708 The (integral) seconds part of the file birth time, if available.
1709 ($atime, $mtime, $ctime, $btime, ...) = IO::AIO::st_xtimensec
1711 Like the functions above, but returns all four times in one go (and
1712 maybe more in future versions).
1713 $counter = IO::AIO::st_gen
1715 Returns the generation counter (in practice this is just a random
1716 number) of the file. This is only available on platforms which have
1717 this member in their "struct stat" (most BSDs at the time of this
1718 writing) and generally only to the root usert. If unsupported, 0 is
1719 returned, but this might change to "undef" in a future version.
1720 Example: print the high resolution modification time of /etc, using
1722 "stat", and "IO::AIO::aio_stat".
1723 if (stat "/etc") {
1725 printf "stat(/etc) mtime: %f\n", IO::AIO::st_mtime;
1726 }
1727 IO::AIO::aio_stat "/etc", sub {
1729 $_[0]
1730 and return;
1731 printf "aio_stat(/etc) mtime: %d.%09d\n", (stat _)[9], IO::AIO::st_mtimensec;
1733 };
1734 IO::AIO::flush;
1736 Output of the awbove on my system, showing reduced and full accuracy:
1738 stat(/etc) mtime: 1534043702.020808
1740 aio_stat(/etc) mtime: 1534043702.020807792
1741 MISCELLANEOUS FUNCTIONS
1743 IO::AIO implements some functions that are useful when you want to use
1745 some "Advanced I/O" function not available to in Perl, without going
1746 the "Asynchronous I/O" route. Many of these have an asynchronous
1747 "aio_*" counterpart.
1748 $numfd = IO::AIO::get_fdlimit
1750 Tries to find the current file descriptor limit and returns it, or
1751 "undef" and sets $! in case of an error. The limit is one larger
1752 than the highest valid file descriptor number.
1753 IO::AIO::min_fdlimit [$numfd]
1755 Try to increase the current file descriptor limit(s) to at least
1756 $numfd by changing the soft or hard file descriptor resource limit.
1757 If $numfd is missing, it will try to set a very high limit,
1758 although this is not recommended when you know the actual minimum
1759 that you require.
1760 If the limit cannot be raised enough, the function makes a best-
1762 effort attempt to increase the limit as much as possible, using
1763 various tricks, while still failing. You can query the resulting
1764 limit using "IO::AIO::get_fdlimit".
1765 If an error occurs, returns "undef" and sets $!, otherwise returns
1767 true.
1768 IO::AIO::sendfile $ofh, $ifh, $offset, $count
1770 Calls the "eio_sendfile_sync" function, which is like
1771 "aio_sendfile", but is blocking (this makes most sense if you know
1772 the input data is likely cached already and the output filehandle
1773 is set to non-blocking operations).
1774 Returns the number of bytes copied, or "-1" on error.
1776 IO::AIO::fadvise $fh, $offset, $len, $advice
1778 Simply calls the "posix_fadvise" function (see its manpage for
1779 details). The following advice constants are available:
1780 "IO::AIO::FADV_NORMAL", "IO::AIO::FADV_SEQUENTIAL",
1781 "IO::AIO::FADV_RANDOM", "IO::AIO::FADV_NOREUSE",
1782 "IO::AIO::FADV_WILLNEED", "IO::AIO::FADV_DONTNEED".
1783 On systems that do not implement "posix_fadvise", this function
1785 returns ENOSYS, otherwise the return value of "posix_fadvise".
1786 IO::AIO::madvise $scalar, $offset, $len, $advice
1788 Simply calls the "posix_madvise" function (see its manpage for
1789 details). The following advice constants are available:
1790 "IO::AIO::MADV_NORMAL", "IO::AIO::MADV_SEQUENTIAL",
1791 "IO::AIO::MADV_RANDOM", "IO::AIO::MADV_WILLNEED",
1792 "IO::AIO::MADV_DONTNEED".
1793 If $offset is negative, counts from the end. If $length is
1795 negative, the remaining length of the $scalar is used. If possible,
1796 $length will be reduced to fit into the $scalar.
1797 On systems that do not implement "posix_madvise", this function
1799 returns ENOSYS, otherwise the return value of "posix_madvise".
1800 IO::AIO::mprotect $scalar, $offset, $len, $protect
1802 Simply calls the "mprotect" function on the preferably AIO::mmap'ed
1803 $scalar (see its manpage for details). The following protect
1804 constants are available: "IO::AIO::PROT_NONE",
1805 "IO::AIO::PROT_READ", "IO::AIO::PROT_WRITE", "IO::AIO::PROT_EXEC".
1806 If $offset is negative, counts from the end. If $length is
1808 negative, the remaining length of the $scalar is used. If possible,
1809 $length will be reduced to fit into the $scalar.
1810 On systems that do not implement "mprotect", this function returns
1812 ENOSYS, otherwise the return value of "mprotect".
1813 IO::AIO::mmap $scalar, $length, $prot, $flags, $fh[, $offset]
1815 Memory-maps a file (or anonymous memory range) and attaches it to
1816 the given $scalar, which will act like a string scalar. Returns
1817 true on success, and false otherwise.
1818 The scalar must exist, but its contents do not matter - this means
1820 you cannot use a nonexistant array or hash element. When in doubt,
1821 "undef" the scalar first.
1822 The only operations allowed on the mmapped scalar are
1824 "substr"/"vec", which don't change the string length, and most
1825 read-only operations such as copying it or searching it with
1826 regexes and so on.
1827 Anything else is unsafe and will, at best, result in memory leaks.
1829 The memory map associated with the $scalar is automatically removed
1831 when the $scalar is undef'd or destroyed, or when the
1832 "IO::AIO::mmap" or "IO::AIO::munmap" functions are called on it.
1833 This calls the "mmap"(2) function internally. See your system's
1835 manual page for details on the $length, $prot and $flags
1836 parameters.
1837 The $length must be larger than zero and smaller than the actual
1839 filesize.
1840 $prot is a combination of "IO::AIO::PROT_NONE",
1842 "IO::AIO::PROT_EXEC", "IO::AIO::PROT_READ" and/or
1843 "IO::AIO::PROT_WRITE",
1844 $flags can be a combination of "IO::AIO::MAP_SHARED" or
1846 "IO::AIO::MAP_PRIVATE", or a number of system-specific flags (when
1847 not available, the are 0): "IO::AIO::MAP_ANONYMOUS" (which is set
1848 to "MAP_ANON" if your system only provides this constant),
1849 "IO::AIO::MAP_LOCKED", "IO::AIO::MAP_NORESERVE",
1850 "IO::AIO::MAP_POPULATE", "IO::AIO::MAP_NONBLOCK",
1851 "IO::AIO::MAP_FIXED", "IO::AIO::MAP_GROWSDOWN",
1852 "IO::AIO::MAP_32BIT", "IO::AIO::MAP_HUGETLB" or
1853 "IO::AIO::MAP_STACK".
1854 If $fh is "undef", then a file descriptor of "-1" is passed.
1856 $offset is the offset from the start of the file - it generally
1858 must be a multiple of "IO::AIO::PAGESIZE" and defaults to 0.
1859 Example:
1861 use Digest::MD5;
1863 use IO::AIO;
1864 open my $fh, "<verybigfile"
1866 or die "$!";
1867 IO::AIO::mmap my $data, -s $fh, IO::AIO::PROT_READ, IO::AIO::MAP_SHARED, $fh
1869 or die "verybigfile: $!";
1870 my $fast_md5 = md5 $data;
1872 IO::AIO::munmap $scalar
1874 Removes a previous mmap and undefines the $scalar.
1875 IO::AIO::mremap $scalar, $new_length, $flags = MREMAP_MAYMOVE[,
1877 $new_address = 0]
1878 Calls the Linux-specific mremap(2) system call. The $scalar must
1879 have been mapped by "IO::AIO::mmap", and $flags must currently
1880 either be 0 or "IO::AIO::MREMAP_MAYMOVE".
1881 Returns true if successful, and false otherwise. If the underlying
1883 mmapped region has changed address, then the true value has the
1884 numerical value 1, otherwise it has the numerical value 0:
1885 my $success = IO::AIO::mremap $mmapped, 8192, IO::AIO::MREMAP_MAYMOVE
1887 or die "mremap: $!";
1888 if ($success*1) {
1890 warn "scalar has chanegd address in memory\n";
1891 }
1892 "IO::AIO::MREMAP_FIXED" and the $new_address argument are currently
1894 implemented, but not supported and might go away in a future
1895 version.
1896 On systems where this call is not supported or is not emulated,
1898 this call returns falls and sets $! to "ENOSYS".
1899 IO::AIO::mlockall $flags
1901 Calls the "eio_mlockall_sync" function, which is like
1902 "aio_mlockall", but is blocking.
1903 IO::AIO::munlock $scalar, $offset = 0, $length = undef
1905 Calls the "munlock" function, undoing the effects of a previous
1906 "aio_mlock" call (see its description for details).
1907 IO::AIO::munlockall
1909 Calls the "munlockall" function.
1910 On systems that do not implement "munlockall", this function
1912 returns ENOSYS, otherwise the return value of "munlockall".
1913 $fh = IO::AIO::accept4 $r_fh, $sockaddr, $sockaddr_maxlen, $flags
1915 Uses the GNU/Linux accept4(2) syscall, if available, to accept a
1916 socket and return the new file handle on success, or sets $! and
1917 returns "undef" on error.
1918 The remote name of the new socket will be stored in $sockaddr,
1920 which will be extended to allow for at least $sockaddr_maxlen
1921 octets. If the socket name does not fit into $sockaddr_maxlen
1922 octets, this is signaled by returning a longer string in $sockaddr,
1923 which might or might not be truncated.
1924 To accept name-less sockets, use "undef" for $sockaddr and 0 for
1926 $sockaddr_maxlen.
1927 The main reasons to use this syscall rather than portable accept(2)
1929 are that you can specify "SOCK_NONBLOCK" and/or "SOCK_CLOEXEC"
1930 flags and you can accept name-less sockets by specifying 0 for
1931 $sockaddr_maxlen, which is sadly not possible with perl's interface
1932 to "accept".
1933 IO::AIO::splice $r_fh, $r_off, $w_fh, $w_off, $length, $flags
1935 Calls the GNU/Linux splice(2) syscall, if available. If $r_off or
1936 $w_off are "undef", then "NULL" is passed for these, otherwise they
1937 should be the file offset.
1938 $r_fh and $w_fh should not refer to the same file, as splice might
1940 silently corrupt the data in this case.
1941 The following symbol flag values are available:
1943 "IO::AIO::SPLICE_F_MOVE", "IO::AIO::SPLICE_F_NONBLOCK",
1944 "IO::AIO::SPLICE_F_MORE" and "IO::AIO::SPLICE_F_GIFT".
1945 See the splice(2) manpage for details.
1947 IO::AIO::tee $r_fh, $w_fh, $length, $flags
1949 Calls the GNU/Linux tee(2) syscall, see its manpage and the
1950 description for "IO::AIO::splice" above for details.
1951 $actual_size = IO::AIO::pipesize $r_fh[, $new_size]
1953 Attempts to query or change the pipe buffer size. Obviously works
1954 only on pipes, and currently works only on GNU/Linux systems, and
1955 fails with "-1"/"ENOSYS" everywhere else. If anybody knows how to
1956 influence pipe buffer size on other systems, drop me a note.
1957 ($rfh, $wfh) = IO::AIO::pipe2 [$flags]
1959 This is a direct interface to the Linux pipe2(2) system call. If
1960 $flags is missing or 0, then this should be the same as a call to
1961 perl's built-in "pipe" function and create a new pipe, and works on
1962 systems that lack the pipe2 syscall. On win32, this case invokes
1963 "_pipe (..., 4096, O_BINARY)".
1964 If $flags is non-zero, it tries to invoke the pipe2 system call
1966 with the given flags (Linux 2.6.27, glibc 2.9).
1967 On success, the read and write file handles are returned.
1969 On error, nothing will be returned. If the pipe2 syscall is missing
1971 and $flags is non-zero, fails with "ENOSYS".
1972 Please refer to pipe2(2) for more info on the $flags, but at the
1974 time of this writing, "IO::AIO::O_CLOEXEC", "IO::AIO::O_NONBLOCK"
1975 and "IO::AIO::O_DIRECT" (Linux 3.4, for packet-based pipes) were
1976 supported.
1977 Example: create a pipe race-free w.r.t. threads and fork:
1979 my ($rfh, $wfh) = IO::AIO::pipe2 IO::AIO::O_CLOEXEC
1981 or die "pipe2: $!\n";
1982 $fh = IO::AIO::memfd_create $pathname[, $flags]
1984 This is a direct interface to the Linux memfd_create(2) system
1985 call. The (unhelpful) default for $flags is 0, but your default
1986 should be "IO::AIO::MFD_CLOEXEC".
1987 On success, the new memfd filehandle is returned, otherwise returns
1989 "undef". If the memfd_create syscall is missing, fails with
1990 "ENOSYS".
1991 Please refer to memfd_create(2) for more info on this call.
1993 The following $flags values are available: "IO::AIO::MFD_CLOEXEC",
1995 "IO::AIO::MFD_ALLOW_SEALING" and "IO::AIO::MFD_HUGETLB".
1996 Example: create a new memfd.
1998 my $fh = IO::AIO::memfd_create "somenameforprocfd", IO::AIO::MFD_CLOEXEC
2000 or die "memfd_create: $!\n";
2001 $fh = IO::AIO::pidfd_open $pid[, $flags]
2003 This is an interface to the Linux pidfd_open(2) system call. The
2004 default for $flags is 0.
2005 On success, a new pidfd filehandle is returned (that is already set
2007 to close-on-exec), otherwise returns "undef". If the syscall is
2008 missing, fails with "ENOSYS".
2009 Example: open pid 6341 as pidfd.
2011 my $fh = IO::AIO::pidfd_open 6341
2013 or die "pidfd_open: $!\n";
2014 $status = IO::AIO::pidfd_send_signal $pidfh, $signal[, $siginfo[,
2016 $flags]]
2017 This is an interface to the Linux pidfd_send_signal system call.
2018 The default for $siginfo is "undef" and the default for $flags is
2019 0.
2020 Returns the system call status. If the syscall is missing, fails
2022 with "ENOSYS".
2023 When specified, $siginfo must be a reference to a hash with one or
2025 more of the following members:
2026 code - the "si_code" member
2028 pid - the "si_pid" member
2029 uid - the "si_uid" member
2030 value_int - the "si_value.sival_int" member
2031 value_ptr - the "si_value.sival_ptr" member, specified as an
2032 integer
2033 Example: send a SIGKILL to the specified process.
2035 my $status = IO::AIO::pidfd_send_signal $pidfh, 9, undef
2037 and die "pidfd_send_signal: $!\n";
2038 Example: send a SIGKILL to the specified process with extra data.
2040 my $status = IO::AIO::pidfd_send_signal $pidfh, 9, { code => -1, value_int => 7 }
2042 and die "pidfd_send_signal: $!\n";
2043 $fh = IO::AIO::pidfd_getfd $pidfh, $targetfd[, $flags]
2045 This is an interface to the Linux pidfd_getfd system call. The
2046 default for $flags is 0.
2047 On success, returns a dup'ed copy of the target file descriptor
2049 (specified as an integer) returned (that is already set to close-
2050 on-exec), otherwise returns "undef". If the syscall is missing,
2051 fails with "ENOSYS".
2052 Example: get a copy of standard error of another process and print
2054 soemthing to it.
2055 my $errfh = IO::AIO::pidfd_getfd $pidfh, 2
2057 or die "pidfd_getfd: $!\n";
2058 print $errfh "stderr\n";
2059 $fh = IO::AIO::eventfd [$initval, [$flags]]
2061 This is a direct interface to the Linux eventfd(2) system call. The
2062 (unhelpful) defaults for $initval and $flags are 0 for both.
2063 On success, the new eventfd filehandle is returned, otherwise
2065 returns "undef". If the eventfd syscall is missing, fails with
2066 "ENOSYS".
2067 Please refer to eventfd(2) for more info on this call.
2069 The following symbol flag values are available:
2071 "IO::AIO::EFD_CLOEXEC", "IO::AIO::EFD_NONBLOCK" and
2072 "IO::AIO::EFD_SEMAPHORE" (Linux 2.6.30).
2073 Example: create a new eventfd filehandle:
2075 $fh = IO::AIO::eventfd 0, IO::AIO::EFD_CLOEXEC
2077 or die "eventfd: $!\n";
2078 $fh = IO::AIO::timerfd_create $clockid[, $flags]
2080 This is a direct interface to the Linux timerfd_create(2) system
2081 call. The (unhelpful) default for $flags is 0, but your default
2082 should be "IO::AIO::TFD_CLOEXEC".
2083 On success, the new timerfd filehandle is returned, otherwise
2085 returns "undef". If the timerfd_create syscall is missing, fails
2086 with "ENOSYS".
2087 Please refer to timerfd_create(2) for more info on this call.
2089 The following $clockid values are available:
2091 "IO::AIO::CLOCK_REALTIME", "IO::AIO::CLOCK_MONOTONIC"
2092 "IO::AIO::CLOCK_CLOCK_BOOTTIME" (Linux 3.15)
2093 "IO::AIO::CLOCK_CLOCK_REALTIME_ALARM" (Linux 3.11) and
2094 "IO::AIO::CLOCK_CLOCK_BOOTTIME_ALARM" (Linux 3.11).
2095 The following $flags values are available (Linux 2.6.27):
2097 "IO::AIO::TFD_NONBLOCK" and "IO::AIO::TFD_CLOEXEC".
2098 Example: create a new timerfd and set it to one-second repeated
2100 alarms, then wait for two alarms:
2101 my $fh = IO::AIO::timerfd_create IO::AIO::CLOCK_BOOTTIME, IO::AIO::TFD_CLOEXEC
2103 or die "timerfd_create: $!\n";
2104 defined IO::AIO::timerfd_settime $fh, 0, 1, 1
2106 or die "timerfd_settime: $!\n";
2107 for (1..2) {
2109 8 == sysread $fh, my $buf, 8
2110 or die "timerfd read failure\n";
2111 printf "number of expirations (likely 1): %d\n",
2113 unpack "Q", $buf;
2114 }
2115 ($cur_interval, $cur_value) = IO::AIO::timerfd_settime $fh, $flags,
2117 $new_interval, $nbw_value
2118 This is a direct interface to the Linux timerfd_settime(2) system
2119 call. Please refer to its manpage for more info on this call.
2120 The new itimerspec is specified using two (possibly fractional)
2122 second values, $new_interval and $new_value).
2123 On success, the current interval and value are returned (as per
2125 "timerfd_gettime"). On failure, the empty list is returned.
2126 The following $flags values are available:
2128 "IO::AIO::TFD_TIMER_ABSTIME" and
2129 "IO::AIO::TFD_TIMER_CANCEL_ON_SET".
2130 See "IO::AIO::timerfd_create" for a full example.
2132 ($cur_interval, $cur_value) = IO::AIO::timerfd_gettime $fh
2134 This is a direct interface to the Linux timerfd_gettime(2) system
2135 call. Please refer to its manpage for more info on this call.
2136 On success, returns the current values of interval and value for
2138 the given timerfd (as potentially fractional second values). On
2139 failure, the empty list is returned.
2140
2142 It is recommended to use AnyEvent::AIO to integrate IO::AIO
2143 automatically into many event loops:
2144 # AnyEvent integration (EV, Event, Glib, Tk, POE, urxvt, pureperl...)
2146 use AnyEvent::AIO;
2147 You can also integrate IO::AIO manually into many event loops, here are
2149 some examples of how to do this:
2150 # EV integration
2152 my $aio_w = EV::io IO::AIO::poll_fileno, EV::READ, \&IO::AIO::poll_cb;
2153 # Event integration
2155 Event->io (fd => IO::AIO::poll_fileno,
2156 poll => 'r',
2157 cb => \&IO::AIO::poll_cb);
2158 # Glib/Gtk2 integration
2160 add_watch Glib::IO IO::AIO::poll_fileno,
2161 in => sub { IO::AIO::poll_cb; 1 };
2162 # Tk integration
2164 Tk::Event::IO->fileevent (IO::AIO::poll_fileno, "",
2165 readable => \&IO::AIO::poll_cb);
2166 # Danga::Socket integration
2168 Danga::Socket->AddOtherFds (IO::AIO::poll_fileno =>
2169 \&IO::AIO::poll_cb);
2170 FORK BEHAVIOUR
2172 Usage of pthreads in a program changes the semantics of fork
2173 considerably. Specifically, only async-safe functions can be called
2174 after fork. Perl doesn't know about this, so in general, you cannot
2175 call fork with defined behaviour in perl if pthreads are involved.
2176 IO::AIO uses pthreads, so this applies, but many other extensions and
2177 (for inexplicable reasons) perl itself often is linked against
2178 pthreads, so this limitation applies to quite a lot of perls.
2179 This module no longer tries to fight your OS, or POSIX. That means
2181 IO::AIO only works in the process that loaded it. Forking is fully
2182 supported, but using IO::AIO in the child is not.
2183 You might get around by not using IO::AIO before (or after) forking.
2185 You could also try to call the IO::AIO::reinit function in the child:
2186 IO::AIO::reinit
2188 Abandons all current requests and I/O threads and simply
2189 reinitialises all data structures. This is not an operation
2190 supported by any standards, but happens to work on GNU/Linux and
2191 some newer BSD systems.
2192 The only reasonable use for this function is to call it after
2194 forking, if "IO::AIO" was used in the parent. Calling it while
2195 IO::AIO is active in the process will result in undefined
2196 behaviour. Calling it at any time will also result in any undefined
2197 (by POSIX) behaviour.
2198 LINUX-SPECIFIC CALLS
2200 When a call is documented as "linux-specific" then this means it
2201 originated on GNU/Linux. "IO::AIO" will usually try to autodetect the
2202 availability and compatibility of such calls regardless of the platform
2203 it is compiled on, so platforms such as FreeBSD which often implement
2204 these calls will work. When in doubt, call them and see if they fail
2205 wth "ENOSYS".
2206 MEMORY USAGE
2208 Per-request usage:
2209 Each aio request uses - depending on your architecture - around 100-200
2211 bytes of memory. In addition, stat requests need a stat buffer
2212 (possibly a few hundred bytes), readdir requires a result buffer and so
2213 on. Perl scalars and other data passed into aio requests will also be
2214 locked and will consume memory till the request has entered the done
2215 state.
2216 This is not awfully much, so queuing lots of requests is not usually a
2218 problem.
2219 Per-thread usage:
2221 In the execution phase, some aio requests require more memory for
2223 temporary buffers, and each thread requires a stack and other data
2224 structures (usually around 16k-128k, depending on the OS).
2225
2227 Known bugs will be fixed in the next release :)
2228
2230 Calls that try to "import" foreign memory areas (such as
2231 "IO::AIO::mmap" or "IO::AIO::aio_slurp") do not work with generic
2232 lvalues, such as non-created hash slots or other scalars I didn't think
2233 of. It's best to avoid such and either use scalar variables or making
2234 sure that the scalar exists (e.g. by storing "undef") and isn't "funny"
2235 (e.g. tied).
2236 I am not sure anything can be done about this, so this is considered a
2238 known issue, rather than a bug.
2239
2241 AnyEvent::AIO for easy integration into event loops, Coro::AIO for a
2242 more natural syntax and IO::FDPass for file descriptor passing.
2243
2245 Marc Lehmann <schmorp@schmorp.de>
2246 http://home.schmorp.de/
2247perl v5.32.1 2021-01-27 AIO(3)