1AIO(3) User Contributed Perl Documentation AIO(3)
2
6 IO::AIO - Asynchronous/Advanced Input/Output
7
9 use IO::AIO;
10 aio_open "/etc/passwd", IO::AIO::O_RDONLY, 0, sub {
12 my $fh = shift
13 or die "/etc/passwd: $!";
14 ...
15 };
16 aio_unlink "/tmp/file", sub { };
18 aio_read $fh, 30000, 1024, $buffer, 0, sub {
20 $_[0] > 0 or die "read error: $!";
21 };
22 # version 2+ has request and group objects
24 use IO::AIO 2;
25 aioreq_pri 4; # give next request a very high priority
27 my $req = aio_unlink "/tmp/file", sub { };
28 $req->cancel; # cancel request if still in queue
29 my $grp = aio_group sub { print "all stats done\n" };
31 add $grp aio_stat "..." for ...;
32
34 This module implements asynchronous I/O using whatever means your
35 operating system supports. It is implemented as an interface to
36 "libeio" (<http://software.schmorp.de/pkg/libeio.html>).
37 Asynchronous means that operations that can normally block your program
39 (e.g. reading from disk) will be done asynchronously: the operation
40 will still block, but you can do something else in the meantime. This
41 is extremely useful for programs that need to stay interactive even
42 when doing heavy I/O (GUI programs, high performance network servers
43 etc.), but can also be used to easily do operations in parallel that
44 are normally done sequentially, e.g. stat'ing many files, which is much
45 faster on a RAID volume or over NFS when you do a number of stat
46 operations concurrently.
47 While most of this works on all types of file descriptors (for example
49 sockets), using these functions on file descriptors that support
50 nonblocking operation (again, sockets, pipes etc.) is very inefficient.
51 Use an event loop for that (such as the EV module): IO::AIO will
52 naturally fit into such an event loop itself.
53 In this version, a number of threads are started that execute your
55 requests and signal their completion. You don't need thread support in
56 perl, and the threads created by this module will not be visible to
57 perl. In the future, this module might make use of the native aio
58 functions available on many operating systems. However, they are often
59 not well-supported or restricted (GNU/Linux doesn't allow them on
60 normal files currently, for example), and they would only support
61 aio_read and aio_write, so the remaining functionality would have to be
62 implemented using threads anyway.
63 In addition to asynchronous I/O, this module also exports some rather
65 arcane interfaces, such as "madvise" or linux's "splice" system call,
66 which is why the "A" in "AIO" can also mean advanced.
67 Although the module will work in the presence of other (Perl-) threads,
69 it is currently not reentrant in any way, so use appropriate locking
70 yourself, always call "poll_cb" from within the same thread, or never
71 call "poll_cb" (or other "aio_" functions) recursively.
72 EXAMPLE
74 This is a simple example that uses the EV module and loads /etc/passwd
75 asynchronously:
76 use EV;
78 use IO::AIO;
79 # register the IO::AIO callback with EV
81 my $aio_w = EV::io IO::AIO::poll_fileno, EV::READ, \&IO::AIO::poll_cb;
82 # queue the request to open /etc/passwd
84 aio_open "/etc/passwd", IO::AIO::O_RDONLY, 0, sub {
85 my $fh = shift
86 or die "error while opening: $!";
87 # stat'ing filehandles is generally non-blocking
89 my $size = -s $fh;
90 # queue a request to read the file
92 my $contents;
93 aio_read $fh, 0, $size, $contents, 0, sub {
94 $_[0] == $size
95 or die "short read: $!";
96 close $fh;
98 # file contents now in $contents
100 print $contents;
101 # exit event loop and program
103 EV::break;
104 };
105 };
106 # possibly queue up other requests, or open GUI windows,
108 # check for sockets etc. etc.
109 # process events as long as there are some:
111 EV::run;
112
114 Every "aio_*" function creates a request. which is a C data structure
115 not directly visible to Perl.
116 If called in non-void context, every request function returns a Perl
118 object representing the request. In void context, nothing is returned,
119 which saves a bit of memory.
120 The perl object is a fairly standard ref-to-hash object. The hash
122 contents are not used by IO::AIO so you are free to store anything you
123 like in it.
124 During their existance, aio requests travel through the following
126 states, in order:
127 ready
129 Immediately after a request is created it is put into the ready
130 state, waiting for a thread to execute it.
131 execute
133 A thread has accepted the request for processing and is currently
134 executing it (e.g. blocking in read).
135 pending
137 The request has been executed and is waiting for result processing.
138 While request submission and execution is fully asynchronous,
140 result processing is not and relies on the perl interpreter calling
141 "poll_cb" (or another function with the same effect).
142 result
144 The request results are processed synchronously by "poll_cb".
145 The "poll_cb" function will process all outstanding aio requests by
147 calling their callbacks, freeing memory associated with them and
148 managing any groups they are contained in.
149 done
151 Request has reached the end of its lifetime and holds no resources
152 anymore (except possibly for the Perl object, but its connection to
153 the actual aio request is severed and calling its methods will
154 either do nothing or result in a runtime error).
155
157 QUICK OVERVIEW
158 This section simply lists the prototypes most of the functions for
159 quick reference. See the following sections for function-by-function
160 documentation.
161 aio_wd $pathname, $callback->($wd)
163 aio_open $pathname, $flags, $mode, $callback->($fh)
164 aio_close $fh, $callback->($status)
165 aio_seek $fh,$offset,$whence, $callback->($offs)
166 aio_read $fh,$offset,$length, $data,$dataoffset, $callback->($retval)
167 aio_write $fh,$offset,$length, $data,$dataoffset, $callback->($retval)
168 aio_sendfile $out_fh, $in_fh, $in_offset, $length, $callback->($retval)
169 aio_readahead $fh,$offset,$length, $callback->($retval)
170 aio_stat $fh_or_path, $callback->($status)
171 aio_lstat $fh, $callback->($status)
172 aio_statvfs $fh_or_path, $callback->($statvfs)
173 aio_utime $fh_or_path, $atime, $mtime, $callback->($status)
174 aio_chown $fh_or_path, $uid, $gid, $callback->($status)
175 aio_chmod $fh_or_path, $mode, $callback->($status)
176 aio_truncate $fh_or_path, $offset, $callback->($status)
177 aio_allocate $fh, $mode, $offset, $len, $callback->($status)
178 aio_fiemap $fh, $start, $length, $flags, $count, $cb->(\@extents)
179 aio_unlink $pathname, $callback->($status)
180 aio_mknod $pathname, $mode, $dev, $callback->($status)
181 aio_link $srcpath, $dstpath, $callback->($status)
182 aio_symlink $srcpath, $dstpath, $callback->($status)
183 aio_readlink $pathname, $callback->($link)
184 aio_realpath $pathname, $callback->($path)
185 aio_rename $srcpath, $dstpath, $callback->($status)
186 aio_rename2 $srcpath, $dstpath, $flags, $callback->($status)
187 aio_mkdir $pathname, $mode, $callback->($status)
188 aio_rmdir $pathname, $callback->($status)
189 aio_readdir $pathname, $callback->($entries)
190 aio_readdirx $pathname, $flags, $callback->($entries, $flags)
191 IO::AIO::READDIR_DENTS IO::AIO::READDIR_DIRS_FIRST
192 IO::AIO::READDIR_STAT_ORDER IO::AIO::READDIR_FOUND_UNKNOWN
193 aio_scandir $pathname, $maxreq, $callback->($dirs, $nondirs)
194 aio_load $pathname, $data, $callback->($status)
195 aio_copy $srcpath, $dstpath, $callback->($status)
196 aio_move $srcpath, $dstpath, $callback->($status)
197 aio_rmtree $pathname, $callback->($status)
198 aio_fcntl $fh, $cmd, $arg, $callback->($status)
199 aio_ioctl $fh, $request, $buf, $callback->($status)
200 aio_sync $callback->($status)
201 aio_syncfs $fh, $callback->($status)
202 aio_fsync $fh, $callback->($status)
203 aio_fdatasync $fh, $callback->($status)
204 aio_sync_file_range $fh, $offset, $nbytes, $flags, $callback->($status)
205 aio_pathsync $pathname, $callback->($status)
206 aio_msync $scalar, $offset = 0, $length = undef, flags = MS_SYNC, $callback->($status)
207 aio_mtouch $scalar, $offset = 0, $length = undef, flags = 0, $callback->($status)
208 aio_mlock $scalar, $offset = 0, $length = undef, $callback->($status)
209 aio_mlockall $flags, $callback->($status)
210 aio_group $callback->(...)
211 aio_nop $callback->()
212 $prev_pri = aioreq_pri [$pri]
214 aioreq_nice $pri_adjust
215 IO::AIO::poll_wait
217 IO::AIO::poll_cb
218 IO::AIO::poll
219 IO::AIO::flush
220 IO::AIO::max_poll_reqs $nreqs
221 IO::AIO::max_poll_time $seconds
222 IO::AIO::min_parallel $nthreads
223 IO::AIO::max_parallel $nthreads
224 IO::AIO::max_idle $nthreads
225 IO::AIO::idle_timeout $seconds
226 IO::AIO::max_outstanding $maxreqs
227 IO::AIO::nreqs
228 IO::AIO::nready
229 IO::AIO::npending
230 $nfd = IO::AIO::get_fdlimit [EXPERIMENTAL]
231 IO::AIO::min_fdlimit $nfd [EXPERIMENTAL]
232 IO::AIO::sendfile $ofh, $ifh, $offset, $count
234 IO::AIO::fadvise $fh, $offset, $len, $advice
235 IO::AIO::mmap $scalar, $length, $prot, $flags[, $fh[, $offset]]
236 IO::AIO::munmap $scalar
237 IO::AIO::mremap $scalar, $new_length, $flags[, $new_address]
238 IO::AIO::madvise $scalar, $offset, $length, $advice
239 IO::AIO::mprotect $scalar, $offset, $length, $protect
240 IO::AIO::munlock $scalar, $offset = 0, $length = undef
241 IO::AIO::munlockall
242 API NOTES
244 All the "aio_*" calls are more or less thin wrappers around the syscall
245 with the same name (sans "aio_"). The arguments are similar or
246 identical, and they all accept an additional (and optional) $callback
247 argument which must be a code reference. This code reference will be
248 called after the syscall has been executed in an asynchronous fashion.
249 The results of the request will be passed as arguments to the callback
250 (and, if an error occured, in $!) - for most requests the syscall
251 return code (e.g. most syscalls return "-1" on error, unlike perl,
252 which usually delivers "false").
253 Some requests (such as "aio_readdir") pass the actual results and
255 communicate failures by passing "undef".
256 All functions expecting a filehandle keep a copy of the filehandle
258 internally until the request has finished.
259 All functions return request objects of type IO::AIO::REQ that allow
261 further manipulation of those requests while they are in-flight.
262 The pathnames you pass to these routines should be absolute. The reason
264 for this is that at the time the request is being executed, the current
265 working directory could have changed. Alternatively, you can make sure
266 that you never change the current working directory anywhere in the
267 program and then use relative paths. You can also take advantage of
268 IO::AIOs working directory abstraction, that lets you specify paths
269 relative to some previously-opened "working directory object" - see the
270 description of the "IO::AIO::WD" class later in this document.
271 To encode pathnames as octets, either make sure you either: a) always
273 pass in filenames you got from outside (command line, readdir etc.)
274 without tinkering, b) are in your native filesystem encoding, c) use
275 the Encode module and encode your pathnames to the locale (or other)
276 encoding in effect in the user environment, d) use
277 Glib::filename_from_unicode on unicode filenames or e) use something
278 else to ensure your scalar has the correct contents.
279 This works, btw. independent of the internal UTF-8 bit, which IO::AIO
281 handles correctly whether it is set or not.
282 AIO REQUEST FUNCTIONS
284 $prev_pri = aioreq_pri [$pri]
285 Returns the priority value that would be used for the next request
286 and, if $pri is given, sets the priority for the next aio request.
287 The default priority is 0, the minimum and maximum priorities are
289 "-4" and 4, respectively. Requests with higher priority will be
290 serviced first.
291 The priority will be reset to 0 after each call to one of the
293 "aio_*" functions.
294 Example: open a file with low priority, then read something from it
296 with higher priority so the read request is serviced before other
297 low priority open requests (potentially spamming the cache):
298 aioreq_pri -3;
300 aio_open ..., sub {
301 return unless $_[0];
302 aioreq_pri -2;
304 aio_read $_[0], ..., sub {
305 ...
306 };
307 };
308 aioreq_nice $pri_adjust
310 Similar to "aioreq_pri", but subtracts the given value from the
311 current priority, so the effect is cumulative.
312 aio_open $pathname, $flags, $mode, $callback->($fh)
314 Asynchronously open or create a file and call the callback with a
315 newly created filehandle for the file (or "undef" in case of an
316 error).
317 The pathname passed to "aio_open" must be absolute. See API NOTES,
319 above, for an explanation.
320 The $flags argument is a bitmask. See the "Fcntl" module for a
322 list. They are the same as used by "sysopen".
323 Likewise, $mode specifies the mode of the newly created file, if it
325 didn't exist and "O_CREAT" has been given, just like perl's
326 "sysopen", except that it is mandatory (i.e. use 0 if you don't
327 create new files, and 0666 or 0777 if you do). Note that the $mode
328 will be modified by the umask in effect then the request is being
329 executed, so better never change the umask.
330 Example:
332 aio_open "/etc/passwd", IO::AIO::O_RDONLY, 0, sub {
334 if ($_[0]) {
335 print "open successful, fh is $_[0]\n";
336 ...
337 } else {
338 die "open failed: $!\n";
339 }
340 };
341 In addition to all the common open modes/flags ("O_RDONLY",
343 "O_WRONLY", "O_RDWR", "O_CREAT", "O_TRUNC", "O_EXCL" and
344 "O_APPEND"), the following POSIX and non-POSIX constants are
345 available (missing ones on your system are, as usual, 0):
346 "O_ASYNC", "O_DIRECT", "O_NOATIME", "O_CLOEXEC", "O_NOCTTY",
348 "O_NOFOLLOW", "O_NONBLOCK", "O_EXEC", "O_SEARCH", "O_DIRECTORY",
349 "O_DSYNC", "O_RSYNC", "O_SYNC", "O_PATH", "O_TMPFILE", "O_TTY_INIT"
350 and "O_ACCMODE".
351 aio_close $fh, $callback->($status)
353 Asynchronously close a file and call the callback with the result
354 code.
355 Unfortunately, you can't do this to perl. Perl insists very
357 strongly on closing the file descriptor associated with the
358 filehandle itself.
359 Therefore, "aio_close" will not close the filehandle - instead it
361 will use dup2 to overwrite the file descriptor with the write-end
362 of a pipe (the pipe fd will be created on demand and will be
363 cached).
364 Or in other words: the file descriptor will be closed, but it will
366 not be free for reuse until the perl filehandle is closed.
367 aio_seek $fh, $offset, $whence, $callback->($offs)
369 Seeks the filehandle to the new $offset, similarly to perl's
370 "sysseek". The $whence can use the traditional values (0 for
371 "IO::AIO::SEEK_SET", 1 for "IO::AIO::SEEK_CUR" or 2 for
372 "IO::AIO::SEEK_END").
373 The resulting absolute offset will be passed to the callback, or
375 "-1" in case of an error.
376 In theory, the $whence constants could be different than the
378 corresponding values from Fcntl, but perl guarantees they are the
379 same, so don't panic.
380 As a GNU/Linux (and maybe Solaris) extension, also the constants
382 "IO::AIO::SEEK_DATA" and "IO::AIO::SEEK_HOLE" are available, if
383 they could be found. No guarantees about suitability for use in
384 "aio_seek" or Perl's "sysseek" can be made though, although I would
385 naively assume they "just work".
386 aio_read $fh,$offset,$length, $data,$dataoffset, $callback->($retval)
388 aio_write $fh,$offset,$length, $data,$dataoffset, $callback->($retval)
389 Reads or writes $length bytes from or to the specified $fh and
390 $offset into the scalar given by $data and offset $dataoffset and
391 calls the callback with the actual number of bytes transferred (or
392 -1 on error, just like the syscall).
393 "aio_read" will, like "sysread", shrink or grow the $data scalar to
395 offset plus the actual number of bytes read.
396 If $offset is undefined, then the current file descriptor offset
398 will be used (and updated), otherwise the file descriptor offset
399 will not be changed by these calls.
400 If $length is undefined in "aio_write", use the remaining length of
402 $data.
403 If $dataoffset is less than zero, it will be counted from the end
405 of $data.
406 The $data scalar MUST NOT be modified in any way while the request
408 is outstanding. Modifying it can result in segfaults or World War
409 III (if the necessary/optional hardware is installed).
410 Example: Read 15 bytes at offset 7 into scalar $buffer, starting at
412 offset 0 within the scalar:
413 aio_read $fh, 7, 15, $buffer, 0, sub {
415 $_[0] > 0 or die "read error: $!";
416 print "read $_[0] bytes: <$buffer>\n";
417 };
418 aio_sendfile $out_fh, $in_fh, $in_offset, $length, $callback->($retval)
420 Tries to copy $length bytes from $in_fh to $out_fh. It starts
421 reading at byte offset $in_offset, and starts writing at the
422 current file offset of $out_fh. Because of that, it is not safe to
423 issue more than one "aio_sendfile" per $out_fh, as they will
424 interfere with each other. The same $in_fh works fine though, as
425 this function does not move or use the file offset of $in_fh.
426 Please note that "aio_sendfile" can read more bytes from $in_fh
428 than are written, and there is no way to find out how many more
429 bytes have been read from "aio_sendfile" alone, as "aio_sendfile"
430 only provides the number of bytes written to $out_fh. Only if the
431 result value equals $length one can assume that $length bytes have
432 been read.
433 Unlike with other "aio_" functions, it makes a lot of sense to use
435 "aio_sendfile" on non-blocking sockets, as long as one end
436 (typically the $in_fh) is a file - the file I/O will then be
437 asynchronous, while the socket I/O will be non-blocking. Note,
438 however, that you can run into a trap where "aio_sendfile" reads
439 some data with readahead, then fails to write all data, and when
440 the socket is ready the next time, the data in the cache is already
441 lost, forcing "aio_sendfile" to again hit the disk. Explicit
442 "aio_read" + "aio_write" let's you better control resource usage.
443 This call tries to make use of a native "sendfile"-like syscall to
445 provide zero-copy operation. For this to work, $out_fh should refer
446 to a socket, and $in_fh should refer to an mmap'able file.
447 If a native sendfile cannot be found or it fails with "ENOSYS",
449 "EINVAL", "ENOTSUP", "EOPNOTSUPP", "EAFNOSUPPORT", "EPROTOTYPE" or
450 "ENOTSOCK", it will be emulated, so you can call "aio_sendfile" on
451 any type of filehandle regardless of the limitations of the
452 operating system.
453 As native sendfile syscalls (as practically any non-POSIX interface
455 hacked together in a hurry to improve benchmark numbers) tend to be
456 rather buggy on many systems, this implementation tries to work
457 around some known bugs in Linux and FreeBSD kernels (probably
458 others, too), but that might fail, so you really really should
459 check the return value of "aio_sendfile" - fewer bytes than
460 expected might have been transferred.
461 aio_readahead $fh,$offset,$length, $callback->($retval)
463 "aio_readahead" populates the page cache with data from a file so
464 that subsequent reads from that file will not block on disk I/O.
465 The $offset argument specifies the starting point from which data
466 is to be read and $length specifies the number of bytes to be read.
467 I/O is performed in whole pages, so that offset is effectively
468 rounded down to a page boundary and bytes are read up to the next
469 page boundary greater than or equal to (off-set+length).
470 "aio_readahead" does not read beyond the end of the file. The
471 current file offset of the file is left unchanged.
472 If that syscall doesn't exist (likely if your kernel isn't Linux)
474 it will be emulated by simply reading the data, which would have a
475 similar effect.
476 aio_stat $fh_or_path, $callback->($status)
478 aio_lstat $fh, $callback->($status)
479 Works almost exactly like perl's "stat" or "lstat" in void context.
480 The callback will be called after the stat and the results will be
481 available using "stat _" or "-s _" and other tests (with the
482 exception of "-B" and "-T").
483 The pathname passed to "aio_stat" must be absolute. See API NOTES,
485 above, for an explanation.
486 Currently, the stats are always 64-bit-stats, i.e. instead of
488 returning an error when stat'ing a large file, the results will be
489 silently truncated unless perl itself is compiled with large file
490 support.
491 To help interpret the mode and dev/rdev stat values, IO::AIO offers
493 the following constants and functions (if not implemented, the
494 constants will be 0 and the functions will either "croak" or fall
495 back on traditional behaviour).
496 "S_IFMT", "S_IFIFO", "S_IFCHR", "S_IFBLK", "S_IFLNK", "S_IFREG",
498 "S_IFDIR", "S_IFWHT", "S_IFSOCK", "IO::AIO::major $dev_t",
499 "IO::AIO::minor $dev_t", "IO::AIO::makedev $major, $minor".
500 To access higher resolution stat timestamps, see "SUBSECOND STAT
502 TIME ACCESS".
503 Example: Print the length of /etc/passwd:
505 aio_stat "/etc/passwd", sub {
507 $_[0] and die "stat failed: $!";
508 print "size is ", -s _, "\n";
509 };
510 aio_statvfs $fh_or_path, $callback->($statvfs)
512 Works like the POSIX "statvfs" or "fstatvfs" syscalls, depending on
513 whether a file handle or path was passed.
514 On success, the callback is passed a hash reference with the
516 following members: "bsize", "frsize", "blocks", "bfree", "bavail",
517 "files", "ffree", "favail", "fsid", "flag" and "namemax". On
518 failure, "undef" is passed.
519 The following POSIX IO::AIO::ST_* constants are defined:
521 "ST_RDONLY" and "ST_NOSUID".
522 The following non-POSIX IO::AIO::ST_* flag masks are defined to
524 their correct value when available, or to 0 on systems that do not
525 support them: "ST_NODEV", "ST_NOEXEC", "ST_SYNCHRONOUS",
526 "ST_MANDLOCK", "ST_WRITE", "ST_APPEND", "ST_IMMUTABLE",
527 "ST_NOATIME", "ST_NODIRATIME" and "ST_RELATIME".
528 Example: stat "/wd" and dump out the data if successful.
530 aio_statvfs "/wd", sub {
532 my $f = $_[0]
533 or die "statvfs: $!";
534 use Data::Dumper;
536 say Dumper $f;
537 };
538 # result:
540 {
541 bsize => 1024,
542 bfree => 4333064312,
543 blocks => 10253828096,
544 files => 2050765568,
545 flag => 4096,
546 favail => 2042092649,
547 bavail => 4333064312,
548 ffree => 2042092649,
549 namemax => 255,
550 frsize => 1024,
551 fsid => 1810
552 }
553 aio_utime $fh_or_path, $atime, $mtime, $callback->($status)
555 Works like perl's "utime" function (including the special case of
556 $atime and $mtime being undef). Fractional times are supported if
557 the underlying syscalls support them.
558 When called with a pathname, uses utimensat(2) or utimes(2) if
560 available, otherwise utime(2). If called on a file descriptor, uses
561 futimens(2) or futimes(2) if available, otherwise returns ENOSYS,
562 so this is not portable.
563 Examples:
565 # set atime and mtime to current time (basically touch(1)):
567 aio_utime "path", undef, undef;
568 # set atime to current time and mtime to beginning of the epoch:
569 aio_utime "path", time, undef; # undef==0
570 aio_chown $fh_or_path, $uid, $gid, $callback->($status)
572 Works like perl's "chown" function, except that "undef" for either
573 $uid or $gid is being interpreted as "do not change" (but -1 can
574 also be used).
575 Examples:
577 # same as "chown root path" in the shell:
579 aio_chown "path", 0, -1;
580 # same as above:
581 aio_chown "path", 0, undef;
582 aio_truncate $fh_or_path, $offset, $callback->($status)
584 Works like truncate(2) or ftruncate(2).
585 aio_allocate $fh, $mode, $offset, $len, $callback->($status)
587 Allocates or frees disk space according to the $mode argument. See
588 the linux "fallocate" documentation for details.
589 $mode is usually 0 or "IO::AIO::FALLOC_FL_KEEP_SIZE" to allocate
591 space, or "IO::AIO::FALLOC_FL_PUNCH_HOLE |
592 IO::AIO::FALLOC_FL_KEEP_SIZE", to deallocate a file range.
593 IO::AIO also supports "FALLOC_FL_COLLAPSE_RANGE", to remove a range
595 (without leaving a hole), "FALLOC_FL_ZERO_RANGE", to zero a range,
596 "FALLOC_FL_INSERT_RANGE" to insert a range and
597 "FALLOC_FL_UNSHARE_RANGE" to unshare shared blocks (see your
598 fallocate(2) manpage).
599 The file system block size used by "fallocate" is presumably the
601 "f_bsize" returned by "statvfs", but different filesystems and
602 filetypes can dictate other limitations.
603 If "fallocate" isn't available or cannot be emulated (currently no
605 emulation will be attempted), passes "-1" and sets $! to "ENOSYS".
606 aio_chmod $fh_or_path, $mode, $callback->($status)
608 Works like perl's "chmod" function.
609 aio_unlink $pathname, $callback->($status)
611 Asynchronously unlink (delete) a file and call the callback with
612 the result code.
613 aio_mknod $pathname, $mode, $dev, $callback->($status)
615 [EXPERIMENTAL]
616 Asynchronously create a device node (or fifo). See mknod(2).
618 The only (POSIX-) portable way of calling this function is:
620 aio_mknod $pathname, IO::AIO::S_IFIFO | $mode, 0, sub { ...
622 See "aio_stat" for info about some potentially helpful extra
624 constants and functions.
625 aio_link $srcpath, $dstpath, $callback->($status)
627 Asynchronously create a new link to the existing object at $srcpath
628 at the path $dstpath and call the callback with the result code.
629 aio_symlink $srcpath, $dstpath, $callback->($status)
631 Asynchronously create a new symbolic link to the existing object at
632 $srcpath at the path $dstpath and call the callback with the result
633 code.
634 aio_readlink $pathname, $callback->($link)
636 Asynchronously read the symlink specified by $path and pass it to
637 the callback. If an error occurs, nothing or undef gets passed to
638 the callback.
639 aio_realpath $pathname, $callback->($path)
641 Asynchronously make the path absolute and resolve any symlinks in
642 $path. The resulting path only consists of directories (same as
643 Cwd::realpath).
644 This request can be used to get the absolute path of the current
646 working directory by passing it a path of . (a single dot).
647 aio_rename $srcpath, $dstpath, $callback->($status)
649 Asynchronously rename the object at $srcpath to $dstpath, just as
650 rename(2) and call the callback with the result code.
651 On systems that support the AIO::WD working directory abstraction
653 natively, the case "[$wd, "."]" as $srcpath is specialcased -
654 instead of failing, "rename" is called on the absolute path of $wd.
655 aio_rename2 $srcpath, $dstpath, $flags, $callback->($status)
657 Basically a version of "aio_rename" with an additional $flags
658 argument. Calling this with "$flags=0" is the same as calling
659 "aio_rename".
660 Non-zero flags are currently only supported on GNU/Linux systems
662 that support renameat2. Other systems fail with "ENOSYS" in this
663 case.
664 The following constants are available (missing ones are, as usual
666 0), see renameat2(2) for details:
667 "IO::AIO::RENAME_NOREPLACE", "IO::AIO::RENAME_EXCHANGE" and
669 "IO::AIO::RENAME_WHITEOUT".
670 aio_mkdir $pathname, $mode, $callback->($status)
672 Asynchronously mkdir (create) a directory and call the callback
673 with the result code. $mode will be modified by the umask at the
674 time the request is executed, so do not change your umask.
675 aio_rmdir $pathname, $callback->($status)
677 Asynchronously rmdir (delete) a directory and call the callback
678 with the result code.
679 On systems that support the AIO::WD working directory abstraction
681 natively, the case "[$wd, "."]" is specialcased - instead of
682 failing, "rmdir" is called on the absolute path of $wd.
683 aio_readdir $pathname, $callback->($entries)
685 Unlike the POSIX call of the same name, "aio_readdir" reads an
686 entire directory (i.e. opendir + readdir + closedir). The entries
687 will not be sorted, and will NOT include the "." and ".." entries.
688 The callback is passed a single argument which is either "undef" or
690 an array-ref with the filenames.
691 aio_readdirx $pathname, $flags, $callback->($entries, $flags)
693 Quite similar to "aio_readdir", but the $flags argument allows one
694 to tune behaviour and output format. In case of an error, $entries
695 will be "undef".
696 The flags are a combination of the following constants, ORed
698 together (the flags will also be passed to the callback, possibly
699 modified):
700 IO::AIO::READDIR_DENTS
702 Normally the callback gets an arrayref consisting of names only
703 (as with "aio_readdir"). If this flag is set, then the callback
704 gets an arrayref with "[$name, $type, $inode]" arrayrefs, each
705 describing a single directory entry in more detail:
706 $name is the name of the entry.
708 $type is one of the "IO::AIO::DT_xxx" constants:
710 "IO::AIO::DT_UNKNOWN", "IO::AIO::DT_FIFO", "IO::AIO::DT_CHR",
712 "IO::AIO::DT_DIR", "IO::AIO::DT_BLK", "IO::AIO::DT_REG",
713 "IO::AIO::DT_LNK", "IO::AIO::DT_SOCK", "IO::AIO::DT_WHT".
714 "IO::AIO::DT_UNKNOWN" means just that: readdir does not know.
716 If you need to know, you have to run stat yourself. Also, for
717 speed/memory reasons, the $type scalars are read-only: you must
718 not modify them.
719 $inode is the inode number (which might not be exact on systems
721 with 64 bit inode numbers and 32 bit perls). This field has
722 unspecified content on systems that do not deliver the inode
723 information.
724 IO::AIO::READDIR_DIRS_FIRST
726 When this flag is set, then the names will be returned in an
727 order where likely directories come first, in optimal stat
728 order. This is useful when you need to quickly find
729 directories, or you want to find all directories while avoiding
730 to stat() each entry.
731 If the system returns type information in readdir, then this is
733 used to find directories directly. Otherwise, likely
734 directories are names beginning with ".", or otherwise names
735 with no dots, of which names with short names are tried first.
736 IO::AIO::READDIR_STAT_ORDER
738 When this flag is set, then the names will be returned in an
739 order suitable for stat()'ing each one. That is, when you plan
740 to stat() most or all files in the given directory, then the
741 returned order will likely be faster.
742 If both this flag and "IO::AIO::READDIR_DIRS_FIRST" are
744 specified, then the likely dirs come first, resulting in a less
745 optimal stat order for stat'ing all entries, but likely a more
746 optimal order for finding subdirectories.
747 IO::AIO::READDIR_FOUND_UNKNOWN
749 This flag should not be set when calling "aio_readdirx".
750 Instead, it is being set by "aio_readdirx", when any of the
751 $type's found were "IO::AIO::DT_UNKNOWN". The absence of this
752 flag therefore indicates that all $type's are known, which can
753 be used to speed up some algorithms.
754 aio_slurp $pathname, $offset, $length, $data, $callback->($status)
756 Opens, reads and closes the given file. The data is put into $data,
757 which is resized as required.
758 If $offset is negative, then it is counted from the end of the
760 file.
761 If $length is zero, then the remaining length of the file is used.
763 Also, in this case, the same limitations to modifying $data apply
764 as when IO::AIO::mmap is used, i.e. it must only be modified in-
765 place with "substr". If the size of the file is known, specifying a
766 non-zero $length results in a performance advantage.
767 This request is similar to the older "aio_load" request, but since
769 it is a single request, it might be more efficient to use.
770 Example: load /etc/passwd into $passwd.
772 my $passwd;
774 aio_slurp "/etc/passwd", 0, 0, $passwd, sub {
775 $_[0] >= 0
776 or die "/etc/passwd: $!\n";
777 printf "/etc/passwd is %d bytes long, and contains:\n", length $passwd;
779 print $passwd;
780 };
781 IO::AIO::flush;
782 aio_load $pathname, $data, $callback->($status)
784 This is a composite request that tries to fully load the given file
785 into memory. Status is the same as with aio_read.
786 Using "aio_slurp" might be more efficient, as it is a single
788 request.
789 aio_copy $srcpath, $dstpath, $callback->($status)
791 Try to copy the file (directories not supported as either source or
792 destination) from $srcpath to $dstpath and call the callback with a
793 status of 0 (ok) or "-1" (error, see $!).
794 Existing destination files will be truncated.
796 This is a composite request that creates the destination file with
798 mode 0200 and copies the contents of the source file into it using
799 "aio_sendfile", followed by restoring atime, mtime, access mode and
800 uid/gid, in that order.
801 If an error occurs, the partial destination file will be unlinked,
803 if possible, except when setting atime, mtime, access mode and
804 uid/gid, where errors are being ignored.
805 aio_move $srcpath, $dstpath, $callback->($status)
807 Try to move the file (directories not supported as either source or
808 destination) from $srcpath to $dstpath and call the callback with a
809 status of 0 (ok) or "-1" (error, see $!).
810 This is a composite request that tries to rename(2) the file first;
812 if rename fails with "EXDEV", it copies the file with "aio_copy"
813 and, if that is successful, unlinks the $srcpath.
814 aio_scandir $pathname, $maxreq, $callback->($dirs, $nondirs)
816 Scans a directory (similar to "aio_readdir") but additionally tries
817 to efficiently separate the entries of directory $path into two
818 sets of names, directories you can recurse into (directories), and
819 ones you cannot recurse into (everything else, including symlinks
820 to directories).
821 "aio_scandir" is a composite request that generates many sub
823 requests. $maxreq specifies the maximum number of outstanding aio
824 requests that this function generates. If it is "<= 0", then a
825 suitable default will be chosen (currently 4).
826 On error, the callback is called without arguments, otherwise it
828 receives two array-refs with path-relative entry names.
829 Example:
831 aio_scandir $dir, 0, sub {
833 my ($dirs, $nondirs) = @_;
834 print "real directories: @$dirs\n";
835 print "everything else: @$nondirs\n";
836 };
837 Implementation notes.
839 The "aio_readdir" cannot be avoided, but "stat()"'ing every entry
841 can.
842 If readdir returns file type information, then this is used
844 directly to find directories.
845 Otherwise, after reading the directory, the modification time, size
847 etc. of the directory before and after the readdir is checked, and
848 if they match (and isn't the current time), the link count will be
849 used to decide how many entries are directories (if >= 2).
850 Otherwise, no knowledge of the number of subdirectories will be
851 assumed.
852 Then entries will be sorted into likely directories a non-initial
854 dot currently) and likely non-directories (see "aio_readdirx").
855 Then every entry plus an appended "/." will be "stat"'ed, likely
856 directories first, in order of their inode numbers. If that
857 succeeds, it assumes that the entry is a directory or a symlink to
858 directory (which will be checked separately). This is often faster
859 than stat'ing the entry itself because filesystems might detect the
860 type of the entry without reading the inode data (e.g. ext2fs
861 filetype feature), even on systems that cannot return the filetype
862 information on readdir.
863 If the known number of directories (link count - 2) has been
865 reached, the rest of the entries is assumed to be non-directories.
866 This only works with certainty on POSIX (= UNIX) filesystems, which
868 fortunately are the vast majority of filesystems around.
869 It will also likely work on non-POSIX filesystems with reduced
871 efficiency as those tend to return 0 or 1 as link counts, which
872 disables the directory counting heuristic.
873 aio_rmtree $pathname, $callback->($status)
875 Delete a directory tree starting (and including) $path, return the
876 status of the final "rmdir" only. This is a composite request that
877 uses "aio_scandir" to recurse into and rmdir directories, and
878 unlink everything else.
879 aio_fcntl $fh, $cmd, $arg, $callback->($status)
881 aio_ioctl $fh, $request, $buf, $callback->($status)
882 These work just like the "fcntl" and "ioctl" built-in functions,
883 except they execute asynchronously and pass the return value to the
884 callback.
885 Both calls can be used for a lot of things, some of which make more
887 sense to run asynchronously in their own thread, while some others
888 make less sense. For example, calls that block waiting for external
889 events, such as locking, will also lock down an I/O thread while it
890 is waiting, which can deadlock the whole I/O system. At the same
891 time, there might be no alternative to using a thread to wait.
892 So in general, you should only use these calls for things that do
894 (filesystem) I/O, not for things that wait for other events
895 (network, other processes), although if you are careful and know
896 what you are doing, you still can.
897 The following constants are available (missing ones are, as usual
899 0):
900 "F_DUPFD_CLOEXEC",
902 "F_OFD_GETLK", "F_OFD_SETLK", "F_OFD_GETLKW",
904 "FIFREEZE", "FITHAW", "FITRIM", "FICLONE", "FICLONERANGE",
906 "FIDEDUPERANGE".
907 "FS_IOC_GETFLAGS", "FS_IOC_SETFLAGS", "FS_IOC_GETVERSION",
909 "FS_IOC_SETVERSION", "FS_IOC_FIEMAP".
910 "FS_IOC_FSGETXATTR", "FS_IOC_FSSETXATTR",
912 "FS_IOC_SET_ENCRYPTION_POLICY", "FS_IOC_GET_ENCRYPTION_PWSALT",
913 "FS_IOC_GET_ENCRYPTION_POLICY", "FS_KEY_DESCRIPTOR_SIZE".
914 "FS_SECRM_FL", "FS_UNRM_FL", "FS_COMPR_FL", "FS_SYNC_FL",
916 "FS_IMMUTABLE_FL", "FS_APPEND_FL", "FS_NODUMP_FL", "FS_NOATIME_FL",
917 "FS_DIRTY_FL", "FS_COMPRBLK_FL", "FS_NOCOMP_FL", "FS_ENCRYPT_FL",
918 "FS_BTREE_FL", "FS_INDEX_FL", "FS_JOURNAL_DATA_FL", "FS_NOTAIL_FL",
919 "FS_DIRSYNC_FL", "FS_TOPDIR_FL", "FS_FL_USER_MODIFIABLE".
920 "FS_XFLAG_REALTIME", "FS_XFLAG_PREALLOC", "FS_XFLAG_IMMUTABLE",
922 "FS_XFLAG_APPEND", "FS_XFLAG_SYNC", "FS_XFLAG_NOATIME",
923 "FS_XFLAG_NODUMP", "FS_XFLAG_RTINHERIT", "FS_XFLAG_PROJINHERIT",
924 "FS_XFLAG_NOSYMLINKS", "FS_XFLAG_EXTSIZE", "FS_XFLAG_EXTSZINHERIT",
925 "FS_XFLAG_NODEFRAG", "FS_XFLAG_FILESTREAM", "FS_XFLAG_DAX",
926 "FS_XFLAG_HASATTR",
927 aio_sync $callback->($status)
929 Asynchronously call sync and call the callback when finished.
930 aio_fsync $fh, $callback->($status)
932 Asynchronously call fsync on the given filehandle and call the
933 callback with the fsync result code.
934 aio_fdatasync $fh, $callback->($status)
936 Asynchronously call fdatasync on the given filehandle and call the
937 callback with the fdatasync result code.
938 If this call isn't available because your OS lacks it or it
940 couldn't be detected, it will be emulated by calling "fsync"
941 instead.
942 aio_syncfs $fh, $callback->($status)
944 Asynchronously call the syncfs syscall to sync the filesystem
945 associated to the given filehandle and call the callback with the
946 syncfs result code. If syncfs is not available, calls sync(), but
947 returns "-1" and sets errno to "ENOSYS" nevertheless.
948 aio_sync_file_range $fh, $offset, $nbytes, $flags, $callback->($status)
950 Sync the data portion of the file specified by $offset and $length
951 to disk (but NOT the metadata), by calling the Linux-specific
952 sync_file_range call. If sync_file_range is not available or it
953 returns ENOSYS, then fdatasync or fsync is being substituted.
954 $flags can be a combination of
956 "IO::AIO::SYNC_FILE_RANGE_WAIT_BEFORE",
957 "IO::AIO::SYNC_FILE_RANGE_WRITE" and
958 "IO::AIO::SYNC_FILE_RANGE_WAIT_AFTER": refer to the sync_file_range
959 manpage for details.
960 aio_pathsync $pathname, $callback->($status)
962 This request tries to open, fsync and close the given path. This is
963 a composite request intended to sync directories after directory
964 operations (E.g. rename). This might not work on all operating
965 systems or have any specific effect, but usually it makes sure that
966 directory changes get written to disc. It works for anything that
967 can be opened for read-only, not just directories.
968 Future versions of this function might fall back to other methods
970 when "fsync" on the directory fails (such as calling "sync").
971 Passes 0 when everything went ok, and "-1" on error.
973 aio_msync $scalar, $offset = 0, $length = undef, flags = MS_SYNC,
975 $callback->($status)
976 This is a rather advanced IO::AIO call, which only works on
977 mmap(2)ed scalars (see the "IO::AIO::mmap" function, although it
978 also works on data scalars managed by the Sys::Mmap or Mmap
979 modules, note that the scalar must only be modified in-place while
980 an aio operation is pending on it).
981 It calls the "msync" function of your OS, if available, with the
983 memory area starting at $offset in the string and ending $length
984 bytes later. If $length is negative, counts from the end, and if
985 $length is "undef", then it goes till the end of the string. The
986 flags can be either "IO::AIO::MS_ASYNC" or "IO::AIO::MS_SYNC", plus
987 an optional "IO::AIO::MS_INVALIDATE".
988 aio_mtouch $scalar, $offset = 0, $length = undef, flags = 0,
990 $callback->($status)
991 This is a rather advanced IO::AIO call, which works best on
992 mmap(2)ed scalars.
993 It touches (reads or writes) all memory pages in the specified
995 range inside the scalar. All caveats and parameters are the same as
996 for "aio_msync", above, except for flags, which must be either 0
997 (which reads all pages and ensures they are instantiated) or
998 "IO::AIO::MT_MODIFY", which modifies the memory pages (by reading
999 and writing an octet from it, which dirties the page).
1000 aio_mlock $scalar, $offset = 0, $length = undef, $callback->($status)
1002 This is a rather advanced IO::AIO call, which works best on
1003 mmap(2)ed scalars.
1004 It reads in all the pages of the underlying storage into memory (if
1006 any) and locks them, so they are not getting swapped/paged out or
1007 removed.
1008 If $length is undefined, then the scalar will be locked till the
1010 end.
1011 On systems that do not implement "mlock", this function returns
1013 "-1" and sets errno to "ENOSYS".
1014 Note that the corresponding "munlock" is synchronous and is
1016 documented under "MISCELLANEOUS FUNCTIONS".
1017 Example: open a file, mmap and mlock it - both will be undone when
1019 $data gets destroyed.
1020 open my $fh, "<", $path or die "$path: $!";
1022 my $data;
1023 IO::AIO::mmap $data, -s $fh, IO::AIO::PROT_READ, IO::AIO::MAP_SHARED, $fh;
1024 aio_mlock $data; # mlock in background
1025 aio_mlockall $flags, $callback->($status)
1027 Calls the "mlockall" function with the given $flags (a combination
1028 of "IO::AIO::MCL_CURRENT" and "IO::AIO::MCL_FUTURE").
1029 On systems that do not implement "mlockall", this function returns
1031 "-1" and sets errno to "ENOSYS".
1032 Note that the corresponding "munlockall" is synchronous and is
1034 documented under "MISCELLANEOUS FUNCTIONS".
1035 Example: asynchronously lock all current and future pages into
1037 memory.
1038 aio_mlockall IO::AIO::MCL_FUTURE;
1040 aio_fiemap $fh, $start, $length, $flags, $count, $cb->(\@extents)
1042 Queries the extents of the given file (by calling the Linux
1043 "FIEMAP" ioctl, see <http://cvs.schmorp.de/IO-AIO/doc/fiemap.txt>
1044 for details). If the ioctl is not available on your OS, then this
1045 request will fail with "ENOSYS".
1046 $start is the starting offset to query extents for, $length is the
1048 size of the range to query - if it is "undef", then the whole file
1049 will be queried.
1050 $flags is a combination of flags ("IO::AIO::FIEMAP_FLAG_SYNC" or
1052 "IO::AIO::FIEMAP_FLAG_XATTR" - "IO::AIO::FIEMAP_FLAGS_COMPAT" is
1053 also exported), and is normally 0 or "IO::AIO::FIEMAP_FLAG_SYNC" to
1054 query the data portion.
1055 $count is the maximum number of extent records to return. If it is
1057 "undef", then IO::AIO queries all extents of the range. As a very
1058 special case, if it is 0, then the callback receives the number of
1059 extents instead of the extents themselves (which is unreliable, see
1060 below).
1061 If an error occurs, the callback receives no arguments. The special
1063 "errno" value "IO::AIO::EBADR" is available to test for flag
1064 errors.
1065 Otherwise, the callback receives an array reference with extent
1067 structures. Each extent structure is an array reference itself,
1068 with the following members:
1069 [$logical, $physical, $length, $flags]
1071 Flags is any combination of the following flag values (typically
1073 either 0 or "IO::AIO::FIEMAP_EXTENT_LAST" (1)):
1074 "IO::AIO::FIEMAP_EXTENT_LAST", "IO::AIO::FIEMAP_EXTENT_UNKNOWN",
1076 "IO::AIO::FIEMAP_EXTENT_DELALLOC",
1077 "IO::AIO::FIEMAP_EXTENT_ENCODED",
1078 "IO::AIO::FIEMAP_EXTENT_DATA_ENCRYPTED",
1079 "IO::AIO::FIEMAP_EXTENT_NOT_ALIGNED",
1080 "IO::AIO::FIEMAP_EXTENT_DATA_INLINE",
1081 "IO::AIO::FIEMAP_EXTENT_DATA_TAIL",
1082 "IO::AIO::FIEMAP_EXTENT_UNWRITTEN", "IO::AIO::FIEMAP_EXTENT_MERGED"
1083 or "IO::AIO::FIEMAP_EXTENT_SHARED".
1084 At the time of this writing (Linux 3.2), this request is unreliable
1086 unless $count is "undef", as the kernel has all sorts of bugs
1087 preventing it to return all extents of a range for files with a
1088 large number of extents. The code (only) works around all these
1089 issues if $count is "undef".
1090 aio_group $callback->(...)
1092 This is a very special aio request: Instead of doing something, it
1093 is a container for other aio requests, which is useful if you want
1094 to bundle many requests into a single, composite, request with a
1095 definite callback and the ability to cancel the whole request with
1096 its subrequests.
1097 Returns an object of class IO::AIO::GRP. See its documentation
1099 below for more info.
1100 Example:
1102 my $grp = aio_group sub {
1104 print "all stats done\n";
1105 };
1106 add $grp
1108 (aio_stat ...),
1109 (aio_stat ...),
1110 ...;
1111 aio_nop $callback->()
1113 This is a special request - it does nothing in itself and is only
1114 used for side effects, such as when you want to add a dummy request
1115 to a group so that finishing the requests in the group depends on
1116 executing the given code.
1117 While this request does nothing, it still goes through the
1119 execution phase and still requires a worker thread. Thus, the
1120 callback will not be executed immediately but only after other
1121 requests in the queue have entered their execution phase. This can
1122 be used to measure request latency.
1123 IO::AIO::aio_busy $fractional_seconds, $callback->() *NOT EXPORTED*
1125 Mainly used for debugging and benchmarking, this aio request puts
1126 one of the request workers to sleep for the given time.
1127 While it is theoretically handy to have simple I/O scheduling
1129 requests like sleep and file handle readable/writable, the overhead
1130 this creates is immense (it blocks a thread for a long time) so do
1131 not use this function except to put your application under
1132 artificial I/O pressure.
1133 IO::AIO::WD - multiple working directories
1135 Your process only has one current working directory, which is used by
1136 all threads. This makes it hard to use relative paths (some other
1137 component could call "chdir" at any time, and it is hard to control
1138 when the path will be used by IO::AIO).
1139 One solution for this is to always use absolute paths. This usually
1141 works, but can be quite slow (the kernel has to walk the whole path on
1142 every access), and can also be a hassle to implement.
1143 Newer POSIX systems have a number of functions (openat, fdopendir,
1145 futimensat and so on) that make it possible to specify working
1146 directories per operation.
1147 For portability, and because the clowns who "designed", or shall I
1149 write, perpetrated this new interface were obviously half-drunk, this
1150 abstraction cannot be perfect, though.
1151 IO::AIO allows you to convert directory paths into a so-called
1153 IO::AIO::WD object. This object stores the canonicalised, absolute
1154 version of the path, and on systems that allow it, also a directory
1155 file descriptor.
1156 Everywhere where a pathname is accepted by IO::AIO (e.g. in "aio_stat"
1158 or "aio_unlink"), one can specify an array reference with an
1159 IO::AIO::WD object and a pathname instead (or the IO::AIO::WD object
1160 alone, which gets interpreted as "[$wd, "."]"). If the pathname is
1161 absolute, the IO::AIO::WD object is ignored, otherwise the pathname is
1162 resolved relative to that IO::AIO::WD object.
1163 For example, to get a wd object for /etc and then stat passwd inside,
1165 you would write:
1166 aio_wd "/etc", sub {
1168 my $etcdir = shift;
1169 # although $etcdir can be undef on error, there is generally no reason
1171 # to check for errors here, as aio_stat will fail with ENOENT
1172 # when $etcdir is undef.
1173 aio_stat [$etcdir, "passwd"], sub {
1175 # yay
1176 };
1177 };
1178 The fact that "aio_wd" is a request and not a normal function shows
1180 that creating an IO::AIO::WD object is itself a potentially blocking
1181 operation, which is why it is done asynchronously.
1182 To stat the directory obtained with "aio_wd" above, one could write
1184 either of the following three request calls:
1185 aio_lstat "/etc" , sub { ... # pathname as normal string
1187 aio_lstat [$wd, "."], sub { ... # "." relative to $wd (i.e. $wd itself)
1188 aio_lstat $wd , sub { ... # shorthand for the previous
1189 As with normal pathnames, IO::AIO keeps a copy of the working directory
1191 object and the pathname string, so you could write the following
1192 without causing any issues due to $path getting reused:
1193 my $path = [$wd, undef];
1195 for my $name (qw(abc def ghi)) {
1197 $path->[1] = $name;
1198 aio_stat $path, sub {
1199 # ...
1200 };
1201 }
1202 There are some caveats: when directories get renamed (or deleted), the
1204 pathname string doesn't change, so will point to the new directory (or
1205 nowhere at all), while the directory fd, if available on the system,
1206 will still point to the original directory. Most functions accepting a
1207 pathname will use the directory fd on newer systems, and the string on
1208 older systems. Some functions (such as "aio_realpath") will always rely
1209 on the string form of the pathname.
1210 So this functionality is mainly useful to get some protection against
1212 "chdir", to easily get an absolute path out of a relative path for
1213 future reference, and to speed up doing many operations in the same
1214 directory (e.g. when stat'ing all files in a directory).
1215 The following functions implement this working directory abstraction:
1217 aio_wd $pathname, $callback->($wd)
1219 Asynchonously canonicalise the given pathname and convert it to an
1220 IO::AIO::WD object representing it. If possible and supported on
1221 the system, also open a directory fd to speed up pathname
1222 resolution relative to this working directory.
1223 If something goes wrong, then "undef" is passwd to the callback
1225 instead of a working directory object and $! is set appropriately.
1226 Since passing "undef" as working directory component of a pathname
1227 fails the request with "ENOENT", there is often no need for error
1228 checking in the "aio_wd" callback, as future requests using the
1229 value will fail in the expected way.
1230 IO::AIO::CWD
1232 This is a compiletime constant (object) that represents the process
1233 current working directory.
1234 Specifying this object as working directory object for a pathname
1236 is as if the pathname would be specified directly, without a
1237 directory object. For example, these calls are functionally
1238 identical:
1239 aio_stat "somefile", sub { ... };
1241 aio_stat [IO::AIO::CWD, "somefile"], sub { ... };
1242 To recover the path associated with an IO::AIO::WD object, you can use
1244 "aio_realpath":
1245 aio_realpath $wd, sub {
1247 warn "path is $_[0]\n";
1248 };
1249 Currently, "aio_statvfs" always, and "aio_rename" and "aio_rmdir"
1251 sometimes, fall back to using an absolue path.
1252 IO::AIO::REQ CLASS
1254 All non-aggregate "aio_*" functions return an object of this class when
1255 called in non-void context.
1256 cancel $req
1258 Cancels the request, if possible. Has the effect of skipping
1259 execution when entering the execute state and skipping calling the
1260 callback when entering the the result state, but will leave the
1261 request otherwise untouched (with the exception of readdir). That
1262 means that requests that currently execute will not be stopped and
1263 resources held by the request will not be freed prematurely.
1264 cb $req $callback->(...)
1266 Replace (or simply set) the callback registered to the request.
1267 IO::AIO::GRP CLASS
1269 This class is a subclass of IO::AIO::REQ, so all its methods apply to
1270 objects of this class, too.
1271 A IO::AIO::GRP object is a special request that can contain multiple
1273 other aio requests.
1274 You create one by calling the "aio_group" constructing function with a
1276 callback that will be called when all contained requests have entered
1277 the "done" state:
1278 my $grp = aio_group sub {
1280 print "all requests are done\n";
1281 };
1282 You add requests by calling the "add" method with one or more
1284 "IO::AIO::REQ" objects:
1285 $grp->add (aio_unlink "...");
1287 add $grp aio_stat "...", sub {
1289 $_[0] or return $grp->result ("error");
1290 # add another request dynamically, if first succeeded
1292 add $grp aio_open "...", sub {
1293 $grp->result ("ok");
1294 };
1295 };
1296 This makes it very easy to create composite requests (see the source of
1298 "aio_move" for an application) that work and feel like simple requests.
1299 · The IO::AIO::GRP objects will be cleaned up during calls to
1301 "IO::AIO::poll_cb", just like any other request.
1302 · They can be canceled like any other request. Canceling will cancel
1304 not only the request itself, but also all requests it contains.
1305 · They can also can also be added to other IO::AIO::GRP objects.
1307 · You must not add requests to a group from within the group callback
1309 (or any later time).
1310 Their lifetime, simplified, looks like this: when they are empty, they
1312 will finish very quickly. If they contain only requests that are in the
1313 "done" state, they will also finish. Otherwise they will continue to
1314 exist.
1315 That means after creating a group you have some time to add requests
1317 (precisely before the callback has been invoked, which is only done
1318 within the "poll_cb"). And in the callbacks of those requests, you can
1319 add further requests to the group. And only when all those requests
1320 have finished will the the group itself finish.
1321 add $grp ...
1323 $grp->add (...)
1324 Add one or more requests to the group. Any type of IO::AIO::REQ can
1325 be added, including other groups, as long as you do not create
1326 circular dependencies.
1327 Returns all its arguments.
1329 $grp->cancel_subs
1331 Cancel all subrequests and clears any feeder, but not the group
1332 request itself. Useful when you queued a lot of events but got a
1333 result early.
1334 The group request will finish normally (you cannot add requests to
1336 the group).
1337 $grp->result (...)
1339 Set the result value(s) that will be passed to the group callback
1340 when all subrequests have finished and set the groups errno to the
1341 current value of errno (just like calling "errno" without an error
1342 number). By default, no argument will be passed and errno is zero.
1343 $grp->errno ([$errno])
1345 Sets the group errno value to $errno, or the current value of errno
1346 when the argument is missing.
1347 Every aio request has an associated errno value that is restored
1349 when the callback is invoked. This method lets you change this
1350 value from its default (0).
1351 Calling "result" will also set errno, so make sure you either set
1353 $! before the call to "result", or call c<errno> after it.
1354 feed $grp $callback->($grp)
1356 Sets a feeder/generator on this group: every group can have an
1357 attached generator that generates requests if idle. The idea behind
1358 this is that, although you could just queue as many requests as you
1359 want in a group, this might starve other requests for a potentially
1360 long time. For example, "aio_scandir" might generate hundreds of
1361 thousands of "aio_stat" requests, delaying any later requests for a
1362 long time.
1363 To avoid this, and allow incremental generation of requests, you
1365 can instead a group and set a feeder on it that generates those
1366 requests. The feed callback will be called whenever there are few
1367 enough (see "limit", below) requests active in the group itself and
1368 is expected to queue more requests.
1369 The feed callback can queue as many requests as it likes (i.e.
1371 "add" does not impose any limits).
1372 If the feed does not queue more requests when called, it will be
1374 automatically removed from the group.
1375 If the feed limit is 0 when this method is called, it will be set
1377 to 2 automatically.
1378 Example:
1380 # stat all files in @files, but only ever use four aio requests concurrently:
1382 my $grp = aio_group sub { print "finished\n" };
1384 limit $grp 4;
1385 feed $grp sub {
1386 my $file = pop @files
1387 or return;
1388 add $grp aio_stat $file, sub { ... };
1390 };
1391 limit $grp $num
1393 Sets the feeder limit for the group: The feeder will be called
1394 whenever the group contains less than this many requests.
1395 Setting the limit to 0 will pause the feeding process.
1397 The default value for the limit is 0, but note that setting a
1399 feeder automatically bumps it up to 2.
1400 SUPPORT FUNCTIONS
1402 EVENT PROCESSING AND EVENT LOOP INTEGRATION
1403 $fileno = IO::AIO::poll_fileno
1405 Return the request result pipe file descriptor. This filehandle
1406 must be polled for reading by some mechanism outside this module
1407 (e.g. EV, Glib, select and so on, see below or the SYNOPSIS). If
1408 the pipe becomes readable you have to call "poll_cb" to check the
1409 results.
1410 See "poll_cb" for an example.
1412 IO::AIO::poll_cb
1414 Process some requests that have reached the result phase (i.e. they
1415 have been executed but the results are not yet reported). You have
1416 to call this "regularly" to finish outstanding requests.
1417 Returns 0 if all events could be processed (or there were no events
1419 to process), or "-1" if it returned earlier for whatever reason.
1420 Returns immediately when no events are outstanding. The amount of
1421 events processed depends on the settings of
1422 "IO::AIO::max_poll_req", "IO::AIO::max_poll_time" and
1423 "IO::AIO::max_outstanding".
1424 If not all requests were processed for whatever reason, the poll
1426 file descriptor will still be ready when "poll_cb" returns, so
1427 normally you don't have to do anything special to have it called
1428 later.
1429 Apart from calling "IO::AIO::poll_cb" when the event filehandle
1431 becomes ready, it can be beneficial to call this function from
1432 loops which submit a lot of requests, to make sure the results get
1433 processed when they become available and not just when the loop is
1434 finished and the event loop takes over again. This function returns
1435 very fast when there are no outstanding requests.
1436 Example: Install an Event watcher that automatically calls
1438 IO::AIO::poll_cb with high priority (more examples can be found in
1439 the SYNOPSIS section, at the top of this document):
1440 Event->io (fd => IO::AIO::poll_fileno,
1442 poll => 'r', async => 1,
1443 cb => \&IO::AIO::poll_cb);
1444 IO::AIO::poll_wait
1446 Wait until either at least one request is in the result phase or no
1447 requests are outstanding anymore.
1448 This is useful if you want to synchronously wait for some requests
1450 to become ready, without actually handling them.
1451 See "nreqs" for an example.
1453 IO::AIO::poll
1455 Waits until some requests have been handled.
1456 Returns the number of requests processed, but is otherwise strictly
1458 equivalent to:
1459 IO::AIO::poll_wait, IO::AIO::poll_cb
1461 IO::AIO::flush
1463 Wait till all outstanding AIO requests have been handled.
1464 Strictly equivalent to:
1466 IO::AIO::poll_wait, IO::AIO::poll_cb
1468 while IO::AIO::nreqs;
1469 This function can be useful at program aborts, to make sure
1471 outstanding I/O has been done ("IO::AIO" uses an "END" block which
1472 already calls this function on normal exits), or when you are
1473 merely using "IO::AIO" for its more advanced functions, rather than
1474 for async I/O, e.g.:
1475 my ($dirs, $nondirs);
1477 IO::AIO::aio_scandir "/tmp", 0, sub { ($dirs, $nondirs) = @_ };
1478 IO::AIO::flush;
1479 # $dirs, $nondirs are now set
1480 IO::AIO::max_poll_reqs $nreqs
1482 IO::AIO::max_poll_time $seconds
1483 These set the maximum number of requests (default 0, meaning
1484 infinity) that are being processed by "IO::AIO::poll_cb" in one
1485 call, respectively the maximum amount of time (default 0, meaning
1486 infinity) spent in "IO::AIO::poll_cb" to process requests (more
1487 correctly the mininum amount of time "poll_cb" is allowed to use).
1488 Setting "max_poll_time" to a non-zero value creates an overhead of
1490 one syscall per request processed, which is not normally a problem
1491 unless your callbacks are really really fast or your OS is really
1492 really slow (I am not mentioning Solaris here). Using
1493 "max_poll_reqs" incurs no overhead.
1494 Setting these is useful if you want to ensure some level of
1496 interactiveness when perl is not fast enough to process all
1497 requests in time.
1498 For interactive programs, values such as 0.01 to 0.1 should be
1500 fine.
1501 Example: Install an Event watcher that automatically calls
1503 IO::AIO::poll_cb with low priority, to ensure that other parts of
1504 the program get the CPU sometimes even under high AIO load.
1505 # try not to spend much more than 0.1s in poll_cb
1507 IO::AIO::max_poll_time 0.1;
1508 # use a low priority so other tasks have priority
1510 Event->io (fd => IO::AIO::poll_fileno,
1511 poll => 'r', nice => 1,
1512 cb => &IO::AIO::poll_cb);
1513 CONTROLLING THE NUMBER OF THREADS
1515 IO::AIO::min_parallel $nthreads
1517 Set the minimum number of AIO threads to $nthreads. The current
1518 default is 8, which means eight asynchronous operations can execute
1519 concurrently at any one time (the number of outstanding requests,
1520 however, is unlimited).
1521 IO::AIO starts threads only on demand, when an AIO request is
1523 queued and no free thread exists. Please note that queueing up a
1524 hundred requests can create demand for a hundred threads, even if
1525 it turns out that everything is in the cache and could have been
1526 processed faster by a single thread.
1527 It is recommended to keep the number of threads relatively low, as
1529 some Linux kernel versions will scale negatively with the number of
1530 threads (higher parallelity => MUCH higher latency). With current
1531 Linux 2.6 versions, 4-32 threads should be fine.
1532 Under most circumstances you don't need to call this function, as
1534 the module selects a default that is suitable for low to moderate
1535 load.
1536 IO::AIO::max_parallel $nthreads
1538 Sets the maximum number of AIO threads to $nthreads. If more than
1539 the specified number of threads are currently running, this
1540 function kills them. This function blocks until the limit is
1541 reached.
1542 While $nthreads are zero, aio requests get queued but not executed
1544 until the number of threads has been increased again.
1545 This module automatically runs "max_parallel 0" at program end, to
1547 ensure that all threads are killed and that there are no
1548 outstanding requests.
1549 Under normal circumstances you don't need to call this function.
1551 IO::AIO::max_idle $nthreads
1553 Limit the number of threads (default: 4) that are allowed to idle
1554 (i.e., threads that did not get a request to process within the
1555 idle timeout (default: 10 seconds). That means if a thread becomes
1556 idle while $nthreads other threads are also idle, it will free its
1557 resources and exit.
1558 This is useful when you allow a large number of threads (e.g. 100
1560 or 1000) to allow for extremely high load situations, but want to
1561 free resources under normal circumstances (1000 threads can easily
1562 consume 30MB of RAM).
1563 The default is probably ok in most situations, especially if thread
1565 creation is fast. If thread creation is very slow on your system
1566 you might want to use larger values.
1567 IO::AIO::idle_timeout $seconds
1569 Sets the minimum idle timeout (default 10) after which worker
1570 threads are allowed to exit. SEe "IO::AIO::max_idle".
1571 IO::AIO::max_outstanding $maxreqs
1573 Sets the maximum number of outstanding requests to $nreqs. If you
1574 do queue up more than this number of requests, the next call to
1575 "IO::AIO::poll_cb" (and other functions calling "poll_cb", such as
1576 "IO::AIO::flush" or "IO::AIO::poll") will block until the limit is
1577 no longer exceeded.
1578 In other words, this setting does not enforce a queue limit, but
1580 can be used to make poll functions block if the limit is exceeded.
1581 This is a very bad function to use in interactive programs because
1583 it blocks, and a bad way to reduce concurrency because it is
1584 inexact: Better use an "aio_group" together with a feed callback.
1585 Its main use is in scripts without an event loop - when you want to
1587 stat a lot of files, you can write something like this:
1588 IO::AIO::max_outstanding 32;
1590 for my $path (...) {
1592 aio_stat $path , ...;
1593 IO::AIO::poll_cb;
1594 }
1595 IO::AIO::flush;
1597 The call to "poll_cb" inside the loop will normally return
1599 instantly, but as soon as more thna 32 reqeusts are in-flight, it
1600 will block until some requests have been handled. This keeps the
1601 loop from pushing a large number of "aio_stat" requests onto the
1602 queue.
1603 The default value for "max_outstanding" is very large, so there is
1605 no practical limit on the number of outstanding requests.
1606 STATISTICAL INFORMATION
1608 IO::AIO::nreqs
1610 Returns the number of requests currently in the ready, execute or
1611 pending states (i.e. for which their callback has not been invoked
1612 yet).
1613 Example: wait till there are no outstanding requests anymore:
1615 IO::AIO::poll_wait, IO::AIO::poll_cb
1617 while IO::AIO::nreqs;
1618 IO::AIO::nready
1620 Returns the number of requests currently in the ready state (not
1621 yet executed).
1622 IO::AIO::npending
1624 Returns the number of requests currently in the pending state
1625 (executed, but not yet processed by poll_cb).
1626 SUBSECOND STAT TIME ACCESS
1628 Both "aio_stat"/"aio_lstat" and perl's "stat"/"lstat" functions can
1630 generally find access/modification and change times with subsecond time
1631 accuracy of the system supports it, but perl's built-in functions only
1632 return the integer part.
1633 The following functions return the timestamps of the most recent stat
1635 with subsecond precision on most systems and work both after
1636 "aio_stat"/"aio_lstat" and perl's "stat"/"lstat" calls. Their return
1637 value is only meaningful after a successful "stat"/"lstat" call, or
1638 during/after a successful "aio_stat"/"aio_lstat" callback.
1639 This is similar to the Time::HiRes "stat" functions, but can return
1641 full resolution without rounding and work with standard perl "stat",
1642 alleviating the need to call the special "Time::HiRes" functions, which
1643 do not act like their perl counterparts.
1644 On operating systems or file systems where subsecond time resolution is
1646 not supported or could not be detected, a fractional part of 0 is
1647 returned, so it is always safe to call these functions.
1648 $seconds = IO::AIO::st_atime, IO::AIO::st_mtime, IO::AIO::st_ctime,
1650 IO::AIO::st_btime
1651 Return the access, modication, change or birth time, respectively,
1652 including fractional part. Due to the limited precision of floating
1653 point, the accuracy on most platforms is only a bit better than
1654 milliseconds for times around now - see the nsec function family,
1655 below, for full accuracy.
1656 File birth time is only available when the OS and perl support it
1658 (on FreeBSD and NetBSD at the time of this writing, although
1659 support is adaptive, so if your OS/perl gains support, IO::AIO can
1660 take avdantage of it). On systems where it isn't available, 0 is
1661 currently returned, but this might change to "undef" in a future
1662 version.
1663 ($atime, $mtime, $ctime, $btime, ...) = IO::AIO::st_xtime
1665 Returns access, modification, change and birth time all in one go,
1666 and maybe more times in the future version.
1667 $nanoseconds = IO::AIO::st_atimensec, IO::AIO::st_mtimensec,
1669 IO::AIO::st_ctimensec, IO::AIO::st_btimensec
1670 Return the fractional access, modifcation, change or birth time, in
1671 nanoseconds, as an integer in the range 0 to 999999999.
1672 Note that no accessors are provided for access, modification and
1674 change times - you need to get those from "stat _" if required
1675 ("int IO::AIO::st_atime" and so on will not generally give you the
1676 correct value).
1677 $seconds = IO::AIO::st_btimesec
1679 The (integral) seconds part of the file birth time, if available.
1680 ($atime, $mtime, $ctime, $btime, ...) = IO::AIO::st_xtimensec
1682 Like the functions above, but returns all four times in one go (and
1683 maybe more in future versions).
1684 $counter = IO::AIO::st_gen
1686 Returns the generation counter of the file. This is only available
1687 on platforms which have this member in their "struct stat" (most
1688 BSDs at the time of this writing) and generally only to the root
1689 usert. If unsupported, 0 is returned, but this might change to
1690 "undef" in a future version.
1691 Example: print the high resolution modification time of /etc, using
1693 "stat", and "IO::AIO::aio_stat".
1694 if (stat "/etc") {
1696 printf "stat(/etc) mtime: %f\n", IO::AIO::st_mtime;
1697 }
1698 IO::AIO::aio_stat "/etc", sub {
1700 $_[0]
1701 and return;
1702 printf "aio_stat(/etc) mtime: %d.%09d\n", (stat _)[9], IO::AIO::st_mtimensec;
1704 };
1705 IO::AIO::flush;
1707 Output of the awbove on my system, showing reduced and full accuracy:
1709 stat(/etc) mtime: 1534043702.020808
1711 aio_stat(/etc) mtime: 1534043702.020807792
1712 MISCELLANEOUS FUNCTIONS
1714 IO::AIO implements some functions that are useful when you want to use
1716 some "Advanced I/O" function not available to in Perl, without going
1717 the "Asynchronous I/O" route. Many of these have an asynchronous
1718 "aio_*" counterpart.
1719 $numfd = IO::AIO::get_fdlimit
1721 This function is EXPERIMENTAL and subject to change.
1722 Tries to find the current file descriptor limit and returns it, or
1724 "undef" and sets $! in case of an error. The limit is one larger
1725 than the highest valid file descriptor number.
1726 IO::AIO::min_fdlimit [$numfd]
1728 This function is EXPERIMENTAL and subject to change.
1729 Try to increase the current file descriptor limit(s) to at least
1731 $numfd by changing the soft or hard file descriptor resource limit.
1732 If $numfd is missing, it will try to set a very high limit,
1733 although this is not recommended when you know the actual minimum
1734 that you require.
1735 If the limit cannot be raised enough, the function makes a best-
1737 effort attempt to increase the limit as much as possible, using
1738 various tricks, while still failing. You can query the resulting
1739 limit using "IO::AIO::get_fdlimit".
1740 If an error occurs, returns "undef" and sets $!, otherwise returns
1742 true.
1743 IO::AIO::sendfile $ofh, $ifh, $offset, $count
1745 Calls the "eio_sendfile_sync" function, which is like
1746 "aio_sendfile", but is blocking (this makes most sense if you know
1747 the input data is likely cached already and the output filehandle
1748 is set to non-blocking operations).
1749 Returns the number of bytes copied, or "-1" on error.
1751 IO::AIO::fadvise $fh, $offset, $len, $advice
1753 Simply calls the "posix_fadvise" function (see its manpage for
1754 details). The following advice constants are available:
1755 "IO::AIO::FADV_NORMAL", "IO::AIO::FADV_SEQUENTIAL",
1756 "IO::AIO::FADV_RANDOM", "IO::AIO::FADV_NOREUSE",
1757 "IO::AIO::FADV_WILLNEED", "IO::AIO::FADV_DONTNEED".
1758 On systems that do not implement "posix_fadvise", this function
1760 returns ENOSYS, otherwise the return value of "posix_fadvise".
1761 IO::AIO::madvise $scalar, $offset, $len, $advice
1763 Simply calls the "posix_madvise" function (see its manpage for
1764 details). The following advice constants are available:
1765 "IO::AIO::MADV_NORMAL", "IO::AIO::MADV_SEQUENTIAL",
1766 "IO::AIO::MADV_RANDOM", "IO::AIO::MADV_WILLNEED",
1767 "IO::AIO::MADV_DONTNEED".
1768 If $offset is negative, counts from the end. If $length is
1770 negative, the remaining length of the $scalar is used. If possible,
1771 $length will be reduced to fit into the $scalar.
1772 On systems that do not implement "posix_madvise", this function
1774 returns ENOSYS, otherwise the return value of "posix_madvise".
1775 IO::AIO::mprotect $scalar, $offset, $len, $protect
1777 Simply calls the "mprotect" function on the preferably AIO::mmap'ed
1778 $scalar (see its manpage for details). The following protect
1779 constants are available: "IO::AIO::PROT_NONE",
1780 "IO::AIO::PROT_READ", "IO::AIO::PROT_WRITE", "IO::AIO::PROT_EXEC".
1781 If $offset is negative, counts from the end. If $length is
1783 negative, the remaining length of the $scalar is used. If possible,
1784 $length will be reduced to fit into the $scalar.
1785 On systems that do not implement "mprotect", this function returns
1787 ENOSYS, otherwise the return value of "mprotect".
1788 IO::AIO::mmap $scalar, $length, $prot, $flags, $fh[, $offset]
1790 Memory-maps a file (or anonymous memory range) and attaches it to
1791 the given $scalar, which will act like a string scalar. Returns
1792 true on success, and false otherwise.
1793 The scalar must exist, but its contents do not matter - this means
1795 you cannot use a nonexistant array or hash element. When in doubt,
1796 "undef" the scalar first.
1797 The only operations allowed on the mmapped scalar are
1799 "substr"/"vec", which don't change the string length, and most
1800 read-only operations such as copying it or searching it with
1801 regexes and so on.
1802 Anything else is unsafe and will, at best, result in memory leaks.
1804 The memory map associated with the $scalar is automatically removed
1806 when the $scalar is undef'd or destroyed, or when the
1807 "IO::AIO::mmap" or "IO::AIO::munmap" functions are called on it.
1808 This calls the "mmap"(2) function internally. See your system's
1810 manual page for details on the $length, $prot and $flags
1811 parameters.
1812 The $length must be larger than zero and smaller than the actual
1814 filesize.
1815 $prot is a combination of "IO::AIO::PROT_NONE",
1817 "IO::AIO::PROT_EXEC", "IO::AIO::PROT_READ" and/or
1818 "IO::AIO::PROT_WRITE",
1819 $flags can be a combination of "IO::AIO::MAP_SHARED" or
1821 "IO::AIO::MAP_PRIVATE", or a number of system-specific flags (when
1822 not available, the are 0): "IO::AIO::MAP_ANONYMOUS" (which is set
1823 to "MAP_ANON" if your system only provides this constant),
1824 "IO::AIO::MAP_LOCKED", "IO::AIO::MAP_NORESERVE",
1825 "IO::AIO::MAP_POPULATE", "IO::AIO::MAP_NONBLOCK",
1826 "IO::AIO::MAP_FIXED", "IO::AIO::MAP_GROWSDOWN",
1827 "IO::AIO::MAP_32BIT", "IO::AIO::MAP_HUGETLB" or
1828 "IO::AIO::MAP_STACK".
1829 If $fh is "undef", then a file descriptor of "-1" is passed.
1831 $offset is the offset from the start of the file - it generally
1833 must be a multiple of "IO::AIO::PAGESIZE" and defaults to 0.
1834 Example:
1836 use Digest::MD5;
1838 use IO::AIO;
1839 open my $fh, "<verybigfile"
1841 or die "$!";
1842 IO::AIO::mmap my $data, -s $fh, IO::AIO::PROT_READ, IO::AIO::MAP_SHARED, $fh
1844 or die "verybigfile: $!";
1845 my $fast_md5 = md5 $data;
1847 IO::AIO::munmap $scalar
1849 Removes a previous mmap and undefines the $scalar.
1850 IO::AIO::mremap $scalar, $new_length, $flags = MREMAP_MAYMOVE[,
1852 $new_address = 0]
1853 Calls the Linux-specific mremap(2) system call. The $scalar must
1854 have been mapped by "IO::AIO::mmap", and $flags must currently
1855 either be 0 or "IO::AIO::MREMAP_MAYMOVE".
1856 Returns true if successful, and false otherwise. If the underlying
1858 mmapped region has changed address, then the true value has the
1859 numerical value 1, otherwise it has the numerical value 0:
1860 my $success = IO::AIO::mremap $mmapped, 8192, IO::AIO::MREMAP_MAYMOVE
1862 or die "mremap: $!";
1863 if ($success*1) {
1865 warn "scalar has chanegd address in memory\n";
1866 }
1867 "IO::AIO::MREMAP_FIXED" and the $new_address argument are currently
1869 implemented, but not supported and might go away in a future
1870 version.
1871 On systems where this call is not supported or is not emulated,
1873 this call returns falls and sets $! to "ENOSYS".
1874 IO::AIO::munlock $scalar, $offset = 0, $length = undef
1876 Calls the "munlock" function, undoing the effects of a previous
1877 "aio_mlock" call (see its description for details).
1878 IO::AIO::munlockall
1880 Calls the "munlockall" function.
1881 On systems that do not implement "munlockall", this function
1883 returns ENOSYS, otherwise the return value of "munlockall".
1884 IO::AIO::splice $r_fh, $r_off, $w_fh, $w_off, $length, $flags
1886 Calls the GNU/Linux splice(2) syscall, if available. If $r_off or
1887 $w_off are "undef", then "NULL" is passed for these, otherwise they
1888 should be the file offset.
1889 $r_fh and $w_fh should not refer to the same file, as splice might
1891 silently corrupt the data in this case.
1892 The following symbol flag values are available:
1894 "IO::AIO::SPLICE_F_MOVE", "IO::AIO::SPLICE_F_NONBLOCK",
1895 "IO::AIO::SPLICE_F_MORE" and "IO::AIO::SPLICE_F_GIFT".
1896 See the splice(2) manpage for details.
1898 IO::AIO::tee $r_fh, $w_fh, $length, $flags
1900 Calls the GNU/Linux tee(2) syscall, see its manpage and the
1901 description for "IO::AIO::splice" above for details.
1902 $actual_size = IO::AIO::pipesize $r_fh[, $new_size]
1904 Attempts to query or change the pipe buffer size. Obviously works
1905 only on pipes, and currently works only on GNU/Linux systems, and
1906 fails with "-1"/"ENOSYS" everywhere else. If anybody knows how to
1907 influence pipe buffer size on other systems, drop me a note.
1908 ($rfh, $wfh) = IO::AIO::pipe2 [$flags]
1910 This is a direct interface to the Linux pipe2(2) system call. If
1911 $flags is missing or 0, then this should be the same as a call to
1912 perl's built-in "pipe" function and create a new pipe, and works on
1913 systems that lack the pipe2 syscall. On win32, this case invokes
1914 "_pipe (..., 4096, O_BINARY)".
1915 If $flags is non-zero, it tries to invoke the pipe2 system call
1917 with the given flags (Linux 2.6.27, glibc 2.9).
1918 On success, the read and write file handles are returned.
1920 On error, nothing will be returned. If the pipe2 syscall is missing
1922 and $flags is non-zero, fails with "ENOSYS".
1923 Please refer to pipe2(2) for more info on the $flags, but at the
1925 time of this writing, "IO::AIO::O_CLOEXEC", "IO::AIO::O_NONBLOCK"
1926 and "IO::AIO::O_DIRECT" (Linux 3.4, for packet-based pipes) were
1927 supported.
1928 Example: create a pipe race-free w.r.t. threads and fork:
1930 my ($rfh, $wfh) = IO::AIO::pipe2 IO::AIO::O_CLOEXEC
1932 or die "pipe2: $!\n";
1933 $fh = IO::AIO::eventfd [$initval, [$flags]]
1935 This is a direct interface to the Linux eventfd(2) system call. The
1936 (unhelpful) defaults for $initval and $flags are 0 for both.
1937 On success, the new eventfd filehandle is returned, otherwise
1939 returns "undef". If the eventfd syscall is missing, fails with
1940 "ENOSYS".
1941 Please refer to eventfd(2) for more info on this call.
1943 The following symbol flag values are available:
1945 "IO::AIO::EFD_CLOEXEC", "IO::AIO::EFD_NONBLOCK" and
1946 "IO::AIO::EFD_SEMAPHORE" (Linux 2.6.30).
1947 Example: create a new eventfd filehandle:
1949 $fh = IO::AIO::eventfd 0, IO::AIO::O_CLOEXEC
1951 or die "eventfd: $!\n";
1952 $fh = IO::AIO::timerfd_create $clockid[, $flags]
1954 This is a direct interface to the Linux timerfd_create(2) system
1955 call. The (unhelpful) default for $flags is 0.
1956 On success, the new timerfd filehandle is returned, otherwise
1958 returns "undef". If the eventfd syscall is missing, fails with
1959 "ENOSYS".
1960 Please refer to timerfd_create(2) for more info on this call.
1962 The following $clockid values are available:
1964 "IO::AIO::CLOCK_REALTIME", "IO::AIO::CLOCK_MONOTONIC"
1965 "IO::AIO::CLOCK_CLOCK_BOOTTIME" (Linux 3.15)
1966 "IO::AIO::CLOCK_CLOCK_REALTIME_ALARM" (Linux 3.11) and
1967 "IO::AIO::CLOCK_CLOCK_BOOTTIME_ALARM" (Linux 3.11).
1968 The following $flags values are available (Linux 2.6.27):
1970 "IO::AIO::TFD_NONBLOCK" and "IO::AIO::TFD_CLOEXEC".
1971 Example: create a new timerfd and set it to one-second repeated
1973 alarms, then wait for two alarms:
1974 my $fh = IO::AIO::timerfd_create IO::AIO::CLOCK_BOOTTIME, IO::AIO::TFD_CLOEXEC
1976 or die "timerfd_create: $!\n";
1977 defined IO::AIO::timerfd_settime $fh, 0, 1, 1
1979 or die "timerfd_settime: $!\n";
1980 for (1..2) {
1982 8 == sysread $fh, my $buf, 8
1983 or die "timerfd read failure\n";
1984 printf "number of expirations (likely 1): %d\n",
1986 unpack "Q", $buf;
1987 }
1988 ($cur_interval, $cur_value) = IO::AIO::timerfd_settime $fh, $flags,
1990 $new_interval, $nbw_value
1991 This is a direct interface to the Linux timerfd_settime(2) system
1992 call. Please refer to its manpage for more info on this call.
1993 The new itimerspec is specified using two (possibly fractional)
1995 second values, $new_interval and $new_value).
1996 On success, the current interval and value are returned (as per
1998 "timerfd_gettime"). On failure, the empty list is returned.
1999 The following $flags values are available:
2001 "IO::AIO::TFD_TIMER_ABSTIME" and
2002 "IO::AIO::TFD_TIMER_CANCEL_ON_SET".
2003 See "IO::AIO::timerfd_create" for a full example.
2005 ($cur_interval, $cur_value) = IO::AIO::timerfd_gettime $fh
2007 This is a direct interface to the Linux timerfd_gettime(2) system
2008 call. Please refer to its manpage for more info on this call.
2009 On success, returns the current values of interval and value for
2011 the given timerfd (as potentially fractional second values). On
2012 failure, the empty list is returned.
2013
2015 It is recommended to use AnyEvent::AIO to integrate IO::AIO
2016 automatically into many event loops:
2017 # AnyEvent integration (EV, Event, Glib, Tk, POE, urxvt, pureperl...)
2019 use AnyEvent::AIO;
2020 You can also integrate IO::AIO manually into many event loops, here are
2022 some examples of how to do this:
2023 # EV integration
2025 my $aio_w = EV::io IO::AIO::poll_fileno, EV::READ, \&IO::AIO::poll_cb;
2026 # Event integration
2028 Event->io (fd => IO::AIO::poll_fileno,
2029 poll => 'r',
2030 cb => \&IO::AIO::poll_cb);
2031 # Glib/Gtk2 integration
2033 add_watch Glib::IO IO::AIO::poll_fileno,
2034 in => sub { IO::AIO::poll_cb; 1 };
2035 # Tk integration
2037 Tk::Event::IO->fileevent (IO::AIO::poll_fileno, "",
2038 readable => \&IO::AIO::poll_cb);
2039 # Danga::Socket integration
2041 Danga::Socket->AddOtherFds (IO::AIO::poll_fileno =>
2042 \&IO::AIO::poll_cb);
2043 FORK BEHAVIOUR
2045 Usage of pthreads in a program changes the semantics of fork
2046 considerably. Specifically, only async-safe functions can be called
2047 after fork. Perl doesn't know about this, so in general, you cannot
2048 call fork with defined behaviour in perl if pthreads are involved.
2049 IO::AIO uses pthreads, so this applies, but many other extensions and
2050 (for inexplicable reasons) perl itself often is linked against
2051 pthreads, so this limitation applies to quite a lot of perls.
2052 This module no longer tries to fight your OS, or POSIX. That means
2054 IO::AIO only works in the process that loaded it. Forking is fully
2055 supported, but using IO::AIO in the child is not.
2056 You might get around by not using IO::AIO before (or after) forking.
2058 You could also try to call the IO::AIO::reinit function in the child:
2059 IO::AIO::reinit
2061 Abandons all current requests and I/O threads and simply
2062 reinitialises all data structures. This is not an operation
2063 supported by any standards, but happens to work on GNU/Linux and
2064 some newer BSD systems.
2065 The only reasonable use for this function is to call it after
2067 forking, if "IO::AIO" was used in the parent. Calling it while
2068 IO::AIO is active in the process will result in undefined
2069 behaviour. Calling it at any time will also result in any undefined
2070 (by POSIX) behaviour.
2071 LINUX-SPECIFIC CALLS
2073 When a call is documented as "linux-specific" then this means it
2074 originated on GNU/Linux. "IO::AIO" will usually try to autodetect the
2075 availability and compatibility of such calls regardless of the platform
2076 it is compiled on, so platforms such as FreeBSD which often implement
2077 these calls will work. When in doubt, call them and see if they fail
2078 wth "ENOSYS".
2079 MEMORY USAGE
2081 Per-request usage:
2082 Each aio request uses - depending on your architecture - around 100-200
2084 bytes of memory. In addition, stat requests need a stat buffer
2085 (possibly a few hundred bytes), readdir requires a result buffer and so
2086 on. Perl scalars and other data passed into aio requests will also be
2087 locked and will consume memory till the request has entered the done
2088 state.
2089 This is not awfully much, so queuing lots of requests is not usually a
2091 problem.
2092 Per-thread usage:
2094 In the execution phase, some aio requests require more memory for
2096 temporary buffers, and each thread requires a stack and other data
2097 structures (usually around 16k-128k, depending on the OS).
2098
2100 Known bugs will be fixed in the next release :)
2101
2103 Calls that try to "import" foreign memory areas (such as
2104 "IO::AIO::mmap" or "IO::AIO::aio_slurp") do not work with generic
2105 lvalues, such as non-created hash slots or other scalars I didn't think
2106 of. It's best to avoid such and either use scalar variables or making
2107 sure that the scalar exists (e.g. by storing "undef") and isn't "funny"
2108 (e.g. tied).
2109 I am not sure anything can be done about this, so this is considered a
2111 known issue, rather than a bug.
2112
2114 AnyEvent::AIO for easy integration into event loops, Coro::AIO for a
2115 more natural syntax.
2116
2118 Marc Lehmann <schmorp@schmorp.de>
2119 http://home.schmorp.de/
2120perl v5.28.0 2018-08-25 AIO(3)