1AIO(3) User Contributed Perl Documentation AIO(3)
2
6 IO::AIO - Asynchronous/Advanced Input/Output
7
9 use IO::AIO;
10 aio_open "/etc/passwd", IO::AIO::O_RDONLY, 0, sub {
12 my $fh = shift
13 or die "/etc/passwd: $!";
14 ...
15 };
16 aio_unlink "/tmp/file", sub { };
18 aio_read $fh, 30000, 1024, $buffer, 0, sub {
20 $_[0] > 0 or die "read error: $!";
21 };
22 # version 2+ has request and group objects
24 use IO::AIO 2;
25 aioreq_pri 4; # give next request a very high priority
27 my $req = aio_unlink "/tmp/file", sub { };
28 $req->cancel; # cancel request if still in queue
29 my $grp = aio_group sub { print "all stats done\n" };
31 add $grp aio_stat "..." for ...;
32
34 This module implements asynchronous I/O using whatever means your
35 operating system supports. It is implemented as an interface to
36 "libeio" (<http://software.schmorp.de/pkg/libeio.html>).
37 Asynchronous means that operations that can normally block your program
39 (e.g. reading from disk) will be done asynchronously: the operation
40 will still block, but you can do something else in the meantime. This
41 is extremely useful for programs that need to stay interactive even
42 when doing heavy I/O (GUI programs, high performance network servers
43 etc.), but can also be used to easily do operations in parallel that
44 are normally done sequentially, e.g. stat'ing many files, which is much
45 faster on a RAID volume or over NFS when you do a number of stat
46 operations concurrently.
47 While most of this works on all types of file descriptors (for example
49 sockets), using these functions on file descriptors that support
50 nonblocking operation (again, sockets, pipes etc.) is very inefficient.
51 Use an event loop for that (such as the EV module): IO::AIO will
52 naturally fit into such an event loop itself.
53 In this version, a number of threads are started that execute your
55 requests and signal their completion. You don't need thread support in
56 perl, and the threads created by this module will not be visible to
57 perl. In the future, this module might make use of the native aio
58 functions available on many operating systems. However, they are often
59 not well-supported or restricted (GNU/Linux doesn't allow them on
60 normal files currently, for example), and they would only support
61 aio_read and aio_write, so the remaining functionality would have to be
62 implemented using threads anyway.
63 In addition to asynchronous I/O, this module also exports some rather
65 arcane interfaces, such as "madvise" or linux's "splice" system call,
66 which is why the "A" in "AIO" can also mean advanced.
67 Although the module will work in the presence of other (Perl-) threads,
69 it is currently not reentrant in any way, so use appropriate locking
70 yourself, always call "poll_cb" from within the same thread, or never
71 call "poll_cb" (or other "aio_" functions) recursively.
72 EXAMPLE
74 This is a simple example that uses the EV module and loads /etc/passwd
75 asynchronously:
76 use EV;
78 use IO::AIO;
79 # register the IO::AIO callback with EV
81 my $aio_w = EV::io IO::AIO::poll_fileno, EV::READ, \&IO::AIO::poll_cb;
82 # queue the request to open /etc/passwd
84 aio_open "/etc/passwd", IO::AIO::O_RDONLY, 0, sub {
85 my $fh = shift
86 or die "error while opening: $!";
87 # stat'ing filehandles is generally non-blocking
89 my $size = -s $fh;
90 # queue a request to read the file
92 my $contents;
93 aio_read $fh, 0, $size, $contents, 0, sub {
94 $_[0] == $size
95 or die "short read: $!";
96 close $fh;
98 # file contents now in $contents
100 print $contents;
101 # exit event loop and program
103 EV::break;
104 };
105 };
106 # possibly queue up other requests, or open GUI windows,
108 # check for sockets etc. etc.
109 # process events as long as there are some:
111 EV::run;
112
114 Every "aio_*" function creates a request. which is a C data structure
115 not directly visible to Perl.
116 If called in non-void context, every request function returns a Perl
118 object representing the request. In void context, nothing is returned,
119 which saves a bit of memory.
120 The perl object is a fairly standard ref-to-hash object. The hash
122 contents are not used by IO::AIO so you are free to store anything you
123 like in it.
124 During their existance, aio requests travel through the following
126 states, in order:
127 ready
129 Immediately after a request is created it is put into the ready
130 state, waiting for a thread to execute it.
131 execute
133 A thread has accepted the request for processing and is currently
134 executing it (e.g. blocking in read).
135 pending
137 The request has been executed and is waiting for result processing.
138 While request submission and execution is fully asynchronous,
140 result processing is not and relies on the perl interpreter calling
141 "poll_cb" (or another function with the same effect).
142 result
144 The request results are processed synchronously by "poll_cb".
145 The "poll_cb" function will process all outstanding aio requests by
147 calling their callbacks, freeing memory associated with them and
148 managing any groups they are contained in.
149 done
151 Request has reached the end of its lifetime and holds no resources
152 anymore (except possibly for the Perl object, but its connection to
153 the actual aio request is severed and calling its methods will
154 either do nothing or result in a runtime error).
155
157 QUICK OVERVIEW
158 This section simply lists the prototypes most of the functions for
159 quick reference. See the following sections for function-by-function
160 documentation.
161 aio_wd $pathname, $callback->($wd)
163 aio_open $pathname, $flags, $mode, $callback->($fh)
164 aio_close $fh, $callback->($status)
165 aio_seek $fh,$offset,$whence, $callback->($offs)
166 aio_read $fh,$offset,$length, $data,$dataoffset, $callback->($retval)
167 aio_write $fh,$offset,$length, $data,$dataoffset, $callback->($retval)
168 aio_sendfile $out_fh, $in_fh, $in_offset, $length, $callback->($retval)
169 aio_readahead $fh,$offset,$length, $callback->($retval)
170 aio_stat $fh_or_path, $callback->($status)
171 aio_lstat $fh, $callback->($status)
172 aio_statvfs $fh_or_path, $callback->($statvfs)
173 aio_utime $fh_or_path, $atime, $mtime, $callback->($status)
174 aio_chown $fh_or_path, $uid, $gid, $callback->($status)
175 aio_chmod $fh_or_path, $mode, $callback->($status)
176 aio_truncate $fh_or_path, $offset, $callback->($status)
177 aio_allocate $fh, $mode, $offset, $len, $callback->($status)
178 aio_fiemap $fh, $start, $length, $flags, $count, $cb->(\@extents)
179 aio_unlink $pathname, $callback->($status)
180 aio_mknod $pathname, $mode, $dev, $callback->($status)
181 aio_link $srcpath, $dstpath, $callback->($status)
182 aio_symlink $srcpath, $dstpath, $callback->($status)
183 aio_readlink $pathname, $callback->($link)
184 aio_realpath $pathname, $callback->($path)
185 aio_rename $srcpath, $dstpath, $callback->($status)
186 aio_rename2 $srcpath, $dstpath, $flags, $callback->($status)
187 aio_mkdir $pathname, $mode, $callback->($status)
188 aio_rmdir $pathname, $callback->($status)
189 aio_readdir $pathname, $callback->($entries)
190 aio_readdirx $pathname, $flags, $callback->($entries, $flags)
191 IO::AIO::READDIR_DENTS IO::AIO::READDIR_DIRS_FIRST
192 IO::AIO::READDIR_STAT_ORDER IO::AIO::READDIR_FOUND_UNKNOWN
193 aio_scandir $pathname, $maxreq, $callback->($dirs, $nondirs)
194 aio_load $pathname, $data, $callback->($status)
195 aio_copy $srcpath, $dstpath, $callback->($status)
196 aio_move $srcpath, $dstpath, $callback->($status)
197 aio_rmtree $pathname, $callback->($status)
198 aio_fcntl $fh, $cmd, $arg, $callback->($status)
199 aio_ioctl $fh, $request, $buf, $callback->($status)
200 aio_sync $callback->($status)
201 aio_syncfs $fh, $callback->($status)
202 aio_fsync $fh, $callback->($status)
203 aio_fdatasync $fh, $callback->($status)
204 aio_sync_file_range $fh, $offset, $nbytes, $flags, $callback->($status)
205 aio_pathsync $pathname, $callback->($status)
206 aio_msync $scalar, $offset = 0, $length = undef, flags = MS_SYNC, $callback->($status)
207 aio_mtouch $scalar, $offset = 0, $length = undef, flags = 0, $callback->($status)
208 aio_mlock $scalar, $offset = 0, $length = undef, $callback->($status)
209 aio_mlockall $flags, $callback->($status)
210 aio_group $callback->(...)
211 aio_nop $callback->()
212 $prev_pri = aioreq_pri [$pri]
214 aioreq_nice $pri_adjust
215 IO::AIO::poll_wait
217 IO::AIO::poll_cb
218 IO::AIO::poll
219 IO::AIO::flush
220 IO::AIO::max_poll_reqs $nreqs
221 IO::AIO::max_poll_time $seconds
222 IO::AIO::min_parallel $nthreads
223 IO::AIO::max_parallel $nthreads
224 IO::AIO::max_idle $nthreads
225 IO::AIO::idle_timeout $seconds
226 IO::AIO::max_outstanding $maxreqs
227 IO::AIO::nreqs
228 IO::AIO::nready
229 IO::AIO::npending
230 IO::AIO::reinit
231 $nfd = IO::AIO::get_fdlimit [EXPERIMENTAL]
233 IO::AIO::min_fdlimit $nfd [EXPERIMENTAL]
234 IO::AIO::sendfile $ofh, $ifh, $offset, $count
236 IO::AIO::fadvise $fh, $offset, $len, $advice
237 IO::AIO::mmap $scalar, $length, $prot, $flags[, $fh[, $offset]]
239 IO::AIO::munmap $scalar
240 IO::AIO::mremap $scalar, $new_length, $flags[, $new_address]
241 IO::AIO::madvise $scalar, $offset, $length, $advice
242 IO::AIO::mprotect $scalar, $offset, $length, $protect
243 IO::AIO::munlock $scalar, $offset = 0, $length = undef
244 IO::AIO::munlockall
245 # stat extensions
247 $counter = IO::AIO::st_gen
248 $seconds = IO::AIO::st_atime, IO::AIO::st_mtime, IO::AIO::st_ctime, IO::AIO::st_btime
249 ($atime, $mtime, $ctime, $btime, ...) = IO::AIO::st_xtime
250 $nanoseconds = IO::AIO::st_atimensec, IO::AIO::st_mtimensec, IO::AIO::st_ctimensec, IO::AIO::st_btimensec
251 $seconds = IO::AIO::st_btimesec
252 ($atime, $mtime, $ctime, $btime, ...) = IO::AIO::st_xtimensec
253 # very much unportable syscalls
255 IO::AIO::splice $r_fh, $r_off, $w_fh, $w_off, $length, $flags
256 IO::AIO::tee $r_fh, $w_fh, $length, $flags
257 $actual_size = IO::AIO::pipesize $r_fh[, $new_size]
258 ($rfh, $wfh) = IO::AIO::pipe2 [$flags]
259 $fh = IO::AIO::memfd_create $pathname[, $flags]
260 $fh = IO::AIO::eventfd [$initval, [$flags]]
261 $fh = IO::AIO::timerfd_create $clockid[, $flags]
262 ($cur_interval, $cur_value) = IO::AIO::timerfd_settime $fh, $flags, $new_interval, $nbw_value
263 ($cur_interval, $cur_value) = IO::AIO::timerfd_gettime $fh
264 API NOTES
266 All the "aio_*" calls are more or less thin wrappers around the syscall
267 with the same name (sans "aio_"). The arguments are similar or
268 identical, and they all accept an additional (and optional) $callback
269 argument which must be a code reference. This code reference will be
270 called after the syscall has been executed in an asynchronous fashion.
271 The results of the request will be passed as arguments to the callback
272 (and, if an error occured, in $!) - for most requests the syscall
273 return code (e.g. most syscalls return "-1" on error, unlike perl,
274 which usually delivers "false").
275 Some requests (such as "aio_readdir") pass the actual results and
277 communicate failures by passing "undef".
278 All functions expecting a filehandle keep a copy of the filehandle
280 internally until the request has finished.
281 All functions return request objects of type IO::AIO::REQ that allow
283 further manipulation of those requests while they are in-flight.
284 The pathnames you pass to these routines should be absolute. The reason
286 for this is that at the time the request is being executed, the current
287 working directory could have changed. Alternatively, you can make sure
288 that you never change the current working directory anywhere in the
289 program and then use relative paths. You can also take advantage of
290 IO::AIOs working directory abstraction, that lets you specify paths
291 relative to some previously-opened "working directory object" - see the
292 description of the "IO::AIO::WD" class later in this document.
293 To encode pathnames as octets, either make sure you either: a) always
295 pass in filenames you got from outside (command line, readdir etc.)
296 without tinkering, b) are in your native filesystem encoding, c) use
297 the Encode module and encode your pathnames to the locale (or other)
298 encoding in effect in the user environment, d) use
299 Glib::filename_from_unicode on unicode filenames or e) use something
300 else to ensure your scalar has the correct contents.
301 This works, btw. independent of the internal UTF-8 bit, which IO::AIO
303 handles correctly whether it is set or not.
304 AIO REQUEST FUNCTIONS
306 $prev_pri = aioreq_pri [$pri]
307 Returns the priority value that would be used for the next request
308 and, if $pri is given, sets the priority for the next aio request.
309 The default priority is 0, the minimum and maximum priorities are
311 "-4" and 4, respectively. Requests with higher priority will be
312 serviced first.
313 The priority will be reset to 0 after each call to one of the
315 "aio_*" functions.
316 Example: open a file with low priority, then read something from it
318 with higher priority so the read request is serviced before other
319 low priority open requests (potentially spamming the cache):
320 aioreq_pri -3;
322 aio_open ..., sub {
323 return unless $_[0];
324 aioreq_pri -2;
326 aio_read $_[0], ..., sub {
327 ...
328 };
329 };
330 aioreq_nice $pri_adjust
332 Similar to "aioreq_pri", but subtracts the given value from the
333 current priority, so the effect is cumulative.
334 aio_open $pathname, $flags, $mode, $callback->($fh)
336 Asynchronously open or create a file and call the callback with a
337 newly created filehandle for the file (or "undef" in case of an
338 error).
339 The pathname passed to "aio_open" must be absolute. See API NOTES,
341 above, for an explanation.
342 The $flags argument is a bitmask. See the "Fcntl" module for a
344 list. They are the same as used by "sysopen".
345 Likewise, $mode specifies the mode of the newly created file, if it
347 didn't exist and "O_CREAT" has been given, just like perl's
348 "sysopen", except that it is mandatory (i.e. use 0 if you don't
349 create new files, and 0666 or 0777 if you do). Note that the $mode
350 will be modified by the umask in effect then the request is being
351 executed, so better never change the umask.
352 Example:
354 aio_open "/etc/passwd", IO::AIO::O_RDONLY, 0, sub {
356 if ($_[0]) {
357 print "open successful, fh is $_[0]\n";
358 ...
359 } else {
360 die "open failed: $!\n";
361 }
362 };
363 In addition to all the common open modes/flags ("O_RDONLY",
365 "O_WRONLY", "O_RDWR", "O_CREAT", "O_TRUNC", "O_EXCL" and
366 "O_APPEND"), the following POSIX and non-POSIX constants are
367 available (missing ones on your system are, as usual, 0):
368 "O_ASYNC", "O_DIRECT", "O_NOATIME", "O_CLOEXEC", "O_NOCTTY",
370 "O_NOFOLLOW", "O_NONBLOCK", "O_EXEC", "O_SEARCH", "O_DIRECTORY",
371 "O_DSYNC", "O_RSYNC", "O_SYNC", "O_PATH", "O_TMPFILE", "O_TTY_INIT"
372 and "O_ACCMODE".
373 aio_close $fh, $callback->($status)
375 Asynchronously close a file and call the callback with the result
376 code.
377 Unfortunately, you can't do this to perl. Perl insists very
379 strongly on closing the file descriptor associated with the
380 filehandle itself.
381 Therefore, "aio_close" will not close the filehandle - instead it
383 will use dup2 to overwrite the file descriptor with the write-end
384 of a pipe (the pipe fd will be created on demand and will be
385 cached).
386 Or in other words: the file descriptor will be closed, but it will
388 not be free for reuse until the perl filehandle is closed.
389 aio_seek $fh, $offset, $whence, $callback->($offs)
391 Seeks the filehandle to the new $offset, similarly to perl's
392 "sysseek". The $whence can use the traditional values (0 for
393 "IO::AIO::SEEK_SET", 1 for "IO::AIO::SEEK_CUR" or 2 for
394 "IO::AIO::SEEK_END").
395 The resulting absolute offset will be passed to the callback, or
397 "-1" in case of an error.
398 In theory, the $whence constants could be different than the
400 corresponding values from Fcntl, but perl guarantees they are the
401 same, so don't panic.
402 As a GNU/Linux (and maybe Solaris) extension, also the constants
404 "IO::AIO::SEEK_DATA" and "IO::AIO::SEEK_HOLE" are available, if
405 they could be found. No guarantees about suitability for use in
406 "aio_seek" or Perl's "sysseek" can be made though, although I would
407 naively assume they "just work".
408 aio_read $fh,$offset,$length, $data,$dataoffset, $callback->($retval)
410 aio_write $fh,$offset,$length, $data,$dataoffset, $callback->($retval)
411 Reads or writes $length bytes from or to the specified $fh and
412 $offset into the scalar given by $data and offset $dataoffset and
413 calls the callback with the actual number of bytes transferred (or
414 -1 on error, just like the syscall).
415 "aio_read" will, like "sysread", shrink or grow the $data scalar to
417 offset plus the actual number of bytes read.
418 If $offset is undefined, then the current file descriptor offset
420 will be used (and updated), otherwise the file descriptor offset
421 will not be changed by these calls.
422 If $length is undefined in "aio_write", use the remaining length of
424 $data.
425 If $dataoffset is less than zero, it will be counted from the end
427 of $data.
428 The $data scalar MUST NOT be modified in any way while the request
430 is outstanding. Modifying it can result in segfaults or World War
431 III (if the necessary/optional hardware is installed).
432 Example: Read 15 bytes at offset 7 into scalar $buffer, starting at
434 offset 0 within the scalar:
435 aio_read $fh, 7, 15, $buffer, 0, sub {
437 $_[0] > 0 or die "read error: $!";
438 print "read $_[0] bytes: <$buffer>\n";
439 };
440 aio_sendfile $out_fh, $in_fh, $in_offset, $length, $callback->($retval)
442 Tries to copy $length bytes from $in_fh to $out_fh. It starts
443 reading at byte offset $in_offset, and starts writing at the
444 current file offset of $out_fh. Because of that, it is not safe to
445 issue more than one "aio_sendfile" per $out_fh, as they will
446 interfere with each other. The same $in_fh works fine though, as
447 this function does not move or use the file offset of $in_fh.
448 Please note that "aio_sendfile" can read more bytes from $in_fh
450 than are written, and there is no way to find out how many more
451 bytes have been read from "aio_sendfile" alone, as "aio_sendfile"
452 only provides the number of bytes written to $out_fh. Only if the
453 result value equals $length one can assume that $length bytes have
454 been read.
455 Unlike with other "aio_" functions, it makes a lot of sense to use
457 "aio_sendfile" on non-blocking sockets, as long as one end
458 (typically the $in_fh) is a file - the file I/O will then be
459 asynchronous, while the socket I/O will be non-blocking. Note,
460 however, that you can run into a trap where "aio_sendfile" reads
461 some data with readahead, then fails to write all data, and when
462 the socket is ready the next time, the data in the cache is already
463 lost, forcing "aio_sendfile" to again hit the disk. Explicit
464 "aio_read" + "aio_write" let's you better control resource usage.
465 This call tries to make use of a native "sendfile"-like syscall to
467 provide zero-copy operation. For this to work, $out_fh should refer
468 to a socket, and $in_fh should refer to an mmap'able file.
469 If a native sendfile cannot be found or it fails with "ENOSYS",
471 "EINVAL", "ENOTSUP", "EOPNOTSUPP", "EAFNOSUPPORT", "EPROTOTYPE" or
472 "ENOTSOCK", it will be emulated, so you can call "aio_sendfile" on
473 any type of filehandle regardless of the limitations of the
474 operating system.
475 As native sendfile syscalls (as practically any non-POSIX interface
477 hacked together in a hurry to improve benchmark numbers) tend to be
478 rather buggy on many systems, this implementation tries to work
479 around some known bugs in Linux and FreeBSD kernels (probably
480 others, too), but that might fail, so you really really should
481 check the return value of "aio_sendfile" - fewer bytes than
482 expected might have been transferred.
483 aio_readahead $fh,$offset,$length, $callback->($retval)
485 "aio_readahead" populates the page cache with data from a file so
486 that subsequent reads from that file will not block on disk I/O.
487 The $offset argument specifies the starting point from which data
488 is to be read and $length specifies the number of bytes to be read.
489 I/O is performed in whole pages, so that offset is effectively
490 rounded down to a page boundary and bytes are read up to the next
491 page boundary greater than or equal to (off-set+length).
492 "aio_readahead" does not read beyond the end of the file. The
493 current file offset of the file is left unchanged.
494 If that syscall doesn't exist (likely if your kernel isn't Linux)
496 it will be emulated by simply reading the data, which would have a
497 similar effect.
498 aio_stat $fh_or_path, $callback->($status)
500 aio_lstat $fh, $callback->($status)
501 Works almost exactly like perl's "stat" or "lstat" in void context.
502 The callback will be called after the stat and the results will be
503 available using "stat _" or "-s _" and other tests (with the
504 exception of "-B" and "-T").
505 The pathname passed to "aio_stat" must be absolute. See API NOTES,
507 above, for an explanation.
508 Currently, the stats are always 64-bit-stats, i.e. instead of
510 returning an error when stat'ing a large file, the results will be
511 silently truncated unless perl itself is compiled with large file
512 support.
513 To help interpret the mode and dev/rdev stat values, IO::AIO offers
515 the following constants and functions (if not implemented, the
516 constants will be 0 and the functions will either "croak" or fall
517 back on traditional behaviour).
518 "S_IFMT", "S_IFIFO", "S_IFCHR", "S_IFBLK", "S_IFLNK", "S_IFREG",
520 "S_IFDIR", "S_IFWHT", "S_IFSOCK", "IO::AIO::major $dev_t",
521 "IO::AIO::minor $dev_t", "IO::AIO::makedev $major, $minor".
522 To access higher resolution stat timestamps, see "SUBSECOND STAT
524 TIME ACCESS".
525 Example: Print the length of /etc/passwd:
527 aio_stat "/etc/passwd", sub {
529 $_[0] and die "stat failed: $!";
530 print "size is ", -s _, "\n";
531 };
532 aio_statvfs $fh_or_path, $callback->($statvfs)
534 Works like the POSIX "statvfs" or "fstatvfs" syscalls, depending on
535 whether a file handle or path was passed.
536 On success, the callback is passed a hash reference with the
538 following members: "bsize", "frsize", "blocks", "bfree", "bavail",
539 "files", "ffree", "favail", "fsid", "flag" and "namemax". On
540 failure, "undef" is passed.
541 The following POSIX IO::AIO::ST_* constants are defined:
543 "ST_RDONLY" and "ST_NOSUID".
544 The following non-POSIX IO::AIO::ST_* flag masks are defined to
546 their correct value when available, or to 0 on systems that do not
547 support them: "ST_NODEV", "ST_NOEXEC", "ST_SYNCHRONOUS",
548 "ST_MANDLOCK", "ST_WRITE", "ST_APPEND", "ST_IMMUTABLE",
549 "ST_NOATIME", "ST_NODIRATIME" and "ST_RELATIME".
550 Example: stat "/wd" and dump out the data if successful.
552 aio_statvfs "/wd", sub {
554 my $f = $_[0]
555 or die "statvfs: $!";
556 use Data::Dumper;
558 say Dumper $f;
559 };
560 # result:
562 {
563 bsize => 1024,
564 bfree => 4333064312,
565 blocks => 10253828096,
566 files => 2050765568,
567 flag => 4096,
568 favail => 2042092649,
569 bavail => 4333064312,
570 ffree => 2042092649,
571 namemax => 255,
572 frsize => 1024,
573 fsid => 1810
574 }
575 aio_utime $fh_or_path, $atime, $mtime, $callback->($status)
577 Works like perl's "utime" function (including the special case of
578 $atime and $mtime being undef). Fractional times are supported if
579 the underlying syscalls support them.
580 When called with a pathname, uses utimensat(2) or utimes(2) if
582 available, otherwise utime(2). If called on a file descriptor, uses
583 futimens(2) or futimes(2) if available, otherwise returns ENOSYS,
584 so this is not portable.
585 Examples:
587 # set atime and mtime to current time (basically touch(1)):
589 aio_utime "path", undef, undef;
590 # set atime to current time and mtime to beginning of the epoch:
591 aio_utime "path", time, undef; # undef==0
592 aio_chown $fh_or_path, $uid, $gid, $callback->($status)
594 Works like perl's "chown" function, except that "undef" for either
595 $uid or $gid is being interpreted as "do not change" (but -1 can
596 also be used).
597 Examples:
599 # same as "chown root path" in the shell:
601 aio_chown "path", 0, -1;
602 # same as above:
603 aio_chown "path", 0, undef;
604 aio_truncate $fh_or_path, $offset, $callback->($status)
606 Works like truncate(2) or ftruncate(2).
607 aio_allocate $fh, $mode, $offset, $len, $callback->($status)
609 Allocates or frees disk space according to the $mode argument. See
610 the linux "fallocate" documentation for details.
611 $mode is usually 0 or "IO::AIO::FALLOC_FL_KEEP_SIZE" to allocate
613 space, or "IO::AIO::FALLOC_FL_PUNCH_HOLE |
614 IO::AIO::FALLOC_FL_KEEP_SIZE", to deallocate a file range.
615 IO::AIO also supports "FALLOC_FL_COLLAPSE_RANGE", to remove a range
617 (without leaving a hole), "FALLOC_FL_ZERO_RANGE", to zero a range,
618 "FALLOC_FL_INSERT_RANGE" to insert a range and
619 "FALLOC_FL_UNSHARE_RANGE" to unshare shared blocks (see your
620 fallocate(2) manpage).
621 The file system block size used by "fallocate" is presumably the
623 "f_bsize" returned by "statvfs", but different filesystems and
624 filetypes can dictate other limitations.
625 If "fallocate" isn't available or cannot be emulated (currently no
627 emulation will be attempted), passes "-1" and sets $! to "ENOSYS".
628 aio_chmod $fh_or_path, $mode, $callback->($status)
630 Works like perl's "chmod" function.
631 aio_unlink $pathname, $callback->($status)
633 Asynchronously unlink (delete) a file and call the callback with
634 the result code.
635 aio_mknod $pathname, $mode, $dev, $callback->($status)
637 [EXPERIMENTAL]
638 Asynchronously create a device node (or fifo). See mknod(2).
640 The only (POSIX-) portable way of calling this function is:
642 aio_mknod $pathname, IO::AIO::S_IFIFO | $mode, 0, sub { ...
644 See "aio_stat" for info about some potentially helpful extra
646 constants and functions.
647 aio_link $srcpath, $dstpath, $callback->($status)
649 Asynchronously create a new link to the existing object at $srcpath
650 at the path $dstpath and call the callback with the result code.
651 aio_symlink $srcpath, $dstpath, $callback->($status)
653 Asynchronously create a new symbolic link to the existing object at
654 $srcpath at the path $dstpath and call the callback with the result
655 code.
656 aio_readlink $pathname, $callback->($link)
658 Asynchronously read the symlink specified by $path and pass it to
659 the callback. If an error occurs, nothing or undef gets passed to
660 the callback.
661 aio_realpath $pathname, $callback->($path)
663 Asynchronously make the path absolute and resolve any symlinks in
664 $path. The resulting path only consists of directories (same as
665 Cwd::realpath).
666 This request can be used to get the absolute path of the current
668 working directory by passing it a path of . (a single dot).
669 aio_rename $srcpath, $dstpath, $callback->($status)
671 Asynchronously rename the object at $srcpath to $dstpath, just as
672 rename(2) and call the callback with the result code.
673 On systems that support the AIO::WD working directory abstraction
675 natively, the case "[$wd, "."]" as $srcpath is specialcased -
676 instead of failing, "rename" is called on the absolute path of $wd.
677 aio_rename2 $srcpath, $dstpath, $flags, $callback->($status)
679 Basically a version of "aio_rename" with an additional $flags
680 argument. Calling this with "$flags=0" is the same as calling
681 "aio_rename".
682 Non-zero flags are currently only supported on GNU/Linux systems
684 that support renameat2. Other systems fail with "ENOSYS" in this
685 case.
686 The following constants are available (missing ones are, as usual
688 0), see renameat2(2) for details:
689 "IO::AIO::RENAME_NOREPLACE", "IO::AIO::RENAME_EXCHANGE" and
691 "IO::AIO::RENAME_WHITEOUT".
692 aio_mkdir $pathname, $mode, $callback->($status)
694 Asynchronously mkdir (create) a directory and call the callback
695 with the result code. $mode will be modified by the umask at the
696 time the request is executed, so do not change your umask.
697 aio_rmdir $pathname, $callback->($status)
699 Asynchronously rmdir (delete) a directory and call the callback
700 with the result code.
701 On systems that support the AIO::WD working directory abstraction
703 natively, the case "[$wd, "."]" is specialcased - instead of
704 failing, "rmdir" is called on the absolute path of $wd.
705 aio_readdir $pathname, $callback->($entries)
707 Unlike the POSIX call of the same name, "aio_readdir" reads an
708 entire directory (i.e. opendir + readdir + closedir). The entries
709 will not be sorted, and will NOT include the "." and ".." entries.
710 The callback is passed a single argument which is either "undef" or
712 an array-ref with the filenames.
713 aio_readdirx $pathname, $flags, $callback->($entries, $flags)
715 Quite similar to "aio_readdir", but the $flags argument allows one
716 to tune behaviour and output format. In case of an error, $entries
717 will be "undef".
718 The flags are a combination of the following constants, ORed
720 together (the flags will also be passed to the callback, possibly
721 modified):
722 IO::AIO::READDIR_DENTS
724 Normally the callback gets an arrayref consisting of names only
725 (as with "aio_readdir"). If this flag is set, then the callback
726 gets an arrayref with "[$name, $type, $inode]" arrayrefs, each
727 describing a single directory entry in more detail:
728 $name is the name of the entry.
730 $type is one of the "IO::AIO::DT_xxx" constants:
732 "IO::AIO::DT_UNKNOWN", "IO::AIO::DT_FIFO", "IO::AIO::DT_CHR",
734 "IO::AIO::DT_DIR", "IO::AIO::DT_BLK", "IO::AIO::DT_REG",
735 "IO::AIO::DT_LNK", "IO::AIO::DT_SOCK", "IO::AIO::DT_WHT".
736 "IO::AIO::DT_UNKNOWN" means just that: readdir does not know.
738 If you need to know, you have to run stat yourself. Also, for
739 speed/memory reasons, the $type scalars are read-only: you must
740 not modify them.
741 $inode is the inode number (which might not be exact on systems
743 with 64 bit inode numbers and 32 bit perls). This field has
744 unspecified content on systems that do not deliver the inode
745 information.
746 IO::AIO::READDIR_DIRS_FIRST
748 When this flag is set, then the names will be returned in an
749 order where likely directories come first, in optimal stat
750 order. This is useful when you need to quickly find
751 directories, or you want to find all directories while avoiding
752 to stat() each entry.
753 If the system returns type information in readdir, then this is
755 used to find directories directly. Otherwise, likely
756 directories are names beginning with ".", or otherwise names
757 with no dots, of which names with short names are tried first.
758 IO::AIO::READDIR_STAT_ORDER
760 When this flag is set, then the names will be returned in an
761 order suitable for stat()'ing each one. That is, when you plan
762 to stat() most or all files in the given directory, then the
763 returned order will likely be faster.
764 If both this flag and "IO::AIO::READDIR_DIRS_FIRST" are
766 specified, then the likely dirs come first, resulting in a less
767 optimal stat order for stat'ing all entries, but likely a more
768 optimal order for finding subdirectories.
769 IO::AIO::READDIR_FOUND_UNKNOWN
771 This flag should not be set when calling "aio_readdirx".
772 Instead, it is being set by "aio_readdirx", when any of the
773 $type's found were "IO::AIO::DT_UNKNOWN". The absence of this
774 flag therefore indicates that all $type's are known, which can
775 be used to speed up some algorithms.
776 aio_slurp $pathname, $offset, $length, $data, $callback->($status)
778 Opens, reads and closes the given file. The data is put into $data,
779 which is resized as required.
780 If $offset is negative, then it is counted from the end of the
782 file.
783 If $length is zero, then the remaining length of the file is used.
785 Also, in this case, the same limitations to modifying $data apply
786 as when IO::AIO::mmap is used, i.e. it must only be modified in-
787 place with "substr". If the size of the file is known, specifying a
788 non-zero $length results in a performance advantage.
789 This request is similar to the older "aio_load" request, but since
791 it is a single request, it might be more efficient to use.
792 Example: load /etc/passwd into $passwd.
794 my $passwd;
796 aio_slurp "/etc/passwd", 0, 0, $passwd, sub {
797 $_[0] >= 0
798 or die "/etc/passwd: $!\n";
799 printf "/etc/passwd is %d bytes long, and contains:\n", length $passwd;
801 print $passwd;
802 };
803 IO::AIO::flush;
804 aio_load $pathname, $data, $callback->($status)
806 This is a composite request that tries to fully load the given file
807 into memory. Status is the same as with aio_read.
808 Using "aio_slurp" might be more efficient, as it is a single
810 request.
811 aio_copy $srcpath, $dstpath, $callback->($status)
813 Try to copy the file (directories not supported as either source or
814 destination) from $srcpath to $dstpath and call the callback with a
815 status of 0 (ok) or "-1" (error, see $!).
816 Existing destination files will be truncated.
818 This is a composite request that creates the destination file with
820 mode 0200 and copies the contents of the source file into it using
821 "aio_sendfile", followed by restoring atime, mtime, access mode and
822 uid/gid, in that order.
823 If an error occurs, the partial destination file will be unlinked,
825 if possible, except when setting atime, mtime, access mode and
826 uid/gid, where errors are being ignored.
827 aio_move $srcpath, $dstpath, $callback->($status)
829 Try to move the file (directories not supported as either source or
830 destination) from $srcpath to $dstpath and call the callback with a
831 status of 0 (ok) or "-1" (error, see $!).
832 This is a composite request that tries to rename(2) the file first;
834 if rename fails with "EXDEV", it copies the file with "aio_copy"
835 and, if that is successful, unlinks the $srcpath.
836 aio_scandir $pathname, $maxreq, $callback->($dirs, $nondirs)
838 Scans a directory (similar to "aio_readdir") but additionally tries
839 to efficiently separate the entries of directory $path into two
840 sets of names, directories you can recurse into (directories), and
841 ones you cannot recurse into (everything else, including symlinks
842 to directories).
843 "aio_scandir" is a composite request that generates many sub
845 requests. $maxreq specifies the maximum number of outstanding aio
846 requests that this function generates. If it is "<= 0", then a
847 suitable default will be chosen (currently 4).
848 On error, the callback is called without arguments, otherwise it
850 receives two array-refs with path-relative entry names.
851 Example:
853 aio_scandir $dir, 0, sub {
855 my ($dirs, $nondirs) = @_;
856 print "real directories: @$dirs\n";
857 print "everything else: @$nondirs\n";
858 };
859 Implementation notes.
861 The "aio_readdir" cannot be avoided, but "stat()"'ing every entry
863 can.
864 If readdir returns file type information, then this is used
866 directly to find directories.
867 Otherwise, after reading the directory, the modification time, size
869 etc. of the directory before and after the readdir is checked, and
870 if they match (and isn't the current time), the link count will be
871 used to decide how many entries are directories (if >= 2).
872 Otherwise, no knowledge of the number of subdirectories will be
873 assumed.
874 Then entries will be sorted into likely directories a non-initial
876 dot currently) and likely non-directories (see "aio_readdirx").
877 Then every entry plus an appended "/." will be "stat"'ed, likely
878 directories first, in order of their inode numbers. If that
879 succeeds, it assumes that the entry is a directory or a symlink to
880 directory (which will be checked separately). This is often faster
881 than stat'ing the entry itself because filesystems might detect the
882 type of the entry without reading the inode data (e.g. ext2fs
883 filetype feature), even on systems that cannot return the filetype
884 information on readdir.
885 If the known number of directories (link count - 2) has been
887 reached, the rest of the entries is assumed to be non-directories.
888 This only works with certainty on POSIX (= UNIX) filesystems, which
890 fortunately are the vast majority of filesystems around.
891 It will also likely work on non-POSIX filesystems with reduced
893 efficiency as those tend to return 0 or 1 as link counts, which
894 disables the directory counting heuristic.
895 aio_rmtree $pathname, $callback->($status)
897 Delete a directory tree starting (and including) $path, return the
898 status of the final "rmdir" only. This is a composite request that
899 uses "aio_scandir" to recurse into and rmdir directories, and
900 unlink everything else.
901 aio_fcntl $fh, $cmd, $arg, $callback->($status)
903 aio_ioctl $fh, $request, $buf, $callback->($status)
904 These work just like the "fcntl" and "ioctl" built-in functions,
905 except they execute asynchronously and pass the return value to the
906 callback.
907 Both calls can be used for a lot of things, some of which make more
909 sense to run asynchronously in their own thread, while some others
910 make less sense. For example, calls that block waiting for external
911 events, such as locking, will also lock down an I/O thread while it
912 is waiting, which can deadlock the whole I/O system. At the same
913 time, there might be no alternative to using a thread to wait.
914 So in general, you should only use these calls for things that do
916 (filesystem) I/O, not for things that wait for other events
917 (network, other processes), although if you are careful and know
918 what you are doing, you still can.
919 The following constants are available (missing ones are, as usual
921 0):
922 "F_DUPFD_CLOEXEC",
924 "F_OFD_GETLK", "F_OFD_SETLK", "F_OFD_GETLKW",
926 "FIFREEZE", "FITHAW", "FITRIM", "FICLONE", "FICLONERANGE",
928 "FIDEDUPERANGE".
929 "FS_IOC_GETFLAGS", "FS_IOC_SETFLAGS", "FS_IOC_GETVERSION",
931 "FS_IOC_SETVERSION", "FS_IOC_FIEMAP".
932 "FS_IOC_FSGETXATTR", "FS_IOC_FSSETXATTR",
934 "FS_IOC_SET_ENCRYPTION_POLICY", "FS_IOC_GET_ENCRYPTION_PWSALT",
935 "FS_IOC_GET_ENCRYPTION_POLICY", "FS_KEY_DESCRIPTOR_SIZE".
936 "FS_SECRM_FL", "FS_UNRM_FL", "FS_COMPR_FL", "FS_SYNC_FL",
938 "FS_IMMUTABLE_FL", "FS_APPEND_FL", "FS_NODUMP_FL", "FS_NOATIME_FL",
939 "FS_DIRTY_FL", "FS_COMPRBLK_FL", "FS_NOCOMP_FL", "FS_ENCRYPT_FL",
940 "FS_BTREE_FL", "FS_INDEX_FL", "FS_JOURNAL_DATA_FL", "FS_NOTAIL_FL",
941 "FS_DIRSYNC_FL", "FS_TOPDIR_FL", "FS_FL_USER_MODIFIABLE".
942 "FS_XFLAG_REALTIME", "FS_XFLAG_PREALLOC", "FS_XFLAG_IMMUTABLE",
944 "FS_XFLAG_APPEND", "FS_XFLAG_SYNC", "FS_XFLAG_NOATIME",
945 "FS_XFLAG_NODUMP", "FS_XFLAG_RTINHERIT", "FS_XFLAG_PROJINHERIT",
946 "FS_XFLAG_NOSYMLINKS", "FS_XFLAG_EXTSIZE", "FS_XFLAG_EXTSZINHERIT",
947 "FS_XFLAG_NODEFRAG", "FS_XFLAG_FILESTREAM", "FS_XFLAG_DAX",
948 "FS_XFLAG_HASATTR",
949 aio_sync $callback->($status)
951 Asynchronously call sync and call the callback when finished.
952 aio_fsync $fh, $callback->($status)
954 Asynchronously call fsync on the given filehandle and call the
955 callback with the fsync result code.
956 aio_fdatasync $fh, $callback->($status)
958 Asynchronously call fdatasync on the given filehandle and call the
959 callback with the fdatasync result code.
960 If this call isn't available because your OS lacks it or it
962 couldn't be detected, it will be emulated by calling "fsync"
963 instead.
964 aio_syncfs $fh, $callback->($status)
966 Asynchronously call the syncfs syscall to sync the filesystem
967 associated to the given filehandle and call the callback with the
968 syncfs result code. If syncfs is not available, calls sync(), but
969 returns "-1" and sets errno to "ENOSYS" nevertheless.
970 aio_sync_file_range $fh, $offset, $nbytes, $flags, $callback->($status)
972 Sync the data portion of the file specified by $offset and $length
973 to disk (but NOT the metadata), by calling the Linux-specific
974 sync_file_range call. If sync_file_range is not available or it
975 returns ENOSYS, then fdatasync or fsync is being substituted.
976 $flags can be a combination of
978 "IO::AIO::SYNC_FILE_RANGE_WAIT_BEFORE",
979 "IO::AIO::SYNC_FILE_RANGE_WRITE" and
980 "IO::AIO::SYNC_FILE_RANGE_WAIT_AFTER": refer to the sync_file_range
981 manpage for details.
982 aio_pathsync $pathname, $callback->($status)
984 This request tries to open, fsync and close the given path. This is
985 a composite request intended to sync directories after directory
986 operations (E.g. rename). This might not work on all operating
987 systems or have any specific effect, but usually it makes sure that
988 directory changes get written to disc. It works for anything that
989 can be opened for read-only, not just directories.
990 Future versions of this function might fall back to other methods
992 when "fsync" on the directory fails (such as calling "sync").
993 Passes 0 when everything went ok, and "-1" on error.
995 aio_msync $scalar, $offset = 0, $length = undef, flags = MS_SYNC,
997 $callback->($status)
998 This is a rather advanced IO::AIO call, which only works on
999 mmap(2)ed scalars (see the "IO::AIO::mmap" function, although it
1000 also works on data scalars managed by the Sys::Mmap or Mmap
1001 modules, note that the scalar must only be modified in-place while
1002 an aio operation is pending on it).
1003 It calls the "msync" function of your OS, if available, with the
1005 memory area starting at $offset in the string and ending $length
1006 bytes later. If $length is negative, counts from the end, and if
1007 $length is "undef", then it goes till the end of the string. The
1008 flags can be either "IO::AIO::MS_ASYNC" or "IO::AIO::MS_SYNC", plus
1009 an optional "IO::AIO::MS_INVALIDATE".
1010 aio_mtouch $scalar, $offset = 0, $length = undef, flags = 0,
1012 $callback->($status)
1013 This is a rather advanced IO::AIO call, which works best on
1014 mmap(2)ed scalars.
1015 It touches (reads or writes) all memory pages in the specified
1017 range inside the scalar. All caveats and parameters are the same as
1018 for "aio_msync", above, except for flags, which must be either 0
1019 (which reads all pages and ensures they are instantiated) or
1020 "IO::AIO::MT_MODIFY", which modifies the memory pages (by reading
1021 and writing an octet from it, which dirties the page).
1022 aio_mlock $scalar, $offset = 0, $length = undef, $callback->($status)
1024 This is a rather advanced IO::AIO call, which works best on
1025 mmap(2)ed scalars.
1026 It reads in all the pages of the underlying storage into memory (if
1028 any) and locks them, so they are not getting swapped/paged out or
1029 removed.
1030 If $length is undefined, then the scalar will be locked till the
1032 end.
1033 On systems that do not implement "mlock", this function returns
1035 "-1" and sets errno to "ENOSYS".
1036 Note that the corresponding "munlock" is synchronous and is
1038 documented under "MISCELLANEOUS FUNCTIONS".
1039 Example: open a file, mmap and mlock it - both will be undone when
1041 $data gets destroyed.
1042 open my $fh, "<", $path or die "$path: $!";
1044 my $data;
1045 IO::AIO::mmap $data, -s $fh, IO::AIO::PROT_READ, IO::AIO::MAP_SHARED, $fh;
1046 aio_mlock $data; # mlock in background
1047 aio_mlockall $flags, $callback->($status)
1049 Calls the "mlockall" function with the given $flags (a combination
1050 of "IO::AIO::MCL_CURRENT", "IO::AIO::MCL_FUTURE" and
1051 "IO::AIO::MCL_ONFAULT").
1052 On systems that do not implement "mlockall", this function returns
1054 "-1" and sets errno to "ENOSYS". Similarly, flag combinations not
1055 supported by the system result in a return value of "-1" with errno
1056 being set to "EINVAL".
1057 Note that the corresponding "munlockall" is synchronous and is
1059 documented under "MISCELLANEOUS FUNCTIONS".
1060 Example: asynchronously lock all current and future pages into
1062 memory.
1063 aio_mlockall IO::AIO::MCL_FUTURE;
1065 aio_fiemap $fh, $start, $length, $flags, $count, $cb->(\@extents)
1067 Queries the extents of the given file (by calling the Linux
1068 "FIEMAP" ioctl, see <http://cvs.schmorp.de/IO-AIO/doc/fiemap.txt>
1069 for details). If the ioctl is not available on your OS, then this
1070 request will fail with "ENOSYS".
1071 $start is the starting offset to query extents for, $length is the
1073 size of the range to query - if it is "undef", then the whole file
1074 will be queried.
1075 $flags is a combination of flags ("IO::AIO::FIEMAP_FLAG_SYNC" or
1077 "IO::AIO::FIEMAP_FLAG_XATTR" - "IO::AIO::FIEMAP_FLAGS_COMPAT" is
1078 also exported), and is normally 0 or "IO::AIO::FIEMAP_FLAG_SYNC" to
1079 query the data portion.
1080 $count is the maximum number of extent records to return. If it is
1082 "undef", then IO::AIO queries all extents of the range. As a very
1083 special case, if it is 0, then the callback receives the number of
1084 extents instead of the extents themselves (which is unreliable, see
1085 below).
1086 If an error occurs, the callback receives no arguments. The special
1088 "errno" value "IO::AIO::EBADR" is available to test for flag
1089 errors.
1090 Otherwise, the callback receives an array reference with extent
1092 structures. Each extent structure is an array reference itself,
1093 with the following members:
1094 [$logical, $physical, $length, $flags]
1096 Flags is any combination of the following flag values (typically
1098 either 0 or "IO::AIO::FIEMAP_EXTENT_LAST" (1)):
1099 "IO::AIO::FIEMAP_EXTENT_LAST", "IO::AIO::FIEMAP_EXTENT_UNKNOWN",
1101 "IO::AIO::FIEMAP_EXTENT_DELALLOC",
1102 "IO::AIO::FIEMAP_EXTENT_ENCODED",
1103 "IO::AIO::FIEMAP_EXTENT_DATA_ENCRYPTED",
1104 "IO::AIO::FIEMAP_EXTENT_NOT_ALIGNED",
1105 "IO::AIO::FIEMAP_EXTENT_DATA_INLINE",
1106 "IO::AIO::FIEMAP_EXTENT_DATA_TAIL",
1107 "IO::AIO::FIEMAP_EXTENT_UNWRITTEN", "IO::AIO::FIEMAP_EXTENT_MERGED"
1108 or "IO::AIO::FIEMAP_EXTENT_SHARED".
1109 At the time of this writing (Linux 3.2), this request is unreliable
1111 unless $count is "undef", as the kernel has all sorts of bugs
1112 preventing it to return all extents of a range for files with a
1113 large number of extents. The code (only) works around all these
1114 issues if $count is "undef".
1115 aio_group $callback->(...)
1117 This is a very special aio request: Instead of doing something, it
1118 is a container for other aio requests, which is useful if you want
1119 to bundle many requests into a single, composite, request with a
1120 definite callback and the ability to cancel the whole request with
1121 its subrequests.
1122 Returns an object of class IO::AIO::GRP. See its documentation
1124 below for more info.
1125 Example:
1127 my $grp = aio_group sub {
1129 print "all stats done\n";
1130 };
1131 add $grp
1133 (aio_stat ...),
1134 (aio_stat ...),
1135 ...;
1136 aio_nop $callback->()
1138 This is a special request - it does nothing in itself and is only
1139 used for side effects, such as when you want to add a dummy request
1140 to a group so that finishing the requests in the group depends on
1141 executing the given code.
1142 While this request does nothing, it still goes through the
1144 execution phase and still requires a worker thread. Thus, the
1145 callback will not be executed immediately but only after other
1146 requests in the queue have entered their execution phase. This can
1147 be used to measure request latency.
1148 IO::AIO::aio_busy $fractional_seconds, $callback->() *NOT EXPORTED*
1150 Mainly used for debugging and benchmarking, this aio request puts
1151 one of the request workers to sleep for the given time.
1152 While it is theoretically handy to have simple I/O scheduling
1154 requests like sleep and file handle readable/writable, the overhead
1155 this creates is immense (it blocks a thread for a long time) so do
1156 not use this function except to put your application under
1157 artificial I/O pressure.
1158 IO::AIO::WD - multiple working directories
1160 Your process only has one current working directory, which is used by
1161 all threads. This makes it hard to use relative paths (some other
1162 component could call "chdir" at any time, and it is hard to control
1163 when the path will be used by IO::AIO).
1164 One solution for this is to always use absolute paths. This usually
1166 works, but can be quite slow (the kernel has to walk the whole path on
1167 every access), and can also be a hassle to implement.
1168 Newer POSIX systems have a number of functions (openat, fdopendir,
1170 futimensat and so on) that make it possible to specify working
1171 directories per operation.
1172 For portability, and because the clowns who "designed", or shall I
1174 write, perpetrated this new interface were obviously half-drunk, this
1175 abstraction cannot be perfect, though.
1176 IO::AIO allows you to convert directory paths into a so-called
1178 IO::AIO::WD object. This object stores the canonicalised, absolute
1179 version of the path, and on systems that allow it, also a directory
1180 file descriptor.
1181 Everywhere where a pathname is accepted by IO::AIO (e.g. in "aio_stat"
1183 or "aio_unlink"), one can specify an array reference with an
1184 IO::AIO::WD object and a pathname instead (or the IO::AIO::WD object
1185 alone, which gets interpreted as "[$wd, "."]"). If the pathname is
1186 absolute, the IO::AIO::WD object is ignored, otherwise the pathname is
1187 resolved relative to that IO::AIO::WD object.
1188 For example, to get a wd object for /etc and then stat passwd inside,
1190 you would write:
1191 aio_wd "/etc", sub {
1193 my $etcdir = shift;
1194 # although $etcdir can be undef on error, there is generally no reason
1196 # to check for errors here, as aio_stat will fail with ENOENT
1197 # when $etcdir is undef.
1198 aio_stat [$etcdir, "passwd"], sub {
1200 # yay
1201 };
1202 };
1203 The fact that "aio_wd" is a request and not a normal function shows
1205 that creating an IO::AIO::WD object is itself a potentially blocking
1206 operation, which is why it is done asynchronously.
1207 To stat the directory obtained with "aio_wd" above, one could write
1209 either of the following three request calls:
1210 aio_lstat "/etc" , sub { ... # pathname as normal string
1212 aio_lstat [$wd, "."], sub { ... # "." relative to $wd (i.e. $wd itself)
1213 aio_lstat $wd , sub { ... # shorthand for the previous
1214 As with normal pathnames, IO::AIO keeps a copy of the working directory
1216 object and the pathname string, so you could write the following
1217 without causing any issues due to $path getting reused:
1218 my $path = [$wd, undef];
1220 for my $name (qw(abc def ghi)) {
1222 $path->[1] = $name;
1223 aio_stat $path, sub {
1224 # ...
1225 };
1226 }
1227 There are some caveats: when directories get renamed (or deleted), the
1229 pathname string doesn't change, so will point to the new directory (or
1230 nowhere at all), while the directory fd, if available on the system,
1231 will still point to the original directory. Most functions accepting a
1232 pathname will use the directory fd on newer systems, and the string on
1233 older systems. Some functions (such as "aio_realpath") will always rely
1234 on the string form of the pathname.
1235 So this functionality is mainly useful to get some protection against
1237 "chdir", to easily get an absolute path out of a relative path for
1238 future reference, and to speed up doing many operations in the same
1239 directory (e.g. when stat'ing all files in a directory).
1240 The following functions implement this working directory abstraction:
1242 aio_wd $pathname, $callback->($wd)
1244 Asynchonously canonicalise the given pathname and convert it to an
1245 IO::AIO::WD object representing it. If possible and supported on
1246 the system, also open a directory fd to speed up pathname
1247 resolution relative to this working directory.
1248 If something goes wrong, then "undef" is passwd to the callback
1250 instead of a working directory object and $! is set appropriately.
1251 Since passing "undef" as working directory component of a pathname
1252 fails the request with "ENOENT", there is often no need for error
1253 checking in the "aio_wd" callback, as future requests using the
1254 value will fail in the expected way.
1255 IO::AIO::CWD
1257 This is a compiletime constant (object) that represents the process
1258 current working directory.
1259 Specifying this object as working directory object for a pathname
1261 is as if the pathname would be specified directly, without a
1262 directory object. For example, these calls are functionally
1263 identical:
1264 aio_stat "somefile", sub { ... };
1266 aio_stat [IO::AIO::CWD, "somefile"], sub { ... };
1267 To recover the path associated with an IO::AIO::WD object, you can use
1269 "aio_realpath":
1270 aio_realpath $wd, sub {
1272 warn "path is $_[0]\n";
1273 };
1274 Currently, "aio_statvfs" always, and "aio_rename" and "aio_rmdir"
1276 sometimes, fall back to using an absolue path.
1277 IO::AIO::REQ CLASS
1279 All non-aggregate "aio_*" functions return an object of this class when
1280 called in non-void context.
1281 cancel $req
1283 Cancels the request, if possible. Has the effect of skipping
1284 execution when entering the execute state and skipping calling the
1285 callback when entering the the result state, but will leave the
1286 request otherwise untouched (with the exception of readdir). That
1287 means that requests that currently execute will not be stopped and
1288 resources held by the request will not be freed prematurely.
1289 cb $req $callback->(...)
1291 Replace (or simply set) the callback registered to the request.
1292 IO::AIO::GRP CLASS
1294 This class is a subclass of IO::AIO::REQ, so all its methods apply to
1295 objects of this class, too.
1296 A IO::AIO::GRP object is a special request that can contain multiple
1298 other aio requests.
1299 You create one by calling the "aio_group" constructing function with a
1301 callback that will be called when all contained requests have entered
1302 the "done" state:
1303 my $grp = aio_group sub {
1305 print "all requests are done\n";
1306 };
1307 You add requests by calling the "add" method with one or more
1309 "IO::AIO::REQ" objects:
1310 $grp->add (aio_unlink "...");
1312 add $grp aio_stat "...", sub {
1314 $_[0] or return $grp->result ("error");
1315 # add another request dynamically, if first succeeded
1317 add $grp aio_open "...", sub {
1318 $grp->result ("ok");
1319 };
1320 };
1321 This makes it very easy to create composite requests (see the source of
1323 "aio_move" for an application) that work and feel like simple requests.
1324 · The IO::AIO::GRP objects will be cleaned up during calls to
1326 "IO::AIO::poll_cb", just like any other request.
1327 · They can be canceled like any other request. Canceling will cancel
1329 not only the request itself, but also all requests it contains.
1330 · They can also can also be added to other IO::AIO::GRP objects.
1332 · You must not add requests to a group from within the group callback
1334 (or any later time).
1335 Their lifetime, simplified, looks like this: when they are empty, they
1337 will finish very quickly. If they contain only requests that are in the
1338 "done" state, they will also finish. Otherwise they will continue to
1339 exist.
1340 That means after creating a group you have some time to add requests
1342 (precisely before the callback has been invoked, which is only done
1343 within the "poll_cb"). And in the callbacks of those requests, you can
1344 add further requests to the group. And only when all those requests
1345 have finished will the the group itself finish.
1346 add $grp ...
1348 $grp->add (...)
1349 Add one or more requests to the group. Any type of IO::AIO::REQ can
1350 be added, including other groups, as long as you do not create
1351 circular dependencies.
1352 Returns all its arguments.
1354 $grp->cancel_subs
1356 Cancel all subrequests and clears any feeder, but not the group
1357 request itself. Useful when you queued a lot of events but got a
1358 result early.
1359 The group request will finish normally (you cannot add requests to
1361 the group).
1362 $grp->result (...)
1364 Set the result value(s) that will be passed to the group callback
1365 when all subrequests have finished and set the groups errno to the
1366 current value of errno (just like calling "errno" without an error
1367 number). By default, no argument will be passed and errno is zero.
1368 $grp->errno ([$errno])
1370 Sets the group errno value to $errno, or the current value of errno
1371 when the argument is missing.
1372 Every aio request has an associated errno value that is restored
1374 when the callback is invoked. This method lets you change this
1375 value from its default (0).
1376 Calling "result" will also set errno, so make sure you either set
1378 $! before the call to "result", or call c<errno> after it.
1379 feed $grp $callback->($grp)
1381 Sets a feeder/generator on this group: every group can have an
1382 attached generator that generates requests if idle. The idea behind
1383 this is that, although you could just queue as many requests as you
1384 want in a group, this might starve other requests for a potentially
1385 long time. For example, "aio_scandir" might generate hundreds of
1386 thousands of "aio_stat" requests, delaying any later requests for a
1387 long time.
1388 To avoid this, and allow incremental generation of requests, you
1390 can instead a group and set a feeder on it that generates those
1391 requests. The feed callback will be called whenever there are few
1392 enough (see "limit", below) requests active in the group itself and
1393 is expected to queue more requests.
1394 The feed callback can queue as many requests as it likes (i.e.
1396 "add" does not impose any limits).
1397 If the feed does not queue more requests when called, it will be
1399 automatically removed from the group.
1400 If the feed limit is 0 when this method is called, it will be set
1402 to 2 automatically.
1403 Example:
1405 # stat all files in @files, but only ever use four aio requests concurrently:
1407 my $grp = aio_group sub { print "finished\n" };
1409 limit $grp 4;
1410 feed $grp sub {
1411 my $file = pop @files
1412 or return;
1413 add $grp aio_stat $file, sub { ... };
1415 };
1416 limit $grp $num
1418 Sets the feeder limit for the group: The feeder will be called
1419 whenever the group contains less than this many requests.
1420 Setting the limit to 0 will pause the feeding process.
1422 The default value for the limit is 0, but note that setting a
1424 feeder automatically bumps it up to 2.
1425 SUPPORT FUNCTIONS
1427 EVENT PROCESSING AND EVENT LOOP INTEGRATION
1428 $fileno = IO::AIO::poll_fileno
1430 Return the request result pipe file descriptor. This filehandle
1431 must be polled for reading by some mechanism outside this module
1432 (e.g. EV, Glib, select and so on, see below or the SYNOPSIS). If
1433 the pipe becomes readable you have to call "poll_cb" to check the
1434 results.
1435 See "poll_cb" for an example.
1437 IO::AIO::poll_cb
1439 Process some requests that have reached the result phase (i.e. they
1440 have been executed but the results are not yet reported). You have
1441 to call this "regularly" to finish outstanding requests.
1442 Returns 0 if all events could be processed (or there were no events
1444 to process), or "-1" if it returned earlier for whatever reason.
1445 Returns immediately when no events are outstanding. The amount of
1446 events processed depends on the settings of
1447 "IO::AIO::max_poll_req", "IO::AIO::max_poll_time" and
1448 "IO::AIO::max_outstanding".
1449 If not all requests were processed for whatever reason, the poll
1451 file descriptor will still be ready when "poll_cb" returns, so
1452 normally you don't have to do anything special to have it called
1453 later.
1454 Apart from calling "IO::AIO::poll_cb" when the event filehandle
1456 becomes ready, it can be beneficial to call this function from
1457 loops which submit a lot of requests, to make sure the results get
1458 processed when they become available and not just when the loop is
1459 finished and the event loop takes over again. This function returns
1460 very fast when there are no outstanding requests.
1461 Example: Install an Event watcher that automatically calls
1463 IO::AIO::poll_cb with high priority (more examples can be found in
1464 the SYNOPSIS section, at the top of this document):
1465 Event->io (fd => IO::AIO::poll_fileno,
1467 poll => 'r', async => 1,
1468 cb => \&IO::AIO::poll_cb);
1469 IO::AIO::poll_wait
1471 Wait until either at least one request is in the result phase or no
1472 requests are outstanding anymore.
1473 This is useful if you want to synchronously wait for some requests
1475 to become ready, without actually handling them.
1476 See "nreqs" for an example.
1478 IO::AIO::poll
1480 Waits until some requests have been handled.
1481 Returns the number of requests processed, but is otherwise strictly
1483 equivalent to:
1484 IO::AIO::poll_wait, IO::AIO::poll_cb
1486 IO::AIO::flush
1488 Wait till all outstanding AIO requests have been handled.
1489 Strictly equivalent to:
1491 IO::AIO::poll_wait, IO::AIO::poll_cb
1493 while IO::AIO::nreqs;
1494 This function can be useful at program aborts, to make sure
1496 outstanding I/O has been done ("IO::AIO" uses an "END" block which
1497 already calls this function on normal exits), or when you are
1498 merely using "IO::AIO" for its more advanced functions, rather than
1499 for async I/O, e.g.:
1500 my ($dirs, $nondirs);
1502 IO::AIO::aio_scandir "/tmp", 0, sub { ($dirs, $nondirs) = @_ };
1503 IO::AIO::flush;
1504 # $dirs, $nondirs are now set
1505 IO::AIO::max_poll_reqs $nreqs
1507 IO::AIO::max_poll_time $seconds
1508 These set the maximum number of requests (default 0, meaning
1509 infinity) that are being processed by "IO::AIO::poll_cb" in one
1510 call, respectively the maximum amount of time (default 0, meaning
1511 infinity) spent in "IO::AIO::poll_cb" to process requests (more
1512 correctly the mininum amount of time "poll_cb" is allowed to use).
1513 Setting "max_poll_time" to a non-zero value creates an overhead of
1515 one syscall per request processed, which is not normally a problem
1516 unless your callbacks are really really fast or your OS is really
1517 really slow (I am not mentioning Solaris here). Using
1518 "max_poll_reqs" incurs no overhead.
1519 Setting these is useful if you want to ensure some level of
1521 interactiveness when perl is not fast enough to process all
1522 requests in time.
1523 For interactive programs, values such as 0.01 to 0.1 should be
1525 fine.
1526 Example: Install an Event watcher that automatically calls
1528 IO::AIO::poll_cb with low priority, to ensure that other parts of
1529 the program get the CPU sometimes even under high AIO load.
1530 # try not to spend much more than 0.1s in poll_cb
1532 IO::AIO::max_poll_time 0.1;
1533 # use a low priority so other tasks have priority
1535 Event->io (fd => IO::AIO::poll_fileno,
1536 poll => 'r', nice => 1,
1537 cb => &IO::AIO::poll_cb);
1538 CONTROLLING THE NUMBER OF THREADS
1540 IO::AIO::min_parallel $nthreads
1542 Set the minimum number of AIO threads to $nthreads. The current
1543 default is 8, which means eight asynchronous operations can execute
1544 concurrently at any one time (the number of outstanding requests,
1545 however, is unlimited).
1546 IO::AIO starts threads only on demand, when an AIO request is
1548 queued and no free thread exists. Please note that queueing up a
1549 hundred requests can create demand for a hundred threads, even if
1550 it turns out that everything is in the cache and could have been
1551 processed faster by a single thread.
1552 It is recommended to keep the number of threads relatively low, as
1554 some Linux kernel versions will scale negatively with the number of
1555 threads (higher parallelity => MUCH higher latency). With current
1556 Linux 2.6 versions, 4-32 threads should be fine.
1557 Under most circumstances you don't need to call this function, as
1559 the module selects a default that is suitable for low to moderate
1560 load.
1561 IO::AIO::max_parallel $nthreads
1563 Sets the maximum number of AIO threads to $nthreads. If more than
1564 the specified number of threads are currently running, this
1565 function kills them. This function blocks until the limit is
1566 reached.
1567 While $nthreads are zero, aio requests get queued but not executed
1569 until the number of threads has been increased again.
1570 This module automatically runs "max_parallel 0" at program end, to
1572 ensure that all threads are killed and that there are no
1573 outstanding requests.
1574 Under normal circumstances you don't need to call this function.
1576 IO::AIO::max_idle $nthreads
1578 Limit the number of threads (default: 4) that are allowed to idle
1579 (i.e., threads that did not get a request to process within the
1580 idle timeout (default: 10 seconds). That means if a thread becomes
1581 idle while $nthreads other threads are also idle, it will free its
1582 resources and exit.
1583 This is useful when you allow a large number of threads (e.g. 100
1585 or 1000) to allow for extremely high load situations, but want to
1586 free resources under normal circumstances (1000 threads can easily
1587 consume 30MB of RAM).
1588 The default is probably ok in most situations, especially if thread
1590 creation is fast. If thread creation is very slow on your system
1591 you might want to use larger values.
1592 IO::AIO::idle_timeout $seconds
1594 Sets the minimum idle timeout (default 10) after which worker
1595 threads are allowed to exit. SEe "IO::AIO::max_idle".
1596 IO::AIO::max_outstanding $maxreqs
1598 Sets the maximum number of outstanding requests to $nreqs. If you
1599 do queue up more than this number of requests, the next call to
1600 "IO::AIO::poll_cb" (and other functions calling "poll_cb", such as
1601 "IO::AIO::flush" or "IO::AIO::poll") will block until the limit is
1602 no longer exceeded.
1603 In other words, this setting does not enforce a queue limit, but
1605 can be used to make poll functions block if the limit is exceeded.
1606 This is a very bad function to use in interactive programs because
1608 it blocks, and a bad way to reduce concurrency because it is
1609 inexact: Better use an "aio_group" together with a feed callback.
1610 Its main use is in scripts without an event loop - when you want to
1612 stat a lot of files, you can write something like this:
1613 IO::AIO::max_outstanding 32;
1615 for my $path (...) {
1617 aio_stat $path , ...;
1618 IO::AIO::poll_cb;
1619 }
1620 IO::AIO::flush;
1622 The call to "poll_cb" inside the loop will normally return
1624 instantly, but as soon as more thna 32 reqeusts are in-flight, it
1625 will block until some requests have been handled. This keeps the
1626 loop from pushing a large number of "aio_stat" requests onto the
1627 queue.
1628 The default value for "max_outstanding" is very large, so there is
1630 no practical limit on the number of outstanding requests.
1631 STATISTICAL INFORMATION
1633 IO::AIO::nreqs
1635 Returns the number of requests currently in the ready, execute or
1636 pending states (i.e. for which their callback has not been invoked
1637 yet).
1638 Example: wait till there are no outstanding requests anymore:
1640 IO::AIO::poll_wait, IO::AIO::poll_cb
1642 while IO::AIO::nreqs;
1643 IO::AIO::nready
1645 Returns the number of requests currently in the ready state (not
1646 yet executed).
1647 IO::AIO::npending
1649 Returns the number of requests currently in the pending state
1650 (executed, but not yet processed by poll_cb).
1651 SUBSECOND STAT TIME ACCESS
1653 Both "aio_stat"/"aio_lstat" and perl's "stat"/"lstat" functions can
1655 generally find access/modification and change times with subsecond time
1656 accuracy of the system supports it, but perl's built-in functions only
1657 return the integer part.
1658 The following functions return the timestamps of the most recent stat
1660 with subsecond precision on most systems and work both after
1661 "aio_stat"/"aio_lstat" and perl's "stat"/"lstat" calls. Their return
1662 value is only meaningful after a successful "stat"/"lstat" call, or
1663 during/after a successful "aio_stat"/"aio_lstat" callback.
1664 This is similar to the Time::HiRes "stat" functions, but can return
1666 full resolution without rounding and work with standard perl "stat",
1667 alleviating the need to call the special "Time::HiRes" functions, which
1668 do not act like their perl counterparts.
1669 On operating systems or file systems where subsecond time resolution is
1671 not supported or could not be detected, a fractional part of 0 is
1672 returned, so it is always safe to call these functions.
1673 $seconds = IO::AIO::st_atime, IO::AIO::st_mtime, IO::AIO::st_ctime,
1675 IO::AIO::st_btime
1676 Return the access, modication, change or birth time, respectively,
1677 including fractional part. Due to the limited precision of floating
1678 point, the accuracy on most platforms is only a bit better than
1679 milliseconds for times around now - see the nsec function family,
1680 below, for full accuracy.
1681 File birth time is only available when the OS and perl support it
1683 (on FreeBSD and NetBSD at the time of this writing, although
1684 support is adaptive, so if your OS/perl gains support, IO::AIO can
1685 take advantage of it). On systems where it isn't available, 0 is
1686 currently returned, but this might change to "undef" in a future
1687 version.
1688 ($atime, $mtime, $ctime, $btime, ...) = IO::AIO::st_xtime
1690 Returns access, modification, change and birth time all in one go,
1691 and maybe more times in the future version.
1692 $nanoseconds = IO::AIO::st_atimensec, IO::AIO::st_mtimensec,
1694 IO::AIO::st_ctimensec, IO::AIO::st_btimensec
1695 Return the fractional access, modifcation, change or birth time, in
1696 nanoseconds, as an integer in the range 0 to 999999999.
1697 Note that no accessors are provided for access, modification and
1699 change times - you need to get those from "stat _" if required
1700 ("int IO::AIO::st_atime" and so on will not generally give you the
1701 correct value).
1702 $seconds = IO::AIO::st_btimesec
1704 The (integral) seconds part of the file birth time, if available.
1705 ($atime, $mtime, $ctime, $btime, ...) = IO::AIO::st_xtimensec
1707 Like the functions above, but returns all four times in one go (and
1708 maybe more in future versions).
1709 $counter = IO::AIO::st_gen
1711 Returns the generation counter (in practice this is just a random
1712 number) of the file. This is only available on platforms which have
1713 this member in their "struct stat" (most BSDs at the time of this
1714 writing) and generally only to the root usert. If unsupported, 0 is
1715 returned, but this might change to "undef" in a future version.
1716 Example: print the high resolution modification time of /etc, using
1718 "stat", and "IO::AIO::aio_stat".
1719 if (stat "/etc") {
1721 printf "stat(/etc) mtime: %f\n", IO::AIO::st_mtime;
1722 }
1723 IO::AIO::aio_stat "/etc", sub {
1725 $_[0]
1726 and return;
1727 printf "aio_stat(/etc) mtime: %d.%09d\n", (stat _)[9], IO::AIO::st_mtimensec;
1729 };
1730 IO::AIO::flush;
1732 Output of the awbove on my system, showing reduced and full accuracy:
1734 stat(/etc) mtime: 1534043702.020808
1736 aio_stat(/etc) mtime: 1534043702.020807792
1737 MISCELLANEOUS FUNCTIONS
1739 IO::AIO implements some functions that are useful when you want to use
1741 some "Advanced I/O" function not available to in Perl, without going
1742 the "Asynchronous I/O" route. Many of these have an asynchronous
1743 "aio_*" counterpart.
1744 $numfd = IO::AIO::get_fdlimit
1746 This function is EXPERIMENTAL and subject to change.
1747 Tries to find the current file descriptor limit and returns it, or
1749 "undef" and sets $! in case of an error. The limit is one larger
1750 than the highest valid file descriptor number.
1751 IO::AIO::min_fdlimit [$numfd]
1753 This function is EXPERIMENTAL and subject to change.
1754 Try to increase the current file descriptor limit(s) to at least
1756 $numfd by changing the soft or hard file descriptor resource limit.
1757 If $numfd is missing, it will try to set a very high limit,
1758 although this is not recommended when you know the actual minimum
1759 that you require.
1760 If the limit cannot be raised enough, the function makes a best-
1762 effort attempt to increase the limit as much as possible, using
1763 various tricks, while still failing. You can query the resulting
1764 limit using "IO::AIO::get_fdlimit".
1765 If an error occurs, returns "undef" and sets $!, otherwise returns
1767 true.
1768 IO::AIO::sendfile $ofh, $ifh, $offset, $count
1770 Calls the "eio_sendfile_sync" function, which is like
1771 "aio_sendfile", but is blocking (this makes most sense if you know
1772 the input data is likely cached already and the output filehandle
1773 is set to non-blocking operations).
1774 Returns the number of bytes copied, or "-1" on error.
1776 IO::AIO::fadvise $fh, $offset, $len, $advice
1778 Simply calls the "posix_fadvise" function (see its manpage for
1779 details). The following advice constants are available:
1780 "IO::AIO::FADV_NORMAL", "IO::AIO::FADV_SEQUENTIAL",
1781 "IO::AIO::FADV_RANDOM", "IO::AIO::FADV_NOREUSE",
1782 "IO::AIO::FADV_WILLNEED", "IO::AIO::FADV_DONTNEED".
1783 On systems that do not implement "posix_fadvise", this function
1785 returns ENOSYS, otherwise the return value of "posix_fadvise".
1786 IO::AIO::madvise $scalar, $offset, $len, $advice
1788 Simply calls the "posix_madvise" function (see its manpage for
1789 details). The following advice constants are available:
1790 "IO::AIO::MADV_NORMAL", "IO::AIO::MADV_SEQUENTIAL",
1791 "IO::AIO::MADV_RANDOM", "IO::AIO::MADV_WILLNEED",
1792 "IO::AIO::MADV_DONTNEED".
1793 If $offset is negative, counts from the end. If $length is
1795 negative, the remaining length of the $scalar is used. If possible,
1796 $length will be reduced to fit into the $scalar.
1797 On systems that do not implement "posix_madvise", this function
1799 returns ENOSYS, otherwise the return value of "posix_madvise".
1800 IO::AIO::mprotect $scalar, $offset, $len, $protect
1802 Simply calls the "mprotect" function on the preferably AIO::mmap'ed
1803 $scalar (see its manpage for details). The following protect
1804 constants are available: "IO::AIO::PROT_NONE",
1805 "IO::AIO::PROT_READ", "IO::AIO::PROT_WRITE", "IO::AIO::PROT_EXEC".
1806 If $offset is negative, counts from the end. If $length is
1808 negative, the remaining length of the $scalar is used. If possible,
1809 $length will be reduced to fit into the $scalar.
1810 On systems that do not implement "mprotect", this function returns
1812 ENOSYS, otherwise the return value of "mprotect".
1813 IO::AIO::mmap $scalar, $length, $prot, $flags, $fh[, $offset]
1815 Memory-maps a file (or anonymous memory range) and attaches it to
1816 the given $scalar, which will act like a string scalar. Returns
1817 true on success, and false otherwise.
1818 The scalar must exist, but its contents do not matter - this means
1820 you cannot use a nonexistant array or hash element. When in doubt,
1821 "undef" the scalar first.
1822 The only operations allowed on the mmapped scalar are
1824 "substr"/"vec", which don't change the string length, and most
1825 read-only operations such as copying it or searching it with
1826 regexes and so on.
1827 Anything else is unsafe and will, at best, result in memory leaks.
1829 The memory map associated with the $scalar is automatically removed
1831 when the $scalar is undef'd or destroyed, or when the
1832 "IO::AIO::mmap" or "IO::AIO::munmap" functions are called on it.
1833 This calls the "mmap"(2) function internally. See your system's
1835 manual page for details on the $length, $prot and $flags
1836 parameters.
1837 The $length must be larger than zero and smaller than the actual
1839 filesize.
1840 $prot is a combination of "IO::AIO::PROT_NONE",
1842 "IO::AIO::PROT_EXEC", "IO::AIO::PROT_READ" and/or
1843 "IO::AIO::PROT_WRITE",
1844 $flags can be a combination of "IO::AIO::MAP_SHARED" or
1846 "IO::AIO::MAP_PRIVATE", or a number of system-specific flags (when
1847 not available, the are 0): "IO::AIO::MAP_ANONYMOUS" (which is set
1848 to "MAP_ANON" if your system only provides this constant),
1849 "IO::AIO::MAP_LOCKED", "IO::AIO::MAP_NORESERVE",
1850 "IO::AIO::MAP_POPULATE", "IO::AIO::MAP_NONBLOCK",
1851 "IO::AIO::MAP_FIXED", "IO::AIO::MAP_GROWSDOWN",
1852 "IO::AIO::MAP_32BIT", "IO::AIO::MAP_HUGETLB" or
1853 "IO::AIO::MAP_STACK".
1854 If $fh is "undef", then a file descriptor of "-1" is passed.
1856 $offset is the offset from the start of the file - it generally
1858 must be a multiple of "IO::AIO::PAGESIZE" and defaults to 0.
1859 Example:
1861 use Digest::MD5;
1863 use IO::AIO;
1864 open my $fh, "<verybigfile"
1866 or die "$!";
1867 IO::AIO::mmap my $data, -s $fh, IO::AIO::PROT_READ, IO::AIO::MAP_SHARED, $fh
1869 or die "verybigfile: $!";
1870 my $fast_md5 = md5 $data;
1872 IO::AIO::munmap $scalar
1874 Removes a previous mmap and undefines the $scalar.
1875 IO::AIO::mremap $scalar, $new_length, $flags = MREMAP_MAYMOVE[,
1877 $new_address = 0]
1878 Calls the Linux-specific mremap(2) system call. The $scalar must
1879 have been mapped by "IO::AIO::mmap", and $flags must currently
1880 either be 0 or "IO::AIO::MREMAP_MAYMOVE".
1881 Returns true if successful, and false otherwise. If the underlying
1883 mmapped region has changed address, then the true value has the
1884 numerical value 1, otherwise it has the numerical value 0:
1885 my $success = IO::AIO::mremap $mmapped, 8192, IO::AIO::MREMAP_MAYMOVE
1887 or die "mremap: $!";
1888 if ($success*1) {
1890 warn "scalar has chanegd address in memory\n";
1891 }
1892 "IO::AIO::MREMAP_FIXED" and the $new_address argument are currently
1894 implemented, but not supported and might go away in a future
1895 version.
1896 On systems where this call is not supported or is not emulated,
1898 this call returns falls and sets $! to "ENOSYS".
1899 IO::AIO::mlockall $flags
1901 Calls the "eio_mlockall_sync" function, which is like
1902 "aio_mlockall", but is blocking.
1903 IO::AIO::munlock $scalar, $offset = 0, $length = undef
1905 Calls the "munlock" function, undoing the effects of a previous
1906 "aio_mlock" call (see its description for details).
1907 IO::AIO::munlockall
1909 Calls the "munlockall" function.
1910 On systems that do not implement "munlockall", this function
1912 returns ENOSYS, otherwise the return value of "munlockall".
1913 IO::AIO::splice $r_fh, $r_off, $w_fh, $w_off, $length, $flags
1915 Calls the GNU/Linux splice(2) syscall, if available. If $r_off or
1916 $w_off are "undef", then "NULL" is passed for these, otherwise they
1917 should be the file offset.
1918 $r_fh and $w_fh should not refer to the same file, as splice might
1920 silently corrupt the data in this case.
1921 The following symbol flag values are available:
1923 "IO::AIO::SPLICE_F_MOVE", "IO::AIO::SPLICE_F_NONBLOCK",
1924 "IO::AIO::SPLICE_F_MORE" and "IO::AIO::SPLICE_F_GIFT".
1925 See the splice(2) manpage for details.
1927 IO::AIO::tee $r_fh, $w_fh, $length, $flags
1929 Calls the GNU/Linux tee(2) syscall, see its manpage and the
1930 description for "IO::AIO::splice" above for details.
1931 $actual_size = IO::AIO::pipesize $r_fh[, $new_size]
1933 Attempts to query or change the pipe buffer size. Obviously works
1934 only on pipes, and currently works only on GNU/Linux systems, and
1935 fails with "-1"/"ENOSYS" everywhere else. If anybody knows how to
1936 influence pipe buffer size on other systems, drop me a note.
1937 ($rfh, $wfh) = IO::AIO::pipe2 [$flags]
1939 This is a direct interface to the Linux pipe2(2) system call. If
1940 $flags is missing or 0, then this should be the same as a call to
1941 perl's built-in "pipe" function and create a new pipe, and works on
1942 systems that lack the pipe2 syscall. On win32, this case invokes
1943 "_pipe (..., 4096, O_BINARY)".
1944 If $flags is non-zero, it tries to invoke the pipe2 system call
1946 with the given flags (Linux 2.6.27, glibc 2.9).
1947 On success, the read and write file handles are returned.
1949 On error, nothing will be returned. If the pipe2 syscall is missing
1951 and $flags is non-zero, fails with "ENOSYS".
1952 Please refer to pipe2(2) for more info on the $flags, but at the
1954 time of this writing, "IO::AIO::O_CLOEXEC", "IO::AIO::O_NONBLOCK"
1955 and "IO::AIO::O_DIRECT" (Linux 3.4, for packet-based pipes) were
1956 supported.
1957 Example: create a pipe race-free w.r.t. threads and fork:
1959 my ($rfh, $wfh) = IO::AIO::pipe2 IO::AIO::O_CLOEXEC
1961 or die "pipe2: $!\n";
1962 $fh = IO::AIO::memfd_create $pathname[, $flags]
1964 This is a direct interface to the Linux memfd_create(2) system
1965 call. The (unhelpful) default for $flags is 0, but your default
1966 should be "IO::AIO::MFD_CLOEXEC".
1967 On success, the new memfd filehandle is returned, otherwise returns
1969 "undef". If the memfd_create syscall is missing, fails with
1970 "ENOSYS".
1971 Please refer to memfd_create(2) for more info on this call.
1973 The following $flags values are available: "IO::AIO::MFD_CLOEXEC",
1975 "IO::AIO::MFD_ALLOW_SEALING" and "IO::AIO::MFD_HUGETLB".
1976 Example: create a new memfd.
1978 my $fh = IO::AIO::memfd_create "somenameforprocfd", IO::AIO::MFD_CLOEXEC
1980 or die "m,emfd_create: $!\n";
1981 =item $fh = IO::AIO::eventfd [$initval, [$flags]]
1982 This is a direct interface to the Linux eventfd(2) system call. The
1984 (unhelpful) defaults for $initval and $flags are 0 for both.
1985 On success, the new eventfd filehandle is returned, otherwise
1987 returns "undef". If the eventfd syscall is missing, fails with
1988 "ENOSYS".
1989 Please refer to eventfd(2) for more info on this call.
1991 The following symbol flag values are available:
1993 "IO::AIO::EFD_CLOEXEC", "IO::AIO::EFD_NONBLOCK" and
1994 "IO::AIO::EFD_SEMAPHORE" (Linux 2.6.30).
1995 Example: create a new eventfd filehandle:
1997 $fh = IO::AIO::eventfd 0, IO::AIO::EFD_CLOEXEC
1999 or die "eventfd: $!\n";
2000 $fh = IO::AIO::timerfd_create $clockid[, $flags]
2002 This is a direct interface to the Linux timerfd_create(2) system
2003 call. The (unhelpful) default for $flags is 0, but your default
2004 should be "IO::AIO::TFD_CLOEXEC".
2005 On success, the new timerfd filehandle is returned, otherwise
2007 returns "undef". If the timerfd_create syscall is missing, fails
2008 with "ENOSYS".
2009 Please refer to timerfd_create(2) for more info on this call.
2011 The following $clockid values are available:
2013 "IO::AIO::CLOCK_REALTIME", "IO::AIO::CLOCK_MONOTONIC"
2014 "IO::AIO::CLOCK_CLOCK_BOOTTIME" (Linux 3.15)
2015 "IO::AIO::CLOCK_CLOCK_REALTIME_ALARM" (Linux 3.11) and
2016 "IO::AIO::CLOCK_CLOCK_BOOTTIME_ALARM" (Linux 3.11).
2017 The following $flags values are available (Linux 2.6.27):
2019 "IO::AIO::TFD_NONBLOCK" and "IO::AIO::TFD_CLOEXEC".
2020 Example: create a new timerfd and set it to one-second repeated
2022 alarms, then wait for two alarms:
2023 my $fh = IO::AIO::timerfd_create IO::AIO::CLOCK_BOOTTIME, IO::AIO::TFD_CLOEXEC
2025 or die "timerfd_create: $!\n";
2026 defined IO::AIO::timerfd_settime $fh, 0, 1, 1
2028 or die "timerfd_settime: $!\n";
2029 for (1..2) {
2031 8 == sysread $fh, my $buf, 8
2032 or die "timerfd read failure\n";
2033 printf "number of expirations (likely 1): %d\n",
2035 unpack "Q", $buf;
2036 }
2037 ($cur_interval, $cur_value) = IO::AIO::timerfd_settime $fh, $flags,
2039 $new_interval, $nbw_value
2040 This is a direct interface to the Linux timerfd_settime(2) system
2041 call. Please refer to its manpage for more info on this call.
2042 The new itimerspec is specified using two (possibly fractional)
2044 second values, $new_interval and $new_value).
2045 On success, the current interval and value are returned (as per
2047 "timerfd_gettime"). On failure, the empty list is returned.
2048 The following $flags values are available:
2050 "IO::AIO::TFD_TIMER_ABSTIME" and
2051 "IO::AIO::TFD_TIMER_CANCEL_ON_SET".
2052 See "IO::AIO::timerfd_create" for a full example.
2054 ($cur_interval, $cur_value) = IO::AIO::timerfd_gettime $fh
2056 This is a direct interface to the Linux timerfd_gettime(2) system
2057 call. Please refer to its manpage for more info on this call.
2058 On success, returns the current values of interval and value for
2060 the given timerfd (as potentially fractional second values). On
2061 failure, the empty list is returned.
2062
2064 It is recommended to use AnyEvent::AIO to integrate IO::AIO
2065 automatically into many event loops:
2066 # AnyEvent integration (EV, Event, Glib, Tk, POE, urxvt, pureperl...)
2068 use AnyEvent::AIO;
2069 You can also integrate IO::AIO manually into many event loops, here are
2071 some examples of how to do this:
2072 # EV integration
2074 my $aio_w = EV::io IO::AIO::poll_fileno, EV::READ, \&IO::AIO::poll_cb;
2075 # Event integration
2077 Event->io (fd => IO::AIO::poll_fileno,
2078 poll => 'r',
2079 cb => \&IO::AIO::poll_cb);
2080 # Glib/Gtk2 integration
2082 add_watch Glib::IO IO::AIO::poll_fileno,
2083 in => sub { IO::AIO::poll_cb; 1 };
2084 # Tk integration
2086 Tk::Event::IO->fileevent (IO::AIO::poll_fileno, "",
2087 readable => \&IO::AIO::poll_cb);
2088 # Danga::Socket integration
2090 Danga::Socket->AddOtherFds (IO::AIO::poll_fileno =>
2091 \&IO::AIO::poll_cb);
2092 FORK BEHAVIOUR
2094 Usage of pthreads in a program changes the semantics of fork
2095 considerably. Specifically, only async-safe functions can be called
2096 after fork. Perl doesn't know about this, so in general, you cannot
2097 call fork with defined behaviour in perl if pthreads are involved.
2098 IO::AIO uses pthreads, so this applies, but many other extensions and
2099 (for inexplicable reasons) perl itself often is linked against
2100 pthreads, so this limitation applies to quite a lot of perls.
2101 This module no longer tries to fight your OS, or POSIX. That means
2103 IO::AIO only works in the process that loaded it. Forking is fully
2104 supported, but using IO::AIO in the child is not.
2105 You might get around by not using IO::AIO before (or after) forking.
2107 You could also try to call the IO::AIO::reinit function in the child:
2108 IO::AIO::reinit
2110 Abandons all current requests and I/O threads and simply
2111 reinitialises all data structures. This is not an operation
2112 supported by any standards, but happens to work on GNU/Linux and
2113 some newer BSD systems.
2114 The only reasonable use for this function is to call it after
2116 forking, if "IO::AIO" was used in the parent. Calling it while
2117 IO::AIO is active in the process will result in undefined
2118 behaviour. Calling it at any time will also result in any undefined
2119 (by POSIX) behaviour.
2120 LINUX-SPECIFIC CALLS
2122 When a call is documented as "linux-specific" then this means it
2123 originated on GNU/Linux. "IO::AIO" will usually try to autodetect the
2124 availability and compatibility of such calls regardless of the platform
2125 it is compiled on, so platforms such as FreeBSD which often implement
2126 these calls will work. When in doubt, call them and see if they fail
2127 wth "ENOSYS".
2128 MEMORY USAGE
2130 Per-request usage:
2131 Each aio request uses - depending on your architecture - around 100-200
2133 bytes of memory. In addition, stat requests need a stat buffer
2134 (possibly a few hundred bytes), readdir requires a result buffer and so
2135 on. Perl scalars and other data passed into aio requests will also be
2136 locked and will consume memory till the request has entered the done
2137 state.
2138 This is not awfully much, so queuing lots of requests is not usually a
2140 problem.
2141 Per-thread usage:
2143 In the execution phase, some aio requests require more memory for
2145 temporary buffers, and each thread requires a stack and other data
2146 structures (usually around 16k-128k, depending on the OS).
2147
2149 Known bugs will be fixed in the next release :)
2150
2152 Calls that try to "import" foreign memory areas (such as
2153 "IO::AIO::mmap" or "IO::AIO::aio_slurp") do not work with generic
2154 lvalues, such as non-created hash slots or other scalars I didn't think
2155 of. It's best to avoid such and either use scalar variables or making
2156 sure that the scalar exists (e.g. by storing "undef") and isn't "funny"
2157 (e.g. tied).
2158 I am not sure anything can be done about this, so this is considered a
2160 known issue, rather than a bug.
2161
2163 AnyEvent::AIO for easy integration into event loops, Coro::AIO for a
2164 more natural syntax.
2165
2167 Marc Lehmann <schmorp@schmorp.de>
2168 http://home.schmorp.de/
2169perl v5.28.1 2019-04-03 AIO(3)