1AnyEvent::IO(3)       User Contributed Perl Documentation      AnyEvent::IO(3)
2
3
4

NAME

6       AnyEvent::IO - the DBI of asynchronous I/O implementations
7

SYNOPSIS

9          use AnyEvent::IO;
10
11          # load /etc/passwd, call callback with the file data when done.
12          aio_load "/etc/passwd", sub {
13             my ($data) = @_
14                or return AE::log error => "/etc/passwd: $!";
15
16             warn "/etc/passwd contains ", ($data =~ y/://) , " colons.\n";
17          };
18
19          # the rest of the SYNOPSIS does the same, but with individual I/O calls
20
21          # also import O_XXX flags
22          use AnyEvent::IO qw(:DEFAULT :flags);
23
24          my $filedata = AE::cv;
25
26          # first open the file
27          aio_open "/etc/passwd", O_RDONLY, 0, sub {
28             my ($fh) = @_
29                or return AE::log error => "/etc/passwd: $!";
30
31             # now stat the file to get the size
32             aio_stat $fh, sub {
33                @_
34                   or return AE::log error => "/etc/passwd: $!";
35
36                my $size = -s _;
37
38                # now read all the file data
39                aio_read $fh, $size, sub {
40                   my ($data) = @_
41                      or return AE::log error => "/etc/passwd: $!";
42
43                   $size == length $data
44                      or return AE::log error => "/etc/passwd: short read, file changed?";
45
46                   # mostly the same as aio_load, above - $data contains
47                   # the file contents now.
48                   $filedata->($data);
49                };
50             };
51          };
52
53          my $passwd = $filedata->recv;
54          warn length $passwd, " octets.\n";
55

DESCRIPTION

57       This module provides functions that do I/O in an asynchronous fashion.
58       It is to I/O the same as AnyEvent is to event libraries - it only
59       interfaces to other implementations or to a portable pure-perl
60       implementation (which does not, however, do asynchronous I/O).
61
62       The only other implementation that is supported (or even known to the
63       author) is IO::AIO, which is used automatically when it can be loaded
64       (via AnyEvent::AIO, which also needs to be installed). If it is not
65       available, then AnyEvent::IO falls back to its synchronous pure-perl
66       implementation.
67
68       Unlike AnyEvent, which model to use is currently decided at module load
69       time, not at first use. Future releases might change this.
70
71   RATIONALE
72       While disk I/O often seems "instant" compared to, say, socket I/O,
73       there are many situations where your program can block for extended
74       time periods when doing disk I/O. For example, you access a disk on an
75       NFS server and it is gone - can take ages to respond again, if ever. Or
76       your system is extremely busy because it creates or restores a backup -
77       reading data from disk can then take seconds. Or you use Linux, which
78       for so many years has a close-to-broken VM/IO subsystem that can often
79       induce minutes or more of delay for disk I/O, even under what I would
80       consider light I/O loads.
81
82       Whatever the situation, some programs just can't afford to block for
83       long times (say, half a second or more), because they need to respond
84       as fast as possible.
85
86       For those cases, you need asynchronous I/O.
87
88       The problem is, AnyEvent itself sometimes reads disk files (for
89       example, when looking at /etc/hosts), and under the above situations,
90       this can bring your program to a complete halt even if your program
91       otherwise takes care to only use asynchronous I/O for everything (e.g.
92       by using IO::AIO).
93
94       On the other hand, requiring IO::AIO for AnyEvent is clearly
95       impossible, as AnyEvent promises to stay pure-perl, and the overhead of
96       IO::AIO for small programs would be immense, especially when
97       asynchronous I/O isn't even needed.
98
99       Clearly, this calls for an abstraction layer, and that is what you are
100       looking at right now :-)
101
102   ASYNCHRONOUS VS. NON-BLOCKING
103       Many people are continuously confused on what the difference is between
104       asynchronous I/O and non-blocking I/O. In fact, those two terms are not
105       well defined, which often makes it hard to even talk about the
106       difference. Here is a short guideline that should leave you less
107       confused. It only talks about read operations, but the reasoning works
108       with other I/O operations as well.
109
110       Non-blocking I/O means that data is delivered by some external means,
111       automatically - that is, something pushes data towards your file
112       handle, without you having to do anything. Non-blocking means that if
113       your operating system currently has no data (or EOF, or some error)
114       available for you, it will not wait ("block") as it would normally do,
115       but immediately return with an error (e.g. "EWOULDBLOCK" - "I would
116       have blocked, but you forbid it").
117
118       Your program can then wait for data to arrive by other means, for
119       example, an I/O watcher which tells you when to re-attempt the read,
120       after which it can try to read again, and so on.
121
122       Often, you would expect this to work for disk files as well - if the
123       data isn't already in memory, one might want to wait for it and then
124       re-attempt the read for example. While this is sound reasoning, the
125       POSIX API does not support this, because disk drives and file systems
126       do not send data "on their own", and more so, the OS already knows that
127       data is there, it doesn't need to "wait" until it arrives from some
128       external entity, it only needs to transfer the data from disk to your
129       memory buffer.
130
131       So basically, while the concept is sound, the existing OS APIs do not
132       support this. Therefore, it makes no sense to switch a disk file handle
133       into non-blocking mode - it will behave exactly the same as in blocking
134       mode, namely it will block until the data has been read from the disk.
135
136       The alternative to non-blocking I/O that actually works with disk files
137       is usually called asynchronous I/O. Asynchronous, because the actual
138       I/O is done while your program does something else: there is no need to
139       call the read function to see if data is there, you only order the read
140       once, and it will notify you when the read has finished and the data is
141       your buffer - all the work is done in the background.
142
143       This works with disk files, and even with sockets and other sources. It
144       is, however, not very efficient when used with sources that could be
145       driven in a non-blocking way, because it usually has higher overhead in
146       the OS than non-blocking I/O, because it ties memory buffers for a
147       potentially unlimited time and often only a limited number of
148       operations can be done in parallel.
149
150       That's why asynchronous I/O makes most sense when confronted with disk
151       files, and non-blocking I/O only makes sense with sockets, pipes and
152       similar streaming sources.
153

IMPORT TAGS

155       By default, this module exports all "aio_"xxx functions. In addition,
156       the following import tags can be used:
157
158          :aio       all aio_* functions, same as :DEFAULT
159          :flags     the fcntl open flags (O_CREAT, O_RDONLY, ...)
160

API NOTES

162       The functions in this module are not meant to be the most versatile or
163       the highest-performers (they are not very slow either, of course). They
164       are primarily meant to give users of your code the option to do the I/O
165       asynchronously (by installing IO::AIO and AnyEvent::AIO), without
166       adding a dependency on those modules.
167
168   NAMING
169       All the functions in this module implement an I/O operation, usually
170       with the same or similar name as the Perl built-in that they mimic, but
171       with an "aio_" prefix. If you like you can think of the "aio_"xxx
172       functions as "AnyEvent I/O" or "Asynchronous I/O" variants of Perl
173       built-ins.
174
175   CALLING CONVENTIONS AND ERROR REPORTING
176       Each function expects a callback as their last argument. The callback
177       is usually called with the result data or result code. An error is
178       usually signalled by passing no arguments to the callback, which is
179       then free to look at $! for the error code.
180
181       This makes all of the following forms of error checking valid:
182
183          aio_open ...., sub {
184             my $fh = shift   # scalar assignment - will assign undef on error
185                or return AE::log error => "...";
186
187             my ($fh) = @_    # list assignment - will be 0 elements on error
188                or return AE::log error => "...";
189
190             @_               # check the number of elements directly
191                or return AE::log error => "...";
192
193   CAVEAT: RELATIVE PATHS
194       When a path is specified, this path must be an absolute path, unless
195       you make certain that nothing in your process calls "chdir" or an
196       equivalent function while the request executes.
197
198   CAVEAT: OTHER SHARED STATE
199       Changing the "umask" while any requests execute that create files (or
200       otherwise rely on the current umask) results in undefined behaviour -
201       likewise changing anything else that would change the outcome, such as
202       your effective user or group ID.
203
204   CALLBACKS MIGHT BE CALLED BEFORE FUNCTION RETURNS TO CALLER
205       Unlike other functions in the AnyEvent module family, these functions
206       may call your callback instantly, before returning. This should not be
207       a real problem, as these functions never return anything useful.
208
209   BEHAVIOUR AT PROGRAM EXIT
210       Both AnyEvent::IO::Perl and AnyEvent::IO::IOAIO implementations make
211       sure that operations that have started will be finished on a clean
212       programs exit. That makes programs work that start some I/O operations
213       and then exit. For example this complete program:
214
215          use AnyEvent::IO;
216
217          aio_stat "path1", sub {
218             aio_stat "path2", sub {
219                warn "both stats done\n";
220             };
221          };
222
223       Starts a "stat" operation and then exits by "falling off the end" of
224       the program. Nevertheless, both "stat" operations will be executed, as
225       AnyEvent::IO waits for all outstanding requests to finish and you can
226       start new requests from request callbacks.
227
228       In fact, since AnyEvent::IO::Perl is currently synchronous, the program
229       will do both stats before falling off the end, but with
230       AnyEvent::IO::IOAIO, the program first falls of the end, then the stats
231       are executed.
232
233       While not guaranteed, this behaviour will be present in future
234       versions, if reasonably possible (which is extreemly likely :).
235

GLOBAL VARIABLES AND FUNCTIONS

237       $AnyEvent::IO::MODEL
238           Contains the package name of the backend I/O model in use - at the
239           moment, this is usually "AnyEvent::IO::Perl" or
240           "AnyEvent::IO::IOAIO".
241
242       aio_load $path, $cb->($data)
243           Tries to open $path and read its contents into memory (obviously,
244           should only be used on files that are "small enough"), then passes
245           them to the callback as a string.
246
247           Example: load /etc/hosts.
248
249              aio_load "/etc/hosts", sub {
250                 my ($hosts) = @_
251                    or return AE::log error => "/etc/hosts: $!";
252
253                 AE::log info => "/etc/hosts contains ", ($hosts =~ y/\n/), " lines\n";
254              };
255
256       aio_open $path, $flags, $mode, $cb->($fh)
257           Tries to open the file specified by $path with the O_XXX-flags
258           $flags (from the Fcntl module, or see below) and the mode $mode (a
259           good value is 0666 for "O_CREAT", and 0 otherwise).
260
261           The (normal, standard, perl) file handle associated with the opened
262           file is then passed to the callback.
263
264           This works very much like Perl's "sysopen" function.
265
266           Changing the "umask" while this request executes results in
267           undefined behaviour - likewise changing anything else that would
268           change the outcome, such as your effective user or group ID.
269
270           To avoid having to load Fcntl, this module provides constants for
271           "O_RDONLY", "O_WRONLY", "O_RDWR", "O_CREAT", "O_EXCL", "O_TRUNC"
272           and "O_APPEND" - you can either access them directly
273           ("AnyEvent::IO::O_RDONLY") or import them by specifying the
274           ":flags" import tag (see SYNOPSIS).
275
276           Example: securely open a file in /var/tmp, fail if it exists or is
277           a symlink.
278
279              use AnyEvent::IO qw(:flags);
280
281              aio_open "/var/tmp/mytmp$$", O_CREAT | O_EXCL | O_RDWR, 0600, sub {
282                 my ($fh) = @_
283                    or return AE::log error => "$! - denial of service attack?";
284
285                 # now we have $fh
286              };
287
288       aio_close $fh, $cb->($success)
289           Closes the file handle (yes, close can block your process
290           indefinitely) and passes a true value to the callback on success.
291
292           Due to idiosyncrasies in perl, instead of calling "close", the file
293           handle might get closed by "dup2"'ing another file descriptor over
294           it, that is, the $fh might still be open, but can be closed safely
295           afterwards and must not be used for anything.
296
297           Example: close a file handle, and dirty as we are, do not even
298           bother to check for errors.
299
300              aio_close $fh, sub { };
301
302       aio_read $fh, $length, $cb->($data)
303           Tries to read $length octets from the current position from $fh and
304           passes these bytes to $cb. Otherwise the semantics are very much
305           like those of Perl's "sysread".
306
307           If less than $length octets have been read, $data will contain only
308           those bytes actually read. At EOF, $data will be a zero-length
309           string. If an error occurs, then nothing is passed to the callback.
310
311           Obviously, multiple "aio_read"'s or "aio_write"'s at the same time
312           on file handles sharing the underlying open file description
313           results in undefined behaviour, due to sharing of the current file
314           offset (and less obviously so, because OS X is not thread safe and
315           corrupts data when you try).
316
317           Example: read 128 octets from a file.
318
319              aio_read $fh, 128, sub {
320                 my ($data) = @_
321                    or return AE::log error "read from fh: $!";
322
323                 if (length $data) {
324                    print "read ", length $data, " octets.\n";
325                 } else {
326                    print "EOF\n";
327                 }
328              };
329
330       aio_seek $fh, $offset, $whence, $callback->($offs)
331           Seeks the filehandle to the new $offset, similarly to Perl's
332           "sysseek". The $whence are the traditional values (0 to count from
333           start, 1 to count from the current position and 2 to count from the
334           end).
335
336           The resulting absolute offset will be passed to the callback on
337           success.
338
339           Example: measure the size of the file in the old-fashioned way
340           using seek.
341
342              aio_seek $fh, 0, 2, sub {
343                 my ($size) = @_
344                    or return AE::log error => "seek to end failed: $!";
345
346                 # maybe we need to seek to the beginning again?
347                 aio_seek $fh, 0, 0, sub {
348                    # now we are hopefully at the beginning
349                 };
350              };
351
352       aio_write $fh, $data, $cb->($length)
353           Tries to write the octets in $data to the current position of $fh
354           and passes the actual number of bytes written to the $cb. Otherwise
355           the semantics are very much like those of Perl's "syswrite".
356
357           If less than "length $data" octets have been written, $length will
358           reflect that. If an error occurs, then nothing is passed to the
359           callback.
360
361           Obviously, multiple "aio_read"'s or "aio_write"'s at the same time
362           on file handles sharing the underlying open file description
363           results in undefined behaviour, due to sharing of the current file
364           offset (and less obviously so, because OS X is not thread safe and
365           corrupts data when you try).
366
367       aio_truncate $fh_or_path, $new_length, $cb->($success)
368           Calls "truncate" on the path or perl file handle and passes a true
369           value to the callback on success.
370
371           Example: truncate /etc/passwd to zero length - this only works on
372           systems that support "truncate", should not be tried out for
373           obvious reasons and debian will probably open yte another security
374           bug about this example.
375
376              aio_truncate "/etc/passwd", sub {
377                 @_
378                    or return AE::log error => "/etc/passwd: $! - are you root enough?";
379              };
380
381       aio_utime $fh_or_path, $atime, $mtime, $cb->($success)
382           Calls "utime" on the path or perl file handle and passes a true
383           value to the callback on success.
384
385           The special case of both $atime and $mtime being "undef" sets the
386           times to the current time, on systems that support this.
387
388           Example: try to touch file.
389
390              aio_utime "file", undef, undef, sub { };
391
392       aio_chown $fh_or_path, $uid, $gid, $cb->($success)
393           Calls "chown" on the path or perl file handle and passes a true
394           value to the callback on success.
395
396           If $uid or $gid can be specified as "undef", in which case the uid
397           or gid of the file is not changed. This differs from Perl's "chown"
398           built-in, which wants "-1" for this.
399
400           Example: update the group of file to 0 (root), but leave the owner
401           alone.
402
403              aio_chown "file", undef, 0, sub {
404                 @_
405                    or return AE::log error => "chown 'file': $!";
406              };
407
408       aio_chmod $fh_or_path, $perms, $cb->($success)
409           Calls "chmod" on the path or perl file handle and passes a true
410           value to the callback on success.
411
412           Example: change file to be user/group/world-readable, but leave the
413           other flags alone.
414
415              aio_stat "file", sub {
416                 @_
417                    or return AE::log error => "file: $!";
418
419                 aio_chmod "file", (stat _)[2] & 07777 | 00444, sub { };
420              };
421
422       aio_stat $fh_or_path, $cb->($success)
423       aio_lstat $path, $cb->($success)
424           Calls "stat" or "lstat" on the path or perl file handle and passes
425           a true value to the callback on success.
426
427           The stat data will be available by "stat"'ing the "_" file handle
428           (e.g. "-x _", "stat _" and so on).
429
430           Example: see if we can find the number of subdirectories of /etc.
431
432              aio_stat "/etc", sub {
433                 @_
434                    or return AE::log error => "/etc: $!";
435
436                 (stat _)[3] >= 2
437                    or return AE::log warn => "/etc has low link count - non-POSIX filesystem?";
438
439                 print "/etc has ", (stat _)[3] - 2, " subdirectories.\n";
440              };
441
442       aio_link $oldpath, $newpath, $cb->($success)
443           Calls "link" on the paths and passes a true value to the callback
444           on success.
445
446           Example: link "file to file.bak, then rename file.new over file, to
447           atomically replace it.
448
449              aio_link "file", "file.bak", sub {
450                 @_
451                    or return AE::log error => "file: $!";
452
453                 aio_rename "file.new", "file", sub {
454                    @_
455                       or return AE::log error => "file.new: $!";
456
457                    print "file atomically replaced by file.new, backup file.bak\n";
458                 };
459              };
460
461       aio_symlink $oldpath, $newpath, $cb->($success)
462           Calls "symlink" on the paths and passes a true value to the
463           callback on success.
464
465           Example: create a symlink "slink containing "random data".
466
467              aio_symlink "random data", "slink", sub {
468                 @_
469                    or return AE::log error => "slink: $!";
470              };
471
472       aio_readlink $path, $cb->($target)
473           Calls "readlink" on the paths and passes the link target string to
474           the callback.
475
476           Example: read the symlink called Fyslink> and verify that it
477           contains "random data".
478
479             aio_readlink "slink", sub {
480                my ($target) = @_
481                   or return AE::log error => "slink: $!";
482
483                $target eq "random data"
484                   or AE::log critical => "omg, the world will end!";
485             };
486
487       aio_rename $oldpath, $newpath, $cb->($success)
488           Calls "rename" on the paths and passes a true value to the callback
489           on success.
490
491           See "aio_link" for an example.
492
493       aio_unlink $path, $cb->($success)
494           Tries to unlink the object at $path and passes a true value to the
495           callback on success.
496
497           Example: try to delete the file tmpfile.dat~.
498
499              aio_unlink "tmpfile.dat~", sub { };
500
501       aio_mkdir $path, $perms, $cb->($success)
502           Calls "mkdir" on the path with the given permissions $perms (when
503           in doubt, 0777 is a good value) and passes a true value to the
504           callback on success.
505
506           Example: try to create the directory subdir and leave it to
507           whoeveer comes after us to check whether it worked.
508
509              aio_mkdir "subdir", 0777, sub { };
510
511       aio_rmdir $path, $cb->($success)
512           Tries to remove the directory at $path and passes a true value to
513           the callback on success.
514
515           Example: try to remove the directory subdir and don't give a damn
516           if that fails.
517
518              aio_rmdir "subdir", sub { };
519
520       aio_readdir $path, $cb->(\@names)
521           Reads all filenames from the directory specified by $path and
522           passes them to the callback, as an array reference with the names
523           (without a path prefix). The . and .. names will be filtered out
524           first.
525
526           The ordering of the file names is undefined - backends that are
527           capable of it (e.g. IO::AIO) will return the ordering that most
528           likely is fastest to "stat" through, and furthermore put entries
529           that likely are directories first in the array.
530
531           If you need best performance in recursive directory traversal or
532           when looking at really big directories, you are advised to use
533           IO::AIO directly, specifically the "aio_readdirx" and "aio_scandir"
534           functions, which have more options to tune performance.
535
536           Example: recursively scan a directory hierarchy, silently skip
537           diretcories we couldn't read and print all others.
538
539              sub scan($); # visibility-in-next statement is not so useful these days
540              sub scan($) {
541                 my ($path) = @_;
542
543                 aio_readdir $path, sub {
544                    my ($names) = @_
545                       or return;
546
547                    print "$path\n";
548
549                    for my $name (@$names) {
550                       aio_lstat "$path/$name", sub {
551                          scan "$path/$name"
552                             if -d _;
553                       };
554                    }
555                 };
556              }
557
558              scan "/etc";
559

ENVIRONMENT VARIABLES

561       See the description of "PERL_ANYEVENT_IO_MODEL" in the AnyEvent
562       manpage.
563

AUTHOR

565        Marc Lehmann <schmorp@schmorp.de>
566        http://anyevent.schmorp.de
567
568
569
570perl v5.32.0                      2020-07-28                   AnyEvent::IO(3)
Impressum