1PERLIOL(1) Perl Programmers Reference Guide PERLIOL(1)
2
6 perliol - C API for Perl's implementation of IO in Layers.
7
9 /* Defining a layer ... */
10 #include <perliol.h>
11
13 This document describes the behavior and implementation of the PerlIO
14 abstraction described in perlapio when "USE_PERLIO" is defined.
15 History and Background
17 The PerlIO abstraction was introduced in perl5.003_02 but languished as
18 just an abstraction until perl5.7.0. However during that time a number
19 of perl extensions switched to using it, so the API is mostly fixed to
20 maintain (source) compatibility.
21 The aim of the implementation is to provide the PerlIO API in a
23 flexible and platform neutral manner. It is also a trial of an "Object
24 Oriented C, with vtables" approach which may be applied to Raku.
25 Basic Structure
27 PerlIO is a stack of layers.
28 The low levels of the stack work with the low-level operating system
30 calls (file descriptors in C) getting bytes in and out, the higher
31 layers of the stack buffer, filter, and otherwise manipulate the I/O,
32 and return characters (or bytes) to Perl. Terms above and below are
33 used to refer to the relative positioning of the stack layers.
34 A layer contains a "vtable", the table of I/O operations (at C level a
36 table of function pointers), and status flags. The functions in the
37 vtable implement operations like "open", "read", and "write".
38 When I/O, for example "read", is requested, the request goes from Perl
40 first down the stack using "read" functions of each layer, then at the
41 bottom the input is requested from the operating system services, then
42 the result is returned up the stack, finally being interpreted as Perl
43 data.
44 The requests do not necessarily go always all the way down to the
46 operating system: that's where PerlIO buffering comes into play.
47 When you do an open() and specify extra PerlIO layers to be deployed,
49 the layers you specify are "pushed" on top of the already existing
50 default stack. One way to see it is that "operating system is on the
51 left" and "Perl is on the right".
52 What exact layers are in this default stack depends on a lot of things:
54 your operating system, Perl version, Perl compile time configuration,
55 and Perl runtime configuration. See PerlIO, "PERLIO" in perlrun, and
56 open for more information.
57 binmode() operates similarly to open(): by default the specified layers
59 are pushed on top of the existing stack.
60 However, note that even as the specified layers are "pushed on top" for
62 open() and binmode(), this doesn't mean that the effects are limited to
63 the "top": PerlIO layers can be very 'active' and inspect and affect
64 layers also deeper in the stack. As an example there is a layer called
65 "raw" which repeatedly "pops" layers until it reaches the first layer
66 that has declared itself capable of handling binary data. The "pushed"
67 layers are processed in left-to-right order.
68 sysopen() operates (unsurprisingly) at a lower level in the stack than
70 open(). For example in Unix or Unix-like systems sysopen() operates
71 directly at the level of file descriptors: in the terms of PerlIO
72 layers, it uses only the "unix" layer, which is a rather thin wrapper
73 on top of the Unix file descriptors.
74 Layers vs Disciplines
76 Initial discussion of the ability to modify IO streams behaviour used
77 the term "discipline" for the entities which were added. This came (I
78 believe) from the use of the term in "sfio", which in turn borrowed it
79 from "line disciplines" on Unix terminals. However, this document (and
80 the C code) uses the term "layer".
81 This is, I hope, a natural term given the implementation, and should
83 avoid connotations that are inherent in earlier uses of "discipline"
84 for things which are rather different.
85 Data Structures
87 The basic data structure is a PerlIOl:
88 typedef struct _PerlIO PerlIOl;
90 typedef struct _PerlIO_funcs PerlIO_funcs;
91 typedef PerlIOl *PerlIO;
92 struct _PerlIO
94 {
95 PerlIOl * next; /* Lower layer */
96 PerlIO_funcs * tab; /* Functions for this layer */
97 U32 flags; /* Various flags for state */
98 };
99 A "PerlIOl *" is a pointer to the struct, and the application level
101 "PerlIO *" is a pointer to a "PerlIOl *" - i.e. a pointer to a pointer
102 to the struct. This allows the application level "PerlIO *" to remain
103 constant while the actual "PerlIOl *" underneath changes. (Compare
104 perl's "SV *" which remains constant while its "sv_any" field changes
105 as the scalar's type changes.) An IO stream is then in general
106 represented as a pointer to this linked-list of "layers".
107 It should be noted that because of the double indirection in a "PerlIO
109 *", a "&(perlio->next)" "is" a "PerlIO *", and so to some degree at
110 least one layer can use the "standard" API on the next layer down.
111 A "layer" is composed of two parts:
113 1. The functions and attributes of the "layer class".
115 2. The per-instance data for a particular handle.
117 Functions and Attributes
119 The functions and attributes are accessed via the "tab" (for table)
120 member of "PerlIOl". The functions (methods of the layer "class") are
121 fixed, and are defined by the "PerlIO_funcs" type. They are broadly the
122 same as the public "PerlIO_xxxxx" functions:
123 struct _PerlIO_funcs
125 {
126 Size_t fsize;
127 char * name;
128 Size_t size;
129 IV kind;
130 IV (*Pushed)(pTHX_ PerlIO *f,
131 const char *mode,
132 SV *arg,
133 PerlIO_funcs *tab);
134 IV (*Popped)(pTHX_ PerlIO *f);
135 PerlIO * (*Open)(pTHX_ PerlIO_funcs *tab,
136 PerlIO_list_t *layers, IV n,
137 const char *mode,
138 int fd, int imode, int perm,
139 PerlIO *old,
140 int narg, SV **args);
141 IV (*Binmode)(pTHX_ PerlIO *f);
142 SV * (*Getarg)(pTHX_ PerlIO *f, CLONE_PARAMS *param, int flags)
143 IV (*Fileno)(pTHX_ PerlIO *f);
144 PerlIO * (*Dup)(pTHX_ PerlIO *f,
145 PerlIO *o,
146 CLONE_PARAMS *param,
147 int flags)
148 /* Unix-like functions - cf sfio line disciplines */
149 SSize_t (*Read)(pTHX_ PerlIO *f, void *vbuf, Size_t count);
150 SSize_t (*Unread)(pTHX_ PerlIO *f, const void *vbuf, Size_t count);
151 SSize_t (*Write)(pTHX_ PerlIO *f, const void *vbuf, Size_t count);
152 IV (*Seek)(pTHX_ PerlIO *f, Off_t offset, int whence);
153 Off_t (*Tell)(pTHX_ PerlIO *f);
154 IV (*Close)(pTHX_ PerlIO *f);
155 /* Stdio-like buffered IO functions */
156 IV (*Flush)(pTHX_ PerlIO *f);
157 IV (*Fill)(pTHX_ PerlIO *f);
158 IV (*Eof)(pTHX_ PerlIO *f);
159 IV (*Error)(pTHX_ PerlIO *f);
160 void (*Clearerr)(pTHX_ PerlIO *f);
161 void (*Setlinebuf)(pTHX_ PerlIO *f);
162 /* Perl's snooping functions */
163 STDCHAR * (*Get_base)(pTHX_ PerlIO *f);
164 Size_t (*Get_bufsiz)(pTHX_ PerlIO *f);
165 STDCHAR * (*Get_ptr)(pTHX_ PerlIO *f);
166 SSize_t (*Get_cnt)(pTHX_ PerlIO *f);
167 void (*Set_ptrcnt)(pTHX_ PerlIO *f,STDCHAR *ptr,SSize_t cnt);
168 };
169 The first few members of the struct give a function table size for
171 compatibility check "name" for the layer, the size to "malloc" for the
172 per-instance data, and some flags which are attributes of the class as
173 whole (such as whether it is a buffering layer), then follow the
174 functions which fall into four basic groups:
175 1. Opening and setup functions
177 2. Basic IO operations
179 3. Stdio class buffering options.
181 4. Functions to support Perl's traditional "fast" access to the
183 buffer.
184 A layer does not have to implement all the functions, but the whole
186 table has to be present. Unimplemented slots can be NULL (which will
187 result in an error when called) or can be filled in with stubs to
188 "inherit" behaviour from a "base class". This "inheritance" is fixed
189 for all instances of the layer, but as the layer chooses which stubs to
190 populate the table, limited "multiple inheritance" is possible.
191 Per-instance Data
193 The per-instance data are held in memory beyond the basic PerlIOl
194 struct, by making a PerlIOl the first member of the layer's struct
195 thus:
196 typedef struct
198 {
199 struct _PerlIO base; /* Base "class" info */
200 STDCHAR * buf; /* Start of buffer */
201 STDCHAR * end; /* End of valid part of buffer */
202 STDCHAR * ptr; /* Current position in buffer */
203 Off_t posn; /* Offset of buf into the file */
204 Size_t bufsiz; /* Real size of buffer */
205 IV oneword; /* Emergency buffer */
206 } PerlIOBuf;
207 In this way (as for perl's scalars) a pointer to a PerlIOBuf can be
209 treated as a pointer to a PerlIOl.
210 Layers in action.
212 table perlio unix
213 | |
214 +-----------+ +----------+ +--------+
215 PerlIO ->| |--->| next |--->| NULL |
216 +-----------+ +----------+ +--------+
217 | | | buffer | | fd |
218 +-----------+ | | +--------+
219 | | +----------+
220 The above attempts to show how the layer scheme works in a simple case.
222 The application's "PerlIO *" points to an entry in the table(s)
223 representing open (allocated) handles. For example the first three
224 slots in the table correspond to "stdin","stdout" and "stderr". The
225 table in turn points to the current "top" layer for the handle - in
226 this case an instance of the generic buffering layer "perlio". That
227 layer in turn points to the next layer down - in this case the low-
228 level "unix" layer.
229 The above is roughly equivalent to a "stdio" buffered stream, but with
231 much more flexibility:
232 • If Unix level "read"/"write"/"lseek" is not appropriate for (say)
234 sockets then the "unix" layer can be replaced (at open time or even
235 dynamically) with a "socket" layer.
236 • Different handles can have different buffering schemes. The "top"
238 layer could be the "mmap" layer if reading disk files was quicker
239 using "mmap" than "read". An "unbuffered" stream can be implemented
240 simply by not having a buffer layer.
241 • Extra layers can be inserted to process the data as it flows
243 through. This was the driving need for including the scheme in
244 perl 5.7.0+ - we needed a mechanism to allow data to be translated
245 between perl's internal encoding (conceptually at least Unicode as
246 UTF-8), and the "native" format used by the system. This is
247 provided by the ":encoding(xxxx)" layer which typically sits above
248 the buffering layer.
249 • A layer can be added that does "\n" to CRLF translation. This layer
251 can be used on any platform, not just those that normally do such
252 things.
253 Per-instance flag bits
255 The generic flag bits are a hybrid of "O_XXXXX" style flags deduced
256 from the mode string passed to "PerlIO_open()", and state bits for
257 typical buffer layers.
258 PERLIO_F_EOF
260 End of file.
261 PERLIO_F_CANWRITE
263 Writes are permitted, i.e. opened as "w" or "r+" or "a", etc.
264 PERLIO_F_CANREAD
266 Reads are permitted i.e. opened "r" or "w+" (or even "a+" - ick).
267 PERLIO_F_ERROR
269 An error has occurred (for "PerlIO_error()").
270 PERLIO_F_TRUNCATE
272 Truncate file suggested by open mode.
273 PERLIO_F_APPEND
275 All writes should be appends.
276 PERLIO_F_CRLF
278 Layer is performing Win32-like "\n" mapped to CR,LF for output and
279 CR,LF mapped to "\n" for input. Normally the provided "crlf" layer
280 is the only layer that need bother about this. "PerlIO_binmode()"
281 will mess with this flag rather than add/remove layers if the
282 "PERLIO_K_CANCRLF" bit is set for the layers class.
283 PERLIO_F_UTF8
285 Data written to this layer should be UTF-8 encoded; data provided
286 by this layer should be considered UTF-8 encoded. Can be set on any
287 layer by ":utf8" dummy layer. Also set on ":encoding" layer.
288 PERLIO_F_UNBUF
290 Layer is unbuffered - i.e. write to next layer down should occur
291 for each write to this layer.
292 PERLIO_F_WRBUF
294 The buffer for this layer currently holds data written to it but
295 not sent to next layer.
296 PERLIO_F_RDBUF
298 The buffer for this layer currently holds unconsumed data read from
299 layer below.
300 PERLIO_F_LINEBUF
302 Layer is line buffered. Write data should be passed to next layer
303 down whenever a "\n" is seen. Any data beyond the "\n" should then
304 be processed.
305 PERLIO_F_TEMP
307 File has been "unlink()"ed, or should be deleted on "close()".
308 PERLIO_F_OPEN
310 Handle is open.
311 PERLIO_F_FASTGETS
313 This instance of this layer supports the "fast "gets"" interface.
314 Normally set based on "PERLIO_K_FASTGETS" for the class and by the
315 existence of the function(s) in the table. However a class that
316 normally provides that interface may need to avoid it on a
317 particular instance. The "pending" layer needs to do this when it
318 is pushed above a layer which does not support the interface.
319 (Perl's "sv_gets()" does not expect the streams fast "gets"
320 behaviour to change during one "get".)
321 Methods in Detail
323 fsize
324 Size_t fsize;
325 Size of the function table. This is compared against the value
327 PerlIO code "knows" as a compatibility check. Future versions may
328 be able to tolerate layers compiled against an old version of the
329 headers.
330 name
332 char * name;
333 The name of the layer whose open() method Perl should invoke on
335 open(). For example if the layer is called APR, you will call:
336 open $fh, ">:APR", ...
338 and Perl knows that it has to invoke the PerlIOAPR_open() method
340 implemented by the APR layer.
341 size
343 Size_t size;
344 The size of the per-instance data structure, e.g.:
346 sizeof(PerlIOAPR)
348 If this field is zero then "PerlIO_pushed" does not malloc anything
350 and assumes layer's Pushed function will do any required layer
351 stack manipulation - used to avoid malloc/free overhead for dummy
352 layers. If the field is non-zero it must be at least the size of
353 "PerlIOl", "PerlIO_pushed" will allocate memory for the layer's
354 data structures and link new layer onto the stream's stack. (If the
355 layer's Pushed method returns an error indication the layer is
356 popped again.)
357 kind
359 IV kind;
360 • PERLIO_K_BUFFERED
362 The layer is buffered.
364 • PERLIO_K_RAW
366 The layer is acceptable to have in a binmode(FH) stack - i.e.
368 it does not (or will configure itself not to) transform bytes
369 passing through it.
370 • PERLIO_K_CANCRLF
372 Layer can translate between "\n" and CRLF line ends.
374 • PERLIO_K_FASTGETS
376 Layer allows buffer snooping.
378 • PERLIO_K_MULTIARG
380 Used when the layer's open() accepts more arguments than usual.
382 The extra arguments should come not before the "MODE" argument.
383 When this flag is used it's up to the layer to validate the
384 args.
385 Pushed
387 IV (*Pushed)(pTHX_ PerlIO *f,const char *mode, SV *arg);
388 The only absolutely mandatory method. Called when the layer is
390 pushed onto the stack. The "mode" argument may be NULL if this
391 occurs post-open. The "arg" will be non-"NULL" if an argument
392 string was passed. In most cases this should call
393 "PerlIOBase_pushed()" to convert "mode" into the appropriate
394 "PERLIO_F_XXXXX" flags in addition to any actions the layer itself
395 takes. If a layer is not expecting an argument it need neither
396 save the one passed to it, nor provide "Getarg()" (it could perhaps
397 "Perl_warn" that the argument was un-expected).
398 Returns 0 on success. On failure returns -1 and should set errno.
400 Popped
402 IV (*Popped)(pTHX_ PerlIO *f);
403 Called when the layer is popped from the stack. A layer will
405 normally be popped after "Close()" is called. But a layer can be
406 popped without being closed if the program is dynamically managing
407 layers on the stream. In such cases "Popped()" should free any
408 resources (buffers, translation tables, ...) not held directly in
409 the layer's struct. It should also "Unread()" any unconsumed data
410 that has been read and buffered from the layer below back to that
411 layer, so that it can be re-provided to what ever is now above.
412 Returns 0 on success and failure. If "Popped()" returns true then
414 perlio.c assumes that either the layer has popped itself, or the
415 layer is super special and needs to be retained for other reasons.
416 In most cases it should return false.
417 Open
419 PerlIO * (*Open)(...);
420 The "Open()" method has lots of arguments because it combines the
422 functions of perl's "open", "PerlIO_open", perl's "sysopen",
423 "PerlIO_fdopen" and "PerlIO_reopen". The full prototype is as
424 follows:
425 PerlIO * (*Open)(pTHX_ PerlIO_funcs *tab,
427 PerlIO_list_t *layers, IV n,
428 const char *mode,
429 int fd, int imode, int perm,
430 PerlIO *old,
431 int narg, SV **args);
432 Open should (perhaps indirectly) call "PerlIO_allocate()" to
434 allocate a slot in the table and associate it with the layers
435 information for the opened file, by calling "PerlIO_push". The
436 layers is an array of all the layers destined for the "PerlIO *",
437 and any arguments passed to them, n is the index into that array of
438 the layer being called. The macro "PerlIOArg" will return a
439 (possibly "NULL") SV * for the argument passed to the layer.
440 Where a layer opens or takes ownership of a file descriptor, that
442 layer is responsible for getting the file descriptor's close-on-
443 exec flag into the correct state. The flag should be clear for a
444 file descriptor numbered less than or equal to "PL_maxsysfd", and
445 set for any file descriptor numbered higher. For thread safety,
446 when a layer opens a new file descriptor it should if possible open
447 it with the close-on-exec flag initially set.
448 The mode string is an ""fopen()"-like" string which would match the
450 regular expression "/^[I#]?[rwa]\+?[bt]?$/".
451 The 'I' prefix is used during creation of "stdin".."stderr" via
453 special "PerlIO_fdopen" calls; the '#' prefix means that this is
454 "sysopen" and that imode and perm should be passed to
455 "PerlLIO_open3"; 'r' means read, 'w' means write and 'a' means
456 append. The '+' suffix means that both reading and
457 writing/appending are permitted. The 'b' suffix means file should
458 be binary, and 't' means it is text. (Almost all layers should do
459 the IO in binary mode, and ignore the b/t bits. The ":crlf" layer
460 should be pushed to handle the distinction.)
461 If old is not "NULL" then this is a "PerlIO_reopen". Perl itself
463 does not use this (yet?) and semantics are a little vague.
464 If fd not negative then it is the numeric file descriptor fd, which
466 will be open in a manner compatible with the supplied mode string,
467 the call is thus equivalent to "PerlIO_fdopen". In this case nargs
468 will be zero. The file descriptor may have the close-on-exec flag
469 either set or clear; it is the responsibility of the layer that
470 takes ownership of it to get the flag into the correct state.
471 If nargs is greater than zero then it gives the number of arguments
473 passed to "open", otherwise it will be 1 if for example
474 "PerlIO_open" was called. In simple cases SvPV_nolen(*args) is the
475 pathname to open.
476 If a layer provides "Open()" it should normally call the "Open()"
478 method of next layer down (if any) and then push itself on top if
479 that succeeds. "PerlIOBase_open" is provided to do exactly that,
480 so in most cases you don't have to write your own "Open()" method.
481 If this method is not defined, other layers may have difficulty
482 pushing themselves on top of it during open.
483 If "PerlIO_push" was performed and open has failed, it must
485 "PerlIO_pop" itself, since if it's not, the layer won't be removed
486 and may cause bad problems.
487 Returns "NULL" on failure.
489 Binmode
491 IV (*Binmode)(pTHX_ PerlIO *f);
492 Optional. Used when ":raw" layer is pushed (explicitly or as a
494 result of binmode(FH)). If not present layer will be popped. If
495 present should configure layer as binary (or pop itself) and return
496 0. If it returns -1 for error "binmode" will fail with layer still
497 on the stack.
498 Getarg
500 SV * (*Getarg)(pTHX_ PerlIO *f,
501 CLONE_PARAMS *param, int flags);
502 Optional. If present should return an SV * representing the string
504 argument passed to the layer when it was pushed. e.g.
505 ":encoding(ascii)" would return an SvPV with value "ascii". (param
506 and flags arguments can be ignored in most cases)
507 "Dup" uses "Getarg" to retrieve the argument originally passed to
509 "Pushed", so you must implement this function if your layer has an
510 extra argument to "Pushed" and will ever be "Dup"ed.
511 Fileno
513 IV (*Fileno)(pTHX_ PerlIO *f);
514 Returns the Unix/Posix numeric file descriptor for the handle.
516 Normally "PerlIOBase_fileno()" (which just asks next layer down)
517 will suffice for this.
518 Returns -1 on error, which is considered to include the case where
520 the layer cannot provide such a file descriptor.
521 Dup
523 PerlIO * (*Dup)(pTHX_ PerlIO *f, PerlIO *o,
524 CLONE_PARAMS *param, int flags);
525 XXX: Needs more docs.
527 Used as part of the "clone" process when a thread is spawned (in
529 which case param will be non-NULL) and when a stream is being
530 duplicated via '&' in the "open".
531 Similar to "Open", returns PerlIO* on success, "NULL" on failure.
533 Read
535 SSize_t (*Read)(pTHX_ PerlIO *f, void *vbuf, Size_t count);
536 Basic read operation.
538 Typically will call "Fill" and manipulate pointers (possibly via
540 the API). "PerlIOBuf_read()" may be suitable for derived classes
541 which provide "fast gets" methods.
542 Returns actual bytes read, or -1 on an error.
544 Unread
546 SSize_t (*Unread)(pTHX_ PerlIO *f,
547 const void *vbuf, Size_t count);
548 A superset of stdio's "ungetc()". Should arrange for future reads
550 to see the bytes in "vbuf". If there is no obviously better
551 implementation then "PerlIOBase_unread()" provides the function by
552 pushing a "fake" "pending" layer above the calling layer.
553 Returns the number of unread chars.
555 Write
557 SSize_t (*Write)(PerlIO *f, const void *vbuf, Size_t count);
558 Basic write operation.
560 Returns bytes written or -1 on an error.
562 Seek
564 IV (*Seek)(pTHX_ PerlIO *f, Off_t offset, int whence);
565 Position the file pointer. Should normally call its own "Flush"
567 method and then the "Seek" method of next layer down.
568 Returns 0 on success, -1 on failure.
570 Tell
572 Off_t (*Tell)(pTHX_ PerlIO *f);
573 Return the file pointer. May be based on layers cached concept of
575 position to avoid overhead.
576 Returns -1 on failure to get the file pointer.
578 Close
580 IV (*Close)(pTHX_ PerlIO *f);
581 Close the stream. Should normally call "PerlIOBase_close()" to
583 flush itself and close layers below, and then deallocate any data
584 structures (buffers, translation tables, ...) not held directly in
585 the data structure.
586 Returns 0 on success, -1 on failure.
588 Flush
590 IV (*Flush)(pTHX_ PerlIO *f);
591 Should make stream's state consistent with layers below. That is,
593 any buffered write data should be written, and file position of
594 lower layers adjusted for data read from below but not actually
595 consumed. (Should perhaps "Unread()" such data to the lower
596 layer.)
597 Returns 0 on success, -1 on failure.
599 Fill
601 IV (*Fill)(pTHX_ PerlIO *f);
602 The buffer for this layer should be filled (for read) from layer
604 below. When you "subclass" PerlIOBuf layer, you want to use its
605 _read method and to supply your own fill method, which fills the
606 PerlIOBuf's buffer.
607 Returns 0 on success, -1 on failure.
609 Eof
611 IV (*Eof)(pTHX_ PerlIO *f);
612 Return end-of-file indicator. "PerlIOBase_eof()" is normally
614 sufficient.
615 Returns 0 on end-of-file, 1 if not end-of-file, -1 on error.
617 Error
619 IV (*Error)(pTHX_ PerlIO *f);
620 Return error indicator. "PerlIOBase_error()" is normally
622 sufficient.
623 Returns 1 if there is an error (usually when "PERLIO_F_ERROR" is
625 set), 0 otherwise.
626 Clearerr
628 void (*Clearerr)(pTHX_ PerlIO *f);
629 Clear end-of-file and error indicators. Should call
631 "PerlIOBase_clearerr()" to set the "PERLIO_F_XXXXX" flags, which
632 may suffice.
633 Setlinebuf
635 void (*Setlinebuf)(pTHX_ PerlIO *f);
636 Mark the stream as line buffered. "PerlIOBase_setlinebuf()" sets
638 the PERLIO_F_LINEBUF flag and is normally sufficient.
639 Get_base
641 STDCHAR * (*Get_base)(pTHX_ PerlIO *f);
642 Allocate (if not already done so) the read buffer for this layer
644 and return pointer to it. Return NULL on failure.
645 Get_bufsiz
647 Size_t (*Get_bufsiz)(pTHX_ PerlIO *f);
648 Return the number of bytes that last "Fill()" put in the buffer.
650 Get_ptr
652 STDCHAR * (*Get_ptr)(pTHX_ PerlIO *f);
653 Return the current read pointer relative to this layer's buffer.
655 Get_cnt
657 SSize_t (*Get_cnt)(pTHX_ PerlIO *f);
658 Return the number of bytes left to be read in the current buffer.
660 Set_ptrcnt
662 void (*Set_ptrcnt)(pTHX_ PerlIO *f,
663 STDCHAR *ptr, SSize_t cnt);
664 Adjust the read pointer and count of bytes to match "ptr" and/or
666 "cnt". The application (or layer above) must ensure they are
667 consistent. (Checking is allowed by the paranoid.)
668 Utilities
670 To ask for the next layer down use PerlIONext(PerlIO *f).
671 To check that a PerlIO* is valid use PerlIOValid(PerlIO *f). (All this
673 does is really just to check that the pointer is non-NULL and that the
674 pointer behind that is non-NULL.)
675 PerlIOBase(PerlIO *f) returns the "Base" pointer, or in other words,
677 the "PerlIOl*" pointer.
678 PerlIOSelf(PerlIO* f, type) return the PerlIOBase cast to a type.
680 Perl_PerlIO_or_Base(PerlIO* f, callback, base, failure, args) either
682 calls the callback from the functions of the layer f (just by the name
683 of the IO function, like "Read") with the args, or if there is no such
684 callback, calls the base version of the callback with the same args, or
685 if the f is invalid, set errno to EBADF and return failure.
686 Perl_PerlIO_or_fail(PerlIO* f, callback, failure, args) either calls
688 the callback of the functions of the layer f with the args, or if there
689 is no such callback, set errno to EINVAL. Or if the f is invalid, set
690 errno to EBADF and return failure.
691 Perl_PerlIO_or_Base_void(PerlIO* f, callback, base, args) either calls
693 the callback of the functions of the layer f with the args, or if there
694 is no such callback, calls the base version of the callback with the
695 same args, or if the f is invalid, set errno to EBADF.
696 Perl_PerlIO_or_fail_void(PerlIO* f, callback, args) either calls the
698 callback of the functions of the layer f with the args, or if there is
699 no such callback, set errno to EINVAL. Or if the f is invalid, set
700 errno to EBADF.
701 Implementing PerlIO Layers
703 If you find the implementation document unclear or not sufficient, look
704 at the existing PerlIO layer implementations, which include:
705 • C implementations
707 The perlio.c and perliol.h in the Perl core implement the "unix",
709 "perlio", "stdio", "crlf", "utf8", "byte", "raw", "pending" layers,
710 and also the "mmap" and "win32" layers if applicable. (The "win32"
711 is currently unfinished and unused, to see what is used instead in
712 Win32, see "Querying the layers of filehandles" in PerlIO .)
713 PerlIO::encoding, PerlIO::scalar, PerlIO::via in the Perl core.
715 PerlIO::gzip and APR::PerlIO (mod_perl 2.0) on CPAN.
717 • Perl implementations
719 PerlIO::via::QuotedPrint in the Perl core and PerlIO::via::* on
721 CPAN.
722 If you are creating a PerlIO layer, you may want to be lazy, in other
724 words, implement only the methods that interest you. The other methods
725 you can either replace with the "blank" methods
726 PerlIOBase_noop_ok
728 PerlIOBase_noop_fail
729 (which do nothing, and return zero and -1, respectively) or for certain
731 methods you may assume a default behaviour by using a NULL method. The
732 Open method looks for help in the 'parent' layer. The following table
733 summarizes the behaviour:
734 method behaviour with NULL
736 Clearerr PerlIOBase_clearerr
738 Close PerlIOBase_close
739 Dup PerlIOBase_dup
740 Eof PerlIOBase_eof
741 Error PerlIOBase_error
742 Fileno PerlIOBase_fileno
743 Fill FAILURE
744 Flush SUCCESS
745 Getarg SUCCESS
746 Get_base FAILURE
747 Get_bufsiz FAILURE
748 Get_cnt FAILURE
749 Get_ptr FAILURE
750 Open INHERITED
751 Popped SUCCESS
752 Pushed SUCCESS
753 Read PerlIOBase_read
754 Seek FAILURE
755 Set_cnt FAILURE
756 Set_ptrcnt FAILURE
757 Setlinebuf PerlIOBase_setlinebuf
758 Tell FAILURE
759 Unread PerlIOBase_unread
760 Write FAILURE
761 FAILURE Set errno (to EINVAL in Unixish, to LIB$_INVARG in VMS)
763 and return -1 (for numeric return values) or NULL (for
764 pointers)
765 INHERITED Inherited from the layer below
766 SUCCESS Return 0 (for numeric return values) or a pointer
767 Core Layers
769 The file "perlio.c" provides the following layers:
770 "unix"
772 A basic non-buffered layer which calls Unix/POSIX "read()",
773 "write()", "lseek()", "close()". No buffering. Even on platforms
774 that distinguish between O_TEXT and O_BINARY this layer is always
775 O_BINARY.
776 "perlio"
778 A very complete generic buffering layer which provides the whole of
779 PerlIO API. It is also intended to be used as a "base class" for
780 other layers. (For example its "Read()" method is implemented in
781 terms of the "Get_cnt()"/"Get_ptr()"/"Set_ptrcnt()" methods).
782 "perlio" over "unix" provides a complete replacement for stdio as
784 seen via PerlIO API. This is the default for USE_PERLIO when
785 system's stdio does not permit perl's "fast gets" access, and which
786 do not distinguish between "O_TEXT" and "O_BINARY".
787 "stdio"
789 A layer which provides the PerlIO API via the layer scheme, but
790 implements it by calling system's stdio. This is (currently) the
791 default if system's stdio provides sufficient access to allow
792 perl's "fast gets" access and which do not distinguish between
793 "O_TEXT" and "O_BINARY".
794 "crlf"
796 A layer derived using "perlio" as a base class. It provides
797 Win32-like "\n" to CR,LF translation. Can either be applied above
798 "perlio" or serve as the buffer layer itself. "crlf" over "unix" is
799 the default if system distinguishes between "O_TEXT" and "O_BINARY"
800 opens. (At some point "unix" will be replaced by a "native" Win32
801 IO layer on that platform, as Win32's read/write layer has various
802 drawbacks.) The "crlf" layer is a reasonable model for a layer
803 which transforms data in some way.
804 "mmap"
806 If Configure detects "mmap()" functions this layer is provided
807 (with "perlio" as a "base") which does "read" operations by
808 mmap()ing the file. Performance improvement is marginal on modern
809 systems, so it is mainly there as a proof of concept. It is likely
810 to be unbundled from the core at some point. The "mmap" layer is a
811 reasonable model for a minimalist "derived" layer.
812 "pending"
814 An "internal" derivative of "perlio" which can be used to provide
815 Unread() function for layers which have no buffer or cannot be
816 bothered. (Basically this layer's "Fill()" pops itself off the
817 stack and so resumes reading from layer below.)
818 "raw"
820 A dummy layer which never exists on the layer stack. Instead when
821 "pushed" it actually pops the stack removing itself, it then calls
822 Binmode function table entry on all the layers in the stack -
823 normally this (via PerlIOBase_binmode) removes any layers which do
824 not have "PERLIO_K_RAW" bit set. Layers can modify that behaviour
825 by defining their own Binmode entry.
826 "utf8"
828 Another dummy layer. When pushed it pops itself and sets the
829 "PERLIO_F_UTF8" flag on the layer which was (and now is once more)
830 the top of the stack.
831 In addition perlio.c also provides a number of "PerlIOBase_xxxx()"
833 functions which are intended to be used in the table slots of classes
834 which do not need to do anything special for a particular method.
835 Extension Layers
837 Layers can be made available by extension modules. When an unknown
838 layer is encountered the PerlIO code will perform the equivalent of :
839 use PerlIO 'layer';
841 Where layer is the unknown layer. PerlIO.pm will then attempt to:
843 require PerlIO::layer;
845 If after that process the layer is still not defined then the "open"
847 will fail.
848 The following extension layers are bundled with perl:
850 ":encoding"
852 use Encoding;
853 makes this layer available, although PerlIO.pm "knows" where to
855 find it. It is an example of a layer which takes an argument as it
856 is called thus:
857 open( $fh, "<:encoding(iso-8859-7)", $pathname );
859 ":scalar"
861 Provides support for reading data from and writing data to a
862 scalar.
863 open( $fh, "+<:scalar", \$scalar );
865 When a handle is so opened, then reads get bytes from the string
867 value of $scalar, and writes change the value. In both cases the
868 position in $scalar starts as zero but can be altered via "seek",
869 and determined via "tell".
870 Please note that this layer is implied when calling open() thus:
872 open( $fh, "+<", \$scalar );
874 ":via"
876 Provided to allow layers to be implemented as Perl code. For
877 instance:
878 use PerlIO::via::StripHTML;
880 open( my $fh, "<:via(StripHTML)", "index.html" );
881 See PerlIO::via for details.
883
885 Things that need to be done to improve this document.
886 • Explain how to make a valid fh without going through open()(i.e.
888 apply a layer). For example if the file is not opened through perl,
889 but we want to get back a fh, like it was opened by Perl.
890 How PerlIO_apply_layera fits in, where its docs, was it made
892 public?
893 Currently the example could be something like this:
895 PerlIO *foo_to_PerlIO(pTHX_ char *mode, ...)
897 {
898 char *mode; /* "w", "r", etc */
899 const char *layers = ":APR"; /* the layer name */
900 PerlIO *f = PerlIO_allocate(aTHX);
901 if (!f) {
902 return NULL;
903 }
904 PerlIO_apply_layers(aTHX_ f, mode, layers);
906 if (f) {
908 PerlIOAPR *st = PerlIOSelf(f, PerlIOAPR);
909 /* fill in the st struct, as in _open() */
910 st->file = file;
911 PerlIOBase(f)->flags |= PERLIO_F_OPEN;
912 return f;
914 }
915 return NULL;
916 }
917 • fix/add the documentation in places marked as XXX.
919 • The handling of errors by the layer is not specified. e.g. when $!
921 should be set explicitly, when the error handling should be just
922 delegated to the top layer.
923 Probably give some hints on using SETERRNO() or pointers to where
925 they can be found.
926 • I think it would help to give some concrete examples to make it
928 easier to understand the API. Of course I agree that the API has to
929 be concise, but since there is no second document that is more of a
930 guide, I think that it'd make it easier to start with the doc which
931 is an API, but has examples in it in places where things are
932 unclear, to a person who is not a PerlIO guru (yet).
933perl v5.32.1 2021-05-31 PERLIOL(1)