1PERLIOL(1) Perl Programmers Reference Guide PERLIOL(1)
2
3
4
6 perliol - C API for Perl's implementation of IO in Layers.
7
9 /* Defining a layer ... */
10 #include <perliol.h>
11
13 This document describes the behavior and implementation of the PerlIO
14 abstraction described in perlapio when "USE_PERLIO" is defined (and
15 "USE_SFIO" is not).
16
17 History and Background
18 The PerlIO abstraction was introduced in perl5.003_02 but languished as
19 just an abstraction until perl5.7.0. However during that time a number
20 of perl extensions switched to using it, so the API is mostly fixed to
21 maintain (source) compatibility.
22
23 The aim of the implementation is to provide the PerlIO API in a
24 flexible and platform neutral manner. It is also a trial of an "Object
25 Oriented C, with vtables" approach which may be applied to Perl 6.
26
27 Basic Structure
28 PerlIO is a stack of layers.
29
30 The low levels of the stack work with the low-level operating system
31 calls (file descriptors in C) getting bytes in and out, the higher
32 layers of the stack buffer, filter, and otherwise manipulate the I/O,
33 and return characters (or bytes) to Perl. Terms above and below are
34 used to refer to the relative positioning of the stack layers.
35
36 A layer contains a "vtable", the table of I/O operations (at C level a
37 table of function pointers), and status flags. The functions in the
38 vtable implement operations like "open", "read", and "write".
39
40 When I/O, for example "read", is requested, the request goes from Perl
41 first down the stack using "read" functions of each layer, then at the
42 bottom the input is requested from the operating system services, then
43 the result is returned up the stack, finally being interpreted as Perl
44 data.
45
46 The requests do not necessarily go always all the way down to the
47 operating system: that's where PerlIO buffering comes into play.
48
49 When you do an open() and specify extra PerlIO layers to be deployed,
50 the layers you specify are "pushed" on top of the already existing
51 default stack. One way to see it is that "operating system is on the
52 left" and "Perl is on the right".
53
54 What exact layers are in this default stack depends on a lot of things:
55 your operating system, Perl version, Perl compile time configuration,
56 and Perl runtime configuration. See PerlIO, "PERLIO" in perlrun, and
57 open for more information.
58
59 binmode() operates similarly to open(): by default the specified layers
60 are pushed on top of the existing stack.
61
62 However, note that even as the specified layers are "pushed on top" for
63 open() and binmode(), this doesn't mean that the effects are limited to
64 the "top": PerlIO layers can be very 'active' and inspect and affect
65 layers also deeper in the stack. As an example there is a layer called
66 "raw" which repeatedly "pops" layers until it reaches the first layer
67 that has declared itself capable of handling binary data. The "pushed"
68 layers are processed in left-to-right order.
69
70 sysopen() operates (unsurprisingly) at a lower level in the stack than
71 open(). For example in Unix or Unix-like systems sysopen() operates
72 directly at the level of file descriptors: in the terms of PerlIO
73 layers, it uses only the "unix" layer, which is a rather thin wrapper
74 on top of the Unix file descriptors.
75
76 Layers vs Disciplines
77 Initial discussion of the ability to modify IO streams behaviour used
78 the term "discipline" for the entities which were added. This came (I
79 believe) from the use of the term in "sfio", which in turn borrowed it
80 from "line disciplines" on Unix terminals. However, this document (and
81 the C code) uses the term "layer".
82
83 This is, I hope, a natural term given the implementation, and should
84 avoid connotations that are inherent in earlier uses of "discipline"
85 for things which are rather different.
86
87 Data Structures
88 The basic data structure is a PerlIOl:
89
90 typedef struct _PerlIO PerlIOl;
91 typedef struct _PerlIO_funcs PerlIO_funcs;
92 typedef PerlIOl *PerlIO;
93
94 struct _PerlIO
95 {
96 PerlIOl * next; /* Lower layer */
97 PerlIO_funcs * tab; /* Functions for this layer */
98 IV flags; /* Various flags for state */
99 };
100
101 A "PerlIOl *" is a pointer to the struct, and the application level
102 "PerlIO *" is a pointer to a "PerlIOl *" - i.e. a pointer to a pointer
103 to the struct. This allows the application level "PerlIO *" to remain
104 constant while the actual "PerlIOl *" underneath changes. (Compare
105 perl's "SV *" which remains constant while its "sv_any" field changes
106 as the scalar's type changes.) An IO stream is then in general
107 represented as a pointer to this linked-list of "layers".
108
109 It should be noted that because of the double indirection in a "PerlIO
110 *", a "&(perlio->next)" "is" a "PerlIO *", and so to some degree at
111 least one layer can use the "standard" API on the next layer down.
112
113 A "layer" is composed of two parts:
114
115 1. The functions and attributes of the "layer class".
116
117 2. The per-instance data for a particular handle.
118
119 Functions and Attributes
120 The functions and attributes are accessed via the "tab" (for table)
121 member of "PerlIOl". The functions (methods of the layer "class") are
122 fixed, and are defined by the "PerlIO_funcs" type. They are broadly the
123 same as the public "PerlIO_xxxxx" functions:
124
125 struct _PerlIO_funcs
126 {
127 Size_t fsize;
128 char * name;
129 Size_t size;
130 IV kind;
131 IV (*Pushed)(pTHX_ PerlIO *f,const char *mode,SV *arg, PerlIO_funcs *tab);
132 IV (*Popped)(pTHX_ PerlIO *f);
133 PerlIO * (*Open)(pTHX_ PerlIO_funcs *tab,
134 PerlIO_list_t *layers, IV n,
135 const char *mode,
136 int fd, int imode, int perm,
137 PerlIO *old,
138 int narg, SV **args);
139 IV (*Binmode)(pTHX_ PerlIO *f);
140 SV * (*Getarg)(pTHX_ PerlIO *f, CLONE_PARAMS *param, int flags)
141 IV (*Fileno)(pTHX_ PerlIO *f);
142 PerlIO * (*Dup)(pTHX_ PerlIO *f, PerlIO *o, CLONE_PARAMS *param, int flags)
143 /* Unix-like functions - cf sfio line disciplines */
144 SSize_t (*Read)(pTHX_ PerlIO *f, void *vbuf, Size_t count);
145 SSize_t (*Unread)(pTHX_ PerlIO *f, const void *vbuf, Size_t count);
146 SSize_t (*Write)(pTHX_ PerlIO *f, const void *vbuf, Size_t count);
147 IV (*Seek)(pTHX_ PerlIO *f, Off_t offset, int whence);
148 Off_t (*Tell)(pTHX_ PerlIO *f);
149 IV (*Close)(pTHX_ PerlIO *f);
150 /* Stdio-like buffered IO functions */
151 IV (*Flush)(pTHX_ PerlIO *f);
152 IV (*Fill)(pTHX_ PerlIO *f);
153 IV (*Eof)(pTHX_ PerlIO *f);
154 IV (*Error)(pTHX_ PerlIO *f);
155 void (*Clearerr)(pTHX_ PerlIO *f);
156 void (*Setlinebuf)(pTHX_ PerlIO *f);
157 /* Perl's snooping functions */
158 STDCHAR * (*Get_base)(pTHX_ PerlIO *f);
159 Size_t (*Get_bufsiz)(pTHX_ PerlIO *f);
160 STDCHAR * (*Get_ptr)(pTHX_ PerlIO *f);
161 SSize_t (*Get_cnt)(pTHX_ PerlIO *f);
162 void (*Set_ptrcnt)(pTHX_ PerlIO *f,STDCHAR *ptr,SSize_t cnt);
163 };
164
165 The first few members of the struct give a function table size for
166 compatibility check "name" for the layer, the size to "malloc" for the
167 per-instance data, and some flags which are attributes of the class as
168 whole (such as whether it is a buffering layer), then follow the
169 functions which fall into four basic groups:
170
171 1. Opening and setup functions
172
173 2. Basic IO operations
174
175 3. Stdio class buffering options.
176
177 4. Functions to support Perl's traditional "fast" access to the
178 buffer.
179
180 A layer does not have to implement all the functions, but the whole
181 table has to be present. Unimplemented slots can be NULL (which will
182 result in an error when called) or can be filled in with stubs to
183 "inherit" behaviour from a "base class". This "inheritance" is fixed
184 for all instances of the layer, but as the layer chooses which stubs to
185 populate the table, limited "multiple inheritance" is possible.
186
187 Per-instance Data
188 The per-instance data are held in memory beyond the basic PerlIOl
189 struct, by making a PerlIOl the first member of the layer's struct
190 thus:
191
192 typedef struct
193 {
194 struct _PerlIO base; /* Base "class" info */
195 STDCHAR * buf; /* Start of buffer */
196 STDCHAR * end; /* End of valid part of buffer */
197 STDCHAR * ptr; /* Current position in buffer */
198 Off_t posn; /* Offset of buf into the file */
199 Size_t bufsiz; /* Real size of buffer */
200 IV oneword; /* Emergency buffer */
201 } PerlIOBuf;
202
203 In this way (as for perl's scalars) a pointer to a PerlIOBuf can be
204 treated as a pointer to a PerlIOl.
205
206 Layers in action.
207 table perlio unix
208 | |
209 +-----------+ +----------+ +--------+
210 PerlIO ->| |--->| next |--->| NULL |
211 +-----------+ +----------+ +--------+
212 | | | buffer | | fd |
213 +-----------+ | | +--------+
214 | | +----------+
215
216 The above attempts to show how the layer scheme works in a simple case.
217 The application's "PerlIO *" points to an entry in the table(s)
218 representing open (allocated) handles. For example the first three
219 slots in the table correspond to "stdin","stdout" and "stderr". The
220 table in turn points to the current "top" layer for the handle - in
221 this case an instance of the generic buffering layer "perlio". That
222 layer in turn points to the next layer down - in this case the low-
223 level "unix" layer.
224
225 The above is roughly equivalent to a "stdio" buffered stream, but with
226 much more flexibility:
227
228 · If Unix level "read"/"write"/"lseek" is not appropriate for (say)
229 sockets then the "unix" layer can be replaced (at open time or even
230 dynamically) with a "socket" layer.
231
232 · Different handles can have different buffering schemes. The "top"
233 layer could be the "mmap" layer if reading disk files was quicker
234 using "mmap" than "read". An "unbuffered" stream can be implemented
235 simply by not having a buffer layer.
236
237 · Extra layers can be inserted to process the data as it flows
238 through. This was the driving need for including the scheme in
239 perl 5.7.0+ - we needed a mechanism to allow data to be translated
240 between perl's internal encoding (conceptually at least Unicode as
241 UTF-8), and the "native" format used by the system. This is
242 provided by the ":encoding(xxxx)" layer which typically sits above
243 the buffering layer.
244
245 · A layer can be added that does "\n" to CRLF translation. This layer
246 can be used on any platform, not just those that normally do such
247 things.
248
249 Per-instance flag bits
250 The generic flag bits are a hybrid of "O_XXXXX" style flags deduced
251 from the mode string passed to "PerlIO_open()", and state bits for
252 typical buffer layers.
253
254 PERLIO_F_EOF
255 End of file.
256
257 PERLIO_F_CANWRITE
258 Writes are permitted, i.e. opened as "w" or "r+" or "a", etc.
259
260 PERLIO_F_CANREAD
261 Reads are permitted i.e. opened "r" or "w+" (or even "a+" - ick).
262
263 PERLIO_F_ERROR
264 An error has occurred (for "PerlIO_error()").
265
266 PERLIO_F_TRUNCATE
267 Truncate file suggested by open mode.
268
269 PERLIO_F_APPEND
270 All writes should be appends.
271
272 PERLIO_F_CRLF
273 Layer is performing Win32-like "\n" mapped to CR,LF for output and
274 CR,LF mapped to "\n" for input. Normally the provided "crlf" layer
275 is the only layer that need bother about this. "PerlIO_binmode()"
276 will mess with this flag rather than add/remove layers if the
277 "PERLIO_K_CANCRLF" bit is set for the layers class.
278
279 PERLIO_F_UTF8
280 Data written to this layer should be UTF-8 encoded; data provided
281 by this layer should be considered UTF-8 encoded. Can be set on any
282 layer by ":utf8" dummy layer. Also set on ":encoding" layer.
283
284 PERLIO_F_UNBUF
285 Layer is unbuffered - i.e. write to next layer down should occur
286 for each write to this layer.
287
288 PERLIO_F_WRBUF
289 The buffer for this layer currently holds data written to it but
290 not sent to next layer.
291
292 PERLIO_F_RDBUF
293 The buffer for this layer currently holds unconsumed data read from
294 layer below.
295
296 PERLIO_F_LINEBUF
297 Layer is line buffered. Write data should be passed to next layer
298 down whenever a "\n" is seen. Any data beyond the "\n" should then
299 be processed.
300
301 PERLIO_F_TEMP
302 File has been "unlink()"ed, or should be deleted on "close()".
303
304 PERLIO_F_OPEN
305 Handle is open.
306
307 PERLIO_F_FASTGETS
308 This instance of this layer supports the "fast "gets"" interface.
309 Normally set based on "PERLIO_K_FASTGETS" for the class and by the
310 existence of the function(s) in the table. However a class that
311 normally provides that interface may need to avoid it on a
312 particular instance. The "pending" layer needs to do this when it
313 is pushed above a layer which does not support the interface.
314 (Perl's "sv_gets()" does not expect the streams fast "gets"
315 behaviour to change during one "get".)
316
317 Methods in Detail
318 fsize
319 Size_t fsize;
320
321 Size of the function table. This is compared against the value
322 PerlIO code "knows" as a compatibility check. Future versions may
323 be able to tolerate layers compiled against an old version of the
324 headers.
325
326 name
327 char * name;
328
329 The name of the layer whose open() method Perl should invoke on
330 open(). For example if the layer is called APR, you will call:
331
332 open $fh, ">:APR", ...
333
334 and Perl knows that it has to invoke the PerlIOAPR_open() method
335 implemented by the APR layer.
336
337 size
338 Size_t size;
339
340 The size of the per-instance data structure, e.g.:
341
342 sizeof(PerlIOAPR)
343
344 If this field is zero then "PerlIO_pushed" does not malloc anything
345 and assumes layer's Pushed function will do any required layer
346 stack manipulation - used to avoid malloc/free overhead for dummy
347 layers. If the field is non-zero it must be at least the size of
348 "PerlIOl", "PerlIO_pushed" will allocate memory for the layer's
349 data structures and link new layer onto the stream's stack. (If the
350 layer's Pushed method returns an error indication the layer is
351 popped again.)
352
353 kind
354 IV kind;
355
356 · PERLIO_K_BUFFERED
357
358 The layer is buffered.
359
360 · PERLIO_K_RAW
361
362 The layer is acceptable to have in a binmode(FH) stack - i.e.
363 it does not (or will configure itself not to) transform bytes
364 passing through it.
365
366 · PERLIO_K_CANCRLF
367
368 Layer can translate between "\n" and CRLF line ends.
369
370 · PERLIO_K_FASTGETS
371
372 Layer allows buffer snooping.
373
374 · PERLIO_K_MULTIARG
375
376 Used when the layer's open() accepts more arguments than usual.
377 The extra arguments should come not before the "MODE" argument.
378 When this flag is used it's up to the layer to validate the
379 args.
380
381 Pushed
382 IV (*Pushed)(pTHX_ PerlIO *f,const char *mode, SV *arg);
383
384 The only absolutely mandatory method. Called when the layer is
385 pushed onto the stack. The "mode" argument may be NULL if this
386 occurs post-open. The "arg" will be non-"NULL" if an argument
387 string was passed. In most cases this should call
388 "PerlIOBase_pushed()" to convert "mode" into the appropriate
389 "PERLIO_F_XXXXX" flags in addition to any actions the layer itself
390 takes. If a layer is not expecting an argument it need neither
391 save the one passed to it, nor provide "Getarg()" (it could perhaps
392 "Perl_warn" that the argument was un-expected).
393
394 Returns 0 on success. On failure returns -1 and should set errno.
395
396 Popped
397 IV (*Popped)(pTHX_ PerlIO *f);
398
399 Called when the layer is popped from the stack. A layer will
400 normally be popped after "Close()" is called. But a layer can be
401 popped without being closed if the program is dynamically managing
402 layers on the stream. In such cases "Popped()" should free any
403 resources (buffers, translation tables, ...) not held directly in
404 the layer's struct. It should also "Unread()" any unconsumed data
405 that has been read and buffered from the layer below back to that
406 layer, so that it can be re-provided to what ever is now above.
407
408 Returns 0 on success and failure. If "Popped()" returns true then
409 perlio.c assumes that either the layer has popped itself, or the
410 layer is super special and needs to be retained for other reasons.
411 In most cases it should return false.
412
413 Open
414 PerlIO * (*Open)(...);
415
416 The "Open()" method has lots of arguments because it combines the
417 functions of perl's "open", "PerlIO_open", perl's "sysopen",
418 "PerlIO_fdopen" and "PerlIO_reopen". The full prototype is as
419 follows:
420
421 PerlIO * (*Open)(pTHX_ PerlIO_funcs *tab,
422 PerlIO_list_t *layers, IV n,
423 const char *mode,
424 int fd, int imode, int perm,
425 PerlIO *old,
426 int narg, SV **args);
427
428 Open should (perhaps indirectly) call "PerlIO_allocate()" to
429 allocate a slot in the table and associate it with the layers
430 information for the opened file, by calling "PerlIO_push". The
431 layers is an array of all the layers destined for the "PerlIO *",
432 and any arguments passed to them, n is the index into that array of
433 the layer being called. The macro "PerlIOArg" will return a
434 (possibly "NULL") SV * for the argument passed to the layer.
435
436 The mode string is an ""fopen()"-like" string which would match the
437 regular expression "/^[I#]?[rwa]\+?[bt]?$/".
438
439 The 'I' prefix is used during creation of "stdin".."stderr" via
440 special "PerlIO_fdopen" calls; the '#' prefix means that this is
441 "sysopen" and that imode and perm should be passed to
442 "PerlLIO_open3"; 'r' means read, 'w' means write and 'a' means
443 append. The '+' suffix means that both reading and
444 writing/appending are permitted. The 'b' suffix means file should
445 be binary, and 't' means it is text. (Almost all layers should do
446 the IO in binary mode, and ignore the b/t bits. The ":crlf" layer
447 should be pushed to handle the distinction.)
448
449 If old is not "NULL" then this is a "PerlIO_reopen". Perl itself
450 does not use this (yet?) and semantics are a little vague.
451
452 If fd not negative then it is the numeric file descriptor fd, which
453 will be open in a manner compatible with the supplied mode string,
454 the call is thus equivalent to "PerlIO_fdopen". In this case nargs
455 will be zero.
456
457 If nargs is greater than zero then it gives the number of arguments
458 passed to "open", otherwise it will be 1 if for example
459 "PerlIO_open" was called. In simple cases SvPV_nolen(*args) is the
460 pathname to open.
461
462 If a layer provides "Open()" it should normally call the "Open()"
463 method of next layer down (if any) and then push itself on top if
464 that succeeds. "PerlIOBase_open" is provided to do exactly that,
465 so in most cases you don't have to write your own "Open()" method.
466 If this method is not defined, other layers may have difficulty
467 pushing themselves on top of it during open.
468
469 If "PerlIO_push" was performed and open has failed, it must
470 "PerlIO_pop" itself, since if it's not, the layer won't be removed
471 and may cause bad problems.
472
473 Returns "NULL" on failure.
474
475 Binmode
476 IV (*Binmode)(pTHX_ PerlIO *f);
477
478 Optional. Used when ":raw" layer is pushed (explicitly or as a
479 result of binmode(FH)). If not present layer will be popped. If
480 present should configure layer as binary (or pop itself) and return
481 0. If it returns -1 for error "binmode" will fail with layer still
482 on the stack.
483
484 Getarg
485 SV * (*Getarg)(pTHX_ PerlIO *f,
486 CLONE_PARAMS *param, int flags);
487
488 Optional. If present should return an SV * representing the string
489 argument passed to the layer when it was pushed. e.g.
490 ":encoding(ascii)" would return an SvPV with value "ascii". (param
491 and flags arguments can be ignored in most cases)
492
493 "Dup" uses "Getarg" to retrieve the argument originally passed to
494 "Pushed", so you must implement this function if your layer has an
495 extra argument to "Pushed" and will ever be "Dup"ed.
496
497 Fileno
498 IV (*Fileno)(pTHX_ PerlIO *f);
499
500 Returns the Unix/Posix numeric file descriptor for the handle.
501 Normally "PerlIOBase_fileno()" (which just asks next layer down)
502 will suffice for this.
503
504 Returns -1 on error, which is considered to include the case where
505 the layer cannot provide such a file descriptor.
506
507 Dup
508 PerlIO * (*Dup)(pTHX_ PerlIO *f, PerlIO *o,
509 CLONE_PARAMS *param, int flags);
510
511 XXX: Needs more docs.
512
513 Used as part of the "clone" process when a thread is spawned (in
514 which case param will be non-NULL) and when a stream is being
515 duplicated via '&' in the "open".
516
517 Similar to "Open", returns PerlIO* on success, "NULL" on failure.
518
519 Read
520 SSize_t (*Read)(pTHX_ PerlIO *f, void *vbuf, Size_t count);
521
522 Basic read operation.
523
524 Typically will call "Fill" and manipulate pointers (possibly via
525 the API). "PerlIOBuf_read()" may be suitable for derived classes
526 which provide "fast gets" methods.
527
528 Returns actual bytes read, or -1 on an error.
529
530 Unread
531 SSize_t (*Unread)(pTHX_ PerlIO *f,
532 const void *vbuf, Size_t count);
533
534 A superset of stdio's "ungetc()". Should arrange for future reads
535 to see the bytes in "vbuf". If there is no obviously better
536 implementation then "PerlIOBase_unread()" provides the function by
537 pushing a "fake" "pending" layer above the calling layer.
538
539 Returns the number of unread chars.
540
541 Write
542 SSize_t (*Write)(PerlIO *f, const void *vbuf, Size_t count);
543
544 Basic write operation.
545
546 Returns bytes written or -1 on an error.
547
548 Seek
549 IV (*Seek)(pTHX_ PerlIO *f, Off_t offset, int whence);
550
551 Position the file pointer. Should normally call its own "Flush"
552 method and then the "Seek" method of next layer down.
553
554 Returns 0 on success, -1 on failure.
555
556 Tell
557 Off_t (*Tell)(pTHX_ PerlIO *f);
558
559 Return the file pointer. May be based on layers cached concept of
560 position to avoid overhead.
561
562 Returns -1 on failure to get the file pointer.
563
564 Close
565 IV (*Close)(pTHX_ PerlIO *f);
566
567 Close the stream. Should normally call "PerlIOBase_close()" to
568 flush itself and close layers below, and then deallocate any data
569 structures (buffers, translation tables, ...) not held directly in
570 the data structure.
571
572 Returns 0 on success, -1 on failure.
573
574 Flush
575 IV (*Flush)(pTHX_ PerlIO *f);
576
577 Should make stream's state consistent with layers below. That is,
578 any buffered write data should be written, and file position of
579 lower layers adjusted for data read from below but not actually
580 consumed. (Should perhaps "Unread()" such data to the lower
581 layer.)
582
583 Returns 0 on success, -1 on failure.
584
585 Fill
586 IV (*Fill)(pTHX_ PerlIO *f);
587
588 The buffer for this layer should be filled (for read) from layer
589 below. When you "subclass" PerlIOBuf layer, you want to use its
590 _read method and to supply your own fill method, which fills the
591 PerlIOBuf's buffer.
592
593 Returns 0 on success, -1 on failure.
594
595 Eof
596 IV (*Eof)(pTHX_ PerlIO *f);
597
598 Return end-of-file indicator. "PerlIOBase_eof()" is normally
599 sufficient.
600
601 Returns 0 on end-of-file, 1 if not end-of-file, -1 on error.
602
603 Error
604 IV (*Error)(pTHX_ PerlIO *f);
605
606 Return error indicator. "PerlIOBase_error()" is normally
607 sufficient.
608
609 Returns 1 if there is an error (usually when "PERLIO_F_ERROR" is
610 set, 0 otherwise.
611
612 Clearerr
613 void (*Clearerr)(pTHX_ PerlIO *f);
614
615 Clear end-of-file and error indicators. Should call
616 "PerlIOBase_clearerr()" to set the "PERLIO_F_XXXXX" flags, which
617 may suffice.
618
619 Setlinebuf
620 void (*Setlinebuf)(pTHX_ PerlIO *f);
621
622 Mark the stream as line buffered. "PerlIOBase_setlinebuf()" sets
623 the PERLIO_F_LINEBUF flag and is normally sufficient.
624
625 Get_base
626 STDCHAR * (*Get_base)(pTHX_ PerlIO *f);
627
628 Allocate (if not already done so) the read buffer for this layer
629 and return pointer to it. Return NULL on failure.
630
631 Get_bufsiz
632 Size_t (*Get_bufsiz)(pTHX_ PerlIO *f);
633
634 Return the number of bytes that last "Fill()" put in the buffer.
635
636 Get_ptr
637 STDCHAR * (*Get_ptr)(pTHX_ PerlIO *f);
638
639 Return the current read pointer relative to this layer's buffer.
640
641 Get_cnt
642 SSize_t (*Get_cnt)(pTHX_ PerlIO *f);
643
644 Return the number of bytes left to be read in the current buffer.
645
646 Set_ptrcnt
647 void (*Set_ptrcnt)(pTHX_ PerlIO *f,
648 STDCHAR *ptr, SSize_t cnt);
649
650 Adjust the read pointer and count of bytes to match "ptr" and/or
651 "cnt". The application (or layer above) must ensure they are
652 consistent. (Checking is allowed by the paranoid.)
653
654 Utilities
655 To ask for the next layer down use PerlIONext(PerlIO *f).
656
657 To check that a PerlIO* is valid use PerlIOValid(PerlIO *f). (All this
658 does is really just to check that the pointer is non-NULL and that the
659 pointer behind that is non-NULL.)
660
661 PerlIOBase(PerlIO *f) returns the "Base" pointer, or in other words,
662 the "PerlIOl*" pointer.
663
664 PerlIOSelf(PerlIO* f, type) return the PerlIOBase cast to a type.
665
666 Perl_PerlIO_or_Base(PerlIO* f, callback, base, failure, args) either
667 calls the callback from the functions of the layer f (just by the name
668 of the IO function, like "Read") with the args, or if there is no such
669 callback, calls the base version of the callback with the same args, or
670 if the f is invalid, set errno to EBADF and return failure.
671
672 Perl_PerlIO_or_fail(PerlIO* f, callback, failure, args) either calls
673 the callback of the functions of the layer f with the args, or if there
674 is no such callback, set errno to EINVAL. Or if the f is invalid, set
675 errno to EBADF and return failure.
676
677 Perl_PerlIO_or_Base_void(PerlIO* f, callback, base, args) either calls
678 the callback of the functions of the layer f with the args, or if there
679 is no such callback, calls the base version of the callback with the
680 same args, or if the f is invalid, set errno to EBADF.
681
682 Perl_PerlIO_or_fail_void(PerlIO* f, callback, args) either calls the
683 callback of the functions of the layer f with the args, or if there is
684 no such callback, set errno to EINVAL. Or if the f is invalid, set
685 errno to EBADF.
686
687 Implementing PerlIO Layers
688 If you find the implementation document unclear or not sufficient, look
689 at the existing PerlIO layer implementations, which include:
690
691 · C implementations
692
693 The perlio.c and perliol.h in the Perl core implement the "unix",
694 "perlio", "stdio", "crlf", "utf8", "byte", "raw", "pending" layers,
695 and also the "mmap" and "win32" layers if applicable. (The "win32"
696 is currently unfinished and unused, to see what is used instead in
697 Win32, see "Querying the layers of filehandles" in PerlIO .)
698
699 PerlIO::encoding, PerlIO::scalar, PerlIO::via in the Perl core.
700
701 PerlIO::gzip and APR::PerlIO (mod_perl 2.0) on CPAN.
702
703 · Perl implementations
704
705 PerlIO::via::QuotedPrint in the Perl core and PerlIO::via::* on
706 CPAN.
707
708 If you are creating a PerlIO layer, you may want to be lazy, in other
709 words, implement only the methods that interest you. The other methods
710 you can either replace with the "blank" methods
711
712 PerlIOBase_noop_ok
713 PerlIOBase_noop_fail
714
715 (which do nothing, and return zero and -1, respectively) or for certain
716 methods you may assume a default behaviour by using a NULL method. The
717 Open method looks for help in the 'parent' layer. The following table
718 summarizes the behaviour:
719
720 method behaviour with NULL
721
722 Clearerr PerlIOBase_clearerr
723 Close PerlIOBase_close
724 Dup PerlIOBase_dup
725 Eof PerlIOBase_eof
726 Error PerlIOBase_error
727 Fileno PerlIOBase_fileno
728 Fill FAILURE
729 Flush SUCCESS
730 Getarg SUCCESS
731 Get_base FAILURE
732 Get_bufsiz FAILURE
733 Get_cnt FAILURE
734 Get_ptr FAILURE
735 Open INHERITED
736 Popped SUCCESS
737 Pushed SUCCESS
738 Read PerlIOBase_read
739 Seek FAILURE
740 Set_cnt FAILURE
741 Set_ptrcnt FAILURE
742 Setlinebuf PerlIOBase_setlinebuf
743 Tell FAILURE
744 Unread PerlIOBase_unread
745 Write FAILURE
746
747 FAILURE Set errno (to EINVAL in Unixish, to LIB$_INVARG in VMS) and
748 return -1 (for numeric return values) or NULL (for pointers)
749 INHERITED Inherited from the layer below
750 SUCCESS Return 0 (for numeric return values) or a pointer
751
752 Core Layers
753 The file "perlio.c" provides the following layers:
754
755 "unix"
756 A basic non-buffered layer which calls Unix/POSIX "read()",
757 "write()", "lseek()", "close()". No buffering. Even on platforms
758 that distinguish between O_TEXT and O_BINARY this layer is always
759 O_BINARY.
760
761 "perlio"
762 A very complete generic buffering layer which provides the whole of
763 PerlIO API. It is also intended to be used as a "base class" for
764 other layers. (For example its "Read()" method is implemented in
765 terms of the "Get_cnt()"/"Get_ptr()"/"Set_ptrcnt()" methods).
766
767 "perlio" over "unix" provides a complete replacement for stdio as
768 seen via PerlIO API. This is the default for USE_PERLIO when
769 system's stdio does not permit perl's "fast gets" access, and which
770 do not distinguish between "O_TEXT" and "O_BINARY".
771
772 "stdio"
773 A layer which provides the PerlIO API via the layer scheme, but
774 implements it by calling system's stdio. This is (currently) the
775 default if system's stdio provides sufficient access to allow
776 perl's "fast gets" access and which do not distinguish between
777 "O_TEXT" and "O_BINARY".
778
779 "crlf"
780 A layer derived using "perlio" as a base class. It provides
781 Win32-like "\n" to CR,LF translation. Can either be applied above
782 "perlio" or serve as the buffer layer itself. "crlf" over "unix" is
783 the default if system distinguishes between "O_TEXT" and "O_BINARY"
784 opens. (At some point "unix" will be replaced by a "native" Win32
785 IO layer on that platform, as Win32's read/write layer has various
786 drawbacks.) The "crlf" layer is a reasonable model for a layer
787 which transforms data in some way.
788
789 "mmap"
790 If Configure detects "mmap()" functions this layer is provided
791 (with "perlio" as a "base") which does "read" operations by
792 mmap()ing the file. Performance improvement is marginal on modern
793 systems, so it is mainly there as a proof of concept. It is likely
794 to be unbundled from the core at some point. The "mmap" layer is a
795 reasonable model for a minimalist "derived" layer.
796
797 "pending"
798 An "internal" derivative of "perlio" which can be used to provide
799 Unread() function for layers which have no buffer or cannot be
800 bothered. (Basically this layer's "Fill()" pops itself off the
801 stack and so resumes reading from layer below.)
802
803 "raw"
804 A dummy layer which never exists on the layer stack. Instead when
805 "pushed" it actually pops the stack removing itself, it then calls
806 Binmode function table entry on all the layers in the stack -
807 normally this (via PerlIOBase_binmode) removes any layers which do
808 not have "PERLIO_K_RAW" bit set. Layers can modify that behaviour
809 by defining their own Binmode entry.
810
811 "utf8"
812 Another dummy layer. When pushed it pops itself and sets the
813 "PERLIO_F_UTF8" flag on the layer which was (and now is once more)
814 the top of the stack.
815
816 In addition perlio.c also provides a number of "PerlIOBase_xxxx()"
817 functions which are intended to be used in the table slots of classes
818 which do not need to do anything special for a particular method.
819
820 Extension Layers
821 Layers can be made available by extension modules. When an unknown
822 layer is encountered the PerlIO code will perform the equivalent of :
823
824 use PerlIO 'layer';
825
826 Where layer is the unknown layer. PerlIO.pm will then attempt to:
827
828 require PerlIO::layer;
829
830 If after that process the layer is still not defined then the "open"
831 will fail.
832
833 The following extension layers are bundled with perl:
834
835 ":encoding"
836 use Encoding;
837
838 makes this layer available, although PerlIO.pm "knows" where to
839 find it. It is an example of a layer which takes an argument as it
840 is called thus:
841
842 open( $fh, "<:encoding(iso-8859-7)", $pathname );
843
844 ":scalar"
845 Provides support for reading data from and writing data to a
846 scalar.
847
848 open( $fh, "+<:scalar", \$scalar );
849
850 When a handle is so opened, then reads get bytes from the string
851 value of $scalar, and writes change the value. In both cases the
852 position in $scalar starts as zero but can be altered via "seek",
853 and determined via "tell".
854
855 Please note that this layer is implied when calling open() thus:
856
857 open( $fh, "+<", \$scalar );
858
859 ":via"
860 Provided to allow layers to be implemented as Perl code. For
861 instance:
862
863 use PerlIO::via::StripHTML;
864 open( my $fh, "<:via(StripHTML)", "index.html" );
865
866 See PerlIO::via for details.
867
869 Things that need to be done to improve this document.
870
871 · Explain how to make a valid fh without going through open()(i.e.
872 apply a layer). For example if the file is not opened through perl,
873 but we want to get back a fh, like it was opened by Perl.
874
875 How PerlIO_apply_layera fits in, where its docs, was it made
876 public?
877
878 Currently the example could be something like this:
879
880 PerlIO *foo_to_PerlIO(pTHX_ char *mode, ...)
881 {
882 char *mode; /* "w", "r", etc */
883 const char *layers = ":APR"; /* the layer name */
884 PerlIO *f = PerlIO_allocate(aTHX);
885 if (!f) {
886 return NULL;
887 }
888
889 PerlIO_apply_layers(aTHX_ f, mode, layers);
890
891 if (f) {
892 PerlIOAPR *st = PerlIOSelf(f, PerlIOAPR);
893 /* fill in the st struct, as in _open() */
894 st->file = file;
895 PerlIOBase(f)->flags |= PERLIO_F_OPEN;
896
897 return f;
898 }
899 return NULL;
900 }
901
902 · fix/add the documentation in places marked as XXX.
903
904 · The handling of errors by the layer is not specified. e.g. when $!
905 should be set explicitly, when the error handling should be just
906 delegated to the top layer.
907
908 Probably give some hints on using SETERRNO() or pointers to where
909 they can be found.
910
911 · I think it would help to give some concrete examples to make it
912 easier to understand the API. Of course I agree that the API has to
913 be concise, but since there is no second document that is more of a
914 guide, I think that it'd make it easier to start with the doc which
915 is an API, but has examples in it in places where things are
916 unclear, to a person who is not a PerlIO guru (yet).
917
918
919
920perl v5.16.3 2013-03-04 PERLIOL(1)