1PERLIOL(1) Perl Programmers Reference Guide PERLIOL(1)
2
3
4
6 perliol - C API for Perl's implementation of IO in Layers.
7
9 /* Defining a layer ... */
10 #include <perliol.h>
11
13 This document describes the behavior and implementation of the PerlIO
14 abstraction described in perlapio when "USE_PERLIO" is defined (and
15 "USE_SFIO" is not).
16
17 History and Background
18
19 The PerlIO abstraction was introduced in perl5.003_02 but languished as
20 just an abstraction until perl5.7.0. However during that time a number
21 of perl extensions switched to using it, so the API is mostly fixed to
22 maintain (source) compatibility.
23
24 The aim of the implementation is to provide the PerlIO API in a flexi‐
25 ble and platform neutral manner. It is also a trial of an "Object Ori‐
26 ented C, with vtables" approach which may be applied to perl6.
27
28 Basic Structure
29
30 PerlIO is a stack of layers.
31
32 The low levels of the stack work with the low-level operating system
33 calls (file descriptors in C) getting bytes in and out, the higher lay‐
34 ers of the stack buffer, filter, and otherwise manipulate the I/O, and
35 return characters (or bytes) to Perl. Terms above and below are used
36 to refer to the relative positioning of the stack layers.
37
38 A layer contains a "vtable", the table of I/O operations (at C level a
39 table of function pointers), and status flags. The functions in the
40 vtable implement operations like "open", "read", and "write".
41
42 When I/O, for example "read", is requested, the request goes from Perl
43 first down the stack using "read" functions of each layer, then at the
44 bottom the input is requested from the operating system services, then
45 the result is returned up the stack, finally being interpreted as Perl
46 data.
47
48 The requests do not necessarily go always all the way down to the oper‐
49 ating system: that's where PerlIO buffering comes into play.
50
51 When you do an open() and specify extra PerlIO layers to be deployed,
52 the layers you specify are "pushed" on top of the already existing
53 default stack. One way to see it is that "operating system is on the
54 left" and "Perl is on the right".
55
56 What exact layers are in this default stack depends on a lot of things:
57 your operating system, Perl version, Perl compile time configuration,
58 and Perl runtime configuration. See PerlIO, "PERLIO" in perlrun, and
59 open for more information.
60
61 binmode() operates similarly to open(): by default the specified layers
62 are pushed on top of the existing stack.
63
64 However, note that even as the specified layers are "pushed on top" for
65 open() and binmode(), this doesn't mean that the effects are limited to
66 the "top": PerlIO layers can be very 'active' and inspect and affect
67 layers also deeper in the stack. As an example there is a layer called
68 "raw" which repeatedly "pops" layers until it reaches the first layer
69 that has declared itself capable of handling binary data. The "pushed"
70 layers are processed in left-to-right order.
71
72 sysopen() operates (unsurprisingly) at a lower level in the stack than
73 open(). For example in UNIX or UNIX-like systems sysopen() operates
74 directly at the level of file descriptors: in the terms of PerlIO lay‐
75 ers, it uses only the "unix" layer, which is a rather thin wrapper on
76 top of the UNIX file descriptors.
77
78 Layers vs Disciplines
79
80 Initial discussion of the ability to modify IO streams behaviour used
81 the term "discipline" for the entities which were added. This came (I
82 believe) from the use of the term in "sfio", which in turn borrowed it
83 from "line disciplines" on Unix terminals. However, this document (and
84 the C code) uses the term "layer".
85
86 This is, I hope, a natural term given the implementation, and should
87 avoid connotations that are inherent in earlier uses of "discipline"
88 for things which are rather different.
89
90 Data Structures
91
92 The basic data structure is a PerlIOl:
93
94 typedef struct _PerlIO PerlIOl;
95 typedef struct _PerlIO_funcs PerlIO_funcs;
96 typedef PerlIOl *PerlIO;
97
98 struct _PerlIO
99 {
100 PerlIOl * next; /* Lower layer */
101 PerlIO_funcs * tab; /* Functions for this layer */
102 IV flags; /* Various flags for state */
103 };
104
105 A "PerlIOl *" is a pointer to the struct, and the application level
106 "PerlIO *" is a pointer to a "PerlIOl *" - i.e. a pointer to a pointer
107 to the struct. This allows the application level "PerlIO *" to remain
108 constant while the actual "PerlIOl *" underneath changes. (Compare
109 perl's "SV *" which remains constant while its "sv_any" field changes
110 as the scalar's type changes.) An IO stream is then in general repre‐
111 sented as a pointer to this linked-list of "layers".
112
113 It should be noted that because of the double indirection in a "PerlIO
114 *", a "&(perlio->next)" "is" a "PerlIO *", and so to some degree at
115 least one layer can use the "standard" API on the next layer down.
116
117 A "layer" is composed of two parts:
118
119 1. The functions and attributes of the "layer class".
120
121 2. The per-instance data for a particular handle.
122
123 Functions and Attributes
124
125 The functions and attributes are accessed via the "tab" (for table)
126 member of "PerlIOl". The functions (methods of the layer "class") are
127 fixed, and are defined by the "PerlIO_funcs" type. They are broadly the
128 same as the public "PerlIO_xxxxx" functions:
129
130 struct _PerlIO_funcs
131 {
132 Size_t fsize;
133 char * name;
134 Size_t size;
135 IV kind;
136 IV (*Pushed)(pTHX_ PerlIO *f,const char *mode,SV *arg, PerlIO_funcs *tab);
137 IV (*Popped)(pTHX_ PerlIO *f);
138 PerlIO * (*Open)(pTHX_ PerlIO_funcs *tab,
139 AV *layers, IV n,
140 const char *mode,
141 int fd, int imode, int perm,
142 PerlIO *old,
143 int narg, SV **args);
144 IV (*Binmode)(pTHX_ PerlIO *f);
145 SV * (*Getarg)(pTHX_ PerlIO *f, CLONE_PARAMS *param, int flags)
146 IV (*Fileno)(pTHX_ PerlIO *f);
147 PerlIO * (*Dup)(pTHX_ PerlIO *f, PerlIO *o, CLONE_PARAMS *param, int flags)
148 /* Unix-like functions - cf sfio line disciplines */
149 SSize_t (*Read)(pTHX_ PerlIO *f, void *vbuf, Size_t count);
150 SSize_t (*Unread)(pTHX_ PerlIO *f, const void *vbuf, Size_t count);
151 SSize_t (*Write)(pTHX_ PerlIO *f, const void *vbuf, Size_t count);
152 IV (*Seek)(pTHX_ PerlIO *f, Off_t offset, int whence);
153 Off_t (*Tell)(pTHX_ PerlIO *f);
154 IV (*Close)(pTHX_ PerlIO *f);
155 /* Stdio-like buffered IO functions */
156 IV (*Flush)(pTHX_ PerlIO *f);
157 IV (*Fill)(pTHX_ PerlIO *f);
158 IV (*Eof)(pTHX_ PerlIO *f);
159 IV (*Error)(pTHX_ PerlIO *f);
160 void (*Clearerr)(pTHX_ PerlIO *f);
161 void (*Setlinebuf)(pTHX_ PerlIO *f);
162 /* Perl's snooping functions */
163 STDCHAR * (*Get_base)(pTHX_ PerlIO *f);
164 Size_t (*Get_bufsiz)(pTHX_ PerlIO *f);
165 STDCHAR * (*Get_ptr)(pTHX_ PerlIO *f);
166 SSize_t (*Get_cnt)(pTHX_ PerlIO *f);
167 void (*Set_ptrcnt)(pTHX_ PerlIO *f,STDCHAR *ptr,SSize_t cnt);
168 };
169
170 The first few members of the struct give a function table size for com‐
171 patibility check "name" for the layer, the size to "malloc" for the
172 per-instance data, and some flags which are attributes of the class as
173 whole (such as whether it is a buffering layer), then follow the func‐
174 tions which fall into four basic groups:
175
176 1. Opening and setup functions
177
178 2. Basic IO operations
179
180 3. Stdio class buffering options.
181
182 4. Functions to support Perl's traditional "fast" access to the buf‐
183 fer.
184
185 A layer does not have to implement all the functions, but the whole ta‐
186 ble has to be present. Unimplemented slots can be NULL (which will
187 result in an error when called) or can be filled in with stubs to
188 "inherit" behaviour from a "base class". This "inheritance" is fixed
189 for all instances of the layer, but as the layer chooses which stubs to
190 populate the table, limited "multiple inheritance" is possible.
191
192 Per-instance Data
193
194 The per-instance data are held in memory beyond the basic PerlIOl
195 struct, by making a PerlIOl the first member of the layer's struct
196 thus:
197
198 typedef struct
199 {
200 struct _PerlIO base; /* Base "class" info */
201 STDCHAR * buf; /* Start of buffer */
202 STDCHAR * end; /* End of valid part of buffer */
203 STDCHAR * ptr; /* Current position in buffer */
204 Off_t posn; /* Offset of buf into the file */
205 Size_t bufsiz; /* Real size of buffer */
206 IV oneword; /* Emergency buffer */
207 } PerlIOBuf;
208
209 In this way (as for perl's scalars) a pointer to a PerlIOBuf can be
210 treated as a pointer to a PerlIOl.
211
212 Layers in action.
213
214 table perlio unix
215 ⎪ ⎪
216 +-----------+ +----------+ +--------+
217 PerlIO ->⎪ ⎪--->⎪ next ⎪--->⎪ NULL ⎪
218 +-----------+ +----------+ +--------+
219 ⎪ ⎪ ⎪ buffer ⎪ ⎪ fd ⎪
220 +-----------+ ⎪ ⎪ +--------+
221 ⎪ ⎪ +----------+
222
223 The above attempts to show how the layer scheme works in a simple case.
224 The application's "PerlIO *" points to an entry in the table(s) repre‐
225 senting open (allocated) handles. For example the first three slots in
226 the table correspond to "stdin","stdout" and "stderr". The table in
227 turn points to the current "top" layer for the handle - in this case an
228 instance of the generic buffering layer "perlio". That layer in turn
229 points to the next layer down - in this case the lowlevel "unix" layer.
230
231 The above is roughly equivalent to a "stdio" buffered stream, but with
232 much more flexibility:
233
234 · If Unix level "read"/"write"/"lseek" is not appropriate for (say)
235 sockets then the "unix" layer can be replaced (at open time or even
236 dynamically) with a "socket" layer.
237
238 · Different handles can have different buffering schemes. The "top"
239 layer could be the "mmap" layer if reading disk files was quicker
240 using "mmap" than "read". An "unbuffered" stream can be implemented
241 simply by not having a buffer layer.
242
243 · Extra layers can be inserted to process the data as it flows
244 through. This was the driving need for including the scheme in
245 perl 5.7.0+ - we needed a mechanism to allow data to be translated
246 between perl's internal encoding (conceptually at least Unicode as
247 UTF-8), and the "native" format used by the system. This is pro‐
248 vided by the ":encoding(xxxx)" layer which typically sits above the
249 buffering layer.
250
251 · A layer can be added that does "\n" to CRLF translation. This layer
252 can be used on any platform, not just those that normally do such
253 things.
254
255 Per-instance flag bits
256
257 The generic flag bits are a hybrid of "O_XXXXX" style flags deduced
258 from the mode string passed to "PerlIO_open()", and state bits for typ‐
259 ical buffer layers.
260
261 PERLIO_F_EOF
262 End of file.
263
264 PERLIO_F_CANWRITE
265 Writes are permitted, i.e. opened as "w" or "r+" or "a", etc.
266
267 PERLIO_F_CANREAD
268 Reads are permitted i.e. opened "r" or "w+" (or even "a+" - ick).
269
270 PERLIO_F_ERROR
271 An error has occurred (for "PerlIO_error()").
272
273 PERLIO_F_TRUNCATE
274 Truncate file suggested by open mode.
275
276 PERLIO_F_APPEND
277 All writes should be appends.
278
279 PERLIO_F_CRLF
280 Layer is performing Win32-like "\n" mapped to CR,LF for output and
281 CR,LF mapped to "\n" for input. Normally the provided "crlf" layer
282 is the only layer that need bother about this. "PerlIO_binmode()"
283 will mess with this flag rather than add/remove layers if the "PER‐
284 LIO_K_CANCRLF" bit is set for the layers class.
285
286 PERLIO_F_UTF8
287 Data written to this layer should be UTF-8 encoded; data provided
288 by this layer should be considered UTF-8 encoded. Can be set on any
289 layer by ":utf8" dummy layer. Also set on ":encoding" layer.
290
291 PERLIO_F_UNBUF
292 Layer is unbuffered - i.e. write to next layer down should occur
293 for each write to this layer.
294
295 PERLIO_F_WRBUF
296 The buffer for this layer currently holds data written to it but
297 not sent to next layer.
298
299 PERLIO_F_RDBUF
300 The buffer for this layer currently holds unconsumed data read from
301 layer below.
302
303 PERLIO_F_LINEBUF
304 Layer is line buffered. Write data should be passed to next layer
305 down whenever a "\n" is seen. Any data beyond the "\n" should then
306 be processed.
307
308 PERLIO_F_TEMP
309 File has been "unlink()"ed, or should be deleted on "close()".
310
311 PERLIO_F_OPEN
312 Handle is open.
313
314 PERLIO_F_FASTGETS
315 This instance of this layer supports the "fast "gets"" interface.
316 Normally set based on "PERLIO_K_FASTGETS" for the class and by the
317 existence of the function(s) in the table. However a class that
318 normally provides that interface may need to avoid it on a particu‐
319 lar instance. The "pending" layer needs to do this when it is
320 pushed above a layer which does not support the interface. (Perl's
321 "sv_gets()" does not expect the streams fast "gets" behaviour to
322 change during one "get".)
323
324 Methods in Detail
325
326 fsize
327 Size_t fsize;
328
329 Size of the function table. This is compared against the value Per‐
330 lIO code "knows" as a compatibility check. Future versions may be
331 able to tolerate layers compiled against an old version of the
332 headers.
333
334 name
335 char * name;
336
337 The name of the layer whose open() method Perl should invoke on
338 open(). For example if the layer is called APR, you will call:
339
340 open $fh, ">:APR", ...
341
342 and Perl knows that it has to invoke the PerlIOAPR_open() method
343 implemented by the APR layer.
344
345 size
346 Size_t size;
347
348 The size of the per-instance data structure, e.g.:
349
350 sizeof(PerlIOAPR)
351
352 If this field is zero then "PerlIO_pushed" does not malloc anything
353 and assumes layer's Pushed function will do any required layer
354 stack manipulation - used to avoid malloc/free overhead for dummy
355 layers. If the field is non-zero it must be at least the size of
356 "PerlIOl", "PerlIO_pushed" will allocate memory for the layer's
357 data structures and link new layer onto the stream's stack. (If the
358 layer's Pushed method returns an error indication the layer is
359 popped again.)
360
361 kind
362 IV kind;
363
364 * PERLIO_K_BUFFERED
365 The layer is buffered.
366
367 * PERLIO_K_RAW
368 The layer is acceptable to have in a binmode(FH) stack - i.e.
369 it does not (or will configure itself not to) transform bytes
370 passing through it.
371
372 * PERLIO_K_CANCRLF
373 Layer can translate between "\n" and CRLF line ends.
374
375 * PERLIO_K_FASTGETS
376 Layer allows buffer snooping.
377
378 * PERLIO_K_MULTIARG
379 Used when the layer's open() accepts more arguments than usual.
380 The extra arguments should come not before the "MODE" argument.
381 When this flag is used it's up to the layer to validate the
382 args.
383
384 Pushed
385 IV (*Pushed)(pTHX_ PerlIO *f,const char *mode, SV *arg);
386
387 The only absolutely mandatory method. Called when the layer is
388 pushed onto the stack. The "mode" argument may be NULL if this
389 occurs post-open. The "arg" will be non-"NULL" if an argument
390 string was passed. In most cases this should call "Per‐
391 lIOBase_pushed()" to convert "mode" into the appropriate "PER‐
392 LIO_F_XXXXX" flags in addition to any actions the layer itself
393 takes. If a layer is not expecting an argument it need neither
394 save the one passed to it, nor provide "Getarg()" (it could perhaps
395 "Perl_warn" that the argument was un-expected).
396
397 Returns 0 on success. On failure returns -1 and should set errno.
398
399 Popped
400 IV (*Popped)(pTHX_ PerlIO *f);
401
402 Called when the layer is popped from the stack. A layer will nor‐
403 mally be popped after "Close()" is called. But a layer can be
404 popped without being closed if the program is dynamically managing
405 layers on the stream. In such cases "Popped()" should free any
406 resources (buffers, translation tables, ...) not held directly in
407 the layer's struct. It should also "Unread()" any unconsumed data
408 that has been read and buffered from the layer below back to that
409 layer, so that it can be re-provided to what ever is now above.
410
411 Returns 0 on success and failure. If "Popped()" returns true then
412 perlio.c assumes that either the layer has popped itself, or the
413 layer is super special and needs to be retained for other reasons.
414 In most cases it should return false.
415
416 Open
417 PerlIO * (*Open)(...);
418
419 The "Open()" method has lots of arguments because it combines the
420 functions of perl's "open", "PerlIO_open", perl's "sysopen", "Per‐
421 lIO_fdopen" and "PerlIO_reopen". The full prototype is as follows:
422
423 PerlIO * (*Open)(pTHX_ PerlIO_funcs *tab,
424 AV *layers, IV n,
425 const char *mode,
426 int fd, int imode, int perm,
427 PerlIO *old,
428 int narg, SV **args);
429
430 Open should (perhaps indirectly) call "PerlIO_allocate()" to allo‐
431 cate a slot in the table and associate it with the layers informa‐
432 tion for the opened file, by calling "PerlIO_push". The layers AV
433 is an array of all the layers destined for the "PerlIO *", and any
434 arguments passed to them, n is the index into that array of the
435 layer being called. The macro "PerlIOArg" will return a (possibly
436 "NULL") SV * for the argument passed to the layer.
437
438 The mode string is an ""fopen()"-like" string which would match the
439 regular expression "/^[I#]?[rwa]\+?[bt]?$/".
440
441 The 'I' prefix is used during creation of "stdin".."stderr" via
442 special "PerlIO_fdopen" calls; the '#' prefix means that this is
443 "sysopen" and that imode and perm should be passed to "Perl‐
444 LIO_open3"; 'r' means read, 'w' means write and 'a' means append.
445 The '+' suffix means that both reading and writing/appending are
446 permitted. The 'b' suffix means file should be binary, and 't'
447 means it is text. (Almost all layers should do the IO in binary
448 mode, and ignore the b/t bits. The ":crlf" layer should be pushed
449 to handle the distinction.)
450
451 If old is not "NULL" then this is a "PerlIO_reopen". Perl itself
452 does not use this (yet?) and semantics are a little vague.
453
454 If fd not negative then it is the numeric file descriptor fd, which
455 will be open in a manner compatible with the supplied mode string,
456 the call is thus equivalent to "PerlIO_fdopen". In this case nargs
457 will be zero.
458
459 If nargs is greater than zero then it gives the number of arguments
460 passed to "open", otherwise it will be 1 if for example "Per‐
461 lIO_open" was called. In simple cases SvPV_nolen(*args) is the
462 pathname to open.
463
464 Having said all that translation-only layers do not need to provide
465 "Open()" at all, but rather leave the opening to a lower level
466 layer and wait to be "pushed". If a layer does provide "Open()" it
467 should normally call the "Open()" method of next layer down (if
468 any) and then push itself on top if that succeeds.
469
470 If "PerlIO_push" was performed and open has failed, it must "Per‐
471 lIO_pop" itself, since if it's not, the layer won't be removed and
472 may cause bad problems.
473
474 Returns "NULL" on failure.
475
476 Binmode
477 IV (*Binmode)(pTHX_ PerlIO *f);
478
479 Optional. Used when ":raw" layer is pushed (explicitly or as a
480 result of binmode(FH)). If not present layer will be popped. If
481 present should configure layer as binary (or pop itself) and return
482 0. If it returns -1 for error "binmode" will fail with layer still
483 on the stack.
484
485 Getarg
486 SV * (*Getarg)(pTHX_ PerlIO *f,
487 CLONE_PARAMS *param, int flags);
488
489 Optional. If present should return an SV * representing the string
490 argument passed to the layer when it was pushed. e.g. ":encod‐
491 ing(ascii)" would return an SvPV with value "ascii". (param and
492 flags arguments can be ignored in most cases)
493
494 "Dup" uses "Getarg" to retrieve the argument originally passed to
495 "Pushed", so you must implement this function if your layer has an
496 extra argument to "Pushed" and will ever be "Dup"ed.
497
498 Fileno
499 IV (*Fileno)(pTHX_ PerlIO *f);
500
501 Returns the Unix/Posix numeric file descriptor for the handle. Nor‐
502 mally "PerlIOBase_fileno()" (which just asks next layer down) will
503 suffice for this.
504
505 Returns -1 on error, which is considered to include the case where
506 the layer cannot provide such a file descriptor.
507
508 Dup
509 PerlIO * (*Dup)(pTHX_ PerlIO *f, PerlIO *o,
510 CLONE_PARAMS *param, int flags);
511
512 XXX: Needs more docs.
513
514 Used as part of the "clone" process when a thread is spawned (in
515 which case param will be non-NULL) and when a stream is being
516 duplicated via '&' in the "open".
517
518 Similar to "Open", returns PerlIO* on success, "NULL" on failure.
519
520 Read
521 SSize_t (*Read)(pTHX_ PerlIO *f, void *vbuf, Size_t count);
522
523 Basic read operation.
524
525 Typically will call "Fill" and manipulate pointers (possibly via
526 the API). "PerlIOBuf_read()" may be suitable for derived classes
527 which provide "fast gets" methods.
528
529 Returns actual bytes read, or -1 on an error.
530
531 Unread
532 SSize_t (*Unread)(pTHX_ PerlIO *f,
533 const void *vbuf, Size_t count);
534
535 A superset of stdio's "ungetc()". Should arrange for future reads
536 to see the bytes in "vbuf". If there is no obviously better imple‐
537 mentation then "PerlIOBase_unread()" provides the function by push‐
538 ing a "fake" "pending" layer above the calling layer.
539
540 Returns the number of unread chars.
541
542 Write
543 SSize_t (*Write)(PerlIO *f, const void *vbuf, Size_t count);
544
545 Basic write operation.
546
547 Returns bytes written or -1 on an error.
548
549 Seek
550 IV (*Seek)(pTHX_ PerlIO *f, Off_t offset, int whence);
551
552 Position the file pointer. Should normally call its own "Flush"
553 method and then the "Seek" method of next layer down.
554
555 Returns 0 on success, -1 on failure.
556
557 Tell
558 Off_t (*Tell)(pTHX_ PerlIO *f);
559
560 Return the file pointer. May be based on layers cached concept of
561 position to avoid overhead.
562
563 Returns -1 on failure to get the file pointer.
564
565 Close
566 IV (*Close)(pTHX_ PerlIO *f);
567
568 Close the stream. Should normally call "PerlIOBase_close()" to
569 flush itself and close layers below, and then deallocate any data
570 structures (buffers, translation tables, ...) not held directly in
571 the data structure.
572
573 Returns 0 on success, -1 on failure.
574
575 Flush
576 IV (*Flush)(pTHX_ PerlIO *f);
577
578 Should make stream's state consistent with layers below. That is,
579 any buffered write data should be written, and file position of
580 lower layers adjusted for data read from below but not actually
581 consumed. (Should perhaps "Unread()" such data to the lower
582 layer.)
583
584 Returns 0 on success, -1 on failure.
585
586 Fill
587 IV (*Fill)(pTHX_ PerlIO *f);
588
589 The buffer for this layer should be filled (for read) from layer
590 below. When you "subclass" PerlIOBuf layer, you want to use its
591 _read method and to supply your own fill method, which fills the
592 PerlIOBuf's buffer.
593
594 Returns 0 on success, -1 on failure.
595
596 Eof
597 IV (*Eof)(pTHX_ PerlIO *f);
598
599 Return end-of-file indicator. "PerlIOBase_eof()" is normally suffi‐
600 cient.
601
602 Returns 0 on end-of-file, 1 if not end-of-file, -1 on error.
603
604 Error
605 IV (*Error)(pTHX_ PerlIO *f);
606
607 Return error indicator. "PerlIOBase_error()" is normally suffi‐
608 cient.
609
610 Returns 1 if there is an error (usually when "PERLIO_F_ERROR" is
611 set, 0 otherwise.
612
613 Clearerr
614 void (*Clearerr)(pTHX_ PerlIO *f);
615
616 Clear end-of-file and error indicators. Should call "Per‐
617 lIOBase_clearerr()" to set the "PERLIO_F_XXXXX" flags, which may
618 suffice.
619
620 Setlinebuf
621 void (*Setlinebuf)(pTHX_ PerlIO *f);
622
623 Mark the stream as line buffered. "PerlIOBase_setlinebuf()" sets
624 the PERLIO_F_LINEBUF flag and is normally sufficient.
625
626 Get_base
627 STDCHAR * (*Get_base)(pTHX_ PerlIO *f);
628
629 Allocate (if not already done so) the read buffer for this layer
630 and return pointer to it. Return NULL on failure.
631
632 Get_bufsiz
633 Size_t (*Get_bufsiz)(pTHX_ PerlIO *f);
634
635 Return the number of bytes that last "Fill()" put in the buffer.
636
637 Get_ptr
638 STDCHAR * (*Get_ptr)(pTHX_ PerlIO *f);
639
640 Return the current read pointer relative to this layer's buffer.
641
642 Get_cnt
643 SSize_t (*Get_cnt)(pTHX_ PerlIO *f);
644
645 Return the number of bytes left to be read in the current buffer.
646
647 Set_ptrcnt
648 void (*Set_ptrcnt)(pTHX_ PerlIO *f,
649 STDCHAR *ptr, SSize_t cnt);
650
651 Adjust the read pointer and count of bytes to match "ptr" and/or
652 "cnt". The application (or layer above) must ensure they are con‐
653 sistent. (Checking is allowed by the paranoid.)
654
655 Utilities
656
657 To ask for the next layer down use PerlIONext(PerlIO *f).
658
659 To check that a PerlIO* is valid use PerlIOValid(PerlIO *f). (All this
660 does is really just to check that the pointer is non-NULL and that the
661 pointer behind that is non-NULL.)
662
663 PerlIOBase(PerlIO *f) returns the "Base" pointer, or in other words,
664 the "PerlIOl*" pointer.
665
666 PerlIOSelf(PerlIO* f, type) return the PerlIOBase cast to a type.
667
668 Perl_PerlIO_or_Base(PerlIO* f, callback, base, failure, args) either
669 calls the callback from the functions of the layer f (just by the name
670 of the IO function, like "Read") with the args, or if there is no such
671 callback, calls the base version of the callback with the same args, or
672 if the f is invalid, set errno to EBADF and return failure.
673
674 Perl_PerlIO_or_fail(PerlIO* f, callback, failure, args) either calls
675 the callback of the functions of the layer f with the args, or if there
676 is no such callback, set errno to EINVAL. Or if the f is invalid, set
677 errno to EBADF and return failure.
678
679 Perl_PerlIO_or_Base_void(PerlIO* f, callback, base, args) either calls
680 the callback of the functions of the layer f with the args, or if there
681 is no such callback, calls the base version of the callback with the
682 same args, or if the f is invalid, set errno to EBADF.
683
684 Perl_PerlIO_or_fail_void(PerlIO* f, callback, args) either calls the
685 callback of the functions of the layer f with the args, or if there is
686 no such callback, set errno to EINVAL. Or if the f is invalid, set
687 errno to EBADF.
688
689 Implementing PerlIO Layers
690
691 If you find the implementation document unclear or not sufficient, look
692 at the existing PerlIO layer implementations, which include:
693
694 * C implementations
695 The perlio.c and perliol.h in the Perl core implement the "unix",
696 "perlio", "stdio", "crlf", "utf8", "byte", "raw", "pending" layers,
697 and also the "mmap" and "win32" layers if applicable. (The "win32"
698 is currently unfinished and unused, to see what is used instead in
699 Win32, see "Querying the layers of filehandles" in PerlIO .)
700
701 PerlIO::encoding, PerlIO::scalar, PerlIO::via in the Perl core.
702
703 PerlIO::gzip and APR::PerlIO (mod_perl 2.0) on CPAN.
704
705 * Perl implementations
706 PerlIO::via::QuotedPrint in the Perl core and PerlIO::via::* on
707 CPAN.
708
709 If you are creating a PerlIO layer, you may want to be lazy, in other
710 words, implement only the methods that interest you. The other methods
711 you can either replace with the "blank" methods
712
713 PerlIOBase_noop_ok
714 PerlIOBase_noop_fail
715
716 (which do nothing, and return zero and -1, respectively) or for certain
717 methods you may assume a default behaviour by using a NULL method. The
718 Open method looks for help in the 'parent' layer. The following table
719 summarizes the behaviour:
720
721 method behaviour with NULL
722
723 Clearerr PerlIOBase_clearerr
724 Close PerlIOBase_close
725 Dup PerlIOBase_dup
726 Eof PerlIOBase_eof
727 Error PerlIOBase_error
728 Fileno PerlIOBase_fileno
729 Fill FAILURE
730 Flush SUCCESS
731 Getarg SUCCESS
732 Get_base FAILURE
733 Get_bufsiz FAILURE
734 Get_cnt FAILURE
735 Get_ptr FAILURE
736 Open INHERITED
737 Popped SUCCESS
738 Pushed SUCCESS
739 Read PerlIOBase_read
740 Seek FAILURE
741 Set_cnt FAILURE
742 Set_ptrcnt FAILURE
743 Setlinebuf PerlIOBase_setlinebuf
744 Tell FAILURE
745 Unread PerlIOBase_unread
746 Write FAILURE
747
748 FAILURE Set errno (to EINVAL in UNIXish, to LIB$_INVARG in VMS) and
749 return -1 (for numeric return values) or NULL (for pointers)
750 INHERITED Inherited from the layer below
751 SUCCESS Return 0 (for numeric return values) or a pointer
752
753 Core Layers
754
755 The file "perlio.c" provides the following layers:
756
757 "unix"
758 A basic non-buffered layer which calls Unix/POSIX "read()",
759 "write()", "lseek()", "close()". No buffering. Even on platforms
760 that distinguish between O_TEXT and O_BINARY this layer is always
761 O_BINARY.
762
763 "perlio"
764 A very complete generic buffering layer which provides the whole of
765 PerlIO API. It is also intended to be used as a "base class" for
766 other layers. (For example its "Read()" method is implemented in
767 terms of the "Get_cnt()"/"Get_ptr()"/"Set_ptrcnt()" methods).
768
769 "perlio" over "unix" provides a complete replacement for stdio as
770 seen via PerlIO API. This is the default for USE_PERLIO when sys‐
771 tem's stdio does not permit perl's "fast gets" access, and which do
772 not distinguish between "O_TEXT" and "O_BINARY".
773
774 "stdio"
775 A layer which provides the PerlIO API via the layer scheme, but
776 implements it by calling system's stdio. This is (currently) the
777 default if system's stdio provides sufficient access to allow
778 perl's "fast gets" access and which do not distinguish between
779 "O_TEXT" and "O_BINARY".
780
781 "crlf"
782 A layer derived using "perlio" as a base class. It provides
783 Win32-like "\n" to CR,LF translation. Can either be applied above
784 "perlio" or serve as the buffer layer itself. "crlf" over "unix" is
785 the default if system distinguishes between "O_TEXT" and "O_BINARY"
786 opens. (At some point "unix" will be replaced by a "native" Win32
787 IO layer on that platform, as Win32's read/write layer has various
788 drawbacks.) The "crlf" layer is a reasonable model for a layer
789 which transforms data in some way.
790
791 "mmap"
792 If Configure detects "mmap()" functions this layer is provided
793 (with "perlio" as a "base") which does "read" operations by
794 mmap()ing the file. Performance improvement is marginal on modern
795 systems, so it is mainly there as a proof of concept. It is likely
796 to be unbundled from the core at some point. The "mmap" layer is a
797 reasonable model for a minimalist "derived" layer.
798
799 "pending"
800 An "internal" derivative of "perlio" which can be used to provide
801 Unread() function for layers which have no buffer or cannot be
802 bothered. (Basically this layer's "Fill()" pops itself off the
803 stack and so resumes reading from layer below.)
804
805 "raw"
806 A dummy layer which never exists on the layer stack. Instead when
807 "pushed" it actually pops the stack removing itself, it then calls
808 Binmode function table entry on all the layers in the stack - nor‐
809 mally this (via PerlIOBase_binmode) removes any layers which do not
810 have "PERLIO_K_RAW" bit set. Layers can modify that behaviour by
811 defining their own Binmode entry.
812
813 "utf8"
814 Another dummy layer. When pushed it pops itself and sets the "PER‐
815 LIO_F_UTF8" flag on the layer which was (and now is once more) the
816 top of the stack.
817
818 In addition perlio.c also provides a number of "PerlIOBase_xxxx()"
819 functions which are intended to be used in the table slots of classes
820 which do not need to do anything special for a particular method.
821
822 Extension Layers
823
824 Layers can made available by extension modules. When an unknown layer
825 is encountered the PerlIO code will perform the equivalent of :
826
827 use PerlIO 'layer';
828
829 Where layer is the unknown layer. PerlIO.pm will then attempt to:
830
831 require PerlIO::layer;
832
833 If after that process the layer is still not defined then the "open"
834 will fail.
835
836 The following extension layers are bundled with perl:
837
838 ":encoding"
839 use Encoding;
840
841 makes this layer available, although PerlIO.pm "knows" where to
842 find it. It is an example of a layer which takes an argument as it
843 is called thus:
844
845 open( $fh, "<:encoding(iso-8859-7)", $pathname );
846
847 ":scalar"
848 Provides support for reading data from and writing data to a
849 scalar.
850
851 open( $fh, "+<:scalar", \$scalar );
852
853 When a handle is so opened, then reads get bytes from the string
854 value of $scalar, and writes change the value. In both cases the
855 position in $scalar starts as zero but can be altered via "seek",
856 and determined via "tell".
857
858 Please note that this layer is implied when calling open() thus:
859
860 open( $fh, "+<", \$scalar );
861
862 ":via"
863 Provided to allow layers to be implemented as Perl code. For
864 instance:
865
866 use PerlIO::via::StripHTML;
867 open( my $fh, "<:via(StripHTML)", "index.html" );
868
869 See PerlIO::via for details.
870
872 Things that need to be done to improve this document.
873
874 · Explain how to make a valid fh without going through open()(i.e.
875 apply a layer). For example if the file is not opened through perl,
876 but we want to get back a fh, like it was opened by Perl.
877
878 How PerlIO_apply_layera fits in, where its docs, was it made pub‐
879 lic?
880
881 Currently the example could be something like this:
882
883 PerlIO *foo_to_PerlIO(pTHX_ char *mode, ...)
884 {
885 char *mode; /* "w", "r", etc */
886 const char *layers = ":APR"; /* the layer name */
887 PerlIO *f = PerlIO_allocate(aTHX);
888 if (!f) {
889 return NULL;
890 }
891
892 PerlIO_apply_layers(aTHX_ f, mode, layers);
893
894 if (f) {
895 PerlIOAPR *st = PerlIOSelf(f, PerlIOAPR);
896 /* fill in the st struct, as in _open() */
897 st->file = file;
898 PerlIOBase(f)->flags ⎪= PERLIO_F_OPEN;
899
900 return f;
901 }
902 return NULL;
903 }
904
905 · fix/add the documentation in places marked as XXX.
906
907 · The handling of errors by the layer is not specified. e.g. when $!
908 should be set explicitly, when the error handling should be just
909 delegated to the top layer.
910
911 Probably give some hints on using SETERRNO() or pointers to where
912 they can be found.
913
914 · I think it would help to give some concrete examples to make it
915 easier to understand the API. Of course I agree that the API has to
916 be concise, but since there is no second document that is more of a
917 guide, I think that it'd make it easier to start with the doc which
918 is an API, but has examples in it in places where things are
919 unclear, to a person who is not a PerlIO guru (yet).
920
921
922
923perl v5.8.8 2006-01-07 PERLIOL(1)