1PERLXSTUT(1) Perl Programmers Reference Guide PERLXSTUT(1)
2
6 perlXStut - Tutorial for writing XSUBs
7
9 This tutorial will educate the reader on the steps involved in creating
10 a Perl extension. The reader is assumed to have access to perlguts,
11 perlapi and perlxs.
12 This tutorial starts with very simple examples and becomes more com‐
14 plex, with each new example adding new features. Certain concepts may
15 not be completely explained until later in the tutorial in order to
16 slowly ease the reader into building extensions.
17 This tutorial was written from a Unix point of view. Where I know them
19 to be otherwise different for other platforms (e.g. Win32), I will list
20 them. If you find something that was missed, please let me know.
21
23 make
24 This tutorial assumes that the make program that Perl is configured to
26 use is called "make". Instead of running "make" in the examples that
27 follow, you may have to substitute whatever make program Perl has been
28 configured to use. Running perl -V:make should tell you what it is.
29 Version caveat
31 When writing a Perl extension for general consumption, one should
33 expect that the extension will be used with versions of Perl different
34 from the version available on your machine. Since you are reading this
35 document, the version of Perl on your machine is probably 5.005 or
36 later, but the users of your extension may have more ancient versions.
37 To understand what kinds of incompatibilities one may expect, and in
39 the rare case that the version of Perl on your machine is older than
40 this document, see the section on "Troubleshooting these Examples" for
41 more information.
42 If your extension uses some features of Perl which are not available on
44 older releases of Perl, your users would appreciate an early meaningful
45 warning. You would probably put this information into the README file,
46 but nowadays installation of extensions may be performed automatically,
47 guided by CPAN.pm module or other tools.
48 In MakeMaker-based installations, Makefile.PL provides the earliest
50 opportunity to perform version checks. One can put something like this
51 in Makefile.PL for this purpose:
52 eval { require 5.007 }
54 or die <<EOD;
55 ############
56 ### This module uses frobnication framework which is not available before
57 ### version 5.007 of Perl. Upgrade your Perl before installing Kara::Mba.
58 ############
59 EOD
60 Dynamic Loading versus Static Loading
62 It is commonly thought that if a system does not have the capability to
64 dynamically load a library, you cannot build XSUBs. This is incorrect.
65 You can build them, but you must link the XSUBs subroutines with the
66 rest of Perl, creating a new executable. This situation is similar to
67 Perl 4.
68 This tutorial can still be used on such a system. The XSUB build mech‐
70 anism will check the system and build a dynamically-loadable library if
71 possible, or else a static library and then, optionally, a new stati‐
72 cally-linked executable with that static library linked in.
73 Should you wish to build a statically-linked executable on a system
75 which can dynamically load libraries, you may, in all the following
76 examples, where the command ""make"" with no arguments is executed, run
77 the command ""make perl"" instead.
78 If you have generated such a statically-linked executable by choice,
80 then instead of saying ""make test"", you should say ""make
81 test_static"". On systems that cannot build dynamically-loadable
82 libraries at all, simply saying ""make test"" is sufficient.
83
85 Now let's go on with the show!
86 EXAMPLE 1
88 Our first extension will be very simple. When we call the routine in
90 the extension, it will print out a well-known message and return.
91 Run ""h2xs -A -n Mytest"". This creates a directory named Mytest, pos‐
93 sibly under ext/ if that directory exists in the current working direc‐
94 tory. Several files will be created in the Mytest dir, including MANI‐
95 FEST, Makefile.PL, Mytest.pm, Mytest.xs, test.pl, and Changes.
96 The MANIFEST file contains the names of all the files just created in
98 the Mytest directory.
99 The file Makefile.PL should look something like this:
101 use ExtUtils::MakeMaker;
103 # See lib/ExtUtils/MakeMaker.pm for details of how to influence
104 # the contents of the Makefile that is written.
105 WriteMakefile(
106 NAME => 'Mytest',
107 VERSION_FROM => 'Mytest.pm', # finds $VERSION
108 LIBS => [''], # e.g., '-lm'
109 DEFINE => '', # e.g., '-DHAVE_SOMETHING'
110 INC => '', # e.g., '-I/usr/include/other'
111 );
112 The file Mytest.pm should start with something like this:
114 package Mytest;
116 use strict;
118 use warnings;
119 require Exporter;
121 require DynaLoader;
122 our @ISA = qw(Exporter DynaLoader);
124 # Items to export into callers namespace by default. Note: do not export
125 # names by default without a very good reason. Use EXPORT_OK instead.
126 # Do not simply export all your public functions/methods/constants.
127 our @EXPORT = qw(
128 );
130 our $VERSION = '0.01';
131 bootstrap Mytest $VERSION;
133 # Preloaded methods go here.
135 # Autoload methods go after __END__, and are processed by the autosplit program.
137 1;
139 __END__
140 # Below is the stub of documentation for your module. You better edit it!
141 The rest of the .pm file contains sample code for providing documenta‐
143 tion for the extension.
144 Finally, the Mytest.xs file should look something like this:
146 #include "EXTERN.h"
148 #include "perl.h"
149 #include "XSUB.h"
150 MODULE = Mytest PACKAGE = Mytest
152 Let's edit the .xs file by adding this to the end of the file:
154 void
156 hello()
157 CODE:
158 printf("Hello, world!\n");
159 It is okay for the lines starting at the "CODE:" line to not be
161 indented. However, for readability purposes, it is suggested that you
162 indent CODE: one level and the lines following one more level.
163 Now we'll run ""perl Makefile.PL"". This will create a real Makefile,
165 which make needs. Its output looks something like:
166 % perl Makefile.PL
168 Checking if your kit is complete...
169 Looks good
170 Writing Makefile for Mytest
171 %
172 Now, running make will produce output that looks something like this
174 (some long lines have been shortened for clarity and some extraneous
175 lines have been deleted):
176 % make
178 umask 0 && cp Mytest.pm ./blib/Mytest.pm
179 perl xsubpp -typemap typemap Mytest.xs >Mytest.tc && mv Mytest.tc Mytest.c
180 Please specify prototyping behavior for Mytest.xs (see perlxs manual)
181 cc -c Mytest.c
182 Running Mkbootstrap for Mytest ()
183 chmod 644 Mytest.bs
184 LD_RUN_PATH="" ld -o ./blib/PA-RISC1.1/auto/Mytest/Mytest.sl -b Mytest.o
185 chmod 755 ./blib/PA-RISC1.1/auto/Mytest/Mytest.sl
186 cp Mytest.bs ./blib/PA-RISC1.1/auto/Mytest/Mytest.bs
187 chmod 644 ./blib/PA-RISC1.1/auto/Mytest/Mytest.bs
188 Manifying ./blib/man3/Mytest.3
189 %
190 You can safely ignore the line about "prototyping behavior" - it is
192 explained in the section "The PROTOTYPES: Keyword" in perlxs.
193 If you are on a Win32 system, and the build process fails with linker
195 errors for functions in the C library, check if your Perl is configured
196 to use PerlCRT (running perl -V:libc should show you if this is the
197 case). If Perl is configured to use PerlCRT, you have to make sure
198 PerlCRT.lib is copied to the same location that msvcrt.lib lives in, so
199 that the compiler can find it on its own. msvcrt.lib is usually found
200 in the Visual C compiler's lib directory (e.g. C:/DevStudio/VC/lib).
201 Perl has its own special way of easily writing test scripts, but for
203 this example only, we'll create our own test script. Create a file
204 called hello that looks like this:
205 #! /opt/perl5/bin/perl
207 use ExtUtils::testlib;
209 use Mytest;
211 Mytest::hello();
213 Now we make the script executable ("chmod +x hello"), run the script
215 and we should see the following output:
216 % ./hello
218 Hello, world!
219 %
220 EXAMPLE 2
222 Now let's add to our extension a subroutine that will take a single
224 numeric argument as input and return 0 if the number is even or 1 if
225 the number is odd.
226 Add the following to the end of Mytest.xs:
228 int
230 is_even(input)
231 int input
232 CODE:
233 RETVAL = (input % 2 == 0);
234 OUTPUT:
235 RETVAL
236 There does not need to be whitespace at the start of the ""int input""
238 line, but it is useful for improving readability. Placing a semi-colon
239 at the end of that line is also optional. Any amount and kind of
240 whitespace may be placed between the ""int"" and ""input"".
241 Now re-run make to rebuild our new shared library.
243 Now perform the same steps as before, generating a Makefile from the
245 Makefile.PL file, and running make.
246 In order to test that our extension works, we now need to look at the
248 file test.pl. This file is set up to imitate the same kind of testing
249 structure that Perl itself has. Within the test script, you perform a
250 number of tests to confirm the behavior of the extension, printing "ok"
251 when the test is correct, "not ok" when it is not. Change the print
252 statement in the BEGIN block to print "1..4", and add the following
253 code to the end of the file:
254 print &Mytest::is_even(0) == 1 ? "ok 2" : "not ok 2", "\n";
256 print &Mytest::is_even(1) == 0 ? "ok 3" : "not ok 3", "\n";
257 print &Mytest::is_even(2) == 1 ? "ok 4" : "not ok 4", "\n";
258 We will be calling the test script through the command ""make test"".
260 You should see output that looks something like this:
261 % make test
263 PERL_DL_NONLAZY=1 /opt/perl5.004/bin/perl (lots of -I arguments) test.pl
264 1..4
265 ok 1
266 ok 2
267 ok 3
268 ok 4
269 %
270 What has gone on?
272 The program h2xs is the starting point for creating extensions. In
274 later examples we'll see how we can use h2xs to read header files and
275 generate templates to connect to C routines.
276 h2xs creates a number of files in the extension directory. The file
278 Makefile.PL is a perl script which will generate a true Makefile to
279 build the extension. We'll take a closer look at it later.
280 The .pm and .xs files contain the meat of the extension. The .xs file
282 holds the C routines that make up the extension. The .pm file contains
283 routines that tell Perl how to load your extension.
284 Generating the Makefile and running "make" created a directory called
286 blib (which stands for "build library") in the current working direc‐
287 tory. This directory will contain the shared library that we will
288 build. Once we have tested it, we can install it into its final loca‐
289 tion.
290 Invoking the test script via ""make test"" did something very impor‐
292 tant. It invoked perl with all those "-I" arguments so that it could
293 find the various files that are part of the extension. It is very
294 important that while you are still testing extensions that you use
295 ""make test"". If you try to run the test script all by itself, you
296 will get a fatal error. Another reason it is important to use ""make
297 test"" to run your test script is that if you are testing an upgrade to
298 an already-existing version, using ""make test"" insures that you will
299 test your new extension, not the already-existing version.
300 When Perl sees a "use extension;", it searches for a file with the same
302 name as the "use"'d extension that has a .pm suffix. If that file can‐
303 not be found, Perl dies with a fatal error. The default search path is
304 contained in the @INC array.
305 In our case, Mytest.pm tells perl that it will need the Exporter and
307 Dynamic Loader extensions. It then sets the @ISA and @EXPORT arrays
308 and the $VERSION scalar; finally it tells perl to bootstrap the module.
309 Perl will call its dynamic loader routine (if there is one) and load
310 the shared library.
311 The two arrays @ISA and @EXPORT are very important. The @ISA array
313 contains a list of other packages in which to search for methods (or
314 subroutines) that do not exist in the current package. This is usually
315 only important for object-oriented extensions (which we will talk about
316 much later), and so usually doesn't need to be modified.
317 The @EXPORT array tells Perl which of the extension's variables and
319 subroutines should be placed into the calling package's namespace.
320 Because you don't know if the user has already used your variable and
321 subroutine names, it's vitally important to carefully select what to
322 export. Do not export method or variable names by default without a
323 good reason.
324 As a general rule, if the module is trying to be object-oriented then
326 don't export anything. If it's just a collection of functions and
327 variables, then you can export them via another array, called
328 @EXPORT_OK. This array does not automatically place its subroutine and
329 variable names into the namespace unless the user specifically requests
330 that this be done.
331 See perlmod for more information.
333 The $VERSION variable is used to ensure that the .pm file and the
335 shared library are "in sync" with each other. Any time you make
336 changes to the .pm or .xs files, you should increment the value of this
337 variable.
338 Writing good test scripts
340 The importance of writing good test scripts cannot be overemphasized.
342 You should closely follow the "ok/not ok" style that Perl itself uses,
343 so that it is very easy and unambiguous to determine the outcome of
344 each test case. When you find and fix a bug, make sure you add a test
345 case for it.
346 By running ""make test"", you ensure that your test.pl script runs and
348 uses the correct version of your extension. If you have many test
349 cases, you might want to copy Perl's test style. Create a directory
350 named "t" in the extension's directory and append the suffix ".t" to
351 the names of your test files. When you run ""make test"", all of these
352 test files will be executed.
353 EXAMPLE 3
355 Our third extension will take one argument as its input, round off that
357 value, and set the argument to the rounded value.
358 Add the following to the end of Mytest.xs:
360 void
362 round(arg)
363 double arg
364 CODE:
365 if (arg > 0.0) {
366 arg = floor(arg + 0.5);
367 } else if (arg < 0.0) {
368 arg = ceil(arg - 0.5);
369 } else {
370 arg = 0.0;
371 }
372 OUTPUT:
373 arg
374 Edit the Makefile.PL file so that the corresponding line looks like
376 this:
377 'LIBS' => ['-lm'], # e.g., '-lm'
379 Generate the Makefile and run make. Change the BEGIN block to print
381 "1..9" and add the following to test.pl:
382 $i = -1.5; &Mytest::round($i); print $i == -2.0 ? "ok 5" : "not ok 5", "\n";
384 $i = -1.1; &Mytest::round($i); print $i == -1.0 ? "ok 6" : "not ok 6", "\n";
385 $i = 0.0; &Mytest::round($i); print $i == 0.0 ? "ok 7" : "not ok 7", "\n";
386 $i = 0.5; &Mytest::round($i); print $i == 1.0 ? "ok 8" : "not ok 8", "\n";
387 $i = 1.2; &Mytest::round($i); print $i == 1.0 ? "ok 9" : "not ok 9", "\n";
388 Running ""make test"" should now print out that all nine tests are
390 okay.
391 Notice that in these new test cases, the argument passed to round was a
393 scalar variable. You might be wondering if you can round a constant or
394 literal. To see what happens, temporarily add the following line to
395 test.pl:
396 &Mytest::round(3);
398 Run ""make test"" and notice that Perl dies with a fatal error. Perl
400 won't let you change the value of constants!
401 What's new here?
403 · We've made some changes to Makefile.PL. In this case, we've speci‐
405 fied an extra library to be linked into the extension's shared
406 library, the math library libm in this case. We'll talk later
407 about how to write XSUBs that can call every routine in a library.
408 · The value of the function is not being passed back as the func‐
410 tion's return value, but by changing the value of the variable that
411 was passed into the function. You might have guessed that when you
412 saw that the return value of round is of type "void".
413 Input and Output Parameters
415 You specify the parameters that will be passed into the XSUB on the
417 line(s) after you declare the function's return value and name. Each
418 input parameter line starts with optional whitespace, and may have an
419 optional terminating semicolon.
420 The list of output parameters occurs at the very end of the function,
422 just before after the OUTPUT: directive. The use of RETVAL tells Perl
423 that you wish to send this value back as the return value of the XSUB
424 function. In Example 3, we wanted the "return value" placed in the
425 original variable which we passed in, so we listed it (and not RETVAL)
426 in the OUTPUT: section.
427 The XSUBPP Program
429 The xsubpp program takes the XS code in the .xs file and translates it
431 into C code, placing it in a file whose suffix is .c. The C code cre‐
432 ated makes heavy use of the C functions within Perl.
433 The TYPEMAP file
435 The xsubpp program uses rules to convert from Perl's data types
437 (scalar, array, etc.) to C's data types (int, char, etc.). These rules
438 are stored in the typemap file ($PERLLIB/ExtUtils/typemap). This file
439 is split into three parts.
440 The first section maps various C data types to a name, which corre‐
442 sponds somewhat with the various Perl types. The second section con‐
443 tains C code which xsubpp uses to handle input parameters. The third
444 section contains C code which xsubpp uses to handle output parameters.
445 Let's take a look at a portion of the .c file created for our exten‐
447 sion. The file name is Mytest.c:
448 XS(XS_Mytest_round)
450 {
451 dXSARGS;
452 if (items != 1)
453 croak("Usage: Mytest::round(arg)");
454 {
455 double arg = (double)SvNV(ST(0)); /* XXXXX */
456 if (arg > 0.0) {
457 arg = floor(arg + 0.5);
458 } else if (arg < 0.0) {
459 arg = ceil(arg - 0.5);
460 } else {
461 arg = 0.0;
462 }
463 sv_setnv(ST(0), (double)arg); /* XXXXX */
464 }
465 XSRETURN(1);
466 }
467 Notice the two lines commented with "XXXXX". If you check the first
469 section of the typemap file, you'll see that doubles are of type T_DOU‐
470 BLE. In the INPUT section, an argument that is T_DOUBLE is assigned to
471 the variable arg by calling the routine SvNV on something, then casting
472 it to double, then assigned to the variable arg. Similarly, in the
473 OUTPUT section, once arg has its final value, it is passed to the
474 sv_setnv function to be passed back to the calling subroutine. These
475 two functions are explained in perlguts; we'll talk more later about
476 what that "ST(0)" means in the section on the argument stack.
477 Warning about Output Arguments
479 In general, it's not a good idea to write extensions that modify their
481 input parameters, as in Example 3. Instead, you should probably return
482 multiple values in an array and let the caller handle them (we'll do
483 this in a later example). However, in order to better accommodate
484 calling pre-existing C routines, which often do modify their input
485 parameters, this behavior is tolerated.
486 EXAMPLE 4
488 In this example, we'll now begin to write XSUBs that will interact with
490 pre-defined C libraries. To begin with, we will build a small library
491 of our own, then let h2xs write our .pm and .xs files for us.
492 Create a new directory called Mytest2 at the same level as the direc‐
494 tory Mytest. In the Mytest2 directory, create another directory called
495 mylib, and cd into that directory.
496 Here we'll create some files that will generate a test library. These
498 will include a C source file and a header file. We'll also create a
499 Makefile.PL in this directory. Then we'll make sure that running make
500 at the Mytest2 level will automatically run this Makefile.PL file and
501 the resulting Makefile.
502 In the mylib directory, create a file mylib.h that looks like this:
504 #define TESTVAL 4
506 extern double foo(int, long, const char*);
508 Also create a file mylib.c that looks like this:
510 #include <stdlib.h>
512 #include "./mylib.h"
513 double
515 foo(int a, long b, const char *c)
516 {
517 return (a + b + atof(c) + TESTVAL);
518 }
519 And finally create a file Makefile.PL that looks like this:
521 use ExtUtils::MakeMaker;
523 $Verbose = 1;
524 WriteMakefile(
525 NAME => 'Mytest2::mylib',
526 SKIP => [qw(all static static_lib dynamic dynamic_lib)],
527 clean => {'FILES' => 'libmylib$(LIB_EXT)'},
528 );
529 sub MY::top_targets {
531 '
532 all :: static
533 pure_all :: static
535 static :: libmylib$(LIB_EXT)
537 libmylib$(LIB_EXT): $(O_FILES)
539 $(AR) cr libmylib$(LIB_EXT) $(O_FILES)
540 $(RANLIB) libmylib$(LIB_EXT)
541 ';
543 }
544 Make sure you use a tab and not spaces on the lines beginning with
546 "$(AR)" and "$(RANLIB)". Make will not function properly if you use
547 spaces. It has also been reported that the "cr" argument to $(AR) is
548 unnecessary on Win32 systems.
549 We will now create the main top-level Mytest2 files. Change to the
551 directory above Mytest2 and run the following command:
552 % h2xs -O -n Mytest2 ./Mytest2/mylib/mylib.h
554 This will print out a warning about overwriting Mytest2, but that's
556 okay. Our files are stored in Mytest2/mylib, and will be untouched.
557 The normal Makefile.PL that h2xs generates doesn't know about the mylib
559 directory. We need to tell it that there is a subdirectory and that we
560 will be generating a library in it. Let's add the argument MYEXTLIB to
561 the WriteMakefile call so that it looks like this:
562 WriteMakefile(
564 'NAME' => 'Mytest2',
565 'VERSION_FROM' => 'Mytest2.pm', # finds $VERSION
566 'LIBS' => [''], # e.g., '-lm'
567 'DEFINE' => '', # e.g., '-DHAVE_SOMETHING'
568 'INC' => '', # e.g., '-I/usr/include/other'
569 'MYEXTLIB' => 'mylib/libmylib$(LIB_EXT)',
570 );
571 and then at the end add a subroutine (which will override the pre-
573 existing subroutine). Remember to use a tab character to indent the
574 line beginning with "cd"!
575 sub MY::postamble {
577 '
578 $(MYEXTLIB): mylib/Makefile
579 cd mylib && $(MAKE) $(PASSTHRU)
580 ';
581 }
582 Let's also fix the MANIFEST file so that it accurately reflects the
584 contents of our extension. The single line that says "mylib" should be
585 replaced by the following three lines:
586 mylib/Makefile.PL
588 mylib/mylib.c
589 mylib/mylib.h
590 To keep our namespace nice and unpolluted, edit the .pm file and change
592 the variable @EXPORT to @EXPORT_OK. Finally, in the .xs file, edit the
593 #include line to read:
594 #include "mylib/mylib.h"
596 And also add the following function definition to the end of the .xs
598 file:
599 double
601 foo(a,b,c)
602 int a
603 long b
604 const char * c
605 OUTPUT:
606 RETVAL
607 Now we also need to create a typemap file because the default Perl
609 doesn't currently support the const char * type. Create a file called
610 typemap in the Mytest2 directory and place the following in it:
611 const char * T_PV
613 Now run perl on the top-level Makefile.PL. Notice that it also created
615 a Makefile in the mylib directory. Run make and watch that it does cd
616 into the mylib directory and run make in there as well.
617 Now edit the test.pl script and change the BEGIN block to print "1..4",
619 and add the following lines to the end of the script:
620 print &Mytest2::foo(1, 2, "Hello, world!") == 7 ? "ok 2\n" : "not ok 2\n";
622 print &Mytest2::foo(1, 2, "0.0") == 7 ? "ok 3\n" : "not ok 3\n";
623 print abs(&Mytest2::foo(0, 0, "-3.4") - 0.6) <= 0.01 ? "ok 4\n" : "not ok 4\n";
624 (When dealing with floating-point comparisons, it is best to not check
626 for equality, but rather that the difference between the expected and
627 actual result is below a certain amount (called epsilon) which is 0.01
628 in this case)
629 Run ""make test"" and all should be well.
631 What has happened here?
633 Unlike previous examples, we've now run h2xs on a real include file.
635 This has caused some extra goodies to appear in both the .pm and .xs
636 files.
637 · In the .xs file, there's now a #include directive with the absolute
639 path to the mylib.h header file. We changed this to a relative
640 path so that we could move the extension directory if we wanted to.
641 · There's now some new C code that's been added to the .xs file. The
643 purpose of the "constant" routine is to make the values that are
644 #define'd in the header file accessible by the Perl script (by
645 calling either "TESTVAL" or &Mytest2::TESTVAL). There's also some
646 XS code to allow calls to the "constant" routine.
647 · The .pm file originally exported the name "TESTVAL" in the @EXPORT
649 array. This could lead to name clashes. A good rule of thumb is
650 that if the #define is only going to be used by the C routines
651 themselves, and not by the user, they should be removed from the
652 @EXPORT array. Alternately, if you don't mind using the "fully
653 qualified name" of a variable, you could move most or all of the
654 items from the @EXPORT array into the @EXPORT_OK array.
655 · If our include file had contained #include directives, these would
657 not have been processed by h2xs. There is no good solution to this
658 right now.
659 · We've also told Perl about the library that we built in the mylib
661 subdirectory. That required only the addition of the "MYEXTLIB"
662 variable to the WriteMakefile call and the replacement of the post‐
663 amble subroutine to cd into the subdirectory and run make. The
664 Makefile.PL for the library is a bit more complicated, but not
665 excessively so. Again we replaced the postamble subroutine to
666 insert our own code. This code simply specified that the library
667 to be created here was a static archive library (as opposed to a
668 dynamically loadable library) and provided the commands to build
669 it.
670 Anatomy of .xs file
672 The .xs file of "EXAMPLE 4" contained some new elements. To understand
674 the meaning of these elements, pay attention to the line which reads
675 MODULE = Mytest2 PACKAGE = Mytest2
677 Anything before this line is plain C code which describes which headers
679 to include, and defines some convenience functions. No translations
680 are performed on this part, apart from having embedded POD documenta‐
681 tion skipped over (see perlpod) it goes into the generated output C
682 file as is.
683 Anything after this line is the description of XSUB functions. These
685 descriptions are translated by xsubpp into C code which implements
686 these functions using Perl calling conventions, and which makes these
687 functions visible from Perl interpreter.
688 Pay a special attention to the function "constant". This name appears
690 twice in the generated .xs file: once in the first part, as a static C
691 function, then another time in the second part, when an XSUB interface
692 to this static C function is defined.
693 This is quite typical for .xs files: usually the .xs file provides an
695 interface to an existing C function. Then this C function is defined
696 somewhere (either in an external library, or in the first part of .xs
697 file), and a Perl interface to this function (i.e. "Perl glue") is
698 described in the second part of .xs file. The situation in "EXAMPLE
699 1", "EXAMPLE 2", and "EXAMPLE 3", when all the work is done inside the
700 "Perl glue", is somewhat of an exception rather than the rule.
701 Getting the fat out of XSUBs
703 In "EXAMPLE 4" the second part of .xs file contained the following
705 description of an XSUB:
706 double
708 foo(a,b,c)
709 int a
710 long b
711 const char * c
712 OUTPUT:
713 RETVAL
714 Note that in contrast with "EXAMPLE 1", "EXAMPLE 2" and "EXAMPLE 3",
716 this description does not contain the actual code for what is done is
717 done during a call to Perl function foo(). To understand what is going
718 on here, one can add a CODE section to this XSUB:
719 double
721 foo(a,b,c)
722 int a
723 long b
724 const char * c
725 CODE:
726 RETVAL = foo(a,b,c);
727 OUTPUT:
728 RETVAL
729 However, these two XSUBs provide almost identical generated C code:
731 xsubpp compiler is smart enough to figure out the "CODE:" section from
732 the first two lines of the description of XSUB. What about "OUTPUT:"
733 section? In fact, that is absolutely the same! The "OUTPUT:" section
734 can be removed as well, as far as "CODE:" section or "PPCODE:" section
735 is not specified: xsubpp can see that it needs to generate a function
736 call section, and will autogenerate the OUTPUT section too. Thus one
737 can shortcut the XSUB to become:
738 double
740 foo(a,b,c)
741 int a
742 long b
743 const char * c
744 Can we do the same with an XSUB
746 int
748 is_even(input)
749 int input
750 CODE:
751 RETVAL = (input % 2 == 0);
752 OUTPUT:
753 RETVAL
754 of "EXAMPLE 2"? To do this, one needs to define a C function "int
756 is_even(int input)". As we saw in "Anatomy of .xs file", a proper
757 place for this definition is in the first part of .xs file. In fact a
758 C function
759 int
761 is_even(int arg)
762 {
763 return (arg % 2 == 0);
764 }
765 is probably overkill for this. Something as simple as a "#define" will
767 do too:
768 #define is_even(arg) ((arg) % 2 == 0)
770 After having this in the first part of .xs file, the "Perl glue" part
772 becomes as simple as
773 int
775 is_even(input)
776 int input
777 This technique of separation of the glue part from the workhorse part
779 has obvious tradeoffs: if you want to change a Perl interface, you need
780 to change two places in your code. However, it removes a lot of clut‐
781 ter, and makes the workhorse part independent from idiosyncrasies of
782 Perl calling convention. (In fact, there is nothing Perl-specific in
783 the above description, a different version of xsubpp might have trans‐
784 lated this to TCL glue or Python glue as well.)
785 More about XSUB arguments
787 With the completion of Example 4, we now have an easy way to simulate
789 some real-life libraries whose interfaces may not be the cleanest in
790 the world. We shall now continue with a discussion of the arguments
791 passed to the xsubpp compiler.
792 When you specify arguments to routines in the .xs file, you are really
794 passing three pieces of information for each argument listed. The
795 first piece is the order of that argument relative to the others
796 (first, second, etc). The second is the type of argument, and consists
797 of the type declaration of the argument (e.g., int, char*, etc). The
798 third piece is the calling convention for the argument in the call to
799 the library function.
800 While Perl passes arguments to functions by reference, C passes argu‐
802 ments by value; to implement a C function which modifies data of one of
803 the "arguments", the actual argument of this C function would be a
804 pointer to the data. Thus two C functions with declarations
805 int string_length(char *s);
807 int upper_case_char(char *cp);
808 may have completely different semantics: the first one may inspect an
810 array of chars pointed by s, and the second one may immediately deref‐
811 erence "cp" and manipulate *cp only (using the return value as, say, a
812 success indicator). From Perl one would use these functions in a com‐
813 pletely different manner.
814 One conveys this info to xsubpp by replacing "*" before the argument by
816 "&". "&" means that the argument should be passed to a library func‐
817 tion by its address. The above two function may be XSUB-ified as
818 int
820 string_length(s)
821 char * s
822 int
824 upper_case_char(cp)
825 char &cp
826 For example, consider:
828 int
830 foo(a,b)
831 char &a
832 char * b
833 The first Perl argument to this function would be treated as a char and
835 assigned to the variable a, and its address would be passed into the
836 function foo. The second Perl argument would be treated as a string
837 pointer and assigned to the variable b. The value of b would be passed
838 into the function foo. The actual call to the function foo that xsubpp
839 generates would look like this:
840 foo(&a, b);
842 xsubpp will parse the following function argument lists identically:
844 char &a
846 char&a
847 char & a
848 However, to help ease understanding, it is suggested that you place a
850 "&" next to the variable name and away from the variable type), and
851 place a "*" near the variable type, but away from the variable name (as
852 in the call to foo above). By doing so, it is easy to understand
853 exactly what will be passed to the C function -- it will be whatever is
854 in the "last column".
855 You should take great pains to try to pass the function the type of
857 variable it wants, when possible. It will save you a lot of trouble in
858 the long run.
859 The Argument Stack
861 If we look at any of the C code generated by any of the examples except
863 example 1, you will notice a number of references to ST(n), where n is
864 usually 0. "ST" is actually a macro that points to the n'th argument
865 on the argument stack. ST(0) is thus the first argument on the stack
866 and therefore the first argument passed to the XSUB, ST(1) is the sec‐
867 ond argument, and so on.
868 When you list the arguments to the XSUB in the .xs file, that tells
870 xsubpp which argument corresponds to which of the argument stack (i.e.,
871 the first one listed is the first argument, and so on). You invite
872 disaster if you do not list them in the same order as the function
873 expects them.
874 The actual values on the argument stack are pointers to the values
876 passed in. When an argument is listed as being an OUTPUT value, its
877 corresponding value on the stack (i.e., ST(0) if it was the first argu‐
878 ment) is changed. You can verify this by looking at the C code gener‐
879 ated for Example 3. The code for the round() XSUB routine contains
880 lines that look like this:
881 double arg = (double)SvNV(ST(0));
883 /* Round the contents of the variable arg */
884 sv_setnv(ST(0), (double)arg);
885 The arg variable is initially set by taking the value from ST(0), then
887 is stored back into ST(0) at the end of the routine.
888 XSUBs are also allowed to return lists, not just scalars. This must be
890 done by manipulating stack values ST(0), ST(1), etc, in a subtly dif‐
891 ferent way. See perlxs for details.
892 XSUBs are also allowed to avoid automatic conversion of Perl function
894 arguments to C function arguments. See perlxs for details. Some peo‐
895 ple prefer manual conversion by inspecting ST(i) even in the cases when
896 automatic conversion will do, arguing that this makes the logic of an
897 XSUB call clearer. Compare with "Getting the fat out of XSUBs" for a
898 similar tradeoff of a complete separation of "Perl glue" and "work‐
899 horse" parts of an XSUB.
900 While experts may argue about these idioms, a novice to Perl guts may
902 prefer a way which is as little Perl-guts-specific as possible, meaning
903 automatic conversion and automatic call generation, as in "Getting the
904 fat out of XSUBs". This approach has the additional benefit of pro‐
905 tecting the XSUB writer from future changes to the Perl API.
906 Extending your Extension
908 Sometimes you might want to provide some extra methods or subroutines
910 to assist in making the interface between Perl and your extension sim‐
911 pler or easier to understand. These routines should live in the .pm
912 file. Whether they are automatically loaded when the extension itself
913 is loaded or only loaded when called depends on where in the .pm file
914 the subroutine definition is placed. You can also consult AutoLoader
915 for an alternate way to store and load your extra subroutines.
916 Documenting your Extension
918 There is absolutely no excuse for not documenting your extension. Doc‐
920 umentation belongs in the .pm file. This file will be fed to pod2man,
921 and the embedded documentation will be converted to the manpage format,
922 then placed in the blib directory. It will be copied to Perl's manpage
923 directory when the extension is installed.
924 You may intersperse documentation and Perl code within the .pm file.
926 In fact, if you want to use method autoloading, you must do this, as
927 the comment inside the .pm file explains.
928 See perlpod for more information about the pod format.
930 Installing your Extension
932 Once your extension is complete and passes all its tests, installing it
934 is quite simple: you simply run "make install". You will either need
935 to have write permission into the directories where Perl is installed,
936 or ask your system administrator to run the make for you.
937 Alternately, you can specify the exact directory to place the exten‐
939 sion's files by placing a "PREFIX=/destination/directory" after the
940 make install. (or in between the make and install if you have a brain-
941 dead version of make). This can be very useful if you are building an
942 extension that will eventually be distributed to multiple systems. You
943 can then just archive the files in the destination directory and dis‐
944 tribute them to your destination systems.
945 EXAMPLE 5
947 In this example, we'll do some more work with the argument stack. The
949 previous examples have all returned only a single value. We'll now
950 create an extension that returns an array.
951 This extension is very Unix-oriented (struct statfs and the statfs sys‐
953 tem call). If you are not running on a Unix system, you can substitute
954 for statfs any other function that returns multiple values, you can
955 hard-code values to be returned to the caller (although this will be a
956 bit harder to test the error case), or you can simply not do this exam‐
957 ple. If you change the XSUB, be sure to fix the test cases to match
958 the changes.
959 Return to the Mytest directory and add the following code to the end of
961 Mytest.xs:
962 void
964 statfs(path)
965 char * path
966 INIT:
967 int i;
968 struct statfs buf;
969 PPCODE:
971 i = statfs(path, &buf);
972 if (i == 0) {
973 XPUSHs(sv_2mortal(newSVnv(buf.f_bavail)));
974 XPUSHs(sv_2mortal(newSVnv(buf.f_bfree)));
975 XPUSHs(sv_2mortal(newSVnv(buf.f_blocks)));
976 XPUSHs(sv_2mortal(newSVnv(buf.f_bsize)));
977 XPUSHs(sv_2mortal(newSVnv(buf.f_ffree)));
978 XPUSHs(sv_2mortal(newSVnv(buf.f_files)));
979 XPUSHs(sv_2mortal(newSVnv(buf.f_type)));
980 XPUSHs(sv_2mortal(newSVnv(buf.f_fsid[0])));
981 XPUSHs(sv_2mortal(newSVnv(buf.f_fsid[1])));
982 } else {
983 XPUSHs(sv_2mortal(newSVnv(errno)));
984 }
985 You'll also need to add the following code to the top of the .xs file,
987 just after the include of "XSUB.h":
988 #include <sys/vfs.h>
990 Also add the following code segment to test.pl while incrementing the
992 "1..9" string in the BEGIN block to "1..11":
993 @a = &Mytest::statfs("/blech");
995 print ((scalar(@a) == 1 && $a[0] == 2) ? "ok 10\n" : "not ok 10\n");
996 @a = &Mytest::statfs("/");
997 print scalar(@a) == 9 ? "ok 11\n" : "not ok 11\n";
998 New Things in this Example
1000 This example added quite a few new concepts. We'll take them one at a
1002 time.
1003 · The INIT: directive contains code that will be placed immediately
1005 after the argument stack is decoded. C does not allow variable
1006 declarations at arbitrary locations inside a function, so this is
1007 usually the best way to declare local variables needed by the XSUB.
1008 (Alternatively, one could put the whole "PPCODE:" section into
1009 braces, and put these declarations on top.)
1010 · This routine also returns a different number of arguments depending
1012 on the success or failure of the call to statfs. If there is an
1013 error, the error number is returned as a single-element array. If
1014 the call is successful, then a 9-element array is returned. Since
1015 only one argument is passed into this function, we need room on the
1016 stack to hold the 9 values which may be returned.
1017 We do this by using the PPCODE: directive, rather than the CODE:
1019 directive. This tells xsubpp that we will be managing the return
1020 values that will be put on the argument stack by ourselves.
1021 · When we want to place values to be returned to the caller onto the
1023 stack, we use the series of macros that begin with "XPUSH". There
1024 are five different versions, for placing integers, unsigned inte‐
1025 gers, doubles, strings, and Perl scalars on the stack. In our
1026 example, we placed a Perl scalar onto the stack. (In fact this is
1027 the only macro which can be used to return multiple values.)
1028 The XPUSH* macros will automatically extend the return stack to
1030 prevent it from being overrun. You push values onto the stack in
1031 the order you want them seen by the calling program.
1032 · The values pushed onto the return stack of the XSUB are actually
1034 mortal SV's. They are made mortal so that once the values are
1035 copied by the calling program, the SV's that held the returned val‐
1036 ues can be deallocated. If they were not mortal, then they would
1037 continue to exist after the XSUB routine returned, but would not be
1038 accessible. This is a memory leak.
1039 · If we were interested in performance, not in code compactness, in
1041 the success branch we would not use "XPUSHs" macros, but "PUSHs"
1042 macros, and would pre-extend the stack before pushing the return
1043 values:
1044 EXTEND(SP, 9);
1046 The tradeoff is that one needs to calculate the number of return
1048 values in advance (though overextending the stack will not typi‐
1049 cally hurt anything but memory consumption).
1050 Similarly, in the failure branch we could use "PUSHs" without
1052 extending the stack: the Perl function reference comes to an XSUB
1053 on the stack, thus the stack is always large enough to take one
1054 return value.
1055 EXAMPLE 6
1057 In this example, we will accept a reference to an array as an input
1059 parameter, and return a reference to an array of hashes. This will
1060 demonstrate manipulation of complex Perl data types from an XSUB.
1061 This extension is somewhat contrived. It is based on the code in the
1063 previous example. It calls the statfs function multiple times, accept‐
1064 ing a reference to an array of filenames as input, and returning a ref‐
1065 erence to an array of hashes containing the data for each of the
1066 filesystems.
1067 Return to the Mytest directory and add the following code to the end of
1069 Mytest.xs:
1070 SV *
1072 multi_statfs(paths)
1073 SV * paths
1074 INIT:
1075 AV * results;
1076 I32 numpaths = 0;
1077 int i, n;
1078 struct statfs buf;
1079 if ((!SvROK(paths))
1081 ⎪⎪ (SvTYPE(SvRV(paths)) != SVt_PVAV)
1082 ⎪⎪ ((numpaths = av_len((AV *)SvRV(paths))) < 0))
1083 {
1084 XSRETURN_UNDEF;
1085 }
1086 results = (AV *)sv_2mortal((SV *)newAV());
1087 CODE:
1088 for (n = 0; n <= numpaths; n++) {
1089 HV * rh;
1090 STRLEN l;
1091 char * fn = SvPV(*av_fetch((AV *)SvRV(paths), n, 0), l);
1092 i = statfs(fn, &buf);
1094 if (i != 0) {
1095 av_push(results, newSVnv(errno));
1096 continue;
1097 }
1098 rh = (HV *)sv_2mortal((SV *)newHV());
1100 hv_store(rh, "f_bavail", 8, newSVnv(buf.f_bavail), 0);
1102 hv_store(rh, "f_bfree", 7, newSVnv(buf.f_bfree), 0);
1103 hv_store(rh, "f_blocks", 8, newSVnv(buf.f_blocks), 0);
1104 hv_store(rh, "f_bsize", 7, newSVnv(buf.f_bsize), 0);
1105 hv_store(rh, "f_ffree", 7, newSVnv(buf.f_ffree), 0);
1106 hv_store(rh, "f_files", 7, newSVnv(buf.f_files), 0);
1107 hv_store(rh, "f_type", 6, newSVnv(buf.f_type), 0);
1108 av_push(results, newRV((SV *)rh));
1110 }
1111 RETVAL = newRV((SV *)results);
1112 OUTPUT:
1113 RETVAL
1114 And add the following code to test.pl, while incrementing the "1..11"
1116 string in the BEGIN block to "1..13":
1117 $results = Mytest::multi_statfs([ '/', '/blech' ]);
1119 print ((ref $results->[0]) ? "ok 12\n" : "not ok 12\n");
1120 print ((! ref $results->[1]) ? "ok 13\n" : "not ok 13\n");
1121 New Things in this Example
1123 There are a number of new concepts introduced here, described below:
1125 · This function does not use a typemap. Instead, we declare it as
1127 accepting one SV* (scalar) parameter, and returning an SV* value,
1128 and we take care of populating these scalars within the code.
1129 Because we are only returning one value, we don't need a "PPCODE:"
1130 directive - instead, we use "CODE:" and "OUTPUT:" directives.
1131 · When dealing with references, it is important to handle them with
1133 caution. The "INIT:" block first checks that "SvROK" returns true,
1134 which indicates that paths is a valid reference. It then verifies
1135 that the object referenced by paths is an array, using "SvRV" to
1136 dereference paths, and "SvTYPE" to discover its type. As an added
1137 test, it checks that the array referenced by paths is non-empty,
1138 using the "av_len" function (which returns -1 if the array is
1139 empty). The XSRETURN_UNDEF macro is used to abort the XSUB and
1140 return the undefined value whenever all three of these conditions
1141 are not met.
1142 · We manipulate several arrays in this XSUB. Note that an array is
1144 represented internally by an AV* pointer. The functions and macros
1145 for manipulating arrays are similar to the functions in Perl:
1146 "av_len" returns the highest index in an AV*, much like $#array;
1147 "av_fetch" fetches a single scalar value from an array, given its
1148 index; "av_push" pushes a scalar value onto the end of the array,
1149 automatically extending the array as necessary.
1150 Specifically, we read pathnames one at a time from the input array,
1152 and store the results in an output array (results) in the same
1153 order. If statfs fails, the element pushed onto the return array
1154 is the value of errno after the failure. If statfs succeeds,
1155 though, the value pushed onto the return array is a reference to a
1156 hash containing some of the information in the statfs structure.
1157 As with the return stack, it would be possible (and a small perfor‐
1159 mance win) to pre-extend the return array before pushing data into
1160 it, since we know how many elements we will return:
1161 av_extend(results, numpaths);
1163 · We are performing only one hash operation in this function, which
1165 is storing a new scalar under a key using "hv_store". A hash is
1166 represented by an HV* pointer. Like arrays, the functions for
1167 manipulating hashes from an XSUB mirror the functionality available
1168 from Perl. See perlguts and perlapi for details.
1169 · To create a reference, we use the "newRV" function. Note that you
1171 can cast an AV* or an HV* to type SV* in this case (and many oth‐
1172 ers). This allows you to take references to arrays, hashes and
1173 scalars with the same function. Conversely, the "SvRV" function
1174 always returns an SV*, which may need to be cast to the appropriate
1175 type if it is something other than a scalar (check with "SvTYPE").
1176 · At this point, xsubpp is doing very little work - the differences
1178 between Mytest.xs and Mytest.c are minimal.
1179 EXAMPLE 7 (Coming Soon)
1181 XPUSH args AND set RETVAL AND assign return value to array
1183 EXAMPLE 8 (Coming Soon)
1185 Setting $!
1187 EXAMPLE 9 Passing open files to XSes
1189 You would think passing files to an XS is difficult, with all the type‐
1191 globs and stuff. Well, it isn't.
1192 Suppose that for some strange reason we need a wrapper around the stan‐
1194 dard C library function "fputs()". This is all we need:
1195 #define PERLIO_NOT_STDIO 0
1197 #include "EXTERN.h"
1198 #include "perl.h"
1199 #include "XSUB.h"
1200 #include <stdio.h>
1202 int
1204 fputs(s, stream)
1205 char * s
1206 FILE * stream
1207 The real work is done in the standard typemap.
1209 But you loose all the fine stuff done by the perlio layers. This calls
1211 the stdio function "fputs()", which knows nothing about them.
1212 The standard typemap offers three variants of PerlIO *: "InputStream"
1214 (T_IN), "InOutStream" (T_INOUT) and "OutputStream" (T_OUT). A bare
1215 "PerlIO *" is considered a T_INOUT. If it matters in your code (see
1216 below for why it might) #define or typedef one of the specific names
1217 and use that as the argument or result type in your XS file.
1218 The standard typemap does not contain PerlIO * before perl 5.7, but it
1220 has the three stream variants. Using a PerlIO * directly is not back‐
1221 wards compatible unless you provide your own typemap.
1222 For streams coming from perl the main difference is that "OutputStream"
1224 will get the output PerlIO * - which may make a difference on a socket.
1225 Like in our example...
1226 For streams being handed to perl a new file handle is created (i.e. a
1228 reference to a new glob) and associated with the PerlIO * provided. If
1229 the read/write state of the PerlIO * is not correct then you may get
1230 errors or warnings from when the file handle is used. So if you opened
1231 the PerlIO * as "w" it should really be an "OutputStream" if open as
1232 "r" it should be an "InputStream".
1233 Now, suppose you want to use perlio layers in your XS. We'll use the
1235 perlio "PerlIO_puts()" function as an example.
1236 In the C part of the XS file (above the first MODULE line) you have
1238 #define OutputStream PerlIO *
1240 or
1241 typedef PerlIO * OutputStream;
1242 And this is the XS code:
1244 int
1246 perlioputs(s, stream)
1247 char * s
1248 OutputStream stream
1249 CODE:
1250 RETVAL = PerlIO_puts(stream, s);
1251 OUTPUT:
1252 RETVAL
1253 We have to use a "CODE" section because "PerlIO_puts()" has the argu‐
1255 ments reversed compared to "fputs()", and we want to keep the arguments
1256 the same.
1257 Wanting to explore this thoroughly, we want to use the stdio "fputs()"
1259 on a PerlIO *. This means we have to ask the perlio system for a stdio
1260 "FILE *":
1261 int
1263 perliofputs(s, stream)
1264 char * s
1265 OutputStream stream
1266 PREINIT:
1267 FILE *fp = PerlIO_findFILE(stream);
1268 CODE:
1269 if (fp != (FILE*) 0) {
1270 RETVAL = fputs(s, fp);
1271 } else {
1272 RETVAL = -1;
1273 }
1274 OUTPUT:
1275 RETVAL
1276 Note: "PerlIO_findFILE()" will search the layers for a stdio layer. If
1278 it can't find one, it will call "PerlIO_exportFILE()" to generate a new
1279 stdio "FILE". Please only call "PerlIO_exportFILE()" if you want a new
1280 "FILE". It will generate one on each call and push a new stdio layer.
1281 So don't call it repeatedly on the same file. "PerlIO()"_findFILE will
1282 retrieve the stdio layer once it has been generated by "PerlIO_export‐
1283 FILE()".
1284 This applies to the perlio system only. For versions before 5.7, "Per‐
1286 lIO_exportFILE()" is equivalent to "PerlIO_findFILE()".
1287 Troubleshooting these Examples
1289 As mentioned at the top of this document, if you are having problems
1291 with these example extensions, you might see if any of these help you.
1292 · In versions of 5.002 prior to the gamma version, the test script in
1294 Example 1 will not function properly. You need to change the "use
1295 lib" line to read:
1296 use lib './blib';
1298 · In versions of 5.002 prior to version 5.002b1h, the test.pl file
1300 was not automatically created by h2xs. This means that you cannot
1301 say "make test" to run the test script. You will need to add the
1302 following line before the "use extension" statement:
1303 use lib './blib';
1305 · In versions 5.000 and 5.001, instead of using the above line, you
1307 will need to use the following line:
1308 BEGIN { unshift(@INC, "./blib") }
1310 · This document assumes that the executable named "perl" is Perl ver‐
1312 sion 5. Some systems may have installed Perl version 5 as "perl5".
1313
1315 For more information, consult perlguts, perlapi, perlxs, perlmod, and
1316 perlpod.
1317
1319 Jeff Okamoto <okamoto@corp.hp.com>
1320 Reviewed and assisted by Dean Roehrich, Ilya Zakharevich, Andreas
1322 Koenig, and Tim Bunce.
1323 PerlIO material contributed by Lupe Christoph, with some clarification
1325 by Nick Ing-Simmons.
1326 Last Changed
1328 2002/05/08
1330perl v5.8.8 2006-01-07 PERLXSTUT(1)