1PERLXSTUT(1) Perl Programmers Reference Guide PERLXSTUT(1)
2
6 perlxstut - Tutorial for writing XSUBs
7
9 This tutorial will educate the reader on the steps involved in creating
10 a Perl extension. The reader is assumed to have access to perlguts,
11 perlapi and perlxs.
12 This tutorial starts with very simple examples and becomes more
14 complex, with each new example adding new features. Certain concepts
15 may not be completely explained until later in the tutorial in order to
16 slowly ease the reader into building extensions.
17 This tutorial was written from a Unix point of view. Where I know them
19 to be otherwise different for other platforms (e.g. Win32), I will list
20 them. If you find something that was missed, please let me know.
21
23 make
24 This tutorial assumes that the make program that Perl is configured to
25 use is called "make". Instead of running "make" in the examples that
26 follow, you may have to substitute whatever make program Perl has been
27 configured to use. Running perl -V:make should tell you what it is.
28 Version caveat
30 When writing a Perl extension for general consumption, one should
31 expect that the extension will be used with versions of Perl different
32 from the version available on your machine. Since you are reading this
33 document, the version of Perl on your machine is probably 5.005 or
34 later, but the users of your extension may have more ancient versions.
35 To understand what kinds of incompatibilities one may expect, and in
37 the rare case that the version of Perl on your machine is older than
38 this document, see the section on "Troubleshooting these Examples" for
39 more information.
40 If your extension uses some features of Perl which are not available on
42 older releases of Perl, your users would appreciate an early meaningful
43 warning. You would probably put this information into the README file,
44 but nowadays installation of extensions may be performed automatically,
45 guided by CPAN.pm module or other tools.
46 In MakeMaker-based installations, Makefile.PL provides the earliest
48 opportunity to perform version checks. One can put something like this
49 in Makefile.PL for this purpose:
50 eval { require 5.007 }
52 or die <<EOD;
53 ############
54 ### This module uses frobnication framework which is not available before
55 ### version 5.007 of Perl. Upgrade your Perl before installing Kara::Mba.
56 ############
57 EOD
58 Dynamic Loading versus Static Loading
60 It is commonly thought that if a system does not have the capability to
61 dynamically load a library, you cannot build XSUBs. This is incorrect.
62 You can build them, but you must link the XSUBs subroutines with the
63 rest of Perl, creating a new executable. This situation is similar to
64 Perl 4.
65 This tutorial can still be used on such a system. The XSUB build
67 mechanism will check the system and build a dynamically-loadable
68 library if possible, or else a static library and then, optionally, a
69 new statically-linked executable with that static library linked in.
70 Should you wish to build a statically-linked executable on a system
72 which can dynamically load libraries, you may, in all the following
73 examples, where the command ""make"" with no arguments is executed, run
74 the command ""make perl"" instead.
75 If you have generated such a statically-linked executable by choice,
77 then instead of saying ""make test"", you should say ""make
78 test_static"". On systems that cannot build dynamically-loadable
79 libraries at all, simply saying ""make test"" is sufficient.
80
82 Now let's go on with the show!
83 EXAMPLE 1
85 Our first extension will be very simple. When we call the routine in
86 the extension, it will print out a well-known message and return.
87 Run ""h2xs -A -n Mytest"". This creates a directory named Mytest,
89 possibly under ext/ if that directory exists in the current working
90 directory. Several files will be created under the Mytest dir,
91 including MANIFEST, Makefile.PL, lib/Mytest.pm, Mytest.xs, t/Mytest.t,
92 and Changes.
93 The MANIFEST file contains the names of all the files just created in
95 the Mytest directory.
96 The file Makefile.PL should look something like this:
98 use ExtUtils::MakeMaker;
100 # See lib/ExtUtils/MakeMaker.pm for details of how to influence
101 # the contents of the Makefile that is written.
102 WriteMakefile(
103 NAME => 'Mytest',
104 VERSION_FROM => 'Mytest.pm', # finds $VERSION
105 LIBS => [''], # e.g., '-lm'
106 DEFINE => '', # e.g., '-DHAVE_SOMETHING'
107 INC => '', # e.g., '-I/usr/include/other'
108 );
109 The file Mytest.pm should start with something like this:
111 package Mytest;
113 use 5.008008;
115 use strict;
116 use warnings;
117 require Exporter;
119 our @ISA = qw(Exporter);
121 our %EXPORT_TAGS = ( 'all' => [ qw(
122 ) ] );
124 our @EXPORT_OK = ( @{ $EXPORT_TAGS{'all'} } );
126 our @EXPORT = qw(
128 );
130 our $VERSION = '0.01';
132 require XSLoader;
134 XSLoader::load('Mytest', $VERSION);
135 # Preloaded methods go here.
137 1;
139 __END__
140 # Below is the stub of documentation for your module. You better edit it!
141 The rest of the .pm file contains sample code for providing
143 documentation for the extension.
144 Finally, the Mytest.xs file should look something like this:
146 #include "EXTERN.h"
148 #include "perl.h"
149 #include "XSUB.h"
150 #include "ppport.h"
152 MODULE = Mytest PACKAGE = Mytest
154 Let's edit the .xs file by adding this to the end of the file:
156 void
158 hello()
159 CODE:
160 printf("Hello, world!\n");
161 It is okay for the lines starting at the "CODE:" line to not be
163 indented. However, for readability purposes, it is suggested that you
164 indent CODE: one level and the lines following one more level.
165 Now we'll run ""perl Makefile.PL"". This will create a real Makefile,
167 which make needs. Its output looks something like:
168 % perl Makefile.PL
170 Checking if your kit is complete...
171 Looks good
172 Writing Makefile for Mytest
173 %
174 Now, running make will produce output that looks something like this
176 (some long lines have been shortened for clarity and some extraneous
177 lines have been deleted):
178 % make
180 cp lib/Mytest.pm blib/lib/Mytest.pm
181 perl xsubpp -typemap typemap Mytest.xs > Mytest.xsc && mv Mytest.xsc Mytest.c
182 Please specify prototyping behavior for Mytest.xs (see perlxs manual)
183 cc -c Mytest.c
184 Running Mkbootstrap for Mytest ()
185 chmod 644 Mytest.bs
186 rm -f blib/arch/auto/Mytest/Mytest.so
187 cc -shared -L/usr/local/lib Mytest.o -o blib/arch/auto/Mytest/Mytest.so \
188 \
189 chmod 755 blib/arch/auto/Mytest/Mytest.so
191 cp Mytest.bs blib/arch/auto/Mytest/Mytest.bs
192 chmod 644 blib/arch/auto/Mytest/Mytest.bs
193 Manifying blib/man3/Mytest.3pm
194 %
195 You can safely ignore the line about "prototyping behavior" - it is
197 explained in "The PROTOTYPES: Keyword" in perlxs.
198 Perl has its own special way of easily writing test scripts, but for
200 this example only, we'll create our own test script. Create a file
201 called hello that looks like this:
202 #! /opt/perl5/bin/perl
204 use ExtUtils::testlib;
206 use Mytest;
208 Mytest::hello();
210 Now we make the script executable ("chmod +x hello"), run the script
212 and we should see the following output:
213 % ./hello
215 Hello, world!
216 %
217 EXAMPLE 2
219 Now let's add to our extension a subroutine that will take a single
220 numeric argument as input and return 1 if the number is even or 0 if
221 the number is odd.
222 Add the following to the end of Mytest.xs:
224 int
226 is_even(input)
227 int input
228 CODE:
229 RETVAL = (input % 2 == 0);
230 OUTPUT:
231 RETVAL
232 There does not need to be whitespace at the start of the ""int input""
234 line, but it is useful for improving readability. Placing a semi-colon
235 at the end of that line is also optional. Any amount and kind of
236 whitespace may be placed between the ""int"" and ""input"".
237 Now re-run make to rebuild our new shared library.
239 Now perform the same steps as before, generating a Makefile from the
241 Makefile.PL file, and running make.
242 In order to test that our extension works, we now need to look at the
244 file Mytest.t. This file is set up to imitate the same kind of testing
245 structure that Perl itself has. Within the test script, you perform a
246 number of tests to confirm the behavior of the extension, printing "ok"
247 when the test is correct, "not ok" when it is not.
248 use Test::More tests => 4;
250 BEGIN { use_ok('Mytest') };
251 #########################
253 # Insert your test code below, the Test::More module is use()ed here so read
255 # its man page ( perldoc Test::More ) for help writing this test script.
256 is(&Mytest::is_even(0), 1);
258 is(&Mytest::is_even(1), 0);
259 is(&Mytest::is_even(2), 1);
260 We will be calling the test script through the command ""make test"".
262 You should see output that looks something like this:
263 %make test
265 PERL_DL_NONLAZY=1 /usr/bin/perl "-MExtUtils::Command::MM" "-e" "test_harness(0, 'blib/lib', 'blib/arch')" t/*.t
266 t/Mytest....ok
267 All tests successful.
268 Files=1, Tests=4, 0 wallclock secs ( 0.03 cusr + 0.00 csys = 0.03 CPU)
269 %
270 What has gone on?
272 The program h2xs is the starting point for creating extensions. In
273 later examples we'll see how we can use h2xs to read header files and
274 generate templates to connect to C routines.
275 h2xs creates a number of files in the extension directory. The file
277 Makefile.PL is a perl script which will generate a true Makefile to
278 build the extension. We'll take a closer look at it later.
279 The .pm and .xs files contain the meat of the extension. The .xs file
281 holds the C routines that make up the extension. The .pm file contains
282 routines that tell Perl how to load your extension.
283 Generating the Makefile and running "make" created a directory called
285 blib (which stands for "build library") in the current working
286 directory. This directory will contain the shared library that we will
287 build. Once we have tested it, we can install it into its final
288 location.
289 Invoking the test script via ""make test"" did something very
291 important. It invoked perl with all those "-I" arguments so that it
292 could find the various files that are part of the extension. It is
293 very important that while you are still testing extensions that you use
294 ""make test"". If you try to run the test script all by itself, you
295 will get a fatal error. Another reason it is important to use ""make
296 test"" to run your test script is that if you are testing an upgrade to
297 an already-existing version, using ""make test"" ensures that you will
298 test your new extension, not the already-existing version.
299 When Perl sees a "use extension;", it searches for a file with the same
301 name as the "use"'d extension that has a .pm suffix. If that file
302 cannot be found, Perl dies with a fatal error. The default search path
303 is contained in the @INC array.
304 In our case, Mytest.pm tells perl that it will need the Exporter and
306 Dynamic Loader extensions. It then sets the @ISA and @EXPORT arrays
307 and the $VERSION scalar; finally it tells perl to bootstrap the module.
308 Perl will call its dynamic loader routine (if there is one) and load
309 the shared library.
310 The two arrays @ISA and @EXPORT are very important. The @ISA array
312 contains a list of other packages in which to search for methods (or
313 subroutines) that do not exist in the current package. This is usually
314 only important for object-oriented extensions (which we will talk about
315 much later), and so usually doesn't need to be modified.
316 The @EXPORT array tells Perl which of the extension's variables and
318 subroutines should be placed into the calling package's namespace.
319 Because you don't know if the user has already used your variable and
320 subroutine names, it's vitally important to carefully select what to
321 export. Do not export method or variable names by default without a
322 good reason.
323 As a general rule, if the module is trying to be object-oriented then
325 don't export anything. If it's just a collection of functions and
326 variables, then you can export them via another array, called
327 @EXPORT_OK. This array does not automatically place its subroutine and
328 variable names into the namespace unless the user specifically requests
329 that this be done.
330 See perlmod for more information.
332 The $VERSION variable is used to ensure that the .pm file and the
334 shared library are "in sync" with each other. Any time you make
335 changes to the .pm or .xs files, you should increment the value of this
336 variable.
337 Writing good test scripts
339 The importance of writing good test scripts cannot be over-emphasized.
340 You should closely follow the "ok/not ok" style that Perl itself uses,
341 so that it is very easy and unambiguous to determine the outcome of
342 each test case. When you find and fix a bug, make sure you add a test
343 case for it.
344 By running ""make test"", you ensure that your Mytest.t script runs and
346 uses the correct version of your extension. If you have many test
347 cases, save your test files in the "t" directory and use the suffix
348 ".t". When you run ""make test"", all of these test files will be
349 executed.
350 EXAMPLE 3
352 Our third extension will take one argument as its input, round off that
353 value, and set the argument to the rounded value.
354 Add the following to the end of Mytest.xs:
356 void
358 round(arg)
359 double arg
360 CODE:
361 if (arg > 0.0) {
362 arg = floor(arg + 0.5);
363 } else if (arg < 0.0) {
364 arg = ceil(arg - 0.5);
365 } else {
366 arg = 0.0;
367 }
368 OUTPUT:
369 arg
370 Edit the Makefile.PL file so that the corresponding line looks like
372 this:
373 'LIBS' => ['-lm'], # e.g., '-lm'
375 Generate the Makefile and run make. Change the test number in Mytest.t
377 to "9" and add the following tests:
378 $i = -1.5; &Mytest::round($i); is( $i, -2.0 );
380 $i = -1.1; &Mytest::round($i); is( $i, -1.0 );
381 $i = 0.0; &Mytest::round($i); is( $i, 0.0 );
382 $i = 0.5; &Mytest::round($i); is( $i, 1.0 );
383 $i = 1.2; &Mytest::round($i); is( $i, 1.0 );
384 Running ""make test"" should now print out that all nine tests are
386 okay.
387 Notice that in these new test cases, the argument passed to round was a
389 scalar variable. You might be wondering if you can round a constant or
390 literal. To see what happens, temporarily add the following line to
391 Mytest.t:
392 &Mytest::round(3);
394 Run ""make test"" and notice that Perl dies with a fatal error. Perl
396 won't let you change the value of constants!
397 What's new here?
399 · We've made some changes to Makefile.PL. In this case, we've
400 specified an extra library to be linked into the extension's shared
401 library, the math library libm in this case. We'll talk later
402 about how to write XSUBs that can call every routine in a library.
403 · The value of the function is not being passed back as the
405 function's return value, but by changing the value of the variable
406 that was passed into the function. You might have guessed that
407 when you saw that the return value of round is of type "void".
408 Input and Output Parameters
410 You specify the parameters that will be passed into the XSUB on the
411 line(s) after you declare the function's return value and name. Each
412 input parameter line starts with optional whitespace, and may have an
413 optional terminating semicolon.
414 The list of output parameters occurs at the very end of the function,
416 just after the OUTPUT: directive. The use of RETVAL tells Perl that
417 you wish to send this value back as the return value of the XSUB
418 function. In Example 3, we wanted the "return value" placed in the
419 original variable which we passed in, so we listed it (and not RETVAL)
420 in the OUTPUT: section.
421 The XSUBPP Program
423 The xsubpp program takes the XS code in the .xs file and translates it
424 into C code, placing it in a file whose suffix is .c. The C code
425 created makes heavy use of the C functions within Perl.
426 The TYPEMAP file
428 The xsubpp program uses rules to convert from Perl's data types
429 (scalar, array, etc.) to C's data types (int, char, etc.). These rules
430 are stored in the typemap file ($PERLLIB/ExtUtils/typemap). There's a
431 brief discussion below, but all the nitty-gritty details can be found
432 in perlxstypemap. If you have a new-enough version of perl (5.16 and
433 up) or an upgraded XS compiler ("ExtUtils::ParseXS" 3.13_01 or better),
434 then you can inline typemaps in your XS instead of writing separate
435 files. Either way, this typemap thing is split into three parts:
436 The first section maps various C data types to a name, which
438 corresponds somewhat with the various Perl types. The second section
439 contains C code which xsubpp uses to handle input parameters. The
440 third section contains C code which xsubpp uses to handle output
441 parameters.
442 Let's take a look at a portion of the .c file created for our
444 extension. The file name is Mytest.c:
445 XS(XS_Mytest_round)
447 {
448 dXSARGS;
449 if (items != 1)
450 Perl_croak(aTHX_ "Usage: Mytest::round(arg)");
451 PERL_UNUSED_VAR(cv); /* -W */
452 {
453 double arg = (double)SvNV(ST(0)); /* XXXXX */
454 if (arg > 0.0) {
455 arg = floor(arg + 0.5);
456 } else if (arg < 0.0) {
457 arg = ceil(arg - 0.5);
458 } else {
459 arg = 0.0;
460 }
461 sv_setnv(ST(0), (double)arg); /* XXXXX */
462 SvSETMAGIC(ST(0));
463 }
464 XSRETURN_EMPTY;
465 }
466 Notice the two lines commented with "XXXXX". If you check the first
468 part of the typemap file (or section), you'll see that doubles are of
469 type T_DOUBLE. In the INPUT part of the typemap, an argument that is
470 T_DOUBLE is assigned to the variable arg by calling the routine SvNV on
471 something, then casting it to double, then assigned to the variable
472 arg. Similarly, in the OUTPUT section, once arg has its final value,
473 it is passed to the sv_setnv function to be passed back to the calling
474 subroutine. These two functions are explained in perlguts; we'll talk
475 more later about what that "ST(0)" means in the section on the argument
476 stack.
477 Warning about Output Arguments
479 In general, it's not a good idea to write extensions that modify their
480 input parameters, as in Example 3. Instead, you should probably return
481 multiple values in an array and let the caller handle them (we'll do
482 this in a later example). However, in order to better accommodate
483 calling pre-existing C routines, which often do modify their input
484 parameters, this behavior is tolerated.
485 EXAMPLE 4
487 In this example, we'll now begin to write XSUBs that will interact with
488 pre-defined C libraries. To begin with, we will build a small library
489 of our own, then let h2xs write our .pm and .xs files for us.
490 Create a new directory called Mytest2 at the same level as the
492 directory Mytest. In the Mytest2 directory, create another directory
493 called mylib, and cd into that directory.
494 Here we'll create some files that will generate a test library. These
496 will include a C source file and a header file. We'll also create a
497 Makefile.PL in this directory. Then we'll make sure that running make
498 at the Mytest2 level will automatically run this Makefile.PL file and
499 the resulting Makefile.
500 In the mylib directory, create a file mylib.h that looks like this:
502 #define TESTVAL 4
504 extern double foo(int, long, const char*);
506 Also create a file mylib.c that looks like this:
508 #include <stdlib.h>
510 #include "./mylib.h"
511 double
513 foo(int a, long b, const char *c)
514 {
515 return (a + b + atof(c) + TESTVAL);
516 }
517 And finally create a file Makefile.PL that looks like this:
519 use ExtUtils::MakeMaker;
521 $Verbose = 1;
522 WriteMakefile(
523 NAME => 'Mytest2::mylib',
524 SKIP => [qw(all static static_lib dynamic dynamic_lib)],
525 clean => {'FILES' => 'libmylib$(LIB_EXT)'},
526 );
527 sub MY::top_targets {
530 '
531 all :: static
532 pure_all :: static
534 static :: libmylib$(LIB_EXT)
536 libmylib$(LIB_EXT): $(O_FILES)
538 $(AR) cr libmylib$(LIB_EXT) $(O_FILES)
539 $(RANLIB) libmylib$(LIB_EXT)
540 ';
542 }
543 Make sure you use a tab and not spaces on the lines beginning with
545 "$(AR)" and "$(RANLIB)". Make will not function properly if you use
546 spaces. It has also been reported that the "cr" argument to $(AR) is
547 unnecessary on Win32 systems.
548 We will now create the main top-level Mytest2 files. Change to the
550 directory above Mytest2 and run the following command:
551 % h2xs -O -n Mytest2 ./Mytest2/mylib/mylib.h
553 This will print out a warning about overwriting Mytest2, but that's
555 okay. Our files are stored in Mytest2/mylib, and will be untouched.
556 The normal Makefile.PL that h2xs generates doesn't know about the mylib
558 directory. We need to tell it that there is a subdirectory and that we
559 will be generating a library in it. Let's add the argument MYEXTLIB to
560 the WriteMakefile call so that it looks like this:
561 WriteMakefile(
563 'NAME' => 'Mytest2',
564 'VERSION_FROM' => 'Mytest2.pm', # finds $VERSION
565 'LIBS' => [''], # e.g., '-lm'
566 'DEFINE' => '', # e.g., '-DHAVE_SOMETHING'
567 'INC' => '', # e.g., '-I/usr/include/other'
568 'MYEXTLIB' => 'mylib/libmylib$(LIB_EXT)',
569 );
570 and then at the end add a subroutine (which will override the pre-
572 existing subroutine). Remember to use a tab character to indent the
573 line beginning with "cd"!
574 sub MY::postamble {
576 '
577 $(MYEXTLIB): mylib/Makefile
578 cd mylib && $(MAKE) $(PASSTHRU)
579 ';
580 }
581 Let's also fix the MANIFEST file so that it accurately reflects the
583 contents of our extension. The single line that says "mylib" should be
584 replaced by the following three lines:
585 mylib/Makefile.PL
587 mylib/mylib.c
588 mylib/mylib.h
589 To keep our namespace nice and unpolluted, edit the .pm file and change
591 the variable @EXPORT to @EXPORT_OK. Finally, in the .xs file, edit the
592 #include line to read:
593 #include "mylib/mylib.h"
595 And also add the following function definition to the end of the .xs
597 file:
598 double
600 foo(a,b,c)
601 int a
602 long b
603 const char * c
604 OUTPUT:
605 RETVAL
606 Now we also need to create a typemap because the default Perl doesn't
608 currently support the "const char *" type. Include a new TYPEMAP
609 section in your XS code before the above function:
610 TYPEMAP: <<END;
612 const char * T_PV
613 END
614 Now run perl on the top-level Makefile.PL. Notice that it also created
616 a Makefile in the mylib directory. Run make and watch that it does cd
617 into the mylib directory and run make in there as well.
618 Now edit the Mytest2.t script and change the number of tests to "4",
620 and add the following lines to the end of the script:
621 is( &Mytest2::foo(1, 2, "Hello, world!"), 7 );
623 is( &Mytest2::foo(1, 2, "0.0"), 7 );
624 ok( abs(&Mytest2::foo(0, 0, "-3.4") - 0.6) <= 0.01 );
625 (When dealing with floating-point comparisons, it is best to not check
627 for equality, but rather that the difference between the expected and
628 actual result is below a certain amount (called epsilon) which is 0.01
629 in this case)
630 Run ""make test"" and all should be well. There are some warnings on
632 missing tests for the Mytest2::mylib extension, but you can ignore
633 them.
634 What has happened here?
636 Unlike previous examples, we've now run h2xs on a real include file.
637 This has caused some extra goodies to appear in both the .pm and .xs
638 files.
639 · In the .xs file, there's now a #include directive with the absolute
641 path to the mylib.h header file. We changed this to a relative
642 path so that we could move the extension directory if we wanted to.
643 · There's now some new C code that's been added to the .xs file. The
645 purpose of the "constant" routine is to make the values that are
646 #define'd in the header file accessible by the Perl script (by
647 calling either "TESTVAL" or &Mytest2::TESTVAL). There's also some
648 XS code to allow calls to the "constant" routine.
649 · The .pm file originally exported the name "TESTVAL" in the @EXPORT
651 array. This could lead to name clashes. A good rule of thumb is
652 that if the #define is only going to be used by the C routines
653 themselves, and not by the user, they should be removed from the
654 @EXPORT array. Alternately, if you don't mind using the "fully
655 qualified name" of a variable, you could move most or all of the
656 items from the @EXPORT array into the @EXPORT_OK array.
657 · If our include file had contained #include directives, these would
659 not have been processed by h2xs. There is no good solution to this
660 right now.
661 · We've also told Perl about the library that we built in the mylib
663 subdirectory. That required only the addition of the "MYEXTLIB"
664 variable to the WriteMakefile call and the replacement of the
665 postamble subroutine to cd into the subdirectory and run make. The
666 Makefile.PL for the library is a bit more complicated, but not
667 excessively so. Again we replaced the postamble subroutine to
668 insert our own code. This code simply specified that the library
669 to be created here was a static archive library (as opposed to a
670 dynamically loadable library) and provided the commands to build
671 it.
672 Anatomy of .xs file
674 The .xs file of "EXAMPLE 4" contained some new elements. To understand
675 the meaning of these elements, pay attention to the line which reads
676 MODULE = Mytest2 PACKAGE = Mytest2
678 Anything before this line is plain C code which describes which headers
680 to include, and defines some convenience functions. No translations
681 are performed on this part, apart from having embedded POD
682 documentation skipped over (see perlpod) it goes into the generated
683 output C file as is.
684 Anything after this line is the description of XSUB functions. These
686 descriptions are translated by xsubpp into C code which implements
687 these functions using Perl calling conventions, and which makes these
688 functions visible from Perl interpreter.
689 Pay a special attention to the function "constant". This name appears
691 twice in the generated .xs file: once in the first part, as a static C
692 function, then another time in the second part, when an XSUB interface
693 to this static C function is defined.
694 This is quite typical for .xs files: usually the .xs file provides an
696 interface to an existing C function. Then this C function is defined
697 somewhere (either in an external library, or in the first part of .xs
698 file), and a Perl interface to this function (i.e. "Perl glue") is
699 described in the second part of .xs file. The situation in "EXAMPLE
700 1", "EXAMPLE 2", and "EXAMPLE 3", when all the work is done inside the
701 "Perl glue", is somewhat of an exception rather than the rule.
702 Getting the fat out of XSUBs
704 In "EXAMPLE 4" the second part of .xs file contained the following
705 description of an XSUB:
706 double
708 foo(a,b,c)
709 int a
710 long b
711 const char * c
712 OUTPUT:
713 RETVAL
714 Note that in contrast with "EXAMPLE 1", "EXAMPLE 2" and "EXAMPLE 3",
716 this description does not contain the actual code for what is done
717 during a call to Perl function foo(). To understand what is going on
718 here, one can add a CODE section to this XSUB:
719 double
721 foo(a,b,c)
722 int a
723 long b
724 const char * c
725 CODE:
726 RETVAL = foo(a,b,c);
727 OUTPUT:
728 RETVAL
729 However, these two XSUBs provide almost identical generated C code:
731 xsubpp compiler is smart enough to figure out the "CODE:" section from
732 the first two lines of the description of XSUB. What about "OUTPUT:"
733 section? In fact, that is absolutely the same! The "OUTPUT:" section
734 can be removed as well, as far as "CODE:" section or "PPCODE:" section
735 is not specified: xsubpp can see that it needs to generate a function
736 call section, and will autogenerate the OUTPUT section too. Thus one
737 can shortcut the XSUB to become:
738 double
740 foo(a,b,c)
741 int a
742 long b
743 const char * c
744 Can we do the same with an XSUB
746 int
748 is_even(input)
749 int input
750 CODE:
751 RETVAL = (input % 2 == 0);
752 OUTPUT:
753 RETVAL
754 of "EXAMPLE 2"? To do this, one needs to define a C function "int
756 is_even(int input)". As we saw in "Anatomy of .xs file", a proper
757 place for this definition is in the first part of .xs file. In fact a
758 C function
759 int
761 is_even(int arg)
762 {
763 return (arg % 2 == 0);
764 }
765 is probably overkill for this. Something as simple as a "#define" will
767 do too:
768 #define is_even(arg) ((arg) % 2 == 0)
770 After having this in the first part of .xs file, the "Perl glue" part
772 becomes as simple as
773 int
775 is_even(input)
776 int input
777 This technique of separation of the glue part from the workhorse part
779 has obvious tradeoffs: if you want to change a Perl interface, you need
780 to change two places in your code. However, it removes a lot of
781 clutter, and makes the workhorse part independent from idiosyncrasies
782 of Perl calling convention. (In fact, there is nothing Perl-specific
783 in the above description, a different version of xsubpp might have
784 translated this to TCL glue or Python glue as well.)
785 More about XSUB arguments
787 With the completion of Example 4, we now have an easy way to simulate
788 some real-life libraries whose interfaces may not be the cleanest in
789 the world. We shall now continue with a discussion of the arguments
790 passed to the xsubpp compiler.
791 When you specify arguments to routines in the .xs file, you are really
793 passing three pieces of information for each argument listed. The
794 first piece is the order of that argument relative to the others
795 (first, second, etc). The second is the type of argument, and consists
796 of the type declaration of the argument (e.g., int, char*, etc). The
797 third piece is the calling convention for the argument in the call to
798 the library function.
799 While Perl passes arguments to functions by reference, C passes
801 arguments by value; to implement a C function which modifies data of
802 one of the "arguments", the actual argument of this C function would be
803 a pointer to the data. Thus two C functions with declarations
804 int string_length(char *s);
806 int upper_case_char(char *cp);
807 may have completely different semantics: the first one may inspect an
809 array of chars pointed by s, and the second one may immediately
810 dereference "cp" and manipulate *cp only (using the return value as,
811 say, a success indicator). From Perl one would use these functions in
812 a completely different manner.
813 One conveys this info to xsubpp by replacing "*" before the argument by
815 "&". "&" means that the argument should be passed to a library
816 function by its address. The above two function may be XSUB-ified as
817 int
819 string_length(s)
820 char * s
821 int
823 upper_case_char(cp)
824 char &cp
825 For example, consider:
827 int
829 foo(a,b)
830 char &a
831 char * b
832 The first Perl argument to this function would be treated as a char and
834 assigned to the variable a, and its address would be passed into the
835 function foo. The second Perl argument would be treated as a string
836 pointer and assigned to the variable b. The value of b would be passed
837 into the function foo. The actual call to the function foo that xsubpp
838 generates would look like this:
839 foo(&a, b);
841 xsubpp will parse the following function argument lists identically:
843 char &a
845 char&a
846 char & a
847 However, to help ease understanding, it is suggested that you place a
849 "&" next to the variable name and away from the variable type), and
850 place a "*" near the variable type, but away from the variable name (as
851 in the call to foo above). By doing so, it is easy to understand
852 exactly what will be passed to the C function; it will be whatever is
853 in the "last column".
854 You should take great pains to try to pass the function the type of
856 variable it wants, when possible. It will save you a lot of trouble in
857 the long run.
858 The Argument Stack
860 If we look at any of the C code generated by any of the examples except
861 example 1, you will notice a number of references to ST(n), where n is
862 usually 0. "ST" is actually a macro that points to the n'th argument
863 on the argument stack. ST(0) is thus the first argument on the stack
864 and therefore the first argument passed to the XSUB, ST(1) is the
865 second argument, and so on.
866 When you list the arguments to the XSUB in the .xs file, that tells
868 xsubpp which argument corresponds to which of the argument stack (i.e.,
869 the first one listed is the first argument, and so on). You invite
870 disaster if you do not list them in the same order as the function
871 expects them.
872 The actual values on the argument stack are pointers to the values
874 passed in. When an argument is listed as being an OUTPUT value, its
875 corresponding value on the stack (i.e., ST(0) if it was the first
876 argument) is changed. You can verify this by looking at the C code
877 generated for Example 3. The code for the round() XSUB routine
878 contains lines that look like this:
879 double arg = (double)SvNV(ST(0));
881 /* Round the contents of the variable arg */
882 sv_setnv(ST(0), (double)arg);
883 The arg variable is initially set by taking the value from ST(0), then
885 is stored back into ST(0) at the end of the routine.
886 XSUBs are also allowed to return lists, not just scalars. This must be
888 done by manipulating stack values ST(0), ST(1), etc, in a subtly
889 different way. See perlxs for details.
890 XSUBs are also allowed to avoid automatic conversion of Perl function
892 arguments to C function arguments. See perlxs for details. Some
893 people prefer manual conversion by inspecting ST(i) even in the cases
894 when automatic conversion will do, arguing that this makes the logic of
895 an XSUB call clearer. Compare with "Getting the fat out of XSUBs" for
896 a similar tradeoff of a complete separation of "Perl glue" and
897 "workhorse" parts of an XSUB.
898 While experts may argue about these idioms, a novice to Perl guts may
900 prefer a way which is as little Perl-guts-specific as possible, meaning
901 automatic conversion and automatic call generation, as in "Getting the
902 fat out of XSUBs". This approach has the additional benefit of
903 protecting the XSUB writer from future changes to the Perl API.
904 Extending your Extension
906 Sometimes you might want to provide some extra methods or subroutines
907 to assist in making the interface between Perl and your extension
908 simpler or easier to understand. These routines should live in the .pm
909 file. Whether they are automatically loaded when the extension itself
910 is loaded or only loaded when called depends on where in the .pm file
911 the subroutine definition is placed. You can also consult AutoLoader
912 for an alternate way to store and load your extra subroutines.
913 Documenting your Extension
915 There is absolutely no excuse for not documenting your extension.
916 Documentation belongs in the .pm file. This file will be fed to
917 pod2man, and the embedded documentation will be converted to the
918 manpage format, then placed in the blib directory. It will be copied
919 to Perl's manpage directory when the extension is installed.
920 You may intersperse documentation and Perl code within the .pm file.
922 In fact, if you want to use method autoloading, you must do this, as
923 the comment inside the .pm file explains.
924 See perlpod for more information about the pod format.
926 Installing your Extension
928 Once your extension is complete and passes all its tests, installing it
929 is quite simple: you simply run "make install". You will either need
930 to have write permission into the directories where Perl is installed,
931 or ask your system administrator to run the make for you.
932 Alternately, you can specify the exact directory to place the
934 extension's files by placing a "PREFIX=/destination/directory" after
935 the make install. (or in between the make and install if you have a
936 brain-dead version of make). This can be very useful if you are
937 building an extension that will eventually be distributed to multiple
938 systems. You can then just archive the files in the destination
939 directory and distribute them to your destination systems.
940 EXAMPLE 5
942 In this example, we'll do some more work with the argument stack. The
943 previous examples have all returned only a single value. We'll now
944 create an extension that returns an array.
945 This extension is very Unix-oriented (struct statfs and the statfs
947 system call). If you are not running on a Unix system, you can
948 substitute for statfs any other function that returns multiple values,
949 you can hard-code values to be returned to the caller (although this
950 will be a bit harder to test the error case), or you can simply not do
951 this example. If you change the XSUB, be sure to fix the test cases to
952 match the changes.
953 Return to the Mytest directory and add the following code to the end of
955 Mytest.xs:
956 void
958 statfs(path)
959 char * path
960 INIT:
961 int i;
962 struct statfs buf;
963 PPCODE:
965 i = statfs(path, &buf);
966 if (i == 0) {
967 XPUSHs(sv_2mortal(newSVnv(buf.f_bavail)));
968 XPUSHs(sv_2mortal(newSVnv(buf.f_bfree)));
969 XPUSHs(sv_2mortal(newSVnv(buf.f_blocks)));
970 XPUSHs(sv_2mortal(newSVnv(buf.f_bsize)));
971 XPUSHs(sv_2mortal(newSVnv(buf.f_ffree)));
972 XPUSHs(sv_2mortal(newSVnv(buf.f_files)));
973 XPUSHs(sv_2mortal(newSVnv(buf.f_type)));
974 } else {
975 XPUSHs(sv_2mortal(newSVnv(errno)));
976 }
977 You'll also need to add the following code to the top of the .xs file,
979 just after the include of "XSUB.h":
980 #include <sys/vfs.h>
982 Also add the following code segment to Mytest.t while incrementing the
984 "9" tests to "11":
985 @a = &Mytest::statfs("/blech");
987 ok( scalar(@a) == 1 && $a[0] == 2 );
988 @a = &Mytest::statfs("/");
989 is( scalar(@a), 7 );
990 New Things in this Example
992 This example added quite a few new concepts. We'll take them one at a
993 time.
994 · The INIT: directive contains code that will be placed immediately
996 after the argument stack is decoded. C does not allow variable
997 declarations at arbitrary locations inside a function, so this is
998 usually the best way to declare local variables needed by the XSUB.
999 (Alternatively, one could put the whole "PPCODE:" section into
1000 braces, and put these declarations on top.)
1001 · This routine also returns a different number of arguments depending
1003 on the success or failure of the call to statfs. If there is an
1004 error, the error number is returned as a single-element array. If
1005 the call is successful, then a 7-element array is returned. Since
1006 only one argument is passed into this function, we need room on the
1007 stack to hold the 7 values which may be returned.
1008 We do this by using the PPCODE: directive, rather than the CODE:
1010 directive. This tells xsubpp that we will be managing the return
1011 values that will be put on the argument stack by ourselves.
1012 · When we want to place values to be returned to the caller onto the
1014 stack, we use the series of macros that begin with "XPUSH". There
1015 are five different versions, for placing integers, unsigned
1016 integers, doubles, strings, and Perl scalars on the stack. In our
1017 example, we placed a Perl scalar onto the stack. (In fact this is
1018 the only macro which can be used to return multiple values.)
1019 The XPUSH* macros will automatically extend the return stack to
1021 prevent it from being overrun. You push values onto the stack in
1022 the order you want them seen by the calling program.
1023 · The values pushed onto the return stack of the XSUB are actually
1025 mortal SV's. They are made mortal so that once the values are
1026 copied by the calling program, the SV's that held the returned
1027 values can be deallocated. If they were not mortal, then they
1028 would continue to exist after the XSUB routine returned, but would
1029 not be accessible. This is a memory leak.
1030 · If we were interested in performance, not in code compactness, in
1032 the success branch we would not use "XPUSHs" macros, but "PUSHs"
1033 macros, and would pre-extend the stack before pushing the return
1034 values:
1035 EXTEND(SP, 7);
1037 The tradeoff is that one needs to calculate the number of return
1039 values in advance (though overextending the stack will not
1040 typically hurt anything but memory consumption).
1041 Similarly, in the failure branch we could use "PUSHs" without
1043 extending the stack: the Perl function reference comes to an XSUB
1044 on the stack, thus the stack is always large enough to take one
1045 return value.
1046 EXAMPLE 6
1048 In this example, we will accept a reference to an array as an input
1049 parameter, and return a reference to an array of hashes. This will
1050 demonstrate manipulation of complex Perl data types from an XSUB.
1051 This extension is somewhat contrived. It is based on the code in the
1053 previous example. It calls the statfs function multiple times,
1054 accepting a reference to an array of filenames as input, and returning
1055 a reference to an array of hashes containing the data for each of the
1056 filesystems.
1057 Return to the Mytest directory and add the following code to the end of
1059 Mytest.xs:
1060 SV *
1062 multi_statfs(paths)
1063 SV * paths
1064 INIT:
1065 AV * results;
1066 I32 numpaths = 0;
1067 int i, n;
1068 struct statfs buf;
1069 SvGETMAGIC(paths);
1071 if ((!SvROK(paths))
1072 || (SvTYPE(SvRV(paths)) != SVt_PVAV)
1073 || ((numpaths = av_len((AV *)SvRV(paths))) < 0))
1074 {
1075 XSRETURN_UNDEF;
1076 }
1077 results = (AV *)sv_2mortal((SV *)newAV());
1078 CODE:
1079 for (n = 0; n <= numpaths; n++) {
1080 HV * rh;
1081 STRLEN l;
1082 char * fn = SvPV(*av_fetch((AV *)SvRV(paths), n, 0), l);
1083 i = statfs(fn, &buf);
1085 if (i != 0) {
1086 av_push(results, newSVnv(errno));
1087 continue;
1088 }
1089 rh = (HV *)sv_2mortal((SV *)newHV());
1091 hv_store(rh, "f_bavail", 8, newSVnv(buf.f_bavail), 0);
1093 hv_store(rh, "f_bfree", 7, newSVnv(buf.f_bfree), 0);
1094 hv_store(rh, "f_blocks", 8, newSVnv(buf.f_blocks), 0);
1095 hv_store(rh, "f_bsize", 7, newSVnv(buf.f_bsize), 0);
1096 hv_store(rh, "f_ffree", 7, newSVnv(buf.f_ffree), 0);
1097 hv_store(rh, "f_files", 7, newSVnv(buf.f_files), 0);
1098 hv_store(rh, "f_type", 6, newSVnv(buf.f_type), 0);
1099 av_push(results, newRV((SV *)rh));
1101 }
1102 RETVAL = newRV((SV *)results);
1103 OUTPUT:
1104 RETVAL
1105 And add the following code to Mytest.t, while incrementing the "11"
1107 tests to "13":
1108 $results = Mytest::multi_statfs([ '/', '/blech' ]);
1110 ok( ref $results->[0] );
1111 ok( ! ref $results->[1] );
1112 New Things in this Example
1114 There are a number of new concepts introduced here, described below:
1115 · This function does not use a typemap. Instead, we declare it as
1117 accepting one SV* (scalar) parameter, and returning an SV* value,
1118 and we take care of populating these scalars within the code.
1119 Because we are only returning one value, we don't need a "PPCODE:"
1120 directive - instead, we use "CODE:" and "OUTPUT:" directives.
1121 · When dealing with references, it is important to handle them with
1123 caution. The "INIT:" block first calls SvGETMAGIC(paths), in case
1124 paths is a tied variable. Then it checks that "SvROK" returns
1125 true, which indicates that paths is a valid reference. (Simply
1126 checking "SvROK" won't trigger FETCH on a tied variable.) It then
1127 verifies that the object referenced by paths is an array, using
1128 "SvRV" to dereference paths, and "SvTYPE" to discover its type. As
1129 an added test, it checks that the array referenced by paths is non-
1130 empty, using the "av_len" function (which returns -1 if the array
1131 is empty). The XSRETURN_UNDEF macro is used to abort the XSUB and
1132 return the undefined value whenever all three of these conditions
1133 are not met.
1134 · We manipulate several arrays in this XSUB. Note that an array is
1136 represented internally by an AV* pointer. The functions and macros
1137 for manipulating arrays are similar to the functions in Perl:
1138 "av_len" returns the highest index in an AV*, much like $#array;
1139 "av_fetch" fetches a single scalar value from an array, given its
1140 index; "av_push" pushes a scalar value onto the end of the array,
1141 automatically extending the array as necessary.
1142 Specifically, we read pathnames one at a time from the input array,
1144 and store the results in an output array (results) in the same
1145 order. If statfs fails, the element pushed onto the return array
1146 is the value of errno after the failure. If statfs succeeds,
1147 though, the value pushed onto the return array is a reference to a
1148 hash containing some of the information in the statfs structure.
1149 As with the return stack, it would be possible (and a small
1151 performance win) to pre-extend the return array before pushing data
1152 into it, since we know how many elements we will return:
1153 av_extend(results, numpaths);
1155 · We are performing only one hash operation in this function, which
1157 is storing a new scalar under a key using "hv_store". A hash is
1158 represented by an HV* pointer. Like arrays, the functions for
1159 manipulating hashes from an XSUB mirror the functionality available
1160 from Perl. See perlguts and perlapi for details.
1161 · To create a reference, we use the "newRV" function. Note that you
1163 can cast an AV* or an HV* to type SV* in this case (and many
1164 others). This allows you to take references to arrays, hashes and
1165 scalars with the same function. Conversely, the "SvRV" function
1166 always returns an SV*, which may need to be cast to the appropriate
1167 type if it is something other than a scalar (check with "SvTYPE").
1168 · At this point, xsubpp is doing very little work - the differences
1170 between Mytest.xs and Mytest.c are minimal.
1171 EXAMPLE 7 (Coming Soon)
1173 XPUSH args AND set RETVAL AND assign return value to array
1174 EXAMPLE 8 (Coming Soon)
1176 Setting $!
1177 EXAMPLE 9 Passing open files to XSes
1179 You would think passing files to an XS is difficult, with all the
1180 typeglobs and stuff. Well, it isn't.
1181 Suppose that for some strange reason we need a wrapper around the
1183 standard C library function "fputs()". This is all we need:
1184 #define PERLIO_NOT_STDIO 0
1186 #include "EXTERN.h"
1187 #include "perl.h"
1188 #include "XSUB.h"
1189 #include <stdio.h>
1191 int
1193 fputs(s, stream)
1194 char * s
1195 FILE * stream
1196 The real work is done in the standard typemap.
1198 But you loose all the fine stuff done by the perlio layers. This calls
1200 the stdio function "fputs()", which knows nothing about them.
1201 The standard typemap offers three variants of PerlIO *: "InputStream"
1203 (T_IN), "InOutStream" (T_INOUT) and "OutputStream" (T_OUT). A bare
1204 "PerlIO *" is considered a T_INOUT. If it matters in your code (see
1205 below for why it might) #define or typedef one of the specific names
1206 and use that as the argument or result type in your XS file.
1207 The standard typemap does not contain PerlIO * before perl 5.7, but it
1209 has the three stream variants. Using a PerlIO * directly is not
1210 backwards compatible unless you provide your own typemap.
1211 For streams coming from perl the main difference is that "OutputStream"
1213 will get the output PerlIO * - which may make a difference on a socket.
1214 Like in our example...
1215 For streams being handed to perl a new file handle is created (i.e. a
1217 reference to a new glob) and associated with the PerlIO * provided. If
1218 the read/write state of the PerlIO * is not correct then you may get
1219 errors or warnings from when the file handle is used. So if you opened
1220 the PerlIO * as "w" it should really be an "OutputStream" if open as
1221 "r" it should be an "InputStream".
1222 Now, suppose you want to use perlio layers in your XS. We'll use the
1224 perlio "PerlIO_puts()" function as an example.
1225 In the C part of the XS file (above the first MODULE line) you have
1227 #define OutputStream PerlIO *
1229 or
1230 typedef PerlIO * OutputStream;
1231 And this is the XS code:
1233 int
1235 perlioputs(s, stream)
1236 char * s
1237 OutputStream stream
1238 CODE:
1239 RETVAL = PerlIO_puts(stream, s);
1240 OUTPUT:
1241 RETVAL
1242 We have to use a "CODE" section because "PerlIO_puts()" has the
1244 arguments reversed compared to "fputs()", and we want to keep the
1245 arguments the same.
1246 Wanting to explore this thoroughly, we want to use the stdio "fputs()"
1248 on a PerlIO *. This means we have to ask the perlio system for a stdio
1249 "FILE *":
1250 int
1252 perliofputs(s, stream)
1253 char * s
1254 OutputStream stream
1255 PREINIT:
1256 FILE *fp = PerlIO_findFILE(stream);
1257 CODE:
1258 if (fp != (FILE*) 0) {
1259 RETVAL = fputs(s, fp);
1260 } else {
1261 RETVAL = -1;
1262 }
1263 OUTPUT:
1264 RETVAL
1265 Note: "PerlIO_findFILE()" will search the layers for a stdio layer. If
1267 it can't find one, it will call "PerlIO_exportFILE()" to generate a new
1268 stdio "FILE". Please only call "PerlIO_exportFILE()" if you want a new
1269 "FILE". It will generate one on each call and push a new stdio layer.
1270 So don't call it repeatedly on the same file. "PerlIO_findFILE()" will
1271 retrieve the stdio layer once it has been generated by
1272 "PerlIO_exportFILE()".
1273 This applies to the perlio system only. For versions before 5.7,
1275 "PerlIO_exportFILE()" is equivalent to "PerlIO_findFILE()".
1276 Troubleshooting these Examples
1278 As mentioned at the top of this document, if you are having problems
1279 with these example extensions, you might see if any of these help you.
1280 · In versions of 5.002 prior to the gamma version, the test script in
1282 Example 1 will not function properly. You need to change the "use
1283 lib" line to read:
1284 use lib './blib';
1286 · In versions of 5.002 prior to version 5.002b1h, the test.pl file
1288 was not automatically created by h2xs. This means that you cannot
1289 say "make test" to run the test script. You will need to add the
1290 following line before the "use extension" statement:
1291 use lib './blib';
1293 · In versions 5.000 and 5.001, instead of using the above line, you
1295 will need to use the following line:
1296 BEGIN { unshift(@INC, "./blib") }
1298 · This document assumes that the executable named "perl" is Perl
1300 version 5. Some systems may have installed Perl version 5 as
1301 "perl5".
1302
1304 For more information, consult perlguts, perlapi, perlxs, perlmod, and
1305 perlpod.
1306
1308 Jeff Okamoto <okamoto@corp.hp.com>
1309 Reviewed and assisted by Dean Roehrich, Ilya Zakharevich, Andreas
1311 Koenig, and Tim Bunce.
1312 PerlIO material contributed by Lupe Christoph, with some clarification
1314 by Nick Ing-Simmons.
1315 Changes for h2xs as of Perl 5.8.x by Renee Baecker
1317 Last Changed
1319 2012-01-20
1320perl v5.16.3 2013-03-04 PERLXSTUT(1)