1PERLXSTUT(1) Perl Programmers Reference Guide PERLXSTUT(1)
2
6 perlXStut - Tutorial for writing XSUBs
7
9 This tutorial will educate the reader on the steps involved in creating
10 a Perl extension. The reader is assumed to have access to perlguts,
11 perlapi and perlxs.
12 This tutorial starts with very simple examples and becomes more
14 complex, with each new example adding new features. Certain concepts
15 may not be completely explained until later in the tutorial in order to
16 slowly ease the reader into building extensions.
17 This tutorial was written from a Unix point of view. Where I know them
19 to be otherwise different for other platforms (e.g. Win32), I will list
20 them. If you find something that was missed, please let me know.
21
23 make
24 This tutorial assumes that the make program that Perl is configured to
25 use is called "make". Instead of running "make" in the examples that
26 follow, you may have to substitute whatever make program Perl has been
27 configured to use. Running perl -V:make should tell you what it is.
28 Version caveat
30 When writing a Perl extension for general consumption, one should
31 expect that the extension will be used with versions of Perl different
32 from the version available on your machine. Since you are reading this
33 document, the version of Perl on your machine is probably 5.005 or
34 later, but the users of your extension may have more ancient versions.
35 To understand what kinds of incompatibilities one may expect, and in
37 the rare case that the version of Perl on your machine is older than
38 this document, see the section on "Troubleshooting these Examples" for
39 more information.
40 If your extension uses some features of Perl which are not available on
42 older releases of Perl, your users would appreciate an early meaningful
43 warning. You would probably put this information into the README file,
44 but nowadays installation of extensions may be performed automatically,
45 guided by CPAN.pm module or other tools.
46 In MakeMaker-based installations, Makefile.PL provides the earliest
48 opportunity to perform version checks. One can put something like this
49 in Makefile.PL for this purpose:
50 eval { require 5.007 }
52 or die <<EOD;
53 ############
54 ### This module uses frobnication framework which is not available before
55 ### version 5.007 of Perl. Upgrade your Perl before installing Kara::Mba.
56 ############
57 EOD
58 Dynamic Loading versus Static Loading
60 It is commonly thought that if a system does not have the capability to
61 dynamically load a library, you cannot build XSUBs. This is incorrect.
62 You can build them, but you must link the XSUBs subroutines with the
63 rest of Perl, creating a new executable. This situation is similar to
64 Perl 4.
65 This tutorial can still be used on such a system. The XSUB build
67 mechanism will check the system and build a dynamically-loadable
68 library if possible, or else a static library and then, optionally, a
69 new statically-linked executable with that static library linked in.
70 Should you wish to build a statically-linked executable on a system
72 which can dynamically load libraries, you may, in all the following
73 examples, where the command ""make"" with no arguments is executed, run
74 the command ""make perl"" instead.
75 If you have generated such a statically-linked executable by choice,
77 then instead of saying ""make test"", you should say ""make
78 test_static"". On systems that cannot build dynamically-loadable
79 libraries at all, simply saying ""make test"" is sufficient.
80
82 Now let's go on with the show!
83 EXAMPLE 1
85 Our first extension will be very simple. When we call the routine in
86 the extension, it will print out a well-known message and return.
87 Run ""h2xs -A -n Mytest"". This creates a directory named Mytest,
89 possibly under ext/ if that directory exists in the current working
90 directory. Several files will be created under the Mytest dir,
91 including MANIFEST, Makefile.PL, lib/Mytest.pm, Mytest.xs, t/Mytest.t,
92 and Changes.
93 The MANIFEST file contains the names of all the files just created in
95 the Mytest directory.
96 The file Makefile.PL should look something like this:
98 use ExtUtils::MakeMaker;
100 # See lib/ExtUtils/MakeMaker.pm for details of how to influence
101 # the contents of the Makefile that is written.
102 WriteMakefile(
103 NAME => 'Mytest',
104 VERSION_FROM => 'Mytest.pm', # finds $VERSION
105 LIBS => [''], # e.g., '-lm'
106 DEFINE => '', # e.g., '-DHAVE_SOMETHING'
107 INC => '', # e.g., '-I/usr/include/other'
108 );
109 The file Mytest.pm should start with something like this:
111 package Mytest;
113 use 5.008008;
115 use strict;
116 use warnings;
117 require Exporter;
119 our @ISA = qw(Exporter);
121 our %EXPORT_TAGS = ( 'all' => [ qw(
122 ) ] );
124 our @EXPORT_OK = ( @{ $EXPORT_TAGS{'all'} } );
126 our @EXPORT = qw(
128 );
130 our $VERSION = '0.01';
132 require XSLoader;
134 XSLoader::load('Mytest', $VERSION);
135 # Preloaded methods go here.
137 1;
139 __END__
140 # Below is the stub of documentation for your module. You better edit it!
141 The rest of the .pm file contains sample code for providing
143 documentation for the extension.
144 Finally, the Mytest.xs file should look something like this:
146 #include "EXTERN.h"
148 #include "perl.h"
149 #include "XSUB.h"
150 #include "ppport.h"
152 MODULE = Mytest PACKAGE = Mytest
154 Let's edit the .xs file by adding this to the end of the file:
156 void
158 hello()
159 CODE:
160 printf("Hello, world!\n");
161 It is okay for the lines starting at the "CODE:" line to not be
163 indented. However, for readability purposes, it is suggested that you
164 indent CODE: one level and the lines following one more level.
165 Now we'll run ""perl Makefile.PL"". This will create a real Makefile,
167 which make needs. Its output looks something like:
168 % perl Makefile.PL
170 Checking if your kit is complete...
171 Looks good
172 Writing Makefile for Mytest
173 %
174 Now, running make will produce output that looks something like this
176 (some long lines have been shortened for clarity and some extraneous
177 lines have been deleted):
178 % make
180 cp lib/Mytest.pm blib/lib/Mytest.pm
181 perl xsubpp -typemap typemap Mytest.xs > Mytest.xsc && mv Mytest.xsc Mytest.c
182 Please specify prototyping behavior for Mytest.xs (see perlxs manual)
183 cc -c Mytest.c
184 Running Mkbootstrap for Mytest ()
185 chmod 644 Mytest.bs
186 rm -f blib/arch/auto/Mytest/Mytest.so
187 cc -shared -L/usr/local/lib Mytest.o -o blib/arch/auto/Mytest/Mytest.so \
188 \
189 chmod 755 blib/arch/auto/Mytest/Mytest.so
191 cp Mytest.bs blib/arch/auto/Mytest/Mytest.bs
192 chmod 644 blib/arch/auto/Mytest/Mytest.bs
193 Manifying blib/man3/Mytest.3pm
194 %
195 You can safely ignore the line about "prototyping behavior" - it is
197 explained in "The PROTOTYPES: Keyword" in perlxs.
198 If you are on a Win32 system, and the build process fails with linker
200 errors for functions in the C library, check if your Perl is configured
201 to use PerlCRT (running perl -V:libc should show you if this is the
202 case). If Perl is configured to use PerlCRT, you have to make sure
203 PerlCRT.lib is copied to the same location that msvcrt.lib lives in, so
204 that the compiler can find it on its own. msvcrt.lib is usually found
205 in the Visual C compiler's lib directory (e.g. C:/DevStudio/VC/lib).
206 Perl has its own special way of easily writing test scripts, but for
208 this example only, we'll create our own test script. Create a file
209 called hello that looks like this:
210 #! /opt/perl5/bin/perl
212 use ExtUtils::testlib;
214 use Mytest;
216 Mytest::hello();
218 Now we make the script executable ("chmod +x hello"), run the script
220 and we should see the following output:
221 % ./hello
223 Hello, world!
224 %
225 EXAMPLE 2
227 Now let's add to our extension a subroutine that will take a single
228 numeric argument as input and return 0 if the number is even or 1 if
229 the number is odd.
230 Add the following to the end of Mytest.xs:
232 int
234 is_even(input)
235 int input
236 CODE:
237 RETVAL = (input % 2 == 0);
238 OUTPUT:
239 RETVAL
240 There does not need to be whitespace at the start of the ""int input""
242 line, but it is useful for improving readability. Placing a semi-colon
243 at the end of that line is also optional. Any amount and kind of
244 whitespace may be placed between the ""int"" and ""input"".
245 Now re-run make to rebuild our new shared library.
247 Now perform the same steps as before, generating a Makefile from the
249 Makefile.PL file, and running make.
250 In order to test that our extension works, we now need to look at the
252 file Mytest.t. This file is set up to imitate the same kind of testing
253 structure that Perl itself has. Within the test script, you perform a
254 number of tests to confirm the behavior of the extension, printing "ok"
255 when the test is correct, "not ok" when it is not.
256 use Test::More tests => 4;
258 BEGIN { use_ok('Mytest') };
259 #########################
261 # Insert your test code below, the Test::More module is use()ed here so read
263 # its man page ( perldoc Test::More ) for help writing this test script.
264 is(&Mytest::is_even(0), 1);
266 is(&Mytest::is_even(1), 0);
267 is(&Mytest::is_even(2), 1);
268 We will be calling the test script through the command ""make test"".
270 You should see output that looks something like this:
271 %make test
273 PERL_DL_NONLAZY=1 /usr/bin/perl "-MExtUtils::Command::MM" "-e" "test_harness(0, 'blib/lib', 'blib/arch')" t/*.t
274 t/Mytest....ok
275 All tests successful.
276 Files=1, Tests=4, 0 wallclock secs ( 0.03 cusr + 0.00 csys = 0.03 CPU)
277 %
278 What has gone on?
280 The program h2xs is the starting point for creating extensions. In
281 later examples we'll see how we can use h2xs to read header files and
282 generate templates to connect to C routines.
283 h2xs creates a number of files in the extension directory. The file
285 Makefile.PL is a perl script which will generate a true Makefile to
286 build the extension. We'll take a closer look at it later.
287 The .pm and .xs files contain the meat of the extension. The .xs file
289 holds the C routines that make up the extension. The .pm file contains
290 routines that tell Perl how to load your extension.
291 Generating the Makefile and running "make" created a directory called
293 blib (which stands for "build library") in the current working
294 directory. This directory will contain the shared library that we will
295 build. Once we have tested it, we can install it into its final
296 location.
297 Invoking the test script via ""make test"" did something very
299 important. It invoked perl with all those "-I" arguments so that it
300 could find the various files that are part of the extension. It is
301 very important that while you are still testing extensions that you use
302 ""make test"". If you try to run the test script all by itself, you
303 will get a fatal error. Another reason it is important to use ""make
304 test"" to run your test script is that if you are testing an upgrade to
305 an already-existing version, using ""make test"" ensures that you will
306 test your new extension, not the already-existing version.
307 When Perl sees a "use extension;", it searches for a file with the same
309 name as the "use"'d extension that has a .pm suffix. If that file
310 cannot be found, Perl dies with a fatal error. The default search path
311 is contained in the @INC array.
312 In our case, Mytest.pm tells perl that it will need the Exporter and
314 Dynamic Loader extensions. It then sets the @ISA and @EXPORT arrays
315 and the $VERSION scalar; finally it tells perl to bootstrap the module.
316 Perl will call its dynamic loader routine (if there is one) and load
317 the shared library.
318 The two arrays @ISA and @EXPORT are very important. The @ISA array
320 contains a list of other packages in which to search for methods (or
321 subroutines) that do not exist in the current package. This is usually
322 only important for object-oriented extensions (which we will talk about
323 much later), and so usually doesn't need to be modified.
324 The @EXPORT array tells Perl which of the extension's variables and
326 subroutines should be placed into the calling package's namespace.
327 Because you don't know if the user has already used your variable and
328 subroutine names, it's vitally important to carefully select what to
329 export. Do not export method or variable names by default without a
330 good reason.
331 As a general rule, if the module is trying to be object-oriented then
333 don't export anything. If it's just a collection of functions and
334 variables, then you can export them via another array, called
335 @EXPORT_OK. This array does not automatically place its subroutine and
336 variable names into the namespace unless the user specifically requests
337 that this be done.
338 See perlmod for more information.
340 The $VERSION variable is used to ensure that the .pm file and the
342 shared library are "in sync" with each other. Any time you make
343 changes to the .pm or .xs files, you should increment the value of this
344 variable.
345 Writing good test scripts
347 The importance of writing good test scripts cannot be over-emphasized.
348 You should closely follow the "ok/not ok" style that Perl itself uses,
349 so that it is very easy and unambiguous to determine the outcome of
350 each test case. When you find and fix a bug, make sure you add a test
351 case for it.
352 By running ""make test"", you ensure that your Mytest.t script runs and
354 uses the correct version of your extension. If you have many test
355 cases, save your test files in the "t" directory and use the suffix
356 ".t". When you run ""make test"", all of these test files will be
357 executed.
358 EXAMPLE 3
360 Our third extension will take one argument as its input, round off that
361 value, and set the argument to the rounded value.
362 Add the following to the end of Mytest.xs:
364 void
366 round(arg)
367 double arg
368 CODE:
369 if (arg > 0.0) {
370 arg = floor(arg + 0.5);
371 } else if (arg < 0.0) {
372 arg = ceil(arg - 0.5);
373 } else {
374 arg = 0.0;
375 }
376 OUTPUT:
377 arg
378 Edit the Makefile.PL file so that the corresponding line looks like
380 this:
381 'LIBS' => ['-lm'], # e.g., '-lm'
383 Generate the Makefile and run make. Change the test number in Mytest.t
385 to "9" and add the following tests:
386 $i = -1.5; &Mytest::round($i); is( $i, -2.0 );
388 $i = -1.1; &Mytest::round($i); is( $i, -1.0 );
389 $i = 0.0; &Mytest::round($i); is( $i, 0.0 );
390 $i = 0.5; &Mytest::round($i); is( $i, 1.0 );
391 $i = 1.2; &Mytest::round($i); is( $i, 1.0 );
392 Running ""make test"" should now print out that all nine tests are
394 okay.
395 Notice that in these new test cases, the argument passed to round was a
397 scalar variable. You might be wondering if you can round a constant or
398 literal. To see what happens, temporarily add the following line to
399 Mytest.t:
400 &Mytest::round(3);
402 Run ""make test"" and notice that Perl dies with a fatal error. Perl
404 won't let you change the value of constants!
405 What's new here?
407 · We've made some changes to Makefile.PL. In this case, we've
408 specified an extra library to be linked into the extension's shared
409 library, the math library libm in this case. We'll talk later
410 about how to write XSUBs that can call every routine in a library.
411 · The value of the function is not being passed back as the
413 function's return value, but by changing the value of the variable
414 that was passed into the function. You might have guessed that
415 when you saw that the return value of round is of type "void".
416 Input and Output Parameters
418 You specify the parameters that will be passed into the XSUB on the
419 line(s) after you declare the function's return value and name. Each
420 input parameter line starts with optional whitespace, and may have an
421 optional terminating semicolon.
422 The list of output parameters occurs at the very end of the function,
424 just after the OUTPUT: directive. The use of RETVAL tells Perl that
425 you wish to send this value back as the return value of the XSUB
426 function. In Example 3, we wanted the "return value" placed in the
427 original variable which we passed in, so we listed it (and not RETVAL)
428 in the OUTPUT: section.
429 The XSUBPP Program
431 The xsubpp program takes the XS code in the .xs file and translates it
432 into C code, placing it in a file whose suffix is .c. The C code
433 created makes heavy use of the C functions within Perl.
434 The TYPEMAP file
436 The xsubpp program uses rules to convert from Perl's data types
437 (scalar, array, etc.) to C's data types (int, char, etc.). These rules
438 are stored in the typemap file ($PERLLIB/ExtUtils/typemap). This file
439 is split into three parts.
440 The first section maps various C data types to a name, which
442 corresponds somewhat with the various Perl types. The second section
443 contains C code which xsubpp uses to handle input parameters. The
444 third section contains C code which xsubpp uses to handle output
445 parameters.
446 Let's take a look at a portion of the .c file created for our
448 extension. The file name is Mytest.c:
449 XS(XS_Mytest_round)
451 {
452 dXSARGS;
453 if (items != 1)
454 Perl_croak(aTHX_ "Usage: Mytest::round(arg)");
455 PERL_UNUSED_VAR(cv); /* -W */
456 {
457 double arg = (double)SvNV(ST(0)); /* XXXXX */
458 if (arg > 0.0) {
459 arg = floor(arg + 0.5);
460 } else if (arg < 0.0) {
461 arg = ceil(arg - 0.5);
462 } else {
463 arg = 0.0;
464 }
465 sv_setnv(ST(0), (double)arg); /* XXXXX */
466 SvSETMAGIC(ST(0));
467 }
468 XSRETURN_EMPTY;
469 }
470 Notice the two lines commented with "XXXXX". If you check the first
472 section of the typemap file, you'll see that doubles are of type
473 T_DOUBLE. In the INPUT section, an argument that is T_DOUBLE is
474 assigned to the variable arg by calling the routine SvNV on something,
475 then casting it to double, then assigned to the variable arg.
476 Similarly, in the OUTPUT section, once arg has its final value, it is
477 passed to the sv_setnv function to be passed back to the calling
478 subroutine. These two functions are explained in perlguts; we'll talk
479 more later about what that "ST(0)" means in the section on the argument
480 stack.
481 Warning about Output Arguments
483 In general, it's not a good idea to write extensions that modify their
484 input parameters, as in Example 3. Instead, you should probably return
485 multiple values in an array and let the caller handle them (we'll do
486 this in a later example). However, in order to better accommodate
487 calling pre-existing C routines, which often do modify their input
488 parameters, this behavior is tolerated.
489 EXAMPLE 4
491 In this example, we'll now begin to write XSUBs that will interact with
492 pre-defined C libraries. To begin with, we will build a small library
493 of our own, then let h2xs write our .pm and .xs files for us.
494 Create a new directory called Mytest2 at the same level as the
496 directory Mytest. In the Mytest2 directory, create another directory
497 called mylib, and cd into that directory.
498 Here we'll create some files that will generate a test library. These
500 will include a C source file and a header file. We'll also create a
501 Makefile.PL in this directory. Then we'll make sure that running make
502 at the Mytest2 level will automatically run this Makefile.PL file and
503 the resulting Makefile.
504 In the mylib directory, create a file mylib.h that looks like this:
506 #define TESTVAL 4
508 extern double foo(int, long, const char*);
510 Also create a file mylib.c that looks like this:
512 #include <stdlib.h>
514 #include "./mylib.h"
515 double
517 foo(int a, long b, const char *c)
518 {
519 return (a + b + atof(c) + TESTVAL);
520 }
521 And finally create a file Makefile.PL that looks like this:
523 use ExtUtils::MakeMaker;
525 $Verbose = 1;
526 WriteMakefile(
527 NAME => 'Mytest2::mylib',
528 SKIP => [qw(all static static_lib dynamic dynamic_lib)],
529 clean => {'FILES' => 'libmylib$(LIB_EXT)'},
530 );
531 sub MY::top_targets {
534 '
535 all :: static
536 pure_all :: static
538 static :: libmylib$(LIB_EXT)
540 libmylib$(LIB_EXT): $(O_FILES)
542 $(AR) cr libmylib$(LIB_EXT) $(O_FILES)
543 $(RANLIB) libmylib$(LIB_EXT)
544 ';
546 }
547 Make sure you use a tab and not spaces on the lines beginning with
549 "$(AR)" and "$(RANLIB)". Make will not function properly if you use
550 spaces. It has also been reported that the "cr" argument to $(AR) is
551 unnecessary on Win32 systems.
552 We will now create the main top-level Mytest2 files. Change to the
554 directory above Mytest2 and run the following command:
555 % h2xs -O -n Mytest2 ./Mytest2/mylib/mylib.h
557 This will print out a warning about overwriting Mytest2, but that's
559 okay. Our files are stored in Mytest2/mylib, and will be untouched.
560 The normal Makefile.PL that h2xs generates doesn't know about the mylib
562 directory. We need to tell it that there is a subdirectory and that we
563 will be generating a library in it. Let's add the argument MYEXTLIB to
564 the WriteMakefile call so that it looks like this:
565 WriteMakefile(
567 'NAME' => 'Mytest2',
568 'VERSION_FROM' => 'Mytest2.pm', # finds $VERSION
569 'LIBS' => [''], # e.g., '-lm'
570 'DEFINE' => '', # e.g., '-DHAVE_SOMETHING'
571 'INC' => '', # e.g., '-I/usr/include/other'
572 'MYEXTLIB' => 'mylib/libmylib$(LIB_EXT)',
573 );
574 and then at the end add a subroutine (which will override the pre-
576 existing subroutine). Remember to use a tab character to indent the
577 line beginning with "cd"!
578 sub MY::postamble {
580 '
581 $(MYEXTLIB): mylib/Makefile
582 cd mylib && $(MAKE) $(PASSTHRU)
583 ';
584 }
585 Let's also fix the MANIFEST file so that it accurately reflects the
587 contents of our extension. The single line that says "mylib" should be
588 replaced by the following three lines:
589 mylib/Makefile.PL
591 mylib/mylib.c
592 mylib/mylib.h
593 To keep our namespace nice and unpolluted, edit the .pm file and change
595 the variable @EXPORT to @EXPORT_OK. Finally, in the .xs file, edit the
596 #include line to read:
597 #include "mylib/mylib.h"
599 And also add the following function definition to the end of the .xs
601 file:
602 double
604 foo(a,b,c)
605 int a
606 long b
607 const char * c
608 OUTPUT:
609 RETVAL
610 Now we also need to create a typemap file because the default Perl
612 doesn't currently support the const char * type. Create a file called
613 typemap in the Mytest2 directory and place the following in it:
614 const char * T_PV
616 Now run perl on the top-level Makefile.PL. Notice that it also created
618 a Makefile in the mylib directory. Run make and watch that it does cd
619 into the mylib directory and run make in there as well.
620 Now edit the Mytest2.t script and change the number of tests to "4",
622 and add the following lines to the end of the script:
623 is( &Mytest2::foo(1, 2, "Hello, world!"), 7 );
625 is( &Mytest2::foo(1, 2, "0.0"), 7 );
626 ok( abs(&Mytest2::foo(0, 0, "-3.4") - 0.6) <= 0.01 );
627 (When dealing with floating-point comparisons, it is best to not check
629 for equality, but rather that the difference between the expected and
630 actual result is below a certain amount (called epsilon) which is 0.01
631 in this case)
632 Run ""make test"" and all should be well. There are some warnings on
634 missing tests for the Mytest2::mylib extension, but you can ignore
635 them.
636 What has happened here?
638 Unlike previous examples, we've now run h2xs on a real include file.
639 This has caused some extra goodies to appear in both the .pm and .xs
640 files.
641 · In the .xs file, there's now a #include directive with the absolute
643 path to the mylib.h header file. We changed this to a relative
644 path so that we could move the extension directory if we wanted to.
645 · There's now some new C code that's been added to the .xs file. The
647 purpose of the "constant" routine is to make the values that are
648 #define'd in the header file accessible by the Perl script (by
649 calling either "TESTVAL" or &Mytest2::TESTVAL). There's also some
650 XS code to allow calls to the "constant" routine.
651 · The .pm file originally exported the name "TESTVAL" in the @EXPORT
653 array. This could lead to name clashes. A good rule of thumb is
654 that if the #define is only going to be used by the C routines
655 themselves, and not by the user, they should be removed from the
656 @EXPORT array. Alternately, if you don't mind using the "fully
657 qualified name" of a variable, you could move most or all of the
658 items from the @EXPORT array into the @EXPORT_OK array.
659 · If our include file had contained #include directives, these would
661 not have been processed by h2xs. There is no good solution to this
662 right now.
663 · We've also told Perl about the library that we built in the mylib
665 subdirectory. That required only the addition of the "MYEXTLIB"
666 variable to the WriteMakefile call and the replacement of the
667 postamble subroutine to cd into the subdirectory and run make. The
668 Makefile.PL for the library is a bit more complicated, but not
669 excessively so. Again we replaced the postamble subroutine to
670 insert our own code. This code simply specified that the library
671 to be created here was a static archive library (as opposed to a
672 dynamically loadable library) and provided the commands to build
673 it.
674 Anatomy of .xs file
676 The .xs file of "EXAMPLE 4" contained some new elements. To understand
677 the meaning of these elements, pay attention to the line which reads
678 MODULE = Mytest2 PACKAGE = Mytest2
680 Anything before this line is plain C code which describes which headers
682 to include, and defines some convenience functions. No translations
683 are performed on this part, apart from having embedded POD
684 documentation skipped over (see perlpod) it goes into the generated
685 output C file as is.
686 Anything after this line is the description of XSUB functions. These
688 descriptions are translated by xsubpp into C code which implements
689 these functions using Perl calling conventions, and which makes these
690 functions visible from Perl interpreter.
691 Pay a special attention to the function "constant". This name appears
693 twice in the generated .xs file: once in the first part, as a static C
694 function, then another time in the second part, when an XSUB interface
695 to this static C function is defined.
696 This is quite typical for .xs files: usually the .xs file provides an
698 interface to an existing C function. Then this C function is defined
699 somewhere (either in an external library, or in the first part of .xs
700 file), and a Perl interface to this function (i.e. "Perl glue") is
701 described in the second part of .xs file. The situation in "EXAMPLE
702 1", "EXAMPLE 2", and "EXAMPLE 3", when all the work is done inside the
703 "Perl glue", is somewhat of an exception rather than the rule.
704 Getting the fat out of XSUBs
706 In "EXAMPLE 4" the second part of .xs file contained the following
707 description of an XSUB:
708 double
710 foo(a,b,c)
711 int a
712 long b
713 const char * c
714 OUTPUT:
715 RETVAL
716 Note that in contrast with "EXAMPLE 1", "EXAMPLE 2" and "EXAMPLE 3",
718 this description does not contain the actual code for what is done is
719 done during a call to Perl function foo(). To understand what is going
720 on here, one can add a CODE section to this XSUB:
721 double
723 foo(a,b,c)
724 int a
725 long b
726 const char * c
727 CODE:
728 RETVAL = foo(a,b,c);
729 OUTPUT:
730 RETVAL
731 However, these two XSUBs provide almost identical generated C code:
733 xsubpp compiler is smart enough to figure out the "CODE:" section from
734 the first two lines of the description of XSUB. What about "OUTPUT:"
735 section? In fact, that is absolutely the same! The "OUTPUT:" section
736 can be removed as well, as far as "CODE:" section or "PPCODE:" section
737 is not specified: xsubpp can see that it needs to generate a function
738 call section, and will autogenerate the OUTPUT section too. Thus one
739 can shortcut the XSUB to become:
740 double
742 foo(a,b,c)
743 int a
744 long b
745 const char * c
746 Can we do the same with an XSUB
748 int
750 is_even(input)
751 int input
752 CODE:
753 RETVAL = (input % 2 == 0);
754 OUTPUT:
755 RETVAL
756 of "EXAMPLE 2"? To do this, one needs to define a C function "int
758 is_even(int input)". As we saw in "Anatomy of .xs file", a proper
759 place for this definition is in the first part of .xs file. In fact a
760 C function
761 int
763 is_even(int arg)
764 {
765 return (arg % 2 == 0);
766 }
767 is probably overkill for this. Something as simple as a "#define" will
769 do too:
770 #define is_even(arg) ((arg) % 2 == 0)
772 After having this in the first part of .xs file, the "Perl glue" part
774 becomes as simple as
775 int
777 is_even(input)
778 int input
779 This technique of separation of the glue part from the workhorse part
781 has obvious tradeoffs: if you want to change a Perl interface, you need
782 to change two places in your code. However, it removes a lot of
783 clutter, and makes the workhorse part independent from idiosyncrasies
784 of Perl calling convention. (In fact, there is nothing Perl-specific
785 in the above description, a different version of xsubpp might have
786 translated this to TCL glue or Python glue as well.)
787 More about XSUB arguments
789 With the completion of Example 4, we now have an easy way to simulate
790 some real-life libraries whose interfaces may not be the cleanest in
791 the world. We shall now continue with a discussion of the arguments
792 passed to the xsubpp compiler.
793 When you specify arguments to routines in the .xs file, you are really
795 passing three pieces of information for each argument listed. The
796 first piece is the order of that argument relative to the others
797 (first, second, etc). The second is the type of argument, and consists
798 of the type declaration of the argument (e.g., int, char*, etc). The
799 third piece is the calling convention for the argument in the call to
800 the library function.
801 While Perl passes arguments to functions by reference, C passes
803 arguments by value; to implement a C function which modifies data of
804 one of the "arguments", the actual argument of this C function would be
805 a pointer to the data. Thus two C functions with declarations
806 int string_length(char *s);
808 int upper_case_char(char *cp);
809 may have completely different semantics: the first one may inspect an
811 array of chars pointed by s, and the second one may immediately
812 dereference "cp" and manipulate *cp only (using the return value as,
813 say, a success indicator). From Perl one would use these functions in
814 a completely different manner.
815 One conveys this info to xsubpp by replacing "*" before the argument by
817 "&". "&" means that the argument should be passed to a library
818 function by its address. The above two function may be XSUB-ified as
819 int
821 string_length(s)
822 char * s
823 int
825 upper_case_char(cp)
826 char &cp
827 For example, consider:
829 int
831 foo(a,b)
832 char &a
833 char * b
834 The first Perl argument to this function would be treated as a char and
836 assigned to the variable a, and its address would be passed into the
837 function foo. The second Perl argument would be treated as a string
838 pointer and assigned to the variable b. The value of b would be passed
839 into the function foo. The actual call to the function foo that xsubpp
840 generates would look like this:
841 foo(&a, b);
843 xsubpp will parse the following function argument lists identically:
845 char &a
847 char&a
848 char & a
849 However, to help ease understanding, it is suggested that you place a
851 "&" next to the variable name and away from the variable type), and
852 place a "*" near the variable type, but away from the variable name (as
853 in the call to foo above). By doing so, it is easy to understand
854 exactly what will be passed to the C function -- it will be whatever is
855 in the "last column".
856 You should take great pains to try to pass the function the type of
858 variable it wants, when possible. It will save you a lot of trouble in
859 the long run.
860 The Argument Stack
862 If we look at any of the C code generated by any of the examples except
863 example 1, you will notice a number of references to ST(n), where n is
864 usually 0. "ST" is actually a macro that points to the n'th argument
865 on the argument stack. ST(0) is thus the first argument on the stack
866 and therefore the first argument passed to the XSUB, ST(1) is the
867 second argument, and so on.
868 When you list the arguments to the XSUB in the .xs file, that tells
870 xsubpp which argument corresponds to which of the argument stack (i.e.,
871 the first one listed is the first argument, and so on). You invite
872 disaster if you do not list them in the same order as the function
873 expects them.
874 The actual values on the argument stack are pointers to the values
876 passed in. When an argument is listed as being an OUTPUT value, its
877 corresponding value on the stack (i.e., ST(0) if it was the first
878 argument) is changed. You can verify this by looking at the C code
879 generated for Example 3. The code for the round() XSUB routine
880 contains lines that look like this:
881 double arg = (double)SvNV(ST(0));
883 /* Round the contents of the variable arg */
884 sv_setnv(ST(0), (double)arg);
885 The arg variable is initially set by taking the value from ST(0), then
887 is stored back into ST(0) at the end of the routine.
888 XSUBs are also allowed to return lists, not just scalars. This must be
890 done by manipulating stack values ST(0), ST(1), etc, in a subtly
891 different way. See perlxs for details.
892 XSUBs are also allowed to avoid automatic conversion of Perl function
894 arguments to C function arguments. See perlxs for details. Some
895 people prefer manual conversion by inspecting ST(i) even in the cases
896 when automatic conversion will do, arguing that this makes the logic of
897 an XSUB call clearer. Compare with "Getting the fat out of XSUBs" for
898 a similar tradeoff of a complete separation of "Perl glue" and
899 "workhorse" parts of an XSUB.
900 While experts may argue about these idioms, a novice to Perl guts may
902 prefer a way which is as little Perl-guts-specific as possible, meaning
903 automatic conversion and automatic call generation, as in "Getting the
904 fat out of XSUBs". This approach has the additional benefit of
905 protecting the XSUB writer from future changes to the Perl API.
906 Extending your Extension
908 Sometimes you might want to provide some extra methods or subroutines
909 to assist in making the interface between Perl and your extension
910 simpler or easier to understand. These routines should live in the .pm
911 file. Whether they are automatically loaded when the extension itself
912 is loaded or only loaded when called depends on where in the .pm file
913 the subroutine definition is placed. You can also consult AutoLoader
914 for an alternate way to store and load your extra subroutines.
915 Documenting your Extension
917 There is absolutely no excuse for not documenting your extension.
918 Documentation belongs in the .pm file. This file will be fed to
919 pod2man, and the embedded documentation will be converted to the
920 manpage format, then placed in the blib directory. It will be copied
921 to Perl's manpage directory when the extension is installed.
922 You may intersperse documentation and Perl code within the .pm file.
924 In fact, if you want to use method autoloading, you must do this, as
925 the comment inside the .pm file explains.
926 See perlpod for more information about the pod format.
928 Installing your Extension
930 Once your extension is complete and passes all its tests, installing it
931 is quite simple: you simply run "make install". You will either need
932 to have write permission into the directories where Perl is installed,
933 or ask your system administrator to run the make for you.
934 Alternately, you can specify the exact directory to place the
936 extension's files by placing a "PREFIX=/destination/directory" after
937 the make install. (or in between the make and install if you have a
938 brain-dead version of make). This can be very useful if you are
939 building an extension that will eventually be distributed to multiple
940 systems. You can then just archive the files in the destination
941 directory and distribute them to your destination systems.
942 EXAMPLE 5
944 In this example, we'll do some more work with the argument stack. The
945 previous examples have all returned only a single value. We'll now
946 create an extension that returns an array.
947 This extension is very Unix-oriented (struct statfs and the statfs
949 system call). If you are not running on a Unix system, you can
950 substitute for statfs any other function that returns multiple values,
951 you can hard-code values to be returned to the caller (although this
952 will be a bit harder to test the error case), or you can simply not do
953 this example. If you change the XSUB, be sure to fix the test cases to
954 match the changes.
955 Return to the Mytest directory and add the following code to the end of
957 Mytest.xs:
958 void
960 statfs(path)
961 char * path
962 INIT:
963 int i;
964 struct statfs buf;
965 PPCODE:
967 i = statfs(path, &buf);
968 if (i == 0) {
969 XPUSHs(sv_2mortal(newSVnv(buf.f_bavail)));
970 XPUSHs(sv_2mortal(newSVnv(buf.f_bfree)));
971 XPUSHs(sv_2mortal(newSVnv(buf.f_blocks)));
972 XPUSHs(sv_2mortal(newSVnv(buf.f_bsize)));
973 XPUSHs(sv_2mortal(newSVnv(buf.f_ffree)));
974 XPUSHs(sv_2mortal(newSVnv(buf.f_files)));
975 XPUSHs(sv_2mortal(newSVnv(buf.f_type)));
976 } else {
977 XPUSHs(sv_2mortal(newSVnv(errno)));
978 }
979 You'll also need to add the following code to the top of the .xs file,
981 just after the include of "XSUB.h":
982 #include <sys/vfs.h>
984 Also add the following code segment to Mytest.t while incrementing the
986 "9" tests to "11":
987 @a = &Mytest::statfs("/blech");
989 ok( scalar(@a) == 1 && $a[0] == 2 );
990 @a = &Mytest::statfs("/");
991 is( scalar(@a), 7 );
992 New Things in this Example
994 This example added quite a few new concepts. We'll take them one at a
995 time.
996 · The INIT: directive contains code that will be placed immediately
998 after the argument stack is decoded. C does not allow variable
999 declarations at arbitrary locations inside a function, so this is
1000 usually the best way to declare local variables needed by the XSUB.
1001 (Alternatively, one could put the whole "PPCODE:" section into
1002 braces, and put these declarations on top.)
1003 · This routine also returns a different number of arguments depending
1005 on the success or failure of the call to statfs. If there is an
1006 error, the error number is returned as a single-element array. If
1007 the call is successful, then a 9-element array is returned. Since
1008 only one argument is passed into this function, we need room on the
1009 stack to hold the 9 values which may be returned.
1010 We do this by using the PPCODE: directive, rather than the CODE:
1012 directive. This tells xsubpp that we will be managing the return
1013 values that will be put on the argument stack by ourselves.
1014 · When we want to place values to be returned to the caller onto the
1016 stack, we use the series of macros that begin with "XPUSH". There
1017 are five different versions, for placing integers, unsigned
1018 integers, doubles, strings, and Perl scalars on the stack. In our
1019 example, we placed a Perl scalar onto the stack. (In fact this is
1020 the only macro which can be used to return multiple values.)
1021 The XPUSH* macros will automatically extend the return stack to
1023 prevent it from being overrun. You push values onto the stack in
1024 the order you want them seen by the calling program.
1025 · The values pushed onto the return stack of the XSUB are actually
1027 mortal SV's. They are made mortal so that once the values are
1028 copied by the calling program, the SV's that held the returned
1029 values can be deallocated. If they were not mortal, then they
1030 would continue to exist after the XSUB routine returned, but would
1031 not be accessible. This is a memory leak.
1032 · If we were interested in performance, not in code compactness, in
1034 the success branch we would not use "XPUSHs" macros, but "PUSHs"
1035 macros, and would pre-extend the stack before pushing the return
1036 values:
1037 EXTEND(SP, 7);
1039 The tradeoff is that one needs to calculate the number of return
1041 values in advance (though overextending the stack will not
1042 typically hurt anything but memory consumption).
1043 Similarly, in the failure branch we could use "PUSHs" without
1045 extending the stack: the Perl function reference comes to an XSUB
1046 on the stack, thus the stack is always large enough to take one
1047 return value.
1048 EXAMPLE 6
1050 In this example, we will accept a reference to an array as an input
1051 parameter, and return a reference to an array of hashes. This will
1052 demonstrate manipulation of complex Perl data types from an XSUB.
1053 This extension is somewhat contrived. It is based on the code in the
1055 previous example. It calls the statfs function multiple times,
1056 accepting a reference to an array of filenames as input, and returning
1057 a reference to an array of hashes containing the data for each of the
1058 filesystems.
1059 Return to the Mytest directory and add the following code to the end of
1061 Mytest.xs:
1062 SV *
1064 multi_statfs(paths)
1065 SV * paths
1066 INIT:
1067 AV * results;
1068 I32 numpaths = 0;
1069 int i, n;
1070 struct statfs buf;
1071 if ((!SvROK(paths))
1073 || (SvTYPE(SvRV(paths)) != SVt_PVAV)
1074 || ((numpaths = av_len((AV *)SvRV(paths))) < 0))
1075 {
1076 XSRETURN_UNDEF;
1077 }
1078 results = (AV *)sv_2mortal((SV *)newAV());
1079 CODE:
1080 for (n = 0; n <= numpaths; n++) {
1081 HV * rh;
1082 STRLEN l;
1083 char * fn = SvPV(*av_fetch((AV *)SvRV(paths), n, 0), l);
1084 i = statfs(fn, &buf);
1086 if (i != 0) {
1087 av_push(results, newSVnv(errno));
1088 continue;
1089 }
1090 rh = (HV *)sv_2mortal((SV *)newHV());
1092 hv_store(rh, "f_bavail", 8, newSVnv(buf.f_bavail), 0);
1094 hv_store(rh, "f_bfree", 7, newSVnv(buf.f_bfree), 0);
1095 hv_store(rh, "f_blocks", 8, newSVnv(buf.f_blocks), 0);
1096 hv_store(rh, "f_bsize", 7, newSVnv(buf.f_bsize), 0);
1097 hv_store(rh, "f_ffree", 7, newSVnv(buf.f_ffree), 0);
1098 hv_store(rh, "f_files", 7, newSVnv(buf.f_files), 0);
1099 hv_store(rh, "f_type", 6, newSVnv(buf.f_type), 0);
1100 av_push(results, newRV((SV *)rh));
1102 }
1103 RETVAL = newRV((SV *)results);
1104 OUTPUT:
1105 RETVAL
1106 And add the following code to Mytest.t, while incrementing the "11"
1108 tests to "13":
1109 $results = Mytest::multi_statfs([ '/', '/blech' ]);
1111 ok( ref $results->[0]) );
1112 ok( ! ref $results->[1] );
1113 New Things in this Example
1115 There are a number of new concepts introduced here, described below:
1116 · This function does not use a typemap. Instead, we declare it as
1118 accepting one SV* (scalar) parameter, and returning an SV* value,
1119 and we take care of populating these scalars within the code.
1120 Because we are only returning one value, we don't need a "PPCODE:"
1121 directive - instead, we use "CODE:" and "OUTPUT:" directives.
1122 · When dealing with references, it is important to handle them with
1124 caution. The "INIT:" block first checks that "SvROK" returns true,
1125 which indicates that paths is a valid reference. It then verifies
1126 that the object referenced by paths is an array, using "SvRV" to
1127 dereference paths, and "SvTYPE" to discover its type. As an added
1128 test, it checks that the array referenced by paths is non-empty,
1129 using the "av_len" function (which returns -1 if the array is
1130 empty). The XSRETURN_UNDEF macro is used to abort the XSUB and
1131 return the undefined value whenever all three of these conditions
1132 are not met.
1133 · We manipulate several arrays in this XSUB. Note that an array is
1135 represented internally by an AV* pointer. The functions and macros
1136 for manipulating arrays are similar to the functions in Perl:
1137 "av_len" returns the highest index in an AV*, much like $#array;
1138 "av_fetch" fetches a single scalar value from an array, given its
1139 index; "av_push" pushes a scalar value onto the end of the array,
1140 automatically extending the array as necessary.
1141 Specifically, we read pathnames one at a time from the input array,
1143 and store the results in an output array (results) in the same
1144 order. If statfs fails, the element pushed onto the return array
1145 is the value of errno after the failure. If statfs succeeds,
1146 though, the value pushed onto the return array is a reference to a
1147 hash containing some of the information in the statfs structure.
1148 As with the return stack, it would be possible (and a small
1150 performance win) to pre-extend the return array before pushing data
1151 into it, since we know how many elements we will return:
1152 av_extend(results, numpaths);
1154 · We are performing only one hash operation in this function, which
1156 is storing a new scalar under a key using "hv_store". A hash is
1157 represented by an HV* pointer. Like arrays, the functions for
1158 manipulating hashes from an XSUB mirror the functionality available
1159 from Perl. See perlguts and perlapi for details.
1160 · To create a reference, we use the "newRV" function. Note that you
1162 can cast an AV* or an HV* to type SV* in this case (and many
1163 others). This allows you to take references to arrays, hashes and
1164 scalars with the same function. Conversely, the "SvRV" function
1165 always returns an SV*, which may need to be cast to the appropriate
1166 type if it is something other than a scalar (check with "SvTYPE").
1167 · At this point, xsubpp is doing very little work - the differences
1169 between Mytest.xs and Mytest.c are minimal.
1170 EXAMPLE 7 (Coming Soon)
1172 XPUSH args AND set RETVAL AND assign return value to array
1173 EXAMPLE 8 (Coming Soon)
1175 Setting $!
1176 EXAMPLE 9 Passing open files to XSes
1178 You would think passing files to an XS is difficult, with all the
1179 typeglobs and stuff. Well, it isn't.
1180 Suppose that for some strange reason we need a wrapper around the
1182 standard C library function "fputs()". This is all we need:
1183 #define PERLIO_NOT_STDIO 0
1185 #include "EXTERN.h"
1186 #include "perl.h"
1187 #include "XSUB.h"
1188 #include <stdio.h>
1190 int
1192 fputs(s, stream)
1193 char * s
1194 FILE * stream
1195 The real work is done in the standard typemap.
1197 But you loose all the fine stuff done by the perlio layers. This calls
1199 the stdio function "fputs()", which knows nothing about them.
1200 The standard typemap offers three variants of PerlIO *: "InputStream"
1202 (T_IN), "InOutStream" (T_INOUT) and "OutputStream" (T_OUT). A bare
1203 "PerlIO *" is considered a T_INOUT. If it matters in your code (see
1204 below for why it might) #define or typedef one of the specific names
1205 and use that as the argument or result type in your XS file.
1206 The standard typemap does not contain PerlIO * before perl 5.7, but it
1208 has the three stream variants. Using a PerlIO * directly is not
1209 backwards compatible unless you provide your own typemap.
1210 For streams coming from perl the main difference is that "OutputStream"
1212 will get the output PerlIO * - which may make a difference on a socket.
1213 Like in our example...
1214 For streams being handed to perl a new file handle is created (i.e. a
1216 reference to a new glob) and associated with the PerlIO * provided. If
1217 the read/write state of the PerlIO * is not correct then you may get
1218 errors or warnings from when the file handle is used. So if you opened
1219 the PerlIO * as "w" it should really be an "OutputStream" if open as
1220 "r" it should be an "InputStream".
1221 Now, suppose you want to use perlio layers in your XS. We'll use the
1223 perlio "PerlIO_puts()" function as an example.
1224 In the C part of the XS file (above the first MODULE line) you have
1226 #define OutputStream PerlIO *
1228 or
1229 typedef PerlIO * OutputStream;
1230 And this is the XS code:
1232 int
1234 perlioputs(s, stream)
1235 char * s
1236 OutputStream stream
1237 CODE:
1238 RETVAL = PerlIO_puts(stream, s);
1239 OUTPUT:
1240 RETVAL
1241 We have to use a "CODE" section because "PerlIO_puts()" has the
1243 arguments reversed compared to "fputs()", and we want to keep the
1244 arguments the same.
1245 Wanting to explore this thoroughly, we want to use the stdio "fputs()"
1247 on a PerlIO *. This means we have to ask the perlio system for a stdio
1248 "FILE *":
1249 int
1251 perliofputs(s, stream)
1252 char * s
1253 OutputStream stream
1254 PREINIT:
1255 FILE *fp = PerlIO_findFILE(stream);
1256 CODE:
1257 if (fp != (FILE*) 0) {
1258 RETVAL = fputs(s, fp);
1259 } else {
1260 RETVAL = -1;
1261 }
1262 OUTPUT:
1263 RETVAL
1264 Note: "PerlIO_findFILE()" will search the layers for a stdio layer. If
1266 it can't find one, it will call "PerlIO_exportFILE()" to generate a new
1267 stdio "FILE". Please only call "PerlIO_exportFILE()" if you want a new
1268 "FILE". It will generate one on each call and push a new stdio layer.
1269 So don't call it repeatedly on the same file. "PerlIO()"_findFILE will
1270 retrieve the stdio layer once it has been generated by
1271 "PerlIO_exportFILE()".
1272 This applies to the perlio system only. For versions before 5.7,
1274 "PerlIO_exportFILE()" is equivalent to "PerlIO_findFILE()".
1275 Troubleshooting these Examples
1277 As mentioned at the top of this document, if you are having problems
1278 with these example extensions, you might see if any of these help you.
1279 · In versions of 5.002 prior to the gamma version, the test script in
1281 Example 1 will not function properly. You need to change the "use
1282 lib" line to read:
1283 use lib './blib';
1285 · In versions of 5.002 prior to version 5.002b1h, the test.pl file
1287 was not automatically created by h2xs. This means that you cannot
1288 say "make test" to run the test script. You will need to add the
1289 following line before the "use extension" statement:
1290 use lib './blib';
1292 · In versions 5.000 and 5.001, instead of using the above line, you
1294 will need to use the following line:
1295 BEGIN { unshift(@INC, "./blib") }
1297 · This document assumes that the executable named "perl" is Perl
1299 version 5. Some systems may have installed Perl version 5 as
1300 "perl5".
1301
1303 For more information, consult perlguts, perlapi, perlxs, perlmod, and
1304 perlpod.
1305
1307 Jeff Okamoto <okamoto@corp.hp.com>
1308 Reviewed and assisted by Dean Roehrich, Ilya Zakharevich, Andreas
1310 Koenig, and Tim Bunce.
1311 PerlIO material contributed by Lupe Christoph, with some clarification
1313 by Nick Ing-Simmons.
1314 Changes for h2xs as of Perl 5.8.x by Renee Baecker
1316 Last Changed
1318 2007/10/11
1319perl v5.10.1 2009-04-22 PERLXSTUT(1)