1perlxstut(3) User Contributed Perl Documentation perlxstut(3)
2
6 perlxstut - Tutorial for writing XSUBs
7
9 This tutorial will educate the reader on the steps involved in creating
10 a Perl extension. The reader is assumed to have access to perlguts,
11 perlapi and perlxs.
12 This tutorial starts with very simple examples and becomes more
14 complex, with each new example adding new features. Certain concepts
15 may not be completely explained until later in the tutorial in order to
16 slowly ease the reader into building extensions.
17 This tutorial was written from a Unix point of view. Where I know them
19 to be otherwise different for other platforms (e.g. Win32), I will list
20 them. If you find something that was missed, please let me know.
21
23 make
24 This tutorial assumes that the make program that Perl is configured to
25 use is called "make". Instead of running "make" in the examples that
26 follow, you may have to substitute whatever make program Perl has been
27 configured to use. Running perl -V:make should tell you what it is.
28 Version caveat
30 When writing a Perl extension for general consumption, one should
31 expect that the extension will be used with versions of Perl different
32 from the version available on your machine. Since you are reading this
33 document, the version of Perl on your machine is probably 5.005 or
34 later, but the users of your extension may have more ancient versions.
35 To understand what kinds of incompatibilities one may expect, and in
37 the rare case that the version of Perl on your machine is older than
38 this document, see the section on "Troubleshooting these Examples" for
39 more information.
40 If your extension uses some features of Perl which are not available on
42 older releases of Perl, your users would appreciate an early meaningful
43 warning. You would probably put this information into the README file,
44 but nowadays installation of extensions may be performed automatically,
45 guided by CPAN.pm module or other tools.
46 In MakeMaker-based installations, Makefile.PL provides the earliest
48 opportunity to perform version checks. One can put something like this
49 in Makefile.PL for this purpose:
50 eval { require 5.007 }
52 or die <<EOD;
53 ############
54 ### This module uses frobnication framework which is not available
55 ### before version 5.007 of Perl. Upgrade your Perl before
56 ### installing Kara::Mba.
57 ############
58 EOD
59 Dynamic Loading versus Static Loading
61 It is commonly thought that if a system does not have the capability to
62 dynamically load a library, you cannot build XSUBs. This is incorrect.
63 You can build them, but you must link the XSUBs subroutines with the
64 rest of Perl, creating a new executable. This situation is similar to
65 Perl 4.
66 This tutorial can still be used on such a system. The XSUB build
68 mechanism will check the system and build a dynamically-loadable
69 library if possible, or else a static library and then, optionally, a
70 new statically-linked executable with that static library linked in.
71 Should you wish to build a statically-linked executable on a system
73 which can dynamically load libraries, you may, in all the following
74 examples, where the command ""make"" with no arguments is executed, run
75 the command ""make perl"" instead.
76 If you have generated such a statically-linked executable by choice,
78 then instead of saying ""make test"", you should say ""make
79 test_static"". On systems that cannot build dynamically-loadable
80 libraries at all, simply saying ""make test"" is sufficient.
81 Threads and PERL_NO_GET_CONTEXT
83 For threaded builds, perl requires the context pointer for the current
84 thread, without "PERL_NO_GET_CONTEXT", perl will call a function to
85 retrieve the context.
86 For improved performance, include:
88 #define PERL_NO_GET_CONTEXT
90 as shown below.
92 For more details, see perlguts.
94
96 Now let's go on with the show!
97 EXAMPLE 1
99 Our first extension will be very simple. When we call the routine in
100 the extension, it will print out a well-known message and return.
101 Run ""h2xs -A -n Mytest"". This creates a directory named Mytest,
103 possibly under ext/ if that directory exists in the current working
104 directory. Several files will be created under the Mytest dir,
105 including MANIFEST, Makefile.PL, lib/Mytest.pm, Mytest.xs, t/Mytest.t,
106 and Changes.
107 The MANIFEST file contains the names of all the files just created in
109 the Mytest directory.
110 The file Makefile.PL should look something like this:
112 use ExtUtils::MakeMaker;
114 # See lib/ExtUtils/MakeMaker.pm for details of how to influence
116 # the contents of the Makefile that is written.
117 WriteMakefile(
118 NAME => 'Mytest',
119 VERSION_FROM => 'Mytest.pm', # finds $VERSION
120 LIBS => [''], # e.g., '-lm'
121 DEFINE => '', # e.g., '-DHAVE_SOMETHING'
122 INC => '-I', # e.g., '-I. -I/usr/include/other'
123 );
124 The file Mytest.pm should start with something like this:
126 package Mytest;
128 use 5.008008;
130 use strict;
131 use warnings;
132 require Exporter;
134 our @ISA = qw(Exporter);
136 our %EXPORT_TAGS = ( 'all' => [ qw(
137 ) ] );
139 our @EXPORT_OK = ( @{ $EXPORT_TAGS{'all'} } );
141 our @EXPORT = qw(
143 );
145 our $VERSION = '0.01';
147 require XSLoader;
149 XSLoader::load('Mytest', $VERSION);
150 # Preloaded methods go here.
152 1;
154 __END__
155 # Below is the stub of documentation for your module. You better
156 # edit it!
157 The rest of the .pm file contains sample code for providing
159 documentation for the extension.
160 Finally, the Mytest.xs file should look something like this:
162 #define PERL_NO_GET_CONTEXT
164 #include "EXTERN.h"
165 #include "perl.h"
166 #include "XSUB.h"
167 #include "ppport.h"
169 MODULE = Mytest PACKAGE = Mytest
171 Let's edit the .xs file by adding this to the end of the file:
173 void
175 hello()
176 CODE:
177 printf("Hello, world!\n");
178 It is okay for the lines starting at the "CODE:" line to not be
180 indented. However, for readability purposes, it is suggested that you
181 indent CODE: one level and the lines following one more level.
182 Now we'll run ""perl Makefile.PL"". This will create a real Makefile,
184 which make needs. Its output looks something like:
185 % perl Makefile.PL
187 Checking if your kit is complete...
188 Looks good
189 Writing Makefile for Mytest
190 %
191 Now, running make will produce output that looks something like this
193 (some long lines have been shortened for clarity and some extraneous
194 lines have been deleted):
195 % make
197 cp lib/Mytest.pm blib/lib/Mytest.pm
198 perl xsubpp -typemap typemap Mytest.xs > Mytest.xsc && \
199 mv Mytest.xsc Mytest.c
200 Please specify prototyping behavior for Mytest.xs (see perlxs manual)
201 cc -c Mytest.c
202 Running Mkbootstrap for Mytest ()
203 chmod 644 Mytest.bs
204 rm -f blib/arch/auto/Mytest/Mytest.so
205 cc -shared -L/usr/local/lib Mytest.o -o blib/arch/auto/Mytest/Mytest.so
206 chmod 755 blib/arch/auto/Mytest/Mytest.so
208 cp Mytest.bs blib/arch/auto/Mytest/Mytest.bs
209 chmod 644 blib/arch/auto/Mytest/Mytest.bs
210 Manifying blib/man3/Mytest.3pm
211 %
212 You can safely ignore the line about "prototyping behavior" - it is
214 explained in "The PROTOTYPES: Keyword" in perlxs.
215 Perl has its own special way of easily writing test scripts, but for
217 this example only, we'll create our own test script. Create a file
218 called hello that looks like this:
219 #! /opt/perl5/bin/perl
221 use ExtUtils::testlib;
223 use Mytest;
225 Mytest::hello();
227 Now we make the script executable ("chmod +x hello"), run the script
229 and we should see the following output:
230 % ./hello
232 Hello, world!
233 %
234 EXAMPLE 2
236 Now let's add to our extension a subroutine that will take a single
237 numeric argument as input and return 1 if the number is even or 0 if
238 the number is odd.
239 Add the following to the end of Mytest.xs:
241 int
243 is_even(input)
244 int input
245 CODE:
246 RETVAL = (input % 2 == 0);
247 OUTPUT:
248 RETVAL
249 There does not need to be whitespace at the start of the ""int input""
251 line, but it is useful for improving readability. Placing a semi-colon
252 at the end of that line is also optional. Any amount and kind of
253 whitespace may be placed between the ""int"" and ""input"".
254 Now re-run make to rebuild our new shared library.
256 Now perform the same steps as before, generating a Makefile from the
258 Makefile.PL file, and running make.
259 In order to test that our extension works, we now need to look at the
261 file Mytest.t. This file is set up to imitate the same kind of testing
262 structure that Perl itself has. Within the test script, you perform a
263 number of tests to confirm the behavior of the extension, printing "ok"
264 when the test is correct, "not ok" when it is not.
265 use Test::More tests => 4;
267 BEGIN { use_ok('Mytest') };
268 #########################
270 # Insert your test code below, the Test::More module is use()ed here
272 # so read its man page ( perldoc Test::More ) for help writing this
273 # test script.
274 is( Mytest::is_even(0), 1 );
276 is( Mytest::is_even(1), 0 );
277 is( Mytest::is_even(2), 1 );
278 We will be calling the test script through the command ""make test"".
280 You should see output that looks something like this:
281 %make test
283 PERL_DL_NONLAZY=1 /usr/bin/perl "-MExtUtils::Command::MM" "-e"
284 "test_harness(0, 'blib/lib', 'blib/arch')" t/*.t
285 t/Mytest....ok
286 All tests successful.
287 Files=1, Tests=4, 0 wallclock secs ( 0.03 cusr + 0.00 csys = 0.03 CPU)
288 %
289 What has gone on?
291 The program h2xs is the starting point for creating extensions. In
292 later examples we'll see how we can use h2xs to read header files and
293 generate templates to connect to C routines.
294 h2xs creates a number of files in the extension directory. The file
296 Makefile.PL is a perl script which will generate a true Makefile to
297 build the extension. We'll take a closer look at it later.
298 The .pm and .xs files contain the meat of the extension. The .xs file
300 holds the C routines that make up the extension. The .pm file contains
301 routines that tell Perl how to load your extension.
302 Generating the Makefile and running "make" created a directory called
304 blib (which stands for "build library") in the current working
305 directory. This directory will contain the shared library that we will
306 build. Once we have tested it, we can install it into its final
307 location.
308 Invoking the test script via ""make test"" did something very
310 important. It invoked perl with all those "-I" arguments so that it
311 could find the various files that are part of the extension. It is
312 very important that while you are still testing extensions that you use
313 ""make test"". If you try to run the test script all by itself, you
314 will get a fatal error. Another reason it is important to use ""make
315 test"" to run your test script is that if you are testing an upgrade to
316 an already-existing version, using ""make test"" ensures that you will
317 test your new extension, not the already-existing version.
318 When Perl sees a "use extension;", it searches for a file with the same
320 name as the "use"'d extension that has a .pm suffix. If that file
321 cannot be found, Perl dies with a fatal error. The default search path
322 is contained in the @INC array.
323 In our case, Mytest.pm tells perl that it will need the Exporter and
325 Dynamic Loader extensions. It then sets the @ISA and @EXPORT arrays
326 and the $VERSION scalar; finally it tells perl to bootstrap the module.
327 Perl will call its dynamic loader routine (if there is one) and load
328 the shared library.
329 The two arrays @ISA and @EXPORT are very important. The @ISA array
331 contains a list of other packages in which to search for methods (or
332 subroutines) that do not exist in the current package. This is usually
333 only important for object-oriented extensions (which we will talk about
334 much later), and so usually doesn't need to be modified.
335 The @EXPORT array tells Perl which of the extension's variables and
337 subroutines should be placed into the calling package's namespace.
338 Because you don't know if the user has already used your variable and
339 subroutine names, it's vitally important to carefully select what to
340 export. Do not export method or variable names by default without a
341 good reason.
342 As a general rule, if the module is trying to be object-oriented then
344 don't export anything. If it's just a collection of functions and
345 variables, then you can export them via another array, called
346 @EXPORT_OK. This array does not automatically place its subroutine and
347 variable names into the namespace unless the user specifically requests
348 that this be done.
349 See perlmod for more information.
351 The $VERSION variable is used to ensure that the .pm file and the
353 shared library are "in sync" with each other. Any time you make
354 changes to the .pm or .xs files, you should increment the value of this
355 variable.
356 Writing good test scripts
358 The importance of writing good test scripts cannot be over-emphasized.
359 You should closely follow the "ok/not ok" style that Perl itself uses,
360 so that it is very easy and unambiguous to determine the outcome of
361 each test case. When you find and fix a bug, make sure you add a test
362 case for it.
363 By running ""make test"", you ensure that your Mytest.t script runs and
365 uses the correct version of your extension. If you have many test
366 cases, save your test files in the "t" directory and use the suffix
367 ".t". When you run ""make test"", all of these test files will be
368 executed.
369 EXAMPLE 3
371 Our third extension will take one argument as its input, round off that
372 value, and set the argument to the rounded value.
373 Add the following to the end of Mytest.xs:
375 void
377 round(arg)
378 double arg
379 CODE:
380 if (arg > 0.0) {
381 arg = floor(arg + 0.5);
382 } else if (arg < 0.0) {
383 arg = ceil(arg - 0.5);
384 } else {
385 arg = 0.0;
386 }
387 OUTPUT:
388 arg
389 Edit the Makefile.PL file so that the corresponding line looks like
391 this:
392 LIBS => ['-lm'], # e.g., '-lm'
394 Generate the Makefile and run make. Change the test number in Mytest.t
396 to "9" and add the following tests:
397 my $i;
399 $i = -1.5;
401 Mytest::round($i);
402 is( $i, -2.0, 'Rounding -1.5 to -2.0' );
403 $i = -1.1;
405 Mytest::round($i);
406 is( $i, -1.0, 'Rounding -1.1 to -1.0' );
407 $i = 0.0;
409 Mytest::round($i);
410 is( $i, 0.0, 'Rounding 0.0 to 0.0' );
411 $i = 0.5;
413 Mytest::round($i);
414 is( $i, 1.0, 'Rounding 0.5 to 1.0' );
415 $i = 1.2;
417 Mytest::round($i);
418 is( $i, 1.0, 'Rounding 1.2 to 1.0' );
419 Running ""make test"" should now print out that all nine tests are
421 okay.
422 Notice that in these new test cases, the argument passed to round was a
424 scalar variable. You might be wondering if you can round a constant or
425 literal. To see what happens, temporarily add the following line to
426 Mytest.t:
427 Mytest::round(3);
429 Run ""make test"" and notice that Perl dies with a fatal error. Perl
431 won't let you change the value of constants!
432 What's new here?
434 • We've made some changes to Makefile.PL. In this case, we've
435 specified an extra library to be linked into the extension's shared
436 library, the math library libm in this case. We'll talk later
437 about how to write XSUBs that can call every routine in a library.
438 • The value of the function is not being passed back as the
440 function's return value, but by changing the value of the variable
441 that was passed into the function. You might have guessed that
442 when you saw that the return value of round is of type "void".
443 Input and Output Parameters
445 You specify the parameters that will be passed into the XSUB on the
446 line(s) after you declare the function's return value and name. Each
447 input parameter line starts with optional whitespace, and may have an
448 optional terminating semicolon.
449 The list of output parameters occurs at the very end of the function,
451 just after the OUTPUT: directive. The use of RETVAL tells Perl that
452 you wish to send this value back as the return value of the XSUB
453 function. In Example 3, we wanted the "return value" placed in the
454 original variable which we passed in, so we listed it (and not RETVAL)
455 in the OUTPUT: section.
456 The XSUBPP Program
458 The xsubpp program takes the XS code in the .xs file and translates it
459 into C code, placing it in a file whose suffix is .c. The C code
460 created makes heavy use of the C functions within Perl.
461 The TYPEMAP file
463 The xsubpp program uses rules to convert from Perl's data types
464 (scalar, array, etc.) to C's data types (int, char, etc.). These rules
465 are stored in the typemap file ($PERLLIB/ExtUtils/typemap). There's a
466 brief discussion below, but all the nitty-gritty details can be found
467 in perlxstypemap. If you have a new-enough version of perl (5.16 and
468 up) or an upgraded XS compiler ("ExtUtils::ParseXS" 3.13_01 or better),
469 then you can inline typemaps in your XS instead of writing separate
470 files. Either way, this typemap thing is split into three parts:
471 The first section maps various C data types to a name, which
473 corresponds somewhat with the various Perl types. The second section
474 contains C code which xsubpp uses to handle input parameters. The
475 third section contains C code which xsubpp uses to handle output
476 parameters.
477 Let's take a look at a portion of the .c file created for our
479 extension. The file name is Mytest.c:
480 XS(XS_Mytest_round)
482 {
483 dXSARGS;
484 if (items != 1)
485 Perl_croak(aTHX_ "Usage: Mytest::round(arg)");
486 PERL_UNUSED_VAR(cv); /* -W */
487 {
488 double arg = (double)SvNV(ST(0)); /* XXXXX */
489 if (arg > 0.0) {
490 arg = floor(arg + 0.5);
491 } else if (arg < 0.0) {
492 arg = ceil(arg - 0.5);
493 } else {
494 arg = 0.0;
495 }
496 sv_setnv(ST(0), (double)arg); /* XXXXX */
497 SvSETMAGIC(ST(0));
498 }
499 XSRETURN_EMPTY;
500 }
501 Notice the two lines commented with "XXXXX". If you check the first
503 part of the typemap file (or section), you'll see that doubles are of
504 type T_DOUBLE. In the INPUT part of the typemap, an argument that is
505 T_DOUBLE is assigned to the variable arg by calling the routine SvNV on
506 something, then casting it to double, then assigned to the variable
507 arg. Similarly, in the OUTPUT section, once arg has its final value,
508 it is passed to the sv_setnv function to be passed back to the calling
509 subroutine. These two functions are explained in perlguts; we'll talk
510 more later about what that "ST(0)" means in the section on the argument
511 stack.
512 Warning about Output Arguments
514 In general, it's not a good idea to write extensions that modify their
515 input parameters, as in Example 3. Instead, you should probably return
516 multiple values in an array and let the caller handle them (we'll do
517 this in a later example). However, in order to better accommodate
518 calling pre-existing C routines, which often do modify their input
519 parameters, this behavior is tolerated.
520 EXAMPLE 4
522 In this example, we'll now begin to write XSUBs that will interact with
523 pre-defined C libraries. To begin with, we will build a small library
524 of our own, then let h2xs write our .pm and .xs files for us.
525 Create a new directory called Mytest2 at the same level as the
527 directory Mytest. In the Mytest2 directory, create another directory
528 called mylib, and cd into that directory.
529 Here we'll create some files that will generate a test library. These
531 will include a C source file and a header file. We'll also create a
532 Makefile.PL in this directory. Then we'll make sure that running make
533 at the Mytest2 level will automatically run this Makefile.PL file and
534 the resulting Makefile.
535 In the mylib directory, create a file mylib.h that looks like this:
537 #define TESTVAL 4
539 extern double foo(int, long, const char*);
541 Also create a file mylib.c that looks like this:
543 #include <stdlib.h>
545 #include "mylib.h"
546 double
548 foo(int a, long b, const char *c)
549 {
550 return (a + b + atof(c) + TESTVAL);
551 }
552 And finally create a file Makefile.PL that looks like this:
554 use ExtUtils::MakeMaker;
556 $Verbose = 1;
557 WriteMakefile(
558 NAME => 'Mytest2::mylib',
559 SKIP => [qw(all static static_lib dynamic dynamic_lib)],
560 clean => {'FILES' => 'libmylib$(LIB_EXT)'},
561 );
562 sub MY::top_targets {
565 '
566 all :: static
567 pure_all :: static
569 static :: libmylib$(LIB_EXT)
571 libmylib$(LIB_EXT): $(O_FILES)
573 $(AR) cr libmylib$(LIB_EXT) $(O_FILES)
574 $(RANLIB) libmylib$(LIB_EXT)
575 ';
577 }
578 Make sure you use a tab and not spaces on the lines beginning with
580 "$(AR)" and "$(RANLIB)". Make will not function properly if you use
581 spaces. It has also been reported that the "cr" argument to $(AR) is
582 unnecessary on Win32 systems.
583 We will now create the main top-level Mytest2 files. Change to the
585 directory above Mytest2 and run the following command:
586 % h2xs -O -n Mytest2 Mytest2/mylib/mylib.h
588 This will print out a warning about overwriting Mytest2, but that's
590 okay. Our files are stored in Mytest2/mylib, and will be untouched.
591 The normal Makefile.PL that h2xs generates doesn't know about the mylib
593 directory. We need to tell it that there is a subdirectory and that we
594 will be generating a library in it. Let's add the argument MYEXTLIB to
595 the WriteMakefile call so that it looks like this:
596 WriteMakefile(
598 NAME => 'Mytest2',
599 VERSION_FROM => 'Mytest2.pm', # finds $VERSION
600 LIBS => [''], # e.g., '-lm'
601 DEFINE => '', # e.g., '-DHAVE_SOMETHING'
602 INC => '', # e.g., '-I/usr/include/other'
603 MYEXTLIB => 'mylib/libmylib$(LIB_EXT)',
604 );
605 and then at the end add a subroutine (which will override the pre-
607 existing subroutine). Remember to use a tab character to indent the
608 line beginning with "cd"!
609 sub MY::postamble {
611 '
612 $(MYEXTLIB): mylib/Makefile
613 cd mylib && $(MAKE) $(PASSTHRU)
614 ';
615 }
616 Let's also fix the MANIFEST file by appending the following three
618 lines:
619 mylib/Makefile.PL
621 mylib/mylib.c
622 mylib/mylib.h
623 To keep our namespace nice and unpolluted, edit the .pm file and change
625 the variable @EXPORT to @EXPORT_OK. Finally, in the .xs file, edit the
626 #include line to read:
627 #include "mylib/mylib.h"
629 And also add the following function definition to the end of the .xs
631 file:
632 double
634 foo(a,b,c)
635 int a
636 long b
637 const char * c
638 OUTPUT:
639 RETVAL
640 Now we also need to create a typemap because the default Perl doesn't
642 currently support the "const char *" type. Include a new TYPEMAP
643 section in your XS code before the above function:
644 TYPEMAP: <<END
646 const char * T_PV
647 END
648 Now run perl on the top-level Makefile.PL. Notice that it also created
650 a Makefile in the mylib directory. Run make and watch that it does cd
651 into the mylib directory and run make in there as well.
652 Now edit the Mytest2.t script and change the number of tests to "5",
654 and add the following lines to the end of the script:
655 is( Mytest2::foo( 1, 2, "Hello, world!" ), 7 );
657 is( Mytest2::foo( 1, 2, "0.0" ), 7 );
658 ok( abs( Mytest2::foo( 0, 0, "-3.4" ) - 0.6 ) <= 0.01 );
659 (When dealing with floating-point comparisons, it is best to not check
661 for equality, but rather that the difference between the expected and
662 actual result is below a certain amount (called epsilon) which is 0.01
663 in this case)
664 Run ""make test"" and all should be well. There are some warnings on
666 missing tests for the Mytest2::mylib extension, but you can ignore
667 them.
668 What has happened here?
670 Unlike previous examples, we've now run h2xs on a real include file.
671 This has caused some extra goodies to appear in both the .pm and .xs
672 files.
673 • In the .xs file, there's now a #include directive with the absolute
675 path to the mylib.h header file. We changed this to a relative
676 path so that we could move the extension directory if we wanted to.
677 • There's now some new C code that's been added to the .xs file. The
679 purpose of the "constant" routine is to make the values that are
680 #define'd in the header file accessible by the Perl script (by
681 calling either "TESTVAL" or &Mytest2::TESTVAL). There's also some
682 XS code to allow calls to the "constant" routine.
683 • The .pm file originally exported the name "TESTVAL" in the @EXPORT
685 array. This could lead to name clashes. A good rule of thumb is
686 that if the #define is only going to be used by the C routines
687 themselves, and not by the user, they should be removed from the
688 @EXPORT array. Alternately, if you don't mind using the "fully
689 qualified name" of a variable, you could move most or all of the
690 items from the @EXPORT array into the @EXPORT_OK array.
691 • If our include file had contained #include directives, these would
693 not have been processed by h2xs. There is no good solution to this
694 right now.
695 • We've also told Perl about the library that we built in the mylib
697 subdirectory. That required only the addition of the "MYEXTLIB"
698 variable to the WriteMakefile call and the replacement of the
699 postamble subroutine to cd into the subdirectory and run make. The
700 Makefile.PL for the library is a bit more complicated, but not
701 excessively so. Again we replaced the postamble subroutine to
702 insert our own code. This code simply specified that the library
703 to be created here was a static archive library (as opposed to a
704 dynamically loadable library) and provided the commands to build
705 it.
706 Anatomy of .xs file
708 The .xs file of "EXAMPLE 4" contained some new elements. To understand
709 the meaning of these elements, pay attention to the line which reads
710 MODULE = Mytest2 PACKAGE = Mytest2
712 Anything before this line is plain C code which describes which headers
714 to include, and defines some convenience functions. No translations
715 are performed on this part, apart from having embedded POD
716 documentation skipped over (see perlpod) it goes into the generated
717 output C file as is.
718 Anything after this line is the description of XSUB functions. These
720 descriptions are translated by xsubpp into C code which implements
721 these functions using Perl calling conventions, and which makes these
722 functions visible from Perl interpreter.
723 Pay a special attention to the function "constant". This name appears
725 twice in the generated .xs file: once in the first part, as a static C
726 function, then another time in the second part, when an XSUB interface
727 to this static C function is defined.
728 This is quite typical for .xs files: usually the .xs file provides an
730 interface to an existing C function. Then this C function is defined
731 somewhere (either in an external library, or in the first part of .xs
732 file), and a Perl interface to this function (i.e. "Perl glue") is
733 described in the second part of .xs file. The situation in "EXAMPLE
734 1", "EXAMPLE 2", and "EXAMPLE 3", when all the work is done inside the
735 "Perl glue", is somewhat of an exception rather than the rule.
736 Getting the fat out of XSUBs
738 In "EXAMPLE 4" the second part of .xs file contained the following
739 description of an XSUB:
740 double
742 foo(a,b,c)
743 int a
744 long b
745 const char * c
746 OUTPUT:
747 RETVAL
748 Note that in contrast with "EXAMPLE 1", "EXAMPLE 2" and "EXAMPLE 3",
750 this description does not contain the actual code for what is done
751 during a call to Perl function foo(). To understand what is going on
752 here, one can add a CODE section to this XSUB:
753 double
755 foo(a,b,c)
756 int a
757 long b
758 const char * c
759 CODE:
760 RETVAL = foo(a,b,c);
761 OUTPUT:
762 RETVAL
763 However, these two XSUBs provide almost identical generated C code:
765 xsubpp compiler is smart enough to figure out the "CODE:" section from
766 the first two lines of the description of XSUB. What about "OUTPUT:"
767 section? In fact, that is absolutely the same! The "OUTPUT:" section
768 can be removed as well, as far as "CODE:" section or "PPCODE:" section
769 is not specified: xsubpp can see that it needs to generate a function
770 call section, and will autogenerate the OUTPUT section too. Thus one
771 can shortcut the XSUB to become:
772 double
774 foo(a,b,c)
775 int a
776 long b
777 const char * c
778 Can we do the same with an XSUB
780 int
782 is_even(input)
783 int input
784 CODE:
785 RETVAL = (input % 2 == 0);
786 OUTPUT:
787 RETVAL
788 of "EXAMPLE 2"? To do this, one needs to define a C function "int
790 is_even(int input)". As we saw in "Anatomy of .xs file", a proper
791 place for this definition is in the first part of .xs file. In fact a
792 C function
793 int
795 is_even(int arg)
796 {
797 return (arg % 2 == 0);
798 }
799 is probably overkill for this. Something as simple as a "#define" will
801 do too:
802 #define is_even(arg) ((arg) % 2 == 0)
804 After having this in the first part of .xs file, the "Perl glue" part
806 becomes as simple as
807 int
809 is_even(input)
810 int input
811 This technique of separation of the glue part from the workhorse part
813 has obvious tradeoffs: if you want to change a Perl interface, you need
814 to change two places in your code. However, it removes a lot of
815 clutter, and makes the workhorse part independent from idiosyncrasies
816 of Perl calling convention. (In fact, there is nothing Perl-specific
817 in the above description, a different version of xsubpp might have
818 translated this to TCL glue or Python glue as well.)
819 More about XSUB arguments
821 With the completion of Example 4, we now have an easy way to simulate
822 some real-life libraries whose interfaces may not be the cleanest in
823 the world. We shall now continue with a discussion of the arguments
824 passed to the xsubpp compiler.
825 When you specify arguments to routines in the .xs file, you are really
827 passing three pieces of information for each argument listed. The
828 first piece is the order of that argument relative to the others
829 (first, second, etc). The second is the type of argument, and consists
830 of the type declaration of the argument (e.g., int, char*, etc). The
831 third piece is the calling convention for the argument in the call to
832 the library function.
833 While Perl passes arguments to functions by reference, C passes
835 arguments by value; to implement a C function which modifies data of
836 one of the "arguments", the actual argument of this C function would be
837 a pointer to the data. Thus two C functions with declarations
838 int string_length(char *s);
840 int upper_case_char(char *cp);
841 may have completely different semantics: the first one may inspect an
843 array of chars pointed by s, and the second one may immediately
844 dereference "cp" and manipulate *cp only (using the return value as,
845 say, a success indicator). From Perl one would use these functions in
846 a completely different manner.
847 One conveys this info to xsubpp by replacing "*" before the argument by
849 "&". "&" means that the argument should be passed to a library
850 function by its address. The above two function may be XSUB-ified as
851 int
853 string_length(s)
854 char * s
855 int
857 upper_case_char(cp)
858 char &cp
859 For example, consider:
861 int
863 foo(a,b)
864 char &a
865 char * b
866 The first Perl argument to this function would be treated as a char and
868 assigned to the variable a, and its address would be passed into the
869 function foo. The second Perl argument would be treated as a string
870 pointer and assigned to the variable b. The value of b would be passed
871 into the function foo. The actual call to the function foo that xsubpp
872 generates would look like this:
873 foo(&a, b);
875 xsubpp will parse the following function argument lists identically:
877 char &a
879 char&a
880 char & a
881 However, to help ease understanding, it is suggested that you place a
883 "&" next to the variable name and away from the variable type), and
884 place a "*" near the variable type, but away from the variable name (as
885 in the call to foo above). By doing so, it is easy to understand
886 exactly what will be passed to the C function; it will be whatever is
887 in the "last column".
888 You should take great pains to try to pass the function the type of
890 variable it wants, when possible. It will save you a lot of trouble in
891 the long run.
892 The Argument Stack
894 If we look at any of the C code generated by any of the examples except
895 example 1, you will notice a number of references to ST(n), where n is
896 usually 0. "ST" is actually a macro that points to the n'th argument
897 on the argument stack. ST(0) is thus the first argument on the stack
898 and therefore the first argument passed to the XSUB, ST(1) is the
899 second argument, and so on.
900 When you list the arguments to the XSUB in the .xs file, that tells
902 xsubpp which argument corresponds to which of the argument stack (i.e.,
903 the first one listed is the first argument, and so on). You invite
904 disaster if you do not list them in the same order as the function
905 expects them.
906 The actual values on the argument stack are pointers to the values
908 passed in. When an argument is listed as being an OUTPUT value, its
909 corresponding value on the stack (i.e., ST(0) if it was the first
910 argument) is changed. You can verify this by looking at the C code
911 generated for Example 3. The code for the round() XSUB routine
912 contains lines that look like this:
913 double arg = (double)SvNV(ST(0));
915 /* Round the contents of the variable arg */
916 sv_setnv(ST(0), (double)arg);
917 The arg variable is initially set by taking the value from ST(0), then
919 is stored back into ST(0) at the end of the routine.
920 XSUBs are also allowed to return lists, not just scalars. This must be
922 done by manipulating stack values ST(0), ST(1), etc, in a subtly
923 different way. See perlxs for details.
924 XSUBs are also allowed to avoid automatic conversion of Perl function
926 arguments to C function arguments. See perlxs for details. Some
927 people prefer manual conversion by inspecting ST(i) even in the cases
928 when automatic conversion will do, arguing that this makes the logic of
929 an XSUB call clearer. Compare with "Getting the fat out of XSUBs" for
930 a similar tradeoff of a complete separation of "Perl glue" and
931 "workhorse" parts of an XSUB.
932 While experts may argue about these idioms, a novice to Perl guts may
934 prefer a way which is as little Perl-guts-specific as possible, meaning
935 automatic conversion and automatic call generation, as in "Getting the
936 fat out of XSUBs". This approach has the additional benefit of
937 protecting the XSUB writer from future changes to the Perl API.
938 Extending your Extension
940 Sometimes you might want to provide some extra methods or subroutines
941 to assist in making the interface between Perl and your extension
942 simpler or easier to understand. These routines should live in the .pm
943 file. Whether they are automatically loaded when the extension itself
944 is loaded or only loaded when called depends on where in the .pm file
945 the subroutine definition is placed. You can also consult AutoLoader
946 for an alternate way to store and load your extra subroutines.
947 Documenting your Extension
949 There is absolutely no excuse for not documenting your extension.
950 Documentation belongs in the .pm file. This file will be fed to
951 pod2man, and the embedded documentation will be converted to the
952 manpage format, then placed in the blib directory. It will be copied
953 to Perl's manpage directory when the extension is installed.
954 You may intersperse documentation and Perl code within the .pm file.
956 In fact, if you want to use method autoloading, you must do this, as
957 the comment inside the .pm file explains.
958 See perlpod for more information about the pod format.
960 Installing your Extension
962 Once your extension is complete and passes all its tests, installing it
963 is quite simple: you simply run "make install". You will either need
964 to have write permission into the directories where Perl is installed,
965 or ask your system administrator to run the make for you.
966 Alternately, you can specify the exact directory to place the
968 extension's files by placing a "PREFIX=/destination/directory" after
969 the make install (or in between the make and install if you have a
970 brain-dead version of make). This can be very useful if you are
971 building an extension that will eventually be distributed to multiple
972 systems. You can then just archive the files in the destination
973 directory and distribute them to your destination systems.
974 EXAMPLE 5
976 In this example, we'll do some more work with the argument stack. The
977 previous examples have all returned only a single value. We'll now
978 create an extension that returns an array.
979 This extension is very Unix-oriented (struct statfs and the statfs
981 system call). If you are not running on a Unix system, you can
982 substitute for statfs any other function that returns multiple values,
983 you can hard-code values to be returned to the caller (although this
984 will be a bit harder to test the error case), or you can simply not do
985 this example. If you change the XSUB, be sure to fix the test cases to
986 match the changes.
987 Return to the Mytest directory and add the following code to the end of
989 Mytest.xs:
990 void
992 statfs(path)
993 char * path
994 INIT:
995 int i;
996 struct statfs buf;
997 PPCODE:
999 i = statfs(path, &buf);
1000 if (i == 0) {
1001 XPUSHs(sv_2mortal(newSVnv(buf.f_bavail)));
1002 XPUSHs(sv_2mortal(newSVnv(buf.f_bfree)));
1003 XPUSHs(sv_2mortal(newSVnv(buf.f_blocks)));
1004 XPUSHs(sv_2mortal(newSVnv(buf.f_bsize)));
1005 XPUSHs(sv_2mortal(newSVnv(buf.f_ffree)));
1006 XPUSHs(sv_2mortal(newSVnv(buf.f_files)));
1007 XPUSHs(sv_2mortal(newSVnv(buf.f_type)));
1008 } else {
1009 XPUSHs(sv_2mortal(newSVnv(errno)));
1010 }
1011 You'll also need to add the following code to the top of the .xs file,
1013 just after the include of "XSUB.h":
1014 #include <sys/vfs.h>
1016 Also add the following code segment to Mytest.t while incrementing the
1018 "9" tests to "11":
1019 my @a;
1021 @a = Mytest::statfs("/blech");
1023 ok( scalar(@a) == 1 && $a[0] == 2 );
1024 @a = Mytest::statfs("/");
1026 is( scalar(@a), 7 );
1027 New Things in this Example
1029 This example added quite a few new concepts. We'll take them one at a
1030 time.
1031 • The INIT: directive contains code that will be placed immediately
1033 after the argument stack is decoded. C does not allow variable
1034 declarations at arbitrary locations inside a function, so this is
1035 usually the best way to declare local variables needed by the XSUB.
1036 (Alternatively, one could put the whole "PPCODE:" section into
1037 braces, and put these declarations on top.)
1038 • This routine also returns a different number of arguments depending
1040 on the success or failure of the call to statfs. If there is an
1041 error, the error number is returned as a single-element array. If
1042 the call is successful, then a 7-element array is returned. Since
1043 only one argument is passed into this function, we need room on the
1044 stack to hold the 7 values which may be returned.
1045 We do this by using the PPCODE: directive, rather than the CODE:
1047 directive. This tells xsubpp that we will be managing the return
1048 values that will be put on the argument stack by ourselves.
1049 • When we want to place values to be returned to the caller onto the
1051 stack, we use the series of macros that begin with "XPUSH". There
1052 are five different versions, for placing integers, unsigned
1053 integers, doubles, strings, and Perl scalars on the stack. In our
1054 example, we placed a Perl scalar onto the stack. (In fact this is
1055 the only macro which can be used to return multiple values.)
1056 The XPUSH* macros will automatically extend the return stack to
1058 prevent it from being overrun. You push values onto the stack in
1059 the order you want them seen by the calling program.
1060 • The values pushed onto the return stack of the XSUB are actually
1062 mortal SV's. They are made mortal so that once the values are
1063 copied by the calling program, the SV's that held the returned
1064 values can be deallocated. If they were not mortal, then they
1065 would continue to exist after the XSUB routine returned, but would
1066 not be accessible. This is a memory leak.
1067 • If we were interested in performance, not in code compactness, in
1069 the success branch we would not use "XPUSHs" macros, but "PUSHs"
1070 macros, and would pre-extend the stack before pushing the return
1071 values:
1072 EXTEND(SP, 7);
1074 The tradeoff is that one needs to calculate the number of return
1076 values in advance (though overextending the stack will not
1077 typically hurt anything but memory consumption).
1078 Similarly, in the failure branch we could use "PUSHs" without
1080 extending the stack: the Perl function reference comes to an XSUB
1081 on the stack, thus the stack is always large enough to take one
1082 return value.
1083 EXAMPLE 6
1085 In this example, we will accept a reference to an array as an input
1086 parameter, and return a reference to an array of hashes. This will
1087 demonstrate manipulation of complex Perl data types from an XSUB.
1088 This extension is somewhat contrived. It is based on the code in the
1090 previous example. It calls the statfs function multiple times,
1091 accepting a reference to an array of filenames as input, and returning
1092 a reference to an array of hashes containing the data for each of the
1093 filesystems.
1094 Return to the Mytest directory and add the following code to the end of
1096 Mytest.xs:
1097 SV *
1099 multi_statfs(paths)
1100 SV * paths
1101 INIT:
1102 AV * results;
1103 SSize_t numpaths = 0, n;
1104 int i;
1105 struct statfs buf;
1106 SvGETMAGIC(paths);
1108 if ((!SvROK(paths))
1109 || (SvTYPE(SvRV(paths)) != SVt_PVAV)
1110 || ((numpaths = av_top_index((AV *)SvRV(paths))) < 0))
1111 {
1112 XSRETURN_UNDEF;
1113 }
1114 results = (AV *)sv_2mortal((SV *)newAV());
1115 CODE:
1116 for (n = 0; n <= numpaths; n++) {
1117 HV * rh;
1118 STRLEN l;
1119 SV * path = *av_fetch((AV *)SvRV(paths), n, 0);
1120 char * fn = SvPVbyte(path, l);
1121 i = statfs(fn, &buf);
1123 if (i != 0) {
1124 av_push(results, newSVnv(errno));
1125 continue;
1126 }
1127 rh = (HV *)sv_2mortal((SV *)newHV());
1129 hv_store(rh, "f_bavail", 8, newSVnv(buf.f_bavail), 0);
1131 hv_store(rh, "f_bfree", 7, newSVnv(buf.f_bfree), 0);
1132 hv_store(rh, "f_blocks", 8, newSVnv(buf.f_blocks), 0);
1133 hv_store(rh, "f_bsize", 7, newSVnv(buf.f_bsize), 0);
1134 hv_store(rh, "f_ffree", 7, newSVnv(buf.f_ffree), 0);
1135 hv_store(rh, "f_files", 7, newSVnv(buf.f_files), 0);
1136 hv_store(rh, "f_type", 6, newSVnv(buf.f_type), 0);
1137 av_push(results, newRV_inc((SV *)rh));
1139 }
1140 RETVAL = newRV_inc((SV *)results);
1141 OUTPUT:
1142 RETVAL
1143 And add the following code to Mytest.t, while incrementing the "11"
1145 tests to "13":
1146 my $results = Mytest::multi_statfs([ '/', '/blech' ]);
1148 ok( ref $results->[0] );
1149 ok( ! ref $results->[1] );
1150 New Things in this Example
1152 There are a number of new concepts introduced here, described below:
1153 • This function does not use a typemap. Instead, we declare it as
1155 accepting one SV* (scalar) parameter, and returning an SV* value,
1156 and we take care of populating these scalars within the code.
1157 Because we are only returning one value, we don't need a "PPCODE:"
1158 directive - instead, we use "CODE:" and "OUTPUT:" directives.
1159 • When dealing with references, it is important to handle them with
1161 caution. The "INIT:" block first calls SvGETMAGIC(paths), in case
1162 paths is a tied variable. Then it checks that "SvROK" returns
1163 true, which indicates that paths is a valid reference. (Simply
1164 checking "SvROK" won't trigger FETCH on a tied variable.) It then
1165 verifies that the object referenced by paths is an array, using
1166 "SvRV" to dereference paths, and "SvTYPE" to discover its type. As
1167 an added test, it checks that the array referenced by paths is non-
1168 empty, using the "av_top_index" function (which returns -1 if the
1169 array is empty). The XSRETURN_UNDEF macro is used to abort the XSUB
1170 and return the undefined value whenever all three of these
1171 conditions are not met.
1172 • We manipulate several arrays in this XSUB. Note that an array is
1174 represented internally by an AV* pointer. The functions and macros
1175 for manipulating arrays are similar to the functions in Perl:
1176 "av_top_index" returns the highest index in an AV*, much like
1177 $#array; "av_fetch" fetches a single scalar value from an array,
1178 given its index; "av_push" pushes a scalar value onto the end of
1179 the array, automatically extending the array as necessary.
1180 Specifically, we read pathnames one at a time from the input array,
1182 and store the results in an output array (results) in the same
1183 order. If statfs fails, the element pushed onto the return array
1184 is the value of errno after the failure. If statfs succeeds,
1185 though, the value pushed onto the return array is a reference to a
1186 hash containing some of the information in the statfs structure.
1187 As with the return stack, it would be possible (and a small
1189 performance win) to pre-extend the return array before pushing data
1190 into it, since we know how many elements we will return:
1191 av_extend(results, numpaths);
1193 • We are performing only one hash operation in this function, which
1195 is storing a new scalar under a key using "hv_store". A hash is
1196 represented by an HV* pointer. Like arrays, the functions for
1197 manipulating hashes from an XSUB mirror the functionality available
1198 from Perl. See perlguts and perlapi for details.
1199 • To create a reference, we use the "newRV_inc" function. Note that
1201 you can cast an AV* or an HV* to type SV* in this case (and many
1202 others). This allows you to take references to arrays, hashes and
1203 scalars with the same function. Conversely, the "SvRV" function
1204 always returns an SV*, which may need to be cast to the appropriate
1205 type if it is something other than a scalar (check with "SvTYPE").
1206 • At this point, xsubpp is doing very little work - the differences
1208 between Mytest.xs and Mytest.c are minimal.
1209 EXAMPLE 7 (Coming Soon)
1211 XPUSH args AND set RETVAL AND assign return value to array
1212 EXAMPLE 8 (Coming Soon)
1214 Setting $!
1215 EXAMPLE 9 Passing open files to XSes
1217 You would think passing files to an XS is difficult, with all the
1218 typeglobs and stuff. Well, it isn't.
1219 Suppose that for some strange reason we need a wrapper around the
1221 standard C library function "fputs()". This is all we need:
1222 #define PERLIO_NOT_STDIO 0 /* For co-existence with stdio only */
1224 #define PERL_NO_GET_CONTEXT /* This is more efficient */
1225 #include "EXTERN.h"
1226 #include "perl.h"
1227 #include "XSUB.h"
1228 #include <stdio.h>
1230 int
1232 fputs(s, stream)
1233 char * s
1234 FILE * stream
1235 The real work is done in the standard typemap.
1237 For more details, see "Co-existence with stdio" in perlapio.
1239 But you lose all the fine stuff done by the perlio layers. This calls
1241 the stdio function "fputs()", which knows nothing about them.
1242 The standard typemap offers three variants of PerlIO *: "InputStream"
1244 (T_IN), "InOutStream" (T_INOUT) and "OutputStream" (T_OUT). A bare
1245 "PerlIO *" is considered a T_INOUT. If it matters in your code (see
1246 below for why it might) #define or typedef one of the specific names
1247 and use that as the argument or result type in your XS file.
1248 The standard typemap does not contain PerlIO * before perl 5.7, but it
1250 has the three stream variants. Using a PerlIO * directly is not
1251 backwards compatible unless you provide your own typemap.
1252 For streams coming from perl the main difference is that "OutputStream"
1254 will get the output PerlIO * - which may make a difference on a socket.
1255 Like in our example...
1256 For streams being handed to perl a new file handle is created (i.e. a
1258 reference to a new glob) and associated with the PerlIO * provided. If
1259 the read/write state of the PerlIO * is not correct then you may get
1260 errors or warnings from when the file handle is used. So if you opened
1261 the PerlIO * as "w" it should really be an "OutputStream" if open as
1262 "r" it should be an "InputStream".
1263 Now, suppose you want to use perlio layers in your XS. We'll use the
1265 perlio "PerlIO_puts()" function as an example.
1266 In the C part of the XS file (above the first MODULE line) you have
1268 #define OutputStream PerlIO *
1270 or
1271 typedef PerlIO * OutputStream;
1272 And this is the XS code:
1274 int
1276 perlioputs(s, stream)
1277 char * s
1278 OutputStream stream
1279 CODE:
1280 RETVAL = PerlIO_puts(stream, s);
1281 OUTPUT:
1282 RETVAL
1283 We have to use a "CODE" section because "PerlIO_puts()" has the
1285 arguments reversed compared to "fputs()", and we want to keep the
1286 arguments the same.
1287 Wanting to explore this thoroughly, we want to use the stdio "fputs()"
1289 on a PerlIO *. This means we have to ask the perlio system for a stdio
1290 "FILE *":
1291 int
1293 perliofputs(s, stream)
1294 char * s
1295 OutputStream stream
1296 PREINIT:
1297 FILE *fp = PerlIO_findFILE(stream);
1298 CODE:
1299 if (fp != (FILE*) 0) {
1300 RETVAL = fputs(s, fp);
1301 } else {
1302 RETVAL = -1;
1303 }
1304 OUTPUT:
1305 RETVAL
1306 Note: "PerlIO_findFILE()" will search the layers for a stdio layer. If
1308 it can't find one, it will call "PerlIO_exportFILE()" to generate a new
1309 stdio "FILE". Please only call "PerlIO_exportFILE()" if you want a new
1310 "FILE". It will generate one on each call and push a new stdio layer.
1311 So don't call it repeatedly on the same file. "PerlIO_findFILE()" will
1312 retrieve the stdio layer once it has been generated by
1313 "PerlIO_exportFILE()".
1314 This applies to the perlio system only. For versions before 5.7,
1316 "PerlIO_exportFILE()" is equivalent to "PerlIO_findFILE()".
1317 Troubleshooting these Examples
1319 As mentioned at the top of this document, if you are having problems
1320 with these example extensions, you might see if any of these help you.
1321 • In versions of 5.002 prior to the gamma version, the test script in
1323 Example 1 will not function properly. You need to change the "use
1324 lib" line to read:
1325 use lib './blib';
1327 • In versions of 5.002 prior to version 5.002b1h, the test.pl file
1329 was not automatically created by h2xs. This means that you cannot
1330 say "make test" to run the test script. You will need to add the
1331 following line before the "use extension" statement:
1332 use lib './blib';
1334 • In versions 5.000 and 5.001, instead of using the above line, you
1336 will need to use the following line:
1337 BEGIN { unshift(@INC, "./blib") }
1339 • This document assumes that the executable named "perl" is Perl
1341 version 5. Some systems may have installed Perl version 5 as
1342 "perl5".
1343
1345 For more information, consult perlguts, perlapi, perlxs, perlmod,
1346 perlapio, and perlpod
1347
1349 Jeff Okamoto <okamoto@corp.hp.com>
1350 Reviewed and assisted by Dean Roehrich, Ilya Zakharevich, Andreas
1352 Koenig, and Tim Bunce.
1353 PerlIO material contributed by Lupe Christoph, with some clarification
1355 by Nick Ing-Simmons.
1356 Changes for h2xs as of Perl 5.8.x by Renee Baecker
1358 This document is now maintained as part of Perl itself.
1360 Last Changed
1362 2020-10-05
1363perl v5.36.0 2022-07-22 perlxstut(3)