1perlxstut(3) User Contributed Perl Documentation perlxstut(3)
2
6 perlxstut - Tutorial for writing XSUBs
7
9 This tutorial will educate the reader on the steps involved in creating
10 a Perl extension. The reader is assumed to have access to perlguts,
11 perlapi and perlxs.
12 This tutorial starts with very simple examples and becomes more
14 complex, with each new example adding new features. Certain concepts
15 may not be completely explained until later in the tutorial in order to
16 slowly ease the reader into building extensions.
17 This tutorial was written from a Unix point of view. Where I know them
19 to be otherwise different for other platforms (e.g. Win32), I will list
20 them. If you find something that was missed, please let me know.
21
23 make
24 This tutorial assumes that the make program that Perl is configured to
25 use is called "make". Instead of running "make" in the examples that
26 follow, you may have to substitute whatever make program Perl has been
27 configured to use. Running perl -V:make should tell you what it is.
28 Version caveat
30 When writing a Perl extension for general consumption, one should
31 expect that the extension will be used with versions of Perl different
32 from the version available on your machine. Since you are reading this
33 document, the version of Perl on your machine is probably 5.005 or
34 later, but the users of your extension may have more ancient versions.
35 To understand what kinds of incompatibilities one may expect, and in
37 the rare case that the version of Perl on your machine is older than
38 this document, see the section on "Troubleshooting these Examples" for
39 more information.
40 If your extension uses some features of Perl which are not available on
42 older releases of Perl, your users would appreciate an early meaningful
43 warning. You would probably put this information into the README file,
44 but nowadays installation of extensions may be performed automatically,
45 guided by CPAN.pm module or other tools.
46 In MakeMaker-based installations, Makefile.PL provides the earliest
48 opportunity to perform version checks. One can put something like this
49 in Makefile.PL for this purpose:
50 eval { require 5.007 }
52 or die <<EOD;
53 ############
54 ### This module uses frobnication framework which is not available
55 ### before version 5.007 of Perl. Upgrade your Perl before
56 ### installing Kara::Mba.
57 ############
58 EOD
59 Dynamic Loading versus Static Loading
61 It is commonly thought that if a system does not have the capability to
62 dynamically load a library, you cannot build XSUBs. This is incorrect.
63 You can build them, but you must link the XSUBs subroutines with the
64 rest of Perl, creating a new executable. This situation is similar to
65 Perl 4.
66 This tutorial can still be used on such a system. The XSUB build
68 mechanism will check the system and build a dynamically-loadable
69 library if possible, or else a static library and then, optionally, a
70 new statically-linked executable with that static library linked in.
71 Should you wish to build a statically-linked executable on a system
73 which can dynamically load libraries, you may, in all the following
74 examples, where the command ""make"" with no arguments is executed, run
75 the command ""make perl"" instead.
76 If you have generated such a statically-linked executable by choice,
78 then instead of saying ""make test"", you should say ""make
79 test_static"". On systems that cannot build dynamically-loadable
80 libraries at all, simply saying ""make test"" is sufficient.
81 Threads and PERL_NO_GET_CONTEXT
83 For threaded builds, perl requires the context pointer for the current
84 thread, without "PERL_NO_GET_CONTEXT", perl will call a function to
85 retrieve the context.
86 For improved performance, include:
88 #define PERL_NO_GET_CONTEXT
90 as shown below.
92 For more details, see perlguts.
94
96 Now let's go on with the show!
97 EXAMPLE 1
99 Our first extension will be very simple. When we call the routine in
100 the extension, it will print out a well-known message and return.
101 Run ""h2xs -A -n Mytest"". This creates a directory named Mytest,
103 possibly under ext/ if that directory exists in the current working
104 directory. Several files will be created under the Mytest dir,
105 including MANIFEST, Makefile.PL, lib/Mytest.pm, Mytest.xs, t/Mytest.t,
106 and Changes.
107 The MANIFEST file contains the names of all the files just created in
109 the Mytest directory.
110 The file Makefile.PL should look something like this:
112 use ExtUtils::MakeMaker;
114 # See lib/ExtUtils/MakeMaker.pm for details of how to influence
115 # the contents of the Makefile that is written.
116 WriteMakefile(
117 NAME => 'Mytest',
118 VERSION_FROM => 'Mytest.pm', # finds $VERSION
119 LIBS => [''], # e.g., '-lm'
120 DEFINE => '', # e.g., '-DHAVE_SOMETHING'
121 INC => '', # e.g., '-I/usr/include/other'
122 );
123 The file Mytest.pm should start with something like this:
125 package Mytest;
127 use 5.008008;
129 use strict;
130 use warnings;
131 require Exporter;
133 our @ISA = qw(Exporter);
135 our %EXPORT_TAGS = ( 'all' => [ qw(
136 ) ] );
138 our @EXPORT_OK = ( @{ $EXPORT_TAGS{'all'} } );
140 our @EXPORT = qw(
142 );
144 our $VERSION = '0.01';
146 require XSLoader;
148 XSLoader::load('Mytest', $VERSION);
149 # Preloaded methods go here.
151 1;
153 __END__
154 # Below is the stub of documentation for your module. You better
155 # edit it!
156 The rest of the .pm file contains sample code for providing
158 documentation for the extension.
159 Finally, the Mytest.xs file should look something like this:
161 #define PERL_NO_GET_CONTEXT
163 #include "EXTERN.h"
164 #include "perl.h"
165 #include "XSUB.h"
166 #include "ppport.h"
168 MODULE = Mytest PACKAGE = Mytest
170 Let's edit the .xs file by adding this to the end of the file:
172 void
174 hello()
175 CODE:
176 printf("Hello, world!\n");
177 It is okay for the lines starting at the "CODE:" line to not be
179 indented. However, for readability purposes, it is suggested that you
180 indent CODE: one level and the lines following one more level.
181 Now we'll run ""perl Makefile.PL"". This will create a real Makefile,
183 which make needs. Its output looks something like:
184 % perl Makefile.PL
186 Checking if your kit is complete...
187 Looks good
188 Writing Makefile for Mytest
189 %
190 Now, running make will produce output that looks something like this
192 (some long lines have been shortened for clarity and some extraneous
193 lines have been deleted):
194 % make
196 cp lib/Mytest.pm blib/lib/Mytest.pm
197 perl xsubpp -typemap typemap Mytest.xs > Mytest.xsc && \
198 mv Mytest.xsc Mytest.c
199 Please specify prototyping behavior for Mytest.xs (see perlxs manual)
200 cc -c Mytest.c
201 Running Mkbootstrap for Mytest ()
202 chmod 644 Mytest.bs
203 rm -f blib/arch/auto/Mytest/Mytest.so
204 cc -shared -L/usr/local/lib Mytest.o -o blib/arch/auto/Mytest/Mytest.so
205 chmod 755 blib/arch/auto/Mytest/Mytest.so
207 cp Mytest.bs blib/arch/auto/Mytest/Mytest.bs
208 chmod 644 blib/arch/auto/Mytest/Mytest.bs
209 Manifying blib/man3/Mytest.3pm
210 %
211 You can safely ignore the line about "prototyping behavior" - it is
213 explained in "The PROTOTYPES: Keyword" in perlxs.
214 Perl has its own special way of easily writing test scripts, but for
216 this example only, we'll create our own test script. Create a file
217 called hello that looks like this:
218 #! /opt/perl5/bin/perl
220 use ExtUtils::testlib;
222 use Mytest;
224 Mytest::hello();
226 Now we make the script executable ("chmod +x hello"), run the script
228 and we should see the following output:
229 % ./hello
231 Hello, world!
232 %
233 EXAMPLE 2
235 Now let's add to our extension a subroutine that will take a single
236 numeric argument as input and return 1 if the number is even or 0 if
237 the number is odd.
238 Add the following to the end of Mytest.xs:
240 int
242 is_even(input)
243 int input
244 CODE:
245 RETVAL = (input % 2 == 0);
246 OUTPUT:
247 RETVAL
248 There does not need to be whitespace at the start of the ""int input""
250 line, but it is useful for improving readability. Placing a semi-colon
251 at the end of that line is also optional. Any amount and kind of
252 whitespace may be placed between the ""int"" and ""input"".
253 Now re-run make to rebuild our new shared library.
255 Now perform the same steps as before, generating a Makefile from the
257 Makefile.PL file, and running make.
258 In order to test that our extension works, we now need to look at the
260 file Mytest.t. This file is set up to imitate the same kind of testing
261 structure that Perl itself has. Within the test script, you perform a
262 number of tests to confirm the behavior of the extension, printing "ok"
263 when the test is correct, "not ok" when it is not.
264 use Test::More tests => 4;
266 BEGIN { use_ok('Mytest') };
267 #########################
269 # Insert your test code below, the Test::More module is use()ed here
271 # so read its man page ( perldoc Test::More ) for help writing this
272 # test script.
273 is(&Mytest::is_even(0), 1);
275 is(&Mytest::is_even(1), 0);
276 is(&Mytest::is_even(2), 1);
277 We will be calling the test script through the command ""make test"".
279 You should see output that looks something like this:
280 %make test
282 PERL_DL_NONLAZY=1 /usr/bin/perl "-MExtUtils::Command::MM" "-e"
283 "test_harness(0, 'blib/lib', 'blib/arch')" t/*.t
284 t/Mytest....ok
285 All tests successful.
286 Files=1, Tests=4, 0 wallclock secs ( 0.03 cusr + 0.00 csys = 0.03 CPU)
287 %
288 What has gone on?
290 The program h2xs is the starting point for creating extensions. In
291 later examples we'll see how we can use h2xs to read header files and
292 generate templates to connect to C routines.
293 h2xs creates a number of files in the extension directory. The file
295 Makefile.PL is a perl script which will generate a true Makefile to
296 build the extension. We'll take a closer look at it later.
297 The .pm and .xs files contain the meat of the extension. The .xs file
299 holds the C routines that make up the extension. The .pm file contains
300 routines that tell Perl how to load your extension.
301 Generating the Makefile and running "make" created a directory called
303 blib (which stands for "build library") in the current working
304 directory. This directory will contain the shared library that we will
305 build. Once we have tested it, we can install it into its final
306 location.
307 Invoking the test script via ""make test"" did something very
309 important. It invoked perl with all those "-I" arguments so that it
310 could find the various files that are part of the extension. It is
311 very important that while you are still testing extensions that you use
312 ""make test"". If you try to run the test script all by itself, you
313 will get a fatal error. Another reason it is important to use ""make
314 test"" to run your test script is that if you are testing an upgrade to
315 an already-existing version, using ""make test"" ensures that you will
316 test your new extension, not the already-existing version.
317 When Perl sees a "use extension;", it searches for a file with the same
319 name as the "use"'d extension that has a .pm suffix. If that file
320 cannot be found, Perl dies with a fatal error. The default search path
321 is contained in the @INC array.
322 In our case, Mytest.pm tells perl that it will need the Exporter and
324 Dynamic Loader extensions. It then sets the @ISA and @EXPORT arrays
325 and the $VERSION scalar; finally it tells perl to bootstrap the module.
326 Perl will call its dynamic loader routine (if there is one) and load
327 the shared library.
328 The two arrays @ISA and @EXPORT are very important. The @ISA array
330 contains a list of other packages in which to search for methods (or
331 subroutines) that do not exist in the current package. This is usually
332 only important for object-oriented extensions (which we will talk about
333 much later), and so usually doesn't need to be modified.
334 The @EXPORT array tells Perl which of the extension's variables and
336 subroutines should be placed into the calling package's namespace.
337 Because you don't know if the user has already used your variable and
338 subroutine names, it's vitally important to carefully select what to
339 export. Do not export method or variable names by default without a
340 good reason.
341 As a general rule, if the module is trying to be object-oriented then
343 don't export anything. If it's just a collection of functions and
344 variables, then you can export them via another array, called
345 @EXPORT_OK. This array does not automatically place its subroutine and
346 variable names into the namespace unless the user specifically requests
347 that this be done.
348 See perlmod for more information.
350 The $VERSION variable is used to ensure that the .pm file and the
352 shared library are "in sync" with each other. Any time you make
353 changes to the .pm or .xs files, you should increment the value of this
354 variable.
355 Writing good test scripts
357 The importance of writing good test scripts cannot be over-emphasized.
358 You should closely follow the "ok/not ok" style that Perl itself uses,
359 so that it is very easy and unambiguous to determine the outcome of
360 each test case. When you find and fix a bug, make sure you add a test
361 case for it.
362 By running ""make test"", you ensure that your Mytest.t script runs and
364 uses the correct version of your extension. If you have many test
365 cases, save your test files in the "t" directory and use the suffix
366 ".t". When you run ""make test"", all of these test files will be
367 executed.
368 EXAMPLE 3
370 Our third extension will take one argument as its input, round off that
371 value, and set the argument to the rounded value.
372 Add the following to the end of Mytest.xs:
374 void
376 round(arg)
377 double arg
378 CODE:
379 if (arg > 0.0) {
380 arg = floor(arg + 0.5);
381 } else if (arg < 0.0) {
382 arg = ceil(arg - 0.5);
383 } else {
384 arg = 0.0;
385 }
386 OUTPUT:
387 arg
388 Edit the Makefile.PL file so that the corresponding line looks like
390 this:
391 'LIBS' => ['-lm'], # e.g., '-lm'
393 Generate the Makefile and run make. Change the test number in Mytest.t
395 to "9" and add the following tests:
396 $i = -1.5; &Mytest::round($i); is( $i, -2.0 );
398 $i = -1.1; &Mytest::round($i); is( $i, -1.0 );
399 $i = 0.0; &Mytest::round($i); is( $i, 0.0 );
400 $i = 0.5; &Mytest::round($i); is( $i, 1.0 );
401 $i = 1.2; &Mytest::round($i); is( $i, 1.0 );
402 Running ""make test"" should now print out that all nine tests are
404 okay.
405 Notice that in these new test cases, the argument passed to round was a
407 scalar variable. You might be wondering if you can round a constant or
408 literal. To see what happens, temporarily add the following line to
409 Mytest.t:
410 &Mytest::round(3);
412 Run ""make test"" and notice that Perl dies with a fatal error. Perl
414 won't let you change the value of constants!
415 What's new here?
417 · We've made some changes to Makefile.PL. In this case, we've
418 specified an extra library to be linked into the extension's shared
419 library, the math library libm in this case. We'll talk later
420 about how to write XSUBs that can call every routine in a library.
421 · The value of the function is not being passed back as the
423 function's return value, but by changing the value of the variable
424 that was passed into the function. You might have guessed that
425 when you saw that the return value of round is of type "void".
426 Input and Output Parameters
428 You specify the parameters that will be passed into the XSUB on the
429 line(s) after you declare the function's return value and name. Each
430 input parameter line starts with optional whitespace, and may have an
431 optional terminating semicolon.
432 The list of output parameters occurs at the very end of the function,
434 just after the OUTPUT: directive. The use of RETVAL tells Perl that
435 you wish to send this value back as the return value of the XSUB
436 function. In Example 3, we wanted the "return value" placed in the
437 original variable which we passed in, so we listed it (and not RETVAL)
438 in the OUTPUT: section.
439 The XSUBPP Program
441 The xsubpp program takes the XS code in the .xs file and translates it
442 into C code, placing it in a file whose suffix is .c. The C code
443 created makes heavy use of the C functions within Perl.
444 The TYPEMAP file
446 The xsubpp program uses rules to convert from Perl's data types
447 (scalar, array, etc.) to C's data types (int, char, etc.). These rules
448 are stored in the typemap file ($PERLLIB/ExtUtils/typemap). There's a
449 brief discussion below, but all the nitty-gritty details can be found
450 in perlxstypemap. If you have a new-enough version of perl (5.16 and
451 up) or an upgraded XS compiler ("ExtUtils::ParseXS" 3.13_01 or better),
452 then you can inline typemaps in your XS instead of writing separate
453 files. Either way, this typemap thing is split into three parts:
454 The first section maps various C data types to a name, which
456 corresponds somewhat with the various Perl types. The second section
457 contains C code which xsubpp uses to handle input parameters. The
458 third section contains C code which xsubpp uses to handle output
459 parameters.
460 Let's take a look at a portion of the .c file created for our
462 extension. The file name is Mytest.c:
463 XS(XS_Mytest_round)
465 {
466 dXSARGS;
467 if (items != 1)
468 Perl_croak(aTHX_ "Usage: Mytest::round(arg)");
469 PERL_UNUSED_VAR(cv); /* -W */
470 {
471 double arg = (double)SvNV(ST(0)); /* XXXXX */
472 if (arg > 0.0) {
473 arg = floor(arg + 0.5);
474 } else if (arg < 0.0) {
475 arg = ceil(arg - 0.5);
476 } else {
477 arg = 0.0;
478 }
479 sv_setnv(ST(0), (double)arg); /* XXXXX */
480 SvSETMAGIC(ST(0));
481 }
482 XSRETURN_EMPTY;
483 }
484 Notice the two lines commented with "XXXXX". If you check the first
486 part of the typemap file (or section), you'll see that doubles are of
487 type T_DOUBLE. In the INPUT part of the typemap, an argument that is
488 T_DOUBLE is assigned to the variable arg by calling the routine SvNV on
489 something, then casting it to double, then assigned to the variable
490 arg. Similarly, in the OUTPUT section, once arg has its final value,
491 it is passed to the sv_setnv function to be passed back to the calling
492 subroutine. These two functions are explained in perlguts; we'll talk
493 more later about what that "ST(0)" means in the section on the argument
494 stack.
495 Warning about Output Arguments
497 In general, it's not a good idea to write extensions that modify their
498 input parameters, as in Example 3. Instead, you should probably return
499 multiple values in an array and let the caller handle them (we'll do
500 this in a later example). However, in order to better accommodate
501 calling pre-existing C routines, which often do modify their input
502 parameters, this behavior is tolerated.
503 EXAMPLE 4
505 In this example, we'll now begin to write XSUBs that will interact with
506 pre-defined C libraries. To begin with, we will build a small library
507 of our own, then let h2xs write our .pm and .xs files for us.
508 Create a new directory called Mytest2 at the same level as the
510 directory Mytest. In the Mytest2 directory, create another directory
511 called mylib, and cd into that directory.
512 Here we'll create some files that will generate a test library. These
514 will include a C source file and a header file. We'll also create a
515 Makefile.PL in this directory. Then we'll make sure that running make
516 at the Mytest2 level will automatically run this Makefile.PL file and
517 the resulting Makefile.
518 In the mylib directory, create a file mylib.h that looks like this:
520 #define TESTVAL 4
522 extern double foo(int, long, const char*);
524 Also create a file mylib.c that looks like this:
526 #include <stdlib.h>
528 #include "./mylib.h"
529 double
531 foo(int a, long b, const char *c)
532 {
533 return (a + b + atof(c) + TESTVAL);
534 }
535 And finally create a file Makefile.PL that looks like this:
537 use ExtUtils::MakeMaker;
539 $Verbose = 1;
540 WriteMakefile(
541 NAME => 'Mytest2::mylib',
542 SKIP => [qw(all static static_lib dynamic dynamic_lib)],
543 clean => {'FILES' => 'libmylib$(LIB_EXT)'},
544 );
545 sub MY::top_targets {
548 '
549 all :: static
550 pure_all :: static
552 static :: libmylib$(LIB_EXT)
554 libmylib$(LIB_EXT): $(O_FILES)
556 $(AR) cr libmylib$(LIB_EXT) $(O_FILES)
557 $(RANLIB) libmylib$(LIB_EXT)
558 ';
560 }
561 Make sure you use a tab and not spaces on the lines beginning with
563 "$(AR)" and "$(RANLIB)". Make will not function properly if you use
564 spaces. It has also been reported that the "cr" argument to $(AR) is
565 unnecessary on Win32 systems.
566 We will now create the main top-level Mytest2 files. Change to the
568 directory above Mytest2 and run the following command:
569 % h2xs -O -n Mytest2 ./Mytest2/mylib/mylib.h
571 This will print out a warning about overwriting Mytest2, but that's
573 okay. Our files are stored in Mytest2/mylib, and will be untouched.
574 The normal Makefile.PL that h2xs generates doesn't know about the mylib
576 directory. We need to tell it that there is a subdirectory and that we
577 will be generating a library in it. Let's add the argument MYEXTLIB to
578 the WriteMakefile call so that it looks like this:
579 WriteMakefile(
581 'NAME' => 'Mytest2',
582 'VERSION_FROM' => 'Mytest2.pm', # finds $VERSION
583 'LIBS' => [''], # e.g., '-lm'
584 'DEFINE' => '', # e.g., '-DHAVE_SOMETHING'
585 'INC' => '', # e.g., '-I/usr/include/other'
586 'MYEXTLIB' => 'mylib/libmylib$(LIB_EXT)',
587 );
588 and then at the end add a subroutine (which will override the pre-
590 existing subroutine). Remember to use a tab character to indent the
591 line beginning with "cd"!
592 sub MY::postamble {
594 '
595 $(MYEXTLIB): mylib/Makefile
596 cd mylib && $(MAKE) $(PASSTHRU)
597 ';
598 }
599 Let's also fix the MANIFEST file so that it accurately reflects the
601 contents of our extension. The single line that says "mylib" should be
602 replaced by the following three lines:
603 mylib/Makefile.PL
605 mylib/mylib.c
606 mylib/mylib.h
607 To keep our namespace nice and unpolluted, edit the .pm file and change
609 the variable @EXPORT to @EXPORT_OK. Finally, in the .xs file, edit the
610 #include line to read:
611 #include "mylib/mylib.h"
613 And also add the following function definition to the end of the .xs
615 file:
616 double
618 foo(a,b,c)
619 int a
620 long b
621 const char * c
622 OUTPUT:
623 RETVAL
624 Now we also need to create a typemap because the default Perl doesn't
626 currently support the "const char *" type. Include a new TYPEMAP
627 section in your XS code before the above function:
628 TYPEMAP: <<END
630 const char * T_PV
631 END
632 Now run perl on the top-level Makefile.PL. Notice that it also created
634 a Makefile in the mylib directory. Run make and watch that it does cd
635 into the mylib directory and run make in there as well.
636 Now edit the Mytest2.t script and change the number of tests to "4",
638 and add the following lines to the end of the script:
639 is( &Mytest2::foo(1, 2, "Hello, world!"), 7 );
641 is( &Mytest2::foo(1, 2, "0.0"), 7 );
642 ok( abs(&Mytest2::foo(0, 0, "-3.4") - 0.6) <= 0.01 );
643 (When dealing with floating-point comparisons, it is best to not check
645 for equality, but rather that the difference between the expected and
646 actual result is below a certain amount (called epsilon) which is 0.01
647 in this case)
648 Run ""make test"" and all should be well. There are some warnings on
650 missing tests for the Mytest2::mylib extension, but you can ignore
651 them.
652 What has happened here?
654 Unlike previous examples, we've now run h2xs on a real include file.
655 This has caused some extra goodies to appear in both the .pm and .xs
656 files.
657 · In the .xs file, there's now a #include directive with the absolute
659 path to the mylib.h header file. We changed this to a relative
660 path so that we could move the extension directory if we wanted to.
661 · There's now some new C code that's been added to the .xs file. The
663 purpose of the "constant" routine is to make the values that are
664 #define'd in the header file accessible by the Perl script (by
665 calling either "TESTVAL" or &Mytest2::TESTVAL). There's also some
666 XS code to allow calls to the "constant" routine.
667 · The .pm file originally exported the name "TESTVAL" in the @EXPORT
669 array. This could lead to name clashes. A good rule of thumb is
670 that if the #define is only going to be used by the C routines
671 themselves, and not by the user, they should be removed from the
672 @EXPORT array. Alternately, if you don't mind using the "fully
673 qualified name" of a variable, you could move most or all of the
674 items from the @EXPORT array into the @EXPORT_OK array.
675 · If our include file had contained #include directives, these would
677 not have been processed by h2xs. There is no good solution to this
678 right now.
679 · We've also told Perl about the library that we built in the mylib
681 subdirectory. That required only the addition of the "MYEXTLIB"
682 variable to the WriteMakefile call and the replacement of the
683 postamble subroutine to cd into the subdirectory and run make. The
684 Makefile.PL for the library is a bit more complicated, but not
685 excessively so. Again we replaced the postamble subroutine to
686 insert our own code. This code simply specified that the library
687 to be created here was a static archive library (as opposed to a
688 dynamically loadable library) and provided the commands to build
689 it.
690 Anatomy of .xs file
692 The .xs file of "EXAMPLE 4" contained some new elements. To understand
693 the meaning of these elements, pay attention to the line which reads
694 MODULE = Mytest2 PACKAGE = Mytest2
696 Anything before this line is plain C code which describes which headers
698 to include, and defines some convenience functions. No translations
699 are performed on this part, apart from having embedded POD
700 documentation skipped over (see perlpod) it goes into the generated
701 output C file as is.
702 Anything after this line is the description of XSUB functions. These
704 descriptions are translated by xsubpp into C code which implements
705 these functions using Perl calling conventions, and which makes these
706 functions visible from Perl interpreter.
707 Pay a special attention to the function "constant". This name appears
709 twice in the generated .xs file: once in the first part, as a static C
710 function, then another time in the second part, when an XSUB interface
711 to this static C function is defined.
712 This is quite typical for .xs files: usually the .xs file provides an
714 interface to an existing C function. Then this C function is defined
715 somewhere (either in an external library, or in the first part of .xs
716 file), and a Perl interface to this function (i.e. "Perl glue") is
717 described in the second part of .xs file. The situation in "EXAMPLE
718 1", "EXAMPLE 2", and "EXAMPLE 3", when all the work is done inside the
719 "Perl glue", is somewhat of an exception rather than the rule.
720 Getting the fat out of XSUBs
722 In "EXAMPLE 4" the second part of .xs file contained the following
723 description of an XSUB:
724 double
726 foo(a,b,c)
727 int a
728 long b
729 const char * c
730 OUTPUT:
731 RETVAL
732 Note that in contrast with "EXAMPLE 1", "EXAMPLE 2" and "EXAMPLE 3",
734 this description does not contain the actual code for what is done
735 during a call to Perl function foo(). To understand what is going on
736 here, one can add a CODE section to this XSUB:
737 double
739 foo(a,b,c)
740 int a
741 long b
742 const char * c
743 CODE:
744 RETVAL = foo(a,b,c);
745 OUTPUT:
746 RETVAL
747 However, these two XSUBs provide almost identical generated C code:
749 xsubpp compiler is smart enough to figure out the "CODE:" section from
750 the first two lines of the description of XSUB. What about "OUTPUT:"
751 section? In fact, that is absolutely the same! The "OUTPUT:" section
752 can be removed as well, as far as "CODE:" section or "PPCODE:" section
753 is not specified: xsubpp can see that it needs to generate a function
754 call section, and will autogenerate the OUTPUT section too. Thus one
755 can shortcut the XSUB to become:
756 double
758 foo(a,b,c)
759 int a
760 long b
761 const char * c
762 Can we do the same with an XSUB
764 int
766 is_even(input)
767 int input
768 CODE:
769 RETVAL = (input % 2 == 0);
770 OUTPUT:
771 RETVAL
772 of "EXAMPLE 2"? To do this, one needs to define a C function "int
774 is_even(int input)". As we saw in "Anatomy of .xs file", a proper
775 place for this definition is in the first part of .xs file. In fact a
776 C function
777 int
779 is_even(int arg)
780 {
781 return (arg % 2 == 0);
782 }
783 is probably overkill for this. Something as simple as a "#define" will
785 do too:
786 #define is_even(arg) ((arg) % 2 == 0)
788 After having this in the first part of .xs file, the "Perl glue" part
790 becomes as simple as
791 int
793 is_even(input)
794 int input
795 This technique of separation of the glue part from the workhorse part
797 has obvious tradeoffs: if you want to change a Perl interface, you need
798 to change two places in your code. However, it removes a lot of
799 clutter, and makes the workhorse part independent from idiosyncrasies
800 of Perl calling convention. (In fact, there is nothing Perl-specific
801 in the above description, a different version of xsubpp might have
802 translated this to TCL glue or Python glue as well.)
803 More about XSUB arguments
805 With the completion of Example 4, we now have an easy way to simulate
806 some real-life libraries whose interfaces may not be the cleanest in
807 the world. We shall now continue with a discussion of the arguments
808 passed to the xsubpp compiler.
809 When you specify arguments to routines in the .xs file, you are really
811 passing three pieces of information for each argument listed. The
812 first piece is the order of that argument relative to the others
813 (first, second, etc). The second is the type of argument, and consists
814 of the type declaration of the argument (e.g., int, char*, etc). The
815 third piece is the calling convention for the argument in the call to
816 the library function.
817 While Perl passes arguments to functions by reference, C passes
819 arguments by value; to implement a C function which modifies data of
820 one of the "arguments", the actual argument of this C function would be
821 a pointer to the data. Thus two C functions with declarations
822 int string_length(char *s);
824 int upper_case_char(char *cp);
825 may have completely different semantics: the first one may inspect an
827 array of chars pointed by s, and the second one may immediately
828 dereference "cp" and manipulate *cp only (using the return value as,
829 say, a success indicator). From Perl one would use these functions in
830 a completely different manner.
831 One conveys this info to xsubpp by replacing "*" before the argument by
833 "&". "&" means that the argument should be passed to a library
834 function by its address. The above two function may be XSUB-ified as
835 int
837 string_length(s)
838 char * s
839 int
841 upper_case_char(cp)
842 char &cp
843 For example, consider:
845 int
847 foo(a,b)
848 char &a
849 char * b
850 The first Perl argument to this function would be treated as a char and
852 assigned to the variable a, and its address would be passed into the
853 function foo. The second Perl argument would be treated as a string
854 pointer and assigned to the variable b. The value of b would be passed
855 into the function foo. The actual call to the function foo that xsubpp
856 generates would look like this:
857 foo(&a, b);
859 xsubpp will parse the following function argument lists identically:
861 char &a
863 char&a
864 char & a
865 However, to help ease understanding, it is suggested that you place a
867 "&" next to the variable name and away from the variable type), and
868 place a "*" near the variable type, but away from the variable name (as
869 in the call to foo above). By doing so, it is easy to understand
870 exactly what will be passed to the C function; it will be whatever is
871 in the "last column".
872 You should take great pains to try to pass the function the type of
874 variable it wants, when possible. It will save you a lot of trouble in
875 the long run.
876 The Argument Stack
878 If we look at any of the C code generated by any of the examples except
879 example 1, you will notice a number of references to ST(n), where n is
880 usually 0. "ST" is actually a macro that points to the n'th argument
881 on the argument stack. ST(0) is thus the first argument on the stack
882 and therefore the first argument passed to the XSUB, ST(1) is the
883 second argument, and so on.
884 When you list the arguments to the XSUB in the .xs file, that tells
886 xsubpp which argument corresponds to which of the argument stack (i.e.,
887 the first one listed is the first argument, and so on). You invite
888 disaster if you do not list them in the same order as the function
889 expects them.
890 The actual values on the argument stack are pointers to the values
892 passed in. When an argument is listed as being an OUTPUT value, its
893 corresponding value on the stack (i.e., ST(0) if it was the first
894 argument) is changed. You can verify this by looking at the C code
895 generated for Example 3. The code for the round() XSUB routine
896 contains lines that look like this:
897 double arg = (double)SvNV(ST(0));
899 /* Round the contents of the variable arg */
900 sv_setnv(ST(0), (double)arg);
901 The arg variable is initially set by taking the value from ST(0), then
903 is stored back into ST(0) at the end of the routine.
904 XSUBs are also allowed to return lists, not just scalars. This must be
906 done by manipulating stack values ST(0), ST(1), etc, in a subtly
907 different way. See perlxs for details.
908 XSUBs are also allowed to avoid automatic conversion of Perl function
910 arguments to C function arguments. See perlxs for details. Some
911 people prefer manual conversion by inspecting ST(i) even in the cases
912 when automatic conversion will do, arguing that this makes the logic of
913 an XSUB call clearer. Compare with "Getting the fat out of XSUBs" for
914 a similar tradeoff of a complete separation of "Perl glue" and
915 "workhorse" parts of an XSUB.
916 While experts may argue about these idioms, a novice to Perl guts may
918 prefer a way which is as little Perl-guts-specific as possible, meaning
919 automatic conversion and automatic call generation, as in "Getting the
920 fat out of XSUBs". This approach has the additional benefit of
921 protecting the XSUB writer from future changes to the Perl API.
922 Extending your Extension
924 Sometimes you might want to provide some extra methods or subroutines
925 to assist in making the interface between Perl and your extension
926 simpler or easier to understand. These routines should live in the .pm
927 file. Whether they are automatically loaded when the extension itself
928 is loaded or only loaded when called depends on where in the .pm file
929 the subroutine definition is placed. You can also consult AutoLoader
930 for an alternate way to store and load your extra subroutines.
931 Documenting your Extension
933 There is absolutely no excuse for not documenting your extension.
934 Documentation belongs in the .pm file. This file will be fed to
935 pod2man, and the embedded documentation will be converted to the
936 manpage format, then placed in the blib directory. It will be copied
937 to Perl's manpage directory when the extension is installed.
938 You may intersperse documentation and Perl code within the .pm file.
940 In fact, if you want to use method autoloading, you must do this, as
941 the comment inside the .pm file explains.
942 See perlpod for more information about the pod format.
944 Installing your Extension
946 Once your extension is complete and passes all its tests, installing it
947 is quite simple: you simply run "make install". You will either need
948 to have write permission into the directories where Perl is installed,
949 or ask your system administrator to run the make for you.
950 Alternately, you can specify the exact directory to place the
952 extension's files by placing a "PREFIX=/destination/directory" after
953 the make install (or in between the make and install if you have a
954 brain-dead version of make). This can be very useful if you are
955 building an extension that will eventually be distributed to multiple
956 systems. You can then just archive the files in the destination
957 directory and distribute them to your destination systems.
958 EXAMPLE 5
960 In this example, we'll do some more work with the argument stack. The
961 previous examples have all returned only a single value. We'll now
962 create an extension that returns an array.
963 This extension is very Unix-oriented (struct statfs and the statfs
965 system call). If you are not running on a Unix system, you can
966 substitute for statfs any other function that returns multiple values,
967 you can hard-code values to be returned to the caller (although this
968 will be a bit harder to test the error case), or you can simply not do
969 this example. If you change the XSUB, be sure to fix the test cases to
970 match the changes.
971 Return to the Mytest directory and add the following code to the end of
973 Mytest.xs:
974 void
976 statfs(path)
977 char * path
978 INIT:
979 int i;
980 struct statfs buf;
981 PPCODE:
983 i = statfs(path, &buf);
984 if (i == 0) {
985 XPUSHs(sv_2mortal(newSVnv(buf.f_bavail)));
986 XPUSHs(sv_2mortal(newSVnv(buf.f_bfree)));
987 XPUSHs(sv_2mortal(newSVnv(buf.f_blocks)));
988 XPUSHs(sv_2mortal(newSVnv(buf.f_bsize)));
989 XPUSHs(sv_2mortal(newSVnv(buf.f_ffree)));
990 XPUSHs(sv_2mortal(newSVnv(buf.f_files)));
991 XPUSHs(sv_2mortal(newSVnv(buf.f_type)));
992 } else {
993 XPUSHs(sv_2mortal(newSVnv(errno)));
994 }
995 You'll also need to add the following code to the top of the .xs file,
997 just after the include of "XSUB.h":
998 #include <sys/vfs.h>
1000 Also add the following code segment to Mytest.t while incrementing the
1002 "9" tests to "11":
1003 @a = &Mytest::statfs("/blech");
1005 ok( scalar(@a) == 1 && $a[0] == 2 );
1006 @a = &Mytest::statfs("/");
1007 is( scalar(@a), 7 );
1008 New Things in this Example
1010 This example added quite a few new concepts. We'll take them one at a
1011 time.
1012 · The INIT: directive contains code that will be placed immediately
1014 after the argument stack is decoded. C does not allow variable
1015 declarations at arbitrary locations inside a function, so this is
1016 usually the best way to declare local variables needed by the XSUB.
1017 (Alternatively, one could put the whole "PPCODE:" section into
1018 braces, and put these declarations on top.)
1019 · This routine also returns a different number of arguments depending
1021 on the success or failure of the call to statfs. If there is an
1022 error, the error number is returned as a single-element array. If
1023 the call is successful, then a 7-element array is returned. Since
1024 only one argument is passed into this function, we need room on the
1025 stack to hold the 7 values which may be returned.
1026 We do this by using the PPCODE: directive, rather than the CODE:
1028 directive. This tells xsubpp that we will be managing the return
1029 values that will be put on the argument stack by ourselves.
1030 · When we want to place values to be returned to the caller onto the
1032 stack, we use the series of macros that begin with "XPUSH". There
1033 are five different versions, for placing integers, unsigned
1034 integers, doubles, strings, and Perl scalars on the stack. In our
1035 example, we placed a Perl scalar onto the stack. (In fact this is
1036 the only macro which can be used to return multiple values.)
1037 The XPUSH* macros will automatically extend the return stack to
1039 prevent it from being overrun. You push values onto the stack in
1040 the order you want them seen by the calling program.
1041 · The values pushed onto the return stack of the XSUB are actually
1043 mortal SV's. They are made mortal so that once the values are
1044 copied by the calling program, the SV's that held the returned
1045 values can be deallocated. If they were not mortal, then they
1046 would continue to exist after the XSUB routine returned, but would
1047 not be accessible. This is a memory leak.
1048 · If we were interested in performance, not in code compactness, in
1050 the success branch we would not use "XPUSHs" macros, but "PUSHs"
1051 macros, and would pre-extend the stack before pushing the return
1052 values:
1053 EXTEND(SP, 7);
1055 The tradeoff is that one needs to calculate the number of return
1057 values in advance (though overextending the stack will not
1058 typically hurt anything but memory consumption).
1059 Similarly, in the failure branch we could use "PUSHs" without
1061 extending the stack: the Perl function reference comes to an XSUB
1062 on the stack, thus the stack is always large enough to take one
1063 return value.
1064 EXAMPLE 6
1066 In this example, we will accept a reference to an array as an input
1067 parameter, and return a reference to an array of hashes. This will
1068 demonstrate manipulation of complex Perl data types from an XSUB.
1069 This extension is somewhat contrived. It is based on the code in the
1071 previous example. It calls the statfs function multiple times,
1072 accepting a reference to an array of filenames as input, and returning
1073 a reference to an array of hashes containing the data for each of the
1074 filesystems.
1075 Return to the Mytest directory and add the following code to the end of
1077 Mytest.xs:
1078 SV *
1080 multi_statfs(paths)
1081 SV * paths
1082 INIT:
1083 AV * results;
1084 SSize_t numpaths = 0, n;
1085 int i;
1086 struct statfs buf;
1087 SvGETMAGIC(paths);
1089 if ((!SvROK(paths))
1090 || (SvTYPE(SvRV(paths)) != SVt_PVAV)
1091 || ((numpaths = av_top_index((AV *)SvRV(paths))) < 0))
1092 {
1093 XSRETURN_UNDEF;
1094 }
1095 results = (AV *)sv_2mortal((SV *)newAV());
1096 CODE:
1097 for (n = 0; n <= numpaths; n++) {
1098 HV * rh;
1099 STRLEN l;
1100 char * fn = SvPV(*av_fetch((AV *)SvRV(paths), n, 0), l);
1101 i = statfs(fn, &buf);
1103 if (i != 0) {
1104 av_push(results, newSVnv(errno));
1105 continue;
1106 }
1107 rh = (HV *)sv_2mortal((SV *)newHV());
1109 hv_store(rh, "f_bavail", 8, newSVnv(buf.f_bavail), 0);
1111 hv_store(rh, "f_bfree", 7, newSVnv(buf.f_bfree), 0);
1112 hv_store(rh, "f_blocks", 8, newSVnv(buf.f_blocks), 0);
1113 hv_store(rh, "f_bsize", 7, newSVnv(buf.f_bsize), 0);
1114 hv_store(rh, "f_ffree", 7, newSVnv(buf.f_ffree), 0);
1115 hv_store(rh, "f_files", 7, newSVnv(buf.f_files), 0);
1116 hv_store(rh, "f_type", 6, newSVnv(buf.f_type), 0);
1117 av_push(results, newRV_inc((SV *)rh));
1119 }
1120 RETVAL = newRV_inc((SV *)results);
1121 OUTPUT:
1122 RETVAL
1123 And add the following code to Mytest.t, while incrementing the "11"
1125 tests to "13":
1126 $results = Mytest::multi_statfs([ '/', '/blech' ]);
1128 ok( ref $results->[0] );
1129 ok( ! ref $results->[1] );
1130 New Things in this Example
1132 There are a number of new concepts introduced here, described below:
1133 · This function does not use a typemap. Instead, we declare it as
1135 accepting one SV* (scalar) parameter, and returning an SV* value,
1136 and we take care of populating these scalars within the code.
1137 Because we are only returning one value, we don't need a "PPCODE:"
1138 directive - instead, we use "CODE:" and "OUTPUT:" directives.
1139 · When dealing with references, it is important to handle them with
1141 caution. The "INIT:" block first calls SvGETMAGIC(paths), in case
1142 paths is a tied variable. Then it checks that "SvROK" returns
1143 true, which indicates that paths is a valid reference. (Simply
1144 checking "SvROK" won't trigger FETCH on a tied variable.) It then
1145 verifies that the object referenced by paths is an array, using
1146 "SvRV" to dereference paths, and "SvTYPE" to discover its type. As
1147 an added test, it checks that the array referenced by paths is non-
1148 empty, using the "av_top_index" function (which returns -1 if the
1149 array is empty). The XSRETURN_UNDEF macro is used to abort the XSUB
1150 and return the undefined value whenever all three of these
1151 conditions are not met.
1152 · We manipulate several arrays in this XSUB. Note that an array is
1154 represented internally by an AV* pointer. The functions and macros
1155 for manipulating arrays are similar to the functions in Perl:
1156 "av_top_index" returns the highest index in an AV*, much like
1157 $#array; "av_fetch" fetches a single scalar value from an array,
1158 given its index; "av_push" pushes a scalar value onto the end of
1159 the array, automatically extending the array as necessary.
1160 Specifically, we read pathnames one at a time from the input array,
1162 and store the results in an output array (results) in the same
1163 order. If statfs fails, the element pushed onto the return array
1164 is the value of errno after the failure. If statfs succeeds,
1165 though, the value pushed onto the return array is a reference to a
1166 hash containing some of the information in the statfs structure.
1167 As with the return stack, it would be possible (and a small
1169 performance win) to pre-extend the return array before pushing data
1170 into it, since we know how many elements we will return:
1171 av_extend(results, numpaths);
1173 · We are performing only one hash operation in this function, which
1175 is storing a new scalar under a key using "hv_store". A hash is
1176 represented by an HV* pointer. Like arrays, the functions for
1177 manipulating hashes from an XSUB mirror the functionality available
1178 from Perl. See perlguts and perlapi for details.
1179 · To create a reference, we use the "newRV_inc" function. Note that
1181 you can cast an AV* or an HV* to type SV* in this case (and many
1182 others). This allows you to take references to arrays, hashes and
1183 scalars with the same function. Conversely, the "SvRV" function
1184 always returns an SV*, which may need to be cast to the appropriate
1185 type if it is something other than a scalar (check with "SvTYPE").
1186 · At this point, xsubpp is doing very little work - the differences
1188 between Mytest.xs and Mytest.c are minimal.
1189 EXAMPLE 7 (Coming Soon)
1191 XPUSH args AND set RETVAL AND assign return value to array
1192 EXAMPLE 8 (Coming Soon)
1194 Setting $!
1195 EXAMPLE 9 Passing open files to XSes
1197 You would think passing files to an XS is difficult, with all the
1198 typeglobs and stuff. Well, it isn't.
1199 Suppose that for some strange reason we need a wrapper around the
1201 standard C library function "fputs()". This is all we need:
1202 #define PERLIO_NOT_STDIO 0
1204 #define PERL_NO_GET_CONTEXT
1205 #include "EXTERN.h"
1206 #include "perl.h"
1207 #include "XSUB.h"
1208 #include <stdio.h>
1210 int
1212 fputs(s, stream)
1213 char * s
1214 FILE * stream
1215 The real work is done in the standard typemap.
1217 But you lose all the fine stuff done by the perlio layers. This calls
1219 the stdio function "fputs()", which knows nothing about them.
1220 The standard typemap offers three variants of PerlIO *: "InputStream"
1222 (T_IN), "InOutStream" (T_INOUT) and "OutputStream" (T_OUT). A bare
1223 "PerlIO *" is considered a T_INOUT. If it matters in your code (see
1224 below for why it might) #define or typedef one of the specific names
1225 and use that as the argument or result type in your XS file.
1226 The standard typemap does not contain PerlIO * before perl 5.7, but it
1228 has the three stream variants. Using a PerlIO * directly is not
1229 backwards compatible unless you provide your own typemap.
1230 For streams coming from perl the main difference is that "OutputStream"
1232 will get the output PerlIO * - which may make a difference on a socket.
1233 Like in our example...
1234 For streams being handed to perl a new file handle is created (i.e. a
1236 reference to a new glob) and associated with the PerlIO * provided. If
1237 the read/write state of the PerlIO * is not correct then you may get
1238 errors or warnings from when the file handle is used. So if you opened
1239 the PerlIO * as "w" it should really be an "OutputStream" if open as
1240 "r" it should be an "InputStream".
1241 Now, suppose you want to use perlio layers in your XS. We'll use the
1243 perlio "PerlIO_puts()" function as an example.
1244 In the C part of the XS file (above the first MODULE line) you have
1246 #define OutputStream PerlIO *
1248 or
1249 typedef PerlIO * OutputStream;
1250 And this is the XS code:
1252 int
1254 perlioputs(s, stream)
1255 char * s
1256 OutputStream stream
1257 CODE:
1258 RETVAL = PerlIO_puts(stream, s);
1259 OUTPUT:
1260 RETVAL
1261 We have to use a "CODE" section because "PerlIO_puts()" has the
1263 arguments reversed compared to "fputs()", and we want to keep the
1264 arguments the same.
1265 Wanting to explore this thoroughly, we want to use the stdio "fputs()"
1267 on a PerlIO *. This means we have to ask the perlio system for a stdio
1268 "FILE *":
1269 int
1271 perliofputs(s, stream)
1272 char * s
1273 OutputStream stream
1274 PREINIT:
1275 FILE *fp = PerlIO_findFILE(stream);
1276 CODE:
1277 if (fp != (FILE*) 0) {
1278 RETVAL = fputs(s, fp);
1279 } else {
1280 RETVAL = -1;
1281 }
1282 OUTPUT:
1283 RETVAL
1284 Note: "PerlIO_findFILE()" will search the layers for a stdio layer. If
1286 it can't find one, it will call "PerlIO_exportFILE()" to generate a new
1287 stdio "FILE". Please only call "PerlIO_exportFILE()" if you want a new
1288 "FILE". It will generate one on each call and push a new stdio layer.
1289 So don't call it repeatedly on the same file. "PerlIO_findFILE()" will
1290 retrieve the stdio layer once it has been generated by
1291 "PerlIO_exportFILE()".
1292 This applies to the perlio system only. For versions before 5.7,
1294 "PerlIO_exportFILE()" is equivalent to "PerlIO_findFILE()".
1295 Troubleshooting these Examples
1297 As mentioned at the top of this document, if you are having problems
1298 with these example extensions, you might see if any of these help you.
1299 · In versions of 5.002 prior to the gamma version, the test script in
1301 Example 1 will not function properly. You need to change the "use
1302 lib" line to read:
1303 use lib './blib';
1305 · In versions of 5.002 prior to version 5.002b1h, the test.pl file
1307 was not automatically created by h2xs. This means that you cannot
1308 say "make test" to run the test script. You will need to add the
1309 following line before the "use extension" statement:
1310 use lib './blib';
1312 · In versions 5.000 and 5.001, instead of using the above line, you
1314 will need to use the following line:
1315 BEGIN { unshift(@INC, "./blib") }
1317 · This document assumes that the executable named "perl" is Perl
1319 version 5. Some systems may have installed Perl version 5 as
1320 "perl5".
1321
1323 For more information, consult perlguts, perlapi, perlxs, perlmod, and
1324 perlpod.
1325
1327 Jeff Okamoto <okamoto@corp.hp.com>
1328 Reviewed and assisted by Dean Roehrich, Ilya Zakharevich, Andreas
1330 Koenig, and Tim Bunce.
1331 PerlIO material contributed by Lupe Christoph, with some clarification
1333 by Nick Ing-Simmons.
1334 Changes for h2xs as of Perl 5.8.x by Renee Baecker
1336 Last Changed
1338 2012-01-20
1339perl v5.30.1 2020-01-30 perlxstut(3)