1PP(1) User Contributed Perl Documentation PP(1)
2
6 PDL::PP - Generate PDL routines from concise descriptions
7
9 # let PDL::PP tell you what it's doing
10 $::PP_VERBOSE = 1;
11 pp_def(
12 'sumover',
13 Pars => 'a(n); [o]b();',
14 Code => q{
15 double tmp=0;
16 loop(n) %{
17 tmp += $a();
18 %}
19 $b() = tmp;
20 },
21 );
22 pp_done();
24 # do not call exit() as some processing can be done in same process
25
27 Here is a quick reference list of the functions provided by PDL::PP.
28 pp_add_boot
30 Add code to the BOOT section of generated XS file
31 pp_add_exported
33 Add functions to the list of exported functions
34 pp_add_isa
36 Add entries to the @ISA list
37 pp_addbegin
39 Sets code to be added at the top of the generate .pm file
40 pp_addhdr
42 Add code and includes to C section of the generated XS file.
43 When used in a module that is "multi-C" (one .c file per "pp_def"ed
45 function), you need to bear in mind that as each one is generated, all
46 the "pp_addhdr" so far will be included. Therefore, if you add C
47 functions, make sure to make them "static" to avoid clashes with later
48 .c files. But a better practice is make them be separate C files, with
49 any necessary .h to be included by them and the .pd file. You can then
50 add them to your Makefile.PL (note this is the "_int" version, see
51 separate notes on how to "opt-in" for your own modules):
52 my @pack = (["pnm.pd", qw(Pnm PDL::IO::Pnm)]);
54 my %hash = pdlpp_stdargs_int(@pack);
55 $hash{OBJECT} .= ' get$(OBJ_EXT)';
56 sub MY::postamble { pdlpp_postamble_int(@pack); };
57 WriteMakefile(%hash);
58 pp_addpm
60 Add code to the generated .pm file
61 pp_addxs
63 Add extra XS code to the generated XS file
64 pp_add_macros
66 Add extra "$MACRO()" definitions for these functions. Note these
67 generate C code.
68 pp_add_macros(SUCC => sub { "($_[0] + 1)" });
70 # ...
71 Code => '$a() = $SUCC($b());',
72 pp_beginwrap
74 Add BEGIN-block wrapping to code for the generated .pm file
75 pp_bless
77 Sets the package to which the XS code is added (default is PDL)
78 pp_boundscheck
80 Control state of PDL bounds checking activity
81 pp_core_importList
83 Specify what is imported from PDL::Core
84 pp_def
86 Define a new PDL function
87 pp_deprecate_module
89 Add runtime and POD warnings about a module being deprecated
90 pp_done
92 Mark the end of PDL::PP definitions in the file
93 pp_export_nothing
95 Clear out the export list for your generated module
96 pp_line_numbers
98 Add line number information to simplify debugging of PDL::PP code
99
101 For an alternate introduction to PDL::PP, see Practical Magick with C,
102 PDL, and PDL::PP -- a guide to compiled add-ons for PDL
103 <https://arxiv.org/abs/1702.07753>.
104 Why do we need PP? Several reasons: firstly, we want to be able to
106 generate subroutine code for each of the PDL datatypes (PDL_Byte,
107 PDL_Short, etc). AUTOMATICALLY. Secondly, when referring to slices of
108 PDL arrays in Perl (e.g. "$x->slice('0:10:2,:')" or other things such
109 as transposes) it is nice to be able to do this transparently and to be
110 able to do this 'in-place' - i.e, not to have to make a memory copy of
111 the section. PP handles all the necessary element and offset arithmetic
112 for you. There are also the notions of broadcasting (repeated calling
113 of the same routine for multiple slices, see PDL::Indexing) and
114 dataflow (see PDL::Dataflow, and "DefaultFlow") which use of PP allows.
115 In much of what follows we will assume familiarity of the reader with
117 the concepts of implicit and explicit broadcasting and index
118 manipulations within PDL. If you have not yet heard of these concepts
119 or are not very comfortable with them it is time to check
120 PDL::Indexing.
121 As you may appreciate from its name PDL::PP is a Pre-Processor, i.e.
123 it expands code via substitutions to make real C-code. Technically, the
124 output is XS code (see perlxs) but that is very close to C.
125 So how do you use PP? Well for the most part you just write ordinary C
127 code except for special PP constructs which take the form:
128 $something(something else)
130 or:
132 PPfunction %{
134 <stuff>
135 %}
136 The most important PP construct is the form "$array()". Consider the
138 very simple PP function to sum the elements of a 1D vector (in fact
139 this is very similar to the actual code used by 'sumover'):
140 pp_def('sumit',
142 Pars => 'a(n); [o]b();',
143 Code => q{
144 double tmp;
145 tmp = 0;
146 loop(n) %{
147 tmp += $a();
148 %}
149 $b() = tmp;
150 }
151 );
152 What's going on? The "Pars =>" line is very important for PP - it
154 specifies all the arguments and their dimensionality. We call this the
155 signature of the PP function (compare also the explanations in
156 PDL::Indexing). In this case the routine takes a 1-D function as input
157 and returns a 0-D scalar as output. The "$a()" PP construct is used to
158 access elements of the array a(n) for you - PP fills in all the
159 required C code.
160 You will notice that we are using the "q{}" single-quote operator. This
162 is not an accident. You generally want to use single quotes to denote
163 your PP Code sections. PDL::PP uses "$var()" for its parsing and if you
164 don't use single quotes, Perl will try to interpolate "$var()". Also,
165 using the single quote "q" operator with curly braces makes it look
166 like you are creating a code block, which is What You Mean. (Perl is
167 smart enough to look for nested curly braces and not close the quote
168 until it finds the matching curly brace, so it's safe to have nested
169 blocks.) Under other circumstances, such as when you're stitching
170 together a Code block using string concatenations, it's often easiest
171 to use real single quotes as
172 Code => 'something'.$interpolatable.'somethingelse;'
174 In the simple case here where all elements are accessed the PP
176 construct "loop(n) %{ ... %}" is used to loop over all elements in
177 dimension "n". Note this feature of PP: ALL DIMENSIONS ARE SPECIFIED
178 BY NAME.
179 This is made clearer if we avoid the PP loop() construct and write the
181 loop explicitly using conventional C:
182 pp_def('sumit',
184 Pars => 'a(n); [o]b();',
185 Code => q{
186 PDL_Indx i,n_size;
187 double tmp;
188 n_size = $SIZE(n);
189 tmp = 0;
190 for(i=0; i<n_size; i++) {
191 tmp += $a(n=>i);
192 }
193 $b() = tmp;
194 },
195 );
196 which does the same as before, but is more long-winded. You can see to
198 get element "i" of a() we say "$a(n=>i)" - we are specifying the
199 dimension by name "n". In 2D we might say:
200 Pars=>'a(m,n);',
202 ...
203 tmp += $a(m=>i,n=>j);
204 ...
205 The syntax "m=>i" borrows from Perl hashes, which are in fact used in
207 the implementation of PP. One could also say "$a(n=>j,m=>i)" as order
208 is not important.
209 You can also see in the above example the use of another PP construct -
211 $SIZE(n) to get the length of the dimension "n".
212 It should, however, be noted that you shouldn't write an explicit
214 C-loop when you could have used the PP "loop" construct since PDL::PP
215 checks automatically the loop limits for you, usage of "loop" makes the
216 code more concise, etc. But there are certainly situations where you
217 need explicit control of the loop and now you know how to do it ;).
218 To revisit 'Why PP?' - the above code for sumit() will be generated for
220 each data-type. It will operate on slices of arrays 'in-place'. It will
221 broadcast automatically - e.g. if a 2D array is given it will be called
222 repeatedly for each 1D row (again check PDL::Indexing for the details
223 of broadcasting). And then b() will be a 1D array of sums of each row.
224 We could call it with $x->transpose to sum the columns instead. And
225 Dataflow tracing etc. will be available.
226 You can see PP saves the programmer from writing a lot of needlessly
228 repetitive C-code -- in our opinion this is one of the best features of
229 PDL making writing new C subroutines for PDL an amazingly concise
230 exercise. A second reason is the ability to make PP expand your concise
231 code definitions into different C code based on the needs of the
232 computer architecture in question. Imagine for example you are lucky to
233 have a supercomputer at your hands; in that case you want PDL::PP
234 certainly to generate code that takes advantage of the
235 vectorising/parallel computing features of your machine (this a project
236 for the future). In any case, the bottom line is that your unchanged
237 code should still expand to working XS code even if the internals of
238 PDL changed.
239 Also, because you are generating the code in an actual Perl script,
241 there are many fun things that you can do. Let's say that you need to
242 write both sumit (as above) and multit. With a little bit of
243 creativity, we can do
244 for({Name => 'sumit', Init => '0', Op => '+='},
246 {Name => 'multit', Init => '1', Op => '*='}) {
247 pp_def($_->{Name},
248 Pars => 'a(n); [o]b();',
249 Code => '
250 double tmp;
251 tmp = '.$_->{Init}.';
252 loop(n) %{
253 tmp '.$_->{Op}.' $a();
254 %}
255 $b() = tmp;
256 ');
257 }
258 which defines both the functions easily. Now, if you later need to
260 change the signature or dimensionality or whatever, you only need to
261 change one place in your code. Yeah, sure, your editor does have 'cut
262 and paste' and 'search and replace' but it's still less bothersome and
263 definitely more difficult to forget just one place and have strange
264 bugs creep in. Also, adding 'orit' (bitwise or) later is a one-liner.
265 And remember, you really have Perl's full abilities with you - you can
267 very easily read any input file and make routines from the information
268 in that file. For simple cases like the above, the author (Tjl)
269 currently favors the hash syntax like the above - it's not too much
270 more characters than the corresponding array syntax but much easier to
271 understand and change.
272 As of 2.064, the "Code" must not just "return", since the signature of
274 the generated functions has changed from returning "void" to returning
275 a "pdl_error", which is pre-initialised to a successful return value.
276 You can easily just replace the "return;" with "return PDL_err;", which
277 is the variable's name.
278 We should mention here also the ability to get the pointer to the
280 beginning of the data in memory - a prerequisite for interfacing PDL to
281 some libraries. This is handled with the "$P(var)" directive, see
282 below.
283 When starting work on a new pp_def'ined function, if you make a
285 mistake, you will usually find a pile of compiler errors indicating
286 line numbers in the generated XS file. If you know how to read XS files
287 (or if you want to learn the hard way), you could open the generated XS
288 file and search for the line number with the error. However, a recent
289 addition to PDL::PP helps report the correct line number of your
290 errors: "pp_line_numbers". Working with the original summit example, if
291 you had a mis-spelling of tmp in your code, you could change the
292 (erroneous) code to something like this and the compiler would give you
293 much more useful information:
294 pp_def('sumit',
296 Pars => 'a(n); [o]b();',
297 Code => pp_line_numbers(__LINE__, q{
298 double tmp;
299 tmp = 0;
300 loop(n) %{
301 tmp += $a();
302 %}
303 $b() = rmp;
304 })
305 );
306 For the above situation, my compiler tells me:
308 ...
310 test.pd:15: error: 'rmp' undeclared (first use in this function)
311 ...
312 In my example script (called test.pd), line 15 is exactly the line at
314 which I made my typo: "rmp" instead of "tmp".
315 So, after this quick overview of the general flavour of programming PDL
317 routines using PDL::PP let's summarise in which circumstances you
318 should actually use this preprocessor/precompiler. You should use
319 PDL::PP if you want to
320 • interface PDL to some external library
322 • write some algorithm that would be slow if coded in Perl (this is
324 not as often as you think; take a look at broadcasting and dataflow
325 first).
326 • be a PDL developer (and even then it's not obligatory)
328
330 Because of its architecture, PDL::PP can be both flexible and easy to
331 use on the one hand, yet exuberantly complicated at the same time.
332 Currently, part of the problem is that error messages are not very
333 informative and if something goes wrong, you'd better know what you are
334 doing and be able to hack your way through the internals (or be able to
335 figure out by trial and error what is wrong with your args to
336 "pp_def"). Although work is being done to produce better warnings, do
337 not be afraid to send your questions to the mailing list if you run
338 into trouble.
339
341 Now that you have some idea how to use "pp_def" to define new PDL
342 functions it is time to explain the general syntax of "pp_def".
343 "pp_def" takes as arguments first the name of the function you are
344 defining and then a hash list that can contain various keys.
345 Based on these keys PP generates XS code and a .pm file. The function
347 "pp_done" (see example in the SYNOPSIS) is used to tell PDL::PP that
348 there are no more definitions in this file and it is time to generate
349 the .xs and
350 .pm file.
351 As a consequence, there may be several pp_def() calls inside a file (by
353 convention files with PP code have the extension .pd or .pp) but
354 generally only one pp_done().
355 There are two main different types of usage of pp_def(), the 'data
357 operation' and 'slice operation' prototypes.
358 The 'data operation' is used to take some data, mangle it and output
360 some other data; this includes for example the '+' operation, matrix
361 inverse, sumover etc and all the examples we have talked about in this
362 document so far. Implicit and explicit broadcasting and the creation of
363 the result are taken care of automatically in those operations. You can
364 even do dataflow with "sumit", "sumover", etc (don't be dismayed if you
365 don't understand the concept of dataflow in PDL very well yet; it is
366 still very much experimental).
367 The 'slice operation' is a different kind of operation: in a slice
369 operation, you are not changing any data, you are defining
370 correspondences between different elements of two ndarrays (examples
371 include the index manipulation/slicing function definitions in the file
372 slices.pd that is part of the PDL distribution; but beware, this is not
373 introductory level stuff).
374 To support bad values, additional keys are required for "pp_def", as
376 explained below.
377 If you are just interested in communicating with some external library
379 (for example some linear algebra/matrix library), you'll usually want
380 the 'data operation' so we are going to discuss that first.
381
383 A simple example
384 In the data operation, you must know what dimensions of data you need.
385 First, an example with scalars:
386 pp_def('add',
388 Pars => 'a(); b(); [o]c();',
389 Code => '$c() = $a() + $b();'
390 );
391 That looks a little strange but let's dissect it. The first line is
393 easy: we're defining a routine with the name 'add'. The second line
394 simply declares our parameters and the parentheses mean that they are
395 scalars. We call the string that defines our parameters and their
396 dimensionality the signature of that function. For its relevance with
397 regard to broadcasting and index manipulations check the PDL::Indexing
398 man page.
399 The third line is the actual operation. You need to use the dollar
401 signs and parentheses to refer to your parameters (this will probably
402 change at some point in the future, once a good syntax is found).
403 These lines are all that is necessary to actually define the function
405 for PDL (well, actually it isn't; you additionally need to write a
406 Makefile.PL (see below) and build the module (something like 'perl
407 Makefile.PL; make'); but let's ignore that for the moment). So now you
408 can do
409 use MyModule;
411 $x = pdl 2,3,4;
412 $y = pdl 5;
413 $c = add($x,$y);
415 # or
416 add($x,$y,($c=null)); # Alternative form, useful if $c has been
417 # preset to something big, not useful here.
418 and have broadcasting work correctly (the result is $c == [7 8 9]).
420 The Pars section: the signature of a PP function
422 Seeing the above example code you will most probably ask: what is this
423 strange "$c=null" syntax in the second call to our new "add" function?
424 If you take another look at the definition of "add" you will notice
425 that the third argument "c" is flagged with the qualifier "[o]" which
426 tells PDL::PP that this is an output argument. So the above call to add
427 means 'create a new $c from scratch with correct dimensions' - "null"
428 is a special token for 'empty ndarray' (you might ask why we haven't
429 used the value "undef" to flag this instead of the PDL specific "null";
430 we are currently thinking about it ;).
431 [This should be explained in some other section of the manual as
433 well!!] The reason for having this syntax as an alternative is that if
434 you have really huge ndarrays, you can do
435 $c = PDL->null;
437 for(some long loop) {
438 # munge a,b
439 add($x,$y,$c);
440 # munge c, put something back to x,y
441 }
442 and avoid allocating and deallocating $c each time. It is allocated
444 once at the first add() and thereafter the memory stays until $c is
445 destroyed.
446 If you just say
448 $c = add($x,$y);
450 the code generated by PP will automatically fill in "$c=null" and
452 return the result. If you want to learn more about the reasons why
453 PDL::PP supports this style where output arguments are given as last
454 arguments check the PDL::Indexing man page.
455 "[o]" is not the only qualifier a pdl argument can have in the
457 signature. Another important qualifier is the "[t]" option which flags
458 a pdl as temporary. What does that mean? You tell PDL::PP that this
459 pdl is only used for temporary results in the course of the calculation
460 and you are not interested in its value after the computation has been
461 completed. But why should PDL::PP want to know about this in the first
462 place? The reason is closely related to the concepts of pdl auto
463 creation (you heard about that above) and implicit broadcasting. If you
464 use implicit broadcasting the dimensionality of automatically created
465 pdls is actually larger than that specified in the signature. With
466 "[o]" flagged pdls will be created so that they have the additional
467 dimensions as required by the number of implicit broadcast dimensions.
468 When creating a temporary pdl, however, it will always only be made big
469 enough so that it can hold the result for one iteration in a broadcast
470 loop, i.e. as large as required by the signature. So less memory is
471 wasted when you flag a pdl as temporary. Secondly, you can use output
472 auto creation with temporary pdls even when you are using explicit
473 broadcasting which is forbidden for normal output pdls flagged with
474 "[o]" (see PDL::Indexing).
475 As of 2.073, the user is unable to pass a "[t]" parameter, and PDL will
477 create and size it to its notional size, times the number of threads.
478 Here is an example where we use the "[t]" qualifier. We define the
480 function "callf" that calls a C routine "f" which needs a temporary
481 array of the same size and type as the array "a" (sorry about the
482 forward reference for $P; it's a pointer access, see below) :
483 pp_def('callf',
485 Pars => 'a(n); [t] tmp(n); [o] b()',
486 Code => 'PDL_Indx ns = $SIZE(n);
487 f($P(a),$P(b),$P(tmp),ns);
488 '
489 );
490 Another possible qualifier is "[phys]". If given, this means the pdl
492 will have "make_physical" in PDL::Core called on it.
493 Additionally, if it has a specified dimension "d" that has value 1, "d"
495 will not magically be grown if "d" is larger in another pdl with
496 specified dimension "d", and instead an exception will be thrown. E.g.:
497 pp_def('callf',
499 Pars => 'a(n); [phys] b(n); [o] c()',
500 # ...
501 );
502 If "a" had lead dimension of 2 and "b" of 3, an exception will always
504 be thrown. However, if "b" has lead dimension of 1, it would be
505 silently repeated as if it were 2, if it were not a "phys" parameter.
506 Argument dimensions and the signature
508 Now we have just talked about dimensions of pdls and the signature. How
509 are they related? Let's say that we want to add a scalar + the index
510 number to a vector:
511 pp_def('add2',
513 Pars => 'a(n); b(); [o]c(n);',
514 Code => 'loop(n) %{
515 $c() = $a() + $b() + n;
516 %}'
517 );
518 There are several points to notice here: first, the "Pars" argument now
520 contains the n arguments to show that we have a single dimensions in a
521 and c. It is important to note that dimensions are actual entities that
522 are accessed by name so this declares a and c to have the same first
523 dimensions. In most PP definitions the size of named dimensions will be
524 set from the respective dimensions of non-output pdls (those with no
525 "[o]" flag) but sometimes you might want to set the size of a named
526 dimension explicitly through an integer parameter. See below in the
527 description of the "OtherPars" section how that works.
528 Constant argument dimensions in the signature
530 Suppose you want an output ndarray to be created automatically and you
531 know that on every call its dimension will have the same size (say 9)
532 regardless of the dimensions of the input ndarrays. In this case you
533 use the following syntax in the Pars section to specify the size of the
534 dimension:
535 ' [o] y(n=9); '
537 As expected, extra dimensions required by broadcasting will be created
539 if necessary. If you need to assign a named dimension according to a
540 more complicated formula (than a constant) you must use the
541 "RedoDimsCode" key described below.
542 Type conversions and the signature
544 The signature also determines the type conversions that will be
545 performed when a PP function is invoked. So what happens when we invoke
546 one of our previously defined functions with pdls of different type,
547 e.g.
548 add2($x,$y,($ret=null));
550 where $x is of type "PDL_Float" and $y of type "PDL_Short"? With the
552 signature as shown in the definition of "add2" above the datatype of
553 the operation (as determined at runtime) is that of the pdl with the
554 'highest' type (sequence is byte < short < ushort < long < float <
555 double). In the add2 example the datatype of the operation is float ($x
556 has that datatype). All pdl arguments are then type converted to that
557 datatype (they are not converted inplace but a copy with the right type
558 is created if a pdl argument doesn't have the type of the operation).
559 Null pdls don't contribute a type in the determination of the type of
560 the operation. However, they will be created with the datatype of the
561 operation; here, for example, $ret will be of type float. You should be
562 aware of these rules when calling PP functions with pdls of different
563 types to take the additional storage and runtime requirements into
564 account.
565 These type conversions are correct for most functions you normally
567 define with "pp_def". However, there are certain cases where slightly
568 modified type conversion behaviour is desired. For these cases
569 additional qualifiers in the signature can be used to specify the
570 desired properties with regard to type conversion. These qualifiers can
571 be combined with those we have encountered already (the creation
572 qualifiers "[o]" and "[t]"). Let's go through the list of qualifiers
573 that change type conversion behaviour.
574 The most important is the "indx" qualifier which comes in handy when a
576 pdl argument represents indices into another pdl. Let's take a look at
577 an example from "PDL::Ufunc":
578 pp_def('maximum_ind',
580 Pars => 'a(n); indx [o] b()',
581 Code => '$GENERIC() cur;
582 PDL_Indx curind;
583 loop(n) %{
584 if (!n || $a() > cur) {cur = $a(); curind = n;}
585 %}
586 $b() = curind;',
587 );
588 The function "maximum_ind" finds the index of the largest element of a
590 vector. If you look at the signature you notice that the output
591 argument "b" has been declared with the additional "indx" qualifier.
592 This has the following consequences for type conversions: regardless of
593 the type of the input pdl "a" the output pdl "b" will be of type
594 "PDL_Indx" which makes sense since "b" will represent an index into
595 "a".
596 Note that 'curind' is declared as type "PDL_Indx" and not "indx".
598 While most datatype declarations in the 'Pars' section use the same
599 name as the underlying C type, "indx" is a type which is sufficient to
600 handle PDL indexing operations. For 32-bit installs, it can be a
601 32-bit integer type. For 64-bit installs, it will be a 64-bit integer
602 type.
603 Furthermore, if you call the function with an existing output pdl "b"
605 its type will not influence the datatype of the operation (see above).
606 Hence, even if "a" is of a smaller type than "b" it will not be
607 converted to match the type of "b" but stays untouched, which saves
608 memory and CPU cycles and is the right thing to do when "b" represents
609 indices. Also note that you can use the 'indx' qualifier together with
610 other qualifiers (the "[o]" and "[t]" qualifiers). Order is significant
611 -- type qualifiers precede creation qualifiers ("[o]" and "[t]").
612 The above example also demonstrates typical usage of the "$GENERIC()"
614 macro. It expands to the current type in a so called generic loop.
615 What is a generic loop? As you already heard a PP function has a
616 runtime datatype as determined by the type of the pdl arguments it has
617 been invoked with. The PP generated XS code for this function
618 therefore contains a switch like "switch (type) {case PDL_Byte: ...
619 case PDL_Double: ...}" that selects a case based on the runtime
620 datatype of the function (it's called a type ``loop'' because there is
621 a loop in PP code that generates the cases). In any case your code is
622 inserted once for each PDL type into this switch statement. The
623 "$GENERIC()" macro just expands to the respective type in each copy of
624 your parsed code in this "switch" statement, e.g., in the "case
625 PDL_Byte" section "cur" will expand to "PDL_Byte" and so on for the
626 other case statements. I guess you realise that this is a useful macro
627 to hold values of pdls in some code.
628 There are a couple of other qualifiers with similar effects as "indx".
630 For your convenience there are the "float" and "double" qualifiers with
631 analogous consequences on type conversions as "indx". Let's assume you
632 have a very large array for which you want to compute row and column
633 sums with an equivalent of the "sumover" function. However, with the
634 normal definition of "sumover" you might run into problems when your
635 data is, e.g. of type short. A call like
636 sumover($large_pdl,($sums = null));
638 will result in $sums be of type short and is therefore prone to
640 overflow errors if $large_pdl is a very large array. On the other hand
641 calling
642 @dims = $large_pdl->dims; shift @dims;
644 sumover($large_pdl,($sums = zeroes(double,@dims)));
645 is not a good alternative either. Now we don't have overflow problems
647 with $sums but at the expense of a type conversion of $large_pdl to
648 double, something bad if this is really a large pdl. That's where
649 "double" comes in handy:
650 pp_def('sumoverd',
652 Pars => 'a(n); double [o] b()',
653 Code => 'double tmp=0;
654 loop(n) %{ tmp += a(); %}
655 $b() = tmp;',
656 );
657 This gets us around the type conversion and overflow problems. Again,
659 analogous to the "indx" qualifier "double" results in "b" always being
660 of type double regardless of the type of "a" without leading to a type
661 conversion of "a" as a side effect.
662 There is also a special type, "real". The others above are all actual
664 PDL/C datatypes, but "real" is a modifier; if the operation type is
665 real, it has no effect; if it is complex, then the parameter will be
666 the real version - so "cdouble" becomes "double", etc.
667 There is also the converse, "complex". If the operation is already
669 complex, there is no effect; if not, the output will be promoted to the
670 type's "complexversion" in PDL::Type, which defaults to "cfloat". Note
671 this is controlled both by the PDL::Types data, and the code in
672 PDL::PP. NB Because this outputs floating-point data, the inputs will
673 by definition be turned into such. Therefore, it only makes sense to
674 have floating-point "GenericTypes" inputs. If you want to default to
675 coercing inputs to "float", give that as the last "GenericTypes" as the
676 generated XS function defaults to the last-given one. Hence (with the
677 "PMCode" and "Doc" omitted):
678 pp_def('r2C',
680 GenericTypes=>[reverse qw(F D G C)], # last one is default so here = F
681 Pars => 'r(); complex [o]c()',
682 Code => '$c() = $r();'
683 );
684 Finally, there are the "type+" qualifiers where type is one of "int" or
686 "float". What shall that mean. Let's illustrate the "int+" qualifier
687 with the actual definition of sumover:
688 pp_def('sumover',
690 Pars => 'a(n); int+ [o] b()',
691 Code => '$GENERIC(b) tmp=0;
692 loop(n) %{ tmp += a(); %}
693 $b() = tmp;',
694 );
695 As we had already seen for the "int", "float" and "double" qualifiers,
697 a pdl marked with a "type+" qualifier does not influence the datatype
698 of the pdl operation. Its meaning is "make this pdl at least of type
699 "type" or higher, as required by the type of the operation". In the
700 sumover example this means that when you call the function with an "a"
701 of type PDL_Short the output pdl will be of type PDL_Long (just as
702 would have been the case with the "int" qualifier). This again tries to
703 avoid overflow problems when using small datatypes (e.g. byte images).
704 However, when the datatype of the operation is higher than the type
705 specified in the "type+" qualifier "b" will be created with the
706 datatype of the operation, e.g. when "a" is of type double then "b"
707 will be double as well. We hope you agree that this is sensible
708 behaviour for "sumover". It should be obvious how the "float+"
709 qualifier works by analogy. It may become necessary to be able to
710 specify a set of alternative types for the parameters. However, this
711 will probably not be implemented until someone comes up with a
712 reasonable use for it.
713 Note that we now had to specify the $GENERIC macro with the name of the
715 pdl to derive the type from that argument. Why is that? If you
716 carefully followed our explanations you will have realised that in some
717 cases "b" will have a different type than the type of the operation.
718 Calling the '$GENERIC' macro with "b" as argument makes sure that the
719 type will always the same as that of "b" in that part of the generic
720 loop.
721 This is about all there is to say about the "Pars" section in a
723 "pp_def" call. You should remember that this section defines the
724 signature of a PP defined function, you can use several options to
725 qualify certain arguments as output and temporary args and all
726 dimensions that you can later refer to in the "Code" section are
727 defined by name.
728 It is important that you understand the meaning of the signature since
730 in the latest PDL versions you can use it to define broadcasting
731 functions from within Perl, i.e. what we call Perl level broadcasting.
732 Please check PDL::Indexing for details.
733 The Code section
735 The "Code" section contains the actual XS code that will be in the
736 innermost part of a broadcast loop (if you don't know what a broadcast
737 loop is then you still haven't read PDL::Indexing; do it now ;) after
738 any PP macros (like $GENERIC) and PP functions have been expanded (like
739 the "loop" function we are going to explain next).
740 Let's quickly reiterate the "sumover" example:
742 pp_def('sumover',
744 Pars => 'a(n); int+ [o] b()',
745 Code => '$GENERIC(b) tmp=0;
746 loop(n) %{ tmp += a(); %}
747 $b() = tmp;',
748 );
749 The "loop" construct in the "Code" section also refers to the dimension
751 name so you don't need to specify any limits: the loop is correctly
752 sized and everything is done for you, again.
753 Next, there is the surprising fact that "$a()" and "$b()" do not
755 contain the index. This is not necessary because we're looping over n
756 and both variables know which dimensions they have so they
757 automatically know they're being looped over.
758 This feature comes in very handy in many places and makes for much
760 shorter code. Of course, there are times when you want to circumvent
761 this; here is a function which make a matrix symmetric and serves as an
762 example of how to code explicit looping:
763 pp_def('symm',
765 Pars => 'a(n,n); [o]c(n,n);',
766 Code => 'loop(n) %{
767 int n2;
768 for(n2=n; n2<$SIZE(n); n2++) {
769 $c(n0 => n, n1 => n2) =
770 $c(n0 => n2, n1 => n) =
771 $a(n0 => n, n1 => n2);
772 }
773 %}
774 '
775 );
776 Let's dissect what is happening. Firstly, what is this function
778 supposed to do? From its signature you see that it takes a 2D matrix
779 with equal numbers of columns and rows and outputs a matrix of the same
780 size. From a given input matrix $a it computes a symmetric output
781 matrix $c (symmetric in the matrix sense that A^T = A where ^T means
782 matrix transpose, or in PDL parlance $c == $c->transpose). It does this
783 by using only the values on and below the diagonal of $a. In the output
784 matrix $c all values on and below the diagonal are the same as those in
785 $a while those above the diagonal are a mirror image of those below the
786 diagonal (above and below are here interpreted in the way that PDL
787 prints 2D pdls). If this explanation still sounds a bit strange just go
788 ahead, make a little file into which you write this definition, build
789 the new PDL extension (see section on Makefiles for PP code) and try it
790 out with a couple of examples.
791 Having explained what the function is supposed to do there are a couple
793 of points worth noting from the syntactical point of view. First, we
794 get the size of the dimension named "n" again by using the $SIZE macro.
795 Second, there are suddenly these funny "n0" and "n1" index names in the
796 code though the signature defines only the dimension "n". Why this? The
797 reason becomes clear when you note that both the first and second
798 dimension of $a and $b are named "n" in the signature of "symm". This
799 tells PDL::PP that the first and second dimension of these arguments
800 should have the same size. Otherwise the generated function will raise
801 a runtime error. However, now in an access to $a and $c PDL::PP cannot
802 figure out which index "n" refers to any more just from the name of the
803 index. Therefore, the indices with equal dimension names get numbered
804 from left to right starting at 0, e.g. in the above example "n0" refers
805 to the first dimension of $a and $c, "n1" to the second and so on.
806 In all examples so far, we have only used the "Pars" and "Code" members
808 of the hash that was passed to "pp_def". There are certainly other keys
809 that are recognised by PDL::PP and we will hear about some of them in
810 the course of this document. Find a (non-exhaustive) list of keys in
811 Appendix A. A list of macros and PPfunctions (we have only encountered
812 some of those in the examples above yet) that are expanded in values of
813 the hash argument to "pp_def" is summarised in Appendix B.
814 At this point, it might be appropriate to mention that PDL::PP is not a
816 completely static, well designed set of routines (as Tuomas puts it:
817 "stop thinking of PP as a set of routines carved in stone") but rather
818 a collection of things that the PDL::PP author (Tuomas J. Lukka)
819 considered he would have to write often into his PDL extension
820 routines. PP tries to be expandable so that in the future, as new needs
821 arise, new common code can be abstracted back into it. If you want to
822 learn more on why you might want to change PDL::PP and how to do it
823 check the section on PDL::PP internals.
824 Handling bad values
826 There are several keys and macros used when writing code to handle bad
827 values. The first one is the "HandleBad" key:
828 HandleBad => 0
830 This flags a pp-routine as NOT handling bad values. If this routine
831 is sent ndarrays with their "badflag" set, then a warning message
832 is printed to STDOUT and the ndarrays are processed as if the value
833 used to represent bad values is a valid number. The "badflag" value
834 is not propagated to the output ndarrays.
835 An example of when this is used is for FFT routines, which
837 generally do not have a way of ignoring part of the data.
838 HandleBad => 1
840 This causes PDL::PP to write extra code that ensures the BadCode
841 section is used, and that the "$ISBAD()" macro (and its brethren)
842 work. If no "BadCode" is supplied, the "Code" section will be used,
843 on the assumption it will use "PDL_IF_BAD" to handle bad values.
844 HandleBad is not given
846 If any of the input ndarrays have their "badflag" set, then the
847 output ndarrays will have their "badflag" set, but any supplied
848 BadCode is ignored.
849 The value of "HandleBad" is used to define the contents of the "BadDoc"
851 key, if it is not given.
852 To handle bad values, code must be written somewhat differently; for
854 instance,
855 $c() = $a() + $b();
857 becomes something like
859 if ( $a() != BADVAL && $b() != BADVAL ) {
861 $c() = $a() + $b();
862 } else {
863 $c() = BADVAL;
864 }
865 However, we only want the second version if bad values are present in
867 the input ndarrays (and that bad-value support is wanted!) - otherwise
868 we actually want the original code. This is where the "BadCode" key
869 comes in; you use it to specify the code to execute if bad values may
870 be present, and PP uses both it and the "Code" section to create
871 something like:
872 if ( bad_values_are_present ) {
874 fancy_broadcastloop_stuff {
875 BadCode
876 }
877 } else {
878 fancy_broadcastloop_stuff {
879 Code
880 }
881 }
882 This approach means that there is virtually no overhead when bad values
884 are not present (i.e. the badflag routine returns 0).
885 The C preprocessor symbol "PDL_BAD_CODE" is defined when the bad code
887 is compiled, so that you can reduce the amount of code you write. The
888 BadCode section can use the same macros and looping constructs as the
889 Code section. As of 2.073, you can also use
890 "PDL_IF_BAD(iftrue,iffalse)".
891 Other bad-value macros
893 However, it wouldn't be much use without the following additional
894 macros:
895 $ISBAD(var)
897 To check whether an ndarray's value is bad, use the $ISBAD macro:
899 if ( $ISBAD(a()) ) { printf("a() is bad\n"); }
901 You can also access given elements of an ndarray:
903 if ( $ISBAD(a(n=>l)) ) { printf("element %d of a() is bad\n", l); }
905 $ISGOOD(var)
907 This is the opposite of the $ISBAD macro.
909 $SETBAD(var)
911 For when you want to set an element of an ndarray bad.
913 $ISBADVAR(c_var,pdl)
915 If you have cached the value of an ndarray "$a()" into a c-variable
917 ("foo" say), then to check whether it is bad, use "$ISBADVAR(foo,a)".
918 $ISGOODVAR(c_var,pdl)
920 As above, but this time checking that the cached value isn't bad.
922 $SETBADVAR(c_var,pdl)
924 To copy the bad value for an ndarray into a c variable, use
926 "$SETBADVAR(foo,a)".
927 TODO: mention the "FindBadStatusCode" and "CopyBadStatusCode" options
929 to "pp_def", as well as the "BadDoc" key.
930 TODO: mention "$PPISBAD()" etc macros.
932 PDL STATE macros
934 If you want access to the value of the badflag for a given ndarray, you
935 can use the PDL STATE macros:
936 $ISPDLSTATEBAD(pdl)
938 $ISPDLSTATEGOOD(pdl)
939 $SETPDLSTATEBAD(pdl)
940 $SETPDLSTATEGOOD(pdl)
941 Bad-value examples
943 Using these macros, the above code could be specified as:
944 Code => '$c() = $a() + $b();',
946 BadCode => '
947 if ( $ISBAD(a()) || $ISBAD(b()) ) {
948 $SETBAD(c());
949 } else {
950 $c() = $a() + $b();
951 }',
952 Since this is Perl, TMTOWTDI, so you could also write:
954 BadCode => '
956 if ( $ISGOOD(a()) && $ISGOOD(b()) ) {
957 $c() = $a() + $b();
958 } else {
959 $SETBAD(c());
960 }',
961 You can reduce code repetition using the C "PDL_BAD_CODE" macro,
963 supplying only the "Code" section:
964 Code => '
966 #ifdef PDL_BAD_CODE
967 if ( $ISGOOD(a()) && $ISGOOD(b()) ) {
968 #endif PDL_BAD_CODE
969 $c() = $a() + $b();
970 #ifdef PDL_BAD_CODE
971 } else {
972 $SETBAD(c());
973 }
974 #endif PDL_BAD_CODE
975 ',
976 As of 2.073, you can also use "PDL_IF_BAD(iftrue,iffalse)":
978 Code => '
980 PDL_IF_BAD(if ( $ISGOOD(a()) && $ISGOOD(b()) ) {,)
981 $c() = $a() + $b();
982 PDL_IF_BAD(} else $SETBAD(c());,)
983 ',
984 Interfacing your own/library functions using PP
986 Now, consider the following: you have your own C function (that may in
987 fact be part of some library you want to interface to PDL) which takes
988 as arguments two pointers to vectors of double:
989 void myfunc(int n,double *v1,double *v2);
991 The correct way of defining the PDL function is
993 pp_def('myfunc',
995 Pars => 'a(n); [o]b(n);',
996 GenericTypes => ['D'],
997 Code => 'myfunc($SIZE(n),$P(a),$P(b));'
998 );
999 The "$P("par")" syntax returns a pointer to the first element and the
1001 other elements are guaranteed to lie after that.
1002 Notice that here it is possible to make many mistakes. First, $SIZE(n)
1004 must be used instead of "n". Second, you shouldn't put any loops in
1005 this code. Third, here we encounter a new hash key recognised by
1006 PDL::PP : the "GenericTypes" declaration tells PDL::PP to ONLY GENERATE
1007 THE TYPELOOP FOP THE LIST OF TYPES SPECIFIED. In this case "double".
1008 This has two advantages. Firstly the size of the compiled code is
1009 reduced vastly, secondly if non-double arguments are passed to
1010 "myfunc()" PDL will automatically convert them to double before passing
1011 to the external C routine and convert them back afterwards.
1012 One can also use "Pars" to qualify the types of individual arguments.
1014 Thus one could also write this as:
1015 pp_def('myfunc',
1017 Pars => 'double a(n); double [o]b(n);',
1018 Code => 'myfunc($SIZE(n),$P(a),$P(b));'
1019 );
1020 The type specification in "Pars" exempts the argument from variation in
1022 the typeloop - rather it is automatically converted to and from the
1023 type specified. This is obviously useful in a more general example,
1024 e.g.:
1025 void myfunc(int n,float *v1,long *v2);
1027 pp_def('myfunc',
1029 Pars => 'float a(n); long [o]b(n);',
1030 GenericTypes => ['F'],
1031 Code => 'myfunc($SIZE(n),$P(a),$P(b));'
1032 );
1033 Note we still use "GenericTypes" to reduce the size of the type loop,
1035 obviously PP could in principle spot this and do it automatically
1036 though the code has yet to attain that level of sophistication!
1037 Finally note when types are converted automatically one MUST use the
1039 "[o]" qualifier for output variables or you hard-won changes will get
1040 optimised away by PP!
1041 If you interface a large library you can automate the interfacing even
1043 further. Perl can help you again(!) in doing this. In many libraries
1044 you have certain calling conventions. This can be exploited. In short,
1045 you can write a little parser (which is really not difficult in Perl)
1046 that then generates the calls to "pp_def" from parsed descriptions of
1047 the functions in that library. For an example, please check the Slatec
1048 interface in the "Lib" tree of the PDL distribution. If you want to
1049 check (during debugging) which calls to PP functions your Perl code
1050 generated a little helper package comes in handy which replaces the PP
1051 functions by identically named ones that dump their arguments to
1052 stdout.
1053 Just say
1055 perl -MPDL::PP::Dump myfile.pd
1057 to see the calls to "pp_def" and friends. Try it with ops.pd and
1059 slatec.pd. If you're interested (or want to enhance it), the source is
1060 in Basic/Gen/PP/Dump.pm
1061 Other macros in the Code section
1063 Macros: So far we have encountered the $SIZE, $GENERIC and $P macros.
1064 Now we are going to quickly explain the other macros that are expanded
1065 in the "Code" section of PDL::PP along with examples of their usage.
1066 $T
1068 The $T macro is used for type switches. This is very useful when you
1070 have to use different external (e.g. library) functions depending on
1071 the input type of arguments. The general syntax is
1072 $Ttypeletters(type_alternatives)
1074 where "typeletters" is a permutation of a subset of the letters
1076 "BSULNQFDGC" which stand for Byte, Short, Ushort, etc. and
1077 "type_alternatives" are the expansions when the type of the PP
1078 operation is equal to that indicated by the respective letter. Let's
1079 illustrate this incomprehensible description by an example. Assuming
1080 you have two C functions with prototypes
1081 void float_func(float *in, float *out);
1083 void double_func(double *in, double *out);
1084 which do basically the same thing but one accepts float and the other
1086 double pointers. You could interface them to PDL by defining a generic
1087 function "foofunc" (which will call the correct function depending on
1088 the type of the transformation):
1089 pp_def('foofunc',
1091 Pars => ' a(n); [o] b();',
1092 Code => ' $TFD(float,double)_func ($P(a),$P(b));'
1093 GenericTypes => [qw(F D)],
1094 );
1095 There is a limitation that the comma-separated values cannot have
1097 parentheses.
1098 $PP
1100 The $PP macro is used for a so called physical pointer access. The
1102 physical refers to some internal optimisations of PDL (for those who
1103 are familiar with the PDL core we are talking about the vaffine
1104 optimisations). This macro is mainly for internal use and you shouldn't
1105 need to use it in any of your normal code.
1106 $PPSYM
1108 The "$PPSYM()" macro is replaced by the value of "ppsym" in PDL::Types
1110 for the loop type, or that of the given parameter, similar to
1111 "$GENERIC()". This is useful for e.g. macros that vary by that,
1112 avoiding the need for things like "$TXY(X,Y)". Another benefit is that
1113 if an operation's GenericTypes get extended, this macro will still be
1114 correct.
1115 $COMP (and the OtherPars section)
1117 The $COMP macro is used to access non-pdl values in the code section.
1119 Its name is derived from the implementation of transformations in PDL.
1120 The variables you can refer to using $COMP are members of the
1121 ``compiled'' structure that represents the PDL transformation in
1122 question but does not yet contain any information about dimensions (for
1123 further details check PDL::Internals). However, you can treat $COMP
1124 just as a black box without knowing anything about the implementation
1125 of transformations in PDL. So when would you use this macro? Its main
1126 usage is to access values of arguments that are declared in the
1127 "OtherPars" section of a "pp_def" definition. But then you haven't
1128 heard about the "OtherPars" key yet?! Let's have another example that
1129 illustrates typical usage of both new features:
1130 pp_def('pnmout',
1132 Pars => 'a(m)',
1133 OtherPars => "PerlIO *fp",
1134 GenericTypes => [qw(B U S L)],
1135 Code => '
1136 if (PerlIO_write($COMP(fp),$P(a),len) != len)
1137 $CROAK("Error writing pnm file");
1138 ');
1139 This function is used to write data from a pdl to a file. The file
1141 descriptor is passed as a string into this function. This parameter
1142 does not go into the "Pars" section since it cannot be usefully treated
1143 like a pdl but rather into the aptly named "OtherPars" section.
1144 Parameters in the "OtherPars" section follow those in the "Pars"
1145 section when invoking the function, i.e.
1146 open FILE,">out.dat" or die "couldn't open out.dat";
1148 pnmout($pdl,'FILE');
1149 When you want to access this parameter inside the code section you have
1151 to tell PP by using the $COMP macro, i.e. you write "$COMP(fp)" as in
1152 the example. Otherwise PP wouldn't know that the "fp" you are referring
1153 to is the same as that specified in the "OtherPars" section.
1154 Another use for the "OtherPars" section is to set a named dimension in
1156 the signature. Let's have an example how that is done:
1157 pp_def('setdim',
1159 Pars => '[o] a(n)',
1160 OtherPars => 'int ns => n',
1161 Code => 'loop(n) %{ $a() = n; %}',
1162 );
1163 This says that the named dimension "n" will be initialised from the
1165 value of the other parameter "ns" which is of integer type (I guess you
1166 have realised that we use the "CType From => named_dim" syntax). Now
1167 you can call this function in the usual way:
1168 setdim(($x=null),5);
1170 print $x;
1171 [ 0 1 2 3 4 ]
1172 Admittedly this function is not very useful but it demonstrates how it
1174 works. If you call the function with an existing pdl and you don't need
1175 to explicitly specify the size of "n" since PDL::PP can figure it out
1176 from the dimensions of the non-null pdl. In that case you just give the
1177 dimension parameter as "-1":
1178 $x = hist($y);
1180 setdim($x,-1);
1181 The default values available via "$COMP()" are the "OtherPars" as noted
1183 above, which get copied in. However, this can be added to (previous to
1184 2.058, replaced) by supplying "Comp" and/or "MakeComp" keys (the
1185 defaults will happen first):
1186 pp_def(
1188 'diagonal',
1189 OtherPars => 'SV *list',
1190 Comp => 'PDL_Indx whichdims_count; PDL_Indx whichdims[$COMP(whichdims_count)];',
1191 MakeComp => '
1192 PDL_Indx i;
1193 PDL_Indx *tmp= PDL->packdims(list,&($COMP(whichdims_count)));
1194 if (!tmp) $CROAK("Failed to packdims for creating");
1195 if ($COMP(whichdims_count) < 1)
1196 $CROAK("Diagonal: must have at least 1 dimension");
1197 $DOCOMPALLOC(); /* malloc()s the whichdims */
1198 for(i=0; i<$COMP(whichdims_count); i++)
1199 $COMP(whichdims)[i] = tmp[i];
1200 free(tmp);
1201 /* ... */
1202 ',
1203 # ...
1204 );
1205 The "MakeComp" code is placed in the "pdl_(funcname)_run", so access to
1207 "Pars" (which will just be "pdl *"s)/"OtherPars" values is just via
1208 their names, not a macro.
1209 As of 2.058, you can instead give a C99 "incomplete array" type
1211 parameter as an "OtherPars" entry:
1212 pp_def(
1214 'diagonal',
1215 OtherPars => 'PDL_Indx whichdims[]',
1216 MakeComp => '
1217 if ($COMP(whichdims_count) < 1)
1218 $CROAK("Diagonal: must have at least 1 dimension");
1219 /* ... */
1220 ',
1221 # ...
1222 );
1223 There is an XS typemap entry (only for "PDL_Indx" array types for now)
1225 that adds a "(varname)_count" variable having extracted the index
1226 numbers from an array-ref parameter, and sets the count variable to the
1227 right value. PP then makes a copy of the data available. The C function
1228 (here, "pdl_diagonal_run")'s caller (here, the generated XS function)
1229 is responsible for freeing the array passed in (here, PDL's "smalloc"
1230 function is used, so the user need do nothing different).
1231 Other functions in the Code section
1233 The only PP function that we have used in the examples so far is
1234 "loop". Additionally, there are currently two other functions which
1235 are recognised in the "Code" section:
1236 broadcastloop
1238 As we heard above the signature of a PP defined function defines the
1240 dimensions of all the pdl arguments involved in a primitive operation.
1241 However, you often call the functions that you defined with PP with
1242 pdls that have more dimensions than those specified in the signature.
1243 In this case the primitive operation is performed on all subslices of
1244 appropriate dimensionality in what is called a broadcast loop (see also
1245 overview above and PDL::Indexing). Assuming you have some notion of
1246 this concept you will probably appreciate that the operation specified
1247 in the code section should be optimised since this is the tightest loop
1248 inside a broadcast loop. However, if you revisit the example where we
1249 define the "pnmout" function, you will quickly realise that looking up
1250 the "IO" file descriptor in the inner broadcast loop is not very
1251 efficient when writing a pdl with many rows. A better approach would be
1252 to look up the "IO" descriptor once outside the broadcast loop and use
1253 its value then inside the tightest broadcast loop. This is exactly
1254 where the "broadcastloop" function comes in handy. Here is an improved
1255 definition of "pnmout" which uses this function:
1256 pp_def('pnmout',
1258 Pars => 'a(m)',
1259 OtherPars => "PerlIO *fp",
1260 GenericTypes => [qw(B U S L)],
1261 Code => '
1262 int len;
1263 len = $SIZE(m) * sizeof($GENERIC());
1264 broadcastloop %{
1265 if (PerlIO_write($COMP(fp),$P(a),len) != len)
1266 $CROAK("Error writing pnm file");
1267 %}
1268 ');
1269 This works as follows. Normally the C code you write inside the "Code"
1271 section is placed inside a broadcast loop (i.e. PP generates the
1272 appropriate wrapping C code around it). However, when you explicitly
1273 use the "broadcastloop" function, PDL::PP recognises this and doesn't
1274 wrap your code with an additional broadcast loop. This has the effect
1275 that code you write outside the broadcast loop is only executed once
1276 per transformation and just the code with in the surrounding "%{ ...
1277 %}" pair is placed within the tightest broadcast loop. This also comes
1278 in handy when you want to perform a decision (or any other code,
1279 especially CPU intensive code) only once per thread, i.e.
1280 pp_addhdr('
1282 #define RAW 0
1283 #define ASCII 1
1284 ');
1285 pp_def('do_raworascii',
1286 Pars => 'a(); b(); [o]c()',
1287 OtherPars => 'int mode',
1288 Code => ' switch ($COMP(mode)) {
1289 case RAW:
1290 broadcastloop %{
1291 /* do raw stuff */
1292 %}
1293 break;
1294 case ASCII:
1295 broadcastloop %{
1296 /* do ASCII stuff */
1297 %}
1298 break;
1299 default:
1300 $CROAK("unknown mode");
1301 }'
1302 );
1303 types
1305 The types function works similar to the $T macro. However, with the
1307 "types" function the code in the following block (delimited by "%{" and
1308 "%}" as usual) is executed for all those cases in which the datatype of
1309 the operation is any of the types represented by the letters in the
1310 argument to "type", e.g.
1311 Code => '...
1313 types(BSUL) %{
1315 /* do integer type operation */
1316 %}
1317 types(FD) %{
1318 /* do floating point operation */
1319 %}
1320 ...'
1321 You are encouraged to use this idiom (from PDL::Math) in order to
1323 minimise effort needed to make your code work with new types:
1324 use PDL::Types qw(types);
1326 my @Rtypes = grep $_->real, types();
1327 my @Ctypes = grep !$_->real, types();
1328 # ...
1329 my $got_complex = PDL::Core::Dev::got_complex_version($name, 2);
1330 my $complex_bit = join "\n",
1331 map 'types('.$_->ppsym.') %{$'.$c.'() = c'.$name.$_->floatsuffix.'($'.$x.'(),$'.$y.'());%}',
1332 @Ctypes;
1333 my $real_bit = join "\n",
1334 map 'types('.$_->ppsym.') %{$'.$c.'() = '.$name.'($'.$x.'(),$'.$y.'());%}',
1335 @Rtypes;
1336 ($got_complex ? $complex_bit : '') . $real_bit;
1337 (although you should first check whether tgmath.h already has a type-
1339 generic version of the function you want to call, in which case the
1340 above becomes unnecessary).
1341 The RedoDimsCode Section
1343 The "RedoDimsCode" key is an optional key that is used to compute
1344 dimensions of ndarrays at runtime in case the standard rules for
1345 computing dimensions from the signature are not sufficient. The
1346 contents of the "RedoDimsCode" entry is interpreted in the same way
1347 that the Code section is interpreted-- i.e., PP macros are expanded and
1348 the result is interpreted as C code. The purpose of the code is to set
1349 the size of some dimensions that appear in the signature. Storage
1350 allocation and broadcastloops and so forth will be set up as if the
1351 computed dimension had appeared in the signature. In your code, you
1352 first compute the desired size of a named dimension in the signature
1353 according to your needs and then assign that value to it via the
1354 $SIZE() macro.
1355 As an example, consider the following situation. You are interfacing an
1357 external library routine that requires an temporary array for workspace
1358 to be passed as an argument. Two input data arrays that are passed are
1359 p(m) and x(n). The output data array is y(n). The routine requires a
1360 workspace array with a length of n+m*m, and you'd like the storage
1361 created automatically just like it would be for any ndarray flagged
1362 with [t] or [o]. What you'd like is to say something like
1363 pp_def( "myexternalfunc",
1365 Pars => " p(m); x(n); [o] y; [t] work(n+m*m); ", ...
1366 but that won't work, because PP can't interpret expressions with
1368 arithmetic in the signature. Instead you write
1369 pp_def(
1371 "myexternalfunc",
1372 Pars => ' p(m); x(n); [o] y(); [t] work(wn); ',
1373 RedoDimsCode => '$SIZE(wn) = $SIZE(n) + $SIZE(m) * $SIZE(m);',
1374 Code => '
1375 externalfunc( $P(p), $P(x), $SIZE(m), $SIZE(n), $P(work) );
1376 '
1377 );
1378 As of 2.075, you can use the dimensions of passed-in ndarrays as they
1380 are available when the "RedoDimsCode" is run. Before the code in the
1381 Code section is executed PP will create the proper storage for "work"
1382 if it does not exist. Note that you only took the first dimension of
1383 "p" and "x" because the user may have sent ndarrays with extra
1384 broadcasting dimensions. Of course, the temporary ndarray "work" (note
1385 the [t] flag) should not be given any broadcast dimensions anyway.
1386 You can also use "RedoDimsCode" to set the dimension of a ndarray
1388 flagged with [o]. In this case you set the dimensions for the named
1389 dimension in the signature using $SIZE() as in the preceding example.
1390 However, because the ndarray is flagged with [o] instead of [t],
1391 broadcasting dimensions will be added if required just as if the size
1392 of the dimension were computed from the signature according to the
1393 usual rules. Here is an example from PDL::Math
1394 pp_def("polyroots",
1396 Pars => 'cr(n); ci(n); [o]rr(m); [o]ri(m);',
1397 RedoDimsCode => '$SIZE(m) = $SIZE(n)-1;',
1398 The input ndarrays are the real and imaginary parts of complex
1400 coefficients of a polynomial. The output ndarrays are real and
1401 imaginary parts of the roots. There are "n" roots to an "n"th order
1402 polynomial and such a polynomial has "n+1" coefficients (the zero-th
1403 through the "n"th). In this example, broadcasting will work correctly.
1404 That is, the first dimension of the output ndarray with have its
1405 dimension adjusted, but other broadcasting dimensions will be assigned
1406 just as if there were no "RedoDimsCode".
1407 RedoDims passed directly
1409 A "RedoDimsCode" value as above gets processed, including expanding
1411 macros, and adding type-generic loops. For very specific purposes, you
1412 may not want this processing done to your dimension-updating code,
1413 probably in "slice"-like functions.
1414 Then, instead of passing a "RedoDimsCode" value, you can pass a
1416 "RedoDims" value (which the "RedoDimsCode" would otherwise get
1417 processed into). Because you will probably want to access the ndarrays,
1418 the following macros are provided. They are named assuming you will
1419 have the first parameter as "PARENT" and the second as "CHILD", which
1420 is the case if you passed a true "P2Child" value, which you will
1421 basically always want to do for this scenario.
1422 $CHILD_PTR()
1424 $PARENT_PTR()
1425 These expand to a "pdl *" pointer, e.g. to pass to a PDL API
1426 function.
1427 $CHILD()
1429 $PARENT()
1430 These expand to a lookup into "pdl" struct, e.g. "$CHILD(ndims)"
1431 looks up the "ndims" value for the child parameter.
1432 RedoDims generated from EquivPDimExpr and EquivDimCheck
1434 Another way to generate the "RedoDims" code is to supply a
1436 "EquivPDimExpr" and maybe a "EquivDimCheck":
1437 pp_def(
1439 'xchg',
1440 OtherPars => 'PDL_Indx n1; PDL_Indx n2;',
1441 TwoWay => 1,
1442 P2Child => 1,
1443 AffinePriv => 1,
1444 EquivDimCheck => '
1445 if ($COMP(n1) <0) $COMP(n1) += $PARENT(broadcastids[0]);
1446 if ($COMP(n2) <0) $COMP(n2) += $PARENT(broadcastids[0]);
1447 if (PDLMIN($COMP(n1),$COMP(n2)) <0 ||
1448 PDLMAX($COMP(n1),$COMP(n2)) >= $PARENT(broadcastids[0]))
1449 $CROAK("One of dims %d, %d out of range: should be 0<=dim<%d",
1450 $COMP(n1),$COMP(n2),$PARENT(broadcastids[0]));',
1451 EquivPDimExpr => '
1452 (($CDIM == $COMP(n1)) ? $COMP(n2) :
1453 ($CDIM == $COMP(n2)) ? $COMP(n1) :
1454 $CDIM)
1455 ',
1456 );
1457 "EquivPDimExpr" is evaluated within a loop, and the value of the
1459 relevant dimension is available using the macro $CDIM as shown above.
1460 Typemap handling in the OtherPars section
1462 The "OtherPars" section discussed above is very often absolutely
1463 crucial when you interface external libraries with PDL. However in many
1464 cases the external libraries either use derived types or pointers of
1465 various types.
1466 The standard way to handle this in Perl is to use a typemap file. This
1468 is discussed in some detail in perlxs in the standard Perl
1469 documentation. In PP the functionality is very similar, so you can
1470 create a typemap file in the directory where your PP file resides and
1471 when it is built it is automatically read in to figure out the
1472 appropriate translation between the C type and Perl's built-in type.
1473 For instance the "gsl_spline_init" function has the following C
1475 declaration:
1476 int gsl_spline_init(gsl_spline * spline,
1478 const double xa[], const double ya[], size_t size);
1479 Clearly the "xa" and "ya" arrays are candidates for being passed in as
1481 ndarrays and the "size" argument is just the length of these ndarrays
1482 so that can be handled by the "$SIZE()" macro in PP. Write an
1483 "OtherPars" declaration of the form
1484 OtherPars => 'gsl_spline *spl'
1486 and write a short typemap file which handles this type:
1488 TYPEMAP
1490 gsl_spline * T_PTR
1491 and use it in the code:
1493 pp_def('init_meat',
1495 Pars => 'double x(n); double y(n);',
1496 OtherPars => 'gsl_spline *spl',
1497 Code =>'gsl_spline_init,($COMP(spl),$P(x),$P(y),$SIZE(n)));'
1498 );
1499 where I have removed a macro wrapper call, but that would obscure the
1501 discussion.
1502 Other useful PP keys in data operation definitions
1504 You have already heard about the "OtherPars" key. Currently, there are
1505 not many other keys for a data operation that will be useful in normal
1506 (whatever that is) PP programming. In fact, it would be interesting to
1507 hear about a case where you think you need more than what is provided
1508 at the moment. Please speak up on one of the PDL mailing lists. Most
1509 other keys recognised by "pp_def" are only really useful for what we
1510 call slice operations (see also above).
1511 One thing that is strongly being planned is variable number of
1513 arguments, which will be a little tricky.
1514 An incomplete list of the available keys:
1516 Inplace
1518 Setting this key marks the routine as working inplace - ie the input
1520 and output ndarrays are the same. An example is "$x->inplace->sqrt()"
1521 (or "sqrt(inplace($x))").
1522 Inplace => 1
1524 Use when the routine is a unary function, such as "sqrt".
1525 Inplace => ['a']
1527 If there are more than one input ndarrays, specify the name of the
1528 one that can be changed inplace using an array reference.
1529 Inplace => ['a','b']
1531 If there are more than one output ndarray, specify the name of the
1532 input ndarray and output ndarray in a 2-element array reference.
1533 This probably isn't needed, but left in for completeness.
1534 If bad values are being used, care must be taken to ensure the
1536 propagation of the badflag when inplace is being used; consider this
1537 excerpt from Basic/Bad/bad.pd:
1538 pp_def('replacebad',HandleBad => 1,
1540 Pars => 'a(); [o]b();',
1541 OtherPars => 'double newval',
1542 Inplace => 1,
1543 CopyBadStatusCode =>
1544 '/* propagate badflag if inplace AND it has changed */
1545 if ( a == b && $ISPDLSTATEBAD(a) )
1546 PDL->propagate_badflag( b, 0 );
1547 /* always make sure the output is "good" */
1549 $SETPDLSTATEGOOD(b);
1550 ',
1551 ...
1552 Since this routine removes all bad values, the output ndarray had its
1554 bad flag cleared. If run inplace (so "a == b"), then we have to tell
1555 all the children of "a" that the bad flag has been cleared (to save
1556 time we make sure that we call "PDL->propagate_badflag" only if the
1557 input ndarray had its bad flag set).
1558 NOTE: one idea is that the documentation for the routine could be
1560 automatically flagged to indicate that it can be executed inplace, ie
1561 something similar to how "HandleBad" sets "BadDoc" if it's not supplied
1562 (it's not an ideal solution).
1563 FTypes
1565 # in slices.pd
1567 FTypes => {CHILD => '$COMP(totype)'},
1568 The value is a hash-ref mapping parameter-names to an expression giving
1570 an override of the type for that parameter. The example above shows the
1571 type being overridden to the "OtherPars" "totype".
1572 OtherParsDefaults
1574 OtherPars => 'int a; int b',
1576 OtherParsDefaults => { b => 0 },
1577 Allows specifying default values for "OtherPars". It is an error to
1579 specify a default for one that is before another that does not have a
1580 default.
1581 Other PDL::PP functions to support concise package definition
1583 So far, we have described the "pp_def" and "pp_done" functions. PDL::PP
1584 exports a few other functions to aid you in writing concise PDL
1585 extension package definitions.
1586 pp_addhdr
1588 Often when you interface library functions as in the above example you
1590 have to include additional C include files. Since the XS file is
1591 generated by PP we need some means to make PP insert the appropriate
1592 include directives in the right place into the generated XS file. To
1593 this end there is the "pp_addhdr" function. This is also the function
1594 to use when you want to define some C functions for internal use by
1595 some of the XS functions (which are mostly functions defined by
1596 "pp_def"). By including these functions here you make sure that
1597 PDL::PP inserts your code before the point where the actual XS module
1598 section begins and will therefore be left untouched by xsubpp (cf.
1599 perlxs and perlxstut man pages).
1600 A typical call would be
1602 pp_addhdr('
1604 #include <unistd.h> /* we need defs of XXXX */
1605 #include "libprotos.h" /* prototypes of library functions */
1606 #include "mylocaldecs.h" /* Local decs */
1607 static void do_the real_work(PDL_Byte * in, PDL_Byte * out, int n)
1609 {
1610 /* do some calculations with the data */
1611 }
1612 ');
1613 This ensures that all the constants and prototypes you need will be
1615 properly included and that you can use the internal functions defined
1616 here in the "pp_def"s, e.g.:
1617 pp_def('barfoo',
1619 Pars => ' a(n); [o] b(n)',
1620 GenericTypes => ['B'],
1621 Code => ' PDL_Indx ns = $SIZE(n);
1622 do_the_real_work($P(a),$P(b),ns);
1623 ',
1624 );
1625 pp_addpm
1627 In many cases the actual PP code (meaning the arguments to "pp_def"
1629 calls) is only part of the package you are currently implementing.
1630 Often there is additional Perl code and XS code you would normally have
1631 written into the pm and XS files which are now automatically generated
1632 by PP. So how to get this stuff into those dynamically generated files?
1633 Fortunately, there are a couple of functions, generally called
1634 "pp_addXXX" that assist you in doing this.
1635 Let's assume you have additional Perl code that should go into the
1637 generated pm-file. This is easily achieved with the "pp_addpm" command:
1638 pp_addpm(<<'EOD');
1640 =head1 NAME
1642 PDL::Lib::Mylib -- a PDL interface to the Mylib library
1644 =head1 DESCRIPTION
1646 This package implements an interface to the Mylib package with full
1648 broadcasting and indexing support (see L<PDL::Indexing>).
1649 =cut
1651 use PGPLOT;
1653 =head2 use_myfunc
1655 this function applies the myfunc operation to all the
1656 elements of the input pdl regardless of dimensions
1657 and returns the sum of the result
1658 =cut
1659 sub use_myfunc {
1661 my $pdl = shift;
1662 myfunc($pdl->clump(-1),($res=null));
1664 return $res->sum;
1666 }
1667 EOD
1669 pp_add_exported
1671 You have probably got the idea. In some cases you also want to export
1673 your additional functions. To avoid getting into trouble with PP which
1674 also messes around with the @EXPORT array you just tell PP to add your
1675 functions to the list of exported functions:
1676 pp_add_exported('use_myfunc gethynx');
1678 pp_add_isa
1680 The "pp_add_isa" command works like the the "pp_add_exported" function.
1682 The arguments to "pp_add_isa" are added the @ISA list, e.g.
1683 pp_add_isa(' Some::Other::Class ');
1685 pp_bless
1687 If your pp_def routines are to be used as object methods use "pp_bless"
1689 to specify the package (i.e. class) to which your pp_defed methods will
1690 be added. For example, "pp_bless('PDL::MyClass')". The default is "PDL"
1691 if this is omitted.
1692 pp_addxs
1694 Sometimes you want to add extra XS code of your own (that is generally
1696 not involved with any broadcasting/indexing issues but supplies some
1697 other functionality you want to access from the Perl side) to the
1698 generated XS file, for example
1699 pp_addxs('','
1701 # Determine endianness of machine
1703 int
1705 isbigendian()
1706 CODE:
1707 unsigned short i;
1708 PDL_Byte *b;
1709 i = 42; b = (PDL_Byte*) (void*) &i;
1711 if (*b == 42)
1713 RETVAL = 0;
1714 else if (*(b+1) == 42)
1715 RETVAL = 1;
1716 else
1717 croak("Impossible - machine is neither big nor little endian!!\n");
1718 OUTPUT:
1719 RETVAL
1720 ');
1721 Especially "pp_add_exported" and "pp_addxs" should be used with care.
1723 PP uses PDL::Exporter, hence letting PP export your function means that
1724 they get added to the standard list of function exported by default
1725 (the list defined by the export tag ``:Func''). If you use "pp_addxs"
1726 you shouldn't try to do anything that involves broadcasting or indexing
1727 directly. PP is much better at generating the appropriate code from
1728 your definitions.
1729 pp_add_boot
1731 Finally, you may want to add some code to the BOOT section of the XS
1733 file (if you don't know what that is check perlxs). This is easily done
1734 with the "pp_add_boot" command:
1735 pp_add_boot(<<EOB);
1737 descrip = mylib_initialize(KEEP_OPEN);
1738 if (descrip == NULL)
1740 croak("Can't initialize library");
1741 GlobalStruc->descrip = descrip;
1743 GlobalStruc->maxfiles = 200;
1744 EOB
1745 pp_export_nothing
1747 By default, PP.pm puts all subs defined using the pp_def function into
1749 the output .pm file's EXPORT list. This can create problems if you are
1750 creating a subclassed object where you don't want any methods exported.
1751 (i.e. the methods will only be called using the $object->method
1752 syntax).
1753 For these cases you can call pp_export_nothing() to clear out the
1755 export list. Example (At the end of the .pd file):
1756 pp_export_nothing();
1758 pp_done();
1759 pp_core_importList
1761 By default, PP.pm puts the 'use Core;' line into the output .pm file.
1763 This imports Core's exported names into the current namespace, which
1764 can create problems if you are over-riding one of Core's methods in the
1765 current file. You end up getting messages like "Warning: sub sumover
1766 redefined in file subclass.pm" when running the program.
1767 For these cases the pp_core_importList can be used to change what is
1769 imported from Core.pm. For example:
1770 pp_core_importList('()')
1772 This would result in
1774 use Core();
1776 being generated in the output .pm file. This would result in no names
1778 being imported from Core.pm. Similarly, calling
1779 pp_core_importList(' qw/ barf /')
1781 would result in
1783 use Core qw/ barf/;
1785 being generated in the output .pm file. This would result in just
1787 'barf' being imported from Core.pm.
1788 pp_setversion
1790 Simultaneously set the .pm and .xs files' versions, thus avoiding
1792 unnecessary version-skew between the two. To use this, simply do this
1793 in your .pd file, probably near the top:
1794 our $VERSION = '0.0.3';
1796 pp_setversion($VERSION);
1797 # Then, in your Makefile.PL:
1799 my @package = qw(FFTW3.pd FFTW3 PDL::FFTW3);
1800 my %descriptor = pdlpp_stdargs(\@package);
1801 $descriptor{VERSION_FROM} = 'FFTW3.pd'; # EUMM can parse the format above
1802 However, don't use this if you use Module::Build::PDL. See that
1804 module's documentation for details.
1805 pp_deprecate_module
1807 If a particular module is deemed obsolete, this function can be used to
1809 mark it as deprecated. This has the effect of emitting a warning when a
1810 user tries to "use" the module. The generated POD for this module also
1811 carries a deprecation notice. The replacement module can be passed as
1812 an argument like this:
1813 pp_deprecate_module( infavor => "PDL::NewNonDeprecatedModule" );
1815 Note that function affects only the runtime warning and the POD.
1817
1819 Let's say that you have a function in your module called PDL::foo that
1820 uses the PP function "bar_pp" to do the heavy lifting. But you don't
1821 want to advertise that "bar_pp" exists. To do this, you must move your
1822 PP function to the top of your module file, then call
1823 pp_export_nothing()
1825 to clear the "EXPORT" list. To ensure that no documentation (even the
1827 default PP docs) is generated, set
1828 Doc => undef
1830 and to prevent the function from being added to the symbol table, set
1832 PMFunc => ''
1834 in your pp_def declaration (see Image2D.pd for an example). This will
1836 effectively make your PP function "private." However, it is always
1837 accessible via PDL::bar_pp due to Perl's module design. But making it
1838 private will cause the user to go very far out of his or her way to use
1839 it, so he or she shoulders the consequences!
1840
1842 The slice operations require a much more intimate knowledge of PDL
1843 internals than the data operations. Furthermore, the complexity of the
1844 issues involved is considerably higher than that in the average data
1845 operation. Nevertheless, functions generated using the slice operations
1846 are at the heart of the index manipulation and dataflow capabilities of
1847 PDL. You can get started by reading the section on "P2Child".
1848 Also, there are a lot of dirty issues with virtual ndarrays and
1850 vaffines which we shall entirely skip here.
1851 Slices and bad values
1853 Slice operations need to be able to handle bad values. The easiest
1854 thing to do is look at Basic/Slices/slices.pd to see how this works.
1855 Along with "BadCode", there are also the "BadBackCode" and
1857 "BadRedoDimsCode" keys for "pp_def". However, any "EquivCPOffsCode"
1858 should not need changing, since any changes are absorbed into the
1859 definition of the "$EQUIVCPOFFS()" macro (i.e. it is handled
1860 automatically by PDL::PP).
1861
1863 For printing warning messages or aborting/dieing, you can call "warn"
1864 or "barf" from PP code. However, you should be aware that these calls
1865 have been redefined using C preprocessor macros to "PDL->barf" and
1866 "PDL->warn". These redefinitions are in place to keep you from
1867 inadvertently calling perl's "warn" or "barf" directly, which can cause
1868 segfaults during pthreading (i.e. processor multi-threading).
1869 PDL's own versions of "barf" and "warn" will queue-up warning or barf
1871 messages until after pthreading is completed, and then call the perl
1872 versions of these routines.
1873 See PDL::ParallelCPU for more information on pthreading.
1875 NB As of 2.064, it is highly recommended that you do not call "barf" at
1877 all in PP code, but instead use "$CROAK()". This will return a
1878 "pdl_error" which will transparently be used to throw the correct
1879 exception in Perl code, but can be handled suitably by non-Perl
1880 callers.
1881
1883 If you are going to generate a package from your PP file (typical file
1884 extensions are ".pd" or ".pp" for the files containing PP code) it is
1885 easiest and safest to leave generation of the appropriate commands to
1886 the Makefile. In the following we will outline the typical format of a
1887 Perl Makefile to automatically build and install your package from a
1888 description in a PP file. Most of the rules to build the xs, pm and
1889 other required files from the PP file are already predefined in the
1890 PDL::Core::Dev package. We just have to tell MakeMaker to use it.
1891 In most cases you can define your Makefile like
1893 use PDL::Core::Dev; # Pick up development utilities
1895 use ExtUtils::MakeMaker;
1896 $package = ["mylib.pd",Mylib,PDL::Lib::Mylib,'',1];
1898 %hash = pdlpp_stdargs($package);
1899 $hash{OBJECT} .= ' additional_Ccode$(OBJ_EXT) ';
1900 WriteMakefile(%hash);
1901 sub MY::postamble { pdlpp_postamble($package); }
1903 # additional_Ccode.c
1905 #include "pdl.h"
1906 void ppcp(PDL_Byte *dst, PDL_Byte *src, int len)
1907 {
1908 int i;
1909 for (i=0;i<len;i++) *dst++=*src++;
1910 }
1911 Here, the list in $package is: first: PP source file name, then the
1913 prefix for the produced files, the whole package name, the package to
1914 add XS functions to (empty string to use the same as the PP functions),
1915 and a boolean to dictate whether to have PDL generate a separate C file
1916 for each PP function (for faster compilation). The last feature is
1917 opt-in as you have to avoid duplicate symbols when linking the library
1918 (so separate out C functions into their own file). You can modify the
1919 hash in whatever way you like but it would be reasonable to stay within
1920 some limits so that your package will continue to work with later
1921 versions of PDL.
1922 To make life even easier PDL::Core::Dev defines the function
1924 "pdlpp_stdargs" that returns a hash with default values that can be
1925 passed (either directly or after appropriate modification) to a call to
1926 WriteMakefile. Currently, "pdlpp_stdargs" returns a hash where the
1927 keys are filled in as follows:
1928 (
1930 'NAME' => $mod,
1931 VERSION_FROM => $src,
1932 'TYPEMAPS' => [&PDL_TYPEMAP()],
1933 'OBJECT' => "$pref\$(OBJ_EXT)",
1934 PM => {"$pref.pm" => "\$(INST_LIBDIR)/$pref.pm"},
1935 MAN3PODS => {"$pref.pm" => "\$(INST_MAN3DIR)/$mod.\$(MAN3EXT)"},
1936 'INC' => &PDL_INCLUDE(),
1937 'LIBS' => [''],
1938 'clean' => {'FILES' => "$pref.xs $pref.pm $pref\$(OBJ_EXT)"},
1939 )
1940 Here, $src is the name of the source file with PP code, $pref the
1942 prefix for the generated .pm and .xs files and $mod the name of the
1943 extension module to generate.
1944 If your "VERSION_FROM" provides a version, PP will use that to set the
1946 "XS_VERSION". If you need to influence the value of that variable so
1947 that XSLoader etc don't reject the loaded dynamic library, you can use
1948 this workaround in a "pp_addpm" (the "BEGIN" is because the "bootstrap"
1949 happens at runtime, and your code appears after that call, but with a
1950 "BEGIN" it will take place beforehand):
1951 our $VERSION; BEGIN { $VERSION = '2.019106' };
1953 our $XS_VERSION; BEGIN { $XS_VERSION = $VERSION };
1954
1956 The internals of the current version consist of a large table which
1957 gives the rules according to which things are translated and the subs
1958 which implement these rules.
1959 Later on, it would be good to make the table modifiable by the user so
1961 that different things may be tried.
1962 [Meta comment: here will hopefully be more in the future; currently,
1964 your best bet will be to read the source code :-( or ask on the list
1965 (try the latter first) ]
1966
1968 As well as the above-mentioned "PDL_BAD_CODE" and
1969 "PDL_IF_BAD(iftrue,iffalse)", there are also
1970 "PDL_IF_GENTYPE_REAL(iftrue,iffalse)", "PDL_IF_GENTYPE_UNSIGNED", and
1971 "PDL_IF_GENTYPE_INTEGER":
1972 $b() = PDL_IF_GENTYPE_INTEGER(0,NAN);
1974
1976 Unless otherwise specified, the arguments are strings.
1977 Pars
1979 define the signature of your function
1981 OtherPars
1983 arguments which are not pdls. Default: nothing. This is a semi-colon
1985 separated list of arguments, e.g., "OtherPars=>'int k; double value;
1986 PerlIO *fp'". See $COMP(x) and also the same entry in Appendix B.
1987 Code
1989 the actual code that implements the functionality; several PP macros
1991 and PP functions are recognised in the string value
1992 HandleBad
1994 If set to 1, the routine is assumed to support bad values and the code
1996 in the BadCode key is used if bad values are present; it also sets
1997 things up so that the "$ISBAD()" etc macros can be used. If set to 0,
1998 cause the routine to print a warning if any of the input ndarrays have
1999 their bad flag set.
2000 BadCode
2002 Give the code to be used if bad values may be present in the input
2004 ndarrays. Only used if "HandleBad => 1". If "HandleBad" is true and
2005 "BadCode" is not supplied, the "Code" section will be reused, on the
2006 assumption it will use "#ifdef PDL_BAD_CODE" to handle bad values. As
2007 of 2.073, you can also use "PDL_IF_BAD(iftrue,iffalse)".
2008 GenericTypes
2010 An array reference. The array may contain any subset of the one-
2012 character strings given below, which specify which types your operation
2013 will accept. The meaning of each type is:
2014 B - signed byte (i.e. signed char)
2016 S - signed short (two-byte integer)
2017 U - unsigned short
2018 L - signed long (four-byte integer, int on 32 bit systems)
2019 N - signed integer for indexing ndarray elements (platform & Perl-dependent size)
2020 Q - signed long long (eight byte integer)
2021 F - float
2022 D - double
2023 G - complex float
2024 C - complex double
2025 This is very useful (and important!) when interfacing an external
2027 library. Default: [qw/B S U L N Q F D/]
2028 Inplace
2030 Mark a function as being able to work inplace.
2032 Inplace => 1 if Pars => 'a(); [o]b();'
2034 Inplace => ['a'] if Pars => 'a(); b(); [o]c();'
2035 Inplace => ['a','c'] if Pars => 'a(); b(); [o]c(); [o]d();'
2036 If bad values are being used, care must be taken to ensure the
2038 propagation of the badflag when inplace is being used; for instance see
2039 the code for "replacebad" in Basic/Bad/bad.pd.
2040 Doc
2042 Used to specify a documentation string in Pod format. See PDL::Doc for
2044 information on PDL documentation conventions. Note: in the special case
2045 where the PP 'Doc' string is one line this is implicitly used for the
2046 quick reference AND the documentation!
2047 If the Doc field is omitted PP will generate default documentation
2049 (after all it knows about the Signature).
2050 If you really want the function NOT to be documented in any way at this
2052 point (e.g. for an internal routine, or because you are doing it
2053 elsewhere in the code) explicitly specify "Doc=>undef".
2054 BadDoc
2056 Contains the text returned by the "badinfo" command (in "perldl") or
2058 the "-b" switch to the "pdldoc" shell script. In many cases, you will
2059 not need to specify this, since the information can be automatically
2060 created by PDL::PP. However, as befits computer-generated text, it's
2061 rather stilted; it may be much better to do it yourself!
2062 NoPthread
2064 Optional flag to indicate the PDL function should not use processor
2066 threads (i.e. pthreads or POSIX threads) to split up work across
2067 multiple CPU cores. This option is typically set to 1 if the underlying
2068 PDL function is not threadsafe. If this option isn't present, then the
2069 function is assumed to be threadsafe. This option only applies if PDL
2070 has been compiled with POSIX threads enabled.
2071 PMCode
2073 pp_def('funcname',
2075 Pars => 'a(); [o] b();',
2076 PMCode => 'sub PDL::funcname {
2077 return PDL::_funcname_int(@_) if @_ == 2; # output arg "b" supplied
2078 PDL::_funcname_int(@_, my $out = PDL->null);
2079 $out;
2080 }',
2081 # ...
2082 );
2083 PDL functions allow "[o]" ndarray arguments into which you want the
2085 output saved. This is handy because you can allocate an output ndarray
2086 once and reuse it many times; the alternative would be for PDL to
2087 create a new ndarray each time, which may waste compute cycles or, more
2088 likely, RAM.
2089 PDL functions check the number of arguments they are given, and call
2091 "croak" if given the wrong number. By default (with no "PMCode"
2092 supplied), any output arguments may be omitted, and PDL::PP provides
2093 code that can handle this by creating "null" objects, passing them to
2094 your code, then returning them on the stack.
2095 If you do supply "PMCode", the rest of PDL::PP assumes it will be a
2097 string that defines a Perl function with the function's name in the
2098 "pp_bless" package ("PDL" by default). As the example implies, the PP-
2099 generated function name will change from "<funcname>", to
2100 "_<funcname>_int". As also shown above, you will need to supply all
2101 ndarrays in the exact order specified in the signature: output ndarrays
2102 are not optional, and the PP-generated function will not return
2103 anything.
2104 PMFunc
2106 When pp_def generates functions, it typically defines them in the PDL
2108 package. Then, in the .pm file that it generates for your module, it
2109 typically adds a line that essentially copies that function into your
2110 current package's symbol table with code that looks like this:
2111 *func_name = \&PDL::func_name;
2113 It's a little bit smarter than that (it knows when to wrap that sort of
2115 thing in a BEGIN block, for example, and if you specified something
2116 different for pp_bless), but that's the gist of it. If you don't care
2117 to import the function into your current package's symbol table, you
2118 can specify
2119 PMFunc => '',
2121 PMFunc has no other side-effects, so you could use it to insert
2123 arbitrary Perl code into your module if you like. However, you should
2124 use pp_addpm if you want to add Perl code to your module.
2125 ReadDataFuncName
2127 Allows overriding the default function-name, for reading data
2129 transformed by this operation. Mainly used internally to set it to
2130 "NULL", in which case a default affine-orientated function will be
2131 called instead.
2132 WriteBackDataFuncName
2134 As above, but for writing transformed data from a child of this
2136 transformation back to the parent when "BackCode" is supplied.
2137 AffinePriv
2139 Flag to indicate this is an affine transformation whose "Priv"
2141 (contents of the "pdl_trans") contains data that will need allocating
2142 and freeing.
2143 GlobalNew
2145 If supplied, will prevent generation of an XS function, and assigns the
2147 generated C "run" function into the named slot in the "Core" struct.
2148 This is not used as of 2.058, and instead the relevant C functions are
2149 in pdlaffine.c.
2150 P2Child
2152 Forces "Pars" to be "PARENT" and "CHILD", the function's "GenericTypes"
2154 to be all of them, no "HaveBroadcasting" or "CallCopy", and turns on
2155 "DefaultFlow" (so do not supply any of those args). Intended for
2156 affine transformations with dataflow.
2157 DefaultFlow
2159 If true, sets in the "pdl_transvtable" (see PDL::Internals) the
2161 "iflags" such that the trans will start with dataflow both forwards and
2162 backwards.
2163 HaveBroadcasting
2165 Default true. If so, generate code implementing broadcasting (see
2167 PDL::Indexing).
2168 CallCopy
2170 For parameters that get created, normally the "PDL->initialize" will be
2172 used (or on a subclass). If this is true (which is the default for
2173 simple functions i.e. 2-arg with 0-dim signatures), instead the first
2174 argument's "copy" method will be used.
2175 TwoWay
2177 If true, sets in the "pdl_transvtable" (see PDL::Internals) the
2179 "iflags" such as to inform the trans's error checks connected to
2180 dataflow.
2181 Identity
2183 If true, sets "RedoDims" "EquivCPOffsCode" "HandleBad" "P2Child"
2185 "TwoWay" such that the function is a dataflowing identity
2186 transformation.
2187 BackCode
2189 For dataflowing functions, this value (which gets parsed) overrides the
2191 operation of that from children ndarrays to parents.
2192 BadBackCode
2194 Same but taking account of bad values.
2196 EquivCPOffsCode
2198 If supplied, allows concise control of copying to Child from Parent the
2200 data considered Equivalent at each given Offset (hence the name); the
2201 "Code" and "BackCode" will be generated from this.
2202 Example:
2204 pp_def(
2206 '_clump_int',
2207 OtherPars => 'int n',
2208 P2Child => 1,
2209 RedoDims => # omitted
2210 EquivCPOffsCode => '
2211 PDL_Indx i;
2212 for(i=0; i<$PDL(CHILD)->nvals; i++) $EQUIVCPOFFS(i,i);
2213 ',
2214 );
2215
2217 Macros
2218 $variablename_from_sig()
2219 access a pdl (by its name) that was specified in the signature
2221 $COMP(x)
2223 access a value in the private data structure of this transformation
2225 (mainly used to use an argument that is specified in the "OtherPars"
2226 section)
2227 $SIZE(n)
2229 replaced at runtime by the actual size of a named dimension (as
2231 specified in the signature)
2232 $GENERIC()
2234 replaced by the C type that is equal to the runtime type of the
2236 operation
2237 $P(a)
2239 a pointer to the data of the PDL named "a" in the signature. Useful for
2241 interfacing to C functions
2242 $PP(a)
2244 a physical pointer access to pdl "a"; mainly for internal use
2246 $TXYZ(AlternativeX,AlternativeY,AlternativeZ)
2248 expansion alternatives according to runtime type of operation, where
2250 XXX is some string that is matched by "/[BSULNQFD+]/".
2251 $PDL(a)
2253 return a pointer to the pdl data structure (pdl *) of ndarray "a"
2255 $ISBAD(a())
2257 returns true if the value stored in "a()" equals the bad value for this
2259 ndarray. Requires "HandleBad" being set to 1.
2260 $ISGOOD(a())
2262 returns true if the value stored in "a()" does not equal the bad value
2264 for this ndarray. Requires "HandleBad" being set to 1.
2265 $SETBAD(a())
2267 Sets "a()" to equal the bad value for this ndarray. Requires
2269 "HandleBad" being set to 1.
2270 $PRIV()
2272 To access fields in the "pdl_trans", eg "$PRIV(offs)".
2274 $CROAK()
2276 Returns a "pdl_error" with the supplied (var-args) message, adding the
2278 function name at the start, which will cause a "barf" within the Perl
2279 code. This is (as of 2.064) a change in PDL functions' API, so that
2280 callers can handle exceptions in their preferred way, which may not use
2281 Perl at all.
2282 $EQUIVCPOFFS()
2284 Copy from the "PARENT" parameter at the first given offset, to the
2286 "CHILD" parameter at the second given offset.
2287 $EQUIVCPTRUNC()
2289 Similar, but if the expression given as the third parameter is false,
2291 instead set the "CHILD"'s value to 0.
2292 $DOCOMPALLOC()
2294 Allocates memory for any "Comp" arrays, after their size has been
2296 determined, e.g. here after "$COMP(whichdims_count)" has been set:
2297 Comp => 'PDL_Indx whichdims[$COMP(whichdims_count)]',
2299 $DOPRIVALLOC()
2301 As above, except the key is "Priv"; because it is "Priv", this is only
2303 for entries in the "pdl_trans" itself, and almost certainly only for
2304 operations where "AffinePriv" is true.
2305 $SETNDIMS()
2307 For affine transformations (specifically, ones which set P2Child to
2309 true), set the child's "ndims" to the given value and allocate a
2310 suitably-sized array of dimension values.
2311 $SETDIMS()
2313 Similarly for affine transformations, after the above and then the
2315 actual dimension sizes are set, use this to resize the child ndarray to
2316 the right size.
2317 $SETDELTABROADCASTIDS()
2319 Similarly again, this sets the child's "nbroadcastids" to the same as
2321 the parent's, allocates space for the "broadcastids", then sets the
2322 child's ones to the same as the parent's plus the given value.
2323 To get a flavour of what "broadcastids" are for, in the normal way of
2325 things the first (0th) one in the parent is the highest dimension-
2326 number in it. See PDL::Indexing for more.
2327 functions
2329 "loop(DIMS) %{ ... %}"
2330 loop over named dimensions; limits are generated automatically by PP
2332 "broadcastloop %{ ... %}"
2334 enclose following code in a broadcast loop
2336 As of 2.075, "threadloop" is a deprecated alias for this.
2338 "types(TYPES) %{ ... %}"
2340 execute following code if type of operation is any of "TYPES"
2342
2344 A number of functions are imported when you "use PDL::PP". These
2345 include functions that control the generated C or XS code, functions
2346 that control the generated Perl code, and functions that manipulate the
2347 packages and symbol tables into which the code is created.
2348 Generating C and XS Code
2350 PDL::PP's main purpose is to make it easy for you to wrap the
2351 broadcasting engine around your own C code, but you can do some other
2352 things, too.
2353 pp_def
2355 Used to wrap the broadcasting engine around your C code. Virtually all
2357 of this document discusses the use of pp_def.
2358 pp_done
2360 Indicates you are done with PDL::PP and that it should generate its .xs
2362 and .pm files based upon the other pp_* functions that you have called.
2363 This function takes no arguments.
2364 pp_addxs
2366 This lets you add XS code to your .xs file. This is useful if you want
2368 to create Perl-accessible functions that invoke C code but cannot or
2369 should not invoke the broadcasting engine. XS is the standard means by
2370 which you wrap Perl-accessible C code. You can learn more at perlxs.
2371 pp_add_boot
2373 This function adds whatever string you pass to the XS BOOT section. The
2375 BOOT section is C code that gets called by Perl when your module is
2376 loaded and is useful for automatic initialization. You can learn more
2377 about XS and the BOOT section at perlxs.
2378 pp_addhdr
2380 Adds pure-C code to your XS file. XS files are structured such that
2382 pure C code must come before XS specifications. This allows you to
2383 specify such C code.
2384 pp_boundscheck
2386 PDL normally checks the bounds of your accesses before making them. You
2388 can turn that on or off at runtime by setting
2389 MyPackage::set_boundscheck. This function allows you to remove that
2390 runtime flexibility and never do bounds checking. It also returns the
2391 current boundschecking status if called without any argumens.
2392 NOTE: I have not found anything about bounds checking in other
2394 documentation. That needs to be addressed.
2395 Generating Perl Code
2397 Many functions imported when you use PDL::PP allow you to modify the
2398 contents of the generated .pm file. In addition to pp_def and pp_done,
2399 the role of these functions is primarily to add code to various parts
2400 of your generated .pm file.
2401 pp_addpm
2403 Adds Perl code to the generated .pm file. PDL::PP actually keeps track
2405 of three different sections of generated code: the Top, the Middle, and
2406 the Bottom. You can add Perl code to the Middle section using the one-
2407 argument form, where the argument is the Perl code you want to supply.
2408 In the two-argument form, the first argument is an anonymous hash with
2409 only one key that specifies where to put the second argument, which is
2410 the string that you want to add to the .pm file. The hash is one of
2411 these three:
2412 {At => 'Top'}
2414 {At => 'Middle'}
2415 {At => 'Bot'}
2416 For example:
2418 pp_addpm({At => 'Bot'}, <<POD);
2420 =head1 Some documentation
2422 I know I'm typing this in the middle of my file, but it'll go at
2424 the bottom.
2425 =cut
2427 POD
2429 Warning: If, in the middle of your .pd file, you put documentation
2431 meant for the bottom of your pod, you will thoroughly confuse CPAN. On
2432 the other hand, if in the middle of your .pd file, you add some Perl
2433 code destined for the bottom or top of your .pm file, you only have
2434 yourself to confuse. :-)
2435 pp_beginwrap
2437 Adds BEGIN-block wrapping. Certain declarations can be wrapped in BEGIN
2439 blocks, though the default behavior is to have no such wrapping.
2440 pp_addbegin
2442 Sets code to be added to the top of your .pm file, even above code that
2444 you specify with "pp_addpm({At => 'Top'}, ...)". Unlike pp_addpm,
2445 calling this overwrites whatever was there before. Generally, you
2446 probably shouldn't use it.
2447 Tracking Line Numbers
2449 When you get compile errors, either from your C-like code or your Perl
2450 code, it can help to make those errors back to the line numbers in the
2451 source file at which the error occurred.
2452 pp_line_numbers
2454 Takes a line number and a (usually long) string of code. The line
2456 number should indicate the line at which the quote begins. This is
2457 usually Perl's "__LINE__" literal, unless you are using heredocs, in
2458 which case it is "__LINE__ + 1". The returned string has #line
2459 directives interspersed to help the compiler report errors on the
2460 proper line.
2461 Modifying the Symbol Table and Export Behavior
2463 PDL::PP usually exports all functions generated using pp_def, and
2464 usually installs them into the PDL symbol table. However, you can
2465 modify this behavior with these functions.
2466 pp_bless
2468 Sets the package (symbol table) to which the XS code is added. The
2470 default is PDL, which is generally what you want. If you use the
2471 default blessing and you create a function myfunc, then you can do the
2472 following:
2473 $ndarray->myfunc(<args>);
2475 PDL::myfunc($ndarray, <args>);
2476 On the other hand, if you bless your functions into another package,
2478 you cannot invoke them as PDL methods, and must invoke them as:
2479 MyPackage::myfunc($ndarray, <args>);
2481 Of course, you could always use the PMFunc key to add your function to
2483 the PDL symbol table, but why do that?
2484 pp_add_isa
2486 Adds to the list of modules from which your module inherits. The
2488 default list is
2489 qw(PDL::Exporter DynaLoader)
2491 pp_core_importlist
2493 At the top of your generated .pm file is a line that looks like this:
2495 use PDL::Core;
2497 You can modify that by specifying a string to pp_core_importlist. For
2499 example,
2500 pp_core_importlist('::Blarg');
2502 will result in
2504 use PDL::Core::Blarg;
2506 You can use this, for example, to add a list of symbols to import from
2508 PDL::Core. For example:
2509 pp_core_importlist(" ':Internal'");
2511 will lead to the following use statement:
2513 use PDL::Core ':Internal';
2515 pp_setversion
2517 Sets your module's version. The version must be consistent between the
2519 .xs and the .pm file, and is used to ensure that your Perl's libraries
2520 do not suffer from version skew.
2521 pp_add_exported
2523 Adds to the export list whatever names you give it. Functions created
2525 using pp_def are automatically added to the list. This function is
2526 useful if you define any Perl functions using pp_addpm or pp_addxs that
2527 you want exported as well.
2528 pp_export_nothing
2530 This resets the list of exported symbols to nothing. This is probably
2532 better called "pp_export_clear", since you can add exported symbols
2533 after calling "pp_export_nothing". When called just before calling
2534 pp_done, this ensures that your module does not export anything, for
2535 example, if you only want programmers to use your functions as methods.
2536
2538 For the concepts of broadcasting and slicing check PDL::Indexing.
2539 PDL::Internals
2541 PDL::BadValues for information on bad values
2543 perlxs, perlxstut
2545 Practical Magick with C, PDL, and PDL::PP -- a guide to compiled add-
2547 ons for PDL <https://arxiv.org/abs/1702.07753>
2548
2550 Although PDL::PP is quite flexible and thoroughly used, there are
2551 surely bugs. First amongst them: this documentation needs a thorough
2552 revision.
2553
2555 Copyright(C) 1997 Tuomas J. Lukka (lukka@fas.harvard.edu), Karl
2556 Glaazebrook (kgb@aaocbn1.aao.GOV.AU) and Christian Soeller
2557 (c.soeller@auckland.ac.nz). All rights reserved. Documentation updates
2558 Copyright(C) 2011 David Mertens (dcmertens.perl@gmail.com). This
2559 documentation is licensed under the same terms as Perl itself.
2560perl v5.34.0 2022-02-28 PP(1)