1PP(1) User Contributed Perl Documentation PP(1)
2
6 PDL::PP - Generate PDL routines from concise descriptions
7
9 # let PDL::PP tell you what it's doing
10 $::PP_VERBOSE = 1;
11 pp_def(
12 'sumover',
13 Pars => 'a(n); [o]b();',
14 Code => q{
15 double tmp=0;
16 loop(n) %{
17 tmp += $a();
18 %}
19 $b() = tmp;
20 },
21 );
22 pp_done();
24 # do not call exit() as some processing can be done in same process
25
27 Here is a quick reference list of the functions provided by PDL::PP.
28 pp_add_boot
30 Add code to the BOOT section of generated XS file
31 pp_add_exported
33 Add functions to the list of exported functions
34 pp_add_isa
36 Add entries to the @ISA list
37 pp_addbegin
39 Sets code to be added at the top of the generate .pm file
40 pp_addhdr
42 Add code and includes to C section of the generated XS file.
43 When used in a module that is "multi-C" (one .c file per "pp_def"ed
45 function), you need to bear in mind that as each one is generated, all
46 the "pp_addhdr" so far will be included. Therefore, if you add C
47 functions, make sure to make them "static" to avoid clashes with later
48 .c files. But a better practice is make them be separate C files, with
49 any necessary .h to be included by them and the .pd file. You can then
50 add them to your Makefile.PL (note this is the "_int" version, see
51 separate notes on how to "opt-in" for your own modules):
52 my @pack = (["pnm.pd", qw(Pnm PDL::IO::Pnm)]);
54 my %hash = pdlpp_stdargs_int(@pack);
55 $hash{OBJECT} .= ' get$(OBJ_EXT)';
56 sub MY::postamble { pdlpp_postamble_int(@pack); };
57 WriteMakefile(%hash);
58 pp_addpm
60 Add code to the generated .pm file
61 pp_addxs
63 Add extra XS code to the generated XS file
64 pp_add_macros
66 Add extra $MACRO() definitions for these functions. Note these generate
67 C code. As of 2.080, they will be passed the list of arguments they
68 were called with, rather than a single string, split like the C pre-
69 processor on commas except if in "" or "()", with leading and trailing
70 whitespace removed.
71 pp_add_macros(SUCC => sub { "($_[0] + 1)" });
73 # ...
74 Code => '$a() = $SUCC($b());',
75 pp_beginwrap
77 Add BEGIN-block wrapping to code for the generated .pm file
78 pp_bless
80 Sets the package to which the XS code is added (default is PDL)
81 pp_boundscheck
83 Control state of PDL bounds checking activity
84 pp_core_importList
86 Specify what is imported from PDL::Core
87 pp_def
89 Define a new PDL function
90 pp_deprecate_module
92 Add runtime and POD warnings about a module being deprecated
93 pp_done
95 Mark the end of PDL::PP definitions in the file
96 pp_export_nothing
98 Clear out the export list for your generated module
99 pp_line_numbers
101 Add line number information to simplify debugging of PDL::PP code
102
104 For an alternate introduction to PDL::PP, see Practical Magick with C,
105 PDL, and PDL::PP -- a guide to compiled add-ons for PDL
106 <https://arxiv.org/abs/1702.07753>.
107 Why do we need PP? Several reasons: firstly, we want to be able to
109 generate subroutine code for each of the PDL datatypes (PDL_Byte,
110 PDL_Short, etc). AUTOMATICALLY. Secondly, when referring to slices of
111 PDL arrays in Perl (e.g. "$x->slice('0:10:2,:')" or other things such
112 as transposes) it is nice to be able to do this transparently and to be
113 able to do this 'in-place' - i.e, not to have to make a memory copy of
114 the section. PP handles all the necessary element and offset arithmetic
115 for you. There are also the notions of broadcasting (repeated calling
116 of the same routine for multiple slices, see PDL::Indexing) and
117 dataflow (see PDL::Dataflow, and "DefaultFlow") which use of PP allows.
118 In much of what follows we will assume familiarity of the reader with
120 the concepts of implicit and explicit broadcasting and index
121 manipulations within PDL. If you have not yet heard of these concepts
122 or are not very comfortable with them it is time to check
123 PDL::Indexing.
124 As you may appreciate from its name PDL::PP is a Pre-Processor, i.e.
126 it expands code via substitutions to make real C-code. Technically, the
127 output is XS code (see perlxs) but that is very close to C.
128 So how do you use PP? Well for the most part you just write ordinary C
130 code except for special PP constructs which take the form:
131 $something(something else)
133 or:
135 PPfunction %{
137 <stuff>
138 %}
139 The most important PP construct is the form $array(). Consider the very
141 simple PP function to sum the elements of a 1D vector (in fact this is
142 very similar to the actual code used by 'sumover'):
143 pp_def('sumit',
145 Pars => 'a(n); [o]b();',
146 Code => q{
147 double tmp;
148 tmp = 0;
149 loop(n) %{
150 tmp += $a();
151 %}
152 $b() = tmp;
153 }
154 );
155 What's going on? The "Pars =>" line is very important for PP - it
157 specifies all the arguments and their dimensionality. We call this the
158 signature of the PP function (compare also the explanations in
159 PDL::Indexing). In this case the routine takes a 1-D function as input
160 and returns a 0-D scalar as output. The $a() PP construct is used to
161 access elements of the array a(n) for you - PP fills in all the
162 required C code.
163 You will notice that we are using the "q{}" single-quote operator. This
165 is not an accident. You generally want to use single quotes to denote
166 your PP Code sections. PDL::PP uses $var() for its parsing and if you
167 don't use single quotes, Perl will try to interpolate $var(). Also,
168 using the single quote "q" operator with curly braces makes it look
169 like you are creating a code block, which is What You Mean. (Perl is
170 smart enough to look for nested curly braces and not close the quote
171 until it finds the matching curly brace, so it's safe to have nested
172 blocks.) Under other circumstances, such as when you're stitching
173 together a Code block using string concatenations, it's often easiest
174 to use real single quotes as
175 Code => 'something'.$interpolatable.'somethingelse;'
177 In the simple case here where all elements are accessed the PP
179 construct "loop(n) %{ ... %}" is used to loop over all elements in
180 dimension "n". Note this feature of PP: ALL DIMENSIONS ARE SPECIFIED
181 BY NAME.
182 This is made clearer if we avoid the PP loop() construct and write the
184 loop explicitly using conventional C:
185 pp_def('sumit',
187 Pars => 'a(n); [o]b();',
188 Code => q{
189 PDL_Indx i,n_size;
190 double tmp;
191 n_size = $SIZE(n);
192 tmp = 0;
193 for(i=0; i<n_size; i++) {
194 tmp += $a(n=>i);
195 }
196 $b() = tmp;
197 },
198 );
199 which does the same as before, but is more long-winded. You can see to
201 get element "i" of a() we say $a(n=>i) - we are specifying the
202 dimension by name "n". In 2D we might say:
203 Pars=>'a(m,n);',
205 ...
206 tmp += $a(m=>i,n=>j);
207 ...
208 The syntax "m=>i" borrows from Perl hashes, which are in fact used in
210 the implementation of PP. One could also say "$a(n=>j,m=>i)" as order
211 is not important.
212 You can also see in the above example the use of another PP construct -
214 $SIZE(n) to get the length of the dimension "n".
215 It should, however, be noted that you shouldn't write an explicit
217 C-loop when you could have used the PP "loop" construct since PDL::PP
218 checks automatically the loop limits for you, usage of "loop" makes the
219 code more concise, etc. But there are certainly situations where you
220 need explicit control of the loop and now you know how to do it ;).
221 To revisit 'Why PP?' - the above code for sumit() will be generated for
223 each data-type. It will operate on slices of arrays 'in-place'. It will
224 broadcast automatically - e.g. if a 2D array is given it will be called
225 repeatedly for each 1D row (again check PDL::Indexing for the details
226 of broadcasting). And then b() will be a 1D array of sums of each row.
227 We could call it with $x->transpose to sum the columns instead. And
228 Dataflow tracing etc. will be available.
229 You can see PP saves the programmer from writing a lot of needlessly
231 repetitive C-code -- in our opinion this is one of the best features of
232 PDL making writing new C subroutines for PDL an amazingly concise
233 exercise. A second reason is the ability to make PP expand your concise
234 code definitions into different C code based on the needs of the
235 computer architecture in question. Imagine for example you are lucky to
236 have a supercomputer at your hands; in that case you want PDL::PP
237 certainly to generate code that takes advantage of the
238 vectorising/parallel computing features of your machine (this a project
239 for the future). In any case, the bottom line is that your unchanged
240 code should still expand to working XS code even if the internals of
241 PDL changed.
242 Also, because you are generating the code in an actual Perl script,
244 there are many fun things that you can do. Let's say that you need to
245 write both sumit (as above) and multit. With a little bit of
246 creativity, we can do
247 for({Name => 'sumit', Init => '0', Op => '+='},
249 {Name => 'multit', Init => '1', Op => '*='}) {
250 pp_def($_->{Name},
251 Pars => 'a(n); [o]b();',
252 Code => '
253 double tmp;
254 tmp = '.$_->{Init}.';
255 loop(n) %{
256 tmp '.$_->{Op}.' $a();
257 %}
258 $b() = tmp;
259 ');
260 }
261 which defines both the functions easily. Now, if you later need to
263 change the signature or dimensionality or whatever, you only need to
264 change one place in your code. Yeah, sure, your editor does have 'cut
265 and paste' and 'search and replace' but it's still less bothersome and
266 definitely more difficult to forget just one place and have strange
267 bugs creep in. Also, adding 'orit' (bitwise or) later is a one-liner.
268 And remember, you really have Perl's full abilities with you - you can
270 very easily read any input file and make routines from the information
271 in that file. For simple cases like the above, the author (Tjl)
272 currently favors the hash syntax like the above - it's not too much
273 more characters than the corresponding array syntax but much easier to
274 understand and change.
275 As of 2.064, the "Code" must not just "return", since the signature of
277 the generated functions has changed from returning "void" to returning
278 a "pdl_error", which is pre-initialised to a successful return value.
279 You can easily just replace the "return;" with "return PDL_err;", which
280 is the variable's name.
281 We should mention here also the ability to get the pointer to the
283 beginning of the data in memory - a prerequisite for interfacing PDL to
284 some libraries. This is handled with the $P(var) directive, see below.
285 When starting work on a new pp_def'ined function, if you make a
287 mistake, you will usually find a pile of compiler errors indicating
288 line numbers in the generated XS file. If you know how to read XS files
289 (or if you want to learn the hard way), you could open the generated XS
290 file and search for the line number with the error. However, a recent
291 addition to PDL::PP helps report the correct line number of your
292 errors: "pp_line_numbers". Working with the original summit example, if
293 you had a mis-spelling of tmp in your code, you could change the
294 (erroneous) code to something like this and the compiler would give you
295 much more useful information:
296 pp_def('sumit',
298 Pars => 'a(n); [o]b();',
299 Code => pp_line_numbers(__LINE__, q{
300 double tmp;
301 tmp = 0;
302 loop(n) %{
303 tmp += $a();
304 %}
305 $b() = rmp;
306 })
307 );
308 For the above situation, my compiler tells me:
310 ...
312 test.pd:15: error: 'rmp' undeclared (first use in this function)
313 ...
314 In my example script (called test.pd), line 15 is exactly the line at
316 which I made my typo: "rmp" instead of "tmp".
317 So, after this quick overview of the general flavour of programming PDL
319 routines using PDL::PP let's summarise in which circumstances you
320 should actually use this preprocessor/precompiler. You should use
321 PDL::PP if you want to
322 • interface PDL to some external library
324 • write some algorithm that would be slow if coded in Perl (this is
326 not as often as you think; take a look at broadcasting and dataflow
327 first).
328 • be a PDL developer (and even then it's not obligatory)
330
332 Because of its architecture, PDL::PP can be both flexible and easy to
333 use on the one hand, yet exuberantly complicated at the same time.
334 Currently, part of the problem is that error messages are not very
335 informative and if something goes wrong, you'd better know what you are
336 doing and be able to hack your way through the internals (or be able to
337 figure out by trial and error what is wrong with your args to
338 "pp_def"). Although work is being done to produce better warnings, do
339 not be afraid to send your questions to the mailing list if you run
340 into trouble.
341
343 Now that you have some idea how to use "pp_def" to define new PDL
344 functions it is time to explain the general syntax of "pp_def".
345 "pp_def" takes as arguments first the name of the function you are
346 defining and then a hash list that can contain various keys.
347 Based on these keys PP generates XS code and a .pm file. The function
349 "pp_done" (see example in the SYNOPSIS) is used to tell PDL::PP that
350 there are no more definitions in this file and it is time to generate
351 the .xs and
352 .pm file.
353 As a consequence, there may be several pp_def() calls inside a file (by
355 convention files with PP code have the extension .pd or .pp) but
356 generally only one pp_done().
357 There are two main different types of usage of pp_def(), the 'data
359 operation' and 'slice operation' prototypes.
360 The 'data operation' is used to take some data, mangle it and output
362 some other data; this includes for example the '+' operation, matrix
363 inverse, sumover etc and all the examples we have talked about in this
364 document so far. Implicit and explicit broadcasting and the creation of
365 the result are taken care of automatically in those operations. You can
366 even do dataflow with "sumit", "sumover", etc (don't be dismayed if you
367 don't understand the concept of dataflow in PDL very well yet; it is
368 still very much experimental).
369 The 'slice operation' is a different kind of operation: in a slice
371 operation, you are not changing any data, you are defining
372 correspondences between different elements of two ndarrays (examples
373 include the index manipulation/slicing function definitions in the file
374 slices.pd that is part of the PDL distribution; but beware, this is not
375 introductory level stuff).
376 To support bad values, additional keys are required for "pp_def", as
378 explained below.
379 If you are just interested in communicating with some external library
381 (for example some linear algebra/matrix library), you'll usually want
382 the 'data operation' so we are going to discuss that first.
383
385 A simple example
386 In the data operation, you must know what dimensions of data you need.
387 First, an example with scalars:
388 pp_def('add',
390 Pars => 'a(); b(); [o]c();',
391 Code => '$c() = $a() + $b();'
392 );
393 That looks a little strange but let's dissect it. The first line is
395 easy: we're defining a routine with the name 'add'. The second line
396 simply declares our parameters and the parentheses mean that they are
397 scalars. We call the string that defines our parameters and their
398 dimensionality the signature of that function. For its relevance with
399 regard to broadcasting and index manipulations check the PDL::Indexing
400 man page.
401 The third line is the actual operation. You need to use the dollar
403 signs and parentheses to refer to your parameters (this will probably
404 change at some point in the future, once a good syntax is found).
405 These lines are all that is necessary to actually define the function
407 for PDL (well, actually it isn't; you additionally need to write a
408 Makefile.PL (see below) and build the module (something like 'perl
409 Makefile.PL; make'); but let's ignore that for the moment). So now you
410 can do
411 use MyModule;
413 $x = pdl 2,3,4;
414 $y = pdl 5;
415 $c = add($x,$y);
417 # or
418 add($x,$y,($c=null)); # Alternative form, useful if $c has been
419 # preset to something big, not useful here.
420 and have broadcasting work correctly (the result is $c == [7 8 9]).
422 The Pars section: the signature of a PP function
424 Seeing the above example code you will most probably ask: what is this
425 strange "$c=null" syntax in the second call to our new "add" function?
426 If you take another look at the definition of "add" you will notice
427 that the third argument "c" is flagged with the qualifier "[o]" which
428 tells PDL::PP that this is an output argument. So the above call to add
429 means 'create a new $c from scratch with correct dimensions' - "null"
430 is a special token for 'empty ndarray' (you might ask why we haven't
431 used the value "undef" to flag this instead of the PDL specific "null";
432 we are currently thinking about it ;).
433 [This should be explained in some other section of the manual as
435 well!!] The reason for having this syntax as an alternative is that if
436 you have really huge ndarrays, you can do
437 $c = PDL->null;
439 for(some long loop) {
440 # munge a,b
441 add($x,$y,$c);
442 # munge c, put something back to x,y
443 }
444 and avoid allocating and deallocating $c each time. It is allocated
446 once at the first add() and thereafter the memory stays until $c is
447 destroyed.
448 If you just say
450 $c = add($x,$y);
452 the code generated by PP will automatically fill in "$c=null" and
454 return the result. If you want to learn more about the reasons why
455 PDL::PP supports this style where output arguments are given as last
456 arguments check the PDL::Indexing man page.
457 "[o]" is not the only qualifier a pdl argument can have in the
459 signature. Another important qualifier is the "[t]" option which flags
460 a pdl as temporary. What does that mean? You tell PDL::PP that this
461 pdl is only used for temporary results in the course of the calculation
462 and you are not interested in its value after the computation has been
463 completed. But why should PDL::PP want to know about this in the first
464 place? The reason is closely related to the concepts of pdl auto
465 creation (you heard about that above) and implicit broadcasting. If you
466 use implicit broadcasting the dimensionality of automatically created
467 pdls is actually larger than that specified in the signature. With
468 "[o]" flagged pdls will be created so that they have the additional
469 dimensions as required by the number of implicit broadcast dimensions.
470 When creating a temporary pdl, however, it will always only be made big
471 enough so that it can hold the result for one iteration in a broadcast
472 loop, i.e. as large as required by the signature. So less memory is
473 wasted when you flag a pdl as temporary. Secondly, you can use output
474 auto creation with temporary pdls even when you are using explicit
475 broadcasting which is forbidden for normal output pdls flagged with
476 "[o]" (see PDL::Indexing).
477 As of 2.073, the user is unable to pass a "[t]" parameter, and PDL will
479 create and size it to its notional size, times the number of threads.
480 Here is an example where we use the "[t]" qualifier. We define the
482 function "callf" that calls a C routine "f" which needs a temporary
483 array of the same size and type as the array "a" (sorry about the
484 forward reference for $P; it's a pointer access, see below) :
485 pp_def('callf',
487 Pars => 'a(n); [t] tmp(n); [o] b()',
488 Code => 'PDL_Indx ns = $SIZE(n);
489 f($P(a),$P(b),$P(tmp),ns);
490 '
491 );
492 Another possible qualifier is "[phys]". If given, this means the pdl
494 will have "make_physical" in PDL::Core called on it.
495 Additionally, if it has a specified dimension "d" that has value 1, "d"
497 will not magically be grown if "d" is larger in another pdl with
498 specified dimension "d", and instead an exception will be thrown. E.g.:
499 pp_def('callf',
501 Pars => 'a(n); [phys] b(n); [o] c()',
502 # ...
503 );
504 If "a" had lead dimension of 2 and "b" of 3, an exception will always
506 be thrown. However, if "b" has lead dimension of 1, it would be
507 silently repeated as if it were 2, if it were not a "phys" parameter.
508 Argument dimensions and the signature
510 Now we have just talked about dimensions of pdls and the signature. How
511 are they related? Let's say that we want to add a scalar + the index
512 number to a vector:
513 pp_def('add2',
515 Pars => 'a(n); b(); [o]c(n);',
516 Code => 'loop(n) %{
517 $c() = $a() + $b() + n;
518 %}'
519 );
520 There are several points to notice here: first, the "Pars" argument now
522 contains the n arguments to show that we have a single dimensions in a
523 and c. It is important to note that dimensions are actual entities that
524 are accessed by name so this declares a and c to have the same first
525 dimensions. In most PP definitions the size of named dimensions will be
526 set from the respective dimensions of non-output pdls (those with no
527 "[o]" flag) but sometimes you might want to set the size of a named
528 dimension explicitly through an integer parameter. See below in the
529 description of the "OtherPars" section how that works.
530 Constant argument dimensions in the signature
532 Suppose you want an output ndarray to be created automatically and you
533 know that on every call its dimension will have the same size (say 9)
534 regardless of the dimensions of the input ndarrays. In this case you
535 use the following syntax in the Pars section to specify the size of the
536 dimension:
537 ' [o] y(n=9); '
539 As expected, extra dimensions required by broadcasting will be created
541 if necessary. If you need to assign a named dimension according to a
542 more complicated formula (than a constant) you must use the
543 "RedoDimsCode" key described below.
544 Type conversions and the signature
546 The signature also determines the type conversions that will be
547 performed when a PP function is invoked. So what happens when we invoke
548 one of our previously defined functions with pdls of different type,
549 e.g.
550 add2($x,$y,($ret=null));
552 where $x is of type "PDL_Float" and $y of type "PDL_Short"? With the
554 signature as shown in the definition of "add2" above the datatype of
555 the operation (as determined at runtime) is that of the pdl with the
556 'highest' type (sequence is byte < short < ushort < long < float <
557 double). In the add2 example the datatype of the operation is float ($x
558 has that datatype). All pdl arguments are then type converted to that
559 datatype (they are not converted inplace but a copy with the right type
560 is created if a pdl argument doesn't have the type of the operation).
561 Null pdls don't contribute a type in the determination of the type of
562 the operation. However, they will be created with the datatype of the
563 operation; here, for example, $ret will be of type float. You should be
564 aware of these rules when calling PP functions with pdls of different
565 types to take the additional storage and runtime requirements into
566 account.
567 These type conversions are correct for most functions you normally
569 define with "pp_def". However, there are certain cases where slightly
570 modified type conversion behaviour is desired. For these cases
571 additional qualifiers in the signature can be used to specify the
572 desired properties with regard to type conversion. These qualifiers can
573 be combined with those we have encountered already (the creation
574 qualifiers "[o]" and "[t]"). Let's go through the list of qualifiers
575 that change type conversion behaviour.
576 The most important is the "indx" qualifier which comes in handy when a
578 pdl argument represents indices into another pdl. Let's take a look at
579 an example from "PDL::Ufunc":
580 pp_def('maximum_ind',
582 Pars => 'a(n); indx [o] b()',
583 Code => '$GENERIC() cur;
584 PDL_Indx curind;
585 loop(n) %{
586 if (!n || $a() > cur) {cur = $a(); curind = n;}
587 %}
588 $b() = curind;',
589 );
590 The function "maximum_ind" finds the index of the largest element of a
592 vector. If you look at the signature you notice that the output
593 argument "b" has been declared with the additional "indx" qualifier.
594 This has the following consequences for type conversions: regardless of
595 the type of the input pdl "a" the output pdl "b" will be of type
596 "PDL_Indx" which makes sense since "b" will represent an index into
597 "a".
598 Note that 'curind' is declared as type "PDL_Indx" and not "indx".
600 While most datatype declarations in the 'Pars' section use the same
601 name as the underlying C type, "indx" is a type which is sufficient to
602 handle PDL indexing operations. For 32-bit installs, it can be a
603 32-bit integer type. For 64-bit installs, it will be a 64-bit integer
604 type.
605 Furthermore, if you call the function with an existing output pdl "b"
607 its type will not influence the datatype of the operation (see above).
608 Hence, even if "a" is of a smaller type than "b" it will not be
609 converted to match the type of "b" but stays untouched, which saves
610 memory and CPU cycles and is the right thing to do when "b" represents
611 indices. Also note that you can use the 'indx' qualifier together with
612 other qualifiers (the "[o]" and "[t]" qualifiers). Order is significant
613 -- type qualifiers precede creation qualifiers ("[o]" and "[t]").
614 The above example also demonstrates typical usage of the $GENERIC()
616 macro. It expands to the current type in a so called generic loop.
617 What is a generic loop? As you already heard a PP function has a
618 runtime datatype as determined by the type of the pdl arguments it has
619 been invoked with. The PP generated XS code for this function
620 therefore contains a switch like "switch (type) {case PDL_Byte: ...
621 case PDL_Double: ...}" that selects a case based on the runtime
622 datatype of the function (it's called a type ``loop'' because there is
623 a loop in PP code that generates the cases). In any case your code is
624 inserted once for each PDL type into this switch statement. The
625 $GENERIC() macro just expands to the respective type in each copy of
626 your parsed code in this "switch" statement, e.g., in the "case
627 PDL_Byte" section "cur" will expand to "PDL_Byte" and so on for the
628 other case statements. I guess you realise that this is a useful macro
629 to hold values of pdls in some code.
630 There are a couple of other qualifiers with similar effects as "indx".
632 For your convenience there are the "float" and "double" qualifiers with
633 analogous consequences on type conversions as "indx". Let's assume you
634 have a very large array for which you want to compute row and column
635 sums with an equivalent of the "sumover" function. However, with the
636 normal definition of "sumover" you might run into problems when your
637 data is, e.g. of type short. A call like
638 sumover($large_pdl,($sums = null));
640 will result in $sums be of type short and is therefore prone to
642 overflow errors if $large_pdl is a very large array. On the other hand
643 calling
644 @dims = $large_pdl->dims; shift @dims;
646 sumover($large_pdl,($sums = zeroes(double,@dims)));
647 is not a good alternative either. Now we don't have overflow problems
649 with $sums but at the expense of a type conversion of $large_pdl to
650 double, something bad if this is really a large pdl. That's where
651 "double" comes in handy:
652 pp_def('sumoverd',
654 Pars => 'a(n); double [o] b()',
655 Code => 'double tmp=0;
656 loop(n) %{ tmp += a(); %}
657 $b() = tmp;',
658 );
659 This gets us around the type conversion and overflow problems. Again,
661 analogous to the "indx" qualifier "double" results in "b" always being
662 of type double regardless of the type of "a" without leading to a type
663 conversion of "a" as a side effect.
664 There is also a special type, "real". The others above are all actual
666 PDL/C datatypes, but "real" is a modifier; if the operation type is
667 real, it has no effect; if it is complex, then the parameter will be
668 the real version - so "cdouble" becomes "double", etc.
669 There is also the converse, "complex". If the operation is already
671 complex, there is no effect; if not, the output will be promoted to the
672 type's "complexversion" in PDL::Type, which defaults to "cfloat". Note
673 this is controlled both by the PDL::Types data, and the code in
674 PDL::PP. NB Because this outputs floating-point data, the inputs will
675 by definition be turned into such. Therefore, it only makes sense to
676 have floating-point "GenericTypes" inputs. If you want to default to
677 coercing inputs to "float", give that as the last "GenericTypes" as the
678 generated XS function defaults to the last-given one. Hence (with the
679 "PMCode" and "Doc" omitted):
680 pp_def('r2C',
682 GenericTypes=>[reverse qw(F D G C)], # last one is default so here = F
683 Pars => 'r(); complex [o]c()',
684 Code => '$c() = $r();'
685 );
686 Finally, there are the "type+" qualifiers where type is one of "int" or
688 "float". What shall that mean. Let's illustrate the "int+" qualifier
689 with the actual definition of sumover:
690 pp_def('sumover',
692 Pars => 'a(n); int+ [o] b()',
693 Code => '$GENERIC(b) tmp=0;
694 loop(n) %{ tmp += a(); %}
695 $b() = tmp;',
696 );
697 As we had already seen for the "int", "float" and "double" qualifiers,
699 a pdl marked with a "type+" qualifier does not influence the datatype
700 of the pdl operation. Its meaning is "make this pdl at least of type
701 "type" or higher, as required by the type of the operation". In the
702 sumover example this means that when you call the function with an "a"
703 of type PDL_Short the output pdl will be of type PDL_Long (just as
704 would have been the case with the "int" qualifier). This again tries to
705 avoid overflow problems when using small datatypes (e.g. byte images).
706 However, when the datatype of the operation is higher than the type
707 specified in the "type+" qualifier "b" will be created with the
708 datatype of the operation, e.g. when "a" is of type double then "b"
709 will be double as well. We hope you agree that this is sensible
710 behaviour for "sumover". It should be obvious how the "float+"
711 qualifier works by analogy. It may become necessary to be able to
712 specify a set of alternative types for the parameters. However, this
713 will probably not be implemented until someone comes up with a
714 reasonable use for it.
715 Note that we now had to specify the $GENERIC macro with the name of the
717 pdl to derive the type from that argument. Why is that? If you
718 carefully followed our explanations you will have realised that in some
719 cases "b" will have a different type than the type of the operation.
720 Calling the '$GENERIC' macro with "b" as argument makes sure that the
721 type will always the same as that of "b" in that part of the generic
722 loop.
723 This is about all there is to say about the "Pars" section in a
725 "pp_def" call. You should remember that this section defines the
726 signature of a PP defined function, you can use several options to
727 qualify certain arguments as output and temporary args and all
728 dimensions that you can later refer to in the "Code" section are
729 defined by name.
730 It is important that you understand the meaning of the signature since
732 in the latest PDL versions you can use it to define broadcasting
733 functions from within Perl, i.e. what we call Perl level broadcasting.
734 Please check PDL::Indexing for details.
735 The Code section
737 The "Code" section contains the actual XS code that will be in the
738 innermost part of a broadcast loop (if you don't know what a broadcast
739 loop is then you still haven't read PDL::Indexing; do it now ;) after
740 any PP macros (like $GENERIC) and PP functions have been expanded (like
741 the "loop" function we are going to explain next).
742 Let's quickly reiterate the "sumover" example:
744 pp_def('sumover',
746 Pars => 'a(n); int+ [o] b()',
747 Code => '$GENERIC(b) tmp=0;
748 loop(n) %{ tmp += a(); %}
749 $b() = tmp;',
750 );
751 The "loop" construct in the "Code" section also refers to the dimension
753 name so you don't need to specify any limits: the loop is correctly
754 sized and everything is done for you, again.
755 Next, there is the surprising fact that $a() and $b() do not contain
757 the index. This is not necessary because we're looping over n and both
758 variables know which dimensions they have so they automatically know
759 they're being looped over.
760 This feature comes in very handy in many places and makes for much
762 shorter code. Of course, there are times when you want to circumvent
763 this; here is a function which make a matrix symmetric and serves as an
764 example of how to code explicit looping:
765 pp_def('symm',
767 Pars => 'a(n,n); [o]c(n,n);',
768 Code => 'loop(n) %{
769 int n2;
770 for(n2=n; n2<$SIZE(n); n2++) {
771 $c(n0 => n, n1 => n2) =
772 $c(n0 => n2, n1 => n) =
773 $a(n0 => n, n1 => n2);
774 }
775 %}
776 '
777 );
778 Let's dissect what is happening. Firstly, what is this function
780 supposed to do? From its signature you see that it takes a 2D matrix
781 with equal numbers of columns and rows and outputs a matrix of the same
782 size. From a given input matrix $a it computes a symmetric output
783 matrix $c (symmetric in the matrix sense that A^T = A where ^T means
784 matrix transpose, or in PDL parlance $c == $c->transpose). It does this
785 by using only the values on and below the diagonal of $a. In the output
786 matrix $c all values on and below the diagonal are the same as those in
787 $a while those above the diagonal are a mirror image of those below the
788 diagonal (above and below are here interpreted in the way that PDL
789 prints 2D pdls). If this explanation still sounds a bit strange just go
790 ahead, make a little file into which you write this definition, build
791 the new PDL extension (see section on Makefiles for PP code) and try it
792 out with a couple of examples.
793 Having explained what the function is supposed to do there are a couple
795 of points worth noting from the syntactical point of view. First, we
796 get the size of the dimension named "n" again by using the $SIZE macro.
797 Second, there are suddenly these funny "n0" and "n1" index names in the
798 code though the signature defines only the dimension "n". Why this? The
799 reason becomes clear when you note that both the first and second
800 dimension of $a and $b are named "n" in the signature of "symm". This
801 tells PDL::PP that the first and second dimension of these arguments
802 should have the same size. Otherwise the generated function will raise
803 a runtime error. However, now in an access to $a and $c PDL::PP cannot
804 figure out which index "n" refers to any more just from the name of the
805 index. Therefore, the indices with equal dimension names get numbered
806 from left to right starting at 0, e.g. in the above example "n0" refers
807 to the first dimension of $a and $c, "n1" to the second and so on.
808 In all examples so far, we have only used the "Pars" and "Code" members
810 of the hash that was passed to "pp_def". There are certainly other keys
811 that are recognised by PDL::PP and we will hear about some of them in
812 the course of this document. Find a (non-exhaustive) list of keys in
813 Appendix A. A list of macros and PPfunctions (we have only encountered
814 some of those in the examples above yet) that are expanded in values of
815 the hash argument to "pp_def" is summarised in Appendix B.
816 At this point, it might be appropriate to mention that PDL::PP is not a
818 completely static, well designed set of routines (as Tuomas puts it:
819 "stop thinking of PP as a set of routines carved in stone") but rather
820 a collection of things that the PDL::PP author (Tuomas J. Lukka)
821 considered he would have to write often into his PDL extension
822 routines. PP tries to be expandable so that in the future, as new needs
823 arise, new common code can be abstracted back into it. If you want to
824 learn more on why you might want to change PDL::PP and how to do it
825 check the section on PDL::PP internals.
826 Handling bad values
828 There are several keys and macros used when writing code to handle bad
829 values. The first one is the "HandleBad" key:
830 HandleBad => 0
832 This flags a pp-routine as NOT handling bad values. If this routine
833 is sent ndarrays with their "badflag" set, then a warning message
834 is printed to STDOUT and the ndarrays are processed as if the value
835 used to represent bad values is a valid number. The "badflag" value
836 is not propagated to the output ndarrays.
837 An example of when this is used is for FFT routines, which
839 generally do not have a way of ignoring part of the data.
840 HandleBad => 1
842 This causes PDL::PP to write extra code that ensures the BadCode
843 section is used, and that the $ISBAD() macro (and its brethren)
844 work. If no "BadCode" is supplied, the "Code" section will be used,
845 on the assumption it will use "PDL_IF_BAD" to handle bad values.
846 HandleBad is not given
848 If any of the input ndarrays have their "badflag" set, then the
849 output ndarrays will have their "badflag" set, but any supplied
850 BadCode is ignored.
851 The value of "HandleBad" is used to define the contents of the "BadDoc"
853 key, if it is not given.
854 To handle bad values, code must be written somewhat differently; for
856 instance,
857 $c() = $a() + $b();
859 becomes something like
861 if ( $a() != BADVAL && $b() != BADVAL ) {
863 $c() = $a() + $b();
864 } else {
865 $c() = BADVAL;
866 }
867 However, we only want the second version if bad values are present in
869 the input ndarrays (and that bad-value support is wanted!) - otherwise
870 we actually want the original code. This is where the "BadCode" key
871 comes in; you use it to specify the code to execute if bad values may
872 be present, and PP uses both it and the "Code" section to create
873 something like:
874 if ( bad_values_are_present ) {
876 fancy_broadcastloop_stuff {
877 BadCode
878 }
879 } else {
880 fancy_broadcastloop_stuff {
881 Code
882 }
883 }
884 This approach means that there is virtually no overhead when bad values
886 are not present (i.e. the badflag routine returns 0).
887 The C preprocessor symbol "PDL_BAD_CODE" is defined when the bad code
889 is compiled, so that you can reduce the amount of code you write. The
890 BadCode section can use the same macros and looping constructs as the
891 Code section. As of 2.073, you can also use
892 "PDL_IF_BAD(iftrue,iffalse)".
893 Other bad-value macros
895 However, it wouldn't be much use without the following additional
896 macros:
897 $ISBAD(var)
899 To check whether an ndarray's value is bad, use the $ISBAD macro:
901 if ( $ISBAD(a()) ) { printf("a() is bad\n"); }
903 You can also access given elements of an ndarray:
905 if ( $ISBAD(a(n=>l)) ) { printf("element %d of a() is bad\n", l); }
907 $ISGOOD(var)
909 This is the opposite of the $ISBAD macro.
911 $SETBAD(var)
913 For when you want to set an element of an ndarray bad.
915 $ISBADVAR(c_var,pdl)
917 If you have cached the value of an ndarray $a() into a c-variable
919 ("foo" say), then to check whether it is bad, use "$ISBADVAR(foo,a)".
920 $ISGOODVAR(c_var,pdl)
922 As above, but this time checking that the cached value isn't bad.
924 $SETBADVAR(c_var,pdl)
926 To copy the bad value for an ndarray into a c variable, use
928 "$SETBADVAR(foo,a)".
929 TODO: mention $PPISBAD() etc macros.
931 PDL STATE macros
933 If you want access to the value of the badflag for a given ndarray, you
934 can use the PDL STATE macros, for use in "CopyBadStatusCode" and
935 "FindBadStatusCode".
936 $ISPDLSTATEBAD(pdl)
938 $ISPDLSTATEGOOD(pdl)
939 $SETPDLSTATEBAD(pdl)
940 $SETPDLSTATEGOOD(pdl)
941 And for use in "Code" sections:
943 $PDLSTATEISBAD(pdl)
945 $PDLSTATEISGOOD(pdl)
946 $PDLSTATESETBAD(pdl)
947 $PDLSTATESETGOOD(pdl)
948 Bad-value examples
950 Using these macros, the above code could be specified as:
951 Code => '$c() = $a() + $b();',
953 BadCode => '
954 if ( $ISBAD(a()) || $ISBAD(b()) ) {
955 $SETBAD(c());
956 } else {
957 $c() = $a() + $b();
958 }',
959 Since this is Perl, TMTOWTDI, so you could also write:
961 BadCode => '
963 if ( $ISGOOD(a()) && $ISGOOD(b()) ) {
964 $c() = $a() + $b();
965 } else {
966 $SETBAD(c());
967 }',
968 You can reduce code repetition using the C "PDL_BAD_CODE" macro,
970 supplying only the "Code" section:
971 Code => '
973 #ifdef PDL_BAD_CODE
974 if ( $ISGOOD(a()) && $ISGOOD(b()) ) {
975 #endif PDL_BAD_CODE
976 $c() = $a() + $b();
977 #ifdef PDL_BAD_CODE
978 } else {
979 $SETBAD(c());
980 }
981 #endif PDL_BAD_CODE
982 ',
983 As of 2.073, you can also use "PDL_IF_BAD(iftrue,iffalse)":
985 Code => '
987 PDL_IF_BAD(if ( $ISGOOD(a()) && $ISGOOD(b()) ) {,)
988 $c() = $a() + $b();
989 PDL_IF_BAD(} else $SETBAD(c());,)
990 ',
991 Interfacing your own/library functions using PP
993 Now, consider the following: you have your own C function (that may in
994 fact be part of some library you want to interface to PDL) which takes
995 as arguments two pointers to vectors of double:
996 void myfunc(int n,double *v1,double *v2);
998 The correct way of defining the PDL function is
1000 pp_def('myfunc',
1002 Pars => 'a(n); [o]b(n);',
1003 GenericTypes => ['D'],
1004 Code => 'myfunc($SIZE(n),$P(a),$P(b));'
1005 );
1006 The "$P("par")" syntax returns a pointer to the first element and the
1008 other elements are guaranteed to lie after that.
1009 Notice that here it is possible to make many mistakes. First, $SIZE(n)
1011 must be used instead of "n". Second, you shouldn't put any loops in
1012 this code. Third, here we encounter a new hash key recognised by
1013 PDL::PP : the "GenericTypes" declaration tells PDL::PP to ONLY GENERATE
1014 THE TYPELOOP FOP THE LIST OF TYPES SPECIFIED. In this case "double".
1015 This has two advantages. Firstly the size of the compiled code is
1016 reduced vastly, secondly if non-double arguments are passed to myfunc()
1017 PDL will automatically convert them to double before passing to the
1018 external C routine and convert them back afterwards.
1019 One can also use "Pars" to qualify the types of individual arguments.
1021 Thus one could also write this as:
1022 pp_def('myfunc',
1024 Pars => 'double a(n); double [o]b(n);',
1025 Code => 'myfunc($SIZE(n),$P(a),$P(b));'
1026 );
1027 The type specification in "Pars" exempts the argument from variation in
1029 the typeloop - rather it is automatically converted to and from the
1030 type specified. This is obviously useful in a more general example,
1031 e.g.:
1032 void myfunc(int n,float *v1,long *v2);
1034 pp_def('myfunc',
1036 Pars => 'float a(n); long [o]b(n);',
1037 GenericTypes => ['F'],
1038 Code => 'myfunc($SIZE(n),$P(a),$P(b));'
1039 );
1040 Note we still use "GenericTypes" to reduce the size of the type loop,
1042 obviously PP could in principle spot this and do it automatically
1043 though the code has yet to attain that level of sophistication!
1044 Finally note when types are converted automatically one MUST use the
1046 "[o]" qualifier for output variables or you hard-won changes will get
1047 optimised away by PP!
1048 If you interface a large library you can automate the interfacing even
1050 further. Perl can help you again(!) in doing this. In many libraries
1051 you have certain calling conventions. This can be exploited. In short,
1052 you can write a little parser (which is really not difficult in Perl)
1053 that then generates the calls to "pp_def" from parsed descriptions of
1054 the functions in that library. For an example, please check the Slatec
1055 interface in the "Lib" tree of the PDL distribution. If you want to
1056 check (during debugging) which calls to PP functions your Perl code
1057 generated a little helper package comes in handy which replaces the PP
1058 functions by identically named ones that dump their arguments to
1059 stdout.
1060 Just say
1062 perl -MPDL::PP::Dump myfile.pd
1064 to see the calls to "pp_def" and friends. Try it with ops.pd and
1066 slatec.pd. If you're interested (or want to enhance it), the source is
1067 in Basic/Gen/PP/Dump.pm
1068 Other macros in the Code section
1070 Macros: So far we have encountered the $SIZE, $GENERIC and $P macros.
1071 Now we are going to quickly explain the other macros that are expanded
1072 in the "Code" section of PDL::PP along with examples of their usage.
1073 $T
1075 The $T macro is used for type switches. This is very useful when you
1077 have to use different external (e.g. library) functions depending on
1078 the input type of arguments. The general syntax is
1079 $Ttypeletters(type_alternatives)
1081 where "typeletters" is a permutation of a subset of the letters
1083 "BSULNQFDGC" which stand for Byte, Short, Ushort, etc. and
1084 "type_alternatives" are the expansions when the type of the PP
1085 operation is equal to that indicated by the respective letter. Let's
1086 illustrate this incomprehensible description by an example. Assuming
1087 you have two C functions with prototypes
1088 void float_func(float *in, float *out);
1090 void double_func(double *in, double *out);
1091 which do basically the same thing but one accepts float and the other
1093 double pointers. You could interface them to PDL by defining a generic
1094 function "foofunc" (which will call the correct function depending on
1095 the type of the transformation):
1096 pp_def('foofunc',
1098 Pars => ' a(n); [o] b();',
1099 Code => ' $TFD(float,double)_func ($P(a),$P(b));'
1100 GenericTypes => [qw(F D)],
1101 );
1102 There is a limitation that the comma-separated values cannot have
1104 parentheses.
1105 $PP
1107 The $PP macro is used for a so called physical pointer access. The
1109 physical refers to some internal optimisations of PDL (for those who
1110 are familiar with the PDL core we are talking about the vaffine
1111 optimisations). This macro is mainly for internal use and you shouldn't
1112 need to use it in any of your normal code.
1113 $PPSYM
1115 The $PPSYM() macro is replaced by the value of "ppsym" in PDL::Types
1117 for the loop type, or that of the given parameter, similar to
1118 $GENERIC(). This is useful for e.g. macros that vary by that, avoiding
1119 the need for things like "$TXY(X,Y)". Another benefit is that if an
1120 operation's GenericTypes get extended, this macro will still be
1121 correct.
1122 $COMP (and the OtherPars section)
1124 The $COMP macro is used to access non-pdl values in the code section.
1126 Its name is derived from the implementation of transformations in PDL.
1127 The variables you can refer to using $COMP are members of the
1128 ``compiled'' structure that represents the PDL transformation in
1129 question but does not yet contain any information about dimensions (for
1130 further details check PDL::Internals). However, you can treat $COMP
1131 just as a black box without knowing anything about the implementation
1132 of transformations in PDL. So when would you use this macro? Its main
1133 usage is to access values of arguments that are declared in the
1134 "OtherPars" section of a "pp_def" definition. But then you haven't
1135 heard about the "OtherPars" key yet?! Let's have another example that
1136 illustrates typical usage of both new features:
1137 pp_def('pnmout',
1139 Pars => 'a(m)',
1140 OtherPars => "PerlIO *fp",
1141 GenericTypes => [qw(B U S L)],
1142 Code => '
1143 if (PerlIO_write($COMP(fp),$P(a),len) != len)
1144 $CROAK("Error writing pnm file");
1145 ');
1146 This function is used to write data from a pdl to a file. The file
1148 descriptor is passed as a string into this function. This parameter
1149 does not go into the "Pars" section since it cannot be usefully treated
1150 like a pdl but rather into the aptly named "OtherPars" section.
1151 Parameters in the "OtherPars" section follow those in the "Pars"
1152 section when invoking the function, i.e.
1153 open FILE,">out.dat" or die "couldn't open out.dat";
1155 pnmout($pdl,'FILE');
1156 When you want to access this parameter inside the code section you have
1158 to tell PP by using the $COMP macro, i.e. you write $COMP(fp) as in the
1159 example. Otherwise PP wouldn't know that the "fp" you are referring to
1160 is the same as that specified in the "OtherPars" section.
1161 Another use for the "OtherPars" section is to set a named dimension in
1163 the signature. Let's have an example how that is done:
1164 pp_def('setdim',
1166 Pars => '[o] a(n)',
1167 OtherPars => 'int ns => n',
1168 Code => 'loop(n) %{ $a() = n; %}',
1169 );
1170 This says that the named dimension "n" will be initialised from the
1172 value of the other parameter "ns" which is of integer type (I guess you
1173 have realised that we use the "CType From => named_dim" syntax). Now
1174 you can call this function in the usual way:
1175 setdim(($x=null),5);
1177 print $x;
1178 [ 0 1 2 3 4 ]
1179 Admittedly this function is not very useful but it demonstrates how it
1181 works. If you call the function with an existing pdl and you don't need
1182 to explicitly specify the size of "n" since PDL::PP can figure it out
1183 from the dimensions of the non-null pdl. In that case you just give the
1184 dimension parameter as -1:
1185 $x = hist($y);
1187 setdim($x,-1);
1188 The default values available via $COMP() are the "OtherPars" as noted
1190 above, which get copied in. However, this can be added to (previous to
1191 2.058, replaced) by supplying "Comp" and/or "MakeComp" keys (the
1192 defaults will happen first):
1193 pp_def(
1195 'diagonal',
1196 OtherPars => 'SV *list',
1197 Comp => 'PDL_Indx whichdims_count; PDL_Indx whichdims[$COMP(whichdims_count)];',
1198 MakeComp => '
1199 PDL_Indx i;
1200 PDL_Indx *tmp= PDL->packdims(list,&($COMP(whichdims_count)));
1201 if (!tmp) $CROAK("Failed to packdims for creating");
1202 if ($COMP(whichdims_count) < 1)
1203 $CROAK("Diagonal: must have at least 1 dimension");
1204 $DOCOMPALLOC(); /* malloc()s the whichdims */
1205 for(i=0; i<$COMP(whichdims_count); i++)
1206 $COMP(whichdims)[i] = tmp[i];
1207 free(tmp);
1208 /* ... */
1209 ',
1210 # ...
1211 );
1212 The "MakeComp" code is placed in the "pdl_(funcname)_run", so access to
1214 "Pars" (which will just be "pdl *"s)/"OtherPars" values is just via
1215 their names, not a macro.
1216 As of 2.058, you can instead give a C99 "incomplete array" type
1218 parameter as an "OtherPars" entry:
1219 pp_def(
1221 'diagonal',
1222 OtherPars => 'PDL_Indx whichdims[]',
1223 MakeComp => '
1224 if ($COMP(whichdims_count) < 1)
1225 $CROAK("Diagonal: must have at least 1 dimension");
1226 /* ... */
1227 ',
1228 # ...
1229 );
1230 There is an XS typemap entry (only for "PDL_Indx" array types for now)
1232 that adds a "(varname)_count" variable having extracted the index
1233 numbers from an array-ref parameter, and sets the count variable to the
1234 right value. PP then makes a copy of the data available. The C function
1235 (here, "pdl_diagonal_run")'s caller (here, the generated XS function)
1236 is responsible for freeing the array passed in (here, PDL's "smalloc"
1237 function is used, so the user need do nothing different).
1238 Other functions in the Code section
1240 The only PP function that we have used in the examples so far is
1241 "loop". Additionally, there are currently two other functions which
1242 are recognised in the "Code" section:
1243 broadcastloop
1245 As we heard above the signature of a PP defined function defines the
1247 dimensions of all the pdl arguments involved in a primitive operation.
1248 However, you often call the functions that you defined with PP with
1249 pdls that have more dimensions than those specified in the signature.
1250 In this case the primitive operation is performed on all subslices of
1251 appropriate dimensionality in what is called a broadcast loop (see also
1252 overview above and PDL::Indexing). Assuming you have some notion of
1253 this concept you will probably appreciate that the operation specified
1254 in the code section should be optimised since this is the tightest loop
1255 inside a broadcast loop. However, if you revisit the example where we
1256 define the "pnmout" function, you will quickly realise that looking up
1257 the "IO" file descriptor in the inner broadcast loop is not very
1258 efficient when writing a pdl with many rows. A better approach would be
1259 to look up the "IO" descriptor once outside the broadcast loop and use
1260 its value then inside the tightest broadcast loop. This is exactly
1261 where the "broadcastloop" function comes in handy. Here is an improved
1262 definition of "pnmout" which uses this function:
1263 pp_def('pnmout',
1265 Pars => 'a(m)',
1266 OtherPars => "PerlIO *fp",
1267 GenericTypes => [qw(B U S L)],
1268 Code => '
1269 int len;
1270 len = $SIZE(m) * sizeof($GENERIC());
1271 broadcastloop %{
1272 if (PerlIO_write($COMP(fp),$P(a),len) != len)
1273 $CROAK("Error writing pnm file");
1274 %}
1275 ');
1276 This works as follows. Normally the C code you write inside the "Code"
1278 section is placed inside a broadcast loop (i.e. PP generates the
1279 appropriate wrapping C code around it). However, when you explicitly
1280 use the "broadcastloop" function, PDL::PP recognises this and doesn't
1281 wrap your code with an additional broadcast loop. This has the effect
1282 that code you write outside the broadcast loop is only executed once
1283 per transformation and just the code with in the surrounding "%{ ...
1284 %}" pair is placed within the tightest broadcast loop. This also comes
1285 in handy when you want to perform a decision (or any other code,
1286 especially CPU intensive code) only once per thread, i.e.
1287 pp_addhdr('
1289 #define RAW 0
1290 #define ASCII 1
1291 ');
1292 pp_def('do_raworascii',
1293 Pars => 'a(); b(); [o]c()',
1294 OtherPars => 'int mode',
1295 Code => ' switch ($COMP(mode)) {
1296 case RAW:
1297 broadcastloop %{
1298 /* do raw stuff */
1299 %}
1300 break;
1301 case ASCII:
1302 broadcastloop %{
1303 /* do ASCII stuff */
1304 %}
1305 break;
1306 default:
1307 $CROAK("unknown mode");
1308 }'
1309 );
1310 types
1312 The types function works similar to the $T macro. However, with the
1314 "types" function the code in the following block (delimited by "%{" and
1315 "%}" as usual) is executed for all those cases in which the datatype of
1316 the operation is any of the types represented by the letters in the
1317 argument to "type", e.g.
1318 Code => '...
1320 types(BSUL) %{
1322 /* do integer type operation */
1323 %}
1324 types(FD) %{
1325 /* do floating point operation */
1326 %}
1327 ...'
1328 You are encouraged to use this idiom (from PDL::Math) in order to
1330 minimise effort needed to make your code work with new types:
1331 use PDL::Types qw(types);
1333 my @Rtypes = grep $_->real, types();
1334 my @Ctypes = grep !$_->real, types();
1335 # ...
1336 my $got_complex = PDL::Core::Dev::got_complex_version($name, 2);
1337 my $complex_bit = join "\n",
1338 map 'types('.$_->ppsym.') %{$'.$c.'() = c'.$name.$_->floatsuffix.'($'.$x.'(),$'.$y.'());%}',
1339 @Ctypes;
1340 my $real_bit = join "\n",
1341 map 'types('.$_->ppsym.') %{$'.$c.'() = '.$name.'($'.$x.'(),$'.$y.'());%}',
1342 @Rtypes;
1343 ($got_complex ? $complex_bit : '') . $real_bit;
1344 (although you should first check whether tgmath.h already has a type-
1346 generic version of the function you want to call, in which case the
1347 above becomes unnecessary).
1348 The RedoDimsCode Section
1350 The "RedoDimsCode" key is an optional key that is used to compute
1351 dimensions of ndarrays at runtime in case the standard rules for
1352 computing dimensions from the signature are not sufficient. The
1353 contents of the "RedoDimsCode" entry is interpreted in the same way
1354 that the Code section is interpreted-- i.e., PP macros are expanded and
1355 the result is interpreted as C code. The purpose of the code is to set
1356 the size of some dimensions that appear in the signature. Storage
1357 allocation and broadcastloops and so forth will be set up as if the
1358 computed dimension had appeared in the signature. In your code, you
1359 first compute the desired size of a named dimension in the signature
1360 according to your needs and then assign that value to it via the
1361 $SIZE() macro.
1362 As an example, consider the following situation. You are interfacing an
1364 external library routine that requires an temporary array for workspace
1365 to be passed as an argument. Two input data arrays that are passed are
1366 p(m) and x(n). The output data array is y(n). The routine requires a
1367 workspace array with a length of n+m*m, and you'd like the storage
1368 created automatically just like it would be for any ndarray flagged
1369 with [t] or [o]. What you'd like is to say something like
1370 pp_def( "myexternalfunc",
1372 Pars => " p(m); x(n); [o] y; [t] work(n+m*m); ", ...
1373 but that won't work, because PP can't interpret expressions with
1375 arithmetic in the signature. Instead you write
1376 pp_def(
1378 "myexternalfunc",
1379 Pars => ' p(m); x(n); [o] y(); [t] work(wn); ',
1380 RedoDimsCode => '$SIZE(wn) = $SIZE(n) + $SIZE(m) * $SIZE(m);',
1381 Code => '
1382 externalfunc( $P(p), $P(x), $SIZE(m), $SIZE(n), $P(work) );
1383 '
1384 );
1385 As of 2.075, you can use the dimensions of passed-in ndarrays as they
1387 are available when the "RedoDimsCode" is run. Before the code in the
1388 Code section is executed PP will create the proper storage for "work"
1389 if it does not exist. Note that you only took the first dimension of
1390 "p" and "x" because the user may have sent ndarrays with extra
1391 broadcasting dimensions. Of course, the temporary ndarray "work" (note
1392 the [t] flag) should not be given any broadcast dimensions anyway.
1393 You can also use "RedoDimsCode" to set the dimension of a ndarray
1395 flagged with [o]. In this case you set the dimensions for the named
1396 dimension in the signature using $SIZE() as in the preceding example.
1397 However, because the ndarray is flagged with [o] instead of [t],
1398 broadcasting dimensions will be added if required just as if the size
1399 of the dimension were computed from the signature according to the
1400 usual rules. Here is an example from PDL::Math
1401 pp_def("polyroots",
1403 Pars => 'cr(n); ci(n); [o]rr(m); [o]ri(m);',
1404 RedoDimsCode => '$SIZE(m) = $SIZE(n)-1;',
1405 The input ndarrays are the real and imaginary parts of complex
1407 coefficients of a polynomial. The output ndarrays are real and
1408 imaginary parts of the roots. There are "n" roots to an "n"th order
1409 polynomial and such a polynomial has "n+1" coefficients (the zero-th
1410 through the "n"th). In this example, broadcasting will work correctly.
1411 That is, the first dimension of the output ndarray with have its
1412 dimension adjusted, but other broadcasting dimensions will be assigned
1413 just as if there were no "RedoDimsCode".
1414 RedoDims passed directly
1416 A "RedoDimsCode" value as above gets processed, including expanding
1418 macros, and adding type-generic loops. For very specific purposes, you
1419 may not want this processing done to your dimension-updating code,
1420 probably in "slice"-like functions.
1421 Then, instead of passing a "RedoDimsCode" value, you can pass a
1423 "RedoDims" value (which the "RedoDimsCode" would otherwise get
1424 processed into). Because you will probably want to access the ndarrays,
1425 the following macros are provided. They are named assuming you will
1426 have the first parameter as "PARENT" and the second as "CHILD", which
1427 is the case if you passed a true "P2Child" value, which you will
1428 basically always want to do for this scenario.
1429 RedoDims generated from EquivPDimExpr and EquivDimCheck
1431 Another way to generate the "RedoDims" code is to supply a
1433 "EquivPDimExpr" and maybe a "EquivDimCheck":
1434 pp_def(
1436 'xchg',
1437 OtherPars => 'PDL_Indx n1; PDL_Indx n2;',
1438 TwoWay => 1,
1439 P2Child => 1,
1440 AffinePriv => 1,
1441 EquivDimCheck => '
1442 if ($COMP(n1) <0) $COMP(n1) += $PARENT(broadcastids[0]);
1443 if ($COMP(n2) <0) $COMP(n2) += $PARENT(broadcastids[0]);
1444 if (PDLMIN($COMP(n1),$COMP(n2)) <0 ||
1445 PDLMAX($COMP(n1),$COMP(n2)) >= $PARENT(broadcastids[0]))
1446 $CROAK("One of dims %d, %d out of range: should be 0<=dim<%d",
1447 $COMP(n1),$COMP(n2),$PARENT(broadcastids[0]));',
1448 EquivPDimExpr => '
1449 (($CDIM == $COMP(n1)) ? $COMP(n2) :
1450 ($CDIM == $COMP(n2)) ? $COMP(n1) :
1451 $CDIM)
1452 ',
1453 );
1454 "EquivPDimExpr" is evaluated within a loop, and the value of the
1456 relevant dimension is available using the macro $CDIM as shown above.
1457 Typemap handling in the OtherPars section
1459 The "OtherPars" section discussed above is very often absolutely
1460 crucial when you interface external libraries with PDL. However in many
1461 cases the external libraries either use derived types or pointers of
1462 various types.
1463 The standard way to handle this in Perl is to use a typemap file. This
1465 is discussed in some detail in perlxs in the standard Perl
1466 documentation. In PP the functionality is very similar, so you can
1467 create a typemap file in the directory where your PP file resides and
1468 when it is built it is automatically read in to figure out the
1469 appropriate translation between the C type and Perl's built-in type.
1470 For instance the "gsl_spline_init" function has the following C
1472 declaration:
1473 int gsl_spline_init(gsl_spline * spline,
1475 const double xa[], const double ya[], size_t size);
1476 Clearly the "xa" and "ya" arrays are candidates for being passed in as
1478 ndarrays and the "size" argument is just the length of these ndarrays
1479 so that can be handled by the $SIZE() macro in PP. Write an
1480 "OtherPars" declaration of the form
1481 OtherPars => 'gsl_spline *spl'
1483 and write a short typemap file which handles this type:
1485 TYPEMAP
1487 gsl_spline * T_PTR
1488 and use it in the code:
1490 pp_def('init_meat',
1492 Pars => 'double x(n); double y(n);',
1493 OtherPars => 'gsl_spline *spl',
1494 Code =>'gsl_spline_init,($COMP(spl),$P(x),$P(y),$SIZE(n)));'
1495 );
1496 where I have removed a macro wrapper call, but that would obscure the
1498 discussion.
1499 OtherPars as outputs
1501 As of 2.081, you can specify an "OtherPar" as an output. This looks
1503 like:
1504 pp_def('output_op',
1506 Pars => 'in(n=2)',
1507 OtherPars => '[o] PDL_Anyval v0; [o] PDL_Anyval v1',
1508 Code => '
1509 pdl_datatypes dt = $PDL(in)->datatype;
1510 ANYVAL_FROM_CTYPE($COMP(v0), dt, $in(n=>0));
1511 ANYVAL_FROM_CTYPE($COMP(v1), dt, $in(n=>1));
1512 ',
1513 );
1514 The passed-in stack SV will be mutated in place, so this code will then
1516 work:
1517 output_op([5,7], my $v0, my $v1);
1519 is_deeply [$v0,$v1], [5,7], 'output OtherPars work';
1520 An operation with output "OtherPars" cannot broadcast, since that would
1522 cause undefined results. A runtime check is generated that throws an
1523 exception if any "Par" would cause broadcasting.
1524 Note the syntax for "OtherPars" has "[o]" go before the type, while it
1526 goes after the type in "Pars". It was felt this was the best way to
1527 avoid ambiguity given C types can have "[]" in them.
1528 This relies on the relevant "OtherPar" having an "OUTPUT" entry in an
1530 XS typemap.
1531 Other useful PP keys in data operation definitions
1533 You have already heard about the "OtherPars" key. Currently, there are
1534 not many other keys for a data operation that will be useful in normal
1535 (whatever that is) PP programming. In fact, it would be interesting to
1536 hear about a case where you think you need more than what is provided
1537 at the moment. Please speak up on one of the PDL mailing lists. Most
1538 other keys recognised by "pp_def" are only really useful for what we
1539 call slice operations (see also above).
1540 One thing that is strongly being planned is variable number of
1542 arguments, which will be a little tricky.
1543 An incomplete list of the available keys:
1545 Inplace
1547 Setting this key marks the routine as working inplace - ie the input
1549 and output ndarrays are the same. An example is "$x->inplace->sqrt()"
1550 (or "sqrt(inplace($x))").
1551 Inplace => 1
1553 Use when the routine is a unary function, such as "sqrt".
1554 Inplace => ['a']
1556 If there are more than one input ndarrays, specify the name of the
1557 one that can be changed inplace using an array reference.
1558 Inplace => ['a','b']
1560 If there are more than one output ndarray, specify the name of the
1561 input ndarray and output ndarray in a 2-element array reference.
1562 This probably isn't needed, but left in for completeness.
1563 If bad values are being used, care must be taken to ensure the
1565 propagation of the badflag when inplace is being used; consider this
1566 excerpt from Basic/Bad/bad.pd:
1567 pp_def('setbadtoval',HandleBad => 1,
1569 Pars => 'a(); [o]b();',
1570 OtherPars => 'double newval',
1571 Inplace => 1,
1572 CopyBadStatusCode => 'PDL->propagate_badflag( b, 0 );',
1573 ...
1574 Since this routine removes all bad values, the output ndarray had its
1576 bad flag cleared. This is then propagated to both parents and children.
1577 NOTE: one idea is that the documentation for the routine could be
1579 automatically flagged to indicate that it can be executed inplace, ie
1580 something similar to how "HandleBad" sets "BadDoc" if it's not supplied
1581 (it's not an ideal solution).
1582 FTypes
1584 # in slices.pd
1586 FTypes => {CHILD => '$COMP(totype)'},
1587 The value is a hash-ref mapping parameter-names to an expression giving
1589 an override of the type for that parameter. The example above shows the
1590 type being overridden to the "OtherPars" "totype".
1591 OtherParsDefaults
1593 OtherPars => 'int a; int b',
1595 OtherParsDefaults => { b => 0 },
1596 Allows specifying default values for "OtherPars". It is an error to
1598 specify a default for one that is before another that does not have a
1599 default.
1600 Other PDL::PP functions to support concise package definition
1602 So far, we have described the "pp_def" and "pp_done" functions. PDL::PP
1603 exports a few other functions to aid you in writing concise PDL
1604 extension package definitions.
1605 pp_addhdr
1607 Often when you interface library functions as in the above example you
1609 have to include additional C include files. Since the XS file is
1610 generated by PP we need some means to make PP insert the appropriate
1611 include directives in the right place into the generated XS file. To
1612 this end there is the "pp_addhdr" function. This is also the function
1613 to use when you want to define some C functions for internal use by
1614 some of the XS functions (which are mostly functions defined by
1615 "pp_def"). By including these functions here you make sure that
1616 PDL::PP inserts your code before the point where the actual XS module
1617 section begins and will therefore be left untouched by xsubpp (cf.
1618 perlxs and perlxstut man pages).
1619 A typical call would be
1621 pp_addhdr('
1623 #include <unistd.h> /* we need defs of XXXX */
1624 #include "libprotos.h" /* prototypes of library functions */
1625 #include "mylocaldecs.h" /* Local decs */
1626 static void do_the real_work(PDL_Byte * in, PDL_Byte * out, int n)
1628 {
1629 /* do some calculations with the data */
1630 }
1631 ');
1632 This ensures that all the constants and prototypes you need will be
1634 properly included and that you can use the internal functions defined
1635 here in the "pp_def"s, e.g.:
1636 pp_def('barfoo',
1638 Pars => ' a(n); [o] b(n)',
1639 GenericTypes => ['B'],
1640 Code => ' PDL_Indx ns = $SIZE(n);
1641 do_the_real_work($P(a),$P(b),ns);
1642 ',
1643 );
1644 pp_addpm
1646 In many cases the actual PP code (meaning the arguments to "pp_def"
1648 calls) is only part of the package you are currently implementing.
1649 Often there is additional Perl code and XS code you would normally have
1650 written into the pm and XS files which are now automatically generated
1651 by PP. So how to get this stuff into those dynamically generated files?
1652 Fortunately, there are a couple of functions, generally called
1653 "pp_addXXX" that assist you in doing this.
1654 Let's assume you have additional Perl code that should go into the
1656 generated pm-file. This is easily achieved with the "pp_addpm" command:
1657 pp_addpm(<<'EOD');
1659 =head1 NAME
1661 PDL::Lib::Mylib -- a PDL interface to the Mylib library
1663 =head1 DESCRIPTION
1665 This package implements an interface to the Mylib package with full
1667 broadcasting and indexing support (see L<PDL::Indexing>).
1668 =cut
1670 use PGPLOT;
1672 =head2 use_myfunc
1674 this function applies the myfunc operation to all the
1675 elements of the input pdl regardless of dimensions
1676 and returns the sum of the result
1677 =cut
1678 sub use_myfunc {
1680 my $pdl = shift;
1681 myfunc($pdl->clump(-1),($res=null));
1683 return $res->sum;
1685 }
1686 EOD
1688 pp_add_exported
1690 You have probably got the idea. In some cases you also want to export
1692 your additional functions. To avoid getting into trouble with PP which
1693 also messes around with the @EXPORT array you just tell PP to add your
1694 functions to the list of exported functions:
1695 pp_add_exported('use_myfunc gethynx');
1697 pp_add_isa
1699 The "pp_add_isa" command works like the the "pp_add_exported" function.
1701 The arguments to "pp_add_isa" are added the @ISA list, e.g.
1702 pp_add_isa(' Some::Other::Class ');
1704 pp_bless
1706 If your pp_def routines are to be used as object methods use "pp_bless"
1708 to specify the package (i.e. class) to which your pp_defed methods will
1709 be added. For example, pp_bless('PDL::MyClass'). The default is "PDL"
1710 if this is omitted.
1711 pp_addxs
1713 Sometimes you want to add extra XS code of your own (that is generally
1715 not involved with any broadcasting/indexing issues but supplies some
1716 other functionality you want to access from the Perl side) to the
1717 generated XS file, for example
1718 pp_addxs('','
1720 # Determine endianness of machine
1722 int
1724 isbigendian()
1725 CODE:
1726 unsigned short i;
1727 PDL_Byte *b;
1728 i = 42; b = (PDL_Byte*) (void*) &i;
1730 if (*b == 42)
1732 RETVAL = 0;
1733 else if (*(b+1) == 42)
1734 RETVAL = 1;
1735 else
1736 croak("Impossible - machine is neither big nor little endian!!\n");
1737 OUTPUT:
1738 RETVAL
1739 ');
1740 Especially "pp_add_exported" and "pp_addxs" should be used with care.
1742 PP uses PDL::Exporter, hence letting PP export your function means that
1743 they get added to the standard list of function exported by default
1744 (the list defined by the export tag ``:Func''). If you use "pp_addxs"
1745 you shouldn't try to do anything that involves broadcasting or indexing
1746 directly. PP is much better at generating the appropriate code from
1747 your definitions.
1748 pp_add_boot
1750 Finally, you may want to add some code to the BOOT section of the XS
1752 file (if you don't know what that is check perlxs). This is easily done
1753 with the "pp_add_boot" command:
1754 pp_add_boot(<<EOB);
1756 descrip = mylib_initialize(KEEP_OPEN);
1757 if (descrip == NULL)
1759 croak("Can't initialize library");
1760 GlobalStruc->descrip = descrip;
1762 GlobalStruc->maxfiles = 200;
1763 EOB
1764 pp_export_nothing
1766 By default, PP.pm puts all subs defined using the pp_def function into
1768 the output .pm file's EXPORT list. This can create problems if you are
1769 creating a subclassed object where you don't want any methods exported.
1770 (i.e. the methods will only be called using the $object->method
1771 syntax).
1772 For these cases you can call pp_export_nothing() to clear out the
1774 export list. Example (At the end of the .pd file):
1775 pp_export_nothing();
1777 pp_done();
1778 pp_core_importList
1780 By default, PP.pm puts the 'use Core;' line into the output .pm file.
1782 This imports Core's exported names into the current namespace, which
1783 can create problems if you are over-riding one of Core's methods in the
1784 current file. You end up getting messages like "Warning: sub sumover
1785 redefined in file subclass.pm" when running the program.
1786 For these cases the pp_core_importList can be used to change what is
1788 imported from Core.pm. For example:
1789 pp_core_importList('()')
1791 This would result in
1793 use Core();
1795 being generated in the output .pm file. This would result in no names
1797 being imported from Core.pm. Similarly, calling
1798 pp_core_importList(' qw/ barf /')
1800 would result in
1802 use Core qw/ barf/;
1804 being generated in the output .pm file. This would result in just
1806 'barf' being imported from Core.pm.
1807 pp_setversion
1809 Simultaneously set the .pm and .xs files' versions, thus avoiding
1811 unnecessary version-skew between the two. To use this, simply do this
1812 in your .pd file, probably near the top:
1813 our $VERSION = '0.0.3';
1815 pp_setversion($VERSION);
1816 # Then, in your Makefile.PL:
1818 my @package = qw(FFTW3.pd FFTW3 PDL::FFTW3);
1819 my %descriptor = pdlpp_stdargs(\@package);
1820 $descriptor{VERSION_FROM} = 'FFTW3.pd'; # EUMM can parse the format above
1821 However, don't use this if you use Module::Build::PDL. See that
1823 module's documentation for details.
1824 pp_deprecate_module
1826 If a particular module is deemed obsolete, this function can be used to
1828 mark it as deprecated. This has the effect of emitting a warning when a
1829 user tries to "use" the module. The generated POD for this module also
1830 carries a deprecation notice. The replacement module can be passed as
1831 an argument like this:
1832 pp_deprecate_module( infavor => "PDL::NewNonDeprecatedModule" );
1834 Note that function affects only the runtime warning and the POD.
1836
1838 Let's say that you have a function in your module called PDL::foo that
1839 uses the PP function "bar_pp" to do the heavy lifting. But you don't
1840 want to advertise that "bar_pp" exists. To do this, you must move your
1841 PP function to the top of your module file, then call
1842 pp_export_nothing()
1844 to clear the "EXPORT" list. To ensure that no documentation (even the
1846 default PP docs) is generated, set
1847 Doc => undef
1849 and to prevent the function from being added to the symbol table, set
1851 PMFunc => ''
1853 in your pp_def declaration (see Image2D.pd for an example). This will
1855 effectively make your PP function "private." However, it is always
1856 accessible via PDL::bar_pp due to Perl's module design. But making it
1857 private will cause the user to go very far out of their way to use it,
1858 so they shoulder the consequences!
1859
1861 The slice operations require a much more intimate knowledge of PDL
1862 internals than the data operations. Furthermore, the complexity of the
1863 issues involved is considerably higher than that in the average data
1864 operation. Nevertheless, functions generated using the slice operations
1865 are at the heart of the index manipulation and dataflow capabilities of
1866 PDL. You can get started by reading the section on "P2Child".
1867 Also, there are a lot of dirty issues with virtual ndarrays and
1869 vaffines which we shall entirely skip here.
1870 Slices and bad values
1872 Slice operations need to be able to handle bad values. The easiest
1873 thing to do is look at Basic/Slices/slices.pd to see how this works.
1874 Along with "BadCode", there are also the "BadBackCode" and
1876 "BadRedoDimsCode" keys for "pp_def". However, any "EquivCPOffsCode"
1877 should not need changing, since any changes are absorbed into the
1878 definition of the $EQUIVCPOFFS() macro (i.e. it is handled
1879 automatically by PDL::PP).
1880
1882 For printing warning messages or aborting/dieing, you can call "warn"
1883 or "barf" from PP code. However, you should be aware that these calls
1884 have been redefined using C preprocessor macros to "PDL->barf" and
1885 "PDL->warn". These redefinitions are in place to keep you from
1886 inadvertently calling perl's "warn" or "barf" directly, which can cause
1887 segfaults during pthreading (i.e. processor multi-threading).
1888 PDL's own versions of "barf" and "warn" will queue-up warning or barf
1890 messages until after pthreading is completed, and then call the perl
1891 versions of these routines.
1892 See PDL::ParallelCPU for more information on pthreading.
1894 NB As of 2.064, it is highly recommended that you do not call "barf" at
1896 all in PP code, but instead use $CROAK(). This will return a
1897 "pdl_error" which will transparently be used to throw the correct
1898 exception in Perl code, but can be handled suitably by non-Perl
1899 callers.
1900
1902 If you are going to generate a package from your PP file (typical file
1903 extensions are ".pd" or ".pp" for the files containing PP code) it is
1904 easiest and safest to leave generation of the appropriate commands to
1905 the Makefile. In the following we will outline the typical format of a
1906 Perl Makefile to automatically build and install your package from a
1907 description in a PP file. Most of the rules to build the xs, pm and
1908 other required files from the PP file are already predefined in the
1909 PDL::Core::Dev package. We just have to tell MakeMaker to use it.
1910 In most cases you can define your Makefile like
1912 use PDL::Core::Dev; # Pick up development utilities
1914 use ExtUtils::MakeMaker;
1915 $package = ["mylib.pd",Mylib,PDL::Lib::Mylib,'',1];
1917 %hash = pdlpp_stdargs($package);
1918 $hash{OBJECT} .= ' additional_Ccode$(OBJ_EXT) ';
1919 WriteMakefile(%hash);
1920 sub MY::postamble { pdlpp_postamble($package); }
1922 # additional_Ccode.c
1924 #include "pdl.h"
1925 void ppcp(PDL_Byte *dst, PDL_Byte *src, int len)
1926 {
1927 int i;
1928 for (i=0;i<len;i++) *dst++=*src++;
1929 }
1930 Here, the list in $package is: first: PP source file name, then the
1932 prefix for the produced files, the whole package name, the package to
1933 add XS functions to (empty string to use the same as the PP functions),
1934 and a boolean to dictate whether to have PDL generate a separate C file
1935 for each PP function (for faster compilation). The last feature is
1936 opt-in as you have to avoid duplicate symbols when linking the library
1937 (so separate out C functions into their own file). You can modify the
1938 hash in whatever way you like but it would be reasonable to stay within
1939 some limits so that your package will continue to work with later
1940 versions of PDL.
1941 To make life even easier PDL::Core::Dev defines the function
1943 "pdlpp_stdargs" that returns a hash with default values that can be
1944 passed (either directly or after appropriate modification) to a call to
1945 WriteMakefile. Currently, "pdlpp_stdargs" returns a hash where the
1946 keys are filled in as follows:
1947 (
1949 'NAME' => $mod,
1950 VERSION_FROM => $src,
1951 'TYPEMAPS' => [&PDL_TYPEMAP()],
1952 'OBJECT' => "$pref\$(OBJ_EXT)",
1953 PM => {"$pref.pm" => "\$(INST_LIBDIR)/$pref.pm"},
1954 MAN3PODS => {"$pref.pm" => "\$(INST_MAN3DIR)/$mod.\$(MAN3EXT)"},
1955 'INC' => &PDL_INCLUDE(),
1956 'LIBS' => [''],
1957 'clean' => {'FILES' => "$pref.xs $pref.pm $pref\$(OBJ_EXT)"},
1958 )
1959 Here, $src is the name of the source file with PP code, $pref the
1961 prefix for the generated .pm and .xs files and $mod the name of the
1962 extension module to generate.
1963 If your "VERSION_FROM" provides a version, PP will use that to set the
1965 "XS_VERSION". If you need to influence the value of that variable so
1966 that XSLoader etc don't reject the loaded dynamic library, you can use
1967 this workaround in a "pp_addpm" (the "BEGIN" is because the "bootstrap"
1968 happens at runtime, and your code appears after that call, but with a
1969 "BEGIN" it will take place beforehand):
1970 our $VERSION; BEGIN { $VERSION = '2.019106' };
1972 our $XS_VERSION; BEGIN { $XS_VERSION = $VERSION };
1973
1975 The internals of the current version consist of a large table which
1976 gives the rules according to which things are translated and the subs
1977 which implement these rules.
1978 Later on, it would be good to make the table modifiable by the user so
1980 that different things may be tried.
1981 [Meta comment: here will hopefully be more in the future; currently,
1983 your best bet will be to read the source code :-( or ask on the list
1984 (try the latter first) ]
1985
1987 As well as the above-mentioned "PDL_BAD_CODE" and
1988 "PDL_IF_BAD(iftrue,iffalse)", there are also
1989 "PDL_IF_GENTYPE_REAL(iftrue,iffalse)", "PDL_IF_GENTYPE_UNSIGNED", and
1990 "PDL_IF_GENTYPE_INTEGER":
1991 $b() = PDL_IF_GENTYPE_INTEGER(0,NAN);
1993
1995 Unless otherwise specified, the arguments are strings.
1996 Pars
1998 define the signature of your function
2000 OtherPars
2002 arguments which are not pdls. Default: nothing. This is a semi-colon
2004 separated list of arguments, e.g., "OtherPars=>'int k; double value;
2005 PerlIO *fp'". See $COMP(x) and also the same entry in Appendix B.
2006 Code
2008 the actual code that implements the functionality; several PP macros
2010 and PP functions are recognised in the string value
2011 HandleBad
2013 If set to 1, the routine is assumed to support bad values and the code
2015 in the BadCode key is used if bad values are present; it also sets
2016 things up so that the $ISBAD() etc macros can be used. If set to 0,
2017 cause the routine to print a warning if any of the input ndarrays have
2018 their bad flag set.
2019 BadCode
2021 Give the code to be used if bad values may be present in the input
2023 ndarrays. Only used if "HandleBad => 1". If "HandleBad" is true and
2024 "BadCode" is not supplied, the "Code" section will be reused, on the
2025 assumption it will use "#ifdef PDL_BAD_CODE" to handle bad values. As
2026 of 2.073, you can, and are recommended to, use
2027 "PDL_IF_BAD(iftrue,iffalse)".
2028 CopyBadStatusCode
2030 As of 2.079, this is deprecated due to being largely unnecessary;
2032 instead, just use $PDLSTATESETBAD(pdlname) in your "Code" section and
2033 the badflag setting will be propagated to all its parents and children.
2034 The default code here sets the bad flag of the output ndarrays if
2036 $BADFLAGCACHE() is true after the code has been evaluated. Sometimes
2037 "CopyBadStatusCode" is set to an empty string, with the responsibility
2038 of setting the badflag of the output ndarray left to the "BadCode"
2039 section (e.g. the "xxxover" routines in Basic/Primitive/primitive.pd).
2040 GenericTypes
2042 An array reference. The array may contain any subset of the one-
2044 character strings given below, which specify which types your operation
2045 will accept. The meaning of each type is:
2046 B - signed byte (i.e. signed char)
2048 S - signed short (two-byte integer)
2049 U - unsigned short
2050 L - signed long (four-byte integer, int on 32 bit systems)
2051 N - signed integer for indexing ndarray elements (platform & Perl-dependent size)
2052 Q - signed long long (eight byte integer)
2053 F - float
2054 D - double
2055 G - complex float
2056 C - complex double
2057 This is very useful (and important!) when interfacing an external
2059 library. Default: [qw/B S U L N Q F D/]
2060 Inplace
2062 Mark a function as being able to work inplace.
2064 Inplace => 1 if Pars => 'a(); [o]b();'
2066 Inplace => ['a'] if Pars => 'a(); b(); [o]c();'
2067 Inplace => ['a','c'] if Pars => 'a(); b(); [o]c(); [o]d();'
2068 If bad values are being used, care must be taken to ensure the
2070 propagation of the badflag when inplace is being used; for instance see
2071 the code for "setbadtoval" in Basic/Bad/bad.pd.
2072 Doc
2074 Used to specify a documentation string in Pod format. See PDL::Doc for
2076 information on PDL documentation conventions. Note: in the special case
2077 where the PP 'Doc' string is one line this is implicitly used for the
2078 quick reference AND the documentation!
2079 If the Doc field is omitted PP will generate default documentation
2081 (after all it knows about the Signature).
2082 If you really want the function NOT to be documented in any way at this
2084 point (e.g. for an internal routine, or because you are doing it
2085 elsewhere in the code) explicitly specify "Doc=>undef".
2086 BadDoc
2088 Contains the text returned by the "badinfo" command (in "perldl") or
2090 the "-b" switch to the "pdldoc" shell script. In many cases, you will
2091 not need to specify this, since the information can be automatically
2092 created by PDL::PP. However, as befits computer-generated text, it's
2093 rather stilted; it may be much better to do it yourself!
2094 NoPthread
2096 Optional flag to indicate the PDL function should not use processor
2098 threads (i.e. pthreads or POSIX threads) to split up work across
2099 multiple CPU cores. This option is typically set to 1 if the underlying
2100 PDL function is not threadsafe. If this option isn't present, then the
2101 function is assumed to be threadsafe. This option only applies if PDL
2102 has been compiled with POSIX threads enabled.
2103 PMCode
2105 pp_def('funcname',
2107 Pars => 'a(); [o] b();',
2108 PMCode => 'sub PDL::funcname {
2109 return PDL::_funcname_int(@_) if @_ == 2; # output arg "b" supplied
2110 PDL::_funcname_int(@_, my $out = PDL->null);
2111 $out;
2112 }',
2113 # ...
2114 );
2115 PDL functions allow "[o]" ndarray arguments into which you want the
2117 output saved. This is handy because you can allocate an output ndarray
2118 once and reuse it many times; the alternative would be for PDL to
2119 create a new ndarray each time, which may waste compute cycles or, more
2120 likely, RAM.
2121 PDL functions check the number of arguments they are given, and call
2123 "croak" if given the wrong number. By default (with no "PMCode"
2124 supplied), any output arguments may be omitted, and PDL::PP provides
2125 code that can handle this by creating "null" objects, passing them to
2126 your code, then returning them on the stack.
2127 If you do supply "PMCode", the rest of PDL::PP assumes it will be a
2129 string that defines a Perl function with the function's name in the
2130 "pp_bless" package ("PDL" by default). As the example implies, the PP-
2131 generated function name will change from "<funcname>", to
2132 "_<funcname>_int". As also shown above, you will need to supply all
2133 ndarrays in the exact order specified in the signature: output ndarrays
2134 are not optional, and the PP-generated function will not return
2135 anything.
2136 PMFunc
2138 When pp_def generates functions, it typically defines them in the PDL
2140 package. Then, in the .pm file that it generates for your module, it
2141 typically adds a line that essentially copies that function into your
2142 current package's symbol table with code that looks like this:
2143 *func_name = \&PDL::func_name;
2145 It's a little bit smarter than that (it knows when to wrap that sort of
2147 thing in a BEGIN block, for example, and if you specified something
2148 different for pp_bless), but that's the gist of it. If you don't care
2149 to import the function into your current package's symbol table, you
2150 can specify
2151 PMFunc => '',
2153 PMFunc has no other side-effects, so you could use it to insert
2155 arbitrary Perl code into your module if you like. However, you should
2156 use pp_addpm if you want to add Perl code to your module.
2157 ReadDataFuncName
2159 Allows overriding the default function-name, for reading data
2161 transformed by this operation. Mainly used internally to set it to
2162 "NULL", in which case a default affine-orientated function will be
2163 called instead.
2164 WriteBackDataFuncName
2166 As above, but for writing transformed data from a child of this
2168 transformation back to the parent when "BackCode" is supplied.
2169 AffinePriv
2171 Flag to indicate this is an affine transformation whose "Priv"
2173 (contents of the "pdl_trans") contains data that will need allocating
2174 and freeing.
2175 GlobalNew
2177 If supplied, will prevent generation of an XS function, and assigns the
2179 generated C "run" function into the named slot in the "Core" struct.
2180 This is not used as of 2.058, and instead the relevant C functions are
2181 in pdlaffine.c.
2182 P2Child
2184 Forces "Pars" to be "PARENT" and "CHILD", the function's "GenericTypes"
2186 to be all of them, no "HaveBroadcasting" or "CallCopy", and turns on
2187 "DefaultFlow" (so do not supply any of those args). Intended for
2188 affine transformations with dataflow.
2189 DefaultFlow
2191 If true, sets in the "pdl_transvtable" (see PDL::Internals) the
2193 "iflags" such that the trans will start with dataflow both forwards and
2194 backwards.
2195 HaveBroadcasting
2197 Default true. If so, generate code implementing broadcasting (see
2199 PDL::Indexing).
2200 CallCopy
2202 For parameters that get created, normally the "PDL->initialize" will be
2204 used (or on a subclass). If this is true (which is the default for
2205 simple functions i.e. 2-arg with 0-dim signatures), instead the first
2206 argument's "copy" method will be used.
2207 TwoWay
2209 If true, sets in the "pdl_transvtable" (see PDL::Internals) the
2211 "iflags" such as to inform the trans's error checks connected to
2212 dataflow.
2213 Identity
2215 If true, sets "RedoDims" "EquivCPOffsCode" "HandleBad" "P2Child"
2217 "TwoWay" such that the function is a dataflowing identity
2218 transformation.
2219 BackCode
2221 For dataflowing functions, this value (which gets parsed) overrides the
2223 operation of that from children ndarrays to parents.
2224 BadBackCode
2226 Same but taking account of bad values.
2228 EquivCPOffsCode
2230 If supplied, allows concise control of copying to Child from Parent the
2232 data considered Equivalent at each given Offset (hence the name); the
2233 "Code" and "BackCode" will be generated from this.
2234 Example:
2236 pp_def(
2238 '_clump_int',
2239 OtherPars => 'int n',
2240 P2Child => 1,
2241 RedoDims => # omitted
2242 EquivCPOffsCode => '
2243 PDL_Indx i;
2244 for(i=0; i<$PDL(CHILD)->nvals; i++) $EQUIVCPOFFS(i,i);
2245 ',
2246 );
2247
2249 Macros
2250 $variablename_from_sig()
2251 access a pdl (by its name) that was specified in the signature
2253 $COMP(x)
2255 access a value in the private data structure of this transformation
2257 (mainly used to use an argument that is specified in the "OtherPars"
2258 section)
2259 $SIZE(n)
2261 replaced at runtime by the actual size of a named dimension (as
2263 specified in the signature)
2264 $GENERIC()
2266 replaced by the C type that is equal to the runtime type of the
2268 operation
2269 $P(a)
2271 a pointer to the data of the PDL named "a" in the signature. Useful for
2273 interfacing to C functions
2274 $PP(a)
2276 a physical pointer access to pdl "a"; mainly for internal use
2278 $TXYZ(AlternativeX,AlternativeY,AlternativeZ)
2280 expansion alternatives according to runtime type of operation, where
2282 XXX is some string that is matched by "/[BSULNQFD+]/".
2283 $PDL(a)
2285 return a pointer to the pdl data structure (pdl *) of ndarray "a"
2287 $ISBAD(a())
2289 returns true if the value stored in a() equals the bad value for this
2291 ndarray. Requires "HandleBad" being set to 1.
2292 $ISGOOD(a())
2294 returns true if the value stored in a() does not equal the bad value
2296 for this ndarray. Requires "HandleBad" being set to 1.
2297 $SETBAD(a())
2299 Sets a() to equal the bad value for this ndarray. Requires "HandleBad"
2301 being set to 1.
2302 $PRIV()
2304 To access fields in the "pdl_trans", eg $PRIV(offs).
2306 $CROAK()
2308 Returns a "pdl_error" with the supplied (var-args) message, adding the
2310 function name at the start, which will cause a "barf" within the Perl
2311 code. This is (as of 2.064) a change in PDL functions' API, so that
2312 callers can handle exceptions in their preferred way, which may not use
2313 Perl at all.
2314 $EQUIVCPOFFS()
2316 Copy from the "PARENT" parameter at the first given offset, to the
2318 "CHILD" parameter at the second given offset.
2319 $EQUIVCPTRUNC()
2321 Similar, but if the expression given as the third parameter is false,
2323 instead set the "CHILD"'s value to 0.
2324 $DOCOMPALLOC()
2326 Allocates memory for any "Comp" arrays, after their size has been
2328 determined, e.g. here after $COMP(whichdims_count) has been set:
2329 Comp => 'PDL_Indx whichdims[$COMP(whichdims_count)]',
2331 $DOPRIVALLOC()
2333 As above, except the key is "Priv"; because it is "Priv", this is only
2335 for entries in the "pdl_trans" itself, and almost certainly only for
2336 operations where "AffinePriv" is true.
2337 $SETNDIMS()
2339 For affine transformations (specifically, ones which set P2Child to
2341 true), set the child's "ndims" to the given value and allocate a
2342 suitably-sized array of dimension values.
2343 $SETDIMS()
2345 Similarly for affine transformations, after the above and then the
2347 actual dimension sizes are set, use this to resize the child ndarray to
2348 the right size.
2349 $SETDELTABROADCASTIDS()
2351 Similarly again, this sets the child's "nbroadcastids" to the same as
2353 the parent's, allocates space for the "broadcastids", then sets the
2354 child's ones to the same as the parent's plus the given value.
2355 To get a flavour of what "broadcastids" are for, in the normal way of
2357 things the first (0th) one in the parent is the highest dimension-
2358 number in it. See PDL::Indexing for more.
2359 functions
2361 "loop(DIMS) %{ ... %}"
2362 loop over named dimensions; limits are generated automatically by PP
2364 "broadcastloop %{ ... %}"
2366 enclose following code in a broadcast loop
2368 As of 2.075, "threadloop" is a deprecated alias for this.
2370 "types(TYPES) %{ ... %}"
2372 execute following code if type of operation is any of "TYPES"
2374
2376 A number of functions are imported when you "use PDL::PP". These
2377 include functions that control the generated C or XS code, functions
2378 that control the generated Perl code, and functions that manipulate the
2379 packages and symbol tables into which the code is created.
2380 Generating C and XS Code
2382 PDL::PP's main purpose is to make it easy for you to wrap the
2383 broadcasting engine around your own C code, but you can do some other
2384 things, too.
2385 pp_def
2387 Used to wrap the broadcasting engine around your C code. Virtually all
2389 of this document discusses the use of pp_def.
2390 pp_done
2392 Indicates you are done with PDL::PP and that it should generate its .xs
2394 and .pm files based upon the other pp_* functions that you have called.
2395 This function takes no arguments.
2396 pp_addxs
2398 This lets you add XS code to your .xs file. This is useful if you want
2400 to create Perl-accessible functions that invoke C code but cannot or
2401 should not invoke the broadcasting engine. XS is the standard means by
2402 which you wrap Perl-accessible C code. You can learn more at perlxs.
2403 pp_add_boot
2405 This function adds whatever string you pass to the XS BOOT section. The
2407 BOOT section is C code that gets called by Perl when your module is
2408 loaded and is useful for automatic initialization. You can learn more
2409 about XS and the BOOT section at perlxs.
2410 pp_addhdr
2412 Adds pure-C code to your XS file. XS files are structured such that
2414 pure C code must come before XS specifications. This allows you to
2415 specify such C code.
2416 pp_boundscheck
2418 PDL normally checks the bounds of your accesses before making them. You
2420 can turn that on or off at runtime by setting
2421 MyPackage::set_boundscheck. This function allows you to remove that
2422 runtime flexibility and never do bounds checking. It also returns the
2423 current boundschecking status if called without any argumens.
2424 NOTE: I have not found anything about bounds checking in other
2426 documentation. That needs to be addressed.
2427 Generating Perl Code
2429 Many functions imported when you use PDL::PP allow you to modify the
2430 contents of the generated .pm file. In addition to pp_def and pp_done,
2431 the role of these functions is primarily to add code to various parts
2432 of your generated .pm file.
2433 pp_addpm
2435 Adds Perl code to the generated .pm file. PDL::PP actually keeps track
2437 of three different sections of generated code: the Top, the Middle, and
2438 the Bottom. You can add Perl code to the Middle section using the one-
2439 argument form, where the argument is the Perl code you want to supply.
2440 In the two-argument form, the first argument is an anonymous hash with
2441 only one key that specifies where to put the second argument, which is
2442 the string that you want to add to the .pm file. The hash is one of
2443 these three:
2444 {At => 'Top'}
2446 {At => 'Middle'}
2447 {At => 'Bot'}
2448 For example:
2450 pp_addpm({At => 'Bot'}, <<POD);
2452 =head1 Some documentation
2454 I know I'm typing this in the middle of my file, but it'll go at
2456 the bottom.
2457 =cut
2459 POD
2461 Warning: If, in the middle of your .pd file, you put documentation
2463 meant for the bottom of your pod, you will thoroughly confuse CPAN. On
2464 the other hand, if in the middle of your .pd file, you add some Perl
2465 code destined for the bottom or top of your .pm file, you only have
2466 yourself to confuse. :-)
2467 pp_beginwrap
2469 Adds BEGIN-block wrapping. Certain declarations can be wrapped in BEGIN
2471 blocks, though the default behavior is to have no such wrapping.
2472 pp_addbegin
2474 Sets code to be added to the top of your .pm file, even above code that
2476 you specify with "pp_addpm({At => 'Top'}, ...)". Unlike pp_addpm,
2477 calling this overwrites whatever was there before. Generally, you
2478 probably shouldn't use it.
2479 Tracking Line Numbers
2481 When you get compile errors, either from your C-like code or your Perl
2482 code, it can help to make those errors back to the line numbers in the
2483 source file at which the error occurred.
2484 pp_line_numbers
2486 Takes a line number and a (usually long) string of code. The line
2488 number should indicate the line at which the quote begins. This is
2489 usually Perl's "__LINE__" literal, unless you are using heredocs, in
2490 which case it is "__LINE__ + 1". The returned string has #line
2491 directives interspersed to help the compiler report errors on the
2492 proper line.
2493 Modifying the Symbol Table and Export Behavior
2495 PDL::PP usually exports all functions generated using pp_def, and
2496 usually installs them into the PDL symbol table. However, you can
2497 modify this behavior with these functions.
2498 pp_bless
2500 Sets the package (symbol table) to which the XS code is added. The
2502 default is PDL, which is generally what you want. If you use the
2503 default blessing and you create a function myfunc, then you can do the
2504 following:
2505 $ndarray->myfunc(<args>);
2507 PDL::myfunc($ndarray, <args>);
2508 On the other hand, if you bless your functions into another package,
2510 you cannot invoke them as PDL methods, and must invoke them as:
2511 MyPackage::myfunc($ndarray, <args>);
2513 Of course, you could always use the PMFunc key to add your function to
2515 the PDL symbol table, but why do that?
2516 pp_add_isa
2518 Adds to the list of modules from which your module inherits. The
2520 default list is
2521 qw(PDL::Exporter DynaLoader)
2523 pp_core_importlist
2525 At the top of your generated .pm file is a line that looks like this:
2527 use PDL::Core;
2529 You can modify that by specifying a string to pp_core_importlist. For
2531 example,
2532 pp_core_importlist('::Blarg');
2534 will result in
2536 use PDL::Core::Blarg;
2538 You can use this, for example, to add a list of symbols to import from
2540 PDL::Core. For example:
2541 pp_core_importlist(" ':Internal'");
2543 will lead to the following use statement:
2545 use PDL::Core ':Internal';
2547 pp_setversion
2549 Sets your module's version. The version must be consistent between the
2551 .xs and the .pm file, and is used to ensure that your Perl's libraries
2552 do not suffer from version skew.
2553 pp_add_exported
2555 Adds to the export list whatever names you give it. Functions created
2557 using pp_def are automatically added to the list. This function is
2558 useful if you define any Perl functions using pp_addpm or pp_addxs that
2559 you want exported as well.
2560 pp_export_nothing
2562 This resets the list of exported symbols to nothing. This is probably
2564 better called "pp_export_clear", since you can add exported symbols
2565 after calling "pp_export_nothing". When called just before calling
2566 pp_done, this ensures that your module does not export anything, for
2567 example, if you only want programmers to use your functions as methods.
2568
2570 For the concepts of broadcasting and slicing check PDL::Indexing.
2571 PDL::Internals
2573 PDL::BadValues for information on bad values
2575 perlxs, perlxstut
2577 Practical Magick with C, PDL, and PDL::PP -- a guide to compiled add-
2579 ons for PDL <https://arxiv.org/abs/1702.07753>
2580
2582 Although PDL::PP is quite flexible and thoroughly used, there are
2583 surely bugs. First amongst them: this documentation needs a thorough
2584 revision.
2585
2587 Copyright(C) 1997 Tuomas J. Lukka (lukka@fas.harvard.edu), Karl
2588 Glaazebrook (kgb@aaocbn1.aao.GOV.AU) and Christian Soeller
2589 (c.soeller@auckland.ac.nz). All rights reserved. Documentation updates
2590 Copyright(C) 2011 David Mertens (dcmertens.perl@gmail.com). This
2591 documentation is licensed under the same terms as Perl itself.
2592perl v5.36.0 2023-01-20 PP(1)