1PP(1) User Contributed Perl Documentation PP(1)
2
6 PDL::PP - Generate PDL routines from concise descriptions
7
9 # let PDL::PP tell you what it's doing
10 $::PP_VERBOSE = 1;
11 pp_def(
12 'sumover',
13 Pars => 'a(n); [o]b();',
14 Code => q{
15 double tmp=0;
16 loop(n) %{
17 tmp += $a();
18 %}
19 $b() = tmp;
20 },
21 );
22 pp_done();
24 # do not call exit() as some processing can be done in same process
25
27 Here is a quick reference list of the functions provided by PDL::PP.
28 pp_add_boot
30 Add code to the BOOT section of generated XS file
31 pp_add_exported
33 Add functions to the list of exported functions
34 pp_add_isa
36 Add entries to the @ISA list
37 pp_addbegin
39 Sets code to be added at the top of the generate .pm file
40 pp_addhdr
42 Add code and includes to C section of the generated XS file.
43 When used in a module that is "multi-C" (one .c file per "pp_def"ed
45 function), you need to bear in mind that as each one is generated, all
46 the "pp_addhdr" so far will be included. Therefore, if you add C
47 functions, make sure to make them "static" to avoid clashes with later
48 .c files. But a better practice is make them be separate C files, with
49 any necessary .h to be included by them and the .pd file. You can then
50 add them to your Makefile.PL (note this is the "_int" version, see
51 separate notes on how to "opt-in" for your own modules):
52 my @pack = (["pnm.pd", qw(Pnm PDL::IO::Pnm)]);
54 my %hash = pdlpp_stdargs_int(@pack);
55 $hash{OBJECT} .= ' get$(OBJ_EXT)';
56 sub MY::postamble { pdlpp_postamble_int(@pack); };
57 WriteMakefile(%hash);
58 pp_addpm
60 Add code to the generated .pm file
61 pp_addxs
63 Add extra XS code to the generated XS file
64 pp_add_macros
66 Add extra "$MACRO()" definitions for these functions. Note these
67 generate C code. As of 2.080, they will be passed the list of arguments
68 they were called with, rather than a single string, split like the C
69 pre-processor on commas except if in "" or "()", with leading and
70 trailing whitespace removed.
71 pp_add_macros(SUCC => sub { "($_[0] + 1)" });
73 # ...
74 Code => '$a() = $SUCC($b());',
75 pp_beginwrap
77 Add BEGIN-block wrapping to code for the generated .pm file
78 pp_bless
80 Sets the package to which the XS code is added (default is PDL)
81 pp_boundscheck
83 Control state of PDL bounds checking activity
84 pp_core_importList
86 Specify what is imported from PDL::Core
87 pp_def
89 Define a new PDL function
90 pp_deprecate_module
92 Add runtime and POD warnings about a module being deprecated
93 pp_done
95 Mark the end of PDL::PP definitions in the file
96 pp_export_nothing
98 Clear out the export list for your generated module
99 pp_line_numbers
101 Add line number information to simplify debugging of PDL::PP code
102
104 For an alternate introduction to PDL::PP, see Practical Magick with C,
105 PDL, and PDL::PP -- a guide to compiled add-ons for PDL
106 <https://arxiv.org/abs/1702.07753>.
107 Why do we need PP? Several reasons: firstly, we want to be able to
109 generate subroutine code for each of the PDL datatypes (PDL_Byte,
110 PDL_Short, etc). AUTOMATICALLY. Secondly, when referring to slices of
111 PDL arrays in Perl (e.g. "$x->slice('0:10:2,:')" or other things such
112 as transposes) it is nice to be able to do this transparently and to be
113 able to do this 'in-place' - i.e, not to have to make a memory copy of
114 the section. PP handles all the necessary element and offset arithmetic
115 for you. There are also the notions of broadcasting (repeated calling
116 of the same routine for multiple slices, see PDL::Indexing) and
117 dataflow (see PDL::Dataflow, and "DefaultFlow") which use of PP allows.
118 In much of what follows we will assume familiarity of the reader with
120 the concepts of implicit and explicit broadcasting and index
121 manipulations within PDL. If you have not yet heard of these concepts
122 or are not very comfortable with them it is time to check
123 PDL::Indexing.
124 As you may appreciate from its name PDL::PP is a Pre-Processor, i.e.
126 it expands code via substitutions to make real C-code. Technically, the
127 output is XS code (see perlxs) but that is very close to C.
128 So how do you use PP? Well for the most part you just write ordinary C
130 code except for special PP constructs which take the form:
131 $something(something else)
133 or:
135 PPfunction %{
137 <stuff>
138 %}
139 The most important PP construct is the form "$array()". Consider the
141 very simple PP function to sum the elements of a 1D vector (in fact
142 this is very similar to the actual code used by 'sumover'):
143 pp_def('sumit',
145 Pars => 'a(n); [o]b();',
146 Code => q{
147 double tmp;
148 tmp = 0;
149 loop(n) %{
150 tmp += $a();
151 %}
152 $b() = tmp;
153 }
154 );
155 What's going on? The "Pars =>" line is very important for PP - it
157 specifies all the arguments and their dimensionality. We call this the
158 signature of the PP function (compare also the explanations in
159 PDL::Indexing). In this case the routine takes a 1-D function as input
160 and returns a 0-D scalar as output. The "$a()" PP construct is used to
161 access elements of the array a(n) for you - PP fills in all the
162 required C code.
163 You will notice that we are using the "q{}" single-quote operator. This
165 is not an accident. You generally want to use single quotes to denote
166 your PP Code sections. PDL::PP uses "$var()" for its parsing and if you
167 don't use single quotes, Perl will try to interpolate "$var()". Also,
168 using the single quote "q" operator with curly braces makes it look
169 like you are creating a code block, which is What You Mean. (Perl is
170 smart enough to look for nested curly braces and not close the quote
171 until it finds the matching curly brace, so it's safe to have nested
172 blocks.) Under other circumstances, such as when you're stitching
173 together a Code block using string concatenations, it's often easiest
174 to use real single quotes as
175 Code => 'something'.$interpolatable.'somethingelse;'
177 In the simple case here where all elements are accessed the PP
179 construct "loop(n) %{ ... %}" is used to loop over all elements in
180 dimension "n". Note this feature of PP: ALL DIMENSIONS ARE SPECIFIED
181 BY NAME.
182 This is made clearer if we avoid the PP loop() construct and write the
184 loop explicitly using conventional C:
185 pp_def('sumit',
187 Pars => 'a(n); [o]b();',
188 Code => q{
189 PDL_Indx i,n_size;
190 double tmp;
191 n_size = $SIZE(n);
192 tmp = 0;
193 for(i=0; i<n_size; i++) {
194 tmp += $a(n=>i);
195 }
196 $b() = tmp;
197 },
198 );
199 which does the same as before, but is more long-winded. You can see to
201 get element "i" of a() we say "$a(n=>i)" - we are specifying the
202 dimension by name "n". In 2D we might say:
203 Pars=>'a(m,n);',
205 ...
206 tmp += $a(m=>i,n=>j);
207 ...
208 The syntax "m=>i" borrows from Perl hashes, which are in fact used in
210 the implementation of PP. One could also say "$a(n=>j,m=>i)" as order
211 is not important.
212 You can also see in the above example the use of another PP construct -
214 $SIZE(n) to get the length of the dimension "n".
215 It should, however, be noted that you shouldn't write an explicit
217 C-loop when you could have used the PP "loop" construct since PDL::PP
218 checks automatically the loop limits for you, usage of "loop" makes the
219 code more concise, etc. But there are certainly situations where you
220 need explicit control of the loop and now you know how to do it ;).
221 To revisit 'Why PP?' - the above code for sumit() will be generated for
223 each data-type. It will operate on slices of arrays 'in-place'. It will
224 broadcast automatically - e.g. if a 2D array is given it will be called
225 repeatedly for each 1D row (again check PDL::Indexing for the details
226 of broadcasting). And then b() will be a 1D array of sums of each row.
227 We could call it with $x->transpose to sum the columns instead. And
228 Dataflow tracing etc. will be available.
229 You can see PP saves the programmer from writing a lot of needlessly
231 repetitive C-code -- in our opinion this is one of the best features of
232 PDL making writing new C subroutines for PDL an amazingly concise
233 exercise. A second reason is the ability to make PP expand your concise
234 code definitions into different C code based on the needs of the
235 computer architecture in question. Imagine for example you are lucky to
236 have a supercomputer at your hands; in that case you want PDL::PP
237 certainly to generate code that takes advantage of the
238 vectorising/parallel computing features of your machine (this a project
239 for the future). In any case, the bottom line is that your unchanged
240 code should still expand to working XS code even if the internals of
241 PDL changed.
242 Also, because you are generating the code in an actual Perl script,
244 there are many fun things that you can do. Let's say that you need to
245 write both sumit (as above) and multit. With a little bit of
246 creativity, we can do
247 for({Name => 'sumit', Init => '0', Op => '+='},
249 {Name => 'multit', Init => '1', Op => '*='}) {
250 pp_def($_->{Name},
251 Pars => 'a(n); [o]b();',
252 Code => '
253 double tmp;
254 tmp = '.$_->{Init}.';
255 loop(n) %{
256 tmp '.$_->{Op}.' $a();
257 %}
258 $b() = tmp;
259 ');
260 }
261 which defines both the functions easily. Now, if you later need to
263 change the signature or dimensionality or whatever, you only need to
264 change one place in your code. Yeah, sure, your editor does have 'cut
265 and paste' and 'search and replace' but it's still less bothersome and
266 definitely more difficult to forget just one place and have strange
267 bugs creep in. Also, adding 'orit' (bitwise or) later is a one-liner.
268 And remember, you really have Perl's full abilities with you - you can
270 very easily read any input file and make routines from the information
271 in that file. For simple cases like the above, the author (Tjl)
272 currently favors the hash syntax like the above - it's not too much
273 more characters than the corresponding array syntax but much easier to
274 understand and change.
275 As of 2.064, the "Code" must not just "return", since the signature of
277 the generated functions has changed from returning "void" to returning
278 a "pdl_error", which is pre-initialised to a successful return value.
279 You can easily just replace the "return;" with "return PDL_err;", which
280 is the variable's name.
281 We should mention here also the ability to get the pointer to the
283 beginning of the data in memory - a prerequisite for interfacing PDL to
284 some libraries. This is handled with the "$P(var)" directive, see
285 below.
286 When starting work on a new pp_def'ined function, if you make a
288 mistake, you will usually find a pile of compiler errors indicating
289 line numbers in the generated XS file. If you know how to read XS files
290 (or if you want to learn the hard way), you could open the generated XS
291 file and search for the line number with the error. However, a recent
292 addition to PDL::PP helps report the correct line number of your
293 errors: "pp_line_numbers". Working with the original summit example, if
294 you had a mis-spelling of tmp in your code, you could change the
295 (erroneous) code to something like this and the compiler would give you
296 much more useful information:
297 pp_def('sumit',
299 Pars => 'a(n); [o]b();',
300 Code => pp_line_numbers(__LINE__, q{
301 double tmp;
302 tmp = 0;
303 loop(n) %{
304 tmp += $a();
305 %}
306 $b() = rmp;
307 })
308 );
309 For the above situation, my compiler tells me:
311 ...
313 test.pd:15: error: 'rmp' undeclared (first use in this function)
314 ...
315 In my example script (called test.pd), line 15 is exactly the line at
317 which I made my typo: "rmp" instead of "tmp".
318 So, after this quick overview of the general flavour of programming PDL
320 routines using PDL::PP let's summarise in which circumstances you
321 should actually use this preprocessor/precompiler. You should use
322 PDL::PP if you want to
323 • interface PDL to some external library
325 • write some algorithm that would be slow if coded in Perl (this is
327 not as often as you think; take a look at broadcasting and dataflow
328 first).
329 • be a PDL developer (and even then it's not obligatory)
331
333 Because of its architecture, PDL::PP can be both flexible and easy to
334 use on the one hand, yet exuberantly complicated at the same time.
335 Currently, part of the problem is that error messages are not very
336 informative and if something goes wrong, you'd better know what you are
337 doing and be able to hack your way through the internals (or be able to
338 figure out by trial and error what is wrong with your args to
339 "pp_def"). Although work is being done to produce better warnings, do
340 not be afraid to send your questions to the mailing list if you run
341 into trouble.
342
344 Now that you have some idea how to use "pp_def" to define new PDL
345 functions it is time to explain the general syntax of "pp_def".
346 "pp_def" takes as arguments first the name of the function you are
347 defining and then a hash list that can contain various keys.
348 Based on these keys PP generates XS code and a .pm file. The function
350 "pp_done" (see example in the SYNOPSIS) is used to tell PDL::PP that
351 there are no more definitions in this file and it is time to generate
352 the .xs and
353 .pm file.
354 As a consequence, there may be several pp_def() calls inside a file (by
356 convention files with PP code have the extension .pd or .pp) but
357 generally only one pp_done().
358 There are two main different types of usage of pp_def(), the 'data
360 operation' and 'slice operation' prototypes.
361 The 'data operation' is used to take some data, mangle it and output
363 some other data; this includes for example the '+' operation, matrix
364 inverse, sumover etc and all the examples we have talked about in this
365 document so far. Implicit and explicit broadcasting and the creation of
366 the result are taken care of automatically in those operations. You can
367 even do dataflow with "sumit", "sumover", etc (don't be dismayed if you
368 don't understand the concept of dataflow in PDL very well yet; it is
369 still very much experimental).
370 The 'slice operation' is a different kind of operation: in a slice
372 operation, you are not changing any data, you are defining
373 correspondences between different elements of two ndarrays (examples
374 include the index manipulation/slicing function definitions in the file
375 slices.pd that is part of the PDL distribution; but beware, this is not
376 introductory level stuff).
377 To support bad values, additional keys are required for "pp_def", as
379 explained below.
380 If you are just interested in communicating with some external library
382 (for example some linear algebra/matrix library), you'll usually want
383 the 'data operation' so we are going to discuss that first.
384
386 A simple example
387 In the data operation, you must know what dimensions of data you need.
388 First, an example with scalars:
389 pp_def('add',
391 Pars => 'a(); b(); [o]c();',
392 Code => '$c() = $a() + $b();'
393 );
394 That looks a little strange but let's dissect it. The first line is
396 easy: we're defining a routine with the name 'add'. The second line
397 simply declares our parameters and the parentheses mean that they are
398 scalars. We call the string that defines our parameters and their
399 dimensionality the signature of that function. For its relevance with
400 regard to broadcasting and index manipulations check the PDL::Indexing
401 man page.
402 The third line is the actual operation. You need to use the dollar
404 signs and parentheses to refer to your parameters (this will probably
405 change at some point in the future, once a good syntax is found).
406 These lines are all that is necessary to actually define the function
408 for PDL (well, actually it isn't; you additionally need to write a
409 Makefile.PL (see below) and build the module (something like 'perl
410 Makefile.PL; make'); but let's ignore that for the moment). So now you
411 can do
412 use MyModule;
414 $x = pdl 2,3,4;
415 $y = pdl 5;
416 $c = add($x,$y);
418 # or
419 add($x,$y,($c=null)); # Alternative form, useful if $c has been
420 # preset to something big, not useful here.
421 and have broadcasting work correctly (the result is $c == [7 8 9]).
423 The Pars section: the signature of a PP function
425 Seeing the above example code you will most probably ask: what is this
426 strange "$c=null" syntax in the second call to our new "add" function?
427 If you take another look at the definition of "add" you will notice
428 that the third argument "c" is flagged with the qualifier "[o]" which
429 tells PDL::PP that this is an output argument. So the above call to add
430 means 'create a new $c from scratch with correct dimensions' - "null"
431 is a special token for 'empty ndarray' (you might ask why we haven't
432 used the value "undef" to flag this instead of the PDL specific "null";
433 we are currently thinking about it ;).
434 [This should be explained in some other section of the manual as
436 well!!] The reason for having this syntax as an alternative is that if
437 you have really huge ndarrays, you can do
438 $c = PDL->null;
440 for(some long loop) {
441 # munge a,b
442 add($x,$y,$c);
443 # munge c, put something back to x,y
444 }
445 and avoid allocating and deallocating $c each time. It is allocated
447 once at the first add() and thereafter the memory stays until $c is
448 destroyed.
449 If you just say
451 $c = add($x,$y);
453 the code generated by PP will automatically fill in "$c=null" and
455 return the result. If you want to learn more about the reasons why
456 PDL::PP supports this style where output arguments are given as last
457 arguments check the PDL::Indexing man page.
458 "[o]" is not the only qualifier a pdl argument can have in the
460 signature. Another important qualifier is the "[t]" option which flags
461 a pdl as temporary. What does that mean? You tell PDL::PP that this
462 pdl is only used for temporary results in the course of the calculation
463 and you are not interested in its value after the computation has been
464 completed. But why should PDL::PP want to know about this in the first
465 place? The reason is closely related to the concepts of pdl auto
466 creation (you heard about that above) and implicit broadcasting. If you
467 use implicit broadcasting the dimensionality of automatically created
468 pdls is actually larger than that specified in the signature. With
469 "[o]" flagged pdls will be created so that they have the additional
470 dimensions as required by the number of implicit broadcast dimensions.
471 When creating a temporary pdl, however, it will always only be made big
472 enough so that it can hold the result for one iteration in a broadcast
473 loop, i.e. as large as required by the signature. So less memory is
474 wasted when you flag a pdl as temporary. Secondly, you can use output
475 auto creation with temporary pdls even when you are using explicit
476 broadcasting which is forbidden for normal output pdls flagged with
477 "[o]" (see PDL::Indexing).
478 As of 2.073, the user is unable to pass a "[t]" parameter, and PDL will
480 create and size it to its notional size, times the number of threads.
481 Here is an example where we use the "[t]" qualifier. We define the
483 function "callf" that calls a C routine "f" which needs a temporary
484 array of the same size and type as the array "a" (sorry about the
485 forward reference for $P; it's a pointer access, see below) :
486 pp_def('callf',
488 Pars => 'a(n); [t] tmp(n); [o] b()',
489 Code => 'PDL_Indx ns = $SIZE(n);
490 f($P(a),$P(b),$P(tmp),ns);
491 '
492 );
493 Another possible qualifier is "[phys]". If given, this means the pdl
495 will have "make_physical" in PDL::Core called on it.
496 Additionally, if it has a specified dimension "d" that has value 1, "d"
498 will not magically be grown if "d" is larger in another pdl with
499 specified dimension "d", and instead an exception will be thrown. E.g.:
500 pp_def('callf',
502 Pars => 'a(n); [phys] b(n); [o] c()',
503 # ...
504 );
505 If "a" had lead dimension of 2 and "b" of 3, an exception will always
507 be thrown. However, if "b" has lead dimension of 1, it would be
508 silently repeated as if it were 2, if it were not a "phys" parameter.
509 Argument dimensions and the signature
511 Now we have just talked about dimensions of pdls and the signature. How
512 are they related? Let's say that we want to add a scalar + the index
513 number to a vector:
514 pp_def('add2',
516 Pars => 'a(n); b(); [o]c(n);',
517 Code => 'loop(n) %{
518 $c() = $a() + $b() + n;
519 %}'
520 );
521 There are several points to notice here: first, the "Pars" argument now
523 contains the n arguments to show that we have a single dimensions in a
524 and c. It is important to note that dimensions are actual entities that
525 are accessed by name so this declares a and c to have the same first
526 dimensions. In most PP definitions the size of named dimensions will be
527 set from the respective dimensions of non-output pdls (those with no
528 "[o]" flag) but sometimes you might want to set the size of a named
529 dimension explicitly through an integer parameter. See below in the
530 description of the "OtherPars" section how that works.
531 Constant argument dimensions in the signature
533 Suppose you want an output ndarray to be created automatically and you
534 know that on every call its dimension will have the same size (say 9)
535 regardless of the dimensions of the input ndarrays. In this case you
536 use the following syntax in the Pars section to specify the size of the
537 dimension:
538 ' [o] y(n=9); '
540 As expected, extra dimensions required by broadcasting will be created
542 if necessary. If you need to assign a named dimension according to a
543 more complicated formula (than a constant) you must use the
544 "RedoDimsCode" key described below.
545 Type conversions and the signature
547 The signature also determines the type conversions that will be
548 performed when a PP function is invoked. So what happens when we invoke
549 one of our previously defined functions with pdls of different type,
550 e.g.
551 add2($x,$y,($ret=null));
553 where $x is of type "PDL_Float" and $y of type "PDL_Short"? With the
555 signature as shown in the definition of "add2" above the datatype of
556 the operation (as determined at runtime) is that of the pdl with the
557 'highest' type (sequence is byte < short < ushort < long < float <
558 double). In the add2 example the datatype of the operation is float ($x
559 has that datatype). All pdl arguments are then type converted to that
560 datatype (they are not converted inplace but a copy with the right type
561 is created if a pdl argument doesn't have the type of the operation).
562 Null pdls don't contribute a type in the determination of the type of
563 the operation. However, they will be created with the datatype of the
564 operation; here, for example, $ret will be of type float. You should be
565 aware of these rules when calling PP functions with pdls of different
566 types to take the additional storage and runtime requirements into
567 account.
568 These type conversions are correct for most functions you normally
570 define with "pp_def". However, there are certain cases where slightly
571 modified type conversion behaviour is desired. For these cases
572 additional qualifiers in the signature can be used to specify the
573 desired properties with regard to type conversion. These qualifiers can
574 be combined with those we have encountered already (the creation
575 qualifiers "[o]" and "[t]"). Let's go through the list of qualifiers
576 that change type conversion behaviour.
577 The most important is the "indx" qualifier which comes in handy when a
579 pdl argument represents indices into another pdl. Let's take a look at
580 an example from "PDL::Ufunc":
581 pp_def('maximum_ind',
583 Pars => 'a(n); indx [o] b()',
584 Code => '$GENERIC() cur;
585 PDL_Indx curind;
586 loop(n) %{
587 if (!n || $a() > cur) {cur = $a(); curind = n;}
588 %}
589 $b() = curind;',
590 );
591 The function "maximum_ind" finds the index of the largest element of a
593 vector. If you look at the signature you notice that the output
594 argument "b" has been declared with the additional "indx" qualifier.
595 This has the following consequences for type conversions: regardless of
596 the type of the input pdl "a" the output pdl "b" will be of type
597 "PDL_Indx" which makes sense since "b" will represent an index into
598 "a".
599 Note that 'curind' is declared as type "PDL_Indx" and not "indx".
601 While most datatype declarations in the 'Pars' section use the same
602 name as the underlying C type, "indx" is a type which is sufficient to
603 handle PDL indexing operations. For 32-bit installs, it can be a
604 32-bit integer type. For 64-bit installs, it will be a 64-bit integer
605 type.
606 Furthermore, if you call the function with an existing output pdl "b"
608 its type will not influence the datatype of the operation (see above).
609 Hence, even if "a" is of a smaller type than "b" it will not be
610 converted to match the type of "b" but stays untouched, which saves
611 memory and CPU cycles and is the right thing to do when "b" represents
612 indices. Also note that you can use the 'indx' qualifier together with
613 other qualifiers (the "[o]" and "[t]" qualifiers). Order is significant
614 -- type qualifiers precede creation qualifiers ("[o]" and "[t]").
615 The above example also demonstrates typical usage of the "$GENERIC()"
617 macro. It expands to the current type in a so called generic loop.
618 What is a generic loop? As you already heard a PP function has a
619 runtime datatype as determined by the type of the pdl arguments it has
620 been invoked with. The PP generated XS code for this function
621 therefore contains a switch like "switch (type) {case PDL_Byte: ...
622 case PDL_Double: ...}" that selects a case based on the runtime
623 datatype of the function (it's called a type ``loop'' because there is
624 a loop in PP code that generates the cases). In any case your code is
625 inserted once for each PDL type into this switch statement. The
626 "$GENERIC()" macro just expands to the respective type in each copy of
627 your parsed code in this "switch" statement, e.g., in the "case
628 PDL_Byte" section "cur" will expand to "PDL_Byte" and so on for the
629 other case statements. I guess you realise that this is a useful macro
630 to hold values of pdls in some code.
631 There are a couple of other qualifiers with similar effects as "indx".
633 For your convenience there are the "float" and "double" qualifiers with
634 analogous consequences on type conversions as "indx". Let's assume you
635 have a very large array for which you want to compute row and column
636 sums with an equivalent of the "sumover" function. However, with the
637 normal definition of "sumover" you might run into problems when your
638 data is, e.g. of type short. A call like
639 sumover($large_pdl,($sums = null));
641 will result in $sums be of type short and is therefore prone to
643 overflow errors if $large_pdl is a very large array. On the other hand
644 calling
645 @dims = $large_pdl->dims; shift @dims;
647 sumover($large_pdl,($sums = zeroes(double,@dims)));
648 is not a good alternative either. Now we don't have overflow problems
650 with $sums but at the expense of a type conversion of $large_pdl to
651 double, something bad if this is really a large pdl. That's where
652 "double" comes in handy:
653 pp_def('sumoverd',
655 Pars => 'a(n); double [o] b()',
656 Code => 'double tmp=0;
657 loop(n) %{ tmp += a(); %}
658 $b() = tmp;',
659 );
660 This gets us around the type conversion and overflow problems. Again,
662 analogous to the "indx" qualifier "double" results in "b" always being
663 of type double regardless of the type of "a" without leading to a type
664 conversion of "a" as a side effect.
665 There is also a special type, "real". The others above are all actual
667 PDL/C datatypes, but "real" is a modifier; if the operation type is
668 real, it has no effect; if it is complex, then the parameter will be
669 the real version - so "cdouble" becomes "double", etc.
670 There is also the converse, "complex". If the operation is already
672 complex, there is no effect; if not, the output will be promoted to the
673 type's "complexversion" in PDL::Type, which defaults to "cfloat". Note
674 this is controlled both by the PDL::Types data, and the code in
675 PDL::PP. NB Because this outputs floating-point data, the inputs will
676 by definition be turned into such. Therefore, it only makes sense to
677 have floating-point "GenericTypes" inputs. If you want to default to
678 coercing inputs to "float", give that as the last "GenericTypes" as the
679 generated XS function defaults to the last-given one. Hence (with the
680 "PMCode" and "Doc" omitted):
681 pp_def('r2C',
683 GenericTypes=>[reverse qw(F D G C)], # last one is default so here = F
684 Pars => 'r(); complex [o]c()',
685 Code => '$c() = $r();'
686 );
687 Finally, there are the "type+" qualifiers where type is one of "int" or
689 "float". What shall that mean. Let's illustrate the "int+" qualifier
690 with the actual definition of sumover:
691 pp_def('sumover',
693 Pars => 'a(n); int+ [o] b()',
694 Code => '$GENERIC(b) tmp=0;
695 loop(n) %{ tmp += a(); %}
696 $b() = tmp;',
697 );
698 As we had already seen for the "int", "float" and "double" qualifiers,
700 a pdl marked with a "type+" qualifier does not influence the datatype
701 of the pdl operation. Its meaning is "make this pdl at least of type
702 "type" or higher, as required by the type of the operation". In the
703 sumover example this means that when you call the function with an "a"
704 of type PDL_Short the output pdl will be of type PDL_Long (just as
705 would have been the case with the "int" qualifier). This again tries to
706 avoid overflow problems when using small datatypes (e.g. byte images).
707 However, when the datatype of the operation is higher than the type
708 specified in the "type+" qualifier "b" will be created with the
709 datatype of the operation, e.g. when "a" is of type double then "b"
710 will be double as well. We hope you agree that this is sensible
711 behaviour for "sumover". It should be obvious how the "float+"
712 qualifier works by analogy. It may become necessary to be able to
713 specify a set of alternative types for the parameters. However, this
714 will probably not be implemented until someone comes up with a
715 reasonable use for it.
716 Note that we now had to specify the $GENERIC macro with the name of the
718 pdl to derive the type from that argument. Why is that? If you
719 carefully followed our explanations you will have realised that in some
720 cases "b" will have a different type than the type of the operation.
721 Calling the '$GENERIC' macro with "b" as argument makes sure that the
722 type will always the same as that of "b" in that part of the generic
723 loop.
724 This is about all there is to say about the "Pars" section in a
726 "pp_def" call. You should remember that this section defines the
727 signature of a PP defined function, you can use several options to
728 qualify certain arguments as output and temporary args and all
729 dimensions that you can later refer to in the "Code" section are
730 defined by name.
731 It is important that you understand the meaning of the signature since
733 in the latest PDL versions you can use it to define broadcasting
734 functions from within Perl, i.e. what we call Perl level broadcasting.
735 Please check PDL::Indexing for details.
736 The Code section
738 The "Code" section contains the actual XS code that will be in the
739 innermost part of a broadcast loop (if you don't know what a broadcast
740 loop is then you still haven't read PDL::Indexing; do it now ;) after
741 any PP macros (like $GENERIC) and PP functions have been expanded (like
742 the "loop" function we are going to explain next).
743 Let's quickly reiterate the "sumover" example:
745 pp_def('sumover',
747 Pars => 'a(n); int+ [o] b()',
748 Code => '$GENERIC(b) tmp=0;
749 loop(n) %{ tmp += a(); %}
750 $b() = tmp;',
751 );
752 The "loop" construct in the "Code" section also refers to the dimension
754 name so you don't need to specify any limits: the loop is correctly
755 sized and everything is done for you, again.
756 Next, there is the surprising fact that "$a()" and "$b()" do not
758 contain the index. This is not necessary because we're looping over n
759 and both variables know which dimensions they have so they
760 automatically know they're being looped over.
761 This feature comes in very handy in many places and makes for much
763 shorter code. Of course, there are times when you want to circumvent
764 this; here is a function which make a matrix symmetric and serves as an
765 example of how to code explicit looping:
766 pp_def('symm',
768 Pars => 'a(n,n); [o]c(n,n);',
769 Code => 'loop(n) %{
770 int n2;
771 for(n2=n; n2<$SIZE(n); n2++) {
772 $c(n0 => n, n1 => n2) =
773 $c(n0 => n2, n1 => n) =
774 $a(n0 => n, n1 => n2);
775 }
776 %}
777 '
778 );
779 Let's dissect what is happening. Firstly, what is this function
781 supposed to do? From its signature you see that it takes a 2D matrix
782 with equal numbers of columns and rows and outputs a matrix of the same
783 size. From a given input matrix $a it computes a symmetric output
784 matrix $c (symmetric in the matrix sense that A^T = A where ^T means
785 matrix transpose, or in PDL parlance $c == $c->transpose). It does this
786 by using only the values on and below the diagonal of $a. In the output
787 matrix $c all values on and below the diagonal are the same as those in
788 $a while those above the diagonal are a mirror image of those below the
789 diagonal (above and below are here interpreted in the way that PDL
790 prints 2D pdls). If this explanation still sounds a bit strange just go
791 ahead, make a little file into which you write this definition, build
792 the new PDL extension (see section on Makefiles for PP code) and try it
793 out with a couple of examples.
794 Having explained what the function is supposed to do there are a couple
796 of points worth noting from the syntactical point of view. First, we
797 get the size of the dimension named "n" again by using the $SIZE macro.
798 Second, there are suddenly these funny "n0" and "n1" index names in the
799 code though the signature defines only the dimension "n". Why this? The
800 reason becomes clear when you note that both the first and second
801 dimension of $a and $b are named "n" in the signature of "symm". This
802 tells PDL::PP that the first and second dimension of these arguments
803 should have the same size. Otherwise the generated function will raise
804 a runtime error. However, now in an access to $a and $c PDL::PP cannot
805 figure out which index "n" refers to any more just from the name of the
806 index. Therefore, the indices with equal dimension names get numbered
807 from left to right starting at 0, e.g. in the above example "n0" refers
808 to the first dimension of $a and $c, "n1" to the second and so on.
809 In all examples so far, we have only used the "Pars" and "Code" members
811 of the hash that was passed to "pp_def". There are certainly other keys
812 that are recognised by PDL::PP and we will hear about some of them in
813 the course of this document. Find a (non-exhaustive) list of keys in
814 Appendix A. A list of macros and PPfunctions (we have only encountered
815 some of those in the examples above yet) that are expanded in values of
816 the hash argument to "pp_def" is summarised in Appendix B.
817 At this point, it might be appropriate to mention that PDL::PP is not a
819 completely static, well designed set of routines (as Tuomas puts it:
820 "stop thinking of PP as a set of routines carved in stone") but rather
821 a collection of things that the PDL::PP author (Tuomas J. Lukka)
822 considered he would have to write often into his PDL extension
823 routines. PP tries to be expandable so that in the future, as new needs
824 arise, new common code can be abstracted back into it. If you want to
825 learn more on why you might want to change PDL::PP and how to do it
826 check the section on PDL::PP internals.
827 Handling bad values
829 There are several keys and macros used when writing code to handle bad
830 values. The first one is the "HandleBad" key:
831 HandleBad => 0
833 This flags a pp-routine as NOT handling bad values. If this routine
834 is sent ndarrays with their "badflag" set, then a warning message
835 is printed to STDOUT and the ndarrays are processed as if the value
836 used to represent bad values is a valid number. The "badflag" value
837 is not propagated to the output ndarrays.
838 An example of when this is used is for FFT routines, which
840 generally do not have a way of ignoring part of the data.
841 HandleBad => 1
843 This causes PDL::PP to write extra code that ensures the BadCode
844 section is used, and that the "$ISBAD()" macro (and its brethren)
845 work. If no "BadCode" is supplied, the "Code" section will be used,
846 on the assumption it will use "PDL_IF_BAD" to handle bad values.
847 HandleBad is not given
849 If any of the input ndarrays have their "badflag" set, then the
850 output ndarrays will have their "badflag" set, but any supplied
851 BadCode is ignored.
852 The value of "HandleBad" is used to define the contents of the "BadDoc"
854 key, if it is not given.
855 To handle bad values, code must be written somewhat differently; for
857 instance,
858 $c() = $a() + $b();
860 becomes something like
862 if ( $a() != BADVAL && $b() != BADVAL ) {
864 $c() = $a() + $b();
865 } else {
866 $c() = BADVAL;
867 }
868 However, we only want the second version if bad values are present in
870 the input ndarrays (and that bad-value support is wanted!) - otherwise
871 we actually want the original code. This is where the "BadCode" key
872 comes in; you use it to specify the code to execute if bad values may
873 be present, and PP uses both it and the "Code" section to create
874 something like:
875 if ( bad_values_are_present ) {
877 fancy_broadcastloop_stuff {
878 BadCode
879 }
880 } else {
881 fancy_broadcastloop_stuff {
882 Code
883 }
884 }
885 This approach means that there is virtually no overhead when bad values
887 are not present (i.e. the badflag routine returns 0).
888 The C preprocessor symbol "PDL_BAD_CODE" is defined when the bad code
890 is compiled, so that you can reduce the amount of code you write. The
891 BadCode section can use the same macros and looping constructs as the
892 Code section. As of 2.073, you can also use
893 "PDL_IF_BAD(iftrue,iffalse)".
894 Other bad-value macros
896 However, it wouldn't be much use without the following additional
897 macros:
898 $ISBAD(var)
900 To check whether an ndarray's value is bad, use the $ISBAD macro:
902 if ( $ISBAD(a()) ) { printf("a() is bad\n"); }
904 You can also access given elements of an ndarray:
906 if ( $ISBAD(a(n=>l)) ) { printf("element %d of a() is bad\n", l); }
908 $ISGOOD(var)
910 This is the opposite of the $ISBAD macro.
912 $SETBAD(var)
914 For when you want to set an element of an ndarray bad.
916 $ISBADVAR(c_var,pdl)
918 If you have cached the value of an ndarray "$a()" into a c-variable
920 ("foo" say), then to check whether it is bad, use "$ISBADVAR(foo,a)".
921 $ISGOODVAR(c_var,pdl)
923 As above, but this time checking that the cached value isn't bad.
925 $SETBADVAR(c_var,pdl)
927 To copy the bad value for an ndarray into a c variable, use
929 "$SETBADVAR(foo,a)".
930 TODO: mention "$PPISBAD()" etc macros.
932 PDL STATE macros
934 If you want access to the value of the badflag for a given ndarray, you
935 can use the PDL STATE macros, for use in "CopyBadStatusCode" and
936 "FindBadStatusCode".
937 $ISPDLSTATEBAD(pdl)
939 $ISPDLSTATEGOOD(pdl)
940 $SETPDLSTATEBAD(pdl)
941 $SETPDLSTATEGOOD(pdl)
942 And for use in "Code" sections:
944 $PDLSTATEISBAD(pdl)
946 $PDLSTATEISGOOD(pdl)
947 $PDLSTATESETBAD(pdl)
948 $PDLSTATESETGOOD(pdl)
949 Bad-value examples
951 Using these macros, the above code could be specified as:
952 Code => '$c() = $a() + $b();',
954 BadCode => '
955 if ( $ISBAD(a()) || $ISBAD(b()) ) {
956 $SETBAD(c());
957 } else {
958 $c() = $a() + $b();
959 }',
960 Since this is Perl, TMTOWTDI, so you could also write:
962 BadCode => '
964 if ( $ISGOOD(a()) && $ISGOOD(b()) ) {
965 $c() = $a() + $b();
966 } else {
967 $SETBAD(c());
968 }',
969 You can reduce code repetition using the C "PDL_BAD_CODE" macro,
971 supplying only the "Code" section:
972 Code => '
974 #ifdef PDL_BAD_CODE
975 if ( $ISGOOD(a()) && $ISGOOD(b()) ) {
976 #endif PDL_BAD_CODE
977 $c() = $a() + $b();
978 #ifdef PDL_BAD_CODE
979 } else {
980 $SETBAD(c());
981 }
982 #endif PDL_BAD_CODE
983 ',
984 As of 2.073, you can also use "PDL_IF_BAD(iftrue,iffalse)":
986 Code => '
988 PDL_IF_BAD(if ( $ISGOOD(a()) && $ISGOOD(b()) ) {,)
989 $c() = $a() + $b();
990 PDL_IF_BAD(} else $SETBAD(c());,)
991 ',
992 Interfacing your own/library functions using PP
994 Now, consider the following: you have your own C function (that may in
995 fact be part of some library you want to interface to PDL) which takes
996 as arguments two pointers to vectors of double:
997 void myfunc(int n,double *v1,double *v2);
999 The correct way of defining the PDL function is
1001 pp_def('myfunc',
1003 Pars => 'a(n); [o]b(n);',
1004 GenericTypes => ['D'],
1005 Code => 'myfunc($SIZE(n),$P(a),$P(b));'
1006 );
1007 The "$P("par")" syntax returns a pointer to the first element and the
1009 other elements are guaranteed to lie after that.
1010 Notice that here it is possible to make many mistakes. First, $SIZE(n)
1012 must be used instead of "n". Second, you shouldn't put any loops in
1013 this code. Third, here we encounter a new hash key recognised by
1014 PDL::PP : the "GenericTypes" declaration tells PDL::PP to ONLY GENERATE
1015 THE TYPELOOP FOP THE LIST OF TYPES SPECIFIED. In this case "double".
1016 This has two advantages. Firstly the size of the compiled code is
1017 reduced vastly, secondly if non-double arguments are passed to
1018 "myfunc()" PDL will automatically convert them to double before passing
1019 to the external C routine and convert them back afterwards.
1020 One can also use "Pars" to qualify the types of individual arguments.
1022 Thus one could also write this as:
1023 pp_def('myfunc',
1025 Pars => 'double a(n); double [o]b(n);',
1026 Code => 'myfunc($SIZE(n),$P(a),$P(b));'
1027 );
1028 The type specification in "Pars" exempts the argument from variation in
1030 the typeloop - rather it is automatically converted to and from the
1031 type specified. This is obviously useful in a more general example,
1032 e.g.:
1033 void myfunc(int n,float *v1,long *v2);
1035 pp_def('myfunc',
1037 Pars => 'float a(n); long [o]b(n);',
1038 GenericTypes => ['F'],
1039 Code => 'myfunc($SIZE(n),$P(a),$P(b));'
1040 );
1041 Note we still use "GenericTypes" to reduce the size of the type loop,
1043 obviously PP could in principle spot this and do it automatically
1044 though the code has yet to attain that level of sophistication!
1045 Finally note when types are converted automatically one MUST use the
1047 "[o]" qualifier for output variables or you hard-won changes will get
1048 optimised away by PP!
1049 If you interface a large library you can automate the interfacing even
1051 further. Perl can help you again(!) in doing this. In many libraries
1052 you have certain calling conventions. This can be exploited. In short,
1053 you can write a little parser (which is really not difficult in Perl)
1054 that then generates the calls to "pp_def" from parsed descriptions of
1055 the functions in that library. For an example, please check the Slatec
1056 interface in the "Lib" tree of the PDL distribution. If you want to
1057 check (during debugging) which calls to PP functions your Perl code
1058 generated a little helper package comes in handy which replaces the PP
1059 functions by identically named ones that dump their arguments to
1060 stdout.
1061 Just say
1063 perl -MPDL::PP::Dump myfile.pd
1065 to see the calls to "pp_def" and friends. Try it with ops.pd and
1067 slatec.pd. If you're interested (or want to enhance it), the source is
1068 in Basic/Gen/PP/Dump.pm
1069 Other macros in the Code section
1071 Macros: So far we have encountered the $SIZE, $GENERIC and $P macros.
1072 Now we are going to quickly explain the other macros that are expanded
1073 in the "Code" section of PDL::PP along with examples of their usage.
1074 $T
1076 The $T macro is used for type switches. This is very useful when you
1078 have to use different external (e.g. library) functions depending on
1079 the input type of arguments. The general syntax is
1080 $Ttypeletters(type_alternatives)
1082 where "typeletters" is a permutation of a subset of the letters
1084 "BSULNQFDGC" which stand for Byte, Short, Ushort, etc. and
1085 "type_alternatives" are the expansions when the type of the PP
1086 operation is equal to that indicated by the respective letter. Let's
1087 illustrate this incomprehensible description by an example. Assuming
1088 you have two C functions with prototypes
1089 void float_func(float *in, float *out);
1091 void double_func(double *in, double *out);
1092 which do basically the same thing but one accepts float and the other
1094 double pointers. You could interface them to PDL by defining a generic
1095 function "foofunc" (which will call the correct function depending on
1096 the type of the transformation):
1097 pp_def('foofunc',
1099 Pars => ' a(n); [o] b();',
1100 Code => ' $TFD(float,double)_func ($P(a),$P(b));'
1101 GenericTypes => [qw(F D)],
1102 );
1103 There is a limitation that the comma-separated values cannot have
1105 parentheses.
1106 $PP
1108 The $PP macro is used for a so called physical pointer access. The
1110 physical refers to some internal optimisations of PDL (for those who
1111 are familiar with the PDL core we are talking about the vaffine
1112 optimisations). This macro is mainly for internal use and you shouldn't
1113 need to use it in any of your normal code.
1114 $PPSYM
1116 The "$PPSYM()" macro is replaced by the value of "ppsym" in PDL::Types
1118 for the loop type, or that of the given parameter, similar to
1119 "$GENERIC()". This is useful for e.g. macros that vary by that,
1120 avoiding the need for things like "$TXY(X,Y)". Another benefit is that
1121 if an operation's GenericTypes get extended, this macro will still be
1122 correct.
1123 $COMP (and the OtherPars section)
1125 The $COMP macro is used to access non-pdl values in the code section.
1127 Its name is derived from the implementation of transformations in PDL.
1128 The variables you can refer to using $COMP are members of the
1129 ``compiled'' structure that represents the PDL transformation in
1130 question but does not yet contain any information about dimensions (for
1131 further details check PDL::Internals). However, you can treat $COMP
1132 just as a black box without knowing anything about the implementation
1133 of transformations in PDL. So when would you use this macro? Its main
1134 usage is to access values of arguments that are declared in the
1135 "OtherPars" section of a "pp_def" definition. But then you haven't
1136 heard about the "OtherPars" key yet?! Let's have another example that
1137 illustrates typical usage of both new features:
1138 pp_def('pnmout',
1140 Pars => 'a(m)',
1141 OtherPars => "PerlIO *fp",
1142 GenericTypes => [qw(B U S L)],
1143 Code => '
1144 if (PerlIO_write($COMP(fp),$P(a),len) != len)
1145 $CROAK("Error writing pnm file");
1146 ');
1147 This function is used to write data from a pdl to a file. The file
1149 descriptor is passed as a string into this function. This parameter
1150 does not go into the "Pars" section since it cannot be usefully treated
1151 like a pdl but rather into the aptly named "OtherPars" section.
1152 Parameters in the "OtherPars" section follow those in the "Pars"
1153 section when invoking the function, i.e.
1154 open FILE,">out.dat" or die "couldn't open out.dat";
1156 pnmout($pdl,'FILE');
1157 When you want to access this parameter inside the code section you have
1159 to tell PP by using the $COMP macro, i.e. you write "$COMP(fp)" as in
1160 the example. Otherwise PP wouldn't know that the "fp" you are referring
1161 to is the same as that specified in the "OtherPars" section.
1162 Another use for the "OtherPars" section is to set a named dimension in
1164 the signature. Let's have an example how that is done:
1165 pp_def('setdim',
1167 Pars => '[o] a(n)',
1168 OtherPars => 'int ns => n',
1169 Code => 'loop(n) %{ $a() = n; %}',
1170 );
1171 This says that the named dimension "n" will be initialised from the
1173 value of the other parameter "ns" which is of integer type (I guess you
1174 have realised that we use the "CType From => named_dim" syntax). Now
1175 you can call this function in the usual way:
1176 setdim(($x=null),5);
1178 print $x;
1179 [ 0 1 2 3 4 ]
1180 Admittedly this function is not very useful but it demonstrates how it
1182 works. If you call the function with an existing pdl and you don't need
1183 to explicitly specify the size of "n" since PDL::PP can figure it out
1184 from the dimensions of the non-null pdl. In that case you just give the
1185 dimension parameter as "-1":
1186 $x = hist($y);
1188 setdim($x,-1);
1189 The default values available via "$COMP()" are the "OtherPars" as noted
1191 above, which get copied in. However, this can be added to (previous to
1192 2.058, replaced) by supplying "Comp" and/or "MakeComp" keys (the
1193 defaults will happen first):
1194 pp_def(
1196 'diagonal',
1197 OtherPars => 'SV *list',
1198 Comp => 'PDL_Indx whichdims_count; PDL_Indx whichdims[$COMP(whichdims_count)];',
1199 MakeComp => '
1200 PDL_Indx i;
1201 PDL_Indx *tmp= PDL->packdims(list,&($COMP(whichdims_count)));
1202 if (!tmp) $CROAK("Failed to packdims for creating");
1203 if ($COMP(whichdims_count) < 1)
1204 $CROAK("Diagonal: must have at least 1 dimension");
1205 $DOCOMPALLOC(); /* malloc()s the whichdims */
1206 for(i=0; i<$COMP(whichdims_count); i++)
1207 $COMP(whichdims)[i] = tmp[i];
1208 free(tmp);
1209 /* ... */
1210 ',
1211 # ...
1212 );
1213 The "MakeComp" code is placed in the "pdl_(funcname)_run", so access to
1215 "Pars" (which will just be "pdl *"s)/"OtherPars" values is just via
1216 their names, not a macro.
1217 As of 2.058, you can instead give a C99 "incomplete array" type
1219 parameter as an "OtherPars" entry:
1220 pp_def(
1222 'diagonal',
1223 OtherPars => 'PDL_Indx whichdims[]',
1224 MakeComp => '
1225 if ($COMP(whichdims_count) < 1)
1226 $CROAK("Diagonal: must have at least 1 dimension");
1227 /* ... */
1228 ',
1229 # ...
1230 );
1231 There is an XS typemap entry (only for "PDL_Indx" array types for now)
1233 that adds a "(varname)_count" variable having extracted the index
1234 numbers from an array-ref parameter, and sets the count variable to the
1235 right value. PP then makes a copy of the data available. The C function
1236 (here, "pdl_diagonal_run")'s caller (here, the generated XS function)
1237 is responsible for freeing the array passed in (here, PDL's "smalloc"
1238 function is used, so the user need do nothing different).
1239 Other functions in the Code section
1241 The only PP function that we have used in the examples so far is
1242 "loop". Additionally, there are currently two other functions which
1243 are recognised in the "Code" section:
1244 broadcastloop
1246 As we heard above the signature of a PP defined function defines the
1248 dimensions of all the pdl arguments involved in a primitive operation.
1249 However, you often call the functions that you defined with PP with
1250 pdls that have more dimensions than those specified in the signature.
1251 In this case the primitive operation is performed on all subslices of
1252 appropriate dimensionality in what is called a broadcast loop (see also
1253 overview above and PDL::Indexing). Assuming you have some notion of
1254 this concept you will probably appreciate that the operation specified
1255 in the code section should be optimised since this is the tightest loop
1256 inside a broadcast loop. However, if you revisit the example where we
1257 define the "pnmout" function, you will quickly realise that looking up
1258 the "IO" file descriptor in the inner broadcast loop is not very
1259 efficient when writing a pdl with many rows. A better approach would be
1260 to look up the "IO" descriptor once outside the broadcast loop and use
1261 its value then inside the tightest broadcast loop. This is exactly
1262 where the "broadcastloop" function comes in handy. Here is an improved
1263 definition of "pnmout" which uses this function:
1264 pp_def('pnmout',
1266 Pars => 'a(m)',
1267 OtherPars => "PerlIO *fp",
1268 GenericTypes => [qw(B U S L)],
1269 Code => '
1270 int len;
1271 len = $SIZE(m) * sizeof($GENERIC());
1272 broadcastloop %{
1273 if (PerlIO_write($COMP(fp),$P(a),len) != len)
1274 $CROAK("Error writing pnm file");
1275 %}
1276 ');
1277 This works as follows. Normally the C code you write inside the "Code"
1279 section is placed inside a broadcast loop (i.e. PP generates the
1280 appropriate wrapping C code around it). However, when you explicitly
1281 use the "broadcastloop" function, PDL::PP recognises this and doesn't
1282 wrap your code with an additional broadcast loop. This has the effect
1283 that code you write outside the broadcast loop is only executed once
1284 per transformation and just the code with in the surrounding "%{ ...
1285 %}" pair is placed within the tightest broadcast loop. This also comes
1286 in handy when you want to perform a decision (or any other code,
1287 especially CPU intensive code) only once per thread, i.e.
1288 pp_addhdr('
1290 #define RAW 0
1291 #define ASCII 1
1292 ');
1293 pp_def('do_raworascii',
1294 Pars => 'a(); b(); [o]c()',
1295 OtherPars => 'int mode',
1296 Code => ' switch ($COMP(mode)) {
1297 case RAW:
1298 broadcastloop %{
1299 /* do raw stuff */
1300 %}
1301 break;
1302 case ASCII:
1303 broadcastloop %{
1304 /* do ASCII stuff */
1305 %}
1306 break;
1307 default:
1308 $CROAK("unknown mode");
1309 }'
1310 );
1311 types
1313 The types function works similar to the $T macro. However, with the
1315 "types" function the code in the following block (delimited by "%{" and
1316 "%}" as usual) is executed for all those cases in which the datatype of
1317 the operation is any of the types represented by the letters in the
1318 argument to "type", e.g.
1319 Code => '...
1321 types(BSUL) %{
1323 /* do integer type operation */
1324 %}
1325 types(FD) %{
1326 /* do floating point operation */
1327 %}
1328 ...'
1329 You are encouraged to use this idiom (from PDL::Math) in order to
1331 minimise effort needed to make your code work with new types:
1332 use PDL::Types qw(types);
1334 my @Rtypes = grep $_->real, types();
1335 my @Ctypes = grep !$_->real, types();
1336 # ...
1337 my $got_complex = PDL::Core::Dev::got_complex_version($name, 2);
1338 my $complex_bit = join "\n",
1339 map 'types('.$_->ppsym.') %{$'.$c.'() = c'.$name.$_->floatsuffix.'($'.$x.'(),$'.$y.'());%}',
1340 @Ctypes;
1341 my $real_bit = join "\n",
1342 map 'types('.$_->ppsym.') %{$'.$c.'() = '.$name.'($'.$x.'(),$'.$y.'());%}',
1343 @Rtypes;
1344 ($got_complex ? $complex_bit : '') . $real_bit;
1345 (although you should first check whether tgmath.h already has a type-
1347 generic version of the function you want to call, in which case the
1348 above becomes unnecessary).
1349 The RedoDimsCode Section
1351 The "RedoDimsCode" key is an optional key that is used to compute
1352 dimensions of ndarrays at runtime in case the standard rules for
1353 computing dimensions from the signature are not sufficient. The
1354 contents of the "RedoDimsCode" entry is interpreted in the same way
1355 that the Code section is interpreted-- i.e., PP macros are expanded and
1356 the result is interpreted as C code. The purpose of the code is to set
1357 the size of some dimensions that appear in the signature. Storage
1358 allocation and broadcastloops and so forth will be set up as if the
1359 computed dimension had appeared in the signature. In your code, you
1360 first compute the desired size of a named dimension in the signature
1361 according to your needs and then assign that value to it via the
1362 $SIZE() macro.
1363 As an example, consider the following situation. You are interfacing an
1365 external library routine that requires an temporary array for workspace
1366 to be passed as an argument. Two input data arrays that are passed are
1367 p(m) and x(n). The output data array is y(n). The routine requires a
1368 workspace array with a length of n+m*m, and you'd like the storage
1369 created automatically just like it would be for any ndarray flagged
1370 with [t] or [o]. What you'd like is to say something like
1371 pp_def( "myexternalfunc",
1373 Pars => " p(m); x(n); [o] y; [t] work(n+m*m); ", ...
1374 but that won't work, because PP can't interpret expressions with
1376 arithmetic in the signature. Instead you write
1377 pp_def(
1379 "myexternalfunc",
1380 Pars => ' p(m); x(n); [o] y(); [t] work(wn); ',
1381 RedoDimsCode => '$SIZE(wn) = $SIZE(n) + $SIZE(m) * $SIZE(m);',
1382 Code => '
1383 externalfunc( $P(p), $P(x), $SIZE(m), $SIZE(n), $P(work) );
1384 '
1385 );
1386 As of 2.075, you can use the dimensions of passed-in ndarrays as they
1388 are available when the "RedoDimsCode" is run. Before the code in the
1389 Code section is executed PP will create the proper storage for "work"
1390 if it does not exist. Note that you only took the first dimension of
1391 "p" and "x" because the user may have sent ndarrays with extra
1392 broadcasting dimensions. Of course, the temporary ndarray "work" (note
1393 the [t] flag) should not be given any broadcast dimensions anyway.
1394 You can also use "RedoDimsCode" to set the dimension of a ndarray
1396 flagged with [o]. In this case you set the dimensions for the named
1397 dimension in the signature using $SIZE() as in the preceding example.
1398 However, because the ndarray is flagged with [o] instead of [t],
1399 broadcasting dimensions will be added if required just as if the size
1400 of the dimension were computed from the signature according to the
1401 usual rules. Here is an example from PDL::Math
1402 pp_def("polyroots",
1404 Pars => 'cr(n); ci(n); [o]rr(m); [o]ri(m);',
1405 RedoDimsCode => '$SIZE(m) = $SIZE(n)-1;',
1406 The input ndarrays are the real and imaginary parts of complex
1408 coefficients of a polynomial. The output ndarrays are real and
1409 imaginary parts of the roots. There are "n" roots to an "n"th order
1410 polynomial and such a polynomial has "n+1" coefficients (the zero-th
1411 through the "n"th). In this example, broadcasting will work correctly.
1412 That is, the first dimension of the output ndarray with have its
1413 dimension adjusted, but other broadcasting dimensions will be assigned
1414 just as if there were no "RedoDimsCode".
1415 RedoDims passed directly
1417 A "RedoDimsCode" value as above gets processed, including expanding
1419 macros, and adding type-generic loops. For very specific purposes, you
1420 may not want this processing done to your dimension-updating code,
1421 probably in "slice"-like functions.
1422 Then, instead of passing a "RedoDimsCode" value, you can pass a
1424 "RedoDims" value (which the "RedoDimsCode" would otherwise get
1425 processed into). Because you will probably want to access the ndarrays,
1426 the following macros are provided. They are named assuming you will
1427 have the first parameter as "PARENT" and the second as "CHILD", which
1428 is the case if you passed a true "P2Child" value, which you will
1429 basically always want to do for this scenario.
1430 RedoDims generated from EquivPDimExpr and EquivDimCheck
1432 Another way to generate the "RedoDims" code is to supply a
1434 "EquivPDimExpr" and maybe a "EquivDimCheck":
1435 pp_def(
1437 'xchg',
1438 OtherPars => 'PDL_Indx n1; PDL_Indx n2;',
1439 TwoWay => 1,
1440 P2Child => 1,
1441 AffinePriv => 1,
1442 EquivDimCheck => '
1443 if ($COMP(n1) <0) $COMP(n1) += $PARENT(broadcastids[0]);
1444 if ($COMP(n2) <0) $COMP(n2) += $PARENT(broadcastids[0]);
1445 if (PDLMIN($COMP(n1),$COMP(n2)) <0 ||
1446 PDLMAX($COMP(n1),$COMP(n2)) >= $PARENT(broadcastids[0]))
1447 $CROAK("One of dims %d, %d out of range: should be 0<=dim<%d",
1448 $COMP(n1),$COMP(n2),$PARENT(broadcastids[0]));',
1449 EquivPDimExpr => '
1450 (($CDIM == $COMP(n1)) ? $COMP(n2) :
1451 ($CDIM == $COMP(n2)) ? $COMP(n1) :
1452 $CDIM)
1453 ',
1454 );
1455 "EquivPDimExpr" is evaluated within a loop, and the value of the
1457 relevant dimension is available using the macro $CDIM as shown above.
1458 Typemap handling in the OtherPars section
1460 The "OtherPars" section discussed above is very often absolutely
1461 crucial when you interface external libraries with PDL. However in many
1462 cases the external libraries either use derived types or pointers of
1463 various types.
1464 The standard way to handle this in Perl is to use a typemap file. This
1466 is discussed in some detail in perlxs in the standard Perl
1467 documentation. In PP the functionality is very similar, so you can
1468 create a typemap file in the directory where your PP file resides and
1469 when it is built it is automatically read in to figure out the
1470 appropriate translation between the C type and Perl's built-in type.
1471 For instance the "gsl_spline_init" function has the following C
1473 declaration:
1474 int gsl_spline_init(gsl_spline * spline,
1476 const double xa[], const double ya[], size_t size);
1477 Clearly the "xa" and "ya" arrays are candidates for being passed in as
1479 ndarrays and the "size" argument is just the length of these ndarrays
1480 so that can be handled by the "$SIZE()" macro in PP. Write an
1481 "OtherPars" declaration of the form
1482 OtherPars => 'gsl_spline *spl'
1484 and write a short typemap file which handles this type:
1486 TYPEMAP
1488 gsl_spline * T_PTR
1489 and use it in the code:
1491 pp_def('init_meat',
1493 Pars => 'double x(n); double y(n);',
1494 OtherPars => 'gsl_spline *spl',
1495 Code =>'gsl_spline_init,($COMP(spl),$P(x),$P(y),$SIZE(n)));'
1496 );
1497 where I have removed a macro wrapper call, but that would obscure the
1499 discussion.
1500 Other useful PP keys in data operation definitions
1502 You have already heard about the "OtherPars" key. Currently, there are
1503 not many other keys for a data operation that will be useful in normal
1504 (whatever that is) PP programming. In fact, it would be interesting to
1505 hear about a case where you think you need more than what is provided
1506 at the moment. Please speak up on one of the PDL mailing lists. Most
1507 other keys recognised by "pp_def" are only really useful for what we
1508 call slice operations (see also above).
1509 One thing that is strongly being planned is variable number of
1511 arguments, which will be a little tricky.
1512 An incomplete list of the available keys:
1514 Inplace
1516 Setting this key marks the routine as working inplace - ie the input
1518 and output ndarrays are the same. An example is "$x->inplace->sqrt()"
1519 (or "sqrt(inplace($x))").
1520 Inplace => 1
1522 Use when the routine is a unary function, such as "sqrt".
1523 Inplace => ['a']
1525 If there are more than one input ndarrays, specify the name of the
1526 one that can be changed inplace using an array reference.
1527 Inplace => ['a','b']
1529 If there are more than one output ndarray, specify the name of the
1530 input ndarray and output ndarray in a 2-element array reference.
1531 This probably isn't needed, but left in for completeness.
1532 If bad values are being used, care must be taken to ensure the
1534 propagation of the badflag when inplace is being used; consider this
1535 excerpt from Basic/Bad/bad.pd:
1536 pp_def('setbadtoval',HandleBad => 1,
1538 Pars => 'a(); [o]b();',
1539 OtherPars => 'double newval',
1540 Inplace => 1,
1541 CopyBadStatusCode => 'PDL->propagate_badflag( b, 0 );',
1542 ...
1543 Since this routine removes all bad values, the output ndarray had its
1545 bad flag cleared. This is then propagated to both parents and children.
1546 NOTE: one idea is that the documentation for the routine could be
1548 automatically flagged to indicate that it can be executed inplace, ie
1549 something similar to how "HandleBad" sets "BadDoc" if it's not supplied
1550 (it's not an ideal solution).
1551 FTypes
1553 # in slices.pd
1555 FTypes => {CHILD => '$COMP(totype)'},
1556 The value is a hash-ref mapping parameter-names to an expression giving
1558 an override of the type for that parameter. The example above shows the
1559 type being overridden to the "OtherPars" "totype".
1560 OtherParsDefaults
1562 OtherPars => 'int a; int b',
1564 OtherParsDefaults => { b => 0 },
1565 Allows specifying default values for "OtherPars". It is an error to
1567 specify a default for one that is before another that does not have a
1568 default.
1569 Other PDL::PP functions to support concise package definition
1571 So far, we have described the "pp_def" and "pp_done" functions. PDL::PP
1572 exports a few other functions to aid you in writing concise PDL
1573 extension package definitions.
1574 pp_addhdr
1576 Often when you interface library functions as in the above example you
1578 have to include additional C include files. Since the XS file is
1579 generated by PP we need some means to make PP insert the appropriate
1580 include directives in the right place into the generated XS file. To
1581 this end there is the "pp_addhdr" function. This is also the function
1582 to use when you want to define some C functions for internal use by
1583 some of the XS functions (which are mostly functions defined by
1584 "pp_def"). By including these functions here you make sure that
1585 PDL::PP inserts your code before the point where the actual XS module
1586 section begins and will therefore be left untouched by xsubpp (cf.
1587 perlxs and perlxstut man pages).
1588 A typical call would be
1590 pp_addhdr('
1592 #include <unistd.h> /* we need defs of XXXX */
1593 #include "libprotos.h" /* prototypes of library functions */
1594 #include "mylocaldecs.h" /* Local decs */
1595 static void do_the real_work(PDL_Byte * in, PDL_Byte * out, int n)
1597 {
1598 /* do some calculations with the data */
1599 }
1600 ');
1601 This ensures that all the constants and prototypes you need will be
1603 properly included and that you can use the internal functions defined
1604 here in the "pp_def"s, e.g.:
1605 pp_def('barfoo',
1607 Pars => ' a(n); [o] b(n)',
1608 GenericTypes => ['B'],
1609 Code => ' PDL_Indx ns = $SIZE(n);
1610 do_the_real_work($P(a),$P(b),ns);
1611 ',
1612 );
1613 pp_addpm
1615 In many cases the actual PP code (meaning the arguments to "pp_def"
1617 calls) is only part of the package you are currently implementing.
1618 Often there is additional Perl code and XS code you would normally have
1619 written into the pm and XS files which are now automatically generated
1620 by PP. So how to get this stuff into those dynamically generated files?
1621 Fortunately, there are a couple of functions, generally called
1622 "pp_addXXX" that assist you in doing this.
1623 Let's assume you have additional Perl code that should go into the
1625 generated pm-file. This is easily achieved with the "pp_addpm" command:
1626 pp_addpm(<<'EOD');
1628 =head1 NAME
1630 PDL::Lib::Mylib -- a PDL interface to the Mylib library
1632 =head1 DESCRIPTION
1634 This package implements an interface to the Mylib package with full
1636 broadcasting and indexing support (see L<PDL::Indexing>).
1637 =cut
1639 use PGPLOT;
1641 =head2 use_myfunc
1643 this function applies the myfunc operation to all the
1644 elements of the input pdl regardless of dimensions
1645 and returns the sum of the result
1646 =cut
1647 sub use_myfunc {
1649 my $pdl = shift;
1650 myfunc($pdl->clump(-1),($res=null));
1652 return $res->sum;
1654 }
1655 EOD
1657 pp_add_exported
1659 You have probably got the idea. In some cases you also want to export
1661 your additional functions. To avoid getting into trouble with PP which
1662 also messes around with the @EXPORT array you just tell PP to add your
1663 functions to the list of exported functions:
1664 pp_add_exported('use_myfunc gethynx');
1666 pp_add_isa
1668 The "pp_add_isa" command works like the the "pp_add_exported" function.
1670 The arguments to "pp_add_isa" are added the @ISA list, e.g.
1671 pp_add_isa(' Some::Other::Class ');
1673 pp_bless
1675 If your pp_def routines are to be used as object methods use "pp_bless"
1677 to specify the package (i.e. class) to which your pp_defed methods will
1678 be added. For example, "pp_bless('PDL::MyClass')". The default is "PDL"
1679 if this is omitted.
1680 pp_addxs
1682 Sometimes you want to add extra XS code of your own (that is generally
1684 not involved with any broadcasting/indexing issues but supplies some
1685 other functionality you want to access from the Perl side) to the
1686 generated XS file, for example
1687 pp_addxs('','
1689 # Determine endianness of machine
1691 int
1693 isbigendian()
1694 CODE:
1695 unsigned short i;
1696 PDL_Byte *b;
1697 i = 42; b = (PDL_Byte*) (void*) &i;
1699 if (*b == 42)
1701 RETVAL = 0;
1702 else if (*(b+1) == 42)
1703 RETVAL = 1;
1704 else
1705 croak("Impossible - machine is neither big nor little endian!!\n");
1706 OUTPUT:
1707 RETVAL
1708 ');
1709 Especially "pp_add_exported" and "pp_addxs" should be used with care.
1711 PP uses PDL::Exporter, hence letting PP export your function means that
1712 they get added to the standard list of function exported by default
1713 (the list defined by the export tag ``:Func''). If you use "pp_addxs"
1714 you shouldn't try to do anything that involves broadcasting or indexing
1715 directly. PP is much better at generating the appropriate code from
1716 your definitions.
1717 pp_add_boot
1719 Finally, you may want to add some code to the BOOT section of the XS
1721 file (if you don't know what that is check perlxs). This is easily done
1722 with the "pp_add_boot" command:
1723 pp_add_boot(<<EOB);
1725 descrip = mylib_initialize(KEEP_OPEN);
1726 if (descrip == NULL)
1728 croak("Can't initialize library");
1729 GlobalStruc->descrip = descrip;
1731 GlobalStruc->maxfiles = 200;
1732 EOB
1733 pp_export_nothing
1735 By default, PP.pm puts all subs defined using the pp_def function into
1737 the output .pm file's EXPORT list. This can create problems if you are
1738 creating a subclassed object where you don't want any methods exported.
1739 (i.e. the methods will only be called using the $object->method
1740 syntax).
1741 For these cases you can call pp_export_nothing() to clear out the
1743 export list. Example (At the end of the .pd file):
1744 pp_export_nothing();
1746 pp_done();
1747 pp_core_importList
1749 By default, PP.pm puts the 'use Core;' line into the output .pm file.
1751 This imports Core's exported names into the current namespace, which
1752 can create problems if you are over-riding one of Core's methods in the
1753 current file. You end up getting messages like "Warning: sub sumover
1754 redefined in file subclass.pm" when running the program.
1755 For these cases the pp_core_importList can be used to change what is
1757 imported from Core.pm. For example:
1758 pp_core_importList('()')
1760 This would result in
1762 use Core();
1764 being generated in the output .pm file. This would result in no names
1766 being imported from Core.pm. Similarly, calling
1767 pp_core_importList(' qw/ barf /')
1769 would result in
1771 use Core qw/ barf/;
1773 being generated in the output .pm file. This would result in just
1775 'barf' being imported from Core.pm.
1776 pp_setversion
1778 Simultaneously set the .pm and .xs files' versions, thus avoiding
1780 unnecessary version-skew between the two. To use this, simply do this
1781 in your .pd file, probably near the top:
1782 our $VERSION = '0.0.3';
1784 pp_setversion($VERSION);
1785 # Then, in your Makefile.PL:
1787 my @package = qw(FFTW3.pd FFTW3 PDL::FFTW3);
1788 my %descriptor = pdlpp_stdargs(\@package);
1789 $descriptor{VERSION_FROM} = 'FFTW3.pd'; # EUMM can parse the format above
1790 However, don't use this if you use Module::Build::PDL. See that
1792 module's documentation for details.
1793 pp_deprecate_module
1795 If a particular module is deemed obsolete, this function can be used to
1797 mark it as deprecated. This has the effect of emitting a warning when a
1798 user tries to "use" the module. The generated POD for this module also
1799 carries a deprecation notice. The replacement module can be passed as
1800 an argument like this:
1801 pp_deprecate_module( infavor => "PDL::NewNonDeprecatedModule" );
1803 Note that function affects only the runtime warning and the POD.
1805
1807 Let's say that you have a function in your module called PDL::foo that
1808 uses the PP function "bar_pp" to do the heavy lifting. But you don't
1809 want to advertise that "bar_pp" exists. To do this, you must move your
1810 PP function to the top of your module file, then call
1811 pp_export_nothing()
1813 to clear the "EXPORT" list. To ensure that no documentation (even the
1815 default PP docs) is generated, set
1816 Doc => undef
1818 and to prevent the function from being added to the symbol table, set
1820 PMFunc => ''
1822 in your pp_def declaration (see Image2D.pd for an example). This will
1824 effectively make your PP function "private." However, it is always
1825 accessible via PDL::bar_pp due to Perl's module design. But making it
1826 private will cause the user to go very far out of their way to use it,
1827 so they shoulder the consequences!
1828
1830 The slice operations require a much more intimate knowledge of PDL
1831 internals than the data operations. Furthermore, the complexity of the
1832 issues involved is considerably higher than that in the average data
1833 operation. Nevertheless, functions generated using the slice operations
1834 are at the heart of the index manipulation and dataflow capabilities of
1835 PDL. You can get started by reading the section on "P2Child".
1836 Also, there are a lot of dirty issues with virtual ndarrays and
1838 vaffines which we shall entirely skip here.
1839 Slices and bad values
1841 Slice operations need to be able to handle bad values. The easiest
1842 thing to do is look at Basic/Slices/slices.pd to see how this works.
1843 Along with "BadCode", there are also the "BadBackCode" and
1845 "BadRedoDimsCode" keys for "pp_def". However, any "EquivCPOffsCode"
1846 should not need changing, since any changes are absorbed into the
1847 definition of the "$EQUIVCPOFFS()" macro (i.e. it is handled
1848 automatically by PDL::PP).
1849
1851 For printing warning messages or aborting/dieing, you can call "warn"
1852 or "barf" from PP code. However, you should be aware that these calls
1853 have been redefined using C preprocessor macros to "PDL->barf" and
1854 "PDL->warn". These redefinitions are in place to keep you from
1855 inadvertently calling perl's "warn" or "barf" directly, which can cause
1856 segfaults during pthreading (i.e. processor multi-threading).
1857 PDL's own versions of "barf" and "warn" will queue-up warning or barf
1859 messages until after pthreading is completed, and then call the perl
1860 versions of these routines.
1861 See PDL::ParallelCPU for more information on pthreading.
1863 NB As of 2.064, it is highly recommended that you do not call "barf" at
1865 all in PP code, but instead use "$CROAK()". This will return a
1866 "pdl_error" which will transparently be used to throw the correct
1867 exception in Perl code, but can be handled suitably by non-Perl
1868 callers.
1869
1871 If you are going to generate a package from your PP file (typical file
1872 extensions are ".pd" or ".pp" for the files containing PP code) it is
1873 easiest and safest to leave generation of the appropriate commands to
1874 the Makefile. In the following we will outline the typical format of a
1875 Perl Makefile to automatically build and install your package from a
1876 description in a PP file. Most of the rules to build the xs, pm and
1877 other required files from the PP file are already predefined in the
1878 PDL::Core::Dev package. We just have to tell MakeMaker to use it.
1879 In most cases you can define your Makefile like
1881 use PDL::Core::Dev; # Pick up development utilities
1883 use ExtUtils::MakeMaker;
1884 $package = ["mylib.pd",Mylib,PDL::Lib::Mylib,'',1];
1886 %hash = pdlpp_stdargs($package);
1887 $hash{OBJECT} .= ' additional_Ccode$(OBJ_EXT) ';
1888 WriteMakefile(%hash);
1889 sub MY::postamble { pdlpp_postamble($package); }
1891 # additional_Ccode.c
1893 #include "pdl.h"
1894 void ppcp(PDL_Byte *dst, PDL_Byte *src, int len)
1895 {
1896 int i;
1897 for (i=0;i<len;i++) *dst++=*src++;
1898 }
1899 Here, the list in $package is: first: PP source file name, then the
1901 prefix for the produced files, the whole package name, the package to
1902 add XS functions to (empty string to use the same as the PP functions),
1903 and a boolean to dictate whether to have PDL generate a separate C file
1904 for each PP function (for faster compilation). The last feature is
1905 opt-in as you have to avoid duplicate symbols when linking the library
1906 (so separate out C functions into their own file). You can modify the
1907 hash in whatever way you like but it would be reasonable to stay within
1908 some limits so that your package will continue to work with later
1909 versions of PDL.
1910 To make life even easier PDL::Core::Dev defines the function
1912 "pdlpp_stdargs" that returns a hash with default values that can be
1913 passed (either directly or after appropriate modification) to a call to
1914 WriteMakefile. Currently, "pdlpp_stdargs" returns a hash where the
1915 keys are filled in as follows:
1916 (
1918 'NAME' => $mod,
1919 VERSION_FROM => $src,
1920 'TYPEMAPS' => [&PDL_TYPEMAP()],
1921 'OBJECT' => "$pref\$(OBJ_EXT)",
1922 PM => {"$pref.pm" => "\$(INST_LIBDIR)/$pref.pm"},
1923 MAN3PODS => {"$pref.pm" => "\$(INST_MAN3DIR)/$mod.\$(MAN3EXT)"},
1924 'INC' => &PDL_INCLUDE(),
1925 'LIBS' => [''],
1926 'clean' => {'FILES' => "$pref.xs $pref.pm $pref\$(OBJ_EXT)"},
1927 )
1928 Here, $src is the name of the source file with PP code, $pref the
1930 prefix for the generated .pm and .xs files and $mod the name of the
1931 extension module to generate.
1932 If your "VERSION_FROM" provides a version, PP will use that to set the
1934 "XS_VERSION". If you need to influence the value of that variable so
1935 that XSLoader etc don't reject the loaded dynamic library, you can use
1936 this workaround in a "pp_addpm" (the "BEGIN" is because the "bootstrap"
1937 happens at runtime, and your code appears after that call, but with a
1938 "BEGIN" it will take place beforehand):
1939 our $VERSION; BEGIN { $VERSION = '2.019106' };
1941 our $XS_VERSION; BEGIN { $XS_VERSION = $VERSION };
1942
1944 The internals of the current version consist of a large table which
1945 gives the rules according to which things are translated and the subs
1946 which implement these rules.
1947 Later on, it would be good to make the table modifiable by the user so
1949 that different things may be tried.
1950 [Meta comment: here will hopefully be more in the future; currently,
1952 your best bet will be to read the source code :-( or ask on the list
1953 (try the latter first) ]
1954
1956 As well as the above-mentioned "PDL_BAD_CODE" and
1957 "PDL_IF_BAD(iftrue,iffalse)", there are also
1958 "PDL_IF_GENTYPE_REAL(iftrue,iffalse)", "PDL_IF_GENTYPE_UNSIGNED", and
1959 "PDL_IF_GENTYPE_INTEGER":
1960 $b() = PDL_IF_GENTYPE_INTEGER(0,NAN);
1962
1964 Unless otherwise specified, the arguments are strings.
1965 Pars
1967 define the signature of your function
1969 OtherPars
1971 arguments which are not pdls. Default: nothing. This is a semi-colon
1973 separated list of arguments, e.g., "OtherPars=>'int k; double value;
1974 PerlIO *fp'". See $COMP(x) and also the same entry in Appendix B.
1975 Code
1977 the actual code that implements the functionality; several PP macros
1979 and PP functions are recognised in the string value
1980 HandleBad
1982 If set to 1, the routine is assumed to support bad values and the code
1984 in the BadCode key is used if bad values are present; it also sets
1985 things up so that the "$ISBAD()" etc macros can be used. If set to 0,
1986 cause the routine to print a warning if any of the input ndarrays have
1987 their bad flag set.
1988 BadCode
1990 Give the code to be used if bad values may be present in the input
1992 ndarrays. Only used if "HandleBad => 1". If "HandleBad" is true and
1993 "BadCode" is not supplied, the "Code" section will be reused, on the
1994 assumption it will use "#ifdef PDL_BAD_CODE" to handle bad values. As
1995 of 2.073, you can, and are recommended to, use
1996 "PDL_IF_BAD(iftrue,iffalse)".
1997 CopyBadStatusCode
1999 As of 2.079, this is deprecated due to being largely unnecessary;
2001 instead, just use "$PDLSTATESETBAD(pdlname)" in your "Code" section and
2002 the badflag setting will be propagated to all its parents and children.
2003 The default code here sets the bad flag of the output ndarrays if
2005 "$BADFLAGCACHE()" is true after the code has been evaluated. Sometimes
2006 "CopyBadStatusCode" is set to an empty string, with the responsibility
2007 of setting the badflag of the output ndarray left to the "BadCode"
2008 section (e.g. the "xxxover" routines in Basic/Primitive/primitive.pd).
2009 GenericTypes
2011 An array reference. The array may contain any subset of the one-
2013 character strings given below, which specify which types your operation
2014 will accept. The meaning of each type is:
2015 B - signed byte (i.e. signed char)
2017 S - signed short (two-byte integer)
2018 U - unsigned short
2019 L - signed long (four-byte integer, int on 32 bit systems)
2020 N - signed integer for indexing ndarray elements (platform & Perl-dependent size)
2021 Q - signed long long (eight byte integer)
2022 F - float
2023 D - double
2024 G - complex float
2025 C - complex double
2026 This is very useful (and important!) when interfacing an external
2028 library. Default: [qw/B S U L N Q F D/]
2029 Inplace
2031 Mark a function as being able to work inplace.
2033 Inplace => 1 if Pars => 'a(); [o]b();'
2035 Inplace => ['a'] if Pars => 'a(); b(); [o]c();'
2036 Inplace => ['a','c'] if Pars => 'a(); b(); [o]c(); [o]d();'
2037 If bad values are being used, care must be taken to ensure the
2039 propagation of the badflag when inplace is being used; for instance see
2040 the code for "setbadtoval" in Basic/Bad/bad.pd.
2041 Doc
2043 Used to specify a documentation string in Pod format. See PDL::Doc for
2045 information on PDL documentation conventions. Note: in the special case
2046 where the PP 'Doc' string is one line this is implicitly used for the
2047 quick reference AND the documentation!
2048 If the Doc field is omitted PP will generate default documentation
2050 (after all it knows about the Signature).
2051 If you really want the function NOT to be documented in any way at this
2053 point (e.g. for an internal routine, or because you are doing it
2054 elsewhere in the code) explicitly specify "Doc=>undef".
2055 BadDoc
2057 Contains the text returned by the "badinfo" command (in "perldl") or
2059 the "-b" switch to the "pdldoc" shell script. In many cases, you will
2060 not need to specify this, since the information can be automatically
2061 created by PDL::PP. However, as befits computer-generated text, it's
2062 rather stilted; it may be much better to do it yourself!
2063 NoPthread
2065 Optional flag to indicate the PDL function should not use processor
2067 threads (i.e. pthreads or POSIX threads) to split up work across
2068 multiple CPU cores. This option is typically set to 1 if the underlying
2069 PDL function is not threadsafe. If this option isn't present, then the
2070 function is assumed to be threadsafe. This option only applies if PDL
2071 has been compiled with POSIX threads enabled.
2072 PMCode
2074 pp_def('funcname',
2076 Pars => 'a(); [o] b();',
2077 PMCode => 'sub PDL::funcname {
2078 return PDL::_funcname_int(@_) if @_ == 2; # output arg "b" supplied
2079 PDL::_funcname_int(@_, my $out = PDL->null);
2080 $out;
2081 }',
2082 # ...
2083 );
2084 PDL functions allow "[o]" ndarray arguments into which you want the
2086 output saved. This is handy because you can allocate an output ndarray
2087 once and reuse it many times; the alternative would be for PDL to
2088 create a new ndarray each time, which may waste compute cycles or, more
2089 likely, RAM.
2090 PDL functions check the number of arguments they are given, and call
2092 "croak" if given the wrong number. By default (with no "PMCode"
2093 supplied), any output arguments may be omitted, and PDL::PP provides
2094 code that can handle this by creating "null" objects, passing them to
2095 your code, then returning them on the stack.
2096 If you do supply "PMCode", the rest of PDL::PP assumes it will be a
2098 string that defines a Perl function with the function's name in the
2099 "pp_bless" package ("PDL" by default). As the example implies, the PP-
2100 generated function name will change from "<funcname>", to
2101 "_<funcname>_int". As also shown above, you will need to supply all
2102 ndarrays in the exact order specified in the signature: output ndarrays
2103 are not optional, and the PP-generated function will not return
2104 anything.
2105 PMFunc
2107 When pp_def generates functions, it typically defines them in the PDL
2109 package. Then, in the .pm file that it generates for your module, it
2110 typically adds a line that essentially copies that function into your
2111 current package's symbol table with code that looks like this:
2112 *func_name = \&PDL::func_name;
2114 It's a little bit smarter than that (it knows when to wrap that sort of
2116 thing in a BEGIN block, for example, and if you specified something
2117 different for pp_bless), but that's the gist of it. If you don't care
2118 to import the function into your current package's symbol table, you
2119 can specify
2120 PMFunc => '',
2122 PMFunc has no other side-effects, so you could use it to insert
2124 arbitrary Perl code into your module if you like. However, you should
2125 use pp_addpm if you want to add Perl code to your module.
2126 ReadDataFuncName
2128 Allows overriding the default function-name, for reading data
2130 transformed by this operation. Mainly used internally to set it to
2131 "NULL", in which case a default affine-orientated function will be
2132 called instead.
2133 WriteBackDataFuncName
2135 As above, but for writing transformed data from a child of this
2137 transformation back to the parent when "BackCode" is supplied.
2138 AffinePriv
2140 Flag to indicate this is an affine transformation whose "Priv"
2142 (contents of the "pdl_trans") contains data that will need allocating
2143 and freeing.
2144 GlobalNew
2146 If supplied, will prevent generation of an XS function, and assigns the
2148 generated C "run" function into the named slot in the "Core" struct.
2149 This is not used as of 2.058, and instead the relevant C functions are
2150 in pdlaffine.c.
2151 P2Child
2153 Forces "Pars" to be "PARENT" and "CHILD", the function's "GenericTypes"
2155 to be all of them, no "HaveBroadcasting" or "CallCopy", and turns on
2156 "DefaultFlow" (so do not supply any of those args). Intended for
2157 affine transformations with dataflow.
2158 DefaultFlow
2160 If true, sets in the "pdl_transvtable" (see PDL::Internals) the
2162 "iflags" such that the trans will start with dataflow both forwards and
2163 backwards.
2164 HaveBroadcasting
2166 Default true. If so, generate code implementing broadcasting (see
2168 PDL::Indexing).
2169 CallCopy
2171 For parameters that get created, normally the "PDL->initialize" will be
2173 used (or on a subclass). If this is true (which is the default for
2174 simple functions i.e. 2-arg with 0-dim signatures), instead the first
2175 argument's "copy" method will be used.
2176 TwoWay
2178 If true, sets in the "pdl_transvtable" (see PDL::Internals) the
2180 "iflags" such as to inform the trans's error checks connected to
2181 dataflow.
2182 Identity
2184 If true, sets "RedoDims" "EquivCPOffsCode" "HandleBad" "P2Child"
2186 "TwoWay" such that the function is a dataflowing identity
2187 transformation.
2188 BackCode
2190 For dataflowing functions, this value (which gets parsed) overrides the
2192 operation of that from children ndarrays to parents.
2193 BadBackCode
2195 Same but taking account of bad values.
2197 EquivCPOffsCode
2199 If supplied, allows concise control of copying to Child from Parent the
2201 data considered Equivalent at each given Offset (hence the name); the
2202 "Code" and "BackCode" will be generated from this.
2203 Example:
2205 pp_def(
2207 '_clump_int',
2208 OtherPars => 'int n',
2209 P2Child => 1,
2210 RedoDims => # omitted
2211 EquivCPOffsCode => '
2212 PDL_Indx i;
2213 for(i=0; i<$PDL(CHILD)->nvals; i++) $EQUIVCPOFFS(i,i);
2214 ',
2215 );
2216
2218 Macros
2219 $variablename_from_sig()
2220 access a pdl (by its name) that was specified in the signature
2222 $COMP(x)
2224 access a value in the private data structure of this transformation
2226 (mainly used to use an argument that is specified in the "OtherPars"
2227 section)
2228 $SIZE(n)
2230 replaced at runtime by the actual size of a named dimension (as
2232 specified in the signature)
2233 $GENERIC()
2235 replaced by the C type that is equal to the runtime type of the
2237 operation
2238 $P(a)
2240 a pointer to the data of the PDL named "a" in the signature. Useful for
2242 interfacing to C functions
2243 $PP(a)
2245 a physical pointer access to pdl "a"; mainly for internal use
2247 $TXYZ(AlternativeX,AlternativeY,AlternativeZ)
2249 expansion alternatives according to runtime type of operation, where
2251 XXX is some string that is matched by "/[BSULNQFD+]/".
2252 $PDL(a)
2254 return a pointer to the pdl data structure (pdl *) of ndarray "a"
2256 $ISBAD(a())
2258 returns true if the value stored in "a()" equals the bad value for this
2260 ndarray. Requires "HandleBad" being set to 1.
2261 $ISGOOD(a())
2263 returns true if the value stored in "a()" does not equal the bad value
2265 for this ndarray. Requires "HandleBad" being set to 1.
2266 $SETBAD(a())
2268 Sets "a()" to equal the bad value for this ndarray. Requires
2270 "HandleBad" being set to 1.
2271 $PRIV()
2273 To access fields in the "pdl_trans", eg "$PRIV(offs)".
2275 $CROAK()
2277 Returns a "pdl_error" with the supplied (var-args) message, adding the
2279 function name at the start, which will cause a "barf" within the Perl
2280 code. This is (as of 2.064) a change in PDL functions' API, so that
2281 callers can handle exceptions in their preferred way, which may not use
2282 Perl at all.
2283 $EQUIVCPOFFS()
2285 Copy from the "PARENT" parameter at the first given offset, to the
2287 "CHILD" parameter at the second given offset.
2288 $EQUIVCPTRUNC()
2290 Similar, but if the expression given as the third parameter is false,
2292 instead set the "CHILD"'s value to 0.
2293 $DOCOMPALLOC()
2295 Allocates memory for any "Comp" arrays, after their size has been
2297 determined, e.g. here after "$COMP(whichdims_count)" has been set:
2298 Comp => 'PDL_Indx whichdims[$COMP(whichdims_count)]',
2300 $DOPRIVALLOC()
2302 As above, except the key is "Priv"; because it is "Priv", this is only
2304 for entries in the "pdl_trans" itself, and almost certainly only for
2305 operations where "AffinePriv" is true.
2306 $SETNDIMS()
2308 For affine transformations (specifically, ones which set P2Child to
2310 true), set the child's "ndims" to the given value and allocate a
2311 suitably-sized array of dimension values.
2312 $SETDIMS()
2314 Similarly for affine transformations, after the above and then the
2316 actual dimension sizes are set, use this to resize the child ndarray to
2317 the right size.
2318 $SETDELTABROADCASTIDS()
2320 Similarly again, this sets the child's "nbroadcastids" to the same as
2322 the parent's, allocates space for the "broadcastids", then sets the
2323 child's ones to the same as the parent's plus the given value.
2324 To get a flavour of what "broadcastids" are for, in the normal way of
2326 things the first (0th) one in the parent is the highest dimension-
2327 number in it. See PDL::Indexing for more.
2328 functions
2330 "loop(DIMS) %{ ... %}"
2331 loop over named dimensions; limits are generated automatically by PP
2333 "broadcastloop %{ ... %}"
2335 enclose following code in a broadcast loop
2337 As of 2.075, "threadloop" is a deprecated alias for this.
2339 "types(TYPES) %{ ... %}"
2341 execute following code if type of operation is any of "TYPES"
2343
2345 A number of functions are imported when you "use PDL::PP". These
2346 include functions that control the generated C or XS code, functions
2347 that control the generated Perl code, and functions that manipulate the
2348 packages and symbol tables into which the code is created.
2349 Generating C and XS Code
2351 PDL::PP's main purpose is to make it easy for you to wrap the
2352 broadcasting engine around your own C code, but you can do some other
2353 things, too.
2354 pp_def
2356 Used to wrap the broadcasting engine around your C code. Virtually all
2358 of this document discusses the use of pp_def.
2359 pp_done
2361 Indicates you are done with PDL::PP and that it should generate its .xs
2363 and .pm files based upon the other pp_* functions that you have called.
2364 This function takes no arguments.
2365 pp_addxs
2367 This lets you add XS code to your .xs file. This is useful if you want
2369 to create Perl-accessible functions that invoke C code but cannot or
2370 should not invoke the broadcasting engine. XS is the standard means by
2371 which you wrap Perl-accessible C code. You can learn more at perlxs.
2372 pp_add_boot
2374 This function adds whatever string you pass to the XS BOOT section. The
2376 BOOT section is C code that gets called by Perl when your module is
2377 loaded and is useful for automatic initialization. You can learn more
2378 about XS and the BOOT section at perlxs.
2379 pp_addhdr
2381 Adds pure-C code to your XS file. XS files are structured such that
2383 pure C code must come before XS specifications. This allows you to
2384 specify such C code.
2385 pp_boundscheck
2387 PDL normally checks the bounds of your accesses before making them. You
2389 can turn that on or off at runtime by setting
2390 MyPackage::set_boundscheck. This function allows you to remove that
2391 runtime flexibility and never do bounds checking. It also returns the
2392 current boundschecking status if called without any argumens.
2393 NOTE: I have not found anything about bounds checking in other
2395 documentation. That needs to be addressed.
2396 Generating Perl Code
2398 Many functions imported when you use PDL::PP allow you to modify the
2399 contents of the generated .pm file. In addition to pp_def and pp_done,
2400 the role of these functions is primarily to add code to various parts
2401 of your generated .pm file.
2402 pp_addpm
2404 Adds Perl code to the generated .pm file. PDL::PP actually keeps track
2406 of three different sections of generated code: the Top, the Middle, and
2407 the Bottom. You can add Perl code to the Middle section using the one-
2408 argument form, where the argument is the Perl code you want to supply.
2409 In the two-argument form, the first argument is an anonymous hash with
2410 only one key that specifies where to put the second argument, which is
2411 the string that you want to add to the .pm file. The hash is one of
2412 these three:
2413 {At => 'Top'}
2415 {At => 'Middle'}
2416 {At => 'Bot'}
2417 For example:
2419 pp_addpm({At => 'Bot'}, <<POD);
2421 =head1 Some documentation
2423 I know I'm typing this in the middle of my file, but it'll go at
2425 the bottom.
2426 =cut
2428 POD
2430 Warning: If, in the middle of your .pd file, you put documentation
2432 meant for the bottom of your pod, you will thoroughly confuse CPAN. On
2433 the other hand, if in the middle of your .pd file, you add some Perl
2434 code destined for the bottom or top of your .pm file, you only have
2435 yourself to confuse. :-)
2436 pp_beginwrap
2438 Adds BEGIN-block wrapping. Certain declarations can be wrapped in BEGIN
2440 blocks, though the default behavior is to have no such wrapping.
2441 pp_addbegin
2443 Sets code to be added to the top of your .pm file, even above code that
2445 you specify with "pp_addpm({At => 'Top'}, ...)". Unlike pp_addpm,
2446 calling this overwrites whatever was there before. Generally, you
2447 probably shouldn't use it.
2448 Tracking Line Numbers
2450 When you get compile errors, either from your C-like code or your Perl
2451 code, it can help to make those errors back to the line numbers in the
2452 source file at which the error occurred.
2453 pp_line_numbers
2455 Takes a line number and a (usually long) string of code. The line
2457 number should indicate the line at which the quote begins. This is
2458 usually Perl's "__LINE__" literal, unless you are using heredocs, in
2459 which case it is "__LINE__ + 1". The returned string has #line
2460 directives interspersed to help the compiler report errors on the
2461 proper line.
2462 Modifying the Symbol Table and Export Behavior
2464 PDL::PP usually exports all functions generated using pp_def, and
2465 usually installs them into the PDL symbol table. However, you can
2466 modify this behavior with these functions.
2467 pp_bless
2469 Sets the package (symbol table) to which the XS code is added. The
2471 default is PDL, which is generally what you want. If you use the
2472 default blessing and you create a function myfunc, then you can do the
2473 following:
2474 $ndarray->myfunc(<args>);
2476 PDL::myfunc($ndarray, <args>);
2477 On the other hand, if you bless your functions into another package,
2479 you cannot invoke them as PDL methods, and must invoke them as:
2480 MyPackage::myfunc($ndarray, <args>);
2482 Of course, you could always use the PMFunc key to add your function to
2484 the PDL symbol table, but why do that?
2485 pp_add_isa
2487 Adds to the list of modules from which your module inherits. The
2489 default list is
2490 qw(PDL::Exporter DynaLoader)
2492 pp_core_importlist
2494 At the top of your generated .pm file is a line that looks like this:
2496 use PDL::Core;
2498 You can modify that by specifying a string to pp_core_importlist. For
2500 example,
2501 pp_core_importlist('::Blarg');
2503 will result in
2505 use PDL::Core::Blarg;
2507 You can use this, for example, to add a list of symbols to import from
2509 PDL::Core. For example:
2510 pp_core_importlist(" ':Internal'");
2512 will lead to the following use statement:
2514 use PDL::Core ':Internal';
2516 pp_setversion
2518 Sets your module's version. The version must be consistent between the
2520 .xs and the .pm file, and is used to ensure that your Perl's libraries
2521 do not suffer from version skew.
2522 pp_add_exported
2524 Adds to the export list whatever names you give it. Functions created
2526 using pp_def are automatically added to the list. This function is
2527 useful if you define any Perl functions using pp_addpm or pp_addxs that
2528 you want exported as well.
2529 pp_export_nothing
2531 This resets the list of exported symbols to nothing. This is probably
2533 better called "pp_export_clear", since you can add exported symbols
2534 after calling "pp_export_nothing". When called just before calling
2535 pp_done, this ensures that your module does not export anything, for
2536 example, if you only want programmers to use your functions as methods.
2537
2539 For the concepts of broadcasting and slicing check PDL::Indexing.
2540 PDL::Internals
2542 PDL::BadValues for information on bad values
2544 perlxs, perlxstut
2546 Practical Magick with C, PDL, and PDL::PP -- a guide to compiled add-
2548 ons for PDL <https://arxiv.org/abs/1702.07753>
2549
2551 Although PDL::PP is quite flexible and thoroughly used, there are
2552 surely bugs. First amongst them: this documentation needs a thorough
2553 revision.
2554
2556 Copyright(C) 1997 Tuomas J. Lukka (lukka@fas.harvard.edu), Karl
2557 Glaazebrook (kgb@aaocbn1.aao.GOV.AU) and Christian Soeller
2558 (c.soeller@auckland.ac.nz). All rights reserved. Documentation updates
2559 Copyright(C) 2011 David Mertens (dcmertens.perl@gmail.com). This
2560 documentation is licensed under the same terms as Perl itself.
2561perl v5.36.0 2022-07-22 PP(1)