1PP(1) User Contributed Perl Documentation PP(1)
2
6 PDL::PP - Generate PDL routines from concise descriptions
7
9 e.g.
10 pp_def(
12 'sumover',
13 Pars => 'a(n); [o]b();',
14 Code => q{
15 double tmp=0;
16 loop(n) %{
17 tmp += $a();
18 %}
19 $b() = tmp;
20 },
21 );
22 pp_done();
24
26 Here is a quick reference list of the functions provided by PDL::PP.
27 pp_add_boot
29 Add code to the BOOT section of generated XS file
30 pp_add_exported
32 Add functions to the list of exported functions
33 pp_add_isa
35 Add entries to the @ISA list
36 pp_addbegin
38 Sets code to be added at the top of the generate .pm file
39 pp_addhdr
41 Add code and includes to C section of the generated XS file
42 pp_addpm
44 Add code to the generated .pm file
45 pp_addxs
47 Add extra XS code to the generated XS file
48 pp_beginwrap
50 Add BEGIN-block wrapping to code for the generated .pm file
51 pp_bless
53 Sets the package to which the XS code is added (default is PDL)
54 pp_boundscheck
56 Control state of PDL bounds checking activity
57 pp_core_importList
59 Specify what is imported from PDL::Core
60 pp_def
62 Define a new PDL function
63 pp_deprecate_module
65 Add runtime and POD warnings about a module being deprecated
66 pp_done
68 Mark the end of PDL::PP definitions in the file
69 pp_export_nothing
71 Clear out the export list for your generated module
72 pp_line_numbers
74 Add line number information to simplify debugging of PDL::PP code
75 pp_setversion
77 Set the version for .pm and .xs files
78
80 Why do we need PP? Several reasons: firstly, we want to be able to
81 generate subroutine code for each of the PDL datatypes (PDL_Byte,
82 PDL_Short, etc). AUTOMATICALLY. Secondly, when referring to slices of
83 PDL arrays in Perl (e.g. "$a->slice('0:10:2,:')" or other things such
84 as transposes) it is nice to be able to do this transparently and to be
85 able to do this 'in-place' - i.e, not to have to make a memory copy of
86 the section. PP handles all the necessary element and offset arithmetic
87 for you. There are also the notions of threading (repeated calling of
88 the same routine for multiple slices, see PDL::Indexing) and dataflow
89 (see PDL::Dataflow) which use of PP allows.
90 In much of what follows we will assume familiarity of the reader with
92 the concepts of implicit and explicit threading and index manipulations
93 within PDL. If you have not yet heard of these concepts or are not very
94 comfortable with them it is time to check PDL::Indexing.
95 As you may appreciate from its name PDL::PP is a Pre-Processor, i.e.
97 it expands code via substitutions to make real C-code. Technically, the
98 output is XS code (see perlxs) but that is very close to C.
99 So how do you use PP? Well for the most part you just write ordinary C
101 code except for special PP constructs which take the form:
102 $something(something else)
104 or:
106 PPfunction %{
108 <stuff>
109 %}
110 The most important PP construct is the form "$array()". Consider the
112 very simple PP function to sum the elements of a 1D vector (in fact
113 this is very similar to the actual code used by 'sumover'):
114 pp_def('sumit',
116 Pars => 'a(n); [o]b();',
117 Code => q{
118 double tmp;
119 tmp = 0;
120 loop(n) %{
121 tmp += $a();
122 %}
123 $b() = tmp;
124 }
125 );
126 What's going on? The "Pars =>" line is very important for PP - it
128 specifies all the arguments and their dimensionality. We call this the
129 signature of the PP function (compare also the explanations in
130 PDL::Indexing). In this case the routine takes a 1-D function as input
131 and returns a 0-D scalar as output. The "$a()" PP construct is used to
132 access elements of the array a(n) for you - PP fills in all the
133 required C code.
134 You will notice that we are using the "q{}" single-quote operator. This
136 is not an accident. You generally want to use single quotes to denote
137 your PP Code sections. PDL::PP uses "$var()" for its parsing and if you
138 don't use single quotes, Perl will try to interpolate "$var()". Also,
139 using the single quote "q" operator with curly braces makes it look
140 like you are creating a code block, which is What You Mean. (Perl is
141 smart enough to look for nested curly braces and not close the quote
142 until it finds the matching curly brace, so it's safe to have nested
143 blocks.) Under other circumstances, such as when you're stitching
144 together a Code block using string concatenations, it's often easiest
145 to use real single quotes as
146 Code => 'something'.$interpolatable.'somethingelse;'
148 In the simple case here where all elements are accessed the PP
150 construct "loop(n) %{ ... %}" is used to loop over all elements in
151 dimension "n". Note this feature of PP: ALL DIMENSIONS ARE SPECIFIED
152 BY NAME.
153 This is made clearer if we avoid the PP loop() construct and write the
155 loop explicitly using conventional C:
156 pp_def('sumit',
158 Pars => 'a(n); [o]b();',
159 Code => q{
160 PDL_Indx i,n_size;
161 double tmp;
162 n_size = $SIZE(n);
163 tmp = 0;
164 for(i=0; i<n_size; i++) {
165 tmp += $a(n=>i);
166 }
167 $b() = tmp;
168 },
169 );
170 which does the same as before, but is more long-winded. You can see to
172 get element "i" of a() we say "$a(n=>i)" - we are specifying the
173 dimension by name "n". In 2D we might say:
174 Pars=>'a(m,n);',
176 ...
177 tmp += $a(m=>i,n=>j);
178 ...
179 The syntax "m=>i" borrows from Perl hashes, which are in fact used in
181 the implementation of PP. One could also say "$a(n=>j,m=>i)" as order
182 is not important.
183 You can also see in the above example the use of another PP construct -
185 $SIZE(n) to get the length of the dimension "n".
186 It should, however, be noted that you shouldn't write an explicit
188 C-loop when you could have used the PP "loop" construct since PDL::PP
189 checks automatically the loop limits for you, usage of "loop" makes the
190 code more concise, etc. But there are certainly situations where you
191 need explicit control of the loop and now you know how to do it ;).
192 To revisit 'Why PP?' - the above code for sumit() will be generated for
194 each data-type. It will operate on slices of arrays 'in-place'. It will
195 thread automatically - e.g. if a 2D array is given it will be called
196 repeatedly for each 1D row (again check PDL::Indexing for the details
197 of threading). And then b() will be a 1D array of sums of each row.
198 We could call it with $a->xchg(0,1) to sum the columns instead. And
199 Dataflow tracing etc. will be available.
200 You can see PP saves the programmer from writing a lot of needlessly
202 repetitive C-code -- in our opinion this is one of the best features of
203 PDL making writing new C subroutines for PDL an amazingly concise
204 exercise. A second reason is the ability to make PP expand your concise
205 code definitions into different C code based on the needs of the
206 computer architecture in question. Imagine for example you are lucky to
207 have a supercomputer at your hands; in that case you want PDL::PP
208 certainly to generate code that takes advantage of the
209 vectorising/parallel computing features of your machine (this a project
210 for the future). In any case, the bottom line is that your unchanged
211 code should still expand to working XS code even if the internals of
212 PDL changed.
213 Also, because you are generating the code in an actual Perl script,
215 there are many fun things that you can do. Let's say that you need to
216 write both sumit (as above) and multit. With a little bit of
217 creativity, we can do
218 for({Name => 'sumit', Init => '0', Op => '+='},
220 {Name => 'multit', Init => '1', Op => '*='}) {
221 pp_def($_->{Name},
222 Pars => 'a(n); [o]b();',
223 Code => '
224 double tmp;
225 tmp = '.$_->{Init}.';
226 loop(n) %{
227 tmp '.$_->{Op}.' $a();
228 %}
229 $b() = tmp;
230 ');
231 }
232 which defines both the functions easily. Now, if you later need to
234 change the signature or dimensionality or whatever, you only need to
235 change one place in your code. Yeah, sure, your editor does have 'cut
236 and paste' and 'search and replace' but it's still less bothersome and
237 definitely more difficult to forget just one place and have strange
238 bugs creep in. Also, adding 'orit' (bitwise or) later is a one-liner.
239 And remember, you really have Perl's full abilities with you - you can
241 very easily read any input file and make routines from the information
242 in that file. For simple cases like the above, the author (Tjl)
243 currently favors the hash syntax like the above - it's not too much
244 more characters than the corresponding array syntax but much easier to
245 understand and change.
246 We should mention here also the ability to get the pointer to the
248 beginning of the data in memory - a prerequisite for interfacing PDL to
249 some libraries. This is handled with the "$P(var)" directive, see
250 below.
251 When starting work on a new pp_def'ined function, if you make a
253 mistake, you will usually find a pile of compiler errors indicating
254 line numbers in the generated XS file. If you know how to read XS files
255 (or if you want to learn the hard way), you could open the generated XS
256 file and search for the line number with the error. However, a recent
257 addition to PDL::PP helps report the correct line number of your
258 errors: "pp_line_numbers". Working with the original summit example, if
259 you had a mis-spelling of tmp in your code, you could change the
260 (erroneous) code to something like this and the compiler would give you
261 much more useful information:
262 pp_def('sumit',
264 Pars => 'a(n); [o]b();',
265 Code => pp_line_numbers(__LINE__, q{
266 double tmp;
267 tmp = 0;
268 loop(n) %{
269 tmp += $a();
270 %}
271 $b() = rmp;
272 })
273 );
274 For the above situation, my compiler tells me:
276 ...
278 test.pd:15: error: 'rmp' undeclared (first use in this function)
279 ...
280 In my example script (called test.pd), line 15 is exactly the line at
282 which I made my typo: "rmp" instead of "tmp".
283 So, after this quick overview of the general flavour of programming PDL
285 routines using PDL::PP let's summarise in which circumstances you
286 should actually use this preprocessor/precompiler. You should use
287 PDL::PP if you want to
288 · interface PDL to some external library
290 · write some algorithm that would be slow if coded in Perl (this is
292 not as often as you think; take a look at threading and dataflow
293 first).
294 · be a PDL developer (and even then it's not obligatory)
296
298 Because of its architecture, PDL::PP can be both flexible and easy to
299 use on the one hand, yet exuberantly complicated at the same time.
300 Currently, part of the problem is that error messages are not very
301 informative and if something goes wrong, you'd better know what you are
302 doing and be able to hack your way through the internals (or be able to
303 figure out by trial and error what is wrong with your args to
304 "pp_def"). Although work is being done to produce better warnings, do
305 not be afraid to send your questions to the mailing list if you run
306 into trouble.
307
309 Now that you have some idea how to use "pp_def" to define new PDL
310 functions it is time to explain the general syntax of "pp_def".
311 "pp_def" takes as arguments first the name of the function you are
312 defining and then a hash list that can contain various keys.
313 Based on these keys PP generates XS code and a .pm file. The function
315 "pp_done" (see example in the SYNOPSIS) is used to tell PDL::PP that
316 there are no more definitions in this file and it is time to generate
317 the .xs and
318 .pm file.
319 As a consequence, there may be several pp_def() calls inside a file (by
321 convention files with PP code have the extension .pd or .pp) but
322 generally only one pp_done().
323 There are two main different types of usage of pp_def(), the 'data
325 operation' and 'slice operation' prototypes.
326 The 'data operation' is used to take some data, mangle it and output
328 some other data; this includes for example the '+' operation, matrix
329 inverse, sumover etc and all the examples we have talked about in this
330 document so far. Implicit and explicit threading and the creation of
331 the result are taken care of automatically in those operations. You can
332 even do dataflow with "sumit", "sumover", etc (don't be dismayed if you
333 don't understand the concept of dataflow in PDL very well yet; it is
334 still very much experimental).
335 The 'slice operation' is a different kind of operation: in a slice
337 operation, you are not changing any data, you are defining
338 correspondences between different elements of two piddles (examples
339 include the index manipulation/slicing function definitions in the file
340 slices.pd that is part of the PDL distribution; but beware, this is not
341 introductory level stuff).
342 If PDL was compiled with support for bad values (i.e. "WITH_BADVAL =>
344 1"), then additional keys are required for "pp_def", as explained
345 below.
346 If you are just interested in communicating with some external library
348 (for example some linear algebra/matrix library), you'll usually want
349 the 'data operation' so we are going to discuss that first.
350
352 A simple example
353 In the data operation, you must know what dimensions of data you need.
354 First, an example with scalars:
355 pp_def('add',
357 Pars => 'a(); b(); [o]c();',
358 Code => '$c() = $a() + $b();'
359 );
360 That looks a little strange but let's dissect it. The first line is
362 easy: we're defining a routine with the name 'add'. The second line
363 simply declares our parameters and the parentheses mean that they are
364 scalars. We call the string that defines our parameters and their
365 dimensionality the signature of that function. For its relevance with
366 regard to threading and index manipulations check the PDL::Indexing man
367 page.
368 The third line is the actual operation. You need to use the dollar
370 signs and parentheses to refer to your parameters (this will probably
371 change at some point in the future, once a good syntax is found).
372 These lines are all that is necessary to actually define the function
374 for PDL (well, actually it isn't; you additionally need to write a
375 Makefile.PL (see below) and build the module (something like 'perl
376 Makefile.PL; make'); but let's ignore that for the moment). So now you
377 can do
378 use MyModule;
380 $a = pdl 2,3,4;
381 $b = pdl 5;
382 $c = add($a,$b);
384 # or
385 add($a,$b,($c=null)); # Alternative form, useful if $c has been
386 # preset to something big, not useful here.
387 and have threading work correctly (the result is $c == [7 8 9]).
389 The Pars section: the signature of a PP function
391 Seeing the above example code you will most probably ask: what is this
392 strange "$c=null" syntax in the second call to our new "add" function?
393 If you take another look at the definition of "add" you will notice
394 that the third argument "c" is flagged with the qualifier "[o]" which
395 tells PDL::PP that this is an output argument. So the above call to add
396 means 'create a new $c from scratch with correct dimensions' - "null"
397 is a special token for 'empty piddle' (you might ask why we haven't
398 used the value "undef" to flag this instead of the PDL specific "null";
399 we are currently thinking about it ;).
400 [This should be explained in some other section of the manual as
402 well!!] The reason for having this syntax as an alternative is that if
403 you have really huge piddles, you can do
404 $c = PDL->null;
406 for(some long loop) {
407 # munge a,b
408 add($a,$b,$c);
409 # munge c, put something back to a,b
410 }
411 and avoid allocating and deallocating $c each time. It is allocated
413 once at the first add() and thereafter the memory stays until $c is
414 destroyed.
415 If you just say
417 $c = add($a,$b);
419 the code generated by PP will automatically fill in "$c=null" and
421 return the result. If you want to learn more about the reasons why
422 PDL::PP supports this style where output arguments are given as last
423 arguments check the PDL::Indexing man page.
424 "[o]" is not the only qualifier a pdl argument can have in the
426 signature. Another important qualifier is the "[t]" option which flags
427 a pdl as temporary. What does that mean? You tell PDL::PP that this
428 pdl is only used for temporary results in the course of the calculation
429 and you are not interested in its value after the computation has been
430 completed. But why should PDL::PP want to know about this in the first
431 place? The reason is closely related to the concepts of pdl auto
432 creation (you heard about that above) and implicit threading. If you
433 use implicit threading the dimensionality of automatically created pdls
434 is actually larger than that specified in the signature. With "[o]"
435 flagged pdls will be created so that they have the additional
436 dimensions as required by the number of implicit thread dimensions.
437 When creating a temporary pdl, however, it will always only be made big
438 enough so that it can hold the result for one iteration in a thread
439 loop, i.e. as large as required by the signature. So less memory is
440 wasted when you flag a pdl as temporary. Secondly, you can use output
441 auto creation with temporary pdls even when you are using explicit
442 threading which is forbidden for normal output pdls flagged with "[o]"
443 (see PDL::Indexing).
444 Here is an example where we use the [t] qualifier. We define the
446 function "callf" that calls a C routine "f" which needs a temporary
447 array of the same size and type as the array "a" (sorry about the
448 forward reference for $P; it's a pointer access, see below) :
449 pp_def('callf',
451 Pars => 'a(n); [t] tmp(n); [o] b()',
452 Code => 'PDL_Indx ns = $SIZE(n);
453 f($P(a),$P(b),$P(tmp),ns);
454 '
455 );
456 Argument dimensions and the signature
458 Now we have just talked about dimensions of pdls and the signature. How
459 are they related? Let's say that we want to add a scalar + the index
460 number to a vector:
461 pp_def('add2',
463 Pars => 'a(n); b(); [o]c(n);',
464 Code => 'loop(n) %{
465 $c() = $a() + $b() + n;
466 %}'
467 );
468 There are several points to notice here: first, the "Pars" argument now
470 contains the n arguments to show that we have a single dimensions in a
471 and c. It is important to note that dimensions are actual entities that
472 are accessed by name so this declares a and c to have the same first
473 dimensions. In most PP definitions the size of named dimensions will be
474 set from the respective dimensions of non-output pdls (those with no
475 "[o]" flag) but sometimes you might want to set the size of a named
476 dimension explicitly through an integer parameter. See below in the
477 description of the "OtherPars" section how that works.
478 Constant argument dimensions in the signature
480 Suppose you want an output piddle to be created automatically and you
481 know that on every call its dimension will have the same size (say 9)
482 regardless of the dimensions of the input piddles. In this case you use
483 the following syntax in the Pars section to specify the size of the
484 dimension:
485 ' [o] y(n=9); '
487 As expected, extra dimensions required by threading will be created if
489 necessary. If you need to assign a named dimension according to a more
490 complicated formula (than a constant) you must use the "RedoDimsCode"
491 key described below.
492 Type conversions and the signature
494 The signature also determines the type conversions that will be
495 performed when a PP function is invoked. So what happens when we invoke
496 one of our previously defined functions with pdls of different type,
497 e.g.
498 add2($a,$b,($ret=null));
500 where $a is of type "PDL_Float" and $b of type "PDL_Short"? With the
502 signature as shown in the definition of "add2" above the datatype of
503 the operation (as determined at runtime) is that of the pdl with the
504 'highest' type (sequence is byte < short < ushort < long < float <
505 double). In the add2 example the datatype of the operation is float ($a
506 has that datatype). All pdl arguments are then type converted to that
507 datatype (they are not converted inplace but a copy with the right type
508 is created if a pdl argument doesn't have the type of the operation).
509 Null pdls don't contribute a type in the determination of the type of
510 the operation. However, they will be created with the datatype of the
511 operation; here, for example, $ret will be of type float. You should be
512 aware of these rules when calling PP functions with pdls of different
513 types to take the additional storage and runtime requirements into
514 account.
515 These type conversions are correct for most functions you normally
517 define with "pp_def". However, there are certain cases where slightly
518 modified type conversion behaviour is desired. For these cases
519 additional qualifiers in the signature can be used to specify the
520 desired properties with regard to type conversion. These qualifiers can
521 be combined with those we have encountered already (the creation
522 qualifiers "[o]" and "[t]"). Let's go through the list of qualifiers
523 that change type conversion behaviour.
524 The most important is the "indx" qualifier which comes in handy when a
526 pdl argument represents indices into another pdl. Let's take a look at
527 an example from "PDL::Ufunc":
528 pp_def('maximum_ind',
530 Pars => 'a(n); indx [o] b()',
531 Code => '$GENERIC() cur;
532 PDL_Indx curind;
533 loop(n) %{
534 if (!n || $a() > cur) {cur = $a(); curind = n;}
535 %}
536 $b() = curind;',
537 );
538 The function "maximum_ind" finds the index of the largest element of a
540 vector. If you look at the signature you notice that the output
541 argument "b" has been declared with the additional "indx" qualifier.
542 This has the following consequences for type conversions: regardless of
543 the type of the input pdl "a" the output pdl "b" will be of type
544 "PDL_Indx" which makes sense since "b" will represent an index into
545 "a".
546 Note that 'curind' is declared as type "PDL_Indx" and not "indx".
548 While most datatype declarations in the 'Pars' section use the same
549 name as the underlying C type, "indx" is a type which is sufficient to
550 handle PDL indexing operations. For 32-bit installs, it can be a
551 32-bit integer type. For 64-bit installs, it will be a 64-bit integer
552 type.
553 Furthermore, if you call the function with an existing output pdl "b"
555 its type will not influence the datatype of the operation (see above).
556 Hence, even if "a" is of a smaller type than "b" it will not be
557 converted to match the type of "b" but stays untouched, which saves
558 memory and CPU cycles and is the right thing to do when "b" represents
559 indices. Also note that you can use the 'indx' qualifier together with
560 other qualifiers (the "[o]" and "[t]" qualifiers). Order is significant
561 -- type qualifiers precede creation qualifiers ("[o]" and "[t]").
562 The above example also demonstrates typical usage of the "$GENERIC()"
564 macro. It expands to the current type in a so called generic loop.
565 What is a generic loop? As you already heard a PP function has a
566 runtime datatype as determined by the type of the pdl arguments it has
567 been invoked with. The PP generated XS code for this function
568 therefore contains a switch like "switch (type) {case PDL_Byte: ...
569 case PDL_Double: ...}" that selects a case based on the runtime
570 datatype of the function (it's called a type ``loop'' because there is
571 a loop in PP code that generates the cases). In any case your code is
572 inserted once for each PDL type into this switch statement. The
573 "$GENERIC()" macro just expands to the respective type in each copy of
574 your parsed code in this "switch" statement, e.g., in the "case
575 PDL_Byte" section "cur" will expand to "PDL_Byte" and so on for the
576 other case statements. I guess you realise that this is a useful macro
577 to hold values of pdls in some code.
578 There are a couple of other qualifiers with similar effects as "indx".
580 For your convenience there are the "float" and "double" qualifiers with
581 analogous consequences on type conversions as "indx". Let's assume you
582 have a very large array for which you want to compute row and column
583 sums with an equivalent of the "sumover" function. However, with the
584 normal definition of "sumover" you might run into problems when your
585 data is, e.g. of type short. A call like
586 sumover($large_pdl,($sums = null));
588 will result in $sums be of type short and is therefore prone to
590 overflow errors if $large_pdl is a very large array. On the other hand
591 calling
592 @dims = $large_pdl->dims; shift @dims;
594 sumover($large_pdl,($sums = zeroes(double,@dims)));
595 is not a good alternative either. Now we don't have overflow problems
597 with $sums but at the expense of a type conversion of $large_pdl to
598 double, something bad if this is really a large pdl. That's where
599 "double" comes in handy:
600 pp_def('sumoverd',
602 Pars => 'a(n); double [o] b()',
603 Code => 'double tmp=0;
604 loop(n) %{ tmp += a(); %}
605 $b() = tmp;',
606 );
607 This gets us around the type conversion and overflow problems. Again,
609 analogous to the "indx" qualifier "double" results in "b" always being
610 of type double regardless of the type of "a" without leading to a type
611 conversion of "a" as a side effect.
612 Finally, there are the "type+" qualifiers where type is one of "int" or
614 "float". What shall that mean. Let's illustrate the "int+" qualifier
615 with the actual definition of sumover:
616 pp_def('sumover',
618 Pars => 'a(n); int+ [o] b()',
619 Code => '$GENERIC(b) tmp=0;
620 loop(n) %{ tmp += a(); %}
621 $b() = tmp;',
622 );
623 As we had already seen for the "int", "float" and "double" qualifiers,
625 a pdl marked with a "type+" qualifier does not influence the datatype
626 of the pdl operation. Its meaning is "make this pdl at least of type
627 "type" or higher, as required by the type of the operation". In the
628 sumover example this means that when you call the function with an "a"
629 of type PDL_Short the output pdl will be of type PDL_Long (just as
630 would have been the case with the "int" qualifier). This again tries to
631 avoid overflow problems when using small datatypes (e.g. byte images).
632 However, when the datatype of the operation is higher than the type
633 specified in the "type+" qualifier "b" will be created with the
634 datatype of the operation, e.g. when "a" is of type double then "b"
635 will be double as well. We hope you agree that this is sensible
636 behaviour for "sumover". It should be obvious how the "float+"
637 qualifier works by analogy. It may become necessary to be able to
638 specify a set of alternative types for the parameters. However, this
639 will probably not be implemented until someone comes up with a
640 reasonable use for it.
641 Note that we now had to specify the $GENERIC macro with the name of the
643 pdl to derive the type from that argument. Why is that? If you
644 carefully followed our explanations you will have realised that in some
645 cases "b" will have a different type than the type of the operation.
646 Calling the '$GENERIC' macro with "b" as argument makes sure that the
647 type will always the same as that of "b" in that part of the generic
648 loop.
649 This is about all there is to say about the "Pars" section in a
651 "pp_def" call. You should remember that this section defines the
652 signature of a PP defined function, you can use several options to
653 qualify certain arguments as output and temporary args and all
654 dimensions that you can later refer to in the "Code" section are
655 defined by name.
656 It is important that you understand the meaning of the signature since
658 in the latest PDL versions you can use it to define threaded functions
659 from within Perl, i.e. what we call Perl level threading. Please check
660 PDL::Indexing for details.
661 The Code section
663 The "Code" section contains the actual XS code that will be in the
664 innermost part of a thread loop (if you don't know what a thread loop
665 is then you still haven't read PDL::Indexing; do it now ;) after any PP
666 macros (like $GENERIC) and PP functions have been expanded (like the
667 "loop" function we are going to explain next).
668 Let's quickly reiterate the "sumover" example:
670 pp_def('sumover',
672 Pars => 'a(n); int+ [o] b()',
673 Code => '$GENERIC(b) tmp=0;
674 loop(n) %{ tmp += a(); %}
675 $b() = tmp;',
676 );
677 The "loop" construct in the "Code" section also refers to the dimension
679 name so you don't need to specify any limits: the loop is correctly
680 sized and everything is done for you, again.
681 Next, there is the surprising fact that "$a()" and "$b()" do not
683 contain the index. This is not necessary because we're looping over n
684 and both variables know which dimensions they have so they
685 automatically know they're being looped over.
686 This feature comes in very handy in many places and makes for much
688 shorter code. Of course, there are times when you want to circumvent
689 this; here is a function which make a matrix symmetric and serves as an
690 example of how to code explicit looping:
691 pp_def('symm',
693 Pars => 'a(n,n); [o]c(n,n);',
694 Code => 'loop(n) %{
695 int n2;
696 for(n2=n; n2<$SIZE(n); n2++) {
697 $c(n0 => n, n1 => n2) =
698 $c(n0 => n2, n1 => n) =
699 $a(n0 => n, n1 => n2);
700 }
701 %}
702 '
703 );
704 Let's dissect what is happening. Firstly, what is this function
706 supposed to do? From its signature you see that it takes a 2D matrix
707 with equal numbers of columns and rows and outputs a matrix of the same
708 size. From a given input matrix $a it computes a symmetric output
709 matrix $c (symmetric in the matrix sense that A^T = A where ^T means
710 matrix transpose, or in PDL parlance $c == $c->xchg(0,1)). It does this
711 by using only the values on and below the diagonal of $a. In the output
712 matrix $c all values on and below the diagonal are the same as those in
713 $a while those above the diagonal are a mirror image of those below the
714 diagonal (above and below are here interpreted in the way that PDL
715 prints 2D pdls). If this explanation still sounds a bit strange just go
716 ahead, make a little file into which you write this definition, build
717 the new PDL extension (see section on Makefiles for PP code) and try it
718 out with a couple of examples.
719 Having explained what the function is supposed to do there are a couple
721 of points worth noting from the syntactical point of view. First, we
722 get the size of the dimension named "n" again by using the $SIZE macro.
723 Second, there are suddenly these funny "n0" and "n1" index names in the
724 code though the signature defines only the dimension "n". Why this? The
725 reason becomes clear when you note that both the first and second
726 dimension of $a and $b are named "n" in the signature of "symm". This
727 tells PDL::PP that the first and second dimension of these arguments
728 should have the same size. Otherwise the generated function will raise
729 a runtime error. However, now in an access to $a and $c PDL::PP cannot
730 figure out which index "n" refers to any more just from the name of the
731 index. Therefore, the indices with equal dimension names get numbered
732 from left to right starting at 0, e.g. in the above example "n0" refers
733 to the first dimension of $a and $c, "n1" to the second and so on.
734 In all examples so far, we have only used the "Pars" and "Code" members
736 of the hash that was passed to "pp_def". There are certainly other keys
737 that are recognised by PDL::PP and we will hear about some of them in
738 the course of this document. Find a (non-exhaustive) list of keys in
739 Appendix A. A list of macros and PPfunctions (we have only encountered
740 some of those in the examples above yet) that are expanded in values of
741 the hash argument to "pp_def" is summarised in Appendix B.
742 At this point, it might be appropriate to mention that PDL::PP is not a
744 completely static, well designed set of routines (as Tuomas puts it:
745 "stop thinking of PP as a set of routines carved in stone") but rather
746 a collection of things that the PDL::PP author (Tuomas J. Lukka)
747 considered he would have to write often into his PDL extension
748 routines. PP tries to be expandable so that in the future, as new needs
749 arise, new common code can be abstracted back into it. If you want to
750 learn more on why you might want to change PDL::PP and how to do it
751 check the section on PDL::PP internals.
752 Handling bad values
754 If you do not have bad-value support compiled into PDL you can ignore
755 this section and the related keys: "BadCode", "HandleBad", ... (try
756 printing out the value of $PDL::Bad::Status - if it equals 0 then move
757 straight on).
758 There are several keys and macros used when writing code to handle bad
760 values. The first one is the "HandleBad" key:
761 HandleBad => 0
763 This flags a pp-routine as NOT handling bad values. If this routine
764 is sent piddles with their "badflag" set, then a warning message is
765 printed to STDOUT and the piddles are processed as if the value
766 used to represent bad values is a valid number. The "badflag" value
767 is not propagated to the output piddles.
768 An example of when this is used is for FFT routines, which
770 generally do not have a way of ignoring part of the data.
771 HandleBad => 1
773 This causes PDL::PP to write extra code that ensures the BadCode
774 section is used, and that the "$ISBAD()" macro (and its brethren)
775 work.
776 HandleBad is not given
778 If any of the input piddles have their "badflag" set, then the
779 output piddles will have their "badflag" set, but any supplied
780 BadCode is ignored.
781 The value of "HandleBad" is used to define the contents of the "BadDoc"
783 key, if it is not given.
784 To handle bad values, code must be written somewhat differently; for
786 instance,
787 $c() = $a() + $b();
789 becomes something like
791 if ( $a() != BADVAL && $b() != BADVAL ) {
793 $c() = $a() + $b();
794 } else {
795 $c() = BADVAL;
796 }
797 However, we only want the second version if bad values are present in
799 the input piddles (and that bad-value support is wanted!) - otherwise
800 we actually want the original code. This is where the "BadCode" key
801 comes in; you use it to specify the code to execute if bad values may
802 be present, and PP uses both it and the "Code" section to create
803 something like:
804 if ( bad_values_are_present ) {
806 fancy_threadloop_stuff {
807 BadCode
808 }
809 } else {
810 fancy_threadloop_stuff {
811 Code
812 }
813 }
814 This approach means that there is virtually no overhead when bad values
816 are not present (i.e. the badflag routine returns 0).
817 The C preprocessor symbol "PDL_BAD_CODE" is defined when the bad code
819 is compiled, so that you can reduce the amount of code you write. The
820 BadCode section can use the same macros and looping constructs as the
821 Code section. However, it wouldn't be much use without the following
822 additional macros:
823 $ISBAD(var)
825 To check whether a piddle's value is bad, use the $ISBAD macro:
826 if ( $ISBAD(a()) ) { printf("a() is bad\n"); }
828 You can also access given elements of a piddle:
830 if ( $ISBAD(a(n=>l)) ) { printf("element %d of a() is bad\n", l); }
832 $ISGOOD(var)
834 This is the opposite of the $ISBAD macro.
835 $SETBAD(var)
837 For when you want to set an element of a piddle bad.
838 $ISBADVAR(c_var,pdl)
840 If you have cached the value of a piddle "$a()" into a c-variable
841 ("foo" say), then to check whether it is bad, use
842 "$ISBADVAR(foo,a)".
843 $ISGOODVAR(c_var,pdl)
845 As above, but this time checking that the cached value isn't bad.
846 $SETBADVAR(c_var,pdl)
848 To copy the bad value for a piddle into a c variable, use
849 "$SETBADVAR(foo,a)".
850 TODO: mention "$PPISBAD()" etc macros.
852 Using these macros, the above code could be specified as:
854 Code => '$c() = $a() + $b();',
856 BadCode => '
857 if ( $ISBAD(a()) || $ISBAD(b()) ) {
858 $SETBAD(c());
859 } else {
860 $c() = $a() + $b();
861 }',
862 Since this is Perl, TMTOWTDI, so you could also write:
864 BadCode => '
866 if ( $ISGOOD(a()) && $ISGOOD(b()) ) {
867 $c() = $a() + $b();
868 } else {
869 $SETBAD(c());
870 }',
871 You can reduce code repition using the C "PDL_BAD_CODE" macro, using
873 the same code for both of the "Code" and "BadCode" sections:
874 #ifdef PDL_BAD_CODE
876 if ( $ISGOOD(a()) && $ISGOOD(b()) ) {
877 #endif PDL_BAD_CODE
878 $c() = $a() + $b();
880 #ifdef PDL_BAD_CODE
882 } else {
883 $SETBAD(c());
884 }
885 #endif PDL_BAD_CODE
886 If you want access to the value of the badflag for a given piddle, you
888 can use the PDL STATE macros:
889 $ISPDLSTATEBAD(pdl)
891 $ISPDLSTATEGOOD(pdl)
892 $SETPDLSTATEBAD(pdl)
893 $SETPDLSTATEGOOD(pdl)
894 TODO: mention the "FindBadStatusCode" and "CopyBadStatusCode" options
896 to "pp_def", as well as the "BadDoc" key.
897 Interfacing your own/library functions using PP
899 Now, consider the following: you have your own C function (that may in
900 fact be part of some library you want to interface to PDL) which takes
901 as arguments two pointers to vectors of double:
902 void myfunc(int n,double *v1,double *v2);
904 The correct way of defining the PDL function is
906 pp_def('myfunc',
908 Pars => 'a(n); [o]b(n);',
909 GenericTypes => ['D'],
910 Code => 'myfunc($SIZE(n),$P(a),$P(b));'
911 );
912 The "$P("par")" syntax returns a pointer to the first element and the
914 other elements are guaranteed to lie after that.
915 Notice that here it is possible to make many mistakes. First, $SIZE(n)
917 must be used instead of "n". Second, you shouldn't put any loops in
918 this code. Third, here we encounter a new hash key recognised by
919 PDL::PP : the "GenericTypes" declaration tells PDL::PP to ONLY GENERATE
920 THE TYPELOOP FOP THE LIST OF TYPES SPECIFIED. In this case "double".
921 This has two advantages. Firstly the size of the compiled code is
922 reduced vastly, secondly if non-double arguments are passed to
923 "myfunc()" PDL will automatically convert them to double before passing
924 to the external C routine and convert them back afterwards.
925 One can also use "Pars" to qualify the types of individual arguments.
927 Thus one could also write this as:
928 pp_def('myfunc',
930 Pars => 'double a(n); double [o]b(n);',
931 Code => 'myfunc($SIZE(n),$P(a),$P(b));'
932 );
933 The type specification in "Pars" exempts the argument from variation in
935 the typeloop - rather it is automatically converted too and from the
936 type specified. This is obviously useful in a more general example,
937 e.g.:
938 void myfunc(int n,float *v1,long *v2);
940 pp_def('myfunc',
942 Pars => 'float a(n); long [o]b(n);',
943 GenericTypes => ['F'],
944 Code => 'myfunc($SIZE(n),$P(a),$P(b));'
945 );
946 Note we still use "GenericTypes" to reduce the size of the type loop,
948 obviously PP could in principle spot this and do it automatically
949 though the code has yet to attain that level of sophistication!
950 Finally note when types are converted automatically one MUST use the
952 "[o]" qualifier for output variables or you hard one changes will get
953 optimised away by PP!
954 If you interface a large library you can automate the interfacing even
956 further. Perl can help you again(!) in doing this. In many libraries
957 you have certain calling conventions. This can be exploited. In short,
958 you can write a little parser (which is really not difficult in Perl)
959 that then generates the calls to "pp_def" from parsed descriptions of
960 the functions in that library. For an example, please check the Slatec
961 interface in the "Lib" tree of the PDL distribution. If you want to
962 check (during debugging) which calls to PP functions your Perl code
963 generated a little helper package comes in handy which replaces the PP
964 functions by identically named ones that dump their arguments to
965 stdout.
966 Just say
968 perl -MPDL::PP::Dump myfile.pd
970 to see the calls to "pp_def" and friends. Try it with ops.pd and
972 slatec.pd. If you're interested (or want to enhance it), the source is
973 in Basic/Gen/PP/Dump.pm
974 Other macros and functions in the Code section
976 Macros: So far we have encountered the $SIZE, $GENERIC and $P macros.
977 Now we are going to quickly explain the other macros that are expanded
978 in the "Code" section of PDL::PP along with examples of their usage.
979 $T The $T macro is used for type switches. This is very useful when you
981 have to use different external (e.g. library) functions depending on
982 the input type of arguments. The general syntax is
983 $Ttypeletters(type_alternatives)
985 where "typeletters" is a permutation of a subset of the letters
987 "BSULFD" which stand for Byte, Short, Ushort, etc. and
988 "type_alternatives" are the expansions when the type of the PP
989 operation is equal to that indicated by the respective letter. Let's
990 illustrate this incomprehensible description by an example. Assuming
991 you have two C functions with prototypes
992 void float_func(float *in, float *out);
994 void double_func(double *in, double *out);
995 which do basically the same thing but one accepts float and the
997 other double pointers. You could interface them to PDL by defining a
998 generic function "foofunc" (which will call the correct function
999 depending on the type of the transformation):
1000 pp_def('foofunc',
1002 Pars => ' a(n); [o] b();',
1003 Code => ' $TFD(float_func,double_func) ($P(a),$P(b));'
1004 GenericTypes => [qw(F D)],
1005 );
1006 Please note that you can't say
1008 Code => ' $TFD(float,double)_func ($P(a),$P(b));'
1010 since the $T macro expands with trailing spaces, analogously to C
1012 preprocessor macros. The slightly longer form illustrated above is
1013 correct. If you really want brevity, you can of course do
1014 '$TBSULFD('.(join ',',map {"long_identifier_name_$_"}
1016 qw/byt short unseigned lounge flotte dubble/).');'
1017 $PP
1019 The $PP macro is used for a so called physical pointer access. The
1020 physical refers to some internal optimisations of PDL (for those who
1021 are familiar with the PDL core we are talking about the vaffine
1022 optimisations). This macro is mainly for internal use and you
1023 shouldn't need to use it in any of your normal code.
1024 $COMP (and the "OtherPars" section)
1026 The $COMP macro is used to access non-pdl values in the code
1027 section. Its name is derived from the implementation of
1028 transformations in PDL. The variables you can refer to using $COMP
1029 are members of the ``compiled'' structure that represents the PDL
1030 transformation in question but does not yet contain any information
1031 about dimensions (for further details check PDL::Internals).
1032 However, you can treat $COMP just as a black box without knowing
1033 anything about the implementation of transformations in PDL. So when
1034 would you use this macro? Its main usage is to access values of
1035 arguments that are declared in the "OtherPars" section of a "pp_def"
1036 definition. But then you haven't heard about the "OtherPars" key
1037 yet?! Let's have another example that illustrates typical usage of
1038 both new features:
1039 pp_def('pnmout',
1041 Pars => 'a(m)',
1042 OtherPars => "char* fd",
1043 GenericTypes => [qw(B U S L)],
1044 Code => 'PerlIO *fp;
1045 IO *io;
1046 io = GvIO(gv_fetchpv($COMP(fd),FALSE,SVt_PVIO));
1048 if (!io || !(fp = IoIFP(io)))
1049 croak("Can\'t figure out FP");
1050 if (PerlIO_write(fp,$P(a),len) != len)
1052 croak("Error writing pnm file");
1053 ');
1054 This function is used to write data from a pdl to a file. The file
1056 descriptor is passed as a string into this function. This parameter
1057 does not go into the "Pars" section since it cannot be usefully
1058 treated like a pdl but rather into the aptly named "OtherPars"
1059 section. Parameters in the "OtherPars" section follow those in the
1060 "Pars" section when invoking the function, i.e.
1061 open FILE,">out.dat" or die "couldn't open out.dat";
1063 pnmout($pdl,'FILE');
1064 When you want to access this parameter inside the code section you
1066 have to tell PP by using the $COMP macro, i.e. you write "$COMP(fd)"
1067 as in the example. Otherwise PP wouldn't know that the "fd" you are
1068 referring to is the same as that specified in the "OtherPars"
1069 section.
1070 Another use for the "OtherPars" section is to set a named dimension
1072 in the signature. Let's have an example how that is done:
1073 pp_def('setdim',
1075 Pars => '[o] a(n)',
1076 OtherPars => 'int ns => n',
1077 Code => 'loop(n) %{ $a() = n; %}',
1078 );
1079 This says that the named dimension "n" will be initialised from the
1081 value of the other parameter "ns" which is of integer type (I guess
1082 you have realised that we use the "CType From => named_dim" syntax).
1083 Now you can call this function in the usual way:
1084 setdim(($a=null),5);
1086 print $a;
1087 [ 0 1 2 3 4 ]
1088 Admittedly this function is not very useful but it demonstrates how
1090 it works. If you call the function with an existing pdl and you
1091 don't need to explicitly specify the size of "n" since PDL::PP can
1092 figure it out from the dimensions of the non-null pdl. In that case
1093 you just give the dimension parameter as "-1":
1094 $a = hist($b);
1096 setdim($a,-1);
1097 That should do it.
1099 The only PP function that we have used in the examples so far is
1101 "loop". Additionally, there are currently two other functions which
1102 are recognised in the "Code" section:
1103 threadloop
1105 As we heard above the signature of a PP defined function defines the
1106 dimensions of all the pdl arguments involved in a primitive
1107 operation. However, you often call the functions that you defined
1108 with PP with pdls that have more dimensions than those specified in
1109 the signature. In this case the primitive operation is performed on
1110 all subslices of appropriate dimensionality in what is called a
1111 thread loop (see also overview above and PDL::Indexing). Assuming you
1112 have some notion of this concept you will probably appreciate that
1113 the operation specified in the code section should be optimised since
1114 this is the tightest loop inside a thread loop. However, if you
1115 revisit the example where we define the "pnmout" function, you will
1116 quickly realise that looking up the "IO" file descriptor in the inner
1117 thread loop is not very efficient when writing a pdl with many rows.
1118 A better approach would be to look up the "IO" descriptor once
1119 outside the thread loop and use its value then inside the tightest
1120 thread loop. This is exactly where the "threadloop" function comes in
1121 handy. Here is an improved definition of "pnmout" which uses this
1122 function:
1123 pp_def('pnmout',
1125 Pars => 'a(m)',
1126 OtherPars => "char* fd",
1127 GenericTypes => [qw(B U S L)],
1128 Code => 'PerlIO *fp;
1129 IO *io;
1130 int len;
1131 io = GvIO(gv_fetchpv($COMP(fd),FALSE,SVt_PVIO));
1133 if (!io || !(fp = IoIFP(io)))
1134 croak("Can\'t figure out FP");
1135 len = $SIZE(m) * sizeof($GENERIC());
1137 threadloop %{
1139 if (PerlIO_write(fp,$P(a),len) != len)
1140 croak("Error writing pnm file");
1141 %}
1142 ');
1143 This works as follows. Normally the C code you write inside the
1145 "Code" section is placed inside a thread loop (i.e. PP generates the
1146 appropriate wrapping XS code around it). However, when you explicitly
1147 use the "threadloop" function, PDL::PP recognises this and doesn't
1148 wrap your code with an additional thread loop. This has the effect
1149 that code you write outside the thread loop is only executed once per
1150 transformation and just the code with in the surrounding "%{ ... %}"
1151 pair is placed within the tightest thread loop. This also comes in
1152 handy when you want to perform a decision (or any other code,
1153 especially CPU intensive code) only once per thread, i.e.
1154 pp_addhdr('
1156 #define RAW 0
1157 #define ASCII 1
1158 ');
1159 pp_def('do_raworascii',
1160 Pars => 'a(); b(); [o]c()',
1161 OtherPars => 'int mode',
1162 Code => ' switch ($COMP(mode)) {
1163 case RAW:
1164 threadloop %{
1165 /* do raw stuff */
1166 %}
1167 break;
1168 case ASCII:
1169 threadloop %{
1170 /* do ASCII stuff */
1171 %}
1172 break;
1173 default:
1174 croak("unknown mode");
1175 }'
1176 );
1177 types
1179 The types function works similar to the $T macro. However, with the
1180 "types" function the code in the following block (delimited by "%{"
1181 and "%}" as usual) is executed for all those cases in which the
1182 datatype of the operation is any of the types represented by the
1183 letters in the argument to "type", e.g.
1184 Code => '...
1186 types(BSUL) %{
1188 /* do integer type operation */
1189 %}
1190 types(FD) %{
1191 /* do floating point operation */
1192 %}
1193 ...'
1194 The RedoDimsCode Section
1196 The "RedoDimsCode" key is an optional key that is used to compute
1197 dimensions of piddles at runtime in case the standard rules for
1198 computing dimensions from the signature are not sufficient. The
1199 contents of the "RedoDimsCode" entry is interpreted in the same way
1200 that the Code section is interpreted-- i.e., PP macros are expanded and
1201 the result is interpreted as C code. The purpose of the code is to set
1202 the size of some dimensions that appear in the signature. Storage
1203 allocation and threadloops and so forth will be set up as if the
1204 computed dimension had appeared in the signature. In your code, you
1205 first compute the desired size of a named dimension in the signature
1206 according to your needs and then assign that value to it via the
1207 $SIZE() macro.
1208 As an example, consider the following situation. You are interfacing an
1210 external library routine that requires an temporary array for workspace
1211 to be passed as an argument. Two input data arrays that are passed are
1212 p(m) and x(n). The output data array is y(n). The routine requires a
1213 workspace array with a length of n+m*m, and you'd like the storage
1214 created automatically just like it would be for any piddle flagged with
1215 [t] or [o]. What you'd like is to say something like
1216 pp_def( "myexternalfunc",
1218 Pars => " p(m); x(n); [o] y; [t] work(n+m*m); ", ...
1219 but that won't work, because PP can't interpret expressions with
1221 arithmetic in the signature. Instead you write
1222 pp_def(
1224 "myexternalfunc",
1225 Pars => ' p(m); x(n); [o] y(); [t] work(wn); ',
1226 RedoDimsCode => '
1227 PDL_Indx im = $PDL(p)->dims[0];
1228 PDL_Indx in = $PDL(x)->dims[0];
1229 PDL_Indx min = in + im * im;
1230 PDL_Indx inw = $PDL(work)->dims[0];
1231 $SIZE(wn) = inw >= min ? inw : min;
1232 ',
1233 Code => '
1234 externalfunc( $P(p), $P(x), $SIZE(m), $SIZE(n), $P(work) );
1235 '
1236 );
1237 This code works as follows: The macro $PDL(p) expands to a pointer to
1239 the pdl struct for the piddle p. You don't want a pointer to the data
1240 ( ie $P ) in this case, because you want to access the methods for the
1241 piddle on the C level. You get the first dimension of each of the
1242 piddles and store them in integers. Then you compute the minimum length
1243 the work array can be. If the user sent a piddle "work" with sufficient
1244 storage, then leave it alone. If the user sent, say a null pdl, or no
1245 pdl at all, then the size of wn will be zero and you reset it to the
1246 minimum value. Before the code in the Code section is executed PP will
1247 create the proper storage for "work" if it does not exist. Note that
1248 you only took the first dimension of "p" and "x" because the user may
1249 have sent piddles with extra threading dimensions. Of course, the
1250 temporary piddle "work" (note the [t] flag) should not be given any
1251 thread dimensions anyway.
1252 You can also use "RedoDimsCode" to set the dimension of a piddle
1254 flagged with [o]. In this case you set the dimensions for the named
1255 dimension in the signature using $SIZE() as in the preceding example.
1256 However, because the piddle is flagged with [o] instead of [t],
1257 threading dimensions will be added if required just as if the size of
1258 the dimension were computed from the signature according to the usual
1259 rules. Here is an example from PDL::Math
1260 pp_def("polyroots",
1262 Pars => 'cr(n); ci(n); [o]rr(m); [o]ri(m);',
1263 RedoDimsCode => 'PDL_Indx sn = $PDL(cr)->dims[0]; $SIZE(m) = sn-1;',
1264 The input piddles are the real and imaginary parts of complex
1266 coefficients of a polynomial. The output piddles are real and imaginary
1267 parts of the roots. There are "n" roots to an "n"th order polynomial
1268 and such a polynomial has "n+1" coefficients (the zeoreth through the
1269 "n"th). In this example, threading will work correctly. That is, the
1270 first dimension of the output piddle with have its dimension adjusted,
1271 but other threading dimensions will be assigned just as if there were
1272 no "RedoDimsCode".
1273 Typemap handling in the "OtherPars" section
1275 The "OtherPars" section discussed above is very often absolutely
1276 crucial when you interface external libraries with PDL. However in many
1277 cases the external libraries either use derived types or pointers of
1278 various types.
1279 The standard way to handle this in Perl is to use a "typemap" file.
1281 This is discussed in some detail in perlxs in the standard Perl
1282 documentation. In PP the functionality is very similar, so you can
1283 create a "typemap" file in the directory where your PP file resides and
1284 when it is built it is automatically read in to figure out the
1285 appropriate translation between the C type and Perl's built-in type.
1286 That said, there are a couple of important differences from the general
1288 handling of types in XS. The first, and probably most important, is
1289 that at the moment pointers to types are not allowed in the "OtherPars"
1290 section. To get around this limitation you must use the "IV" type
1291 (thanks to Judd Taylor for pointing out that this is necessary for
1292 portability).
1293 It is probably best to illustrate this with a couple of code-snippets:
1295 For instance the "gsl_spline_init" function has the following C
1297 declaration:
1298 int gsl_spline_init(gsl_spline * spline,
1300 const double xa[], const double ya[], size_t size);
1301 Clearly the "xa" and "ya" arrays are candidates for being passed in as
1303 piddles and the "size" argument is just the length of these piddles so
1304 that can be handled by the "$SIZE()" macro in PP. The problem is the
1305 pointer to the "gsl_spline" type. The natural solution would be to
1306 write an "OtherPars" declaration of the form
1307 OtherPars => 'gsl_spline *spl'
1309 and write a short "typemap" file which handled this type. This does not
1311 work at present however! So what you have to do is to go around the
1312 problem slightly (and in some ways this is easier too!):
1313 The solution is to declare "spline" in the "OtherPars" section using an
1315 "Integer Value", "IV". This hides the nature of the variable from PP
1316 and you then need to (well to avoid compiler warnings at least!)
1317 perform a type cast when you use the variable in your code. Thus
1318 "OtherPars" should take the form:
1319 OtherPars => 'IV spl'
1321 and when you use it in the code you will write
1323 INT2PTR(gsl_spline *, $COMP(spl))
1325 where the Perl API macro "INT2PTR" has been used to handle the pointer
1327 cast to avoid compiler warnings and problems for machines with mixed
1328 32bit and 64bit Perl configurations. Putting this together as Andres
1329 Jordan has done (with the modification using "IV" by Judd Taylor) in
1330 the "gsl_interp.pd" in the distribution source you get:
1331 pp_def('init_meat',
1333 Pars => 'double x(n); double y(n);',
1334 OtherPars => 'IV spl',
1335 Code =>'
1336 gsl_spline_init,( INT2PTR(gsl_spline *, $COMP(spl)), $P(x),$P(y),$SIZE(n)));'
1337 );
1338 where I have removed a macro wrapper call, but that would obscure the
1340 discussion.
1341 The other minor difference as compared to the standard typemap handling
1343 in Perl, is that the user cannot specify non-standard typemap locations
1344 or typemap filenames using the "TYPEMAPS" option in MakeMaker... Thus
1345 you can only use a file called "typemap" and/or the "IV" trick above.
1346 Other useful PP keys in data operation definitions
1348 You have already heard about the "OtherPars" key. Currently, there are
1349 not many other keys for a data operation that will be useful in normal
1350 (whatever that is) PP programming. In fact, it would be interesting to
1351 hear about a case where you think you need more than what is provided
1352 at the moment. Please speak up on one of the PDL mailing lists. Most
1353 other keys recognised by "pp_def" are only really useful for what we
1354 call slice operations (see also above).
1355 One thing that is strongly being planned is variable number of
1357 arguments, which will be a little tricky.
1358 An incomplete list of the available keys:
1360 Inplace
1362 Setting this key marks the routine as working inplace - ie the
1363 input and output piddles are the same. An example is
1364 "$a->inplace->sqrt()" (or "sqrt(inplace($a))").
1365 Inplace => 1
1367 Use when the routine is a unary function, such as "sqrt".
1368 Inplace => ['a']
1370 If there are more than one input piddles, specify the name of
1371 the one that can be changed inplace using an array reference.
1372 Inplace => ['a','b']
1374 If there are more than one output piddle, specify the name of
1375 the input piddle and output piddle in a 2-element array
1376 reference. This probably isn't needed, but left in for
1377 completeness.
1378 If bad values are being used, care must be taken to ensure the
1380 propagation of the badflag when inplace is being used; consider
1381 this excerpt from Basic/Bad/bad.pd:
1382 pp_def('replacebad',HandleBad => 1,
1384 Pars => 'a(); [o]b();',
1385 OtherPars => 'double newval',
1386 Inplace => 1,
1387 CopyBadStatusCode =>
1388 '/* propagate badflag if inplace AND it has changed */
1389 if ( a == b && $ISPDLSTATEBAD(a) )
1390 PDL->propagate_badflag( b, 0 );
1391 /* always make sure the output is "good" */
1393 $SETPDLSTATEGOOD(b);
1394 ',
1395 ...
1396 Since this routine removes all bad values, then the output piddle
1398 had its bad flag cleared. If run inplace (so "a == b"), then we
1399 have to tell all the children of "a" that the bad flag has been
1400 cleared (to save time we make sure that we call
1401 "PDL->propagate_badgflag" only if the input piddle had its bad flag
1402 set).
1403 NOTE: one idea is that the documentation for the routine could be
1405 automatically flagged to indicate that it can be executed inplace,
1406 ie something similar to how "HandleBad" sets "BadDoc" if it's not
1407 supplied (it's not an ideal solution).
1408 Other PDL::PP functions to support concise package definition
1410 So far, we have described the "pp_def" and "pp_done" functions. PDL::PP
1411 exports a few other functions to aid you in writing concise PDL
1412 extension package definitions.
1413 pp_addhdr
1415 Often when you interface library functions as in the above example you
1417 have to include additional C include files. Since the XS file is
1418 generated by PP we need some means to make PP insert the appropriate
1419 include directives in the right place into the generated XS file. To
1420 this end there is the "pp_addhdr" function. This is also the function
1421 to use when you want to define some C functions for internal use by
1422 some of the XS functions (which are mostly functions defined by
1423 "pp_def"). By including these functions here you make sure that
1424 PDL::PP inserts your code before the point where the actual XS module
1425 section begins and will therefore be left untouched by xsubpp (cf.
1426 perlxs and perlxstut man pages).
1427 A typical call would be
1429 pp_addhdr('
1431 #include <unistd.h> /* we need defs of XXXX */
1432 #include "libprotos.h" /* prototypes of library functions */
1433 #include "mylocaldecs.h" /* Local decs */
1434 static void do_the real_work(PDL_Byte * in, PDL_Byte * out, int n)
1436 {
1437 /* do some calculations with the data */
1438 }
1439 ');
1440 This ensures that all the constants and prototypes you need will be
1442 properly included and that you can use the internal functions defined
1443 here in the "pp_def"s, e.g.:
1444 pp_def('barfoo',
1446 Pars => ' a(n); [o] b(n)',
1447 GenericTypes => ['B'],
1448 Code => ' PDL_Indx ns = $SIZE(n);
1449 do_the_real_work($P(a),$P(b),ns);
1450 ',
1451 );
1452 pp_addpm
1454 In many cases the actual PP code (meaning the arguments to "pp_def"
1456 calls) is only part of the package you are currently implementing.
1457 Often there is additional Perl code and XS code you would normally have
1458 written into the pm and XS files which are now automatically generated
1459 by PP. So how to get this stuff into those dynamically generated files?
1460 Fortunately, there are a couple of functions, generally called
1461 "pp_addXXX" that assist you in doing this.
1462 Let's assume you have additional Perl code that should go into the
1464 generated pm-file. This is easily achieved with the "pp_addpm" command:
1465 pp_addpm(<<'EOD');
1467 =head1 NAME
1469 PDL::Lib::Mylib -- a PDL interface to the Mylib library
1471 =head1 DESCRIPTION
1473 This package implements an interface to the Mylib package with full
1475 threading and indexing support (see L<PDL::Indexing>).
1476 =cut
1478 use PGPLOT;
1480 =head2 use_myfunc
1482 this function applies the myfunc operation to all the
1483 elements of the input pdl regardless of dimensions
1484 and returns the sum of the result
1485 =cut
1486 sub use_myfunc {
1488 my $pdl = shift;
1489 myfunc($pdl->clump(-1),($res=null));
1491 return $res->sum;
1493 }
1494 EOD
1496 pp_add_exported
1498 You have probably got the idea. In some cases you also want to export
1500 your additional functions. To avoid getting into trouble with PP which
1501 also messes around with the @EXPORT array you just tell PP to add your
1502 functions to the list of exported functions:
1503 pp_add_exported('use_myfunc gethynx');
1505 pp_add_isa
1507 The "pp_add_isa" command works like the the "pp_add_exported" function.
1509 The arguments to "pp_add_isa" are added the @ISA list, e.g.
1510 pp_add_isa(' Some::Other::Class ');
1512 pp_bless
1514 If your pp_def routines are to be used as object methods use "pp_bless"
1516 to specify the package (i.e. class) to which your pp_defed methods will
1517 be added. For example, "pp_bless('PDL::MyClass')". The default is "PDL"
1518 if this is omitted.
1519 pp_addxs
1521 Sometimes you want to add extra XS code of your own (that is generally
1523 not involved with any threading/indexing issues but supplies some other
1524 functionality you want to access from the Perl side) to the generated
1525 XS file, for example
1526 pp_addxs('','
1528 # Determine endianness of machine
1530 int
1532 isbigendian()
1533 CODE:
1534 unsigned short i;
1535 PDL_Byte *b;
1536 i = 42; b = (PDL_Byte*) (void*) &i;
1538 if (*b == 42)
1540 RETVAL = 0;
1541 else if (*(b+1) == 42)
1542 RETVAL = 1;
1543 else
1544 croak("Impossible - machine is neither big nor little endian!!\n");
1545 OUTPUT:
1546 RETVAL
1547 ');
1548 Especially "pp_add_exported" and "pp_addxs" should be used with care.
1550 PP uses PDL::Exporter, hence letting PP export your function means that
1551 they get added to the standard list of function exported by default
1552 (the list defined by the export tag ``:Func''). If you use "pp_addxs"
1553 you shouldn't try to do anything that involves threading or indexing
1554 directly. PP is much better at generating the appropriate code from
1555 your definitions.
1556 pp_add_boot
1558 Finally, you may want to add some code to the BOOT section of the XS
1560 file (if you don't know what that is check perlxs). This is easily done
1561 with the "pp_add_boot" command:
1562 pp_add_boot(<<EOB);
1564 descrip = mylib_initialize(KEEP_OPEN);
1565 if (descrip == NULL)
1567 croak("Can't initialize library");
1568 GlobalStruc->descrip = descrip;
1570 GlobalStruc->maxfiles = 200;
1571 EOB
1572 pp_export_nothing
1574 By default, PP.pm puts all subs defined using the pp_def function into
1576 the output .pm file's EXPORT list. This can create problems if you are
1577 creating a subclassed object where you don't want any methods exported.
1578 (i.e. the methods will only be called using the $object->method
1579 syntax).
1580 For these cases you can call pp_export_nothing() to clear out the
1582 export list. Example (At the end of the .pd file):
1583 pp_export_nothing();
1585 pp_done();
1586 pp_core_importList
1588 By default, PP.pm puts the 'use Core;' line into the output .pm file.
1590 This imports Core's exported names into the current namespace, which
1591 can create problems if you are over-riding one of Core's methods in the
1592 current file. You end up getting messages like "Warning: sub sumover
1593 redefined in file subclass.pm" when running the program.
1594 For these cases the pp_core_importList can be used to change what is
1596 imported from Core.pm. For example:
1597 pp_core_importList('()')
1599 This would result in
1601 use Core();
1603 being generated in the output .pm file. This would result in no names
1605 being imported from Core.pm. Similarly, calling
1606 pp_core_importList(' qw/ barf /')
1608 would result in
1610 use Core qw/ barf/;
1612 being generated in the output .pm file. This would result in just
1614 'barf' being imported from Core.pm.
1615 pp_setversion
1617 I am pretty sure that this allows you to simultaneously set the .pm and
1619 .xs files' versions, thus avoiding unnecessary version-skew between the
1620 two. To use this, simply have the following line at some point in your
1621 .pd file:
1622 pp_setversion('0.0.3');
1624 However, don't use this if you use Module::Build::PDL. See that
1626 module's documentation for details.
1627 pp_deprecate_module
1629 If a particular module is deemed obsolete, this function can be used to
1631 mark it as deprecated. This has the effect of emitting a warning when a
1632 user tries to "use" the module. The generated POD for this module also
1633 carries a deprecation notice. The replacement module can be passed as
1634 an argument like this:
1635 pp_deprecate_module( infavor => "PDL::NewNonDeprecatedModule" );
1637 Note that function affects only the runtime warning and the POD.
1639
1641 Let's say that you have a function in your module called PDL::foo that
1642 uses the PP function "bar_pp" to do the heavy lifting. But you don't
1643 want to advertise that "bar_pp" exists. To do this, you must move your
1644 PP function to the top of your module file, then call
1645 pp_export_nothing()
1647 to clear the "EXPORT" list. To ensure that no documentation (even the
1649 default PP docs) is generated, set
1650 Doc => undef
1652 and to prevent the function from being added to the symbol table, set
1654 PMFunc => ''
1656 in your pp_def declaration (see Image2D.pd for an example). This will
1658 effectively make your PP function "private." However, it is always
1659 accessible via PDL::bar_pp due to Perl's module design. But making it
1660 private will cause the user to go very far out of his or her way to use
1661 it, so he or she shoulders the consequences!
1662
1664 The slice operation section of this manual is provided using dataflow
1665 and lazy evaluation: when you need it, ask Tjl to write it. a delivery
1666 in a week from when I receive the email is 95% probable and two week
1667 delivery is 99% probable.
1668 And anyway, the slice operations require a much more intimate knowledge
1670 of PDL internals than the data operations. Furthermore, the complexity
1671 of the issues involved is considerably higher than that in the average
1672 data operation. If you would like to convince yourself of this fact
1673 take a look at the Basic/Slices/slices.pd file in the PDL distribution
1674 :-). Nevertheless, functions generated using the slice operations are
1675 at the heart of the index manipulation and dataflow capabilities of
1676 PDL.
1677 Also, there are a lot of dirty issues with virtual piddles and vaffines
1679 which we shall entirely skip here.
1680 Slices and bad values
1682 Slice operations need to be able to handle bad values (if support is
1683 compiled into PDL). The easiest thing to do is look at
1684 Basic/Slices/slices.pd to see how this works.
1685 Along with "BadCode", there are also the "BadBackCode" and
1687 "BadRedoDimsCode" keys for "pp_def". However, any "EquivCPOffsCode"
1688 should not need changing, since any changes are absorbed into the
1689 definition of the "$EQUIVCPOFFS()" macro (i.e. it is handled
1690 automatically by PDL::PP).
1691 A few notes on writing a slicing routine...
1693 The following few paragraphs describe writing of a new slicing routine
1694 ('range'); any errors are CED's. (--CED 26-Aug-2002)
1695
1697 For printing warning messages or aborting/dieing, you can call "warn"
1698 or "barf" from PP code. However, you should be aware that these calls
1699 have been redefined using C preprocessor macros to "PDL->barf" and
1700 "PDL->warn". These redefinitions are in place to keep you from
1701 inadvertently calling perl's "warn" or "barf" directly, which can cause
1702 segfaults during pthreading (i.e. processor multi-threading).
1703 PDL's own versions of "barf" and "warn" will queue-up warning or barf
1705 messages until after pthreading is completed, and then call the perl
1706 versions of these routines.
1707 See PDL::ParallelCPU for more information on pthreading.
1709
1711 The PDL "Core" structure, defined in Basic/Core/pdlcore.h.PL, contains
1712 pointers to a number of routines that may be useful to you. The
1713 majority of these routines deal with manipulating piddles, but some are
1714 more general:
1715 PDL->qsort_B( PDL_Byte *xx, PDL_Indx a, PDL_Indx b )
1717 Sort the array "xx" between the indices "a" and "b". There are
1718 also versions for the other PDL datatypes, with postfix "_S", "_U",
1719 "_L", "_N", "_Q", "_F", and "_D". Any module using this must
1720 ensure that "PDL::Ufunc" is loaded.
1721 PDL->qsort_ind_B( PDL_Byte *xx, PDL_Indx *ix, PDL_Indx a, PDL_Indx b )
1723 As for "PDL->qsort_B", but this time sorting the indices rather
1724 than the data.
1725 The routine "med2d" in Lib/Image2D/image2d.pd shows how such routines
1727 are used.
1728
1730 If you are going to generate a package from your PP file (typical file
1731 extensions are ".pd" or ".pp" for the files containing PP code) it is
1732 easiest and safest to leave generation of the appropriate commands to
1733 the Makefile. In the following we will outline the typical format of a
1734 Perl Makefile to automatically build and install your package from a
1735 description in a PP file. Most of the rules to build the xs, pm and
1736 other required files from the PP file are already predefined in the
1737 PDL::Core::Dev package. We just have to tell MakeMaker to use it.
1738 In most cases you can define your Makefile like
1740 # Makefile.PL for a package defined by PP code.
1742 use PDL::Core::Dev; # Pick up development utilities
1744 use ExtUtils::MakeMaker;
1745 $package = ["mylib.pd",Mylib,PDL::Lib::Mylib];
1747 %hash = pdlpp_stdargs($package);
1748 $hash{OBJECT} .= ' additional_Ccode$(OBJ_EXT) ';
1749 $hash{clean}->{FILES} .= ' todelete_Ccode$(OBJ_EXT) ';
1750 $hash{'VERSION_FROM'} = 'mylib.pd';
1751 WriteMakefile(%hash);
1752 sub MY::postamble { pdlpp_postamble($package); }
1754 Here, the list in $package is: first: PP source file name, then the
1756 prefix for the produced files and finally the whole package name. You
1757 can modify the hash in whatever way you like but it would be reasonable
1758 to stay within some limits so that your package will continue to work
1759 with later versions of PDL.
1760 If you don't want to use prepackaged arguments, here is a generic
1762 Makefile.PL that you can adapt for your own needs:
1763 # Makefile.PL for a package defined by PP code.
1765 use PDL::Core::Dev; # Pick up development utilities
1767 use ExtUtils::MakeMaker;
1768 WriteMakefile(
1770 'NAME' => 'PDL::Lib::Mylib',
1771 'VERSION_FROM' => 'mylib.pd',
1772 'TYPEMAPS' => [&PDL_TYPEMAP()],
1773 'OBJECT' => 'mylib$(OBJ_EXT) additional_Ccode$(OBJ_EXT)',
1774 'PM' => { 'Mylib.pm' => '$(INST_LIBDIR)/Mylib.pm'},
1775 'INC' => &PDL_INCLUDE(), # add include dirs as required by your lib
1776 'LIBS' => [''], # add link directives as necessary
1777 'clean' => {'FILES' =>
1778 'Mylib.pm Mylib.xs Mylib$(OBJ_EXT)
1779 additional_Ccode$(OBJ_EXT)'},
1780 );
1781 # Add genpp rule; this will invoke PDL::PP on our PP file
1783 # the argument is an array reference where the array has three string elements:
1784 # arg1: name of the source file that contains the PP code
1785 # arg2: basename of the xs and pm files to be generated
1786 # arg3: name of the package that is to be generated
1787 sub MY::postamble { pdlpp_postamble(["mylib.pd",Mylib,PDL::Lib::Mylib]); }
1788 To make life even easier PDL::Core::Dev defines the function
1790 "pdlpp_stdargs" that returns a hash with default values that can be
1791 passed (either directly or after appropriate modification) to a call to
1792 WriteMakefile. Currently, "pdlpp_stdargs" returns a hash where the
1793 keys are filled in as follows:
1794 (
1796 'NAME' => $mod,
1797 'TYPEMAPS' => [&PDL_TYPEMAP()],
1798 'OBJECT' => "$pref\$(OBJ_EXT)",
1799 PM => {"$pref.pm" => "\$(INST_LIBDIR)/$pref.pm"},
1800 MAN3PODS => {"$src" => "\$(INST_MAN3DIR)/$mod.\$(MAN3EXT)"},
1801 'INC' => &PDL_INCLUDE(),
1802 'LIBS' => [''],
1803 'clean' => {'FILES' => "$pref.xs $pref.pm $pref\$(OBJ_EXT)"},
1804 )
1805 Here, $src is the name of the source file with PP code, $pref the
1807 prefix for the generated .pm and .xs files and $mod the name of the
1808 extension module to generate.
1809
1811 The internals of the current version consist of a large table which
1812 gives the rules according to which things are translated and the subs
1813 which implement these rules.
1814 Later on, it would be good to make the table modifiable by the user so
1816 that different things may be tried.
1817 [Meta comment: here will hopefully be more in the future; currently,
1819 your best bet will be to read the source code :-( or ask on the list
1820 (try the latter first) ]
1821
1823 Unless otherwise specified, the arguments are strings. Keys marked with
1824 (bad) are only used if bad-value support is compiled into PDL.
1825 Pars
1827 define the signature of your function
1828 OtherPars
1830 arguments which are not pdls. Default: nothing. This is a semi-
1831 colon separated list of arguments, e.g., "OtherPars=>'int k; double
1832 value; char* fd'". See $COMP(x) and also the same entry in Appendix
1833 B.
1834 Code
1836 the actual code that implements the functionality; several PP
1837 macros and PP functions are recognised in the string value
1838 HandleBad (bad)
1840 If set to 1, the routine is assumed to support bad values and the
1841 code in the BadCode key is used if bad values are present; it also
1842 sets things up so that the "$ISBAD()" etc macros can be used. If
1843 set to 0, cause the routine to print a warning if any of the input
1844 piddles have their bad flag set.
1845 BadCode (bad)
1847 Give the code to be used if bad values may be present in the input
1848 piddles. Only used if "HandleBad => 1".
1849 GenericTypes
1851 An array reference. The array may contain any subset of the one-
1852 character strings `B', `S', `U', `L', `Q', `F' and `D', which
1853 specify which types your operation will accept. The meaning of each
1854 type is:
1855 B - signed byte (i.e. signed char)
1857 S - signed short (two-byte integer)
1858 U - unsigned short
1859 L - signed long (four-byte integer, int on 32 bit systems)
1860 N - signed integer for indexing piddle elements (platform & Perl-dependent size)
1861 Q - signed long long (eight byte integer)
1862 F - float
1863 D - double
1864 This is very useful (and important!) when interfacing an external
1866 library. Default: [qw/B S U L N Q F D/]
1867 Inplace
1869 Mark a function as being able to work inplace.
1870 Inplace => 1 if Pars => 'a(); [o]b();'
1872 Inplace => ['a'] if Pars => 'a(); b(); [o]c();'
1873 Inplace => ['a','b'] if Pars => 'a(); b(); [o]c(); [o]d();'
1874 If bad values are being used, care must be taken to ensure the
1876 propagation of the badflag when inplace is being used; for instance
1877 see the code for "replacebad" in Basic/Bad/bad.pd.
1878 Doc Used to specify a documentation string in Pod format. See PDL::Doc
1880 for information on PDL documentation conventions. Note: in the
1881 special case where the PP 'Doc' string is one line this is
1882 implicitly used for the quick reference AND the documentation!
1883 If the Doc field is omitted PP will generate default documentation
1885 (after all it knows about the Signature).
1886 If you really want the function NOT to be documented in any way at
1888 this point (e.g. for an internal routine, or because you are doing
1889 it elsewhere in the code) explicitly specify "Doc=>undef".
1890 BadDoc (bad)
1892 Contains the text returned by the "badinfo" command (in "perldl")
1893 or the "-b" switch to the "pdldoc" shell script. In many cases, you
1894 will not need to specify this, since the information can be
1895 automatically created by PDL::PP. However, as befits computer-
1896 generated text, it's rather stilted; it may be much better to do it
1897 yourself!
1898 NoPthread
1900 Optional flag to indicate the PDL function should not use processor
1901 threads (i.e. pthreads or POSIX threads) to split up work across
1902 multiple CPU cores. This option is typically set to 1 if the
1903 underlying PDL function is not threadsafe. If this option isn't
1904 present, then the function is assumed to be threadsafe. This option
1905 only applies if PDL has been compiled with POSIX threads enabled.
1906 PMCode
1908 PDL functions allow you to pass in a piddle into which you want the
1909 output saved. This is handy because you can allocate an output
1910 piddle once and reuse it many times; the alternative would be for
1911 PDL to create a new piddle each time, which may waste compute
1912 cycles or, more likely, RAM. This added flexibility comes at the
1913 cost of more complexity: PDL::PP has to write functions that are
1914 smart enough to count the arguments passed to it and create new
1915 piddles on the fly, but only if you want them.
1916 PDL::PP is smart enough to do that, but there are restrictions on
1918 argument order and the like. If you want a more flexible function,
1919 you can write your own Perl-side wrapper and specify it in the
1920 PMCode key. The string that you supply must (should) define a Perl
1921 function with a name that matches what you gave to pp_def in the
1922 first place. When you wish to eventually invoke the PP-generated
1923 function, you will need to supply all piddles in the exact order
1924 specified in the signature: output piddles are not optional, and
1925 the PP-generated function will not return anything. The obfuscated
1926 name that you will call is _<funcname>_int.
1927 I believe this documentation needs further clarification, but this
1929 will have to do. :-(
1930 PMFunc
1932 When pp_def generates functions, it typically defines them in the
1933 PDL package. Then, in the .pm file that it generates for your
1934 module, it typically adds a line that essentially copies that
1935 function into your current package's symbol table with code that
1936 looks like this:
1937 *func_name = \&PDL::func_name;
1939 It's a little bit smarter than that (it knows when to wrap that
1941 sort of thing in a BEGIN block, for example, and if you specified
1942 something different for pp_bless), but that's the gist of it. If
1943 you don't care to import the function into your current package's
1944 symbol table, you can specify
1945 PMFunc => '',
1947 PMFunc has no other side-effects, so you could use it to insert
1949 arbitrary Perl code into your module if you like. However, you
1950 should use pp_addpm if you want to add Perl code to your module.
1951
1953 Macros
1954 Macros labeled by (bad) are only used if bad-value support is compiled
1955 into PDL.
1956 $variablename_from_sig()
1958 access a pdl (by its name) that was specified in the signature
1959 $COMP(x)
1961 access a value in the private data structure of this
1962 transformation (mainly used to use an argument that is specified
1963 in the "OtherPars" section)
1964 $SIZE(n)
1966 replaced at runtime by the actual size of a named dimension (as
1967 specified in the signature)
1968 $GENERIC()
1970 replaced by the C type that is equal to the runtime type of the
1971 operation
1972 $P(a) a pointer access to the PDL named "a" in the signature. Useful
1974 for interfacing to C functions
1975 $PP(a) a physical pointer access to pdl "a"; mainly for internal use
1977 $TXXX(Alternative,Alternative)
1979 expansion alternatives according to runtime type of operation,
1980 where XXX is some string that is matched by "/[BSULNQFD+]/".
1981 $PDL(a)
1983 return a pointer to the pdl data structure (pdl *) of piddle "a"
1984 $ISBAD(a()) (bad)
1986 returns true if the value stored in "a()" equals the bad value
1987 for this piddle. Requires "HandleBad" being set to 1.
1988 $ISGOOD(a()) (bad)
1990 returns true if the value stored in "a()" does not equal the bad
1991 value for this piddle. Requires "HandleBad" being set to 1.
1992 $SETBAD(a()) (bad)
1994 Sets "a()" to equal the bad value for this piddle. Requires
1995 "HandleBad" being set to 1.
1996 functions
1998 "loop(DIMS) %{ ... %}"
1999 loop over named dimensions; limits are generated automatically by PP
2000 "threadloop %{ ... %}"
2002 enclose following code in a thread loop
2003 "types(TYPES) %{ ... %}"
2005 execute following code if type of operation is any of "TYPES"
2006
2008 A number of functions are imported when you "use PDL::PP". These
2009 include functions that control the generated C or XS code, functions
2010 that control the generated Perl code, and functions that manipulate the
2011 packages and symbol tables into which the code is created.
2012 Generating C and XS Code
2014 PDL::PP's main purpose is to make it easy for you to wrap the threading
2015 engine around your own C code, but you can do some other things, too.
2016 pp_def
2018 Used to wrap the threading engine around your C code. Virtually all
2019 of this document discusses the use of pp_def.
2020 pp_done
2022 Indicates you are done with PDL::PP and that it should generate its
2023 .xs and .pm files based upon the other pp_* functions that you have
2024 called. This function takes no arguments.
2025 pp_addxs
2027 This lets you add XS code to your .xs file. This is useful if you
2028 want to create Perl-accessible functions that invoke C code but
2029 cannot or should not invoke the threading engine. XS is the
2030 standard means by which you wrap Perl-accessible C code. You can
2031 learn more at perlxs.
2032 pp_add_boot
2034 This function adds whatever string you pass to the XS BOOT section.
2035 The BOOT section is C code that gets called by Perl when your
2036 module is loaded and is useful for automatic initialization. You
2037 can learn more about XS and the BOOT section at perlxs.
2038 pp_addhdr
2040 Adds pure-C code to your XS file. XS files are structured such that
2041 pure C code must come before XS specifications. This allows you to
2042 specify such C code.
2043 pp_boundscheck
2045 PDL normally checks the bounds of your accesses before making them.
2046 You can turn that on or off at runtime by setting
2047 MyPackage::set_boundscheck. This function allows you to remove that
2048 runtime flexibility and never do bounds checking. It also returns
2049 the current boundschecking status if called without any argumens.
2050 NOTE: I have not found anything about bounds checking in other
2052 documentation. That needs to be addressed.
2053 Generating Perl Code
2055 Many functions imported when you use PDL::PP allow you to modify the
2056 contents of the generated .pm file. In addition to pp_def and pp_done,
2057 the role of these functions is primarily to add code to various parts
2058 of your generated .pm file.
2059 pp_addpm
2061 Adds Perl code to the generated .pm file. PDL::PP actually keeps
2062 track of three different sections of generated code: the Top, the
2063 Middle, and the Bottom. You can add Perl code to the Middle section
2064 using the one-argument form, where the argument is the Perl code
2065 you want to supply. In the two-argument form, the first argument is
2066 an anonymous hash with only one key that specifies where to put the
2067 second argument, which is the string that you want to add to the
2068 .pm file. The hash is one of these three:
2069 {At => 'Top'}
2071 {At => 'Middle'}
2072 {At => 'Bot'}
2073 For example:
2075 pp_addpm({At => 'Bot'}, <<POD);
2077 =head1 Some documentation
2079 I know I'm typing this in the middle of my file, but it'll go at
2081 the bottom.
2082 =cut
2084 POD
2086 Warning: If, in the middle of your .pd file, you put documentation
2088 meant for the bottom of your pod, you will thoroughly confuse CPAN.
2089 On the other hand, if in the middle of your .pd file, you add some
2090 Perl code destined for the bottom or top of your .pm file, you only
2091 have yourself to confuse. :-)
2092 pp_beginwrap
2094 Adds BEGIN-block wrapping. Certain declarations can be wrapped in
2095 BEGIN blocks, though the default behavior is to have no such
2096 wrapping.
2097 pp_addbegin
2099 Sets code to be added to the top of your .pm file, even above code
2100 that you specify with "pp_addpm({At => 'Top'}, ...)". Unlike
2101 pp_addpm, calling this overwrites whatever was there before.
2102 Generally, you probably shouldn't use it.
2103 Tracking Line Numbers
2105 When you get compile errors, either from your C-like code or your Perl
2106 code, it can help to make those errors back to the line numbers in the
2107 source file at which the error occurred.
2108 pp_line_numbers
2110 Takes a line number and a (usually long) string of code. The line
2111 number should indicate the line at which the quote begins. This is
2112 usually Perl's "__LINE__" literal, unless you are using heredocs,
2113 in which case it is "__LINE__ + 1". The returned string has #line
2114 directives interspersed to help the compiler report errors on the
2115 proper line.
2116 Modifying the Symbol Table and Export Behavior
2118 PDL::PP usually exports all functions generated using pp_def, and
2119 usually installs them into the PDL symbol table. However, you can
2120 modify this behavior with these functions.
2121 pp_bless
2123 Sets the package (symbol table) to which the XS code is added. The
2124 default is PDL, which is generally what you want. If you use the
2125 default blessing and you create a function myfunc, then you can do
2126 the following:
2127 $piddle->myfunc(<args>);
2129 PDL::myfunc($piddle, <args>);
2130 On the other hand, if you bless your functions into another
2132 package, you cannot invoke them as PDL methods, and must invoke
2133 them as:
2134 MyPackage::myfunc($piddle, <args>);
2136 Of course, you could always use the PMFunc key to add your function
2138 to the PDL symbol table, but why do that?
2139 pp_add_isa
2141 Adds to the list of modules from which your module inherits. The
2142 default list is
2143 qw(PDL::Exporter DynaLoader)
2145 pp_core_importlist
2147 At the top of your generated .pm file is a line that looks like
2148 this:
2149 use PDL::Core;
2151 You can modify that by specifying a string to pp_core_importlist.
2153 For example,
2154 pp_core_importlist('::Blarg');
2156 will result in
2158 use PDL::Core::Blarg;
2160 You can use this, for example, to add a list of symbols to import
2162 from PDL::Core. For example:
2163 pp_core_importlist(" ':Internal'");
2165 will lead to the following use statement:
2167 use PDL::Core ':Internal';
2169 pp_setversion
2171 Sets your module's version. The version must be consistent between
2172 the .xs and the .pm file, and is used to ensure that your Perl's
2173 libraries do not suffer from version skew.
2174 pp_add_exported
2176 Adds to the export list whatever names you give it. Functions
2177 created using pp_def are automatically added to the list. This
2178 function is useful if you define any Perl functions using pp_addpm
2179 or pp_addxs that you want exported as well.
2180 pp_export_nothing
2182 This resets the list of exported symbols to nothing. This is
2183 probably better called "pp_export_clear", since you can add
2184 exported symbols after calling "pp_export_nothing". When called
2185 just before calling pp_done, this ensures that your module does not
2186 export anything, for example, if you only want programmers to use
2187 your functions as methods.
2188
2190 PDL
2191 For the concepts of threading and slicing check PDL::Indexing.
2193 PDL::Internals
2195 PDL::BadValues for information on bad values
2197 perlxs, perlxstut
2199
2201 Almost everything having to do with "Slice operation". This includes
2202 much of the following (each entry is followed by a guess/description of
2203 where it is used or defined):
2204 MACROS
2206 $CDIM()
2207 $CHILD()
2209 PDL::PP::Rule::Substitute::Usual
2210 $CHILD_P()
2212 PDL::PP::Rule::Substitute::Usual
2213 $CHILD_PTR()
2215 PDL::PP::Rule::Substitute::Usual
2216 $COPYDIMS()
2218 $COPYINDS()
2220 $CROAK()
2222 PDL::PP::Rule::Substitute::dosubst_private()
2223 $DOCOMPDIMS()
2225 Used in slices.pd, defined where?
2226 $DOPRIVDIMS()
2228 Used in slices.pd, defined where?
2229 Code comes from PDL::PP::CType::get_malloc, which is called by
2230 PDL::PP::CType::get_copy, which is called by PDL::PP::CopyOtherPars,
2231 PDL::PP::NT2Copies__, and PDL::PP::make_incsize_copy. But none of
2232 those three at first glance seem to have anything to do with
2233 $DOPRIVDIMS
2234 $EQUIVCPOFFS()
2236 $EQUIVCPTRUNC()
2238 $PARENT()
2240 PDL::PP::Rule::Substitute::Usual
2241 $PARENT_P()
2243 PDL::PP::Rule::Substitute::Usual
2244 $PARENT_PTR()
2246 PDL::PP::Rule::Substitute::Usual
2247 $PDIM()
2249 $PRIV()
2251 PDL::PP::Rule::Substitute::dosubst_private()
2252 $RESIZE()
2254 $SETDELTATHREADIDS()
2256 PDL::PP::Rule::MakeComp
2257 $SETDIMS()
2259 PDL::PP::Rule::MakeComp
2260 $SETNDIMS()
2262 PDL::PP::Rule::MakeComp
2263 $SETREVERSIBLE()
2265 PDL::PP::Rule::Substitute::dosubst_private()
2266 Keys
2268 AffinePriv
2269 BackCode
2271 BadBackCode
2273 CallCopy
2275 Comp (related to $COMP()?)
2277 DefaultFlow
2279 EquivCDimExpr
2281 EquivCPOffsCode
2283 EquivDimCheck
2285 EquivPDimExpr
2287 FTypes (see comment in this POD's source file between NoPthread and
2289 PMCode.)
2290 GlobalNew
2292 Identity
2294 MakeComp
2296 NoPdlThread
2298 P2Child
2300 ParentInds
2302 Priv
2304 ReadDataFuncName
2306 RedoDims (related to RedoDimsCode ?)
2308 Reversible
2310 WriteBckDataFuncName
2312 XCHGOnly
2314
2316 Although PDL::PP is quite flexible and thoroughly used, there are
2317 surely bugs. First amongst them: this documentation needs a thorough
2318 revision.
2319
2321 Copyright(C) 1997 Tuomas J. Lukka (lukka@fas.harvard.edu), Karl
2322 Glaazebrook (kgb@aaocbn1.aao.GOV.AU) and Christian Soeller
2323 (c.soeller@auckland.ac.nz). All rights reserved. Documentation updates
2324 Copyright(C) 2011 David Mertens (dcmertens.perl@gmail.com). This
2325 documentation is licensed under the same terms as Perl itself.
2326perl v5.30.0 2019-09-05 PP(1)