1FAQ(1) User Contributed Perl Documentation FAQ(1)
2
6 PDL::FAQ - Frequently asked questions about PDL
7
9 Current FAQ version: 1.008
10
12 This is version 1.008 of the PDL FAQ, a collection of frequently asked
13 questions about PDL - the Perl Data Language.
14
16 Q: 1.1 Where to find this document
17 You can find the latest version of this document at
18 <http://pdl.perl.org/?docs=FAQ&title=Frequently%20Asked%20Questions> .
19 Q: 1.2 How to contribute to this document
21 This is a considerably reworked version of the PDL FAQ. As such many
22 errors might have crept in and many updates might not have made it in.
23 You are explicitly encouraged to let us know about questions which you
24 think should be answered in this document but currently aren't.
25 Similarly, if you think parts of this document are unclear, please tell
27 the FAQ maintainer about it. Where a specific answer is taken in full
28 from someones posting the authorship should be indicated, let the FAQ
29 maintainer know if it isn't. For more general information explicit
30 acknowledgment is not made in the text, but rather there is an
31 incomplete list of contributors at the end of this document. Please
32 contact the FAQ maintainer if you feel hard done by.
33 Send your comments, additions, suggestions or corrections to the PDL
35 mailing list at pdl-general@lists.sourceforge.net. See Q: 3.2 below
36 for instructions on how to join the mailing lists.
37
39 Q: 2.1 What is PDL ?
40 PDL stands for Perl Data Language . To say it with the words of Karl
41 Glazebrook, initiator of the PDL project:
42 The PDL concept is to give standard perl5 the ability
44 to COMPACTLY store and SPEEDILY manipulate the large
45 N-dimensional data sets which are the bread and butter
46 of scientific computing. e.g. $x=$y+$z can add two
47 2048x2048 images in only a fraction of a second.
48 It provides tons of useful functionality for scientific and numeric
50 analysis.
51 For readers familiar with other scientific data evaluation packages it
53 may be helpful to add that PDL is in many respects similar to IDL,
54 MATLAB and similar packages. However, it tries to improve on a number
55 of issues which were perceived (by the authors of PDL) as shortcomings
56 of those existing packages.
57 Q: 2.2 Who supports PDL? Who develops it?
59 PDL is supported by its users. General informal support for PDL is
60 provided through the PDL mailing list
61 (pdl-general@lists.sourceforge.net , see below).
62 As a Perl extension (see Q: 2.5 below) it is devoted to the idea of
64 free and open development put forth by the Perl community. PDL was and
65 is being actively developed by a loosely knit group of people around
66 the world who coordinate their activities through the PDL development
67 mailing list (pdl-devel@lists.sourceforge.net , see Q: 3.2 below). If
68 you would like to join in the ongoing efforts to improve PDL please
69 join this list.
70 Q: 2.3 Why yet another Data Language ?
72 There are actually several reasons and everyone should decide for
73 himself which are the most important ones:
74 · PDL is "free software". The authors of PDL think that this concept
76 has several advantages: everyone has access to the sources ->
77 better debugging, easily adaptable to your own needs, extensible
78 for your purposes, etc... In comparison with commercial packages
79 such as MATLAB and IDL this is of considerable importance for
80 workers who want to do some work at home and cannot afford the
81 considerable cost to buy commercial packages for personal use.
82 · PDL is based on a powerful and well designed scripting language:
84 Perl. In contrast to other scientific/numeric data analysis
85 languages it has been designed using the features of a proven
86 language instead of having grown into existence from scratch.
87 Defining the control structures while features were added during
88 development leads to languages that often appear clumsy and badly
89 planned for most existing packages with similar scope as PDL.
90 · Using Perl as the basis a PDL programmer has all the powerful
92 features of Perl at his hand, right from the start. This includes
93 regular expressions, associative arrays (hashes), well designed
94 interfaces to the operating system, network, etc. Experience has
95 shown that even in mainly numerically oriented programming it is
96 often extremely handy if you have easy access to powerful semi-
97 numerical or completely non-numerical functionality as well. For
98 example, you might want to offer the results of a complicated
99 computation as a server process to other processes on the network,
100 perhaps directly accepting input from other processes on the
101 network. Using Perl and existing Perl extension packages things
102 like this are no problem at all (and it all will fit into your "PDL
103 script").
104 · Extremely easy extensibility and interoperability as PDL is a Perl
106 extension; development support for Perl extensions is an integral
107 part of Perl and there are already numerous extensions to standard
108 Perl freely available on the network.
109 · Integral language features of Perl (regular expressions, hashes,
111 object modules) immensely facilitated development and
112 implementation of key concepts of PDL. One of the most striking
113 examples for this point is probably PDL::PP (see Q: 6.16 below), a
114 code generator/parser/pre-processor that generates PDL functions
115 from concise descriptions.
116 · None of the existing data languages follow the Perl language rules,
118 which the authors firmly believe in:
119 · TIMTOWTDI: There is more than one way to do it. Minimalist
121 languages are interesting for computer scientists, but for
122 users, a little bit of redundancy makes things wildly easier to
123 cope with and allows individual programming styles - just as
124 people speak in different ways. For many people this will
125 undoubtedly be a reason to avoid PDL ;)
126 · Simple things are simple, complicated things possible: Things
128 that are often done should be easy to do in the language,
129 whereas seldom done things shouldn't be too cumbersome.
130 All existing languages violate at least one of these rules.
132 · As a project for the future PDL should be able to use super
134 computer features, e.g. vector capabilities/parallel processing,
135 GPGPU acceleration. This will probably be achieved by having
136 PDL::PP (see Q: 6.16 below) generate appropriate code on such
137 architectures to exploit these features.
138 · [ fill in your personal 111 favourite reasons here...]
140 Q: 2.4 What is PDL good for ?
142 Just in case you do not yet know what the main features of PDL are and
143 what one could do with them, here is a (necessarily selective) list of
144 key features:
145 PDL is well suited for matrix computations, general handling of
147 multidimensional data, image processing, general scientific
148 computation, numerical applications. It supports I/O for many popular
149 image and data formats, 1D (line plots), 2D (images) and 3D (volume
150 visualization, surface plots via OpenGL - for instance implemented
151 using Mesa or video card OpenGL drivers), graphics display capabilities
152 and implements many numerical and semi-numerical algorithms.
153 Through the powerful pre-processor it is also easy to interface Perl to
155 your favorite C routines, more of that further below.
156 Q: 2.5 What is the connection between PDL and Perl ?
158 PDL is a Perl5 extension package. As such it needs an existing Perl5
159 installation (see below) to run. Furthermore, much of PDL is written in
160 Perl (+ some core functionality that is written in C). PDL programs are
161 (syntactically) just Perl scripts that happen to use some of the
162 functionality implemented by the package "PDL".
163 Q: 2.6 What do I need to run PDL on my machine ?
165 Since PDL is just a Perl5 package you need first of all an installation
166 of Perl5 on your machine. As of this writing PDL requires version
167 5.10.x of perl, or higher. More information on where and how to get a
168 Perl installation can be found at the Perl home page
169 <http://www.perl.org> and at many CPAN sites (if you do not know what
170 CPAN is, check the answer to the next question).
171 To build PDL you also need a working C compiler, support for Xsubs, and
173 the package Extutils::MakeMaker. If you don't have a compiler there
174 might be a binary distribution available, see "Binary distributions"
175 below.
176 If you can (or cannot) get PDL working on a new (previously
178 unsupported) platform we would like to hear about it. Please, report
179 your success/failure to the PDL mailing list at
180 pdl-general@lists.sourceforge.net . We will do our best to assist you
181 in porting PDL to a new system.
182 Q: 2.7 Where do I get it?
184 PDL is available as source distribution in the Comprehensive Perl
185 Archive Network (or CPAN) and from the GitHub project page at
186 <https://github.com/PDLPorters/pdl>. The CPAN archives contains not
187 only the PDL distribution but also just about everything else that is
188 Perl-related. CPAN is mirrored by dozens of sites all over the world.
189 The main site is <http://www.cpan.org>, and local CPAN sites (mirrors)
190 can be found there. PDL's homepage is at <http://pdl.perl.org> and the
191 latest version can also be downloaded from there.
192 Q: 2.8 What do I have to pay to get PDL?
194 We are delighted to be able to give you the nicest possible answer on a
195 question like this: PDL is *free software* and all sources are publicly
196 available. But still, there are some copyrights to comply with. So
197 please, try to be as nice as we (the PDL authors) are and try to comply
198 with them.
199 Oh, before you think it is *completely* free: you have to invest some
201 time to pull the distribution from the net, compile and install it and
202 (maybe) read the manuals.
203
205 Q: 3.1 Where can I get information on PDL?
206 The complete PDL documentation is available with the PDL distribution.
207 Use the command "perldoc PDL" to start learning about PDL.
208 The easiest way by far, however, to get familiar with PDL is to use the
210 PDL on-line help facility from within the PDL shell, "pdl2" Just type
211 "pdl2" at your system prompt. Once you are inside the "pdl2" shell type
212 "help" . Using the "help" and "apropos" commands inside the shell you
213 should be able to find the way round the documentation.
214 Even better, you can immediately try your newly acquired knowledge
216 about PDL by issuing PDL/Perl commands directly at the command line. To
217 illustrate this process, here is the record of a typical "pdl2" session
218 of a PDL beginner (lengthy output is only symbolically reproduced in
219 braces ( <... ...> ) ):
220 unix> pdl2
222 pdl> help
223 < ... help output ... >
224 pdl> help PDL::QuickStart
225 < ... perldoc page ... >
226 pdl> $x = pdl (1,5,7.3,1.0)
227 pdl> $y = sequence float, 4, 4
228 pdl> help inner
229 < ... help on the 'inner' function ... >
230 pdl> $c = inner $x, $y
231 pdl> p $c
232 [22.6 79.8 137 194.2]
233 For further sources of information that are accessible through the
235 Internet see next question.
236 Q: 3.2 Are there other PDL information sources on the Internet?
238 First of all, for all purely Perl-related questions there are tons of
239 sources on the net. Good points to start are <http://www.perl.com> and
240 <http://www.perl.org> .
241 The PDL home site can be accessed by pointing your web browser to
243 <http://pdl.perl.org> . It has tons of goodies for anyone interested in
244 PDL:
245 · PDL distributions
247 · On-line documentation
249 · Pointers to an HTML archive of the PDL mailing lists
251 · A list of platforms on which PDL has been successfully tested.
253 · News about recently added features, ported libraries, etc.
255 · Name of the current pumpkin holders for the different PDL modules
257 (if you want to know what that means you better had a look at the
258 web pages).
259 If you are interested in PDL in general you can join the pdl-general
261 mailing list. This is a forum to discuss programming issues in PDL,
262 report bugs, seek assistance with PDL related problems, etc.
263 If you are interested in all the technical details of the ongoing PDL
265 development you can join the pdl-devel mailing list.
266 Subscription and current archive links to both mailing lists can be
268 found at <http://pdl.perl.org/?page=mailing-lists>.
269 Cross-posting between these lists should be avoided unless there is a
271 very good reason for doing that.
272 The PDL project, begun in the late 1990s, has undergone considerable
274 evolution since that time, and the support for it has as well. Thus
275 mailing-list archives are in several places. Originally pdl-general
276 was called 'perldl', and pdl-devel was called 'pdl-porters'.
277 |Time Period | URL |
279 |------------|-------------------------------------------------------|
280 |1996 - 2004 | http://www.xray.mpe.mpg.de/mailing-lists/perldl/ |
281 |1997 - 2004 | http://www.xray.mpe.mpg.de/mailing-lists/pdl-porters/ |
282 |2005 - 2015 | http://perldl.jach.hawaii.narkive.com/ |
283 |2005 - 2015 | http://pdl-porters.jach.hawaii.narkive.com/ |
284 |2015 - | https://sourceforge.net/p/pdl/mailman/pdl-general/ |
285 |2015 - | https://sourceforge.net/p/pdl/mailman/pdl-devel/ |
286 |--------------------------------------------------------------------|
287 Q: 3.3 What is the current version of PDL ?
289 As of this writing (FAQ version 1.008 of 21 May 2017) the latest stable
290 version is 2.018. The latest stable version should always be available
291 from a CPAN mirror site near you (see Question 2.7 for info on where to
292 get PDL).
293 The most current (possibly unstable) version of PDL can be obtained
295 from the Git repository, see Question 4.10 and periodic CPAN developers
296 releases of the Git code will be made for testing purposes and more
297 general availability.
298 Q: 3.4 How can PDL-2.2 be older than PDL-2.007?
300 Over its development, PDL has used both a single floating point version
301 number (from the versions 1.x through 2.005) at which point it switched
302 to a dotted triple version for 2.1.1 onward---EXCEPT for version 2.2
303 which came out which should have been 2.2.0. To simplify and unify
304 things, PDL has reverted to a single float version representation with
305 PDL-2.006. This can cause dependency problems for modules that set a
306 minimum PDL version of 2.2. The work around it, note that all extant
307 PDL releases have version numbers greater than 2.2.1 so that using 0 as
308 the minimum version will work.
309 Q: 3.5 I want to contribute to the further development of PDL. How can I
311 help?
312 Two ways that you could help almost immediately are (1) participate in
313 CPAN Testers for PDL and related modules, and (2) proofreading and
314 clarifying the PDL documentation so that it is most useable for PDL
315 users, especially new users.
316 To participate in CPAN Testers and contribute test reports, the page
318 <http://wiki.cpantesters.org/wiki/QuickStart> has instructions for
319 starting for either "CPAN" or "CPANPLUS" users.
320 If you have a certain project in mind you should check if somebody else
322 is already working on it or if you could benefit from existing modules.
323 Do so by posting your planned project to the PDL developers mailing
324 list at pdl-devel@lists.sourceforge.net . See the subscription
325 instructions in Question 3.2. We are always looking for people to
326 write code and/or documentation ;).
327 Q: 3.6 I think I have found a bug in the current version of PDL. What
329 shall I do?
330 First, make sure that the bug/problem you came across has not already
331 been dealt with somewhere else in this FAQ. Secondly, you can check
332 the searchable archive of the PDL mailing lists to find whether this
333 bug has already been discussed. If you still haven't found any
334 explanations you can post a bug report to
335 pdl-general@lists.sourceforge.net , or through the Bugs link on
336 <http://pdl.perl.org> . See the BUGS file in the PDL distribution for
337 what information to include. If you are unsure, discussions via the
338 perldl mailing list can be most helpful.
339
341 Q: 4.1 I have problems installing PDL. What shall I do?
342 First make sure you have read the file INSTALL in the distribution.
343 This contains a list of common problems which are unnecessary to repeat
344 here.
345 Next, check the file perldl.conf to see if by editing the configuration
347 options in that file you will be able to successfully build PDL. Some
348 of the modules need additional software installed, please refer to the
349 file DEPENDENCIES for further details. Make sure to edit the location
350 of these packages in perldl.conf if you have them in non-standard
351 locations.
352 N.B. Unix shell specific: If you would like to save an edited
354 perldl.conf for future builds just copy it as ~/.perldl.conf into your
355 home directory where it will be picked up automatically during the PDL
356 build process.
357 Also, check for another, pre-existing version of PDL on the build
359 system. Multiple PDL installs in the same PATH or @INC can cause
360 puzzling test or build failures.
361 If you still can't make it work properly please submit a bug report
363 including detailed information on the problems you encountered to the
364 perldl mailing list ( pdl-general@lists.sourceforge.net , see also
365 above). Response is often rapid.
366 Q: 4.2 Are there configuration files for PDL I have to edit?
368 Most users should not have to edit any configuration files manually.
369 However, in some cases you might have to supply some information about
370 awkwardly placed include files/libraries or you might want to
371 explicitly disable building some of the optional PDL modules. Check
372 the files INSTALL and perldl.conf for details.
373 If you had to manually edit perldl.conf and are happy with the results
375 you can keep the file handy for future reference. Place it in
376 ~/.perldl.conf where it will be picked up automatically or use "perl
377 Makefile.PL PDLCONF=your_file_name" next time you build PDL.
378 Q: 4.3 Do I need other software for successful operation?
380 For the basic PDL functionality you don't need any additional software.
381 However, some of the optional PDL modules included in the distribution
382 (notably most graphics and some I/O modules) require certain other
383 libraries/programs to be installed. Check the file DEPENDENCIES in the
384 distribution for details and directions on how to get these.
385 Q: 4.4 How can I install PDL in a non-standard location?
387 To install PDL in a non-standard location, use the INSTALL_BASE option
388 in the "perl Makefile.PL" configure step. For example, "perl
389 Makefile.PL INSTALL_BASE=/mydir/perl5" will configure PDL to install
390 into the tree rooted at "/mydir/perl5". For more details see "How do I
391 keep my own module/library directory?" in perlfaq8 and subsequent
392 sections. Another alternative is to use local::lib to do the heavy
393 listing for the needed configuration.
394 Q: 4.5 How can I force a completely clean installation?
396 To guarantee a completely clean installation of PDL, you will need to
397 first delete the current installation files and folders. These will be
398 all directories named "PDL" in the Perl @INC path, files named
399 "*Pdlpp*" in any "Inline" directories, and the programs "pdl, pdldoc,
400 pdl2, perldl, and pptemplate". Then just build and install as usual.
401 This is much easier to keep track of if you always install "PDL" into a
402 non-standard location. See Q: 4.4 above.
403
405 Q: 4.5 What binary distributions are available?
406 Information about binary distributions of PDL can be found on
407 <http://pdl.perl.org> . At present there are binary distributions of
408 PDL for Linux (RedHat and Debian), FreeBSD, Mac OS X and Windows,
409 though they might not be the most recent version.
410 If someone is interested in providing binary distributions for other
412 architectures, that would be very welcome. Let us know on the
413 pdl-devel@lists.sourceforge.net mailing list. Also check your Linux
414 distribution's package manager as many now include PDL. PPMs for win32
415 versions (both 32bit and 64bit) are also available.
416 Q: 4.6 Does PDL run on Linux? (And what about packages?)
418 Yes, PDL does run on Linux and indeed much of the development has been
419 done under Linux. On <http://pdl.perl.org> you can find links to
420 packages for some of the major distributions. Also check your
421 distribution's package manager (yum, apt, urpmi, ...) as PDL is now
422 found by many of these.
423 Q: 4.7 Does PDL run under Windows?
425 PDL builds fine on Win32 using MinGW or Microsoft compilers. See the
426 win32/INSTALL file in the PDL source distribution for details. Other
427 compilers have not been tested--input is welcome. There is also a
428 distribution of PDL through ActiveState's ppm, though it might not
429 always be the latest version. PDL-2.018 builds out of the box on
430 Strawberry Perl and ActiveState Perl and there are distributions of
431 Strawberry Perl with bundled PDL (see
432 <http://strawberryperl.com/releases.html>).
433
435 Q: 4.8 Can I get PDL via CVS?
436 No. PDL development was conducted with a CVS repository from December
437 1999 to April 2009. In April 2009 the project switched to the Git
438 version control system (see <http://git-scm.com>).
439 Q: 4.9 How do I get PDL via Git?
441 Assume you have Git installed on your system and want to download the
442 project source code into the directory "PDL". To get read-only access
443 to the repository, you type at the command line
444 git clone git://github.com/PDLPorters/pdl
446 If you wish to submit changes to PDL, you should "fork" the repository
448 from <https://github.com/PDLPorters/pdl>, then clone your fork in the
449 normal fashion.
450 To become an official PDL developer, you will need to be added to the
452 GitHub "PDLPorters" organisation.
453 For official PDL developers, to get read/write access to the repository
455 type at the command line
456 git clone git://github.com/PDLPorters/pdl
458 They can still use their own fork; at least one active developer uses
460 that model rather than branches on the main repository.
461 Q: 4.10 I had a problem with the Git version, how do I check if someone
463 has submitted a patch?
464 The best way is to check <https://github.com/PDLPorters/pdl/pulls> to
465 see if somebody has submitted a pull request related to your problem.
466 In addition, if you are not subscribing to the mailing list, check the
468 archive of the "pdl-devel" and "pdl-general" mailing lists. See
469 Question 3.2 for details.
470 Q: 4.11 I have gotten developer access to Git, how do I upload my
472 changes?
473 The first thing you should do is to read the Git documentation and
474 learn the basics about Git. There are many sources available online.
475 It is very important that you use Git "best practice", with branches,
476 but fortunately this is very easy! Here are the basics.
477 Make sure your copy is up to date with the main repo:
479 git checkout master
481 git pull --rebase # rebase in case you wrongly changed your own master
482 Make a branch:
484 git checkout -b mybranch-name
486 Commit your changes locally:
488 git add <file1> <file2> ...
490 git commit
491 or combine these two with:
493 git commit -a
495 Test the PDL before you push it to the main repository. If the code is
497 broken for you, then it is most likely broken for others. Luckily, the
498 rest of this process will test that automatically to help you catch
499 such errors.
500 Then update the shared repository with your changes:
502 git push -u origin mybranch-name
504 This will still leave your changes on a branch, but this is good. Now
506 go to the GitHub page, <https://github.com/PDLPorters/pdl>. It will ask
507 you whether you want to make a "pull request" - you do. Follow the
508 prompts. This will then initiate the automated "continuous integration"
509 tests, on Linux and Windows, with various versions of Perl, with
510 various compilers. You will also want to get at least one other
511 developer to review your changes.
512 Once this review process is successfully completed, you can merge your
514 changes to the master branch!
515
517 Q: 5.1 What is threading (is PDL a newsreader) ?
518 Unfortunately, in the context of PDL the term threading can have two
519 different (but related) meanings:
520 · When mentioned in the INSTALL directions and possibly during the
522 build process we have the usual computer science meaning of multi-
523 threading in mind (useful mainly on multiprocessor machines or
524 clusters)
525 · PDL threading of operations on piddles (as mentioned in the
527 indexing docs) is the iteration of a basic operation over
528 appropriate sub-slices of piddles, e.g. the inner product "inner
529 $x, $y" of a (3) pdl $x and a (3,5,4) pdl $y results in a (5,4)
530 piddle where each value is the result of an inner product of the
531 (3) pdl with a (3) sub-slice of the (3,5,4) piddle. For details
532 check PDL::Indexing
533 PDL threading leads naturally to potentially parallel code which can
535 make use of multi threading on multiprocessor machines/networks; there
536 you have the connection between the two types of use of the term.
537 Q: 5.2 What is a piddle?
539 Well, PDL scalar variables (which are instances of a particular class
540 of Perl objects, i.e. blessed thingies (see "perldoc perlobj" )) are in
541 common PDL parlance often called piddles (for example, check the
542 mailing list archives). Err, clear? If not, simply use the term
543 piddle when you refer to a PDL variable (an instance of a PDL object as
544 you might remember) regardless of what actual data the PDL variable
545 contains.
546
548 Q: 6.1 What is perldl? What is pdl2?
549 Sometimes "perldl" ("pdl2") is used as a synonym for PDL. Strictly
550 speaking, however, the name "perldl" ("pdl2") is reserved for the
551 little shell that comes with the PDL distribution and is supposed to be
552 used for the interactive prototyping of PDL scripts. For details check
553 perldl or pdl2.
554 Q: 6.2 How do I get on-line help for PDL?
556 Just type "help" (shortcut = "?") at the "pdl2" shell prompt and
557 proceed from there. Another useful command is the "apropos" (shortcut =
558 "??") command. Also try the "demo" command in the "perldl" or "pdl2"
559 shell if you are new to PDL.
560
562 Q: 6.3 I want to access the third element of a pdl but $x[2] doesn't
563 work ?!
564 See answer to the next question why the normal Perl array syntax
565 doesn't work for piddles.
566 Q: 6.4 The docs say piddles are some kind of array. But why doesn't the
568 Perl array syntax work with piddles then ?
569 OK, you are right in a way. The docs say that piddles can be thought of
570 arrays. More specifically, it says ( PDL::QuickStart ):
571 I find when using the Perl Data Language it is most useful
573 to think of standard Perl @x variables as "lists" of generic
574 "things" and PDL variables like $x as "arrays" which can be
575 contained in lists or hashes.
576 So, while piddles can be thought of as some kind of multi-dimensional
578 array they are not arrays in the Perl sense. Rather, from the point of
579 view of Perl they are some special class (which is currently
580 implemented as an opaque pointer to some stuff in memory) and therefore
581 need special functions (or 'methods' if you are using the OO version)
582 to access individual elements or a range of elements. The
583 functions/methods to check are "at" / "set" (see the section 'Sections'
584 in PDL::QuickStart ) or the powerful "slice" function and friends (see
585 PDL::Slices and PDL::Indexing and especially PDL::NiceSlice ).
586 Finally, to confuse you completely, you can have Perl arrays of
588 piddles, e.g. $spec[3] can refer to a pdl representing ,e.g, a
589 spectrum, where $spec[3] is the fourth element of the Perl list (or
590 array ;) @spec . This may be confusing but is very useful !
591 Q: 6.5 How do I concatenate piddles?
593 Most people will try to form new piddles from old piddles using some
594 variation over the theme: "$x = pdl([$y, 0, 2])". This does work, but
595 may not work in the way that a novice user would expect. (If $y has N
596 dimensions then $x will have N+1 dimensions.) Other ways to concatenate
597 piddles are to use the functions "cat", "append", and "glue". Similarly
598 you can split piddles using the command "dog".
599 Q: 6.6 Sometimes I am getting these strange results when using inplace
601 operations?
602 This question is related to the "inplace" function. From the
603 documentation (see PDL::QuickStart):
604 Most functions, e.g. log(), return a result which is a
606 transformation of their argument. This makes for good
607 programming practice. However many operations can be done
608 "in-place" and this may be required when large arrays are in
609 use and memory is at a premium. For these circumstances the
610 operator inplace() is provided which prevents the extra copy
611 and allows the argument to be modified. e.g.:
612 $x = log($array); # $array unaffected
614 log( inplace($bigarray) ); # $bigarray changed in situ
615 And also from the doc !!:
617 Obviously when used with some functions which can not be
619 applied in situ (e.g. convolve()) unexpected effects may
620 occur!
621 Q: 6.7 What is this strange usage of the string concatenation operator
623 ".=" in PDL scripts?
624 See next question on assignment in PDL.
625 Q: 6.8 Why are there two different kinds of assignment in PDL ?
627 This is caused by the fact that currently the assignment operator "="
628 allows only restricted overloading. For some purposes of PDL it turned
629 out to be necessary to have more control over the overloading of an
630 assignment operator. Therefore, PDL peruses the operator ".=" for
631 certain types of assignments.
632 Q: 6.9 How do I set a set of values in a piddle?
634 In Perl 5.6.7 and higher this assignment can be made using lvalue
635 subroutines:
636 pdl> $x = sequence(5); p $x
638 [0 1 2 3 4]
639 pdl> $x->slice('1:2') .= pdl([5,6])
640 pdl> p $x
641 [0 5 6 3 4]
642 see PDL::Lvalue for more info. PDL also supports a more matrix-like
644 slice syntax via the PDL::NiceSlice module:
645 pdl> $x(1:2) .= pdl([5,6])
647 pdl> p $x
648 [0 5 6 3 4]
649 With versions of Perl prior to 5.6.7 or when running under the perl
651 debugger this has to be done using a temporary variable:
652 pdl> $x = sequence(5); p $x
654 [0 1 2 3 4]
655 pdl> $tmp = $x->slice('1:2'); p $tmp;
656 [1 2]
657 pdl> $tmp .= pdl([5, 6]); # Note .= !!
658 pdl> p $x
659 [0 5 6 3 4]
660 This can also be made into one expression, which is often seen in PDL
662 code:
663 pdl> ($tmp = $x->slice('1:2')) .= pdl([5,6])
665 pdl> p $x
666 [0 5 6 3 4]
667 Q: 6.10 Can I use a piddle in a conditional expression?
669 Yes you can, but not in the way you probably tried first. It is not
670 possible to use a piddle directly in a conditional expression since
671 this is usually poorly defined. Instead PDL has two very useful
672 functions: "any" and "all" . Use these to test if any or all elements
673 in a piddle fulfills some criterion:
674 pdl> $x=pdl ( 1, -2, 3);
676 pdl> print '$x has at least one element < 0' if (any $x < 0);
677 $x has at least one element < 0
678 pdl> print '$x is not positive definite' unless (all $x > 0);
680 $x is not positive definite
681 Q: 6.11 Logical operators and piddles - '||' and '&&' don't work!
683 It is a common problem that you try to make a mask array or something
684 similar using a construct such as
685 $mask = which($piddle > 1 && $piddle < 2); # incorrect
687 This does not work! What you are looking for is the bitwise logical
689 operators '|' and '&' which work on an element-by-element basis. So it
690 is really very simple: Do not use logical operators on multi-element
691 piddles since that really doesn't make sense, instead write the example
692 as:
693 $mask = which($piddle > 1 & $piddle < 2);
695 which works correctly.
697
699 Q: 6.12 What is a null pdl ?
700 "null" is a special token for 'empty piddle'. A null pdl can be used to
701 flag to a PDL function that it should create an appropriately sized and
702 typed piddle. Null piddles can be used in places where a PDL function
703 expects an output or temporary argument. Output and temporary arguments
704 are flagged in the signature of a PDL function with the "[o]" and "[t]"
705 qualifiers (see next question if you don't know what the signature of a
706 PDL function is). For example, you can invoke the "sumover" function
707 as follows:
708 sumover $x, $y=null;
710 which is equivalent to
712 $y = sumover $x;
714 If this seems still a bit murky check PDL::Indexing and PDL::PP for
716 details about calling conventions, the signature and threading (see
717 also below).
718 Q: 6.13 What is the signature of a PDL function ?
720 The signature of a function is an important concept in PDL. Many (but
721 not all) PDL function have a signature which specifies the arguments
722 and their (minimal) dimensionality. As an example, look at the
723 signature of the "maximum" function:
724 'a(n); [o] b;'
726 this says that "maximum" takes two arguments, the first of which is (at
728 least) one-dimensional while the second one is zero-dimensional and an
729 output argument (flagged by the "[o]" qualifier). If the function is
730 called with piddles of higher dimension the function will be repeatedly
731 called with slices of these piddles of appropriate dimension(this is
732 called threading in PDL).
733 For details and further explanations consult PDL::Indexing and PDL::PP
735 .
736 Q: 6.14 How can I subclass (inherit from) piddles?
738 The short answer is: read PDL::Objects (e.g. type "help PDL::Objects"
739 in the perldl or pdl2 shell).
740 The longer answer (extracted from PDL::Objects ): Since a PDL object is
742 an opaque reference to a C struct, it is not possible to extend the PDL
743 class by e.g. extra data via sub-classing (as you could do with a hash
744 based Perl object). To circumvent this problem PDL has built-in
745 support to extend the PDL class via the has-a relation for blessed
746 hashes. You can get the HAS-A to behave like IS-A simply in that you
747 assign the PDL object to the attribute named "PDL" and redefine the
748 method initialize(). For example:
749 package FOO;
751 @FOO::ISA = qw(PDL);
753 sub initialize {
754 my $class = shift;
755 my $self = {
756 creation_time => time(), # necessary extension :-)
757 PDL => PDL->null, # used to store PDL object
758 };
759 bless $self, $class;
760 }
761 For another example check the script t/subclass.t in the PDL
763 distribution.
764 Q: 6.15 What on earth is this dataflow stuff ?
766 Dataflow is an experimental project that you don't need to concern
767 yourself with (it should not interfere with your usual programming).
768 However, if you want to know, have a look at PDL::Dataflow . There are
769 applications which will benefit from this feature (and it is already at
770 work behind the scenes).
771 Q: 6.16 What is PDL::PP?
773 Simple answer: PDL::PP is both a glue between external libraries and
774 PDL and a concise language for writing PDL functions.
775 Slightly longer answer: PDL::PP is used to compile very concise
777 definitions into XSUB routines implemented in C that can easily be
778 called from PDL and which automatically support threading, dataflow and
779 other things without you having to worry about it.
780 For further details check PDL::PP and the section below on Extensions
782 of PDL.
783 Q: 6.17 What happens when I have several references to the same PDL
785 object in different variables (cloning, etc?) ?
786 Piddles behave like Perl references in many respects. So when you say
787 $x = pdl [0,1,2,3];
789 $y = $x;
790 then both $y and $x point to the same object, e.g. then saying
792 $y++;
794 will *not* create a copy of the original piddle but just increment in
796 place, of which you can convince yourself by saying
797 print $x;
799 [1 2 3 4]
800 This should not be mistaken for dataflow which connects several
802 *different* objects so that data changes are propagated between the so
803 linked piddles (though, under certain circumstances, dataflown piddles
804 can share physically the same data).
805 It is important to keep the "reference nature" of piddles in mind when
807 passing piddles into subroutines. If you modify the input piddles you
808 modify the original argument, not a copy of it. This is different from
809 some other array processing languages but makes for very efficient
810 passing of piddles between subroutines. If you do not want to modify
811 the original argument but rather a copy of it just create a copy
812 explicitly (this example also demonstrates how to properly check for an
813 explicit request to process inplace, assuming your routine can work
814 inplace):
815 sub myfunc {
817 my $pdl = shift;
818 if ($pdl->is_inplace) {
819 $pdl->set_inplace(0)
820 } else {
821 # modify a copy by default
822 $pdl = $pdl->copy
823 }
824 $pdl->set(0,0);
825 return $pdl;
826 }
827
829 Q: 6.18 What I/O formats are supported by PDL ?
830 The current versions of PDL already support quite a number of different
831 I/O formats. However, it is not always obvious which module implements
832 which formats. To help you find the right module for the format you
833 require, here is a short list of the current list of I/O formats and a
834 hint in which module to find the implementation:
835 · A home brew fast raw (binary) I/O format for PDL is implemented by
837 the FastRaw module
838 · The FlexRaw module implements generic methods for the input and
840 output of `raw' data arrays. In particular, it is designed to read
841 output from FORTRAN 77 UNFORMATTED files and the low-level C
842 "write" function, even if the files are compressed or gzipped.
843 It is possible that the FastRaw functionality will be included in
845 the FlexRaw module at some time in the future.
846 · FITS I/O is implemented by the "wfits"/"rfits" functions in
848 PDL::IO::FITS .
849 · ASCII file I/O in various formats can be achieved by using the
851 "rcols" and "rgrep" functions, also in PDL::IO::Misc .
852 · PDL::IO::Pic implements an interface to the NetPBM/PBM+ filters to
854 read/write several popular image formats; also supported is output
855 of image sequences as MPEG movies, animated GIFs and a wide variety
856 of other video formats.
857 · On CPAN you can find the PDL::NetCDF module that works with PDL
859 2.007.
860 For further details consult the more detailed list in the PDL::IO
862 documentation or the documentation for the individual modules.
863 Q: 6.19 How can I stack a set of 2D arrays (images) into a 3D piddle?
865 Assuming all arrays are of the same size and in some format recognized
866 by "rpic" (see PDL::IO::Pic ) you could say:
867 use PDL::IO::Pic;
869 @names = qw/name1.tif .... nameN.tif/; # some file names
870 $dummy = PDL->rpic($names[0]);
871 $cube = PDL->zeroes($dummy->type,$dummy->dims,$#names+1); # make 3D piddle
872 for (0..$#names) {
873 # this is the slice assignment
874 ($tmp = $cube->slice(":,:,($_)")) .= PDL->rpic($names[$_]);
875 }
876 or
878 $cube(:,:,($_)) .= PDL->rpic($names[$_]);
880 for the slice assignment using the new PDL::NiceSlice syntax and Lvalue
882 assignments.
883 The for loop reads the actual images into a temporary 2D piddle whose
885 values are then assigned (using the overloaded ".=" operator) to the
886 appropriate slices of the 3D piddle $cube .
887 Q: 6.20 Where are test files for the graphics modules?
889 This answer applies mainly to PDL::Graphics::TriD (PDL's device
890 independent 3D graphics model) which is the trickiest one in this
891 respect. You find some test scripts in Demos/TriD in the distribution.
892 There are also 3dtest.pl and line3d.pl in the PDL/Example/TriD
893 directory. After you have built PDL you can do:
894 perl -Mblib Example/TriD/3dtest.pl
896 perl -Mblib Example/TriD/line3d.pl
898 to try the two TriD test programs. They only exercise one TriD
900 function each but their simplicity makes it easy to debug if needed
901 with the Perl debugger, see perldbug.
902 The programs in the Demo directory can be run most easily from the
904 "perldl" or "pdl2" interactive shell:
905 perl -Mblib perldl or perl -Mblib Perldl2/pdl2
907 followed by "demo 3d" or "demo 3d2" at the prompt. "demo" by itself
909 will give you a list of the available PDL demos.
910 You can run the test scripts in the Demos/TriD directory manually by
912 changing to that directory and running
913 perl -Mblib <testfile>
915 where "testfile" ; should match the pattern "test[3-9].p" and watch the
917 results. Some of the tests should bring up a window where you can
918 control (twiddle) the 3D objects with the mouse. Try using mouse button
919 1 for turning the objects in 3D space, mouse button 3 to zoom in and
920 out, and 'q' to advance to the next stage of the test.
921 Q: 6.21 What is TriD or PDL::TriD or PDL::Graphics::TriD?
923 Questions like this should be a thing of the past with the PDL on-line
924 help system in place. Just try (after installation):
925 un*x> pdl2
927 pdl> apropos trid
928 Check the output for promising hits and then try to look up some of
930 them, e.g.
931 pdl> help PDL::Graphics::TriD
933 Note that case matters with "help" but not with "apropos" .
935 Q: 6.22 PGPLOT does not write out PNG files.
937 There are a few sources of trouble with PGPLOT and PNG files. First,
938 when compiling the pgplot libraries, make sure you uncomment the PNG
939 entries in the drivers.list file. Then when running 'make' you probably
940 got an error like
941 C<make: *** No rule to make target `png.h', needed by `pndriv.o'. Stop.>
943 To fix this, find the line in the 'makefile' that starts with
945 'pndriv.o:' (it's near the bottom). Change, for example, ./png.h to
946 /usr/include/png.h, if that is where your header files are (you do have
947 the libpng and libz devel packages, don't you?). Do this for all four
948 entries on that line, then go back and run "make".
949 Second, if you already have the PGPLOT Perl module and PDL installed,
951 you probably tried to write out a PNG file and got fatal error message
952 like:
953 C<undefined symbol: png_create_write_struct>
955 This is because the PGPLOT Perl module does not automatically link
957 against the png and z libraries. So when you are installing the PGPLOT
958 Perl module (version 2.19) from CPAN, don't do "install PGPLOT", but
959 just do "get PGPLOT". Then exit from CPAN and manually install PGPLOT,
960 calling the makefile thusly:
961 C<perl Makefile.PL EXLIB=png,z EXDIR=/usr/lib>
963 assuming that there exist files such as /usr/lib/libpng.so.*,
965 /usr/lib/libz.so.*. Then do the standard "make;make test;make install;"
966 sequence. Now you can write png files from PDL!
967
969 Q: 7.1 I am looking for a package to do XXX in PDL. Where shall I look
970 for it?
971 The first stop is again "perldl" or "pdl2" and the on-line help or the
972 PDL documentation. There is already a lot of functionality in PDL which
973 you might not be aware of. The easiest way to look for functionality
974 is to use the "apropos" command:
975 pdl> apropos 'integral'
977 ceil Round to integral values in floating-point format
978 floor Round to integral values in floating-point format
979 intover Project via integral to N-1 dimensions
980 rint Round to integral values in floating-point format
981 Since the apropos command is no sophisticated search engine make sure
983 that you search on a couple of related topics and use short phrases.
984 However there is a good chance that what you need is not part of the
986 PDL distribution. You are then well advised to check out
987 <http://pdl.perl.org> where there is a list of packages using PDL. If
988 that does not solve your problem, ask on the mailing-list, if nothing
989 else you might get assistance which will let you interface your package
990 with PDL yourself, see also the next question.
991 Q: 7.2 Can I access my C/FORTRAN library routines in PDL?
993 Yes, you can, in fact it is very simple for many simple applications.
994 What you want is the PDL pre-processor PP (PDL::PP ). This will allow
995 you to make a simple interface to your C routine.
996 The two functions you need to learn (at least first) are "pp_def" which
998 defines the calling interface to the function, specifying input and
999 output parameters, and contains the code that links to the external
1000 library. The other command is "pp_end" which finishes the PP
1001 definitions. For details see the PDL::PP man-page, but we also have a
1002 worked example here.
1003 double eight_sum(int n)
1005 {
1006 int i;
1007 double sum, x;
1008 sum = 0.0; x=0.0;
1010 for (i=1; i<=n; i++) {
1011 x++;
1012 sum += x/((4.0*x*x-1.0)*(4.0*x*x-1.0));
1013 }
1014 return 1.0/sum;
1015 }
1016 We will here show you an example of how you interface C code with PDL.
1018 This is the first example and will show you how to approximate the
1019 number 8...
1020 The C code is shown above and is a simple function returning a double,
1022 and expecting an integer - the number of terms in the sum - as input.
1023 This function could be defined in a library or, as we do here, as an
1024 inline function.
1025 We will postpone the writing of the Makefile till later. First we will
1027 construct the ".pd" file. This is the file containing PDL::PP code. We
1028 call this "eight.pd" .
1029 #
1031 # pp_def defines a PDL function.
1032 #
1033 pp_addhdr (
1034 '
1035 double eight_sum(int n)
1036 {
1037 int i;
1038 double sum, x;
1039 sum = 0.0; x=0.0;
1041 for (i=1; i<=n; i++) {
1042 x++;
1043 sum += x/((4.0*x*x-1.0)*(4.0*x*x-1.0));
1044 }
1045 return 1.0/sum;
1046 }
1048 ');
1049 pp_def (
1051 'eight',
1052 Pars => 'int a(); double [o]b();',
1053 Code => '$b()=eight_sum($a());'
1054 );
1055 # Always make sure that you finish your PP declarations with
1057 # pp_done
1058 pp_done();
1060 A peculiarity with our example is that we have included the entire code
1062 with "pp_addhdr" instead of linking it in. This is only for the
1063 purposes of example, in a typical application you will use "pp_addhdr"
1064 to include header files. Note that the argument to "pp_addhdr" is
1065 enclosed in quotes.
1066 What is most important in this example is however the "pp_def" command.
1068 The first argument to this is the name of the new function eight ,
1069 then comes a hash which the real meat:
1070 · This gives the input parameters (here "a") and the output
1072 parameters (here "b"). The latter are indicated by the "[o]"
1073 specifier. Both arguments can have a type specification as shown
1074 here.
1075 Many variations and further flexibility in the interface can be
1077 specified. See "perldoc PDL::PP" for details.
1078 · This switch contains the code that should be executed. As you can
1080 see this is a rather peculiar mix of C and Perl, but essentially it
1081 is just as you would write it in C, but the variables that are
1082 passed from PDL are treated differently and have to be referred to
1083 with a preceding '$'.
1084 There are also simple macros to pass pointers to data and to obtain
1086 the values of other Perl quantities, see the manual page for
1087 further details.
1088 Finally note the call to "pp_done()" at the end of the file. This is
1090 necessary in all PP files.
1091 OK. So now we have a file with code that we dearly would like to use in
1093 Perl via PDL. To do this we need to compile the function, and to do
1094 that we need a Makefile.
1095 use PDL::Core::Dev;
1097 use ExtUtils::MakeMaker;
1098 PDL::Core::Dev->import();
1099 $package = ["eight.pd",Eight,PDL::Eight];
1101 %hash = pdlpp_stdargs($package);
1102 WriteMakefile( %hash );
1104 sub MY::postamble {pdlpp_postamble($package)};
1106 The code above should go in a file called Makefile.PL, which should
1108 subsequently be called in the standard Perl way: "perl Makefile.PL" .
1109 This should give you a Makefile and running "make" should compile the
1110 module for you and "make install" will install it for you.
1111 Q: 7.3 How can I interface package XXX in PDL?
1113 This question is closely related to the previous one, and as we said
1114 there, the PDL::PP pre-processor is the standard way of interfacing
1115 external packages with PDL. The most usual way to use PDL::PP is to
1116 write a short interface routine, see the PDL::PP perldoc page and the
1117 answer to the previous question for examples.
1118 However it is also possible to interface a package to PDL by re-writing
1120 your function in PDL::PP directly. This can be convenient in certain
1121 situations, in particular if you have a routine that expects a function
1122 as input and you would like to pass the function a Perl function for
1123 convenience.
1124 The PDL::PP perldoc page is the main source of information for writing
1126 PDL::PP extensions, but it is very useful to look for files in the
1127 distribution of PDL as many of the core functions are written in
1128 PDL::PP. Look for files that end in ".pd" which is the generally
1129 accepted suffix for PDL::PP files. But we also have a simple example
1130 here.
1131 The following example will show you how to write a simple function that
1133 automatically allows threading. To make this concise the example is of
1134 an almost trivial function, but the intention is to show the basics of
1135 writing a PDL::PP interface.
1136 We will write a simple function that calculates the minimum, maximum
1138 and average of a piddle. On my machine the resulting function is 8
1139 times faster than the built-in function "stats" (of course the latter
1140 also calculates the median).
1141 Let's jump straight in. Here is the code (from a file called
1143 "quickstats.pd" )
1144 #
1146 pp_def('quickstats',
1147 Pars => 'a(n); [o]avg(); [o]max(); [o]min()',
1148 Code => '$GENERIC(a) curmax, curmin;
1149 $GENERIC(a) tmp=0;
1150 loop(n) %{
1151 tmp += $a();
1152 if (!n || $a() > curmax) { curmax = $a();}
1153 if (!n || $a() < curmin) { curmin = $a();}
1154 %}
1155 $avg() = tmp/$SIZE(n);
1156 $max() = curmax;
1157 $min() = curmin;
1158 '
1159 );
1160 pp_done();
1162 The above might look like a confusing mixture of C and Perl, but behind
1164 the peculiar syntax lies a very powerful language. Let us take it line
1165 by line.
1166 The first line declares that we are starting the definition of a PDL:PP
1168 function called "quickstats" .
1169 The second line is very important as it specifies the input and output
1171 parameters of the function. a(n) tells us that there is one input
1172 parameter that we will refer to as "a" which is expected to be a vector
1173 of length n (likewise matrices, both square and rectangular would be
1174 written as "a(n,n)" and "a(n,m)" respectively). To indicate that
1175 something is an output parameter we put "[o]" in front of their names,
1176 so referring back to the code we see that avg, max and min are three
1177 output parameters, all of which are scalar (since they have no
1178 dimensional size indicated.
1179 The third line starts the code definition which is essentially pure C
1181 but with a couple of convenient functions. $GENERIC is a function that
1182 returns the C type of its argument - here the input parameter a. Thus
1183 the first two lines of the code section are variable declarations.
1184 The loop(n) construct is a convenience function that loops over the
1186 dimension called n in the parameter section. Inside this loop we
1187 calculate the cumulative sum of the input vector and keep track of the
1188 maximum and minimum values. Finally we assign the resulting values to
1189 the output parameters.
1190 Finally we finish our function declaration with "pp_done()" .
1192 To compile our new function we need to create a Makefile, which we will
1194 just list since its creation is discussed in an earlier question.
1195 use PDL::Core::Dev;
1197 use ExtUtils::MakeMaker;
1198 PDL::Core::Dev->import();
1199 $package = ["quickstats.pd",Quickstats,PDL::Quickstats];
1201 %hash = pdlpp_stdargs($package);
1202 WriteMakefile( %hash );
1204 sub MY::postamble {pdlpp_postamble($package)};
1206 An example Makefile.PL
1208 Our new statistic function should now compile using the tried and
1210 tested Perl way: "perl Makefile.PL; make" .
1211 You should experiment with this function, changing the calculations and
1213 input and output parameters. In conjunction with the PDL::PP perldoc
1214 page this should allow you to quickly write more advanced routines
1215 directly in PDL::PP.
1216
1218 If you find any inaccuracies in this document (or dis-functional URLs)
1219 please report to the perldl mailing list
1220 pdl-general@lists.sourceforge.net.
1221
1223 Achim Bohnet (ach@mpe.mpg.de ) for suggesting CoolHTML as a
1224 prettypodder (although we have switched to XML now) and various other
1225 improvements. Suggestions for some questions were taken from Perl FAQ
1226 and adapted for PDL.
1227
1229 Many people have contributed or given feedback on the current version
1230 of the FAQ, here is an incomplete list of individuals whose
1231 contributions or posts to the mailing-list have improved this FAQ at
1232 some point in time alphabetically listed by first name: Christian
1233 Soeller, Chris Marshall, Doug Burke, Doug Hunt, Frank Schmauder, Jarle
1234 Brinchmann, John Cerney, Karl Glazebrook, Kurt Starsinic, Thomas
1235 Yengst, Tuomas J. Lukka.
1236
1238 This document emerged from a joint effort of several PDL developers
1239 (Karl Glazebrook, Tuomas J. Lukka, Christian Soeller) to compile a list
1240 of the most frequently asked questions about PDL with answers.
1241 Permission is granted for verbatim copying (and formatting) of this
1242 material as part of PDL.
1243 Permission is explicitly not granted for distribution in book or any
1245 corresponding form. Ask on the PDL mailing list
1246 pdl-general@lists.sourceforge.net if some of the issues covered in here
1247 are unclear.
1248perl v5.30.2 2020-04-02 FAQ(1)