1QUICKSTART(1) User Contributed Perl Documentation QUICKSTART(1)
2
6 PDL::QuickStart - Quick introduction to PDL features.
7
9 A brief summary of the main PDL features and how to use them.
10
12 Introduction
13 Perl is an extremely good and versatile scripting language, well suited
14 to beginners and allows rapid prototyping. However until recently it
15 did not support data structures which allowed it to do fast number
16 crunching.
17 However with the development of Perl v5, Perl acquired 'Objects'. To
19 put it simply users can define their own special data types, and write
20 custom routines to manipulate them either in low level languages (C and
21 Fortran) or in Perl itself.
22 This has been fully exploited by the PerlDL developers. The 'PDL'
24 module is a complete Object-Oriented extension to Perl (although you
25 don't have to know what an object is to use it) which allows large
26 N-dimensional data sets, such as large images, spectra, time series,
27 etc to be stored efficiently and manipulated en masse. For example
28 with the PDL module we can write the Perl code "$a = $b + $c", where $b
29 and $c are large datasets (e.g. 2048x2048 images), and get the result
30 in only a fraction of a second.
31 PDL variables (or 'piddles' as they have come to be known) support a
33 wide range of fundamental data types - arrays can be bytes, short
34 integers (signed or unsigned), long integers, floats or double
35 precision floats. And because of the Object-Oriented nature of PDL new
36 customised datatypes can be derived from them.
37 As well as the PDL modules, that can be used by normal Perl programs,
39 PerlDL comes with a command line Perl shell, called 'perldl', which
40 supports command line editing. In combination with the various PDL
41 graphics modules this allows data to be easily played with and
42 visualised.
43 Help
45 PDL contains extensive documentation, available both within the perldl
46 or pdl2 shells and from the command line, using the "pdldoc" program.
47 For further information try either of:
48 pdl> help help
50 $ pdldoc
51 HTML copies of the documentation should also be available. To find
53 their location, try the following:
54 pdl> foreach ( map{"$_/PDL/HtmlDocs"}@INC ) { p "$_\n" if -d $_ }
56 Perl Datatypes and how PDL extends them
58 The fundamental Perl data structures are scalar variables, e.g. $x,
59 which can hold numbers or strings, lists or arrays of scalars, e.g. @x,
60 and associative arrays/hashes of scalars, e.g. %x.
61 Perl v5 introduces to Perl data structures and objects. A simple scalar
63 variable $x now be a user-defined data type or full blown object (it
64 actually holds a reference (a smart "pointer") to this but that is not
65 relevant for ordinary use of perlDL)
66 The fundamental idea behind perlDL is to allow $x to hold a whole 1D
68 spectrum, or a 2D image, a 3D data cube, and so on up to large
69 N-dimensional data sets. These can be manipulated all at once, e.g.
70 "$a = $b + 2" does a vector operation on each value in the
71 spectrum/image/etc.
72 You may well ask: "Why not just store a spectrum as a simple Perl @x
74 style list with each pixel being a list item?" The two key answers to
75 this are memory and speed. Because we know our spectrum consists of
76 pure numbers we can compactly store them in a single block of memory
77 corresponding to a C style numeric array. This takes up a LOT less
78 memory than the equivalent Perl list. It is then easy to pass this
79 block of memory to a fast addition routine, or to any other C function
80 which deals with arrays. As a result perlDL is very fast --- for
81 example one can multiply a 2048*2048 image in exactly the same time as
82 it would take in C or FORTRAN (0.1 sec on my SPARC). A further
83 advantage of this is that for simple operations (e.g. "$x += 2") one
84 can manipulate the whole array without caring about its dimensionality.
85 I find when using perlDL it is most useful to think of standard Perl @x
87 variables as "lists" of generic "things" and PDL variables like $x as
88 "arrays" which can be contained in lists or hashes. Quite often in my
89 perlDL scripts I have @x contain a list of spectra, or a list of images
90 (or even a mix!). Or perhaps one could have a hash (e.g. %x) of
91 images... the only limit is memory!
92 perlDL variables support a range of data types - arrays can be bytes,
94 short integers (signed or unsigned), long integers, floats or double
95 precision floats.
96 Usage
98 PerlDL is loaded into your Perl script using this command:
99 use PDL; # in Perl scripts: use the standard perlDL modules
101 There are also a lot of extension modules, e.g. PDL::Graphics::TriD.
103 Most of these (but not all as sometimes it is not appropriate) follow a
104 standard convention. If you say:
105 use PDL::Graphics::TriD;
107 You import everything in a standard list from the module. Sometimes you
109 might want to import nothing (e.g. if you want to use OO syntax all the
110 time and save the import tax). For these you say:
111 use PDL::Graphics::TriD qw();
113 And the empty "qw()" quotes are recognised as meaning 'nothing'. You
115 can also specify a list of functions to import in the normal Perl way.
116 There is also an interactive shell, "perldl" or "pdl2", see perldl or
118 pdl2 for details.
119 To create a new PDL variable
121 Here are some ways of creating a PDL variable:
122 $a = pdl [1..10]; # 1D array
124 $a = pdl (1,2,3,4); # Ditto
125 $a = pdl '[1 2 3 4]'; # Ditto
126 $b = pdl [[1,2,3],[4,5,6]]; # 2D 3x2 array
127 $b = pdl '[1 2 3; 4 5 6]'; # Ditto
128 $b = pdl q[1,2,3; 4,5,6]; # Ditto
129 $b = pdl <<NEWPDL # Ditto
130 [1 2 3]
131 [4 5 6]
132 NEWPDL
133 $c = pdl q[1 -2]; # 2-element piddle containing 1 and -2
134 $c = pdl q[1 - 2]; # 2-element piddle containing 1 and -2
135 $b = pdl 42 # 0-dimensional scalar
136 $c = pdl $a; # Make a new copy
137 $d = byte [1..10]; # See "Type conversion"
139 $e = zeroes(3,2,4); # 3x2x4 zero-filled array
140 $c = rfits $file; # Read FITS file
142 @x = ( pdl(42), zeroes(3,2,4), rfits($file) ); # Is a LIST of PDL variables!
144 The pdl() function is used to initialise a PDL variable from a scalar,
146 list, list reference, another PDL variable, or a properly formatted
147 string.
148 In addition all PDL functions automatically convert normal Perl scalars
150 to PDL variables on-the-fly.
151 (also see "Type Conversion" and "Input/Output" sections below)
153 Arithmetic (and boolean expressions)
155 $a = $b + 2; $a++; $a = $b / $c; # Etc.
156 $c=sqrt($a); $d = log10($b+100); # Etc
158 $e = $a>42; # Vector conditional
160 $e = 42*($a>42) + $a*($a<=42); # Cap top
162 $b = $a->log10 unless any ($a <= 0); # avoid floating point error
164 $a = $a / ( max($a) - min($a) );
166 $f = where($a, $a > 10); # where returns a piddle of elements for
168 # which the condition is true
169 print $a; # $a in string context prints it in a N-dimensional format
171 (and other Perl operators/functions)
173 When using piddles in conditional expressions (i.e. "if", "unless" and
175 "while" constructs) only piddles with exactly one element are allowed,
176 e.g.
177 $a = pdl (1,0,0,1);
179 print "is set" if $a->index(2);
180 Note that the boolean operators return in general multi-element
182 piddles. Therefore, the following will raise an error
183 print "is ok" if $a > 3;
185 since "$a > 3" is a piddle with 4 elements. Rather use all or any to
187 test if all or any of the elements fulfill the condition:
188 print "some are > 3" if any $a>3;
190 print "can't take logarithm" unless all $a>0;
191 There are also many predefined functions, which are described on other
193 man pages. Check PDL::Index.
194 Matrix functions
196 'x' is hijacked as the matrix multiplication operator. e.g. "$c = $a x
197 $b";
198 perlDL is row-major not column major so this is actually "c(i,j) =
200 sum_k a(k,j) b(i,k)" - but when matrices are printed the results will
201 look right. Just remember the indices are reversed. e.g.:
202 $a = [ $b = [
204 [ 1 2 3 0] [1 1]
205 [ 1 -1 2 7] [0 2]
206 [ 1 0 0 1] [0 2]
207 ] [1 1]
208 ]
209 gives $c = [
211 [ 1 11]
212 [ 8 10]
213 [ 2 2]
214 ]
215 Note: transpose() does what it says and is a convenient way to turn row
217 vectors into column vectors.
218 How to write a simple function
220 sub dotproduct {
221 my ($a,$b) = @_;
222 return sum($a*$b) ;
223 }
224 1;
225 If put in file dotproduct.pdl would be autoloaded if you are using
227 PDL::AutoLoader (see below).
228 Of course, this function is already available as the inner function,
230 see PDL::Primitive.
231 Type Conversion
233 Default for pdl() is double. Conversions are:
234 $a = float($b);
236 $c = long($d); # "long" is generally a 4 byte int
237 $d = byte($a);
238 Also double(), short(), ushort(), indx().
240 NOTE: The indx() routine is a special integer type that
242 is the correct size for a PDL index value (dimension size,
243 index, or offest) which can be either a 32bit (long) or
244 64bit (longlong) quantity depending on whether the perl
245 is built with 32bit or 64bit support.
246 These routines also automatically convert Perl lists to allow the
248 convenient shorthand:
249 $a = byte [[1..10],[1..10]]; # Create 2D byte array
251 $a = float [1..1000]; # Create 1D float array
252 etc.
254 Printing
256 Automatically expands array in N-dimensional format:
257 print $a;
259 $b = "Answer is = $a ";
261 Sections
263 PDL has very powerful multidimensional slicing and sectioning
264 operators; see the PDL::Slices(3) man page for details; we'll describe
265 the most important one here.
266 PDL shows its Perl/C heritage in that arrays are zero-offset. Thus a
268 100x100 image has indices "0..99,0..99". (The convention is that the
269 center of pixel (0,0) is at coordinate (0.0,0.0). All PDL graphics
270 functions conform to this definition and hide away the unit offsets of,
271 for example, the PGPLOT FORTRAN library.
272 Following the usual convention coordinate (0,0) is displayed at the
274 bottom left when displaying an image. It appears at the top left when
275 using ""print $a"" etc.
276 Simple sectioning uses a syntax extension to Perl, PDL::NiceSlice, that
278 allows you to specify subranges via a null-method modifier to a PDL:
279 $b = $a->($x1:$x2,$y1:$y2,($z1)); # Take subsection
281 Here, $a is a 3-dimensional variable, and $b gets a planar cutout that
283 is defined by the limits $x1, $x2, $y1, $y2, at the location $z1. The
284 parenthesis around $z1 cause the trivial index to be omitted --
285 otherwise $b would be three-dimensional with a third dimension of order
286 1.
287 You can put PDL slices on either side of the element-wise assignment
289 operator ".=", like so:
290 # Set part of $bigimage to values from $smallimage
292 $bigimage->($xa:$xb,$ya:$yb) .= $smallimage;
293 Some other miscellany:
295 $c = nelem($a); # Number of pixels
297 $val = at($object, $x,$y,$z...) # Pixel value at position, as a Perl scalar
299 $val = $object->at($x,$y,$z...) # equivalent (method syntax OK)
300 $b = xvals($a); # Fill array with X-coord values (also yvals(), zvals(),
302 # axisvals($x,$axis) and rvals() for radial distance
303 # from centre).
304 Input/Output
306 The "PDL::IO" modules implement several useful IO format functions. It
307 would be too much to give examples of each, but you can find a nice
308 overview at PDL::IO. Here is a sample of some of the supported IO
309 formats in PDL.
310 PDL::IO::Misc
312 Ascii, FITS and FIGARO/NDF IO routines.
313 PDL::IO::FastRaw
315 Using the raw data types of your machine, an unportable but
316 blindingly fast IO format. Also supports memory mapping to
317 conserve memory as well as get more speed.
318 PDL::IO::FlexRaw
320 General raw data formats. Like FastRaw, only better.
321 PDL::IO::Browser
323 A Curses browser for arrays.
324 PDL::IO::Pnm
326 Portaple bitmap and pixmap support.
327 PDL::IO::Pic
329 Using the previous module and netpbm, makes it possible to
330 easily write GIF, jpeg and whatever with simple commands.
331 Graphics
333 The philosophy behind perlDL is to make it work with a variety of
334 existing graphics libraries since no single package will satisfy all
335 needs and all people and this allows one to work with packages one
336 already knows and likes. Obviously there will be some overlaps in
337 functionality and some lack of consistency and uniformity. However this
338 allows PDL to keep up with a rapidly developing field - the latest PDL
339 modules provide interfaces to OpenGL and VRML graphics!
340 PDL::Graphics::PGPLOT
342 PGPLOT provides a simple library for line graphics and image
343 display.
344 There is an easy interface to this in the internal module
346 PDL::Graphics::PGPLOT, which calls routines in the separately
347 available PGPLOT top-level module.
348 PDL::Graphics::PLplot
350 PLplot provides a simple library for creating graphics with
351 multiple output drivers, including a direct-to-piddle driver.
352 This module provides both high-level and low-level functionality
354 built on PLplot. The low-level commands are pretty much direct
355 bindings to PLplot's C interface. Read more at
356 PDL::Graphics::PLplot.
357 PDL::Graphics::IIS
359 Many astronomers like to use SAOimage and Ximtool (or there
360 derivations/clones). These are useful free widgets for inspection
361 and visualisation of images. (They are not provided with perlDL but
362 can easily be obtained from their official sites off the Net.)
363 The PDL::Graphics::IIS package provides allows one to display
365 images in these ("IIS" is the name of an ancient item of image
366 display hardware whose protocols these tools conform to.)
367 PDL::Graphics::TriD
369 See PDL::Graphics::TriD, this is a collection of 3D routines for
370 OpenGL and (soon) VRML and other 3D formats which allow 3D point,
371 line, and surface plots from PDL.
372 Autoloading
374 See PDL::AutoLoader. This allows one to autoload functions on demand,
375 in a way perhaps familiar to users of MatLab.
376 One can also write PDL extensions as normal Perl modules.
378 PDL shells
380 The Perl script "pdl2" (or "perldl") provides a simple command line
381 interface to PDL. If the latest Readlines/ReadKey modules have been
382 installed "pdl2" detects this and enables command line recall and
383 editing. See the man page for details.
384 e.g.:
386 % perldl
388 perlDL shell v1.354
389 PDL comes with ABSOLUTELY NO WARRANTY. For details, see the file
390 'COPYING' in the PDL distribution. This is free software and you
391 are welcome to redistribute it under certain conditions, see
392 the same file for details.
393 ReadLines, NiceSlice, MultiLines enabled
394 Reading PDL/default.perldlrc...
395 Found docs database /home/pdl/dev/lib/perl5/site_perl/PDL/pdldoc.db
396 Type 'help' for online help
397 Type 'demo' for online demos
398 Loaded PDL v2.4.9_003 (supports bad values)
399 pdl> $x = rfits 'm51.fits'
400 Reading IMAGE data...
401 BITPIX = 32 size = 147456 pixels
402 Reading 589824 bytes
403 BSCALE = && BZERO =
404 pdl> use PDL::Graphics::PGPLOT;
406 pdl> imag $x
407 Displaying 384 x 384 image from 40 to 761, using 84 colors (16-99)...
408 You can also run it from the Perl debugger ("perl -MPDL -d -e 1") if
410 you want.
411 Miscellaneous shell features:
413 p The shell aliases "p" to be a convenient short form of "print",
415 e.g.
416 pdl> p ones 5,3
418 [
419 [1 1 1 1 1]
420 [1 1 1 1 1]
421 [1 1 1 1 1]
422 ]
423 Initialization
425 The files "~/.perldlrc" and "local.perldlrc" (in the current
426 directory) are sourced if found. This allows the user to have
427 global and local PDL code for startup.
428 Help
430 Type 'help'! One can search the PDL documentation, and look up
431 documentation on any function.
432 Escape
434 Any line starting with the "#" character is treated as a shell
435 escape. This character is configurable by setting the Perl variable
436 $PERLDL_ESCAPE. This could, for example, be set in "~/.perldlrc".
437 Overload operators
439 The following builtin Perl operators and functions have been overloaded
440 to work on PDL variables:
441 + - * / > < >= <= << >> & | ^ == != <=> ** % ! ~
443 sin log abs atan2 sqrt cos exp
444 [All the unary functions (sin etc.) may be used with inplace() - see
446 "Memory" below.]
447 Object-Orientation and perlDL
449 PDL operations are available as functions and methods. Thus one can
450 derive new types of object, to represent custom data classes.
451 By using overloading one can make mathematical operators do whatever
453 you please, and PDL has some built-in tricks which allow existing PDL
454 functions to work unchanged, even if the underlying data representation
455 is vastly changed! See PDL::Objects
456 Memory usage and references
458 Messing around with really huge data arrays may require some care.
459 perlDL provides many facilities to let you perform operations on big
460 arrays without generating extra copies though this does require a bit
461 more thought and care from the programmer.
462 NOTE: On some most systems it is better to configure Perl (during the
464 build options) to use the system "malloc()" function rather than Perl's
465 built-in one. This is because Perl's one is optimised for speed rather
466 than consumption of virtual memory - this can result in a factor of two
467 improvement in the amount of memory storage you can use. The Perl
468 malloc in 5.004 and later does have a number of compile-time options
469 you can use to tune the behaviour.
470 Simple arithmetic
472 If $a is a big image (e.g. occupying 10MB) then the command
473 $a = $a + 1;
475 eats up another 10MB of memory. This is because the expression
477 "$a+1" creates a temporary copy of $a to hold the result, then $a
478 is assigned a reference to that. After this, the original $a is
479 destroyed so there is no permanent memory waste. But on a small
480 machine, the growth in the memory footprint can be considerable.
481 It is obviously done this way so "$c=$a+1" works as expected.
482 Also if one says:
484 $b = $a; # $b and $a now point to same data
486 $a = $a + 1;
487 Then $b and $a end up being different, as one naively expects,
489 because a new reference is created and $a is assigned to it.
490 However if $a was a huge memory hog (e.g. a 3D volume) creating a
492 copy of it may not be a good thing. One can avoid this memory
493 overhead in the above example by saying:
494 $a++;
496 The operations "++,+=,--,-=", etc. all call a special "in-place"
498 version of the arithmetic subroutine. This means no more memory is
499 needed - the downside of this is that if "$b=$a" then $b is also
500 incremented. To force a copy explicitly:
501 $b = pdl $a; # Real copy
503 or, alternatively, perhaps better style:
505 $b = $a->copy;
507 Functions
509 Most functions, e.g. "log()", return a result which is a
510 transformation of their argument. This makes for good programming
511 practice. However many operations can be done "in-place" and this
512 may be required when large arrays are in use and memory is at a
513 premium. For these circumstances the operator inplace() is provided
514 which prevents the extra copy and allows the argument to be
515 modified. e.g.:
516 $x = log($array); # $array unaffected
518 log( inplace($bigarray) ); # $bigarray changed in situ
519 WARNINGS:
521 1. The usual caveats about duplicate references apply.
523 2. Obviously when used with some functions which can not be
525 applied in situ (e.g. "convolve()") unexpected effects may
526 occur! We try to indicate "inplace()"-safe functions in the
527 documentation.
528 3. Type conversions, such as"float()", may cause hidden copying.
530 Ensuring piddleness
532 If you have written a simple function and you don't want it to blow up
533 in your face if you pass it a simple number rather than a PDL variable.
534 Simply call the function topdl() first to make it safe. e.g.:
535 sub myfiddle { my $pdl = topdl(shift); $pdl->fiddle_foo(...); ... }
537 "topdl()" does NOT perform a copy if a pdl variable is passed - it just
539 falls through - which is obviously the desired behaviour. The routine
540 is not of course necessary in normal user defined functions which do
541 not care about internals.
542
544 Copyright (C) Karl Glazebrook (kgb@aaoepp.aao.gov.au), Tuomas J. Lukka,
545 (lukka@husc.harvard.edu) and Christian Soeller
546 (c.soeller@auckland.ac.nz) 1997. Commercial reproduction of this
547 documentation in a different format is forbidden.
548perl v5.28.1 2018-05-05 QUICKSTART(1)