1IMPATIENT(1) User Contributed Perl Documentation IMPATIENT(1)
2
6 PDL::Impatient - PDL for the impatient (quick overview)
7
9 A brief summary of the main PDL features and how to use them.
10
12 Introduction
13 Perl is an extremely good and versatile scripting language, well suited
15 to beginners and allows rapid prototyping. However until recently it
16 did not support data structures which allowed it to do fast number
17 crunching.
18 However with the development of Perl v5, Perl acquired 'Objects'. To
20 put it simply users can define their own special data types, and write
21 custom routines to manipulate them either in low level languages (C and
22 Fortran) or in Perl itself.
23 This has been fully exploited by the PerlDL developers. The 'PDL' mod‐
25 ule is a complete Object-Oriented extension to Perl (although you don't
26 have to know what an object is to use it) which allows large N-dimen‐
27 sional data sets, such as large images, spectra, time series, etc to be
28 stored efficiently and manipulated en masse. For example with the
29 PDL module we can write the perl code "$a=$b+$c", where $b and $c are
30 large datasets (e.g. 2048x2048 images), and get the result in only a
31 fraction of a second.
32 PDL variables (or 'piddles' as they have come to be known) support a
34 wide range of fundamental data types - arrays can be bytes, short inte‐
35 gers (signed or unsigned), long integers, floats or double precision
36 floats. And because of the Object-Oriented nature of PDL new customised
37 datatypes can be derived from them.
38 As well as the PDL modules, that can be used by normal perl programs,
40 PerlDL comes with a command line perl shell, called 'perldl', which
41 supports command line editing. In combination with the various PDL
42 graphics modules this allows data to be easily played with and visu‐
43 alised.
44 Help
46 PDL contains extensive documentation, available both within the perldl
48 shell and from the command line, using the "pdldoc" program. For fur‐
49 ther information try either of:
50 perldl> help help
52 $ pdldoc
53 HTML copies of the documentation should also be available. To find
55 their location, try the following:
56 perldl> foreach ( map{"$_/PDL/HtmlDocs"}@INC ) { p "$_\n" if -d $_ }
58 Perl Datatypes and how PDL extends them
60 The fundamental perl data structures are scalar variables, e.g. $x,
62 which can hold numbers or strings, lists or arrays of scalars, e.g. @x,
63 and associative arrays/hashes of scalars, e.g. %x.
64 perl v5 introduces to perl data structures and objects. A simple scalar
66 variable $x now be a user-defined data type or full blown object (it
67 actually holds a reference (a smart "pointer") to this but that is not
68 relevant for ordinary use of perlDL)
69 The fundamental idea behind perlDL is to allow $x to hold a whole 1D
71 spectrum, or a 2D image, a 3D data cube, and so on up to large N-dimen‐
72 sional data sets. These can be manipulated all at once, e.g. "$a = $b
73 + 2" does a vector operation on each value in the spectrum/image/etc.
74 You may well ask: "Why not just store a spectrum as a simple perl @x
76 style list with each pixel being a list item?" The two key answers to
77 this are memory and speed. Because we know our spectrum consists of
78 pure numbers we can compactly store them in a single block of memory
79 corresponding to a C style numeric array. This takes up a LOT less mem‐
80 ory than the equivalent perl list. It is then easy to pass this block
81 of memory to a fast addition routine, or to any other C function which
82 deals with arrays. As a result perlDL is very fast --- for example one
83 can mulitiply a 2048*2048 image in exactly the same time as it would
84 take in C or FORTRAN (0.1 sec on my SPARC). A further advantage of this
85 is that for simple operations (e.g. "$x += 2") one can manipulate the
86 whole array without caring about its dimensionality.
87 I find when using perlDL it is most useful to think of standard perl @x
89 variables as "lists" of generic "things" and PDL variables like $x as
90 "arrays" which can be contained in lists or hashes. Quite often in my
91 perlDL scripts I have @x contain a list of spectra, or a list of images
92 (or even a mix!). Or perhaps one could have a hash (e.g. %x) of
93 images... the only limit is memory!
94 perlDL variables support a range of data types - arrays can be bytes,
96 short integers (signed or unsigned), long integers, floats or double
97 precision floats.
98 Usage
100 PerlDL is loaded into your perl script using this command:
102 use PDL; # in perl scripts: use the standard perlDL modules
104 There are also a lot of extension modules, e.g. PDL::Graphics::TriD.
106 Most of these (but not all as sometimes it is not appropriate) follow a
107 standard convention. If you say:
108 use PDL::Graphics::TriD;
110 You import everything in a standard list from the module. Sometimes you
112 might want to import nothing (e.g. if you want to use OO syntax all the
113 time and save the import tax). For these you say:
114 use PDL::Graphics::TriD '';
116 And the blank quotes '' are regonised as meaning 'nothing'. You can
118 also specify a list of functions to import in the normal Perl way.
119 There is also an interactive shell, "perldl", see perldl.
121 To create a new PDL variable
123 Here are some ways of creating a PDL variable:
125 $a = pdl [1..10]; # 1D array
127 $a = pdl (1,2,3,4); # Ditto
128 $b = pdl [[1,2,3],[4,5,6]]; # 2D 3x2 array
129 $b = pdl 42 # 0-dimensional scalar
130 $c = pdl $a; # Make a new copy
131 $d = byte [1..10]; # See "Type conversion"
133 $e = zeroes(3,2,4); # 3x2x4 zero-filled array
134 $c = rfits $file; # Read FITS file
136 @x = ( pdl(42), zeroes(3,2,4), rfits($file) ); # Is a LIST of PDL variables!
138 The pdl() function is used to initialise a PDL variable from a scalar,
140 list, list reference or another PDL variable.
141 In addition all PDL functions automatically convert normal perl scalars
143 to PDL variables on-the-fly.
144 (also see "Type Conversion" and "Input/Output" sections below)
146 Arithmetic (and boolean expressions)
148 $a = $b + 2; $a++; $a = $b / $c; # Etc.
150 $c=sqrt($a); $d = log10($b+100); # Etc
152 $e = $a>42; # Vector conditional
154 $e = 42*($a>42) + $a*($a<=42); # Cap top
156 $b = $a->log10 unless any ($a <= 0); # avoid floating point error
158 $a = $a / ( max($a) - min($a) );
160 $f = where($a, $a > 10); # where returns a piddle of elements for
162 # which the condition is true
163 print $a; # $a in string context prints it in a N-dimensional format
165 (and other perl operators/functions)
167 When using piddles in conditional expressions (i.e. "if", "unless" and
169 "while" constructs) only piddles with exactly one element are allowed,
170 e.g.
171 $a = pdl (1,0,0,1);
173 print "is set" if $a->index(2);
174 Note that the boolean operators return in general multielement piddles.
176 Therefore, the following will raise an error
177 print "is ok" if $a > 3;
179 since "$a > 3" is a piddle with 4 elements. Rather use all or any to
181 test if all or any of the elements fulfill the condition:
182 print "some are > 3" if any $a>3;
184 print "can't take logarithm" unless all $a>0;
185 There are also many predefined functions, which are described on other
187 manpages. Check PDL::Index.
188 Matrix functions
190 'x' is hijacked as the matrix multiplication operator. e.g. "$c = $a x
192 $b";
193 perlDL is row-major not column major so this is actually "c(i,j) =
195 sum_k a(k,j) b(i,k)" - but when matrices are printed the results will
196 look right. Just remember the indices are reversed. e.g.:
197 $a = [ $b = [
199 [ 1 2 3 0] [1 1]
200 [ 1 -1 2 7] [0 2]
201 [ 1 0 0 1] [0 2]
202 ] [1 1]
203 ]
204 gives $c = [
206 [ 1 11]
207 [ 8 10]
208 [ 2 2]
209 ]
210 Note: transpose() does what it says and is a convenient way to turn row
212 vectors into column vectors. It is bound to the unary operator '~' for
213 convenience.
214 How to write a simple function
216 sub dotproduct {
218 my ($a,$b) = @_;
219 return sum($a*$b) ;
220 }
221 1;
222 If put in file dotproduct.pdl would be autoloaded if you are using
224 PDL::AutoLoader (see below).
225 Of course, this function is already available as the inner function,
227 see PDL::Primitive.
228 Type Conversion
230 Default for pdl() is double. Conversions are:
232 $a = float($b);
234 $c = long($d); # "long" is generally a 4 byte int
235 $d = byte($a);
236 Also double(), short(), ushort().
238 These routines also automatically convert perl lists to allow the con‐
240 venient shorthand:
241 $a = byte [[1..10],[1..10]]; # Create 2D byte array
243 $a = float [1..1000]; # Create 1D float array
244 etc.
246 Piddles and boolean expressions
248 Printing
250 Automatically expands array in N-dimensional format:
252 print $a;
254 $b = "Answer is = $a ";
256 Sections
258 PDL has very powerful multidimensional slicing and sectioning opera‐
260 tors; see the PDL::Slices(3) man page for details; we'll describe the
261 most imporant one here.
262 PDL shows its perl/C heritage in that arrays are zero-offset. Thus a
264 100x100 image has indices "0..99,0..99". (The convention is that the
265 center of pixel (0,0) is at coordinate (0.0,0.0). All PDL graphics
266 functions conform to this definition and hide away the unit-offsetness
267 of, for example, the PGPLOT FORTRAN library.
268 Following the usual convention coordinate (0,0) is displayed at the
270 bottom left when displaying an image. It appears at the top left when
271 using ""print $a"" etc.
272 Simple sectioning uses a syntax extension to perl, PDL::NiceSlice, that
274 allows you to specify subranges via a null-method modifier to a PDL:
275 $b = $a->($x1:$x2,$y1:$y2,($z1)); # Take subsection
277 Here, $a is a 3-dimensional variable, and $b gets a planar cutout that
279 is defined by the limits $x1, $x2, $y1, $y2, at the location $z1. The
280 parenthesis around $z1 cause the trivial index to be omitted -- other‐
281 wise $b would be three-dimensional with a third dimension of order 1.
282 You can put PDL slices on either side of the elementwise-assignment
284 operator ".=", like so:
285 # Set part of $bigimage to values from $smallimage
287 $bigimage->($xa:$xb,$ya:$yb) .= $smallimage;
288 Some other miscellany:
290 $c = nelem($a); # Number of pixels
292 $val = at($object, $x,$y,$z...) # Pixel value at position, as a perl scalar
294 $val = $object->at($x,$y,$z...) # equivalent (method syntax OK)
295 $b = xvals($a); # Fill array with X-coord values (also yvals(), zvals(),
297 # axisvals($x,$axis) and rvals() for radial distance
298 # from centre).
299 Input/Output
301 The "PDL::IO" modules implement several useful IO format functions. It
303 would be too much to give examples of each so you are referred to the
304 individual manpages for details.
305 PDL::IO::Misc
307 Ascii, FITS and FIGARO/NDF IO routines.
308 PDL::IO::FastRaw
310 Using the raw data types of your machine, an unportable but
311 blindingly fast IO format. Also supports memory mapping to con‐
312 serve memory as well as get more speed.
313 PDL::IO::FlexRaw
315 General raw data formats.
316 PDL::IO::Browser
318 A Curses browser for arrays.
319 PDL::IO::Pnm
321 Portaple bitmap and pixmap support.
322 PDL::IO::Pic
324 Using the previous module and netpbm, makes it possible to eas‐
325 ily write GIF, jpeg and whatever with simple commands.
326 Graphics
328 The philosophy behind perlDL is to make it work with a variety of
330 existing graphics libraries since no single package will satisfy all
331 needs and all people and this allows one to work with packages one
332 already knows and likes. Obviously there will be some overlaps in
333 functionality and some lack of consistency and uniformity. However this
334 allows PDL to keep up with a rapidly developing field - the latest PDL
335 modules provide interfaces to OpenGL and VRML graphics!
336 PDL::Graphics::PGPLOT
338 PGPLOT provdes a simple library for line graphics and image dis‐
339 play.
340 There is an easy interface to this in the internal module
342 PDL::Graphics::PGPLOT, which calls routines in the separately
343 available PGPLOT top-level module.
344 PDL::Graphics::IIS
346 Many astronomers like to use SAOimage and Ximtool (or there deriva‐
347 tions/clones). These are useful free widgets for inspection and
348 visualisation of images. (They are not provided with perlDL but can
349 easily be obtained from their official sites off the Net.)
350 The PDL::Graphics::IIS package provides allows one to display
352 images in these ("IIS" is the name of an ancient item of image dis‐
353 play hardware whose protocols these tools conform to.)
354 Karma
356 The PDL::Graphics::Karma module provides an interface to the Karma
357 visualisation suite. This is a set of GUI applications which are
358 specially designed for visualising noisy 2D and 3D data sets.
359 PDL::Graphics::TriD
361 See PDL::Graphics::TriD (the name sucks...). this is a collection
362 of 3D routines for OpenGL and (soon) VRML and other 3D formats
363 which allow 3D point, line, and surface plots from PDL.
364 Autoloading
366 See PDL::AutoLoader. This allows one to autoload functions on demand,
368 in a way perhaps familiar to users of MatLab.
369 One can also write PDL extensions as normal Perl modules.
371 perldl shell
373 The perl script "perldl" provides a simple command line - if the latest
375 Readlines/ReadKey modules have beeen installed "perldl" detects this
376 and enables command line recall and editing. See the manpage for
377 details.
378 e.g.:
380 jhereg% perldl
382 perlDL shell v1.30
383 PDL comes with ABSOLUTELY NO WARRANTY. For details, see the file
384 'COPYING' in the PDL distribution. This is free software and you
385 are welcome to redistribute it under certain conditions, see
386 the same file for details.
387 ReadLines enabled
388 Reading PDL/default.perldlrc...
389 Found docs database /home/kgb/soft/dev/lib/perl5/site_perl/PDL/pdldoc.db
390 Type 'help' for online help
391 Type 'demo' for online demos
392 Loaded PDL v2.005
393 perldl> $x = rfits 'm51.fits'
394 BITPIX = 16 size = 65536 pixels
395 Reading 131072 bytes
396 BSCALE = 1.0000000000E0 && BZERO = 0.0000000000E0
397 perldl> imag $x
399 Loaded PGPLOT
400 Displaying 256 x 256 image from 24 to 500 ...
401 You can also run it from the perl debugger ("perl -MPDL -d -e 1") if
403 you want.
404 Miscellaneous shell features:
406 p The shell aliases "p" to be a convenient short form of "print",
408 e.g.
409 perldl> p ones 5,3
411 [
412 [1 1 1 1 1]
413 [1 1 1 1 1]
414 [1 1 1 1 1]
415 ]
416 Initialization
418 The files "~/.perldlrc" and "local.perldlrc" (in the current direc‐
419 tory) are sourced if found. This allows the user to have global and
420 local PDL code for startup.
421 Help
423 Type 'help'! One can search the PDL documentation, and look up doc‐
424 umentation on any function.
425 Escape
427 Any line starting with the "#" character is treated as a shell
428 escape. This character is configurable by setting the perl variable
429 $PERLDL_ESCAPE. This could, for example, be set in "~/.perldlrc".
430 Overload operators
432 The following builtin perl operators and functions have been overloaded
434 to work on PDL variables:
435 + - * / > < >= <= << >> & ⎪ ^ == != <=> ** % ! ~
437 sin log abs atan2 sqrt cos exp
438 [All the unary functions (sin etc.) may be used with inplace() - see
440 "Memory" below.]
441 Object-Orientation and perlDL
443 PDL operations are available as functions and methods. Thus one can
445 derive new types of object, to represent custom data classes.
446 By using overloading one can make mathematical operators do whatever
448 you please, and PDL has some built-in tricks which allow existing PDL
449 functions to work unchanged, even if the underlying data representation
450 is vastly changed! See PDL::Objects
451 Memory usage and references
453 Messing around with really huge data arrays may require some care.
455 perlDL provides many facilities to let you perform operations on big
456 arrays without generating extra copies though this does require a bit
457 more thought and care from the programmer.
458 NOTE: On some most systems it is better to configure perl (during the
460 build options) to use the system "malloc()" function rather than perl's
461 built-in one. This is because perl's one is optimised for speed rather
462 than consumption of virtual memory - this can result in a factor of two
463 improvement in the amount of memory storage you can use. The Perl mal‐
464 loc in 5.004 and later does have a number of compile-time options you
465 can use to tune the behaviour.
466 Simple arithmetic
468 If $a is a big image (e.g. occupying 10MB) then the command
469 $a = $a + 1;
471 eats up another 10MB of memory. This is because the expression
473 "$a+1" creates a temporary copy of $a to hold the result, then $a
474 is assigned a reference to that. After this, the original $a is
475 destroyed so there is no permanent memory waste. But on a small
476 machine, the growth in the memory footprint can be considerable.
477 It is obviously done this way so "$c=$a+1" works as expected.
478 Also if one says:
480 $b = $a; # $b and $a now point to same data
482 $a = $a + 1;
483 Then $b and $a end up being different, as one naively expects,
485 because a new reference is created and $a is assigned to it.
486 However if $a was a huge memory hog (e.g. a 3D volume) creating a
488 copy of it may not be a good thing. One can avoid this memory over‐
489 head in the above example by saying:
490 $a++;
492 The operations "++,+=,--,-=", etc. all call a special "in-place"
494 version of the arithmetic subroutine. This means no more memory is
495 needed - the downside of this is that if "$b=$a" then $b is also
496 incremented. To force a copy explicitly:
497 $b = pdl $a; # Real copy
499 or, alternatively, perhaps better style:
501 $b = $a->copy;
503 Functions
505 Most functions, e.g. "log()", return a result which is a transfor‐
506 mation of their argument. This makes for good programming practice.
507 However many operations can be done "in-place" and this may be
508 required when large arrays are in use and memory is at a premium.
509 For these circumstances the operator inplace() is provided which
510 prevents the extra copy and allows the argument to be modified.
511 e.g.:
512 $x = log($array); # $array unaffected
514 log( inplace($bigarray) ); # $bigarray changed in situ
515 WARNINGS:
517 1 The usual caveats about duplicate references apply.
519 2 Obviously when used with some functions which can not be
521 applied in situ (e.g. "convolve()") unexpected effects may
522 occur! We try to indicate "inplace()"-safe functions in the
523 documentation.
524 3 Type conversions, such as"float()", may cause hidden copying.
526 Ensuring piddleness
528 If you have written a simple function and you don't want it to blow up
530 in your face if you pass it a simple number rather than a PDL variable.
531 Simply call the function topdl() first to make it safe. e.g.:
532 sub myfiddle { my $pdl = topdl(shift); $pdl->fiddle_foo(...); ... }
534 "topdl()" does NOT perform a copy if a pdl variable is passed - it just
536 falls through - which is obviously the desired behaviour. The routine
537 is not of course necessary in normal user defined functions which do
538 not care about internals.
539
541 Copyright (C) Karl Glazebrook (kgb@aaoepp.aao.gov.au), Tuomas J. Lukka,
542 (lukka@husc.harvard.edu) and Christian Soeller (c.soeller@auck‐
543 land.ac.nz) 1997. Commercial reproduction of this documentation in a
544 different format is forbidden.
545perl v5.8.8 2003-05-21 IMPATIENT(1)