1IMPATIENT(1) User Contributed Perl Documentation IMPATIENT(1)
2
6 PDL::Impatient - PDL for the impatient (quick overview)
7
9 A brief summary of the main PDL features and how to use them.
10
12 Introduction
13 Perl is an extremely good and versatile scripting language, well suited
14 to beginners and allows rapid prototyping. However until recently it
15 did not support data structures which allowed it to do fast number
16 crunching.
17 However with the development of Perl v5, Perl acquired 'Objects'. To
19 put it simply users can define their own special data types, and write
20 custom routines to manipulate them either in low level languages (C and
21 Fortran) or in Perl itself.
22 This has been fully exploited by the PerlDL developers. The 'PDL'
24 module is a complete Object-Oriented extension to Perl (although you
25 don't have to know what an object is to use it) which allows large
26 N-dimensional data sets, such as large images, spectra, time series,
27 etc to be stored efficiently and manipulated en masse. For example
28 with the PDL module we can write the perl code "$a=$b+$c", where $b and
29 $c are large datasets (e.g. 2048x2048 images), and get the result in
30 only a fraction of a second.
31 PDL variables (or 'piddles' as they have come to be known) support a
33 wide range of fundamental data types - arrays can be bytes, short
34 integers (signed or unsigned), long integers, floats or double
35 precision floats. And because of the Object-Oriented nature of PDL new
36 customised datatypes can be derived from them.
37 As well as the PDL modules, that can be used by normal perl programs,
39 PerlDL comes with a command line perl shell, called 'perldl', which
40 supports command line editing. In combination with the various PDL
41 graphics modules this allows data to be easily played with and
42 visualised.
43 Help
45 PDL contains extensive documentation, available both within the perldl
46 shell and from the command line, using the "pdldoc" program. For
47 further information try either of:
48 perldl> help help
50 $ pdldoc
51 HTML copies of the documentation should also be available. To find
53 their location, try the following:
54 perldl> foreach ( map{"$_/PDL/HtmlDocs"}@INC ) { p "$_\n" if -d $_ }
56 Perl Datatypes and how PDL extends them
58 The fundamental perl data structures are scalar variables, e.g. $x,
59 which can hold numbers or strings, lists or arrays of scalars, e.g. @x,
60 and associative arrays/hashes of scalars, e.g. %x.
61 perl v5 introduces to perl data structures and objects. A simple scalar
63 variable $x now be a user-defined data type or full blown object (it
64 actually holds a reference (a smart "pointer") to this but that is not
65 relevant for ordinary use of perlDL)
66 The fundamental idea behind perlDL is to allow $x to hold a whole 1D
68 spectrum, or a 2D image, a 3D data cube, and so on up to large
69 N-dimensional data sets. These can be manipulated all at once, e.g.
70 "$a = $b + 2" does a vector operation on each value in the
71 spectrum/image/etc.
72 You may well ask: "Why not just store a spectrum as a simple perl @x
74 style list with each pixel being a list item?" The two key answers to
75 this are memory and speed. Because we know our spectrum consists of
76 pure numbers we can compactly store them in a single block of memory
77 corresponding to a C style numeric array. This takes up a LOT less
78 memory than the equivalent perl list. It is then easy to pass this
79 block of memory to a fast addition routine, or to any other C function
80 which deals with arrays. As a result perlDL is very fast --- for
81 example one can mulitiply a 2048*2048 image in exactly the same time as
82 it would take in C or FORTRAN (0.1 sec on my SPARC). A further
83 advantage of this is that for simple operations (e.g. "$x += 2") one
84 can manipulate the whole array without caring about its dimensionality.
85 I find when using perlDL it is most useful to think of standard perl @x
87 variables as "lists" of generic "things" and PDL variables like $x as
88 "arrays" which can be contained in lists or hashes. Quite often in my
89 perlDL scripts I have @x contain a list of spectra, or a list of images
90 (or even a mix!). Or perhaps one could have a hash (e.g. %x) of
91 images... the only limit is memory!
92 perlDL variables support a range of data types - arrays can be bytes,
94 short integers (signed or unsigned), long integers, floats or double
95 precision floats.
96 Usage
98 PerlDL is loaded into your perl script using this command:
99 use PDL; # in perl scripts: use the standard perlDL modules
101 There are also a lot of extension modules, e.g. PDL::Graphics::TriD.
103 Most of these (but not all as sometimes it is not appropriate) follow a
104 standard convention. If you say:
105 use PDL::Graphics::TriD;
107 You import everything in a standard list from the module. Sometimes you
109 might want to import nothing (e.g. if you want to use OO syntax all the
110 time and save the import tax). For these you say:
111 use PDL::Graphics::TriD '';
113 And the blank quotes '' are regonised as meaning 'nothing'. You can
115 also specify a list of functions to import in the normal Perl way.
116 There is also an interactive shell, "perldl", see perldl.
118 To create a new PDL variable
120 Here are some ways of creating a PDL variable:
121 $a = pdl [1..10]; # 1D array
123 $a = pdl (1,2,3,4); # Ditto
124 $b = pdl [[1,2,3],[4,5,6]]; # 2D 3x2 array
125 $b = pdl 42 # 0-dimensional scalar
126 $c = pdl $a; # Make a new copy
127 $d = byte [1..10]; # See "Type conversion"
129 $e = zeroes(3,2,4); # 3x2x4 zero-filled array
130 $c = rfits $file; # Read FITS file
132 @x = ( pdl(42), zeroes(3,2,4), rfits($file) ); # Is a LIST of PDL variables!
134 The pdl() function is used to initialise a PDL variable from a scalar,
136 list, list reference or another PDL variable.
137 In addition all PDL functions automatically convert normal perl scalars
139 to PDL variables on-the-fly.
140 (also see "Type Conversion" and "Input/Output" sections below)
142 Arithmetic (and boolean expressions)
144 $a = $b + 2; $a++; $a = $b / $c; # Etc.
145 $c=sqrt($a); $d = log10($b+100); # Etc
147 $e = $a>42; # Vector conditional
149 $e = 42*($a>42) + $a*($a<=42); # Cap top
151 $b = $a->log10 unless any ($a <= 0); # avoid floating point error
153 $a = $a / ( max($a) - min($a) );
155 $f = where($a, $a > 10); # where returns a piddle of elements for
157 # which the condition is true
158 print $a; # $a in string context prints it in a N-dimensional format
160 (and other perl operators/functions)
162 When using piddles in conditional expressions (i.e. "if", "unless" and
164 "while" constructs) only piddles with exactly one element are allowed,
165 e.g.
166 $a = pdl (1,0,0,1);
168 print "is set" if $a->index(2);
169 Note that the boolean operators return in general multielement piddles.
171 Therefore, the following will raise an error
172 print "is ok" if $a > 3;
174 since "$a > 3" is a piddle with 4 elements. Rather use all or any to
176 test if all or any of the elements fulfill the condition:
177 print "some are > 3" if any $a>3;
179 print "can't take logarithm" unless all $a>0;
180 There are also many predefined functions, which are described on other
182 manpages. Check PDL::Index.
183 Matrix functions
185 'x' is hijacked as the matrix multiplication operator. e.g. "$c = $a x
186 $b";
187 perlDL is row-major not column major so this is actually "c(i,j) =
189 sum_k a(k,j) b(i,k)" - but when matrices are printed the results will
190 look right. Just remember the indices are reversed. e.g.:
191 $a = [ $b = [
193 [ 1 2 3 0] [1 1]
194 [ 1 -1 2 7] [0 2]
195 [ 1 0 0 1] [0 2]
196 ] [1 1]
197 ]
198 gives $c = [
200 [ 1 11]
201 [ 8 10]
202 [ 2 2]
203 ]
204 Note: transpose() does what it says and is a convenient way to turn row
206 vectors into column vectors.
207 How to write a simple function
209 sub dotproduct {
210 my ($a,$b) = @_;
211 return sum($a*$b) ;
212 }
213 1;
214 If put in file dotproduct.pdl would be autoloaded if you are using
216 PDL::AutoLoader (see below).
217 Of course, this function is already available as the inner function,
219 see PDL::Primitive.
220 Type Conversion
222 Default for pdl() is double. Conversions are:
223 $a = float($b);
225 $c = long($d); # "long" is generally a 4 byte int
226 $d = byte($a);
227 Also double(), short(), ushort().
229 These routines also automatically convert perl lists to allow the
231 convenient shorthand:
232 $a = byte [[1..10],[1..10]]; # Create 2D byte array
234 $a = float [1..1000]; # Create 1D float array
235 etc.
237 Piddles and boolean expressions
239 Printing
240 Automatically expands array in N-dimensional format:
241 print $a;
243 $b = "Answer is = $a ";
245 Sections
247 PDL has very powerful multidimensional slicing and sectioning
248 operators; see the PDL::Slices(3) man page for details; we'll describe
249 the most imporant one here.
250 PDL shows its perl/C heritage in that arrays are zero-offset. Thus a
252 100x100 image has indices "0..99,0..99". (The convention is that the
253 center of pixel (0,0) is at coordinate (0.0,0.0). All PDL graphics
254 functions conform to this definition and hide away the unit-offsetness
255 of, for example, the PGPLOT FORTRAN library.
256 Following the usual convention coordinate (0,0) is displayed at the
258 bottom left when displaying an image. It appears at the top left when
259 using ""print $a"" etc.
260 Simple sectioning uses a syntax extension to perl, PDL::NiceSlice, that
262 allows you to specify subranges via a null-method modifier to a PDL:
263 $b = $a->($x1:$x2,$y1:$y2,($z1)); # Take subsection
265 Here, $a is a 3-dimensional variable, and $b gets a planar cutout that
267 is defined by the limits $x1, $x2, $y1, $y2, at the location $z1. The
268 parenthesis around $z1 cause the trivial index to be omitted --
269 otherwise $b would be three-dimensional with a third dimension of order
270 1.
271 You can put PDL slices on either side of the elementwise-assignment
273 operator ".=", like so:
274 # Set part of $bigimage to values from $smallimage
276 $bigimage->($xa:$xb,$ya:$yb) .= $smallimage;
277 Some other miscellany:
279 $c = nelem($a); # Number of pixels
281 $val = at($object, $x,$y,$z...) # Pixel value at position, as a perl scalar
283 $val = $object->at($x,$y,$z...) # equivalent (method syntax OK)
284 $b = xvals($a); # Fill array with X-coord values (also yvals(), zvals(),
286 # axisvals($x,$axis) and rvals() for radial distance
287 # from centre).
288 Input/Output
290 The "PDL::IO" modules implement several useful IO format functions. It
291 would be too much to give examples of each so you are referred to the
292 individual manpages for details.
293 PDL::IO::Misc
295 Ascii, FITS and FIGARO/NDF IO routines.
296 PDL::IO::FastRaw
298 Using the raw data types of your machine, an unportable but
299 blindingly fast IO format. Also supports memory mapping to
300 conserve memory as well as get more speed.
301 PDL::IO::FlexRaw
303 General raw data formats.
304 PDL::IO::Browser
306 A Curses browser for arrays.
307 PDL::IO::Pnm
309 Portaple bitmap and pixmap support.
310 PDL::IO::Pic
312 Using the previous module and netpbm, makes it possible to
313 easily write GIF, jpeg and whatever with simple commands.
314 Graphics
316 The philosophy behind perlDL is to make it work with a variety of
317 existing graphics libraries since no single package will satisfy all
318 needs and all people and this allows one to work with packages one
319 already knows and likes. Obviously there will be some overlaps in
320 functionality and some lack of consistency and uniformity. However this
321 allows PDL to keep up with a rapidly developing field - the latest PDL
322 modules provide interfaces to OpenGL and VRML graphics!
323 PDL::Graphics::PGPLOT
325 PGPLOT provdes a simple library for line graphics and image
326 display.
327 There is an easy interface to this in the internal module
329 PDL::Graphics::PGPLOT, which calls routines in the separately
330 available PGPLOT top-level module.
331 PDL::Graphics::IIS
333 Many astronomers like to use SAOimage and Ximtool (or there
334 derivations/clones). These are useful free widgets for inspection
335 and visualisation of images. (They are not provided with perlDL but
336 can easily be obtained from their official sites off the Net.)
337 The PDL::Graphics::IIS package provides allows one to display
339 images in these ("IIS" is the name of an ancient item of image
340 display hardware whose protocols these tools conform to.)
341 Karma
343 The PDL::Graphics::Karma module provides an interface to the Karma
344 visualisation suite. This is a set of GUI applications which are
345 specially designed for visualising noisy 2D and 3D data sets.
346 PDL::Graphics::TriD
348 See PDL::Graphics::TriD (the name sucks...). this is a collection
349 of 3D routines for OpenGL and (soon) VRML and other 3D formats
350 which allow 3D point, line, and surface plots from PDL.
351 Autoloading
353 See PDL::AutoLoader. This allows one to autoload functions on demand,
354 in a way perhaps familiar to users of MatLab.
355 One can also write PDL extensions as normal Perl modules.
357 perldl shell
359 The perl script "perldl" provides a simple command line - if the latest
360 Readlines/ReadKey modules have beeen installed "perldl" detects this
361 and enables command line recall and editing. See the manpage for
362 details.
363 e.g.:
365 jhereg% perldl
367 perlDL shell v1.30
368 PDL comes with ABSOLUTELY NO WARRANTY. For details, see the file
369 'COPYING' in the PDL distribution. This is free software and you
370 are welcome to redistribute it under certain conditions, see
371 the same file for details.
372 ReadLines enabled
373 Reading PDL/default.perldlrc...
374 Found docs database /home/kgb/soft/dev/lib/perl5/site_perl/PDL/pdldoc.db
375 Type 'help' for online help
376 Type 'demo' for online demos
377 Loaded PDL v2.005
378 perldl> $x = rfits 'm51.fits'
379 BITPIX = 16 size = 65536 pixels
380 Reading 131072 bytes
381 BSCALE = 1.0000000000E0 && BZERO = 0.0000000000E0
382 perldl> imag $x
384 Loaded PGPLOT
385 Displaying 256 x 256 image from 24 to 500 ...
386 You can also run it from the perl debugger ("perl -MPDL -d -e 1") if
388 you want.
389 Miscellaneous shell features:
391 p The shell aliases "p" to be a convenient short form of "print",
393 e.g.
394 perldl> p ones 5,3
396 [
397 [1 1 1 1 1]
398 [1 1 1 1 1]
399 [1 1 1 1 1]
400 ]
401 Initialization
403 The files "~/.perldlrc" and "local.perldlrc" (in the current
404 directory) are sourced if found. This allows the user to have
405 global and local PDL code for startup.
406 Help
408 Type 'help'! One can search the PDL documentation, and look up
409 documentation on any function.
410 Escape
412 Any line starting with the "#" character is treated as a shell
413 escape. This character is configurable by setting the perl variable
414 $PERLDL_ESCAPE. This could, for example, be set in "~/.perldlrc".
415 Overload operators
417 The following builtin perl operators and functions have been overloaded
418 to work on PDL variables:
419 + - * / > < >= <= << >> & | ^ == != <=> ** % ! ~
421 sin log abs atan2 sqrt cos exp
422 [All the unary functions (sin etc.) may be used with inplace() - see
424 "Memory" below.]
425 Object-Orientation and perlDL
427 PDL operations are available as functions and methods. Thus one can
428 derive new types of object, to represent custom data classes.
429 By using overloading one can make mathematical operators do whatever
431 you please, and PDL has some built-in tricks which allow existing PDL
432 functions to work unchanged, even if the underlying data representation
433 is vastly changed! See PDL::Objects
434 Memory usage and references
436 Messing around with really huge data arrays may require some care.
437 perlDL provides many facilities to let you perform operations on big
438 arrays without generating extra copies though this does require a bit
439 more thought and care from the programmer.
440 NOTE: On some most systems it is better to configure perl (during the
442 build options) to use the system "malloc()" function rather than perl's
443 built-in one. This is because perl's one is optimised for speed rather
444 than consumption of virtual memory - this can result in a factor of two
445 improvement in the amount of memory storage you can use. The Perl
446 malloc in 5.004 and later does have a number of compile-time options
447 you can use to tune the behaviour.
448 Simple arithmetic
450 If $a is a big image (e.g. occupying 10MB) then the command
451 $a = $a + 1;
453 eats up another 10MB of memory. This is because the expression
455 "$a+1" creates a temporary copy of $a to hold the result, then $a
456 is assigned a reference to that. After this, the original $a is
457 destroyed so there is no permanent memory waste. But on a small
458 machine, the growth in the memory footprint can be considerable.
459 It is obviously done this way so "$c=$a+1" works as expected.
460 Also if one says:
462 $b = $a; # $b and $a now point to same data
464 $a = $a + 1;
465 Then $b and $a end up being different, as one naively expects,
467 because a new reference is created and $a is assigned to it.
468 However if $a was a huge memory hog (e.g. a 3D volume) creating a
470 copy of it may not be a good thing. One can avoid this memory
471 overhead in the above example by saying:
472 $a++;
474 The operations "++,+=,--,-=", etc. all call a special "in-place"
476 version of the arithmetic subroutine. This means no more memory is
477 needed - the downside of this is that if "$b=$a" then $b is also
478 incremented. To force a copy explicitly:
479 $b = pdl $a; # Real copy
481 or, alternatively, perhaps better style:
483 $b = $a->copy;
485 Functions
487 Most functions, e.g. "log()", return a result which is a
488 transformation of their argument. This makes for good programming
489 practice. However many operations can be done "in-place" and this
490 may be required when large arrays are in use and memory is at a
491 premium. For these circumstances the operator inplace() is provided
492 which prevents the extra copy and allows the argument to be
493 modified. e.g.:
494 $x = log($array); # $array unaffected
496 log( inplace($bigarray) ); # $bigarray changed in situ
497 WARNINGS:
499 1. The usual caveats about duplicate references apply.
501 2. Obviously when used with some functions which can not be
503 applied in situ (e.g. "convolve()") unexpected effects may
504 occur! We try to indicate "inplace()"-safe functions in the
505 documentation.
506 3. Type conversions, such as"float()", may cause hidden copying.
508 Ensuring piddleness
510 If you have written a simple function and you don't want it to blow up
511 in your face if you pass it a simple number rather than a PDL variable.
512 Simply call the function topdl() first to make it safe. e.g.:
513 sub myfiddle { my $pdl = topdl(shift); $pdl->fiddle_foo(...); ... }
515 "topdl()" does NOT perform a copy if a pdl variable is passed - it just
517 falls through - which is obviously the desired behaviour. The routine
518 is not of course necessary in normal user defined functions which do
519 not care about internals.
520
522 Copyright (C) Karl Glazebrook (kgb@aaoepp.aao.gov.au), Tuomas J. Lukka,
523 (lukka@husc.harvard.edu) and Christian Soeller
524 (c.soeller@auckland.ac.nz) 1997. Commercial reproduction of this
525 documentation in a different format is forbidden.
526perl v5.12.3 2009-10-17 IMPATIENT(1)