1QUICKSTART(1)         User Contributed Perl Documentation        QUICKSTART(1)
2
3
4

NAME

6       PDL::QuickStart - Quick introduction to PDL features.
7

SYNOPSIS

9       A brief summary of the main PDL features and how to use them.
10

DESCRIPTION

12   Introduction
13       Perl is an extremely good and versatile scripting language, well suited
14       to beginners and allows rapid prototyping. However until recently it
15       did not support data structures which allowed it to do fast number
16       crunching.
17
18       However with the development of Perl v5, Perl acquired 'Objects'. To
19       put it simply users can define their own special data types, and write
20       custom routines to manipulate them either in low level languages (C and
21       Fortran) or in Perl itself.
22
23       This has been fully exploited by the PerlDL developers. The 'PDL'
24       module is a complete Object-Oriented extension to Perl (although you
25       don't have to know what an object is to use it) which allows large
26       N-dimensional data sets, such as large images, spectra, time series,
27       etc to be stored  efficiently and manipulated en masse.  For example
28       with the PDL module we can write the Perl code "$x = $y + $z", where $y
29       and $z are large datasets (e.g. 2048x2048 images), and get the result
30       in only a fraction of a second.
31
32       PDL variables (or 'ndarrays' as they have come to be known) support a
33       wide range of fundamental data types - arrays can be bytes, short
34       integers (signed or unsigned), long integers, floats or double
35       precision floats. And because of the Object-Oriented nature of PDL new
36       customised datatypes can be derived from them.
37
38       As well as the PDL modules, that can be used by normal Perl programs,
39       PerlDL comes with a command line Perl shell, called 'perldl', which
40       supports command line editing. In combination with the various PDL
41       graphics modules this allows data to be easily played with and
42       visualised.
43
44   Help
45       PDL contains extensive documentation, available both within the perldl
46       or pdl2 shells and from the command line, using the "pdldoc" program.
47       For further information try either of:
48
49        pdl> help help
50        $ pdldoc
51
52       HTML copies of the documentation should also be available.  To find
53       their location, try the following:
54
55        pdl> foreach ( map{"$_/PDL/HtmlDocs"}@INC ) { p "$_\n" if -d $_ }
56
57   Perl Datatypes and how PDL extends them
58       The fundamental Perl data structures are scalar variables, e.g. $x,
59       which can hold numbers or strings, lists or arrays of scalars, e.g. @x,
60       and associative arrays/hashes of scalars, e.g. %x.
61
62       Perl v5 introduces to Perl data structures and objects. A simple scalar
63       variable $x now be a user-defined data type or full blown object (it
64       actually holds a reference (a smart "pointer") to this but that is not
65       relevant for ordinary use of perlDL)
66
67       The fundamental idea behind perlDL is to allow $x to hold a whole 1D
68       spectrum, or a 2D image, a 3D data cube, and so on up to large
69       N-dimensional data sets. These can be manipulated all at once, e.g.
70       "$x = $y + 2" does a vector operation on each value in the
71       spectrum/image/etc.
72
73       You may well ask: "Why not just store a spectrum as a simple Perl @x
74       style list with each pixel being a list item?"  The two key answers to
75       this are memory and speed.  Because we know our spectrum consists of
76       pure numbers we can compactly store them in a single block of memory
77       corresponding to a C style numeric array. This takes up a LOT less
78       memory than the equivalent Perl list. It is then easy to pass this
79       block of memory to a fast addition routine, or to any other C function
80       which deals with arrays.  As a result perlDL is very fast --- for
81       example one can multiply a 2048*2048 image in exactly the same time as
82       it would take in C or FORTRAN (0.1 sec on my SPARC). A further
83       advantage of this is that for simple operations (e.g. "$x += 2") one
84       can manipulate the whole array without caring about its dimensionality.
85
86       I find when using perlDL it is most useful to think of standard Perl @x
87       variables as "lists" of generic "things" and PDL variables like $x as
88       "arrays" which can be contained in lists or hashes. Quite often in my
89       perlDL scripts I have @x contain a list of spectra, or a list of images
90       (or even a mix!). Or perhaps one could have a hash (e.g.  %x) of
91       images... the only limit is memory!
92
93       perlDL variables support a range of data types - arrays can be bytes,
94       short integers (signed or unsigned), long integers, floats or double
95       precision floats.
96
97   Usage
98       PerlDL is loaded into your Perl script using this command:
99
100        use PDL;  # in Perl scripts: use the standard perlDL modules
101
102       There are also a lot of extension modules, e.g.  PDL::Graphics::TriD.
103       Most of these (but not all as sometimes it is not appropriate) follow a
104       standard convention. If you say:
105
106        use PDL::Graphics::TriD;
107
108       You import everything in a standard list from the module. Sometimes you
109       might want to import nothing (e.g. if you want to use OO syntax all the
110       time and save the import tax). For these you say:
111
112        use PDL::Graphics::TriD qw();
113
114       And the empty "qw()"  quotes are recognised as meaning 'nothing'.  You
115       can also specify a list of functions to import in the normal Perl way.
116
117       There is also an interactive shell, "perldl" or "pdl2", see perldl or
118       pdl2 for details.
119
120   To create a new PDL variable
121       Here are some ways of creating a PDL variable:
122
123        $x = pdl [1..10];             # 1D array
124        $x = pdl (1,2,3,4);           # Ditto
125        $x = pdl '[1 2 3 4]';         # Ditto
126        $y = pdl [[1,2,3],[4,5,6]];   # 2D 3x2 array
127        $y = pdl '[1 2 3; 4 5 6]';    # Ditto
128        $y = pdl q[1,2,3; 4,5,6];     # Ditto
129        $y = pdl <<NEWPDL             # Ditto
130          [1 2 3]
131          [4 5 6]
132        NEWPDL
133        $c = pdl q[1 -2];             # 2-element ndarray containing 1 and -2
134        $c = pdl q[1 - 2];            # 2-element ndarray containing 1 and -2
135        $y = pdl 42                   # 0-dimensional scalar
136        $c = pdl $x;                  # Make a new copy
137
138        $d = byte [1..10];            # See "Type conversion"
139        $e = zeroes(3,2,4);           # 3x2x4 zero-filled array
140
141        $c = rfits $file;             # Read FITS file
142
143        @x = ( pdl(42), zeroes(3,2,4), rfits($file) ); # Is a LIST of PDL variables!
144
145       The pdl() function is used to initialise a PDL variable from a scalar,
146       list, list reference, another PDL variable, or a properly formatted
147       string.
148
149       In addition all PDL functions automatically convert normal Perl scalars
150       to PDL variables on-the-fly.
151
152       (also see "Type Conversion" and "Input/Output" sections below)
153
154   Arithmetic (and boolean expressions)
155        $x = $y + 2; $x++; $x = $y / $c; # Etc.
156
157        $c=sqrt($x); $d = log10($y+100); # Etc
158
159        $e = $x>42; # Vector conditional
160
161        $e = 42*($x>42) + $x*($x<=42); # Cap top
162
163        $y = $x->log10 unless any ($x <= 0); # avoid floating point error
164
165        $x = $x / ( max($x) - min($x) );
166
167        $f = where($x, $x > 10); # where returns an ndarray of elements for
168                                 # which the condition is true
169
170        print $x; # $x in string context prints it in a N-dimensional format
171
172       (and other Perl operators/functions)
173
174       When using ndarrays in conditional expressions (i.e. "if", "unless" and
175       "while" constructs) only ndarrays with exactly one element are allowed,
176       e.g.
177
178        $x = pdl (1,0,0,1);
179        print "is set" if $x->index(2);
180
181       Note that the boolean operators return in general multi-element
182       ndarrays. Therefore, the following will raise an error
183
184        print "is ok" if $x > 3;
185
186       since "$x > 3" is an ndarray with 4 elements. Rather use all or any to
187       test if all or any of the elements fulfill the condition:
188
189        print "some are > 3" if any $x>3;
190        print "can't take logarithm" unless all $x>0;
191
192       There are also many predefined functions, which are described on other
193       man pages. Check PDL::Index.
194
195   Matrix functions
196       'x' is hijacked as the matrix multiplication operator. e.g.  "$c = $x x
197       $y";
198
199       perlDL is row-major not column major so this is actually "c(i,j) =
200       sum_k x(k,j) y(i,k)" - but when matrices are printed the results will
201       look right. Just remember the indices are reversed.  e.g.:
202
203        $x = [                   $y = [
204              [ 1  2  3  0]            [1 1]
205              [ 1 -1  2  7]            [0 2]
206              [ 1  0  0  1]            [0 2]
207             ]                         [1 1]
208                                      ]
209
210        gives $c = [
211                    [ 1 11]
212                    [ 8 10]
213                    [ 2  2]
214                   ]
215
216       Note: transpose() does what it says and is a convenient way to turn row
217       vectors into column vectors.
218
219   How to write a simple function
220        sub dotproduct {
221            my ($x,$y) = @_;
222            return sum($x*$y) ;
223        }
224        1;
225
226       If put in file dotproduct.pdl would be autoloaded if you are using
227       PDL::AutoLoader (see below).
228
229       Of course, this function is already available as the inner function,
230       see PDL::Primitive.
231
232   Type Conversion
233       Default for pdl() is double. Conversions are:
234
235        $x = float($y);
236        $c = long($d);   # "long" is generally a 4 byte int
237        $d = byte($x);
238
239       Also double(), short(), ushort(), indx().
240
241         NOTE: The indx() routine is a special integer type that
242         is the correct size for a PDL index value (dimension size,
243         index, or offest) which can be either a 32bit (long) or
244         64bit (longlong) quantity depending on whether the perl
245         is built with 32bit or 64bit support.
246
247       These routines also automatically convert Perl lists to allow the
248       convenient shorthand:
249
250        $x = byte [[1..10],[1..10]];  # Create 2D byte array
251        $x = float [1..1000];         # Create 1D float array
252
253       etc.
254
255   Printing
256       Automatically expands array in N-dimensional format:
257
258        print $x;
259
260        $y = "Answer is = $x ";
261
262   Sections
263       PDL has very powerful multidimensional slicing and sectioning
264       operators; see the PDL::Slices(3) man page for details; we'll describe
265       the most important one here.
266
267       PDL shows its Perl/C heritage in that arrays are zero-offset.  Thus a
268       100x100 image has indices "0..99,0..99".  (The convention is that the
269       center of pixel (0,0) is at coordinate (0.0,0.0). All PDL graphics
270       functions conform to this definition and hide away the unit offsets of,
271       for example, the PGPLOT FORTRAN library.
272
273       Following the usual convention coordinate (0,0) is displayed at the
274       bottom left when displaying an image. It appears at the top left when
275       using ""print $x"" etc.
276
277       Simple sectioning uses a syntax extension to Perl, PDL::NiceSlice, that
278       allows you to specify subranges via a null-method modifier to a PDL:
279
280         $g = $f->($x1:$x2,$y1:$y2,($z1)); # Take subsection
281
282       Here, $f is a 3-dimensional variable, and $g gets a planar cutout that
283       is defined by the limits $x1, $x2, $y1, $y2, at the location $z1.  The
284       parenthesis around $z1 cause the trivial index to be omitted --
285       otherwise $g would be three-dimensional with a third dimension of order
286       1.
287
288       You can put PDL slices on either side of the element-wise assignment
289       operator ".=", like so:
290
291         # Set part of $bigimage to values from $smallimage
292         $bigimage->($xa:$xb,$ya:$yb) .= $smallimage;
293
294       Some other miscellany:
295
296        $c  = nelem($x); # Number of pixels
297
298        $val = at($object, $x,$y,$z...)    # Pixel value at position, as a Perl scalar
299        $val = $object->at($x,$y,$z...)    # equivalent (method syntax OK)
300
301        $y = xvals($x); # Fill array with X-coord values (also yvals(), zvals(),
302                        # axisvals($x,$axis) and rvals() for radial distance
303                        # from centre).
304
305   Input/Output
306       The "PDL::IO" modules implement several useful IO format functions.  It
307       would be too much to give examples of each, but you can find a nice
308       overview at PDL::IO. Here is a sample of some of the supported IO
309       formats in PDL.
310
311       PDL::IO::Misc
312               Ascii, FITS and FIGARO/NDF IO routines.
313
314       PDL::IO::FastRaw
315               Using the raw data types of your machine, an unportable but
316               blindingly fast IO format. Also supports memory mapping to
317               conserve memory as well as get more speed.
318
319       PDL::IO::FlexRaw
320               General raw data formats. Like FastRaw, only better.
321
322       PDL::IO::Browser
323               A Curses browser for arrays.
324
325       PDL::IO::Pnm
326               Portaple bitmap and pixmap support.
327
328       PDL::IO::Pic
329               Using the previous module and netpbm, makes it possible to
330               easily write GIF, jpeg and whatever with simple commands.
331
332   Graphics
333       The philosophy behind perlDL is to make it work with a variety of
334       existing graphics libraries since no single package will satisfy all
335       needs and all people and this allows one to work with packages one
336       already knows and likes.  Obviously there will be some overlaps in
337       functionality and some lack of consistency and uniformity. However this
338       allows PDL to keep up with a rapidly developing field - the latest PDL
339       modules provide interfaces to OpenGL and VRML graphics!
340
341       PDL::Graphics::PGPLOT
342           PGPLOT provides a simple library for line graphics and image
343           display.
344
345           There is an easy interface to this in the internal module
346           PDL::Graphics::PGPLOT, which calls routines in the separately
347           available PGPLOT top-level module.
348
349       PDL::Graphics::PLplot
350           PLplot provides a simple library for creating graphics with
351           multiple output drivers, including a direct-to-ndarray driver.
352
353           This module provides both high-level and low-level functionality
354           built on PLplot. The low-level commands are pretty much direct
355           bindings to PLplot's C interface. Read more at
356           PDL::Graphics::PLplot.
357
358       PDL::Graphics::IIS
359           Many astronomers like to use SAOimage and Ximtool (or there
360           derivations/clones). These are useful free widgets for inspection
361           and visualisation of images. (They are not provided with perlDL but
362           can easily be obtained from their official sites off the Net.)
363
364           The PDL::Graphics::IIS package provides allows one to display
365           images in these ("IIS" is the name of an ancient item of image
366           display hardware whose protocols these tools conform to.)
367
368       PDL::Graphics::TriD
369           See PDL::Graphics::TriD, this is a collection of 3D routines for
370           OpenGL and (soon) VRML and other 3D formats which allow 3D point,
371           line, and surface plots from PDL.
372
373   Autoloading
374       See PDL::AutoLoader. This allows one to autoload functions on demand,
375       in a way perhaps familiar to users of MatLab.
376
377       One can also write PDL extensions as normal Perl modules.
378
379   PDL shells
380       The Perl script "pdl2" (or "perldl") provides a simple command line
381       interface to PDL.  If the latest Readlines/ReadKey modules have been
382       installed "pdl2" detects this and enables command line recall and
383       editing.  See the man page for details.
384
385       e.g.:
386
387        % perldl
388        perlDL shell v1.354
389         PDL comes with ABSOLUTELY NO WARRANTY. For details, see the file
390         'COPYING' in the PDL distribution. This is free software and you
391         are welcome to redistribute it under certain conditions, see
392         the same file for details.
393        ReadLines, NiceSlice, MultiLines  enabled
394        Reading PDL/default.perldlrc...
395        Found docs database /home/pdl/dev/lib/perl5/site_perl/PDL/pdldoc.db
396        Type 'help' for online help
397        Type 'demo' for online demos
398        Loaded PDL v2.4.9_003 (supports bad values)
399        pdl> $x = rfits 'm51.fits'
400        Reading IMAGE data...
401        BITPIX =  32  size = 147456 pixels
402        Reading  589824  bytes
403        BSCALE =  &&  BZERO =
404
405        pdl> use PDL::Graphics::PGPLOT;
406        pdl> imag $x
407        Displaying 384 x 384 image from 40 to 761, using 84 colors (16-99)...
408
409       You can also run it from the Perl debugger ("perl -MPDL -d -e 1") if
410       you want.
411
412       Miscellaneous shell features:
413
414       p   The shell aliases "p" to be a convenient short form of "print",
415           e.g.
416
417              pdl> p ones 5,3
418              [
419               [1 1 1 1 1]
420               [1 1 1 1 1]
421               [1 1 1 1 1]
422              ]
423
424       Initialization
425           The files "~/.perldlrc" and "local.perldlrc" (in the current
426           directory) are sourced if found. This allows the user to have
427           global and local PDL code for startup.
428
429       Help
430           Type 'help'! One can search the PDL documentation, and look up
431           documentation on any function.
432
433       Escape
434           Any line starting with the "#" character is treated as a shell
435           escape. This character is configurable by setting the Perl variable
436           $PERLDL_ESCAPE. This could, for example, be set in "~/.perldlrc".
437
438   Overload operators
439       The following builtin Perl operators and functions have been overloaded
440       to work on PDL variables:
441
442        + - * / > < >= <= << >> & | ^ == != <=> ** % ! ~
443        sin log abs atan2 sqrt cos exp
444
445       [All the unary functions (sin etc.) may be used with inplace() - see
446       "Memory" below.]
447
448   Object-Orientation and perlDL
449       PDL operations are available as functions and methods.  Thus one can
450       derive new types of object, to represent custom data classes.
451
452       By using overloading one can make mathematical operators do whatever
453       you please, and PDL has some built-in tricks which allow existing PDL
454       functions to work unchanged, even if the underlying data representation
455       is vastly changed!  See PDL::Objects
456
457   Memory usage and references
458       Messing around with really huge data arrays may require some care.
459       perlDL provides many facilities to let you perform operations on big
460       arrays without generating extra copies though this does require a bit
461       more thought and care from the programmer.
462
463       NOTE: On some most systems it is better to configure Perl (during the
464       build options) to use the system "malloc()" function rather than Perl's
465       built-in one. This is because Perl's one is optimised for speed rather
466       than consumption of virtual memory - this can result in a factor of two
467       improvement in the amount of memory storage you can use.  The Perl
468       malloc in 5.004 and later does have a number of compile-time options
469       you can use to tune the behaviour.
470
471       Simple arithmetic
472           If $x is a big image (e.g. occupying 10MB) then the command
473
474            $x = $x + 1;
475
476           eats up another 10MB of memory. This is because the expression
477           "$x+1" creates a temporary copy of $x to hold the result, then $x
478           is assigned a reference to that.  After this, the original $x is
479           destroyed so there is no permanent memory waste. But on a small
480           machine, the growth in the memory footprint can be considerable.
481           It is obviously done this way so "$c=$x+1" works as expected.
482
483           Also if one says:
484
485            $y = $x;     # $y and $x now point to same data
486            $x = $x + 1;
487
488           Then $y and $x end up being different, as one naively expects,
489           because a new reference is created and $x is assigned to it.
490
491           However if $x was a huge memory hog (e.g. a 3D volume) creating a
492           copy of it may not be a good thing. One can avoid this memory
493           overhead in the above example by saying:
494
495            $x++;
496
497           The operations "++,+=,--,-=", etc. all call a special "in-place"
498           version of the arithmetic subroutine. This means no more memory is
499           needed - the downside of this is that if "$y=$x" then $y is also
500           incremented. To force a copy explicitly:
501
502            $y = pdl $x; # Real copy
503
504           or, alternatively, perhaps better style:
505
506            $y = $x->copy;
507
508       Functions
509           Most functions, e.g. "log()", return a result which is a
510           transformation of their argument. This makes for good programming
511           practice. However many operations can be done "in-place" and this
512           may be required when large arrays are in use and memory is at a
513           premium. For these circumstances the operator inplace() is provided
514           which prevents the extra copy and allows the argument to be
515           modified. e.g.:
516
517            $x = log($array);          # $array unaffected
518            log( inplace($bigarray) ); # $bigarray changed in situ
519
520           WARNINGS:
521
522           1.  The usual caveats about duplicate references apply.
523
524           2.  Obviously when used with some functions which can not be
525               applied in situ (e.g. "convolve()") unexpected effects may
526               occur! We try to indicate "inplace()"-safe functions in the
527               documentation.
528
529           3.  Type conversions, such as"float()", may cause hidden copying.
530
531   Ensuring ndarrayness
532       If you have written a simple function and you don't want it to blow up
533       in your face if you pass it a simple number rather than a PDL variable.
534       Simply call the function topdl() first to make it safe. e.g.:
535
536        sub myfiddle { my $pdl = topdl(shift); $pdl->fiddle_foo(...); ... }
537
538       "topdl()" does NOT perform a copy if a pdl variable is passed - it just
539       falls through - which is obviously the desired behaviour. The routine
540       is not of course necessary in normal user defined functions which do
541       not care about internals.
542

AUTHOR

544       Copyright (C) Karl Glazebrook (kgb@aaoepp.aao.gov.au), Tuomas J. Lukka,
545       (lukka@husc.harvard.edu) and Christian Soeller
546       (c.soeller@auckland.ac.nz) 1997.  Commercial reproduction of this
547       documentation in a different format is forbidden.
548
549
550
551perl v5.34.0                      2021-08-16                     QUICKSTART(1)
Impressum