1PHILOSOPHY(1)         User Contributed Perl Documentation        PHILOSOPHY(1)
2
3
4

NAME

6       PDL::Philosophy -- Why did we write PDL?
7

DESCRIPTION

9       Some history from the creator of PDL, leading into the philosophy and
10       motivation behind this data language.  This is an attempt to summarize
11       some of the common spirit between pdl developers in order to answer the
12       question "Why PDL"?
13
14   The Start of PDL
15       "Why is it that we entertain the belief that for every purpose odd
16       numbers are the most effectual?" - Pliny the Elder
17
18       The PDL project began in February 1996, when I decided to experiment
19       with writing my own `Data Language'.  I am an astronomer. My day job
20       involves a lot of analysis of digital data accumulated on many nights
21       observing on telescopes around the world. Such data might for example
22       be images containing millions of pixels and thousands of images of
23       distant stars and galaxies. Or more abstrusely, many hundreds of
24       digital spectra revealing the secrets of the composition and properties
25       of these distant objects.
26
27       Obviously many astronomers have dealt with these problems before, and a
28       large amount of software has been constructed to facilitate their
29       analysis. However, like many of my colleagues, I was constantly
30       frustrated by the lack of generality and flexibility of these programs
31       and the difficulty of doing anything out of the ordinary quickly and
32       easily. What I wanted had a name: "Data Language", i.e. a language
33       which allowed the manipulation of large amounts of data with simple
34       arithmetic expressions.  In fact some commercial software worked like
35       this, and I was impressed with the capabilities but not with the price
36       tag. And I thought I could do better.
37
38       As a fairly computer literate astronomer (read "nerd" or "geek"
39       according to your local argot) I was very familiar with "Perl", a
40       computer language which now seems to fill the shelves of many
41       bookstores around the world.  I was impressed by its power and
42       flexibility, and especially its ease of use.  I had even explored the
43       depths of its internals and written an interface to allow graphics, the
44       ease with which I could then create charts and graphs, for my papers,
45       was refreshing.
46
47       Version 5 of Perl had just been released, and I was fascinated by the
48       new features available. Especially the support of arbitrary data
49       structures (or "objects" in modern parlance) and the ability to
50       "overload" operators - i.e. make mathematical symbols like  "+-*/" do
51       whatever you felt like.  It seemed to me it ought to be possible to
52       write an extension to Perl where I could play with my data in a general
53       way: for example using the maths operators manipulate whole images at
54       once.
55
56       One slow night at an observatory I thought I would try a little
57       experiment.  In a bored moment I fired up a text editor and started to
58       create a file called `PDL.xs' - a Perl extension module to manipulate
59       data vectors. A few hours later I actually had something half decent
60       working, where I could add two images in the Perl language, fast! This
61       was something I could not let rest, and it probably cost me one or two
62       scientific papers worth of productivity. A few weeks later the Perl
63       Data Language version 1.0 was born. It was a pretty bare infant: very
64       little was there apart from the basic arithmetic operators.  But
65       encouraged I made it available on the Internet to see what people
66       thought.
67
68       People were fairly critical - among the most vocal were Tuomas Lukka
69       and Christian Soeller. Unfortunately for them they were both Perl
70       enthusiasts too and soon found themselves improving my code to
71       implement all the features they thought PDL ought to have and I had
72       heinously neglected. PDL is a prime example of that modern phenomenon
73       of authoring large free software packages via the Internet. Large
74       numbers of people, most of whom have never met, have made contributions
75       ranging for core functionality to large modules to the smallest of bug
76       patches. PDL version 2.0 is now here (though it should perhaps have
77       been called version 10 to reflect the amount of growth in size and
78       functionality) and the phenomenon continues.  I firmly believe that PDL
79       is a great tool for tackling general problems of data analysis. It is
80       powerful, fast, easy to add too and freely available to anyone.  I wish
81       I had had it when I was a graduate student!  I hope you too will find
82       it of immense value, I hope it will save you from heaps of time and
83       frustration in solving complex problems. Of course it can't do
84       everything, but it provides the framework, the hammers and the nails
85       for building solutions without having to reinvent wheels or levers.
86
87       --- Karl Glazebook, the creator of PDL
88
89   Major ideas
90       The first tenet of our philosophy is the "free software" idea: software
91       being free has several advantages (less bugs because more people see
92       the code, you can have the source and port it to your own working
93       environment with you, ... and of course, that you don't need to pay
94       anything).
95
96       The second idea is a pet peeve of many: many languages like Matlab are
97       pretty well suited for their specific tasks but for a different
98       application, you need to change to an entirely different tool and
99       regear yourself mentally. Not to speak about doing an application that
100       does two things at once...  Because we use Perl, we have the power and
101       ease of Perl syntax, regular expressions, hash tables, etc. at our
102       fingertips at all times.  By extending an existing language, we start
103       from a much healthier base than languages like Matlab which have grown
104       into existence from a very small functionality at first and expanded
105       little by little, making things look badly planned. We stand by the
106       Perl sayings: "simple things should be simple but complicated things
107       should be possible" and "There is more than one way to do it"
108       (TIMTOWTDI).
109
110       The third idea is interoperability: we want to be able to use PDL to
111       drive as many tools as possible, we can connect to OpenGL or Mesa for
112       graphics or whatever. There isn't anything out there that's really
113       satisfactory as a tool and can do everything we want easily. And be
114       portable.
115
116       The fourth idea is related to "PDL::PP" and is Tuomas's personal
117       favorite: code should only specify as little as possible redundant
118       info. If you find yourself writing very similar-looking code much of
119       the time, all that code could probably be generated by a simple Perl
120       script. The PDL C preprocessor takes this to an extreme.
121
122   Minor goals and purposes
123       We want speed. Optimally, it should ultimately (e.g. with the Perl
124       compiler) be possible to compile "PDL::PP" subs to C and obtain the top
125       vectorized speeds on supercomputers. Also, we want to be able to
126       calculate things at near top speed from inside Perl, by using dataflow
127       to avoid memory allocation and deallocation (the overhead should
128       ultimately be only a little over one indirect function call plus couple
129       of ifs per function in the pipe).
130
131   Go on, try it!
132       Well, that's the philosophy behind PDL - speed, conciseness, free,
133       expandable, and integrated with the wide base of modules and libraries
134       that Perl provides. Feel free to download it, install it, run through
135       some of the tutorials and introductions and have a play with it.
136
137       Enjoy!
138

AUTHOR

140       Copyright(C) 1997 Tuomas J. Lukka (lukka@fas.harvard.edu).  Same terms
141       as the rest of PDL.
142
143       Added Karl Glazebrook (2001), contributions by Matthew Kenworthy
144
145
146
147perl v5.36.0                      2023-01-20                     PHILOSOPHY(1)
Impressum