1DATAFLOW(1) User Contributed Perl Documentation DATAFLOW(1)
2
3
4
6 PDL::Dataflow -- description of the dataflow philosophy
7
9 perldl> $a = zeroes(10);
10 perldl> $b = $a->slice("2:4:2");
11 perldl> $b ++;
12 perldl> print $a;
13 [0 0 1 0 1 0 0 0 0 0]
14
16 Dataflow is very experimental. Many features of it are disabled for
17 2.0, particularly families for one-directional dataflow. If you wish to
18 use one-directional dataflow for something, please contact the author
19 first and we'll work out how to make it functional again.
20
21 Two-directional dataflow (which implements ->slice() etc.) is fully
22 functional, however. Just about any function which returns some subset
23 of the values in some piddle will make a binding so that
24
25 $a = some piddle
26 $b = $a->slice("some parts");
27 $b->set(3,3,10);
28
29 also changes the corresponding element in $a. $b has become effectively
30 a window to some subelements of $a. You can also define your own rou‐
31 tines that do different types of subsets. If you don't want $b to be a
32 window to $a, you must do
33
34 $b = $a->slice("some parts")->copy;
35
36 The copying turns off all dataflow between the two piddles.
37
38 The difficulties with one-directional dataflow are related to sequences
39 like
40
41 $b = $a + 1;
42 $b ++;
43
44 where there are several possible outcomes and the semantics get a lit‐
45 tle murky.
46
48 Dataflow is new to PDL2.0. The basic philosophy behind dataflow is that
49
50 > $a = pdl 2,3,4;
51 > $b = $a * 2;
52 > print $b
53 [2 3 4]
54 > $a->set(0,5);
55 > print $b;
56 [10 3 4]
57
58 should work. It doesn't. It was considered that doing this might be too
59 confusing for novices and occasional users of the language. Therefore,
60 you need to explicitly turn on dataflow, so
61
62 > $a = pdl 2,3,4;
63 > $a->doflow();
64 > $b = $a * 2;
65 ...
66
67 produces the (un)expected result. The rest of this documents explains
68 various features and details of the dataflow implementation.
69
71 When you calculate something like the above
72
73 > $a = pdl 2,3,4;
74 > $a->doflow();
75 > $b = $a * 2;
76
77 nothing will have been calculated at this point. Even the memory for
78 the contents of $b has not been allocated. Only the command
79
80 > print $b
81
82 will actually cause $b to be calculated. This is important to bear in
83 mind when doing performance measurements and benchmarks as well as when
84 tracking errors.
85
86 There is an explanation for this behaviour: it may save cycles but more
87 importantly, imagine the following:
88
89 > $a = pdl 2,3,4;
90 > $b = pdl 5,6,7;
91 > $c = $a + $b;
92 ...
93 > $a->resize(4);
94 > $b->resize(4);
95 > print $c;
96
97 Now, if $c were evaluated between the two resizes, an error condition
98 of incompatible sizes would occur.
99
100 What happens in the current version is that resizing $a raises a flag
101 in $c: "PDL_PARENTDIMSCHANGED" and $b just raises the same flag again.
102 When $c is next evaluated, the flags are checked and it is found that a
103 recalculation is needed.
104
105 Of course, lazy evaluation can sometimes make debugging more painful
106 because errors may occur somewhere where you'd not expect them. A bet‐
107 ter stack trace for errors is in the works for PDL, probably so that
108 you can toggle a switch $PDL::traceevals and get a good trace of where
109 the error actually was.
110
112 This is one of the more intricate concepts of one-directional dataflow.
113 Consider the following code ($a and $b are pdls that have dataflow
114 enabled):
115
116 $c = $a + $b;
117 $e = $c + 1;
118 $d = $c->diagonal();
119 $d ++;
120 $f = $c + 1;
121
122 What should $e and $f contain now? What about when $a is changed and a
123 recalculation is triggered.
124
125 In order to make dataflow work like you'd expect, a rather strange con‐
126 cept must be introduced: families. Let us make a diagram:
127
128 a b
129 \ /
130 c
131 /⎪
132 / ⎪
133 e d
134
135 This is what PDL actually has in memory after the first three lines.
136 When $d is changed, we want $c to change but we don't want $e to change
137 because it already is on the graph. It may not be clear now why you
138 don't want it to change but if there were 40 lines of code between the
139 2nd and 4th lines, you would. So we need to make a copy of $c and $d:
140
141 a b
142 \ /
143 c' . . . c
144 /⎪ ⎪\
145 / ⎪ ⎪ \
146 e d' . . . d f
147
148 Notice that we primed the original c and d, because they do not corre‐
149 spond to the objects in $c and $d any more. Also, notice the dotted
150 lines between the two objects: when $a is changed and this diagram is
151 re-evaluated, $c really does get the value of c' with the diagonal
152 incremented.
153
154 To generalize on the above, whenever a piddle is mutated i.e. when its
155 actual *value* is forcibly changed (not just the reference:
156
157 $d = $d + 1
158
159 would produce a completely different result ($c and $d would not be
160 bound any more whereas
161
162 $d .= $d + 1
163
164 would yield the same as $d++), a "family" consisting of all other pid‐
165 dles joined to the mutated piddle by a two-way transformation is cre‐
166 ated and all those are copied.
167
168 All slices or transformations that simply select a subset of the origi‐
169 nal pdl are two-way. Matrix inverse should be. No arithmetic operators
170 are.
171
173 What you were told in the previous section is not quite true: the be‐
174 haviour described is not *always* what you want. Sometimes you would
175 probably like to have a data "source":
176
177 $a = pdl 2,3,4; $b = pdl 5,6,7;
178 $c = $a + $b;
179 line($c);
180
181 Now, if you know that $a is going to change and that you want its chil‐
182 dren to change with it, you can declare it into a data source (XXX
183 unimplemented in current version):
184
185 $a->datasource(1);
186
187 After this, $a++ or $a .= something will not create a new family but
188 will alter $a and cut its relation with its previous parents. All its
189 children will follow its current value.
190
191 So if $c in the previous section had been declared as a source, $e and
192 $f would remain equal.
193
195 A dataflow mechanism would not be very useful without the ability to
196 bind events onto changed data. Therefore, we provide such a mechanism:
197
198 > $a = pdl 2,3,4
199 > $b = $a + 1;
200 > $c = $b * 2;
201 > $c->bind( sub { print "A now: $a, C now: $c\n" } )
202 > PDL::dowhenidle();
203 A now: [2,3,4], C now: [6 8 10]
204 > $a->set(0,1);
205 > $a->set(1,1);
206 > PDL::dowhenidle();
207 A now: [1,1,4], C now: [4 4 10]
208
209 Notice how the callbacks only get called during PDL::dowhenidle. An
210 easy way to interface this to Perl event loop mechanisms (such as Tk)
211 is being planned.
212
213 There are many kinds of uses for this feature: self-updating graphs,
214 for instance.
215
216 Bla bla bla XXX more explanation
217
219 Dataflow as such is a fairly limited addition on top of Perl. To get a
220 more refined addition, the internals of perl need to be hacked a lit‐
221 tle. A true implementation would enable flow of everything, including
222
223 data
224 data size
225 datatype
226 operations
227
228 At the moment we only have the first two (hey, 50% in a couple of
229 months is not bad ;) but even this is useful by itself. However, espe‐
230 cially the last one is desirable since it would add the possibility of
231 flowing closures from place to place and would make many things more
232 flexible.
233
234 To get the rest working, the internals of dataflow probably need to be
235 changed to be a more general framework.
236
237 Additionally, it would be nice to be able to flow data in time, lucid-
238 like (so you could easily define all kinds of signal processing
239 things).
240
242 Copyright(C) 1997 Tuomas J. Lukka (lukka@fas.harvard.edu). Redistribu‐
243 tion in the same form is allowed provided that the copyright notice
244 stays intact but reprinting requires a permission from the author.
245
246
247
248perl v5.8.8 2003-05-21 DATAFLOW(1)