1DATAFLOW(1)           User Contributed Perl Documentation          DATAFLOW(1)
2
3
4

NAME

6       PDL::Dataflow -- description of the dataflow philosophy
7

SYNOPSIS

9               pdl> $a = zeroes(10);
10               pdl> $b = $a->slice("2:4:2");
11               pdl> $b ++;
12               pdl> print $a;
13               [0 0 1 0 1 0 0 0 0 0]
14

WARNING

16       Dataflow is very experimental. Many features of it are disabled for
17       2.0, particularly families for one-directional dataflow. If you wish to
18       use one-directional dataflow for something, please contact the author
19       first and we'll work out how to make it functional again.
20
21       Two-directional dataflow (which implements ->slice() etc.)  is fully
22       functional, however. Just about any function which returns some subset
23       of the values in some piddle will make a binding so that
24
25               $a = some piddle
26               $b = $a->slice("some parts");
27               $b->set(3,3,10);
28
29       also changes the corresponding element in $a. $b has become effectively
30       a window to some sub-elements of $a. You can also define your own
31       routines that do different types of subsets. If you don't want $b to be
32       a window to $a, you must do
33
34               $b = $a->slice("some parts")->copy;
35
36       The copying turns off all dataflow between the two piddles.
37
38       The difficulties with one-directional dataflow are related to sequences
39       like
40
41               $b = $a + 1;
42               $b ++;
43
44       where there are several possible outcomes and the semantics get a
45       little murky.
46

DESCRIPTION

48       Dataflow is new to PDL2.0. The basic philosophy behind dataflow is that
49
50               > $a = pdl 2,3,4;
51               > $b = $a * 2;
52               > print $b
53               [2 3 4]
54               > $a->set(0,5);
55               > print $b;
56               [10 3 4]
57
58       should work. It doesn't. It was considered that doing this might be too
59       confusing for novices and occasional users of the language.  Therefore,
60       you need to explicitly turn on dataflow, so
61
62               > $a = pdl 2,3,4;
63               > $a->doflow();
64               > $b = $a * 2;
65               ...
66
67       produces the unexpected result. The rest of this documents explains
68       various features and details of the dataflow implementation.
69

Lazy evaluation

71       When you calculate something like the above
72
73               > $a = pdl 2,3,4;
74               > $a->doflow();
75               > $b = $a * 2;
76
77       nothing will have been calculated at this point. Even the memory for
78       the contents of $b has not been allocated. Only the command
79
80               > print $b
81
82       will actually cause $b to be calculated. This is important to bear in
83       mind when doing performance measurements and benchmarks as well as when
84       tracking errors.
85
86       There is an explanation for this behaviour: it may save cycles but more
87       importantly, imagine the following:
88
89               > $a = pdl 2,3,4;
90               > $b = pdl 5,6,7;
91               > $c = $a + $b;
92               ...
93               > $a->resize(4);
94               > $b->resize(4);
95               > print $c;
96
97       Now, if $c were evaluated between the two resizes, an error condition
98       of incompatible sizes would occur.
99
100       What happens in the current version is that resizing $a raises a flag
101       in $c: "PDL_PARENTDIMSCHANGED" and $b just raises the same flag again.
102       When $c is next evaluated, the flags are checked and it is found that a
103       recalculation is needed.
104
105       Of course, lazy evaluation can sometimes make debugging more painful
106       because errors may occur somewhere where you'd not expect them.  A
107       better stack trace for errors is in the works for PDL, probably so that
108       you can toggle a switch $PDL::traceevals and get a good trace of where
109       the error actually was.
110

Families

112       This is one of the more intricate concepts of one-directional dataflow.
113       Consider the following code ($a and $b are pdls that have dataflow
114       enabled):
115
116               $c = $a + $b;
117               $e = $c + 1;
118               $d = $c->diagonal();
119               $d ++;
120               $f = $c + 1;
121
122       What should $e and $f contain now? What about when $a is changed and a
123       recalculation is triggered.
124
125       In order to make dataflow work like you'd expect, a rather strange
126       concept must be introduced: families. Let us make a diagram:
127
128               a   b
129                \ /
130                 c
131                /|
132               / |
133              e  d
134
135       This is what PDL actually has in memory after the first three lines.
136       When $d is changed, we want $c to change but we don't want $e to change
137       because it already is on the graph. It may not be clear now why you
138       don't want it to change but if there were 40 lines of code between the
139       2nd and 4th lines, you would. So we need to make a copy of $c and $d:
140
141               a   b
142                \ /
143                 c' . . . c
144                /|        |\
145               / |        | \
146              e  d' . . . d  f
147
148       Notice that we primed the original c and d, because they do not
149       correspond to the objects in $c and $d any more. Also, notice the
150       dotted lines between the two objects: when $a is changed and this
151       diagram is re-evaluated, $c really does get the value of c' with the
152       diagonal incremented.
153
154       To generalize on the above, whenever a piddle is mutated i.e.  when its
155       actual *value* is forcibly changed (not just the reference:
156
157               $d = $d + 1
158
159       would produce a completely different result ($c and $d would not be
160       bound any more whereas
161
162               $d .= $d + 1
163
164       would yield the same as $d++), a "family" consisting of all other
165       piddles joined to the mutated piddle by a two-way transformation is
166       created and all those are copied.
167
168       All slices or transformations that simply select a subset of the
169       original pdl are two-way. Matrix inverse should be. No arithmetic
170       operators are.
171

Sources

173       What you were told in the previous section is not quite true: the
174       behaviour described is not *always* what you want. Sometimes you would
175       probably like to have a data "source":
176
177               $a = pdl 2,3,4; $b = pdl 5,6,7;
178               $c = $a + $b;
179               line($c);
180
181       Now, if you know that $a is going to change and that you want its
182       children to change with it, you can declare it into a data source (XXX
183       unimplemented in current version):
184
185               $a->datasource(1);
186
187       After this, $a++ or $a .= something will not create a new family but
188       will alter $a and cut its relation with its previous parents.  All its
189       children will follow its current value.
190
191       So if $c in the previous section had been declared as a source, $e and
192       $f would remain equal.
193

Binding

195       A dataflow mechanism would not be very useful without the ability to
196       bind events onto changed data. Therefore, we provide such a mechanism:
197
198               > $a = pdl 2,3,4
199               > $b = $a + 1;
200               > $c = $b * 2;
201               > $c->bind( sub { print "A now: $a, C now: $c\n" } )
202               > PDL::dowhenidle();
203               A now: [2,3,4], C now: [6 8 10]
204               > $a->set(0,1);
205               > $a->set(1,1);
206               > PDL::dowhenidle();
207               A now: [1,1,4], C now: [4 4 10]
208
209       Notice how the callbacks only get called during PDL::dowhenidle.  An
210       easy way to interface this to Perl event loop mechanisms (such as Tk)
211       is being planned.
212
213       There are many kinds of uses for this feature: self-updating graphs,
214       for instance.
215
216       Blah blah blah XXX more explanation
217

Limitations

219       Dataflow as such is a fairly limited addition on top of Perl.  To get a
220       more refined addition, the internals of Perl need to be hacked a
221       little. A true implementation would enable flow of everything,
222       including
223
224       data
225       data size
226       datatype
227       operations
228
229       At the moment we only have the first two (hey, 50% in a couple of
230       months is not bad ;) but even this is useful by itself. However,
231       especially the last one is desirable since it would add the possibility
232       of flowing closures from place to place and would make many things more
233       flexible.
234
235       To get the rest working, the internals of dataflow probably need to be
236       changed to be a more general framework.
237
238       Additionally, it would be nice to be able to flow data in time, lucid-
239       like (so you could easily define all kinds of signal processing
240       things).
241

AUTHOR

243       Copyright(C) 1997 Tuomas J. Lukka (lukka@fas.harvard.edu).
244       Redistribution in the same form is allowed provided that the copyright
245       notice stays intact but reprinting requires a permission from the
246       author.
247
248
249
250perl v5.30.0                      2019-09-05                       DATAFLOW(1)
Impressum