1BADVALUES(1) User Contributed Perl Documentation BADVALUES(1)
2
6 PDL::BadValues - Discussion of bad value support in PDL
7
9 What are bad values and why should I bother with them?
10 Sometimes it's useful to be able to specify a certain value is 'bad' or
11 'missing'; for example CCDs used in astronomy produce 2D images which
12 are not perfect since certain areas contain invalid data due to
13 imperfections in the detector. Whilst PDL's powerful index routines
14 and all the complicated business with dataflow, slices, etc etc mean
15 that these regions can be ignored in processing, it's awkward to do. It
16 would be much easier to be able to say "$c = $x + $y" and leave all the
17 hassle to the computer.
18 If you're not interested in this, then you may (rightly) be concerned
20 with how this affects the speed of PDL, since the overhead of checking
21 for a bad value at each operation can be large. Because of this, the
22 code has been written to be as fast as possible - particularly when
23 operating on ndarrays which do not contain bad values. In fact, you
24 should notice essentially no speed difference when working with
25 ndarrays which do not contain bad values.
26 You may also ask 'well, my computer supports IEEE NaN, so I already
28 have this'. They are different things; a bad value signifies "leave
29 this out of processing", whereas NaN is the result of a mathematically-
30 invalid operation.
31 Many routines, such as "y=sin(x)", will propagate NaN's without the
33 user having to code differently, but routines such as "qsort", or
34 finding the median of an array, need to be re-coded to handle bad
35 values. For floating-point datatypes, "NaN" and "Inf" can be used to
36 flag bad values, but by default special values are used (Default bad
37 values).
38 There is one default bad value for each datatype, but as of PDL 2.040,
40 you can have different bad values for separate ndarrays of the same
41 type.
42 You can use "NaN" as the bad value for any floating-point type,
44 including complex.
45 A quick overview
47 pdl> $x = sequence(4,3);
48 pdl> p $x
49 [
50 [ 0 1 2 3]
51 [ 4 5 6 7]
52 [ 8 9 10 11]
53 ]
54 pdl> $x = $x->setbadif( $x % 3 == 2 )
55 pdl> p $x
56 [
57 [ 0 1 BAD 3]
58 [ 4 BAD 6 7]
59 [BAD 9 10 BAD]
60 ]
61 pdl> $x *= 3
62 pdl> p $x
63 [
64 [ 0 3 BAD 9]
65 [ 12 BAD 18 21]
66 [BAD 27 30 BAD]
67 ]
68 pdl> p $x->sum
69 120
70 "demo bad" within perldl or pdl2 gives a demonstration of some of the
72 things possible with bad values. These are also available on PDL's
73 web-site, at http://pdl.perl.org/demos/. See PDL::Bad for useful
74 routines for working with bad values and t/bad.t to see them in action.
75 To find out if a routine supports bad values, use the "badinfo" command
77 in perldl or pdl2 or the "-b" option to pdldoc.
78 Each ndarray contains a flag - accessible via "$pdl->badflag" - to say
80 whether there's any bad data present:
81 • If false/0, which means there's no bad data here, the code supplied
83 by the "Code" option to pp_def() is executed.
84 • If true/1, then this says there MAY be bad data in the ndarray, so
86 use the code in the "BadCode" option (assuming that the pp_def()
87 for this routine has been updated to have a BadCode key). You get
88 all the advantages of broadcasting, as with the "Code" option, but
89 it will run slower since you are going to have to handle the
90 presence of bad values.
91 If you create an ndarray, it will have its bad-value flag set to 0. To
93 change this, use "$pdl->badflag($new_bad_status)", where
94 $new_bad_status can be 0 or 1. When a routine creates an ndarray, its
95 bad-value flag will depend on the input ndarrays: unless over-ridden
96 (see the "CopyBadStatusCode" option to "pp_def"), the bad-value flag
97 will be set true if any of the input ndarrays contain bad values. To
98 check that an ndarray really contains bad data, use the "check_badflag"
99 method.
100 NOTE: propagation of the badflag
102 If you change the badflag of an ndarray, this change is propagated to
104 all the children of an ndarray, so
105 pdl> $x = zeroes(20,30);
107 pdl> $y = $x->slice('0:10,0:10');
108 pdl> $c = $y->slice(',(2)');
109 pdl> print ">>c: ", $c->badflag, "\n";
110 >>c: 0
111 pdl> $x->badflag(1);
112 pdl> print ">>c: ", $c->badflag, "\n";
113 >>c: 1
114 This is also propagated to the parents of an ndarray, so
116 pdl> print ">>a: ", $x->badflag, "\n";
118 >>a: 1
119 pdl> $c->badflag(0);
120 pdl> print ">>a: ", $x->badflag, "\n";
121 >>a: 0
122 There's also the issue of what happens if you change the badvalue of an
124 ndarray - should these propagate to children/parents (yes) or whether
125 you should only be able to change the badvalue at the 'top' level -
126 i.e. those ndarrays which do not have parents.
127 The orig_badvalue() method returns the compile-time value for a given
129 datatype. It works on ndarrays, PDL::Type objects, and numbers - eg
130 $pdl->orig_badvalue(), byte->orig_badvalue(), and orig_badvalue(4).
132 To get the current bad value, use the badvalue() method - it has the
134 same syntax as orig_badvalue().
135 To change the current bad value, supply the new number to badvalue - eg
137 $pdl->badvalue(2.3), byte->badvalue(2), badvalue(5,-3e34).
139 Note: the value is silently converted to the correct C type, and
141 returned - i.e. "byte->badvalue(-26)" returns 230 on my Linux machine.
142 Note that changes to the bad value are NOT propagated to previously-
144 created ndarrays - they will still have the bad flag set, but suddenly
145 the elements that were bad will become 'good', but containing the old
146 bad value. See discussion below.
147 Bad values and boolean operators
149 For those boolean operators in PDL::Ops, evaluation on a bad value
150 returns the bad value. This:
151 $mask = $img > $thresh;
153 correctly propagates bad values. This will omit any bad values, but
155 return a bad value if there are no good ones:
156 $bool = any( $img > $thresh );
158 As of 2.077, a bad value used as a boolean will throw an exception.
160 When using one of the 'projection' functions in PDL::Ufunc - such as
162 orover - bad values are skipped over (see the documentation of these
163 functions for the current handling of the case when all elements are
164 bad).
165
167 A new flag has been added to the state of an ndarray - "PDL_BADVAL". If
168 unset, then the ndarray does not contain bad values, and so all the
169 support code can be ignored. If set, it does not guarantee that bad
170 values are present, just that they should be checked for.
171 The "pdl_trans" structure has been extended to include an integer
173 value, "bvalflag", which acts as a switch to tell the code whether to
174 handle bad values or not. This value is set if any of the input
175 ndarrays have their "PDL_BADVAL" flag set (although this code can be
176 replaced by setting "FindBadStateCode" in pp_def).
177 Default bad values
179 The default bad values are now stored in a structure within the Core
180 PDL structure - "PDL.bvals" (eg Basic/Core/pdlcore.h.PL); see also
181 "typedef badvals" in Basic/Core/pdl.h.PL and the BOOT code of
182 Basic/Core/Core.xs.PL where the values are initialised to (hopefully)
183 sensible values. See "badvalue" in PDL::Bad and "orig_badvalue" in
184 PDL::Bad for read/write routines to the values.
185 The default/original bad values are set to the C type's maximum
187 (unsigned integers) or the minimum (floating-point and signed
188 integers).
189 How do I change a routine to handle bad values?
191 See "BadCode" in PDL::PP and "HandleBad" in PDL::PP.
192 If you have a routine that you want to be able to use as in-place, look
194 at the routines in bad.pd (or ops.pd) which use the "in-place" option
195 to see how the bad flag is propagated to children using the
196 "xxxBadStatusCode" options. I decided not to automate this as rules
197 would be a little complex, since not every in-place op will need to
198 propagate the badflag (eg unary functions).
199 This all means that you can change
201 Code => '$a() = $b() + $c();'
203 to
205 BadCode => 'if ( $ISBAD(b()) || $ISBAD(c()) ) {
207 $SETBAD(a());
208 } else {
209 $a() = $b() + $c();
210 }'
211 leaving Code as it is. PP::PDLCode will then create code something like
213 if ( __trans->bvalflag ) {
215 broadcastloop over BadCode
216 } else {
217 broadcastloop over Code
218 }
219
221 One of the strengths of PDL is its on-line documentation. The aim is to
222 use this system to provide information on how/if a routine supports bad
223 values: in many cases pp_def() contains all the information anyway, so
224 the function-writer doesn't need to do anything at all! For the cases
225 when this is not sufficient, there's the "BadDoc" option. For code
226 written at the Perl level - i.e. in a .pm file - use the "=for bad" pod
227 directive.
228 This information will be available via man/pod2man/html documentation.
230 It's also accessible from the "perldl" or "pdl2" shells - using the
231 "badinfo" command - and the "pdldoc" shell command - using the "-b"
232 option.
233
235 Copyright (C) Doug Burke (djburke@cpan.org), 2000, 2006.
236 The per-ndarray bad value support is by Heiko Klein (2006).
238perl v5.38.0 2023-07-21 BADVALUES(1)