1Fsdb::Filter::dbrowevalU(s3e)r Contributed Perl DocumentaFtsidobn::Filter::dbroweval(3)
2
6 dbroweval - evaluate code for each row of a fsdb file
7
9 dbroweval [-f CodeFile] code [code...]
10
12 Evaluate code for each row of the data.
13 Typical actions are things like reformatting and other data
15 transformations.
16 Code can include embedded column names preceded by underscores; these
18 result in the value of that column for the current row.
19 The values of the last row's columns are retrieved with _last_foo where
21 foo is the column name.
22 Even more perverse, _columname(N) is the value of the Nth column after
24 columnname [so _columnname(0) is the also the column's value.
25
27 -b CODE
28 Run CODE before reading any data (like awk BEGIN blocks).
29 -e CODE
31 Run CODE at the end of all data (like awk END blocks).
32 -f FILE
34 Read code from the FILE.
35 -n or --no-output
37 no output except for comments and what is in the provided code
38 -N or --no-output-even-comments
40 no output at all, except for what is in the provided code
41 -m or --manual-output
43 The user must setup output, allowing arbitrary comments. See
44 example 2 below for details.
45 -w or --warnings
47 Enable warnings in user supplied code.
48 --saveoutput $OUT_REF
50 Save output writer (for integration with other fsdb filters).
51 This module also supports the standard fsdb options:
53 -d Enable debugging output.
55 -i or --input InputSource
57 Read from InputSource, typically a file name, or "-" for standard
58 input, or (if in Perl) a IO::Handle, Fsdb::IO or Fsdb::BoundedQueue
59 objects.
60 -o or --output OutputDestination
62 Write to OutputDestination, typically a file name, or "-" for
63 standard output, or (if in Perl) a IO::Handle, Fsdb::IO or
64 Fsdb::BoundedQueue objects.
65 --autorun or --noautorun
67 By default, programs process automatically, but Fsdb::Filter
68 objects in Perl do not run until you invoke the run() method. The
69 "--(no)autorun" option controls that behavior within Perl.
70 --header H
72 Use H as the full Fsdb header, rather than reading a header from
73 then input.
74 --help
76 Show help.
77 --man
79 Show full manual.
80
82 Typically dbroweval outputs a line in the same schema for each input
83 line. For advanced usage, one can violate each of these assumptions.
84 Some fun:
86 omitting a line
88 Add the code "next row if ($your condition);"
89 outputting an extra line
91 Call "&$write_fastpath_sub($fref)". You may find $fref, the input
92 row, useful.
93 changing the schema
95 See the examples below in "Command 2: Changing the Schema"
96
98 Input:
99 #fsdb size mean stddev pct_rsd
100 1024 1.4962e+06 2.8497e+05 19.047
101 10240 5.0286e+06 6.0103e+05 11.952
102 102400 4.9216e+06 3.0939e+05 6.2863
103 # | dbsetheader size bw
104 # | /home/johnh/BIN/DB/dbmultistats size bw
105 # | /home/johnh/BIN/DB/dbcol size mean stddev pct_rsd
106 Command:
108 cat data.fsdb | dbroweval '_mean = sprintf("%8.0f", _mean); _stddev = sprintf("%8.0f", _stddev);'
109 Output:
111 #fsdb size mean stddev pct_rsd
112 1024 1496200 284970 19.047
113 10240 5028600 601030 11.952
114 102400 4921600 309390 6.2863
115 # | dbsetheader size bw
116 # | /home/johnh/BIN/DB/dbmultistats size bw
117 # | /home/johnh/BIN/DB/dbcol size mean stddev pct_rsd
118 # | /home/johnh/BIN/DB/dbroweval { _mean = sprintf("%8.0f", _mean); _stddev = sprintf("%8.0f", _stddev); }
119 Command 2: Changing the Schema
121 By default, dbroweval reads and writes the same format file. The
122 recommended method of adding and removing columns is to do so before or
123 after dbroweval. I.e.,
124 cat data.fsdb |
126 dbcolcreate divisible_by_ten |
127 dbroweval '_divisible_by_ten = (_size % 10 == 0);' |
128 dbrow '_divisible_by_ten == 1' |
129 dbcol size mean divisible_by_ten
130 Another approach is to use the "next row" command to skip output of a
132 row. I.e., the equivalent:
133 cat data.fsdb |
135 dbcolcreate divisible_by_ten |
136 dbroweval '_divisible_by_ten = (_size % 10 == 0); next row if (!_divisible_by_ten);' |
137 dbcol size mean divisible_by_ten
138 However, neither of these approachs work very well when the output is a
140 completely different schema.
141 The recommended method for schema-changing commands is to write a full
143 filter, but a full filter is a bit heavy weight. As an alternative,
144 one can use the "-m" option to request manual configuration of the
145 output, then use @out_args to define the output schema (it specifies
146 the "Fsdb::IO::Writer" arguments), and $ofref is the output row. It
147 may also reference <$in>, the input "Fsdb::IO::Reader" argument, and
148 <$fref> as an aref to the current line. Note that newly created
149 columns do not have underscore-names
150 Thus a third equivalent is:
152 cat data.fsdb | \
154 dbroweval -m -b '@out_args = ( -clone => $in, \
155 -cols => ($in->cols, divisible_by_ten); ' \
156 'my $div_by_10 = (_size % 10 == 0); \
157 $ofref = [ @$fref, $div_by_10 ] if ($div_by_ten);' |
158 dbcol size mean divisible_by_ten
159 or
161 cat data.fsdb | \
163 dbroweval -m -b '@out_args = ( -clone => $in, \
164 -cols => [qw(size mean divisible_by_ten)] ); ' \
165 'my $div_by_10 = (_size % 10 == 0); \
166 $ofref = [ _mean, _size, $div_by_10 ] if ($div_by_ten);'
167 Finally, one can write different a completely different schema,
169 although it's more work:
170 cat data.fsdb | \
172 dbroweval -m -b '@out_args = (-cols => [qw(size n)]);' \
173 '$ofref = [ _size, 1 ];'
174 writes different columns, and
176 cat data.fsdb | \
178 dbroweval -n -m -b '@out_args = (-cols => [qw(n)]); \
179 my $count = 0;' -e '$ofref = [ $count ];' '$count++;'
180 Is a fancy way to count lines.
182 The begin code block should setup @out_args to be the arguments to a
184 "Fsdb::IO::Writer::new" call, and whatever is in $ofref (if anything)
185 is written for each input line, and once at the end.
186 Command 3: Fun With Suppressing Output
188 The "-n" option suppresses default output. Thus, a simple equivalent
189 to tail -1 is:
190 dbroweval -n -e '$ofref = $lfref;'
192 Where $ofref is the output fields, which are copied from $lfref, the
194 hereby documented internal representation of the last row. Yes, this
195 is a bit unappetizing, but, in for a penny with $ofref, in for a pound.
196 Command 4: Extra Ouptut
198 Calling "&$write_fastpath_sub($fref)" will do extra output, so this
199 simple program will duplicate each line of input (one extra output,
200 plus one regular output for each line of input):
201 dbroweval '&$write_fastpath_sub($fref)'
203
205 Handling of code in files isn't very elegant.
206
208 Fsdb(3)
209
211 new
212 $filter = new Fsdb::Filter::dbroweval(@arguments);
213 set_defaults
215 $filter->set_defaults();
216 Internal: set up defaults.
218 _confirm_ending_semicolon
220 Not a method; but an internal routine to make sure code compiles.
221 parse_options
223 $filter->parse_options(@ARGV);
224 Internal: parse options
226 setup
228 $filter->setup();
229 Internal: setup, parse headers.
231 run
233 $filter->run();
234 Internal: run over all IO
236 finish
238 $filter->finish();
239 Internal: write trailer.
241 compute_program_log
243 $log = $filter->figure_program_log();
244 Override compute_program_log to do pretty-printed arguments.
246
248 Copyright (C) 1991-2018 by John Heidemann <johnh@isi.edu>
249 This program is distributed under terms of the GNU general public
251 license, version 2. See the file COPYING with the distribution for
252 details.
253perl v5.30.1 2020-01-30 Fsdb::Filter::dbroweval(3)