1Fsdb::Filter::dbrowevalU(s3e)r Contributed Perl DocumentaFtsidobn::Filter::dbroweval(3)
2
3
4

NAME

6       dbroweval - evaluate code for each row of a fsdb file
7

SYNOPSIS

9           dbroweval [-f CodeFile] code [code...]
10

DESCRIPTION

12       Evaluate code for each row of the data.
13
14       Typical actions are things like reformatting and other data
15       transformations.
16
17       Code can include embedded column names preceded by underscores; these
18       result in the value of that column for the current row.
19
20       The values of the last row's columns are retrieved with _last_foo where
21       foo is the column name.
22
23       Even more perverse, _columname(N) is the value of the Nth column after
24       columnname [so _columnname(0) is the also the column's value.
25

OPTIONS

27       -b CODE
28           Run CODE before reading any data (like awk BEGIN blocks).
29
30       -e CODE
31           Run CODE at the end of all data (like awk END blocks).
32
33       -f FILE
34           Read code from the FILE.
35
36       -n or --no-output
37           no output except for comments and what is in the provided code
38
39       -N or --no-output-even-comments
40           no output at all, except for what is in the provided code
41
42       -m or --manual-output
43           The user must setup output, allowing arbitrary comments.  See
44           example 2 below for details.
45
46       -w or --warnings
47           Enable warnings in user supplied code.
48
49       --saveoutput $OUT_REF
50           Save output writer (for integration with other fsdb filters).
51
52       This module also supports the standard fsdb options:
53
54       -d  Enable debugging output.
55
56       -i or --input InputSource
57           Read from InputSource, typically a file name, or "-" for standard
58           input, or (if in Perl) a IO::Handle, Fsdb::IO or Fsdb::BoundedQueue
59           objects.
60
61       -o or --output OutputDestination
62           Write to OutputDestination, typically a file name, or "-" for
63           standard output, or (if in Perl) a IO::Handle, Fsdb::IO or
64           Fsdb::BoundedQueue objects.
65
66       --autorun or --noautorun
67           By default, programs process automatically, but Fsdb::Filter
68           objects in Perl do not run until you invoke the run() method.  The
69           "--(no)autorun" option controls that behavior within Perl.
70
71       --header H
72           Use H as the full Fsdb header, rather than reading a header from
73           then input.
74
75       --help
76           Show help.
77
78       --man
79           Show full manual.
80

ADVANCED USAGE

82       Typically dbroweval outputs a line in the same schema for each input
83       line.  For advanced usage, one can violate each of these assumptions.
84
85       Some fun:
86
87       omitting a line
88           Add the code "next row if ($your condition);"
89
90       outputting an extra line
91           Call "&$write_fastpath_sub($fref)".  You may find $fref, the input
92           row, useful.
93
94       changing the schema
95           See the examples below in "Command 2: Changing the Schema"
96

SAMPLE USAGE

98   Input:
99           #fsdb      size    mean    stddev  pct_rsd
100           1024    1.4962e+06      2.8497e+05      19.047
101           10240   5.0286e+06      6.0103e+05      11.952
102           102400  4.9216e+06      3.0939e+05      6.2863
103           #  | dbsetheader size bw
104           #  | /home/johnh/BIN/DB/dbmultistats size bw
105           #  | /home/johnh/BIN/DB/dbcol size mean stddev pct_rsd
106
107   Command:
108           cat data.fsdb | dbroweval '_mean = sprintf("%8.0f", _mean); _stddev = sprintf("%8.0f", _stddev);'
109
110   Output:
111           #fsdb      size    mean    stddev  pct_rsd
112           1024     1496200          284970        19.047
113           10240    5028600          601030        11.952
114           102400   4921600          309390        6.2863
115           #  | dbsetheader size bw
116           #  | /home/johnh/BIN/DB/dbmultistats size bw
117           #  | /home/johnh/BIN/DB/dbcol size mean stddev pct_rsd
118           #  | /home/johnh/BIN/DB/dbroweval   { _mean = sprintf("%8.0f", _mean); _stddev = sprintf("%8.0f", _stddev); }
119
120   Command 2: Changing the Schema
121       By default, dbroweval reads and writes the same format file.  The
122       recommended method of adding and removing columns is to do so before or
123       after dbroweval.  I.e.,
124
125           cat data.fsdb |
126               dbcolcreate divisible_by_ten |
127               dbroweval '_divisible_by_ten = (_size % 10 == 0);' |
128               dbrow '_divisible_by_ten == 1' |
129               dbcol size mean divisible_by_ten
130
131       Another approach is to use the "next row" command to skip output of a
132       row.  I.e., the equivalent:
133
134           cat data.fsdb |
135               dbcolcreate divisible_by_ten |
136               dbroweval '_divisible_by_ten = (_size % 10 == 0); next row if (!_divisible_by_ten);' |
137               dbcol size mean divisible_by_ten
138
139       However, neither of these approachs work very well when the output is a
140       completely different schema.
141
142       The recommended method for schema-changing commands is to write a full
143       filter, but a full filter is a bit heavy weight.  As an alternative,
144       one can use the "-m" option to request manual configuration of the
145       output, then use @out_args to define the output schema (it specifies
146       the "Fsdb::IO::Writer" arguments), and $ofref is the output row.  It
147       may also reference <$in>, the input "Fsdb::IO::Reader" argument, and
148       <$fref> as an aref to the current line.  Note that newly created
149       columns do not have underscore-names
150
151       Thus a third equivalent is:
152
153           cat data.fsdb | \
154               dbroweval -m -b '@out_args = ( -clone => $in, \
155                        -cols => ($in->cols, divisible_by_ten); ' \
156                   'my $div_by_10 = (_size % 10 == 0); \
157                   $ofref = [ @$fref, $div_by_10 ] if ($div_by_ten);' |
158               dbcol size mean divisible_by_ten
159
160       or
161
162           cat data.fsdb | \
163               dbroweval -m -b '@out_args = ( -clone => $in, \
164                       -cols => [qw(size mean divisible_by_ten)] ); ' \
165                   'my $div_by_10 = (_size % 10 == 0);  \
166                   $ofref = [ _mean, _size, $div_by_10 ] if ($div_by_ten);'
167
168       Finally, one can write different a completely different schema,
169       although it's more work:
170
171           cat data.fsdb | \
172               dbroweval -m -b '@out_args = (-cols => [qw(size n)]);' \
173                   '$ofref = [ _size, 1 ];'
174
175       writes different columns, and
176
177           cat data.fsdb | \
178               dbroweval -n -m -b '@out_args = (-cols => [qw(n)]);  \
179                   my $count = 0;' -e '$ofref = [ $count ];' '$count++;'
180
181       Is a fancy way to count lines.
182
183       The begin code block should setup @out_args to be the arguments to a
184       "Fsdb::IO::Writer::new" call, and whatever is in $ofref (if anything)
185       is written for each input line, and once at the end.
186
187   Command 3: Fun With Suppressing Output
188       The "-n" option suppresses default output.  Thus, a simple equivalent
189       to tail -1 is:
190
191           dbroweval -n -e '$ofref = $lfref;'
192
193       Where $ofref is the output fields, which are copied from $lfref, the
194       hereby documented internal representation of the last row.  Yes, this
195       is a bit unappetizing, but, in for a penny with $ofref, in for a pound.
196
197   Command 4: Extra Ouptut
198       Calling "&$write_fastpath_sub($fref)" will do extra output, so this
199       simple program will duplicate each line of input (one extra output,
200       plus one regular output for each line of input):
201
202           dbroweval  '&$write_fastpath_sub($fref)'
203

BUGS

205       Handling of code in files isn't very elegant.
206

SEE ALSO

208       Fsdb(3)
209

CLASS FUNCTIONS

211   new
212           $filter = new Fsdb::Filter::dbroweval(@arguments);
213
214   set_defaults
215           $filter->set_defaults();
216
217       Internal: set up defaults.
218
219   _confirm_ending_semicolon
220       Not a method; but an internal routine to make sure code compiles.
221
222   parse_options
223           $filter->parse_options(@ARGV);
224
225       Internal: parse options
226
227   setup
228           $filter->setup();
229
230       Internal: setup, parse headers.
231
232   run
233           $filter->run();
234
235       Internal: run over all IO
236
237   finish
238           $filter->finish();
239
240       Internal: write trailer.
241
242   compute_program_log
243           $log = $filter->figure_program_log();
244
245       Override compute_program_log to do pretty-printed arguments.
246
248       Copyright (C) 1991-2018 by John Heidemann <johnh@isi.edu>
249
250       This program is distributed under terms of the GNU general public
251       license, version 2.  See the file COPYING with the distribution for
252       details.
253
254
255
256perl v5.32.0                      2020-11-16        Fsdb::Filter::dbroweval(3)
Impressum