1DBCOLMOVINGSTATS(1)   User Contributed Perl Documentation  DBCOLMOVINGSTATS(1)
2
3
4

NAME

6       dbcolmovingstats - compute moving statistics over a window of a column
7       of data
8

SYNOPSIS

10       dbcolmovingstats [-am] [-w WINDOW] [-e EmptyValue] [-k KEY] column
11

DESCRIPTION

13       Compute moving statistics over a COLUMN of data.  Records containing
14       non-numeric data are considered null do not contribute to the stats
15       (optionally they are treated as zeros with "-a").
16
17       Statitics are computed over a WINDOW of samples of data.
18
19       [In progress 2020-11-12, but not completed: Alternatively, if a key
20       column is given with "-k KEY", then a we treat the key column as a time
21       value and compute the time-weighted mean.]
22
23       Currently we compute mean and sample standard deviation.  (Note we only
24       compute sample standard deviation, not full population.)  Optionally,
25       with "-m" we also compute median.  (Currently there is no support for
26       generalized quantiles.)
27
28       Values before a sufficient number have been accumulated are given the
29       empty value (if specified with "-e").  If no empty value is given,
30       stats are computed on as many are possible if no empty value is
31       specified.
32
33       Dbcolmovingstats runs in O(1) memory, but must buffer a full window of
34       data.  Quantiles currently will repeatedly sort the window and so may
35       perform poorly with wide windows.
36

OPTIONS

38       -a or --include-non-numeric
39           Compute stats over all records (treat non-numeric records as zero
40           rather than just ignoring them).
41
42       -w or --window WINDOW
43           WINDOW of how many items to accumulate (defaults to 10).  (For
44           compatibility with fsdb-1.x, -n is also supported.)
45
46       -k or --key KEY
47           The KEY specifies a field that is used to evaluate the window---a
48           window must span at most this range of value so the key field.
49           (For example, if KEY is the time and window is 60, then enough
50           samples will be added to make at most 60s of observations.  With a
51           key, sampling can be irregular.)  If key is specified, we also
52           output a moving_n field  for how many samples are in each window.
53
54       -m or --median
55           Show median of the window in addition to mean.
56
57       -e E or --empty E
58           Give value E as the value for empty (null) records.  This null
59           value is then output before a full window is accumulated.
60
61       -f FORMAT or --format FORMAT
62           Specify a printf(3)-style format for output mean and standard
63           deviation.  Defaults to "%.5g".
64
65       Eventually we expect to support other options of dbcolstats.
66
67       This module also supports the standard fsdb options:
68
69       -d  Enable debugging output.
70
71       -i or --input InputSource
72           Read from InputSource, typically a file name, or "-" for standard
73           input, or (if in Perl) a IO::Handle, Fsdb::IO or Fsdb::BoundedQueue
74           objects.
75
76       -o or --output OutputDestination
77           Write to OutputDestination, typically a file name, or "-" for
78           standard output, or (if in Perl) a IO::Handle, Fsdb::IO or
79           Fsdb::BoundedQueue objects.
80
81       --autorun or --noautorun
82           By default, programs process automatically, but Fsdb::Filter
83           objects in Perl do not run until you invoke the run() method.  The
84           "--(no)autorun" option controls that behavior within Perl.
85
86       --help
87           Show help.
88
89       --man
90           Show full manual.
91

SAMPLE USAGE

93   Input:
94           #fsdb date  epoch count
95           19980201        886320000       6
96           19980202        886406400       8
97           19980203        886492800       19
98           19980204        886579200       53
99           19980205        886665600       20
100           19980206        886752000       18
101           19980207        886838400       5
102           19980208        886924800       9
103           19980209        887011200       22
104           19980210        887097600       22
105           19980211        887184000       36
106           19980212        887270400       26
107           19980213        887356800       23
108           19980214        887443200       6
109
110   Command:
111           cat data.fsdb | dbmovingstats -e - -w 4 count
112
113   Output:
114               #fsdb date epoch count moving_mean moving_stddev
115               19980201        886320000       6       -       -
116               19980202        886406400       8       -       -
117               19980203        886492800       19      -       -
118               19980204        886579200       53      21.5    21.764
119               19980205        886665600       20      25      19.442
120               19980206        886752000       18      27.5    17.02
121               19980207        886838400       5       24      20.445
122               19980208        886924800       9       13      7.1647
123               19980209        887011200       22      13.5    7.8528
124               19980210        887097600       22      14.5    8.8129
125               19980211        887184000       36      22.25   11.026
126               19980212        887270400       26      26.5    6.6081
127               19980213        887356800       23      26.75   6.3966
128               19980214        887443200       6       22.75   12.473
129               #   | dbcolmovingstats -e - -n 4 count
130

SEE ALSO

132       Fsdb.  dbcolstats.  dbmultistats.  dbrowdiff.
133

BUGS

135       Currently there is no support for generalized quantiles.
136
138       Copyright (C) 1991-2018 by John Heidemann <johnh@isi.edu>
139
140       This program is distributed under terms of the GNU general public
141       license, version 2.  See the file COPYING with the distribution for
142       details.
143
144
145
146perl v5.32.0                      2020-11-16               DBCOLMOVINGSTATS(1)
Impressum