1Fsdb::Filter::dbrowuniqU(s3e)r Contributed Perl DocumentaFtsidobn::Filter::dbrowuniq(3)
2
6 dbrowuniq - eliminate adjacent rows with duplicate fields, maybe
7 counting
8
10 dbrowuniq [-cFLB] [uniquifying fields...]
11
13 Eliminate adjacent rows with duplicate fields, perhaps counting them.
14 Roughly equivalent to the Unix uniq command, but optionally only
15 operating on the specified fields.
16 By default, all columns must be unique. If column names are specified,
18 only those columns must be unique and the first row with those columns
19 is returned.
20 Dbrowuniq eliminates only identical rows that adjacent. If you want to
22 eliminate identical rows across the entirefile, you must make them
23 adajcent, perhaps by using dbsort on your uniquifying field. (That is,
24 the input with three lines a/b/a will produce three lines of output
25 with both a's, but if you dbsort it, it will become a/a/b and dbrowuniq
26 will output a/b.
27 By default, dbrowuniq outputs the first unique row. Optionally, with
29 "-L", it will output the last unique row, or with "-B" it outputs both
30 first and last. (This choice only matters when uniqueness is
31 determined by specific fields.)
32 dbrowuniq can also count how many unique, adjacent lines it finds with
34 "-c", with the count going to a new column (defaulting to "count").
35 Incremental counting, when the "count" column already exists, is
36 possible with "-I". With incremental counting, the existing count
37 column is summed.
38
40 -c or --count
41 Create a new column (count) which counts the number of times each
42 line occurred.
43 The new column is named by the "-N" argument, defaulting to
45 "count".
46 -N on --new-name
48 Specify the name of the count column, if any. Please specify the
49 type with the name, if desired (allowing one to pick sizes smaller
50 than the default quad, if desired). (Default is "count:q".)
51 -I on --incremental
53 Incremental counting. If the count column exists, it is assumed to
54 have a partial count and the count accumulates. If the count
55 column doesn't exist, it is created.
56 -L or --last
58 Output the last unique row only. By default, it outputs the first
59 unique row.
60 -F or --first
62 Output the first unique row only. (This output is the default.)
63 -B or --both
65 Output both the first and last unique rows.
66 This module also supports the standard fsdb options:
68 -d Enable debugging output.
70 -i or --input InputSource
72 Read from InputSource, typically a file name, or "-" for standard
73 input, or (if in Perl) a IO::Handle, Fsdb::IO or Fsdb::BoundedQueue
74 objects.
75 -o or --output OutputDestination
77 Write to OutputDestination, typically a file name, or "-" for
78 standard output, or (if in Perl) a IO::Handle, Fsdb::IO or
79 Fsdb::BoundedQueue objects.
80 --autorun or --noautorun
82 By default, programs process automatically, but Fsdb::Filter
83 objects in Perl do not run until you invoke the run() method. The
84 "--(no)autorun" option controls that behavior within Perl.
85 --header H
87 Use H as the full Fsdb header, rather than reading a header from
88 then input.
89 --help
91 Show help.
92 --man
94 Show full manual.
95
97 Input:
98 #fsdb event
99 _null_getpage+128
100 _null_getpage+128
101 _null_getpage+128
102 _null_getpage+128
103 _null_getpage+128
104 _null_getpage+128
105 _null_getpage+4
106 _null_getpage+4
107 _null_getpage+4
108 _null_getpage+4
109 _null_getpage+4
110 _null_getpage+4
111 # | /home/johnh/BIN/DB/dbcol event
112 # | /home/johnh/BIN/DB/dbsort event
113 Command:
115 cat data.fsdb | dbrowuniq -c
116 Output:
118 #fsdb event count
119 _null_getpage+128 6
120 _null_getpage+4 6
121 # 2 /home/johnh/BIN/DB/dbcol event
122 # | /home/johnh/BIN/DB/dbrowuniq -c
123
125 Retaining the last unique row as an example.
126 Input:
128 #fsdb event i
129 _null_getpage+128 10
130 _null_getpage+128 11
131 _null_getpage+128 12
132 _null_getpage+128 13
133 _null_getpage+128 14
134 _null_getpage+128 15
135 _null_getpage+4 16
136 _null_getpage+4 17
137 _null_getpage+4 18
138 _null_getpage+4 19
139 _null_getpage+4 20
140 _null_getpage+4 21
141 # | /home/johnh/BIN/DB/dbcol event
142 # | /home/johnh/BIN/DB/dbsort event
143 Command:
145 cat data.fsdb | dbrowuniq -c -L event
146 Output:
148 #fsdb event i count
149 _null_getpage+128 15 6
150 # | /home/johnh/BIN/DB/dbcol event
151 # | /home/johnh/BIN/DB/dbsort event
152 _null_getpage+4 21 6
153 # | dbrowuniq -c
154
156 Incremental counting.
157 Input:
159 #fsdb event count
160 _null_getpage+128 6
161 _null_getpage+128 6
162 _null_getpage+4 6
163 _null_getpage+4 6
164 # /home/johnh/BIN/DB/dbcol event
165 # | /home/johnh/BIN/DB/dbrowuniq -c
166 Command:
168 cat data.fsdb | dbrowuniq -I -c event
169 Output:
171 #fsdb event count
172 _null_getpage+128 12
173 _null_getpage+4 12
174 # /home/johnh/BIN/DB/dbcol event
175 # | /home/johnh/BIN/DB/dbrowuniq -c
176 # | dbrowuniq -I -c event
177
179 Fsdb.
180
182 new
183 $filter = new Fsdb::Filter::dbrowuniq(@arguments);
184 Create a new dbrowuniq object, taking command-line arguments.
186 set_defaults
188 $filter->set_defaults();
189 Internal: set up defaults.
191 parse_options
193 $filter->parse_options(@ARGV);
194 Internal: parse command-line arguments.
196 setup
198 $filter->setup();
199 Internal: setup, parse headers.
201 run
203 $filter->run();
204 Internal: run over each rows.
206
208 Copyright (C) 1997-2022 by John Heidemann <johnh@isi.edu>
209 This program is distributed under terms of the GNU general public
211 license, version 2. See the file COPYING with the distribution for
212 details.
213perl v5.36.0 2022-11-22 Fsdb::Filter::dbrowuniq(3)