1Fsdb::Filter::dbrowuniqU(s3e)r Contributed Perl DocumentaFtsidobn::Filter::dbrowuniq(3)
2
6 dbrowuniq - eliminate adjacent rows with duplicate fields, maybe
7 counting
8
10 dbrowuniq [-cFLB] [uniquifying fields...]
11
13 Eliminate adjacent rows with duplicate fields, perhaps counting them.
14 Roughly equivalent to the Unix uniq command, but optionally only
15 operating on the specified fields.
16 By default, all columns must be unique. If column names are specified,
18 only those columns must be unique and the first row with those columns
19 is returned.
20 Dbrowuniq eliminates only identical rows that adjacent. If you want to
22 eliminate identical rows across the entirefile, you must make them
23 adajcent, perhaps by using dbsort on your uniquifying field. (That is,
24 the input with three lines a/b/a will produce three lines of output
25 with both a's, but if you dbsort it, it will become a/a/b and dbrowuniq
26 will output a/b.
27 By default, dbrowuniq outputs the first unique row. Optionally, with
29 "-L", it will output the last unique row, or with "-B" it outputs both
30 first and last. (This choice only matters when uniqueness is
31 determined by specific fields.)
32 dbrowuniq can also count how many unique, adjacent lines it finds with
34 "-c", with the count going to a new column (defaulting to "count").
35 Incremental counting, when the "count" column already exists, is
36 possible with "-I". With incremental counting, the existing count
37 column is summed.
38
40 -c or --count
41 Create a new column (count) which counts the number of times each
42 line occurred.
43 The new column is named by the "-N" argument, defaulting to
45 "count".
46 -N on --new-name
48 Specify the name of the count column, if any. (Default is
49 "count".)
50 -I on --incremental
52 Incremental counting. If the count column exists, it is assumed to
53 have a partial count and the count accumulates. If the count
54 column doesn't exist, it is created.
55 -L or --last
57 Output the last unique row only. By default, it outputs the first
58 unique row.
59 -F or --first
61 Output the first unique row only. (This output is the default.)
62 -B or --both
64 Output both the first and last unique rows.
65 This module also supports the standard fsdb options:
67 -d Enable debugging output.
69 -i or --input InputSource
71 Read from InputSource, typically a file name, or "-" for standard
72 input, or (if in Perl) a IO::Handle, Fsdb::IO or Fsdb::BoundedQueue
73 objects.
74 -o or --output OutputDestination
76 Write to OutputDestination, typically a file name, or "-" for
77 standard output, or (if in Perl) a IO::Handle, Fsdb::IO or
78 Fsdb::BoundedQueue objects.
79 --autorun or --noautorun
81 By default, programs process automatically, but Fsdb::Filter
82 objects in Perl do not run until you invoke the run() method. The
83 "--(no)autorun" option controls that behavior within Perl.
84 --header H
86 Use H as the full Fsdb header, rather than reading a header from
87 then input.
88 --help
90 Show help.
91 --man
93 Show full manual.
94
96 Input:
97 #fsdb event
98 _null_getpage+128
99 _null_getpage+128
100 _null_getpage+128
101 _null_getpage+128
102 _null_getpage+128
103 _null_getpage+128
104 _null_getpage+4
105 _null_getpage+4
106 _null_getpage+4
107 _null_getpage+4
108 _null_getpage+4
109 _null_getpage+4
110 # | /home/johnh/BIN/DB/dbcol event
111 # | /home/johnh/BIN/DB/dbsort event
112 Command:
114 cat data.fsdb | dbrowuniq -c
115 Output:
117 #fsdb event count
118 _null_getpage+128 6
119 _null_getpage+4 6
120 # 2 /home/johnh/BIN/DB/dbcol event
121 # | /home/johnh/BIN/DB/dbrowuniq -c
122
124 Retaining the last unique row as an example.
125 Input:
127 #fsdb event i
128 _null_getpage+128 10
129 _null_getpage+128 11
130 _null_getpage+128 12
131 _null_getpage+128 13
132 _null_getpage+128 14
133 _null_getpage+128 15
134 _null_getpage+4 16
135 _null_getpage+4 17
136 _null_getpage+4 18
137 _null_getpage+4 19
138 _null_getpage+4 20
139 _null_getpage+4 21
140 # | /home/johnh/BIN/DB/dbcol event
141 # | /home/johnh/BIN/DB/dbsort event
142 Command:
144 cat data.fsdb | dbrowuniq -c -L event
145 Output:
147 #fsdb event i count
148 _null_getpage+128 15 6
149 # | /home/johnh/BIN/DB/dbcol event
150 # | /home/johnh/BIN/DB/dbsort event
151 _null_getpage+4 21 6
152 # | dbrowuniq -c
153
155 Incremental counting.
156 Input:
158 #fsdb event count
159 _null_getpage+128 6
160 _null_getpage+128 6
161 _null_getpage+4 6
162 _null_getpage+4 6
163 # /home/johnh/BIN/DB/dbcol event
164 # | /home/johnh/BIN/DB/dbrowuniq -c
165 Command:
167 cat data.fsdb | dbrowuniq -I -c event
168 Output:
170 #fsdb event count
171 _null_getpage+128 12
172 _null_getpage+4 12
173 # /home/johnh/BIN/DB/dbcol event
174 # | /home/johnh/BIN/DB/dbrowuniq -c
175 # | dbrowuniq -I -c event
176
178 Fsdb.
179
181 new
182 $filter = new Fsdb::Filter::dbrowuniq(@arguments);
183 Create a new dbrowuniq object, taking command-line arguments.
185 set_defaults
187 $filter->set_defaults();
188 Internal: set up defaults.
190 parse_options
192 $filter->parse_options(@ARGV);
193 Internal: parse command-line arguments.
195 setup
197 $filter->setup();
198 Internal: setup, parse headers.
200 run
202 $filter->run();
203 Internal: run over each rows.
205
207 Copyright (C) 1997-2020 by John Heidemann <johnh@isi.edu>
208 This program is distributed under terms of the GNU general public
210 license, version 2. See the file COPYING with the distribution for
211 details.
212perl v5.32.0 2020-11-16 Fsdb::Filter::dbrowuniq(3)