1Text::RecordParser(3) User Contributed Perl DocumentationText::RecordParser(3)
2
6 Text::RecordParser - read record-oriented files
7
9 use Text::RecordParser;
10 # use default record (\n) and field (,) separators
12 my $p = Text::RecordParser->new( $file );
13 # or be explicit
15 my $p = Text::RecordParser->new({
16 filename => $file,
17 field_separator => "\t",
18 });
19 $p->filename('foo.csv');
21 # Split records on two newlines
23 $p->record_separator("\n\n");
24 # Split fields on tabs
26 $p->field_separator("\t");
27 # Skip lines beginning with hashes
29 $p->comment( qr/^#/ );
30 # Trim whitespace
32 $p->trim(1);
33 # Use the fields in the first line as column names
35 $p->bind_header;
36 # Get a list of the header fields (in order)
38 my @columns = $p->field_list;
39 # Extract a particular field from the next row
41 my ( $name, $age ) = $p->extract( qw[name age] );
42 # Return all the fields from the next row
44 my @fields = $p->fetchrow_array;
45 # Define a field alias
47 $p->set_field_alias( name => 'handle' );
48 # Return all the fields from the next row as a hashref
50 my $record = $p->fetchrow_hashref;
51 print $record->{'name'};
52 # or
53 print $record->{'handle'};
54 # Return the record as an object with fields as accessors
56 my $object = $p->fetchrow_object;
57 print $object->name; # or $object->handle;
58 # Get all data as arrayref of arrayrefs
60 my $data = $p->fetchall_arrayref;
61 # Get all data as arrayref of hashrefs
63 my $data = $p->fetchall_arrayref( { Columns => {} } );
64 # Get all data as hashref of hashrefs
66 my $data = $p->fetchall_hashref('name');
67
69 This module is for reading record-oriented data in a delimited text
70 file. The most common example have records separated by newlines and
71 fields separated by commas or tabs, but this module aims to provide a
72 consistent interface for handling sequential records in a file however
73 they may be delimited. Typically this data lists the fields in the
74 first line of the file, in which case you should call "bind_header" to
75 bind the field name (or not, and it will be called implicitly). If the
76 first line contains data, you can still bind your own field names via
77 "bind_fields". Either way, you can then use many methods to get at the
78 data as arrays or hashes.
79
81 new
82 This is the object constructor. It takes a hash (or hashref) of
83 arguments. Each argument can also be set through the method of the
84 same name.
85 • filename
87 The path to the file being read. If the filename is passed and the
89 fh is not, then it will open a filehandle on that file and sets
90 "fh" accordingly.
91 • comment
93 A compiled regular expression identifying comment lines that should
95 be skipped.
96 • data
98 The data to read.
100 • fh
102 The filehandle of the file to read.
104 • field_separator | fs
106 The field separator (default is comma).
108 • record_separator | rs
110 The record separator (default is newline).
112 • field_filter
114 A callback applied to all the fields as they are read.
116 • header_filter
118 A callback applied to the column names.
120 • trim
122 Boolean to enable trimming of leading and trailing whitespace from
124 fields (useful if splitting on whitespace only).
125 See methods for each argument name for more information.
127 Alternately, if you supply a single argument to "new", it will be
129 treated as the "filename" argument.
130 bind_fields
132 $p->bind_fields( qw[ name rank serial_number ] );
133 Takes an array of field names and memorizes the field positions for
135 later use. If the input file has no header line but you still wish to
136 retrieve the fields by name (or even if you want to call "bind_header"
137 and then give your own field names), simply pass in the an array of
138 field names you wish to use.
139 Pass in an empty array reference to unset:
141 $p->bind_field( [] ); # unsets fields
143 bind_header
145 $p->bind_header;
146 my $name = $p->extract('name');
147 Takes the fields from the next row under the cursor and assigns the
149 field names to the values. Usually you would call this immediately
150 after opening the file in order to bind the field names in the first
151 row.
152 comment
154 $p->comment( qr/^#/ ); # Perl-style comments
155 $p->comment( qr/^--/ ); # SQL-style comments
156 Takes a regex to apply to a record to see if it looks like a comment to
158 skip.
159 data
161 $p->data( $string );
162 $p->data( \$string );
163 $p->data( @lines );
164 $p->data( [$line1, $line2, $line3] );
165 $p->data( IO::File->new('<data') );
166 Allows a scalar, scalar reference, glob, array, or array reference as
168 the thing to read instead of a file handle.
169 It's not advised to pass a filehandle to "data" as it will read the
171 entire contents of the file rather than one line at a time if you set
172 it via "fh".
173 extract
175 my ( $foo, $bar, $baz ) = $p->extract( qw[ foo bar baz ] );
176 Extracts a list of fields out of the last row read. The field names
178 must correspond to the field names bound either via "bind_fields" or
179 "bind_header".
180 fetchrow_array
182 my @values = $p->fetchrow_array;
183 Reads a row from the file and returns an array or array reference of
185 the fields.
186 fetchrow_hashref
188 my $record = $p->fetchrow_hashref;
189 print "Name = ", $record->{'name'}, "\n";
190 Reads a line of the file and returns it as a hash reference. The keys
192 of the hashref are the field names bound via "bind_fields" or
193 "bind_header". If you do not bind fields prior to calling this method,
194 the "bind_header" method will be implicitly called for you.
195 fetchrow_object
197 while ( my $object = $p->fetchrow_object ) {
198 my $id = $object->id;
199 my $name = $object->naem; # <-- this will throw a runtime error
200 }
201 This will return the next data record as a Text::RecordParser::Object
203 object that has read-only accessor methods of the field names and any
204 aliases. This allows you to enforce field names, further helping
205 ensure that your code is reading the input file correctly. That is, if
206 you are using the "fetchrow_hashref" method to read each line, you may
207 misspell the hash key and introduce a bug in your code. With this
208 method, Perl will throw an error if you attempt to read a field not
209 defined in the file's headers. Additionally, any defined field aliases
210 will be created as additional accessor methods.
211 fetchall_arrayref
213 my $records = $p->fetchall_arrayref;
214 for my $record ( @$records ) {
215 print "Name = ", $record->[0], "\n";
216 }
217 my $records = $p->fetchall_arrayref( { Columns => {} } );
219 for my $record ( @$records ) {
220 print "Name = ", $record->{'name'}, "\n";
221 }
222 Like DBI's fetchall_arrayref, returns an arrayref of arrayrefs. Also
224 accepts optional "{ Columns => {} }" argument to return an arrayref of
225 hashrefs.
226 fetchall_hashref
228 my $records = $p->fetchall_hashref('id');
229 for my $id ( keys %$records ) {
230 my $record = $records->{ $id };
231 print "Name = ", $record->{'name'}, "\n";
232 }
233 Like DBI's fetchall_hashref, this returns a hash reference of hash
235 references. The keys of the top-level hashref are the field values of
236 the field argument you supply. The field name you supply can be a
237 field created by a "field_compute".
238 fh
240 open my $fh, '<', $file or die $!;
241 $p->fh( $fh );
242 Gets or sets the filehandle of the file being read.
244 field_compute
246 A callback applied to the fields identified by position (or field name
247 if "bind_fields" or "bind_header" was called).
248 The callback will be passed two arguments:
250 1. The current field
252 2. A reference to all the other fields, either as an array or hash
254 reference, depending on the method which you called.
255 If data looks like this:
257 parent children
259 Mike Greg,Peter,Bobby
260 Carol Marcia,Jane,Cindy
261 You could split the "children" field into an array reference with the
263 values like so:
264 $p->field_compute( 'children', sub { [ split /,/, shift() ] } );
266 The field position or name doesn't actually have to exist, which means
268 you could create new, computed fields on-the-fly. E.g., if you data
269 looks like this:
270 1,3,5
272 32,4,1
273 9,5,4
274 You could write a field_compute like this:
276 $p->field_compute( 3,
278 sub {
279 my ( $cur, $others ) = @_;
280 my $sum;
281 $sum += $_ for @$others;
282 return $sum;
283 }
284 );
285 Field "3" will be created as the sum of the other fields. This allows
287 you to further write:
288 my $data = $p->fetchall_arrayref;
290 for my $rec ( @$data ) {
291 print "$rec->[0] + $rec->[1] + $rec->[2] = $rec->[3]\n";
292 }
293 Prints:
295 1 + 3 + 5 = 9
297 32 + 4 + 1 = 37
298 9 + 5 + 4 = 18
299 field_filter
301 $p->field_filter( sub { $_ = shift; uc(lc($_)) } );
302 A callback which is applied to each field. The callback will be passed
304 the current value of the field. Whatever is passed back will become
305 the new value of the field. The above example capitalizes field
306 values. To unset the filter, pass in the empty string.
307 field_list
309 $p->bind_fields( qw[ foo bar baz ] );
310 my @fields = $p->field_list;
311 print join ', ', @fields; # prints "foo, bar, baz"
312 Returns the fields bound via "bind_fields" (or "bind_header").
314 field_positions
316 my %positions = $p->field_positions;
317 Returns a hash of the fields and their positions bound via
319 "bind_fields" (or "bind_header"). Mostly for internal use.
320 field_separator
322 $p->field_separator("\t"); # splits fields on tabs
323 $p->field_separator('::'); # splits fields on double colons
324 $p->field_separator(qr/\s+/); # splits fields on whitespace
325 my $sep = $p->field_separator; # returns the current separator
326 Gets and sets the token to use as the field delimiter. Regular
328 expressions can be specified using qr//. If not specified, it will
329 take a guess based on the filename extension ("comma" for ".txt,"
330 ".dat," or ".csv"; "tab" for ".tab"). The default is a comma.
331 filename
333 $p->filename('/path/to/file.dat');
334 Gets or sets the complete path to the file to be read. If a file is
336 already opened, then the handle on it will be closed and a new one
337 opened on the new file.
338 get_field_aliases
340 my @aliases = $p->get_field_aliases('name');
341 Allows you to define alternate names for fields, e.g., sometimes your
343 input file calls city "town" or "township," sometimes a file uses
344 "Moniker" instead of "name."
345 header_filter
347 $p->header_filter( sub { $_ = shift; s/\s+/_/g; lc $_ } );
348 A callback applied to column header names. The callback will be passed
350 the current value of the header. Whatever is returned will become the
351 new value of the header. The above example collapses spaces into a
352 single underscore and lowercases the letters. To unset a filter, pass
353 in the empty string.
354 record_separator
356 $p->record_separator("\n//\n");
357 $p->field_separator("\n");
358 Gets and sets the token to use as the record separator. The default is
360 a newline ("\n").
361 The above example would read a file that looks like this:
363 field1
365 field2
366 field3
367 //
368 data1
369 data2
370 data3
371 //
372 set_field_alias
374 $p->set_field_alias({
375 name => 'Moniker,handle', # comma-separated string
376 city => [ qw( town township ) ], # or anonymous arrayref
377 });
378 Allows you to define alternate names for fields, e.g., sometimes your
380 input file calls city "town" or "township," sometimes a file uses
381 "Moniker" instead of "name."
382 trim
384 my $trim_value = $p->trim(1);
385 Provide "true" argument to remove leading and trailing whitespace from
387 fields. Use a "false" argument to disable.
388
390 Ken Youens-Clark <kclark@cpan.org>
391
393 http://github.com/kyclark/text-recordparser
394
396 Thanks to the following:
397 • Benjamin Tilly
399 For Text::xSV, the inspirado for this module
401 • Tim Bunce et al.
403 For DBI, from which many of the methods were shamelessly stolen
405 • Tom Aldcroft
407 For contributing code to make it easy to parse whitespace-delimited
409 data
410 • Liya Ren
412 For catching the column-ordering error when parsing with "no-
414 headers"
415 • Sharon Wei
417 For catching bug in "extract" that sets up infinite loops
419 • Lars Thegler
421 For bug report on missing "script_files" arg in Build.PL
423
425 None known. Please use http://rt.cpan.org/ for reporting bugs.
426
428 Copyright (C) 2006-10 Ken Youens-Clark. All rights reserved.
429 This program is free software; you can redistribute it and/or modify it
431 under the terms of the GNU General Public License as published by the
432 Free Software Foundation; version 2.
433 This program is distributed in the hope that it will be useful, but
435 WITHOUT ANY WARRANTY; without even the implied warranty of
436 MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
437 General Public License for more details.
438perl v5.34.0 2022-01-28 Text::RecordParser(3)