1CSV_XS(3) User Contributed Perl Documentation CSV_XS(3)
2
6 Text::CSV_XS - comma-separated values manipulation routines
7
9 # Functional interface
10 use Text::CSV_XS qw( csv );
11 # Read whole file in memory
13 my $aoa = csv (in => "data.csv"); # as array of array
14 my $aoh = csv (in => "data.csv",
15 headers => "auto"); # as array of hash
16 # Write array of arrays as csv file
18 csv (in => $aoa, out => "file.csv", sep_char=> ";");
19 # Only show lines where "code" is odd
21 csv (in => "data.csv", filter => { code => sub { $_ % 2 }});
22 # Object interface
25 use Text::CSV_XS;
26 my @rows;
28 # Read/parse CSV
29 my $csv = Text::CSV_XS->new ({ binary => 1, auto_diag => 1 });
30 open my $fh, "<:encoding(utf8)", "test.csv" or die "test.csv: $!";
31 while (my $row = $csv->getline ($fh)) {
32 $row->[2] =~ m/pattern/ or next; # 3rd field should match
33 push @rows, $row;
34 }
35 close $fh;
36 # and write as CSV
38 open $fh, ">:encoding(utf8)", "new.csv" or die "new.csv: $!";
39 $csv->say ($fh, $_) for @rows;
40 close $fh or die "new.csv: $!";
41
43 Text::CSV_XS provides facilities for the composition and
44 decomposition of comma-separated values. An instance of the
45 Text::CSV_XS class will combine fields into a "CSV" string and parse a
46 "CSV" string into fields.
47 The module accepts either strings or files as input and support the
49 use of user-specified characters for delimiters, separators, and
50 escapes.
51 Embedded newlines
53 Important Note: The default behavior is to accept only ASCII
54 characters in the range from 0x20 (space) to 0x7E (tilde). This means
55 that the fields can not contain newlines. If your data contains
56 newlines embedded in fields, or characters above 0x7E (tilde), or
57 binary data, you must set "binary => 1" in the call to "new". To cover
58 the widest range of parsing options, you will always want to set
59 binary.
60 But you still have the problem that you have to pass a correct line to
62 the "parse" method, which is more complicated from the usual point of
63 usage:
64 my $csv = Text::CSV_XS->new ({ binary => 1, eol => $/ });
66 while (<>) { # WRONG!
67 $csv->parse ($_);
68 my @fields = $csv->fields ();
69 }
70 this will break, as the "while" might read broken lines: it does not
72 care about the quoting. If you need to support embedded newlines, the
73 way to go is to not pass "eol" in the parser (it accepts "\n", "\r",
74 and "\r\n" by default) and then
75 my $csv = Text::CSV_XS->new ({ binary => 1 });
77 open my $fh, "<", $file or die "$file: $!";
78 while (my $row = $csv->getline ($fh)) {
79 my @fields = @$row;
80 }
81 The old(er) way of using global file handles is still supported
83 while (my $row = $csv->getline (*ARGV)) { ... }
85 Unicode
87 Unicode is only tested to work with perl-5.8.2 and up.
88 See also "BOM".
90 The simplest way to ensure the correct encoding is used for in- and
92 output is by either setting layers on the filehandles, or setting the
93 "encoding" argument for "csv".
94 open my $fh, "<:encoding(UTF-8)", "in.csv" or die "in.csv: $!";
96 or
97 my $aoa = csv (in => "in.csv", encoding => "UTF-8");
98 open my $fh, ">:encoding(UTF-8)", "out.csv" or die "out.csv: $!";
100 or
101 csv (in => $aoa, out => "out.csv", encoding => "UTF-8");
102 On parsing (both for "getline" and "parse"), if the source is marked
104 being UTF8, then all fields that are marked binary will also be marked
105 UTF8.
106 On combining ("print" and "combine"): if any of the combining fields
108 was marked UTF8, the resulting string will be marked as UTF8. Note
109 however that all fields before the first field marked UTF8 and
110 contained 8-bit characters that were not upgraded to UTF8, these will
111 be "bytes" in the resulting string too, possibly causing unexpected
112 errors. If you pass data of different encoding, or you don't know if
113 there is different encoding, force it to be upgraded before you pass
114 them on:
115 $csv->print ($fh, [ map { utf8::upgrade (my $x = $_); $x } @data ]);
117 For complete control over encoding, please use Text::CSV::Encoded:
119 use Text::CSV::Encoded;
121 my $csv = Text::CSV::Encoded->new ({
122 encoding_in => "iso-8859-1", # the encoding comes into Perl
123 encoding_out => "cp1252", # the encoding comes out of Perl
124 });
125 $csv = Text::CSV::Encoded->new ({ encoding => "utf8" });
127 # combine () and print () accept *literally* utf8 encoded data
128 # parse () and getline () return *literally* utf8 encoded data
129 $csv = Text::CSV::Encoded->new ({ encoding => undef }); # default
131 # combine () and print () accept UTF8 marked data
132 # parse () and getline () return UTF8 marked data
133 BOM
135 BOM (or Byte Order Mark) handling is available only inside the
136 "header" method. This method supports the following encodings:
137 "utf-8", "utf-1", "utf-32be", "utf-32le", "utf-16be", "utf-16le",
138 "utf-ebcdic", "scsu", "bocu-1", and "gb-18030". See Wikipedia
139 <https://en.wikipedia.org/wiki/Byte_order_mark>.
140 If a file has a BOM, the easiest way to deal with that is
142 my $aoh = csv (in => $file, detect_bom => 1);
144 All records will be encoded based on the detected BOM.
146 This implies a call to the "header" method, which defaults to also
148 set the "column_names". So this is not the same as
149 my $aoh = csv (in => $file, headers => "auto");
151 which only reads the first record to set "column_names" but ignores
153 any meaning of possible present BOM.
154
156 While no formal specification for CSV exists, RFC 4180
157 <http://tools.ietf.org/html/rfc4180> (1) describes the common format
158 and establishes "text/csv" as the MIME type registered with the IANA.
159 RFC 7111 <http://tools.ietf.org/html/rfc7111> (2) adds fragments to
160 CSV.
161 Many informal documents exist that describe the "CSV" format. "How
163 To: The Comma Separated Value (CSV) File Format"
164 <http://www.creativyst.com/Doc/Articles/CSV/CSV01.htm> (3) provides an
165 overview of the "CSV" format in the most widely used applications and
166 explains how it can best be used and supported.
167 1) http://tools.ietf.org/html/rfc4180
169 2) http://tools.ietf.org/html/rfc7111
170 3) http://www.creativyst.com/Doc/Articles/CSV/CSV01.htm
171 The basic rules are as follows:
173 CSV is a delimited data format that has fields/columns separated by
175 the comma character and records/rows separated by newlines. Fields that
176 contain a special character (comma, newline, or double quote), must be
177 enclosed in double quotes. However, if a line contains a single entry
178 that is the empty string, it may be enclosed in double quotes. If a
179 field's value contains a double quote character it is escaped by
180 placing another double quote character next to it. The "CSV" file
181 format does not require a specific character encoding, byte order, or
182 line terminator format.
183 · Each record is a single line ended by a line feed (ASCII/"LF"=0x0A)
185 or a carriage return and line feed pair (ASCII/"CRLF"="0x0D 0x0A"),
186 however, line-breaks may be embedded.
187 · Fields are separated by commas.
189 · Allowable characters within a "CSV" field include 0x09 ("TAB") and
191 the inclusive range of 0x20 (space) through 0x7E (tilde). In binary
192 mode all characters are accepted, at least in quoted fields.
193 · A field within "CSV" must be surrounded by double-quotes to
195 contain a separator character (comma).
196 Though this is the most clear and restrictive definition, Text::CSV_XS
198 is way more liberal than this, and allows extension:
199 · Line termination by a single carriage return is accepted by default
201 · The separation-, escape-, and escape- characters can be any ASCII
203 character in the range from 0x20 (space) to 0x7E (tilde).
204 Characters outside this range may or may not work as expected.
205 Multibyte characters, like UTF "U+060C" (ARABIC COMMA), "U+FF0C"
206 (FULLWIDTH COMMA), "U+241B" (SYMBOL FOR ESCAPE), "U+2424" (SYMBOL
207 FOR NEWLINE), "U+FF02" (FULLWIDTH QUOTATION MARK), and "U+201C" (LEFT
208 DOUBLE QUOTATION MARK) (to give some examples of what might look
209 promising) work for newer versions of perl for "sep_char", and
210 "quote_char" but not for "escape_char".
211 If you use perl-5.8.2 or higher these three attributes are
213 utf8-decoded, to increase the likelihood of success. This way
214 "U+00FE" will be allowed as a quote character.
215 · A field in "CSV" must be surrounded by double-quotes to make an
217 embedded double-quote, represented by a pair of consecutive double-
218 quotes, valid. In binary mode you may additionally use the sequence
219 ""0" for representation of a NULL byte. Using 0x00 in binary mode is
220 just as valid.
221 · Several violations of the above specification may be lifted by
223 passing some options as attributes to the object constructor.
224
226 version
227 (Class method) Returns the current module version.
228 new
230 (Class method) Returns a new instance of class Text::CSV_XS. The
231 attributes are described by the (optional) hash ref "\%attr".
232 my $csv = Text::CSV_XS->new ({ attributes ... });
234 The following attributes are available:
236 eol
238 my $csv = Text::CSV_XS->new ({ eol => $/ });
240 $csv->eol (undef);
241 my $eol = $csv->eol;
242 The end-of-line string to add to rows for "print" or the record
244 separator for "getline".
245 When not passed in a parser instance, the default behavior is to
247 accept "\n", "\r", and "\r\n", so it is probably safer to not specify
248 "eol" at all. Passing "undef" or the empty string behave the same.
249 When not passed in a generating instance, records are not terminated
251 at all, so it is probably wise to pass something you expect. A safe
252 choice for "eol" on output is either $/ or "\r\n".
253 Common values for "eol" are "\012" ("\n" or Line Feed), "\015\012"
255 ("\r\n" or Carriage Return, Line Feed), and "\015" ("\r" or Carriage
256 Return). The "eol" attribute cannot exceed 7 (ASCII) characters.
257 If both $/ and "eol" equal "\015", parsing lines that end on only a
259 Carriage Return without Line Feed, will be "parse"d correct.
260 sep_char
262 my $csv = Text::CSV_XS->new ({ sep_char => ";" });
264 $csv->sep_char (";");
265 my $c = $csv->sep_char;
266 The char used to separate fields, by default a comma. (","). Limited
268 to a single-byte character, usually in the range from 0x20 (space) to
269 0x7E (tilde). When longer sequences are required, use "sep".
270 The separation character can not be equal to the quote character or to
272 the escape character.
273 See also "CAVEATS"
275 sep
277 my $csv = Text::CSV_XS->new ({ sep => "\N{FULLWIDTH COMMA}" });
279 $csv->sep (";");
280 my $sep = $csv->sep;
281 The chars used to separate fields, by default undefined. Limited to 8
283 bytes.
284 When set, overrules "sep_char". If its length is one byte it acts as
286 an alias to "sep_char".
287 See also "CAVEATS"
289 quote_char
291 my $csv = Text::CSV_XS->new ({ quote_char => "'" });
293 $csv->quote_char (undef);
294 my $c = $csv->quote_char;
295 The character to quote fields containing blanks or binary data, by
297 default the double quote character ("""). A value of undef suppresses
298 quote chars (for simple cases only). Limited to a single-byte
299 character, usually in the range from 0x20 (space) to 0x7E (tilde).
300 When longer sequences are required, use "quote".
301 "quote_char" can not be equal to "sep_char".
303 quote
305 my $csv = Text::CSV_XS->new ({ quote => "\N{FULLWIDTH QUOTATION MARK}" });
307 $csv->quote ("'");
308 my $quote = $csv->quote;
309 The chars used to quote fields, by default undefined. Limited to 8
311 bytes.
312 When set, overrules "quote_char". If its length is one byte it acts as
314 an alias to "quote_char".
315 See also "CAVEATS"
317 escape_char
319 my $csv = Text::CSV_XS->new ({ escape_char => "\\" });
321 $csv->escape_char (":");
322 my $c = $csv->escape_char;
323 The character to escape certain characters inside quoted fields.
325 This is limited to a single-byte character, usually in the range
326 from 0x20 (space) to 0x7E (tilde).
327 The "escape_char" defaults to being the double-quote mark ("""). In
329 other words the same as the default "quote_char". This means that
330 doubling the quote mark in a field escapes it:
331 "foo","bar","Escape ""quote mark"" with two ""quote marks""","baz"
333 If you change the "quote_char" without changing the
335 "escape_char", the "escape_char" will still be the double-quote
336 ("""). If instead you want to escape the "quote_char" by doubling it
337 you will need to also change the "escape_char" to be the same as what
338 you have changed the "quote_char" to.
339 Setting "escape_char" to <undef> or "" will disable escaping completely
341 and is greatly discouraged. This will also disable "escape_null".
342 The escape character can not be equal to the separation character.
344 binary
346 my $csv = Text::CSV_XS->new ({ binary => 1 });
348 $csv->binary (0);
349 my $f = $csv->binary;
350 If this attribute is 1, you may use binary characters in quoted
352 fields, including line feeds, carriage returns and "NULL" bytes. (The
353 latter could be escaped as ""0".) By default this feature is off.
354 If a string is marked UTF8, "binary" will be turned on automatically
356 when binary characters other than "CR" and "NL" are encountered. Note
357 that a simple string like "\x{00a0}" might still be binary, but not
358 marked UTF8, so setting "{ binary => 1 }" is still a wise option.
359 strict
361 my $csv = Text::CSV_XS->new ({ strict => 1 });
363 $csv->strict (0);
364 my $f = $csv->strict;
365 If this attribute is set to 1, any row that parses to a different
367 number of fields than the previous row will cause the parser to throw
368 error 2014.
369 formula_handling
371 formula
373 my $csv = Text::CSV_XS->new ({ formula => "none" });
375 $csv->formula ("none");
376 my $f = $csv->formula;
377 This defines the behavior of fields containing formulas. As formulas
379 are considered dangerous in spreadsheets, this attribute can define an
380 optional action to be taken if a field starts with an equal sign ("=").
381 For purpose of code-readability, this can also be written as
383 my $csv = Text::CSV_XS->new ({ formula_handling => "none" });
385 $csv->formula_handling ("none");
386 my $f = $csv->formula_handling;
387 Possible values for this attribute are
389 none
391 Take no specific action. This is the default.
392 $csv->formula ("none");
394 die
396 Cause the process to "die" whenever a leading "=" is encountered.
397 $csv->formula ("die");
399 croak
401 Cause the process to "croak" whenever a leading "=" is encountered.
402 (See Carp)
403 $csv->formula ("croak");
405 diag
407 Report position and content of the field whenever a leading "=" is
408 found. The value of the field is unchanged.
409 $csv->formula ("diag");
411 empty
413 Replace the content of fields that start with a "=" with the empty
414 string.
415 $csv->formula ("empty");
417 $csv->formula ("");
418 undef
420 Replace the content of fields that start with a "=" with "undef".
421 $csv->formula ("undef");
423 $csv->formula (undef);
424 All other values will give a warning and then fallback to "diag".
426 decode_utf8
428 my $csv = Text::CSV_XS->new ({ decode_utf8 => 1 });
430 $csv->decode_utf8 (0);
431 my $f = $csv->decode_utf8;
432 This attributes defaults to TRUE.
434 While parsing, fields that are valid UTF-8, are automatically set to
436 be UTF-8, so that
437 $csv->parse ("\xC4\xA8\n");
439 results in
441 PV("\304\250"\0) [UTF8 "\x{128}"]
443 Sometimes it might not be a desired action. To prevent those upgrades,
445 set this attribute to false, and the result will be
446 PV("\304\250"\0)
448 auto_diag
450 my $csv = Text::CSV_XS->new ({ auto_diag => 1 });
452 $csv->auto_diag (2);
453 my $l = $csv->auto_diag;
454 Set this attribute to a number between 1 and 9 causes "error_diag" to
456 be automatically called in void context upon errors.
457 In case of error "2012 - EOF", this call will be void.
459 If "auto_diag" is set to a numeric value greater than 1, it will "die"
461 on errors instead of "warn". If set to anything unrecognized, it will
462 be silently ignored.
463 Future extensions to this feature will include more reliable auto-
465 detection of "autodie" being active in the scope of which the error
466 occurred which will increment the value of "auto_diag" with 1 the
467 moment the error is detected.
468 diag_verbose
470 my $csv = Text::CSV_XS->new ({ diag_verbose => 1 });
472 $csv->diag_verbose (2);
473 my $l = $csv->diag_verbose;
474 Set the verbosity of the output triggered by "auto_diag". Currently
476 only adds the current input-record-number (if known) to the
477 diagnostic output with an indication of the position of the error.
478 blank_is_undef
480 my $csv = Text::CSV_XS->new ({ blank_is_undef => 1 });
482 $csv->blank_is_undef (0);
483 my $f = $csv->blank_is_undef;
484 Under normal circumstances, "CSV" data makes no distinction between
486 quoted- and unquoted empty fields. These both end up in an empty
487 string field once read, thus
488 1,"",," ",2
490 is read as
492 ("1", "", "", " ", "2")
494 When writing "CSV" files with either "always_quote" or "quote_empty"
496 set, the unquoted empty field is the result of an undefined value.
497 To enable this distinction when reading "CSV" data, the
498 "blank_is_undef" attribute will cause unquoted empty fields to be set
499 to "undef", causing the above to be parsed as
500 ("1", "", undef, " ", "2")
502 note that this is specifically important when loading "CSV" fields
504 into a database that allows "NULL" values, as the perl equivalent for
505 "NULL" is "undef" in DBI land.
506 empty_is_undef
508 my $csv = Text::CSV_XS->new ({ empty_is_undef => 1 });
510 $csv->empty_is_undef (0);
511 my $f = $csv->empty_is_undef;
512 Going one step further than "blank_is_undef", this attribute
514 converts all empty fields to "undef", so
515 1,"",," ",2
517 is read as
519 (1, undef, undef, " ", 2)
521 Note that this effects only fields that are originally empty, not
523 fields that are empty after stripping allowed whitespace. YMMV.
524 allow_whitespace
526 my $csv = Text::CSV_XS->new ({ allow_whitespace => 1 });
528 $csv->allow_whitespace (0);
529 my $f = $csv->allow_whitespace;
530 When this option is set to true, the whitespace ("TAB"'s and
532 "SPACE"'s) surrounding the separation character is removed when
533 parsing. If either "TAB" or "SPACE" is one of the three characters
534 "sep_char", "quote_char", or "escape_char" it will not be considered
535 whitespace.
536 Now lines like:
538 1 , "foo" , bar , 3 , zapp
540 are parsed as valid "CSV", even though it violates the "CSV" specs.
542 Note that all whitespace is stripped from both start and end of
544 each field. That would make it more than a feature to enable parsing
545 bad "CSV" lines, as
546 1, 2.0, 3, ape , monkey
548 will now be parsed as
550 ("1", "2.0", "3", "ape", "monkey")
552 even if the original line was perfectly acceptable "CSV".
554 allow_loose_quotes
556 my $csv = Text::CSV_XS->new ({ allow_loose_quotes => 1 });
558 $csv->allow_loose_quotes (0);
559 my $f = $csv->allow_loose_quotes;
560 By default, parsing unquoted fields containing "quote_char" characters
562 like
563 1,foo "bar" baz,42
565 would result in parse error 2034. Though it is still bad practice to
567 allow this format, we cannot help the fact that some vendors
568 make their applications spit out lines styled this way.
569 If there is really bad "CSV" data, like
571 1,"foo "bar" baz",42
573 or
575 1,""foo bar baz"",42
577 there is a way to get this data-line parsed and leave the quotes inside
579 the quoted field as-is. This can be achieved by setting
580 "allow_loose_quotes" AND making sure that the "escape_char" is not
581 equal to "quote_char".
582 allow_loose_escapes
584 my $csv = Text::CSV_XS->new ({ allow_loose_escapes => 1 });
586 $csv->allow_loose_escapes (0);
587 my $f = $csv->allow_loose_escapes;
588 Parsing fields that have "escape_char" characters that escape
590 characters that do not need to be escaped, like:
591 my $csv = Text::CSV_XS->new ({ escape_char => "\\" });
593 $csv->parse (qq{1,"my bar\'s",baz,42});
594 would result in parse error 2025. Though it is bad practice to allow
596 this format, this attribute enables you to treat all escape character
597 sequences equal.
598 allow_unquoted_escape
600 my $csv = Text::CSV_XS->new ({ allow_unquoted_escape => 1 });
602 $csv->allow_unquoted_escape (0);
603 my $f = $csv->allow_unquoted_escape;
604 A backward compatibility issue where "escape_char" differs from
606 "quote_char" prevents "escape_char" to be in the first position of a
607 field. If "quote_char" is equal to the default """ and "escape_char"
608 is set to "\", this would be illegal:
609 1,\0,2
611 Setting this attribute to 1 might help to overcome issues with
613 backward compatibility and allow this style.
614 always_quote
616 my $csv = Text::CSV_XS->new ({ always_quote => 1 });
618 $csv->always_quote (0);
619 my $f = $csv->always_quote;
620 By default the generated fields are quoted only if they need to be.
622 For example, if they contain the separator character. If you set this
623 attribute to 1 then all defined fields will be quoted. ("undef" fields
624 are not quoted, see "blank_is_undef"). This makes it quite often easier
625 to handle exported data in external applications. (Poor creatures who
626 are better to use Text::CSV_XS. :)
627 quote_space
629 my $csv = Text::CSV_XS->new ({ quote_space => 1 });
631 $csv->quote_space (0);
632 my $f = $csv->quote_space;
633 By default, a space in a field would trigger quotation. As no rule
635 exists this to be forced in "CSV", nor any for the opposite, the
636 default is true for safety. You can exclude the space from this
637 trigger by setting this attribute to 0.
638 quote_empty
640 my $csv = Text::CSV_XS->new ({ quote_empty => 1 });
642 $csv->quote_empty (0);
643 my $f = $csv->quote_empty;
644 By default the generated fields are quoted only if they need to be.
646 An empty (defined) field does not need quotation. If you set this
647 attribute to 1 then empty defined fields will be quoted. ("undef"
648 fields are not quoted, see "blank_is_undef"). See also "always_quote".
649 quote_binary
651 my $csv = Text::CSV_XS->new ({ quote_binary => 1 });
653 $csv->quote_binary (0);
654 my $f = $csv->quote_binary;
655 By default, all "unsafe" bytes inside a string cause the combined
657 field to be quoted. By setting this attribute to 0, you can disable
658 that trigger for bytes >= 0x7F.
659 escape_null
661 my $csv = Text::CSV_XS->new ({ escape_null => 1 });
663 $csv->escape_null (0);
664 my $f = $csv->escape_null;
665 By default, a "NULL" byte in a field would be escaped. This option
667 enables you to treat the "NULL" byte as a simple binary character in
668 binary mode (the "{ binary => 1 }" is set). The default is true. You
669 can prevent "NULL" escapes by setting this attribute to 0.
670 When the "escape_char" attribute is set to undefined, this attribute
672 will be set to false.
673 The default setting will encode "=\x00=" as
675 "="0="
677 With "escape_null" set, this will result in
679 "=\x00="
681 The default when using the "csv" function is "false".
683 For backward compatibility reasons, the deprecated old name
685 "quote_null" is still recognized.
686 keep_meta_info
688 my $csv = Text::CSV_XS->new ({ keep_meta_info => 1 });
690 $csv->keep_meta_info (0);
691 my $f = $csv->keep_meta_info;
692 By default, the parsing of input records is as simple and fast as
694 possible. However, some parsing information - like quotation of the
695 original field - is lost in that process. Setting this flag to true
696 enables retrieving that information after parsing with the methods
697 "meta_info", "is_quoted", and "is_binary" described below. Default is
698 false for performance.
699 If you set this attribute to a value greater than 9, than you can
701 control output quotation style like it was used in the input of the the
702 last parsed record (unless quotation was added because of other
703 reasons).
704 my $csv = Text::CSV_XS->new ({
706 binary => 1,
707 keep_meta_info => 1,
708 quote_space => 0,
709 });
710 my $row = $csv->parse (q{1,,"", ," ",f,"g","h""h",help,"help"});
712 $csv->print (*STDOUT, \@row);
714 # 1,,, , ,f,g,"h""h",help,help
715 $csv->keep_meta_info (11);
716 $csv->print (*STDOUT, \@row);
717 # 1,,"", ," ",f,"g","h""h",help,"help"
718 undef_str
720 my $csv = Text::CSV_XS->new ({ undef_str => "\\N" });
722 $csv->undef_str (undef);
723 my $s = $csv->undef_str;
724 This attribute optionally defines the output of undefined fields. The
726 value passed is not changed at all, so if it needs quotation, the
727 quotation needs to be included in the value of the attribute. Use with
728 caution, as passing a value like ",",,,,""" will for sure mess up
729 your output. The default for this attribute is "undef", meaning no
730 special treatment.
731 This attribute is useful when exporting CSV data to be imported in
733 custom loaders, like for MySQL, that recognize special sequences for
734 "NULL" data.
735 This attribute has no meaning when parsing CSV data.
737 verbatim
739 my $csv = Text::CSV_XS->new ({ verbatim => 1 });
741 $csv->verbatim (0);
742 my $f = $csv->verbatim;
743 This is a quite controversial attribute to set, but makes some hard
745 things possible.
746 The rationale behind this attribute is to tell the parser that the
748 normally special characters newline ("NL") and Carriage Return ("CR")
749 will not be special when this flag is set, and be dealt with as being
750 ordinary binary characters. This will ease working with data with
751 embedded newlines.
752 When "verbatim" is used with "getline", "getline" auto-"chomp"'s
754 every line.
755 Imagine a file format like
757 M^^Hans^Janssen^Klas 2\n2A^Ja^11-06-2007#\r\n
759 where, the line ending is a very specific "#\r\n", and the sep_char is
761 a "^" (caret). None of the fields is quoted, but embedded binary
762 data is likely to be present. With the specific line ending, this
763 should not be too hard to detect.
764 By default, Text::CSV_XS' parse function is instructed to only know
766 about "\n" and "\r" to be legal line endings, and so has to deal with
767 the embedded newline as a real "end-of-line", so it can scan the next
768 line if binary is true, and the newline is inside a quoted field. With
769 this option, we tell "parse" to parse the line as if "\n" is just
770 nothing more than a binary character.
771 For "parse" this means that the parser has no more idea about line
773 ending and "getline" "chomp"s line endings on reading.
774 types
776 A set of column types; the attribute is immediately passed to the
778 "types" method.
779 callbacks
781 See the "Callbacks" section below.
783 accessors
785 To sum it up,
787 $csv = Text::CSV_XS->new ();
789 is equivalent to
791 $csv = Text::CSV_XS->new ({
793 eol => undef, # \r, \n, or \r\n
794 sep_char => ',',
795 sep => undef,
796 quote_char => '"',
797 quote => undef,
798 escape_char => '"',
799 binary => 0,
800 decode_utf8 => 1,
801 auto_diag => 0,
802 diag_verbose => 0,
803 blank_is_undef => 0,
804 empty_is_undef => 0,
805 allow_whitespace => 0,
806 allow_loose_quotes => 0,
807 allow_loose_escapes => 0,
808 allow_unquoted_escape => 0,
809 always_quote => 0,
810 quote_empty => 0,
811 quote_space => 1,
812 escape_null => 1,
813 quote_binary => 1,
814 keep_meta_info => 0,
815 strict => 0,
816 formula => 0,
817 verbatim => 0,
818 undef_str => undef,
819 types => undef,
820 callbacks => undef,
821 });
822 For all of the above mentioned flags, an accessor method is available
824 where you can inquire the current value, or change the value
825 my $quote = $csv->quote_char;
827 $csv->binary (1);
828 It is not wise to change these settings halfway through writing "CSV"
830 data to a stream. If however you want to create a new stream using the
831 available "CSV" object, there is no harm in changing them.
832 If the "new" constructor call fails, it returns "undef", and makes
834 the fail reason available through the "error_diag" method.
835 $csv = Text::CSV_XS->new ({ ecs_char => 1 }) or
837 die "".Text::CSV_XS->error_diag ();
838 "error_diag" will return a string like
840 "INI - Unknown attribute 'ecs_char'"
842 known_attributes
844 @attr = Text::CSV_XS->known_attributes;
845 @attr = Text::CSV_XS::known_attributes;
846 @attr = $csv->known_attributes;
847 This method will return an ordered list of all the supported
849 attributes as described above. This can be useful for knowing what
850 attributes are valid in classes that use or extend Text::CSV_XS.
851 print
853 $status = $csv->print ($fh, $colref);
854 Similar to "combine" + "string" + "print", but much more efficient.
856 It expects an array ref as input (not an array!) and the resulting
857 string is not really created, but immediately written to the $fh
858 object, typically an IO handle or any other object that offers a
859 "print" method.
860 For performance reasons "print" does not create a result string, so
862 all "string", "status", "fields", and "error_input" methods will return
863 undefined information after executing this method.
864 If $colref is "undef" (explicit, not through a variable argument) and
866 "bind_columns" was used to specify fields to be printed, it is
867 possible to make performance improvements, as otherwise data would have
868 to be copied as arguments to the method call:
869 $csv->bind_columns (\($foo, $bar));
871 $status = $csv->print ($fh, undef);
872 A short benchmark
874 my @data = ("aa" .. "zz");
876 $csv->bind_columns (\(@data));
877 $csv->print ($fh, [ @data ]); # 11800 recs/sec
879 $csv->print ($fh, \@data ); # 57600 recs/sec
880 $csv->print ($fh, undef ); # 48500 recs/sec
881 say
883 $status = $csv->say ($fh, $colref);
884 Like "print", but "eol" defaults to "$\".
886 print_hr
888 $csv->print_hr ($fh, $ref);
889 Provides an easy way to print a $ref (as fetched with "getline_hr")
891 provided the column names are set with "column_names".
892 It is just a wrapper method with basic parameter checks over
894 $csv->print ($fh, [ map { $ref->{$_} } $csv->column_names ]);
896 combine
898 $status = $csv->combine (@fields);
899 This method constructs a "CSV" record from @fields, returning success
901 or failure. Failure can result from lack of arguments or an argument
902 that contains an invalid character. Upon success, "string" can be
903 called to retrieve the resultant "CSV" string. Upon failure, the
904 value returned by "string" is undefined and "error_input" could be
905 called to retrieve the invalid argument.
906 string
908 $line = $csv->string ();
909 This method returns the input to "parse" or the resultant "CSV"
911 string of "combine", whichever was called more recently.
912 getline
914 $colref = $csv->getline ($fh);
915 This is the counterpart to "print", as "parse" is the counterpart to
917 "combine": it parses a row from the $fh handle using the "getline"
918 method associated with $fh and parses this row into an array ref.
919 This array ref is returned by the function or "undef" for failure.
920 When $fh does not support "getline", you are likely to hit errors.
921 When fields are bound with "bind_columns" the return value is a
923 reference to an empty list.
924 The "string", "fields", and "status" methods are meaningless again.
926 getline_all
928 $arrayref = $csv->getline_all ($fh);
929 $arrayref = $csv->getline_all ($fh, $offset);
930 $arrayref = $csv->getline_all ($fh, $offset, $length);
931 This will return a reference to a list of getline ($fh) results. In
933 this call, "keep_meta_info" is disabled. If $offset is negative, as
934 with "splice", only the last "abs ($offset)" records of $fh are taken
935 into consideration.
936 Given a CSV file with 10 lines:
938 lines call
940 ----- ---------------------------------------------------------
941 0..9 $csv->getline_all ($fh) # all
942 0..9 $csv->getline_all ($fh, 0) # all
943 8..9 $csv->getline_all ($fh, 8) # start at 8
944 - $csv->getline_all ($fh, 0, 0) # start at 0 first 0 rows
945 0..4 $csv->getline_all ($fh, 0, 5) # start at 0 first 5 rows
946 4..5 $csv->getline_all ($fh, 4, 2) # start at 4 first 2 rows
947 8..9 $csv->getline_all ($fh, -2) # last 2 rows
948 6..7 $csv->getline_all ($fh, -4, 2) # first 2 of last 4 rows
949 getline_hr
951 The "getline_hr" and "column_names" methods work together to allow you
952 to have rows returned as hashrefs. You must call "column_names" first
953 to declare your column names.
954 $csv->column_names (qw( code name price description ));
956 $hr = $csv->getline_hr ($fh);
957 print "Price for $hr->{name} is $hr->{price} EUR\n";
958 "getline_hr" will croak if called before "column_names".
960 Note that "getline_hr" creates a hashref for every row and will be
962 much slower than the combined use of "bind_columns" and "getline" but
963 still offering the same ease of use hashref inside the loop:
964 my @cols = @{$csv->getline ($fh)};
966 $csv->column_names (@cols);
967 while (my $row = $csv->getline_hr ($fh)) {
968 print $row->{price};
969 }
970 Could easily be rewritten to the much faster:
972 my @cols = @{$csv->getline ($fh)};
974 my $row = {};
975 $csv->bind_columns (\@{$row}{@cols});
976 while ($csv->getline ($fh)) {
977 print $row->{price};
978 }
979 Your mileage may vary for the size of the data and the number of rows.
981 With perl-5.14.2 the comparison for a 100_000 line file with 14 rows:
982 Rate hashrefs getlines
984 hashrefs 1.00/s -- -76%
985 getlines 4.15/s 313% --
986 getline_hr_all
988 $arrayref = $csv->getline_hr_all ($fh);
989 $arrayref = $csv->getline_hr_all ($fh, $offset);
990 $arrayref = $csv->getline_hr_all ($fh, $offset, $length);
991 This will return a reference to a list of getline_hr ($fh) results.
993 In this call, "keep_meta_info" is disabled.
994 parse
996 $status = $csv->parse ($line);
997 This method decomposes a "CSV" string into fields, returning success
999 or failure. Failure can result from a lack of argument or the given
1000 "CSV" string is improperly formatted. Upon success, "fields" can be
1001 called to retrieve the decomposed fields. Upon failure calling "fields"
1002 will return undefined data and "error_input" can be called to
1003 retrieve the invalid argument.
1004 You may use the "types" method for setting column types. See "types"'
1006 description below.
1007 The $line argument is supposed to be a simple scalar. Everything else
1009 is supposed to croak and set error 1500.
1010 fragment
1012 This function tries to implement RFC7111 (URI Fragment Identifiers for
1013 the text/csv Media Type) - http://tools.ietf.org/html/rfc7111
1014 my $AoA = $csv->fragment ($fh, $spec);
1016 In specifications, "*" is used to specify the last item, a dash ("-")
1018 to indicate a range. All indices are 1-based: the first row or
1019 column has index 1. Selections can be combined with the semi-colon
1020 (";").
1021 When using this method in combination with "column_names", the
1023 returned reference will point to a list of hashes instead of a list
1024 of lists. A disjointed cell-based combined selection might return
1025 rows with different number of columns making the use of hashes
1026 unpredictable.
1027 $csv->column_names ("Name", "Age");
1029 my $AoH = $csv->fragment ($fh, "col=3;8");
1030 If the "after_parse" callback is active, it is also called on every
1032 line parsed and skipped before the fragment.
1033 row
1035 row=4
1036 row=5-7
1037 row=6-*
1038 row=1-2;4;6-*
1039 col
1041 col=2
1042 col=1-3
1043 col=4-*
1044 col=1-2;4;7-*
1045 cell
1047 In cell-based selection, the comma (",") is used to pair row and
1048 column
1049 cell=4,1
1051 The range operator ("-") using "cell"s can be used to define top-left
1053 and bottom-right "cell" location
1054 cell=3,1-4,6
1056 The "*" is only allowed in the second part of a pair
1058 cell=3,2-*,2 # row 3 till end, only column 2
1060 cell=3,2-3,* # column 2 till end, only row 3
1061 cell=3,2-*,* # strip row 1 and 2, and column 1
1062 Cells and cell ranges may be combined with ";", possibly resulting in
1064 rows with different number of columns
1065 cell=1,1-2,2;3,3-4,4;1,4;4,1
1067 Disjointed selections will only return selected cells. The cells
1069 that are not specified will not be included in the returned
1070 set, not even as "undef". As an example given a "CSV" like
1071 11,12,13,...19
1073 21,22,...28,29
1074 : :
1075 91,...97,98,99
1076 with "cell=1,1-2,2;3,3-4,4;1,4;4,1" will return:
1078 11,12,14
1080 21,22
1081 33,34
1082 41,43,44
1083 Overlapping cell-specs will return those cells only once, So
1085 "cell=1,1-3,3;2,2-4,4;2,3;4,2" will return:
1086 11,12,13
1088 21,22,23,24
1089 31,32,33,34
1090 42,43,44
1091 RFC7111 <http://tools.ietf.org/html/rfc7111> does not allow different
1093 types of specs to be combined (either "row" or "col" or "cell").
1094 Passing an invalid fragment specification will croak and set error
1095 2013.
1096 column_names
1098 Set the "keys" that will be used in the "getline_hr" calls. If no
1099 keys (column names) are passed, it will return the current setting as a
1100 list.
1101 "column_names" accepts a list of scalars (the column names) or a
1103 single array_ref, so you can pass the return value from "getline" too:
1104 $csv->column_names ($csv->getline ($fh));
1106 "column_names" does no checking on duplicates at all, which might lead
1108 to unexpected results. Undefined entries will be replaced with the
1109 string "\cAUNDEF\cA", so
1110 $csv->column_names (undef, "", "name", "name");
1112 $hr = $csv->getline_hr ($fh);
1113 Will set "$hr->{"\cAUNDEF\cA"}" to the 1st field, "$hr->{""}" to the
1115 2nd field, and "$hr->{name}" to the 4th field, discarding the 3rd
1116 field.
1117 "column_names" croaks on invalid arguments.
1119 header
1121 This method does NOT work in perl-5.6.x
1122 Parse the CSV header and set "sep", column_names and encoding.
1124 my @hdr = $csv->header ($fh);
1126 $csv->header ($fh, { sep_set => [ ";", ",", "|", "\t" ] });
1127 $csv->header ($fh, { detect_bom => 1, munge_column_names => "lc" });
1128 The first argument should be a file handle.
1130 This method resets some object properties, as it is supposed to be
1132 invoked only once per file or stream. It will leave attributes
1133 "column_names" and "bound_columns" alone of setting column names is
1134 disabled. Reading headers on previously process objects might fail on
1135 perl-5.8.0 and older.
1136 Assuming that the file opened for parsing has a header, and the header
1138 does not contain problematic characters like embedded newlines, read
1139 the first line from the open handle then auto-detect whether the header
1140 separates the column names with a character from the allowed separator
1141 list.
1142 If any of the allowed separators matches, and none of the other
1144 allowed separators match, set "sep" to that separator for the
1145 current CSV_XS instance and use it to parse the first line, map those
1146 to lowercase, and use that to set the instance "column_names":
1147 my $csv = Text::CSV_XS->new ({ binary => 1, auto_diag => 1 });
1149 open my $fh, "<", "file.csv";
1150 binmode $fh; # for Windows
1151 $csv->header ($fh);
1152 while (my $row = $csv->getline_hr ($fh)) {
1153 ...
1154 }
1155 If the header is empty, contains more than one unique separator out of
1157 the allowed set, contains empty fields, or contains identical fields
1158 (after folding), it will croak with error 1010, 1011, 1012, or 1013
1159 respectively.
1160 If the header contains embedded newlines or is not valid CSV in any
1162 other way, this method will croak and leave the parse error untouched.
1163 A successful call to "header" will always set the "sep" of the $csv
1165 object. This behavior can not be disabled.
1166 return value
1168 On error this method will croak.
1170 In list context, the headers will be returned whether they are used to
1172 set "column_names" or not.
1173 In scalar context, the instance itself is returned. Note: the values
1175 as found in the header will effectively be lost if "set_column_names"
1176 is false.
1177 Options
1179 sep_set
1181 $csv->header ($fh, { sep_set => [ ";", ",", "|", "\t" ] });
1182 The list of legal separators defaults to "[ ";", "," ]" and can be
1184 changed by this option. As this is probably the most often used
1185 option, it can be passed on its own as an unnamed argument:
1186 $csv->header ($fh, [ ";", ",", "|", "\t", "::", "\x{2063}" ]);
1188 Multi-byte sequences are allowed, both multi-character and
1190 Unicode. See "sep".
1191 detect_bom
1193 $csv->header ($fh, { detect_bom => 1 });
1194 The default behavior is to detect if the header line starts with a
1196 BOM. If the header has a BOM, use that to set the encoding of $fh.
1197 This default behavior can be disabled by passing a false value to
1198 "detect_bom".
1199 Supported encodings from BOM are: UTF-8, UTF-16BE, UTF-16LE,
1201 UTF-32BE, and UTF-32LE. BOM's also support UTF-1, UTF-EBCDIC, SCSU,
1202 BOCU-1, and GB-18030 but Encode does not (yet). UTF-7 is not
1203 supported.
1204 If a supported BOM was detected as start of the stream, it is stored
1206 in the abject attribute "ENCODING".
1207 my $enc = $csv->{ENCODING};
1209 The encoding is used with "binmode" on $fh.
1211 If the handle was opened in a (correct) encoding, this method will
1213 not alter the encoding, as it checks the leading bytes of the first
1214 line. In case the stream starts with a decode BOM ("U+FEFF"),
1215 "{ENCODING}" will be "" (empty) instead of the default "undef".
1216 munge_column_names
1218 This option offers the means to modify the column names into
1219 something that is most useful to the application. The default is to
1220 map all column names to lower case.
1221 $csv->header ($fh, { munge_column_names => "lc" });
1223 The following values are available:
1225 lc - lower case
1227 uc - upper case
1228 none - do not change
1229 \%hash - supply a mapping
1230 \&cb - supply a callback
1231 Literal:
1233 $csv->header ($fh, { munge_column_names => "none" });
1235 Hash:
1237 $csv->header ($fh, { munge_column_names => { foo => "sombrero" });
1239 if a value does not exist, the original value is used unchanged
1241 Callback:
1243 $csv->header ($fh, { munge_column_names => sub { fc } });
1245 $csv->header ($fh, { munge_column_names => sub { "column_".$col++ } });
1246 $csv->header ($fh, { munge_column_names => sub { lc (s/\W+/_/gr) } });
1247 As this callback is called in a "map", you can use $_ directly.
1249 set_column_names
1251 $csv->header ($fh, { set_column_names => 1 });
1252 The default is to set the instances column names using
1254 "column_names" if the method is successful, so subsequent calls to
1255 "getline_hr" can return a hash. Disable setting the header can be
1256 forced by using a false value for this option.
1257 As described in "return value" above, content is lost in scalar
1259 context.
1260 Validation
1262 When receiving CSV files from external sources, this method can be
1264 used to protect against changes in the layout by restricting to known
1265 headers (and typos in the header fields).
1266 my %known = (
1268 "record key" => "c_rec",
1269 "rec id" => "c_rec",
1270 "id_rec" => "c_rec",
1271 "kode" => "code",
1272 "code" => "code",
1273 "vaule" => "value",
1274 "value" => "value",
1275 );
1276 my $csv = Text::CSV_XS->new ({ binary => 1, auto_diag => 1 });
1277 open my $fh, "<", $source or die "$source: $!";
1278 $csv->header ($fh, { munge_column_names => sub {
1279 s/\s+$//;
1280 s/^\s+//;
1281 $known{lc $_} or die "Unknown column '$_' in $source";
1282 }});
1283 while (my $row = $csv->getline_hr ($fh)) {
1284 say join "\t", $row->{c_rec}, $row->{code}, $row->{value};
1285 }
1286 bind_columns
1288 Takes a list of scalar references to be used for output with "print"
1289 or to store in the fields fetched by "getline". When you do not pass
1290 enough references to store the fetched fields in, "getline" will fail
1291 with error 3006. If you pass more than there are fields to return,
1292 the content of the remaining references is left untouched.
1293 $csv->bind_columns (\$code, \$name, \$price, \$description);
1295 while ($csv->getline ($fh)) {
1296 print "The price of a $name is \x{20ac} $price\n";
1297 }
1298 To reset or clear all column binding, call "bind_columns" with the
1300 single argument "undef". This will also clear column names.
1301 $csv->bind_columns (undef);
1303 If no arguments are passed at all, "bind_columns" will return the list
1305 of current bindings or "undef" if no binds are active.
1306 Note that in parsing with "bind_columns", the fields are set on the
1308 fly. That implies that if the third field of a row causes an error
1309 (or this row has just two fields where the previous row had more), the
1310 first two fields already have been assigned the values of the current
1311 row, while the rest of the fields will still hold the values of the
1312 previous row. If you want the parser to fail in these cases, use the
1313 "strict" attribute.
1314 eof
1316 $eof = $csv->eof ();
1317 If "parse" or "getline" was used with an IO stream, this method will
1319 return true (1) if the last call hit end of file, otherwise it will
1320 return false (''). This is useful to see the difference between a
1321 failure and end of file.
1322 Note that if the parsing of the last line caused an error, "eof" is
1324 still true. That means that if you are not using "auto_diag", an idiom
1325 like
1326 while (my $row = $csv->getline ($fh)) {
1328 # ...
1329 }
1330 $csv->eof or $csv->error_diag;
1331 will not report the error. You would have to change that to
1333 while (my $row = $csv->getline ($fh)) {
1335 # ...
1336 }
1337 +$csv->error_diag and $csv->error_diag;
1338 types
1340 $csv->types (\@tref);
1341 This method is used to force that (all) columns are of a given type.
1343 For example, if you have an integer column, two columns with
1344 doubles and a string column, then you might do a
1345 $csv->types ([Text::CSV_XS::IV (),
1347 Text::CSV_XS::NV (),
1348 Text::CSV_XS::NV (),
1349 Text::CSV_XS::PV ()]);
1350 Column types are used only for decoding columns while parsing, in
1352 other words by the "parse" and "getline" methods.
1353 You can unset column types by doing a
1355 $csv->types (undef);
1357 or fetch the current type settings with
1359 $types = $csv->types ();
1361 IV Set field type to integer.
1363 NV Set field type to numeric/float.
1365 PV Set field type to string.
1367 fields
1369 @columns = $csv->fields ();
1370 This method returns the input to "combine" or the resultant
1372 decomposed fields of a successful "parse", whichever was called more
1373 recently.
1374 Note that the return value is undefined after using "getline", which
1376 does not fill the data structures returned by "parse".
1377 meta_info
1379 @flags = $csv->meta_info ();
1380 This method returns the "flags" of the input to "combine" or the flags
1382 of the resultant decomposed fields of "parse", whichever was called
1383 more recently.
1384 For each field, a meta_info field will hold flags that inform
1386 something about the field returned by the "fields" method or
1387 passed to the "combine" method. The flags are bit-wise-"or"'d like:
1388 " "0x0001
1390 The field was quoted.
1391 " "0x0002
1393 The field was binary.
1394 See the "is_***" methods below.
1396 is_quoted
1398 my $quoted = $csv->is_quoted ($column_idx);
1399 Where $column_idx is the (zero-based) index of the column in the
1401 last result of "parse".
1402 This returns a true value if the data in the indicated column was
1404 enclosed in "quote_char" quotes. This might be important for fields
1405 where content ",20070108," is to be treated as a numeric value, and
1406 where ","20070108"," is explicitly marked as character string data.
1407 This method is only valid when "keep_meta_info" is set to a true value.
1409 is_binary
1411 my $binary = $csv->is_binary ($column_idx);
1412 Where $column_idx is the (zero-based) index of the column in the
1414 last result of "parse".
1415 This returns a true value if the data in the indicated column contained
1417 any byte in the range "[\x00-\x08,\x10-\x1F,\x7F-\xFF]".
1418 This method is only valid when "keep_meta_info" is set to a true value.
1420 is_missing
1422 my $missing = $csv->is_missing ($column_idx);
1423 Where $column_idx is the (zero-based) index of the column in the
1425 last result of "getline_hr".
1426 $csv->keep_meta_info (1);
1428 while (my $hr = $csv->getline_hr ($fh)) {
1429 $csv->is_missing (0) and next; # This was an empty line
1430 }
1431 When using "getline_hr", it is impossible to tell if the parsed
1433 fields are "undef" because they where not filled in the "CSV" stream
1434 or because they were not read at all, as all the fields defined by
1435 "column_names" are set in the hash-ref. If you still need to know if
1436 all fields in each row are provided, you should enable "keep_meta_info"
1437 so you can check the flags.
1438 If "keep_meta_info" is "false", "is_missing" will always return
1440 "undef", regardless of $column_idx being valid or not. If this
1441 attribute is "true" it will return either 0 (the field is present) or 1
1442 (the field is missing).
1443 A special case is the empty line. If the line is completely empty -
1445 after dealing with the flags - this is still a valid CSV line: it is a
1446 record of just one single empty field. However, if "keep_meta_info" is
1447 set, invoking "is_missing" with index 0 will now return true.
1448 status
1450 $status = $csv->status ();
1451 This method returns the status of the last invoked "combine" or "parse"
1453 call. Status is success (true: 1) or failure (false: "undef" or 0).
1454 error_input
1456 $bad_argument = $csv->error_input ();
1457 This method returns the erroneous argument (if it exists) of "combine"
1459 or "parse", whichever was called more recently. If the last
1460 invocation was successful, "error_input" will return "undef".
1461 error_diag
1463 Text::CSV_XS->error_diag ();
1464 $csv->error_diag ();
1465 $error_code = 0 + $csv->error_diag ();
1466 $error_str = "" . $csv->error_diag ();
1467 ($cde, $str, $pos, $rec, $fld) = $csv->error_diag ();
1468 If (and only if) an error occurred, this function returns the
1470 diagnostics of that error.
1471 If called in void context, this will print the internal error code and
1473 the associated error message to STDERR.
1474 If called in list context, this will return the error code and the
1476 error message in that order. If the last error was from parsing, the
1477 rest of the values returned are a best guess at the location within
1478 the line that was being parsed. Their values are 1-based. The
1479 position currently is index of the byte at which the parsing failed in
1480 the current record. It might change to be the index of the current
1481 character in a later release. The records is the index of the record
1482 parsed by the csv instance. The field number is the index of the field
1483 the parser thinks it is currently trying to parse. See
1484 examples/csv-check for how this can be used.
1485 If called in scalar context, it will return the diagnostics in a
1487 single scalar, a-la $!. It will contain the error code in numeric
1488 context, and the diagnostics message in string context.
1489 When called as a class method or a direct function call, the
1491 diagnostics are that of the last "new" call.
1492 record_number
1494 $recno = $csv->record_number ();
1495 Returns the records parsed by this csv instance. This value should be
1497 more accurate than $. when embedded newlines come in play. Records
1498 written by this instance are not counted.
1499 SetDiag
1501 $csv->SetDiag (0);
1502 Use to reset the diagnostics if you are dealing with errors.
1504
1506 csv
1507 This function is not exported by default and should be explicitly
1508 requested:
1509 use Text::CSV_XS qw( csv );
1511 This is an high-level function that aims at simple (user) interfaces.
1513 This can be used to read/parse a "CSV" file or stream (the default
1514 behavior) or to produce a file or write to a stream (define the "out"
1515 attribute). It returns an array- or hash-reference on parsing (or
1516 "undef" on fail) or the numeric value of "error_diag" on writing.
1517 When this function fails you can get to the error using the class call
1518 to "error_diag"
1519 my $aoa = csv (in => "test.csv") or
1521 die Text::CSV_XS->error_diag;
1522 This function takes the arguments as key-value pairs. This can be
1524 passed as a list or as an anonymous hash:
1525 my $aoa = csv ( in => "test.csv", sep_char => ";");
1527 my $aoh = csv ({ in => $fh, headers => "auto" });
1528 The arguments passed consist of two parts: the arguments to "csv"
1530 itself and the optional attributes to the "CSV" object used inside
1531 the function as enumerated and explained in "new".
1532 If not overridden, the default option used for CSV is
1534 auto_diag => 1
1536 escape_null => 0
1537 The option that is always set and cannot be altered is
1539 binary => 1
1541 As this function will likely be used in one-liners, it allows "quote"
1543 to be abbreviated as "quo", and "escape_char" to be abbreviated as
1544 "esc" or "escape".
1545 Alternative invocations:
1547 my $aoa = Text::CSV_XS::csv (in => "file.csv");
1549 my $csv = Text::CSV_XS->new ();
1551 my $aoa = $csv->csv (in => "file.csv");
1552 In the latter case, the object attributes are used from the existing
1554 object and the attribute arguments in the function call are ignored:
1555 my $csv = Text::CSV_XS->new ({ sep_char => ";" });
1557 my $aoh = $csv->csv (in => "file.csv", bom => 1);
1558 will parse using ";" as "sep_char", not ",".
1560 in
1562 Used to specify the source. "in" can be a file name (e.g. "file.csv"),
1564 which will be opened for reading and closed when finished, a file
1565 handle (e.g. $fh or "FH"), a reference to a glob (e.g. "\*ARGV"),
1566 the glob itself (e.g. *STDIN), or a reference to a scalar (e.g.
1567 "\q{1,2,"csv"}").
1568 When used with "out", "in" should be a reference to a CSV structure
1570 (AoA or AoH) or a CODE-ref that returns an array-reference or a hash-
1571 reference. The code-ref will be invoked with no arguments.
1572 my $aoa = csv (in => "file.csv");
1574 open my $fh, "<", "file.csv";
1576 my $aoa = csv (in => $fh);
1577 my $csv = [ [qw( Foo Bar )], [ 1, 2 ], [ 2, 3 ]];
1579 my $err = csv (in => $csv, out => "file.csv");
1580 If called in void context without the "out" attribute, the resulting
1582 ref will be used as input to a subsequent call to csv:
1583 csv (in => "file.csv", filter => { 2 => sub { length > 2 }})
1585 will be a shortcut to
1587 csv (in => csv (in => "file.csv", filter => { 2 => sub { length > 2 }}))
1589 where, in the absence of the "out" attribute, this is a shortcut to
1591 csv (in => csv (in => "file.csv", filter => { 2 => sub { length > 2 }}),
1593 out => *STDOUT)
1594 out
1596 csv (in => $aoa, out => "file.csv");
1598 csv (in => $aoa, out => $fh);
1599 csv (in => $aoa, out => STDOUT);
1600 csv (in => $aoa, out => *STDOUT);
1601 csv (in => $aoa, out => \*STDOUT);
1602 csv (in => $aoa, out => \my $data);
1603 csv (in => $aoa, out => undef);
1604 csv (in => $aoa, out => \"skip");
1605 In output mode, the default CSV options when producing CSV are
1607 eol => "\r\n"
1609 The "fragment" attribute is ignored in output mode.
1611 "out" can be a file name (e.g. "file.csv"), which will be opened for
1613 writing and closed when finished, a file handle (e.g. $fh or "FH"), a
1614 reference to a glob (e.g. "\*STDOUT"), the glob itself (e.g. *STDOUT),
1615 or a reference to a scalar (e.g. "\my $data").
1616 csv (in => sub { $sth->fetch }, out => "dump.csv");
1618 csv (in => sub { $sth->fetchrow_hashref }, out => "dump.csv",
1619 headers => $sth->{NAME_lc});
1620 When a code-ref is used for "in", the output is generated per
1622 invocation, so no buffering is involved. This implies that there is no
1623 size restriction on the number of records. The "csv" function ends when
1624 the coderef returns a false value.
1625 If "out" is set to a reference of the literal string "skip", the output
1627 will be suppressed completely, which might be useful in combination
1628 with a filter for side effects only.
1629 my %cache;
1631 csv (in => "dump.csv",
1632 out => \"skip",
1633 on_in => sub { $cache{$_[1][1]}++ });
1634 Currently, setting "out" to any false value ("undef", "", 0) will be
1636 equivalent to "\"skip"".
1637 encoding
1639 If passed, it should be an encoding accepted by the ":encoding()"
1641 option to "open". There is no default value. This attribute does not
1642 work in perl 5.6.x. "encoding" can be abbreviated to "enc" for ease of
1643 use in command line invocations.
1644 If "encoding" is set to the literal value "auto", the method "header"
1646 will be invoked on the opened stream to check if there is a BOM and set
1647 the encoding accordingly. This is equal to passing a true value in
1648 the option "detect_bom".
1649 detect_bom
1651 If "detect_bom" is given, the method "header" will be invoked on
1653 the opened stream to check if there is a BOM and set the encoding
1654 accordingly.
1655 "detect_bom" can be abbreviated to "bom".
1657 This is the same as setting "encoding" to "auto".
1659 Note that as the method "header" is invoked, its default is to also
1661 set the headers.
1662 headers
1664 If this attribute is not given, the default behavior is to produce an
1666 array of arrays.
1667 If "headers" is supplied, it should be an anonymous list of column
1669 names, an anonymous hashref, a coderef, or a literal flag: "auto",
1670 "lc", "uc", or "skip".
1671 skip
1673 When "skip" is used, the header will not be included in the output.
1674 my $aoa = csv (in => $fh, headers => "skip");
1676 auto
1678 If "auto" is used, the first line of the "CSV" source will be read as
1679 the list of field headers and used to produce an array of hashes.
1680 my $aoh = csv (in => $fh, headers => "auto");
1682 lc
1684 If "lc" is used, the first line of the "CSV" source will be read as
1685 the list of field headers mapped to lower case and used to produce
1686 an array of hashes. This is a variation of "auto".
1687 my $aoh = csv (in => $fh, headers => "lc");
1689 uc
1691 If "uc" is used, the first line of the "CSV" source will be read as
1692 the list of field headers mapped to upper case and used to produce
1693 an array of hashes. This is a variation of "auto".
1694 my $aoh = csv (in => $fh, headers => "uc");
1696 CODE
1698 If a coderef is used, the first line of the "CSV" source will be
1699 read as the list of mangled field headers in which each field is
1700 passed as the only argument to the coderef. This list is used to
1701 produce an array of hashes.
1702 my $aoh = csv (in => $fh,
1704 headers => sub { lc ($_[0]) =~ s/kode/code/gr });
1705 this example is a variation of using "lc" where all occurrences of
1707 "kode" are replaced with "code".
1708 ARRAY
1710 If "headers" is an anonymous list, the entries in the list will be
1711 used as field names. The first line is considered data instead of
1712 headers.
1713 my $aoh = csv (in => $fh, headers => [qw( Foo Bar )]);
1715 csv (in => $aoa, out => $fh, headers => [qw( code description price )]);
1716 HASH
1718 If "headers" is an hash reference, this implies "auto", but header
1719 fields for that exist as key in the hashref will be replaced by the
1720 value for that key. Given a CSV file like
1721 post-kode,city,name,id number,fubble
1723 1234AA,Duckstad,Donald,13,"X313DF"
1724 using
1726 csv (headers => { "post-kode" => "pc", "id number" => "ID" }, ...
1728 will return an entry like
1730 { pc => "1234AA",
1732 city => "Duckstad",
1733 name => "Donald",
1734 ID => "13",
1735 fubble => "X313DF",
1736 }
1737 See also "munge_column_names" and "set_column_names".
1739 munge_column_names
1741 If "munge_column_names" is set, the method "header" is invoked on
1743 the opened stream with all matching arguments to detect and set the
1744 headers.
1745 "munge_column_names" can be abbreviated to "munge".
1747 key
1749 If passed, will default "headers" to "auto" and return a hashref
1751 instead of an array of hashes. Allowed values are simple scalars or
1752 array-references where the first element is the joiner and the rest are
1753 the fields to join to combine the key.
1754 my $ref = csv (in => "test.csv", key => "code");
1756 my $ref = csv (in => "test.csv", key => [ ":" => "code", "color" ]);
1757 with test.csv like
1759 code,product,price,color
1761 1,pc,850,gray
1762 2,keyboard,12,white
1763 3,mouse,5,black
1764 the first example will return
1766 { 1 => {
1768 code => 1,
1769 color => 'gray',
1770 price => 850,
1771 product => 'pc'
1772 },
1773 2 => {
1774 code => 2,
1775 color => 'white',
1776 price => 12,
1777 product => 'keyboard'
1778 },
1779 3 => {
1780 code => 3,
1781 color => 'black',
1782 price => 5,
1783 product => 'mouse'
1784 }
1785 }
1786 the second example will return
1788 { "1:gray" => {
1790 code => 1,
1791 color => 'gray',
1792 price => 850,
1793 product => 'pc'
1794 },
1795 "2:white" => {
1796 code => 2,
1797 color => 'white',
1798 price => 12,
1799 product => 'keyboard'
1800 },
1801 "3:black" => {
1802 code => 3,
1803 color => 'black',
1804 price => 5,
1805 product => 'mouse'
1806 }
1807 }
1808 The "key" attribute can be combined with "headers" for "CSV" date that
1810 has no header line, like
1811 my $ref = csv (
1813 in => "foo.csv",
1814 headers => [qw( c_foo foo bar description stock )],
1815 key => "c_foo",
1816 );
1817 value
1819 Used to create key-value hashes.
1821 Only allowed when "key" is valid. A "value" can be either a single
1823 column label or an anonymous list of column labels. In the first case,
1824 the value will be a simple scalar value, in the latter case, it will be
1825 a hashref.
1826 my $ref = csv (in => "test.csv", key => "code",
1828 value => "price");
1829 my $ref = csv (in => "test.csv", key => "code",
1830 value => [ "product", "price" ]);
1831 my $ref = csv (in => "test.csv", key => [ ":" => "code", "color" ],
1832 value => "price");
1833 my $ref = csv (in => "test.csv", key => [ ":" => "code", "color" ],
1834 value => [ "product", "price" ]);
1835 with test.csv like
1837 code,product,price,color
1839 1,pc,850,gray
1840 2,keyboard,12,white
1841 3,mouse,5,black
1842 the first example will return
1844 { 1 => 850,
1846 2 => 12,
1847 3 => 5,
1848 }
1849 the second example will return
1851 { 1 => {
1853 price => 850,
1854 product => 'pc'
1855 },
1856 2 => {
1857 price => 12,
1858 product => 'keyboard'
1859 },
1860 3 => {
1861 price => 5,
1862 product => 'mouse'
1863 }
1864 }
1865 the third example will return
1867 { "1:gray" => 850,
1869 "2:white" => 12,
1870 "3:black" => 5,
1871 }
1872 the fourth example will return
1874 { "1:gray" => {
1876 price => 850,
1877 product => 'pc'
1878 },
1879 "2:white" => {
1880 price => 12,
1881 product => 'keyboard'
1882 },
1883 "3:black" => {
1884 price => 5,
1885 product => 'mouse'
1886 }
1887 }
1888 keep_headers
1890 When using hashes, keep the column names into the arrayref passed, so
1892 all headers are available after the call in the original order.
1893 my $aoh = csv (in => "file.csv", keep_headers => \my @hdr);
1895 This attribute can be abbreviated to "kh" or passed as
1897 "keep_column_names".
1898 This attribute implies a default of "auto" for the "headers" attribute.
1900 fragment
1902 Only output the fragment as defined in the "fragment" method. This
1904 option is ignored when generating "CSV". See "out".
1905 Combining all of them could give something like
1907 use Text::CSV_XS qw( csv );
1909 my $aoh = csv (
1910 in => "test.txt",
1911 encoding => "utf-8",
1912 headers => "auto",
1913 sep_char => "|",
1914 fragment => "row=3;6-9;15-*",
1915 );
1916 say $aoh->[15]{Foo};
1917 sep_set
1919 If "sep_set" is set, the method "header" is invoked on the opened
1921 stream to detect and set "sep_char" with the given set.
1922 "sep_set" can be abbreviated to "seps".
1924 Note that as the "header" method is invoked, its default is to also
1926 set the headers.
1927 set_column_names
1929 If "set_column_names" is passed, the method "header" is invoked on
1931 the opened stream with all arguments meant for "header".
1932 If "set_column_names" is passed as a false value, the content of the
1934 first row is only preserved if the output is AoA:
1935 With an input-file like
1937 bAr,foo
1939 1,2
1940 3,4,5
1941 This call
1943 my $aoa = csv (in => $file, set_column_names => 0);
1945 will result in
1947 [[ "bar", "foo" ],
1949 [ "1", "2" ],
1950 [ "3", "4", "5" ]]
1951 and
1953 my $aoa = csv (in => $file, set_column_names => 0, munge => "none");
1955 will result in
1957 [[ "bAr", "foo" ],
1959 [ "1", "2" ],
1960 [ "3", "4", "5" ]]
1961 Callbacks
1963 Callbacks enable actions triggered from the inside of Text::CSV_XS.
1964 While most of what this enables can easily be done in an unrolled
1966 loop as described in the "SYNOPSIS" callbacks can be used to meet
1967 special demands or enhance the "csv" function.
1968 error
1970 $csv->callbacks (error => sub { $csv->SetDiag (0) });
1971 the "error" callback is invoked when an error occurs, but only
1973 when "auto_diag" is set to a true value. A callback is invoked with
1974 the values returned by "error_diag":
1975 my ($c, $s);
1977 sub ignore3006
1979 {
1980 my ($err, $msg, $pos, $recno, $fldno) = @_;
1981 if ($err == 3006) {
1982 # ignore this error
1983 ($c, $s) = (undef, undef);
1984 Text::CSV_XS->SetDiag (0);
1985 }
1986 # Any other error
1987 return;
1988 } # ignore3006
1989 $csv->callbacks (error => \&ignore3006);
1991 $csv->bind_columns (\$c, \$s);
1992 while ($csv->getline ($fh)) {
1993 # Error 3006 will not stop the loop
1994 }
1995 after_parse
1997 $csv->callbacks (after_parse => sub { push @{$_[1]}, "NEW" });
1998 while (my $row = $csv->getline ($fh)) {
1999 $row->[-1] eq "NEW";
2000 }
2001 This callback is invoked after parsing with "getline" only if no
2003 error occurred. The callback is invoked with two arguments: the
2004 current "CSV" parser object and an array reference to the fields
2005 parsed.
2006 The return code of the callback is ignored unless it is a reference
2008 to the string "skip", in which case the record will be skipped in
2009 "getline_all".
2010 sub add_from_db
2012 {
2013 my ($csv, $row) = @_;
2014 $sth->execute ($row->[4]);
2015 push @$row, $sth->fetchrow_array;
2016 } # add_from_db
2017 my $aoa = csv (in => "file.csv", callbacks => {
2019 after_parse => \&add_from_db });
2020 This hook can be used for validation:
2022 FAIL
2024 Die if any of the records does not validate a rule:
2025 after_parse => sub {
2027 $_[1][4] =~ m/^[0-9]{4}\s?[A-Z]{2}$/ or
2028 die "5th field does not have a valid Dutch zipcode";
2029 }
2030 DEFAULT
2032 Replace invalid fields with a default value:
2033 after_parse => sub { $_[1][2] =~ m/^\d+$/ or $_[1][2] = 0 }
2035 SKIP
2037 Skip records that have invalid fields (only applies to
2038 "getline_all"):
2039 after_parse => sub { $_[1][0] =~ m/^\d+$/ or return \"skip"; }
2041 before_print
2043 my $idx = 1;
2044 $csv->callbacks (before_print => sub { $_[1][0] = $idx++ });
2045 $csv->print (*STDOUT, [ 0, $_ ]) for @members;
2046 This callback is invoked before printing with "print" only if no
2048 error occurred. The callback is invoked with two arguments: the
2049 current "CSV" parser object and an array reference to the fields
2050 passed.
2051 The return code of the callback is ignored.
2053 sub max_4_fields
2055 {
2056 my ($csv, $row) = @_;
2057 @$row > 4 and splice @$row, 4;
2058 } # max_4_fields
2059 csv (in => csv (in => "file.csv"), out => *STDOUT,
2061 callbacks => { before print => \&max_4_fields });
2062 This callback is not active for "combine".
2064 Callbacks for csv ()
2066 The "csv" allows for some callbacks that do not integrate in XS
2068 internals but only feature the "csv" function.
2069 csv (in => "file.csv",
2071 callbacks => {
2072 filter => { 6 => sub { $_ > 15 } }, # first
2073 after_parse => sub { say "AFTER PARSE"; }, # first
2074 after_in => sub { say "AFTER IN"; }, # second
2075 on_in => sub { say "ON IN"; }, # third
2076 },
2077 );
2078 csv (in => $aoh,
2080 out => "file.csv",
2081 callbacks => {
2082 on_in => sub { say "ON IN"; }, # first
2083 before_out => sub { say "BEFORE OUT"; }, # second
2084 before_print => sub { say "BEFORE PRINT"; }, # third
2085 },
2086 );
2087 filter
2089 This callback can be used to filter records. It is called just after
2090 a new record has been scanned. The callback accepts a:
2091 hashref
2093 The keys are the index to the row (the field name or field number,
2094 1-based) and the values are subs to return a true or false value.
2095 csv (in => "file.csv", filter => {
2097 3 => sub { m/a/ }, # third field should contain an "a"
2098 5 => sub { length > 4 }, # length of the 5th field minimal 5
2099 });
2100 csv (in => "file.csv", filter => { foo => sub { $_ > 4 }});
2102 If the keys to the filter hash contain any character that is not a
2104 digit it will also implicitly set "headers" to "auto" unless
2105 "headers" was already passed as argument. When headers are
2106 active, returning an array of hashes, the filter is not applicable
2107 to the header itself.
2108 All sub results should match, as in AND.
2110 The context of the callback sets $_ localized to the field
2112 indicated by the filter. The two arguments are as with all other
2113 callbacks, so the other fields in the current row can be seen:
2114 filter => { 3 => sub { $_ > 100 ? $_[1][1] =~ m/A/ : $_[1][6] =~ m/B/ }}
2116 If the context is set to return a list of hashes ("headers" is
2118 defined), the current record will also be available in the
2119 localized %_:
2120 filter => { 3 => sub { $_ > 100 && $_{foo} =~ m/A/ && $_{bar} < 1000 }}
2122 If the filter is used to alter the content by changing $_, make
2124 sure that the sub returns true in order not to have that record
2125 skipped:
2126 filter => { 2 => sub { $_ = uc }}
2128 will upper-case the second field, and then skip it if the resulting
2130 content evaluates to false. To always accept, end with truth:
2131 filter => { 2 => sub { $_ = uc; 1 }}
2133 coderef
2135 csv (in => "file.csv", filter => sub { $n++; 0; });
2136 If the argument to "filter" is a coderef, it is an alias or
2138 shortcut to a filter on column 0:
2139 csv (filter => sub { $n++; 0 });
2141 is equal to
2143 csv (filter => { 0 => sub { $n++; 0 });
2145 filter-name
2147 csv (in => "file.csv", filter => "not_blank");
2148 csv (in => "file.csv", filter => "not_empty");
2149 csv (in => "file.csv", filter => "filled");
2150 These are predefined filters
2152 Given a file like (line numbers prefixed for doc purpose only):
2154 1:1,2,3
2156 2:
2157 3:,
2158 4:""
2159 5:,,
2160 6:, ,
2161 7:"",
2162 8:" "
2163 9:4,5,6
2164 not_blank
2166 Filter out the blank lines
2167 This filter is a shortcut for
2169 filter => { 0 => sub { @{$_[1]} > 1 or
2171 defined $_[1][0] && $_[1][0] ne "" } }
2172 Due to the implementation, it is currently impossible to also
2174 filter lines that consists only of a quoted empty field. These
2175 lines are also considered blank lines.
2176 With the given example, lines 2 and 4 will be skipped.
2178 not_empty
2180 Filter out lines where all the fields are empty.
2181 This filter is a shortcut for
2183 filter => { 0 => sub { grep { defined && $_ ne "" } @{$_[1]} } }
2185 A space is not regarded being empty, so given the example data,
2187 lines 2, 3, 4, 5, and 7 are skipped.
2188 filled
2190 Filter out lines that have no visible data
2191 This filter is a shortcut for
2193 filter => { 0 => sub { grep { defined && m/\S/ } @{$_[1]} } }
2195 This filter rejects all lines that not have at least one field
2197 that does not evaluate to the empty string.
2198 With the given example data, this filter would skip lines 2
2200 through 8.
2201 after_in
2203 This callback is invoked for each record after all records have been
2204 parsed but before returning the reference to the caller. The hook is
2205 invoked with two arguments: the current "CSV" parser object and a
2206 reference to the record. The reference can be a reference to a
2207 HASH or a reference to an ARRAY as determined by the arguments.
2208 This callback can also be passed as an attribute without the
2210 "callbacks" wrapper.
2211 before_out
2213 This callback is invoked for each record before the record is
2214 printed. The hook is invoked with two arguments: the current "CSV"
2215 parser object and a reference to the record. The reference can be a
2216 reference to a HASH or a reference to an ARRAY as determined by the
2217 arguments.
2218 This callback can also be passed as an attribute without the
2220 "callbacks" wrapper.
2221 This callback makes the row available in %_ if the row is a hashref.
2223 In this case %_ is writable and will change the original row.
2224 on_in
2226 This callback acts exactly as the "after_in" or the "before_out"
2227 hooks.
2228 This callback can also be passed as an attribute without the
2230 "callbacks" wrapper.
2231 This callback makes the row available in %_ if the row is a hashref.
2233 In this case %_ is writable and will change the original row. So e.g.
2234 with
2235 my $aoh = csv (
2237 in => \"foo\n1\n2\n",
2238 headers => "auto",
2239 on_in => sub { $_{bar} = 2; },
2240 );
2241 $aoh will be:
2243 [ { foo => 1,
2245 bar => 2,
2246 }
2247 { foo => 2,
2248 bar => 2,
2249 }
2250 ]
2251 csv
2253 The function "csv" can also be called as a method or with an
2254 existing Text::CSV_XS object. This could help if the function is to
2255 be invoked a lot of times and the overhead of creating the object
2256 internally over and over again would be prevented by passing an
2257 existing instance.
2258 my $csv = Text::CSV_XS->new ({ binary => 1, auto_diag => 1 });
2260 my $aoa = $csv->csv (in => $fh);
2262 my $aoa = csv (in => $fh, csv => $csv);
2263 both act the same. Running this 20000 times on a 20 lines CSV file,
2265 showed a 53% speedup.
2266
2268 Combine (...)
2269 Parse (...)
2270 The arguments to these internal functions are deliberately not
2272 described or documented in order to enable the module authors make
2273 changes it when they feel the need for it. Using them is highly
2274 discouraged as the API may change in future releases.
2275
2277 Reading a CSV file line by line:
2278 my $csv = Text::CSV_XS->new ({ binary => 1, auto_diag => 1 });
2279 open my $fh, "<", "file.csv" or die "file.csv: $!";
2280 while (my $row = $csv->getline ($fh)) {
2281 # do something with @$row
2282 }
2283 close $fh or die "file.csv: $!";
2284 or
2286 my $aoa = csv (in => "file.csv", on_in => sub {
2288 # do something with %_
2289 });
2290 Reading only a single column
2292 my $csv = Text::CSV_XS->new ({ binary => 1, auto_diag => 1 });
2294 open my $fh, "<", "file.csv" or die "file.csv: $!";
2295 # get only the 4th column
2296 my @column = map { $_->[3] } @{$csv->getline_all ($fh)};
2297 close $fh or die "file.csv: $!";
2298 with "csv", you could do
2300 my @column = map { $_->[0] }
2302 @{csv (in => "file.csv", fragment => "col=4")};
2303 Parsing CSV strings:
2305 my $csv = Text::CSV_XS->new ({ keep_meta_info => 1, binary => 1 });
2306 my $sample_input_string =
2308 qq{"I said, ""Hi!""",Yes,"",2.34,,"1.09","\x{20ac}",};
2309 if ($csv->parse ($sample_input_string)) {
2310 my @field = $csv->fields;
2311 foreach my $col (0 .. $#field) {
2312 my $quo = $csv->is_quoted ($col) ? $csv->{quote_char} : "";
2313 printf "%2d: %s%s%s\n", $col, $quo, $field[$col], $quo;
2314 }
2315 }
2316 else {
2317 print STDERR "parse () failed on argument: ",
2318 $csv->error_input, "\n";
2319 $csv->error_diag ();
2320 }
2321 Parsing CSV from memory
2323 Given a complete CSV data-set in scalar $data, generate a list of
2325 lists to represent the rows and fields
2326 # The data
2328 my $data = join "\r\n" => map { join "," => 0 .. 5 } 0 .. 5;
2329 # in a loop
2331 my $csv = Text::CSV_XS->new ({ binary => 1, auto_diag => 1 });
2332 open my $fh, "<", \$data;
2333 my @foo;
2334 while (my $row = $csv->getline ($fh)) {
2335 push @foo, $row;
2336 }
2337 close $fh;
2338 # a single call
2340 my $foo = csv (in => \$data);
2341 Printing CSV data
2343 The fast way: using "print"
2344 An example for creating "CSV" files using the "print" method:
2346 my $csv = Text::CSV_XS->new ({ binary => 1, eol => $/ });
2348 open my $fh, ">", "foo.csv" or die "foo.csv: $!";
2349 for (1 .. 10) {
2350 $csv->print ($fh, [ $_, "$_" ]) or $csv->error_diag;
2351 }
2352 close $fh or die "$tbl.csv: $!";
2353 The slow way: using "combine" and "string"
2355 or using the slower "combine" and "string" methods:
2357 my $csv = Text::CSV_XS->new;
2359 open my $csv_fh, ">", "hello.csv" or die "hello.csv: $!";
2361 my @sample_input_fields = (
2363 'You said, "Hello!"', 5.67,
2364 '"Surely"', '', '3.14159');
2365 if ($csv->combine (@sample_input_fields)) {
2366 print $csv_fh $csv->string, "\n";
2367 }
2368 else {
2369 print "combine () failed on argument: ",
2370 $csv->error_input, "\n";
2371 }
2372 close $csv_fh or die "hello.csv: $!";
2373 Generating CSV into memory
2375 Format a data-set (@foo) into a scalar value in memory ($data):
2377 # The data
2379 my @foo = map { [ 0 .. 5 ] } 0 .. 3;
2380 # in a loop
2382 my $csv = Text::CSV_XS->new ({ binary => 1, auto_diag => 1, eol => "\r\n" });
2383 open my $fh, ">", \my $data;
2384 $csv->print ($fh, $_) for @foo;
2385 close $fh;
2386 # a single call
2388 csv (in => \@foo, out => \my $data);
2389 Rewriting CSV
2391 Rewrite "CSV" files with ";" as separator character to well-formed
2392 "CSV":
2393 use Text::CSV_XS qw( csv );
2395 csv (in => csv (in => "bad.csv", sep_char => ";"), out => *STDOUT);
2396 As "STDOUT" is now default in "csv", a one-liner converting a UTF-16
2398 CSV file with BOM and TAB-separation to valid UTF-8 CSV could be:
2399 $ perl -C3 -MText::CSV_XS=csv -we\
2401 'csv(in=>"utf16tab.csv",encoding=>"utf16",sep=>"\t")' >utf8.csv
2402 Dumping database tables to CSV
2404 Dumping a database table can be simple as this (TIMTOWTDI):
2405 my $dbh = DBI->connect (...);
2407 my $sql = "select * from foo";
2408 # using your own loop
2410 open my $fh, ">", "foo.csv" or die "foo.csv: $!\n";
2411 my $csv = Text::CSV_XS->new ({ binary => 1, eol => "\r\n" });
2412 my $sth = $dbh->prepare ($sql); $sth->execute;
2413 $csv->print ($fh, $sth->{NAME_lc});
2414 while (my $row = $sth->fetch) {
2415 $csv->print ($fh, $row);
2416 }
2417 # using the csv function, all in memory
2419 csv (out => "foo.csv", in => $dbh->selectall_arrayref ($sql));
2420 # using the csv function, streaming with callbacks
2422 my $sth = $dbh->prepare ($sql); $sth->execute;
2423 csv (out => "foo.csv", in => sub { $sth->fetch });
2424 csv (out => "foo.csv", in => sub { $sth->fetchrow_hashref });
2425 Note that this does not discriminate between "empty" values and NULL-
2427 values from the database, as both will be the same empty field in CSV.
2428 To enable distinction between the two, use "quote_empty".
2429 csv (out => "foo.csv", in => sub { $sth->fetch }, quote_empty => 1);
2431 If the database import utility supports special sequences to insert
2433 "NULL" values into the database, like MySQL/MariaDB supports "\N",
2434 use a filter or a map
2435 csv (out => "foo.csv", in => sub { $sth->fetch },
2437 on_in => sub { $_ //= "\\N" for @{$_[1]} });
2438 while (my $row = $sth->fetch) {
2440 $csv->print ($fh, [ map { $_ // "\\N" } @$row ]);
2441 }
2442 note that this will not work as expected when choosing the backslash
2444 ("\") as "escape_char", as that will cause the "\" to need to be
2445 escaped by yet another "\", which will cause the field to need
2446 quotation and thus ending up as "\\N" instead of "\N". See also
2447 "undef_str".
2448 csv (out => "foo.csv", in => sub { $sth->fetch }, undef_str => "\\N");
2450 these special sequences are not recognized by Text::CSV_XS on parsing
2452 the CSV generated like this, but map and filter are your friends again
2453 while (my $row = $csv->getline ($fh)) {
2455 $sth->execute (map { $_ eq "\\N" ? undef : $_ } @$row);
2456 }
2457 csv (in => "foo.csv", filter => { 1 => sub {
2459 $sth->execute (map { $_ eq "\\N" ? undef : $_ } @{$_[1]}); 0; }});
2460 The examples folder
2462 For more extended examples, see the examples/ 1. sub-directory in the
2463 original distribution or the git repository 2.
2464 1. https://github.com/Tux/Text-CSV_XS/tree/master/examples
2466 2. https://github.com/Tux/Text-CSV_XS
2467 The following files can be found there:
2469 parser-xs.pl
2471 This can be used as a boilerplate to parse invalid "CSV" and parse
2472 beyond (expected) errors alternative to using the "error" callback.
2473 $ perl examples/parser-xs.pl bad.csv >good.csv
2475 csv-check
2477 This is a command-line tool that uses parser-xs.pl techniques to
2478 check the "CSV" file and report on its content.
2479 $ csv-check files/utf8.csv
2481 Checked files/utf8.csv with csv-check 1.9
2482 using Text::CSV_XS 1.32 with perl 5.26.0 and Unicode 9.0.0
2483 OK: rows: 1, columns: 2
2484 sep = <,>, quo = <">, bin = <1>, eol = <"\n">
2485 csv2xls
2487 A script to convert "CSV" to Microsoft Excel ("XLS"). This requires
2488 extra modules Date::Calc and Spreadsheet::WriteExcel. The converter
2489 accepts various options and can produce UTF-8 compliant Excel files.
2490 csv2xlsx
2492 A script to convert "CSV" to Microsoft Excel ("XLSX"). This requires
2493 the modules Date::Calc and Spreadsheet::Writer::XLSX. The converter
2494 does accept various options including merging several "CSV" files
2495 into a single Excel file.
2496 csvdiff
2498 A script that provides colorized diff on sorted CSV files, assuming
2499 first line is header and first field is the key. Output options
2500 include colorized ANSI escape codes or HTML.
2501 $ csvdiff --html --output=diff.html file1.csv file2.csv
2503 rewrite.pl
2505 A script to rewrite (in)valid CSV into valid CSV files. Script has
2506 options to generate confusing CSV files or CSV files that conform to
2507 Dutch MS-Excel exports (using ";" as separation).
2508 Script - by default - honors BOM and auto-detects separation
2510 converting it to default standard CSV with "," as separator.
2511
2513 Text::CSV_XS is not designed to detect the characters used to quote
2514 and separate fields. The parsing is done using predefined (default)
2515 settings. In the examples sub-directory, you can find scripts that
2516 demonstrate how you could try to detect these characters yourself.
2517 Microsoft Excel
2519 The import/export from Microsoft Excel is a risky task, according to
2520 the documentation in "Text::CSV::Separator". Microsoft uses the
2521 system's list separator defined in the regional settings, which happens
2522 to be a semicolon for Dutch, German and Spanish (and probably some
2523 others as well). For the English locale, the default is a comma.
2524 In Windows however, the user is free to choose a predefined locale,
2525 and then change every individual setting in it, so checking the
2526 locale is no solution.
2527 As of version 1.17, a lone first line with just
2529 sep=;
2531 will be recognized and honored when parsing with "getline".
2533
2535 More Errors & Warnings
2536 New extensions ought to be clear and concise in reporting what
2537 error has occurred where and why, and maybe also offer a remedy to
2538 the problem.
2539 "error_diag" is a (very) good start, but there is more work to be
2541 done in this area.
2542 Basic calls should croak or warn on illegal parameters. Errors
2544 should be documented.
2545 setting meta info
2547 Future extensions might include extending the "meta_info",
2548 "is_quoted", and "is_binary" to accept setting these flags for
2549 fields, so you can specify which fields are quoted in the
2550 "combine"/"string" combination.
2551 $csv->meta_info (0, 1, 1, 3, 0, 0);
2553 $csv->is_quoted (3, 1);
2554 Metadata Vocabulary for Tabular Data
2556 <http://w3c.github.io/csvw/metadata/> (a W3C editor's draft) could be
2557 an example for supporting more metadata.
2558 Parse the whole file at once
2560 Implement new methods or functions that enable parsing of a
2561 complete file at once, returning a list of hashes. Possible extension
2562 to this could be to enable a column selection on the call:
2563 my @AoH = $csv->parse_file ($filename, { cols => [ 1, 4..8, 12 ]});
2565 Returning something like
2567 [ { fields => [ 1, 2, "foo", 4.5, undef, "", 8 ],
2569 flags => [ ... ],
2570 },
2571 { fields => [ ... ],
2572 .
2573 },
2574 ]
2575 Note that the "csv" function already supports most of this, but does
2577 not return flags. "getline_all" returns all rows for an open stream,
2578 but this will not return flags either. "fragment" can reduce the
2579 required rows or columns, but cannot combine them.
2580 Cookbook
2582 Write a document that has recipes for most known non-standard (and
2583 maybe some standard) "CSV" formats, including formats that use
2584 "TAB", ";", "|", or other non-comma separators.
2585 Examples could be taken from W3C's CSV on the Web: Use Cases and
2587 Requirements <http://w3c.github.io/csvw/use-cases-and-
2588 requirements/index.html>
2589 Steal
2591 Steal good new ideas and features from PapaParse
2592 <http://papaparse.com> or csvkit <http://csvkit.readthedocs.org>.
2593 Perl6 support
2595 I'm already working on perl6 support here
2596 <https://github.com/Tux/CSV>. No promises yet on when it is finished
2597 (or fast). Trying to keep the API alike as much as possible.
2598 NOT TODO
2600 combined methods
2601 Requests for adding means (methods) that combine "combine" and
2602 "string" in a single call will not be honored (use "print" instead).
2603 Likewise for "parse" and "fields" (use "getline" instead), given the
2604 problems with embedded newlines.
2605 Release plan
2607 No guarantees, but this is what I had in mind some time ago:
2608 · DIAGNOSTICS section in pod to *describe* the errors (see below)
2610
2612 The current hard-coding of characters and character ranges makes this
2613 code unusable on "EBCDIC" systems. Recent work in perl-5.20 might
2614 change that.
2615 Opening "EBCDIC" encoded files on "ASCII"+ systems is likely to
2617 succeed using Encode's "cp37", "cp1047", or "posix-bc":
2618 open my $fh, "<:encoding(cp1047)", "ebcdic_file.csv" or die "...";
2620
2622 Still under construction ...
2623 If an error occurs, "$csv->error_diag" can be used to get information
2625 on the cause of the failure. Note that for speed reasons the internal
2626 value is never cleared on success, so using the value returned by
2627 "error_diag" in normal cases - when no error occurred - may cause
2628 unexpected results.
2629 If the constructor failed, the cause can be found using "error_diag" as
2631 a class method, like "Text::CSV_XS->error_diag".
2632 The "$csv->error_diag" method is automatically invoked upon error when
2634 the contractor was called with "auto_diag" set to 1 or 2, or when
2635 autodie is in effect. When set to 1, this will cause a "warn" with the
2636 error message, when set to 2, it will "die". "2012 - EOF" is excluded
2637 from "auto_diag" reports.
2638 Errors can be (individually) caught using the "error" callback.
2640 The errors as described below are available. I have tried to make the
2642 error itself explanatory enough, but more descriptions will be added.
2643 For most of these errors, the first three capitals describe the error
2644 category:
2645 · INI
2647 Initialization error or option conflict.
2649 · ECR
2651 Carriage-Return related parse error.
2653 · EOF
2655 End-Of-File related parse error.
2657 · EIQ
2659 Parse error inside quotation.
2661 · EIF
2663 Parse error inside field.
2665 · ECB
2667 Combine error.
2669 · EHR
2671 HashRef parse related error.
2673 And below should be the complete list of error codes that can be
2675 returned:
2676 · 1001 "INI - sep_char is equal to quote_char or escape_char"
2678 The separation character cannot be equal to the quotation
2680 character or to the escape character, as this would invalidate all
2681 parsing rules.
2682 · 1002 "INI - allow_whitespace with escape_char or quote_char SP or
2684 TAB"
2685 Using the "allow_whitespace" attribute when either "quote_char" or
2687 "escape_char" is equal to "SPACE" or "TAB" is too ambiguous to
2688 allow.
2689 · 1003 "INI - \r or \n in main attr not allowed"
2691 Using default "eol" characters in either "sep_char", "quote_char",
2693 or "escape_char" is not allowed.
2694 · 1004 "INI - callbacks should be undef or a hashref"
2696 The "callbacks" attribute only allows one to be "undef" or a hash
2698 reference.
2699 · 1005 "INI - EOL too long"
2701 The value passed for EOL is exceeding its maximum length (16).
2703 · 1006 "INI - SEP too long"
2705 The value passed for SEP is exceeding its maximum length (16).
2707 · 1007 "INI - QUOTE too long"
2709 The value passed for QUOTE is exceeding its maximum length (16).
2711 · 1008 "INI - SEP undefined"
2713 The value passed for SEP should be defined and not empty.
2715 · 1010 "INI - the header is empty"
2717 The header line parsed in the "header" is empty.
2719 · 1011 "INI - the header contains more than one valid separator"
2721 The header line parsed in the "header" contains more than one
2723 (unique) separator character out of the allowed set of separators.
2724 · 1012 "INI - the header contains an empty field"
2726 The header line parsed in the "header" is contains an empty field.
2728 · 1013 "INI - the header contains nun-unique fields"
2730 The header line parsed in the "header" contains at least two
2732 identical fields.
2733 · 1014 "INI - header called on undefined stream"
2735 The header line cannot be parsed from an undefined sources.
2737 · 1500 "PRM - Invalid/unsupported argument(s)"
2739 Function or method called with invalid argument(s) or parameter(s).
2741 · 1501 "PRM - The key attribute is passed as an unsupported type"
2743 The "key" attribute is of an unsupported type.
2745 · 1502 "PRM - The value attribute is passed without the key attribute"
2747 The "value" attribute is only allowed when a valid key is given.
2749 · 1503 "PRM - The value attribute is passed as an unsupported type"
2751 The "value" attribute is of an unsupported type.
2753 · 2010 "ECR - QUO char inside quotes followed by CR not part of EOL"
2755 When "eol" has been set to anything but the default, like
2757 "\r\t\n", and the "\r" is following the second (closing)
2758 "quote_char", where the characters following the "\r" do not make up
2759 the "eol" sequence, this is an error.
2760 · 2011 "ECR - Characters after end of quoted field"
2762 Sequences like "1,foo,"bar"baz,22,1" are not allowed. "bar" is a
2764 quoted field and after the closing double-quote, there should be
2765 either a new-line sequence or a separation character.
2766 · 2012 "EOF - End of data in parsing input stream"
2768 Self-explaining. End-of-file while inside parsing a stream. Can
2770 happen only when reading from streams with "getline", as using
2771 "parse" is done on strings that are not required to have a trailing
2772 "eol".
2773 · 2013 "INI - Specification error for fragments RFC7111"
2775 Invalid specification for URI "fragment" specification.
2777 · 2014 "ENF - Inconsistent number of fields"
2779 Inconsistent number of fields under strict parsing.
2781 · 2021 "EIQ - NL char inside quotes, binary off"
2783 Sequences like "1,"foo\nbar",22,1" are allowed only when the binary
2785 option has been selected with the constructor.
2786 · 2022 "EIQ - CR char inside quotes, binary off"
2788 Sequences like "1,"foo\rbar",22,1" are allowed only when the binary
2790 option has been selected with the constructor.
2791 · 2023 "EIQ - QUO character not allowed"
2793 Sequences like ""foo "bar" baz",qu" and "2023,",2008-04-05,"Foo,
2795 Bar",\n" will cause this error.
2796 · 2024 "EIQ - EOF cannot be escaped, not even inside quotes"
2798 The escape character is not allowed as last character in an input
2800 stream.
2801 · 2025 "EIQ - Loose unescaped escape"
2803 An escape character should escape only characters that need escaping.
2805 Allowing the escape for other characters is possible with the
2807 attribute "allow_loose_escapes".
2808 · 2026 "EIQ - Binary character inside quoted field, binary off"
2810 Binary characters are not allowed by default. Exceptions are
2812 fields that contain valid UTF-8, that will automatically be upgraded
2813 if the content is valid UTF-8. Set "binary" to 1 to accept binary
2814 data.
2815 · 2027 "EIQ - Quoted field not terminated"
2817 When parsing a field that started with a quotation character, the
2819 field is expected to be closed with a quotation character. When the
2820 parsed line is exhausted before the quote is found, that field is not
2821 terminated.
2822 · 2030 "EIF - NL char inside unquoted verbatim, binary off"
2824 · 2031 "EIF - CR char is first char of field, not part of EOL"
2826 · 2032 "EIF - CR char inside unquoted, not part of EOL"
2828 · 2034 "EIF - Loose unescaped quote"
2830 · 2035 "EIF - Escaped EOF in unquoted field"
2832 · 2036 "EIF - ESC error"
2834 · 2037 "EIF - Binary character in unquoted field, binary off"
2836 · 2110 "ECB - Binary character in Combine, binary off"
2838 · 2200 "EIO - print to IO failed. See errno"
2840 · 3001 "EHR - Unsupported syntax for column_names ()"
2842 · 3002 "EHR - getline_hr () called before column_names ()"
2844 · 3003 "EHR - bind_columns () and column_names () fields count
2846 mismatch"
2847 · 3004 "EHR - bind_columns () only accepts refs to scalars"
2849 · 3006 "EHR - bind_columns () did not pass enough refs for parsed
2851 fields"
2852 · 3007 "EHR - bind_columns needs refs to writable scalars"
2854 · 3008 "EHR - unexpected error in bound fields"
2856 · 3009 "EHR - print_hr () called before column_names ()"
2858 · 3010 "EHR - print_hr () called with invalid arguments"
2860
2862 IO::File, IO::Handle, IO::Wrap, Text::CSV, Text::CSV_PP,
2863 Text::CSV::Encoded, Text::CSV::Separator, Text::CSV::Slurp,
2864 Spreadsheet::CSV and Spreadsheet::Read, and of course perl.
2865 If you are using perl6, you can have a look at "Text::CSV" in the
2867 perl6 ecosystem, offering the same features.
2868 non-perl
2870 A CSV parser in JavaScript, also used by W3C <http://www.w3.org>, is
2872 the multi-threaded in-browser PapaParse <http://papaparse.com/>.
2873 csvkit <http://csvkit.readthedocs.org> is a python CSV parsing toolkit.
2875
2877 Alan Citterman <alan@mfgrtl.com> wrote the original Perl module.
2878 Please don't send mail concerning Text::CSV_XS to Alan, who is not
2879 involved in the C/XS part that is now the main part of the module.
2880 Jochen Wiedmann <joe@ispsoft.de> rewrote the en- and decoding in C by
2882 implementing a simple finite-state machine. He added variable quote,
2883 escape and separator characters, the binary mode and the print and
2884 getline methods. See ChangeLog releases 0.10 through 0.23.
2885 H.Merijn Brand <h.m.brand@xs4all.nl> cleaned up the code, added the
2887 field flags methods, wrote the major part of the test suite, completed
2888 the documentation, fixed most RT bugs, added all the allow flags and
2889 the "csv" function. See ChangeLog releases 0.25 and on.
2890
2892 Copyright (C) 2007-2019 H.Merijn Brand. All rights reserved.
2893 Copyright (C) 1998-2001 Jochen Wiedmann. All rights reserved.
2894 Copyright (C) 1997 Alan Citterman. All rights reserved.
2895 This library is free software; you can redistribute and/or modify it
2897 under the same terms as Perl itself.
2898perl v5.28.1 2019-02-27 CSV_XS(3)