1IO::HTML(3)           User Contributed Perl Documentation          IO::HTML(3)
2
3
4

NAME

6       IO::HTML - Open an HTML file with automatic charset detection
7

VERSION

9       This document describes version 1.00 of IO::HTML, released February 23,
10       2013.
11

SYNOPSIS

13         use IO::HTML;                 # exports html_file by default
14         use HTML::TreeBuilder;
15
16         my $tree = HTML::TreeBuilder->new_from_file(
17                      html_file('foo.html')
18                    );
19
20         # Alternative interface:
21         open(my $in, '<:raw', 'bar.html');
22         my $encoding = IO::HTML::sniff_encoding($in, 'bar.html');
23

DESCRIPTION

25       IO::HTML provides an easy way to open a file containing HTML while
26       automatically determining its encoding.  It uses the HTML5 encoding
27       sniffing algorithm specified in section 8.2.2.1 of the draft standard.
28
29       The algorithm as implemented here is:
30
31       1.  If the file begins with a byte order mark indicating UTF-16LE,
32           UTF-16BE, or UTF-8, then that is the encoding.
33
34       2.  If the first 1024 bytes of the file contain a "<meta>" tag that
35           indicates the charset, and Encode recognizes the specified charset
36           name, then that is the encoding.  (This portion of the algorithm is
37           implemented by "find_charset_in".)
38
39           The "<meta>" tag can be in one of two formats:
40
41             <meta charset="...">
42             <meta http-equiv="Content-Type" content="...charset=...">
43
44           The search is case-insensitive, and the order of attributes within
45           the tag is irrelevant.  Any additional attributes of the tag are
46           ignored.  The first matching tag with a recognized encoding ends
47           the search.
48
49       3.  If the first 1024 bytes of the file are valid UTF-8 (with at least
50           1 non-ASCII character), then the encoding is UTF-8.
51
52       4.  If all else fails, use the default character encoding.  The HTML5
53           standard suggests the default encoding should be locale dependent,
54           but currently it is always "cp1252" unless you set
55           $IO::HTML::default_encoding to a different value.  Note:
56           "sniff_encoding" does not apply this step; only "html_file" does
57           that.
58

SUBROUTINES

60   html_file
61         $filehandle = html_file($filename, \%options);
62
63       This function (exported by default) is the primary entry point.  It
64       opens the file specified by $filename for reading, uses
65       "sniff_encoding" to find a suitable encoding layer, and applies it.  It
66       also applies the ":crlf" layer.  If the file begins with a BOM, the
67       filehandle is positioned just after the BOM.
68
69       The optional second argument is a hashref containing options.  The
70       possible keys are described under "find_charset_in".
71
72       If "sniff_encoding" is unable to determine the encoding, it defaults to
73       $IO::HTML::default_encoding, which is set to "cp1252" (a.k.a.
74       Windows-1252) by default.  According to the standard, the default
75       should be locale dependent, but that is not currently implemented.
76
77       It dies if the file cannot be opened.
78
79   html_file_and_encoding
80         ($filehandle, $encoding, $bom)
81           = html_file_and_encoding($filename, \%options);
82
83       This function (exported only by request) is just like "html_file", but
84       returns more information.  In addition to the filehandle, it returns
85       the name of the encoding used, and a flag indicating whether a byte
86       order mark was found (if $bom is true, the file began with a BOM).
87       This may be useful if you want to write the file out again (especially
88       in conjunction with the "html_outfile" function).
89
90       The optional second argument is a hashref containing options.  The
91       possible keys are described under "find_charset_in".
92
93       It dies if the file cannot be opened.  The result of calling it in
94       scalar context is undefined.
95
96   html_outfile
97         $filehandle = html_outfile($filename, $encoding, $bom);
98
99       This function (exported only by request) opens $filename for output
100       using $encoding, and writes a BOM to it if $bom is true.  If $encoding
101       is "undef", it defaults to $IO::HTML::default_encoding.  $encoding may
102       be either an encoding name or an Encode::Encoding object.
103
104       It dies if the file cannot be opened.
105
106   sniff_encoding
107         ($encoding, $bom) = sniff_encoding($filehandle, $filename, \%options);
108
109       This function (exported only by request) runs the HTML5 encoding
110       sniffing algorithm on $filehandle (which must be seekable, and should
111       have been opened in ":raw" mode).  $filename is used only for error
112       messages (if there's a problem using the filehandle), and defaults to
113       "file" if omitted.  The optional third argument is a hashref containing
114       options.  The possible keys are described under "find_charset_in".
115
116       It returns Perl's canonical name for the encoding, which is not
117       necessarily the same as the MIME or IANA charset name.  It returns
118       "undef" if the encoding cannot be determined.  $bom is true if the file
119       began with a byte order mark.  In scalar context, it returns only
120       $encoding.
121
122       The filehandle's position is restored to its original position
123       (normally the beginning of the file) unless $bom is true.  In that
124       case, the position is immediately after the BOM.
125
126       Tip: If you want to run "sniff_encoding" on a file you've already
127       loaded into a string, open an in-memory file on the string, and pass
128       that handle:
129
130         ($encoding, $bom) = do {
131           open(my $fh, '<', \$string);  sniff_encoding($fh)
132         };
133
134       (This only makes sense if $string contains bytes, not characters.)
135
136   find_charset_in
137         $encoding = find_charset_in($string_containing_HTML, \%options);
138
139       This function (exported only by request) looks for charset information
140       in a "<meta>" tag in a possibly incomplete HTML document using the "two
141       step" algorithm specified by HTML5.  It does not look for a BOM.  Only
142       the first 1024 bytes of the string are checked.
143
144       It returns Perl's canonical name for the encoding, which is not
145       necessarily the same as the MIME or IANA charset name.  It returns
146       "undef" if no charset is specified or if the specified charset is not
147       recognized by the Encode module.
148
149       The optional second argument is a hashref containing options.  The
150       following keys are recognized:
151
152       "encoding"
153           If true, return the Encode::Encoding object instead of its name.
154           Defaults to false.
155
156       "need_pragma"
157           If true (the default), follow the HTML5 spec and examine the
158           "content" attribute only of "<meta http-equiv="Content-Type"".  If
159           set to 0, relax the HTML5 spec, and look for "charset=" in the
160           "content" attribute of every meta tag.
161

EXPORTS

163       By default, only "html_file" is exported.  Other functions may be
164       exported on request.
165
166       For people who prefer not to export functions, all functions beginning
167       with "html_" have an alias without that prefix (e.g. you can call
168       "IO::HTML::file(...)" instead of "IO::HTML::html_file(...)".  These
169       aliases are not exportable.
170
171       The following export tags are available:
172
173       ":all"
174           All exportable functions.
175
176       ":rw"
177           "html_file", "html_file_and_encoding", "html_outfile".
178

SEE ALSO

180       The HTML5 specification, section 8.2.2.1 Determining the character
181       encoding:
182       <http://www.w3.org/TR/html5/parsing.html#determining-the-character-encoding>
183

DIAGNOSTICS

185       "Could not read %s: %s"
186           The specified file could not be read from for the reason specified
187           by $!.
188
189       "Could not seek %s: %s"
190           The specified file could not be rewound for the reason specified by
191           $!.
192
193       "Failed to open %s: %s"
194           The specified file could not be opened for reading for the reason
195           specified by $!.
196
197       "No default encoding specified"
198           The "sniff_encoding" algorithm didn't find an encoding to use, and
199           you set $IO::HTML::default_encoding to "undef".
200

CONFIGURATION AND ENVIRONMENT

202       IO::HTML requires no configuration files or environment variables.
203

DEPENDENCIES

205       IO::HTML has no non-core dependencies for Perl 5.8.7+.  With earlier
206       versions of Perl 5.8, you need to upgrade Encode to at least version
207       2.10, and you may need to upgrade Exporter to at least version 5.57.
208

INCOMPATIBILITIES

210       None reported.
211

BUGS AND LIMITATIONS

213       No bugs have been reported.
214

AUTHOR

216       Christopher J. Madsen  "<perl AT cjmweb.net>"
217
218       Please report any bugs or feature requests to
219       "<bug-IO-HTML AT rt.cpan.org>" or through the web interface at
220       <http://rt.cpan.org/Public/Bug/Report.html?Queue=IO-HTML>.
221
222       You can follow or contribute to IO-HTML's development at
223       <http://github.com/madsen/io-html>.
224
226       This software is copyright (c) 2013 by Christopher J. Madsen.
227
228       This is free software; you can redistribute it and/or modify it under
229       the same terms as the Perl 5 programming language system itself.
230

DISCLAIMER OF WARRANTY

232       BECAUSE THIS SOFTWARE IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY
233       FOR THE SOFTWARE, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT
234       WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER
235       PARTIES PROVIDE THE SOFTWARE "AS IS" WITHOUT WARRANTY OF ANY KIND,
236       EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
237       WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE
238       ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE SOFTWARE IS WITH
239       YOU. SHOULD THE SOFTWARE PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL
240       NECESSARY SERVICING, REPAIR, OR CORRECTION.
241
242       IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
243       WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR
244       REDISTRIBUTE THE SOFTWARE AS PERMITTED BY THE ABOVE LICENSE, BE LIABLE
245       TO YOU FOR DAMAGES, INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL, OR
246       CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OR INABILITY TO USE THE
247       SOFTWARE (INCLUDING BUT NOT LIMITED TO LOSS OF DATA OR DATA BEING
248       RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD PARTIES OR A
249       FAILURE OF THE SOFTWARE TO OPERATE WITH ANY OTHER SOFTWARE), EVEN IF
250       SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH
251       DAMAGES.
252
253
254
255perl v5.16.3                      2013-02-23                       IO::HTML(3)
Impressum