encoding::warnings(3pm)

1encoding::warnings(3pm)Perl Programmers Reference Guideencoding::warnings(3pm)
2
3
4

NAME

6       encoding::warnings - Warn on implicit encoding conversions
7

VERSION

9       This document describes version 0.13 of encoding::warnings, released
10       June 20, 2016.
11

NOTICE

13       As of Perl 5.26.0, this module has no effect.  The internal Perl
14       feature that was used to implement this module has been removed.  In
15       recent years, much work has been done on the Perl core to eliminate
16       discrepancies in the treatment of upgraded versus downgraded strings.
17       In addition, the encoding pragma, which caused many of the problems, is
18       no longer supported.  Thus, the warnings this module produced are no
19       longer necessary.
20
21       Hence, if you load this module on Perl 5.26.0, you will get one warning
22       that the module is no longer supported; and the module will do nothing
23       thereafter.
24

SYNOPSIS

26           use encoding::warnings; # or 'FATAL' to raise fatal exceptions
27
28           utf8::encode($a = chr(20000));  # a byte-string (raw bytes)
29           $b = chr(20000);                # a unicode-string (wide characters)
30
31           # "Bytes implicitly upgraded into wide characters as iso-8859-1"
32           $c = $a . $b;
33

DESCRIPTION

35   Overview of the problem
36       By default, there is a fundamental asymmetry in Perl's unicode model:
37       implicit upgrading from byte-strings to unicode-strings assumes that
38       they were encoded in ISO 8859-1 (Latin-1), but unicode-strings are
39       downgraded with UTF-8 encoding.  This happens because the first 256
40       codepoints in Unicode happens to agree with Latin-1.
41
42       However, this silent upgrading can easily cause problems, if you happen
43       to mix unicode strings with non-Latin1 data -- i.e. byte-strings
44       encoded in UTF-8 or other encodings.  The error will not manifest until
45       the combined string is written to output, at which time it would be
46       impossible to see where did the silent upgrading occur.
47
48   Detecting the problem
49       This module simplifies the process of diagnosing such problems.  Just
50       put this line on top of your main program:
51
52           use encoding::warnings;
53
54       Afterwards, implicit upgrading of high-bit bytes will raise a warning.
55       Ex.: "Bytes implicitly upgraded into wide characters as iso-8859-1 at -
56       line 7".
57
58       However, strings composed purely of ASCII code points (0x00..0x7F) will
59       not trigger this warning.
60
61       You can also make the warnings fatal by importing this module as:
62
63           use encoding::warnings 'FATAL';
64
65   Solving the problem
66       Most of the time, this warning occurs when a byte-string is
67       concatenated with a unicode-string.  There are a number of ways to
68       solve it:
69
70       •   Upgrade both sides to unicode-strings
71
72           If your program does not need compatibility for Perl 5.6 and
73           earlier, the recommended approach is to apply appropriate IO
74           disciplines, so all data in your program become unicode-strings.
75           See encoding, open and "binmode" in perlfunc for how.
76
77       •   Downgrade both sides to byte-strings
78
79           The other way works too, especially if you are sure that all your
80           data are under the same encoding, or if compatibility with older
81           versions of Perl is desired.
82
83           You may downgrade strings with "Encode::encode" and "utf8::encode".
84           See Encode and utf8 for details.
85
86       •   Specify the encoding for implicit byte-string upgrading
87
88           If you are confident that all byte-strings will be in a specific
89           encoding like UTF-8, and need not support older versions of Perl,
90           use the "encoding" pragma:
91
92               use encoding 'utf8';
93
94           Similarly, this will silence warnings from this module, and
95           preserve the default behaviour:
96
97               use encoding 'iso-8859-1';
98
99           However, note that "use encoding" actually had three distinct
100           effects:
101
102           •   PerlIO layers for STDIN and STDOUT
103
104               This is similar to what open pragma does.
105
106           •   Literal conversions
107
108               This turns all literal string in your program into unicode-
109               strings (equivalent to a "use utf8"), by decoding them using
110               the specified encoding.
111
112           •   Implicit upgrading for byte-strings
113
114               This will silence warnings from this module, as shown above.
115
116           Because literal conversions also work on empty strings, it may
117           surprise some people:
118
119               use encoding 'big5';
120
121               my $byte_string = pack("C*", 0xA4, 0x40);
122               print length $a;    # 2 here.
123               $a .= "";           # concatenating with a unicode string...
124               print length $a;    # 1 here!
125
126           In other words, do not "use encoding" unless you are certain that
127           the program will not deal with any raw, 8-bit binary data at all.
128
129           However, the "Filter => 1" flavor of "use encoding" will not affect
130           implicit upgrading for byte-strings, and is thus incapable of
131           silencing warnings from this module.  See encoding for more
132           details.
133

CAVEATS

135       For Perl 5.9.4 or later, this module's effect is lexical.
136
137       For Perl versions prior to 5.9.4, this module affects the whole script,
138       instead of inside its lexical block.
139

AUTHORS

146       Audrey Tang
147

COPYRIGHT

149       Copyright 2004, 2005, 2006, 2007 by Audrey Tang <cpan@audreyt.org>.
150
151       This program is free software; you can redistribute it and/or modify it
152       under the same terms as Perl itself.
153
154       See <http://www.perl.com/perl/misc/Artistic.html>
155
156
157
158perl v5.34.1                      2022-03-15           encoding::warnings(3pm)