1Encode::Locale(3) User Contributed Perl Documentation Encode::Locale(3)
2
6 Encode::Locale - Determine the locale encoding
7
9 use Encode::Locale;
10 use Encode;
11 $string = decode(locale => $bytes);
13 $bytes = encode(locale => $string);
14 if (-t) {
16 binmode(STDIN, ":encoding(console_in)");
17 binmode(STDOUT, ":encoding(console_out)");
18 binmode(STDERR, ":encoding(console_out)");
19 }
20 # Processing file names passed in as arguments
22 my $uni_filename = decode(locale => $ARGV[0]);
23 open(my $fh, "<", encode(locale_fs => $uni_filename))
24 || die "Can't open '$uni_filename': $!";
25 binmode($fh, ":encoding(locale)");
26 ...
27
29 In many applications it's wise to let Perl use Unicode for the strings
30 it processes. Most of the interfaces Perl has to the outside world are
31 still byte based. Programs therefore need to decode byte strings that
32 enter the program from the outside and encode them again on the way
33 out.
34 The POSIX locale system is used to specify both the language
36 conventions requested by the user and the preferred character set to
37 consume and output. The "Encode::Locale" module looks up the charset
38 and encoding (called a CODESET in the locale jargon) and arranges for
39 the Encode module to know this encoding under the name "locale". It
40 means bytes obtained from the environment can be converted to Unicode
41 strings by calling "Encode::encode(locale => $bytes)" and converted
42 back again with "Encode::decode(locale => $string)".
43 Where file systems interfaces pass file names in and out of the program
45 we also need care. The trend is for operating systems to use a fixed
46 file encoding that don't actually depend on the locale; and this module
47 determines the most appropriate encoding for file names. The Encode
48 module will know this encoding under the name "locale_fs". For
49 traditional Unix systems this will be an alias to the same encoding as
50 "locale".
51 For programs running in a terminal window (called a "Console" on some
53 systems) the "locale" encoding is usually a good choice for what to
54 expect as input and output. Some systems allows us to query the
55 encoding set for the terminal and "Encode::Locale" will do that if
56 available and make these encodings known under the "Encode" aliases
57 "console_in" and "console_out". For systems where we can't determine
58 the terminal encoding these will be aliased as the same encoding as
59 "locale". The advice is to use "console_in" for input known to come
60 from the terminal and "console_out" for output known to go from the
61 terminal.
62 In addition to arranging for various Encode aliases the following
64 functions and variables are provided:
65 decode_argv( )
67 decode_argv( Encode::FB_CROAK )
68 This will decode the command line arguments to perl (the @ARGV
69 array) in-place.
70 The function will by default replace characters that can't be
72 decoded by "\x{FFFD}", the Unicode replacement character.
73 Any argument provided is passed as CHECK to underlying
75 Encode::decode() call. Pass the value "Encode::FB_CROAK" to have
76 the decoding croak if not all the command line arguments can be
77 decoded. See "Handling Malformed Data" in Encode for details on
78 other options for CHECK.
79 env( $uni_key )
81 env( $uni_key => $uni_value )
82 Interface to get/set environment variables. Returns the current
83 value as a Unicode string. The $uni_key and $uni_value arguments
84 are expected to be Unicode strings as well. Passing "undef" as
85 $uni_value deletes the environment variable named $uni_key.
86 The returned value will have the characters that can't be decoded
88 replaced by "\x{FFFD}", the Unicode replacement character.
89 There is no interface to request alternative CHECK behavior as for
91 decode_argv(). If you need that you need to call encode/decode
92 yourself. For example:
93 my $key = Encode::encode(locale => $uni_key, Encode::FB_CROAK);
95 my $uni_value = Encode::decode(locale => $ENV{$key}, Encode::FB_CROAK);
96 reinit( )
98 reinit( $encoding )
99 Reinitialize the encodings from the locale. You want to call this
100 function if you changed anything in the environment that might
101 influence the locale.
102 This function will croak if the determined encoding isn't
104 recognized by the Encode module.
105 With argument force $ENCODING_... variables to set to the given
107 value.
108 $ENCODING_LOCALE
110 The encoding name determined to be suitable for the current locale.
111 Encode know this encoding as "locale".
112 $ENCODING_LOCALE_FS
114 The encoding name determined to be suiteable for file system
115 interfaces involving file names. Encode know this encoding as
116 "locale_fs".
117 $ENCODING_CONSOLE_IN
119 $ENCODING_CONSOLE_OUT
120 The encodings to be used for reading and writing output to the a
121 console. Encode know these encodings as "console_in" and
122 "console_out".
123
125 This table summarizes the mapping of the encodings set up by the
126 "Encode::Locale" module:
127 Encode | | |
129 Alias | Windows | Mac OS X | POSIX
130 ------------+---------+--------------+------------
131 locale | ANSI | nl_langinfo | nl_langinfo
132 locale_fs | ANSI | UTF-8 | nl_langinfo
133 console_in | OEM | nl_langinfo | nl_langinfo
134 console_out | OEM | nl_langinfo | nl_langinfo
135 Windows
137 Windows has basically 2 sets of APIs. A wide API (based on passing
138 UTF-16 strings) and a byte based API based a character set called ANSI.
139 The regular Perl interfaces to the OS currently only uses the ANSI
140 APIs. Unfortunately ANSI is not a single character set.
141 The encoding that corresponds to ANSI varies between different editions
143 of Windows. For many western editions of Windows ANSI corresponds to
144 CP-1252 which is a character set similar to ISO-8859-1. Conceptually
145 the ANSI character set is a similar concept to the POSIX locale CODESET
146 so this module figures out what the ANSI code page is and make this
147 available as $ENCODING_LOCALE and the "locale" Encoding alias.
148 Windows systems also operate with another byte based character set.
150 It's called the OEM code page. This is the encoding that the Console
151 takes as input and output. It's common for the OEM code page to differ
152 from the ANSI code page.
153 Mac OS X
155 On Mac OS X the file system encoding is always UTF-8 while the locale
156 can otherwise be set up as normal for POSIX systems.
157 File names on Mac OS X will at the OS-level be converted to NFD-form.
159 A file created by passing a NFC-filename will come in NFD-form from
160 readdir(). See Unicode::Normalize for details of NFD/NFC.
161 Actually, Apple does not follow the Unicode NFD standard since not all
163 character ranges are decomposed. The claim is that this avoids
164 problems with round trip conversions from old Mac text encodings. See
165 Encode::UTF8Mac for details.
166 POSIX (Linux and other Unixes)
168 File systems might vary in what encoding is to be used for filenames.
169 Since this module has no way to actually figure out what the is correct
170 it goes with the best guess which is to assume filenames are encoding
171 according to the current locale. Users are advised to always specify
172 UTF-8 as the locale charset.
173
175 I18N::Langinfo, Encode
176
178 Copyright 2010 Gisle Aas <gisle@aas.no>.
179 This library is free software; you can redistribute it and/or modify it
181 under the same terms as Perl itself.
182perl v5.16.3 2012-02-11 Encode::Locale(3)