1Encode::Locale(3)     User Contributed Perl Documentation    Encode::Locale(3)
2
3
4

NAME

6       Encode::Locale - Determine the locale encoding
7

SYNOPSIS

9         use Encode::Locale;
10         use Encode;
11
12         $string = decode(locale => $bytes);
13         $bytes = encode(locale => $string);
14
15         if (-t) {
16             binmode(STDIN, ":encoding(console_in)");
17             binmode(STDOUT, ":encoding(console_out)");
18             binmode(STDERR, ":encoding(console_out)");
19         }
20
21         # Processing file names passed in as arguments
22         my $uni_filename = decode(locale => $ARGV[0]);
23         open(my $fh, "<", encode(locale_fs => $uni_filename))
24            || die "Can't open '$uni_filename': $!";
25         binmode($fh, ":encoding(locale)");
26         ...
27

DESCRIPTION

29       In many applications it's wise to let Perl use Unicode for the strings
30       it processes.  Most of the interfaces Perl has to the outside world are
31       still byte based.  Programs therefore need to decode byte strings that
32       enter the program from the outside and encode them again on the way
33       out.
34
35       The POSIX locale system is used to specify both the language
36       conventions requested by the user and the preferred character set to
37       consume and output.  The "Encode::Locale" module looks up the charset
38       and encoding (called a CODESET in the locale jargon) and arranges for
39       the Encode module to know this encoding under the name "locale".  It
40       means bytes obtained from the environment can be converted to Unicode
41       strings by calling "Encode::encode(locale => $bytes)" and converted
42       back again with "Encode::decode(locale => $string)".
43
44       Where file systems interfaces pass file names in and out of the program
45       we also need care.  The trend is for operating systems to use a fixed
46       file encoding that don't actually depend on the locale; and this module
47       determines the most appropriate encoding for file names. The Encode
48       module will know this encoding under the name "locale_fs".  For
49       traditional Unix systems this will be an alias to the same encoding as
50       "locale".
51
52       For programs running in a terminal window (called a "Console" on some
53       systems) the "locale" encoding is usually a good choice for what to
54       expect as input and output.  Some systems allows us to query the
55       encoding set for the terminal and "Encode::Locale" will do that if
56       available and make these encodings known under the "Encode" aliases
57       "console_in" and "console_out".  For systems where we can't determine
58       the terminal encoding these will be aliased as the same encoding as
59       "locale".  The advice is to use "console_in" for input known to come
60       from the terminal and "console_out" for output to the terminal.
61
62       In addition to arranging for various Encode aliases the following
63       functions and variables are provided:
64
65       decode_argv( )
66       decode_argv( Encode::FB_CROAK )
67           This will decode the command line arguments to perl (the @ARGV
68           array) in-place.
69
70           The function will by default replace characters that can't be
71           decoded by "\x{FFFD}", the Unicode replacement character.
72
73           Any argument provided is passed as CHECK to underlying
74           Encode::decode() call.  Pass the value "Encode::FB_CROAK" to have
75           the decoding croak if not all the command line arguments can be
76           decoded.  See "Handling Malformed Data" in Encode for details on
77           other options for CHECK.
78
79       env( $uni_key )
80       env( $uni_key => $uni_value )
81           Interface to get/set environment variables.  Returns the current
82           value as a Unicode string. The $uni_key and $uni_value arguments
83           are expected to be Unicode strings as well.  Passing "undef" as
84           $uni_value deletes the environment variable named $uni_key.
85
86           The returned value will have the characters that can't be decoded
87           replaced by "\x{FFFD}", the Unicode replacement character.
88
89           There is no interface to request alternative CHECK behavior as for
90           decode_argv().  If you need that you need to call encode/decode
91           yourself.  For example:
92
93               my $key = Encode::encode(locale => $uni_key, Encode::FB_CROAK);
94               my $uni_value = Encode::decode(locale => $ENV{$key}, Encode::FB_CROAK);
95
96       reinit( )
97       reinit( $encoding )
98           Reinitialize the encodings from the locale.  You want to call this
99           function if you changed anything in the environment that might
100           influence the locale.
101
102           This function will croak if the determined encoding isn't
103           recognized by the Encode module.
104
105           With argument force $ENCODING_... variables to set to the given
106           value.
107
108       $ENCODING_LOCALE
109           The encoding name determined to be suitable for the current locale.
110           Encode know this encoding as "locale".
111
112       $ENCODING_LOCALE_FS
113           The encoding name determined to be suitable for file system
114           interfaces involving file names.  Encode know this encoding as
115           "locale_fs".
116
117       $ENCODING_CONSOLE_IN
118       $ENCODING_CONSOLE_OUT
119           The encodings to be used for reading and writing output to the a
120           console.  Encode know these encodings as "console_in" and
121           "console_out".
122

NOTES

124       This table summarizes the mapping of the encodings set up by the
125       "Encode::Locale" module:
126
127         Encode      |         |              |
128         Alias       | Windows | Mac OS X     | POSIX
129         ------------+---------+--------------+------------
130         locale      | ANSI    | nl_langinfo  | nl_langinfo
131         locale_fs   | ANSI    | UTF-8        | nl_langinfo
132         console_in  | OEM     | nl_langinfo  | nl_langinfo
133         console_out | OEM     | nl_langinfo  | nl_langinfo
134
135   Windows
136       Windows has basically 2 sets of APIs.  A wide API (based on passing
137       UTF-16 strings) and a byte based API based a character set called ANSI.
138       The regular Perl interfaces to the OS currently only uses the ANSI
139       APIs.  Unfortunately ANSI is not a single character set.
140
141       The encoding that corresponds to ANSI varies between different editions
142       of Windows.  For many western editions of Windows ANSI corresponds to
143       CP-1252 which is a character set similar to ISO-8859-1.  Conceptually
144       the ANSI character set is a similar concept to the POSIX locale CODESET
145       so this module figures out what the ANSI code page is and make this
146       available as $ENCODING_LOCALE and the "locale" Encoding alias.
147
148       Windows systems also operate with another byte based character set.
149       It's called the OEM code page.  This is the encoding that the Console
150       takes as input and output.  It's common for the OEM code page to differ
151       from the ANSI code page.
152
153   Mac OS X
154       On Mac OS X the file system encoding is always UTF-8 while the locale
155       can otherwise be set up as normal for POSIX systems.
156
157       File names on Mac OS X will at the OS-level be converted to NFD-form.
158       A file created by passing a NFC-filename will come in NFD-form from
159       readdir().  See Unicode::Normalize for details of NFD/NFC.
160
161       Actually, Apple does not follow the Unicode NFD standard since not all
162       character ranges are decomposed.  The claim is that this avoids
163       problems with round trip conversions from old Mac text encodings.  See
164       Encode::UTF8Mac for details.
165
166   POSIX (Linux and other Unixes)
167       File systems might vary in what encoding is to be used for filenames.
168       Since this module has no way to actually figure out what the is correct
169       it goes with the best guess which is to assume filenames are encoding
170       according to the current locale.  Users are advised to always specify
171       UTF-8 as the locale charset.
172

SEE ALSO

174       I18N::Langinfo, Encode, Term::Encoding
175

AUTHOR

177       Copyright 2010 Gisle Aas <gisle@aas.no>.
178
179       This library is free software; you can redistribute it and/or modify it
180       under the same terms as Perl itself.
181
182
183
184perl v5.36.0                      2023-01-20                 Encode::Locale(3)
Impressum