1Encode(3) User Contributed Perl Documentation Encode(3)
2
3
4
6 Encode - character encodings in Perl
7
9 use Encode qw(decode encode);
10 $characters = decode('UTF-8', $octets, Encode::FB_CROAK);
11 $octets = encode('UTF-8', $characters, Encode::FB_CROAK);
12
13 Table of Contents
14 Encode consists of a collection of modules whose details are too
15 extensive to fit in one document. This one itself explains the top-
16 level APIs and general topics at a glance. For other topics and more
17 details, see the documentation for these modules:
18
19 Encode::Alias - Alias definitions to encodings
20 Encode::Encoding - Encode Implementation Base Class
21 Encode::Supported - List of Supported Encodings
22 Encode::CN - Simplified Chinese Encodings
23 Encode::JP - Japanese Encodings
24 Encode::KR - Korean Encodings
25 Encode::TW - Traditional Chinese Encodings
26
28 The "Encode" module provides the interface between Perl strings and the
29 rest of the system. Perl strings are sequences of characters.
30
31 The repertoire of characters that Perl can represent is a superset of
32 those defined by the Unicode Consortium. On most platforms the ordinal
33 values of a character as returned by "ord(S)" is the Unicode codepoint
34 for that character. The exceptions are platforms where the legacy
35 encoding is some variant of EBCDIC rather than a superset of ASCII; see
36 perlebcdic.
37
38 During recent history, data is moved around a computer in 8-bit chunks,
39 often called "bytes" but also known as "octets" in standards documents.
40 Perl is widely used to manipulate data of many types: not only strings
41 of characters representing human or computer languages, but also
42 "binary" data, being the machine's representation of numbers, pixels in
43 an image, or just about anything.
44
45 When Perl is processing "binary data", the programmer wants Perl to
46 process "sequences of bytes". This is not a problem for Perl: because a
47 byte has 256 possible values, it easily fits in Perl's much larger
48 "logical character".
49
50 This document mostly explains the how. perlunitut and perlunifaq
51 explain the why.
52
53 TERMINOLOGY
54 character
55
56 A character in the range 0 .. 2**32-1 (or more); what Perl's strings
57 are made of.
58
59 byte
60
61 A character in the range 0..255; a special case of a Perl character.
62
63 octet
64
65 8 bits of data, with ordinal values 0..255; term for bytes passed to or
66 from a non-Perl context, such as a disk file, standard I/O stream,
67 database, command-line argument, environment variable, socket etc.
68
70 Basic methods
71 encode
72
73 $octets = encode(ENCODING, STRING[, CHECK])
74
75 Encodes the scalar value STRING from Perl's internal form into ENCODING
76 and returns a sequence of octets. ENCODING can be either a canonical
77 name or an alias. For encoding names and aliases, see "Defining
78 Aliases". For CHECK, see "Handling Malformed Data".
79
80 CAVEAT: the input scalar STRING might be modified in-place depending on
81 what is set in CHECK. See "LEAVE_SRC" if you want your inputs to be
82 left unchanged.
83
84 For example, to convert a string from Perl's internal format into
85 ISO-8859-1, also known as Latin1:
86
87 $octets = encode("iso-8859-1", $string);
88
89 CAVEAT: When you run "$octets = encode("UTF-8", $string)", then $octets
90 might not be equal to $string. Though both contain the same data, the
91 UTF8 flag for $octets is always off. When you encode anything, the
92 UTF8 flag on the result is always off, even when it contains a
93 completely valid UTF-8 string. See "The UTF8 flag" below.
94
95 If the $string is "undef", then "undef" is returned.
96
97 "str2bytes" may be used as an alias for "encode".
98
99 decode
100
101 $string = decode(ENCODING, OCTETS[, CHECK])
102
103 This function returns the string that results from decoding the scalar
104 value OCTETS, assumed to be a sequence of octets in ENCODING, into
105 Perl's internal form. As with encode(), ENCODING can be either a
106 canonical name or an alias. For encoding names and aliases, see
107 "Defining Aliases"; for CHECK, see "Handling Malformed Data".
108
109 CAVEAT: the input scalar OCTETS might be modified in-place depending on
110 what is set in CHECK. See "LEAVE_SRC" if you want your inputs to be
111 left unchanged.
112
113 For example, to convert ISO-8859-1 data into a string in Perl's
114 internal format:
115
116 $string = decode("iso-8859-1", $octets);
117
118 CAVEAT: When you run "$string = decode("UTF-8", $octets)", then $string
119 might not be equal to $octets. Though both contain the same data, the
120 UTF8 flag for $string is on. See "The UTF8 flag" below.
121
122 If the $string is "undef", then "undef" is returned.
123
124 "bytes2str" may be used as an alias for "decode".
125
126 find_encoding
127
128 [$obj =] find_encoding(ENCODING)
129
130 Returns the encoding object corresponding to ENCODING. Returns "undef"
131 if no matching ENCODING is find. The returned object is what does the
132 actual encoding or decoding.
133
134 $string = decode($name, $bytes);
135
136 is in fact
137
138 $string = do {
139 $obj = find_encoding($name);
140 croak qq(encoding "$name" not found) unless ref $obj;
141 $obj->decode($bytes);
142 };
143
144 with more error checking.
145
146 You can therefore save time by reusing this object as follows;
147
148 my $enc = find_encoding("iso-8859-1");
149 while(<>) {
150 my $string = $enc->decode($_);
151 ... # now do something with $string;
152 }
153
154 Besides "decode" and "encode", other methods are available as well.
155 For instance, "name()" returns the canonical name of the encoding
156 object.
157
158 find_encoding("latin1")->name; # iso-8859-1
159
160 See Encode::Encoding for details.
161
162 find_mime_encoding
163
164 [$obj =] find_mime_encoding(MIME_ENCODING)
165
166 Returns the encoding object corresponding to MIME_ENCODING. Acts same
167 as "find_encoding()" but "mime_name()" of returned object must match to
168 MIME_ENCODING. So as opposite of "find_encoding()" canonical names and
169 aliases are not used when searching for object.
170
171 find_mime_encoding("utf8"); # returns undef because "utf8" is not valid I<MIME_ENCODING>
172 find_mime_encoding("utf-8"); # returns encode object "utf-8-strict"
173 find_mime_encoding("UTF-8"); # same as "utf-8" because I<MIME_ENCODING> is case insensitive
174 find_mime_encoding("utf-8-strict"); returns undef because "utf-8-strict" is not valid I<MIME_ENCODING>
175
176 from_to
177
178 [$length =] from_to($octets, FROM_ENC, TO_ENC [, CHECK])
179
180 Converts in-place data between two encodings. The data in $octets must
181 be encoded as octets and not as characters in Perl's internal format.
182 For example, to convert ISO-8859-1 data into Microsoft's CP1250
183 encoding:
184
185 from_to($octets, "iso-8859-1", "cp1250");
186
187 and to convert it back:
188
189 from_to($octets, "cp1250", "iso-8859-1");
190
191 Because the conversion happens in place, the data to be converted
192 cannot be a string constant: it must be a scalar variable.
193
194 "from_to()" returns the length of the converted string in octets on
195 success, and "undef" on error.
196
197 CAVEAT: The following operations may look the same, but are not:
198
199 from_to($data, "iso-8859-1", "UTF-8"); #1
200 $data = decode("iso-8859-1", $data); #2
201
202 Both #1 and #2 make $data consist of a completely valid UTF-8 string,
203 but only #2 turns the UTF8 flag on. #1 is equivalent to:
204
205 $data = encode("UTF-8", decode("iso-8859-1", $data));
206
207 See "The UTF8 flag" below.
208
209 Also note that:
210
211 from_to($octets, $from, $to, $check);
212
213 is equivalent to:
214
215 $octets = encode($to, decode($from, $octets), $check);
216
217 Yes, it does not respect the $check during decoding. It is
218 deliberately done that way. If you need minute control, use "decode"
219 followed by "encode" as follows:
220
221 $octets = encode($to, decode($from, $octets, $check_from), $check_to);
222
223 encode_utf8
224
225 $octets = encode_utf8($string);
226
227 WARNING: This function can produce invalid UTF-8! Do not use it for
228 data exchange. Unless you want Perl's older "lax" mode, prefer
229 "$octets = encode("UTF-8", $string)".
230
231 Equivalent to "$octets = encode("utf8", $string)". The characters in
232 $string are encoded in Perl's internal format, and the result is
233 returned as a sequence of octets. Because all possible characters in
234 Perl have a (loose, not strict) utf8 representation, this function
235 cannot fail.
236
237 decode_utf8
238
239 $string = decode_utf8($octets [, CHECK]);
240
241 WARNING: This function accepts invalid UTF-8! Do not use it for data
242 exchange. Unless you want Perl's older "lax" mode, prefer "$string =
243 decode("UTF-8", $octets [, CHECK])".
244
245 Equivalent to "$string = decode("utf8", $octets [, CHECK])". The
246 sequence of octets represented by $octets is decoded from (loose, not
247 strict) utf8 into a sequence of logical characters. Because not all
248 sequences of octets are valid not strict utf8, it is quite possible for
249 this function to fail. For CHECK, see "Handling Malformed Data".
250
251 CAVEAT: the input $octets might be modified in-place depending on what
252 is set in CHECK. See "LEAVE_SRC" if you want your inputs to be left
253 unchanged.
254
255 Listing available encodings
256 use Encode;
257 @list = Encode->encodings();
258
259 Returns a list of canonical names of available encodings that have
260 already been loaded. To get a list of all available encodings
261 including those that have not yet been loaded, say:
262
263 @all_encodings = Encode->encodings(":all");
264
265 Or you can give the name of a specific module:
266
267 @with_jp = Encode->encodings("Encode::JP");
268
269 When ""::"" is not in the name, ""Encode::"" is assumed.
270
271 @ebcdic = Encode->encodings("EBCDIC");
272
273 To find out in detail which encodings are supported by this package,
274 see Encode::Supported.
275
276 Defining Aliases
277 To add a new alias to a given encoding, use:
278
279 use Encode;
280 use Encode::Alias;
281 define_alias(NEWNAME => ENCODING);
282
283 After that, NEWNAME can be used as an alias for ENCODING. ENCODING may
284 be either the name of an encoding or an encoding object.
285
286 Before you do that, first make sure the alias is nonexistent using
287 "resolve_alias()", which returns the canonical name thereof. For
288 example:
289
290 Encode::resolve_alias("latin1") eq "iso-8859-1" # true
291 Encode::resolve_alias("iso-8859-12") # false; nonexistent
292 Encode::resolve_alias($name) eq $name # true if $name is canonical
293
294 "resolve_alias()" does not need "use Encode::Alias"; it can be imported
295 via "use Encode qw(resolve_alias)".
296
297 See Encode::Alias for details.
298
299 Finding IANA Character Set Registry names
300 The canonical name of a given encoding does not necessarily agree with
301 IANA Character Set Registry, commonly seen as "Content-Type:
302 text/plain; charset=WHATEVER". For most cases, the canonical name
303 works, but sometimes it does not, most notably with "utf-8-strict".
304
305 As of "Encode" version 2.21, a new method "mime_name()" is therefore
306 added.
307
308 use Encode;
309 my $enc = find_encoding("UTF-8");
310 warn $enc->name; # utf-8-strict
311 warn $enc->mime_name; # UTF-8
312
313 See also: Encode::Encoding
314
316 If your perl supports "PerlIO" (which is the default), you can use a
317 "PerlIO" layer to decode and encode directly via a filehandle. The
318 following two examples are fully identical in functionality:
319
320 ### Version 1 via PerlIO
321 open(INPUT, "< :encoding(shiftjis)", $infile)
322 || die "Can't open < $infile for reading: $!";
323 open(OUTPUT, "> :encoding(euc-jp)", $outfile)
324 || die "Can't open > $output for writing: $!";
325 while (<INPUT>) { # auto decodes $_
326 print OUTPUT; # auto encodes $_
327 }
328 close(INPUT) || die "can't close $infile: $!";
329 close(OUTPUT) || die "can't close $outfile: $!";
330
331 ### Version 2 via from_to()
332 open(INPUT, "< :raw", $infile)
333 || die "Can't open < $infile for reading: $!";
334 open(OUTPUT, "> :raw", $outfile)
335 || die "Can't open > $output for writing: $!";
336
337 while (<INPUT>) {
338 from_to($_, "shiftjis", "euc-jp", 1); # switch encoding
339 print OUTPUT; # emit raw (but properly encoded) data
340 }
341 close(INPUT) || die "can't close $infile: $!";
342 close(OUTPUT) || die "can't close $outfile: $!";
343
344 In the first version above, you let the appropriate encoding layer
345 handle the conversion. In the second, you explicitly translate from
346 one encoding to the other.
347
348 Unfortunately, it may be that encodings are not "PerlIO"-savvy. You
349 can check to see whether your encoding is supported by "PerlIO" by
350 invoking the "perlio_ok" method on it:
351
352 Encode::perlio_ok("hz"); # false
353 find_encoding("euc-cn")->perlio_ok; # true wherever PerlIO is available
354
355 use Encode qw(perlio_ok); # imported upon request
356 perlio_ok("euc-jp")
357
358 Fortunately, all encodings that come with "Encode" core are
359 "PerlIO"-savvy except for "hz" and "ISO-2022-kr". For the gory
360 details, see Encode::Encoding and Encode::PerlIO.
361
363 The optional CHECK argument tells "Encode" what to do when encountering
364 malformed data. Without CHECK, "Encode::FB_DEFAULT" (== 0) is assumed.
365
366 As of version 2.12, "Encode" supports coderef values for "CHECK"; see
367 below.
368
369 NOTE: Not all encodings support this feature. Some encodings ignore
370 the CHECK argument. For example, Encode::Unicode ignores CHECK and it
371 always croaks on error.
372
373 List of CHECK values
374 FB_DEFAULT
375
376 I<CHECK> = Encode::FB_DEFAULT ( == 0)
377
378 If CHECK is 0, encoding and decoding replace any malformed character
379 with a substitution character. When you encode, SUBCHAR is used. When
380 you decode, the Unicode REPLACEMENT CHARACTER, code point U+FFFD, is
381 used. If the data is supposed to be UTF-8, an optional lexical warning
382 of warning category "utf8" is given.
383
384 FB_CROAK
385
386 I<CHECK> = Encode::FB_CROAK ( == 1)
387
388 If CHECK is 1, methods immediately die with an error message.
389 Therefore, when CHECK is 1, you should trap exceptions with "eval{}",
390 unless you really want to let it "die".
391
392 FB_QUIET
393
394 I<CHECK> = Encode::FB_QUIET
395
396 If CHECK is set to "Encode::FB_QUIET", encoding and decoding
397 immediately return the portion of the data that has been processed so
398 far when an error occurs. The data argument is overwritten with
399 everything after that point; that is, the unprocessed portion of the
400 data. This is handy when you have to call "decode" repeatedly in the
401 case where your source data may contain partial multi-byte character
402 sequences, (that is, you are reading with a fixed-width buffer). Here's
403 some sample code to do exactly that:
404
405 my($buffer, $string) = ("", "");
406 while (read($fh, $buffer, 256, length($buffer))) {
407 $string .= decode($encoding, $buffer, Encode::FB_QUIET);
408 # $buffer now contains the unprocessed partial character
409 }
410
411 FB_WARN
412
413 I<CHECK> = Encode::FB_WARN
414
415 This is the same as "FB_QUIET" above, except that instead of being
416 silent on errors, it issues a warning. This is handy for when you are
417 debugging.
418
419 CAVEAT: All warnings from Encode module are reported, independently of
420 pragma warnings settings. If you want to follow settings of lexical
421 warnings configured by pragma warnings then append also check value
422 "ENCODE::ONLY_PRAGMA_WARNINGS". This value is available since Encode
423 version 2.99.
424
425 FB_PERLQQ FB_HTMLCREF FB_XMLCREF
426
427 perlqq mode (CHECK = Encode::FB_PERLQQ)
428 HTML charref mode (CHECK = Encode::FB_HTMLCREF)
429 XML charref mode (CHECK = Encode::FB_XMLCREF)
430
431 For encodings that are implemented by the "Encode::XS" module, "CHECK"
432 "==" "Encode::FB_PERLQQ" puts "encode" and "decode" into "perlqq"
433 fallback mode.
434
435 When you decode, "\xHH" is inserted for a malformed character, where HH
436 is the hex representation of the octet that could not be decoded to
437 utf8. When you encode, "\x{HHHH}" will be inserted, where HHHH is the
438 Unicode code point (in any number of hex digits) of the character that
439 cannot be found in the character repertoire of the encoding.
440
441 The HTML/XML character reference modes are about the same. In place of
442 "\x{HHHH}", HTML uses "&#NNN;" where NNN is a decimal number, and XML
443 uses "&#xHHHH;" where HHHH is the hexadecimal number.
444
445 In "Encode" 2.10 or later, "LEAVE_SRC" is also implied.
446
447 The bitmask
448
449 These modes are all actually set via a bitmask. Here is how the
450 "FB_XXX" constants are laid out. You can import the "FB_XXX" constants
451 via "use Encode qw(:fallbacks)", and you can import the generic bitmask
452 constants via "use Encode qw(:fallback_all)".
453
454 FB_DEFAULT FB_CROAK FB_QUIET FB_WARN FB_PERLQQ
455 DIE_ON_ERR 0x0001 X
456 WARN_ON_ERR 0x0002 X
457 RETURN_ON_ERR 0x0004 X X
458 LEAVE_SRC 0x0008 X
459 PERLQQ 0x0100 X
460 HTMLCREF 0x0200
461 XMLCREF 0x0400
462
463 LEAVE_SRC
464
465 Encode::LEAVE_SRC
466
467 If the "Encode::LEAVE_SRC" bit is not set but CHECK is set, then the
468 source string to encode() or decode() will be overwritten in place. If
469 you're not interested in this, then bitwise-OR it with the bitmask.
470
471 coderef for CHECK
472 As of "Encode" 2.12, "CHECK" can also be a code reference which takes
473 the ordinal value of the unmapped character as an argument and returns
474 octets that represent the fallback character. For instance:
475
476 $ascii = encode("ascii", $utf8, sub{ sprintf "<U+%04X>", shift });
477
478 Acts like "FB_PERLQQ" but U+XXXX is used instead of "\x{XXXX}".
479
480 Fallback for "decode" must return decoded string (sequence of
481 characters) and takes a list of ordinal values as its arguments. So for
482 example if you wish to decode octets as UTF-8, and use ISO-8859-15 as a
483 fallback for bytes that are not valid UTF-8, you could write
484
485 $str = decode 'UTF-8', $octets, sub {
486 my $tmp = join '', map chr, @_;
487 return decode 'ISO-8859-15', $tmp;
488 };
489
491 To define a new encoding, use:
492
493 use Encode qw(define_encoding);
494 define_encoding($object, CANONICAL_NAME [, alias...]);
495
496 CANONICAL_NAME will be associated with $object. The object should
497 provide the interface described in Encode::Encoding. If more than two
498 arguments are provided, additional arguments are considered aliases for
499 $object.
500
501 See Encode::Encoding for details.
502
504 Before the introduction of Unicode support in Perl, The "eq" operator
505 just compared the strings represented by two scalars. Beginning with
506 Perl 5.8, "eq" compares two strings with simultaneous consideration of
507 the UTF8 flag. To explain why we made it so, I quote from page 402 of
508 Programming Perl, 3rd ed.
509
510 Goal #1:
511 Old byte-oriented programs should not spontaneously break on the old
512 byte-oriented data they used to work on.
513
514 Goal #2:
515 Old byte-oriented programs should magically start working on the new
516 character-oriented data when appropriate.
517
518 Goal #3:
519 Programs should run just as fast in the new character-oriented mode
520 as in the old byte-oriented mode.
521
522 Goal #4:
523 Perl should remain one language, rather than forking into a byte-
524 oriented Perl and a character-oriented Perl.
525
526 When Programming Perl, 3rd ed. was written, not even Perl 5.6.0 had
527 been born yet, many features documented in the book remained
528 unimplemented for a long time. Perl 5.8 corrected much of this, and
529 the introduction of the UTF8 flag is one of them. You can think of
530 there being two fundamentally different kinds of strings and string-
531 operations in Perl: one a byte-oriented mode for when the internal
532 UTF8 flag is off, and the other a character-oriented mode for when the
533 internal UTF8 flag is on.
534
535 This UTF8 flag is not visible in Perl scripts, exactly for the same
536 reason you cannot (or rather, you don't have to) see whether a scalar
537 contains a string, an integer, or a floating-point number. But you
538 can still peek and poke these if you will. See the next section.
539
540 Messing with Perl's Internals
541 The following API uses parts of Perl's internals in the current
542 implementation. As such, they are efficient but may change in a future
543 release.
544
545 is_utf8
546
547 is_utf8(STRING [, CHECK])
548
549 [INTERNAL] Tests whether the UTF8 flag is turned on in the STRING. If
550 CHECK is true, also checks whether STRING contains well-formed UTF-8.
551 Returns true if successful, false otherwise.
552
553 Typically only necessary for debugging and testing. Don't use this
554 flag as a marker to distinguish character and binary data, that should
555 be decided for each variable when you write your code.
556
557 CAVEAT: If STRING has UTF8 flag set, it does NOT mean that STRING is
558 UTF-8 encoded and vice-versa.
559
560 As of Perl 5.8.1, utf8 also has the "utf8::is_utf8" function.
561
562 _utf8_on
563
564 _utf8_on(STRING)
565
566 [INTERNAL] Turns the STRING's internal UTF8 flag on. The STRING is not
567 checked for containing only well-formed UTF-8. Do not use this unless
568 you know with absolute certainty that the STRING holds only well-formed
569 UTF-8. Returns the previous state of the UTF8 flag (so please don't
570 treat the return value as indicating success or failure), or "undef" if
571 STRING is not a string.
572
573 NOTE: For security reasons, this function does not work on tainted
574 values.
575
576 _utf8_off
577
578 _utf8_off(STRING)
579
580 [INTERNAL] Turns the STRING's internal UTF8 flag off. Do not use
581 frivolously. Returns the previous state of the UTF8 flag, or "undef"
582 if STRING is not a string. Do not treat the return value as indicative
583 of success or failure, because that isn't what it means: it is only the
584 previous setting.
585
586 NOTE: For security reasons, this function does not work on tainted
587 values.
588
590 ....We now view strings not as sequences of bytes, but as sequences
591 of numbers in the range 0 .. 2**32-1 (or in the case of 64-bit
592 computers, 0 .. 2**64-1) -- Programming Perl, 3rd ed.
593
594 That has historically been Perl's notion of UTF-8, as that is how UTF-8
595 was first conceived by Ken Thompson when he invented it. However,
596 thanks to later revisions to the applicable standards, official UTF-8
597 is now rather stricter than that. For example, its range is much
598 narrower (0 .. 0x10_FFFF to cover only 21 bits instead of 32 or 64
599 bits) and some sequences are not allowed, like those used in surrogate
600 pairs, the 31 non-character code points 0xFDD0 .. 0xFDEF, the last two
601 code points in any plane (0xXX_FFFE and 0xXX_FFFF), all non-shortest
602 encodings, etc.
603
604 The former default in which Perl would always use a loose
605 interpretation of UTF-8 has now been overruled:
606
607 From: Larry Wall <larry@wall.org>
608 Date: December 04, 2004 11:51:58 JST
609 To: perl-unicode@perl.org
610 Subject: Re: Make Encode.pm support the real UTF-8
611 Message-Id: <20041204025158.GA28754@wall.org>
612
613 On Fri, Dec 03, 2004 at 10:12:12PM +0000, Tim Bunce wrote:
614 : I've no problem with 'utf8' being perl's unrestricted uft8 encoding,
615 : but "UTF-8" is the name of the standard and should give the
616 : corresponding behaviour.
617
618 For what it's worth, that's how I've always kept them straight in my
619 head.
620
621 Also for what it's worth, Perl 6 will mostly default to strict but
622 make it easy to switch back to lax.
623
624 Larry
625
626 Got that? As of Perl 5.8.7, "UTF-8" means UTF-8 in its current sense,
627 which is conservative and strict and security-conscious, whereas "utf8"
628 means UTF-8 in its former sense, which was liberal and loose and lax.
629 "Encode" version 2.10 or later thus groks this subtle but critically
630 important distinction between "UTF-8" and "utf8".
631
632 encode("utf8", "\x{FFFF_FFFF}", 1); # okay
633 encode("UTF-8", "\x{FFFF_FFFF}", 1); # croaks
634
635 This distinction is also important for decoding. In the following, $s
636 stores character U+200000, which exceeds UTF-8's allowed range. $s
637 thus stores an invalid Unicode code point:
638
639 $s = decode("utf8", "\xf8\x88\x80\x80\x80");
640
641 "UTF-8", by contrast, will either coerce the input to something valid:
642
643 $s = decode("UTF-8", "\xf8\x88\x80\x80\x80"); # U+FFFD
644
645 .. or croak:
646
647 decode("UTF-8", "\xf8\x88\x80\x80\x80", FB_CROAK|LEAVE_SRC);
648
649 In the "Encode" module, "UTF-8" is actually a canonical name for
650 "utf-8-strict". That hyphen between the "UTF" and the "8" is critical;
651 without it, "Encode" goes "liberal" and (perhaps overly-)permissive:
652
653 find_encoding("UTF-8")->name # is 'utf-8-strict'
654 find_encoding("utf-8")->name # ditto. names are case insensitive
655 find_encoding("utf_8")->name # ditto. "_" are treated as "-"
656 find_encoding("UTF8")->name # is 'utf8'.
657
658 Perl's internal UTF8 flag is called "UTF8", without a hyphen. It
659 indicates whether a string is internally encoded as "utf8", also
660 without a hyphen.
661
663 Encode::Encoding, Encode::Supported, Encode::PerlIO, encoding,
664 perlebcdic, "open" in perlfunc, perlunicode, perluniintro, perlunifaq,
665 perlunitut utf8, the Perl Unicode Mailing List
666 <http://lists.perl.org/list/perl-unicode.html>
667
669 This project was originated by the late Nick Ing-Simmons and later
670 maintained by Dan Kogai <dankogai@cpan.org>. See AUTHORS for a full
671 list of people involved. For any questions, send mail to
672 <perl-unicode@perl.org> so that we can all share.
673
674 While Dan Kogai retains the copyright as a maintainer, credit should go
675 to all those involved. See AUTHORS for a list of those who submitted
676 code to the project.
677
679 Copyright 2002-2014 Dan Kogai <dankogai@cpan.org>.
680
681 This library is free software; you can redistribute it and/or modify it
682 under the same terms as Perl itself.
683
684
685
686perl v5.34.0 2021-10-10 Encode(3)