1Scalar::String(3) User Contributed Perl Documentation Scalar::String(3)
2
6 Scalar::String - string aspects of scalars
7
9 use Scalar::String
10 qw(sclstr_is_upgraded sclstr_is_downgraded);
11 if(sclstr_is_upgraded($value)) { ...
13 if(sclstr_is_downgraded($value)) { ...
14 use Scalar::String qw(
16 sclstr_upgrade_inplace sclstr_upgraded
17 sclstr_downgrade_inplace sclstr_downgraded
18 );
19 sclstr_upgrade_inplace($value);
21 $value = sclstr_upgraded($value);
22 sclstr_downgrade_inplace($value);
23 $value = sclstr_downgraded($value);
24
26 This module is about the string part of plain Perl scalars. A scalar
27 has a string value, which is notionally a sequence of Unicode
28 codepoints, but may be internally encoded in either ISO-8859-1 or
29 UTF-8. In places, and more so in older versions of Perl, the internal
30 encoding shows through. To fully understand Perl strings it is
31 necessary to understand these implementation details.
32 This module provides functions to classify a string by encoding and to
34 encode a string in a desired way.
35 This module is implemented in XS, with a pure Perl backup version for
37 systems that can't handle XS.
38
40 ISO-8859-1 is a simple 8-bit character encoding, which represents the
41 first 256 Unicode characters (codepoints 0x00 to 0xff) in one octet
42 each. This is how strings were historically represented in Perl. A
43 string represented this way is referred to as "downgraded".
44 UTF-8 is a variable-width character encoding, which represents all
46 possible Unicode codepoints in differing numbers of octets. A design
47 feature of UTF-8 is that ASCII characters (codepoints 0x00 to 0x7f) are
48 each represented in a single octet, identically to their ISO-8859-1
49 encoding. Perl has its own variant of UTF-8, which can handle a wider
50 range of codepoints than Unicode formally allows. A string represented
51 in this variant UTF-8 is referred to as "upgraded".
52 A Perl string is physically represented as a string of octets along
54 with a flag that says whether the string is downgraded or upgraded. At
55 this level, to determine the Unicode codepoints that are represented
56 requires examining both parts of the representation. If the string
57 contains only ASCII characters then the octet sequence is identical in
58 either encoding, but Perl still maintains an encoding flag on such a
59 string. A string is always either downgraded or upgraded; it is never
60 both or neither.
61 When handling string input, it is good form to operate only on the
63 Unicode characters represented by the string, ignoring the manner in
64 which they are encoded. Basic string operations such as concatenation
65 work this way (except for a bug in perl 5.6.0), so simple code written
66 in pure Perl is generally safe on this front. Pieces of character-
67 based code can pass around strings among themselves, and always get
68 consistent behaviour, without worrying about the way in which the
69 characters are encoded.
70 However, due to an historical accident, a lot of C code that interfaces
72 with Perl looks at the octets used to represent a string without also
73 examining the encoding flag. Such code gives inconsistent behaviour
74 for the same character sequence represented in the different ways. In
75 perl 5.6, many pure Perl operations (such as regular expression
76 matching) also work this way, though some of them can be induced to
77 work correctly by using the utf8 pragma. In perl 5.8, regular
78 expression matching is character-based by default, but many I/O
79 functions (such as "open") are still octet-based.
80 Where code that operates on the octets of a string must be used by code
82 that operates on characters, the latter needs to pay attention to the
83 encoding of its strings. Commonly, the octet-based code expects its
84 input to be represented in a particular encoding, in which case the
85 character-based code must oblige by forcing strings to that encoding
86 before they are passed in. There are other usage patterns too.
87 You will be least confused if you think about a Perl string as a
89 character sequence plus an encoding flag. You should normally operate
90 on the character sequence and not care about the encoding flag.
91 Occasionally you must pay attention to the flag in addition to the
92 characters. Unless you are writing C code, you should try not to think
93 about a string the other way round, as an octet sequence plus encoding
94 flag.
95
97 Each "sclstr_" function takes one or more scalar string arguments to
98 operate on. These arguments must be strings; giving non-string
99 arguments will cause mayhem. See "is_string" in Params::Classify for a
100 way to check for stringness. Only the string value of the scalar is
101 used; the numeric value is completely ignored, so dualvars are not a
102 problem.
103 Classification
105 sclstr_is_upgraded(VALUE)
106 Returns a truth value indicating whether the provided string VALUE
107 is in upgraded form.
108 sclstr_is_downgraded(VALUE)
110 Returns a truth value indicating whether the provided string VALUE
111 is in downgraded form.
112 Regrading
114 sclstr_upgrade_inplace(VALUE)
115 Modifies the string VALUE in-place, so that it is in upgraded form,
116 regardless of how it was encoded before. The character sequence
117 that it represents is unchanged.
118 A cleaner interface to this operation is the non-mutating
120 "sclstr_upgraded".
121 sclstr_upgraded(VALUE)
123 Returns a string that represents the same character sequence as the
124 string VALUE, and is in upgraded form (regardless of how VALUE is
125 encoded).
126 sclstr_downgrade_inplace(VALUE[, FAIL_OK])
128 Modifies the string VALUE in-place, so that it is in downgraded
129 form, regardless of how it was encoded before. The character
130 sequence that it represents is unchanged. If the string cannot be
131 downgraded, because it contains a non-ISO-8859-1 character, then by
132 default the function "die"s, but if FAIL_OK is present and true
133 then it will return leaving VALUE unmodified.
134 A cleaner interface to this operation is the non-mutating
136 "sclstr_downgraded".
137 sclstr_downgraded(VALUE[, FAIL_OK])
139 Returns a string that represents the same character sequence as the
140 string VALUE, and is in downgraded form (regardless of how VALUE is
141 encoded). If the string cannot be represented in downgraded form,
142 because it contains a non-ISO-8859-1 character, then by default the
143 function "die"s, but if FAIL_OK is present and true then it will
144 return VALUE in its original upgraded form.
145
147 utf8
148
150 Andrew Main (Zefram) <zefram@fysh.org>
151
153 Copyright (C) 2009, 2010, 2011, 2017 Andrew Main (Zefram)
154 <zefram@fysh.org>
155
157 This module is free software; you can redistribute it and/or modify it
158 under the same terms as Perl itself.
159perl v5.34.0 2021-07-22 Scalar::String(3)