1Locale::Maketext(3) User Contributed Perl Documentation Locale::Maketext(3)
2
6 Locale::Maketext - framework for localization
7
9 package MyProgram;
10 use strict;
11 use MyProgram::L10N;
12 # ...which inherits from Locale::Maketext
13 my $lh = MyProgram::L10N->get_handle() || die "What language?";
14 ...
15 # And then any messages your program emits, like:
16 warn $lh->maketext( "Can't open file [_1]: [_2]\n", $f, $! );
17 ...
18
20 It is a common feature of applications (whether run directly, or via
21 the Web) for them to be "localized" -- i.e., for them to a present an
22 English interface to an English-speaker, a German interface to a
23 German-speaker, and so on for all languages it's programmed with.
24 Locale::Maketext is a framework for software localization; it provides
25 you with the tools for organizing and accessing the bits of text and
26 text-processing code that you need for producing localized
27 applications.
28 In order to make sense of Maketext and how all its components fit
30 together, you should probably go read Locale::Maketext::TPJ13, and then
31 read the following documentation.
32 You may also want to read over the source for "File::Findgrep" and its
34 constituent modules -- they are a complete (if small) example
35 application that uses Maketext.
36
38 The basic design of Locale::Maketext is object-oriented, and
39 Locale::Maketext is an abstract base class, from which you derive a
40 "project class". The project class (with a name like
41 "TkBocciBall::Localize", which you then use in your module) is in turn
42 the base class for all the "language classes" for your project (with
43 names "TkBocciBall::Localize::it", "TkBocciBall::Localize::en",
44 "TkBocciBall::Localize::fr", etc.).
45 A language class is a class containing a lexicon of phrases as class
47 data, and possibly also some methods that are of use in interpreting
48 phrases in the lexicon, or otherwise dealing with text in that
49 language.
50 An object belonging to a language class is called a "language handle";
52 it's typically a flyweight object.
53 The normal course of action is to call:
55 use TkBocciBall::Localize; # the localization project class
57 $lh = TkBocciBall::Localize->get_handle();
58 # Depending on the user's locale, etc., this will
59 # make a language handle from among the classes available,
60 # and any defaults that you declare.
61 die "Couldn't make a language handle??" unless $lh;
62 From then on, you use the "maketext" function to access entries in
64 whatever lexicon(s) belong to the language handle you got. So, this:
65 print $lh->maketext("You won!"), "\n";
67 ...emits the right text for this language. If the object in $lh
69 belongs to class "TkBocciBall::Localize::fr" and
70 %TkBocciBall::Localize::fr::Lexicon contains "("You won!" => "Tu as
71 gagné!")", then the above code happily tells the user "Tu as gagné!".
72
74 Locale::Maketext offers a variety of methods, which fall into three
75 categories:
76 • Methods to do with constructing language handles.
78 • "maketext" and other methods to do with accessing %Lexicon data for
80 a given language handle.
81 • Methods that you may find it handy to use, from routines of yours
83 that you put in %Lexicon entries.
84 These are covered in the following section.
86 Construction Methods
88 These are to do with constructing a language handle:
89 • $lh = YourProjClass->get_handle( ...langtags... ) || die "lg-
91 handle?";
92 This tries loading classes based on the language-tags you give
94 (like "("en-US", "sk", "kon", "es-MX", "ja", "i-klingon")", and for
95 the first class that succeeds, returns
96 YourProjClass::language->new().
97 If it runs thru the entire given list of language-tags, and finds
99 no classes for those exact terms, it then tries "superordinate"
100 language classes. So if no "en-US" class (i.e.,
101 YourProjClass::en_us) was found, nor classes for anything else in
102 that list, we then try its superordinate, "en" (i.e.,
103 YourProjClass::en), and so on thru the other language-tags in the
104 given list: "es". (The other language-tags in our example list:
105 happen to have no superordinates.)
106 If none of those language-tags leads to loadable classes, we then
108 try classes derived from YourProjClass->fallback_languages() and
109 then if nothing comes of that, we use classes named by
110 YourProjClass->fallback_language_classes(). Then in the (probably
111 quite unlikely) event that that fails, we just return undef.
112 • $lh = YourProjClass->get_handle() || die "lg-handle?";
114 When "get_handle" is called with an empty parameter list, magic
116 happens:
117 If "get_handle" senses that it's running in program that was
119 invoked as a CGI, then it tries to get language-tags out of the
120 environment variable "HTTP_ACCEPT_LANGUAGE", and it pretends that
121 those were the languages passed as parameters to "get_handle".
122 Otherwise (i.e., if not a CGI), this tries various OS-specific ways
124 to get the language-tags for the current locale/language, and then
125 pretends that those were the value(s) passed to "get_handle".
126 Currently this OS-specific stuff consists of looking in the
128 environment variables "LANG" and "LANGUAGE"; and on MSWin machines
129 (where those variables are typically unused), this also tries using
130 the module Win32::Locale to get a language-tag for whatever
131 language/locale is currently selected in the "Regional Settings"
132 (or "International"?) Control Panel. I welcome further
133 suggestions for making this do the Right Thing under other
134 operating systems that support localization.
135 If you're using localization in an application that keeps a
137 configuration file, you might consider something like this in your
138 project class:
139 sub get_handle_via_config {
141 my $class = $_[0];
142 my $chosen_language = $Config_settings{'language'};
143 my $lh;
144 if($chosen_language) {
145 $lh = $class->get_handle($chosen_language)
146 || die "No language handle for \"$chosen_language\""
147 . " or the like";
148 } else {
149 # Config file missing, maybe?
150 $lh = $class->get_handle()
151 || die "Can't get a language handle";
152 }
153 return $lh;
154 }
155 • $lh = YourProjClass::langname->new();
157 This constructs a language handle. You usually don't call this
159 directly, but instead let "get_handle" find a language class to
160 "use" and to then call ->new on.
161 • $lh->init();
163 This is called by ->new to initialize newly-constructed language
165 handles. If you define an init method in your class, remember that
166 it's usually considered a good idea to call $lh->SUPER::init in it
167 (presumably at the beginning), so that all classes get a chance to
168 initialize a new object however they see fit.
169 • YourProjClass->fallback_languages()
171 "get_handle" appends the return value of this to the end of
173 whatever list of languages you pass "get_handle". Unless you
174 override this method, your project class will inherit
175 Locale::Maketext's "fallback_languages", which currently returns
176 "('i-default', 'en', 'en-US')". ("i-default" is defined in RFC
177 2277).
178 This method (by having it return the name of a language-tag that
180 has an existing language class) can be used for making sure that
181 "get_handle" will always manage to construct a language handle
182 (assuming your language classes are in an appropriate @INC
183 directory). Or you can use the next method:
184 • YourProjClass->fallback_language_classes()
186 "get_handle" appends the return value of this to the end of the
188 list of classes it will try using. Unless you override this
189 method, your project class will inherit Locale::Maketext's
190 "fallback_language_classes", which currently returns an empty list,
191 "()". By setting this to some value (namely, the name of a
192 loadable language class), you can be sure that "get_handle" will
193 always manage to construct a language handle.
194 The "maketext" Method
196 This is the most important method in Locale::Maketext:
197 $text = $lh->maketext(I<key>, ...parameters for this phrase...);
199 This looks in the %Lexicon of the language handle $lh and all its
201 superclasses, looking for an entry whose key is the string key.
202 Assuming such an entry is found, various things then happen, depending
203 on the value found:
204 If the value is a scalarref, the scalar is dereferenced and returned
206 (and any parameters are ignored).
207 If the value is a coderef, we return &$value($lh, ...parameters...).
209 If the value is a string that doesn't look like it's in Bracket
211 Notation, we return it (after replacing it with a scalarref, in its
212 %Lexicon).
213 If the value does look like it's in Bracket Notation, then we compile
215 it into a sub, replace the string in the %Lexicon with the new coderef,
216 and then we return &$new_sub($lh, ...parameters...).
217 Bracket Notation is discussed in a later section. Note that trying to
219 compile a string into Bracket Notation can throw an exception if the
220 string is not syntactically valid (say, by not balancing brackets
221 right.)
222 Also, calling &$coderef($lh, ...parameters...) can throw any sort of
224 exception (if, say, code in that sub tries to divide by zero). But a
225 very common exception occurs when you have Bracket Notation text that
226 says to call a method "foo", but there is no such method. (E.g., "You
227 have [quatn,_1,ball]." will throw an exception on trying to call
228 $lh->quatn($_[1],'ball') -- you presumably meant "quant".) "maketext"
229 catches these exceptions, but only to make the error message more
230 readable, at which point it rethrows the exception.
231 An exception may be thrown if key is not found in any of $lh's %Lexicon
233 hashes. What happens if a key is not found, is discussed in a later
234 section, "Controlling Lookup Failure".
235 Note that you might find it useful in some cases to override the
237 "maketext" method with an "after method", if you want to translate
238 encodings, or even scripts:
239 package YrProj::zh_cn; # Chinese with PRC-style glyphs
241 use base ('YrProj::zh_tw'); # Taiwan-style
242 sub maketext {
243 my $self = shift(@_);
244 my $value = $self->maketext(@_);
245 return Chineeze::taiwan2mainland($value);
246 }
247 Or you may want to override it with something that traps any
249 exceptions, if that's critical to your program:
250 sub maketext {
252 my($lh, @stuff) = @_;
253 my $out;
254 eval { $out = $lh->SUPER::maketext(@stuff) };
255 return $out unless $@;
256 ...otherwise deal with the exception...
257 }
258 Other than those two situations, I don't imagine that it's useful to
260 override the "maketext" method. (If you run into a situation where it
261 is useful, I'd be interested in hearing about it.)
262 $lh->fail_with or $lh->fail_with(PARAM)
264 $lh->failure_handler_auto
265 These two methods are discussed in the section "Controlling Lookup
266 Failure".
267 $lh->denylist(@list) <or> $lh->blacklist(@list)
269 $lh->allowlist(@list) <or> $lh->whitelist(@list)
270 These methods are discussed in the section "Bracket Notation
271 Security".
272 Utility Methods
274 These are methods that you may find it handy to use, generally from
275 %Lexicon routines of yours (whether expressed as Bracket Notation or
276 not).
277 $language->quant($number, $singular)
279 $language->quant($number, $singular, $plural)
280 $language->quant($number, $singular, $plural, $negative)
281 This is generally meant to be called from inside Bracket Notation
282 (which is discussed later), as in
283 "Your search matched [quant,_1,document]!"
285 It's for quantifying a noun (i.e., saying how much of it there is,
287 while giving the correct form of it). The behavior of this method
288 is handy for English and a few other Western European languages,
289 and you should override it for languages where it's not suitable.
290 You can feel free to read the source, but the current
291 implementation is basically as this pseudocode describes:
292 if $number is 0 and there's a $negative,
294 return $negative;
295 elsif $number is 1,
296 return "1 $singular";
297 elsif there's a $plural,
298 return "$number $plural";
299 else
300 return "$number " . $singular . "s";
301 #
302 # ...except that we actually call numf to
303 # stringify $number before returning it.
304 So for English (with Bracket Notation) "...[quant,_1,file]..." is
306 fine (for 0 it returns "0 files", for 1 it returns "1 file", and
307 for more it returns "2 files", etc.)
308 But for "directory", you'd want "[quant,_1,directory,directories]"
310 so that our elementary "quant" method doesn't think that the plural
311 of "directory" is "directorys". And you might find that the output
312 may sound better if you specify a negative form, as in:
313 "[quant,_1,file,files,No files] matched your query.\n"
315 Remember to keep in mind verb agreement (or adjectives too, in
317 other languages), as in:
318 "[quant,_1,document] were matched.\n"
320 Because if _1 is one, you get "1 document were matched". An
322 acceptable hack here is to do something like this:
323 "[quant,_1,document was, documents were] matched.\n"
325 $language->numf($number)
327 This returns the given number formatted nicely according to this
328 language's conventions. Maketext's default method is mostly to
329 just take the normal string form of the number (applying sprintf
330 "%G" for only very large numbers), and then to add commas as
331 necessary. (Except that we apply "tr/,./.,/" if
332 $language->{'numf_comma'} is true; that's a bit of a hack that's
333 useful for languages that express two million as "2.000.000" and
334 not as "2,000,000").
335 If you want anything fancier, consider overriding this with
337 something that uses Number::Format, or does something else
338 entirely.
339 Note that numf is called by quant for stringifying all quantifying
341 numbers.
342 $language->numerate($number, $singular, $plural, $negative)
344 This returns the given noun form which is appropriate for the
345 quantity $number according to this language's conventions.
346 "numerate" is used internally by "quant" to quantify nouns. Use it
347 directly -- usually from bracket notation -- to avoid "quant"'s
348 implicit call to "numf" and output of a numeric quantity.
349 $language->sprintf($format, @items)
351 This is just a wrapper around Perl's normal "sprintf" function.
352 It's provided so that you can use "sprintf" in Bracket Notation:
353 "Couldn't access datanode [sprintf,%10x=~[%s~],_1,_2]!\n"
355 returning...
357 Couldn't access datanode Stuff=[thangamabob]!
359 $language->language_tag()
361 Currently this just takes the last bit of "ref($language)", turns
362 underscores to dashes, and returns it. So if $language is an
363 object of class Hee::HOO::Haw::en_us, $language->language_tag()
364 returns "en-us". (Yes, the usual representation for that language
365 tag is "en-US", but case is never considered meaningful in
366 language-tag comparison.)
367 You may override this as you like; Maketext doesn't use it for
369 anything.
370 $language->encoding()
372 Currently this isn't used for anything, but it's provided (with
373 default value of "(ref($language) && $language->{'encoding'})) or
374 "iso-8859-1"" ) as a sort of suggestion that it may be
375 useful/necessary to associate encodings with your language handles
376 (whether on a per-class or even per-handle basis.)
377 Language Handle Attributes and Internals
379 A language handle is a flyweight object -- i.e., it doesn't
380 (necessarily) carry any data of interest, other than just being a
381 member of whatever class it belongs to.
382 A language handle is implemented as a blessed hash. Subclasses of
384 yours can store whatever data you want in the hash. Currently the only
385 hash entry used by any crucial Maketext method is "fail", so feel free
386 to use anything else as you like.
387 Remember: Don't be afraid to read the Maketext source if there's any
389 point on which this documentation is unclear. This documentation is
390 vastly longer than the module source itself.
391
393 These are Locale::Maketext's assumptions about the class hierarchy
394 formed by all your language classes:
395 • You must have a project base class, which you load, and which you
397 then use as the first argument in the call to
398 YourProjClass->get_handle(...). It should derive (whether directly
399 or indirectly) from Locale::Maketext. It doesn't matter how you
400 name this class, although assuming this is the localization
401 component of your Super Mega Program, good names for your project
402 class might be SuperMegaProgram::Localization,
403 SuperMegaProgram::L10N, SuperMegaProgram::I18N,
404 SuperMegaProgram::International, or even
405 SuperMegaProgram::Languages or SuperMegaProgram::Messages.
406 • Language classes are what YourProjClass->get_handle will try to
408 load. It will look for them by taking each language-tag (skipping
409 it if it doesn't look like a language-tag or locale-tag!), turning
410 it to all lowercase, turning dashes to underscores, and appending
411 it to YourProjClass . "::". So this:
412 $lh = YourProjClass->get_handle(
414 'en-US', 'fr', 'kon', 'i-klingon', 'i-klingon-romanized'
415 );
416 will try loading the classes YourProjClass::en_us (note
418 lowercase!), YourProjClass::fr, YourProjClass::kon,
419 YourProjClass::i_klingon and YourProjClass::i_klingon_romanized.
420 (And it'll stop at the first one that actually loads.)
421 • I assume that each language class derives (directly or indirectly)
423 from your project class, and also defines its @ISA, its %Lexicon,
424 or both. But I anticipate no dire consequences if these
425 assumptions do not hold.
426 • Language classes may derive from other language classes (although
428 they should have "use Thatclassname" or "use base
429 qw(...classes...)"). They may derive from the project class. They
430 may derive from some other class altogether. Or via multiple
431 inheritance, it may derive from any mixture of these.
432 • I foresee no problems with having multiple inheritance in your
434 hierarchy of language classes. (As usual, however, Perl will
435 complain bitterly if you have a cycle in the hierarchy: i.e., if
436 any class is its own ancestor.)
437
439 A typical %Lexicon entry is meant to signify a phrase, taking some
440 number (0 or more) of parameters. An entry is meant to be accessed by
441 via a string key in $lh->maketext(key, ...parameters...), which should
442 return a string that is generally meant for be used for "output" to the
443 user -- regardless of whether this actually means printing to STDOUT,
444 writing to a file, or putting into a GUI widget.
445 While the key must be a string value (since that's a basic restriction
447 that Perl places on hash keys), the value in the lexicon can currently
448 be of several types: a defined scalar, scalarref, or coderef. The use
449 of these is explained above, in the section 'The "maketext" Method',
450 and Bracket Notation for strings is discussed in the next section.
451 While you can use arbitrary unique IDs for lexicon keys (like
453 "_min_larger_max_error"), it is often useful for if an entry's key is
454 itself a valid value, like this example error message:
455 "Minimum ([_1]) is larger than maximum ([_2])!\n",
457 Compare this code that uses an arbitrary ID...
459 die $lh->maketext( "_min_larger_max_error", $min, $max )
461 if $min > $max;
462 ...to this code that uses a key-as-value:
464 die $lh->maketext(
466 "Minimum ([_1]) is larger than maximum ([_2])!\n",
467 $min, $max
468 ) if $min > $max;
469 The second is, in short, more readable. In particular, it's obvious
471 that the number of parameters you're feeding to that phrase (two) is
472 the number of parameters that it wants to be fed. (Since you see _1
473 and a _2 being used in the key there.)
474 Also, once a project is otherwise complete and you start to localize
476 it, you can scrape together all the various keys you use, and pass it
477 to a translator; and then the translator's work will go faster if what
478 he's presented is this:
479 "Minimum ([_1]) is larger than maximum ([_2])!\n",
481 => "", # fill in something here, Jacques!
482 rather than this more cryptic mess:
484 "_min_larger_max_error"
486 => "", # fill in something here, Jacques
487 I think that keys as lexicon values makes the completed lexicon entries
489 more readable:
490 "Minimum ([_1]) is larger than maximum ([_2])!\n",
492 => "Le minimum ([_1]) est plus grand que le maximum ([_2])!\n",
493 Also, having valid values as keys becomes very useful if you set up an
495 _AUTO lexicon. _AUTO lexicons are discussed in a later section.
496 I almost always use keys that are themselves valid lexicon values. One
498 notable exception is when the value is quite long. For example, to get
499 the screenful of data that a command-line program might return when
500 given an unknown switch, I often just use a brief, self-explanatory key
501 such as "_USAGE_MESSAGE". At that point I then go and immediately to
502 define that lexicon entry in the ProjectClass::L10N::en lexicon (since
503 English is always my "project language"):
504 '_USAGE_MESSAGE' => <<'EOSTUFF',
506 ...long long message...
507 EOSTUFF
508 and then I can use it as:
510 getopt('oDI', \%opts) or die $lh->maketext('_USAGE_MESSAGE');
512 Incidentally, note that each class's %Lexicon inherits-and-extends the
514 lexicons in its superclasses. This is not because these are special
515 hashes per se, but because you access them via the "maketext" method,
516 which looks for entries across all the %Lexicon hashes in a language
517 class and all its ancestor classes. (This is because the idea of
518 "class data" isn't directly implemented in Perl, but is instead left to
519 individual class-systems to implement as they see fit..)
520 Note that you may have things stored in a lexicon besides just phrases
522 for output: for example, if your program takes input from the
523 keyboard, asking a "(Y/N)" question, you probably need to know what the
524 equivalent of "Y[es]/N[o]" is in whatever language. You probably also
525 need to know what the equivalents of the answers "y" and "n" are. You
526 can store that information in the lexicon (say, under the keys
527 "~answer_y" and "~answer_n", and the long forms as "~answer_yes" and
528 "~answer_no", where "~" is just an ad-hoc character meant to indicate
529 to programmers/translators that these are not phrases for output).
530 Or instead of storing this in the language class's lexicon, you can
532 (and, in some cases, really should) represent the same bit of knowledge
533 as code in a method in the language class. (That leaves a tidy
534 distinction between the lexicon as the things we know how to say, and
535 the rest of the things in the lexicon class as things that we know how
536 to do.) Consider this example of a processor for responses to French
537 "oui/non" questions:
538 sub y_or_n {
540 return undef unless defined $_[1] and length $_[1];
541 my $answer = lc $_[1]; # smash case
542 return 1 if $answer eq 'o' or $answer eq 'oui';
543 return 0 if $answer eq 'n' or $answer eq 'non';
544 return undef;
545 }
546 ...which you'd then call in a construct like this:
548 my $response;
550 until(defined $response) {
551 print $lh->maketext("Open the pod bay door (y/n)? ");
552 $response = $lh->y_or_n( get_input_from_keyboard_somehow() );
553 }
554 if($response) { $pod_bay_door->open() }
555 else { $pod_bay_door->leave_closed() }
556 Other data worth storing in a lexicon might be things like filenames
558 for language-targetted resources:
559 ...
561 "_main_splash_png"
562 => "/styles/en_us/main_splash.png",
563 "_main_splash_imagemap"
564 => "/styles/en_us/main_splash.incl",
565 "_general_graphics_path"
566 => "/styles/en_us/",
567 "_alert_sound"
568 => "/styles/en_us/hey_there.wav",
569 "_forward_icon"
570 => "left_arrow.png",
571 "_backward_icon"
572 => "right_arrow.png",
573 # In some other languages, left equals
574 # BACKwards, and right is FOREwards.
575 ...
576 You might want to do the same thing for expressing key bindings or the
578 like (since hardwiring "q" as the binding for the function that quits a
579 screen/menu/program is useful only if your language happens to
580 associate "q" with "quit"!)
581
583 Bracket Notation is a crucial feature of Locale::Maketext. I mean
584 Bracket Notation to provide a replacement for the use of sprintf
585 formatting. Everything you do with Bracket Notation could be done with
586 a sub block, but bracket notation is meant to be much more concise.
587 Bracket Notation is a like a miniature "template" system (in the sense
589 of Text::Template, not in the sense of C++ templates), where normal
590 text is passed thru basically as is, but text in special regions is
591 specially interpreted. In Bracket Notation, you use square brackets
592 ("[...]"), not curly braces ("{...}") to note sections that are
593 specially interpreted.
594 For example, here all the areas that are taken literally are underlined
596 with a "^", and all the in-bracket special regions are underlined with
597 an X:
598 "Minimum ([_1]) is larger than maximum ([_2])!\n",
600 ^^^^^^^^^ XX ^^^^^^^^^^^^^^^^^^^^^^^^^^ XX ^^^^
601 When that string is compiled from bracket notation into a real Perl
603 sub, it's basically turned into:
604 sub {
606 my $lh = $_[0];
607 my @params = @_;
608 return join '',
609 "Minimum (",
610 ...some code here...
611 ") is larger than maximum (",
612 ...some code here...
613 ")!\n",
614 }
615 # to be called by $lh->maketext(KEY, params...)
616 In other words, text outside bracket groups is turned into string
618 literals. Text in brackets is rather more complex, and currently
619 follows these rules:
620 • Bracket groups that are empty, or which consist only of whitespace,
622 are ignored. (Examples: "[]", "[ ]", or a [ and a ] with
623 returns and/or tabs and/or spaces between them.
624 Otherwise, each group is taken to be a comma-separated group of
626 items, and each item is interpreted as follows:
627 • An item that is "_digits" or "_-digits" is interpreted as
629 $_[value]. I.e., "_1" becomes with $_[1], and "_-3" is interpreted
630 as $_[-3] (in which case @_ should have at least three elements in
631 it). Note that $_[0] is the language handle, and is typically not
632 named directly.
633 • An item "_*" is interpreted to mean "all of @_ except $_[0]".
635 I.e., @_[1..$#_]. Note that this is an empty list in the case of
636 calls like $lh->maketext(key) where there are no parameters (except
637 $_[0], the language handle).
638 • Otherwise, each item is interpreted as a string literal.
640 The group as a whole is interpreted as follows:
642 • If the first item in a bracket group looks like a method name, then
644 that group is interpreted like this:
645 $lh->that_method_name(
647 ...rest of items in this group...
648 ),
649 • If the first item in a bracket group is "*", it's taken as
651 shorthand for the so commonly called "quant" method. Similarly, if
652 the first item in a bracket group is "#", it's taken to be
653 shorthand for "numf".
654 • If the first item in a bracket group is the empty-string, or "_*"
656 or "_digits" or "_-digits", then that group is interpreted as just
657 the interpolation of all its items:
658 join('',
660 ...rest of items in this group...
661 ),
662 Examples: "[_1]" and "[,_1]", which are synonymous; and
664 ""[,ID-(,_4,-,_2,)]"", which compiles as "join "", "ID-(", $_[4],
665 "-", $_[2], ")"".
666 • Otherwise this bracket group is invalid. For example, in the group
668 "[!@#,whatever]", the first item "!@#" is neither the empty-string,
669 "_number", "_-number", "_*", nor a valid method name; and so
670 Locale::Maketext will throw an exception of you try compiling an
671 expression containing this bracket group.
672 Note, incidentally, that items in each group are comma-separated, not
674 "/\s*,\s*/"-separated. That is, you might expect that this bracket
675 group:
676 "Hoohah [foo, _1 , bar ,baz]!"
678 would compile to this:
680 sub {
682 my $lh = $_[0];
683 return join '',
684 "Hoohah ",
685 $lh->foo( $_[1], "bar", "baz"),
686 "!",
687 }
688 But it actually compiles as this:
690 sub {
692 my $lh = $_[0];
693 return join '',
694 "Hoohah ",
695 $lh->foo(" _1 ", " bar ", "baz"), # note the <space> in " bar "
696 "!",
697 }
698 In the notation discussed so far, the characters "[" and "]" are given
700 special meaning, for opening and closing bracket groups, and "," has a
701 special meaning inside bracket groups, where it separates items in the
702 group. This begs the question of how you'd express a literal "[" or
703 "]" in a Bracket Notation string, and how you'd express a literal comma
704 inside a bracket group. For this purpose I've adopted "~" (tilde) as
705 an escape character: "~[" means a literal '[' character anywhere in
706 Bracket Notation (i.e., regardless of whether you're in a bracket group
707 or not), and ditto for "~]" meaning a literal ']', and "~," meaning a
708 literal comma. (Altho "," means a literal comma outside of bracket
709 groups -- it's only inside bracket groups that commas are special.)
710 And on the off chance you need a literal tilde in a bracket expression,
712 you get it with "~~".
713 Currently, an unescaped "~" before a character other than a bracket or
715 a comma is taken to mean just a "~" and that character. I.e., "~X"
716 means the same as "~~X" -- i.e., one literal tilde, and then one
717 literal "X". However, by using "~X", you are assuming that no future
718 version of Maketext will use "~X" as a magic escape sequence. In
719 practice this is not a great problem, since first off you can just
720 write "~~X" and not worry about it; second off, I doubt I'll add lots
721 of new magic characters to bracket notation; and third off, you aren't
722 likely to want literal "~" characters in your messages anyway, since
723 it's not a character with wide use in natural language text.
724 Brackets must be balanced -- every openbracket must have one matching
726 closebracket, and vice versa. So these are all invalid:
727 "I ate [quant,_1,rhubarb pie."
729 "I ate [quant,_1,rhubarb pie[."
730 "I ate quant,_1,rhubarb pie]."
731 "I ate quant,_1,rhubarb pie[."
732 Currently, bracket groups do not nest. That is, you cannot say:
734 "Foo [bar,baz,[quux,quuux]]\n";
736 If you need a notation that's that powerful, use normal Perl:
738 %Lexicon = (
740 ...
741 "some_key" => sub {
742 my $lh = $_[0];
743 join '',
744 "Foo ",
745 $lh->bar('baz', $lh->quux('quuux')),
746 "\n",
747 },
748 ...
749 );
750 Or write the "bar" method so you don't need to pass it the output from
752 calling quux.
753 I do not anticipate that you will need (or particularly want) to nest
755 bracket groups, but you are welcome to email me with convincing (real-
756 life) arguments to the contrary.
757
759 Locale::Maketext does not use any special syntax to differentiate
760 bracket notation methods from normal class or object methods. This
761 design makes it vulnerable to format string attacks whenever it is used
762 to process strings provided by untrusted users.
763 Locale::Maketext does support denylist and allowlist functionality to
765 limit which methods may be called as bracket notation methods.
766 By default, Locale::Maketext denies all methods in the Locale::Maketext
768 namespace that begin with the '_' character, and all methods which
769 include Perl's namespace separator characters.
770 The default denylist for Locale::Maketext also prevents use of the
772 following methods in bracket notation:
773 denylist
775 encoding
776 fail_with
777 failure_handler_auto
778 fallback_language_classes
779 fallback_languages
780 get_handle
781 init
782 language_tag
783 maketext
784 new
785 allowlist
786 whitelist
787 blacklist
788 This list can be extended by either deny-listing additional "known bad"
790 methods, or allow-listing only "known good" methods.
791 To prevent specific methods from being called in bracket notation, use
793 the denylist() method:
794 my $lh = MyProgram::L10N->get_handle();
796 $lh->denylist(qw{my_internal_method my_other_method});
797 $lh->maketext('[my_internal_method]'); # dies
798 To limit the allowed bracked notation methods to a specific list, use
800 the allowlist() method:
801 my $lh = MyProgram::L10N->get_handle();
803 $lh->allowlist('numerate', 'numf');
804 $lh->maketext('[_1] [numerate, _1,shoe,shoes]', 12); # works
805 $lh->maketext('[my_internal_method]'); # dies
806 The denylist() and allowlist() methods extend their internal lists
808 whenever they are called. To reset the denylist or allowlist, create a
809 new maketext object.
810 my $lh = MyProgram::L10N->get_handle();
812 $lh->denylist('numerate');
813 $lh->denylist('numf');
814 $lh->maketext('[_1] [numerate,_1,shoe,shoes]', 12); # dies
815 For lexicons that use an internal cache, translations which have
817 already been cached in their compiled form are not affected by
818 subsequent changes to the allowlist or denylist settings. Lexicons that
819 use an external cache will have their cache cleared whenever the
820 allowlist or denylist settings change. The difference between the two
821 types of caching is explained in the "Readonly Lexicons" section.
822 Methods disallowed by the denylist cannot be permitted by the
824 allowlist.
825 NOTE: denylist() is the preferred method name to use instead of the
827 historical and non-inclusive method blacklist(). blacklist() may be
828 removed in a future release of this package and so it's use should be
829 removed from usage.
830 NOTE: allowlist() is the preferred method name to use instead of the
832 historical and non-inclusive method whitelist(). whitelist() may be
833 removed in a future release of this package and so it's use should be
834 removed from usage.
835
837 If maketext goes to look in an individual %Lexicon for an entry for key
838 (where key does not start with an underscore), and sees none, but does
839 see an entry of "_AUTO" => some_true_value, then we actually define
840 $Lexicon{key} = key right then and there, and then use that value as if
841 it had been there all along. This happens before we even look in any
842 superclass %Lexicons!
843 (This is meant to be somewhat like the AUTOLOAD mechanism in Perl's
845 function call system -- or, looked at another way, like the AutoLoader
846 module.)
847 I can picture all sorts of circumstances where you just do not want
849 lookup to be able to fail (since failing normally means that maketext
850 throws a "die", although see the next section for greater control over
851 that). But here's one circumstance where _AUTO lexicons are meant to
852 be especially useful:
853 As you're writing an application, you decide as you go what messages
855 you need to emit. Normally you'd go to write this:
856 if(-e $filename) {
858 go_process_file($filename)
859 } else {
860 print qq{Couldn't find file "$filename"!\n};
861 }
862 but since you anticipate localizing this, you write:
864 use ThisProject::I18N;
866 my $lh = ThisProject::I18N->get_handle();
867 # For the moment, assume that things are set up so
868 # that we load class ThisProject::I18N::en
869 # and that that's the class that $lh belongs to.
870 ...
871 if(-e $filename) {
872 go_process_file($filename)
873 } else {
874 print $lh->maketext(
875 qq{Couldn't find file "[_1]"!\n}, $filename
876 );
877 }
878 Now, right after you've just written the above lines, you'd normally
880 have to go open the file ThisProject/I18N/en.pm, and immediately add an
881 entry:
882 "Couldn't find file \"[_1]\"!\n"
884 => "Couldn't find file \"[_1]\"!\n",
885 But I consider that somewhat of a distraction from the work of getting
887 the main code working -- to say nothing of the fact that I often have
888 to play with the program a few times before I can decide exactly what
889 wording I want in the messages (which in this case would require me to
890 go changing three lines of code: the call to maketext with that key,
891 and then the two lines in ThisProject/I18N/en.pm).
892 However, if you set "_AUTO => 1" in the %Lexicon in,
894 ThisProject/I18N/en.pm (assuming that English (en) is the language that
895 all your programmers will be using for this project's internal message
896 keys), then you don't ever have to go adding lines like this
897 "Couldn't find file \"[_1]\"!\n"
899 => "Couldn't find file \"[_1]\"!\n",
900 to ThisProject/I18N/en.pm, because if _AUTO is true there, then just
902 looking for an entry with the key "Couldn't find file \"[_1]\"!\n" in
903 that lexicon will cause it to be added, with that value!
904 Note that the reason that keys that start with "_" are immune to _AUTO
906 isn't anything generally magical about the underscore character -- I
907 just wanted a way to have most lexicon keys be autoable, except for
908 possibly a few, and I arbitrarily decided to use a leading underscore
909 as a signal to distinguish those few.
910
912 If your lexicon is a tied hash the simple act of caching the compiled
913 value can be fatal.
914 For example a GDBM_File GDBM_READER tied hash will die with something
916 like:
917 gdbm store returned -1, errno 2, key "..." at ...
919 All you need to do is turn on caching outside of the lexicon hash
921 itself like so:
922 sub init {
924 my ($lh) = @_;
925 ...
926 $lh->{'use_external_lex_cache'} = 1;
927 ...
928 }
929 And then instead of storing the compiled value in the lexicon hash it
931 will store it in $lh->{'_external_lex_cache'}
932
934 If you call $lh->maketext(key, ...parameters...), and there's no entry
935 key in $lh's class's %Lexicon, nor in the superclass %Lexicon hash, and
936 if we can't auto-make key (because either it starts with a "_", or
937 because none of its lexicons have "_AUTO => 1,"), then we have failed
938 to find a normal way to maketext key. What then happens in these
939 failure conditions, depends on the $lh object's "fail" attribute.
940 If the language handle has no "fail" attribute, maketext will simply
942 throw an exception (i.e., it calls "die", mentioning the key whose
943 lookup failed, and naming the line number where the calling
944 $lh->maketext(key,...) was.
945 If the language handle has a "fail" attribute whose value is a coderef,
947 then $lh->maketext(key,...params...) gives up and calls:
948 return $that_subref->($lh, $key, @params);
950 Otherwise, the "fail" attribute's value should be a string denoting a
952 method name, so that $lh->maketext(key,...params...) can give up with:
953 return $lh->$that_method_name($phrase, @params);
955 The "fail" attribute can be accessed with the "fail_with" method:
957 # Set to a coderef:
959 $lh->fail_with( \&failure_handler );
960 # Set to a method name:
962 $lh->fail_with( 'failure_method' );
963 # Set to nothing (i.e., so failure throws a plain exception)
965 $lh->fail_with( undef );
966 # Get the current value
968 $handler = $lh->fail_with();
969 Now, as to what you may want to do with these handlers: Maybe you'd
971 want to log what key failed for what class, and then die. Maybe you
972 don't like "die" and instead you want to send the error message to
973 STDOUT (or wherever) and then merely "exit()".
974 Or maybe you don't want to "die" at all! Maybe you could use a handler
976 like this:
977 # Make all lookups fall back onto an English value,
979 # but only after we log it for later fingerpointing.
980 my $lh_backup = ThisProject->get_handle('en');
981 open(LEX_FAIL_LOG, ">>wherever/lex.log") || die "GNAARGH $!";
982 sub lex_fail {
983 my($failing_lh, $key, $params) = @_;
984 print LEX_FAIL_LOG scalar(localtime), "\t",
985 ref($failing_lh), "\t", $key, "\n";
986 return $lh_backup->maketext($key,@params);
987 }
988 Some users have expressed that they think this whole mechanism of
990 having a "fail" attribute at all, seems a rather pointless
991 complication. But I want Locale::Maketext to be usable for software
992 projects of any scale and type; and different software projects have
993 different ideas of what the right thing is to do in failure conditions.
994 I could simply say that failure always throws an exception, and that if
995 you want to be careful, you'll just have to wrap every call to
996 $lh->maketext in an eval { }. However, I want programmers to reserve
997 the right (via the "fail" attribute) to treat lookup failure as
998 something other than an exception of the same level of severity as a
999 config file being unreadable, or some essential resource being
1000 inaccessible.
1001 One possibly useful value for the "fail" attribute is the method name
1003 "failure_handler_auto". This is a method defined in the class
1004 Locale::Maketext itself. You set it with:
1005 $lh->fail_with('failure_handler_auto');
1007 Then when you call $lh->maketext(key, ...parameters...) and there's no
1009 key in any of those lexicons, maketext gives up with
1010 return $lh->failure_handler_auto($key, @params);
1012 But failure_handler_auto, instead of dying or anything, compiles $key,
1014 caching it in
1015 $lh->{'failure_lex'}{$key} = $compiled
1017 and then calls the compiled value, and returns that. (I.e., if $key
1019 looks like bracket notation, $compiled is a sub, and we return
1020 &{$compiled}(@params); but if $key is just a plain string, we just
1021 return that.)
1022 The effect of using "failure_auto_handler" is like an AUTO lexicon,
1024 except that it 1) compiles $key even if it starts with "_", and 2) you
1025 have a record in the new hashref $lh->{'failure_lex'} of all the keys
1026 that have failed for this object. This should avoid your program dying
1027 -- as long as your keys aren't actually invalid as bracket code, and as
1028 long as they don't try calling methods that don't exist.
1029 "failure_auto_handler" may not be exactly what you want, but I hope it
1031 at least shows you that maketext failure can be mitigated in any number
1032 of very flexible ways. If you can formalize exactly what you want, you
1033 should be able to express that as a failure handler. You can even make
1034 it default for every object of a given class, by setting it in that
1035 class's init:
1036 sub init {
1038 my $lh = $_[0]; # a newborn handle
1039 $lh->SUPER::init();
1040 $lh->fail_with('my_clever_failure_handler');
1041 return;
1042 }
1043 sub my_clever_failure_handler {
1044 ...you clever things here...
1045 }
1046
1048 Here is a brief checklist on how to use Maketext to localize
1049 applications:
1050 • Decide what system you'll use for lexicon keys. If you insist, you
1052 can use opaque IDs (if you're nostalgic for "catgets"), but I have
1053 better suggestions in the section "Entries in Each Lexicon", above.
1054 Assuming you opt for meaningful keys that double as values (like
1055 "Minimum ([_1]) is larger than maximum ([_2])!\n"), you'll have to
1056 settle on what language those should be in. For the sake of
1057 argument, I'll call this English, specifically American English,
1058 "en-US".
1059 • Create a class for your localization project. This is the name of
1061 the class that you'll use in the idiom:
1062 use Projname::L10N;
1064 my $lh = Projname::L10N->get_handle(...) || die "Language?";
1065 Assuming you call your class Projname::L10N, create a class
1067 consisting minimally of:
1068 package Projname::L10N;
1070 use base qw(Locale::Maketext);
1071 ...any methods you might want all your languages to share...
1072 # And, assuming you want the base class to be an _AUTO lexicon,
1074 # as is discussed a few sections up:
1075 1;
1077 • Create a class for the language your internal keys are in. Name
1079 the class after the language-tag for that language, in lowercase,
1080 with dashes changed to underscores. Assuming your project's first
1081 language is US English, you should call this Projname::L10N::en_us.
1082 It should consist minimally of:
1083 package Projname::L10N::en_us;
1085 use base qw(Projname::L10N);
1086 %Lexicon = (
1087 '_AUTO' => 1,
1088 );
1089 1;
1090 (For the rest of this section, I'll assume that this "first
1092 language class" of Projname::L10N::en_us has _AUTO lexicon.)
1093 • Go and write your program. Everywhere in your program where you
1095 would say:
1096 print "Foobar $thing stuff\n";
1098 instead do it thru maketext, using no variable interpolation in the
1100 key:
1101 print $lh->maketext("Foobar [_1] stuff\n", $thing);
1103 If you get tired of constantly saying "print $lh->maketext",
1105 consider making a functional wrapper for it, like so:
1106 use Projname::L10N;
1108 our $lh;
1109 $lh = Projname::L10N->get_handle(...) || die "Language?";
1110 sub pmt (@) { print( $lh->maketext(@_)) }
1111 # "pmt" is short for "Print MakeText"
1112 $Carp::Verbose = 1;
1113 # so if maketext fails, we see made the call to pmt
1114 Besides whole phrases meant for output, anything language-dependent
1116 should be put into the class Projname::L10N::en_us, whether as
1117 methods, or as lexicon entries -- this is discussed in the section
1118 "Entries in Each Lexicon", above.
1119 • Once the program is otherwise done, and once its localization for
1121 the first language works right (via the data and methods in
1122 Projname::L10N::en_us), you can get together the data for
1123 translation. If your first language lexicon isn't an _AUTO
1124 lexicon, then you already have all the messages explicitly in the
1125 lexicon (or else you'd be getting exceptions thrown when you call
1126 $lh->maketext to get messages that aren't in there). But if you
1127 were (advisedly) lazy and are using an _AUTO lexicon, then you've
1128 got to make a list of all the phrases that you've so far been
1129 letting _AUTO generate for you. There are very many ways to
1130 assemble such a list. The most straightforward is to simply grep
1131 the source for every occurrence of "maketext" (or calls to wrappers
1132 around it, like the above "pmt" function), and to log the following
1133 phrase.
1134 • You may at this point want to consider whether your base class
1136 (Projname::L10N), from which all lexicons inherit from
1137 (Projname::L10N::en, Projname::L10N::es, etc.), should be an _AUTO
1138 lexicon. It may be true that in theory, all needed messages will
1139 be in each language class; but in the presumably unlikely or
1140 "impossible" case of lookup failure, you should consider whether
1141 your program should throw an exception, emit text in English (or
1142 whatever your project's first language is), or some more complex
1143 solution as described in the section "Controlling Lookup Failure",
1144 above.
1145 • Submit all messages/phrases/etc. to translators.
1147 (You may, in fact, want to start with localizing to one other
1149 language at first, if you're not sure that you've properly
1150 abstracted the language-dependent parts of your code.)
1151 Translators may request clarification of the situation in which a
1153 particular phrase is found. For example, in English we are
1154 entirely happy saying "n files found", regardless of whether we
1155 mean "I looked for files, and found n of them" or the rather
1156 distinct situation of "I looked for something else (like lines in
1157 files), and along the way I saw n files." This may involve
1158 rethinking things that you thought quite clear: should "Edit" on a
1159 toolbar be a noun ("editing") or a verb ("to edit")? Is there
1160 already a conventionalized way to express that menu option,
1161 separate from the target language's normal word for "to edit"?
1162 In all cases where the very common phenomenon of quantification
1164 (saying "N files", for any value of N) is involved, each translator
1165 should make clear what dependencies the number causes in the
1166 sentence. In many cases, dependency is limited to words adjacent
1167 to the number, in places where you might expect them ("I found
1168 the-?PLURAL N empty-?PLURAL directory-?PLURAL"), but in some cases
1169 there are unexpected dependencies ("I found-?PLURAL ..."!) as well
1170 as long-distance dependencies "The N directory-?PLURAL could not be
1171 deleted-?PLURAL"!).
1172 Remind the translators to consider the case where N is 0: "0 files
1174 found" isn't exactly natural-sounding in any language, but it may
1175 be unacceptable in many -- or it may condition special kinds of
1176 agreement (similar to English "I didN'T find ANY files").
1177 Remember to ask your translators about numeral formatting in their
1179 language, so that you can override the "numf" method as
1180 appropriate. Typical variables in number formatting are: what to
1181 use as a decimal point (comma? period?); what to use as a thousands
1182 separator (space? nonbreaking space? comma? period? small middot?
1183 prime? apostrophe?); and even whether the so-called "thousands
1184 separator" is actually for every third digit -- I've heard reports
1185 of two hundred thousand being expressible as "2,00,000" for some
1186 Indian (Subcontinental) languages, besides the less surprising
1187 "200 000", "200.000", "200,000", and "200'000". Also, using a set
1188 of numeral glyphs other than the usual ASCII "0"-"9" might be
1189 appreciated, as via "tr/0-9/\x{0966}-\x{096F}/" for getting digits
1190 in Devanagari script (for Hindi, Konkani, others).
1191 The basic "quant" method that Locale::Maketext provides should be
1193 good for many languages. For some languages, it might be useful to
1194 modify it (or its constituent "numerate" method) to take a plural
1195 form in the two-argument call to "quant" (as in "[quant,_1,files]")
1196 if it's all-around easier to infer the singular form from the
1197 plural, than to infer the plural form from the singular.
1198 But for other languages (as is discussed at length in
1200 Locale::Maketext::TPJ13), simple "quant"/"numf" is not enough. For
1201 the particularly problematic Slavic languages, what you may need is
1202 a method which you provide with the number, the citation form of
1203 the noun to quantify, and the case and gender that the sentence's
1204 syntax projects onto that noun slot. The method would then be
1205 responsible for determining what grammatical number that numeral
1206 projects onto its noun phrase, and what case and gender it may
1207 override the normal case and gender with; and then it would look up
1208 the noun in a lexicon providing all needed inflected forms.
1209 • You may also wish to discuss with the translators the question of
1211 how to relate different subforms of the same language tag,
1212 considering how this reacts with "get_handle"'s treatment of these.
1213 For example, if a user accepts interfaces in "en, fr", and you have
1214 interfaces available in "en-US" and "fr", what should they get?
1215 You may wish to resolve this by establishing that "en" and "en-US"
1216 are effectively synonymous, by having one class zero-derive from
1217 the other.
1218 For some languages this issue may never come up (Danish is rarely
1220 expressed as "da-DK", but instead is just "da"). And for other
1221 languages, the whole concept of a "generic" form may verge on being
1222 uselessly vague, particularly for interfaces involving voice media
1223 in forms of Arabic or Chinese.
1224 • Once you've localized your program/site/etc. for all desired
1226 languages, be sure to show the result (whether live, or via
1227 screenshots) to the translators. Once they approve, make every
1228 effort to have it then checked by at least one other speaker of
1229 that language. This holds true even when (or especially when) the
1230 translation is done by one of your own programmers. Some kinds of
1231 systems may be harder to find testers for than others, depending on
1232 the amount of domain-specific jargon and concepts involved -- it's
1233 easier to find people who can tell you whether they approve of your
1234 translation for "delete this message" in an email-via-Web
1235 interface, than to find people who can give you an informed opinion
1236 on your translation for "attribute value" in an XML query tool's
1237 interface.
1238
1240 I recommend reading all of these:
1241 Locale::Maketext::TPJ13 -- my The Perl Journal article about Maketext.
1243 It explains many important concepts underlying Locale::Maketext's
1244 design, and some insight into why Maketext is better than the plain old
1245 approach of having message catalogs that are just databases of sprintf
1246 formats.
1247 File::Findgrep is a sample application/module that uses
1249 Locale::Maketext to localize its messages. For a larger
1250 internationalized system, see also Apache::MP3.
1251 I18N::LangTags.
1253 Win32::Locale.
1255 RFC 3066, Tags for the Identification of Languages, as at
1257 <http://sunsite.dk/RFC/rfc/rfc3066.html>
1258 RFC 2277, IETF Policy on Character Sets and Languages is at
1260 <http://sunsite.dk/RFC/rfc/rfc2277.html> -- much of it is just things
1261 of interest to protocol designers, but it explains some basic concepts,
1262 like the distinction between locales and language-tags.
1263 The manual for GNU "gettext". The gettext dist is available in
1265 "<ftp://prep.ai.mit.edu/pub/gnu/>" -- get a recent gettext tarball and
1266 look in its "doc/" directory, there's an easily browsable HTML version
1267 in there. The gettext documentation asks lots of questions worth
1268 thinking about, even if some of their answers are sometimes wonky,
1269 particularly where they start talking about pluralization.
1270 The Locale/Maketext.pm source. Observe that the module is much shorter
1272 than its documentation!
1273
1275 Copyright (c) 1999-2004 Sean M. Burke. All rights reserved.
1276 This library is free software; you can redistribute it and/or modify it
1278 under the same terms as Perl itself.
1279 This program is distributed in the hope that it will be useful, but
1281 without any warranty; without even the implied warranty of
1282 merchantability or fitness for a particular purpose.
1283
1285 Sean M. Burke "sburke@cpan.org"
1286perl v5.36.0 2022-08-23 Locale::Maketext(3)