1Locale::Maketext(3pm) Perl Programmers Reference Guide Locale::Maketext(3pm)
2
6 Locale::Maketext - framework for localization
7
9 package MyProgram;
10 use strict;
11 use MyProgram::L10N;
12 # ...which inherits from Locale::Maketext
13 my $lh = MyProgram::L10N->get_handle() ⎪⎪ die "What language?";
14 ...
15 # And then any messages your program emits, like:
16 warn $lh->maketext( "Can't open file [_1]: [_2]\n", $f, $! );
17 ...
18
20 It is a common feature of applications (whether run directly, or via
21 the Web) for them to be "localized" -- i.e., for them to a present an
22 English interface to an English-speaker, a German interface to a Ger‐
23 man-speaker, and so on for all languages it's programmed with.
24 Locale::Maketext is a framework for software localization; it provides
25 you with the tools for organizing and accessing the bits of text and
26 text-processing code that you need for producing localized applica‐
27 tions.
28 In order to make sense of Maketext and how all its components fit
30 together, you should probably go read Locale::Maketext::TPJ13, and then
31 read the following documentation.
32 You may also want to read over the source for "File::Findgrep" and its
34 constituent modules -- they are a complete (if small) example applica‐
35 tion that uses Maketext.
36
38 The basic design of Locale::Maketext is object-oriented, and
39 Locale::Maketext is an abstract base class, from which you derive a
40 "project class". The project class (with a name like "TkBoc‐
41 ciBall::Localize", which you then use in your module) is in turn the
42 base class for all the "language classes" for your project (with names
43 "TkBocciBall::Localize::it", "TkBocciBall::Localize::en", "TkBoc‐
44 ciBall::Localize::fr", etc.).
45 A language class is a class containing a lexicon of phrases as class
47 data, and possibly also some methods that are of use in interpreting
48 phrases in the lexicon, or otherwise dealing with text in that lan‐
49 guage.
50 An object belonging to a language class is called a "language handle";
52 it's typically a flyweight object.
53 The normal course of action is to call:
55 use TkBocciBall::Localize; # the localization project class
57 $lh = TkBocciBall::Localize->get_handle();
58 # Depending on the user's locale, etc., this will
59 # make a language handle from among the classes available,
60 # and any defaults that you declare.
61 die "Couldn't make a language handle??" unless $lh;
62 From then on, you use the "maketext" function to access entries in
64 whatever lexicon(s) belong to the language handle you got. So, this:
65 print $lh->maketext("You won!"), "\n";
67 ...emits the right text for this language. If the object in $lh
69 belongs to class "TkBocciBall::Localize::fr" and %TkBocciBall::Local‐
70 ize::fr::Lexicon contains "("You won!" => "Tu as gagne!")", then the
71 above code happily tells the user "Tu as gagne!".
72
74 Locale::Maketext offers a variety of methods, which fall into three
75 categories:
76 · Methods to do with constructing language handles.
78 · "maketext" and other methods to do with accessing %Lexicon data for
80 a given language handle.
81 · Methods that you may find it handy to use, from routines of yours
83 that you put in %Lexicon entries.
84 These are covered in the following section.
86 Construction Methods
88 These are to do with constructing a language handle:
90 · $lh = YourProjClass->get_handle( ...langtags... ) ⎪⎪ die "lg-han‐
92 dle?";
93 This tries loading classes based on the language-tags you give
95 (like "("en-US", "sk", "kon", "es-MX", "ja", "i-klingon")", and for
96 the first class that succeeds, returns YourProjClass::lan‐
97 guage->new().
98 It runs thru the entire given list of language-tags, and finds no
100 classes for those exact terms, it then tries "superordinate" lan‐
101 guage classes. So if no "en-US" class (i.e., YourProjClass::en_us)
102 was found, nor classes for anything else in that list, we then try
103 its superordinate, "en" (i.e., YourProjClass::en), and so on thru
104 the other language-tags in the given list: "es". (The other lan‐
105 guage-tags in our example list: happen to have no superordinates.)
106 If none of those language-tags leads to loadable classes, we then
108 try classes derived from YourProjClass->fallback_languages() and
109 then if nothing comes of that, we use classes named by YourProj‐
110 Class->fallback_language_classes(). Then in the (probably quite
111 unlikely) event that that fails, we just return undef.
112 · $lh = YourProjClass->get_handle() ⎪⎪ die "lg-handle?";
114 When "get_handle" is called with an empty parameter list, magic
116 happens:
117 If "get_handle" senses that it's running in program that was
119 invoked as a CGI, then it tries to get language-tags out of the
120 environment variable "HTTP_ACCEPT_LANGUAGE", and it pretends that
121 those were the languages passed as parameters to "get_handle".
122 Otherwise (i.e., if not a CGI), this tries various OS-specific ways
124 to get the language-tags for the current locale/language, and then
125 pretends that those were the value(s) passed to "get_handle".
126 Currently this OS-specific stuff consists of looking in the envi‐
128 ronment variables "LANG" and "LANGUAGE"; and on MSWin machines
129 (where those variables are typically unused), this also tries using
130 the module Win32::Locale to get a language-tag for whatever lan‐
131 guage/locale is currently selected in the "Regional Settings" (or
132 "International"?) Control Panel. I welcome further suggestions
133 for making this do the Right Thing under other operating systems
134 that support localization.
135 If you're using localization in an application that keeps a config‐
137 uration file, you might consider something like this in your
138 project class:
139 sub get_handle_via_config {
141 my $class = $_[0];
142 my $preferred_language = $Config_settings{'language'};
143 my $lh;
144 if($preferred_language) {
145 $lh = $class->get_handle($chosen_language)
146 ⎪⎪ die "No language handle for \"$chosen_language\" or the like";
147 } else {
148 # Config file missing, maybe?
149 $lh = $class->get_handle()
150 ⎪⎪ die "Can't get a language handle";
151 }
152 return $lh;
153 }
154 · $lh = YourProjClass::langname->new();
156 This constructs a language handle. You usually don't call this
158 directly, but instead let "get_handle" find a language class to
159 "use" and to then call ->new on.
160 · $lh->init();
162 This is called by ->new to initialize newly-constructed language
164 handles. If you define an init method in your class, remember that
165 it's usually considered a good idea to call $lh->SUPER::init in it
166 (presumably at the beginning), so that all classes get a chance to
167 initialize a new object however they see fit.
168 · YourProjClass->fallback_languages()
170 "get_handle" appends the return value of this to the end of what‐
172 ever list of languages you pass "get_handle". Unless you override
173 this method, your project class will inherit Locale::Maketext's
174 "fallback_languages", which currently returns "('i-default', 'en',
175 'en-US')". ("i-default" is defined in RFC 2277).
176 This method (by having it return the name of a language-tag that
178 has an existing language class) can be used for making sure that
179 "get_handle" will always manage to construct a language handle
180 (assuming your language classes are in an appropriate @INC direc‐
181 tory). Or you can use the next method:
182 · YourProjClass->fallback_language_classes()
184 "get_handle" appends the return value of this to the end of the
186 list of classes it will try using. Unless you override this
187 method, your project class will inherit Locale::Maketext's "fall‐
188 back_language_classes", which currently returns an empty list,
189 "()". By setting this to some value (namely, the name of a load‐
190 able language class), you can be sure that "get_handle" will always
191 manage to construct a language handle.
192 The "maketext" Method
194 This is the most important method in Locale::Maketext:
196 $text = $lh->maketext(key, ...parameters for this phrase...);
198 This looks in the %Lexicon of the language handle $lh and all its
200 superclasses, looking for an entry whose key is the string key. Assum‐
201 ing such an entry is found, various things then happen, depending on
202 the value found:
203 If the value is a scalarref, the scalar is dereferenced and returned
205 (and any parameters are ignored). If the value is a coderef, we return
206 &$value($lh, ...parameters...). If the value is a string that doesn't
207 look like it's in Bracket Notation, we return it (after replacing it
208 with a scalarref, in its %Lexicon). If the value does look like it's
209 in Bracket Notation, then we compile it into a sub, replace the string
210 in the %Lexicon with the new coderef, and then we return &$new_sub($lh,
211 ...parameters...).
212 Bracket Notation is discussed in a later section. Note that trying to
214 compile a string into Bracket Notation can throw an exception if the
215 string is not syntactically valid (say, by not balancing brackets
216 right.)
217 Also, calling &$coderef($lh, ...parameters...) can throw any sort of
219 exception (if, say, code in that sub tries to divide by zero). But a
220 very common exception occurs when you have Bracket Notation text that
221 says to call a method "foo", but there is no such method. (E.g., "You
222 have [quatn,_1,ball]." will throw an exception on trying to call
223 $lh->quatn($_[1],'ball') -- you presumably meant "quant".) "maketext"
224 catches these exceptions, but only to make the error message more read‐
225 able, at which point it rethrows the exception.
226 An exception may be thrown if key is not found in any of $lh's %Lexicon
228 hashes. What happens if a key is not found, is discussed in a later
229 section, "Controlling Lookup Failure".
230 Note that you might find it useful in some cases to override the "make‐
232 text" method with an "after method", if you want to translate encod‐
233 ings, or even scripts:
234 package YrProj::zh_cn; # Chinese with PRC-style glyphs
236 use base ('YrProj::zh_tw'); # Taiwan-style
237 sub maketext {
238 my $self = shift(@_);
239 my $value = $self->maketext(@_);
240 return Chineeze::taiwan2mainland($value);
241 }
242 Or you may want to override it with something that traps any excep‐
244 tions, if that's critical to your program:
245 sub maketext {
247 my($lh, @stuff) = @_;
248 my $out;
249 eval { $out = $lh->SUPER::maketext(@stuff) };
250 return $out unless $@;
251 ...otherwise deal with the exception...
252 }
253 Other than those two situations, I don't imagine that it's useful to
255 override the "maketext" method. (If you run into a situation where it
256 is useful, I'd be interested in hearing about it.)
257 $lh->fail_with or $lh->fail_with(PARAM)
259 $lh->failure_handler_auto
260 These two methods are discussed in the section "Controlling Lookup
261 Failure".
262 Utility Methods
264 These are methods that you may find it handy to use, generally from
266 %Lexicon routines of yours (whether expressed as Bracket Notation or
267 not).
268 $language->quant($number, $singular)
270 $language->quant($number, $singular, $plural)
271 $language->quant($number, $singular, $plural, $negative)
272 This is generally meant to be called from inside Bracket Notation
273 (which is discussed later), as in
274 "Your search matched [quant,_1,document]!"
276 It's for quantifying a noun (i.e., saying how much of it there is,
278 while giving the correct form of it). The behavior of this method
279 is handy for English and a few other Western European languages,
280 and you should override it for languages where it's not suitable.
281 You can feel free to read the source, but the current implementa‐
282 tion is basically as this pseudocode describes:
283 if $number is 0 and there's a $negative,
285 return $negative;
286 elsif $number is 1,
287 return "1 $singular";
288 elsif there's a $plural,
289 return "$number $plural";
290 else
291 return "$number " . $singular . "s";
292 #
293 # ...except that we actually call numf to
294 # stringify $number before returning it.
295 So for English (with Bracket Notation) "...[quant,_1,file]..." is
297 fine (for 0 it returns "0 files", for 1 it returns "1 file", and
298 for more it returns "2 files", etc.)
299 But for "directory", you'd want "[quant,_1,directory,directories]"
301 so that our elementary "quant" method doesn't think that the plural
302 of "directory" is "directorys". And you might find that the output
303 may sound better if you specify a negative form, as in:
304 "[quant,_1,file,files,No files] matched your query.\n"
306 Remember to keep in mind verb agreement (or adjectives too, in
308 other languages), as in:
309 "[quant,_1,document] were matched.\n"
311 Because if _1 is one, you get "1 document were matched". An
313 acceptable hack here is to do something like this:
314 "[quant,_1,document was, documents were] matched.\n"
316 $language->numf($number)
318 This returns the given number formatted nicely according to this
319 language's conventions. Maketext's default method is mostly to
320 just take the normal string form of the number (applying sprintf
321 "%G" for only very large numbers), and then to add commas as neces‐
322 sary. (Except that we apply "tr/,./.,/" if $lan‐
323 guage->{'numf_comma'} is true; that's a bit of a hack that's useful
324 for languages that express two million as "2.000.000" and not as
325 "2,000,000").
326 If you want anything fancier, consider overriding this with some‐
328 thing that uses Number::Format, or does something else entirely.
329 Note that numf is called by quant for stringifying all quantifying
331 numbers.
332 $language->sprintf($format, @items)
334 This is just a wrapper around Perl's normal "sprintf" function.
335 It's provided so that you can use "sprintf" in Bracket Notation:
336 "Couldn't access datanode [sprintf,%10x=~[%s~],_1,_2]!\n"
338 returning...
340 Couldn't access datanode Stuff=[thangamabob]!
342 $language->language_tag()
344 Currently this just takes the last bit of "ref($language)", turns
345 underscores to dashes, and returns it. So if $language is an
346 object of class Hee::HOO::Haw::en_us, $language->language_tag()
347 returns "en-us". (Yes, the usual representation for that language
348 tag is "en-US", but case is never considered meaningful in lan‐
349 guage-tag comparison.)
350 You may override this as you like; Maketext doesn't use it for any‐
352 thing.
353 $language->encoding()
355 Currently this isn't used for anything, but it's provided (with
356 default value of "(ref($language) && $language->{'encoding'})) or
357 "iso-8859-1"" ) as a sort of suggestion that it may be useful/nec‐
358 essary to associate encodings with your language handles (whether
359 on a per-class or even per-handle basis.)
360 Language Handle Attributes and Internals
362 A language handle is a flyweight object -- i.e., it doesn't (necessar‐
364 ily) carry any data of interest, other than just being a member of
365 whatever class it belongs to.
366 A language handle is implemented as a blessed hash. Subclasses of
368 yours can store whatever data you want in the hash. Currently the only
369 hash entry used by any crucial Maketext method is "fail", so feel free
370 to use anything else as you like.
371 Remember: Don't be afraid to read the Maketext source if there's any
373 point on which this documentation is unclear. This documentation is
374 vastly longer than the module source itself.
375
377 These are Locale::Maketext's assumptions about the class hierarchy
378 formed by all your language classes:
379 · You must have a project base class, which you load, and which you
381 then use as the first argument in the call to YourProj‐
382 Class->get_handle(...). It should derive (whether directly or
383 indirectly) from Locale::Maketext. It doesn't matter how you name
384 this class, altho assuming this is the localization component of
385 your Super Mega Program, good names for your project class might be
386 SuperMegaProgram::Localization, SuperMegaProgram::L10N, SuperMe‐
387 gaProgram::I18N, SuperMegaProgram::International, or even SuperMe‐
388 gaProgram::Languages or SuperMegaProgram::Messages.
389 · Language classes are what YourProjClass->get_handle will try to
391 load. It will look for them by taking each language-tag (skipping
392 it if it doesn't look like a language-tag or locale-tag!), turning
393 it to all lowercase, turning and dashes to underscores, and append‐
394 ing it to YourProjClass . "::". So this:
395 $lh = YourProjClass->get_handle(
397 'en-US', 'fr', 'kon', 'i-klingon', 'i-klingon-romanized'
398 );
399 will try loading the classes YourProjClass::en_us (note lower‐
401 case!), YourProjClass::fr, YourProjClass::kon, YourProj‐
402 Class::i_klingon and YourProjClass::i_klingon_romanized. (And
403 it'll stop at the first one that actually loads.)
404 · I assume that each language class derives (directly or indirectly)
406 from your project class, and also defines its @ISA, its %Lexicon,
407 or both. But I anticipate no dire consequences if these assump‐
408 tions do not hold.
409 · Language classes may derive from other language classes (altho they
411 should have "use Thatclassname" or "use base qw(...classes...)").
412 They may derive from the project class. They may derive from some
413 other class altogether. Or via multiple inheritance, it may derive
414 from any mixture of these.
415 · I foresee no problems with having multiple inheritance in your
417 hierarchy of language classes. (As usual, however, Perl will com‐
418 plain bitterly if you have a cycle in the hierarchy: i.e., if any
419 class is its own ancestor.)
420
422 A typical %Lexicon entry is meant to signify a phrase, taking some num‐
423 ber (0 or more) of parameters. An entry is meant to be accessed by via
424 a string key in $lh->maketext(key, ...parameters...), which should
425 return a string that is generally meant for be used for "output" to the
426 user -- regardless of whether this actually means printing to STDOUT,
427 writing to a file, or putting into a GUI widget.
428 While the key must be a string value (since that's a basic restriction
430 that Perl places on hash keys), the value in the lexicon can currently
431 be of several types: a defined scalar, scalarref, or coderef. The use
432 of these is explained above, in the section 'The "maketext" Method',
433 and Bracket Notation for strings is discussed in the next section.
434 While you can use arbitrary unique IDs for lexicon keys (like
436 "_min_larger_max_error"), it is often useful for if an entry's key is
437 itself a valid value, like this example error message:
438 "Minimum ([_1]) is larger than maximum ([_2])!\n",
440 Compare this code that uses an arbitrary ID...
442 die $lh->maketext( "_min_larger_max_error", $min, $max )
444 if $min > $max;
445 ...to this code that uses a key-as-value:
447 die $lh->maketext(
449 "Minimum ([_1]) is larger than maximum ([_2])!\n",
450 $min, $max
451 ) if $min > $max;
452 The second is, in short, more readable. In particular, it's obvious
454 that the number of parameters you're feeding to that phrase (two) is
455 the number of parameters that it wants to be fed. (Since you see _1
456 and a _2 being used in the key there.)
457 Also, once a project is otherwise complete and you start to localize
459 it, you can scrape together all the various keys you use, and pass it
460 to a translator; and then the translator's work will go faster if what
461 he's presented is this:
462 "Minimum ([_1]) is larger than maximum ([_2])!\n",
464 => "", # fill in something here, Jacques!
465 rather than this more cryptic mess:
467 "_min_larger_max_error"
469 => "", # fill in something here, Jacques
470 I think that keys as lexicon values makes the completed lexicon entries
472 more readable:
473 "Minimum ([_1]) is larger than maximum ([_2])!\n",
475 => "Le minimum ([_1]) est plus grand que le maximum ([_2])!\n",
476 Also, having valid values as keys becomes very useful if you set up an
478 _AUTO lexicon. _AUTO lexicons are discussed in a later section.
479 I almost always use keys that are themselves valid lexicon values. One
481 notable exception is when the value is quite long. For example, to get
482 the screenful of data that a command-line program might returns when
483 given an unknown switch, I often just use a key "_USAGE_MESSAGE". At
484 that point I then go and immediately to define that lexicon entry in
485 the ProjectClass::L10N::en lexicon (since English is always my "project
486 language"):
487 '_USAGE_MESSAGE' => <<'EOSTUFF',
489 ...long long message...
490 EOSTUFF
491 and then I can use it as:
493 getopt('oDI', \%opts) or die $lh->maketext('_USAGE_MESSAGE');
495 Incidentally, note that each class's %Lexicon inherits-and-extends the
497 lexicons in its superclasses. This is not because these are special
498 hashes per se, but because you access them via the "maketext" method,
499 which looks for entries across all the %Lexicon's in a language class
500 and all its ancestor classes. (This is because the idea of "class
501 data" isn't directly implemented in Perl, but is instead left to indi‐
502 vidual class-systems to implement as they see fit..)
503 Note that you may have things stored in a lexicon besides just phrases
505 for output: for example, if your program takes input from the key‐
506 board, asking a "(Y/N)" question, you probably need to know what equiv‐
507 alent of "Y[es]/N[o]" is in whatever language. You probably also need
508 to know what the equivalents of the answers "y" and "n" are. You can
509 store that information in the lexicon (say, under the keys "~answer_y"
510 and "~answer_n", and the long forms as "~answer_yes" and "~answer_no",
511 where "~" is just an ad-hoc character meant to indicate to program‐
512 mers/translators that these are not phrases for output).
513 Or instead of storing this in the language class's lexicon, you can
515 (and, in some cases, really should) represent the same bit of knowledge
516 as code is a method in the language class. (That leaves a tidy dis‐
517 tinction between the lexicon as the things we know how to say, and the
518 rest of the things in the lexicon class as things that we know how to
519 do.) Consider this example of a processor for responses to French
520 "oui/non" questions:
521 sub y_or_n {
523 return undef unless defined $_[1] and length $_[1];
524 my $answer = lc $_[1]; # smash case
525 return 1 if $answer eq 'o' or $answer eq 'oui';
526 return 0 if $answer eq 'n' or $answer eq 'non';
527 return undef;
528 }
529 ...which you'd then call in a construct like this:
531 my $response;
533 until(defined $response) {
534 print $lh->maketext("Open the pod bay door (y/n)? ");
535 $response = $lh->y_or_n( get_input_from_keyboard_somehow() );
536 }
537 if($response) { $pod_bay_door->open() }
538 else { $pod_bay_door->leave_closed() }
539 Other data worth storing in a lexicon might be things like filenames
541 for language-targetted resources:
542 ...
544 "_main_splash_png"
545 => "/styles/en_us/main_splash.png",
546 "_main_splash_imagemap"
547 => "/styles/en_us/main_splash.incl",
548 "_general_graphics_path"
549 => "/styles/en_us/",
550 "_alert_sound"
551 => "/styles/en_us/hey_there.wav",
552 "_forward_icon"
553 => "left_arrow.png",
554 "_backward_icon"
555 => "right_arrow.png",
556 # In some other languages, left equals
557 # BACKwards, and right is FOREwards.
558 ...
559 You might want to do the same thing for expressing key bindings or the
561 like (since hardwiring "q" as the binding for the function that quits a
562 screen/menu/program is useful only if your language happens to asso‐
563 ciate "q" with "quit"!)
564
566 Bracket Notation is a crucial feature of Locale::Maketext. I mean
567 Bracket Notation to provide a replacement for sprintf formatting.
568 Everything you do with Bracket Notation could be done with a sub block,
569 but bracket notation is meant to be much more concise.
570 Bracket Notation is a like a miniature "template" system (in the sense
572 of Text::Template, not in the sense of C++ templates), where normal
573 text is passed thru basically as is, but text is special regions is
574 specially interpreted. In Bracket Notation, you use brackets ("[...]"
575 -- not "{...}"!) to note sections that are specially interpreted.
576 For example, here all the areas that are taken literally are underlined
578 with a "^", and all the in-bracket special regions are underlined with
579 an X:
580 "Minimum ([_1]) is larger than maximum ([_2])!\n",
582 ^^^^^^^^^ XX ^^^^^^^^^^^^^^^^^^^^^^^^^^ XX ^^^^
583 When that string is compiled from bracket notation into a real Perl
585 sub, it's basically turned into:
586 sub {
588 my $lh = $_[0];
589 my @params = @_;
590 return join '',
591 "Minimum (",
592 ...some code here...
593 ") is larger than maximum (",
594 ...some code here...
595 ")!\n",
596 }
597 # to be called by $lh->maketext(KEY, params...)
598 In other words, text outside bracket groups is turned into string lit‐
600 erals. Text in brackets is rather more complex, and currently follows
601 these rules:
602 · Bracket groups that are empty, or which consist only of whitespace,
604 are ignored. (Examples: "[]", "[ ]", or a [ and a ] with
605 returns and/or tabs and/or spaces between them.
606 Otherwise, each group is taken to be a comma-separated group of
608 items, and each item is interpreted as follows:
609 · An item that is "_digits" or "_-digits" is interpreted as
611 $_[value]. I.e., "_1" is becomes with $_[1], and "_-3" is inter‐
612 preted as $_[-3] (in which case @_ should have at least three ele‐
613 ments in it). Note that $_[0] is the language handle, and is typi‐
614 cally not named directly.
615 · An item "_*" is interpreted to mean "all of @_ except $_[0]".
617 I.e., @_[1..$#_]. Note that this is an empty list in the case of
618 calls like $lh->maketext(key) where there are no parameters (except
619 $_[0], the language handle).
620 · Otherwise, each item is interpreted as a string literal.
622 The group as a whole is interpreted as follows:
624 · If the first item in a bracket group looks like a method name, then
626 that group is interpreted like this:
627 $lh->that_method_name(
629 ...rest of items in this group...
630 ),
631 · If the first item in a bracket group is "*", it's taken as short‐
633 hand for the so commonly called "quant" method. Similarly, if the
634 first item in a bracket group is "#", it's taken to be shorthand
635 for "numf".
636 · If the first item in a bracket group is empty-string, or "_*" or
638 "_digits" or "_-digits", then that group is interpreted as just the
639 interpolation of all its items:
640 join('',
642 ...rest of items in this group...
643 ),
644 Examples: "[_1]" and "[,_1]", which are synonymous; and
646 ""[,ID-(,_4,-,_2,)]"", which compiles as "join "", "ID-(", $_[4],
647 "-", $_[2], ")"".
648 · Otherwise this bracket group is invalid. For example, in the group
650 "[!@#,whatever]", the first item "!@#" is neither empty-string,
651 "_number", "_-number", "_*", nor a valid method name; and so
652 Locale::Maketext will throw an exception of you try compiling an
653 expression containing this bracket group.
654 Note, incidentally, that items in each group are comma-separated, not
656 "/\s*,\s*/"-separated. That is, you might expect that this bracket
657 group:
658 "Hoohah [foo, _1 , bar ,baz]!"
660 would compile to this:
662 sub {
664 my $lh = $_[0];
665 return join '',
666 "Hoohah ",
667 $lh->foo( $_[1], "bar", "baz"),
668 "!",
669 }
670 But it actually compiles as this:
672 sub {
674 my $lh = $_[0];
675 return join '',
676 "Hoohah ",
677 $lh->foo(" _1 ", " bar ", "baz"), #!!!
678 "!",
679 }
680 In the notation discussed so far, the characters "[" and "]" are given
682 special meaning, for opening and closing bracket groups, and "," has a
683 special meaning inside bracket groups, where it separates items in the
684 group. This begs the question of how you'd express a literal "[" or
685 "]" in a Bracket Notation string, and how you'd express a literal comma
686 inside a bracket group. For this purpose I've adopted "~" (tilde) as
687 an escape character: "~[" means a literal '[' character anywhere in
688 Bracket Notation (i.e., regardless of whether you're in a bracket group
689 or not), and ditto for "~]" meaning a literal ']', and "~," meaning a
690 literal comma. (Altho "," means a literal comma outside of bracket
691 groups -- it's only inside bracket groups that commas are special.)
692 And on the off chance you need a literal tilde in a bracket expression,
694 you get it with "~~".
695 Currently, an unescaped "~" before a character other than a bracket or
697 a comma is taken to mean just a "~" and that character. I.e., "~X"
698 means the same as "~~X" -- i.e., one literal tilde, and then one lit‐
699 eral "X". However, by using "~X", you are assuming that no future ver‐
700 sion of Maketext will use "~X" as a magic escape sequence. In practice
701 this is not a great problem, since first off you can just write "~~X"
702 and not worry about it; second off, I doubt I'll add lots of new magic
703 characters to bracket notation; and third off, you aren't likely to
704 want literal "~" characters in your messages anyway, since it's not a
705 character with wide use in natural language text.
706 Brackets must be balanced -- every openbracket must have one matching
708 closebracket, and vice versa. So these are all invalid:
709 "I ate [quant,_1,rhubarb pie."
711 "I ate [quant,_1,rhubarb pie[."
712 "I ate quant,_1,rhubarb pie]."
713 "I ate quant,_1,rhubarb pie[."
714 Currently, bracket groups do not nest. That is, you cannot say:
716 "Foo [bar,baz,[quux,quuux]]\n";
718 If you need a notation that's that powerful, use normal Perl:
720 %Lexicon = (
722 ...
723 "some_key" => sub {
724 my $lh = $_[0];
725 join '',
726 "Foo ",
727 $lh->bar('baz', $lh->quux('quuux')),
728 "\n",
729 },
730 ...
731 );
732 Or write the "bar" method so you don't need to pass it the output from
734 calling quux.
735 I do not anticipate that you will need (or particularly want) to nest
737 bracket groups, but you are welcome to email me with convincing
738 (real-life) arguments to the contrary.
739
741 If maketext goes to look in an individual %Lexicon for an entry for key
742 (where key does not start with an underscore), and sees none, but does
743 see an entry of "_AUTO" => some_true_value, then we actually define
744 $Lexicon{key} = key right then and there, and then use that value as if
745 it had been there all along. This happens before we even look in any
746 superclass %Lexicons!
747 (This is meant to be somewhat like the AUTOLOAD mechanism in Perl's
749 function call system -- or, looked at another way, like the AutoLoader
750 module.)
751 I can picture all sorts of circumstances where you just do not want
753 lookup to be able to fail (since failing normally means that maketext
754 throws a "die", altho see the next section for greater control over
755 that). But here's one circumstance where _AUTO lexicons are meant to
756 be especially useful:
757 As you're writing an application, you decide as you go what messages
759 you need to emit. Normally you'd go to write this:
760 if(-e $filename) {
762 go_process_file($filename)
763 } else {
764 print "Couldn't find file \"$filename\"!\n";
765 }
766 but since you anticipate localizing this, you write:
768 use ThisProject::I18N;
770 my $lh = ThisProject::I18N->get_handle();
771 # For the moment, assume that things are set up so
772 # that we load class ThisProject::I18N::en
773 # and that that's the class that $lh belongs to.
774 ...
775 if(-e $filename) {
776 go_process_file($filename)
777 } else {
778 print $lh->maketext(
779 "Couldn't find file \"[_1]\"!\n", $filename
780 );
781 }
782 Now, right after you've just written the above lines, you'd normally
784 have to go open the file ThisProject/I18N/en.pm, and immediately add an
785 entry:
786 "Couldn't find file \"[_1]\"!\n"
788 => "Couldn't find file \"[_1]\"!\n",
789 But I consider that somewhat of a distraction from the work of getting
791 the main code working -- to say nothing of the fact that I often have
792 to play with the program a few times before I can decide exactly what
793 wording I want in the messages (which in this case would require me to
794 go changing three lines of code: the call to maketext with that key,
795 and then the two lines in ThisProject/I18N/en.pm).
796 However, if you set "_AUTO => 1" in the %Lexicon in, ThisPro‐
798 ject/I18N/en.pm (assuming that English (en) is the language that all
799 your programmers will be using for this project's internal message
800 keys), then you don't ever have to go adding lines like this
801 "Couldn't find file \"[_1]\"!\n"
803 => "Couldn't find file \"[_1]\"!\n",
804 to ThisProject/I18N/en.pm, because if _AUTO is true there, then just
806 looking for an entry with the key "Couldn't find file \"[_1]\"!\n" in
807 that lexicon will cause it to be added, with that value!
808 Note that the reason that keys that start with "_" are immune to _AUTO
810 isn't anything generally magical about the underscore character -- I
811 just wanted a way to have most lexicon keys be autoable, except for
812 possibly a few, and I arbitrarily decided to use a leading underscore
813 as a signal to distinguish those few.
814
816 If you call $lh->maketext(key, ...parameters...), and there's no entry
817 key in $lh's class's %Lexicon, nor in the superclass %Lexicon hash, and
818 if we can't auto-make key (because either it starts with a "_", or
819 because none of its lexicons have "_AUTO => 1,"), then we have failed
820 to find a normal way to maketext key. What then happens in these fail‐
821 ure conditions, depends on the $lh object "fail" attribute.
822 If the language handle has no "fail" attribute, maketext will simply
824 throw an exception (i.e., it calls "die", mentioning the key whose
825 lookup failed, and naming the line number where the calling $lh->make‐
826 text(key,...) was.
827 If the language handle has a "fail" attribute whose value is a coderef,
829 then $lh->maketext(key,...params...) gives up and calls:
830 return &{$that_subref}($lh, $key, @params);
832 Otherwise, the "fail" attribute's value should be a string denoting a
834 method name, so that $lh->maketext(key,...params...) can give up with:
835 return $lh->$that_method_name($phrase, @params);
837 The "fail" attribute can be accessed with the "fail_with" method:
839 # Set to a coderef:
841 $lh->fail_with( \&failure_handler );
842 # Set to a method name:
844 $lh->fail_with( 'failure_method' );
845 # Set to nothing (i.e., so failure throws a plain exception)
847 $lh->fail_with( undef );
848 # Simply read:
850 $handler = $lh->fail_with();
851 Now, as to what you may want to do with these handlers: Maybe you'd
853 want to log what key failed for what class, and then die. Maybe you
854 don't like "die" and instead you want to send the error message to STD‐
855 OUT (or wherever) and then merely "exit()".
856 Or maybe you don't want to "die" at all! Maybe you could use a handler
858 like this:
859 # Make all lookups fall back onto an English value,
861 # but after we log it for later fingerpointing.
862 my $lh_backup = ThisProject->get_handle('en');
863 open(LEX_FAIL_LOG, ">>wherever/lex.log") ⎪⎪ die "GNAARGH $!";
864 sub lex_fail {
865 my($failing_lh, $key, $params) = @_;
866 print LEX_FAIL_LOG scalar(localtime), "\t",
867 ref($failing_lh), "\t", $key, "\n";
868 return $lh_backup->maketext($key,@params);
869 }
870 Some users have expressed that they think this whole mechanism of hav‐
872 ing a "fail" attribute at all, seems a rather pointless complication.
873 But I want Locale::Maketext to be usable for software projects of any
874 scale and type; and different software projects have different ideas of
875 what the right thing is to do in failure conditions. I could simply
876 say that failure always throws an exception, and that if you want to be
877 careful, you'll just have to wrap every call to $lh->maketext in an
878 eval { }. However, I want programmers to reserve the right (via the
879 "fail" attribute) to treat lookup failure as something other than an
880 exception of the same level of severity as a config file being unread‐
881 able, or some essential resource being inaccessible.
882 One possibly useful value for the "fail" attribute is the method name
884 "failure_handler_auto". This is a method defined in class
885 Locale::Maketext itself. You set it with:
886 $lh->fail_with('failure_handler_auto');
888 Then when you call $lh->maketext(key, ...parameters...) and there's no
890 key in any of those lexicons, maketext gives up with
891 return $lh->failure_handler_auto($key, @params);
893 But failure_handler_auto, instead of dying or anything, compiles $key,
895 caching it in $lh->{'failure_lex'}{$key} = $complied, and then calls
896 the compiled value, and returns that. (I.e., if $key looks like
897 bracket notation, $compiled is a sub, and we return &{$com‐
898 piled}(@params); but if $key is just a plain string, we just return
899 that.)
900 The effect of using "failure_auto_handler" is like an AUTO lexicon,
902 except that it 1) compiles $key even if it starts with "_", and 2) you
903 have a record in the new hashref $lh->{'failure_lex'} of all the keys
904 that have failed for this object. This should avoid your program dying
905 -- as long as your keys aren't actually invalid as bracket code, and as
906 long as they don't try calling methods that don't exist.
907 "failure_auto_handler" may not be exactly what you want, but I hope it
909 at least shows you that maketext failure can be mitigated in any number
910 of very flexible ways. If you can formalize exactly what you want, you
911 should be able to express that as a failure handler. You can even make
912 it default for every object of a given class, by setting it in that
913 class's init:
914 sub init {
916 my $lh = $_[0]; # a newborn handle
917 $lh->SUPER::init();
918 $lh->fail_with('my_clever_failure_handler');
919 return;
920 }
921 sub my_clever_failure_handler {
922 ...you clever things here...
923 }
924
926 Here is a brief checklist on how to use Maketext to localize applica‐
927 tions:
928 · Decide what system you'll use for lexicon keys. If you insist, you
930 can use opaque IDs (if you're nostalgic for "catgets"), but I have
931 better suggestions in the section "Entries in Each Lexicon", above.
932 Assuming you opt for meaningful keys that double as values (like
933 "Minimum ([_1]) is larger than maximum ([_2])!\n"), you'll have to
934 settle on what language those should be in. For the sake of argu‐
935 ment, I'll call this English, specifically American English,
936 "en-US".
937 · Create a class for your localization project. This is the name of
939 the class that you'll use in the idiom:
940 use Projname::L10N;
942 my $lh = Projname::L10N->get_handle(...) ⎪⎪ die "Language?";
943 Assuming your call your class Projname::L10N, create a class con‐
945 sisting minimally of:
946 package Projname::L10N;
948 use base qw(Locale::Maketext);
949 ...any methods you might want all your languages to share...
950 # And, assuming you want the base class to be an _AUTO lexicon,
952 # as is discussed a few sections up:
953 1;
955 · Create a class for the language your internal keys are in. Name
957 the class after the language-tag for that language, in lowercase,
958 with dashes changed to underscores. Assuming your project's first
959 language is US English, you should call this Projname::L10N::en_us.
960 It should consist minimally of:
961 package Projname::L10N::en_us;
963 use base qw(Projname::L10N);
964 %Lexicon = (
965 '_AUTO' => 1,
966 );
967 1;
968 (For the rest of this section, I'll assume that this "first lan‐
970 guage class" of Projname::L10N::en_us has _AUTO lexicon.)
971 · Go and write your program. Everywhere in your program where you
973 would say:
974 print "Foobar $thing stuff\n";
976 instead do it thru maketext, using no variable interpolation in the
978 key:
979 print $lh->maketext("Foobar [_1] stuff\n", $thing);
981 If you get tired of constantly saying "print $lh->maketext", con‐
983 sider making a functional wrapper for it, like so:
984 use Projname::L10N;
986 use vars qw($lh);
987 $lh = Projname::L10N->get_handle(...) ⎪⎪ die "Language?";
988 sub pmt (@) { print( $lh->maketext(@_)) }
989 # "pmt" is short for "Print MakeText"
990 $Carp::Verbose = 1;
991 # so if maketext fails, we see made the call to pmt
992 Besides whole phrases meant for output, anything language-dependent
994 should be put into the class Projname::L10N::en_us, whether as
995 methods, or as lexicon entries -- this is discussed in the section
996 "Entries in Each Lexicon", above.
997 · Once the program is otherwise done, and once its localization for
999 the first language works right (via the data and methods in Proj‐
1000 name::L10N::en_us), you can get together the data for translation.
1001 If your first language lexicon isn't an _AUTO lexicon, then you
1002 already have all the messages explicitly in the lexicon (or else
1003 you'd be getting exceptions thrown when you call $lh->maketext to
1004 get messages that aren't in there). But if you were (advisedly)
1005 lazy and are using an _AUTO lexicon, then you've got to make a list
1006 of all the phrases that you've so far been letting _AUTO generate
1007 for you. There are very many ways to assemble such a list. The
1008 most straightforward is to simply grep the source for every occur‐
1009 rence of "maketext" (or calls to wrappers around it, like the above
1010 "pmt" function), and to log the following phrase.
1011 · You may at this point want to consider whether the your base class
1013 (Projname::L10N) that all lexicons inherit from (Proj‐
1014 name::L10N::en, Projname::L10N::es, etc.) should be an _AUTO lexi‐
1015 con. It may be true that in theory, all needed messages will be in
1016 each language class; but in the presumably unlikely or "impossible"
1017 case of lookup failure, you should consider whether your program
1018 should throw an exception, emit text in English (or whatever your
1019 project's first language is), or some more complex solution as
1020 described in the section "Controlling Lookup Failure", above.
1021 · Submit all messages/phrases/etc. to translators.
1023 (You may, in fact, want to start with localizing to one other lan‐
1025 guage at first, if you're not sure that you've property abstracted
1026 the language-dependent parts of your code.)
1027 Translators may request clarification of the situation in which a
1029 particular phrase is found. For example, in English we are
1030 entirely happy saying "n files found", regardless of whether we
1031 mean "I looked for files, and found n of them" or the rather dis‐
1032 tinct situation of "I looked for something else (like lines in
1033 files), and along the way I saw n files." This may involve
1034 rethinking things that you thought quite clear: should "Edit" on a
1035 toolbar be a noun ("editing") or a verb ("to edit")? Is there
1036 already a conventionalized way to express that menu option, sepa‐
1037 rate from the target language's normal word for "to edit"?
1038 In all cases where the very common phenomenon of quantification
1040 (saying "N files", for any value of N) is involved, each translator
1041 should make clear what dependencies the number causes in the sen‐
1042 tence. In many cases, dependency is limited to words adjacent to
1043 the number, in places where you might expect them ("I found
1044 the-?PLURAL N empty-?PLURAL directory-?PLURAL"), but in some cases
1045 there are unexpected dependencies ("I found-?PLURAL ..."!) as well
1046 as long-distance dependencies "The N directory-?PLURAL could not be
1047 deleted-?PLURAL"!).
1048 Remind the translators to consider the case where N is 0: "0 files
1050 found" isn't exactly natural-sounding in any language, but it may
1051 be unacceptable in many -- or it may condition special kinds of
1052 agreement (similar to English "I didN'T find ANY files").
1053 Remember to ask your translators about numeral formatting in their
1055 language, so that you can override the "numf" method as appropri‐
1056 ate. Typical variables in number formatting are: what to use as a
1057 decimal point (comma? period?); what to use as a thousands separa‐
1058 tor (space? nonbreaking space? comma? period? small middot? prime?
1059 apostrophe?); and even whether the so-called "thousands separator"
1060 is actually for every third digit -- I've heard reports of two hun‐
1061 dred thousand being expressible as "2,00,000" for some Indian (Sub‐
1062 continental) languages, besides the less surprising "200 000",
1063 "200.000", "200,000", and "200'000". Also, using a set of numeral
1064 glyphs other than the usual ASCII "0"-"9" might be appreciated, as
1065 via "tr/0-9/\x{0966}-\x{096F}/" for getting digits in Devanagari
1066 script (for Hindi, Konkani, others).
1067 The basic "quant" method that Locale::Maketext provides should be
1069 good for many languages. For some languages, it might be useful to
1070 modify it (or its constituent "numerate" method) to take a plural
1071 form in the two-argument call to "quant" (as in "[quant,_1,files]")
1072 if it's all-around easier to infer the singular form from the plu‐
1073 ral, than to infer the plural form from the singular.
1074 But for other languages (as is discussed at length in Locale::Make‐
1076 text::TPJ13), simple "quant"/"numerify" is not enough. For the
1077 particularly problematic Slavic languages, what you may need is a
1078 method which you provide with the number, the citation form of the
1079 noun to quantify, and the case and gender that the sentence's syn‐
1080 tax projects onto that noun slot. The method would then be respon‐
1081 sible for determining what grammatical number that numeral projects
1082 onto its noun phrase, and what case and gender it may override the
1083 normal case and gender with; and then it would look up the noun in
1084 a lexicon providing all needed inflected forms.
1085 · You may also wish to discuss with the translators the question of
1087 how to relate different subforms of the same language tag, consid‐
1088 ering how this reacts with "get_handle"'s treatment of these. For
1089 example, if a user accepts interfaces in "en, fr", and you have
1090 interfaces available in "en-US" and "fr", what should they get?
1091 You may wish to resolve this by establishing that "en" and "en-US"
1092 are effectively synonymous, by having one class zero-derive from
1093 the other.
1094 For some languages this issue may never come up (Danish is rarely
1096 expressed as "da-DK", but instead is just "da"). And for other
1097 languages, the whole concept of a "generic" form may verge on being
1098 uselessly vague, particularly for interfaces involving voice media
1099 in forms of Arabic or Chinese.
1100 · Once you've localized your program/site/etc. for all desired lan‐
1102 guages, be sure to show the result (whether live, or via screen‐
1103 shots) to the translators. Once they approve, make every effort to
1104 have it then checked by at least one other speaker of that lan‐
1105 guage. This holds true even when (or especially when) the transla‐
1106 tion is done by one of your own programmers. Some kinds of systems
1107 may be harder to find testers for than others, depending on the
1108 amount of domain-specific jargon and concepts involved -- it's eas‐
1109 ier to find people who can tell you whether they approve of your
1110 translation for "delete this message" in an email-via-Web inter‐
1111 face, than to find people who can give you an informed opinion on
1112 your translation for "attribute value" in an XML query tool's
1113 interface.
1114
1116 I recommend reading all of these:
1117 Locale::Maketext::TPJ13 -- my The Perl Journal article about Maketext.
1119 It explains many important concepts underlying Locale::Maketext's
1120 design, and some insight into why Maketext is better than the plain old
1121 approach of just having message catalogs that are just databases of
1122 sprintf formats.
1123 File::Findgrep is a sample application/module that uses Locale::Make‐
1125 text to localize its messages. For a larger internationalized system,
1126 see also Apache::MP3.
1127 I18N::LangTags.
1129 Win32::Locale.
1131 RFC 3066, Tags for the Identification of Languages, as at http://sun‐
1133 site.dk/RFC/rfc/rfc3066.html
1134 RFC 2277, IETF Policy on Character Sets and Languages is at http://sun‐
1136 site.dk/RFC/rfc/rfc2277.html -- much of it is just things of interest
1137 to protocol designers, but it explains some basic concepts, like the
1138 distinction between locales and language-tags.
1139 The manual for GNU "gettext". The gettext dist is available in
1141 "ftp://prep.ai.mit.edu/pub/gnu/" -- get a recent gettext tarball and
1142 look in its "doc/" directory, there's an easily browsable HTML version
1143 in there. The gettext documentation asks lots of questions worth
1144 thinking about, even if some of their answers are sometimes wonky, par‐
1145 ticularly where they start talking about pluralization.
1146 The Locale/Maketext.pm source. Obverse that the module is much shorter
1148 than its documentation!
1149
1151 Copyright (c) 1999-2004 Sean M. Burke. All rights reserved.
1152 This library is free software; you can redistribute it and/or modify it
1154 under the same terms as Perl itself.
1155 This program is distributed in the hope that it will be useful, but
1157 without any warranty; without even the implied warranty of mer‐
1158 chantability or fitness for a particular purpose.
1159
1161 Sean M. Burke "sburke@cpan.org"
1162perl v5.8.8 2001-09-21 Locale::Maketext(3pm)