1Text::Autoformat(3) User Contributed Perl Documentation Text::Autoformat(3)
2
6 Text::Autoformat - Automatic text wrapping and reformatting
7
9 This document describes version 1.72 of Text::Autoformat
10
12 # Minimal use: read from STDIN, format to STDOUT...
13 use Text::Autoformat;
15 autoformat;
16 # In-memory formatting...
18 $formatted = autoformat $rawtext;
20 # Configuration...
22 $formatted = autoformat $rawtext, { %options };
24 # Margins (1..72 by default)...
26 $formatted = autoformat $rawtext, { left=>8, right=>70 };
28 # Justification (left by default)...
30 $formatted = autoformat $rawtext, { justify => 'left' };
32 $formatted = autoformat $rawtext, { justify => 'right' };
33 $formatted = autoformat $rawtext, { justify => 'full' };
34 $formatted = autoformat $rawtext, { justify => 'centre' };
35 # Filling (does so by default)...
37 $formatted = autoformat $rawtext, { fill=>0 };
39 # Squeezing whitespace (does so by default)...
41 $formatted = autoformat $rawtext, { squeeze=>0 };
43 # Select appropriate tabspacing (default is 8 spaces per tab):
45 $formatted = autoformat $rawtext, { tabspace=>4 };
47 # Case conversions...
49 $formatted = autoformat $rawtext, { case => 'lower' };
51 $formatted = autoformat $rawtext, { case => 'upper' };
52 $formatted = autoformat $rawtext, { case => 'sentence' };
53 $formatted = autoformat $rawtext, { case => 'title' };
54 $formatted = autoformat $rawtext, { case => 'highlight' };
55 $formatted = autoformat $rawtext, { case => \&my_case_func };
56 # Selective reformatting
58 $formatted = autoformat $rawtext, { ignore=>qr/^\t/ };
60
62 The problem
63 Perl plaintext formatters just aren't smart enough. Given a typical
64 piece of plaintext in need of formatting:
65 In comp.lang.perl.misc you wrote:
67 : > <CN = Clooless Noobie> writes:
68 : > CN> PERL sux because:
69 : > CN> * It doesn't have a switch statement and you have to put $
70 : > CN>signs in front of everything
71 : > CN> * There are too many OR operators: having |, || and 'or'
72 : > CN>operators is confusing
73 : > CN> * VB rools, yeah!!!!!!!!!
74 : > CN> So anyway, how can I stop reloads on a web page?
75 : > CN> Email replies only, thanks - I don't read this newsgroup.
76 : >
77 : > Begone, sirrah! You are a pathetic, Bill-loving, microcephalic
78 : > script-infant.
79 : Sheesh, what's with this group - ask a question, get toasted! And how
80 : *dare* you accuse me of Ianuphilia!
81 both the venerable Unix fmt tool and Perl's standard Text::Wrap module
83 produce:
84 In comp.lang.perl.misc you wrote: : > <CN = Clooless Noobie>
86 writes: : > CN> PERL sux because: : > CN> * It doesn't
87 have a switch statement and you have to put $ : > CN>signs in
88 front of everything : > CN> * There are too many OR
89 operators: having |, || and 'or' : > CN>operators is confusing
90 : > CN> * VB rools, yeah!!!!!!!!! : > CN> So anyway, how
91 can I stop reloads on a web page? : > CN> Email replies only,
92 thanks - I don't read this newsgroup. : > : > Begone, sirrah!
93 You are a pathetic, Bill-loving, microcephalic : >
94 script-infant. : Sheesh, what's with this group - ask a
95 question, get toasted! And how : *dare* you accuse me of
96 Ianuphilia!
97 Other formatting modules -- such as Text::Correct and Text::Format --
99 provide more control over their output, but produce equally poor
100 results when applied to arbitrary input. They simply don't understand
101 the structural conventions of the text they're reformatting.
102 The solution
104 The Text::Autoformat module provides a subroutine named "autoformat"
105 that wraps text to specified margins. However, "autoformat" reformats
106 its input by analysing the text's structure, so it wraps the above
107 example like so:
108 In comp.lang.perl.misc you wrote:
110 : > <CN = Clooless Noobie> writes:
111 : > CN> PERL sux because:
112 : > CN> * It doesn't have a switch statement and you
113 : > CN> have to put $ signs in front of everything
114 : > CN> * There are too many OR operators: having |, ||
115 : > CN> and 'or' operators is confusing
116 : > CN> * VB rools, yeah!!!!!!!!! So anyway, how can I
117 : > CN> stop reloads on a web page? Email replies
118 : > CN> only, thanks - I don't read this newsgroup.
119 : >
120 : > Begone, sirrah! You are a pathetic, Bill-loving,
121 : > microcephalic script-infant.
122 : Sheesh, what's with this group - ask a question, get toasted!
123 : And how *dare* you accuse me of Ianuphilia!
124 Note that the various quoting conventions have been observed. In fact,
126 their structure has been used to determine where some paragraphs begin.
127 Furthermore "autoformat" correctly distinguished between the leading
128 '*' bullets of the nested list (which were outdented) and the leading
129 emphatic '*' of "*dare*" (which was inlined).
130
132 Paragraphs
133 The fundamental task of the "autoformat" subroutine is to identify and
134 rearrange independent paragraphs in a text. Paragraphs typically
135 consist of a series of lines containing at least one non-whitespace
136 character, followed by one or more lines containing only optional
137 whitespace. This is a more liberal definition than many other
138 formatters use: most require an empty line to terminate a paragraph.
139 Paragraphs may also be denoted by bulleting, numbering, or quoting (see
140 the following sections).
141 Once a paragraph has been isolated, "autoformat" fills and re-wraps its
143 lines according to the margins that are specified in its argument list.
144 These are placed after the text to be formatted, in a hash reference:
145 $tidied = autoformat($messy, {left=>20, right=>60});
147 By default, "autoformat" uses a left margin of 1 (first column) and a
149 right margin of 72.
150 You can also control whether (and how) "autoformat" breaks words at the
152 end of a line, using the 'break' option:
153 # Turn off all hyphenation
155 use Text::Autoformat qw(autoformat break_wrap);
156 $tidied = autoformat($messy, {break=>break_wrap});
157 # Default hyphenation
159 use Text::Autoformat qw(autoformat break_at);
160 $tidied = autoformat($messy, {break=>break_at('-')});
161 # Use TeX::Hyphen module's hyphenation (module must be installed)
163 use Text::Autoformat qw(autoformat break_TeX);
164 $tidied = autoformat($messy, {break=>break_TeX});
165 Normally, "autoformat" only reformats the first paragraph it
167 encounters, and leaves the remainder of the text unaltered. This
168 behaviour is useful because it allows a one-liner invoking the
169 subroutine to be mapped onto a convenient keystroke in a text editor,
170 to provide one-paragraph-at-a-time reformatting:
171 % cat .exrc
173 map f !Gperl -MText::Autoformat -e'autoformat'
175 (Note that to facilitate such one-liners, if "autoformat" is called in
177 a void context without any text data, it takes its text from "STDIN"
178 and writes its result to "STDOUT").
179 To enable "autoformat" to rearrange the entire input text at once, the
181 "all" argument is used:
182 $tidied_all = autoformat($messy, {left=>20, right=>60, all=>1});
184 "autoformat" can also be directed to selectively reformat paragraphs,
186 using the "ignore" argument:
187 $tidied_some = autoformat($messy, {ignore=>qr/^[ \t]/});
189 The value for "ignore" may be a "qr"'d regex, a subroutine reference,
191 or the special string 'indented'.
192 If a regex is specified, any paragraph whose original text matches that
194 regex will not be reformatted (i.e. it will be printed verbatim).
195 If a subroutine is specified, that subroutine will be called once for
197 each paragraph (with $_ set to the paragraph's text). The subroutine is
198 expected to return a true or false value. If it returns true, the
199 paragraph will not be reformatted.
200 If the value of the "ignore" option is the string 'indented',
202 "autoformat" will ignore any paragraph in which every line begins with
203 a whitespace.
204 You may also specify multiple "ignore" options by including them in an
206 array-ref:
207 $tidied_mesg = autoformat($messy, {ignore=>[qr/1/,'indented']});
209 One other special case of ignorance is ignoring mail headers and
211 signature. This option is specified using the "mail" argument:
212 $tidied_mesg = autoformat($messy_mesg, {mail=>1});
214 Note that the "ignore" or "mail" options automatically imply "all".
216 Bulleting and (re-)numbering
218 Often plaintext will include lists that are either:
219 * bulleted,
221 * simply numbered (i.e. 1., 2., 3., etc.), or
222 * hierarchically numbered (1, 1.1, 1.2, 1.3, 2, 2.1. and so forth).
223 In such lists, each bulleted item is implicitly a separate paragraph,
225 and is formatted individually, with the appropriate indentation:
226 * bulleted,
228 * simply numbered (i.e. 1., 2., 3.,
229 etc.), or
230 * hierarchically numbered (1, 1.1,
231 1.2, 1.3, 2, 2.1. and so forth).
232 More importantly, if the points are numbered, the numbering is checked
234 and reordered. For example, a list whose points have been rearranged:
235 1. Analyze problem
237 3. Design algorithm
238 1. Code solution
239 5. Test
240 4. Ship
241 would be renumbered automatically by "autoformat":
243 1. Analyze problem
245 2. Design algorithm
246 3. Code solution
247 4. Test
248 5. Ship
249 The same reordering would be performed if the "numbering" was by
251 letters ("a." "b." "c." etc.) or Roman numerals ("i." "ii." "iii.)" or
252 by some combination of these ("1a." "1b." "2a." "2b." etc.) Handling
253 disordered lists of letters and Roman numerals presents an interesting
254 challenge. A list such as:
255 A. Put cat in box.
257 D. Close lid.
258 E. Activate Geiger counter.
259 should be reordered as "A." "B." "C.," whereas:
261 I. Put cat in box.
263 D. Close lid.
264 XLI. Activate Geiger counter.
265 should be reordered "I." "II." "III."
267 The "autoformat" subroutine solves this problem by always interpreting
269 alphabetic bullets as being letters, unless the full list consists only
270 of valid Roman numerals, at least one of which is two or more
271 characters long.
272 Note that renumbering starts at the first number actually given, rather
274 than restarting at the first possible number. To renumber from 1 (or
275 A.) you must change the first numbered bullet to that.
276 If automatic renumbering isn't wanted, just specify the 'renumber'
278 option with a false value.
279 Note that normal numbers above 1000 at the start of a line are no
281 longer considered to be paragraph numbering. Numbered paragraphs
282 running that high are exceptionally rare, and much rarer than
283 paragraphs that look like these:
284 Although it has long been popular (especially in the year
286 2001) to point out that we now live in the Future, many
287 of the promised miracles of Future Life have failed to
288 eventuate. This is a new phenomenon (it didn't happen in
289 1001) because the idea that the future might be different
290 is a new phenomenon.
291 which the former numbering rules caused to be formatted like this:
293 Although it has long been popular (especially in the year
295 2001) to point out that we now live in the Future, many of the
297 promised miracles of Future Life have failed to eventuate.
298 This is a new phenomenon (it didn't happen in
299 2002) because the idea that the future might be different is a
301 new phenomenon.
302 but which are now formatted:
304 Although it has long been popular (especially in the year 2001)
306 to point out that we now live in the Future, many of the
307 promised miracles of Future Life have failed to eventuate. This
308 is a new phenomenon (it didn't happen in 1001) because the idea
309 that the future might be different is a new phenomenon.
310 If you want numbers less than 1000 (or other characters strings
312 currently treated as bullets) to be ignored in this way, you can turn
313 of list formatting entirely by setting the 'lists' option to a false
314 value.
315 You can also select which kinds of lists are recognized, by using a
317 string as the value of lists:
318 # Don't recognize Roman numerals or alphabetics as list markers...
320 autoformat { lists => 'number, bullet' }, $text;
321 # Don't recognize bullets or numbers as list markers...
323 autoformat { lists => 'roman, alpha' }, $text;
324 # Recognize everything except Roman numerals as list markers...
326 autoformat { lists => 'number, bullet, alpha' }, $text;
327 The string should contain one or more of the following words: "number",
329 "bullet", "alpha", "roman". "autoformat()" will ignore any list type
330 that doesn't appear in the 'lists' string.
331 Quoting
333 Another case in which contiguous lines may be interpreted as belonging
334 to different paragraphs, is where they are quoted with distinct
335 quoters. For example:
336 : > CN> So anyway, how can I stop reloads on a web page? Email
338 : > CN> replies only, thanks - I don't read this newsgroup.
339 : > Begone, sirrah! You are a pathetic, Bill-loving,
340 : > microcephalic script-infant.
341 : Sheesh, what's with this group - ask a question, get toasted!
342 : And how *dare* you accuse me of Ianuphilia!
343 "autoformat" recognizes the various quoting conventions used in this
345 example and treats it as three paragraphs to be independently
346 reformatted.
347 You may also override the default set of recognized quoters by
349 specifying a 'quoter' argument when calling "autoformat()". For
350 example, to format lines such as:
351 // This is a comment
353 // in the standard C(++)
354 // comment-to-EOL
355 // format
356 specify:
358 autoformat($text, { quoter =E<gt> qr{//} })
360 Instead of completely replacing the existing set of quoters, you can
362 extend them by specifying a pattern that includes the metasequence
363 "<QUOTER>", which is then replaced by the module's standard pattern for
364 quoters. So, for example, to add "//" to the set of existing quoters:
365 autoformat($text, { quoter =E<gt> qr{//|<QUOTER>} })
367 Block quotations present a different challenge. A typical formatter
369 would render the following quotation:
370 "We are all of us in the gutter, but some of us are looking at
372 the stars"
373 -- Oscar Wilde
374 like so:
376 "We are all of us in the gutter, but some of us are looking at
378 the stars" -- Oscar Wilde
379 "autoformat" recognizes the quotation structure by matching the
381 following regular expression against the text component of each
382 paragraph:
383 / \A(\s*) # leading whitespace for quotation (["']|``) # opening
385 quotemark (.*) # quotation (''|\2) # closing quotemark \s*?\n #
386 trailing whitespace after quotation (\1[ ]+) # leading
387 whitespace for attribution
388 # (must be indented more than
389 # quotation)
390 (--|-) # attribution introducer ([^\n]*?\n) # first
391 attribution line ((\5[^\n]*?$)*) # other attribution lines
392 # (indented no less than first line)
393 \s*\Z # optional whitespace to end of paragraph /xsm
394 When reformatted (see below), the indentation and the attribution
396 structure will be preserved:
397 "We are all of us in the gutter, but some of us are looking
399 at the stars"
400 -- Oscar Wilde
401 Widow control
403 Note that in the last example, "autoformat" broke the line at column
404 68, four characters earlier than it should have. It did so because, if
405 the full margin width had been used, the formatting would have left the
406 last two words by themselves on an oddly short last line:
407 "We are all of us in the gutter, but some of us are looking at
409 the stars"
410 This phenomenon is known as "widowing" and is heavily frowned upon in
412 typesetting circles. It looks ugly in plaintext too, so "autoformat"
413 avoids it by stealing extra words from earlier lines in a paragraph, so
414 as to leave enough for a reasonable last line. The heuristic used is
415 that final lines must be at least 10 characters long (though this
416 number may be adjusted by passing a "widow => minlength" argument to
417 "autoformat").
418 If the last line is too short, the paragraph's right margin is reduced
420 by one column, and the paragraph is reformatted. This process iterates
421 until either the last line exceeds nine characters or the margins have
422 been narrowed by 10% of their original separation. In the latter case,
423 the reformatter gives up and uses its original formatting.
424 Justification
426 The "autoformat" subroutine also takes a named argument: "{justify =>
427 type}", which specifies how each paragraph is to be justified. The
428 options are: 'left' (the default), "'right'," 'centre' (or 'center'),
429 and 'full'. These act on the complete paragraph text (but not on any
430 quoters before that text). For example, with 'right' justification:
431 R3> Now is the Winter of our discontent made
433 R3> glorious Summer by this son of York. And all
434 R3> the clouds that lour'd upon our house In the
435 R3> deep bosom of the ocean buried.
436 Full justification is interesting in a fixed-width medium like
438 plaintext because it usually results in uneven spacing between words.
439 Typically, formatters provide this by distributing the extra spaces
440 into the first available gaps of each line:
441 R3> Now is the Winter of our discontent made
443 R3> glorious Summer by this son of York. And all
444 R3> the clouds that lour'd upon our house In
445 R3> the deep bosom of the ocean buried.
446 This produces a rather jarring visual effect, so "autoformat" reverses
448 the strategy and inserts extra spaces at the end of lines:
449 R3> Now is the Winter of our discontent made
451 R3> glorious Summer by this son of York. And all
452 R3> the clouds that lour'd upon our house In
453 R3> the deep bosom of the ocean buried.
454 Most readers find this less disconcerting.
456 Implicit centring
458 Even if explicit centring is not specified, "autoformat" will attempt
459 to automatically detect centred paragraphs and preserve their
460 justification. It does this by examining each line of the paragraph and
461 asking: "if this line were part of a centred paragraph, where would the
462 centre line have been?"
463 The answer can be determined by adding the length of leading whitespace
465 before the first word, plus half the length of the full set of words on
466 the line. That is, for a single line:
467 $line =~ /^(\s*)(.*?)(\s*)$/ $centre =
469 length($1)+0.5*length($2);
470 By making the same estimate for every line, and then comparing the
472 estimates, it is possible to deduce whether all the lines are centred
473 with respect to the same axis of symmetry (with an allowance of ±1 to
474 cater for the inevitable rounding when the centre positions of even-
475 length rows were originally computed). If a common axis of symmetry is
476 detected, "autoformat" assumes that the lines are supposed to be
477 centred, and switches to centre-justification mode for that paragraph.
478 Note that this behaviour can to switched off entirely by setting the
480 "autocentre" argument false.
481 Case transformations
483 The "autoformat" subroutine can also optionally perform case
484 conversions on the text it processes. The "{case => type}" argument
485 allows the user to specify six different conversions:
486 'upper'
488 This mode unconditionally converts every letter in the reformatted
489 text to upper-case;
490 'lower'
492 This mode unconditionally converts every letter in the reformatted
493 text to lower-case;
494 'sentence'
496 This mode attempts to generate correctly-cased sentences from the
497 input text. That is, the first letter after a sentence-terminating
498 punctuator is converted to upper-case. Then, each subsequent word
499 in the sentence is converted to lower-case, unless that word is
500 originally mixed-case or contains punctuation. For example, under
501 "{case => 'sentence'}":
502 'POVERTY, MISERY, ETC. are the lot of the PhD candidate. alas!'
504 becomes:
506 'Poverty, misery, etc. are the lot of the PhD candidate. Alas!'
508 Note that "autoformat" is clever enough to recognize that the
510 period after abbreviations such as "etc." is not a sentence
511 terminator.
512 If the argument is specified as 'sentence ' (with one or more
514 trailing whitespace characters) those characters are used to
515 replace the single space that appears at the end of the sentence.
516 For example, "autoformat($text, {case=>'sentence '}") would
517 produce:
518 'Poverty, misery, etc. are the lot of the PhD candidate. Alas!'
520 'title'
522 This mode behaves like 'sentence' except that the first letter of
523 every word is capitalized:
524 'What I Did On My Summer Vacation In Monterey'
526 'highlight'
528 This mode behaves like 'title' except that trivial words are not
529 capitalized:
530 'What I Did on my Summer Vacation in Monterey'
532 "sub{...}"
534 If the argument for 'case' is a subroutine reference, that
535 subroutine is applied to each word and the result replaces the word
536 in the text.
537 For example, to convert a string to hostage-case:
539 my $ransom_note = sub {
541 return join "", # ^ Reconcatenate
542 map {/[aeiou]/i ? lc : uc} # | uPPeR aND LoWeR each
543 split //, # | Break into chars
544 shift; # | Take argument
545 };
546 $text = autoformat($text, {case => $ransom_note });
548 # "FoR eXaMPLe, To CoNVeRT a STRiNG To HoSTaGe-CaSe:"
549 Or to highlight particular words:
551 my @SPECIAL = qw( perl camel wall );
553 sub highlight_specials {
554 my ($word) = @_;
555 return $word ~~ @SPECIAL ? uc($word) : $word;
556 }
557 $text = autoformat($text, {case => \&highlight_specials});
559 # "It is easier for a CAMEL to pass through a WALL of PERL..."
560 Selective reformatting
562 You can select which paragraphs "autoformat" actually reformats (or,
563 rather, those it doesn't reformat) using the "ignore" flag.
564 For example:
566 # Reformat all paras except those containing "verbatim"...
568 print autoformat { all => 1, ignore => qr/verbatim/i }, $text;
569 # Reformat all paras except those less that 3 lines long...
571 print autoformat { all => 1, ignore => sub { tr/\n/\n/ < 3
572 } }, $text;
573 # Reformat all paras except those that are indented...
575 print autoformat { all => 1, ignore => qr/^\s/m }, $text;
576 # Reformat all paras except those that are indented (easier)...
578 print autoformat { all => 1, ignore => 'indented' }, $text;
579 Handling tabs
581 Text::Autoformat replaces any tabs in the text it's formatting with the
582 appropriate number of spaces (using Text::Tabs to do its dirty work).
583 It normally assumes that each tab is equivalent to 8 space characters,
584 but you can change that default using the 'tabspace' option:
585 print autoformat { tabspace => 4 }, $text;
587
589 Text::Reform - provides functions for manual text wrapping and
590 reformatting.
591 Text::Aligner - provides a single function for justifying strings
593 according to various styles.
594 Text::Format - a class that provides methods for formatting text in
596 various ways.
597 Data::Formatter::Text - format various Perl data structures as text, in
599 different ways according to the type of data.
600
602 <https://github.com/neilb/Text-Autoformat>
603
605 Damian Conway (damian@conway.org)
606
608 There are undoubtedly serious bugs lurking somewhere in code this funky
609 :-) Bug reports and other feedback are most welcome.
610
612 Copyright (c) 1997-2007, Damian Conway "<DCONWAY@CPAN.org>". All rights
613 reserved.
614 This module is free software; you can redistribute it and/or modify it
616 under the same terms as Perl itself. See perlartistic.
617
619 BECAUSE THIS SOFTWARE IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY
620 FOR THE SOFTWARE, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT
621 WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER
622 PARTIES PROVIDE THE SOFTWARE "AS IS" WITHOUT WARRANTY OF ANY KIND,
623 EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
624 WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE
625 ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE SOFTWARE IS WITH
626 YOU. SHOULD THE SOFTWARE PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL
627 NECESSARY SERVICING, REPAIR, OR CORRECTION.
628 IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
630 WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR
631 REDISTRIBUTE THE SOFTWARE AS PERMITTED BY THE ABOVE LICENCE, BE LIABLE
632 TO YOU FOR DAMAGES, INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL, OR
633 CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OR INABILITY TO USE THE
634 SOFTWARE (INCLUDING BUT NOT LIMITED TO LOSS OF DATA OR DATA BEING
635 RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD PARTIES OR A
636 FAILURE OF THE SOFTWARE TO OPERATE WITH ANY OTHER SOFTWARE), EVEN IF
637 SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH
638 DAMAGES.
639perl v5.30.1 2020-01-30 Text::Autoformat(3)