1PERLFAQ4(1) Perl Programmers Reference Guide PERLFAQ4(1)
2
6 perlfaq4 - Data Manipulation ($Revision: 1.73 $, $Date: 2005/12/31
7 00:54:37 $)
8
10 This section of the FAQ answers questions related to manipulating num‐
11 bers, dates, strings, arrays, hashes, and miscellaneous data issues.
12
14 Why am I getting long decimals (eg, 19.9499999999999) instead of the
15 numbers I should be getting (eg, 19.95)?
16 Internally, your computer represents floating-point numbers in binary.
18 Digital (as in powers of two) computers cannot store all numbers
19 exactly. Some real numbers lose precision in the process. This is a
20 problem with how computers store numbers and affects all computer lan‐
21 guages, not just Perl.
22 perlnumber show the gory details of number representations and conver‐
24 sions.
25 To limit the number of decimal places in your numbers, you can use the
27 printf or sprintf function. See the "Floating Point Arithmetic" for
28 more details.
29 printf "%.2f", 10/3;
31 my $number = sprintf "%.2f", 10/3;
33 Why is int() broken?
35 Your int() is most probably working just fine. It's the numbers that
37 aren't quite what you think.
38 First, see the above item "Why am I getting long decimals (eg,
40 19.9499999999999) instead of the numbers I should be getting (eg,
41 19.95)?".
42 For example, this
44 print int(0.6/0.2-2), "\n";
46 will in most computers print 0, not 1, because even such simple numbers
48 as 0.6 and 0.2 cannot be presented exactly by floating-point numbers.
49 What you think in the above as 'three' is really more like
50 2.9999999999999995559.
51 Why isn't my octal data interpreted correctly?
53 Perl only understands octal and hex numbers as such when they occur as
55 literals in your program. Octal literals in perl must start with a
56 leading "0" and hexadecimal literals must start with a leading "0x".
57 If they are read in from somewhere and assigned, no automatic conver‐
58 sion takes place. You must explicitly use oct() or hex() if you want
59 the values converted to decimal. oct() interprets hex ("0x350"), octal
60 ("0350" or even without the leading "0", like "377") and binary
61 ("0b1010") numbers, while hex() only converts hexadecimal ones, with or
62 without a leading "0x", like "0x255", "3A", "ff", or "deadbeef". The
63 inverse mapping from decimal to octal can be done with either the "%o"
64 or "%O" sprintf() formats.
65 This problem shows up most often when people try using chmod(),
67 mkdir(), umask(), or sysopen(), which by widespread tradition typically
68 take permissions in octal.
69 chmod(644, $file); # WRONG
71 chmod(0644, $file); # right
72 Note the mistake in the first line was specifying the decimal literal
74 644, rather than the intended octal literal 0644. The problem can be
75 seen with:
76 printf("%#o",644); # prints 01204
78 Surely you had not intended "chmod(01204, $file);" - did you? If you
80 want to use numeric literals as arguments to chmod() et al. then please
81 try to express them as octal constants, that is with a leading zero and
82 with the following digits restricted to the set 0..7.
83 Does Perl have a round() function? What about ceil() and floor()?
85 Trig functions?
86 Remember that int() merely truncates toward 0. For rounding to a cer‐
88 tain number of digits, sprintf() or printf() is usually the easiest
89 route.
90 printf("%.3f", 3.1415926535); # prints 3.142
92 The POSIX module (part of the standard Perl distribution) implements
94 ceil(), floor(), and a number of other mathematical and trigonometric
95 functions.
96 use POSIX;
98 $ceil = ceil(3.5); # 4
99 $floor = floor(3.5); # 3
100 In 5.000 to 5.003 perls, trigonometry was done in the Math::Complex
102 module. With 5.004, the Math::Trig module (part of the standard Perl
103 distribution) implements the trigonometric functions. Internally it
104 uses the Math::Complex module and some functions can break out from the
105 real axis into the complex plane, for example the inverse sine of 2.
106 Rounding in financial applications can have serious implications, and
108 the rounding method used should be specified precisely. In these
109 cases, it probably pays not to trust whichever system rounding is being
110 used by Perl, but to instead implement the rounding function you need
111 yourself.
112 To see why, notice how you'll still have an issue on half-way-point
114 alternation:
115 for ($i = 0; $i < 1.01; $i += 0.05) { printf "%.1f ",$i}
117 0.0 0.1 0.1 0.2 0.2 0.2 0.3 0.3 0.4 0.4 0.5 0.5 0.6 0.7 0.7
119 0.8 0.8 0.9 0.9 1.0 1.0
120 Don't blame Perl. It's the same as in C. IEEE says we have to do
122 this. Perl numbers whose absolute values are integers under 2**31 (on
123 32 bit machines) will work pretty much like mathematical integers.
124 Other numbers are not guaranteed.
125 How do I convert between numeric representations/bases/radixes?
127 As always with Perl there is more than one way to do it. Below are a
129 few examples of approaches to making common conversions between number
130 representations. This is intended to be representational rather than
131 exhaustive.
132 Some of the examples below use the Bit::Vector module from CPAN. The
134 reason you might choose Bit::Vector over the perl built in functions is
135 that it works with numbers of ANY size, that it is optimized for speed
136 on some operations, and for at least some programmers the notation
137 might be familiar.
138 How do I convert hexadecimal into decimal
140 Using perl's built in conversion of 0x notation:
141 $dec = 0xDEADBEEF;
143 Using the hex function:
145 $dec = hex("DEADBEEF");
147 Using pack:
149 $dec = unpack("N", pack("H8", substr("0" x 8 . "DEADBEEF", -8)));
151 Using the CPAN module Bit::Vector:
153 use Bit::Vector;
155 $vec = Bit::Vector->new_Hex(32, "DEADBEEF");
156 $dec = $vec->to_Dec();
157 How do I convert from decimal to hexadecimal
159 Using sprintf:
160 $hex = sprintf("%X", 3735928559); # upper case A-F
162 $hex = sprintf("%x", 3735928559); # lower case a-f
163 Using unpack:
165 $hex = unpack("H*", pack("N", 3735928559));
167 Using Bit::Vector:
169 use Bit::Vector;
171 $vec = Bit::Vector->new_Dec(32, -559038737);
172 $hex = $vec->to_Hex();
173 And Bit::Vector supports odd bit counts:
175 use Bit::Vector;
177 $vec = Bit::Vector->new_Dec(33, 3735928559);
178 $vec->Resize(32); # suppress leading 0 if unwanted
179 $hex = $vec->to_Hex();
180 How do I convert from octal to decimal
182 Using Perl's built in conversion of numbers with leading zeros:
183 $dec = 033653337357; # note the leading 0!
185 Using the oct function:
187 $dec = oct("33653337357");
189 Using Bit::Vector:
191 use Bit::Vector;
193 $vec = Bit::Vector->new(32);
194 $vec->Chunk_List_Store(3, split(//, reverse "33653337357"));
195 $dec = $vec->to_Dec();
196 How do I convert from decimal to octal
198 Using sprintf:
199 $oct = sprintf("%o", 3735928559);
201 Using Bit::Vector:
203 use Bit::Vector;
205 $vec = Bit::Vector->new_Dec(32, -559038737);
206 $oct = reverse join('', $vec->Chunk_List_Read(3));
207 How do I convert from binary to decimal
209 Perl 5.6 lets you write binary numbers directly with the 0b nota‐
210 tion:
211 $number = 0b10110110;
213 Using oct:
215 my $input = "10110110";
217 $decimal = oct( "0b$input" );
218 Using pack and ord:
220 $decimal = ord(pack('B8', '10110110'));
222 Using pack and unpack for larger strings:
224 $int = unpack("N", pack("B32",
226 substr("0" x 32 . "11110101011011011111011101111", -32)));
227 $dec = sprintf("%d", $int);
228 # substr() is used to left pad a 32 character string with zeros.
230 Using Bit::Vector:
232 $vec = Bit::Vector->new_Bin(32, "11011110101011011011111011101111");
234 $dec = $vec->to_Dec();
235 How do I convert from decimal to binary
237 Using sprintf (perl 5.6+):
238 $bin = sprintf("%b", 3735928559);
240 Using unpack:
242 $bin = unpack("B*", pack("N", 3735928559));
244 Using Bit::Vector:
246 use Bit::Vector;
248 $vec = Bit::Vector->new_Dec(32, -559038737);
249 $bin = $vec->to_Bin();
250 The remaining transformations (e.g. hex -> oct, bin -> hex, etc.)
252 are left as an exercise to the inclined reader.
253 Why doesn't & work the way I want it to?
255 The behavior of binary arithmetic operators depends on whether they're
257 used on numbers or strings. The operators treat a string as a series
258 of bits and work with that (the string "3" is the bit pattern
259 00110011). The operators work with the binary form of a number (the
260 number 3 is treated as the bit pattern 00000011).
261 So, saying "11 & 3" performs the "and" operation on numbers (yielding
263 3). Saying "11" & "3" performs the "and" operation on strings (yield‐
264 ing "1").
265 Most problems with "&" and "⎪" arise because the programmer thinks they
267 have a number but really it's a string. The rest arise because the
268 programmer says:
269 if ("\020\020" & "\101\101") {
271 # ...
272 }
273 but a string consisting of two null bytes (the result of ""\020\020" &
275 "\101\101"") is not a false value in Perl. You need:
276 if ( ("\020\020" & "\101\101") !~ /[^\000]/) {
278 # ...
279 }
280 How do I multiply matrices?
282 Use the Math::Matrix or Math::MatrixReal modules (available from CPAN)
284 or the PDL extension (also available from CPAN).
285 How do I perform an operation on a series of integers?
287 To call a function on each element in an array, and collect the
289 results, use:
290 @results = map { my_func($_) } @array;
292 For example:
294 @triple = map { 3 * $_ } @single;
296 To call a function on each element of an array, but ignore the results:
298 foreach $iterator (@array) {
300 some_func($iterator);
301 }
302 To call a function on each integer in a (small) range, you can use:
304 @results = map { some_func($_) } (5 .. 25);
306 but you should be aware that the ".." operator creates an array of all
308 integers in the range. This can take a lot of memory for large ranges.
309 Instead use:
310 @results = ();
312 for ($i=5; $i < 500_005; $i++) {
313 push(@results, some_func($i));
314 }
315 This situation has been fixed in Perl5.005. Use of ".." in a "for" loop
317 will iterate over the range, without creating the entire range.
318 for my $i (5 .. 500_005) {
320 push(@results, some_func($i));
321 }
322 will not create a list of 500,000 integers.
324 How can I output Roman numerals?
326 Get the http://www.cpan.org/modules/by-module/Roman module.
328 Why aren't my random numbers random?
330 If you're using a version of Perl before 5.004, you must call "srand"
332 once at the start of your program to seed the random number generator.
333 BEGIN { srand() if $] < 5.004 }
335 5.004 and later automatically call "srand" at the beginning. Don't
337 call "srand" more than once---you make your numbers less random, rather
338 than more.
339 Computers are good at being predictable and bad at being random
341 (despite appearances caused by bugs in your programs :-). see the ran‐
342 dom article in the "Far More Than You Ever Wanted To Know" collection
343 in http://www.cpan.org/misc/olddoc/FMTEYEWTK.tgz , courtesy of Tom
344 Phoenix, talks more about this. John von Neumann said, "Anyone who
345 attempts to generate random numbers by deterministic means is, of
346 course, living in a state of sin."
347 If you want numbers that are more random than "rand" with "srand" pro‐
349 vides, you should also check out the Math::TrulyRandom module from
350 CPAN. It uses the imperfections in your system's timer to generate
351 random numbers, but this takes quite a while. If you want a better
352 pseudorandom generator than comes with your operating system, look at
353 "Numerical Recipes in C" at http://www.nr.com/ .
354 How do I get a random number between X and Y?
356 "rand($x)" returns a number such that "0 <= rand($x) < $x". Thus what
358 you want to have perl figure out is a random number in the range from 0
359 to the difference between your X and Y.
360 That is, to get a number between 10 and 15, inclusive, you want a ran‐
362 dom number between 0 and 5 that you can then add to 10.
363 my $number = 10 + int rand( 15-10+1 );
365 Hence you derive the following simple function to abstract that. It
367 selects a random integer between the two given integers (inclusive),
368 For example: "random_int_in(50,120)".
369 sub random_int_in ($$) {
371 my($min, $max) = @_;
372 # Assumes that the two arguments are integers themselves!
373 return $min if $min == $max;
374 ($min, $max) = ($max, $min) if $min > $max;
375 return $min + int rand(1 + $max - $min);
376 }
377
379 How do I find the day or week of the year?
380 The localtime function returns the day of the year. Without an argu‐
382 ment localtime uses the current time.
383 $day_of_year = (localtime)[7];
385 The POSIX module can also format a date as the day of the year or week
387 of the year.
388 use POSIX qw/strftime/;
390 my $day_of_year = strftime "%j", localtime;
391 my $week_of_year = strftime "%W", localtime;
392 To get the day of year for any date, use the Time::Local module to get
394 a time in epoch seconds for the argument to localtime.
395 use POSIX qw/strftime/;
397 use Time::Local;
398 my $week_of_year = strftime "%W",
399 localtime( timelocal( 0, 0, 0, 18, 11, 1987 ) );
400 The Date::Calc module provides two functions to calculate these.
402 use Date::Calc;
404 my $day_of_year = Day_of_Year( 1987, 12, 18 );
405 my $week_of_year = Week_of_Year( 1987, 12, 18 );
406 How do I find the current century or millennium?
408 Use the following simple functions:
410 sub get_century {
412 return int((((localtime(shift ⎪⎪ time))[5] + 1999))/100);
413 }
414 sub get_millennium {
416 return 1+int((((localtime(shift ⎪⎪ time))[5] + 1899))/1000);
417 }
418 On some systems, the POSIX module's strftime() function has been
420 extended in a non-standard way to use a %C format, which they sometimes
421 claim is the "century". It isn't, because on most such systems, this
422 is only the first two digits of the four-digit year, and thus cannot be
423 used to reliably determine the current century or millennium.
424 How can I compare two dates and find the difference?
426 (contributed by brian d foy)
428 You could just store all your dates as a number and then subtract. Life
430 isn't always that simple though. If you want to work with formatted
431 dates, the Date::Manip, Date::Calc, or DateTime modules can help you.
432 How can I take a string and turn it into epoch seconds?
434 If it's a regular enough string that it always has the same format, you
436 can split it up and pass the parts to "timelocal" in the standard
437 Time::Local module. Otherwise, you should look into the Date::Calc and
438 Date::Manip modules from CPAN.
439 How can I find the Julian Day?
441 (contributed by brian d foy and Dave Cross)
443 You can use the Time::JulianDay module available on CPAN. Ensure that
445 you really want to find a Julian day, though, as many people have dif‐
446 ferent ideas about Julian days. See http://www.her‐
447 metic.ch/cal_stud/jdn.htm for instance.
448 You can also try the DateTime module, which can convert a date/time to
450 a Julian Day.
451 $ perl -MDateTime -le'print DateTime->today->jd'
453 2453401.5
454 Or the modified Julian Day
456 $ perl -MDateTime -le'print DateTime->today->mjd'
458 53401
459 Or even the day of the year (which is what some people think of as a
461 Julian day)
462 $ perl -MDateTime -le'print DateTime->today->doy'
464 31
465 How do I find yesterday's date?
467 (contributed by brian d foy)
469 Use one of the Date modules. The "DateTime" module makes it simple, and
471 give you the same time of day, only the day before.
472 use DateTime;
474 my $yesterday = DateTime->now->subtract( days => 1 );
476 print "Yesterday was $yesterday\n";
478 You can also use the "Date::Calc" module using its Today_and_Now func‐
480 tion.
481 use Date::Calc qw( Today_and_Now Add_Delta_DHMS );
483 my @date_time = Add_Delta_DHMS( Today_and_Now(), -1, 0, 0, 0 );
485 print "@date\n";
487 Most people try to use the time rather than the calendar to figure out
489 dates, but that assumes that days are twenty-four hours each. For most
490 people, there are two days a year when they aren't: the switch to and
491 from summer time throws this off. Let the modules do the work.
492 Does Perl have a Year 2000 problem? Is Perl Y2K compliant?
494 Short answer: No, Perl does not have a Year 2000 problem. Yes, Perl is
496 Y2K compliant (whatever that means). The programmers you've hired to
497 use it, however, probably are not.
498 Long answer: The question belies a true understanding of the issue.
500 Perl is just as Y2K compliant as your pencil--no more, and no less.
501 Can you use your pencil to write a non-Y2K-compliant memo? Of course
502 you can. Is that the pencil's fault? Of course it isn't.
503 The date and time functions supplied with Perl (gmtime and localtime)
505 supply adequate information to determine the year well beyond 2000
506 (2038 is when trouble strikes for 32-bit machines). The year returned
507 by these functions when used in a list context is the year minus 1900.
508 For years between 1910 and 1999 this happens to be a 2-digit decimal
509 number. To avoid the year 2000 problem simply do not treat the year as
510 a 2-digit number. It isn't.
511 When gmtime() and localtime() are used in scalar context they return a
513 timestamp string that contains a fully-expanded year. For example,
514 "$timestamp = gmtime(1005613200)" sets $timestamp to "Tue Nov 13
515 01:00:00 2001". There's no year 2000 problem here.
516 That doesn't mean that Perl can't be used to create non-Y2K compliant
518 programs. It can. But so can your pencil. It's the fault of the
519 user, not the language. At the risk of inflaming the NRA: "Perl
520 doesn't break Y2K, people do." See http://www.perl.org/about/y2k.html
521 for a longer exposition.
522
524 How do I validate input?
525 (contributed by brian d foy)
527 There are many ways to ensure that values are what you expect or want
529 to accept. Besides the specific examples that we cover in the perlfaq,
530 you can also look at the modules with "Assert" and "Validate" in their
531 names, along with other modules such as "Regexp::Common".
532 Some modules have validation for particular types of input, such as
534 "Business::ISBN", "Business::CreditCard", "Email::Valid", and
535 "Data::Validate::IP".
536 How do I unescape a string?
538 It depends just what you mean by "escape". URL escapes are dealt with
540 in perlfaq9. Shell escapes with the backslash ("\") character are
541 removed with
542 s/\\(.)/$1/g;
544 This won't expand "\n" or "\t" or any other special escapes.
546 How do I remove consecutive pairs of characters?
548 (contributed by brian d foy)
550 You can use the substitution operator to find pairs of characters (or
552 runs of characters) and replace them with a single instance. In this
553 substitution, we find a character in "(.)". The memory parentheses
554 store the matched character in the back-reference "\1" and we use that
555 to require that the same thing immediately follow it. We replace that
556 part of the string with the character in $1.
557 s/(.)\1/$1/g;
559 We can also use the transliteration operator, "tr///". In this example,
561 the search list side of our "tr///" contains nothing, but the "c"
562 option complements that so it contains everything. The replacement list
563 also contains nothing, so the transliteration is almost a no-op since
564 it won't do any replacements (or more exactly, replace the character
565 with itself). However, the "s" option squashes duplicated and consecu‐
566 tive characters in the string so a character does not show up next to
567 itself
568 my $str = 'Haarlem'; # in the Netherlands
570 $str =~ tr///cs; # Now Harlem, like in New York
571 How do I expand function calls in a string?
573 (contributed by brian d foy)
575 This is documented in perlref, and although it's not the easiest thing
577 to read, it does work. In each of these examples, we call the function
578 inside the braces used to dereference a reference. If we have a more
579 than one return value, we can construct and dereference an anonymous
580 array. In this case, we call the function in list context.
581 print "The time values are @{ [localtime] }.\n";
583 If we want to call the function in scalar context, we have to do a bit
585 more work. We can really have any code we like inside the braces, so we
586 simply have to end with the scalar reference, although how you do that
587 is up to you, and you can use code inside the braces.
588 print "The time is ${\(scalar localtime)}.\n"
590 print "The time is ${ my $x = localtime; \$x }.\n";
592 If your function already returns a reference, you don't need to create
594 the reference yourself.
595 sub timestamp { my $t = localtime; \$t }
597 print "The time is ${ timestamp() }.\n";
599 The "Interpolation" module can also do a lot of magic for you. You can
601 specify a variable name, in this case "E", to set up a tied hash that
602 does the interpolation for you. It has several other methods to do this
603 as well.
604 use Interpolation E => 'eval';
606 print "The time values are $E{localtime()}.\n";
607 In most cases, it is probably easier to simply use string concatena‐
609 tion, which also forces scalar context.
610 print "The time is " . localtime . ".\n";
612 How do I find matching/nesting anything?
614 This isn't something that can be done in one regular expression, no
616 matter how complicated. To find something between two single charac‐
617 ters, a pattern like "/x([^x]*)x/" will get the intervening bits in $1.
618 For multiple ones, then something more like "/alpha(.*?)omega/" would
619 be needed. But none of these deals with nested patterns. For balanced
620 expressions using "(", "{", "[" or "<" as delimiters, use the CPAN mod‐
621 ule Regexp::Common, or see "(??{ code })" in perlre. For other cases,
622 you'll have to write a parser.
623 If you are serious about writing a parser, there are a number of mod‐
625 ules or oddities that will make your life a lot easier. There are the
626 CPAN modules Parse::RecDescent, Parse::Yapp, and Text::Balanced; and
627 the byacc program. Starting from perl 5.8 the Text::Balanced is part
628 of the standard distribution.
629 One simple destructive, inside-out approach that you might try is to
631 pull out the smallest nesting parts one at a time:
632 while (s/BEGIN((?:(?!BEGIN)(?!END).)*)END//gs) {
634 # do something with $1
635 }
636 A more complicated and sneaky approach is to make Perl's regular
638 expression engine do it for you. This is courtesy Dean Inada, and
639 rather has the nature of an Obfuscated Perl Contest entry, but it
640 really does work:
641 # $_ contains the string to parse
643 # BEGIN and END are the opening and closing markers for the
644 # nested text.
645 @( = ('(','');
647 @) = (')','');
648 ($re=$_)=~s/((BEGIN)⎪(END)⎪.)/$)[!$3]\Q$1\E$([!$2]/gs;
649 @$ = (eval{/$re/},$@!~/unmatched/i);
650 print join("\n",@$[0..$#$]) if( $$[-1] );
651 How do I reverse a string?
653 Use reverse() in scalar context, as documented in "reverse" in perl‐
655 func.
656 $reversed = reverse $string;
658 How do I expand tabs in a string?
660 You can do it yourself:
662 1 while $string =~ s/\t+/' ' x (length($&) * 8 - length($`) % 8)/e;
664 Or you can just use the Text::Tabs module (part of the standard Perl
666 distribution).
667 use Text::Tabs;
669 @expanded_lines = expand(@lines_with_tabs);
670 How do I reformat a paragraph?
672 Use Text::Wrap (part of the standard Perl distribution):
674 use Text::Wrap;
676 print wrap("\t", ' ', @paragraphs);
677 The paragraphs you give to Text::Wrap should not contain embedded new‐
679 lines. Text::Wrap doesn't justify the lines (flush-right).
680 Or use the CPAN module Text::Autoformat. Formatting files can be eas‐
682 ily done by making a shell alias, like so:
683 alias fmt="perl -i -MText::Autoformat -n0777 \
685 -e 'print autoformat $_, {all=>1}' $*"
686 See the documentation for Text::Autoformat to appreciate its many capa‐
688 bilities.
689 How can I access or change N characters of a string?
691 You can access the first characters of a string with substr(). To get
693 the first character, for example, start at position 0 and grab the
694 string of length 1.
695 $string = "Just another Perl Hacker";
697 $first_char = substr( $string, 0, 1 ); # 'J'
698 To change part of a string, you can use the optional fourth argument
700 which is the replacement string.
701 substr( $string, 13, 4, "Perl 5.8.0" );
703 You can also use substr() as an lvalue.
705 substr( $string, 13, 4 ) = "Perl 5.8.0";
707 How do I change the Nth occurrence of something?
709 You have to keep track of N yourself. For example, let's say you want
711 to change the fifth occurrence of "whoever" or "whomever" into "whoso‐
712 ever" or "whomsoever", case insensitively. These all assume that $_
713 contains the string to be altered.
714 $count = 0;
716 s{((whom?)ever)}{
717 ++$count == 5 # is it the 5th?
718 ? "${2}soever" # yes, swap
719 : $1 # renege and leave it there
720 }ige;
721 In the more general case, you can use the "/g" modifier in a "while"
723 loop, keeping count of matches.
724 $WANT = 3;
726 $count = 0;
727 $_ = "One fish two fish red fish blue fish";
728 while (/(\w+)\s+fish\b/gi) {
729 if (++$count == $WANT) {
730 print "The third fish is a $1 one.\n";
731 }
732 }
733 That prints out: "The third fish is a red one." You can also use a
735 repetition count and repeated pattern like this:
736 /(?:\w+\s+fish\s+){2}(\w+)\s+fish/i;
738 How can I count the number of occurrences of a substring within a
740 string?
741 There are a number of ways, with varying efficiency. If you want a
743 count of a certain single character (X) within a string, you can use
744 the "tr///" function like so:
745 $string = "ThisXlineXhasXsomeXx'sXinXit";
747 $count = ($string =~ tr/X//);
748 print "There are $count X characters in the string";
749 This is fine if you are just looking for a single character. However,
751 if you are trying to count multiple character substrings within a
752 larger string, "tr///" won't work. What you can do is wrap a while()
753 loop around a global pattern match. For example, let's count negative
754 integers:
755 $string = "-9 55 48 -2 23 -76 4 14 -44";
757 while ($string =~ /-\d+/g) { $count++ }
758 print "There are $count negative numbers in the string";
759 Another version uses a global match in list context, then assigns the
761 result to a scalar, producing a count of the number of matches.
762 $count = () = $string =~ /-\d+/g;
764 How do I capitalize all the words on one line?
766 To make the first letter of each word upper case:
768 $line =~ s/\b(\w)/\U$1/g;
770 This has the strange effect of turning ""don't do it"" into ""Don'T Do
772 It"". Sometimes you might want this. Other times you might need a
773 more thorough solution (Suggested by brian d foy):
774 $string =~ s/ (
776 (^\w) #at the beginning of the line
777 ⎪ # or
778 (\s\w) #preceded by whitespace
779 )
780 /\U$1/xg;
781 $string =~ /([\w']+)/\u\L$1/g;
782 To make the whole line upper case:
784 $line = uc($line);
786 To force each word to be lower case, with the first letter upper case:
788 $line =~ s/(\w+)/\u\L$1/g;
790 You can (and probably should) enable locale awareness of those charac‐
792 ters by placing a "use locale" pragma in your program. See perllocale
793 for endless details on locales.
794 This is sometimes referred to as putting something into "title case",
796 but that's not quite accurate. Consider the proper capitalization of
797 the movie Dr. Strangelove or: How I Learned to Stop Worrying and Love
798 the Bomb, for example.
799 Damian Conway's Text::Autoformat module provides some smart case trans‐
801 formations:
802 use Text::Autoformat;
804 my $x = "Dr. Strangelove or: How I Learned to Stop ".
805 "Worrying and Love the Bomb";
806 print $x, "\n";
808 for my $style (qw( sentence title highlight ))
809 {
810 print autoformat($x, { case => $style }), "\n";
811 }
812 How can I split a [character] delimited string except when inside
814 [character]?
815 Several modules can handle this sort of pasing---Text::Balanced,
817 Text::CSV, Text::CSV_XS, and Text::ParseWords, among others.
818 Take the example case of trying to split a string that is comma-sepa‐
820 rated into its different fields. You can't use "split(/,/)" because you
821 shouldn't split if the comma is inside quotes. For example, take a
822 data line like this:
823 SAR001,"","Cimetrix, Inc","Bob Smith","CAM",N,8,1,0,7,"Error, Core Dumped"
825 Due to the restriction of the quotes, this is a fairly complex problem.
827 Thankfully, we have Jeffrey Friedl, author of Mastering Regular Expres‐
828 sions, to handle these for us. He suggests (assuming your string is
829 contained in $text):
830 @new = ();
832 push(@new, $+) while $text =~ m{
833 "([^\"\\]*(?:\\.[^\"\\]*)*)",? # groups the phrase inside the quotes
834 ⎪ ([^,]+),?
835 ⎪ ,
836 }gx;
837 push(@new, undef) if substr($text,-1,1) eq ',';
838 If you want to represent quotation marks inside a quotation-mark-delim‐
840 ited field, escape them with backslashes (eg, "like \"this\"".
841 Alternatively, the Text::ParseWords module (part of the standard Perl
843 distribution) lets you say:
844 use Text::ParseWords;
846 @new = quotewords(",", 0, $text);
847 There's also a Text::CSV (Comma-Separated Values) module on CPAN.
849 How do I strip blank space from the beginning/end of a string?
851 (contributed by brian d foy)
853 A substitution can do this for you. For a single line, you want to
855 replace all the leading or trailing whitespace with nothing. You can do
856 that with a pair of substitutions.
857 s/^\s+//;
859 s/\s+$//;
860 You can also write that as a single substitution, although it turns out
862 the combined statement is slower than the separate ones. That might not
863 matter to you, though.
864 s/^\s+⎪\s+$//g;
866 In this regular expression, the alternation matches either at the
868 beginning or the end of the string since the anchors have a lower
869 precedence than the alternation. With the "/g" flag, the substitution
870 makes all possible matches, so it gets both. Remember, the trailing
871 newline matches the "\s+", and the "$" anchor can match to the physi‐
872 cal end of the string, so the newline disappears too. Just add the new‐
873 line to the output, which has the added benefit of preserving "blank"
874 (consisting entirely of whitespace) lines which the "^\s+" would remove
875 all by itself.
876 while( <> )
878 {
879 s/^\s+⎪\s+$//g;
880 print "$_\n";
881 }
882 For a multi-line string, you can apply the regular expression to each
884 logical line in the string by adding the "/m" flag (for "multi-line").
885 With the "/m" flag, the "$" matches before an embedded newline, so it
886 doesn't remove it. It still removes the newline at the end of the
887 string.
888 $string =~ s/^\s+⎪\s+$//gm;
890 Remember that lines consisting entirely of whitespace will disappear,
892 since the first part of the alternation can match the entire string and
893 replace it with nothing. If need to keep embedded blank lines, you have
894 to do a little more work. Instead of matching any whitespace (since
895 that includes a newline), just match the other whitespace.
896 $string =~ s/^[\t\f ]+⎪[\t\f ]+$//mg;
898 How do I pad a string with blanks or pad a number with zeroes?
900 In the following examples, $pad_len is the length to which you wish to
902 pad the string, $text or $num contains the string to be padded, and
903 $pad_char contains the padding character. You can use a single charac‐
904 ter string constant instead of the $pad_char variable if you know what
905 it is in advance. And in the same way you can use an integer in place
906 of $pad_len if you know the pad length in advance.
907 The simplest method uses the "sprintf" function. It can pad on the left
909 or right with blanks and on the left with zeroes and it will not trun‐
910 cate the result. The "pack" function can only pad strings on the right
911 with blanks and it will truncate the result to a maximum length of
912 $pad_len.
913 # Left padding a string with blanks (no truncation):
915 $padded = sprintf("%${pad_len}s", $text);
916 $padded = sprintf("%*s", $pad_len, $text); # same thing
917 # Right padding a string with blanks (no truncation):
919 $padded = sprintf("%-${pad_len}s", $text);
920 $padded = sprintf("%-*s", $pad_len, $text); # same thing
921 # Left padding a number with 0 (no truncation):
923 $padded = sprintf("%0${pad_len}d", $num);
924 $padded = sprintf("%0*d", $pad_len, $num); # same thing
925 # Right padding a string with blanks using pack (will truncate):
927 $padded = pack("A$pad_len",$text);
928 If you need to pad with a character other than blank or zero you can
930 use one of the following methods. They all generate a pad string with
931 the "x" operator and combine that with $text. These methods do not
932 truncate $text.
933 Left and right padding with any character, creating a new string:
935 $padded = $pad_char x ( $pad_len - length( $text ) ) . $text;
937 $padded = $text . $pad_char x ( $pad_len - length( $text ) );
938 Left and right padding with any character, modifying $text directly:
940 substr( $text, 0, 0 ) = $pad_char x ( $pad_len - length( $text ) );
942 $text .= $pad_char x ( $pad_len - length( $text ) );
943 How do I extract selected columns from a string?
945 Use substr() or unpack(), both documented in perlfunc. If you prefer
947 thinking in terms of columns instead of widths, you can use this kind
948 of thing:
949 # determine the unpack format needed to split Linux ps output
951 # arguments are cut columns
952 my $fmt = cut2fmt(8, 14, 20, 26, 30, 34, 41, 47, 59, 63, 67, 72);
953 sub cut2fmt {
955 my(@positions) = @_;
956 my $template = '';
957 my $lastpos = 1;
958 for my $place (@positions) {
959 $template .= "A" . ($place - $lastpos) . " ";
960 $lastpos = $place;
961 }
962 $template .= "A*";
963 return $template;
964 }
965 How do I find the soundex value of a string?
967 (contributed by brian d foy)
969 You can use the Text::Soundex module. If you want to do fuzzy or close
971 matching, you might also try the String::Approx, and Text::Metaphone,
972 and Text::DoubleMetaphone modules.
973 How can I expand variables in text strings?
975 Let's assume that you have a string that contains placeholder vari‐
977 ables.
978 $text = 'this has a $foo in it and a $bar';
980 You can use a substitution with a double evaluation. The first /e
982 turns $1 into $foo, and the second /e turns $foo into its value. You
983 may want to wrap this in an "eval": if you try to get the value of an
984 undeclared variable while running under "use strict", you get a fatal
985 error.
986 eval { $text =~ s/(\$\w+)/$1/eeg };
988 die if $@;
989 It's probably better in the general case to treat those variables as
991 entries in some special hash. For example:
992 %user_defs = (
994 foo => 23,
995 bar => 19,
996 );
997 $text =~ s/\$(\w+)/$user_defs{$1}/g;
998 What's wrong with always quoting "$vars"?
1000 The problem is that those double-quotes force stringification-- coerc‐
1002 ing numbers and references into strings--even when you don't want them
1003 to be strings. Think of it this way: double-quote expansion is used to
1004 produce new strings. If you already have a string, why do you need
1005 more?
1006 If you get used to writing odd things like these:
1008 print "$var"; # BAD
1010 $new = "$old"; # BAD
1011 somefunc("$var"); # BAD
1012 You'll be in trouble. Those should (in 99.8% of the cases) be the sim‐
1014 pler and more direct:
1015 print $var;
1017 $new = $old;
1018 somefunc($var);
1019 Otherwise, besides slowing you down, you're going to break code when
1021 the thing in the scalar is actually neither a string nor a number, but
1022 a reference:
1023 func(\@array);
1025 sub func {
1026 my $aref = shift;
1027 my $oref = "$aref"; # WRONG
1028 }
1029 You can also get into subtle problems on those few operations in Perl
1031 that actually do care about the difference between a string and a num‐
1032 ber, such as the magical "++" autoincrement operator or the syscall()
1033 function.
1034 Stringification also destroys arrays.
1036 @lines = `command`;
1038 print "@lines"; # WRONG - extra blanks
1039 print @lines; # right
1040 Why don't my <<HERE documents work?
1042 Check for these three things:
1044 There must be no space after the << part.
1046 There (probably) should be a semicolon at the end.
1047 You can't (easily) have any space in front of the tag.
1048 If you want to indent the text in the here document, you can do this:
1050 # all in one
1052 ($VAR = <<HERE_TARGET) =~ s/^\s+//gm;
1053 your text
1054 goes here
1055 HERE_TARGET
1056 But the HERE_TARGET must still be flush against the margin. If you
1058 want that indented also, you'll have to quote in the indentation.
1059 ($quote = <<' FINIS') =~ s/^\s+//gm;
1061 ...we will have peace, when you and all your works have
1062 perished--and the works of your dark master to whom you
1063 would deliver us. You are a liar, Saruman, and a corrupter
1064 of men's hearts. --Theoden in /usr/src/perl/taint.c
1065 FINIS
1066 $quote =~ s/\s+--/\n--/;
1067 A nice general-purpose fixer-upper function for indented here documents
1069 follows. It expects to be called with a here document as its argument.
1070 It looks to see whether each line begins with a common substring, and
1071 if so, strips that substring off. Otherwise, it takes the amount of
1072 leading whitespace found on the first line and removes that much off
1073 each subsequent line.
1074 sub fix {
1076 local $_ = shift;
1077 my ($white, $leader); # common whitespace and common leading string
1078 if (/^\s*(?:([^\w\s]+)(\s*).*\n)(?:\s*\1\2?.*\n)+$/) {
1079 ($white, $leader) = ($2, quotemeta($1));
1080 } else {
1081 ($white, $leader) = (/^(\s+)/, '');
1082 }
1083 s/^\s*?$leader(?:$white)?//gm;
1084 return $_;
1085 }
1086 This works with leading special strings, dynamically determined:
1088 $remember_the_main = fix<<' MAIN_INTERPRETER_LOOP';
1090 @@@ int
1091 @@@ runops() {
1092 @@@ SAVEI32(runlevel);
1093 @@@ runlevel++;
1094 @@@ while ( op = (*op->op_ppaddr)() );
1095 @@@ TAINT_NOT;
1096 @@@ return 0;
1097 @@@ }
1098 MAIN_INTERPRETER_LOOP
1099 Or with a fixed amount of leading whitespace, with remaining indenta‐
1101 tion correctly preserved:
1102 $poem = fix<<EVER_ON_AND_ON;
1104 Now far ahead the Road has gone,
1105 And I must follow, if I can,
1106 Pursuing it with eager feet,
1107 Until it joins some larger way
1108 Where many paths and errands meet.
1109 And whither then? I cannot say.
1110 --Bilbo in /usr/src/perl/pp_ctl.c
1111 EVER_ON_AND_ON
1112
1114 What is the difference between a list and an array?
1115 An array has a changeable length. A list does not. An array is some‐
1117 thing you can push or pop, while a list is a set of values. Some peo‐
1118 ple make the distinction that a list is a value while an array is a
1119 variable. Subroutines are passed and return lists, you put things into
1120 list context, you initialize arrays with lists, and you foreach()
1121 across a list. "@" variables are arrays, anonymous arrays are arrays,
1122 arrays in scalar context behave like the number of elements in them,
1123 subroutines access their arguments through the array @_, and
1124 push/pop/shift only work on arrays.
1125 As a side note, there's no such thing as a list in scalar context.
1127 When you say
1128 $scalar = (2, 5, 7, 9);
1130 you're using the comma operator in scalar context, so it uses the
1132 scalar comma operator. There never was a list there at all! This
1133 causes the last value to be returned: 9.
1134 What is the difference between $array[1] and @array[1]?
1136 The former is a scalar value; the latter an array slice, making it a
1138 list with one (scalar) value. You should use $ when you want a scalar
1139 value (most of the time) and @ when you want a list with one scalar
1140 value in it (very, very rarely; nearly never, in fact).
1141 Sometimes it doesn't make a difference, but sometimes it does. For
1143 example, compare:
1144 $good[0] = `some program that outputs several lines`;
1146 with
1148 @bad[0] = `same program that outputs several lines`;
1150 The "use warnings" pragma and the -w flag will warn you about these
1152 matters.
1153 How can I remove duplicate elements from a list or array?
1155 (contributed by brian d foy)
1157 Use a hash. When you think the words "unique" or "duplicated", think
1159 "hash keys".
1160 If you don't care about the order of the elements, you could just cre‐
1162 ate the hash then extract the keys. It's not important how you create
1163 that hash: just that you use "keys" to get the unique elements.
1164 my %hash = map { $_, 1 } @array;
1166 # or a hash slice: @hash{ @array } = ();
1167 # or a foreach: $hash{$_} = 1 foreach ( @array );
1168 my @unique = keys %hash;
1170 You can also go through each element and skip the ones you've seen
1172 before. Use a hash to keep track. The first time the loop sees an ele‐
1173 ment, that element has no key in %Seen. The "next" statement creates
1174 the key and immediately uses its value, which is "undef", so the loop
1175 continues to the "push" and increments the value for that key. The next
1176 time the loop sees that same element, its key exists in the hash and
1177 the value for that key is true (since it's not 0 or undef), so the next
1178 skips that iteration and the loop goes to the next element.
1179 my @unique = ();
1181 my %seen = ();
1182 foreach my $elem ( @array )
1184 {
1185 next if $seen{ $elem }++;
1186 push @unique, $elem;
1187 }
1188 You can write this more briefly using a grep, which does the same
1190 thing.
1191 my %seen = ();
1193 my @unique = grep { ! $seen{ $_ }++ } @array;
1194 How can I tell whether a certain element is contained in a list or
1196 array?
1197 (portions of this answer contributed by Anno Siegel)
1199 Hearing the word "in" is an indication that you probably should have
1201 used a hash, not a list or array, to store your data. Hashes are
1202 designed to answer this question quickly and efficiently. Arrays
1203 aren't.
1204 That being said, there are several ways to approach this. If you are
1206 going to make this query many times over arbitrary string values, the
1207 fastest way is probably to invert the original array and maintain a
1208 hash whose keys are the first array's values.
1209 @blues = qw/azure cerulean teal turquoise lapis-lazuli/;
1211 %is_blue = ();
1212 for (@blues) { $is_blue{$_} = 1 }
1213 Now you can check whether $is_blue{$some_color}. It might have been a
1215 good idea to keep the blues all in a hash in the first place.
1216 If the values are all small integers, you could use a simple indexed
1218 array. This kind of an array will take up less space:
1219 @primes = (2, 3, 5, 7, 11, 13, 17, 19, 23, 29, 31);
1221 @is_tiny_prime = ();
1222 for (@primes) { $is_tiny_prime[$_] = 1 }
1223 # or simply @istiny_prime[@primes] = (1) x @primes;
1224 Now you check whether $is_tiny_prime[$some_number].
1226 If the values in question are integers instead of strings, you can save
1228 quite a lot of space by using bit strings instead:
1229 @articles = ( 1..10, 150..2000, 2017 );
1231 undef $read;
1232 for (@articles) { vec($read,$_,1) = 1 }
1233 Now check whether "vec($read,$n,1)" is true for some $n.
1235 These methods guarantee fast individual tests but require a re-organi‐
1237 zation of the original list or array. They only pay off if you have to
1238 test multiple values against the same array.
1239 If you are testing only once, the standard module List::Util exports
1241 the function "first" for this purpose. It works by stopping once it
1242 finds the element. It's written in C for speed, and its Perl equivalant
1243 looks like this subroutine:
1244 sub first (&@) {
1246 my $code = shift;
1247 foreach (@_) {
1248 return $_ if &{$code}();
1249 }
1250 undef;
1251 }
1252 If speed is of little concern, the common idiom uses grep in scalar
1254 context (which returns the number of items that passed its condition)
1255 to traverse the entire list. This does have the benefit of telling you
1256 how many matches it found, though.
1257 my $is_there = grep $_ eq $whatever, @array;
1259 If you want to actually extract the matching elements, simply use grep
1261 in list context.
1262 my @matches = grep $_ eq $whatever, @array;
1264 How do I compute the difference of two arrays? How do I compute the
1266 intersection of two arrays?
1267 Use a hash. Here's code to do both and more. It assumes that each
1269 element is unique in a given array:
1270 @union = @intersection = @difference = ();
1272 %count = ();
1273 foreach $element (@array1, @array2) { $count{$element}++ }
1274 foreach $element (keys %count) {
1275 push @union, $element;
1276 push @{ $count{$element} > 1 ? \@intersection : \@difference }, $element;
1277 }
1278 Note that this is the symmetric difference, that is, all elements in
1280 either A or in B but not in both. Think of it as an xor operation.
1281 How do I test whether two arrays or hashes are equal?
1283 The following code works for single-level arrays. It uses a stringwise
1285 comparison, and does not distinguish defined versus undefined empty
1286 strings. Modify if you have other needs.
1287 $are_equal = compare_arrays(\@frogs, \@toads);
1289 sub compare_arrays {
1291 my ($first, $second) = @_;
1292 no warnings; # silence spurious -w undef complaints
1293 return 0 unless @$first == @$second;
1294 for (my $i = 0; $i < @$first; $i++) {
1295 return 0 if $first->[$i] ne $second->[$i];
1296 }
1297 return 1;
1298 }
1299 For multilevel structures, you may wish to use an approach more like
1301 this one. It uses the CPAN module FreezeThaw:
1302 use FreezeThaw qw(cmpStr);
1304 @a = @b = ( "this", "that", [ "more", "stuff" ] );
1305 printf "a and b contain %s arrays\n",
1307 cmpStr(\@a, \@b) == 0
1308 ? "the same"
1309 : "different";
1310 This approach also works for comparing hashes. Here we'll demonstrate
1312 two different answers:
1313 use FreezeThaw qw(cmpStr cmpStrHard);
1315 %a = %b = ( "this" => "that", "extra" => [ "more", "stuff" ] );
1317 $a{EXTRA} = \%b;
1318 $b{EXTRA} = \%a;
1319 printf "a and b contain %s hashes\n",
1321 cmpStr(\%a, \%b) == 0 ? "the same" : "different";
1322 printf "a and b contain %s hashes\n",
1324 cmpStrHard(\%a, \%b) == 0 ? "the same" : "different";
1325 The first reports that both those the hashes contain the same data,
1327 while the second reports that they do not. Which you prefer is left as
1328 an exercise to the reader.
1329 How do I find the first array element for which a condition is true?
1331 To find the first array element which satisfies a condition, you can
1333 use the first() function in the List::Util module, which comes with
1334 Perl 5.8. This example finds the first element that contains "Perl".
1335 use List::Util qw(first);
1337 my $element = first { /Perl/ } @array;
1339 If you cannot use List::Util, you can make your own loop to do the same
1341 thing. Once you find the element, you stop the loop with last.
1342 my $found;
1344 foreach ( @array )
1345 {
1346 if( /Perl/ ) { $found = $_; last }
1347 }
1348 If you want the array index, you can iterate through the indices and
1350 check the array element at each index until you find one that satisfies
1351 the condition.
1352 my( $found, $index ) = ( undef, -1 );
1354 for( $i = 0; $i < @array; $i++ )
1355 {
1356 if( $array[$i] =~ /Perl/ )
1357 {
1358 $found = $array[$i];
1359 $index = $i;
1360 last;
1361 }
1362 }
1363 How do I handle linked lists?
1365 In general, you usually don't need a linked list in Perl, since with
1367 regular arrays, you can push and pop or shift and unshift at either
1368 end, or you can use splice to add and/or remove arbitrary number of
1369 elements at arbitrary points. Both pop and shift are both O(1) opera‐
1370 tions on Perl's dynamic arrays. In the absence of shifts and pops,
1371 push in general needs to reallocate on the order every log(N) times,
1372 and unshift will need to copy pointers each time.
1373 If you really, really wanted, you could use structures as described in
1375 perldsc or perltoot and do just what the algorithm book tells you to
1376 do. For example, imagine a list node like this:
1377 $node = {
1379 VALUE => 42,
1380 LINK => undef,
1381 };
1382 You could walk the list this way:
1384 print "List: ";
1386 for ($node = $head; $node; $node = $node->{LINK}) {
1387 print $node->{VALUE}, " ";
1388 }
1389 print "\n";
1390 You could add to the list this way:
1392 my ($head, $tail);
1394 $tail = append($head, 1); # grow a new head
1395 for $value ( 2 .. 10 ) {
1396 $tail = append($tail, $value);
1397 }
1398 sub append {
1400 my($list, $value) = @_;
1401 my $node = { VALUE => $value };
1402 if ($list) {
1403 $node->{LINK} = $list->{LINK};
1404 $list->{LINK} = $node;
1405 } else {
1406 $_[0] = $node; # replace caller's version
1407 }
1408 return $node;
1409 }
1410 But again, Perl's built-in are virtually always good enough.
1412 How do I handle circular lists?
1414 Circular lists could be handled in the traditional fashion with linked
1416 lists, or you could just do something like this with an array:
1417 unshift(@array, pop(@array)); # the last shall be first
1419 push(@array, shift(@array)); # and vice versa
1420 How do I shuffle an array randomly?
1422 If you either have Perl 5.8.0 or later installed, or if you have
1424 Scalar-List-Utils 1.03 or later installed, you can say:
1425 use List::Util 'shuffle';
1427 @shuffled = shuffle(@list);
1429 If not, you can use a Fisher-Yates shuffle.
1431 sub fisher_yates_shuffle {
1433 my $deck = shift; # $deck is a reference to an array
1434 my $i = @$deck;
1435 while (--$i) {
1436 my $j = int rand ($i+1);
1437 @$deck[$i,$j] = @$deck[$j,$i];
1438 }
1439 }
1440 # shuffle my mpeg collection
1442 #
1443 my @mpeg = <audio/*/*.mp3>;
1444 fisher_yates_shuffle( \@mpeg ); # randomize @mpeg in place
1445 print @mpeg;
1446 Note that the above implementation shuffles an array in place, unlike
1448 the List::Util::shuffle() which takes a list and returns a new shuffled
1449 list.
1450 You've probably seen shuffling algorithms that work using splice, ran‐
1452 domly picking another element to swap the current element with
1453 srand;
1455 @new = ();
1456 @old = 1 .. 10; # just a demo
1457 while (@old) {
1458 push(@new, splice(@old, rand @old, 1));
1459 }
1460 This is bad because splice is already O(N), and since you do it N
1462 times, you just invented a quadratic algorithm; that is, O(N**2). This
1463 does not scale, although Perl is so efficient that you probably won't
1464 notice this until you have rather largish arrays.
1465 How do I process/modify each element of an array?
1467 Use "for"/"foreach":
1469 for (@lines) {
1471 s/foo/bar/; # change that word
1472 tr/XZ/ZX/; # swap those letters
1473 }
1474 Here's another; let's compute spherical volumes:
1476 for (@volumes = @radii) { # @volumes has changed parts
1478 $_ **= 3;
1479 $_ *= (4/3) * 3.14159; # this will be constant folded
1480 }
1481 which can also be done with map() which is made to transform one list
1483 into another:
1484 @volumes = map {$_ ** 3 * (4/3) * 3.14159} @radii;
1486 If you want to do the same thing to modify the values of the hash, you
1488 can use the "values" function. As of Perl 5.6 the values are not
1489 copied, so if you modify $orbit (in this case), you modify the value.
1490 for $orbit ( values %orbits ) {
1492 ($orbit **= 3) *= (4/3) * 3.14159;
1493 }
1494 Prior to perl 5.6 "values" returned copies of the values, so older perl
1496 code often contains constructions such as @orbits{keys %orbits} instead
1497 of "values %orbits" where the hash is to be modified.
1498 How do I select a random element from an array?
1500 Use the rand() function (see "rand" in perlfunc):
1502 $index = rand @array;
1504 $element = $array[$index];
1505 Or, simply:
1507 my $element = $array[ rand @array ];
1508 How do I permute N elements of a list?
1510 Use the List::Permutor module on CPAN. If the list is actually an
1512 array, try the Algorithm::Permute module (also on CPAN). It's written
1513 in XS code and is very efficient.
1514 use Algorithm::Permute;
1516 my @array = 'a'..'d';
1517 my $p_iterator = Algorithm::Permute->new ( \@array );
1518 while (my @perm = $p_iterator->next) {
1519 print "next permutation: (@perm)\n";
1520 }
1521 For even faster execution, you could do:
1523 use Algorithm::Permute;
1525 my @array = 'a'..'d';
1526 Algorithm::Permute::permute {
1527 print "next permutation: (@array)\n";
1528 } @array;
1529 Here's a little program that generates all permutations of all the
1531 words on each line of input. The algorithm embodied in the permute()
1532 function is discussed in Volume 4 (still unpublished) of Knuth's The
1533 Art of Computer Programming and will work on any list:
1534 #!/usr/bin/perl -n
1536 # Fischer-Kause ordered permutation generator
1537 sub permute (&@) {
1539 my $code = shift;
1540 my @idx = 0..$#_;
1541 while ( $code->(@_[@idx]) ) {
1542 my $p = $#idx;
1543 --$p while $idx[$p-1] > $idx[$p];
1544 my $q = $p or return;
1545 push @idx, reverse splice @idx, $p;
1546 ++$q while $idx[$p-1] > $idx[$q];
1547 @idx[$p-1,$q]=@idx[$q,$p-1];
1548 }
1549 }
1550 permute {print"@_\n"} split;
1552 How do I sort an array by (anything)?
1554 Supply a comparison function to sort() (described in "sort" in perl‐
1556 func):
1557 @list = sort { $a <=> $b } @list;
1559 The default sort function is cmp, string comparison, which would sort
1561 "(1, 2, 10)" into "(1, 10, 2)". "<=>", used above, is the numerical
1562 comparison operator.
1563 If you have a complicated function needed to pull out the part you want
1565 to sort on, then don't do it inside the sort function. Pull it out
1566 first, because the sort BLOCK can be called many times for the same
1567 element. Here's an example of how to pull out the first word after the
1568 first number on each item, and then sort those words case-insensi‐
1569 tively.
1570 @idx = ();
1572 for (@data) {
1573 ($item) = /\d+\s*(\S+)/;
1574 push @idx, uc($item);
1575 }
1576 @sorted = @data[ sort { $idx[$a] cmp $idx[$b] } 0 .. $#idx ];
1577 which could also be written this way, using a trick that's come to be
1579 known as the Schwartzian Transform:
1580 @sorted = map { $_->[0] }
1582 sort { $a->[1] cmp $b->[1] }
1583 map { [ $_, uc( (/\d+\s*(\S+)/)[0]) ] } @data;
1584 If you need to sort on several fields, the following paradigm is use‐
1586 ful.
1587 @sorted = sort { field1($a) <=> field1($b) ⎪⎪
1589 field2($a) cmp field2($b) ⎪⎪
1590 field3($a) cmp field3($b)
1591 } @data;
1592 This can be conveniently combined with precalculation of keys as given
1594 above.
1595 See the sort article in the "Far More Than You Ever Wanted To Know"
1597 collection in http://www.cpan.org/misc/olddoc/FMTEYEWTK.tgz for more
1598 about this approach.
1599 See also the question below on sorting hashes.
1601 How do I manipulate arrays of bits?
1603 Use pack() and unpack(), or else vec() and the bitwise operations.
1605 For example, this sets $vec to have bit N set if $ints[N] was set:
1607 $vec = '';
1609 foreach(@ints) { vec($vec,$_,1) = 1 }
1610 Here's how, given a vector in $vec, you can get those bits into your
1612 @ints array:
1613 sub bitvec_to_list {
1615 my $vec = shift;
1616 my @ints;
1617 # Find null-byte density then select best algorithm
1618 if ($vec =~ tr/\0// / length $vec > 0.95) {
1619 use integer;
1620 my $i;
1621 # This method is faster with mostly null-bytes
1622 while($vec =~ /[^\0]/g ) {
1623 $i = -9 + 8 * pos $vec;
1624 push @ints, $i if vec($vec, ++$i, 1);
1625 push @ints, $i if vec($vec, ++$i, 1);
1626 push @ints, $i if vec($vec, ++$i, 1);
1627 push @ints, $i if vec($vec, ++$i, 1);
1628 push @ints, $i if vec($vec, ++$i, 1);
1629 push @ints, $i if vec($vec, ++$i, 1);
1630 push @ints, $i if vec($vec, ++$i, 1);
1631 push @ints, $i if vec($vec, ++$i, 1);
1632 }
1633 } else {
1634 # This method is a fast general algorithm
1635 use integer;
1636 my $bits = unpack "b*", $vec;
1637 push @ints, 0 if $bits =~ s/^(\d)// && $1;
1638 push @ints, pos $bits while($bits =~ /1/g);
1639 }
1640 return \@ints;
1641 }
1642 This method gets faster the more sparse the bit vector is. (Courtesy
1644 of Tim Bunce and Winfried Koenig.)
1645 You can make the while loop a lot shorter with this suggestion from
1647 Benjamin Goldberg:
1648 while($vec =~ /[^\0]+/g ) {
1650 push @ints, grep vec($vec, $_, 1), $-[0] * 8 .. $+[0] * 8;
1651 }
1652 Or use the CPAN module Bit::Vector:
1654 $vector = Bit::Vector->new($num_of_bits);
1656 $vector->Index_List_Store(@ints);
1657 @ints = $vector->Index_List_Read();
1658 Bit::Vector provides efficient methods for bit vector, sets of small
1660 integers and "big int" math.
1661 Here's a more extensive illustration using vec():
1663 # vec demo
1665 $vector = "\xff\x0f\xef\xfe";
1666 print "Ilya's string \\xff\\x0f\\xef\\xfe represents the number ",
1667 unpack("N", $vector), "\n";
1668 $is_set = vec($vector, 23, 1);
1669 print "Its 23rd bit is ", $is_set ? "set" : "clear", ".\n";
1670 pvec($vector);
1671 set_vec(1,1,1);
1673 set_vec(3,1,1);
1674 set_vec(23,1,1);
1675 set_vec(3,1,3);
1677 set_vec(3,2,3);
1678 set_vec(3,4,3);
1679 set_vec(3,4,7);
1680 set_vec(3,8,3);
1681 set_vec(3,8,7);
1682 set_vec(0,32,17);
1684 set_vec(1,32,17);
1685 sub set_vec {
1687 my ($offset, $width, $value) = @_;
1688 my $vector = '';
1689 vec($vector, $offset, $width) = $value;
1690 print "offset=$offset width=$width value=$value\n";
1691 pvec($vector);
1692 }
1693 sub pvec {
1695 my $vector = shift;
1696 my $bits = unpack("b*", $vector);
1697 my $i = 0;
1698 my $BASE = 8;
1699 print "vector length in bytes: ", length($vector), "\n";
1701 @bytes = unpack("A8" x length($vector), $bits);
1702 print "bits are: @bytes\n\n";
1703 }
1704 Why does defined() return true on empty arrays and hashes?
1706 The short story is that you should probably only use defined on scalars
1708 or functions, not on aggregates (arrays and hashes). See "defined" in
1709 perlfunc in the 5.004 release or later of Perl for more detail.
1710
1712 How do I process an entire hash?
1713 Use the each() function (see "each" in perlfunc) if you don't care
1715 whether it's sorted:
1716 while ( ($key, $value) = each %hash) {
1718 print "$key = $value\n";
1719 }
1720 If you want it sorted, you'll have to use foreach() on the result of
1722 sorting the keys as shown in an earlier question.
1723 What happens if I add or remove keys from a hash while iterating over
1725 it?
1726 (contributed by brian d foy)
1728 The easy answer is "Don't do that!"
1730 If you iterate through the hash with each(), you can delete the key
1732 most recently returned without worrying about it. If you delete or add
1733 other keys, the iterator may skip or double up on them since perl may
1734 rearrange the hash table. See the entry for "each()" in perlfunc.
1735 How do I look up a hash element by value?
1737 Create a reverse hash:
1739 %by_value = reverse %by_key;
1741 $key = $by_value{$value};
1742 That's not particularly efficient. It would be more space-efficient to
1744 use:
1745 while (($key, $value) = each %by_key) {
1747 $by_value{$value} = $key;
1748 }
1749 If your hash could have repeated values, the methods above will only
1751 find one of the associated keys. This may or may not worry you. If
1752 it does worry you, you can always reverse the hash into a hash of
1753 arrays instead:
1754 while (($key, $value) = each %by_key) {
1756 push @{$key_list_by_value{$value}}, $key;
1757 }
1758 How can I know how many entries are in a hash?
1760 If you mean how many keys, then all you have to do is use the keys()
1762 function in a scalar context:
1763 $num_keys = keys %hash;
1765 The keys() function also resets the iterator, which means that you may
1767 see strange results if you use this between uses of other hash opera‐
1768 tors such as each().
1769 How do I sort a hash (optionally by value instead of key)?
1771 (contributed by brian d foy)
1773 To sort a hash, start with the keys. In this example, we give the list
1775 of keys to the sort function which then compares them ASCIIbetically
1776 (which might be affected by your locale settings). The output list has
1777 the keys in ASCIIbetical order. Once we have the keys, we can go
1778 through them to create a report which lists the keys in ASCIIbetical
1779 order.
1780 my @keys = sort { $a cmp $b } keys %hash;
1782 foreach my $key ( @keys )
1784 {
1785 printf "%-20s %6d\n", $key, $hash{$value};
1786 }
1787 We could get more fancy in the "sort()" block though. Instead of com‐
1789 paring the keys, we can compute a value with them and use that value as
1790 the comparison.
1791 For instance, to make our report order case-insensitive, we use the
1793 "\L" sequence in a double-quoted string to make everything lowercase.
1794 The "sort()" block then compares the lowercased values to determine in
1795 which order to put the keys.
1796 my @keys = sort { "\L$a" cmp "\L$b" } keys %hash;
1798 Note: if the computation is expensive or the hash has many elements,
1800 you may want to look at the Schwartzian Transform to cache the computa‐
1801 tion results.
1802 If we want to sort by the hash value instead, we use the hash key to
1804 look it up. We still get out a list of keys, but this time they are
1805 ordered by their value.
1806 my @keys = sort { $hash{$a} <=> $hash{$b} } keys %hash;
1808 From there we can get more complex. If the hash values are the same, we
1810 can provide a secondary sort on the hash key.
1811 my @keys = sort {
1813 $hash{$a} <=> $hash{$b}
1814 or
1815 "\L$a" cmp "\L$b"
1816 } keys %hash;
1817 How can I always keep my hash sorted?
1819 You can look into using the DB_File module and tie() using the
1821 $DB_BTREE hash bindings as documented in "In Memory Databases" in
1822 DB_File. The Tie::IxHash module from CPAN might also be instructive.
1823 What's the difference between "delete" and "undef" with hashes?
1825 Hashes contain pairs of scalars: the first is the key, the second is
1827 the value. The key will be coerced to a string, although the value can
1828 be any kind of scalar: string, number, or reference. If a key $key is
1829 present in %hash, "exists($hash{$key})" will return true. The value
1830 for a given key can be "undef", in which case $hash{$key} will be
1831 "undef" while "exists $hash{$key}" will return true. This corresponds
1832 to ($key, "undef") being in the hash.
1833 Pictures help... here's the %hash table:
1835 keys values
1837 +------+------+
1838 ⎪ a ⎪ 3 ⎪
1839 ⎪ x ⎪ 7 ⎪
1840 ⎪ d ⎪ 0 ⎪
1841 ⎪ e ⎪ 2 ⎪
1842 +------+------+
1843 And these conditions hold
1845 $hash{'a'} is true
1847 $hash{'d'} is false
1848 defined $hash{'d'} is true
1849 defined $hash{'a'} is true
1850 exists $hash{'a'} is true (Perl5 only)
1851 grep ($_ eq 'a', keys %hash) is true
1852 If you now say
1854 undef $hash{'a'}
1856 your table now reads:
1858 keys values
1860 +------+------+
1861 ⎪ a ⎪ undef⎪
1862 ⎪ x ⎪ 7 ⎪
1863 ⎪ d ⎪ 0 ⎪
1864 ⎪ e ⎪ 2 ⎪
1865 +------+------+
1866 and these conditions now hold; changes in caps:
1868 $hash{'a'} is FALSE
1870 $hash{'d'} is false
1871 defined $hash{'d'} is true
1872 defined $hash{'a'} is FALSE
1873 exists $hash{'a'} is true (Perl5 only)
1874 grep ($_ eq 'a', keys %hash) is true
1875 Notice the last two: you have an undef value, but a defined key!
1877 Now, consider this:
1879 delete $hash{'a'}
1881 your table now reads:
1883 keys values
1885 +------+------+
1886 ⎪ x ⎪ 7 ⎪
1887 ⎪ d ⎪ 0 ⎪
1888 ⎪ e ⎪ 2 ⎪
1889 +------+------+
1890 and these conditions now hold; changes in caps:
1892 $hash{'a'} is false
1894 $hash{'d'} is false
1895 defined $hash{'d'} is true
1896 defined $hash{'a'} is false
1897 exists $hash{'a'} is FALSE (Perl5 only)
1898 grep ($_ eq 'a', keys %hash) is FALSE
1899 See, the whole entry is gone!
1901 Why don't my tied hashes make the defined/exists distinction?
1903 This depends on the tied hash's implementation of EXISTS(). For exam‐
1905 ple, there isn't the concept of undef with hashes that are tied to DBM*
1906 files. It also means that exists() and defined() do the same thing with
1907 a DBM* file, and what they end up doing is not what they do with ordi‐
1908 nary hashes.
1909 How do I reset an each() operation part-way through?
1911 Using "keys %hash" in scalar context returns the number of keys in the
1913 hash and resets the iterator associated with the hash. You may need to
1914 do this if you use "last" to exit a loop early so that when you re-
1915 enter it, the hash iterator has been reset.
1916 How can I get the unique keys from two hashes?
1918 First you extract the keys from the hashes into lists, then solve the
1920 "removing duplicates" problem described above. For example:
1921 %seen = ();
1923 for $element (keys(%foo), keys(%bar)) {
1924 $seen{$element}++;
1925 }
1926 @uniq = keys %seen;
1927 Or more succinctly:
1929 @uniq = keys %{{%foo,%bar}};
1931 Or if you really want to save space:
1933 %seen = ();
1935 while (defined ($key = each %foo)) {
1936 $seen{$key}++;
1937 }
1938 while (defined ($key = each %bar)) {
1939 $seen{$key}++;
1940 }
1941 @uniq = keys %seen;
1942 How can I store a multidimensional array in a DBM file?
1944 Either stringify the structure yourself (no fun), or else get the MLDBM
1946 (which uses Data::Dumper) module from CPAN and layer it on top of
1947 either DB_File or GDBM_File.
1948 How can I make my hash remember the order I put elements into it?
1950 Use the Tie::IxHash from CPAN.
1952 use Tie::IxHash;
1954 tie my %myhash, 'Tie::IxHash';
1955 for (my $i=0; $i<20; $i++) {
1956 $myhash{$i} = 2*$i;
1957 }
1958 my @keys = keys %myhash;
1959 # @keys = (0,1,2,3,...)
1960 Why does passing a subroutine an undefined element in a hash create it?
1962 If you say something like:
1964 somefunc($hash{"nonesuch key here"});
1966 Then that element "autovivifies"; that is, it springs into existence
1968 whether you store something there or not. That's because functions get
1969 scalars passed in by reference. If somefunc() modifies $_[0], it has
1970 to be ready to write it back into the caller's version.
1971 This has been fixed as of Perl5.004.
1973 Normally, merely accessing a key's value for a nonexistent key does not
1975 cause that key to be forever there. This is different than awk's
1976 behavior.
1977 How can I make the Perl equivalent of a C structure/C++ class/hash or
1979 array of hashes or arrays?
1980 Usually a hash ref, perhaps like this:
1982 $record = {
1984 NAME => "Jason",
1985 EMPNO => 132,
1986 TITLE => "deputy peon",
1987 AGE => 23,
1988 SALARY => 37_000,
1989 PALS => [ "Norbert", "Rhys", "Phineas"],
1990 };
1991 References are documented in perlref and the upcoming perlreftut.
1993 Examples of complex data structures are given in perldsc and perllol.
1994 Examples of structures and object-oriented classes are in perltoot.
1995 How can I use a reference as a hash key?
1997 (contributed by brian d foy)
1999 Hash keys are strings, so you can't really use a reference as the key.
2001 When you try to do that, perl turns the reference into its stringified
2002 form (for instance, "HASH(0xDEADBEEF)"). From there you can't get back
2003 the reference from the stringified form, at least without doing some
2004 extra work on your own. Also remember that hash keys must be unique,
2005 but two different variables can store the same reference (and those
2006 variables can change later).
2007 The Tie::RefHash module, which is distributed with perl, might be what
2009 you want. It handles that extra work.
2010
2012 How do I handle binary data correctly?
2013 Perl is binary clean, so this shouldn't be a problem. For example,
2015 this works fine (assuming the files are found):
2016 if (`cat /vmunix` =~ /gzip/) {
2018 print "Your kernel is GNU-zip enabled!\n";
2019 }
2020 On less elegant (read: Byzantine) systems, however, you have to play
2022 tedious games with "text" versus "binary" files. See "binmode" in
2023 perlfunc or perlopentut.
2024 If you're concerned about 8-bit ASCII data, then see perllocale.
2026 If you want to deal with multibyte characters, however, there are some
2028 gotchas. See the section on Regular Expressions.
2029 How do I determine whether a scalar is a number/whole/integer/float?
2031 Assuming that you don't care about IEEE notations like "NaN" or "Infin‐
2033 ity", you probably just want to use a regular expression.
2034 if (/\D/) { print "has nondigits\n" }
2036 if (/^\d+$/) { print "is a whole number\n" }
2037 if (/^-?\d+$/) { print "is an integer\n" }
2038 if (/^[+-]?\d+$/) { print "is a +/- integer\n" }
2039 if (/^-?\d+\.?\d*$/) { print "is a real number\n" }
2040 if (/^-?(?:\d+(?:\.\d*)?⎪\.\d+)$/) { print "is a decimal number\n" }
2041 if (/^([+-]?)(?=\d⎪\.\d)\d*(\.\d*)?([Ee]([+-]?\d+))?$/)
2042 { print "a C float\n" }
2043 There are also some commonly used modules for the task. Scalar::Util
2045 (distributed with 5.8) provides access to perl's internal function
2046 "looks_like_number" for determining whether a variable looks like a
2047 number. Data::Types exports functions that validate data types using
2048 both the above and other regular expressions. Thirdly, there is "Reg‐
2049 exp::Common" which has regular expressions to match various types of
2050 numbers. Those three modules are available from the CPAN.
2051 If you're on a POSIX system, Perl supports the "POSIX::strtod" func‐
2053 tion. Its semantics are somewhat cumbersome, so here's a "getnum"
2054 wrapper function for more convenient access. This function takes a
2055 string and returns the number it found, or "undef" for input that isn't
2056 a C float. The "is_numeric" function is a front end to "getnum" if you
2057 just want to say, "Is this a float?"
2058 sub getnum {
2060 use POSIX qw(strtod);
2061 my $str = shift;
2062 $str =~ s/^\s+//;
2063 $str =~ s/\s+$//;
2064 $! = 0;
2065 my($num, $unparsed) = strtod($str);
2066 if (($str eq '') ⎪⎪ ($unparsed != 0) ⎪⎪ $!) {
2067 return undef;
2068 } else {
2069 return $num;
2070 }
2071 }
2072 sub is_numeric { defined getnum($_[0]) }
2074 Or you could check out the String::Scanf module on the CPAN instead.
2076 The POSIX module (part of the standard Perl distribution) provides the
2077 "strtod" and "strtol" for converting strings to double and longs,
2078 respectively.
2079 How do I keep persistent data across program calls?
2081 For some specific applications, you can use one of the DBM modules.
2083 See AnyDBM_File. More generically, you should consult the FreezeThaw
2084 or Storable modules from CPAN. Starting from Perl 5.8 Storable is part
2085 of the standard distribution. Here's one example using Storable's
2086 "store" and "retrieve" functions:
2087 use Storable;
2089 store(\%hash, "filename");
2090 # later on...
2092 $href = retrieve("filename"); # by ref
2093 %hash = %{ retrieve("filename") }; # direct to hash
2094 How do I print out or copy a recursive data structure?
2096 The Data::Dumper module on CPAN (or the 5.005 release of Perl) is great
2098 for printing out data structures. The Storable module on CPAN (or the
2099 5.8 release of Perl), provides a function called "dclone" that recur‐
2100 sively copies its argument.
2101 use Storable qw(dclone);
2103 $r2 = dclone($r1);
2104 Where $r1 can be a reference to any kind of data structure you'd like.
2106 It will be deeply copied. Because "dclone" takes and returns refer‐
2107 ences, you'd have to add extra punctuation if you had a hash of arrays
2108 that you wanted to copy.
2109 %newhash = %{ dclone(\%oldhash) };
2111 How do I define methods for every class/object?
2113 Use the UNIVERSAL class (see UNIVERSAL).
2115 How do I verify a credit card checksum?
2117 Get the Business::CreditCard module from CPAN.
2119 How do I pack arrays of doubles or floats for XS code?
2121 The kgbpack.c code in the PGPLOT module on CPAN does just this. If
2123 you're doing a lot of float or double processing, consider using the
2124 PDL module from CPAN instead--it makes number-crunching easy.
2125
2127 Copyright (c) 1997-2006 Tom Christiansen, Nathan Torkington, and other
2128 authors as noted. All rights reserved.
2129 This documentation is free; you can redistribute it and/or modify it
2131 under the same terms as Perl itself.
2132 Irrespective of its distribution, all code examples in this file are
2134 hereby placed into the public domain. You are permitted and encouraged
2135 to use this code in your own programs for fun or for profit as you see
2136 fit. A simple comment in the code giving credit would be courteous but
2137 is not required.
2138perl v5.8.8 2006-01-07 PERLFAQ4(1)