1perlfaq4(3) User Contributed Perl Documentation perlfaq4(3)
2
6 perlfaq4 - Data Manipulation
7
9 version 5.20180605
10
12 This section of the FAQ answers questions related to manipulating
13 numbers, dates, strings, arrays, hashes, and miscellaneous data issues.
14
16 Why am I getting long decimals (eg, 19.9499999999999) instead of the
17 numbers I should be getting (eg, 19.95)?
18 For the long explanation, see David Goldberg's "What Every Computer
19 Scientist Should Know About Floating-Point Arithmetic"
20 (<http://web.cse.msu.edu/~cse320/Documents/FloatingPoint.pdf>).
21 Internally, your computer represents floating-point numbers in binary.
23 Digital (as in powers of two) computers cannot store all numbers
24 exactly. Some real numbers lose precision in the process. This is a
25 problem with how computers store numbers and affects all computer
26 languages, not just Perl.
27 perlnumber shows the gory details of number representations and
29 conversions.
30 To limit the number of decimal places in your numbers, you can use the
32 "printf" or "sprintf" function. See "Floating-point Arithmetic" in
33 perlop for more details.
34 printf "%.2f", 10/3;
36 my $number = sprintf "%.2f", 10/3;
38 Why is int() broken?
40 Your "int()" is most probably working just fine. It's the numbers that
41 aren't quite what you think.
42 First, see the answer to "Why am I getting long decimals (eg,
44 19.9499999999999) instead of the numbers I should be getting (eg,
45 19.95)?".
46 For example, this
48 print int(0.6/0.2-2), "\n";
50 will in most computers print 0, not 1, because even such simple numbers
52 as 0.6 and 0.2 cannot be presented exactly by floating-point numbers.
53 What you think in the above as 'three' is really more like
54 2.9999999999999995559.
55 Why isn't my octal data interpreted correctly?
57 (contributed by brian d foy)
58 You're probably trying to convert a string to a number, which Perl only
60 converts as a decimal number. When Perl converts a string to a number,
61 it ignores leading spaces and zeroes, then assumes the rest of the
62 digits are in base 10:
63 my $string = '0644';
65 print $string + 0; # prints 644
67 print $string + 44; # prints 688, certainly not octal!
69 This problem usually involves one of the Perl built-ins that has the
71 same name a Unix command that uses octal numbers as arguments on the
72 command line. In this example, "chmod" on the command line knows that
73 its first argument is octal because that's what it does:
74 %prompt> chmod 644 file
76 If you want to use the same literal digits (644) in Perl, you have to
78 tell Perl to treat them as octal numbers either by prefixing the digits
79 with a 0 or using "oct":
80 chmod( 0644, $filename ); # right, has leading zero
82 chmod( oct(644), $filename ); # also correct
83 The problem comes in when you take your numbers from something that
85 Perl thinks is a string, such as a command line argument in @ARGV:
86 chmod( $ARGV[0], $filename ); # wrong, even if "0644"
88 chmod( oct($ARGV[0]), $filename ); # correct, treat string as octal
90 You can always check the value you're using by printing it in octal
92 notation to ensure it matches what you think it should be. Print it in
93 octal and decimal format:
94 printf "0%o %d", $number, $number;
96 Does Perl have a round() function? What about ceil() and floor()? Trig
98 functions?
99 Remember that "int()" merely truncates toward 0. For rounding to a
100 certain number of digits, "sprintf()" or "printf()" is usually the
101 easiest route.
102 printf("%.3f", 3.1415926535); # prints 3.142
104 The POSIX module (part of the standard Perl distribution) implements
106 "ceil()", "floor()", and a number of other mathematical and
107 trigonometric functions.
108 use POSIX;
110 my $ceil = ceil(3.5); # 4
111 my $floor = floor(3.5); # 3
112 In 5.000 to 5.003 perls, trigonometry was done in the Math::Complex
114 module. With 5.004, the Math::Trig module (part of the standard Perl
115 distribution) implements the trigonometric functions. Internally it
116 uses the Math::Complex module and some functions can break out from the
117 real axis into the complex plane, for example the inverse sine of 2.
118 Rounding in financial applications can have serious implications, and
120 the rounding method used should be specified precisely. In these cases,
121 it probably pays not to trust whichever system of rounding is being
122 used by Perl, but instead to implement the rounding function you need
123 yourself.
124 To see why, notice how you'll still have an issue on half-way-point
126 alternation:
127 for (my $i = -5; $i <= 5; $i += 0.5) { printf "%.0f ",$i }
129 -5 -4 -4 -4 -3 -2 -2 -2 -1 -0 0 0 1 2 2 2 3 4 4 4 5
131 Don't blame Perl. It's the same as in C. IEEE says we have to do this.
133 Perl numbers whose absolute values are integers under 2**31 (on 32-bit
134 machines) will work pretty much like mathematical integers. Other
135 numbers are not guaranteed.
136 How do I convert between numeric representations/bases/radixes?
138 As always with Perl there is more than one way to do it. Below are a
139 few examples of approaches to making common conversions between number
140 representations. This is intended to be representational rather than
141 exhaustive.
142 Some of the examples later in perlfaq4 use the Bit::Vector module from
144 CPAN. The reason you might choose Bit::Vector over the perl built-in
145 functions is that it works with numbers of ANY size, that it is
146 optimized for speed on some operations, and for at least some
147 programmers the notation might be familiar.
148 How do I convert hexadecimal into decimal
150 Using perl's built in conversion of "0x" notation:
151 my $dec = 0xDEADBEEF;
153 Using the "hex" function:
155 my $dec = hex("DEADBEEF");
157 Using "pack":
159 my $dec = unpack("N", pack("H8", substr("0" x 8 . "DEADBEEF", -8)));
161 Using the CPAN module "Bit::Vector":
163 use Bit::Vector;
165 my $vec = Bit::Vector->new_Hex(32, "DEADBEEF");
166 my $dec = $vec->to_Dec();
167 How do I convert from decimal to hexadecimal
169 Using "sprintf":
170 my $hex = sprintf("%X", 3735928559); # upper case A-F
172 my $hex = sprintf("%x", 3735928559); # lower case a-f
173 Using "unpack":
175 my $hex = unpack("H*", pack("N", 3735928559));
177 Using Bit::Vector:
179 use Bit::Vector;
181 my $vec = Bit::Vector->new_Dec(32, -559038737);
182 my $hex = $vec->to_Hex();
183 And Bit::Vector supports odd bit counts:
185 use Bit::Vector;
187 my $vec = Bit::Vector->new_Dec(33, 3735928559);
188 $vec->Resize(32); # suppress leading 0 if unwanted
189 my $hex = $vec->to_Hex();
190 How do I convert from octal to decimal
192 Using Perl's built in conversion of numbers with leading zeros:
193 my $dec = 033653337357; # note the leading 0!
195 Using the "oct" function:
197 my $dec = oct("33653337357");
199 Using Bit::Vector:
201 use Bit::Vector;
203 my $vec = Bit::Vector->new(32);
204 $vec->Chunk_List_Store(3, split(//, reverse "33653337357"));
205 my $dec = $vec->to_Dec();
206 How do I convert from decimal to octal
208 Using "sprintf":
209 my $oct = sprintf("%o", 3735928559);
211 Using Bit::Vector:
213 use Bit::Vector;
215 my $vec = Bit::Vector->new_Dec(32, -559038737);
216 my $oct = reverse join('', $vec->Chunk_List_Read(3));
217 How do I convert from binary to decimal
219 Perl 5.6 lets you write binary numbers directly with the "0b"
220 notation:
221 my $number = 0b10110110;
223 Using "oct":
225 my $input = "10110110";
227 my $decimal = oct( "0b$input" );
228 Using "pack" and "ord":
230 my $decimal = ord(pack('B8', '10110110'));
232 Using "pack" and "unpack" for larger strings:
234 my $int = unpack("N", pack("B32",
236 substr("0" x 32 . "11110101011011011111011101111", -32)));
237 my $dec = sprintf("%d", $int);
238 # substr() is used to left-pad a 32-character string with zeros.
240 Using Bit::Vector:
242 my $vec = Bit::Vector->new_Bin(32, "11011110101011011011111011101111");
244 my $dec = $vec->to_Dec();
245 How do I convert from decimal to binary
247 Using "sprintf" (perl 5.6+):
248 my $bin = sprintf("%b", 3735928559);
250 Using "unpack":
252 my $bin = unpack("B*", pack("N", 3735928559));
254 Using Bit::Vector:
256 use Bit::Vector;
258 my $vec = Bit::Vector->new_Dec(32, -559038737);
259 my $bin = $vec->to_Bin();
260 The remaining transformations (e.g. hex -> oct, bin -> hex, etc.)
262 are left as an exercise to the inclined reader.
263 Why doesn't & work the way I want it to?
265 The behavior of binary arithmetic operators depends on whether they're
266 used on numbers or strings. The operators treat a string as a series of
267 bits and work with that (the string "3" is the bit pattern 00110011).
268 The operators work with the binary form of a number (the number 3 is
269 treated as the bit pattern 00000011).
270 So, saying "11 & 3" performs the "and" operation on numbers (yielding
272 3). Saying "11" & "3" performs the "and" operation on strings (yielding
273 "1").
274 Most problems with "&" and "|" arise because the programmer thinks they
276 have a number but really it's a string or vice versa. To avoid this,
277 stringify the arguments explicitly (using "" or "qq()") or convert them
278 to numbers explicitly (using "0+$arg"). The rest arise because the
279 programmer says:
280 if ("\020\020" & "\101\101") {
282 # ...
283 }
284 but a string consisting of two null bytes (the result of "\020\020" &
286 "\101\101") is not a false value in Perl. You need:
287 if ( ("\020\020" & "\101\101") !~ /[^\000]/) {
289 # ...
290 }
291 How do I multiply matrices?
293 Use the Math::Matrix or Math::MatrixReal modules (available from CPAN)
294 or the PDL extension (also available from CPAN).
295 How do I perform an operation on a series of integers?
297 To call a function on each element in an array, and collect the
298 results, use:
299 my @results = map { my_func($_) } @array;
301 For example:
303 my @triple = map { 3 * $_ } @single;
305 To call a function on each element of an array, but ignore the results:
307 foreach my $iterator (@array) {
309 some_func($iterator);
310 }
311 To call a function on each integer in a (small) range, you can use:
313 my @results = map { some_func($_) } (5 .. 25);
315 but you should be aware that in this form, the ".." operator creates a
317 list of all integers in the range, which can take a lot of memory for
318 large ranges. However, the problem does not occur when using ".."
319 within a "for" loop, because in that case the range operator is
320 optimized to iterate over the range, without creating the entire list.
321 So
322 my @results = ();
324 for my $i (5 .. 500_005) {
325 push(@results, some_func($i));
326 }
327 or even
329 push(@results, some_func($_)) for 5 .. 500_005;
331 will not create an intermediate list of 500,000 integers.
333 How can I output Roman numerals?
335 Get the <http://www.cpan.org/modules/by-module/Roman> module.
336 Why aren't my random numbers random?
338 If you're using a version of Perl before 5.004, you must call "srand"
339 once at the start of your program to seed the random number generator.
340 BEGIN { srand() if $] < 5.004 }
342 5.004 and later automatically call "srand" at the beginning. Don't call
344 "srand" more than once--you make your numbers less random, rather than
345 more.
346 Computers are good at being predictable and bad at being random
348 (despite appearances caused by bugs in your programs :-). The random
349 article in the "Far More Than You Ever Wanted To Know" collection in
350 <http://www.cpan.org/misc/olddoc/FMTEYEWTK.tgz>, courtesy of Tom
351 Phoenix, talks more about this. John von Neumann said, "Anyone who
352 attempts to generate random numbers by deterministic means is, of
353 course, living in a state of sin."
354 Perl relies on the underlying system for the implementation of "rand"
356 and "srand"; on some systems, the generated numbers are not random
357 enough (especially on Windows : see
358 <http://www.perlmonks.org/?node_id=803632>). Several CPAN modules in
359 the "Math" namespace implement better pseudorandom generators; see for
360 example Math::Random::MT ("Mersenne Twister", fast), or
361 Math::TrulyRandom (uses the imperfections in the system's timer to
362 generate random numbers, which is rather slow). More algorithms for
363 random numbers are described in "Numerical Recipes in C" at
364 <http://www.nr.com/>
365 How do I get a random number between X and Y?
367 To get a random number between two values, you can use the "rand()"
368 built-in to get a random number between 0 and 1. From there, you shift
369 that into the range that you want.
370 "rand($x)" returns a number such that "0 <= rand($x) < $x". Thus what
372 you want to have perl figure out is a random number in the range from 0
373 to the difference between your X and Y.
374 That is, to get a number between 10 and 15, inclusive, you want a
376 random number between 0 and 5 that you can then add to 10.
377 my $number = 10 + int rand( 15-10+1 ); # ( 10,11,12,13,14, or 15 )
379 Hence you derive the following simple function to abstract that. It
381 selects a random integer between the two given integers (inclusive).
382 For example: "random_int_between(50,120)".
383 sub random_int_between {
385 my($min, $max) = @_;
386 # Assumes that the two arguments are integers themselves!
387 return $min if $min == $max;
388 ($min, $max) = ($max, $min) if $min > $max;
389 return $min + int rand(1 + $max - $min);
390 }
391
393 How do I find the day or week of the year?
394 The day of the year is in the list returned by the "localtime"
395 function. Without an argument "localtime" uses the current time.
396 my $day_of_year = (localtime)[7];
398 The POSIX module can also format a date as the day of the year or week
400 of the year.
401 use POSIX qw/strftime/;
403 my $day_of_year = strftime "%j", localtime;
404 my $week_of_year = strftime "%W", localtime;
405 To get the day of year for any date, use POSIX's "mktime" to get a time
407 in epoch seconds for the argument to "localtime".
408 use POSIX qw/mktime strftime/;
410 my $week_of_year = strftime "%W",
411 localtime( mktime( 0, 0, 0, 18, 11, 87 ) );
412 You can also use Time::Piece, which comes with Perl and provides a
414 "localtime" that returns an object:
415 use Time::Piece;
417 my $day_of_year = localtime->yday;
418 my $week_of_year = localtime->week;
419 The Date::Calc module provides two functions to calculate these, too:
421 use Date::Calc;
423 my $day_of_year = Day_of_Year( 1987, 12, 18 );
424 my $week_of_year = Week_of_Year( 1987, 12, 18 );
425 How do I find the current century or millennium?
427 Use the following simple functions:
428 sub get_century {
430 return int((((localtime(shift || time))[5] + 1999))/100);
431 }
432 sub get_millennium {
434 return 1+int((((localtime(shift || time))[5] + 1899))/1000);
435 }
436 On some systems, the POSIX module's "strftime()" function has been
438 extended in a non-standard way to use a %C format, which they sometimes
439 claim is the "century". It isn't, because on most such systems, this is
440 only the first two digits of the four-digit year, and thus cannot be
441 used to determine reliably the current century or millennium.
442 How can I compare two dates and find the difference?
444 (contributed by brian d foy)
445 You could just store all your dates as a number and then subtract.
447 Life isn't always that simple though.
448 The Time::Piece module, which comes with Perl, replaces localtime with
450 a version that returns an object. It also overloads the comparison
451 operators so you can compare them directly:
452 use Time::Piece;
454 my $date1 = localtime( $some_time );
455 my $date2 = localtime( $some_other_time );
456 if( $date1 < $date2 ) {
458 print "The date was in the past\n";
459 }
460 You can also get differences with a subtraction, which returns a
462 Time::Seconds object:
463 my $date_diff = $date1 - $date2;
465 print "The difference is ", $date_diff->days, " days\n";
466 If you want to work with formatted dates, the Date::Manip, Date::Calc,
468 or DateTime modules can help you.
469 How can I take a string and turn it into epoch seconds?
471 If it's a regular enough string that it always has the same format, you
472 can split it up and pass the parts to "timelocal" in the standard
473 Time::Local module. Otherwise, you should look into the Date::Calc,
474 Date::Parse, and Date::Manip modules from CPAN.
475 How can I find the Julian Day?
477 (contributed by brian d foy and Dave Cross)
478 You can use the Time::Piece module, part of the Standard Library, which
480 can convert a date/time to a Julian Day:
481 $ perl -MTime::Piece -le 'print localtime->julian_day'
483 2455607.7959375
484 Or the modified Julian Day:
486 $ perl -MTime::Piece -le 'print localtime->mjd'
488 55607.2961226851
489 Or even the day of the year (which is what some people think of as a
491 Julian day):
492 $ perl -MTime::Piece -le 'print localtime->yday'
494 45
495 You can also do the same things with the DateTime module:
497 $ perl -MDateTime -le'print DateTime->today->jd'
499 2453401.5
500 $ perl -MDateTime -le'print DateTime->today->mjd'
501 53401
502 $ perl -MDateTime -le'print DateTime->today->doy'
503 31
504 You can use the Time::JulianDay module available on CPAN. Ensure that
506 you really want to find a Julian day, though, as many people have
507 different ideas about Julian days (see
508 <http://www.hermetic.ch/cal_stud/jdn.htm> for instance):
509 $ perl -MTime::JulianDay -le 'print local_julian_day( time )'
511 55608
512 How do I find yesterday's date?
514 (contributed by brian d foy)
515 To do it correctly, you can use one of the "Date" modules since they
517 work with calendars instead of times. The DateTime module makes it
518 simple, and give you the same time of day, only the day before, despite
519 daylight saving time changes:
520 use DateTime;
522 my $yesterday = DateTime->now->subtract( days => 1 );
524 print "Yesterday was $yesterday\n";
526 You can also use the Date::Calc module using its "Today_and_Now"
528 function.
529 use Date::Calc qw( Today_and_Now Add_Delta_DHMS );
531 my @date_time = Add_Delta_DHMS( Today_and_Now(), -1, 0, 0, 0 );
533 print "@date_time\n";
535 Most people try to use the time rather than the calendar to figure out
537 dates, but that assumes that days are twenty-four hours each. For most
538 people, there are two days a year when they aren't: the switch to and
539 from summer time throws this off. For example, the rest of the
540 suggestions will be wrong sometimes:
541 Starting with Perl 5.10, Time::Piece and Time::Seconds are part of the
543 standard distribution, so you might think that you could do something
544 like this:
545 use Time::Piece;
547 use Time::Seconds;
548 my $yesterday = localtime() - ONE_DAY; # WRONG
550 print "Yesterday was $yesterday\n";
551 The Time::Piece module exports a new "localtime" that returns an
553 object, and Time::Seconds exports the "ONE_DAY" constant that is a set
554 number of seconds. This means that it always gives the time 24 hours
555 ago, which is not always yesterday. This can cause problems around the
556 end of daylight saving time when there's one day that is 25 hours long.
557 You have the same problem with Time::Local, which will give the wrong
559 answer for those same special cases:
560 # contributed by Gunnar Hjalmarsson
562 use Time::Local;
563 my $today = timelocal 0, 0, 12, ( localtime )[3..5];
564 my ($d, $m, $y) = ( localtime $today-86400 )[3..5]; # WRONG
565 printf "Yesterday: %d-%02d-%02d\n", $y+1900, $m+1, $d;
566 Does Perl have a Year 2000 or 2038 problem? Is Perl Y2K compliant?
568 (contributed by brian d foy)
569 Perl itself never had a Y2K problem, although that never stopped people
571 from creating Y2K problems on their own. See the documentation for
572 "localtime" for its proper use.
573 Starting with Perl 5.12, "localtime" and "gmtime" can handle dates past
575 03:14:08 January 19, 2038, when a 32-bit based time would overflow. You
576 still might get a warning on a 32-bit "perl":
577 % perl5.12 -E 'say scalar localtime( 0x9FFF_FFFFFFFF )'
579 Integer overflow in hexadecimal number at -e line 1.
580 Wed Nov 1 19:42:39 5576711
581 On a 64-bit "perl", you can get even larger dates for those really long
583 running projects:
584 % perl5.12 -E 'say scalar gmtime( 0x9FFF_FFFFFFFF )'
586 Thu Nov 2 00:42:39 5576711
587 You're still out of luck if you need to keep track of decaying protons
589 though.
590
592 How do I validate input?
593 (contributed by brian d foy)
594 There are many ways to ensure that values are what you expect or want
596 to accept. Besides the specific examples that we cover in the perlfaq,
597 you can also look at the modules with "Assert" and "Validate" in their
598 names, along with other modules such as Regexp::Common.
599 Some modules have validation for particular types of input, such as
601 Business::ISBN, Business::CreditCard, Email::Valid, and
602 Data::Validate::IP.
603 How do I unescape a string?
605 It depends just what you mean by "escape". URL escapes are dealt with
606 in perlfaq9. Shell escapes with the backslash ("\") character are
607 removed with
608 s/\\(.)/$1/g;
610 This won't expand "\n" or "\t" or any other special escapes.
612 How do I remove consecutive pairs of characters?
614 (contributed by brian d foy)
615 You can use the substitution operator to find pairs of characters (or
617 runs of characters) and replace them with a single instance. In this
618 substitution, we find a character in "(.)". The memory parentheses
619 store the matched character in the back-reference "\g1" and we use that
620 to require that the same thing immediately follow it. We replace that
621 part of the string with the character in $1.
622 s/(.)\g1/$1/g;
624 We can also use the transliteration operator, "tr///". In this example,
626 the search list side of our "tr///" contains nothing, but the "c"
627 option complements that so it contains everything. The replacement list
628 also contains nothing, so the transliteration is almost a no-op since
629 it won't do any replacements (or more exactly, replace the character
630 with itself). However, the "s" option squashes duplicated and
631 consecutive characters in the string so a character does not show up
632 next to itself
633 my $str = 'Haarlem'; # in the Netherlands
635 $str =~ tr///cs; # Now Harlem, like in New York
636 How do I expand function calls in a string?
638 (contributed by brian d foy)
639 This is documented in perlref, and although it's not the easiest thing
641 to read, it does work. In each of these examples, we call the function
642 inside the braces used to dereference a reference. If we have more than
643 one return value, we can construct and dereference an anonymous array.
644 In this case, we call the function in list context.
645 print "The time values are @{ [localtime] }.\n";
647 If we want to call the function in scalar context, we have to do a bit
649 more work. We can really have any code we like inside the braces, so we
650 simply have to end with the scalar reference, although how you do that
651 is up to you, and you can use code inside the braces. Note that the use
652 of parens creates a list context, so we need "scalar" to force the
653 scalar context on the function:
654 print "The time is ${\(scalar localtime)}.\n"
656 print "The time is ${ my $x = localtime; \$x }.\n";
658 If your function already returns a reference, you don't need to create
660 the reference yourself.
661 sub timestamp { my $t = localtime; \$t }
663 print "The time is ${ timestamp() }.\n";
665 The "Interpolation" module can also do a lot of magic for you. You can
667 specify a variable name, in this case "E", to set up a tied hash that
668 does the interpolation for you. It has several other methods to do this
669 as well.
670 use Interpolation E => 'eval';
672 print "The time values are $E{localtime()}.\n";
673 In most cases, it is probably easier to simply use string
675 concatenation, which also forces scalar context.
676 print "The time is " . localtime() . ".\n";
678 How do I find matching/nesting anything?
680 To find something between two single characters, a pattern like
681 "/x([^x]*)x/" will get the intervening bits in $1. For multiple ones,
682 then something more like "/alpha(.*?)omega/" would be needed. For
683 nested patterns and/or balanced expressions, see the so-called (?PARNO)
684 construct (available since perl 5.10). The CPAN module Regexp::Common
685 can help to build such regular expressions (see in particular
686 Regexp::Common::balanced and Regexp::Common::delimited).
687 More complex cases will require to write a parser, probably using a
689 parsing module from CPAN, like Regexp::Grammars, Parse::RecDescent,
690 Parse::Yapp, Text::Balanced, or Marpa::R2.
691 How do I reverse a string?
693 Use "reverse()" in scalar context, as documented in "reverse" in
694 perlfunc.
695 my $reversed = reverse $string;
697 How do I expand tabs in a string?
699 You can do it yourself:
700 1 while $string =~ s/\t+/' ' x (length($&) * 8 - length($`) % 8)/e;
702 Or you can just use the Text::Tabs module (part of the standard Perl
704 distribution).
705 use Text::Tabs;
707 my @expanded_lines = expand(@lines_with_tabs);
708 How do I reformat a paragraph?
710 Use Text::Wrap (part of the standard Perl distribution):
711 use Text::Wrap;
713 print wrap("\t", ' ', @paragraphs);
714 The paragraphs you give to Text::Wrap should not contain embedded
716 newlines. Text::Wrap doesn't justify the lines (flush-right).
717 Or use the CPAN module Text::Autoformat. Formatting files can be easily
719 done by making a shell alias, like so:
720 alias fmt="perl -i -MText::Autoformat -n0777 \
722 -e 'print autoformat $_, {all=>1}' $*"
723 See the documentation for Text::Autoformat to appreciate its many
725 capabilities.
726 How can I access or change N characters of a string?
728 You can access the first characters of a string with substr(). To get
729 the first character, for example, start at position 0 and grab the
730 string of length 1.
731 my $string = "Just another Perl Hacker";
733 my $first_char = substr( $string, 0, 1 ); # 'J'
734 To change part of a string, you can use the optional fourth argument
736 which is the replacement string.
737 substr( $string, 13, 4, "Perl 5.8.0" );
739 You can also use substr() as an lvalue.
741 substr( $string, 13, 4 ) = "Perl 5.8.0";
743 How do I change the Nth occurrence of something?
745 You have to keep track of N yourself. For example, let's say you want
746 to change the fifth occurrence of "whoever" or "whomever" into
747 "whosoever" or "whomsoever", case insensitively. These all assume that
748 $_ contains the string to be altered.
749 $count = 0;
751 s{((whom?)ever)}{
752 ++$count == 5 # is it the 5th?
753 ? "${2}soever" # yes, swap
754 : $1 # renege and leave it there
755 }ige;
756 In the more general case, you can use the "/g" modifier in a "while"
758 loop, keeping count of matches.
759 $WANT = 3;
761 $count = 0;
762 $_ = "One fish two fish red fish blue fish";
763 while (/(\w+)\s+fish\b/gi) {
764 if (++$count == $WANT) {
765 print "The third fish is a $1 one.\n";
766 }
767 }
768 That prints out: "The third fish is a red one." You can also use a
770 repetition count and repeated pattern like this:
771 /(?:\w+\s+fish\s+){2}(\w+)\s+fish/i;
773 How can I count the number of occurrences of a substring within a string?
775 There are a number of ways, with varying efficiency. If you want a
776 count of a certain single character (X) within a string, you can use
777 the "tr///" function like so:
778 my $string = "ThisXlineXhasXsomeXx'sXinXit";
780 my $count = ($string =~ tr/X//);
781 print "There are $count X characters in the string";
782 This is fine if you are just looking for a single character. However,
784 if you are trying to count multiple character substrings within a
785 larger string, "tr///" won't work. What you can do is wrap a while()
786 loop around a global pattern match. For example, let's count negative
787 integers:
788 my $string = "-9 55 48 -2 23 -76 4 14 -44";
790 my $count = 0;
791 while ($string =~ /-\d+/g) { $count++ }
792 print "There are $count negative numbers in the string";
793 Another version uses a global match in list context, then assigns the
795 result to a scalar, producing a count of the number of matches.
796 my $count = () = $string =~ /-\d+/g;
798 How do I capitalize all the words on one line?
800 (contributed by brian d foy)
801 Damian Conway's Text::Autoformat handles all of the thinking for you.
803 use Text::Autoformat;
805 my $x = "Dr. Strangelove or: How I Learned to Stop ".
806 "Worrying and Love the Bomb";
807 print $x, "\n";
809 for my $style (qw( sentence title highlight )) {
810 print autoformat($x, { case => $style }), "\n";
811 }
812 How do you want to capitalize those words?
814 FRED AND BARNEY'S LODGE # all uppercase
816 Fred And Barney's Lodge # title case
817 Fred and Barney's Lodge # highlight case
818 It's not as easy a problem as it looks. How many words do you think are
820 in there? Wait for it... wait for it.... If you answered 5 you're
821 right. Perl words are groups of "\w+", but that's not what you want to
822 capitalize. How is Perl supposed to know not to capitalize that "s"
823 after the apostrophe? You could try a regular expression:
824 $string =~ s/ (
826 (^\w) #at the beginning of the line
827 | # or
828 (\s\w) #preceded by whitespace
829 )
830 /\U$1/xg;
831 $string =~ s/([\w']+)/\u\L$1/g;
833 Now, what if you don't want to capitalize that "and"? Just use
835 Text::Autoformat and get on with the next problem. :)
836 How can I split a [character]-delimited string except when inside
838 [character]?
839 Several modules can handle this sort of parsing--Text::Balanced,
840 Text::CSV, Text::CSV_XS, and Text::ParseWords, among others.
841 Take the example case of trying to split a string that is comma-
843 separated into its different fields. You can't use "split(/,/)" because
844 you shouldn't split if the comma is inside quotes. For example, take a
845 data line like this:
846 SAR001,"","Cimetrix, Inc","Bob Smith","CAM",N,8,1,0,7,"Error, Core Dumped"
848 Due to the restriction of the quotes, this is a fairly complex problem.
850 Thankfully, we have Jeffrey Friedl, author of Mastering Regular
851 Expressions, to handle these for us. He suggests (assuming your string
852 is contained in $text):
853 my @new = ();
855 push(@new, $+) while $text =~ m{
856 "([^\"\\]*(?:\\.[^\"\\]*)*)",? # groups the phrase inside the quotes
857 | ([^,]+),?
858 | ,
859 }gx;
860 push(@new, undef) if substr($text,-1,1) eq ',';
861 If you want to represent quotation marks inside a quotation-mark-
863 delimited field, escape them with backslashes (eg, "like \"this\"".
864 Alternatively, the Text::ParseWords module (part of the standard Perl
866 distribution) lets you say:
867 use Text::ParseWords;
869 @new = quotewords(",", 0, $text);
870 For parsing or generating CSV, though, using Text::CSV rather than
872 implementing it yourself is highly recommended; you'll save yourself
873 odd bugs popping up later by just using code which has already been
874 tried and tested in production for years.
875 How do I strip blank space from the beginning/end of a string?
877 (contributed by brian d foy)
878 A substitution can do this for you. For a single line, you want to
880 replace all the leading or trailing whitespace with nothing. You can do
881 that with a pair of substitutions:
882 s/^\s+//;
884 s/\s+$//;
885 You can also write that as a single substitution, although it turns out
887 the combined statement is slower than the separate ones. That might not
888 matter to you, though:
889 s/^\s+|\s+$//g;
891 In this regular expression, the alternation matches either at the
893 beginning or the end of the string since the anchors have a lower
894 precedence than the alternation. With the "/g" flag, the substitution
895 makes all possible matches, so it gets both. Remember, the trailing
896 newline matches the "\s+", and the "$" anchor can match to the
897 absolute end of the string, so the newline disappears too. Just add the
898 newline to the output, which has the added benefit of preserving
899 "blank" (consisting entirely of whitespace) lines which the "^\s+"
900 would remove all by itself:
901 while( <> ) {
903 s/^\s+|\s+$//g;
904 print "$_\n";
905 }
906 For a multi-line string, you can apply the regular expression to each
908 logical line in the string by adding the "/m" flag (for "multi-line").
909 With the "/m" flag, the "$" matches before an embedded newline, so it
910 doesn't remove it. This pattern still removes the newline at the end of
911 the string:
912 $string =~ s/^\s+|\s+$//gm;
914 Remember that lines consisting entirely of whitespace will disappear,
916 since the first part of the alternation can match the entire string and
917 replace it with nothing. If you need to keep embedded blank lines, you
918 have to do a little more work. Instead of matching any whitespace
919 (since that includes a newline), just match the other whitespace:
920 $string =~ s/^[\t\f ]+|[\t\f ]+$//mg;
922 How do I pad a string with blanks or pad a number with zeroes?
924 In the following examples, $pad_len is the length to which you wish to
925 pad the string, $text or $num contains the string to be padded, and
926 $pad_char contains the padding character. You can use a single
927 character string constant instead of the $pad_char variable if you know
928 what it is in advance. And in the same way you can use an integer in
929 place of $pad_len if you know the pad length in advance.
930 The simplest method uses the "sprintf" function. It can pad on the left
932 or right with blanks and on the left with zeroes and it will not
933 truncate the result. The "pack" function can only pad strings on the
934 right with blanks and it will truncate the result to a maximum length
935 of $pad_len.
936 # Left padding a string with blanks (no truncation):
938 my $padded = sprintf("%${pad_len}s", $text);
939 my $padded = sprintf("%*s", $pad_len, $text); # same thing
940 # Right padding a string with blanks (no truncation):
942 my $padded = sprintf("%-${pad_len}s", $text);
943 my $padded = sprintf("%-*s", $pad_len, $text); # same thing
944 # Left padding a number with 0 (no truncation):
946 my $padded = sprintf("%0${pad_len}d", $num);
947 my $padded = sprintf("%0*d", $pad_len, $num); # same thing
948 # Right padding a string with blanks using pack (will truncate):
950 my $padded = pack("A$pad_len",$text);
951 If you need to pad with a character other than blank or zero you can
953 use one of the following methods. They all generate a pad string with
954 the "x" operator and combine that with $text. These methods do not
955 truncate $text.
956 Left and right padding with any character, creating a new string:
958 my $padded = $pad_char x ( $pad_len - length( $text ) ) . $text;
960 my $padded = $text . $pad_char x ( $pad_len - length( $text ) );
961 Left and right padding with any character, modifying $text directly:
963 substr( $text, 0, 0 ) = $pad_char x ( $pad_len - length( $text ) );
965 $text .= $pad_char x ( $pad_len - length( $text ) );
966 How do I extract selected columns from a string?
968 (contributed by brian d foy)
969 If you know the columns that contain the data, you can use "substr" to
971 extract a single column.
972 my $column = substr( $line, $start_column, $length );
974 You can use "split" if the columns are separated by whitespace or some
976 other delimiter, as long as whitespace or the delimiter cannot appear
977 as part of the data.
978 my $line = ' fred barney betty ';
980 my @columns = split /\s+/, $line;
981 # ( '', 'fred', 'barney', 'betty' );
982 my $line = 'fred||barney||betty';
984 my @columns = split /\|/, $line;
985 # ( 'fred', '', 'barney', '', 'betty' );
986 If you want to work with comma-separated values, don't do this since
988 that format is a bit more complicated. Use one of the modules that
989 handle that format, such as Text::CSV, Text::CSV_XS, or Text::CSV_PP.
990 If you want to break apart an entire line of fixed columns, you can use
992 "unpack" with the A (ASCII) format. By using a number after the format
993 specifier, you can denote the column width. See the "pack" and "unpack"
994 entries in perlfunc for more details.
995 my @fields = unpack( $line, "A8 A8 A8 A16 A4" );
997 Note that spaces in the format argument to "unpack" do not denote
999 literal spaces. If you have space separated data, you may want "split"
1000 instead.
1001 How do I find the soundex value of a string?
1003 (contributed by brian d foy)
1004 You can use the "Text::Soundex" module. If you want to do fuzzy or
1006 close matching, you might also try the String::Approx, and
1007 Text::Metaphone, and Text::DoubleMetaphone modules.
1008 How can I expand variables in text strings?
1010 (contributed by brian d foy)
1011 If you can avoid it, don't, or if you can use a templating system, such
1013 as Text::Template or Template Toolkit, do that instead. You might even
1014 be able to get the job done with "sprintf" or "printf":
1015 my $string = sprintf 'Say hello to %s and %s', $foo, $bar;
1017 However, for the one-off simple case where I don't want to pull out a
1019 full templating system, I'll use a string that has two Perl scalar
1020 variables in it. In this example, I want to expand $foo and $bar to
1021 their variable's values:
1022 my $foo = 'Fred';
1024 my $bar = 'Barney';
1025 $string = 'Say hello to $foo and $bar';
1026 One way I can do this involves the substitution operator and a double
1028 "/e" flag. The first "/e" evaluates $1 on the replacement side and
1029 turns it into $foo. The second /e starts with $foo and replaces it with
1030 its value. $foo, then, turns into 'Fred', and that's finally what's
1031 left in the string:
1032 $string =~ s/(\$\w+)/$1/eeg; # 'Say hello to Fred and Barney'
1034 The "/e" will also silently ignore violations of strict, replacing
1036 undefined variable names with the empty string. Since I'm using the
1037 "/e" flag (twice even!), I have all of the same security problems I
1038 have with "eval" in its string form. If there's something odd in $foo,
1039 perhaps something like "@{[ system "rm -rf /" ]}", then I could get
1040 myself in trouble.
1041 To get around the security problem, I could also pull the values from a
1043 hash instead of evaluating variable names. Using a single "/e", I can
1044 check the hash to ensure the value exists, and if it doesn't, I can
1045 replace the missing value with a marker, in this case "???" to signal
1046 that I missed something:
1047 my $string = 'This has $foo and $bar';
1049 my %Replacements = (
1051 foo => 'Fred',
1052 );
1053 # $string =~ s/\$(\w+)/$Replacements{$1}/g;
1055 $string =~ s/\$(\w+)/
1056 exists $Replacements{$1} ? $Replacements{$1} : '???'
1057 /eg;
1058 print $string;
1060 What's wrong with always quoting "$vars"?
1062 The problem is that those double-quotes force stringification--coercing
1063 numbers and references into strings--even when you don't want them to
1064 be strings. Think of it this way: double-quote expansion is used to
1065 produce new strings. If you already have a string, why do you need
1066 more?
1067 If you get used to writing odd things like these:
1069 print "$var"; # BAD
1071 my $new = "$old"; # BAD
1072 somefunc("$var"); # BAD
1073 You'll be in trouble. Those should (in 99.8% of the cases) be the
1075 simpler and more direct:
1076 print $var;
1078 my $new = $old;
1079 somefunc($var);
1080 Otherwise, besides slowing you down, you're going to break code when
1082 the thing in the scalar is actually neither a string nor a number, but
1083 a reference:
1084 func(\@array);
1086 sub func {
1087 my $aref = shift;
1088 my $oref = "$aref"; # WRONG
1089 }
1090 You can also get into subtle problems on those few operations in Perl
1092 that actually do care about the difference between a string and a
1093 number, such as the magical "++" autoincrement operator or the
1094 syscall() function.
1095 Stringification also destroys arrays.
1097 my @lines = `command`;
1099 print "@lines"; # WRONG - extra blanks
1100 print @lines; # right
1101 Why don't my <<HERE documents work?
1103 Here documents are found in perlop. Check for these three things:
1104 There must be no space after the << part.
1106 There (probably) should be a semicolon at the end of the opening token
1107 You can't (easily) have any space in front of the tag.
1108 There needs to be at least a line separator after the end token.
1109 If you want to indent the text in the here document, you can do this:
1111 # all in one
1113 (my $VAR = <<HERE_TARGET) =~ s/^\s+//gm;
1114 your text
1115 goes here
1116 HERE_TARGET
1117 But the HERE_TARGET must still be flush against the margin. If you
1119 want that indented also, you'll have to quote in the indentation.
1120 (my $quote = <<' FINIS') =~ s/^\s+//gm;
1122 ...we will have peace, when you and all your works have
1123 perished--and the works of your dark master to whom you
1124 would deliver us. You are a liar, Saruman, and a corrupter
1125 of men's hearts. --Theoden in /usr/src/perl/taint.c
1126 FINIS
1127 $quote =~ s/\s+--/\n--/;
1128 A nice general-purpose fixer-upper function for indented here documents
1130 follows. It expects to be called with a here document as its argument.
1131 It looks to see whether each line begins with a common substring, and
1132 if so, strips that substring off. Otherwise, it takes the amount of
1133 leading whitespace found on the first line and removes that much off
1134 each subsequent line.
1135 sub fix {
1137 local $_ = shift;
1138 my ($white, $leader); # common whitespace and common leading string
1139 if (/^\s*(?:([^\w\s]+)(\s*).*\n)(?:\s*\g1\g2?.*\n)+$/) {
1140 ($white, $leader) = ($2, quotemeta($1));
1141 } else {
1142 ($white, $leader) = (/^(\s+)/, '');
1143 }
1144 s/^\s*?$leader(?:$white)?//gm;
1145 return $_;
1146 }
1147 This works with leading special strings, dynamically determined:
1149 my $remember_the_main = fix<<' MAIN_INTERPRETER_LOOP';
1151 @@@ int
1152 @@@ runops() {
1153 @@@ SAVEI32(runlevel);
1154 @@@ runlevel++;
1155 @@@ while ( op = (*op->op_ppaddr)() );
1156 @@@ TAINT_NOT;
1157 @@@ return 0;
1158 @@@ }
1159 MAIN_INTERPRETER_LOOP
1160 Or with a fixed amount of leading whitespace, with remaining
1162 indentation correctly preserved:
1163 my $poem = fix<<EVER_ON_AND_ON;
1165 Now far ahead the Road has gone,
1166 And I must follow, if I can,
1167 Pursuing it with eager feet,
1168 Until it joins some larger way
1169 Where many paths and errands meet.
1170 And whither then? I cannot say.
1171 --Bilbo in /usr/src/perl/pp_ctl.c
1172 EVER_ON_AND_ON
1173
1175 What is the difference between a list and an array?
1176 (contributed by brian d foy)
1177 A list is a fixed collection of scalars. An array is a variable that
1179 holds a variable collection of scalars. An array can supply its
1180 collection for list operations, so list operations also work on arrays:
1181 # slices
1183 ( 'dog', 'cat', 'bird' )[2,3];
1184 @animals[2,3];
1185 # iteration
1187 foreach ( qw( dog cat bird ) ) { ... }
1188 foreach ( @animals ) { ... }
1189 my @three = grep { length == 3 } qw( dog cat bird );
1191 my @three = grep { length == 3 } @animals;
1192 # supply an argument list
1194 wash_animals( qw( dog cat bird ) );
1195 wash_animals( @animals );
1196 Array operations, which change the scalars, rearrange them, or add or
1198 subtract some scalars, only work on arrays. These can't work on a list,
1199 which is fixed. Array operations include "shift", "unshift", "push",
1200 "pop", and "splice".
1201 An array can also change its length:
1203 $#animals = 1; # truncate to two elements
1205 $#animals = 10000; # pre-extend to 10,001 elements
1206 You can change an array element, but you can't change a list element:
1208 $animals[0] = 'Rottweiler';
1210 qw( dog cat bird )[0] = 'Rottweiler'; # syntax error!
1211 foreach ( @animals ) {
1213 s/^d/fr/; # works fine
1214 }
1215 foreach ( qw( dog cat bird ) ) {
1217 s/^d/fr/; # Error! Modification of read only value!
1218 }
1219 However, if the list element is itself a variable, it appears that you
1221 can change a list element. However, the list element is the variable,
1222 not the data. You're not changing the list element, but something the
1223 list element refers to. The list element itself doesn't change: it's
1224 still the same variable.
1225 You also have to be careful about context. You can assign an array to a
1227 scalar to get the number of elements in the array. This only works for
1228 arrays, though:
1229 my $count = @animals; # only works with arrays
1231 If you try to do the same thing with what you think is a list, you get
1233 a quite different result. Although it looks like you have a list on the
1234 righthand side, Perl actually sees a bunch of scalars separated by a
1235 comma:
1236 my $scalar = ( 'dog', 'cat', 'bird' ); # $scalar gets bird
1238 Since you're assigning to a scalar, the righthand side is in scalar
1240 context. The comma operator (yes, it's an operator!) in scalar context
1241 evaluates its lefthand side, throws away the result, and evaluates it's
1242 righthand side and returns the result. In effect, that list-lookalike
1243 assigns to $scalar it's rightmost value. Many people mess this up
1244 because they choose a list-lookalike whose last element is also the
1245 count they expect:
1246 my $scalar = ( 1, 2, 3 ); # $scalar gets 3, accidentally
1248 What is the difference between $array[1] and @array[1]?
1250 (contributed by brian d foy)
1251 The difference is the sigil, that special character in front of the
1253 array name. The "$" sigil means "exactly one item", while the "@" sigil
1254 means "zero or more items". The "$" gets you a single scalar, while the
1255 "@" gets you a list.
1256 The confusion arises because people incorrectly assume that the sigil
1258 denotes the variable type.
1259 The $array[1] is a single-element access to the array. It's going to
1261 return the item in index 1 (or undef if there is no item there). If
1262 you intend to get exactly one element from the array, this is the form
1263 you should use.
1264 The @array[1] is an array slice, although it has only one index. You
1266 can pull out multiple elements simultaneously by specifying additional
1267 indices as a list, like @array[1,4,3,0].
1268 Using a slice on the lefthand side of the assignment supplies list
1270 context to the righthand side. This can lead to unexpected results.
1271 For instance, if you want to read a single line from a filehandle,
1272 assigning to a scalar value is fine:
1273 $array[1] = <STDIN>;
1275 However, in list context, the line input operator returns all of the
1277 lines as a list. The first line goes into @array[1] and the rest of the
1278 lines mysteriously disappear:
1279 @array[1] = <STDIN>; # most likely not what you want
1281 Either the "use warnings" pragma or the -w flag will warn you when you
1283 use an array slice with a single index.
1284 How can I remove duplicate elements from a list or array?
1286 (contributed by brian d foy)
1287 Use a hash. When you think the words "unique" or "duplicated", think
1289 "hash keys".
1290 If you don't care about the order of the elements, you could just
1292 create the hash then extract the keys. It's not important how you
1293 create that hash: just that you use "keys" to get the unique elements.
1294 my %hash = map { $_, 1 } @array;
1296 # or a hash slice: @hash{ @array } = ();
1297 # or a foreach: $hash{$_} = 1 foreach ( @array );
1298 my @unique = keys %hash;
1300 If you want to use a module, try the "uniq" function from
1302 List::MoreUtils. In list context it returns the unique elements,
1303 preserving their order in the list. In scalar context, it returns the
1304 number of unique elements.
1305 use List::MoreUtils qw(uniq);
1307 my @unique = uniq( 1, 2, 3, 4, 4, 5, 6, 5, 7 ); # 1,2,3,4,5,6,7
1309 my $unique = uniq( 1, 2, 3, 4, 4, 5, 6, 5, 7 ); # 7
1310 You can also go through each element and skip the ones you've seen
1312 before. Use a hash to keep track. The first time the loop sees an
1313 element, that element has no key in %Seen. The "next" statement creates
1314 the key and immediately uses its value, which is "undef", so the loop
1315 continues to the "push" and increments the value for that key. The next
1316 time the loop sees that same element, its key exists in the hash and
1317 the value for that key is true (since it's not 0 or "undef"), so the
1318 next skips that iteration and the loop goes to the next element.
1319 my @unique = ();
1321 my %seen = ();
1322 foreach my $elem ( @array ) {
1324 next if $seen{ $elem }++;
1325 push @unique, $elem;
1326 }
1327 You can write this more briefly using a grep, which does the same
1329 thing.
1330 my %seen = ();
1332 my @unique = grep { ! $seen{ $_ }++ } @array;
1333 How can I tell whether a certain element is contained in a list or array?
1335 (portions of this answer contributed by Anno Siegel and brian d foy)
1336 Hearing the word "in" is an indication that you probably should have
1338 used a hash, not a list or array, to store your data. Hashes are
1339 designed to answer this question quickly and efficiently. Arrays
1340 aren't.
1341 That being said, there are several ways to approach this. In Perl 5.10
1343 and later, you can use the smart match operator to check that an item
1344 is contained in an array or a hash:
1345 use 5.010;
1347 if( $item ~~ @array ) {
1349 say "The array contains $item"
1350 }
1351 if( $item ~~ %hash ) {
1353 say "The hash contains $item"
1354 }
1355 With earlier versions of Perl, you have to do a bit more work. If you
1357 are going to make this query many times over arbitrary string values,
1358 the fastest way is probably to invert the original array and maintain a
1359 hash whose keys are the first array's values:
1360 my @blues = qw/azure cerulean teal turquoise lapis-lazuli/;
1362 my %is_blue = ();
1363 for (@blues) { $is_blue{$_} = 1 }
1364 Now you can check whether $is_blue{$some_color}. It might have been a
1366 good idea to keep the blues all in a hash in the first place.
1367 If the values are all small integers, you could use a simple indexed
1369 array. This kind of an array will take up less space:
1370 my @primes = (2, 3, 5, 7, 11, 13, 17, 19, 23, 29, 31);
1372 my @is_tiny_prime = ();
1373 for (@primes) { $is_tiny_prime[$_] = 1 }
1374 # or simply @istiny_prime[@primes] = (1) x @primes;
1375 Now you check whether $is_tiny_prime[$some_number].
1377 If the values in question are integers instead of strings, you can save
1379 quite a lot of space by using bit strings instead:
1380 my @articles = ( 1..10, 150..2000, 2017 );
1382 undef $read;
1383 for (@articles) { vec($read,$_,1) = 1 }
1384 Now check whether "vec($read,$n,1)" is true for some $n.
1386 These methods guarantee fast individual tests but require a re-
1388 organization of the original list or array. They only pay off if you
1389 have to test multiple values against the same array.
1390 If you are testing only once, the standard module List::Util exports
1392 the function "first" for this purpose. It works by stopping once it
1393 finds the element. It's written in C for speed, and its Perl equivalent
1394 looks like this subroutine:
1395 sub first (&@) {
1397 my $code = shift;
1398 foreach (@_) {
1399 return $_ if &{$code}();
1400 }
1401 undef;
1402 }
1403 If speed is of little concern, the common idiom uses grep in scalar
1405 context (which returns the number of items that passed its condition)
1406 to traverse the entire list. This does have the benefit of telling you
1407 how many matches it found, though.
1408 my $is_there = grep $_ eq $whatever, @array;
1410 If you want to actually extract the matching elements, simply use grep
1412 in list context.
1413 my @matches = grep $_ eq $whatever, @array;
1415 How do I compute the difference of two arrays? How do I compute the
1417 intersection of two arrays?
1418 Use a hash. Here's code to do both and more. It assumes that each
1419 element is unique in a given array:
1420 my (@union, @intersection, @difference);
1422 my %count = ();
1423 foreach my $element (@array1, @array2) { $count{$element}++ }
1424 foreach my $element (keys %count) {
1425 push @union, $element;
1426 push @{ $count{$element} > 1 ? \@intersection : \@difference }, $element;
1427 }
1428 Note that this is the symmetric difference, that is, all elements in
1430 either A or in B but not in both. Think of it as an xor operation.
1431 How do I test whether two arrays or hashes are equal?
1433 With Perl 5.10 and later, the smart match operator can give you the
1434 answer with the least amount of work:
1435 use 5.010;
1437 if( @array1 ~~ @array2 ) {
1439 say "The arrays are the same";
1440 }
1441 if( %hash1 ~~ %hash2 ) # doesn't check values! {
1443 say "The hash keys are the same";
1444 }
1445 The following code works for single-level arrays. It uses a stringwise
1447 comparison, and does not distinguish defined versus undefined empty
1448 strings. Modify if you have other needs.
1449 $are_equal = compare_arrays(\@frogs, \@toads);
1451 sub compare_arrays {
1453 my ($first, $second) = @_;
1454 no warnings; # silence spurious -w undef complaints
1455 return 0 unless @$first == @$second;
1456 for (my $i = 0; $i < @$first; $i++) {
1457 return 0 if $first->[$i] ne $second->[$i];
1458 }
1459 return 1;
1460 }
1461 For multilevel structures, you may wish to use an approach more like
1463 this one. It uses the CPAN module FreezeThaw:
1464 use FreezeThaw qw(cmpStr);
1466 my @a = my @b = ( "this", "that", [ "more", "stuff" ] );
1467 printf "a and b contain %s arrays\n",
1469 cmpStr(\@a, \@b) == 0
1470 ? "the same"
1471 : "different";
1472 This approach also works for comparing hashes. Here we'll demonstrate
1474 two different answers:
1475 use FreezeThaw qw(cmpStr cmpStrHard);
1477 my %a = my %b = ( "this" => "that", "extra" => [ "more", "stuff" ] );
1479 $a{EXTRA} = \%b;
1480 $b{EXTRA} = \%a;
1481 printf "a and b contain %s hashes\n",
1483 cmpStr(\%a, \%b) == 0 ? "the same" : "different";
1484 printf "a and b contain %s hashes\n",
1486 cmpStrHard(\%a, \%b) == 0 ? "the same" : "different";
1487 The first reports that both those the hashes contain the same data,
1489 while the second reports that they do not. Which you prefer is left as
1490 an exercise to the reader.
1491 How do I find the first array element for which a condition is true?
1493 To find the first array element which satisfies a condition, you can
1494 use the "first()" function in the List::Util module, which comes with
1495 Perl 5.8. This example finds the first element that contains "Perl".
1496 use List::Util qw(first);
1498 my $element = first { /Perl/ } @array;
1500 If you cannot use List::Util, you can make your own loop to do the same
1502 thing. Once you find the element, you stop the loop with last.
1503 my $found;
1505 foreach ( @array ) {
1506 if( /Perl/ ) { $found = $_; last }
1507 }
1508 If you want the array index, use the "firstidx()" function from
1510 "List::MoreUtils":
1511 use List::MoreUtils qw(firstidx);
1513 my $index = firstidx { /Perl/ } @array;
1514 Or write it yourself, iterating through the indices and checking the
1516 array element at each index until you find one that satisfies the
1517 condition:
1518 my( $found, $index ) = ( undef, -1 );
1520 for( $i = 0; $i < @array; $i++ ) {
1521 if( $array[$i] =~ /Perl/ ) {
1522 $found = $array[$i];
1523 $index = $i;
1524 last;
1525 }
1526 }
1527 How do I handle linked lists?
1529 (contributed by brian d foy)
1530 Perl's arrays do not have a fixed size, so you don't need linked lists
1532 if you just want to add or remove items. You can use array operations
1533 such as "push", "pop", "shift", "unshift", or "splice" to do that.
1534 Sometimes, however, linked lists can be useful in situations where you
1536 want to "shard" an array so you have many small arrays instead of a
1537 single big array. You can keep arrays longer than Perl's largest array
1538 index, lock smaller arrays separately in threaded programs, reallocate
1539 less memory, or quickly insert elements in the middle of the chain.
1540 Steve Lembark goes through the details in his YAPC::NA 2009 talk "Perly
1542 Linked Lists" ( <http://www.slideshare.net/lembark/perly-linked-lists>
1543 ), although you can just use his LinkedList::Single module.
1544 How do I handle circular lists?
1546 (contributed by brian d foy)
1547 If you want to cycle through an array endlessly, you can increment the
1549 index modulo the number of elements in the array:
1550 my @array = qw( a b c );
1552 my $i = 0;
1553 while( 1 ) {
1555 print $array[ $i++ % @array ], "\n";
1556 last if $i > 20;
1557 }
1558 You can also use Tie::Cycle to use a scalar that always has the next
1560 element of the circular array:
1561 use Tie::Cycle;
1563 tie my $cycle, 'Tie::Cycle', [ qw( FFFFFF 000000 FFFF00 ) ];
1565 print $cycle; # FFFFFF
1567 print $cycle; # 000000
1568 print $cycle; # FFFF00
1569 The Array::Iterator::Circular creates an iterator object for circular
1571 arrays:
1572 use Array::Iterator::Circular;
1574 my $color_iterator = Array::Iterator::Circular->new(
1576 qw(red green blue orange)
1577 );
1578 foreach ( 1 .. 20 ) {
1580 print $color_iterator->next, "\n";
1581 }
1582 How do I shuffle an array randomly?
1584 If you either have Perl 5.8.0 or later installed, or if you have
1585 Scalar-List-Utils 1.03 or later installed, you can say:
1586 use List::Util 'shuffle';
1588 @shuffled = shuffle(@list);
1590 If not, you can use a Fisher-Yates shuffle.
1592 sub fisher_yates_shuffle {
1594 my $deck = shift; # $deck is a reference to an array
1595 return unless @$deck; # must not be empty!
1596 my $i = @$deck;
1598 while (--$i) {
1599 my $j = int rand ($i+1);
1600 @$deck[$i,$j] = @$deck[$j,$i];
1601 }
1602 }
1603 # shuffle my mpeg collection
1605 #
1606 my @mpeg = <audio/*/*.mp3>;
1607 fisher_yates_shuffle( \@mpeg ); # randomize @mpeg in place
1608 print @mpeg;
1609 Note that the above implementation shuffles an array in place, unlike
1611 the "List::Util::shuffle()" which takes a list and returns a new
1612 shuffled list.
1613 You've probably seen shuffling algorithms that work using splice,
1615 randomly picking another element to swap the current element with
1616 srand;
1618 @new = ();
1619 @old = 1 .. 10; # just a demo
1620 while (@old) {
1621 push(@new, splice(@old, rand @old, 1));
1622 }
1623 This is bad because splice is already O(N), and since you do it N
1625 times, you just invented a quadratic algorithm; that is, O(N**2). This
1626 does not scale, although Perl is so efficient that you probably won't
1627 notice this until you have rather largish arrays.
1628 How do I process/modify each element of an array?
1630 Use "for"/"foreach":
1631 for (@lines) {
1633 s/foo/bar/; # change that word
1634 tr/XZ/ZX/; # swap those letters
1635 }
1636 Here's another; let's compute spherical volumes:
1638 my @volumes = @radii;
1640 for (@volumes) { # @volumes has changed parts
1641 $_ **= 3;
1642 $_ *= (4/3) * 3.14159; # this will be constant folded
1643 }
1644 which can also be done with "map()" which is made to transform one list
1646 into another:
1647 my @volumes = map {$_ ** 3 * (4/3) * 3.14159} @radii;
1649 If you want to do the same thing to modify the values of the hash, you
1651 can use the "values" function. As of Perl 5.6 the values are not
1652 copied, so if you modify $orbit (in this case), you modify the value.
1653 for my $orbit ( values %orbits ) {
1655 ($orbit **= 3) *= (4/3) * 3.14159;
1656 }
1657 Prior to perl 5.6 "values" returned copies of the values, so older perl
1659 code often contains constructions such as @orbits{keys %orbits} instead
1660 of "values %orbits" where the hash is to be modified.
1661 How do I select a random element from an array?
1663 Use the "rand()" function (see "rand" in perlfunc):
1664 my $index = rand @array;
1666 my $element = $array[$index];
1667 Or, simply:
1669 my $element = $array[ rand @array ];
1671 How do I permute N elements of a list?
1673 Use the List::Permutor module on CPAN. If the list is actually an
1674 array, try the Algorithm::Permute module (also on CPAN). It's written
1675 in XS code and is very efficient:
1676 use Algorithm::Permute;
1678 my @array = 'a'..'d';
1680 my $p_iterator = Algorithm::Permute->new ( \@array );
1681 while (my @perm = $p_iterator->next) {
1683 print "next permutation: (@perm)\n";
1684 }
1685 For even faster execution, you could do:
1687 use Algorithm::Permute;
1689 my @array = 'a'..'d';
1691 Algorithm::Permute::permute {
1693 print "next permutation: (@array)\n";
1694 } @array;
1695 Here's a little program that generates all permutations of all the
1697 words on each line of input. The algorithm embodied in the "permute()"
1698 function is discussed in Volume 4 (still unpublished) of Knuth's The
1699 Art of Computer Programming and will work on any list:
1700 #!/usr/bin/perl -n
1702 # Fischer-Krause ordered permutation generator
1703 sub permute (&@) {
1705 my $code = shift;
1706 my @idx = 0..$#_;
1707 while ( $code->(@_[@idx]) ) {
1708 my $p = $#idx;
1709 --$p while $idx[$p-1] > $idx[$p];
1710 my $q = $p or return;
1711 push @idx, reverse splice @idx, $p;
1712 ++$q while $idx[$p-1] > $idx[$q];
1713 @idx[$p-1,$q]=@idx[$q,$p-1];
1714 }
1715 }
1716 permute { print "@_\n" } split;
1718 The Algorithm::Loops module also provides the "NextPermute" and
1720 "NextPermuteNum" functions which efficiently find all unique
1721 permutations of an array, even if it contains duplicate values,
1722 modifying it in-place: if its elements are in reverse-sorted order then
1723 the array is reversed, making it sorted, and it returns false;
1724 otherwise the next permutation is returned.
1725 "NextPermute" uses string order and "NextPermuteNum" numeric order, so
1727 you can enumerate all the permutations of 0..9 like this:
1728 use Algorithm::Loops qw(NextPermuteNum);
1730 my @list= 0..9;
1732 do { print "@list\n" } while NextPermuteNum @list;
1733 How do I sort an array by (anything)?
1735 Supply a comparison function to sort() (described in "sort" in
1736 perlfunc):
1737 @list = sort { $a <=> $b } @list;
1739 The default sort function is cmp, string comparison, which would sort
1741 "(1, 2, 10)" into "(1, 10, 2)". "<=>", used above, is the numerical
1742 comparison operator.
1743 If you have a complicated function needed to pull out the part you want
1745 to sort on, then don't do it inside the sort function. Pull it out
1746 first, because the sort BLOCK can be called many times for the same
1747 element. Here's an example of how to pull out the first word after the
1748 first number on each item, and then sort those words case-
1749 insensitively.
1750 my @idx;
1752 for (@data) {
1753 my $item;
1754 ($item) = /\d+\s*(\S+)/;
1755 push @idx, uc($item);
1756 }
1757 my @sorted = @data[ sort { $idx[$a] cmp $idx[$b] } 0 .. $#idx ];
1758 which could also be written this way, using a trick that's come to be
1760 known as the Schwartzian Transform:
1761 my @sorted = map { $_->[0] }
1763 sort { $a->[1] cmp $b->[1] }
1764 map { [ $_, uc( (/\d+\s*(\S+)/)[0]) ] } @data;
1765 If you need to sort on several fields, the following paradigm is
1767 useful.
1768 my @sorted = sort {
1770 field1($a) <=> field1($b) ||
1771 field2($a) cmp field2($b) ||
1772 field3($a) cmp field3($b)
1773 } @data;
1774 This can be conveniently combined with precalculation of keys as given
1776 above.
1777 See the sort article in the "Far More Than You Ever Wanted To Know"
1779 collection in <http://www.cpan.org/misc/olddoc/FMTEYEWTK.tgz> for more
1780 about this approach.
1781 See also the question later in perlfaq4 on sorting hashes.
1783 How do I manipulate arrays of bits?
1785 Use "pack()" and "unpack()", or else "vec()" and the bitwise
1786 operations.
1787 For example, you don't have to store individual bits in an array (which
1789 would mean that you're wasting a lot of space). To convert an array of
1790 bits to a string, use "vec()" to set the right bits. This sets $vec to
1791 have bit N set only if $ints[N] was set:
1792 my @ints = (...); # array of bits, e.g. ( 1, 0, 0, 1, 1, 0 ... )
1794 my $vec = '';
1795 foreach( 0 .. $#ints ) {
1796 vec($vec,$_,1) = 1 if $ints[$_];
1797 }
1798 The string $vec only takes up as many bits as it needs. For instance,
1800 if you had 16 entries in @ints, $vec only needs two bytes to store them
1801 (not counting the scalar variable overhead).
1802 Here's how, given a vector in $vec, you can get those bits into your
1804 @ints array:
1805 sub bitvec_to_list {
1807 my $vec = shift;
1808 my @ints;
1809 # Find null-byte density then select best algorithm
1810 if ($vec =~ tr/\0// / length $vec > 0.95) {
1811 use integer;
1812 my $i;
1813 # This method is faster with mostly null-bytes
1815 while($vec =~ /[^\0]/g ) {
1816 $i = -9 + 8 * pos $vec;
1817 push @ints, $i if vec($vec, ++$i, 1);
1818 push @ints, $i if vec($vec, ++$i, 1);
1819 push @ints, $i if vec($vec, ++$i, 1);
1820 push @ints, $i if vec($vec, ++$i, 1);
1821 push @ints, $i if vec($vec, ++$i, 1);
1822 push @ints, $i if vec($vec, ++$i, 1);
1823 push @ints, $i if vec($vec, ++$i, 1);
1824 push @ints, $i if vec($vec, ++$i, 1);
1825 }
1826 }
1827 else {
1828 # This method is a fast general algorithm
1829 use integer;
1830 my $bits = unpack "b*", $vec;
1831 push @ints, 0 if $bits =~ s/^(\d)// && $1;
1832 push @ints, pos $bits while($bits =~ /1/g);
1833 }
1834 return \@ints;
1836 }
1837 This method gets faster the more sparse the bit vector is. (Courtesy
1839 of Tim Bunce and Winfried Koenig.)
1840 You can make the while loop a lot shorter with this suggestion from
1842 Benjamin Goldberg:
1843 while($vec =~ /[^\0]+/g ) {
1845 push @ints, grep vec($vec, $_, 1), $-[0] * 8 .. $+[0] * 8;
1846 }
1847 Or use the CPAN module Bit::Vector:
1849 my $vector = Bit::Vector->new($num_of_bits);
1851 $vector->Index_List_Store(@ints);
1852 my @ints = $vector->Index_List_Read();
1853 Bit::Vector provides efficient methods for bit vector, sets of small
1855 integers and "big int" math.
1856 Here's a more extensive illustration using vec():
1858 # vec demo
1860 my $vector = "\xff\x0f\xef\xfe";
1861 print "Ilya's string \\xff\\x0f\\xef\\xfe represents the number ",
1862 unpack("N", $vector), "\n";
1863 my $is_set = vec($vector, 23, 1);
1864 print "Its 23rd bit is ", $is_set ? "set" : "clear", ".\n";
1865 pvec($vector);
1866 set_vec(1,1,1);
1868 set_vec(3,1,1);
1869 set_vec(23,1,1);
1870 set_vec(3,1,3);
1872 set_vec(3,2,3);
1873 set_vec(3,4,3);
1874 set_vec(3,4,7);
1875 set_vec(3,8,3);
1876 set_vec(3,8,7);
1877 set_vec(0,32,17);
1879 set_vec(1,32,17);
1880 sub set_vec {
1882 my ($offset, $width, $value) = @_;
1883 my $vector = '';
1884 vec($vector, $offset, $width) = $value;
1885 print "offset=$offset width=$width value=$value\n";
1886 pvec($vector);
1887 }
1888 sub pvec {
1890 my $vector = shift;
1891 my $bits = unpack("b*", $vector);
1892 my $i = 0;
1893 my $BASE = 8;
1894 print "vector length in bytes: ", length($vector), "\n";
1896 @bytes = unpack("A8" x length($vector), $bits);
1897 print "bits are: @bytes\n\n";
1898 }
1899 Why does defined() return true on empty arrays and hashes?
1901 The short story is that you should probably only use defined on scalars
1902 or functions, not on aggregates (arrays and hashes). See "defined" in
1903 perlfunc in the 5.004 release or later of Perl for more detail.
1904
1906 How do I process an entire hash?
1907 (contributed by brian d foy)
1908 There are a couple of ways that you can process an entire hash. You can
1910 get a list of keys, then go through each key, or grab a one key-value
1911 pair at a time.
1912 To go through all of the keys, use the "keys" function. This extracts
1914 all of the keys of the hash and gives them back to you as a list. You
1915 can then get the value through the particular key you're processing:
1916 foreach my $key ( keys %hash ) {
1918 my $value = $hash{$key}
1919 ...
1920 }
1921 Once you have the list of keys, you can process that list before you
1923 process the hash elements. For instance, you can sort the keys so you
1924 can process them in lexical order:
1925 foreach my $key ( sort keys %hash ) {
1927 my $value = $hash{$key}
1928 ...
1929 }
1930 Or, you might want to only process some of the items. If you only want
1932 to deal with the keys that start with "text:", you can select just
1933 those using "grep":
1934 foreach my $key ( grep /^text:/, keys %hash ) {
1936 my $value = $hash{$key}
1937 ...
1938 }
1939 If the hash is very large, you might not want to create a long list of
1941 keys. To save some memory, you can grab one key-value pair at a time
1942 using "each()", which returns a pair you haven't seen yet:
1943 while( my( $key, $value ) = each( %hash ) ) {
1945 ...
1946 }
1947 The "each" operator returns the pairs in apparently random order, so if
1949 ordering matters to you, you'll have to stick with the "keys" method.
1950 The "each()" operator can be a bit tricky though. You can't add or
1952 delete keys of the hash while you're using it without possibly skipping
1953 or re-processing some pairs after Perl internally rehashes all of the
1954 elements. Additionally, a hash has only one iterator, so if you mix
1955 "keys", "values", or "each" on the same hash, you risk resetting the
1956 iterator and messing up your processing. See the "each" entry in
1957 perlfunc for more details.
1958 How do I merge two hashes?
1960 (contributed by brian d foy)
1961 Before you decide to merge two hashes, you have to decide what to do if
1963 both hashes contain keys that are the same and if you want to leave the
1964 original hashes as they were.
1965 If you want to preserve the original hashes, copy one hash (%hash1) to
1967 a new hash (%new_hash), then add the keys from the other hash (%hash2
1968 to the new hash. Checking that the key already exists in %new_hash
1969 gives you a chance to decide what to do with the duplicates:
1970 my %new_hash = %hash1; # make a copy; leave %hash1 alone
1972 foreach my $key2 ( keys %hash2 ) {
1974 if( exists $new_hash{$key2} ) {
1975 warn "Key [$key2] is in both hashes!";
1976 # handle the duplicate (perhaps only warning)
1977 ...
1978 next;
1979 }
1980 else {
1981 $new_hash{$key2} = $hash2{$key2};
1982 }
1983 }
1984 If you don't want to create a new hash, you can still use this looping
1986 technique; just change the %new_hash to %hash1.
1987 foreach my $key2 ( keys %hash2 ) {
1989 if( exists $hash1{$key2} ) {
1990 warn "Key [$key2] is in both hashes!";
1991 # handle the duplicate (perhaps only warning)
1992 ...
1993 next;
1994 }
1995 else {
1996 $hash1{$key2} = $hash2{$key2};
1997 }
1998 }
1999 If you don't care that one hash overwrites keys and values from the
2001 other, you could just use a hash slice to add one hash to another. In
2002 this case, values from %hash2 replace values from %hash1 when they have
2003 keys in common:
2004 @hash1{ keys %hash2 } = values %hash2;
2006 What happens if I add or remove keys from a hash while iterating over it?
2008 (contributed by brian d foy)
2009 The easy answer is "Don't do that!"
2011 If you iterate through the hash with each(), you can delete the key
2013 most recently returned without worrying about it. If you delete or add
2014 other keys, the iterator may skip or double up on them since perl may
2015 rearrange the hash table. See the entry for "each()" in perlfunc.
2016 How do I look up a hash element by value?
2018 Create a reverse hash:
2019 my %by_value = reverse %by_key;
2021 my $key = $by_value{$value};
2022 That's not particularly efficient. It would be more space-efficient to
2024 use:
2025 while (my ($key, $value) = each %by_key) {
2027 $by_value{$value} = $key;
2028 }
2029 If your hash could have repeated values, the methods above will only
2031 find one of the associated keys. This may or may not worry you. If it
2032 does worry you, you can always reverse the hash into a hash of arrays
2033 instead:
2034 while (my ($key, $value) = each %by_key) {
2036 push @{$key_list_by_value{$value}}, $key;
2037 }
2038 How can I know how many entries are in a hash?
2040 (contributed by brian d foy)
2041 This is very similar to "How do I process an entire hash?", also in
2043 perlfaq4, but a bit simpler in the common cases.
2044 You can use the "keys()" built-in function in scalar context to find
2046 out have many entries you have in a hash:
2047 my $key_count = keys %hash; # must be scalar context!
2049 If you want to find out how many entries have a defined value, that's a
2051 bit different. You have to check each value. A "grep" is handy:
2052 my $defined_value_count = grep { defined } values %hash;
2054 You can use that same structure to count the entries any way that you
2056 like. If you want the count of the keys with vowels in them, you just
2057 test for that instead:
2058 my $vowel_count = grep { /[aeiou]/ } keys %hash;
2060 The "grep" in scalar context returns the count. If you want the list of
2062 matching items, just use it in list context instead:
2063 my @defined_values = grep { defined } values %hash;
2065 The "keys()" function also resets the iterator, which means that you
2067 may see strange results if you use this between uses of other hash
2068 operators such as "each()".
2069 How do I sort a hash (optionally by value instead of key)?
2071 (contributed by brian d foy)
2072 To sort a hash, start with the keys. In this example, we give the list
2074 of keys to the sort function which then compares them ASCIIbetically
2075 (which might be affected by your locale settings). The output list has
2076 the keys in ASCIIbetical order. Once we have the keys, we can go
2077 through them to create a report which lists the keys in ASCIIbetical
2078 order.
2079 my @keys = sort { $a cmp $b } keys %hash;
2081 foreach my $key ( @keys ) {
2083 printf "%-20s %6d\n", $key, $hash{$key};
2084 }
2085 We could get more fancy in the "sort()" block though. Instead of
2087 comparing the keys, we can compute a value with them and use that value
2088 as the comparison.
2089 For instance, to make our report order case-insensitive, we use "lc" to
2091 lowercase the keys before comparing them:
2092 my @keys = sort { lc $a cmp lc $b } keys %hash;
2094 Note: if the computation is expensive or the hash has many elements,
2096 you may want to look at the Schwartzian Transform to cache the
2097 computation results.
2098 If we want to sort by the hash value instead, we use the hash key to
2100 look it up. We still get out a list of keys, but this time they are
2101 ordered by their value.
2102 my @keys = sort { $hash{$a} <=> $hash{$b} } keys %hash;
2104 From there we can get more complex. If the hash values are the same, we
2106 can provide a secondary sort on the hash key.
2107 my @keys = sort {
2109 $hash{$a} <=> $hash{$b}
2110 or
2111 "\L$a" cmp "\L$b"
2112 } keys %hash;
2113 How can I always keep my hash sorted?
2115 You can look into using the "DB_File" module and "tie()" using the
2116 $DB_BTREE hash bindings as documented in "In Memory Databases" in
2117 DB_File. The Tie::IxHash module from CPAN might also be instructive.
2118 Although this does keep your hash sorted, you might not like the
2119 slowdown you suffer from the tie interface. Are you sure you need to do
2120 this? :)
2121 What's the difference between "delete" and "undef" with hashes?
2123 Hashes contain pairs of scalars: the first is the key, the second is
2124 the value. The key will be coerced to a string, although the value can
2125 be any kind of scalar: string, number, or reference. If a key $key is
2126 present in %hash, "exists($hash{$key})" will return true. The value for
2127 a given key can be "undef", in which case $hash{$key} will be "undef"
2128 while "exists $hash{$key}" will return true. This corresponds to ($key,
2129 "undef") being in the hash.
2130 Pictures help... Here's the %hash table:
2132 keys values
2134 +------+------+
2135 | a | 3 |
2136 | x | 7 |
2137 | d | 0 |
2138 | e | 2 |
2139 +------+------+
2140 And these conditions hold
2142 $hash{'a'} is true
2144 $hash{'d'} is false
2145 defined $hash{'d'} is true
2146 defined $hash{'a'} is true
2147 exists $hash{'a'} is true (Perl 5 only)
2148 grep ($_ eq 'a', keys %hash) is true
2149 If you now say
2151 undef $hash{'a'}
2153 your table now reads:
2155 keys values
2157 +------+------+
2158 | a | undef|
2159 | x | 7 |
2160 | d | 0 |
2161 | e | 2 |
2162 +------+------+
2163 and these conditions now hold; changes in caps:
2165 $hash{'a'} is FALSE
2167 $hash{'d'} is false
2168 defined $hash{'d'} is true
2169 defined $hash{'a'} is FALSE
2170 exists $hash{'a'} is true (Perl 5 only)
2171 grep ($_ eq 'a', keys %hash) is true
2172 Notice the last two: you have an undef value, but a defined key!
2174 Now, consider this:
2176 delete $hash{'a'}
2178 your table now reads:
2180 keys values
2182 +------+------+
2183 | x | 7 |
2184 | d | 0 |
2185 | e | 2 |
2186 +------+------+
2187 and these conditions now hold; changes in caps:
2189 $hash{'a'} is false
2191 $hash{'d'} is false
2192 defined $hash{'d'} is true
2193 defined $hash{'a'} is false
2194 exists $hash{'a'} is FALSE (Perl 5 only)
2195 grep ($_ eq 'a', keys %hash) is FALSE
2196 See, the whole entry is gone!
2198 Why don't my tied hashes make the defined/exists distinction?
2200 This depends on the tied hash's implementation of EXISTS(). For
2201 example, there isn't the concept of undef with hashes that are tied to
2202 DBM* files. It also means that exists() and defined() do the same thing
2203 with a DBM* file, and what they end up doing is not what they do with
2204 ordinary hashes.
2205 How do I reset an each() operation part-way through?
2207 (contributed by brian d foy)
2208 You can use the "keys" or "values" functions to reset "each". To simply
2210 reset the iterator used by "each" without doing anything else, use one
2211 of them in void context:
2212 keys %hash; # resets iterator, nothing else.
2214 values %hash; # resets iterator, nothing else.
2215 See the documentation for "each" in perlfunc.
2217 How can I get the unique keys from two hashes?
2219 First you extract the keys from the hashes into lists, then solve the
2220 "removing duplicates" problem described above. For example:
2221 my %seen = ();
2223 for my $element (keys(%foo), keys(%bar)) {
2224 $seen{$element}++;
2225 }
2226 my @uniq = keys %seen;
2227 Or more succinctly:
2229 my @uniq = keys %{{%foo,%bar}};
2231 Or if you really want to save space:
2233 my %seen = ();
2235 while (defined ($key = each %foo)) {
2236 $seen{$key}++;
2237 }
2238 while (defined ($key = each %bar)) {
2239 $seen{$key}++;
2240 }
2241 my @uniq = keys %seen;
2242 How can I store a multidimensional array in a DBM file?
2244 Either stringify the structure yourself (no fun), or else get the MLDBM
2245 (which uses Data::Dumper) module from CPAN and layer it on top of
2246 either DB_File or GDBM_File. You might also try DBM::Deep, but it can
2247 be a bit slow.
2248 How can I make my hash remember the order I put elements into it?
2250 Use the Tie::IxHash from CPAN.
2251 use Tie::IxHash;
2253 tie my %myhash, 'Tie::IxHash';
2255 for (my $i=0; $i<20; $i++) {
2257 $myhash{$i} = 2*$i;
2258 }
2259 my @keys = keys %myhash;
2261 # @keys = (0,1,2,3,...)
2262 Why does passing a subroutine an undefined element in a hash create it?
2264 (contributed by brian d foy)
2265 Are you using a really old version of Perl?
2267 Normally, accessing a hash key's value for a nonexistent key will not
2269 create the key.
2270 my %hash = ();
2272 my $value = $hash{ 'foo' };
2273 print "This won't print\n" if exists $hash{ 'foo' };
2274 Passing $hash{ 'foo' } to a subroutine used to be a special case,
2276 though. Since you could assign directly to $_[0], Perl had to be ready
2277 to make that assignment so it created the hash key ahead of time:
2278 my_sub( $hash{ 'foo' } );
2280 print "This will print before 5.004\n" if exists $hash{ 'foo' };
2281 sub my_sub {
2283 # $_[0] = 'bar'; # create hash key in case you do this
2284 1;
2285 }
2286 Since Perl 5.004, however, this situation is a special case and Perl
2288 creates the hash key only when you make the assignment:
2289 my_sub( $hash{ 'foo' } );
2291 print "This will print, even after 5.004\n" if exists $hash{ 'foo' };
2292 sub my_sub {
2294 $_[0] = 'bar';
2295 }
2296 However, if you want the old behavior (and think carefully about that
2298 because it's a weird side effect), you can pass a hash slice instead.
2299 Perl 5.004 didn't make this a special case:
2300 my_sub( @hash{ qw/foo/ } );
2302 How can I make the Perl equivalent of a C structure/C++ class/hash or array
2304 of hashes or arrays?
2305 Usually a hash ref, perhaps like this:
2306 $record = {
2308 NAME => "Jason",
2309 EMPNO => 132,
2310 TITLE => "deputy peon",
2311 AGE => 23,
2312 SALARY => 37_000,
2313 PALS => [ "Norbert", "Rhys", "Phineas"],
2314 };
2315 References are documented in perlref and perlreftut. Examples of
2317 complex data structures are given in perldsc and perllol. Examples of
2318 structures and object-oriented classes are in perlootut.
2319 How can I use a reference as a hash key?
2321 (contributed by brian d foy and Ben Morrow)
2322 Hash keys are strings, so you can't really use a reference as the key.
2324 When you try to do that, perl turns the reference into its stringified
2325 form (for instance, "HASH(0xDEADBEEF)"). From there you can't get back
2326 the reference from the stringified form, at least without doing some
2327 extra work on your own.
2328 Remember that the entry in the hash will still be there even if the
2330 referenced variable goes out of scope, and that it is entirely
2331 possible for Perl to subsequently allocate a different variable at the
2332 same address. This will mean a new variable might accidentally be
2333 associated with the value for an old.
2334 If you have Perl 5.10 or later, and you just want to store a value
2336 against the reference for lookup later, you can use the core
2337 Hash::Util::Fieldhash module. This will also handle renaming the keys
2338 if you use multiple threads (which causes all variables to be
2339 reallocated at new addresses, changing their stringification), and
2340 garbage-collecting the entries when the referenced variable goes out of
2341 scope.
2342 If you actually need to be able to get a real reference back from each
2344 hash entry, you can use the Tie::RefHash module, which does the
2345 required work for you.
2346 How can I check if a key exists in a multilevel hash?
2348 (contributed by brian d foy)
2349 The trick to this problem is avoiding accidental autovivification. If
2351 you want to check three keys deep, you might naïvely try this:
2352 my %hash;
2354 if( exists $hash{key1}{key2}{key3} ) {
2355 ...;
2356 }
2357 Even though you started with a completely empty hash, after that call
2359 to "exists" you've created the structure you needed to check for
2360 "key3":
2361 %hash = (
2363 'key1' => {
2364 'key2' => {}
2365 }
2366 );
2367 That's autovivification. You can get around this in a few ways. The
2369 easiest way is to just turn it off. The lexical "autovivification"
2370 pragma is available on CPAN. Now you don't add to the hash:
2371 {
2373 no autovivification;
2374 my %hash;
2375 if( exists $hash{key1}{key2}{key3} ) {
2376 ...;
2377 }
2378 }
2379 The Data::Diver module on CPAN can do it for you too. Its "Dive"
2381 subroutine can tell you not only if the keys exist but also get the
2382 value:
2383 use Data::Diver qw(Dive);
2385 my @exists = Dive( \%hash, qw(key1 key2 key3) );
2387 if( ! @exists ) {
2388 ...; # keys do not exist
2389 }
2390 elsif( ! defined $exists[0] ) {
2391 ...; # keys exist but value is undef
2392 }
2393 You can easily do this yourself too by checking each level of the hash
2395 before you move onto the next level. This is essentially what
2396 Data::Diver does for you:
2397 if( check_hash( \%hash, qw(key1 key2 key3) ) ) {
2399 ...;
2400 }
2401 sub check_hash {
2403 my( $hash, @keys ) = @_;
2404 return unless @keys;
2406 foreach my $key ( @keys ) {
2408 return unless eval { exists $hash->{$key} };
2409 $hash = $hash->{$key};
2410 }
2411 return 1;
2413 }
2414 How can I prevent addition of unwanted keys into a hash?
2416 Since version 5.8.0, hashes can be restricted to a fixed number of
2417 given keys. Methods for creating and dealing with restricted hashes are
2418 exported by the Hash::Util module.
2419
2421 How do I handle binary data correctly?
2422 Perl is binary-clean, so it can handle binary data just fine. On
2423 Windows or DOS, however, you have to use "binmode" for binary files to
2424 avoid conversions for line endings. In general, you should use
2425 "binmode" any time you want to work with binary data.
2426 Also see "binmode" in perlfunc or perlopentut.
2428 If you're concerned about 8-bit textual data then see perllocale. If
2430 you want to deal with multibyte characters, however, there are some
2431 gotchas. See the section on Regular Expressions.
2432 How do I determine whether a scalar is a number/whole/integer/float?
2434 Assuming that you don't care about IEEE notations like "NaN" or
2435 "Infinity", you probably just want to use a regular expression (see
2436 also perlretut and perlre):
2437 use 5.010;
2439 if ( /\D/ )
2441 { say "\thas nondigits"; }
2442 if ( /^\d+\z/ )
2443 { say "\tis a whole number"; }
2444 if ( /^-?\d+\z/ )
2445 { say "\tis an integer"; }
2446 if ( /^[+-]?\d+\z/ )
2447 { say "\tis a +/- integer"; }
2448 if ( /^-?(?:\d+\.?|\.\d)\d*\z/ )
2449 { say "\tis a real number"; }
2450 if ( /^[+-]?(?=\.?\d)\d*\.?\d*(?:e[+-]?\d+)?\z/i )
2451 { say "\tis a C float" }
2452 There are also some commonly used modules for the task. Scalar::Util
2454 (distributed with 5.8) provides access to perl's internal function
2455 "looks_like_number" for determining whether a variable looks like a
2456 number. Data::Types exports functions that validate data types using
2457 both the above and other regular expressions. Thirdly, there is
2458 Regexp::Common which has regular expressions to match various types of
2459 numbers. Those three modules are available from the CPAN.
2460 If you're on a POSIX system, Perl supports the "POSIX::strtod" function
2462 for converting strings to doubles (and also "POSIX::strtol" for longs).
2463 Its semantics are somewhat cumbersome, so here's a "getnum" wrapper
2464 function for more convenient access. This function takes a string and
2465 returns the number it found, or "undef" for input that isn't a C float.
2466 The "is_numeric" function is a front end to "getnum" if you just want
2467 to say, "Is this a float?"
2468 sub getnum {
2470 use POSIX qw(strtod);
2471 my $str = shift;
2472 $str =~ s/^\s+//;
2473 $str =~ s/\s+$//;
2474 $! = 0;
2475 my($num, $unparsed) = strtod($str);
2476 if (($str eq '') || ($unparsed != 0) || $!) {
2477 return undef;
2478 }
2479 else {
2480 return $num;
2481 }
2482 }
2483 sub is_numeric { defined getnum($_[0]) }
2485 Or you could check out the String::Scanf module on the CPAN instead.
2487 How do I keep persistent data across program calls?
2489 For some specific applications, you can use one of the DBM modules.
2490 See AnyDBM_File. More generically, you should consult the FreezeThaw or
2491 Storable modules from CPAN. Starting from Perl 5.8, Storable is part of
2492 the standard distribution. Here's one example using Storable's "store"
2493 and "retrieve" functions:
2494 use Storable;
2496 store(\%hash, "filename");
2497 # later on...
2499 $href = retrieve("filename"); # by ref
2500 %hash = %{ retrieve("filename") }; # direct to hash
2501 How do I print out or copy a recursive data structure?
2503 The Data::Dumper module on CPAN (or the 5.005 release of Perl) is great
2504 for printing out data structures. The Storable module on CPAN (or the
2505 5.8 release of Perl), provides a function called "dclone" that
2506 recursively copies its argument.
2507 use Storable qw(dclone);
2509 $r2 = dclone($r1);
2510 Where $r1 can be a reference to any kind of data structure you'd like.
2512 It will be deeply copied. Because "dclone" takes and returns
2513 references, you'd have to add extra punctuation if you had a hash of
2514 arrays that you wanted to copy.
2515 %newhash = %{ dclone(\%oldhash) };
2517 How do I define methods for every class/object?
2519 (contributed by Ben Morrow)
2520 You can use the "UNIVERSAL" class (see UNIVERSAL). However, please be
2522 very careful to consider the consequences of doing this: adding methods
2523 to every object is very likely to have unintended consequences. If
2524 possible, it would be better to have all your object inherit from some
2525 common base class, or to use an object system like Moose that supports
2526 roles.
2527 How do I verify a credit card checksum?
2529 Get the Business::CreditCard module from CPAN.
2530 How do I pack arrays of doubles or floats for XS code?
2532 The arrays.h/arrays.c code in the PGPLOT module on CPAN does just this.
2533 If you're doing a lot of float or double processing, consider using the
2534 PDL module from CPAN instead--it makes number-crunching easy.
2535 See <https://metacpan.org/release/PGPLOT> for the code.
2537
2539 Copyright (c) 1997-2010 Tom Christiansen, Nathan Torkington, and other
2540 authors as noted. All rights reserved.
2541 This documentation is free; you can redistribute it and/or modify it
2543 under the same terms as Perl itself.
2544 Irrespective of its distribution, all code examples in this file are
2546 hereby placed into the public domain. You are permitted and encouraged
2547 to use this code in your own programs for fun or for profit as you see
2548 fit. A simple comment in the code giving credit would be courteous but
2549 is not required.
2550perl v5.26.3 2018-06-05 perlfaq4(3)