1perlfaq4(3) User Contributed Perl Documentation perlfaq4(3)
2
6 perlfaq4 - Data Manipulation
7
9 version 5.20200125
10
12 This section of the FAQ answers questions related to manipulating
13 numbers, dates, strings, arrays, hashes, and miscellaneous data issues.
14
16 Why am I getting long decimals (eg, 19.9499999999999) instead of the
17 numbers I should be getting (eg, 19.95)?
18 For the long explanation, see David Goldberg's "What Every Computer
19 Scientist Should Know About Floating-Point Arithmetic"
20 (<http://web.cse.msu.edu/~cse320/Documents/FloatingPoint.pdf>).
21 Internally, your computer represents floating-point numbers in binary.
23 Digital (as in powers of two) computers cannot store all numbers
24 exactly. Some real numbers lose precision in the process. This is a
25 problem with how computers store numbers and affects all computer
26 languages, not just Perl.
27 perlnumber shows the gory details of number representations and
29 conversions.
30 To limit the number of decimal places in your numbers, you can use the
32 "printf" or "sprintf" function. See "Floating-point Arithmetic" in
33 perlop for more details.
34 printf "%.2f", 10/3;
36 my $number = sprintf "%.2f", 10/3;
38 Why is int() broken?
40 Your "int()" is most probably working just fine. It's the numbers that
41 aren't quite what you think.
42 First, see the answer to "Why am I getting long decimals (eg,
44 19.9499999999999) instead of the numbers I should be getting (eg,
45 19.95)?".
46 For example, this
48 print int(0.6/0.2-2), "\n";
50 will in most computers print 0, not 1, because even such simple numbers
52 as 0.6 and 0.2 cannot be presented exactly by floating-point numbers.
53 What you think in the above as 'three' is really more like
54 2.9999999999999995559.
55 Why isn't my octal data interpreted correctly?
57 (contributed by brian d foy)
58 You're probably trying to convert a string to a number, which Perl only
60 converts as a decimal number. When Perl converts a string to a number,
61 it ignores leading spaces and zeroes, then assumes the rest of the
62 digits are in base 10:
63 my $string = '0644';
65 print $string + 0; # prints 644
67 print $string + 44; # prints 688, certainly not octal!
69 This problem usually involves one of the Perl built-ins that has the
71 same name a Unix command that uses octal numbers as arguments on the
72 command line. In this example, "chmod" on the command line knows that
73 its first argument is octal because that's what it does:
74 %prompt> chmod 644 file
76 If you want to use the same literal digits (644) in Perl, you have to
78 tell Perl to treat them as octal numbers either by prefixing the digits
79 with a 0 or using "oct":
80 chmod( 0644, $filename ); # right, has leading zero
82 chmod( oct(644), $filename ); # also correct
83 The problem comes in when you take your numbers from something that
85 Perl thinks is a string, such as a command line argument in @ARGV:
86 chmod( $ARGV[0], $filename ); # wrong, even if "0644"
88 chmod( oct($ARGV[0]), $filename ); # correct, treat string as octal
90 You can always check the value you're using by printing it in octal
92 notation to ensure it matches what you think it should be. Print it in
93 octal and decimal format:
94 printf "0%o %d", $number, $number;
96 Does Perl have a round() function? What about ceil() and floor()? Trig
98 functions?
99 Remember that "int()" merely truncates toward 0. For rounding to a
100 certain number of digits, "sprintf()" or "printf()" is usually the
101 easiest route.
102 printf("%.3f", 3.1415926535); # prints 3.142
104 The POSIX module (part of the standard Perl distribution) implements
106 "ceil()", "floor()", and a number of other mathematical and
107 trigonometric functions.
108 use POSIX;
110 my $ceil = ceil(3.5); # 4
111 my $floor = floor(3.5); # 3
112 In 5.000 to 5.003 perls, trigonometry was done in the Math::Complex
114 module. With 5.004, the Math::Trig module (part of the standard Perl
115 distribution) implements the trigonometric functions. Internally it
116 uses the Math::Complex module and some functions can break out from the
117 real axis into the complex plane, for example the inverse sine of 2.
118 Rounding in financial applications can have serious implications, and
120 the rounding method used should be specified precisely. In these cases,
121 it probably pays not to trust whichever system of rounding is being
122 used by Perl, but instead to implement the rounding function you need
123 yourself.
124 To see why, notice how you'll still have an issue on half-way-point
126 alternation:
127 for (my $i = -5; $i <= 5; $i += 0.5) { printf "%.0f ",$i }
129 -5 -4 -4 -4 -3 -2 -2 -2 -1 -0 0 0 1 2 2 2 3 4 4 4 5
131 Don't blame Perl. It's the same as in C. IEEE says we have to do this.
133 Perl numbers whose absolute values are integers under 2**31 (on 32-bit
134 machines) will work pretty much like mathematical integers. Other
135 numbers are not guaranteed.
136 How do I convert between numeric representations/bases/radixes?
138 As always with Perl there is more than one way to do it. Below are a
139 few examples of approaches to making common conversions between number
140 representations. This is intended to be representational rather than
141 exhaustive.
142 Some of the examples later in perlfaq4 use the Bit::Vector module from
144 CPAN. The reason you might choose Bit::Vector over the perl built-in
145 functions is that it works with numbers of ANY size, that it is
146 optimized for speed on some operations, and for at least some
147 programmers the notation might be familiar.
148 How do I convert hexadecimal into decimal
150 Using perl's built in conversion of "0x" notation:
151 my $dec = 0xDEADBEEF;
153 Using the "hex" function:
155 my $dec = hex("DEADBEEF");
157 Using "pack":
159 my $dec = unpack("N", pack("H8", substr("0" x 8 . "DEADBEEF", -8)));
161 Using the CPAN module "Bit::Vector":
163 use Bit::Vector;
165 my $vec = Bit::Vector->new_Hex(32, "DEADBEEF");
166 my $dec = $vec->to_Dec();
167 How do I convert from decimal to hexadecimal
169 Using "sprintf":
170 my $hex = sprintf("%X", 3735928559); # upper case A-F
172 my $hex = sprintf("%x", 3735928559); # lower case a-f
173 Using "unpack":
175 my $hex = unpack("H*", pack("N", 3735928559));
177 Using Bit::Vector:
179 use Bit::Vector;
181 my $vec = Bit::Vector->new_Dec(32, -559038737);
182 my $hex = $vec->to_Hex();
183 And Bit::Vector supports odd bit counts:
185 use Bit::Vector;
187 my $vec = Bit::Vector->new_Dec(33, 3735928559);
188 $vec->Resize(32); # suppress leading 0 if unwanted
189 my $hex = $vec->to_Hex();
190 How do I convert from octal to decimal
192 Using Perl's built in conversion of numbers with leading zeros:
193 my $dec = 033653337357; # note the leading 0!
195 Using the "oct" function:
197 my $dec = oct("33653337357");
199 Using Bit::Vector:
201 use Bit::Vector;
203 my $vec = Bit::Vector->new(32);
204 $vec->Chunk_List_Store(3, split(//, reverse "33653337357"));
205 my $dec = $vec->to_Dec();
206 How do I convert from decimal to octal
208 Using "sprintf":
209 my $oct = sprintf("%o", 3735928559);
211 Using Bit::Vector:
213 use Bit::Vector;
215 my $vec = Bit::Vector->new_Dec(32, -559038737);
216 my $oct = reverse join('', $vec->Chunk_List_Read(3));
217 How do I convert from binary to decimal
219 Perl 5.6 lets you write binary numbers directly with the "0b"
220 notation:
221 my $number = 0b10110110;
223 Using "oct":
225 my $input = "10110110";
227 my $decimal = oct( "0b$input" );
228 Using "pack" and "ord":
230 my $decimal = ord(pack('B8', '10110110'));
232 Using "pack" and "unpack" for larger strings:
234 my $int = unpack("N", pack("B32",
236 substr("0" x 32 . "11110101011011011111011101111", -32)));
237 my $dec = sprintf("%d", $int);
238 # substr() is used to left-pad a 32-character string with zeros.
240 Using Bit::Vector:
242 my $vec = Bit::Vector->new_Bin(32, "11011110101011011011111011101111");
244 my $dec = $vec->to_Dec();
245 How do I convert from decimal to binary
247 Using "sprintf" (perl 5.6+):
248 my $bin = sprintf("%b", 3735928559);
250 Using "unpack":
252 my $bin = unpack("B*", pack("N", 3735928559));
254 Using Bit::Vector:
256 use Bit::Vector;
258 my $vec = Bit::Vector->new_Dec(32, -559038737);
259 my $bin = $vec->to_Bin();
260 The remaining transformations (e.g. hex -> oct, bin -> hex, etc.)
262 are left as an exercise to the inclined reader.
263 Why doesn't & work the way I want it to?
265 The behavior of binary arithmetic operators depends on whether they're
266 used on numbers or strings. The operators treat a string as a series of
267 bits and work with that (the string "3" is the bit pattern 00110011).
268 The operators work with the binary form of a number (the number 3 is
269 treated as the bit pattern 00000011).
270 So, saying "11 & 3" performs the "and" operation on numbers (yielding
272 3). Saying "11" & "3" performs the "and" operation on strings (yielding
273 "1").
274 Most problems with "&" and "|" arise because the programmer thinks they
276 have a number but really it's a string or vice versa. To avoid this,
277 stringify the arguments explicitly (using "" or "qq()") or convert them
278 to numbers explicitly (using "0+$arg"). The rest arise because the
279 programmer says:
280 if ("\020\020" & "\101\101") {
282 # ...
283 }
284 but a string consisting of two null bytes (the result of "\020\020" &
286 "\101\101") is not a false value in Perl. You need:
287 if ( ("\020\020" & "\101\101") !~ /[^\000]/) {
289 # ...
290 }
291 How do I multiply matrices?
293 Use the Math::Matrix or Math::MatrixReal modules (available from CPAN)
294 or the PDL extension (also available from CPAN).
295 How do I perform an operation on a series of integers?
297 To call a function on each element in an array, and collect the
298 results, use:
299 my @results = map { my_func($_) } @array;
301 For example:
303 my @triple = map { 3 * $_ } @single;
305 To call a function on each element of an array, but ignore the results:
307 foreach my $iterator (@array) {
309 some_func($iterator);
310 }
311 To call a function on each integer in a (small) range, you can use:
313 my @results = map { some_func($_) } (5 .. 25);
315 but you should be aware that in this form, the ".." operator creates a
317 list of all integers in the range, which can take a lot of memory for
318 large ranges. However, the problem does not occur when using ".."
319 within a "for" loop, because in that case the range operator is
320 optimized to iterate over the range, without creating the entire list.
321 So
322 my @results = ();
324 for my $i (5 .. 500_005) {
325 push(@results, some_func($i));
326 }
327 or even
329 push(@results, some_func($_)) for 5 .. 500_005;
331 will not create an intermediate list of 500,000 integers.
333 How can I output Roman numerals?
335 Get the <http://www.cpan.org/modules/by-module/Roman> module.
336 Why aren't my random numbers random?
338 If you're using a version of Perl before 5.004, you must call "srand"
339 once at the start of your program to seed the random number generator.
340 BEGIN { srand() if $] < 5.004 }
342 5.004 and later automatically call "srand" at the beginning. Don't call
344 "srand" more than once--you make your numbers less random, rather than
345 more.
346 Computers are good at being predictable and bad at being random
348 (despite appearances caused by bugs in your programs :-). The random
349 article in the "Far More Than You Ever Wanted To Know" collection in
350 <http://www.cpan.org/misc/olddoc/FMTEYEWTK.tgz>, courtesy of Tom
351 Phoenix, talks more about this. John von Neumann said, "Anyone who
352 attempts to generate random numbers by deterministic means is, of
353 course, living in a state of sin."
354 Perl relies on the underlying system for the implementation of "rand"
356 and "srand"; on some systems, the generated numbers are not random
357 enough (especially on Windows : see
358 <http://www.perlmonks.org/?node_id=803632>). Several CPAN modules in
359 the "Math" namespace implement better pseudorandom generators; see for
360 example Math::Random::MT ("Mersenne Twister", fast), or
361 Math::TrulyRandom (uses the imperfections in the system's timer to
362 generate random numbers, which is rather slow). More algorithms for
363 random numbers are described in "Numerical Recipes in C" at
364 <http://www.nr.com/>
365 How do I get a random number between X and Y?
367 To get a random number between two values, you can use the "rand()"
368 built-in to get a random number between 0 and 1. From there, you shift
369 that into the range that you want.
370 "rand($x)" returns a number such that "0 <= rand($x) < $x". Thus what
372 you want to have perl figure out is a random number in the range from 0
373 to the difference between your X and Y.
374 That is, to get a number between 10 and 15, inclusive, you want a
376 random number between 0 and 5 that you can then add to 10.
377 my $number = 10 + int rand( 15-10+1 ); # ( 10,11,12,13,14, or 15 )
379 Hence you derive the following simple function to abstract that. It
381 selects a random integer between the two given integers (inclusive).
382 For example: "random_int_between(50,120)".
383 sub random_int_between {
385 my($min, $max) = @_;
386 # Assumes that the two arguments are integers themselves!
387 return $min if $min == $max;
388 ($min, $max) = ($max, $min) if $min > $max;
389 return $min + int rand(1 + $max - $min);
390 }
391
393 How do I find the day or week of the year?
394 The day of the year is in the list returned by the "localtime"
395 function. Without an argument "localtime" uses the current time.
396 my $day_of_year = (localtime)[7];
398 The POSIX module can also format a date as the day of the year or week
400 of the year.
401 use POSIX qw/strftime/;
403 my $day_of_year = strftime "%j", localtime;
404 my $week_of_year = strftime "%W", localtime;
405 To get the day of year for any date, use POSIX's "mktime" to get a time
407 in epoch seconds for the argument to "localtime".
408 use POSIX qw/mktime strftime/;
410 my $week_of_year = strftime "%W",
411 localtime( mktime( 0, 0, 0, 18, 11, 87 ) );
412 You can also use Time::Piece, which comes with Perl and provides a
414 "localtime" that returns an object:
415 use Time::Piece;
417 my $day_of_year = localtime->yday;
418 my $week_of_year = localtime->week;
419 The Date::Calc module provides two functions to calculate these, too:
421 use Date::Calc;
423 my $day_of_year = Day_of_Year( 1987, 12, 18 );
424 my $week_of_year = Week_of_Year( 1987, 12, 18 );
425 How do I find the current century or millennium?
427 Use the following simple functions:
428 sub get_century {
430 return int((((localtime(shift || time))[5] + 1999))/100);
431 }
432 sub get_millennium {
434 return 1+int((((localtime(shift || time))[5] + 1899))/1000);
435 }
436 On some systems, the POSIX module's "strftime()" function has been
438 extended in a non-standard way to use a %C format, which they sometimes
439 claim is the "century". It isn't, because on most such systems, this is
440 only the first two digits of the four-digit year, and thus cannot be
441 used to determine reliably the current century or millennium.
442 How can I compare two dates and find the difference?
444 (contributed by brian d foy)
445 You could just store all your dates as a number and then subtract.
447 Life isn't always that simple though.
448 The Time::Piece module, which comes with Perl, replaces localtime with
450 a version that returns an object. It also overloads the comparison
451 operators so you can compare them directly:
452 use Time::Piece;
454 my $date1 = localtime( $some_time );
455 my $date2 = localtime( $some_other_time );
456 if( $date1 < $date2 ) {
458 print "The date was in the past\n";
459 }
460 You can also get differences with a subtraction, which returns a
462 Time::Seconds object:
463 my $date_diff = $date1 - $date2;
465 print "The difference is ", $date_diff->days, " days\n";
466 If you want to work with formatted dates, the Date::Manip, Date::Calc,
468 or DateTime modules can help you.
469 How can I take a string and turn it into epoch seconds?
471 If it's a regular enough string that it always has the same format, you
472 can split it up and pass the parts to "timelocal" in the standard
473 Time::Local module. Otherwise, you should look into the Date::Calc,
474 Date::Parse, and Date::Manip modules from CPAN.
475 How can I find the Julian Day?
477 (contributed by brian d foy and Dave Cross)
478 You can use the Time::Piece module, part of the Standard Library, which
480 can convert a date/time to a Julian Day:
481 $ perl -MTime::Piece -le 'print localtime->julian_day'
483 2455607.7959375
484 Or the modified Julian Day:
486 $ perl -MTime::Piece -le 'print localtime->mjd'
488 55607.2961226851
489 Or even the day of the year (which is what some people think of as a
491 Julian day):
492 $ perl -MTime::Piece -le 'print localtime->yday'
494 45
495 You can also do the same things with the DateTime module:
497 $ perl -MDateTime -le'print DateTime->today->jd'
499 2453401.5
500 $ perl -MDateTime -le'print DateTime->today->mjd'
501 53401
502 $ perl -MDateTime -le'print DateTime->today->doy'
503 31
504 You can use the Time::JulianDay module available on CPAN. Ensure that
506 you really want to find a Julian day, though, as many people have
507 different ideas about Julian days (see
508 <http://www.hermetic.ch/cal_stud/jdn.htm> for instance):
509 $ perl -MTime::JulianDay -le 'print local_julian_day( time )'
511 55608
512 How do I find yesterday's date?
514 (contributed by brian d foy)
515 To do it correctly, you can use one of the "Date" modules since they
517 work with calendars instead of times. The DateTime module makes it
518 simple, and give you the same time of day, only the day before, despite
519 daylight saving time changes:
520 use DateTime;
522 my $yesterday = DateTime->now->subtract( days => 1 );
524 print "Yesterday was $yesterday\n";
526 You can also use the Date::Calc module using its "Today_and_Now"
528 function.
529 use Date::Calc qw( Today_and_Now Add_Delta_DHMS );
531 my @date_time = Add_Delta_DHMS( Today_and_Now(), -1, 0, 0, 0 );
533 print "@date_time\n";
535 Most people try to use the time rather than the calendar to figure out
537 dates, but that assumes that days are twenty-four hours each. For most
538 people, there are two days a year when they aren't: the switch to and
539 from summer time throws this off. For example, the rest of the
540 suggestions will be wrong sometimes:
541 Starting with Perl 5.10, Time::Piece and Time::Seconds are part of the
543 standard distribution, so you might think that you could do something
544 like this:
545 use Time::Piece;
547 use Time::Seconds;
548 my $yesterday = localtime() - ONE_DAY; # WRONG
550 print "Yesterday was $yesterday\n";
551 The Time::Piece module exports a new "localtime" that returns an
553 object, and Time::Seconds exports the "ONE_DAY" constant that is a set
554 number of seconds. This means that it always gives the time 24 hours
555 ago, which is not always yesterday. This can cause problems around the
556 end of daylight saving time when there's one day that is 25 hours long.
557 You have the same problem with Time::Local, which will give the wrong
559 answer for those same special cases:
560 # contributed by Gunnar Hjalmarsson
562 use Time::Local;
563 my $today = timelocal 0, 0, 12, ( localtime )[3..5];
564 my ($d, $m, $y) = ( localtime $today-86400 )[3..5]; # WRONG
565 printf "Yesterday: %d-%02d-%02d\n", $y+1900, $m+1, $d;
566 Does Perl have a Year 2000 or 2038 problem? Is Perl Y2K compliant?
568 (contributed by brian d foy)
569 Perl itself never had a Y2K problem, although that never stopped people
571 from creating Y2K problems on their own. See the documentation for
572 "localtime" for its proper use.
573 Starting with Perl 5.12, "localtime" and "gmtime" can handle dates past
575 03:14:08 January 19, 2038, when a 32-bit based time would overflow. You
576 still might get a warning on a 32-bit "perl":
577 % perl5.12 -E 'say scalar localtime( 0x9FFF_FFFFFFFF )'
579 Integer overflow in hexadecimal number at -e line 1.
580 Wed Nov 1 19:42:39 5576711
581 On a 64-bit "perl", you can get even larger dates for those really long
583 running projects:
584 % perl5.12 -E 'say scalar gmtime( 0x9FFF_FFFFFFFF )'
586 Thu Nov 2 00:42:39 5576711
587 You're still out of luck if you need to keep track of decaying protons
589 though.
590
592 How do I validate input?
593 (contributed by brian d foy)
594 There are many ways to ensure that values are what you expect or want
596 to accept. Besides the specific examples that we cover in the perlfaq,
597 you can also look at the modules with "Assert" and "Validate" in their
598 names, along with other modules such as Regexp::Common.
599 Some modules have validation for particular types of input, such as
601 Business::ISBN, Business::CreditCard, Email::Valid, and
602 Data::Validate::IP.
603 How do I unescape a string?
605 It depends just what you mean by "escape". URL escapes are dealt with
606 in perlfaq9. Shell escapes with the backslash ("\") character are
607 removed with
608 s/\\(.)/$1/g;
610 This won't expand "\n" or "\t" or any other special escapes.
612 How do I remove consecutive pairs of characters?
614 (contributed by brian d foy)
615 You can use the substitution operator to find pairs of characters (or
617 runs of characters) and replace them with a single instance. In this
618 substitution, we find a character in "(.)". The memory parentheses
619 store the matched character in the back-reference "\g1" and we use that
620 to require that the same thing immediately follow it. We replace that
621 part of the string with the character in $1.
622 s/(.)\g1/$1/g;
624 We can also use the transliteration operator, "tr///". In this example,
626 the search list side of our "tr///" contains nothing, but the "c"
627 option complements that so it contains everything. The replacement list
628 also contains nothing, so the transliteration is almost a no-op since
629 it won't do any replacements (or more exactly, replace the character
630 with itself). However, the "s" option squashes duplicated and
631 consecutive characters in the string so a character does not show up
632 next to itself
633 my $str = 'Haarlem'; # in the Netherlands
635 $str =~ tr///cs; # Now Harlem, like in New York
636 How do I expand function calls in a string?
638 (contributed by brian d foy)
639 This is documented in perlref, and although it's not the easiest thing
641 to read, it does work. In each of these examples, we call the function
642 inside the braces used to dereference a reference. If we have more than
643 one return value, we can construct and dereference an anonymous array.
644 In this case, we call the function in list context.
645 print "The time values are @{ [localtime] }.\n";
647 If we want to call the function in scalar context, we have to do a bit
649 more work. We can really have any code we like inside the braces, so we
650 simply have to end with the scalar reference, although how you do that
651 is up to you, and you can use code inside the braces. Note that the use
652 of parens creates a list context, so we need "scalar" to force the
653 scalar context on the function:
654 print "The time is ${\(scalar localtime)}.\n"
656 print "The time is ${ my $x = localtime; \$x }.\n";
658 If your function already returns a reference, you don't need to create
660 the reference yourself.
661 sub timestamp { my $t = localtime; \$t }
663 print "The time is ${ timestamp() }.\n";
665 The "Interpolation" module can also do a lot of magic for you. You can
667 specify a variable name, in this case "E", to set up a tied hash that
668 does the interpolation for you. It has several other methods to do this
669 as well.
670 use Interpolation E => 'eval';
672 print "The time values are $E{localtime()}.\n";
673 In most cases, it is probably easier to simply use string
675 concatenation, which also forces scalar context.
676 print "The time is " . localtime() . ".\n";
678 How do I find matching/nesting anything?
680 To find something between two single characters, a pattern like
681 "/x([^x]*)x/" will get the intervening bits in $1. For multiple ones,
682 then something more like "/alpha(.*?)omega/" would be needed. For
683 nested patterns and/or balanced expressions, see the so-called (?PARNO)
684 construct (available since perl 5.10). The CPAN module Regexp::Common
685 can help to build such regular expressions (see in particular
686 Regexp::Common::balanced and Regexp::Common::delimited).
687 More complex cases will require to write a parser, probably using a
689 parsing module from CPAN, like Regexp::Grammars, Parse::RecDescent,
690 Parse::Yapp, Text::Balanced, or Marpa::R2.
691 How do I reverse a string?
693 Use "reverse()" in scalar context, as documented in "reverse" in
694 perlfunc.
695 my $reversed = reverse $string;
697 How do I expand tabs in a string?
699 You can do it yourself:
700 1 while $string =~ s/\t+/' ' x (length($&) * 8 - length($`) % 8)/e;
702 Or you can just use the Text::Tabs module (part of the standard Perl
704 distribution).
705 use Text::Tabs;
707 my @expanded_lines = expand(@lines_with_tabs);
708 How do I reformat a paragraph?
710 Use Text::Wrap (part of the standard Perl distribution):
711 use Text::Wrap;
713 print wrap("\t", ' ', @paragraphs);
714 The paragraphs you give to Text::Wrap should not contain embedded
716 newlines. Text::Wrap doesn't justify the lines (flush-right).
717 Or use the CPAN module Text::Autoformat. Formatting files can be easily
719 done by making a shell alias, like so:
720 alias fmt="perl -i -MText::Autoformat -n0777 \
722 -e 'print autoformat $_, {all=>1}' $*"
723 See the documentation for Text::Autoformat to appreciate its many
725 capabilities.
726 How can I access or change N characters of a string?
728 You can access the first characters of a string with substr(). To get
729 the first character, for example, start at position 0 and grab the
730 string of length 1.
731 my $string = "Just another Perl Hacker";
733 my $first_char = substr( $string, 0, 1 ); # 'J'
734 To change part of a string, you can use the optional fourth argument
736 which is the replacement string.
737 substr( $string, 13, 4, "Perl 5.8.0" );
739 You can also use substr() as an lvalue.
741 substr( $string, 13, 4 ) = "Perl 5.8.0";
743 How do I change the Nth occurrence of something?
745 You have to keep track of N yourself. For example, let's say you want
746 to change the fifth occurrence of "whoever" or "whomever" into
747 "whosoever" or "whomsoever", case insensitively. These all assume that
748 $_ contains the string to be altered.
749 $count = 0;
751 s{((whom?)ever)}{
752 ++$count == 5 # is it the 5th?
753 ? "${2}soever" # yes, swap
754 : $1 # renege and leave it there
755 }ige;
756 In the more general case, you can use the "/g" modifier in a "while"
758 loop, keeping count of matches.
759 $WANT = 3;
761 $count = 0;
762 $_ = "One fish two fish red fish blue fish";
763 while (/(\w+)\s+fish\b/gi) {
764 if (++$count == $WANT) {
765 print "The third fish is a $1 one.\n";
766 }
767 }
768 That prints out: "The third fish is a red one." You can also use a
770 repetition count and repeated pattern like this:
771 /(?:\w+\s+fish\s+){2}(\w+)\s+fish/i;
773 How can I count the number of occurrences of a substring within a string?
775 There are a number of ways, with varying efficiency. If you want a
776 count of a certain single character (X) within a string, you can use
777 the "tr///" function like so:
778 my $string = "ThisXlineXhasXsomeXx'sXinXit";
780 my $count = ($string =~ tr/X//);
781 print "There are $count X characters in the string";
782 This is fine if you are just looking for a single character. However,
784 if you are trying to count multiple character substrings within a
785 larger string, "tr///" won't work. What you can do is wrap a while()
786 loop around a global pattern match. For example, let's count negative
787 integers:
788 my $string = "-9 55 48 -2 23 -76 4 14 -44";
790 my $count = 0;
791 while ($string =~ /-\d+/g) { $count++ }
792 print "There are $count negative numbers in the string";
793 Another version uses a global match in list context, then assigns the
795 result to a scalar, producing a count of the number of matches.
796 my $count = () = $string =~ /-\d+/g;
798 How do I capitalize all the words on one line?
800 (contributed by brian d foy)
801 Damian Conway's Text::Autoformat handles all of the thinking for you.
803 use Text::Autoformat;
805 my $x = "Dr. Strangelove or: How I Learned to Stop ".
806 "Worrying and Love the Bomb";
807 print $x, "\n";
809 for my $style (qw( sentence title highlight )) {
810 print autoformat($x, { case => $style }), "\n";
811 }
812 How do you want to capitalize those words?
814 FRED AND BARNEY'S LODGE # all uppercase
816 Fred And Barney's Lodge # title case
817 Fred and Barney's Lodge # highlight case
818 It's not as easy a problem as it looks. How many words do you think are
820 in there? Wait for it... wait for it.... If you answered 5 you're
821 right. Perl words are groups of "\w+", but that's not what you want to
822 capitalize. How is Perl supposed to know not to capitalize that "s"
823 after the apostrophe? You could try a regular expression:
824 $string =~ s/ (
826 (^\w) #at the beginning of the line
827 | # or
828 (\s\w) #preceded by whitespace
829 )
830 /\U$1/xg;
831 $string =~ s/([\w']+)/\u\L$1/g;
833 Now, what if you don't want to capitalize that "and"? Just use
835 Text::Autoformat and get on with the next problem. :)
836 How can I split a [character]-delimited string except when inside
838 [character]?
839 Several modules can handle this sort of parsing--Text::Balanced,
840 Text::CSV, Text::CSV_XS, and Text::ParseWords, among others.
841 Take the example case of trying to split a string that is comma-
843 separated into its different fields. You can't use "split(/,/)" because
844 you shouldn't split if the comma is inside quotes. For example, take a
845 data line like this:
846 SAR001,"","Cimetrix, Inc","Bob Smith","CAM",N,8,1,0,7,"Error, Core Dumped"
848 Due to the restriction of the quotes, this is a fairly complex problem.
850 Thankfully, we have Jeffrey Friedl, author of Mastering Regular
851 Expressions, to handle these for us. He suggests (assuming your string
852 is contained in $text):
853 my @new = ();
855 push(@new, $+) while $text =~ m{
856 "([^\"\\]*(?:\\.[^\"\\]*)*)",? # groups the phrase inside the quotes
857 | ([^,]+),?
858 | ,
859 }gx;
860 push(@new, undef) if substr($text,-1,1) eq ',';
861 If you want to represent quotation marks inside a quotation-mark-
863 delimited field, escape them with backslashes (eg, "like \"this\"".
864 Alternatively, the Text::ParseWords module (part of the standard Perl
866 distribution) lets you say:
867 use Text::ParseWords;
869 @new = quotewords(",", 0, $text);
870 For parsing or generating CSV, though, using Text::CSV rather than
872 implementing it yourself is highly recommended; you'll save yourself
873 odd bugs popping up later by just using code which has already been
874 tried and tested in production for years.
875 How do I strip blank space from the beginning/end of a string?
877 (contributed by brian d foy)
878 A substitution can do this for you. For a single line, you want to
880 replace all the leading or trailing whitespace with nothing. You can do
881 that with a pair of substitutions:
882 s/^\s+//;
884 s/\s+$//;
885 You can also write that as a single substitution, although it turns out
887 the combined statement is slower than the separate ones. That might not
888 matter to you, though:
889 s/^\s+|\s+$//g;
891 In this regular expression, the alternation matches either at the
893 beginning or the end of the string since the anchors have a lower
894 precedence than the alternation. With the "/g" flag, the substitution
895 makes all possible matches, so it gets both. Remember, the trailing
896 newline matches the "\s+", and the "$" anchor can match to the
897 absolute end of the string, so the newline disappears too. Just add the
898 newline to the output, which has the added benefit of preserving
899 "blank" (consisting entirely of whitespace) lines which the "^\s+"
900 would remove all by itself:
901 while( <> ) {
903 s/^\s+|\s+$//g;
904 print "$_\n";
905 }
906 For a multi-line string, you can apply the regular expression to each
908 logical line in the string by adding the "/m" flag (for "multi-line").
909 With the "/m" flag, the "$" matches before an embedded newline, so it
910 doesn't remove it. This pattern still removes the newline at the end of
911 the string:
912 $string =~ s/^\s+|\s+$//gm;
914 Remember that lines consisting entirely of whitespace will disappear,
916 since the first part of the alternation can match the entire string and
917 replace it with nothing. If you need to keep embedded blank lines, you
918 have to do a little more work. Instead of matching any whitespace
919 (since that includes a newline), just match the other whitespace:
920 $string =~ s/^[\t\f ]+|[\t\f ]+$//mg;
922 How do I pad a string with blanks or pad a number with zeroes?
924 In the following examples, $pad_len is the length to which you wish to
925 pad the string, $text or $num contains the string to be padded, and
926 $pad_char contains the padding character. You can use a single
927 character string constant instead of the $pad_char variable if you know
928 what it is in advance. And in the same way you can use an integer in
929 place of $pad_len if you know the pad length in advance.
930 The simplest method uses the "sprintf" function. It can pad on the left
932 or right with blanks and on the left with zeroes and it will not
933 truncate the result. The "pack" function can only pad strings on the
934 right with blanks and it will truncate the result to a maximum length
935 of $pad_len.
936 # Left padding a string with blanks (no truncation):
938 my $padded = sprintf("%${pad_len}s", $text);
939 my $padded = sprintf("%*s", $pad_len, $text); # same thing
940 # Right padding a string with blanks (no truncation):
942 my $padded = sprintf("%-${pad_len}s", $text);
943 my $padded = sprintf("%-*s", $pad_len, $text); # same thing
944 # Left padding a number with 0 (no truncation):
946 my $padded = sprintf("%0${pad_len}d", $num);
947 my $padded = sprintf("%0*d", $pad_len, $num); # same thing
948 # Right padding a string with blanks using pack (will truncate):
950 my $padded = pack("A$pad_len",$text);
951 If you need to pad with a character other than blank or zero you can
953 use one of the following methods. They all generate a pad string with
954 the "x" operator and combine that with $text. These methods do not
955 truncate $text.
956 Left and right padding with any character, creating a new string:
958 my $padded = $pad_char x ( $pad_len - length( $text ) ) . $text;
960 my $padded = $text . $pad_char x ( $pad_len - length( $text ) );
961 Left and right padding with any character, modifying $text directly:
963 substr( $text, 0, 0 ) = $pad_char x ( $pad_len - length( $text ) );
965 $text .= $pad_char x ( $pad_len - length( $text ) );
966 How do I extract selected columns from a string?
968 (contributed by brian d foy)
969 If you know the columns that contain the data, you can use "substr" to
971 extract a single column.
972 my $column = substr( $line, $start_column, $length );
974 You can use "split" if the columns are separated by whitespace or some
976 other delimiter, as long as whitespace or the delimiter cannot appear
977 as part of the data.
978 my $line = ' fred barney betty ';
980 my @columns = split /\s+/, $line;
981 # ( '', 'fred', 'barney', 'betty' );
982 my $line = 'fred||barney||betty';
984 my @columns = split /\|/, $line;
985 # ( 'fred', '', 'barney', '', 'betty' );
986 If you want to work with comma-separated values, don't do this since
988 that format is a bit more complicated. Use one of the modules that
989 handle that format, such as Text::CSV, Text::CSV_XS, or Text::CSV_PP.
990 If you want to break apart an entire line of fixed columns, you can use
992 "unpack" with the A (ASCII) format. By using a number after the format
993 specifier, you can denote the column width. See the "pack" and "unpack"
994 entries in perlfunc for more details.
995 my @fields = unpack( $line, "A8 A8 A8 A16 A4" );
997 Note that spaces in the format argument to "unpack" do not denote
999 literal spaces. If you have space separated data, you may want "split"
1000 instead.
1001 How do I find the soundex value of a string?
1003 (contributed by brian d foy)
1004 You can use the "Text::Soundex" module. If you want to do fuzzy or
1006 close matching, you might also try the String::Approx, and
1007 Text::Metaphone, and Text::DoubleMetaphone modules.
1008 How can I expand variables in text strings?
1010 (contributed by brian d foy)
1011 If you can avoid it, don't, or if you can use a templating system, such
1013 as Text::Template or Template Toolkit, do that instead. You might even
1014 be able to get the job done with "sprintf" or "printf":
1015 my $string = sprintf 'Say hello to %s and %s', $foo, $bar;
1017 However, for the one-off simple case where I don't want to pull out a
1019 full templating system, I'll use a string that has two Perl scalar
1020 variables in it. In this example, I want to expand $foo and $bar to
1021 their variable's values:
1022 my $foo = 'Fred';
1024 my $bar = 'Barney';
1025 $string = 'Say hello to $foo and $bar';
1026 One way I can do this involves the substitution operator and a double
1028 "/e" flag. The first "/e" evaluates $1 on the replacement side and
1029 turns it into $foo. The second /e starts with $foo and replaces it with
1030 its value. $foo, then, turns into 'Fred', and that's finally what's
1031 left in the string:
1032 $string =~ s/(\$\w+)/$1/eeg; # 'Say hello to Fred and Barney'
1034 The "/e" will also silently ignore violations of strict, replacing
1036 undefined variable names with the empty string. Since I'm using the
1037 "/e" flag (twice even!), I have all of the same security problems I
1038 have with "eval" in its string form. If there's something odd in $foo,
1039 perhaps something like "@{[ system "rm -rf /" ]}", then I could get
1040 myself in trouble.
1041 To get around the security problem, I could also pull the values from a
1043 hash instead of evaluating variable names. Using a single "/e", I can
1044 check the hash to ensure the value exists, and if it doesn't, I can
1045 replace the missing value with a marker, in this case "???" to signal
1046 that I missed something:
1047 my $string = 'This has $foo and $bar';
1049 my %Replacements = (
1051 foo => 'Fred',
1052 );
1053 # $string =~ s/\$(\w+)/$Replacements{$1}/g;
1055 $string =~ s/\$(\w+)/
1056 exists $Replacements{$1} ? $Replacements{$1} : '???'
1057 /eg;
1058 print $string;
1060 What's wrong with always quoting "$vars"?
1062 The problem is that those double-quotes force stringification--coercing
1063 numbers and references into strings--even when you don't want them to
1064 be strings. Think of it this way: double-quote expansion is used to
1065 produce new strings. If you already have a string, why do you need
1066 more?
1067 If you get used to writing odd things like these:
1069 print "$var"; # BAD
1071 my $new = "$old"; # BAD
1072 somefunc("$var"); # BAD
1073 You'll be in trouble. Those should (in 99.8% of the cases) be the
1075 simpler and more direct:
1076 print $var;
1078 my $new = $old;
1079 somefunc($var);
1080 Otherwise, besides slowing you down, you're going to break code when
1082 the thing in the scalar is actually neither a string nor a number, but
1083 a reference:
1084 func(\@array);
1086 sub func {
1087 my $aref = shift;
1088 my $oref = "$aref"; # WRONG
1089 }
1090 You can also get into subtle problems on those few operations in Perl
1092 that actually do care about the difference between a string and a
1093 number, such as the magical "++" autoincrement operator or the
1094 syscall() function.
1095 Stringification also destroys arrays.
1097 my @lines = `command`;
1099 print "@lines"; # WRONG - extra blanks
1100 print @lines; # right
1101 Why don't my <<HERE documents work?
1103 Here documents are found in perlop. Check for these three things:
1104 There must be no space after the << part.
1106 There (probably) should be a semicolon at the end of the opening token
1107 You can't (easily) have any space in front of the tag.
1108 There needs to be at least a line separator after the end token.
1109 If you want to indent the text in the here document, you can do this:
1111 # all in one
1113 (my $VAR = <<HERE_TARGET) =~ s/^\s+//gm;
1114 your text
1115 goes here
1116 HERE_TARGET
1117 But the HERE_TARGET must still be flush against the margin. If you
1119 want that indented also, you'll have to quote in the indentation.
1120 (my $quote = <<' FINIS') =~ s/^\s+//gm;
1122 ...we will have peace, when you and all your works have
1123 perished--and the works of your dark master to whom you
1124 would deliver us. You are a liar, Saruman, and a corrupter
1125 of men's hearts. --Theoden in /usr/src/perl/taint.c
1126 FINIS
1127 $quote =~ s/\s+--/\n--/;
1128 A nice general-purpose fixer-upper function for indented here documents
1130 follows. It expects to be called with a here document as its argument.
1131 It looks to see whether each line begins with a common substring, and
1132 if so, strips that substring off. Otherwise, it takes the amount of
1133 leading whitespace found on the first line and removes that much off
1134 each subsequent line.
1135 sub fix {
1137 local $_ = shift;
1138 my ($white, $leader); # common whitespace and common leading string
1139 if (/^\s*(?:([^\w\s]+)(\s*).*\n)(?:\s*\g1\g2?.*\n)+$/) {
1140 ($white, $leader) = ($2, quotemeta($1));
1141 } else {
1142 ($white, $leader) = (/^(\s+)/, '');
1143 }
1144 s/^\s*?$leader(?:$white)?//gm;
1145 return $_;
1146 }
1147 This works with leading special strings, dynamically determined:
1149 my $remember_the_main = fix<<' MAIN_INTERPRETER_LOOP';
1151 @@@ int
1152 @@@ runops() {
1153 @@@ SAVEI32(runlevel);
1154 @@@ runlevel++;
1155 @@@ while ( op = (*op->op_ppaddr)() );
1156 @@@ TAINT_NOT;
1157 @@@ return 0;
1158 @@@ }
1159 MAIN_INTERPRETER_LOOP
1160 Or with a fixed amount of leading whitespace, with remaining
1162 indentation correctly preserved:
1163 my $poem = fix<<EVER_ON_AND_ON;
1165 Now far ahead the Road has gone,
1166 And I must follow, if I can,
1167 Pursuing it with eager feet,
1168 Until it joins some larger way
1169 Where many paths and errands meet.
1170 And whither then? I cannot say.
1171 --Bilbo in /usr/src/perl/pp_ctl.c
1172 EVER_ON_AND_ON
1173 Beginning with Perl version 5.26, a much simpler and cleaner way to
1175 write indented here documents has been added to the language: the tilde
1176 (~) modifier. See "Indented Here-docs" in perlop for details.
1177
1179 What is the difference between a list and an array?
1180 (contributed by brian d foy)
1181 A list is a fixed collection of scalars. An array is a variable that
1183 holds a variable collection of scalars. An array can supply its
1184 collection for list operations, so list operations also work on arrays:
1185 # slices
1187 ( 'dog', 'cat', 'bird' )[2,3];
1188 @animals[2,3];
1189 # iteration
1191 foreach ( qw( dog cat bird ) ) { ... }
1192 foreach ( @animals ) { ... }
1193 my @three = grep { length == 3 } qw( dog cat bird );
1195 my @three = grep { length == 3 } @animals;
1196 # supply an argument list
1198 wash_animals( qw( dog cat bird ) );
1199 wash_animals( @animals );
1200 Array operations, which change the scalars, rearrange them, or add or
1202 subtract some scalars, only work on arrays. These can't work on a list,
1203 which is fixed. Array operations include "shift", "unshift", "push",
1204 "pop", and "splice".
1205 An array can also change its length:
1207 $#animals = 1; # truncate to two elements
1209 $#animals = 10000; # pre-extend to 10,001 elements
1210 You can change an array element, but you can't change a list element:
1212 $animals[0] = 'Rottweiler';
1214 qw( dog cat bird )[0] = 'Rottweiler'; # syntax error!
1215 foreach ( @animals ) {
1217 s/^d/fr/; # works fine
1218 }
1219 foreach ( qw( dog cat bird ) ) {
1221 s/^d/fr/; # Error! Modification of read only value!
1222 }
1223 However, if the list element is itself a variable, it appears that you
1225 can change a list element. However, the list element is the variable,
1226 not the data. You're not changing the list element, but something the
1227 list element refers to. The list element itself doesn't change: it's
1228 still the same variable.
1229 You also have to be careful about context. You can assign an array to a
1231 scalar to get the number of elements in the array. This only works for
1232 arrays, though:
1233 my $count = @animals; # only works with arrays
1235 If you try to do the same thing with what you think is a list, you get
1237 a quite different result. Although it looks like you have a list on the
1238 righthand side, Perl actually sees a bunch of scalars separated by a
1239 comma:
1240 my $scalar = ( 'dog', 'cat', 'bird' ); # $scalar gets bird
1242 Since you're assigning to a scalar, the righthand side is in scalar
1244 context. The comma operator (yes, it's an operator!) in scalar context
1245 evaluates its lefthand side, throws away the result, and evaluates it's
1246 righthand side and returns the result. In effect, that list-lookalike
1247 assigns to $scalar it's rightmost value. Many people mess this up
1248 because they choose a list-lookalike whose last element is also the
1249 count they expect:
1250 my $scalar = ( 1, 2, 3 ); # $scalar gets 3, accidentally
1252 What is the difference between $array[1] and @array[1]?
1254 (contributed by brian d foy)
1255 The difference is the sigil, that special character in front of the
1257 array name. The "$" sigil means "exactly one item", while the "@" sigil
1258 means "zero or more items". The "$" gets you a single scalar, while the
1259 "@" gets you a list.
1260 The confusion arises because people incorrectly assume that the sigil
1262 denotes the variable type.
1263 The $array[1] is a single-element access to the array. It's going to
1265 return the item in index 1 (or undef if there is no item there). If
1266 you intend to get exactly one element from the array, this is the form
1267 you should use.
1268 The @array[1] is an array slice, although it has only one index. You
1270 can pull out multiple elements simultaneously by specifying additional
1271 indices as a list, like @array[1,4,3,0].
1272 Using a slice on the lefthand side of the assignment supplies list
1274 context to the righthand side. This can lead to unexpected results.
1275 For instance, if you want to read a single line from a filehandle,
1276 assigning to a scalar value is fine:
1277 $array[1] = <STDIN>;
1279 However, in list context, the line input operator returns all of the
1281 lines as a list. The first line goes into @array[1] and the rest of the
1282 lines mysteriously disappear:
1283 @array[1] = <STDIN>; # most likely not what you want
1285 Either the "use warnings" pragma or the -w flag will warn you when you
1287 use an array slice with a single index.
1288 How can I remove duplicate elements from a list or array?
1290 (contributed by brian d foy)
1291 Use a hash. When you think the words "unique" or "duplicated", think
1293 "hash keys".
1294 If you don't care about the order of the elements, you could just
1296 create the hash then extract the keys. It's not important how you
1297 create that hash: just that you use "keys" to get the unique elements.
1298 my %hash = map { $_, 1 } @array;
1300 # or a hash slice: @hash{ @array } = ();
1301 # or a foreach: $hash{$_} = 1 foreach ( @array );
1302 my @unique = keys %hash;
1304 If you want to use a module, try the "uniq" function from
1306 List::MoreUtils. In list context it returns the unique elements,
1307 preserving their order in the list. In scalar context, it returns the
1308 number of unique elements.
1309 use List::MoreUtils qw(uniq);
1311 my @unique = uniq( 1, 2, 3, 4, 4, 5, 6, 5, 7 ); # 1,2,3,4,5,6,7
1313 my $unique = uniq( 1, 2, 3, 4, 4, 5, 6, 5, 7 ); # 7
1314 You can also go through each element and skip the ones you've seen
1316 before. Use a hash to keep track. The first time the loop sees an
1317 element, that element has no key in %Seen. The "next" statement creates
1318 the key and immediately uses its value, which is "undef", so the loop
1319 continues to the "push" and increments the value for that key. The next
1320 time the loop sees that same element, its key exists in the hash and
1321 the value for that key is true (since it's not 0 or "undef"), so the
1322 next skips that iteration and the loop goes to the next element.
1323 my @unique = ();
1325 my %seen = ();
1326 foreach my $elem ( @array ) {
1328 next if $seen{ $elem }++;
1329 push @unique, $elem;
1330 }
1331 You can write this more briefly using a grep, which does the same
1333 thing.
1334 my %seen = ();
1336 my @unique = grep { ! $seen{ $_ }++ } @array;
1337 How can I tell whether a certain element is contained in a list or array?
1339 (portions of this answer contributed by Anno Siegel and brian d foy)
1340 Hearing the word "in" is an indication that you probably should have
1342 used a hash, not a list or array, to store your data. Hashes are
1343 designed to answer this question quickly and efficiently. Arrays
1344 aren't.
1345 That being said, there are several ways to approach this. If you are
1347 going to make this query many times over arbitrary string values, the
1348 fastest way is probably to invert the original array and maintain a
1349 hash whose keys are the first array's values:
1350 my @blues = qw/azure cerulean teal turquoise lapis-lazuli/;
1352 my %is_blue = ();
1353 for (@blues) { $is_blue{$_} = 1 }
1354 Now you can check whether $is_blue{$some_color}. It might have been a
1356 good idea to keep the blues all in a hash in the first place.
1357 If the values are all small integers, you could use a simple indexed
1359 array. This kind of an array will take up less space:
1360 my @primes = (2, 3, 5, 7, 11, 13, 17, 19, 23, 29, 31);
1362 my @is_tiny_prime = ();
1363 for (@primes) { $is_tiny_prime[$_] = 1 }
1364 # or simply @istiny_prime[@primes] = (1) x @primes;
1365 Now you check whether $is_tiny_prime[$some_number].
1367 If the values in question are integers instead of strings, you can save
1369 quite a lot of space by using bit strings instead:
1370 my @articles = ( 1..10, 150..2000, 2017 );
1372 undef $read;
1373 for (@articles) { vec($read,$_,1) = 1 }
1374 Now check whether "vec($read,$n,1)" is true for some $n.
1376 These methods guarantee fast individual tests but require a re-
1378 organization of the original list or array. They only pay off if you
1379 have to test multiple values against the same array.
1380 If you are testing only once, the standard module List::Util exports
1382 the function "any" for this purpose. It works by stopping once it finds
1383 the element. It's written in C for speed, and its Perl equivalent looks
1384 like this subroutine:
1385 sub any (&@) {
1387 my $code = shift;
1388 foreach (@_) {
1389 return 1 if $code->();
1390 }
1391 return 0;
1392 }
1393 If speed is of little concern, the common idiom uses grep in scalar
1395 context (which returns the number of items that passed its condition)
1396 to traverse the entire list. This does have the benefit of telling you
1397 how many matches it found, though.
1398 my $is_there = grep $_ eq $whatever, @array;
1400 If you want to actually extract the matching elements, simply use grep
1402 in list context.
1403 my @matches = grep $_ eq $whatever, @array;
1405 How do I compute the difference of two arrays? How do I compute the
1407 intersection of two arrays?
1408 Use a hash. Here's code to do both and more. It assumes that each
1409 element is unique in a given array:
1410 my (@union, @intersection, @difference);
1412 my %count = ();
1413 foreach my $element (@array1, @array2) { $count{$element}++ }
1414 foreach my $element (keys %count) {
1415 push @union, $element;
1416 push @{ $count{$element} > 1 ? \@intersection : \@difference }, $element;
1417 }
1418 Note that this is the symmetric difference, that is, all elements in
1420 either A or in B but not in both. Think of it as an xor operation.
1421 How do I test whether two arrays or hashes are equal?
1423 The following code works for single-level arrays. It uses a stringwise
1424 comparison, and does not distinguish defined versus undefined empty
1425 strings. Modify if you have other needs.
1426 $are_equal = compare_arrays(\@frogs, \@toads);
1428 sub compare_arrays {
1430 my ($first, $second) = @_;
1431 no warnings; # silence spurious -w undef complaints
1432 return 0 unless @$first == @$second;
1433 for (my $i = 0; $i < @$first; $i++) {
1434 return 0 if $first->[$i] ne $second->[$i];
1435 }
1436 return 1;
1437 }
1438 For multilevel structures, you may wish to use an approach more like
1440 this one. It uses the CPAN module FreezeThaw:
1441 use FreezeThaw qw(cmpStr);
1443 my @a = my @b = ( "this", "that", [ "more", "stuff" ] );
1444 printf "a and b contain %s arrays\n",
1446 cmpStr(\@a, \@b) == 0
1447 ? "the same"
1448 : "different";
1449 This approach also works for comparing hashes. Here we'll demonstrate
1451 two different answers:
1452 use FreezeThaw qw(cmpStr cmpStrHard);
1454 my %a = my %b = ( "this" => "that", "extra" => [ "more", "stuff" ] );
1456 $a{EXTRA} = \%b;
1457 $b{EXTRA} = \%a;
1458 printf "a and b contain %s hashes\n",
1460 cmpStr(\%a, \%b) == 0 ? "the same" : "different";
1461 printf "a and b contain %s hashes\n",
1463 cmpStrHard(\%a, \%b) == 0 ? "the same" : "different";
1464 The first reports that both those the hashes contain the same data,
1466 while the second reports that they do not. Which you prefer is left as
1467 an exercise to the reader.
1468 How do I find the first array element for which a condition is true?
1470 To find the first array element which satisfies a condition, you can
1471 use the "first()" function in the List::Util module, which comes with
1472 Perl 5.8. This example finds the first element that contains "Perl".
1473 use List::Util qw(first);
1475 my $element = first { /Perl/ } @array;
1477 If you cannot use List::Util, you can make your own loop to do the same
1479 thing. Once you find the element, you stop the loop with last.
1480 my $found;
1482 foreach ( @array ) {
1483 if( /Perl/ ) { $found = $_; last }
1484 }
1485 If you want the array index, use the "firstidx()" function from
1487 "List::MoreUtils":
1488 use List::MoreUtils qw(firstidx);
1490 my $index = firstidx { /Perl/ } @array;
1491 Or write it yourself, iterating through the indices and checking the
1493 array element at each index until you find one that satisfies the
1494 condition:
1495 my( $found, $index ) = ( undef, -1 );
1497 for( $i = 0; $i < @array; $i++ ) {
1498 if( $array[$i] =~ /Perl/ ) {
1499 $found = $array[$i];
1500 $index = $i;
1501 last;
1502 }
1503 }
1504 How do I handle linked lists?
1506 (contributed by brian d foy)
1507 Perl's arrays do not have a fixed size, so you don't need linked lists
1509 if you just want to add or remove items. You can use array operations
1510 such as "push", "pop", "shift", "unshift", or "splice" to do that.
1511 Sometimes, however, linked lists can be useful in situations where you
1513 want to "shard" an array so you have many small arrays instead of a
1514 single big array. You can keep arrays longer than Perl's largest array
1515 index, lock smaller arrays separately in threaded programs, reallocate
1516 less memory, or quickly insert elements in the middle of the chain.
1517 Steve Lembark goes through the details in his YAPC::NA 2009 talk "Perly
1519 Linked Lists" ( <http://www.slideshare.net/lembark/perly-linked-lists>
1520 ), although you can just use his LinkedList::Single module.
1521 How do I handle circular lists?
1523 (contributed by brian d foy)
1524 If you want to cycle through an array endlessly, you can increment the
1526 index modulo the number of elements in the array:
1527 my @array = qw( a b c );
1529 my $i = 0;
1530 while( 1 ) {
1532 print $array[ $i++ % @array ], "\n";
1533 last if $i > 20;
1534 }
1535 You can also use Tie::Cycle to use a scalar that always has the next
1537 element of the circular array:
1538 use Tie::Cycle;
1540 tie my $cycle, 'Tie::Cycle', [ qw( FFFFFF 000000 FFFF00 ) ];
1542 print $cycle; # FFFFFF
1544 print $cycle; # 000000
1545 print $cycle; # FFFF00
1546 The Array::Iterator::Circular creates an iterator object for circular
1548 arrays:
1549 use Array::Iterator::Circular;
1551 my $color_iterator = Array::Iterator::Circular->new(
1553 qw(red green blue orange)
1554 );
1555 foreach ( 1 .. 20 ) {
1557 print $color_iterator->next, "\n";
1558 }
1559 How do I shuffle an array randomly?
1561 If you either have Perl 5.8.0 or later installed, or if you have
1562 Scalar-List-Utils 1.03 or later installed, you can say:
1563 use List::Util 'shuffle';
1565 @shuffled = shuffle(@list);
1567 If not, you can use a Fisher-Yates shuffle.
1569 sub fisher_yates_shuffle {
1571 my $deck = shift; # $deck is a reference to an array
1572 return unless @$deck; # must not be empty!
1573 my $i = @$deck;
1575 while (--$i) {
1576 my $j = int rand ($i+1);
1577 @$deck[$i,$j] = @$deck[$j,$i];
1578 }
1579 }
1580 # shuffle my mpeg collection
1582 #
1583 my @mpeg = <audio/*/*.mp3>;
1584 fisher_yates_shuffle( \@mpeg ); # randomize @mpeg in place
1585 print @mpeg;
1586 Note that the above implementation shuffles an array in place, unlike
1588 the "List::Util::shuffle()" which takes a list and returns a new
1589 shuffled list.
1590 You've probably seen shuffling algorithms that work using splice,
1592 randomly picking another element to swap the current element with
1593 srand;
1595 @new = ();
1596 @old = 1 .. 10; # just a demo
1597 while (@old) {
1598 push(@new, splice(@old, rand @old, 1));
1599 }
1600 This is bad because splice is already O(N), and since you do it N
1602 times, you just invented a quadratic algorithm; that is, O(N**2). This
1603 does not scale, although Perl is so efficient that you probably won't
1604 notice this until you have rather largish arrays.
1605 How do I process/modify each element of an array?
1607 Use "for"/"foreach":
1608 for (@lines) {
1610 s/foo/bar/; # change that word
1611 tr/XZ/ZX/; # swap those letters
1612 }
1613 Here's another; let's compute spherical volumes:
1615 my @volumes = @radii;
1617 for (@volumes) { # @volumes has changed parts
1618 $_ **= 3;
1619 $_ *= (4/3) * 3.14159; # this will be constant folded
1620 }
1621 which can also be done with "map()" which is made to transform one list
1623 into another:
1624 my @volumes = map {$_ ** 3 * (4/3) * 3.14159} @radii;
1626 If you want to do the same thing to modify the values of the hash, you
1628 can use the "values" function. As of Perl 5.6 the values are not
1629 copied, so if you modify $orbit (in this case), you modify the value.
1630 for my $orbit ( values %orbits ) {
1632 ($orbit **= 3) *= (4/3) * 3.14159;
1633 }
1634 Prior to perl 5.6 "values" returned copies of the values, so older perl
1636 code often contains constructions such as @orbits{keys %orbits} instead
1637 of "values %orbits" where the hash is to be modified.
1638 How do I select a random element from an array?
1640 Use the "rand()" function (see "rand" in perlfunc):
1641 my $index = rand @array;
1643 my $element = $array[$index];
1644 Or, simply:
1646 my $element = $array[ rand @array ];
1648 How do I permute N elements of a list?
1650 Use the List::Permutor module on CPAN. If the list is actually an
1651 array, try the Algorithm::Permute module (also on CPAN). It's written
1652 in XS code and is very efficient:
1653 use Algorithm::Permute;
1655 my @array = 'a'..'d';
1657 my $p_iterator = Algorithm::Permute->new ( \@array );
1658 while (my @perm = $p_iterator->next) {
1660 print "next permutation: (@perm)\n";
1661 }
1662 For even faster execution, you could do:
1664 use Algorithm::Permute;
1666 my @array = 'a'..'d';
1668 Algorithm::Permute::permute {
1670 print "next permutation: (@array)\n";
1671 } @array;
1672 Here's a little program that generates all permutations of all the
1674 words on each line of input. The algorithm embodied in the "permute()"
1675 function is discussed in Volume 4 (still unpublished) of Knuth's The
1676 Art of Computer Programming and will work on any list:
1677 #!/usr/bin/perl -n
1679 # Fischer-Krause ordered permutation generator
1680 sub permute (&@) {
1682 my $code = shift;
1683 my @idx = 0..$#_;
1684 while ( $code->(@_[@idx]) ) {
1685 my $p = $#idx;
1686 --$p while $idx[$p-1] > $idx[$p];
1687 my $q = $p or return;
1688 push @idx, reverse splice @idx, $p;
1689 ++$q while $idx[$p-1] > $idx[$q];
1690 @idx[$p-1,$q]=@idx[$q,$p-1];
1691 }
1692 }
1693 permute { print "@_\n" } split;
1695 The Algorithm::Loops module also provides the "NextPermute" and
1697 "NextPermuteNum" functions which efficiently find all unique
1698 permutations of an array, even if it contains duplicate values,
1699 modifying it in-place: if its elements are in reverse-sorted order then
1700 the array is reversed, making it sorted, and it returns false;
1701 otherwise the next permutation is returned.
1702 "NextPermute" uses string order and "NextPermuteNum" numeric order, so
1704 you can enumerate all the permutations of 0..9 like this:
1705 use Algorithm::Loops qw(NextPermuteNum);
1707 my @list= 0..9;
1709 do { print "@list\n" } while NextPermuteNum @list;
1710 How do I sort an array by (anything)?
1712 Supply a comparison function to sort() (described in "sort" in
1713 perlfunc):
1714 @list = sort { $a <=> $b } @list;
1716 The default sort function is cmp, string comparison, which would sort
1718 "(1, 2, 10)" into "(1, 10, 2)". "<=>", used above, is the numerical
1719 comparison operator.
1720 If you have a complicated function needed to pull out the part you want
1722 to sort on, then don't do it inside the sort function. Pull it out
1723 first, because the sort BLOCK can be called many times for the same
1724 element. Here's an example of how to pull out the first word after the
1725 first number on each item, and then sort those words case-
1726 insensitively.
1727 my @idx;
1729 for (@data) {
1730 my $item;
1731 ($item) = /\d+\s*(\S+)/;
1732 push @idx, uc($item);
1733 }
1734 my @sorted = @data[ sort { $idx[$a] cmp $idx[$b] } 0 .. $#idx ];
1735 which could also be written this way, using a trick that's come to be
1737 known as the Schwartzian Transform:
1738 my @sorted = map { $_->[0] }
1740 sort { $a->[1] cmp $b->[1] }
1741 map { [ $_, uc( (/\d+\s*(\S+)/)[0]) ] } @data;
1742 If you need to sort on several fields, the following paradigm is
1744 useful.
1745 my @sorted = sort {
1747 field1($a) <=> field1($b) ||
1748 field2($a) cmp field2($b) ||
1749 field3($a) cmp field3($b)
1750 } @data;
1751 This can be conveniently combined with precalculation of keys as given
1753 above.
1754 See the sort article in the "Far More Than You Ever Wanted To Know"
1756 collection in <http://www.cpan.org/misc/olddoc/FMTEYEWTK.tgz> for more
1757 about this approach.
1758 See also the question later in perlfaq4 on sorting hashes.
1760 How do I manipulate arrays of bits?
1762 Use "pack()" and "unpack()", or else "vec()" and the bitwise
1763 operations.
1764 For example, you don't have to store individual bits in an array (which
1766 would mean that you're wasting a lot of space). To convert an array of
1767 bits to a string, use "vec()" to set the right bits. This sets $vec to
1768 have bit N set only if $ints[N] was set:
1769 my @ints = (...); # array of bits, e.g. ( 1, 0, 0, 1, 1, 0 ... )
1771 my $vec = '';
1772 foreach( 0 .. $#ints ) {
1773 vec($vec,$_,1) = 1 if $ints[$_];
1774 }
1775 The string $vec only takes up as many bits as it needs. For instance,
1777 if you had 16 entries in @ints, $vec only needs two bytes to store them
1778 (not counting the scalar variable overhead).
1779 Here's how, given a vector in $vec, you can get those bits into your
1781 @ints array:
1782 sub bitvec_to_list {
1784 my $vec = shift;
1785 my @ints;
1786 # Find null-byte density then select best algorithm
1787 if ($vec =~ tr/\0// / length $vec > 0.95) {
1788 use integer;
1789 my $i;
1790 # This method is faster with mostly null-bytes
1792 while($vec =~ /[^\0]/g ) {
1793 $i = -9 + 8 * pos $vec;
1794 push @ints, $i if vec($vec, ++$i, 1);
1795 push @ints, $i if vec($vec, ++$i, 1);
1796 push @ints, $i if vec($vec, ++$i, 1);
1797 push @ints, $i if vec($vec, ++$i, 1);
1798 push @ints, $i if vec($vec, ++$i, 1);
1799 push @ints, $i if vec($vec, ++$i, 1);
1800 push @ints, $i if vec($vec, ++$i, 1);
1801 push @ints, $i if vec($vec, ++$i, 1);
1802 }
1803 }
1804 else {
1805 # This method is a fast general algorithm
1806 use integer;
1807 my $bits = unpack "b*", $vec;
1808 push @ints, 0 if $bits =~ s/^(\d)// && $1;
1809 push @ints, pos $bits while($bits =~ /1/g);
1810 }
1811 return \@ints;
1813 }
1814 This method gets faster the more sparse the bit vector is. (Courtesy
1816 of Tim Bunce and Winfried Koenig.)
1817 You can make the while loop a lot shorter with this suggestion from
1819 Benjamin Goldberg:
1820 while($vec =~ /[^\0]+/g ) {
1822 push @ints, grep vec($vec, $_, 1), $-[0] * 8 .. $+[0] * 8;
1823 }
1824 Or use the CPAN module Bit::Vector:
1826 my $vector = Bit::Vector->new($num_of_bits);
1828 $vector->Index_List_Store(@ints);
1829 my @ints = $vector->Index_List_Read();
1830 Bit::Vector provides efficient methods for bit vector, sets of small
1832 integers and "big int" math.
1833 Here's a more extensive illustration using vec():
1835 # vec demo
1837 my $vector = "\xff\x0f\xef\xfe";
1838 print "Ilya's string \\xff\\x0f\\xef\\xfe represents the number ",
1839 unpack("N", $vector), "\n";
1840 my $is_set = vec($vector, 23, 1);
1841 print "Its 23rd bit is ", $is_set ? "set" : "clear", ".\n";
1842 pvec($vector);
1843 set_vec(1,1,1);
1845 set_vec(3,1,1);
1846 set_vec(23,1,1);
1847 set_vec(3,1,3);
1849 set_vec(3,2,3);
1850 set_vec(3,4,3);
1851 set_vec(3,4,7);
1852 set_vec(3,8,3);
1853 set_vec(3,8,7);
1854 set_vec(0,32,17);
1856 set_vec(1,32,17);
1857 sub set_vec {
1859 my ($offset, $width, $value) = @_;
1860 my $vector = '';
1861 vec($vector, $offset, $width) = $value;
1862 print "offset=$offset width=$width value=$value\n";
1863 pvec($vector);
1864 }
1865 sub pvec {
1867 my $vector = shift;
1868 my $bits = unpack("b*", $vector);
1869 my $i = 0;
1870 my $BASE = 8;
1871 print "vector length in bytes: ", length($vector), "\n";
1873 @bytes = unpack("A8" x length($vector), $bits);
1874 print "bits are: @bytes\n\n";
1875 }
1876 Why does defined() return true on empty arrays and hashes?
1878 The short story is that you should probably only use defined on scalars
1879 or functions, not on aggregates (arrays and hashes). See "defined" in
1880 perlfunc in the 5.004 release or later of Perl for more detail.
1881
1883 How do I process an entire hash?
1884 (contributed by brian d foy)
1885 There are a couple of ways that you can process an entire hash. You can
1887 get a list of keys, then go through each key, or grab a one key-value
1888 pair at a time.
1889 To go through all of the keys, use the "keys" function. This extracts
1891 all of the keys of the hash and gives them back to you as a list. You
1892 can then get the value through the particular key you're processing:
1893 foreach my $key ( keys %hash ) {
1895 my $value = $hash{$key}
1896 ...
1897 }
1898 Once you have the list of keys, you can process that list before you
1900 process the hash elements. For instance, you can sort the keys so you
1901 can process them in lexical order:
1902 foreach my $key ( sort keys %hash ) {
1904 my $value = $hash{$key}
1905 ...
1906 }
1907 Or, you might want to only process some of the items. If you only want
1909 to deal with the keys that start with "text:", you can select just
1910 those using "grep":
1911 foreach my $key ( grep /^text:/, keys %hash ) {
1913 my $value = $hash{$key}
1914 ...
1915 }
1916 If the hash is very large, you might not want to create a long list of
1918 keys. To save some memory, you can grab one key-value pair at a time
1919 using "each()", which returns a pair you haven't seen yet:
1920 while( my( $key, $value ) = each( %hash ) ) {
1922 ...
1923 }
1924 The "each" operator returns the pairs in apparently random order, so if
1926 ordering matters to you, you'll have to stick with the "keys" method.
1927 The "each()" operator can be a bit tricky though. You can't add or
1929 delete keys of the hash while you're using it without possibly skipping
1930 or re-processing some pairs after Perl internally rehashes all of the
1931 elements. Additionally, a hash has only one iterator, so if you mix
1932 "keys", "values", or "each" on the same hash, you risk resetting the
1933 iterator and messing up your processing. See the "each" entry in
1934 perlfunc for more details.
1935 How do I merge two hashes?
1937 (contributed by brian d foy)
1938 Before you decide to merge two hashes, you have to decide what to do if
1940 both hashes contain keys that are the same and if you want to leave the
1941 original hashes as they were.
1942 If you want to preserve the original hashes, copy one hash (%hash1) to
1944 a new hash (%new_hash), then add the keys from the other hash (%hash2
1945 to the new hash. Checking that the key already exists in %new_hash
1946 gives you a chance to decide what to do with the duplicates:
1947 my %new_hash = %hash1; # make a copy; leave %hash1 alone
1949 foreach my $key2 ( keys %hash2 ) {
1951 if( exists $new_hash{$key2} ) {
1952 warn "Key [$key2] is in both hashes!";
1953 # handle the duplicate (perhaps only warning)
1954 ...
1955 next;
1956 }
1957 else {
1958 $new_hash{$key2} = $hash2{$key2};
1959 }
1960 }
1961 If you don't want to create a new hash, you can still use this looping
1963 technique; just change the %new_hash to %hash1.
1964 foreach my $key2 ( keys %hash2 ) {
1966 if( exists $hash1{$key2} ) {
1967 warn "Key [$key2] is in both hashes!";
1968 # handle the duplicate (perhaps only warning)
1969 ...
1970 next;
1971 }
1972 else {
1973 $hash1{$key2} = $hash2{$key2};
1974 }
1975 }
1976 If you don't care that one hash overwrites keys and values from the
1978 other, you could just use a hash slice to add one hash to another. In
1979 this case, values from %hash2 replace values from %hash1 when they have
1980 keys in common:
1981 @hash1{ keys %hash2 } = values %hash2;
1983 What happens if I add or remove keys from a hash while iterating over it?
1985 (contributed by brian d foy)
1986 The easy answer is "Don't do that!"
1988 If you iterate through the hash with each(), you can delete the key
1990 most recently returned without worrying about it. If you delete or add
1991 other keys, the iterator may skip or double up on them since perl may
1992 rearrange the hash table. See the entry for "each()" in perlfunc.
1993 How do I look up a hash element by value?
1995 Create a reverse hash:
1996 my %by_value = reverse %by_key;
1998 my $key = $by_value{$value};
1999 That's not particularly efficient. It would be more space-efficient to
2001 use:
2002 while (my ($key, $value) = each %by_key) {
2004 $by_value{$value} = $key;
2005 }
2006 If your hash could have repeated values, the methods above will only
2008 find one of the associated keys. This may or may not worry you. If it
2009 does worry you, you can always reverse the hash into a hash of arrays
2010 instead:
2011 while (my ($key, $value) = each %by_key) {
2013 push @{$key_list_by_value{$value}}, $key;
2014 }
2015 How can I know how many entries are in a hash?
2017 (contributed by brian d foy)
2018 This is very similar to "How do I process an entire hash?", also in
2020 perlfaq4, but a bit simpler in the common cases.
2021 You can use the "keys()" built-in function in scalar context to find
2023 out have many entries you have in a hash:
2024 my $key_count = keys %hash; # must be scalar context!
2026 If you want to find out how many entries have a defined value, that's a
2028 bit different. You have to check each value. A "grep" is handy:
2029 my $defined_value_count = grep { defined } values %hash;
2031 You can use that same structure to count the entries any way that you
2033 like. If you want the count of the keys with vowels in them, you just
2034 test for that instead:
2035 my $vowel_count = grep { /[aeiou]/ } keys %hash;
2037 The "grep" in scalar context returns the count. If you want the list of
2039 matching items, just use it in list context instead:
2040 my @defined_values = grep { defined } values %hash;
2042 The "keys()" function also resets the iterator, which means that you
2044 may see strange results if you use this between uses of other hash
2045 operators such as "each()".
2046 How do I sort a hash (optionally by value instead of key)?
2048 (contributed by brian d foy)
2049 To sort a hash, start with the keys. In this example, we give the list
2051 of keys to the sort function which then compares them ASCIIbetically
2052 (which might be affected by your locale settings). The output list has
2053 the keys in ASCIIbetical order. Once we have the keys, we can go
2054 through them to create a report which lists the keys in ASCIIbetical
2055 order.
2056 my @keys = sort { $a cmp $b } keys %hash;
2058 foreach my $key ( @keys ) {
2060 printf "%-20s %6d\n", $key, $hash{$key};
2061 }
2062 We could get more fancy in the "sort()" block though. Instead of
2064 comparing the keys, we can compute a value with them and use that value
2065 as the comparison.
2066 For instance, to make our report order case-insensitive, we use "lc" to
2068 lowercase the keys before comparing them:
2069 my @keys = sort { lc $a cmp lc $b } keys %hash;
2071 Note: if the computation is expensive or the hash has many elements,
2073 you may want to look at the Schwartzian Transform to cache the
2074 computation results.
2075 If we want to sort by the hash value instead, we use the hash key to
2077 look it up. We still get out a list of keys, but this time they are
2078 ordered by their value.
2079 my @keys = sort { $hash{$a} <=> $hash{$b} } keys %hash;
2081 From there we can get more complex. If the hash values are the same, we
2083 can provide a secondary sort on the hash key.
2084 my @keys = sort {
2086 $hash{$a} <=> $hash{$b}
2087 or
2088 "\L$a" cmp "\L$b"
2089 } keys %hash;
2090 How can I always keep my hash sorted?
2092 You can look into using the "DB_File" module and "tie()" using the
2093 $DB_BTREE hash bindings as documented in "In Memory Databases" in
2094 DB_File. The Tie::IxHash module from CPAN might also be instructive.
2095 Although this does keep your hash sorted, you might not like the
2096 slowdown you suffer from the tie interface. Are you sure you need to do
2097 this? :)
2098 What's the difference between "delete" and "undef" with hashes?
2100 Hashes contain pairs of scalars: the first is the key, the second is
2101 the value. The key will be coerced to a string, although the value can
2102 be any kind of scalar: string, number, or reference. If a key $key is
2103 present in %hash, "exists($hash{$key})" will return true. The value for
2104 a given key can be "undef", in which case $hash{$key} will be "undef"
2105 while "exists $hash{$key}" will return true. This corresponds to ($key,
2106 "undef") being in the hash.
2107 Pictures help... Here's the %hash table:
2109 keys values
2111 +------+------+
2112 | a | 3 |
2113 | x | 7 |
2114 | d | 0 |
2115 | e | 2 |
2116 +------+------+
2117 And these conditions hold
2119 $hash{'a'} is true
2121 $hash{'d'} is false
2122 defined $hash{'d'} is true
2123 defined $hash{'a'} is true
2124 exists $hash{'a'} is true (Perl 5 only)
2125 grep ($_ eq 'a', keys %hash) is true
2126 If you now say
2128 undef $hash{'a'}
2130 your table now reads:
2132 keys values
2134 +------+------+
2135 | a | undef|
2136 | x | 7 |
2137 | d | 0 |
2138 | e | 2 |
2139 +------+------+
2140 and these conditions now hold; changes in caps:
2142 $hash{'a'} is FALSE
2144 $hash{'d'} is false
2145 defined $hash{'d'} is true
2146 defined $hash{'a'} is FALSE
2147 exists $hash{'a'} is true (Perl 5 only)
2148 grep ($_ eq 'a', keys %hash) is true
2149 Notice the last two: you have an undef value, but a defined key!
2151 Now, consider this:
2153 delete $hash{'a'}
2155 your table now reads:
2157 keys values
2159 +------+------+
2160 | x | 7 |
2161 | d | 0 |
2162 | e | 2 |
2163 +------+------+
2164 and these conditions now hold; changes in caps:
2166 $hash{'a'} is false
2168 $hash{'d'} is false
2169 defined $hash{'d'} is true
2170 defined $hash{'a'} is false
2171 exists $hash{'a'} is FALSE (Perl 5 only)
2172 grep ($_ eq 'a', keys %hash) is FALSE
2173 See, the whole entry is gone!
2175 Why don't my tied hashes make the defined/exists distinction?
2177 This depends on the tied hash's implementation of EXISTS(). For
2178 example, there isn't the concept of undef with hashes that are tied to
2179 DBM* files. It also means that exists() and defined() do the same thing
2180 with a DBM* file, and what they end up doing is not what they do with
2181 ordinary hashes.
2182 How do I reset an each() operation part-way through?
2184 (contributed by brian d foy)
2185 You can use the "keys" or "values" functions to reset "each". To simply
2187 reset the iterator used by "each" without doing anything else, use one
2188 of them in void context:
2189 keys %hash; # resets iterator, nothing else.
2191 values %hash; # resets iterator, nothing else.
2192 See the documentation for "each" in perlfunc.
2194 How can I get the unique keys from two hashes?
2196 First you extract the keys from the hashes into lists, then solve the
2197 "removing duplicates" problem described above. For example:
2198 my %seen = ();
2200 for my $element (keys(%foo), keys(%bar)) {
2201 $seen{$element}++;
2202 }
2203 my @uniq = keys %seen;
2204 Or more succinctly:
2206 my @uniq = keys %{{%foo,%bar}};
2208 Or if you really want to save space:
2210 my %seen = ();
2212 while (defined ($key = each %foo)) {
2213 $seen{$key}++;
2214 }
2215 while (defined ($key = each %bar)) {
2216 $seen{$key}++;
2217 }
2218 my @uniq = keys %seen;
2219 How can I store a multidimensional array in a DBM file?
2221 Either stringify the structure yourself (no fun), or else get the MLDBM
2222 (which uses Data::Dumper) module from CPAN and layer it on top of
2223 either DB_File or GDBM_File. You might also try DBM::Deep, but it can
2224 be a bit slow.
2225 How can I make my hash remember the order I put elements into it?
2227 Use the Tie::IxHash from CPAN.
2228 use Tie::IxHash;
2230 tie my %myhash, 'Tie::IxHash';
2232 for (my $i=0; $i<20; $i++) {
2234 $myhash{$i} = 2*$i;
2235 }
2236 my @keys = keys %myhash;
2238 # @keys = (0,1,2,3,...)
2239 Why does passing a subroutine an undefined element in a hash create it?
2241 (contributed by brian d foy)
2242 Are you using a really old version of Perl?
2244 Normally, accessing a hash key's value for a nonexistent key will not
2246 create the key.
2247 my %hash = ();
2249 my $value = $hash{ 'foo' };
2250 print "This won't print\n" if exists $hash{ 'foo' };
2251 Passing $hash{ 'foo' } to a subroutine used to be a special case,
2253 though. Since you could assign directly to $_[0], Perl had to be ready
2254 to make that assignment so it created the hash key ahead of time:
2255 my_sub( $hash{ 'foo' } );
2257 print "This will print before 5.004\n" if exists $hash{ 'foo' };
2258 sub my_sub {
2260 # $_[0] = 'bar'; # create hash key in case you do this
2261 1;
2262 }
2263 Since Perl 5.004, however, this situation is a special case and Perl
2265 creates the hash key only when you make the assignment:
2266 my_sub( $hash{ 'foo' } );
2268 print "This will print, even after 5.004\n" if exists $hash{ 'foo' };
2269 sub my_sub {
2271 $_[0] = 'bar';
2272 }
2273 However, if you want the old behavior (and think carefully about that
2275 because it's a weird side effect), you can pass a hash slice instead.
2276 Perl 5.004 didn't make this a special case:
2277 my_sub( @hash{ qw/foo/ } );
2279 How can I make the Perl equivalent of a C structure/C++ class/hash or array
2281 of hashes or arrays?
2282 Usually a hash ref, perhaps like this:
2283 $record = {
2285 NAME => "Jason",
2286 EMPNO => 132,
2287 TITLE => "deputy peon",
2288 AGE => 23,
2289 SALARY => 37_000,
2290 PALS => [ "Norbert", "Rhys", "Phineas"],
2291 };
2292 References are documented in perlref and perlreftut. Examples of
2294 complex data structures are given in perldsc and perllol. Examples of
2295 structures and object-oriented classes are in perlootut.
2296 How can I use a reference as a hash key?
2298 (contributed by brian d foy and Ben Morrow)
2299 Hash keys are strings, so you can't really use a reference as the key.
2301 When you try to do that, perl turns the reference into its stringified
2302 form (for instance, "HASH(0xDEADBEEF)"). From there you can't get back
2303 the reference from the stringified form, at least without doing some
2304 extra work on your own.
2305 Remember that the entry in the hash will still be there even if the
2307 referenced variable goes out of scope, and that it is entirely
2308 possible for Perl to subsequently allocate a different variable at the
2309 same address. This will mean a new variable might accidentally be
2310 associated with the value for an old.
2311 If you have Perl 5.10 or later, and you just want to store a value
2313 against the reference for lookup later, you can use the core
2314 Hash::Util::Fieldhash module. This will also handle renaming the keys
2315 if you use multiple threads (which causes all variables to be
2316 reallocated at new addresses, changing their stringification), and
2317 garbage-collecting the entries when the referenced variable goes out of
2318 scope.
2319 If you actually need to be able to get a real reference back from each
2321 hash entry, you can use the Tie::RefHash module, which does the
2322 required work for you.
2323 How can I check if a key exists in a multilevel hash?
2325 (contributed by brian d foy)
2326 The trick to this problem is avoiding accidental autovivification. If
2328 you want to check three keys deep, you might naïvely try this:
2329 my %hash;
2331 if( exists $hash{key1}{key2}{key3} ) {
2332 ...;
2333 }
2334 Even though you started with a completely empty hash, after that call
2336 to "exists" you've created the structure you needed to check for
2337 "key3":
2338 %hash = (
2340 'key1' => {
2341 'key2' => {}
2342 }
2343 );
2344 That's autovivification. You can get around this in a few ways. The
2346 easiest way is to just turn it off. The lexical "autovivification"
2347 pragma is available on CPAN. Now you don't add to the hash:
2348 {
2350 no autovivification;
2351 my %hash;
2352 if( exists $hash{key1}{key2}{key3} ) {
2353 ...;
2354 }
2355 }
2356 The Data::Diver module on CPAN can do it for you too. Its "Dive"
2358 subroutine can tell you not only if the keys exist but also get the
2359 value:
2360 use Data::Diver qw(Dive);
2362 my @exists = Dive( \%hash, qw(key1 key2 key3) );
2364 if( ! @exists ) {
2365 ...; # keys do not exist
2366 }
2367 elsif( ! defined $exists[0] ) {
2368 ...; # keys exist but value is undef
2369 }
2370 You can easily do this yourself too by checking each level of the hash
2372 before you move onto the next level. This is essentially what
2373 Data::Diver does for you:
2374 if( check_hash( \%hash, qw(key1 key2 key3) ) ) {
2376 ...;
2377 }
2378 sub check_hash {
2380 my( $hash, @keys ) = @_;
2381 return unless @keys;
2383 foreach my $key ( @keys ) {
2385 return unless eval { exists $hash->{$key} };
2386 $hash = $hash->{$key};
2387 }
2388 return 1;
2390 }
2391 How can I prevent addition of unwanted keys into a hash?
2393 Since version 5.8.0, hashes can be restricted to a fixed number of
2394 given keys. Methods for creating and dealing with restricted hashes are
2395 exported by the Hash::Util module.
2396
2398 How do I handle binary data correctly?
2399 Perl is binary-clean, so it can handle binary data just fine. On
2400 Windows or DOS, however, you have to use "binmode" for binary files to
2401 avoid conversions for line endings. In general, you should use
2402 "binmode" any time you want to work with binary data.
2403 Also see "binmode" in perlfunc or perlopentut.
2405 If you're concerned about 8-bit textual data then see perllocale. If
2407 you want to deal with multibyte characters, however, there are some
2408 gotchas. See the section on Regular Expressions.
2409 How do I determine whether a scalar is a number/whole/integer/float?
2411 Assuming that you don't care about IEEE notations like "NaN" or
2412 "Infinity", you probably just want to use a regular expression (see
2413 also perlretut and perlre):
2414 use 5.010;
2416 if ( /\D/ )
2418 { say "\thas nondigits"; }
2419 if ( /^\d+\z/ )
2420 { say "\tis a whole number"; }
2421 if ( /^-?\d+\z/ )
2422 { say "\tis an integer"; }
2423 if ( /^[+-]?\d+\z/ )
2424 { say "\tis a +/- integer"; }
2425 if ( /^-?(?:\d+\.?|\.\d)\d*\z/ )
2426 { say "\tis a real number"; }
2427 if ( /^[+-]?(?=\.?\d)\d*\.?\d*(?:e[+-]?\d+)?\z/i )
2428 { say "\tis a C float" }
2429 There are also some commonly used modules for the task. Scalar::Util
2431 (distributed with 5.8) provides access to perl's internal function
2432 "looks_like_number" for determining whether a variable looks like a
2433 number. Data::Types exports functions that validate data types using
2434 both the above and other regular expressions. Thirdly, there is
2435 Regexp::Common which has regular expressions to match various types of
2436 numbers. Those three modules are available from the CPAN.
2437 If you're on a POSIX system, Perl supports the "POSIX::strtod" function
2439 for converting strings to doubles (and also "POSIX::strtol" for longs).
2440 Its semantics are somewhat cumbersome, so here's a "getnum" wrapper
2441 function for more convenient access. This function takes a string and
2442 returns the number it found, or "undef" for input that isn't a C float.
2443 The "is_numeric" function is a front end to "getnum" if you just want
2444 to say, "Is this a float?"
2445 sub getnum {
2447 use POSIX qw(strtod);
2448 my $str = shift;
2449 $str =~ s/^\s+//;
2450 $str =~ s/\s+$//;
2451 $! = 0;
2452 my($num, $unparsed) = strtod($str);
2453 if (($str eq '') || ($unparsed != 0) || $!) {
2454 return undef;
2455 }
2456 else {
2457 return $num;
2458 }
2459 }
2460 sub is_numeric { defined getnum($_[0]) }
2462 Or you could check out the String::Scanf module on the CPAN instead.
2464 How do I keep persistent data across program calls?
2466 For some specific applications, you can use one of the DBM modules.
2467 See AnyDBM_File. More generically, you should consult the FreezeThaw or
2468 Storable modules from CPAN. Starting from Perl 5.8, Storable is part of
2469 the standard distribution. Here's one example using Storable's "store"
2470 and "retrieve" functions:
2471 use Storable;
2473 store(\%hash, "filename");
2474 # later on...
2476 $href = retrieve("filename"); # by ref
2477 %hash = %{ retrieve("filename") }; # direct to hash
2478 How do I print out or copy a recursive data structure?
2480 The Data::Dumper module on CPAN (or the 5.005 release of Perl) is great
2481 for printing out data structures. The Storable module on CPAN (or the
2482 5.8 release of Perl), provides a function called "dclone" that
2483 recursively copies its argument.
2484 use Storable qw(dclone);
2486 $r2 = dclone($r1);
2487 Where $r1 can be a reference to any kind of data structure you'd like.
2489 It will be deeply copied. Because "dclone" takes and returns
2490 references, you'd have to add extra punctuation if you had a hash of
2491 arrays that you wanted to copy.
2492 %newhash = %{ dclone(\%oldhash) };
2494 How do I define methods for every class/object?
2496 (contributed by Ben Morrow)
2497 You can use the "UNIVERSAL" class (see UNIVERSAL). However, please be
2499 very careful to consider the consequences of doing this: adding methods
2500 to every object is very likely to have unintended consequences. If
2501 possible, it would be better to have all your object inherit from some
2502 common base class, or to use an object system like Moose that supports
2503 roles.
2504 How do I verify a credit card checksum?
2506 Get the Business::CreditCard module from CPAN.
2507 How do I pack arrays of doubles or floats for XS code?
2509 The arrays.h/arrays.c code in the PGPLOT module on CPAN does just this.
2510 If you're doing a lot of float or double processing, consider using the
2511 PDL module from CPAN instead--it makes number-crunching easy.
2512 See <https://metacpan.org/release/PGPLOT> for the code.
2514
2516 Copyright (c) 1997-2010 Tom Christiansen, Nathan Torkington, and other
2517 authors as noted. All rights reserved.
2518 This documentation is free; you can redistribute it and/or modify it
2520 under the same terms as Perl itself.
2521 Irrespective of its distribution, all code examples in this file are
2523 hereby placed into the public domain. You are permitted and encouraged
2524 to use this code in your own programs for fun or for profit as you see
2525 fit. A simple comment in the code giving credit would be courteous but
2526 is not required.
2527perl v5.30.1 2020-01-30 perlfaq4(3)