1PERLPERF(1)            Perl Programmers Reference Guide            PERLPERF(1)
2
3
4

NAME

6       perlperf - Perl Performance and Optimization Techniques
7

DESCRIPTION

9       This is an introduction to the use of performance and optimization
10       techniques which can be used with particular reference to perl
11       programs.  While many perl developers have come from other languages,
12       and can use their prior knowledge where appropriate, there are many
13       other people who might benefit from a few perl specific pointers.  If
14       you want the condensed version, perhaps the best advice comes from the
15       renowned Japanese Samurai, Miyamoto Musashi, who said:
16
17        "Do Not Engage in Useless Activity"
18
19       in 1645.
20

OVERVIEW

22       Perhaps the most common mistake programmers make is to attempt to
23       optimize their code before a program actually does anything useful -
24       this is a bad idea.  There's no point in having an extremely fast
25       program that doesn't work.  The first job is to get a program to
26       correctly do something useful, (not to mention ensuring the test suite
27       is fully functional), and only then to consider optimizing it.  Having
28       decided to optimize existing working code, there are several simple but
29       essential steps to consider which are intrinsic to any optimization
30       process.
31
32   ONE STEP SIDEWAYS
33       Firstly, you need to establish a baseline time for the existing code,
34       which timing needs to be reliable and repeatable.  You'll probably want
35       to use the "Benchmark" or "Devel::NYTProf" modules, or something
36       similar, for this step, or perhaps the Unix system "time" utility,
37       whichever is appropriate.  See the base of this document for a longer
38       list of benchmarking and profiling modules, and recommended further
39       reading.
40
41   ONE STEP FORWARD
42       Next, having examined the program for hot spots, (places where the code
43       seems to run slowly), change the code with the intention of making it
44       run faster.  Using version control software, like "subversion", will
45       ensure no changes are irreversible.  It's too easy to fiddle here and
46       fiddle there - don't change too much at any one time or you might not
47       discover which piece of code really was the slow bit.
48
49   ANOTHER STEP SIDEWAYS
50       It's not enough to say: "that will make it run faster", you have to
51       check it.  Rerun the code under control of the benchmarking or
52       profiling modules, from the first step above, and check that the new
53       code executed the same task in less time.  Save your work and repeat...
54

GENERAL GUIDELINES

56       The critical thing when considering performance is to remember there is
57       no such thing as a "Golden Bullet", which is why there are no rules,
58       only guidelines.
59
60       It is clear that inline code is going to be faster than subroutine or
61       method calls, because there is less overhead, but this approach has the
62       disadvantage of being less maintainable and comes at the cost of
63       greater memory usage - there is no such thing as a free lunch.  If you
64       are searching for an element in a list, it can be more efficient to
65       store the data in a hash structure, and then simply look to see whether
66       the key is defined, rather than to loop through the entire array using
67       grep() for instance.  substr() may be (a lot) faster than grep() but
68       not as flexible, so you have another trade-off to access.  Your code
69       may contain a line which takes 0.01 of a second to execute which if you
70       call it 1,000 times, quite likely in a program parsing even medium
71       sized files for instance, you already have a 10 second delay, in just
72       one single code location, and if you call that line 100,000 times, your
73       entire program will slow down to an unbearable crawl.
74
75       Using a subroutine as part of your sort is a powerful way to get
76       exactly what you want, but will usually be slower than the built-in
77       alphabetic "cmp" and numeric "<=>" sort operators.  It is possible to
78       make multiple passes over your data, building indices to make the
79       upcoming sort more efficient, and to use what is known as the "OM"
80       (Orcish Maneuver) to cache the sort keys in advance.  The cache lookup,
81       while a good idea, can itself be a source of slowdown by enforcing a
82       double pass over the data - once to setup the cache, and once to sort
83       the data.  Using "pack()" to extract the required sort key into a
84       consistent string can be an efficient way to build a single string to
85       compare, instead of using multiple sort keys, which makes it possible
86       to use the standard, written in "c" and fast, perl "sort()" function on
87       the output, and is the basis of the "GRT" (Guttman Rossler Transform).
88       Some string combinations can slow the "GRT" down, by just being too
89       plain complex for its own good.
90
91       For applications using database backends, the standard "DBIx" namespace
92       has tries to help with keeping things nippy, not least because it tries
93       to not query the database until the latest possible moment, but always
94       read the docs which come with your choice of libraries.  Among the many
95       issues facing developers dealing with databases should remain aware of
96       is to always use "SQL" placeholders and to consider pre-fetching data
97       sets when this might prove advantageous.  Splitting up a large file by
98       assigning multiple processes to parsing a single file, using say "POE",
99       "threads" or "fork" can also be a useful way of optimizing your usage
100       of the available "CPU" resources, though this technique is fraught with
101       concurrency issues and demands high attention to detail.
102
103       Every case has a specific application and one or more exceptions, and
104       there is no replacement for running a few tests and finding out which
105       method works best for your particular environment, this is why writing
106       optimal code is not an exact science, and why we love using Perl so
107       much - TMTOWTDI.
108

BENCHMARKS

110       Here are a few examples to demonstrate usage of Perl's benchmarking
111       tools.
112
113   Assigning and Dereferencing Variables.
114       I'm sure most of us have seen code which looks like, (or worse than),
115       this:
116
117        if ( $obj->{_ref}->{_myscore} >= $obj->{_ref}->{_yourscore} ) {
118            ...
119
120       This sort of code can be a real eyesore to read, as well as being very
121       sensitive to typos, and it's much clearer to dereference the variable
122       explicitly.  We're side-stepping the issue of working with object-
123       oriented programming techniques to encapsulate variable access via
124       methods, only accessible through an object.  Here we're just discussing
125       the technical implementation of choice, and whether this has an effect
126       on performance.  We can see whether this dereferencing operation, has
127       any overhead by putting comparative code in a file and running a
128       "Benchmark" test.
129
130       # dereference
131
132        #!/usr/bin/perl
133
134        use strict;
135        use warnings;
136
137        use Benchmark;
138
139        my $ref = {
140                'ref'   => {
141                    _myscore    => '100 + 1',
142                    _yourscore  => '102 - 1',
143                },
144        };
145
146        timethese(1000000, {
147                'direct'       => sub {
148                  my $x = $ref->{ref}->{_myscore} . $ref->{ref}->{_yourscore} ;
149                },
150                'dereference'  => sub {
151                    my $ref  = $ref->{ref};
152                    my $myscore = $ref->{_myscore};
153                    my $yourscore = $ref->{_yourscore};
154                    my $x = $myscore . $yourscore;
155                },
156        });
157
158       It's essential to run any timing measurements a sufficient number of
159       times so the numbers settle on a numerical average, otherwise each run
160       will naturally fluctuate due to variations in the environment, to
161       reduce the effect of contention for "CPU" resources and network
162       bandwidth for instance.  Running the above code for one million
163       iterations, we can take a look at the report output by the "Benchmark"
164       module, to see which approach is the most effective.
165
166        $> perl dereference
167
168        Benchmark: timing 1000000 iterations of dereference, direct...
169        dereference:  2 wallclock secs ( 1.59 usr +  0.00 sys =  1.59 CPU) @ 628930.82/s (n=1000000)
170            direct:  1 wallclock secs ( 1.20 usr +  0.00 sys =  1.20 CPU) @ 833333.33/s (n=1000000)
171
172       The difference is clear to see and the dereferencing approach is
173       slower.  While it managed to execute an average of 628,930 times a
174       second during our test, the direct approach managed to run an
175       additional 204,403 times, unfortunately.  Unfortunately, because there
176       are many examples of code written using the multiple layer direct
177       variable access, and it's usually horrible.  It is, however, minusculy
178       faster.  The question remains whether the minute gain is actually worth
179       the eyestrain, or the loss of maintainability.
180
181   Search and replace or tr
182       If we have a string which needs to be modified, while a regex will
183       almost always be much more flexible, "tr", an oft underused tool, can
184       still be a useful.  One scenario might be replace all vowels with
185       another character.  The regex solution might look like this:
186
187        $str =~ s/[aeiou]/x/g
188
189       The "tr" alternative might look like this:
190
191        $str =~ tr/aeiou/xxxxx/
192
193       We can put that into a test file which we can run to check which
194       approach is the fastest, using a global $STR variable to assign to the
195       "my $str" variable so as to avoid perl trying to optimize any of the
196       work away by noticing it's assigned only the once.
197
198       # regex-transliterate
199
200        #!/usr/bin/perl
201
202        use strict;
203        use warnings;
204
205        use Benchmark;
206
207        my $STR = "$$-this and that";
208
209        timethese( 1000000, {
210        'sr'  => sub { my $str = $STR; $str =~ s/[aeiou]/x/g; return $str; },
211        'tr'  => sub { my $str = $STR; $str =~ tr/aeiou/xxxxx/; return $str; },
212        });
213
214       Running the code gives us our results:
215
216        $> perl regex-transliterate
217
218        Benchmark: timing 1000000 iterations of sr, tr...
219                sr:  2 wallclock secs ( 1.19 usr +  0.00 sys =  1.19 CPU) @ 840336.13/s (n=1000000)
220                tr:  0 wallclock secs ( 0.49 usr +  0.00 sys =  0.49 CPU) @ 2040816.33/s (n=1000000)
221
222       The "tr" version is a clear winner.  One solution is flexible, the
223       other is fast - and it's appropriately the programmer's choice which to
224       use.
225
226       Check the "Benchmark" docs for further useful techniques.
227

PROFILING TOOLS

229       A slightly larger piece of code will provide something on which a
230       profiler can produce more extensive reporting statistics.  This example
231       uses the simplistic "wordmatch" program which parses a given input file
232       and spews out a short report on the contents.
233
234       # wordmatch
235
236        #!/usr/bin/perl
237
238        use strict;
239        use warnings;
240
241        =head1 NAME
242
243        filewords - word analysis of input file
244
245        =head1 SYNOPSIS
246
247            filewords -f inputfilename [-d]
248
249        =head1 DESCRIPTION
250
251        This program parses the given filename, specified with C<-f>, and
252        displays a simple analysis of the words found therein.  Use the C<-d>
253        switch to enable debugging messages.
254
255        =cut
256
257        use FileHandle;
258        use Getopt::Long;
259
260        my $debug   =  0;
261        my $file    = '';
262
263        my $result = GetOptions (
264            'debug'         => \$debug,
265            'file=s'        => \$file,
266        );
267        die("invalid args") unless $result;
268
269        unless ( -f $file ) {
270            die("Usage: $0 -f filename [-d]");
271        }
272        my $FH = FileHandle->new("< $file")
273                                      or die("unable to open file($file): $!");
274
275        my $i_LINES = 0;
276        my $i_WORDS = 0;
277        my %count   = ();
278
279        my @lines = <$FH>;
280        foreach my $line ( @lines ) {
281            $i_LINES++;
282            $line =~ s/\n//;
283            my @words = split(/ +/, $line);
284            my $i_words = scalar(@words);
285            $i_WORDS = $i_WORDS + $i_words;
286            debug("line: $i_LINES supplying $i_words words: @words");
287            my $i_word = 0;
288            foreach my $word ( @words ) {
289                $i_word++;
290                $count{$i_LINES}{spec} += matches($i_word, $word,
291                                                  '[^a-zA-Z0-9]');
292                $count{$i_LINES}{only} += matches($i_word, $word,
293                                                  '^[^a-zA-Z0-9]+$');
294                $count{$i_LINES}{cons} += matches($i_word, $word,
295                                            '^[(?i:bcdfghjklmnpqrstvwxyz)]+$');
296                $count{$i_LINES}{vows} += matches($i_word, $word,
297                                                  '^[(?i:aeiou)]+$');
298                $count{$i_LINES}{caps} += matches($i_word, $word,
299                                                  '^[(A-Z)]+$');
300            }
301        }
302
303        print report( %count );
304
305        sub matches {
306            my $i_wd  = shift;
307            my $word  = shift;
308            my $regex = shift;
309            my $has = 0;
310
311            if ( $word =~ /($regex)/ ) {
312                $has++ if $1;
313            }
314
315            debug( "word: $i_wd "
316                  . ($has ? 'matches' : 'does not match')
317                  . " chars: /$regex/");
318
319            return $has;
320        }
321
322        sub report {
323            my %report = @_;
324            my %rep;
325
326            foreach my $line ( keys %report ) {
327                foreach my $key ( keys %{ $report{$line} } ) {
328                    $rep{$key} += $report{$line}{$key};
329                }
330            }
331
332            my $report = qq|
333        $0 report for $file:
334        lines in file: $i_LINES
335        words in file: $i_WORDS
336        words with special (non-word) characters: $i_spec
337        words with only special (non-word) characters: $i_only
338        words with only consonants: $i_cons
339        words with only capital letters: $i_caps
340        words with only vowels: $i_vows
341        |;
342
343            return $report;
344        }
345
346        sub debug {
347            my $message = shift;
348
349            if ( $debug ) {
350                print STDERR "DBG: $message\n";
351            }
352        }
353
354        exit 0;
355
356   Devel::DProf
357       This venerable module has been the de-facto standard for Perl code
358       profiling for more than a decade, but has been replaced by a number of
359       other modules which have brought us back to the 21st century.  Although
360       you're recommended to evaluate your tool from the several mentioned
361       here and from the CPAN list at the base of this document, (and
362       currently Devel::NYTProf seems to be the weapon of choice - see below),
363       we'll take a quick look at the output from Devel::DProf first, to set a
364       baseline for Perl profiling tools.  Run the above program under the
365       control of "Devel::DProf" by using the "-d" switch on the command-line.
366
367        $> perl -d:DProf wordmatch -f perl5db.pl
368
369        <...multiple lines snipped...>
370
371        wordmatch report for perl5db.pl:
372        lines in file: 9428
373        words in file: 50243
374        words with special (non-word) characters: 20480
375        words with only special (non-word) characters: 7790
376        words with only consonants: 4801
377        words with only capital letters: 1316
378        words with only vowels: 1701
379
380       "Devel::DProf" produces a special file, called tmon.out by default, and
381       this file is read by the "dprofpp" program, which is already installed
382       as part of the "Devel::DProf" distribution.  If you call "dprofpp" with
383       no options, it will read the tmon.out file in the current directory and
384       produce a human readable statistics report of the run of your program.
385       Note that this may take a little time.
386
387        $> dprofpp
388
389        Total Elapsed Time = 2.951677 Seconds
390          User+System Time = 2.871677 Seconds
391        Exclusive Times
392        %Time ExclSec CumulS #Calls sec/call Csec/c  Name
393         102.   2.945  3.003 251215   0.0000 0.0000  main::matches
394         2.40   0.069  0.069 260643   0.0000 0.0000  main::debug
395         1.74   0.050  0.050      1   0.0500 0.0500  main::report
396         1.04   0.030  0.049      4   0.0075 0.0123  main::BEGIN
397         0.35   0.010  0.010      3   0.0033 0.0033  Exporter::as_heavy
398         0.35   0.010  0.010      7   0.0014 0.0014  IO::File::BEGIN
399         0.00       - -0.000      1        -      -  Getopt::Long::FindOption
400         0.00       - -0.000      1        -      -  Symbol::BEGIN
401         0.00       - -0.000      1        -      -  Fcntl::BEGIN
402         0.00       - -0.000      1        -      -  Fcntl::bootstrap
403         0.00       - -0.000      1        -      -  warnings::BEGIN
404         0.00       - -0.000      1        -      -  IO::bootstrap
405         0.00       - -0.000      1        -      -  Getopt::Long::ConfigDefaults
406         0.00       - -0.000      1        -      -  Getopt::Long::Configure
407         0.00       - -0.000      1        -      -  Symbol::gensym
408
409       "dprofpp" will produce some quite detailed reporting on the activity of
410       the "wordmatch" program.  The wallclock, user and system, times are at
411       the top of the analysis, and after this are the main columns defining
412       which define the report.  Check the "dprofpp" docs for details of the
413       many options it supports.
414
415       See also "Apache::DProf" which hooks "Devel::DProf" into "mod_perl".
416
417   Devel::Profiler
418       Let's take a look at the same program using a different profiler:
419       "Devel::Profiler", a drop-in Perl-only replacement for "Devel::DProf".
420       The usage is very slightly different in that instead of using the
421       special "-d:" flag, you pull "Devel::Profiler" in directly as a module
422       using "-M".
423
424        $> perl -MDevel::Profiler wordmatch -f perl5db.pl
425
426        <...multiple lines snipped...>
427
428        wordmatch report for perl5db.pl:
429        lines in file: 9428
430        words in file: 50243
431        words with special (non-word) characters: 20480
432        words with only special (non-word) characters: 7790
433        words with only consonants: 4801
434        words with only capital letters: 1316
435        words with only vowels: 1701
436
437       "Devel::Profiler" generates a tmon.out file which is compatible with
438       the "dprofpp" program, thus saving the construction of a dedicated
439       statistics reader program.  "dprofpp" usage is therefore identical to
440       the above example.
441
442        $> dprofpp
443
444        Total Elapsed Time =   20.984 Seconds
445          User+System Time =   19.981 Seconds
446        Exclusive Times
447        %Time ExclSec CumulS #Calls sec/call Csec/c  Name
448         49.0   9.792 14.509 251215   0.0000 0.0001  main::matches
449         24.4   4.887  4.887 260643   0.0000 0.0000  main::debug
450         0.25   0.049  0.049      1   0.0490 0.0490  main::report
451         0.00   0.000  0.000      1   0.0000 0.0000  Getopt::Long::GetOptions
452         0.00   0.000  0.000      2   0.0000 0.0000  Getopt::Long::ParseOptionSpec
453         0.00   0.000  0.000      1   0.0000 0.0000  Getopt::Long::FindOption
454         0.00   0.000  0.000      1   0.0000 0.0000  IO::File::new
455         0.00   0.000  0.000      1   0.0000 0.0000  IO::Handle::new
456         0.00   0.000  0.000      1   0.0000 0.0000  Symbol::gensym
457         0.00   0.000  0.000      1   0.0000 0.0000  IO::File::open
458
459       Interestingly we get slightly different results, which is mostly
460       because the algorithm which generates the report is different, even
461       though the output file format was allegedly identical.  The elapsed,
462       user and system times are clearly showing the time it took for
463       "Devel::Profiler" to execute its own run, but the column listings feel
464       more accurate somehow than the ones we had earlier from "Devel::DProf".
465       The 102% figure has disappeared, for example.  This is where we have to
466       use the tools at our disposal, and recognise their pros and cons,
467       before using them.  Interestingly, the numbers of calls for each
468       subroutine are identical in the two reports, it's the percentages which
469       differ.  As the author of "Devel::Proviler" writes:
470
471        ...running HTML::Template's test suite under Devel::DProf shows
472        output() taking NO time but Devel::Profiler shows around 10% of the
473        time is in output().  I don't know which to trust but my gut tells me
474        something is wrong with Devel::DProf.  HTML::Template::output() is a
475        big routine that's called for every test. Either way, something needs
476        fixing.
477
478       YMMV.
479
480       See also "Devel::Apache::Profiler" which hooks "Devel::Profiler" into
481       "mod_perl".
482
483   Devel::SmallProf
484       The "Devel::SmallProf" profiler examines the runtime of your Perl
485       program and produces a line-by-line listing to show how many times each
486       line was called, and how long each line took to execute.  It is called
487       by supplying the familiar "-d" flag to Perl at runtime.
488
489        $> perl -d:SmallProf wordmatch -f perl5db.pl
490
491        <...multiple lines snipped...>
492
493        wordmatch report for perl5db.pl:
494        lines in file: 9428
495        words in file: 50243
496        words with special (non-word) characters: 20480
497        words with only special (non-word) characters: 7790
498        words with only consonants: 4801
499        words with only capital letters: 1316
500        words with only vowels: 1701
501
502       "Devel::SmallProf" writes it's output into a file called smallprof.out,
503       by default.  The format of the file looks like this:
504
505        <num> <time> <ctime> <line>:<text>
506
507       When the program has terminated, the output may be examined and sorted
508       using any standard text filtering utilities.  Something like the
509       following may be sufficient:
510
511        $> cat smallprof.out | grep \d*: | sort -k3 | tac | head -n20
512
513        251215   1.65674   7.68000    75: if ( $word =~ /($regex)/ ) {
514        251215   0.03264   4.40000    79: debug("word: $i_wd ".($has ? 'matches' :
515        251215   0.02693   4.10000    81: return $has;
516        260643   0.02841   4.07000   128: if ( $debug ) {
517        260643   0.02601   4.04000   126: my $message = shift;
518        251215   0.02641   3.91000    73: my $has = 0;
519        251215   0.03311   3.71000    70: my $i_wd  = shift;
520        251215   0.02699   3.69000    72: my $regex = shift;
521        251215   0.02766   3.68000    71: my $word  = shift;
522         50243   0.59726   1.00000    59:  $count{$i_LINES}{cons} =
523         50243   0.48175   0.92000    61:  $count{$i_LINES}{spec} =
524         50243   0.00644   0.89000    56:  my $i_cons = matches($i_word, $word,
525         50243   0.48837   0.88000    63:  $count{$i_LINES}{caps} =
526         50243   0.00516   0.88000    58:  my $i_caps = matches($i_word, $word, '^[(A-
527         50243   0.00631   0.81000    54:  my $i_spec = matches($i_word, $word, '[^a-
528         50243   0.00496   0.80000    57:  my $i_vows = matches($i_word, $word,
529         50243   0.00688   0.80000    53:  $i_word++;
530         50243   0.48469   0.79000    62:  $count{$i_LINES}{only} =
531         50243   0.48928   0.77000    60:  $count{$i_LINES}{vows} =
532         50243   0.00683   0.75000    55:  my $i_only = matches($i_word, $word, '^[^a-
533
534       You can immediately see a slightly different focus to the subroutine
535       profiling modules, and we start to see exactly which line of code is
536       taking the most time.  That regex line is looking a bit suspicious, for
537       example.  Remember that these tools are supposed to be used together,
538       there is no single best way to profile your code, you need to use the
539       best tools for the job.
540
541       See also "Apache::SmallProf" which hooks "Devel::SmallProf" into
542       "mod_perl".
543
544   Devel::FastProf
545       "Devel::FastProf" is another Perl line profiler.  This was written with
546       a view to getting a faster line profiler, than is possible with for
547       example "Devel::SmallProf", because it's written in "C".  To use
548       "Devel::FastProf", supply the "-d" argument to Perl:
549
550        $> perl -d:FastProf wordmatch -f perl5db.pl
551
552        <...multiple lines snipped...>
553
554        wordmatch report for perl5db.pl:
555        lines in file: 9428
556        words in file: 50243
557        words with special (non-word) characters: 20480
558        words with only special (non-word) characters: 7790
559        words with only consonants: 4801
560        words with only capital letters: 1316
561        words with only vowels: 1701
562
563       "Devel::FastProf" writes statistics to the file fastprof.out in the
564       current directory.  The output file, which can be specified, can be
565       interpreted by using the "fprofpp" command-line program.
566
567        $> fprofpp | head -n20
568
569        # fprofpp output format is:
570        # filename:line time count: source
571        wordmatch:75 3.93338 251215: if ( $word =~ /($regex)/ ) {
572        wordmatch:79 1.77774 251215: debug("word: $i_wd ".($has ? 'matches' : 'does not match')." chars: /$regex/");
573        wordmatch:81 1.47604 251215: return $has;
574        wordmatch:126 1.43441 260643: my $message = shift;
575        wordmatch:128 1.42156 260643: if ( $debug ) {
576        wordmatch:70 1.36824 251215: my $i_wd  = shift;
577        wordmatch:71 1.36739 251215: my $word  = shift;
578        wordmatch:72 1.35939 251215: my $regex = shift;
579
580       Straightaway we can see that the number of times each line has been
581       called is identical to the "Devel::SmallProf" output, and the sequence
582       is only very slightly different based on the ordering of the amount of
583       time each line took to execute, "if ( $debug ) { " and "my $message =
584       shift;", for example.  The differences in the actual times recorded
585       might be in the algorithm used internally, or it could be due to system
586       resource limitations or contention.
587
588       See also the DBIx::Profile which will profile database queries running
589       under the "DBIx::*" namespace.
590
591   Devel::NYTProf
592       "Devel::NYTProf" is the next generation of Perl code profiler, fixing
593       many shortcomings in other tools and implementing many cool features.
594       First of all it can be used as either a line profiler, a block or a
595       subroutine profiler, all at once.  It can also use sub-microsecond
596       (100ns) resolution on systems which provide "clock_gettime()".  It can
597       be started and stopped even by the program being profiled.  It's a one-
598       line entry to profile "mod_perl" applications.  It's written in "c" and
599       is probably the fastest profiler available for Perl.  The list of
600       coolness just goes on.  Enough of that, let's see how to it works -
601       just use the familiar "-d" switch to plug it in and run the code.
602
603        $> perl -d:NYTProf wordmatch -f perl5db.pl
604
605        wordmatch report for perl5db.pl:
606        lines in file: 9427
607        words in file: 50243
608        words with special (non-word) characters: 20480
609        words with only special (non-word) characters: 7790
610        words with only consonants: 4801
611        words with only capital letters: 1316
612        words with only vowels: 1701
613
614       "NYTProf" will generate a report database into the file nytprof.out by
615       default.  Human readable reports can be generated from here by using
616       the supplied "nytprofhtml" (HTML output) and "nytprofcsv" (CSV output)
617       programs.  We've used the Unix system "html2text" utility to convert
618       the nytprof/index.html file for convenience here.
619
620        $> html2text nytprof/index.html
621
622        Performance Profile Index
623        For wordmatch
624          Run on Fri Sep 26 13:46:39 2008
625        Reported on Fri Sep 26 13:47:23 2008
626
627                 Top 15 Subroutines -- ordered by exclusive time
628        |Calls |P |F |Inclusive|Exclusive|Subroutine                          |
629        |      |  |  |Time     |Time     |                                    |
630        |251215|5 |1 |13.09263 |10.47692 |main::              |matches        |
631        |260642|2 |1 |2.71199  |2.71199  |main::              |debug          |
632        |1     |1 |1 |0.21404  |0.21404  |main::              |report         |
633        |2     |2 |2 |0.00511  |0.00511  |XSLoader::          |load (xsub)    |
634        |14    |14|7 |0.00304  |0.00298  |Exporter::          |import         |
635        |3     |1 |1 |0.00265  |0.00254  |Exporter::          |as_heavy       |
636        |10    |10|4 |0.00140  |0.00140  |vars::              |import         |
637        |13    |13|1 |0.00129  |0.00109  |constant::          |import         |
638        |1     |1 |1 |0.00360  |0.00096  |FileHandle::        |import         |
639        |3     |3 |3 |0.00086  |0.00074  |warnings::register::|import         |
640        |9     |3 |1 |0.00036  |0.00036  |strict::            |bits           |
641        |13    |13|13|0.00032  |0.00029  |strict::            |import         |
642        |2     |2 |2 |0.00020  |0.00020  |warnings::          |import         |
643        |2     |1 |1 |0.00020  |0.00020  |Getopt::Long::      |ParseOptionSpec|
644        |7     |7 |6 |0.00043  |0.00020  |strict::            |unimport       |
645
646        For more information see the full list of 189 subroutines.
647
648       The first part of the report already shows the critical information
649       regarding which subroutines are using the most time.  The next gives
650       some statistics about the source files profiled.
651
652                Source Code Files -- ordered by exclusive time then name
653        |Stmts  |Exclusive|Avg.   |Reports                     |Source File         |
654        |       |Time     |       |                            |                    |
655        |2699761|15.66654 |6e-06  |line   .    block   .    sub|wordmatch           |
656        |35     |0.02187  |0.00062|line   .    block   .    sub|IO/Handle.pm        |
657        |274    |0.01525  |0.00006|line   .    block   .    sub|Getopt/Long.pm      |
658        |20     |0.00585  |0.00029|line   .    block   .    sub|Fcntl.pm            |
659        |128    |0.00340  |0.00003|line   .    block   .    sub|Exporter/Heavy.pm   |
660        |42     |0.00332  |0.00008|line   .    block   .    sub|IO/File.pm          |
661        |261    |0.00308  |0.00001|line   .    block   .    sub|Exporter.pm         |
662        |323    |0.00248  |8e-06  |line   .    block   .    sub|constant.pm         |
663        |12     |0.00246  |0.00021|line   .    block   .    sub|File/Spec/Unix.pm   |
664        |191    |0.00240  |0.00001|line   .    block   .    sub|vars.pm             |
665        |77     |0.00201  |0.00003|line   .    block   .    sub|FileHandle.pm       |
666        |12     |0.00198  |0.00016|line   .    block   .    sub|Carp.pm             |
667        |14     |0.00175  |0.00013|line   .    block   .    sub|Symbol.pm           |
668        |15     |0.00130  |0.00009|line   .    block   .    sub|IO.pm               |
669        |22     |0.00120  |0.00005|line   .    block   .    sub|IO/Seekable.pm      |
670        |198    |0.00085  |4e-06  |line   .    block   .    sub|warnings/register.pm|
671        |114    |0.00080  |7e-06  |line   .    block   .    sub|strict.pm           |
672        |47     |0.00068  |0.00001|line   .    block   .    sub|warnings.pm         |
673        |27     |0.00054  |0.00002|line   .    block   .    sub|overload.pm         |
674        |9      |0.00047  |0.00005|line   .    block   .    sub|SelectSaver.pm      |
675        |13     |0.00045  |0.00003|line   .    block   .    sub|File/Spec.pm        |
676        |2701595|15.73869 |       |Total                       |
677        |128647 |0.74946  |       |Average                     |
678        |       |0.00201  |0.00003|Median                      |
679        |       |0.00121  |0.00003|Deviation                   |
680
681        Report produced by the NYTProf 2.03 Perl profiler, developed by Tim Bunce and
682        Adam Kaplan.
683
684       At this point, if you're using the html report, you can click through
685       the various links to bore down into each subroutine and each line of
686       code.  Because we're using the text reporting here, and there's a whole
687       directory full of reports built for each source file, we'll just
688       display a part of the corresponding wordmatch-line.html file,
689       sufficient to give an idea of the sort of output you can expect from
690       this cool tool.
691
692        $> html2text nytprof/wordmatch-line.html
693
694        Performance Profile -- -block view-.-line view-.-sub view-
695        For wordmatch
696        Run on Fri Sep 26 13:46:39 2008
697        Reported on Fri Sep 26 13:47:22 2008
698
699        File wordmatch
700
701         Subroutines -- ordered by exclusive time
702        |Calls |P|F|Inclusive|Exclusive|Subroutine    |
703        |      | | |Time     |Time     |              |
704        |251215|5|1|13.09263 |10.47692 |main::|matches|
705        |260642|2|1|2.71199  |2.71199  |main::|debug  |
706        |1     |1|1|0.21404  |0.21404  |main::|report |
707        |0     |0|0|0        |0        |main::|BEGIN  |
708
709
710        |Line|Stmts.|Exclusive|Avg.   |Code                                           |
711        |    |      |Time     |       |                                               |
712        |1   |      |         |       |#!/usr/bin/perl                                |
713        |2   |      |         |       |                                               |
714        |    |      |         |       |use strict;                                    |
715        |3   |3     |0.00086  |0.00029|# spent 0.00003s making 1 calls to strict::    |
716        |    |      |         |       |import                                         |
717        |    |      |         |       |use warnings;                                  |
718        |4   |3     |0.01563  |0.00521|# spent 0.00012s making 1 calls to warnings::  |
719        |    |      |         |       |import                                         |
720        |5   |      |         |       |                                               |
721        |6   |      |         |       |=head1 NAME                                    |
722        |7   |      |         |       |                                               |
723        |8   |      |         |       |filewords - word analysis of input file        |
724        <...snip...>
725        |62  |1     |0.00445  |0.00445|print report( %count );                        |
726        |    |      |         |       |# spent 0.21404s making 1 calls to main::report|
727        |63  |      |         |       |                                               |
728        |    |      |         |       |# spent 23.56955s (10.47692+2.61571) within    |
729        |    |      |         |       |main::matches which was called 251215 times,   |
730        |    |      |         |       |avg 0.00005s/call: # 50243 times               |
731        |    |      |         |       |(2.12134+0.51939s) at line 57 of wordmatch, avg|
732        |    |      |         |       |0.00005s/call # 50243 times (2.17735+0.54550s) |
733        |64  |      |         |       |at line 56 of wordmatch, avg 0.00005s/call #   |
734        |    |      |         |       |50243 times (2.10992+0.51797s) at line 58 of   |
735        |    |      |         |       |wordmatch, avg 0.00005s/call # 50243 times     |
736        |    |      |         |       |(2.12696+0.51598s) at line 55 of wordmatch, avg|
737        |    |      |         |       |0.00005s/call # 50243 times (1.94134+0.51687s) |
738        |    |      |         |       |at line 54 of wordmatch, avg 0.00005s/call     |
739        |    |      |         |       |sub matches {                                  |
740        <...snip...>
741        |102 |      |         |       |                                               |
742        |    |      |         |       |# spent 2.71199s within main::debug which was  |
743        |    |      |         |       |called 260642 times, avg 0.00001s/call: #      |
744        |    |      |         |       |251215 times (2.61571+0s) by main::matches at  |
745        |103 |      |         |       |line 74 of wordmatch, avg 0.00001s/call # 9427 |
746        |    |      |         |       |times (0.09628+0s) at line 50 of wordmatch, avg|
747        |    |      |         |       |0.00001s/call                                  |
748        |    |      |         |       |sub debug {                                    |
749        |104 |260642|0.58496  |2e-06  |my $message = shift;                           |
750        |105 |      |         |       |                                               |
751        |106 |260642|1.09917  |4e-06  |if ( $debug ) {                                |
752        |107 |      |         |       |print STDERR "DBG: $message\n";                |
753        |108 |      |         |       |}                                              |
754        |109 |      |         |       |}                                              |
755        |110 |      |         |       |                                               |
756        |111 |1     |0.01501  |0.01501|exit 0;                                        |
757        |112 |      |         |       |                                               |
758
759       Oodles of very useful information in there - this seems to be the way
760       forward.
761
762       See also "Devel::NYTProf::Apache" which hooks "Devel::NYTProf" into
763       "mod_perl".
764

SORTING

766       Perl modules are not the only tools a performance analyst has at their
767       disposal, system tools like "time" should not be overlooked as the next
768       example shows, where we take a quick look at sorting.  Many books,
769       theses and articles, have been written about efficient sorting
770       algorithms, and this is not the place to repeat such work, there's
771       several good sorting modules which deserve taking a look at too:
772       "Sort::Maker", "Sort::Key" spring to mind.  However, it's still
773       possible to make some observations on certain Perl specific
774       interpretations on issues relating to sorting data sets and give an
775       example or two with regard to how sorting large data volumes can effect
776       performance.  Firstly, an often overlooked point when sorting large
777       amounts of data, one can attempt to reduce the data set to be dealt
778       with and in many cases "grep()" can be quite useful as a simple filter:
779
780        @data = sort grep { /$filter/ } @incoming
781
782       A command such as this can vastly reduce the volume of material to
783       actually sort through in the first place, and should not be too lightly
784       disregarded purely on the basis of its simplicity.  The "KISS"
785       principle is too often overlooked - the next example uses the simple
786       system "time" utility to demonstrate.  Let's take a look at an actual
787       example of sorting the contents of a large file, an apache logfile
788       would do.  This one has over a quarter of a million lines, is 50M in
789       size, and a snippet of it looks like this:
790
791       # logfile
792
793        188.209-65-87.adsl-dyn.isp.belgacom.be - - [08/Feb/2007:12:57:16 +0000] "GET /favicon.ico HTTP/1.1" 404 209 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)"
794        188.209-65-87.adsl-dyn.isp.belgacom.be - - [08/Feb/2007:12:57:16 +0000] "GET /favicon.ico HTTP/1.1" 404 209 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)"
795        151.56.71.198 - - [08/Feb/2007:12:57:41 +0000] "GET /suse-on-vaio.html HTTP/1.1" 200 2858 "http://www.linux-on-laptops.com/sony.html" "Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US; rv:1.8.1.1) Gecko/20061204 Firefox/2.0.0.1"
796        151.56.71.198 - - [08/Feb/2007:12:57:42 +0000] "GET /data/css HTTP/1.1" 404 206 "http://www.rfi.net/suse-on-vaio.html" "Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US; rv:1.8.1.1) Gecko/20061204 Firefox/2.0.0.1"
797        151.56.71.198 - - [08/Feb/2007:12:57:43 +0000] "GET /favicon.ico HTTP/1.1" 404 209 "-" "Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US; rv:1.8.1.1) Gecko/20061204 Firefox/2.0.0.1"
798        217.113.68.60 - - [08/Feb/2007:13:02:15 +0000] "GET / HTTP/1.1" 304 - "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)"
799        217.113.68.60 - - [08/Feb/2007:13:02:16 +0000] "GET /data/css HTTP/1.1" 404 206 "http://www.rfi.net/" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)"
800        debora.to.isac.cnr.it - - [08/Feb/2007:13:03:58 +0000] "GET /suse-on-vaio.html HTTP/1.1" 200 2858 "http://www.linux-on-laptops.com/sony.html" "Mozilla/5.0 (compatible; Konqueror/3.4; Linux) KHTML/3.4.0 (like Gecko)"
801        debora.to.isac.cnr.it - - [08/Feb/2007:13:03:58 +0000] "GET /data/css HTTP/1.1" 404 206 "http://www.rfi.net/suse-on-vaio.html" "Mozilla/5.0 (compatible; Konqueror/3.4; Linux) KHTML/3.4.0 (like Gecko)"
802        debora.to.isac.cnr.it - - [08/Feb/2007:13:03:58 +0000] "GET /favicon.ico HTTP/1.1" 404 209 "-" "Mozilla/5.0 (compatible; Konqueror/3.4; Linux) KHTML/3.4.0 (like Gecko)"
803        195.24.196.99 - - [08/Feb/2007:13:26:48 +0000] "GET / HTTP/1.0" 200 3309 "-" "Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.8.0.9) Gecko/20061206 Firefox/1.5.0.9"
804        195.24.196.99 - - [08/Feb/2007:13:26:58 +0000] "GET /data/css HTTP/1.0" 404 206 "http://www.rfi.net/" "Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.8.0.9) Gecko/20061206 Firefox/1.5.0.9"
805        195.24.196.99 - - [08/Feb/2007:13:26:59 +0000] "GET /favicon.ico HTTP/1.0" 404 209 "-" "Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.8.0.9) Gecko/20061206 Firefox/1.5.0.9"
806        crawl1.cosmixcorp.com - - [08/Feb/2007:13:27:57 +0000] "GET /robots.txt HTTP/1.0" 200 179 "-" "voyager/1.0"
807        crawl1.cosmixcorp.com - - [08/Feb/2007:13:28:25 +0000] "GET /links.html HTTP/1.0" 200 3413 "-" "voyager/1.0"
808        fhm226.internetdsl.tpnet.pl - - [08/Feb/2007:13:37:32 +0000] "GET /suse-on-vaio.html HTTP/1.1" 200 2858 "http://www.linux-on-laptops.com/sony.html" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)"
809        fhm226.internetdsl.tpnet.pl - - [08/Feb/2007:13:37:34 +0000] "GET /data/css HTTP/1.1" 404 206 "http://www.rfi.net/suse-on-vaio.html" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)"
810        80.247.140.134 - - [08/Feb/2007:13:57:35 +0000] "GET / HTTP/1.1" 200 3309 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; .NET CLR 1.1.4322)"
811        80.247.140.134 - - [08/Feb/2007:13:57:37 +0000] "GET /data/css HTTP/1.1" 404 206 "http://www.rfi.net" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; .NET CLR 1.1.4322)"
812        pop.compuscan.co.za - - [08/Feb/2007:14:10:43 +0000] "GET / HTTP/1.1" 200 3309 "-" "www.clamav.net"
813        livebot-207-46-98-57.search.live.com - - [08/Feb/2007:14:12:04 +0000] "GET /robots.txt HTTP/1.0" 200 179 "-" "msnbot/1.0 (+http://search.msn.com/msnbot.htm)"
814        livebot-207-46-98-57.search.live.com - - [08/Feb/2007:14:12:04 +0000] "GET /html/oracle.html HTTP/1.0" 404 214 "-" "msnbot/1.0 (+http://search.msn.com/msnbot.htm)"
815        dslb-088-064-005-154.pools.arcor-ip.net - - [08/Feb/2007:14:12:15 +0000] "GET / HTTP/1.1" 200 3309 "-" "www.clamav.net"
816        196.201.92.41 - - [08/Feb/2007:14:15:01 +0000] "GET / HTTP/1.1" 200 3309 "-" "MOT-L7/08.B7.DCR MIB/2.2.1 Profile/MIDP-2.0 Configuration/CLDC-1.1"
817
818       The specific task here is to sort the 286,525 lines of this file by
819       Response Code, Query, Browser, Referring Url, and lastly Date.  One
820       solution might be to use the following code, which iterates over the
821       files given on the command-line.
822
823       # sort-apache-log
824
825        #!/usr/bin/perl -n
826
827        use strict;
828        use warnings;
829
830        my @data;
831
832        LINE:
833        while ( <> ) {
834            my $line = $_;
835            if (
836                $line =~ m/^(
837                    ([\w\.\-]+)             # client
838                    \s*-\s*-\s*\[
839                    ([^]]+)                 # date
840                    \]\s*"\w+\s*
841                    (\S+)                   # query
842                    [^"]+"\s*
843                    (\d+)                   # status
844                    \s+\S+\s+"[^"]*"\s+"
845                    ([^"]*)                 # browser
846                    "
847                    .*
848                )$/x
849            ) {
850                my @chunks = split(/ +/, $line);
851                my $ip      = $1;
852                my $date    = $2;
853                my $query   = $3;
854                my $status  = $4;
855                my $browser = $5;
856
857                push(@data, [$ip, $date, $query, $status, $browser, $line]);
858            }
859        }
860
861        my @sorted = sort {
862            $a->[3] cmp $b->[3]
863                    ||
864            $a->[2] cmp $b->[2]
865                    ||
866            $a->[0] cmp $b->[0]
867                    ||
868            $a->[1] cmp $b->[1]
869                    ||
870            $a->[4] cmp $b->[4]
871        } @data;
872
873        foreach my $data ( @sorted ) {
874            print $data->[5];
875        }
876
877        exit 0;
878
879       When running this program, redirect "STDOUT" so it is possible to check
880       the output is correct from following test runs and use the system
881       "time" utility to check the overall runtime.
882
883        $> time ./sort-apache-log logfile > out-sort
884
885        real    0m17.371s
886        user    0m15.757s
887        sys     0m0.592s
888
889       The program took just over 17 wallclock seconds to run.  Note the
890       different values "time" outputs, it's important to always use the same
891       one, and to not confuse what each one means.
892
893       Elapsed Real Time
894           The overall, or wallclock, time between when "time" was called, and
895           when it terminates.  The elapsed time includes both user and system
896           times, and time spent waiting for other users and processes on the
897           system.  Inevitably, this is the most approximate of the
898           measurements given.
899
900       User CPU Time
901           The user time is the amount of time the entire process spent on
902           behalf of the user on this system executing this program.
903
904       System CPU Time
905           The system time is the amount of time the kernel itself spent
906           executing routines, or system calls, on behalf of this process
907           user.
908
909       Running this same process as a "Schwarzian Transform" it is possible to
910       eliminate the input and output arrays for storing all the data, and
911       work on the input directly as it arrives too.  Otherwise, the code
912       looks fairly similar:
913
914       # sort-apache-log-schwarzian
915
916        #!/usr/bin/perl -n
917
918        use strict;
919        use warnings;
920
921        print
922
923            map $_->[0] =>
924
925            sort {
926                $a->[4] cmp $b->[4]
927                        ||
928                $a->[3] cmp $b->[3]
929                        ||
930                $a->[1] cmp $b->[1]
931                        ||
932                $a->[2] cmp $b->[2]
933                        ||
934                $a->[5] cmp $b->[5]
935            }
936            map  [ $_, m/^(
937                ([\w\.\-]+)             # client
938                \s*-\s*-\s*\[
939                ([^]]+)                 # date
940                \]\s*"\w+\s*
941                (\S+)                   # query
942                [^"]+"\s*
943                (\d+)                   # status
944                \s+\S+\s+"[^"]*"\s+"
945                ([^"]*)                 # browser
946                "
947                .*
948            )$/xo ]
949
950            => <>;
951
952        exit 0;
953
954       Run the new code against the same logfile, as above, to check the new
955       time.
956
957        $> time ./sort-apache-log-schwarzian logfile > out-schwarz
958
959        real    0m9.664s
960        user    0m8.873s
961        sys     0m0.704s
962
963       The time has been cut in half, which is a respectable speed improvement
964       by any standard.  Naturally, it is important to check the output is
965       consistent with the first program run, this is where the Unix system
966       "cksum" utility comes in.
967
968        $> cksum out-sort out-schwarz
969        3044173777 52029194 out-sort
970        3044173777 52029194 out-schwarz
971
972       BTW. Beware too of pressure from managers who see you speed a program
973       up by 50% of the runtime once, only to get a request one month later to
974       do the same again (true story) - you'll just have to point out you're
975       only human, even if you are a Perl programmer, and you'll see what you
976       can do...
977

LOGGING

979       An essential part of any good development process is appropriate error
980       handling with appropriately informative messages, however there exists
981       a school of thought which suggests that log files should be chatty, as
982       if the chain of unbroken output somehow ensures the survival of the
983       program.  If speed is in any way an issue, this approach is wrong.
984
985       A common sight is code which looks something like this:
986
987        logger->debug( "A logging message via process-id: $$ INC: "
988                                                              . Dumper(\%INC) )
989
990       The problem is that this code will always be parsed and executed, even
991       when the debug level set in the logging configuration file is zero.
992       Once the debug() subroutine has been entered, and the internal $debug
993       variable confirmed to be zero, for example, the message which has been
994       sent in will be discarded and the program will continue.  In the
995       example given though, the "\%INC" hash will already have been dumped,
996       and the message string constructed, all of which work could be bypassed
997       by a debug variable at the statement level, like this:
998
999        logger->debug( "A logging message via process-id: $$ INC: "
1000                                                   . Dumper(\%INC) ) if $DEBUG;
1001
1002       This effect can be demonstrated by setting up a test script with both
1003       forms, including a "debug()" subroutine to emulate typical "logger()"
1004       functionality.
1005
1006       # ifdebug
1007
1008        #!/usr/bin/perl
1009
1010        use strict;
1011        use warnings;
1012
1013        use Benchmark;
1014        use Data::Dumper;
1015        my $DEBUG = 0;
1016
1017        sub debug {
1018            my $msg = shift;
1019
1020            if ( $DEBUG ) {
1021                print "DEBUG: $msg\n";
1022            }
1023        };
1024
1025        timethese(100000, {
1026                'debug'       => sub {
1027                    debug( "A $0 logging message via process-id: $$" . Dumper(\%INC) )
1028                },
1029                'ifdebug'  => sub {
1030                    debug( "A $0 logging message via process-id: $$" . Dumper(\%INC) ) if $DEBUG
1031                },
1032        });
1033
1034       Let's see what "Benchmark" makes of this:
1035
1036        $> perl ifdebug
1037        Benchmark: timing 100000 iterations of constant, sub...
1038           ifdebug:  0 wallclock secs ( 0.01 usr +  0.00 sys =  0.01 CPU) @ 10000000.00/s (n=100000)
1039                    (warning: too few iterations for a reliable count)
1040             debug: 14 wallclock secs (13.18 usr +  0.04 sys = 13.22 CPU) @ 7564.30/s (n=100000)
1041
1042       In the one case the code, which does exactly the same thing as far as
1043       outputting any debugging information is concerned, in other words
1044       nothing, takes 14 seconds, and in the other case the code takes one
1045       hundredth of a second.  Looks fairly definitive.  Use a $DEBUG variable
1046       BEFORE you call the subroutine, rather than relying on the smart
1047       functionality inside it.
1048
1049   Logging if DEBUG (constant)
1050       It's possible to take the previous idea a little further, by using a
1051       compile time "DEBUG" constant.
1052
1053       # ifdebug-constant
1054
1055        #!/usr/bin/perl
1056
1057        use strict;
1058        use warnings;
1059
1060        use Benchmark;
1061        use Data::Dumper;
1062        use constant
1063            DEBUG => 0
1064        ;
1065
1066        sub debug {
1067            if ( DEBUG ) {
1068                my $msg = shift;
1069                print "DEBUG: $msg\n";
1070            }
1071        };
1072
1073        timethese(100000, {
1074                'debug'       => sub {
1075                    debug( "A $0 logging message via process-id: $$" . Dumper(\%INC) )
1076                },
1077                'constant'  => sub {
1078                    debug( "A $0 logging message via process-id: $$" . Dumper(\%INC) ) if DEBUG
1079                },
1080        });
1081
1082       Running this program produces the following output:
1083
1084        $> perl ifdebug-constant
1085        Benchmark: timing 100000 iterations of constant, sub...
1086          constant:  0 wallclock secs (-0.00 usr +  0.00 sys = -0.00 CPU) @ -7205759403792793600000.00/s (n=100000)
1087                    (warning: too few iterations for a reliable count)
1088               sub: 14 wallclock secs (13.09 usr +  0.00 sys = 13.09 CPU) @ 7639.42/s (n=100000)
1089
1090       The "DEBUG" constant wipes the floor with even the $debug variable,
1091       clocking in at minus zero seconds, and generates a "warning: too few
1092       iterations for a reliable count" message into the bargain.  To see what
1093       is really going on, and why we had too few iterations when we thought
1094       we asked for 100000, we can use the very useful "B::Deparse" to inspect
1095       the new code:
1096
1097        $> perl -MO=Deparse ifdebug-constant
1098
1099        use Benchmark;
1100        use Data::Dumper;
1101        use constant ('DEBUG', 0);
1102        sub debug {
1103            use warnings;
1104            use strict 'refs';
1105            0;
1106        }
1107        use warnings;
1108        use strict 'refs';
1109        timethese(100000, {'sub', sub {
1110            debug "A $0 logging message via process-id: $$" . Dumper(\%INC);
1111        }
1112        , 'constant', sub {
1113            0;
1114        }
1115        });
1116        ifdebug-constant syntax OK
1117
1118       The output shows the constant() subroutine we're testing being replaced
1119       with the value of the "DEBUG" constant: zero.  The line to be tested
1120       has been completely optimized away, and you can't get much more
1121       efficient than that.
1122

POSTSCRIPT

1124       This document has provided several way to go about identifying hot-
1125       spots, and checking whether any modifications have improved the runtime
1126       of the code.
1127
1128       As a final thought, remember that it's not (at the time of writing)
1129       possible to produce a useful program which will run in zero or negative
1130       time and this basic principle can be written as: useful programs are
1131       slow by their very definition.  It is of course possible to write a
1132       nearly instantaneous program, but it's not going to do very much,
1133       here's a very efficient one:
1134
1135        $> perl -e 0
1136
1137       Optimizing that any further is a job for "p5p".
1138

SEE ALSO

1140       Further reading can be found using the modules and links below.
1141
1142   PERLDOCS
1143       For example: "perldoc -f sort".
1144
1145       perlfaq4.
1146
1147       perlfork, perlfunc, perlretut, perlthrtut.
1148
1149       threads.
1150
1151   MAN PAGES
1152       "time".
1153
1154   MODULES
1155       It's not possible to individually showcase all the performance related
1156       code for Perl here, naturally, but here's a short list of modules from
1157       the CPAN which deserve further attention.
1158
1159        Apache::DProf
1160        Apache::SmallProf
1161        Benchmark
1162        DBIx::Profile
1163        Devel::AutoProfiler
1164        Devel::DProf
1165        Devel::DProfLB
1166        Devel::FastProf
1167        Devel::GraphVizProf
1168        Devel::NYTProf
1169        Devel::NYTProf::Apache
1170        Devel::Profiler
1171        Devel::Profile
1172        Devel::Profit
1173        Devel::SmallProf
1174        Devel::WxProf
1175        POE::Devel::Profiler
1176        Sort::Key
1177        Sort::Maker
1178
1179   URLS
1180       Very useful online reference material:
1181
1182        http://www.ccl4.org/~nick/P/Fast_Enough/
1183
1184        http://www-128.ibm.com/developerworks/library/l-optperl.html
1185
1186        http://perlbuzz.com/2007/11/bind-output-variables-in-dbi-for-speed-and-safety.html
1187
1188        http://en.wikipedia.org/wiki/Performance_analysis
1189
1190        http://apache.perl.org/docs/1.0/guide/performance.html
1191
1192        http://perlgolf.sourceforge.net/
1193
1194        http://www.sysarch.com/Perl/sort_paper.html
1195

AUTHOR

1197       Richard Foley <richard.foley@rfi.net> Copyright (c) 2008
1198
1199
1200
1201perl v5.26.3                      2018-03-23                       PERLPERF(1)