1PERLPERF(1)            Perl Programmers Reference Guide            PERLPERF(1)
2
3
4

NAME

6       perlperf - Perl Performance and Optimization Techniques
7

DESCRIPTION

9       This is an introduction to the use of performance and optimization
10       techniques which can be used with particular reference to perl
11       programs.  While many perl developers have come from other languages,
12       and can use their prior knowledge where appropriate, there are many
13       other people who might benefit from a few perl specific pointers.  If
14       you want the condensed version, perhaps the best advice comes from the
15       renowned Japanese Samurai, Miyamoto Musashi, who said:
16
17        "Do Not Engage in Useless Activity"
18
19       in 1645.
20

OVERVIEW

22       Perhaps the most common mistake programmers make is to attempt to
23       optimize their code before a program actually does anything useful -
24       this is a bad idea.  There's no point in having an extremely fast
25       program that doesn't work.  The first job is to get a program to
26       correctly do something useful, (not to mention ensuring the test suite
27       is fully functional), and only then to consider optimizing it.  Having
28       decided to optimize existing working code, there are several simple but
29       essential steps to consider which are intrinsic to any optimization
30       process.
31
32   ONE STEP SIDEWAYS
33       Firstly, you need to establish a baseline time for the existing code,
34       which timing needs to be reliable and repeatable.  You'll probably want
35       to use the "Benchmark" or "Devel::NYTProf" modules, or something
36       similar, for this step, or perhaps the Unix system "time" utility,
37       whichever is appropriate.  See the base of this document for a longer
38       list of benchmarking and profiling modules, and recommended further
39       reading.
40
41   ONE STEP FORWARD
42       Next, having examined the program for hot spots, (places where the code
43       seems to run slowly), change the code with the intention of making it
44       run faster.  Using version control software, like "subversion", will
45       ensure no changes are irreversible.  It's too easy to fiddle here and
46       fiddle there - don't change too much at any one time or you might not
47       discover which piece of code really was the slow bit.
48
49   ANOTHER STEP SIDEWAYS
50       It's not enough to say: "that will make it run faster", you have to
51       check it.  Rerun the code under control of the benchmarking or
52       profiling modules, from the first step above, and check that the new
53       code executed the same task in less time.  Save your work and repeat...
54

GENERAL GUIDELINES

56       The critical thing when considering performance is to remember there is
57       no such thing as a "Golden Bullet", which is why there are no rules,
58       only guidelines.
59
60       It is clear that inline code is going to be faster than subroutine or
61       method calls, because there is less overhead, but this approach has the
62       disadvantage of being less maintainable and comes at the cost of
63       greater memory usage - there is no such thing as a free lunch.  If you
64       are searching for an element in a list, it can be more efficient to
65       store the data in a hash structure, and then simply look to see whether
66       the key is defined, rather than to loop through the entire array using
67       grep() for instance.  substr() may be (a lot) faster than grep() but
68       not as flexible, so you have another trade-off to access.  Your code
69       may contain a line which takes 0.01 of a second to execute which if you
70       call it 1,000 times, quite likely in a program parsing even medium
71       sized files for instance, you already have a 10 second delay, in just
72       one single code location, and if you call that line 100,000 times, your
73       entire program will slow down to an unbearable crawl.
74
75       Using a subroutine as part of your sort is a powerful way to get
76       exactly what you want, but will usually be slower than the built-in
77       alphabetic "cmp" and numeric "<=>" sort operators.  It is possible to
78       make multiple passes over your data, building indices to make the
79       upcoming sort more efficient, and to use what is known as the "OM"
80       (Orcish Maneuver) to cache the sort keys in advance.  The cache lookup,
81       while a good idea, can itself be a source of slowdown by enforcing a
82       double pass over the data - once to setup the cache, and once to sort
83       the data.  Using "pack()" to extract the required sort key into a
84       consistent string can be an efficient way to build a single string to
85       compare, instead of using multiple sort keys, which makes it possible
86       to use the standard, written in "c" and fast, perl "sort()" function on
87       the output, and is the basis of the "GRT" (Guttman Rossler Transform).
88       Some string combinations can slow the "GRT" down, by just being too
89       plain complex for its own good.
90
91       For applications using database backends, the standard "DBIx" namespace
92       has tries to help with keeping things nippy, not least because it tries
93       to not query the database until the latest possible moment, but always
94       read the docs which come with your choice of libraries.  Among the many
95       issues facing developers dealing with databases should remain aware of
96       is to always use "SQL" placeholders and to consider pre-fetching data
97       sets when this might prove advantageous.  Splitting up a large file by
98       assigning multiple processes to parsing a single file, using say "POE",
99       "threads" or "fork" can also be a useful way of optimizing your usage
100       of the available "CPU" resources, though this technique is fraught with
101       concurrency issues and demands high attention to detail.
102
103       Every case has a specific application and one or more exceptions, and
104       there is no replacement for running a few tests and finding out which
105       method works best for your particular environment, this is why writing
106       optimal code is not an exact science, and why we love using Perl so
107       much - TMTOWTDI.
108

BENCHMARKS

110       Here are a few examples to demonstrate usage of Perl's benchmarking
111       tools.
112
113   Assigning and Dereferencing Variables.
114       I'm sure most of us have seen code which looks like, (or worse than),
115       this:
116
117        if ( $obj->{_ref}->{_myscore} >= $obj->{_ref}->{_yourscore} ) {
118            ...
119
120       This sort of code can be a real eyesore to read, as well as being very
121       sensitive to typos, and it's much clearer to dereference the variable
122       explicitly.  We're side-stepping the issue of working with object-
123       oriented programming techniques to encapsulate variable access via
124       methods, only accessible through an object.  Here we're just discussing
125       the technical implementation of choice, and whether this has an effect
126       on performance.  We can see whether this dereferencing operation, has
127       any overhead by putting comparative code in a file and running a
128       "Benchmark" test.
129
130       # dereference
131
132        #!/usr/bin/perl
133
134        use v5.36;
135
136        use Benchmark;
137
138        my $ref = {
139                'ref'   => {
140                    _myscore    => '100 + 1',
141                    _yourscore  => '102 - 1',
142                },
143        };
144
145        timethese(1000000, {
146                'direct'       => sub {
147                  my $x = $ref->{ref}->{_myscore} . $ref->{ref}->{_yourscore} ;
148                },
149                'dereference'  => sub {
150                    my $ref  = $ref->{ref};
151                    my $myscore = $ref->{_myscore};
152                    my $yourscore = $ref->{_yourscore};
153                    my $x = $myscore . $yourscore;
154                },
155        });
156
157       It's essential to run any timing measurements a sufficient number of
158       times so the numbers settle on a numerical average, otherwise each run
159       will naturally fluctuate due to variations in the environment, to
160       reduce the effect of contention for "CPU" resources and network
161       bandwidth for instance.  Running the above code for one million
162       iterations, we can take a look at the report output by the "Benchmark"
163       module, to see which approach is the most effective.
164
165        $> perl dereference
166
167        Benchmark: timing 1000000 iterations of dereference, direct...
168        dereference:  2 wallclock secs ( 1.59 usr +  0.00 sys =  1.59 CPU) @ 628930.82/s (n=1000000)
169            direct:  1 wallclock secs ( 1.20 usr +  0.00 sys =  1.20 CPU) @ 833333.33/s (n=1000000)
170
171       The difference is clear to see and the dereferencing approach is
172       slower.  While it managed to execute an average of 628,930 times a
173       second during our test, the direct approach managed to run an
174       additional 204,403 times, unfortunately.  Unfortunately, because there
175       are many examples of code written using the multiple layer direct
176       variable access, and it's usually horrible.  It is, however, minusculy
177       faster.  The question remains whether the minute gain is actually worth
178       the eyestrain, or the loss of maintainability.
179
180   Search and replace or tr
181       If we have a string which needs to be modified, while a regex will
182       almost always be much more flexible, "tr", an oft underused tool, can
183       still be a useful.  One scenario might be replace all vowels with
184       another character.  The regex solution might look like this:
185
186        $str =~ s/[aeiou]/x/g
187
188       The "tr" alternative might look like this:
189
190        $str =~ tr/aeiou/xxxxx/
191
192       We can put that into a test file which we can run to check which
193       approach is the fastest, using a global $STR variable to assign to the
194       "my $str" variable so as to avoid perl trying to optimize any of the
195       work away by noticing it's assigned only the once.
196
197       # regex-transliterate
198
199        #!/usr/bin/perl
200
201        use v5.36;
202
203        use Benchmark;
204
205        my $STR = "$$-this and that";
206
207        timethese( 1000000, {
208        'sr'  => sub { my $str = $STR; $str =~ s/[aeiou]/x/g; return $str; },
209        'tr'  => sub { my $str = $STR; $str =~ tr/aeiou/xxxxx/; return $str; },
210        });
211
212       Running the code gives us our results:
213
214        $> perl regex-transliterate
215
216        Benchmark: timing 1000000 iterations of sr, tr...
217                sr:  2 wallclock secs ( 1.19 usr +  0.00 sys =  1.19 CPU) @ 840336.13/s (n=1000000)
218                tr:  0 wallclock secs ( 0.49 usr +  0.00 sys =  0.49 CPU) @ 2040816.33/s (n=1000000)
219
220       The "tr" version is a clear winner.  One solution is flexible, the
221       other is fast - and it's appropriately the programmer's choice which to
222       use.
223
224       Check the "Benchmark" docs for further useful techniques.
225

PROFILING TOOLS

227       A slightly larger piece of code will provide something on which a
228       profiler can produce more extensive reporting statistics.  This example
229       uses the simplistic "wordmatch" program which parses a given input file
230       and spews out a short report on the contents.
231
232       # wordmatch
233
234        #!/usr/bin/perl
235
236        use v5.36;
237
238        =head1 NAME
239
240        filewords - word analysis of input file
241
242        =head1 SYNOPSIS
243
244            filewords -f inputfilename [-d]
245
246        =head1 DESCRIPTION
247
248        This program parses the given filename, specified with C<-f>, and
249        displays a simple analysis of the words found therein.  Use the C<-d>
250        switch to enable debugging messages.
251
252        =cut
253
254        use FileHandle;
255        use Getopt::Long;
256
257        my $debug   =  0;
258        my $file    = '';
259
260        my $result = GetOptions (
261            'debug'         => \$debug,
262            'file=s'        => \$file,
263        );
264        die("invalid args") unless $result;
265
266        unless ( -f $file ) {
267            die("Usage: $0 -f filename [-d]");
268        }
269        my $FH = FileHandle->new("< $file")
270                                      or die("unable to open file($file): $!");
271
272        my $i_LINES = 0;
273        my $i_WORDS = 0;
274        my %count   = ();
275
276        my @lines = <$FH>;
277        foreach my $line ( @lines ) {
278            $i_LINES++;
279            $line =~ s/\n//;
280            my @words = split(/ +/, $line);
281            my $i_words = scalar(@words);
282            $i_WORDS = $i_WORDS + $i_words;
283            debug("line: $i_LINES supplying $i_words words: @words");
284            my $i_word = 0;
285            foreach my $word ( @words ) {
286                $i_word++;
287                $count{$i_LINES}{spec} += matches($i_word, $word,
288                                                  '[^a-zA-Z0-9]');
289                $count{$i_LINES}{only} += matches($i_word, $word,
290                                                  '^[^a-zA-Z0-9]+$');
291                $count{$i_LINES}{cons} += matches($i_word, $word,
292                                            '^[(?i:bcdfghjklmnpqrstvwxyz)]+$');
293                $count{$i_LINES}{vows} += matches($i_word, $word,
294                                                  '^[(?i:aeiou)]+$');
295                $count{$i_LINES}{caps} += matches($i_word, $word,
296                                                  '^[(A-Z)]+$');
297            }
298        }
299
300        print report( %count );
301
302        sub matches {
303            my $i_wd  = shift;
304            my $word  = shift;
305            my $regex = shift;
306            my $has = 0;
307
308            if ( $word =~ /($regex)/ ) {
309                $has++ if $1;
310            }
311
312            debug( "word: $i_wd "
313                  . ($has ? 'matches' : 'does not match')
314                  . " chars: /$regex/");
315
316            return $has;
317        }
318
319        sub report {
320            my %report = @_;
321            my %rep;
322
323            foreach my $line ( keys %report ) {
324                foreach my $key ( keys $report{$line}->%* ) {
325                    $rep{$key} += $report{$line}{$key};
326                }
327            }
328
329            my $report = qq|
330        $0 report for $file:
331        lines in file: $i_LINES
332        words in file: $i_WORDS
333        words with special (non-word) characters: $i_spec
334        words with only special (non-word) characters: $i_only
335        words with only consonants: $i_cons
336        words with only capital letters: $i_caps
337        words with only vowels: $i_vows
338        |;
339
340            return $report;
341        }
342
343        sub debug {
344            my $message = shift;
345
346            if ( $debug ) {
347                print STDERR "DBG: $message\n";
348            }
349        }
350
351        exit 0;
352
353   Devel::DProf
354       This venerable module has been the de-facto standard for Perl code
355       profiling for more than a decade, but has been replaced by a number of
356       other modules which have brought us back to the 21st century.  Although
357       you're recommended to evaluate your tool from the several mentioned
358       here and from the CPAN list at the base of this document, (and
359       currently Devel::NYTProf seems to be the weapon of choice - see below),
360       we'll take a quick look at the output from Devel::DProf first, to set a
361       baseline for Perl profiling tools.  Run the above program under the
362       control of "Devel::DProf" by using the "-d" switch on the command-line.
363
364        $> perl -d:DProf wordmatch -f perl5db.pl
365
366        <...multiple lines snipped...>
367
368        wordmatch report for perl5db.pl:
369        lines in file: 9428
370        words in file: 50243
371        words with special (non-word) characters: 20480
372        words with only special (non-word) characters: 7790
373        words with only consonants: 4801
374        words with only capital letters: 1316
375        words with only vowels: 1701
376
377       "Devel::DProf" produces a special file, called tmon.out by default, and
378       this file is read by the "dprofpp" program, which is already installed
379       as part of the "Devel::DProf" distribution.  If you call "dprofpp" with
380       no options, it will read the tmon.out file in the current directory and
381       produce a human readable statistics report of the run of your program.
382       Note that this may take a little time.
383
384        $> dprofpp
385
386        Total Elapsed Time = 2.951677 Seconds
387          User+System Time = 2.871677 Seconds
388        Exclusive Times
389        %Time ExclSec CumulS #Calls sec/call Csec/c  Name
390         102.   2.945  3.003 251215   0.0000 0.0000  main::matches
391         2.40   0.069  0.069 260643   0.0000 0.0000  main::debug
392         1.74   0.050  0.050      1   0.0500 0.0500  main::report
393         1.04   0.030  0.049      4   0.0075 0.0123  main::BEGIN
394         0.35   0.010  0.010      3   0.0033 0.0033  Exporter::as_heavy
395         0.35   0.010  0.010      7   0.0014 0.0014  IO::File::BEGIN
396         0.00       - -0.000      1        -      -  Getopt::Long::FindOption
397         0.00       - -0.000      1        -      -  Symbol::BEGIN
398         0.00       - -0.000      1        -      -  Fcntl::BEGIN
399         0.00       - -0.000      1        -      -  Fcntl::bootstrap
400         0.00       - -0.000      1        -      -  warnings::BEGIN
401         0.00       - -0.000      1        -      -  IO::bootstrap
402         0.00       - -0.000      1        -      -  Getopt::Long::ConfigDefaults
403         0.00       - -0.000      1        -      -  Getopt::Long::Configure
404         0.00       - -0.000      1        -      -  Symbol::gensym
405
406       "dprofpp" will produce some quite detailed reporting on the activity of
407       the "wordmatch" program.  The wallclock, user and system, times are at
408       the top of the analysis, and after this are the main columns defining
409       which define the report.  Check the "dprofpp" docs for details of the
410       many options it supports.
411
412       See also "Apache::DProf" which hooks "Devel::DProf" into "mod_perl".
413
414   Devel::Profiler
415       Let's take a look at the same program using a different profiler:
416       "Devel::Profiler", a drop-in Perl-only replacement for "Devel::DProf".
417       The usage is very slightly different in that instead of using the
418       special "-d:" flag, you pull "Devel::Profiler" in directly as a module
419       using "-M".
420
421        $> perl -MDevel::Profiler wordmatch -f perl5db.pl
422
423        <...multiple lines snipped...>
424
425        wordmatch report for perl5db.pl:
426        lines in file: 9428
427        words in file: 50243
428        words with special (non-word) characters: 20480
429        words with only special (non-word) characters: 7790
430        words with only consonants: 4801
431        words with only capital letters: 1316
432        words with only vowels: 1701
433
434       "Devel::Profiler" generates a tmon.out file which is compatible with
435       the "dprofpp" program, thus saving the construction of a dedicated
436       statistics reader program.  "dprofpp" usage is therefore identical to
437       the above example.
438
439        $> dprofpp
440
441        Total Elapsed Time =   20.984 Seconds
442          User+System Time =   19.981 Seconds
443        Exclusive Times
444        %Time ExclSec CumulS #Calls sec/call Csec/c  Name
445         49.0   9.792 14.509 251215   0.0000 0.0001  main::matches
446         24.4   4.887  4.887 260643   0.0000 0.0000  main::debug
447         0.25   0.049  0.049      1   0.0490 0.0490  main::report
448         0.00   0.000  0.000      1   0.0000 0.0000  Getopt::Long::GetOptions
449         0.00   0.000  0.000      2   0.0000 0.0000  Getopt::Long::ParseOptionSpec
450         0.00   0.000  0.000      1   0.0000 0.0000  Getopt::Long::FindOption
451         0.00   0.000  0.000      1   0.0000 0.0000  IO::File::new
452         0.00   0.000  0.000      1   0.0000 0.0000  IO::Handle::new
453         0.00   0.000  0.000      1   0.0000 0.0000  Symbol::gensym
454         0.00   0.000  0.000      1   0.0000 0.0000  IO::File::open
455
456       Interestingly we get slightly different results, which is mostly
457       because the algorithm which generates the report is different, even
458       though the output file format was allegedly identical.  The elapsed,
459       user and system times are clearly showing the time it took for
460       "Devel::Profiler" to execute its own run, but the column listings feel
461       more accurate somehow than the ones we had earlier from "Devel::DProf".
462       The 102% figure has disappeared, for example.  This is where we have to
463       use the tools at our disposal, and recognise their pros and cons,
464       before using them.  Interestingly, the numbers of calls for each
465       subroutine are identical in the two reports, it's the percentages which
466       differ.  As the author of "Devel::Proviler" writes:
467
468        ...running HTML::Template's test suite under Devel::DProf shows
469        output() taking NO time but Devel::Profiler shows around 10% of the
470        time is in output().  I don't know which to trust but my gut tells me
471        something is wrong with Devel::DProf.  HTML::Template::output() is a
472        big routine that's called for every test. Either way, something needs
473        fixing.
474
475       YMMV.
476
477       See also "Devel::Apache::Profiler" which hooks "Devel::Profiler" into
478       "mod_perl".
479
480   Devel::SmallProf
481       The "Devel::SmallProf" profiler examines the runtime of your Perl
482       program and produces a line-by-line listing to show how many times each
483       line was called, and how long each line took to execute.  It is called
484       by supplying the familiar "-d" flag to Perl at runtime.
485
486        $> perl -d:SmallProf wordmatch -f perl5db.pl
487
488        <...multiple lines snipped...>
489
490        wordmatch report for perl5db.pl:
491        lines in file: 9428
492        words in file: 50243
493        words with special (non-word) characters: 20480
494        words with only special (non-word) characters: 7790
495        words with only consonants: 4801
496        words with only capital letters: 1316
497        words with only vowels: 1701
498
499       "Devel::SmallProf" writes it's output into a file called smallprof.out,
500       by default.  The format of the file looks like this:
501
502        <num> <time> <ctime> <line>:<text>
503
504       When the program has terminated, the output may be examined and sorted
505       using any standard text filtering utilities.  Something like the
506       following may be sufficient:
507
508        $> cat smallprof.out | grep \d*: | sort -k3 | tac | head -n20
509
510        251215   1.65674   7.68000    75: if ( $word =~ /($regex)/ ) {
511        251215   0.03264   4.40000    79: debug("word: $i_wd ".($has ? 'matches' :
512        251215   0.02693   4.10000    81: return $has;
513        260643   0.02841   4.07000   128: if ( $debug ) {
514        260643   0.02601   4.04000   126: my $message = shift;
515        251215   0.02641   3.91000    73: my $has = 0;
516        251215   0.03311   3.71000    70: my $i_wd  = shift;
517        251215   0.02699   3.69000    72: my $regex = shift;
518        251215   0.02766   3.68000    71: my $word  = shift;
519         50243   0.59726   1.00000    59:  $count{$i_LINES}{cons} =
520         50243   0.48175   0.92000    61:  $count{$i_LINES}{spec} =
521         50243   0.00644   0.89000    56:  my $i_cons = matches($i_word, $word,
522         50243   0.48837   0.88000    63:  $count{$i_LINES}{caps} =
523         50243   0.00516   0.88000    58:  my $i_caps = matches($i_word, $word, '^[(A-
524         50243   0.00631   0.81000    54:  my $i_spec = matches($i_word, $word, '[^a-
525         50243   0.00496   0.80000    57:  my $i_vows = matches($i_word, $word,
526         50243   0.00688   0.80000    53:  $i_word++;
527         50243   0.48469   0.79000    62:  $count{$i_LINES}{only} =
528         50243   0.48928   0.77000    60:  $count{$i_LINES}{vows} =
529         50243   0.00683   0.75000    55:  my $i_only = matches($i_word, $word, '^[^a-
530
531       You can immediately see a slightly different focus to the subroutine
532       profiling modules, and we start to see exactly which line of code is
533       taking the most time.  That regex line is looking a bit suspicious, for
534       example.  Remember that these tools are supposed to be used together,
535       there is no single best way to profile your code, you need to use the
536       best tools for the job.
537
538       See also "Apache::SmallProf" which hooks "Devel::SmallProf" into
539       "mod_perl".
540
541   Devel::FastProf
542       "Devel::FastProf" is another Perl line profiler.  This was written with
543       a view to getting a faster line profiler, than is possible with for
544       example "Devel::SmallProf", because it's written in "C".  To use
545       "Devel::FastProf", supply the "-d" argument to Perl:
546
547        $> perl -d:FastProf wordmatch -f perl5db.pl
548
549        <...multiple lines snipped...>
550
551        wordmatch report for perl5db.pl:
552        lines in file: 9428
553        words in file: 50243
554        words with special (non-word) characters: 20480
555        words with only special (non-word) characters: 7790
556        words with only consonants: 4801
557        words with only capital letters: 1316
558        words with only vowels: 1701
559
560       "Devel::FastProf" writes statistics to the file fastprof.out in the
561       current directory.  The output file, which can be specified, can be
562       interpreted by using the "fprofpp" command-line program.
563
564        $> fprofpp | head -n20
565
566        # fprofpp output format is:
567        # filename:line time count: source
568        wordmatch:75 3.93338 251215: if ( $word =~ /($regex)/ ) {
569        wordmatch:79 1.77774 251215: debug("word: $i_wd ".($has ? 'matches' : 'does not match')." chars: /$regex/");
570        wordmatch:81 1.47604 251215: return $has;
571        wordmatch:126 1.43441 260643: my $message = shift;
572        wordmatch:128 1.42156 260643: if ( $debug ) {
573        wordmatch:70 1.36824 251215: my $i_wd  = shift;
574        wordmatch:71 1.36739 251215: my $word  = shift;
575        wordmatch:72 1.35939 251215: my $regex = shift;
576
577       Straightaway we can see that the number of times each line has been
578       called is identical to the "Devel::SmallProf" output, and the sequence
579       is only very slightly different based on the ordering of the amount of
580       time each line took to execute, "if ( $debug ) { " and "my $message =
581       shift;", for example.  The differences in the actual times recorded
582       might be in the algorithm used internally, or it could be due to system
583       resource limitations or contention.
584
585       See also the DBIx::Profile which will profile database queries running
586       under the "DBIx::*" namespace.
587
588   Devel::NYTProf
589       "Devel::NYTProf" is the next generation of Perl code profiler, fixing
590       many shortcomings in other tools and implementing many cool features.
591       First of all it can be used as either a line profiler, a block or a
592       subroutine profiler, all at once.  It can also use sub-microsecond
593       (100ns) resolution on systems which provide "clock_gettime()".  It can
594       be started and stopped even by the program being profiled.  It's a one-
595       line entry to profile "mod_perl" applications.  It's written in "c" and
596       is probably the fastest profiler available for Perl.  The list of
597       coolness just goes on.  Enough of that, let's see how to it works -
598       just use the familiar "-d" switch to plug it in and run the code.
599
600        $> perl -d:NYTProf wordmatch -f perl5db.pl
601
602        wordmatch report for perl5db.pl:
603        lines in file: 9427
604        words in file: 50243
605        words with special (non-word) characters: 20480
606        words with only special (non-word) characters: 7790
607        words with only consonants: 4801
608        words with only capital letters: 1316
609        words with only vowels: 1701
610
611       "NYTProf" will generate a report database into the file nytprof.out by
612       default.  Human readable reports can be generated from here by using
613       the supplied "nytprofhtml" (HTML output) and "nytprofcsv" (CSV output)
614       programs.  We've used the Unix system "html2text" utility to convert
615       the nytprof/index.html file for convenience here.
616
617        $> html2text nytprof/index.html
618
619        Performance Profile Index
620        For wordmatch
621          Run on Fri Sep 26 13:46:39 2008
622        Reported on Fri Sep 26 13:47:23 2008
623
624                 Top 15 Subroutines -- ordered by exclusive time
625        |Calls |P |F |Inclusive|Exclusive|Subroutine                          |
626        |      |  |  |Time     |Time     |                                    |
627        |251215|5 |1 |13.09263 |10.47692 |main::              |matches        |
628        |260642|2 |1 |2.71199  |2.71199  |main::              |debug          |
629        |1     |1 |1 |0.21404  |0.21404  |main::              |report         |
630        |2     |2 |2 |0.00511  |0.00511  |XSLoader::          |load (xsub)    |
631        |14    |14|7 |0.00304  |0.00298  |Exporter::          |import         |
632        |3     |1 |1 |0.00265  |0.00254  |Exporter::          |as_heavy       |
633        |10    |10|4 |0.00140  |0.00140  |vars::              |import         |
634        |13    |13|1 |0.00129  |0.00109  |constant::          |import         |
635        |1     |1 |1 |0.00360  |0.00096  |FileHandle::        |import         |
636        |3     |3 |3 |0.00086  |0.00074  |warnings::register::|import         |
637        |9     |3 |1 |0.00036  |0.00036  |strict::            |bits           |
638        |13    |13|13|0.00032  |0.00029  |strict::            |import         |
639        |2     |2 |2 |0.00020  |0.00020  |warnings::          |import         |
640        |2     |1 |1 |0.00020  |0.00020  |Getopt::Long::      |ParseOptionSpec|
641        |7     |7 |6 |0.00043  |0.00020  |strict::            |unimport       |
642
643        For more information see the full list of 189 subroutines.
644
645       The first part of the report already shows the critical information
646       regarding which subroutines are using the most time.  The next gives
647       some statistics about the source files profiled.
648
649                Source Code Files -- ordered by exclusive time then name
650        |Stmts  |Exclusive|Avg.   |Reports                     |Source File         |
651        |       |Time     |       |                            |                    |
652        |2699761|15.66654 |6e-06  |line   .    block   .    sub|wordmatch           |
653        |35     |0.02187  |0.00062|line   .    block   .    sub|IO/Handle.pm        |
654        |274    |0.01525  |0.00006|line   .    block   .    sub|Getopt/Long.pm      |
655        |20     |0.00585  |0.00029|line   .    block   .    sub|Fcntl.pm            |
656        |128    |0.00340  |0.00003|line   .    block   .    sub|Exporter/Heavy.pm   |
657        |42     |0.00332  |0.00008|line   .    block   .    sub|IO/File.pm          |
658        |261    |0.00308  |0.00001|line   .    block   .    sub|Exporter.pm         |
659        |323    |0.00248  |8e-06  |line   .    block   .    sub|constant.pm         |
660        |12     |0.00246  |0.00021|line   .    block   .    sub|File/Spec/Unix.pm   |
661        |191    |0.00240  |0.00001|line   .    block   .    sub|vars.pm             |
662        |77     |0.00201  |0.00003|line   .    block   .    sub|FileHandle.pm       |
663        |12     |0.00198  |0.00016|line   .    block   .    sub|Carp.pm             |
664        |14     |0.00175  |0.00013|line   .    block   .    sub|Symbol.pm           |
665        |15     |0.00130  |0.00009|line   .    block   .    sub|IO.pm               |
666        |22     |0.00120  |0.00005|line   .    block   .    sub|IO/Seekable.pm      |
667        |198    |0.00085  |4e-06  |line   .    block   .    sub|warnings/register.pm|
668        |114    |0.00080  |7e-06  |line   .    block   .    sub|strict.pm           |
669        |47     |0.00068  |0.00001|line   .    block   .    sub|warnings.pm         |
670        |27     |0.00054  |0.00002|line   .    block   .    sub|overload.pm         |
671        |9      |0.00047  |0.00005|line   .    block   .    sub|SelectSaver.pm      |
672        |13     |0.00045  |0.00003|line   .    block   .    sub|File/Spec.pm        |
673        |2701595|15.73869 |       |Total                       |
674        |128647 |0.74946  |       |Average                     |
675        |       |0.00201  |0.00003|Median                      |
676        |       |0.00121  |0.00003|Deviation                   |
677
678        Report produced by the NYTProf 2.03 Perl profiler, developed by Tim Bunce and
679        Adam Kaplan.
680
681       At this point, if you're using the html report, you can click through
682       the various links to bore down into each subroutine and each line of
683       code.  Because we're using the text reporting here, and there's a whole
684       directory full of reports built for each source file, we'll just
685       display a part of the corresponding wordmatch-line.html file,
686       sufficient to give an idea of the sort of output you can expect from
687       this cool tool.
688
689        $> html2text nytprof/wordmatch-line.html
690
691        Performance Profile -- -block view-.-line view-.-sub view-
692        For wordmatch
693        Run on Fri Sep 26 13:46:39 2008
694        Reported on Fri Sep 26 13:47:22 2008
695
696        File wordmatch
697
698         Subroutines -- ordered by exclusive time
699        |Calls |P|F|Inclusive|Exclusive|Subroutine    |
700        |      | | |Time     |Time     |              |
701        |251215|5|1|13.09263 |10.47692 |main::|matches|
702        |260642|2|1|2.71199  |2.71199  |main::|debug  |
703        |1     |1|1|0.21404  |0.21404  |main::|report |
704        |0     |0|0|0        |0        |main::|BEGIN  |
705
706
707        |Line|Stmts.|Exclusive|Avg.   |Code                                           |
708        |    |      |Time     |       |                                               |
709        |1   |      |         |       |#!/usr/bin/perl                                |
710        |2   |      |         |       |                                               |
711        |    |      |         |       |use strict;                                    |
712        |3   |3     |0.00086  |0.00029|# spent 0.00003s making 1 calls to strict::    |
713        |    |      |         |       |import                                         |
714        |    |      |         |       |use warnings;                                  |
715        |4   |3     |0.01563  |0.00521|# spent 0.00012s making 1 calls to warnings::  |
716        |    |      |         |       |import                                         |
717        |5   |      |         |       |                                               |
718        |6   |      |         |       |=head1 NAME                                    |
719        |7   |      |         |       |                                               |
720        |8   |      |         |       |filewords - word analysis of input file        |
721        <...snip...>
722        |62  |1     |0.00445  |0.00445|print report( %count );                        |
723        |    |      |         |       |# spent 0.21404s making 1 calls to main::report|
724        |63  |      |         |       |                                               |
725        |    |      |         |       |# spent 23.56955s (10.47692+2.61571) within    |
726        |    |      |         |       |main::matches which was called 251215 times,   |
727        |    |      |         |       |avg 0.00005s/call: # 50243 times               |
728        |    |      |         |       |(2.12134+0.51939s) at line 57 of wordmatch, avg|
729        |    |      |         |       |0.00005s/call # 50243 times (2.17735+0.54550s) |
730        |64  |      |         |       |at line 56 of wordmatch, avg 0.00005s/call #   |
731        |    |      |         |       |50243 times (2.10992+0.51797s) at line 58 of   |
732        |    |      |         |       |wordmatch, avg 0.00005s/call # 50243 times     |
733        |    |      |         |       |(2.12696+0.51598s) at line 55 of wordmatch, avg|
734        |    |      |         |       |0.00005s/call # 50243 times (1.94134+0.51687s) |
735        |    |      |         |       |at line 54 of wordmatch, avg 0.00005s/call     |
736        |    |      |         |       |sub matches {                                  |
737        <...snip...>
738        |102 |      |         |       |                                               |
739        |    |      |         |       |# spent 2.71199s within main::debug which was  |
740        |    |      |         |       |called 260642 times, avg 0.00001s/call: #      |
741        |    |      |         |       |251215 times (2.61571+0s) by main::matches at  |
742        |103 |      |         |       |line 74 of wordmatch, avg 0.00001s/call # 9427 |
743        |    |      |         |       |times (0.09628+0s) at line 50 of wordmatch, avg|
744        |    |      |         |       |0.00001s/call                                  |
745        |    |      |         |       |sub debug {                                    |
746        |104 |260642|0.58496  |2e-06  |my $message = shift;                           |
747        |105 |      |         |       |                                               |
748        |106 |260642|1.09917  |4e-06  |if ( $debug ) {                                |
749        |107 |      |         |       |print STDERR "DBG: $message\n";                |
750        |108 |      |         |       |}                                              |
751        |109 |      |         |       |}                                              |
752        |110 |      |         |       |                                               |
753        |111 |1     |0.01501  |0.01501|exit 0;                                        |
754        |112 |      |         |       |                                               |
755
756       Oodles of very useful information in there - this seems to be the way
757       forward.
758
759       See also "Devel::NYTProf::Apache" which hooks "Devel::NYTProf" into
760       "mod_perl".
761

SORTING

763       Perl modules are not the only tools a performance analyst has at their
764       disposal, system tools like "time" should not be overlooked as the next
765       example shows, where we take a quick look at sorting.  Many books,
766       theses and articles, have been written about efficient sorting
767       algorithms, and this is not the place to repeat such work, there's
768       several good sorting modules which deserve taking a look at too:
769       "Sort::Maker", "Sort::Key" spring to mind.  However, it's still
770       possible to make some observations on certain Perl specific
771       interpretations on issues relating to sorting data sets and give an
772       example or two with regard to how sorting large data volumes can effect
773       performance.  Firstly, an often overlooked point when sorting large
774       amounts of data, one can attempt to reduce the data set to be dealt
775       with and in many cases "grep()" can be quite useful as a simple filter:
776
777        @data = sort grep { /$filter/ } @incoming
778
779       A command such as this can vastly reduce the volume of material to
780       actually sort through in the first place, and should not be too lightly
781       disregarded purely on the basis of its simplicity.  The "KISS"
782       principle is too often overlooked - the next example uses the simple
783       system "time" utility to demonstrate.  Let's take a look at an actual
784       example of sorting the contents of a large file, an apache logfile
785       would do.  This one has over a quarter of a million lines, is 50M in
786       size, and a snippet of it looks like this:
787
788       # logfile
789
790        188.209-65-87.adsl-dyn.isp.belgacom.be - - [08/Feb/2007:12:57:16 +0000] "GET /favicon.ico HTTP/1.1" 404 209 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)"
791        188.209-65-87.adsl-dyn.isp.belgacom.be - - [08/Feb/2007:12:57:16 +0000] "GET /favicon.ico HTTP/1.1" 404 209 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)"
792        151.56.71.198 - - [08/Feb/2007:12:57:41 +0000] "GET /suse-on-vaio.html HTTP/1.1" 200 2858 "http://www.linux-on-laptops.com/sony.html" "Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US; rv:1.8.1.1) Gecko/20061204 Firefox/2.0.0.1"
793        151.56.71.198 - - [08/Feb/2007:12:57:42 +0000] "GET /data/css HTTP/1.1" 404 206 "http://www.rfi.net/suse-on-vaio.html" "Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US; rv:1.8.1.1) Gecko/20061204 Firefox/2.0.0.1"
794        151.56.71.198 - - [08/Feb/2007:12:57:43 +0000] "GET /favicon.ico HTTP/1.1" 404 209 "-" "Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US; rv:1.8.1.1) Gecko/20061204 Firefox/2.0.0.1"
795        217.113.68.60 - - [08/Feb/2007:13:02:15 +0000] "GET / HTTP/1.1" 304 - "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)"
796        217.113.68.60 - - [08/Feb/2007:13:02:16 +0000] "GET /data/css HTTP/1.1" 404 206 "http://www.rfi.net/" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)"
797        debora.to.isac.cnr.it - - [08/Feb/2007:13:03:58 +0000] "GET /suse-on-vaio.html HTTP/1.1" 200 2858 "http://www.linux-on-laptops.com/sony.html" "Mozilla/5.0 (compatible; Konqueror/3.4; Linux) KHTML/3.4.0 (like Gecko)"
798        debora.to.isac.cnr.it - - [08/Feb/2007:13:03:58 +0000] "GET /data/css HTTP/1.1" 404 206 "http://www.rfi.net/suse-on-vaio.html" "Mozilla/5.0 (compatible; Konqueror/3.4; Linux) KHTML/3.4.0 (like Gecko)"
799        debora.to.isac.cnr.it - - [08/Feb/2007:13:03:58 +0000] "GET /favicon.ico HTTP/1.1" 404 209 "-" "Mozilla/5.0 (compatible; Konqueror/3.4; Linux) KHTML/3.4.0 (like Gecko)"
800        195.24.196.99 - - [08/Feb/2007:13:26:48 +0000] "GET / HTTP/1.0" 200 3309 "-" "Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.8.0.9) Gecko/20061206 Firefox/1.5.0.9"
801        195.24.196.99 - - [08/Feb/2007:13:26:58 +0000] "GET /data/css HTTP/1.0" 404 206 "http://www.rfi.net/" "Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.8.0.9) Gecko/20061206 Firefox/1.5.0.9"
802        195.24.196.99 - - [08/Feb/2007:13:26:59 +0000] "GET /favicon.ico HTTP/1.0" 404 209 "-" "Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.8.0.9) Gecko/20061206 Firefox/1.5.0.9"
803        crawl1.cosmixcorp.com - - [08/Feb/2007:13:27:57 +0000] "GET /robots.txt HTTP/1.0" 200 179 "-" "voyager/1.0"
804        crawl1.cosmixcorp.com - - [08/Feb/2007:13:28:25 +0000] "GET /links.html HTTP/1.0" 200 3413 "-" "voyager/1.0"
805        fhm226.internetdsl.tpnet.pl - - [08/Feb/2007:13:37:32 +0000] "GET /suse-on-vaio.html HTTP/1.1" 200 2858 "http://www.linux-on-laptops.com/sony.html" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)"
806        fhm226.internetdsl.tpnet.pl - - [08/Feb/2007:13:37:34 +0000] "GET /data/css HTTP/1.1" 404 206 "http://www.rfi.net/suse-on-vaio.html" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)"
807        80.247.140.134 - - [08/Feb/2007:13:57:35 +0000] "GET / HTTP/1.1" 200 3309 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; .NET CLR 1.1.4322)"
808        80.247.140.134 - - [08/Feb/2007:13:57:37 +0000] "GET /data/css HTTP/1.1" 404 206 "http://www.rfi.net" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; .NET CLR 1.1.4322)"
809        pop.compuscan.co.za - - [08/Feb/2007:14:10:43 +0000] "GET / HTTP/1.1" 200 3309 "-" "www.clamav.net"
810        livebot-207-46-98-57.search.live.com - - [08/Feb/2007:14:12:04 +0000] "GET /robots.txt HTTP/1.0" 200 179 "-" "msnbot/1.0 (+http://search.msn.com/msnbot.htm)"
811        livebot-207-46-98-57.search.live.com - - [08/Feb/2007:14:12:04 +0000] "GET /html/oracle.html HTTP/1.0" 404 214 "-" "msnbot/1.0 (+http://search.msn.com/msnbot.htm)"
812        dslb-088-064-005-154.pools.arcor-ip.net - - [08/Feb/2007:14:12:15 +0000] "GET / HTTP/1.1" 200 3309 "-" "www.clamav.net"
813        196.201.92.41 - - [08/Feb/2007:14:15:01 +0000] "GET / HTTP/1.1" 200 3309 "-" "MOT-L7/08.B7.DCR MIB/2.2.1 Profile/MIDP-2.0 Configuration/CLDC-1.1"
814
815       The specific task here is to sort the 286,525 lines of this file by
816       Response Code, Query, Browser, Referring Url, and lastly Date.  One
817       solution might be to use the following code, which iterates over the
818       files given on the command-line.
819
820       # sort-apache-log
821
822        #!/usr/bin/perl -n
823
824        use v5.36;
825
826        my @data;
827
828        LINE:
829        while ( <> ) {
830            my $line = $_;
831            if (
832                $line =~ m/^(
833                    ([\w\.\-]+)             # client
834                    \s*-\s*-\s*\[
835                    ([^]]+)                 # date
836                    \]\s*"\w+\s*
837                    (\S+)                   # query
838                    [^"]+"\s*
839                    (\d+)                   # status
840                    \s+\S+\s+"[^"]*"\s+"
841                    ([^"]*)                 # browser
842                    "
843                    .*
844                )$/x
845            ) {
846                my @chunks = split(/ +/, $line);
847                my $ip      = $1;
848                my $date    = $2;
849                my $query   = $3;
850                my $status  = $4;
851                my $browser = $5;
852
853                push(@data, [$ip, $date, $query, $status, $browser, $line]);
854            }
855        }
856
857        my @sorted = sort {
858            $a->[3] cmp $b->[3]
859                    ||
860            $a->[2] cmp $b->[2]
861                    ||
862            $a->[0] cmp $b->[0]
863                    ||
864            $a->[1] cmp $b->[1]
865                    ||
866            $a->[4] cmp $b->[4]
867        } @data;
868
869        foreach my $data ( @sorted ) {
870            print $data->[5];
871        }
872
873        exit 0;
874
875       When running this program, redirect "STDOUT" so it is possible to check
876       the output is correct from following test runs and use the system
877       "time" utility to check the overall runtime.
878
879        $> time ./sort-apache-log logfile > out-sort
880
881        real    0m17.371s
882        user    0m15.757s
883        sys     0m0.592s
884
885       The program took just over 17 wallclock seconds to run.  Note the
886       different values "time" outputs, it's important to always use the same
887       one, and to not confuse what each one means.
888
889       Elapsed Real Time
890           The overall, or wallclock, time between when "time" was called, and
891           when it terminates.  The elapsed time includes both user and system
892           times, and time spent waiting for other users and processes on the
893           system.  Inevitably, this is the most approximate of the
894           measurements given.
895
896       User CPU Time
897           The user time is the amount of time the entire process spent on
898           behalf of the user on this system executing this program.
899
900       System CPU Time
901           The system time is the amount of time the kernel itself spent
902           executing routines, or system calls, on behalf of this process
903           user.
904
905       Running this same process as a "Schwarzian Transform" it is possible to
906       eliminate the input and output arrays for storing all the data, and
907       work on the input directly as it arrives too.  Otherwise, the code
908       looks fairly similar:
909
910       # sort-apache-log-schwarzian
911
912        #!/usr/bin/perl -n
913
914        use v5.36;
915
916        print
917
918            map $_->[0] =>
919
920            sort {
921                $a->[4] cmp $b->[4]
922                        ||
923                $a->[3] cmp $b->[3]
924                        ||
925                $a->[1] cmp $b->[1]
926                        ||
927                $a->[2] cmp $b->[2]
928                        ||
929                $a->[5] cmp $b->[5]
930            }
931            map  [ $_, m/^(
932                ([\w\.\-]+)             # client
933                \s*-\s*-\s*\[
934                ([^]]+)                 # date
935                \]\s*"\w+\s*
936                (\S+)                   # query
937                [^"]+"\s*
938                (\d+)                   # status
939                \s+\S+\s+"[^"]*"\s+"
940                ([^"]*)                 # browser
941                "
942                .*
943            )$/xo ]
944
945            => <>;
946
947        exit 0;
948
949       Run the new code against the same logfile, as above, to check the new
950       time.
951
952        $> time ./sort-apache-log-schwarzian logfile > out-schwarz
953
954        real    0m9.664s
955        user    0m8.873s
956        sys     0m0.704s
957
958       The time has been cut in half, which is a respectable speed improvement
959       by any standard.  Naturally, it is important to check the output is
960       consistent with the first program run, this is where the Unix system
961       "cksum" utility comes in.
962
963        $> cksum out-sort out-schwarz
964        3044173777 52029194 out-sort
965        3044173777 52029194 out-schwarz
966
967       BTW. Beware too of pressure from managers who see you speed a program
968       up by 50% of the runtime once, only to get a request one month later to
969       do the same again (true story) - you'll just have to point out you're
970       only human, even if you are a Perl programmer, and you'll see what you
971       can do...
972

LOGGING

974       An essential part of any good development process is appropriate error
975       handling with appropriately informative messages, however there exists
976       a school of thought which suggests that log files should be chatty, as
977       if the chain of unbroken output somehow ensures the survival of the
978       program.  If speed is in any way an issue, this approach is wrong.
979
980       A common sight is code which looks something like this:
981
982        logger->debug( "A logging message via process-id: $$ INC: "
983                                                              . Dumper(\%INC) )
984
985       The problem is that this code will always be parsed and executed, even
986       when the debug level set in the logging configuration file is zero.
987       Once the debug() subroutine has been entered, and the internal $debug
988       variable confirmed to be zero, for example, the message which has been
989       sent in will be discarded and the program will continue.  In the
990       example given though, the "\%INC" hash will already have been dumped,
991       and the message string constructed, all of which work could be bypassed
992       by a debug variable at the statement level, like this:
993
994        logger->debug( "A logging message via process-id: $$ INC: "
995                                                   . Dumper(\%INC) ) if $DEBUG;
996
997       This effect can be demonstrated by setting up a test script with both
998       forms, including a "debug()" subroutine to emulate typical "logger()"
999       functionality.
1000
1001       # ifdebug
1002
1003        #!/usr/bin/perl
1004
1005        use v5.36;
1006
1007        use Benchmark;
1008        use Data::Dumper;
1009        my $DEBUG = 0;
1010
1011        sub debug {
1012            my $msg = shift;
1013
1014            if ( $DEBUG ) {
1015                print "DEBUG: $msg\n";
1016            }
1017        };
1018
1019        timethese(100000, {
1020                'debug'       => sub {
1021                    debug( "A $0 logging message via process-id: $$" . Dumper(\%INC) )
1022                },
1023                'ifdebug'  => sub {
1024                    debug( "A $0 logging message via process-id: $$" . Dumper(\%INC) ) if $DEBUG
1025                },
1026        });
1027
1028       Let's see what "Benchmark" makes of this:
1029
1030        $> perl ifdebug
1031        Benchmark: timing 100000 iterations of constant, sub...
1032           ifdebug:  0 wallclock secs ( 0.01 usr +  0.00 sys =  0.01 CPU) @ 10000000.00/s (n=100000)
1033                    (warning: too few iterations for a reliable count)
1034             debug: 14 wallclock secs (13.18 usr +  0.04 sys = 13.22 CPU) @ 7564.30/s (n=100000)
1035
1036       In the one case the code, which does exactly the same thing as far as
1037       outputting any debugging information is concerned, in other words
1038       nothing, takes 14 seconds, and in the other case the code takes one
1039       hundredth of a second.  Looks fairly definitive.  Use a $DEBUG variable
1040       BEFORE you call the subroutine, rather than relying on the smart
1041       functionality inside it.
1042
1043   Logging if DEBUG (constant)
1044       It's possible to take the previous idea a little further, by using a
1045       compile time "DEBUG" constant.
1046
1047       # ifdebug-constant
1048
1049        #!/usr/bin/perl
1050
1051        use v5.36;
1052
1053        use Benchmark;
1054        use Data::Dumper;
1055        use constant
1056            DEBUG => 0
1057        ;
1058
1059        sub debug {
1060            if ( DEBUG ) {
1061                my $msg = shift;
1062                print "DEBUG: $msg\n";
1063            }
1064        };
1065
1066        timethese(100000, {
1067                'debug'       => sub {
1068                    debug( "A $0 logging message via process-id: $$" . Dumper(\%INC) )
1069                },
1070                'constant'  => sub {
1071                    debug( "A $0 logging message via process-id: $$" . Dumper(\%INC) ) if DEBUG
1072                },
1073        });
1074
1075       Running this program produces the following output:
1076
1077        $> perl ifdebug-constant
1078        Benchmark: timing 100000 iterations of constant, sub...
1079          constant:  0 wallclock secs (-0.00 usr +  0.00 sys = -0.00 CPU) @ -7205759403792793600000.00/s (n=100000)
1080                    (warning: too few iterations for a reliable count)
1081               sub: 14 wallclock secs (13.09 usr +  0.00 sys = 13.09 CPU) @ 7639.42/s (n=100000)
1082
1083       The "DEBUG" constant wipes the floor with even the $debug variable,
1084       clocking in at minus zero seconds, and generates a "warning: too few
1085       iterations for a reliable count" message into the bargain.  To see what
1086       is really going on, and why we had too few iterations when we thought
1087       we asked for 100000, we can use the very useful "B::Deparse" to inspect
1088       the new code:
1089
1090        $> perl -MO=Deparse ifdebug-constant
1091
1092        use Benchmark;
1093        use Data::Dumper;
1094        use constant ('DEBUG', 0);
1095        sub debug {
1096            use warnings;
1097            use strict 'refs';
1098            0;
1099        }
1100        use warnings;
1101        use strict 'refs';
1102        timethese(100000, {'sub', sub {
1103            debug "A $0 logging message via process-id: $$" . Dumper(\%INC);
1104        }
1105        , 'constant', sub {
1106            0;
1107        }
1108        });
1109        ifdebug-constant syntax OK
1110
1111       The output shows the constant() subroutine we're testing being replaced
1112       with the value of the "DEBUG" constant: zero.  The line to be tested
1113       has been completely optimized away, and you can't get much more
1114       efficient than that.
1115

POSTSCRIPT

1117       This document has provided several way to go about identifying hot-
1118       spots, and checking whether any modifications have improved the runtime
1119       of the code.
1120
1121       As a final thought, remember that it's not (at the time of writing)
1122       possible to produce a useful program which will run in zero or negative
1123       time and this basic principle can be written as: useful programs are
1124       slow by their very definition.  It is of course possible to write a
1125       nearly instantaneous program, but it's not going to do very much,
1126       here's a very efficient one:
1127
1128        $> perl -e 0
1129
1130       Optimizing that any further is a job for "p5p".
1131

SEE ALSO

1133       Further reading can be found using the modules and links below.
1134
1135   PERLDOCS
1136       For example: "perldoc -f sort".
1137
1138       perlfaq4.
1139
1140       perlfork, perlfunc, perlretut, perlthrtut.
1141
1142       threads.
1143
1144   MAN PAGES
1145       "time".
1146
1147   MODULES
1148       It's not possible to individually showcase all the performance related
1149       code for Perl here, naturally, but here's a short list of modules from
1150       the CPAN which deserve further attention.
1151
1152        Apache::DProf
1153        Apache::SmallProf
1154        Benchmark
1155        DBIx::Profile
1156        Devel::AutoProfiler
1157        Devel::DProf
1158        Devel::DProfLB
1159        Devel::FastProf
1160        Devel::GraphVizProf
1161        Devel::NYTProf
1162        Devel::NYTProf::Apache
1163        Devel::Profiler
1164        Devel::Profile
1165        Devel::Profit
1166        Devel::SmallProf
1167        Devel::WxProf
1168        POE::Devel::Profiler
1169        Sort::Key
1170        Sort::Maker
1171
1172   URLS
1173       Very useful online reference material:
1174
1175        https://web.archive.org/web/20120515021937/http://www.ccl4.org/~nick/P/Fast_Enough/
1176
1177        https://web.archive.org/web/20050706081718/http://www-106.ibm.com/developerworks/library/l-optperl.html
1178
1179        https://perlbuzz.com/2007/11/14/bind_output_variables_in_dbi_for_speed_and_safety/
1180
1181        http://en.wikipedia.org/wiki/Performance_analysis
1182
1183        http://apache.perl.org/docs/1.0/guide/performance.html
1184
1185        http://perlgolf.sourceforge.net/
1186
1187        http://www.sysarch.com/Perl/sort_paper.html
1188

AUTHOR

1190       Richard Foley <richard.foley@rfi.net> Copyright (c) 2008
1191
1192
1193
1194perl v5.36.0                      2022-08-30                       PERLPERF(1)
Impressum