1PERLHACKTIPS(1)        Perl Programmers Reference Guide        PERLHACKTIPS(1)
2
3
4

NAME

6       perlhacktips - Tips for Perl core C code hacking
7

DESCRIPTION

9       This document will help you learn the best way to go about hacking on
10       the Perl core C code.  It covers common problems, debugging, profiling,
11       and more.
12
13       If you haven't read perlhack and perlhacktut yet, you might want to do
14       that first.
15

COMMON PROBLEMS

17       Perl source now permits some specific C99 features which we know are
18       supported by all platforms, but mostly plays by ANSI C89 rules.  You
19       don't care about some particular platform having broken Perl? I hear
20       there is still a strong demand for J2EE programmers.
21
22   Perl environment problems
23       •   Not compiling with threading
24
25           Compiling with threading (-Duseithreads) completely rewrites the
26           function prototypes of Perl.  You better try your changes with
27           that.  Related to this is the difference between "Perl_-less" and
28           "Perl_-ly" APIs, for example:
29
30             Perl_sv_setiv(aTHX_ ...);
31             sv_setiv(...);
32
33           The first one explicitly passes in the context, which is needed for
34           e.g. threaded builds.  The second one does that implicitly; do not
35           get them mixed.  If you are not passing in a aTHX_, you will need
36           to do a dTHX as the first thing in the function.
37
38           See "How multiple interpreters and concurrency are supported" in
39           perlguts for further discussion about context.
40
41       •   Not compiling with -DDEBUGGING
42
43           The DEBUGGING define exposes more code to the compiler, therefore
44           more ways for things to go wrong.  You should try it.
45
46       •   Introducing (non-read-only) globals
47
48           Do not introduce any modifiable globals, truly global or file
49           static.  They are bad form and complicate multithreading and other
50           forms of concurrency.  The right way is to introduce them as new
51           interpreter variables, see intrpvar.h (at the very end for binary
52           compatibility).
53
54           Introducing read-only (const) globals is okay, as long as you
55           verify with e.g. "nm libperl.a|egrep -v ' [TURtr] '" (if your "nm"
56           has BSD-style output) that the data you added really is read-only.
57           (If it is, it shouldn't show up in the output of that command.)
58
59           If you want to have static strings, make them constant:
60
61             static const char etc[] = "...";
62
63           If you want to have arrays of constant strings, note carefully the
64           right combination of "const"s:
65
66               static const char * const yippee[] =
67                   {"hi", "ho", "silver"};
68
69       •   Not exporting your new function
70
71           Some platforms (Win32, AIX, VMS, OS/2, to name a few) require any
72           function that is part of the public API (the shared Perl library)
73           to be explicitly marked as exported.  See the discussion about
74           embed.pl in perlguts.
75
76       •   Exporting your new function
77
78           The new shiny result of either genuine new functionality or your
79           arduous refactoring is now ready and correctly exported.  So what
80           could possibly go wrong?
81
82           Maybe simply that your function did not need to be exported in the
83           first place.  Perl has a long and not so glorious history of
84           exporting functions that it should not have.
85
86           If the function is used only inside one source code file, make it
87           static.  See the discussion about embed.pl in perlguts.
88
89           If the function is used across several files, but intended only for
90           Perl's internal use (and this should be the common case), do not
91           export it to the public API.  See the discussion about embed.pl in
92           perlguts.
93
94   C99
95       Starting from 5.35.5 we now permit some C99 features in the core C
96       source.  However, code in dual life extensions still needs to be C89
97       only, because it needs to compile against earlier version of Perl
98       running on older platforms.  Also note that our headers need to also be
99       valid as C++, because XS extensions written in C++ need to include
100       them, hence member structure initialisers can't be used in headers.
101
102       C99 support is still far from complete on all platforms we currently
103       support.  As a baseline we can only assume C89 semantics with the
104       specific C99 features described below, which we've verified work
105       everywhere.  It's fine to probe for additional C99 features and use
106       them where available, providing there is also a fallback for compilers
107       that don't support the feature.  For example, we use C11 thread local
108       storage when available, but fall back to POSIX thread specific APIs
109       otherwise, and we use "char" for booleans if "<stdbool.h>" isn't
110       available.
111
112       Code can use (and rely on) the following C99 features being present
113
114       •   mixed declarations and code
115
116       •   64 bit integer types
117
118           For consistency with the existing source code, use the typedefs
119           "I64" and "U64", instead of using "long long" and "unsigned long
120           long" directly.
121
122       •   variadic macros
123
124               void greet(char *file, unsigned int line, char *format, ...);
125               #define logged_greet(...) greet(__FILE__, __LINE__, __VA_ARGS__);
126
127           Note that "__VA_OPT__" is a gcc extension not yet in any published
128           standard.
129
130       •   declarations in for loops
131
132               for (const char *p = message; *p; ++p) {
133                   putchar(*p);
134               }
135
136       •   member structure initialisers
137
138           But not in headers, as support was only added to C++ relatively
139           recently.
140
141           Hence this is fine in C and XS code, but not headers:
142
143               struct message {
144                   char *action;
145                   char *target;
146               };
147
148               struct message mcguffin = {
149                   .target = "member structure initialisers",
150                   .action = "Built"
151                };
152
153       •   flexible array members
154
155           This is standards conformant:
156
157               struct greeting {
158                   unsigned int len;
159                   char message[];
160               };
161
162           However, the source code already uses the "unwarranted chumminess
163           with the compiler" hack in many places:
164
165               struct greeting {
166                   unsigned int len;
167                   char message[1];
168               };
169
170           Strictly it is undefined behaviour accessing beyond "message[0]",
171           but this has been a commonly used hack since K&R times, and using
172           it hasn't been a practical issue anywhere (in the perl source or
173           any other common C code).  Hence it's unclear what we would gain
174           from actively changing to the C99 approach.
175
176       •   "//" comments
177
178           All compilers we tested support their use. Not all humans we tested
179           support their use.
180
181       Code explicitly should not use any other C99 features. For example
182
183       •   variable length arrays
184
185           Not supported by any MSVC, and this is not going to change.
186
187           Even "variable" length arrays where the variable is a constant
188           expression are syntax errors under MSVC.
189
190       •   C99 types in "<stdint.h>"
191
192           Use "PERL_INT_FAST8_T" etc as defined in handy.h
193
194       •   C99 format strings in "<inttypes.h>"
195
196           "snprintf" in the VMS libc only added support for "PRIdN" etc very
197           recently, meaning that there are live supported installations
198           without this, or formats such as %zu.
199
200           (perl's "sv_catpvf" etc use parser code code in "sv.c", which
201           supports the "z" modifier, along with perl-specific formats such as
202           "SVf".)
203
204       If you want to use a C99 feature not listed above then you need to do
205       one of
206
207       •   Probe for it in Configure, set a variable in config.sh, and add
208           fallback logic in the headers for platforms which don't have it.
209
210       •   Write test code and verify that it works on platforms we need to
211           support, before relying on it unconditionally.
212
213       Likely you want to repeat the same plan as we used to get the current
214       C99 feature set. See the message at
215       https://markmail.org/thread/odr4fjrn72u2fkpz for the C99 probes we used
216       before. Note that the two most "fussy" compilers appear to be MSVC and
217       the vendor compiler on VMS. To date all the *nix compilers have been
218       far more flexible in what they support.
219
220       On *nix platforms, Configure attempts to set compiler flags
221       appropriately.  All vendor compilers that we tested defaulted to C99
222       (or C11) support.  However, older versions of gcc default to C89, or
223       permit most C99 (with warnings), but forbid declarations in for loops
224       unless "-std=gnu99" is added. The alternative "-std=c99" might seem
225       better, but using it on some platforms can prevent "<unistd.h>"
226       declaring some prototypes being declared, which breaks the build. gcc's
227       "-ansi" flag implies "-std=c89" so we can no longer set that, hence the
228       Configure option "-gccansipedantic" now only adds "-pedantic".
229
230       The Perl core source code files (the ones at the top level of the
231       source code distribution) are automatically compiled with as many as
232       possible of the "-std=gnu99", "-pedantic", and a selection of "-W"
233       flags (see cflags.SH). Files in ext/ dist/ cpan/ etc are compiled with
234       the same flags as the installed perl would use to compile XS
235       extensions.
236
237       Basically, it's safe to assume that Configure and cflags.SH have picked
238       the best combination of flags for the version of gcc on the platform,
239       and attempting to add more flags related to enforcing a C dialect will
240       cause problems either locally, or on other systems that the code is
241       shipped to.
242
243       We believe that the C99 support in gcc 3.1 is good enough for us, but
244       we don't have a 19 year old gcc handy to check this :-) If you have
245       ancient vendor compilers that don't default to C99, the flags you might
246       want to try are
247
248       AIX "-qlanglvl=stdc99"
249
250       HP/UX
251           "-AC99"
252
253       Solaris
254           "-xc99"
255
256   Portability problems
257       The following are common causes of compilation and/or execution
258       failures, not common to Perl as such.  The C FAQ is good bedtime
259       reading.  Please test your changes with as many C compilers and
260       platforms as possible; we will, anyway, and it's nice to save oneself
261       from public embarrassment.
262
263       Also study perlport carefully to avoid any bad assumptions about the
264       operating system, filesystems, character set, and so forth.
265
266       Do not assume an operating system indicates a certain compiler.
267
268       •   Casting pointers to integers or casting integers to pointers
269
270               void castaway(U8* p)
271               {
272                 IV i = p;
273
274           or
275
276               void castaway(U8* p)
277               {
278                 IV i = (IV)p;
279
280           Both are bad, and broken, and unportable.  Use the PTR2IV() macro
281           that does it right.  (Likewise, there are PTR2UV(), PTR2NV(),
282           INT2PTR(), and NUM2PTR().)
283
284       •   Casting between function pointers and data pointers
285
286           Technically speaking casting between function pointers and data
287           pointers is unportable and undefined, but practically speaking it
288           seems to work, but you should use the FPTR2DPTR() and DPTR2FPTR()
289           macros.  Sometimes you can also play games with unions.
290
291       •   Assuming sizeof(int) == sizeof(long)
292
293           There are platforms where longs are 64 bits, and platforms where
294           ints are 64 bits, and while we are out to shock you, even platforms
295           where shorts are 64 bits.  This is all legal according to the C
296           standard.  (In other words, "long long" is not a portable way to
297           specify 64 bits, and "long long" is not even guaranteed to be any
298           wider than "long".)
299
300           Instead, use the definitions IV, UV, IVSIZE, I32SIZE, and so forth.
301           Avoid things like I32 because they are not guaranteed to be exactly
302           32 bits, they are at least 32 bits, nor are they guaranteed to be
303           int or long.  If you explicitly need 64-bit variables, use I64 and
304           U64.
305
306       •   Assuming one can dereference any type of pointer for any type of
307           data
308
309             char *p = ...;
310             long pony = *(long *)p;    /* BAD */
311
312           Many platforms, quite rightly so, will give you a core dump instead
313           of a pony if the p happens not to be correctly aligned.
314
315       •   Lvalue casts
316
317             (int)*p = ...;    /* BAD */
318
319           Simply not portable.  Get your lvalue to be of the right type, or
320           maybe use temporary variables, or dirty tricks with unions.
321
322       •   Assume anything about structs (especially the ones you don't
323           control, like the ones coming from the system headers)
324
325           •       That a certain field exists in a struct
326
327           •       That no other fields exist besides the ones you know of
328
329           •       That a field is of certain signedness, sizeof, or type
330
331           •       That the fields are in a certain order
332
333                   •       While C guarantees the ordering specified in the
334                           struct definition, between different platforms the
335                           definitions might differ
336
337           •       That the sizeof(struct) or the alignments are the same
338                   everywhere
339
340                   •       There might be padding bytes between the fields to
341                           align the fields - the bytes can be anything
342
343                   •       Structs are required to be aligned to the maximum
344                           alignment required by the fields - which for native
345                           types is for usually equivalent to sizeof() of the
346                           field
347
348       •   Assuming the character set is ASCIIish
349
350           Perl can compile and run under EBCDIC platforms.  See perlebcdic.
351           This is transparent for the most part, but because the character
352           sets differ, you shouldn't use numeric (decimal, octal, nor hex)
353           constants to refer to characters.  You can safely say 'A', but not
354           0x41.  You can safely say '\n', but not "\012".  However, you can
355           use macros defined in utf8.h to specify any code point portably.
356           "LATIN1_TO_NATIVE(0xDF)" is going to be the code point that means
357           LATIN SMALL LETTER SHARP S on whatever platform you are running on
358           (on ASCII platforms it compiles without adding any extra code, so
359           there is zero performance hit on those).  The acceptable inputs to
360           "LATIN1_TO_NATIVE" are from 0x00 through 0xFF.  If your input isn't
361           guaranteed to be in that range, use "UNICODE_TO_NATIVE" instead.
362           "NATIVE_TO_LATIN1" and "NATIVE_TO_UNICODE" translate the opposite
363           direction.
364
365           If you need the string representation of a character that doesn't
366           have a mnemonic name in C, you should add it to the list in
367           regen/unicode_constants.pl, and have Perl create "#define"'s for
368           you, based on the current platform.
369
370           Note that the "isFOO" and "toFOO" macros in handy.h work properly
371           on native code points and strings.
372
373           Also, the range 'A' - 'Z' in ASCII is an unbroken sequence of 26
374           upper case alphabetic characters.  That is not true in EBCDIC.  Nor
375           for 'a' to 'z'.  But '0' - '9' is an unbroken range in both
376           systems.  Don't assume anything about other ranges.  (Note that
377           special handling of ranges in regular expression patterns and
378           transliterations makes it appear to Perl code that the
379           aforementioned ranges are all unbroken.)
380
381           Many of the comments in the existing code ignore the possibility of
382           EBCDIC, and may be wrong therefore, even if the code works.  This
383           is actually a tribute to the successful transparent insertion of
384           being able to handle EBCDIC without having to change pre-existing
385           code.
386
387           UTF-8 and UTF-EBCDIC are two different encodings used to represent
388           Unicode code points as sequences of bytes.  Macros  with the same
389           names (but different definitions) in utf8.h and utfebcdic.h are
390           used to allow the calling code to think that there is only one such
391           encoding.  This is almost always referred to as "utf8", but it
392           means the EBCDIC version as well.  Again, comments in the code may
393           well be wrong even if the code itself is right.  For example, the
394           concept of UTF-8 "invariant characters" differs between ASCII and
395           EBCDIC.  On ASCII platforms, only characters that do not have the
396           high-order bit set (i.e.  whose ordinals are strict ASCII, 0 - 127)
397           are invariant, and the documentation and comments in the code may
398           assume that, often referring to something like, say, "hibit".  The
399           situation differs and is not so simple on EBCDIC machines, but as
400           long as the code itself uses the "NATIVE_IS_INVARIANT()" macro
401           appropriately, it works, even if the comments are wrong.
402
403           As noted in "TESTING" in perlhack, when writing test scripts, the
404           file t/charset_tools.pl contains some helpful functions for writing
405           tests valid on both ASCII and EBCDIC platforms.  Sometimes, though,
406           a test can't use a function and it's inconvenient to have different
407           test versions depending on the platform.  There are 20 code points
408           that are the same in all 4 character sets currently recognized by
409           Perl (the 3 EBCDIC code pages plus ISO 8859-1 (ASCII/Latin1)).
410           These can be used in such tests, though there is a small
411           possibility that Perl will become available in yet another
412           character set, breaking your test.  All but one of these code
413           points are C0 control characters.  The most significant controls
414           that are the same are "\0", "\r", and "\N{VT}" (also specifiable as
415           "\cK", "\x0B", "\N{U+0B}", or "\013").  The single non-control is
416           U+00B6 PILCROW SIGN.  The controls that are the same have the same
417           bit pattern in all 4 character sets, regardless of the UTF8ness of
418           the string containing them.  The bit pattern for U+B6 is the same
419           in all 4 for non-UTF8 strings, but differs in each when its
420           containing string is UTF-8 encoded.  The only other code points
421           that have some sort of sameness across all 4 character sets are the
422           pair 0xDC and 0xFC.  Together these represent upper- and lowercase
423           LATIN LETTER U WITH DIAERESIS, but which is upper and which is
424           lower may be reversed: 0xDC is the capital in Latin1 and 0xFC is
425           the small letter, while 0xFC is the capital in EBCDIC and 0xDC is
426           the small one.  This factoid may be exploited in writing case
427           insensitive tests that are the same across all 4 character sets.
428
429       •   Assuming the character set is just ASCII
430
431           ASCII is a 7 bit encoding, but bytes have 8 bits in them.  The 128
432           extra characters have different meanings depending on the locale.
433           Absent a locale, currently these extra characters are generally
434           considered to be unassigned, and this has presented some problems.
435           This has being changed starting in 5.12 so that these characters
436           can be considered to be Latin-1 (ISO-8859-1).
437
438       •   Mixing #define and #ifdef
439
440             #define BURGLE(x) ... \
441             #ifdef BURGLE_OLD_STYLE        /* BAD */
442             ... do it the old way ... \
443             #else
444             ... do it the new way ... \
445             #endif
446
447           You cannot portably "stack" cpp directives.  For example in the
448           above you need two separate BURGLE() #defines, one for each #ifdef
449           branch.
450
451       •   Adding non-comment stuff after #endif or #else
452
453             #ifdef SNOSH
454             ...
455             #else !SNOSH    /* BAD */
456             ...
457             #endif SNOSH    /* BAD */
458
459           The #endif and #else cannot portably have anything non-comment
460           after them.  If you want to document what is going (which is a good
461           idea especially if the branches are long), use (C) comments:
462
463             #ifdef SNOSH
464             ...
465             #else /* !SNOSH */
466             ...
467             #endif /* SNOSH */
468
469           The gcc option "-Wendif-labels" warns about the bad variant (by
470           default on starting from Perl 5.9.4).
471
472       •   Having a comma after the last element of an enum list
473
474             enum color {
475               CERULEAN,
476               CHARTREUSE,
477               CINNABAR,     /* BAD */
478             };
479
480           is not portable.  Leave out the last comma.
481
482           Also note that whether enums are implicitly morphable to ints
483           varies between compilers, you might need to (int).
484
485       •   Mixing signed char pointers with unsigned char pointers
486
487             int foo(char *s) { ... }
488             ...
489             unsigned char *t = ...; /* Or U8* t = ... */
490             foo(t);   /* BAD */
491
492           While this is legal practice, it is certainly dubious, and
493           downright fatal in at least one platform: for example VMS cc
494           considers this a fatal error.  One cause for people often making
495           this mistake is that a "naked char" and therefore dereferencing a
496           "naked char pointer" have an undefined signedness: it depends on
497           the compiler and the flags of the compiler and the underlying
498           platform whether the result is signed or unsigned.  For this very
499           same reason using a 'char' as an array index is bad.
500
501       •   Macros that have string constants and their arguments as substrings
502           of the string constants
503
504             #define FOO(n) printf("number = %d\n", n)    /* BAD */
505             FOO(10);
506
507           Pre-ANSI semantics for that was equivalent to
508
509             printf("10umber = %d\10");
510
511           which is probably not what you were expecting.  Unfortunately at
512           least one reasonably common and modern C compiler does "real
513           backward compatibility" here, in AIX that is what still happens
514           even though the rest of the AIX compiler is very happily C89.
515
516       •   Using printf formats for non-basic C types
517
518              IV i = ...;
519              printf("i = %d\n", i);    /* BAD */
520
521           While this might by accident work in some platform (where IV
522           happens to be an "int"), in general it cannot.  IV might be
523           something larger.  Even worse the situation is with more specific
524           types (defined by Perl's configuration step in config.h):
525
526              Uid_t who = ...;
527              printf("who = %d\n", who);    /* BAD */
528
529           The problem here is that Uid_t might be not only not "int"-wide but
530           it might also be unsigned, in which case large uids would be
531           printed as negative values.
532
533           There is no simple solution to this because of printf()'s limited
534           intelligence, but for many types the right format is available as
535           with either 'f' or '_f' suffix, for example:
536
537              IVdf /* IV in decimal */
538              UVxf /* UV is hexadecimal */
539
540              printf("i = %"IVdf"\n", i); /* The IVdf is a string constant. */
541
542              Uid_t_f /* Uid_t in decimal */
543
544              printf("who = %"Uid_t_f"\n", who);
545
546           Or you can try casting to a "wide enough" type:
547
548              printf("i = %"IVdf"\n", (IV)something_very_small_and_signed);
549
550           See "Formatted Printing of Size_t and SSize_t" in perlguts for how
551           to print those.
552
553           Also remember that the %p format really does require a void
554           pointer:
555
556              U8* p = ...;
557              printf("p = %p\n", (void*)p);
558
559           The gcc option "-Wformat" scans for such problems.
560
561       •   Blindly passing va_list
562
563           Not all platforms support passing va_list to further varargs
564           (stdarg) functions.  The right thing to do is to copy the va_list
565           using the Perl_va_copy() if the NEED_VA_COPY is defined.
566
567       •   Using gcc statement expressions
568
569              val = ({...;...;...});    /* BAD */
570
571           While a nice extension, it's not portable.  Historically, Perl used
572           them in macros if available to gain some extra speed (essentially
573           as a funky form of inlining), but we now support (or emulate) C99
574           "static inline" functions, so use them instead. Declare functions
575           as "PERL_STATIC_INLINE" to transparently fall back to emulation
576           where needed.
577
578       •   Binding together several statements in a macro
579
580           Use the macros STMT_START and STMT_END.
581
582              STMT_START {
583                 ...
584              } STMT_END
585
586       •   Testing for operating systems or versions when should be testing
587           for features
588
589             #ifdef __FOONIX__    /* BAD */
590             foo = quux();
591             #endif
592
593           Unless you know with 100% certainty that quux() is only ever
594           available for the "Foonix" operating system and that is available
595           and correctly working for all past, present, and future versions of
596           "Foonix", the above is very wrong.  This is more correct (though
597           still not perfect, because the below is a compile-time check):
598
599             #ifdef HAS_QUUX
600             foo = quux();
601             #endif
602
603           How does the HAS_QUUX become defined where it needs to be?  Well,
604           if Foonix happens to be Unixy enough to be able to run the
605           Configure script, and Configure has been taught about detecting and
606           testing quux(), the HAS_QUUX will be correctly defined.  In other
607           platforms, the corresponding configuration step will hopefully do
608           the same.
609
610           In a pinch, if you cannot wait for Configure to be educated, or if
611           you have a good hunch of where quux() might be available, you can
612           temporarily try the following:
613
614             #if (defined(__FOONIX__) || defined(__BARNIX__))
615             # define HAS_QUUX
616             #endif
617
618             ...
619
620             #ifdef HAS_QUUX
621             foo = quux();
622             #endif
623
624           But in any case, try to keep the features and operating systems
625           separate.
626
627           A good resource on the predefined macros for various operating
628           systems, compilers, and so forth is
629           <http://sourceforge.net/p/predef/wiki/Home/>
630
631       •   Assuming the contents of static memory pointed to by the return
632           values of Perl wrappers for C library functions doesn't change.
633           Many C library functions return pointers to static storage that can
634           be overwritten by subsequent calls to the same or related
635           functions.  Perl has wrappers for some of these functions.
636           Originally many of those wrappers returned those volatile pointers.
637           But over time almost all of them have evolved to return stable
638           copies.  To cope with the remaining ones, do a "savepv" in perlapi
639           to make a copy, thus avoiding these problems.  You will have to
640           free the copy when you're done to avoid memory leaks.  If you don't
641           have control over when it gets freed, you'll need to make the copy
642           in a mortal scalar, like so
643
644            SvPVX(sv_2mortal(newSVpv(volatile_string, 0)))
645
646   Problematic System Interfaces
647       •   Perl strings are NOT the same as C strings:  They may contain "NUL"
648           characters, whereas a C string is terminated by the first "NUL".
649           That is why Perl API functions that deal with strings generally
650           take a pointer to the first byte and either a length or a pointer
651           to the byte just beyond the final one.
652
653           And this is the reason that many of the C library string handling
654           functions should not be used.  They don't cope with the full
655           generality of Perl strings.  It may be that your test cases don't
656           have embedded "NUL"s, and so the tests pass, whereas there may well
657           eventually arise real-world cases where they fail.  A lesson here
658           is to include "NUL"s in your tests.  Now it's fairly rare in most
659           real world cases to get "NUL"s, so your code may seem to work,
660           until one day a "NUL" comes along.
661
662           Here's an example.  It used to be a common paradigm, for decades,
663           in the perl core to use "strchr("list", c)" to see if the character
664           "c" is any of the ones given in "list", a double-quote-enclosed
665           string of the set of characters that we are seeing if "c" is one
666           of.  As long as "c" isn't a "NUL", it works.  But when "c" is a
667           "NUL", "strchr" returns a pointer to the terminating "NUL" in
668           "list".   This likely will result in a segfault or a security issue
669           when the caller uses that end pointer as the starting point to read
670           from.
671
672           A solution to this and many similar issues is to use the "mem"-foo
673           C library functions instead.  In this case "memchr" can be used to
674           see if "c" is in "list" and works even if "c" is "NUL".  These
675           functions need an additional parameter to give the string length.
676           In the case of literal string parameters, perl has defined macros
677           that calculate the length for you.  See "String Handling" in
678           perlapi.
679
680       •   malloc(0), realloc(0), calloc(0, 0) are non-portable.  To be
681           portable allocate at least one byte.  (In general you should rarely
682           need to work at this low level, but instead use the various malloc
683           wrappers.)
684
685snprintf() - the return type is unportable.  Use my_snprintf()
686           instead.
687
688   Security problems
689       Last but not least, here are various tips for safer coding.  See also
690       perlclib for libc/stdio replacements one should use.
691
692       •   Do not use gets()
693
694           Or we will publicly ridicule you.  Seriously.
695
696       •   Do not use tmpfile()
697
698           Use mkstemp() instead.
699
700       •   Do not use strcpy() or strcat() or strncpy() or strncat()
701
702           Use my_strlcpy() and my_strlcat() instead: they either use the
703           native implementation, or Perl's own implementation (borrowed from
704           the public domain implementation of INN).
705
706       •   Do not use sprintf() or vsprintf()
707
708           If you really want just plain byte strings, use my_snprintf() and
709           my_vsnprintf() instead, which will try to use snprintf() and
710           vsnprintf() if those safer APIs are available.  If you want
711           something fancier than a plain byte string, use "Perl_form"() or
712           SVs and "Perl_sv_catpvf()".
713
714           Note that glibc "printf()", "sprintf()", etc. are buggy before
715           glibc version 2.17.  They won't allow a "%.s" format with a
716           precision to create a string that isn't valid UTF-8 if the current
717           underlying locale of the program is UTF-8.  What happens is that
718           the %s and its operand are simply skipped without any notice.
719           <https://sourceware.org/bugzilla/show_bug.cgi?id=6530>.
720
721       •   Do not use atoi()
722
723           Use grok_atoUV() instead.  atoi() has ill-defined behavior on
724           overflows, and cannot be used for incremental parsing.  It is also
725           affected by locale, which is bad.
726
727       •   Do not use strtol() or strtoul()
728
729           Use grok_atoUV() instead.  strtol() or strtoul() (or their
730           IV/UV-friendly macro disguises, Strtol() and Strtoul(), or Atol()
731           and Atoul() are affected by locale, which is bad.
732

DEBUGGING

734       You can compile a special debugging version of Perl, which allows you
735       to use the "-D" option of Perl to tell more about what Perl is doing.
736       But sometimes there is no alternative than to dive in with a debugger,
737       either to see the stack trace of a core dump (very useful in a bug
738       report), or trying to figure out what went wrong before the core dump
739       happened, or how did we end up having wrong or unexpected results.
740
741   Poking at Perl
742       To really poke around with Perl, you'll probably want to build Perl for
743       debugging, like this:
744
745           ./Configure -d -DDEBUGGING
746           make
747
748       "-DDEBUGGING" turns on the C compiler's "-g" flag to have it produce
749       debugging information which will allow us to step through a running
750       program, and to see in which C function we are at (without the
751       debugging information we might see only the numerical addresses of the
752       functions, which is not very helpful). It will also turn on the
753       "DEBUGGING" compilation symbol which enables all the internal debugging
754       code in Perl.  There are a whole bunch of things you can debug with
755       this: perlrun lists them all, and the best way to find out about them
756       is to play about with them.  The most useful options are probably
757
758           l  Context (loop) stack processing
759           s  Stack snapshots (with v, displays all stacks)
760           t  Trace execution
761           o  Method and overloading resolution
762           c  String/numeric conversions
763
764       For example
765
766           $ perl -Dst -e '$a + 1'
767           ....
768           (-e:1)      gvsv(main::a)
769               =>  UNDEF
770           (-e:1)      const(IV(1))
771               =>  UNDEF  IV(1)
772           (-e:1)      add
773               =>  NV(1)
774
775       Some of the functionality of the debugging code can be achieved with a
776       non-debugging perl by using XS modules:
777
778           -Dr => use re 'debug'
779           -Dx => use O 'Debug'
780
781   Using a source-level debugger
782       If the debugging output of "-D" doesn't help you, it's time to step
783       through perl's execution with a source-level debugger.
784
785       •  We'll use "gdb" for our examples here; the principles will apply to
786          any debugger (many vendors call their debugger "dbx"), but check the
787          manual of the one you're using.
788
789       To fire up the debugger, type
790
791           gdb ./perl
792
793       Or if you have a core dump:
794
795           gdb ./perl core
796
797       You'll want to do that in your Perl source tree so the debugger can
798       read the source code.  You should see the copyright message, followed
799       by the prompt.
800
801           (gdb)
802
803       "help" will get you into the documentation, but here are the most
804       useful commands:
805
806       •  run [args]
807
808          Run the program with the given arguments.
809
810       •  break function_name
811
812       •  break source.c:xxx
813
814          Tells the debugger that we'll want to pause execution when we reach
815          either the named function (but see "Internal Functions" in
816          perlguts!) or the given line in the named source file.
817
818       •  step
819
820          Steps through the program a line at a time.
821
822       •  next
823
824          Steps through the program a line at a time, without descending into
825          functions.
826
827       •  continue
828
829          Run until the next breakpoint.
830
831       •  finish
832
833          Run until the end of the current function, then stop again.
834
835       •  'enter'
836
837          Just pressing Enter will do the most recent operation again - it's a
838          blessing when stepping through miles of source code.
839
840       •  ptype
841
842          Prints the C definition of the argument given.
843
844            (gdb) ptype PL_op
845            type = struct op {
846                OP *op_next;
847                OP *op_sibparent;
848                OP *(*op_ppaddr)(void);
849                PADOFFSET op_targ;
850                unsigned int op_type : 9;
851                unsigned int op_opt : 1;
852                unsigned int op_slabbed : 1;
853                unsigned int op_savefree : 1;
854                unsigned int op_static : 1;
855                unsigned int op_folded : 1;
856                unsigned int op_spare : 2;
857                U8 op_flags;
858                U8 op_private;
859            } *
860
861       •  print
862
863          Execute the given C code and print its results.  WARNING: Perl makes
864          heavy use of macros, and gdb does not necessarily support macros
865          (see later "gdb macro support").  You'll have to substitute them
866          yourself, or to invoke cpp on the source code files (see "The .i
867          Targets") So, for instance, you can't say
868
869              print SvPV_nolen(sv)
870
871          but you have to say
872
873              print Perl_sv_2pv_nolen(sv)
874
875       You may find it helpful to have a "macro dictionary", which you can
876       produce by saying "cpp -dM perl.c | sort".  Even then, cpp won't
877       recursively apply those macros for you.
878
879   gdb macro support
880       Recent versions of gdb have fairly good macro support, but in order to
881       use it you'll need to compile perl with macro definitions included in
882       the debugging information.  Using gcc version 3.1, this means
883       configuring with "-Doptimize=-g3".  Other compilers might use a
884       different switch (if they support debugging macros at all).
885
886   Dumping Perl Data Structures
887       One way to get around this macro hell is to use the dumping functions
888       in dump.c; these work a little like an internal Devel::Peek, but they
889       also cover OPs and other structures that you can't get at from Perl.
890       Let's take an example.  We'll use the "$a = $b + $c" we used before,
891       but give it a bit of context: "$b = "6XXXX"; $c = 2.3;".  Where's a
892       good place to stop and poke around?
893
894       What about "pp_add", the function we examined earlier to implement the
895       "+" operator:
896
897           (gdb) break Perl_pp_add
898           Breakpoint 1 at 0x46249f: file pp_hot.c, line 309.
899
900       Notice we use "Perl_pp_add" and not "pp_add" - see "Internal Functions"
901       in perlguts.  With the breakpoint in place, we can run our program:
902
903           (gdb) run -e '$b = "6XXXX"; $c = 2.3; $a = $b + $c'
904
905       Lots of junk will go past as gdb reads in the relevant source files and
906       libraries, and then:
907
908           Breakpoint 1, Perl_pp_add () at pp_hot.c:309
909           1396    dSP; dATARGET; bool useleft; SV *svl, *svr;
910           (gdb) step
911           311           dPOPTOPnnrl_ul;
912           (gdb)
913
914       We looked at this bit of code before, and we said that "dPOPTOPnnrl_ul"
915       arranges for two "NV"s to be placed into "left" and "right" - let's
916       slightly expand it:
917
918        #define dPOPTOPnnrl_ul  NV right = POPn; \
919                                SV *leftsv = TOPs; \
920                                NV left = USE_LEFT(leftsv) ? SvNV(leftsv) : 0.0
921
922       "POPn" takes the SV from the top of the stack and obtains its NV either
923       directly (if "SvNOK" is set) or by calling the "sv_2nv" function.
924       "TOPs" takes the next SV from the top of the stack - yes, "POPn" uses
925       "TOPs" - but doesn't remove it.  We then use "SvNV" to get the NV from
926       "leftsv" in the same way as before - yes, "POPn" uses "SvNV".
927
928       Since we don't have an NV for $b, we'll have to use "sv_2nv" to convert
929       it.  If we step again, we'll find ourselves there:
930
931           (gdb) step
932           Perl_sv_2nv (sv=0xa0675d0) at sv.c:1669
933           1669        if (!sv)
934           (gdb)
935
936       We can now use "Perl_sv_dump" to investigate the SV:
937
938           (gdb) print Perl_sv_dump(sv)
939           SV = PV(0xa057cc0) at 0xa0675d0
940           REFCNT = 1
941           FLAGS = (POK,pPOK)
942           PV = 0xa06a510 "6XXXX"\0
943           CUR = 5
944           LEN = 6
945           $1 = void
946
947       We know we're going to get 6 from this, so let's finish the subroutine:
948
949           (gdb) finish
950           Run till exit from #0  Perl_sv_2nv (sv=0xa0675d0) at sv.c:1671
951           0x462669 in Perl_pp_add () at pp_hot.c:311
952           311           dPOPTOPnnrl_ul;
953
954       We can also dump out this op: the current op is always stored in
955       "PL_op", and we can dump it with "Perl_op_dump".  This'll give us
956       similar output to CPAN module B::Debug.
957
958           (gdb) print Perl_op_dump(PL_op)
959           {
960           13  TYPE = add  ===> 14
961               TARG = 1
962               FLAGS = (SCALAR,KIDS)
963               {
964                   TYPE = null  ===> (12)
965                     (was rv2sv)
966                   FLAGS = (SCALAR,KIDS)
967                   {
968           11          TYPE = gvsv  ===> 12
969                       FLAGS = (SCALAR)
970                       GV = main::b
971                   }
972               }
973
974       # finish this later #
975
976   Using gdb to look at specific parts of a program
977       With the example above, you knew to look for "Perl_pp_add", but what if
978       there were multiple calls to it all over the place, or you didn't know
979       what the op was you were looking for?
980
981       One way to do this is to inject a rare call somewhere near what you're
982       looking for.  For example, you could add "study" before your method:
983
984           study;
985
986       And in gdb do:
987
988           (gdb) break Perl_pp_study
989
990       And then step until you hit what you're looking for.  This works well
991       in a loop if you want to only break at certain iterations:
992
993           for my $c (1..100) {
994               study if $c == 50;
995           }
996
997   Using gdb to look at what the parser/lexer are doing
998       If you want to see what perl is doing when parsing/lexing your code,
999       you can use "BEGIN {}":
1000
1001           print "Before\n";
1002           BEGIN { study; }
1003           print "After\n";
1004
1005       And in gdb:
1006
1007           (gdb) break Perl_pp_study
1008
1009       If you want to see what the parser/lexer is doing inside of "if" blocks
1010       and the like you need to be a little trickier:
1011
1012           if ($a && $b && do { BEGIN { study } 1 } && $c) { ... }
1013

SOURCE CODE STATIC ANALYSIS

1015       Various tools exist for analysing C source code statically, as opposed
1016       to dynamically, that is, without executing the code.  It is possible to
1017       detect resource leaks, undefined behaviour, type mismatches,
1018       portability problems, code paths that would cause illegal memory
1019       accesses, and other similar problems by just parsing the C code and
1020       looking at the resulting graph, what does it tell about the execution
1021       and data flows.  As a matter of fact, this is exactly how C compilers
1022       know to give warnings about dubious code.
1023
1024   lint
1025       The good old C code quality inspector, "lint", is available in several
1026       platforms, but please be aware that there are several different
1027       implementations of it by different vendors, which means that the flags
1028       are not identical across different platforms.
1029
1030       There is a "lint" target in Makefile, but you may have to diddle with
1031       the flags (see above).
1032
1033   Coverity
1034       Coverity (<http://www.coverity.com/>) is a product similar to lint and
1035       as a testbed for their product they periodically check several open
1036       source projects, and they give out accounts to open source developers
1037       to the defect databases.
1038
1039       There is Coverity setup for the perl5 project:
1040       <https://scan.coverity.com/projects/perl5>
1041
1042   HP-UX cadvise (Code Advisor)
1043       HP has a C/C++ static analyzer product for HP-UX caller Code Advisor.
1044       (Link not given here because the URL is horribly long and seems
1045       horribly unstable; use the search engine of your choice to find it.)
1046       The use of the "cadvise_cc" recipe with "Configure ...
1047       -Dcc=./cadvise_cc" (see cadvise "User Guide") is recommended; as is the
1048       use of "+wall".
1049
1050   cpd (cut-and-paste detector)
1051       The cpd tool detects cut-and-paste coding.  If one instance of the cut-
1052       and-pasted code changes, all the other spots should probably be
1053       changed, too.  Therefore such code should probably be turned into a
1054       subroutine or a macro.
1055
1056       cpd (<https://pmd.github.io/latest/pmd_userdocs_cpd.html>) is part of
1057       the pmd project (<https://pmd.github.io/>).  pmd was originally written
1058       for static analysis of Java code, but later the cpd part of it was
1059       extended to parse also C and C++.
1060
1061       Download the pmd-bin-X.Y.zip () from the SourceForge site, extract the
1062       pmd-X.Y.jar from it, and then run that on source code thusly:
1063
1064         java -cp pmd-X.Y.jar net.sourceforge.pmd.cpd.CPD \
1065          --minimum-tokens 100 --files /some/where/src --language c > cpd.txt
1066
1067       You may run into memory limits, in which case you should use the -Xmx
1068       option:
1069
1070         java -Xmx512M ...
1071
1072   gcc warnings
1073       Though much can be written about the inconsistency and coverage
1074       problems of gcc warnings (like "-Wall" not meaning "all the warnings",
1075       or some common portability problems not being covered by "-Wall", or
1076       "-ansi" and "-pedantic" both being a poorly defined collection of
1077       warnings, and so forth), gcc is still a useful tool in keeping our
1078       coding nose clean.
1079
1080       The "-Wall" is by default on.
1081
1082       It would be nice for "-pedantic") to be on always, but unfortunately it
1083       is not safe on all platforms - for example fatal conflicts with the
1084       system headers (Solaris being a prime example).  If Configure
1085       "-Dgccansipedantic" is used, the "cflags" frontend selects "-pedantic"
1086       for the platforms where it is known to be safe.
1087
1088       The following extra flags are added:
1089
1090       •   "-Wendif-labels"
1091
1092       •   "-Wextra"
1093
1094       •   "-Wc++-compat"
1095
1096       •   "-Wwrite-strings"
1097
1098       •   "-Werror=pointer-arith"
1099
1100       •   "-Werror=vla"
1101
1102       The following flags would be nice to have but they would first need
1103       their own Augean stablemaster:
1104
1105       •   "-Wshadow"
1106
1107       •   "-Wstrict-prototypes"
1108
1109       The "-Wtraditional" is another example of the annoying tendency of gcc
1110       to bundle a lot of warnings under one switch (it would be impossible to
1111       deploy in practice because it would complain a lot) but it does contain
1112       some warnings that would be beneficial to have available on their own,
1113       such as the warning about string constants inside macros containing the
1114       macro arguments: this behaved differently pre-ANSI than it does in
1115       ANSI, and some C compilers are still in transition, AIX being an
1116       example.
1117
1118   Warnings of other C compilers
1119       Other C compilers (yes, there are other C compilers than gcc) often
1120       have their "strict ANSI" or "strict ANSI with some portability
1121       extensions" modes on, like for example the Sun Workshop has its "-Xa"
1122       mode on (though implicitly), or the DEC (these days, HP...) has its
1123       "-std1" mode on.
1124

MEMORY DEBUGGERS

1126       NOTE 1: Running under older memory debuggers such as Purify, valgrind
1127       or Third Degree greatly slows down the execution: seconds become
1128       minutes, minutes become hours.  For example as of Perl 5.8.1, the
1129       ext/Encode/t/Unicode.t takes extraordinarily long to complete under
1130       e.g. Purify, Third Degree, and valgrind.  Under valgrind it takes more
1131       than six hours, even on a snappy computer.  The said test must be doing
1132       something that is quite unfriendly for memory debuggers.  If you don't
1133       feel like waiting, that you can simply kill away the perl process.
1134       Roughly valgrind slows down execution by factor 10, AddressSanitizer by
1135       factor 2.
1136
1137       NOTE 2: To minimize the number of memory leak false alarms (see
1138       "PERL_DESTRUCT_LEVEL" for more information), you have to set the
1139       environment variable PERL_DESTRUCT_LEVEL to 2.  For example, like this:
1140
1141           env PERL_DESTRUCT_LEVEL=2 valgrind ./perl -Ilib ...
1142
1143       NOTE 3: There are known memory leaks when there are compile-time errors
1144       within eval or require, seeing "S_doeval" in the call stack is a good
1145       sign of these.  Fixing these leaks is non-trivial, unfortunately, but
1146       they must be fixed eventually.
1147
1148       NOTE 4: DynaLoader will not clean up after itself completely unless
1149       Perl is built with the Configure option
1150       "-Accflags=-DDL_UNLOAD_ALL_AT_EXIT".
1151
1152   valgrind
1153       The valgrind tool can be used to find out both memory leaks and illegal
1154       heap memory accesses.  As of version 3.3.0, Valgrind only supports
1155       Linux on x86, x86-64 and PowerPC and Darwin (OS X) on x86 and x86-64.
1156       The special "test.valgrind" target can be used to run the tests under
1157       valgrind.  Found errors and memory leaks are logged in files named
1158       testfile.valgrind and by default output is displayed inline.
1159
1160       Example usage:
1161
1162           make test.valgrind
1163
1164       Since valgrind adds significant overhead, tests will take much longer
1165       to run.  The valgrind tests support being run in parallel to help with
1166       this:
1167
1168           TEST_JOBS=9 make test.valgrind
1169
1170       Note that the above two invocations will be very verbose as reachable
1171       memory and leak-checking is enabled by default.  If you want to just
1172       see pure errors, try:
1173
1174           VG_OPTS='-q --leak-check=no --show-reachable=no' TEST_JOBS=9 \
1175               make test.valgrind
1176
1177       Valgrind also provides a cachegrind tool, invoked on perl as:
1178
1179           VG_OPTS=--tool=cachegrind make test.valgrind
1180
1181       As system libraries (most notably glibc) are also triggering errors,
1182       valgrind allows to suppress such errors using suppression files.  The
1183       default suppression file that comes with valgrind already catches a lot
1184       of them.  Some additional suppressions are defined in t/perl.supp.
1185
1186       To get valgrind and for more information see
1187
1188           http://valgrind.org/
1189
1190   AddressSanitizer
1191       AddressSanitizer ("ASan") consists of a compiler instrumentation module
1192       and a run-time "malloc" library. ASan is available for a variety of
1193       architectures, operating systems, and compilers (see project link
1194       below).  It checks for unsafe memory usage, such as use after free and
1195       buffer overflow conditions, and is fast enough that you can easily
1196       compile your debugging or optimized perl with it. Modern versions of
1197       ASan check for memory leaks by default on most platforms, otherwise
1198       (e.g. x86_64 OS X) this feature can be enabled via
1199       "ASAN_OPTIONS=detect_leaks=1".
1200
1201       To build perl with AddressSanitizer, your Configure invocation should
1202       look like:
1203
1204           sh Configure -des -Dcc=clang \
1205              -Accflags=-fsanitize=address -Aldflags=-fsanitize=address \
1206              -Alddlflags=-shared\ -fsanitize=address \
1207              -fsanitize-blacklist=`pwd`/asan_ignore
1208
1209       where these arguments mean:
1210
1211       •   -Dcc=clang
1212
1213           This should be replaced by the full path to your clang executable
1214           if it is not in your path.
1215
1216       •   -Accflags=-fsanitize=address
1217
1218           Compile perl and extensions sources with AddressSanitizer.
1219
1220       •   -Aldflags=-fsanitize=address
1221
1222           Link the perl executable with AddressSanitizer.
1223
1224       •   -Alddlflags=-shared\ -fsanitize=address
1225
1226           Link dynamic extensions with AddressSanitizer.  You must manually
1227           specify "-shared" because using "-Alddlflags=-shared" will prevent
1228           Configure from setting a default value for "lddlflags", which
1229           usually contains "-shared" (at least on Linux).
1230
1231       •   -fsanitize-blacklist=`pwd`/asan_ignore
1232
1233           AddressSanitizer will ignore functions listed in the "asan_ignore"
1234           file. (This file should contain a short explanation of why each of
1235           the functions is listed.)
1236
1237       See also <https://github.com/google/sanitizers/wiki/AddressSanitizer>.
1238

PROFILING

1240       Depending on your platform there are various ways of profiling Perl.
1241
1242       There are two commonly used techniques of profiling executables:
1243       statistical time-sampling and basic-block counting.
1244
1245       The first method takes periodically samples of the CPU program counter,
1246       and since the program counter can be correlated with the code generated
1247       for functions, we get a statistical view of in which functions the
1248       program is spending its time.  The caveats are that very small/fast
1249       functions have lower probability of showing up in the profile, and that
1250       periodically interrupting the program (this is usually done rather
1251       frequently, in the scale of milliseconds) imposes an additional
1252       overhead that may skew the results.  The first problem can be
1253       alleviated by running the code for longer (in general this is a good
1254       idea for profiling), the second problem is usually kept in guard by the
1255       profiling tools themselves.
1256
1257       The second method divides up the generated code into basic blocks.
1258       Basic blocks are sections of code that are entered only in the
1259       beginning and exited only at the end.  For example, a conditional jump
1260       starts a basic block.  Basic block profiling usually works by
1261       instrumenting the code by adding enter basic block #nnnn book-keeping
1262       code to the generated code.  During the execution of the code the basic
1263       block counters are then updated appropriately.  The caveat is that the
1264       added extra code can skew the results: again, the profiling tools
1265       usually try to factor their own effects out of the results.
1266
1267   Gprof Profiling
1268       gprof is a profiling tool available in many Unix platforms which uses
1269       statistical time-sampling.  You can build a profiled version of perl by
1270       compiling using gcc with the flag "-pg".  Either edit config.sh or re-
1271       run Configure.  Running the profiled version of Perl will create an
1272       output file called gmon.out which contains the profiling data collected
1273       during the execution.
1274
1275       quick hint:
1276
1277           $ sh Configure -des -Dusedevel -Accflags='-pg' \
1278               -Aldflags='-pg' -Alddlflags='-pg -shared' \
1279               && make perl
1280           $ ./perl ... # creates gmon.out in current directory
1281           $ gprof ./perl > out
1282           $ less out
1283
1284       (you probably need to add "-shared" to the <-Alddlflags> line until RT
1285       #118199 is resolved)
1286
1287       The gprof tool can then display the collected data in various ways.
1288       Usually gprof understands the following options:
1289
1290       •   -a
1291
1292           Suppress statically defined functions from the profile.
1293
1294       •   -b
1295
1296           Suppress the verbose descriptions in the profile.
1297
1298       •   -e routine
1299
1300           Exclude the given routine and its descendants from the profile.
1301
1302       •   -f routine
1303
1304           Display only the given routine and its descendants in the profile.
1305
1306       •   -s
1307
1308           Generate a summary file called gmon.sum which then may be given to
1309           subsequent gprof runs to accumulate data over several runs.
1310
1311       •   -z
1312
1313           Display routines that have zero usage.
1314
1315       For more detailed explanation of the available commands and output
1316       formats, see your own local documentation of gprof.
1317
1318   GCC gcov Profiling
1319       basic block profiling is officially available in gcc 3.0 and later.
1320       You can build a profiled version of perl by compiling using gcc with
1321       the flags "-fprofile-arcs -ftest-coverage".  Either edit config.sh or
1322       re-run Configure.
1323
1324       quick hint:
1325
1326           $ sh Configure -des -Dusedevel -Doptimize='-g' \
1327               -Accflags='-fprofile-arcs -ftest-coverage' \
1328               -Aldflags='-fprofile-arcs -ftest-coverage' \
1329               -Alddlflags='-fprofile-arcs -ftest-coverage -shared' \
1330               && make perl
1331           $ rm -f regexec.c.gcov regexec.gcda
1332           $ ./perl ...
1333           $ gcov regexec.c
1334           $ less regexec.c.gcov
1335
1336       (you probably need to add "-shared" to the <-Alddlflags> line until RT
1337       #118199 is resolved)
1338
1339       Running the profiled version of Perl will cause profile output to be
1340       generated.  For each source file an accompanying .gcda file will be
1341       created.
1342
1343       To display the results you use the gcov utility (which should be
1344       installed if you have gcc 3.0 or newer installed).  gcov is run on
1345       source code files, like this
1346
1347           gcov sv.c
1348
1349       which will cause sv.c.gcov to be created.  The .gcov files contain the
1350       source code annotated with relative frequencies of execution indicated
1351       by "#" markers.  If you want to generate .gcov files for all profiled
1352       object files, you can run something like this:
1353
1354           for file in `find . -name \*.gcno`
1355           do sh -c "cd `dirname $file` && gcov `basename $file .gcno`"
1356           done
1357
1358       Useful options of gcov include "-b" which will summarise the basic
1359       block, branch, and function call coverage, and "-c" which instead of
1360       relative frequencies will use the actual counts.  For more information
1361       on the use of gcov and basic block profiling with gcc, see the latest
1362       GNU CC manual.  As of gcc 4.8, this is at
1363       <http://gcc.gnu.org/onlinedocs/gcc/Gcov-Intro.html#Gcov-Intro>
1364
1365   callgrind profiling
1366       callgrind is a valgrind tool for profiling source code. Paired with
1367       kcachegrind (a Qt based UI), it gives you an overview of where code is
1368       taking up time, as well as the ability to examine callers, call trees,
1369       and more. One of its benefits is you can use it on perl and XS modules
1370       that have not been compiled with debugging symbols.
1371
1372       If perl is compiled with debugging symbols ("-g"), you can view the
1373       annotated source and click around, much like Devel::NYTProf's HTML
1374       output.
1375
1376       For basic usage:
1377
1378           valgrind --tool=callgrind ./perl ...
1379
1380       By default it will write output to callgrind.out.PID, but you can
1381       change that with "--callgrind-out-file=..."
1382
1383       To view the data, do:
1384
1385           kcachegrind callgrind.out.PID
1386
1387       If you'd prefer to view the data in a terminal, you can use
1388       callgrind_annotate. In it's basic form:
1389
1390           callgrind_annotate callgrind.out.PID | less
1391
1392       Some useful options are:
1393
1394       •   --threshold
1395
1396           Percentage of counts (of primary sort event) we are interested in.
1397           The default is 99%, 100% might show things that seem to be missing.
1398
1399       •   --auto
1400
1401           Annotate all source files containing functions that helped reach
1402           the event count threshold.
1403

MISCELLANEOUS TRICKS

1405   PERL_DESTRUCT_LEVEL
1406       If you want to run any of the tests yourself manually using e.g.
1407       valgrind, please note that by default perl does not explicitly cleanup
1408       all the memory it has allocated (such as global memory arenas) but
1409       instead lets the exit() of the whole program "take care" of such
1410       allocations, also known as "global destruction of objects".
1411
1412       There is a way to tell perl to do complete cleanup: set the environment
1413       variable PERL_DESTRUCT_LEVEL to a non-zero value.  The t/TEST wrapper
1414       does set this to 2, and this is what you need to do too, if you don't
1415       want to see the "global leaks": For example, for running under valgrind
1416
1417           env PERL_DESTRUCT_LEVEL=2 valgrind ./perl -Ilib t/foo/bar.t
1418
1419       (Note: the mod_perl apache module uses also this environment variable
1420       for its own purposes and extended its semantics.  Refer to the mod_perl
1421       documentation for more information.  Also, spawned threads do the
1422       equivalent of setting this variable to the value 1.)
1423
1424       If, at the end of a run you get the message N scalars leaked, you can
1425       recompile with "-DDEBUG_LEAKING_SCALARS", ("Configure
1426       -Accflags=-DDEBUG_LEAKING_SCALARS"), which will cause the addresses of
1427       all those leaked SVs to be dumped along with details as to where each
1428       SV was originally allocated.  This information is also displayed by
1429       Devel::Peek.  Note that the extra details recorded with each SV
1430       increases memory usage, so it shouldn't be used in production
1431       environments.  It also converts "new_SV()" from a macro into a real
1432       function, so you can use your favourite debugger to discover where
1433       those pesky SVs were allocated.
1434
1435       If you see that you're leaking memory at runtime, but neither valgrind
1436       nor "-DDEBUG_LEAKING_SCALARS" will find anything, you're probably
1437       leaking SVs that are still reachable and will be properly cleaned up
1438       during destruction of the interpreter.  In such cases, using the "-Dm"
1439       switch can point you to the source of the leak.  If the executable was
1440       built with "-DDEBUG_LEAKING_SCALARS", "-Dm" will output SV allocations
1441       in addition to memory allocations.  Each SV allocation has a distinct
1442       serial number that will be written on creation and destruction of the
1443       SV.  So if you're executing the leaking code in a loop, you need to
1444       look for SVs that are created, but never destroyed between each cycle.
1445       If such an SV is found, set a conditional breakpoint within "new_SV()"
1446       and make it break only when "PL_sv_serial" is equal to the serial
1447       number of the leaking SV.  Then you will catch the interpreter in
1448       exactly the state where the leaking SV is allocated, which is
1449       sufficient in many cases to find the source of the leak.
1450
1451       As "-Dm" is using the PerlIO layer for output, it will by itself
1452       allocate quite a bunch of SVs, which are hidden to avoid recursion.
1453       You can bypass the PerlIO layer if you use the SV logging provided by
1454       "-DPERL_MEM_LOG" instead.
1455
1456   PERL_MEM_LOG
1457       If compiled with "-DPERL_MEM_LOG" ("-Accflags=-DPERL_MEM_LOG"), both
1458       memory and SV allocations go through logging functions, which is handy
1459       for breakpoint setting.
1460
1461       Unless "-DPERL_MEM_LOG_NOIMPL" ("-Accflags=-DPERL_MEM_LOG_NOIMPL") is
1462       also compiled, the logging functions read $ENV{PERL_MEM_LOG} to
1463       determine whether to log the event, and if so how:
1464
1465           $ENV{PERL_MEM_LOG} =~ /m/           Log all memory ops
1466           $ENV{PERL_MEM_LOG} =~ /s/           Log all SV ops
1467           $ENV{PERL_MEM_LOG} =~ /t/           include timestamp in Log
1468           $ENV{PERL_MEM_LOG} =~ /^(\d+)/      write to FD given (default is 2)
1469
1470       Memory logging is somewhat similar to "-Dm" but is independent of
1471       "-DDEBUGGING", and at a higher level; all uses of Newx(), Renew(), and
1472       Safefree() are logged with the caller's source code file and line
1473       number (and C function name, if supported by the C compiler).  In
1474       contrast, "-Dm" is directly at the point of "malloc()".  SV logging is
1475       similar.
1476
1477       Since the logging doesn't use PerlIO, all SV allocations are logged and
1478       no extra SV allocations are introduced by enabling the logging.  If
1479       compiled with "-DDEBUG_LEAKING_SCALARS", the serial number for each SV
1480       allocation is also logged.
1481
1482   DDD over gdb
1483       Those debugging perl with the DDD frontend over gdb may find the
1484       following useful:
1485
1486       You can extend the data conversion shortcuts menu, so for example you
1487       can display an SV's IV value with one click, without doing any typing.
1488       To do that simply edit ~/.ddd/init file and add after:
1489
1490         ! Display shortcuts.
1491         Ddd*gdbDisplayShortcuts: \
1492         /t ()   // Convert to Bin\n\
1493         /d ()   // Convert to Dec\n\
1494         /x ()   // Convert to Hex\n\
1495         /o ()   // Convert to Oct(\n\
1496
1497       the following two lines:
1498
1499         ((XPV*) (())->sv_any )->xpv_pv  // 2pvx\n\
1500         ((XPVIV*) (())->sv_any )->xiv_iv // 2ivx
1501
1502       so now you can do ivx and pvx lookups or you can plug there the sv_peek
1503       "conversion":
1504
1505         Perl_sv_peek(my_perl, (SV*)()) // sv_peek
1506
1507       (The my_perl is for threaded builds.)  Just remember that every line,
1508       but the last one, should end with \n\
1509
1510       Alternatively edit the init file interactively via: 3rd mouse button ->
1511       New Display -> Edit Menu
1512
1513       Note: you can define up to 20 conversion shortcuts in the gdb section.
1514
1515   C backtrace
1516       On some platforms Perl supports retrieving the C level backtrace
1517       (similar to what symbolic debuggers like gdb do).
1518
1519       The backtrace returns the stack trace of the C call frames, with the
1520       symbol names (function names), the object names (like "perl"), and if
1521       it can, also the source code locations (file:line).
1522
1523       The supported platforms are Linux, and OS X (some *BSD might work at
1524       least partly, but they have not yet been tested).
1525
1526       This feature hasn't been tested with multiple threads, but it will only
1527       show the backtrace of the thread doing the backtracing.
1528
1529       The feature needs to be enabled with "Configure -Dusecbacktrace".
1530
1531       The "-Dusecbacktrace" also enables keeping the debug information when
1532       compiling/linking (often: "-g").  Many compilers/linkers do support
1533       having both optimization and keeping the debug information.  The debug
1534       information is needed for the symbol names and the source locations.
1535
1536       Static functions might not be visible for the backtrace.
1537
1538       Source code locations, even if available, can often be missing or
1539       misleading if the compiler has e.g. inlined code.  Optimizer can make
1540       matching the source code and the object code quite challenging.
1541
1542       Linux
1543           You must have the BFD (-lbfd) library installed, otherwise "perl"
1544           will fail to link.  The BFD is usually distributed as part of the
1545           GNU binutils.
1546
1547           Summary: "Configure ... -Dusecbacktrace" and you need "-lbfd".
1548
1549       OS X
1550           The source code locations are supported only if you have the
1551           Developer Tools installed.  (BFD is not needed.)
1552
1553           Summary: "Configure ... -Dusecbacktrace" and installing the
1554           Developer Tools would be good.
1555
1556       Optionally, for trying out the feature, you may want to enable
1557       automatic dumping of the backtrace just before a warning or croak (die)
1558       message is emitted, by adding "-Accflags=-DUSE_C_BACKTRACE_ON_ERROR"
1559       for Configure.
1560
1561       Unless the above additional feature is enabled, nothing about the
1562       backtrace functionality is visible, except for the Perl/XS level.
1563
1564       Furthermore, even if you have enabled this feature to be compiled, you
1565       need to enable it in runtime with an environment variable:
1566       "PERL_C_BACKTRACE_ON_ERROR=10".  It must be an integer higher than
1567       zero, telling the desired frame count.
1568
1569       Retrieving the backtrace from Perl level (using for example an XS
1570       extension) would be much less exciting than one would hope: normally
1571       you would see "runops", "entersub", and not much else.  This API is
1572       intended to be called from within the Perl implementation, not from
1573       Perl level execution.
1574
1575       The C API for the backtrace is as follows:
1576
1577       get_c_backtrace
1578       free_c_backtrace
1579       get_c_backtrace_dump
1580       dump_c_backtrace
1581
1582   Poison
1583       If you see in a debugger a memory area mysteriously full of 0xABABABAB
1584       or 0xEFEFEFEF, you may be seeing the effect of the Poison() macros, see
1585       perlclib.
1586
1587   Read-only optrees
1588       Under ithreads the optree is read only.  If you want to enforce this,
1589       to check for write accesses from buggy code, compile with
1590       "-Accflags=-DPERL_DEBUG_READONLY_OPS" to enable code that allocates op
1591       memory via "mmap", and sets it read-only when it is attached to a
1592       subroutine.  Any write access to an op results in a "SIGBUS" and abort.
1593
1594       This code is intended for development only, and may not be portable
1595       even to all Unix variants.  Also, it is an 80% solution, in that it
1596       isn't able to make all ops read only.  Specifically it does not apply
1597       to op slabs belonging to "BEGIN" blocks.
1598
1599       However, as an 80% solution it is still effective, as it has caught
1600       bugs in the past.
1601
1602   When is a bool not a bool?
1603       There wasn't necessarily a standard "bool" type on compilers prior to
1604       C99, and so some workarounds were created.  The "TRUE" and "FALSE"
1605       macros are still available as alternatives for "true" and "false".  And
1606       the "cBOOL" macro was created to correctly cast to a true/false value
1607       in all circumstances, but should no longer be necessary.  Using
1608       "(bool)" expr> should now always work.
1609
1610       There are no plans to remove any of "TRUE", "FALSE", nor "cBOOL".
1611
1612   Finding unsafe truncations
1613       You may wish to run "Configure" with something like
1614
1615           -Accflags='-Wconversion -Wno-sign-conversion -Wno-shorten-64-to-32'
1616
1617       or your compiler's equivalent to make it easier to spot any unsafe
1618       truncations that show up.
1619
1620   The .i Targets
1621       You can expand the macros in a foo.c file by saying
1622
1623           make foo.i
1624
1625       which will expand the macros using cpp.  Don't be scared by the
1626       results.
1627

AUTHOR

1629       This document was originally written by Nathan Torkington, and is
1630       maintained by the perl5-porters mailing list.
1631
1632
1633
1634perl v5.36.0                      2022-08-30                   PERLHACKTIPS(1)
Impressum