1PAR::Tutorial(3) User Contributed Perl Documentation PAR::Tutorial(3)
2
3
4
6 PAR::Tutorial - Cross-Platform Packaging and Deployment with PAR
7
9 This is a tutorial on PAR, first appeared at the 7th Perl Conference.
10 The HTML version of this tutorial is available online as
11 <http://search.cpan.org/perldoc?PAR::Tutorial>
12
14 On Deploying Perl Applications
15 % sshnuke.pl 10.2.2.2 -rootpw="Z1ON0101"
16 Perl v5.6.1 required--this is only v5.6.0, stopped at sshnuke.pl line 1.
17 BEGIN failed--compilation aborted at sshnuke.pl line 1.
18
19 • Q: "Help! I can't run your program!"
20
21 • A1: Install Perl & "perl -MCPAN -e'install(...)'"
22
23 • How do we know which modules are needed?
24
25 • New versions of CPAN modules may break "sshnuke.pl"
26
27 • A2: Install Perl & "tar zxf my_perllib.tgz"
28
29 • Possibly overwriting existing modules; not cross-platform at
30 all
31
32 • A3: Use the executable generated by "perlcc sshnuke.pl"
33
34 • Impossible to debug; "perlcc" usually does not work anyway
35
36 PAR, the Perl Archive Toolkit
37 • Do what JAR (Java Archive) does for Perl
38
39 • Aggregates modules, scripts and other files into a Zip file
40
41 • Easy to generate, update and extract
42
43 • Version consistency: solves forward-compatibility problems
44
45 • Developed by community: "par@perl.org"
46
47 • PAR files can be packed into self-contained scripts
48
49 • Automatically scans perl script for dependencies
50
51 • Bundles all necessary 3rd-party modules with it
52
53 • Requires only core Perl to run on the target machine
54
55 • PAR also comes with "pp", the Perl Packager:
56
57 % pp -o sshnuke.exe sshnuke.pl # stand-alone executable!
58
59 Simple Packaging
60 • PAR files are just Zip files with modules in it
61
62 • Any Zip tools can generate them:
63
64 % zip foo.par Hello.pm World.pm # pack two modules
65 % zip -r bar.par lib/ # grab all modules in lib/
66
67 • To load modules from PAR files:
68
69 use PAR;
70 use lib "foo.par"; # the .par part is optional
71 use Hello;
72
73 • This also works:
74
75 use PAR "/home/mylibs/*.par"; # put all of them into @INC
76 use Hello;
77
78 PAR Loaders
79 • Use "par.pl" to run files inside a PAR archive:
80
81 % par.pl foo.par # looks for 'main.pl' by default
82 % par.pl foo.par test.pl # runs script/test.pl in foo.par
83
84 • Same thing, with the stand-alone "parl" or "parl.exe":
85
86 % parl foo.par # no perl or PAR.pm needed!
87 % parl foo.par test.pl # ditto
88
89 • The PAR loader can prepend itself to a PAR file:
90
91 • "-b" bundles non-core modules needed by "PAR.pm":
92
93 % par.pl -b -O./foo.pl foo.par # self-contained script
94
95 • "-B" bundles core modules in addition to "-b":
96
97 % parl -B -O./foo.exe foo.par # self-contained binary
98
99 Dependency Scanning
100 • Recursively scan dependencies with "scandeps.pl":
101
102 % scandeps.pl sshnuke.pl
103 # Legend: [C]ore [X]ternal [S]ubmodule [?]NotOnCPAN
104 'Crypt::SSLeay' => '0', # X #
105 'Net::HTTP' => '0', # #
106 'Crypt::SSLeay::X509' => '0', # S # Crypt::SSLeay
107 'Net::HTTP::Methods' => '0', # S # Net::HTTP
108 'Compress::Zlib' => '0', # X # Net::HTTP::Methods
109
110 • Scan an one-liner, list all involved files:
111
112 % scandeps.pl -V -e "use Dynaloader;"
113 ...
114 # auto/DynaLoader/dl_findfile.al [autoload]
115 # auto/DynaLoader/extralibs.ld [autoload]
116 # auto/File/Glob/Glob.bs [data]
117 # auto/File/Glob/Glob.so [shared]
118 ...
119
120 Perl Packager: "pp"
121 • Combines scanning, zipping and loader-embedding:
122
123 % pp -o out.exe src.pl # self-contained .exe
124 % out.exe # runs anywhere on the same OS
125
126 • Bundle additional modules:
127
128 % pp -o out.exe -M CGI src.pl # pack CGI + its dependencies, too
129
130 • Pack one-liners:
131
132 % pp -o out.exe -e 'print "Hi!"' # turns one-liner into executable
133
134 • Generate PAR files instead of executables:
135
136 % pp -p src.pl # makes 'source.par'
137 % pp -B -p src.pl # include core modules
138
139 How it works
140 • Command-line options are almost identical to "perlcc"'s
141
142 • Also supports "gcc"-style long options:
143
144 % pp --gui --verbose --output=out.exe src.pl
145
146 • Small initial overhead; no runtime overhead
147
148 • Dependencies are POD-stripped before packing
149
150 • Loads modules directly into memory on demand
151
152 • Shared libraries (DLLs) are extracted with File::Temp
153
154 • Works on Perl 5.6.0 or above
155
156 • Tested on Win32 (VC++ and MinGW), FreeBSD, NetBSD, Linux, MacOSX,
157 Cygwin, AIX, Solaris, HP-UX, Tru64...
158
159 Aggregating multiple programs
160 • A common question:
161
162 > I have used pp to make several standalone applications which work
163 > great, the only problem is that for each executable that I make, I am
164 > assuming the parl.exe is somehow bundled into the resulting exe.
165
166 • The obvious workaround:
167
168 You can ship parl.exe by itself, along with .par files built
169 by "pp -p", and run those PAR files by associating them to parl.exe.
170
171 • On platforms that have "ln", there is a better solution:
172
173 % pp --output=a.out a.pl b.pl # two scripts in one!
174 % ln a.out b.out # symlink also works
175 % ./a.out # runs a.pl
176 % ./b.out # runs b.pl
177
178 Cross-platform Packages
179 • Of course, there is no cross-platform binary format
180
181 • Pure-perl PAR packages are cross-platform by default
182
183 • However, XS modules are specific to Perl version and platform
184
185 • Multiple versions of a XS module can co-exist in a PAR file
186
187 • Suppose we need "out.par" on both Win32 and Finix:
188
189 C:\> pp --multiarch --output=out.par src.pl
190 ...copy src.pl and out.par to a Finix machine...
191 % pp --multiarch --output=out.par src.pl
192
193 • Now it works on both platforms:
194
195 % parl out.par # runs src.pl
196 % perl -MPAR=out.par -e '...' # uses modules inside out.par
197
198 The Anatomy of a PAR file
199 • Modules can reside in several directories:
200
201 / # casual packaging only
202 /lib/ # standard location
203 /arch/ # for creating from blib/
204 /i386-freebsd/ # i.e. $Config{archname}
205 /5.8.0/ # i.e. Perl version number
206 /5.8.0/i386-freebsd/ # combination of the two above
207
208 • Scripts are stored in one of the two locations:
209
210 / # casual packaging only
211 /script/ # standard location
212
213 • Shared libraries may be architecture- or perl-version-specific:
214
215 /shlib/(5.8.0/)?(i386-freebsd/)?
216
217 • PAR files may recursively contain other PAR files:
218
219 /par/(5.8.0/)?(i386-freebsd/)?
220
221 Special files
222 • MANIFEST
223
224 • Index of all files inside PAR
225
226 • Can be parsed with "ExtUtils::Manifest"
227
228 • META.yml
229
230 • Dependency, license, runtime options
231
232 • Can be parsed with "YAML"
233
234 • SIGNATURE
235
236 • OpenPGP-signed digital signature
237
238 • Can be parsed and verified with "Module::Signature"
239
240 Advantages over perlcc, PerlApp and Perl2exe
241 • This is not meant to be a flame
242
243 • All three maintainers have contributed to PAR directly; I'm
244 grateful
245
246 • perlcc
247
248 • "The code generated in this way is not guaranteed to work...
249 Use for production purposes is strongly discouraged." (from
250 perldoc perlcc)
251
252 • Guaranteed to not work is more like it
253
254 • PerlApp / Perl2exe
255
256 • Expensive: Need to pay for each upgrade
257
258 • Non-portable: Only available for limited platforms
259
260 • Proprietary: Cannot extend its features or fix bugs
261
262 • Obfuscated: Vendor and black-hats can see your code, but you
263 can't
264
265 • Inflexible: Does not work with existing Perl installations
266
267 MANIFEST: Best viewed with Mozilla
268 • The URL of "MANIFEST" inside "/home/autrijus/foo.par":
269
270 jar:file:///home/autrijus/foo.par!/MANIFEST
271
272 • Open it in a Gecko browser (e.g. Netscape 6+) with Javascript
273 enabled:
274
275 • No needed to unzip anything; just click on files to view them
276
277 META.yml: Metadata galore
278 • Static, machine-readable distribution metadata
279
280 • Supported by "Module::Build", "ExtUtils::MakeMaker",
281 "Module::Install"
282
283 • A typical "pp"-generated "META.yml" looks like this:
284
285 build_requires: {}
286 conflicts: {}
287 dist_name: out.par
288 distribution_type: par
289 dynamic_config: 0
290 generated_by: 'Perl Packager version 0.03'
291 license: unknown
292 par:
293 clean: 0
294 signature: ''
295 verbatim: 0
296 version: 0.68
297
298 • The "par:" settings controls its runtime behavior
299
300 SIGNATURE: Signing and verifying packages
301 • OpenPGP clear-signed manifest with SHA1 digests
302
303 • Supported by "Module::Signature", "CPANPLUS" and
304 "Module::Build"
305
306 • A typical "SIGNATURE" looks like this:
307
308 -----BEGIN PGP SIGNED MESSAGE-----
309 Hash: SHA1
310
311 SHA1 8a014cd6d0f6775552a01d1e6354a69eb6826046 AUTHORS
312 ...
313 -----BEGIN PGP SIGNATURE-----
314 ...
315 -----END PGP SIGNATURE-----
316
317 • Use "pp" and "cpansign" to work with signatures:
318
319 % pp -s -o foo.par bar.pl # make and sign foo.par from bar.pl
320 % cpansign -s foo.par # sign this PAR file
321 % cpansign -v foo.par # verify this PAR file
322
323 Perl Servlets with Apache::PAR
324 • Framework for self-contained Web applications
325
326 • Similar to Java's "Web Application Archive" (WAR) files
327
328 • Works with mod_perl 1.x or 2.x
329
330 • A complete web application inside a ".par" file
331
332 • Apache configuration, static files, Perl modules...
333
334 • Supports Static, Registry and PerlRun handlers
335
336 • Can also load all PARs under a directory
337
338 • One additional special file: "web.conf"
339
340 Alias /myapp/cgi-perl/ ##PARFILE##/
341 <Location /myapp/cgi-perl>
342 Options +ExecCGI
343 SetHandler perl-script
344 PerlHandler Apache::PAR::Registry
345 </Location>
346
347 Hon Dah, A-par-che!
348 • First, make a "hondah.par" from an one-liner:
349
350 # use the "web.conf" from the previous slide
351 % pp -p -o hondah.par -e 'print "Hon Dah!\n"' \
352 --add web.conf
353 % chmod a+x hondah.par
354
355 • Add this to "httpd.conf", then restart apache:
356
357 <IfDefine MODPERL2>
358 PerlModule Apache2
359 </IfDefine>
360 PerlAddVar PARInclude /home/autrijus/hondah.par
361 PerlModule Apache::PAR
362
363 • Test it out:
364
365 % GET http://localhost/myapp/cgi-perl/main.pl
366 Hon Dah!
367
368 • Instant one-liner web application that works!
369
370 On-demand library fetching
371 • With LWP installed, your can use remote PAR files:
372
373 use PAR;
374 use lib 'http://aut.dyndns.org/par/DBI-latest.par';
375 use DBI; # always up to date!
376
377 • Modules are now cached under $ENV{PAR_GLOBAL_TEMP}
378
379 • Auto-updates with "LWP::Simple::mirror"
380
381 • Download only if modified
382
383 • Safe for offline use after the first time
384
385 • May use "SIGNATURE" to prevent DNS-spoofing
386
387 • Makes large-scale deployment a breeze
388
389 • Upgrades from a central location
390
391 • No installers needed
392
393 Code Obfuscation
394 • Also known as source-hiding techniques
395
396 • It is not encryption
397
398 • Offered by PerlApp, Perl2Exe, Stunnix...
399
400 • Usually easy to defeat
401
402 • Take optree dump from memory, feed to "B::Deparse"
403
404 • If you just want to stop a casual "grep", "deflate" already
405 works
406
407 • PAR now supports pluggable input filters with "pp -f"
408
409 • Bundled examples: Bleach, PodStrip and PatchContent
410
411 • True encryption using "Crypt::*"
412
413 • Or even _product activation_ over the internet
414
415 • Alternatively, just keep core logic in your server and use RPC
416
417 Accessing packed files
418 • To get the host archive from a packed program:
419
420 my $zip = PAR::par_handle($0); # an Archive::Zip object
421 my $content = $zip->contents('MANIFEST');
422
423 • Same thing, but with "read_file()":
424
425 my $content = PAR::read_file('MANIFEST');
426
427 • Loaded PAR files are stored in %PAR::LibCache:
428
429 use PAR '/home/mylibs/*.par';
430 while (my ($filename, $zip) = each %PAR::LibCache) {
431 print "[$filename - MANIFEST]\n";
432 print $zip->contents('MANIFEST');
433 }
434
435 Packing GUI applications
436 • GUI toolkits often need to link with shared libraries:
437
438 # search for libncurses under library paths and pack it
439 % pp -l ncurses curses_app.pl # same for Tk, Wx, Gtk, Qt...
440
441 • Use "pp --gui" on Win32 to eliminate the console window:
442
443 # pack 'src.pl' into a console-less 'out.exe' (Win32 only)
444 % pp --gui -o out.exe src.pl
445
446 • "Can't locate Foo/Widget/Bar.pm in @INC"?
447
448 • Some toolkits (notably Tk) autoloads modules without "use" or
449 "require"
450
451 • Hence "pp" and "Module::ScanDeps" may fail to detect them
452
453 • Tk problems mostly fixed by now, but other toolkits may still
454 break
455
456 • You can work around it with "pp -M" or an explicit "require"
457
458 • Or better, send a short test-case to "par@perl.org" so we can
459 fix it
460
461 Precompiled CPAN distributions
462 • Installing XS extensions from CPAN was difficult
463
464 • Some platforms do not come with a compiler (Win32, MacOSX...)
465
466 • Some headers or libraries may be missing
467
468 • PAR.pm itself used to suffer from both problems
469
470 • ...but not anymore -- "Module::Install" to the rescue!
471
472 # same old Makefile.PL, with a few changes
473 use inc::Module::Install; # was "use ExtUtils::MakeMaker;"
474 WriteMakefile( ... ); # same as the original
475 check_nmake(); # make sure the user have nmake
476 par_base('AUTRIJUS'); # your CPAN ID or a URL
477 fetch_par() unless can_cc(); # use precompiled PAR only if necessary
478
479 • Users will not notice anything, except now it works
480
481 • Of course, you still need to type "make par" and upload the
482 precompiled package
483
484 • PAR users can also install it directly with "parl -i"
485
486 Thank you!
487 • Additional resources
488
489 • Mailing list: "par@perl.org"
490
491 • Subscribe: Send a blank email to "par-subscribe@perl.org"
492
493 • List archive: <http://nntp.x.perl.org/group/perl.par>
494
495 • PAR::Intro: <http://search.cpan.org/dist/PAR/lib/PAR/Intro.pod>
496
497 • Apache::PAR: <http://search.cpan.org/dist/Apache-PAR/>
498
499 • Module::Install: <http://search.cpan.org/dist/Module-Install/>
500
501 • Any questions?
502
503 Overview of PAR.pm's Implementation
504 • Here begins the scary part
505
506 • Grues, Dragons and Jabberwocks abound...
507
508 • You are going to learn weird things about Perl internals
509
510 • PAR invokes four areas of Perl arcana:
511
512 • @INC code references
513
514 • On-the-fly source filtering
515
516 • Overriding "DynaLoader::bootstrap()" to handle XS modules
517
518 • Making self-bootstrapping binary executables
519
520 • The first two only works on 5.6 or later
521
522 • DynaLoader and %INC are there since Perl 5 was born
523
524 • PAR currently needs 5.6, but a 5.005 port is possible
525
526 Code References in @INC
527 • On 1999-07-19, Ken Fox submitted a patch to P5P
528
529 • To _enable using remote modules_ by putting hooks in @INC
530
531 • It's accepted to come in Perl 5.6, but undocumented until 5.8
532
533 • Type "perldoc -f require" to read the nitty-gritty details
534
535 • Coderefs in @INC may return a fh, or undef to 'pass':
536
537 push @INC, sub {
538 my ($coderef, $filename) = @_; # $coderef is \&my_sub
539 open my $fh, "wget ftp://example.com/$filename |";
540 return $fh; # using remote modules, indeed!
541 };
542
543 • Perl 5.8 let you open a file handle to a string, so we just use
544 that:
545
546 open my $fh, '<', \($zip->memberNamed($filename)->contents);
547 return $fh;
548
549 • But Perl 5.6 does not have that, and I don't want to use temp
550 files...
551
552 Source Filtering without Filter::* Modules
553 • ... Undocumented features to the rescue!
554
555 • It turns out that @INC hooks can return two values
556
557 • The first is still the file handle
558
559 • The second is a code reference for line-by-line source
560 filtering!
561
562 • This is how "Acme::use::strict::with::pride" works:
563
564 # Force all modules used to use strict and warnings
565 open my $fh, "<", $filename or return;
566 my @lines = ("use strict; use warnings;\n", "#line 1 \"$full\"\n");
567 return ($fh, sub {
568 return 0 unless @lines;
569 push @lines, $_; $_ = shift @lines; return length $_;
570 });
571
572 Source Filtering without Filter::* Modules (cont.)
573 • But we don't really have a filehandle for anything
574
575 • Another undocumented feature saves the day!
576
577 • We can actually omit the first return value altogether:
578
579 # Return all contents line-by-line from the file inside PAR
580 my @lines = split(
581 /(?<=\n)/,
582 $zip->memberNamed($filename)->contents
583 );
584 return (sub {
585 $_ = shift(@lines);
586 return length $_;
587 });
588
589 Overriding DynaLoader::bootstrap
590 • XS modules have dynamically loaded libraries
591
592 • They cannot be loaded as part of a zip file, so we extract them
593 out
594
595 • Must intercept DynaLoader's library-finding process
596
597 • Module names are passed to "bootstrap" for XS loading
598
599 • During the process, it calls "dl_findfile" to locate the file
600
601 • So we install pre-hooks around both functions
602
603 • Our "_bootstrap" just checks if the library is in PARs
604
605 • If yes, extract it to a "File::Temp" temp file
606
607 • The file will be automatically cleaned up when the program
608 ends
609
610 • It then pass the arguments to the original "bootstrap"
611
612 • Finally, our "dl_findfile" intercepts known filenames and
613 return it
614
615 Anatomy of a Self-Contained PAR executable
616 • The par script ($0) itself
617
618 • May be in plain-text or native executable format
619
620 • Any number of embedded files
621
622 • Typically used to bootstrap PAR's various dependencies
623
624 • Each section begins with the magic string "FILE"
625
626 • Length of filename in pack('N') format and the filename
627 (auto/.../)
628
629 • File length in pack('N') and the file's content (not
630 compressed)
631
632 • One PAR file
633
634 • Just a regular zip file with the magic string "PK\003\004"
635
636 • Ending section
637
638 • A pack('N') number of the total length of FILE and PAR sections
639
640 • Finally, there must be a 8-bytes magic string: "\012PAR.pm\012"
641
642 Self-Bootstrapping Tricks
643 • All we can expect is a working perl interpreter
644
645 • The self-contained script *must not* use any modules at all
646
647 • But to process PAR files, we need XS modules like
648 Compress::Zlib
649
650 • Answer: bundle all modules + libraries used by PAR.pm
651
652 • That's what the "FILE" section in the previous slide is for
653
654 • Load modules to memory, and write object files to disk
655
656 • Then use a local @INC hook to load them on demand
657
658 • Minimizing the amount of temporary files
659
660 • First, try to load PerlIO::scalar and File::Temp
661
662 • Set up an END hook to unlink all temp files up to this point
663
664 • Load other bundled files, and look in the compressed PAR
665 section
666
667 • This can be much easier with a pure-perl "inflate()"; patches
668 welcome!
669
670 Thank you (again)!
671 • Any questions, please?
672
674 PAR, pp, par.pl, parl
675
676 ex::lib::zip, Acme::use::strict::with::pride
677
678 App::Packer, Apache::PAR, CPANPLUS, Module::Install
679
681 Audrey Tang <cpan@audreyt.org>
682
683 You can write to the mailing list at <par@perl.org>, or send an empty
684 mail to <par-subscribe@perl.org> to participate in the discussion.
685
686 Please submit bug reports to <bug-par@rt.cpan.org>.
687
689 Copyright 2003, 2004, 2005, 2006 by Audrey Tang <cpan@audreyt.org>.
690
691 This document is free documentation; you can redistribute it and/or
692 modify it under the same terms as Perl itself.
693
694 See LICENSE.
695
696
697
698perl v5.36.0 2022-09-29 PAR::Tutorial(3)