1Test::AutoBuild(3) User Contributed Perl Documentation Test::AutoBuild(3)
2
6 Test::AutoBuild - Automated build engine controller
7
9 use Test::AutoBuild;
10 use Config::Record;
11 my $config = new Config::Record (file => $filename);
13 my $builder = new Test::AutoBuild (config => $config);
14 my ($status, $log) = $builder->run($timestamp);
16 if ($status) {
18 print STDERR $log, "\n";
19 }
20 exit $status;
22
24 This module provides the build controller, tieing together various sub‐
25 systems to form an integrated engine. It is wholely reponsible for
26 loading the various runtime objects (stages, modules, repositories,
27 package types, monitors, publishers) based on their definitions in the
28 configuration file and then invoking the build. This object does not,
29 however, contain any logic pertaining to how the build is run, since
30 this is all delegated to the stages defined in the configuration file.
31
33 After installing the modules, the first setup step is to create an
34 unprivileged user to run the build as. By convention the user is called
35 'builder', in a similarly named group and a home directory of
36 /var/lib/builder. So as root, execute the following commands:
37 $ groupadd builder
39 $ useradd -g builder -m -d /var/lib/builder builder
40 NB, with the combined contents of the source checkout, the cache and
42 the virtual installed root, and HTTP site, the disk space requirements
43 can be pretty large for any non-trivial software. Based on the appli‐
44 cations being built, anywhere between 100MB and many GB of disk space
45 make be neccessary. For Linux, making /var/lib/builder a dedicated par‐
46 tition with LVM (Logical Volume Manager) will enable additional space
47 to be easily grafted on without requiring a re-build.
48 The next step is to create the basic directory structure within the
50 user's home directory for storing the various files. There are directo‐
51 ries required for storing the source code, a virtual root directory for
52 installing files, a build archive, package spool directories, and pub‐
53 lishing directories for HTTP and FTP servers. To facilitate quick set‐
54 up, a script is provided to create all the required directories. Run
55 this script as the unprivileged user
56 $ su - builder
58 $ auto-build-make-root /var/lib/builder
59 It will display a list of all the directories it creates, but for
61 advance reference, they are
62 /var/lib/builder/install-root
64 /var/lib/builder/source-root
65 /var/lib/builder/log-root
66 /var/lib/builder/build-archive
67 /var/lib/builder/package-root
69 /var/lib/builder/package-root/rpm
70 /var/lib/builder/package-root/rpm/BUILD
71 /var/lib/builder/package-root/rpm/RPMS
72 /var/lib/builder/package-root/rpm/RPMS/noarch
73 /var/lib/builder/package-root/rpm/RPMS/i386
74 /var/lib/builder/package-root/rpm/RPMS/i486
75 /var/lib/builder/package-root/rpm/RPMS/i586
76 /var/lib/builder/package-root/rpm/RPMS/i686
77 /var/lib/builder/package-root/rpm/RPMS/x86_64
78 /var/lib/builder/package-root/rpm/RPMS/ia32e
79 /var/lib/builder/package-root/rpm/RPMS/ia64
80 /var/lib/builder/package-root/rpm/RPMS/sparc
81 /var/lib/builder/package-root/rpm/SPECS
82 /var/lib/builder/package-root/rpm/SOURCES
83 /var/lib/builder/package-root/rpm/SRPMS
84 /var/lib/builder/package-root/zips
85 /var/lib/builder/package-root/tars
86 /var/lib/builder/package-root/debian
87
89 The configuration file determines all aspects of operation of the build
90 engine, from the modules built, through the package types detected,
91 archival method, to build workflow stages, and much more. The example
92 build configuration file installed by default should provide a fully
93 functional build instance running under /var/lib/builder, which is
94 capable of building Test-AutoBuild, versions 1.0.x and 1.1.x, along
95 with the AutoBuild-Applet. A good sanity check for correct installa‐
96 tion, is to ensure that the example build configuration succeeds when
97 run.
98 The configuration file is split into a number of logical groups, which
100 will be considered in turn below. The minimal level of configuration to
101 get started involves editing the list of modules, along with the source
102 repository definitions.
103 General runtime
105 The following options define miscellaneous aspects of the build engine
107 runtime environment.
108 root
110 The "root" option is a grouping under which core directories of the
111 build engine are defined.
112 root = {
114 ... nested options...
115 }
116 The following nested options are permitted with the "root" option
118 source
120 The location into which modules' source code will be checked
121 out from version control. If not specified this option defaults
122 to the location $HOME/source-root
123 root = {
125 ...
126 source = /var/lib/builder/source-root
127 ...
128 }
129 Thus, a module with a name of 'dbus-dev' would be checked out
131 into the directory
132 /var/lib/builder/source-root/dbus-dev
134 install
136 The location into which a module's build process would install
137 files to be used by dependant modules later in the build cycle.
138 This location is made available to a module's build control
139 file via the environment variable $AUTOBUILD_INSTALL_ROOT. If
140 not specified this option defaults to the location
141 $HOME/install-root
142 root = {
144 ...
145 install = /var/lib/builder/install-root
146 ...
147 }
148 Consider, for example, a module 'mozilla' which depends on a
150 library 'openssl'. The 'openssl' module would be listed as a
151 dependant module so that it is built first. The build of
152 'openssl' would install itself into the install root, perhaps
153 by passing the 'prefix' argument to a configure script:
154 ./configure --prefix=$AUTOBUILD_INSTALL_ROOT
156 The later build of mozilla, would then build against this ver‐
158 sion of openssl, by using
159 ./configure --with-openssl=$AUTOBUILD_INSTALL_ROOT
161 package
163 The location in which a module's build process will create any
164 binary packages it generates, for example RPMs, or Debian pack‐
165 ages. The packages are typically placed into a package type
166 specific sub-directory. This location is made available to a
167 module's build control file via the environment variable $AUTO‐
168 BUILD_PACKAGE_ROOT. If not specified, this option defaults to
169 the location $HOME/package-root
170 root = {
172 ...
173 package = /var/lib/builder/package-root
174 ...
175 }
176 Consider, for example, a module which generates an RPM, of
178 itself. The $AUTOBUILD_PACKAGE_ROOT directory would be used to
179 set the '_topdir' macro for the RPM build process
180 rpmbuild --define '_topdir $AUTOBUILD_PACKAGE_ROOT/rpm' -ta foo.tar.gz
182 log The location in which the output from a module's build control
184 file will be spooled during execution. If not specified, this
185 option defaults to the location $HOME/log-root. The control
186 file's standard output and error streams will be combined into
187 one.
188 root = {
190 ...
191 log = /var/lib/builder/log-root
192 ...
193 }
194 admin-email
196 The email address of the build engine administrator, typically
197 linked from the bottom of the HTML status pages. This is also the
198 address spammed with build status alerts if the Test::Auto‐
199 Build::Stage::EmailAlert module is in use.
200 admin-email = admin@example.com
202 admin-name
204 The full name of the build engine administrator, typically dis‐
205 played on the bottom of the HTML status pages.
206 admin-name = John Doe
208 log4perl
210 A configuration block controlling the output of debug information
211 during execution of the build. The data here is passed straight
212 through to the "init" method in the Log::Log4perl module, so con‐
213 sult that module's manual page for possible configuration options.
214 The example setting, enables display of progress through the build
215 workflow. To get maximum possible debug information, change the
216 "log4perl.rootLogger" option to 'DEBUG' instead of 'WARN'.
217 log4perl = {
219 log4perl.rootLogger = WARN, Screen
220 # To get progress updates
222 log4perl.logger.Test.AutoBuild.Monitor.Log4perl = INFO
223 log4perl.appender.Screen = Log::Log4perl::Appender::Screen
225 log4perl.appender.Screen.stderr = 1
226 log4perl.appender.Screen.layout = Log::Log4perl::Layout::SimpleLayout
227 }
228 counter
230 This configuration block determines the module used to generate the
231 unique build cycle counter.
232 counter = {
234 ..nested options..
235 }
236 The nested options allowed within this block are
238 module
240 The full package name of the subclass of Test::Auto‐
241 Build::Counter used to generate the build cycle counter. Con‐
242 sult that module for a list of known implementations. The
243 exmaple configuration sets the build counter to match the time‐
244 stamp taken at the start of the build cycle.
245 module = Test::AutoBuild::Counter::Timestamp
247 options
249 This is defines a set of options as key, value pairs which are
250 passed to the counter object created. The valid keys for this
251 are specific to the package specified in the "module" parameter
252 above, so consult the manual page corresponding to the module
253 defined there. If using the "Test::AutoBuild::Counter::Change‐
254 List" class, and there is a source repository named, "mysvn",
255 one would set an option such as
256 options = {
258 repository = mysvn
259 }
260 Source repositories
262 The "repositories" configuration block, defines the source repositories
264 from which modules are checked out. The keys in the block form the
265 short name of the repository, and is referenced later when defining the
266 paths for modules' source checkout
267 repositories = {
269 myrepo = {
270 ... definition of myrepo ...
271 }
272 otherrepo = {
273 ... definition of otherrepo ...
274 }
275 }
276 Within each repository definition the following options are supported
278 label
280 The free-text display name of the repository, typically used in the
281 HTML status pages.
282 module
284 The full package name of a subclass of "Test::AutoBuild::Reposi‐
285 tory" which implements the checkout procedure for the repository
286 type. There are repository types for common version control systems
287 such as Subversion, CVS, Perforce and GNUArch, as well as a simple
288 non-versioned repository. Refer to the Test::AutoBuild::Repository
289 module for a list of known repository types and their corresponding
290 package names.
291 module = Test::AutoBuild::Repository::Subversion
293 env Lists a set of environment variables which will be set whenever
295 running any repository commands. The possible environment variable
296 names vary according to the type of repository, so refer to the
297 manual page for the repository module defined in the "module"
298 option. For example, the CVS repository type uses the CVSROOT envi‐
299 ronment variable to specify the repository location.
300 env = {
302 CVSROOT = :pserver:nonymous@cvs.gna.org:/cvs/testautobuild
303 }
304 option
306 A set of configuration options specific to the type of repository
307 configure. Again, refer to the manual page for the repository mod‐
308 ule defined in the "module" option. For example, the GNU Arch
309 repository type supports the 'archive-name' option
310 options = {
312 archive-name = lord@emf.net--2004
313 }
314 Modules
316 The "modules" configuration block defines the list of modules to be
318 checked out of source control and built. The keys in the block form the
319 short names for the modules, used, for example, in creating filenames
320 for assets relating to the module, and the name of the checkout direc‐
321 tory under the source root. If building multiple branches of a module,
322 it is common to post-fix the module name with a version / branch name.
323 modules = {
325 mymod-1.0 = {
326 .. definition of mymod version 1.0..
327 }
328 mymod-dev = {
329 .. definition of mymod development snapshot...
330 }
331 }
332 Within the configuration block of an individual module the following
334 options are permitted
335 label
337 The free-text display name for the module, typically used in HTML
338 status pages, and email alerts.
339 label = Test-AutoBuild (Development branch)
341 source
343 This block defines the repository containing the source to be
344 checked out for the module. There are two keys in the block, the
345 value associated with the key "repository", is the name of a repos‐
346 itory previously defined in the config file. The value associated
347 with the key "path" is the path to checkout within the repository.
348 The syntax for path values is dependant on the type of repository
349 being accessed. For details refer to the manual pages for the cor‐
350 responding modules:
351 CVS Refer to Test::AutoBuild::Repository::CVS
353 GNU Arch
355 Refer to Test::AutoBuild::Repository::GNUArch
356 Subversion
358 Refer to Test::AutoBuild::Repository::Subversion
359 Perforce
361 Refer to Test::AutoBuild::Repository::Perforce
362 Mercurial
364 Refer to Test::AutoBuild::Repository::Mercurial
365 SVK Refer to Test::AutoBuild::Repository::SVK
367 Local disk
369 Refer to Test::AutoBuild::Repository::Disk
370 An example config entry for a module checked out of CVS is
372 source = {
374 repository = gna-cvs
375 path = /testautobuild
376 }
377 sources
379 If a module's source is split amongst several locations, this block
380 is used instead of the "source" block. It allows defintion of a
381 list of source paths to checkout. It is a list, where each entry
382 matches the format of the "source" parameter. For example
383 sources = (
385 {
386 repository = gna-cvs
387 path = /testautobuild
388 }
389 {
390 repository = local-disk
391 path = /testautobuild-autobuild.sh -> autobuild.sh
392 }
393 )
394 NB, not all repository types play nicely together when checking out
396 from multiple paths. Consult manual pages for individual repository
397 types for futher information
398 groups
400 Lists the groups to which the module belongs. The values in the
401 list must be group names, specified earlier in the top level
402 "groups" configuration block.
403 groups = (
405 software
406 perl
407 )
408 env Defines a set of environment variables which will be set whenever
410 running the build/test control files for the module. The only
411 restriction on variables set here, are that none should be named
412 with the prefix AUTOBUILD_, otherwise they are liable to be over‐
413 ridden by variables set by the build engine.
414 env = (
416 SKIP_TESTS = 1
417 )
418 options
420 The options parameter is used to specify module specific data which
421 will be used by stages in the workflow engine. Consult the manual
422 pages for individual stages in use, for further details on which
423 options are possible. The most common option is "control-file"
424 which can be used to override the default name of the command to
425 invoke to run the build. For compatability with version 1.0.x of
426 autobuild, this should be set to 'rollingbuild.sh'
427 options = {
429 control-file = rollingbuild.sh
430 }
431 links
433 The links configuration block defines a simple list of hyperlinks
434 relating to the module. This is typically used to provide links to
435 a graphical front end for the source repository, or a link to a
436 project homepage. The two keys with the block are "href" and
437 "label"
438 links = (
440 {
441 href = http://www.autobuild.org/
442 label = Project homepage
443 }
444 {
445 href = http://cvs.gna.org/viewcvs/testautobuild/testautobuild/?only_with_tag=RELEASE_1_0_0
446 label = Browse source code
447 }
448 )
449 artifacts
451 The artifacts configuration block defines a list of build artifacts
452 which will be published to the distribution sites. This is typi‐
453 cally used to provide access to items such as build reports on code
454 coverage, code complexity, bug analysis, etc, or metadata files
455 such as the module's README, or copyright notices. With the block,
456 the "src" parameter is a filename glob relative to the base of the
457 module's code checkout; the "dst" parameter is the name of the des‐
458 tination file (or directory if the source glob matches multiple
459 files), and will also form the URL string; the "label" key gives a
460 label for hyperlinks to the artifact, and finally the "publisher"
461 is the name of a file publisher, as defined in the top level "pub‐
462 lishers" config block.
463 artifacts = (
465 {
466 src = README
467 dst = README
468 label = Readme
469 publisher = copy
470 }
471 {
472 src = blib/coverage/*
473 dst = coverage/
474 label = Code Test & POD coverage Reports
475 publisher = copy
476 }
477 )
478 Groups
480 The following options define grouping of modules, primarily used for
482 grouping modules in the HTML status display. The keys in the configura‐
483 tion block for the short group name, used when defining group member‐
484 ship in the module configuration.
485 groups = {
487 perl = {
488 label = Perl modules
489 }
490 software = {
491 label = Software
492 }
493 docs = {
494 label = Documentation
495 }
496 }
497 The following options are allowed within each group definition
499 label
501 The free-text display name of the group
502 Package types
504 The following options define binary package types to detect and pub‐
506 lish.
507 Publishers
509 The following options define mechanisms for publishing files to distri‐
511 bution directories.
512 Platforms
514 The following options define aspects of the host platform
516 Build archive
518 The following options define the mechanism used for archiving module
520 build output between build cycles.
521 Workflow stages
523 The following options defined the workflow followed for a build cycle
525
527 $builder = Test::AutoBuild->new(config => $config);
528 Creates a new builder instance, loading configuration parameters
529 from the $config parameter, whose value is an instance of the "Con‐
530 fig::Record" module.
531 $config = $builder->config([$name, [$default]]);
533 If invoked with no arguments returns the Config::Record object
534 storing the builder configuration. If invoked with a single argu‐
535 ment, returns the configuration value with the matching name. An
536 optional default value can be provided in the second argument
537 $builder->run();
539 Executes the build process. This is the heart of the auto build
540 engine. It performs the following actions:
541 * Reads the list of modules, source control repositories,
543 package types and output modules from the configuration
544 file
545 * Initializes the build cache
546 * Takes out an exclusive file lock to prevent > 1 builder
547 running at the same time.
548 * Changes the (nice) priority of the AutoBuild process
549 * Checks the code for each module out of its respective
550 source control repository.
551 * Does a topological sort to determine the build order
552 for all modules
553 * For each module to be built:
554 - Take a snapshot of the package & virtual root install
555 directories
556 - Change to the top level source directory of the module
557 - Run the rollingbuild.sh script
558 - Take another snapshot & compare to determine which
559 files were install in the virtual root & which packages
560 were generated
561 - Save the intsalled files and packages in the cache.
562 * Invoke each requested output module, for example, HTML
563 status generator, package & log file copiers, email
564 alerts
565 my \%groups = $autobuild->load_groups();
567 Creates the "Test::AutoBuild::Publisher" objects for each publisher
568 defined in the build configuration.
569 my \%publishers = $autobuild->load_publishers();
571 Creates the "Test::AutoBuild::Publisher" objects for each publisher
572 defined in the build configuration.
573 my \%monitors = $autobuild->load_monitors();
575 Creates the "Test::AutoBuild::Monitor" objects for each monitor
576 defined in the build configuration.
577 my \%platforms = $autobuild->load_platforms();
579 Creates the "Test::AutoBuild::Platform" objects for each platform
580 defined in the build configuration.
581 my \@repositories = $autobuild->load_repositories();
583 Creates the "Test::AutoBuild::Repository" objects for each reposi‐
584 tory defined in the build configuration.
585 my \%package_types = $autobuild->load_package_types();
587 Creates the "Test::AutoBuild::PackageType" objects for each package
588 type defined in the build configuration.
589 my \%modules = $autobuild->load_modules();
591 Creates the "Test::AutoBuild::Module" obkjects for each module
592 defined in the build configuration.
593
595 The task tracker on the Gna! project site (www.autobuild.org) contains
596 a list of all things we'd like to do.
597 Oh and 100% Pod & code test coverage - Devel::Cover WILL EAT YOUR
599 BRAAAAANE!
600
602 Probably a few, so report them to the bug tracker linked from the Gna!
603 project site www.autobuild.org.
604
606 Daniel P. Berrange, Dennis Gregorovic
607
609 Copyright (C) 2002-2005 Daniel Berrange <dan@berrange.com>
610
612 perl(1), <http://www.autobuild.org>, Test::AutoBuild::Runtime,
613 Test::AutoBuild::Module, Test::AutoBuild::Stage, Test::Auto‐
614 Build::Repository, Test::AutoBuild::PackageType, Test::AutoBuild::Pub‐
615 lisher, Test::AutoBuild::Repository, Test::AutoBuild::Counter,
616 Test::AutoBuild::Group,
617perl v5.8.8 2007-12-09 Test::AutoBuild(3)