1Test::AutoBuild(3) User Contributed Perl Documentation Test::AutoBuild(3)
2
6 Test::AutoBuild - Automated build engine controller
7
9 use Test::AutoBuild;
10 use Config::Record;
11 my $config = new Config::Record (file => $filename);
13 my $builder = new Test::AutoBuild (config => $config);
14 my ($status, $log) = $builder->run($timestamp);
16 if ($status) {
18 print STDERR $log, "\n";
19 }
20 exit $status;
22
24 This module provides the build controller, tieing together various
25 subsystems to form an integrated engine. It is wholely reponsible for
26 loading the various runtime objects (stages, modules, repositories,
27 package types, monitors, publishers) based on their definitions in the
28 configuration file and then invoking the build. This object does not,
29 however, contain any logic pertaining to how the build is run, since
30 this is all delegated to the stages defined in the configuration file.
31
33 After installing the modules, the first setup step is to create an
34 unprivileged user to run the build as. By convention the user is called
35 'builder', in a similarly named group and a home directory of
36 /var/lib/builder. So as root, execute the following commands:
37 $ groupadd builder
39 $ useradd -g builder -m -d /var/lib/builder builder
40 NB, with the combined contents of the source checkout, the cache and
42 the virtual installed root, and HTTP site, the disk space requirements
43 can be pretty large for any non-trivial software. Based on the
44 applications being built, anywhere between 100MB and many GB of disk
45 space make be neccessary. For Linux, making /var/lib/builder a
46 dedicated partition with LVM (Logical Volume Manager) will enable
47 additional space to be easily grafted on without requiring a re-build.
48 The next step is to create the basic directory structure within the
50 user's home directory for storing the various files. There are
51 directories required for storing the source code, a virtual root
52 directory for installing files, a build archive, package spool
53 directories, and publishing directories for HTTP and FTP servers. To
54 facilitate quick setup, a script is provided to create all the required
55 directories. Run this script as the unprivileged user
56 $ su - builder
58 $ auto-build-make-root /var/lib/builder
59 It will display a list of all the directories it creates, but for
61 advance reference, they are
62 /var/lib/builder/install-root
64 /var/lib/builder/source-root
65 /var/lib/builder/log-root
66 /var/lib/builder/build-archive
67 /var/lib/builder/package-root
69 /var/lib/builder/package-root/rpm
70 /var/lib/builder/package-root/rpm/BUILD
71 /var/lib/builder/package-root/rpm/RPMS
72 /var/lib/builder/package-root/rpm/RPMS/noarch
73 /var/lib/builder/package-root/rpm/RPMS/i386
74 /var/lib/builder/package-root/rpm/RPMS/i486
75 /var/lib/builder/package-root/rpm/RPMS/i586
76 /var/lib/builder/package-root/rpm/RPMS/i686
77 /var/lib/builder/package-root/rpm/RPMS/x86_64
78 /var/lib/builder/package-root/rpm/RPMS/ia32e
79 /var/lib/builder/package-root/rpm/RPMS/ia64
80 /var/lib/builder/package-root/rpm/RPMS/sparc
81 /var/lib/builder/package-root/rpm/SPECS
82 /var/lib/builder/package-root/rpm/SOURCES
83 /var/lib/builder/package-root/rpm/SRPMS
84 /var/lib/builder/package-root/zips
85 /var/lib/builder/package-root/tars
86 /var/lib/builder/package-root/debian
87
89 The configuration file determines all aspects of operation of the build
90 engine, from the modules built, through the package types detected,
91 archival method, to build workflow stages, and much more. The example
92 build configuration file installed by default should provide a fully
93 functional build instance running under /var/lib/builder, which is
94 capable of building Test-AutoBuild, versions 1.0.x and 1.1.x, along
95 with the AutoBuild-Applet. A good sanity check for correct
96 installation, is to ensure that the example build configuration
97 succeeds when run.
98 The configuration file is split into a number of logical groups, which
100 will be considered in turn below. The minimal level of configuration to
101 get started involves editing the list of modules, along with the source
102 repository definitions.
103 General runtime
105 The following options define miscellaneous aspects of the build engine
106 runtime environment.
107 root
109 The "root" option is a grouping under which core directories of the
110 build engine are defined.
111 root = {
113 ... nested options...
114 }
115 The following nested options are permitted with the "root" option
117 source
119 The location into which modules' source code will be checked
120 out from version control. If not specified this option defaults
121 to the location $HOME/source-root
122 root = {
124 ...
125 source = /var/lib/builder/source-root
126 ...
127 }
128 Thus, a module with a name of 'dbus-dev' would be checked out
130 into the directory
131 /var/lib/builder/source-root/dbus-dev
133 install
135 The location into which a module's build process would install
136 files to be used by dependant modules later in the build cycle.
137 This location is made available to a module's build control
138 file via the environment variable $AUTOBUILD_INSTALL_ROOT. If
139 not specified this option defaults to the location
140 $HOME/install-root
141 root = {
143 ...
144 install = /var/lib/builder/install-root
145 ...
146 }
147 Consider, for example, a module 'mozilla' which depends on a
149 library 'openssl'. The 'openssl' module would be listed as a
150 dependant module so that it is built first. The build of
151 'openssl' would install itself into the install root, perhaps
152 by passing the 'prefix' argument to a configure script:
153 ./configure --prefix=$AUTOBUILD_INSTALL_ROOT
155 The later build of mozilla, would then build against this
157 version of openssl, by using
158 ./configure --with-openssl=$AUTOBUILD_INSTALL_ROOT
160 package
162 The location in which a module's build process will create any
163 binary packages it generates, for example RPMs, or Debian
164 packages. The packages are typically placed into a package
165 type specific sub-directory. This location is made available to
166 a module's build control file via the environment variable
167 $AUTOBUILD_PACKAGE_ROOT. If not specified, this option
168 defaults to the location $HOME/package-root
169 root = {
171 ...
172 package = /var/lib/builder/package-root
173 ...
174 }
175 Consider, for example, a module which generates an RPM, of
177 itself. The $AUTOBUILD_PACKAGE_ROOT directory would be used to
178 set the '_topdir' macro for the RPM build process
179 rpmbuild --define '_topdir $AUTOBUILD_PACKAGE_ROOT/rpm' -ta foo.tar.gz
181 log The location in which the output from a module's build control
183 file will be spooled during execution. If not specified, this
184 option defaults to the location $HOME/log-root. The control
185 file's standard output and error streams will be combined into
186 one.
187 root = {
189 ...
190 log = /var/lib/builder/log-root
191 ...
192 }
193 admin-email
195 The email address of the build engine administrator, typically
196 linked from the bottom of the HTML status pages. This is also the
197 address spammed with build status alerts if the
198 Test::AutoBuild::Stage::EmailAlert module is in use.
199 admin-email = admin@example.com
201 admin-name
203 The full name of the build engine administrator, typically
204 displayed on the bottom of the HTML status pages.
205 admin-name = John Doe
207 log4perl
209 A configuration block controlling the output of debug information
210 during execution of the build. The data here is passed straight
211 through to the "init" method in the Log::Log4perl module, so
212 consult that module's manual page for possible configuration
213 options. The example setting, enables display of progress through
214 the build workflow. To get maximum possible debug information,
215 change the "log4perl.rootLogger" option to 'DEBUG' instead of
216 'WARN'.
217 log4perl = {
219 log4perl.rootLogger = WARN, Screen
220 # To get progress updates
222 log4perl.logger.Test.AutoBuild.Monitor.Log4perl = INFO
223 log4perl.appender.Screen = Log::Log4perl::Appender::Screen
225 log4perl.appender.Screen.stderr = 1
226 log4perl.appender.Screen.layout = Log::Log4perl::Layout::SimpleLayout
227 }
228 counter
230 This configuration block determines the module used to generate the
231 unique build cycle counter.
232 counter = {
234 ..nested options..
235 }
236 The nested options allowed within this block are
238 module
240 The full package name of the subclass of
241 Test::AutoBuild::Counter used to generate the build cycle
242 counter. Consult that module for a list of known
243 implementations. The exmaple configuration sets the build
244 counter to match the timestamp taken at the start of the build
245 cycle.
246 module = Test::AutoBuild::Counter::Timestamp
248 options
250 This is defines a set of options as key, value pairs which are
251 passed to the counter object created. The valid keys for this
252 are specific to the package specified in the "module" parameter
253 above, so consult the manual page corresponding to the module
254 defined there. If using the
255 "Test::AutoBuild::Counter::ChangeList" class, and there is a
256 source repository named, "mysvn", one would set an option such
257 as
258 options = {
260 repository = mysvn
261 }
262 Source repositories
264 The "repositories" configuration block, defines the source repositories
265 from which modules are checked out. The keys in the block form the
266 short name of the repository, and is referenced later when defining the
267 paths for modules' source checkout
268 repositories = {
270 myrepo = {
271 ... definition of myrepo ...
272 }
273 otherrepo = {
274 ... definition of otherrepo ...
275 }
276 }
277 Within each repository definition the following options are supported
279 label
281 The free-text display name of the repository, typically used in the
282 HTML status pages.
283 module
285 The full package name of a subclass of
286 "Test::AutoBuild::Repository" which implements the checkout
287 procedure for the repository type. There are repository types for
288 common version control systems such as Subversion, CVS, Perforce
289 and GNUArch, as well as a simple non-versioned repository. Refer
290 to the Test::AutoBuild::Repository module for a list of known
291 repository types and their corresponding package names.
292 module = Test::AutoBuild::Repository::Subversion
294 env Lists a set of environment variables which will be set whenever
296 running any repository commands. The possible environment variable
297 names vary according to the type of repository, so refer to the
298 manual page for the repository module defined in the "module"
299 option. For example, the CVS repository type uses the CVSROOT
300 environment variable to specify the repository location.
301 env = {
303 CVSROOT = :pserver:nonymous@cvs.gna.org:/cvs/testautobuild
304 }
305 option
307 A set of configuration options specific to the type of repository
308 configure. Again, refer to the manual page for the repository
309 module defined in the "module" option. For example, the GNU Arch
310 repository type supports the 'archive-name' option
311 options = {
313 archive-name = lord@emf.net--2004
314 }
315 Modules
317 The "modules" configuration block defines the list of modules to be
318 checked out of source control and built. The keys in the block form the
319 short names for the modules, used, for example, in creating filenames
320 for assets relating to the module, and the name of the checkout
321 directory under the source root. If building multiple branches of a
322 module, it is common to post-fix the module name with a version /
323 branch name.
324 modules = {
326 mymod-1.0 = {
327 .. definition of mymod version 1.0..
328 }
329 mymod-dev = {
330 .. definition of mymod development snapshot...
331 }
332 }
333 Within the configuration block of an individual module the following
335 options are permitted
336 label
338 The free-text display name for the module, typically used in HTML
339 status pages, and email alerts.
340 label = Test-AutoBuild (Development branch)
342 source
344 This block defines the repository containing the source to be
345 checked out for the module. There are two keys in the block, the
346 value associated with the key "repository", is the name of a
347 repository previously defined in the config file. The value
348 associated with the key "path" is the path to checkout within the
349 repository. The syntax for path values is dependant on the type of
350 repository being accessed. For details refer to the manual pages
351 for the corresponding modules:
352 CVS Refer to Test::AutoBuild::Repository::CVS
354 GNU Arch
356 Refer to Test::AutoBuild::Repository::GNUArch
357 Subversion
359 Refer to Test::AutoBuild::Repository::Subversion
360 Perforce
362 Refer to Test::AutoBuild::Repository::Perforce
363 Mercurial
365 Refer to Test::AutoBuild::Repository::Mercurial
366 SVK Refer to Test::AutoBuild::Repository::SVK
368 Local disk
370 Refer to Test::AutoBuild::Repository::Disk
371 An example config entry for a module checked out of CVS is
373 source = {
375 repository = gna-cvs
376 path = /testautobuild
377 }
378 sources
380 If a module's source is split amongst several locations, this block
381 is used instead of the "source" block. It allows defintion of a
382 list of source paths to checkout. It is a list, where each entry
383 matches the format of the "source" parameter. For example
384 sources = (
386 {
387 repository = gna-cvs
388 path = /testautobuild
389 }
390 {
391 repository = local-disk
392 path = /testautobuild-autobuild.sh -> autobuild.sh
393 }
394 )
395 NB, not all repository types play nicely together when checking out
397 from multiple paths. Consult manual pages for individual repository
398 types for futher information
399 groups
401 Lists the groups to which the module belongs. The values in the
402 list must be group names, specified earlier in the top level
403 "groups" configuration block.
404 groups = (
406 software
407 perl
408 )
409 env Defines a set of environment variables which will be set whenever
411 running the build/test control files for the module. The only
412 restriction on variables set here, are that none should be named
413 with the prefix AUTOBUILD_, otherwise they are liable to be
414 overridden by variables set by the build engine.
415 env = (
417 SKIP_TESTS = 1
418 )
419 options
421 The options parameter is used to specify module specific data which
422 will be used by stages in the workflow engine. Consult the manual
423 pages for individual stages in use, for further details on which
424 options are possible. The most common option is "control-file"
425 which can be used to override the default name of the command to
426 invoke to run the build. For compatability with version 1.0.x of
427 autobuild, this should be set to 'rollingbuild.sh'
428 options = {
430 control-file = rollingbuild.sh
431 }
432 links
434 The links configuration block defines a simple list of hyperlinks
435 relating to the module. This is typically used to provide links to
436 a graphical front end for the source repository, or a link to a
437 project homepage. The two keys with the block are "href" and
438 "label"
439 links = (
441 {
442 href = http://www.autobuild.org/
443 label = Project homepage
444 }
445 {
446 href = http://cvs.gna.org/viewcvs/testautobuild/testautobuild/?only_with_tag=RELEASE_1_0_0
447 label = Browse source code
448 }
449 )
450 artifacts
452 The artifacts configuration block defines a list of build artifacts
453 which will be published to the distribution sites. This is
454 typically used to provide access to items such as build reports on
455 code coverage, code complexity, bug analysis, etc, or metadata
456 files such as the module's README, or copyright notices. With the
457 block, the "src" parameter is a filename glob relative to the base
458 of the module's code checkout; the "dst" parameter is the name of
459 the destination file (or directory if the source glob matches
460 multiple files), and will also form the URL string; the "label" key
461 gives a label for hyperlinks to the artifact, and finally the
462 "publisher" is the name of a file publisher, as defined in the top
463 level "publishers" config block.
464 artifacts = (
466 {
467 src = README
468 dst = README
469 label = Readme
470 publisher = copy
471 }
472 {
473 src = blib/coverage/*
474 dst = coverage/
475 label = Code Test & POD coverage Reports
476 publisher = copy
477 }
478 )
479 Groups
481 The following options define grouping of modules, primarily used for
482 grouping modules in the HTML status display. The keys in the
483 configuration block for the short group name, used when defining group
484 membership in the module configuration.
485 groups = {
487 perl = {
488 label = Perl modules
489 }
490 software = {
491 label = Software
492 }
493 docs = {
494 label = Documentation
495 }
496 }
497 The following options are allowed within each group definition
499 label
501 The free-text display name of the group
502 Package types
504 The following options define binary package types to detect and
505 publish.
506 Publishers
508 The following options define mechanisms for publishing files to
509 distribution directories.
510 Platforms
512 The following options define aspects of the host platform
513 Build archive
515 The following options define the mechanism used for archiving module
516 build output between build cycles.
517 Workflow stages
519 The following options defined the workflow followed for a build cycle
520
522 $builder = Test::AutoBuild->new(config => $config);
523 Creates a new builder instance, loading configuration parameters
524 from the $config parameter, whose value is an instance of the
525 "Config::Record" module.
526 $config = $builder->config([$name, [$default]]);
528 If invoked with no arguments returns the Config::Record object
529 storing the builder configuration. If invoked with a single
530 argument, returns the configuration value with the matching name.
531 An optional default value can be provided in the second argument
532 $builder->run();
534 Executes the build process. This is the heart of the auto build
535 engine. It performs the following actions:
536 * Reads the list of modules, source control repositories,
538 package types and output modules from the configuration
539 file
540 * Initializes the build cache
541 * Takes out an exclusive file lock to prevent > 1 builder
542 running at the same time.
543 * Changes the (nice) priority of the AutoBuild process
544 * Checks the code for each module out of its respective
545 source control repository.
546 * Does a topological sort to determine the build order
547 for all modules
548 * For each module to be built:
549 - Take a snapshot of the package & virtual root install
550 directories
551 - Change to the top level source directory of the module
552 - Run the rollingbuild.sh script
553 - Take another snapshot & compare to determine which
554 files were install in the virtual root & which packages
555 were generated
556 - Save the intsalled files and packages in the cache.
557 * Invoke each requested output module, for example, HTML
558 status generator, package & log file copiers, email
559 alerts
560 my \%groups = $autobuild->load_groups();
562 Creates the "Test::AutoBuild::Publisher" objects for each publisher
563 defined in the build configuration.
564 my \%publishers = $autobuild->load_publishers();
566 Creates the "Test::AutoBuild::Publisher" objects for each publisher
567 defined in the build configuration.
568 my \%monitors = $autobuild->load_monitors();
570 Creates the "Test::AutoBuild::Monitor" objects for each monitor
571 defined in the build configuration.
572 my \%platforms = $autobuild->load_platforms();
574 Creates the "Test::AutoBuild::Platform" objects for each platform
575 defined in the build configuration.
576 my \@repositories = $autobuild->load_repositories();
578 Creates the "Test::AutoBuild::Repository" objects for each
579 repository defined in the build configuration.
580 my \%package_types = $autobuild->load_package_types();
582 Creates the "Test::AutoBuild::PackageType" objects for each package
583 type defined in the build configuration.
584 my \%modules = $autobuild->load_modules();
586 Creates the "Test::AutoBuild::Module" obkjects for each module
587 defined in the build configuration.
588
590 The task tracker on the Gna! project site (www.autobuild.org) contains
591 a list of all things we'd like to do.
592 Oh and 100% Pod & code test coverage - Devel::Cover WILL EAT YOUR
594 BRAAAAANE!
595
597 Probably a few, so report them to the bug tracker linked from the Gna!
598 project site www.autobuild.org.
599
601 Daniel P. Berrange, Dennis Gregorovic
602
604 Copyright (C) 2002-2005 Daniel Berrange <dan@berrange.com>
605
607 perl(1), <http://www.autobuild.org>, Test::AutoBuild::Runtime,
608 Test::AutoBuild::Module, Test::AutoBuild::Stage,
609 Test::AutoBuild::Repository, Test::AutoBuild::PackageType,
610 Test::AutoBuild::Publisher, Test::AutoBuild::Repository,
611 Test::AutoBuild::Counter, Test::AutoBuild::Group,
612perl v5.12.1 2007-12-17 Test::AutoBuild(3)