1AUTO-BUILD(1) User Contributed Perl Documentation AUTO-BUILD(1)
2
6 auto-build - run automated, unattended software build processes
7
9 # auto-build [OPTION]...
10 Run build using latest code currently in source control
12 # auto-build --config /etc/auto-build.d/auto-build.conf
13 Run build using code committed prior to 12:30 on Dec 12th 2004
15 # auto-build --timestamp '12:30:00 Dec 12th 2004' \
16 --config /etc/auto-build.d/auto-build.conf
17
19 "auto-build" is the command used to execute the Test-AutoBuild software
20 build automation framework. It is intended to execute a single itera‐
21 tion of the build engine and then quit with its exit status providing
22 an indication of the build success / fail status (see the "EXIT STATUS"
23 section later in this manual page). To perform a continous automated
24 build this command would be launched on a frequent basis from cron.
25 Locking will ensure that two instances do not execute concurrently, so
26 it can safely be triggered every 5 minutes.
27
29 The following command line options are valid:
30 --config=FILE
32 Specify an alternative path to the configuration file for the build
33 process. If ommitted, will default to looking for the file
34 $HOME/auto-build.conf. The configuration data is loaded by the Con‐
35 fig::Record module, so refer to that module's manual page for
36 details of permitted syntax. The file will also be pre-processed by
37 the Template module allowing dynamic generation of complex configu‐
38 ration files
39 --log4perl=FILE
41 Specify an alternative file containing configuration settings for
42 the Log::Log4perl logging system. If ommitted, log4perl settings
43 will be loaded from the master autobuild configuration file.
44 --timestamp=DATE
46 Specify the timestamp to use when checking code out of the source
47 control repositories. For repository modules supporting timestamps
48 it is guarenteed that the code checked out will be the latest ver‐
49 sion not newer than this timestamp. If ommitted, the timestamp will
50 default to the current timestamp. The argument can be either an
51 plain integer specifying number of seconds since the epoch, or any
52 of the text representations handled by the Date::Manip module
53 'today'
55 '1st thursday in June 1992'
56 '05/10/93'
57 '12:30 Dec 12th 1880'
58 '8:00pm december tenth'
59 --verbose
61 Increase the verbosity of warning messages and errors. This will
62 ensure that if the builder unexpectedly crashes, a complete stack
63 trace will be dumped along will full arguments.It will also cause
64 the post-processed configuration file to be dumped.
65 --help
67 Display a summary of the command line help options
68
70 The build process provides special handlers for a couple of signals to
71 assist in debugging problems.
72 USR1
74 When receiving this signal the build process will send a stack
75 trace to STDERR, providing an indication of the current task being
76 performed. The build process will continue to run after receiving
77 and handling this signal.
78 USR2
80 When receiving this signal the build process will use Data::Dumper
81 to dump the entire runtime state of the builder to STDERR. The
82 build process will continue to run after receiving and handling
83 this signal
84
86 The exit status codes of the build have the following meanings
87 0 The build process completed succesfully with no failures in any
89 module
90 1 The build process failed on one of more modules, but ran through to
92 completion of all stages
93 2 The build process aborted during processing, and did not complete
95 execution of all stages.
96 3 An invalid/unknown command line option was provided
98 4 The argument to one of the command line options was malformed or
100 invalid.
101
103 The notes which follow assume the software itself has been installed
104 into the /usr hierarchy with all common commands available within the
105 default $PATH. Some minor alterations may be neccessary if this is not
106 the case.
107 HOST SETUP
109 Before using the build engine a handful of things must be configured on
111 the host machine. This shouldn't take more than a few minutes to accom‐
112 plish if the example configurations are followed.
113 BUILD AREA CREATION
115 The first step after software installation is to create an area where
117 builds can be performed. The build engine expects that the build area
118 be populated with a small set of top level directories. To aid in this
119 setup, the command "auto-build-make-root(1)" can be run passing the
120 build area path as a command line parameter. The example configuration
121 file expects that the build area is at "/var/lib/builder", so assuming
122 this is the case, run the command:
123 # auto-build-make-root /var/lib/builder
125 NB, if the software was installed from RPM, this directory may have
127 been created on your behalf.
128 USER ACCOUNT CREATION
130 It is highly recommended that the build engine be run as an unprivi‐
132 leged user. This is primarily intended to protect the host machine
133 against both delibrate and accidental flaws in a module's build script.
134 By convention the user to run the build as will be called "builder". If
135 only a single instance of the build engine is to be run on a host, it
136 is convenient to make this user's home directory would point to the
137 build area root. So to add a user to run the build engine run the com‐
138 mand:
139 # groupadd builder
141 # useradd -d /var/lib/builder -g builder builder
142 # passwd builder
143 WEB STATUS PAGE SETUP
145 The build engine will generate HTML status pages at the end of every
147 run summarizing the status of the build. There are 3 example configura‐
148 tion files suitable for use with Apache located in
149 "/etc/auto-build.d/httpd". The standard "auto-build.conf" file assumes
150 that virtual hosting will be used on the web server, with a suitable
151 Apache config being the file "vhost.conf". Copy it to
152 /etc/httpd/conf.d, and edit it to set the server name of your build
153 host. If virtual hosting is not suitable, then the status pages can be
154 made to appear at a fixed URL "/builder", or as a user directory under
155 "/~builder". The sample config files "aliased.conf" or "user.conf" can
156 be used instead of "vhost.conf", although it will also be neccessary to
157 edit the "auto-build.conf" file to change the "httpPrefix" option.
158 AUTOMATION
160 As mentioned earlier, executing "auto-build" just runs a single itera‐
162 tion of the build engine. To achieve continuous integration, this com‐
163 mand should be scheduled to run on a periodic (frequent) basis. On
164 UNIX, cron is perfectly designed to provide build scheduling, so 3
165 example crontabs are provided in the directory
166 "/etc/auto-build.d/cron". To run the build engine continous 24x7, copy
167 the "continuous.conf" file into "/etc/cron.d". There are also examples
168 for running an hourly, or daily build cycle.
169 NB. before setting up a scheduled build in cron it is a good idea to
171 run a cycle manually to ensure that your installation & configuration
172 is operating normally.
173 BASIC CONFIGURATION
175 The following notes describe how to add simple modules to the build
177 engine.
178 MODULE CONTROL FILES
180 To achieve complete independance from the language and tools used for a
182 project, the build stage of the autobuild cycle is considered a black
183 box. To perform the build, autobuild merely invokes an opaque "control
184 file" provided by the developer. This shell script will typically per‐
185 form four tasks - configure, build, install and package. The autobuild
186 will capture the standard output and error streams, saving to the mod‐
187 ules build log files. In common with standard UNIX behaviour, an exit
188 status of zero indicates a successful build, while non-zero indicates
189 failure.
190 By convention, the script should be called "autobuild.sh" and be placed
192 in the top level directory of the module's source. When invoked a num‐
193 ber of environment variables will be set for use by the control file.
194 AUTOBUILD_MODULE
196 The name of the module which is being built
197 AUTOBUILD_INSTALL_ROOT
199 The path to a virtual root directory where software should be
200 installed. When a module is built, it is guarenteed that the
201 installed files from any dependant modules will be present in this
202 directory. This enables a module to depend on the output of
203 another module. The contents of this directory, however, are not
204 preserved across runs of the build.
205 AUTOBUILD_PACKAGE_ROOT
207 The path to a directory in which a module's build process will cre‐
208 ate any binary packages it generates, for example RPMs, or Debian
209 packages. The packages are typically placed into a package type
210 specific sub-directory.Consider, for example, a module which gener‐
211 ates an RPM, of itself. The $AUTOBUILD_PACKAGE_ROOT directory
212 would be used to set the '_topdir' macro for the RPM build process
213 rpmbuild --define '_topdir $AUTOBUILD_PACKAGE_ROOT/rpm' -ta foo.tar.gz
215 AUTOBUILD_SOURCE_ROOT
217 The path to the directory in which modules are checked out. This
218 can be used in conjunction with $AUTOBUILD_MODULE to identify the
219 root directory for the module
220 AUTOBUILD_COUNTER
222 A counter identifying the current run of the build. Typically this
223 will be the number of seconds since the UNIX at the time the cur‐
224 rent build cycle began, but may alternatively be a version control
225 changelist.
226 AUTOBUILD_TIMESTAMP
228 A counter identifying the timestamp taken at the start of the build
229 cycle. When checking out code, all version control systems are syn‐
230 chronized to no later than this timestamp. For a given value of
231 $AUTOBUILD_TIMESTAMP the source code being built will always be
232 identical, thus this is suitable for use as a unique identifier for
233 a build.
234 The overall goal of the control file is to build the software, run unit
236 tests and then install the software into the autobuild install root.
237 The location of the latter is given by the "AUTOBUILD_INSTALL_ROOT",
238 and would typically be used to set the "prefix" when running "config‐
239 ure" or an equivalent script. The control file should exit when a non-
240 zero exit status if an error occurs at any time. An exit status of
241 zero, indicates a successful build & install.
242 For example of creating a script for GNU AutoTools refer to latter part
244 of this manual page.
245 MODULE CONFIGURATION
247 ADVANCED TOPICS
249 MODULE DEPENDANCIES
251 PUBLISHING UNIT TEST RESULTS
253 As well as a number of environment variables, the control file is also
255 passed a single command line argument. This argument contains the name
256 of a file into which unit test results should be stored. There are
257 (currently) no requirements on what data format should be used when
258 writing to this file, so at its simplest one can just capture the out‐
259 put from a 'make check' command. So, taking the control file written
260 earlier, one would add
261 test -n "$1" && TEST_RESULTS_FILE=$1 ⎪⎪ TEST_RESULTS_FILE=results.log
263 rm -f $TEST_RESULTS_FILE
264 make check ⎪ tee $TEST_RESULTS_FILE
266 There are a couple of things to note here. First, if no argument is
268 given we default to writing results to a file called "results.log".
269 While this is not neccessary if running within the build engine, it
270 does make it easy for a developer to execute the control file ("auto‐
271 build.sh") manually. Second, rather than just redirecting output into
272 the results file, it is piped through the tee(1) command. This simple
273 lets one monitor the progress of tests when running the control file
274 manually.
275 When the HTML status pages are generated for a module, the contest of
277 the unit test results log will be made available for download.
278 CONTROL FILE EXAMPLES
280 A control file using GNU AutoTools (autoconf/automake)
282 With a few notable exceptions (Perl & Apache), C and C++ programmers
284 have been increasingly converging on GNU Auto Tools for configuring and
285 building their software. As with MakeMaker, the key task is to set the
286 installation prefix when running the configure script:
287 ./configure --prefix=$AUTOBUILD_INSTALL_ROOT
289 The task of resolving inter-module build dependancies is somewhat less
291 well defined. There are a number of approaches that may work - some
292 programs may even need a combination of all of them!
293 Helper scripts (aka pkg-config)
295 Some libraries install a small shell script to the bin directory that
297 programs can use to determine the correct compiler and linker flags.
298 For example, the "pkg-config" script can print out the compiler flags
299 required when building against most libraries. It is usually just suf‐
300 ficient to set the PKG_CONFIG_PATH environment variable to point to the
301 builder install root
302 PKG_CONFIG_PATH=$AUTOBUILD_INSTALL_ROOT/lib/pkgconfig
304 Configure flags
306 Another common approach to locating pre-requisite libraries is for the
308 program's configure script to have command line options for specifying
309 include and library search paths. For example, when building the Resin
310 Java servlet container, the configure script has --with-openssl-lib and
311 --with-openssl-include options. Thus when configuring Resin, as well as
312 setting the prefix, we'd set these two options:
313 ./configure --prefix=$AUTOBUILD_INSTALL_ROOT \
315 --with-openssl-lib=$AUTOBUILD_INSTALL_ROOT/lib
316 --with-openssl-include=$AUTOBUILD_INSTALL_ROOT/include
317 An alternative to specifying both library and include search paths is
319 to just tell the configure script the installation prefix of the
320 library:
321 ./configure --prefix=$AUTOBUILD_INSTALL_ROOT \
323 --with-openssl=$AUTOBUILD_INSTALL_ROOT
324 Compiler / linker environment variables
326 If there is no explicit support for specifying the location of pre-req‐
328 uisite libraries then the final approach is to try setting compiler
329 environment variables. The two important ones being <code>CFLAG/code
330 and <code>LDFLAG/code:
331 CFLAGS=-I$AUTOBUILD_INSTALL_ROOT/include
333 LDFLAGS=-L$AUTOBUILD_INSTALL_ROOT/lib
334 Complete AutoTools example
336 #!/bin/sh
338 set -e
340 # Pull in config scripts
342 PATH=$AUTOBUILD_INSTALL_ROOT/bin:$PATH
343 export PATH
344 # Clean up build area
346 [ -f Makefile ] && make -k maintainer-clean ⎪⎪:
347 # Re-generate autotools scripts
349 autoconf
350 automake -a
351 # Configure the build
353 ./configure --prefix=$AUTOBUILD_INSTALL_ROOT \
354 --with-openssl=$AUTOBUILD_INSTALL_ROOT
355 # Make & install
357 make
358 make install
359 # Create source code dist
361 make dist
362
364 Written by Daniel P. Berrange and Dennis Gregorovic, with contributions
365 from Richard W.M. Jones.
366
368 Report bugs to the bug tracker provided on the GNA! project site linked
369 from <http://www.autobuild.org>.
370
372 Copyright (C) 2002-2006 Daniel P. Berrange, Dennis Gregorovic, Red Hat.
373 Refer to individual source files for further details of copyright hold‐
374 ers.
375 Parts of this software are derived from Rolling builds version 2.0
377 Copyright (C) Richard W.M. Jones <rich@annexia.org>
378 Test-AutoBuild is distributed under the terms of the GNU GPL v2+. This
380 is free software; see the source for copying conditions. There is NO
381 warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR
382 PURPOSE.
383
385 For configuration information auto-build.conf(5); for setting up a
386 build root auto-build-make-root(1); for developers wishing to extend
387 the build framework Test::AutoBuild(3pm); for further project informa‐
388 tion "http://www.autobuild.org"; for developer questions "mailto:tes‐
389 tautobuild-devel@gna.org".
390perl v5.8.8 2007-12-08 AUTO-BUILD(1)