1AUTO-BUILD(1) User Contributed Perl Documentation AUTO-BUILD(1)
2
6 auto-build - run automated, unattended software build processes
7
9 # auto-build [OPTION]...
10 Run build using latest code currently in source control
12 # auto-build --config /etc/auto-build.d/auto-build.conf
13 Run build using code committed prior to 12:30 on Dec 12th 2004
15 # auto-build --timestamp '12:30:00 Dec 12th 2004' \
16 --config /etc/auto-build.d/auto-build.conf
17
19 "auto-build" is the command used to execute the Test-AutoBuild software
20 build automation framework. It is intended to execute a single
21 iteration of the build engine and then quit with its exit status
22 providing an indication of the build success / fail status (see the
23 "EXIT STATUS" section later in this manual page). To perform a
24 continous automated build this command would be launched on a frequent
25 basis from cron. Locking will ensure that two instances do not execute
26 concurrently, so it can safely be triggered every 5 minutes.
27
29 The following command line options are valid:
30 --config=FILE
32 Specify an alternative path to the configuration file for the build
33 process. If ommitted, will default to looking for the file
34 $HOME/auto-build.conf. The configuration data is loaded by the
35 Config::Record module, so refer to that module's manual page for
36 details of permitted syntax. The file will also be pre-processed by
37 the Template module allowing dynamic generation of complex
38 configuration files
39 --log4perl=FILE
41 Specify an alternative file containing configuration settings for
42 the Log::Log4perl logging system. If ommitted, log4perl settings
43 will be loaded from the master autobuild configuration file.
44 --timestamp=DATE
46 Specify the timestamp to use when checking code out of the source
47 control repositories. For repository modules supporting timestamps
48 it is guarenteed that the code checked out will be the latest
49 version not newer than this timestamp. If ommitted, the timestamp
50 will default to the current timestamp. The argument can be either
51 an plain integer specifying number of seconds since the epoch, or
52 any of the text representations handled by the Date::Manip module
53 'today'
55 '1st thursday in June 1992'
56 '05/10/93'
57 '12:30 Dec 12th 1880'
58 '8:00pm december tenth'
59 --verbose
61 Increase the verbosity of warning messages and errors. This will
62 ensure that if the builder unexpectedly crashes, a complete stack
63 trace will be dumped along will full arguments.It will also cause
64 the post-processed configuration file to be dumped.
65 --help
67 Display a summary of the command line help options
68
70 The build process provides special handlers for a couple of signals to
71 assist in debugging problems.
72 USR1
74 When receiving this signal the build process will send a stack
75 trace to STDERR, providing an indication of the current task being
76 performed. The build process will continue to run after receiving
77 and handling this signal.
78 USR2
80 When receiving this signal the build process will use Data::Dumper
81 to dump the entire runtime state of the builder to STDERR. The
82 build process will continue to run after receiving and handling
83 this signal
84
86 The exit status codes of the build have the following meanings
87 0 The build process completed succesfully with no failures in any
89 module
90 1 The build process failed on one of more modules, but ran through to
92 completion of all stages
93 2 The build process aborted during processing, and did not complete
95 execution of all stages.
96 3 An invalid/unknown command line option was provided
98 4 The argument to one of the command line options was malformed or
100 invalid.
101
103 The notes which follow assume the software itself has been installed
104 into the /usr hierarchy with all common commands available within the
105 default $PATH. Some minor alterations may be neccessary if this is not
106 the case.
107 HOST SETUP
109 Before using the build engine a handful of things must be configured on
110 the host machine. This shouldn't take more than a few minutes to
111 accomplish if the example configurations are followed.
112 BUILD AREA CREATION
114 The first step after software installation is to create an area where
116 builds can be performed. The build engine expects that the build area
117 be populated with a small set of top level directories. To aid in this
118 setup, the command "auto-build-make-root(1)" can be run passing the
119 build area path as a command line parameter. The example configuration
120 file expects that the build area is at "/var/lib/builder", so assuming
121 this is the case, run the command:
122 # auto-build-make-root /var/lib/builder
124 NB, if the software was installed from RPM, this directory may have
126 been created on your behalf.
127 USER ACCOUNT CREATION
129 It is highly recommended that the build engine be run as an
131 unprivileged user. This is primarily intended to protect the host
132 machine against both delibrate and accidental flaws in a module's build
133 script. By convention the user to run the build as will be called
134 "builder". If only a single instance of the build engine is to be run
135 on a host, it is convenient to make this user's home directory would
136 point to the build area root. So to add a user to run the build engine
137 run the command:
138 # groupadd builder
140 # useradd -d /var/lib/builder -g builder builder
141 # passwd builder
142 WEB STATUS PAGE SETUP
144 The build engine will generate HTML status pages at the end of every
146 run summarizing the status of the build. There are 3 example
147 configuration files suitable for use with Apache located in
148 "/etc/auto-build.d/httpd". The standard "auto-build.conf" file assumes
149 that virtual hosting will be used on the web server, with a suitable
150 Apache config being the file "vhost.conf". Copy it to
151 /etc/httpd/conf.d, and edit it to set the server name of your build
152 host. If virtual hosting is not suitable, then the status pages can be
153 made to appear at a fixed URL "/builder", or as a user directory under
154 "/~builder". The sample config files "aliased.conf" or "user.conf" can
155 be used instead of "vhost.conf", although it will also be neccessary to
156 edit the "auto-build.conf" file to change the "httpPrefix" option.
157 AUTOMATION
159 As mentioned earlier, executing "auto-build" just runs a single
161 iteration of the build engine. To achieve continuous integration, this
162 command should be scheduled to run on a periodic (frequent) basis. On
163 UNIX, cron is perfectly designed to provide build scheduling, so 3
164 example crontabs are provided in the directory
165 "/etc/auto-build.d/cron". To run the build engine continous 24x7, copy
166 the "continuous.conf" file into "/etc/cron.d". There are also examples
167 for running an hourly, or daily build cycle.
168 NB. before setting up a scheduled build in cron it is a good idea to
170 run a cycle manually to ensure that your installation & configuration
171 is operating normally.
172 BASIC CONFIGURATION
174 The following notes describe how to add simple modules to the build
175 engine.
176 MODULE CONTROL FILES
178 To achieve complete independance from the language and tools used for a
180 project, the build stage of the autobuild cycle is considered a black
181 box. To perform the build, autobuild merely invokes an opaque "control
182 file" provided by the developer. This shell script will typically
183 perform four tasks - configure, build, install and package. The
184 autobuild will capture the standard output and error streams, saving to
185 the modules build log files. In common with standard UNIX behaviour, an
186 exit status of zero indicates a successful build, while non-zero
187 indicates failure.
188 By convention, the script should be called "autobuild.sh" and be placed
190 in the top level directory of the module's source. When invoked a
191 number of environment variables will be set for use by the control
192 file.
193 AUTOBUILD_MODULE
195 The name of the module which is being built
196 AUTOBUILD_INSTALL_ROOT
198 The path to a virtual root directory where software should be
199 installed. When a module is built, it is guarenteed that the
200 installed files from any dependant modules will be present in this
201 directory. This enables a module to depend on the output of
202 another module. The contents of this directory, however, are not
203 preserved across runs of the build.
204 AUTOBUILD_PACKAGE_ROOT
206 The path to a directory in which a module's build process will
207 create any binary packages it generates, for example RPMs, or
208 Debian packages. The packages are typically placed into a package
209 type specific sub-directory.Consider, for example, a module which
210 generates an RPM, of itself. The $AUTOBUILD_PACKAGE_ROOT directory
211 would be used to set the '_topdir' macro for the RPM build process
212 rpmbuild --define '_topdir $AUTOBUILD_PACKAGE_ROOT/rpm' -ta foo.tar.gz
214 AUTOBUILD_SOURCE_ROOT
216 The path to the directory in which modules are checked out. This
217 can be used in conjunction with $AUTOBUILD_MODULE to identify the
218 root directory for the module
219 AUTOBUILD_COUNTER
221 A counter identifying the current run of the build. Typically this
222 will be the number of seconds since the UNIX at the time the
223 current build cycle began, but may alternatively be a version
224 control changelist.
225 AUTOBUILD_TIMESTAMP
227 A counter identifying the timestamp taken at the start of the build
228 cycle. When checking out code, all version control systems are
229 synchronized to no later than this timestamp. For a given value of
230 $AUTOBUILD_TIMESTAMP the source code being built will always be
231 identical, thus this is suitable for use as a unique identifier for
232 a build.
233 The overall goal of the control file is to build the software, run unit
235 tests and then install the software into the autobuild install root.
236 The location of the latter is given by the "AUTOBUILD_INSTALL_ROOT",
237 and would typically be used to set the "prefix" when running
238 "configure" or an equivalent script. The control file should exit when
239 a non-zero exit status if an error occurs at any time. An exit status
240 of zero, indicates a successful build & install.
241 For example of creating a script for GNU AutoTools refer to latter part
243 of this manual page.
244 MODULE CONFIGURATION
246 ADVANCED TOPICS
248 MODULE DEPENDANCIES
249 PUBLISHING UNIT TEST RESULTS
251 As well as a number of environment variables, the control file is also
253 passed a single command line argument. This argument contains the name
254 of a file into which unit test results should be stored. There are
255 (currently) no requirements on what data format should be used when
256 writing to this file, so at its simplest one can just capture the
257 output from a 'make check' command. So, taking the control file written
258 earlier, one would add
259 test -n "$1" && TEST_RESULTS_FILE=$1 || TEST_RESULTS_FILE=results.log
261 rm -f $TEST_RESULTS_FILE
262 make check | tee $TEST_RESULTS_FILE
264 There are a couple of things to note here. First, if no argument is
266 given we default to writing results to a file called "results.log".
267 While this is not neccessary if running within the build engine, it
268 does make it easy for a developer to execute the control file
269 ("autobuild.sh") manually. Second, rather than just redirecting output
270 into the results file, it is piped through the tee(1) command. This
271 simple lets one monitor the progress of tests when running the control
272 file manually.
273 When the HTML status pages are generated for a module, the contest of
275 the unit test results log will be made available for download.
276 CONTROL FILE EXAMPLES
278 A control file using GNU AutoTools (autoconf/automake)
279 With a few notable exceptions (Perl & Apache), C and C++ programmers
281 have been increasingly converging on GNU Auto Tools for configuring and
282 building their software. As with MakeMaker, the key task is to set the
283 installation prefix when running the configure script:
284 ./configure --prefix=$AUTOBUILD_INSTALL_ROOT
286 The task of resolving inter-module build dependancies is somewhat less
288 well defined. There are a number of approaches that may work - some
289 programs may even need a combination of all of them!
290 Helper scripts (aka pkg-config)
292 Some libraries install a small shell script to the bin directory that
294 programs can use to determine the correct compiler and linker flags.
295 For example, the "pkg-config" script can print out the compiler flags
296 required when building against most libraries. It is usually just
297 sufficient to set the PKG_CONFIG_PATH environment variable to point to
298 the builder install root
299 PKG_CONFIG_PATH=$AUTOBUILD_INSTALL_ROOT/lib/pkgconfig
301 Configure flags
303 Another common approach to locating pre-requisite libraries is for the
305 program's configure script to have command line options for specifying
306 include and library search paths. For example, when building the Resin
307 Java servlet container, the configure script has --with-openssl-lib and
308 --with-openssl-include options. Thus when configuring Resin, as well as
309 setting the prefix, we'd set these two options:
310 ./configure --prefix=$AUTOBUILD_INSTALL_ROOT \
312 --with-openssl-lib=$AUTOBUILD_INSTALL_ROOT/lib
313 --with-openssl-include=$AUTOBUILD_INSTALL_ROOT/include
314 An alternative to specifying both library and include search paths is
316 to just tell the configure script the installation prefix of the
317 library:
318 ./configure --prefix=$AUTOBUILD_INSTALL_ROOT \
320 --with-openssl=$AUTOBUILD_INSTALL_ROOT
321 Compiler / linker environment variables
323 If there is no explicit support for specifying the location of pre-
325 requisite libraries then the final approach is to try setting compiler
326 environment variables. The two important ones being <code>CFLAG/code
327 and <code>LDFLAG/code:
328 CFLAGS=-I$AUTOBUILD_INSTALL_ROOT/include
330 LDFLAGS=-L$AUTOBUILD_INSTALL_ROOT/lib
331 Complete AutoTools example
333 #!/bin/sh
335 set -e
337 # Pull in config scripts
339 PATH=$AUTOBUILD_INSTALL_ROOT/bin:$PATH
340 export PATH
341 # Clean up build area
343 [ -f Makefile ] && make -k maintainer-clean ||:
344 # Re-generate autotools scripts
346 autoconf
347 automake -a
348 # Configure the build
350 ./configure --prefix=$AUTOBUILD_INSTALL_ROOT \
351 --with-openssl=$AUTOBUILD_INSTALL_ROOT
352 # Make & install
354 make
355 make install
356 # Create source code dist
358 make dist
359
361 Written by Daniel P. Berrange and Dennis Gregorovic, with contributions
362 from Richard W.M. Jones.
363
365 Report bugs to the bug tracker provided on the GNA! project site linked
366 from <http://www.autobuild.org>.
367
369 Copyright (C) 2002-2006 Daniel P. Berrange, Dennis Gregorovic, Red Hat.
370 Refer to individual source files for further details of copyright
371 holders.
372 Parts of this software are derived from Rolling builds version 2.0
374 Copyright (C) Richard W.M. Jones <rich@annexia.org>
375 Test-AutoBuild is distributed under the terms of the GNU GPL v2+. This
377 is free software; see the source for copying conditions. There is NO
378 warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR
379 PURPOSE.
380
382 For configuration information auto-build.conf(5); for setting up a
383 build root auto-build-make-root(1); for developers wishing to extend
384 the build framework Test::AutoBuild(3pm); for further project
385 information "http://www.autobuild.org"; for developer questions
386 "mailto:testautobuild-devel@gna.org".
387perl v5.12.1 2007-12-08 AUTO-BUILD(1)