1MAKE-REDIR(1) User Commands MAKE-REDIR(1)
2
6 make-redir - run "make" but automatically redirect its results to DEST‐
7 DIR
8
10 make-redir [MAKE_ARGUMENT*]...[install]
11 * DESTDIR=directory and -k/--keep-going are especially common.
13
15 The make-redir program runs "make" in a way that redirects file instal‐
16 lations to descend from the $DESTDIR directory, even if the provided
17 makefile doesn't natively support $DESTDIR. In most cases, simply
18 replace "make install" with "make-redir DESTDIR=directory install"
19 (adding options as necessary).
20 The make-redir program attempts to solve a common problem for packagers
22 of software and for users who directly install source code: Inadequate
23 DESTDIR support. Software is typically installed on Unix/Linux from a
24 source code package by typing "make install". Historically, this
25 script would write directly to privileged directories to perform the
26 installation (e.g., /usr/bin), or at least to a directory that merged
27 all installed files (e.g., $HOME/bin). However, packagers of software
28 using most modern packaging systems (such as .rpm and .deb) instead
29 require that files be written to some intermediate directory, even
30 though they will be run from a different filesystem location. Simi‐
31 larly, users who directly install source code often use programs such
32 as GNU stow, encap, or toast. These programs install each package in
33 its own separate directory, but work best if the installed programs are
34 invoked through a different directory (which merges the contents of
35 multiple packages, simplifying the PATH and supporting plug-ins).
36 Thus, for both many packagers of software and for many users who
37 directly install source code, a program should be installed in one
38 directory, yet actually be run from a different directory. This is
39 easy to do if the installation script supports the "DESTDIR conven‐
40 tion"; simply run "make DESTDIR=something install", which would install
41 the files descending from DESTDIR, even though the files are configured
42 to run without the DESTDIR prefix. Unfortunately, many source packages
43 don't support the DESTDIR convention.
44 The auto-destdir tools "make-redir" and "run-redir" will automatically
46 redirect file reads/writes to follow the DESTDIR convention, without
47 requiring any changes to the build files (e.g., a makefile). Attempts
48 to write to directories that are normally root-privileged (such as
49 /usr/bin) will instead be redirected to descend from a designated des‐
50 tination directory. This destination directory is specified by the
51 environment variable REDIR_DESTDIR or DESTDIR, with REDIR_DESTDIR tak‐
52 ing precedence.
53 You should use make-redir (not run-redir) if you are redirecting
55 "make", because make-redir performs additional helpful actions. If you
56 pass DESTDIR=something as a parameter, it automatically sets the DEST‐
57 DIR environment variable. The make-redir command also sets the over‐
58 rides INSTALL="install -p" CP="cp -p" MKDIR="mkdir" MKDIR_P="mkdir -p"
59 mkdir_p="mkdir -p" LN="ln", so that makefiles with absolute pathnames
60 of common commands will work correctly and preserve timestamps. (Add
61 overrides on the command line if these are not acceptable.) By default
62 make-redir also sets special SHELL and MAKESHELL values, so that even
63 invocations with a full path name (like /bin/cp) will be redirected if
64 they begin a line in a makefile.
65 Internally, make-redir runs "make" using run-redir; run-redir does the
67 actual file redirection. Use run-redir if you need to redirect file
68 writes without first running "make".
69 The auto-destdir tools are designed to "just work" given the limita‐
71 tions described below (in particular, only "wrapped" commands should
72 read/write files that must be redirected). Filenames will not be redi‐
73 rected twice, so it's safe to use run-redir and make-redir even with
74 makefiles that do support DESTDIR. If a write is redirected, all par‐
75 ent directories are first automatically created if they exist in the
76 unredirected filesystem. This means that an attempt to "cp" some file
77 to /usr/bin/xyz will create the directory "$DESTDIR/usr/bin" if
78 /usr/bin already exists, then copy the file to "$DESTDIR/usr/bin/xyz".
79 If a file is being read, a file written inside $DESTDIR will have
80 precedence over a file that isn't; this nicely simulates the overwrit‐
81 ing of an old file with a new one. Since make-redir and run-redir set
82 the PATH, they will work even if there are many levels of indirection,
83 and even if they use tools other than make (as long as they end up
84 eventually invoking the usual commands such as cp and install).
85 If the "make install" works except that it uses a few "unwrapped" com‐
87 mands, consider using the "-k" ("--keep-going") option, so that make-
88 redir will keep going instead of failing on an error. Then, re-perform
89 the actions that would have failed. Alternatively, you may be able to
90 perform one or two actions "by hand" first, and then invoke make-redir
91 to do the rest.
92 The programs make-redir and run-redir can be run as root, but in most
94 cases, you should not run them as root. The installation program
95 itself may require root to run (e.g., to set ownership), even if files
96 are written to DESTDIR. If you just want to record permissions (e.g.,
97 chown, chmod), consider also using "fakeroot" to record the intended
98 permissions instead of running as root. When creating an rpm package,
99 the permission-setting is typically irrelevant, since in most cases it
100 is the rpm spec file that will ultimately set the permissions. If the
101 permission-setting is irrelevant, you may want to run make-redir as
102 non-root and pass it the "-k" ("--keep-going") option so that make will
103 keep going instead of failing on an error.
104 A file/directory is not redirected if its full pathname begins as a
106 typical user directory (specifically /home/ or /Users/) or temporary
107 directory (/tmp/, /usr/tmp/, /var/tmp/, or /opt/tmp/). A file/direc‐
108 tory is redirected if its full pathname begins with a directory name
109 that is normally privileged: /usr/, /etc/, /boot/, /bin*/, /dev/,
110 /lib*/, /opt/, /sbin*/, /selinux/, /srv/, /sys/, and /var/. If neither
111 case applies, it is not redirected.
112 You can override some values, though few will need to do so. By
114 default, make-redir invokes "make"; this can be changed by setting the
115 environment variable REALMAKE to whatever "make" program should be
116 invoked instead. By default, make-redir sets a special SHELL value (so
117 it can override commands that give full pathnames); this special value
118 does some processing and then invokes /bin/bash. Set the environment
119 variable REALMAKESHELL to the name of the shell to use after doing this
120 filtering if you do not want it to use /bin/bash. You can disable this
121 filtering entirely by setting the make variables SHELL and MAKESHELL on
122 the command line to the name of the shell program to use (e.g., make-
123 redir SHELL=/bin/sh MAKESHELL=/bin/sh); if make can be called recur‐
124 sively, also set the environment variables with the same name. If you
125 do not want any makefile variable overrides, set the environment vari‐
126 able DESTDIR and use "run-redir make ..." instead.
127
129 The following configures, builds, and then "installs" a program. The
130 "installed" files will descend from /tmp/stuff, e.g., directory
131 /tmp/stuff/usr/bin will contain files that were intended to be
132 installed in /usr/bin:
133 ./configure --prefix=/usr
135 make
136 make-redir DESTDIR=/tmp/stuff install
137 For example, if "cp myprogram /usr/bin" is one of the instructions in
139 the makefile for the "install" target, then (presuming /usr/bin exists)
140 it will actually perform the equivalent of:
141 /bin/mkdir -p /tmp/stuff/usr/bin
142 /bin/cp myprogram /tmp/stuff/usr/bin
143 You can use make-redir inside an RPM .spec file. Be sure to "Buil‐
145 dRequires" auto-destdir, and set the DESTDIR to be the RPM build root,
146 as follows in the spec file:
147 ...
149 BuildRequires: auto-destdir
150 ...
151 %install
152 rm -rf "$RPM_BUILD_ROOT"
153 make-redir DESTDIR="$RPM_BUILD_ROOT" install
154 Some RPM spec authors may prefer to use %{buildroot} instead of
156 $RPM_BUILD_ROOT.
157
159 The auto-destdir tools (including run-redir and make-redir) work by
160 modifying the PATH environment variable and providing a few "wrappers"
161 that override typical installation commands like "cp" and "install".
162 These commands with wrappers, called "wrapped commands", use the values
163 of $REDIR_DESTDIR or $DESTDIR (with REDIR_DESTDIR taking precedence),
164 and will redirect reads and writes of files under certain conditions.
165 The "redirection" comes by simply changing the name of the file source
166 or destination, and then calling the "real" program being wrapped. The
167 internal function redir_name takes the original filename, and how it
168 will be used (e.g., for reading or writing), and determines if it
169 should be renamed (and if so, to what). In addition, make-redir redi‐
170 rects all shell commands (using SHELL and a special wrapper for shell
171 commands); any command that begins with the prefix /usr/ or /usr/bin/,
172 followed by a wrapped command's name, will have that prefix removed.
173 This may seem to be an odd implementation approach, but it has many
175 advantages. The "obvious" way to implement redirection is to manipu‐
176 late LD_PRELOAD and wrap the open() and creat() calls. But as the
177 fakeroot(1) documentation notes, this approach creates "problems, as
178 demonstrated by the libtricks package. This package wrapped many more
179 functions, and tried to do a lot more than fakeroot. It turned out
180 that a minor upgrade of libc (from one where the stat() function didn't
181 use open() to one with a stat() function that did (in some cases) use
182 open()), would cause unexplainable segfaults (that is, the libc6 stat()
183 called the wrapped open(), which would then call the libc6 stat(),
184 etc). Fixing them wasn't all that easy, but once fixed, it was just a
185 matter of time before another function started to use open()...".
186 LD_PRELOAD-based approaches fail on commands that are statically
187 linked, and the commands usually used for installation (e.g., cp) are
188 often statically linked. Others tools (like fakeroot) also use
189 LD_PRELOAD, so using LD_PRELOAD means that these other tools cannot be
190 used simultaneously. Other "obvious" approaches are to use ptrace(),
191 or redirect file writes using the kernel (e.g. with chroot), but these
192 turn out to have many problems too, as discussed in
193 <http://www.dwheeler.com/essays/automating-destdir.html>.
194 In contrast, since run-redir and make-redir don't manipulate LD_PRELOAD
196 to manipulate open() or creat(), they are immune to deadlocks caused by
197 changes in the C library, work with statically-linked commands, and
198 work with tools like fakeroot. No special privileges are needed to run
199 these programs, nor do they require setuid installation, eliminating
200 many security concerns (compared to a chroot, setuid, or kernel-based
201 approach). No kernel driver is used, nor is ptrace() used, so these
202 tools should be fairly portable to any POSIX-based system with bash.
203 All work happens very quickly, since nearly all commands simply run as
204 usual, and even the wrapped commands merely run a short shell script
205 before running as usual. The implementation is entirely a set of small
206 bash scripts with trivial dependencies, so the auto-destdir tools can
207 be easily included in a package's build process without slowing the
208 build process down (even if the build process requires re-installing
209 dependencies on each run).
210 In practice, this approach is sufficient to automate DESTDIR for typi‐
212 cal installation scripts. Typical "make install" programs, even if
213 they are deeply recursive, only use a very few commands to actually
214 write files to their final locations in trusted directories, and usu‐
215 ally invoke commands without using the full pathnames of the commands
216 (e.g., "install -p myprog /usr/bin"). As a result, typical installa‐
217 tion scripts are easily redirected by this program.
218 More information about various implementation alternatives, their pros,
220 and their cons, is here: http://www.dwheeler.com/essays/automating-
221 destdir.html
222
224 Only wrapped programs will be redirected to the new destination direc‐
225 tory; any other program will not be redirected. Currently, the
226 "wrapped" commands are (alphabetically):
227 chmod, cp, install, ln, mkdir, mv, and touch.
229 If the makefile invokes wrapped programs using absolute pathnames
231 (e.g., calling "/bin/cp" instead of "cp"), then the command will be
232 redirected anyway if it is the absolutely first command at the begin‐
233 ning of a line in a makefile and is invoked directly or via makefile
234 variable expansions like $(CP). Absolute pathnames at the beginning of
235 a line are handled because make-redir sets SHELL to a special shell
236 wrapper that handles this particular case. However, absolute pathnames
237 for wrapped programs are not handled in any other place (doing so would
238 require the shell wrapper to re-implement shell parsing); e.g., program
239 names stored in shell variable expansions will not be intercepted.
240 Thus, makefile lines like "cd X; /usr/bin/install myprogram /usr/bin"
241 will not be redirected. In these cases, you should change absolute
242 pathname invocations like /usr/bin/install to unnamed-path invocations
243 like install.
244 Capabilities built into the shell are not redirected, including I/O re‐
246 direction such as "<", ">", and ">>". This means that "cat x >
247 /usr/bin/program" will not be redirected; use "cp x /usr/bin/program"
248 or similar instead. Similarly, "test" and "[" (which are normally
249 shell built-ins) aren't redirected. Changing this would require auto-
250 destdir to re-implement a full shell.
251 The wrappers presume GNU command line options, and will not understand
253 other options even if the underlying tool would. In practice, this is
254 almost always irrelevant; "make install" typically only uses highly-
255 portable commands to copy files and make directories, and does not use
256 unusual options.
257 The "rm" and "rmdir" commands are not redirected simply because these
259 are unlikely commands during a "make install". Similarly, there are no
260 "whiteout" files to record when a file is removed from $DESTDIR,
261 because this does not seem useful for "make install".
262 In cases where this wrapper is not sufficient, you may need to modify
264 the installation script. If significant modifications to the installa‐
265 tion script are required, you may as well add DESTDIR support directly
266 to the installation script. The auto-destdir tools are designed to
267 handle common cases, so that people can focus their time on the cases
268 that need special handling.
269
272 When writing the commands for "make install":
273 · Only use the wrapped commands to install files and directories in
275 their final positions (e.g., install, cp, ln, and mkdir).
276 · Don't use full pathnames of wrapped commands (e.g.,
278 "/usr/bin/install"); use their basename (e.g., "install").
279 · Don't use redirection (e.g., ">") to write to files in privileged
281 directories; use redirection to write to a "local" file (typically
282 during the build process), and then use "install" or "cp" to install
283 the result.
284 Supporting DESTDIR is still a good idea.
286
288 As noted above, only "wrapped" commands are redirected. Some commands
289 that aren't wrapped, and perhaps should be, include: install-info,
290 ldconfig, rm, rmdir, chown, chgrp, and libtool. For permissions-
291 related commands (like chmod, chown, and chgrp), consider also using
292 fakeroot or make's "-k" option.
293 Options with optional arguments cannot accept arguments that begin with
295 "-", due to limitations of getopt(1).
296 cp's --parents argument is supported, but it won't work if the original
298 files are being copied from a redirected location.
299 When running make-redir, the makefile's SHELL value is ignored. If you
301 need to use the SHELL value provided by the makefile, run make-redir
302 and provide "SHELL=..." with the correct shell value. If you do this,
303 wrapped commands must never be invoked using their full pathname.
304
306 David A. Wheeler
307
309 Report bugs to <dwheeler, at, dwheeler dot com>
310
312 Copyright © 2009 Institute for Defense Analyses. Released using the
313 MIT license.
314 This is free software: you are free to change and redistribute it.
315 There is NO WARRANTY, to the extent permitted by law.
316
318 run-redir(1)
319auto-destdir August 2009 MAKE-REDIR(1)