1MAKE-REDIR(1)                    User Commands                   MAKE-REDIR(1)
2
3
4

NAME

6       make-redir - run "make" but automatically redirect its results to DEST‐
7       DIR
8

SYNOPSIS

10       make-redir [MAKE_ARGUMENT*]...[install]
11
12              * DESTDIR=directory and -k/--keep-going are especially common.
13

DESCRIPTION

15       The make-redir program runs "make" in a way that redirects file instal‐
16       lations  to  descend  from the $DESTDIR directory, even if the provided
17       makefile doesn't natively support  $DESTDIR.   In  most  cases,  simply
18       replace  "make  install"  with  "make-redir  DESTDIR=directory install"
19       (adding options as necessary).
20
21       The make-redir program attempts to solve a common problem for packagers
22       of  software and for users who directly install source code: Inadequate
23       DESTDIR support.  Software is typically installed on Unix/Linux from  a
24       source  code  package  by  typing  "make  install".  Historically, this
25       script would write directly to privileged directories  to  perform  the
26       installation  (e.g.,  /usr/bin), or at least to a directory that merged
27       all installed files (e.g., $HOME/bin).  However, packagers of  software
28       using  most  modern  packaging  systems (such as .rpm and .deb) instead
29       require that files be written  to  some  intermediate  directory,  even
30       though  they  will  be run from a different filesystem location.  Simi‐
31       larly, users who directly install source code often use  programs  such
32       as  GNU  stow, encap, or toast.  These programs install each package in
33       its own separate directory, but work best if the installed programs are
34       invoked  through  a  different  directory (which merges the contents of
35       multiple packages,  simplifying  the  PATH  and  supporting  plug-ins).
36       Thus,  for  both  many  packagers  of  software  and for many users who
37       directly install source code, a program  should  be  installed  in  one
38       directory,  yet  actually  be  run from a different directory.  This is
39       easy to do if the installation script  supports  the  "DESTDIR  conven‐
40       tion"; simply run "make DESTDIR=something install", which would install
41       the files descending from DESTDIR, even though the files are configured
42       to run without the DESTDIR prefix.  Unfortunately, many source packages
43       don't support the DESTDIR convention.
44
45       The auto-destdir tools "make-redir" and "run-redir" will  automatically
46       redirect  file  reads/writes  to follow the DESTDIR convention, without
47       requiring any changes to the build files (e.g., a makefile).   Attempts
48       to  write  to  directories  that  are normally root-privileged (such as
49       /usr/bin) will instead be redirected to descend from a designated  des‐
50       tination  directory.   This  destination  directory is specified by the
51       environment variable REDIR_DESTDIR or DESTDIR, with REDIR_DESTDIR  tak‐
52       ing precedence.
53
54       You  should  use  make-redir  (not  run-redir)  if  you are redirecting
55       "make", because make-redir performs additional helpful actions.  If you
56       pass  DESTDIR=something as a parameter, it automatically sets the DEST‐
57       DIR environment variable.  The make-redir command also sets  the  over‐
58       rides  INSTALL="install -p" CP="cp -p" MKDIR="mkdir" MKDIR_P="mkdir -p"
59       mkdir_p="mkdir -p" LN="ln", so that makefiles with  absolute  pathnames
60       of  common  commands will work correctly and preserve timestamps.  (Add
61       overrides on the command line if these are not acceptable.)  By default
62       make-redir  also  sets special SHELL and MAKESHELL values, so that even
63       invocations with a full path name (like /bin/cp) will be redirected  if
64       they begin a line in a makefile.
65
66       Internally,  make-redir runs "make" using run-redir; run-redir does the
67       actual file redirection.  Use run-redir if you need  to  redirect  file
68       writes without first running "make".
69
70       The  auto-destdir  tools  are designed to "just work" given the limita‐
71       tions described below (in particular, only  "wrapped"  commands  should
72       read/write files that must be redirected).  Filenames will not be redi‐
73       rected twice, so it's safe to use run-redir and  make-redir  even  with
74       makefiles  that do support DESTDIR.  If a write is redirected, all par‐
75       ent directories are first automatically created if they  exist  in  the
76       unredirected  filesystem.  This means that an attempt to "cp" some file
77       to  /usr/bin/xyz  will  create  the  directory  "$DESTDIR/usr/bin"   if
78       /usr/bin  already exists, then copy the file to "$DESTDIR/usr/bin/xyz".
79       If a file is being read, a  file  written  inside  $DESTDIR  will  have
80       precedence  over a file that isn't; this nicely simulates the overwrit‐
81       ing of an old file with a new one.  Since make-redir and run-redir  set
82       the  PATH, they will work even if there are many levels of indirection,
83       and even if they use tools other than make (as  long  as  they  end  up
84       eventually invoking the usual commands such as cp and install).
85
86       If  the "make install" works except that it uses a few "unwrapped" com‐
87       mands, consider using the "-k" ("--keep-going") option, so  that  make-
88       redir will keep going instead of failing on an error.  Then, re-perform
89       the actions that would have failed.  Alternatively, you may be able  to
90       perform  one or two actions "by hand" first, and then invoke make-redir
91       to do the rest.
92
93       The programs make-redir and run-redir can be run as root, but  in  most
94       cases,  you  should  not  run  them  as root.  The installation program
95       itself may require root to run (e.g., to set ownership), even if  files
96       are  written to DESTDIR.  If you just want to record permissions (e.g.,
97       chown, chmod), consider also using "fakeroot" to  record  the  intended
98       permissions  instead of running as root.  When creating an rpm package,
99       the permission-setting is typically irrelevant, since in most cases  it
100       is  the rpm spec file that will ultimately set the permissions.  If the
101       permission-setting is irrelevant, you may want  to  run  make-redir  as
102       non-root and pass it the "-k" ("--keep-going") option so that make will
103       keep going instead of failing on an error.
104
105       A file/directory is not redirected if its full  pathname  begins  as  a
106       typical  user  directory  (specifically /home/ or /Users/) or temporary
107       directory (/tmp/, /usr/tmp/, /var/tmp/, or /opt/tmp/).   A  file/direc‐
108       tory  is  redirected  if its full pathname begins with a directory name
109       that is normally  privileged:  /usr/,  /etc/,  /boot/,  /bin*/,  /dev/,
110       /lib*/, /opt/, /sbin*/, /selinux/, /srv/, /sys/, and /var/.  If neither
111       case applies, it is not redirected.
112
113       You can override some values, though  few  will  need  to  do  so.   By
114       default,  make-redir invokes "make"; this can be changed by setting the
115       environment variable REALMAKE to  whatever  "make"  program  should  be
116       invoked instead.  By default, make-redir sets a special SHELL value (so
117       it can override commands that give full pathnames); this special  value
118       does  some  processing and then invokes /bin/bash.  Set the environment
119       variable REALMAKESHELL to the name of the shell to use after doing this
120       filtering if you do not want it to use /bin/bash.  You can disable this
121       filtering entirely by setting the make variables SHELL and MAKESHELL on
122       the  command  line to the name of the shell program to use (e.g., make-
123       redir SHELL=/bin/sh MAKESHELL=/bin/sh); if make can  be  called  recur‐
124       sively,  also set the environment variables with the same name.  If you
125       do not want any makefile variable overrides, set the environment  vari‐
126       able DESTDIR and use "run-redir make ..." instead.
127

EXAMPLES

129       The  following  configures, builds, and then "installs" a program.  The
130       "installed"  files  will  descend  from  /tmp/stuff,  e.g.,   directory
131       /tmp/stuff/usr/bin   will  contain  files  that  were  intended  to  be
132       installed in /usr/bin:
133
134               ./configure --prefix=/usr
135               make
136               make-redir DESTDIR=/tmp/stuff install
137
138       For example, if "cp myprogram /usr/bin" is one of the  instructions  in
139       the makefile for the "install" target, then (presuming /usr/bin exists)
140       it will actually perform the equivalent of:
141               /bin/mkdir -p /tmp/stuff/usr/bin
142               /bin/cp myprogram /tmp/stuff/usr/bin
143
144       You can use make-redir inside an RPM .spec file.   Be  sure  to  "Buil‐
145       dRequires"  auto-destdir, and set the DESTDIR to be the RPM build root,
146       as follows in the spec file:
147
148               ...
149               BuildRequires: auto-destdir
150               ...
151               %install
152               rm -rf "$RPM_BUILD_ROOT"
153               make-redir DESTDIR="$RPM_BUILD_ROOT" install
154
155       Some RPM spec  authors  may  prefer  to  use  %{buildroot}  instead  of
156       $RPM_BUILD_ROOT.
157

IMPLEMENTATION APPROACH

159       The  auto-destdir  tools  (including  run-redir and make-redir) work by
160       modifying the PATH environment variable and providing a few  "wrappers"
161       that  override  typical  installation commands like "cp" and "install".
162       These commands with wrappers, called "wrapped commands", use the values
163       of  $REDIR_DESTDIR  or $DESTDIR (with REDIR_DESTDIR taking precedence),
164       and will redirect reads and writes of files under  certain  conditions.
165       The  "redirection" comes by simply changing the name of the file source
166       or destination, and then calling the "real" program being wrapped.  The
167       internal  function  redir_name  takes the original filename, and how it
168       will be used (e.g., for reading  or  writing),  and  determines  if  it
169       should  be renamed (and if so, to what).  In addition, make-redir redi‐
170       rects all shell commands (using SHELL and a special wrapper  for  shell
171       commands);  any command that begins with the prefix /usr/ or /usr/bin/,
172       followed by a wrapped command's name, will have that prefix removed.
173
174       This may seem to be an odd implementation approach,  but  it  has  many
175       advantages.   The  "obvious" way to implement redirection is to manipu‐
176       late LD_PRELOAD and wrap the open() and  creat()  calls.   But  as  the
177       fakeroot(1)  documentation  notes,  this approach creates "problems, as
178       demonstrated by the libtricks package.  This package wrapped many  more
179       functions,  and  tried  to  do a lot more than fakeroot.  It turned out
180       that a minor upgrade of libc (from one where the stat() function didn't
181       use  open()  to one with a stat() function that did (in some cases) use
182       open()), would cause unexplainable segfaults (that is, the libc6 stat()
183       called  the  wrapped  open(),  which  would then call the libc6 stat(),
184       etc).  Fixing them wasn't all that easy, but once fixed, it was just  a
185       matter  of  time  before  another  function  started to use open()...".
186       LD_PRELOAD-based  approaches  fail  on  commands  that  are  statically
187       linked,  and  the commands usually used for installation (e.g., cp) are
188       often  statically  linked.   Others  tools  (like  fakeroot)  also  use
189       LD_PRELOAD,  so using LD_PRELOAD means that these other tools cannot be
190       used simultaneously.  Other "obvious" approaches are to  use  ptrace(),
191       or  redirect file writes using the kernel (e.g. with chroot), but these
192       turn   out   to   have   many   problems   too,   as    discussed    in
193       <http://www.dwheeler.com/essays/automating-destdir.html>.
194
195       In contrast, since run-redir and make-redir don't manipulate LD_PRELOAD
196       to manipulate open() or creat(), they are immune to deadlocks caused by
197       changes  in  the  C  library, work with statically-linked commands, and
198       work with tools like fakeroot.  No special privileges are needed to run
199       these  programs,  nor  do they require setuid installation, eliminating
200       many security concerns (compared to a chroot, setuid,  or  kernel-based
201       approach).   No  kernel  driver is used, nor is ptrace() used, so these
202       tools should be fairly portable to any POSIX-based  system  with  bash.
203       All  work happens very quickly, since nearly all commands simply run as
204       usual, and even the wrapped commands merely run a  short  shell  script
205       before running as usual.  The implementation is entirely a set of small
206       bash scripts with trivial dependencies, so the auto-destdir  tools  can
207       be  easily  included  in  a package's build process without slowing the
208       build process down (even if the build  process  requires  re-installing
209       dependencies on each run).
210
211       In  practice, this approach is sufficient to automate DESTDIR for typi‐
212       cal installation scripts.  Typical "make  install"  programs,  even  if
213       they  are  deeply  recursive,  only use a very few commands to actually
214       write files to their final locations in trusted directories,  and  usu‐
215       ally  invoke  commands without using the full pathnames of the commands
216       (e.g., "install -p myprog /usr/bin").  As a result,  typical  installa‐
217       tion scripts are easily redirected by this program.
218
219       More information about various implementation alternatives, their pros,
220       and their  cons,  is  here:  http://www.dwheeler.com/essays/automating-
221       destdir.html
222

LIMITATIONS

224       Only  wrapped programs will be redirected to the new destination direc‐
225       tory; any  other  program  will  not  be  redirected.   Currently,  the
226       "wrapped" commands are (alphabetically):
227
228              chmod, cp, install, ln, mkdir, mv, and touch.
229
230       If  the  makefile  invokes  wrapped  programs  using absolute pathnames
231       (e.g., calling "/bin/cp" instead of "cp"), then  the  command  will  be
232       redirected  anyway  if it is the absolutely first command at the begin‐
233       ning of a line in a makefile and is invoked directly  or  via  makefile
234       variable expansions like $(CP).  Absolute pathnames at the beginning of
235       a line are handled because make-redir sets SHELL  to  a  special  shell
236       wrapper that handles this particular case.  However, absolute pathnames
237       for wrapped programs are not handled in any other place (doing so would
238       require the shell wrapper to re-implement shell parsing); e.g., program
239       names stored in shell variable  expansions  will  not  be  intercepted.
240       Thus,  makefile  lines like "cd X; /usr/bin/install myprogram /usr/bin"
241       will not be redirected.  In these cases,  you  should  change  absolute
242       pathname  invocations like /usr/bin/install to unnamed-path invocations
243       like install.
244
245       Capabilities built into the shell are not redirected, including I/O re‐
246       direction  such  as  "<",  ">",  and  ">>".   This  means that "cat x >
247       /usr/bin/program" will not be redirected; use "cp  x  /usr/bin/program"
248       or  similar  instead.   Similarly,  "test"  and "[" (which are normally
249       shell built-ins) aren't redirected.  Changing this would require  auto-
250       destdir to re-implement a full shell.
251
252       The  wrappers presume GNU command line options, and will not understand
253       other options even if the underlying tool would.  In practice, this  is
254       almost  always  irrelevant;  "make install" typically only uses highly-
255       portable commands to copy files and make directories, and does not  use
256       unusual options.
257
258       The  "rm"  and "rmdir" commands are not redirected simply because these
259       are unlikely commands during a "make install".  Similarly, there are no
260       "whiteout"  files  to  record  when  a  file  is removed from $DESTDIR,
261       because this does not seem useful for "make install".
262
263       In cases where this wrapper is not sufficient, you may need  to  modify
264       the installation script.  If significant modifications to the installa‐
265       tion script are required, you may as well add DESTDIR support  directly
266       to  the  installation  script.   The auto-destdir tools are designed to
267       handle common cases, so that people can focus their time on  the  cases
268       that need special handling.
269
270

GUIDANCE FOR SOFTWARE CREATORS

272       When writing the commands for "make install":
273
274       ·  Only  use  the  wrapped commands to install files and directories in
275          their final positions (e.g., install, cp, ln, and mkdir).
276
277       ·  Don't   use   full   pathnames   of    wrapped    commands    (e.g.,
278          "/usr/bin/install"); use their basename (e.g., "install").
279
280       ·  Don't  use  redirection  (e.g., ">") to write to files in privileged
281          directories; use redirection to write to a "local"  file  (typically
282          during the build process), and then use "install" or "cp" to install
283          the result.
284
285       Supporting DESTDIR is still a good idea.
286

BUGS

288       As noted above, only "wrapped" commands are redirected.  Some  commands
289       that  aren't  wrapped,  and  perhaps  should be, include: install-info,
290       ldconfig, rm, rmdir,  chown,  chgrp,  and  libtool.   For  permissions-
291       related  commands  (like  chmod, chown, and chgrp), consider also using
292       fakeroot or make's "-k" option.
293
294       Options with optional arguments cannot accept arguments that begin with
295       "-", due to limitations of getopt(1).
296
297       cp's --parents argument is supported, but it won't work if the original
298       files are being copied from a redirected location.
299
300       When running make-redir, the makefile's SHELL value is ignored.  If you
301       need  to  use  the SHELL value provided by the makefile, run make-redir
302       and provide "SHELL=..." with the correct shell value.  If you do  this,
303       wrapped commands must never be invoked using their full pathname.
304

AUTHOR

306       David A. Wheeler
307

REPORTING BUGS

309       Report bugs to <dwheeler, at, dwheeler dot com>
310
312       Copyright  ©  2009  Institute for Defense Analyses.  Released using the
313       MIT license.
314       This is free software: you are free  to  change  and  redistribute  it.
315       There is NO WARRANTY, to the extent permitted by law.
316

SEE ALSO

318       run-redir(1)
319
320
321
322
323auto-destdir                      August 2009                    MAKE-REDIR(1)
Impressum