1CH-FROMHOST(1) Charliecloud CH-FROMHOST(1)
2
6 ch-fromhost - Inject files from the host into an image directory, with
7 various magic
8
10 $ ch-fromhost [OPTION ...] [FILE_OPTION ...] IMGDIR
11
13 NOTE:
14 This command is experimental. Features may be incomplete and/or
15 buggy. Please report any issues you find, so we can fix them!
16 Inject files from the host into the Charliecloud image directory
18 IMGDIR.
19 The purpose of this command is to inject files into a container image
21 that are necessary to run the container on a specific host; e.g., GPU
22 libraries that are tied to a specific kernel version. It is not a gen‐
23 eral copy-to-image tool; see further discussion on use cases below. It
24 should be run after ch-tar2dir and before ch-run. After invocation, the
25 image is no longer portable to other hosts.
26 Injection is not atomic; if an error occurs partway through injection,
28 the image is left in an undefined state. Injection is currently imple‐
29 mented using a simple file copy, but that may change in the future.
30 By default, file paths that contain the strings /bin or /sbin are as‐
32 sumed to be executables and placed in /usr/bin within the container.
33 File paths that contain the strings /lib or .so are assumed to be
34 shared libraries and are placed in the first-priority directory re‐
35 ported by ldconfig (see --lib-path below). Other files are placed in
36 the directory specified by --dest.
37 If any shared libraries are injected, run ldconfig inside the container
39 (using ch-run -w) after injection.
40
42 To specify which files to inject
43 -c, --cmd CMD
44 Inject files listed in the standard output of command CMD.
45 -f, --file FILE
47 Inject files listed in the file FILE.
48 -p, --path PATH
50 Inject the file at PATH.
51 --cray-mpi
53 Cray-enable MPICH/OpenMPI installed inside the image. See im‐
54 portant details below.
55 --nvidia
57 Use nvidia-container-cli list (from libnvidia-container) to
58 find executables and libraries to inject.
59 These can be repeated, and at least one must be specified.
61 To specify the destination within the image
63 -d, --dest DST
64 Place files specified later in directory IMGDIR/DST, overrid‐
65 ing the inferred destination, if any. If a file’s destination
66 cannot be inferred and --dest has not been specified, exit
67 with an error. This can be repeated to place files in varying
68 destinations.
69 Additional arguments
71 --lib-path
72 Print the guest destination path for shared libraries in‐
73 ferred as described above.
74 --no-ldconfig
76 Don’t run ldconfig even if we appear to have injected shared
77 libraries.
78 -h, --help
80 Print help and exit.
81 -v, --verbose
83 List the injected files.
84 --version
86 Print version and exit.
87
89 This command does a lot of heuristic magic; while it can copy arbitrary
90 files into an image, this usage is discouraged and prone to error. Here
91 are some use cases and the recommended approach:
92 1. I have some files on my build host that I want to include in the im‐
94 age. Use the COPY instruction within your Dockerfile. Note that
95 it’s OK to build an image that meets your specific needs but isn’t
96 generally portable, e.g., only runs on specific micro-architectures
97 you’re using.
98 2. I have an already built image and want to install a program I com‐
100 piled separately into the image. Consider whether a building a new
101 derived image with a Dockerfile is appropriate. Another good option
102 is to bind-mount the directory containing your program at run time.
103 A less good option is to cp(1) the program into your image, because
104 this permanently alters the image in a non-reproducible way.
105 3. I have some shared libraries that I need in the image for function‐
107 ality or performance, and they aren’t available in a place where I
108 can use COPY. This is the intended use case of ch-fromhost. You can
109 use --cmd, --file, and/or --path to put together a custom solution.
110 But, please consider filing an issue so we can package your func‐
111 tionality with a tidy option like --cray-mpi or --nvidia.
112
114 The implementation of --cray-mpi is messy, foul smelling, and brittle.
115 It replaces or overrides the MPICH or OpenMPI libraries installed in
116 the container. Users should be aware of the following.
117 1. Containers must have the following software installed:
119 a. Corresponding open source MPI implementation. (MPICH and
121 OpenMPI.)
122 b. PatchELF with our patches. Use the shrink-soname branch. (MPICH
124 only.)
125 c. libgfortran.so.3, because Cray’s libmpi.so.12 links to it. (MPICH
127 only.)
128 2. Applications must be dynamically linked to libmpi.so.12 (not e.g.
130 libmpich.so.12).
131 a. How to configure MPICH to accomplish this is not yet clear to us;
133 test/Dockerfile.mpich does it, while the Debian packages do not.
134 (MPICH only.)
135 3. An ABI compatible module for the given MPI implementation must be
137 loaded when ch-fromhost is invoked.
138 a. Load the cray-mpich-abi module. (MPICH only.)
140 b. We recommend loading the module of a version as close to what is
142 installed in the image as possible. This OpenMPI install needs to
143 be built such that libmpi contains all needed plugins (as opposed
144 to them being standalone shared libraries). See OpenMPI’s docu‐
145 mentation for how to do this. (OpenMPI only.)
146 4. Tested only for C programs compiled with GCC, and it probably won’t
148 work otherwise. If you’d like to use another compiler or another
149 programming language, please get in touch so we can implement the
150 necessary support.
151 Please file a bug if we missed anything above or if you know how to
153 make the code better.
154
156 Symbolic links are dereferenced, i.e., the files pointed to are in‐
157 jected, not the links themselves.
158 As a corollary, do not include symlinks to shared libraries. These will
160 be re-created by ldconfig.
161 There are two alternate approaches for nVidia GPU libraries:
163 1. Link libnvidia-containers into ch-run and call the library func‐
165 tions directly. However, this would mean that Charliecloud would
166 either (a) need to be compiled differently on machines with and
167 without nVidia GPUs or (b) have libnvidia-containers available
168 even on machines without nVidia GPUs. Neither of these is consis‐
169 tent with Charliecloud’s philosophies of simplicity and minimal
170 dependencies.
171 2. Use nvidia-container-cli configure to do the injecting. This
173 would require that containers have a half-started state, where
174 the namespaces are active and everything is mounted but
175 pivot_root(2) has not been performed. This is not feasible be‐
176 cause Charliecloud has no notion of a half-started container.
177 Further, while these alternate approaches would simplify or eliminate
179 this script for nVidia GPUs, they would not solve the problem for other
180 situations.
181
183 File paths may not contain colons or newlines.
184 ldconfig tends to print stat errors; these are typically non-fatal and
186 occur when trying to probe common library paths. See issue #732.
187
189 Place shared library /usr/lib64/libfoo.so at path /usr/lib/libfoo.so
190 (assuming /usr/lib is the first directory searched by the dynamic
191 loader in the image), within the image /var/tmp/baz and executable
192 /bin/bar at path /usr/bin/bar. Then, create appropriate symlinks to
193 libfoo and update the ld.so cache.
194 $ cat qux.txt
196 /bin/bar
197 /usr/lib64/libfoo.so
198 $ ch-fromhost --file qux.txt /var/tmp/baz
199 Same as above:
201 $ ch-fromhost --cmd 'cat qux.txt' /var/tmp/baz
203 Same as above:
205 $ ch-fromhost --path /bin/bar --path /usr/lib64/libfoo.so /var/tmp/baz
207 Same as above, but place the files into /corge instead (and the shared
209 library will not be found by ldconfig):
210 $ ch-fromhost --dest /corge --file qux.txt /var/tmp/baz
212 Same as above, and also place file /etc/quux at /etc/quux within the
214 container:
215 $ ch-fromhost --file qux.txt --dest /etc --path /etc/quux /var/tmp/baz
217 Inject the executables and libraries recommended by nVidia into the im‐
219 age, and then run ldconfig:
220 $ ch-fromhost --nvidia /var/tmp/baz
222 asking ldconfig for shared library destination
223 /sbin/ldconfig: Can't stat /libx32: No such file or directory
224 /sbin/ldconfig: Can't stat /usr/libx32: No such file or directory
225 shared library destination: /usr/lib64//bind9-export
226 injecting into image: /var/tmp/baz
227 /usr/bin/nvidia-smi -> /usr/bin (inferred)
228 /usr/bin/nvidia-debugdump -> /usr/bin (inferred)
229 /usr/bin/nvidia-persistenced -> /usr/bin (inferred)
230 /usr/bin/nvidia-cuda-mps-control -> /usr/bin (inferred)
231 /usr/bin/nvidia-cuda-mps-server -> /usr/bin (inferred)
232 /usr/lib64/libnvidia-ml.so.460.32.03 -> /usr/lib64//bind9-export (inferred)
233 /usr/lib64/libnvidia-cfg.so.460.32.03 -> /usr/lib64//bind9-export (inferred)
234 [...]
235 /usr/lib64/libGLESv2_nvidia.so.460.32.03 -> /usr/lib64//bind9-export (inferred)
236 /usr/lib64/libGLESv1_CM_nvidia.so.460.32.03 -> /usr/lib64//bind9-export (inferred)
237 running ldconfig
238 Inject the Cray-enabled MPI libraries into the image, and then run ld‐
240 config:
241 $ ch-fromhost --cray-mpi /var/tmp/baz
243 asking ldconfig for shared library destination
244 /sbin/ldconfig: Can't stat /libx32: No such file or directory
245 /sbin/ldconfig: Can't stat /usr/libx32: No such file or directory
246 shared library destination: /usr/lib64//bind9-export
247 found shared library: /usr/lib64/liblustreapi.so.1
248 found shared library: /opt/cray/xpmem/default/lib64/libxpmem.so.0
249 [...]
250 injecting into image: /var/tmp/baz
251 rm -f /var/tmp/openmpi/usr/lib64//bind9-export/libopen-rte.so.40
252 rm -f /var/tmp/openmpi/usr/lib64/bind9-export/libopen-rte.so.40
253 [...]
254 mkdir -p /var/tmp/openmpi/var/opt/cray/alps/spool
255 mkdir -p /var/tmp/openmpi/etc/opt/cray/wlm_detect
256 [...]
257 /usr/lib64/liblustreapi.so.1 -> /usr/lib64//bind9-export (inferred)
258 /opt/cray/xpmem/default/lib64/libxpmem.so.0 -> /usr/lib64//bind9-export (inferred)
259 [...]
260 /etc/opt/cray/wlm_detect/active_wlm -> /etc/opt/cray/wlm_detect
261 running ldconfig
262
264 This command was inspired by the similar Shifter feature that allows
265 Shifter containers to use the Cray Aries network. We particularly ap‐
266 preciate the help provided by Shane Canon and Doug Jacobsen during our
267 implementation of --cray-mpi.
268 We appreciate the advice of Ryan Olson at nVidia on implementing
270 --nvidia.
271
273 If Charliecloud was obtained from your Linux distribution, use your
274 distribution’s bug reporting procedures.
275 Otherwise, report bugs to: https://github.com/hpc/charliecloud/issues
277
279 charliecloud(7)
280 Full documentation at: <https://hpc.github.io/charliecloud>
282
284 2014–2021, Triad National Security, LLC
2850.26 2022-01-24 00:00 UTC CH-FROMHOST(1)