1CH-IMAGE(1) Charliecloud CH-IMAGE(1)
2
6 ch-image - Build and manage images; completely unprivileged
7
9 $ ch-image [...] build [-t TAG] [-f DOCKERFILE] [...] CONTEXT
10 $ ch-image [...] delete IMAGE_REF
11 $ ch-image [...] list
12 $ ch-image [...] pull [...] IMAGE_REF [IMAGE_DIR]
13 $ ch-image [...] push [--image DIR] IMAGE_REF [DEST_REF]
14 $ ch-image [...] storage-path
15 $ ch-image { --help | --version | --dependencies }
16
18 ch-image is a tool for building and manipulating container images, but
19 not running them (for that you want ch-run). It is completely unprivi‐
20 leged, with no setuid/setgid/setcap helpers.
21 Options that print brief information and then exit:
23 -h, --help
25 Print help and exit successfully.
26 --dependencies
28 Report dependency problems on standard output, if any, and
29 exit. If all is well, there is no output and the exit is suc‐
30 cessful; in case of problems, the exit is unsuccessful.
31 --version
33 Print version number and exit successfully.
34 Common options placed before the sub-command:
36 --no-cache
38 Download everything needed, ignoring the cache.
39 -s, --storage DIR
41 Set the storage directory (see below for important details).
42 --tls-no-verify
44 Don’t verify TLS certificates of the repository. (Do not use
45 this option unless you understand the risks.)
46 -v, --verbose
48 Print extra chatter; can be repeated.
49
51 ch-image maintains state using normal files and directories, including
52 unpacked container images, located in its storage directory. There is
53 no notion of storage drivers, graph drivers, etc., to select and/or
54 configure. In descending order of priority, this directory is located
55 at:
56 -s, --storage DIR
58 Command line option.
59 $CH_IMAGE_STORAGE
61 Environment variable.
62 /var/tmp/$USER/ch-image
64 Default.
65 The storage directory can reside on any filesystem. However, it con‐
67 tains lots of small files and metadata traffic can be intense. For
68 example, the Charliecloud test suite uses approximately 400,000 files
69 and directories in the storage directory as of this writing. Place it
70 on a filesystem appropriate for this; tmpfs’es such as /var/tmp are a
71 good choice if you have enough RAM (/tmp is not recommended because
72 ch-run bind-mounts it into containers by default).
73 While you can currently poke around in the storage directory and find
75 unpacked images runnable with ch-run, this is not a supported use case.
76 The supported workflow uses ch-builder2tar or ch-builder2squash to
77 obtain a packed image; see the tutorial for details.
78 WARNING:
80 Network filesystems, especially Lustre, are typically bad choices
81 for the storage directory. This is a site-specific question and your
82 local support will likely have strong opinions.
83
85 build
86 Build an image from a Dockerfile and put it in the storage directory.
87 Use ch-run(1) to execute RUN instructions. Note that FROM implicitly
88 pulls the base image if needed, so you may want to read about the pull
89 subcommand below as well.
90 Required argument:
92 CONTEXT
94 Path to context directory; this is the root of COPY and ADD
95 instructions in the Dockerfile.
96 Options:
98 -b, --bind SRC[:DST]
100 Bind-mount host directory SRC at container directory DST dur‐
101 ing RUN instructions. Can be repeated; the default destina‐
102 tion if DST is omitted is /mnt/0, /mnt/1, etc.
103 Note: This applies only to RUN instructions. Other instruc‐
105 tions that modify the image filesystem, e.g. COPY, can only
106 access host files from the context directory.
107 --build-arg KEY[=VALUE]
109 Set build-time variable KEY defined by ARG instruction to
110 VALUE. If VALUE not specified, use the value of environment
111 variable KEY.
112 -f, --file DOCKERFILE
114 Use DOCKERFILE instead of CONTEXT/Dockerfile. Specify a sin‐
115 gle hyphen (-) to use standard input; note that in this case,
116 the context directory is still provided, which matches docker
117 build -f - behavior.
118 --force
120 Inject the unprivileged build workarounds; see discussion
121 later in this section for details on what this does and when
122 you might need it. If a build fails and ch-image thinks
123 --force would help, it will suggest it.
124 -n, --dry-run
126 Don’t actually execute any Dockerfile instructions.
127 --no-force-detect
129 Don’t try to detect if the workarounds in --force would help.
130 --parse-only
132 Stop after parsing the Dockerfile.
133 -t, -tag TAG
135 Name of image to create. If not specified, use the final com‐
136 ponent of path CONTEXT. Append :latest if no colon present.
137 ch-image is a fully unprivileged image builder. It does not use any
139 setuid or setcap helper programs, and it does not use configuration
140 files /etc/subuid or /etc/subgid. This contrasts with the “rootless” or
141 “fakeroot” modes of some competing builders, which do require privi‐
142 leged supporting code or utilities.
143 This approach does yield some quirks. We provide built-in workarounds
145 that should mostly work (i.e., --force), but it can be helpful to
146 understand what is going on.
147 ch-image executes all instructions as the normal user who invokes it.
149 For RUN, this is accomplished with ch-run -w --uid=0 --gid=0 (and some
150 other arguments), i.e., your host EUID and EGID both mapped to zero
151 inside the container, and only one UID (zero) and GID (zero) are avail‐
152 able inside the container. Under this arrangement, processes running in
153 the container for each RUN appear to be running as root, but many priv‐
154 ileged system calls will fail without the workarounds described below.
155 This affects any fully unprivileged container build, not just Char‐
156 liecloud.
157 The most common time to see this is installing packages. For example,
159 here is RPM failing to chown(2) a file, which makes the package update
160 fail:
161 Updating : 1:dbus-1.10.24-13.el7_6.x86_64 2/4
163 Error unpacking rpm package 1:dbus-1.10.24-13.el7_6.x86_64
164 error: unpacking of archive failed on file /usr/libexec/dbus-1/dbus-daemon-launch-helper;5cffd726: cpio: chown
165 Cleanup : 1:dbus-libs-1.10.24-12.el7.x86_64 3/4
166 error: dbus-1:1.10.24-13.el7_6.x86_64: install failed
167 This one is (ironically) apt-get failing to drop privileges:
169 E: setgroups 65534 failed - setgroups (1: Operation not permitted)
171 E: setegid 65534 failed - setegid (22: Invalid argument)
172 E: seteuid 100 failed - seteuid (22: Invalid argument)
173 E: setgroups 0 failed - setgroups (1: Operation not permitted)
174 By default, nothing is done to avoid these problems, though ch-image
176 does try to detect if the workarounds could help. --force activates the
177 workarounds: ch-image injects extra commands to intercept these system
178 calls and fake a successful result, using fakeroot(1). There are three
179 basic steps:
180 1. After FROM, analyze the image to see what distribution it con‐
182 tains, which determines the specific workarounds.
183 2. Before the user command in the first RUN instruction where the
185 injection seems needed, install fakeroot(1) in the image, if one
186 is not already installed, as well as any other necessary initial‐
187 ization commands. For example, we turn off the apt sandbox (for
188 Debian Buster) and configure EPEL but leave it disabled (for Cen‐
189 tOS/RHEL).
190 3. Prepend fakeroot to RUN instructions that seem to need it, e.g.
192 ones that contain apt, apt-get, dpkg for Debian derivatives and
193 dnf, rpm, or yum for RPM-based distributions.
194 The details are specific to each distribution. ch-image analyzes image
196 content (e.g., grepping /etc/debian_version) to select a configuration;
197 see lib/fakeroot.py for details. ch-image prints exactly what it is
198 doing.
199 delete
201 Delete the image described by the image reference IMAGE_REF from the
202 storage directory.
203 pull
205 Pull the image described by the image reference IMAGE_REF from a repos‐
206 itory to the local filesystem. See the FAQ for the gory details on
207 specifying image references.
208 Destination:
210 IMAGE_DIR
212 If specified, place the unpacked image at this path; it is
213 then ready for use by ch-run or other tools. The storage
214 directory will not contain a copy of the image, i.e., it is
215 only unpacked once.
216 Options:
218 --last-layer N
220 Unpack only N layers, leaving an incomplete image. This
221 option is intended for debugging.
222 --parse-only
224 Parse IMAGE_REF, print a parse report, and exit successfully
225 without talking to the internet or touching the storage
226 directory.
227 This script does a fair amount of validation and fixing of the layer
229 tarballs before flattening in order to support unprivileged use despite
230 image problems we frequently see in the wild. For example, device files
231 are ignored, and file and directory permissions are increased to a min‐
232 imum of rwx------ and rw------- respectively. Note, however, that sym‐
233 links pointing outside the image are permitted, because they are not
234 resolved until runtime within a container.
235 The following metadata in the pulled image is retained; all other meta‐
237 data is currently ignored. (If you have a need for additional metadata,
238 please let us know!)
239 · Current working directory set with WORKDIR is effective in down‐
241 stream Dockerfiles.
242 · Environment variables set with ENV are effective in downstream
244 Dockerfiles and also written to /ch/environment for use in ch-run
245 --set-env.
246 · Mount point directories specified with VOLUME are created in the
248 image if they don’t exist, but no other action is taken.
249 Note that some images (e.g., those with a “version 1 manifest”) do not
251 contain metadata. A warning is printed in this case.
252 push
254 Push the image described by the image reference IMAGE_REF from the
255 local filesystem to a repository. See the FAQ for the gory details on
256 specifying image references.
257 Because Charliecloud is fully unprivileged, the owner and group of
259 files in its images are not meaningful in the broader ecosystem. Thus,
260 when pushed, everything in the image is flattened to user:group
261 root:root. Also, setuid/setgid bits are removed, to avoid surprises if
262 the image is pulled by a privileged container implementation.
263 Destination:
265 DEST_REF
267 If specified, use this as the destination image reference,
268 rather than IMAGE_REF. This lets you push to a repository
269 without permanently adding a tag to the image.
270 Options:
272 --image DIR
274 Use the unpacked image located at DIR rather than an image in
275 the storage directory named IMAGE_REF.
276 storage-path
278 Print the storage directory path and exit.
279
281 ch-image is an independent implementation and shares no code with other
282 Dockerfile interpreters. It uses a formal Dockerfile parsing grammar
283 developed from the Dockerfile reference documentation and miscellaneous
284 other sources, which you can examine in the source code.
285 We believe this independence is valuable for several reasons. First, it
287 helps the community examine Dockerfile syntax and semantics critically,
288 think rigorously about what is really needed, and build a more robust
289 standard. Second, it yields disjoint sets of bugs (note that Podman,
290 Buildah, and Docker all share the same Dockerfile parser). Third,
291 because it is a much smaller code base, it illustrates how Dockerfiles
292 work more clearly. Finally, it allows straightforward extensions if
293 needed to support scientific computing.
294 ch-image tries hard to be compatible with Docker and other inter‐
296 preters, though as an independent implementation, it is not bug-compat‐
297 ible.
298 This section describes differences from the Dockerfile reference that
300 we expect to be approximately permanent. For an overview of features we
301 have not yet implemented and our plans, see our road map on GitHub.
302 Plain old bugs are in our GitHub issues.
303 None of these are set in stone. We are very interested in feedback on
305 our assessments and open questions. This helps us prioritize new fea‐
306 tures and revise our thinking about what is needed for HPC containers.
307 Context directory
309 The context directory is bind-mounted into the build, rather than
310 copied like Docker. Thus, the size of the context is immaterial, and
311 the build reads directly from storage like any other local process
312 would. However, you still can’t access anything outside the context
313 directory.
314 Authentication
316 ch-image can authenticate using one-time passwords, e.g. those provided
317 by a security token. Unlike docker login, it does not assume passwords
318 are persistent.
319 Environment variables
321 Variable substitution happens for all instructions, not just the ones
322 listed in the Dockerfile reference.
323 ARG and ENV cause cache misses upon definition, in contrast with Docker
325 where these variables miss upon use, except for certain cache-excluded
326 variables that never cause misses, listed below.
327 Like Docker, ch-image pre-defines the following proxy variables, which
329 do not require an ARG instruction. However, they are available if the
330 same-named environment variable is defined; --build-arg is not
331 required. Changes to these variables do not cause a cache miss.
332 HTTP_PROXY
334 http_proxy
335 HTTPS_PROXY
336 https_proxy
337 FTP_PROXY
338 ftp_proxy
339 NO_PROXY
340 no_proxy
341 The following variables are also pre-defined:
343 PATH=/ch/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
345 TAR_OPTIONS=--no-same-owner
346 Note that ARG and ENV have different syntax despite very similar seman‐
348 tics.
349 COPY
351 Especially for people used to UNIX cp(1), the semantics of the Docker‐
352 file COPY instruction can be confusing.
353 Most notably, when a source of the copy is a directory, the contents of
355 that directory, not the directory itself, are copied. This is docu‐
356 mented, but it’s a real gotcha because that’s not what cp(1) does, and
357 it means that many things you can do in one cp(1) command require mul‐
358 tiple COPY instructions.
359 Also, the reference documentation is incomplete. In our experience,
361 Docker also behaves as follows; ch-image does the same in an attempt to
362 be bug-compatible.
363 1. You can use absolute paths in the source; the root is the context
365 directory.
366 2. Destination directories are created if they don’t exist in the fol‐
368 lowing situations:
369 1. If the destination path ends in slash. (Documented.)
371 2. If the number of sources is greater than 1, either by wildcard or
373 explicitly, regardless of whether the destination ends in slash.
374 (Not documented.)
375 3. If there is a single source and it is a directory. (Not docu‐
377 mented.)
378 3. Symbolic links behave differently depending on how deep in the
380 copied tree they are. (Not documented.)
381 1. Symlinks at the top level — i.e., named as the destination or the
383 source, either explicitly or by wildcards — are dereferenced.
384 They are followed, and whatever they point to is used as the des‐
385 tination or source, respectively.
386 2. Symlinks at deeper levels are not dereferenced, i.e., the symlink
388 itself is copied.
389 4. If a directory appears at the same path in source and destination,
391 and is at the 2nd level or deeper, the source directory’s metadata
392 (e.g., permissions) are copied to the destination directory. (Not
393 documented.)
394 5. If an object appears in both the source and destination, and is at
396 the 2nd level or deeper, and is of different types in the source and
397 destination, then the source object will overwrite the destination
398 object. (Not documented.) For example, if /tmp/foo/bar is a regular
399 file, and /tmp is the context directory, then the following Docker‐
400 file snippet will result in a file in the container at /foo/bar
401 (copied from /tmp/foo/bar); the directory and all its contents will
402 be lost.
403 RUN mkdir -p /foo/bar && touch /foo/bar/baz
405 COPY foo /foo
406 We expect the following differences to be permanent:
408 · Wildcards use Python glob semantics, not the Go semantics.
410 · COPY --chown is ignored, because it doesn’t make sense in an unprivi‐
412 leged build.
413 Features we do not plan to support
415 · Parser directives are not supported. We have not identified a need
416 for any of them.
417 · EXPOSE: Charliecloud does not use the network namespace, so con‐
419 tainerized processes can simply listen on a host port like other
420 unprivileged processes.
421 · HEALTHCHECK: This instruction’s main use case is monitoring server
423 processes rather than applications. Also, implementing it requires a
424 container supervisor daemon, which we have no plans to add.
425 · MAINTAINER is deprecated.
427 · STOPSIGNAL requires a container supervisor daemon process, which we
429 have no plans to add.
430 · USER does not make sense for unprivileged builds.
432 · VOLUME: This instruction is not currently supported. Charliecloud has
434 good support for bind mounts; we anticipate that it will continue to
435 focus on that and will not introduce the volume management features
436 that Docker has.
437
439 CH_LOG_FILE
440 If set, append log chatter to this file, rather than standard error.
441 This is useful for debugging situations where standard error is con‐
442 sumed or lost.
443 Also sets verbose mode if not already set (equivalent to --verbose).
445 CH_LOG_FESTOON
447 If set, prepend PID and timestamp to logged chatter.
448
450 build
451 Build image bar using ./foo/bar/Dockerfile and context directory
452 ./foo/bar:
453 $ ch-image build -t bar -f ./foo/bar/Dockerfile ./foo/bar
455 [...]
456 grown in 4 instructions: bar
457 Same, but infer the image name and Dockerfile from the context direc‐
459 tory path:
460 $ ch-image build ./foo/bar
462 [...]
463 grown in 4 instructions: bar
464 Build using humongous vendor compilers you want to bind-mount instead
466 of installing into a layer:
467 $ ch-image build --bind /opt/bigvendor:/opt .
469 $ cat Dockerfile
470 FROM centos:7
471 RUN /opt/bin/cc hello.c
473 #COPY /opt/lib/*.so /usr/local/lib # fail: COPY doesn't bind mount
474 RUN cp /opt/lib/*.so /usr/local/lib # possible workaround
475 RUN ldconfig
476 pull
478 Download the Debian Buster image and place it in the storage directory:
479 $ ch-image pull debian:buster
481 pulling image: debian:buster
482 manifest: downloading
484 layer 1/1: d6ff36c: downloading
485 layer 1/1: d6ff36c: listing
486 validating tarball members
487 resolving whiteouts
488 flattening image
489 layer 1/1: d6ff36c: extracting
490 done
491 Same, except place the image in /tmp/buster:
493 $ ch-image pull debian:buster /tmp/buster
495 [...]
496 $ ls /tmp/buster
497 bin dev home lib64 mnt proc run srv tmp var
498 boot etc lib media opt root sbin sys usr
499 push
501 Push a local image to the registry example.com:5000 at path /foo/bar
502 with tag latest. Note that in this form, the local image must be named
503 to match that remote reference.
504 $ ch-image push example.com:5000/foo/bar:latest
506 pushing image: example.com:5000/foo/bar:latest
507 layer 1/1: gathering
508 layer 1/1: preparing
509 preparing metadata
510 starting upload
511 layer 1/1: a1664c4: checking if already in repository
512 layer 1/1: a1664c4: not present, uploading
513 config: 89315a2: checking if already in repository
514 config: 89315a2: not present, uploading
515 manifest: uploading
516 cleaning up
517 done
518 Same, except use local image alpine:3.9. In this form, the local image
520 name does not have to match the destination reference.
521 $ ch-image push alpine:3.9 example.com:5000/foo/bar:latest
523 pushing image: alpine:3.9
524 destination: example.com:5000/foo/bar:latest
525 layer 1/1: gathering
526 layer 1/1: preparing
527 preparing metadata
528 starting upload
529 layer 1/1: a1664c4: checking if already in repository
530 layer 1/1: a1664c4: not present, uploading
531 config: 89315a2: checking if already in repository
532 config: 89315a2: not present, uploading
533 manifest: uploading
534 cleaning up
535 done
536 Same, except use unpacked image located at /var/tmp/image rather than
538 an image in ch-image storage. (Also, the sole layer is already present
539 in the remote registry, so we don’t upload it again.)
540 $ ch-image push --image /var/tmp/image example.com:5000/foo/bar:latest
542 pushing image: example.com:5000/foo/bar:latest
543 image path: /var/tmp/image
544 layer 1/1: gathering
545 layer 1/1: preparing
546 preparing metadata
547 starting upload
548 layer 1/1: 892e38d: checking if already in repository
549 layer 1/1: 892e38d: already present
550 config: 546f447: checking if already in repository
551 config: 546f447: not present, uploading
552 manifest: uploading
553 cleaning up
554 done
555
557 If Charliecloud was obtained from your Linux distribution, use your
558 distribution’s bug reporting procedures.
559 Otherwise, report bugs to: <https://github.com/hpc/charliecloud/issues>
561
563 charliecloud(7)
564 Full documentation at: <https://hpc.github.io/charliecloud>
566
568 2014–2020, Triad National Security, LLC
5690.22 2021-03-08 00:00 Coordinated Universal Time CH-IMAGE(1)