1CH-TEST(1) Charliecloud CH-TEST(1)
2
6 ch-test - Run some or all of the Charliecloud test suite
7
9 $ ch-test [PHASE] [--scope SCOPE] [--pack-fmt FMT] [ARGS]
10
12 Charliecloud comes with a comprehensive test suite that exercises the
13 container workflow itself as well as a few example applications.
14 ch-test coordinates running the test suite.
15 While the CLI has lots of options, the defaults are reasonable, and
17 bare ch-test will give useful results in a few minutes on single-node,
18 internet-connected systems with a few GB available in /var/tmp.
19 The test suite requires a few GB (standard scope) or tens of GB (full
21 scope) of storage for test fixtures:
22 • Builder storage (e.g., layer cache). This goes wherever the builder
24 puts it.
25 • Packed images directory: image tarballs or SquashFS files.
27 • Unpacked images directory. Images are unpacked into and then run from
29 here.
30 • Filesystem permissions directories. These are used to test that the
32 kernel is enforcing permissions correctly. Note that this exercises
33 the kernel, not Charliecloud, and can be omitted from routine Char‐
34 liecloud testing.
35 The first three are created when needed if they don’t exist, while the
37 filesystem permissions fixtures must be created manually, in order to
38 accommodate configurations where sudo is not available via the same lo‐
39 gin path used for running tests.
40 The packed and unpacked image directories specified for testing are
42 volatile. The contents of these directories are deleted before the
43 build and run phases, respectively.
44 In all four cases, when creating directories, only the final path com‐
46 ponent is created. Parent directories must already exist, i.e., ch-test
47 uses the behavior of mkdir rather than mkdir -p.
48 Some of the tests exercise parallel functionality. If ch-test is run on
50 a single node, multiple cores will be used; if in a Slurm allocation,
51 multiple nodes too.
52 The subset of tests to run mostly splits along two key dimensions. The
54 phase is which parts of the workflow to run. Different parts of the
55 workflow can be tested on different systems by copying the necessary
56 artifacts between them, e.g. by building images on one system and run‐
57 ning them on another. The scope allows trading off thoroughness versus
58 time.
59 PHASE must be one of the following:
61 build Image building and associated functionality, with the se‐
63 lected builder.
64 run Running containers and associated functionality. This re‐
66 quires a packed images directory produced by a successful
67 build phase, which can be copied from the build system if
68 it’s not also the run system.
69 examples
71 Example applications. Requires an unpacked images directory
72 produced by a successful run phase.
73 all Execute phases build, run, and examples, in that order.
75 mk-perm-dirs
77 Create the filesystem permissions directories. Requires
78 --perm-dirs.
79 clean Delete automatically-generated test files, and packed and un‐
81 packed image directories.
82 rm-perm-dirs
84 Remove the filesystem permissions directories. Requires
85 --perm-dirs.
86 -f, --file FILE[:TEST]
88 Run the tests in the given file only, which can be an arbi‐
89 trary .bats file, except for test.bats under examples, where
90 you must specify the corresponding Dockerfile or Build file
91 instead. This is somewhat brittle and typically used for de‐
92 velopment or debugging. For example, it does not check
93 whether the pre-requisites of whatever is in the file are
94 satisfied. Often running build and run first is sufficient,
95 but this varies.
96 If TEST is also given, then run only tests with name contain‐
98 ing that string, skipping the others. The separator is a lit‐
99 eral colon. If the string contains shell metacharacters such
100 as space, you’ll need to quote the argument to protect it
101 from the shell.
102 Scope is specified with:
104 -s, --scope SCOPE
106 SCOPE must be one of the following:
107 • quick: Most important subset of workflow. Handy for devel‐
109 opment.
110 • standard: All tested workflow functionality and a selection
112 of more important examples. (Default.)
113 • full: All available tests, including all examples.
115 Image format is specified with:
117 --pack-fmt FMT
119 FMT must be one of the following:
120 • squash-mount or 🐘: SquashFS archive, run directly from the
122 archive using ch-run’s internal SquashFUSE functionality.
123 In this mode, tests that require writing to the image are
124 skipped.
125 • tar-unpack or 📠: Tarball, and the images are unpacked be‐
127 fore running.
128 • squash-unpack or 🎃: SquashFS, and the images are unpacked
130 before running.
131 Default: $CH_TEST_PACK_FMT if set. Otherwise, if mk‐
133 squashfs(1) is available and ch-run was built with libsquash‐
134 fuse support, then squash-mount, else tar-unpack.
135 Additional arguments:
137 -b, --builder BUILDER
139 Image builder to use. See ch-build(1) for how the default is
140 selected.
141 --dry-run
143 Print summary of what would be tested and then exit.
144 -h, --help
146 Print usage and then exit.
147 --img-dir DIR
149 Set unpacked images directory to DIR. In a multi-node alloca‐
150 tion, this directory may not be shared between nodes. De‐
151 fault: $CH_TEST_IMGDIR if set; otherwise /var/tmp/img.
152 --lustre DIR
154 Use DIR for run-phase Lustre tests. Default: CH_TEST_LUS‐
155 TREDIR if set; otherwise skip them.
156 The tests will create, populate, and delete a new subdirec‐
158 tory under DIR, leaving everything else in DIR untouched.
159 --pack-dir DIR
161 Set packed images directory to DIR. Default: $CH_TEST_TARDIR
162 if set; otherwise /var/tmp/pack.
163 --pedantic (yes|no)
165 Some tests require configurations that are very specific
166 (e.g., being a member of at least two groups) or unusual
167 (e.g., sudo to a non-root group). If yes, then fail if the
168 requirement is not met; if no, then skip. The default is yes
169 for CI environments or people listed in README.md, no other‐
170 wise.
171 If yes and sudo seems to be available, implies --sudo.
173 --perm-dir DIR
175 Add DIR to filesystem permission fixture directories; can be
176 specified multiple times. We recommend one such directory per
177 mounted filesystem type whose kernel module you do not trust;
178 e.g., you probably don’t need to test your tmpfses, but
179 out-of-tree filesystems very likely need this.
180 Implies --sudo. Default: CH_TEST_PERMDIRS if set; otherwise
182 skip the filesystem permissions tests.
183 --sudo Enable things that require sudo, such as certain privilege
185 escalation tests and creating/removing the filesystem permis‐
186 sions fixtures. Requires generic sudo capabilities. Note that
187 the Docker builder uses sudo docker even without this option.
188
190 Zero if all tests passed; non-zero if any failed. For setup and tear‐
191 down phases, zero if everything was created or deleted correctly,
192 non-zero otherwise.
193
195 Bats will wait until all descendant processes finish before exiting, so
196 if you get into a failure mode where a test sequence doesn’t clean up
197 all its processes, ch-test will hang.
198
200 Many systems can simply use the defaults. To run the build, run, and
201 examples phases on a single system, without the filesystem permissions
202 tests:
203 $ ch-test
205 ch-test version 0.12
206 ch-run: 0.12 /usr/local/bin/ch-run
208 bats: 0.4.0 /usr/bin/bats
209 tests: /usr/local/libexec/charliecloud/test
210 phase: build run examples
212 scope: standard (default)
213 builder: docker (default)
214 use generic sudo: no (default)
215 unpacked images dir: /var/tmp/img (default)
216 packed images dir: /var/tmp/tar (default)
217 fs permissions dirs: skip (default)
218 checking namespaces ...
220 ok
221 checking builder ...
223 found: /usr/bin/docker 19.03.2
224 bats build.bats build_auto.bats build_post.bats
226 ✓ documentation seems sane
227 ✓ version number seems sane
228 [...]
229 All tests passed.
230 The next example is for a more complex setup like you might find in HPC
232 centers:
233 • Non-default fixture directories.
235 • Non-default scope.
237 • Different build and run systems.
239 • Run the filesystem permissions tests.
241 Output has been omitted.
243 (mybox)$ ssh hpc-admin
245 (hpc-admin)$ ch-test mk-perm-dirs --perm-dir /scratch/$USER/perms \
246 --perm-dir /home/$USER/perms
247 (hpc-admin)$ exit
248 (mybox)$ ch-test build --scope full
249 (mybox)$ scp -r /var/tmp/pack hpc:/scratch/$USER/pack
250 (mybox)$ ssh hpc
251 (hpc)$ salloc -N2
252 (cn001)$ export CH_TEST_TARDIR=/scratch/$USER/pack
253 (cn001)$ export CH_TEST_IMGDIR=/local/tmp
254 (cn001)$ export CH_TEST_PERMDIRS="/scratch/$USER/perms /home/$USER/perms"
255 (cn001)$ export CH_TEST_SCOPE=full
256 (cn001)$ ch-test run
257 (cn001)$ ch-test examples
258
260 If Charliecloud was obtained from your Linux distribution, use your
261 distribution’s bug reporting procedures.
262 Otherwise, report bugs to: https://github.com/hpc/charliecloud/issues
264
266 charliecloud(7)
267 Full documentation at: <https://hpc.github.io/charliecloud>
269
271 2014–2021, Triad National Security, LLC
2720.26 2022-07-20 00:00 UTC CH-TEST(1)