1CH-TEST(1) Charliecloud CH-TEST(1)
2
6 ch-test - Run some or all of the Charliecloud test suite
7
9 $ ch-test [PHASE] [--scope SCOPE] [--pack-fmt FMT] [ARGS]
10
12 Charliecloud comes with a comprehensive test suite that exercises the
13 container workflow itself as well as a few example applications.
14 ch-test coordinates running the test suite.
15 While the CLI has lots of options, the defaults are reasonable, and
17 bare ch-test will give useful results in a few minutes on single-node,
18 internet-connected systems with a few GB available in /var/tmp.
19 The test suite requires a few GB (standard scope) or tens of GB (full
21 scope) of storage for test fixtures:
22 • Builder storage (e.g., layer cache). This goes wherever the builder
24 puts it.
25 • Packed images directory: image tarballs or SquashFS files.
27 • Unpacked images directory. Images are unpacked into and then run from
29 here.
30 • Filesystem permissions directories. These are used to test that the
32 kernel is enforcing permissions correctly. Note that this exercises
33 the kernel, not Charliecloud, and can be omitted from routine Char‐
34 liecloud testing.
35 The first three are created when needed if they don’t exist, while the
37 filesystem permissions fixtures must be created manually, in order to
38 accommodate configurations where sudo is not available via the same lo‐
39 gin path used for running tests.
40 The packed and unpacked image directories specified for testing are
42 volatile. The contents of these directories are deleted before the
43 build and run phases, respectively.
44 In all four cases, when creating directories, only the final path com‐
46 ponent is created. Parent directories must already exist, i.e., ch-test
47 uses the behavior of mkdir rather than mkdir -p.
48 Some of the tests exercise parallel functionality. If ch-test is run on
50 a single node, multiple cores will be used; if in a Slurm allocation,
51 multiple nodes too.
52 The subset of tests to run mostly splits along two key dimensions. The
54 phase is which parts of the workflow to run. Different parts of the
55 workflow can be tested on different systems by copying the necessary
56 artifacts between them, e.g. by building images on one system and run‐
57 ning them on another. The scope allows trading off thoroughness versus
58 time.
59 PHASE must be one of the following:
61 build Image building and associated functionality, with the se‐
63 lected builder.
64 run Running containers and associated functionality. This re‐
66 quires a packed images directory produced by a successful
67 build phase, which can be copied from the build system if
68 it’s not also the run system.
69 examples
71 Example applications. Requires an unpacked images directory
72 produced by a successful run phase.
73 all Execute phases build, run, and examples, in that order.
75 mk-perm-dirs
77 Create the filesystem permissions directories. Requires
78 --perm-dirs.
79 build-images
81 Build images from build phase, without running the associated
82 tests.
83 clean Delete automatically-generated test files, and packed and un‐
85 packed image directories.
86 rm-perm-dirs
88 Remove the filesystem permissions directories. Requires
89 --perm-dirs.
90 -f, --file FILE[:TEST]
92 Run the tests in the given file only, which can be an arbi‐
93 trary .bats file, except for test.bats under examples, where
94 you must specify the corresponding Dockerfile or Build file
95 instead. This is somewhat brittle and typically used for de‐
96 velopment or debugging. For example, it does not check
97 whether the pre-requisites of whatever is in the file are
98 satisfied. Often running build and run first is sufficient,
99 but this varies.
100 If TEST is also given, then run only tests with name contain‐
102 ing that string, skipping the others. The separator is a lit‐
103 eral colon. If the string contains shell metacharacters such
104 as space, you’ll need to quote the argument to protect it
105 from the shell.
106 Scope is specified with:
108 -s, --scope SCOPE
110 SCOPE must be one of the following:
111 • quick: Most important subset of workflow. Handy for devel‐
113 opment.
114 • standard: All tested workflow functionality and a selection
116 of more important examples. (Default.)
117 • full: All available tests, including all examples.
119 Image format is specified with:
121 --pack-fmt FMT
123 FMT must be one of the following:
124 • squash-mount or 🐘: SquashFS archive, run directly from the
126 archive using ch-run’s internal SquashFUSE functionality.
127 In this mode, tests that require writing to the image are
128 skipped.
129 • tar-unpack or 📠: Tarball, and the images are unpacked be‐
131 fore running.
132 • squash-unpack or 🎃: SquashFS, and the images are unpacked
134 before running.
135 Default: $CH_TEST_PACK_FMT if set. Otherwise, if mk‐
137 squashfs(1) is available and ch-run was built with libsquash‐
138 fuse support, then squash-mount, else tar-unpack.
139 Additional arguments:
141 -b, --builder BUILDER
143 Image builder to use. Default: $CH_TEST_BUILDER if set, oth‐
144 erwise ch-image.
145 --dry-run
147 Print summary of what would be tested and then exit.
148 -h, --help
150 Print usage and then exit.
151 --img-dir DIR
153 Set unpacked images directory to DIR. In a multi-node alloca‐
154 tion, this directory may not be shared between nodes. De‐
155 fault: $CH_TEST_IMGDIR if set; otherwise
156 /var/tmp/${USER}.img.
157 --lustre DIR
159 Use DIR for run-phase Lustre tests. Default: CH_TEST_LUS‐
160 TREDIR if set; otherwise skip them.
161 The tests will create, populate, and delete a new subdirec‐
163 tory under DIR, leaving everything else in DIR untouched.
164 --pack-dir DIR
166 Set packed images directory to DIR. Default: $CH_TEST_TARDIR
167 if set; otherwise /var/tmp/${USER}.pack.
168 --pedantic (yes|no)
170 Some tests require configurations that are very specific
171 (e.g., being a member of at least two groups) or unusual
172 (e.g., sudo to a non-root group). If yes, then fail if the
173 requirement is not met; if no, then skip. The default is yes
174 for CI environments or people listed in README.md, no other‐
175 wise.
176 If yes and sudo seems to be available, implies --sudo.
178 --perm-dir DIR
180 Add DIR to filesystem permission fixture directories; can be
181 specified multiple times. We recommend one such directory per
182 mounted filesystem type whose kernel module you do not trust;
183 e.g., you probably don’t need to test your tmpfses, but
184 out-of-tree filesystems very likely need this.
185 Implies --sudo. Default: CH_TEST_PERMDIRS if set; otherwise
187 skip the filesystem permissions tests.
188 --sudo Enable things that require sudo, such as certain privilege
190 escalation tests and creating/removing the filesystem permis‐
191 sions fixtures. Requires generic sudo capabilities. Note that
192 the Docker builder uses sudo docker even without this option.
193
195 Zero if all tests passed; non-zero if any failed. For setup and tear‐
196 down phases, zero if everything was created or deleted correctly,
197 non-zero otherwise.
198
200 Bats will wait until all descendant processes finish before exiting, so
201 if you get into a failure mode where a test sequence doesn’t clean up
202 all its processes, ch-test will hang.
203
205 Many systems can simply use the defaults. To run the build, run, and
206 examples phases on a single system, without the filesystem permissions
207 tests:
208 $ ch-test
210 ch-test version 0.12
211 ch-run: 0.12 /usr/local/bin/ch-run
213 bats: 0.4.0 /usr/bin/bats
214 tests: /usr/local/libexec/charliecloud/test
215 phase: build run examples
217 scope: standard (default)
218 builder: docker (default)
219 use generic sudo: no (default)
220 unpacked images dir: /var/tmp/img (default)
221 packed images dir: /var/tmp/tar (default)
222 fs permissions dirs: skip (default)
223 checking namespaces ...
225 ok
226 checking builder ...
228 found: /usr/bin/docker 19.03.2
229 bats build.bats build_auto.bats build_post.bats
231 ✓ documentation seems sane
232 ✓ version number seems sane
233 [...]
234 All tests passed.
235 The next example is for a more complex setup like you might find in HPC
237 centers:
238 • Non-default fixture directories.
240 • Non-default scope.
242 • Different build and run systems.
244 • Run the filesystem permissions tests.
246 Output has been omitted.
248 (mybox)$ ssh hpc-admin
250 (hpc-admin)$ ch-test mk-perm-dirs --perm-dir /scratch/$USER/perms \
251 --perm-dir /home/$USER/perms
252 (hpc-admin)$ exit
253 (mybox)$ ch-test build --scope full
254 (mybox)$ scp -r /var/tmp/pack hpc:/scratch/$USER/pack
255 (mybox)$ ssh hpc
256 (hpc)$ salloc -N2
257 (cn001)$ export CH_TEST_TARDIR=/scratch/$USER/pack
258 (cn001)$ export CH_TEST_IMGDIR=/local/tmp
259 (cn001)$ export CH_TEST_PERMDIRS="/scratch/$USER/perms /home/$USER/perms"
260 (cn001)$ export CH_TEST_SCOPE=full
261 (cn001)$ ch-test run
262 (cn001)$ ch-test examples
263
265 If Charliecloud was obtained from your Linux distribution, use your
266 distribution’s bug reporting procedures.
267 Otherwise, report bugs to: https://github.com/hpc/charliecloud/issues
269
271 charliecloud(7)
272 Full documentation at: <https://hpc.github.io/charliecloud>
274
276 2014–2022, Triad National Security, LLC and others
2770.32 2023-07-19 00:00 UTC CH-TEST(1)