1CH-TEST(1)                       Charliecloud                       CH-TEST(1)
2
3
4

NAME

6       ch-test - Run some or all of the Charliecloud test suite
7

SYNOPSIS

9          $ ch-test [PHASE] [--scope SCOPE] [--pack-fmt FMT] [ARGS]
10

DESCRIPTION

12       Charliecloud  comes  with a comprehensive test suite that exercises the
13       container workflow itself  as  well  as  a  few  example  applications.
14       ch-test coordinates running the test suite.
15
16       While  the  CLI  has  lots of options, the defaults are reasonable, and
17       bare ch-test will give useful results in a few minutes on  single-node,
18       internet-connected systems with a few GB available in /var/tmp.
19
20       The  test  suite requires a few GB (standard scope) or tens of GB (full
21       scope) of storage for test fixtures:
22
23Builder storage (e.g., layer cache). This goes wherever  the  builder
24         puts it.
25
26Packed images directory: image tarballs or SquashFS files.
27
28Unpacked images directory. Images are unpacked into and then run from
29         here.
30
31Filesystem permissions directories. These are used to test  that  the
32         kernel  is  enforcing permissions correctly. Note that this exercises
33         the kernel, not Charliecloud, and can be omitted from  routine  Char‐
34         liecloud testing.
35
36       The  first three are created when needed if they don’t exist, while the
37       filesystem permissions fixtures must be created manually, in  order  to
38       accommodate configurations where sudo is not available via the same lo‐
39       gin path used for running tests.
40
41       The packed and unpacked image directories  specified  for  testing  are
42       volatile.   The  contents  of  these directories are deleted before the
43       build and run phases, respectively.
44
45       In all four cases, when creating directories, only the final path  com‐
46       ponent is created. Parent directories must already exist, i.e., ch-test
47       uses the behavior of mkdir rather than mkdir -p.
48
49       Some of the tests exercise parallel functionality. If ch-test is run on
50       a  single  node, multiple cores will be used; if in a Slurm allocation,
51       multiple nodes too.
52
53       The subset of tests to run mostly splits along two key dimensions.  The
54       phase  is  which  parts  of the workflow to run. Different parts of the
55       workflow can be tested on different systems by  copying  the  necessary
56       artifacts  between them, e.g. by building images on one system and run‐
57       ning them on another. The scope allows trading off thoroughness  versus
58       time.
59
60       PHASE must be one of the following:
61
62          build  Image  building  and  associated  functionality, with the se‐
63                 lected builder.
64
65          run    Running containers and  associated  functionality.  This  re‐
66                 quires  a  packed  images  directory produced by a successful
67                 build phase, which can be copied from  the  build  system  if
68                 it’s not also the run system.
69
70          examples
71                 Example  applications.  Requires an unpacked images directory
72                 produced by a successful run phase.
73
74          all    Execute phases build, run, and examples, in that order.
75
76          mk-perm-dirs
77                 Create  the  filesystem  permissions  directories.   Requires
78                 --perm-dirs.
79
80          clean  Delete automatically-generated test files, and packed and un‐
81                 packed image directories.
82
83          rm-perm-dirs
84                 Remove  the  filesystem  permissions  directories.   Requires
85                 --perm-dirs.
86
87          -f, --file FILE[:TEST]
88                 Run  the  tests in the given file only, which can be an arbi‐
89                 trary .bats file, except for test.bats under examples,  where
90                 you  must  specify the corresponding Dockerfile or Build file
91                 instead. This is somewhat brittle and typically used for  de‐
92                 velopment  or  debugging.  For  example,  it  does  not check
93                 whether the pre-requisites of whatever is  in  the  file  are
94                 satisfied.  Often  running build and run first is sufficient,
95                 but this varies.
96
97                 If TEST is also given, then run only tests with name contain‐
98                 ing that string, skipping the others. The separator is a lit‐
99                 eral colon. If the string contains shell metacharacters  such
100                 as  space,  you’ll  need  to quote the argument to protect it
101                 from the shell.
102
103       Scope is specified with:
104
105          -s, --scope SCOPE
106                 SCOPE must be one of the following:
107
108quick: Most important subset of workflow. Handy for  devel‐
109                   opment.
110
111standard: All tested workflow functionality and a selection
112                   of more important examples. (Default.)
113
114full: All available tests, including all examples.
115
116       Image format is specified with:
117
118          --pack-fmt FMT
119                 FMT must be one of the following:
120
121squash-mount or 🐘: SquashFS archive, run directly from the
122                   archive  using  ch-run’s internal SquashFUSE functionality.
123                   In this mode, tests that require writing to the  image  are
124                   skipped.
125
126tar-unpack  or 📠: Tarball, and the images are unpacked be‐
127                   fore running.
128
129squash-unpack or 🎃: SquashFS, and the images are  unpacked
130                   before running.
131
132                 Default:   $CH_TEST_PACK_FMT   if   set.  Otherwise,  if  mk‐
133                 squashfs(1) is available and ch-run was built with libsquash‐
134                 fuse support, then squash-mount, else tar-unpack.
135
136       Additional arguments:
137
138          -b, --builder BUILDER
139                 Image  builder to use. See ch-build(1) for how the default is
140                 selected.
141
142          --dry-run
143                 Print summary of what would be tested and then exit.
144
145          -h, --help
146                 Print usage and then exit.
147
148          --img-dir DIR
149                 Set unpacked images directory to DIR. In a multi-node alloca‐
150                 tion,  this  directory  may  not be shared between nodes. De‐
151                 fault: $CH_TEST_IMGDIR if set; otherwise /var/tmp/img.
152
153          --lustre DIR
154                 Use DIR for run-phase  Lustre  tests.  Default:  CH_TEST_LUS‐
155                 TREDIR if set; otherwise skip them.
156
157                 The  tests  will create, populate, and delete a new subdirec‐
158                 tory under DIR, leaving everything else in DIR untouched.
159
160          --pack-dir DIR
161                 Set packed images directory to DIR. Default:  $CH_TEST_TARDIR
162                 if set; otherwise /var/tmp/pack.
163
164          --pedantic (yes|no)
165                 Some  tests  require  configurations  that  are very specific
166                 (e.g., being a member of at  least  two  groups)  or  unusual
167                 (e.g.,  sudo  to  a non-root group). If yes, then fail if the
168                 requirement is not met; if no, then skip. The default is  yes
169                 for  CI environments or people listed in README.md, no other‐
170                 wise.
171
172                 If yes and sudo seems to be available, implies --sudo.
173
174          --perm-dir DIR
175                 Add DIR to filesystem permission fixture directories; can  be
176                 specified multiple times. We recommend one such directory per
177                 mounted filesystem type whose kernel module you do not trust;
178                 e.g.,  you  probably  don’t  need  to  test your tmpfses, but
179                 out-of-tree filesystems very likely need this.
180
181                 Implies --sudo. Default: CH_TEST_PERMDIRS if  set;  otherwise
182                 skip the filesystem permissions tests.
183
184          --sudo Enable  things  that  require sudo, such as certain privilege
185                 escalation tests and creating/removing the filesystem permis‐
186                 sions fixtures. Requires generic sudo capabilities. Note that
187                 the Docker builder uses sudo docker even without this option.
188

EXIT STATUS

190       Zero if all tests passed; non-zero if any failed. For setup  and  tear‐
191       down  phases,  zero  if  everything  was  created or deleted correctly,
192       non-zero otherwise.
193

BUGS

195       Bats will wait until all descendant processes finish before exiting, so
196       if  you  get into a failure mode where a test sequence doesn’t clean up
197       all its processes, ch-test will hang.
198

EXAMPLES

200       Many systems can simply use the defaults. To run the  build,  run,  and
201       examples  phases on a single system, without the filesystem permissions
202       tests:
203
204          $ ch-test
205          ch-test version 0.12
206
207          ch-run: 0.12 /usr/local/bin/ch-run
208          bats:   0.4.0 /usr/bin/bats
209          tests:  /usr/local/libexec/charliecloud/test
210
211          phase:                build run examples
212          scope:                standard (default)
213          builder:              docker (default)
214          use generic sudo:     no (default)
215          unpacked images dir:  /var/tmp/img (default)
216          packed images dir:    /var/tmp/tar (default)
217          fs permissions dirs:  skip (default)
218
219          checking namespaces ...
220          ok
221
222          checking builder ...
223          found: /usr/bin/docker 19.03.2
224
225          bats build.bats build_auto.bats build_post.bats
226           ✓ documentation seems sane
227           ✓ version number seems sane
228          [...]
229          All tests passed.
230
231       The next example is for a more complex setup like you might find in HPC
232       centers:
233
234          • Non-default fixture directories.
235
236          • Non-default scope.
237
238          • Different build and run systems.
239
240          • Run the filesystem permissions tests.
241
242       Output has been omitted.
243
244          (mybox)$ ssh hpc-admin
245          (hpc-admin)$ ch-test mk-perm-dirs --perm-dir /scratch/$USER/perms \
246                                            --perm-dir /home/$USER/perms
247          (hpc-admin)$ exit
248          (mybox)$ ch-test build --scope full
249          (mybox)$ scp -r /var/tmp/pack hpc:/scratch/$USER/pack
250          (mybox)$ ssh hpc
251          (hpc)$ salloc -N2
252          (cn001)$ export CH_TEST_TARDIR=/scratch/$USER/pack
253          (cn001)$ export CH_TEST_IMGDIR=/local/tmp
254          (cn001)$ export CH_TEST_PERMDIRS="/scratch/$USER/perms /home/$USER/perms"
255          (cn001)$ export CH_TEST_SCOPE=full
256          (cn001)$ ch-test run
257          (cn001)$ ch-test examples
258

REPORTING BUGS

260       If  Charliecloud  was  obtained  from your Linux distribution, use your
261       distribution’s bug reporting procedures.
262
263       Otherwise, report bugs to: https://github.com/hpc/charliecloud/issues
264

SEE ALSO

266       charliecloud(7)
267
268       Full documentation at: <https://hpc.github.io/charliecloud>
269
271       2014–2021, Triad National Security, LLC
272
273
274
275
2760.26                         2023-01-18 00:00 UTC                   CH-TEST(1)
Impressum