1CH-TEST(1)                       Charliecloud                       CH-TEST(1)
2
3
4

NAME

6       ch-test - Run some or all of the Charliecloud test suite
7

SYNOPSIS

9          $ ch-test [PHASE] [--scope SCOPE] [ARGS]
10

DESCRIPTION

12       Charliecloud  comes  with a comprehensive test suite that exercises the
13       container workflow itself  as  well  as  a  few  example  applications.
14       ch-test coordinates running the test suite.
15
16       While  the  CLI  has  lots of options, the defaults are reasonable, and
17       bare ch-test will give useful results in a few minutes on  single-node,
18       internet-connected systems with a few GB available in /var/tmp.
19
20       The  test  suite requires a few GB (standard scope) or tens of GB (full
21       scope) of storage for test fixtures:
22
23       · Builder storage (e.g., layer cache). This goes wherever  the  builder
24         puts it.
25
26       · Packed images directory: image tarballs or SquashFS files.
27
28       · Unpacked images directory. Images are unpacked into and then run from
29         here.
30
31       · Filesystem permissions directories. These are used to test  that  the
32         kernel  is  enforcing permissions correctly. Note that this exercises
33         the kernel, not Charliecloud, and can be omitted from  routine  Char‐
34         liecloud testing.
35
36       The  first three are created when needed if they don’t exist, while the
37       filesystem permissions fixtures must be created manually, in  order  to
38       accommodate  configurations  where  sudo  is not available via the same
39       login path used for running tests.
40
41       The packed and unpacked image directories  specified  for  testing  are
42       volatile.   The  contents  of  these directories are deleted before the
43       build and run phases, respectively.
44
45       In all four cases, when creating directories, only the final path  com‐
46       ponent is created. Parent directories must already exist, i.e., ch-test
47       uses the behavior of mkdir rather than mkdir -p.
48
49       Some of the tests exercise parallel functionality. If ch-test is run on
50       a  single  node, multiple cores will be used; if in a Slurm allocation,
51       multiple nodes too.
52
53       The subset of tests to run mostly splits along two key dimensions.  The
54       phase  is  which  parts  of the workflow to run. Different parts of the
55       workflow can be tested on different systems by  copying  the  necessary
56       artifacts  between them, e.g. by building images on one system and run‐
57       ning them on another. The scope allows trading off thoroughness  versus
58       time.
59
60       PHASE must be one of the following:
61
62          build  Image   building   and  associated  functionality,  with  the
63                 selected builder.
64
65          run    Running  containers  and   associated   functionality.   This
66                 requires  a  packed images directory produced by a successful
67                 build phase, which can be copied from  the  build  system  if
68                 it’s not also the run system.
69
70          examples
71                 Example  applications.  Requires an unpacked images directory
72                 produced by a successful run phase.
73
74          all    Execute phases build, run, and examples, in that order.
75
76          mk-perm-dirs
77                 Create  the  filesystem  permissions  directories.   Requires
78                 --perm-dirs.
79
80          clean  Delete  automatically-generated  test  files,  and packed and
81                 unpacked image directories.
82
83          rm-perm-dirs
84                 Remove  the  filesystem  permissions  directories.   Requires
85                 --perm-dirs.
86
87          -f, --file FILE[:TEST]
88                 Run  the  tests in the given file only, which can be an arbi‐
89                 trary .bats file, except for test.bats under examples,  where
90                 you  must  specify the corresponding Dockerfile or Build file
91                 instead. This is somewhat  brittle  and  typically  used  for
92                 development  or  debugging.  For  example,  it does not check
93                 whether the pre-requisites of whatever is  in  the  file  are
94                 satisfied.  Often  running build and run first is sufficient,
95                 but this varies.
96
97                 If TEST is also given, then run only the test with that name,
98                 skipping  the  others. The separator is a literal colon. Most
99                 test names contain spaces, so you’ll usually  need  to  quote
100                 the argument to protect it from the shell.
101
102       Scope is specified with:
103
104          -s, --scope SCOPE
105                 SCOPE must be one of the following; the default is standard.
106
107                 · quick:  Most important subset of workflow. Handy for devel‐
108                   opment.  Completion time: 1–2 minutes.
109
110                 · standard: All tested workflow functionality and a selection
111                   of more important examples. Completion time: 5–10 minutes.
112
113                 · full:  All available tests, including all examples. Comple‐
114                   tion time, hot cache: 7–15 minutes; cold cache: 1–2 hours.
115
116       Additional arguments:
117
118          -b, --builder BUILDER
119                 Image builder to use. See ch-build(1) for how the default  is
120                 selected.
121
122          --dry-run
123                 Print summary of what would be tested and then exit.
124
125          -h, --help
126                 Print usage and then exit.
127
128          --img-dir DIR
129                 Set unpacked images directory to DIR. In a multi-node alloca‐
130                 tion,  this  directory  may  not  be  shared  between  nodes.
131                 Default: $CH_TEST_IMGDIR if set; otherwise /var/tmp/img.
132
133          --pack-dir DIR
134                 Set  packed images directory to DIR. Default: $CH_TEST_TARDIR
135                 if set; otherwise /var/tmp/pack.
136
137          --pedantic (yes|no)
138                 Some tests require  configurations  that  are  very  specific
139                 (e.g.,  being  a  member  of  at least two groups) or unusual
140                 (e.g., sudo to a non-root group). If yes, then  fail  if  the
141                 requirement  is not met; if no, then skip. The default is yes
142                 for CI environments or people listed in README.md, no  other‐
143                 wise.
144
145                 If yes and sudo seems to be available, implies --sudo.
146
147          --perm-dir DIR
148                 Add  DIR to filesystem permission fixture directories; can be
149                 specified multiple times. We recommend one such directory per
150                 mounted filesystem type whose kernel module you do not trust;
151                 e.g., you probably don’t  need  to  test  your  tmpfses,  but
152                 out-of-tree filesystems very likely need this.
153
154                 Implies  --sudo.  Default: CH_TEST_PERMDIRS if set; otherwise
155                 skip the filesystem permissions tests.
156
157          --pack-fmt FMT
158                 Use packed image format FMT (squash or tar).
159
160          --sudo Enable things that require sudo, such  as  certain  privilege
161                 escalation tests and creating/removing the filesystem permis‐
162                 sions fixtures. Requires generic sudo capabilities. Note that
163                 the Docker builder uses sudo docker even without this option.
164
165          --lustre DIR
166                 Use  DIR  for  run-phase  Lustre tests. Default: CH_TEST_LUS‐
167                 TREDIR if set; otherwise skip them.
168
169                 The tests will create, populate, and delete a  new  subdirec‐
170                 tory under DIR, leaving everything else in DIR untouched.
171

EXIT STATUS

173       Zero  if  all tests passed; non-zero if any failed. For setup and tear‐
174       down phases, zero if  everything  was  created  or  deleted  correctly,
175       non-zero otherwise.
176

BUGS

178       Bats will wait until all descendant processes finish before exiting, so
179       if you get into a failure mode where a test sequence doesn’t  clean  up
180       all its processes, ch-test will hang.
181

EXAMPLES

183       Many  systems  can  simply use the defaults. To run the build, run, and
184       examples phases on a single system, without the filesystem  permissions
185       tests:
186
187          $ ch-test
188          ch-test version 0.12
189
190          ch-run: 0.12 /usr/local/bin/ch-run
191          bats:   0.4.0 /usr/bin/bats
192          tests:  /usr/local/libexec/charliecloud/test
193
194          phase:                build run examples
195          scope:                standard (default)
196          builder:              docker (default)
197          use generic sudo:     no (default)
198          unpacked images dir:  /var/tmp/img (default)
199          packed images dir:    /var/tmp/tar (default)
200          fs permissions dirs:  skip (default)
201
202          checking namespaces ...
203          ok
204
205          checking builder ...
206          found: /usr/bin/docker 19.03.2
207
208          bats build.bats build_auto.bats build_post.bats
209           ✓ documentation seems sane
210           ✓ version number seems sane
211          [...]
212          All tests passed.
213
214       The next example is for a more complex setup like you might find in HPC
215       centers:
216
217          · Non-default fixture directories.
218
219          · Non-default scope.
220
221          · Different build and run systems.
222
223          · Run the filesystem permissions tests.
224
225       Output has been omitted.
226
227          (mybox)$ ssh hpc-admin
228          (hpc-admin)$ ch-test mk-perm-dirs --perm-dir /scratch/$USER/perms \
229                                            --perm-dir /home/$USER/perms
230          (hpc-admin)$ exit
231          (mybox)$ ch-test build --scope full
232          (mybox)$ scp -r /var/tmp/pack hpc:/scratch/$USER/pack
233          (mybox)$ ssh hpc
234          (hpc)$ salloc -N2
235          (cn001)$ export CH_TEST_TARDIR=/scratch/$USER/pack
236          (cn001)$ export CH_TEST_IMGDIR=/local/tmp
237          (cn001)$ export CH_TEST_PERMDIRS="/scratch/$USER/perms /home/$USER/perms"
238          (cn001)$ export CH_TEST_SCOPE=full
239          (cn001)$ ch-test run
240          (cn001)$ ch-test examples
241

REPORTING BUGS

243       If Charliecloud was obtained from your  Linux  distribution,  use  your
244       distribution’s bug reporting procedures.
245
246       Otherwise, report bugs to: <https://github.com/hpc/charliecloud/issues>
247

SEE ALSO

249       charliecloud(7)
250
251       Full documentation at: <https://hpc.github.io/charliecloud>
252
254       2014–2020, Triad National Security, LLC
255
256
257
258
2590.22              2021-03-08 00:00 Coordinated Universal Time       CH-TEST(1)
Impressum