1ct_suite(3) Erlang Module Definition ct_suite(3)
2
6 ct_suite - -behaviour(ct_suite).
7
10 The following section describes the mandatory and optional test suite
11 functions that Common Test calls during test execution. For more de‐
12 tails, see section Writing Test Suites in the User's Guide.
13
15 ct_testname() = atom()
16 The name of the testcase function.
18 ct_groupname() = atom()
20 The name of the test group.
22 ct_config() = [{Key :: atom(), Value :: term()}]
24 The configuration data that can be modified.
26 ct_status() = ok | skipped | failed
28 The status value for a nested subgroup.
30 ct_group_def()
32 The test group definition, as returned by Module:groups/0.
34 ct_test_def()
36 The test suite definition, as returned by Module:all/0.
38 ct_info()
40 The test suite information, as returned by Module:suite/0, Mod‐
42 ule:group/1 and Module:Testcase/0.
43
45 The following functions are to be exported from a ct_suite callback
46 module in order to define the callback interface for a test suite.
47
49 Module:all() -> [ct_test_def()] | {skip, Reason}
50 Types:
52 ct_test_def() = TestCase | {group, GroupName} | {group,
54 GroupName, Properties} | {group, GroupName, Properties, Sub‐
55 Groups}
56 TestCase = ct_testname()
57 GroupName = ct_groupname()
58 Properties = [parallel | sequence | Shuffle | {RepeatType,
59 N}] | default
60 SubGroups = [{GroupName, Properties} | {GroupName, Proper‐
61 ties, SubGroups}]
62 Shuffle = shuffle | {shuffle, Seed}
63 Seed = {integer(), integer(), integer()}
64 RepeatType = repeat | repeat_until_all_ok | repeat_un‐
65 til_all_fail | repeat_until_any_ok | repeat_until_any_fail
66 N = integer() | forever
67 Reason = term()
68 MANDATORY
70 Returns the list of all test cases and test case groups in the
72 test suite module to be executed. This list also specifies the
73 order the cases and groups are executed by Common Test. A test
74 case is represented by an atom, the name of the test case func‐
75 tion, or a testcase tuple indicating that the test case shall be
76 repeated. A test case group is represented by a group tuple,
77 where GroupName, an atom, is the name of the group (defined in
78 Module:groups/0). Execution properties for groups can also be
79 specified, both for a top-level group and for any of its sub‐
80 groups. Group execution properties specified here override prop‐
81 erties in the group definition (see Module:groups/0). (With
82 value default, the group definition properties are used).
83 If {skip, Reason} is returned, all test cases in the module are
85 skipped and Reason is printed on the HTML result page.
86 For details on groups, see section Test Case Groups in the
88 User's Guide.
89 Module:groups() -> [ct_group_def()]
91 Types:
93 ct_group_def() = {GroupName, Properties, GroupsAndTestCases}
95 GroupName = ct_groupname()
96 Properties = [parallel | sequence | Shuffle | {RepeatType,
97 N}]
98 GroupsAndTestCases = [Group | {group, GroupName} | TestCase]
99 TestCase = ct_testname()
100 Shuffle = shuffle | {shuffle, Seed}
101 Seed = {integer(), integer(), integer()}
102 RepeatType = repeat | repeat_until_all_ok | repeat_un‐
103 til_all_fail | repeat_until_any_ok | repeat_until_any_fail
104 N = integer() | forever
105 OPTIONAL
107 Defines test case groups. For details, see section Test Case
109 Groups in the User's Guide.
110 Module:suite() -> [ct_info()]
112 Types:
114 ct_info() = {timetrap, Time} | {require, Required} | {re‐
116 quire, Name, Required} | {userdata, UserData} | {silent_con‐
117 nections, Conns} | {stylesheet, CSSFile} | {ct_hooks, CTHs}
118 Time = TimeVal | TimeFunc
119 TimeVal = MilliSec | {seconds, integer()} | {minutes, inte‐
120 ger()} | {hours, integer()}
121 TimeFunc = {Mod, Func, Args} | Fun
122 MilliSec = integer()
123 Mod = atom()
124 Func = atom()
125 Args = list()
126 Fun = fun()
127 Required = Key | {Key, SubKeys} | {Key, SubKey} | {Key, Sub‐
128 Key, SubKeys}
129 Key = atom()
130 SubKeys = SubKey | [SubKey]
131 SubKey = atom()
132 Name = atom()
133 UserData = term()
134 Conns = [atom()]
135 CSSFile = string()
136 CTHs = [CTHModule |
137 {CTHModule, CTHInitArgs} |
138 {CTHModule, CTHInitArgs, CTHPriority}]
139 CTHModule = atom()
140 CTHInitArgs = term()
141 CTHPriority = integer()
142 OPTIONAL
144 The test suite information function. Returns a list of tagged
146 tuples specifying various properties related to the execution of
147 this test suite (common for all test cases in the suite).
148 Tag timetrap sets the maximum time that each test case is al‐
150 lowed to execute (including Module:init_per_testcase/2 and Mod‐
151 ule:end_per_testcase/2). If the timetrap time is exceeded, the
152 test case fails with reason timetrap_timeout. A TimeFunc func‐
153 tion can be used to set a new timetrap by returning a TimeVal.
154 It can also be used to trigger a timetrap time-out by, at some
155 point, returning a value other than a TimeVal. For details, see
156 section Timetrap Time-Outs in the User's Guide.
157 Tag require specifies configuration variables required by test
159 cases (or configuration functions) in the suite. If the required
160 configuration variables are not found in any of the configura‐
161 tion files, all test cases are skipped. For details about the
162 require functionality, see function ct:require/1,2.
163 With userdata, the user can specify any test suite-related in‐
165 formation, which can be read by calling ct:userdata/2.
166 Tag ct_hooks specifies the Common Test Hooks to be run with this
168 suite.
169 Other tuples than the ones defined are ignored.
171 For details about the test suite information function, see sec‐
173 tion Test Suite Information Function in the User's Guide.
174 Module:init_per_suite(Config) -> NewConfig | {skip, Reason} |
176 {skip_and_save, Reason, SaveConfig}
177 Types:
179 Config = NewConfig = SaveConfig = ct_config()
181 Reason = term()
182 OPTIONAL; if this function is defined, then Mod‐
184 ule:end_per_suite/1 must also be defined.
185 This configuration function is called as the first function in
187 the suite. It typically contains initializations that are common
188 for all test cases in the suite, and that must only be done
189 once. Parameter Config is the configuration data that can be
190 modified. Whatever is returned from this function is specified
191 as Config to all configuration functions and test cases in the
192 suite.
193 If {skip, Reason} is returned, all test cases in the suite are
195 skipped and Reason is printed in the overview log for the suite.
196 For information on save_config and skip_and_save, see section
198 Saving Configuration Data in the User's Guide.
199 Module:end_per_suite(Config) -> term() | {save_config, SaveConfig}
201 Types:
203 Config = SaveConfig = ct_config()
205 OPTIONAL; if this function is defined, then Mod‐
207 ule:init_per_suite/1 must also be defined.
208 This function is called as the last test case in the suite. It
210 is meant to be used for cleaning up after Mod‐
211 ule:init_per_suite/1.
212 For information on save_config, see section Saving Configuration
214 Data in the User's Guide.
215 Module:group(GroupName) -> [ct_info()]
217 Types:
219 GroupName = ct_groupname()
221 ct_info() = {timetrap, Time} | {require, Required} | {re‐
222 quire, Name, Required} | {userdata, UserData} | {silent_con‐
223 nections, Conns} | {stylesheet, CSSFile} | {ct_hooks, CTHs}
224 Time = TimeVal | TimeFunc
225 TimeVal = MilliSec | {seconds, integer()} | {minutes, inte‐
226 ger()} | {hours, integer()}
227 TimeFunc = {Mod, Func, Args} | Fun
228 MilliSec = integer()
229 Mod = atom()
230 Func = atom()
231 Args = list()
232 Fun = fun()
233 Required = Key | {Key, SubKeys} | {Key, SubKey} | {Key, Sub‐
234 Key, SubKeys}
235 Key = atom()
236 SubKeys = SubKey | [SubKey]
237 SubKey = atom()
238 Name = atom()
239 UserData = term()
240 Conns = [atom()]
241 CSSFile = string()
242 CTHs = [CTHModule |
243 {CTHModule, CTHInitArgs} |
244 {CTHModule, CTHInitArgs, CTHPriority}]
245 CTHModule = atom()
246 CTHInitArgs = term()
247 CTHPriority = integer()
248 OPTIONAL
250 The test case group information function. It is supposed to re‐
252 turn a list of tagged tuples that specify various properties re‐
253 lated to the execution of a test case group (that is, its test
254 cases and subgroups). Properties set by Module:group/1 override
255 properties with the same key that have been set previously by
256 Module:suite/0.
257 Tag timetrap sets the maximum time that each test case is al‐
259 lowed to execute (including Module:init_per_testcase/2 and Mod‐
260 ule:end_per_testcase/2). If the timetrap time is exceeded, the
261 test case fails with reason timetrap_timeout. A TimeFunc func‐
262 tion can be used to set a new timetrap by returning a TimeVal.
263 It can also be used to trigger a timetrap time-out by, at some
264 point, returning a value other than a TimeVal. For details, see
265 section Timetrap Time-Outs in the User's Guide.
266 Tag require specifies configuration variables required by test
268 cases (or configuration functions) in the suite. If the required
269 configuration variables are not found in any of the configura‐
270 tion files, all test cases in this group are skipped. For de‐
271 tails about the require functionality, see function ct:re‐
272 quire/1,2.
273 With userdata, the user can specify any test case group related
275 information that can be read by calling ct:userdata/2.
276 Tag ct_hooks specifies the Common Test Hooks to be run with this
278 suite.
279 Other tuples than the ones defined are ignored.
281 For details about the test case group information function, see
283 section Group Information Function in the User's Guide.
284 Module:init_per_group(GroupName, Config) -> NewConfig | {skip, Reason}
286 Types:
288 GroupName = ct_groupname()
290 Config = NewConfig = ct_config()
291 Reason = term()
292 OPTIONAL; if this function is defined, then Mod‐
294 ule:end_per_group/2 must also be defined.
295 This configuration function is called before execution of a test
297 case group. It typically contains initializations that are com‐
298 mon for all test cases and subgroups in the group, and that must
299 only be performed once. GroupName is the name of the group, as
300 specified in the group definition (see Module:groups/0). Parame‐
301 ter Config is the configuration data that can be modified. The
302 return value of this function is given as Config to all test
303 cases and subgroups in the group.
304 If {skip, Reason} is returned, all test cases in the group are
306 skipped and Reason is printed in the overview log for the group.
307 For information about test case groups, see section Test Case
309 Groups in the User's Guide.
310 Module:end_per_group(GroupName, Config) -> term() | {return_group_re‐
312 sult, Status}
313 Types:
315 GroupName = ct_groupname()
317 Config = ct_config()
318 Status = ct_status()
319 OPTIONAL; if this function is defined, then Mod‐
321 ule:init_per_group/2 must also be defined.
322 This function is called after the execution of a test case group
324 is finished. It is meant to be used for cleaning up after Mod‐
325 ule:init_per_group/2. A status value for a nested subgroup can
326 be returned with {return_group_result, Status}. The status can
327 be retrieved in Module:end_per_group/2 for the group on the
328 level above. The status is also used by Common Test for deciding
329 if execution of a group is to proceed if property sequence or
330 repeat_until_* is set.
331 For details about test case groups, see section Test Case Groups
333 in the User's Guide.
334 Module:init_per_testcase(TestCase, Config) -> NewConfig | {fail, Rea‐
336 son} | {skip, Reason}
337 Types:
339 TestCase = ct_testname()
341 Config = NewConfig = ct_config()
342 Reason = term()
343 OPTIONAL; if this function is defined, then Module:end_per_test‐
345 case/2 must also be defined.
346 This function is called before each test case. Argument TestCase
348 is the test case name, and Config (list of key-value tuples) is
349 the configuration data that can be modified. The NewConfig list
350 returned from this function is given as Config to the test case.
351 If {fail, Reason} is returned, the test case is marked as failed
352 without being executed.
353 If {skip, Reason} is returned, the test case is skipped and Rea‐
355 son is printed in the overview log for the suite.
356 Module:end_per_testcase(TestCase, Config) -> term() | {fail, Reason} |
358 {save_config, SaveConfig}
359 Types:
361 TestCase = ct_testname()
363 Config = SaveConfig = ct_config()
364 Reason = term()
365 OPTIONAL; if this function is defined, then Mod‐
367 ule:init_per_testcase/2 must also be defined.
368 This function is called after each test case, and can be used to
370 clean up after Module:init_per_testcase/2 and the test case. Any
371 return value (besides {fail, Reason} and {save_config, SaveCon‐
372 fig}) is ignored. By returning {fail, Reason}, TestCase is
373 marked as faulty (even though it was successful in the sense
374 that it returned a value instead of terminating).
375 For information on save_config, see section Saving Configuration
377 Data in the User's Guide.
378 Module:Testcase() -> [ct_info()]
380 Types:
382 ct_info() = {timetrap, Time} | {require, Required} | {re‐
384 quire, Name, Required} | {userdata, UserData} | {silent_con‐
385 nections, Conns} | {stylesheet, CSSFile} | {ct_hooks, CTHs}
386 Time = TimeVal | TimeFunc
387 TimeVal = MilliSec | {seconds, integer()} | {minutes, inte‐
388 ger()} | {hours, integer()}
389 TimeFunc = {Mod, Func, Args} | Fun
390 MilliSec = integer()
391 Mod = atom()
392 Func = atom()
393 Args = list()
394 Fun = fun()
395 Required = Key | {Key, SubKeys} | {Key, SubKey} | {Key, Sub‐
396 Key, SubKeys}
397 Key = atom()
398 SubKeys = SubKey | [SubKey]
399 SubKey = atom()
400 Name = atom()
401 UserData = term()
402 Conns = [atom()]
403 CSSFile = string()
404 CTHs = [CTHModule |
405 {CTHModule, CTHInitArgs} |
406 {CTHModule, CTHInitArgs, CTHPriority}]
407 CTHModule = atom()
408 CTHInitArgs = term()
409 CTHPriority = integer()
410 OPTIONAL
412 The test case information function. It is supposed to return a
414 list of tagged tuples that specify various properties related to
415 the execution of this particular test case. Properties set by
416 Module:Testcase/0 override properties set previously for the
417 test case by Module:group/1 or Module:suite/0.
418 Tag timetrap sets the maximum time that the test case is allowed
420 to execute. If the timetrap time is exceeded, the test case
421 fails with reason timetrap_timeout. Module:init_per_testcase/2
422 and Module:end_per_testcase/2 are included in the timetrap time.
423 A TimeFunc function can be used to set a new timetrap by return‐
424 ing a TimeVal. It can also be used to trigger a timetrap time-
425 out by, at some point, returning a value other than a TimeVal.
426 For details, see section Timetrap Time-Outs in the User's Guide.
427 Tag require specifies configuration variables that are required
429 by the test case (or init_per_testcase/2 or end_per_testcase/2).
430 If the required configuration variables are not found in any of
431 the configuration files, the test case is skipped. For details
432 about the require functionality, see function ct:require/1,2.
433 If timetrap or require is not set, the default values specified
435 by Module:suite/0 (or Module:group/1) are used.
436 With userdata, the user can specify any test case-related infor‐
438 mation that can be read by calling ct:userdata/3.
439 Other tuples than the ones defined are ignored.
441 For details about the test case information function, see sec‐
443 tion Test Case Information Function in the User's Guide.
444 Module:Testcase(Config) -> term() | {skip, Reason} | {comment, Comment}
446 | {save_config, SaveConfig} | {skip_and_save, Reason, SaveConfig} |
447 exit()
448 Types:
450 Config = SaveConfig = ct_config()
452 Reason = term()
453 Comment = string()
454 MANDATORY
456 The implementation of a test case. Call the functions to test
458 and check the result. If something fails, ensure the function
459 causes a runtime error or call ct:fail/1,2 (which also causes
460 the test case process to terminate).
461 Elements from the Config list can, for example, be read with
463 proplists:get_value/2 in STDLIB (or the macro ?config defined in
464 ct.hrl).
465 If you decide not to run the test case after all, return {skip,
467 Reason}. Reason is then printed in field Comment on the HTML re‐
468 sult page.
469 To print some information in field Comment on the HTML result
471 page, return {comment, Comment}.
472 If the function returns anything else, the test case is consid‐
474 ered successful. The return value always gets printed in the
475 test case log file.
476 For details about test case implementation, see section Test
478 Cases in the User's Guide.
479 For information on save_config and skip_and_save, see section
481 Saving Configuration Data in the User's Guide.
482Ericsson AB common_test 1.24.0.1 ct_suite(3)