1ct_suite(3) Erlang Module Definition ct_suite(3)
2
6 ct_suite - -behaviour(ct_suite).
7
10 The following section describes the mandatory and optional test suite
11 functions that Common Test calls during test execution. For more de‐
12 tails, see section Writing Test Suites in the User's Guide.
13
15 ct_testname() = atom()
16 The name of the testcase function.
18 ct_groupname() = atom()
20 The name of the test group.
22 ct_config() = [{Key :: atom(), Value :: term()}]
24 The configuration data that can be modified.
26 ct_status() = ok | skipped | failed
28 The status value for a nested subgroup.
30 ct_group_def()
32 The test group definition, as returned by Module:groups/0.
34 ct_test_def()
36 The test suite definition, as returned by Module:all/0.
38 ct_info()
40 The test suite information, as returned by Module:suite/0, Mod‐
42 ule:group/1 and Module:Testcase/0.
43
45 The following functions are to be exported from a ct_suite callback
46 module in order to define the callback interface for a test suite.
47
49 Module:all() -> [ct_test_def()] | {skip, Reason}
50 Types:
52 ct_test_def() = TestCase | {group, GroupName} | {group,
54 GroupName, Properties} | {group, GroupName, Properties, Sub‐
55 Groups} | {testcase, TestCase, TestCaseRepeatType}
56 TestCase = ct_testname()
57 GroupName = ct_groupname()
58 Properties = [parallel | sequence | Shuffle | {RepeatType,
59 N}] | default
60 SubGroups = [{GroupName, Properties} | {GroupName, Proper‐
61 ties, SubGroups}]
62 Shuffle = shuffle | {shuffle, Seed}
63 Seed = {integer(), integer(), integer()}
64 RepeatType = repeat | repeat_until_all_ok | repeat_un‐
65 til_all_fail | repeat_until_any_ok | repeat_until_any_fail
66 TestCaseRepeatType = [{repeat, N} | {repeat_until_ok, N} |
67 {repeat_until_fail, N}]
68 N = integer() | forever
69 Reason = term()
70 MANDATORY
72 Returns the list of all test cases and test case groups in the
74 test suite module to be executed. This list also specifies the
75 order the cases and groups are executed by Common Test. A test
76 case is represented by an atom, the name of the test case func‐
77 tion, or a testcase tuple indicating that the test case shall be
78 repeated. A test case group is represented by a group tuple,
79 where GroupName, an atom, is the name of the group (defined in
80 Module:groups/0). Execution properties for groups can also be
81 specified, both for a top-level group and for any of its sub‐
82 groups. Group execution properties specified here override prop‐
83 erties in the group definition (see Module:groups/0). (With
84 value default, the group definition properties are used).
85 If {skip, Reason} is returned, all test cases in the module are
87 skipped and Reason is printed on the HTML result page.
88 For details on groups, see section Test Case Groups in the
90 User's Guide.
91 Module:groups() -> [ct_group_def()]
93 Types:
95 ct_group_def() = {GroupName, Properties, GroupsAndTestCases}
97 GroupName = ct_groupname()
98 Properties = [parallel | sequence | Shuffle | {RepeatType,
99 N}]
100 GroupsAndTestCases = [Group | {group, GroupName} | TestCase |
101 {testcase, TestCase, TestCaseRepeatType}]
102 TestCase = ct_testname()
103 Shuffle = shuffle | {shuffle, Seed}
104 Seed = {integer(), integer(), integer()}
105 RepeatType = repeat | repeat_until_all_ok | repeat_un‐
106 til_all_fail | repeat_until_any_ok | repeat_until_any_fail
107 TestCaseRepeatType = [{repeat, N} | {repeat_until_ok, N} |
108 {repeat_until_fail, N}]
109 N = integer() | forever
110 OPTIONAL
112 Defines test case groups. For details, see section Test Case
114 Groups in the User's Guide.
115 Module:suite() -> [ct_info()]
117 Types:
119 ct_info() = {timetrap, Time} | {require, Required} | {re‐
121 quire, Name, Required} | {userdata, UserData} | {silent_con‐
122 nections, Conns} | {stylesheet, CSSFile} | {ct_hooks, CTHs}
123 Time = TimeVal | TimeFunc
124 TimeVal = MilliSec | {seconds, integer()} | {minutes, inte‐
125 ger()} | {hours, integer()}
126 TimeFunc = {Mod, Func, Args} | Fun
127 MilliSec = integer()
128 Mod = atom()
129 Func = atom()
130 Args = list()
131 Fun = fun()
132 Required = Key | {Key, SubKeys} | {Key, SubKey} | {Key, Sub‐
133 Key, SubKeys}
134 Key = atom()
135 SubKeys = SubKey | [SubKey]
136 SubKey = atom()
137 Name = atom()
138 UserData = term()
139 Conns = [atom()]
140 CSSFile = string()
141 CTHs = [CTHModule |
142 {CTHModule, CTHInitArgs} |
143 {CTHModule, CTHInitArgs, CTHPriority}]
144 CTHModule = atom()
145 CTHInitArgs = term()
146 CTHPriority = integer()
147 OPTIONAL
149 The test suite information function. Returns a list of tagged
151 tuples specifying various properties related to the execution of
152 this test suite (common for all test cases in the suite).
153 Tag timetrap sets the maximum time that each test case is al‐
155 lowed to execute (including Module:init_per_testcase/2 and Mod‐
156 ule:end_per_testcase/2). If the timetrap time is exceeded, the
157 test case fails with reason timetrap_timeout. A TimeFunc func‐
158 tion can be used to set a new timetrap by returning a TimeVal.
159 It can also be used to trigger a timetrap time-out by, at some
160 point, returning a value other than a TimeVal. For details, see
161 section Timetrap Time-Outs in the User's Guide.
162 Tag require specifies configuration variables required by test
164 cases (or configuration functions) in the suite. If the required
165 configuration variables are not found in any of the configura‐
166 tion files, all test cases are skipped. For details about the
167 require functionality, see function ct:require/1,2.
168 With userdata, the user can specify any test suite-related in‐
170 formation, which can be read by calling ct:userdata/2.
171 Tag ct_hooks specifies the Common Test Hooks to be run with this
173 suite.
174 Other tuples than the ones defined are ignored.
176 For details about the test suite information function, see sec‐
178 tion Test Suite Information Function in the User's Guide.
179 Module:init_per_suite(Config) -> NewConfig | {skip, Reason} |
181 {skip_and_save, Reason, SaveConfig}
182 Types:
184 Config = NewConfig = SaveConfig = ct_config()
186 Reason = term()
187 OPTIONAL; if this function is defined, then Mod‐
189 ule:end_per_suite/1 must also be defined.
190 This configuration function is called as the first function in
192 the suite. It typically contains initializations that are common
193 for all test cases in the suite, and that must only be done
194 once. Parameter Config is the configuration data that can be
195 modified. Whatever is returned from this function is specified
196 as Config to all configuration functions and test cases in the
197 suite.
198 If {skip, Reason} is returned, all test cases in the suite are
200 skipped and Reason is printed in the overview log for the suite.
201 For information on save_config and skip_and_save, see section
203 Saving Configuration Data in the User's Guide.
204 Module:end_per_suite(Config) -> term() | {save_config, SaveConfig}
206 Types:
208 Config = SaveConfig = ct_config()
210 OPTIONAL; if this function is defined, then Mod‐
212 ule:init_per_suite/1 must also be defined.
213 This function is called as the last test case in the suite. It
215 is meant to be used for cleaning up after Mod‐
216 ule:init_per_suite/1.
217 For information on save_config, see section Saving Configuration
219 Data in the User's Guide.
220 Module:group(GroupName) -> [ct_info()]
222 Types:
224 GroupName = ct_groupname()
226 ct_info() = {timetrap, Time} | {require, Required} | {re‐
227 quire, Name, Required} | {userdata, UserData} | {silent_con‐
228 nections, Conns} | {stylesheet, CSSFile} | {ct_hooks, CTHs}
229 Time = TimeVal | TimeFunc
230 TimeVal = MilliSec | {seconds, integer()} | {minutes, inte‐
231 ger()} | {hours, integer()}
232 TimeFunc = {Mod, Func, Args} | Fun
233 MilliSec = integer()
234 Mod = atom()
235 Func = atom()
236 Args = list()
237 Fun = fun()
238 Required = Key | {Key, SubKeys} | {Key, SubKey} | {Key, Sub‐
239 Key, SubKeys}
240 Key = atom()
241 SubKeys = SubKey | [SubKey]
242 SubKey = atom()
243 Name = atom()
244 UserData = term()
245 Conns = [atom()]
246 CSSFile = string()
247 CTHs = [CTHModule |
248 {CTHModule, CTHInitArgs} |
249 {CTHModule, CTHInitArgs, CTHPriority}]
250 CTHModule = atom()
251 CTHInitArgs = term()
252 CTHPriority = integer()
253 OPTIONAL
255 The test case group information function. It is supposed to re‐
257 turn a list of tagged tuples that specify various properties re‐
258 lated to the execution of a test case group (that is, its test
259 cases and subgroups). Properties set by Module:group/1 override
260 properties with the same key that have been set previously by
261 Module:suite/0.
262 Tag timetrap sets the maximum time that each test case is al‐
264 lowed to execute (including Module:init_per_testcase/2 and Mod‐
265 ule:end_per_testcase/2). If the timetrap time is exceeded, the
266 test case fails with reason timetrap_timeout. A TimeFunc func‐
267 tion can be used to set a new timetrap by returning a TimeVal.
268 It can also be used to trigger a timetrap time-out by, at some
269 point, returning a value other than a TimeVal. For details, see
270 section Timetrap Time-Outs in the User's Guide.
271 Tag require specifies configuration variables required by test
273 cases (or configuration functions) in the suite. If the required
274 configuration variables are not found in any of the configura‐
275 tion files, all test cases in this group are skipped. For de‐
276 tails about the require functionality, see function ct:re‐
277 quire/1,2.
278 With userdata, the user can specify any test case group related
280 information that can be read by calling ct:userdata/2.
281 Tag ct_hooks specifies the Common Test Hooks to be run with this
283 suite.
284 Other tuples than the ones defined are ignored.
286 For details about the test case group information function, see
288 section Group Information Function in the User's Guide.
289 Module:init_per_group(GroupName, Config) -> NewConfig | {skip, Reason}
291 Types:
293 GroupName = ct_groupname()
295 Config = NewConfig = ct_config()
296 Reason = term()
297 OPTIONAL; if this function is defined, then Mod‐
299 ule:end_per_group/2 must also be defined.
300 This configuration function is called before execution of a test
302 case group. It typically contains initializations that are com‐
303 mon for all test cases and subgroups in the group, and that must
304 only be performed once. GroupName is the name of the group, as
305 specified in the group definition (see Module:groups/0). Parame‐
306 ter Config is the configuration data that can be modified. The
307 return value of this function is given as Config to all test
308 cases and subgroups in the group.
309 If {skip, Reason} is returned, all test cases in the group are
311 skipped and Reason is printed in the overview log for the group.
312 For information about test case groups, see section Test Case
314 Groups in the User's Guide.
315 Module:end_per_group(GroupName, Config) -> term() | {return_group_re‐
317 sult, Status}
318 Types:
320 GroupName = ct_groupname()
322 Config = ct_config()
323 Status = ct_status()
324 OPTIONAL; if this function is defined, then Mod‐
326 ule:init_per_group/2 must also be defined.
327 This function is called after the execution of a test case group
329 is finished. It is meant to be used for cleaning up after Mod‐
330 ule:init_per_group/2. A status value for a nested subgroup can
331 be returned with {return_group_result, Status}. The status can
332 be retrieved in Module:end_per_group/2 for the group on the
333 level above. The status is also used by Common Test for deciding
334 if execution of a group is to proceed if property sequence or
335 repeat_until_* is set.
336 For details about test case groups, see section Test Case Groups
338 in the User's Guide.
339 Module:init_per_testcase(TestCase, Config) -> NewConfig | {fail, Rea‐
341 son} | {skip, Reason}
342 Types:
344 TestCase = ct_testname()
346 Config = NewConfig = ct_config()
347 Reason = term()
348 OPTIONAL; if this function is defined, then Module:end_per_test‐
350 case/2 must also be defined.
351 This function is called before each test case. Argument TestCase
353 is the test case name, and Config (list of key-value tuples) is
354 the configuration data that can be modified. The NewConfig list
355 returned from this function is given as Config to the test case.
356 If {fail, Reason} is returned, the test case is marked as failed
357 without being executed.
358 If {skip, Reason} is returned, the test case is skipped and Rea‐
360 son is printed in the overview log for the suite.
361 Module:end_per_testcase(TestCase, Config) -> term() | {fail, Reason} |
363 {save_config, SaveConfig}
364 Types:
366 TestCase = ct_testname()
368 Config = SaveConfig = ct_config()
369 Reason = term()
370 OPTIONAL; if this function is defined, then Mod‐
372 ule:init_per_testcase/2 must also be defined.
373 This function is called after each test case, and can be used to
375 clean up after Module:init_per_testcase/2 and the test case. Any
376 return value (besides {fail, Reason} and {save_config, SaveCon‐
377 fig}) is ignored. By returning {fail, Reason}, TestCase is
378 marked as faulty (even though it was successful in the sense
379 that it returned a value instead of terminating).
380 For information on save_config, see section Saving Configuration
382 Data in the User's Guide.
383 Module:Testcase() -> [ct_info()]
385 Types:
387 ct_info() = {timetrap, Time} | {require, Required} | {re‐
389 quire, Name, Required} | {userdata, UserData} | {silent_con‐
390 nections, Conns} | {stylesheet, CSSFile} | {ct_hooks, CTHs}
391 Time = TimeVal | TimeFunc
392 TimeVal = MilliSec | {seconds, integer()} | {minutes, inte‐
393 ger()} | {hours, integer()}
394 TimeFunc = {Mod, Func, Args} | Fun
395 MilliSec = integer()
396 Mod = atom()
397 Func = atom()
398 Args = list()
399 Fun = fun()
400 Required = Key | {Key, SubKeys} | {Key, SubKey} | {Key, Sub‐
401 Key, SubKeys}
402 Key = atom()
403 SubKeys = SubKey | [SubKey]
404 SubKey = atom()
405 Name = atom()
406 UserData = term()
407 Conns = [atom()]
408 CSSFile = string()
409 CTHs = [CTHModule |
410 {CTHModule, CTHInitArgs} |
411 {CTHModule, CTHInitArgs, CTHPriority}]
412 CTHModule = atom()
413 CTHInitArgs = term()
414 CTHPriority = integer()
415 OPTIONAL
417 The test case information function. It is supposed to return a
419 list of tagged tuples that specify various properties related to
420 the execution of this particular test case. Properties set by
421 Module:Testcase/0 override properties set previously for the
422 test case by Module:group/1 or Module:suite/0.
423 Tag timetrap sets the maximum time that the test case is allowed
425 to execute. If the timetrap time is exceeded, the test case
426 fails with reason timetrap_timeout. Module:init_per_testcase/2
427 and Module:end_per_testcase/2 are included in the timetrap time.
428 A TimeFunc function can be used to set a new timetrap by return‐
429 ing a TimeVal. It can also be used to trigger a timetrap time-
430 out by, at some point, returning a value other than a TimeVal.
431 For details, see section Timetrap Time-Outs in the User's Guide.
432 Tag require specifies configuration variables that are required
434 by the test case (or init_per_testcase/2 or end_per_testcase/2).
435 If the required configuration variables are not found in any of
436 the configuration files, the test case is skipped. For details
437 about the require functionality, see function ct:require/1,2.
438 If timetrap or require is not set, the default values specified
440 by Module:suite/0 (or Module:group/1) are used.
441 With userdata, the user can specify any test case-related infor‐
443 mation that can be read by calling ct:userdata/3.
444 Other tuples than the ones defined are ignored.
446 For details about the test case information function, see sec‐
448 tion Test Case Information Function in the User's Guide.
449 Module:Testcase(Config) -> term() | {skip, Reason} | {comment, Comment}
451 | {save_config, SaveConfig} | {skip_and_save, Reason, SaveConfig} |
452 exit()
453 Types:
455 Config = SaveConfig = ct_config()
457 Reason = term()
458 Comment = string()
459 MANDATORY
461 The implementation of a test case. Call the functions to test
463 and check the result. If something fails, ensure the function
464 causes a runtime error or call ct:fail/1,2 (which also causes
465 the test case process to terminate).
466 Elements from the Config list can, for example, be read with
468 proplists:get_value/2 in STDLIB (or the macro ?config defined in
469 ct.hrl).
470 If you decide not to run the test case after all, return {skip,
472 Reason}. Reason is then printed in field Comment on the HTML re‐
473 sult page.
474 To print some information in field Comment on the HTML result
476 page, return {comment, Comment}.
477 If the function returns anything else, the test case is consid‐
479 ered successful. The return value always gets printed in the
480 test case log file.
481 For details about test case implementation, see section Test
483 Cases in the User's Guide.
484 For information on save_config and skip_and_save, see section
486 Saving Configuration Data in the User's Guide.
487Ericsson AB common_test 1.25.1 ct_suite(3)