1ct_suite(3) Erlang Module Definition ct_suite(3)
2
6 ct_suite - -behaviour(ct_suite).
7
10 The following section describes the mandatory and optional test suite
11 functions that Common Test calls during test execution. For more
12 details, see section Writing Test Suites in the User's Guide.
13
15 ct_testname() = atom()
16 The name of the testcase function.
18 ct_groupname() = atom()
20 The name of the test group.
22 ct_config() = [{Key :: atom(), Value :: term()}]
24 The configuration data that can be modified.
26 ct_status() = ok | skipped | failed
28 The status value for a nested subgroup.
30 ct_group_def()
32 The test group definition, as returned by Module:groups/0.
34 ct_test_def()
36 The test suite definition, as returned by Module:all/0.
38 ct_info()
40 The test suite information, as returned by Module:suite/0, Mod‐
42 ule:group/1 and Module:Testcase/0.
43
45 The following functions are to be exported from a ct_suite callback
46 module in order to define the callback interface for a test suite.
47
49 Module:all() -> [ct_test_def()] | {skip, Reason}
50 Types:
52 ct_test_def() = TestCase | {group, GroupName} | {group,
54 GroupName, Properties} | {group, GroupName, Properties, Sub‐
55 Groups}
56 TestCase = ct_testname()
57 GroupName = ct_groupname()
58 Properties = [parallel | sequence | Shuffle | {RepeatType,
59 N}] | default
60 SubGroups = [{GroupName, Properties} | {GroupName, Proper‐
61 ties, SubGroups}]
62 Shuffle = shuffle | {shuffle, Seed}
63 Seed = {integer(), integer(), integer()}
64 RepeatType = repeat | repeat_until_all_ok |
65 repeat_until_all_fail | repeat_until_any_ok |
66 repeat_until_any_fail
67 N = integer() | forever
68 Reason = term()
69 MANDATORY
71 Returns the list of all test cases and test case groups in the
73 test suite module to be executed. This list also specifies the
74 order the cases and groups are executed by Common Test. A test
75 case is represented by an atom, the name of the test case func‐
76 tion, or a testcase tuple indicating that the test case shall be
77 repeated. A test case group is represented by a group tuple,
78 where GroupName, an atom, is the name of the group (defined in
79 Module:groups/0). Execution properties for groups can also be
80 specified, both for a top-level group and for any of its sub‐
81 groups. Group execution properties specified here override prop‐
82 erties in the group definition (see Module:groups/0). (With
83 value default, the group definition properties are used).
84 If {skip, Reason} is returned, all test cases in the module are
86 skipped and Reason is printed on the HTML result page.
87 For details on groups, see section Test Case Groups in the
89 User's Guide.
90 Module:groups() -> [ct_group_def()]
92 Types:
94 ct_group_def() = {GroupName, Properties, GroupsAndTestCases}
96 GroupName = ct_groupname()
97 Properties = [parallel | sequence | Shuffle | {RepeatType,
98 N}]
99 GroupsAndTestCases = [Group | {group, GroupName} | TestCase]
100 TestCase = ct_testname()
101 Shuffle = shuffle | {shuffle, Seed}
102 Seed = {integer(), integer(), integer()}
103 RepeatType = repeat | repeat_until_all_ok |
104 repeat_until_all_fail | repeat_until_any_ok |
105 repeat_until_any_fail
106 N = integer() | forever
107 OPTIONAL
109 Defines test case groups. For details, see section Test Case
111 Groups in the User's Guide.
112 Module:suite() -> [ct_info()]
114 Types:
116 ct_info() = {timetrap, Time} | {require, Required} |
118 {require, Name, Required} | {userdata, UserData} |
119 {silent_connections, Conns} | {stylesheet, CSSFile} |
120 {ct_hooks, CTHs}
121 Time = TimeVal | TimeFunc
122 TimeVal = MilliSec | {seconds, integer()} | {minutes, inte‐
123 ger()} | {hours, integer()}
124 TimeFunc = {Mod, Func, Args} | Fun
125 MilliSec = integer()
126 Mod = atom()
127 Func = atom()
128 Args = list()
129 Fun = fun()
130 Required = Key | {Key, SubKeys} | {Key, SubKey} | {Key, Sub‐
131 Key, SubKeys}
132 Key = atom()
133 SubKeys = SubKey | [SubKey]
134 SubKey = atom()
135 Name = atom()
136 UserData = term()
137 Conns = [atom()]
138 CSSFile = string()
139 CTHs = [CTHModule |
140 {CTHModule, CTHInitArgs} |
141 {CTHModule, CTHInitArgs, CTHPriority}]
142 CTHModule = atom()
143 CTHInitArgs = term()
144 CTHPriority = integer()
145 OPTIONAL
147 The test suite information function. Returns a list of tagged
149 tuples specifying various properties related to the execution of
150 this test suite (common for all test cases in the suite).
151 Tag timetrap sets the maximum time that each test case is
153 allowed to execute (including Module:init_per_testcase/2 and
154 Module:end_per_testcase/2). If the timetrap time is exceeded,
155 the test case fails with reason timetrap_timeout. A TimeFunc
156 function can be used to set a new timetrap by returning a
157 TimeVal. It can also be used to trigger a timetrap time-out by,
158 at some point, returning a value other than a TimeVal. For
159 details, see section Timetrap Time-Outs in the User's Guide.
160 Tag require specifies configuration variables required by test
162 cases (or configuration functions) in the suite. If the required
163 configuration variables are not found in any of the configura‐
164 tion files, all test cases are skipped. For details about the
165 require functionality, see funtion ct:require/1,2.
166 With userdata, the user can specify any test suite-related
168 information, which can be read by calling ct:userdata/2.
169 Tag ct_hooks specifies the Common Test Hooks to be run with this
171 suite.
172 Other tuples than the ones defined are ignored.
174 For details about the test suite information function, see sec‐
176 tion Test Suite Information Function in the User's Guide.
177 Module:init_per_suite(Config) -> NewConfig | {skip, Reason} |
179 {skip_and_save, Reason, SaveConfig}
180 Types:
182 Config = NewConfig = SaveConfig = ct_config()
184 Reason = term()
185 OPTIONAL; if this function is defined, then Mod‐
187 ule:end_per_suite/1 must also be defined.
188 This configuration function is called as the first function in
190 the suite. It typically contains initializations that are common
191 for all test cases in the suite, and that must only be done
192 once. Parameter Config is the configuration data that can be
193 modified. Whatever is returned from this function is specified
194 as Config to all configuration functions and test cases in the
195 suite.
196 If {skip, Reason} is returned, all test cases in the suite are
198 skipped and Reason is printed in the overview log for the suite.
199 For information on save_config and skip_and_save, see section
201 Saving Configuration Data in the User's Guide.
202 Module:end_per_suite(Config) -> term() | {save_config, SaveConfig}
204 Types:
206 Config = SaveConfig = ct_config()
208 OPTIONAL; if this function is defined, then Mod‐
210 ule:init_per_suite/1 must also be defined.
211 This function is called as the last test case in the suite. It
213 is meant to be used for cleaning up after Mod‐
214 ule:init_per_suite/1.
215 For information on save_config, see section Saving Configuration
217 Data in the User's Guide.
218 Module:group(GroupName) -> [ct_info()]
220 Types:
222 GroupName = ct_groupname()
224 ct_info() = {timetrap, Time} | {require, Required} |
225 {require, Name, Required} | {userdata, UserData} |
226 {silent_connections, Conns} | {stylesheet, CSSFile} |
227 {ct_hooks, CTHs}
228 Time = TimeVal | TimeFunc
229 TimeVal = MilliSec | {seconds, integer()} | {minutes, inte‐
230 ger()} | {hours, integer()}
231 TimeFunc = {Mod, Func, Args} | Fun
232 MilliSec = integer()
233 Mod = atom()
234 Func = atom()
235 Args = list()
236 Fun = fun()
237 Required = Key | {Key, SubKeys} | {Key, SubKey} | {Key, Sub‐
238 Key, SubKeys}
239 Key = atom()
240 SubKeys = SubKey | [SubKey]
241 SubKey = atom()
242 Name = atom()
243 UserData = term()
244 Conns = [atom()]
245 CSSFile = string()
246 CTHs = [CTHModule |
247 {CTHModule, CTHInitArgs} |
248 {CTHModule, CTHInitArgs, CTHPriority}]
249 CTHModule = atom()
250 CTHInitArgs = term()
251 CTHPriority = integer()
252 OPTIONAL
254 The test case group information function. It is supposed to
256 return a list of tagged tuples that specify various properties
257 related to the execution of a test case group (that is, its test
258 cases and subgroups). Properties set by Module:group/1 override
259 properties with the same key that have been set previously by
260 Module:suite/0.
261 Tag timetrap sets the maximum time that each test case is
263 allowed to execute (including Module:init_per_testcase/2 and
264 Module:end_per_testcase/2). If the timetrap time is exceeded,
265 the test case fails with reason timetrap_timeout. A TimeFunc
266 function can be used to set a new timetrap by returning a
267 TimeVal. It can also be used to trigger a timetrap time-out by,
268 at some point, returning a value other than a TimeVal. For
269 details, see section Timetrap Time-Outs in the User's Guide.
270 Tag require specifies configuration variables required by test
272 cases (or configuration functions) in the suite. If the required
273 configuration variables are not found in any of the configura‐
274 tion files, all test cases in this group are skipped. For
275 details about the require functionality, see function
276 ct:require/1,2.
277 With userdata, the user can specify any test case group related
279 information that can be read by calling ct:userdata/2.
280 Tag ct_hooks specifies the Common Test Hooks to be run with this
282 suite.
283 Other tuples than the ones defined are ignored.
285 For details about the test case group information function, see
287 section Group Information Function in the User's Guide.
288 Module:init_per_group(GroupName, Config) -> NewConfig | {skip, Reason}
290 Types:
292 GroupName = ct_groupname()
294 Config = NewConfig = ct_config()
295 Reason = term()
296 OPTIONAL; if this function is defined, then Mod‐
298 ule:end_per_group/2 must also be defined.
299 This configuration function is called before execution of a test
301 case group. It typically contains initializations that are com‐
302 mon for all test cases and subgroups in the group, and that must
303 only be performed once. GroupName is the name of the group, as
304 specified in the group definition (see Module:groups/0). Parame‐
305 ter Config is the configuration data that can be modified. The
306 return value of this function is given as Config to all test
307 cases and subgroups in the group.
308 If {skip, Reason} is returned, all test cases in the group are
310 skipped and Reason is printed in the overview log for the group.
311 For information about test case groups, see section Test Case
313 Groups in the User's Guide.
314 Module:end_per_group(GroupName, Config) -> term() |
316 {return_group_result, Status}
317 Types:
319 GroupName = ct_groupname()
321 Config = ct_config()
322 Status = ct_status()
323 OPTIONAL; if this function is defined, then Mod‐
325 ule:init_per_group/2 must also be defined.
326 This function is called after the execution of a test case group
328 is finished. It is meant to be used for cleaning up after Mod‐
329 ule:init_per_group/2. A status value for a nested subgroup can
330 be returned with {return_group_result, Status}. The status can
331 be retrieved in Module:end_per_group/2 for the group on the
332 level above. The status is also used by Common Test for deciding
333 if execution of a group is to proceed if property sequence or
334 repeat_until_* is set.
335 For details about test case groups, see section Test Case Groups
337 in the User's Guide.
338 Module:init_per_testcase(TestCase, Config) -> NewConfig | {fail, Rea‐
340 son} | {skip, Reason}
341 Types:
343 TestCase = ct_testname()
345 Config = NewConfig = ct_config()
346 Reason = term()
347 OPTIONAL; if this function is defined, then Module:end_per_test‐
349 case/2 must also be defined.
350 This function is called before each test case. Argument TestCase
352 is the test case name, and Config (list of key-value tuples) is
353 the configuration data that can be modified. The NewConfig list
354 returned from this function is given as Config to the test case.
355 If {fail, Reason} is returned, the test case is marked as failed
356 without being executed.
357 If {skip, Reason} is returned, the test case is skipped and Rea‐
359 son is printed in the overview log for the suite.
360 Module:end_per_testcase(TestCase, Config) -> term() | {fail, Reason} |
362 {save_config, SaveConfig}
363 Types:
365 TestCase = ct_testname()
367 Config = SaveConfig = ct_config()
368 Reason = term()
369 OPTIONAL; if this function is defined, then Mod‐
371 ule:init_per_testcase/2 must also be defined.
372 This function is called after each test case, and can be used to
374 clean up after Module:init_per_testcase/2 and the test case. Any
375 return value (besides {fail, Reason} and {save_config, SaveCon‐
376 fig}) is ignored. By returning {fail, Reason}, TestCase is
377 marked as faulty (even though it was successful in the sense
378 that it returned a value instead of terminating).
379 For information on save_config, see section Saving Configuration
381 Data in the User's Guide.
382 Module:Testcase() -> [ct_info()]
384 Types:
386 ct_info() = {timetrap, Time} | {require, Required} |
388 {require, Name, Required} | {userdata, UserData} |
389 {silent_connections, Conns} | {stylesheet, CSSFile} |
390 {ct_hooks, CTHs}
391 Time = TimeVal | TimeFunc
392 TimeVal = MilliSec | {seconds, integer()} | {minutes, inte‐
393 ger()} | {hours, integer()}
394 TimeFunc = {Mod, Func, Args} | Fun
395 MilliSec = integer()
396 Mod = atom()
397 Func = atom()
398 Args = list()
399 Fun = fun()
400 Required = Key | {Key, SubKeys} | {Key, SubKey} | {Key, Sub‐
401 Key, SubKeys}
402 Key = atom()
403 SubKeys = SubKey | [SubKey]
404 SubKey = atom()
405 Name = atom()
406 UserData = term()
407 Conns = [atom()]
408 CSSFile = string()
409 CTHs = [CTHModule |
410 {CTHModule, CTHInitArgs} |
411 {CTHModule, CTHInitArgs, CTHPriority}]
412 CTHModule = atom()
413 CTHInitArgs = term()
414 CTHPriority = integer()
415 OPTIONAL
417 The test case information function. It is supposed to return a
419 list of tagged tuples that specify various properties related to
420 the execution of this particular test case. Properties set by
421 Module:Testcase/0 override properties set previously for the
422 test case by Module:group/1 or Module:suite/0.
423 Tag timetrap sets the maximum time that the test case is allowed
425 to execute. If the timetrap time is exceeded, the test case
426 fails with reason timetrap_timeout. Module:init_per_testcase/2
427 and Module:end_per_testcase/2 are included in the timetrap time.
428 A TimeFunc function can be used to set a new timetrap by return‐
429 ing a TimeVal. It can also be used to trigger a timetrap time-
430 out by, at some point, returning a value other than a TimeVal.
431 For details, see section Timetrap Time-Outs in the User's Guide.
432 Tag require specifies configuration variables that are required
434 by the test case (or init_per_testcase/2 or end_per_testcase/2).
435 If the required configuration variables are not found in any of
436 the configuration files, the test case is skipped. For details
437 about the require functionality, see function ct:require/1,2.
438 If timetrap or require is not set, the default values specified
440 by Module:suite/0 (or Module:group/1) are used.
441 With userdata, the user can specify any test case-related infor‐
443 mation that can be read by calling ct:userdata/3.
444 Other tuples than the ones defined are ignored.
446 For details about the test case information function, see sec‐
448 tion Test Case Information Function in the User's Guide.
449 Module:Testcase(Config) -> term() | {skip, Reason} | {comment, Comment}
451 | {save_config, SaveConfig} | {skip_and_save, Reason, SaveConfig} |
452 exit()
453 Types:
455 Config = SaveConfig = ct_config()
457 Reason = term()
458 Comment = string()
459 MANDATORY
461 The implementation of a test case. Call the functions to test
463 and check the result. If something fails, ensure the function
464 causes a runtime error or call ct:fail/1,2 (which also causes
465 the test case process to terminate).
466 Elements from the Config list can, for example, be read with
468 proplists:get_value/2 in STDLIB (or the macro ?config defined in
469 ct.hrl).
470 If you decide not to run the test case after all, return {skip,
472 Reason}. Reason is then printed in field Comment on the HTML
473 result page.
474 To print some information in field Comment on the HTML result
476 page, return {comment, Comment}.
477 If the function returns anything else, the test case is consid‐
479 ered successful. The return value always gets printed in the
480 test case log file.
481 For details about test case implementation, see section Test
483 Cases in the User's Guide.
484 For information on save_config and skip_and_save, see section
486 Saving Configuration Data in the User's Guide.
487Ericsson AB common_test 1.20 ct_suite(3)