1ct_suite(3)                Erlang Module Definition                ct_suite(3)
2
3
4

NAME

6       ct_suite - -behaviour(ct_suite).
7
8

DESCRIPTION

10       The  following  section describes the mandatory and optional test suite
11       functions that Common Test calls during test execution.  For  more  de‐
12       tails, see section Writing Test Suites in the User's Guide.
13

DATA TYPES

15       ct_testname() = atom()
16
17              The name of the testcase function.
18
19       ct_groupname() = atom()
20
21              The name of the test group.
22
23       ct_config() = [{Key :: atom(), Value :: term()}]
24
25              The configuration data that can be modified.
26
27       ct_status() = ok | skipped | failed
28
29              The status value for a nested subgroup.
30
31       ct_group_def()
32
33              The test group definition, as returned by Module:groups/0.
34
35       ct_test_def()
36
37              The test suite definition, as returned by Module:all/0.
38
39       ct_info()
40
41              The  test suite information, as returned by Module:suite/0, Mod‐
42              ule:group/1 and Module:Testcase/0.
43

CALLBACK FUNCTIONS

45       The following functions are to be exported  from  a  ct_suite  callback
46       module in order to define the callback interface for a test suite.
47

EXPORTS

49       Module:all() -> [ct_test_def()] | {skip, Reason}
50
51              Types:
52
53                 ct_test_def()  =  TestCase  |  {group,  GroupName}  | {group,
54                 GroupName, Properties} | {group, GroupName, Properties,  Sub‐
55                 Groups}
56                 TestCase = ct_testname()
57                 GroupName = ct_groupname()
58                 Properties  =  [parallel  | sequence | Shuffle | {RepeatType,
59                 N}] | default
60                 SubGroups = [{GroupName, Properties}  |  {GroupName,  Proper‐
61                 ties, SubGroups}]
62                 Shuffle = shuffle | {shuffle, Seed}
63                 Seed = {integer(), integer(), integer()}
64                 RepeatType   =  repeat  |  repeat_until_all_ok  |  repeat_un‐
65                 til_all_fail | repeat_until_any_ok | repeat_until_any_fail
66                 N = integer() | forever
67                 Reason = term()
68
69              MANDATORY
70
71              Returns the list of all test cases and test case groups  in  the
72              test  suite  module to be executed. This list also specifies the
73              order the cases and groups are executed by Common Test.  A  test
74              case  is represented by an atom, the name of the test case func‐
75              tion, or a testcase tuple indicating that the test case shall be
76              repeated.  A  test  case  group is represented by a group tuple,
77              where GroupName, an atom, is the name of the group  (defined  in
78              Module:groups/0).  Execution  properties  for groups can also be
79              specified, both for a top-level group and for any  of  its  sub‐
80              groups. Group execution properties specified here override prop‐
81              erties in the  group  definition  (see  Module:groups/0).  (With
82              value default, the group definition properties are used).
83
84              If  {skip, Reason} is returned, all test cases in the module are
85              skipped and Reason is printed on the HTML result page.
86
87              For details on groups, see  section  Test  Case  Groups  in  the
88              User's Guide.
89
90       Module:groups() -> [ct_group_def()]
91
92              Types:
93
94                 ct_group_def() = {GroupName, Properties, GroupsAndTestCases}
95                 GroupName = ct_groupname()
96                 Properties  =  [parallel  | sequence | Shuffle | {RepeatType,
97                 N}]
98                 GroupsAndTestCases = [Group | {group, GroupName} | TestCase]
99                 TestCase = ct_testname()
100                 Shuffle = shuffle | {shuffle, Seed}
101                 Seed = {integer(), integer(), integer()}
102                 RepeatType  =  repeat  |  repeat_until_all_ok  |   repeat_un‐
103                 til_all_fail | repeat_until_any_ok | repeat_until_any_fail
104                 N = integer() | forever
105
106              OPTIONAL
107
108              Defines  test  case  groups.  For details, see section Test Case
109              Groups in the User's Guide.
110
111       Module:suite() -> [ct_info()]
112
113              Types:
114
115                 ct_info() = {timetrap, Time} |  {require,  Required}  |  {re‐
116                 quire,  Name, Required} | {userdata, UserData} | {silent_con‐
117                 nections, Conns} | {stylesheet, CSSFile} | {ct_hooks, CTHs}
118                 Time = TimeVal | TimeFunc
119                 TimeVal = MilliSec | {seconds, integer()} |  {minutes,  inte‐
120                 ger()} | {hours, integer()}
121                 TimeFunc = {Mod, Func, Args} | Fun
122                 MilliSec = integer()
123                 Mod = atom()
124                 Func = atom()
125                 Args = list()
126                 Fun = fun()
127                 Required  = Key | {Key, SubKeys} | {Key, SubKey} | {Key, Sub‐
128                 Key, SubKeys}
129                 Key = atom()
130                 SubKeys = SubKey | [SubKey]
131                 SubKey = atom()
132                 Name = atom()
133                 UserData = term()
134                 Conns = [atom()]
135                 CSSFile = string()
136                 CTHs = [CTHModule |
137                  {CTHModule, CTHInitArgs} |
138                  {CTHModule, CTHInitArgs, CTHPriority}]
139                 CTHModule = atom()
140                 CTHInitArgs = term()
141                 CTHPriority = integer()
142
143              OPTIONAL
144
145              The test suite information function. Returns a  list  of  tagged
146              tuples specifying various properties related to the execution of
147              this test suite (common for all test cases in the suite).
148
149              Tag timetrap sets the maximum time that each test  case  is  al‐
150              lowed  to execute (including Module:init_per_testcase/2 and Mod‐
151              ule:end_per_testcase/2). If the timetrap time is  exceeded,  the
152              test  case  fails with reason timetrap_timeout. A TimeFunc func‐
153              tion can be used to set a new timetrap by returning  a  TimeVal.
154              It  can  also be used to trigger a timetrap time-out by, at some
155              point, returning a value other than a TimeVal. For details,  see
156              section Timetrap Time-Outs in the User's Guide.
157
158              Tag  require  specifies configuration variables required by test
159              cases (or configuration functions) in the suite. If the required
160              configuration  variables  are not found in any of the configura‐
161              tion files, all test cases are skipped. For  details  about  the
162              require functionality, see function ct:require/1,2.
163
164              With  userdata,  the user can specify any test suite-related in‐
165              formation, which can be read by calling ct:userdata/2.
166
167              Tag ct_hooks specifies the Common Test Hooks to be run with this
168              suite.
169
170              Other tuples than the ones defined are ignored.
171
172              For  details about the test suite information function, see sec‐
173              tion Test Suite Information Function in the User's Guide.
174
175       Module:init_per_suite(Config)  ->  NewConfig   |   {skip,   Reason}   |
176       {skip_and_save, Reason, SaveConfig}
177
178              Types:
179
180                 Config = NewConfig = SaveConfig = ct_config()
181                 Reason = term()
182
183              OPTIONAL;    if    this   function   is   defined,   then   Mod‐
184              ule:end_per_suite/1 must also be defined.
185
186              This configuration function is called as the first  function  in
187              the suite. It typically contains initializations that are common
188              for all test cases in the suite, and  that  must  only  be  done
189              once.  Parameter  Config  is  the configuration data that can be
190              modified. Whatever is returned from this function  is  specified
191              as  Config  to all configuration functions and test cases in the
192              suite.
193
194              If {skip, Reason} is returned, all test cases in the  suite  are
195              skipped and Reason is printed in the overview log for the suite.
196
197              For  information  on  save_config and skip_and_save, see section
198              Saving Configuration Data in the User's Guide.
199
200       Module:end_per_suite(Config) -> term() | {save_config, SaveConfig}
201
202              Types:
203
204                 Config = SaveConfig = ct_config()
205
206              OPTIONAL;   if   this   function   is   defined,    then    Mod‐
207              ule:init_per_suite/1 must also be defined.
208
209              This  function  is called as the last test case in the suite. It
210              is   meant   to   be   used   for   cleaning   up   after   Mod‐
211              ule:init_per_suite/1.
212
213              For information on save_config, see section Saving Configuration
214              Data in the User's Guide.
215
216       Module:group(GroupName) -> [ct_info()]
217
218              Types:
219
220                 GroupName = ct_groupname()
221                 ct_info() = {timetrap, Time} |  {require,  Required}  |  {re‐
222                 quire,  Name, Required} | {userdata, UserData} | {silent_con‐
223                 nections, Conns} | {stylesheet, CSSFile} | {ct_hooks, CTHs}
224                 Time = TimeVal | TimeFunc
225                 TimeVal = MilliSec | {seconds, integer()} |  {minutes,  inte‐
226                 ger()} | {hours, integer()}
227                 TimeFunc = {Mod, Func, Args} | Fun
228                 MilliSec = integer()
229                 Mod = atom()
230                 Func = atom()
231                 Args = list()
232                 Fun = fun()
233                 Required  = Key | {Key, SubKeys} | {Key, SubKey} | {Key, Sub‐
234                 Key, SubKeys}
235                 Key = atom()
236                 SubKeys = SubKey | [SubKey]
237                 SubKey = atom()
238                 Name = atom()
239                 UserData = term()
240                 Conns = [atom()]
241                 CSSFile = string()
242                 CTHs = [CTHModule |
243                  {CTHModule, CTHInitArgs} |
244                  {CTHModule, CTHInitArgs, CTHPriority}]
245                 CTHModule = atom()
246                 CTHInitArgs = term()
247                 CTHPriority = integer()
248
249              OPTIONAL
250
251              The test case group information function. It is supposed to  re‐
252              turn a list of tagged tuples that specify various properties re‐
253              lated to the execution of a test case group (that is,  its  test
254              cases  and subgroups). Properties set by Module:group/1 override
255              properties with the same key that have been  set  previously  by
256              Module:suite/0.
257
258              Tag  timetrap  sets  the maximum time that each test case is al‐
259              lowed to execute (including Module:init_per_testcase/2 and  Mod‐
260              ule:end_per_testcase/2).  If  the timetrap time is exceeded, the
261              test case fails with reason timetrap_timeout. A  TimeFunc  func‐
262              tion  can  be used to set a new timetrap by returning a TimeVal.
263              It can also be used to trigger a timetrap time-out by,  at  some
264              point,  returning a value other than a TimeVal. For details, see
265              section Timetrap Time-Outs in the User's Guide.
266
267              Tag require specifies configuration variables required  by  test
268              cases (or configuration functions) in the suite. If the required
269              configuration variables are not found in any of  the  configura‐
270              tion  files,  all  test cases in this group are skipped. For de‐
271              tails about  the  require  functionality,  see  function  ct:re‐
272              quire/1,2.
273
274              With  userdata, the user can specify any test case group related
275              information that can be read by calling ct:userdata/2.
276
277              Tag ct_hooks specifies the Common Test Hooks to be run with this
278              suite.
279
280              Other tuples than the ones defined are ignored.
281
282              For  details about the test case group information function, see
283              section Group Information Function in the User's Guide.
284
285       Module:init_per_group(GroupName, Config) -> NewConfig | {skip, Reason}
286
287              Types:
288
289                 GroupName = ct_groupname()
290                 Config = NewConfig = ct_config()
291                 Reason = term()
292
293              OPTIONAL;   if   this   function   is   defined,    then    Mod‐
294              ule:end_per_group/2 must also be defined.
295
296              This configuration function is called before execution of a test
297              case group. It typically contains initializations that are  com‐
298              mon for all test cases and subgroups in the group, and that must
299              only be performed once. GroupName is the name of the  group,  as
300              specified in the group definition (see Module:groups/0). Parame‐
301              ter Config is the configuration data that can be  modified.  The
302              return  value  of  this  function is given as Config to all test
303              cases and subgroups in the group.
304
305              If {skip, Reason} is returned, all test cases in the  group  are
306              skipped and Reason is printed in the overview log for the group.
307
308              For  information  about  test case groups, see section Test Case
309              Groups in the User's Guide.
310
311       Module:end_per_group(GroupName, Config) -> term()  |  {return_group_re‐
312       sult, Status}
313
314              Types:
315
316                 GroupName = ct_groupname()
317                 Config = ct_config()
318                 Status = ct_status()
319
320              OPTIONAL;    if    this   function   is   defined,   then   Mod‐
321              ule:init_per_group/2 must also be defined.
322
323              This function is called after the execution of a test case group
324              is  finished.  It is meant to be used for cleaning up after Mod‐
325              ule:init_per_group/2. A status value for a nested  subgroup  can
326              be  returned  with {return_group_result, Status}. The status can
327              be retrieved in Module:end_per_group/2  for  the  group  on  the
328              level above. The status is also used by Common Test for deciding
329              if execution of a group is to proceed if  property  sequence  or
330              repeat_until_* is set.
331
332              For details about test case groups, see section Test Case Groups
333              in the User's Guide.
334
335       Module:init_per_testcase(TestCase, Config) -> NewConfig |  {fail,  Rea‐
336       son} | {skip, Reason}
337
338              Types:
339
340                 TestCase = ct_testname()
341                 Config = NewConfig = ct_config()
342                 Reason = term()
343
344              OPTIONAL; if this function is defined, then Module:end_per_test‐
345              case/2 must also be defined.
346
347              This function is called before each test case. Argument TestCase
348              is  the test case name, and Config (list of key-value tuples) is
349              the configuration data that can be modified. The NewConfig  list
350              returned from this function is given as Config to the test case.
351              If {fail, Reason} is returned, the test case is marked as failed
352              without being executed.
353
354              If {skip, Reason} is returned, the test case is skipped and Rea‐
355              son is printed in the overview log for the suite.
356
357       Module:end_per_testcase(TestCase, Config) -> term() | {fail, Reason}  |
358       {save_config, SaveConfig}
359
360              Types:
361
362                 TestCase = ct_testname()
363                 Config = SaveConfig = ct_config()
364                 Reason = term()
365
366              OPTIONAL;    if    this   function   is   defined,   then   Mod‐
367              ule:init_per_testcase/2 must also be defined.
368
369              This function is called after each test case, and can be used to
370              clean up after Module:init_per_testcase/2 and the test case. Any
371              return value (besides {fail, Reason} and {save_config,  SaveCon‐
372              fig})  is  ignored.  By  returning  {fail,  Reason}, TestCase is
373              marked as faulty (even though it was  successful  in  the  sense
374              that it returned a value instead of terminating).
375
376              For information on save_config, see section Saving Configuration
377              Data in the User's Guide.
378
379       Module:Testcase() -> [ct_info()]
380
381              Types:
382
383                 ct_info() = {timetrap, Time} |  {require,  Required}  |  {re‐
384                 quire,  Name, Required} | {userdata, UserData} | {silent_con‐
385                 nections, Conns} | {stylesheet, CSSFile} | {ct_hooks, CTHs}
386                 Time = TimeVal | TimeFunc
387                 TimeVal = MilliSec | {seconds, integer()} |  {minutes,  inte‐
388                 ger()} | {hours, integer()}
389                 TimeFunc = {Mod, Func, Args} | Fun
390                 MilliSec = integer()
391                 Mod = atom()
392                 Func = atom()
393                 Args = list()
394                 Fun = fun()
395                 Required  = Key | {Key, SubKeys} | {Key, SubKey} | {Key, Sub‐
396                 Key, SubKeys}
397                 Key = atom()
398                 SubKeys = SubKey | [SubKey]
399                 SubKey = atom()
400                 Name = atom()
401                 UserData = term()
402                 Conns = [atom()]
403                 CSSFile = string()
404                 CTHs = [CTHModule |
405                  {CTHModule, CTHInitArgs} |
406                  {CTHModule, CTHInitArgs, CTHPriority}]
407                 CTHModule = atom()
408                 CTHInitArgs = term()
409                 CTHPriority = integer()
410
411              OPTIONAL
412
413              The test case information function. It is supposed to  return  a
414              list of tagged tuples that specify various properties related to
415              the execution of this particular test case.  Properties  set  by
416              Module:Testcase/0  override  properties  set  previously for the
417              test case by Module:group/1 or Module:suite/0.
418
419              Tag timetrap sets the maximum time that the test case is allowed
420              to  execute.  If  the  timetrap  time is exceeded, the test case
421              fails with reason  timetrap_timeout.  Module:init_per_testcase/2
422              and Module:end_per_testcase/2 are included in the timetrap time.
423              A TimeFunc function can be used to set a new timetrap by return‐
424              ing  a  TimeVal. It can also be used to trigger a timetrap time-
425              out by, at some point, returning a value other than  a  TimeVal.
426              For details, see section Timetrap Time-Outs in the User's Guide.
427
428              Tag  require specifies configuration variables that are required
429              by the test case (or init_per_testcase/2 or end_per_testcase/2).
430              If  the required configuration variables are not found in any of
431              the configuration files, the test case is skipped.  For  details
432              about the require functionality, see function ct:require/1,2.
433
434              If  timetrap or require is not set, the default values specified
435              by Module:suite/0 (or Module:group/1) are used.
436
437              With userdata, the user can specify any test case-related infor‐
438              mation that can be read by calling ct:userdata/3.
439
440              Other tuples than the ones defined are ignored.
441
442              For  details  about the test case information function, see sec‐
443              tion Test Case Information Function in the User's Guide.
444
445       Module:Testcase(Config) -> term() | {skip, Reason} | {comment, Comment}
446       |  {save_config,  SaveConfig}  |  {skip_and_save, Reason, SaveConfig} |
447       exit()
448
449              Types:
450
451                 Config = SaveConfig = ct_config()
452                 Reason = term()
453                 Comment = string()
454
455              MANDATORY
456
457              The implementation of a test case. Call the  functions  to  test
458              and  check  the  result. If something fails, ensure the function
459              causes a runtime error or call ct:fail/1,2  (which  also  causes
460              the test case process to terminate).
461
462              Elements  from  the  Config  list can, for example, be read with
463              proplists:get_value/2 in STDLIB (or the macro ?config defined in
464              ct.hrl).
465
466              If  you decide not to run the test case after all, return {skip,
467              Reason}. Reason is then printed in field Comment on the HTML re‐
468              sult page.
469
470              To  print  some  information in field Comment on the HTML result
471              page, return {comment, Comment}.
472
473              If the function returns anything else, the test case is  consid‐
474              ered  successful.  The  return  value always gets printed in the
475              test case log file.
476
477              For details about test case  implementation,  see  section  Test
478              Cases in the User's Guide.
479
480              For  information  on  save_config and skip_and_save, see section
481              Saving Configuration Data in the User's Guide.
482
483
484
485Ericsson AB                   common_test 1.23.3                   ct_suite(3)
Impressum