1ct_hooks(3)                Erlang Module Definition                ct_hooks(3)
2
3
4

NAME

6       ct_hooks - A callback interface on top of Common Test.
7

DESCRIPTION

9       The  Common  Test Hook (CTH) framework allows extensions of the default
10       behavior of Common Test by callbacks before and after  all  test  suite
11       calls.  It  is  intended  for advanced users of Common Test who want to
12       abstract out behavior that is common to multiple test suites.
13
14       In brief, CTH allows you to:
15
16         * Manipulate the runtime configuration before each  suite  configura‐
17           tion call.
18
19         * Manipulate  the  return  of  all  suite  configuration calls and by
20           extension the result of the test themselves.
21
22       The following sections describe the mandatory and  optional  CTH  func‐
23       tions  that  Common Test calls during test execution. For more details,
24       see section Common Test Hooks in the User's Guide.
25
26       For information about how to add a  CTH  to  your  suite,  see  section
27       Installing a CTH in the User's Guide.
28
29   Note:
30       For  a  minimal example of a CTH, see section Example CTH in the User's
31       Guide.
32
33

CALLBACK FUNCTIONS

35       The following functions define the callback interface for a CTH.
36

EXPORTS

38       Module:init(Id, Opts) -> {ok, State} | {ok, State, Priority}
39
40              Types:
41
42                 Id = reference() | term()
43                 Opts = term()
44                 State = term()
45                 Priority = integer()
46
47              MANDATORY
48
49              This function is always called before any other  callback  func‐
50              tion.  Use  it  to  initiate any common state. It is to return a
51              state for this CTH.
52
53              Id is either the return value of ct_hooks:id/1, or  a  reference
54              (created  using  erlang:make_ref/0  in ERTS) if ct_hooks:id/1 is
55              not implemented.
56
57              Priority is the relative priority of this  hook.  Hooks  with  a
58              lower  priority are executed first. If no priority is specified,
59              it is set to 0.
60
61              For details about when init is called, see section CTH Scope  in
62              the User's Guide.
63
64       Module:post_groups(SuiteName, GroupDefs) -> NewGroupDefs
65
66              Types:
67
68                 SuiteName = atom()
69                 GroupDefs = NewGroupDefs = [Group]
70                 Group = {GroupName,Properties,GroupsAndTestCases}
71                 GroupName = atom()
72                 Properties  =  [parallel | sequence | Shuffle | {GroupRepeat‐
73                 Type,N}]
74                 GroupsAndTestCases = [Group | {group,GroupName} | TestCase  |
75                 {testcase,TestCase,TCRepeatProps}]
76                 TestCase = atom()
77                 TCRepeatProps   =   [{repeat,N}   |   {repeat_until_ok,N}   |
78                 {repeat_until_fail,N}]
79                 Shuffle = shuffle | {shuffle,Seed}
80                 Seed = {integer(),integer(),integer()}
81                 GroupRepeatType   =   repeat    |    repeat_until_all_ok    |
82                 repeat_until_all_fail       |      repeat_until_any_ok      |
83                 repeat_until_any_fail
84                 N = integer() | forever
85
86              OPTIONAL
87
88              This function is called after groups/0. It is used to modify the
89              test  group definitions, for instance to add or remove groups or
90              change group properties.
91
92              GroupDefs is what groups/0 returned, that is, a  list  of  group
93              definitions.
94
95              NewGroupDefs is the possibly modified version of this list.
96
97              This  function  is  called  only  if  the  CTH  is  added before
98              init_per_suite is run. For details, see section CTH Scope in the
99              User's Guide.
100
101              Notice  that for CTHs that are installed by means of the suite/0
102              function, post_groups/2 is called before the init/2  hook  func‐
103              tion.  However,  for  CTHs that are installed by means of the CT
104              start flag, the init/2 function is called first.
105
106          Note:
107              Prior to each test execution, Common Test does a simulated  test
108              run  in order to count test suites, groups and cases for logging
109              purposes. This causes the post_groups/2 hook function to  always
110              be  called twice. For this reason, side effects are best avoided
111              in this callback.
112
113
114       Module:post_all(SuiteName, Return, GroupDefs) -> NewReturn
115
116              Types:
117
118                 SuiteName = atom()
119                 Return = NewReturn = Tests | {skip,Reason}
120                 Tests  =  [TestCase  |  {testcase,TestCase,TCRepeatProps}   |
121                 {group,GroupName}     |     {group,GroupName,Properties}    |
122                 {group,GroupName,Properties,SubGroups}]
123                 TestCase = atom()
124                 TCRepeatProps   =   [{repeat,N}   |   {repeat_until_ok,N}   |
125                 {repeat_until_fail,N}]
126                 GroupName = atom()
127                 Properties = GroupProperties | default
128                 SubGroups   =  [{GroupName,Properties}  |  {GroupName,Proper‐
129                 ties,SubGroups}]
130                 Shuffle = shuffle | {shuffle,Seed}
131                 Seed = {integer(),integer(),integer()}
132                 GroupRepeatType   =   repeat    |    repeat_until_all_ok    |
133                 repeat_until_all_fail       |      repeat_until_any_ok      |
134                 repeat_until_any_fail
135                 N = integer() | forever
136                 GroupDefs = NewGroupDefs = [Group]
137                 Group = {GroupName,GroupProperties,GroupsAndTestCases}
138                 GroupProperties = [parallel | sequence | Shuffle |  {GroupRe‐
139                 peatType,N}]
140                 GroupsAndTestCases = [Group | {group,GroupName} | TestCase]
141                 Reason = term()
142
143              OPTIONAL
144
145              This  function  is  called after all/0. It is used to modify the
146              set of test cases and test group to be executed, for instance to
147              add or remove test cases and groups, change group properties, or
148              even skip all tests in the suite.
149
150              Return is what all/0 returned, that is, a list of test cases and
151              groups to be executed, or a tuple {skip,Reason}.
152
153              GroupDefs  is  what groups/0 or the post_groups/2 hook returned,
154              that is, a list of group definitions.
155
156              NewReturn is the possibly modified version of Return.
157
158              This function  is  called  only  if  the  CTH  is  added  before
159              init_per_suite is run. For details, see section CTH Scope in the
160              User's Guide.
161
162              Notice that for CTHs that are installed by means of the  suite/0
163              function,  post_all/2 is called before the init/2 hook function.
164              However, for CTHs that are installed by means of  the  CT  start
165              flag, the init/2 function is called first.
166
167          Note:
168              Prior  to each test execution, Common Test does a simulated test
169              run in order to count test suites, groups and cases for  logging
170              purposes.  This causes the post_all/3 hook function to always be
171              called twice. For this reason, side effects are best avoided  in
172              this callback.
173
174
175       Module:pre_init_per_suite(SuiteName, InitData, CTHState) -> Result
176
177              Types:
178
179                 SuiteName = atom()
180                 InitData = Config | SkipOrFail
181                 Config = NewConfig = [{Key,Value}]
182                 CTHState = NewCTHState = term()
183                 Result = {Return, NewCTHState}
184                 Return = NewConfig | SkipOrFail
185                 SkipOrFail = {fail, Reason} | {skip, Reason}
186                 Key = atom()
187                 Value = term()
188                 Reason = term()
189
190              OPTIONAL
191
192              This  function  is called before init_per_suite if it exists. It
193              typically contains  initialization/logging  that  must  be  done
194              before  init_per_suite is called. If {skip,Reason} or {fail,Rea‐
195              son} is returned, init_per_suite and all test cases of the suite
196              are skipped and Reason printed in the overview log of the suite.
197
198              SuiteName is the name of the suite to be run.
199
200              InitData  is  the original configuration list of the test suite,
201              or a SkipOrFail tuple if a previous CTH has returned this.
202
203              CTHState is the current internal state of the CTH.
204
205              Return is the result of the init_per_suite function.  If  it  is
206              {skip,Reason}  or {fail,Reason}, init_per_suite is never called,
207              instead the initiation is considered to be  skipped  or  failed,
208              respectively. If a NewConfig list is returned, init_per_suite is
209              called with that NewConfig list. For more details,  see  section
210              Pre Hooks in the User's Guide.
211
212              This  function  is  called  only  if  the  CTH  is  added before
213              init_per_suite is run. For details, see section CTH Scope in the
214              User's Guide.
215
216       Module:post_init_per_suite(SuiteName,   Config,  Return,  CTHState)  ->
217       Result
218
219              Types:
220
221                 SuiteName = atom()
222                 Config = [{Key,Value}]
223                 Return = NewReturn = Config | SkipOrFail | term()
224                 SkipOrFail = {fail, Reason} | {skip, Reason} | term()
225                 CTHState = NewCTHState = term()
226                 Result = {NewReturn, NewCTHState}
227                 Key = atom()
228                 Value = term()
229                 Reason = term()
230
231              OPTIONAL
232
233              This function is called after init_per_suite if  it  exists.  It
234              typically  contains  extra checks to ensure that all the correct
235              dependencies are started correctly.
236
237              Return is what init_per_suite returned, that is,  {fail,Reason},
238              {skip,Reason},   a   Config  list,  or  a  term  describing  how
239              init_per_suite failed.
240
241              NewReturn   is   the   possibly   modified   return   value   of
242              init_per_suite.  To  recover  from  a failure in init_per_suite,
243              return ConfigList with the tc_status element removed.  For  more
244              details,  see  Post Hooks in section "Manipulating Tests" in the
245              User's Guide.
246
247              CTHState is the current internal state of the CTH.
248
249              This function is called only if the CTH is added  before  or  in
250              init_per_suite. For details, see section CTH Scope in the User's
251              Guide.
252
253       Module:pre_init_per_group(SuiteName, GroupName, InitData, CTHState)  ->
254       Result
255
256              Types:
257
258                 SuiteName = atom()
259                 GroupName = atom()
260                 InitData = Config | SkipOrFail
261                 Config = NewConfig = [{Key,Value}]
262                 CTHState = NewCTHState = term()
263                 Result = {NewConfig | SkipOrFail, NewCTHState}
264                 SkipOrFail = {fail,Reason} | {skip, Reason}
265                 Key = atom()
266                 Value = term()
267                 Reason = term()
268
269              OPTIONAL
270
271              This  function  is called before init_per_group if it exists. It
272              behaves the same way as  pre_init_per_suite,  but  for  function
273              init_per_group instead.
274
275              If Module:pre_init_per_group/4 is not exported, common_test will
276              attempt to call  Module:pre_init_per_group(GroupName,  InitData,
277              CTHState) instead. This is for backwards compatibility.
278
279       Module:post_init_per_group(SuiteName,  GroupName,  Config, Return, CTH‐
280       State) -> Result
281
282              Types:
283
284                 SuiteName = atom()
285                 GroupName = atom()
286                 Config = [{Key,Value}]
287                 Return = NewReturn = Config | SkipOrFail | term()
288                 SkipOrFail = {fail,Reason} | {skip, Reason}
289                 CTHState = NewCTHState = term()
290                 Result = {NewReturn, NewCTHState}
291                 Key = atom()
292                 Value = term()
293                 Reason = term()
294
295              OPTIONAL
296
297              This function is called after init_per_group if  it  exists.  It
298              behaves  the  same  way as post_init_per_suite, but for function
299              init_per_group instead.
300
301              If Module:post_init_per_group/5  is  not  exported,  common_test
302              will  attempt to call Module:post_init_per_group(GroupName, Con‐
303              fig, Return, CTHState) instead. This is for  backwards  compati‐
304              bility.
305
306       Module:pre_init_per_testcase(SuiteName,  TestcaseName,  InitData,  CTH‐
307       State) -> Result
308
309              Types:
310
311                 SuiteName = atom()
312                 TestcaseName = atom()
313                 InitData = Config | SkipOrFail
314                 Config = NewConfig = [{Key,Value}]
315                 CTHState = NewCTHState = term()
316                 Result = {NewConfig | SkipOrFail, NewCTHState}
317                 SkipOrFail = {fail,Reason} | {skip, Reason}
318                 Key = atom()
319                 Value = term()
320                 Reason = term()
321
322              OPTIONAL
323
324              This function is called before init_per_testcase if  it  exists.
325              It  behaves the same way as pre_init_per_suite, but for function
326              init_per_testcase instead.
327
328              If Module:pre_init_per_testcase/4 is not  exported,  common_test
329              will  attempt to call Module:pre_init_per_testcase(TestcaseName,
330              InitData, CTHState) instead. This is for  backwards  compatibil‐
331              ity.
332
333              CTHs  cannot  be added here right now. That feature may be added
334              in a later release, but it would right now break backwards  com‐
335              patibility.
336
337       Module:post_init_per_testcase(SuiteName,  TestcaseName, Config, Return,
338       CTHState) -> Result
339
340              Types:
341
342                 SuiteName = atom()
343                 TestcaseName = atom()
344                 Config = [{Key,Value}]
345                 Return = NewReturn = Config | SkipOrFail | term()
346                 SkipOrFail = {fail,Reason} | {skip, Reason}
347                 CTHState = NewCTHState = term()
348                 Result = {NewReturn, NewCTHState}
349                 Key = atom()
350                 Value = term()
351                 Reason = term()
352
353              OPTIONAL
354
355              This function is called after init_per_testcase if it exists. It
356              behaves  the  same  way as post_init_per_suite, but for function
357              init_per_testcase instead.
358
359              If Module:post_init_per_testcase/5 is not exported,  common_test
360              will attempt to call Module:post_init_per_testcase(TestcaseName,
361              Config, Return, CTHState) instead. This is for backwards compat‐
362              ibility.
363
364       Module:pre_end_per_testcase(SuiteName, TestcaseName, EndData, CTHState)
365       -> Result
366
367              Types:
368
369                 SuiteName = atom()
370                 TestcaseName = atom()
371                 EndData = Config
372                 Config = NewConfig = [{Key,Value}]
373                 CTHState = NewCTHState = term()
374                 Result = {NewConfig, NewCTHState}
375                 Key = atom()
376                 Value = term()
377                 Reason = term()
378
379              OPTIONAL
380
381              This function is called before end_per_testcase if it exists. It
382              behaves  the  same  way  as  pre_end_per_suite, but for function
383              end_per_testcase instead.
384
385              This function cannot change the  result  of  the  test  case  by
386              returning skip or fail tuples, but it may insert items in Config
387              that can be read in end_per_testcase/2 or in  post_end_per_test‐
388              case/5.
389
390              If  Module:pre_end_per_testcase/4  is  not exported, common_test
391              will attempt to  call  Module:pre_end_per_testcase(TestcaseName,
392              EndData, CTHState) instead. This is for backwards compatibility.
393
394       Module:post_end_per_testcase(SuiteName,  TestcaseName,  Config, Return,
395       CTHState) -> Result
396
397              Types:
398
399                 SuiteName = atom()
400                 TestcaseName = atom()
401                 Config = [{Key,Value}]
402                 Return = NewReturn = Config | SkipOrFail | term()
403                 SkipOrFail = {fail,Reason} | {skip, Reason}
404                 CTHState = NewCTHState = term()
405                 Result = {NewReturn, NewCTHState}
406                 Key = atom()
407                 Value = term()
408                 Reason = term()
409
410              OPTIONAL
411
412              This function is called after end_per_testcase if it exists.  It
413              behaves  the  same  way  as post_end_per_suite, but for function
414              end_per_testcase instead.
415
416              If Module:post_end_per_testcase/5 is not  exported,  common_test
417              will  attempt to call Module:post_end_per_testcase(TestcaseName,
418              Config, Return, CTHState) instead. This is for backwards compat‐
419              ibility.
420
421       Module:pre_end_per_group(SuiteName,  GroupName,  EndData,  CTHState) ->
422       Result
423
424              Types:
425
426                 SuiteName = atom()
427                 GroupName = atom()
428                 EndData = Config | SkipOrFail
429                 Config = NewConfig = [{Key,Value}]
430                 CTHState = NewCTHState = term()
431                 Result = {NewConfig | SkipOrFail, NewCTHState}
432                 SkipOrFail = {fail,Reason} | {skip, Reason}
433                 Key = atom()
434                 Value = term()
435                 Reason = term()
436
437              OPTIONAL
438
439              This function is called before end_per_group if  it  exists.  It
440              behaves  the  same  way  as pre_init_per_suite, but for function
441              end_per_group instead.
442
443              If Module:pre_end_per_group/4 is not exported, common_test  will
444              attempt  to  call  Module:pre_end_per_group(GroupName,  EndData,
445              CTHState) instead. This is for backwards compatibility.
446
447       Module:post_end_per_group(SuiteName, GroupName,  Config,  Return,  CTH‐
448       State) -> Result
449
450              Types:
451
452                 SuiteName = atom()
453                 GroupName = atom()
454                 Config = [{Key,Value}]
455                 Return = NewReturn = Config | SkipOrFail | term()
456                 SkipOrFail = {fail,Reason} | {skip, Reason}
457                 CTHState = NewCTHState = term()
458                 Result = {NewReturn, NewCTHState}
459                 Key = atom()
460                 Value = term()
461                 Reason = term()
462
463              OPTIONAL
464
465              This  function  is  called  after end_per_group if it exists. It
466              behaves the same way as post_init_per_suite,  but  for  function
467              end_per_group instead.
468
469              If Module:post_end_per_group/5 is not exported, common_test will
470              attempt  to  call  Module:post_end_per_group(GroupName,  Config,
471              Return, CTHState) instead. This is for backwards compatibility.
472
473       Module:pre_end_per_suite(SuiteName, EndData, CTHState) -> Result
474
475              Types:
476
477                 SuiteName = atom()
478                 EndData = Config | SkipOrFail
479                 Config = NewConfig = [{Key,Value}]
480                 CTHState = NewCTHState = term()
481                 Result = {NewConfig | SkipOrFail, NewCTHState}
482                 SkipOrFail = {fail,Reason} | {skip, Reason}
483                 Key = atom()
484                 Value = term()
485                 Reason = term()
486
487              OPTIONAL
488
489              This  function  is  called before end_per_suite if it exists. It
490              behaves the same way as  pre_init_per_suite,  but  for  function
491              end_per_suite instead.
492
493       Module:post_end_per_suite(SuiteName,   Config,   Return,  CTHState)  ->
494       Result
495
496              Types:
497
498                 SuiteName = atom()
499                 Config = [{Key,Value}]
500                 Return = NewReturn = Config | SkipOrFail | term()
501                 SkipOrFail = {fail,Reason} | {skip, Reason}
502                 CTHState = NewCTHState = term()
503                 Result = {NewReturn, NewCTHState}
504                 Key = atom()
505                 Value = term()
506                 Reason = term()
507
508              OPTIONAL
509
510              This function is called after end_per_suite  if  it  exists.  It
511              behaves  the  same  way as post_init_per_suite, but for function
512              end_per_suite instead.
513
514       Module:on_tc_fail(SuiteName, TestName, Reason, CTHState) -> NewCTHState
515
516              Types:
517
518                 SuiteName = atom()
519                 TestName    =    init_per_suite     |     end_per_suite     |
520                 {init_per_group,GroupName}   |   {end_per_group,GroupName}  |
521                 {FuncName,GroupName} | FuncName
522                 FuncName = atom()
523                 GroupName = atom()
524                 Reason = term()
525                 CTHState = NewCTHState = term()
526
527              OPTIONAL
528
529              This function is called whenever a test case  (or  configuration
530              function)  fails. It is called after the post function is called
531              for the failed test case, that is:
532
533                * If init_per_suite  fails,  this  function  is  called  after
534                  post_init_per_suite.
535
536                * If   a  test  case  fails,  this  funcion  is  called  after
537                  post_end_per_testcase.
538
539              If the failed test case belongs to a test case group, the  first
540              argument  is  a  tuple  {FuncName,GroupName}, otherwise only the
541              function name.
542
543              The data that comes with Reason follows the same format as Fail‐
544              Reason in event tc_done. For details, see section Event Handling
545              in the User's Guide.
546
547              If Module:on_tc_fail/4 is not exported, common_test will attempt
548              to  call  Module:on_tc_fail(TestName, Reason, CTHState) instead.
549              This is for backwards compatibility.
550
551       Module:on_tc_skip(SuiteName, TestName, Reason, CTHState) -> NewCTHState
552
553              Types:
554
555                 SuiteName = atom()
556                 TestName    =    init_per_suite     |     end_per_suite     |
557                 {init_per_group,GroupName}   |   {end_per_group,GroupName}  |
558                 {FuncName,GroupName} | FuncName
559                 FuncName = atom()
560                 GroupName = atom()
561                 Reason = {tc_auto_skip | tc_user_skip, term()}
562                 CTHState = NewCTHState = term()
563
564              OPTIONAL
565
566              This function is called whenever a test case  (or  configuration
567              function)  is  skipped.  It is called after the post function is
568              called for the skipped test case, that is:
569
570                * If init_per_group is skipped, this function is called  after
571                  post_init_per_group.
572
573                * If  a  test  case  is skipped, this function is called after
574                  post_end_per_testcase.
575
576              If the skipped test case belongs to a test case group, the first
577              argument  is  a  tuple  {FuncName,GroupName}, otherwise only the
578              function name.
579
580              The data that comes with  Reason  follows  the  same  format  as
581              events  tc_auto_skip  and  tc_user_skip For details, see section
582              Event Handling in the User's Guide.
583
584              If Module:on_tc_skip/4 is not exported, common_test will attempt
585              to  call  Module:on_tc_skip(TestName, Reason, CTHState) instead.
586              This is for backwards compatibility.
587
588       Module:terminate(CTHState)
589
590              Types:
591
592                 CTHState = term()
593
594              OPTIONAL
595
596              This function is called at the end of a CTH scope.
597
598       Module:id(Opts) -> Id
599
600              Types:
601
602                 Opts = term()
603                 Id = term()
604
605              OPTIONAL
606
607              The Id identifies a CTH instance uniquely. If  two  CTHs  return
608              the  same  Id, the second CTH is ignored and subsequent calls to
609              the CTH are only made to the first instance.  For  details,  see
610              section Installing a CTH in the User's Guide.
611
612              This  function  is  not  to  have any side effects, as it can be
613              called multiple times by Common Test.
614
615              If not implemented, the CTH acts as if this function returned  a
616              call to make_ref/0.
617
618
619
620Ericsson AB                    common_test 1.20                    ct_hooks(3)