1Test2::Harness::Runner:U:sReersoCuorncter(i3b)uted PerlTDeosctu2m:e:nHtaartnieosns::Runner::Resource(3)
2
6 Test2::Harness::Runner::Resource - Base class for resource management
7 classes
8
10 Sometimes you have limited resources that must be shared/divided
11 between tests that run concurrently. Resource classes give you a way to
12 leverage the IPC system used by Test2::Harness to manage resource
13 assignment and recovery.
14
16 Here is a resource class that simply assigns an integer to each test.
17 It would be possible to re-use integers, but since there are infinite
18 integers this example is kept simple and just always grabs the next
19 one.
20 package Test2::Harness::Runner::Resource::Foo;
22 use strict;
23 use warnings;
24 use parent 'Test2::Harness::Runner::Resource';
26 sub available {
28 my $self = shift;
29 my ($task) = @_;
30 # There are an infinite amount of integers, so we always return true
32 return 1;
33 }
34 sub assign {
36 my $self = shift;
37 my ($task, $state) = @_;
38 # Next ID, do not record the state change yet!
40 my $id = 1 + ($self->{ID} //= 0);
41 print "ASSIGN: $id = $task->{job_id}\n";
43 # 'record' should get whatever we need to record the resource, whatever you
45 # pass in will become the argument to the record() sub below. This may be a
46 # scalar, a hash, an array, etc. It will be serialized to JSON before
47 # record() sees it.
48 $state->{record} = $id;
49 # Pass the resource into the test, this can be done as envronment variables
51 # and/or arguments to the test (@ARGV).
52 $state->{env_vars}->{FOO_ID} = $id;
53 push @{$state->{args}} => $id;
54 # The return is ignored.
56 return;
57 }
58 sub record {
60 my $self = shift;
61 my ($job_id, $record_arg_from_assign) = @_;
62 # The ID from $state->{record}->{$pkg} in assign.
64 my $id = $record_arg_from_assign;
65 # Update our internal state to reflect the new ID.
67 $self->{ID} = $id;
68 # Add a mapping of what job ID gets what integer ID.
70 $self->{ID_TO_JOB_ID}->{$id} = $job_id;
71 $self->{JOB_ID_TO_ID}->{$job_id} = $id;
72 print "RECORD: $id = $job_id\n";
74 # The return is ignored
76 }
77 sub release {
79 my $self = shift;
80 my ($job_id) = @_;
81 # Clear the internal mapping, the integer ID is now free. Theoretically it
83 # can be reused, but this example is not that complex.
84 my $id = delete $self->{JOB_ID_TO_ID}->{$job_id};
85 # This is called for all tests that complete, even if they did not use
87 # this resource, so we return if the job_id is not applicable.
88 return unless defined $id;
89 delete $self->{ID_TO_JOB_ID}->{$id};
91 print " FREE: $id = $job_id\n";
93 # The return is ignored
95 }
96 sub cleanup {
98 my $self = shift;
99 print "CLEANUP!\n";
101 }
102 1;
104 The print statements generated will look like this when running 2 tests
106 concurrently:
107 yath test -R Foo -j2 t/testA.t t/testB.t
109 [...]
110 (INTERNAL) ASSIGN: 1 = 4F7CF5F6-E43F-11EA-9199-24FCBF610F44
111 (INTERNAL) RECORD: 1 = 4F7CF5F6-E43F-11EA-9199-24FCBF610F44
112 (INTERNAL) ASSIGN: 2 = E19CD98C-E436-11EA-8469-8DF0BF610F44
113 (INTERNAL) RECORD: 2 = E19CD98C-E436-11EA-8469-8DF0BF610F44
114 (INTERNAL) FREE: 1 = 4F7CF5F6-E43F-11EA-9199-24FCBF610F44
115 (INTERNAL) FREE: 2 = E19CD98C-E436-11EA-8469-8DF0BF610F44
116 (INTERNAL) CLEANUP!
117 [...]
118 Depending on the tests run the 'FREE' prints may be out of order.
120
122 HOW STATE IS MANAGED
123 Depending on your preload configuration, yath may have several runners
124 launching tests. If a runner has nothing to do it will lock the queue
125 and try to find the next test that should be run. Only 1 of the runners
126 will be in control of the queue at any given time, but the control of
127 the queue may pass between runners. To manage this there is a mechanism
128 to record messages that allow each runner to maintain a copy of the
129 current state.
130 CHECK IF RESOURCES ARE AVAILABLE
132 Each runner will have an instance of your resource class. When the
133 runner is in control of the queue, and wants to designate the next test
134 to run, it will check with the resource classes to make sure the
135 correct resources are available. To do that it will call
136 "available($task)" on each resource instance.
137 The $task will contain the specification for the test, it is a hashref,
139 and you SHOULD NOT modify it. The only key most people care about is
140 the 'file' key, which has the test file that will be run if resources
141 are available.
142 If resources are available, or if the specific file does not need the
144 resource, the "available()" method should return true. If the file does
145 need your resource(s), and none are available, this should return
146 false. If any resource class returns false it means the test cannot be
147 run yet and the runner will look for another test to run.
148 ASSIGN A RESOURCE
150 If the runner has determined the test can be run, and all necessary
151 resources are available, it will then call "assign($task, $state)" on
152 all resource class instances. At this time the resource class should
153 decide what resource(s) to assign to the class.
154 CRITICAL NOTE: the "assing()" method MUST NOT alter any internal state
156 on the resource class instance. State modification must wait for the
157 "record()" method to be called. This is because the "assign()" method
158 is only called in one runner process, the "record()" method call will
159 happen in every runner process to insure they all have the same
160 internal state.
161 The assign() sub should modify the $state hash, which has 3 keys:
163 env_vars => {}
165 Env vars to set for the test
166 args => []
168 Arguments to pass to the test
169 record => ...
171 Data needed to record the state change for resource classes. Can be
172 a scalar, hashref, arrayref, etc. It will be serialized to JSON to
173 be passed between processes.
174 RECORD A RESOURCE
176 Once a resource is assigned, a message will be sent to all runner
177 processes INCLUDING THE ONE THAT DID THE ASSIGN that says it should
178 call "record($job_id, $record_val)" on your resource class instance.
179 Your resource class instance must use this to update the state so that
180 once done ALL processes will have the proper internal state.
181 The $record_val is whatever you put into "$state->{record}" in the
183 "assign()" method above.
184 QUEUE MANAGEMENT IS UNLOCKED
186 Once the above has been done, queue management will be unlocked. You
187 can be guarenteed that only one process will be run the "available()",
188 and "assign()" sequence at a time, and that they will be called in
189 order, though "assign()" may not be called if another resource was not
190 available. If "assign()" is called, you can be guarenteed that all
191 processes, including the one that called "assign()" will have their
192 "record()" called with the proper argument BEFORE they try to manage
193 the queue (which is the only place resources are checked or assigned).
194 RELEASE A RESOURCE
196 Whenever a process that is using a resource exits, the runner that
197 waits on that process will eventually send an IPC message announcing
198 that the job_id has completed. Every time a job_id completes the
199 "release($job_id)" method will be called on your resource class in all
200 runner processes. This allows the state to be updated to reflect the
201 freed resource.
202 You can be guarenteed that any process that locks the queue to run a
204 new test will eventually see the message. The message may come in
205 during a loop that is checking for resources, in which case the state
206 will not reflect the resource being available, however in such cases
207 the loop will end and be called again later with the message having
208 been receieved. There will be no deadlock due to a queue manager
209 waiting for the message.
210 There are no guarentees about what order resources will be released in.
212
214 $res = $class->new(settings => $settings);
215 A default new method, returns a blessed hashref with the settings
216 key set to the Test2::Harness::Settings instance.
217 $bool = $res->available(\%task)
219 DO NOT MODIFY ANY INTERNAL STATE IN THIS METHOD
220 DO NOT MODIFY THE TASK HASHREF
222 Return true if the resource is available, or if the task does not
224 require the resource.
225 Return false if the resource is not available, but is needed.
227 The only key in "\%task" hashref that most resources will care
229 about is the 'file' key, which contains the test file to be run.
230 $res->assign(\%task, \%state)
232 DO NOT MODIFY THE TASK HASHREF
233 DO NOT MODIFY ANY INTERNAL STATE IN THIS METHOD
235 If the task does not need any resources you may simply return.
237 If resources are needed you should deduce what resources to assign.
239 You should put any data needed to update the internal state of your
241 resource instance in the "$state->{record}" hash key. It WILL be
242 serialized to JSON before being used as an argument to "record()".
243 $state->{record} = $id;
245 If you do not set the 'record' key, or set it to undef, then the
247 "record()" method will not be called.
248 If your tests need to know what resources to use, you may set
250 environment variables and/or command line arguments to pass into
251 the test (@ARGV).
252 $state->{env_vars}->{FOO_ID} = $id;
254 push @{$state->{args}} => $id;
255 The "\%state" hashref is used only by your instance, you are free
257 to fully replace the 'env_vars' and 'args' keys. They will
258 eventually be merged into a master state along with those of other
259 resources, but this ref is exclusive to you in this method.
260 $inst->record($job_id, $record_arg_from_assign)
262 NOTE: THIS MAY BE CALLED IN MUTLIPLE PROCESSES CONCURRENTLY.
263 This will be called in all processes so that your instance can
265 update any internal state.
266 The $job_id variable contains the id for the job to which the
268 resource was assigned. You should use this to record any internal
269 state. The $job_id will be passed to "release()" when the job
270 completes and no longer needs the resource.
271 This is intended only for modifying internal state, you should not
273 do anything in this sub that will explode if it is also done in
274 another process at the same time with the same arguments. For
275 example creating a database should not be done here, multiple
276 processes will fight to do the create. The creation, if necessary
277 should be done in "assign()" which will be called in only one
278 process.
279 $inst->release($job_id)
281 NOTE: THIS MAY BE CALLED IN MUTLIPLE PROCESSES CONCURRENTLY.
282 This will be called for every test job that completes, even if it
284 did not use this resource. If the job_id did not use the resource
285 you may simply return, otherwise update the internal state to
286 reflect that the resource is no longer in use.
287 This is intended only for modifying internal state, you should not
289 do anything in this sub that will explode if it is also done in
290 another process at the same time with the same arguments. For
291 example deleting a database should not be done here, multiple
292 processes will fight to do the delete. "assign()" is the only
293 method that will be run in a single process, so if a database needs
294 to be cleaned before it can be used you should clean it there. Any
295 final cleanup should be done in "cleanup()" which will only be
296 called by one process at the very end.
297 $inst->cleanup()
299 This will be called once by the parent runner process just before
300 it exits. This is your chance to do any final cleanup tasks such
301 as deleting databases that are no longer going to be used by tests
302 as no more will be run.
303
305 The source code repository for Test2-Harness can be found at
306 http://github.com/Test-More/Test2-Harness/.
307
309 Chad Granum <exodist@cpan.org>
310
312 Chad Granum <exodist@cpan.org>
313
315 Copyright 2020 Chad Granum <exodist7@gmail.com>.
316 This program is free software; you can redistribute it and/or modify it
318 under the same terms as Perl itself.
319 See http://dev.perl.org/licenses/
321perl v5.32.1 2021-03-12Test2::Harness::Runner::Resource(3)