1ForkManager(3) User Contributed Perl Documentation ForkManager(3)
2
6 Parallel::ForkManager - A simple parallel processing fork manager
7
9 use Parallel::ForkManager;
10 $pm = new Parallel::ForkManager($MAX_PROCESSES);
12 foreach $data (@all_data) {
14 # Forks and returns the pid for the child:
15 my $pid = $pm->start and next;
16 ... do some work with $data in the child process ...
18 $pm->finish; # Terminates the child process
20 }
21
23 This module is intended for use in operations that can be done in par‐
24 allel where the number of processes to be forked off should be limited.
25 Typical use is a downloader which will be retrieving hundreds/thousands
26 of files.
27 The code for a downloader would look something like this:
29 use LWP::Simple;
31 use Parallel::ForkManager;
32 ...
34 @links=(
36 ["http://www.foo.bar/rulez.data","rulez_data.txt"],
37 ["http://new.host/more_data.doc","more_data.doc"],
38 ...
39 );
40 ...
42 # Max 30 processes for parallel download
44 my $pm = new Parallel::ForkManager(30);
45 foreach my $linkarray (@links) {
47 $pm->start and next; # do the fork
48 my ($link,$fn) = @$linkarray;
50 warn "Cannot get $fn from $link"
51 if getstore($link,$fn) != RC_OK;
52 $pm->finish; # do the exit in the child process
54 }
55 $pm->wait_all_children;
56 First you need to instantiate the ForkManager with the "new" construc‐
58 tor. You must specify the maximum number of processes to be created.
59 If you specify 0, then NO fork will be done; this is good for debugging
60 purposes.
61 Next, use $pm->start to do the fork. $pm returns 0 for the child
63 process, and child pid for the parent process (see also "fork()" in
64 perlfunc(1p)). The "and next" skips the internal loop in the parent
65 process. NOTE: $pm->start dies if the fork fails.
66 $pm->finish terminates the child process (assuming a fork was done in
68 the "start").
69 NOTE: You cannot use $pm->start if you are already in the child
71 process. If you want to manage another set of subprocesses in the
72 child process, you must instantiate another Parallel::ForkManager
73 object!
74
76 new $processes
77 Instantiate a new Parallel::ForkManager object. You must specify
78 the maximum number of children to fork off. If you specify 0
79 (zero), then no children will be forked. This is intended for
80 debugging purposes.
81 start [ $process_identifier ]
83 This method does the fork. It returns the pid of the child process
84 for the parent, and 0 for the child process. If the $processes
85 parameter for the constructor is 0 then, assuming you're in the
86 child process, $pm->start simply returns 0.
87 An optional $process_identifier can be provided to this method...
89 It is used by the "run_on_finish" callback (see CALLBACKS) for
90 identifying the finished process.
91 finish [ $exit_code ]
93 Closes the child process by exiting and accepts an optional exit
94 code (default exit code is 0) which can be retrieved in the parent
95 via callback. If you use the program in debug mode ($processes ==
96 0), this method doesn't do anything.
97 set_max_procs $processes
99 Allows you to set a new maximum number of children to maintain.
100 Returns the previous setting.
101 wait_all_children
103 You can call this method to wait for all the processes which have
104 been forked. This is a blocking wait.
105
107 You can define callbacks in the code, which are called on events like
108 starting a process or upon finish.
109 The callbacks can be defined with the following methods:
111 run_on_finish $code [, $pid ]
113 You can define a subroutine which is called when a child is termi‐
114 nated. It is called in the parent process.
115 The paremeters of the $code are the following:
117 - pid of the process, which is terminated
119 - exit code of the program
120 - identification of the process (if provided in the "start" method)
121 - exit signal (0-127: signal name)
122 - core dump (1 if there was core dump at exit)
123 run_on_start $code
125 You can define a subroutine which is called when a child is
126 started. It called after the successful startup of a child in the
127 parent process.
128 The parameters of the $code are the following:
130 - pid of the process which has been started
132 - identification of the process (if provided in the "start" method)
133 run_on_wait $code, [$period]
135 You can define a subroutine which is called when the child process
136 needs to wait for the startup. If $period is not defined, then one
137 call is done per child. If $period is defined, then $code is called
138 periodically and the module waits for $period seconds betwen the
139 two calls. Note, $period can be fractional number also. The exact
140 "$period seconds" is not guarranteed, signals can shorten and the
141 process scheduler can make it longer (on busy systems).
142 The $code called in the "start" and the "wait_all_children" method
144 also.
145 No parameters are passed to the $code on the call.
147
149 Parallel get
150 This small example can be used to get URLs in parallel.
152 use Parallel::ForkManager;
154 use LWP::Simple;
155 my $pm=new Parallel::ForkManager(10);
156 for my $link (@ARGV) {
157 $pm->start and next;
158 my ($fn)= $link =~ /^.*\/(.*?)$/;
159 if (!$fn) {
160 warn "Cannot determine filename from $fn\n";
161 } else {
162 $0.=" ".$fn;
163 print "Getting $fn from $link\n";
164 my $rc=getstore($link,$fn);
165 print "$link downloaded. response code: $rc\n";
166 };
167 $pm->finish;
168 };
169 Callbacks
171 Example of a program using callbacks to get child exit codes:
173 use strict;
175 use Parallel::ForkManager;
176 my $max_procs = 5;
178 my @names = qw( Fred Jim Lily Steve Jessica Bob Dave Christine Rico Sara );
179 # hash to resolve PID's back to child specific information
180 my $pm = new Parallel::ForkManager($max_procs);
182 # Setup a callback for when a child finishes up so we can
184 # get it's exit code
185 $pm->run_on_finish(
186 sub { my ($pid, $exit_code, $ident) = @_;
187 print "** $ident just got out of the pool ".
188 "with PID $pid and exit code: $exit_code\n";
189 }
190 );
191 $pm->run_on_start(
193 sub { my ($pid,$ident)=@_;
194 print "** $ident started, pid: $pid\n";
195 }
196 );
197 $pm->run_on_wait(
199 sub {
200 print "** Have to wait for one children ...\n"
201 },
202 0.5
203 );
204 foreach my $child ( 0 .. $#names ) {
206 my $pid = $pm->start($names[$child]) and next;
207 # This code is the child process
209 print "This is $names[$child], Child number $child\n";
210 sleep ( 2 * $child );
211 print "$names[$child], Child $child is about to get out...\n";
212 sleep 1;
213 $pm->finish($child); # pass an exit code to finish
214 }
215 print "Waiting for Children...\n";
217 $pm->wait_all_children;
218 print "Everybody is out of the pool!\n";
219
221 Do not use Parallel::ForkManager in an environment, where other child
222 processes can affect the run of the main program, so using this module
223 is not recommended in an environment where fork() / wait() is already
224 used.
225 If you want to use more than one copies of the Parallel::ForkManager,
227 then you have to make sure that all children processes are terminated,
228 before you use the second object in the main program.
229 You are free to use a new copy of Parallel::ForkManager in the child
231 processes, although I don't think it makes sense.
232
234 Copyright (c) 2000 Szabó, Balázs (dLux)
235 All right reserved. This program is free software; you can redistribute
237 it and/or modify it under the same terms as Perl itself.
238
240 dLux (Szabó, Balázs) <dlux@kapu.hu>
241
243 Noah Robin <sitz@onastick.net> (documentation tweaks)
244 Chuck Hirstius <chirstius@megapathdsl.net> (callback exit status, example)
245 Grant Hopwood <hopwoodg@valero.com> (win32 port)
246 Mark Southern <mark_southern@merck.com> (bugfix)
247perl v5.8.8 2002-12-25 ForkManager(3)