1ForkManager(3) User Contributed Perl Documentation ForkManager(3)
2
6 Parallel::ForkManager - A simple parallel processing fork manager
7
9 use Parallel::ForkManager;
10 $pm = new Parallel::ForkManager($MAX_PROCESSES);
12 foreach $data (@all_data) {
14 # Forks and returns the pid for the child:
15 my $pid = $pm->start and next;
16 ... do some work with $data in the child process ...
18 $pm->finish; # Terminates the child process
20 }
21
23 This module is intended for use in operations that can be done in
24 parallel where the number of processes to be forked off should be
25 limited. Typical use is a downloader which will be retrieving
26 hundreds/thousands of files.
27 The code for a downloader would look something like this:
29 use LWP::Simple;
31 use Parallel::ForkManager;
32 ...
34 @links=(
36 ["http://www.foo.bar/rulez.data","rulez_data.txt"],
37 ["http://new.host/more_data.doc","more_data.doc"],
38 ...
39 );
40 ...
42 # Max 30 processes for parallel download
44 my $pm = new Parallel::ForkManager(30);
45 foreach my $linkarray (@links) {
47 $pm->start and next; # do the fork
48 my ($link,$fn) = @$linkarray;
50 warn "Cannot get $fn from $link"
51 if getstore($link,$fn) != RC_OK;
52 $pm->finish; # do the exit in the child process
54 }
55 $pm->wait_all_children;
56 First you need to instantiate the ForkManager with the "new"
58 constructor. You must specify the maximum number of processes to be
59 created. If you specify 0, then NO fork will be done; this is good for
60 debugging purposes.
61 Next, use $pm->start to do the fork. $pm returns 0 for the child
63 process, and child pid for the parent process (see also "fork()" in
64 perlfunc(1p)). The "and next" skips the internal loop in the parent
65 process. NOTE: $pm->start dies if the fork fails.
66 $pm->finish terminates the child process (assuming a fork was done in
68 the "start").
69 NOTE: You cannot use $pm->start if you are already in the child
71 process. If you want to manage another set of subprocesses in the
72 child process, you must instantiate another Parallel::ForkManager
73 object!
74
76 new $processes
77 Instantiate a new Parallel::ForkManager object. You must specify
78 the maximum number of children to fork off. If you specify 0
79 (zero), then no children will be forked. This is intended for
80 debugging purposes.
81 start [ $process_identifier ]
83 This method does the fork. It returns the pid of the child process
84 for the parent, and 0 for the child process. If the $processes
85 parameter for the constructor is 0 then, assuming you're in the
86 child process, $pm->start simply returns 0.
87 An optional $process_identifier can be provided to this method...
89 It is used by the "run_on_finish" callback (see CALLBACKS) for
90 identifying the finished process.
91 finish [ $exit_code ]
93 Closes the child process by exiting and accepts an optional exit
94 code (default exit code is 0) which can be retrieved in the parent
95 via callback. If you use the program in debug mode ($processes ==
96 0), this method doesn't do anything.
97 set_max_procs $processes
99 Allows you to set a new maximum number of children to maintain.
100 Returns the previous setting.
101 wait_all_children
103 You can call this method to wait for all the processes which have
104 been forked. This is a blocking wait.
105
107 You can define callbacks in the code, which are called on events like
108 starting a process or upon finish.
109 The callbacks can be defined with the following methods:
111 run_on_finish $code [, $pid ]
113 You can define a subroutine which is called when a child is
114 terminated. It is called in the parent process.
115 The paremeters of the $code are the following:
117 - pid of the process, which is terminated
119 - exit code of the program
120 - identification of the process (if provided in the "start" method)
121 - exit signal (0-127: signal name)
122 - core dump (1 if there was core dump at exit)
123 run_on_start $code
125 You can define a subroutine which is called when a child is
126 started. It called after the successful startup of a child in the
127 parent process.
128 The parameters of the $code are the following:
130 - pid of the process which has been started
132 - identification of the process (if provided in the "start" method)
133 run_on_wait $code, [$period]
135 You can define a subroutine which is called when the child process
136 needs to wait for the startup. If $period is not defined, then one
137 call is done per child. If $period is defined, then $code is called
138 periodically and the module waits for $period seconds betwen the
139 two calls. Note, $period can be fractional number also. The exact
140 "$period seconds" is not guarranteed, signals can shorten and the
141 process scheduler can make it longer (on busy systems).
142 The $code called in the "start" and the "wait_all_children" method
144 also.
145 No parameters are passed to the $code on the call.
147
149 Parallel get
150 This small example can be used to get URLs in parallel.
151 use Parallel::ForkManager;
153 use LWP::Simple;
154 my $pm=new Parallel::ForkManager(10);
155 for my $link (@ARGV) {
156 $pm->start and next;
157 my ($fn)= $link =~ /^.*\/(.*?)$/;
158 if (!$fn) {
159 warn "Cannot determine filename from $fn\n";
160 } else {
161 $0.=" ".$fn;
162 print "Getting $fn from $link\n";
163 my $rc=getstore($link,$fn);
164 print "$link downloaded. response code: $rc\n";
165 };
166 $pm->finish;
167 };
168 Callbacks
170 Example of a program using callbacks to get child exit codes:
171 use strict;
173 use Parallel::ForkManager;
174 my $max_procs = 5;
176 my @names = qw( Fred Jim Lily Steve Jessica Bob Dave Christine Rico Sara );
177 # hash to resolve PID's back to child specific information
178 my $pm = new Parallel::ForkManager($max_procs);
180 # Setup a callback for when a child finishes up so we can
182 # get it's exit code
183 $pm->run_on_finish(
184 sub { my ($pid, $exit_code, $ident) = @_;
185 print "** $ident just got out of the pool ".
186 "with PID $pid and exit code: $exit_code\n";
187 }
188 );
189 $pm->run_on_start(
191 sub { my ($pid,$ident)=@_;
192 print "** $ident started, pid: $pid\n";
193 }
194 );
195 $pm->run_on_wait(
197 sub {
198 print "** Have to wait for one children ...\n"
199 },
200 0.5
201 );
202 foreach my $child ( 0 .. $#names ) {
204 my $pid = $pm->start($names[$child]) and next;
205 # This code is the child process
207 print "This is $names[$child], Child number $child\n";
208 sleep ( 2 * $child );
209 print "$names[$child], Child $child is about to get out...\n";
210 sleep 1;
211 $pm->finish($child); # pass an exit code to finish
212 }
213 print "Waiting for Children...\n";
215 $pm->wait_all_children;
216 print "Everybody is out of the pool!\n";
217
219 Do not use Parallel::ForkManager in an environment, where other child
220 processes can affect the run of the main program, so using this module
221 is not recommended in an environment where fork() / wait() is already
222 used.
223 If you want to use more than one copies of the Parallel::ForkManager,
225 then you have to make sure that all children processes are terminated,
226 before you use the second object in the main program.
227 You are free to use a new copy of Parallel::ForkManager in the child
229 processes, although I don't think it makes sense.
230
232 Copyright (c) 2000 SzabA~X, BalA~Xzs (dLux)
233 All right reserved. This program is free software; you can redistribute
235 it and/or modify it under the same terms as Perl itself.
236
238 dLux (SzabA~X, BalA~Xzs) <dlux@kapu.hu>
239
241 Noah Robin <sitz@onastick.net> (documentation tweaks)
242 Chuck Hirstius <chirstius@megapathdsl.net> (callback exit status, example)
243 Grant Hopwood <hopwoodg@valero.com> (win32 port)
244 Mark Southern <mark_southern@merck.com> (bugfix)
245perl v5.12.0 2002-12-25 ForkManager(3)