1ForkManager(3)        User Contributed Perl Documentation       ForkManager(3)
2
3
4

NAME

6       Parallel::ForkManager - A simple parallel processing fork manager
7

SYNOPSIS

9         use Parallel::ForkManager;
10
11         $pm = new Parallel::ForkManager($MAX_PROCESSES);
12
13         foreach $data (@all_data) {
14           # Forks and returns the pid for the child:
15           my $pid = $pm->start and next;
16
17           ... do some work with $data in the child process ...
18
19           $pm->finish; # Terminates the child process
20         }
21

DESCRIPTION

23       This module is intended for use in operations that can be done in
24       parallel where the number of processes to be forked off should be
25       limited. Typical use is a downloader which will be retrieving
26       hundreds/thousands of files.
27
28       The code for a downloader would look something like this:
29
30         use LWP::Simple;
31         use Parallel::ForkManager;
32
33         ...
34
35         @links=(
36           ["http://www.foo.bar/rulez.data","rulez_data.txt"],
37           ["http://new.host/more_data.doc","more_data.doc"],
38           ...
39         );
40
41         ...
42
43         # Max 30 processes for parallel download
44         my $pm = new Parallel::ForkManager(30);
45
46         foreach my $linkarray (@links) {
47           $pm->start and next; # do the fork
48
49           my ($link,$fn) = @$linkarray;
50           warn "Cannot get $fn from $link"
51             if getstore($link,$fn) != RC_OK;
52
53           $pm->finish; # do the exit in the child process
54         }
55         $pm->wait_all_children;
56
57       First you need to instantiate the ForkManager with the "new"
58       constructor.  You must specify the maximum number of processes to be
59       created. If you specify 0, then NO fork will be done; this is good for
60       debugging purposes.
61
62       Next, use $pm->start to do the fork. $pm returns 0 for the child
63       process, and child pid for the parent process (see also "fork()" in
64       perlfunc(1p)).  The "and next" skips the internal loop in the parent
65       process. NOTE: $pm->start dies if the fork fails.
66
67       $pm->finish terminates the child process (assuming a fork was done in
68       the "start").
69
70       NOTE: You cannot use $pm->start if you are already in the child
71       process.  If you want to manage another set of subprocesses in the
72       child process, you must instantiate another Parallel::ForkManager
73       object!
74

METHODS

76       new $processes
77            Instantiate a new Parallel::ForkManager object. You must specify
78            the maximum number of children to fork off. If you specify 0
79            (zero), then no children will be forked. This is intended for
80            debugging purposes.
81
82       start [ $process_identifier ]
83            This method does the fork. It returns the pid of the child process
84            for the parent, and 0 for the child process. If the $processes
85            parameter for the constructor is 0 then, assuming you're in the
86            child process, $pm->start simply returns 0.
87
88            An optional $process_identifier can be provided to this method...
89            It is used by the "run_on_finish" callback (see CALLBACKS) for
90            identifying the finished process.
91
92       finish [ $exit_code ]
93            Closes the child process by exiting and accepts an optional exit
94            code (default exit code is 0) which can be retrieved in the parent
95            via callback.  If you use the program in debug mode ($processes ==
96            0), this method doesn't do anything.
97
98       set_max_procs $processes
99            Allows you to set a new maximum number of children to maintain.
100            Returns the previous setting.
101
102       wait_all_children
103            You can call this method to wait for all the processes which have
104            been forked. This is a blocking wait.
105

CALLBACKS

107       You can define callbacks in the code, which are called on events like
108       starting a process or upon finish.
109
110       The callbacks can be defined with the following methods:
111
112       run_on_finish $code [, $pid ]
113           You can define a subroutine which is called when a child is
114           terminated. It is called in the parent process.
115
116           The paremeters of the $code are the following:
117
118             - pid of the process, which is terminated
119             - exit code of the program
120             - identification of the process (if provided in the "start" method)
121             - exit signal (0-127: signal name)
122             - core dump (1 if there was core dump at exit)
123
124       run_on_start $code
125           You can define a subroutine which is called when a child is
126           started. It called after the successful startup of a child in the
127           parent process.
128
129           The parameters of the $code are the following:
130
131             - pid of the process which has been started
132             - identification of the process (if provided in the "start" method)
133
134       run_on_wait $code, [$period]
135           You can define a subroutine which is called when the child process
136           needs to wait for the startup. If $period is not defined, then one
137           call is done per child. If $period is defined, then $code is called
138           periodically and the module waits for $period seconds betwen the
139           two calls. Note, $period can be fractional number also. The exact
140           "$period seconds" is not guarranteed, signals can shorten and the
141           process scheduler can make it longer (on busy systems).
142
143           The $code called in the "start" and the "wait_all_children" method
144           also.
145
146           No parameters are passed to the $code on the call.
147

EXAMPLE

149   Parallel get
150       This small example can be used to get URLs in parallel.
151
152         use Parallel::ForkManager;
153         use LWP::Simple;
154         my $pm=new Parallel::ForkManager(10);
155         for my $link (@ARGV) {
156           $pm->start and next;
157           my ($fn)= $link =~ /^.*\/(.*?)$/;
158           if (!$fn) {
159             warn "Cannot determine filename from $fn\n";
160           } else {
161             $0.=" ".$fn;
162             print "Getting $fn from $link\n";
163             my $rc=getstore($link,$fn);
164             print "$link downloaded. response code: $rc\n";
165           };
166           $pm->finish;
167         };
168
169   Callbacks
170       Example of a program using callbacks to get child exit codes:
171
172         use strict;
173         use Parallel::ForkManager;
174
175         my $max_procs = 5;
176         my @names = qw( Fred Jim Lily Steve Jessica Bob Dave Christine Rico Sara );
177         # hash to resolve PID's back to child specific information
178
179         my $pm =  new Parallel::ForkManager($max_procs);
180
181         # Setup a callback for when a child finishes up so we can
182         # get it's exit code
183         $pm->run_on_finish(
184           sub { my ($pid, $exit_code, $ident) = @_;
185             print "** $ident just got out of the pool ".
186               "with PID $pid and exit code: $exit_code\n";
187           }
188         );
189
190         $pm->run_on_start(
191           sub { my ($pid,$ident)=@_;
192             print "** $ident started, pid: $pid\n";
193           }
194         );
195
196         $pm->run_on_wait(
197           sub {
198             print "** Have to wait for one children ...\n"
199           },
200           0.5
201         );
202
203         foreach my $child ( 0 .. $#names ) {
204           my $pid = $pm->start($names[$child]) and next;
205
206           # This code is the child process
207           print "This is $names[$child], Child number $child\n";
208           sleep ( 2 * $child );
209           print "$names[$child], Child $child is about to get out...\n";
210           sleep 1;
211           $pm->finish($child); # pass an exit code to finish
212         }
213
214         print "Waiting for Children...\n";
215         $pm->wait_all_children;
216         print "Everybody is out of the pool!\n";
217

BUGS AND LIMITATIONS

219       Do not use Parallel::ForkManager in an environment, where other child
220       processes can affect the run of the main program, so using this module
221       is not recommended in an environment where fork() / wait() is already
222       used.
223
224       If you want to use more than one copies of the Parallel::ForkManager,
225       then you have to make sure that all children processes are terminated,
226       before you use the second object in the main program.
227
228       You are free to use a new copy of Parallel::ForkManager in the child
229       processes, although I don't think it makes sense.
230
232       Copyright (c) 2000 SzabA~X, BalA~Xzs (dLux)
233
234       All right reserved. This program is free software; you can redistribute
235       it and/or modify it under the same terms as Perl itself.
236

AUTHOR

238         dLux (SzabA~X, BalA~Xzs) <dlux@kapu.hu>
239

CREDITS

241         Noah Robin <sitz@onastick.net> (documentation tweaks)
242         Chuck Hirstius <chirstius@megapathdsl.net> (callback exit status, example)
243         Grant Hopwood <hopwoodg@valero.com> (win32 port)
244         Mark Southern <mark_southern@merck.com> (bugfix)
245
246
247
248perl v5.12.0                      2002-12-25                    ForkManager(3)
Impressum