1MPIRUN(1) LAM COMMANDS MPIRUN(1)
2
6 mpirun - Run MPI programs on LAM nodes.
7
9 mpirun [-fhvO] [-c <#> | -np <#>] [-D | -wd <dir>] [-ger | -nger]
10 [-sigs | -nsigs] [-ssi <key> <value>] [-nw | -w] [-nx] [-pty |
11 -npty] [-s <node>] [-t | -toff | -ton] [-tv] [-x
12 VAR1[=VALUE1][,VAR2[=VALUE2],...]] [[-p <prefix_str>] [-sa |
13 -sf]] [<where>] <program> [-- <args>]
14 Note: Although each are individually optional, at least one of <where>,
16 -np, or -c must be specified in the above form (i.e., when a
17 schema is not used).
18 mpirun [-fhvO] [-D | -wd <dir>] [-ger | -nger] [-sigs | -nsigs] [-ssi
20 <key> <value>] [-nw | -w] [-nx] [-pty | -npty] [-t | -toff |
21 -ton] [-tv] [-x VAR1[=VALUE1][,VAR2[=VALUE2],...]] <schema>
22 Note: The -c2c and -lamd options are now obsolete. Use -ssi instead.
24 See the "SSI" section, below.
25
27 If you're simply looking for how to run an MPI application, you proba‐
28 bly want to use the following command line:
29 % mpirun C my_mpi_application
31 This will run one copy of my_mpi_application on every CPU in the cur‐
33 rent LAM universe. Alternatively, "N" can be used in place of "C", in‐
34 dicating that one copy of my_mpi_application should be run on every
35 node (as opposed to CPU) in the current LAM universe. Finally:
36 % mpirun -np 4 my_mpi_application
38 can be used to tell LAM to explicitly run four copies of my_mpi_appli‐
40 cation, scheduling in a round-robin fashion by CPU in the LAM universe.
41 See the rest of this page for more details, particularly the "Location
42 Nomenclature" section.
43
45 There are two forms of the mpirun command -- one for programs (i.e.,
46 SPMD-style applications), and one for application schemas (see app‐
47 schema(5)). Both forms of mpirun use the following options by default:
48 -nger -w. These may each be overriden by their counterpart options,
49 described below.
50 Additionally, mpirun will send the name of the directory where it was
52 invoked on the local node to each of the remote nodes, and attempt to
53 change to that directory. See the "Current Working Directory" section,
54 below.
55 -c <#> Synonym for -np (see below).
57 -D Use the executable program location as the current working
59 directory for created processes. The current working direc‐
60 tory of the created processes will be set before the user's
61 program is invoked. This option is mutually exclusive with
62 -wd.
63 -f Do not configure standard I/O file descriptors - use de‐
65 faults.
66 -h Print useful information on this command.
68 -ger Enable GER (Guaranteed Envelope Resources) communication pro‐
70 tocol and error reporting. See MPI(7) for a description of
71 GER. This option is mutually exclusive with -nger.
72 -nger Disable GER (Guaranteed Envelope Resources). This option is
74 mutually exclusive with -ger.
75 -nsigs Do not have LAM catch signals in the user application. This
77 is the default, and is mutually exclusive with -sigs.
78 -np <#> Run this many copies of the program on the given nodes. This
80 option indicates that the specified file is an executable
81 program and not an application schema. If no nodes are spec‐
82 ified, all LAM nodes are considered for scheduling; LAM will
83 schedule the programs in a round-robin fashion, "wrapping
84 around" (and scheduling multiple copies on a single node) if
85 necessary.
86 -npty Disable pseudo-tty support. Unless you are having problems
88 with pseudo-tty support, you probably do not need this op‐
89 tion. Mutually exlclusive with -pty.
90 -nw Do not wait for all processes to complete before exiting
92 mpirun. This option is mutually exclusive with -w.
93 -nx Do not automatically export LAM_MPI_*, LAM_IMPI_*, or IMPI_*
95 environment variables to the remote nodes.
96 -O Multicomputer is homogeneous. Do no data conversion when
98 passing messages. THIS FLAG IS NOW OBSOLETE.
99 -pty Enable pseudo-tty support. Among other things, this enabled
101 line-buffered output (which is probably what you want). This
102 is the default. Mutually exclusive with -npty.
103 -s <node> Load the program from this node. This option is not valid on
105 the command line if an application schema is specified.
106 -sigs Have LAM catch signals in the user process. This options is
108 mutually exclusive with -nsigs.
109 -ssi <key> <value>
111 Send arguments to various SSI modules. See the "SSI" sec‐
112 tion, below.
113 -t, -ton Enable execution trace generation for all processes. Trace
115 generation will proceed with no further action. These op‐
116 tions are mutually exclusive with -toff.
117 -toff Enable execution trace generation for all processes. Trace
119 generation for message passing traffic will begin after pro‐
120 cesses collectively call MPIL_Trace_on(2). Note that trace
121 generation for datatypes and communicators will proceed re‐
122 gardless of whether trace generation is enabled for messages
123 or not. This option is mutually exclusive with -t and -ton.
124 -tv Launch processes under the TotalView Debugger.
126 -v Be verbose; report on important steps as they are done.
128 -w Wait for all applications to exit before mpirun exits.
130 -wd <dir> Change to the directory <dir> before the user's program exe‐
132 cutes. Note that if the -wd option appears both on the com‐
133 mand line and in an application schema, the schema will take
134 precendence over the command line. This option is mutually
135 exclusive with -D.
136 -x Export the specified environment variables to the remote
138 nodes before executing the program. Existing environment
139 variables can be specified (see the Examples section, below),
140 or new variable names specified with corresponding values.
141 The parser for the -x option is not very sophisticated; it
142 does not even understand quoted values. Users are advised to
143 set variables in the environment, and then use -x to export
144 (not define) them.
145 -sa Display the exit status of all MPI processes irrespecive of
147 whether they fail or run successfully.
148 -sf Display the exit status of all processes only if one of them
150 fails.
151 -p <prefix_str>
153 Prefixes each process status line displayed by [-sa] and
154 [-sf] by the <prefix_str>.
155 <where> A set of node and/or CPU identifiers indicating where to
157 start <program>. See bhost(5) for a description of the node
158 and CPU identifiers. mpirun will schedule adjoining ranks in
159 MPI_COMM_WORLD on the same node when CPU identifiers are
160 used. For example, if LAM was booted with a CPU count of 4
161 on n0 and a CPU count of 2 on n1 and <where> is C, ranks 0
162 through 3 will be placed on n0, and ranks 4 and 5 will be
163 placed on n1.
164 <args> Pass these runtime arguments to every new process. These
166 must always be the last arguments to mpirun. This option is
167 not valid on the command line if an application schema is
168 specified.
169
171 One invocation of mpirun starts an MPI application running under LAM.
172 If the application is simply SPMD, the application can be specified on
173 the mpirun command line. If the application is MIMD, comprising multi‐
174 ple programs, an application schema is required in a separate file.
175 See appschema(5) for a description of the application schema syntax,
176 but it essentially contains multiple mpirun command lines, less the
177 command name itself. The ability to specify different options for dif‐
178 ferent instantiations of a program is another reason to use an applica‐
179 tion schema.
180 Location Nomenclature
182 As described above, mpirun can specify arbitrary locations in the cur‐
183 rent LAM universe. Locations can be specified either by CPU or by node
184 (noted by the "<where>" in the SYNTAX section, above). Note that LAM
185 does not bind processes to CPUs -- specifying a location "by CPU" is
186 really a convenience mechanism for SMPs that ultimately maps down to a
187 specific node.
188 Note that LAM effectively numbers MPI_COMM_WORLD ranks from left-to-
190 right in the <where>, regardless of which nomenclature is used. This
191 can be important because typical MPI programs tend to communicate more
192 with their immediate neighbors (i.e., myrank +/- X) than distant neigh‐
193 bors. When neighbors end up on the same node, the shmem RPIs can be
194 used for communication rather than the network RPIs, which can result
195 in faster MPI performance.
196 Specifying locations by node will launch one copy of an executable per
198 specified node. Using a capitol "N" tells LAM to use all available
199 nodes that were lambooted (see lamboot(1)). Ranges of specific nodes
200 can also be specified in the form "nR[,R]*", where R specifies either a
201 single node number or a valid range of node numbers in the range of [0,
202 num_nodes). For example:
203 mpirun N a.out
205 Runs one copy of the the executable a.out on all available nodes in
206 the LAM universe. MPI_COMM_WORLD rank 0 will be on n0, rank 1 will
207 be on n1, etc.
208 mpirun n0-3 a.out
210 Runs one copy of the the executable a.out on nodes 0 through 3.
211 MPI_COMM_WORLD rank 0 will be on n0, rank 1 will be on n1, etc.
212 mpirun n0-3,8-11,15 a.out
214 Runs one copy of the the executable a.out on nodes 0 through 3, 8
215 through 11, and 15. MPI_COMM_WORLD ranks will be ordered as fol‐
216 lows: (0, n0), (1, n1), (2, n2), (3, n3), (4, n8), (5, n9), (6,
217 n10), (7, n11), (8, n15).
218 Specifying by CPU is the preferred method of launching MPI jobs. The
220 intent is that the boot schema used with lamboot(1) will indicate how
221 many CPUs are available on each node, and then a single, simple mpirun
222 command can be used to launch across all of them. As noted above,
223 specifying CPUs does not actually bind processes to CPUs -- it is only
224 a convenience mechanism for launching on SMPs. Otherwise, the by-CPU
225 notation is the same as the by-node notation, except that "C" and "c"
226 are used instead of "N" and "n".
227 Assume in the following example that the LAM universe consists of four
229 4-way SMPs. So c0-3 are on n0, c4-7 are on n1, c8-11 are on n2, and
230 13-15 are on n3.
231 mpirun C a.out
233 Runs one copy of the the executable a.out on all available CPUs in
234 the LAM universe. This is typically the simplest (and preferred)
235 method of launching all MPI jobs (even if it resolves to one
236 process per node). MPI_COMM_WORLD ranks 0-3 will be on n0, ranks
237 4-7 will be on n1, ranks 8-11 will be on n2, and ranks 13-15 will
238 be on n3.
239 mpirun c0-3 a.out
241 Runs one copy of the the executable a.out on CPUs 0 through 3. All
242 four ranks of MPI_COMM_WORLD will be on MPI_COMM_WORLD.
243 mpirun c0-3,8-11,15 a.out
245 Runs one copy of the the executable a.out on CPUs 0 through 3, 8
246 through 11, and 15. MPI_COMM_WORLD ranks 0-3 will be on n0, 4-7
247 will be on n2, and 8 will be on n3.
248 The reason that the by-CPU nomenclature is preferred over the by-node
250 nomenclature is best shown through example. Consider trying to run the
251 first CPU example (with the same MPI_COMM_WORLD mapping) with the by-
252 node nomenclature -- run one copy of a.out for every available CPU, and
253 maximize the number of local neighbors to potentially maximize MPI per‐
254 formance. One solution would be to use the following command:
255 mpirun n0,0,0,0,1,1,1,1,2,2,2,2,3,3,3,3 a.out
257 This works, but is definitely klunky to type. It is typically easier
259 to use the by-CPU notation. One might think that the following is
260 equivalent:
261 mpirun N -np 16 a.out
263 This is not equivalent because the MPI_COMM_WORLD rank mappings will be
265 assigned by node rather than by CPU. Hence rank 0 will be on n0, rank
266 1 will be on n1, etc. Note that the following, however, is equivalent,
267 because LAM interprets lack of a <where> as "C":
268 mpirun -np 16 a.out
270 However, a "C" can tend to be more convenient, especially for batch-
272 queuing scripts because the exact number of processes may vary between
273 queue submissions. Since the batch system will determine the final
274 number of CPUs available, having a generic script that effectively says
275 "run on everything you gave me" may lead to more portable / re-usable
276 scripts.
277 Finally, it should be noted that specifying multiple <where> clauses
279 are perfectly acceptable. As such, mixing of the by-node and by-CPU
280 syntax is also valid, albiet typically not useful. For example:
281 mpirun C N a.out
283 However, in some cases, specifying multiple <where> clauses can be use‐
285 ful. Consider a parallel application where MPI_COMM_WORLD rank 0 will
286 be a "manager" and therefore consume very few CPU cycles because it is
287 usually waiting for "worker" processes to return results. Hence, it is
288 probably desirable to run one "worker" process on all available CPUs,
289 and run one extra process that will be the "manager":
290 mpirun c0 C manager-worker-program
292 Application Schema or Executable Program?
294 To distinguish the two different forms, mpirun looks on the command
295 line for <where> or the -c option. If neither is specified, then the
296 file named on the command line is assumed to be an application schema.
297 If either one or both are specified, then the file is assumed to be an
298 executable program. If <where> and -c both are specified, then copies
299 of the program are started on the specified nodes/CPUs according to an
300 internal LAM scheduling policy. Specifying just one node effectively
301 forces LAM to run all copies of the program in one place. If -c is
302 given, but not <where>, then all available CPUs on all LAM nodes are
303 used. If <where> is given, but not -c, then one copy of the program is
304 run on each node.
305 Program Transfer
307 By default, LAM searches for executable programs on the target node
308 where a particular instantiation will run. If the file system is not
309 shared, the target nodes are homogeneous, and the program is frequently
310 recompiled, it can be convenient to have LAM transfer the program from
311 a source node (usually the local node) to each target node. The -s op‐
312 tion specifies this behavior and identifies the single source node.
313 Locating Files
315 LAM looks for an executable program by searching the directories in the
316 user's PATH environment variable as defined on the source node(s).
317 This behavior is consistent with logging into the source node and exe‐
318 cuting the program from the shell. On remote nodes, the "." path is
319 the home directory.
320 LAM looks for an application schema in three directories: the local di‐
322 rectory, the value of the LAMAPPLDIR environment variable, and lamin‐
323 stalldir/boot, where "laminstalldir" is the directory where LAM/MPI was
324 installed.
325 Standard I/O
327 LAM directs UNIX standard input to /dev/null on all remote nodes. On
328 the local node that invoked mpirun, standard input is inherited from
329 mpirun. The default is what used to be the -w option to prevent con‐
330 flicting access to the terminal.
331 LAM directs UNIX standard output and error to the LAM daemon on all re‐
333 mote nodes. LAM ships all captured output/error to the node that in‐
334 voked mpirun and prints it on the standard output/error of mpirun. Lo‐
335 cal processes inherit the standard output/error of mpirun and transfer
336 to it directly.
337 Thus it is possible to redirect standard I/O for LAM applications by
339 using the typical shell redirection procedure on mpirun.
340 % mpirun C my_app < my_input > my_output
342 Note that in this example only the local node (i.e., the node where
344 mpirun was invoked from) will receive the stream from my_input on
345 stdin. The stdin on all the other nodes will be tied to /dev/null.
346 However, the stdout from all nodes will be collected into the my_output
347 file.
348 The -f option avoids all the setup required to support standard I/O de‐
350 scribed above. Remote processes are completely directed to /dev/null
351 and local processes inherit file descriptors from lamboot(1).
352 Pseudo-tty support
354 The -pty option enabled pseudo-tty support for process output (it is
355 also enabled by default). This allows, among other things, for line
356 buffered output from remote nodes (which is probably what you want).
357 This option can be disabled with the -npty switch.
358 Process Termination / Signal Handling
360 During the run of an MPI application, if any rank dies abnormally (ei‐
361 ther exiting before invoking MPI_FINALIZE, or dying as the result of a
362 signal), mpirun will print out an error message and kill the rest of
363 the MPI application.
364 By default, LAM/MPI only installs a signal handler for one signal in
366 user programs (SIGUSR2 by default, but this can be overridden when LAM
367 is configured and built). Therefore, it is safe for users to install
368 their own signal handlers in LAM/MPI programs (LAM notices death-by-
369 signal cases by examining the process' return status provided by the
370 operating system).
371 User signal handlers should probably avoid trying to cleanup MPI state
373 -- LAM is neither thread-safe nor async-signal-safe. For example, if a
374 seg fault occurs in MPI_SEND (perhaps because a bad buffer was passed
375 in) and a user signal handler is invoked, if this user handler attempts
376 to invoke MPI_FINALIZE, Bad Things could happen since LAM/MPI was al‐
377 ready "in" MPI when the error occurred. Since mpirun will notice that
378 the process died due to a signal, it is probably not necessary (and
379 safest) for the user to only clean up non-MPI state.
380 If the -sigs option is used with mpirun, LAM/MPI will install several
382 signal handlers to locally on each rank to catch signals, print out er‐
383 ror messages, and kill the rest of the MPI application. This is some‐
384 what redundant behavior since this is now all handled by mpirun, but it
385 has been left for backwards compatability.
386 Process Exit Statuses
388 The -sa, -sf, and -p parameters can be used to display the exist sta‐
389 tuses of the individual MPI processes as they terminate. -sa forces
390 the exit statuses to be displayed for all processes; -sf only displays
391 the exist statuses if at least one process terminates either by a sig‐
392 nal or a non-zero exit status (note that exiting before invoking
393 MPI_FINALIZE will cause a non-zero exit status).
394 The status of each process is printed out, one per line, in the follow‐
396 ing format:
397 prefix_string node pid killed status
399 If killed is 1, then status is the signal number. If killed is 0, then
401 status is the exit status of the process.
402 The default prefix_string is "mpirun:", but the -p option can be used
404 override this string.
405 Current Working Directory
407 The default behavior of mpirun has changed with respect to the directo‐
408 ry that processes will be started in.
409 The -wd option to mpirun allows the user to change to an arbitrary di‐
411 rectory before their program is invoked. It can also be used in appli‐
412 cation schema files to specify working directories on specific nodes
413 and/or for specific applications.
414 If the -wd option appears both in a schema file and on the command
416 line, the schema file directory will override the command line value.
417 The -D option will change the current working directory to the directo‐
419 ry where the executable resides. It cannot be used in application
420 schema files. -wd is mutually exclusive with -D.
421 If neither -wd nor -D are specified, the local node will send the di‐
423 rectory name where mpirun was invoked from to each of the remote nodes.
424 The remote nodes will then try to change to that directory. If they
425 fail (e.g., if the directory does not exists on that node), they will
426 start with from the user's home directory.
427 All directory changing occurs before the user's program is invoked; it
429 does not wait until MPI_INIT is called.
430 Process Environment
432 Processes in the MPI application inherit their environment from the LAM
433 daemon upon the node on which they are running. The environment of a
434 LAM daemon is fixed upon booting of the LAM with lamboot(1) and is typ‐
435 ically inherited from the user's shell. On the origin node, this will
436 be the shell from which lamboot(1) was invoked; on remote nodes, the
437 exact environment is determined by the boot SSI module used by lam‐
438 boot(1). The rsh boot module, for example, uses either rsh/ssh to
439 launch the LAM daemon on remote nodes, and typically executes one or
440 more of the user's shell-setup files before launching the LAM daemon.
441 When running dynamically linked applications which require the LD_LI‐
442 BRARY_PATH environment variable to be set, care must be taken to ensure
443 that it is correctly set when booting the LAM.
444 Exported Environment Variables
446 All environment variables that are named in the form LAM_MPI_*,
447 LAM_IMPI_*, or IMPI_* will automatically be exported to new processes
448 on the local and remote nodes. This exporting may be inhibited with
449 the -nx option.
450 Additionally, the -x option to mpirun can be used to export specific
452 environment variables to the new processes. While the syntax of the -x
453 option allows the definition of new variables, note that the parser for
454 this option is currently not very sophisticated - it does not even un‐
455 derstand quoted values. Users are advised to set variables in the en‐
456 vironment and use -x to export them; not to define them.
457 Trace Generation
459 Two switches control trace generation from processes running under LAM
460 and both must be in the on position for traces to actually be generat‐
461 ed. The first switch is controlled by mpirun and the second switch is
462 initially set by mpirun but can be toggled at runtime with
463 MPIL_Trace_on(2) and MPIL_Trace_off(2). The -t (-ton is equivalent)
464 and -toff options all turn on the first switch. Otherwise the first
465 switch is off and calls to MPIL_Trace_on(2) in the application program
466 are ineffective. The -t option also turns on the second switch. The
467 -toff option turns off the second switch. See MPIL_Trace_on(2) and
468 lamtrace(1) for more details.
469 MPI Data Conversion
471 LAM's MPI library converts MPI messages from local representation to
472 LAM representation upon sending them and then back to local representa‐
473 tion upon receiving them. If the case of a LAM consisting of a homoge‐
474 neous network of machines where the local representation differs from
475 the LAM representation this can result in unnecessary conversions.
476 The -O switch used to be necessary to indicate to LAM whether the
478 mulitcomputer was homogeneous or not. LAM now automatically determines
479 whether a given MPI job is homogeneous or not. The -O flag will
480 silently be accepted for backwards compatability, but it is ignored.
481 SSI (System Services Interface)
483 The -ssi switch allows the passing of parameters to various SSI mod‐
484 ules. LAM's SSI modules are described in detail in lamssi(7). SSI
485 modules have direct impact on MPI programs because they allow tunable
486 parameters to be set at run time (such as which RPI communication de‐
487 vice driver to use, what parameters to pass to that RPI, etc.).
488 The -ssi switch takes two arguments: <key> and <value>. The <key> ar‐
490 gument generally specifies which SSI module will receive the value.
491 For example, the <key> "rpi" is used to select which RPI to be used for
492 transporting MPI messages. The <value> argument is the value that is
493 passed. For example:
494 mpirun -ssi rpi lamd N foo
496 Tells LAM to use the "lamd" RPI and to run a single copy of "foo"
497 on every node.
498 mpirun -ssi rpi tcp N foo
500 Tells LAM to use the "tcp" RPI.
501 mpirun -ssi rpi sysv N foo
503 Tells LAM to use the "sysv" RPI.
504 And so on. LAM's RPI SSI modules are described in lamssi_rpi(7).
506 The -ssi switch can be used multiple times to specify different <key>
508 and/or <value> arguments. If the same <key> is specified more than
509 once, the <value>s are concatenated with a comma (",") separating them.
510 Note that the -ssi switch is simply a shortcut for setting environment
512 variables. The same effect may be accomplished by setting correspond‐
513 ing environment variables before running mpirun. The form of the envi‐
514 ronment variables that LAM sets are: LAM_MPI_SSI_<key>=<value>.
515 Note that the -ssi switch overrides any previously set environment
517 variables. Also note that unknown <key> arguments are still set as en‐
518 vironment variable -- they are not checked (by mpirun) for correctness.
519 Illegal or incorrect <value> arguments may or may not be reported -- it
520 depends on the specific SSI module.
521 The -ssi switch obsoletes the old -c2c and -lamd switches. These
523 switches used to be relevant because LAM could only have two RPI's
524 available at a time: the lamd RPI and one of the C2C RPIs. This is no
525 longer true -- all RPI's are now available and choosable at run-time.
526 Selecting the lamd RPI is shown in the examples above. The -c2c switch
527 has no direct translation since "C2C" used to refer to all other RPI's
528 that were not the lamd RPI. As such, -ssi rpi <value> must be used to
529 select the specific desired RPI (whether it is "lamd" or one of the
530 other RPI's).
531 Guaranteed Envelope Resources
533 By default, LAM will guarantee a minimum amount of message envelope
534 buffering to each MPI process pair and will impede or report an error
535 to a process that attempts to overflow this system resource. This ro‐
536 bustness and debugging feature is implemented in a machine specific
537 manner when direct communication is used. For normal LAM communication
538 via the LAM daemon, a protocol is used. The -nger option disables GER
539 and the measures taken to support it. The minimum GER is configured by
540 the system administrator when LAM is installed. See MPI(7) for more
541 details.
542
544 Be sure to also see the examples in the "Location Nomenclature" sec‐
545 tion, above.
546 mpirun N prog1
548 Load and execute prog1 on all nodes. Search the user's $PATH for
549 the executable file on each node.
550 mpirun -c 8 prog1
552 Run 8 copies of prog1 wherever LAM wants to run them.
553 mpirun n8-10 -v -nw -s n3 prog1 -q
555 Load and execute prog1 on nodes 8, 9, and 10. Search for prog1 on
556 node 3 and transfer it to the three target nodes. Report as each
557 process is created. Give "-q" as a command line to each new
558 process. Do not wait for the processes to complete before exiting
559 mpirun.
560 mpirun -v myapp
562 Parse the application schema, myapp, and start all processes speci‐
563 fied in it. Report as each process is created.
564 mpirun -npty -wd /work/output -x DISPLAY C my_application
566 Start one copy of "my_application" on each available CPU. The num‐
568 ber of available CPUs on each node was previously specified when
569 LAM was booted with lamboot(1). As noted above, mpirun will sched‐
570 ule adjoining rank in MPI_COMM_WORLD on the same node where possi‐
571 ble. For example, if n0 has a CPU count of 8, and n1 has a CPU
572 count of 4, mpirun will place MPI_COMM_WORLD ranks 0 through 7 on
573 n0, and 8 through 11 on n1. This tends to maximize on-node commu‐
574 nication for many parallel applications; when used in conjunction
575 with the multi-protocol network/shared memory RPIs in LAM (see the
576 RELEASE_NOTES and INSTALL files with the LAM distribution), overall
577 communication performance can be quite good. Also disable pseudo-
578 tty support, change directory to /work/output, and export the DIS‐
579 PLAY variable to the new processes (perhaps my_application will in‐
580 voke an X application such as xv to display output).
581
583 mpirun: Exec format error
584 This usually means that either a number of processes or an appro‐
585 priate <where> clause was not specified, indicating that LAM does
586 not know how many processes to run. See the EXAMPLES and "Location
587 Nomenclature" sections, above, for examples on how to specify how
588 many processes to run, and/or where to run them. However, it can
589 also mean that a non-ASCII character was detected in the applica‐
590 tion schema. This is usually a command line usage error where
591 mpirun is expecting an application schema and an executable file
592 was given.
593 mpirun: syntax error in application schema, line XXX
595 The application schema cannot be parsed because of a usage or syn‐
596 tax error on the given line in the file.
597 <filename>: No such file or directory
599 This error can occur in two cases. Either the named file cannot be
600 located or it has been found but the user does not have sufficient
601 permissions to execute the program or read the application schema.
602
604 mpirun returns 0 if all ranks started by mpirun exit after calling
605 MPI_FINALIZE. A non-zero value is returned if an internal error oc‐
606 curred in mpirun, or one or more ranks exited before calling MPI_FINAL‐
607 IZE. If an internal error occurred in mpirun, the corresponding error
608 code is returned. In the event that one or more ranks exit before
609 calling MPI_FINALIZE, the return value of the rank of the process that
610 mpirun first notices died before calling MPI_FINALIZE will be returned.
611 Note that, in general, this will be the first rank that died but is not
612 guaranteed to be so.
613 However, note that if the -nw switch is used, the return value from
615 mpirun does not indicate the exit status of the ranks.
616
618 bhost(5), lamexec(1), lamssi(7), lamssi_rpi(7), lamtrace(1), loadgo(1),
619 MPIL_Trace_on(2), mpimsg(1), mpitask(1)
620LAM 7.1.2 March, 2006 MPIRUN(1)