1SEM(1) parallel SEM(1)
2
6 sem - semaphore for executing shell command lines in parallel
7
9 sem [--fg] [--id <id>] [--semaphoretimeout <secs>] [-j <num>] [--wait]
10 command
11
13 GNU sem is an alias for GNU parallel --semaphore.
14 GNU sem acts as a counting semaphore. When GNU sem is called with
16 command it starts the command in the background. When num number of
17 commands are running in the background, GNU sem waits for one of these
18 to complete before starting the command.
19 GNU sem does not read any arguments to build the command (no -a, :::,
21 and ::::). It simply waits for a semaphore to become available and then
22 runs the command given.
23 Before looking at the options you may want to check out the examples
25 after the list of options. That will give you an idea of what GNU sem
26 is capable of.
27
29 command Command to execute. The command may be followed by arguments
30 for the command.
31 --bg Run command in background thus GNU sem will not wait for
33 completion of the command before exiting. This is the default.
34 In toilet analogy: GNU sem waits for a toilet to be available,
36 gives the toilet to a person, and exits immediately.
37 See also: --fg
39 --jobs N
41 -j N
42 --max-procs N
43 -P N Run up to N commands in parallel. Default is 1 thus acting
44 like a mutex.
45 In toilet analogy: -j is the number of toilets.
47 --jobs +N
49 -j +N
50 --max-procs +N
51 -P +N Add N to the number of CPU cores. Run up to this many jobs in
52 parallel. For compute intensive jobs -j +0 is useful as it
53 will run number-of-cpu-cores jobs simultaneously.
54 --jobs -N
56 -j -N
57 --max-procs -N
58 -P -N Subtract N from the number of CPU cores. Run up to this many
59 jobs in parallel. If the evaluated number is less than 1 then
60 1 will be used. See also --use-cpus-instead-of-cores.
61 --jobs N%
63 -j N%
64 --max-procs N%
65 -P N% Multiply N% with the number of CPU cores. Run up to this many
66 jobs in parallel. If the evaluated number is less than 1 then
67 1 will be used. See also --use-cpus-instead-of-cores.
68 --jobs procfile
70 -j procfile
71 --max-procs procfile
72 -P procfile
73 Read parameter from file. Use the content of procfile as
74 parameter for -j. E.g. procfile could contain the string 100%
75 or +2 or 10.
76 --pipe Pass stdin (standard input) to command.
78 If command read from stdin (standard input), use --pipe.
80 --semaphorename name
82 --id name
83 Use name as the name of the semaphore. Default is the name of
84 the controlling tty (output from tty).
85 The default normally works as expected when used
87 interactively, but when used in a script name should be set.
88 $$ or my_task_name are often a good value.
89 The semaphore is stored in ~/.parallel/semaphores/
91 In toilet analogy the name corresponds to different types of
93 toilets: e.g. male, female, customer, staff.
94 --fg Do not put command in background.
96 In toilet analogy: GNU sem waits for a toilet to be available,
98 takes a person to the toilet, waits for the person to finish,
99 and exits.
100 --semaphoretimeout secs
102 --st secs
103 If secs > 0: If the semaphore is not released within secs
104 seconds, take it anyway.
105 If secs < 0: If the semaphore is not released within secs
107 seconds, exit.
108 In toilet analogy: secs > 0: If no toilet becomes available
110 within secs seconds, pee on the floor. secs < 0: If no toilet
111 becomes available within secs seconds, exit without doing
112 anything.
113 --wait Wait for all commands to complete.
115 In toilet analogy: Wait until all toilets are empty, then
117 exit.
118
120 Try the following example:
121 sem -j 2 'sleep 1;echo 1 finished'; echo sem 1 exited
123 sem -j 2 'sleep 2;echo 2 finished'; echo sem 2 exited
124 sem -j 2 'sleep 3;echo 3 finished'; echo sem 3 exited
125 sem -j 2 'sleep 4;echo 4 finished'; echo sem 4 exited
126 sem --wait; echo sem --wait done
127 In toilet analogy this uses 2 toilets (-j 2). GNU sem takes '1' to a
129 toilet, and exits immediately. While '1' is sleeping, another GNU sem
130 takes '2' to a toilet, and exits immediately.
131 While '1' and '2' are sleeping, another GNU sem waits for a free
133 toilet. When '1' finishes, a toilet becomes available, and this GNU sem
134 stops waiting, and takes '3' to a toilet, and exits immediately.
135 While '2' and '3' are sleeping, another GNU sem waits for a free
137 toilet. When '2' finishes, a toilet becomes available, and this GNU
138 sem stops waiting, and takes '4' to a toilet, and exits immediately.
139 Finally another GNU sem waits for all toilets to become free.
141
143 Run one gzip process per CPU core. Block until a CPU core becomes
144 available.
145 for i in *.log ; do
147 echo $i
148 sem -j+0 gzip $i ";" echo done
149 done
150 sem --wait
151
153 pod2html creates two files: pod2htmd.tmp and pod2htmi.tmp which it does
154 not clean up. It uses these two files for a short time. But if you run
155 multiple pod2html in parallel (e.g. in a Makefile with make -j) there
156 is a risk that two different instances of pod2html will write to the
157 files at the same time:
158 # This may fail due to shared pod2htmd.tmp/pod2htmi.tmp files
160 foo.html:
161 pod2html foo.pod --outfile foo.html
162 bar.html:
164 pod2html bar.pod --outfile bar.html
165 $ make -j foo.html bar.html
167 You need to protect pod2html from running twice at the same time. sem
169 running as a mutex will make sure only one runs:
170 foo.html:
172 sem --id pod2html pod2html foo.pod --outfile foo.html
173 bar.html:
175 sem --id pod2html pod2html bar.pod --outfile bar.html
176 clean: foo.html bar.html
178 sem --id pod2html --wait
179 rm -f pod2htmd.tmp pod2htmi.tmp
180 $ make -j foo.html bar.html clean
182
184 None known.
185
187 Report bugs to <bug-parallel@gnu.org>.
188
190 Copyright (C) 2010-2023 Ole Tange, http://ole.tange.dk and Free
191 Software Foundation, Inc.
192
194 This program is free software; you can redistribute it and/or modify it
195 under the terms of the GNU General Public License as published by the
196 Free Software Foundation; either version 3 of the License, or at your
197 option any later version.
198 This program is distributed in the hope that it will be useful, but
200 WITHOUT ANY WARRANTY; without even the implied warranty of
201 MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
202 General Public License for more details.
203 You should have received a copy of the GNU General Public License along
205 with this program. If not, see <https://www.gnu.org/licenses/>.
206 Documentation license I
208 Permission is granted to copy, distribute and/or modify this
209 documentation under the terms of the GNU Free Documentation License,
210 Version 1.3 or any later version published by the Free Software
211 Foundation; with no Invariant Sections, with no Front-Cover Texts, and
212 with no Back-Cover Texts. A copy of the license is included in the
213 file LICENSES/GFDL-1.3-or-later.txt.
214 Documentation license II
216 You are free:
217 to Share to copy, distribute and transmit the work
219 to Remix to adapt the work
221 Under the following conditions:
223 Attribution
225 You must attribute the work in the manner specified by the
226 author or licensor (but not in any way that suggests that they
227 endorse you or your use of the work).
228 Share Alike
230 If you alter, transform, or build upon this work, you may
231 distribute the resulting work only under the same, similar or
232 a compatible license.
233 With the understanding that:
235 Waiver Any of the above conditions can be waived if you get
237 permission from the copyright holder.
238 Public Domain
240 Where the work or any of its elements is in the public domain
241 under applicable law, that status is in no way affected by the
242 license.
243 Other Rights
245 In no way are any of the following rights affected by the
246 license:
247 • Your fair dealing or fair use rights, or other applicable
249 copyright exceptions and limitations;
250 • The author's moral rights;
252 • Rights other persons may have either in the work itself or
254 in how the work is used, such as publicity or privacy
255 rights.
256 Notice For any reuse or distribution, you must make clear to others
258 the license terms of this work.
259 A copy of the full license is included in the file as
261 LICENCES/CC-BY-SA-4.0.txt
262
264 GNU sem uses Perl, and the Perl modules Getopt::Long, Symbol, Fcntl.
265
267 parallel(1)
26820230722 2023-07-28 SEM(1)