1SEM(1) parallel SEM(1)
2
6 sem - semaphore for executing shell command lines in parallel
7
9 sem [--fg] [--id <id>] [--semaphoretimeout <secs>] [-j <num>] [--wait]
10 command
11
13 GNU sem is an alias for GNU parallel --semaphore.
14 GNU sem acts as a counting semaphore. When GNU sem is called with
16 command it starts the command in the background. When num number of
17 commands are running in the background, GNU sem waits for one of these
18 to complete before starting the command.
19 GNU sem does not read any arguments to build the command (no -a, :::,
21 and ::::). It simply waits for a semaphore to become available and then
22 runs the command given.
23 Before looking at the options you may want to check out the examples
25 after the list of options. That will give you an idea of what GNU sem
26 is capable of.
27
29 command Command to execute. The command may be followed by arguments
30 for the command.
31 --bg Run command in background thus GNU sem will not wait for
33 completion of the command before exiting. This is the default.
34 In toilet analogy: GNU sem waits for a toilet to be available,
36 gives the toilet to a person, and exits immediately.
37 See also: --fg
39 --jobs N
41 -j N
42 --max-procs N
43 -P N Run up to N commands in parallel. Default is 1 thus acting
44 like a mutex.
45 In toilet analogy: -j is the number of toilets.
47 --jobs +N
49 -j +N
50 --max-procs +N
51 -P +N Add N to the number of CPU cores. Run up to this many jobs in
52 parallel. For compute intensive jobs -j +0 is useful as it
53 will run number-of-cpu-cores jobs simultaneously.
54 --jobs -N
56 -j -N
57 --max-procs -N
58 -P -N Subtract N from the number of CPU cores. Run up to this many
59 jobs in parallel. If the evaluated number is less than 1 then
60 1 will be used. See also --use-cpus-instead-of-cores.
61 --jobs N%
63 -j N%
64 --max-procs N%
65 -P N% Multiply N% with the number of CPU cores. Run up to this many
66 jobs in parallel. If the evaluated number is less than 1 then
67 1 will be used. See also --use-cpus-instead-of-cores.
68 --jobs procfile
70 -j procfile
71 --max-procs procfile
72 -P procfile
73 Read parameter from file. Use the content of procfile as
74 parameter for -j. E.g. procfile could contain the string 100%
75 or +2 or 10.
76 --semaphorename name
78 --id name
79 Use name as the name of the semaphore. Default is the name of
80 the controlling tty (output from tty).
81 The default normally works as expected when used
83 interactively, but when used in a script name should be set.
84 $$ or my_task_name are often a good value.
85 The semaphore is stored in ~/.parallel/semaphores/
87 In toilet analogy the name corresponds to different types of
89 toilets: e.g. male, female, customer, staff.
90 --fg Do not put command in background.
92 In toilet analogy: GNU sem waits for a toilet to be available,
94 takes a person to the toilet, waits for the person to finish,
95 and exits.
96 --semaphoretimeout secs
98 --st secs
99 If secs > 0: If the semaphore is not released within secs
100 seconds, take it anyway.
101 If secs < 0: If the semaphore is not released within secs
103 seconds, exit.
104 In toilet analogy: secs > 0: If no toilet becomes available
106 within secs seconds, pee on the floor. secs < 0: If no toilet
107 becomes available within secs seconds, exit without doing
108 anything.
109 --wait Wait for all commands to complete.
111 In toilet analogy: Wait until all toilets are empty, then
113 exit.
114
116 Try the following example:
117 sem -j 2 'sleep 1;echo 1 finished'; echo sem 1 exited
119 sem -j 2 'sleep 2;echo 2 finished'; echo sem 2 exited
120 sem -j 2 'sleep 3;echo 3 finished'; echo sem 3 exited
121 sem -j 2 'sleep 4;echo 4 finished'; echo sem 4 exited
122 sem --wait; echo sem --wait done
123 In toilet analogy this uses 2 toilets (-j 2). GNU sem takes '1' to a
125 toilet, and exits immediately. While '1' is sleeping, another GNU sem
126 takes '2' to a toilet, and exits immediately.
127 While '1' and '2' are sleeping, another GNU sem waits for a free
129 toilet. When '1' finishes, a toilet becomes available, and this GNU sem
130 stops waiting, and takes '3' to a toilet, and exits immediately.
131 While '2' and '3' are sleeping, another GNU sem waits for a free
133 toilet. When '2' finishes, a toilet becomes available, and this GNU
134 sem stops waiting, and takes '4' to a toilet, and exits immediately.
135 Finally another GNU sem waits for all toilets to become free.
137
139 Run one gzip process per CPU core. Block until a CPU core becomes
140 available.
141 for i in *.log ; do
143 echo $i
144 sem -j+0 gzip $i ";" echo done
145 done
146 sem --wait
147
149 pod2html creates two files: pod2htmd.tmp and pod2htmi.tmp which it does
150 not clean up. It uses these two files for a short time. But if you run
151 multiple pod2html in parallel (e.g. in a Makefile with make -j) there
152 is a risk that two different instances of pod2html will write to the
153 files at the same time:
154 # This may fail due to shared pod2htmd.tmp/pod2htmi.tmp files
156 foo.html:
157 pod2html foo.pod --outfile foo.html
158 bar.html:
160 pod2html bar.pod --outfile bar.html
161 $ make -j foo.html bar.html
163 You need to protect pod2html from running twice at the same time. sem
165 running as a mutex will make sure only one runs:
166 foo.html:
168 sem --id pod2html pod2html foo.pod --outfile foo.html
169 bar.html:
171 sem --id pod2html pod2html bar.pod --outfile bar.html
172 clean: foo.html bar.html
174 sem --id pod2html --wait
175 rm -f pod2htmd.tmp pod2htmi.tmp
176 $ make -j foo.html bar.html clean
178
180 None known.
181
183 Report bugs to <bug-parallel@gnu.org>.
184
186 Copyright (C) 2010-2022 Ole Tange, http://ole.tange.dk and Free
187 Software Foundation, Inc.
188
190 This program is free software; you can redistribute it and/or modify it
191 under the terms of the GNU General Public License as published by the
192 Free Software Foundation; either version 3 of the License, or at your
193 option any later version.
194 This program is distributed in the hope that it will be useful, but
196 WITHOUT ANY WARRANTY; without even the implied warranty of
197 MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
198 General Public License for more details.
199 You should have received a copy of the GNU General Public License along
201 with this program. If not, see <https://www.gnu.org/licenses/>.
202 Documentation license I
204 Permission is granted to copy, distribute and/or modify this
205 documentation under the terms of the GNU Free Documentation License,
206 Version 1.3 or any later version published by the Free Software
207 Foundation; with no Invariant Sections, with no Front-Cover Texts, and
208 with no Back-Cover Texts. A copy of the license is included in the
209 file LICENSES/GFDL-1.3-or-later.txt.
210 Documentation license II
212 You are free:
213 to Share to copy, distribute and transmit the work
215 to Remix to adapt the work
217 Under the following conditions:
219 Attribution
221 You must attribute the work in the manner specified by the
222 author or licensor (but not in any way that suggests that they
223 endorse you or your use of the work).
224 Share Alike
226 If you alter, transform, or build upon this work, you may
227 distribute the resulting work only under the same, similar or
228 a compatible license.
229 With the understanding that:
231 Waiver Any of the above conditions can be waived if you get
233 permission from the copyright holder.
234 Public Domain
236 Where the work or any of its elements is in the public domain
237 under applicable law, that status is in no way affected by the
238 license.
239 Other Rights
241 In no way are any of the following rights affected by the
242 license:
243 • Your fair dealing or fair use rights, or other applicable
245 copyright exceptions and limitations;
246 • The author's moral rights;
248 • Rights other persons may have either in the work itself or
250 in how the work is used, such as publicity or privacy
251 rights.
252 Notice For any reuse or distribution, you must make clear to others
254 the license terms of this work.
255 A copy of the full license is included in the file as
257 LICENCES/CC-BY-SA-4.0.txt
258
260 GNU sem uses Perl, and the Perl modules Getopt::Long, Symbol, Fcntl.
261
263 parallel(1)
26420211222 2021-12-25 SEM(1)