1SEM(1)                             parallel                             SEM(1)
2
3
4

NAME

6       sem - semaphore for executing shell command lines in parallel
7

SYNOPSIS

9       sem [--fg] [--id <id>] [--semaphoretimeout <secs>] [-j <num>] [--wait]
10       command
11

DESCRIPTION

13       GNU sem is an alias for GNU parallel --semaphore.
14
15       GNU sem acts as a counting semaphore. When GNU sem is called with
16       command it starts the command in the background. When num number of
17       commands are running in the background, GNU sem waits for one of these
18       to complete before starting the command.
19
20       GNU sem does not read any arguments to build the command (no -a, :::,
21       and ::::). It simply waits for a semaphore to become available and then
22       runs the command given.
23
24       Before looking at the options you may want to check out the examples
25       after the list of options. That will give you an idea of what GNU sem
26       is capable of.
27

OPTIONS

29       command  Command to execute. The command may be followed by arguments
30                for the command.
31
32       --bg     Run command in background thus GNU sem will not wait for
33                completion of the command before exiting. This is the default.
34
35                In toilet analogy: GNU sem waits for a toilet to be available,
36                gives the toilet to a person, and exits immediately.
37
38                See also: --fg
39
40       --jobs N
41       -j N
42       --max-procs N
43       -P N     Run up to N commands in parallel. Default is 1 thus acting
44                like a mutex.
45
46                In toilet analogy: -j is the number of toilets.
47
48       --jobs +N
49       -j +N
50       --max-procs +N
51       -P +N    Add N to the number of CPU cores.  Run up to this many jobs in
52                parallel. For compute intensive jobs -j +0 is useful as it
53                will run number-of-cpu-cores jobs simultaneously.
54
55       --jobs -N
56       -j -N
57       --max-procs -N
58       -P -N    Subtract N from the number of CPU cores.  Run up to this many
59                jobs in parallel.  If the evaluated number is less than 1 then
60                1 will be used.  See also --use-cpus-instead-of-cores.
61
62       --jobs N%
63       -j N%
64       --max-procs N%
65       -P N%    Multiply N% with the number of CPU cores.  Run up to this many
66                jobs in parallel.  If the evaluated number is less than 1 then
67                1 will be used.  See also --use-cpus-instead-of-cores.
68
69       --jobs procfile
70       -j procfile
71       --max-procs procfile
72       -P procfile
73                Read parameter from file. Use the content of procfile as
74                parameter for -j. E.g. procfile could contain the string 100%
75                or +2 or 10.
76
77       --semaphorename name
78       --id name
79                Use name as the name of the semaphore. Default is the name of
80                the controlling tty (output from tty).
81
82                The default normally works as expected when used
83                interactively, but when used in a script name should be set.
84                $$ or my_task_name are often a good value.
85
86                The semaphore is stored in ~/.parallel/semaphores/
87
88                In toilet analogy the name corresponds to different types of
89                toilets: e.g. male, female, customer, staff.
90
91       --fg     Do not put command in background.
92
93                In toilet analogy: GNU sem waits for a toilet to be available,
94                takes a person to the toilet, waits for the person to finish,
95                and exits.
96
97       --semaphoretimeout secs
98       --st secs
99                If secs > 0: If the semaphore is not released within secs
100                seconds, take it anyway.
101
102                If secs < 0: If the semaphore is not released within secs
103                seconds, exit.
104
105                In toilet analogy: secs > 0: If no toilet becomes available
106                within secs seconds, pee on the floor. secs < 0: If no toilet
107                becomes available within secs seconds, exit without doing
108                anything.
109
110       --wait   Wait for all commands to complete.
111
112                In toilet analogy: Wait until all toilets are empty, then
113                exit.
114

UNDERSTANDING A SEMAPHORE

116       Try the following example:
117
118         sem -j 2 'sleep 1;echo 1 finished';   echo sem 1 exited
119         sem -j 2 'sleep 2;echo 2 finished';   echo sem 2 exited
120         sem -j 2 'sleep 3;echo 3 finished';   echo sem 3 exited
121         sem -j 2 'sleep 4;echo 4 finished';   echo sem 4 exited
122         sem --wait; echo sem --wait done
123
124       In toilet analogy this uses 2 toilets (-j 2). GNU sem takes '1' to a
125       toilet, and exits immediately. While '1' is sleeping, another GNU sem
126       takes '2' to a toilet, and exits immediately.
127
128       While '1' and '2' are sleeping, another GNU sem waits for a free
129       toilet. When '1' finishes, a toilet becomes available, and this GNU sem
130       stops waiting, and takes '3' to a toilet, and exits immediately.
131
132       While '2' and '3' are sleeping, another GNU sem waits for a free
133       toilet.  When '2' finishes, a toilet becomes available, and this GNU
134       sem stops waiting, and takes '4' to a toilet, and exits immediately.
135
136       Finally another GNU sem waits for all toilets to become free.
137

EXAMPLE: Gzipping *.log

139       Run one gzip process per CPU core. Block until a CPU core becomes
140       available.
141
142         for i in *.log ; do
143           echo $i
144           sem -j+0 gzip $i ";" echo done
145         done
146         sem --wait
147

EXAMPLE: Protecting pod2html from itself

149       pod2html creates two files: pod2htmd.tmp and pod2htmi.tmp which it does
150       not clean up. It uses these two files for a short time. But if you run
151       multiple pod2html in parallel (e.g. in a Makefile with make -j) there
152       is a risk that two different instances of pod2html will write to the
153       files at the same time:
154
155         # This may fail due to shared pod2htmd.tmp/pod2htmi.tmp files
156         foo.html:
157                 pod2html foo.pod --outfile foo.html
158
159         bar.html:
160                 pod2html bar.pod --outfile bar.html
161
162         $ make -j foo.html bar.html
163
164       You need to protect pod2html from running twice at the same time.  sem
165       running as a mutex will make sure only one runs:
166
167         foo.html:
168                 sem --id pod2html pod2html foo.pod --outfile foo.html
169
170         bar.html:
171                 sem --id pod2html pod2html bar.pod --outfile bar.html
172
173         clean: foo.html bar.html
174                 sem --id pod2html --wait
175                 rm -f pod2htmd.tmp pod2htmi.tmp
176
177         $ make -j foo.html bar.html clean
178

BUGS

180       None known.
181

REPORTING BUGS

183       Report bugs to <bug-parallel@gnu.org>.
184

AUTHOR

186       Copyright (C) 2010-2022 Ole Tange, http://ole.tange.dk and Free
187       Software Foundation, Inc.
188

LICENSE

190       This program is free software; you can redistribute it and/or modify it
191       under the terms of the GNU General Public License as published by the
192       Free Software Foundation; either version 3 of the License, or at your
193       option any later version.
194
195       This program is distributed in the hope that it will be useful, but
196       WITHOUT ANY WARRANTY; without even the implied warranty of
197       MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
198       General Public License for more details.
199
200       You should have received a copy of the GNU General Public License along
201       with this program.  If not, see <https://www.gnu.org/licenses/>.
202
203   Documentation license I
204       Permission is granted to copy, distribute and/or modify this
205       documentation under the terms of the GNU Free Documentation License,
206       Version 1.3 or any later version published by the Free Software
207       Foundation; with no Invariant Sections, with no Front-Cover Texts, and
208       with no Back-Cover Texts.  A copy of the license is included in the
209       file LICENSES/GFDL-1.3-or-later.txt.
210
211   Documentation license II
212       You are free:
213
214       to Share to copy, distribute and transmit the work
215
216       to Remix to adapt the work
217
218       Under the following conditions:
219
220       Attribution
221                You must attribute the work in the manner specified by the
222                author or licensor (but not in any way that suggests that they
223                endorse you or your use of the work).
224
225       Share Alike
226                If you alter, transform, or build upon this work, you may
227                distribute the resulting work only under the same, similar or
228                a compatible license.
229
230       With the understanding that:
231
232       Waiver   Any of the above conditions can be waived if you get
233                permission from the copyright holder.
234
235       Public Domain
236                Where the work or any of its elements is in the public domain
237                under applicable law, that status is in no way affected by the
238                license.
239
240       Other Rights
241                In no way are any of the following rights affected by the
242                license:
243
244                • Your fair dealing or fair use rights, or other applicable
245                  copyright exceptions and limitations;
246
247                • The author's moral rights;
248
249                • Rights other persons may have either in the work itself or
250                  in how the work is used, such as publicity or privacy
251                  rights.
252
253       Notice   For any reuse or distribution, you must make clear to others
254                the license terms of this work.
255
256       A copy of the full license is included in the file as
257       LICENCES/CC-BY-SA-4.0.txt
258

DEPENDENCIES

260       GNU sem uses Perl, and the Perl modules Getopt::Long, Symbol, Fcntl.
261

SEE ALSO

263       parallel(1)
264
265
266
26720211222                          2021-12-25                            SEM(1)
Impressum