1ENV_PARALLEL(1) parallel ENV_PARALLEL(1)
2
3
4
6 env_parallel - export environment to GNU parallel
7
9 env_parallel [--record-env|--session|--end-session]
10 [options for GNU Parallel]
11
13 env_parallel is a shell function that exports the current environment
14 to GNU parallel.
15
16 If the shell function is not loaded, a dummy script will be run instead
17 that explains how to install the function.
18
19 env_parallel is 100 ms slower at startup than pure GNU parallel, and
20 takes up to 30% longer to start a job (typically 15 ms).
21
22 Due to the problem with environment space (see below) the recommended
23 usage is either:
24
25 # Do --record-env into $PARALLEL_IGNORED_NAMES
26 env_parallel --session
27
28 # Define whatever you want to use
29 myfunc() { myalias and functions $myvar work. $1.; }
30 alias myalias='echo Aliases'
31 myvar='and variables'
32
33 # env_parallel will not export names in $PARALLEL_IGNORED_NAMES
34 env_parallel -S localhost myfunc ::: Hooray
35
36 Or:
37
38 # Record the "clean" environment (this only needs to be run once)
39 env_parallel --record-env
40
41 # Optionally edit ~/.parallel/ignored_vars (only needed once)
42
43 # Define whatever you want to use
44 myfunc() { myalias and functions $myvar work. $1.; }
45 alias myalias='echo Aliases'
46 myvar='and variables'
47
48 # Use --env _ to only transfer the names not in the "empty" environment
49 env_parallel --env _ -S localhost myfunc ::: Hooray
50
51 In csh --session is not supported:
52
53 # Record the "clean" environment - this only needs to be run once
54 env_parallel --record-env
55
56 # Optionally edit ~/.parallel/ignored_vars - only needed once
57
58 # Define whatever you want to use
59 alias myalias 'echo Aliases $myvar \!*.'
60 set myvar='and variables'
61
62 # Use --env _ to only transfer the names not in the "empty" environment
63 env_parallel --env _ -S localhost myalias ::: work
64
65 Environment space
66 By default env_parallel will export all environment variables, arrays,
67 aliases, functions and shell options (see details for the individual
68 shells below).
69
70 But this only works if the size of the current environment is smaller
71 than the maximal length of a command and smaller than half of the max
72 if running remotely. E.g. The max size of Bash's command is 128 KB, so
73 env_parallel will fail if 'set | wc -c' is bigger than 128 KB.
74 Technically the limit is in execve(1) which IPC::open3 uses.
75
76 Bash completion functions are well-known for taking up well over 128 KB
77 of environment space and the primary reason for causing env_parallel to
78 fail.
79
80 Instead you can use --env to specify which variables, arrays, aliases
81 and functions to export as this will only export those with the given
82 name. Or follow the recommended usage in shown in DESCRIPTION.
83
85 Same as GNU parallel in addition to these:
86
87 --end-session
88 Undo last --session
89
90 --record-env
91 Record all names currently defined to be ignored every time running
92 env_parallel in the future.
93
94 --session
95 Ignore all names currently defined. Aliases, variables, arrays, and
96 functions currently defined will not be transferred.
97
98 But names defined after running parallel --session will be
99 transferred.
100
101 This is only valid in the running shell, and can be undone with
102 parallel --end-session.
103
104 You can run multiple --session inside each other:
105
106 env_parallel --session
107 var=not
108 # var is transferred
109 env_parallel -Slocalhost 'echo var is $var' ::: ignored
110 env_parallel --session
111 # var is not transferred
112 env_parallel -Slocalhost 'echo var is $var' ::: ignored
113 env_parallel --end-session
114 # var is transferred again
115 env_parallel -Slocalhost 'echo var is $var' ::: ignored
116
118 Ash
119 Installation
120
121 Put this in $HOME/.profile:
122
123 . `which env_parallel.ash`
124
125 E.g. by doing:
126
127 echo '. `which env_parallel.ash`' >> $HOME/.profile
128
129 Supported use
130
131 --env is supported to export only the variable, or alias with the given
132 name. Multiple --envs can be given.
133
134 --session is supported.
135
136 aliases
137 alias myecho='echo aliases'
138 env_parallel myecho ::: work
139 env_parallel -S server myecho ::: work
140 env_parallel --env myecho myecho ::: work
141 env_parallel --env myecho -S server myecho ::: work
142
143 alias multiline='echo multiline
144 echo aliases'
145 env_parallel multiline ::: work
146 env_parallel -S server multiline ::: work
147 env_parallel --env multiline multiline ::: work
148 env_parallel --env multiline -S server multiline ::: work
149
150 functions
151 ash cannot list defined functions - thus is not supported.
152
153 variables
154 myvar=variables
155 env_parallel echo '$myvar' ::: work
156 env_parallel -S server echo '$myvar' ::: work
157 env_parallel --env myvar echo '$myvar' ::: work
158 env_parallel --env myvar -S server echo '$myvar' ::: work
159
160 arrays Arrays are not supported by Ash.
161
162 Bash
163 Installation
164
165 Put this in $HOME/.bashrc:
166
167 . `which env_parallel.bash`
168
169 E.g. by doing:
170
171 echo '. `which env_parallel.bash`' >> $HOME/.bashrc
172
173 Supported use
174
175 --env is supported to export only the variable, alias, function, or
176 array with the given name. Multiple --envs can be given.
177
178 --session is supported.
179
180 aliases
181 alias myecho='echo aliases'
182 env_parallel myecho ::: work
183 env_parallel -S server myecho ::: work
184 env_parallel --env myecho myecho ::: work
185 env_parallel --env myecho -S server myecho ::: work
186
187 alias multiline='echo multiline
188 echo aliases'
189 env_parallel 'multiline {};
190 echo but only when followed by a newline' ::: work
191 env_parallel -S server 'multiline {};
192 echo but only when followed by a newline' ::: work
193 env_parallel --env multiline 'multiline {};
194 echo but only when followed by a newline' ::: work
195 env_parallel --env multiline -S server 'multiline {};
196 echo but only when followed by a newline' ::: work
197
198 functions
199 myfunc() { echo functions $*; }
200 env_parallel myfunc ::: work
201 env_parallel -S server myfunc ::: work
202 env_parallel --env myfunc myfunc ::: work
203 env_parallel --env myfunc -S server myfunc ::: work
204
205 variables
206 myvar=variables
207 env_parallel echo '$myvar' ::: work
208 env_parallel -S server echo '$myvar' ::: work
209 env_parallel --env myvar echo '$myvar' ::: work
210 env_parallel --env myvar -S server echo '$myvar' ::: work
211
212 arrays
213 myarray=(arrays work, too)
214 env_parallel -k echo '${myarray[{}]}' ::: 0 1 2
215 env_parallel -k -S server echo '${myarray[{}]}' ::: 0 1 2
216 env_parallel -k --env myarray echo '${myarray[{}]}' ::: 0 1 2
217 env_parallel -k --env myarray -S server \
218 echo '${myarray[{}]}' ::: 0 1 2
219
220 BUGS
221
222 Due to a bug in Bash, aliases containing newlines must be followed by a
223 newline in the command. Some systems are not affected by this bug, but
224 will print a warning anyway.
225
226 csh
227 env_parallel for csh breaks $PARALLEL, so do not use $PARALLEL.
228
229 Installation
230
231 Put this in $HOME/.cshrc:
232
233 source `which env_parallel.csh`
234
235 E.g. by doing:
236
237 echo 'source `which env_parallel.csh`' >> $HOME/.cshrc
238
239 Supported use
240
241 --env is supported to export only the variable, alias, or array with
242 the given name. Multiple --envs can be given.
243
244 aliases
245 alias myecho 'echo aliases'
246 env_parallel myecho ::: work
247 env_parallel -S server myecho ::: work
248 env_parallel --env myecho myecho ::: work
249 env_parallel --env myecho -S server myecho ::: work
250
251 functions
252 Not supported by csh.
253
254 variables
255 set myvar=variables
256 env_parallel echo '$myvar' ::: work
257 env_parallel -S server echo '$myvar' ::: work
258 env_parallel --env myvar echo '$myvar' ::: work
259 env_parallel --env myvar -S server echo '$myvar' ::: work
260
261 arrays with no special chars
262 set myarray=(arrays work, too)
263 env_parallel -k echo \$'{myarray[{}]}' ::: 1 2 3
264 env_parallel -k -S server echo \$'{myarray[{}]}' ::: 1 2 3
265 env_parallel -k --env myarray echo \$'{myarray[{}]}' ::: 1 2 3
266 env_parallel -k --env myarray -S server \
267 echo \$'{myarray[{}]}' ::: 1 2 3
268
269 Dash
270 Installation
271
272 Put this in $HOME/.profile:
273
274 . `which env_parallel.dash`
275
276 E.g. by doing:
277
278 echo '. `which env_parallel.dash`' >> $HOME/.profile
279
280 Supported use
281
282 --env is supported to export only the variable, or alias with the given
283 name. Multiple --envs can be given.
284
285 --session is supported.
286
287 aliases
288 alias myecho='echo aliases'
289 env_parallel myecho ::: work
290 env_parallel -S server myecho ::: work
291 env_parallel --env myecho myecho ::: work
292 env_parallel --env myecho -S server myecho ::: work
293
294 alias multiline='echo multiline
295 echo aliases'
296 env_parallel multiline ::: work
297 env_parallel -S server multiline ::: work
298 env_parallel --env multiline multiline ::: work
299 env_parallel --env multiline -S server multiline ::: work
300
301 functions
302 dash cannot list defined functions - thus is not supported.
303
304 variables
305 myvar=variables
306 env_parallel echo '$myvar' ::: work
307 env_parallel -S server echo '$myvar' ::: work
308 env_parallel --env myvar echo '$myvar' ::: work
309 env_parallel --env myvar -S server echo '$myvar' ::: work
310
311 arrays
312 dash does not support arrays.
313
314 fish
315 Installation
316
317 Put this in $HOME/.config/fish/config.fish:
318
319 source (which env_parallel.fish)
320
321 E.g. by doing:
322
323 echo 'source (which env_parallel.fish)' \
324 >> $HOME/.config/fish/config.fish
325
326 Supported use
327
328 --env is supported to export only the variable, alias, function, or
329 array with the given name. Multiple --envs can be given.
330
331 --session is supported.
332
333 aliases
334 alias myecho 'echo aliases'
335 env_parallel myecho ::: work
336 env_parallel -S server myecho ::: work
337 env_parallel --env myecho myecho ::: work
338 env_parallel --env myecho -S server myecho ::: work
339
340 functions
341 function myfunc
342 echo functions $argv
343 end
344 env_parallel myfunc ::: work
345 env_parallel -S server myfunc ::: work
346 env_parallel --env myfunc myfunc ::: work
347 env_parallel --env myfunc -S server myfunc ::: work
348
349 variables
350 set myvar variables
351 env_parallel echo '$myvar' ::: work
352 env_parallel -S server echo '$myvar' ::: work
353 env_parallel --env myvar echo '$myvar' ::: work
354 env_parallel --env myvar -S server echo '$myvar' ::: work
355
356 arrays
357 set myarray arrays work, too
358 env_parallel -k echo '$myarray[{}]' ::: 1 2 3
359 env_parallel -k -S server echo '$myarray[{}]' ::: 1 2 3
360 env_parallel -k --env myarray echo '$myarray[{}]' ::: 1 2 3
361 env_parallel -k --env myarray -S server \
362 echo '$myarray[{}]' ::: 1 2 3
363
364 ksh
365 Installation
366
367 Put this in $HOME/.kshrc:
368
369 source `which env_parallel.ksh`
370
371 E.g. by doing:
372
373 echo 'source `which env_parallel.ksh`' >> $HOME/.kshrc
374
375 Supported use
376
377 --env is supported to export only the variable, alias, function, or
378 array with the given name. Multiple --envs can be given.
379
380 --session is supported.
381
382 aliases
383 alias myecho='echo aliases'
384 env_parallel myecho ::: work
385 env_parallel -S server myecho ::: work
386 env_parallel --env myecho myecho ::: work
387 env_parallel --env myecho -S server myecho ::: work
388
389 alias multiline='echo multiline
390 echo aliases'
391 env_parallel multiline ::: work
392 env_parallel -S server multiline ::: work
393 env_parallel --env multiline multiline ::: work
394 env_parallel --env multiline -S server multiline ::: work
395
396 functions
397 myfunc() { echo functions $*; }
398 env_parallel myfunc ::: work
399 env_parallel -S server myfunc ::: work
400 env_parallel --env myfunc myfunc ::: work
401 env_parallel --env myfunc -S server myfunc ::: work
402
403 variables
404 myvar=variables
405 env_parallel echo '$myvar' ::: work
406 env_parallel -S server echo '$myvar' ::: work
407 env_parallel --env myvar echo '$myvar' ::: work
408 env_parallel --env myvar -S server echo '$myvar' ::: work
409
410 arrays
411 myarray=(arrays work, too)
412 env_parallel -k echo '${myarray[{}]}' ::: 0 1 2
413 env_parallel -k -S server echo '${myarray[{}]}' ::: 0 1 2
414 env_parallel -k --env myarray echo '${myarray[{}]}' ::: 0 1 2
415 env_parallel -k --env myarray -S server \
416 echo '${myarray[{}]}' ::: 0 1 2
417
418 mksh
419 Installation
420
421 Put this in $HOME/.mkshrc:
422
423 source `which env_parallel.mksh`
424
425 E.g. by doing:
426
427 echo 'source `which env_parallel.mksh`' >> $HOME/.mkshrc
428
429 Supported use
430
431 --env is supported to export only the variable, alias, function, or
432 array with the given name. Multiple --envs can be given.
433
434 --session is supported.
435
436 aliases
437 alias myecho='echo aliases'
438 env_parallel myecho ::: work
439 env_parallel -S server myecho ::: work
440 env_parallel --env myecho myecho ::: work
441 env_parallel --env myecho -S server myecho ::: work
442
443 alias multiline='echo multiline
444 echo aliases'
445 env_parallel multiline ::: work
446 env_parallel -S server multiline ::: work
447 env_parallel --env multiline multiline ::: work
448 env_parallel --env multiline -S server multiline ::: work
449
450 functions
451 myfunc() { echo functions $*; }
452 env_parallel myfunc ::: work
453 env_parallel -S server myfunc ::: work
454 env_parallel --env myfunc myfunc ::: work
455 env_parallel --env myfunc -S server myfunc ::: work
456
457 variables
458 myvar=variables
459 env_parallel echo '$myvar' ::: work
460 env_parallel -S server echo '$myvar' ::: work
461 env_parallel --env myvar echo '$myvar' ::: work
462 env_parallel --env myvar -S server echo '$myvar' ::: work
463
464 arrays
465 myarray=(arrays work, too)
466 env_parallel -k echo '${myarray[{}]}' ::: 0 1 2
467 env_parallel -k -S server echo '${myarray[{}]}' ::: 0 1 2
468 env_parallel -k --env myarray echo '${myarray[{}]}' ::: 0 1 2
469 env_parallel -k --env myarray -S server \
470 echo '${myarray[{}]}' ::: 0 1 2
471
472 pdksh
473 Installation
474
475 Put this in $HOME/.profile:
476
477 source `which env_parallel.pdksh`
478
479 E.g. by doing:
480
481 echo 'source `which env_parallel.pdksh`' >> $HOME/.profile
482
483 Supported use
484
485 --env is supported to export only the variable, alias, function, or
486 array with the given name. Multiple --envs can be given.
487
488 --session is supported.
489
490 aliases
491 alias myecho="echo aliases";
492 env_parallel myecho ::: work;
493 env_parallel -S server myecho ::: work;
494 env_parallel --env myecho myecho ::: work;
495 env_parallel --env myecho -S server myecho ::: work
496
497 functions
498 myfunc() { echo functions $*; };
499 env_parallel myfunc ::: work;
500 env_parallel -S server myfunc ::: work;
501 env_parallel --env myfunc myfunc ::: work;
502 env_parallel --env myfunc -S server myfunc ::: work
503
504 variables
505 myvar=variables;
506 env_parallel echo "\$myvar" ::: work;
507 env_parallel -S server echo "\$myvar" ::: work;
508 env_parallel --env myvar echo "\$myvar" ::: work;
509 env_parallel --env myvar -S server echo "\$myvar" ::: work
510
511 arrays
512 myarray=(arrays work, too);
513 env_parallel -k echo "\${myarray[{}]}" ::: 0 1 2;
514 env_parallel -k -S server echo "\${myarray[{}]}" ::: 0 1 2;
515 env_parallel -k --env myarray echo "\${myarray[{}]}" ::: 0 1 2;
516 env_parallel -k --env myarray -S server \
517 echo "\${myarray[{}]}" ::: 0 1 2
518
519 sh
520 Installation
521
522 Put this in $HOME/.profile:
523
524 . `which env_parallel.sh`
525
526 E.g. by doing:
527
528 echo '. `which env_parallel.sh`' >> $HOME/.profile
529
530 Supported use
531
532 --env is supported to export only the variable, or alias with the given
533 name. Multiple --envs can be given.
534
535 --session is supported.
536
537 aliases
538 sh does not support aliases.
539
540 functions
541 myfunc() { echo functions $*; }
542 env_parallel myfunc ::: work
543 env_parallel -S server myfunc ::: work
544 env_parallel --env myfunc myfunc ::: work
545 env_parallel --env myfunc -S server myfunc ::: work
546
547 variables
548 myvar=variables
549 env_parallel echo '$myvar' ::: work
550 env_parallel -S server echo '$myvar' ::: work
551 env_parallel --env myvar echo '$myvar' ::: work
552 env_parallel --env myvar -S server echo '$myvar' ::: work
553
554 arrays
555 sh does not support arrays.
556
557 tcsh
558 env_parallel for tcsh breaks $PARALLEL, so do not use $PARALLEL.
559
560 Installation
561
562 Put this in $HOME/.tcshrc:
563
564 source `which env_parallel.tcsh`
565
566 E.g. by doing:
567
568 echo 'source `which env_parallel.tcsh`' >> $HOME/.tcshrc
569
570 Supported use
571
572 --env is supported to export only the variable, alias, or array with
573 the given name. Multiple --envs can be given.
574
575 aliases
576 alias myecho 'echo aliases'
577 env_parallel myecho ::: work
578 env_parallel -S server myecho ::: work
579 env_parallel --env myecho myecho ::: work
580 env_parallel --env myecho -S server myecho ::: work
581
582 functions
583 Not supported by tcsh.
584
585 variables
586 set myvar=variables
587 env_parallel echo '$myvar' ::: work
588 env_parallel -S server echo '$myvar' ::: work
589 env_parallel --env myvar echo '$myvar' ::: work
590 env_parallel --env myvar -S server echo '$myvar' ::: work
591
592 arrays with no special chars
593 set myarray=(arrays work, too)
594 env_parallel -k echo \$'{myarray[{}]}' ::: 1 2 3
595 env_parallel -k -S server echo \$'{myarray[{}]}' ::: 1 2 3
596 env_parallel -k --env myarray echo \$'{myarray[{}]}' ::: 1 2 3
597 env_parallel -k --env myarray -S server \
598 echo \$'{myarray[{}]}' ::: 1 2 3
599
600 Zsh
601 Installation
602
603 Put this in $HOME/.zshrc:
604
605 . `which env_parallel.zsh`
606
607 E.g. by doing:
608
609 echo '. `which env_parallel.zsh`' >> $HOME/.zshenv
610
611 Supported use
612
613 --env is supported to export only the variable, alias, function, or
614 array with the given name. Multiple --envs can be given.
615
616 --session is supported.
617
618 aliases
619 alias myecho='echo aliases'
620 env_parallel myecho ::: work
621 env_parallel -S server myecho ::: work
622 env_parallel --env myecho myecho ::: work
623 env_parallel --env myecho -S server myecho ::: work
624
625 alias multiline='echo multiline
626 echo aliases'
627 env_parallel multiline ::: work
628 env_parallel -S server multiline ::: work
629 env_parallel --env multiline multiline ::: work
630 env_parallel --env multiline -S server multiline ::: work
631
632 functions
633 myfunc() { echo functions $*; }
634 env_parallel myfunc ::: work
635 env_parallel -S server myfunc ::: work
636 env_parallel --env myfunc myfunc ::: work
637 env_parallel --env myfunc -S server myfunc ::: work
638
639 variables
640 myvar=variables
641 env_parallel echo '$myvar' ::: work
642 env_parallel -S server echo '$myvar' ::: work
643 env_parallel --env myvar echo '$myvar' ::: work
644 env_parallel --env myvar -S server echo '$myvar' ::: work
645
646 arrays
647 myarray=(arrays work, too)
648 env_parallel -k echo '${myarray[{}]}' ::: 1 2 3
649 env_parallel -k -S server echo '${myarray[{}]}' ::: 1 2 3
650 env_parallel -k --env myarray echo '${myarray[{}]}' ::: 1 2 3
651 env_parallel -k --env myarray -S server \
652 echo '${myarray[{}]}' ::: 1 2 3
653
655 Same as GNU parallel.
656
658 When using GNU env_parallel for a publication please cite:
659
660 O. Tange (2018): GNU Parallel 2018, March 2018, ISBN 9781387509881,
661 DOI: 10.5281/zenodo.1146014.
662
663 This helps funding further development; and it won't cost you a cent.
664 If you pay 10000 EUR you should feel free to use GNU Parallel without
665 citing.
666
667 Copyright (C) 2007-10-18 Ole Tange, http://ole.tange.dk
668
669 Copyright (C) 2008-2010 Ole Tange, http://ole.tange.dk
670
671 Copyright (C) 2010-2022 Ole Tange, http://ole.tange.dk and Free
672 Software Foundation, Inc.
673
675 This program is free software; you can redistribute it and/or modify it
676 under the terms of the GNU General Public License as published by the
677 Free Software Foundation; either version 3 of the License, or at your
678 option any later version.
679
680 This program is distributed in the hope that it will be useful, but
681 WITHOUT ANY WARRANTY; without even the implied warranty of
682 MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
683 General Public License for more details.
684
685 You should have received a copy of the GNU General Public License along
686 with this program. If not, see <http://www.gnu.org/licenses/>.
687
688 Documentation license I
689 Permission is granted to copy, distribute and/or modify this
690 documentation under the terms of the GNU Free Documentation License,
691 Version 1.3 or any later version published by the Free Software
692 Foundation; with no Invariant Sections, with no Front-Cover Texts, and
693 with no Back-Cover Texts. A copy of the license is included in the
694 file LICENSES/GFDL-1.3-or-later.txt.
695
696 Documentation license II
697 You are free:
698
699 to Share to copy, distribute and transmit the work
700
701 to Remix to adapt the work
702
703 Under the following conditions:
704
705 Attribution
706 You must attribute the work in the manner specified by the
707 author or licensor (but not in any way that suggests that they
708 endorse you or your use of the work).
709
710 Share Alike
711 If you alter, transform, or build upon this work, you may
712 distribute the resulting work only under the same, similar or
713 a compatible license.
714
715 With the understanding that:
716
717 Waiver Any of the above conditions can be waived if you get
718 permission from the copyright holder.
719
720 Public Domain
721 Where the work or any of its elements is in the public domain
722 under applicable law, that status is in no way affected by the
723 license.
724
725 Other Rights
726 In no way are any of the following rights affected by the
727 license:
728
729 • Your fair dealing or fair use rights, or other applicable
730 copyright exceptions and limitations;
731
732 • The author's moral rights;
733
734 • Rights other persons may have either in the work itself or
735 in how the work is used, such as publicity or privacy
736 rights.
737
738 Notice For any reuse or distribution, you must make clear to others
739 the license terms of this work.
740
741 A copy of the full license is included in the file as
742 LICENCES/CC-BY-SA-4.0.txt
743
745 env_parallel uses GNU parallel.
746
748 parallel(1), ash(1), bash(1), csh(1), dash(1), fish(1), ksh(1),
749 pdksh(1) tcsh(1), zsh(1).
750
751
752
75320220322 2022-03-30 ENV_PARALLEL(1)