env_parallel(1)

1ENV_PARALLEL(1)                    parallel                    ENV_PARALLEL(1)
2
3
4

NAME

6       env_parallel - export environment to GNU parallel
7

SYNOPSIS

9       env_parallel [--record-env|--session|--end-session]
10         [options for GNU Parallel]
11

DESCRIPTION

13       env_parallel is a shell function that exports the current environment
14       to GNU parallel.
15
16       If the shell function is not loaded, a dummy script will be run instead
17       that explains how to install the function.
18
19       env_parallel is 100 ms slower at startup than pure GNU parallel, and
20       takes up to 30% longer to start a job (typically 15 ms).
21
22       Due to the problem with environment space (see below) the recommended
23       usage is either:
24
25         # Do --record-env into $PARALLEL_IGNORED_NAMES
26         env_parallel --session
27
28         # Define whatever you want to use
29         myfunc() { myalias and functions $myvar work. $1.; }
30         alias myalias='echo Aliases'
31         myvar='and variables'
32
33         # env_parallel will not export names in $PARALLEL_IGNORED_NAMES
34         env_parallel -S localhost myfunc ::: Hooray
35
36       Or:
37
38         # Record the "clean" environment (this only needs to be run once)
39         env_parallel --record-env
40
41         # Optionally edit ~/.parallel/ignored_vars (only needed once)
42
43         # Define whatever you want to use
44         myfunc() { myalias and functions $myvar work. $1.; }
45         alias myalias='echo Aliases'
46         myvar='and variables'
47
48         # Use --env _ to only transfer the names not in the "empty" environment
49         env_parallel --env _ -S localhost myfunc ::: Hooray
50
51       In csh --session is not supported:
52
53         # Record the "clean" environment - this only needs to be run once
54         env_parallel --record-env
55
56         # Optionally edit ~/.parallel/ignored_vars - only needed once
57
58         # Define whatever you want to use
59         alias myalias 'echo Aliases $myvar \!*.'
60         set myvar='and variables'
61
62         # Use --env _ to only transfer the names not in the "empty" environment
63         env_parallel --env _ -S localhost myalias ::: work
64
65   Environment space
66       By default env_parallel will export all environment variables, arrays,
67       aliases, functions and shell options (see details for the individual
68       shells below).
69
70       But this only works if the size of the current environment is smaller
71       than the maximal length of a command and smaller than half of the max
72       if running remotely. E.g. The max size of Bash's command is 128 KB, so
73       env_parallel will fail if 'set | wc -c' is bigger than 128 KB.
74       Technically the limit is in execve(1) which IPC::open3 uses.
75
76       Bash completion functions are well-known for taking up well over 128 KB
77       of environment space and the primary reason for causing env_parallel to
78       fail.
79
80       Instead you can use --env to specify which variables, arrays, aliases
81       and functions to export as this will only export those with the given
82       name. Or follow the recommended usage in shown in DESCRIPTION.
83

OPTIONS

85       Same as GNU parallel in addition to these:
86
87       --end-session
88           Undo last --session
89
90       --record-env
91           Record all names currently defined to be ignored every time running
92           env_parallel in the future.
93
94       --session
95           Ignore all names currently defined. Aliases, variables, arrays, and
96           functions currently defined will not be transferred.
97
98           But names defined after running parallel --session will be
99           transferred.
100
101           This is only valid in the running shell, and can be undone with
102           parallel --end-session.
103
104           You can run multiple --session inside each other:
105
106             env_parallel --session
107             var=not
108             # var is transferred
109             env_parallel -Slocalhost 'echo var is $var' ::: ignored
110             env_parallel --session
111             # var is not transferred
112             env_parallel -Slocalhost 'echo var is $var' ::: ignored
113             env_parallel --end-session
114             # var is transferred again
115             env_parallel -Slocalhost 'echo var is $var' ::: ignored
116

SUPPORTED SHELLS

118   Ash
119       Installation
120
121       Put this in $HOME/.profile:
122
123         . env_parallel.ash
124
125       E.g. by doing:
126
127         echo '. env_parallel.ash' >> $HOME/.profile
128
129       Supported use
130
131       --env is supported to export only the variable, or alias with the given
132       name. Multiple --envs can be given.
133
134       --session is supported.
135
136       aliases
137                 alias myecho='echo aliases'
138                 env_parallel myecho ::: work
139                 env_parallel -S server myecho ::: work
140                 env_parallel --env myecho myecho ::: work
141                 env_parallel --env myecho -S server myecho ::: work
142
143                 alias multiline='echo multiline
144                   echo aliases'
145                 env_parallel multiline ::: work
146                 env_parallel -S server multiline ::: work
147                 env_parallel --env multiline multiline ::: work
148                 env_parallel --env multiline -S server multiline ::: work
149
150       functions
151                 ash cannot list defined functions - thus is not supported.
152
153       variables
154                 myvar=variables
155                 env_parallel echo '$myvar' ::: work
156                 env_parallel -S server echo '$myvar' ::: work
157                 env_parallel --env myvar echo '$myvar' ::: work
158                 env_parallel --env myvar -S server echo '$myvar' ::: work
159
160       arrays  Arrays are not supported by Ash.
161
162   Bash
163       Installation
164
165       Put this in $HOME/.bashrc:
166
167         . env_parallel.bash
168
169       E.g. by doing:
170
171         echo '. env_parallel.bash' >> $HOME/.bashrc
172
173       Supported use
174
175       --env is supported to export only the variable, alias, function, or
176       array with the given name. Multiple --envs can be given.
177
178       --session is supported.
179
180       aliases
181                 alias myecho='echo aliases'
182                 env_parallel myecho ::: work
183                 env_parallel -S server myecho ::: work
184                 env_parallel --env myecho myecho ::: work
185                 env_parallel --env myecho -S server myecho ::: work
186
187                 alias multiline='echo multiline
188                   echo aliases'
189                 env_parallel 'multiline {};
190                   echo but only when followed by a newline' ::: work
191                 env_parallel -S server 'multiline {};
192                   echo but only when followed by a newline' ::: work
193                 env_parallel --env multiline 'multiline {};
194                   echo but only when followed by a newline' ::: work
195                 env_parallel --env multiline -S server 'multiline {};
196                   echo but only when followed by a newline' ::: work
197
198       functions
199                 myfunc() { echo functions $*; }
200                 env_parallel myfunc ::: work
201                 env_parallel -S server myfunc ::: work
202                 env_parallel --env myfunc myfunc ::: work
203                 env_parallel --env myfunc -S server myfunc ::: work
204
205       variables
206                 myvar=variables
207                 env_parallel echo '$myvar' ::: work
208                 env_parallel -S server echo '$myvar' ::: work
209                 env_parallel --env myvar echo '$myvar' ::: work
210                 env_parallel --env myvar -S server echo '$myvar' ::: work
211
212       arrays
213                 myarray=(arrays work, too)
214                 env_parallel -k echo '${myarray[{}]}' ::: 0 1 2
215                 env_parallel -k -S server echo '${myarray[{}]}' ::: 0 1 2
216                 env_parallel -k --env myarray echo '${myarray[{}]}' ::: 0 1 2
217                 env_parallel -k --env myarray -S server \
218                   echo '${myarray[{}]}' ::: 0 1 2
219
220       BUGS
221
222       Due to a bug in Bash, aliases containing newlines must be followed by a
223       newline in the command. Some systems are not affected by this bug, but
224       will print a warning anyway.
225
226   csh
227       env_parallel for csh breaks $PARALLEL, so do not use $PARALLEL.
228
229       Installation
230
231       Put this in $HOME/.cshrc:
232
233         source `which env_parallel.csh`
234
235       E.g. by doing:
236
237         echo 'source `which env_parallel.csh`' >> $HOME/.cshrc
238
239       Supported use
240
241       --env is supported to export only the variable, alias, or array with
242       the given name. Multiple --envs can be given.
243
244       aliases
245                 alias myecho 'echo aliases'
246                 env_parallel myecho ::: work
247                 env_parallel -S server myecho ::: work
248                 env_parallel --env myecho myecho ::: work
249                 env_parallel --env myecho -S server myecho ::: work
250
251       functions
252               Not supported by csh.
253
254       variables
255                 set myvar=variables
256                 env_parallel echo '$myvar' ::: work
257                 env_parallel -S server echo '$myvar' ::: work
258                 env_parallel --env myvar echo '$myvar' ::: work
259                 env_parallel --env myvar -S server echo '$myvar' ::: work
260
261       arrays with no special chars
262                 set myarray=(arrays work, too)
263                 env_parallel -k echo \$'{myarray[{}]}' ::: 1 2 3
264                 env_parallel -k -S server echo \$'{myarray[{}]}' ::: 1 2 3
265                 env_parallel -k --env myarray echo \$'{myarray[{}]}' ::: 1 2 3
266                 env_parallel -k --env myarray -S server \
267                   echo \$'{myarray[{}]}' ::: 1 2 3
268
269   Dash
270       Installation
271
272       Put this in $HOME/.profile:
273
274         . env_parallel.dash
275
276       E.g. by doing:
277
278         echo '. env_parallel.dash' >> $HOME/.profile
279
280       Supported use
281
282       --env is supported to export only the variable, or alias with the given
283       name. Multiple --envs can be given.
284
285       --session is supported.
286
287       aliases
288                 alias myecho='echo aliases'
289                 env_parallel myecho ::: work
290                 env_parallel -S server myecho ::: work
291                 env_parallel --env myecho myecho ::: work
292                 env_parallel --env myecho -S server myecho ::: work
293
294                 alias multiline='echo multiline
295                   echo aliases'
296                 env_parallel multiline ::: work
297                 env_parallel -S server multiline ::: work
298                 env_parallel --env multiline multiline ::: work
299                 env_parallel --env multiline -S server multiline ::: work
300
301       functions
302                 dash cannot list defined functions - thus is not supported.
303
304       variables
305                 myvar=variables
306                 env_parallel echo '$myvar' ::: work
307                 env_parallel -S server echo '$myvar' ::: work
308                 env_parallel --env myvar echo '$myvar' ::: work
309                 env_parallel --env myvar -S server echo '$myvar' ::: work
310
311       arrays
312                 dash does not support arrays.
313
314   fish
315       Installation
316
317       Put this in $HOME/.config/fish/config.fish:
318
319         source (which env_parallel.fish)
320
321       E.g. by doing:
322
323         echo 'source (which env_parallel.fish)' \
324           >> $HOME/.config/fish/config.fish
325
326       Supported use
327
328       --env is supported to export only the variable, alias, function, or
329       array with the given name. Multiple --envs can be given.
330
331       --session is supported.
332
333       aliases
334                 alias myecho 'echo aliases'
335                 env_parallel myecho ::: work
336                 env_parallel -S server myecho ::: work
337                 env_parallel --env myecho myecho ::: work
338                 env_parallel --env myecho -S server myecho ::: work
339
340       functions
341                 function myfunc
342                   echo functions $argv
343                 end
344                 env_parallel myfunc ::: work
345                 env_parallel -S server myfunc ::: work
346                 env_parallel --env myfunc myfunc ::: work
347                 env_parallel --env myfunc -S server myfunc ::: work
348
349       variables
350                 set myvar variables
351                 env_parallel echo '$myvar' ::: work
352                 env_parallel -S server echo '$myvar' ::: work
353                 env_parallel --env myvar echo '$myvar' ::: work
354                 env_parallel --env myvar -S server echo '$myvar' ::: work
355
356       arrays
357                 set myarray arrays work, too
358                 env_parallel -k echo '$myarray[{}]' ::: 1 2 3
359                 env_parallel -k -S server echo '$myarray[{}]' ::: 1 2 3
360                 env_parallel -k --env myarray echo '$myarray[{}]' ::: 1 2 3
361                 env_parallel -k --env myarray -S server \
362                   echo '$myarray[{}]' ::: 1 2 3
363
364   ksh
365       Installation
366
367       Put this in $HOME/.kshrc:
368
369         source env_parallel.ksh
370
371       E.g. by doing:
372
373         echo 'source env_parallel.ksh' >> $HOME/.kshrc
374
375       Supported use
376
377       --env is supported to export only the variable, alias, function, or
378       array with the given name. Multiple --envs can be given.
379
380       --session is supported.
381
382       aliases
383                 alias myecho='echo aliases'
384                 env_parallel myecho ::: work
385                 env_parallel -S server myecho ::: work
386                 env_parallel --env myecho myecho ::: work
387                 env_parallel --env myecho -S server myecho ::: work
388
389                 alias multiline='echo multiline
390                   echo aliases'
391                 env_parallel multiline ::: work
392                 env_parallel -S server multiline ::: work
393                 env_parallel --env multiline multiline ::: work
394                 env_parallel --env multiline -S server multiline ::: work
395
396       functions
397                 myfunc() { echo functions $*; }
398                 env_parallel myfunc ::: work
399                 env_parallel -S server myfunc ::: work
400                 env_parallel --env myfunc myfunc ::: work
401                 env_parallel --env myfunc -S server myfunc ::: work
402
403       variables
404                 myvar=variables
405                 env_parallel echo '$myvar' ::: work
406                 env_parallel -S server echo '$myvar' ::: work
407                 env_parallel --env myvar echo '$myvar' ::: work
408                 env_parallel --env myvar -S server echo '$myvar' ::: work
409
410       arrays
411                 myarray=(arrays work, too)
412                 env_parallel -k echo '${myarray[{}]}' ::: 0 1 2
413                 env_parallel -k -S server echo '${myarray[{}]}' ::: 0 1 2
414                 env_parallel -k --env myarray echo '${myarray[{}]}' ::: 0 1 2
415                 env_parallel -k --env myarray -S server \
416                   echo '${myarray[{}]}' ::: 0 1 2
417
418   mksh
419       Installation
420
421       Put this in $HOME/.mkshrc:
422
423         source env_parallel.mksh
424
425       E.g. by doing:
426
427         echo 'source env_parallel.mksh' >> $HOME/.mkshrc
428
429       Supported use
430
431       --env is supported to export only the variable, alias, function, or
432       array with the given name. Multiple --envs can be given.
433
434       --session is supported.
435
436       aliases
437                 alias myecho='echo aliases'
438                 env_parallel myecho ::: work
439                 env_parallel -S server myecho ::: work
440                 env_parallel --env myecho myecho ::: work
441                 env_parallel --env myecho -S server myecho ::: work
442
443                 alias multiline='echo multiline
444                   echo aliases'
445                 env_parallel multiline ::: work
446                 env_parallel -S server multiline ::: work
447                 env_parallel --env multiline multiline ::: work
448                 env_parallel --env multiline -S server multiline ::: work
449
450       functions
451                 myfunc() { echo functions $*; }
452                 env_parallel myfunc ::: work
453                 env_parallel -S server myfunc ::: work
454                 env_parallel --env myfunc myfunc ::: work
455                 env_parallel --env myfunc -S server myfunc ::: work
456
457       variables
458                 myvar=variables
459                 env_parallel echo '$myvar' ::: work
460                 env_parallel -S server echo '$myvar' ::: work
461                 env_parallel --env myvar echo '$myvar' ::: work
462                 env_parallel --env myvar -S server echo '$myvar' ::: work
463
464       arrays
465                 myarray=(arrays work, too)
466                 env_parallel -k echo '${myarray[{}]}' ::: 0 1 2
467                 env_parallel -k -S server echo '${myarray[{}]}' ::: 0 1 2
468                 env_parallel -k --env myarray echo '${myarray[{}]}' ::: 0 1 2
469                 env_parallel -k --env myarray -S server \
470                   echo '${myarray[{}]}' ::: 0 1 2
471
472   pdksh
473       Installation
474
475       Put this in $HOME/.profile:
476
477         source env_parallel.pdksh
478
479       E.g. by doing:
480
481         echo 'source env_parallel.pdksh' >> $HOME/.profile
482
483       Supported use
484
485       --env is supported to export only the variable, alias, function, or
486       array with the given name. Multiple --envs can be given.
487
488       --session is supported.
489
490       aliases
491                 alias myecho="echo aliases";
492                 env_parallel myecho ::: work;
493                 env_parallel -S server myecho ::: work;
494                 env_parallel --env myecho myecho ::: work;
495                 env_parallel --env myecho -S server myecho ::: work
496
497       functions
498                 myfunc() { echo functions $*; };
499                 env_parallel myfunc ::: work;
500                 env_parallel -S server myfunc ::: work;
501                 env_parallel --env myfunc myfunc ::: work;
502                 env_parallel --env myfunc -S server myfunc ::: work
503
504       variables
505                 myvar=variables;
506                 env_parallel echo "\$myvar" ::: work;
507                 env_parallel -S server echo "\$myvar" ::: work;
508                 env_parallel --env myvar echo "\$myvar" ::: work;
509                 env_parallel --env myvar -S server echo "\$myvar" ::: work
510
511       arrays
512                 myarray=(arrays work, too);
513                 env_parallel -k echo "\${myarray[{}]}" ::: 0 1 2;
514                 env_parallel -k -S server echo "\${myarray[{}]}" ::: 0 1 2;
515                 env_parallel -k --env myarray echo "\${myarray[{}]}" ::: 0 1 2;
516                 env_parallel -k --env myarray -S server \
517                   echo "\${myarray[{}]}" ::: 0 1 2
518
519   sh
520       Installation
521
522       Put this in $HOME/.profile:
523
524         . env_parallel.sh
525
526       E.g. by doing:
527
528         echo '. env_parallel.sh' >> $HOME/.profile
529
530       Supported use
531
532       --env is supported to export only the variable, or alias with the given
533       name. Multiple --envs can be given.
534
535       --session is supported.
536
537       aliases
538                 sh does not support aliases.
539
540       functions
541                 myfunc() { echo functions $*; }
542                 env_parallel myfunc ::: work
543                 env_parallel -S server myfunc ::: work
544                 env_parallel --env myfunc myfunc ::: work
545                 env_parallel --env myfunc -S server myfunc ::: work
546
547       variables
548                 myvar=variables
549                 env_parallel echo '$myvar' ::: work
550                 env_parallel -S server echo '$myvar' ::: work
551                 env_parallel --env myvar echo '$myvar' ::: work
552                 env_parallel --env myvar -S server echo '$myvar' ::: work
553
554       arrays
555                 sh does not support arrays.
556
557   tcsh
558       env_parallel for tcsh breaks $PARALLEL, so do not use $PARALLEL.
559
560       Installation
561
562       Put this in $HOME/.tcshrc:
563
564         source `which env_parallel.tcsh`
565
566       E.g. by doing:
567
568         echo 'source `which env_parallel.tcsh`' >> $HOME/.tcshrc
569
570       Supported use
571
572       --env is supported to export only the variable, alias, or array with
573       the given name. Multiple --envs can be given.
574
575       aliases
576                 alias myecho 'echo aliases'
577                 env_parallel myecho ::: work
578                 env_parallel -S server myecho ::: work
579                 env_parallel --env myecho myecho ::: work
580                 env_parallel --env myecho -S server myecho ::: work
581
582       functions
583               Not supported by tcsh.
584
585       variables
586                 set myvar=variables
587                 env_parallel echo '$myvar' ::: work
588                 env_parallel -S server echo '$myvar' ::: work
589                 env_parallel --env myvar echo '$myvar' ::: work
590                 env_parallel --env myvar -S server echo '$myvar' ::: work
591
592       arrays with no special chars
593                 set myarray=(arrays work, too)
594                 env_parallel -k echo \$'{myarray[{}]}' ::: 1 2 3
595                 env_parallel -k -S server echo \$'{myarray[{}]}' ::: 1 2 3
596                 env_parallel -k --env myarray echo \$'{myarray[{}]}' ::: 1 2 3
597                 env_parallel -k --env myarray -S server \
598                   echo \$'{myarray[{}]}' ::: 1 2 3
599
600   Zsh
601       Installation
602
603       Put this in $HOME/.zshrc:
604
605         . env_parallel.zsh
606
607       E.g. by doing:
608
609         echo '. env_parallel.zsh' >> $HOME/.zshenv
610
611       Supported use
612
613       --env is supported to export only the variable, alias, function, or
614       array with the given name. Multiple --envs can be given.
615
616       --session is supported.
617
618       aliases
619                 alias myecho='echo aliases'
620                 env_parallel myecho ::: work
621                 env_parallel -S server myecho ::: work
622                 env_parallel --env myecho myecho ::: work
623                 env_parallel --env myecho -S server myecho ::: work
624
625                 alias multiline='echo multiline
626                   echo aliases'
627                 env_parallel multiline ::: work
628                 env_parallel -S server multiline ::: work
629                 env_parallel --env multiline multiline ::: work
630                 env_parallel --env multiline -S server multiline ::: work
631
632       functions
633                 myfunc() { echo functions $*; }
634                 env_parallel myfunc ::: work
635                 env_parallel -S server myfunc ::: work
636                 env_parallel --env myfunc myfunc ::: work
637                 env_parallel --env myfunc -S server myfunc ::: work
638
639       variables
640                 myvar=variables
641                 env_parallel echo '$myvar' ::: work
642                 env_parallel -S server echo '$myvar' ::: work
643                 env_parallel --env myvar echo '$myvar' ::: work
644                 env_parallel --env myvar -S server echo '$myvar' ::: work
645
646       arrays
647                 myarray=(arrays work, too)
648                 env_parallel -k echo '${myarray[{}]}' ::: 1 2 3
649                 env_parallel -k -S server echo '${myarray[{}]}' ::: 1 2 3
650                 env_parallel -k --env myarray echo '${myarray[{}]}' ::: 1 2 3
651                 env_parallel -k --env myarray -S server \
652                   echo '${myarray[{}]}' ::: 1 2 3
653

EXIT STATUS

655       Same as GNU parallel.
656

AUTHOR

658       When using GNU env_parallel for a publication please cite:
659
660       O. Tange (2018): GNU Parallel 2018, March 2018, ISBN 9781387509881,
661       DOI: 10.5281/zenodo.1146014.
662
663       This helps funding further development; and it won't cost you a cent.
664       If you pay 10000 EUR you should feel free to use GNU Parallel without
665       citing.
666
667       Copyright (C) 2007-10-18 Ole Tange, http://ole.tange.dk
668
669       Copyright (C) 2008-2010 Ole Tange, http://ole.tange.dk
670
671       Copyright (C) 2010-2023 Ole Tange, http://ole.tange.dk and Free
672       Software Foundation, Inc.
673

LICENSE

675       This program is free software; you can redistribute it and/or modify it
676       under the terms of the GNU General Public License as published by the
677       Free Software Foundation; either version 3 of the License, or at your
678       option any later version.
679
680       This program is distributed in the hope that it will be useful, but
681       WITHOUT ANY WARRANTY; without even the implied warranty of
682       MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
683       General Public License for more details.
684
685       You should have received a copy of the GNU General Public License along
686       with this program.  If not, see <http://www.gnu.org/licenses/>.
687
688   Documentation license I
689       Permission is granted to copy, distribute and/or modify this
690       documentation under the terms of the GNU Free Documentation License,
691       Version 1.3 or any later version published by the Free Software
692       Foundation; with no Invariant Sections, with no Front-Cover Texts, and
693       with no Back-Cover Texts.  A copy of the license is included in the
694       file LICENSES/GFDL-1.3-or-later.txt.
695
696   Documentation license II
697       You are free:
698
699       to Share to copy, distribute and transmit the work
700
701       to Remix to adapt the work
702
703       Under the following conditions:
704
705       Attribution
706                You must attribute the work in the manner specified by the
707                author or licensor (but not in any way that suggests that they
708                endorse you or your use of the work).
709
710       Share Alike
711                If you alter, transform, or build upon this work, you may
712                distribute the resulting work only under the same, similar or
713                a compatible license.
714
715       With the understanding that:
716
717       Waiver   Any of the above conditions can be waived if you get
718                permission from the copyright holder.
719
720       Public Domain
721                Where the work or any of its elements is in the public domain
722                under applicable law, that status is in no way affected by the
723                license.
724
725       Other Rights
726                In no way are any of the following rights affected by the
727                license:
728
729                • Your fair dealing or fair use rights, or other applicable
730                  copyright exceptions and limitations;
731
732                • The author's moral rights;
733
734                • Rights other persons may have either in the work itself or
735                  in how the work is used, such as publicity or privacy
736                  rights.
737
738       Notice   For any reuse or distribution, you must make clear to others
739                the license terms of this work.
740
741       A copy of the full license is included in the file as
742       LICENCES/CC-BY-SA-4.0.txt
743

DEPENDENCIES

745       env_parallel uses GNU parallel.
746