1NQ(1)                     BSD General Commands Manual                    NQ(1)
2

NAME

4     nq — job queue utility
5

SYNOPSIS

7     nq [-c] [-q] command line ...
8     nq -t job id ...
9     nq -w job id ...
10

DESCRIPTION

12     The nq utility provides a very lightweight queuing system without requir‐
13     ing setup, maintenance, supervision or any long-running processes.
14
15     Job order is enforced by a timestamp nq gets immediately when started.
16     Synchronization happens on file-system level.  Timer resolution is mil‐
17     liseconds.  No sub-second file system time stamps are required.  Polling
18     is not used.  Exclusive execution is maintained strictly.
19
20     You enqueue(!) new jobs into the queue by running
21
22           nq command line ...
23
24     The job id (a file name relative to NQDIR, which defaults to the current
25     directory) is output (unless suppressed using -q) and nq detaches from
26     the terminal immediately, running the job in the background.  Standard
27     output and standard error are redirected into the job id file.  fq(1) can
28     be used to conveniently watch the log files.
29
30     The options are as follows:
31
32     -c      Clean up job id file when process exited with status 0.
33
34     -q      Suppress output of the job id after spawning new job.
35
36     -t      Enter test mode: exit with status 0 when all of the listed job
37             ids are already done, else with status 1.
38
39     -w      Enter waiting mode: wait in the foreground until all listed job
40             ids are done.
41

ENVIRONMENT

43     NQDIR      Directory where lock files/job output resides.  Each NQDIR can
44                be considered a separate queue.  The current working directory
45                is used when NQDIR is unset.  NQDIR is created if needed.
46
47     NQDONEDIR  When set, specifies a directory (must be on the same file
48                system as NQDIR) where lock files/job output is moved to after
49                successful execution of the job.
50
51                Ignored when -c is used.
52
53     NQJOBID    The job id of the currently running job, exposed to the job
54                itself.
55

FILES

57     nq owns all files in NQDIR (respectively .) which start with “,” or “.,”.
58     These files are created according to the following scheme:
59
60           ,hexadecimal-time-stamp.pid
61

EXIT STATUS

63     The nq utility exits 0 on success, and >0 if an error occurs; unless test
64     mode is used, in which case exit status 1 means there is a job running.
65
66     On fatal errors, exit codes 111 and 222 are used.
67

EXAMPLES

69     Build make(1) targets clean, depends, all, without occupying the termi‐
70     nal:
71
72           % nq make clean
73           % nq make depends
74           % nq make all
75           % fq
76           ... look at output, can interrupt with C-c any time
77           without stopping the build ...
78
79     Simple download queue, accessible from multiple terminals:
80
81           % alias qget='NQDIR=/tmp/downloads nq wget'
82           % alias qwait='NQDIR=/tmp/downloads fq -q'
83           window1% qget http://mymirror/big1.iso
84           window2% qget http://mymirror/big2.iso
85           window3% qget http://mymirror/big3.iso
86           % qwait
87           ... wait for all downloads to finish ...
88
89     As nohup(1) replacement (The benchmark will run in background, every run
90     gets a different output file, and the command line you ran is logged
91     too.):
92
93           % ssh remote
94           remote% nq ./run-benchmark
95           ,14f6f3034f8.17035
96           remote% ^D
97           % ssh remote
98           remote% fq
99           ... see output, fq exits when job finished ...
100

TRICKS

102     The "file extension" of the log file is actually the PID of the job.  nq
103     runs all jobs in a separate process group, so you can kill an entire job
104     process tree at once using kill(1) with a negative PID.  Before the job
105     is started, it is the PID of nq, so you can cancel a queued job by
106     killing it as well.
107
108     Thanks to the initial exec line in the log files, you can resubmit a job
109     by executing it as a shell command file, i.e. running
110
111           sh job id
112
113     Creating nq wrappers setting NQDIR to provide different queues for dif‐
114     ferent purposes is encouraged.
115

INTERNALS

117     Enforcing job order works like this:
118     -   every job has an output file locked using flock(2) and named accord‐
119         ing to FILES.
120     -   every job starts only after all earlier flocked files are unlocked.
121     -   the lock is released by the kernel after the job terminates.
122

ASSUMPTIONS

124     nq will only work correctly when:
125
126     -   NQDIR (respectively .) is writable.
127
128     -   flock(2) works correctly in NQDIR (respectively .).
129
130     -   gettimeofday(2) behaves monotonic (using CLOCK_MONOTONIC would create
131         confusing file names after reboot).
132
133     -   No other programs put files matching ,* into NQDIR (respectively .).
134

SEE ALSO

136     fq(1), tq(1).
137
138     Alternatives to the nq system include batch(1), qsub(1), schedule(1),
139     srun(1), and ts(1).
140

AUTHORS

142     Leah Neukirchen <leah@vuxu.org>
143

CAVEATS

145     All reliable queue status information is in main memory only, which makes
146     restarting a job queue after a reboot difficult.
147

LICENSE

149     nq is in the public domain.
150
151     To the extent possible under law, the creator of this work has waived all
152     copyright and related or neighboring rights to this work.
153
154     http://creativecommons.org/publicdomain/zero/1.0/
155
156BSD                             March 13, 2021                             BSD