1HTCONDOR(1) HTCondor Manual HTCONDOR(1)
2
6 htcondor - HTCondor Manual
7 Manage HTCondor jobs, job sets, dags, event logs, and resources
9
12 htcondor [ -h | --help ] [ -v | -q ]
13 htcondor job submit [--resource resource-type] [--runtime time-seconds] [--email email-address] submit_file
14 htcondor job status [--resource resource-type] [--skip-history] job_id
15 htcondor job resources [--resource resource-type] [--skip-history] job_id
16 htcondor jobset submit description-file
18 htcondor jobset list [--allusers]
19 htcondor jobset status job-set-name [--owner user-name] [--nobatch] [--skip-history]
20 htcondor jobset remove job-set-name [--owner user-name]
21 htcondor dag submit dag-file
23 htcondor dag status dagman-job-id
24 htcondor eventlog read [ -csv | -json] [ --groupby attribute ] eventlog
26 htcondor eventlog follow [ -csv | -json] [ --groupby attribute ] eventlog
27
30 htcondor is a tool for managing HTCondor jobs, job sets, resources,
31 event logs, and DAGs. It can replace condor_submit, condor_submit_dag,
32 condor_q, condor_status, and condor_userlog, as well as all-new func‐
33 tionality and features. The user interface is more consistent than its
34 predecessor tools.
35 The first argument of the htcondor command (ignoring any global op‐
37 tions) is the noun representing an object in the HTCondor system to be
38 operated on. The nouns include an individual job, jobset, eventlog, or
39 a dag. Each noun is then followed by a noun-specific verb that de‐
40 scribe the operation on that noun.
41 One of the following optional global option may appear before the noun:
43
45 htcondor -h, htcondor --help
46 Display the help message. Can also be specified after any
47 verb to display the options available for each verb.
48 htcondor -q ...
50 Reduce verbosity of log messages.
51 htcondor -v ...
53 Increase verbosity of log messages.
54 A noun-specific verb appears after each noun; the verbs are sorted by
56 noun in the list, which includes with their individual option flags.
57
59 htcondor job submit submit_file
60 Takes as an argument a submit file in the condor_submit job
61 submit description language, and places a new job in an Ac‐
62 cess Point
63 htcondor job submit options
65 htcondor job submit --resource resource_type submit_file
67 Resource type used to run this job. Currently sup‐
68 ports Slurm and EC2. Assumes the necessary setup
69 is complete and security tokens available.
70 htcondor job submit --runtime runtime_in_seconds sub‐
72 mit_file
73 Amount of time in seconds to allocate resources.
74 Used in conjunction with the --resource flag.
75 htcondor job submit --email address submit_file
77 Email address to receive notification messages.
78 Used in conjunction with the --resource flag.
79 htcondor job status
81 Takes as an argument a job id in the form of clusterid.pro‐
82 cid, and returns a human readable presentation of the status
83 of that job.
84 job status option
86 htcondor job status --skip-history job.id
87 Passed to the status verb to skip checking history if
88 job not found in the active job queue.
89 htcondor job resources
91 Takes as an argument a job id in the form of clusterid.pro‐
92 cid, and returns a human readable presentation the machine
93 resource used by this job.
94
96 htcondor jobset submit submit_file
97 Takes as an argument a submit file in the condor_submit job
98 submit description language, and places a new job set in an
99 Access Point
100 htcondor jobset list
102 Succinctly lists all the jobsets in the queue which are owned
103 by the current user.
104 htcondor jobset list options
105 htcondor jobset list --allusers
107 Shows jobs from all users, not just those owned
108 by the current user.
109 htcondor jobset status submit_file
111 Takes as an argument a job set name, and shows detailed in‐
112 formation about that job set.
113 htcondor jobset status options
115 htcondor jobset status --nobatch
117 Shows jobs in a more detailed view, one line per
118 job
119 htcondor jobset status --owner ownername
121 Shows jobs from the specified job owner.
122 htcondor jobset status --skiphistory
124 Shows detailed information only about active jobs
125 in the queue, and ignore historical jobs which
126 have left the queue. This runs much faster.
127 htcondor jobset remove job_name
129 Takes as an argument a job_name in the queue, and removes it
130 from the Access Point.
131 htcondor jobsets remove options
133 htcondor jobset remove --owner=owner_name Removes all
134 jobs owned by the given owner.
135
137 htcondor eventlog read logfile
138 Takes as an argument an event log to process. It may be the
139 per-job or per-jobset eventlog, which was specified by the
140 log = some_file in the submit description language. For a
141 dag, it may also be the nodes.log file that all dags gener‐
142 ate. Or, if the global event log is enabled by an adminis‐
143 trator with the EVENT_LOG configuration knob, it may be the
144 global event log, containing information about all jobs on
145 the Access point.
146 Given this file, htcondor eventlog read returns information
148 about all the contained jobs, and their status. It runs much
149 faster than condor_history, because these logs are more con‐
150 cise than the history files. Unlike condor_history, it will
151 also show information about jobs that have not yet left the
152 queue.
153 htcondor eventlog follow logfile
155 Takes as an argument an event log to process, as above, but
156 instead of processing that file to completion, it does the
157 equivalent of tail -f, and runs until interruption, emitting
158 information about jobs as it appears in the file.
159 Eventlog Options
161 --csv By default, htcondor eventlog read emits a ta‐
163 ble of information in human readable format.
164 With this option, the output is in a command
165 separated value format, suitable for injestion
166 by a spreadsheet or database.
167 --json Emits output in the json format. Only one of
169 -csv or -json should be given.
170 --group-by attributeName
172 With a job ad attribute name, instead of one line
173 per job, emit one line summarizing all jobs that
174 share the same value for the attribute name given.
175 In the OSG, the GLIDEIN_SITE attribute is injected
176 into all jobs, so one can quickly get a count of
177 all jobs running, idle and exitted per site by us‐
178 ing this option.
179
181 $ htcondor eventlog read logfile
182 Job Host Start Time Evict Time Evictions Wall Time Good Time CPU Usage
184 19989.0 slot1_1@speedy 5/18 12:34 5/18 12:54 0 0+00:20:00 0+00:20:00 0+00:00:00
185 19990.0 slot1_1@lumpy 5/22 18:51 5/22 18:51 1 0+00:02:00 0+00:00:00 0+00:00:43
186 20003.0 slot1_1@chtc 8/9 23:33 8/9 23:37 1 0+00:04:00 0+00:00:00 0+00:00:00
187 20004.0 slot1_1@wisc 8/9 23:38 8/9 23:58 0 0+00:20:00 0+00:20:00 0+00:00:00
188
190 htcondor will exit with a non-zero status value if it fails and zero
191 status if it succeeds.
192
194 HTCondor Team
195
197 1990-2023, Center for High Throughput Computing, Computer Sciences De‐
198 partment, University of Wisconsin-Madison, Madison, WI, US. Licensed
199 under the Apache License, Version 2.0.
200 Oct 02, 2023 HTCONDOR(1)