1CONDOR_ANNEX(1) HTCondor Manual CONDOR_ANNEX(1)
2
3
4
6 condor_annex - HTCondor Manual
7
8 Add cloud resources to the pool.
9
10
12 condor_annex -help
13
14 condor_annex [-aws-region <region>] -setup [FROM IN‐
15 STANCE|[/full/path/to/access/key/file [/full/path/to/secret/key/file]]]
16
17 condor_annex [-aws-on-demand ] -annex-name <name of the annex> -count
18 <integer number of instances> [-aws-on-demand-* ] [common options ]
19
20 condor_annex [-aws-spot-fleet ] -annex-name <name of the annex> -slots
21 <integer weight> [-aws-spot-fleet-* ] [common options ]
22
23 condor_annex -annex-name <name of the annex> -duration hours
24
25 condor_annex [-annex-name <name of the annex>] -status [-classad ]
26
27 condor_annex -check-setup
28
29 condor_annex <condor_annex options> status <condor_status options>
30
32 condor_annex adds clouds resources to the pool. ("The pool" is deter‐
33 mined in the usual manner for HTCondor daemons and tools.) This version
34 supports only Amazon Web Services ('AWS'). To add "on-demand" in‐
35 stances, use the third form listed above; to add "spot" instances, use
36 the fourth. For an explanation of terms, consult either the HTCondor
37 manual in the /cloud-computing/index chapter or the AWS documentation.
38
39 Using condor_annex with AWS requires a one-time setup procedure per‐
40 formed by invoking condor_annex with the -setup flag (the second form
41 listed above). You may check if this procedure has been performed with
42 the -check-setup flag (the seventh form listed above). If you use the
43 setup flag on an instance whose role gives it sufficient privileges,
44 you may, instead of specifying your API keys, pass FROM INSTANCE to
45 -setup to ask condor_annex to use the instance's role credentials.
46
47 To reset the lease on an existing annex, invoke condor_annex with only
48 the -annex-name option and -duration flag (the fifth form listed
49 above).
50
51 To determine which of the instances previously requested for a particu‐
52 lar annex are not currently in the pool, invoke condor_annex with the
53 -status flag and the -annex-name option (the sixth form listed above).
54 The output of this command is intended to be human-readable; specifying
55 the -classad flag will produce the same information in ClassAd format.
56 If you omit -annex-name, information for all annexes will be returned.
57
58 Starting in 8.7.3, you may instead invoke condor_annex with status as a
59 command argument (the eighth form listed above). This will cause con‐
60 dor_annex to use condor_status to present annex instance data. Argu‐
61 ments and options on the command line after status will be passed un‐
62 modified to condor_status, but not all arguments and options will be‐
63 have as expected. (See below.) condor_annex will construct an ad for
64 each annex instance and pass that information to condor_status; con‐
65 dor_status will (unless you specify otherwise using its command line)
66 query the collector for more information about the instances. Informa‐
67 tion from the collector will be presented as usual; instances which did
68 not have ads in the collector will be presented last, in their own ta‐
69 ble. These instances can not be presented in the usual way because the
70 annex instance ads generated by condor_annex do not (and can not) have
71 the same information in them as ads generated by a condor_startd run‐
72 ning in the instance. See the /man-pages/condor_status manual page for
73 details about the "merge" mode of condor_status used by this command
74 argument. Note that both condor_annex and condor_status have -an‐
75 nex-name options; if you're interested in a particular annex, put this
76 flag on the command line before the status command argument to avoid
77 confusing results.
78
79 Common options are listed first, followed by options specific to AWS,
80 followed by options specific to AWS' on-demand instances, followed by
81 options specific to AWS' spot instances, followed by options intended
82 for use by experts.
83
85 -help Print a usage reminder.
86
87 -setup [/full/path/to/access/key/file/full/path/to/secret/key/file]
88 Do the first-time setup.
89
90 -duration hours
91 Set the maximum lease duration in decimal hours. After this
92 amount of time, all instances will terminated, regardless of
93 their idleness. Defaults to 50 minutes.
94
95 -idle hours
96 Set the maximum idle duration in decimal hours. An instance
97 idle for longer than this duration will terminate itself. De‐
98 faults to 15 minutes.
99
100 -config-dir /full/path/to/directory
101 Copy the contents of /full/path/to/directory to each in‐
102 stance's configuration directory.
103
104 -owner owner[, owner]*
105 Configure the annex so that only owner may start jobs there.
106 By default, configure the annex so that only the user running
107 condor_annex may start jobs there.
108
109 -no-owner
110 Configure the annex so that anyone in the pool may use the
111 annex.
112
113 -aws-region region
114 Specify the region in which to create the annex.
115
116 -aws-user-data user-data
117 Set the instance user data to user-data.
118
119 -aws-user-data-file /full/path/to/file
120 Set the instance user data to the contents of the file
121 /full/path/to/file.
122
123 -aws-default-user-data user-data
124 Set the instance user data to user-data, if it's not already
125 set. Only applies to spot fleet requests.
126
127 -aws-default-user-data-file /full/path/to/file
128 Set the instance user data to the contents of the file
129 /full/path/to/file, if it's not already set. Only applies to
130 spot fleet requests.
131
132 -aws-on-demand-instance-type instance-type
133 This annex will requests instances of type instance-type. The
134 default for v8.7.1 is 'm4.large'.
135
136 -aws-on-demand-ami-id ami-id
137 This annex will start instances of the AMI ami-id. The de‐
138 fault for v8.7.1 is 'ami-35b13223', a GPU-compatible Amazon
139 Linux image with HTCondor pre-installed.
140
141 -aws-on-demand-security-group-ids group-id[,group-id]
142 This annex will start instances with the listed security
143 group IDs. The default is the security group created by
144 -setup.
145
146 -aws-on-demand-key-name key-name
147 This annex will start instances with the key pair named
148 key-name. The default is the key pair created by -setup.
149
150 -aws-spot-fleet-config-file /full/path/to/file
151 Use the JSON blob in /full/path/to/file for the spot fleet
152 request.
153
154 -aws-access-key-file /full/path/to/access-key-file
155 Experts only.
156
157 -aws-secret-key-file /full/path/to/secret-key-file
158 Experts only.
159
160 -aws-ec2-url https://ec2.<region>.amazonaws.com
161 Experts only.
162
163 -aws-events-url https://events.<region>.amazonaws.com
164 Experts only.
165
166 -aws-lambda-url https://lambda.<region>.amazonaws.com
167 Experts only.
168
169 -aws-s3-url https://s3.<region>.amazonaws.com
170 Experts only.
171
172 -aws-spot-fleet-lease-function-arn sfr-lease-function-arn
173 Developers only.
174
175 -aws-on-demand-lease-function-arn odi-lease-function-arn
176 Developers only.
177
178 -aws-on-demand-instance-profile-arn instance-profile-arn
179 Developers only.
180
182 Currently, only AWS is supported. The AMI configured by setup runs HT‐
183 Condor v8.6.10 on Amazon Linux 2016.09, and the default instance type
184 is "m4.large". The default AMI has the appropriate drivers for AWS' GPU
185 instance types.
186
188 To start an on-demand annex named 'MyFirstAnnex' with one core, using
189 the default AMI and instance type, run
190
191 condor_annex -count 1 -annex-name MyFirstAnnex
192
193 You will be asked to confirm that the defaults are what you want.
194
195 As of 2017-04-17, the following example will cost a minimum of $90.
196
197 To start an on-demand annex with 100 GPUs that job owners 'big' and
198 'little' may use (be sure to include yourself!), run
199
200 condor_annex -count 100 -annex-name MySecondAnnex \
201 -aws-on-demand-instance-type p2.xlarge -owner "big, little"
202
204 condor_annex will exit with a status value of 0 (zero) on success.
205
207 HTCondor Team
208
210 1990-2021, Center for High Throughput Computing, Computer Sciences De‐
211 partment, University of Wisconsin-Madison, Madison, WI, US. Licensed
212 under the Apache License, Version 2.0.
213
214
215
216
2178.8 Aug 23, 2021 CONDOR_ANNEX(1)