1CONDOR_ANNEX(1) HTCondor Manual CONDOR_ANNEX(1)
2
6 condor_annex - HTCondor Manual
7 Add cloud resources to the pool.
9
12 condor_annex -help
13 condor_annex [-aws-region <region>] -setup [FROM IN‐
15 STANCE|[/full/path/to/access/key/file [/full/path/to/secret/key/file]]]
16 condor_annex [-aws-on-demand ] -annex-name <name of the annex> -count
18 <integer number of instances> [-aws-on-demand-* ] [common options ]
19 condor_annex [-aws-spot-fleet ] -annex-name <name of the annex> -slots
21 <integer weight> [-aws-spot-fleet-* ] [common options ]
22 condor_annex -annex-name <name of the annex> -duration hours
24 condor_annex [-annex-name <name of the annex>] -status [-classad ]
26 condor_annex -check-setup
28 condor_annex <condor_annex options> status <condor_status options>
30
32 condor_annex adds clouds resources to the pool. ("The pool" is deter‐
33 mined in the usual manner for HTCondor daemons and tools.) This version
34 supports only Amazon Web Services ('AWS'). To add "on-demand" in‐
35 stances, use the third form listed above; to add "spot" instances, use
36 the fourth. For an explanation of terms, consult either the HTCondor
37 manual in the Cloud Computing chapter or the AWS documentation.
38 Using condor_annex with AWS requires a one-time setup procedure per‐
40 formed by invoking condor_annex with the -setup flag (the second form
41 listed above). You may check if this procedure has been performed with
42 the -check-setup flag (the seventh form listed above). If you use the
43 setup flag on an instance whose role gives it sufficient privileges,
44 you may, instead of specifying your API keys, pass FROM INSTANCE to
45 -setup to ask condor_annex to use the instance's role credentials.
46 To reset the lease on an existing annex, invoke condor_annex with only
48 the -annex-name option and -duration flag (the fifth form listed
49 above).
50 To determine which of the instances previously requested for a particu‐
52 lar annex are not currently in the pool, invoke condor_annex with the
53 -status flag and the -annex-name option (the sixth form listed above).
54 The output of this command is intended to be human-readable; specifying
55 the -classad flag will produce the same information in ClassAd format.
56 If you omit -annex-name, information for all annexes will be returned.
57 Starting in 8.7.3, you may instead invoke condor_annex with status as a
59 command argument (the eighth form listed above). This will cause con‐
60 dor_annex to use condor_status to present annex instance data. Argu‐
61 ments and options on the command line after status will be passed un‐
62 modified to condor_status, but not all arguments and options will be‐
63 have as expected. (See below.) condor_annex will construct an ad for
64 each annex instance and pass that information to condor_status; con‐
65 dor_status will (unless you specify otherwise using its command line)
66 query the collector for more information about the instances. Informa‐
67 tion from the collector will be presented as usual; instances which did
68 not have ads in the collector will be presented last, in their own ta‐
69 ble. These instances can not be presented in the usual way because the
70 annex instance ads generated by condor_annex do not (and can not) have
71 the same information in them as ads generated by a condor_startd run‐
72 ning in the instance. See the condor_status manual page for details
73 about the "merge" mode of condor_status used by this command argument.
74 Note that both condor_annex and condor_status have -annex-name options;
75 if you're interested in a particular annex, put this flag on the com‐
76 mand line before the status command argument to avoid confusing re‐
77 sults.
78 Common options are listed first, followed by options specific to AWS,
80 followed by options specific to AWS' on-demand instances, followed by
81 options specific to AWS' spot instances, followed by options intended
82 for use by experts.
83
85 -help Print a usage reminder.
86 -setup [/full/path/to/access/key/file/full/path/to/secret/key/file]
88 Do the first-time setup.
89 -duration hours
91 Set the maximum lease duration in decimal hours. After this
92 amount of time, all instances will terminated, regardless of
93 their idleness. Defaults to 50 minutes.
94 -idle hours
96 Set the maximum idle duration in decimal hours. An instance
97 idle for longer than this duration will terminate itself. De‐
98 faults to 15 minutes.
99 -yes Start the annex automatically without a yes/no confirmation
101 prompt.
102 -tag name value
104 Add a tag named name with value value to each instance in the
105 requested annex. Only works at annex creation. This option
106 may be specified more than once.
107 -config-dir /full/path/to/directory
109 Copy the contents of /full/path/to/directory to each in‐
110 stance's configuration directory.
111 -owner owner[, owner]*
113 Configure the annex so that only owner may start jobs there.
114 By default, configure the annex so that only the user running
115 condor_annex may start jobs there.
116 -no-owner
118 Configure the annex so that anyone in the pool may use the
119 annex.
120 -aws-region region
122 Specify the region in which to create the annex.
123 -aws-user-data user-data
125 Set the instance user data to user-data.
126 -aws-user-data-file /full/path/to/file
128 Set the instance user data to the contents of the file
129 /full/path/to/file.
130 -aws-default-user-data user-data
132 Set the instance user data to user-data, if it's not already
133 set. Only applies to spot fleet requests.
134 -aws-default-user-data-file /full/path/to/file
136 Set the instance user data to the contents of the file
137 /full/path/to/file, if it's not already set. Only applies to
138 spot fleet requests.
139 -aws-on-demand-instance-type instance-type
141 This annex will requests instances of type instance-type. The
142 default for v8.7.1 is 'm4.large'.
143 -aws-on-demand-ami-id ami-id
145 This annex will start instances of the AMI ami-id. The de‐
146 fault for v8.7.1 is 'ami-35b13223', a GPU-compatible Amazon
147 Linux image with HTCondor pre-installed.
148 -aws-on-demand-security-group-ids group-id[,group-id]
150 This annex will start instances with the listed security
151 group IDs. The default is the security group created by
152 -setup.
153 -aws-on-demand-key-name key-name
155 This annex will start instances with the key pair named
156 key-name. The default is the key pair created by -setup.
157 -aws-spot-fleet-config-file /full/path/to/file
159 Use the JSON blob in /full/path/to/file for the spot fleet
160 request.
161 -aws-access-key-file /full/path/to/access-key-file
163 Experts only.
164 -aws-secret-key-file /full/path/to/secret-key-file
166 Experts only.
167 -aws-ec2-url https://ec2.<region>.amazonaws.com
169 Experts only.
170 -aws-events-url https://events.<region>.amazonaws.com
172 Experts only.
173 -aws-lambda-url https://lambda.<region>.amazonaws.com
175 Experts only.
176 -aws-s3-url https://s3.<region>.amazonaws.com
178 Experts only.
179 -aws-spot-fleet-lease-function-arn sfr-lease-function-arn
181 Developers only.
182 -aws-on-demand-lease-function-arn odi-lease-function-arn
184 Developers only.
185 -aws-on-demand-instance-profile-arn instance-profile-arn
187 Developers only.
188
190 Currently, only AWS is supported. The AMI configured by setup runs HT‐
191 Condor v8.6.10 on Amazon Linux 2016.09, and the default instance type
192 is "m4.large". The default AMI has the appropriate drivers for AWS' GPU
193 instance types.
194
196 To start an on-demand annex named 'MyFirstAnnex' with one core, using
197 the default AMI and instance type, run
198 $ condor_annex -count 1 -annex-name MyFirstAnnex
200 You will be asked to confirm that the defaults are what you want.
202 As of 2017-04-17, the following example will cost a minimum of $90.
204 To start an on-demand annex with 100 GPUs that job owners 'big' and
206 'little' may use (be sure to include yourself!), run
207 $ condor_annex -count 100 -annex-name MySecondAnnex \
209 -aws-on-demand-instance-type p2.xlarge -owner "big, little"
210
212 condor_annex will exit with a status value of 0 (zero) on success.
213
215 HTCondor Team
216
218 1990-2023, Center for High Throughput Computing, Computer Sciences De‐
219 partment, University of Wisconsin-Madison, Madison, WI, US. Licensed
220 under the Apache License, Version 2.0.
221 Oct 02, 2023 CONDOR_ANNEX(1)