1CONDOR_ANNEX(1) HTCondor Manual CONDOR_ANNEX(1)
2
6 condor_annex - HTCondor Manual
7 Add cloud resources to the pool.
9
12 condor_annex -help
13 condor_annex [-aws-region <region>] -setup [FROM
15 INSTANCE|[/full/path/to/access/key/file
16 [/full/path/to/secret/key/file]]]
17 condor_annex [-aws-on-demand ] -annex-name <name of the annex> -count
19 <integer number of instances> [-aws-on-demand-* ] [common options ]
20 condor_annex [-aws-spot-fleet ] -annex-name <name of the annex> -slots
22 <integer weight> [-aws-spot-fleet-* ] [common options ]
23 condor_annex -annex-name <name of the annex> -duration hours
25 condor_annex [-annex-name <name of the annex>] -status [-classad ]
27 condor_annex -check-setup
29 condor_annex <condor_annex options> status <condor_status options>
31
33 condor_annex adds clouds resources to the pool. ("The pool" is deter‐
34 mined in the usual manner for HTCondor daemons and tools.) This version
35 supports only Amazon Web Services ('AWS'). To add "on-demand"
36 instances, use the third form listed above; to add "spot" instances,
37 use the fourth. For an explanation of terms, consult either the HTCon‐
38 dor manual in the /cloud-computing/index chapter or the AWS documenta‐
39 tion.
40 Using condor_annex with AWS requires a one-time setup procedure per‐
42 formed by invoking condor_annex with the -setup flag (the second form
43 listed above). You may check if this procedure has been performed with
44 the -check-setup flag (the seventh form listed above). If you use the
45 setup flag on an instance whose role gives it sufficient privileges,
46 you may, instead of specifying your API keys, pass FROM INSTANCE to
47 -setup to ask condor_annex to use the instance's role credentials.
48 To reset the lease on an existing annex, invoke condor_annex with only
50 the -annex-name option and -duration flag (the fifth form listed
51 above).
52 To determine which of the instances previously requested for a particu‐
54 lar annex are not currently in the pool, invoke condor_annex with the
55 -status flag and the -annex-name option (the sixth form listed above).
56 The output of this command is intended to be human-readable; specifying
57 the -classad flag will produce the same information in ClassAd format.
58 If you omit -annex-name, information for all annexes will be returned.
59 Starting in 8.7.3, you may instead invoke condor_annex with status as a
61 command argument (the eighth form listed above). This will cause con‐
62 dor_annex to use condor_status to present annex instance data. Argu‐
63 ments and options on the command line after status will be passed
64 unmodified to condor_status, but not all arguments and options will
65 behave as expected. (See below.) condor_annex will construct an ad for
66 each annex instance and pass that information to condor_status; con‐
67 dor_status will (unless you specify otherwise using its command line)
68 query the collector for more information about the instances. Informa‐
69 tion from the collector will be presented as usual; instances which did
70 not have ads in the collector will be presented last, in their own ta‐
71 ble. These instances can not be presented in the usual way because the
72 annex instance ads generated by condor_annex do not (and can not) have
73 the same information in them as ads generated by a condor_startd run‐
74 ning in the instance. See the /man-pages/condor_status manual page for
75 details about the "merge" mode of condor_status used by this command
76 argument. Note that both condor_annex and condor_status have
77 -annex-name options; if you're interested in a particular annex, put
78 this flag on the command line before the status command argument to
79 avoid confusing results.
80 Common options are listed first, followed by options specific to AWS,
82 followed by options specific to AWS' on-demand instances, followed by
83 options specific to AWS' spot instances, followed by options intended
84 for use by experts.
85
87 -help Print a usage reminder.
88 -setup [/full/path/to/access/key/file/full/path/to/secret/key/file]
90 Do the first-time setup.
91 -duration hours
93 Set the maximum lease duration in decimal hours. After this
94 amount of time, all instances will terminated, regardless of
95 their idleness. Defaults to 50 minutes.
96 -idle hours
98 Set the maximum idle duration in decimal hours. An instance
99 idle for longer than this duration will terminate itself.
100 Defaults to 15 minutes.
101 -config-dir /full/path/to/directory
103 Copy the contents of /full/path/to/directory to each
104 instance's configuration directory.
105 -owner owner[, owner]*
107 Configure the annex so that only owner may start jobs there.
108 By default, configure the annex so that only the user running
109 condor_annex may start jobs there.
110 -no-owner
112 Configure the annex so that anyone in the pool may use the
113 annex.
114 -aws-region region
116 Specify the region in which to create the annex.
117 -aws-user-data user-data
119 Set the instance user data to user-data.
120 -aws-user-data-file /full/path/to/file
122 Set the instance user data to the contents of the file
123 /full/path/to/file.
124 -aws-default-user-data user-data
126 Set the instance user data to user-data, if it's not already
127 set. Only applies to spot fleet requests.
128 -aws-default-user-data-file /full/path/to/file
130 Set the instance user data to the contents of the file
131 /full/path/to/file, if it's not already set. Only applies to
132 spot fleet requests.
133 -aws-on-demand-instance-type instance-type
135 This annex will requests instances of type instance-type. The
136 default for v8.7.1 is 'm4.large'.
137 -aws-on-demand-ami-id ami-id
139 This annex will start instances of the AMI ami-id. The
140 default for v8.7.1 is 'ami-35b13223', a GPU-compatible Amazon
141 Linux image with HTCondor pre-installed.
142 -aws-on-demand-security-group-ids group-id[,group-id]
144 This annex will start instances with the listed security
145 group IDs. The default is the security group created by
146 -setup.
147 -aws-on-demand-key-name key-name
149 This annex will start instances with the key pair named
150 key-name. The default is the key pair created by -setup.
151 -aws-spot-fleet-config-file /full/path/to/file
153 Use the JSON blob in /full/path/to/file for the spot fleet
154 request.
155 -aws-access-key-file /full/path/to/access-key-file
157 Experts only.
158 -aws-secret-key-file /full/path/to/secret-key-file
160 Experts only.
161 -aws-ec2-url https://ec2.<region>.amazonaws.com
163 Experts only.
164 -aws-events-url https://events.<region>.amazonaws.com
166 Experts only.
167 -aws-lambda-url https://lambda.<region>.amazonaws.com
169 Experts only.
170 -aws-s3-url https://s3.<region>.amazonaws.com
172 Experts only.
173 -aws-spot-fleet-lease-function-arn sfr-lease-function-arn
175 Developers only.
176 -aws-on-demand-lease-function-arn odi-lease-function-arn
178 Developers only.
179 -aws-on-demand-instance-profile-arn instance-profile-arn
181 Developers only.
182
184 Currently, only AWS is supported. The AMI configured by setup runs
185 HTCondor v8.6.10 on Amazon Linux 2016.09, and the default instance type
186 is "m4.large". The default AMI has the appropriate drivers for AWS' GPU
187 instance types.
188
190 To start an on-demand annex named 'MyFirstAnnex' with one core, using
191 the default AMI and instance type, run
192 condor_annex -count 1 -annex-name MyFirstAnnex
194 You will be asked to confirm that the defaults are what you want.
196 As of 2017-04-17, the following example will cost a minimum of $90.
198 To start an on-demand annex with 100 GPUs that job owners 'big' and
200 'little' may use (be sure to include yourself!), run
201 condor_annex -count 100 -annex-name MySecondAnnex \
203 -aws-on-demand-instance-type p2.xlarge -owner "big, little"
204
206 condor_annex will exit with a status value of 0 (zero) on success.
207
209 HTCondor Team
210
212 1990-2020, Center for High Throughput Computing, Computer Sciences
213 Department, University of Wisconsin-Madison, Madison, WI, US. Licensed
214 under the Apache License, Version 2.0.
2158.8 Aug 06, 2020 CONDOR_ANNEX(1)