1CONDOR_ANNEX(1)                 HTCondor Manual                CONDOR_ANNEX(1)
2
3
4

NAME

6       condor_annex - HTCondor Manual
7
8       Add cloud resources to the pool.
9
10

SYNOPSIS

12       condor_annex -help
13
14       condor_annex     [-aws-region     <region>]     -setup     [FROM    IN‐
15       STANCE|[/full/path/to/access/key/file [/full/path/to/secret/key/file]]]
16
17       condor_annex [-aws-on-demand ] -annex-name <name of the  annex>  -count
18       <integer number of instances> [-aws-on-demand-* ] [common options ]
19
20       condor_annex  [-aws-spot-fleet ] -annex-name <name of the annex> -slots
21       <integer weight> [-aws-spot-fleet-* ] [common options ]
22
23       condor_annex -annex-name <name of the annex> -duration hours
24
25       condor_annex [-annex-name <name of the annex>] -status [-classad ]
26
27       condor_annex -check-setup
28
29       condor_annex <condor_annex options> status <condor_status options>
30

DESCRIPTION

32       condor_annex adds clouds resources to the pool. ("The pool"  is  deter‐
33       mined in the usual manner for HTCondor daemons and tools.) This version
34       supports only Amazon Web  Services  ('AWS').  To  add  "on-demand"  in‐
35       stances,  use the third form listed above; to add "spot" instances, use
36       the fourth. For an explanation of terms, consult  either  the  HTCondor
37       manual in the /cloud-computing/index chapter or the AWS documentation.
38
39       Using  condor_annex  with  AWS requires a one-time setup procedure per‐
40       formed by invoking condor_annex with the -setup flag (the  second  form
41       listed  above). You may check if this procedure has been performed with
42       the -check-setup flag (the seventh form listed above). If you  use  the
43       setup  flag  on  an instance whose role gives it sufficient privileges,
44       you may, instead of specifying your API keys,  pass  FROM  INSTANCE  to
45       -setup to ask condor_annex to use the instance's role credentials.
46
47       To  reset the lease on an existing annex, invoke condor_annex with only
48       the -annex-name option  and  -duration  flag  (the  fifth  form  listed
49       above).
50
51       To determine which of the instances previously requested for a particu‐
52       lar annex are not currently in the pool, invoke condor_annex  with  the
53       -status  flag and the -annex-name option (the sixth form listed above).
54       The output of this command is intended to be human-readable; specifying
55       the  -classad flag will produce the same information in ClassAd format.
56       If you omit -annex-name, information for all annexes will be returned.
57
58       Starting in 8.7.3, you may instead invoke condor_annex with status as a
59       command  argument  (the eighth form listed above). This will cause con‐
60       dor_annex to use condor_status to present annex  instance  data.  Argu‐
61       ments  and  options on the command line after status will be passed un‐
62       modified to condor_status, but not all arguments and options  will  be‐
63       have  as  expected. (See below.)  condor_annex will construct an ad for
64       each annex instance and pass that information  to  condor_status;  con‐
65       dor_status  will  (unless you specify otherwise using its command line)
66       query the collector for more information about the instances.  Informa‐
67       tion from the collector will be presented as usual; instances which did
68       not have ads in the collector will be presented last, in their own  ta‐
69       ble.  These instances can not be presented in the usual way because the
70       annex instance ads generated by condor_annex do not (and can not)  have
71       the  same  information in them as ads generated by a condor_startd run‐
72       ning in the instance. See the /man-pages/condor_status manual page  for
73       details  about  the  "merge" mode of condor_status used by this command
74       argument. Note that  both  condor_annex  and  condor_status  have  -an‐
75       nex-name  options; if you're interested in a particular annex, put this
76       flag on the command line before the status command  argument  to  avoid
77       confusing results.
78
79       Common  options  are listed first, followed by options specific to AWS,
80       followed by options specific to AWS' on-demand instances,  followed  by
81       options  specific  to AWS' spot instances, followed by options intended
82       for use by experts.
83

OPTIONS

85          -help  Print a usage reminder.
86
87          -setup [/full/path/to/access/key/file/full/path/to/secret/key/file]
88                 Do the first-time setup.
89
90          -duration hours
91                 Set the maximum lease duration in decimal hours.  After  this
92                 amount  of time, all instances will terminated, regardless of
93                 their idleness. Defaults to 50 minutes.
94
95          -idle hours
96                 Set the maximum idle duration in decimal hours.  An  instance
97                 idle for longer than this duration will terminate itself. De‐
98                 faults to 15 minutes.
99
100          -config-dir /full/path/to/directory
101                 Copy the contents  of  /full/path/to/directory  to  each  in‐
102                 stance's configuration directory.
103
104          -owner owner[, owner]*
105                 Configure  the annex so that only owner may start jobs there.
106                 By default, configure the annex so that only the user running
107                 condor_annex may start jobs there.
108
109          -no-owner
110                 Configure  the  annex  so that anyone in the pool may use the
111                 annex.
112
113          -aws-region region
114                 Specify the region in which to create the annex.
115
116          -aws-user-data user-data
117                 Set the instance user data to user-data.
118
119          -aws-user-data-file /full/path/to/file
120                 Set the instance user  data  to  the  contents  of  the  file
121                 /full/path/to/file.
122
123          -aws-default-user-data user-data
124                 Set  the instance user data to user-data, if it's not already
125                 set.  Only applies to spot fleet requests.
126
127          -aws-default-user-data-file /full/path/to/file
128                 Set the instance user  data  to  the  contents  of  the  file
129                 /full/path/to/file,  if it's not already set. Only applies to
130                 spot fleet requests.
131
132          -aws-on-demand-instance-type instance-type
133                 This annex will requests instances of type instance-type. The
134                 default for v8.7.1 is 'm4.large'.
135
136          -aws-on-demand-ami-id ami-id
137                 This  annex  will  start instances of the AMI ami-id. The de‐
138                 fault for v8.7.1 is 'ami-35b13223', a  GPU-compatible  Amazon
139                 Linux image with HTCondor pre-installed.
140
141          -aws-on-demand-security-group-ids group-id[,group-id]
142                 This  annex  will  start  instances  with the listed security
143                 group IDs.  The default is  the  security  group  created  by
144                 -setup.
145
146          -aws-on-demand-key-name key-name
147                 This  annex  will  start  instances  with  the key pair named
148                 key-name.  The default is the key pair created by -setup.
149
150          -aws-spot-fleet-config-file /full/path/to/file
151                 Use the JSON blob in /full/path/to/file for  the  spot  fleet
152                 request.
153
154          -aws-access-key-file /full/path/to/access-key-file
155                 Experts only.
156
157          -aws-secret-key-file /full/path/to/secret-key-file
158                 Experts only.
159
160          -aws-ec2-url https://ec2.<region>.amazonaws.com
161                 Experts only.
162
163          -aws-events-url https://events.<region>.amazonaws.com
164                 Experts only.
165
166          -aws-lambda-url https://lambda.<region>.amazonaws.com
167                 Experts only.
168
169          -aws-s3-url https://s3.<region>.amazonaws.com
170                 Experts only.
171
172          -aws-spot-fleet-lease-function-arn sfr-lease-function-arn
173                 Developers only.
174
175          -aws-on-demand-lease-function-arn odi-lease-function-arn
176                 Developers only.
177
178          -aws-on-demand-instance-profile-arn instance-profile-arn
179                 Developers only.
180

GENERAL REMARKS

182       Currently,  only AWS is supported. The AMI configured by setup runs HT‐
183       Condor v8.6.10 on Amazon Linux 2016.09, and the default  instance  type
184       is "m4.large". The default AMI has the appropriate drivers for AWS' GPU
185       instance types.
186

EXAMPLES

188       To start an on-demand annex named 'MyFirstAnnex' with one  core,  using
189       the default AMI and instance type, run
190
191          condor_annex -count 1 -annex-name MyFirstAnnex
192
193       You will be asked to confirm that the defaults are what you want.
194
195       As of 2017-04-17, the following example will cost a minimum of $90.
196
197       To  start  an  on-demand  annex with 100 GPUs that job owners 'big' and
198       'little' may use (be sure to include yourself!), run
199
200          condor_annex -count 100 -annex-name MySecondAnnex \
201            -aws-on-demand-instance-type p2.xlarge -owner "big, little"
202

EXIT STATUS

204       condor_annex will exit with a status value of 0 (zero) on success.
205

AUTHOR

207       HTCondor Team
208
210       1990-2021, Center for High Throughput Computing, Computer Sciences  De‐
211       partment,  University  of  Wisconsin-Madison, Madison, WI, US. Licensed
212       under the Apache License, Version 2.0.
213
214
215
216
2178.8                              Jan 26, 2021                  CONDOR_ANNEX(1)
Impressum