1ARCSUB(1) NorduGrid Users Manual ARCSUB(1)
2
6 arcsub - ARC Submission
7
10 The arcsub command is used for submitting jobs to grid enabled comput‐
11 ing resources.
12
15 arcsub [options] [filename ...]
16
19 -c, --cluster=[-]name
20 explicitly select or reject a specific cluster
21 -g, --index=[-]name
23 explicitly select or reject an index server
24 -e, --jobdescrstring=String
26 jobdescription string describing the job to be submitted
27 -f, --jobdescrfile=filename
29 jobdescription file describing the job to be submitted
30 -j, --joblist=filename
32 the file storing information about active jobs (default
33 ~/.arc/jobs.xml)
34 -o, --jobids-to-file=filename
36 the IDs of the submitted jobs will be appended to this file
37 -D, --dryrun
39 submit jobs as dry run (no submission to batch system)
40 -x, --dumpdescription
42 do not submit - dump job description in the language accepted by
43 the target
44 -P, --listplugins
46 list the available plugins
47 -t, --timeout=seconds
49 timeout in seconds (default 20)
50 -z, --conffile=filename
52 configuration file (default ~/.arc/client.conf)
53 -d, --debug=debuglevel
55 FATAL, ERROR, WARNING, INFO, VERBOSE or DEBUG
56 -b, --broker=broker
58 selected broker: Random (default), FastestQueue or custom
59 -v, --version
61 print version information
62 -?, --help
64 print help
65
67 filename ...
68 job description files describing the jobs to be submitted
69
71 arcsub is the key command when submitting jobs to grid enabled comput‐
72 ing resources with the ARC client. As default arcsub is able to submit
73 jobs to A-REX and CREAM computing elements (CEs), and as always for
74 successful submission you need to be authenticated at the computing
75 element. Since arcsub is build on a modular library, modules can be
76 installed which enables submission to other targets, e.g. the classic
77 ARC CE Grid-Manager.
78 Job submission can be accomplished by invoking the arcsub command with
80 the --cluster option set to the target CE, and the job description file
81 given as argument. The --cluster option has the following format:
82 [<middleware_flavour>:]<URL>
83 where middleware_flavour can specify the CE type, e.g. for A-REX 'ARC1'
84 should be used, 'CREAM' for CREAM and 'ARC0' for Grid-Manager, and
85 <URL> specifies the information endpoint of the CE, which may include
86 the protocol (e.g. http, ldap, etc.). If the middleware_flavour is not
87 specified, then the client will try all supported flavours to find the
88 correct one. Multiple CEs can be targeted for submission by specifying
89 the --cluster option multiple times, in which case the CEs will be
90 ranked by the resource broker. If no --cluster or --index (see below)
91 options have been specified, then the defaults, which is read from the
92 configuration file (see the files section), will be used. That also
93 makes it possible to define own defaults, so it makes it superfluous to
94 specify the CE usually used for submission. In the same manner aliases,
95 defined in the configuration file, can be utilized, and should be used
96 as argument to the --cluster or --index options.
97 The format of 'ARC0' cluster URLs:
99 [ARC0:][ldap://]<hostname>[:2135/nordugrid-cluster-name=<hostname>,Mds-
100 Vo-name=local,o=grid]
101 Only the hostname part has to be specified, the rest of the URL is
102 automatically generated.
103 The format of 'ARC1' cluster URLs:
105 [ARC1:][https://]<hostname>:<port>/<path>
106 The format of 'CREAM' cluster URLs:
108 [CREAM:][ldap://]<hostname>[:2170/o=Grid??base?(objectClass=*)]
109 Only the hostname part has to be specified, the rest of the URL is
110 automatically generated.
111 Job descriptions can also be specified using the --jobdescrfile option
113 which expect the file name of the description as argument, or the
114 --jobdescrstring option which expect as argument a the job description
115 as a string, and both options can be specified multiple times and one
116 does not exclude the other. The default supported job description lan‐
117 guages are XRSL, JSDL and JDL.
118 If the job description is successfully submitted a job-ID is returned
120 and printed. This job-ID uniquely identifies the job while it is being
121 executed. On the other hand it is also possible that no CEs matches
122 the constraints defined in the description in which case no submission
123 will be done. Upon successful submission, the job-ID along with more
124 technical job information is stored in the job-list file (described
125 below). The stored information enables the job management commands of
126 the ARC client to manage jobs easily, and thus the job-ID need not to
127 be saved manually. By default the job-list file is stored in the .arc
128 directory in the home directory of the user, however another location
129 can be specified using the --joblist option taking the location of this
130 file as argument. If the --joblist option was used during submission,
131 it should also be specified in the consecutive commands when managing
132 the job.
133 Instead of directly targeting a CE for submission, using the --cluster
135 option, an information index server can be specified with the --index
136 option. Specifying one or multiple index servers instructs the arcsub
137 command to query the servers for registered CEs, the returned CEs will
138 then be matched against the job description and those matching will be
139 ranked by the chosen broker (see below) and submission will be tried in
140 order until successful or reaching the end. The format of the argument
141 to the --index option follows the one for the --cluster option
142 described above, however here middleware_flavour designates the type of
143 information index server, e.g. for ISIS 'ARC1' should be used, 'CREAM'
144 for BDII and 'ARC0' for GIIS. From the returned list of CEs it might
145 happen that a troublesome or undesirable CE is selected for submission,
146 in that case it possible to specify that cluster using the --cluster
147 option and prepending the argument it with a '-', which will disregard
148 that CE as a target for submission.
149 When multiple CEs are targeted for submission, the resource broker will
151 be used to rank the CEs. The broker used by default will rank the CEs
152 randomly, however a different broker can be chosen by using the --bro‐
153 ker option, which takes the name of the broker as argument. Two other
154 brokers are by default available. The FastestQueue and Benchmark bro‐
155 kers. The former ranks CEs according to the length of the job queue at
156 the CEs, ranking those with shortest queue first/highest. The latter
157 ranks CEs according to a specified benchmark, which should be specified
158 by appending the broker name with ':' and then the name of the bench‐
159 mark. If no option is given to the Benchmark broker then CEs will be
160 ranked according to the 'specint2000' benchmark. For examples see the
161 respective section.
162 Before submission, arcsub performs an intelligent modification of the
164 job description (adding or modifying attributes, even converting the
165 description language to fit the needs of the CE) ensuring that it is
166 valid. The modified job description can be printed by specifying the
167 --dumpdescription option. The format, i.e. job description language, of
168 the printed job description cannot be specified, and will be that which
169 will be sent to and accepted by the chosen target. Further information
170 from arcsub can be obtained by increasing the verbosity, which is done
171 with the --debug option where the default verbosity level is WARNING.
172 Setting the level to DEBUG will show all messages, while setting it to
173 FATAL will only show fatal log messages.
174 To validate your job description without actually submitting a job, use
176 the --dryrun option: it will capture possible syntax or other errors,
177 but will instruct the site not to submit the job for execution. Only
178 the grid-manager (ARC0) and A-REX (ARC1) CEs support this feature.
179
182 Submission of a job description file "helloworld.jsdl" to the A-REX CE
183 located at "example.com":
184 arcsub --cluster ARC1:example.com:60000/arex helloworld.jsdl
185 Similar submission to a CREAM CE:
187 arcsub --cluster CREAM:example.com helloworld.jsdl
188 And if the 'ARC0' module is installed submission to a Grid-Manager CE
190 can be done as:
191 arcsub --cluster ARC0:example.com helloworld.jsdl
192 Specifying the flavour is not mandatory, so this command will submit
194 the job to "example.com" regardless of the type of the CE (by trying
195 all possibilites):
196 arcsub --cluster example.com helloworld.jsdl
197 A ISIS information index server can also be queried for CEs to submit
199 to:
200 arcsub --index example.com/isis helloworld.jsdl
201 The job description can also be specified directly on the command line
203 as shown in the example, using the XRSL job description language:
204 arcsub --cluster example.com/arex --jobdescrstring \
205 ´&(executable="/bin/echo")(arguments="Hello World!")´
206 When submitting against CEs retrieved from information index servers it
208 might be useful to do resource brokering:
209 arcsub --index example.com/isis --broker FastestQueue \
210 helloworld.jsdl
211 Disregarding a specific CE for submission submitting against an infor‐
213 mation index server:
214 arcsub --index example.com/isis \
215 Dumping the job description is done as follows:
217 arcsub --cluster example.com/arex --dumpdescription \
218 helloworld.jsdl
219
223 ~/.arc/client.conf
224 Some options can be given default values by specifying them in
225 the ARC client configuration file. The default services can be
226 specified by the 'defaultservices' attribute, thus when no
227 --cluster or --index options are given these will be used for
228 submission. It is also possible to define aliases, and a alias
229 can consist of multiple CEs and index servers. By using the
230 --conffile option a different configuration file can be used
231 than the default. Note that some installations also have a sys‐
232 tem client configuration file, however attributes in the client
233 one takes precedence, and then command line options takes prece‐
234 dence over configuration file attributes.
235 ~/.arc/jobs.xml
238 This a local list of the user's active jobs. When a job is suc‐
239 cessfully submitted it is added to this list and when it is
240 removed from the remote cluster it is removed from this list.
241 This list is used as the list of all active jobs when the user
242 specifies the --all option to the various NorduGrid ARC user
243 interface commands. By using the --joblist option a different
244 file can be used than the default.
245
248 X509_USER_PROXY
249 The location of the user's grid proxy file. Shouldn't be set
250 unless the proxy is in a non-standard location.
251 ARC_LOCATION
254 The location where ARC is installed can be specified by this
255 variable. If not specified the install location will be deter‐
256 mined from the path to the command being executed, and if this
257 fails a WARNING will be given stating the location which will be
258 used.
259 ARC_PLUGIN_PATH
262 The location of ARC plugins can be specified by this variable.
263 Multiple locations can be specified by separating them by : (;
264 in Windows). The default location is $ARC_LOCATION/lib/arc (\ in
265 Windows).
266
269 APACHE LICENSE Version 2.0
270
273 Mattias Ellert <mattias.ellert@fysast.uu.se>
274
277 arccat(1), arcclean(1), arccp(1), arcget(1), arcinfo(1), arckill(1),
278 arcls(1), arcmigrate(1), arcproxy(1), arcrenew(1), arcresub(1), arcre‐
279 sume(1), arcrm(1), arcstat(1), arcsync(1), arctest(1)
280NorduGrid ARC 1.1.0 2011-10-24 ARCSUB(1)