1xrdcopy(1) General Commands Manual xrdcopy(1)
2
6 xrdcp - copy files
7
9 xrdcp [options] source destination
10 options: [--cksum args] [--debug lvl]
12 [--coerce] [--force] [--help] [--license]
13 [--nopbar] [--posc] [--proxy ipaddr:port]
14 [--recursive] [--retry time] [--server]
15 [--silent] [--sources num] [--streams num]
16 [--tpc [delegate] first|only] [--verbose]
17 [--version] [--xrate rate] [--zip file]
18 [--dynamic-src] [--infiles fn]
19 [--parallel n] [--allow-http] [--xattr]
20 [--notlsok] [--tlsnodata] [--tlsmetalink]
21 [--zip-mtln-cksum] [--rm-bad-cksum] [--continue]
22 [--xrate-threshold] [--retry-policy]
23 legacy options: [-adler] [-DSparm string] [-DIparm number]
25 [-md5] [-np] [-ODcgi] [-OScgi] [-x]
26
29 The xrdcp utility copies one or more files from one location to an‐
30 other. The data source and destination may be a local or remote file or
31 directory. Additionally, the data source may also reside on multiple
32 servers.
33
35 -C | --cksum type[:value|print|source]
36 obtains the checksum of type (i.e. adler32, crc32, md5 or zcrc32)
37 from the source, computes the checksum at the destination, and
38 verifies that they are the same. If auto is chosen as the checksum
39 type, xrdcp will try to automatically infer the right checksum
40 type based on source/destination configuration, source file type
41 (metalink, ZIP), and available checksum plug-ins. If a value is
42 specified, it is used as the source checksum. When print is spec‐
43 ified, the checksum at the destination is printed but is not veri‐
44 fied.
45 -d | --debug lvl
47 debug level: 1 (low), 2 (medium), 3 (high)
48 -F | --coerce
50 ignores locking semantics on the destination file. This option may
51 lead to file corruption if not properly used.
52 -f | --force
54 re-creates a file if it is already present.
55 -h | --help
57 displays usage information.
58 -H | --license
60 displays license terms and conditions.
61 -N | --nopbar
63 does not display the progress bar.
64 -P | --posc
66 requests POSC (persist-on-successful-close) processing to create a
67 new file. Files are automatically deleted should they not be suc‐
68 cessfully closed.
69 -D | --proxy proxyaddr:proxyport
71 [NOT YET IMPLEMENTED]
72 use proxyaddr:proxyport as a SOCKS4 proxy. Only numerical ad‐
74 dresses are supported.
75 -r | --recursive
77 recursively copy all files starting at the given source directory.
78 --retry
80 retry failed copy-jobs.
81 --server
83 runs as if in a server environment. Used only for server-side
84 third party copy support.
85 -s | --silent
87 neither produces summary information nor displays the progress
88 bar.
89 -y | --sources num
91 uses up to num sources to copy the file.
92 -S | --streams num
94 use num parallel data streams to do the transfer (the main stream
95 is in this case not used to carry out the transfer). The maximum
96 value is 15. The default is 0 (i.e., use only the control stream).
97 --tpc [delegate] first|only
99 copies the file from remote server to remote server using third-
100 party-copy protocol (i.e., data flows from server to server). The
101 source and destination servers must support third party copies.
102 Additional security restrictions may apply and may cause the copy
103 to fail if they cannot be satisfied. Argument 'first' tries tpc
104 and if it fails, does a normal copy; while 'only' fails the copy
105 unless tpc succeeds. When 'delegate' is specified, the copy dele‐
106 gates the command issuer's credentials to the target server which
107 uses those credentials to authenticate with the source server.
108 Delegation is ignored if the target server is not configured to
109 use delegated credentials. Currently, only gsi credentials can be
110 delegated.
111 -v | --verbose
113 displays summary output.
114 -V | -version
116 displays version information and immediately exits.
117 -z | --zip file
119 copy given file from a ZIP archive (same as xrdcl.unzip opaque
120 info).
121 -X | --xrate rate
123 limits the copy speed to the specified rate. The rate may be qual‐
124 ified with the letter k, m, or g to indicate kilo, mega, or giga
125 bytes, respectively. The option only applies when the source or
126 destination is local.
127 -X | --xrate-threshold rate
129 If the transfer rate drops bellow given threshold force the client
130 to use different source or if no more sources are available fail
131 the transfer.
132 -Z | --dynamic-src
134 file size may change during the copy
135 -I | --infiles fn
137 specifies the file that contains a list of input files
138 --parallel n
140 number of copy jobs to be run simultaneously
141 --allow-http
143 allow HTTP as source or destination protocol. Requires the Xrd‐
144 ClHttp client plugin
145 --xattr
147 preserve extended attributes
148 --notlsok
150 If server is too old to support TLS encryption fallback to unen‐
151 crypted communication.
152 --tlsnodata
154 In case of roots/xroots protocol, encrypt only the control stream
155 and leave the data streams unencrypted.
156 --tlsmetalink
158 Treat all URLs in metalink as roots/xroots.
159 --zip-mtln-cksum
161 Use the checksum available in a metalink file even if a file is
162 being extracted from a ZIP archive.
163 --rm-bad-cksum
165 Remove the target file if checksum verification failed (enables
166 also POSC semantics).
167 --continue
169 Continue copying a file from the point where the previous copy was
170 interrupted.
171
174 Legacy options are provided for backward compatability. These are now
175 deprecated and should be avoided.
176 -adler
177 equivalent to "--cksum adler32:source".
178 -DIpname numberval
180 set the internal parameter pname with the numeric value numberval.
181 -DSpname stringval
183 set the internal parameter pname with the string value stringval.
184 -md5
186 equivalent to "--cksum md5:source".
187 -np
189 equivalent to "--nopbar".
190 -ODcgi
192 add cgi information cgi to any destination xrootd URL. You should
193 specify the opaque information directly on the destination URL.
194 -OScgi
196 add cgi information cgi to any source xrootd URL.
197 -x
199 equivalent to "--sources 12".
200
203 source
204 a dash (i.e. -) indicating stanard in, a local file, a local di‐
205 rectory name suffixed by /, or an xrootd URL in the form of
206 xroot://[user@]host[:port]/absolutepath
207 The absolutepath can be a directory.
208 destination
210 a dash (i.e. -) indicating stanard out, a local file, a local di‐
211 rectory name suffixed by /, or an xrootd URL in the form
212 xroot://[user@]host[:port]/absolutepath
213 The absolutepath can be a directory.
214
218 The following environment variables are supported. They apply to xrdfs
219 and any other application using the libXrdCl library, unless specified
220 otherwise. The text in the brackets is a name of the corresponding
221 xrdcp commandline parameter.
222 XRD_LOGLEVEL
224 Detemines the amout of diagnostics that should be printed. Valid
225 values are: Dump, Debug, Info, Warning, and Error.
226 XRD_LOGFILE
228 If set, the diagnostics will be printed to the specified file in‐
229 stead of stderr.
230 XRD_LOGMASK
232 Determines which diagnostics topics should be printed at all lev‐
233 els. It's a "|" separated list of topics. The first element may be
234 "All" in which case all the topics are enabled and the subsequent
235 elements may turn them off, or "None" in which case all the topics
236 are disabled and the subsequent flags may turn them on. If the
237 topic name is prefixed with "^", then it means that the topic
238 should be disabled. If the topic name is not prefixed, then it
239 means that the topic should be enabled.
240 The log mask may as well be handled for each diagnostic level sep‐
242 arately by setting one or more of the following variables:
243 XRD_LOGMASK_ERROR, XRD_LOGMASK_WARNING, XRD_LOGMASK_INFO, XRD_LOG‐
244 MASK_DEBUG, and XRD_LOGMASK_DUMP. The default for each level is
245 "All", except for the Dump level, where the default is
246 "All|^PollerMsg". This means that, at the Dump level, all the top‐
247 ics but "PollerMsg" are enabled.
248 Available topics: AppMsg, UtilityMsg, FileMsg, PollerMsg, PostMas‐
250 terMsg, XRootDTransportMsg, TaskMgrMsg, XRootDMsg, FileSystemMsg,
251 AsyncSockMsg, TlsMsg
252 XRD_TLSDBGLVL
254 Determine the debug level for the TLS component. Valid values are:
255 OFF, CTX, SOK, SIO, ALL and OUT. The default value is OFF.
256 XRD_PARALLELEVTLOOP
258 The number of event loops.
259 XRD_READRECOVERY
261 Determines if read recovery should be enabled or disabled (enabled
262 by default).
263 XRD_WRITERECOVERY
265 Determines if write recovery should be enabled or disabled (en‐
266 abled by default).
267 XRD_OPENRECOVERY
269 Determines if open recovery should be enabled or disabled for mu‐
270 table (truncate or create) opens (enabled by default).
271 XRD_CONNECTIONWINDOW (-DIConnectionWindow)
273 A time window for the connection establishment. A connection fail‐
274 ure is declared if the connection is not established within the
275 time window. If a connection failure happens earlier then another
276 connection attempt will only be made at the beginning of the next
277 window.
278 XRD_CONNECTIONRETRY (-DIConnectionRetry)
280 Number of connection attempts that should be made (number of
281 available connection windows) before declaring a permanent fail‐
282 ure.
283 XRD_REQUESTTIMEOUT (-DIRequestTimeout)
285 Default value for the time after which an error is declared if it
286 was impossible to get a response to a request.
287 XRD_STREAMTIMEOUT (-DIStreamTimeout)
289 Default value for the time after which a connection error is de‐
290 clared (and a recovery attempted) if there are unfulfilled re‐
291 quests and there is no socket activity or a registered wait time‐
292 out.
293 XRD_SUBSTREAMSPERCHANNEL (-DISubStreamsPerChannel)
295 Number of streams per session.
296 XRD_TIMEOUTRESOLUTION (-DITimeoutResolution)
298 Resolution for the timeout events. Ie. timeout events will be pro‐
299 cessed only every XRD_TIMEOUTRESOLUTION seconds.
300 XRD_STREAMERRORWINDOW (-DIStreamErrorWindow)
302 Time after which the permanent failure flags are cleared out and a
303 new connection may be attempted if needed.
304 XRD_RUNFORKHANDLER (-DIRunForkHandler)
306 Determines whether the fork handlers should be enabled, making the
307 API fork safe.
308 XRD_REDIRECTLIMIT (-DIRedirectLimit)
310 Maximum number of allowed redirections.
311 XRD_NOTAUTHORIZEDRETRYLIMIT (-dNotAuthorizedRetryLimit)
313 Maximum number of allowed retries at a meta-manager for not-autho‐
314 rized error.
315 XRD_POLLERPREFERENCE (-DSPollerPreference)
317 A comma separated list of poller implementations in order of pref‐
318 erence. The default is: built-in.
319 XRD_CLIENTMONITOR (-DSClientMonitor)
321 Path to the client monitor library.
322 XRD_CLIENTMONITORPARAM (-DSClientMonitorParam)
324 Additional optional parameters that will be passed to the monitor‐
325 ing object on initialization.
326 XRD_WORKERTHREADS (-DIWorkerThreads)
328 Number of threads processing user callbacks.
329 XRD_CPPARALLELCHUNKS (-DICPParallelChunks)
331 Maximum number of asynchronous requests being processed by the
332 xrdcp command per connected channel substream (adjusted in real-
333 time).
334 XRD_CPCHUNKSIZE (-DICPChunkSize)
336 Size of a single data chunk handled by xrdcp.
337 XRD_NETWORKSTACK (-DSNetworkStack)
339 The network stack that the client should use to connect to the
340 server. Possible values are:
341 IPAuto - automatically detect which IP stack to use
343 IPAll - use IPv6 stack (AF_INET6 sockets) and both IPv6 and IPv4
345 (mapped to IPv6) addresses
346 IPv6 - use only IPv6 stack and addresses
348 IPv4 - use only IPv4 stack (AF_INET sockets) and addresses
350 IPv4Mapped6 - use IPv6 stack and mapped IPv4 addresses
352 XRD_DATASERVERTTL (-DIDataServerTTL)
354 Time period after which an idle connection to a data server should
355 be closed.
356 XRD_LOADBALANCERTTL (-DILoadBalancerTTL)
358 Time period after which an idle connection to a manager or a load
359 balancer should be closed.
360 XRD_APPNAME (-DSAppName)
362 Override the application name reported to the server.
363 XRD_PLUGINCONFDIR
365 A custom location containing client plug-in config files.
366 XRD_PLUGIN
368 A default client plug-in to be used.
369 XRD_CPINITTIMEOUT (-DICPInitTimeout)
371 Maximum time allowed for the copy process to initialize, ie. open
372 the source and destination files.
373 XRD_CPTPCTIMEOUT (-DICPTPCTimeout)
375 Maximum time allowed for a third-party copy operation to finish.
376 XRD_TCPKEEPALIVE (-DITCPKeepAlive)
378 Enable/Disable the TCP keep alive functionality
379 XRD_TCPKEEPALIVETIME (-DITCPKeepAliveTime)
381 Time between last data packet sent and the first keepalive probe
382 (Linux only)
383 XRD_TCPKEEPALIVEINTERVAL (-DITCPKeepAliveInterval)
385 Interval between subsequent keepalive probes (Linux only)
386 XRD_TCPKEEPALIVEPROBES (-DITCPKeepProbes)
388 Number of unacknowledged probes before considering the connection
389 dead (Linux only)
390 XRD_METALINKPROCESSING
392 Enable/Disable Metalink processing (enabled by default)
393 XRD_LOCALMETALINKFILE
395 Enable/Disable local Metalink file processing (by convention the
396 following URL schema has to be used: root://localfile//path/file‐
397 name.meta4) The 'localfile' semantic is now deprecated, use
398 file://localhost/path/filename.meta4 instead!
399 XRD_GLFNREDIRECTOR
401 The redirector will be used as a last resort if the GLFN tag is
402 specified in a Metalink file.
403 XRD_XCPBLOCKSIZE
405 Maximu size of a data block assigned to a single source in case of
406 an extreme copy transfer.
407 XRD_NODELAY
409 Disables the Nagle algorithm if set to 1 (default), enables it if
410 set to 0.
411 XRD_PREFERIPV4
413 If set the client tries first IPv4 address (turned off by de‐
414 fault).
415 XRD_MAXMETALINKWAIT
417 The maximum time in seconds a clinet can be stalled by the server
418 if a Metalink redirector is available (defaults to 60s).
419 XRD_PRESERVELOCATETRIED
421 If set to 1 XRootD client will preserve tired/triedrc cgi opaque
422 info for kXR_locate request accross redirects/retries, if set to 0
423 XRootD client will treat kXR_locate as any other passive request.
424 XRD_PRESERVEXATTRS
426 If set to 1 (default) xrdcp will preverve file extended attribues,
427 if set to 0 file extended attributes wont be preserved.
428 XRD_NOTLSOK
430 If set to 1 and the server is to old to support TLS encryption,
431 xrdcp will fallback to unencrypted transmission although
432 roots/xroots protocol was used. By default set to 0.
433 XRD_TLSNODATA
435 If set to 1, in case of roots/xroots protocol only the control
436 stream will be encrypted, data streams will be left unencrypted.
437 By default set to 0.
438 XRD_TLSMETALINK
440 If set to 1 all URLs in metalink will be treated as roots/xroots.
441 By default set to 0.
442 XRD_ZIPMTLNCKSUM
444 If set to 1, use the checksum available in a metalink file even if
445 a file is being extracted from a ZIP archive. By defaut set to 0;
446 XRD_CPTIMEOUT
448 Timeout for a classical (not TPC) copy job. By defaut set to 0
449 (disabled);
450 XRD_CLCONFDIR
452 User defined directory with config files (*.conf).
453 XRD_CLCONFFILE
455 User defined config file.
456 XRD_CPRETRY
458 Maximum number of times to retry failed copy-jobs (default: 0).
459 XRD_CPRETRYPOLICY
461 Copy job retry policy, either force or retry (default: force).
462
465 50 : generic error (e.g. config, internal, data, OS, command line op‐
466 tion)
467 51 : socket related error
469 52 : postmaster related error
471 53 : XRootD related error
473 54 : redirection error
475 55 : query response was negative (this is not an error)
477
480 Documentation for all components associated with xrdcp can be found at
481 http://xrootd.org/docs.html
482
485 Errors yield an error message and a non-zero exit status.
486
489 LGPL
490
493 The xrdcp command is supported by the xrootd collaboration. Contact
494 information can be found at:
495 http://xrootd.org/contact.html
497 v5.3.3 xrdcopy(1)