1xrdcopy(1) General Commands Manual xrdcopy(1)
2
6 xrdcp - copy files
7
9 xrdcp [options] source destination
10 options: [--cksum args] [--debug lvl]
12 [--coerce] [--force] [--help] [--license]
13 [--nopbar] [--posc] [--proxy ipaddr:port]
14 [--recursive] [--retry time] [--server]
15 [--silent] [--sources num] [--streams num]
16 [--tpc [delegate] first|only] [--verbose]
17 [--version] [--xrate rate] [--zip file]
18 [--dynamic-src] [--infiles fn]
19 [--parallel n] [--allow-http] [--xattr]
20 [--notlsok] [--tlsnodata] [--tlsmetalink]
21 [--zip-mtln-cksum] [--rm-bad-cksum] [--continue]
22 [--xrate-threshold] [--retry-policy]
23 legacy options: [-adler] [-DSparm string] [-DIparm number]
25 [-md5] [-np] [-ODcgi] [-OScgi] [-x]
26
29 The xrdcp utility copies one or more files from one location to an‐
30 other. The data source and destination may be a local or remote file or
31 directory. Additionally, the data source may also reside on multiple
32 servers.
33
35 -C | --cksum type[:value|print|source]
36 obtains the checksum of type (i.e. adler32, crc32, md5 or zcrc32)
37 from the source, computes the checksum at the destination, and
38 verifies that they are the same. If auto is chosen as the checksum
39 type, xrdcp will try to automatically infer the right checksum
40 type based on source/destination configuration, source file type
41 (metalink, ZIP), and available checksum plug-ins. If a value is
42 specified, it is used as the source checksum. When print is spec‐
43 ified, the checksum at the destination is printed but is not veri‐
44 fied.
45 -d | --debug lvl
47 debug level: 1 (low), 2 (medium), 3 (high)
48 -F | --coerce
50 ignores locking semantics on the destination file. This option may
51 lead to file corruption if not properly used.
52 -f | --force
54 re-creates a file if it is already present.
55 -h | --help
57 displays usage information.
58 -H | --license
60 displays license terms and conditions.
61 -N | --nopbar
63 does not display the progress bar.
64 -P | --posc
66 requests POSC (persist-on-successful-close) processing to create a
67 new file. Files are automatically deleted should they not be suc‐
68 cessfully closed.
69 -D | --proxy proxyaddr:proxyport
71 [NOT YET IMPLEMENTED]
72 use proxyaddr:proxyport as a SOCKS4 proxy. Only numerical ad‐
74 dresses are supported.
75 -r | --recursive
77 recursively copy all files starting at the given source directory.
78 --retry
80 retry failed copy-jobs.
81 --server
83 runs as if in a server environment. Used only for server-side
84 third party copy support.
85 -s | --silent
87 neither produces summary information nor displays the progress
88 bar.
89 -y | --sources num
91 uses up to num sources to copy the file.
92 -S | --streams num
94 use num parallel data streams to do the transfer (the main stream
95 is in this case not used to carry out the transfer). The maximum
96 value is 15. The default is 0 (i.e., use only the control stream).
97 --tpc [delegate] first|only
99 copies the file from remote server to remote server using third-
100 party-copy protocol (i.e., data flows from server to server). The
101 source and destination servers must support third party copies.
102 Additional security restrictions may apply and may cause the copy
103 to fail if they cannot be satisfied. Argument 'first' tries tpc
104 and if it fails, does a normal copy; while 'only' fails the copy
105 unless tpc succeeds. When 'delegate' is specified, the copy dele‐
106 gates the command issuer's credentials to the target server which
107 uses those credentials to authenticate with the source server.
108 Delegation is ignored if the target server is not configured to
109 use delegated credentials. Currently, only gsi credentials can be
110 delegated.
111 -v | --verbose
113 displays summary output.
114 -V | -version
116 displays version information and immediately exits.
117 -z | --zip file
119 copy given file from a ZIP archive (same as xrdcl.unzip opaque
120 info).
121 -X | --xrate rate
123 limits the copy speed to the specified rate. The rate may be qual‐
124 ified with the letter k, m, or g to indicate kilo, mega, or giga
125 bytes, respectively. The option only applies when the source or
126 destination is local.
127 -X | --xrate-threshold rate
129 If the transfer rate drops below given threshold force the client
130 to use different source or if no more sources are available fail
131 the transfer.
132 -Z | --dynamic-src
134 file size may change during the copy
135 -I | --infiles fn
137 specifies the file that contains a list of input files
138 --parallel n
140 number of copy jobs to be run simultaneously
141 --allow-http
143 allow HTTP as source or destination protocol. Requires the Xrd‐
144 ClHttp client plugin
145 --xattr
147 preserve extended attributes
148 --notlsok
150 If server is too old to support TLS encryption fallback to unen‐
151 crypted communication.
152 --tlsnodata
154 In case of roots/xroots protocol, encrypt only the control stream
155 and leave the data streams unencrypted.
156 --tlsmetalink
158 Treat all URLs in metalink as roots/xroots.
159 --zip-mtln-cksum
161 Use the checksum available in a metalink file even if a file is
162 being extracted from a ZIP archive.
163 --rm-bad-cksum
165 Remove the target file if checksum verification failed (enables
166 also POSC semantics).
167 --continue
169 Continue copying a file from the point where the previous copy was
170 interrupted.
171
174 Legacy options are provided for backward compatibility. These are now
175 deprecated and should be avoided.
176 -adler
177 equivalent to "--cksum adler32:source".
178 -DIpname numberval
180 set the internal parameter pname with the numeric value numberval.
181 -DSpname stringval
183 set the internal parameter pname with the string value stringval.
184 -md5
186 equivalent to "--cksum md5:source".
187 -np
189 equivalent to "--nopbar".
190 -ODcgi
192 add cgi information cgi to any destination xrootd URL. You should
193 specify the opaque information directly on the destination URL.
194 -OScgi
196 add cgi information cgi to any source xrootd URL.
197 -x
199 equivalent to "--sources 12".
200
203 source
204 a dash (i.e. -) indicating stanard in, a local file, a local di‐
205 rectory name suffixed by /, or an xrootd URL in the form of
206 xroot://[user@]host[:port]/absolutepath
207 The absolutepath can be a directory.
208 destination
210 a dash (i.e. -) indicating stanard out, a local file, a local di‐
211 rectory name suffixed by /, or an xrootd URL in the form
212 xroot://[user@]host[:port]/absolutepath
213 The absolutepath can be a directory.
214
218 The following environment variables are supported. They apply to xrdfs
219 and any other application using the libXrdCl library, unless specified
220 otherwise. The text in the brackets is a name of the corresponding
221 xrdcp commandline parameter.
222 XRD_LOGLEVEL
224 Determines the amount of diagnostics that should be printed. Valid
225 values are: Dump, Debug, Info, Warning, and Error.
226 XRD_LOGFILE
228 If set, the diagnostics will be printed to the specified file in‐
229 stead of stderr.
230 XRD_LOGMASK
232 Determines which diagnostics topics should be printed at all lev‐
233 els. It's a "|" separated list of topics. The first element may be
234 "All" in which case all the topics are enabled and the subsequent
235 elements may turn them off, or "None" in which case all the topics
236 are disabled and the subsequent flags may turn them on. If the
237 topic name is prefixed with "^", then it means that the topic
238 should be disabled. If the topic name is not prefixed, then it
239 means that the topic should be enabled.
240 The log mask may as well be handled for each diagnostic level sep‐
242 arately by setting one or more of the following variables:
243 XRD_LOGMASK_ERROR, XRD_LOGMASK_WARNING, XRD_LOGMASK_INFO, XRD_LOG‐
244 MASK_DEBUG, and XRD_LOGMASK_DUMP. The default for each level is
245 "All", except for the Dump level, where the default is
246 "All|^PollerMsg". This means that, at the Dump level, all the top‐
247 ics but "PollerMsg" are enabled.
248 Available topics: AppMsg, UtilityMsg, FileMsg, PollerMsg, PostMas‐
250 terMsg, XRootDTransportMsg, TaskMgrMsg, XRootDMsg, FileSystemMsg,
251 AsyncSockMsg, TlsMsg
252 XRD_TLSDBGLVL
254 Determine the debug level for the TLS component. Valid values are:
255 OFF, CTX, SOK, SIO, ALL and OUT. The default value is OFF.
256 XRD_PARALLELEVTLOOP
258 The number of event loops (i.e. the number of threads handling re‐
259 quests). Default number is 10.
260 XRD_READRECOVERY
262 Determines if read recovery should be enabled or disabled (enabled
263 by default).
264 XRD_WRITERECOVERY
266 Determines if write recovery should be enabled or disabled (en‐
267 abled by default).
268 XRD_OPENRECOVERY
270 Determines if open recovery should be enabled or disabled for mu‐
271 table (truncate or create) opens (enabled by default).
272 XRD_CONNECTIONWINDOW (-DIConnectionWindow)
274 A time window for the connection establishment. A connection fail‐
275 ure is declared if the connection is not established within the
276 time window. If a connection failure happens earlier then another
277 connection attempt will only be made at the beginning of the next
278 window.
279 XRD_CONNECTIONRETRY (-DIConnectionRetry)
281 Number of connection attempts that should be made (number of
282 available connection windows) before declaring a permanent fail‐
283 ure.
284 XRD_REQUESTTIMEOUT (-DIRequestTimeout)
286 Default value for the time after which an error is declared if it
287 was impossible to get a response to a request.
288 XRD_STREAMTIMEOUT (-DIStreamTimeout)
290 Default value for the time after which a connection error is de‐
291 clared (and a recovery attempted) if there are unfulfilled re‐
292 quests and there is no socket activity or a registered wait time‐
293 out.
294 XRD_SUBSTREAMSPERCHANNEL (-DISubStreamsPerChannel)
296 Number of streams per session.
297 XRD_TIMEOUTRESOLUTION (-DITimeoutResolution)
299 Resolution for the timeout events. Ie. timeout events will be pro‐
300 cessed only every XRD_TIMEOUTRESOLUTION seconds.
301 XRD_STREAMERRORWINDOW (-DIStreamErrorWindow)
303 Time after which the permanent failure flags are cleared out and a
304 new connection may be attempted if needed.
305 XRD_RUNFORKHANDLER (-DIRunForkHandler)
307 Determines whether the fork handlers should be enabled, making the
308 API fork safe.
309 XRD_REDIRECTLIMIT (-DIRedirectLimit)
311 Maximum number of allowed redirections.
312 XRD_NOTAUTHORIZEDRETRYLIMIT (-dNotAuthorizedRetryLimit)
314 Maximum number of allowed retries at a meta-manager for not-autho‐
315 rized error.
316 XRD_POLLERPREFERENCE (-DSPollerPreference)
318 A comma separated list of poller implementations in order of pref‐
319 erence. The default is: built-in.
320 XRD_CLIENTMONITOR (-DSClientMonitor)
322 Path to the client monitor library.
323 XRD_CLIENTMONITORPARAM (-DSClientMonitorParam)
325 Additional optional parameters that will be passed to the monitor‐
326 ing object on initialization.
327 XRD_WORKERTHREADS (-DIWorkerThreads)
329 Number of threads processing user callbacks.
330 XRD_CPPARALLELCHUNKS (-DICPParallelChunks)
332 Maximum number of asynchronous requests being processed by the
333 xrdcp command per connected channel substream (adjusted in real-
334 time).
335 XRD_CPCHUNKSIZE (-DICPChunkSize)
337 Size of a single data chunk handled by xrdcp.
338 XRD_NETWORKSTACK (-DSNetworkStack)
340 The network stack that the client should use to connect to the
341 server. Possible values are:
342 IPAuto - automatically detect which IP stack to use
344 IPAll - use IPv6 stack (AF_INET6 sockets) and both IPv6 and IPv4
346 (mapped to IPv6) addresses
347 IPv6 - use only IPv6 stack and addresses
349 IPv4 - use only IPv4 stack (AF_INET sockets) and addresses
351 IPv4Mapped6 - use IPv6 stack and mapped IPv4 addresses
353 XRD_DATASERVERTTL (-DIDataServerTTL)
355 Time period after which an idle connection to a data server should
356 be closed.
357 XRD_LOADBALANCERTTL (-DILoadBalancerTTL)
359 Time period after which an idle connection to a manager or a load
360 balancer should be closed.
361 XRD_APPNAME (-DSAppName)
363 Override the application name reported to the server.
364 XRD_PLUGINCONFDIR
366 A custom location containing client plug-in config files.
367 XRD_PLUGIN
369 A default client plug-in to be used.
370 XRD_CPINITTIMEOUT (-DICPInitTimeout)
372 Maximum time allowed for the copy process to initialize, ie. open
373 the source and destination files.
374 XRD_CPTPCTIMEOUT (-DICPTPCTimeout)
376 Maximum time allowed for a third-party copy operation to finish.
377 XRD_TCPKEEPALIVE (-DITCPKeepAlive)
379 Enable/Disable the TCP keep alive functionality
380 XRD_TCPKEEPALIVETIME (-DITCPKeepAliveTime)
382 Time between last data packet sent and the first keepalive probe
383 (Linux only)
384 XRD_TCPKEEPALIVEINTERVAL (-DITCPKeepAliveInterval)
386 Interval between subsequent keepalive probes (Linux only)
387 XRD_TCPKEEPALIVEPROBES (-DITCPKeepProbes)
389 Number of unacknowledged probes before considering the connection
390 dead (Linux only)
391 XRD_METALINKPROCESSING
393 Enable/Disable Metalink processing (enabled by default)
394 XRD_LOCALMETALINKFILE
396 Enable/Disable local Metalink file processing (by convention the
397 following URL schema has to be used: root://localfile//path/file‐
398 name.meta4) The 'localfile' semantic is now deprecated, use
399 file://localhost/path/filename.meta4 instead!
400 XRD_GLFNREDIRECTOR
402 The redirector will be used as a last resort if the GLFN tag is
403 specified in a Metalink file.
404 XRD_XCPBLOCKSIZE
406 Maximu size of a data block assigned to a single source in case of
407 an extreme copy transfer.
408 XRD_NODELAY
410 Disables the Nagle algorithm if set to 1 (default), enables it if
411 set to 0.
412 XRD_PREFERIPV4
414 If set the client tries first IPv4 address (turned off by de‐
415 fault).
416 XRD_MAXMETALINKWAIT
418 The maximum time in seconds a clinet can be stalled by the server
419 if a Metalink redirector is available (defaults to 60s).
420 XRD_PRESERVELOCATETRIED
422 If set to 1 XRootD client will preserve tried/triedrc cgi opaque
423 info for kXR_locate request across redirects/retries, if set to 0
424 XRootD client will treat kXR_locate as any other passive request.
425 XRD_PRESERVEXATTRS
427 If set to 1 (default) xrdcp will preserve file extended attribues,
428 if set to 0 file extended attributes won't be preserved.
429 XRD_NOTLSOK
431 If set to 1 and the server is to old to support TLS encryption,
432 xrdcp will fallback to unencrypted transmission although
433 roots/xroots protocol was used. By default set to 0.
434 XRD_TLSNODATA
436 If set to 1, in case of roots/xroots protocol only the control
437 stream will be encrypted, data streams will be left unencrypted.
438 By default set to 0.
439 XRD_TLSMETALINK
441 If set to 1 all URLs in metalink will be treated as roots/xroots.
442 By default set to 0.
443 XRD_ZIPMTLNCKSUM
445 If set to 1, use the checksum available in a metalink file even if
446 a file is being extracted from a ZIP archive. By default set to
447 0;
448 XRD_CPTIMEOUT
450 Timeout for a classical (not TPC) copy job. By default set to 0
451 (disabled);
452 XRD_CLCONFDIR
454 User defined directory with config files (*.conf).
455 XRD_CLCONFFILE
457 User defined config file.
458 XRD_CPRETRY
460 Maximum number of times to retry failed copy-jobs (default: 0).
461 XRD_CPRETRYPOLICY
463 Copy job retry policy, either force or retry (default: force).
464 XRD_CPUSEPGWRTRD
466 Enable in-fly error correction of corrupted pages (default: 1).
467
470 50 : generic error (e.g. config, internal, data, OS, command line op‐
471 tion)
472 51 : socket related error
474 52 : postmaster related error
476 53 : XRootD related error
478 54 : redirection error
480 55 : query response was negative (this is not an error)
482
485 Documentation for all components associated with xrdcp can be found at
486 http://xrootd.org/docs.html
487
490 Errors yield an error message and a non-zero exit status.
491
494 LGPL
495
498 The xrdcp command is supported by the xrootd collaboration. Contact
499 information can be found at:
500 http://xrootd.org/contact.html
502 v5.6.3 xrdcopy(1)