1xrdcopy(1) General Commands Manual xrdcopy(1)
2
6 xrdcp - copy files
7
9 xrdcp [options] source destination
10 options: [--cksum args] [--debug lvl]
12 [--coerce] [--force] [--help] [--license]
13 [--nopbar] [--posc] [--proxy ipaddr:port]
14 [--recursive] [--retry time] [--server]
15 [--silent] [--sources num] [--streams num]
16 [--tpc [delegate] first|only] [--verbose]
17 [--version] [--xrate rate] [--zip file]
18 [--dynamic-src] [--infiles fn]
19 [--parallel n] [--allow-http] [--xattr]
20 [--notlsok] [--tlsnodata] [--tlsmetalink]
21 [--zip-mtln-cksum] [--rm-bad-cksum] [--continue]
22 [--xrate-threshold]
23 legacy options: [-adler] [-DSparm string] [-DIparm number]
25 [-md5] [-np] [-ODcgi] [-OScgi] [-x]
26
29 The xrdcp utility copies one or more files from one location to an‐
30 other. The data source and destination may be a local or remote file or
31 directory. Additionally, the data source may also reside on multiple
32 servers.
33
35 -C | --cksum type[:value|print|source]
36 obtains the checksum of type (i.e. adler32, crc32, md5 or zcrc32)
37 from the source, computes the checksum at the destination, and
38 verifies that they are the same. If auto is chosen as the checksum
39 type, xrdcp will try to automatically infer the right checksum
40 type based on source/destination configuration, source file type
41 (metalink, ZIP), and available checksum plug-ins. If a value is
42 specified, it is used as the source checksum. When print is spec‐
43 ified, the checksum at the destination is printed but is not veri‐
44 fied.
45 -d | --debug lvl
47 debug level: 1 (low), 2 (medium), 3 (high)
48 -F | --coerce
50 ignores locking semantics on the destination file. This option may
51 lead to file corruption if not properly used.
52 -f | --force
54 re-creates a file if it is already present.
55 -h | --help
57 displays usage information.
58 -H | --license
60 displays license terms and conditions.
61 -N | --nopbar
63 does not display the progress bar.
64 -P | --posc
66 requests POSC (persist-on-successful-close) processing to create a
67 new file. Files are automatically deleted should they not be suc‐
68 cessfully closed.
69 -D | --proxy proxyaddr:proxyport
71 [NOT YET IMPLEMENTED]
72 use proxyaddr:proxyport as a SOCKS4 proxy. Only numerical ad‐
74 dresses are supported.
75 -r | --recursive
77 recursively copy all files starting at the given source directory.
78 --server
80 runs as if in a server environment. Used only for server-side
81 third party copy support.
82 -s | --silent
84 neither produces summary information nor displays the progress
85 bar.
86 -y | --sources num
88 uses up to num sources to copy the file.
89 -S | --streams num
91 use num parallel data streams to do the transfer (the main stream
92 is in this case not used to carry out the transfer). The maximum
93 value is 15. The default is 0 (i.e., use only the control stream).
94 --tpc [delegate] first|only
96 copies the file from remote server to remote server using third-
97 party-copy protocol (i.e., data flows from server to server). The
98 source and destination servers must support third party copies.
99 Additional security restrictions may apply and may cause the copy
100 to fail if they cannot be satisfied. Argument 'first' tries tpc
101 and if it fails, does a normal copy; while 'only' fails the copy
102 unless tpc succeeds. When 'delegate' is specified, the copy dele‐
103 gates the command issuer's credentials to the target server which
104 uses those credentials to authenticate with the source server.
105 Delegation is ignored if the target server is not configured to
106 use delegated credentials. Currently, only gsi credentials can be
107 delegated.
108 -v | --verbose
110 displays summary output.
111 -V | -version
113 displays version information and immediately exits.
114 -z | --zip file
116 copy given file from a ZIP archive (same as xrdcl.unzip opaque
117 info).
118 -X | --xrate rate
120 limits the copy speed to the specified rate. The rate may be qual‐
121 ified with the letter k, m, or g to indicate kilo, mega, or giga
122 bytes, respectively. The option only applies when the source or
123 destination is local.
124 -X | --xrate-threshold rate
126 If the transfer rate drops bellow given threshold force the client
127 to use different source or if no more sources are available fail
128 the transfer.
129 -Z | --dynamic-src
131 file size may change during the copy
132 -I | --infiles fn
134 specifies the file that contains a list of input files
135 --parallel n
137 number of copy jobs to be run simultaneously
138 --allow-http
140 allow HTTP as source or destination protocol. Requires the Xrd‐
141 ClHttp client plugin
142 --xattr
144 preserve extended attributes
145 --notlsok
147 If server is too old to support TLS encryption fallback to unen‐
148 crypted communication.
149 --tlsnodata
151 In case of roots/xroots protocol, encrypt only the control stream
152 and leave the data streams unencrypted.
153 --tlsmetalink
155 Treat all URLs in metalink as roots/xroots.
156 --zip-mtln-cksum
158 Use the checksum available in a metalink file even if a file is
159 being extracted from a ZIP archive.
160 --rm-bad-cksum
162 Remove the target file if checksum verification failed (enables
163 also POSC semantics).
164 --continue
166 Continue copying a file from the point where the previous copy was
167 interrupted.
168
171 Legacy options are provided for backward compatability. These are now
172 deprecated and should be avoided.
173 -adler
174 equivalent to "--cksum adler32:source".
175 -DIpname numberval
177 set the internal parameter pname with the numeric value numberval.
178 -DSpname stringval
180 set the internal parameter pname with the string value stringval.
181 -md5
183 equivalent to "--cksum md5:source".
184 -np
186 equivalent to "--nopbar".
187 -ODcgi
189 add cgi information cgi to any destination xrootd URL. You should
190 specify the opaque information directly on the destination URL.
191 -OScgi
193 add cgi information cgi to any source xrootd URL.
194 -x
196 equivalent to "--sources 12".
197
200 source
201 a dash (i.e. -) indicating stanard in, a local file, a local di‐
202 rectory name suffixed by /, or an xrootd URL in the form of
203 xroot://[user@]host[:port]/absolutepath
204 The absolutepath can be a directory.
205 destination
207 a dash (i.e. -) indicating stanard out, a local file, a local di‐
208 rectory name suffixed by /, or an xrootd URL in the form
209 xroot://[user@]host[:port]/absolutepath
210 The absolutepath can be a directory.
211
215 The following environment variables are supported. They apply to xrdfs
216 and any other application using the libXrdCl library, unless specified
217 otherwise. The text in the brackets is a name of the corresponding
218 xrdcp commandline parameter.
219 XRD_LOGLEVEL
221 Detemines the amout of diagnostics that should be printed. Valid
222 values are: Dump, Debug, Info, Warning, and Error.
223 XRD_LOGFILE
225 If set, the diagnostics will be printed to the specified file in‐
226 stead of stderr.
227 XRD_LOGMASK
229 Determines which diagnostics topics should be printed at all lev‐
230 els. It's a "|" separated list of topics. The first element may be
231 "All" in which case all the topics are enabled and the subsequent
232 elements may turn them off, or "None" in which case all the topics
233 are disabled and the subsequent flags may turn them on. If the
234 topic name is prefixed with "^", then it means that the topic
235 should be disabled. If the topic name is not prefixed, then it
236 means that the topic should be enabled.
237 The log mask may as well be handled for each diagnostic level sep‐
239 arately by setting one or more of the following variables:
240 XRD_LOGMASK_ERROR, XRD_LOGMASK_WARNING, XRD_LOGMASK_INFO, XRD_LOG‐
241 MASK_DEBUG, and XRD_LOGMASK_DUMP. The default for each level is
242 "All", except for the Dump level, where the default is
243 "All|^PollerMsg". This means that, at the Dump level, all the top‐
244 ics but "PollerMsg" are enabled.
245 Available topics: AppMsg, UtilityMsg, FileMsg, PollerMsg, PostMas‐
247 terMsg, XRootDTransportMsg, TaskMgrMsg, XRootDMsg, FileSystemMsg,
248 AsyncSockMsg, TlsMsg
249 XRD_TLSDBGLVL
251 Determine the debug level for the TLS component. Valid values are:
252 OFF, CTX, SOK, SIO, ALL and OUT. The default value is OFF.
253 XRD_PARALLELEVTLOOP
255 The number of event loops.
256 XRD_READRECOVERY
258 Determines if read recovery should be enabled or disabled (enabled
259 by default).
260 XRD_WRITERECOVERY
262 Determines if write recovery should be enabled or disabled (en‐
263 abled by default).
264 XRD_OPENRECOVERY
266 Determines if open recovery should be enabled or disabled for mu‐
267 table (truncate or create) opens (enabled by default).
268 XRD_CONNECTIONWINDOW (-DIConnectionWindow)
270 A time window for the connection establishment. A connection fail‐
271 ure is declared if the connection is not established within the
272 time window. If a connection failure happens earlier then another
273 connection attempt will only be made at the beginning of the next
274 window.
275 XRD_CONNECTIONRETRY (-DIConnectionRetry)
277 Number of connection attempts that should be made (number of
278 available connection windows) before declaring a permanent fail‐
279 ure.
280 XRD_REQUESTTIMEOUT (-DIRequestTimeout)
282 Default value for the time after which an error is declared if it
283 was impossible to get a response to a request.
284 XRD_STREAMTIMEOUT (-DIStreamTimeout)
286 Default value for the time after which a connection error is de‐
287 clared (and a recovery attempted) if there are unfulfilled re‐
288 quests and there is no socket activity or a registered wait time‐
289 out.
290 XRD_SUBSTREAMSPERCHANNEL (-DISubStreamsPerChannel)
292 Number of streams per session.
293 XRD_TIMEOUTRESOLUTION (-DITimeoutResolution)
295 Resolution for the timeout events. Ie. timeout events will be pro‐
296 cessed only every XRD_TIMEOUTRESOLUTION seconds.
297 XRD_STREAMERRORWINDOW (-DIStreamErrorWindow)
299 Time after which the permanent failure flags are cleared out and a
300 new connection may be attempted if needed.
301 XRD_RUNFORKHANDLER (-DIRunForkHandler)
303 Determines whether the fork handlers should be enabled, making the
304 API fork safe.
305 XRD_REDIRECTLIMIT (-DIRedirectLimit)
307 Maximum number of allowed redirections.
308 XRD_NOTAUTHORIZEDRETRYLIMIT (-dNotAuthorizedRetryLimit)
310 Maximum number of allowed retries at a meta-manager for not-autho‐
311 rized error.
312 XRD_POLLERPREFERENCE (-DSPollerPreference)
314 A comma separated list of poller implementations in order of pref‐
315 erence. The default is: built-in.
316 XRD_CLIENTMONITOR (-DSClientMonitor)
318 Path to the client monitor library.
319 XRD_CLIENTMONITORPARAM (-DSClientMonitorParam)
321 Additional optional parameters that will be passed to the monitor‐
322 ing object on initialization.
323 XRD_WORKERTHREADS (-DIWorkerThreads)
325 Number of threads processing user callbacks.
326 XRD_CPPARALLELCHUNKS (-DICPParallelChunks)
328 Maximum number of asynchronous requests being processed by the
329 xrdcp command per connected channel substream (adjusted in real-
330 time).
331 XRD_CPCHUNKSIZE (-DICPChunkSize)
333 Size of a single data chunk handled by xrdcp.
334 XRD_NETWORKSTACK (-DSNetworkStack)
336 The network stack that the client should use to connect to the
337 server. Possible values are:
338 IPAuto - automatically detect which IP stack to use
340 IPAll - use IPv6 stack (AF_INET6 sockets) and both IPv6 and IPv4
342 (mapped to IPv6) addresses
343 IPv6 - use only IPv6 stack and addresses
345 IPv4 - use only IPv4 stack (AF_INET sockets) and addresses
347 IPv4Mapped6 - use IPv6 stack and mapped IPv4 addresses
349 XRD_DATASERVERTTL (-DIDataServerTTL)
351 Time period after which an idle connection to a data server should
352 be closed.
353 XRD_LOADBALANCERTTL (-DILoadBalancerTTL)
355 Time period after which an idle connection to a manager or a load
356 balancer should be closed.
357 XRD_APPNAME (-DSAppName)
359 Override the application name reported to the server.
360 XRD_PLUGINCONFDIR
362 A custom location containing client plug-in config files.
363 XRD_PLUGIN
365 A default client plug-in to be used.
366 XRD_CPINITTIMEOUT (-DICPInitTimeout)
368 Maximum time allowed for the copy process to initialize, ie. open
369 the source and destination files.
370 XRD_CPTPCTIMEOUT (-DICPTPCTimeout)
372 Maximum time allowed for a third-party copy operation to finish.
373 XRD_TCPKEEPALIVE (-DITCPKeepAlive)
375 Enable/Disable the TCP keep alive functionality
376 XRD_TCPKEEPALIVETIME (-DITCPKeepAliveTime)
378 Time between last data packet sent and the first keepalive probe
379 (Linux only)
380 XRD_TCPKEEPALIVEINTERVAL (-DITCPKeepAliveInterval)
382 Interval between subsequent keepalive probes (Linux only)
383 XRD_TCPKEEPALIVEPROBES (-DITCPKeepProbes)
385 Number of unacknowledged probes before considering the connection
386 dead (Linux only)
387 XRD_METALINKPROCESSING
389 Enable/Disable Metalink processing (enabled by default)
390 XRD_LOCALMETALINKFILE
392 Enable/Disable local Metalink file processing (by convention the
393 following URL schema has to be used: root://localfile//path/file‐
394 name.meta4) The 'localfile' semantic is now deprecated, use
395 file://localhost/path/filename.meta4 instead!
396 XRD_GLFNREDIRECTOR
398 The redirector will be used as a last resort if the GLFN tag is
399 specified in a Metalink file.
400 XRD_XCPBLOCKSIZE
402 Maximu size of a data block assigned to a single source in case of
403 an extreme copy transfer.
404 XRD_NODELAY
406 Disables the Nagle algorithm if set to 1 (default), enables it if
407 set to 0.
408 XRD_PREFERIPV4
410 If set the client tries first IPv4 address (turned off by de‐
411 fault).
412 XRD_MAXMETALINKWAIT
414 The maximum time in seconds a clinet can be stalled by the server
415 if a Metalink redirector is available (defaults to 60s).
416 XRD_PRESERVELOCATETRIED
418 If set to 1 XRootD client will preserve tired/triedrc cgi opaque
419 info for kXR_locate request accross redirects/retries, if set to 0
420 XRootD client will treat kXR_locate as any other passive request.
421 XRD_PRESERVEXATTRS
423 If set to 1 (default) xrdcp will preverve file extended attribues,
424 if set to 0 file extended attributes wont be preserved.
425 XRD_NOTLSOK
427 If set to 1 and the server is to old to support TLS encryption,
428 xrdcp will fallback to unencrypted transmission although
429 roots/xroots protocol was used. By default set to 0.
430 XRD_TLSNODATA
432 If set to 1, in case of roots/xroots protocol only the control
433 stream will be encrypted, data streams will be left unencrypted.
434 By default set to 0.
435 XRD_TLSMETALINK
437 If set to 1 all URLs in metalink will be treated as roots/xroots.
438 By default set to 0.
439 XRD_ZIPMTLNCKSUM
441 If set to 1, use the checksum available in a metalink file even if
442 a file is being extracted from a ZIP archive. By defaut set to 0;
443 XRD_CPTIMEOUT
445 Timeout for a classical (not TPC) copy job. By defaut set to 0
446 (disabled);
447 XRD_CLCONFDIR
449 User defined directory with config files (*.conf).
450 XRD_CLCONFFILE
452 User defined config file.
453
456 50 : generic error (e.g. config, internal, data, OS, command line op‐
457 tion)
458 51 : socket related error
460 52 : postmaster related error
462 53 : XRootD related error
464 54 : redirection error
466 55 : query response was negative (this is not an error)
468
471 Documentation for all components associated with xrdcp can be found at
472 http://xrootd.org/docs.html
473
476 Errors yield an error message and a non-zero exit status.
477
480 LGPL
481
484 The xrdcp command is supported by the xrootd collaboration. Contact
485 information can be found at:
486 http://xrootd.org/contact.html
488 v5.2.0 xrdcopy(1)