1xrdcopy(1) General Commands Manual xrdcopy(1)
2
6 xrdcp - copy files
7
9 xrdcp [options] source destination
10 options: [--cksum args] [--debug lvl]
12 [--coerce] [--force] [--help] [--license]
13 [--nopbar] [--posc] [--proxy ipaddr:port]
14 [--recursive] [--retry time] [--server]
15 [--silent] [--sources num] [--streams num]
16 [--tpc [delegate] first|only] [--verbose]
17 [--version] [--xrate rate] [--zip file]
18 [--dynamic-src] [--infiles fn]
19 [--parallel n] [--allow-http] [--xattr]
20 [--notlsok] [--tlsnodata] [--tlsmetalink]
21 [--zip-mtln-cksum] [--rm-bad-cksum] [--continue]
22 legacy options: [-adler] [-DSparm string] [-DIparm number]
24 [-md5] [-np] [-ODcgi] [-OScgi] [-x]
25
28 The xrdcp utility copies one or more files from one location to
29 another. The data source and destination may be a local or remote file
30 or directory. Additionally, the data source may also reside on multi‐
31 ple servers.
32
34 -C | --cksum type[:value|print|source]
35 obtains the checksum of type (i.e. adler32, crc32, md5 or zcrc32)
36 from the source, computes the checksum at the destination, and
37 verifies that they are the same. If auto is chosen as the checksum
38 type, xrdcp will try to automatically infer the right checksum
39 type based on source/destination configuration, source file type
40 (metalink, ZIP), and available checksum plug-ins. If a value is
41 specified, it is used as the source checksum. When print is spec‐
42 ified, the checksum at the destination is printed but is not veri‐
43 fied.
44 -d | --debug lvl
46 debug level: 1 (low), 2 (medium), 3 (high)
47 -F | --coerce
49 ignores locking semantics on the destination file. This option may
50 lead to file corruption if not properly used.
51 -f | --force
53 re-creates a file if it is already present.
54 -h | --help
56 displays usage information.
57 -H | --license
59 displays license terms and conditions.
60 -N | --nopbar
62 does not display the progress bar.
63 -P | --posc
65 requests POSC (persist-on-successful-close) processing to create a
66 new file. Files are automatically deleted should they not be suc‐
67 cessfully closed.
68 -D | --proxy proxyaddr:proxyport
70 [NOT YET IMPLEMENTED]
71 use proxyaddr:proxyport as a SOCKS4 proxy. Only numerical
73 addresses are supported.
74 -r | --recursive
76 recursively copy all files starting at the given source directory.
77 --server
79 runs as if in a server environment. Used only for server-side
80 third party copy support.
81 -s | --silent
83 neither produces summary information nor displays the progress
84 bar.
85 -y | --sources num
87 uses up to num sources to copy the file.
88 -S | --streams num
90 use num parallel data streams to do the transfer (the main stream
91 is in this case not used to carry out the transfer). The maximum
92 value is 15. The default is 0 (i.e., use only the control stream).
93 --tpc [delegate] first|only
95 copies the file from remote server to remote server using third-
96 party-copy protocol (i.e., data flows from server to server). The
97 source and destination servers must support third party copies.
98 Additional security restrictions may apply and may cause the copy
99 to fail if they cannot be satisfied. Argument 'first' tries tpc
100 and if it fails, does a normal copy; while 'only' fails the copy
101 unless tpc succeeds. When 'delegate' is specified, the copy dele‐
102 gates the command issuer's credentials to the target server which
103 uses those credentials to authenticate with the source server.
104 Delegation is ignored if the target server is not configured to
105 use delegated credentials. Currently, only gsi credentials can be
106 delegated.
107 -v | --verbose
109 displays summary output.
110 -V | -version
112 displays version information and immediately exits.
113 -z | --zip file
115 copy given file from a ZIP archive (same as xrdcl.unzip opaque
116 info).
117 -X | --xrate rate
119 limits the copy speed to the specified rate. The rate may be qual‐
120 ified with the letter k, m, or g to indicate kilo, mega, or giga
121 bytes, respectively. The option only applies when the source or
122 destination is local.
123 -Z | --dynamic-src
125 file size may change during the copy
126 -I | --infiles fn
128 specifies the file that contains a list of input files
129 --parallel n
131 number of copy jobs to be run simultaneously
132 --allow-http
134 allow HTTP as source or destination protocol. Requires the Xrd‐
135 ClHttp client plugin
136 --xattr
138 preserve extended attributes
139 --notlsok
141 If server is too old to support TLS encryption fallback to unen‐
142 crypted communication.
143 --tlsnodata
145 In case of roots/xroots protocol, encrypt only the control stream
146 and leave the data streams unencrypted.
147 --tlsmetalink
149 Treat all URLs in metalink as roots/xroots.
150 --zip-mtln-cksum
152 Use the checksum available in a metalink file even if a file is
153 being extracted from a ZIP archive.
154 --rm-bad-cksum
156 Remove the target file if checksum verification failed (enables
157 also POSC semantics).
158 --continue
160 Continue copying a file from the point where the previous copy was
161 interrupted.
162
165 Legacy options are provided for backward compatability. These are now
166 deprecated and should be avoided.
167 -adler
168 equivalent to "--cksum adler32:source".
169 -DIpname numberval
171 set the internal parameter pname with the numeric value numberval.
172 -DSpname stringval
174 set the internal parameter pname with the string value stringval.
175 -md5
177 equivalent to "--cksum md5:source".
178 -np
180 equivalent to "--nopbar".
181 -ODcgi
183 add cgi information cgi to any destination xrootd URL. You should
184 specify the opaque information directly on the destination URL.
185 -OScgi
187 add cgi information cgi to any source xrootd URL.
188 -x
190 equivalent to "--sources 12".
191
194 source
195 a dash (i.e. -) indicating stanard in, a local file, a local
196 directory name suffixed by /, or an xrootd URL in the form of
197 xroot://[user@]host[:port]/absolutepath
198 The absolutepath can be a directory.
199 destination
201 a dash (i.e. -) indicating stanard out, a local file, a local
202 directory name suffixed by /, or an xrootd URL in the form
203 xroot://[user@]host[:port]/absolutepath
204 The absolutepath can be a directory.
205
209 The following environment variables are supported. They apply to xrdfs
210 and any other application using the libXrdCl library, unless specified
211 otherwise. The text in the brackets is a name of the corresponding
212 xrdcp commandline parameter.
213 XRD_LOGLEVEL
215 Detemines the amout of diagnostics that should be printed. Valid
216 values are: Dump, Debug, Info, Warning, and Error.
217 XRD_LOGFILE
219 If set, the diagnostics will be printed to the specified file
220 instead of stderr.
221 XRD_LOGMASK
223 Determines which diagnostics topics should be printed at all lev‐
224 els. It's a "|" separated list of topics. The first element may be
225 "All" in which case all the topics are enabled and the subsequent
226 elements may turn them off, or "None" in which case all the topics
227 are disabled and the subsequent flags may turn them on. If the
228 topic name is prefixed with "^", then it means that the topic
229 should be disabled. If the topic name is not prefixed, then it
230 means that the topic should be enabled.
231 The log mask may as well be handled for each diagnostic level sep‐
233 arately by setting one or more of the following variables:
234 XRD_LOGMASK_ERROR, XRD_LOGMASK_WARNING, XRD_LOGMASK_INFO, XRD_LOG‐
235 MASK_DEBUG, and XRD_LOGMASK_DUMP. The default for each level is
236 "All", except for the Dump level, where the default is
237 "All|^PollerMsg". This means that, at the Dump level, all the top‐
238 ics but "PollerMsg" are enabled.
239 Available topics: AppMsg, UtilityMsg, FileMsg, PollerMsg, PostMas‐
241 terMsg, XRootDTransportMsg, TaskMgrMsg, XRootDMsg, FileSystemMsg,
242 AsyncSockMsg
243 XRD_TLSDBGLVL
245 Determine the debug level for the TLS component. Valid values are:
246 OFF, CTX, SOK, SIO, ALL and OUT. The default value is OFF.
247 XRD_PARALLELEVTLOOP
249 The number of event loops.
250 XRD_READRECOVERY
252 Determines if read recovery should be enabled or disabled (enabled
253 by default).
254 XRD_WRITERECOVERY
256 Determines if write recovery should be enabled or disabled
257 (enabled by default).
258 XRD_OPENRECOVERY
260 Determines if open recovery should be enabled or disabled for
261 mutable (truncate or create) opens (enabled by default).
262 XRD_CONNECTIONWINDOW (-DIConnectionWindow)
264 A time window for the connection establishment. A connection fail‐
265 ure is declared if the connection is not established within the
266 time window. If a connection failure happens earlier then another
267 connection attempt will only be made at the beginning of the next
268 window.
269 XRD_CONNECTIONRETRY (-DIConnectionRetry)
271 Number of connection attempts that should be made (number of
272 available connection windows) before declaring a permanent fail‐
273 ure.
274 XRD_REQUESTTIMEOUT (-DIRequestTimeout)
276 Default value for the time after which an error is declared if it
277 was impossible to get a response to a request.
278 XRD_STREAMTIMEOUT (-DIStreamTimeout)
280 Default value for the time after which a connection error is
281 declared (and a recovery attempted) if there are unfulfilled
282 requests and there is no socket activity or a registered wait
283 timeout.
284 XRD_SUBSTREAMSPERCHANNEL (-DISubStreamsPerChannel)
286 Number of streams per session.
287 XRD_TIMEOUTRESOLUTION (-DITimeoutResolution)
289 Resolution for the timeout events. Ie. timeout events will be pro‐
290 cessed only every XRD_TIMEOUTRESOLUTION seconds.
291 XRD_STREAMERRORWINDOW (-DIStreamErrorWindow)
293 Time after which the permanent failure flags are cleared out and a
294 new connection may be attempted if needed.
295 XRD_RUNFORKHANDLER (-DIRunForkHandler)
297 Determines whether the fork handlers should be enabled, making the
298 API fork safe.
299 XRD_REDIRECTLIMIT (-DIRedirectLimit)
301 Maximum number of allowed redirections.
302 XRD_NOTAUTHORIZEDRETRYLIMIT (-dNotAuthorizedRetryLimit)
304 Maximum number of allowed retries at a meta-manager for not-autho‐
305 rized error.
306 XRD_POLLERPREFERENCE (-DSPollerPreference)
308 A comma separated list of poller implementations in order of pref‐
309 erence. The default is: built-in.
310 XRD_CLIENTMONITOR (-DSClientMonitor)
312 Path to the client monitor library.
313 XRD_CLIENTMONITORPARAM (-DSClientMonitorParam)
315 Additional optional parameters that will be passed to the monitor‐
316 ing object on initialization.
317 XRD_WORKERTHREADS (-DIWorkerThreads)
319 Number of threads processing user callbacks.
320 XRD_CPPARALLELCHUNKS (-DICPParallelChunks)
322 Maximum number of asynchronous requests being processed by the
323 xrdcp command per connected channel substream (adjusted in real-
324 time).
325 XRD_CPCHUNKSIZE (-DICPChunkSize)
327 Size of a single data chunk handled by xrdcp.
328 XRD_NETWORKSTACK (-DSNetworkStack)
330 The network stack that the client should use to connect to the
331 server. Possible values are:
332 IPAuto - automatically detect which IP stack to use
334 IPAll - use IPv6 stack (AF_INET6 sockets) and both IPv6 and IPv4
336 (mapped to IPv6) addresses
337 IPv6 - use only IPv6 stack and addresses
339 IPv4 - use only IPv4 stack (AF_INET sockets) and addresses
341 IPv4Mapped6 - use IPv6 stack and mapped IPv4 addresses
343 XRD_DATASERVERTTL (-DIDataServerTTL)
345 Time period after which an idle connection to a data server should
346 be closed.
347 XRD_LOADBALANCERTTL (-DILoadBalancerTTL)
349 Time period after which an idle connection to a manager or a load
350 balancer should be closed.
351 XRD_APPNAME (-DSAppName)
353 Override the application name reported to the server.
354 XRD_PLUGINCONFDIR
356 A custom location containing client plug-in config files.
357 XRD_PLUGIN
359 A default client plug-in to be used.
360 XRD_CPINITTIMEOUT (-DICPInitTimeout)
362 Maximum time allowed for the copy process to initialize, ie. open
363 the source and destination files.
364 XRD_CPTPCTIMEOUT (-DICPTPCTimeout)
366 Maximum time allowed for a third-party copy operation to finish.
367 XRD_TCPKEEPALIVE (-DITCPKeepAlive)
369 Enable/Disable the TCP keep alive functionality
370 XRD_TCPKEEPALIVETIME (-DITCPKeepAliveTime)
372 Time between last data packet sent and the first keepalive probe
373 (Linux only)
374 XRD_TCPKEEPALIVEINTERVAL (-DITCPKeepAliveInterval)
376 Interval between subsequent keepalive probes (Linux only)
377 XRD_TCPKEEPALIVEPROBES (-DITCPKeepProbes)
379 Number of unacknowledged probes before considering the connection
380 dead (Linux only)
381 XRD_METALINKPROCESSING
383 Enable/Disable Metalink processing (enabled by default)
384 XRD_LOCALMETALINKFILE
386 Enable/Disable local Metalink file processing (by convention the
387 following URL schema has to be used: root://localfile//path/file‐
388 name.meta4) The 'localfile' semantic is now deprecated, use
389 file://localhost/path/filename.meta4 instead!
390 XRD_GLFNREDIRECTOR
392 The redirector will be used as a last resort if the GLFN tag is
393 specified in a Metalink file.
394 XRD_XCPBLOCKSIZE
396 Maximu size of a data block assigned to a single source in case of
397 an extreme copy transfer.
398 XRD_NODELAY
400 Disables the Nagle algorithm if set to 1 (default), enables it if
401 set to 0.
402 XRD_PREFERIPV4
404 If set the client tries first IPv4 address (turned off by
405 default).
406 XRD_MAXMETALINKWAIT
408 The maximum time in seconds a clinet can be stalled by the server
409 if a Metalink redirector is available (defaults to 60s).
410 XRD_PRESERVELOCATETRIED
412 If set to 1 XRootD client will preserve tired/triedrc cgi opaque
413 info for kXR_locate request accross redirects/retries, if set to 0
414 XRootD client will treat kXR_locate as any other passive request.
415 XRD_PRESERVEXATTRS
417 If set to 1 (default) xrdcp will preverve file extended attribues,
418 if set to 0 file extended attributes wont be preserved.
419 XRD_NOTLSOK
421 If set to 1 and the server is to old to support TLS encryption,
422 xrdcp will fallback to unencrypted transmission although
423 roots/xroots protocol was used. By default set to 0.
424 XRD_TLSNODATA
426 If set to 1, in case of roots/xroots protocol only the control
427 stream will be encrypted, data streams will be left unencrypted.
428 By default set to 0.
429 XRD_TLSMETALINK
431 If set to 1 all URLs in metalink will be treated as roots/xroots.
432 By default set to 0.
433 XRD_ZIPMTLNCKSUM
435 If set to 1, use the checksum available in a metalink file even if
436 a file is being extracted from a ZIP archive. By defaut set to 0;
437
440 50 : generic error (e.g. config, internal, data, OS, command line
441 option)
442 51 : socket related error
444 52 : postmaster related error
446 53 : XRootD related error
448 54 : redirection error
450 55 : query response was negative (this is not an error)
452
455 Documentation for all components associated with xrdcp can be found at
456 http://xrootd.org/docs.html
457
460 Errors yield an error message and a non-zero exit status.
461
464 LGPL
465
468 The xrdcp command is supported by the xrootd collaboration. Contact
469 information can be found at:
470 http://xrootd.org/contact.html
472 v5.1.1 xrdcopy(1)