1xrdcopy(1)                  General Commands Manual                 xrdcopy(1)
2
3
4

NAME

6       xrdcp - copy files
7

SYNOPSIS

9       xrdcp [options] source destination
10
11       options: [--cksum args] [--debug lvl]
12       [--coerce] [--force] [--help] [--license]
13       [--nopbar] [--posc] [--proxy ipaddr:port]
14       [--recursive] [--retry time] [--server]
15       [--silent] [--sources num] [--streams num]
16       [--tpc [delegate] first|only] [--verbose]
17       [--version] [--xrate rate] [--zip file]
18       [--dynamic-src] [--infiles fn]
19       [--parallel n] [--allow-http] [--xattr]
20       [--notlsok] [--tlsnodata] [--tlsmetalink]
21       [--zip-mtln-cksum] [--rm-bad-cksum] [--continue]
22       [--xrate-threshold] [--retry-policy]
23
24       legacy options: [-adler] [-DSparm string] [-DIparm number]
25       [-md5] [-np] [-ODcgi] [-OScgi] [-x]
26
27

DESCRIPTION

29       The xrdcp utility copies one or more files from one location to an‐
30       other. The data source and destination may be a local or remote file or
31       directory.  Additionally, the data source may also reside on multiple
32       servers.
33

OPTIONS

35       -C | --cksum type[:value|print|source]
36            obtains the checksum of type (i.e. adler32, crc32, md5 or zcrc32)
37            from the source, computes the checksum at the destination, and
38            verifies that they are the same. If auto is chosen as the checksum
39            type, xrdcp will try to automatically infer the right checksum
40            type based on source/destination configuration, source file type
41            (metalink, ZIP), and available checksum plug-ins. If a value is
42            specified, it is used as the source checksum.  When print is spec‐
43            ified, the checksum at the destination is printed but is not veri‐
44            fied.
45
46       -d | --debug lvl
47            debug level: 1 (low), 2 (medium), 3 (high)
48
49       -F | --coerce
50            ignores locking semantics on the destination file. This option may
51            lead to file corruption if not properly used.
52
53       -f | --force
54            re-creates a file if it is already present.
55
56       -h | --help
57            displays usage information.
58
59       -H | --license
60            displays license terms and conditions.
61
62       -N | --nopbar
63            does not display the progress bar.
64
65       -P | --posc
66            requests POSC (persist-on-successful-close) processing to create a
67            new file. Files are automatically deleted should they not be suc‐
68            cessfully closed.
69
70       -D | --proxy proxyaddr:proxyport
71            [NOT YET IMPLEMENTED]
72
73            use proxyaddr:proxyport as a SOCKS4 proxy. Only numerical ad‐
74            dresses are supported.
75
76       -r | --recursive
77            recursively copy all files starting at the given source directory.
78
79       --retry
80            retry failed copy-jobs.
81
82       --server
83            runs as if in a server environment. Used only for server-side
84            third party copy support.
85
86       -s | --silent
87            neither produces summary information nor displays the progress
88            bar.
89
90       -y | --sources num
91            uses up to num sources to copy the file.
92
93       -S | --streams num
94            use num parallel data streams to do the transfer (the main stream
95            is in this case not used to carry out the transfer).  The maximum
96            value is 15. The default is 0 (i.e., use only the control stream).
97
98       --tpc [delegate] first|only
99            copies the file from remote server to remote server using third-
100            party-copy protocol (i.e., data flows from server to server). The
101            source and destination servers must support third party copies.
102            Additional security restrictions may apply and may cause the copy
103            to fail if they cannot be satisfied.  Argument 'first' tries tpc
104            and if it fails, does a normal copy; while 'only' fails the copy
105            unless tpc succeeds. When 'delegate' is specified, the copy dele‐
106            gates the command issuer's credentials to the target server which
107            uses those credentials to authenticate with the source server.
108            Delegation is ignored if the target server is not configured to
109            use delegated credentials. Currently, only gsi credentials can be
110            delegated.
111
112       -v | --verbose
113            displays summary output.
114
115       -V | -version
116            displays version information and immediately exits.
117
118       -z | --zip file
119            copy given file from a ZIP archive (same as xrdcl.unzip opaque
120            info).
121
122       -X | --xrate rate
123            limits the copy speed to the specified rate. The rate may be qual‐
124            ified with the letter k, m, or g to indicate kilo, mega, or giga
125            bytes, respectively. The option only applies when the source or
126            destination is local.
127
128       -X | --xrate-threshold rate
129            If the transfer rate drops bellow given threshold force the client
130            to use different source or if no more sources are available fail
131            the transfer.
132
133       -Z | --dynamic-src
134            file size may change during the copy
135
136       -I | --infiles fn
137            specifies the file that contains a list of input files
138
139       --parallel n
140            number of copy jobs to be run simultaneously
141
142       --allow-http
143            allow HTTP as source or destination protocol. Requires the Xrd‐
144            ClHttp client plugin
145
146       --xattr
147            preserve extended attributes
148
149       --notlsok
150            If server is too old to support TLS encryption fallback to unen‐
151            crypted communication.
152
153       --tlsnodata
154            In case of roots/xroots protocol, encrypt only the control stream
155            and leave the data streams unencrypted.
156
157       --tlsmetalink
158            Treat all URLs in metalink as roots/xroots.
159
160       --zip-mtln-cksum
161            Use the checksum available in a metalink file even if a file is
162            being extracted from a ZIP archive.
163
164       --rm-bad-cksum
165            Remove the target file if checksum verification failed (enables
166            also POSC semantics).
167
168       --continue
169            Continue copying a file from the point where the previous copy was
170            interrupted.
171
172

LEGACY OPTIONS

174       Legacy options are provided for backward compatability. These are now
175       deprecated and should be avoided.
176       -adler
177            equivalent to "--cksum adler32:source".
178
179       -DIpname numberval
180            set the internal parameter pname with the numeric value numberval.
181
182       -DSpname stringval
183            set the internal parameter pname with the string value stringval.
184
185       -md5
186            equivalent to "--cksum md5:source".
187
188       -np
189            equivalent to "--nopbar".
190
191       -ODcgi
192            add cgi information cgi to any destination xrootd URL.  You should
193            specify the opaque information directly on the destination URL.
194
195       -OScgi
196            add cgi information cgi to any source xrootd URL.
197
198       -x
199            equivalent to "--sources 12".
200
201

OPERANDS

203       source
204            a dash (i.e. -) indicating stanard in, a local file, a local di‐
205            rectory name suffixed by /, or an xrootd URL in the form of
206                         xroot://[user@]host[:port]/absolutepath
207            The absolutepath can be a directory.
208
209       destination
210            a dash (i.e. -) indicating stanard out, a local file, a local di‐
211            rectory name suffixed by /, or an xrootd URL in the form
212                         xroot://[user@]host[:port]/absolutepath
213            The absolutepath can be a directory.
214
215
216

ENVIRONMENT

218       The following environment variables are supported. They apply to xrdfs
219       and any other application using the libXrdCl library, unless specified
220       otherwise. The text in the brackets is a name of the corresponding
221       xrdcp commandline parameter.
222
223       XRD_LOGLEVEL
224            Detemines the amout of diagnostics that should be printed. Valid
225            values are: Dump, Debug, Info, Warning, and Error.
226
227       XRD_LOGFILE
228            If set, the diagnostics will be printed to the specified file in‐
229            stead of stderr.
230
231       XRD_LOGMASK
232            Determines which diagnostics topics should be printed at all lev‐
233            els. It's a "|" separated list of topics. The first element may be
234            "All" in which case all the topics are enabled and the subsequent
235            elements may turn them off, or "None" in which case all the topics
236            are disabled and the subsequent flags may turn them on. If the
237            topic name is prefixed with "^", then it means that the topic
238            should be disabled. If the topic name is not prefixed, then it
239            means that the topic should be enabled.
240
241            The log mask may as well be handled for each diagnostic level sep‐
242            arately by setting one or more of the following variables:
243            XRD_LOGMASK_ERROR, XRD_LOGMASK_WARNING, XRD_LOGMASK_INFO, XRD_LOG‐
244            MASK_DEBUG, and XRD_LOGMASK_DUMP. The default for each level is
245            "All", except for the Dump level, where the default is
246            "All|^PollerMsg". This means that, at the Dump level, all the top‐
247            ics but "PollerMsg" are enabled.
248
249            Available topics: AppMsg, UtilityMsg, FileMsg, PollerMsg, PostMas‐
250            terMsg, XRootDTransportMsg, TaskMgrMsg, XRootDMsg, FileSystemMsg,
251            AsyncSockMsg, TlsMsg
252
253       XRD_TLSDBGLVL
254            Determine the debug level for the TLS component. Valid values are:
255            OFF, CTX, SOK, SIO, ALL and OUT.  The default value is OFF.
256
257       XRD_PARALLELEVTLOOP
258            The number of event loops.
259
260       XRD_READRECOVERY
261            Determines if read recovery should be enabled or disabled (enabled
262            by default).
263
264       XRD_WRITERECOVERY
265            Determines if write recovery should be enabled or disabled (en‐
266            abled by default).
267
268       XRD_OPENRECOVERY
269            Determines if open recovery should be enabled or disabled for mu‐
270            table (truncate or create) opens (enabled by default).
271
272       XRD_CONNECTIONWINDOW (-DIConnectionWindow)
273            A time window for the connection establishment. A connection fail‐
274            ure is declared if the connection is not established within the
275            time window. If a connection failure happens earlier then another
276            connection attempt will only be made at the beginning of the next
277            window.
278
279       XRD_CONNECTIONRETRY (-DIConnectionRetry)
280            Number of connection attempts that should be made (number of
281            available connection windows) before declaring a permanent fail‐
282            ure.
283
284       XRD_REQUESTTIMEOUT (-DIRequestTimeout)
285            Default value for the time after which an error is declared if it
286            was impossible to get a response to a request.
287
288       XRD_STREAMTIMEOUT (-DIStreamTimeout)
289            Default value for the time after which a connection error is de‐
290            clared (and a recovery attempted) if there are unfulfilled re‐
291            quests and there is no socket activity or a registered wait time‐
292            out.
293
294       XRD_SUBSTREAMSPERCHANNEL (-DISubStreamsPerChannel)
295            Number of streams per session.
296
297       XRD_TIMEOUTRESOLUTION (-DITimeoutResolution)
298            Resolution for the timeout events. Ie. timeout events will be pro‐
299            cessed only every XRD_TIMEOUTRESOLUTION seconds.
300
301       XRD_STREAMERRORWINDOW (-DIStreamErrorWindow)
302            Time after which the permanent failure flags are cleared out and a
303            new connection may be attempted if needed.
304
305       XRD_RUNFORKHANDLER (-DIRunForkHandler)
306            Determines whether the fork handlers should be enabled, making the
307            API fork safe.
308
309       XRD_REDIRECTLIMIT (-DIRedirectLimit)
310            Maximum number of allowed redirections.
311
312       XRD_NOTAUTHORIZEDRETRYLIMIT (-dNotAuthorizedRetryLimit)
313            Maximum number of allowed retries at a meta-manager for not-autho‐
314            rized error.
315
316       XRD_POLLERPREFERENCE (-DSPollerPreference)
317            A comma separated list of poller implementations in order of pref‐
318            erence. The default is: built-in.
319
320       XRD_CLIENTMONITOR (-DSClientMonitor)
321            Path to the client monitor library.
322
323       XRD_CLIENTMONITORPARAM (-DSClientMonitorParam)
324            Additional optional parameters that will be passed to the monitor‐
325            ing object on initialization.
326
327       XRD_WORKERTHREADS (-DIWorkerThreads)
328            Number of threads processing user callbacks.
329
330       XRD_CPPARALLELCHUNKS (-DICPParallelChunks)
331            Maximum number of asynchronous requests being processed by the
332            xrdcp command per connected channel substream (adjusted in real-
333            time).
334
335       XRD_CPCHUNKSIZE (-DICPChunkSize)
336            Size of a single data chunk handled by xrdcp.
337
338       XRD_NETWORKSTACK (-DSNetworkStack)
339            The network stack that the client should use to connect to the
340            server. Possible values are:
341
342            IPAuto - automatically detect which IP stack to use
343
344            IPAll - use IPv6 stack (AF_INET6 sockets) and both IPv6 and IPv4
345            (mapped to IPv6) addresses
346
347            IPv6 - use only IPv6 stack and addresses
348
349            IPv4 - use only IPv4 stack (AF_INET sockets) and addresses
350
351            IPv4Mapped6 - use IPv6 stack and mapped IPv4 addresses
352
353       XRD_DATASERVERTTL (-DIDataServerTTL)
354            Time period after which an idle connection to a data server should
355            be closed.
356
357       XRD_LOADBALANCERTTL (-DILoadBalancerTTL)
358            Time period after which an idle connection to a manager or a load
359            balancer should be closed.
360
361       XRD_APPNAME (-DSAppName)
362            Override the application name reported to the server.
363
364       XRD_PLUGINCONFDIR
365            A custom location containing client plug-in config files.
366
367       XRD_PLUGIN
368            A default client plug-in to be used.
369
370       XRD_CPINITTIMEOUT (-DICPInitTimeout)
371            Maximum time allowed for the copy process to initialize, ie. open
372            the source and destination files.
373
374       XRD_CPTPCTIMEOUT (-DICPTPCTimeout)
375            Maximum time allowed for a third-party copy operation to finish.
376
377       XRD_TCPKEEPALIVE (-DITCPKeepAlive)
378            Enable/Disable the TCP keep alive functionality
379
380       XRD_TCPKEEPALIVETIME (-DITCPKeepAliveTime)
381            Time between last data packet sent and the first keepalive probe
382            (Linux only)
383
384       XRD_TCPKEEPALIVEINTERVAL (-DITCPKeepAliveInterval)
385            Interval between subsequent keepalive probes (Linux only)
386
387       XRD_TCPKEEPALIVEPROBES (-DITCPKeepProbes)
388            Number of unacknowledged probes before considering the connection
389            dead (Linux only)
390
391       XRD_METALINKPROCESSING
392            Enable/Disable Metalink processing (enabled by default)
393
394       XRD_LOCALMETALINKFILE
395            Enable/Disable local Metalink file processing (by convention the
396            following URL schema has to be used: root://localfile//path/file‐
397            name.meta4) The 'localfile' semantic is now deprecated, use
398            file://localhost/path/filename.meta4 instead!
399
400       XRD_GLFNREDIRECTOR
401            The redirector will be used as a last resort if the GLFN tag is
402            specified in a Metalink file.
403
404       XRD_XCPBLOCKSIZE
405            Maximu size of a data block assigned to a single source in case of
406            an extreme copy transfer.
407
408       XRD_NODELAY
409            Disables the Nagle algorithm if set to 1 (default), enables it if
410            set to 0.
411
412       XRD_PREFERIPV4
413            If set the client tries first IPv4 address (turned off by de‐
414            fault).
415
416       XRD_MAXMETALINKWAIT
417            The maximum time in seconds a clinet can be stalled by the server
418            if a Metalink redirector is available (defaults to 60s).
419
420       XRD_PRESERVELOCATETRIED
421            If set to 1 XRootD client will preserve tired/triedrc cgi opaque
422            info for kXR_locate request accross redirects/retries, if set to 0
423            XRootD client will treat kXR_locate as any other passive request.
424
425       XRD_PRESERVEXATTRS
426            If set to 1 (default) xrdcp will preverve file extended attribues,
427            if set to 0 file extended attributes wont be preserved.
428
429       XRD_NOTLSOK
430            If set to 1 and the server is to old to support TLS encryption,
431            xrdcp will fallback to unencrypted transmission although
432            roots/xroots protocol was used.  By default set to 0.
433
434       XRD_TLSNODATA
435            If set to 1, in case of roots/xroots protocol only the control
436            stream will be encrypted, data streams will be left unencrypted.
437            By default set to 0.
438
439       XRD_TLSMETALINK
440            If set to 1 all URLs in metalink will be treated as roots/xroots.
441            By default set to 0.
442
443       XRD_ZIPMTLNCKSUM
444            If set to 1, use the checksum available in a metalink file even if
445            a file is being extracted from a ZIP archive.  By defaut set to 0;
446
447       XRD_CPTIMEOUT
448            Timeout for a classical (not TPC) copy job.  By defaut set to 0
449            (disabled);
450
451       XRD_CLCONFDIR
452            User defined directory with config files (*.conf).
453
454       XRD_CLCONFFILE
455            User defined config file.
456
457       XRD_CPRETRY
458            Maximum number of times to retry failed copy-jobs (default: 0).
459
460       XRD_CPRETRYPOLICY
461            Copy job retry policy, either force or retry (default: force).
462
463

RETURN CODES

465       50  : generic error (e.g. config, internal, data, OS, command line op‐
466       tion)
467
468       51  : socket related error
469
470       52  : postmaster related error
471
472       53  : XRootD related error
473
474       54  : redirection error
475
476       55  : query response was negative (this is not an error)
477
478

NOTES

480       Documentation for all components associated with xrdcp can be found at
481       http://xrootd.org/docs.html
482
483

DIAGNOSTICS

485       Errors yield an error message and a non-zero exit status.
486
487

LICENSE

489       LGPL
490
491

SUPPORT LEVEL

493       The xrdcp command is supported by the xrootd collaboration.  Contact
494       information can be found at:
495
496                           http://xrootd.org/contact.html
497
498
499
500                                    v5.3.3                          xrdcopy(1)
Impressum