1GLOBUS-URL-COPY(1) Grid Community Toolkit Manual GLOBUS-URL-COPY(1)
2
6 globus-url-copy - globus-url-copy
7
9 globus-url-copy [options] SOURCE-URL DESTINATION-URL
10
12 The globus-url-copy program is a command line tool for multi-protocol
13 data movement. It supports gsiftp:// (GridFTP), ftp://, http://,
14 https://, sshftp:// and file:/// protocol specifiers in the URL.
15
17 -help, -usage
18 Print help.
19 -versions
21 Print the versions of all modules that this program uses
22 -c, -continue-on-error
24 Do not die after any errors. By default, program will exit after
25 most errors.
26 -a, -ascii
28 Convert the file to/from ASCII format to/from local file format
29 -b, -binary
31 Do not apply any conversion to the files. default
32 -f FILENAME
34 Read a list of URL pairs from filename. Each line should contain
35 sourceURL destURL. Enclose URLs with spaces in double qoutes (").
36 Blank lines and lines beginning with # will be ignored.
37 -cd, -create-dest
39 Create destination directory if needed.
40 -r
42 Copy files in subdirectories
43 -fast
45 Recommended when using GridFTP servers. Use MODE E for all data
46 transfers, including reusing data channels between list and
47 transfer operations.
48 -t SECONDS
50 Run the transfer for this number of seconds and then end. Useful
51 for performance testing or forced restart loops.
52 -q, -quiet
54 Suppress all output for successful operation.
55 -v, -verbose
57 Display URLs being transferred
58 -vb, -verbose-perf
60 During the transfer, display the number of bytes transferred and
61 the transfer rate per second. Show URLs being transferred
62 -dbg, -debugftp
64 Debug ftp connections. Prints control channel communication to
65 stderr
66 -rst, -restart
68 Restart failed ftp operations.
69 -rst-retries RETRIES
71 The maximum number of times to retry the operation before giving up
72 on the transfer. Use 0 for infinite. Default is 5.
73 -rst-interval SECONDS
75 The interval in seconds to wait after a failure before retrying the
76 transfer. Use 0 for an exponential backoff. Default is 0.
77 -rst-timeout SECONDS
79 Maximum time after a failure to keep retrying. Use 0 for no
80 timeout. Default is 0.
81 -stall-timeout SECONDS, -st SECONDS
83 How long before cancelling/restarting a transfer with no data
84 movement. Set to 0 to disable. Default is 600 seconds.
85 -df FILENAME, -dumpfile FILENAME
87 Path to a file where untransferred URLs will be saved for later
88 restarting. Resulting file is the same format as the -f input file.
89 If file exists, it will be read and all other URL input will be
90 ignored.
91 -do FILENAME, -dump-only FILENAME
93 Perform no write operations on the destination. Instead, all files
94 that would be transferred are enumerated and dumped to the
95 specified file. Resulting file is the same format as the -f input
96 file. Note: if you intend to use this file as input for a future
97 transfer, the -create-dest option will be required if any
98 destination directories do not already exist.
99 -rp, -relative-paths
101 The path portion of ftp URLs will be interpreted as relative to the
102 user’s starting directory on the server. By default, all paths are
103 root-relative. When this flag is set, the path portion of the ftp
104 URL must start with %2F if it designates a root-relative path.
105 -s SUBJECT, -subject SUBJECT
107 Use this subject to match with both the source and dest servers.
108 -ss SUBJECT, -source-subject SUBJECT
110 Use this subject to match with the source server
111 -ds SUBJECT, -dest-subject SUBJECT
113 Use this subject to match with the destination server.
114 -tcp-bs SIZE, -tcp-buffer-size SIZE
116 Specify the size (in bytes) of the buffer to be used by the
117 underlying ftp data channels.
118 -bs block SIZE, -block-size block SIZE
120 Specify the size (in bytes) of the buffer to be used by the
121 underlying transfer methods.
122 -p PARALLELISM, -parallel PARALLELISM
124 Specify the number of parallel data connections should be used.
125 -notpt, -no-third-party-transfers
127 Turn third-party transfers off (on by default).
128 -nodcau, -no-data-channel-authentication
130 Turn off data channel authentication for ftp transfers.
131 -dcsafe, -data-channel-safe
133 Set data channel protection mode to SAFE
134 -dcpriv, -data-channel-private
136 Set data channel protection mode to PRIVATE
137 -off, -partial-offset
139 Offset for partial ftp file transfers, defaults to 0.
140 -len, -partial-length
142 Length for partial ftp file transfers, used only for the source
143 url, defaults the full file.
144 -list URL
146 List the files located at URL.
147 -stripe
149 Enable striped transfers on supported servers.
150 -striped-block-size, -sbs
152 Set layout mode and block size for striped transfers. If not set,
153 server defaults will be used. If set to 0, Partitioned mode will be
154 used. If set to > 0, Blocked mode will be used, with this as the
155 block size.
156 -ipv6
158 Use ipv6 when available (EXPERIMENTAL)
159 -udt
161 Use UDT, a reliable udp based transport protocol, for data
162 transfers
163 -g2, -gridftp2
165 Use GridFTP v2 protocol enhancements when possible.
166 -dp, -delayed-pasv
168 Enable delayed passive.
169 -mn NAME, -module-name NAME
171 Set the back-end storage module to use for both the source and
172 destination in a GridFTP transfer.
173 -mp PARAMETERS, -module-parameters PARAMETERS
175 Set the back-end storage module arguments to use for both the
176 source and destination in a GridFTP transfer.
177 -smn NAME, -src-module-name NAME
179 Set the back-end storage module to use for the source in a GridFTP
180 transfer.
181 -smp PARAMETERS, -src-module-parameters PARAMETERS
183 Set the back-end storage module arguments to use for the source in
184 a GridFTP transfer.
185 -dmn NAME, -dst-module-name NAME
187 Set the back-end storage module to use for the destination in a
188 GridFTP transfer.
189 -dmp PARAMETERS, -dst-module-parameters PARAMETERS
191 Set the back-end storage module arguments to use for the
192 destination in a GridFTP transfer.
193 -aa FILE, -authz-assert FILE
195 Use the assertions in FILE to authorize the access with both source
196 and destination servers.
197 -saa FILE, -src-authz-assert FILE
199 Use the assertions in this file to authorize the access with source
200 server.
201 -daa FILE, -dst-authz-assert FILE
203 Use the assertions in this file to authorize the access with dest
204 server.
205 -cache-aa, -cache-authz-assert
207 Cache the authz assertion for subsequent transfers.
208 -cache-saa, -cache-src-authz-assert
210 Cache the src authz assertion for subsequent transfers.
211 -cache-daa, -cache-dst-authz-assert
213 Cache the dst authz assertion for subsequent transfers.
214 -pipeline, -pp
216 Enable pipelining support for multi-file ftp transfers. Currently
217 third-party transfers benefit from this. EXPERIMENTAL
218 -concurrency, -cc
220 Number of concurrent ftp connections to use for multiple transfers.
221 -nl-bottleneck, -nlb
223 Use NetLogger to estimate speeds of disk and network read/write
224 system calls, and attempt to determine the bottleneck component.
225 -sp COMMANDS, -src-pipe COMMANDS
227 Set the source end of a remote transfer to use piped in input with
228 the given command line. Do not use with -fsstack.
229 -DP COMMANDS, -dst-pipe COMMANDS
231 Set the destination end of a remote transfer to write data to then
232 standard input of the program run via the given command line. Do
233 not use with -fsstack.
234 -pipe COMMANDS
236 Sets both -src-pipe and -dst-pipe to the same thing.
237 -dcstack STACK, -data-channel-stack STACK
239 Set the XIO driver stack for the network on both the source and the
240 destination. Both must be GridFTP servers. The stack should contain
241 all network drivers to use, in the order specified from bottom to
242 top (e.g. -dcstack tcp,gsi). If the gsi driver is not included in
243 the stack and data channel authentication is enabled, it will be
244 inserted above the transport driver in the stack.
245 -fsstack STACK, -file-system-stack STACK
247 Set the XIO driver stack for the disk on both the source and the
248 destination. Both must be GridFTP servers. The stack should contain
249 all file system drivers to use, in the order specified from bottom
250 to top.
251 -src-dcstack STACK, -source-data-channel-stack STACK
253 Set the XIO driver stack for the network on the source GridFTP
254 server. See -dcstack above for description of the STACK string.
255 -src-fsstack STACK, -source-file-system-stack STACK
257 Set the XIO driver stack for the disk on the source GridFTP server.
258 See -fsstack above for description of the STACK string.
259 -dst-dcstack STACK, -dest-data-channel-stack STACK
261 Set the XIO driver stack for the network on the destination GridFTP
262 server. See -dcstack above for description of the STACK string.
263 -dst-fsstack STACK, -dest-file-system-stack STACK
265 Set the XIO driver stack for the disk on the destination GridFTP
266 server. See -fsstack above for description of the STACK string.
267 -cred PATH
269 Set the credentials to use for both ftp connections.
270 -src-cred CRED-FILE, -sc CRED-FILE
272 Set the credentials to use for source ftp connections.
273 -dst-cred CRED-FILE, -dc CRED-FILE
275 Set the credentials to use for destination ftp connections.
276 -af FILENAME, -alias-file FILENAME
278 File with mapping of logical host aliases to lists of physical
279 hosts. When used with multiple concurrent connections, each
280 connection uses the next host in the list. Each line should either
281 be an alias, noted with the @ symbol, or a hostname[:port].
282 Currently, only the aliases @source and @destination are valid, and
283 they are used for every source or destination URL.
284 -sync
286 Only transfer files where the destination does not exist or differs
287 from the source. -sync-level controls how to determine if files
288 differ.
289 -sync-level number
291 Criteria for determining if files differ when performing a sync
292 transfer. The default sync level is 2. The available levels are:
293 · Level 0 will only transfer if the destination does not exist.
295 · Level 1 will transfer if the size of the destination does not
297 match the size of the source.
298 · Level 2 will transfer if the time stamp of the destination is
300 older than the time stamp of the source.
301 · Level 3 will perform a checksum of the source and destination
303 and transfer if the checksums do not match. The default
304 algorithm used for this checksum is MD5, but other algorithms
305 can be specified with the -algo parameter.
306 -checksum-alg CHECKSUM-ALGORITHM
308 Set the algorithm type to use for all checksum operations during
309 the transfer.
310 -verify-checksum
312 Perform a checksum on the source and destination after each file
313 transfer and compare the two. If they do not match, fail the
314 transfer. The default algorithm used for this checksum is MD5, but
315 other algorithms can be specified with the -checksum-alg parameter.
316
318 Copyright © 1999-2016 University of Chicago
319Grid Community Toolkit 6 03/31/2018 GLOBUS-URL-COPY(1)