1rclone(1) rclone(1)
2
3
4
6 • About rclone
7
8 • What can rclone do for you?
9
10 • What features does rclone have?
11
12 • What providers does rclone support?
13
14 • Download (https://rclone.org/downloads/)
15
16 • Install (https://rclone.org/install/)
17
18 • Donate. (https://rclone.org/donate/)
19
20 About rclone
21 Rclone is a command-line program to manage files on cloud storage. It
22 is a feature-rich alternative to cloud vendors' web storage interfaces.
23 Over 40 cloud storage products support rclone including S3 object
24 stores, business & consumer file storage services, as well as standard
25 transfer protocols.
26
27 Rclone has powerful cloud equivalents to the unix commands rsync, cp,
28 mv, mount, ls, ncdu, tree, rm, and cat. Rclone's familiar syntax in‐
29 cludes shell pipeline support, and --dry-run protection. It is used at
30 the command line, in scripts or via its API.
31
32 Users call rclone "The Swiss army knife of cloud storage", and "Tech‐
33 nology indistinguishable from magic".
34
35 Rclone really looks after your data. It preserves timestamps and veri‐
36 fies checksums at all times. Transfers over limited bandwidth; inter‐
37 mittent connections, or subject to quota can be restarted, from the
38 last good file transferred. You can check (https://rclone.org/com‐
39 mands/rclone_check/) the integrity of your files. Where possible,
40 rclone employs server-side transfers to minimise local bandwidth use
41 and transfers from one provider to another without using local disk.
42
43 Virtual backends wrap local and cloud file systems to apply encryption
44 (https://rclone.org/crypt/), compression (https://rclone.org/com‐
45 press/), chunking (https://rclone.org/chunker/), hashing
46 (https://rclone.org/hasher/) and joining (https://rclone.org/union/).
47
48 Rclone mounts (https://rclone.org/commands/rclone_mount/) any local,
49 cloud or virtual filesystem as a disk on Windows, macOS, linux and
50 FreeBSD, and also serves these over SFTP (https://rclone.org/com‐
51 mands/rclone_serve_sftp/), HTTP (https://rclone.org/com‐
52 mands/rclone_serve_http/), WebDAV (https://rclone.org/com‐
53 mands/rclone_serve_webdav/), FTP (https://rclone.org/com‐
54 mands/rclone_serve_ftp/) and DLNA (https://rclone.org/com‐
55 mands/rclone_serve_dlna/).
56
57 Rclone is mature, open-source software originally inspired by rsync and
58 written in Go (https://golang.org). The friendly support community is
59 familiar with varied use cases. Official Ubuntu, Debian, Fedora, Brew
60 and Chocolatey repos. include rclone. For the latest version down‐
61 loading from rclone.org (https://rclone.org/downloads/) is recommended.
62
63 Rclone is widely used on Linux, Windows and Mac. Third-party develop‐
64 ers create innovative backup, restore, GUI and business process solu‐
65 tions using the rclone command line or API.
66
67 Rclone does the heavy lifting of communicating with cloud storage.
68
69 What can rclone do for you?
70 Rclone helps you:
71
72 • Backup (and encrypt) files to cloud storage
73
74 • Restore (and decrypt) files from cloud storage
75
76 • Mirror cloud data to other cloud services or locally
77
78 • Migrate data to the cloud, or between cloud storage vendors
79
80 • Mount multiple, encrypted, cached or diverse cloud storage as a disk
81
82 • Analyse and account for data held on cloud storage using lsf
83 (https://rclone.org/commands/rclone_lsf/), ljson
84 (https://rclone.org/commands/rclone_lsjson/), size
85 (https://rclone.org/commands/rclone_size/), ncdu
86 (https://rclone.org/commands/rclone_ncdu/)
87
88 • Union (https://rclone.org/union/) file systems together to present
89 multiple local and/or cloud file systems as one
90
91 Features
92 • Transfers
93
94 • MD5, SHA1 hashes are checked at all times for file integrity
95
96 • Timestamps are preserved on files
97
98 • Operations can be restarted at any time
99
100 • Can be to and from network, e.g. two different cloud providers
101
102 • Can use multi-threaded downloads to local disk
103
104 • Copy (https://rclone.org/commands/rclone_copy/) new or changed files
105 to cloud storage
106
107 • Sync (https://rclone.org/commands/rclone_sync/) (one way) to make a
108 directory identical
109
110 • Move (https://rclone.org/commands/rclone_move/) files to cloud stor‐
111 age deleting the local after verification
112
113 • Check (https://rclone.org/commands/rclone_check/) hashes and for
114 missing/extra files
115
116 • Mount (https://rclone.org/commands/rclone_mount/) your cloud storage
117 as a network disk
118
119 • Serve (https://rclone.org/commands/rclone_serve/) local or remote
120 files over HTTP (https://rclone.org/commands/rclone_serve_http/)/Web‐
121 Dav (https://rclone.org/commands/rclone_serve_webdav/)/FTP
122 (https://rclone.org/commands/rclone_serve_ftp/)/SFTP
123 (https://rclone.org/commands/rclone_serve_sftp/)/DLNA
124 (https://rclone.org/commands/rclone_serve_dlna/)
125
126 • Experimental Web based GUI (https://rclone.org/gui/)
127
128 Supported providers
129 (There are many others, built on standard protocols such as WebDAV or
130 S3, that work out of the box.)
131
132 • 1Fichier
133
134 • Akamai Netstorage
135
136 • Alibaba Cloud (Aliyun) Object Storage System (OSS)
137
138 • Amazon Drive
139
140 • Amazon S3
141
142 • Backblaze B2
143
144 • Box
145
146 • Ceph
147
148 • China Mobile Ecloud Elastic Object Storage (EOS)
149
150 • Arvan Cloud Object Storage (AOS)
151
152 • Citrix ShareFile
153
154 • Cloudflare R2
155
156 • DigitalOcean Spaces
157
158 • Digi Storage
159
160 • Dreamhost
161
162 • Dropbox
163
164 • Enterprise File Fabric
165
166 • FTP
167
168 • Google Cloud Storage
169
170 • Google Drive
171
172 • Google Photos
173
174 • HDFS
175
176 • Hetzner Storage Box
177
178 • HiDrive
179
180 • HTTP
181
182 • Internet Archive
183
184 • Jottacloud
185
186 • IBM COS S3
187
188 • IDrive e2
189
190 • IONOS Cloud
191
192 • Koofr
193
194 • Mail.ru Cloud
195
196 • Memset Memstore
197
198 • Mega
199
200 • Memory
201
202 • Microsoft Azure Blob Storage
203
204 • Microsoft OneDrive
205
206 • Minio
207
208 • Nextcloud
209
210 • OVH
211
212 • OpenDrive
213
214 • OpenStack Swift
215
216 • Oracle Cloud Storage Swift
217
218 • Oracle Object Storage
219
220 • ownCloud
221
222 • pCloud
223
224 • premiumize.me
225
226 • put.io
227
228 • QingStor
229
230 • Qiniu Cloud Object Storage (Kodo)
231
232 • Rackspace Cloud Files
233
234 • rsync.net
235
236 • Scaleway
237
238 • Seafile
239
240 • Seagate Lyve Cloud
241
242 • SeaweedFS
243
244 • SFTP
245
246 • Sia
247
248 • SMB / CIFS
249
250 • StackPath
251
252 • Storj
253
254 • SugarSync
255
256 • Tencent Cloud Object Storage (COS)
257
258 • Uptobox
259
260 • Wasabi
261
262 • WebDAV
263
264 • Yandex Disk
265
266 • Zoho WorkDrive
267
268 • The local filesystem
269
270 Virtual providers
271 These backends adapt or modify other storage providers:
272
273 • Alias: Rename existing remotes
274
275 • Cache: Cache remotes (DEPRECATED)
276
277 • Chunker: Split large files
278
279 • Combine: Combine multiple remotes into a directory tree
280
281 • Compress: Compress files
282
283 • Crypt: Encrypt files
284
285 • Hasher: Hash files
286
287 • Union: Join multiple remotes to work together
288
289 Links
290 • Home page (https://rclone.org/)
291
292 • GitHub project page for source and bug tracker
293 (https://github.com/rclone/rclone)
294
295 • Rclone Forum (https://forum.rclone.org)
296
297 • Downloads (https://rclone.org/downloads/)
298
300 Rclone is a Go program and comes as a single binary file.
301
302 Quickstart
303 • Download (https://rclone.org/downloads/) the relevant binary.
304
305 • Extract the rclone executable, rclone.exe on Windows, from the ar‐
306 chive.
307
308 • Run rclone config to setup. See rclone config docs
309 (https://rclone.org/docs/) for more details.
310
311 • Optionally configure automatic execution.
312
313 See below for some expanded Linux / macOS / Windows instructions.
314
315 See the usage (https://rclone.org/docs/) docs for how to use rclone, or
316 run rclone -h.
317
318 Already installed rclone can be easily updated to the latest version
319 using the rclone selfupdate (https://rclone.org/commands/rclone_selfup‐
320 date/) command.
321
322 Script installation
323 To install rclone on Linux/macOS/BSD systems, run:
324
325 sudo -v ; curl https://rclone.org/install.sh | sudo bash
326
327 For beta installation, run:
328
329 sudo -v ; curl https://rclone.org/install.sh | sudo bash -s beta
330
331 Note that this script checks the version of rclone installed first and
332 won't re-download if not needed.
333
334 Linux installation
335 Precompiled binary
336 Fetch and unpack
337
338 curl -O https://downloads.rclone.org/rclone-current-linux-amd64.zip
339 unzip rclone-current-linux-amd64.zip
340 cd rclone-*-linux-amd64
341
342 Copy binary file
343
344 sudo cp rclone /usr/bin/
345 sudo chown root:root /usr/bin/rclone
346 sudo chmod 755 /usr/bin/rclone
347
348 Install manpage
349
350 sudo mkdir -p /usr/local/share/man/man1
351 sudo cp rclone.1 /usr/local/share/man/man1/
352 sudo mandb
353
354 Run rclone config to setup. See rclone config docs
355 (https://rclone.org/docs/) for more details.
356
357 rclone config
358
359 macOS installation
360 Installation with brew
361 brew install rclone
362
363 NOTE: This version of rclone will not support mount any more (see #5373
364 (https://github.com/rclone/rclone/issues/5373)). If mounting is wanted
365 on macOS, either install a precompiled binary or enable the relevant
366 option when installing from source.
367
368 Note that this is a third party installer not controlled by the rclone
369 developers so it may be out of date. Its current version is as below.
370
371 [IMAGE: Homebrew package (https://repology.org/badge/version-for-re‐
372 po/homebrew/rclone.svg)] (https://repology.org/project/rclone/versions)
373
374 Precompiled binary, using curl
375 To avoid problems with macOS gatekeeper enforcing the binary to be
376 signed and notarized it is enough to download with curl.
377
378 Download the latest version of rclone.
379
380 cd && curl -O https://downloads.rclone.org/rclone-current-osx-amd64.zip
381
382 Unzip the download and cd to the extracted folder.
383
384 unzip -a rclone-current-osx-amd64.zip && cd rclone-*-osx-amd64
385
386 Move rclone to your $PATH. You will be prompted for your password.
387
388 sudo mkdir -p /usr/local/bin
389 sudo mv rclone /usr/local/bin/
390
391 (the mkdir command is safe to run, even if the directory already ex‐
392 ists).
393
394 Remove the leftover files.
395
396 cd .. && rm -rf rclone-*-osx-amd64 rclone-current-osx-amd64.zip
397
398 Run rclone config to setup. See rclone config docs
399 (https://rclone.org/docs/) for more details.
400
401 rclone config
402
403 Precompiled binary, using a web browser
404 When downloading a binary with a web browser, the browser will set the
405 macOS gatekeeper quarantine attribute. Starting from Catalina, when
406 attempting to run rclone, a pop-up will appear saying:
407
408 "rclone" cannot be opened because the developer cannot be verified.
409 macOS cannot verify that this app is free from malware.
410
411 The simplest fix is to run
412
413 xattr -d com.apple.quarantine rclone
414
415 Windows installation
416 Precompiled binary
417 Fetch the correct binary for your processor type by clicking on these
418 links. If not sure, use the first link.
419
420 • Intel/AMD - 64 Bit (https://downloads.rclone.org/rclone-current-win‐
421 dows-amd64.zip)
422
423 • Intel/AMD - 32 Bit (https://downloads.rclone.org/rclone-current-win‐
424 dows-386.zip)
425
426 • ARM - 64 Bit (https://downloads.rclone.org/rclone-current-windows-
427 arm64.zip)
428
429 Open this file in the Explorer and extract rclone.exe. Rclone is a
430 portable executable so you can place it wherever is convenient.
431
432 Open a CMD window (or powershell) and run the binary. Note that rclone
433 does not launch a GUI by default, it runs in the CMD Window.
434
435 • Run rclone.exe config to setup. See rclone config docs
436 (https://rclone.org/docs/) for more details.
437
438 • Optionally configure automatic execution.
439
440 If you are planning to use the rclone mount (https://rclone.org/com‐
441 mands/rclone_mount/) feature then you will need to install the third
442 party utility WinFsp (https://winfsp.dev/) also.
443
444 Chocolatey package manager
445 Make sure you have Choco (https://chocolatey.org/) installed
446
447 choco search rclone
448 choco install rclone
449
450 This will install rclone on your Windows machine. If you are planning
451 to use rclone mount (https://rclone.org/commands/rclone_mount/) then
452
453 choco install winfsp
454
455 will install that too.
456
457 Note that this is a third party installer not controlled by the rclone
458 developers so it may be out of date. Its current version is as below.
459
460 [IMAGE: Chocolatey package (https://repology.org/badge/version-for-re‐
461 po/chocolatey/rclone.svg)] (https://repology.org/project/rclone/ver‐
462 sions)
463
464 Package manager installation
465 Many Linux, Windows, macOS and other OS distributions package and dis‐
466 tribute rclone.
467
468 The distributed versions of rclone are often quite out of date and for
469 this reason we recommend one of the other installation methods if pos‐
470 sible.
471
472 You can get an idea of how up to date or not your OS distribution's
473 package is here.
474
475 [IMAGE: Packaging status (https://repology.org/badge/vertical-allre‐
476 pos/rclone.svg?columns=3)] (https://repology.org/project/rclone/ver‐
477 sions)
478
479 Docker installation
480 The rclone developers maintain a docker image for rclone
481 (https://hub.docker.com/r/rclone/rclone).
482
483 These images are built as part of the release process based on a mini‐
484 mal Alpine Linux.
485
486 The :latest tag will always point to the latest stable release. You
487 can use the :beta tag to get the latest build from master. You can al‐
488 so use version tags, e.g. :1.49.1, :1.49 or :1.
489
490 $ docker pull rclone/rclone:latest
491 latest: Pulling from rclone/rclone
492 Digest: sha256:0e0ced72671989bb837fea8e88578b3fc48371aa45d209663683e24cfdaa0e11
493 ...
494 $ docker run --rm rclone/rclone:latest version
495 rclone v1.49.1
496 - os/arch: linux/amd64
497 - go version: go1.12.9
498
499 There are a few command line options to consider when starting an
500 rclone Docker container from the rclone image.
501
502 • You need to mount the host rclone config dir at /config/rclone into
503 the Docker container. Due to the fact that rclone updates tokens in‐
504 side its config file, and that the update process involves a file re‐
505 name, you need to mount the whole host rclone config dir, not just
506 the single host rclone config file.
507
508 • You need to mount a host data dir at /data into the Docker container.
509
510 • By default, the rclone binary inside a Docker container runs with
511 UID=0 (root). As a result, all files created in a run will have
512 UID=0. If your config and data files reside on the host with a non-
513 root UID:GID, you need to pass these on the container start command
514 line.
515
516 • If you want to access the RC interface (either via the API or the Web
517 UI), it is required to set the --rc-addr to :5572 in order to connect
518 to it from outside the container. An explanation about why this is
519 necessary is present here (https://web.ar‐
520 chive.org/web/20200808071950/https://pythonspeed.com/articles/docker-
521 connection-refused/).
522
523 • NOTE: Users running this container with the docker network set to
524 host should probably set it to listen to localhost only, with
525 127.0.0.1:5572 as the value for --rc-addr
526
527 • It is possible to use rclone mount inside a userspace Docker contain‐
528 er, and expose the resulting fuse mount to the host. The exact dock‐
529 er run options to do that might vary slightly between hosts. See,
530 e.g. the discussion in this thread (https://github.com/moby/moby/is‐
531 sues/9448).
532
533 You also need to mount the host /etc/passwd and /etc/group for fuse
534 to work inside the container.
535
536 Here are some commands tested on an Ubuntu 18.04.3 host:
537
538 # config on host at ~/.config/rclone/rclone.conf
539 # data on host at ~/data
540
541 # make sure the config is ok by listing the remotes
542 docker run --rm \
543 --volume ~/.config/rclone:/config/rclone \
544 --volume ~/data:/data:shared \
545 --user $(id -u):$(id -g) \
546 rclone/rclone \
547 listremotes
548
549 # perform mount inside Docker container, expose result to host
550 mkdir -p ~/data/mount
551 docker run --rm \
552 --volume ~/.config/rclone:/config/rclone \
553 --volume ~/data:/data:shared \
554 --user $(id -u):$(id -g) \
555 --volume /etc/passwd:/etc/passwd:ro --volume /etc/group:/etc/group:ro \
556 --device /dev/fuse --cap-add SYS_ADMIN --security-opt apparmor:unconfined \
557 rclone/rclone \
558 mount dropbox:Photos /data/mount &
559 ls ~/data/mount
560 kill %1
561
562 Source installation
563 Make sure you have git and Go (https://golang.org/) installed. Go ver‐
564 sion 1.17 or newer is required, latest release is recommended. You can
565 get it from your package manager, or download it from golang.org/dl
566 (https://golang.org/dl/). Then you can run the following:
567
568 git clone https://github.com/rclone/rclone.git
569 cd rclone
570 go build
571
572 This will check out the rclone source in subfolder rclone, which you
573 can later modify and send pull requests with. Then it will build the
574 rclone executable in the same folder. As an initial check you can now
575 run ./rclone version (.\rclone version on Windows).
576
577 Note that on macOS and Windows the mount (https://rclone.org/com‐
578 mands/rclone_mount/) command will not be available unless you specify
579 an additional build tag cmount.
580
581 go build -tags cmount
582
583 This assumes you have a GCC compatible C compiler (GCC or Clang) in
584 your PATH, as it uses cgo (https://pkg.go.dev/cmd/cgo). But on Win‐
585 dows, the cgofuse (https://github.com/winfsp/cgofuse) library that the
586 cmount implementation is based on, also supports building without cgo
587 (https://github.com/golang/go/wiki/WindowsDLLs), i.e. by setting envi‐
588 ronment variable CGO_ENABLED to value 0 (static linking). This is how
589 the official Windows release of rclone is being built, starting with
590 version 1.59. It is still possible to build with cgo on Windows as
591 well, by using the MinGW port of GCC, e.g. by installing it in a MSYS2
592 (https://www.msys2.org) distribution (make sure you install it in the
593 classic mingw64 subsystem, the ucrt64 version is not compatible).
594
595 Additionally, on Windows, you must install the third party utility Win‐
596 Fsp (https://winfsp.dev/), with the "Developer" feature selected. If
597 building with cgo, you must also set environment variable CPATH point‐
598 ing to the fuse include directory within the WinFsp installation (nor‐
599 mally C:\Program Files (x86)\WinFsp\inc\fuse).
600
601 You may also add arguments -ldflags -s (with or without -tags cmount),
602 to omit symbol table and debug information, making the executable file
603 smaller, and -trimpath to remove references to local file system paths.
604 This is how the official rclone releases are built.
605
606 go build -trimpath -ldflags -s -tags cmount
607
608 Instead of executing the go build command directly, you can run it via
609 the Makefile. It changes the version number suffix from "-DEV" to
610 "-beta" and appends commit details. It also copies the resulting
611 rclone executable into your GOPATH bin folder ($(go env GOPATH)/bin,
612 which corresponds to ~/go/bin/rclone by default).
613
614 make
615
616 To include mount command on macOS and Windows with Makefile build:
617
618 make GOTAGS=cmount
619
620 There are other make targets that can be used for more advanced builds,
621 such as cross-compiling for all supported os/architectures, embedding
622 icon and version info resources into windows executable, and packaging
623 results into release artifacts. See Makefile
624 (https://github.com/rclone/rclone/blob/master/Makefile) and cross-com‐
625 pile.go (https://github.com/rclone/rclone/blob/master/bin/cross-com‐
626 pile.go) for details.
627
628 Another alternative is to download the source, build and install rclone
629 in one operation, as a regular Go package. The source will be stored
630 it in the Go module cache, and the resulting executable will be in your
631 GOPATH bin folder ($(go env GOPATH)/bin, which corresponds to
632 ~/go/bin/rclone by default).
633
634 With Go version 1.17 or newer:
635
636 go install github.com/rclone/rclone@latest
637
638 With Go versions older than 1.17 (do not use the -u flag, it causes Go
639 to try to update the dependencies that rclone uses and sometimes these
640 don't work with the current version):
641
642 go get github.com/rclone/rclone
643
644 Ansible installation
645 This can be done with Stefan Weichinger's ansible role
646 (https://github.com/stefangweichinger/ansible-rclone).
647
648 Instructions
649
650 1. git clone https://github.com/stefangweichinger/ansible-rclone.git
651 into your local roles-directory
652
653 2. add the role to the hosts you want rclone installed to:
654
655 - hosts: rclone-hosts
656 roles:
657 - rclone
658
659 Portable installation
660 As mentioned above (https://rclone.org/install/#quickstart), rclone is
661 single executable (rclone, or rclone.exe on Windows) that you can down‐
662 load as a zip archive and extract into a location of your choosing.
663 When executing different commands, it may create files in different lo‐
664 cations, such as a configuration file and various temporary files. By
665 default the locations for these are according to your operating system,
666 e.g. configuration file in your user profile directory and temporary
667 files in the standard temporary directory, but you can customize all of
668 them, e.g. to make a completely self-contained, portable installation.
669
670 Run the config paths (https://rclone.org/commands/rclone_config_paths/)
671 command to see the locations that rclone will use.
672
673 To override them set the corresponding options (as command-line argu‐
674 ments, or as environment variables (https://rclone.org/docs/#environ‐
675 ment-variables)): - --config (https://rclone.org/docs/#config-config-
676 file) - --cache-dir (https://rclone.org/docs/#cache-dir-dir) - --temp-
677 dir (https://rclone.org/docs/#temp-dir-dir)
678
679 Autostart
680 After installing and configuring rclone, as described above, you are
681 ready to use rclone as an interactive command line utility. If your
682 goal is to perform periodic operations, such as a regular sync
683 (https://rclone.org/commands/rclone_sync/), you will probably want to
684 configure your rclone command in your operating system's scheduler. If
685 you need to expose service-like features, such as remote control
686 (https://rclone.org/rc/), GUI (https://rclone.org/gui/), serve
687 (https://rclone.org/commands/rclone_serve/) or mount
688 (https://rclone.org/commands/rclone_mount/), you will often want an
689 rclone command always running in the background, and configuring it to
690 run in a service infrastructure may be a better option. Below are some
691 alternatives on how to achieve this on different operating systems.
692
693 NOTE: Before setting up autorun it is highly recommended that you have
694 tested your command manually from a Command Prompt first.
695
696 Autostart on Windows
697 The most relevant alternatives for autostart on Windows are: - Run at
698 user log on using the Startup folder - Run at user log on, at system
699 startup or at schedule using Task Scheduler - Run at system startup us‐
700 ing Windows service
701
702 Running in background
703 Rclone is a console application, so if not starting from an existing
704 Command Prompt, e.g. when starting rclone.exe from a shortcut, it will
705 open a Command Prompt window. When configuring rclone to run from task
706 scheduler and windows service you are able to set it to run hidden in
707 background. From rclone version 1.54 you can also make it run hidden
708 from anywhere by adding option --no-console (it may still flash briefly
709 when the program starts). Since rclone normally writes information and
710 any error messages to the console, you must redirect this to a file to
711 be able to see it. Rclone has a built-in option --log-file for that.
712
713 Example command to run a sync in background:
714
715 c:\rclone\rclone.exe sync c:\files remote:/files --no-console --log-file c:\rclone\logs\sync_files.txt
716
717 User account
718 As mentioned in the mount (https://rclone.org/commands/rclone_mount/)
719 documentation, mounted drives created as Administrator are not visible
720 to other accounts, not even the account that was elevated as Adminis‐
721 trator. By running the mount command as the built-in SYSTEM user ac‐
722 count, it will create drives accessible for everyone on the system.
723 Both scheduled task and Windows service can be used to achieve this.
724
725 NOTE: Remember that when rclone runs as the SYSTEM user, the user pro‐
726 file that it sees will not be yours. This means that if you normally
727 run rclone with configuration file in the default location, to be able
728 to use the same configuration when running as the system user you must
729 explicitly tell rclone where to find it with the --config
730 (https://rclone.org/docs/#config-config-file) option, or else it will
731 look in the system users profile path (C:\Windows\System32\config\sys‐
732 temprofile). To test your command manually from a Command Prompt, you
733 can run it with the PsExec (https://docs.microsoft.com/en-us/sysinter‐
734 nals/downloads/psexec) utility from Microsoft's Sysinternals suite,
735 which takes option -s to execute commands as the SYSTEM user.
736
737 Start from Startup folder
738 To quickly execute an rclone command you can simply create a standard
739 Windows Explorer shortcut for the complete rclone command you want to
740 run. If you store this shortcut in the special "Startup" start-menu
741 folder, Windows will automatically run it at login. To open this fold‐
742 er in Windows Explorer, enter path %APPDATA%\Microsoft\Windows\Start
743 Menu\Programs\Startup, or C:\ProgramData\Microsoft\Windows\Start
744 Menu\Programs\StartUp if you want the command to start for every user
745 that logs in.
746
747 This is the easiest approach to autostarting of rclone, but it offers
748 no functionality to set it to run as different user, or to set condi‐
749 tions or actions on certain events. Setting up a scheduled task as de‐
750 scribed below will often give you better results.
751
752 Start from Task Scheduler
753 Task Scheduler is an administrative tool built into Windows, and it can
754 be used to configure rclone to be started automatically in a highly
755 configurable way, e.g. periodically on a schedule, on user log on, or
756 at system startup. It can run be configured to run as the current us‐
757 er, or for a mount command that needs to be available to all users it
758 can run as the SYSTEM user. For technical information, see
759 https://docs.microsoft.com/windows/win32/taskschd/task-scheduler-start-
760 page.
761
762 Run as service
763 For running rclone at system startup, you can create a Windows service
764 that executes your rclone command, as an alternative to scheduled task
765 configured to run at startup.
766
767 Mount command built-in service integration
768 For mount commands, rclone has a built-in Windows service integration
769 via the third-party WinFsp library it uses. Registering as a regular
770 Windows service easy, as you just have to execute the built-in Power‐
771 Shell command New-Service (requires administrative privileges).
772
773 Example of a PowerShell command that creates a Windows service for
774 mounting some remote:/files as drive letter X:, for all users (service
775 will be running as the local system account):
776
777 New-Service -Name Rclone -BinaryPathName 'c:\rclone\rclone.exe mount remote:/files X: --config c:\rclone\config\rclone.conf --log-file c:\rclone\logs\mount.txt'
778
779 The WinFsp service infrastructure (https://github.com/billziss-gh/winf‐
780 sp/wiki/WinFsp-Service-Architecture) supports incorporating services
781 for file system implementations, such as rclone, into its own launcher
782 service, as kind of "child services". This has the additional advan‐
783 tage that it also implements a network provider that integrates into
784 Windows standard methods for managing network drives. This is current‐
785 ly not officially supported by Rclone, but with WinFsp version 2019.3
786 B2 / v1.5B2 or later it should be possible through path rewriting as
787 described here (https://github.com/rclone/rclone/issues/3340).
788
789 Third-party service integration
790 To Windows service running any rclone command, the excellent third-par‐
791 ty utility NSSM (http://nssm.cc), the "Non-Sucking Service Manager",
792 can be used. It includes some advanced features such as adjusting
793 process priority, defining process environment variables, redirect to
794 file anything written to stdout, and customized response to different
795 exit codes, with a GUI to configure everything from (although it can
796 also be used from command line ).
797
798 There are also several other alternatives. To mention one more, WinSW
799 (https://github.com/winsw/winsw), "Windows Service Wrapper", is worth
800 checking out. It requires .NET Framework, but it is preinstalled on
801 newer versions of Windows, and it also provides alternative standalone
802 distributions which includes necessary runtime (.NET 5). WinSW is a
803 command-line only utility, where you have to manually create an XML
804 file with service configuration. This may be a drawback for some, but
805 it can also be an advantage as it is easy to back up and re-use the
806 configuration settings, without having go through manual steps in a
807 GUI. One thing to note is that by default it does not restart the ser‐
808 vice on error, one have to explicit enable this in the configuration
809 file (via the "onfailure" parameter).
810
811 Autostart on Linux
812 Start as a service
813 To always run rclone in background, relevant for mount commands etc,
814 you can use systemd to set up rclone as a system or user service. Run‐
815 ning as a system service ensures that it is run at startup even if the
816 user it is running as has no active session. Running rclone as a user
817 service ensures that it only starts after the configured user has
818 logged into the system.
819
820 Run periodically from cron
821 To run a periodic command, such as a copy/sync, you can set up a cron
822 job.
823
825 Rclone is a command line program to manage files on cloud storage. Af‐
826 ter download (https://rclone.org/downloads/) and install, continue here
827 to learn how to use it: Initial configuration, what the basic syntax
828 looks like, describes the various subcommands, the various options, and
829 more.
830
831 Configure
832 First, you'll need to configure rclone. As the object storage systems
833 have quite complicated authentication these are kept in a config file.
834 (See the --config entry for how to find the config file and choose its
835 location.)
836
837 The easiest way to make the config is to run rclone with the config op‐
838 tion:
839
840 rclone config
841
842 See the following for detailed instructions for
843
844 • 1Fichier (https://rclone.org/fichier/)
845
846 • Akamai Netstorage (https://rclone.org/netstorage/)
847
848 • Alias (https://rclone.org/alias/)
849
850 • Amazon Drive (https://rclone.org/amazonclouddrive/)
851
852 • Amazon S3 (https://rclone.org/s3/)
853
854 • Backblaze B2 (https://rclone.org/b2/)
855
856 • Box (https://rclone.org/box/)
857
858 • Chunker (https://rclone.org/chunker/) - transparently splits large
859 files for other remotes
860
861 • Citrix ShareFile (https://rclone.org/sharefile/)
862
863 • Compress (https://rclone.org/compress/)
864
865 • Combine (https://rclone.org/combine/)
866
867 • Crypt (https://rclone.org/crypt/) - to encrypt other remotes
868
869 • DigitalOcean Spaces (https://rclone.org/s3/#digitalocean-spaces)
870
871 • Digi Storage (https://rclone.org/koofr/#digi-storage)
872
873 • Dropbox (https://rclone.org/dropbox/)
874
875 • Enterprise File Fabric (https://rclone.org/filefabric/)
876
877 • FTP (https://rclone.org/ftp/)
878
879 • Google Cloud Storage (https://rclone.org/googlecloudstorage/)
880
881 • Google Drive (https://rclone.org/drive/)
882
883 • Google Photos (https://rclone.org/googlephotos/)
884
885 • Hasher (https://rclone.org/hasher/) - to handle checksums for other
886 remotes
887
888 • HDFS (https://rclone.org/hdfs/)
889
890 • HiDrive (https://rclone.org/hidrive/)
891
892 • HTTP (https://rclone.org/http/)
893
894 • Internet Archive (https://rclone.org/internetarchive/)
895
896 • Jottacloud (https://rclone.org/jottacloud/)
897
898 • Koofr (https://rclone.org/koofr/)
899
900 • Mail.ru Cloud (https://rclone.org/mailru/)
901
902 • Mega (https://rclone.org/mega/)
903
904 • Memory (https://rclone.org/memory/)
905
906 • Microsoft Azure Blob Storage (https://rclone.org/azureblob/)
907
908 • Microsoft OneDrive (https://rclone.org/onedrive/)
909
910 • OpenStack Swift / Rackspace Cloudfiles / Memset Memstore
911 (https://rclone.org/swift/)
912
913 • OpenDrive (https://rclone.org/opendrive/)
914
915 • Oracle Object Storage (https://rclone.org/oracleobjectstorage/)
916
917 • Pcloud (https://rclone.org/pcloud/)
918
919 • premiumize.me (https://rclone.org/premiumizeme/)
920
921 • put.io (https://rclone.org/putio/)
922
923 • QingStor (https://rclone.org/qingstor/)
924
925 • Seafile (https://rclone.org/seafile/)
926
927 • SFTP (https://rclone.org/sftp/)
928
929 • Sia (https://rclone.org/sia/)
930
931 • SMB (https://rclone.org/smb/)
932
933 • Storj (https://rclone.org/storj/)
934
935 • SugarSync (https://rclone.org/sugarsync/)
936
937 • Union (https://rclone.org/union/)
938
939 • Uptobox (https://rclone.org/uptobox/)
940
941 • WebDAV (https://rclone.org/webdav/)
942
943 • Yandex Disk (https://rclone.org/yandex/)
944
945 • Zoho WorkDrive (https://rclone.org/zoho/)
946
947 • The local filesystem (https://rclone.org/local/)
948
949 Basic syntax
950 Rclone syncs a directory tree from one storage system to another.
951
952 Its syntax is like this
953
954 Syntax: [options] subcommand <parameters> <parameters...>
955
956 Source and destination paths are specified by the name you gave the
957 storage system in the config file then the sub path, e.g. "drive:my‐
958 folder" to look at "myfolder" in Google drive.
959
960 You can define as many storage paths as you like in the config file.
961
962 Please use the -i / --interactive flag while learning rclone to avoid
963 accidental data loss.
964
965 Subcommands
966 rclone uses a system of subcommands. For example
967
968 rclone ls remote:path # lists a remote
969 rclone copy /local/path remote:path # copies /local/path to the remote
970 rclone sync -i /local/path remote:path # syncs /local/path to the remote
971
973 Enter an interactive configuration session.
974
975 Synopsis
976 Enter an interactive configuration session where you can setup new re‐
977 motes and manage existing ones. You may also set or remove a password
978 to protect your configuration.
979
980 rclone config [flags]
981
982 Options
983 -h, --help help for config
984
985 See the global flags page (https://rclone.org/flags/) for global op‐
986 tions not listed here.
987
988 SEE ALSO
989 • rclone (https://rclone.org/commands/rclone/) - Show help for rclone
990 commands, flags and backends.
991
992 • rclone config create (https://rclone.org/commands/rclone_config_cre‐
993 ate/) - Create a new remote with name, type and options.
994
995 • rclone config delete (https://rclone.org/commands/rclone_con‐
996 fig_delete/) - Delete an existing remote.
997
998 • rclone config disconnect (https://rclone.org/commands/rclone_con‐
999 fig_disconnect/) - Disconnects user from remote
1000
1001 • rclone config dump (https://rclone.org/commands/rclone_config_dump/)
1002 - Dump the config file as JSON.
1003
1004 • rclone config file (https://rclone.org/commands/rclone_config_file/)
1005 - Show path of configuration file in use.
1006
1007 • rclone config password (https://rclone.org/commands/rclone_con‐
1008 fig_password/) - Update password in an existing remote.
1009
1010 • rclone config paths (https://rclone.org/commands/rclone_con‐
1011 fig_paths/) - Show paths used for configuration, cache, temp etc.
1012
1013 • rclone config providers (https://rclone.org/commands/rclone_con‐
1014 fig_providers/) - List in JSON format all the providers and options.
1015
1016 • rclone config reconnect (https://rclone.org/commands/rclone_con‐
1017 fig_reconnect/) - Re-authenticates user with remote.
1018
1019 • rclone config show (https://rclone.org/commands/rclone_config_show/)
1020 - Print (decrypted) config file, or the config for a single remote.
1021
1022 • rclone config touch (https://rclone.org/commands/rclone_con‐
1023 fig_touch/) - Ensure configuration file exists.
1024
1025 • rclone config update (https://rclone.org/commands/rclone_config_up‐
1026 date/) - Update options in an existing remote.
1027
1028 • rclone config userinfo (https://rclone.org/commands/rclone_con‐
1029 fig_userinfo/) - Prints info about logged in user of remote.
1030
1032 Copy files from source to dest, skipping identical files.
1033
1034 Synopsis
1035 Copy the source to the destination. Does not transfer files that are
1036 identical on source and destination, testing by size and modification
1037 time or MD5SUM. Doesn't delete files from the destination. If you
1038 want to also delete files from destination, to make it match source,
1039 use the sync (https://rclone.org/commands/rclone_sync/) command in‐
1040 stead.
1041
1042 Note that it is always the contents of the directory that is synced,
1043 not the directory itself. So when source:path is a directory, it's the
1044 contents of source:path that are copied, not the directory name and
1045 contents.
1046
1047 To copy single files, use the copyto (https://rclone.org/com‐
1048 mands/rclone_copyto/) command instead.
1049
1050 If dest:path doesn't exist, it is created and the source:path contents
1051 go there.
1052
1053 For example
1054
1055 rclone copy source:sourcepath dest:destpath
1056
1057 Let's say there are two files in sourcepath
1058
1059 sourcepath/one.txt
1060 sourcepath/two.txt
1061
1062 This copies them to
1063
1064 destpath/one.txt
1065 destpath/two.txt
1066
1067 Not to
1068
1069 destpath/sourcepath/one.txt
1070 destpath/sourcepath/two.txt
1071
1072 If you are familiar with rsync, rclone always works as if you had writ‐
1073 ten a trailing / - meaning "copy the contents of this directory". This
1074 applies to all commands and whether you are talking about the source or
1075 destination.
1076
1077 See the --no-traverse (https://rclone.org/docs/#no-traverse) option for
1078 controlling whether rclone lists the destination directory or not.
1079 Supplying this option when copying a small number of files into a large
1080 destination can speed transfers up greatly.
1081
1082 For example, if you have many files in /path/to/src but only a few of
1083 them change every day, you can copy all the files which have changed
1084 recently very efficiently like this:
1085
1086 rclone copy --max-age 24h --no-traverse /path/to/src remote:
1087
1088 Note: Use the -P/--progress flag to view real-time transfer statistics.
1089
1090 Note: Use the --dry-run or the --interactive/-i flag to test without
1091 copying anything.
1092
1093 rclone copy source:path dest:path [flags]
1094
1095 Options
1096 --create-empty-src-dirs Create empty source dirs on destination after copy
1097 -h, --help help for copy
1098
1099 See the global flags page (https://rclone.org/flags/) for global op‐
1100 tions not listed here.
1101
1102 SEE ALSO
1103 • rclone (https://rclone.org/commands/rclone/) - Show help for rclone
1104 commands, flags and backends.
1105
1107 Make source and dest identical, modifying destination only.
1108
1109 Synopsis
1110 Sync the source to the destination, changing the destination only.
1111 Doesn't transfer files that are identical on source and destination,
1112 testing by size and modification time or MD5SUM. Destination is updat‐
1113 ed to match source, including deleting files if necessary (except du‐
1114 plicate objects, see below). If you don't want to delete files from
1115 destination, use the copy (https://rclone.org/commands/rclone_copy/)
1116 command instead.
1117
1118 Important: Since this can cause data loss, test first with the --dry-
1119 run or the --interactive/-i flag.
1120
1121 rclone sync -i SOURCE remote:DESTINATION
1122
1123 Note that files in the destination won't be deleted if there were any
1124 errors at any point. Duplicate objects (files with the same name, on
1125 those providers that support it) are also not yet handled.
1126
1127 It is always the contents of the directory that is synced, not the di‐
1128 rectory itself. So when source:path is a directory, it's the contents
1129 of source:path that are copied, not the directory name and contents.
1130 See extended explanation in the copy (https://rclone.org/com‐
1131 mands/rclone_copy/) command if unsure.
1132
1133 If dest:path doesn't exist, it is created and the source:path contents
1134 go there.
1135
1136 It is not possible to sync overlapping remotes. However, you may ex‐
1137 clude the destination from the sync with a filter rule or by putting an
1138 exclude-if-present file inside the destination directory and sync to a
1139 destination that is inside the source directory.
1140
1141 Note: Use the -P/--progress flag to view real-time transfer statistics
1142
1143 Note: Use the rclone dedupe command to deal with "Duplicate object/di‐
1144 rectory found in source/destination - ignoring" errors. See this forum
1145 post (https://forum.rclone.org/t/sync-not-clearing-duplicates/14372)
1146 for more info.
1147
1148 rclone sync source:path dest:path [flags]
1149
1150 Options
1151 --create-empty-src-dirs Create empty source dirs on destination after sync
1152 -h, --help help for sync
1153
1154 See the global flags page (https://rclone.org/flags/) for global op‐
1155 tions not listed here.
1156
1157 SEE ALSO
1158 • rclone (https://rclone.org/commands/rclone/) - Show help for rclone
1159 commands, flags and backends.
1160
1162 Move files from source to dest.
1163
1164 Synopsis
1165 Moves the contents of the source directory to the destination directo‐
1166 ry. Rclone will error if the source and destination overlap and the
1167 remote does not support a server-side directory move operation.
1168
1169 To move single files, use the moveto (https://rclone.org/com‐
1170 mands/rclone_moveto/) command instead.
1171
1172 If no filters are in use and if possible this will server-side move
1173 source:path into dest:path. After this source:path will no longer ex‐
1174 ist.
1175
1176 Otherwise for each file in source:path selected by the filters (if any)
1177 this will move it into dest:path. If possible a server-side move will
1178 be used, otherwise it will copy it (server-side if possible) into
1179 dest:path then delete the original (if no errors on copy) in
1180 source:path.
1181
1182 If you want to delete empty source directories after move, use the
1183 --delete-empty-src-dirs flag.
1184
1185 See the --no-traverse (https://rclone.org/docs/#no-traverse) option for
1186 controlling whether rclone lists the destination directory or not.
1187 Supplying this option when moving a small number of files into a large
1188 destination can speed transfers up greatly.
1189
1190 Important: Since this can cause data loss, test first with the --dry-
1191 run or the --interactive/-i flag.
1192
1193 Note: Use the -P/--progress flag to view real-time transfer statistics.
1194
1195 rclone move source:path dest:path [flags]
1196
1197 Options
1198 --create-empty-src-dirs Create empty source dirs on destination after move
1199 --delete-empty-src-dirs Delete empty source dirs after move
1200 -h, --help help for move
1201
1202 See the global flags page (https://rclone.org/flags/) for global op‐
1203 tions not listed here.
1204
1205 SEE ALSO
1206 • rclone (https://rclone.org/commands/rclone/) - Show help for rclone
1207 commands, flags and backends.
1208
1210 Remove the files in path.
1211
1212 Synopsis
1213 Remove the files in path. Unlike purge (https://rclone.org/com‐
1214 mands/rclone_purge/) it obeys include/exclude filters so can be used to
1215 selectively delete files.
1216
1217 rclone delete only deletes files but leaves the directory structure
1218 alone. If you want to delete a directory and all of its contents use
1219 the purge (https://rclone.org/commands/rclone_purge/) command.
1220
1221 If you supply the --rmdirs flag, it will remove all empty directories
1222 along with it. You can also use the separate command rmdir
1223 (https://rclone.org/commands/rclone_rmdir/) or rmdirs
1224 (https://rclone.org/commands/rclone_rmdirs/) to delete empty directo‐
1225 ries only.
1226
1227 For example, to delete all files bigger than 100 MiB, you may first
1228 want to check what would be deleted (use either):
1229
1230 rclone --min-size 100M lsl remote:path
1231 rclone --dry-run --min-size 100M delete remote:path
1232
1233 Then proceed with the actual delete:
1234
1235 rclone --min-size 100M delete remote:path
1236
1237 That reads "delete everything with a minimum size of 100 MiB", hence
1238 delete all files bigger than 100 MiB.
1239
1240 Important: Since this can cause data loss, test first with the --dry-
1241 run or the --interactive/-i flag.
1242
1243 rclone delete remote:path [flags]
1244
1245 Options
1246 -h, --help help for delete
1247 --rmdirs rmdirs removes empty directories but leaves root intact
1248
1249 See the global flags page (https://rclone.org/flags/) for global op‐
1250 tions not listed here.
1251
1252 SEE ALSO
1253 • rclone (https://rclone.org/commands/rclone/) - Show help for rclone
1254 commands, flags and backends.
1255
1257 Remove the path and all of its contents.
1258
1259 Synopsis
1260 Remove the path and all of its contents. Note that this does not obey
1261 include/exclude filters - everything will be removed. Use the delete
1262 (https://rclone.org/commands/rclone_delete/) command if you want to se‐
1263 lectively delete files. To delete empty directories only, use command
1264 rmdir (https://rclone.org/commands/rclone_rmdir/) or rmdirs
1265 (https://rclone.org/commands/rclone_rmdirs/).
1266
1267 Important: Since this can cause data loss, test first with the --dry-
1268 run or the --interactive/-i flag.
1269
1270 rclone purge remote:path [flags]
1271
1272 Options
1273 -h, --help help for purge
1274
1275 See the global flags page (https://rclone.org/flags/) for global op‐
1276 tions not listed here.
1277
1278 SEE ALSO
1279 • rclone (https://rclone.org/commands/rclone/) - Show help for rclone
1280 commands, flags and backends.
1281
1283 Make the path if it doesn't already exist.
1284
1285 rclone mkdir remote:path [flags]
1286
1287 Options
1288 -h, --help help for mkdir
1289
1290 See the global flags page (https://rclone.org/flags/) for global op‐
1291 tions not listed here.
1292
1293 SEE ALSO
1294 • rclone (https://rclone.org/commands/rclone/) - Show help for rclone
1295 commands, flags and backends.
1296
1298 Remove the empty directory at path.
1299
1300 Synopsis
1301 This removes empty directory given by path. Will not remove the path
1302 if it has any objects in it, not even empty subdirectories. Use com‐
1303 mand rmdirs (https://rclone.org/commands/rclone_rmdirs/) (or delete
1304 (https://rclone.org/commands/rclone_delete/) with option --rmdirs) to
1305 do that.
1306
1307 To delete a path and any objects in it, use purge
1308 (https://rclone.org/commands/rclone_purge/) command.
1309
1310 rclone rmdir remote:path [flags]
1311
1312 Options
1313 -h, --help help for rmdir
1314
1315 See the global flags page (https://rclone.org/flags/) for global op‐
1316 tions not listed here.
1317
1318 SEE ALSO
1319 • rclone (https://rclone.org/commands/rclone/) - Show help for rclone
1320 commands, flags and backends.
1321
1323 Checks the files in the source and destination match.
1324
1325 Synopsis
1326 Checks the files in the source and destination match. It compares
1327 sizes and hashes (MD5 or SHA1) and logs a report of files that don't
1328 match. It doesn't alter the source or destination.
1329
1330 For the crypt (https://rclone.org/crypt/) remote there is a dedicated
1331 command, cryptcheck (https://rclone.org/commands/rclone_cryptcheck/),
1332 that are able to check the checksums of the crypted files.
1333
1334 If you supply the --size-only flag, it will only compare the sizes not
1335 the hashes as well. Use this for a quick check.
1336
1337 If you supply the --download flag, it will download the data from both
1338 remotes and check them against each other on the fly. This can be use‐
1339 ful for remotes that don't support hashes or if you really want to
1340 check all the data.
1341
1342 If you supply the --checkfile HASH flag with a valid hash name, the
1343 source:path must point to a text file in the SUM format.
1344
1345 If you supply the --one-way flag, it will only check that files in the
1346 source match the files in the destination, not the other way around.
1347 This means that extra files in the destination that are not in the
1348 source will not be detected.
1349
1350 The --differ, --missing-on-dst, --missing-on-src, --match and --error
1351 flags write paths, one per line, to the file name (or stdout if it is
1352 -) supplied. What they write is described in the help below. For ex‐
1353 ample --differ will write all paths which are present on both the
1354 source and destination but different.
1355
1356 The --combined flag will write a file (or stdout) which contains all
1357 file paths with a symbol and then a space and then the path to tell you
1358 what happened to it. These are reminiscent of diff files.
1359
1360 • = path means path was found in source and destination and was identi‐
1361 cal
1362
1363 • `- path` means path was missing on the source, so only in the desti‐
1364 nation
1365
1366 • `+ path` means path was missing on the destination, so only in the
1367 source
1368
1369 • `* path` means path was present in source and destination but differ‐
1370 ent.
1371
1372 • ! path means there was an error reading or hashing the source or
1373 dest.
1374
1375 rclone check source:path dest:path [flags]
1376
1377 Options
1378 -C, --checkfile string Treat source:path as a SUM file with hashes of given type
1379 --combined string Make a combined report of changes to this file
1380 --differ string Report all non-matching files to this file
1381 --download Check by downloading rather than with hash
1382 --error string Report all files with errors (hashing or reading) to this file
1383 -h, --help help for check
1384 --match string Report all matching files to this file
1385 --missing-on-dst string Report all files missing from the destination to this file
1386 --missing-on-src string Report all files missing from the source to this file
1387 --one-way Check one way only, source files must exist on remote
1388
1389 See the global flags page (https://rclone.org/flags/) for global op‐
1390 tions not listed here.
1391
1392 SEE ALSO
1393 • rclone (https://rclone.org/commands/rclone/) - Show help for rclone
1394 commands, flags and backends.
1395
1397 List the objects in the path with size and path.
1398
1399 Synopsis
1400 Lists the objects in the source path to standard output in a human
1401 readable format with size and path. Recurses by default.
1402
1403 Eg
1404
1405 $ rclone ls swift:bucket
1406 60295 bevajer5jef
1407 90613 canole
1408 94467 diwogej7
1409 37600 fubuwic
1410
1411 Any of the filtering options can be applied to this command.
1412
1413 There are several related list commands
1414
1415 • ls to list size and path of objects only
1416
1417 • lsl to list modification time, size and path of objects only
1418
1419 • lsd to list directories only
1420
1421 • lsf to list objects and directories in easy to parse format
1422
1423 • lsjson to list objects and directories in JSON format
1424
1425 ls,lsl,lsd are designed to be human-readable. lsf is designed to be
1426 human and machine-readable. lsjson is designed to be machine-readable.
1427
1428 Note that ls and lsl recurse by default - use --max-depth 1 to stop the
1429 recursion.
1430
1431 The other list commands lsd,lsf,lsjson do not recurse by default - use
1432 -R to make them recurse.
1433
1434 Listing a nonexistent directory will produce an error except for re‐
1435 motes which can't have empty directories (e.g. s3, swift, or gcs - the
1436 bucket-based remotes).
1437
1438 rclone ls remote:path [flags]
1439
1440 Options
1441 -h, --help help for ls
1442
1443 See the global flags page (https://rclone.org/flags/) for global op‐
1444 tions not listed here.
1445
1446 SEE ALSO
1447 • rclone (https://rclone.org/commands/rclone/) - Show help for rclone
1448 commands, flags and backends.
1449
1451 List all directories/containers/buckets in the path.
1452
1453 Synopsis
1454 Lists the directories in the source path to standard output. Does not
1455 recurse by default. Use the -R flag to recurse.
1456
1457 This command lists the total size of the directory (if known, -1 if
1458 not), the modification time (if known, the current time if not), the
1459 number of objects in the directory (if known, -1 if not) and the name
1460 of the directory, Eg
1461
1462 $ rclone lsd swift:
1463 494000 2018-04-26 08:43:20 10000 10000files
1464 65 2018-04-26 08:43:20 1 1File
1465
1466 Or
1467
1468 $ rclone lsd drive:test
1469 -1 2016-10-17 17:41:53 -1 1000files
1470 -1 2017-01-03 14:40:54 -1 2500files
1471 -1 2017-07-08 14:39:28 -1 4000files
1472
1473 If you just want the directory names use rclone lsf --dirs-only.
1474
1475 Any of the filtering options can be applied to this command.
1476
1477 There are several related list commands
1478
1479 • ls to list size and path of objects only
1480
1481 • lsl to list modification time, size and path of objects only
1482
1483 • lsd to list directories only
1484
1485 • lsf to list objects and directories in easy to parse format
1486
1487 • lsjson to list objects and directories in JSON format
1488
1489 ls,lsl,lsd are designed to be human-readable. lsf is designed to be
1490 human and machine-readable. lsjson is designed to be machine-readable.
1491
1492 Note that ls and lsl recurse by default - use --max-depth 1 to stop the
1493 recursion.
1494
1495 The other list commands lsd,lsf,lsjson do not recurse by default - use
1496 -R to make them recurse.
1497
1498 Listing a nonexistent directory will produce an error except for re‐
1499 motes which can't have empty directories (e.g. s3, swift, or gcs - the
1500 bucket-based remotes).
1501
1502 rclone lsd remote:path [flags]
1503
1504 Options
1505 -h, --help help for lsd
1506 -R, --recursive Recurse into the listing
1507
1508 See the global flags page (https://rclone.org/flags/) for global op‐
1509 tions not listed here.
1510
1511 SEE ALSO
1512 • rclone (https://rclone.org/commands/rclone/) - Show help for rclone
1513 commands, flags and backends.
1514
1516 List the objects in path with modification time, size and path.
1517
1518 Synopsis
1519 Lists the objects in the source path to standard output in a human
1520 readable format with modification time, size and path. Recurses by de‐
1521 fault.
1522
1523 Eg
1524
1525 $ rclone lsl swift:bucket
1526 60295 2016-06-25 18:55:41.062626927 bevajer5jef
1527 90613 2016-06-25 18:55:43.302607074 canole
1528 94467 2016-06-25 18:55:43.046609333 diwogej7
1529 37600 2016-06-25 18:55:40.814629136 fubuwic
1530
1531 Any of the filtering options can be applied to this command.
1532
1533 There are several related list commands
1534
1535 • ls to list size and path of objects only
1536
1537 • lsl to list modification time, size and path of objects only
1538
1539 • lsd to list directories only
1540
1541 • lsf to list objects and directories in easy to parse format
1542
1543 • lsjson to list objects and directories in JSON format
1544
1545 ls,lsl,lsd are designed to be human-readable. lsf is designed to be
1546 human and machine-readable. lsjson is designed to be machine-readable.
1547
1548 Note that ls and lsl recurse by default - use --max-depth 1 to stop the
1549 recursion.
1550
1551 The other list commands lsd,lsf,lsjson do not recurse by default - use
1552 -R to make them recurse.
1553
1554 Listing a nonexistent directory will produce an error except for re‐
1555 motes which can't have empty directories (e.g. s3, swift, or gcs - the
1556 bucket-based remotes).
1557
1558 rclone lsl remote:path [flags]
1559
1560 Options
1561 -h, --help help for lsl
1562
1563 See the global flags page (https://rclone.org/flags/) for global op‐
1564 tions not listed here.
1565
1566 SEE ALSO
1567 • rclone (https://rclone.org/commands/rclone/) - Show help for rclone
1568 commands, flags and backends.
1569
1571 Produces an md5sum file for all the objects in the path.
1572
1573 Synopsis
1574 Produces an md5sum file for all the objects in the path. This is in
1575 the same format as the standard md5sum tool produces.
1576
1577 By default, the hash is requested from the remote. If MD5 is not sup‐
1578 ported by the remote, no hash will be returned. With the download
1579 flag, the file will be downloaded from the remote and hashed locally
1580 enabling MD5 for any remote.
1581
1582 For other algorithms, see the hashsum (https://rclone.org/com‐
1583 mands/rclone_hashsum/) command. Running rclone md5sum remote:path is
1584 equivalent to running rclone hashsum MD5 remote:path.
1585
1586 This command can also hash data received on standard input (stdin), by
1587 not passing a remote:path, or by passing a hyphen as remote:path when
1588 there is data to read (if not, the hyphen will be treated literally, as
1589 a relative path).
1590
1591 rclone md5sum remote:path [flags]
1592
1593 Options
1594 --base64 Output base64 encoded hashsum
1595 -C, --checkfile string Validate hashes against a given SUM file instead of printing them
1596 --download Download the file and hash it locally; if this flag is not specified, the hash is requested from the remote
1597 -h, --help help for md5sum
1598 --output-file string Output hashsums to a file rather than the terminal
1599
1600 See the global flags page (https://rclone.org/flags/) for global op‐
1601 tions not listed here.
1602
1603 SEE ALSO
1604 • rclone (https://rclone.org/commands/rclone/) - Show help for rclone
1605 commands, flags and backends.
1606
1608 Produces an sha1sum file for all the objects in the path.
1609
1610 Synopsis
1611 Produces an sha1sum file for all the objects in the path. This is in
1612 the same format as the standard sha1sum tool produces.
1613
1614 By default, the hash is requested from the remote. If SHA-1 is not
1615 supported by the remote, no hash will be returned. With the download
1616 flag, the file will be downloaded from the remote and hashed locally
1617 enabling SHA-1 for any remote.
1618
1619 For other algorithms, see the hashsum (https://rclone.org/com‐
1620 mands/rclone_hashsum/) command. Running rclone sha1sum remote:path is
1621 equivalent to running rclone hashsum SHA1 remote:path.
1622
1623 This command can also hash data received on standard input (stdin), by
1624 not passing a remote:path, or by passing a hyphen as remote:path when
1625 there is data to read (if not, the hyphen will be treated literally, as
1626 a relative path).
1627
1628 This command can also hash data received on STDIN, if not passing a re‐
1629 mote:path.
1630
1631 rclone sha1sum remote:path [flags]
1632
1633 Options
1634 --base64 Output base64 encoded hashsum
1635 -C, --checkfile string Validate hashes against a given SUM file instead of printing them
1636 --download Download the file and hash it locally; if this flag is not specified, the hash is requested from the remote
1637 -h, --help help for sha1sum
1638 --output-file string Output hashsums to a file rather than the terminal
1639
1640 See the global flags page (https://rclone.org/flags/) for global op‐
1641 tions not listed here.
1642
1643 SEE ALSO
1644 • rclone (https://rclone.org/commands/rclone/) - Show help for rclone
1645 commands, flags and backends.
1646
1648 Prints the total size and number of objects in remote:path.
1649
1650 Synopsis
1651 Counts objects in the path and calculates the total size. Prints the
1652 result to standard output.
1653
1654 By default the output is in human-readable format, but shows values in
1655 both human-readable format as well as the raw numbers (global option
1656 --human-readable is not considered). Use option --json to format out‐
1657 put as JSON instead.
1658
1659 Recurses by default, use --max-depth 1 to stop the recursion.
1660
1661 Some backends do not always provide file sizes, see for example Google
1662 Photos (https://rclone.org/googlephotos/#size) and Google Drive
1663 (https://rclone.org/drive/#limitations-of-google-docs). Rclone will
1664 then show a notice in the log indicating how many such files were en‐
1665 countered, and count them in as empty files in the output of the size
1666 command.
1667
1668 rclone size remote:path [flags]
1669
1670 Options
1671 -h, --help help for size
1672 --json Format output as JSON
1673
1674 See the global flags page (https://rclone.org/flags/) for global op‐
1675 tions not listed here.
1676
1677 SEE ALSO
1678 • rclone (https://rclone.org/commands/rclone/) - Show help for rclone
1679 commands, flags and backends.
1680
1682 Show the version number.
1683
1684 Synopsis
1685 Show the rclone version number, the go version, the build target OS and
1686 architecture, the runtime OS and kernel version and bitness, build tags
1687 and the type of executable (static or dynamic).
1688
1689 For example:
1690
1691 $ rclone version
1692 rclone v1.55.0
1693 - os/version: ubuntu 18.04 (64 bit)
1694 - os/kernel: 4.15.0-136-generic (x86_64)
1695 - os/type: linux
1696 - os/arch: amd64
1697 - go/version: go1.16
1698 - go/linking: static
1699 - go/tags: none
1700
1701 Note: before rclone version 1.55 the os/type and os/arch lines were
1702 merged, and the "go/version" line was tagged as "go version".
1703
1704 If you supply the --check flag, then it will do an online check to com‐
1705 pare your version with the latest release and the latest beta.
1706
1707 $ rclone version --check
1708 yours: 1.42.0.6
1709 latest: 1.42 (released 2018-06-16)
1710 beta: 1.42.0.5 (released 2018-06-17)
1711
1712 Or
1713
1714 $ rclone version --check
1715 yours: 1.41
1716 latest: 1.42 (released 2018-06-16)
1717 upgrade: https://downloads.rclone.org/v1.42
1718 beta: 1.42.0.5 (released 2018-06-17)
1719 upgrade: https://beta.rclone.org/v1.42-005-g56e1e820
1720
1721 rclone version [flags]
1722
1723 Options
1724 --check Check for new version
1725 -h, --help help for version
1726
1727 See the global flags page (https://rclone.org/flags/) for global op‐
1728 tions not listed here.
1729
1730 SEE ALSO
1731 • rclone (https://rclone.org/commands/rclone/) - Show help for rclone
1732 commands, flags and backends.
1733
1735 Clean up the remote if possible.
1736
1737 Synopsis
1738 Clean up the remote if possible. Empty the trash or delete old file
1739 versions. Not supported by all remotes.
1740
1741 rclone cleanup remote:path [flags]
1742
1743 Options
1744 -h, --help help for cleanup
1745
1746 See the global flags page (https://rclone.org/flags/) for global op‐
1747 tions not listed here.
1748
1749 SEE ALSO
1750 • rclone (https://rclone.org/commands/rclone/) - Show help for rclone
1751 commands, flags and backends.
1752
1754 Interactively find duplicate filenames and delete/rename them.
1755
1756 Synopsis
1757 By default dedupe interactively finds files with duplicate names and
1758 offers to delete all but one or rename them to be different. This is
1759 known as deduping by name.
1760
1761 Deduping by name is only useful with a small group of backends (e.g.
1762 Google Drive, Opendrive) that can have duplicate file names. It can be
1763 run on wrapping backends (e.g. crypt) if they wrap a backend which
1764 supports duplicate file names.
1765
1766 However if --by-hash is passed in then dedupe will find files with du‐
1767 plicate hashes instead which will work on any backend which supports at
1768 least one hash. This can be used to find files with duplicate content.
1769 This is known as deduping by hash.
1770
1771 If deduping by name, first rclone will merge directories with the same
1772 name. It will do this iteratively until all the identically named di‐
1773 rectories have been merged.
1774
1775 Next, if deduping by name, for every group of duplicate file names /
1776 hashes, it will delete all but one identical file it finds without con‐
1777 firmation. This means that for most duplicated files the dedupe com‐
1778 mand will not be interactive.
1779
1780 dedupe considers files to be identical if they have the same file path
1781 and the same hash. If the backend does not support hashes (e.g. crypt
1782 wrapping Google Drive) then they will never be found to be identical.
1783 If you use the --size-only flag then files will be considered identical
1784 if they have the same size (any hash will be ignored). This can be
1785 useful on crypt backends which do not support hashes.
1786
1787 Next rclone will resolve the remaining duplicates. Exactly which ac‐
1788 tion is taken depends on the dedupe mode. By default, rclone will in‐
1789 teractively query the user for each one.
1790
1791 Important: Since this can cause data loss, test first with the --dry-
1792 run or the --interactive/-i flag.
1793
1794 Here is an example run.
1795
1796 Before - with duplicates
1797
1798 $ rclone lsl drive:dupes
1799 6048320 2016-03-05 16:23:16.798000000 one.txt
1800 6048320 2016-03-05 16:23:11.775000000 one.txt
1801 564374 2016-03-05 16:23:06.731000000 one.txt
1802 6048320 2016-03-05 16:18:26.092000000 one.txt
1803 6048320 2016-03-05 16:22:46.185000000 two.txt
1804 1744073 2016-03-05 16:22:38.104000000 two.txt
1805 564374 2016-03-05 16:22:52.118000000 two.txt
1806
1807 Now the dedupe session
1808
1809 $ rclone dedupe drive:dupes
1810 2016/03/05 16:24:37 Google drive root 'dupes': Looking for duplicates using interactive mode.
1811 one.txt: Found 4 files with duplicate names
1812 one.txt: Deleting 2/3 identical duplicates (MD5 "1eedaa9fe86fd4b8632e2ac549403b36")
1813 one.txt: 2 duplicates remain
1814 1: 6048320 bytes, 2016-03-05 16:23:16.798000000, MD5 1eedaa9fe86fd4b8632e2ac549403b36
1815 2: 564374 bytes, 2016-03-05 16:23:06.731000000, MD5 7594e7dc9fc28f727c42ee3e0749de81
1816 s) Skip and do nothing
1817 k) Keep just one (choose which in next step)
1818 r) Rename all to be different (by changing file.jpg to file-1.jpg)
1819 s/k/r> k
1820 Enter the number of the file to keep> 1
1821 one.txt: Deleted 1 extra copies
1822 two.txt: Found 3 files with duplicate names
1823 two.txt: 3 duplicates remain
1824 1: 564374 bytes, 2016-03-05 16:22:52.118000000, MD5 7594e7dc9fc28f727c42ee3e0749de81
1825 2: 6048320 bytes, 2016-03-05 16:22:46.185000000, MD5 1eedaa9fe86fd4b8632e2ac549403b36
1826 3: 1744073 bytes, 2016-03-05 16:22:38.104000000, MD5 851957f7fb6f0bc4ce76be966d336802
1827 s) Skip and do nothing
1828 k) Keep just one (choose which in next step)
1829 r) Rename all to be different (by changing file.jpg to file-1.jpg)
1830 s/k/r> r
1831 two-1.txt: renamed from: two.txt
1832 two-2.txt: renamed from: two.txt
1833 two-3.txt: renamed from: two.txt
1834
1835 The result being
1836
1837 $ rclone lsl drive:dupes
1838 6048320 2016-03-05 16:23:16.798000000 one.txt
1839 564374 2016-03-05 16:22:52.118000000 two-1.txt
1840 6048320 2016-03-05 16:22:46.185000000 two-2.txt
1841 1744073 2016-03-05 16:22:38.104000000 two-3.txt
1842
1843 Dedupe can be run non interactively using the --dedupe-mode flag or by
1844 using an extra parameter with the same value
1845
1846 • --dedupe-mode interactive - interactive as above.
1847
1848 • --dedupe-mode skip - removes identical files then skips anything
1849 left.
1850
1851 • --dedupe-mode first - removes identical files then keeps the first
1852 one.
1853
1854 • --dedupe-mode newest - removes identical files then keeps the newest
1855 one.
1856
1857 • --dedupe-mode oldest - removes identical files then keeps the oldest
1858 one.
1859
1860 • --dedupe-mode largest - removes identical files then keeps the
1861 largest one.
1862
1863 • --dedupe-mode smallest - removes identical files then keeps the
1864 smallest one.
1865
1866 • --dedupe-mode rename - removes identical files then renames the rest
1867 to be different.
1868
1869 • --dedupe-mode list - lists duplicate dirs and files only and changes
1870 nothing.
1871
1872 For example, to rename all the identically named photos in your Google
1873 Photos directory, do
1874
1875 rclone dedupe --dedupe-mode rename "drive:Google Photos"
1876
1877 Or
1878
1879 rclone dedupe rename "drive:Google Photos"
1880
1881 rclone dedupe [mode] remote:path [flags]
1882
1883 Options
1884 --by-hash Find identical hashes rather than names
1885 --dedupe-mode string Dedupe mode interactive|skip|first|newest|oldest|largest|smallest|rename (default "interactive")
1886 -h, --help help for dedupe
1887
1888 See the global flags page (https://rclone.org/flags/) for global op‐
1889 tions not listed here.
1890
1891 SEE ALSO
1892 • rclone (https://rclone.org/commands/rclone/) - Show help for rclone
1893 commands, flags and backends.
1894
1896 Get quota information from the remote.
1897
1898 Synopsis
1899 rclone about prints quota information about a remote to standard out‐
1900 put. The output is typically used, free, quota and trash contents.
1901
1902 E.g. Typical output from rclone about remote: is:
1903
1904 Total: 17 GiB
1905 Used: 7.444 GiB
1906 Free: 1.315 GiB
1907 Trashed: 100.000 MiB
1908 Other: 8.241 GiB
1909
1910 Where the fields are:
1911
1912 • Total: Total size available.
1913
1914 • Used: Total size used.
1915
1916 • Free: Total space available to this user.
1917
1918 • Trashed: Total space used by trash.
1919
1920 • Other: Total amount in other storage (e.g. Gmail, Google Photos).
1921
1922 • Objects: Total number of objects in the storage.
1923
1924 All sizes are in number of bytes.
1925
1926 Applying a --full flag to the command prints the bytes in full, e.g.
1927
1928 Total: 18253611008
1929 Used: 7993453766
1930 Free: 1411001220
1931 Trashed: 104857602
1932 Other: 8849156022
1933
1934 A --json flag generates conveniently machine-readable output, e.g.
1935
1936 {
1937 "total": 18253611008,
1938 "used": 7993453766,
1939 "trashed": 104857602,
1940 "other": 8849156022,
1941 "free": 1411001220
1942 }
1943
1944 Not all backends print all fields. Information is not included if it
1945 is not provided by a backend. Where the value is unlimited it is omit‐
1946 ted.
1947
1948 Some backends does not support the rclone about command at all, see
1949 complete list in documentation (https://rclone.org/overview/#optional-
1950 features).
1951
1952 rclone about remote: [flags]
1953
1954 Options
1955 --full Full numbers instead of human-readable
1956 -h, --help help for about
1957 --json Format output as JSON
1958
1959 See the global flags page (https://rclone.org/flags/) for global op‐
1960 tions not listed here.
1961
1962 SEE ALSO
1963 • rclone (https://rclone.org/commands/rclone/) - Show help for rclone
1964 commands, flags and backends.
1965
1967 Remote authorization.
1968
1969 Synopsis
1970 Remote authorization. Used to authorize a remote or headless rclone
1971 from a machine with a browser - use as instructed by rclone config.
1972
1973 Use the --auth-no-open-browser to prevent rclone to open auth link in
1974 default browser automatically.
1975
1976 rclone authorize [flags]
1977
1978 Options
1979 --auth-no-open-browser Do not automatically open auth link in default browser
1980 -h, --help help for authorize
1981
1982 See the global flags page (https://rclone.org/flags/) for global op‐
1983 tions not listed here.
1984
1985 SEE ALSO
1986 • rclone (https://rclone.org/commands/rclone/) - Show help for rclone
1987 commands, flags and backends.
1988
1990 Run a backend-specific command.
1991
1992 Synopsis
1993 This runs a backend-specific command. The commands themselves (except
1994 for "help" and "features") are defined by the backends and you should
1995 see the backend docs for definitions.
1996
1997 You can discover what commands a backend implements by using
1998
1999 rclone backend help remote:
2000 rclone backend help <backendname>
2001
2002 You can also discover information about the backend using (see opera‐
2003 tions/fsinfo (https://rclone.org/rc/#operations-fsinfo) in the remote
2004 control docs for more info).
2005
2006 rclone backend features remote:
2007
2008 Pass options to the backend command with -o. This should be key=value
2009 or key, e.g.:
2010
2011 rclone backend stats remote:path stats -o format=json -o long
2012
2013 Pass arguments to the backend by placing them on the end of the line
2014
2015 rclone backend cleanup remote:path file1 file2 file3
2016
2017 Note to run these commands on a running backend then see backend/com‐
2018 mand (https://rclone.org/rc/#backend-command) in the rc docs.
2019
2020 rclone backend <command> remote:path [opts] <args> [flags]
2021
2022 Options
2023 -h, --help help for backend
2024 --json Always output in JSON format
2025 -o, --option stringArray Option in the form name=value or name
2026
2027 See the global flags page (https://rclone.org/flags/) for global op‐
2028 tions not listed here.
2029
2030 SEE ALSO
2031 • rclone (https://rclone.org/commands/rclone/) - Show help for rclone
2032 commands, flags and backends.
2033
2035 Perform bidirectional synchronization between two paths.
2036
2037 Synopsis
2038 Perform bidirectional synchronization between two paths.
2039
2040 Bisync (https://rclone.org/bisync/) provides a bidirectional cloud sync
2041 solution in rclone. It retains the Path1 and Path2 filesystem listings
2042 from the prior run. On each successive run it will: - list files on
2043 Path1 and Path2, and check for changes on each side. Changes include
2044 New, Newer, Older, and Deleted files. - Propagate changes on Path1 to
2045 Path2, and vice-versa.
2046
2047 See full bisync description (https://rclone.org/bisync/) for details.
2048
2049 rclone bisync remote1:path1 remote2:path2 [flags]
2050
2051 Options
2052 --check-access Ensure expected RCLONE_TEST files are found on both Path1 and Path2 filesystems, else abort.
2053 --check-filename string Filename for --check-access (default: RCLONE_TEST)
2054 --check-sync string Controls comparison of final listings: true|false|only (default: true) (default "true")
2055 --filters-file string Read filtering patterns from a file
2056 --force Bypass --max-delete safety check and run the sync. Consider using with --verbose
2057 -h, --help help for bisync
2058 --localtime Use local time in listings (default: UTC)
2059 --no-cleanup Retain working files (useful for troubleshooting and testing).
2060 --remove-empty-dirs Remove empty directories at the final cleanup step.
2061 -1, --resync Performs the resync run. Path1 files may overwrite Path2 versions. Consider using --verbose or --dry-run first.
2062 --workdir string Use custom working dir - useful for testing. (default: $HOME/.cache/rclone/bisync)
2063
2064 See the global flags page (https://rclone.org/flags/) for global op‐
2065 tions not listed here.
2066
2067 SEE ALSO
2068 • rclone (https://rclone.org/commands/rclone/) - Show help for rclone
2069 commands, flags and backends.
2070
2072 Concatenates any files and sends them to stdout.
2073
2074 Synopsis
2075 rclone cat sends any files to standard output.
2076
2077 You can use it like this to output a single file
2078
2079 rclone cat remote:path/to/file
2080
2081 Or like this to output any file in dir or its subdirectories.
2082
2083 rclone cat remote:path/to/dir
2084
2085 Or like this to output any .txt files in dir or its subdirectories.
2086
2087 rclone --include "*.txt" cat remote:path/to/dir
2088
2089 Use the --head flag to print characters only at the start, --tail for
2090 the end and --offset and --count to print a section in the middle.
2091 Note that if offset is negative it will count from the end, so --offset
2092 -1 --count 1 is equivalent to --tail 1.
2093
2094 rclone cat remote:path [flags]
2095
2096 Options
2097 --count int Only print N characters (default -1)
2098 --discard Discard the output instead of printing
2099 --head int Only print the first N characters
2100 -h, --help help for cat
2101 --offset int Start printing at offset N (or from end if -ve)
2102 --tail int Only print the last N characters
2103
2104 See the global flags page (https://rclone.org/flags/) for global op‐
2105 tions not listed here.
2106
2107 SEE ALSO
2108 • rclone (https://rclone.org/commands/rclone/) - Show help for rclone
2109 commands, flags and backends.
2110
2112 Checks the files in the source against a SUM file.
2113
2114 Synopsis
2115 Checks that hashsums of source files match the SUM file. It compares
2116 hashes (MD5, SHA1, etc) and logs a report of files which don't match.
2117 It doesn't alter the file system.
2118
2119 If you supply the --download flag, it will download the data from re‐
2120 mote and calculate the contents hash on the fly. This can be useful
2121 for remotes that don't support hashes or if you really want to check
2122 all the data.
2123
2124 Note that hash values in the SUM file are treated as case insensitive.
2125
2126 If you supply the --one-way flag, it will only check that files in the
2127 source match the files in the destination, not the other way around.
2128 This means that extra files in the destination that are not in the
2129 source will not be detected.
2130
2131 The --differ, --missing-on-dst, --missing-on-src, --match and --error
2132 flags write paths, one per line, to the file name (or stdout if it is
2133 -) supplied. What they write is described in the help below. For ex‐
2134 ample --differ will write all paths which are present on both the
2135 source and destination but different.
2136
2137 The --combined flag will write a file (or stdout) which contains all
2138 file paths with a symbol and then a space and then the path to tell you
2139 what happened to it. These are reminiscent of diff files.
2140
2141 • = path means path was found in source and destination and was identi‐
2142 cal
2143
2144 • `- path` means path was missing on the source, so only in the desti‐
2145 nation
2146
2147 • `+ path` means path was missing on the destination, so only in the
2148 source
2149
2150 • `* path` means path was present in source and destination but differ‐
2151 ent.
2152
2153 • ! path means there was an error reading or hashing the source or
2154 dest.
2155
2156 rclone checksum <hash> sumfile src:path [flags]
2157
2158 Options
2159 --combined string Make a combined report of changes to this file
2160 --differ string Report all non-matching files to this file
2161 --download Check by hashing the contents
2162 --error string Report all files with errors (hashing or reading) to this file
2163 -h, --help help for checksum
2164 --match string Report all matching files to this file
2165 --missing-on-dst string Report all files missing from the destination to this file
2166 --missing-on-src string Report all files missing from the source to this file
2167 --one-way Check one way only, source files must exist on remote
2168
2169 See the global flags page (https://rclone.org/flags/) for global op‐
2170 tions not listed here.
2171
2172 SEE ALSO
2173 • rclone (https://rclone.org/commands/rclone/) - Show help for rclone
2174 commands, flags and backends.
2175
2177 Generate the autocompletion script for the specified shell
2178
2179 Synopsis
2180 Generate the autocompletion script for rclone for the specified shell.
2181 See each sub-command's help for details on how to use the generated
2182 script.
2183
2184 Options
2185 -h, --help help for completion
2186
2187 See the global flags page (https://rclone.org/flags/) for global op‐
2188 tions not listed here.
2189
2190 SEE ALSO
2191 • rclone (https://rclone.org/commands/rclone/) - Show help for rclone
2192 commands, flags and backends.
2193
2194 • rclone completion bash (https://rclone.org/commands/rclone_comple‐
2195 tion_bash/) - Generate the autocompletion script for bash
2196
2197 • rclone completion fish (https://rclone.org/commands/rclone_comple‐
2198 tion_fish/) - Generate the autocompletion script for fish
2199
2200 • rclone completion powershell (https://rclone.org/commands/rclone_com‐
2201 pletion_powershell/) - Generate the autocompletion script for power‐
2202 shell
2203
2204 • rclone completion zsh (https://rclone.org/commands/rclone_comple‐
2205 tion_zsh/) - Generate the autocompletion script for zsh
2206
2208 Generate the autocompletion script for bash
2209
2210 Synopsis
2211 Generate the autocompletion script for the bash shell.
2212
2213 This script depends on the 'bash-completion' package. If it is not in‐
2214 stalled already, you can install it via your OS's package manager.
2215
2216 To load completions in your current shell session:
2217
2218 source <(rclone completion bash)
2219
2220 To load completions for every new session, execute once:
2221
2222 Linux:
2223 rclone completion bash > /etc/bash_completion.d/rclone
2224
2225 macOS:
2226 rclone completion bash > $(brew --prefix)/etc/bash_completion.d/rclone
2227
2228 You will need to start a new shell for this setup to take effect.
2229
2230 rclone completion bash
2231
2232 Options
2233 -h, --help help for bash
2234 --no-descriptions disable completion descriptions
2235
2236 See the global flags page (https://rclone.org/flags/) for global op‐
2237 tions not listed here.
2238
2239 SEE ALSO
2240 • rclone completion (https://rclone.org/commands/rclone_completion/) -
2241 Generate the autocompletion script for the specified shell
2242
2244 Generate the autocompletion script for fish
2245
2246 Synopsis
2247 Generate the autocompletion script for the fish shell.
2248
2249 To load completions in your current shell session:
2250
2251 rclone completion fish | source
2252
2253 To load completions for every new session, execute once:
2254
2255 rclone completion fish > ~/.config/fish/completions/rclone.fish
2256
2257 You will need to start a new shell for this setup to take effect.
2258
2259 rclone completion fish [flags]
2260
2261 Options
2262 -h, --help help for fish
2263 --no-descriptions disable completion descriptions
2264
2265 See the global flags page (https://rclone.org/flags/) for global op‐
2266 tions not listed here.
2267
2268 SEE ALSO
2269 • rclone completion (https://rclone.org/commands/rclone_completion/) -
2270 Generate the autocompletion script for the specified shell
2271
2273 Generate the autocompletion script for powershell
2274
2275 Synopsis
2276 Generate the autocompletion script for powershell.
2277
2278 To load completions in your current shell session:
2279
2280 rclone completion powershell | Out-String | Invoke-Expression
2281
2282 To load completions for every new session, add the output of the above
2283 command to your powershell profile.
2284
2285 rclone completion powershell [flags]
2286
2287 Options
2288 -h, --help help for powershell
2289 --no-descriptions disable completion descriptions
2290
2291 See the global flags page (https://rclone.org/flags/) for global op‐
2292 tions not listed here.
2293
2294 SEE ALSO
2295 • rclone completion (https://rclone.org/commands/rclone_completion/) -
2296 Generate the autocompletion script for the specified shell
2297
2299 Generate the autocompletion script for zsh
2300
2301 Synopsis
2302 Generate the autocompletion script for the zsh shell.
2303
2304 If shell completion is not already enabled in your environment you will
2305 need to enable it. You can execute the following once:
2306
2307 echo "autoload -U compinit; compinit" >> ~/.zshrc
2308
2309 To load completions in your current shell session:
2310
2311 source <(rclone completion zsh); compdef _rclone rclone
2312
2313 To load completions for every new session, execute once:
2314
2315 Linux:
2316 rclone completion zsh > "${fpath[1]}/_rclone"
2317
2318 macOS:
2319 rclone completion zsh > $(brew --prefix)/share/zsh/site-functions/_rclone
2320
2321 You will need to start a new shell for this setup to take effect.
2322
2323 rclone completion zsh [flags]
2324
2325 Options
2326 -h, --help help for zsh
2327 --no-descriptions disable completion descriptions
2328
2329 See the global flags page (https://rclone.org/flags/) for global op‐
2330 tions not listed here.
2331
2332 SEE ALSO
2333 • rclone completion (https://rclone.org/commands/rclone_completion/) -
2334 Generate the autocompletion script for the specified shell
2335
2337 Create a new remote with name, type and options.
2338
2339 Synopsis
2340 Create a new remote of name with type and options. The options should
2341 be passed in pairs of key value or as key=value.
2342
2343 For example, to make a swift remote of name myremote using auto config
2344 you would do:
2345
2346 rclone config create myremote swift env_auth true
2347 rclone config create myremote swift env_auth=true
2348
2349 So for example if you wanted to configure a Google Drive remote but us‐
2350 ing remote authorization you would do this:
2351
2352 rclone config create mydrive drive config_is_local=false
2353
2354 Note that if the config process would normally ask a question the de‐
2355 fault is taken (unless --non-interactive is used). Each time that hap‐
2356 pens rclone will print or DEBUG a message saying how to affect the val‐
2357 ue taken.
2358
2359 If any of the parameters passed is a password field, then rclone will
2360 automatically obscure them if they aren't already obscured before
2361 putting them in the config file.
2362
2363 NB If the password parameter is 22 characters or longer and consists
2364 only of base64 characters then rclone can get confused about whether
2365 the password is already obscured or not and put unobscured passwords
2366 into the config file. If you want to be 100% certain that the pass‐
2367 words get obscured then use the --obscure flag, or if you are 100% cer‐
2368 tain you are already passing obscured passwords then use --no-obscure.
2369 You can also set obscured passwords using the rclone config password
2370 command.
2371
2372 The flag --non-interactive is for use by applications that wish to con‐
2373 figure rclone themselves, rather than using rclone's text based config‐
2374 uration questions. If this flag is set, and rclone needs to ask the
2375 user a question, a JSON blob will be returned with the question in it.
2376
2377 This will look something like (some irrelevant detail removed):
2378
2379 {
2380 "State": "*oauth-islocal,teamdrive,,",
2381 "Option": {
2382 "Name": "config_is_local",
2383 "Help": "Use auto config?\n * Say Y if not sure\n * Say N if you are working on a remote or headless machine\n",
2384 "Default": true,
2385 "Examples": [
2386 {
2387 "Value": "true",
2388 "Help": "Yes"
2389 },
2390 {
2391 "Value": "false",
2392 "Help": "No"
2393 }
2394 ],
2395 "Required": false,
2396 "IsPassword": false,
2397 "Type": "bool",
2398 "Exclusive": true,
2399 },
2400 "Error": "",
2401 }
2402
2403 The format of Option is the same as returned by rclone config
2404 providers. The question should be asked to the user and returned to
2405 rclone as the --result option along with the --state parameter.
2406
2407 The keys of Option are used as follows:
2408
2409 • Name - name of variable - show to user
2410
2411 • Help - help text. Hard wrapped at 80 chars. Any URLs should be
2412 clicky.
2413
2414 • Default - default value - return this if the user just wants the de‐
2415 fault.
2416
2417 • Examples - the user should be able to choose one of these
2418
2419 • Required - the value should be non-empty
2420
2421 • IsPassword - the value is a password and should be edited as such
2422
2423 • Type - type of value, eg bool, string, int and others
2424
2425 • Exclusive - if set no free-form entry allowed only the Examples
2426
2427 • Irrelevant keys Provider, ShortOpt, Hide, NoPrefix, Advanced
2428
2429 If Error is set then it should be shown to the user at the same time as
2430 the question.
2431
2432 rclone config update name --continue --state "*oauth-islocal,teamdrive,," --result "true"
2433
2434 Note that when using --continue all passwords should be passed in the
2435 clear (not obscured). Any default config values should be passed in
2436 with each invocation of --continue.
2437
2438 At the end of the non interactive process, rclone will return a result
2439 with State as empty string.
2440
2441 If --all is passed then rclone will ask all the config questions, not
2442 just the post config questions. Any parameters are used as defaults
2443 for questions as usual.
2444
2445 Note that bin/config.py in the rclone source implements this protocol
2446 as a readable demonstration.
2447
2448 rclone config create name type [key value]* [flags]
2449
2450 Options
2451 --all Ask the full set of config questions
2452 --continue Continue the configuration process with an answer
2453 -h, --help help for create
2454 --no-obscure Force any passwords not to be obscured
2455 --non-interactive Don't interact with user and return questions
2456 --obscure Force any passwords to be obscured
2457 --result string Result - use with --continue
2458 --state string State - use with --continue
2459
2460 See the global flags page (https://rclone.org/flags/) for global op‐
2461 tions not listed here.
2462
2463 SEE ALSO
2464 • rclone config (https://rclone.org/commands/rclone_config/) - Enter an
2465 interactive configuration session.
2466
2468 Delete an existing remote.
2469
2470 rclone config delete name [flags]
2471
2472 Options
2473 -h, --help help for delete
2474
2475 See the global flags page (https://rclone.org/flags/) for global op‐
2476 tions not listed here.
2477
2478 SEE ALSO
2479 • rclone config (https://rclone.org/commands/rclone_config/) - Enter an
2480 interactive configuration session.
2481
2483 Disconnects user from remote
2484
2485 Synopsis
2486 This disconnects the remote: passed in to the cloud storage system.
2487
2488 This normally means revoking the oauth token.
2489
2490 To reconnect use "rclone config reconnect".
2491
2492 rclone config disconnect remote: [flags]
2493
2494 Options
2495 -h, --help help for disconnect
2496
2497 See the global flags page (https://rclone.org/flags/) for global op‐
2498 tions not listed here.
2499
2500 SEE ALSO
2501 • rclone config (https://rclone.org/commands/rclone_config/) - Enter an
2502 interactive configuration session.
2503
2505 Dump the config file as JSON.
2506
2507 rclone config dump [flags]
2508
2509 Options
2510 -h, --help help for dump
2511
2512 See the global flags page (https://rclone.org/flags/) for global op‐
2513 tions not listed here.
2514
2515 SEE ALSO
2516 • rclone config (https://rclone.org/commands/rclone_config/) - Enter an
2517 interactive configuration session.
2518
2520 Enter an interactive configuration session.
2521
2523 Enter an interactive configuration session where you can setup new re‐
2524 motes and manage existing ones. You may also set or remove a password
2525 to protect your configuration.
2526
2527 rclone config edit [flags]
2528
2530 -h, --help help for edit
2531
2532 See the global flags page (https://rclone.org/flags/) for global op‐
2533 tions not listed here.
2534
2536 • rclone config (https://rclone.org/commands/rclone_config/) - Enter an
2537 interactive configuration session.
2538
2540 Show path of configuration file in use.
2541
2542 rclone config file [flags]
2543
2544 Options
2545 -h, --help help for file
2546
2547 See the global flags page (https://rclone.org/flags/) for global op‐
2548 tions not listed here.
2549
2550 SEE ALSO
2551 • rclone config (https://rclone.org/commands/rclone_config/) - Enter an
2552 interactive configuration session.
2553
2555 Update password in an existing remote.
2556
2557 Synopsis
2558 Update an existing remote's password. The password should be passed in
2559 pairs of key password or as key=password. The password should be
2560 passed in in clear (unobscured).
2561
2562 For example, to set password of a remote of name myremote you would do:
2563
2564 rclone config password myremote fieldname mypassword
2565 rclone config password myremote fieldname=mypassword
2566
2567 This command is obsolete now that "config update" and "config create"
2568 both support obscuring passwords directly.
2569
2570 rclone config password name [key value]+ [flags]
2571
2572 Options
2573 -h, --help help for password
2574
2575 See the global flags page (https://rclone.org/flags/) for global op‐
2576 tions not listed here.
2577
2578 SEE ALSO
2579 • rclone config (https://rclone.org/commands/rclone_config/) - Enter an
2580 interactive configuration session.
2581
2583 Show paths used for configuration, cache, temp etc.
2584
2585 rclone config paths [flags]
2586
2587 Options
2588 -h, --help help for paths
2589
2590 See the global flags page (https://rclone.org/flags/) for global op‐
2591 tions not listed here.
2592
2593 SEE ALSO
2594 • rclone config (https://rclone.org/commands/rclone_config/) - Enter an
2595 interactive configuration session.
2596
2598 List in JSON format all the providers and options.
2599
2600 rclone config providers [flags]
2601
2602 Options
2603 -h, --help help for providers
2604
2605 See the global flags page (https://rclone.org/flags/) for global op‐
2606 tions not listed here.
2607
2608 SEE ALSO
2609 • rclone config (https://rclone.org/commands/rclone_config/) - Enter an
2610 interactive configuration session.
2611
2613 Re-authenticates user with remote.
2614
2615 Synopsis
2616 This reconnects remote: passed in to the cloud storage system.
2617
2618 To disconnect the remote use "rclone config disconnect".
2619
2620 This normally means going through the interactive oauth flow again.
2621
2622 rclone config reconnect remote: [flags]
2623
2624 Options
2625 -h, --help help for reconnect
2626
2627 See the global flags page (https://rclone.org/flags/) for global op‐
2628 tions not listed here.
2629
2630 SEE ALSO
2631 • rclone config (https://rclone.org/commands/rclone_config/) - Enter an
2632 interactive configuration session.
2633
2635 Print (decrypted) config file, or the config for a single remote.
2636
2637 rclone config show [<remote>] [flags]
2638
2639 Options
2640 -h, --help help for show
2641
2642 See the global flags page (https://rclone.org/flags/) for global op‐
2643 tions not listed here.
2644
2645 SEE ALSO
2646 • rclone config (https://rclone.org/commands/rclone_config/) - Enter an
2647 interactive configuration session.
2648
2650 Ensure configuration file exists.
2651
2652 rclone config touch [flags]
2653
2654 Options
2655 -h, --help help for touch
2656
2657 See the global flags page (https://rclone.org/flags/) for global op‐
2658 tions not listed here.
2659
2660 SEE ALSO
2661 • rclone config (https://rclone.org/commands/rclone_config/) - Enter an
2662 interactive configuration session.
2663
2665 Update options in an existing remote.
2666
2667 Synopsis
2668 Update an existing remote's options. The options should be passed in
2669 pairs of key value or as key=value.
2670
2671 For example, to update the env_auth field of a remote of name myremote
2672 you would do:
2673
2674 rclone config update myremote env_auth true
2675 rclone config update myremote env_auth=true
2676
2677 If the remote uses OAuth the token will be updated, if you don't re‐
2678 quire this add an extra parameter thus:
2679
2680 rclone config update myremote env_auth=true config_refresh_token=false
2681
2682 Note that if the config process would normally ask a question the de‐
2683 fault is taken (unless --non-interactive is used). Each time that hap‐
2684 pens rclone will print or DEBUG a message saying how to affect the val‐
2685 ue taken.
2686
2687 If any of the parameters passed is a password field, then rclone will
2688 automatically obscure them if they aren't already obscured before
2689 putting them in the config file.
2690
2691 NB If the password parameter is 22 characters or longer and consists
2692 only of base64 characters then rclone can get confused about whether
2693 the password is already obscured or not and put unobscured passwords
2694 into the config file. If you want to be 100% certain that the pass‐
2695 words get obscured then use the --obscure flag, or if you are 100% cer‐
2696 tain you are already passing obscured passwords then use --no-obscure.
2697 You can also set obscured passwords using the rclone config password
2698 command.
2699
2700 The flag --non-interactive is for use by applications that wish to con‐
2701 figure rclone themselves, rather than using rclone's text based config‐
2702 uration questions. If this flag is set, and rclone needs to ask the
2703 user a question, a JSON blob will be returned with the question in it.
2704
2705 This will look something like (some irrelevant detail removed):
2706
2707 {
2708 "State": "*oauth-islocal,teamdrive,,",
2709 "Option": {
2710 "Name": "config_is_local",
2711 "Help": "Use auto config?\n * Say Y if not sure\n * Say N if you are working on a remote or headless machine\n",
2712 "Default": true,
2713 "Examples": [
2714 {
2715 "Value": "true",
2716 "Help": "Yes"
2717 },
2718 {
2719 "Value": "false",
2720 "Help": "No"
2721 }
2722 ],
2723 "Required": false,
2724 "IsPassword": false,
2725 "Type": "bool",
2726 "Exclusive": true,
2727 },
2728 "Error": "",
2729 }
2730
2731 The format of Option is the same as returned by rclone config
2732 providers. The question should be asked to the user and returned to
2733 rclone as the --result option along with the --state parameter.
2734
2735 The keys of Option are used as follows:
2736
2737 • Name - name of variable - show to user
2738
2739 • Help - help text. Hard wrapped at 80 chars. Any URLs should be
2740 clicky.
2741
2742 • Default - default value - return this if the user just wants the de‐
2743 fault.
2744
2745 • Examples - the user should be able to choose one of these
2746
2747 • Required - the value should be non-empty
2748
2749 • IsPassword - the value is a password and should be edited as such
2750
2751 • Type - type of value, eg bool, string, int and others
2752
2753 • Exclusive - if set no free-form entry allowed only the Examples
2754
2755 • Irrelevant keys Provider, ShortOpt, Hide, NoPrefix, Advanced
2756
2757 If Error is set then it should be shown to the user at the same time as
2758 the question.
2759
2760 rclone config update name --continue --state "*oauth-islocal,teamdrive,," --result "true"
2761
2762 Note that when using --continue all passwords should be passed in the
2763 clear (not obscured). Any default config values should be passed in
2764 with each invocation of --continue.
2765
2766 At the end of the non interactive process, rclone will return a result
2767 with State as empty string.
2768
2769 If --all is passed then rclone will ask all the config questions, not
2770 just the post config questions. Any parameters are used as defaults
2771 for questions as usual.
2772
2773 Note that bin/config.py in the rclone source implements this protocol
2774 as a readable demonstration.
2775
2776 rclone config update name [key value]+ [flags]
2777
2778 Options
2779 --all Ask the full set of config questions
2780 --continue Continue the configuration process with an answer
2781 -h, --help help for update
2782 --no-obscure Force any passwords not to be obscured
2783 --non-interactive Don't interact with user and return questions
2784 --obscure Force any passwords to be obscured
2785 --result string Result - use with --continue
2786 --state string State - use with --continue
2787
2788 See the global flags page (https://rclone.org/flags/) for global op‐
2789 tions not listed here.
2790
2791 SEE ALSO
2792 • rclone config (https://rclone.org/commands/rclone_config/) - Enter an
2793 interactive configuration session.
2794
2796 Prints info about logged in user of remote.
2797
2798 Synopsis
2799 This prints the details of the person logged in to the cloud storage
2800 system.
2801
2802 rclone config userinfo remote: [flags]
2803
2804 Options
2805 -h, --help help for userinfo
2806 --json Format output as JSON
2807
2808 See the global flags page (https://rclone.org/flags/) for global op‐
2809 tions not listed here.
2810
2811 SEE ALSO
2812 • rclone config (https://rclone.org/commands/rclone_config/) - Enter an
2813 interactive configuration session.
2814
2816 Copy files from source to dest, skipping identical files.
2817
2818 Synopsis
2819 If source:path is a file or directory then it copies it to a file or
2820 directory named dest:path.
2821
2822 This can be used to upload single files to other than their current
2823 name. If the source is a directory then it acts exactly like the copy
2824 (https://rclone.org/commands/rclone_copy/) command.
2825
2826 So
2827
2828 rclone copyto src dst
2829
2830 where src and dst are rclone paths, either remote:path or /path/to/lo‐
2831 cal or C:.
2832
2833 This will:
2834
2835 if src is file
2836 copy it to dst, overwriting an existing file if it exists
2837 if src is directory
2838 copy it to dst, overwriting existing files if they exist
2839 see copy command for full details
2840
2841 This doesn't transfer files that are identical on src and dst, testing
2842 by size and modification time or MD5SUM. It doesn't delete files from
2843 the destination.
2844
2845 Note: Use the -P/--progress flag to view real-time transfer statistics
2846
2847 rclone copyto source:path dest:path [flags]
2848
2849 Options
2850 -h, --help help for copyto
2851
2852 See the global flags page (https://rclone.org/flags/) for global op‐
2853 tions not listed here.
2854
2855 SEE ALSO
2856 • rclone (https://rclone.org/commands/rclone/) - Show help for rclone
2857 commands, flags and backends.
2858
2860 Copy url content to dest.
2861
2862 Synopsis
2863 Download a URL's content and copy it to the destination without saving
2864 it in temporary storage.
2865
2866 Setting --auto-filename will attempt to automatically determine the
2867 filename from the URL (after any redirections) and used in the destina‐
2868 tion path. With --auto-filename-header in addition, if a specific
2869 filename is set in HTTP headers, it will be used instead of the name
2870 from the URL. With --print-filename in addition, the resulting file
2871 name will be printed.
2872
2873 Setting --no-clobber will prevent overwriting file on the destination
2874 if there is one with the same name.
2875
2876 Setting --stdout or making the output file name - will cause the output
2877 to be written to standard output.
2878
2879 rclone copyurl https://example.com dest:path [flags]
2880
2881 Options
2882 -a, --auto-filename Get the file name from the URL and use it for destination file path
2883 --header-filename Get the file name from the Content-Disposition header
2884 -h, --help help for copyurl
2885 --no-clobber Prevent overwriting file with same name
2886 -p, --print-filename Print the resulting name from --auto-filename
2887 --stdout Write the output to stdout rather than a file
2888
2889 See the global flags page (https://rclone.org/flags/) for global op‐
2890 tions not listed here.
2891
2892 SEE ALSO
2893 • rclone (https://rclone.org/commands/rclone/) - Show help for rclone
2894 commands, flags and backends.
2895
2897 Cryptcheck checks the integrity of a crypted remote.
2898
2899 Synopsis
2900 rclone cryptcheck checks a remote against a crypted
2901 (https://rclone.org/crypt/) remote. This is the equivalent of running
2902 rclone check (https://rclone.org/commands/rclone_check/), but able to
2903 check the checksums of the crypted remote.
2904
2905 For it to work the underlying remote of the cryptedremote must support
2906 some kind of checksum.
2907
2908 It works by reading the nonce from each file on the cryptedremote: and
2909 using that to encrypt each file on the remote:. It then checks the
2910 checksum of the underlying file on the cryptedremote: against the
2911 checksum of the file it has just encrypted.
2912
2913 Use it like this
2914
2915 rclone cryptcheck /path/to/files encryptedremote:path
2916
2917 You can use it like this also, but that will involve downloading all
2918 the files in remote:path.
2919
2920 rclone cryptcheck remote:path encryptedremote:path
2921
2922 After it has run it will log the status of the encryptedremote:.
2923
2924 If you supply the --one-way flag, it will only check that files in the
2925 source match the files in the destination, not the other way around.
2926 This means that extra files in the destination that are not in the
2927 source will not be detected.
2928
2929 The --differ, --missing-on-dst, --missing-on-src, --match and --error
2930 flags write paths, one per line, to the file name (or stdout if it is
2931 -) supplied. What they write is described in the help below. For ex‐
2932 ample --differ will write all paths which are present on both the
2933 source and destination but different.
2934
2935 The --combined flag will write a file (or stdout) which contains all
2936 file paths with a symbol and then a space and then the path to tell you
2937 what happened to it. These are reminiscent of diff files.
2938
2939 • = path means path was found in source and destination and was identi‐
2940 cal
2941
2942 • `- path` means path was missing on the source, so only in the desti‐
2943 nation
2944
2945 • `+ path` means path was missing on the destination, so only in the
2946 source
2947
2948 • `* path` means path was present in source and destination but differ‐
2949 ent.
2950
2951 • ! path means there was an error reading or hashing the source or
2952 dest.
2953
2954 rclone cryptcheck remote:path cryptedremote:path [flags]
2955
2956 Options
2957 --combined string Make a combined report of changes to this file
2958 --differ string Report all non-matching files to this file
2959 --error string Report all files with errors (hashing or reading) to this file
2960 -h, --help help for cryptcheck
2961 --match string Report all matching files to this file
2962 --missing-on-dst string Report all files missing from the destination to this file
2963 --missing-on-src string Report all files missing from the source to this file
2964 --one-way Check one way only, source files must exist on remote
2965
2966 See the global flags page (https://rclone.org/flags/) for global op‐
2967 tions not listed here.
2968
2969 SEE ALSO
2970 • rclone (https://rclone.org/commands/rclone/) - Show help for rclone
2971 commands, flags and backends.
2972
2974 Cryptdecode returns unencrypted file names.
2975
2976 Synopsis
2977 rclone cryptdecode returns unencrypted file names when provided with a
2978 list of encrypted file names. List limit is 10 items.
2979
2980 If you supply the --reverse flag, it will return encrypted file names.
2981
2982 use it like this
2983
2984 rclone cryptdecode encryptedremote: encryptedfilename1 encryptedfilename2
2985
2986 rclone cryptdecode --reverse encryptedremote: filename1 filename2
2987
2988 Another way to accomplish this is by using the rclone backend encode
2989 (or decode) command. See the documentation on the crypt
2990 (https://rclone.org/crypt/) overlay for more info.
2991
2992 rclone cryptdecode encryptedremote: encryptedfilename [flags]
2993
2994 Options
2995 -h, --help help for cryptdecode
2996 --reverse Reverse cryptdecode, encrypts filenames
2997
2998 See the global flags page (https://rclone.org/flags/) for global op‐
2999 tions not listed here.
3000
3001 SEE ALSO
3002 • rclone (https://rclone.org/commands/rclone/) - Show help for rclone
3003 commands, flags and backends.
3004
3006 Remove a single file from remote.
3007
3008 Synopsis
3009 Remove a single file from remote. Unlike delete it cannot be used to
3010 remove a directory and it doesn't obey include/exclude filters - if the
3011 specified file exists, it will always be removed.
3012
3013 rclone deletefile remote:path [flags]
3014
3015 Options
3016 -h, --help help for deletefile
3017
3018 See the global flags page (https://rclone.org/flags/) for global op‐
3019 tions not listed here.
3020
3021 SEE ALSO
3022 • rclone (https://rclone.org/commands/rclone/) - Show help for rclone
3023 commands, flags and backends.
3024
3026 Output completion script for a given shell.
3027
3028 Synopsis
3029 Generates a shell completion script for rclone. Run with --help to
3030 list the supported shells.
3031
3032 Options
3033 -h, --help help for genautocomplete
3034
3035 See the global flags page (https://rclone.org/flags/) for global op‐
3036 tions not listed here.
3037
3038 SEE ALSO
3039 • rclone (https://rclone.org/commands/rclone/) - Show help for rclone
3040 commands, flags and backends.
3041
3042 • rclone genautocomplete bash (https://rclone.org/commands/rclone_ge‐
3043 nautocomplete_bash/) - Output bash completion script for rclone.
3044
3045 • rclone genautocomplete fish (https://rclone.org/commands/rclone_ge‐
3046 nautocomplete_fish/) - Output fish completion script for rclone.
3047
3048 • rclone genautocomplete zsh (https://rclone.org/commands/rclone_genau‐
3049 tocomplete_zsh/) - Output zsh completion script for rclone.
3050
3052 Output bash completion script for rclone.
3053
3054 Synopsis
3055 Generates a bash shell autocompletion script for rclone.
3056
3057 This writes to /etc/bash_completion.d/rclone by default so will proba‐
3058 bly need to be run with sudo or as root, e.g.
3059
3060 sudo rclone genautocomplete bash
3061
3062 Logout and login again to use the autocompletion scripts, or source
3063 them directly
3064
3065 . /etc/bash_completion
3066
3067 If you supply a command line argument the script will be written there.
3068
3069 If output_file is "-", then the output will be written to stdout.
3070
3071 rclone genautocomplete bash [output_file] [flags]
3072
3073 Options
3074 -h, --help help for bash
3075
3076 See the global flags page (https://rclone.org/flags/) for global op‐
3077 tions not listed here.
3078
3079 SEE ALSO
3080 • rclone genautocomplete (https://rclone.org/commands/rclone_genauto‐
3081 complete/) - Output completion script for a given shell.
3082
3084 Output fish completion script for rclone.
3085
3086 Synopsis
3087 Generates a fish autocompletion script for rclone.
3088
3089 This writes to /etc/fish/completions/rclone.fish by default so will
3090 probably need to be run with sudo or as root, e.g.
3091
3092 sudo rclone genautocomplete fish
3093
3094 Logout and login again to use the autocompletion scripts, or source
3095 them directly
3096
3097 . /etc/fish/completions/rclone.fish
3098
3099 If you supply a command line argument the script will be written there.
3100
3101 If output_file is "-", then the output will be written to stdout.
3102
3103 rclone genautocomplete fish [output_file] [flags]
3104
3105 Options
3106 -h, --help help for fish
3107
3108 See the global flags page (https://rclone.org/flags/) for global op‐
3109 tions not listed here.
3110
3111 SEE ALSO
3112 • rclone genautocomplete (https://rclone.org/commands/rclone_genauto‐
3113 complete/) - Output completion script for a given shell.
3114
3116 Output zsh completion script for rclone.
3117
3118 Synopsis
3119 Generates a zsh autocompletion script for rclone.
3120
3121 This writes to /usr/share/zsh/vendor-completions/_rclone by default so
3122 will probably need to be run with sudo or as root, e.g.
3123
3124 sudo rclone genautocomplete zsh
3125
3126 Logout and login again to use the autocompletion scripts, or source
3127 them directly
3128
3129 autoload -U compinit && compinit
3130
3131 If you supply a command line argument the script will be written there.
3132
3133 If output_file is "-", then the output will be written to stdout.
3134
3135 rclone genautocomplete zsh [output_file] [flags]
3136
3137 Options
3138 -h, --help help for zsh
3139
3140 See the global flags page (https://rclone.org/flags/) for global op‐
3141 tions not listed here.
3142
3143 SEE ALSO
3144 • rclone genautocomplete (https://rclone.org/commands/rclone_genauto‐
3145 complete/) - Output completion script for a given shell.
3146
3148 Output markdown docs for rclone to the directory supplied.
3149
3150 Synopsis
3151 This produces markdown docs for the rclone commands to the directory
3152 supplied. These are in a format suitable for hugo to render into the
3153 rclone.org website.
3154
3155 rclone gendocs output_directory [flags]
3156
3157 Options
3158 -h, --help help for gendocs
3159
3160 See the global flags page (https://rclone.org/flags/) for global op‐
3161 tions not listed here.
3162
3163 SEE ALSO
3164 • rclone (https://rclone.org/commands/rclone/) - Show help for rclone
3165 commands, flags and backends.
3166
3168 Produces a hashsum file for all the objects in the path.
3169
3170 Synopsis
3171 Produces a hash file for all the objects in the path using the hash
3172 named. The output is in the same format as the standard md5sum/sha1sum
3173 tool.
3174
3175 By default, the hash is requested from the remote. If the hash is not
3176 supported by the remote, no hash will be returned. With the download
3177 flag, the file will be downloaded from the remote and hashed locally
3178 enabling any hash for any remote.
3179
3180 For the MD5 and SHA1 algorithms there are also dedicated commands,
3181 md5sum (https://rclone.org/commands/rclone_md5sum/) and sha1sum
3182 (https://rclone.org/commands/rclone_sha1sum/).
3183
3184 This command can also hash data received on standard input (stdin), by
3185 not passing a remote:path, or by passing a hyphen as remote:path when
3186 there is data to read (if not, the hyphen will be treated literally, as
3187 a relative path).
3188
3189 Run without a hash to see the list of all supported hashes, e.g.
3190
3191 $ rclone hashsum
3192 Supported hashes are:
3193 * md5
3194 * sha1
3195 * whirlpool
3196 * crc32
3197 * sha256
3198 * dropbox
3199 * hidrive
3200 * mailru
3201 * quickxor
3202
3203 Then
3204
3205 $ rclone hashsum MD5 remote:path
3206
3207 Note that hash names are case insensitive and values are output in low‐
3208 er case.
3209
3210 rclone hashsum <hash> remote:path [flags]
3211
3212 Options
3213 --base64 Output base64 encoded hashsum
3214 -C, --checkfile string Validate hashes against a given SUM file instead of printing them
3215 --download Download the file and hash it locally; if this flag is not specified, the hash is requested from the remote
3216 -h, --help help for hashsum
3217 --output-file string Output hashsums to a file rather than the terminal
3218
3219 See the global flags page (https://rclone.org/flags/) for global op‐
3220 tions not listed here.
3221
3222 SEE ALSO
3223 • rclone (https://rclone.org/commands/rclone/) - Show help for rclone
3224 commands, flags and backends.
3225
3227 Generate public link to file/folder.
3228
3229 Synopsis
3230 rclone link will create, retrieve or remove a public link to the given
3231 file or folder.
3232
3233 rclone link remote:path/to/file
3234 rclone link remote:path/to/folder/
3235 rclone link --unlink remote:path/to/folder/
3236 rclone link --expire 1d remote:path/to/file
3237
3238 If you supply the --expire flag, it will set the expiration time other‐
3239 wise it will use the default (100 years). Note not all backends sup‐
3240 port the --expire flag - if the backend doesn't support it then the
3241 link returned won't expire.
3242
3243 Use the --unlink flag to remove existing public links to the file or
3244 folder. Note not all backends support "--unlink" flag - those that
3245 don't will just ignore it.
3246
3247 If successful, the last line of the output will contain the link. Ex‐
3248 act capabilities depend on the remote, but the link will always by de‐
3249 fault be created with the least constraints – e.g. no expiry, no pass‐
3250 word protection, accessible without account.
3251
3252 rclone link remote:path [flags]
3253
3254 Options
3255 --expire Duration The amount of time that the link will be valid (default off)
3256 -h, --help help for link
3257 --unlink Remove existing public link to file/folder
3258
3259 See the global flags page (https://rclone.org/flags/) for global op‐
3260 tions not listed here.
3261
3262 SEE ALSO
3263 • rclone (https://rclone.org/commands/rclone/) - Show help for rclone
3264 commands, flags and backends.
3265
3267 List all the remotes in the config file.
3268
3269 Synopsis
3270 rclone listremotes lists all the available remotes from the config
3271 file.
3272
3273 When used with the --long flag it lists the types too.
3274
3275 rclone listremotes [flags]
3276
3277 Options
3278 -h, --help help for listremotes
3279 --long Show the type as well as names
3280
3281 See the global flags page (https://rclone.org/flags/) for global op‐
3282 tions not listed here.
3283
3284 SEE ALSO
3285 • rclone (https://rclone.org/commands/rclone/) - Show help for rclone
3286 commands, flags and backends.
3287
3289 List directories and objects in remote:path formatted for parsing.
3290
3291 Synopsis
3292 List the contents of the source path (directories and objects) to stan‐
3293 dard output in a form which is easy to parse by scripts. By default
3294 this will just be the names of the objects and directories, one per
3295 line. The directories will have a / suffix.
3296
3297 Eg
3298
3299 $ rclone lsf swift:bucket
3300 bevajer5jef
3301 canole
3302 diwogej7
3303 ferejej3gux/
3304 fubuwic
3305
3306 Use the --format option to control what gets listed. By default this
3307 is just the path, but you can use these parameters to control the out‐
3308 put:
3309
3310 p - path
3311 s - size
3312 t - modification time
3313 h - hash
3314 i - ID of object
3315 o - Original ID of underlying object
3316 m - MimeType of object if known
3317 e - encrypted name
3318 T - tier of storage if known, e.g. "Hot" or "Cool"
3319 M - Metadata of object in JSON blob format, eg {"key":"value"}
3320
3321 So if you wanted the path, size and modification time, you would use
3322 --format "pst", or maybe --format "tsp" to put the path last.
3323
3324 Eg
3325
3326 $ rclone lsf --format "tsp" swift:bucket
3327 2016-06-25 18:55:41;60295;bevajer5jef
3328 2016-06-25 18:55:43;90613;canole
3329 2016-06-25 18:55:43;94467;diwogej7
3330 2018-04-26 08:50:45;0;ferejej3gux/
3331 2016-06-25 18:55:40;37600;fubuwic
3332
3333 If you specify "h" in the format you will get the MD5 hash by default,
3334 use the --hash flag to change which hash you want. Note that this can
3335 be returned as an empty string if it isn't available on the object (and
3336 for directories), "ERROR" if there was an error reading it from the ob‐
3337 ject and "UNSUPPORTED" if that object does not support that hash type.
3338
3339 For example, to emulate the md5sum command you can use
3340
3341 rclone lsf -R --hash MD5 --format hp --separator " " --files-only .
3342
3343 Eg
3344
3345 $ rclone lsf -R --hash MD5 --format hp --separator " " --files-only swift:bucket
3346 7908e352297f0f530b84a756f188baa3 bevajer5jef
3347 cd65ac234e6fea5925974a51cdd865cc canole
3348 03b5341b4f234b9d984d03ad076bae91 diwogej7
3349 8fd37c3810dd660778137ac3a66cc06d fubuwic
3350 99713e14a4c4ff553acaf1930fad985b gixacuh7ku
3351
3352 (Though "rclone md5sum ." is an easier way of typing this.)
3353
3354 By default the separator is ";" this can be changed with the --separa‐
3355 tor flag. Note that separators aren't escaped in the path so putting
3356 it last is a good strategy.
3357
3358 Eg
3359
3360 $ rclone lsf --separator "," --format "tshp" swift:bucket
3361 2016-06-25 18:55:41,60295,7908e352297f0f530b84a756f188baa3,bevajer5jef
3362 2016-06-25 18:55:43,90613,cd65ac234e6fea5925974a51cdd865cc,canole
3363 2016-06-25 18:55:43,94467,03b5341b4f234b9d984d03ad076bae91,diwogej7
3364 2018-04-26 08:52:53,0,,ferejej3gux/
3365 2016-06-25 18:55:40,37600,8fd37c3810dd660778137ac3a66cc06d,fubuwic
3366
3367 You can output in CSV standard format. This will escape things in " if
3368 they contain ,
3369
3370 Eg
3371
3372 $ rclone lsf --csv --files-only --format ps remote:path
3373 test.log,22355
3374 test.sh,449
3375 "this file contains a comma, in the file name.txt",6
3376
3377 Note that the --absolute parameter is useful for making lists of files
3378 to pass to an rclone copy with the --files-from-raw flag.
3379
3380 For example, to find all the files modified within one day and copy
3381 those only (without traversing the whole directory structure):
3382
3383 rclone lsf --absolute --files-only --max-age 1d /path/to/local > new_files
3384 rclone copy --files-from-raw new_files /path/to/local remote:path
3385
3386 Any of the filtering options can be applied to this command.
3387
3388 There are several related list commands
3389
3390 • ls to list size and path of objects only
3391
3392 • lsl to list modification time, size and path of objects only
3393
3394 • lsd to list directories only
3395
3396 • lsf to list objects and directories in easy to parse format
3397
3398 • lsjson to list objects and directories in JSON format
3399
3400 ls,lsl,lsd are designed to be human-readable. lsf is designed to be
3401 human and machine-readable. lsjson is designed to be machine-readable.
3402
3403 Note that ls and lsl recurse by default - use --max-depth 1 to stop the
3404 recursion.
3405
3406 The other list commands lsd,lsf,lsjson do not recurse by default - use
3407 -R to make them recurse.
3408
3409 Listing a nonexistent directory will produce an error except for re‐
3410 motes which can't have empty directories (e.g. s3, swift, or gcs - the
3411 bucket-based remotes).
3412
3413 rclone lsf remote:path [flags]
3414
3415 Options
3416 --absolute Put a leading / in front of path names
3417 --csv Output in CSV format
3418 -d, --dir-slash Append a slash to directory names (default true)
3419 --dirs-only Only list directories
3420 --files-only Only list files
3421 -F, --format string Output format - see help for details (default "p")
3422 --hash h Use this hash when h is used in the format MD5|SHA-1|DropboxHash (default "md5")
3423 -h, --help help for lsf
3424 -R, --recursive Recurse into the listing
3425 -s, --separator string Separator for the items in the format (default ";")
3426
3427 See the global flags page (https://rclone.org/flags/) for global op‐
3428 tions not listed here.
3429
3430 SEE ALSO
3431 • rclone (https://rclone.org/commands/rclone/) - Show help for rclone
3432 commands, flags and backends.
3433
3435 List directories and objects in the path in JSON format.
3436
3437 Synopsis
3438 List directories and objects in the path in JSON format.
3439
3440 The output is an array of Items, where each Item looks like this
3441
3442 {
3443 "Hashes" : {
3444 "SHA-1" : "f572d396fae9206628714fb2ce00f72e94f2258f",
3445 "MD5" : "b1946ac92492d2347c6235b4d2611184",
3446 "DropboxHash" : "ecb65bb98f9d905b70458986c39fcbad7715e5f2fcc3b1f07767d7c83e2438cc"
3447 },
3448 "ID": "y2djkhiujf83u33",
3449 "OrigID": "UYOJVTUW00Q1RzTDA",
3450 "IsBucket" : false,
3451 "IsDir" : false,
3452 "MimeType" : "application/octet-stream",
3453 "ModTime" : "2017-05-31T16:15:57.034468261+01:00",
3454 "Name" : "file.txt",
3455 "Encrypted" : "v0qpsdq8anpci8n929v3uu9338",
3456 "EncryptedPath" : "kja9098349023498/v0qpsdq8anpci8n929v3uu9338",
3457 "Path" : "full/path/goes/here/file.txt",
3458 "Size" : 6,
3459 "Tier" : "hot",
3460 }
3461
3462 If --hash is not specified the Hashes property won't be emitted. The
3463 types of hash can be specified with the --hash-type parameter (which
3464 may be repeated). If --hash-type is set then it implies --hash.
3465
3466 If --no-modtime is specified then ModTime will be blank. This can
3467 speed things up on remotes where reading the ModTime takes an extra re‐
3468 quest (e.g. s3, swift).
3469
3470 If --no-mimetype is specified then MimeType will be blank. This can
3471 speed things up on remotes where reading the MimeType takes an extra
3472 request (e.g. s3, swift).
3473
3474 If --encrypted is not specified the Encrypted won't be emitted.
3475
3476 If --dirs-only is not specified files in addition to directories are
3477 returned
3478
3479 If --files-only is not specified directories in addition to the files
3480 will be returned.
3481
3482 If --metadata is set then an additional Metadata key will be returned.
3483 This will have metadata in rclone standard format as a JSON object.
3484
3485 if --stat is set then a single JSON blob will be returned about the
3486 item pointed to. This will return an error if the item isn't found.
3487 However on bucket based backends (like s3, gcs, b2, azureblob etc) if
3488 the item isn't found it will return an empty directory as it isn't pos‐
3489 sible to tell empty directories from missing directories there.
3490
3491 The Path field will only show folders below the remote path being list‐
3492 ed. If "remote:path" contains the file "subfolder/file.txt", the Path
3493 for "file.txt" will be "subfolder/file.txt", not "remote:path/subfold‐
3494 er/file.txt". When used without --recursive the Path will always be
3495 the same as Name.
3496
3497 If the directory is a bucket in a bucket-based backend, then "IsBucket"
3498 will be set to true. This key won't be present unless it is "true".
3499
3500 The time is in RFC3339 format with up to nanosecond precision. The
3501 number of decimal digits in the seconds will depend on the precision
3502 that the remote can hold the times, so if times are accurate to the
3503 nearest millisecond (e.g. Google Drive) then 3 digits will always be
3504 shown ("2017-05-31T16:15:57.034+01:00") whereas if the times are accu‐
3505 rate to the nearest second (Dropbox, Box, WebDav, etc.) no digits will
3506 be shown ("2017-05-31T16:15:57+01:00").
3507
3508 The whole output can be processed as a JSON blob, or alternatively it
3509 can be processed line by line as each item is written one to a line.
3510
3511 Any of the filtering options can be applied to this command.
3512
3513 There are several related list commands
3514
3515 • ls to list size and path of objects only
3516
3517 • lsl to list modification time, size and path of objects only
3518
3519 • lsd to list directories only
3520
3521 • lsf to list objects and directories in easy to parse format
3522
3523 • lsjson to list objects and directories in JSON format
3524
3525 ls,lsl,lsd are designed to be human-readable. lsf is designed to be
3526 human and machine-readable. lsjson is designed to be machine-readable.
3527
3528 Note that ls and lsl recurse by default - use --max-depth 1 to stop the
3529 recursion.
3530
3531 The other list commands lsd,lsf,lsjson do not recurse by default - use
3532 -R to make them recurse.
3533
3534 Listing a nonexistent directory will produce an error except for re‐
3535 motes which can't have empty directories (e.g. s3, swift, or gcs - the
3536 bucket-based remotes).
3537
3538 rclone lsjson remote:path [flags]
3539
3540 Options
3541 --dirs-only Show only directories in the listing
3542 --encrypted Show the encrypted names
3543 --files-only Show only files in the listing
3544 --hash Include hashes in the output (may take longer)
3545 --hash-type stringArray Show only this hash type (may be repeated)
3546 -h, --help help for lsjson
3547 --no-mimetype Don't read the mime type (can speed things up)
3548 --no-modtime Don't read the modification time (can speed things up)
3549 --original Show the ID of the underlying Object
3550 -R, --recursive Recurse into the listing
3551 --stat Just return the info for the pointed to file
3552
3553 See the global flags page (https://rclone.org/flags/) for global op‐
3554 tions not listed here.
3555
3556 SEE ALSO
3557 • rclone (https://rclone.org/commands/rclone/) - Show help for rclone
3558 commands, flags and backends.
3559
3561 Mount the remote as file system on a mountpoint.
3562
3563 Synopsis
3564 rclone mount allows Linux, FreeBSD, macOS and Windows to mount any of
3565 Rclone's cloud storage systems as a file system with FUSE.
3566
3567 First set up your remote using rclone config. Check it works with
3568 rclone ls etc.
3569
3570 On Linux and macOS, you can run mount in either foreground or back‐
3571 ground (aka daemon) mode. Mount runs in foreground mode by default.
3572 Use the --daemon flag to force background mode. On Windows you can run
3573 mount in foreground only, the flag is ignored.
3574
3575 In background mode rclone acts as a generic Unix mount program: the
3576 main program starts, spawns background rclone process to setup and
3577 maintain the mount, waits until success or timeout and exits with ap‐
3578 propriate code (killing the child process if it fails).
3579
3580 On Linux/macOS/FreeBSD start the mount like this, where /path/to/lo‐
3581 cal/mount is an empty existing directory:
3582
3583 rclone mount remote:path/to/files /path/to/local/mount
3584
3585 On Windows you can start a mount in different ways. See below for de‐
3586 tails. If foreground mount is used interactively from a console win‐
3587 dow, rclone will serve the mount and occupy the console so another win‐
3588 dow should be used to work with the mount until rclone is interrupted
3589 e.g. by pressing Ctrl-C.
3590
3591 The following examples will mount to an automatically assigned drive,
3592 to specific drive letter X:, to path C:\path\parent\mount (where parent
3593 directory or drive must exist, and mount must not exist, and is not
3594 supported when mounting as a network drive), and the last example will
3595 mount as network share \\cloud\remote and map it to an automatically
3596 assigned drive:
3597
3598 rclone mount remote:path/to/files *
3599 rclone mount remote:path/to/files X:
3600 rclone mount remote:path/to/files C:\path\parent\mount
3601 rclone mount remote:path/to/files \\cloud\remote
3602
3603 When the program ends while in foreground mode, either via Ctrl+C or
3604 receiving a SIGINT or SIGTERM signal, the mount should be automatically
3605 stopped.
3606
3607 When running in background mode the user will have to stop the mount
3608 manually:
3609
3610 # Linux
3611 fusermount -u /path/to/local/mount
3612 # OS X
3613 umount /path/to/local/mount
3614
3615 The umount operation can fail, for example when the mountpoint is busy.
3616 When that happens, it is the user's responsibility to stop the mount
3617 manually.
3618
3619 The size of the mounted file system will be set according to informa‐
3620 tion retrieved from the remote, the same as returned by the rclone
3621 about (https://rclone.org/commands/rclone_about/) command. Remotes
3622 with unlimited storage may report the used size only, then an addition‐
3623 al 1 PiB of free space is assumed. If the remote does not support
3624 (https://rclone.org/overview/#optional-features) the about feature at
3625 all, then 1 PiB is set as both the total and the free size.
3626
3627 Installing on Windows
3628 To run rclone mount on Windows, you will need to download and install
3629 WinFsp (http://www.secfs.net/winfsp/).
3630
3631 WinFsp (https://github.com/winfsp/winfsp) is an open-source Windows
3632 File System Proxy which makes it easy to write user space file systems
3633 for Windows. It provides a FUSE emulation layer which rclone uses com‐
3634 bination with cgofuse (https://github.com/winfsp/cgofuse). Both of
3635 these packages are by Bill Zissimopoulos who was very helpful during
3636 the implementation of rclone mount for Windows.
3637
3638 Mounting modes on windows
3639 Unlike other operating systems, Microsoft Windows provides a different
3640 filesystem type for network and fixed drives. It optimises access on
3641 the assumption fixed disk drives are fast and reliable, while network
3642 drives have relatively high latency and less reliability. Some set‐
3643 tings can also be differentiated between the two types, for example
3644 that Windows Explorer should just display icons and not create preview
3645 thumbnails for image and video files on network drives.
3646
3647 In most cases, rclone will mount the remote as a normal, fixed disk
3648 drive by default. However, you can also choose to mount it as a remote
3649 network drive, often described as a network share. If you mount an
3650 rclone remote using the default, fixed drive mode and experience unex‐
3651 pected program errors, freezes or other issues, consider mounting as a
3652 network drive instead.
3653
3654 When mounting as a fixed disk drive you can either mount to an unused
3655 drive letter, or to a path representing a nonexistent subdirectory of
3656 an existing parent directory or drive. Using the special value * will
3657 tell rclone to automatically assign the next available drive letter,
3658 starting with Z: and moving backward. Examples:
3659
3660 rclone mount remote:path/to/files *
3661 rclone mount remote:path/to/files X:
3662 rclone mount remote:path/to/files C:\path\parent\mount
3663 rclone mount remote:path/to/files X:
3664
3665 Option --volname can be used to set a custom volume name for the mount‐
3666 ed file system. The default is to use the remote name and path.
3667
3668 To mount as network drive, you can add option --network-mode to your
3669 mount command. Mounting to a directory path is not supported in this
3670 mode, it is a limitation Windows imposes on junctions, so the remote
3671 must always be mounted to a drive letter.
3672
3673 rclone mount remote:path/to/files X: --network-mode
3674
3675 A volume name specified with --volname will be used to create the net‐
3676 work share path. A complete UNC path, such as \\cloud\remote, option‐
3677 ally with path \\cloud\remote\madeup\path, will be used as is. Any
3678 other string will be used as the share part, after a default prefix
3679 \\server\. If no volume name is specified then \\server\share will be
3680 used. You must make sure the volume name is unique when you are mount‐
3681 ing more than one drive, or else the mount command will fail. The
3682 share name will treated as the volume label for the mapped drive, shown
3683 in Windows Explorer etc, while the complete \\server\share will be re‐
3684 ported as the remote UNC path by net use etc, just like a normal net‐
3685 work drive mapping.
3686
3687 If you specify a full network share UNC path with --volname, this will
3688 implicitly set the --network-mode option, so the following two examples
3689 have same result:
3690
3691 rclone mount remote:path/to/files X: --network-mode
3692 rclone mount remote:path/to/files X: --volname \\server\share
3693
3694 You may also specify the network share UNC path as the mountpoint it‐
3695 self. Then rclone will automatically assign a drive letter, same as
3696 with * and use that as mountpoint, and instead use the UNC path speci‐
3697 fied as the volume name, as if it were specified with the --volname op‐
3698 tion. This will also implicitly set the --network-mode option. This
3699 means the following two examples have same result:
3700
3701 rclone mount remote:path/to/files \\cloud\remote
3702 rclone mount remote:path/to/files * --volname \\cloud\remote
3703
3704 There is yet another way to enable network mode, and to set the share
3705 path, and that is to pass the "native" libfuse/WinFsp option directly:
3706 --fuse-flag --VolumePrefix=\server\share. Note that the path must be
3707 with just a single backslash prefix in this case.
3708
3709 Note: In previous versions of rclone this was the only supported meth‐
3710 od.
3711
3712 Read more about drive mapping (https://en.wikipedia.org/wiki/Drive_map‐
3713 ping)
3714
3715 See also Limitations section below.
3716
3717 Windows filesystem permissions
3718 The FUSE emulation layer on Windows must convert between the POSIX-
3719 based permission model used in FUSE, and the permission model used in
3720 Windows, based on access-control lists (ACL).
3721
3722 The mounted filesystem will normally get three entries in its access-
3723 control list (ACL), representing permissions for the POSIX permission
3724 scopes: Owner, group and others. By default, the owner and group will
3725 be taken from the current user, and the built-in group "Everyone" will
3726 be used to represent others. The user/group can be customized with
3727 FUSE options "UserName" and "GroupName", e.g. -o UserName=user123 -o
3728 GroupName="Authenticated Users". The permissions on each entry will be
3729 set according to options --dir-perms and --file-perms, which takes a
3730 value in traditional numeric notation (https://en.wikipedia.org/wi‐
3731 ki/File-system_permissions#Numeric_notation).
3732
3733 The default permissions corresponds to --file-perms 0666 --dir-perms
3734 0777, i.e. read and write permissions to everyone. This means you
3735 will not be able to start any programs from the mount. To be able to
3736 do that you must add execute permissions, e.g. --file-perms 0777
3737 --dir-perms 0777 to add it to everyone. If the program needs to write
3738 files, chances are you will have to enable VFS File Caching as well
3739 (see also limitations).
3740
3741 Note that the mapping of permissions is not always trivial, and the re‐
3742 sult you see in Windows Explorer may not be exactly like you expected.
3743 For example, when setting a value that includes write access, this will
3744 be mapped to individual permissions "write attributes", "write data"
3745 and "append data", but not "write extended attributes". Windows will
3746 then show this as basic permission "Special" instead of "Write", be‐
3747 cause "Write" includes the "write extended attributes" permission.
3748
3749 If you set POSIX permissions for only allowing access to the owner, us‐
3750 ing --file-perms 0600 --dir-perms 0700, the user group and the built-in
3751 "Everyone" group will still be given some special permissions, such as
3752 "read attributes" and "read permissions", in Windows. This is done for
3753 compatibility reasons, e.g. to allow users without additional permis‐
3754 sions to be able to read basic metadata about files like in UNIX. One
3755 case that may arise is that other programs (incorrectly) interprets
3756 this as the file being accessible by everyone. For example an SSH
3757 client may warn about "unprotected private key file".
3758
3759 WinFsp 2021 (version 1.9) introduces a new FUSE option "FileSecurity",
3760 that allows the complete specification of file security descriptors us‐
3761 ing SDDL (https://docs.microsoft.com/en-us/windows/win32/secauthz/secu‐
3762 rity-descriptor-string-format). With this you can work around issues
3763 such as the mentioned "unprotected private key file" by specifying -o
3764 FileSecurity="D:P(A;;FA;;;OW)", for file all access (FA) to the owner
3765 (OW).
3766
3767 Windows caveats
3768 Drives created as Administrator are not visible to other accounts, not
3769 even an account that was elevated to Administrator with the User Ac‐
3770 count Control (UAC) feature. A result of this is that if you mount to
3771 a drive letter from a Command Prompt run as Administrator, and then try
3772 to access the same drive from Windows Explorer (which does not run as
3773 Administrator), you will not be able to see the mounted drive.
3774
3775 If you don't need to access the drive from applications running with
3776 administrative privileges, the easiest way around this is to always
3777 create the mount from a non-elevated command prompt.
3778
3779 To make mapped drives available to the user account that created them
3780 regardless if elevated or not, there is a special Windows setting
3781 called linked connections (https://docs.microsoft.com/en-us/trou‐
3782 bleshoot/windows-client/networking/mapped-drives-not-available-from-el‐
3783 evated-command#detail-to-configure-the-enablelinkedconnections-reg‐
3784 istry-entry) that can be enabled.
3785
3786 It is also possible to make a drive mount available to everyone on the
3787 system, by running the process creating it as the built-in SYSTEM ac‐
3788 count. There are several ways to do this: One is to use the command-
3789 line utility PsExec (https://docs.microsoft.com/en-us/sysinter‐
3790 nals/downloads/psexec), from Microsoft's Sysinternals suite, which has
3791 option -s to start processes as the SYSTEM account. Another alterna‐
3792 tive is to run the mount command from a Windows Scheduled Task, or a
3793 Windows Service, configured to run as the SYSTEM account. A third al‐
3794 ternative is to use the WinFsp.Launcher infrastructure
3795 (https://github.com/winfsp/winfsp/wiki/WinFsp-Service-Architecture)).
3796 Note that when running rclone as another user, it will not use the con‐
3797 figuration file from your profile unless you tell it to with the --con‐
3798 fig (https://rclone.org/docs/#config-config-file) option. Read more in
3799 the install documentation (https://rclone.org/install/).
3800
3801 Note that mapping to a directory path, instead of a drive letter, does
3802 not suffer from the same limitations.
3803
3804 Limitations
3805 Without the use of --vfs-cache-mode this can only write files sequen‐
3806 tially, it can only seek when reading. This means that many applica‐
3807 tions won't work with their files on an rclone mount without --vfs-
3808 cache-mode writes or --vfs-cache-mode full. See the VFS File Caching
3809 section for more info.
3810
3811 The bucket-based remotes (e.g. Swift, S3, Google Compute Storage, B2)
3812 do not support the concept of empty directories, so empty directories
3813 will have a tendency to disappear once they fall out of the directory
3814 cache.
3815
3816 When rclone mount is invoked on Unix with --daemon flag, the main
3817 rclone program will wait for the background mount to become ready or
3818 until the timeout specified by the --daemon-wait flag. On Linux it can
3819 check mount status using ProcFS so the flag in fact sets maximum time
3820 to wait, while the real wait can be less. On macOS / BSD the time to
3821 wait is constant and the check is performed only at the end. We advise
3822 you to set wait time on macOS reasonably.
3823
3824 Only supported on Linux, FreeBSD, OS X and Windows at the moment.
3825
3826 rclone mount vs rclone sync/copy
3827 File systems expect things to be 100% reliable, whereas cloud storage
3828 systems are a long way from 100% reliable. The rclone sync/copy com‐
3829 mands cope with this with lots of retries. However rclone mount can't
3830 use retries in the same way without making local copies of the uploads.
3831 Look at the VFS File Caching for solutions to make mount more reliable.
3832
3833 Attribute caching
3834 You can use the flag --attr-timeout to set the time the kernel caches
3835 the attributes (size, modification time, etc.) for directory entries.
3836
3837 The default is 1s which caches files just long enough to avoid too many
3838 callbacks to rclone from the kernel.
3839
3840 In theory 0s should be the correct value for filesystems which can
3841 change outside the control of the kernel. However this causes quite a
3842 few problems such as rclone using too much memory
3843 (https://github.com/rclone/rclone/issues/2157), rclone not serving
3844 files to samba (https://forum.rclone.org/t/rclone-1-39-vs-1-40-mount-
3845 issue/5112) and excessive time listing directories
3846 (https://github.com/rclone/rclone/issues/2095#issuecomment-371141147).
3847
3848 The kernel can cache the info about a file for the time given by --at‐
3849 tr-timeout. You may see corruption if the remote file changes length
3850 during this window. It will show up as either a truncated file or a
3851 file with garbage on the end. With --attr-timeout 1s this is very un‐
3852 likely but not impossible. The higher you set --attr-timeout the more
3853 likely it is. The default setting of "1s" is the lowest setting which
3854 mitigates the problems above.
3855
3856 If you set it higher (10s or 1m say) then the kernel will call back to
3857 rclone less often making it more efficient, however there is more
3858 chance of the corruption issue above.
3859
3860 If files don't change on the remote outside of the control of rclone
3861 then there is no chance of corruption.
3862
3863 This is the same as setting the attr_timeout option in mount.fuse.
3864
3865 Filters
3866 Note that all the rclone filters can be used to select a subset of the
3867 files to be visible in the mount.
3868
3869 systemd
3870 When running rclone mount as a systemd service, it is possible to use
3871 Type=notify. In this case the service will enter the started state af‐
3872 ter the mountpoint has been successfully set up. Units having the
3873 rclone mount service specified as a requirement will see all files and
3874 folders immediately in this mode.
3875
3876 Note that systemd runs mount units without any environment variables
3877 including PATH or HOME. This means that tilde (~) expansion will not
3878 work and you should provide --config and --cache-dir explicitly as ab‐
3879 solute paths via rclone arguments. Since mounting requires the fuser‐
3880 mount program, rclone will use the fallback PATH of /bin:/usr/bin in
3881 this scenario. Please ensure that fusermount is present on this PATH.
3882
3883 Rclone as Unix mount helper
3884 The core Unix program /bin/mount normally takes the -t FSTYPE argument
3885 then runs the /sbin/mount.FSTYPE helper program passing it mount op‐
3886 tions as -o key=val,... or --opt=.... Automount (classic or systemd)
3887 behaves in a similar way.
3888
3889 rclone by default expects GNU-style flags --key val. To run it as a
3890 mount helper you should symlink rclone binary to /sbin/mount.rclone and
3891 optionally /usr/bin/rclonefs, e.g. ln -s /usr/bin/rclone
3892 /sbin/mount.rclone. rclone will detect it and translate command-line
3893 arguments appropriately.
3894
3895 Now you can run classic mounts like this:
3896
3897 mount sftp1:subdir /mnt/data -t rclone -o vfs_cache_mode=writes,sftp_key_file=/path/to/pem
3898
3899 or create systemd mount units:
3900
3901 # /etc/systemd/system/mnt-data.mount
3902 [Unit]
3903 After=network-online.target
3904 [Mount]
3905 Type=rclone
3906 What=sftp1:subdir
3907 Where=/mnt/data
3908 Options=rw,allow_other,args2env,vfs-cache-mode=writes,config=/etc/rclone.conf,cache-dir=/var/rclone
3909
3910 optionally accompanied by systemd automount unit
3911
3912 # /etc/systemd/system/mnt-data.automount
3913 [Unit]
3914 After=network-online.target
3915 Before=remote-fs.target
3916 [Automount]
3917 Where=/mnt/data
3918 TimeoutIdleSec=600
3919 [Install]
3920 WantedBy=multi-user.target
3921
3922 or add in /etc/fstab a line like
3923
3924 sftp1:subdir /mnt/data rclone rw,noauto,nofail,_netdev,x-systemd.automount,args2env,vfs_cache_mode=writes,config=/etc/rclone.conf,cache_dir=/var/cache/rclone 0 0
3925
3926 or use classic Automountd. Remember to provide explicit con‐
3927 fig=...,cache-dir=... as a workaround for mount units being run without
3928 HOME.
3929
3930 Rclone in the mount helper mode will split -o argument(s) by comma, re‐
3931 place _ by - and prepend -- to get the command-line flags. Options
3932 containing commas or spaces can be wrapped in single or double quotes.
3933 Any inner quotes inside outer quotes of the same type should be dou‐
3934 bled.
3935
3936 Mount option syntax includes a few extra options treated specially:
3937
3938 • env.NAME=VALUE will set an environment variable for the mount
3939 process. This helps with Automountd and Systemd.mount which don't
3940 allow setting custom environment for mount helpers. Typically you
3941 will use env.HTTPS_PROXY=proxy.host:3128 or env.HOME=/root
3942
3943 • command=cmount can be used to run cmount or any other rclone command
3944 rather than the default mount.
3945
3946 • args2env will pass mount options to the mount helper running in back‐
3947 ground via environment variables instead of command line arguments.
3948 This allows to hide secrets from such commands as ps or pgrep.
3949
3950 • vv... will be transformed into appropriate --verbose=N
3951
3952 • standard mount options like x-systemd.automount, _netdev, nosuid and
3953 alike are intended only for Automountd and ignored by rclone.
3954
3955 VFS - Virtual File System
3956 This command uses the VFS layer. This adapts the cloud storage objects
3957 that rclone uses into something which looks much more like a disk fil‐
3958 ing system.
3959
3960 Cloud storage objects have lots of properties which aren't like disk
3961 files - you can't extend them or write to the middle of them, so the
3962 VFS layer has to deal with that. Because there is no one right way of
3963 doing this there are various options explained below.
3964
3965 The VFS layer also implements a directory cache - this caches info
3966 about files and directories (but not the data) in memory.
3967
3968 VFS Directory Cache
3969 Using the --dir-cache-time flag, you can control how long a directory
3970 should be considered up to date and not refreshed from the backend.
3971 Changes made through the VFS will appear immediately or invalidate the
3972 cache.
3973
3974 --dir-cache-time duration Time to cache directory entries for (default 5m0s)
3975 --poll-interval duration Time to wait between polling for changes. Must be smaller than dir-cache-time. Only on supported remotes. Set to 0 to disable (default 1m0s)
3976
3977 However, changes made directly on the cloud storage by the web inter‐
3978 face or a different copy of rclone will only be picked up once the di‐
3979 rectory cache expires if the backend configured does not support
3980 polling for changes. If the backend supports polling, changes will be
3981 picked up within the polling interval.
3982
3983 You can send a SIGHUP signal to rclone for it to flush all directory
3984 caches, regardless of how old they are. Assuming only one rclone in‐
3985 stance is running, you can reset the cache like this:
3986
3987 kill -SIGHUP $(pidof rclone)
3988
3989 If you configure rclone with a remote control then you can use rclone
3990 rc to flush the whole directory cache:
3991
3992 rclone rc vfs/forget
3993
3994 Or individual files or directories:
3995
3996 rclone rc vfs/forget file=path/to/file dir=path/to/dir
3997
3998 VFS File Buffering
3999 The --buffer-size flag determines the amount of memory, that will be
4000 used to buffer data in advance.
4001
4002 Each open file will try to keep the specified amount of data in memory
4003 at all times. The buffered data is bound to one open file and won't be
4004 shared.
4005
4006 This flag is a upper limit for the used memory per open file. The buf‐
4007 fer will only use memory for data that is downloaded but not not yet
4008 read. If the buffer is empty, only a small amount of memory will be
4009 used.
4010
4011 The maximum memory used by rclone for buffering can be up to --buffer-
4012 size * open files.
4013
4014 VFS File Caching
4015 These flags control the VFS file caching options. File caching is nec‐
4016 essary to make the VFS layer appear compatible with a normal file sys‐
4017 tem. It can be disabled at the cost of some compatibility.
4018
4019 For example you'll need to enable VFS caching if you want to read and
4020 write simultaneously to a file. See below for more details.
4021
4022 Note that the VFS cache is separate from the cache backend and you may
4023 find that you need one or the other or both.
4024
4025 --cache-dir string Directory rclone will use for caching.
4026 --vfs-cache-mode CacheMode Cache mode off|minimal|writes|full (default off)
4027 --vfs-cache-max-age duration Max age of objects in the cache (default 1h0m0s)
4028 --vfs-cache-max-size SizeSuffix Max total size of objects in the cache (default off)
4029 --vfs-cache-poll-interval duration Interval to poll the cache for stale objects (default 1m0s)
4030 --vfs-write-back duration Time to writeback files after last use when using cache (default 5s)
4031
4032 If run with -vv rclone will print the location of the file cache. The
4033 files are stored in the user cache file area which is OS dependent but
4034 can be controlled with --cache-dir or setting the appropriate environ‐
4035 ment variable.
4036
4037 The cache has 4 different modes selected by --vfs-cache-mode. The
4038 higher the cache mode the more compatible rclone becomes at the cost of
4039 using disk space.
4040
4041 Note that files are written back to the remote only when they are
4042 closed and if they haven't been accessed for --vfs-write-back seconds.
4043 If rclone is quit or dies with files that haven't been uploaded, these
4044 will be uploaded next time rclone is run with the same flags.
4045
4046 If using --vfs-cache-max-size note that the cache may exceed this size
4047 for two reasons. Firstly because it is only checked every --vfs-cache-
4048 poll-interval. Secondly because open files cannot be evicted from the
4049 cache.
4050
4051 You should not run two copies of rclone using the same VFS cache with
4052 the same or overlapping remotes if using --vfs-cache-mode > off. This
4053 can potentially cause data corruption if you do. You can work around
4054 this by giving each rclone its own cache hierarchy with --cache-dir.
4055 You don't need to worry about this if the remotes in use don't overlap.
4056
4057 --vfs-cache-mode off
4058 In this mode (the default) the cache will read directly from the remote
4059 and write directly to the remote without caching anything on disk.
4060
4061 This will mean some operations are not possible
4062
4063 • Files can't be opened for both read AND write
4064
4065 • Files opened for write can't be seeked
4066
4067 • Existing files opened for write must have O_TRUNC set
4068
4069 • Files open for read with O_TRUNC will be opened write only
4070
4071 • Files open for write only will behave as if O_TRUNC was supplied
4072
4073 • Open modes O_APPEND, O_TRUNC are ignored
4074
4075 • If an upload fails it can't be retried
4076
4077 --vfs-cache-mode minimal
4078 This is very similar to "off" except that files opened for read AND
4079 write will be buffered to disk. This means that files opened for write
4080 will be a lot more compatible, but uses the minimal disk space.
4081
4082 These operations are not possible
4083
4084 • Files opened for write only can't be seeked
4085
4086 • Existing files opened for write must have O_TRUNC set
4087
4088 • Files opened for write only will ignore O_APPEND, O_TRUNC
4089
4090 • If an upload fails it can't be retried
4091
4092 --vfs-cache-mode writes
4093 In this mode files opened for read only are still read directly from
4094 the remote, write only and read/write files are buffered to disk first.
4095
4096 This mode should support all normal file system operations.
4097
4098 If an upload fails it will be retried at exponentially increasing in‐
4099 tervals up to 1 minute.
4100
4101 --vfs-cache-mode full
4102 In this mode all reads and writes are buffered to and from disk. When
4103 data is read from the remote this is buffered to disk as well.
4104
4105 In this mode the files in the cache will be sparse files and rclone
4106 will keep track of which bits of the files it has downloaded.
4107
4108 So if an application only reads the starts of each file, then rclone
4109 will only buffer the start of the file. These files will appear to be
4110 their full size in the cache, but they will be sparse files with only
4111 the data that has been downloaded present in them.
4112
4113 This mode should support all normal file system operations and is oth‐
4114 erwise identical to --vfs-cache-mode writes.
4115
4116 When reading a file rclone will read --buffer-size plus --vfs-read-
4117 ahead bytes ahead. The --buffer-size is buffered in memory whereas the
4118 --vfs-read-ahead is buffered on disk.
4119
4120 When using this mode it is recommended that --buffer-size is not set
4121 too large and --vfs-read-ahead is set large if required.
4122
4123 IMPORTANT not all file systems support sparse files. In particular
4124 FAT/exFAT do not. Rclone will perform very badly if the cache directo‐
4125 ry is on a filesystem which doesn't support sparse files and it will
4126 log an ERROR message if one is detected.
4127
4128 Fingerprinting
4129 Various parts of the VFS use fingerprinting to see if a local file copy
4130 has changed relative to a remote file. Fingerprints are made from:
4131
4132 • size
4133
4134 • modification time
4135
4136 • hash
4137
4138 where available on an object.
4139
4140 On some backends some of these attributes are slow to read (they take
4141 an extra API call per object, or extra work per object).
4142
4143 For example hash is slow with the local and sftp backends as they have
4144 to read the entire file and hash it, and modtime is slow with the s3,
4145 swift, ftp and qinqstor backends because they need to do an extra API
4146 call to fetch it.
4147
4148 If you use the --vfs-fast-fingerprint flag then rclone will not include
4149 the slow operations in the fingerprint. This makes the fingerprinting
4150 less accurate but much faster and will improve the opening time of
4151 cached files.
4152
4153 If you are running a vfs cache over local, s3 or swift backends then
4154 using this flag is recommended.
4155
4156 Note that if you change the value of this flag, the fingerprints of the
4157 files in the cache may be invalidated and the files will need to be
4158 downloaded again.
4159
4160 VFS Chunked Reading
4161 When rclone reads files from a remote it reads them in chunks. This
4162 means that rather than requesting the whole file rclone reads the chunk
4163 specified. This can reduce the used download quota for some remotes by
4164 requesting only chunks from the remote that are actually read, at the
4165 cost of an increased number of requests.
4166
4167 These flags control the chunking:
4168
4169 --vfs-read-chunk-size SizeSuffix Read the source objects in chunks (default 128M)
4170 --vfs-read-chunk-size-limit SizeSuffix Max chunk doubling size (default off)
4171
4172 Rclone will start reading a chunk of size --vfs-read-chunk-size, and
4173 then double the size for each read. When --vfs-read-chunk-size-limit
4174 is specified, and greater than --vfs-read-chunk-size, the chunk size
4175 for each open file will get doubled only until the specified value is
4176 reached. If the value is "off", which is the default, the limit is
4177 disabled and the chunk size will grow indefinitely.
4178
4179 With --vfs-read-chunk-size 100M and --vfs-read-chunk-size-limit 0 the
4180 following parts will be downloaded: 0-100M, 100M-200M, 200M-300M,
4181 300M-400M and so on. When --vfs-read-chunk-size-limit 500M is speci‐
4182 fied, the result would be 0-100M, 100M-300M, 300M-700M, 700M-1200M,
4183 1200M-1700M and so on.
4184
4185 Setting --vfs-read-chunk-size to 0 or "off" disables chunked reading.
4186
4187 VFS Performance
4188 These flags may be used to enable/disable features of the VFS for per‐
4189 formance or other reasons. See also the chunked reading feature.
4190
4191 In particular S3 and Swift benefit hugely from the --no-modtime flag
4192 (or use --use-server-modtime for a slightly different effect) as each
4193 read of the modification time takes a transaction.
4194
4195 --no-checksum Don't compare checksums on up/download.
4196 --no-modtime Don't read/write the modification time (can speed things up).
4197 --no-seek Don't allow seeking in files.
4198 --read-only Only allow read-only access.
4199
4200 Sometimes rclone is delivered reads or writes out of order. Rather
4201 than seeking rclone will wait a short time for the in sequence read or
4202 write to come in. These flags only come into effect when not using an
4203 on disk cache file.
4204
4205 --vfs-read-wait duration Time to wait for in-sequence read before seeking (default 20ms)
4206 --vfs-write-wait duration Time to wait for in-sequence write before giving error (default 1s)
4207
4208 When using VFS write caching (--vfs-cache-mode with value writes or
4209 full), the global flag --transfers can be set to adjust the number of
4210 parallel uploads of modified files from the cache (the related global
4211 flag --checkers has no effect on the VFS).
4212
4213 --transfers int Number of file transfers to run in parallel (default 4)
4214
4215 VFS Case Sensitivity
4216 Linux file systems are case-sensitive: two files can differ only by
4217 case, and the exact case must be used when opening a file.
4218
4219 File systems in modern Windows are case-insensitive but case-preserv‐
4220 ing: although existing files can be opened using any case, the exact
4221 case used to create the file is preserved and available for programs to
4222 query. It is not allowed for two files in the same directory to differ
4223 only by case.
4224
4225 Usually file systems on macOS are case-insensitive. It is possible to
4226 make macOS file systems case-sensitive but that is not the default.
4227
4228 The --vfs-case-insensitive VFS flag controls how rclone handles these
4229 two cases. If its value is "false", rclone passes file names to the
4230 remote as-is. If the flag is "true" (or appears without a value on the
4231 command line), rclone may perform a "fixup" as explained below.
4232
4233 The user may specify a file name to open/delete/rename/etc with a case
4234 different than what is stored on the remote. If an argument refers to
4235 an existing file with exactly the same name, then the case of the ex‐
4236 isting file on the disk will be used. However, if a file name with ex‐
4237 actly the same name is not found but a name differing only by case ex‐
4238 ists, rclone will transparently fixup the name. This fixup happens on‐
4239 ly when an existing file is requested. Case sensitivity of file names
4240 created anew by rclone is controlled by the underlying remote.
4241
4242 Note that case sensitivity of the operating system running rclone (the
4243 target) may differ from case sensitivity of a file system presented by
4244 rclone (the source). The flag controls whether "fixup" is performed to
4245 satisfy the target.
4246
4247 If the flag is not provided on the command line, then its default value
4248 depends on the operating system where rclone runs: "true" on Windows
4249 and macOS, "false" otherwise. If the flag is provided without a value,
4250 then it is "true".
4251
4252 VFS Disk Options
4253 This flag allows you to manually set the statistics about the filing
4254 system. It can be useful when those statistics cannot be read correct‐
4255 ly automatically.
4256
4257 --vfs-disk-space-total-size Manually set the total disk space size (example: 256G, default: -1)
4258
4259 Alternate report of used bytes
4260 Some backends, most notably S3, do not report the amount of bytes used.
4261 If you need this information to be available when running df on the
4262 filesystem, then pass the flag --vfs-used-is-size to rclone. With this
4263 flag set, instead of relying on the backend to report this information,
4264 rclone will scan the whole remote similar to rclone size and compute
4265 the total used space itself.
4266
4267 WARNING. Contrary to rclone size, this flag ignores filters so that the
4268 result is accurate. However, this is very inefficient and may cost
4269 lots of API calls resulting in extra charges. Use it as a last resort
4270 and only with caching.
4271
4272 rclone mount remote:path /path/to/mountpoint [flags]
4273
4274 Options
4275 --allow-non-empty Allow mounting over a non-empty directory (not supported on Windows)
4276 --allow-other Allow access to other users (not supported on Windows)
4277 --allow-root Allow access to root user (not supported on Windows)
4278 --async-read Use asynchronous reads (not supported on Windows) (default true)
4279 --attr-timeout duration Time for which file/directory attributes are cached (default 1s)
4280 --daemon Run mount in background and exit parent process (as background output is suppressed, use --log-file with --log-format=pid,... to monitor) (not supported on Windows)
4281 --daemon-timeout duration Time limit for rclone to respond to kernel (not supported on Windows)
4282 --daemon-wait duration Time to wait for ready mount from daemon (maximum time on Linux, constant sleep time on OSX/BSD) (not supported on Windows) (default 1m0s)
4283 --debug-fuse Debug the FUSE internals - needs -v
4284 --default-permissions Makes kernel enforce access control based on the file mode (not supported on Windows)
4285 --devname string Set the device name - default is remote:path
4286 --dir-cache-time duration Time to cache directory entries for (default 5m0s)
4287 --dir-perms FileMode Directory permissions (default 0777)
4288 --file-perms FileMode File permissions (default 0666)
4289 --fuse-flag stringArray Flags or arguments to be passed direct to libfuse/WinFsp (repeat if required)
4290 --gid uint32 Override the gid field set by the filesystem (not supported on Windows) (default 1000)
4291 -h, --help help for mount
4292 --max-read-ahead SizeSuffix The number of bytes that can be prefetched for sequential reads (not supported on Windows) (default 128Ki)
4293 --network-mode Mount as remote network drive, instead of fixed disk drive (supported on Windows only)
4294 --no-checksum Don't compare checksums on up/download
4295 --no-modtime Don't read/write the modification time (can speed things up)
4296 --no-seek Don't allow seeking in files
4297 --noappledouble Ignore Apple Double (._) and .DS_Store files (supported on OSX only) (default true)
4298 --noapplexattr Ignore all "com.apple.*" extended attributes (supported on OSX only)
4299 -o, --option stringArray Option for libfuse/WinFsp (repeat if required)
4300 --poll-interval duration Time to wait between polling for changes, must be smaller than dir-cache-time and only on supported remotes (set 0 to disable) (default 1m0s)
4301 --read-only Only allow read-only access
4302 --uid uint32 Override the uid field set by the filesystem (not supported on Windows) (default 1000)
4303 --umask int Override the permission bits set by the filesystem (not supported on Windows) (default 2)
4304 --vfs-cache-max-age duration Max age of objects in the cache (default 1h0m0s)
4305 --vfs-cache-max-size SizeSuffix Max total size of objects in the cache (default off)
4306 --vfs-cache-mode CacheMode Cache mode off|minimal|writes|full (default off)
4307 --vfs-cache-poll-interval duration Interval to poll the cache for stale objects (default 1m0s)
4308 --vfs-case-insensitive If a file name not found, find a case insensitive match
4309 --vfs-disk-space-total-size SizeSuffix Specify the total space of disk (default off)
4310 --vfs-fast-fingerprint Use fast (less accurate) fingerprints for change detection
4311 --vfs-read-ahead SizeSuffix Extra read ahead over --buffer-size when using cache-mode full
4312 --vfs-read-chunk-size SizeSuffix Read the source objects in chunks (default 128Mi)
4313 --vfs-read-chunk-size-limit SizeSuffix If greater than --vfs-read-chunk-size, double the chunk size after each chunk read, until the limit is reached ('off' is unlimited) (default off)
4314 --vfs-read-wait duration Time to wait for in-sequence read before seeking (default 20ms)
4315 --vfs-used-is-size rclone size Use the rclone size algorithm for Used size
4316 --vfs-write-back duration Time to writeback files after last use when using cache (default 5s)
4317 --vfs-write-wait duration Time to wait for in-sequence write before giving error (default 1s)
4318 --volname string Set the volume name (supported on Windows and OSX only)
4319 --write-back-cache Makes kernel buffer writes before sending them to rclone (without this, writethrough caching is used) (not supported on Windows)
4320
4321 See the global flags page (https://rclone.org/flags/) for global op‐
4322 tions not listed here.
4323
4324 SEE ALSO
4325 • rclone (https://rclone.org/commands/rclone/) - Show help for rclone
4326 commands, flags and backends.
4327
4329 Move file or directory from source to dest.
4330
4331 Synopsis
4332 If source:path is a file or directory then it moves it to a file or di‐
4333 rectory named dest:path.
4334
4335 This can be used to rename files or upload single files to other than
4336 their existing name. If the source is a directory then it acts exactly
4337 like the move (https://rclone.org/commands/rclone_move/) command.
4338
4339 So
4340
4341 rclone moveto src dst
4342
4343 where src and dst are rclone paths, either remote:path or /path/to/lo‐
4344 cal or C:.
4345
4346 This will:
4347
4348 if src is file
4349 move it to dst, overwriting an existing file if it exists
4350 if src is directory
4351 move it to dst, overwriting existing files if they exist
4352 see move command for full details
4353
4354 This doesn't transfer files that are identical on src and dst, testing
4355 by size and modification time or MD5SUM. src will be deleted on suc‐
4356 cessful transfer.
4357
4358 Important: Since this can cause data loss, test first with the --dry-
4359 run or the --interactive/-i flag.
4360
4361 Note: Use the -P/--progress flag to view real-time transfer statistics.
4362
4363 rclone moveto source:path dest:path [flags]
4364
4365 Options
4366 -h, --help help for moveto
4367
4368 See the global flags page (https://rclone.org/flags/) for global op‐
4369 tions not listed here.
4370
4371 SEE ALSO
4372 • rclone (https://rclone.org/commands/rclone/) - Show help for rclone
4373 commands, flags and backends.
4374
4376 Explore a remote with a text based user interface.
4377
4378 Synopsis
4379 This displays a text based user interface allowing the navigation of a
4380 remote. It is most useful for answering the question - "What is using
4381 all my disk space?".
4382
4383 To make the user interface it first scans the entire remote given and
4384 builds an in memory representation. rclone ncdu can be used during
4385 this scanning phase and you will see it building up the directory
4386 structure as it goes along.
4387
4388 You can interact with the user interface using key presses, press '?'
4389 to toggle the help on and off. The supported keys are:
4390
4391 ↑,↓ or k,j to Move
4392 →,l to enter
4393 ←,h to return
4394 c toggle counts
4395 g toggle graph
4396 a toggle average size in directory
4397 u toggle human-readable format
4398 n,s,C,A sort by name,size,count,average size
4399 d delete file/directory
4400 v select file/directory
4401 V enter visual select mode
4402 D delete selected files/directories
4403 y copy current path to clipboard
4404 Y display current path
4405 ^L refresh screen (fix screen corruption)
4406 ? to toggle help on and off
4407 q/ESC/^c to quit
4408
4409 Listed files/directories may be prefixed by a one-character flag, some
4410 of them combined with a description in brackets at end of line. These
4411 flags have the following meaning:
4412
4413 e means this is an empty directory, i.e. contains no files (but
4414 may contain empty subdirectories)
4415 ~ means this is a directory where some of the files (possibly in
4416 subdirectories) have unknown size, and therefore the directory
4417 size may be underestimated (and average size inaccurate, as it
4418 is average of the files with known sizes).
4419 . means an error occurred while reading a subdirectory, and
4420 therefore the directory size may be underestimated (and average
4421 size inaccurate)
4422 ! means an error occurred while reading this directory
4423
4424 This an homage to the ncdu tool (https://dev.yorhel.nl/ncdu) but for
4425 rclone remotes. It is missing lots of features at the moment but is
4426 useful as it stands.
4427
4428 Note that it might take some time to delete big files/directories. The
4429 UI won't respond in the meantime since the deletion is done syn‐
4430 chronously.
4431
4432 For a non-interactive listing of the remote, see the tree
4433 (https://rclone.org/commands/rclone_tree/) command. To just get the
4434 total size of the remote you can also use the size
4435 (https://rclone.org/commands/rclone_size/) command.
4436
4437 rclone ncdu remote:path [flags]
4438
4439 Options
4440 -h, --help help for ncdu
4441
4442 See the global flags page (https://rclone.org/flags/) for global op‐
4443 tions not listed here.
4444
4445 SEE ALSO
4446 • rclone (https://rclone.org/commands/rclone/) - Show help for rclone
4447 commands, flags and backends.
4448
4450 Obscure password for use in the rclone config file.
4451
4452 Synopsis
4453 In the rclone config file, human-readable passwords are obscured. Ob‐
4454 scuring them is done by encrypting them and writing them out in base64.
4455 This is not a secure way of encrypting these passwords as rclone can
4456 decrypt them - it is to prevent "eyedropping" - namely someone seeing a
4457 password in the rclone config file by accident.
4458
4459 Many equally important things (like access tokens) are not obscured in
4460 the config file. However it is very hard to shoulder surf a 64 charac‐
4461 ter hex token.
4462
4463 This command can also accept a password through STDIN instead of an ar‐
4464 gument by passing a hyphen as an argument. This will use the first
4465 line of STDIN as the password not including the trailing newline.
4466
4467 echo "secretpassword" | rclone obscure -
4468
4469 If there is no data on STDIN to read, rclone obscure will default to
4470 obfuscating the hyphen itself.
4471
4472 If you want to encrypt the config file then please use config file en‐
4473 cryption - see rclone config (https://rclone.org/commands/rclone_con‐
4474 fig/) for more info.
4475
4476 rclone obscure password [flags]
4477
4478 Options
4479 -h, --help help for obscure
4480
4481 See the global flags page (https://rclone.org/flags/) for global op‐
4482 tions not listed here.
4483
4484 SEE ALSO
4485 • rclone (https://rclone.org/commands/rclone/) - Show help for rclone
4486 commands, flags and backends.
4487
4489 Run a command against a running rclone.
4490
4491 Synopsis
4492 This runs a command against a running rclone. Use the --url flag to
4493 specify an non default URL to connect on. This can be either a ":port"
4494 which is taken to mean "http://localhost:port" or a "host:port" which
4495 is taken to mean "http://host:port"
4496
4497 A username and password can be passed in with --user and --pass.
4498
4499 Note that --rc-addr, --rc-user, --rc-pass will be read also for --url,
4500 --user, --pass.
4501
4502 Arguments should be passed in as parameter=value.
4503
4504 The result will be returned as a JSON object by default.
4505
4506 The --json parameter can be used to pass in a JSON blob as an input in‐
4507 stead of key=value arguments. This is the only way of passing in more
4508 complicated values.
4509
4510 The -o/--opt option can be used to set a key "opt" with key, value op‐
4511 tions in the form -o key=value or -o key. It can be repeated as many
4512 times as required. This is useful for rc commands which take the "opt"
4513 parameter which by convention is a dictionary of strings.
4514
4515 -o key=value -o key2
4516
4517 Will place this in the "opt" value
4518
4519 {"key":"value", "key2","")
4520
4521 The -a/--arg option can be used to set strings in the "arg" value. It
4522 can be repeated as many times as required. This is useful for rc com‐
4523 mands which take the "arg" parameter which by convention is a list of
4524 strings.
4525
4526 -a value -a value2
4527
4528 Will place this in the "arg" value
4529
4530 ["value", "value2"]
4531
4532 Use --loopback to connect to the rclone instance running rclone rc.
4533 This is very useful for testing commands without having to run an
4534 rclone rc server, e.g.:
4535
4536 rclone rc --loopback operations/about fs=/
4537
4538 Use rclone rc to see a list of all possible commands.
4539
4540 rclone rc commands parameter [flags]
4541
4542 Options
4543 -a, --arg stringArray Argument placed in the "arg" array
4544 -h, --help help for rc
4545 --json string Input JSON - use instead of key=value args
4546 --loopback If set connect to this rclone instance not via HTTP
4547 --no-output If set, don't output the JSON result
4548 -o, --opt stringArray Option in the form name=value or name placed in the "opt" array
4549 --pass string Password to use to connect to rclone remote control
4550 --url string URL to connect to rclone remote control (default "http://localhost:5572/")
4551 --user string Username to use to rclone remote control
4552
4553 See the global flags page (https://rclone.org/flags/) for global op‐
4554 tions not listed here.
4555
4556 SEE ALSO
4557 • rclone (https://rclone.org/commands/rclone/) - Show help for rclone
4558 commands, flags and backends.
4559
4561 Copies standard input to file on remote.
4562
4563 Synopsis
4564 rclone rcat reads from standard input (stdin) and copies it to a single
4565 remote file.
4566
4567 echo "hello world" | rclone rcat remote:path/to/file
4568 ffmpeg - | rclone rcat remote:path/to/file
4569
4570 If the remote file already exists, it will be overwritten.
4571
4572 rcat will try to upload small files in a single request, which is usu‐
4573 ally more efficient than the streaming/chunked upload endpoints, which
4574 use multiple requests. Exact behaviour depends on the remote. What is
4575 considered a small file may be set through --streaming-upload-cutoff.
4576 Uploading only starts after the cutoff is reached or if the file ends
4577 before that. The data must fit into RAM. The cutoff needs to be small
4578 enough to adhere the limits of your remote, please see there. General‐
4579 ly speaking, setting this cutoff too high will decrease your perfor‐
4580 mance.
4581
4582 Use the --size flag to preallocate the file in advance at the remote
4583 end and actually stream it, even if remote backend doesn't support
4584 streaming.
4585
4586 --size should be the exact size of the input stream in bytes. If the
4587 size of the stream is different in length to the --size passed in then
4588 the transfer will likely fail.
4589
4590 Note that the upload can also not be retried because the data is not
4591 kept around until the upload succeeds. If you need to transfer a lot
4592 of data, you're better off caching locally and then rclone move it to
4593 the destination.
4594
4595 rclone rcat remote:path [flags]
4596
4597 Options
4598 -h, --help help for rcat
4599 --size int File size hint to preallocate (default -1)
4600
4601 See the global flags page (https://rclone.org/flags/) for global op‐
4602 tions not listed here.
4603
4604 SEE ALSO
4605 • rclone (https://rclone.org/commands/rclone/) - Show help for rclone
4606 commands, flags and backends.
4607
4609 Run rclone listening to remote control commands only.
4610
4611 Synopsis
4612 This runs rclone so that it only listens to remote control commands.
4613
4614 This is useful if you are controlling rclone via the rc API.
4615
4616 If you pass in a path to a directory, rclone will serve that directory
4617 for GET requests on the URL passed in. It will also open the URL in
4618 the browser when rclone is run.
4619
4620 See the rc documentation (https://rclone.org/rc/) for more info on the
4621 rc flags.
4622
4623 rclone rcd <path to files to serve>* [flags]
4624
4625 Options
4626 -h, --help help for rcd
4627
4628 See the global flags page (https://rclone.org/flags/) for global op‐
4629 tions not listed here.
4630
4631 SEE ALSO
4632 • rclone (https://rclone.org/commands/rclone/) - Show help for rclone
4633 commands, flags and backends.
4634
4636 Remove empty directories under the path.
4637
4638 Synopsis
4639 This recursively removes any empty directories (including directories
4640 that only contain empty directories), that it finds under the path.
4641 The root path itself will also be removed if it is empty, unless you
4642 supply the --leave-root flag.
4643
4644 Use command rmdir (https://rclone.org/commands/rclone_rmdir/) to delete
4645 just the empty directory given by path, not recurse.
4646
4647 This is useful for tidying up remotes that rclone has left a lot of
4648 empty directories in. For example the delete (https://rclone.org/com‐
4649 mands/rclone_delete/) command will delete files but leave the directory
4650 structure (unless used with option --rmdirs).
4651
4652 To delete a path and any objects in it, use purge
4653 (https://rclone.org/commands/rclone_purge/) command.
4654
4655 rclone rmdirs remote:path [flags]
4656
4657 Options
4658 -h, --help help for rmdirs
4659 --leave-root Do not remove root directory if empty
4660
4661 See the global flags page (https://rclone.org/flags/) for global op‐
4662 tions not listed here.
4663
4664 SEE ALSO
4665 • rclone (https://rclone.org/commands/rclone/) - Show help for rclone
4666 commands, flags and backends.
4667
4669 Update the rclone binary.
4670
4671 Synopsis
4672 This command downloads the latest release of rclone and replaces the
4673 currently running binary. The download is verified with a hashsum and
4674 cryptographically signed signature.
4675
4676 If used without flags (or with implied --stable flag), this command
4677 will install the latest stable release. However, some issues may be
4678 fixed (or features added) only in the latest beta release. In such
4679 cases you should run the command with the --beta flag, i.e. rclone
4680 selfupdate --beta. You can check in advance what version would be in‐
4681 stalled by adding the --check flag, then repeat the command without it
4682 when you are satisfied.
4683
4684 Sometimes the rclone team may recommend you a concrete beta or stable
4685 rclone release to troubleshoot your issue or add a bleeding edge fea‐
4686 ture. The --version VER flag, if given, will update to the concrete
4687 version instead of the latest one. If you omit micro version from VER
4688 (for example 1.53), the latest matching micro version will be used.
4689
4690 Upon successful update rclone will print a message that contains a pre‐
4691 vious version number. You will need it if you later decide to revert
4692 your update for some reason. Then you'll have to note the previous
4693 version and run the following command: rclone selfupdate [--beta] OLD‐
4694 VER. If the old version contains only dots and digits (for example
4695 v1.54.0) then it's a stable release so you won't need the --beta flag.
4696 Beta releases have an additional information similar to v1.54.0-be‐
4697 ta.5111.06f1c0c61. (if you are a developer and use a locally built
4698 rclone, the version number will end with -DEV, you will have to rebuild
4699 it as it obviously can't be distributed).
4700
4701 If you previously installed rclone via a package manager, the package
4702 may include local documentation or configure services. You may wish to
4703 update with the flag --package deb or --package rpm (whichever is cor‐
4704 rect for your OS) to update these too. This command with the default
4705 --package zip will update only the rclone executable so the local manu‐
4706 al may become inaccurate after it.
4707
4708 The rclone mount command (https://rclone.org/commands/rclone_mount/)
4709 may or may not support extended FUSE options depending on the build and
4710 OS. selfupdate will refuse to update if the capability would be dis‐
4711 carded.
4712
4713 Note: Windows forbids deletion of a currently running executable so
4714 this command will rename the old executable to 'rclone.old.exe' upon
4715 success.
4716
4717 Please note that this command was not available before rclone version
4718 1.55. If it fails for you with the message unknown command "selfup‐
4719 date" then you will need to update manually following the install in‐
4720 structions located at https://rclone.org/install/
4721
4722 rclone selfupdate [flags]
4723
4724 Options
4725 --beta Install beta release
4726 --check Check for latest release, do not download
4727 -h, --help help for selfupdate
4728 --output string Save the downloaded binary at a given path (default: replace running binary)
4729 --package string Package format: zip|deb|rpm (default: zip)
4730 --stable Install stable release (this is the default)
4731 --version string Install the given rclone version (default: latest)
4732
4733 See the global flags page (https://rclone.org/flags/) for global op‐
4734 tions not listed here.
4735
4736 SEE ALSO
4737 • rclone (https://rclone.org/commands/rclone/) - Show help for rclone
4738 commands, flags and backends.
4739
4741 Serve a remote over a protocol.
4742
4743 Synopsis
4744 Serve a remote over a given protocol. Requires the use of a subcommand
4745 to specify the protocol, e.g.
4746
4747 rclone serve http remote:
4748
4749 Each subcommand has its own options which you can see in their help.
4750
4751 rclone serve <protocol> [opts] <remote> [flags]
4752
4753 Options
4754 -h, --help help for serve
4755
4756 See the global flags page (https://rclone.org/flags/) for global op‐
4757 tions not listed here.
4758
4759 SEE ALSO
4760 • rclone (https://rclone.org/commands/rclone/) - Show help for rclone
4761 commands, flags and backends.
4762
4763 • rclone serve dlna (https://rclone.org/commands/rclone_serve_dlna/) -
4764 Serve remote:path over DLNA
4765
4766 • rclone serve docker (https://rclone.org/commands/rclone_serve_dock‐
4767 er/) - Serve any remote on docker's volume plugin API.
4768
4769 • rclone serve ftp (https://rclone.org/commands/rclone_serve_ftp/) -
4770 Serve remote:path over FTP.
4771
4772 • rclone serve http (https://rclone.org/commands/rclone_serve_http/) -
4773 Serve the remote over HTTP.
4774
4775 • rclone serve restic (https://rclone.org/com‐
4776 mands/rclone_serve_restic/) - Serve the remote for restic's REST API.
4777
4778 • rclone serve sftp (https://rclone.org/commands/rclone_serve_sftp/) -
4779 Serve the remote over SFTP.
4780
4781 • rclone serve webdav (https://rclone.org/commands/rclone_serve_web‐
4782 dav/) - Serve remote:path over WebDAV.
4783
4785 Serve remote:path over DLNA
4786
4787 Synopsis
4788 Run a DLNA media server for media stored in an rclone remote. Many de‐
4789 vices, such as the Xbox and PlayStation, can automatically discover
4790 this server in the LAN and play audio/video from it. VLC is also sup‐
4791 ported. Service discovery uses UDP multicast packets (SSDP) and will
4792 thus only work on LANs.
4793
4794 Rclone will list all files present in the remote, without filtering
4795 based on media formats or file extensions. Additionally, there is no
4796 media transcoding support. This means that some players might show
4797 files that they are not able to play back correctly.
4798
4799 Server options
4800 Use --addr to specify which IP address and port the server should lis‐
4801 ten on, e.g. --addr 1.2.3.4:8000 or --addr :8080 to listen to all IPs.
4802
4803 Use --name to choose the friendly server name, which is by default
4804 "rclone (hostname)".
4805
4806 Use --log-trace in conjunction with -vv to enable additional debug log‐
4807 ging of all UPNP traffic.
4808
4809 VFS - Virtual File System
4810 This command uses the VFS layer. This adapts the cloud storage objects
4811 that rclone uses into something which looks much more like a disk fil‐
4812 ing system.
4813
4814 Cloud storage objects have lots of properties which aren't like disk
4815 files - you can't extend them or write to the middle of them, so the
4816 VFS layer has to deal with that. Because there is no one right way of
4817 doing this there are various options explained below.
4818
4819 The VFS layer also implements a directory cache - this caches info
4820 about files and directories (but not the data) in memory.
4821
4822 VFS Directory Cache
4823 Using the --dir-cache-time flag, you can control how long a directory
4824 should be considered up to date and not refreshed from the backend.
4825 Changes made through the VFS will appear immediately or invalidate the
4826 cache.
4827
4828 --dir-cache-time duration Time to cache directory entries for (default 5m0s)
4829 --poll-interval duration Time to wait between polling for changes. Must be smaller than dir-cache-time. Only on supported remotes. Set to 0 to disable (default 1m0s)
4830
4831 However, changes made directly on the cloud storage by the web inter‐
4832 face or a different copy of rclone will only be picked up once the di‐
4833 rectory cache expires if the backend configured does not support
4834 polling for changes. If the backend supports polling, changes will be
4835 picked up within the polling interval.
4836
4837 You can send a SIGHUP signal to rclone for it to flush all directory
4838 caches, regardless of how old they are. Assuming only one rclone in‐
4839 stance is running, you can reset the cache like this:
4840
4841 kill -SIGHUP $(pidof rclone)
4842
4843 If you configure rclone with a remote control then you can use rclone
4844 rc to flush the whole directory cache:
4845
4846 rclone rc vfs/forget
4847
4848 Or individual files or directories:
4849
4850 rclone rc vfs/forget file=path/to/file dir=path/to/dir
4851
4852 VFS File Buffering
4853 The --buffer-size flag determines the amount of memory, that will be
4854 used to buffer data in advance.
4855
4856 Each open file will try to keep the specified amount of data in memory
4857 at all times. The buffered data is bound to one open file and won't be
4858 shared.
4859
4860 This flag is a upper limit for the used memory per open file. The buf‐
4861 fer will only use memory for data that is downloaded but not not yet
4862 read. If the buffer is empty, only a small amount of memory will be
4863 used.
4864
4865 The maximum memory used by rclone for buffering can be up to --buffer-
4866 size * open files.
4867
4868 VFS File Caching
4869 These flags control the VFS file caching options. File caching is nec‐
4870 essary to make the VFS layer appear compatible with a normal file sys‐
4871 tem. It can be disabled at the cost of some compatibility.
4872
4873 For example you'll need to enable VFS caching if you want to read and
4874 write simultaneously to a file. See below for more details.
4875
4876 Note that the VFS cache is separate from the cache backend and you may
4877 find that you need one or the other or both.
4878
4879 --cache-dir string Directory rclone will use for caching.
4880 --vfs-cache-mode CacheMode Cache mode off|minimal|writes|full (default off)
4881 --vfs-cache-max-age duration Max age of objects in the cache (default 1h0m0s)
4882 --vfs-cache-max-size SizeSuffix Max total size of objects in the cache (default off)
4883 --vfs-cache-poll-interval duration Interval to poll the cache for stale objects (default 1m0s)
4884 --vfs-write-back duration Time to writeback files after last use when using cache (default 5s)
4885
4886 If run with -vv rclone will print the location of the file cache. The
4887 files are stored in the user cache file area which is OS dependent but
4888 can be controlled with --cache-dir or setting the appropriate environ‐
4889 ment variable.
4890
4891 The cache has 4 different modes selected by --vfs-cache-mode. The
4892 higher the cache mode the more compatible rclone becomes at the cost of
4893 using disk space.
4894
4895 Note that files are written back to the remote only when they are
4896 closed and if they haven't been accessed for --vfs-write-back seconds.
4897 If rclone is quit or dies with files that haven't been uploaded, these
4898 will be uploaded next time rclone is run with the same flags.
4899
4900 If using --vfs-cache-max-size note that the cache may exceed this size
4901 for two reasons. Firstly because it is only checked every --vfs-cache-
4902 poll-interval. Secondly because open files cannot be evicted from the
4903 cache.
4904
4905 You should not run two copies of rclone using the same VFS cache with
4906 the same or overlapping remotes if using --vfs-cache-mode > off. This
4907 can potentially cause data corruption if you do. You can work around
4908 this by giving each rclone its own cache hierarchy with --cache-dir.
4909 You don't need to worry about this if the remotes in use don't overlap.
4910
4911 --vfs-cache-mode off
4912 In this mode (the default) the cache will read directly from the remote
4913 and write directly to the remote without caching anything on disk.
4914
4915 This will mean some operations are not possible
4916
4917 • Files can't be opened for both read AND write
4918
4919 • Files opened for write can't be seeked
4920
4921 • Existing files opened for write must have O_TRUNC set
4922
4923 • Files open for read with O_TRUNC will be opened write only
4924
4925 • Files open for write only will behave as if O_TRUNC was supplied
4926
4927 • Open modes O_APPEND, O_TRUNC are ignored
4928
4929 • If an upload fails it can't be retried
4930
4931 --vfs-cache-mode minimal
4932 This is very similar to "off" except that files opened for read AND
4933 write will be buffered to disk. This means that files opened for write
4934 will be a lot more compatible, but uses the minimal disk space.
4935
4936 These operations are not possible
4937
4938 • Files opened for write only can't be seeked
4939
4940 • Existing files opened for write must have O_TRUNC set
4941
4942 • Files opened for write only will ignore O_APPEND, O_TRUNC
4943
4944 • If an upload fails it can't be retried
4945
4946 --vfs-cache-mode writes
4947 In this mode files opened for read only are still read directly from
4948 the remote, write only and read/write files are buffered to disk first.
4949
4950 This mode should support all normal file system operations.
4951
4952 If an upload fails it will be retried at exponentially increasing in‐
4953 tervals up to 1 minute.
4954
4955 --vfs-cache-mode full
4956 In this mode all reads and writes are buffered to and from disk. When
4957 data is read from the remote this is buffered to disk as well.
4958
4959 In this mode the files in the cache will be sparse files and rclone
4960 will keep track of which bits of the files it has downloaded.
4961
4962 So if an application only reads the starts of each file, then rclone
4963 will only buffer the start of the file. These files will appear to be
4964 their full size in the cache, but they will be sparse files with only
4965 the data that has been downloaded present in them.
4966
4967 This mode should support all normal file system operations and is oth‐
4968 erwise identical to --vfs-cache-mode writes.
4969
4970 When reading a file rclone will read --buffer-size plus --vfs-read-
4971 ahead bytes ahead. The --buffer-size is buffered in memory whereas the
4972 --vfs-read-ahead is buffered on disk.
4973
4974 When using this mode it is recommended that --buffer-size is not set
4975 too large and --vfs-read-ahead is set large if required.
4976
4977 IMPORTANT not all file systems support sparse files. In particular
4978 FAT/exFAT do not. Rclone will perform very badly if the cache directo‐
4979 ry is on a filesystem which doesn't support sparse files and it will
4980 log an ERROR message if one is detected.
4981
4982 Fingerprinting
4983 Various parts of the VFS use fingerprinting to see if a local file copy
4984 has changed relative to a remote file. Fingerprints are made from:
4985
4986 • size
4987
4988 • modification time
4989
4990 • hash
4991
4992 where available on an object.
4993
4994 On some backends some of these attributes are slow to read (they take
4995 an extra API call per object, or extra work per object).
4996
4997 For example hash is slow with the local and sftp backends as they have
4998 to read the entire file and hash it, and modtime is slow with the s3,
4999 swift, ftp and qinqstor backends because they need to do an extra API
5000 call to fetch it.
5001
5002 If you use the --vfs-fast-fingerprint flag then rclone will not include
5003 the slow operations in the fingerprint. This makes the fingerprinting
5004 less accurate but much faster and will improve the opening time of
5005 cached files.
5006
5007 If you are running a vfs cache over local, s3 or swift backends then
5008 using this flag is recommended.
5009
5010 Note that if you change the value of this flag, the fingerprints of the
5011 files in the cache may be invalidated and the files will need to be
5012 downloaded again.
5013
5014 VFS Chunked Reading
5015 When rclone reads files from a remote it reads them in chunks. This
5016 means that rather than requesting the whole file rclone reads the chunk
5017 specified. This can reduce the used download quota for some remotes by
5018 requesting only chunks from the remote that are actually read, at the
5019 cost of an increased number of requests.
5020
5021 These flags control the chunking:
5022
5023 --vfs-read-chunk-size SizeSuffix Read the source objects in chunks (default 128M)
5024 --vfs-read-chunk-size-limit SizeSuffix Max chunk doubling size (default off)
5025
5026 Rclone will start reading a chunk of size --vfs-read-chunk-size, and
5027 then double the size for each read. When --vfs-read-chunk-size-limit
5028 is specified, and greater than --vfs-read-chunk-size, the chunk size
5029 for each open file will get doubled only until the specified value is
5030 reached. If the value is "off", which is the default, the limit is
5031 disabled and the chunk size will grow indefinitely.
5032
5033 With --vfs-read-chunk-size 100M and --vfs-read-chunk-size-limit 0 the
5034 following parts will be downloaded: 0-100M, 100M-200M, 200M-300M,
5035 300M-400M and so on. When --vfs-read-chunk-size-limit 500M is speci‐
5036 fied, the result would be 0-100M, 100M-300M, 300M-700M, 700M-1200M,
5037 1200M-1700M and so on.
5038
5039 Setting --vfs-read-chunk-size to 0 or "off" disables chunked reading.
5040
5041 VFS Performance
5042 These flags may be used to enable/disable features of the VFS for per‐
5043 formance or other reasons. See also the chunked reading feature.
5044
5045 In particular S3 and Swift benefit hugely from the --no-modtime flag
5046 (or use --use-server-modtime for a slightly different effect) as each
5047 read of the modification time takes a transaction.
5048
5049 --no-checksum Don't compare checksums on up/download.
5050 --no-modtime Don't read/write the modification time (can speed things up).
5051 --no-seek Don't allow seeking in files.
5052 --read-only Only allow read-only access.
5053
5054 Sometimes rclone is delivered reads or writes out of order. Rather
5055 than seeking rclone will wait a short time for the in sequence read or
5056 write to come in. These flags only come into effect when not using an
5057 on disk cache file.
5058
5059 --vfs-read-wait duration Time to wait for in-sequence read before seeking (default 20ms)
5060 --vfs-write-wait duration Time to wait for in-sequence write before giving error (default 1s)
5061
5062 When using VFS write caching (--vfs-cache-mode with value writes or
5063 full), the global flag --transfers can be set to adjust the number of
5064 parallel uploads of modified files from the cache (the related global
5065 flag --checkers has no effect on the VFS).
5066
5067 --transfers int Number of file transfers to run in parallel (default 4)
5068
5069 VFS Case Sensitivity
5070 Linux file systems are case-sensitive: two files can differ only by
5071 case, and the exact case must be used when opening a file.
5072
5073 File systems in modern Windows are case-insensitive but case-preserv‐
5074 ing: although existing files can be opened using any case, the exact
5075 case used to create the file is preserved and available for programs to
5076 query. It is not allowed for two files in the same directory to differ
5077 only by case.
5078
5079 Usually file systems on macOS are case-insensitive. It is possible to
5080 make macOS file systems case-sensitive but that is not the default.
5081
5082 The --vfs-case-insensitive VFS flag controls how rclone handles these
5083 two cases. If its value is "false", rclone passes file names to the
5084 remote as-is. If the flag is "true" (or appears without a value on the
5085 command line), rclone may perform a "fixup" as explained below.
5086
5087 The user may specify a file name to open/delete/rename/etc with a case
5088 different than what is stored on the remote. If an argument refers to
5089 an existing file with exactly the same name, then the case of the ex‐
5090 isting file on the disk will be used. However, if a file name with ex‐
5091 actly the same name is not found but a name differing only by case ex‐
5092 ists, rclone will transparently fixup the name. This fixup happens on‐
5093 ly when an existing file is requested. Case sensitivity of file names
5094 created anew by rclone is controlled by the underlying remote.
5095
5096 Note that case sensitivity of the operating system running rclone (the
5097 target) may differ from case sensitivity of a file system presented by
5098 rclone (the source). The flag controls whether "fixup" is performed to
5099 satisfy the target.
5100
5101 If the flag is not provided on the command line, then its default value
5102 depends on the operating system where rclone runs: "true" on Windows
5103 and macOS, "false" otherwise. If the flag is provided without a value,
5104 then it is "true".
5105
5106 VFS Disk Options
5107 This flag allows you to manually set the statistics about the filing
5108 system. It can be useful when those statistics cannot be read correct‐
5109 ly automatically.
5110
5111 --vfs-disk-space-total-size Manually set the total disk space size (example: 256G, default: -1)
5112
5113 Alternate report of used bytes
5114 Some backends, most notably S3, do not report the amount of bytes used.
5115 If you need this information to be available when running df on the
5116 filesystem, then pass the flag --vfs-used-is-size to rclone. With this
5117 flag set, instead of relying on the backend to report this information,
5118 rclone will scan the whole remote similar to rclone size and compute
5119 the total used space itself.
5120
5121 WARNING. Contrary to rclone size, this flag ignores filters so that the
5122 result is accurate. However, this is very inefficient and may cost
5123 lots of API calls resulting in extra charges. Use it as a last resort
5124 and only with caching.
5125
5126 rclone serve dlna remote:path [flags]
5127
5128 Options
5129 --addr string The ip:port or :port to bind the DLNA http server to (default ":7879")
5130 --announce-interval duration The interval between SSDP announcements (default 12m0s)
5131 --dir-cache-time duration Time to cache directory entries for (default 5m0s)
5132 --dir-perms FileMode Directory permissions (default 0777)
5133 --file-perms FileMode File permissions (default 0666)
5134 --gid uint32 Override the gid field set by the filesystem (not supported on Windows) (default 1000)
5135 -h, --help help for dlna
5136 --interface stringArray The interface to use for SSDP (repeat as necessary)
5137 --log-trace Enable trace logging of SOAP traffic
5138 --name string Name of DLNA server
5139 --no-checksum Don't compare checksums on up/download
5140 --no-modtime Don't read/write the modification time (can speed things up)
5141 --no-seek Don't allow seeking in files
5142 --poll-interval duration Time to wait between polling for changes, must be smaller than dir-cache-time and only on supported remotes (set 0 to disable) (default 1m0s)
5143 --read-only Only allow read-only access
5144 --uid uint32 Override the uid field set by the filesystem (not supported on Windows) (default 1000)
5145 --umask int Override the permission bits set by the filesystem (not supported on Windows) (default 2)
5146 --vfs-cache-max-age duration Max age of objects in the cache (default 1h0m0s)
5147 --vfs-cache-max-size SizeSuffix Max total size of objects in the cache (default off)
5148 --vfs-cache-mode CacheMode Cache mode off|minimal|writes|full (default off)
5149 --vfs-cache-poll-interval duration Interval to poll the cache for stale objects (default 1m0s)
5150 --vfs-case-insensitive If a file name not found, find a case insensitive match
5151 --vfs-disk-space-total-size SizeSuffix Specify the total space of disk (default off)
5152 --vfs-fast-fingerprint Use fast (less accurate) fingerprints for change detection
5153 --vfs-read-ahead SizeSuffix Extra read ahead over --buffer-size when using cache-mode full
5154 --vfs-read-chunk-size SizeSuffix Read the source objects in chunks (default 128Mi)
5155 --vfs-read-chunk-size-limit SizeSuffix If greater than --vfs-read-chunk-size, double the chunk size after each chunk read, until the limit is reached ('off' is unlimited) (default off)
5156 --vfs-read-wait duration Time to wait for in-sequence read before seeking (default 20ms)
5157 --vfs-used-is-size rclone size Use the rclone size algorithm for Used size
5158 --vfs-write-back duration Time to writeback files after last use when using cache (default 5s)
5159 --vfs-write-wait duration Time to wait for in-sequence write before giving error (default 1s)
5160
5161 See the global flags page (https://rclone.org/flags/) for global op‐
5162 tions not listed here.
5163
5164 SEE ALSO
5165 • rclone serve (https://rclone.org/commands/rclone_serve/) - Serve a
5166 remote over a protocol.
5167
5169 Serve any remote on docker's volume plugin API.
5170
5171 Synopsis
5172 This command implements the Docker volume plugin API allowing docker to
5173 use rclone as a data storage mechanism for various cloud providers.
5174 rclone provides docker volume plugin based on it.
5175
5176 To create a docker plugin, one must create a Unix or TCP socket that
5177 Docker will look for when you use the plugin and then it listens for
5178 commands from docker daemon and runs the corresponding code when neces‐
5179 sary. Docker plugins can run as a managed plugin under control of the
5180 docker daemon or as an independent native service. For testing, you
5181 can just run it directly from the command line, for example:
5182
5183 sudo rclone serve docker --base-dir /tmp/rclone-volumes --socket-addr localhost:8787 -vv
5184
5185 Running rclone serve docker will create the said socket, listening for
5186 commands from Docker to create the necessary Volumes. Normally you
5187 need not give the --socket-addr flag. The API will listen on the unix
5188 domain socket at /run/docker/plugins/rclone.sock. In the example above
5189 rclone will create a TCP socket and a small file /etc/docker/plug‐
5190 ins/rclone.spec containing the socket address. We use sudo because
5191 both paths are writeable only by the root user.
5192
5193 If you later decide to change listening socket, the docker daemon must
5194 be restarted to reconnect to /run/docker/plugins/rclone.sock or parse
5195 new /etc/docker/plugins/rclone.spec. Until you restart, any volume re‐
5196 lated docker commands will timeout trying to access the old socket.
5197 Running directly is supported on Linux only, not on Windows or MacOS.
5198 This is not a problem with managed plugin mode described in details in
5199 the full documentation (https://rclone.org/docker).
5200
5201 The command will create volume mounts under the path given by --base-
5202 dir (by default /var/lib/docker-volumes/rclone available only to root)
5203 and maintain the JSON formatted file docker-plugin.state in the rclone
5204 cache directory with book-keeping records of created and mounted vol‐
5205 umes.
5206
5207 All mount and VFS options are submitted by the docker daemon via API,
5208 but you can also provide defaults on the command line as well as set
5209 path to the config file and cache directory or adjust logging verbosi‐
5210 ty.
5211
5212 VFS - Virtual File System
5213 This command uses the VFS layer. This adapts the cloud storage objects
5214 that rclone uses into something which looks much more like a disk fil‐
5215 ing system.
5216
5217 Cloud storage objects have lots of properties which aren't like disk
5218 files - you can't extend them or write to the middle of them, so the
5219 VFS layer has to deal with that. Because there is no one right way of
5220 doing this there are various options explained below.
5221
5222 The VFS layer also implements a directory cache - this caches info
5223 about files and directories (but not the data) in memory.
5224
5225 VFS Directory Cache
5226 Using the --dir-cache-time flag, you can control how long a directory
5227 should be considered up to date and not refreshed from the backend.
5228 Changes made through the VFS will appear immediately or invalidate the
5229 cache.
5230
5231 --dir-cache-time duration Time to cache directory entries for (default 5m0s)
5232 --poll-interval duration Time to wait between polling for changes. Must be smaller than dir-cache-time. Only on supported remotes. Set to 0 to disable (default 1m0s)
5233
5234 However, changes made directly on the cloud storage by the web inter‐
5235 face or a different copy of rclone will only be picked up once the di‐
5236 rectory cache expires if the backend configured does not support
5237 polling for changes. If the backend supports polling, changes will be
5238 picked up within the polling interval.
5239
5240 You can send a SIGHUP signal to rclone for it to flush all directory
5241 caches, regardless of how old they are. Assuming only one rclone in‐
5242 stance is running, you can reset the cache like this:
5243
5244 kill -SIGHUP $(pidof rclone)
5245
5246 If you configure rclone with a remote control then you can use rclone
5247 rc to flush the whole directory cache:
5248
5249 rclone rc vfs/forget
5250
5251 Or individual files or directories:
5252
5253 rclone rc vfs/forget file=path/to/file dir=path/to/dir
5254
5255 VFS File Buffering
5256 The --buffer-size flag determines the amount of memory, that will be
5257 used to buffer data in advance.
5258
5259 Each open file will try to keep the specified amount of data in memory
5260 at all times. The buffered data is bound to one open file and won't be
5261 shared.
5262
5263 This flag is a upper limit for the used memory per open file. The buf‐
5264 fer will only use memory for data that is downloaded but not not yet
5265 read. If the buffer is empty, only a small amount of memory will be
5266 used.
5267
5268 The maximum memory used by rclone for buffering can be up to --buffer-
5269 size * open files.
5270
5271 VFS File Caching
5272 These flags control the VFS file caching options. File caching is nec‐
5273 essary to make the VFS layer appear compatible with a normal file sys‐
5274 tem. It can be disabled at the cost of some compatibility.
5275
5276 For example you'll need to enable VFS caching if you want to read and
5277 write simultaneously to a file. See below for more details.
5278
5279 Note that the VFS cache is separate from the cache backend and you may
5280 find that you need one or the other or both.
5281
5282 --cache-dir string Directory rclone will use for caching.
5283 --vfs-cache-mode CacheMode Cache mode off|minimal|writes|full (default off)
5284 --vfs-cache-max-age duration Max age of objects in the cache (default 1h0m0s)
5285 --vfs-cache-max-size SizeSuffix Max total size of objects in the cache (default off)
5286 --vfs-cache-poll-interval duration Interval to poll the cache for stale objects (default 1m0s)
5287 --vfs-write-back duration Time to writeback files after last use when using cache (default 5s)
5288
5289 If run with -vv rclone will print the location of the file cache. The
5290 files are stored in the user cache file area which is OS dependent but
5291 can be controlled with --cache-dir or setting the appropriate environ‐
5292 ment variable.
5293
5294 The cache has 4 different modes selected by --vfs-cache-mode. The
5295 higher the cache mode the more compatible rclone becomes at the cost of
5296 using disk space.
5297
5298 Note that files are written back to the remote only when they are
5299 closed and if they haven't been accessed for --vfs-write-back seconds.
5300 If rclone is quit or dies with files that haven't been uploaded, these
5301 will be uploaded next time rclone is run with the same flags.
5302
5303 If using --vfs-cache-max-size note that the cache may exceed this size
5304 for two reasons. Firstly because it is only checked every --vfs-cache-
5305 poll-interval. Secondly because open files cannot be evicted from the
5306 cache.
5307
5308 You should not run two copies of rclone using the same VFS cache with
5309 the same or overlapping remotes if using --vfs-cache-mode > off. This
5310 can potentially cause data corruption if you do. You can work around
5311 this by giving each rclone its own cache hierarchy with --cache-dir.
5312 You don't need to worry about this if the remotes in use don't overlap.
5313
5314 --vfs-cache-mode off
5315 In this mode (the default) the cache will read directly from the remote
5316 and write directly to the remote without caching anything on disk.
5317
5318 This will mean some operations are not possible
5319
5320 • Files can't be opened for both read AND write
5321
5322 • Files opened for write can't be seeked
5323
5324 • Existing files opened for write must have O_TRUNC set
5325
5326 • Files open for read with O_TRUNC will be opened write only
5327
5328 • Files open for write only will behave as if O_TRUNC was supplied
5329
5330 • Open modes O_APPEND, O_TRUNC are ignored
5331
5332 • If an upload fails it can't be retried
5333
5334 --vfs-cache-mode minimal
5335 This is very similar to "off" except that files opened for read AND
5336 write will be buffered to disk. This means that files opened for write
5337 will be a lot more compatible, but uses the minimal disk space.
5338
5339 These operations are not possible
5340
5341 • Files opened for write only can't be seeked
5342
5343 • Existing files opened for write must have O_TRUNC set
5344
5345 • Files opened for write only will ignore O_APPEND, O_TRUNC
5346
5347 • If an upload fails it can't be retried
5348
5349 --vfs-cache-mode writes
5350 In this mode files opened for read only are still read directly from
5351 the remote, write only and read/write files are buffered to disk first.
5352
5353 This mode should support all normal file system operations.
5354
5355 If an upload fails it will be retried at exponentially increasing in‐
5356 tervals up to 1 minute.
5357
5358 --vfs-cache-mode full
5359 In this mode all reads and writes are buffered to and from disk. When
5360 data is read from the remote this is buffered to disk as well.
5361
5362 In this mode the files in the cache will be sparse files and rclone
5363 will keep track of which bits of the files it has downloaded.
5364
5365 So if an application only reads the starts of each file, then rclone
5366 will only buffer the start of the file. These files will appear to be
5367 their full size in the cache, but they will be sparse files with only
5368 the data that has been downloaded present in them.
5369
5370 This mode should support all normal file system operations and is oth‐
5371 erwise identical to --vfs-cache-mode writes.
5372
5373 When reading a file rclone will read --buffer-size plus --vfs-read-
5374 ahead bytes ahead. The --buffer-size is buffered in memory whereas the
5375 --vfs-read-ahead is buffered on disk.
5376
5377 When using this mode it is recommended that --buffer-size is not set
5378 too large and --vfs-read-ahead is set large if required.
5379
5380 IMPORTANT not all file systems support sparse files. In particular
5381 FAT/exFAT do not. Rclone will perform very badly if the cache directo‐
5382 ry is on a filesystem which doesn't support sparse files and it will
5383 log an ERROR message if one is detected.
5384
5385 Fingerprinting
5386 Various parts of the VFS use fingerprinting to see if a local file copy
5387 has changed relative to a remote file. Fingerprints are made from:
5388
5389 • size
5390
5391 • modification time
5392
5393 • hash
5394
5395 where available on an object.
5396
5397 On some backends some of these attributes are slow to read (they take
5398 an extra API call per object, or extra work per object).
5399
5400 For example hash is slow with the local and sftp backends as they have
5401 to read the entire file and hash it, and modtime is slow with the s3,
5402 swift, ftp and qinqstor backends because they need to do an extra API
5403 call to fetch it.
5404
5405 If you use the --vfs-fast-fingerprint flag then rclone will not include
5406 the slow operations in the fingerprint. This makes the fingerprinting
5407 less accurate but much faster and will improve the opening time of
5408 cached files.
5409
5410 If you are running a vfs cache over local, s3 or swift backends then
5411 using this flag is recommended.
5412
5413 Note that if you change the value of this flag, the fingerprints of the
5414 files in the cache may be invalidated and the files will need to be
5415 downloaded again.
5416
5417 VFS Chunked Reading
5418 When rclone reads files from a remote it reads them in chunks. This
5419 means that rather than requesting the whole file rclone reads the chunk
5420 specified. This can reduce the used download quota for some remotes by
5421 requesting only chunks from the remote that are actually read, at the
5422 cost of an increased number of requests.
5423
5424 These flags control the chunking:
5425
5426 --vfs-read-chunk-size SizeSuffix Read the source objects in chunks (default 128M)
5427 --vfs-read-chunk-size-limit SizeSuffix Max chunk doubling size (default off)
5428
5429 Rclone will start reading a chunk of size --vfs-read-chunk-size, and
5430 then double the size for each read. When --vfs-read-chunk-size-limit
5431 is specified, and greater than --vfs-read-chunk-size, the chunk size
5432 for each open file will get doubled only until the specified value is
5433 reached. If the value is "off", which is the default, the limit is
5434 disabled and the chunk size will grow indefinitely.
5435
5436 With --vfs-read-chunk-size 100M and --vfs-read-chunk-size-limit 0 the
5437 following parts will be downloaded: 0-100M, 100M-200M, 200M-300M,
5438 300M-400M and so on. When --vfs-read-chunk-size-limit 500M is speci‐
5439 fied, the result would be 0-100M, 100M-300M, 300M-700M, 700M-1200M,
5440 1200M-1700M and so on.
5441
5442 Setting --vfs-read-chunk-size to 0 or "off" disables chunked reading.
5443
5444 VFS Performance
5445 These flags may be used to enable/disable features of the VFS for per‐
5446 formance or other reasons. See also the chunked reading feature.
5447
5448 In particular S3 and Swift benefit hugely from the --no-modtime flag
5449 (or use --use-server-modtime for a slightly different effect) as each
5450 read of the modification time takes a transaction.
5451
5452 --no-checksum Don't compare checksums on up/download.
5453 --no-modtime Don't read/write the modification time (can speed things up).
5454 --no-seek Don't allow seeking in files.
5455 --read-only Only allow read-only access.
5456
5457 Sometimes rclone is delivered reads or writes out of order. Rather
5458 than seeking rclone will wait a short time for the in sequence read or
5459 write to come in. These flags only come into effect when not using an
5460 on disk cache file.
5461
5462 --vfs-read-wait duration Time to wait for in-sequence read before seeking (default 20ms)
5463 --vfs-write-wait duration Time to wait for in-sequence write before giving error (default 1s)
5464
5465 When using VFS write caching (--vfs-cache-mode with value writes or
5466 full), the global flag --transfers can be set to adjust the number of
5467 parallel uploads of modified files from the cache (the related global
5468 flag --checkers has no effect on the VFS).
5469
5470 --transfers int Number of file transfers to run in parallel (default 4)
5471
5472 VFS Case Sensitivity
5473 Linux file systems are case-sensitive: two files can differ only by
5474 case, and the exact case must be used when opening a file.
5475
5476 File systems in modern Windows are case-insensitive but case-preserv‐
5477 ing: although existing files can be opened using any case, the exact
5478 case used to create the file is preserved and available for programs to
5479 query. It is not allowed for two files in the same directory to differ
5480 only by case.
5481
5482 Usually file systems on macOS are case-insensitive. It is possible to
5483 make macOS file systems case-sensitive but that is not the default.
5484
5485 The --vfs-case-insensitive VFS flag controls how rclone handles these
5486 two cases. If its value is "false", rclone passes file names to the
5487 remote as-is. If the flag is "true" (or appears without a value on the
5488 command line), rclone may perform a "fixup" as explained below.
5489
5490 The user may specify a file name to open/delete/rename/etc with a case
5491 different than what is stored on the remote. If an argument refers to
5492 an existing file with exactly the same name, then the case of the ex‐
5493 isting file on the disk will be used. However, if a file name with ex‐
5494 actly the same name is not found but a name differing only by case ex‐
5495 ists, rclone will transparently fixup the name. This fixup happens on‐
5496 ly when an existing file is requested. Case sensitivity of file names
5497 created anew by rclone is controlled by the underlying remote.
5498
5499 Note that case sensitivity of the operating system running rclone (the
5500 target) may differ from case sensitivity of a file system presented by
5501 rclone (the source). The flag controls whether "fixup" is performed to
5502 satisfy the target.
5503
5504 If the flag is not provided on the command line, then its default value
5505 depends on the operating system where rclone runs: "true" on Windows
5506 and macOS, "false" otherwise. If the flag is provided without a value,
5507 then it is "true".
5508
5509 VFS Disk Options
5510 This flag allows you to manually set the statistics about the filing
5511 system. It can be useful when those statistics cannot be read correct‐
5512 ly automatically.
5513
5514 --vfs-disk-space-total-size Manually set the total disk space size (example: 256G, default: -1)
5515
5516 Alternate report of used bytes
5517 Some backends, most notably S3, do not report the amount of bytes used.
5518 If you need this information to be available when running df on the
5519 filesystem, then pass the flag --vfs-used-is-size to rclone. With this
5520 flag set, instead of relying on the backend to report this information,
5521 rclone will scan the whole remote similar to rclone size and compute
5522 the total used space itself.
5523
5524 WARNING. Contrary to rclone size, this flag ignores filters so that the
5525 result is accurate. However, this is very inefficient and may cost
5526 lots of API calls resulting in extra charges. Use it as a last resort
5527 and only with caching.
5528
5529 rclone serve docker [flags]
5530
5531 Options
5532 --allow-non-empty Allow mounting over a non-empty directory (not supported on Windows)
5533 --allow-other Allow access to other users (not supported on Windows)
5534 --allow-root Allow access to root user (not supported on Windows)
5535 --async-read Use asynchronous reads (not supported on Windows) (default true)
5536 --attr-timeout duration Time for which file/directory attributes are cached (default 1s)
5537 --base-dir string Base directory for volumes (default "/var/lib/docker-volumes/rclone")
5538 --daemon Run mount in background and exit parent process (as background output is suppressed, use --log-file with --log-format=pid,... to monitor) (not supported on Windows)
5539 --daemon-timeout duration Time limit for rclone to respond to kernel (not supported on Windows)
5540 --daemon-wait duration Time to wait for ready mount from daemon (maximum time on Linux, constant sleep time on OSX/BSD) (not supported on Windows) (default 1m0s)
5541 --debug-fuse Debug the FUSE internals - needs -v
5542 --default-permissions Makes kernel enforce access control based on the file mode (not supported on Windows)
5543 --devname string Set the device name - default is remote:path
5544 --dir-cache-time duration Time to cache directory entries for (default 5m0s)
5545 --dir-perms FileMode Directory permissions (default 0777)
5546 --file-perms FileMode File permissions (default 0666)
5547 --forget-state Skip restoring previous state
5548 --fuse-flag stringArray Flags or arguments to be passed direct to libfuse/WinFsp (repeat if required)
5549 --gid uint32 Override the gid field set by the filesystem (not supported on Windows) (default 1000)
5550 -h, --help help for docker
5551 --max-read-ahead SizeSuffix The number of bytes that can be prefetched for sequential reads (not supported on Windows) (default 128Ki)
5552 --network-mode Mount as remote network drive, instead of fixed disk drive (supported on Windows only)
5553 --no-checksum Don't compare checksums on up/download
5554 --no-modtime Don't read/write the modification time (can speed things up)
5555 --no-seek Don't allow seeking in files
5556 --no-spec Do not write spec file
5557 --noappledouble Ignore Apple Double (._) and .DS_Store files (supported on OSX only) (default true)
5558 --noapplexattr Ignore all "com.apple.*" extended attributes (supported on OSX only)
5559 -o, --option stringArray Option for libfuse/WinFsp (repeat if required)
5560 --poll-interval duration Time to wait between polling for changes, must be smaller than dir-cache-time and only on supported remotes (set 0 to disable) (default 1m0s)
5561 --read-only Only allow read-only access
5562 --socket-addr string Address <host:port> or absolute path (default: /run/docker/plugins/rclone.sock)
5563 --socket-gid int GID for unix socket (default: current process GID) (default 1000)
5564 --uid uint32 Override the uid field set by the filesystem (not supported on Windows) (default 1000)
5565 --umask int Override the permission bits set by the filesystem (not supported on Windows) (default 2)
5566 --vfs-cache-max-age duration Max age of objects in the cache (default 1h0m0s)
5567 --vfs-cache-max-size SizeSuffix Max total size of objects in the cache (default off)
5568 --vfs-cache-mode CacheMode Cache mode off|minimal|writes|full (default off)
5569 --vfs-cache-poll-interval duration Interval to poll the cache for stale objects (default 1m0s)
5570 --vfs-case-insensitive If a file name not found, find a case insensitive match
5571 --vfs-disk-space-total-size SizeSuffix Specify the total space of disk (default off)
5572 --vfs-fast-fingerprint Use fast (less accurate) fingerprints for change detection
5573 --vfs-read-ahead SizeSuffix Extra read ahead over --buffer-size when using cache-mode full
5574 --vfs-read-chunk-size SizeSuffix Read the source objects in chunks (default 128Mi)
5575 --vfs-read-chunk-size-limit SizeSuffix If greater than --vfs-read-chunk-size, double the chunk size after each chunk read, until the limit is reached ('off' is unlimited) (default off)
5576 --vfs-read-wait duration Time to wait for in-sequence read before seeking (default 20ms)
5577 --vfs-used-is-size rclone size Use the rclone size algorithm for Used size
5578 --vfs-write-back duration Time to writeback files after last use when using cache (default 5s)
5579 --vfs-write-wait duration Time to wait for in-sequence write before giving error (default 1s)
5580 --volname string Set the volume name (supported on Windows and OSX only)
5581 --write-back-cache Makes kernel buffer writes before sending them to rclone (without this, writethrough caching is used) (not supported on Windows)
5582
5583 See the global flags page (https://rclone.org/flags/) for global op‐
5584 tions not listed here.
5585
5586 SEE ALSO
5587 • rclone serve (https://rclone.org/commands/rclone_serve/) - Serve a
5588 remote over a protocol.
5589
5591 Serve remote:path over FTP.
5592
5593 Synopsis
5594 Run a basic FTP server to serve a remote over FTP protocol. This can
5595 be viewed with a FTP client or you can make a remote of type FTP to
5596 read and write it.
5597
5598 Server options
5599 Use --addr to specify which IP address and port the server should lis‐
5600 ten on, e.g. --addr 1.2.3.4:8000 or --addr :8080 to listen to all IPs.
5601 By default it only listens on localhost. You can use port :0 to let
5602 the OS choose an available port.
5603
5604 If you set --addr to listen on a public or LAN accessible IP address
5605 then using Authentication is advised - see the next section for info.
5606
5607 Authentication
5608 By default this will serve files without needing a login.
5609
5610 You can set a single username and password with the --user and --pass
5611 flags.
5612
5613 VFS - Virtual File System
5614 This command uses the VFS layer. This adapts the cloud storage objects
5615 that rclone uses into something which looks much more like a disk fil‐
5616 ing system.
5617
5618 Cloud storage objects have lots of properties which aren't like disk
5619 files - you can't extend them or write to the middle of them, so the
5620 VFS layer has to deal with that. Because there is no one right way of
5621 doing this there are various options explained below.
5622
5623 The VFS layer also implements a directory cache - this caches info
5624 about files and directories (but not the data) in memory.
5625
5626 VFS Directory Cache
5627 Using the --dir-cache-time flag, you can control how long a directory
5628 should be considered up to date and not refreshed from the backend.
5629 Changes made through the VFS will appear immediately or invalidate the
5630 cache.
5631
5632 --dir-cache-time duration Time to cache directory entries for (default 5m0s)
5633 --poll-interval duration Time to wait between polling for changes. Must be smaller than dir-cache-time. Only on supported remotes. Set to 0 to disable (default 1m0s)
5634
5635 However, changes made directly on the cloud storage by the web inter‐
5636 face or a different copy of rclone will only be picked up once the di‐
5637 rectory cache expires if the backend configured does not support
5638 polling for changes. If the backend supports polling, changes will be
5639 picked up within the polling interval.
5640
5641 You can send a SIGHUP signal to rclone for it to flush all directory
5642 caches, regardless of how old they are. Assuming only one rclone in‐
5643 stance is running, you can reset the cache like this:
5644
5645 kill -SIGHUP $(pidof rclone)
5646
5647 If you configure rclone with a remote control then you can use rclone
5648 rc to flush the whole directory cache:
5649
5650 rclone rc vfs/forget
5651
5652 Or individual files or directories:
5653
5654 rclone rc vfs/forget file=path/to/file dir=path/to/dir
5655
5656 VFS File Buffering
5657 The --buffer-size flag determines the amount of memory, that will be
5658 used to buffer data in advance.
5659
5660 Each open file will try to keep the specified amount of data in memory
5661 at all times. The buffered data is bound to one open file and won't be
5662 shared.
5663
5664 This flag is a upper limit for the used memory per open file. The buf‐
5665 fer will only use memory for data that is downloaded but not not yet
5666 read. If the buffer is empty, only a small amount of memory will be
5667 used.
5668
5669 The maximum memory used by rclone for buffering can be up to --buffer-
5670 size * open files.
5671
5672 VFS File Caching
5673 These flags control the VFS file caching options. File caching is nec‐
5674 essary to make the VFS layer appear compatible with a normal file sys‐
5675 tem. It can be disabled at the cost of some compatibility.
5676
5677 For example you'll need to enable VFS caching if you want to read and
5678 write simultaneously to a file. See below for more details.
5679
5680 Note that the VFS cache is separate from the cache backend and you may
5681 find that you need one or the other or both.
5682
5683 --cache-dir string Directory rclone will use for caching.
5684 --vfs-cache-mode CacheMode Cache mode off|minimal|writes|full (default off)
5685 --vfs-cache-max-age duration Max age of objects in the cache (default 1h0m0s)
5686 --vfs-cache-max-size SizeSuffix Max total size of objects in the cache (default off)
5687 --vfs-cache-poll-interval duration Interval to poll the cache for stale objects (default 1m0s)
5688 --vfs-write-back duration Time to writeback files after last use when using cache (default 5s)
5689
5690 If run with -vv rclone will print the location of the file cache. The
5691 files are stored in the user cache file area which is OS dependent but
5692 can be controlled with --cache-dir or setting the appropriate environ‐
5693 ment variable.
5694
5695 The cache has 4 different modes selected by --vfs-cache-mode. The
5696 higher the cache mode the more compatible rclone becomes at the cost of
5697 using disk space.
5698
5699 Note that files are written back to the remote only when they are
5700 closed and if they haven't been accessed for --vfs-write-back seconds.
5701 If rclone is quit or dies with files that haven't been uploaded, these
5702 will be uploaded next time rclone is run with the same flags.
5703
5704 If using --vfs-cache-max-size note that the cache may exceed this size
5705 for two reasons. Firstly because it is only checked every --vfs-cache-
5706 poll-interval. Secondly because open files cannot be evicted from the
5707 cache.
5708
5709 You should not run two copies of rclone using the same VFS cache with
5710 the same or overlapping remotes if using --vfs-cache-mode > off. This
5711 can potentially cause data corruption if you do. You can work around
5712 this by giving each rclone its own cache hierarchy with --cache-dir.
5713 You don't need to worry about this if the remotes in use don't overlap.
5714
5715 --vfs-cache-mode off
5716 In this mode (the default) the cache will read directly from the remote
5717 and write directly to the remote without caching anything on disk.
5718
5719 This will mean some operations are not possible
5720
5721 • Files can't be opened for both read AND write
5722
5723 • Files opened for write can't be seeked
5724
5725 • Existing files opened for write must have O_TRUNC set
5726
5727 • Files open for read with O_TRUNC will be opened write only
5728
5729 • Files open for write only will behave as if O_TRUNC was supplied
5730
5731 • Open modes O_APPEND, O_TRUNC are ignored
5732
5733 • If an upload fails it can't be retried
5734
5735 --vfs-cache-mode minimal
5736 This is very similar to "off" except that files opened for read AND
5737 write will be buffered to disk. This means that files opened for write
5738 will be a lot more compatible, but uses the minimal disk space.
5739
5740 These operations are not possible
5741
5742 • Files opened for write only can't be seeked
5743
5744 • Existing files opened for write must have O_TRUNC set
5745
5746 • Files opened for write only will ignore O_APPEND, O_TRUNC
5747
5748 • If an upload fails it can't be retried
5749
5750 --vfs-cache-mode writes
5751 In this mode files opened for read only are still read directly from
5752 the remote, write only and read/write files are buffered to disk first.
5753
5754 This mode should support all normal file system operations.
5755
5756 If an upload fails it will be retried at exponentially increasing in‐
5757 tervals up to 1 minute.
5758
5759 --vfs-cache-mode full
5760 In this mode all reads and writes are buffered to and from disk. When
5761 data is read from the remote this is buffered to disk as well.
5762
5763 In this mode the files in the cache will be sparse files and rclone
5764 will keep track of which bits of the files it has downloaded.
5765
5766 So if an application only reads the starts of each file, then rclone
5767 will only buffer the start of the file. These files will appear to be
5768 their full size in the cache, but they will be sparse files with only
5769 the data that has been downloaded present in them.
5770
5771 This mode should support all normal file system operations and is oth‐
5772 erwise identical to --vfs-cache-mode writes.
5773
5774 When reading a file rclone will read --buffer-size plus --vfs-read-
5775 ahead bytes ahead. The --buffer-size is buffered in memory whereas the
5776 --vfs-read-ahead is buffered on disk.
5777
5778 When using this mode it is recommended that --buffer-size is not set
5779 too large and --vfs-read-ahead is set large if required.
5780
5781 IMPORTANT not all file systems support sparse files. In particular
5782 FAT/exFAT do not. Rclone will perform very badly if the cache directo‐
5783 ry is on a filesystem which doesn't support sparse files and it will
5784 log an ERROR message if one is detected.
5785
5786 Fingerprinting
5787 Various parts of the VFS use fingerprinting to see if a local file copy
5788 has changed relative to a remote file. Fingerprints are made from:
5789
5790 • size
5791
5792 • modification time
5793
5794 • hash
5795
5796 where available on an object.
5797
5798 On some backends some of these attributes are slow to read (they take
5799 an extra API call per object, or extra work per object).
5800
5801 For example hash is slow with the local and sftp backends as they have
5802 to read the entire file and hash it, and modtime is slow with the s3,
5803 swift, ftp and qinqstor backends because they need to do an extra API
5804 call to fetch it.
5805
5806 If you use the --vfs-fast-fingerprint flag then rclone will not include
5807 the slow operations in the fingerprint. This makes the fingerprinting
5808 less accurate but much faster and will improve the opening time of
5809 cached files.
5810
5811 If you are running a vfs cache over local, s3 or swift backends then
5812 using this flag is recommended.
5813
5814 Note that if you change the value of this flag, the fingerprints of the
5815 files in the cache may be invalidated and the files will need to be
5816 downloaded again.
5817
5818 VFS Chunked Reading
5819 When rclone reads files from a remote it reads them in chunks. This
5820 means that rather than requesting the whole file rclone reads the chunk
5821 specified. This can reduce the used download quota for some remotes by
5822 requesting only chunks from the remote that are actually read, at the
5823 cost of an increased number of requests.
5824
5825 These flags control the chunking:
5826
5827 --vfs-read-chunk-size SizeSuffix Read the source objects in chunks (default 128M)
5828 --vfs-read-chunk-size-limit SizeSuffix Max chunk doubling size (default off)
5829
5830 Rclone will start reading a chunk of size --vfs-read-chunk-size, and
5831 then double the size for each read. When --vfs-read-chunk-size-limit
5832 is specified, and greater than --vfs-read-chunk-size, the chunk size
5833 for each open file will get doubled only until the specified value is
5834 reached. If the value is "off", which is the default, the limit is
5835 disabled and the chunk size will grow indefinitely.
5836
5837 With --vfs-read-chunk-size 100M and --vfs-read-chunk-size-limit 0 the
5838 following parts will be downloaded: 0-100M, 100M-200M, 200M-300M,
5839 300M-400M and so on. When --vfs-read-chunk-size-limit 500M is speci‐
5840 fied, the result would be 0-100M, 100M-300M, 300M-700M, 700M-1200M,
5841 1200M-1700M and so on.
5842
5843 Setting --vfs-read-chunk-size to 0 or "off" disables chunked reading.
5844
5845 VFS Performance
5846 These flags may be used to enable/disable features of the VFS for per‐
5847 formance or other reasons. See also the chunked reading feature.
5848
5849 In particular S3 and Swift benefit hugely from the --no-modtime flag
5850 (or use --use-server-modtime for a slightly different effect) as each
5851 read of the modification time takes a transaction.
5852
5853 --no-checksum Don't compare checksums on up/download.
5854 --no-modtime Don't read/write the modification time (can speed things up).
5855 --no-seek Don't allow seeking in files.
5856 --read-only Only allow read-only access.
5857
5858 Sometimes rclone is delivered reads or writes out of order. Rather
5859 than seeking rclone will wait a short time for the in sequence read or
5860 write to come in. These flags only come into effect when not using an
5861 on disk cache file.
5862
5863 --vfs-read-wait duration Time to wait for in-sequence read before seeking (default 20ms)
5864 --vfs-write-wait duration Time to wait for in-sequence write before giving error (default 1s)
5865
5866 When using VFS write caching (--vfs-cache-mode with value writes or
5867 full), the global flag --transfers can be set to adjust the number of
5868 parallel uploads of modified files from the cache (the related global
5869 flag --checkers has no effect on the VFS).
5870
5871 --transfers int Number of file transfers to run in parallel (default 4)
5872
5873 VFS Case Sensitivity
5874 Linux file systems are case-sensitive: two files can differ only by
5875 case, and the exact case must be used when opening a file.
5876
5877 File systems in modern Windows are case-insensitive but case-preserv‐
5878 ing: although existing files can be opened using any case, the exact
5879 case used to create the file is preserved and available for programs to
5880 query. It is not allowed for two files in the same directory to differ
5881 only by case.
5882
5883 Usually file systems on macOS are case-insensitive. It is possible to
5884 make macOS file systems case-sensitive but that is not the default.
5885
5886 The --vfs-case-insensitive VFS flag controls how rclone handles these
5887 two cases. If its value is "false", rclone passes file names to the
5888 remote as-is. If the flag is "true" (or appears without a value on the
5889 command line), rclone may perform a "fixup" as explained below.
5890
5891 The user may specify a file name to open/delete/rename/etc with a case
5892 different than what is stored on the remote. If an argument refers to
5893 an existing file with exactly the same name, then the case of the ex‐
5894 isting file on the disk will be used. However, if a file name with ex‐
5895 actly the same name is not found but a name differing only by case ex‐
5896 ists, rclone will transparently fixup the name. This fixup happens on‐
5897 ly when an existing file is requested. Case sensitivity of file names
5898 created anew by rclone is controlled by the underlying remote.
5899
5900 Note that case sensitivity of the operating system running rclone (the
5901 target) may differ from case sensitivity of a file system presented by
5902 rclone (the source). The flag controls whether "fixup" is performed to
5903 satisfy the target.
5904
5905 If the flag is not provided on the command line, then its default value
5906 depends on the operating system where rclone runs: "true" on Windows
5907 and macOS, "false" otherwise. If the flag is provided without a value,
5908 then it is "true".
5909
5910 VFS Disk Options
5911 This flag allows you to manually set the statistics about the filing
5912 system. It can be useful when those statistics cannot be read correct‐
5913 ly automatically.
5914
5915 --vfs-disk-space-total-size Manually set the total disk space size (example: 256G, default: -1)
5916
5917 Alternate report of used bytes
5918 Some backends, most notably S3, do not report the amount of bytes used.
5919 If you need this information to be available when running df on the
5920 filesystem, then pass the flag --vfs-used-is-size to rclone. With this
5921 flag set, instead of relying on the backend to report this information,
5922 rclone will scan the whole remote similar to rclone size and compute
5923 the total used space itself.
5924
5925 WARNING. Contrary to rclone size, this flag ignores filters so that the
5926 result is accurate. However, this is very inefficient and may cost
5927 lots of API calls resulting in extra charges. Use it as a last resort
5928 and only with caching.
5929
5930 Auth Proxy
5931 If you supply the parameter --auth-proxy /path/to/program then rclone
5932 will use that program to generate backends on the fly which then are
5933 used to authenticate incoming requests. This uses a simple JSON based
5934 protocol with input on STDIN and output on STDOUT.
5935
5936 PLEASE NOTE: --auth-proxy and --authorized-keys cannot be used togeth‐
5937 er, if --auth-proxy is set the authorized keys option will be ignored.
5938
5939 There is an example program bin/test_proxy.py
5940 (https://github.com/rclone/rclone/blob/master/test_proxy.py) in the
5941 rclone source code.
5942
5943 The program's job is to take a user and pass on the input and turn
5944 those into the config for a backend on STDOUT in JSON format. This
5945 config will have any default parameters for the backend added, but it
5946 won't use configuration from environment variables or command line op‐
5947 tions - it is the job of the proxy program to make a complete config.
5948
5949 This config generated must have this extra parameter - _root - root to
5950 use for the backend
5951
5952 And it may have this parameter - _obscure - comma separated strings for
5953 parameters to obscure
5954
5955 If password authentication was used by the client, input to the proxy
5956 process (on STDIN) would look similar to this:
5957
5958 {
5959 "user": "me",
5960 "pass": "mypassword"
5961 }
5962
5963 If public-key authentication was used by the client, input to the proxy
5964 process (on STDIN) would look similar to this:
5965
5966 {
5967 "user": "me",
5968 "public_key": "AAAAB3NzaC1yc2EAAAADAQABAAABAQDuwESFdAe14hVS6omeyX7edc...JQdf"
5969 }
5970
5971 And as an example return this on STDOUT
5972
5973 {
5974 "type": "sftp",
5975 "_root": "",
5976 "_obscure": "pass",
5977 "user": "me",
5978 "pass": "mypassword",
5979 "host": "sftp.example.com"
5980 }
5981
5982 This would mean that an SFTP backend would be created on the fly for
5983 the user and pass/public_key returned in the output to the host given.
5984 Note that since _obscure is set to pass, rclone will obscure the pass
5985 parameter before creating the backend (which is required for sftp back‐
5986 ends).
5987
5988 The program can manipulate the supplied user in any way, for example to
5989 make proxy to many different sftp backends, you could make the user be
5990 user@example.com and then set the host to example.com in the output and
5991 the user to user. For security you'd probably want to restrict the
5992 host to a limited list.
5993
5994 Note that an internal cache is keyed on user so only use that for con‐
5995 figuration, don't use pass or public_key. This also means that if a
5996 user's password or public-key is changed the cache will need to expire
5997 (which takes 5 mins) before it takes effect.
5998
5999 This can be used to build general purpose proxies to any kind of back‐
6000 end that rclone supports.
6001
6002 rclone serve ftp remote:path [flags]
6003
6004 Options
6005 --addr string IPaddress:Port or :Port to bind server to (default "localhost:2121")
6006 --auth-proxy string A program to use to create the backend from the auth
6007 --cert string TLS PEM key (concatenation of certificate and CA certificate)
6008 --dir-cache-time duration Time to cache directory entries for (default 5m0s)
6009 --dir-perms FileMode Directory permissions (default 0777)
6010 --file-perms FileMode File permissions (default 0666)
6011 --gid uint32 Override the gid field set by the filesystem (not supported on Windows) (default 1000)
6012 -h, --help help for ftp
6013 --key string TLS PEM Private key
6014 --no-checksum Don't compare checksums on up/download
6015 --no-modtime Don't read/write the modification time (can speed things up)
6016 --no-seek Don't allow seeking in files
6017 --pass string Password for authentication (empty value allow every password)
6018 --passive-port string Passive port range to use (default "30000-32000")
6019 --poll-interval duration Time to wait between polling for changes, must be smaller than dir-cache-time and only on supported remotes (set 0 to disable) (default 1m0s)
6020 --public-ip string Public IP address to advertise for passive connections
6021 --read-only Only allow read-only access
6022 --uid uint32 Override the uid field set by the filesystem (not supported on Windows) (default 1000)
6023 --umask int Override the permission bits set by the filesystem (not supported on Windows) (default 2)
6024 --user string User name for authentication (default "anonymous")
6025 --vfs-cache-max-age duration Max age of objects in the cache (default 1h0m0s)
6026 --vfs-cache-max-size SizeSuffix Max total size of objects in the cache (default off)
6027 --vfs-cache-mode CacheMode Cache mode off|minimal|writes|full (default off)
6028 --vfs-cache-poll-interval duration Interval to poll the cache for stale objects (default 1m0s)
6029 --vfs-case-insensitive If a file name not found, find a case insensitive match
6030 --vfs-disk-space-total-size SizeSuffix Specify the total space of disk (default off)
6031 --vfs-fast-fingerprint Use fast (less accurate) fingerprints for change detection
6032 --vfs-read-ahead SizeSuffix Extra read ahead over --buffer-size when using cache-mode full
6033 --vfs-read-chunk-size SizeSuffix Read the source objects in chunks (default 128Mi)
6034 --vfs-read-chunk-size-limit SizeSuffix If greater than --vfs-read-chunk-size, double the chunk size after each chunk read, until the limit is reached ('off' is unlimited) (default off)
6035 --vfs-read-wait duration Time to wait for in-sequence read before seeking (default 20ms)
6036 --vfs-used-is-size rclone size Use the rclone size algorithm for Used size
6037 --vfs-write-back duration Time to writeback files after last use when using cache (default 5s)
6038 --vfs-write-wait duration Time to wait for in-sequence write before giving error (default 1s)
6039
6040 See the global flags page (https://rclone.org/flags/) for global op‐
6041 tions not listed here.
6042
6043 SEE ALSO
6044 • rclone serve (https://rclone.org/commands/rclone_serve/) - Serve a
6045 remote over a protocol.
6046
6048 Serve the remote over HTTP.
6049
6050 Synopsis
6051 Run a basic web server to serve a remote over HTTP. This can be viewed
6052 in a web browser or you can make a remote of type http read from it.
6053
6054 You can use the filter flags (e.g. --include, --exclude) to control
6055 what is served.
6056
6057 The server will log errors. Use -v to see access logs.
6058
6059 --bwlimit will be respected for file transfers. Use --stats to control
6060 the stats printing.
6061
6062 Server options
6063 Use --addr to specify which IP address and port the server should lis‐
6064 ten on, eg --addr 1.2.3.4:8000 or --addr :8080 to listen to all IPs.
6065 By default it only listens on localhost. You can use port :0 to let
6066 the OS choose an available port.
6067
6068 If you set --addr to listen on a public or LAN accessible IP address
6069 then using Authentication is advised - see the next section for info.
6070
6071 --server-read-timeout and --server-write-timeout can be used to control
6072 the timeouts on the server. Note that this is the total time for a
6073 transfer.
6074
6075 --max-header-bytes controls the maximum number of bytes the server will
6076 accept in the HTTP header.
6077
6078 --baseurl controls the URL prefix that rclone serves from. By default
6079 rclone will serve from the root. If you used --baseurl "/rclone" then
6080 rclone would serve from a URL starting with "/rclone/". This is useful
6081 if you wish to proxy rclone serve. Rclone automatically inserts lead‐
6082 ing and trailing "/" on --baseurl, so --baseurl "rclone", --baseurl
6083 "/rclone" and --baseurl "/rclone/" are all treated identically.
6084
6085 SSL/TLS
6086 By default this will serve over http. If you want you can serve over
6087 https. You will need to supply the --cert and --key flags. If you
6088 wish to do client side certificate validation then you will need to
6089 supply --client-ca also.
6090
6091 --cert should be a either a PEM encoded certificate or a concatenation
6092 of that with the CA certificate. --key should be the PEM encoded pri‐
6093 vate key and --client-ca should be the PEM encoded client certificate
6094 authority certificate.
6095
6096 --min-tls-version is minimum TLS version that is acceptable. Valid
6097 values are "tls1.0", "tls1.1", "tls1.2" and "tls1.3" (default
6098 "tls1.0").
6099
6100 Template
6101 --template allows a user to specify a custom markup template for HTTP
6102 and WebDAV serve functions. The server exports the following markup to
6103 be used within the template to server pages:
6104
6105 Parameter Description
6106 ──────────────────────────────────────────────────────────────────────────
6107 .Name The full path of a file/directory.
6108 .Title Directory listing of .Name
6109 .Sort The current sort used. This is
6110 changeable via ?sort= parameter
6111 Sort Options:
6112 namedirfirst,name,size,time (de‐
6113 fault namedirfirst)
6114 .Order The current ordering used. This is
6115 changeable via ?order= parameter
6116 Order Options: asc,desc (default
6117 asc)
6118 .Query Currently unused.
6119 .Breadcrumb Allows for creating a relative nav‐
6120 igation
6121 -- .Link The relative to the root link of
6122 the Text.
6123 -- .Text The Name of the directory.
6124 .Entries Information about a specific
6125 file/directory.
6126 -- .URL The 'url' of an entry.
6127 -- .Leaf Currently same as 'URL' but intend‐
6128 ed to be 'just' the name.
6129
6130 -- .IsDir Boolean for if an entry is a direc‐
6131 tory or not.
6132 -- .Size Size in Bytes of the entry.
6133 -- .ModTime The UTC timestamp of an entry.
6134
6135 Authentication
6136 By default this will serve files without needing a login.
6137
6138 You can either use an htpasswd file which can take lots of users, or
6139 set a single username and password with the --user and --pass flags.
6140
6141 Use --htpasswd /path/to/htpasswd to provide an htpasswd file. This is
6142 in standard apache format and supports MD5, SHA1 and BCrypt for basic
6143 authentication. Bcrypt is recommended.
6144
6145 To create an htpasswd file:
6146
6147 touch htpasswd
6148 htpasswd -B htpasswd user
6149 htpasswd -B htpasswd anotherUser
6150
6151 The password file can be updated while rclone is running.
6152
6153 Use --realm to set the authentication realm.
6154
6155 Use --salt to change the password hashing salt from the default.
6156
6157 VFS - Virtual File System
6158 This command uses the VFS layer. This adapts the cloud storage objects
6159 that rclone uses into something which looks much more like a disk fil‐
6160 ing system.
6161
6162 Cloud storage objects have lots of properties which aren't like disk
6163 files - you can't extend them or write to the middle of them, so the
6164 VFS layer has to deal with that. Because there is no one right way of
6165 doing this there are various options explained below.
6166
6167 The VFS layer also implements a directory cache - this caches info
6168 about files and directories (but not the data) in memory.
6169
6170 VFS Directory Cache
6171 Using the --dir-cache-time flag, you can control how long a directory
6172 should be considered up to date and not refreshed from the backend.
6173 Changes made through the VFS will appear immediately or invalidate the
6174 cache.
6175
6176 --dir-cache-time duration Time to cache directory entries for (default 5m0s)
6177 --poll-interval duration Time to wait between polling for changes. Must be smaller than dir-cache-time. Only on supported remotes. Set to 0 to disable (default 1m0s)
6178
6179 However, changes made directly on the cloud storage by the web inter‐
6180 face or a different copy of rclone will only be picked up once the di‐
6181 rectory cache expires if the backend configured does not support
6182 polling for changes. If the backend supports polling, changes will be
6183 picked up within the polling interval.
6184
6185 You can send a SIGHUP signal to rclone for it to flush all directory
6186 caches, regardless of how old they are. Assuming only one rclone in‐
6187 stance is running, you can reset the cache like this:
6188
6189 kill -SIGHUP $(pidof rclone)
6190
6191 If you configure rclone with a remote control then you can use rclone
6192 rc to flush the whole directory cache:
6193
6194 rclone rc vfs/forget
6195
6196 Or individual files or directories:
6197
6198 rclone rc vfs/forget file=path/to/file dir=path/to/dir
6199
6200 VFS File Buffering
6201 The --buffer-size flag determines the amount of memory, that will be
6202 used to buffer data in advance.
6203
6204 Each open file will try to keep the specified amount of data in memory
6205 at all times. The buffered data is bound to one open file and won't be
6206 shared.
6207
6208 This flag is a upper limit for the used memory per open file. The buf‐
6209 fer will only use memory for data that is downloaded but not not yet
6210 read. If the buffer is empty, only a small amount of memory will be
6211 used.
6212
6213 The maximum memory used by rclone for buffering can be up to --buffer-
6214 size * open files.
6215
6216 VFS File Caching
6217 These flags control the VFS file caching options. File caching is nec‐
6218 essary to make the VFS layer appear compatible with a normal file sys‐
6219 tem. It can be disabled at the cost of some compatibility.
6220
6221 For example you'll need to enable VFS caching if you want to read and
6222 write simultaneously to a file. See below for more details.
6223
6224 Note that the VFS cache is separate from the cache backend and you may
6225 find that you need one or the other or both.
6226
6227 --cache-dir string Directory rclone will use for caching.
6228 --vfs-cache-mode CacheMode Cache mode off|minimal|writes|full (default off)
6229 --vfs-cache-max-age duration Max age of objects in the cache (default 1h0m0s)
6230 --vfs-cache-max-size SizeSuffix Max total size of objects in the cache (default off)
6231 --vfs-cache-poll-interval duration Interval to poll the cache for stale objects (default 1m0s)
6232 --vfs-write-back duration Time to writeback files after last use when using cache (default 5s)
6233
6234 If run with -vv rclone will print the location of the file cache. The
6235 files are stored in the user cache file area which is OS dependent but
6236 can be controlled with --cache-dir or setting the appropriate environ‐
6237 ment variable.
6238
6239 The cache has 4 different modes selected by --vfs-cache-mode. The
6240 higher the cache mode the more compatible rclone becomes at the cost of
6241 using disk space.
6242
6243 Note that files are written back to the remote only when they are
6244 closed and if they haven't been accessed for --vfs-write-back seconds.
6245 If rclone is quit or dies with files that haven't been uploaded, these
6246 will be uploaded next time rclone is run with the same flags.
6247
6248 If using --vfs-cache-max-size note that the cache may exceed this size
6249 for two reasons. Firstly because it is only checked every --vfs-cache-
6250 poll-interval. Secondly because open files cannot be evicted from the
6251 cache.
6252
6253 You should not run two copies of rclone using the same VFS cache with
6254 the same or overlapping remotes if using --vfs-cache-mode > off. This
6255 can potentially cause data corruption if you do. You can work around
6256 this by giving each rclone its own cache hierarchy with --cache-dir.
6257 You don't need to worry about this if the remotes in use don't overlap.
6258
6259 --vfs-cache-mode off
6260 In this mode (the default) the cache will read directly from the remote
6261 and write directly to the remote without caching anything on disk.
6262
6263 This will mean some operations are not possible
6264
6265 • Files can't be opened for both read AND write
6266
6267 • Files opened for write can't be seeked
6268
6269 • Existing files opened for write must have O_TRUNC set
6270
6271 • Files open for read with O_TRUNC will be opened write only
6272
6273 • Files open for write only will behave as if O_TRUNC was supplied
6274
6275 • Open modes O_APPEND, O_TRUNC are ignored
6276
6277 • If an upload fails it can't be retried
6278
6279 --vfs-cache-mode minimal
6280 This is very similar to "off" except that files opened for read AND
6281 write will be buffered to disk. This means that files opened for write
6282 will be a lot more compatible, but uses the minimal disk space.
6283
6284 These operations are not possible
6285
6286 • Files opened for write only can't be seeked
6287
6288 • Existing files opened for write must have O_TRUNC set
6289
6290 • Files opened for write only will ignore O_APPEND, O_TRUNC
6291
6292 • If an upload fails it can't be retried
6293
6294 --vfs-cache-mode writes
6295 In this mode files opened for read only are still read directly from
6296 the remote, write only and read/write files are buffered to disk first.
6297
6298 This mode should support all normal file system operations.
6299
6300 If an upload fails it will be retried at exponentially increasing in‐
6301 tervals up to 1 minute.
6302
6303 --vfs-cache-mode full
6304 In this mode all reads and writes are buffered to and from disk. When
6305 data is read from the remote this is buffered to disk as well.
6306
6307 In this mode the files in the cache will be sparse files and rclone
6308 will keep track of which bits of the files it has downloaded.
6309
6310 So if an application only reads the starts of each file, then rclone
6311 will only buffer the start of the file. These files will appear to be
6312 their full size in the cache, but they will be sparse files with only
6313 the data that has been downloaded present in them.
6314
6315 This mode should support all normal file system operations and is oth‐
6316 erwise identical to --vfs-cache-mode writes.
6317
6318 When reading a file rclone will read --buffer-size plus --vfs-read-
6319 ahead bytes ahead. The --buffer-size is buffered in memory whereas the
6320 --vfs-read-ahead is buffered on disk.
6321
6322 When using this mode it is recommended that --buffer-size is not set
6323 too large and --vfs-read-ahead is set large if required.
6324
6325 IMPORTANT not all file systems support sparse files. In particular
6326 FAT/exFAT do not. Rclone will perform very badly if the cache directo‐
6327 ry is on a filesystem which doesn't support sparse files and it will
6328 log an ERROR message if one is detected.
6329
6330 Fingerprinting
6331 Various parts of the VFS use fingerprinting to see if a local file copy
6332 has changed relative to a remote file. Fingerprints are made from:
6333
6334 • size
6335
6336 • modification time
6337
6338 • hash
6339
6340 where available on an object.
6341
6342 On some backends some of these attributes are slow to read (they take
6343 an extra API call per object, or extra work per object).
6344
6345 For example hash is slow with the local and sftp backends as they have
6346 to read the entire file and hash it, and modtime is slow with the s3,
6347 swift, ftp and qinqstor backends because they need to do an extra API
6348 call to fetch it.
6349
6350 If you use the --vfs-fast-fingerprint flag then rclone will not include
6351 the slow operations in the fingerprint. This makes the fingerprinting
6352 less accurate but much faster and will improve the opening time of
6353 cached files.
6354
6355 If you are running a vfs cache over local, s3 or swift backends then
6356 using this flag is recommended.
6357
6358 Note that if you change the value of this flag, the fingerprints of the
6359 files in the cache may be invalidated and the files will need to be
6360 downloaded again.
6361
6362 VFS Chunked Reading
6363 When rclone reads files from a remote it reads them in chunks. This
6364 means that rather than requesting the whole file rclone reads the chunk
6365 specified. This can reduce the used download quota for some remotes by
6366 requesting only chunks from the remote that are actually read, at the
6367 cost of an increased number of requests.
6368
6369 These flags control the chunking:
6370
6371 --vfs-read-chunk-size SizeSuffix Read the source objects in chunks (default 128M)
6372 --vfs-read-chunk-size-limit SizeSuffix Max chunk doubling size (default off)
6373
6374 Rclone will start reading a chunk of size --vfs-read-chunk-size, and
6375 then double the size for each read. When --vfs-read-chunk-size-limit
6376 is specified, and greater than --vfs-read-chunk-size, the chunk size
6377 for each open file will get doubled only until the specified value is
6378 reached. If the value is "off", which is the default, the limit is
6379 disabled and the chunk size will grow indefinitely.
6380
6381 With --vfs-read-chunk-size 100M and --vfs-read-chunk-size-limit 0 the
6382 following parts will be downloaded: 0-100M, 100M-200M, 200M-300M,
6383 300M-400M and so on. When --vfs-read-chunk-size-limit 500M is speci‐
6384 fied, the result would be 0-100M, 100M-300M, 300M-700M, 700M-1200M,
6385 1200M-1700M and so on.
6386
6387 Setting --vfs-read-chunk-size to 0 or "off" disables chunked reading.
6388
6389 VFS Performance
6390 These flags may be used to enable/disable features of the VFS for per‐
6391 formance or other reasons. See also the chunked reading feature.
6392
6393 In particular S3 and Swift benefit hugely from the --no-modtime flag
6394 (or use --use-server-modtime for a slightly different effect) as each
6395 read of the modification time takes a transaction.
6396
6397 --no-checksum Don't compare checksums on up/download.
6398 --no-modtime Don't read/write the modification time (can speed things up).
6399 --no-seek Don't allow seeking in files.
6400 --read-only Only allow read-only access.
6401
6402 Sometimes rclone is delivered reads or writes out of order. Rather
6403 than seeking rclone will wait a short time for the in sequence read or
6404 write to come in. These flags only come into effect when not using an
6405 on disk cache file.
6406
6407 --vfs-read-wait duration Time to wait for in-sequence read before seeking (default 20ms)
6408 --vfs-write-wait duration Time to wait for in-sequence write before giving error (default 1s)
6409
6410 When using VFS write caching (--vfs-cache-mode with value writes or
6411 full), the global flag --transfers can be set to adjust the number of
6412 parallel uploads of modified files from the cache (the related global
6413 flag --checkers has no effect on the VFS).
6414
6415 --transfers int Number of file transfers to run in parallel (default 4)
6416
6417 VFS Case Sensitivity
6418 Linux file systems are case-sensitive: two files can differ only by
6419 case, and the exact case must be used when opening a file.
6420
6421 File systems in modern Windows are case-insensitive but case-preserv‐
6422 ing: although existing files can be opened using any case, the exact
6423 case used to create the file is preserved and available for programs to
6424 query. It is not allowed for two files in the same directory to differ
6425 only by case.
6426
6427 Usually file systems on macOS are case-insensitive. It is possible to
6428 make macOS file systems case-sensitive but that is not the default.
6429
6430 The --vfs-case-insensitive VFS flag controls how rclone handles these
6431 two cases. If its value is "false", rclone passes file names to the
6432 remote as-is. If the flag is "true" (or appears without a value on the
6433 command line), rclone may perform a "fixup" as explained below.
6434
6435 The user may specify a file name to open/delete/rename/etc with a case
6436 different than what is stored on the remote. If an argument refers to
6437 an existing file with exactly the same name, then the case of the ex‐
6438 isting file on the disk will be used. However, if a file name with ex‐
6439 actly the same name is not found but a name differing only by case ex‐
6440 ists, rclone will transparently fixup the name. This fixup happens on‐
6441 ly when an existing file is requested. Case sensitivity of file names
6442 created anew by rclone is controlled by the underlying remote.
6443
6444 Note that case sensitivity of the operating system running rclone (the
6445 target) may differ from case sensitivity of a file system presented by
6446 rclone (the source). The flag controls whether "fixup" is performed to
6447 satisfy the target.
6448
6449 If the flag is not provided on the command line, then its default value
6450 depends on the operating system where rclone runs: "true" on Windows
6451 and macOS, "false" otherwise. If the flag is provided without a value,
6452 then it is "true".
6453
6454 VFS Disk Options
6455 This flag allows you to manually set the statistics about the filing
6456 system. It can be useful when those statistics cannot be read correct‐
6457 ly automatically.
6458
6459 --vfs-disk-space-total-size Manually set the total disk space size (example: 256G, default: -1)
6460
6461 Alternate report of used bytes
6462 Some backends, most notably S3, do not report the amount of bytes used.
6463 If you need this information to be available when running df on the
6464 filesystem, then pass the flag --vfs-used-is-size to rclone. With this
6465 flag set, instead of relying on the backend to report this information,
6466 rclone will scan the whole remote similar to rclone size and compute
6467 the total used space itself.
6468
6469 WARNING. Contrary to rclone size, this flag ignores filters so that the
6470 result is accurate. However, this is very inefficient and may cost
6471 lots of API calls resulting in extra charges. Use it as a last resort
6472 and only with caching.
6473
6474 rclone serve http remote:path [flags]
6475
6476 Options
6477 --addr string IPaddress:Port or :Port to bind server to (default "127.0.0.1:8080")
6478 --baseurl string Prefix for URLs - leave blank for root
6479 --cert string SSL PEM key (concatenation of certificate and CA certificate)
6480 --client-ca string Client certificate authority to verify clients with
6481 --dir-cache-time duration Time to cache directory entries for (default 5m0s)
6482 --dir-perms FileMode Directory permissions (default 0777)
6483 --file-perms FileMode File permissions (default 0666)
6484 --gid uint32 Override the gid field set by the filesystem (not supported on Windows) (default 1000)
6485 -h, --help help for http
6486 --htpasswd string A htpasswd file - if not provided no authentication is done
6487 --key string SSL PEM Private key
6488 --max-header-bytes int Maximum size of request header (default 4096)
6489 --min-tls-version string Minimum TLS version that is acceptable (default "tls1.0")
6490 --no-checksum Don't compare checksums on up/download
6491 --no-modtime Don't read/write the modification time (can speed things up)
6492 --no-seek Don't allow seeking in files
6493 --pass string Password for authentication
6494 --poll-interval duration Time to wait between polling for changes, must be smaller than dir-cache-time and only on supported remotes (set 0 to disable) (default 1m0s)
6495 --read-only Only allow read-only access
6496 --realm string Realm for authentication
6497 --salt string Password hashing salt (default "dlPL2MqE")
6498 --server-read-timeout duration Timeout for server reading data (default 1h0m0s)
6499 --server-write-timeout duration Timeout for server writing data (default 1h0m0s)
6500 --template string User-specified template
6501 --uid uint32 Override the uid field set by the filesystem (not supported on Windows) (default 1000)
6502 --umask int Override the permission bits set by the filesystem (not supported on Windows) (default 2)
6503 --user string User name for authentication
6504 --vfs-cache-max-age duration Max age of objects in the cache (default 1h0m0s)
6505 --vfs-cache-max-size SizeSuffix Max total size of objects in the cache (default off)
6506 --vfs-cache-mode CacheMode Cache mode off|minimal|writes|full (default off)
6507 --vfs-cache-poll-interval duration Interval to poll the cache for stale objects (default 1m0s)
6508 --vfs-case-insensitive If a file name not found, find a case insensitive match
6509 --vfs-disk-space-total-size SizeSuffix Specify the total space of disk (default off)
6510 --vfs-fast-fingerprint Use fast (less accurate) fingerprints for change detection
6511 --vfs-read-ahead SizeSuffix Extra read ahead over --buffer-size when using cache-mode full
6512 --vfs-read-chunk-size SizeSuffix Read the source objects in chunks (default 128Mi)
6513 --vfs-read-chunk-size-limit SizeSuffix If greater than --vfs-read-chunk-size, double the chunk size after each chunk read, until the limit is reached ('off' is unlimited) (default off)
6514 --vfs-read-wait duration Time to wait for in-sequence read before seeking (default 20ms)
6515 --vfs-used-is-size rclone size Use the rclone size algorithm for Used size
6516 --vfs-write-back duration Time to writeback files after last use when using cache (default 5s)
6517 --vfs-write-wait duration Time to wait for in-sequence write before giving error (default 1s)
6518
6519 See the global flags page (https://rclone.org/flags/) for global op‐
6520 tions not listed here.
6521
6522 SEE ALSO
6523 • rclone serve (https://rclone.org/commands/rclone_serve/) - Serve a
6524 remote over a protocol.
6525
6527 Serve the remote for restic's REST API.
6528
6529 Synopsis
6530 Run a basic web server to serve a remove over restic's REST backend API
6531 over HTTP. This allows restic to use rclone as a data storage mecha‐
6532 nism for cloud providers that restic does not support directly.
6533
6534 Restic (https://restic.net/) is a command-line program for doing back‐
6535 ups.
6536
6537 The server will log errors. Use -v to see access logs.
6538
6539 --bwlimit will be respected for file transfers. Use --stats to control
6540 the stats printing.
6541
6542 Setting up rclone for use by restic
6543 First set up a remote for your chosen cloud provider
6544 (https://rclone.org/docs/#configure).
6545
6546 Once you have set up the remote, check it is working with, for example
6547 "rclone lsd remote:". You may have called the remote something other
6548 than "remote:" - just substitute whatever you called it in the follow‐
6549 ing instructions.
6550
6551 Now start the rclone restic server
6552
6553 rclone serve restic -v remote:backup
6554
6555 Where you can replace "backup" in the above by whatever path in the re‐
6556 mote you wish to use.
6557
6558 By default this will serve on "localhost:8080" you can change this with
6559 use of the --addr flag.
6560
6561 You might wish to start this server on boot.
6562
6563 Adding --cache-objects=false will cause rclone to stop caching objects
6564 returned from the List call. Caching is normally desirable as it
6565 speeds up downloading objects, saves transactions and uses very little
6566 memory.
6567
6568 Setting up restic to use rclone
6569 Now you can follow the restic instructions (http://restic.readthedo‐
6570 cs.io/en/latest/030_preparing_a_new_repo.html#rest-server) on setting
6571 up restic.
6572
6573 Note that you will need restic 0.8.2 or later to interoperate with
6574 rclone.
6575
6576 For the example above you will want to use "http://localhost:8080/" as
6577 the URL for the REST server.
6578
6579 For example:
6580
6581 $ export RESTIC_REPOSITORY=rest:http://localhost:8080/
6582 $ export RESTIC_PASSWORD=yourpassword
6583 $ restic init
6584 created restic backend 8b1a4b56ae at rest:http://localhost:8080/
6585
6586 Please note that knowledge of your password is required to access
6587 the repository. Losing your password means that your data is
6588 irrecoverably lost.
6589 $ restic backup /path/to/files/to/backup
6590 scan [/path/to/files/to/backup]
6591 scanned 189 directories, 312 files in 0:00
6592 [0:00] 100.00% 38.128 MiB / 38.128 MiB 501 / 501 items 0 errors ETA 0:00
6593 duration: 0:00
6594 snapshot 45c8fdd8 saved
6595
6596 Multiple repositories
6597 Note that you can use the endpoint to host multiple repositories. Do
6598 this by adding a directory name or path after the URL. Note that these
6599 must end with /. Eg
6600
6601 $ export RESTIC_REPOSITORY=rest:http://localhost:8080/user1repo/
6602 # backup user1 stuff
6603 $ export RESTIC_REPOSITORY=rest:http://localhost:8080/user2repo/
6604 # backup user2 stuff
6605
6606 Private repositories
6607 The--private-repos flag can be used to limit users to repositories
6608 starting with a path of /<username>/.
6609
6610 Server options
6611 Use --addr to specify which IP address and port the server should lis‐
6612 ten on, e.g. --addr 1.2.3.4:8000 or --addr :8080 to listen to all IPs.
6613 By default it only listens on localhost. You can use port :0 to let
6614 the OS choose an available port.
6615
6616 If you set --addr to listen on a public or LAN accessible IP address
6617 then using Authentication is advised - see the next section for info.
6618
6619 --server-read-timeout and --server-write-timeout can be used to control
6620 the timeouts on the server. Note that this is the total time for a
6621 transfer.
6622
6623 --max-header-bytes controls the maximum number of bytes the server will
6624 accept in the HTTP header.
6625
6626 --baseurl controls the URL prefix that rclone serves from. By default
6627 rclone will serve from the root. If you used --baseurl "/rclone" then
6628 rclone would serve from a URL starting with "/rclone/". This is useful
6629 if you wish to proxy rclone serve. Rclone automatically inserts lead‐
6630 ing and trailing "/" on --baseurl, so --baseurl "rclone", --baseurl
6631 "/rclone" and --baseurl "/rclone/" are all treated identically.
6632
6633 --template allows a user to specify a custom markup template for HTTP
6634 and WebDAV serve functions. The server exports the following markup to
6635 be used within the template to server pages:
6636
6637 Parameter Description
6638 ──────────────────────────────────────────────────────────────────────────
6639 .Name The full path of a file/directory.
6640 .Title Directory listing of .Name
6641 .Sort The current sort used. This is
6642 changeable via ?sort= parameter
6643 Sort Options:
6644 namedirfirst,name,size,time (de‐
6645 fault namedirfirst)
6646 .Order The current ordering used. This is
6647 changeable via ?order= parameter
6648 Order Options: asc,desc (default
6649 asc)
6650 .Query Currently unused.
6651 .Breadcrumb Allows for creating a relative nav‐
6652 igation
6653 -- .Link The relative to the root link of
6654 the Text.
6655 -- .Text The Name of the directory.
6656 .Entries Information about a specific
6657 file/directory.
6658 -- .URL The 'url' of an entry.
6659 -- .Leaf Currently same as 'URL' but intend‐
6660 ed to be 'just' the name.
6661 -- .IsDir Boolean for if an entry is a direc‐
6662 tory or not.
6663 -- .Size Size in Bytes of the entry.
6664 -- .ModTime The UTC timestamp of an entry.
6665
6666 Authentication
6667 By default this will serve files without needing a login.
6668
6669 You can either use an htpasswd file which can take lots of users, or
6670 set a single username and password with the --user and --pass flags.
6671
6672 Use --htpasswd /path/to/htpasswd to provide an htpasswd file. This is
6673 in standard apache format and supports MD5, SHA1 and BCrypt for basic
6674 authentication. Bcrypt is recommended.
6675
6676 To create an htpasswd file:
6677
6678 touch htpasswd
6679 htpasswd -B htpasswd user
6680 htpasswd -B htpasswd anotherUser
6681
6682 The password file can be updated while rclone is running.
6683
6684 Use --realm to set the authentication realm.
6685
6686 SSL/TLS
6687 By default this will serve over HTTP. If you want you can serve over
6688 HTTPS. You will need to supply the --cert and --key flags. If you
6689 wish to do client side certificate validation then you will need to
6690 supply --client-ca also.
6691
6692 --cert should be either a PEM encoded certificate or a concatenation of
6693 that with the CA certificate. --key should be the PEM encoded private
6694 key and --client-ca should be the PEM encoded client certificate au‐
6695 thority certificate.
6696
6697 --min-tls-version is minimum TLS version that is acceptable. Valid
6698 values are "tls1.0", "tls1.1", "tls1.2" and "tls1.3" (default
6699 "tls1.0").
6700
6701 rclone serve restic remote:path [flags]
6702
6703 Options
6704 --addr string IPaddress:Port or :Port to bind server to (default "localhost:8080")
6705 --append-only Disallow deletion of repository data
6706 --baseurl string Prefix for URLs - leave blank for root
6707 --cache-objects Cache listed objects (default true)
6708 --cert string SSL PEM key (concatenation of certificate and CA certificate)
6709 --client-ca string Client certificate authority to verify clients with
6710 -h, --help help for restic
6711 --htpasswd string htpasswd file - if not provided no authentication is done
6712 --key string SSL PEM Private key
6713 --max-header-bytes int Maximum size of request header (default 4096)
6714 --min-tls-version string Minimum TLS version that is acceptable (default "tls1.0")
6715 --pass string Password for authentication
6716 --private-repos Users can only access their private repo
6717 --realm string Realm for authentication (default "rclone")
6718 --server-read-timeout duration Timeout for server reading data (default 1h0m0s)
6719 --server-write-timeout duration Timeout for server writing data (default 1h0m0s)
6720 --stdio Run an HTTP2 server on stdin/stdout
6721 --template string User-specified template
6722 --user string User name for authentication
6723
6724 See the global flags page (https://rclone.org/flags/) for global op‐
6725 tions not listed here.
6726
6727 SEE ALSO
6728 • rclone serve (https://rclone.org/commands/rclone_serve/) - Serve a
6729 remote over a protocol.
6730
6732 Serve the remote over SFTP.
6733
6734 Synopsis
6735 Run an SFTP server to serve a remote over SFTP. This can be used with
6736 an SFTP client or you can make a remote of type sftp to use with it.
6737
6738 You can use the filter flags (e.g. --include, --exclude) to control
6739 what is served.
6740
6741 The server will respond to a small number of shell commands, mainly
6742 md5sum, sha1sum and df, which enable it to provide support for check‐
6743 sums and the about feature when accessed from an sftp remote.
6744
6745 Note that this server uses standard 32 KiB packet payload size, which
6746 means you must not configure the client to expect anything else, e.g.
6747 with the chunk_size (https://rclone.org/sftp/#sftp-chunk-size) option
6748 on an sftp remote.
6749
6750 The server will log errors. Use -v to see access logs.
6751
6752 --bwlimit will be respected for file transfers. Use --stats to control
6753 the stats printing.
6754
6755 You must provide some means of authentication, either with --us‐
6756 er/--pass, an authorized keys file (specify location with --authorized-
6757 keys - the default is the same as ssh), an --auth-proxy, or set the
6758 --no-auth flag for no authentication when logging in.
6759
6760 If you don't supply a host --key then rclone will generate rsa, ecdsa
6761 and ed25519 variants, and cache them for later use in rclone's cache
6762 directory (see rclone help flags cache-dir) in the "serve-sftp" direc‐
6763 tory.
6764
6765 By default the server binds to localhost:2022 - if you want it to be
6766 reachable externally then supply --addr :2022 for example.
6767
6768 Note that the default of --vfs-cache-mode off is fine for the rclone
6769 sftp backend, but it may not be with other SFTP clients.
6770
6771 If --stdio is specified, rclone will serve SFTP over stdio, which can
6772 be used with sshd via ~/.ssh/authorized_keys, for example:
6773
6774 restrict,command="rclone serve sftp --stdio ./photos" ssh-rsa ...
6775
6776 On the client you need to set --transfers 1 when using --stdio. Other‐
6777 wise multiple instances of the rclone server are started by OpenSSH
6778 which can lead to "corrupted on transfer" errors. This is the case be‐
6779 cause the client chooses indiscriminately which server to send commands
6780 to while the servers all have different views of the state of the fil‐
6781 ing system.
6782
6783 The "restrict" in authorized_keys prevents SHA1SUMs and MD5SUMs from
6784 beeing used. Omitting "restrict" and using --sftp-path-override to en‐
6785 able checksumming is possible but less secure and you could use the
6786 SFTP server provided by OpenSSH in this case.
6787
6788 VFS - Virtual File System
6789 This command uses the VFS layer. This adapts the cloud storage objects
6790 that rclone uses into something which looks much more like a disk fil‐
6791 ing system.
6792
6793 Cloud storage objects have lots of properties which aren't like disk
6794 files - you can't extend them or write to the middle of them, so the
6795 VFS layer has to deal with that. Because there is no one right way of
6796 doing this there are various options explained below.
6797
6798 The VFS layer also implements a directory cache - this caches info
6799 about files and directories (but not the data) in memory.
6800
6801 VFS Directory Cache
6802 Using the --dir-cache-time flag, you can control how long a directory
6803 should be considered up to date and not refreshed from the backend.
6804 Changes made through the VFS will appear immediately or invalidate the
6805 cache.
6806
6807 --dir-cache-time duration Time to cache directory entries for (default 5m0s)
6808 --poll-interval duration Time to wait between polling for changes. Must be smaller than dir-cache-time. Only on supported remotes. Set to 0 to disable (default 1m0s)
6809
6810 However, changes made directly on the cloud storage by the web inter‐
6811 face or a different copy of rclone will only be picked up once the di‐
6812 rectory cache expires if the backend configured does not support
6813 polling for changes. If the backend supports polling, changes will be
6814 picked up within the polling interval.
6815
6816 You can send a SIGHUP signal to rclone for it to flush all directory
6817 caches, regardless of how old they are. Assuming only one rclone in‐
6818 stance is running, you can reset the cache like this:
6819
6820 kill -SIGHUP $(pidof rclone)
6821
6822 If you configure rclone with a remote control then you can use rclone
6823 rc to flush the whole directory cache:
6824
6825 rclone rc vfs/forget
6826
6827 Or individual files or directories:
6828
6829 rclone rc vfs/forget file=path/to/file dir=path/to/dir
6830
6831 VFS File Buffering
6832 The --buffer-size flag determines the amount of memory, that will be
6833 used to buffer data in advance.
6834
6835 Each open file will try to keep the specified amount of data in memory
6836 at all times. The buffered data is bound to one open file and won't be
6837 shared.
6838
6839 This flag is a upper limit for the used memory per open file. The buf‐
6840 fer will only use memory for data that is downloaded but not not yet
6841 read. If the buffer is empty, only a small amount of memory will be
6842 used.
6843
6844 The maximum memory used by rclone for buffering can be up to --buffer-
6845 size * open files.
6846
6847 VFS File Caching
6848 These flags control the VFS file caching options. File caching is nec‐
6849 essary to make the VFS layer appear compatible with a normal file sys‐
6850 tem. It can be disabled at the cost of some compatibility.
6851
6852 For example you'll need to enable VFS caching if you want to read and
6853 write simultaneously to a file. See below for more details.
6854
6855 Note that the VFS cache is separate from the cache backend and you may
6856 find that you need one or the other or both.
6857
6858 --cache-dir string Directory rclone will use for caching.
6859 --vfs-cache-mode CacheMode Cache mode off|minimal|writes|full (default off)
6860 --vfs-cache-max-age duration Max age of objects in the cache (default 1h0m0s)
6861 --vfs-cache-max-size SizeSuffix Max total size of objects in the cache (default off)
6862 --vfs-cache-poll-interval duration Interval to poll the cache for stale objects (default 1m0s)
6863 --vfs-write-back duration Time to writeback files after last use when using cache (default 5s)
6864
6865 If run with -vv rclone will print the location of the file cache. The
6866 files are stored in the user cache file area which is OS dependent but
6867 can be controlled with --cache-dir or setting the appropriate environ‐
6868 ment variable.
6869
6870 The cache has 4 different modes selected by --vfs-cache-mode. The
6871 higher the cache mode the more compatible rclone becomes at the cost of
6872 using disk space.
6873
6874 Note that files are written back to the remote only when they are
6875 closed and if they haven't been accessed for --vfs-write-back seconds.
6876 If rclone is quit or dies with files that haven't been uploaded, these
6877 will be uploaded next time rclone is run with the same flags.
6878
6879 If using --vfs-cache-max-size note that the cache may exceed this size
6880 for two reasons. Firstly because it is only checked every --vfs-cache-
6881 poll-interval. Secondly because open files cannot be evicted from the
6882 cache.
6883
6884 You should not run two copies of rclone using the same VFS cache with
6885 the same or overlapping remotes if using --vfs-cache-mode > off. This
6886 can potentially cause data corruption if you do. You can work around
6887 this by giving each rclone its own cache hierarchy with --cache-dir.
6888 You don't need to worry about this if the remotes in use don't overlap.
6889
6890 --vfs-cache-mode off
6891 In this mode (the default) the cache will read directly from the remote
6892 and write directly to the remote without caching anything on disk.
6893
6894 This will mean some operations are not possible
6895
6896 • Files can't be opened for both read AND write
6897
6898 • Files opened for write can't be seeked
6899
6900 • Existing files opened for write must have O_TRUNC set
6901
6902 • Files open for read with O_TRUNC will be opened write only
6903
6904 • Files open for write only will behave as if O_TRUNC was supplied
6905
6906 • Open modes O_APPEND, O_TRUNC are ignored
6907
6908 • If an upload fails it can't be retried
6909
6910 --vfs-cache-mode minimal
6911 This is very similar to "off" except that files opened for read AND
6912 write will be buffered to disk. This means that files opened for write
6913 will be a lot more compatible, but uses the minimal disk space.
6914
6915 These operations are not possible
6916
6917 • Files opened for write only can't be seeked
6918
6919 • Existing files opened for write must have O_TRUNC set
6920
6921 • Files opened for write only will ignore O_APPEND, O_TRUNC
6922
6923 • If an upload fails it can't be retried
6924
6925 --vfs-cache-mode writes
6926 In this mode files opened for read only are still read directly from
6927 the remote, write only and read/write files are buffered to disk first.
6928
6929 This mode should support all normal file system operations.
6930
6931 If an upload fails it will be retried at exponentially increasing in‐
6932 tervals up to 1 minute.
6933
6934 --vfs-cache-mode full
6935 In this mode all reads and writes are buffered to and from disk. When
6936 data is read from the remote this is buffered to disk as well.
6937
6938 In this mode the files in the cache will be sparse files and rclone
6939 will keep track of which bits of the files it has downloaded.
6940
6941 So if an application only reads the starts of each file, then rclone
6942 will only buffer the start of the file. These files will appear to be
6943 their full size in the cache, but they will be sparse files with only
6944 the data that has been downloaded present in them.
6945
6946 This mode should support all normal file system operations and is oth‐
6947 erwise identical to --vfs-cache-mode writes.
6948
6949 When reading a file rclone will read --buffer-size plus --vfs-read-
6950 ahead bytes ahead. The --buffer-size is buffered in memory whereas the
6951 --vfs-read-ahead is buffered on disk.
6952
6953 When using this mode it is recommended that --buffer-size is not set
6954 too large and --vfs-read-ahead is set large if required.
6955
6956 IMPORTANT not all file systems support sparse files. In particular
6957 FAT/exFAT do not. Rclone will perform very badly if the cache directo‐
6958 ry is on a filesystem which doesn't support sparse files and it will
6959 log an ERROR message if one is detected.
6960
6961 Fingerprinting
6962 Various parts of the VFS use fingerprinting to see if a local file copy
6963 has changed relative to a remote file. Fingerprints are made from:
6964
6965 • size
6966
6967 • modification time
6968
6969 • hash
6970
6971 where available on an object.
6972
6973 On some backends some of these attributes are slow to read (they take
6974 an extra API call per object, or extra work per object).
6975
6976 For example hash is slow with the local and sftp backends as they have
6977 to read the entire file and hash it, and modtime is slow with the s3,
6978 swift, ftp and qinqstor backends because they need to do an extra API
6979 call to fetch it.
6980
6981 If you use the --vfs-fast-fingerprint flag then rclone will not include
6982 the slow operations in the fingerprint. This makes the fingerprinting
6983 less accurate but much faster and will improve the opening time of
6984 cached files.
6985
6986 If you are running a vfs cache over local, s3 or swift backends then
6987 using this flag is recommended.
6988
6989 Note that if you change the value of this flag, the fingerprints of the
6990 files in the cache may be invalidated and the files will need to be
6991 downloaded again.
6992
6993 VFS Chunked Reading
6994 When rclone reads files from a remote it reads them in chunks. This
6995 means that rather than requesting the whole file rclone reads the chunk
6996 specified. This can reduce the used download quota for some remotes by
6997 requesting only chunks from the remote that are actually read, at the
6998 cost of an increased number of requests.
6999
7000 These flags control the chunking:
7001
7002 --vfs-read-chunk-size SizeSuffix Read the source objects in chunks (default 128M)
7003 --vfs-read-chunk-size-limit SizeSuffix Max chunk doubling size (default off)
7004
7005 Rclone will start reading a chunk of size --vfs-read-chunk-size, and
7006 then double the size for each read. When --vfs-read-chunk-size-limit
7007 is specified, and greater than --vfs-read-chunk-size, the chunk size
7008 for each open file will get doubled only until the specified value is
7009 reached. If the value is "off", which is the default, the limit is
7010 disabled and the chunk size will grow indefinitely.
7011
7012 With --vfs-read-chunk-size 100M and --vfs-read-chunk-size-limit 0 the
7013 following parts will be downloaded: 0-100M, 100M-200M, 200M-300M,
7014 300M-400M and so on. When --vfs-read-chunk-size-limit 500M is speci‐
7015 fied, the result would be 0-100M, 100M-300M, 300M-700M, 700M-1200M,
7016 1200M-1700M and so on.
7017
7018 Setting --vfs-read-chunk-size to 0 or "off" disables chunked reading.
7019
7020 VFS Performance
7021 These flags may be used to enable/disable features of the VFS for per‐
7022 formance or other reasons. See also the chunked reading feature.
7023
7024 In particular S3 and Swift benefit hugely from the --no-modtime flag
7025 (or use --use-server-modtime for a slightly different effect) as each
7026 read of the modification time takes a transaction.
7027
7028 --no-checksum Don't compare checksums on up/download.
7029 --no-modtime Don't read/write the modification time (can speed things up).
7030 --no-seek Don't allow seeking in files.
7031 --read-only Only allow read-only access.
7032
7033 Sometimes rclone is delivered reads or writes out of order. Rather
7034 than seeking rclone will wait a short time for the in sequence read or
7035 write to come in. These flags only come into effect when not using an
7036 on disk cache file.
7037
7038 --vfs-read-wait duration Time to wait for in-sequence read before seeking (default 20ms)
7039 --vfs-write-wait duration Time to wait for in-sequence write before giving error (default 1s)
7040
7041 When using VFS write caching (--vfs-cache-mode with value writes or
7042 full), the global flag --transfers can be set to adjust the number of
7043 parallel uploads of modified files from the cache (the related global
7044 flag --checkers has no effect on the VFS).
7045
7046 --transfers int Number of file transfers to run in parallel (default 4)
7047
7048 VFS Case Sensitivity
7049 Linux file systems are case-sensitive: two files can differ only by
7050 case, and the exact case must be used when opening a file.
7051
7052 File systems in modern Windows are case-insensitive but case-preserv‐
7053 ing: although existing files can be opened using any case, the exact
7054 case used to create the file is preserved and available for programs to
7055 query. It is not allowed for two files in the same directory to differ
7056 only by case.
7057
7058 Usually file systems on macOS are case-insensitive. It is possible to
7059 make macOS file systems case-sensitive but that is not the default.
7060
7061 The --vfs-case-insensitive VFS flag controls how rclone handles these
7062 two cases. If its value is "false", rclone passes file names to the
7063 remote as-is. If the flag is "true" (or appears without a value on the
7064 command line), rclone may perform a "fixup" as explained below.
7065
7066 The user may specify a file name to open/delete/rename/etc with a case
7067 different than what is stored on the remote. If an argument refers to
7068 an existing file with exactly the same name, then the case of the ex‐
7069 isting file on the disk will be used. However, if a file name with ex‐
7070 actly the same name is not found but a name differing only by case ex‐
7071 ists, rclone will transparently fixup the name. This fixup happens on‐
7072 ly when an existing file is requested. Case sensitivity of file names
7073 created anew by rclone is controlled by the underlying remote.
7074
7075 Note that case sensitivity of the operating system running rclone (the
7076 target) may differ from case sensitivity of a file system presented by
7077 rclone (the source). The flag controls whether "fixup" is performed to
7078 satisfy the target.
7079
7080 If the flag is not provided on the command line, then its default value
7081 depends on the operating system where rclone runs: "true" on Windows
7082 and macOS, "false" otherwise. If the flag is provided without a value,
7083 then it is "true".
7084
7085 VFS Disk Options
7086 This flag allows you to manually set the statistics about the filing
7087 system. It can be useful when those statistics cannot be read correct‐
7088 ly automatically.
7089
7090 --vfs-disk-space-total-size Manually set the total disk space size (example: 256G, default: -1)
7091
7092 Alternate report of used bytes
7093 Some backends, most notably S3, do not report the amount of bytes used.
7094 If you need this information to be available when running df on the
7095 filesystem, then pass the flag --vfs-used-is-size to rclone. With this
7096 flag set, instead of relying on the backend to report this information,
7097 rclone will scan the whole remote similar to rclone size and compute
7098 the total used space itself.
7099
7100 WARNING. Contrary to rclone size, this flag ignores filters so that the
7101 result is accurate. However, this is very inefficient and may cost
7102 lots of API calls resulting in extra charges. Use it as a last resort
7103 and only with caching.
7104
7105 Auth Proxy
7106 If you supply the parameter --auth-proxy /path/to/program then rclone
7107 will use that program to generate backends on the fly which then are
7108 used to authenticate incoming requests. This uses a simple JSON based
7109 protocol with input on STDIN and output on STDOUT.
7110
7111 PLEASE NOTE: --auth-proxy and --authorized-keys cannot be used togeth‐
7112 er, if --auth-proxy is set the authorized keys option will be ignored.
7113
7114 There is an example program bin/test_proxy.py
7115 (https://github.com/rclone/rclone/blob/master/test_proxy.py) in the
7116 rclone source code.
7117
7118 The program's job is to take a user and pass on the input and turn
7119 those into the config for a backend on STDOUT in JSON format. This
7120 config will have any default parameters for the backend added, but it
7121 won't use configuration from environment variables or command line op‐
7122 tions - it is the job of the proxy program to make a complete config.
7123
7124 This config generated must have this extra parameter - _root - root to
7125 use for the backend
7126
7127 And it may have this parameter - _obscure - comma separated strings for
7128 parameters to obscure
7129
7130 If password authentication was used by the client, input to the proxy
7131 process (on STDIN) would look similar to this:
7132
7133 {
7134 "user": "me",
7135 "pass": "mypassword"
7136 }
7137
7138 If public-key authentication was used by the client, input to the proxy
7139 process (on STDIN) would look similar to this:
7140
7141 {
7142 "user": "me",
7143 "public_key": "AAAAB3NzaC1yc2EAAAADAQABAAABAQDuwESFdAe14hVS6omeyX7edc...JQdf"
7144 }
7145
7146 And as an example return this on STDOUT
7147
7148 {
7149 "type": "sftp",
7150 "_root": "",
7151 "_obscure": "pass",
7152 "user": "me",
7153 "pass": "mypassword",
7154 "host": "sftp.example.com"
7155 }
7156
7157 This would mean that an SFTP backend would be created on the fly for
7158 the user and pass/public_key returned in the output to the host given.
7159 Note that since _obscure is set to pass, rclone will obscure the pass
7160 parameter before creating the backend (which is required for sftp back‐
7161 ends).
7162
7163 The program can manipulate the supplied user in any way, for example to
7164 make proxy to many different sftp backends, you could make the user be
7165 user@example.com and then set the host to example.com in the output and
7166 the user to user. For security you'd probably want to restrict the
7167 host to a limited list.
7168
7169 Note that an internal cache is keyed on user so only use that for con‐
7170 figuration, don't use pass or public_key. This also means that if a
7171 user's password or public-key is changed the cache will need to expire
7172 (which takes 5 mins) before it takes effect.
7173
7174 This can be used to build general purpose proxies to any kind of back‐
7175 end that rclone supports.
7176
7177 rclone serve sftp remote:path [flags]
7178
7179 Options
7180 --addr string IPaddress:Port or :Port to bind server to (default "localhost:2022")
7181 --auth-proxy string A program to use to create the backend from the auth
7182 --authorized-keys string Authorized keys file (default "~/.ssh/authorized_keys")
7183 --dir-cache-time duration Time to cache directory entries for (default 5m0s)
7184 --dir-perms FileMode Directory permissions (default 0777)
7185 --file-perms FileMode File permissions (default 0666)
7186 --gid uint32 Override the gid field set by the filesystem (not supported on Windows) (default 1000)
7187 -h, --help help for sftp
7188 --key stringArray SSH private host key file (Can be multi-valued, leave blank to auto generate)
7189 --no-auth Allow connections with no authentication if set
7190 --no-checksum Don't compare checksums on up/download
7191 --no-modtime Don't read/write the modification time (can speed things up)
7192 --no-seek Don't allow seeking in files
7193 --pass string Password for authentication
7194 --poll-interval duration Time to wait between polling for changes, must be smaller than dir-cache-time and only on supported remotes (set 0 to disable) (default 1m0s)
7195 --read-only Only allow read-only access
7196 --stdio Run an sftp server on stdin/stdout
7197 --uid uint32 Override the uid field set by the filesystem (not supported on Windows) (default 1000)
7198 --umask int Override the permission bits set by the filesystem (not supported on Windows) (default 2)
7199 --user string User name for authentication
7200 --vfs-cache-max-age duration Max age of objects in the cache (default 1h0m0s)
7201 --vfs-cache-max-size SizeSuffix Max total size of objects in the cache (default off)
7202 --vfs-cache-mode CacheMode Cache mode off|minimal|writes|full (default off)
7203 --vfs-cache-poll-interval duration Interval to poll the cache for stale objects (default 1m0s)
7204 --vfs-case-insensitive If a file name not found, find a case insensitive match
7205 --vfs-disk-space-total-size SizeSuffix Specify the total space of disk (default off)
7206 --vfs-fast-fingerprint Use fast (less accurate) fingerprints for change detection
7207 --vfs-read-ahead SizeSuffix Extra read ahead over --buffer-size when using cache-mode full
7208 --vfs-read-chunk-size SizeSuffix Read the source objects in chunks (default 128Mi)
7209 --vfs-read-chunk-size-limit SizeSuffix If greater than --vfs-read-chunk-size, double the chunk size after each chunk read, until the limit is reached ('off' is unlimited) (default off)
7210 --vfs-read-wait duration Time to wait for in-sequence read before seeking (default 20ms)
7211 --vfs-used-is-size rclone size Use the rclone size algorithm for Used size
7212 --vfs-write-back duration Time to writeback files after last use when using cache (default 5s)
7213 --vfs-write-wait duration Time to wait for in-sequence write before giving error (default 1s)
7214
7215 See the global flags page (https://rclone.org/flags/) for global op‐
7216 tions not listed here.
7217
7218 SEE ALSO
7219 • rclone serve (https://rclone.org/commands/rclone_serve/) - Serve a
7220 remote over a protocol.
7221
7223 Serve remote:path over WebDAV.
7224
7225 Synopsis
7226 Run a basic WebDAV server to serve a remote over HTTP via the WebDAV
7227 protocol. This can be viewed with a WebDAV client, through a web
7228 browser, or you can make a remote of type WebDAV to read and write it.
7229
7230 WebDAV options
7231 --etag-hash
7232 This controls the ETag header. Without this flag the ETag will be
7233 based on the ModTime and Size of the object.
7234
7235 If this flag is set to "auto" then rclone will choose the first sup‐
7236 ported hash on the backend or you can use a named hash such as "MD5" or
7237 "SHA-1". Use the hashsum (https://rclone.org/commands/rclone_hashsum/)
7238 command to see the full list.
7239
7240 Server options
7241 Use --addr to specify which IP address and port the server should lis‐
7242 ten on, e.g. --addr 1.2.3.4:8000 or --addr :8080 to listen to all IPs.
7243 By default it only listens on localhost. You can use port :0 to let
7244 the OS choose an available port.
7245
7246 If you set --addr to listen on a public or LAN accessible IP address
7247 then using Authentication is advised - see the next section for info.
7248
7249 --server-read-timeout and --server-write-timeout can be used to control
7250 the timeouts on the server. Note that this is the total time for a
7251 transfer.
7252
7253 --max-header-bytes controls the maximum number of bytes the server will
7254 accept in the HTTP header.
7255
7256 --baseurl controls the URL prefix that rclone serves from. By default
7257 rclone will serve from the root. If you used --baseurl "/rclone" then
7258 rclone would serve from a URL starting with "/rclone/". This is useful
7259 if you wish to proxy rclone serve. Rclone automatically inserts lead‐
7260 ing and trailing "/" on --baseurl, so --baseurl "rclone", --baseurl
7261 "/rclone" and --baseurl "/rclone/" are all treated identically.
7262
7263 --template allows a user to specify a custom markup template for HTTP
7264 and WebDAV serve functions. The server exports the following markup to
7265 be used within the template to server pages:
7266
7267 Parameter Description
7268 ──────────────────────────────────────────────────────────────────────────
7269 .Name The full path of a file/directory.
7270 .Title Directory listing of .Name
7271 .Sort The current sort used. This is
7272 changeable via ?sort= parameter
7273 Sort Options:
7274 namedirfirst,name,size,time (de‐
7275 fault namedirfirst)
7276 .Order The current ordering used. This is
7277 changeable via ?order= parameter
7278 Order Options: asc,desc (default
7279 asc)
7280 .Query Currently unused.
7281 .Breadcrumb Allows for creating a relative nav‐
7282 igation
7283 -- .Link The relative to the root link of
7284 the Text.
7285 -- .Text The Name of the directory.
7286 .Entries Information about a specific
7287 file/directory.
7288 -- .URL The 'url' of an entry.
7289 -- .Leaf Currently same as 'URL' but intend‐
7290 ed to be 'just' the name.
7291 -- .IsDir Boolean for if an entry is a direc‐
7292 tory or not.
7293 -- .Size Size in Bytes of the entry.
7294 -- .ModTime The UTC timestamp of an entry.
7295
7296 Authentication
7297 By default this will serve files without needing a login.
7298
7299 You can either use an htpasswd file which can take lots of users, or
7300 set a single username and password with the --user and --pass flags.
7301
7302 Use --htpasswd /path/to/htpasswd to provide an htpasswd file. This is
7303 in standard apache format and supports MD5, SHA1 and BCrypt for basic
7304 authentication. Bcrypt is recommended.
7305
7306 To create an htpasswd file:
7307
7308 touch htpasswd
7309 htpasswd -B htpasswd user
7310 htpasswd -B htpasswd anotherUser
7311
7312 The password file can be updated while rclone is running.
7313
7314 Use --realm to set the authentication realm.
7315
7316 SSL/TLS
7317 By default this will serve over HTTP. If you want you can serve over
7318 HTTPS. You will need to supply the --cert and --key flags. If you
7319 wish to do client side certificate validation then you will need to
7320 supply --client-ca also.
7321
7322 --cert should be either a PEM encoded certificate or a concatenation of
7323 that with the CA certificate. --key should be the PEM encoded private
7324 key and --client-ca should be the PEM encoded client certificate au‐
7325 thority certificate.
7326
7327 --min-tls-version is minimum TLS version that is acceptable. Valid
7328 values are "tls1.0", "tls1.1", "tls1.2" and "tls1.3" (default
7329 "tls1.0").
7330
7331 VFS - Virtual File System
7332 This command uses the VFS layer. This adapts the cloud storage objects
7333 that rclone uses into something which looks much more like a disk fil‐
7334 ing system.
7335
7336 Cloud storage objects have lots of properties which aren't like disk
7337 files - you can't extend them or write to the middle of them, so the
7338 VFS layer has to deal with that. Because there is no one right way of
7339 doing this there are various options explained below.
7340
7341 The VFS layer also implements a directory cache - this caches info
7342 about files and directories (but not the data) in memory.
7343
7344 VFS Directory Cache
7345 Using the --dir-cache-time flag, you can control how long a directory
7346 should be considered up to date and not refreshed from the backend.
7347 Changes made through the VFS will appear immediately or invalidate the
7348 cache.
7349
7350 --dir-cache-time duration Time to cache directory entries for (default 5m0s)
7351 --poll-interval duration Time to wait between polling for changes. Must be smaller than dir-cache-time. Only on supported remotes. Set to 0 to disable (default 1m0s)
7352
7353 However, changes made directly on the cloud storage by the web inter‐
7354 face or a different copy of rclone will only be picked up once the di‐
7355 rectory cache expires if the backend configured does not support
7356 polling for changes. If the backend supports polling, changes will be
7357 picked up within the polling interval.
7358
7359 You can send a SIGHUP signal to rclone for it to flush all directory
7360 caches, regardless of how old they are. Assuming only one rclone in‐
7361 stance is running, you can reset the cache like this:
7362
7363 kill -SIGHUP $(pidof rclone)
7364
7365 If you configure rclone with a remote control then you can use rclone
7366 rc to flush the whole directory cache:
7367
7368 rclone rc vfs/forget
7369
7370 Or individual files or directories:
7371
7372 rclone rc vfs/forget file=path/to/file dir=path/to/dir
7373
7374 VFS File Buffering
7375 The --buffer-size flag determines the amount of memory, that will be
7376 used to buffer data in advance.
7377
7378 Each open file will try to keep the specified amount of data in memory
7379 at all times. The buffered data is bound to one open file and won't be
7380 shared.
7381
7382 This flag is a upper limit for the used memory per open file. The buf‐
7383 fer will only use memory for data that is downloaded but not not yet
7384 read. If the buffer is empty, only a small amount of memory will be
7385 used.
7386
7387 The maximum memory used by rclone for buffering can be up to --buffer-
7388 size * open files.
7389
7390 VFS File Caching
7391 These flags control the VFS file caching options. File caching is nec‐
7392 essary to make the VFS layer appear compatible with a normal file sys‐
7393 tem. It can be disabled at the cost of some compatibility.
7394
7395 For example you'll need to enable VFS caching if you want to read and
7396 write simultaneously to a file. See below for more details.
7397
7398 Note that the VFS cache is separate from the cache backend and you may
7399 find that you need one or the other or both.
7400
7401 --cache-dir string Directory rclone will use for caching.
7402 --vfs-cache-mode CacheMode Cache mode off|minimal|writes|full (default off)
7403 --vfs-cache-max-age duration Max age of objects in the cache (default 1h0m0s)
7404 --vfs-cache-max-size SizeSuffix Max total size of objects in the cache (default off)
7405 --vfs-cache-poll-interval duration Interval to poll the cache for stale objects (default 1m0s)
7406 --vfs-write-back duration Time to writeback files after last use when using cache (default 5s)
7407
7408 If run with -vv rclone will print the location of the file cache. The
7409 files are stored in the user cache file area which is OS dependent but
7410 can be controlled with --cache-dir or setting the appropriate environ‐
7411 ment variable.
7412
7413 The cache has 4 different modes selected by --vfs-cache-mode. The
7414 higher the cache mode the more compatible rclone becomes at the cost of
7415 using disk space.
7416
7417 Note that files are written back to the remote only when they are
7418 closed and if they haven't been accessed for --vfs-write-back seconds.
7419 If rclone is quit or dies with files that haven't been uploaded, these
7420 will be uploaded next time rclone is run with the same flags.
7421
7422 If using --vfs-cache-max-size note that the cache may exceed this size
7423 for two reasons. Firstly because it is only checked every --vfs-cache-
7424 poll-interval. Secondly because open files cannot be evicted from the
7425 cache.
7426
7427 You should not run two copies of rclone using the same VFS cache with
7428 the same or overlapping remotes if using --vfs-cache-mode > off. This
7429 can potentially cause data corruption if you do. You can work around
7430 this by giving each rclone its own cache hierarchy with --cache-dir.
7431 You don't need to worry about this if the remotes in use don't overlap.
7432
7433 --vfs-cache-mode off
7434 In this mode (the default) the cache will read directly from the remote
7435 and write directly to the remote without caching anything on disk.
7436
7437 This will mean some operations are not possible
7438
7439 • Files can't be opened for both read AND write
7440
7441 • Files opened for write can't be seeked
7442
7443 • Existing files opened for write must have O_TRUNC set
7444
7445 • Files open for read with O_TRUNC will be opened write only
7446
7447 • Files open for write only will behave as if O_TRUNC was supplied
7448
7449 • Open modes O_APPEND, O_TRUNC are ignored
7450
7451 • If an upload fails it can't be retried
7452
7453 --vfs-cache-mode minimal
7454 This is very similar to "off" except that files opened for read AND
7455 write will be buffered to disk. This means that files opened for write
7456 will be a lot more compatible, but uses the minimal disk space.
7457
7458 These operations are not possible
7459
7460 • Files opened for write only can't be seeked
7461
7462 • Existing files opened for write must have O_TRUNC set
7463
7464 • Files opened for write only will ignore O_APPEND, O_TRUNC
7465
7466 • If an upload fails it can't be retried
7467
7468 --vfs-cache-mode writes
7469 In this mode files opened for read only are still read directly from
7470 the remote, write only and read/write files are buffered to disk first.
7471
7472 This mode should support all normal file system operations.
7473
7474 If an upload fails it will be retried at exponentially increasing in‐
7475 tervals up to 1 minute.
7476
7477 --vfs-cache-mode full
7478 In this mode all reads and writes are buffered to and from disk. When
7479 data is read from the remote this is buffered to disk as well.
7480
7481 In this mode the files in the cache will be sparse files and rclone
7482 will keep track of which bits of the files it has downloaded.
7483
7484 So if an application only reads the starts of each file, then rclone
7485 will only buffer the start of the file. These files will appear to be
7486 their full size in the cache, but they will be sparse files with only
7487 the data that has been downloaded present in them.
7488
7489 This mode should support all normal file system operations and is oth‐
7490 erwise identical to --vfs-cache-mode writes.
7491
7492 When reading a file rclone will read --buffer-size plus --vfs-read-
7493 ahead bytes ahead. The --buffer-size is buffered in memory whereas the
7494 --vfs-read-ahead is buffered on disk.
7495
7496 When using this mode it is recommended that --buffer-size is not set
7497 too large and --vfs-read-ahead is set large if required.
7498
7499 IMPORTANT not all file systems support sparse files. In particular
7500 FAT/exFAT do not. Rclone will perform very badly if the cache directo‐
7501 ry is on a filesystem which doesn't support sparse files and it will
7502 log an ERROR message if one is detected.
7503
7504 Fingerprinting
7505 Various parts of the VFS use fingerprinting to see if a local file copy
7506 has changed relative to a remote file. Fingerprints are made from:
7507
7508 • size
7509
7510 • modification time
7511
7512 • hash
7513
7514 where available on an object.
7515
7516 On some backends some of these attributes are slow to read (they take
7517 an extra API call per object, or extra work per object).
7518
7519 For example hash is slow with the local and sftp backends as they have
7520 to read the entire file and hash it, and modtime is slow with the s3,
7521 swift, ftp and qinqstor backends because they need to do an extra API
7522 call to fetch it.
7523
7524 If you use the --vfs-fast-fingerprint flag then rclone will not include
7525 the slow operations in the fingerprint. This makes the fingerprinting
7526 less accurate but much faster and will improve the opening time of
7527 cached files.
7528
7529 If you are running a vfs cache over local, s3 or swift backends then
7530 using this flag is recommended.
7531
7532 Note that if you change the value of this flag, the fingerprints of the
7533 files in the cache may be invalidated and the files will need to be
7534 downloaded again.
7535
7536 VFS Chunked Reading
7537 When rclone reads files from a remote it reads them in chunks. This
7538 means that rather than requesting the whole file rclone reads the chunk
7539 specified. This can reduce the used download quota for some remotes by
7540 requesting only chunks from the remote that are actually read, at the
7541 cost of an increased number of requests.
7542
7543 These flags control the chunking:
7544
7545 --vfs-read-chunk-size SizeSuffix Read the source objects in chunks (default 128M)
7546 --vfs-read-chunk-size-limit SizeSuffix Max chunk doubling size (default off)
7547
7548 Rclone will start reading a chunk of size --vfs-read-chunk-size, and
7549 then double the size for each read. When --vfs-read-chunk-size-limit
7550 is specified, and greater than --vfs-read-chunk-size, the chunk size
7551 for each open file will get doubled only until the specified value is
7552 reached. If the value is "off", which is the default, the limit is
7553 disabled and the chunk size will grow indefinitely.
7554
7555 With --vfs-read-chunk-size 100M and --vfs-read-chunk-size-limit 0 the
7556 following parts will be downloaded: 0-100M, 100M-200M, 200M-300M,
7557 300M-400M and so on. When --vfs-read-chunk-size-limit 500M is speci‐
7558 fied, the result would be 0-100M, 100M-300M, 300M-700M, 700M-1200M,
7559 1200M-1700M and so on.
7560
7561 Setting --vfs-read-chunk-size to 0 or "off" disables chunked reading.
7562
7563 VFS Performance
7564 These flags may be used to enable/disable features of the VFS for per‐
7565 formance or other reasons. See also the chunked reading feature.
7566
7567 In particular S3 and Swift benefit hugely from the --no-modtime flag
7568 (or use --use-server-modtime for a slightly different effect) as each
7569 read of the modification time takes a transaction.
7570
7571 --no-checksum Don't compare checksums on up/download.
7572 --no-modtime Don't read/write the modification time (can speed things up).
7573 --no-seek Don't allow seeking in files.
7574 --read-only Only allow read-only access.
7575
7576 Sometimes rclone is delivered reads or writes out of order. Rather
7577 than seeking rclone will wait a short time for the in sequence read or
7578 write to come in. These flags only come into effect when not using an
7579 on disk cache file.
7580
7581 --vfs-read-wait duration Time to wait for in-sequence read before seeking (default 20ms)
7582 --vfs-write-wait duration Time to wait for in-sequence write before giving error (default 1s)
7583
7584 When using VFS write caching (--vfs-cache-mode with value writes or
7585 full), the global flag --transfers can be set to adjust the number of
7586 parallel uploads of modified files from the cache (the related global
7587 flag --checkers has no effect on the VFS).
7588
7589 --transfers int Number of file transfers to run in parallel (default 4)
7590
7591 VFS Case Sensitivity
7592 Linux file systems are case-sensitive: two files can differ only by
7593 case, and the exact case must be used when opening a file.
7594
7595 File systems in modern Windows are case-insensitive but case-preserv‐
7596 ing: although existing files can be opened using any case, the exact
7597 case used to create the file is preserved and available for programs to
7598 query. It is not allowed for two files in the same directory to differ
7599 only by case.
7600
7601 Usually file systems on macOS are case-insensitive. It is possible to
7602 make macOS file systems case-sensitive but that is not the default.
7603
7604 The --vfs-case-insensitive VFS flag controls how rclone handles these
7605 two cases. If its value is "false", rclone passes file names to the
7606 remote as-is. If the flag is "true" (or appears without a value on the
7607 command line), rclone may perform a "fixup" as explained below.
7608
7609 The user may specify a file name to open/delete/rename/etc with a case
7610 different than what is stored on the remote. If an argument refers to
7611 an existing file with exactly the same name, then the case of the ex‐
7612 isting file on the disk will be used. However, if a file name with ex‐
7613 actly the same name is not found but a name differing only by case ex‐
7614 ists, rclone will transparently fixup the name. This fixup happens on‐
7615 ly when an existing file is requested. Case sensitivity of file names
7616 created anew by rclone is controlled by the underlying remote.
7617
7618 Note that case sensitivity of the operating system running rclone (the
7619 target) may differ from case sensitivity of a file system presented by
7620 rclone (the source). The flag controls whether "fixup" is performed to
7621 satisfy the target.
7622
7623 If the flag is not provided on the command line, then its default value
7624 depends on the operating system where rclone runs: "true" on Windows
7625 and macOS, "false" otherwise. If the flag is provided without a value,
7626 then it is "true".
7627
7628 VFS Disk Options
7629 This flag allows you to manually set the statistics about the filing
7630 system. It can be useful when those statistics cannot be read correct‐
7631 ly automatically.
7632
7633 --vfs-disk-space-total-size Manually set the total disk space size (example: 256G, default: -1)
7634
7635 Alternate report of used bytes
7636 Some backends, most notably S3, do not report the amount of bytes used.
7637 If you need this information to be available when running df on the
7638 filesystem, then pass the flag --vfs-used-is-size to rclone. With this
7639 flag set, instead of relying on the backend to report this information,
7640 rclone will scan the whole remote similar to rclone size and compute
7641 the total used space itself.
7642
7643 WARNING. Contrary to rclone size, this flag ignores filters so that the
7644 result is accurate. However, this is very inefficient and may cost
7645 lots of API calls resulting in extra charges. Use it as a last resort
7646 and only with caching.
7647
7648 Auth Proxy
7649 If you supply the parameter --auth-proxy /path/to/program then rclone
7650 will use that program to generate backends on the fly which then are
7651 used to authenticate incoming requests. This uses a simple JSON based
7652 protocol with input on STDIN and output on STDOUT.
7653
7654 PLEASE NOTE: --auth-proxy and --authorized-keys cannot be used togeth‐
7655 er, if --auth-proxy is set the authorized keys option will be ignored.
7656
7657 There is an example program bin/test_proxy.py
7658 (https://github.com/rclone/rclone/blob/master/test_proxy.py) in the
7659 rclone source code.
7660
7661 The program's job is to take a user and pass on the input and turn
7662 those into the config for a backend on STDOUT in JSON format. This
7663 config will have any default parameters for the backend added, but it
7664 won't use configuration from environment variables or command line op‐
7665 tions - it is the job of the proxy program to make a complete config.
7666
7667 This config generated must have this extra parameter - _root - root to
7668 use for the backend
7669
7670 And it may have this parameter - _obscure - comma separated strings for
7671 parameters to obscure
7672
7673 If password authentication was used by the client, input to the proxy
7674 process (on STDIN) would look similar to this:
7675
7676 {
7677 "user": "me",
7678 "pass": "mypassword"
7679 }
7680
7681 If public-key authentication was used by the client, input to the proxy
7682 process (on STDIN) would look similar to this:
7683
7684 {
7685 "user": "me",
7686 "public_key": "AAAAB3NzaC1yc2EAAAADAQABAAABAQDuwESFdAe14hVS6omeyX7edc...JQdf"
7687 }
7688
7689 And as an example return this on STDOUT
7690
7691 {
7692 "type": "sftp",
7693 "_root": "",
7694 "_obscure": "pass",
7695 "user": "me",
7696 "pass": "mypassword",
7697 "host": "sftp.example.com"
7698 }
7699
7700 This would mean that an SFTP backend would be created on the fly for
7701 the user and pass/public_key returned in the output to the host given.
7702 Note that since _obscure is set to pass, rclone will obscure the pass
7703 parameter before creating the backend (which is required for sftp back‐
7704 ends).
7705
7706 The program can manipulate the supplied user in any way, for example to
7707 make proxy to many different sftp backends, you could make the user be
7708 user@example.com and then set the host to example.com in the output and
7709 the user to user. For security you'd probably want to restrict the
7710 host to a limited list.
7711
7712 Note that an internal cache is keyed on user so only use that for con‐
7713 figuration, don't use pass or public_key. This also means that if a
7714 user's password or public-key is changed the cache will need to expire
7715 (which takes 5 mins) before it takes effect.
7716
7717 This can be used to build general purpose proxies to any kind of back‐
7718 end that rclone supports.
7719
7720 rclone serve webdav remote:path [flags]
7721
7722 Options
7723 --addr string IPaddress:Port or :Port to bind server to (default "localhost:8080")
7724 --auth-proxy string A program to use to create the backend from the auth
7725 --baseurl string Prefix for URLs - leave blank for root
7726 --cert string SSL PEM key (concatenation of certificate and CA certificate)
7727 --client-ca string Client certificate authority to verify clients with
7728 --dir-cache-time duration Time to cache directory entries for (default 5m0s)
7729 --dir-perms FileMode Directory permissions (default 0777)
7730 --disable-dir-list Disable HTML directory list on GET request for a directory
7731 --etag-hash string Which hash to use for the ETag, or auto or blank for off
7732 --file-perms FileMode File permissions (default 0666)
7733 --gid uint32 Override the gid field set by the filesystem (not supported on Windows) (default 1000)
7734 -h, --help help for webdav
7735 --htpasswd string htpasswd file - if not provided no authentication is done
7736 --key string SSL PEM Private key
7737 --max-header-bytes int Maximum size of request header (default 4096)
7738 --min-tls-version string Minimum TLS version that is acceptable (default "tls1.0")
7739 --no-checksum Don't compare checksums on up/download
7740 --no-modtime Don't read/write the modification time (can speed things up)
7741 --no-seek Don't allow seeking in files
7742 --pass string Password for authentication
7743 --poll-interval duration Time to wait between polling for changes, must be smaller than dir-cache-time and only on supported remotes (set 0 to disable) (default 1m0s)
7744 --read-only Only allow read-only access
7745 --realm string Realm for authentication (default "rclone")
7746 --server-read-timeout duration Timeout for server reading data (default 1h0m0s)
7747 --server-write-timeout duration Timeout for server writing data (default 1h0m0s)
7748 --template string User-specified template
7749 --uid uint32 Override the uid field set by the filesystem (not supported on Windows) (default 1000)
7750 --umask int Override the permission bits set by the filesystem (not supported on Windows) (default 2)
7751 --user string User name for authentication
7752 --vfs-cache-max-age duration Max age of objects in the cache (default 1h0m0s)
7753 --vfs-cache-max-size SizeSuffix Max total size of objects in the cache (default off)
7754 --vfs-cache-mode CacheMode Cache mode off|minimal|writes|full (default off)
7755 --vfs-cache-poll-interval duration Interval to poll the cache for stale objects (default 1m0s)
7756 --vfs-case-insensitive If a file name not found, find a case insensitive match
7757 --vfs-disk-space-total-size SizeSuffix Specify the total space of disk (default off)
7758 --vfs-fast-fingerprint Use fast (less accurate) fingerprints for change detection
7759 --vfs-read-ahead SizeSuffix Extra read ahead over --buffer-size when using cache-mode full
7760 --vfs-read-chunk-size SizeSuffix Read the source objects in chunks (default 128Mi)
7761 --vfs-read-chunk-size-limit SizeSuffix If greater than --vfs-read-chunk-size, double the chunk size after each chunk read, until the limit is reached ('off' is unlimited) (default off)
7762 --vfs-read-wait duration Time to wait for in-sequence read before seeking (default 20ms)
7763 --vfs-used-is-size rclone size Use the rclone size algorithm for Used size
7764 --vfs-write-back duration Time to writeback files after last use when using cache (default 5s)
7765 --vfs-write-wait duration Time to wait for in-sequence write before giving error (default 1s)
7766
7767 See the global flags page (https://rclone.org/flags/) for global op‐
7768 tions not listed here.
7769
7770 SEE ALSO
7771 • rclone serve (https://rclone.org/commands/rclone_serve/) - Serve a
7772 remote over a protocol.
7773
7775 Changes storage class/tier of objects in remote.
7776
7777 Synopsis
7778 rclone settier changes storage tier or class at remote if supported.
7779 Few cloud storage services provides different storage classes on ob‐
7780 jects, for example AWS S3 and Glacier, Azure Blob storage - Hot, Cool
7781 and Archive, Google Cloud Storage, Regional Storage, Nearline, Coldline
7782 etc.
7783
7784 Note that, certain tier changes make objects not available to access
7785 immediately. For example tiering to archive in azure blob storage
7786 makes objects in frozen state, user can restore by setting tier to
7787 Hot/Cool, similarly S3 to Glacier makes object inaccessible.true
7788
7789 You can use it to tier single object
7790
7791 rclone settier Cool remote:path/file
7792
7793 Or use rclone filters to set tier on only specific files
7794
7795 rclone --include "*.txt" settier Hot remote:path/dir
7796
7797 Or just provide remote directory and all files in directory will be
7798 tiered
7799
7800 rclone settier tier remote:path/dir
7801
7802 rclone settier tier remote:path [flags]
7803
7804 Options
7805 -h, --help help for settier
7806
7807 See the global flags page (https://rclone.org/flags/) for global op‐
7808 tions not listed here.
7809
7810 SEE ALSO
7811 • rclone (https://rclone.org/commands/rclone/) - Show help for rclone
7812 commands, flags and backends.
7813
7815 Run a test command
7816
7817 Synopsis
7818 Rclone test is used to run test commands.
7819
7820 Select which test comand you want with the subcommand, eg
7821
7822 rclone test memory remote:
7823
7824 Each subcommand has its own options which you can see in their help.
7825
7826 NB Be careful running these commands, they may do strange things so
7827 reading their documentation first is recommended.
7828
7829 Options
7830 -h, --help help for test
7831
7832 See the global flags page (https://rclone.org/flags/) for global op‐
7833 tions not listed here.
7834
7835 SEE ALSO
7836 • rclone (https://rclone.org/commands/rclone/) - Show help for rclone
7837 commands, flags and backends.
7838
7839 • rclone test changenotify (https://rclone.org/com‐
7840 mands/rclone_test_changenotify/) - Log any change notify requests for
7841 the remote passed in.
7842
7843 • rclone test histogram (https://rclone.org/commands/rclone_test_his‐
7844 togram/) - Makes a histogram of file name characters.
7845
7846 • rclone test info (https://rclone.org/commands/rclone_test_info/) -
7847 Discovers file name or other limitations for paths.
7848
7849 • rclone test makefile (https://rclone.org/commands/rclone_test_make‐
7850 file/) - Make files with random contents of the size given
7851
7852 • rclone test makefiles (https://rclone.org/commands/rclone_test_make‐
7853 files/) - Make a random file hierarchy in a directory
7854
7855 • rclone test memory (https://rclone.org/commands/rclone_test_memory/)
7856 - Load all the objects at remote:path into memory and report memory
7857 stats.
7858
7860 Log any change notify requests for the remote passed in.
7861
7862 rclone test changenotify remote: [flags]
7863
7864 Options
7865 -h, --help help for changenotify
7866 --poll-interval duration Time to wait between polling for changes (default 10s)
7867
7868 See the global flags page (https://rclone.org/flags/) for global op‐
7869 tions not listed here.
7870
7871 SEE ALSO
7872 • rclone test (https://rclone.org/commands/rclone_test/) - Run a test
7873 command
7874
7876 Makes a histogram of file name characters.
7877
7878 Synopsis
7879 This command outputs JSON which shows the histogram of characters used
7880 in filenames in the remote:path specified.
7881
7882 The data doesn't contain any identifying information but is useful for
7883 the rclone developers when developing filename compression.
7884
7885 rclone test histogram [remote:path] [flags]
7886
7887 Options
7888 -h, --help help for histogram
7889
7890 See the global flags page (https://rclone.org/flags/) for global op‐
7891 tions not listed here.
7892
7893 SEE ALSO
7894 • rclone test (https://rclone.org/commands/rclone_test/) - Run a test
7895 command
7896
7898 Discovers file name or other limitations for paths.
7899
7900 Synopsis
7901 rclone info discovers what filenames and upload methods are possible to
7902 write to the paths passed in and how long they can be. It can take
7903 some time. It will write test files into the remote:path passed in.
7904 It outputs a bit of go code for each one.
7905
7906 NB this can create undeletable files and other hazards - use with care
7907
7908 rclone test info [remote:path]+ [flags]
7909
7910 Options
7911 --all Run all tests
7912 --check-control Check control characters
7913 --check-length Check max filename length
7914 --check-normalization Check UTF-8 Normalization
7915 --check-streaming Check uploads with indeterminate file size
7916 -h, --help help for info
7917 --upload-wait duration Wait after writing a file
7918 --write-json string Write results to file
7919
7920 See the global flags page (https://rclone.org/flags/) for global op‐
7921 tions not listed here.
7922
7923 SEE ALSO
7924 • rclone test (https://rclone.org/commands/rclone_test/) - Run a test
7925 command
7926
7928 Make files with random contents of the size given
7929
7930 rclone test makefile <size> [<file>]+ [flags]
7931
7932 Options
7933 --ascii Fill files with random ASCII printable bytes only
7934 --chargen Fill files with a ASCII chargen pattern
7935 -h, --help help for makefile
7936 --pattern Fill files with a periodic pattern
7937 --seed int Seed for the random number generator (0 for random) (default 1)
7938 --sparse Make the files sparse (appear to be filled with ASCII 0x00)
7939 --zero Fill files with ASCII 0x00
7940
7941 See the global flags page (https://rclone.org/flags/) for global op‐
7942 tions not listed here.
7943
7944 SEE ALSO
7945 • rclone test (https://rclone.org/commands/rclone_test/) - Run a test
7946 command
7947
7949 Make a random file hierarchy in a directory
7950
7951 rclone test makefiles <dir> [flags]
7952
7953 Options
7954 --ascii Fill files with random ASCII printable bytes only
7955 --chargen Fill files with a ASCII chargen pattern
7956 --files int Number of files to create (default 1000)
7957 --files-per-directory int Average number of files per directory (default 10)
7958 -h, --help help for makefiles
7959 --max-file-size SizeSuffix Maximum size of files to create (default 100)
7960 --max-name-length int Maximum size of file names (default 12)
7961 --min-file-size SizeSuffix Minimum size of file to create
7962 --min-name-length int Minimum size of file names (default 4)
7963 --pattern Fill files with a periodic pattern
7964 --seed int Seed for the random number generator (0 for random) (default 1)
7965 --sparse Make the files sparse (appear to be filled with ASCII 0x00)
7966 --zero Fill files with ASCII 0x00
7967
7968 See the global flags page (https://rclone.org/flags/) for global op‐
7969 tions not listed here.
7970
7971 SEE ALSO
7972 • rclone test (https://rclone.org/commands/rclone_test/) - Run a test
7973 command
7974
7976 Load all the objects at remote:path into memory and report memory
7977 stats.
7978
7979 rclone test memory remote:path [flags]
7980
7981 Options
7982 -h, --help help for memory
7983
7984 See the global flags page (https://rclone.org/flags/) for global op‐
7985 tions not listed here.
7986
7987 SEE ALSO
7988 • rclone test (https://rclone.org/commands/rclone_test/) - Run a test
7989 command
7990
7992 Create new file or change file modification time.
7993
7994 Synopsis
7995 Set the modification time on file(s) as specified by remote:path to
7996 have the current time.
7997
7998 If remote:path does not exist then a zero sized file will be created,
7999 unless --no-create or --recursive is provided.
8000
8001 If --recursive is used then recursively sets the modification time on
8002 all existing files that is found under the path. Filters are support‐
8003 ed, and you can test with the --dry-run or the --interactive flag.
8004
8005 If --timestamp is used then sets the modification time to that time in‐
8006 stead of the current time. Times may be specified as one of:
8007
8008 • 'YYMMDD' - e.g. 17.10.30
8009
8010 • 'YYYY-MM-DDTHH:MM:SS' - e.g. 2006-01-02T15:04:05
8011
8012 • 'YYYY-MM-DDTHH:MM:SS.SSS' - e.g. 2006-01-02T15:04:05.123456789
8013
8014 Note that value of --timestamp is in UTC. If you want local time then
8015 add the --localtime flag.
8016
8017 rclone touch remote:path [flags]
8018
8019 Options
8020 -h, --help help for touch
8021 --localtime Use localtime for timestamp, not UTC
8022 -C, --no-create Do not create the file if it does not exist (implied with --recursive)
8023 -R, --recursive Recursively touch all files
8024 -t, --timestamp string Use specified time instead of the current time of day
8025
8026 See the global flags page (https://rclone.org/flags/) for global op‐
8027 tions not listed here.
8028
8029 SEE ALSO
8030 • rclone (https://rclone.org/commands/rclone/) - Show help for rclone
8031 commands, flags and backends.
8032
8034 List the contents of the remote in a tree like fashion.
8035
8036 Synopsis
8037 rclone tree lists the contents of a remote in a similar way to the unix
8038 tree command.
8039
8040 For example
8041
8042 $ rclone tree remote:path
8043 /
8044 ├── file1
8045 ├── file2
8046 ├── file3
8047 └── subdir
8048 ├── file4
8049 └── file5
8050
8051 1 directories, 5 files
8052
8053 You can use any of the filtering options with the tree command (e.g.
8054 --include and --exclude. You can also use --fast-list.
8055
8056 The tree command has many options for controlling the listing which are
8057 compatible with the tree command, for example you can include file
8058 sizes with --size. Note that not all of them have short options as
8059 they conflict with rclone's short options.
8060
8061 For a more interactive navigation of the remote see the ncdu
8062 (https://rclone.org/commands/rclone_ncdu/) command.
8063
8064 rclone tree remote:path [flags]
8065
8066 Options
8067 -a, --all All files are listed (list . files too)
8068 -C, --color Turn colorization on always
8069 -d, --dirs-only List directories only
8070 --dirsfirst List directories before files (-U disables)
8071 --full-path Print the full path prefix for each file
8072 -h, --help help for tree
8073 --level int Descend only level directories deep
8074 -D, --modtime Print the date of last modification.
8075 --noindent Don't print indentation lines
8076 --noreport Turn off file/directory count at end of tree listing
8077 -o, --output string Output to file instead of stdout
8078 -p, --protections Print the protections for each file.
8079 -Q, --quote Quote filenames with double quotes.
8080 -s, --size Print the size in bytes of each file.
8081 --sort string Select sort: name,version,size,mtime,ctime
8082 --sort-ctime Sort files by last status change time
8083 -t, --sort-modtime Sort files by last modification time
8084 -r, --sort-reverse Reverse the order of the sort
8085 -U, --unsorted Leave files unsorted
8086 --version Sort files alphanumerically by version
8087
8088 See the global flags page (https://rclone.org/flags/) for global op‐
8089 tions not listed here.
8090
8091 SEE ALSO
8092 • rclone (https://rclone.org/commands/rclone/) - Show help for rclone
8093 commands, flags and backends.
8094
8095 Copying single files
8096 rclone normally syncs or copies directories. However, if the source
8097 remote points to a file, rclone will just copy that file. The destina‐
8098 tion remote must point to a directory - rclone will give the error
8099 Failed to create file system for "remote:file": is a file not a direc‐
8100 tory if it isn't.
8101
8102 For example, suppose you have a remote with a file in called test.jpg,
8103 then you could copy just that file like this
8104
8105 rclone copy remote:test.jpg /tmp/download
8106
8107 The file test.jpg will be placed inside /tmp/download.
8108
8109 This is equivalent to specifying
8110
8111 rclone copy --files-from /tmp/files remote: /tmp/download
8112
8113 Where /tmp/files contains the single line
8114
8115 test.jpg
8116
8117 It is recommended to use copy when copying individual files, not sync.
8118 They have pretty much the same effect but copy will use a lot less mem‐
8119 ory.
8120
8121 Syntax of remote paths
8122 The syntax of the paths passed to the rclone command are as follows.
8123
8124 /path/to/dir
8125 This refers to the local file system.
8126
8127 On Windows \ may be used instead of / in local paths only, non local
8128 paths must use /. See local filesystem (https://rclone.org/lo‐
8129 cal/#paths-on-windows) documentation for more about Windows-specific
8130 paths.
8131
8132 These paths needn't start with a leading / - if they don't then they
8133 will be relative to the current directory.
8134
8135 remote:path/to/dir
8136 This refers to a directory path/to/dir on remote: as defined in the
8137 config file (configured with rclone config).
8138
8139 remote:/path/to/dir
8140 On most backends this is refers to the same directory as re‐
8141 mote:path/to/dir and that format should be preferred. On a very small
8142 number of remotes (FTP, SFTP, Dropbox for business) this will refer to
8143 a different directory. On these, paths without a leading / will refer
8144 to your "home" directory and paths with a leading / will refer to the
8145 root.
8146
8147 :backend:path/to/dir
8148 This is an advanced form for creating remotes on the fly. backend
8149 should be the name or prefix of a backend (the type in the config file)
8150 and all the configuration for the backend should be provided on the
8151 command line (or in environment variables).
8152
8153 Here are some examples:
8154
8155 rclone lsd --http-url https://pub.rclone.org :http:
8156
8157 To list all the directories in the root of https://pub.rclone.org/.
8158
8159 rclone lsf --http-url https://example.com :http:path/to/dir
8160
8161 To list files and directories in https://example.com/path/to/dir/
8162
8163 rclone copy --http-url https://example.com :http:path/to/dir /tmp/dir
8164
8165 To copy files and directories in https://example.com/path/to/dir to
8166 /tmp/dir.
8167
8168 rclone copy --sftp-host example.com :sftp:path/to/dir /tmp/dir
8169
8170 To copy files and directories from example.com in the relative directo‐
8171 ry path/to/dir to /tmp/dir using sftp.
8172
8173 Connection strings
8174 The above examples can also be written using a connection string syn‐
8175 tax, so instead of providing the arguments as command line parameters
8176 --http-url https://pub.rclone.org they are provided as part of the re‐
8177 mote specification as a kind of connection string.
8178
8179 rclone lsd ":http,url='https://pub.rclone.org':"
8180 rclone lsf ":http,url='https://example.com':path/to/dir"
8181 rclone copy ":http,url='https://example.com':path/to/dir" /tmp/dir
8182 rclone copy :sftp,host=example.com:path/to/dir /tmp/dir
8183
8184 These can apply to modify existing remotes as well as create new re‐
8185 motes with the on the fly syntax. This example is equivalent to adding
8186 the --drive-shared-with-me parameter to the remote gdrive:.
8187
8188 rclone lsf "gdrive,shared_with_me:path/to/dir"
8189
8190 The major advantage to using the connection string style syntax is that
8191 it only applies to the remote, not to all the remotes of that type of
8192 the command line. A common confusion is this attempt to copy a file
8193 shared on google drive to the normal drive which does not work because
8194 the --drive-shared-with-me flag applies to both the source and the des‐
8195 tination.
8196
8197 rclone copy --drive-shared-with-me gdrive:shared-file.txt gdrive:
8198
8199 However using the connection string syntax, this does work.
8200
8201 rclone copy "gdrive,shared_with_me:shared-file.txt" gdrive:
8202
8203 Note that the connection string only affects the options of the immedi‐
8204 ate backend. If for example gdriveCrypt is a crypt based on gdrive,
8205 then the following command will not work as intended, because
8206 shared_with_me is ignored by the crypt backend:
8207
8208 rclone copy "gdriveCrypt,shared_with_me:shared-file.txt" gdriveCrypt:
8209
8210 The connection strings have the following syntax
8211
8212 remote,parameter=value,parameter2=value2:path/to/dir
8213 :backend,parameter=value,parameter2=value2:path/to/dir
8214
8215 If the parameter has a : or , then it must be placed in quotes " or ',
8216 so
8217
8218 remote,parameter="colon:value",parameter2="comma,value":path/to/dir
8219 :backend,parameter='colon:value',parameter2='comma,value':path/to/dir
8220
8221 If a quoted value needs to include that quote, then it should be dou‐
8222 bled, so
8223
8224 remote,parameter="with""quote",parameter2='with''quote':path/to/dir
8225
8226 This will make parameter be with"quote and parameter2 be with'quote.
8227
8228 If you leave off the =parameter then rclone will substitute =true which
8229 works very well with flags. For example, to use s3 configured in the
8230 environment you could use:
8231
8232 rclone lsd :s3,env_auth:
8233
8234 Which is equivalent to
8235
8236 rclone lsd :s3,env_auth=true:
8237
8238 Note that on the command line you might need to surround these connec‐
8239 tion strings with " or ' to stop the shell interpreting any special
8240 characters within them.
8241
8242 If you are a shell master then you'll know which strings are OK and
8243 which aren't, but if you aren't sure then enclose them in " and use '
8244 as the inside quote. This syntax works on all OSes.
8245
8246 rclone copy ":http,url='https://example.com':path/to/dir" /tmp/dir
8247
8248 On Linux/macOS some characters are still interpreted inside " strings
8249 in the shell (notably \ and $ and ") so if your strings contain those
8250 you can swap the roles of " and ' thus. (This syntax does not work on
8251 Windows.)
8252
8253 rclone copy ':http,url="https://example.com":path/to/dir' /tmp/dir
8254
8255 Connection strings, config and logging
8256 If you supply extra configuration to a backend by command line flag,
8257 environment variable or connection string then rclone will add a suffix
8258 based on the hash of the config to the name of the remote, eg
8259
8260 rclone -vv lsf --s3-chunk-size 20M s3:
8261
8262 Has the log message
8263
8264 DEBUG : s3: detected overridden config - adding "{Srj1p}" suffix to name
8265
8266 This is so rclone can tell the modified remote apart from the unmodi‐
8267 fied remote when caching the backends.
8268
8269 This should only be noticeable in the logs.
8270
8271 This means that on the fly backends such as
8272
8273 rclone -vv lsf :s3,env_auth:
8274
8275 Will get their own names
8276
8277 DEBUG : :s3: detected overridden config - adding "{YTu53}" suffix to name
8278
8279 Valid remote names
8280 Remote names are case sensitive, and must adhere to the following
8281 rules: - May only contain 0-9, A-Z, a-z, _, -, . and space. - May not
8282 start with - or space.
8283
8284 Quoting and the shell
8285 When you are typing commands to your computer you are using something
8286 called the command line shell. This interprets various characters in
8287 an OS specific way.
8288
8289 Here are some gotchas which may help users unfamiliar with the shell
8290 rules
8291
8292 Linux / OSX
8293 If your names have spaces or shell metacharacters (e.g. *, ?, $, ', ",
8294 etc.) then you must quote them. Use single quotes ' by default.
8295
8296 rclone copy 'Important files?' remote:backup
8297
8298 If you want to send a ' you will need to use ", e.g.
8299
8300 rclone copy "O'Reilly Reviews" remote:backup
8301
8302 The rules for quoting metacharacters are complicated and if you want
8303 the full details you'll have to consult the manual page for your shell.
8304
8305 Windows
8306 If your names have spaces in you need to put them in ", e.g.
8307
8308 rclone copy "E:\folder name\folder name\folder name" remote:backup
8309
8310 If you are using the root directory on its own then don't quote it (see
8311 #464 (https://github.com/rclone/rclone/issues/464) for why), e.g.
8312
8313 rclone copy E:\ remote:backup
8314
8315 Copying files or directories with : in the names
8316 rclone uses : to mark a remote name. This is, however, a valid file‐
8317 name component in non-Windows OSes. The remote name parser will only
8318 search for a : up to the first / so if you need to act on a file or di‐
8319 rectory like this then use the full path starting with a /, or use ./
8320 as a current directory prefix.
8321
8322 So to sync a directory called sync:me to a remote called remote: use
8323
8324 rclone sync -i ./sync:me remote:path
8325
8326 or
8327
8328 rclone sync -i /full/path/to/sync:me remote:path
8329
8330 Server Side Copy
8331 Most remotes (but not all - see the overview (https://rclone.org/over‐
8332 view/#optional-features)) support server-side copy.
8333
8334 This means if you want to copy one folder to another then rclone won't
8335 download all the files and re-upload them; it will instruct the server
8336 to copy them in place.
8337
8338 Eg
8339
8340 rclone copy s3:oldbucket s3:newbucket
8341
8342 Will copy the contents of oldbucket to newbucket without downloading
8343 and re-uploading.
8344
8345 Remotes which don't support server-side copy will download and re-up‐
8346 load in this case.
8347
8348 Server side copies are used with sync and copy and will be identified
8349 in the log when using the -v flag. The move command may also use them
8350 if remote doesn't support server-side move directly. This is done by
8351 issuing a server-side copy then a delete which is much quicker than a
8352 download and re-upload.
8353
8354 Server side copies will only be attempted if the remote names are the
8355 same.
8356
8357 This can be used when scripting to make aged backups efficiently, e.g.
8358
8359 rclone sync -i remote:current-backup remote:previous-backup
8360 rclone sync -i /path/to/files remote:current-backup
8361
8362 Metadata support
8363 Metadata is data about a file which isn't the contents of the file.
8364 Normally rclone only preserves the modification time and the content
8365 (MIME) type where possible.
8366
8367 Rclone supports preserving all the available metadata on files (not di‐
8368 rectories) when using the --metadata or -M flag.
8369
8370 Exactly what metadata is supported and what that support means depends
8371 on the backend. Backends that support metadata have a metadata section
8372 in their docs and are listed in the features table
8373 (https://rclone.org/overview/#features) (Eg local
8374 (https://rclone.org/local/#metadata), s3)
8375
8376 Rclone only supports a one-time sync of metadata. This means that
8377 metadata will be synced from the source object to the destination ob‐
8378 ject only when the source object has changed and needs to be re-upload‐
8379 ed. If the metadata subsequently changes on the source object without
8380 changing the object itself then it won't be synced to the destination
8381 object. This is in line with the way rclone syncs Content-Type without
8382 the --metadata flag.
8383
8384 Using --metadata when syncing from local to local will preserve file
8385 attributes such as file mode, owner, extended attributes (not Windows).
8386
8387 Note that arbitrary metadata may be added to objects using the --meta‐
8388 data-set key=value flag when the object is first uploaded. This flag
8389 can be repeated as many times as necessary.
8390
8391 Types of metadata
8392 Metadata is divided into two type. System metadata and User metadata.
8393
8394 Metadata which the backend uses itself is called system metadata. For
8395 example on the local backend the system metadata uid will store the us‐
8396 er ID of the file when used on a unix based platform.
8397
8398 Arbitrary metadata is called user metadata and this can be set however
8399 is desired.
8400
8401 When objects are copied from backend to backend, they will attempt to
8402 interpret system metadata if it is supplied. Metadata may change from
8403 being user metadata to system metadata as objects are copied between
8404 different backends. For example copying an object from s3 sets the
8405 content-type metadata. In a backend which understands this (like
8406 azureblob) this will become the Content-Type of the object. In a back‐
8407 end which doesn't understand this (like the local backend) this will
8408 become user metadata. However should the local object be copied back
8409 to s3, the Content-Type will be set correctly.
8410
8411 Metadata framework
8412 Rclone implements a metadata framework which can read metadata from an
8413 object and write it to the object when (and only when) it is being up‐
8414 loaded.
8415
8416 This metadata is stored as a dictionary with string keys and string
8417 values.
8418
8419 There are some limits on the names of the keys (these may be clarified
8420 further in the future).
8421
8422 • must be lower case
8423
8424 • may be a-z 0-9 containing . - or _
8425
8426 • length is backend dependent
8427
8428 Each backend can provide system metadata that it understands. Some
8429 backends can also store arbitrary user metadata.
8430
8431 Where possible the key names are standardized, so, for example, it is
8432 possible to copy object metadata from s3 to azureblob for example and
8433 metadata will be translated appropriately.
8434
8435 Some backends have limits on the size of the metadata and rclone will
8436 give errors on upload if they are exceeded.
8437
8438 Metadata preservation
8439 The goal of the implementation is to
8440
8441 1. Preserve metadata if at all possible
8442
8443 2. Interpret metadata if at all possible
8444
8445 The consequences of 1 is that you can copy an S3 object to a local disk
8446 then back to S3 losslessly. Likewise you can copy a local file with
8447 file attributes and xattrs from local disk to s3 and back again loss‐
8448 lessly.
8449
8450 The consequence of 2 is that you can copy an S3 object with metadata to
8451 Azureblob (say) and have the metadata appear on the Azureblob object
8452 also.
8453
8454 Standard system metadata
8455 Here is a table of standard system metadata which, if appropriate, a
8456 backend may implement.
8457
8458 key description example
8459 ─────────────────────────────────────────────────────────────────────────────────────────────────
8460 mode File type and mode: 0100664
8461 octal, unix style
8462 uid User ID of owner: 500
8463 decimal number
8464 gid Group ID of owner: 500
8465 decimal number
8466 rdev Device ID (if special 0
8467 file) => hexadecimal
8468 atime Time of last access: 2006-01-02T15:04:05.999999999Z07:00
8469 RFC 3339
8470 mtime Time of last modifi‐ 2006-01-02T15:04:05.999999999Z07:00
8471 cation: RFC 3339
8472 btime Time of file creation 2006-01-02T15:04:05.999999999Z07:00
8473 (birth): RFC 3339
8474 cache-control Cache-Control header no-cache
8475 content-disposition Content-Disposition inline
8476 header
8477 content-encoding Content-Encoding gzip
8478 header
8479 content-language Content-Language en-US
8480 header
8481 content-type Content-Type header text/plain
8482
8483 The metadata keys mtime and content-type will take precedence if sup‐
8484 plied in the metadata over reading the Content-Type or modification
8485 time of the source object.
8486
8487 Hashes are not included in system metadata as there is a well defined
8488 way of reading those already.
8489
8490 Options
8491 Rclone has a number of options to control its behaviour.
8492
8493 Options that take parameters can have the values passed in two ways,
8494 --option=value or --option value. However boolean (true/false) options
8495 behave slightly differently to the other options in that --boolean sets
8496 the option to true and the absence of the flag sets it to false. It is
8497 also possible to specify --boolean=false or --boolean=true. Note that
8498 --boolean false is not valid - this is parsed as --boolean and the
8499 false is parsed as an extra command line argument for rclone.
8500
8501 Time or duration options
8502 TIME or DURATION options can be specified as a duration string or a
8503 time string.
8504
8505 A duration string is a possibly signed sequence of decimal numbers,
8506 each with optional fraction and a unit suffix, such as "300ms", "-1.5h"
8507 or "2h45m". Default units are seconds or the following abbreviations
8508 are valid:
8509
8510 • ms - Milliseconds
8511
8512 • s - Seconds
8513
8514 • m - Minutes
8515
8516 • h - Hours
8517
8518 • d - Days
8519
8520 • w - Weeks
8521
8522 • M - Months
8523
8524 • y - Years
8525
8526 These can also be specified as an absolute time in the following for‐
8527 mats:
8528
8529 • RFC3339 - e.g. 2006-01-02T15:04:05Z or 2006-01-02T15:04:05+07:00
8530
8531 • ISO8601 Date and time, local timezone - 2006-01-02T15:04:05
8532
8533 • ISO8601 Date and time, local timezone - 2006-01-02 15:04:05
8534
8535 • ISO8601 Date - 2006-01-02 (YYYY-MM-DD)
8536
8537 Size options
8538 Options which use SIZE use KiB (multiples of 1024 bytes) by default.
8539 However, a suffix of B for Byte, K for KiB, M for MiB, G for GiB, T for
8540 TiB and P for PiB may be used. These are the binary units, e.g. 1,
8541 2**10, 2**20, 2**30 respectively.
8542
8543 --backup-dir=DIR
8544 When using sync, copy or move any files which would have been overwrit‐
8545 ten or deleted are moved in their original hierarchy into this directo‐
8546 ry.
8547
8548 If --suffix is set, then the moved files will have the suffix added to
8549 them. If there is a file with the same path (after the suffix has been
8550 added) in DIR, then it will be overwritten.
8551
8552 The remote in use must support server-side move or copy and you must
8553 use the same remote as the destination of the sync. The backup direc‐
8554 tory must not overlap the destination directory without it being ex‐
8555 cluded by a filter rule.
8556
8557 For example
8558
8559 rclone sync -i /path/to/local remote:current --backup-dir remote:old
8560
8561 will sync /path/to/local to remote:current, but for any files which
8562 would have been updated or deleted will be stored in remote:old.
8563
8564 If running rclone from a script you might want to use today's date as
8565 the directory name passed to --backup-dir to store the old files, or
8566 you might want to pass --suffix with today's date.
8567
8568 See --compare-dest and --copy-dest.
8569
8570 --bind string
8571 Local address to bind to for outgoing connections. This can be an IPv4
8572 address (1.2.3.4), an IPv6 address (1234::789A) or host name. If the
8573 host name doesn't resolve or resolves to more than one IP address it
8574 will give an error.
8575
8576 --bwlimit=BANDWIDTH_SPEC
8577 This option controls the bandwidth limit. For example
8578
8579 --bwlimit 10M
8580
8581 would mean limit the upload and download bandwidth to 10 MiB/s. NB
8582 this is bytes per second not bits per second. To use a single limit,
8583 specify the desired bandwidth in KiB/s, or use a suffix B|K|M|G|T|P.
8584 The default is 0 which means to not limit bandwidth.
8585
8586 The upload and download bandwidth can be specified separately, as
8587 --bwlimit UP:DOWN, so
8588
8589 --bwlimit 10M:100k
8590
8591 would mean limit the upload bandwidth to 10 MiB/s and the download
8592 bandwidth to 100 KiB/s. Either limit can be "off" meaning no limit, so
8593 to just limit the upload bandwidth you would use
8594
8595 --bwlimit 10M:off
8596
8597 this would limit the upload bandwidth to 10 MiB/s but the download
8598 bandwidth would be unlimited.
8599
8600 When specified as above the bandwidth limits last for the duration of
8601 run of the rclone binary.
8602
8603 It is also possible to specify a "timetable" of limits, which will
8604 cause certain limits to be applied at certain times. To specify a
8605 timetable, format your entries as WEEKDAY-HH:MM,BANDWIDTH WEEKDAY-
8606 HH:MM,BANDWIDTH... where: WEEKDAY is optional element.
8607
8608 • BANDWIDTH can be a single number, e.g.100k or a pair of numbers for
8609 upload:download, e.g.10M:1M.
8610
8611 • WEEKDAY can be written as the whole word or only using the first 3
8612 characters. It is optional.
8613
8614 • HH:MM is an hour from 00:00 to 23:59.
8615
8616 An example of a typical timetable to avoid link saturation during day‐
8617 time working hours could be:
8618
8619 --bwlimit "08:00,512k 12:00,10M 13:00,512k 18:00,30M 23:00,off"
8620
8621 In this example, the transfer bandwidth will be set to 512 KiB/s at 8am
8622 every day. At noon, it will rise to 10 MiB/s, and drop back to 512
8623 KiB/sec at 1pm. At 6pm, the bandwidth limit will be set to 30 MiB/s,
8624 and at 11pm it will be completely disabled (full speed). Anything be‐
8625 tween 11pm and 8am will remain unlimited.
8626
8627 An example of timetable with WEEKDAY could be:
8628
8629 --bwlimit "Mon-00:00,512 Fri-23:59,10M Sat-10:00,1M Sun-20:00,off"
8630
8631 It means that, the transfer bandwidth will be set to 512 KiB/s on Mon‐
8632 day. It will rise to 10 MiB/s before the end of Friday. At 10:00 on
8633 Saturday it will be set to 1 MiB/s. From 20:00 on Sunday it will be
8634 unlimited.
8635
8636 Timeslots without WEEKDAY are extended to the whole week. So this ex‐
8637 ample:
8638
8639 --bwlimit "Mon-00:00,512 12:00,1M Sun-20:00,off"
8640
8641 Is equivalent to this:
8642
8643 --bwlimit "Mon-00:00,512Mon-12:00,1M Tue-12:00,1M Wed-12:00,1M
8644 Thu-12:00,1M Fri-12:00,1M Sat-12:00,1M Sun-12:00,1M Sun-20:00,off"
8645
8646 Bandwidth limit apply to the data transfer for all backends. For most
8647 backends the directory listing bandwidth is also included (exceptions
8648 being the non HTTP backends, ftp, sftp and storj).
8649
8650 Note that the units are Byte/s, not bit/s. Typically connections are
8651 measured in bit/s - to convert divide by 8. For example, let's say you
8652 have a 10 Mbit/s connection and you wish rclone to use half of it - 5
8653 Mbit/s. This is 5/8 = 0.625 MiB/s so you would use a --bwlimit 0.625M
8654 parameter for rclone.
8655
8656 On Unix systems (Linux, macOS, ...) the bandwidth limiter can be tog‐
8657 gled by sending a SIGUSR2 signal to rclone. This allows to remove the
8658 limitations of a long running rclone transfer and to restore it back to
8659 the value specified with --bwlimit quickly when needed. Assuming there
8660 is only one rclone instance running, you can toggle the limiter like
8661 this:
8662
8663 kill -SIGUSR2 $(pidof rclone)
8664
8665 If you configure rclone with a remote control then you can use change
8666 the bwlimit dynamically:
8667
8668 rclone rc core/bwlimit rate=1M
8669
8670 --bwlimit-file=BANDWIDTH_SPEC
8671 This option controls per file bandwidth limit. For the options see the
8672 --bwlimit flag.
8673
8674 For example use this to allow no transfers to be faster than 1 MiB/s
8675
8676 --bwlimit-file 1M
8677
8678 This can be used in conjunction with --bwlimit.
8679
8680 Note that if a schedule is provided the file will use the schedule in
8681 effect at the start of the transfer.
8682
8683 --buffer-size=SIZE
8684 Use this sized buffer to speed up file transfers. Each --transfer will
8685 use this much memory for buffering.
8686
8687 When using mount or cmount each open file descriptor will use this much
8688 memory for buffering. See the mount (https://rclone.org/com‐
8689 mands/rclone_mount/#file-buffering) documentation for more details.
8690
8691 Set to 0 to disable the buffering for the minimum memory usage.
8692
8693 Note that the memory allocation of the buffers is influenced by the
8694 --use-mmap flag.
8695
8696 --cache-dir=DIR
8697 Specify the directory rclone will use for caching, to override the de‐
8698 fault.
8699
8700 Default value is depending on operating system: - Windows %LocalAppDa‐
8701 ta%\rclone, if LocalAppData is defined. - macOS $HOME/Li‐
8702 brary/Caches/rclone if HOME is defined. - Unix $XDG_CACHE_HOME/rclone
8703 if XDG_CACHE_HOME is defined, else $HOME/.cache/rclone if HOME is de‐
8704 fined. - Fallback (on all OS) to $TMPDIR/rclone, where TMPDIR is the
8705 value from --temp-dir.
8706
8707 You can use the config paths (https://rclone.org/commands/rclone_con‐
8708 fig_paths/) command to see the current value.
8709
8710 Cache directory is heavily used by the VFS File Caching
8711 (https://rclone.org/commands/rclone_mount/#vfs-file-caching) mount fea‐
8712 ture, but also by serve (https://rclone.org/commands/rclone_serve/),
8713 GUI and other parts of rclone.
8714
8715 --check-first
8716 If this flag is set then in a sync, copy or move, rclone will do all
8717 the checks to see whether files need to be transferred before doing any
8718 of the transfers. Normally rclone would start running transfers as
8719 soon as possible.
8720
8721 This flag can be useful on IO limited systems where transfers interfere
8722 with checking.
8723
8724 It can also be useful to ensure perfect ordering when using --order-by.
8725
8726 Using this flag can use more memory as it effectively sets --max-back‐
8727 log to infinite. This means that all the info on the objects to trans‐
8728 fer is held in memory before the transfers start.
8729
8730 --checkers=N
8731 Originally controlling just the number of file checkers to run in par‐
8732 allel, e.g. by rclone copy. Now a fairly universal parallelism con‐
8733 trol used by rclone in several places.
8734
8735 Note: checkers do the equality checking of files during a sync. For
8736 some storage systems (e.g. S3, Swift, Dropbox) this can take a signif‐
8737 icant amount of time so they are run in parallel.
8738
8739 The default is to run 8 checkers in parallel. However, in case of
8740 slow-reacting backends you may need to lower (rather than increase)
8741 this default by setting --checkers to 4 or less threads. This is espe‐
8742 cially advised if you are experiencing backend server crashes during
8743 file checking phase (e.g. on subsequent or top-up backups where little
8744 or no file copying is done and checking takes up most of the time).
8745 Increase this setting only with utmost care, while monitoring your
8746 server health and file checking throughput.
8747
8748 -c, --checksum
8749 Normally rclone will look at modification time and size of files to see
8750 if they are equal. If you set this flag then rclone will check the
8751 file hash and size to determine if files are equal.
8752
8753 This is useful when the remote doesn't support setting modified time
8754 and a more accurate sync is desired than just checking the file size.
8755
8756 This is very useful when transferring between remotes which store the
8757 same hash type on the object, e.g. Drive and Swift. For details of
8758 which remotes support which hash type see the table in the overview
8759 section (https://rclone.org/overview/).
8760
8761 Eg rclone --checksum sync s3:/bucket swift:/bucket would run much
8762 quicker than without the --checksum flag.
8763
8764 When using this flag, rclone won't update mtimes of remote files if
8765 they are incorrect as it would normally.
8766
8767 --compare-dest=DIR
8768 When using sync, copy or move DIR is checked in addition to the desti‐
8769 nation for files. If a file identical to the source is found that file
8770 is NOT copied from source. This is useful to copy just files that have
8771 changed since the last backup.
8772
8773 You must use the same remote as the destination of the sync. The com‐
8774 pare directory must not overlap the destination directory.
8775
8776 See --copy-dest and --backup-dir.
8777
8778 --config=CONFIG_FILE
8779 Specify the location of the rclone configuration file, to override the
8780 default. E.g. rclone config --config="rclone.conf".
8781
8782 The exact default is a bit complex to describe, due to changes intro‐
8783 duced through different versions of rclone while preserving backwards
8784 compatibility, but in most cases it is as simple as:
8785
8786 • %APPDATA%/rclone/rclone.conf on Windows
8787
8788 • ~/.config/rclone/rclone.conf on other
8789
8790 The complete logic is as follows: Rclone will look for an existing con‐
8791 figuration file in any of the following locations, in priority order:
8792
8793 1. rclone.conf (in program directory, where rclone executable is)
8794
8795 2. %APPDATA%/rclone/rclone.conf (only on Windows)
8796
8797 3. $XDG_CONFIG_HOME/rclone/rclone.conf (on all systems, including Win‐
8798 dows)
8799
8800 4. ~/.config/rclone/rclone.conf (see below for explanation of ~ symbol)
8801
8802 5. ~/.rclone.conf
8803
8804 If no existing configuration file is found, then a new one will be cre‐
8805 ated in the following location:
8806
8807 • On Windows: Location 2 listed above, except in the unlikely event
8808 that APPDATA is not defined, then location 4 is used instead.
8809
8810 • On Unix: Location 3 if XDG_CONFIG_HOME is defined, else location 4.
8811
8812 • Fallback to location 5 (on all OS), when the rclone directory cannot
8813 be created, but if also a home directory was not found then path
8814 .rclone.conf relative to current working directory will be used as a
8815 final resort.
8816
8817 The ~ symbol in paths above represent the home directory of the current
8818 user on any OS, and the value is defined as following:
8819
8820 • On Windows: %HOME% if defined, else %USERPROFILE%, or else
8821 %HOMEDRIVE%\%HOMEPATH%.
8822
8823 • On Unix: $HOME if defined, else by looking up current user in OS-spe‐
8824 cific user database (e.g. passwd file), or else use the result from
8825 shell command cd && pwd.
8826
8827 If you run rclone config file you will see where the default location
8828 is for you.
8829
8830 The fact that an existing file rclone.conf in the same directory as the
8831 rclone executable is always preferred, means that it is easy to run in
8832 "portable" mode by downloading rclone executable to a writable directo‐
8833 ry and then create an empty file rclone.conf in the same directory.
8834
8835 If the location is set to empty string "" or path to a file with name
8836 notfound, or the os null device represented by value NUL on Windows and
8837 /dev/null on Unix systems, then rclone will keep the config file in
8838 memory only.
8839
8840 The file format is basic INI (https://en.wikipedia.org/wi‐
8841 ki/INI_file#Format): Sections of text, led by a [section] header and
8842 followed by key=value entries on separate lines. In rclone each remote
8843 is represented by its own section, where the section name defines the
8844 name of the remote. Options are specified as the key=value entries,
8845 where the key is the option name without the --backend- prefix, in low‐
8846 ercase and with _ instead of -. E.g. option --mega-hard-delete corre‐
8847 sponds to key hard_delete. Only backend options can be specified. A
8848 special, and required, key type identifies the storage system
8849 (https://rclone.org/overview/), where the value is the internal lower‐
8850 case name as returned by command rclone help backends. Comments are
8851 indicated by ; or # at the beginning of a line.
8852
8853 Example:
8854
8855 [megaremote]
8856 type = mega
8857 user = you@example.com
8858 pass = PDPcQVVjVtzFY-GTdDFozqBhTdsPg3qH
8859
8860 Note that passwords are in obscured (https://rclone.org/com‐
8861 mands/rclone_obscure/) form. Also, many storage systems uses token-
8862 based authentication instead of passwords, and this requires additional
8863 steps. It is easier, and safer, to use the interactive command rclone
8864 config instead of manually editing the configuration file.
8865
8866 The configuration file will typically contain login information, and
8867 should therefore have restricted permissions so that only the current
8868 user can read it. Rclone tries to ensure this when it writes the file.
8869 You may also choose to encrypt the file.
8870
8871 When token-based authentication are used, the configuration file must
8872 be writable, because rclone needs to update the tokens inside it.
8873
8874 --contimeout=TIME
8875 Set the connection timeout. This should be in go time format which
8876 looks like 5s for 5 seconds, 10m for 10 minutes, or 3h30m.
8877
8878 The connection timeout is the amount of time rclone will wait for a
8879 connection to go through to a remote object storage system. It is 1m
8880 by default.
8881
8882 --copy-dest=DIR
8883 When using sync, copy or move DIR is checked in addition to the desti‐
8884 nation for files. If a file identical to the source is found that file
8885 is server-side copied from DIR to the destination. This is useful for
8886 incremental backup.
8887
8888 The remote in use must support server-side copy and you must use the
8889 same remote as the destination of the sync. The compare directory must
8890 not overlap the destination directory.
8891
8892 See --compare-dest and --backup-dir.
8893
8894 --dedupe-mode MODE
8895 Mode to run dedupe command in. One of interactive, skip, first, new‐
8896 est, oldest, rename. The default is interactive.
8897 See the dedupe command for more information as to what these options
8898 mean.
8899
8900 --disable FEATURE,FEATURE,...
8901 This disables a comma separated list of optional features. For example
8902 to disable server-side move and server-side copy use:
8903
8904 --disable move,copy
8905
8906 The features can be put in any case.
8907
8908 To see a list of which features can be disabled use:
8909
8910 --disable help
8911
8912 See the overview features (https://rclone.org/overview/#features) and
8913 optional features (https://rclone.org/overview/#optional-features) to
8914 get an idea of which feature does what.
8915
8916 This flag can be useful for debugging and in exceptional circumstances
8917 (e.g. Google Drive limiting the total volume of Server Side Copies to
8918 100 GiB/day).
8919
8920 --disable-http2
8921 This stops rclone from trying to use HTTP/2 if available. This can
8922 sometimes speed up transfers due to a problem in the Go standard li‐
8923 brary (https://github.com/golang/go/issues/37373).
8924
8925 --dscp VALUE
8926 Specify a DSCP value or name to use in connections. This could help
8927 QoS system to identify traffic class. BE, EF, DF, LE, CSx and AFxx are
8928 allowed.
8929
8930 See the description of differentiated services
8931 (https://en.wikipedia.org/wiki/Differentiated_services) to get an idea
8932 of this field. Setting this to 1 (LE) to identify the flow to SCAV‐
8933 ENGER class can avoid occupying too much bandwidth in a network with
8934 DiffServ support (RFC 8622 (https://tools.ietf.org/html/rfc8622)).
8935
8936 For example, if you configured QoS on router to handle LE properly.
8937 Running:
8938
8939 rclone copy --dscp LE from:/from to:/to
8940
8941 would make the priority lower than usual internet flows.
8942
8943 This option has no effect on Windows (see golang/go#42728
8944 (https://github.com/golang/go/issues/42728)).
8945
8946 -n, --dry-run
8947 Do a trial run with no permanent changes. Use this to see what rclone
8948 would do without actually doing it. Useful when setting up the sync
8949 command which deletes files in the destination.
8950
8951 --expect-continue-timeout=TIME
8952 This specifies the amount of time to wait for a server's first response
8953 headers after fully writing the request headers if the request has an
8954 "Expect: 100-continue" header. Not all backends support using this.
8955
8956 Zero means no timeout and causes the body to be sent immediately, with‐
8957 out waiting for the server to approve. This time does not include the
8958 time to send the request header.
8959
8960 The default is 1s. Set to 0 to disable.
8961
8962 --error-on-no-transfer
8963 By default, rclone will exit with return code 0 if there were no er‐
8964 rors.
8965
8966 This option allows rclone to return exit code 9 if no files were trans‐
8967 ferred between the source and destination. This allows using rclone in
8968 scripts, and triggering follow-on actions if data was copied, or skip‐
8969 ping if not.
8970
8971 NB: Enabling this option turns a usually non-fatal error into a poten‐
8972 tially fatal one - please check and adjust your scripts accordingly!
8973
8974 --fs-cache-expire-duration=TIME
8975 When using rclone via the API rclone caches created remotes for 5 min‐
8976 utes by default in the "fs cache". This means that if you do repeated
8977 actions on the same remote then rclone won't have to build it again
8978 from scratch, which makes it more efficient.
8979
8980 This flag sets the time that the remotes are cached for. If you set it
8981 to 0 (or negative) then rclone won't cache the remotes at all.
8982
8983 Note that if you use some flags, eg --backup-dir and if this is set to
8984 0 rclone may build two remotes (one for the source or destination and
8985 one for the --backup-dir where it may have only built one before.
8986
8987 --fs-cache-expire-interval=TIME
8988 This controls how often rclone checks for cached remotes to expire.
8989 See the --fs-cache-expire-duration documentation above for more info.
8990 The default is 60s, set to 0 to disable expiry.
8991
8992 --header
8993 Add an HTTP header for all transactions. The flag can be repeated to
8994 add multiple headers.
8995
8996 If you want to add headers only for uploads use --header-upload and if
8997 you want to add headers only for downloads use --header-download.
8998
8999 This flag is supported for all HTTP based backends even those not sup‐
9000 ported by --header-upload and --header-download so may be used as a
9001 workaround for those with care.
9002
9003 rclone ls remote:test --header "X-Rclone: Foo" --header "X-LetMeIn: Yes"
9004
9005 --header-download
9006 Add an HTTP header for all download transactions. The flag can be re‐
9007 peated to add multiple headers.
9008
9009 rclone sync -i s3:test/src ~/dst --header-download "X-Amz-Meta-Test: Foo" --header-download "X-Amz-Meta-Test2: Bar"
9010
9011 See the GitHub issue here (https://github.com/rclone/rclone/issues/59)
9012 for currently supported backends.
9013
9014 --header-upload
9015 Add an HTTP header for all upload transactions. The flag can be re‐
9016 peated to add multiple headers.
9017
9018 rclone sync -i ~/src s3:test/dst --header-upload "Content-Disposition: attachment; filename='cool.html'" --header-upload "X-Amz-Meta-Test: FooBar"
9019
9020 See the GitHub issue here (https://github.com/rclone/rclone/issues/59)
9021 for currently supported backends.
9022
9023 --human-readable
9024 Rclone commands output values for sizes (e.g. number of bytes) and
9025 counts (e.g. number of files) either as raw numbers, or in human-read‐
9026 able format.
9027
9028 In human-readable format the values are scaled to larger units, indi‐
9029 cated with a suffix shown after the value, and rounded to three deci‐
9030 mals. Rclone consistently uses binary units (powers of 2) for sizes
9031 and decimal units (powers of 10) for counts. The unit prefix for size
9032 is according to IEC standard notation, e.g. Ki for kibi. Used with
9033 byte unit, 1 KiB means 1024 Byte. In list type of output, only the
9034 unit prefix appended to the value (e.g. 9.762Ki), while in more textu‐
9035 al output the full unit is shown (e.g. 9.762 KiB). For counts the SI
9036 standard notation is used, e.g. prefix k for kilo. Used with file
9037 counts, 1k means 1000 files.
9038
9039 The various list (https://rclone.org/commands/rclone_ls/) commands out‐
9040 put raw numbers by default. Option --human-readable will make them
9041 output values in human-readable format instead (with the short unit
9042 prefix).
9043
9044 The about (https://rclone.org/commands/rclone_about/) command outputs
9045 human-readable by default, with a command-specific option --full to
9046 output the raw numbers instead.
9047
9048 Command size (https://rclone.org/commands/rclone_size/) outputs both
9049 human-readable and raw numbers in the same output.
9050
9051 The tree (https://rclone.org/commands/rclone_tree/) command also con‐
9052 siders --human-readable, but it will not use the exact same notation as
9053 the other commands: It rounds to one decimal, and uses single letter
9054 suffix, e.g. K instead of Ki. The reason for this is that it relies
9055 on an external library.
9056
9057 The interactive command ncdu (https://rclone.org/commands/rclone_ncdu/)
9058 shows human-readable by default, and responds to key u for toggling hu‐
9059 man-readable format.
9060
9061 --ignore-case-sync
9062 Using this option will cause rclone to ignore the case of the files
9063 when synchronizing so files will not be copied/synced when the existing
9064 filenames are the same, even if the casing is different.
9065
9066 --ignore-checksum
9067 Normally rclone will check that the checksums of transferred files
9068 match, and give an error "corrupted on transfer" if they don't.
9069
9070 You can use this option to skip that check. You should only use it if
9071 you have had the "corrupted on transfer" error message and you are sure
9072 you might want to transfer potentially corrupted data.
9073
9074 --ignore-existing
9075 Using this option will make rclone unconditionally skip all files that
9076 exist on the destination, no matter the content of these files.
9077
9078 While this isn't a generally recommended option, it can be useful in
9079 cases where your files change due to encryption. However, it cannot
9080 correct partial transfers in case a transfer was interrupted.
9081
9082 When performing a move/moveto command, this flag will leave skipped
9083 files in the source location unchanged when a file with the same name
9084 exists on the destination.
9085
9086 --ignore-size
9087 Normally rclone will look at modification time and size of files to see
9088 if they are equal. If you set this flag then rclone will check only
9089 the modification time. If --checksum is set then it only checks the
9090 checksum.
9091
9092 It will also cause rclone to skip verifying the sizes are the same af‐
9093 ter transfer.
9094
9095 This can be useful for transferring files to and from OneDrive which
9096 occasionally misreports the size of image files (see #399
9097 (https://github.com/rclone/rclone/issues/399) for more info).
9098
9099 -I, --ignore-times
9100 Using this option will cause rclone to unconditionally upload all files
9101 regardless of the state of files on the destination.
9102
9103 Normally rclone would skip any files that have the same modification
9104 time and are the same size (or have the same checksum if using --check‐
9105 sum).
9106
9107 --immutable
9108 Treat source and destination files as immutable and disallow modifica‐
9109 tion.
9110
9111 With this option set, files will be created and deleted as requested,
9112 but existing files will never be updated. If an existing file does not
9113 match between the source and destination, rclone will give the error
9114 Source and destination exist but do not match: immutable file modified.
9115
9116 Note that only commands which transfer files (e.g. sync, copy, move)
9117 are affected by this behavior, and only modification is disallowed.
9118 Files may still be deleted explicitly (e.g. delete, purge) or implic‐
9119 itly (e.g. sync, move). Use copy --immutable if it is desired to
9120 avoid deletion as well as modification.
9121
9122 This can be useful as an additional layer of protection for immutable
9123 or append-only data sets (notably backup archives), where modification
9124 implies corruption and should not be propagated.
9125
9126 -i / --interactive
9127 This flag can be used to tell rclone that you wish a manual confirma‐
9128 tion before destructive operations.
9129
9130 It is recommended that you use this flag while learning rclone espe‐
9131 cially with rclone sync.
9132
9133 For example
9134
9135 $ rclone delete -i /tmp/dir
9136 rclone: delete "important-file.txt"?
9137 y) Yes, this is OK (default)
9138 n) No, skip this
9139 s) Skip all delete operations with no more questions
9140 !) Do all delete operations with no more questions
9141 q) Exit rclone now.
9142 y/n/s/!/q> n
9143
9144 The options mean
9145
9146 • y: Yes, this operation should go ahead. You can also press Return
9147 for this to happen. You'll be asked every time unless you choose s
9148 or !.
9149
9150 • n: No, do not do this operation. You'll be asked every time unless
9151 you choose s or !.
9152
9153 • s: Skip all the following operations of this type with no more ques‐
9154 tions. This takes effect until rclone exits. If there are any dif‐
9155 ferent kind of operations you'll be prompted for them.
9156
9157 • !: Do all the following operations with no more questions. Useful if
9158 you've decided that you don't mind rclone doing that kind of opera‐
9159 tion. This takes effect until rclone exits . If there are any dif‐
9160 ferent kind of operations you'll be prompted for them.
9161
9162 • q: Quit rclone now, just in case!
9163
9164 --leave-root
9165 During rmdirs it will not remove root directory, even if it's empty.
9166
9167 --log-file=FILE
9168 Log all of rclone's output to FILE. This is not active by default.
9169 This can be useful for tracking down problems with syncs in combination
9170 with the -v flag. See the Logging section for more info.
9171
9172 If FILE exists then rclone will append to it.
9173
9174 Note that if you are using the logrotate program to manage rclone's
9175 logs, then you should use the copytruncate option as rclone doesn't
9176 have a signal to rotate logs.
9177
9178 --log-format LIST
9179 Comma separated list of log format options. Accepted options are date,
9180 time, microseconds, pid, longfile, shortfile, UTC. Any other keywords
9181 will be silently ignored. pid will tag log messages with process iden‐
9182 tifier which useful with rclone mount --daemon. Other accepted options
9183 are explained in the go documentation (https://pkg.go.dev/log#pkg-con‐
9184 stants). The default log format is "date,time".
9185
9186 --log-level LEVEL
9187 This sets the log level for rclone. The default log level is NOTICE.
9188
9189 DEBUG is equivalent to -vv. It outputs lots of debug info - useful for
9190 bug reports and really finding out what rclone is doing.
9191
9192 INFO is equivalent to -v. It outputs information about each transfer
9193 and prints stats once a minute by default.
9194
9195 NOTICE is the default log level if no logging flags are supplied. It
9196 outputs very little when things are working normally. It outputs warn‐
9197 ings and significant events.
9198
9199 ERROR is equivalent to -q. It only outputs error messages.
9200
9201 --use-json-log
9202 This switches the log format to JSON for rclone. The fields of json
9203 log are level, msg, source, time.
9204
9205 --low-level-retries NUMBER
9206 This controls the number of low level retries rclone does.
9207
9208 A low level retry is used to retry a failing operation - typically one
9209 HTTP request. This might be uploading a chunk of a big file for exam‐
9210 ple. You will see low level retries in the log with the -v flag.
9211
9212 This shouldn't need to be changed from the default in normal opera‐
9213 tions. However, if you get a lot of low level retries you may wish to
9214 reduce the value so rclone moves on to a high level retry (see the
9215 --retries flag) quicker.
9216
9217 Disable low level retries with --low-level-retries 1.
9218
9219 --max-backlog=N
9220 This is the maximum allowable backlog of files in a sync/copy/move
9221 queued for being checked or transferred.
9222
9223 This can be set arbitrarily large. It will only use memory when the
9224 queue is in use. Note that it will use in the order of N KiB of memory
9225 when the backlog is in use.
9226
9227 Setting this large allows rclone to calculate how many files are pend‐
9228 ing more accurately, give a more accurate estimated finish time and
9229 make --order-by work more accurately.
9230
9231 Setting this small will make rclone more synchronous to the listings of
9232 the remote which may be desirable.
9233
9234 Setting this to a negative number will make the backlog as large as
9235 possible.
9236
9237 --max-delete=N
9238 This tells rclone not to delete more than N files. If that limit is
9239 exceeded then a fatal error will be generated and rclone will stop the
9240 operation in progress.
9241
9242 --max-depth=N
9243 This modifies the recursion depth for all the commands except purge.
9244
9245 So if you do rclone --max-depth 1 ls remote:path you will see only the
9246 files in the top level directory. Using --max-depth 2 means you will
9247 see all the files in first two directory levels and so on.
9248
9249 For historical reasons the lsd command defaults to using a --max-depth
9250 of 1 - you can override this with the command line flag.
9251
9252 You can use this command to disable recursion (with --max-depth 1).
9253
9254 Note that if you use this with sync and --delete-excluded the files not
9255 recursed through are considered excluded and will be deleted on the
9256 destination. Test first with --dry-run if you are not sure what will
9257 happen.
9258
9259 --max-duration=TIME
9260 Rclone will stop scheduling new transfers when it has run for the dura‐
9261 tion specified.
9262
9263 Defaults to off.
9264
9265 When the limit is reached any existing transfers will complete.
9266
9267 Rclone won't exit with an error if the transfer limit is reached.
9268
9269 --max-transfer=SIZE
9270 Rclone will stop transferring when it has reached the size specified.
9271 Defaults to off.
9272
9273 When the limit is reached all transfers will stop immediately.
9274
9275 Rclone will exit with exit code 8 if the transfer limit is reached.
9276
9277 --metadata / -M
9278 Setting this flag enables rclone to copy the metadata from the source
9279 to the destination. For local backends this is ownership, permissions,
9280 xattr etc. See the #metadata for more info.
9281
9282 --metadata-set key=value
9283 Add metadata key = value when uploading. This can be repeated as many
9284 times as required. See the #metadata for more info.
9285
9286 --cutoff-mode=hard|soft|cautious
9287 This modifies the behavior of --max-transfer Defaults to --cutoff-
9288 mode=hard.
9289
9290 Specifying --cutoff-mode=hard will stop transferring immediately when
9291 Rclone reaches the limit.
9292
9293 Specifying --cutoff-mode=soft will stop starting new transfers when
9294 Rclone reaches the limit.
9295
9296 Specifying --cutoff-mode=cautious will try to prevent Rclone from
9297 reaching the limit.
9298
9299 --modify-window=TIME
9300 When checking whether a file has been modified, this is the maximum al‐
9301 lowed time difference that a file can have and still be considered
9302 equivalent.
9303
9304 The default is 1ns unless this is overridden by a remote. For example
9305 OS X only stores modification times to the nearest second so if you are
9306 reading and writing to an OS X filing system this will be 1s by de‐
9307 fault.
9308
9309 This command line flag allows you to override that computed default.
9310
9311 --multi-thread-cutoff=SIZE
9312 When downloading files to the local backend above this size, rclone
9313 will use multiple threads to download the file (default 250M).
9314
9315 Rclone preallocates the file (using fallocate(FALLOC_FL_KEEP_SIZE) on
9316 unix or NTSetInformationFile on Windows both of which takes no time)
9317 then each thread writes directly into the file at the correct place.
9318 This means that rclone won't create fragmented or sparse files and
9319 there won't be any assembly time at the end of the transfer.
9320
9321 The number of threads used to download is controlled by --multi-thread-
9322 streams.
9323
9324 Use -vv if you wish to see info about the threads.
9325
9326 This will work with the sync/copy/move commands and friends copy‐
9327 to/moveto. Multi thread downloads will be used with rclone mount and
9328 rclone serve if --vfs-cache-mode is set to writes or above.
9329
9330 NB that this only works for a local destination but will work with any
9331 source.
9332
9333 NB that multi thread copies are disabled for local to local copies as
9334 they are faster without unless --multi-thread-streams is set explicit‐
9335 ly.
9336
9337 NB on Windows using multi-thread downloads will cause the resulting
9338 files to be sparse (https://en.wikipedia.org/wiki/Sparse_file). Use
9339 --local-no-sparse to disable sparse files (which may cause long delays
9340 at the start of downloads) or disable multi-thread downloads with
9341 --multi-thread-streams 0
9342
9343 --multi-thread-streams=N
9344 When using multi thread downloads (see above --multi-thread-cutoff)
9345 this sets the maximum number of streams to use. Set to 0 to disable
9346 multi thread downloads (Default 4).
9347
9348 Exactly how many streams rclone uses for the download depends on the
9349 size of the file. To calculate the number of download streams Rclone
9350 divides the size of the file by the --multi-thread-cutoff and rounds
9351 up, up to the maximum set with --multi-thread-streams.
9352
9353 So if --multi-thread-cutoff 250M and --multi-thread-streams 4 are in
9354 effect (the defaults):
9355
9356 • 0..250 MiB files will be downloaded with 1 stream
9357
9358 • 250..500 MiB files will be downloaded with 2 streams
9359
9360 • 500..750 MiB files will be downloaded with 3 streams
9361
9362 • 750+ MiB files will be downloaded with 4 streams
9363
9364 --no-check-dest
9365 The --no-check-dest can be used with move or copy and it causes rclone
9366 not to check the destination at all when copying files.
9367
9368 This means that:
9369
9370 • the destination is not listed minimising the API calls
9371
9372 • files are always transferred
9373
9374 • this can cause duplicates on remotes which allow it (e.g. Google
9375 Drive)
9376
9377 • --retries 1 is recommended otherwise you'll transfer everything again
9378 on a retry
9379
9380 This flag is useful to minimise the transactions if you know that none
9381 of the files are on the destination.
9382
9383 This is a specialized flag which should be ignored by most users!
9384
9385 --no-gzip-encoding
9386 Don't set Accept-Encoding: gzip. This means that rclone won't ask the
9387 server for compressed files automatically. Useful if you've set the
9388 server to return files with Content-Encoding: gzip but you uploaded
9389 compressed files.
9390
9391 There is no need to set this in normal operation, and doing so will de‐
9392 crease the network transfer efficiency of rclone.
9393
9394 --no-traverse
9395 The --no-traverse flag controls whether the destination file system is
9396 traversed when using the copy or move commands. --no-traverse is not
9397 compatible with sync and will be ignored if you supply it with sync.
9398
9399 If you are only copying a small number of files (or are filtering most
9400 of the files) and/or have a large number of files on the destination
9401 then --no-traverse will stop rclone listing the destination and save
9402 time.
9403
9404 However, if you are copying a large number of files, especially if you
9405 are doing a copy where lots of the files under consideration haven't
9406 changed and won't need copying then you shouldn't use --no-traverse.
9407
9408 See rclone copy (https://rclone.org/commands/rclone_copy/) for an exam‐
9409 ple of how to use it.
9410
9411 --no-unicode-normalization
9412 Don't normalize unicode characters in filenames during the sync rou‐
9413 tine.
9414
9415 Sometimes, an operating system will store filenames containing unicode
9416 parts in their decomposed form (particularly macOS). Some cloud stor‐
9417 age systems will then recompose the unicode, resulting in duplicate
9418 files if the data is ever copied back to a local filesystem.
9419
9420 Using this flag will disable that functionality, treating each unicode
9421 character as unique. For example, by default é and é will be normal‐
9422 ized into the same character. With --no-unicode-normalization they
9423 will be treated as unique characters.
9424
9425 --no-update-modtime
9426 When using this flag, rclone won't update modification times of remote
9427 files if they are incorrect as it would normally.
9428
9429 This can be used if the remote is being synced with another tool also
9430 (e.g. the Google Drive client).
9431
9432 --order-by string
9433 The --order-by flag controls the order in which files in the backlog
9434 are processed in rclone sync, rclone copy and rclone move.
9435
9436 The order by string is constructed like this. The first part describes
9437 what aspect is being measured:
9438
9439 • size - order by the size of the files
9440
9441 • name - order by the full path of the files
9442
9443 • modtime - order by the modification date of the files
9444
9445 This can have a modifier appended with a comma:
9446
9447 • ascending or asc - order so that the smallest (or oldest) is pro‐
9448 cessed first
9449
9450 • descending or desc - order so that the largest (or newest) is pro‐
9451 cessed first
9452
9453 • mixed - order so that the smallest is processed first for some
9454 threads and the largest for others
9455
9456 If the modifier is mixed then it can have an optional percentage (which
9457 defaults to 50), e.g. size,mixed,25 which means that 25% of the
9458 threads should be taking the smallest items and 75% the largest. The
9459 threads which take the smallest first will always take the smallest
9460 first and likewise the largest first threads. The mixed mode can be
9461 useful to minimise the transfer time when you are transferring a mix‐
9462 ture of large and small files - the large files are guaranteed upload
9463 threads and bandwidth and the small files will be processed continuous‐
9464 ly.
9465
9466 If no modifier is supplied then the order is ascending.
9467
9468 For example
9469
9470 • --order-by size,desc - send the largest files first
9471
9472 • --order-by modtime,ascending - send the oldest files first
9473
9474 • --order-by name - send the files with alphabetically by path first
9475
9476 If the --order-by flag is not supplied or it is supplied with an empty
9477 string then the default ordering will be used which is as scanned.
9478 With --checkers 1 this is mostly alphabetical, however with the default
9479 --checkers 8 it is somewhat random.
9480
9481 Limitations
9482 The --order-by flag does not do a separate pass over the data. This
9483 means that it may transfer some files out of the order specified if
9484
9485 • there are no files in the backlog or the source has not been fully
9486 scanned yet
9487
9488 • there are more than --max-backlog files in the backlog
9489
9490 Rclone will do its best to transfer the best file it has so in practice
9491 this should not cause a problem. Think of --order-by as being more of
9492 a best efforts flag rather than a perfect ordering.
9493
9494 If you want perfect ordering then you will need to specify --check-
9495 first which will find all the files which need transferring first be‐
9496 fore transferring any.
9497
9498 --password-command SpaceSepList
9499 This flag supplies a program which should supply the config password
9500 when run. This is an alternative to rclone prompting for the password
9501 or setting the RCLONE_CONFIG_PASS variable.
9502
9503 The argument to this should be a command with a space separated list of
9504 arguments. If one of the arguments has a space in then enclose it in
9505 ", if you want a literal " in an argument then enclose the argument in
9506 " and double the ". See CSV encoding (https://godoc.org/encoding/csv)
9507 for more info.
9508
9509 Eg
9510
9511 --password-command echo hello
9512 --password-command echo "hello with space"
9513 --password-command echo "hello with ""quotes"" and space"
9514
9515 See the Configuration Encryption for more info.
9516
9517 See a Windows PowerShell example on the Wiki
9518 (https://github.com/rclone/rclone/wiki/Windows-Powershell-use-rclone-
9519 password-command-for-Config-file-password).
9520
9521 -P, --progress
9522 This flag makes rclone update the stats in a static block in the termi‐
9523 nal providing a realtime overview of the transfer.
9524
9525 Any log messages will scroll above the static block. Log messages will
9526 push the static block down to the bottom of the terminal where it will
9527 stay.
9528
9529 Normally this is updated every 500mS but this period can be overridden
9530 with the --stats flag.
9531
9532 This can be used with the --stats-one-line flag for a simpler display.
9533
9534 Note: On Windows until this bug (https://github.com/Azure/go-an‐
9535 siterm/issues/26) is fixed all non-ASCII characters will be replaced
9536 with . when --progress is in use.
9537
9538 --progress-terminal-title
9539 This flag, when used with -P/--progress, will print the string ETA: %s
9540 to the terminal title.
9541
9542 -q, --quiet
9543 This flag will limit rclone's output to error messages only.
9544
9545 --refresh-times
9546 The --refresh-times flag can be used to update modification times of
9547 existing files when they are out of sync on backends which don't sup‐
9548 port hashes.
9549
9550 This is useful if you uploaded files with the incorrect timestamps and
9551 you now wish to correct them.
9552
9553 This flag is only useful for destinations which don't support hashes
9554 (e.g. crypt).
9555
9556 This can be used any of the sync commands sync, copy or move.
9557
9558 To use this flag you will need to be doing a modification time sync (so
9559 not using --size-only or --checksum). The flag will have no effect
9560 when using --size-only or --checksum.
9561
9562 If this flag is used when rclone comes to upload a file it will check
9563 to see if there is an existing file on the destination. If this file
9564 matches the source with size (and checksum if available) but has a dif‐
9565 fering timestamp then instead of re-uploading it, rclone will update
9566 the timestamp on the destination file. If the checksum does not match
9567 rclone will upload the new file. If the checksum is absent (e.g. on a
9568 crypt backend) then rclone will update the timestamp.
9569
9570 Note that some remotes can't set the modification time without re-up‐
9571 loading the file so this flag is less useful on them.
9572
9573 Normally if you are doing a modification time sync rclone will update
9574 modification times without --refresh-times provided that the remote
9575 supports checksums and the checksums match on the file. However if the
9576 checksums are absent then rclone will upload the file rather than set‐
9577 ting the timestamp as this is the safe behaviour.
9578
9579 --retries int
9580 Retry the entire sync if it fails this many times it fails (default 3).
9581
9582 Some remotes can be unreliable and a few retries help pick up the files
9583 which didn't get transferred because of errors.
9584
9585 Disable retries with --retries 1.
9586
9587 --retries-sleep=TIME
9588 This sets the interval between each retry specified by --retries
9589
9590 The default is 0. Use 0 to disable.
9591
9592 --server-side-across-configs
9593 Allow server-side operations (e.g. copy or move) to work across dif‐
9594 ferent configurations.
9595
9596 This can be useful if you wish to do a server-side copy or move between
9597 two remotes which use the same backend but are configured differently.
9598
9599 Note that this isn't enabled by default because it isn't easy for
9600 rclone to tell if it will work between any two configurations.
9601
9602 --size-only
9603 Normally rclone will look at modification time and size of files to see
9604 if they are equal. If you set this flag then rclone will check only
9605 the size.
9606
9607 This can be useful transferring files from Dropbox which have been mod‐
9608 ified by the desktop sync client which doesn't set checksums of modifi‐
9609 cation times in the same way as rclone.
9610
9611 --stats=TIME
9612 Commands which transfer data (sync, copy, copyto, move, moveto) will
9613 print data transfer stats at regular intervals to show their progress.
9614
9615 This sets the interval.
9616
9617 The default is 1m. Use 0 to disable.
9618
9619 If you set the stats interval then all commands can show stats. This
9620 can be useful when running other commands, check or mount for example.
9621
9622 Stats are logged at INFO level by default which means they won't show
9623 at default log level NOTICE. Use --stats-log-level NOTICE or -v to
9624 make them show. See the Logging section for more info on log levels.
9625
9626 Note that on macOS you can send a SIGINFO (which is normally ctrl-T in
9627 the terminal) to make the stats print immediately.
9628
9629 --stats-file-name-length integer
9630 By default, the --stats output will truncate file names and paths
9631 longer than 40 characters. This is equivalent to providing --stats-
9632 file-name-length 40. Use --stats-file-name-length 0 to disable any
9633 truncation of file names printed by stats.
9634
9635 --stats-log-level string
9636 Log level to show --stats output at. This can be DEBUG, INFO, NOTICE,
9637 or ERROR. The default is INFO. This means at the default level of
9638 logging which is NOTICE the stats won't show - if you want them to then
9639 use --stats-log-level NOTICE. See the Logging section for more info on
9640 log levels.
9641
9642 --stats-one-line
9643 When this is specified, rclone condenses the stats into a single line
9644 showing the most important stats only.
9645
9646 --stats-one-line-date
9647 When this is specified, rclone enables the single-line stats and
9648 prepends the display with a date string. The default is 2006/01/02
9649 15:04:05 -
9650
9651 --stats-one-line-date-format
9652 When this is specified, rclone enables the single-line stats and
9653 prepends the display with a user-supplied date string. The date string
9654 MUST be enclosed in quotes. Follow golang specs
9655 (https://golang.org/pkg/time/#Time.Format) for date formatting syntax.
9656
9657 --stats-unit=bits|bytes
9658 By default, data transfer rates will be printed in bytes per second.
9659
9660 This option allows the data rate to be printed in bits per second.
9661
9662 Data transfer volume will still be reported in bytes.
9663
9664 The rate is reported as a binary unit, not SI unit. So 1 Mbit/s equals
9665 1,048,576 bit/s and not 1,000,000 bit/s.
9666
9667 The default is bytes.
9668
9669 --suffix=SUFFIX
9670 When using sync, copy or move any files which would have been overwrit‐
9671 ten or deleted will have the suffix added to them. If there is a file
9672 with the same path (after the suffix has been added), then it will be
9673 overwritten.
9674
9675 The remote in use must support server-side move or copy and you must
9676 use the same remote as the destination of the sync.
9677
9678 This is for use with files to add the suffix in the current directory
9679 or with --backup-dir. See --backup-dir for more info.
9680
9681 For example
9682
9683 rclone copy -i /path/to/local/file remote:current --suffix .bak
9684
9685 will copy /path/to/local to remote:current, but for any files which
9686 would have been updated or deleted have .bak added.
9687
9688 If using rclone sync with --suffix and without --backup-dir then it is
9689 recommended to put a filter rule in excluding the suffix otherwise the
9690 sync will delete the backup files.
9691
9692 rclone sync -i /path/to/local/file remote:current --suffix .bak --exclude "*.bak"
9693
9694 --suffix-keep-extension
9695 When using --suffix, setting this causes rclone put the SUFFIX before
9696 the extension of the files that it backs up rather than after.
9697
9698 So let's say we had --suffix -2019-01-01, without the flag file.txt
9699 would be backed up to file.txt-2019-01-01 and with the flag it would be
9700 backed up to file-2019-01-01.txt. This can be helpful to make sure the
9701 suffixed files can still be opened.
9702
9703 --syslog
9704 On capable OSes (not Windows or Plan9) send all log output to syslog.
9705
9706 This can be useful for running rclone in a script or rclone mount.
9707
9708 --syslog-facility string
9709 If using --syslog this sets the syslog facility (e.g. KERN, USER).
9710 See man syslog for a list of possible facilities. The default facility
9711 is DAEMON.
9712
9713 --temp-dir=DIR
9714 Specify the directory rclone will use for temporary files, to override
9715 the default. Make sure the directory exists and have accessible per‐
9716 missions.
9717
9718 By default the operating system's temp directory will be used: - On
9719 Unix systems, $TMPDIR if non-empty, else /tmp. - On Windows, the first
9720 non-empty value from %TMP%, %TEMP%, %USERPROFILE%, or the Windows di‐
9721 rectory.
9722
9723 When overriding the default with this option, the specified path will
9724 be set as value of environment variable TMPDIR on Unix systems and TMP
9725 and TEMP on Windows.
9726
9727 You can use the config paths (https://rclone.org/commands/rclone_con‐
9728 fig_paths/) command to see the current value.
9729
9730 --tpslimit float
9731 Limit transactions per second to this number. Default is 0 which is
9732 used to mean unlimited transactions per second.
9733
9734 A transaction is roughly defined as an API call; its exact meaning will
9735 depend on the backend. For HTTP based backends it is an HTTP
9736 PUT/GET/POST/etc and its response. For FTP/SFTP it is a round trip
9737 transaction over TCP.
9738
9739 For example, to limit rclone to 10 transactions per second use --tp‐
9740 slimit 10, or to 1 transaction every 2 seconds use --tpslimit 0.5.
9741
9742 Use this when the number of transactions per second from rclone is
9743 causing a problem with the cloud storage provider (e.g. getting you
9744 banned or rate limited).
9745
9746 This can be very useful for rclone mount to control the behaviour of
9747 applications using it.
9748
9749 This limit applies to all HTTP based backends and to the FTP and SFTP
9750 backends. It does not apply to the local backend or the Storj backend.
9751
9752 See also --tpslimit-burst.
9753
9754 --tpslimit-burst int
9755 Max burst of transactions for --tpslimit (default 1).
9756
9757 Normally --tpslimit will do exactly the number of transaction per sec‐
9758 ond specified. However if you supply --tps-burst then rclone can save
9759 up some transactions from when it was idle giving a burst of up to the
9760 parameter supplied.
9761
9762 For example if you provide --tpslimit-burst 10 then if rclone has been
9763 idle for more than 10*--tpslimit then it can do 10 transactions very
9764 quickly before they are limited again.
9765
9766 This may be used to increase performance of --tpslimit without changing
9767 the long term average number of transactions per second.
9768
9769 --track-renames
9770 By default, rclone doesn't keep track of renamed files, so if you re‐
9771 name a file locally then sync it to a remote, rclone will delete the
9772 old file on the remote and upload a new copy.
9773
9774 An rclone sync with --track-renames runs like a normal sync, but keeps
9775 track of objects which exist in the destination but not in the source
9776 (which would normally be deleted), and which objects exist in the
9777 source but not the destination (which would normally be transferred).
9778 These objects are then candidates for renaming.
9779
9780 After the sync, rclone matches up the source only and destination only
9781 objects using the --track-renames-strategy specified and either renames
9782 the destination object or transfers the source and deletes the destina‐
9783 tion object. --track-renames is stateless like all of rclone's syncs.
9784
9785 To use this flag the destination must support server-side copy or serv‐
9786 er-side move, and to use a hash based --track-renames-strategy (the de‐
9787 fault) the source and the destination must have a compatible hash.
9788
9789 If the destination does not support server-side copy or move, rclone
9790 will fall back to the default behaviour and log an error level message
9791 to the console.
9792
9793 Encrypted destinations are not currently supported by --track-renames
9794 if --track-renames-strategy includes hash.
9795
9796 Note that --track-renames is incompatible with --no-traverse and that
9797 it uses extra memory to keep track of all the rename candidates.
9798
9799 Note also that --track-renames is incompatible with --delete-before and
9800 will select --delete-after instead of --delete-during.
9801
9802 --track-renames-strategy (hash,modtime,leaf,size)
9803 This option changes the file matching criteria for --track-renames.
9804
9805 The matching is controlled by a comma separated selection of these to‐
9806 kens:
9807
9808 • modtime - the modification time of the file - not supported on all
9809 backends
9810
9811 • hash - the hash of the file contents - not supported on all backends
9812
9813 • leaf - the name of the file not including its directory name
9814
9815 • size - the size of the file (this is always enabled)
9816
9817 The default option is hash.
9818
9819 Using --track-renames-strategy modtime,leaf would match files based on
9820 modification time, the leaf of the file name and the size only.
9821
9822 Using --track-renames-strategy modtime or leaf can enable --track-re‐
9823 names support for encrypted destinations.
9824
9825 Note that the hash strategy is not supported with encrypted destina‐
9826 tions.
9827
9828 --delete-(before,during,after)
9829 This option allows you to specify when files on your destination are
9830 deleted when you sync folders.
9831
9832 Specifying the value --delete-before will delete all files present on
9833 the destination, but not on the source before starting the transfer of
9834 any new or updated files. This uses two passes through the file sys‐
9835 tems, one for the deletions and one for the copies.
9836
9837 Specifying --delete-during will delete files while checking and upload‐
9838 ing files. This is the fastest option and uses the least memory.
9839
9840 Specifying --delete-after (the default value) will delay deletion of
9841 files until all new/updated files have been successfully transferred.
9842 The files to be deleted are collected in the copy pass then deleted af‐
9843 ter the copy pass has completed successfully. The files to be deleted
9844 are held in memory so this mode may use more memory. This is the
9845 safest mode as it will only delete files if there have been no errors
9846 subsequent to that. If there have been errors before the deletions
9847 start then you will get the message not deleting files as there were IO
9848 errors.
9849
9850 --fast-list
9851 When doing anything which involves a directory listing (e.g. sync,
9852 copy, ls - in fact nearly every command), rclone normally lists a di‐
9853 rectory and processes it before using more directory lists to process
9854 any subdirectories. This can be parallelised and works very quickly
9855 using the least amount of memory.
9856
9857 However, some remotes have a way of listing all files beneath a direc‐
9858 tory in one (or a small number) of transactions. These tend to be the
9859 bucket-based remotes (e.g. S3, B2, GCS, Swift).
9860
9861 If you use the --fast-list flag then rclone will use this method for
9862 listing directories. This will have the following consequences for the
9863 listing:
9864
9865 • It will use fewer transactions (important if you pay for them)
9866
9867 • It will use more memory. Rclone has to load the whole listing into
9868 memory.
9869
9870 • It may be faster because it uses fewer transactions
9871
9872 • It may be slower because it can't be parallelized
9873
9874 rclone should always give identical results with and without --fast-
9875 list.
9876
9877 If you pay for transactions and can fit your entire sync listing into
9878 memory then --fast-list is recommended. If you have a very big sync to
9879 do then don't use --fast-list otherwise you will run out of memory.
9880
9881 If you use --fast-list on a remote which doesn't support it, then
9882 rclone will just ignore it.
9883
9884 --timeout=TIME
9885 This sets the IO idle timeout. If a transfer has started but then be‐
9886 comes idle for this long it is considered broken and disconnected.
9887
9888 The default is 5m. Set to 0 to disable.
9889
9890 --transfers=N
9891 The number of file transfers to run in parallel. It can sometimes be
9892 useful to set this to a smaller number if the remote is giving a lot of
9893 timeouts or bigger if you have lots of bandwidth and a fast remote.
9894
9895 The default is to run 4 file transfers in parallel.
9896
9897 Look at --multi-thread-streams if you would like to control single file
9898 transfers.
9899
9900 -u, --update
9901 This forces rclone to skip any files which exist on the destination and
9902 have a modified time that is newer than the source file.
9903
9904 This can be useful in avoiding needless transfers when transferring to
9905 a remote which doesn't support modification times directly (or when us‐
9906 ing --use-server-modtime to avoid extra API calls) as it is more accu‐
9907 rate than a --size-only check and faster than using --checksum. On
9908 such remotes (or when using --use-server-modtime) the time checked will
9909 be the uploaded time.
9910
9911 If an existing destination file has a modification time older than the
9912 source file's, it will be updated if the sizes are different. If the
9913 sizes are the same, it will be updated if the checksum is different or
9914 not available.
9915
9916 If an existing destination file has a modification time equal (within
9917 the computed modify window) to the source file's, it will be updated if
9918 the sizes are different. The checksum will not be checked in this case
9919 unless the --checksum flag is provided.
9920
9921 In all other cases the file will not be updated.
9922
9923 Consider using the --modify-window flag to compensate for time skews
9924 between the source and the backend, for backends that do not support
9925 mod times, and instead use uploaded times. However, if the backend
9926 does not support checksums, note that syncing or copying within the
9927 time skew window may still result in additional transfers for safety.
9928
9929 --use-mmap
9930 If this flag is set then rclone will use anonymous memory allocated by
9931 mmap on Unix based platforms and VirtualAlloc on Windows for its trans‐
9932 fer buffers (size controlled by --buffer-size). Memory allocated like
9933 this does not go on the Go heap and can be returned to the OS immedi‐
9934 ately when it is finished with.
9935
9936 If this flag is not set then rclone will allocate and free the buffers
9937 using the Go memory allocator which may use more memory as memory pages
9938 are returned less aggressively to the OS.
9939
9940 It is possible this does not work well on all platforms so it is dis‐
9941 abled by default; in the future it may be enabled by default.
9942
9943 --use-server-modtime
9944 Some object-store backends (e.g, Swift, S3) do not preserve file modi‐
9945 fication times (modtime). On these backends, rclone stores the origi‐
9946 nal modtime as additional metadata on the object. By default it will
9947 make an API call to retrieve the metadata when the modtime is needed by
9948 an operation.
9949
9950 Use this flag to disable the extra API call and rely instead on the
9951 server's modified time. In cases such as a local to remote sync using
9952 --update, knowing the local file is newer than the time it was last up‐
9953 loaded to the remote is sufficient. In those cases, this flag can
9954 speed up the process and reduce the number of API calls necessary.
9955
9956 Using this flag on a sync operation without also using --update would
9957 cause all files modified at any time other than the last upload time to
9958 be uploaded again, which is probably not what you want.
9959
9960 -v, -vv, --verbose
9961 With -v rclone will tell you about each file that is transferred and a
9962 small number of significant events.
9963
9964 With -vv rclone will become very verbose telling you about every file
9965 it considers and transfers. Please send bug reports with a log with
9966 this setting.
9967
9968 When setting verbosity as an environment variable, use RCLONE_VERBOSE=1
9969 or RCLONE_VERBOSE=2 for -v and -vv respectively.
9970
9971 -V, --version
9972 Prints the version number
9973
9974 SSL/TLS options
9975 The outgoing SSL/TLS connections rclone makes can be controlled with
9976 these options. For example this can be very useful with the HTTP or
9977 WebDAV backends. Rclone HTTP servers have their own set of configura‐
9978 tion for SSL/TLS which you can find in their documentation.
9979
9980 --ca-cert string
9981 This loads the PEM encoded certificate authority certificate and uses
9982 it to verify the certificates of the servers rclone connects to.
9983
9984 If you have generated certificates signed with a local CA then you will
9985 need this flag to connect to servers using those certificates.
9986
9987 --client-cert string
9988 This loads the PEM encoded client side certificate.
9989
9990 This is used for mutual TLS authentication
9991 (https://en.wikipedia.org/wiki/Mutual_authentication).
9992
9993 The --client-key flag is required too when using this.
9994
9995 --client-key string
9996 This loads the PEM encoded client side private key used for mutual TLS
9997 authentication. Used in conjunction with --client-cert.
9998
9999 --no-check-certificate=true/false
10000 --no-check-certificate controls whether a client verifies the server's
10001 certificate chain and host name. If --no-check-certificate is true,
10002 TLS accepts any certificate presented by the server and any host name
10003 in that certificate. In this mode, TLS is susceptible to man-in-the-
10004 middle attacks.
10005
10006 This option defaults to false.
10007
10008 This should be used only for testing.
10009
10010 Configuration Encryption
10011 Your configuration file contains information for logging in to your
10012 cloud services. This means that you should keep your rclone.conf file
10013 in a secure location.
10014
10015 If you are in an environment where that isn't possible, you can add a
10016 password to your configuration. This means that you will have to sup‐
10017 ply the password every time you start rclone.
10018
10019 To add a password to your rclone configuration, execute rclone config.
10020
10021 >rclone config
10022 Current remotes:
10023
10024 e) Edit existing remote
10025 n) New remote
10026 d) Delete remote
10027 s) Set configuration password
10028 q) Quit config
10029 e/n/d/s/q>
10030
10031 Go into s, Set configuration password:
10032
10033 e/n/d/s/q> s
10034 Your configuration is not encrypted.
10035 If you add a password, you will protect your login information to cloud services.
10036 a) Add Password
10037 q) Quit to main menu
10038 a/q> a
10039 Enter NEW configuration password:
10040 password:
10041 Confirm NEW password:
10042 password:
10043 Password set
10044 Your configuration is encrypted.
10045 c) Change Password
10046 u) Unencrypt configuration
10047 q) Quit to main menu
10048 c/u/q>
10049
10050 Your configuration is now encrypted, and every time you start rclone
10051 you will have to supply the password. See below for details. In the
10052 same menu, you can change the password or completely remove encryption
10053 from your configuration.
10054
10055 There is no way to recover the configuration if you lose your password.
10056
10057 rclone uses nacl secretbox (https://godoc.org/golang.org/x/crypto/na‐
10058 cl/secretbox) which in turn uses XSalsa20 and Poly1305 to encrypt and
10059 authenticate your configuration with secret-key cryptography. The
10060 password is SHA-256 hashed, which produces the key for secretbox. The
10061 hashed password is not stored.
10062
10063 While this provides very good security, we do not recommend storing
10064 your encrypted rclone configuration in public if it contains sensitive
10065 information, maybe except if you use a very strong password.
10066
10067 If it is safe in your environment, you can set the RCLONE_CONFIG_PASS
10068 environment variable to contain your password, in which case it will be
10069 used for decrypting the configuration.
10070
10071 You can set this for a session from a script. For unix like systems
10072 save this to a file called set-rclone-password:
10073
10074 #!/bin/echo Source this file don't run it
10075
10076 read -s RCLONE_CONFIG_PASS
10077 export RCLONE_CONFIG_PASS
10078
10079 Then source the file when you want to use it. From the shell you would
10080 do source set-rclone-password. It will then ask you for the password
10081 and set it in the environment variable.
10082
10083 An alternate means of supplying the password is to provide a script
10084 which will retrieve the password and print on standard output. This
10085 script should have a fully specified path name and not rely on any en‐
10086 vironment variables. The script is supplied either via --password-com‐
10087 mand="..." command line argument or via the RCLONE_PASSWORD_COMMAND en‐
10088 vironment variable.
10089
10090 One useful example of this is using the passwordstore application to
10091 retrieve the password:
10092
10093 export RCLONE_PASSWORD_COMMAND="pass rclone/config"
10094
10095 If the passwordstore password manager holds the password for the rclone
10096 configuration, using the script method means the password is primarily
10097 protected by the passwordstore system, and is never embedded in the
10098 clear in scripts, nor available for examination using the standard com‐
10099 mands available. It is quite possible with long running rclone ses‐
10100 sions for copies of passwords to be innocently captured in log files or
10101 terminal scroll buffers, etc. Using the script method of supplying the
10102 password enhances the security of the config password considerably.
10103
10104 If you are running rclone inside a script, unless you are using the
10105 --password-command method, you might want to disable password prompts.
10106 To do that, pass the parameter --ask-password=false to rclone. This
10107 will make rclone fail instead of asking for a password if RCLONE_CON‐
10108 FIG_PASS doesn't contain a valid password, and --password-command has
10109 not been supplied.
10110
10111 Whenever running commands that may be affected by options in a configu‐
10112 ration file, rclone will look for an existing file according to the
10113 rules described above, and load any it finds. If an encrypted file is
10114 found, this includes decrypting it, with the possible consequence of a
10115 password prompt. When executing a command line that you know are not
10116 actually using anything from such a configuration file, you can avoid
10117 it being loaded by overriding the location, e.g. with one of the docu‐
10118 mented special values for memory-only configuration. Since only back‐
10119 end options can be stored in configuration files, this is normally un‐
10120 necessary for commands that do not operate on backends, e.g. genauto‐
10121 complete. However, it will be relevant for commands that do operate on
10122 backends in general, but are used without referencing a stored remote,
10123 e.g. listing local filesystem paths, or connection strings: rclone
10124 --config="" ls .
10125
10126 Developer options
10127 These options are useful when developing or debugging rclone. There
10128 are also some more remote specific options which aren't documented here
10129 which are used for testing. These start with remote name e.g.
10130 --drive-test-option - see the docs for the remote in question.
10131
10132 --cpuprofile=FILE
10133 Write CPU profile to file. This can be analysed with go tool pprof.
10134
10135 --dump flag,flag,flag
10136 The --dump flag takes a comma separated list of flags to dump info
10137 about.
10138
10139 Note that some headers including Accept-Encoding as shown may not be
10140 correct in the request and the response may not show Content-Encoding
10141 if the go standard libraries auto gzip encoding was in effect. In this
10142 case the body of the request will be gunzipped before showing it.
10143
10144 The available flags are:
10145
10146 --dump headers
10147 Dump HTTP headers with Authorization: lines removed. May still contain
10148 sensitive info. Can be very verbose. Useful for debugging only.
10149
10150 Use --dump auth if you do want the Authorization: headers.
10151
10152 --dump bodies
10153 Dump HTTP headers and bodies - may contain sensitive info. Can be very
10154 verbose. Useful for debugging only.
10155
10156 Note that the bodies are buffered in memory so don't use this for enor‐
10157 mous files.
10158
10159 --dump requests
10160 Like --dump bodies but dumps the request bodies and the response head‐
10161 ers. Useful for debugging download problems.
10162
10163 --dump responses
10164 Like --dump bodies but dumps the response bodies and the request head‐
10165 ers. Useful for debugging upload problems.
10166
10167 --dump auth
10168 Dump HTTP headers - will contain sensitive info such as Authorization:
10169 headers - use --dump headers to dump without Authorization: headers.
10170 Can be very verbose. Useful for debugging only.
10171
10172 --dump filters
10173 Dump the filters to the output. Useful to see exactly what include and
10174 exclude options are filtering on.
10175
10176 --dump goroutines
10177 This dumps a list of the running go-routines at the end of the command
10178 to standard output.
10179
10180 --dump openfiles
10181 This dumps a list of the open files at the end of the command. It uses
10182 the lsof command to do that so you'll need that installed to use it.
10183
10184 --memprofile=FILE
10185 Write memory profile to file. This can be analysed with go tool pprof.
10186
10187 Filtering
10188 For the filtering options
10189
10190 • --delete-excluded
10191
10192 • --filter
10193
10194 • --filter-from
10195
10196 • --exclude
10197
10198 • --exclude-from
10199
10200 • --exclude-if-present
10201
10202 • --include
10203
10204 • --include-from
10205
10206 • --files-from
10207
10208 • --files-from-raw
10209
10210 • --min-size
10211
10212 • --max-size
10213
10214 • --min-age
10215
10216 • --max-age
10217
10218 • --dump filters
10219
10220 See the filtering section (https://rclone.org/filtering/).
10221
10222 Remote control
10223 For the remote control options and for instructions on how to remote
10224 control rclone
10225
10226 • --rc
10227
10228 • and anything starting with --rc-
10229
10230 See the remote control section (https://rclone.org/rc/).
10231
10232 Logging
10233 rclone has 4 levels of logging, ERROR, NOTICE, INFO and DEBUG.
10234
10235 By default, rclone logs to standard error. This means you can redirect
10236 standard error and still see the normal output of rclone commands (e.g.
10237 rclone ls).
10238
10239 By default, rclone will produce Error and Notice level messages.
10240
10241 If you use the -q flag, rclone will only produce Error messages.
10242
10243 If you use the -v flag, rclone will produce Error, Notice and Info mes‐
10244 sages.
10245
10246 If you use the -vv flag, rclone will produce Error, Notice, Info and
10247 Debug messages.
10248
10249 You can also control the log levels with the --log-level flag.
10250
10251 If you use the --log-file=FILE option, rclone will redirect Error, Info
10252 and Debug messages along with standard error to FILE.
10253
10254 If you use the --syslog flag then rclone will log to syslog and the
10255 --syslog-facility control which facility it uses.
10256
10257 Rclone prefixes all log messages with their level in capitals, e.g.
10258 INFO which makes it easy to grep the log file for different kinds of
10259 information.
10260
10261 Exit Code
10262 If any errors occur during the command execution, rclone will exit with
10263 a non-zero exit code. This allows scripts to detect when rclone opera‐
10264 tions have failed.
10265
10266 During the startup phase, rclone will exit immediately if an error is
10267 detected in the configuration. There will always be a log message im‐
10268 mediately before exiting.
10269
10270 When rclone is running it will accumulate errors as it goes along, and
10271 only exit with a non-zero exit code if (after retries) there were still
10272 failed transfers. For every error counted there will be a high priori‐
10273 ty log message (visible with -q) showing the message and which file
10274 caused the problem. A high priority message is also shown when start‐
10275 ing a retry so the user can see that any previous error messages may
10276 not be valid after the retry. If rclone has done a retry it will log a
10277 high priority message if the retry was successful.
10278
10279 List of exit codes
10280 • 0 - success
10281
10282 • 1 - Syntax or usage error
10283
10284 • 2 - Error not otherwise categorised
10285
10286 • 3 - Directory not found
10287
10288 • 4 - File not found
10289
10290 • 5 - Temporary error (one that more retries might fix) (Retry errors)
10291
10292 • 6 - Less serious errors (like 461 errors from dropbox) (NoRetry er‐
10293 rors)
10294
10295 • 7 - Fatal error (one that more retries won't fix, like account sus‐
10296 pended) (Fatal errors)
10297
10298 • 8 - Transfer exceeded - limit set by --max-transfer reached
10299
10300 • 9 - Operation successful, but no files transferred
10301
10302 Environment Variables
10303 Rclone can be configured entirely using environment variables. These
10304 can be used to set defaults for options or config file entries.
10305
10306 Options
10307 Every option in rclone can have its default set by environment vari‐
10308 able.
10309
10310 To find the name of the environment variable, first, take the long op‐
10311 tion name, strip the leading --, change - to _, make upper case and
10312 prepend RCLONE_.
10313
10314 For example, to always set --stats 5s, set the environment variable
10315 RCLONE_STATS=5s. If you set stats on the command line this will over‐
10316 ride the environment variable setting.
10317
10318 Or to always use the trash in drive --drive-use-trash, set
10319 RCLONE_DRIVE_USE_TRASH=true.
10320
10321 Verbosity is slightly different, the environment variable equivalent of
10322 --verbose or -v is RCLONE_VERBOSE=1, or for -vv, RCLONE_VERBOSE=2.
10323
10324 The same parser is used for the options and the environment variables
10325 so they take exactly the same form.
10326
10327 The options set by environment variables can be seen with the -vv flag,
10328 e.g. rclone version -vv.
10329
10330 Config file
10331 You can set defaults for values in the config file on an individual re‐
10332 mote basis. The names of the config items are documented in the page
10333 for each backend.
10334
10335 To find the name of the environment variable, you need to set, take
10336 RCLONE_CONFIG_ + name of remote + _ + name of config file option and
10337 make it all uppercase.
10338
10339 For example, to configure an S3 remote named mys3: without a config
10340 file (using unix ways of setting environment variables):
10341
10342 $ export RCLONE_CONFIG_MYS3_TYPE=s3
10343 $ export RCLONE_CONFIG_MYS3_ACCESS_KEY_ID=XXX
10344 $ export RCLONE_CONFIG_MYS3_SECRET_ACCESS_KEY=XXX
10345 $ rclone lsd mys3:
10346 -1 2016-09-21 12:54:21 -1 my-bucket
10347 $ rclone listremotes | grep mys3
10348 mys3:
10349
10350 Note that if you want to create a remote using environment variables
10351 you must create the ..._TYPE variable as above.
10352
10353 Note that the name of a remote created using environment variable is
10354 case insensitive, in contrast to regular remotes stored in config file
10355 as documented above. You must write the name in uppercase in the envi‐
10356 ronment variable, but as seen from example above it will be listed and
10357 can be accessed in lowercase, while you can also refer to the same re‐
10358 mote in uppercase:
10359
10360 $ rclone lsd mys3:
10361 -1 2016-09-21 12:54:21 -1 my-bucket
10362 $ rclone lsd MYS3:
10363 -1 2016-09-21 12:54:21 -1 my-bucket
10364
10365 Note that you can only set the options of the immediate backend, so
10366 RCLONE_CONFIG_MYS3CRYPT_ACCESS_KEY_ID has no effect, if myS3Crypt is a
10367 crypt remote based on an S3 remote. However RCLONE_S3_ACCESS_KEY_ID
10368 will set the access key of all remotes using S3, including myS3Crypt.
10369
10370 Note also that now rclone has connection strings, it is probably easier
10371 to use those instead which makes the above example
10372
10373 rclone lsd :s3,access_key_id=XXX,secret_access_key=XXX:
10374
10375 Precedence
10376 The various different methods of backend configuration are read in this
10377 order and the first one with a value is used.
10378
10379 • Parameters in connection strings, e.g. myRemote,skip_links:
10380
10381 • Flag values as supplied on the command line, e.g. --skip-links
10382
10383 • Remote specific environment vars, e.g. RCLONE_CONFIG_MYRE‐
10384 MOTE_SKIP_LINKS (see above).
10385
10386 • Backend-specific environment vars, e.g. RCLONE_LOCAL_SKIP_LINKS.
10387
10388 • Backend generic environment vars, e.g. RCLONE_SKIP_LINKS.
10389
10390 • Config file, e.g. skip_links = true.
10391
10392 • Default values, e.g. false - these can't be changed.
10393
10394 So if both --skip-links is supplied on the command line and an environ‐
10395 ment variable RCLONE_LOCAL_SKIP_LINKS is set, the command line flag
10396 will take preference.
10397
10398 The backend configurations set by environment variables can be seen
10399 with the -vv flag, e.g. rclone about myRemote: -vv.
10400
10401 For non backend configuration the order is as follows:
10402
10403 • Flag values as supplied on the command line, e.g. --stats 5s.
10404
10405 • Environment vars, e.g. RCLONE_STATS=5s.
10406
10407 • Default values, e.g. 1m - these can't be changed.
10408
10409 Other environment variables
10410 • RCLONE_CONFIG_PASS set to contain your config file password (see Con‐
10411 figuration Encryption section)
10412
10413 • HTTP_PROXY, HTTPS_PROXY and NO_PROXY (or the lowercase versions
10414 thereof).
10415
10416 • HTTPS_PROXY takes precedence over HTTP_PROXY for https requests.
10417
10418 • The environment values may be either a complete URL or a
10419 "host[:port]" for, in which case the "http" scheme is assumed.
10420
10421 • USER and LOGNAME values are used as fallbacks for current username.
10422 The primary method for looking up username is OS-specific: Windows
10423 API on Windows, real user ID in /etc/passwd on Unix systems. In the
10424 documentation the current username is simply referred to as $USER.
10425
10426 • RCLONE_CONFIG_DIR - rclone sets this variable for use in config files
10427 and sub processes to point to the directory holding the config file.
10428
10429 The options set by environment variables can be seen with the -vv and
10430 --log-level=DEBUG flags, e.g. rclone version -vv.
10431
10433 Some of the configurations (those involving oauth2) require an Internet
10434 connected web browser.
10435
10436 If you are trying to set rclone up on a remote or headless box with no
10437 browser available on it (e.g. a NAS or a server in a datacenter) then
10438 you will need to use an alternative means of configuration. There are
10439 two ways of doing it, described below.
10440
10441 Configuring using rclone authorize
10442 On the headless box run rclone config but answer N to the Use auto con‐
10443 fig? question.
10444
10445 ...
10446 Remote config
10447 Use auto config?
10448 * Say Y if not sure
10449 * Say N if you are working on a remote or headless machine
10450 y) Yes (default)
10451 n) No
10452 y/n> n
10453 For this to work, you will need rclone available on a machine that has
10454 a web browser available.
10455
10456 For more help and alternate methods see: https://rclone.org/remote_setup/
10457
10458 Execute the following on the machine with the web browser (same rclone
10459 version recommended):
10460
10461 rclone authorize "amazon cloud drive"
10462
10463 Then paste the result below:
10464 result>
10465
10466 Then on your main desktop machine
10467
10468 rclone authorize "amazon cloud drive"
10469 If your browser doesn't open automatically go to the following link: http://127.0.0.1:53682/auth
10470 Log in and authorize rclone for access
10471 Waiting for code...
10472 Got code
10473 Paste the following into your remote machine --->
10474 SECRET_TOKEN
10475 <---End paste
10476
10477 Then back to the headless box, paste in the code
10478
10479 result> SECRET_TOKEN
10480 --------------------
10481 [acd12]
10482 client_id =
10483 client_secret =
10484 token = SECRET_TOKEN
10485 --------------------
10486 y) Yes this is OK
10487 e) Edit this remote
10488 d) Delete this remote
10489 y/e/d>
10490
10491 Configuring by copying the config file
10492 Rclone stores all of its config in a single configuration file. This
10493 can easily be copied to configure a remote rclone.
10494
10495 So first configure rclone on your desktop machine with
10496
10497 rclone config
10498
10499 to set up the config file.
10500
10501 Find the config file by running rclone config file, for example
10502
10503 $ rclone config file
10504 Configuration file is stored at:
10505 /home/user/.rclone.conf
10506
10507 Now transfer it to the remote box (scp, cut paste, ftp, sftp, etc.) and
10508 place it in the correct place (use rclone config file on the remote box
10509 to find out where).
10510
10511 Configuring using SSH Tunnel
10512 Linux and MacOS users can utilize SSH Tunnel to redirect the headless
10513 box port 53682 to local machine by using the following command:
10514
10515 ssh -L localhost:53682:localhost:53682 username@remote_server
10516
10517 Then on the headless box run rclone config and answer Y to the Use auto
10518 config? question.
10519
10520 ...
10521 Remote config
10522 Use auto config?
10523 * Say Y if not sure
10524 * Say N if you are working on a remote or headless machine
10525 y) Yes (default)
10526 n) No
10527 y/n> y
10528
10529 Then copy and paste the auth url
10530 http://127.0.0.1:53682/auth?state=xxxxxxxxxxxx to the browser on your
10531 local machine, complete the auth and it is done.
10532
10534 Filter flags determine which files rclone sync, move, ls, lsl, md5sum,
10535 sha1sum, size, delete, check and similar commands apply to.
10536
10537 They are specified in terms of path/file name patterns; path/file
10538 lists; file age and size, or presence of a file in a directory. Bucket
10539 based remotes without the concept of directory apply filters to object
10540 key, age and size in an analogous way.
10541
10542 Rclone purge does not obey filters.
10543
10544 To test filters without risk of damage to data, apply them to rclone
10545 ls, or with the --dry-run and -vv flags.
10546
10547 Rclone filter patterns can only be used in filter command line options,
10548 not in the specification of a remote.
10549
10550 E.g. rclone copy "remote:dir*.jpg" /path/to/dir does not have a filter
10551 effect. rclone copy remote:dir /path/to/dir --include "*.jpg" does.
10552
10553 Important Avoid mixing any two of --include..., --exclude... or --fil‐
10554 ter... flags in an rclone command. The results may not be what you ex‐
10555 pect. Instead use a --filter... flag.
10556
10557 Patterns for matching path/file names
10558 Pattern syntax
10559 Here is a formal definition of the pattern syntax, examples are below.
10560
10561 Rclone matching rules follow a glob style:
10562
10563 * matches any sequence of non-separator (/) characters
10564 ** matches any sequence of characters including / separators
10565 ? matches any single non-separator (/) character
10566 [ [ ! ] { character-range } ]
10567 character class (must be non-empty)
10568 { pattern-list }
10569 pattern alternatives
10570 {{ regexp }}
10571 regular expression to match
10572 c matches character c (c != *, **, ?, \, [, {, })
10573 \c matches reserved character c (c = *, **, ?, \, [, {, }) or character class
10574
10575 character-range:
10576
10577 c matches character c (c != \, -, ])
10578 \c matches reserved character c (c = \, -, ])
10579 lo - hi matches character c for lo <= c <= hi
10580
10581 pattern-list:
10582
10583 pattern { , pattern }
10584 comma-separated (without spaces) patterns
10585
10586 character classes (see Go regular expression reference
10587 (https://golang.org/pkg/regexp/syntax/)) include:
10588
10589 Named character classes (e.g. [\d], [^\d], [\D], [^\D])
10590 Perl character classes (e.g. \s, \S, \w, \W)
10591 ASCII character classes (e.g. [[:alnum:]], [[:alpha:]], [[:punct:]], [[:xdigit:]])
10592
10593 regexp for advanced users to insert a regular expression - see below
10594 for more info:
10595
10596 Any re2 regular expression not containing `}}`
10597
10598 If the filter pattern starts with a / then it only matches at the top
10599 level of the directory tree, relative to the root of the remote (not
10600 necessarily the root of the drive). If it does not start with / then
10601 it is matched starting at the end of the path/file name but it only
10602 matches a complete path element - it must match from a / separator or
10603 the beginning of the path/file.
10604
10605 file.jpg - matches "file.jpg"
10606 - matches "directory/file.jpg"
10607 - doesn't match "afile.jpg"
10608 - doesn't match "directory/afile.jpg"
10609 /file.jpg - matches "file.jpg" in the root directory of the remote
10610 - doesn't match "afile.jpg"
10611 - doesn't match "directory/file.jpg"
10612
10613 The top level of the remote may not be the top level of the drive.
10614
10615 E.g. for a Microsoft Windows local directory structure
10616
10617 F:
10618 ├── bkp
10619 ├── data
10620 │ ├── excl
10621 │ │ ├── 123.jpg
10622 │ │ └── 456.jpg
10623 │ ├── incl
10624 │ │ └── document.pdf
10625
10626 To copy the contents of folder data into folder bkp excluding the con‐
10627 tents of subfolder exclthe following command treats F:\data and F:\bkp
10628 as top level for filtering.
10629
10630 rclone copy F:\data\ F:\bkp\ --exclude=/excl/**
10631
10632 Important Use / in path/file name patterns and not \ even if running on
10633 Microsoft Windows.
10634
10635 Simple patterns are case sensitive unless the --ignore-case flag is
10636 used.
10637
10638 Without --ignore-case (default)
10639
10640 potato - matches "potato"
10641 - doesn't match "POTATO"
10642
10643 With --ignore-case
10644
10645 potato - matches "potato"
10646 - matches "POTATO"
10647
10648 Using regular expressions in filter patterns
10649 The syntax of filter patterns is glob style matching (like bash uses)
10650 to make things easy for users. However this does not provide absolute
10651 control over the matching, so for advanced users rclone also provides a
10652 regular expression syntax.
10653
10654 The regular expressions used are as defined in the Go regular expres‐
10655 sion reference (https://golang.org/pkg/regexp/syntax/). Regular ex‐
10656 pressions should be enclosed in {{ }}. They will match only the last
10657 path segment if the glob doesn't start with / or the whole path name if
10658 it does. Note that rclone does not attempt to parse the supplied regu‐
10659 lar expression, meaning that using any regular expression filter will
10660 prevent rclone from using directory filter rules, as it will instead
10661 check every path against the supplied regular expression(s).
10662
10663 Here is how the {{regexp}} is transformed into an full regular expres‐
10664 sion to match the entire path:
10665
10666 {{regexp}} becomes (^|/)(regexp)$
10667 /{{regexp}} becomes ^(regexp)$
10668
10669 Regexp syntax can be mixed with glob syntax, for example
10670
10671 *.{{jpe?g}} to match file.jpg, file.jpeg but not file.png
10672
10673 You can also use regexp flags - to set case insensitive, for example
10674
10675 *.{{(?i)jpg}} to match file.jpg, file.JPG but not file.png
10676
10677 Be careful with wildcards in regular expressions - you don't want them
10678 to match path separators normally. To match any file name starting
10679 with start and ending with end write
10680
10681 {{start[^/]*end\.jpg}}
10682
10683 Not
10684
10685 {{start.*end\.jpg}}
10686
10687 Which will match a directory called start with a file called end.jpg in
10688 it as the .* will match / characters.
10689
10690 Note that you can use -vv --dump filters to show the filter patterns in
10691 regexp format - rclone implements the glob patters by transforming them
10692 into regular expressions.
10693
10694 Filter pattern examples
10695 Description Pattern Matches Does not match
10696 ─────────────────────────────────────────────────────────────────────
10697 Wildcard *.jpg /file.jpg /file.png
10698 /dir/file.jpg /dir/file.png
10699 Rooted /*.jpg /file.jpg /file.png
10700 /file2.jpg /dir/file.jpg
10701 Alternates *.{jpg,png} /file.jpg /file.gif
10702 /dir/file.png /dir/file.gif
10703 Path Wildcard dir/** /dir/anyfile file.png
10704 /subdir/dir/sub‐ /subdir/file.png
10705 subdir/anyfile
10706 Any Char *.t?t /file.txt /file.qxt
10707 /dir/file.tzt /dir/file.png
10708 Range *.[a-z] /file.a /file.0
10709 /dir/file.b /dir/file.1
10710 Escape *.\?\?\? /file.??? /file.abc
10711 /dir/file.??? /dir/file.def
10712 Class *.\d\d\d /file.012 /file.abc
10713 /dir/file.345 /dir/file.def
10714 Regexp *.{{jpe?g}} /file.jpeg /file.png
10715 /dir/file.jpg /dir/file.jpeeg
10716 Rooted Regexp /{{.*\.jpe?g}} /file.jpeg /file.png
10717 /file.jpg /dir/file.jpg
10718
10719 How filter rules are applied to files
10720 Rclone path/file name filters are made up of one or more of the follow‐
10721 ing flags:
10722
10723 • --include
10724
10725 • --include-from
10726
10727 • --exclude
10728
10729 • --exclude-from
10730
10731 • --filter
10732
10733 • --filter-from
10734
10735 There can be more than one instance of individual flags.
10736
10737 Rclone internally uses a combined list of all the include and exclude
10738 rules. The order in which rules are processed can influence the result
10739 of the filter.
10740
10741 All flags of the same type are processed together in the order above,
10742 regardless of what order the different types of flags are included on
10743 the command line.
10744
10745 Multiple instances of the same flag are processed from left to right
10746 according to their position in the command line.
10747
10748 To mix up the order of processing includes and excludes use --filter...
10749 flags.
10750
10751 Within --include-from, --exclude-from and --filter-from flags rules are
10752 processed from top to bottom of the referenced file.
10753
10754 If there is an --include or --include-from flag specified, rclone im‐
10755 plies a - ** rule which it adds to the bottom of the internal rule
10756 list. Specifying a + rule with a --filter... flag does not imply that
10757 rule.
10758
10759 Each path/file name passed through rclone is matched against the com‐
10760 bined filter list. At first match to a rule the path/file name is in‐
10761 cluded or excluded and no further filter rules are processed for that
10762 path/file.
10763
10764 If rclone does not find a match, after testing against all rules (in‐
10765 cluding the implied rule if appropriate), the path/file name is includ‐
10766 ed.
10767
10768 Any path/file included at that stage is processed by the rclone com‐
10769 mand.
10770
10771 --files-from and --files-from-raw flags over-ride and cannot be com‐
10772 bined with other filter options.
10773
10774 To see the internal combined rule list, in regular expression form, for
10775 a command add the --dump filters flag. Running an rclone command with
10776 --dump filters and -vv flags lists the internal filter elements and
10777 shows how they are applied to each source path/file. There is not cur‐
10778 rently a means provided to pass regular expression filter options into
10779 rclone directly though character class filter rules contain character
10780 classes. Go regular expression reference (https://golang.org/pkg/reg‐
10781 exp/syntax/)
10782
10783 How filter rules are applied to directories
10784 Rclone commands are applied to path/file names not directories. The
10785 entire contents of a directory can be matched to a filter by the pat‐
10786 tern directory/* or recursively by directory/**.
10787
10788 Directory filter rules are defined with a closing / separator.
10789
10790 E.g. /directory/subdirectory/ is an rclone directory filter rule.
10791
10792 Rclone commands can use directory filter rules to determine whether
10793 they recurse into subdirectories. This potentially optimises access to
10794 a remote by avoiding listing unnecessary directories. Whether optimi‐
10795 sation is desirable depends on the specific filter rules and source re‐
10796 mote content.
10797
10798 If any regular expression filters are in use, then no directory recur‐
10799 sion optimisation is possible, as rclone must check every path against
10800 the supplied regular expression(s).
10801
10802 Directory recursion optimisation occurs if either:
10803
10804 • A source remote does not support the rclone ListR primitive. local,
10805 sftp, Microsoft OneDrive and WebDAV do not support ListR. Google
10806 Drive and most bucket type storage do. Full list
10807 (https://rclone.org/overview/#optional-features)
10808
10809 • On other remotes (those that support ListR), if the rclone command is
10810 not naturally recursive, and provided it is not run with the --fast-
10811 list flag. ls, lsf -R and size are naturally recursive but sync,
10812 copy and move are not.
10813
10814 • Whenever the --disable ListR flag is applied to an rclone command.
10815
10816 Rclone commands imply directory filter rules from path/file filter
10817 rules. To view the directory filter rules rclone has implied for a
10818 command specify the --dump filters flag.
10819
10820 E.g. for an include rule
10821
10822 /a/*.jpg
10823
10824 Rclone implies the directory include rule
10825
10826 /a/
10827
10828 Directory filter rules specified in an rclone command can limit the
10829 scope of an rclone command but path/file filters still have to be spec‐
10830 ified.
10831
10832 E.g. rclone ls remote: --include /directory/ will not match any files.
10833 Because it is an --include option the --exclude ** rule is implied, and
10834 the /directory/ pattern serves only to optimise access to the remote by
10835 ignoring everything outside of that directory.
10836
10837 E.g. rclone ls remote: --filter-from filter-list.txt with a file fil‐
10838 ter-list.txt:
10839
10840 - /dir1/
10841 - /dir2/
10842 + *.pdf
10843 - **
10844
10845 All files in directories dir1 or dir2 or their subdirectories are com‐
10846 pletely excluded from the listing. Only files of suffix pdf in the
10847 root of remote: or its subdirectories are listed. The - ** rule pre‐
10848 vents listing of any path/files not previously matched by the rules
10849 above.
10850
10851 Option exclude-if-present creates a directory exclude rule based on the
10852 presence of a file in a directory and takes precedence over other
10853 rclone directory filter rules.
10854
10855 When using pattern list syntax, if a pattern item contains either / or
10856 **, then rclone will not able to imply a directory filter rule from
10857 this pattern list.
10858
10859 E.g. for an include rule
10860
10861 {dir1/**,dir2/**}
10862
10863 Rclone will match files below directories dir1 or dir2 only, but will
10864 not be able to use this filter to exclude a directory dir3 from being
10865 traversed.
10866
10867 Directory recursion optimisation may affect performance, but normally
10868 not the result. One exception to this is sync operations with option
10869 --create-empty-src-dirs, where any traversed empty directories will be
10870 created. With the pattern list example {dir1/**,dir2/**} above, this
10871 would create an empty directory dir3 on destination (when it exists on
10872 source). Changing the filter to {dir1,dir2}/**, or splitting it into
10873 two include rules --include dir1/** --include dir2/**, will match the
10874 same files while also filtering directories, with the result that an
10875 empty directory dir3 will no longer be created.
10876
10877 --exclude - Exclude files matching pattern
10878 Excludes path/file names from an rclone command based on a single ex‐
10879 clude rule.
10880
10881 This flag can be repeated. See above for the order filter flags are
10882 processed in.
10883
10884 --exclude should not be used with --include, --include-from, --filter
10885 or --filter-from flags.
10886
10887 --exclude has no effect when combined with --files-from or --files-
10888 from-raw flags.
10889
10890 E.g. rclone ls remote: --exclude *.bak excludes all .bak files from
10891 listing.
10892
10893 E.g. rclone size remote: "--exclude /dir/**" returns the total size of
10894 all files on remote: excluding those in root directory dir and sub di‐
10895 rectories.
10896
10897 E.g. on Microsoft Windows rclone ls remote: --exclude
10898 "*\[{JP,KR,HK}\]*" lists the files in remote: with [JP] or [KR] or [HK]
10899 in their name. Quotes prevent the shell from interpreting the \ char‐
10900 acters.\ characters escape the [ and ] so an rclone filter treats them
10901 literally rather than as a character-range. The { and } define an
10902 rclone pattern list. For other operating systems single quotes are re‐
10903 quired ie rclone ls remote: --exclude '*\[{JP,KR,HK}\]*'
10904
10905 --exclude-from - Read exclude patterns from file
10906 Excludes path/file names from an rclone command based on rules in a
10907 named file. The file contains a list of remarks and pattern rules.
10908
10909 For an example exclude-file.txt:
10910
10911 # a sample exclude rule file
10912 *.bak
10913 file2.jpg
10914
10915 rclone ls remote: --exclude-from exclude-file.txt lists the files on
10916 remote: except those named file2.jpg or with a suffix .bak. That is
10917 equivalent to rclone ls remote: --exclude file2.jpg --exclude "*.bak".
10918
10919 This flag can be repeated. See above for the order filter flags are
10920 processed in.
10921
10922 The --exclude-from flag is useful where multiple exclude filter rules
10923 are applied to an rclone command.
10924
10925 --exclude-from should not be used with --include, --include-from,
10926 --filter or --filter-from flags.
10927
10928 --exclude-from has no effect when combined with --files-from or
10929 --files-from-raw flags.
10930
10931 --exclude-from followed by - reads filter rules from standard input.
10932
10933 --include - Include files matching pattern
10934 Adds a single include rule based on path/file names to an rclone com‐
10935 mand.
10936
10937 This flag can be repeated. See above for the order filter flags are
10938 processed in.
10939
10940 --include has no effect when combined with --files-from or --files-
10941 from-raw flags.
10942
10943 --include implies --exclude ** at the end of an rclone internal filter
10944 list. Therefore if you mix --include and --include-from flags with
10945 --exclude, --exclude-from, --filter or --filter-from, you must use in‐
10946 clude rules for all the files you want in the include statement. For
10947 more flexibility use the --filter-from flag.
10948
10949 E.g. rclone ls remote: --include "*.{png,jpg}" lists the files on re‐
10950 mote: with suffix .png and .jpg. All other files are excluded.
10951
10952 E.g. multiple rclone copy commands can be combined with --include and
10953 a pattern-list.
10954
10955 rclone copy /vol1/A remote:A
10956 rclone copy /vol1/B remote:B
10957
10958 is equivalent to:
10959
10960 rclone copy /vol1 remote: --include "{A,B}/**"
10961
10962 E.g. rclone ls remote:/wheat --include "??[^[:punct:]]*" lists the
10963 files remote: directory wheat (and subdirectories) whose third charac‐
10964 ter is not punctuation. This example uses an ASCII character class
10965 (https://golang.org/pkg/regexp/syntax/).
10966
10967 --include-from - Read include patterns from file
10968 Adds path/file names to an rclone command based on rules in a named
10969 file. The file contains a list of remarks and pattern rules.
10970
10971 For an example include-file.txt:
10972
10973 # a sample include rule file
10974 *.jpg
10975 file2.avi
10976
10977 rclone ls remote: --include-from include-file.txt lists the files on
10978 remote: with name file2.avi or suffix .jpg. That is equivalent to
10979 rclone ls remote: --include file2.avi --include "*.jpg".
10980
10981 This flag can be repeated. See above for the order filter flags are
10982 processed in.
10983
10984 The --include-from flag is useful where multiple include filter rules
10985 are applied to an rclone command.
10986
10987 --include-from implies --exclude ** at the end of an rclone internal
10988 filter list. Therefore if you mix --include and --include-from flags
10989 with --exclude, --exclude-from, --filter or --filter-from, you must use
10990 include rules for all the files you want in the include statement. For
10991 more flexibility use the --filter-from flag.
10992
10993 --exclude-from has no effect when combined with --files-from or
10994 --files-from-raw flags.
10995
10996 --exclude-from followed by - reads filter rules from standard input.
10997
10998 --filter - Add a file-filtering rule
10999 Specifies path/file names to an rclone command, based on a single in‐
11000 clude or exclude rule, in + or - format.
11001
11002 This flag can be repeated. See above for the order filter flags are
11003 processed in.
11004
11005 --filter + differs from --include. In the case of --include rclone im‐
11006 plies an --exclude * rule which it adds to the bottom of the internal
11007 rule list. --filter...+ does not imply that rule.
11008
11009 --filter has no effect when combined with --files-from or --files-from-
11010 raw flags.
11011
11012 --filter should not be used with --include, --include-from, --exclude
11013 or --exclude-from flags.
11014
11015 E.g. rclone ls remote: --filter "- *.bak" excludes all .bak files from
11016 a list of remote:.
11017
11018 --filter-from - Read filtering patterns from a file
11019 Adds path/file names to an rclone command based on rules in a named
11020 file. The file contains a list of remarks and pattern rules. Include
11021 rules start with + and exclude rules with -. ! clears existing rules.
11022 Rules are processed in the order they are defined.
11023
11024 This flag can be repeated. See above for the order filter flags are
11025 processed in.
11026
11027 Arrange the order of filter rules with the most restrictive first and
11028 work down.
11029
11030 E.g. for filter-file.txt:
11031
11032 # a sample filter rule file
11033 - secret*.jpg
11034 + *.jpg
11035 + *.png
11036 + file2.avi
11037 - /dir/Trash/**
11038 + /dir/**
11039 # exclude everything else
11040 - *
11041
11042 rclone ls remote: --filter-from filter-file.txt lists the path/files on
11043 remote: including all jpg and png files, excluding any matching se‐
11044 cret*.jpg and including file2.avi. It also includes everything in the
11045 directory dir at the root of remote, except remote:dir/Trash which it
11046 excludes. Everything else is excluded.
11047
11048 E.g. for an alternative filter-file.txt:
11049
11050 - secret*.jpg
11051 + *.jpg
11052 + *.png
11053 + file2.avi
11054 - *
11055
11056 Files file1.jpg, file3.png and file2.avi are listed whilst secret17.jpg
11057 and files without the suffix .jpgor.png` are excluded.
11058
11059 E.g. for an alternative filter-file.txt:
11060
11061 + *.jpg
11062 + *.gif
11063 !
11064 + 42.doc
11065 - *
11066
11067 Only file 42.doc is listed. Prior rules are cleared by the !.
11068
11069 --files-from - Read list of source-file names
11070 Adds path/files to an rclone command from a list in a named file.
11071 Rclone processes the path/file names in the order of the list, and no
11072 others.
11073
11074 Other filter flags (--include, --include-from, --exclude, --exclude-
11075 from, --filter and --filter-from) are ignored when --files-from is
11076 used.
11077
11078 --files-from expects a list of files as its input. Leading or trailing
11079 whitespace is stripped from the input lines. Lines starting with # or
11080 ; are ignored.
11081
11082 Rclone commands with a --files-from flag traverse the remote, treating
11083 the names in --files-from as a set of filters.
11084
11085 If the --no-traverse and --files-from flags are used together an rclone
11086 command does not traverse the remote. Instead it addresses each
11087 path/file named in the file individually. For each path/file name,
11088 that requires typically 1 API call. This can be efficient for a short
11089 --files-from list and a remote containing many files.
11090
11091 Rclone commands do not error if any names in the --files-from file are
11092 missing from the source remote.
11093
11094 The --files-from flag can be repeated in a single rclone command to
11095 read path/file names from more than one file. The files are read from
11096 left to right along the command line.
11097
11098 Paths within the --files-from file are interpreted as starting with the
11099 root specified in the rclone command. Leading / separators are ig‐
11100 nored. See --files-from-raw if you need the input to be processed in a
11101 raw manner.
11102
11103 E.g. for a file files-from.txt:
11104
11105 # comment
11106 file1.jpg
11107 subdir/file2.jpg
11108
11109 rclone copy --files-from files-from.txt /home/me/pics remote:pics
11110 copies the following, if they exist, and only those files.
11111
11112 /home/me/pics/file1.jpg → remote:pics/file1.jpg
11113 /home/me/pics/subdir/file2.jpg → remote:pics/subdir/file2.jpg
11114
11115 E.g. to copy the following files referenced by their absolute paths:
11116
11117 /home/user1/42
11118 /home/user1/dir/ford
11119 /home/user2/prefect
11120
11121 First find a common subdirectory - in this case /home and put the re‐
11122 maining files in files-from.txt with or without leading /, e.g.
11123
11124 user1/42
11125 user1/dir/ford
11126 user2/prefect
11127
11128 Then copy these to a remote:
11129
11130 rclone copy --files-from files-from.txt /home remote:backup
11131
11132 The three files are transferred as follows:
11133
11134 /home/user1/42 → remote:backup/user1/important
11135 /home/user1/dir/ford → remote:backup/user1/dir/file
11136 /home/user2/prefect → remote:backup/user2/stuff
11137
11138 Alternatively if / is chosen as root files-from.txt will be:
11139
11140 /home/user1/42
11141 /home/user1/dir/ford
11142 /home/user2/prefect
11143
11144 The copy command will be:
11145
11146 rclone copy --files-from files-from.txt / remote:backup
11147
11148 Then there will be an extra home directory on the remote:
11149
11150 /home/user1/42 → remote:backup/home/user1/42
11151 /home/user1/dir/ford → remote:backup/home/user1/dir/ford
11152 /home/user2/prefect → remote:backup/home/user2/prefect
11153
11154 --files-from-raw - Read list of source-file names without any processing
11155 This flag is the same as --files-from except that input is read in a
11156 raw manner. Lines with leading / trailing whitespace, and lines start‐
11157 ing with ; or # are read without any processing. rclone lsf
11158 (https://rclone.org/commands/rclone_lsf/) has a compatible format that
11159 can be used to export file lists from remotes for input to --files-
11160 from-raw.
11161
11162 --ignore-case - make searches case insensitive
11163 By default, rclone filter patterns are case sensitive. The --ignore-
11164 case flag makes all of the filters patterns on the command line case
11165 insensitive.
11166
11167 E.g. --include "zaphod.txt" does not match a file Zaphod.txt. With
11168 --ignore-case a match is made.
11169
11170 Quoting shell metacharacters
11171 Rclone commands with filter patterns containing shell metacharacters
11172 may not as work as expected in your shell and may require quoting.
11173
11174 E.g. linux, OSX (* metacharacter)
11175
11176 • --include \*.jpg
11177
11178 • --include '*.jpg'
11179
11180 • --include='*.jpg'
11181
11182 Microsoft Windows expansion is done by the command, not shell, so --in‐
11183 clude *.jpg does not require quoting.
11184
11185 If the rclone error Command .... needs .... arguments maximum: you pro‐
11186 vided .... non flag arguments: is encountered, the cause is commonly
11187 spaces within the name of a remote or flag value. The fix then is to
11188 quote values containing spaces.
11189
11190 Other filters
11191 --min-size - Don't transfer any file smaller than this
11192 Controls the minimum size file within the scope of an rclone command.
11193 Default units are KiB but abbreviations K, M, G, T or P are valid.
11194
11195 E.g. rclone ls remote: --min-size 50k lists files on remote: of 50 KiB
11196 size or larger.
11197
11198 See the size option docs (https://rclone.org/docs/#size-option) for
11199 more info.
11200
11201 --max-size - Don't transfer any file larger than this
11202 Controls the maximum size file within the scope of an rclone command.
11203 Default units are KiB but abbreviations K, M, G, T or P are valid.
11204
11205 E.g. rclone ls remote: --max-size 1G lists files on remote: of 1 GiB
11206 size or smaller.
11207
11208 See the size option docs (https://rclone.org/docs/#size-option) for
11209 more info.
11210
11211 --max-age - Don't transfer any file older than this
11212 Controls the maximum age of files within the scope of an rclone com‐
11213 mand.
11214
11215 --max-age applies only to files and not to directories.
11216
11217 E.g. rclone ls remote: --max-age 2d lists files on remote: of 2 days
11218 old or less.
11219
11220 See the time option docs (https://rclone.org/docs/#time-option) for
11221 valid formats.
11222
11223 --min-age - Don't transfer any file younger than this
11224 Controls the minimum age of files within the scope of an rclone com‐
11225 mand. (see --max-age for valid formats)
11226
11227 --min-age applies only to files and not to directories.
11228
11229 E.g. rclone ls remote: --min-age 2d lists files on remote: of 2 days
11230 old or more.
11231
11232 See the time option docs (https://rclone.org/docs/#time-option) for
11233 valid formats.
11234
11235 Other flags
11236 --delete-excluded - Delete files on dest excluded from sync
11237 Important this flag is dangerous to your data - use with --dry-run and
11238 -v first.
11239
11240 In conjunction with rclone sync, --delete-excluded deletes any files on
11241 the destination which are excluded from the command.
11242
11243 E.g. the scope of rclone sync -i A: B: can be restricted:
11244
11245 rclone --min-size 50k --delete-excluded sync A: B:
11246
11247 All files on B: which are less than 50 KiB are deleted because they are
11248 excluded from the rclone sync command.
11249
11250 --dump filters - dump the filters to the output
11251 Dumps the defined filters to standard output in regular expression for‐
11252 mat.
11253
11254 Useful for debugging.
11255
11256 Exclude directory based on a file
11257 The --exclude-if-present flag controls whether a directory is within
11258 the scope of an rclone command based on the presence of a named file
11259 within it. The flag can be repeated to check for multiple file names,
11260 presence of any of them will exclude the directory.
11261
11262 This flag has a priority over other filter flags.
11263
11264 E.g. for the following directory structure:
11265
11266 dir1/file1
11267 dir1/dir2/file2
11268 dir1/dir2/dir3/file3
11269 dir1/dir2/dir3/.ignore
11270
11271 The command rclone ls --exclude-if-present .ignore dir1 does not list
11272 dir3, file3 or .ignore.
11273
11274 Common pitfalls
11275 The most frequent filter support issues on the rclone forum
11276 (https://forum.rclone.org/) are:
11277
11278 • Not using paths relative to the root of the remote
11279
11280 • Not using / to match from the root of a remote
11281
11282 • Not using ** to match the contents of a directory
11283
11285 Rclone can serve a web based GUI (graphical user interface). This is
11286 somewhat experimental at the moment so things may be subject to change.
11287
11288 Run this command in a terminal and rclone will download and then dis‐
11289 play the GUI in a web browser.
11290
11291 rclone rcd --rc-web-gui
11292
11293 This will produce logs like this and rclone needs to continue to run to
11294 serve the GUI:
11295
11296 2019/08/25 11:40:14 NOTICE: A new release for gui is present at https://github.com/rclone/rclone-webui-react/releases/download/v0.0.6/currentbuild.zip
11297 2019/08/25 11:40:14 NOTICE: Downloading webgui binary. Please wait. [Size: 3813937, Path : /home/USER/.cache/rclone/webgui/v0.0.6.zip]
11298 2019/08/25 11:40:16 NOTICE: Unzipping
11299 2019/08/25 11:40:16 NOTICE: Serving remote control on http://127.0.0.1:5572/
11300
11301 This assumes you are running rclone locally on your machine. It is
11302 possible to separate the rclone and the GUI - see below for details.
11303
11304 If you wish to check for updates then you can add --rc-web-gui-update
11305 to the command line.
11306
11307 If you find your GUI broken, you may force it to update by add --rc-
11308 web-gui-force-update.
11309
11310 By default, rclone will open your browser. Add --rc-web-gui-no-open-
11311 browser to disable this feature.
11312
11313 Using the GUI
11314 Once the GUI opens, you will be looking at the dashboard which has an
11315 overall overview.
11316
11317 On the left hand side you will see a series of view buttons you can
11318 click on:
11319
11320 • Dashboard - main overview
11321
11322 • Configs - examine and create new configurations
11323
11324 • Explorer - view, download and upload files to the cloud storage sys‐
11325 tems
11326
11327 • Backend - view or alter the backend config
11328
11329 • Log out
11330
11331 (More docs and walkthrough video to come!)
11332
11333 How it works
11334 When you run the rclone rcd --rc-web-gui this is what happens
11335
11336 • Rclone starts but only runs the remote control API ("rc").
11337
11338 • The API is bound to localhost with an auto-generated username and
11339 password.
11340
11341 • If the API bundle is missing then rclone will download it.
11342
11343 • rclone will start serving the files from the API bundle over the same
11344 port as the API
11345
11346 • rclone will open the browser with a login_token so it can log
11347 straight in.
11348
11349 Advanced use
11350 The rclone rcd may use any of the flags documented on the rc page
11351 (https://rclone.org/rc/#supported-parameters).
11352
11353 The flag --rc-web-gui is shorthand for
11354
11355 • Download the web GUI if necessary
11356
11357 • Check we are using some authentication
11358
11359 • --rc-user gui
11360
11361 • --rc-pass <random password>
11362
11363 • --rc-serve
11364
11365 These flags can be overridden as desired.
11366
11367 See also the rclone rcd documentation (https://rclone.org/com‐
11368 mands/rclone_rcd/).
11369
11370 Example: Running a public GUI
11371 For example the GUI could be served on a public port over SSL using an
11372 htpasswd file using the following flags:
11373
11374 • --rc-web-gui
11375
11376 • --rc-addr :443
11377
11378 • --rc-htpasswd /path/to/htpasswd
11379
11380 • --rc-cert /path/to/ssl.crt
11381
11382 • --rc-key /path/to/ssl.key
11383
11384 Example: Running a GUI behind a proxy
11385 If you want to run the GUI behind a proxy at /rclone you could use
11386 these flags:
11387
11388 • --rc-web-gui
11389
11390 • --rc-baseurl rclone
11391
11392 • --rc-htpasswd /path/to/htpasswd
11393
11394 Or instead of htpasswd if you just want a single user and password:
11395
11396 • --rc-user me
11397
11398 • --rc-pass mypassword
11399
11400 Project
11401 The GUI is being developed in the: rclone/rclone-webui-react repository
11402 (https://github.com/rclone/rclone-webui-react).
11403
11404 Bug reports and contributions are very welcome :-)
11405
11406 If you have questions then please ask them on the rclone forum
11407 (https://forum.rclone.org/).
11408
11410 If rclone is run with the --rc flag then it starts an HTTP server which
11411 can be used to remote control rclone using its API.
11412
11413 You can either use the rc command to access the API or use HTTP direct‐
11414 ly.
11415
11416 If you just want to run a remote control then see the rcd
11417 (https://rclone.org/commands/rclone_rcd/) command.
11418
11419 Supported parameters
11420 --rc
11421 Flag to start the http server listen on remote requests
11422
11423 --rc-addr=IP
11424 IPaddress:Port or :Port to bind server to. (default "localhost:5572")
11425
11426 --rc-cert=KEY
11427 SSL PEM key (concatenation of certificate and CA certificate)
11428
11429 --rc-client-ca=PATH
11430 Client certificate authority to verify clients with
11431
11432 --rc-htpasswd=PATH
11433 htpasswd file - if not provided no authentication is done
11434
11435 --rc-key=PATH
11436 SSL PEM Private key
11437
11438 --rc-max-header-bytes=VALUE
11439 Maximum size of request header (default 4096)
11440
11441 --rc-min-tls-version=VALUE
11442 The minimum TLS version that is acceptable. Valid values are "tls1.0",
11443 "tls1.1", "tls1.2" and "tls1.3" (default "tls1.0").
11444
11445 --rc-user=VALUE
11446 User name for authentication.
11447
11448 --rc-pass=VALUE
11449 Password for authentication.
11450
11451 --rc-realm=VALUE
11452 Realm for authentication (default "rclone")
11453
11454 --rc-server-read-timeout=DURATION
11455 Timeout for server reading data (default 1h0m0s)
11456
11457 --rc-server-write-timeout=DURATION
11458 Timeout for server writing data (default 1h0m0s)
11459
11460 --rc-serve
11461 Enable the serving of remote objects via the HTTP interface. This
11462 means objects will be accessible at http://127.0.0.1:5572/ by default,
11463 so you can browse to http://127.0.0.1:5572/ or http://127.0.0.1:5572/*
11464 to see a listing of the remotes. Objects may be requested from remotes
11465 using this syntax http://127.0.0.1:5572/[remote:path]/path/to/object
11466
11467 Default Off.
11468
11469 --rc-files /path/to/directory
11470 Path to local files to serve on the HTTP server.
11471
11472 If this is set then rclone will serve the files in that directory. It
11473 will also open the root in the web browser if specified. This is for
11474 implementing browser based GUIs for rclone functions.
11475
11476 If --rc-user or --rc-pass is set then the URL that is opened will have
11477 the authorization in the URL in the http://user:pass@localhost/ style.
11478
11479 Default Off.
11480
11481 --rc-enable-metrics
11482 Enable OpenMetrics/Prometheus compatible endpoint at /metrics.
11483
11484 Default Off.
11485
11486 --rc-web-gui
11487 Set this flag to serve the default web gui on the same port as rclone.
11488
11489 Default Off.
11490
11491 --rc-allow-origin
11492 Set the allowed Access-Control-Allow-Origin for rc requests.
11493
11494 Can be used with --rc-web-gui if the rclone is running on different IP
11495 than the web-gui.
11496
11497 Default is IP address on which rc is running.
11498
11499 --rc-web-fetch-url
11500 Set the URL to fetch the rclone-web-gui files from.
11501
11502 Default https://api.github.com/repos/rclone/rclone-webui-react/releas‐
11503 es/latest.
11504
11505 --rc-web-gui-update
11506 Set this flag to check and update rclone-webui-react from the rc-web-
11507 fetch-url.
11508
11509 Default Off.
11510
11511 --rc-web-gui-force-update
11512 Set this flag to force update rclone-webui-react from the rc-web-fetch-
11513 url.
11514
11515 Default Off.
11516
11517 --rc-web-gui-no-open-browser
11518 Set this flag to disable opening browser automatically when using web-
11519 gui.
11520
11521 Default Off.
11522
11523 --rc-job-expire-duration=DURATION
11524 Expire finished async jobs older than DURATION (default 60s).
11525
11526 --rc-job-expire-interval=DURATION
11527 Interval duration to check for expired async jobs (default 10s).
11528
11529 --rc-no-auth
11530 By default rclone will require authorisation to have been set up on the
11531 rc interface in order to use any methods which access any rclone re‐
11532 motes. Eg operations/list is denied as it involved creating a remote
11533 as is sync/copy.
11534
11535 If this is set then no authorisation will be required on the server to
11536 use these methods. The alternative is to use --rc-user and --rc-pass
11537 and use these credentials in the request.
11538
11539 Default Off.
11540
11541 --rc-baseurl
11542 Prefix for URLs.
11543
11544 Default is root
11545
11546 --rc-template
11547 User-specified template.
11548
11549 Accessing the remote control via the rclone rc command
11550 Rclone itself implements the remote control protocol in its rclone rc
11551 command.
11552
11553 You can use it like this
11554
11555 $ rclone rc rc/noop param1=one param2=two
11556 {
11557 "param1": "one",
11558 "param2": "two"
11559 }
11560
11561 Run rclone rc on its own to see the help for the installed remote con‐
11562 trol commands.
11563
11564 JSON input
11565 rclone rc also supports a --json flag which can be used to send more
11566 complicated input parameters.
11567
11568 $ rclone rc --json '{ "p1": [1,"2",null,4], "p2": { "a":1, "b":2 } }' rc/noop
11569 {
11570 "p1": [
11571 1,
11572 "2",
11573 null,
11574 4
11575 ],
11576 "p2": {
11577 "a": 1,
11578 "b": 2
11579 }
11580 }
11581
11582 If the parameter being passed is an object then it can be passed as a
11583 JSON string rather than using the --json flag which simplifies the com‐
11584 mand line.
11585
11586 rclone rc operations/list fs=/tmp remote=test opt='{"showHash": true}'
11587
11588 Rather than
11589
11590 rclone rc operations/list --json '{"fs": "/tmp", "remote": "test", "opt": {"showHash": true}}'
11591
11592 Special parameters
11593 The rc interface supports some special parameters which apply to all
11594 commands. These start with _ to show they are different.
11595
11596 Running asynchronous jobs with _async = true
11597 Each rc call is classified as a job and it is assigned its own id. By
11598 default jobs are executed immediately as they are created or syn‐
11599 chronously.
11600
11601 If _async has a true value when supplied to an rc call then it will re‐
11602 turn immediately with a job id and the task will be run in the back‐
11603 ground. The job/status call can be used to get information of the
11604 background job. The job can be queried for up to 1 minute after it has
11605 finished.
11606
11607 It is recommended that potentially long running jobs, e.g. sync/sync,
11608 sync/copy, sync/move, operations/purge are run with the _async flag to
11609 avoid any potential problems with the HTTP request and response timing
11610 out.
11611
11612 Starting a job with the _async flag:
11613
11614 $ rclone rc --json '{ "p1": [1,"2",null,4], "p2": { "a":1, "b":2 }, "_async": true }' rc/noop
11615 {
11616 "jobid": 2
11617 }
11618
11619 Query the status to see if the job has finished. For more information
11620 on the meaning of these return parameters see the job/status call.
11621
11622 $ rclone rc --json '{ "jobid":2 }' job/status
11623 {
11624 "duration": 0.000124163,
11625 "endTime": "2018-10-27T11:38:07.911245881+01:00",
11626 "error": "",
11627 "finished": true,
11628 "id": 2,
11629 "output": {
11630 "_async": true,
11631 "p1": [
11632 1,
11633 "2",
11634 null,
11635 4
11636 ],
11637 "p2": {
11638 "a": 1,
11639 "b": 2
11640 }
11641 },
11642 "startTime": "2018-10-27T11:38:07.911121728+01:00",
11643 "success": true
11644 }
11645
11646 job/list can be used to show the running or recently completed jobs
11647
11648 $ rclone rc job/list
11649 {
11650 "jobids": [
11651 2
11652 ]
11653 }
11654
11655 Setting config flags with _config
11656 If you wish to set config (the equivalent of the global flags) for the
11657 duration of an rc call only then pass in the _config parameter.
11658
11659 This should be in the same format as the config key returned by op‐
11660 tions/get.
11661
11662 For example, if you wished to run a sync with the --checksum parameter,
11663 you would pass this parameter in your JSON blob.
11664
11665 "_config":{"CheckSum": true}
11666
11667 If using rclone rc this could be passed as
11668
11669 rclone rc operations/sync ... _config='{"CheckSum": true}'
11670
11671 Any config parameters you don't set will inherit the global defaults
11672 which were set with command line flags or environment variables.
11673
11674 Note that it is possible to set some values as strings or integers -
11675 see data types for more info. Here is an example setting the equiva‐
11676 lent of --buffer-size in string or integer format.
11677
11678 "_config":{"BufferSize": "42M"}
11679 "_config":{"BufferSize": 44040192}
11680
11681 If you wish to check the _config assignment has worked properly then
11682 calling options/local will show what the value got set to.
11683
11684 Setting filter flags with _filter
11685 If you wish to set filters for the duration of an rc call only then
11686 pass in the _filter parameter.
11687
11688 This should be in the same format as the filter key returned by op‐
11689 tions/get.
11690
11691 For example, if you wished to run a sync with these flags
11692
11693 --max-size 1M --max-age 42s --include "a" --include "b"
11694
11695 you would pass this parameter in your JSON blob.
11696
11697 "_filter":{"MaxSize":"1M", "IncludeRule":["a","b"], "MaxAge":"42s"}
11698
11699 If using rclone rc this could be passed as
11700
11701 rclone rc ... _filter='{"MaxSize":"1M", "IncludeRule":["a","b"], "MaxAge":"42s"}'
11702
11703 Any filter parameters you don't set will inherit the global defaults
11704 which were set with command line flags or environment variables.
11705
11706 Note that it is possible to set some values as strings or integers -
11707 see data types for more info. Here is an example setting the equiva‐
11708 lent of --buffer-size in string or integer format.
11709
11710 "_filter":{"MinSize": "42M"}
11711 "_filter":{"MinSize": 44040192}
11712
11713 If you wish to check the _filter assignment has worked properly then
11714 calling options/local will show what the value got set to.
11715
11716 Assigning operations to groups with _group = value
11717 Each rc call has its own stats group for tracking its metrics. By de‐
11718 fault grouping is done by the composite group name from prefix job/ and
11719 id of the job like so job/1.
11720
11721 If _group has a value then stats for that request will be grouped under
11722 that value. This allows caller to group stats under their own name.
11723
11724 Stats for specific group can be accessed by passing group to
11725 core/stats:
11726
11727 $ rclone rc --json '{ "group": "job/1" }' core/stats
11728 {
11729 "speed": 12345
11730 ...
11731 }
11732
11733 Data types
11734 When the API returns types, these will mostly be straight forward inte‐
11735 ger, string or boolean types.
11736
11737 However some of the types returned by the options/get call and taken by
11738 the options/set calls as well as the vfsOpt, mountOpt and the _config
11739 parameters.
11740
11741 • Duration - these are returned as an integer duration in nanoseconds.
11742 They may be set as an integer, or they may be set with time string,
11743 eg "5s". See the options section (https://rclone.org/docs/#options)
11744 for more info.
11745
11746 • Size - these are returned as an integer number of bytes. They may be
11747 set as an integer or they may be set with a size suffix string, eg
11748 "10M". See the options section (https://rclone.org/docs/#options)
11749 for more info.
11750
11751 • Enumerated type (such as CutoffMode, DumpFlags, LogLevel, Vfs‐
11752 CacheMode - these will be returned as an integer and may be set as an
11753 integer but more conveniently they can be set as a string, eg "HARD"
11754 for CutoffMode or DEBUG for LogLevel.
11755
11756 • BandwidthSpec - this will be set and returned as a string, eg "1M".
11757
11758 Specifying remotes to work on
11759 Remotes are specified with the fs=, srcFs=, dstFs= parameters depending
11760 on the command being used.
11761
11762 The parameters can be a string as per the rest of rclone, eg s3:buck‐
11763 et/path or :sftp:/my/dir. They can also be specified as JSON blobs.
11764
11765 If specifying a JSON blob it should be a object mapping strings to
11766 strings. These values will be used to configure the remote. There are
11767 3 special values which may be set:
11768
11769 • type - set to type to specify a remote called :type:
11770
11771 • _name - set to name to specify a remote called name:
11772
11773 • _root - sets the root of the remote - may be empty
11774
11775 One of _name or type should normally be set. If the local backend is
11776 desired then type should be set to local. If _root isn't specified
11777 then it defaults to the root of the remote.
11778
11779 For example this JSON is equivalent to remote:/tmp
11780
11781 {
11782 "_name": "remote",
11783 "_path": "/tmp"
11784 }
11785
11786 And this is equivalent to :sftp,host='example.com':/tmp
11787
11788 {
11789 "type": "sftp",
11790 "host": "example.com",
11791 "_path": "/tmp"
11792 }
11793
11794 And this is equivalent to /tmp/dir
11795
11796 {
11797 type = "local",
11798 _ path = "/tmp/dir"
11799 }
11800
11801 Supported commands
11802 backend/command: Runs a backend command.
11803 This takes the following parameters:
11804
11805 • command - a string with the command name
11806
11807 • fs - a remote name string e.g. "drive:"
11808
11809 • arg - a list of arguments for the backend command
11810
11811 • opt - a map of string to string of options
11812
11813 Returns:
11814
11815 • result - result from the backend command
11816
11817 Example:
11818
11819 rclone rc backend/command command=noop fs=. -o echo=yes -o blue -a path1 -a path2
11820
11821 Returns
11822
11823 {
11824 "result": {
11825 "arg": [
11826 "path1",
11827 "path2"
11828 ],
11829 "name": "noop",
11830 "opt": {
11831 "blue": "",
11832 "echo": "yes"
11833 }
11834 }
11835 }
11836
11837 Note that this is the direct equivalent of using this "backend" com‐
11838 mand:
11839
11840 rclone backend noop . -o echo=yes -o blue path1 path2
11841
11842 Note that arguments must be preceded by the "-a" flag
11843
11844 See the backend (https://rclone.org/commands/rclone_backend/) command
11845 for more information.
11846
11847 Authentication is required for this call.
11848
11849 cache/expire: Purge a remote from cache
11850 Purge a remote from the cache backend. Supports either a directory or
11851 a file. Params: - remote = path to remote (required) - withData =
11852 true/false to delete cached data (chunks) as well (optional)
11853
11854 Eg
11855
11856 rclone rc cache/expire remote=path/to/sub/folder/
11857 rclone rc cache/expire remote=/ withData=true
11858
11859 cache/fetch: Fetch file chunks
11860 Ensure the specified file chunks are cached on disk.
11861
11862 The chunks= parameter specifies the file chunks to check. It takes a
11863 comma separated list of array slice indices. The slice indices are
11864 similar to Python slices: start[:end]
11865
11866 start is the 0 based chunk number from the beginning of the file to
11867 fetch inclusive. end is 0 based chunk number from the beginning of the
11868 file to fetch exclusive. Both values can be negative, in which case
11869 they count from the back of the file. The value "-5:" represents the
11870 last 5 chunks of a file.
11871
11872 Some valid examples are: ":5,-5:" -> the first and last five chunks
11873 "0,-2" -> the first and the second last chunk "0:10" -> the first ten
11874 chunks
11875
11876 Any parameter with a key that starts with "file" can be used to specify
11877 files to fetch, e.g.
11878
11879 rclone rc cache/fetch chunks=0 file=hello file2=home/goodbye
11880
11881 File names will automatically be encrypted when the a crypt remote is
11882 used on top of the cache.
11883
11884 cache/stats: Get cache stats
11885 Show statistics for the cache remote.
11886
11887 config/create: create the config for a remote.
11888 This takes the following parameters:
11889
11890 • name - name of remote
11891
11892 • parameters - a map of { "key": "value" } pairs
11893
11894 • type - type of the new remote
11895
11896 • opt - a dictionary of options to control the configuration
11897
11898 • obscure - declare passwords are plain and need obscuring
11899
11900 • noObscure - declare passwords are already obscured and don't need
11901 obscuring
11902
11903 • nonInteractive - don't interact with a user, return questions
11904
11905 • continue - continue the config process with an answer
11906
11907 • all - ask all the config questions not just the post config ones
11908
11909 • state - state to restart with - used with continue
11910
11911 • result - result to restart with - used with continue
11912
11913 See the config create (https://rclone.org/commands/rclone_config_cre‐
11914 ate/) command for more information on the above.
11915
11916 Authentication is required for this call.
11917
11918 config/delete: Delete a remote in the config file.
11919 Parameters:
11920
11921 • name - name of remote to delete
11922
11923 See the config delete (https://rclone.org/commands/rclone_con‐
11924 fig_delete/) command for more information on the above.
11925
11926 Authentication is required for this call.
11927
11928 config/dump: Dumps the config file.
11929 Returns a JSON object: - key: value
11930
11931 Where keys are remote names and values are the config parameters.
11932
11933 See the config dump (https://rclone.org/commands/rclone_config_dump/)
11934 command for more information on the above.
11935
11936 Authentication is required for this call.
11937
11938 config/get: Get a remote in the config file.
11939 Parameters:
11940
11941 • name - name of remote to get
11942
11943 See the config dump (https://rclone.org/commands/rclone_config_dump/)
11944 command for more information on the above.
11945
11946 Authentication is required for this call.
11947
11948 config/listremotes: Lists the remotes in the config file.
11949 Returns - remotes - array of remote names
11950
11951 See the listremotes (https://rclone.org/commands/rclone_listremotes/)
11952 command for more information on the above.
11953
11954 Authentication is required for this call.
11955
11956 config/password: password the config for a remote.
11957 This takes the following parameters:
11958
11959 • name - name of remote
11960
11961 • parameters - a map of { "key": "value" } pairs
11962
11963 See the config password (https://rclone.org/commands/rclone_con‐
11964 fig_password/) command for more information on the above.
11965
11966 Authentication is required for this call.
11967
11968 config/providers: Shows how providers are configured in the config file.
11969 Returns a JSON object: - providers - array of objects
11970
11971 See the config providers (https://rclone.org/commands/rclone_con‐
11972 fig_providers/) command for more information on the above.
11973
11974 Authentication is required for this call.
11975
11976 config/update: update the config for a remote.
11977 This takes the following parameters:
11978
11979 • name - name of remote
11980
11981 • parameters - a map of { "key": "value" } pairs
11982
11983 • opt - a dictionary of options to control the configuration
11984
11985 • obscure - declare passwords are plain and need obscuring
11986
11987 • noObscure - declare passwords are already obscured and don't need
11988 obscuring
11989
11990 • nonInteractive - don't interact with a user, return questions
11991
11992 • continue - continue the config process with an answer
11993
11994 • all - ask all the config questions not just the post config ones
11995
11996 • state - state to restart with - used with continue
11997
11998 • result - result to restart with - used with continue
11999
12000 See the config update (https://rclone.org/commands/rclone_config_up‐
12001 date/) command for more information on the above.
12002
12003 Authentication is required for this call.
12004
12005 core/bwlimit: Set the bandwidth limit.
12006 This sets the bandwidth limit to the string passed in. This should be
12007 a single bandwidth limit entry or a pair of upload:download bandwidth.
12008
12009 Eg
12010
12011 rclone rc core/bwlimit rate=off
12012 {
12013 "bytesPerSecond": -1,
12014 "bytesPerSecondTx": -1,
12015 "bytesPerSecondRx": -1,
12016 "rate": "off"
12017 }
12018 rclone rc core/bwlimit rate=1M
12019 {
12020 "bytesPerSecond": 1048576,
12021 "bytesPerSecondTx": 1048576,
12022 "bytesPerSecondRx": 1048576,
12023 "rate": "1M"
12024 }
12025 rclone rc core/bwlimit rate=1M:100k
12026 {
12027 "bytesPerSecond": 1048576,
12028 "bytesPerSecondTx": 1048576,
12029 "bytesPerSecondRx": 131072,
12030 "rate": "1M"
12031 }
12032
12033 If the rate parameter is not supplied then the bandwidth is queried
12034
12035 rclone rc core/bwlimit
12036 {
12037 "bytesPerSecond": 1048576,
12038 "bytesPerSecondTx": 1048576,
12039 "bytesPerSecondRx": 1048576,
12040 "rate": "1M"
12041 }
12042
12043 The format of the parameter is exactly the same as passed to --bwlimit
12044 except only one bandwidth may be specified.
12045
12046 In either case "rate" is returned as a human-readable string, and
12047 "bytesPerSecond" is returned as a number.
12048
12049 core/command: Run a rclone terminal command over rc.
12050 This takes the following parameters:
12051
12052 • command - a string with the command name.
12053
12054 • arg - a list of arguments for the backend command.
12055
12056 • opt - a map of string to string of options.
12057
12058 • returnType - one of ("COMBINED_OUTPUT", "STREAM", "STREAM_ONLY_STD‐
12059 OUT", "STREAM_ONLY_STDERR").
12060
12061 • Defaults to "COMBINED_OUTPUT" if not set.
12062
12063 • The STREAM returnTypes will write the output to the body of the
12064 HTTP message.
12065
12066 • The COMBINED_OUTPUT will write the output to the "result" parame‐
12067 ter.
12068
12069 Returns:
12070
12071 • result - result from the backend command.
12072
12073 • Only set when using returnType "COMBINED_OUTPUT".
12074
12075 • error - set if rclone exits with an error code.
12076
12077 • returnType - one of ("COMBINED_OUTPUT", "STREAM", "STREAM_ONLY_STD‐
12078 OUT", "STREAM_ONLY_STDERR").
12079
12080 Example:
12081
12082 rclone rc core/command command=ls -a mydrive:/ -o max-depth=1
12083 rclone rc core/command -a ls -a mydrive:/ -o max-depth=1
12084
12085 Returns:
12086
12087 {
12088 "error": false,
12089 "result": "<Raw command line output>"
12090 }
12091
12092 OR
12093 {
12094 "error": true,
12095 "result": "<Raw command line output>"
12096 }
12097
12098 Authentication is required for this call.
12099
12100 core/gc: Runs a garbage collection.
12101 This tells the go runtime to do a garbage collection run. It isn't
12102 necessary to call this normally, but it can be useful for debugging
12103 memory problems.
12104
12105 core/group-list: Returns list of stats.
12106 This returns list of stats groups currently in memory.
12107
12108 Returns the following values:
12109
12110 {
12111 "groups": an array of group names:
12112 [
12113 "group1",
12114 "group2",
12115 ...
12116 ]
12117 }
12118
12119 core/memstats: Returns the memory statistics
12120 This returns the memory statistics of the running program. What the
12121 values mean are explained in the go docs: https://golang.org/pkg/run‐
12122 time/#MemStats
12123
12124 The most interesting values for most people are:
12125
12126 • HeapAlloc - this is the amount of memory rclone is actually using
12127
12128 • HeapSys - this is the amount of memory rclone has obtained from the
12129 OS
12130
12131 • Sys - this is the total amount of memory requested from the OS
12132
12133 • It is virtual memory so may include unused memory
12134
12135 core/obscure: Obscures a string passed in.
12136 Pass a clear string and rclone will obscure it for the config file: -
12137 clear - string
12138
12139 Returns: - obscured - string
12140
12141 core/pid: Return PID of current process
12142 This returns PID of current process. Useful for stopping rclone
12143 process.
12144
12145 core/quit: Terminates the app.
12146 (Optional) Pass an exit code to be used for terminating the app: - ex‐
12147 itCode - int
12148
12149 core/stats: Returns stats about current transfers.
12150 This returns all available stats:
12151
12152 rclone rc core/stats
12153
12154 If group is not provided then summed up stats for all groups will be
12155 returned.
12156
12157 Parameters
12158
12159 • group - name of the stats group (string)
12160
12161 Returns the following values:
12162
12163 {
12164 "bytes": total transferred bytes since the start of the group,
12165 "checks": number of files checked,
12166 "deletes" : number of files deleted,
12167 "elapsedTime": time in floating point seconds since rclone was started,
12168 "errors": number of errors,
12169 "eta": estimated time in seconds until the group completes,
12170 "fatalError": boolean whether there has been at least one fatal error,
12171 "lastError": last error string,
12172 "renames" : number of files renamed,
12173 "retryError": boolean showing whether there has been at least one non-NoRetryError,
12174 "speed": average speed in bytes per second since start of the group,
12175 "totalBytes": total number of bytes in the group,
12176 "totalChecks": total number of checks in the group,
12177 "totalTransfers": total number of transfers in the group,
12178 "transferTime" : total time spent on running jobs,
12179 "transfers": number of transferred files,
12180 "transferring": an array of currently active file transfers:
12181 [
12182 {
12183 "bytes": total transferred bytes for this file,
12184 "eta": estimated time in seconds until file transfer completion
12185 "name": name of the file,
12186 "percentage": progress of the file transfer in percent,
12187 "speed": average speed over the whole transfer in bytes per second,
12188 "speedAvg": current speed in bytes per second as an exponentially weighted moving average,
12189 "size": size of the file in bytes
12190 }
12191 ],
12192 "checking": an array of names of currently active file checks
12193 []
12194 }
12195
12196 Values for "transferring", "checking" and "lastError" are only assigned
12197 if data is available. The value for "eta" is null if an eta cannot be
12198 determined.
12199
12200 core/stats-delete: Delete stats group.
12201 This deletes entire stats group.
12202
12203 Parameters
12204
12205 • group - name of the stats group (string)
12206
12207 core/stats-reset: Reset stats.
12208 This clears counters, errors and finished transfers for all stats or
12209 specific stats group if group is provided.
12210
12211 Parameters
12212
12213 • group - name of the stats group (string)
12214
12215 core/transferred: Returns stats about completed transfers.
12216 This returns stats about completed transfers:
12217
12218 rclone rc core/transferred
12219
12220 If group is not provided then completed transfers for all groups will
12221 be returned.
12222
12223 Note only the last 100 completed transfers are returned.
12224
12225 Parameters
12226
12227 • group - name of the stats group (string)
12228
12229 Returns the following values:
12230
12231 {
12232 "transferred": an array of completed transfers (including failed ones):
12233 [
12234 {
12235 "name": name of the file,
12236 "size": size of the file in bytes,
12237 "bytes": total transferred bytes for this file,
12238 "checked": if the transfer is only checked (skipped, deleted),
12239 "timestamp": integer representing millisecond unix epoch,
12240 "error": string description of the error (empty if successful),
12241 "jobid": id of the job that this transfer belongs to
12242 }
12243 ]
12244 }
12245
12246 core/version: Shows the current version of rclone and the go runtime.
12247 This shows the current version of go and the go runtime:
12248
12249 • version - rclone version, e.g. "v1.53.0"
12250
12251 • decomposed - version number as [major, minor, patch]
12252
12253 • isGit - boolean - true if this was compiled from the git version
12254
12255 • isBeta - boolean - true if this is a beta version
12256
12257 • os - OS in use as according to Go
12258
12259 • arch - cpu architecture in use according to Go
12260
12261 • goVersion - version of Go runtime in use
12262
12263 • linking - type of rclone executable (static or dynamic)
12264
12265 • goTags - space separated build tags or "none"
12266
12267 debug/set-block-profile-rate: Set runtime.SetBlockProfileRate for blocking
12268 profiling.
12269 SetBlockProfileRate controls the fraction of goroutine blocking events
12270 that are reported in the blocking profile. The profiler aims to sample
12271 an average of one blocking event per rate nanoseconds spent blocked.
12272
12273 To include every blocking event in the profile, pass rate = 1. To turn
12274 off profiling entirely, pass rate <= 0.
12275
12276 After calling this you can use this to see the blocking profile:
12277
12278 go tool pprof http://localhost:5572/debug/pprof/block
12279
12280 Parameters:
12281
12282 • rate - int
12283
12284 debug/set-mutex-profile-fraction: Set runtime.SetMutexProfileFraction for
12285 mutex profiling.
12286 SetMutexProfileFraction controls the fraction of mutex contention
12287 events that are reported in the mutex profile. On average 1/rate
12288 events are reported. The previous rate is returned.
12289
12290 To turn off profiling entirely, pass rate 0. To just read the current
12291 rate, pass rate < 0. (For n>1 the details of sampling may change.)
12292
12293 Once this is set you can look use this to profile the mutex contention:
12294
12295 go tool pprof http://localhost:5572/debug/pprof/mutex
12296
12297 Parameters:
12298
12299 • rate - int
12300
12301 Results:
12302
12303 • previousRate - int
12304
12305 fscache/clear: Clear the Fs cache.
12306 This clears the fs cache. This is where remotes created from backends
12307 are cached for a short while to make repeated rc calls more efficient.
12308
12309 If you change the parameters of a backend then you may want to call
12310 this to clear an existing remote out of the cache before re-creating
12311 it.
12312
12313 Authentication is required for this call.
12314
12315 fscache/entries: Returns the number of entries in the fs cache.
12316 This returns the number of entries in the fs cache.
12317
12318 Returns - entries - number of items in the cache
12319
12320 Authentication is required for this call.
12321
12322 job/list: Lists the IDs of the running jobs
12323 Parameters: None.
12324
12325 Results:
12326
12327 • jobids - array of integer job ids.
12328
12329 job/status: Reads the status of the job ID
12330 Parameters:
12331
12332 • jobid - id of the job (integer).
12333
12334 Results:
12335
12336 • finished - boolean
12337
12338 • duration - time in seconds that the job ran for
12339
12340 • endTime - time the job finished (e.g.
12341 "2018-10-26T18:50:20.528746884+01:00")
12342
12343 • error - error from the job or empty string for no error
12344
12345 • finished - boolean whether the job has finished or not
12346
12347 • id - as passed in above
12348
12349 • startTime - time the job started (e.g.
12350 "2018-10-26T18:50:20.528336039+01:00")
12351
12352 • success - boolean - true for success false otherwise
12353
12354 • output - output of the job as would have been returned if called syn‐
12355 chronously
12356
12357 • progress - output of the progress related to the underlying job
12358
12359 job/stop: Stop the running job
12360 Parameters:
12361
12362 • jobid - id of the job (integer).
12363
12364 job/stopgroup: Stop all running jobs in a group
12365 Parameters:
12366
12367 • group - name of the group (string).
12368
12369 mount/listmounts: Show current mount points
12370 This shows currently mounted points, which can be used for performing
12371 an unmount.
12372
12373 This takes no parameters and returns
12374
12375 • mountPoints: list of current mount points
12376
12377 Eg
12378
12379 rclone rc mount/listmounts
12380
12381 Authentication is required for this call.
12382
12383 mount/mount: Create a new mount point
12384 rclone allows Linux, FreeBSD, macOS and Windows to mount any of
12385 Rclone's cloud storage systems as a file system with FUSE.
12386
12387 If no mountType is provided, the priority is given as follows: 1.
12388 mount 2.cmount 3.mount2
12389
12390 This takes the following parameters:
12391
12392 • fs - a remote path to be mounted (required)
12393
12394 • mountPoint: valid path on the local machine (required)
12395
12396 • mountType: one of the values (mount, cmount, mount2) specifies the
12397 mount implementation to use
12398
12399 • mountOpt: a JSON object with Mount options in.
12400
12401 • vfsOpt: a JSON object with VFS options in.
12402
12403 Example:
12404
12405 rclone rc mount/mount fs=mydrive: mountPoint=/home/<user>/mountPoint
12406 rclone rc mount/mount fs=mydrive: mountPoint=/home/<user>/mountPoint mountType=mount
12407 rclone rc mount/mount fs=TestDrive: mountPoint=/mnt/tmp vfsOpt='{"CacheMode": 2}' mountOpt='{"AllowOther": true}'
12408
12409 The vfsOpt are as described in options/get and can be seen in the the
12410 "vfs" section when running and the mountOpt can be seen in the "mount"
12411 section:
12412
12413 rclone rc options/get
12414
12415 Authentication is required for this call.
12416
12417 mount/types: Show all possible mount types
12418 This shows all possible mount types and returns them as a list.
12419
12420 This takes no parameters and returns
12421
12422 • mountTypes: list of mount types
12423
12424 The mount types are strings like "mount", "mount2", "cmount" and can be
12425 passed to mount/mount as the mountType parameter.
12426
12427 Eg
12428
12429 rclone rc mount/types
12430
12431 Authentication is required for this call.
12432
12433 mount/unmount: Unmount selected active mount
12434 rclone allows Linux, FreeBSD, macOS and Windows to mount any of
12435 Rclone's cloud storage systems as a file system with FUSE.
12436
12437 This takes the following parameters:
12438
12439 • mountPoint: valid path on the local machine where the mount was cre‐
12440 ated (required)
12441
12442 Example:
12443
12444 rclone rc mount/unmount mountPoint=/home/<user>/mountPoint
12445
12446 Authentication is required for this call.
12447
12448 mount/unmountall: Unmount all active mounts
12449 rclone allows Linux, FreeBSD, macOS and Windows to mount any of
12450 Rclone's cloud storage systems as a file system with FUSE.
12451
12452 This takes no parameters and returns error if unmount does not succeed.
12453
12454 Eg
12455
12456 rclone rc mount/unmountall
12457
12458 Authentication is required for this call.
12459
12460 operations/about: Return the space used on the remote
12461 This takes the following parameters:
12462
12463 • fs - a remote name string e.g. "drive:"
12464
12465 The result is as returned from rclone about --json
12466
12467 See the about (https://rclone.org/commands/rclone_about/) command for
12468 more information on the above.
12469
12470 Authentication is required for this call.
12471
12472 operations/cleanup: Remove trashed files in the remote or path
12473 This takes the following parameters:
12474
12475 • fs - a remote name string e.g. "drive:"
12476
12477 See the cleanup (https://rclone.org/commands/rclone_cleanup/) command
12478 for more information on the above.
12479
12480 Authentication is required for this call.
12481
12482 operations/copyfile: Copy a file from source remote to destination remote
12483 This takes the following parameters:
12484
12485 • srcFs - a remote name string e.g. "drive:" for the source
12486
12487 • srcRemote - a path within that remote e.g. "file.txt" for the source
12488
12489 • dstFs - a remote name string e.g. "drive2:" for the destination
12490
12491 • dstRemote - a path within that remote e.g. "file2.txt" for the des‐
12492 tination
12493
12494 Authentication is required for this call.
12495
12496 operations/copyurl: Copy the URL to the object
12497 This takes the following parameters:
12498
12499 • fs - a remote name string e.g. "drive:"
12500
12501 • remote - a path within that remote e.g. "dir"
12502
12503 • url - string, URL to read from
12504
12505 • autoFilename - boolean, set to true to retrieve destination file name
12506 from url
12507
12508 See the copyurl (https://rclone.org/commands/rclone_copyurl/) command
12509 for more information on the above.
12510
12511 Authentication is required for this call.
12512
12513 operations/delete: Remove files in the path
12514 This takes the following parameters:
12515
12516 • fs - a remote name string e.g. "drive:"
12517
12518 See the delete (https://rclone.org/commands/rclone_delete/) command for
12519 more information on the above.
12520
12521 Authentication is required for this call.
12522
12523 operations/deletefile: Remove the single file pointed to
12524 This takes the following parameters:
12525
12526 • fs - a remote name string e.g. "drive:"
12527
12528 • remote - a path within that remote e.g. "dir"
12529
12530 See the deletefile (https://rclone.org/commands/rclone_deletefile/)
12531 command for more information on the above.
12532
12533 Authentication is required for this call.
12534
12535 operations/fsinfo: Return information about the remote
12536 This takes the following parameters:
12537
12538 • fs - a remote name string e.g. "drive:"
12539
12540 This returns info about the remote passed in;
12541
12542 {
12543 // optional features and whether they are available or not
12544 "Features": {
12545 "About": true,
12546 "BucketBased": false,
12547 "BucketBasedRootOK": false,
12548 "CanHaveEmptyDirectories": true,
12549 "CaseInsensitive": false,
12550 "ChangeNotify": false,
12551 "CleanUp": false,
12552 "Command": true,
12553 "Copy": false,
12554 "DirCacheFlush": false,
12555 "DirMove": true,
12556 "Disconnect": false,
12557 "DuplicateFiles": false,
12558 "GetTier": false,
12559 "IsLocal": true,
12560 "ListR": false,
12561 "MergeDirs": false,
12562 "MetadataInfo": true,
12563 "Move": true,
12564 "OpenWriterAt": true,
12565 "PublicLink": false,
12566 "Purge": true,
12567 "PutStream": true,
12568 "PutUnchecked": false,
12569 "ReadMetadata": true,
12570 "ReadMimeType": false,
12571 "ServerSideAcrossConfigs": false,
12572 "SetTier": false,
12573 "SetWrapper": false,
12574 "Shutdown": false,
12575 "SlowHash": true,
12576 "SlowModTime": false,
12577 "UnWrap": false,
12578 "UserInfo": false,
12579 "UserMetadata": true,
12580 "WrapFs": false,
12581 "WriteMetadata": true,
12582 "WriteMimeType": false
12583 },
12584 // Names of hashes available
12585 "Hashes": [
12586 "md5",
12587 "sha1",
12588 "whirlpool",
12589 "crc32",
12590 "sha256",
12591 "dropbox",
12592 "mailru",
12593 "quickxor"
12594 ],
12595 "Name": "local", // Name as created
12596 "Precision": 1, // Precision of timestamps in ns
12597 "Root": "/", // Path as created
12598 "String": "Local file system at /", // how the remote will appear in logs
12599 // Information about the system metadata for this backend
12600 "MetadataInfo": {
12601 "System": {
12602 "atime": {
12603 "Help": "Time of last access",
12604 "Type": "RFC 3339",
12605 "Example": "2006-01-02T15:04:05.999999999Z07:00"
12606 },
12607 "btime": {
12608 "Help": "Time of file birth (creation)",
12609 "Type": "RFC 3339",
12610 "Example": "2006-01-02T15:04:05.999999999Z07:00"
12611 },
12612 "gid": {
12613 "Help": "Group ID of owner",
12614 "Type": "decimal number",
12615 "Example": "500"
12616 },
12617 "mode": {
12618 "Help": "File type and mode",
12619 "Type": "octal, unix style",
12620 "Example": "0100664"
12621 },
12622 "mtime": {
12623 "Help": "Time of last modification",
12624 "Type": "RFC 3339",
12625 "Example": "2006-01-02T15:04:05.999999999Z07:00"
12626 },
12627 "rdev": {
12628 "Help": "Device ID (if special file)",
12629 "Type": "hexadecimal",
12630 "Example": "1abc"
12631 },
12632 "uid": {
12633 "Help": "User ID of owner",
12634 "Type": "decimal number",
12635 "Example": "500"
12636 }
12637 },
12638 "Help": "Textual help string\n"
12639 }
12640 }
12641
12642 This command does not have a command line equivalent so use this in‐
12643 stead:
12644
12645 rclone rc --loopback operations/fsinfo fs=remote:
12646
12647 operations/list: List the given remote and path in JSON format
12648 This takes the following parameters:
12649
12650 • fs - a remote name string e.g. "drive:"
12651
12652 • remote - a path within that remote e.g. "dir"
12653
12654 • opt - a dictionary of options to control the listing (optional)
12655
12656 • recurse - If set recurse directories
12657
12658 • noModTime - If set return modification time
12659
12660 • showEncrypted - If set show decrypted names
12661
12662 • showOrigIDs - If set show the IDs for each item if known
12663
12664 • showHash - If set return a dictionary of hashes
12665
12666 • noMimeType - If set don't show mime types
12667
12668 • dirsOnly - If set only show directories
12669
12670 • filesOnly - If set only show files
12671
12672 • metadata - If set return metadata of objects also
12673
12674 • hashTypes - array of strings of hash types to show if showHash set
12675
12676 Returns:
12677
12678 • list
12679
12680 • This is an array of objects as described in the lsjson command
12681
12682 See the lsjson (https://rclone.org/commands/rclone_lsjson/) command for
12683 more information on the above and examples.
12684
12685 Authentication is required for this call.
12686
12687 operations/mkdir: Make a destination directory or container
12688 This takes the following parameters:
12689
12690 • fs - a remote name string e.g. "drive:"
12691
12692 • remote - a path within that remote e.g. "dir"
12693
12694 See the mkdir (https://rclone.org/commands/rclone_mkdir/) command for
12695 more information on the above.
12696
12697 Authentication is required for this call.
12698
12699 operations/movefile: Move a file from source remote to destination remote
12700 This takes the following parameters:
12701
12702 • srcFs - a remote name string e.g. "drive:" for the source
12703
12704 • srcRemote - a path within that remote e.g. "file.txt" for the source
12705
12706 • dstFs - a remote name string e.g. "drive2:" for the destination
12707
12708 • dstRemote - a path within that remote e.g. "file2.txt" for the des‐
12709 tination
12710
12711 Authentication is required for this call.
12712
12713 operations/publiclink: Create or retrieve a public link to the given file
12714 or folder.
12715 This takes the following parameters:
12716
12717 • fs - a remote name string e.g. "drive:"
12718
12719 • remote - a path within that remote e.g. "dir"
12720
12721 • unlink - boolean - if set removes the link rather than adding it (op‐
12722 tional)
12723
12724 • expire - string - the expiry time of the link e.g. "1d" (optional)
12725
12726 Returns:
12727
12728 • url - URL of the resource
12729
12730 See the link (https://rclone.org/commands/rclone_link/) command for
12731 more information on the above.
12732
12733 Authentication is required for this call.
12734
12735 operations/purge: Remove a directory or container and all of its contents
12736 This takes the following parameters:
12737
12738 • fs - a remote name string e.g. "drive:"
12739
12740 • remote - a path within that remote e.g. "dir"
12741
12742 See the purge (https://rclone.org/commands/rclone_purge/) command for
12743 more information on the above.
12744
12745 Authentication is required for this call.
12746
12747 operations/rmdir: Remove an empty directory or container
12748 This takes the following parameters:
12749
12750 • fs - a remote name string e.g. "drive:"
12751
12752 • remote - a path within that remote e.g. "dir"
12753
12754 See the rmdir (https://rclone.org/commands/rclone_rmdir/) command for
12755 more information on the above.
12756
12757 Authentication is required for this call.
12758
12759 operations/rmdirs: Remove all the empty directories in the path
12760 This takes the following parameters:
12761
12762 • fs - a remote name string e.g. "drive:"
12763
12764 • remote - a path within that remote e.g. "dir"
12765
12766 • leaveRoot - boolean, set to true not to delete the root
12767
12768 See the rmdirs (https://rclone.org/commands/rclone_rmdirs/) command for
12769 more information on the above.
12770
12771 Authentication is required for this call.
12772
12773 operations/size: Count the number of bytes and files in remote
12774 This takes the following parameters:
12775
12776 • fs - a remote name string e.g. "drive:path/to/dir"
12777
12778 Returns:
12779
12780 • count - number of files
12781
12782 • bytes - number of bytes in those files
12783
12784 See the size (https://rclone.org/commands/rclone_size/) command for
12785 more information on the above.
12786
12787 Authentication is required for this call.
12788
12789 operations/stat: Give information about the supplied file or directory
12790 This takes the following parameters
12791
12792 • fs - a remote name string eg "drive:"
12793
12794 • remote - a path within that remote eg "dir"
12795
12796 • opt - a dictionary of options to control the listing (optional)
12797
12798 • see operations/list for the options
12799
12800 The result is
12801
12802 • item - an object as described in the lsjson command. Will be null if
12803 not found.
12804
12805 Note that if you are only interested in files then it is much more ef‐
12806 ficient to set the filesOnly flag in the options.
12807
12808 See the lsjson (https://rclone.org/commands/rclone_lsjson/) command for
12809 more information on the above and examples.
12810
12811 Authentication is required for this call.
12812
12813 operations/uploadfile: Upload file using multiform/form-data
12814 This takes the following parameters:
12815
12816 • fs - a remote name string e.g. "drive:"
12817
12818 • remote - a path within that remote e.g. "dir"
12819
12820 • each part in body represents a file to be uploaded
12821
12822 See the uploadfile (https://rclone.org/commands/rclone_uploadfile/)
12823 command for more information on the above.
12824
12825 Authentication is required for this call.
12826
12827 options/blocks: List all the option blocks
12828 Returns: - options - a list of the options block names
12829
12830 options/get: Get all the global options
12831 Returns an object where keys are option block names and values are an
12832 object with the current option values in.
12833
12834 Note that these are the global options which are unaffected by use of
12835 the _config and _filter parameters. If you wish to read the parameters
12836 set in _config then use options/config and for _filter use options/fil‐
12837 ter.
12838
12839 This shows the internal names of the option within rclone which should
12840 map to the external options very easily with a few exceptions.
12841
12842 options/local: Get the currently active config for this call
12843 Returns an object with the keys "config" and "filter". The "config"
12844 key contains the local config and the "filter" key contains the local
12845 filters.
12846
12847 Note that these are the local options specific to this rc call. If
12848 _config was not supplied then they will be the global options. Like‐
12849 wise with "_filter".
12850
12851 This call is mostly useful for seeing if _config and _filter passing is
12852 working.
12853
12854 This shows the internal names of the option within rclone which should
12855 map to the external options very easily with a few exceptions.
12856
12857 options/set: Set an option
12858 Parameters:
12859
12860 • option block name containing an object with
12861
12862 • key: value
12863
12864 Repeated as often as required.
12865
12866 Only supply the options you wish to change. If an option is unknown it
12867 will be silently ignored. Not all options will have an effect when
12868 changed like this.
12869
12870 For example:
12871
12872 This sets DEBUG level logs (-vv) (these can be set by number or string)
12873
12874 rclone rc options/set --json '{"main": {"LogLevel": "DEBUG"}}'
12875 rclone rc options/set --json '{"main": {"LogLevel": 8}}'
12876
12877 And this sets INFO level logs (-v)
12878
12879 rclone rc options/set --json '{"main": {"LogLevel": "INFO"}}'
12880
12881 And this sets NOTICE level logs (normal without -v)
12882
12883 rclone rc options/set --json '{"main": {"LogLevel": "NOTICE"}}'
12884
12885 pluginsctl/addPlugin: Add a plugin using url
12886 Used for adding a plugin to the webgui.
12887
12888 This takes the following parameters:
12889
12890 • url - http url of the github repo where the plugin is hosted
12891 (http://github.com/rclone/rclone-webui-react).
12892
12893 Example:
12894
12895 rclone rc pluginsctl/addPlugin
12896
12897 Authentication is required for this call.
12898
12899 pluginsctl/getPluginsForType: Get plugins with type criteria
12900 This shows all possible plugins by a mime type.
12901
12902 This takes the following parameters:
12903
12904 • type - supported mime type by a loaded plugin e.g. (video/mp4, au‐
12905 dio/mp3).
12906
12907 • pluginType - filter plugins based on their type e.g. (DASHBOARD,
12908 FILE_HANDLER, TERMINAL).
12909
12910 Returns:
12911
12912 • loadedPlugins - list of current production plugins.
12913
12914 • testPlugins - list of temporarily loaded development plugins, usually
12915 running on a different server.
12916
12917 Example:
12918
12919 rclone rc pluginsctl/getPluginsForType type=video/mp4
12920
12921 Authentication is required for this call.
12922
12923 pluginsctl/listPlugins: Get the list of currently loaded plugins
12924 This allows you to get the currently enabled plugins and their details.
12925
12926 This takes no parameters and returns:
12927
12928 • loadedPlugins - list of current production plugins.
12929
12930 • testPlugins - list of temporarily loaded development plugins, usually
12931 running on a different server.
12932
12933 E.g.
12934
12935 rclone rc pluginsctl/listPlugins
12936
12937 Authentication is required for this call.
12938
12939 pluginsctl/listTestPlugins: Show currently loaded test plugins
12940 Allows listing of test plugins with the rclone.test set to true in
12941 package.json of the plugin.
12942
12943 This takes no parameters and returns:
12944
12945 • loadedTestPlugins - list of currently available test plugins.
12946
12947 E.g.
12948
12949 rclone rc pluginsctl/listTestPlugins
12950
12951 Authentication is required for this call.
12952
12953 pluginsctl/removePlugin: Remove a loaded plugin
12954 This allows you to remove a plugin using it's name.
12955
12956 This takes parameters:
12957
12958 • name - name of the plugin in the format author/plugin_name.
12959
12960 E.g.
12961
12962 rclone rc pluginsctl/removePlugin name=rclone/video-plugin
12963
12964 Authentication is required for this call.
12965
12966 pluginsctl/removeTestPlugin: Remove a test plugin
12967 This allows you to remove a plugin using it's name.
12968
12969 This takes the following parameters:
12970
12971 • name - name of the plugin in the format author/plugin_name.
12972
12973 Example:
12974
12975 rclone rc pluginsctl/removeTestPlugin name=rclone/rclone-webui-react
12976
12977 Authentication is required for this call.
12978
12979 rc/error: This returns an error
12980 This returns an error with the input as part of its error string. Use‐
12981 ful for testing error handling.
12982
12983 rc/list: List all the registered remote control commands
12984 This lists all the registered remote control commands as a JSON map in
12985 the commands response.
12986
12987 rc/noop: Echo the input to the output parameters
12988 This echoes the input parameters to the output parameters for testing
12989 purposes. It can be used to check that rclone is still alive and to
12990 check that parameter passing is working properly.
12991
12992 rc/noopauth: Echo the input to the output parameters requiring auth
12993 This echoes the input parameters to the output parameters for testing
12994 purposes. It can be used to check that rclone is still alive and to
12995 check that parameter passing is working properly.
12996
12997 Authentication is required for this call.
12998
12999 sync/bisync: Perform bidirectional synchronization between two paths.
13000 This takes the following parameters
13001
13002 • path1 - a remote directory string e.g. drive:path1
13003
13004 • path2 - a remote directory string e.g. drive:path2
13005
13006 • dryRun - dry-run mode
13007
13008 • resync - performs the resync run
13009
13010 • checkAccess - abort if RCLONE_TEST files are not found on both
13011 filesystems
13012
13013 • checkFilename - file name for checkAccess (default: RCLONE_TEST)
13014
13015 • maxDelete - abort sync if percentage of deleted files is above this
13016 threshold (default: 50)
13017
13018 • force - maxDelete safety check and run the sync
13019
13020 • checkSync - true by default, false disables comparison of final list‐
13021 ings, only will skip sync, only compare listings from the last run
13022
13023 • removeEmptyDirs - remove empty directories at the final cleanup step
13024
13025 • filtersFile - read filtering patterns from a file
13026
13027 • workdir - server directory for history files (default:
13028 /home/ncw/.cache/rclone/bisync)
13029
13030 • noCleanup - retain working files
13031
13032 See bisync command help (https://rclone.org/commands/rclone_bisync/)
13033 and full bisync description (https://rclone.org/bisync/) for more in‐
13034 formation.
13035
13036 Authentication is required for this call.
13037
13038 sync/copy: copy a directory from source remote to destination remote
13039 This takes the following parameters:
13040
13041 • srcFs - a remote name string e.g. "drive:src" for the source
13042
13043 • dstFs - a remote name string e.g. "drive:dst" for the destination
13044
13045 • createEmptySrcDirs - create empty src directories on destination if
13046 set
13047
13048 See the copy (https://rclone.org/commands/rclone_copy/) command for
13049 more information on the above.
13050
13051 Authentication is required for this call.
13052
13053 sync/move: move a directory from source remote to destination remote
13054 This takes the following parameters:
13055
13056 • srcFs - a remote name string e.g. "drive:src" for the source
13057
13058 • dstFs - a remote name string e.g. "drive:dst" for the destination
13059
13060 • createEmptySrcDirs - create empty src directories on destination if
13061 set
13062
13063 • deleteEmptySrcDirs - delete empty src directories if set
13064
13065 See the move (https://rclone.org/commands/rclone_move/) command for
13066 more information on the above.
13067
13068 Authentication is required for this call.
13069
13070 sync/sync: sync a directory from source remote to destination remote
13071 This takes the following parameters:
13072
13073 • srcFs - a remote name string e.g. "drive:src" for the source
13074
13075 • dstFs - a remote name string e.g. "drive:dst" for the destination
13076
13077 • createEmptySrcDirs - create empty src directories on destination if
13078 set
13079
13080 See the sync (https://rclone.org/commands/rclone_sync/) command for
13081 more information on the above.
13082
13083 Authentication is required for this call.
13084
13085 vfs/forget: Forget files or directories in the directory cache.
13086 This forgets the paths in the directory cache causing them to be re-
13087 read from the remote when needed.
13088
13089 If no paths are passed in then it will forget all the paths in the di‐
13090 rectory cache.
13091
13092 rclone rc vfs/forget
13093
13094 Otherwise pass files or dirs in as file=path or dir=path. Any parame‐
13095 ter key starting with file will forget that file and any starting with
13096 dir will forget that dir, e.g.
13097
13098 rclone rc vfs/forget file=hello file2=goodbye dir=home/junk
13099
13100 This command takes an "fs" parameter. If this parameter is not sup‐
13101 plied and if there is only one VFS in use then that VFS will be used.
13102 If there is more than one VFS in use then the "fs" parameter must be
13103 supplied.
13104
13105 vfs/list: List active VFSes.
13106 This lists the active VFSes.
13107
13108 It returns a list under the key "vfses" where the values are the VFS
13109 names that could be passed to the other VFS commands in the "fs" param‐
13110 eter.
13111
13112 vfs/poll-interval: Get the status or update the value of the poll-interval
13113 option.
13114 Without any parameter given this returns the current status of the
13115 poll-interval setting.
13116
13117 When the interval=duration parameter is set, the poll-interval value is
13118 updated and the polling function is notified. Setting interval=0 dis‐
13119 ables poll-interval.
13120
13121 rclone rc vfs/poll-interval interval=5m
13122
13123 The timeout=duration parameter can be used to specify a time to wait
13124 for the current poll function to apply the new value. If timeout is
13125 less or equal 0, which is the default, wait indefinitely.
13126
13127 The new poll-interval value will only be active when the timeout is not
13128 reached.
13129
13130 If poll-interval is updated or disabled temporarily, some changes might
13131 not get picked up by the polling function, depending on the used re‐
13132 mote.
13133
13134 This command takes an "fs" parameter. If this parameter is not sup‐
13135 plied and if there is only one VFS in use then that VFS will be used.
13136 If there is more than one VFS in use then the "fs" parameter must be
13137 supplied.
13138
13139 vfs/refresh: Refresh the directory cache.
13140 This reads the directories for the specified paths and freshens the di‐
13141 rectory cache.
13142
13143 If no paths are passed in then it will refresh the root directory.
13144
13145 rclone rc vfs/refresh
13146
13147 Otherwise pass directories in as dir=path. Any parameter key starting
13148 with dir will refresh that directory, e.g.
13149
13150 rclone rc vfs/refresh dir=home/junk dir2=data/misc
13151
13152 If the parameter recursive=true is given the whole directory tree will
13153 get refreshed. This refresh will use --fast-list if enabled.
13154
13155 This command takes an "fs" parameter. If this parameter is not sup‐
13156 plied and if there is only one VFS in use then that VFS will be used.
13157 If there is more than one VFS in use then the "fs" parameter must be
13158 supplied.
13159
13160 vfs/stats: Stats for a VFS.
13161 This returns stats for the selected VFS.
13162
13163 {
13164 // Status of the disk cache - only present if --vfs-cache-mode > off
13165 "diskCache": {
13166 "bytesUsed": 0,
13167 "erroredFiles": 0,
13168 "files": 0,
13169 "hashType": 1,
13170 "outOfSpace": false,
13171 "path": "/home/user/.cache/rclone/vfs/local/mnt/a",
13172 "pathMeta": "/home/user/.cache/rclone/vfsMeta/local/mnt/a",
13173 "uploadsInProgress": 0,
13174 "uploadsQueued": 0
13175 },
13176 "fs": "/mnt/a",
13177 "inUse": 1,
13178 // Status of the in memory metadata cache
13179 "metadataCache": {
13180 "dirs": 1,
13181 "files": 0
13182 },
13183 // Options as returned by options/get
13184 "opt": {
13185 "CacheMaxAge": 3600000000000,
13186 // ...
13187 "WriteWait": 1000000000
13188 }
13189 }
13190
13191 This command takes an "fs" parameter. If this parameter is not sup‐
13192 plied and if there is only one VFS in use then that VFS will be used.
13193 If there is more than one VFS in use then the "fs" parameter must be
13194 supplied.
13195
13196 Accessing the remote control via HTTP
13197 Rclone implements a simple HTTP based protocol.
13198
13199 Each endpoint takes an JSON object and returns a JSON object or an er‐
13200 ror. The JSON objects are essentially a map of string names to values.
13201
13202 All calls must made using POST.
13203
13204 The input objects can be supplied using URL parameters, POST parameters
13205 or by supplying "Content-Type: application/json" and a JSON blob in the
13206 body. There are examples of these below using curl.
13207
13208 The response will be a JSON blob in the body of the response. This is
13209 formatted to be reasonably human-readable.
13210
13211 Error returns
13212 If an error occurs then there will be an HTTP error status (e.g. 500)
13213 and the body of the response will contain a JSON encoded error object,
13214 e.g.
13215
13216 {
13217 "error": "Expecting string value for key \"remote\" (was float64)",
13218 "input": {
13219 "fs": "/tmp",
13220 "remote": 3
13221 },
13222 "status": 400
13223 "path": "operations/rmdir",
13224 }
13225
13226 The keys in the error response are - error - error string - input - the
13227 input parameters to the call - status - the HTTP status code - path -
13228 the path of the call
13229
13230 CORS
13231 The sever implements basic CORS support and allows all origins for
13232 that. The response to a preflight OPTIONS request will echo the re‐
13233 quested "Access-Control-Request-Headers" back.
13234
13235 Using POST with URL parameters only
13236 curl -X POST 'http://localhost:5572/rc/noop?potato=1&sausage=2'
13237
13238 Response
13239
13240 {
13241 "potato": "1",
13242 "sausage": "2"
13243 }
13244
13245 Here is what an error response looks like:
13246
13247 curl -X POST 'http://localhost:5572/rc/error?potato=1&sausage=2'
13248
13249 {
13250 "error": "arbitrary error on input map[potato:1 sausage:2]",
13251 "input": {
13252 "potato": "1",
13253 "sausage": "2"
13254 }
13255 }
13256
13257 Note that curl doesn't return errors to the shell unless you use the -f
13258 option
13259
13260 $ curl -f -X POST 'http://localhost:5572/rc/error?potato=1&sausage=2'
13261 curl: (22) The requested URL returned error: 400 Bad Request
13262 $ echo $?
13263 22
13264
13265 Using POST with a form
13266 curl --data "potato=1" --data "sausage=2" http://localhost:5572/rc/noop
13267
13268 Response
13269
13270 {
13271 "potato": "1",
13272 "sausage": "2"
13273 }
13274
13275 Note that you can combine these with URL parameters too with the POST
13276 parameters taking precedence.
13277
13278 curl --data "potato=1" --data "sausage=2" "http://localhost:5572/rc/noop?rutabaga=3&sausage=4"
13279
13280 Response
13281
13282 {
13283 "potato": "1",
13284 "rutabaga": "3",
13285 "sausage": "4"
13286 }
13287
13288 Using POST with a JSON blob
13289 curl -H "Content-Type: application/json" -X POST -d '{"potato":2,"sausage":1}' http://localhost:5572/rc/noop
13290
13291 response
13292
13293 {
13294 "password": "xyz",
13295 "username": "xyz"
13296 }
13297
13298 This can be combined with URL parameters too if required. The JSON
13299 blob takes precedence.
13300
13301 curl -H "Content-Type: application/json" -X POST -d '{"potato":2,"sausage":1}' 'http://localhost:5572/rc/noop?rutabaga=3&potato=4'
13302
13303 {
13304 "potato": 2,
13305 "rutabaga": "3",
13306 "sausage": 1
13307 }
13308
13309 Debugging rclone with pprof
13310 If you use the --rc flag this will also enable the use of the go pro‐
13311 filing tools on the same port.
13312
13313 To use these, first install go (https://golang.org/doc/install).
13314
13315 Debugging memory use
13316 To profile rclone's memory use you can run:
13317
13318 go tool pprof -web http://localhost:5572/debug/pprof/heap
13319
13320 This should open a page in your browser showing what is using what mem‐
13321 ory.
13322
13323 You can also use the -text flag to produce a textual summary
13324
13325 $ go tool pprof -text http://localhost:5572/debug/pprof/heap
13326 Showing nodes accounting for 1537.03kB, 100% of 1537.03kB total
13327 flat flat% sum% cum cum%
13328 1024.03kB 66.62% 66.62% 1024.03kB 66.62% github.com/rclone/rclone/vendor/golang.org/x/net/http2/hpack.addDecoderNode
13329 513kB 33.38% 100% 513kB 33.38% net/http.newBufioWriterSize
13330 0 0% 100% 1024.03kB 66.62% github.com/rclone/rclone/cmd/all.init
13331 0 0% 100% 1024.03kB 66.62% github.com/rclone/rclone/cmd/serve.init
13332 0 0% 100% 1024.03kB 66.62% github.com/rclone/rclone/cmd/serve/restic.init
13333 0 0% 100% 1024.03kB 66.62% github.com/rclone/rclone/vendor/golang.org/x/net/http2.init
13334 0 0% 100% 1024.03kB 66.62% github.com/rclone/rclone/vendor/golang.org/x/net/http2/hpack.init
13335 0 0% 100% 1024.03kB 66.62% github.com/rclone/rclone/vendor/golang.org/x/net/http2/hpack.init.0
13336 0 0% 100% 1024.03kB 66.62% main.init
13337 0 0% 100% 513kB 33.38% net/http.(*conn).readRequest
13338 0 0% 100% 513kB 33.38% net/http.(*conn).serve
13339 0 0% 100% 1024.03kB 66.62% runtime.main
13340
13341 Debugging go routine leaks
13342 Memory leaks are most often caused by go routine leaks keeping memory
13343 alive which should have been garbage collected.
13344
13345 See all active go routines using
13346
13347 curl http://localhost:5572/debug/pprof/goroutine?debug=1
13348
13349 Or go to http://localhost:5572/debug/pprof/goroutine?debug=1 in your
13350 browser.
13351
13352 Other profiles to look at
13353 You can see a summary of profiles available at http://local‐
13354 host:5572/debug/pprof/
13355
13356 Here is how to use some of them:
13357
13358 • Memory: go tool pprof http://localhost:5572/debug/pprof/heap
13359
13360 • Go routines: curl http://localhost:5572/debug/pprof/goroutine?debug=1
13361
13362 • 30-second CPU profile: go tool pprof http://localhost:5572/de‐
13363 bug/pprof/profile
13364
13365 • 5-second execution trace: wget http://localhost:5572/de‐
13366 bug/pprof/trace?seconds=5
13367
13368 • Goroutine blocking profile
13369
13370 • Enable first with: rclone rc debug/set-block-profile-rate rate=1
13371 (docs)
13372
13373 • go tool pprof http://localhost:5572/debug/pprof/block
13374
13375 • Contended mutexes:
13376
13377 • Enable first with: rclone rc debug/set-mutex-profile-fraction
13378 rate=1 (docs)
13379
13380 • go tool pprof http://localhost:5572/debug/pprof/mutex
13381
13382 See the net/http/pprof docs (https://golang.org/pkg/net/http/pprof/)
13383 for more info on how to use the profiling and for a general overview
13384 see the Go team's blog post on profiling go programs
13385 (https://blog.golang.org/profiling-go-programs).
13386
13387 The profiling hook is zero overhead unless it is used (https://stack‐
13388 overflow.com/q/26545159/164234).
13389
13391 Each cloud storage system is slightly different. Rclone attempts to
13392 provide a unified interface to them, but some underlying differences
13393 show through.
13394
13395 Features
13396 Here is an overview of the major features of each cloud storage system.
13397
13398 Name Hash ModTime Case In‐ Duplicate MIME Type Metadata
13399 sensitive Files
13400 ──────────────────────────────────────────────────────────────────────────────────
13401 1Fichier Whirlpool - No Yes R -
13402 Akamai MD5, R/W No No R -
13403 Netstorage SHA256
13404 Amazon MD5 - Yes No R -
13405 Drive
13406 Amazon S3 MD5 R/W No No R/W RWU
13407 (or S3
13408 compati‐
13409 ble)
13410 Backblaze SHA1 R/W No No R/W -
13411 B2
13412 Box SHA1 R/W Yes No - -
13413 Citrix MD5 R/W Yes No - -
13414 ShareFile
13415 Dropbox DBHASH ¹ R Yes No - -
13416 Enterprise - R/W Yes No R/W -
13417 File Fab‐
13418 ric
13419 FTP - R/W ¹⁰ No No - -
13420 Google MD5 R/W No No R/W -
13421 Cloud
13422 Storage
13423 Google MD5 R/W No Yes R/W -
13424 Drive
13425 Google - - No Yes R -
13426 Photos
13427 HDFS - R/W No No - -
13428 HiDrive HiDrive ¹² R/W No No - -
13429 HTTP - R No No R -
13430 Internet MD5, SHA1, R/W ¹¹ No No - RWU
13431 Archive CRC32
13432 Jottacloud MD5 R/W Yes No R -
13433 Koofr MD5 - Yes No - -
13434 Mail.ru Mailru ⁶ R/W Yes No - -
13435 Cloud
13436 Mega - - No Yes - -
13437 Memory MD5 R/W No No - -
13438 Microsoft MD5 R/W No No R/W -
13439 Azure Blob
13440 Storage
13441 Microsoft SHA1 ⁵ R/W Yes No R -
13442 OneDrive
13443 OpenDrive MD5 R/W Yes Partial ⁸ - -
13444 OpenStack MD5 R/W No No R/W -
13445 Swift
13446 Oracle Ob‐ MD5 R/W No No R/W -
13447 ject Stor‐
13448 age
13449 pCloud MD5, SHA1 R No No W -
13450 ⁷
13451 premiu‐ - - Yes No R -
13452 mize.me
13453 put.io CRC-32 R/W No Yes R -
13454 QingStor MD5 - ⁹ No No R/W -
13455 Seafile - - No No - -
13456 SFTP MD5, SHA1 R/W Depends No - -
13457 ²
13458 Sia - - No No - -
13459 SMB - - Yes No - -
13460 SugarSync - - No No - -
13461 Storj - R No No - -
13462 Uptobox - - No Yes - -
13463 WebDAV MD5, SHA1 R ⁴ Depends No - -
13464 ³
13465 Yandex MD5 R/W No No R -
13466 Disk
13467 Zoho Work‐ - - No No - -
13468 Drive
13469 The local All R/W Depends No - RWU
13470 filesystem
13471
13472 Notes
13473 ¹ Dropbox supports its own custom hash (https://www.dropbox.com/devel‐
13474 opers/reference/content-hash). This is an SHA256 sum of all the 4 MiB
13475 block SHA256s.
13476
13477 ² SFTP supports checksums if the same login has shell access and md5sum
13478 or sha1sum as well as echo are in the remote's PATH.
13479
13480 ³ WebDAV supports hashes when used with Owncloud and Nextcloud only.
13481
13482 ⁴ WebDAV supports modtimes when used with Owncloud and Nextcloud only.
13483
13484 ⁵ Microsoft OneDrive Personal supports SHA1 hashes, whereas OneDrive
13485 for business and SharePoint server support Microsoft's own QuickXorHash
13486 (https://docs.microsoft.com/en-us/onedrive/developer/code-snip‐
13487 pets/quickxorhash).
13488
13489 ⁶ Mail.ru uses its own modified SHA1 hash
13490
13491 ⁷ pCloud only supports SHA1 (not MD5) in its EU region
13492
13493 ⁸ Opendrive does not support creation of duplicate files using their
13494 web client interface or other stock clients, but the underlying storage
13495 platform has been determined to allow duplicate files, and it is possi‐
13496 ble to create them with rclone. It may be that this is a mistake or an
13497 unsupported feature.
13498
13499 ⁹ QingStor does not support SetModTime for objects bigger than 5 GiB.
13500
13501 ¹⁰ FTP supports modtimes for the major FTP servers, and also others if
13502 they advertised required protocol extensions. See this
13503 (https://rclone.org/ftp/#modified-time) for more details.
13504
13505 ¹¹ Internet Archive requires option wait_archive to be set to a non-ze‐
13506 ro value for full modtime support.
13507
13508 ¹² HiDrive supports its own custom hash (https://stat‐
13509 ic.hidrive.com/dev/0001). It combines SHA1 sums for each 4 KiB block
13510 hierarchically to a single top-level sum.
13511
13512 Hash
13513 The cloud storage system supports various hash types of the objects.
13514 The hashes are used when transferring data as an integrity check and
13515 can be specifically used with the --checksum flag in syncs and in the
13516 check command.
13517
13518 To use the verify checksums when transferring between cloud storage
13519 systems they must support a common hash type.
13520
13521 ModTime
13522 Almost all cloud storage systems store some sort of timestamp on ob‐
13523 jects, but several of them not something that is appropriate to use for
13524 syncing. E.g. some backends will only write a timestamp that repre‐
13525 sent the time of the upload. To be relevant for syncing it should be
13526 able to store the modification time of the source object. If this is
13527 not the case, rclone will only check the file size by default, though
13528 can be configured to check the file hash (with the --checksum flag).
13529 Ideally it should also be possible to change the timestamp of an exist‐
13530 ing file without having to re-upload it.
13531
13532 Storage systems with a - in the ModTime column, means the modification
13533 read on objects is not the modification time of the file when uploaded.
13534 It is most likely the time the file was uploaded, or possibly something
13535 else (like the time the picture was taken in Google Photos).
13536
13537 Storage systems with a R (for read-only) in the ModTime column, means
13538 the it keeps modification times on objects, and updates them when up‐
13539 loading objects, but it does not support changing only the modification
13540 time (SetModTime operation) without re-uploading, possibly not even
13541 without deleting existing first. Some operations in rclone, such as
13542 copy and sync commands, will automatically check for SetModTime support
13543 and re-upload if necessary to keep the modification times in sync.
13544 Other commands will not work without SetModTime support, e.g. touch
13545 command on an existing file will fail, and changes to modification time
13546 only on a files in a mount will be silently ignored.
13547
13548 Storage systems with R/W (for read/write) in the ModTime column, means
13549 they do also support modtime-only operations.
13550
13551 Case Insensitive
13552 If a cloud storage systems is case sensitive then it is possible to
13553 have two files which differ only in case, e.g. file.txt and FILE.txt.
13554 If a cloud storage system is case insensitive then that isn't possible.
13555
13556 This can cause problems when syncing between a case insensitive system
13557 and a case sensitive system. The symptom of this is that no matter how
13558 many times you run the sync it never completes fully.
13559
13560 The local filesystem and SFTP may or may not be case sensitive depend‐
13561 ing on OS.
13562
13563 • Windows - usually case insensitive, though case is preserved
13564
13565 • OSX - usually case insensitive, though it is possible to format case
13566 sensitive
13567
13568 • Linux - usually case sensitive, but there are case insensitive file
13569 systems (e.g. FAT formatted USB keys)
13570
13571 Most of the time this doesn't cause any problems as people tend to
13572 avoid files whose name differs only by case even on case sensitive sys‐
13573 tems.
13574
13575 Duplicate files
13576 If a cloud storage system allows duplicate files then it can have two
13577 objects with the same name.
13578
13579 This confuses rclone greatly when syncing - use the rclone dedupe com‐
13580 mand to rename or remove duplicates.
13581
13582 Restricted filenames
13583 Some cloud storage systems might have restrictions on the characters
13584 that are usable in file or directory names. When rclone detects such a
13585 name during a file upload, it will transparently replace the restricted
13586 characters with similar looking Unicode characters. To handle the dif‐
13587 ferent sets of restricted characters for different backends, rclone us‐
13588 es something it calls encoding.
13589
13590 This process is designed to avoid ambiguous file names as much as pos‐
13591 sible and allow to move files between many cloud storage systems trans‐
13592 parently.
13593
13594 The name shown by rclone to the user or during log output will only
13595 contain a minimal set of replaced characters to ensure correct format‐
13596 ting and not necessarily the actual name used on the cloud storage.
13597
13598 This transformation is reversed when downloading a file or parsing
13599 rclone arguments. For example, when uploading a file named my
13600 file?.txt to Onedrive, it will be displayed as my file?.txt on the con‐
13601 sole, but stored as my file?.txt to Onedrive (the ? gets replaced by
13602 the similar looking ? character, the so-called "fullwidth question
13603 mark"). The reverse transformation allows to read a file unusu‐
13604 al/name.txt from Google Drive, by passing the name unusual/name.txt on
13605 the command line (the / needs to be replaced by the similar looking /
13606 character).
13607
13608 Caveats
13609 The filename encoding system works well in most cases, at least where
13610 file names are written in English or similar languages. You might not
13611 even notice it: It just works. In some cases it may lead to issues,
13612 though. E.g. when file names are written in Chinese, or Japanese,
13613 where it is always the Unicode fullwidth variants of the punctuation
13614 marks that are used.
13615
13616 On Windows, the characters :, * and ? are examples of restricted char‐
13617 acters. If these are used in filenames on a remote that supports it,
13618 Rclone will transparently convert them to their fullwidth Unicode vari‐
13619 ants *, ? and : when downloading to Windows, and back again when up‐
13620 loading. This way files with names that are not allowed on Windows can
13621 still be stored.
13622
13623 However, if you have files on your Windows system originally with these
13624 same Unicode characters in their names, they will be included in the
13625 same conversion process. E.g. if you create a file in your Windows
13626 filesystem with name Test:1.jpg, where : is the Unicode fullwidth
13627 colon symbol, and use rclone to upload it to Google Drive, which sup‐
13628 ports regular : (halfwidth question mark), rclone will replace the
13629 fullwidth : with the halfwidth : and store the file as Test:1.jpg in
13630 Google Drive. Since both Windows and Google Drive allows the name
13631 Test:1.jpg, it would probably be better if rclone just kept the name
13632 as is in this case.
13633
13634 With the opposite situation; if you have a file named Test:1.jpg, in
13635 your Google Drive, e.g. uploaded from a Linux system where : is valid
13636 in file names. Then later use rclone to copy this file to your Windows
13637 computer you will notice that on your local disk it gets renamed to
13638 Test:1.jpg. The original filename is not legal on Windows, due to the
13639 :, and rclone therefore renames it to make the copy possible. That is
13640 all good. However, this can also lead to an issue: If you already had
13641 a different file named Test:1.jpg on Windows, and then use rclone to
13642 copy either way. Rclone will then treat the file originally named
13643 Test:1.jpg on Google Drive and the file originally named Test:1.jpg on
13644 Windows as the same file, and replace the contents from one with the
13645 other.
13646
13647 Its virtually impossible to handle all cases like these correctly in
13648 all situations, but by customizing the encoding option, changing the
13649 set of characters that rclone should convert, you should be able to
13650 create a configuration that works well for your specific situation.
13651 See also the example (https://rclone.org/overview/#encoding-example-
13652 windows) below.
13653
13654 (Windows was used as an example of a file system with many restricted
13655 characters, and Google drive a storage system with few.)
13656
13657 Default restricted characters
13658 The table below shows the characters that are replaced by default.
13659
13660 When a replacement character is found in a filename, this character
13661 will be escaped with the ‛ character to avoid ambiguous file names.
13662 (e.g. a file named ␀.txt would shown as ‛␀.txt)
13663
13664 Each cloud storage backend can use a different set of characters, which
13665 will be specified in the documentation for each backend.
13666
13667 Character Value Replacement
13668 ────────────────────────────────
13669 NUL 0x00 ␀
13670 SOH 0x01 ␁
13671 STX 0x02 ␂
13672 ETX 0x03 ␃
13673 EOT 0x04 ␄
13674 ENQ 0x05 ␅
13675 ACK 0x06 ␆
13676 BEL 0x07 ␇
13677 BS 0x08 ␈
13678 HT 0x09 ␉
13679 LF 0x0A ␊
13680 VT 0x0B ␋
13681 FF 0x0C ␌
13682 CR 0x0D ␍
13683 SO 0x0E ␎
13684 SI 0x0F ␏
13685 DLE 0x10 ␐
13686 DC1 0x11 ␑
13687 DC2 0x12 ␒
13688 DC3 0x13 ␓
13689 DC4 0x14 ␔
13690 NAK 0x15 ␕
13691 SYN 0x16 ␖
13692 ETB 0x17 ␗
13693 CAN 0x18 ␘
13694 EM 0x19 ␙
13695 SUB 0x1A ␚
13696
13697 ESC 0x1B ␛
13698 FS 0x1C ␜
13699 GS 0x1D ␝
13700 RS 0x1E ␞
13701 US 0x1F ␟
13702 / 0x2F /
13703 DEL 0x7F ␡
13704
13705 The default encoding will also encode these file names as they are
13706 problematic with many cloud storage systems.
13707
13708 File name Replacement
13709 ────────────────────────
13710 . .
13711 .. ..
13712
13713 Invalid UTF-8 bytes
13714 Some backends only support a sequence of well formed UTF-8 bytes as
13715 file or directory names.
13716
13717 In this case all invalid UTF-8 bytes will be replaced with a quoted
13718 representation of the byte value to allow uploading a file to such a
13719 backend. For example, the invalid byte 0xFE will be encoded as ‛FE.
13720
13721 A common source of invalid UTF-8 bytes are local filesystems, that
13722 store names in a different encoding than UTF-8 or UTF-16, like latin1.
13723 See the local filenames (https://rclone.org/local/#filenames) section
13724 for details.
13725
13726 Encoding option
13727 Most backends have an encoding option, specified as a flag --backend-
13728 encoding where backend is the name of the backend, or as a config pa‐
13729 rameter encoding (you'll need to select the Advanced config in rclone
13730 config to see it).
13731
13732 This will have default value which encodes and decodes characters in
13733 such a way as to preserve the maximum number of characters (see above).
13734
13735 However this can be incorrect in some scenarios, for example if you
13736 have a Windows file system with Unicode fullwidth characters *, ? or
13737 :, that you want to remain as those characters on the remote rather
13738 than being translated to regular (halfwidth) *, ? and :.
13739
13740 The --backend-encoding flags allow you to change that. You can disable
13741 the encoding completely with --backend-encoding None or set encoding =
13742 None in the config file.
13743
13744 Encoding takes a comma separated list of encodings. You can see the
13745 list of all possible values by passing an invalid value to this flag,
13746 e.g. --local-encoding "help". The command rclone help flags encoding
13747 will show you the defaults for the backends.
13748
13749 Encoding Characters Encoded as
13750 ─────────────────────────────────────────────────────────────────────────────────────
13751 Asterisk * *
13752 BackQuote ` `
13753 BackSlash \ \
13754 Colon : :
13755 CrLf CR 0x0D, LF 0x0A ␍, ␊
13756 Ctl All control characters ␀␁␂␃␄␅␆␇␈␉␊␋␌␍␎␏␐␑␒␓␔␕␖␗␘␙␚␛␜␝␞␟
13757 0x00-0x1F
13758 Del DEL 0x7F ␡
13759 Dollar $ $
13760 Dot . or .. as entire string ., ..
13761 DoubleQuote " "
13762 Hash # #
13763 InvalidUtf8 An invalid UTF-8 charac‐ �
13764 ter (e.g. latin1)
13765 LeftCrLfHtVt CR 0x0D, LF 0x0A, HT ␍, ␊, ␉, ␋
13766 0x09, VT 0x0B on the
13767 left of a string
13768 LeftPeriod . on the left of a .
13769 string
13770 LeftSpace SPACE on the left of a ␠
13771 string
13772 LeftTilde ~ on the left of a ~
13773 string
13774 LtGt <, > <, >
13775 None No characters are encod‐
13776 ed
13777 Percent % %
13778 Pipe | |
13779 Question ? ?
13780 RightCrLfHtVt CR 0x0D, LF 0x0A, HT ␍, ␊, ␉, ␋
13781 0x09, VT 0x0B on the
13782 right of a string
13783 RightPeriod . on the right of a .
13784 string
13785 RightSpace SPACE on the right of a ␠
13786 string
13787 Semicolon ; ;
13788 SingleQuote ' '
13789 Slash / /
13790 SquareBracket [, ] [, ]
13791
13792 Encoding example: FTP
13793 To take a specific example, the FTP backend's default encoding is
13794
13795 --ftp-encoding "Slash,Del,Ctl,RightSpace,Dot"
13796
13797 However, let's say the FTP server is running on Windows and can't have
13798 any of the invalid Windows characters in file names. You are backing
13799 up Linux servers to this FTP server which do have those characters in
13800 file names. So you would add the Windows set which are
13801
13802 Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,BackSlash,Ctl,RightSpace,RightPeriod,InvalidUtf8,Dot
13803
13804 to the existing ones, giving:
13805
13806 Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,BackSlash,Ctl,RightSpace,RightPeriod,InvalidUtf8,Dot,Del,RightSpace
13807
13808 This can be specified using the --ftp-encoding flag or using an encod‐
13809 ing parameter in the config file.
13810
13811 Encoding example: Windows
13812 As a nother example, take a Windows system where there is a file with
13813 name Test:1.jpg, where : is the Unicode fullwidth colon symbol. When
13814 using rclone to copy this to a remote which supports :, the regular
13815 (halfwidth) colon (such as Google Drive), you will notice that the file
13816 gets renamed to Test:1.jpg.
13817
13818 To avoid this you can change the set of characters rclone should con‐
13819 vert for the local filesystem, using command-line argument --local-en‐
13820 coding. Rclone's default behavior on Windows corresponds to
13821
13822 --local-encoding "Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,BackSlash,Ctl,RightSpace,RightPeriod,InvalidUtf8,Dot"
13823
13824 If you want to use fullwidth characters :, * and ? in your filenames
13825 without rclone changing them when uploading to a remote, then set the
13826 same as the default value but without Colon,Question,Asterisk:
13827
13828 --local-encoding "Slash,LtGt,DoubleQuote,Pipe,BackSlash,Ctl,RightSpace,RightPeriod,InvalidUtf8,Dot"
13829
13830 Alternatively, you can disable the conversion of any characters with
13831 --local-encoding None.
13832
13833 Instead of using command-line argument --local-encoding, you may also
13834 set it as environment variable (https://rclone.org/docs/#environment-
13835 variables) RCLONE_LOCAL_ENCODING, or configure
13836 (https://rclone.org/docs/#configure) a remote of type local in your
13837 config, and set the encoding option there.
13838
13839 The risk by doing this is that if you have a filename with the regular
13840 (halfwidth) :, * and ? in your cloud storage, and you try to download
13841 it to your Windows filesystem, this will fail. These characters are
13842 not valid in filenames on Windows, and you have told rclone not to work
13843 around this by converting them to valid fullwidth variants.
13844
13845 MIME Type
13846 MIME types (also known as media types) classify types of documents us‐
13847 ing a simple text classification, e.g. text/html or application/pdf.
13848
13849 Some cloud storage systems support reading (R) the MIME type of objects
13850 and some support writing (W) the MIME type of objects.
13851
13852 The MIME type can be important if you are serving files directly to
13853 HTTP from the storage system.
13854
13855 If you are copying from a remote which supports reading (R) to a remote
13856 which supports writing (W) then rclone will preserve the MIME types.
13857 Otherwise they will be guessed from the extension, or the remote itself
13858 may assign the MIME type.
13859
13860 Metadata
13861 Backends may or may support reading or writing metadata. They may sup‐
13862 port reading and writing system metadata (metadata intrinsic to that
13863 backend) and/or user metadata (general purpose metadata).
13864
13865 The levels of metadata support are
13866
13867 Key Explanation
13868 ─────────────────────────────────
13869 R Read only System Metadata
13870 RW Read and write System
13871 Metadata
13872 RWU Read and write System
13873 Metadata and read and
13874 write User Metadata
13875
13876 See the metadata docs (https://rclone.org/docs/#metadata) for more in‐
13877 fo.
13878
13879 Optional Features
13880 All rclone remotes support a base command set. Other features depend
13881 upon backend-specific capabilities.
13882
13883 Name Purge Copy Move DirMove CleanUp ListR StreamU‐ LinkShar‐ About Empty‐
13884 pload ing Dir
13885 ───────────────────────────────────────────────────────────────────────────────────────────────────
13886 1Fichi‐ No Yes Yes No No No No Yes No Yes
13887 er
13888 Akamai Yes No No No No Yes Yes No No Yes
13889 Net‐
13890 storage
13891 Amazon Yes No Yes Yes No No No No No Yes
13892 Drive
13893 Amazon No Yes No No Yes Yes Yes Yes No No
13894 S3 (or
13895 S3 com‐
13896 pati‐
13897 ble)
13898 Back‐ No Yes No No Yes Yes Yes Yes No No
13899 blaze
13900 B2
13901 Box Yes Yes Yes Yes Yes ‡‡ No Yes Yes Yes Yes
13902 Citrix Yes Yes Yes Yes No No Yes No No Yes
13903 Share‐
13904 File
13905 Dropbox Yes Yes Yes Yes No No Yes Yes Yes Yes
13906
13907 Enter‐ Yes Yes Yes Yes Yes No No No No Yes
13908 prise
13909 File
13910 Fabric
13911 FTP No No Yes Yes No No Yes No No Yes
13912 Google Yes Yes No No No Yes Yes No No No
13913 Cloud
13914 Storage
13915 Google Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes
13916 Drive
13917 Google No No No No No No No No No No
13918 Photos
13919 HDFS Yes No Yes Yes No No Yes No Yes Yes
13920 HiDrive Yes Yes Yes Yes No No Yes No No Yes
13921 HTTP No No No No No No No No No Yes
13922 Inter‐ No Yes No No Yes Yes No Yes Yes No
13923 net Ar‐
13924 chive
13925 Jotta‐ Yes Yes Yes Yes Yes Yes No Yes Yes Yes
13926 cloud
13927 Koofr Yes Yes Yes Yes No No Yes Yes Yes Yes
13928 Mail.ru Yes Yes Yes Yes Yes No No Yes Yes Yes
13929 Cloud
13930 Mega Yes No Yes Yes Yes No No Yes Yes Yes
13931 Memory No Yes No No No Yes Yes No No No
13932 Micro‐ Yes Yes No No No Yes Yes No No No
13933 soft
13934 Azure
13935 Blob
13936 Storage
13937 Micro‐ Yes Yes Yes Yes Yes No No Yes Yes Yes
13938 soft
13939 OneDrive
13940 Open‐ Yes Yes Yes Yes No No No No No Yes
13941 Drive
13942 Open‐ Yes † Yes No No No Yes Yes No Yes No
13943 Stack
13944 Swift
13945 Oracle No Yes No No Yes Yes Yes No No No
13946 Object
13947 Storage
13948 pCloud Yes Yes Yes Yes Yes No No Yes Yes Yes
13949 premiu‐ Yes No Yes Yes No No No Yes Yes Yes
13950 mize.me
13951 put.io Yes No Yes Yes Yes No Yes No Yes Yes
13952 QingStor No Yes No No Yes Yes No No No No
13953 Seafile Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes
13954 SFTP No No Yes Yes No No Yes No Yes Yes
13955 Sia No No No No No No Yes No No Yes
13956 SMB No No Yes Yes No No Yes No No Yes
13957 Sug‐ Yes Yes Yes Yes No No Yes Yes No Yes
13958 arSync
13959 Storj Yes † No Yes No No Yes Yes No No No
13960 Uptobox No Yes Yes Yes No No No No No No
13961 WebDAV Yes Yes Yes Yes No No Yes ‡ No Yes Yes
13962 Yandex Yes Yes Yes Yes Yes No Yes Yes Yes Yes
13963 Disk
13964 Zoho Yes Yes Yes Yes No No No No Yes Yes
13965 Work‐
13966 Drive
13967 The lo‐ Yes No Yes Yes No No Yes No Yes Yes
13968 cal
13969 filesys‐
13970 tem
13971
13972 Purge
13973 This deletes a directory quicker than just deleting all the files in
13974 the directory.
13975
13976 † Note Swift and Storj implement this in order to delete directory
13977 markers but they don't actually have a quicker way of deleting files
13978 other than deleting them individually.
13979
13980 ‡ StreamUpload is not supported with Nextcloud
13981
13982 Copy
13983 Used when copying an object to and from the same remote. This known as
13984 a server-side copy so you can copy a file without downloading it and
13985 uploading it again. It is used if you use rclone copy or rclone move
13986 if the remote doesn't support Move directly.
13987
13988 If the server doesn't support Copy directly then for copy operations
13989 the file is downloaded then re-uploaded.
13990
13991 Move
13992 Used when moving/renaming an object on the same remote. This is known
13993 as a server-side move of a file. This is used in rclone move if the
13994 server doesn't support DirMove.
13995
13996 If the server isn't capable of Move then rclone simulates it with Copy
13997 then delete. If the server doesn't support Copy then rclone will down‐
13998 load the file and re-upload it.
13999
14000 DirMove
14001 This is used to implement rclone move to move a directory if possible.
14002 If it isn't then it will use Move on each file (which falls back to
14003 Copy then download and upload - see Move section).
14004
14005 CleanUp
14006 This is used for emptying the trash for a remote by rclone cleanup.
14007
14008 If the server can't do CleanUp then rclone cleanup will return an er‐
14009 ror.
14010
14011 ‡‡ Note that while Box implements this it has to delete every file in‐
14012 dividually so it will be slower than emptying the trash via the WebUI
14013
14014 ListR
14015 The remote supports a recursive list to list all the contents beneath a
14016 directory quickly. This enables the --fast-list flag to work. See the
14017 rclone docs (https://rclone.org/docs/#fast-list) for more details.
14018
14019 StreamUpload
14020 Some remotes allow files to be uploaded without knowing the file size
14021 in advance. This allows certain operations to work without spooling
14022 the file to local disk first, e.g. rclone rcat.
14023
14024 LinkSharing
14025 Sets the necessary permissions on a file or folder and prints a link
14026 that allows others to access them, even if they don't have an account
14027 on the particular cloud provider.
14028
14029 About
14030 Rclone about prints quota information for a remote. Typical output in‐
14031 cludes bytes used, free, quota and in trash.
14032
14033 If a remote lacks about capability rclone about remote:returns an er‐
14034 ror.
14035
14036 Backends without about capability cannot determine free space for an
14037 rclone mount, or use policy mfs (most free space) as a member of an
14038 rclone union remote.
14039
14040 See rclone about command (https://rclone.org/commands/rclone_about/)
14041
14042 EmptyDir
14043 The remote supports empty directories. See Limitations
14044 (https://rclone.org/bugs/#limitations) for details. Most Object/Buck‐
14045 et-based remotes do not support this.
14046
14048 This describes the global flags available to every rclone command split
14049 into two groups, non backend and backend flags.
14050
14051 Non Backend Flags
14052 These flags are available for every command.
14053
14054 --ask-password Allow prompt for password for encrypted configuration (default true)
14055 --auto-confirm If enabled, do not request console confirmation
14056 --backup-dir string Make backups into hierarchy based in DIR
14057 --bind string Local address to bind to for outgoing connections, IPv4, IPv6 or name
14058 --buffer-size SizeSuffix In memory buffer size when reading files for each --transfer (default 16Mi)
14059 --bwlimit BwTimetable Bandwidth limit in KiB/s, or use suffix B|K|M|G|T|P or a full timetable
14060 --bwlimit-file BwTimetable Bandwidth limit per file in KiB/s, or use suffix B|K|M|G|T|P or a full timetable
14061 --ca-cert string CA certificate used to verify servers
14062 --cache-dir string Directory rclone will use for caching (default "$HOME/.cache/rclone")
14063 --check-first Do all the checks before starting transfers
14064 --checkers int Number of checkers to run in parallel (default 8)
14065 -c, --checksum Skip based on checksum (if available) & size, not mod-time & size
14066 --client-cert string Client SSL certificate (PEM) for mutual TLS auth
14067 --client-key string Client SSL private key (PEM) for mutual TLS auth
14068 --compare-dest stringArray Include additional comma separated server-side paths during comparison
14069 --config string Config file (default "$HOME/.config/rclone/rclone.conf")
14070 --contimeout duration Connect timeout (default 1m0s)
14071 --copy-dest stringArray Implies --compare-dest but also copies files from paths into destination
14072 --cpuprofile string Write cpu profile to file
14073 --cutoff-mode string Mode to stop transfers when reaching the max transfer limit HARD|SOFT|CAUTIOUS (default "HARD")
14074 --delete-after When synchronizing, delete files on destination after transferring (default)
14075 --delete-before When synchronizing, delete files on destination before transferring
14076 --delete-during When synchronizing, delete files during transfer
14077 --delete-excluded Delete files on dest excluded from sync
14078 --disable string Disable a comma separated list of features (use --disable help to see a list)
14079 --disable-http-keep-alives Disable HTTP keep-alives and use each connection once.
14080 --disable-http2 Disable HTTP/2 in the global transport
14081 -n, --dry-run Do a trial run with no permanent changes
14082 --dscp string Set DSCP value to connections, value or name, e.g. CS1, LE, DF, AF21
14083 --dump DumpFlags List of items to dump from: headers,bodies,requests,responses,auth,filters,goroutines,openfiles
14084 --dump-bodies Dump HTTP headers and bodies - may contain sensitive info
14085 --dump-headers Dump HTTP headers - may contain sensitive info
14086 --error-on-no-transfer Sets exit code 9 if no files are transferred, useful in scripts
14087 --exclude stringArray Exclude files matching pattern
14088 --exclude-from stringArray Read exclude patterns from file (use - to read from stdin)
14089 --exclude-if-present stringArray Exclude directories if filename is present
14090 --expect-continue-timeout duration Timeout when using expect / 100-continue in HTTP (default 1s)
14091 --fast-list Use recursive list if available; uses more memory but fewer transactions
14092 --files-from stringArray Read list of source-file names from file (use - to read from stdin)
14093 --files-from-raw stringArray Read list of source-file names from file without any processing of lines (use - to read from stdin)
14094 -f, --filter stringArray Add a file-filtering rule
14095 --filter-from stringArray Read filtering patterns from a file (use - to read from stdin)
14096 --fs-cache-expire-duration duration Cache remotes for this long (0 to disable caching) (default 5m0s)
14097 --fs-cache-expire-interval duration Interval to check for expired remotes (default 1m0s)
14098 --header stringArray Set HTTP header for all transactions
14099 --header-download stringArray Set HTTP header for download transactions
14100 --header-upload stringArray Set HTTP header for upload transactions
14101 --human-readable Print numbers in a human-readable format, sizes with suffix Ki|Mi|Gi|Ti|Pi
14102 --ignore-case Ignore case in filters (case insensitive)
14103 --ignore-case-sync Ignore case when synchronizing
14104 --ignore-checksum Skip post copy check of checksums
14105 --ignore-errors Delete even if there are I/O errors
14106 --ignore-existing Skip all files that exist on destination
14107 --ignore-size Ignore size when skipping use mod-time or checksum
14108 -I, --ignore-times Don't skip files that match size and time - transfer all files
14109 --immutable Do not modify files, fail if existing files have been modified
14110 --include stringArray Include files matching pattern
14111 --include-from stringArray Read include patterns from file (use - to read from stdin)
14112 -i, --interactive Enable interactive mode
14113 --kv-lock-time duration Maximum time to keep key-value database locked by process (default 1s)
14114 --log-file string Log everything to this file
14115 --log-format string Comma separated list of log format options (default "date,time")
14116 --log-level string Log level DEBUG|INFO|NOTICE|ERROR (default "NOTICE")
14117 --log-systemd Activate systemd integration for the logger
14118 --low-level-retries int Number of low level retries to do (default 10)
14119 --max-age Duration Only transfer files younger than this in s or suffix ms|s|m|h|d|w|M|y (default off)
14120 --max-backlog int Maximum number of objects in sync or check backlog (default 10000)
14121 --max-delete int When synchronizing, limit the number of deletes (default -1)
14122 --max-depth int If set limits the recursion depth to this (default -1)
14123 --max-duration duration Maximum duration rclone will transfer data for
14124 --max-size SizeSuffix Only transfer files smaller than this in KiB or suffix B|K|M|G|T|P (default off)
14125 --max-stats-groups int Maximum number of stats groups to keep in memory, on max oldest is discarded (default 1000)
14126 --max-transfer SizeSuffix Maximum size of data to transfer (default off)
14127 --memprofile string Write memory profile to file
14128 -M, --metadata If set, preserve metadata when copying objects
14129 --metadata-set stringArray Add metadata key=value when uploading
14130 --min-age Duration Only transfer files older than this in s or suffix ms|s|m|h|d|w|M|y (default off)
14131 --min-size SizeSuffix Only transfer files bigger than this in KiB or suffix B|K|M|G|T|P (default off)
14132 --modify-window duration Max time diff to be considered the same (default 1ns)
14133 --multi-thread-cutoff SizeSuffix Use multi-thread downloads for files above this size (default 250Mi)
14134 --multi-thread-streams int Max number of streams to use for multi-thread downloads (default 4)
14135 --no-check-certificate Do not verify the server SSL certificate (insecure)
14136 --no-check-dest Don't check the destination, copy regardless
14137 --no-console Hide console window (supported on Windows only)
14138 --no-gzip-encoding Don't set Accept-Encoding: gzip
14139 --no-traverse Don't traverse destination file system on copy
14140 --no-unicode-normalization Don't normalize unicode characters in filenames
14141 --no-update-modtime Don't update destination mod-time if files identical
14142 --order-by string Instructions on how to order the transfers, e.g. 'size,descending'
14143 --password-command SpaceSepList Command for supplying password for encrypted configuration
14144 -P, --progress Show progress during transfer
14145 --progress-terminal-title Show progress on the terminal title (requires -P/--progress)
14146 -q, --quiet Print as little stuff as possible
14147 --rc Enable the remote control server
14148 --rc-addr string IPaddress:Port or :Port to bind server to (default "localhost:5572")
14149 --rc-allow-origin string Set the allowed origin for CORS
14150 --rc-baseurl string Prefix for URLs - leave blank for root
14151 --rc-cert string SSL PEM key (concatenation of certificate and CA certificate)
14152 --rc-client-ca string Client certificate authority to verify clients with
14153 --rc-enable-metrics Enable prometheus metrics on /metrics
14154 --rc-files string Path to local files to serve on the HTTP server
14155 --rc-htpasswd string htpasswd file - if not provided no authentication is done
14156 --rc-job-expire-duration duration Expire finished async jobs older than this value (default 1m0s)
14157 --rc-job-expire-interval duration Interval to check for expired async jobs (default 10s)
14158 --rc-key string SSL PEM Private key
14159 --rc-max-header-bytes int Maximum size of request header (default 4096)
14160 --rc-min-tls-version string Minimum TLS version that is acceptable (default "tls1.0")
14161 --rc-no-auth Don't require auth for certain methods
14162 --rc-pass string Password for authentication
14163 --rc-realm string Realm for authentication (default "rclone")
14164 --rc-serve Enable the serving of remote objects
14165 --rc-server-read-timeout duration Timeout for server reading data (default 1h0m0s)
14166 --rc-server-write-timeout duration Timeout for server writing data (default 1h0m0s)
14167 --rc-template string User-specified template
14168 --rc-user string User name for authentication
14169 --rc-web-fetch-url string URL to fetch the releases for webgui (default "https://api.github.com/repos/rclone/rclone-webui-react/releases/latest")
14170 --rc-web-gui Launch WebGUI on localhost
14171 --rc-web-gui-force-update Force update to latest version of web gui
14172 --rc-web-gui-no-open-browser Don't open the browser automatically
14173 --rc-web-gui-update Check and update to latest version of web gui
14174 --refresh-times Refresh the modtime of remote files
14175 --retries int Retry operations this many times if they fail (default 3)
14176 --retries-sleep duration Interval between retrying operations if they fail, e.g. 500ms, 60s, 5m (0 to disable)
14177 --server-side-across-configs Allow server-side operations (e.g. copy) to work across different configs
14178 --size-only Skip based on size only, not mod-time or checksum
14179 --stats duration Interval between printing stats, e.g. 500ms, 60s, 5m (0 to disable) (default 1m0s)
14180 --stats-file-name-length int Max file name length in stats (0 for no limit) (default 45)
14181 --stats-log-level string Log level to show --stats output DEBUG|INFO|NOTICE|ERROR (default "INFO")
14182 --stats-one-line Make the stats fit on one line
14183 --stats-one-line-date Enable --stats-one-line and add current date/time prefix
14184 --stats-one-line-date-format string Enable --stats-one-line-date and use custom formatted date: Enclose date string in double quotes ("), see https://golang.org/pkg/time/#Time.Format
14185 --stats-unit string Show data rate in stats as either 'bits' or 'bytes' per second (default "bytes")
14186 --streaming-upload-cutoff SizeSuffix Cutoff for switching to chunked upload if file size is unknown, upload starts after reaching cutoff or when file ends (default 100Ki)
14187 --suffix string Suffix to add to changed files
14188 --suffix-keep-extension Preserve the extension when using --suffix
14189 --syslog Use Syslog for logging
14190 --syslog-facility string Facility for syslog, e.g. KERN,USER,... (default "DAEMON")
14191 --temp-dir string Directory rclone will use for temporary files (default "/tmp")
14192 --timeout duration IO idle timeout (default 5m0s)
14193 --tpslimit float Limit HTTP transactions per second to this
14194 --tpslimit-burst int Max burst of transactions for --tpslimit (default 1)
14195 --track-renames When synchronizing, track file renames and do a server-side move if possible
14196 --track-renames-strategy string Strategies to use when synchronizing using track-renames hash|modtime|leaf (default "hash")
14197 --transfers int Number of file transfers to run in parallel (default 4)
14198 -u, --update Skip files that are newer on the destination
14199 --use-cookies Enable session cookiejar
14200 --use-json-log Use json log format
14201 --use-mmap Use mmap allocator (see docs)
14202 --use-server-modtime Use server modified time instead of object metadata
14203 --user-agent string Set the user-agent to a specified string (default "rclone/v1.60.1")
14204 -v, --verbose count Print lots more stuff (repeat for more)
14205
14206 Backend Flags
14207 These flags are available for every command. They control the backends
14208 and may be set in the config file.
14209
14210 --acd-auth-url string Auth server URL
14211 --acd-client-id string OAuth Client Id
14212 --acd-client-secret string OAuth Client Secret
14213 --acd-encoding MultiEncoder The encoding for the backend (default Slash,InvalidUtf8,Dot)
14214 --acd-templink-threshold SizeSuffix Files >= this size will be downloaded via their tempLink (default 9Gi)
14215 --acd-token string OAuth Access Token as a JSON blob
14216 --acd-token-url string Token server url
14217 --acd-upload-wait-per-gb Duration Additional time per GiB to wait after a failed complete upload to see if it appears (default 3m0s)
14218 --alias-remote string Remote or path to alias
14219 --azureblob-access-tier string Access tier of blob: hot, cool or archive
14220 --azureblob-account string Storage Account Name
14221 --azureblob-archive-tier-delete Delete archive tier blobs before overwriting
14222 --azureblob-chunk-size SizeSuffix Upload chunk size (default 4Mi)
14223 --azureblob-disable-checksum Don't store MD5 checksum with object metadata
14224 --azureblob-encoding MultiEncoder The encoding for the backend (default Slash,BackSlash,Del,Ctl,RightPeriod,InvalidUtf8)
14225 --azureblob-endpoint string Endpoint for the service
14226 --azureblob-key string Storage Account Key
14227 --azureblob-list-chunk int Size of blob list (default 5000)
14228 --azureblob-memory-pool-flush-time Duration How often internal memory buffer pools will be flushed (default 1m0s)
14229 --azureblob-memory-pool-use-mmap Whether to use mmap buffers in internal memory pool
14230 --azureblob-msi-client-id string Object ID of the user-assigned MSI to use, if any
14231 --azureblob-msi-mi-res-id string Azure resource ID of the user-assigned MSI to use, if any
14232 --azureblob-msi-object-id string Object ID of the user-assigned MSI to use, if any
14233 --azureblob-no-head-object If set, do not do HEAD before GET when getting objects
14234 --azureblob-public-access string Public access level of a container: blob or container
14235 --azureblob-sas-url string SAS URL for container level access only
14236 --azureblob-service-principal-file string Path to file containing credentials for use with a service principal
14237 --azureblob-upload-concurrency int Concurrency for multipart uploads (default 16)
14238 --azureblob-upload-cutoff string Cutoff for switching to chunked upload (<= 256 MiB) (deprecated)
14239 --azureblob-use-emulator Uses local storage emulator if provided as 'true'
14240 --azureblob-use-msi Use a managed service identity to authenticate (only works in Azure)
14241 --b2-account string Account ID or Application Key ID
14242 --b2-chunk-size SizeSuffix Upload chunk size (default 96Mi)
14243 --b2-copy-cutoff SizeSuffix Cutoff for switching to multipart copy (default 4Gi)
14244 --b2-disable-checksum Disable checksums for large (> upload cutoff) files
14245 --b2-download-auth-duration Duration Time before the authorization token will expire in s or suffix ms|s|m|h|d (default 1w)
14246 --b2-download-url string Custom endpoint for downloads
14247 --b2-encoding MultiEncoder The encoding for the backend (default Slash,BackSlash,Del,Ctl,InvalidUtf8,Dot)
14248 --b2-endpoint string Endpoint for the service
14249 --b2-hard-delete Permanently delete files on remote removal, otherwise hide files
14250 --b2-key string Application Key
14251 --b2-memory-pool-flush-time Duration How often internal memory buffer pools will be flushed (default 1m0s)
14252 --b2-memory-pool-use-mmap Whether to use mmap buffers in internal memory pool
14253 --b2-test-mode string A flag string for X-Bz-Test-Mode header for debugging
14254 --b2-upload-cutoff SizeSuffix Cutoff for switching to chunked upload (default 200Mi)
14255 --b2-version-at Time Show file versions as they were at the specified time (default off)
14256 --b2-versions Include old versions in directory listings
14257 --box-access-token string Box App Primary Access Token
14258 --box-auth-url string Auth server URL
14259 --box-box-config-file string Box App config.json location
14260 --box-box-sub-type string (default "user")
14261 --box-client-id string OAuth Client Id
14262 --box-client-secret string OAuth Client Secret
14263 --box-commit-retries int Max number of times to try committing a multipart file (default 100)
14264 --box-encoding MultiEncoder The encoding for the backend (default Slash,BackSlash,Del,Ctl,RightSpace,InvalidUtf8,Dot)
14265 --box-list-chunk int Size of listing chunk 1-1000 (default 1000)
14266 --box-owned-by string Only show items owned by the login (email address) passed in
14267 --box-root-folder-id string Fill in for rclone to use a non root folder as its starting point
14268 --box-token string OAuth Access Token as a JSON blob
14269 --box-token-url string Token server url
14270 --box-upload-cutoff SizeSuffix Cutoff for switching to multipart upload (>= 50 MiB) (default 50Mi)
14271 --cache-chunk-clean-interval Duration How often should the cache perform cleanups of the chunk storage (default 1m0s)
14272 --cache-chunk-no-memory Disable the in-memory cache for storing chunks during streaming
14273 --cache-chunk-path string Directory to cache chunk files (default "$HOME/.cache/rclone/cache-backend")
14274 --cache-chunk-size SizeSuffix The size of a chunk (partial file data) (default 5Mi)
14275 --cache-chunk-total-size SizeSuffix The total size that the chunks can take up on the local disk (default 10Gi)
14276 --cache-db-path string Directory to store file structure metadata DB (default "$HOME/.cache/rclone/cache-backend")
14277 --cache-db-purge Clear all the cached data for this remote on start
14278 --cache-db-wait-time Duration How long to wait for the DB to be available - 0 is unlimited (default 1s)
14279 --cache-info-age Duration How long to cache file structure information (directory listings, file size, times, etc.) (default 6h0m0s)
14280 --cache-plex-insecure string Skip all certificate verification when connecting to the Plex server
14281 --cache-plex-password string The password of the Plex user (obscured)
14282 --cache-plex-url string The URL of the Plex server
14283 --cache-plex-username string The username of the Plex user
14284 --cache-read-retries int How many times to retry a read from a cache storage (default 10)
14285 --cache-remote string Remote to cache
14286 --cache-rps int Limits the number of requests per second to the source FS (-1 to disable) (default -1)
14287 --cache-tmp-upload-path string Directory to keep temporary files until they are uploaded
14288 --cache-tmp-wait-time Duration How long should files be stored in local cache before being uploaded (default 15s)
14289 --cache-workers int How many workers should run in parallel to download chunks (default 4)
14290 --cache-writes Cache file data on writes through the FS
14291 --chunker-chunk-size SizeSuffix Files larger than chunk size will be split in chunks (default 2Gi)
14292 --chunker-fail-hard Choose how chunker should handle files with missing or invalid chunks
14293 --chunker-hash-type string Choose how chunker handles hash sums (default "md5")
14294 --chunker-remote string Remote to chunk/unchunk
14295 --combine-upstreams SpaceSepList Upstreams for combining
14296 --compress-level int GZIP compression level (-2 to 9) (default -1)
14297 --compress-mode string Compression mode (default "gzip")
14298 --compress-ram-cache-limit SizeSuffix Some remotes don't allow the upload of files with unknown size (default 20Mi)
14299 --compress-remote string Remote to compress
14300 -L, --copy-links Follow symlinks and copy the pointed to item
14301 --crypt-directory-name-encryption Option to either encrypt directory names or leave them intact (default true)
14302 --crypt-filename-encoding string How to encode the encrypted filename to text string (default "base32")
14303 --crypt-filename-encryption string How to encrypt the filenames (default "standard")
14304 --crypt-no-data-encryption Option to either encrypt file data or leave it unencrypted
14305 --crypt-password string Password or pass phrase for encryption (obscured)
14306 --crypt-password2 string Password or pass phrase for salt (obscured)
14307 --crypt-remote string Remote to encrypt/decrypt
14308 --crypt-server-side-across-configs Allow server-side operations (e.g. copy) to work across different crypt configs
14309 --crypt-show-mapping For all files listed show how the names encrypt
14310 --drive-acknowledge-abuse Set to allow files which return cannotDownloadAbusiveFile to be downloaded
14311 --drive-allow-import-name-change Allow the filetype to change when uploading Google docs
14312 --drive-auth-owner-only Only consider files owned by the authenticated user
14313 --drive-auth-url string Auth server URL
14314 --drive-chunk-size SizeSuffix Upload chunk size (default 8Mi)
14315 --drive-client-id string Google Application Client Id
14316 --drive-client-secret string OAuth Client Secret
14317 --drive-copy-shortcut-content Server side copy contents of shortcuts instead of the shortcut
14318 --drive-disable-http2 Disable drive using http2 (default true)
14319 --drive-encoding MultiEncoder The encoding for the backend (default InvalidUtf8)
14320 --drive-export-formats string Comma separated list of preferred formats for downloading Google docs (default "docx,xlsx,pptx,svg")
14321 --drive-formats string Deprecated: See export_formats
14322 --drive-impersonate string Impersonate this user when using a service account
14323 --drive-import-formats string Comma separated list of preferred formats for uploading Google docs
14324 --drive-keep-revision-forever Keep new head revision of each file forever
14325 --drive-list-chunk int Size of listing chunk 100-1000, 0 to disable (default 1000)
14326 --drive-pacer-burst int Number of API calls to allow without sleeping (default 100)
14327 --drive-pacer-min-sleep Duration Minimum time to sleep between API calls (default 100ms)
14328 --drive-resource-key string Resource key for accessing a link-shared file
14329 --drive-root-folder-id string ID of the root folder
14330 --drive-scope string Scope that rclone should use when requesting access from drive
14331 --drive-server-side-across-configs Allow server-side operations (e.g. copy) to work across different drive configs
14332 --drive-service-account-credentials string Service Account Credentials JSON blob
14333 --drive-service-account-file string Service Account Credentials JSON file path
14334 --drive-shared-with-me Only show files that are shared with me
14335 --drive-size-as-quota Show sizes as storage quota usage, not actual size
14336 --drive-skip-checksum-gphotos Skip MD5 checksum on Google photos and videos only
14337 --drive-skip-dangling-shortcuts If set skip dangling shortcut files
14338 --drive-skip-gdocs Skip google documents in all listings
14339 --drive-skip-shortcuts If set skip shortcut files
14340 --drive-starred-only Only show files that are starred
14341 --drive-stop-on-download-limit Make download limit errors be fatal
14342 --drive-stop-on-upload-limit Make upload limit errors be fatal
14343 --drive-team-drive string ID of the Shared Drive (Team Drive)
14344 --drive-token string OAuth Access Token as a JSON blob
14345 --drive-token-url string Token server url
14346 --drive-trashed-only Only show files that are in the trash
14347 --drive-upload-cutoff SizeSuffix Cutoff for switching to chunked upload (default 8Mi)
14348 --drive-use-created-date Use file created date instead of modified date
14349 --drive-use-shared-date Use date file was shared instead of modified date
14350 --drive-use-trash Send files to the trash instead of deleting permanently (default true)
14351 --drive-v2-download-min-size SizeSuffix If Object's are greater, use drive v2 API to download (default off)
14352 --dropbox-auth-url string Auth server URL
14353 --dropbox-batch-commit-timeout Duration Max time to wait for a batch to finish committing (default 10m0s)
14354 --dropbox-batch-mode string Upload file batching sync|async|off (default "sync")
14355 --dropbox-batch-size int Max number of files in upload batch
14356 --dropbox-batch-timeout Duration Max time to allow an idle upload batch before uploading (default 0s)
14357 --dropbox-chunk-size SizeSuffix Upload chunk size (< 150Mi) (default 48Mi)
14358 --dropbox-client-id string OAuth Client Id
14359 --dropbox-client-secret string OAuth Client Secret
14360 --dropbox-encoding MultiEncoder The encoding for the backend (default Slash,BackSlash,Del,RightSpace,InvalidUtf8,Dot)
14361 --dropbox-impersonate string Impersonate this user when using a business account
14362 --dropbox-shared-files Instructs rclone to work on individual shared files
14363 --dropbox-shared-folders Instructs rclone to work on shared folders
14364 --dropbox-token string OAuth Access Token as a JSON blob
14365 --dropbox-token-url string Token server url
14366 --fichier-api-key string Your API Key, get it from https://1fichier.com/console/params.pl
14367 --fichier-encoding MultiEncoder The encoding for the backend (default Slash,LtGt,DoubleQuote,SingleQuote,BackQuote,Dollar,BackSlash,Del,Ctl,LeftSpace,RightSpace,InvalidUtf8,Dot)
14368 --fichier-file-password string If you want to download a shared file that is password protected, add this parameter (obscured)
14369 --fichier-folder-password string If you want to list the files in a shared folder that is password protected, add this parameter (obscured)
14370 --fichier-shared-folder string If you want to download a shared folder, add this parameter
14371 --filefabric-encoding MultiEncoder The encoding for the backend (default Slash,Del,Ctl,InvalidUtf8,Dot)
14372 --filefabric-permanent-token string Permanent Authentication Token
14373 --filefabric-root-folder-id string ID of the root folder
14374 --filefabric-token string Session Token
14375 --filefabric-token-expiry string Token expiry time
14376 --filefabric-url string URL of the Enterprise File Fabric to connect to
14377 --filefabric-version string Version read from the file fabric
14378 --ftp-ask-password Allow asking for FTP password when needed
14379 --ftp-close-timeout Duration Maximum time to wait for a response to close (default 1m0s)
14380 --ftp-concurrency int Maximum number of FTP simultaneous connections, 0 for unlimited
14381 --ftp-disable-epsv Disable using EPSV even if server advertises support
14382 --ftp-disable-mlsd Disable using MLSD even if server advertises support
14383 --ftp-disable-tls13 Disable TLS 1.3 (workaround for FTP servers with buggy TLS)
14384 --ftp-disable-utf8 Disable using UTF-8 even if server advertises support
14385 --ftp-encoding MultiEncoder The encoding for the backend (default Slash,Del,Ctl,RightSpace,Dot)
14386 --ftp-explicit-tls Use Explicit FTPS (FTP over TLS)
14387 --ftp-force-list-hidden Use LIST -a to force listing of hidden files and folders. This will disable the use of MLSD
14388 --ftp-host string FTP host to connect to
14389 --ftp-idle-timeout Duration Max time before closing idle connections (default 1m0s)
14390 --ftp-no-check-certificate Do not verify the TLS certificate of the server
14391 --ftp-pass string FTP password (obscured)
14392 --ftp-port int FTP port number (default 21)
14393 --ftp-shut-timeout Duration Maximum time to wait for data connection closing status (default 1m0s)
14394 --ftp-tls Use Implicit FTPS (FTP over TLS)
14395 --ftp-tls-cache-size int Size of TLS session cache for all control and data connections (default 32)
14396 --ftp-user string FTP username (default "$USER")
14397 --ftp-writing-mdtm Use MDTM to set modification time (VsFtpd quirk)
14398 --gcs-anonymous Access public buckets and objects without credentials
14399 --gcs-auth-url string Auth server URL
14400 --gcs-bucket-acl string Access Control List for new buckets
14401 --gcs-bucket-policy-only Access checks should use bucket-level IAM policies
14402 --gcs-client-id string OAuth Client Id
14403 --gcs-client-secret string OAuth Client Secret
14404 --gcs-decompress If set this will decompress gzip encoded objects
14405 --gcs-encoding MultiEncoder The encoding for the backend (default Slash,CrLf,InvalidUtf8,Dot)
14406 --gcs-endpoint string Endpoint for the service
14407 --gcs-location string Location for the newly created buckets
14408 --gcs-no-check-bucket If set, don't attempt to check the bucket exists or create it
14409 --gcs-object-acl string Access Control List for new objects
14410 --gcs-project-number string Project number
14411 --gcs-service-account-file string Service Account Credentials JSON file path
14412 --gcs-storage-class string The storage class to use when storing objects in Google Cloud Storage
14413 --gcs-token string OAuth Access Token as a JSON blob
14414 --gcs-token-url string Token server url
14415 --gphotos-auth-url string Auth server URL
14416 --gphotos-client-id string OAuth Client Id
14417 --gphotos-client-secret string OAuth Client Secret
14418 --gphotos-encoding MultiEncoder The encoding for the backend (default Slash,CrLf,InvalidUtf8,Dot)
14419 --gphotos-include-archived Also view and download archived media
14420 --gphotos-read-only Set to make the Google Photos backend read only
14421 --gphotos-read-size Set to read the size of media items
14422 --gphotos-start-year int Year limits the photos to be downloaded to those which are uploaded after the given year (default 2000)
14423 --gphotos-token string OAuth Access Token as a JSON blob
14424 --gphotos-token-url string Token server url
14425 --hasher-auto-size SizeSuffix Auto-update checksum for files smaller than this size (disabled by default)
14426 --hasher-hashes CommaSepList Comma separated list of supported checksum types (default md5,sha1)
14427 --hasher-max-age Duration Maximum time to keep checksums in cache (0 = no cache, off = cache forever) (default off)
14428 --hasher-remote string Remote to cache checksums for (e.g. myRemote:path)
14429 --hdfs-data-transfer-protection string Kerberos data transfer protection: authentication|integrity|privacy
14430 --hdfs-encoding MultiEncoder The encoding for the backend (default Slash,Colon,Del,Ctl,InvalidUtf8,Dot)
14431 --hdfs-namenode string Hadoop name node and port
14432 --hdfs-service-principal-name string Kerberos service principal name for the namenode
14433 --hdfs-username string Hadoop user name
14434 --hidrive-auth-url string Auth server URL
14435 --hidrive-chunk-size SizeSuffix Chunksize for chunked uploads (default 48Mi)
14436 --hidrive-client-id string OAuth Client Id
14437 --hidrive-client-secret string OAuth Client Secret
14438 --hidrive-disable-fetching-member-count Do not fetch number of objects in directories unless it is absolutely necessary
14439 --hidrive-encoding MultiEncoder The encoding for the backend (default Slash,Dot)
14440 --hidrive-endpoint string Endpoint for the service (default "https://api.hidrive.strato.com/2.1")
14441 --hidrive-root-prefix string The root/parent folder for all paths (default "/")
14442 --hidrive-scope-access string Access permissions that rclone should use when requesting access from HiDrive (default "rw")
14443 --hidrive-scope-role string User-level that rclone should use when requesting access from HiDrive (default "user")
14444 --hidrive-token string OAuth Access Token as a JSON blob
14445 --hidrive-token-url string Token server url
14446 --hidrive-upload-concurrency int Concurrency for chunked uploads (default 4)
14447 --hidrive-upload-cutoff SizeSuffix Cutoff/Threshold for chunked uploads (default 96Mi)
14448 --http-headers CommaSepList Set HTTP headers for all transactions
14449 --http-no-head Don't use HEAD requests
14450 --http-no-slash Set this if the site doesn't end directories with /
14451 --http-url string URL of HTTP host to connect to
14452 --internetarchive-access-key-id string IAS3 Access Key
14453 --internetarchive-disable-checksum Don't ask the server to test against MD5 checksum calculated by rclone (default true)
14454 --internetarchive-encoding MultiEncoder The encoding for the backend (default Slash,LtGt,CrLf,Del,Ctl,InvalidUtf8,Dot)
14455 --internetarchive-endpoint string IAS3 Endpoint (default "https://s3.us.archive.org")
14456 --internetarchive-front-endpoint string Host of InternetArchive Frontend (default "https://archive.org")
14457 --internetarchive-secret-access-key string IAS3 Secret Key (password)
14458 --internetarchive-wait-archive Duration Timeout for waiting the server's processing tasks (specifically archive and book_op) to finish (default 0s)
14459 --jottacloud-encoding MultiEncoder The encoding for the backend (default Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,Del,Ctl,InvalidUtf8,Dot)
14460 --jottacloud-hard-delete Delete files permanently rather than putting them into the trash
14461 --jottacloud-md5-memory-limit SizeSuffix Files bigger than this will be cached on disk to calculate the MD5 if required (default 10Mi)
14462 --jottacloud-no-versions Avoid server side versioning by deleting files and recreating files instead of overwriting them
14463 --jottacloud-trashed-only Only show files that are in the trash
14464 --jottacloud-upload-resume-limit SizeSuffix Files bigger than this can be resumed if the upload fail's (default 10Mi)
14465 --koofr-encoding MultiEncoder The encoding for the backend (default Slash,BackSlash,Del,Ctl,InvalidUtf8,Dot)
14466 --koofr-endpoint string The Koofr API endpoint to use
14467 --koofr-mountid string Mount ID of the mount to use
14468 --koofr-password string Your password for rclone (generate one at https://app.koofr.net/app/admin/preferences/password) (obscured)
14469 --koofr-provider string Choose your storage provider
14470 --koofr-setmtime Does the backend support setting modification time (default true)
14471 --koofr-user string Your user name
14472 -l, --links Translate symlinks to/from regular files with a '.rclonelink' extension
14473 --local-case-insensitive Force the filesystem to report itself as case insensitive
14474 --local-case-sensitive Force the filesystem to report itself as case sensitive
14475 --local-encoding MultiEncoder The encoding for the backend (default Slash,Dot)
14476 --local-no-check-updated Don't check to see if the files change during upload
14477 --local-no-preallocate Disable preallocation of disk space for transferred files
14478 --local-no-set-modtime Disable setting modtime
14479 --local-no-sparse Disable sparse files for multi-thread downloads
14480 --local-nounc Disable UNC (long path names) conversion on Windows
14481 --local-unicode-normalization Apply unicode NFC normalization to paths and filenames
14482 --local-zero-size-links Assume the Stat size of links is zero (and read them instead) (deprecated)
14483 --mailru-check-hash What should copy do if file checksum is mismatched or invalid (default true)
14484 --mailru-encoding MultiEncoder The encoding for the backend (default Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,BackSlash,Del,Ctl,InvalidUtf8,Dot)
14485 --mailru-pass string Password (obscured)
14486 --mailru-speedup-enable Skip full upload if there is another file with same data hash (default true)
14487 --mailru-speedup-file-patterns string Comma separated list of file name patterns eligible for speedup (put by hash) (default "*.mkv,*.avi,*.mp4,*.mp3,*.zip,*.gz,*.rar,*.pdf")
14488 --mailru-speedup-max-disk SizeSuffix This option allows you to disable speedup (put by hash) for large files (default 3Gi)
14489 --mailru-speedup-max-memory SizeSuffix Files larger than the size given below will always be hashed on disk (default 32Mi)
14490 --mailru-user string User name (usually email)
14491 --mega-debug Output more debug from Mega
14492 --mega-encoding MultiEncoder The encoding for the backend (default Slash,InvalidUtf8,Dot)
14493 --mega-hard-delete Delete files permanently rather than putting them into the trash
14494 --mega-pass string Password (obscured)
14495 --mega-user string User name
14496 --netstorage-account string Set the NetStorage account name
14497 --netstorage-host string Domain+path of NetStorage host to connect to
14498 --netstorage-protocol string Select between HTTP or HTTPS protocol (default "https")
14499 --netstorage-secret string Set the NetStorage account secret/G2O key for authentication (obscured)
14500 -x, --one-file-system Don't cross filesystem boundaries (unix/macOS only)
14501 --onedrive-access-scopes SpaceSepList Set scopes to be requested by rclone (default Files.Read Files.ReadWrite Files.Read.All Files.ReadWrite.All Sites.Read.All offline_access)
14502 --onedrive-auth-url string Auth server URL
14503 --onedrive-chunk-size SizeSuffix Chunk size to upload files with - must be multiple of 320k (327,680 bytes) (default 10Mi)
14504 --onedrive-client-id string OAuth Client Id
14505 --onedrive-client-secret string OAuth Client Secret
14506 --onedrive-drive-id string The ID of the drive to use
14507 --onedrive-drive-type string The type of the drive (personal | business | documentLibrary)
14508 --onedrive-encoding MultiEncoder The encoding for the backend (default Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,BackSlash,Del,Ctl,LeftSpace,LeftTilde,RightSpace,RightPeriod,InvalidUtf8,Dot)
14509 --onedrive-expose-onenote-files Set to make OneNote files show up in directory listings
14510 --onedrive-link-password string Set the password for links created by the link command
14511 --onedrive-link-scope string Set the scope of the links created by the link command (default "anonymous")
14512 --onedrive-link-type string Set the type of the links created by the link command (default "view")
14513 --onedrive-list-chunk int Size of listing chunk (default 1000)
14514 --onedrive-no-versions Remove all versions on modifying operations
14515 --onedrive-region string Choose national cloud region for OneDrive (default "global")
14516 --onedrive-root-folder-id string ID of the root folder
14517 --onedrive-server-side-across-configs Allow server-side operations (e.g. copy) to work across different onedrive configs
14518 --onedrive-token string OAuth Access Token as a JSON blob
14519 --onedrive-token-url string Token server url
14520 --oos-chunk-size SizeSuffix Chunk size to use for uploading (default 5Mi)
14521 --oos-compartment string Object storage compartment OCID
14522 --oos-config-file string Path to OCI config file (default "~/.oci/config")
14523 --oos-config-profile string Profile name inside the oci config file (default "Default")
14524 --oos-copy-cutoff SizeSuffix Cutoff for switching to multipart copy (default 4.656Gi)
14525 --oos-copy-timeout Duration Timeout for copy (default 1m0s)
14526 --oos-disable-checksum Don't store MD5 checksum with object metadata
14527 --oos-encoding MultiEncoder The encoding for the backend (default Slash,InvalidUtf8,Dot)
14528 --oos-endpoint string Endpoint for Object storage API
14529 --oos-leave-parts-on-error If true avoid calling abort upload on a failure, leaving all successfully uploaded parts on S3 for manual recovery
14530 --oos-namespace string Object storage namespace
14531 --oos-no-check-bucket If set, don't attempt to check the bucket exists or create it
14532 --oos-provider string Choose your Auth Provider (default "env_auth")
14533 --oos-region string Object storage Region
14534 --oos-upload-concurrency int Concurrency for multipart uploads (default 10)
14535 --oos-upload-cutoff SizeSuffix Cutoff for switching to chunked upload (default 200Mi)
14536 --opendrive-chunk-size SizeSuffix Files will be uploaded in chunks this size (default 10Mi)
14537 --opendrive-encoding MultiEncoder The encoding for the backend (default Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,BackSlash,LeftSpace,LeftCrLfHtVt,RightSpace,RightCrLfHtVt,InvalidUtf8,Dot)
14538 --opendrive-password string Password (obscured)
14539 --opendrive-username string Username
14540 --pcloud-auth-url string Auth server URL
14541 --pcloud-client-id string OAuth Client Id
14542 --pcloud-client-secret string OAuth Client Secret
14543 --pcloud-encoding MultiEncoder The encoding for the backend (default Slash,BackSlash,Del,Ctl,InvalidUtf8,Dot)
14544 --pcloud-hostname string Hostname to connect to (default "api.pcloud.com")
14545 --pcloud-password string Your pcloud password (obscured)
14546 --pcloud-root-folder-id string Fill in for rclone to use a non root folder as its starting point (default "d0")
14547 --pcloud-token string OAuth Access Token as a JSON blob
14548 --pcloud-token-url string Token server url
14549 --pcloud-username string Your pcloud username
14550 --premiumizeme-encoding MultiEncoder The encoding for the backend (default Slash,DoubleQuote,BackSlash,Del,Ctl,InvalidUtf8,Dot)
14551 --putio-encoding MultiEncoder The encoding for the backend (default Slash,BackSlash,Del,Ctl,InvalidUtf8,Dot)
14552 --qingstor-access-key-id string QingStor Access Key ID
14553 --qingstor-chunk-size SizeSuffix Chunk size to use for uploading (default 4Mi)
14554 --qingstor-connection-retries int Number of connection retries (default 3)
14555 --qingstor-encoding MultiEncoder The encoding for the backend (default Slash,Ctl,InvalidUtf8)
14556 --qingstor-endpoint string Enter an endpoint URL to connection QingStor API
14557 --qingstor-env-auth Get QingStor credentials from runtime
14558 --qingstor-secret-access-key string QingStor Secret Access Key (password)
14559 --qingstor-upload-concurrency int Concurrency for multipart uploads (default 1)
14560 --qingstor-upload-cutoff SizeSuffix Cutoff for switching to chunked upload (default 200Mi)
14561 --qingstor-zone string Zone to connect to
14562 --s3-access-key-id string AWS Access Key ID
14563 --s3-acl string Canned ACL used when creating buckets and storing or copying objects
14564 --s3-bucket-acl string Canned ACL used when creating buckets
14565 --s3-chunk-size SizeSuffix Chunk size to use for uploading (default 5Mi)
14566 --s3-copy-cutoff SizeSuffix Cutoff for switching to multipart copy (default 4.656Gi)
14567 --s3-decompress If set this will decompress gzip encoded objects
14568 --s3-disable-checksum Don't store MD5 checksum with object metadata
14569 --s3-disable-http2 Disable usage of http2 for S3 backends
14570 --s3-download-url string Custom endpoint for downloads
14571 --s3-encoding MultiEncoder The encoding for the backend (default Slash,InvalidUtf8,Dot)
14572 --s3-endpoint string Endpoint for S3 API
14573 --s3-env-auth Get AWS credentials from runtime (environment variables or EC2/ECS meta data if no env vars)
14574 --s3-force-path-style If true use path style access if false use virtual hosted style (default true)
14575 --s3-leave-parts-on-error If true avoid calling abort upload on a failure, leaving all successfully uploaded parts on S3 for manual recovery
14576 --s3-list-chunk int Size of listing chunk (response list for each ListObject S3 request) (default 1000)
14577 --s3-list-url-encode Tristate Whether to url encode listings: true/false/unset (default unset)
14578 --s3-list-version int Version of ListObjects to use: 1,2 or 0 for auto
14579 --s3-location-constraint string Location constraint - must be set to match the Region
14580 --s3-max-upload-parts int Maximum number of parts in a multipart upload (default 10000)
14581 --s3-memory-pool-flush-time Duration How often internal memory buffer pools will be flushed (default 1m0s)
14582 --s3-memory-pool-use-mmap Whether to use mmap buffers in internal memory pool
14583 --s3-might-gzip Tristate Set this if the backend might gzip objects (default unset)
14584 --s3-no-check-bucket If set, don't attempt to check the bucket exists or create it
14585 --s3-no-head If set, don't HEAD uploaded objects to check integrity
14586 --s3-no-head-object If set, do not do HEAD before GET when getting objects
14587 --s3-no-system-metadata Suppress setting and reading of system metadata
14588 --s3-profile string Profile to use in the shared credentials file
14589 --s3-provider string Choose your S3 provider
14590 --s3-region string Region to connect to
14591 --s3-requester-pays Enables requester pays option when interacting with S3 bucket
14592 --s3-secret-access-key string AWS Secret Access Key (password)
14593 --s3-server-side-encryption string The server-side encryption algorithm used when storing this object in S3
14594 --s3-session-token string An AWS session token
14595 --s3-shared-credentials-file string Path to the shared credentials file
14596 --s3-sse-customer-algorithm string If using SSE-C, the server-side encryption algorithm used when storing this object in S3
14597 --s3-sse-customer-key string To use SSE-C you may provide the secret encryption key used to encrypt/decrypt your data
14598 --s3-sse-customer-key-base64 string If using SSE-C you must provide the secret encryption key encoded in base64 format to encrypt/decrypt your data
14599 --s3-sse-customer-key-md5 string If using SSE-C you may provide the secret encryption key MD5 checksum (optional)
14600 --s3-sse-kms-key-id string If using KMS ID you must provide the ARN of Key
14601 --s3-storage-class string The storage class to use when storing new objects in S3
14602 --s3-upload-concurrency int Concurrency for multipart uploads (default 4)
14603 --s3-upload-cutoff SizeSuffix Cutoff for switching to chunked upload (default 200Mi)
14604 --s3-use-accelerate-endpoint If true use the AWS S3 accelerated endpoint
14605 --s3-use-multipart-etag Tristate Whether to use ETag in multipart uploads for verification (default unset)
14606 --s3-use-presigned-request Whether to use a presigned request or PutObject for single part uploads
14607 --s3-v2-auth If true use v2 authentication
14608 --s3-version-at Time Show file versions as they were at the specified time (default off)
14609 --s3-versions Include old versions in directory listings
14610 --seafile-2fa Two-factor authentication ('true' if the account has 2FA enabled)
14611 --seafile-create-library Should rclone create a library if it doesn't exist
14612 --seafile-encoding MultiEncoder The encoding for the backend (default Slash,DoubleQuote,BackSlash,Ctl,InvalidUtf8)
14613 --seafile-library string Name of the library
14614 --seafile-library-key string Library password (for encrypted libraries only) (obscured)
14615 --seafile-pass string Password (obscured)
14616 --seafile-url string URL of seafile host to connect to
14617 --seafile-user string User name (usually email address)
14618 --sftp-ask-password Allow asking for SFTP password when needed
14619 --sftp-chunk-size SizeSuffix Upload and download chunk size (default 32Ki)
14620 --sftp-concurrency int The maximum number of outstanding requests for one file (default 64)
14621 --sftp-disable-concurrent-reads If set don't use concurrent reads
14622 --sftp-disable-concurrent-writes If set don't use concurrent writes
14623 --sftp-disable-hashcheck Disable the execution of SSH commands to determine if remote file hashing is available
14624 --sftp-host string SSH host to connect to
14625 --sftp-idle-timeout Duration Max time before closing idle connections (default 1m0s)
14626 --sftp-key-file string Path to PEM-encoded private key file
14627 --sftp-key-file-pass string The passphrase to decrypt the PEM-encoded private key file (obscured)
14628 --sftp-key-pem string Raw PEM-encoded private key
14629 --sftp-key-use-agent When set forces the usage of the ssh-agent
14630 --sftp-known-hosts-file string Optional path to known_hosts file
14631 --sftp-md5sum-command string The command used to read md5 hashes
14632 --sftp-pass string SSH password, leave blank to use ssh-agent (obscured)
14633 --sftp-path-override string Override path used by SSH shell commands
14634 --sftp-port int SSH port number (default 22)
14635 --sftp-pubkey-file string Optional path to public key file
14636 --sftp-server-command string Specifies the path or command to run a sftp server on the remote host
14637 --sftp-set-env SpaceSepList Environment variables to pass to sftp and commands
14638 --sftp-set-modtime Set the modified time on the remote if set (default true)
14639 --sftp-sha1sum-command string The command used to read sha1 hashes
14640 --sftp-shell-type string The type of SSH shell on remote server, if any
14641 --sftp-skip-links Set to skip any symlinks and any other non regular files
14642 --sftp-subsystem string Specifies the SSH2 subsystem on the remote host (default "sftp")
14643 --sftp-use-fstat If set use fstat instead of stat
14644 --sftp-use-insecure-cipher Enable the use of insecure ciphers and key exchange methods
14645 --sftp-user string SSH username (default "$USER")
14646 --sharefile-chunk-size SizeSuffix Upload chunk size (default 64Mi)
14647 --sharefile-encoding MultiEncoder The encoding for the backend (default Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,BackSlash,Ctl,LeftSpace,LeftPeriod,RightSpace,RightPeriod,InvalidUtf8,Dot)
14648 --sharefile-endpoint string Endpoint for API calls
14649 --sharefile-root-folder-id string ID of the root folder
14650 --sharefile-upload-cutoff SizeSuffix Cutoff for switching to multipart upload (default 128Mi)
14651 --sia-api-password string Sia Daemon API Password (obscured)
14652 --sia-api-url string Sia daemon API URL, like http://sia.daemon.host:9980 (default "http://127.0.0.1:9980")
14653 --sia-encoding MultiEncoder The encoding for the backend (default Slash,Question,Hash,Percent,Del,Ctl,InvalidUtf8,Dot)
14654 --sia-user-agent string Siad User Agent (default "Sia-Agent")
14655 --skip-links Don't warn about skipped symlinks
14656 --smb-case-insensitive Whether the server is configured to be case-insensitive (default true)
14657 --smb-domain string Domain name for NTLM authentication (default "WORKGROUP")
14658 --smb-encoding MultiEncoder The encoding for the backend (default Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,BackSlash,Ctl,RightSpace,RightPeriod,InvalidUtf8,Dot)
14659 --smb-hide-special-share Hide special shares (e.g. print$) which users aren't supposed to access (default true)
14660 --smb-host string SMB server hostname to connect to
14661 --smb-idle-timeout Duration Max time before closing idle connections (default 1m0s)
14662 --smb-pass string SMB password (obscured)
14663 --smb-port int SMB port number (default 445)
14664 --smb-user string SMB username (default "$USER")
14665 --storj-access-grant string Access grant
14666 --storj-api-key string API key
14667 --storj-passphrase string Encryption passphrase
14668 --storj-provider string Choose an authentication method (default "existing")
14669 --storj-satellite-address string Satellite address (default "us-central-1.storj.io")
14670 --sugarsync-access-key-id string Sugarsync Access Key ID
14671 --sugarsync-app-id string Sugarsync App ID
14672 --sugarsync-authorization string Sugarsync authorization
14673 --sugarsync-authorization-expiry string Sugarsync authorization expiry
14674 --sugarsync-deleted-id string Sugarsync deleted folder id
14675 --sugarsync-encoding MultiEncoder The encoding for the backend (default Slash,Ctl,InvalidUtf8,Dot)
14676 --sugarsync-hard-delete Permanently delete files if true
14677 --sugarsync-private-access-key string Sugarsync Private Access Key
14678 --sugarsync-refresh-token string Sugarsync refresh token
14679 --sugarsync-root-id string Sugarsync root id
14680 --sugarsync-user string Sugarsync user
14681 --swift-application-credential-id string Application Credential ID (OS_APPLICATION_CREDENTIAL_ID)
14682 --swift-application-credential-name string Application Credential Name (OS_APPLICATION_CREDENTIAL_NAME)
14683 --swift-application-credential-secret string Application Credential Secret (OS_APPLICATION_CREDENTIAL_SECRET)
14684 --swift-auth string Authentication URL for server (OS_AUTH_URL)
14685 --swift-auth-token string Auth Token from alternate authentication - optional (OS_AUTH_TOKEN)
14686 --swift-auth-version int AuthVersion - optional - set to (1,2,3) if your auth URL has no version (ST_AUTH_VERSION)
14687 --swift-chunk-size SizeSuffix Above this size files will be chunked into a _segments container (default 5Gi)
14688 --swift-domain string User domain - optional (v3 auth) (OS_USER_DOMAIN_NAME)
14689 --swift-encoding MultiEncoder The encoding for the backend (default Slash,InvalidUtf8)
14690 --swift-endpoint-type string Endpoint type to choose from the service catalogue (OS_ENDPOINT_TYPE) (default "public")
14691 --swift-env-auth Get swift credentials from environment variables in standard OpenStack form
14692 --swift-key string API key or password (OS_PASSWORD)
14693 --swift-leave-parts-on-error If true avoid calling abort upload on a failure
14694 --swift-no-chunk Don't chunk files during streaming upload
14695 --swift-no-large-objects Disable support for static and dynamic large objects
14696 --swift-region string Region name - optional (OS_REGION_NAME)
14697 --swift-storage-policy string The storage policy to use when creating a new container
14698 --swift-storage-url string Storage URL - optional (OS_STORAGE_URL)
14699 --swift-tenant string Tenant name - optional for v1 auth, this or tenant_id required otherwise (OS_TENANT_NAME or OS_PROJECT_NAME)
14700 --swift-tenant-domain string Tenant domain - optional (v3 auth) (OS_PROJECT_DOMAIN_NAME)
14701 --swift-tenant-id string Tenant ID - optional for v1 auth, this or tenant required otherwise (OS_TENANT_ID)
14702 --swift-user string User name to log in (OS_USERNAME)
14703 --swift-user-id string User ID to log in - optional - most swift systems use user and leave this blank (v3 auth) (OS_USER_ID)
14704 --union-action-policy string Policy to choose upstream on ACTION category (default "epall")
14705 --union-cache-time int Cache time of usage and free space (in seconds) (default 120)
14706 --union-create-policy string Policy to choose upstream on CREATE category (default "epmfs")
14707 --union-min-free-space SizeSuffix Minimum viable free space for lfs/eplfs policies (default 1Gi)
14708 --union-search-policy string Policy to choose upstream on SEARCH category (default "ff")
14709 --union-upstreams string List of space separated upstreams
14710 --uptobox-access-token string Your access token
14711 --uptobox-encoding MultiEncoder The encoding for the backend (default Slash,LtGt,DoubleQuote,BackQuote,Del,Ctl,LeftSpace,InvalidUtf8,Dot)
14712 --webdav-bearer-token string Bearer token instead of user/pass (e.g. a Macaroon)
14713 --webdav-bearer-token-command string Command to run to get a bearer token
14714 --webdav-encoding string The encoding for the backend
14715 --webdav-headers CommaSepList Set HTTP headers for all transactions
14716 --webdav-pass string Password (obscured)
14717 --webdav-url string URL of http host to connect to
14718 --webdav-user string User name
14719 --webdav-vendor string Name of the WebDAV site/service/software you are using
14720 --yandex-auth-url string Auth server URL
14721 --yandex-client-id string OAuth Client Id
14722 --yandex-client-secret string OAuth Client Secret
14723 --yandex-encoding MultiEncoder The encoding for the backend (default Slash,Del,Ctl,InvalidUtf8,Dot)
14724 --yandex-hard-delete Delete files permanently rather than putting them into the trash
14725 --yandex-token string OAuth Access Token as a JSON blob
14726 --yandex-token-url string Token server url
14727 --zoho-auth-url string Auth server URL
14728 --zoho-client-id string OAuth Client Id
14729 --zoho-client-secret string OAuth Client Secret
14730 --zoho-encoding MultiEncoder The encoding for the backend (default Del,Ctl,InvalidUtf8)
14731 --zoho-region string Zoho region to connect to
14732 --zoho-token string OAuth Access Token as a JSON blob
14733 --zoho-token-url string Token server url
14734
14736 Introduction
14737 Docker 1.9 has added support for creating named volumes
14738 (https://docs.docker.com/storage/volumes/) via command-line interface
14739 (https://docs.docker.com/engine/reference/commandline/volume_create/)
14740 and mounting them in containers as a way to share data between them.
14741 Since Docker 1.10 you can create named volumes with Docker Compose
14742 (https://docs.docker.com/compose/) by descriptions in docker-com‐
14743 pose.yml (https://docs.docker.com/compose/compose-file/compose-file-
14744 v2/#volume-configuration-reference) files for use by container groups
14745 on a single host. As of Docker 1.12 volumes are supported by Docker
14746 Swarm (https://docs.docker.com/engine/swarm/key-concepts/) included
14747 with Docker Engine and created from descriptions in swarm compose v3
14748 (https://docs.docker.com/compose/compose-file/compose-file-v3/#volume-
14749 configuration-reference) files for use with swarm stacks across multi‐
14750 ple cluster nodes.
14751
14752 Docker Volume Plugins (https://docs.docker.com/engine/extend/plug‐
14753 ins_volume/) augment the default local volume driver included in Docker
14754 with stateful volumes shared across containers and hosts. Unlike local
14755 volumes, your data will not be deleted when such volume is removed.
14756 Plugins can run managed by the docker daemon, as a native system ser‐
14757 vice (under systemd, sysv or upstart) or as a standalone executable.
14758 Rclone can run as docker volume plugin in all these modes. It inter‐
14759 acts with the local docker daemon via plugin API (https://docs.dock‐
14760 er.com/engine/extend/plugin_api/) and handles mounting of remote file
14761 systems into docker containers so it must run on the same host as the
14762 docker daemon or on every Swarm node.
14763
14764 Getting started
14765 In the first example we will use the SFTP (https://rclone.org/sftp/)
14766 rclone volume with Docker engine on a standalone Ubuntu machine.
14767
14768 Start from installing Docker (https://docs.docker.com/engine/install/)
14769 on the host.
14770
14771 The FUSE driver is a prerequisite for rclone mounting and should be in‐
14772 stalled on host:
14773
14774 sudo apt-get -y install fuse
14775
14776 Create two directories required by rclone docker plugin:
14777
14778 sudo mkdir -p /var/lib/docker-plugins/rclone/config
14779 sudo mkdir -p /var/lib/docker-plugins/rclone/cache
14780
14781 Install the managed rclone docker plugin for your architecture (here
14782 amd64):
14783
14784 docker plugin install rclone/docker-volume-rclone:amd64 args="-v" --alias rclone --grant-all-permissions
14785 docker plugin list
14786
14787 Create your SFTP volume (https://rclone.org/sftp/#standard-options):
14788
14789 docker volume create firstvolume -d rclone -o type=sftp -o sftp-host=_hostname_ -o sftp-user=_username_ -o sftp-pass=_password_ -o allow-other=true
14790
14791 Note that since all options are static, you don't even have to run
14792 rclone config or create the rclone.conf file (but the config directory
14793 should still be present). In the simplest case you can use localhost
14794 as hostname and your SSH credentials as username and password. You can
14795 also change the remote path to your home directory on the host, for ex‐
14796 ample -o path=/home/username.
14797
14798 Time to create a test container and mount the volume into it:
14799
14800 docker run --rm -it -v firstvolume:/mnt --workdir /mnt ubuntu:latest bash
14801
14802 If all goes well, you will enter the new container and change right to
14803 the mounted SFTP remote. You can type ls to list the mounted directory
14804 or otherwise play with it. Type exit when you are done. The container
14805 will stop but the volume will stay, ready to be reused. When it's not
14806 needed anymore, remove it:
14807
14808 docker volume list
14809 docker volume remove firstvolume
14810
14811 Now let us try something more elaborate: Google Drive
14812 (https://rclone.org/drive/) volume on multi-node Docker Swarm.
14813
14814 You should start from installing Docker and FUSE, creating plugin di‐
14815 rectories and installing rclone plugin on every swarm node. Then setup
14816 the Swarm (https://docs.docker.com/engine/swarm/swarm-mode/).
14817
14818 Google Drive volumes need an access token which can be setup via web
14819 browser and will be periodically renewed by rclone. The managed plugin
14820 cannot run a browser so we will use a technique similar to the rclone
14821 setup on a headless box (https://rclone.org/remote_setup/).
14822
14823 Run rclone config (https://rclone.org/commands/rclone_config_create/)
14824 on another machine equipped with web browser and graphical user inter‐
14825 face. Create the Google Drive remote (https://rclone.org/drive/#stan‐
14826 dard-options). When done, transfer the resulting rclone.conf to the
14827 Swarm cluster and save as /var/lib/docker-plugins/rclone/con‐
14828 fig/rclone.conf on every node. By default this location is accessible
14829 only to the root user so you will need appropriate privileges. The re‐
14830 sulting config will look like this:
14831
14832 [gdrive]
14833 type = drive
14834 scope = drive
14835 drive_id = 1234567...
14836 root_folder_id = 0Abcd...
14837 token = {"access_token":...}
14838
14839 Now create the file named example.yml with a swarm stack description
14840 like this:
14841
14842 version: '3'
14843 services:
14844 heimdall:
14845 image: linuxserver/heimdall:latest
14846 ports: [8080:80]
14847 volumes: [configdata:/config]
14848 volumes:
14849 configdata:
14850 driver: rclone
14851 driver_opts:
14852 remote: 'gdrive:heimdall'
14853 allow_other: 'true'
14854 vfs_cache_mode: full
14855 poll_interval: 0
14856
14857 and run the stack:
14858
14859 docker stack deploy example -c ./example.yml
14860
14861 After a few seconds docker will spread the parsed stack description
14862 over cluster, create the example_heimdall service on port 8080, run
14863 service containers on one or more cluster nodes and request the exam‐
14864 ple_configdata volume from rclone plugins on the node hosts. You can
14865 use the following commands to confirm results:
14866
14867 docker service ls
14868 docker service ps example_heimdall
14869 docker volume ls
14870
14871 Point your browser to http://cluster.host.address:8080 and play with
14872 the service. Stop it with docker stack remove example when you are
14873 done. Note that the example_configdata volume(s) created on demand at
14874 the cluster nodes will not be automatically removed together with the
14875 stack but stay for future reuse. You can remove them manually by in‐
14876 voking the docker volume remove example_configdata command on every
14877 node.
14878
14879 Creating Volumes via CLI
14880 Volumes can be created with docker volume create (https://docs.dock‐
14881 er.com/engine/reference/commandline/volume_create/). Here are a few
14882 examples:
14883
14884 docker volume create vol1 -d rclone -o remote=storj: -o vfs-cache-mode=full
14885 docker volume create vol2 -d rclone -o remote=:storj,access_grant=xxx:heimdall
14886 docker volume create vol3 -d rclone -o type=storj -o path=heimdall -o storj-access-grant=xxx -o poll-interval=0
14887
14888 Note the -d rclone flag that tells docker to request volume from the
14889 rclone driver. This works even if you installed managed driver by its
14890 full name rclone/docker-volume-rclone because you provided the --alias
14891 rclone option.
14892
14893 Volumes can be inspected as follows:
14894
14895 docker volume list
14896 docker volume inspect vol1
14897
14898 Volume Configuration
14899 Rclone flags and volume options are set via the -o flag to the docker
14900 volume create command. They include backend-specific parameters as
14901 well as mount and VFS options. Also there are a few special -o op‐
14902 tions: remote, fs, type, path, mount-type and persist.
14903
14904 remote determines an existing remote name from the config file, with
14905 trailing colon and optionally with a remote path. See the full syntax
14906 in the rclone documentation (https://rclone.org/docs/#syntax-of-remote-
14907 paths). This option can be aliased as fs to prevent confusion with the
14908 remote parameter of such backends as crypt or alias.
14909
14910 The remote=:backend:dir/subdir syntax can be used to create on-the-fly
14911 (config-less) remotes (https://rclone.org/docs/#backend-path-to-dir),
14912 while the type and path options provide a simpler alternative for this.
14913 Using two split options
14914
14915 -o type=backend -o path=dir/subdir
14916
14917 is equivalent to the combined syntax
14918
14919 -o remote=:backend:dir/subdir
14920
14921 but is arguably easier to parameterize in scripts. The path part is
14922 optional.
14923
14924 Mount and VFS options (https://rclone.org/commands/rclone_serve_dock‐
14925 er/#options) as well as backend parameters
14926 (https://rclone.org/flags/#backend-flags) are named like their twin
14927 command-line flags without the -- CLI prefix. Optionally you can use
14928 underscores instead of dashes in option names. For example, --vfs-
14929 cache-mode full becomes -o vfs-cache-mode=full or -o
14930 vfs_cache_mode=full. Boolean CLI flags without value will gain the
14931 true value, e.g. --allow-other becomes -o allow-other=true or -o al‐
14932 low_other=true.
14933
14934 Please note that you can provide parameters only for the backend imme‐
14935 diately referenced by the backend type of mounted remote. If this is a
14936 wrapping backend like alias, chunker or crypt, you cannot provide op‐
14937 tions for the referred to remote or backend. This limitation is im‐
14938 posed by the rclone connection string parser. The only workaround is
14939 to feed plugin with rclone.conf or configure plugin arguments (see be‐
14940 low).
14941
14942 Special Volume Options
14943 mount-type determines the mount method and in general can be one of:
14944 mount, cmount, or mount2. This can be aliased as mount_type. It
14945 should be noted that the managed rclone docker plugin currently does
14946 not support the cmount method and mount2 is rarely needed. This option
14947 defaults to the first found method, which is usually mount so you gen‐
14948 erally won't need it.
14949
14950 persist is a reserved boolean (true/false) option. In future it will
14951 allow to persist on-the-fly remotes in the plugin rclone.conf file.
14952
14953 Connection Strings
14954 The remote value can be extended with connection strings
14955 (https://rclone.org/docs/#connection-strings) as an alternative way to
14956 supply backend parameters. This is equivalent to the -o backend op‐
14957 tions with one syntactic difference. Inside connection string the
14958 backend prefix must be dropped from parameter names but in the -o
14959 param=value array it must be present. For instance, compare the fol‐
14960 lowing option array
14961
14962 -o remote=:sftp:/home -o sftp-host=localhost
14963
14964 with equivalent connection string:
14965
14966 -o remote=:sftp,host=localhost:/home
14967
14968 This difference exists because flag options -o key=val include not only
14969 backend parameters but also mount/VFS flags and possibly other set‐
14970 tings. Also it allows to discriminate the remote option from the
14971 crypt-remote (or similarly named backend parameters) and arguably sim‐
14972 plifies scripting due to clearer value substitution.
14973
14974 Using with Swarm or Compose
14975 Both Docker Swarm and Docker Compose use YAML
14976 (http://yaml.org/spec/1.2/spec.html)-formatted text files to describe
14977 groups (stacks) of containers, their properties, networks and volumes.
14978 Compose uses the compose v2 (https://docs.docker.com/compose/compose-
14979 file/compose-file-v2/#volume-configuration-reference) format, Swarm us‐
14980 es the compose v3 (https://docs.docker.com/compose/compose-file/com‐
14981 pose-file-v3/#volume-configuration-reference) format. They are mostly
14982 similar, differences are explained in the docker documentation
14983 (https://docs.docker.com/compose/compose-file/compose-versioning/#up‐
14984 grading).
14985
14986 Volumes are described by the children of the top-level volumes: node.
14987 Each of them should be named after its volume and have at least two el‐
14988 ements, the self-explanatory driver: rclone value and the driver_opts:
14989 structure playing the same role as -o key=val CLI flags:
14990
14991 volumes:
14992 volume_name_1:
14993 driver: rclone
14994 driver_opts:
14995 remote: 'gdrive:'
14996 allow_other: 'true'
14997 vfs_cache_mode: full
14998 token: '{"type": "borrower", "expires": "2021-12-31"}'
14999 poll_interval: 0
15000
15001 Notice a few important details: - YAML prefers _ in option names in‐
15002 stead of -. - YAML treats single and double quotes interchangeably.
15003 Simple strings and integers can be left unquoted. - Boolean values
15004 must be quoted like 'true' or "false" because these two words are re‐
15005 served by YAML. - The filesystem string is keyed with remote (or with
15006 fs). Normally you can omit quotes here, but if the string ends with
15007 colon, you must quote it like remote: "storage_box:". - YAML is picky
15008 about surrounding braces in values as this is in fact another syntax
15009 for key/value mappings (http://yaml.org/spec/1.2/spec.html#id2790832).
15010 For example, JSON access tokens usually contain double quotes and sur‐
15011 rounding braces, so you must put them in single quotes.
15012
15013 Installing as Managed Plugin
15014 Docker daemon can install plugins from an image registry and run them
15015 managed. We maintain the docker-volume-rclone (https://hub.dock‐
15016 er.com/p/rclone/docker-volume-rclone/) plugin image on Docker Hub
15017 (https://hub.docker.com).
15018
15019 Rclone volume plugin requires Docker Engine >= 19.03.15
15020
15021 The plugin requires presence of two directories on the host before it
15022 can be installed. Note that plugin will not create them automatically.
15023 By default they must exist on host at the following locations (though
15024 you can tweak the paths): - /var/lib/docker-plugins/rclone/config is
15025 reserved for the rclone.conf config file and must exist even if it's
15026 empty and the config file is not present. - /var/lib/docker-plug‐
15027 ins/rclone/cache holds the plugin state file as well as optional VFS
15028 caches.
15029
15030 You can install managed plugin (https://docs.docker.com/engine/refer‐
15031 ence/commandline/plugin_install/) with default settings as follows:
15032
15033 docker plugin install rclone/docker-volume-rclone:amd64 --grant-all-permissions --alias rclone
15034
15035 The :amd64 part of the image specification after colon is called a tag.
15036 Usually you will want to install the latest plugin for your architec‐
15037 ture. In this case the tag will just name it, like amd64 above. The
15038 following plugin architectures are currently available: - amd64 - arm64
15039 - arm-v7
15040
15041 Sometimes you might want a concrete plugin version, not the latest one.
15042 Then you should use image tag in the form :ARCHITECTURE-VERSION. For
15043 example, to install plugin version v1.56.2 on architecture arm64 you
15044 will use tag arm64-1.56.2 (note the removed v) so the full image speci‐
15045 fication becomes rclone/docker-volume-rclone:arm64-1.56.2.
15046
15047 We also provide the latest plugin tag, but since docker does not sup‐
15048 port multi-architecture plugins as of the time of this writing, this
15049 tag is currently an alias for amd64. By convention the latest tag is
15050 the default one and can be omitted, thus both rclone/docker-volume-
15051 rclone:latest and just rclone/docker-volume-rclone will refer to the
15052 latest plugin release for the amd64 platform.
15053
15054 Also the amd64 part can be omitted from the versioned rclone plugin
15055 tags. For example, rclone image reference rclone/docker-volume-
15056 rclone:amd64-1.56.2 can be abbreviated as rclone/docker-volume-
15057 rclone:1.56.2 for convenience. However, for non-intel architectures
15058 you still have to use the full tag as amd64 or latest will fail to
15059 start.
15060
15061 Managed plugin is in fact a special container running in a namespace
15062 separate from normal docker containers. Inside it runs the rclone
15063 serve docker command. The config and cache directories are bind-mount‐
15064 ed into the container at start. The docker daemon connects to a unix
15065 socket created by the command inside the container. The command cre‐
15066 ates on-demand remote mounts right inside, then docker machinery propa‐
15067 gates them through kernel mount namespaces and bind-mounts into re‐
15068 questing user containers.
15069
15070 You can tweak a few plugin settings after installation when it's dis‐
15071 abled (not in use), for instance:
15072
15073 docker plugin disable rclone
15074 docker plugin set rclone RCLONE_VERBOSE=2 config=/etc/rclone args="--vfs-cache-mode=writes --allow-other"
15075 docker plugin enable rclone
15076 docker plugin inspect rclone
15077
15078 Note that if docker refuses to disable the plugin, you should find and
15079 remove all active volumes connected with it as well as containers and
15080 swarm services that use them. This is rather tedious so please care‐
15081 fully plan in advance.
15082
15083 You can tweak the following settings: args, config, cache, HTTP_PROXY,
15084 HTTPS_PROXY, NO_PROXY and RCLONE_VERBOSE. It's your task to keep plug‐
15085 in settings in sync across swarm cluster nodes.
15086
15087 args sets command-line arguments for the rclone serve docker command
15088 (none by default). Arguments should be separated by space so you will
15089 normally want to put them in quotes on the docker plugin set
15090 (https://docs.docker.com/engine/reference/commandline/plugin_set/) com‐
15091 mand line. Both serve docker flags (https://rclone.org/com‐
15092 mands/rclone_serve_docker/#options) and generic rclone flags
15093 (https://rclone.org/flags/) are supported, including backend parameters
15094 that will be used as defaults for volume creation. Note that plugin
15095 will fail (due to this docker bug (https://github.com/moby/mo‐
15096 by/blob/v20.10.7/plugin/v2/plugin.go#L195)) if the args value is empty.
15097 Use e.g. args="-v" as a workaround.
15098
15099 config=/host/dir sets alternative host location for the config directo‐
15100 ry. Plugin will look for rclone.conf here. It's not an error if the
15101 config file is not present but the directory must exist. Please note
15102 that plugin can periodically rewrite the config file, for example when
15103 it renews storage access tokens. Keep this in mind and try to avoid
15104 races between the plugin and other instances of rclone on the host that
15105 might try to change the config simultaneously resulting in corrupted
15106 rclone.conf. You can also put stuff like private key files for SFTP
15107 remotes in this directory. Just note that it's bind-mounted inside the
15108 plugin container at the predefined path /data/config. For example, if
15109 your key file is named sftp-box1.key on the host, the corresponding
15110 volume config option should read -o sftp-key-file=/data/config/sftp-
15111 box1.key.
15112
15113 cache=/host/dir sets alternative host location for the cache directory.
15114 The plugin will keep VFS caches here. Also it will create and maintain
15115 the docker-plugin.state file in this directory. When the plugin is
15116 restarted or reinstalled, it will look in this file to recreate any
15117 volumes that existed previously. However, they will not be re-mounted
15118 into consuming containers after restart. Usually this is not a problem
15119 as the docker daemon normally will restart affected user containers af‐
15120 ter failures, daemon restarts or host reboots.
15121
15122 RCLONE_VERBOSE sets plugin verbosity from 0 (errors only, by default)
15123 to 2 (debugging). Verbosity can be also tweaked via args="-v [-v]
15124 ...". Since arguments are more generic, you will rarely need this set‐
15125 ting. The plugin output by default feeds the docker daemon log on lo‐
15126 cal host. Log entries are reflected as errors in the docker log but
15127 retain their actual level assigned by rclone in the encapsulated mes‐
15128 sage string.
15129
15130 HTTP_PROXY, HTTPS_PROXY, NO_PROXY customize the plugin proxy settings.
15131
15132 You can set custom plugin options right when you install it, in one go:
15133
15134 docker plugin remove rclone
15135 docker plugin install rclone/docker-volume-rclone:amd64 \
15136 --alias rclone --grant-all-permissions \
15137 args="-v --allow-other" config=/etc/rclone
15138 docker plugin inspect rclone
15139
15140 Healthchecks
15141 The docker plugin volume protocol doesn't provide a way for plugins to
15142 inform the docker daemon that a volume is (un-)available. As a work‐
15143 around you can setup a healthcheck to verify that the mount is respond‐
15144 ing, for example:
15145
15146 services:
15147 my_service:
15148 image: my_image
15149 healthcheck:
15150 test: ls /path/to/rclone/mount || exit 1
15151 interval: 1m
15152 timeout: 15s
15153 retries: 3
15154 start_period: 15s
15155
15156 Running Plugin under Systemd
15157 In most cases you should prefer managed mode. Moreover, MacOS and Win‐
15158 dows do not support native Docker plugins. Please use managed mode on
15159 these systems. Proceed further only if you are on Linux.
15160
15161 First, install rclone (https://rclone.org/install/). You can just run
15162 it (type rclone serve docker and hit enter) for the test.
15163
15164 Install FUSE:
15165
15166 sudo apt-get -y install fuse
15167
15168 Download two systemd configuration files: docker-volume-rclone.service
15169 (https://raw.githubusercontent.com/rclone/rclone/master/contrib/docker-
15170 plugin/systemd/docker-volume-rclone.service) and docker-volume-
15171 rclone.socket (https://raw.githubusercontent.com/rclone/rclone/mas‐
15172 ter/contrib/docker-plugin/systemd/docker-volume-rclone.socket).
15173
15174 Put them to the /etc/systemd/system/ directory:
15175
15176 cp docker-volume-plugin.service /etc/systemd/system/
15177 cp docker-volume-plugin.socket /etc/systemd/system/
15178
15179 Please note that all commands in this section must be run as root but
15180 we omit sudo prefix for brevity. Now create directories required by
15181 the service:
15182
15183 mkdir -p /var/lib/docker-volumes/rclone
15184 mkdir -p /var/lib/docker-plugins/rclone/config
15185 mkdir -p /var/lib/docker-plugins/rclone/cache
15186
15187 Run the docker plugin service in the socket activated mode:
15188
15189 systemctl daemon-reload
15190 systemctl start docker-volume-rclone.service
15191 systemctl enable docker-volume-rclone.socket
15192 systemctl start docker-volume-rclone.socket
15193 systemctl restart docker
15194
15195 Or run the service directly: - run systemctl daemon-reload to let sys‐
15196 temd pick up new config - run systemctl enable docker-volume-
15197 rclone.service to make the new service start automatically when you
15198 power on your machine. - run systemctl start docker-volume-rclone.ser‐
15199 vice to start the service now. - run systemctl restart docker to
15200 restart docker daemon and let it detect the new plugin socket. Note
15201 that this step is not needed in managed mode where docker knows about
15202 plugin state changes.
15203
15204 The two methods are equivalent from the user perspective, but I person‐
15205 ally prefer socket activation.
15206
15207 Troubleshooting
15208 You can see managed plugin settings (https://docs.docker.com/engine/ex‐
15209 tend/#debugging-plugins) with
15210
15211 docker plugin list
15212 docker plugin inspect rclone
15213
15214 Note that docker (including latest 20.10.7) will not show actual values
15215 of args, just the defaults.
15216
15217 Use journalctl --unit docker to see managed plugin output as part of
15218 the docker daemon log. Note that docker reflects plugin lines as er‐
15219 rors but their actual level can be seen from encapsulated message
15220 string.
15221
15222 You will usually install the latest version of managed plugin for your
15223 platform. Use the following commands to print the actual installed
15224 version:
15225
15226 PLUGID=$(docker plugin list --no-trunc | awk '/rclone/{print$1}')
15227 sudo runc --root /run/docker/runtime-runc/plugins.moby exec $PLUGID rclone version
15228
15229 You can even use runc to run shell inside the plugin container:
15230
15231 sudo runc --root /run/docker/runtime-runc/plugins.moby exec --tty $PLUGID bash
15232
15233 Also you can use curl to check the plugin socket connectivity:
15234
15235 docker plugin list --no-trunc
15236 PLUGID=123abc...
15237 sudo curl -H Content-Type:application/json -XPOST -d {} --unix-socket /run/docker/plugins/$PLUGID/rclone.sock http://localhost/Plugin.Activate
15238
15239 though this is rarely needed.
15240
15241 Caveats
15242 Finally I'd like to mention a caveat with updating volume settings.
15243 Docker CLI does not have a dedicated command like docker volume update.
15244 It may be tempting to invoke docker volume create with updated options
15245 on existing volume, but there is a gotcha. The command will do noth‐
15246 ing, it won't even return an error. I hope that docker maintainers
15247 will fix this some day. In the meantime be aware that you must remove
15248 your volume before recreating it with new settings:
15249
15250 docker volume remove my_vol
15251 docker volume create my_vol -d rclone -o opt1=new_val1 ...
15252
15253 and verify that settings did update:
15254
15255 docker volume list
15256 docker volume inspect my_vol
15257
15258 If docker refuses to remove the volume, you should find containers or
15259 swarm services that use it and stop them first.
15260
15261 Getting started
15262 • Install rclone (https://rclone.org/install/) and setup your remotes.
15263
15264 • Bisync will create its working directory at ~/.cache/rclone/bisync on
15265 Linux or C:\Users\MyLogin\AppData\Local\rclone\bisync on Windows.
15266 Make sure that this location is writable.
15267
15268 • Run bisync with the --resync flag, specifying the paths to the local
15269 and remote sync directory roots.
15270
15271 • For successive sync runs, leave off the --resync flag.
15272
15273 • Consider using a filters file for excluding unnecessary files and di‐
15274 rectories from the sync.
15275
15276 • Consider setting up the --check-access feature for safety.
15277
15278 • On Linux, consider setting up a crontab entry. bisync can safely run
15279 in concurrent cron jobs thanks to lock files it maintains.
15280
15281 Here is a typical run log (with timestamps removed for clarity):
15282
15283 rclone bisync /testdir/path1/ /testdir/path2/ --verbose
15284 INFO : Synching Path1 "/testdir/path1/" with Path2 "/testdir/path2/"
15285 INFO : Path1 checking for diffs
15286 INFO : - Path1 File is new - file11.txt
15287 INFO : - Path1 File is newer - file2.txt
15288 INFO : - Path1 File is newer - file5.txt
15289 INFO : - Path1 File is newer - file7.txt
15290 INFO : - Path1 File was deleted - file4.txt
15291 INFO : - Path1 File was deleted - file6.txt
15292 INFO : - Path1 File was deleted - file8.txt
15293 INFO : Path1: 7 changes: 1 new, 3 newer, 0 older, 3 deleted
15294 INFO : Path2 checking for diffs
15295 INFO : - Path2 File is new - file10.txt
15296 INFO : - Path2 File is newer - file1.txt
15297 INFO : - Path2 File is newer - file5.txt
15298 INFO : - Path2 File is newer - file6.txt
15299 INFO : - Path2 File was deleted - file3.txt
15300 INFO : - Path2 File was deleted - file7.txt
15301 INFO : - Path2 File was deleted - file8.txt
15302 INFO : Path2: 7 changes: 1 new, 3 newer, 0 older, 3 deleted
15303 INFO : Applying changes
15304 INFO : - Path1 Queue copy to Path2 - /testdir/path2/file11.txt
15305 INFO : - Path1 Queue copy to Path2 - /testdir/path2/file2.txt
15306 INFO : - Path2 Queue delete - /testdir/path2/file4.txt
15307 NOTICE: - WARNING New or changed in both paths - file5.txt
15308 NOTICE: - Path1 Renaming Path1 copy - /testdir/path1/file5.txt..path1
15309 NOTICE: - Path1 Queue copy to Path2 - /testdir/path2/file5.txt..path1
15310 NOTICE: - Path2 Renaming Path2 copy - /testdir/path2/file5.txt..path2
15311 NOTICE: - Path2 Queue copy to Path1 - /testdir/path1/file5.txt..path2
15312 INFO : - Path2 Queue copy to Path1 - /testdir/path1/file6.txt
15313 INFO : - Path1 Queue copy to Path2 - /testdir/path2/file7.txt
15314 INFO : - Path2 Queue copy to Path1 - /testdir/path1/file1.txt
15315 INFO : - Path2 Queue copy to Path1 - /testdir/path1/file10.txt
15316 INFO : - Path1 Queue delete - /testdir/path1/file3.txt
15317 INFO : - Path2 Do queued copies to - Path1
15318 INFO : - Path1 Do queued copies to - Path2
15319 INFO : - Do queued deletes on - Path1
15320 INFO : - Do queued deletes on - Path2
15321 INFO : Updating listings
15322 INFO : Validating listings for Path1 "/testdir/path1/" vs Path2 "/testdir/path2/"
15323 INFO : Bisync successful
15324
15325 Command line syntax
15326 $ rclone bisync --help
15327 Usage:
15328 rclone bisync remote1:path1 remote2:path2 [flags]
15329
15330 Positional arguments:
15331 Path1, Path2 Local path, or remote storage with ':' plus optional path.
15332 Type 'rclone listremotes' for list of configured remotes.
15333
15334 Optional Flags:
15335 --check-access Ensure expected `RCLONE_TEST` files are found on
15336 both Path1 and Path2 filesystems, else abort.
15337 --check-filename FILENAME Filename for `--check-access` (default: `RCLONE_TEST`)
15338 --check-sync CHOICE Controls comparison of final listings:
15339 `true | false | only` (default: true)
15340 If set to `only`, bisync will only compare listings
15341 from the last run but skip actual sync.
15342 --filters-file PATH Read filtering patterns from a file
15343 --max-delete PERCENT Safety check on maximum percentage of deleted files allowed.
15344 If exceeded, the bisync run will abort. (default: 50%)
15345 --force Bypass `--max-delete` safety check and run the sync.
15346 Consider using with `--verbose`
15347 --remove-empty-dirs Remove empty directories at the final cleanup step.
15348 -1, --resync Performs the resync run.
15349 Warning: Path1 files may overwrite Path2 versions.
15350 Consider using `--verbose` or `--dry-run` first.
15351 --localtime Use local time in listings (default: UTC)
15352 --no-cleanup Retain working files (useful for troubleshooting and testing).
15353 --workdir PATH Use custom working directory (useful for testing).
15354 (default: `~/.cache/rclone/bisync`)
15355 -n, --dry-run Go through the motions - No files are copied/deleted.
15356 -v, --verbose Increases logging verbosity.
15357 May be specified more than once for more details.
15358 -h, --help help for bisync
15359
15360 Arbitrary rclone flags may be specified on the bisync command line
15361 (https://rclone.org/commands/rclone_bisync/), for example rclone bisync
15362 ./testdir/path1/ gdrive:testdir/path2/ --drive-skip-gdocs -v -v --time‐
15363 out 10s Note that interactions of various rclone flags with bisync
15364 process flow has not been fully tested yet.
15365
15366 Paths
15367 Path1 and Path2 arguments may be references to any mix of local direc‐
15368 tory paths (absolute or relative), UNC paths (//server/share/path),
15369 Windows drive paths (with a drive letter and :) or configured remotes
15370 (https://rclone.org/docs/#syntax-of-remote-paths) with optional subdi‐
15371 rectory paths. Cloud references are distinguished by having a : in the
15372 argument (see Windows support below).
15373
15374 Path1 and Path2 are treated equally, in that neither has priority for
15375 file changes, and access efficiency does not change whether a remote is
15376 on Path1 or Path2.
15377
15378 The listings in bisync working directory (default:
15379 ~/.cache/rclone/bisync) are named based on the Path1 and Path2 argu‐
15380 ments so that separate syncs to individual directories within the tree
15381 may be set up, e.g.: path_to_local_tree..dropbox_subdir.lst.
15382
15383 Any empty directories after the sync on both the Path1 and Path2
15384 filesystems are not deleted by default. If the --remove-empty-dirs
15385 flag is specified, then both paths will have any empty directories
15386 purged as the last step in the process.
15387
15388 Command-line flags
15389 --resync
15390 This will effectively make both Path1 and Path2 filesystems contain a
15391 matching superset of all files. Path2 files that do not exist in Path1
15392 will be copied to Path1, and the process will then sync the Path1 tree
15393 to Path2.
15394
15395 The base directories on the both Path1 and Path2 filesystems must exist
15396 or bisync will fail. This is required for safety - that bisync can
15397 verify that both paths are valid.
15398
15399 When using --resync a newer version of a file on the Path2 filesystem
15400 will be overwritten by the Path1 filesystem version. Carefully evalu‐
15401 ate deltas using --dry-run (https://rclone.org/flags/#non-backend-
15402 flags).
15403
15404 For a resync run, one of the paths may be empty (no files in the path
15405 tree). The resync run should result in files on both paths, else a
15406 normal non-resync run will fail.
15407
15408 For a non-resync run, either path being empty (no files in the tree)
15409 fails with Empty current PathN listing. Cannot sync to an empty direc‐
15410 tory: X.pathN.lst This is a safety check that an unexpected empty path
15411 does not result in deleting everything in the other path.
15412
15413 --check-access
15414 Access check files are an additional safety measure against data loss.
15415 bisync will ensure it can find matching RCLONE_TEST files in the same
15416 places in the Path1 and Path2 filesystems. Time stamps and file con‐
15417 tents are not important, just the names and locations. Place one or
15418 more RCLONE_TEST files in the Path1 or Path2 filesystem and then do ei‐
15419 ther a run without --check-access or a --resync to set matching files
15420 on both filesystems. If you have symbolic links in your sync tree it
15421 is recommended to place RCLONE_TEST files in the linked-to directory
15422 tree to protect against bisync assuming a bunch of deleted files if the
15423 linked-to tree should not be accessible. Also see the --check-filename
15424 flag.
15425
15426 --max-delete
15427 As a safety check, if greater than the --max-delete percent of files
15428 were deleted on either the Path1 or Path2 filesystem, then bisync will
15429 abort with a warning message, without making any changes. The default
15430 --max-delete is 50%. One way to trigger this limit is to rename a di‐
15431 rectory that contains more than half of your files. This will appear
15432 to bisync as a bunch of deleted files and a bunch of new files. This
15433 safety check is intended to block bisync from deleting all of the files
15434 on both filesystems due to a temporary network access issue, or if the
15435 user had inadvertently deleted the files on one side or the other. To
15436 force the sync either set a different delete percentage limit, e.g.
15437 --max-delete 75 (allows up to 75% deletion), or use --force to bypass
15438 the check.
15439
15440 Also see the all files changed check.
15441
15442 --filters-file
15443 By using rclone filter features you can exclude file types or directory
15444 sub-trees from the sync. See the bisync filters section and generic
15445 --filter-from (https://rclone.org/filtering/#filter-from-read-filter‐
15446 ing-patterns-from-a-file) documentation. An example filters file con‐
15447 tains filters for non-allowed files for synching with Dropbox.
15448
15449 If you make changes to your filters file then bisync requires a run
15450 with --resync. This is a safety feature, which avoids existing files
15451 on the Path1 and/or Path2 side from seeming to disappear from view
15452 (since they are excluded in the new listings), which would fool bisync
15453 into seeing them as deleted (as compared to the prior run listings),
15454 and then bisync would proceed to delete them for real.
15455
15456 To block this from happening bisync calculates an MD5 hash of the fil‐
15457 ters file and stores the hash in a .md5 file in the same place as your
15458 filters file. On the next runs with --filters-file set, bisync re-cal‐
15459 culates the MD5 hash of the current filters file and compares it to the
15460 hash stored in .md5 file. If they don't match the run aborts with a
15461 critical error and thus forces you to do a --resync, likely avoiding a
15462 disaster.
15463
15464 --check-sync
15465 Enabled by default, the check-sync function checks that all of the same
15466 files exist in both the Path1 and Path2 history listings. This check-
15467 sync integrity check is performed at the end of the sync run by de‐
15468 fault. Any untrapped failing copy/deletes between the two paths might
15469 result in differences between the two listings and in the untracked
15470 file content differences between the two paths. A resync run would
15471 correct the error.
15472
15473 Note that the default-enabled integrity check locally executes a load
15474 of both the final Path1 and Path2 listings, and thus adds to the run
15475 time of a sync. Using --check-sync=false will disable it and may sig‐
15476 nificantly reduce the sync run times for very large numbers of files.
15477
15478 The check may be run manually with --check-sync=only. It runs only the
15479 integrity check and terminates without actually synching.
15480
15481 Operation
15482 Runtime flow details
15483 bisync retains the listings of the Path1 and Path2 filesystems from the
15484 prior run. On each successive run it will:
15485
15486 • list files on path1 and path2, and check for changes on each side.
15487 Changes include New, Newer, Older, and Deleted files.
15488
15489 • Propagate changes on path1 to path2, and vice-versa.
15490
15491 Safety measures
15492 • Lock file prevents multiple simultaneous runs when taking a while.
15493 This can be particularly useful if bisync is run by cron scheduler.
15494
15495 • Handle change conflicts non-destructively by creating ..path1 and
15496 ..path2 file versions.
15497
15498 • File system access health check using RCLONE_TEST files (see the
15499 --check-access flag).
15500
15501 • Abort on excessive deletes - protects against a failed listing being
15502 interpreted as all the files were deleted. See the --max-delete and
15503 --force flags.
15504
15505 • If something evil happens, bisync goes into a safe state to block
15506 damage by later runs. (See Error Handling)
15507
15508 Normal sync checks
15509 Type Description Result Implementation
15510 ────────────────────────────────────────────────────────────────────────────────
15511 Path2 File is new on Path2, does Path2 version rclone copy Path2
15512 new not exist on Path1 survives to Path1
15513 Path2 File is newer on Path2, un‐ Path2 version rclone copy Path2
15514 newer changed on Path1 survives to Path1
15515 Path2 File is deleted on Path2, File is deleted rclone delete
15516 deleted unchanged on Path1 Path1
15517 Path1 File is new on Path1, does Path1 version rclone copy Path1
15518 new not exist on Path2 survives to Path2
15519 Path1 File is newer on Path1, un‐ Path1 version rclone copy Path1
15520 newer changed on Path2 survives to Path2
15521 Path1 File is older on Path1, un‐ Path1 version rclone copy Path1
15522 older changed on Path2 survives to Path2
15523 Path2 File is older on Path2, un‐ Path2 version rclone copy Path2
15524 older changed on Path1 survives to Path1
15525 Path1 File no longer exists on File is deleted rclone delete
15526 deleted Path1 Path2
15527
15528 Unusual sync checks
15529 Type Description Result Implementa‐
15530 tion
15531 ────────────────────────────────────────────────────────────────────────────────
15532 Path1 new AND File is new on Path1 Files renamed to rclone copy
15533 Path2 new AND new on Path2 _Path1 and _Path2 _Path2 file
15534 to Path1,
15535 rclone copy
15536 _Path1 file
15537 to Path2
15538 Path2 newer AND File is newer on Files renamed to rclone copy
15539 Path1 changed Path2 AND also _Path1 and _Path2 _Path2 file
15540 changed (newer/old‐ to Path1,
15541 er/size) on Path1 rclone copy
15542 _Path1 file
15543 to Path2
15544 Path2 newer AND File is newer on Path2 version sur‐ rclone copy
15545 Path1 deleted Path2 AND also delet‐ vives Path2 to
15546 ed on Path1 Path1
15547 Path2 deleted AND File is deleted on Path1 version sur‐ rclone copy
15548 Path1 changed Path2 AND changed vives Path1 to
15549 (newer/older/size) on Path2
15550 Path1
15551 Path1 deleted AND File is deleted on Path2 version sur‐ rclone copy
15552 Path2 changed Path1 AND changed vives Path2 to
15553 (newer/older/size) on Path1
15554 Path2
15555
15556 All files changed check
15557 if all prior existing files on either of the filesystems have changed
15558 (e.g. timestamps have changed due to changing the system's timezone)
15559 then bisync will abort without making any changes. Any new files are
15560 not considered for this check. You could use --force to force the sync
15561 (whichever side has the changed timestamp files wins). Alternately, a
15562 --resync may be used (Path1 versions will be pushed to Path2). Consid‐
15563 er the situation carefully and perhaps use --dry-run before you commit
15564 to the changes.
15565
15566 Modification time
15567 Bisync relies on file timestamps to identify changed files and will
15568 refuse to operate if backend lacks the modification time support.
15569
15570 If you or your application should change the content of a file without
15571 changing the modification time then bisync will not notice the change,
15572 and thus will not copy it to the other side.
15573
15574 Note that on some cloud storage systems it is not possible to have file
15575 timestamps that match precisely between the local and other filesys‐
15576 tems.
15577
15578 Bisync's approach to this problem is by tracking the changes on each
15579 side separately over time with a local database of files in that side
15580 then applying the resulting changes on the other side.
15581
15582 Error handling
15583 Certain bisync critical errors, such as file copy/move failing, will
15584 result in a bisync lockout of following runs. The lockout is asserted
15585 because the sync status and history of the Path1 and Path2 filesystems
15586 cannot be trusted, so it is safer to block any further changes until
15587 someone checks things out. The recovery is to do a --resync again.
15588
15589 It is recommended to use --resync --dry-run --verbose initially and
15590 carefully review what changes will be made before running the --resync
15591 without --dry-run.
15592
15593 Most of these events come up due to a error status from an internal
15594 call. On such a critical error the {...}.path1.lst and {...}.path2.lst
15595 listing files are renamed to extension .lst-err, which blocks any fu‐
15596 ture bisync runs (since the normal .lst files are not found). Bisync
15597 keeps them under bisync subdirectory of the rclone cache directory,
15598 typically at ${HOME}/.cache/rclone/bisync/ on Linux.
15599
15600 Some errors are considered temporary and re-running the bisync is not
15601 blocked. The critical return blocks further bisync runs.
15602
15603 Lock file
15604 When bisync is running, a lock file is created in the bisync working
15605 directory, typically at ~/.cache/rclone/bisync/PATH1..PATH2.lck on Lin‐
15606 ux. If bisync should crash or hang, the lock file will remain in place
15607 and block any further runs of bisync for the same paths. Delete the
15608 lock file as part of debugging the situation. The lock file effective‐
15609 ly blocks follow-on (e.g., scheduled by cron) runs when the prior invo‐
15610 cation is taking a long time. The lock file contains PID of the block‐
15611 ing process, which may help in debug.
15612
15613 Note that while concurrent bisync runs are allowed, be very cautious
15614 that there is no overlap in the trees being synched between concurrent
15615 runs, lest there be replicated files, deleted files and general mayhem.
15616
15617 Return codes
15618 rclone bisync returns the following codes to calling program: - 0 on a
15619 successful run, - 1 for a non-critical failing run (a rerun may be suc‐
15620 cessful), - 2 for a critically aborted run (requires a --resync to re‐
15621 cover).
15622
15623 Limitations
15624 Supported backends
15625 Bisync is considered BETA and has been tested with the following back‐
15626 ends: - Local filesystem - Google Drive - Dropbox - OneDrive - S3 -
15627 SFTP - Yandex Disk
15628
15629 It has not been fully tested with other services yet. If it works, or
15630 sorta works, please let us know and we'll update the list. Run the
15631 test suite to check for proper operation as described below.
15632
15633 First release of rclone bisync requires that underlying backend sup‐
15634 ported the modification time feature and will refuse to run otherwise.
15635 This limitation will be lifted in a future rclone bisync release.
15636
15637 Concurrent modifications
15638 When using Local, FTP or SFTP remotes rclone does not create temporary
15639 files at the destination when copying, and thus if the connection is
15640 lost the created file may be corrupt, which will likely propagate back
15641 to the original path on the next sync, resulting in data loss. This
15642 will be solved in a future release, there is no workaround at the mo‐
15643 ment.
15644
15645 Files that change during a bisync run may result in data loss. This
15646 has been seen in a highly dynamic environment, where the filesystem is
15647 getting hammered by running processes during the sync. The solution is
15648 to sync at quiet times or filter out unnecessary directories and files.
15649
15650 Empty directories
15651 New empty directories on one path are not propagated to the other side.
15652 This is because bisync (and rclone) natively works on files not direc‐
15653 tories. The following sequence is a workaround but will not propagate
15654 the delete of an empty directory to the other side:
15655
15656 rclone bisync PATH1 PATH2
15657 rclone copy PATH1 PATH2 --filter "+ */" --filter "- **" --create-empty-src-dirs
15658 rclone copy PATH2 PATH2 --filter "+ */" --filter "- **" --create-empty-src-dirs
15659
15660 Renamed directories
15661 Renaming a folder on the Path1 side results is deleting all files on
15662 the Path2 side and then copying all files again from Path1 to Path2.
15663 Bisync sees this as all files in the old directory name as deleted and
15664 all files in the new directory name as new. Similarly, renaming a di‐
15665 rectory on both sides to the same name will result in creating ..path1
15666 and ..path2 files on both sides. Currently the most effective and ef‐
15667 ficient method of renaming a directory is to rename it on both sides,
15668 then do a --resync.
15669
15670 Case sensitivity
15671 Synching with case-insensitive filesystems, such as Windows or Box, can
15672 result in file name conflicts. This will be fixed in a future release.
15673 The near term workaround is to make sure that files on both sides don't
15674 have spelling case differences (Smile.jpg vs. smile.jpg).
15675
15676 Windows support
15677 Bisync has been tested on Windows 8.1, Windows 10 Pro 64-bit and on
15678 Windows GitHub runners.
15679
15680 Drive letters are allowed, including drive letters mapped to network
15681 drives (rclone bisync J:\localsync GDrive:). If a drive letter is
15682 omitted, the shell current drive is the default. Drive letters are a
15683 single character follows by :, so cloud names must be more than one
15684 character long.
15685
15686 Absolute paths (with or without a drive letter), and relative paths
15687 (with or without a drive letter) are supported.
15688
15689 Working directory is created at C:\Users\MyLogin\AppData\Lo‐
15690 cal\rclone\bisync.
15691
15692 Note that bisync output may show a mix of forward / and back \ slashes.
15693
15694 Be careful of case independent directory and file naming on Windows vs.
15695 case dependent Linux
15696
15697 Filtering
15698 See filtering documentation (https://rclone.org/filtering/) for how
15699 filter rules are written and interpreted.
15700
15701 Bisync's --filters-file flag slightly extends the rclone's --filter-
15702 from (https://rclone.org/filtering/#filter-from-read-filtering-pat‐
15703 terns-from-a-file) filtering mechanism. For a given bisync run you may
15704 provide only one --filters-file. The --include*, --exclude*, and
15705 --filter flags are also supported.
15706
15707 How to filter directories
15708 Filtering portions of the directory tree is a critical feature for
15709 synching.
15710
15711 Examples of directory trees (always beneath the Path1/Path2 root level)
15712 you may want to exclude from your sync: - Directory trees containing
15713 only software build intermediate files. - Directory trees containing
15714 application temporary files and data such as the Windows C:\Users\MyLo‐
15715 gin\AppData\ tree. - Directory trees containing files that are large,
15716 less important, or are getting thrashed continuously by ongoing pro‐
15717 cesses.
15718
15719 On the other hand, there may be only select directories that you actu‐
15720 ally want to sync, and exclude all others. See the Example include-
15721 style filters for Windows user directories below.
15722
15723 Filters file writing guidelines
15724 1. Begin with excluding directory trees:
15725
15726 • e.g. `- /AppData/`
15727
15728 • ** on the end is not necessary. Once a given directory level is
15729 excluded then everything beneath it won't be looked at by rclone.
15730
15731 • Exclude such directories that are unneeded, are big, dynamically
15732 thrashed, or where there may be access permission issues.
15733
15734 • Excluding such dirs first will make rclone operations (much)
15735 faster.
15736
15737 • Specific files may also be excluded, as with the Dropbox exclu‐
15738 sions example below.
15739
15740 2. Decide if its easier (or cleaner) to:
15741
15742 • Include select directories and therefore exclude everything else
15743 -- or --
15744
15745 • Exclude select directories and therefore include everything else
15746
15747 3. Include select directories:
15748
15749 • Add lines like: `+ /Documents/PersonalFiles/**` to select which
15750 directories to include in the sync.
15751
15752 • ** on the end specifies to include the full depth of the speci‐
15753 fied tree.
15754
15755 • With Include-style filters, files at the Path1/Path2 root are not
15756 included. They may be included with `+ /*`.
15757
15758 • Place RCLONE_TEST files within these included directory trees.
15759 They will only be looked for in these directory trees.
15760
15761 • Finish by excluding everything else by adding `- **` at the end
15762 of the filters file.
15763
15764 • Disregard step 4.
15765
15766 4. Exclude select directories:
15767
15768 • Add more lines like in step 1. For example: -/Desktop/temp‐
15769 files/, or `- /testdir/. Again, a**` on the end is not neces‐
15770 sary.
15771
15772 • Do not add a `- **` in the file. Without this line, everything
15773 will be included that has not be explicitly excluded.
15774
15775 • Disregard step 3.
15776
15777 A few rules for the syntax of a filter file expanding on filtering doc‐
15778 umentation (https://rclone.org/filtering/):
15779
15780 • Lines may start with spaces and tabs - rclone strips leading white‐
15781 space.
15782
15783 • If the first non-whitespace character is a # then the line is a com‐
15784 ment and will be ignored.
15785
15786 • Blank lines are ignored.
15787
15788 • The first non-whitespace character on a filter line must be a + or -.
15789
15790 • Exactly 1 space is allowed between the +/- and the path term.
15791
15792 • Only forward slashes (/) are used in path terms, even on Windows.
15793
15794 • The rest of the line is taken as the path term. Trailing whitespace
15795 is taken literally, and probably is an error.
15796
15797 Example include-style filters for Windows user directories
15798 This Windows include-style example is based on the sync root (Path1)
15799 set to C:\Users\MyLogin. The strategy is to select specific directo‐
15800 ries to be synched with a network drive (Path2).
15801
15802 • `- /AppData/` excludes an entire tree of Windows stored stuff that
15803 need not be synched. In my case, AppData has >11 GB of stuff I don't
15804 care about, and there are some subdirectories beneath AppData that
15805 are not accessible to my user login, resulting in bisync critical
15806 aborts.
15807
15808 • Windows creates cache files starting with both upper and lowercase
15809 NTUSER at C:\Users\MyLogin. These files may be dynamic, locked, and
15810 are generally don't care.
15811
15812 • There are just a few directories with my data that I do want synched,
15813 in the form of `+ /. By selecting only the directory trees I want
15814 to avoid the dozen plus directories that various apps make atC:`.
15815
15816 • Include files in the root of the sync point, C:\Users\MyLogin, by
15817 adding the `+ /*` line.
15818
15819 • This is an Include-style filters file, therefore it ends with `- **`
15820 which excludes everything not explicitly included.
15821
15822 - /AppData/
15823 - NTUSER*
15824 - ntuser*
15825 + /Documents/Family/**
15826 + /Documents/Sketchup/**
15827 + /Documents/Microcapture_Photo/**
15828 + /Documents/Microcapture_Video/**
15829 + /Desktop/**
15830 + /Pictures/**
15831 + /*
15832 - **
15833
15834 Note also that Windows implements several "library" links such as
15835 C:\Users\MyLogin\My Documents\My Music pointing to C:\Users\MyLogin\Mu‐
15836 sic. rclone sees these as links, so you must add --links to the bisync
15837 command line if you which to follow these links. I find that I get
15838 permission errors in trying to follow the links, so I don't include the
15839 rclone --links flag, but then you get lots of Can't follow symlink...
15840 noise from rclone about not following the links. This noise can be
15841 quashed by adding --quiet to the bisync command line.
15842
15843 Example exclude-style filters files for use with Dropbox
15844 • Dropbox disallows synching the listed temporary and configuration/da‐
15845 ta files. The `- ` filters exclude these files where ever they may
15846 occur in the sync tree. Consider adding similar exclusions for file
15847 types you don't need to sync, such as core dump and software build
15848 files.
15849
15850 • bisync testing creates /testdir/ at the top level of the sync tree,
15851 and usually deletes the tree after the test. If a normal sync should
15852 run while the /testdir/ tree exists the --check-access phase may fail
15853 due to unbalanced RCLONE_TEST files. The `- /testdir/` filter blocks
15854 this tree from being synched. You don't need this exclusion if you
15855 are not doing bisync development testing.
15856
15857 • Everything else beneath the Path1/Path2 root will be synched.
15858
15859 • RCLONE_TEST files may be placed anywhere within the tree, including
15860 the root.
15861
15862 Example filters file for Dropbox
15863 # Filter file for use with bisync
15864 # See https://rclone.org/filtering/ for filtering rules
15865 # NOTICE: If you make changes to this file you MUST do a --resync run.
15866 # Run with --dry-run to see what changes will be made.
15867
15868 # Dropbox wont sync some files so filter them away here.
15869 # See https://help.dropbox.com/installs-integrations/sync-uploads/files-not-syncing
15870 - .dropbox.attr
15871 - ~*.tmp
15872 - ~$*
15873 - .~*
15874 - desktop.ini
15875 - .dropbox
15876
15877 # Used for bisync testing, so excluded from normal runs
15878 - /testdir/
15879
15880 # Other example filters
15881 #- /TiBU/
15882 #- /Photos/
15883
15884 How --check-access handles filters
15885 At the start of a bisync run, listings are gathered for Path1 and Path2
15886 while using the user's --filters-file. During the check access phase,
15887 bisync scans these listings for RCLONE_TEST files. Any RCLONE_TEST
15888 files hidden by the --filters-file are not in the listings and thus not
15889 checked during the check access phase.
15890
15891 Troubleshooting
15892 Reading bisync logs
15893 Here are two normal runs. The first one has a newer file on the re‐
15894 mote. The second has no deltas between local and remote.
15895
15896 2021/05/16 00:24:38 INFO : Synching Path1 "/path/to/local/tree/" with Path2 "dropbox:/"
15897 2021/05/16 00:24:38 INFO : Path1 checking for diffs
15898 2021/05/16 00:24:38 INFO : - Path1 File is new - file.txt
15899 2021/05/16 00:24:38 INFO : Path1: 1 changes: 1 new, 0 newer, 0 older, 0 deleted
15900 2021/05/16 00:24:38 INFO : Path2 checking for diffs
15901 2021/05/16 00:24:38 INFO : Applying changes
15902 2021/05/16 00:24:38 INFO : - Path1 Queue copy to Path2 - dropbox:/file.txt
15903 2021/05/16 00:24:38 INFO : - Path1 Do queued copies to - Path2
15904 2021/05/16 00:24:38 INFO : Updating listings
15905 2021/05/16 00:24:38 INFO : Validating listings for Path1 "/path/to/local/tree/" vs Path2 "dropbox:/"
15906 2021/05/16 00:24:38 INFO : Bisync successful
15907
15908 2021/05/16 00:36:52 INFO : Synching Path1 "/path/to/local/tree/" with Path2 "dropbox:/"
15909 2021/05/16 00:36:52 INFO : Path1 checking for diffs
15910 2021/05/16 00:36:52 INFO : Path2 checking for diffs
15911 2021/05/16 00:36:52 INFO : No changes found
15912 2021/05/16 00:36:52 INFO : Updating listings
15913 2021/05/16 00:36:52 INFO : Validating listings for Path1 "/path/to/local/tree/" vs Path2 "dropbox:/"
15914 2021/05/16 00:36:52 INFO : Bisync successful
15915
15916 Dry run oddity
15917 The --dry-run messages may indicate that it would try to delete some
15918 files. For example, if a file is new on Path2 and does not exist on
15919 Path1 then it would normally be copied to Path1, but with --dry-run en‐
15920 abled those copies don't happen, which leads to the attempted delete on
15921 the Path2, blocked again by --dry-run: ... Not deleting as --dry-run.
15922
15923 This whole confusing situation is an artifact of the --dry-run flag.
15924 Scrutinize the proposed deletes carefully, and if the files would have
15925 been copied to Path1 then the threatened deletes on Path2 may be disre‐
15926 garded.
15927
15928 Retries
15929 Rclone has built in retries. If you run with --verbose you'll see er‐
15930 ror and retry messages such as shown below. This is usually not a bug.
15931 If at the end of the run you see Bisync successful and not Bisync crit‐
15932 ical error or Bisync aborted then the run was successful, and you can
15933 ignore the error messages.
15934
15935 The following run shows an intermittent fail. Lines 5 and _6- are low
15936 level messages. Line 6 is a bubbled-up warning message, conveying the
15937 error. Rclone normally retries failing commands, so there may be nu‐
15938 merous such messages in the log.
15939
15940 Since there are no final error/warning messages on line 7, rclone has
15941 recovered from failure after a retry, and the overall sync was success‐
15942 ful.
15943
15944 1: 2021/05/14 00:44:12 INFO : Synching Path1 "/path/to/local/tree" with Path2 "dropbox:"
15945 2: 2021/05/14 00:44:12 INFO : Path1 checking for diffs
15946 3: 2021/05/14 00:44:12 INFO : Path2 checking for diffs
15947 4: 2021/05/14 00:44:12 INFO : Path2: 113 changes: 22 new, 0 newer, 0 older, 91 deleted
15948 5: 2021/05/14 00:44:12 ERROR : /path/to/local/tree/objects/af: error listing: unexpected end of JSON input
15949 6: 2021/05/14 00:44:12 NOTICE: WARNING listing try 1 failed. - dropbox:
15950 7: 2021/05/14 00:44:12 INFO : Bisync successful
15951
15952 This log shows a Critical failure which requires a --resync to recover
15953 from. See the Runtime Error Handling section.
15954
15955 2021/05/12 00:49:40 INFO : Google drive root '': Waiting for checks to finish
15956 2021/05/12 00:49:40 INFO : Google drive root '': Waiting for transfers to finish
15957 2021/05/12 00:49:40 INFO : Google drive root '': not deleting files as there were IO errors
15958 2021/05/12 00:49:40 ERROR : Attempt 3/3 failed with 3 errors and: not deleting files as there were IO errors
15959 2021/05/12 00:49:40 ERROR : Failed to sync: not deleting files as there were IO errors
15960 2021/05/12 00:49:40 NOTICE: WARNING rclone sync try 3 failed. - /path/to/local/tree/
15961 2021/05/12 00:49:40 ERROR : Bisync aborted. Must run --resync to recover.
15962
15963 Denied downloads of "infected" or "abusive" files
15964 Google Drive has a filter for certain file types (.exe, .apk, et
15965 cetera) that by default cannot be copied from Google Drive to the local
15966 filesystem. If you are having problems, run with --verbose to see
15967 specifically which files are generating complaints. If the error is
15968 This file has been identified as malware or spam and cannot be down‐
15969 loaded, consider using the flag --drive-acknowledge-abuse
15970 (https://rclone.org/drive/#drive-acknowledge-abuse).
15971
15972 Google Doc files
15973 Google docs exist as virtual files on Google Drive and cannot be trans‐
15974 ferred to other filesystems natively. While it is possible to export a
15975 Google doc to a normal file (with .xlsx extension, for example), it is
15976 not possible to import a normal file back into a Google document.
15977
15978 Bisync's handling of Google Doc files is to flag them in the run log
15979 output for user's attention and ignore them for any file transfers,
15980 deletes, or syncs. They will show up with a length of -1 in the list‐
15981 ings. This bisync run is otherwise successful:
15982
15983 2021/05/11 08:23:15 INFO : Synching Path1 "/path/to/local/tree/base/" with Path2 "GDrive:"
15984 2021/05/11 08:23:15 INFO : ...path2.lst-new: Ignoring incorrect line: "- -1 - - 2018-07-29T08:49:30.136000000+0000 GoogleDoc.docx"
15985 2021/05/11 08:23:15 INFO : Bisync successful
15986
15987 Usage examples
15988 Cron
15989 Rclone does not yet have a built-in capability to monitor the local
15990 file system for changes and must be blindly run periodically. On Win‐
15991 dows this can be done using a Task Scheduler, on Linux you can use Cron
15992 which is described below.
15993
15994 The 1st example runs a sync every 5 minutes between a local directory
15995 and an OwnCloud server, with output logged to a runlog file:
15996
15997 # Minute (0-59)
15998 # Hour (0-23)
15999 # Day of Month (1-31)
16000 # Month (1-12 or Jan-Dec)
16001 # Day of Week (0-6 or Sun-Sat)
16002 # Command
16003 */5 * * * * /path/to/rclone bisync /local/files MyCloud: --check-access --filters-file /path/to/bysync-filters.txt --log-file /path/to//bisync.log
16004
16005 See crontab syntax (https://www.man7.org/linux/man-
16006 pages/man1/crontab.1p.html#INPUT_FILES)). for the details of crontab
16007 time interval expressions.
16008
16009 If you run rclone bisync as a cron job, redirect stdout/stderr to a
16010 file. The 2nd example runs a sync to Dropbox every hour and logs all
16011 stdout (via the >>) and stderr (via 2>&1) to a log file.
16012
16013 0 * * * * /path/to/rclone bisync /path/to/local/dropbox Dropbox: --check-access --filters-file /home/user/filters.txt >> /path/to/logs/dropbox-run.log 2>&1
16014
16015 Sharing an encrypted folder tree between hosts
16016 bisync can keep a local folder in sync with a cloud service, but what
16017 if you have some highly sensitive files to be synched?
16018
16019 Usage of a cloud service is for exchanging both routine and sensitive
16020 personal files between one's home network, one's personal notebook when
16021 on the road, and with one's work computer. The routine data is not
16022 sensitive. For the sensitive data, configure an rclone crypt remote
16023 (https://rclone.org/crypt/) to point to a subdirectory within the local
16024 disk tree that is bisync'd to Dropbox, and then set up an bisync for
16025 this local crypt directory to a directory outside of the main sync
16026 tree.
16027
16028 Linux server setup
16029 • /path/to/DBoxroot is the root of my local sync tree. There are nu‐
16030 merous subdirectories.
16031
16032 • /path/to/DBoxroot/crypt is the root subdirectory for files that are
16033 encrypted. This local directory target is setup as an rclone crypt
16034 remote named Dropcrypt:. See rclone.conf snippet below.
16035
16036 • /path/to/my/unencrypted/files is the root of my sensitive files - not
16037 encrypted, not within the tree synched to Dropbox.
16038
16039 • To sync my local unencrypted files with the encrypted Dropbox ver‐
16040 sions I manually run bisync /path/to/my/unencrypted/files DropCrypt:.
16041 This step could be bundled into a script to run before and after the
16042 full Dropbox tree sync in the last step, thus actively keeping the
16043 sensitive files in sync.
16044
16045 • bisync /path/to/DBoxroot Dropbox: runs periodically via cron, keeping
16046 my full local sync tree in sync with Dropbox.
16047
16048 Windows notebook setup
16049 • The Dropbox client runs keeping the local tree C:\Users\MyLogin\Drop‐
16050 box always in sync with Dropbox. I could have used rclone bisync in‐
16051 stead.
16052
16053 • A separate directory tree at C:\Users\MyLogin\Documents\DropLocal
16054 hosts the tree of unencrypted files/folders.
16055
16056 • To sync my local unencrypted files with the encrypted Dropbox ver‐
16057 sions I manually run the following command: rclone bisync
16058 C:\Users\MyLogin\Documents\DropLocal Dropcrypt:.
16059
16060 • The Dropbox client then syncs the changes with Dropbox.
16061
16062 rclone.conf snippet
16063 [Dropbox]
16064 type = dropbox
16065 ...
16066
16067 [Dropcrypt]
16068 type = crypt
16069 remote = /path/to/DBoxroot/crypt # on the Linux server
16070 remote = C:\Users\MyLogin\Dropbox\crypt # on the Windows notebook
16071 filename_encryption = standard
16072 directory_name_encryption = true
16073 password = ...
16074 ...
16075
16076 Testing
16077 You should read this section only if you are developing for rclone.
16078 You need to have rclone source code locally to work with bisync tests.
16079
16080 Bisync has a dedicated test framework implemented in the bisync_test.go
16081 file located in the rclone source tree. The test suite is based on the
16082 go test command. Series of tests are stored in subdirectories below
16083 the cmd/bisync/testdata directory. Individual tests can be invoked by
16084 their directory name, e.g. go test . -case basic -remote local -re‐
16085 mote2 gdrive: -v
16086
16087 Tests will make a temporary folder on remote and purge it afterwards.
16088 If during test run there are intermittent errors and rclone retries,
16089 these errors will be captured and flagged as invalid MISCOMPAREs. Re‐
16090 running the test will let it pass. Consider such failures as noise.
16091
16092 Test command syntax
16093 usage: go test ./cmd/bisync [options...]
16094
16095 Options:
16096 -case NAME Name(s) of the test case(s) to run. Multiple names should
16097 be separated by commas. You can remove the `test_` prefix
16098 and replace `_` by `-` in test name for convenience.
16099 If not `all`, the name(s) should map to a directory under
16100 `./cmd/bisync/testdata`.
16101 Use `all` to run all tests (default: all)
16102 -remote PATH1 `local` or name of cloud service with `:` (default: local)
16103 -remote2 PATH2 `local` or name of cloud service with `:` (default: local)
16104 -no-compare Disable comparing test results with the golden directory
16105 (default: compare)
16106 -no-cleanup Disable cleanup of Path1 and Path2 testdirs.
16107 Useful for troubleshooting. (default: cleanup)
16108 -golden Store results in the golden directory (default: false)
16109 This flag can be used with multiple tests.
16110 -debug Print debug messages
16111 -stop-at NUM Stop test after given step number. (default: run to the end)
16112 Implies `-no-compare` and `-no-cleanup`, if the test really
16113 ends prematurely. Only meaningful for a single test case.
16114 -refresh-times Force refreshing the target modtime, useful for Dropbox
16115 (default: false)
16116 -verbose Run tests verbosely
16117
16118 Note: unlike rclone flags which must be prefixed by double dash (--),
16119 the test command flags can be equally prefixed by a single - or double
16120 dash.
16121
16122 Running tests
16123 • go test . -case basic -remote local -remote2 local runs the test_ba‐
16124 sic test case using only the local filesystem, synching one local di‐
16125 rectory with another local directory. Test script output is to the
16126 console, while commands within scenario.txt have their output sent to
16127 the .../workdir/test.log file, which is finally compared to the gold‐
16128 en copy.
16129
16130 • The first argument after go test should be a relative name of the di‐
16131 rectory containing bisync source code. If you run tests right from
16132 there, the argument will be . (current directory) as in most exam‐
16133 ples below. If you run bisync tests from the rclone source directo‐
16134 ry, the command should be go test ./cmd/bisync ....
16135
16136 • The test engine will mangle rclone output to ensure comparability
16137 with golden listings and logs.
16138
16139 • Test scenarios are located in ./cmd/bisync/testdata. The test -case
16140 argument should match the full name of a subdirectory under that di‐
16141 rectory. Every test subdirectory name on disk must start with test_,
16142 this prefix can be omitted on command line for brevity. Also, under‐
16143 scores in the name can be replaced by dashes for convenience.
16144
16145 • go test . -remote local -remote2 local -case all runs all tests.
16146
16147 • Path1 and Path2 may either be the keyword local or may be names of
16148 configured cloud services. go test . -remote gdrive: -remote2 drop‐
16149 box: -case basic will run the test between these two services, with‐
16150 out transferring any files to the local filesystem.
16151
16152 • Test run stdout and stderr console output may be directed to a file,
16153 e.g. go test . -remote gdrive: -remote2 local -case all > runlog.txt
16154 2>&1
16155
16156 Test execution flow
16157 1. The base setup in the initial directory of the testcase is applied
16158 on the Path1 and Path2 filesystems (via rclone copy the initial di‐
16159 rectory to Path1, then rclone sync Path1 to Path2).
16160
16161 2. The commands in the scenario.txt file are applied, with output di‐
16162 rected to the test.log file in the test working directory. Typical‐
16163 ly, the first actual command in the scenario.txt file is to do a
16164 --resync, which establishes the baseline {...}.path1.lst and
16165 {...}.path2.lst files in the test working directory (.../workdir/
16166 relative to the temporary test directory). Various commands and
16167 listing snapshots are done within the test.
16168
16169 3. Finally, the contents of the test working directory are compared to
16170 the contents of the testcase's golden directory.
16171
16172 Notes about testing
16173 • Test cases are in individual directories beneath ./cmd/bisync/testda‐
16174 ta. A command line reference to a test is understood to reference a
16175 directory beneath testdata. For example, go test ./cmd/bisync -case
16176 dry-run -remote gdrive: -remote2 local refers to the test case in
16177 ./cmd/bisync/testdata/test_dry_run.
16178
16179 • The test working directory is located at .../workdir relative to a
16180 temporary test directory, usually under /tmp on Linux.
16181
16182 • The local test sync tree is created at a temporary directory named
16183 like bisync.XXX under system temporary directory.
16184
16185 • The remote test sync tree is located at a temporary directory under
16186 <remote:>/bisync.XXX/.
16187
16188 • path1 and/or path2 subdirectories are created in a temporary directo‐
16189 ry under the respective local or cloud test remote.
16190
16191 • By default, the Path1 and Path2 test dirs and workdir will be deleted
16192 after each test run. The -no-cleanup flag disables purging these di‐
16193 rectories when validating and debugging a given test. These directo‐
16194 ries will be flushed before running another test, independent of the
16195 -no-cleanup usage.
16196
16197 • You will likely want to add `- /testdir/to your normal bisync--fil‐
16198 ters-fileso that normal syncs do not attempt to sync the test tem‐
16199 porary directories, which may haveRCLONE_TESTmiscompares in some
16200 testcases which would otherwise trip the--check-accesssystem.
16201 The--check-accessmechanism is hard-coded to ignoreRCLONE_TESTfiles
16202 beneathbisync/testdata`, so the test cases may reside on the synched
16203 tree even if there are check file mismatches in the test tree.
16204
16205 • Some Dropbox tests can fail, notably printing the following message:
16206 src and dst identical but can't set mod time without deleting and re-
16207 uploading This is expected and happens due a way Dropbox handles mod‐
16208 ification times. You should use the -refresh-times test flag to make
16209 up for this.
16210
16211 • If Dropbox tests hit request limit for you and print error message
16212 too_many_requests/...: Too many requests or write operations. then
16213 follow the Dropbox App ID instructions (https://rclone.org/drop‐
16214 box/#get-your-own-dropbox-app-id).
16215
16216 Updating golden results
16217 Sometimes even a slight change in the bisync source can cause little
16218 changes spread around many log files. Updating them manually would be
16219 a nightmare.
16220
16221 The -golden flag will store the test.log and *.lst listings from each
16222 test case into respective golden directories. Golden results will au‐
16223 tomatically contain generic strings instead of local or cloud paths
16224 which means that they should match when run with a different cloud ser‐
16225 vice.
16226
16227 Your normal workflow might be as follows: 1. Git-clone the rclone
16228 sources locally 2. Modify bisync source and check that it builds 3.
16229 Run the whole test suite go test ./cmd/bisync -remote local 4. If some
16230 tests show log difference, recheck them individually, e.g.: go test
16231 ./cmd/bisync -remote local -case basic 5. If you are convinced with
16232 the difference, goldenize all tests at once: go test ./cmd/bisync -re‐
16233 mote local -golden 6. Use word diff: git diff --word-diff
16234 ./cmd/bisync/testdata/. Please note that normal line-level diff is
16235 generally useless here. 7. Check the difference carefully! 8. Commit
16236 the change (git commit) only if you are sure. If unsure, save your
16237 code changes then wipe the log diffs from git: git reset [--hard].
16238
16239 Structure of test scenarios
16240 • <testname>/initial/ contains a tree of files that will be set as the
16241 initial condition on both Path1 and Path2 testdirs.
16242
16243 • <testname>/modfiles/ contains files that will be used to modify the
16244 Path1 and/or Path2 filesystems.
16245
16246 • <testname>/golden/ contains the expected content of the test working
16247 directory (workdir) at the completion of the testcase.
16248
16249 • <testname>/scenario.txt contains the body of the test, in the form of
16250 various commands to modify files, run bisync, and snapshot listings.
16251 Output from these commands is captured to .../workdir/test.log for
16252 comparison to the golden files.
16253
16254 Supported test commands
16255 • test <some message> Print the line to the console and to the
16256 test.log: test sync is working correctly with options x, y, z
16257
16258 • copy-listings <prefix> Save a copy of all .lst listings in the test
16259 working directory with the specified prefix: save-listings exclude-
16260 pass-run
16261
16262 • move-listings <prefix> Similar to copy-listings but removes the
16263 source
16264
16265 • purge-children <dir> This will delete all child files and purge all
16266 child subdirs under given directory but keep the parent intact. This
16267 behavior is important for tests with Google Drive because removing
16268 and re-creating the parent would change its ID.
16269
16270 • delete-file <file> Delete a single file.
16271
16272 • delete-glob <dir> <pattern> Delete a group of files located one level
16273 deep in the given directory with names maching a given glob pattern.
16274
16275 • touch-glob YYYY-MM-DD <dir> <pattern> Change modification time on a
16276 group of files.
16277
16278 • touch-copy YYYY-MM-DD <source-file> <dest-dir> Change file modifica‐
16279 tion time then copy it to destination.
16280
16281 • copy-file <source-file> <dest-dir> Copy a single file to given direc‐
16282 tory.
16283
16284 • copy-as <source-file> <dest-file> Similar to above but destination
16285 must include both directory and the new file name at destination.
16286
16287 • copy-dir <src> <dst> and sync-dir <src> <dst> Copy/sync a directory.
16288 Equivalent of rclone copy and rclone sync.
16289
16290 • list-dirs <dir> Equivalent to rclone lsf -R --dirs-only <dir>
16291
16292 • bisync [options] Runs bisync against -remote and -remote2.
16293
16294 Supported substitution terms
16295 • {testdir/} - the root dir of the testcase
16296
16297 • {datadir/} - the modfiles dir under the testcase root
16298
16299 • {workdir/} - the temporary test working directory
16300
16301 • {path1/} - the root of the Path1 test directory tree
16302
16303 • {path2/} - the root of the Path2 test directory tree
16304
16305 • {session} - base name of the test listings
16306
16307 • {/} - OS-specific path separator
16308
16309 • {spc}, {tab}, {eol} - whitespace
16310
16311 • {chr:HH} - raw byte with given hexadecimal code
16312
16313 Substitution results of the terms named like {dir/} will end with / (or
16314 backslash on Windows), so it is not necessary to include slash in the
16315 usage, for example delete-file {path1/}file1.txt.
16316
16317 Benchmarks
16318 This section is work in progress.
16319
16320 Here are a few data points for scale, execution times, and memory us‐
16321 age.
16322
16323 The first set of data was taken between a local disk to Dropbox. The
16324 speedtest.net (https://speedtest.net) download speed was ~170 Mbps, and
16325 upload speed was ~10 Mbps. 500 files (~9.5 MB each) had been already
16326 synched. 50 files were added in a new directory, each ~9.5 MB, ~475 MB
16327 total.
16328
16329 Change Operations and times Overall run
16330 time
16331 ─────────────────────────────────────────────────────────────────────────────
16332 500 files synched (noth‐ 1x listings for Path1 & Path2 1.5 sec
16333 ing to move)
16334 500 files synched with 1x listings for Path1 & Path2 1.5 sec
16335 --check-access
16336 50 new files on remote Queued 50 copies down: 27 sec 29 sec
16337 Moved local dir Queued 50 copies up: 410 sec, 50 421 sec
16338 deletes up: 9 sec
16339 Moved remote dir Queued 50 copies down: 31 sec, 50 33 sec
16340 deletes down: <1 sec
16341 Delete local dir Queued 50 deletes up: 9 sec 13 sec
16342
16343 This next data is from a user's application. They had ~400GB of data
16344 over 1.96 million files being sync'ed between a Windows local disk and
16345 some remote cloud. The file full path length was on average 35 charac‐
16346 ters (which factors into load time and RAM required).
16347
16348 • Loading the prior listing into memory (1.96 million files, listing
16349 file size 140 MB) took ~30 sec and occupied about 1 GB of RAM.
16350
16351 • Getting a fresh listing of the local file system (producing the 140
16352 MB output file) took about XXX sec.
16353
16354 • Getting a fresh listing of the remote file system (producing the 140
16355 MB output file) took about XXX sec. The network download speed was
16356 measured at XXX Mb/s.
16357
16358 • Once the prior and current Path1 and Path2 listings were loaded (a
16359 total of four to be loaded, two at a time), determining the deltas
16360 was pretty quick (a few seconds for this test case), and the transfer
16361 time for any files to be copied was dominated by the network band‐
16362 width.
16363
16364 References
16365 rclone's bisync implementation was derived from the rclonesync-V2
16366 (https://github.com/cjnaz/rclonesync-V2) project, including documenta‐
16367 tion and test mechanisms, with [@cjnaz](https://github.com/cjnaz)'s
16368 full support and encouragement.
16369
16370 rclone bisync is similar in nature to a range of other projects:
16371
16372 • unison (https://github.com/bcpierce00/unison)
16373
16374 • syncthing (https://github.com/syncthing/syncthing)
16375
16376 • cjnaz/rclonesync (https://github.com/cjnaz/rclonesync-V2)
16377
16378 • ConorWilliams/rsinc (https://github.com/ConorWilliams/rsinc)
16379
16380 • jwink3101/syncrclone (https://github.com/Jwink3101/syncrclone)
16381
16382 • DavideRossi/upback (https://github.com/DavideRossi/upback)
16383
16384 Bisync adopts the differential synchronization technique, which is
16385 based on keeping history of changes performed by both synchronizing
16386 sides. See the Dual Shadow Method section in the Neil Fraser's article
16387 (https://neil.fraser.name/writing/sync/).
16388
16389 Also note a number of academic publications by Benjamin Pierce
16390 (http://www.cis.upenn.edu/%7Ebcpierce/papers/index.shtml#File%20Syn‐
16391 chronization) about Unison and synchronization in general.
16392
16394 This is a backend for the 1fichier (https://1fichier.com) cloud storage
16395 service. Note that a Premium subscription is required to use the API.
16396
16397 Paths are specified as remote:path
16398
16399 Paths may be as deep as required, e.g. remote:directory/subdirectory.
16400
16401 Configuration
16402 The initial setup for 1Fichier involves getting the API key from the
16403 website which you need to do in your browser.
16404
16405 Here is an example of how to make a remote called remote. First run:
16406
16407 rclone config
16408
16409 This will guide you through an interactive setup process:
16410
16411 No remotes found, make a new one?
16412 n) New remote
16413 s) Set configuration password
16414 q) Quit config
16415 n/s/q> n
16416 name> remote
16417 Type of storage to configure.
16418 Enter a string value. Press Enter for the default ("").
16419 Choose a number from below, or type in your own value
16420 [snip]
16421 XX / 1Fichier
16422 \ "fichier"
16423 [snip]
16424 Storage> fichier
16425 ** See help for fichier backend at: https://rclone.org/fichier/ **
16426
16427 Your API Key, get it from https://1fichier.com/console/params.pl
16428 Enter a string value. Press Enter for the default ("").
16429 api_key> example_key
16430
16431 Edit advanced config? (y/n)
16432 y) Yes
16433 n) No
16434 y/n>
16435 Remote config
16436 --------------------
16437 [remote]
16438 type = fichier
16439 api_key = example_key
16440 --------------------
16441 y) Yes this is OK
16442 e) Edit this remote
16443 d) Delete this remote
16444 y/e/d> y
16445
16446 Once configured you can then use rclone like this,
16447
16448 List directories in top level of your 1Fichier account
16449
16450 rclone lsd remote:
16451
16452 List all the files in your 1Fichier account
16453
16454 rclone ls remote:
16455
16456 To copy a local directory to a 1Fichier directory called backup
16457
16458 rclone copy /home/source remote:backup
16459
16460 Modified time and hashes
16461 1Fichier does not support modification times. It supports the
16462 Whirlpool hash algorithm.
16463
16464 Duplicated files
16465 1Fichier can have two files with exactly the same name and path (unlike
16466 a normal file system).
16467
16468 Duplicated files cause problems with the syncing and you will see mes‐
16469 sages in the log about duplicates.
16470
16471 Restricted filename characters
16472 In addition to the default restricted characters set
16473 (https://rclone.org/overview/#restricted-characters) the following
16474 characters are also replaced:
16475
16476 Character Value Replacement
16477 ────────────────────────────────
16478 \ 0x5C \
16479 < 0x3C <
16480 > 0x3E >
16481 " 0x22 "
16482 $ 0x24 $
16483 ` 0x60 `
16484 ' 0x27 '
16485
16486 File names can also not start or end with the following characters.
16487 These only get replaced if they are the first or last character in the
16488 name:
16489
16490 Character Value Replacement
16491 ────────────────────────────────
16492 SP 0x20 ␠
16493
16494 Invalid UTF-8 bytes will also be replaced (https://rclone.org/over‐
16495 view/#invalid-utf8), as they can't be used in JSON strings.
16496
16497 Standard options
16498 Here are the Standard options specific to fichier (1Fichier).
16499
16500 --fichier-api-key
16501 Your API Key, get it from https://1fichier.com/console/params.pl.
16502
16503 Properties:
16504
16505 • Config: api_key
16506
16507 • Env Var: RCLONE_FICHIER_API_KEY
16508
16509 • Type: string
16510
16511 • Required: false
16512
16513 Advanced options
16514 Here are the Advanced options specific to fichier (1Fichier).
16515
16516 --fichier-shared-folder
16517 If you want to download a shared folder, add this parameter.
16518
16519 Properties:
16520
16521 • Config: shared_folder
16522
16523 • Env Var: RCLONE_FICHIER_SHARED_FOLDER
16524
16525 • Type: string
16526
16527 • Required: false
16528
16529 --fichier-file-password
16530 If you want to download a shared file that is password protected, add
16531 this parameter.
16532
16533 NB Input to this must be obscured - see rclone obscure
16534 (https://rclone.org/commands/rclone_obscure/).
16535
16536 Properties:
16537
16538 • Config: file_password
16539
16540 • Env Var: RCLONE_FICHIER_FILE_PASSWORD
16541
16542 • Type: string
16543
16544 • Required: false
16545
16546 --fichier-folder-password
16547 If you want to list the files in a shared folder that is password pro‐
16548 tected, add this parameter.
16549
16550 NB Input to this must be obscured - see rclone obscure
16551 (https://rclone.org/commands/rclone_obscure/).
16552
16553 Properties:
16554
16555 • Config: folder_password
16556
16557 • Env Var: RCLONE_FICHIER_FOLDER_PASSWORD
16558
16559 • Type: string
16560
16561 • Required: false
16562
16563 --fichier-encoding
16564 The encoding for the backend.
16565
16566 See the encoding section in the overview (https://rclone.org/over‐
16567 view/#encoding) for more info.
16568
16569 Properties:
16570
16571 • Config: encoding
16572
16573 • Env Var: RCLONE_FICHIER_ENCODING
16574
16575 • Type: MultiEncoder
16576
16577 • Default: Slash,LtGt,DoubleQuote,SingleQuote,BackQuote,Dollar,Back‐
16578 Slash,Del,Ctl,LeftSpace,RightSpace,InvalidUtf8,Dot
16579
16580 Limitations
16581 rclone about is not supported by the 1Fichier backend. Backends with‐
16582 out this capability cannot determine free space for an rclone mount or
16583 use policy mfs (most free space) as a member of an rclone union remote.
16584
16585 See List of backends that do not support rclone about
16586 (https://rclone.org/overview/#optional-features) and rclone about
16587 (https://rclone.org/commands/rclone_about/)
16588
16590 The alias remote provides a new name for another remote.
16591
16592 Paths may be as deep as required or a local path, e.g. remote:directo‐
16593 ry/subdirectory or /directory/subdirectory.
16594
16595 During the initial setup with rclone config you will specify the target
16596 remote. The target remote can either be a local path or another re‐
16597 mote.
16598
16599 Subfolders can be used in target remote. Assume an alias remote named
16600 backup with the target mydrive:private/backup. Invoking rclone mkdir
16601 backup:desktop is exactly the same as invoking rclone mkdir my‐
16602 drive:private/backup/desktop.
16603
16604 There will be no special handling of paths containing .. segments.
16605 Invoking rclone mkdir backup:../desktop is exactly the same as invoking
16606 rclone mkdir mydrive:private/backup/../desktop. The empty path is not
16607 allowed as a remote. To alias the current directory use . instead.
16608
16609 The target remote can also be a connection string
16610 (https://rclone.org/docs/#connection-strings). This can be used to
16611 modify the config of a remote for different uses, e.g. the alias my‐
16612 DriveTrash with the target remote myDrive,trashed_only: can be used to
16613 only show the trashed files in myDrive.
16614
16615 Configuration
16616 Here is an example of how to make an alias called remote for local
16617 folder. First run:
16618
16619 rclone config
16620
16621 This will guide you through an interactive setup process:
16622
16623 No remotes found, make a new one?
16624 n) New remote
16625 s) Set configuration password
16626 q) Quit config
16627 n/s/q> n
16628 name> remote
16629 Type of storage to configure.
16630 Choose a number from below, or type in your own value
16631 [snip]
16632 XX / Alias for an existing remote
16633 \ "alias"
16634 [snip]
16635 Storage> alias
16636 Remote or path to alias.
16637 Can be "myremote:path/to/dir", "myremote:bucket", "myremote:" or "/local/path".
16638 remote> /mnt/storage/backup
16639 Remote config
16640 --------------------
16641 [remote]
16642 remote = /mnt/storage/backup
16643 --------------------
16644 y) Yes this is OK
16645 e) Edit this remote
16646 d) Delete this remote
16647 y/e/d> y
16648 Current remotes:
16649
16650 Name Type
16651 ==== ====
16652 remote alias
16653
16654 e) Edit existing remote
16655 n) New remote
16656 d) Delete remote
16657 r) Rename remote
16658 c) Copy remote
16659 s) Set configuration password
16660 q) Quit config
16661 e/n/d/r/c/s/q> q
16662
16663 Once configured you can then use rclone like this,
16664
16665 List directories in top level in /mnt/storage/backup
16666
16667 rclone lsd remote:
16668
16669 List all the files in /mnt/storage/backup
16670
16671 rclone ls remote:
16672
16673 Copy another local directory to the alias directory called source
16674
16675 rclone copy /home/source remote:source
16676
16677 Standard options
16678 Here are the Standard options specific to alias (Alias for an existing
16679 remote).
16680
16681 --alias-remote
16682 Remote or path to alias.
16683
16684 Can be "myremote:path/to/dir", "myremote:bucket", "myremote:" or "/lo‐
16685 cal/path".
16686
16687 Properties:
16688
16689 • Config: remote
16690
16691 • Env Var: RCLONE_ALIAS_REMOTE
16692
16693 • Type: string
16694
16695 • Required: true
16696
16698 Amazon Drive, formerly known as Amazon Cloud Drive, is a cloud storage
16699 service run by Amazon for consumers.
16700
16701 Status
16702 Important: rclone supports Amazon Drive only if you have your own set
16703 of API keys. Unfortunately the Amazon Drive developer program
16704 (https://developer.amazon.com/amazon-drive) is now closed to new en‐
16705 tries so if you don't already have your own set of keys you will not be
16706 able to use rclone with Amazon Drive.
16707
16708 For the history on why rclone no longer has a set of Amazon Drive API
16709 keys see the forum (https://forum.rclone.org/t/rclone-has-been-banned-
16710 from-amazon-drive/2314).
16711
16712 If you happen to know anyone who works at Amazon then please ask them
16713 to re-instate rclone into the Amazon Drive developer program - thanks!
16714
16715 Configuration
16716 The initial setup for Amazon Drive involves getting a token from Amazon
16717 which you need to do in your browser. rclone config walks you through
16718 it.
16719
16720 The configuration process for Amazon Drive may involve using an oauth
16721 proxy (https://github.com/ncw/oauthproxy). This is used to keep the
16722 Amazon credentials out of the source code. The proxy runs in Google's
16723 very secure App Engine environment and doesn't store any credentials
16724 which pass through it.
16725
16726 Since rclone doesn't currently have its own Amazon Drive credentials so
16727 you will either need to have your own client_id and client_secret with
16728 Amazon Drive, or use a third-party oauth proxy in which case you will
16729 need to enter client_id, client_secret, auth_url and token_url.
16730
16731 Note also if you are not using Amazon's auth_url and token_url, (ie you
16732 filled in something for those) then if setting up on a remote machine
16733 you can only use the copying the config method of configuration
16734 (https://rclone.org/remote_setup/#configuring-by-copying-the-config-
16735 file) - rclone authorize will not work.
16736
16737 Here is an example of how to make a remote called remote. First run:
16738
16739 rclone config
16740
16741 This will guide you through an interactive setup process:
16742
16743 No remotes found, make a new one?
16744 n) New remote
16745 r) Rename remote
16746 c) Copy remote
16747 s) Set configuration password
16748 q) Quit config
16749 n/r/c/s/q> n
16750 name> remote
16751 Type of storage to configure.
16752 Choose a number from below, or type in your own value
16753 [snip]
16754 XX / Amazon Drive
16755 \ "amazon cloud drive"
16756 [snip]
16757 Storage> amazon cloud drive
16758 Amazon Application Client Id - required.
16759 client_id> your client ID goes here
16760 Amazon Application Client Secret - required.
16761 client_secret> your client secret goes here
16762 Auth server URL - leave blank to use Amazon's.
16763 auth_url> Optional auth URL
16764 Token server url - leave blank to use Amazon's.
16765 token_url> Optional token URL
16766 Remote config
16767 Make sure your Redirect URL is set to "http://127.0.0.1:53682/" in your custom config.
16768 Use auto config?
16769 * Say Y if not sure
16770 * Say N if you are working on a remote or headless machine
16771 y) Yes
16772 n) No
16773 y/n> y
16774 If your browser doesn't open automatically go to the following link: http://127.0.0.1:53682/auth
16775 Log in and authorize rclone for access
16776 Waiting for code...
16777 Got code
16778 --------------------
16779 [remote]
16780 client_id = your client ID goes here
16781 client_secret = your client secret goes here
16782 auth_url = Optional auth URL
16783 token_url = Optional token URL
16784 token = {"access_token":"xxxxxxxxxxxxxxxxxxxxxxx","token_type":"bearer","refresh_token":"xxxxxxxxxxxxxxxxxx","expiry":"2015-09-06T16:07:39.658438471+01:00"}
16785 --------------------
16786 y) Yes this is OK
16787 e) Edit this remote
16788 d) Delete this remote
16789 y/e/d> y
16790
16791 See the remote setup docs (https://rclone.org/remote_setup/) for how to
16792 set it up on a machine with no Internet browser available.
16793
16794 Note that rclone runs a webserver on your local machine to collect the
16795 token as returned from Amazon. This only runs from the moment it opens
16796 your browser to the moment you get back the verification code. This is
16797 on http://127.0.0.1:53682/ and this it may require you to unblock it
16798 temporarily if you are running a host firewall.
16799
16800 Once configured you can then use rclone like this,
16801
16802 List directories in top level of your Amazon Drive
16803
16804 rclone lsd remote:
16805
16806 List all the files in your Amazon Drive
16807
16808 rclone ls remote:
16809
16810 To copy a local directory to an Amazon Drive directory called backup
16811
16812 rclone copy /home/source remote:backup
16813
16814 Modified time and MD5SUMs
16815 Amazon Drive doesn't allow modification times to be changed via the API
16816 so these won't be accurate or used for syncing.
16817
16818 It does store MD5SUMs so for a more accurate sync, you can use the
16819 --checksum flag.
16820
16821 Restricted filename characters
16822 Character Value Replacement
16823 ────────────────────────────────
16824 NUL 0x00 ␀
16825 / 0x2F /
16826
16827 Invalid UTF-8 bytes will also be replaced (https://rclone.org/over‐
16828 view/#invalid-utf8), as they can't be used in JSON strings.
16829
16830 Deleting files
16831 Any files you delete with rclone will end up in the trash. Amazon
16832 don't provide an API to permanently delete files, nor to empty the
16833 trash, so you will have to do that with one of Amazon's apps or via the
16834 Amazon Drive website. As of November 17, 2016, files are automatically
16835 deleted by Amazon from the trash after 30 days.
16836
16837 Using with non .com Amazon accounts
16838 Let's say you usually use amazon.co.uk. When you authenticate with
16839 rclone it will take you to an amazon.com page to log in. Your ama‐
16840 zon.co.uk email and password should work here just fine.
16841
16842 Standard options
16843 Here are the Standard options specific to amazon cloud drive (Amazon
16844 Drive).
16845
16846 --acd-client-id
16847 OAuth Client Id.
16848
16849 Leave blank normally.
16850
16851 Properties:
16852
16853 • Config: client_id
16854
16855 • Env Var: RCLONE_ACD_CLIENT_ID
16856
16857 • Type: string
16858
16859 • Required: false
16860
16861 --acd-client-secret
16862 OAuth Client Secret.
16863
16864 Leave blank normally.
16865
16866 Properties:
16867
16868 • Config: client_secret
16869
16870 • Env Var: RCLONE_ACD_CLIENT_SECRET
16871
16872 • Type: string
16873
16874 • Required: false
16875
16876 Advanced options
16877 Here are the Advanced options specific to amazon cloud drive (Amazon
16878 Drive).
16879
16880 --acd-token
16881 OAuth Access Token as a JSON blob.
16882
16883 Properties:
16884
16885 • Config: token
16886
16887 • Env Var: RCLONE_ACD_TOKEN
16888
16889 • Type: string
16890
16891 • Required: false
16892
16893 --acd-auth-url
16894 Auth server URL.
16895
16896 Leave blank to use the provider defaults.
16897
16898 Properties:
16899
16900 • Config: auth_url
16901
16902 • Env Var: RCLONE_ACD_AUTH_URL
16903
16904 • Type: string
16905
16906 • Required: false
16907
16908 --acd-token-url
16909 Token server url.
16910
16911 Leave blank to use the provider defaults.
16912
16913 Properties:
16914
16915 • Config: token_url
16916
16917 • Env Var: RCLONE_ACD_TOKEN_URL
16918
16919 • Type: string
16920
16921 • Required: false
16922
16923 --acd-checkpoint
16924 Checkpoint for internal polling (debug).
16925
16926 Properties:
16927
16928 • Config: checkpoint
16929
16930 • Env Var: RCLONE_ACD_CHECKPOINT
16931
16932 • Type: string
16933
16934 • Required: false
16935
16936 --acd-upload-wait-per-gb
16937 Additional time per GiB to wait after a failed complete upload to see
16938 if it appears.
16939
16940 Sometimes Amazon Drive gives an error when a file has been fully up‐
16941 loaded but the file appears anyway after a little while. This happens
16942 sometimes for files over 1 GiB in size and nearly every time for files
16943 bigger than 10 GiB. This parameter controls the time rclone waits for
16944 the file to appear.
16945
16946 The default value for this parameter is 3 minutes per GiB, so by de‐
16947 fault it will wait 3 minutes for every GiB uploaded to see if the file
16948 appears.
16949
16950 You can disable this feature by setting it to 0. This may cause con‐
16951 flict errors as rclone retries the failed upload but the file will most
16952 likely appear correctly eventually.
16953
16954 These values were determined empirically by observing lots of uploads
16955 of big files for a range of file sizes.
16956
16957 Upload with the "-v" flag to see more info about what rclone is doing
16958 in this situation.
16959
16960 Properties:
16961
16962 • Config: upload_wait_per_gb
16963
16964 • Env Var: RCLONE_ACD_UPLOAD_WAIT_PER_GB
16965
16966 • Type: Duration
16967
16968 • Default: 3m0s
16969
16970 --acd-templink-threshold
16971 Files >= this size will be downloaded via their tempLink.
16972
16973 Files this size or more will be downloaded via their "tempLink". This
16974 is to work around a problem with Amazon Drive which blocks downloads of
16975 files bigger than about 10 GiB. The default for this is 9 GiB which
16976 shouldn't need to be changed.
16977
16978 To download files above this threshold, rclone requests a "tempLink"
16979 which downloads the file through a temporary URL directly from the un‐
16980 derlying S3 storage.
16981
16982 Properties:
16983
16984 • Config: templink_threshold
16985
16986 • Env Var: RCLONE_ACD_TEMPLINK_THRESHOLD
16987
16988 • Type: SizeSuffix
16989
16990 • Default: 9Gi
16991
16992 --acd-encoding
16993 The encoding for the backend.
16994
16995 See the encoding section in the overview (https://rclone.org/over‐
16996 view/#encoding) for more info.
16997
16998 Properties:
16999
17000 • Config: encoding
17001
17002 • Env Var: RCLONE_ACD_ENCODING
17003
17004 • Type: MultiEncoder
17005
17006 • Default: Slash,InvalidUtf8,Dot
17007
17008 Limitations
17009 Note that Amazon Drive is case insensitive so you can't have a file
17010 called "Hello.doc" and one called "hello.doc".
17011
17012 Amazon Drive has rate limiting so you may notice errors in the sync
17013 (429 errors). rclone will automatically retry the sync up to 3 times
17014 by default (see --retries flag) which should hopefully work around this
17015 problem.
17016
17017 Amazon Drive has an internal limit of file sizes that can be uploaded
17018 to the service. This limit is not officially published, but all files
17019 larger than this will fail.
17020
17021 At the time of writing (Jan 2016) is in the area of 50 GiB per file.
17022 This means that larger files are likely to fail.
17023
17024 Unfortunately there is no way for rclone to see that this failure is
17025 because of file size, so it will retry the operation, as any other
17026 failure. To avoid this problem, use --max-size 50000M option to limit
17027 the maximum size of uploaded files. Note that --max-size does not
17028 split files into segments, it only ignores files over this size.
17029
17030 rclone about is not supported by the Amazon Drive backend. Backends
17031 without this capability cannot determine free space for an rclone mount
17032 or use policy mfs (most free space) as a member of an rclone union re‐
17033 mote.
17034
17035 See List of backends that do not support rclone about
17036 (https://rclone.org/overview/#optional-features) and rclone about
17037 (https://rclone.org/commands/rclone_about/)
17038
17040 The S3 backend can be used with a number of different providers:
17041
17042 • AWS S3
17043
17044 • Alibaba Cloud (Aliyun) Object Storage System (OSS)
17045
17046 • Ceph
17047
17048 • China Mobile Ecloud Elastic Object Storage (EOS)
17049
17050 • Cloudflare R2
17051
17052 • Arvan Cloud Object Storage (AOS)
17053
17054 • DigitalOcean Spaces
17055
17056 • Dreamhost
17057
17058 • Huawei OBS
17059
17060 • IBM COS S3
17061
17062 • IDrive e2
17063
17064 • IONOS Cloud
17065
17066 • Minio
17067
17068 • Qiniu Cloud Object Storage (Kodo)
17069
17070 • RackCorp Object Storage
17071
17072 • Scaleway
17073
17074 • Seagate Lyve Cloud
17075
17076 • SeaweedFS
17077
17078 • StackPath
17079
17080 • Storj
17081
17082 • Tencent Cloud Object Storage (COS)
17083
17084 • Wasabi
17085
17086 Paths are specified as remote:bucket (or remote: for the lsd command.)
17087 You may put subdirectories in too, e.g. remote:bucket/path/to/dir.
17088
17089 Once you have made a remote (see the provider specific section above)
17090 you can use it like this:
17091
17092 See all buckets
17093
17094 rclone lsd remote:
17095
17096 Make a new bucket
17097
17098 rclone mkdir remote:bucket
17099
17100 List the contents of a bucket
17101
17102 rclone ls remote:bucket
17103
17104 Sync /home/local/directory to the remote bucket, deleting any excess
17105 files in the bucket.
17106
17107 rclone sync -i /home/local/directory remote:bucket
17108
17109 Configuration
17110 Here is an example of making an s3 configuration for the AWS S3
17111 provider. Most applies to the other providers as well, any differences
17112 are described below.
17113
17114 First run
17115
17116 rclone config
17117
17118 This will guide you through an interactive setup process.
17119
17120 No remotes found, make a new one?
17121 n) New remote
17122 s) Set configuration password
17123 q) Quit config
17124 n/s/q> n
17125 name> remote
17126 Type of storage to configure.
17127 Choose a number from below, or type in your own value
17128 [snip]
17129 XX / Amazon S3 Compliant Storage Providers including AWS, Ceph, ChinaMobile, ArvanCloud, Dreamhost, IBM COS, Minio, and Tencent COS
17130 \ "s3"
17131 [snip]
17132 Storage> s3
17133 Choose your S3 provider.
17134 Choose a number from below, or type in your own value
17135 1 / Amazon Web Services (AWS) S3
17136 \ "AWS"
17137 2 / Ceph Object Storage
17138 \ "Ceph"
17139 3 / Digital Ocean Spaces
17140 \ "DigitalOcean"
17141 4 / Dreamhost DreamObjects
17142 \ "Dreamhost"
17143 5 / IBM COS S3
17144 \ "IBMCOS"
17145 6 / Minio Object Storage
17146 \ "Minio"
17147 7 / Wasabi Object Storage
17148 \ "Wasabi"
17149 8 / Any other S3 compatible provider
17150 \ "Other"
17151 provider> 1
17152 Get AWS credentials from runtime (environment variables or EC2/ECS meta data if no env vars). Only applies if access_key_id and secret_access_key is blank.
17153 Choose a number from below, or type in your own value
17154 1 / Enter AWS credentials in the next step
17155 \ "false"
17156 2 / Get AWS credentials from the environment (env vars or IAM)
17157 \ "true"
17158 env_auth> 1
17159 AWS Access Key ID - leave blank for anonymous access or runtime credentials.
17160 access_key_id> XXX
17161 AWS Secret Access Key (password) - leave blank for anonymous access or runtime credentials.
17162 secret_access_key> YYY
17163 Region to connect to.
17164 Choose a number from below, or type in your own value
17165 / The default endpoint - a good choice if you are unsure.
17166 1 | US Region, Northern Virginia, or Pacific Northwest.
17167 | Leave location constraint empty.
17168 \ "us-east-1"
17169 / US East (Ohio) Region
17170 2 | Needs location constraint us-east-2.
17171 \ "us-east-2"
17172 / US West (Oregon) Region
17173 3 | Needs location constraint us-west-2.
17174 \ "us-west-2"
17175 / US West (Northern California) Region
17176 4 | Needs location constraint us-west-1.
17177 \ "us-west-1"
17178 / Canada (Central) Region
17179 5 | Needs location constraint ca-central-1.
17180 \ "ca-central-1"
17181 / EU (Ireland) Region
17182 6 | Needs location constraint EU or eu-west-1.
17183 \ "eu-west-1"
17184 / EU (London) Region
17185 7 | Needs location constraint eu-west-2.
17186 \ "eu-west-2"
17187 / EU (Frankfurt) Region
17188 8 | Needs location constraint eu-central-1.
17189 \ "eu-central-1"
17190 / Asia Pacific (Singapore) Region
17191 9 | Needs location constraint ap-southeast-1.
17192 \ "ap-southeast-1"
17193 / Asia Pacific (Sydney) Region
17194 10 | Needs location constraint ap-southeast-2.
17195 \ "ap-southeast-2"
17196 / Asia Pacific (Tokyo) Region
17197 11 | Needs location constraint ap-northeast-1.
17198 \ "ap-northeast-1"
17199 / Asia Pacific (Seoul)
17200 12 | Needs location constraint ap-northeast-2.
17201 \ "ap-northeast-2"
17202 / Asia Pacific (Mumbai)
17203 13 | Needs location constraint ap-south-1.
17204 \ "ap-south-1"
17205 / Asia Pacific (Hong Kong) Region
17206 14 | Needs location constraint ap-east-1.
17207 \ "ap-east-1"
17208 / South America (Sao Paulo) Region
17209 15 | Needs location constraint sa-east-1.
17210 \ "sa-east-1"
17211 region> 1
17212 Endpoint for S3 API.
17213 Leave blank if using AWS to use the default endpoint for the region.
17214 endpoint>
17215 Location constraint - must be set to match the Region. Used when creating buckets only.
17216 Choose a number from below, or type in your own value
17217 1 / Empty for US Region, Northern Virginia, or Pacific Northwest.
17218 \ ""
17219 2 / US East (Ohio) Region.
17220 \ "us-east-2"
17221 3 / US West (Oregon) Region.
17222 \ "us-west-2"
17223 4 / US West (Northern California) Region.
17224 \ "us-west-1"
17225 5 / Canada (Central) Region.
17226 \ "ca-central-1"
17227 6 / EU (Ireland) Region.
17228 \ "eu-west-1"
17229 7 / EU (London) Region.
17230 \ "eu-west-2"
17231 8 / EU Region.
17232 \ "EU"
17233 9 / Asia Pacific (Singapore) Region.
17234 \ "ap-southeast-1"
17235 10 / Asia Pacific (Sydney) Region.
17236 \ "ap-southeast-2"
17237 11 / Asia Pacific (Tokyo) Region.
17238 \ "ap-northeast-1"
17239 12 / Asia Pacific (Seoul)
17240 \ "ap-northeast-2"
17241 13 / Asia Pacific (Mumbai)
17242 \ "ap-south-1"
17243 14 / Asia Pacific (Hong Kong)
17244 \ "ap-east-1"
17245 15 / South America (Sao Paulo) Region.
17246 \ "sa-east-1"
17247 location_constraint> 1
17248 Canned ACL used when creating buckets and/or storing objects in S3.
17249 For more info visit https://docs.aws.amazon.com/AmazonS3/latest/dev/acl-overview.html#canned-acl
17250 Choose a number from below, or type in your own value
17251 1 / Owner gets FULL_CONTROL. No one else has access rights (default).
17252 \ "private"
17253 2 / Owner gets FULL_CONTROL. The AllUsers group gets READ access.
17254 \ "public-read"
17255 / Owner gets FULL_CONTROL. The AllUsers group gets READ and WRITE access.
17256 3 | Granting this on a bucket is generally not recommended.
17257 \ "public-read-write"
17258 4 / Owner gets FULL_CONTROL. The AuthenticatedUsers group gets READ access.
17259 \ "authenticated-read"
17260 / Object owner gets FULL_CONTROL. Bucket owner gets READ access.
17261 5 | If you specify this canned ACL when creating a bucket, Amazon S3 ignores it.
17262 \ "bucket-owner-read"
17263 / Both the object owner and the bucket owner get FULL_CONTROL over the object.
17264 6 | If you specify this canned ACL when creating a bucket, Amazon S3 ignores it.
17265 \ "bucket-owner-full-control"
17266 acl> 1
17267 The server-side encryption algorithm used when storing this object in S3.
17268 Choose a number from below, or type in your own value
17269 1 / None
17270 \ ""
17271 2 / AES256
17272 \ "AES256"
17273 server_side_encryption> 1
17274 The storage class to use when storing objects in S3.
17275 Choose a number from below, or type in your own value
17276 1 / Default
17277 \ ""
17278 2 / Standard storage class
17279 \ "STANDARD"
17280 3 / Reduced redundancy storage class
17281 \ "REDUCED_REDUNDANCY"
17282 4 / Standard Infrequent Access storage class
17283 \ "STANDARD_IA"
17284 5 / One Zone Infrequent Access storage class
17285 \ "ONEZONE_IA"
17286 6 / Glacier storage class
17287 \ "GLACIER"
17288 7 / Glacier Deep Archive storage class
17289 \ "DEEP_ARCHIVE"
17290 8 / Intelligent-Tiering storage class
17291 \ "INTELLIGENT_TIERING"
17292 9 / Glacier Instant Retrieval storage class
17293 \ "GLACIER_IR"
17294 storage_class> 1
17295 Remote config
17296 --------------------
17297 [remote]
17298 type = s3
17299 provider = AWS
17300 env_auth = false
17301 access_key_id = XXX
17302 secret_access_key = YYY
17303 region = us-east-1
17304 endpoint =
17305 location_constraint =
17306 acl = private
17307 server_side_encryption =
17308 storage_class =
17309 --------------------
17310 y) Yes this is OK
17311 e) Edit this remote
17312 d) Delete this remote
17313 y/e/d>
17314
17315 Modified time
17316 The modified time is stored as metadata on the object as X-Amz-Meta-
17317 Mtime as floating point since the epoch, accurate to 1 ns.
17318
17319 If the modification time needs to be updated rclone will attempt to
17320 perform a server side copy to update the modification if the object can
17321 be copied in a single part. In the case the object is larger than 5Gb
17322 or is in Glacier or Glacier Deep Archive storage the object will be up‐
17323 loaded rather than copied.
17324
17325 Note that reading this from the object takes an additional HEAD request
17326 as the metadata isn't returned in object listings.
17327
17328 Reducing costs
17329 Avoiding HEAD requests to read the modification time
17330 By default, rclone will use the modification time of objects stored in
17331 S3 for syncing. This is stored in object metadata which unfortunately
17332 takes an extra HEAD request to read which can be expensive (in time and
17333 money).
17334
17335 The modification time is used by default for all operations that re‐
17336 quire checking the time a file was last updated. It allows rclone to
17337 treat the remote more like a true filesystem, but it is inefficient on
17338 S3 because it requires an extra API call to retrieve the metadata.
17339
17340 The extra API calls can be avoided when syncing (using rclone sync or
17341 rclone copy) in a few different ways, each with its own tradeoffs.
17342
17343 • --size-only
17344
17345 • Only checks the size of files.
17346
17347 • Uses no extra transactions.
17348
17349 • If the file doesn't change size then rclone won't detect it has
17350 changed.
17351
17352 • rclone sync --size-only /path/to/source s3:bucket
17353
17354 • --checksum
17355
17356 • Checks the size and MD5 checksum of files.
17357
17358 • Uses no extra transactions.
17359
17360 • The most accurate detection of changes possible.
17361
17362 • Will cause the source to read an MD5 checksum which, if it is a lo‐
17363 cal disk, will cause lots of disk activity.
17364
17365 • If the source and destination are both S3 this is the recommended
17366 flag to use for maximum efficiency.
17367
17368 • rclone sync --checksum /path/to/source s3:bucket
17369
17370 • --update --use-server-modtime
17371
17372 • Uses no extra transactions.
17373
17374 • Modification time becomes the time the object was uploaded.
17375
17376 • For many operations this is sufficient to determine if it needs up‐
17377 loading.
17378
17379 • Using --update along with --use-server-modtime, avoids the extra
17380 API call and uploads files whose local modification time is newer
17381 than the time it was last uploaded.
17382
17383 • Files created with timestamps in the past will be missed by the
17384 sync.
17385
17386 • rclone sync --update --use-server-modtime /path/to/source s3:bucket
17387
17388 These flags can and should be used in combination with --fast-list -
17389 see below.
17390
17391 If using rclone mount or any command using the VFS (eg rclone serve)
17392 commands then you might want to consider using the VFS flag --no-mod‐
17393 time which will stop rclone reading the modification time for every ob‐
17394 ject. You could also use --use-server-modtime if you are happy with
17395 the modification times of the objects being the time of upload.
17396
17397 Avoiding GET requests to read directory listings
17398 Rclone's default directory traversal is to process each directory indi‐
17399 vidually. This takes one API call per directory. Using the --fast-
17400 list flag will read all info about the objects into memory first using
17401 a smaller number of API calls (one per 1000 objects). See the rclone
17402 docs (https://rclone.org/docs/#fast-list) for more details.
17403
17404 rclone sync --fast-list --checksum /path/to/source s3:bucket
17405
17406 --fast-list trades off API transactions for memory use. As a rough
17407 guide rclone uses 1k of memory per object stored, so using --fast-list
17408 on a sync of a million objects will use roughly 1 GiB of RAM.
17409
17410 If you are only copying a small number of files into a big repository
17411 then using --no-traverse is a good idea. This finds objects directly
17412 instead of through directory listings. You can do a "top-up" sync very
17413 cheaply by using --max-age and --no-traverse to copy only recent files,
17414 eg
17415
17416 rclone copy --max-age 24h --no-traverse /path/to/source s3:bucket
17417
17418 You'd then do a full rclone sync less often.
17419
17420 Note that --fast-list isn't required in the top-up sync.
17421
17422 Avoiding HEAD requests after PUT
17423 By default, rclone will HEAD every object it uploads. It does this to
17424 check the object got uploaded correctly.
17425
17426 You can disable this with the --s3-no-head option - see there for more
17427 details.
17428
17429 Setting this flag increases the chance for undetected upload failures.
17430
17431 Hashes
17432 For small objects which weren't uploaded as multipart uploads (objects
17433 sized below --s3-upload-cutoff if uploaded with rclone) rclone uses the
17434 ETag: header as an MD5 checksum.
17435
17436 However for objects which were uploaded as multipart uploads or with
17437 server side encryption (SSE-AWS or SSE-C) the ETag header is no longer
17438 the MD5 sum of the data, so rclone adds an additional piece of metadata
17439 X-Amz-Meta-Md5chksum which is a base64 encoded MD5 hash (in the same
17440 format as is required for Content-MD5).
17441
17442 For large objects, calculating this hash can take some time so the ad‐
17443 dition of this hash can be disabled with --s3-disable-checksum. This
17444 will mean that these objects do not have an MD5 checksum.
17445
17446 Note that reading this from the object takes an additional HEAD request
17447 as the metadata isn't returned in object listings.
17448
17449 Versions
17450 When bucket versioning is enabled (this can be done with rclone with
17451 the rclone backend versioning command) when rclone uploads a new ver‐
17452 sion of a file it creates a new version of it (https://docs.aws.ama‐
17453 zon.com/AmazonS3/latest/userguide/Versioning.html) Likewise when you
17454 delete a file, the old version will be marked hidden and still be
17455 available.
17456
17457 Old versions of files, where available, are visible using the --s3-ver‐
17458 sions flag.
17459
17460 It is also possible to view a bucket as it was at a certain point in
17461 time, using the --s3-version-at flag. This will show the file versions
17462 as they were at that time, showing files that have been deleted after‐
17463 wards, and hiding files that were created since.
17464
17465 If you wish to remove all the old versions then you can use the rclone
17466 backend cleanup-hidden remote:bucket command which will delete all the
17467 old hidden versions of files, leaving the current ones intact. You can
17468 also supply a path and only old versions under that path will be delet‐
17469 ed, e.g. rclone backend cleanup-hidden remote:bucket/path/to/stuff.
17470
17471 When you purge a bucket, the current and the old versions will be
17472 deleted then the bucket will be deleted.
17473
17474 However delete will cause the current versions of the files to become
17475 hidden old versions.
17476
17477 Here is a session showing the listing and retrieval of an old version
17478 followed by a cleanup of the old versions.
17479
17480 Show current version and all the versions with --s3-versions flag.
17481
17482 $ rclone -q ls s3:cleanup-test
17483 9 one.txt
17484
17485 $ rclone -q --s3-versions ls s3:cleanup-test
17486 9 one.txt
17487 8 one-v2016-07-04-141032-000.txt
17488 16 one-v2016-07-04-141003-000.txt
17489 15 one-v2016-07-02-155621-000.txt
17490
17491 Retrieve an old version
17492
17493 $ rclone -q --s3-versions copy s3:cleanup-test/one-v2016-07-04-141003-000.txt /tmp
17494
17495 $ ls -l /tmp/one-v2016-07-04-141003-000.txt
17496 -rw-rw-r-- 1 ncw ncw 16 Jul 2 17:46 /tmp/one-v2016-07-04-141003-000.txt
17497
17498 Clean up all the old versions and show that they've gone.
17499
17500 $ rclone -q backend cleanup-hidden s3:cleanup-test
17501
17502 $ rclone -q ls s3:cleanup-test
17503 9 one.txt
17504
17505 $ rclone -q --s3-versions ls s3:cleanup-test
17506 9 one.txt
17507
17508 Cleanup
17509 If you run rclone cleanup s3:bucket then it will remove all pending
17510 multipart uploads older than 24 hours. You can use the -i flag to see
17511 exactly what it will do. If you want more control over the expiry date
17512 then run rclone backend cleanup s3:bucket -o max-age=1h to expire all
17513 uploads older than one hour. You can use rclone backend list-multi‐
17514 part-uploads s3:bucket to see the pending multipart uploads.
17515
17516 Restricted filename characters
17517 S3 allows any valid UTF-8 string as a key.
17518
17519 Invalid UTF-8 bytes will be replaced (https://rclone.org/overview/#in‐
17520 valid-utf8), as they can't be used in XML.
17521
17522 The following characters are replaced since these are problematic when
17523 dealing with the REST API:
17524
17525 Character Value Replacement
17526 ────────────────────────────────
17527 NUL 0x00 ␀
17528 / 0x2F /
17529
17530 The encoding will also encode these file names as they don't seem to
17531 work with the SDK properly:
17532
17533 File name Replacement
17534 ────────────────────────
17535 . .
17536 .. ..
17537
17538 Multipart uploads
17539 rclone supports multipart uploads with S3 which means that it can up‐
17540 load files bigger than 5 GiB.
17541
17542 Note that files uploaded both with multipart upload and through crypt
17543 remotes do not have MD5 sums.
17544
17545 rclone switches from single part uploads to multipart uploads at the
17546 point specified by --s3-upload-cutoff. This can be a maximum of 5 GiB
17547 and a minimum of 0 (ie always upload multipart files).
17548
17549 The chunk sizes used in the multipart upload are specified by
17550 --s3-chunk-size and the number of chunks uploaded concurrently is spec‐
17551 ified by --s3-upload-concurrency.
17552
17553 Multipart uploads will use --transfers * --s3-upload-concurrency *
17554 --s3-chunk-size extra memory. Single part uploads to not use extra
17555 memory.
17556
17557 Single part transfers can be faster than multipart transfers or slower
17558 depending on your latency from S3 - the more latency, the more likely
17559 single part transfers will be faster.
17560
17561 Increasing --s3-upload-concurrency will increase throughput (8 would be
17562 a sensible value) and increasing --s3-chunk-size also increases
17563 throughput (16M would be sensible). Increasing either of these will
17564 use more memory. The default values are high enough to gain most of
17565 the possible performance without using too much memory.
17566
17567 Buckets and Regions
17568 With Amazon S3 you can list buckets (rclone lsd) using any region, but
17569 you can only access the content of a bucket from the region it was cre‐
17570 ated in. If you attempt to access a bucket from the wrong region, you
17571 will get an error, incorrect region, the bucket is not in 'XXX' region.
17572
17573 Authentication
17574 There are a number of ways to supply rclone with a set of AWS creden‐
17575 tials, with and without using the environment.
17576
17577 The different authentication methods are tried in this order:
17578
17579 • Directly in the rclone configuration file (env_auth = false in the
17580 config file):
17581
17582 • access_key_id and secret_access_key are required.
17583
17584 • session_token can be optionally set when using AWS STS.
17585
17586 • Runtime configuration (env_auth = true in the config file):
17587
17588 • Export the following environment variables before running rclone:
17589
17590 • Access Key ID: AWS_ACCESS_KEY_ID or AWS_ACCESS_KEY
17591
17592 • Secret Access Key: AWS_SECRET_ACCESS_KEY or AWS_SECRET_KEY
17593
17594 • Session Token: AWS_SESSION_TOKEN (optional)
17595
17596 • Or, use a named profile (https://docs.aws.amazon.com/cli/lat‐
17597 est/userguide/cli-multiple-profiles.html):
17598
17599 • Profile files are standard files used by AWS CLI tools
17600
17601 • By default it will use the profile in your home directory (e.g.
17602 ~/.aws/credentials on unix based systems) file and the "default"
17603 profile, to change set these environment variables:
17604
17605 • AWS_SHARED_CREDENTIALS_FILE to control which file.
17606
17607 • AWS_PROFILE to control which profile to use.
17608
17609 • Or, run rclone in an ECS task with an IAM role (AWS only).
17610
17611 • Or, run rclone on an EC2 instance with an IAM role (AWS only).
17612
17613 • Or, run rclone in an EKS pod with an IAM role that is associated
17614 with a service account (AWS only).
17615
17616 If none of these option actually end up providing rclone with AWS cre‐
17617 dentials then S3 interaction will be non-authenticated (see below).
17618
17619 S3 Permissions
17620 When using the sync subcommand of rclone the following minimum permis‐
17621 sions are required to be available on the bucket being written to:
17622
17623 • ListBucket
17624
17625 • DeleteObject
17626
17627 • GetObject
17628
17629 • PutObject
17630
17631 • PutObjectACL
17632
17633 When using the lsd subcommand, the ListAllMyBuckets permission is re‐
17634 quired.
17635
17636 Example policy:
17637
17638 {
17639 "Version": "2012-10-17",
17640 "Statement": [
17641 {
17642 "Effect": "Allow",
17643 "Principal": {
17644 "AWS": "arn:aws:iam::USER_SID:user/USER_NAME"
17645 },
17646 "Action": [
17647 "s3:ListBucket",
17648 "s3:DeleteObject",
17649 "s3:GetObject",
17650 "s3:PutObject",
17651 "s3:PutObjectAcl"
17652 ],
17653 "Resource": [
17654 "arn:aws:s3:::BUCKET_NAME/*",
17655 "arn:aws:s3:::BUCKET_NAME"
17656 ]
17657 },
17658 {
17659 "Effect": "Allow",
17660 "Action": "s3:ListAllMyBuckets",
17661 "Resource": "arn:aws:s3:::*"
17662 }
17663 ]
17664 }
17665
17666 Notes on above:
17667
17668 1. This is a policy that can be used when creating bucket. It assumes
17669 that USER_NAME has been created.
17670
17671 2. The Resource entry must include both resource ARNs, as one implies
17672 the bucket and the other implies the bucket's objects.
17673
17674 For reference, here's an Ansible script
17675 (https://gist.github.com/ebridges/ebfc9042dd7c756cd101cfa807b7ae2b)
17676 that will generate one or more buckets that will work with rclone sync.
17677
17678 Key Management System (KMS)
17679 If you are using server-side encryption with KMS then you must make
17680 sure rclone is configured with server_side_encryption = aws:kms other‐
17681 wise you will find you can't transfer small objects - these will create
17682 checksum errors.
17683
17684 Glacier and Glacier Deep Archive
17685 You can upload objects using the glacier storage class or transition
17686 them to glacier using a lifecycle policy (http://docs.aws.ama‐
17687 zon.com/AmazonS3/latest/user-guide/create-lifecycle.html). The bucket
17688 can still be synced or copied into normally, but if rclone tries to ac‐
17689 cess data from the glacier storage class you will see an error like be‐
17690 low.
17691
17692 2017/09/11 19:07:43 Failed to sync: failed to open source object: Object in GLACIER, restore first: path/to/file
17693
17694 In this case you need to restore (http://docs.aws.amazon.com/Ama‐
17695 zonS3/latest/user-guide/restore-archived-objects.html) the object(s) in
17696 question before using rclone.
17697
17698 Note that rclone only speaks the S3 API it does not speak the Glacier
17699 Vault API, so rclone cannot directly access Glacier Vaults.
17700
17701 Object-lock enabled S3 bucket
17702 According to AWS's documentation on S3 Object Lock
17703 (https://docs.aws.amazon.com/AmazonS3/latest/userguide/object-lock-
17704 overview.html#object-lock-permission):
17705
17706 If you configure a default retention period on a bucket, re‐
17707 quests to upload objects in such a bucket must include the Con‐
17708 tent-MD5 header.
17709
17710 As mentioned in the Hashes section, small files that are not uploaded
17711 as multipart, use a different tag, causing the upload to fail. A sim‐
17712 ple solution is to set the --s3-upload-cutoff 0 and force all the files
17713 to be uploaded as multipart.
17714
17715 Standard options
17716 Here are the Standard options specific to s3 (Amazon S3 Compliant Stor‐
17717 age Providers including AWS, Alibaba, Ceph, China Mobile, Cloudflare,
17718 ArvanCloud, Digital Ocean, Dreamhost, Huawei OBS, IBM COS, IDrive e2,
17719 IONOS Cloud, Lyve Cloud, Minio, Netease, RackCorp, Scaleway, SeaweedFS,
17720 StackPath, Storj, Tencent COS, Qiniu and Wasabi).
17721
17722 --s3-provider
17723 Choose your S3 provider.
17724
17725 Properties:
17726
17727 • Config: provider
17728
17729 • Env Var: RCLONE_S3_PROVIDER
17730
17731 • Type: string
17732
17733 • Required: false
17734
17735 • Examples:
17736
17737 • "AWS"
17738
17739 • Amazon Web Services (AWS) S3
17740
17741 • "Alibaba"
17742
17743 • Alibaba Cloud Object Storage System (OSS) formerly Aliyun
17744
17745 • "Ceph"
17746
17747 • Ceph Object Storage
17748
17749 • "ChinaMobile"
17750
17751 • China Mobile Ecloud Elastic Object Storage (EOS)
17752
17753 • "Cloudflare"
17754
17755 • Cloudflare R2 Storage
17756
17757 • "ArvanCloud"
17758
17759 • Arvan Cloud Object Storage (AOS)
17760
17761 • "DigitalOcean"
17762
17763 • Digital Ocean Spaces
17764
17765 • "Dreamhost"
17766
17767 • Dreamhost DreamObjects
17768
17769 • "HuaweiOBS"
17770
17771 • Huawei Object Storage Service
17772
17773 • "IBMCOS"
17774
17775 • IBM COS S3
17776
17777 • "IDrive"
17778
17779 • IDrive e2
17780
17781 • "IONOS"
17782
17783 • IONOS Cloud
17784
17785 • "LyveCloud"
17786
17787 • Seagate Lyve Cloud
17788
17789 • "Minio"
17790
17791 • Minio Object Storage
17792
17793 • "Netease"
17794
17795 • Netease Object Storage (NOS)
17796
17797 • "RackCorp"
17798
17799 • RackCorp Object Storage
17800
17801 • "Scaleway"
17802
17803 • Scaleway Object Storage
17804
17805 • "SeaweedFS"
17806
17807 • SeaweedFS S3
17808
17809 • "StackPath"
17810
17811 • StackPath Object Storage
17812
17813 • "Storj"
17814
17815 • Storj (S3 Compatible Gateway)
17816
17817 • "TencentCOS"
17818
17819 • Tencent Cloud Object Storage (COS)
17820
17821 • "Wasabi"
17822
17823 • Wasabi Object Storage
17824
17825 • "Qiniu"
17826
17827 • Qiniu Object Storage (Kodo)
17828
17829 • "Other"
17830
17831 • Any other S3 compatible provider
17832
17833 --s3-env-auth
17834 Get AWS credentials from runtime (environment variables or EC2/ECS meta
17835 data if no env vars).
17836
17837 Only applies if access_key_id and secret_access_key is blank.
17838
17839 Properties:
17840
17841 • Config: env_auth
17842
17843 • Env Var: RCLONE_S3_ENV_AUTH
17844
17845 • Type: bool
17846
17847 • Default: false
17848
17849 • Examples:
17850
17851 • "false"
17852
17853 • Enter AWS credentials in the next step.
17854
17855 • "true"
17856
17857 • Get AWS credentials from the environment (env vars or IAM).
17858
17859 --s3-access-key-id
17860 AWS Access Key ID.
17861
17862 Leave blank for anonymous access or runtime credentials.
17863
17864 Properties:
17865
17866 • Config: access_key_id
17867
17868 • Env Var: RCLONE_S3_ACCESS_KEY_ID
17869
17870 • Type: string
17871
17872 • Required: false
17873
17874 --s3-secret-access-key
17875 AWS Secret Access Key (password).
17876
17877 Leave blank for anonymous access or runtime credentials.
17878
17879 Properties:
17880
17881 • Config: secret_access_key
17882
17883 • Env Var: RCLONE_S3_SECRET_ACCESS_KEY
17884
17885 • Type: string
17886
17887 • Required: false
17888
17889 --s3-region
17890 Region to connect to.
17891
17892 Properties:
17893
17894 • Config: region
17895
17896 • Env Var: RCLONE_S3_REGION
17897
17898 • Provider: AWS
17899
17900 • Type: string
17901
17902 • Required: false
17903
17904 • Examples:
17905
17906 • "us-east-1"
17907
17908 • The default endpoint - a good choice if you are unsure.
17909
17910 • US Region, Northern Virginia, or Pacific Northwest.
17911
17912 • Leave location constraint empty.
17913
17914 • "us-east-2"
17915
17916 • US East (Ohio) Region.
17917
17918 • Needs location constraint us-east-2.
17919
17920 • "us-west-1"
17921
17922 • US West (Northern California) Region.
17923
17924 • Needs location constraint us-west-1.
17925
17926 • "us-west-2"
17927
17928 • US West (Oregon) Region.
17929
17930 • Needs location constraint us-west-2.
17931
17932 • "ca-central-1"
17933
17934 • Canada (Central) Region.
17935
17936 • Needs location constraint ca-central-1.
17937
17938 • "eu-west-1"
17939
17940 • EU (Ireland) Region.
17941
17942 • Needs location constraint EU or eu-west-1.
17943
17944 • "eu-west-2"
17945
17946 • EU (London) Region.
17947
17948 • Needs location constraint eu-west-2.
17949
17950 • "eu-west-3"
17951
17952 • EU (Paris) Region.
17953
17954 • Needs location constraint eu-west-3.
17955
17956 • "eu-north-1"
17957
17958 • EU (Stockholm) Region.
17959
17960 • Needs location constraint eu-north-1.
17961
17962 • "eu-south-1"
17963
17964 • EU (Milan) Region.
17965
17966 • Needs location constraint eu-south-1.
17967
17968 • "eu-central-1"
17969
17970 • EU (Frankfurt) Region.
17971
17972 • Needs location constraint eu-central-1.
17973
17974 • "ap-southeast-1"
17975
17976 • Asia Pacific (Singapore) Region.
17977
17978 • Needs location constraint ap-southeast-1.
17979
17980 • "ap-southeast-2"
17981
17982 • Asia Pacific (Sydney) Region.
17983
17984 • Needs location constraint ap-southeast-2.
17985
17986 • "ap-northeast-1"
17987
17988 • Asia Pacific (Tokyo) Region.
17989
17990 • Needs location constraint ap-northeast-1.
17991
17992 • "ap-northeast-2"
17993
17994 • Asia Pacific (Seoul).
17995
17996 • Needs location constraint ap-northeast-2.
17997
17998 • "ap-northeast-3"
17999
18000 • Asia Pacific (Osaka-Local).
18001
18002 • Needs location constraint ap-northeast-3.
18003
18004 • "ap-south-1"
18005
18006 • Asia Pacific (Mumbai).
18007
18008 • Needs location constraint ap-south-1.
18009
18010 • "ap-east-1"
18011
18012 • Asia Pacific (Hong Kong) Region.
18013
18014 • Needs location constraint ap-east-1.
18015
18016 • "sa-east-1"
18017
18018 • South America (Sao Paulo) Region.
18019
18020 • Needs location constraint sa-east-1.
18021
18022 • "me-south-1"
18023
18024 • Middle East (Bahrain) Region.
18025
18026 • Needs location constraint me-south-1.
18027
18028 • "af-south-1"
18029
18030 • Africa (Cape Town) Region.
18031
18032 • Needs location constraint af-south-1.
18033
18034 • "cn-north-1"
18035
18036 • China (Beijing) Region.
18037
18038 • Needs location constraint cn-north-1.
18039
18040 • "cn-northwest-1"
18041
18042 • China (Ningxia) Region.
18043
18044 • Needs location constraint cn-northwest-1.
18045
18046 • "us-gov-east-1"
18047
18048 • AWS GovCloud (US-East) Region.
18049
18050 • Needs location constraint us-gov-east-1.
18051
18052 • "us-gov-west-1"
18053
18054 • AWS GovCloud (US) Region.
18055
18056 • Needs location constraint us-gov-west-1.
18057
18058 --s3-region
18059 region - the location where your bucket will be created and your data
18060 stored.
18061
18062 Properties:
18063
18064 • Config: region
18065
18066 • Env Var: RCLONE_S3_REGION
18067
18068 • Provider: RackCorp
18069
18070 • Type: string
18071
18072 • Required: false
18073
18074 • Examples:
18075
18076 • "global"
18077
18078 • Global CDN (All locations) Region
18079
18080 • "au"
18081
18082 • Australia (All states)
18083
18084 • "au-nsw"
18085
18086 • NSW (Australia) Region
18087
18088 • "au-qld"
18089
18090 • QLD (Australia) Region
18091
18092 • "au-vic"
18093
18094 • VIC (Australia) Region
18095
18096 • "au-wa"
18097
18098 • Perth (Australia) Region
18099
18100 • "ph"
18101
18102 • Manila (Philippines) Region
18103
18104 • "th"
18105
18106 • Bangkok (Thailand) Region
18107
18108 • "hk"
18109
18110 • HK (Hong Kong) Region
18111
18112 • "mn"
18113
18114 • Ulaanbaatar (Mongolia) Region
18115
18116 • "kg"
18117
18118 • Bishkek (Kyrgyzstan) Region
18119
18120 • "id"
18121
18122 • Jakarta (Indonesia) Region
18123
18124 • "jp"
18125
18126 • Tokyo (Japan) Region
18127
18128 • "sg"
18129
18130 • SG (Singapore) Region
18131
18132 • "de"
18133
18134 • Frankfurt (Germany) Region
18135
18136 • "us"
18137
18138 • USA (AnyCast) Region
18139
18140 • "us-east-1"
18141
18142 • New York (USA) Region
18143
18144 • "us-west-1"
18145
18146 • Freemont (USA) Region
18147
18148 • "nz"
18149
18150 • Auckland (New Zealand) Region
18151
18152 --s3-region
18153 Region to connect to.
18154
18155 Properties:
18156
18157 • Config: region
18158
18159 • Env Var: RCLONE_S3_REGION
18160
18161 • Provider: Scaleway
18162
18163 • Type: string
18164
18165 • Required: false
18166
18167 • Examples:
18168
18169 • "nl-ams"
18170
18171 • Amsterdam, The Netherlands
18172
18173 • "fr-par"
18174
18175 • Paris, France
18176
18177 • "pl-waw"
18178
18179 • Warsaw, Poland
18180
18181 --s3-region
18182 Region to connect to. - the location where your bucket will be created
18183 and your data stored. Need bo be same with your endpoint.
18184
18185 Properties:
18186
18187 • Config: region
18188
18189 • Env Var: RCLONE_S3_REGION
18190
18191 • Provider: HuaweiOBS
18192
18193 • Type: string
18194
18195 • Required: false
18196
18197 • Examples:
18198
18199 • "af-south-1"
18200
18201 • AF-Johannesburg
18202
18203 • "ap-southeast-2"
18204
18205 • AP-Bangkok
18206
18207 • "ap-southeast-3"
18208
18209 • AP-Singapore
18210
18211 • "cn-east-3"
18212
18213 • CN East-Shanghai1
18214
18215 • "cn-east-2"
18216
18217 • CN East-Shanghai2
18218
18219 • "cn-north-1"
18220
18221 • CN North-Beijing1
18222
18223 • "cn-north-4"
18224
18225 • CN North-Beijing4
18226
18227 • "cn-south-1"
18228
18229 • CN South-Guangzhou
18230
18231 • "ap-southeast-1"
18232
18233 • CN-Hong Kong
18234
18235 • "sa-argentina-1"
18236
18237 • LA-Buenos Aires1
18238
18239 • "sa-peru-1"
18240
18241 • LA-Lima1
18242
18243 • "na-mexico-1"
18244
18245 • LA-Mexico City1
18246
18247 • "sa-chile-1"
18248
18249 • LA-Santiago2
18250
18251 • "sa-brazil-1"
18252
18253 • LA-Sao Paulo1
18254
18255 • "ru-northwest-2"
18256
18257 • RU-Moscow2
18258
18259 --s3-region
18260 Region to connect to.
18261
18262 Properties:
18263
18264 • Config: region
18265
18266 • Env Var: RCLONE_S3_REGION
18267
18268 • Provider: Cloudflare
18269
18270 • Type: string
18271
18272 • Required: false
18273
18274 • Examples:
18275
18276 • "auto"
18277
18278 • R2 buckets are automatically distributed across Cloudflare's data
18279 centers for low latency.
18280
18281 --s3-region
18282 Region to connect to.
18283
18284 Properties:
18285
18286 • Config: region
18287
18288 • Env Var: RCLONE_S3_REGION
18289
18290 • Provider: Qiniu
18291
18292 • Type: string
18293
18294 • Required: false
18295
18296 • Examples:
18297
18298 • "cn-east-1"
18299
18300 • The default endpoint - a good choice if you are unsure.
18301
18302 • East China Region 1.
18303
18304 • Needs location constraint cn-east-1.
18305
18306 • "cn-east-2"
18307
18308 • East China Region 2.
18309
18310 • Needs location constraint cn-east-2.
18311
18312 • "cn-north-1"
18313
18314 • North China Region 1.
18315
18316 • Needs location constraint cn-north-1.
18317
18318 • "cn-south-1"
18319
18320 • South China Region 1.
18321
18322 • Needs location constraint cn-south-1.
18323
18324 • "us-north-1"
18325
18326 • North America Region.
18327
18328 • Needs location constraint us-north-1.
18329
18330 • "ap-southeast-1"
18331
18332 • Southeast Asia Region 1.
18333
18334 • Needs location constraint ap-southeast-1.
18335
18336 • "ap-northeast-1"
18337
18338 • Northeast Asia Region 1.
18339
18340 • Needs location constraint ap-northeast-1.
18341
18342 --s3-region
18343 Region where your bucket will be created and your data stored.
18344
18345 Properties:
18346
18347 • Config: region
18348
18349 • Env Var: RCLONE_S3_REGION
18350
18351 • Provider: IONOS
18352
18353 • Type: string
18354
18355 • Required: false
18356
18357 • Examples:
18358
18359 • "de"
18360
18361 • Frankfurt, Germany
18362
18363 • "eu-central-2"
18364
18365 • Berlin, Germany
18366
18367 • "eu-south-2"
18368
18369 • Logrono, Spain
18370
18371 --s3-region
18372 Region to connect to.
18373
18374 Leave blank if you are using an S3 clone and you don't have a region.
18375
18376 Properties:
18377
18378 • Config: region
18379
18380 • Env Var: RCLONE_S3_REGION
18381
18382 • Provider: !AWS,Alibaba,ChinaMobile,Cloudflare,IONOS,Arvan‐
18383 Cloud,Qiniu,RackCorp,Scaleway,Storj,TencentCOS,HuaweiOBS,IDrive
18384
18385 • Type: string
18386
18387 • Required: false
18388
18389 • Examples:
18390
18391 • ""
18392
18393 • Use this if unsure.
18394
18395 • Will use v4 signatures and an empty region.
18396
18397 • "other-v2-signature"
18398
18399 • Use this only if v4 signatures don't work.
18400
18401 • E.g. pre Jewel/v10 CEPH.
18402
18403 --s3-endpoint
18404 Endpoint for S3 API.
18405
18406 Leave blank if using AWS to use the default endpoint for the region.
18407
18408 Properties:
18409
18410 • Config: endpoint
18411
18412 • Env Var: RCLONE_S3_ENDPOINT
18413
18414 • Provider: AWS
18415
18416 • Type: string
18417
18418 • Required: false
18419
18420 --s3-endpoint
18421 Endpoint for China Mobile Ecloud Elastic Object Storage (EOS) API.
18422
18423 Properties:
18424
18425 • Config: endpoint
18426
18427 • Env Var: RCLONE_S3_ENDPOINT
18428
18429 • Provider: ChinaMobile
18430
18431 • Type: string
18432
18433 • Required: false
18434
18435 • Examples:
18436
18437 • "eos-wuxi-1.cmecloud.cn"
18438
18439 • The default endpoint - a good choice if you are unsure.
18440
18441 • East China (Suzhou)
18442
18443 • "eos-jinan-1.cmecloud.cn"
18444
18445 • East China (Jinan)
18446
18447 • "eos-ningbo-1.cmecloud.cn"
18448
18449 • East China (Hangzhou)
18450
18451 • "eos-shanghai-1.cmecloud.cn"
18452
18453 • East China (Shanghai-1)
18454
18455 • "eos-zhengzhou-1.cmecloud.cn"
18456
18457 • Central China (Zhengzhou)
18458
18459 • "eos-hunan-1.cmecloud.cn"
18460
18461 • Central China (Changsha-1)
18462
18463 • "eos-zhuzhou-1.cmecloud.cn"
18464
18465 • Central China (Changsha-2)
18466
18467 • "eos-guangzhou-1.cmecloud.cn"
18468
18469 • South China (Guangzhou-2)
18470
18471 • "eos-dongguan-1.cmecloud.cn"
18472
18473 • South China (Guangzhou-3)
18474
18475 • "eos-beijing-1.cmecloud.cn"
18476
18477 • North China (Beijing-1)
18478
18479 • "eos-beijing-2.cmecloud.cn"
18480
18481 • North China (Beijing-2)
18482
18483 • "eos-beijing-4.cmecloud.cn"
18484
18485 • North China (Beijing-3)
18486
18487 • "eos-huhehaote-1.cmecloud.cn"
18488
18489 • North China (Huhehaote)
18490
18491 • "eos-chengdu-1.cmecloud.cn"
18492
18493 • Southwest China (Chengdu)
18494
18495 • "eos-chongqing-1.cmecloud.cn"
18496
18497 • Southwest China (Chongqing)
18498
18499 • "eos-guiyang-1.cmecloud.cn"
18500
18501 • Southwest China (Guiyang)
18502
18503 • "eos-xian-1.cmecloud.cn"
18504
18505 • Nouthwest China (Xian)
18506
18507 • "eos-yunnan.cmecloud.cn"
18508
18509 • Yunnan China (Kunming)
18510
18511 • "eos-yunnan-2.cmecloud.cn"
18512
18513 • Yunnan China (Kunming-2)
18514
18515 • "eos-tianjin-1.cmecloud.cn"
18516
18517 • Tianjin China (Tianjin)
18518
18519 • "eos-jilin-1.cmecloud.cn"
18520
18521 • Jilin China (Changchun)
18522
18523 • "eos-hubei-1.cmecloud.cn"
18524
18525 • Hubei China (Xiangyan)
18526
18527 • "eos-jiangxi-1.cmecloud.cn"
18528
18529 • Jiangxi China (Nanchang)
18530
18531 • "eos-gansu-1.cmecloud.cn"
18532
18533 • Gansu China (Lanzhou)
18534
18535 • "eos-shanxi-1.cmecloud.cn"
18536
18537 • Shanxi China (Taiyuan)
18538
18539 • "eos-liaoning-1.cmecloud.cn"
18540
18541 • Liaoning China (Shenyang)
18542
18543 • "eos-hebei-1.cmecloud.cn"
18544
18545 • Hebei China (Shijiazhuang)
18546
18547 • "eos-fujian-1.cmecloud.cn"
18548
18549 • Fujian China (Xiamen)
18550
18551 • "eos-guangxi-1.cmecloud.cn"
18552
18553 • Guangxi China (Nanning)
18554
18555 • "eos-anhui-1.cmecloud.cn"
18556
18557 • Anhui China (Huainan)
18558
18559 --s3-endpoint
18560 Endpoint for Arvan Cloud Object Storage (AOS) API.
18561
18562 Properties:
18563
18564 • Config: endpoint
18565
18566 • Env Var: RCLONE_S3_ENDPOINT
18567
18568 • Provider: ArvanCloud
18569
18570 • Type: string
18571
18572 • Required: false
18573
18574 • Examples:
18575
18576 • "s3.ir-thr-at1.arvanstorage.com"
18577
18578 • The default endpoint - a good choice if you are unsure.
18579
18580 • Tehran Iran (Asiatech)
18581
18582 • "s3.ir-tbz-sh1.arvanstorage.com"
18583
18584 • Tabriz Iran (Shahriar)
18585
18586 --s3-endpoint
18587 Endpoint for IBM COS S3 API.
18588
18589 Specify if using an IBM COS On Premise.
18590
18591 Properties:
18592
18593 • Config: endpoint
18594
18595 • Env Var: RCLONE_S3_ENDPOINT
18596
18597 • Provider: IBMCOS
18598
18599 • Type: string
18600
18601 • Required: false
18602
18603 • Examples:
18604
18605 • "s3.us.cloud-object-storage.appdomain.cloud"
18606
18607 • US Cross Region Endpoint
18608
18609 • "s3.dal.us.cloud-object-storage.appdomain.cloud"
18610
18611 • US Cross Region Dallas Endpoint
18612
18613 • "s3.wdc.us.cloud-object-storage.appdomain.cloud"
18614
18615 • US Cross Region Washington DC Endpoint
18616
18617 • "s3.sjc.us.cloud-object-storage.appdomain.cloud"
18618
18619 • US Cross Region San Jose Endpoint
18620
18621 • "s3.private.us.cloud-object-storage.appdomain.cloud"
18622
18623 • US Cross Region Private Endpoint
18624
18625 • "s3.private.dal.us.cloud-object-storage.appdomain.cloud"
18626
18627 • US Cross Region Dallas Private Endpoint
18628
18629 • "s3.private.wdc.us.cloud-object-storage.appdomain.cloud"
18630
18631 • US Cross Region Washington DC Private Endpoint
18632
18633 • "s3.private.sjc.us.cloud-object-storage.appdomain.cloud"
18634
18635 • US Cross Region San Jose Private Endpoint
18636
18637 • "s3.us-east.cloud-object-storage.appdomain.cloud"
18638
18639 • US Region East Endpoint
18640
18641 • "s3.private.us-east.cloud-object-storage.appdomain.cloud"
18642
18643 • US Region East Private Endpoint
18644
18645 • "s3.us-south.cloud-object-storage.appdomain.cloud"
18646
18647 • US Region South Endpoint
18648
18649 • "s3.private.us-south.cloud-object-storage.appdomain.cloud"
18650
18651 • US Region South Private Endpoint
18652
18653 • "s3.eu.cloud-object-storage.appdomain.cloud"
18654
18655 • EU Cross Region Endpoint
18656
18657 • "s3.fra.eu.cloud-object-storage.appdomain.cloud"
18658
18659 • EU Cross Region Frankfurt Endpoint
18660
18661 • "s3.mil.eu.cloud-object-storage.appdomain.cloud"
18662
18663 • EU Cross Region Milan Endpoint
18664
18665 • "s3.ams.eu.cloud-object-storage.appdomain.cloud"
18666
18667 • EU Cross Region Amsterdam Endpoint
18668
18669 • "s3.private.eu.cloud-object-storage.appdomain.cloud"
18670
18671 • EU Cross Region Private Endpoint
18672
18673 • "s3.private.fra.eu.cloud-object-storage.appdomain.cloud"
18674
18675 • EU Cross Region Frankfurt Private Endpoint
18676
18677 • "s3.private.mil.eu.cloud-object-storage.appdomain.cloud"
18678
18679 • EU Cross Region Milan Private Endpoint
18680
18681 • "s3.private.ams.eu.cloud-object-storage.appdomain.cloud"
18682
18683 • EU Cross Region Amsterdam Private Endpoint
18684
18685 • "s3.eu-gb.cloud-object-storage.appdomain.cloud"
18686
18687 • Great Britain Endpoint
18688
18689 • "s3.private.eu-gb.cloud-object-storage.appdomain.cloud"
18690
18691 • Great Britain Private Endpoint
18692
18693 • "s3.eu-de.cloud-object-storage.appdomain.cloud"
18694
18695 • EU Region DE Endpoint
18696
18697 • "s3.private.eu-de.cloud-object-storage.appdomain.cloud"
18698
18699 • EU Region DE Private Endpoint
18700
18701 • "s3.ap.cloud-object-storage.appdomain.cloud"
18702
18703 • APAC Cross Regional Endpoint
18704
18705 • "s3.tok.ap.cloud-object-storage.appdomain.cloud"
18706
18707 • APAC Cross Regional Tokyo Endpoint
18708
18709 • "s3.hkg.ap.cloud-object-storage.appdomain.cloud"
18710
18711 • APAC Cross Regional HongKong Endpoint
18712
18713 • "s3.seo.ap.cloud-object-storage.appdomain.cloud"
18714
18715 • APAC Cross Regional Seoul Endpoint
18716
18717 • "s3.private.ap.cloud-object-storage.appdomain.cloud"
18718
18719 • APAC Cross Regional Private Endpoint
18720
18721 • "s3.private.tok.ap.cloud-object-storage.appdomain.cloud"
18722
18723 • APAC Cross Regional Tokyo Private Endpoint
18724
18725 • "s3.private.hkg.ap.cloud-object-storage.appdomain.cloud"
18726
18727 • APAC Cross Regional HongKong Private Endpoint
18728
18729 • "s3.private.seo.ap.cloud-object-storage.appdomain.cloud"
18730
18731 • APAC Cross Regional Seoul Private Endpoint
18732
18733 • "s3.jp-tok.cloud-object-storage.appdomain.cloud"
18734
18735 • APAC Region Japan Endpoint
18736
18737 • "s3.private.jp-tok.cloud-object-storage.appdomain.cloud"
18738
18739 • APAC Region Japan Private Endpoint
18740
18741 • "s3.au-syd.cloud-object-storage.appdomain.cloud"
18742
18743 • APAC Region Australia Endpoint
18744
18745 • "s3.private.au-syd.cloud-object-storage.appdomain.cloud"
18746
18747 • APAC Region Australia Private Endpoint
18748
18749 • "s3.ams03.cloud-object-storage.appdomain.cloud"
18750
18751 • Amsterdam Single Site Endpoint
18752
18753 • "s3.private.ams03.cloud-object-storage.appdomain.cloud"
18754
18755 • Amsterdam Single Site Private Endpoint
18756
18757 • "s3.che01.cloud-object-storage.appdomain.cloud"
18758
18759 • Chennai Single Site Endpoint
18760
18761 • "s3.private.che01.cloud-object-storage.appdomain.cloud"
18762
18763 • Chennai Single Site Private Endpoint
18764
18765 • "s3.mel01.cloud-object-storage.appdomain.cloud"
18766
18767 • Melbourne Single Site Endpoint
18768
18769 • "s3.private.mel01.cloud-object-storage.appdomain.cloud"
18770
18771 • Melbourne Single Site Private Endpoint
18772
18773 • "s3.osl01.cloud-object-storage.appdomain.cloud"
18774
18775 • Oslo Single Site Endpoint
18776
18777 • "s3.private.osl01.cloud-object-storage.appdomain.cloud"
18778
18779 • Oslo Single Site Private Endpoint
18780
18781 • "s3.tor01.cloud-object-storage.appdomain.cloud"
18782
18783 • Toronto Single Site Endpoint
18784
18785 • "s3.private.tor01.cloud-object-storage.appdomain.cloud"
18786
18787 • Toronto Single Site Private Endpoint
18788
18789 • "s3.seo01.cloud-object-storage.appdomain.cloud"
18790
18791 • Seoul Single Site Endpoint
18792
18793 • "s3.private.seo01.cloud-object-storage.appdomain.cloud"
18794
18795 • Seoul Single Site Private Endpoint
18796
18797 • "s3.mon01.cloud-object-storage.appdomain.cloud"
18798
18799 • Montreal Single Site Endpoint
18800
18801 • "s3.private.mon01.cloud-object-storage.appdomain.cloud"
18802
18803 • Montreal Single Site Private Endpoint
18804
18805 • "s3.mex01.cloud-object-storage.appdomain.cloud"
18806
18807 • Mexico Single Site Endpoint
18808
18809 • "s3.private.mex01.cloud-object-storage.appdomain.cloud"
18810
18811 • Mexico Single Site Private Endpoint
18812
18813 • "s3.sjc04.cloud-object-storage.appdomain.cloud"
18814
18815 • San Jose Single Site Endpoint
18816
18817 • "s3.private.sjc04.cloud-object-storage.appdomain.cloud"
18818
18819 • San Jose Single Site Private Endpoint
18820
18821 • "s3.mil01.cloud-object-storage.appdomain.cloud"
18822
18823 • Milan Single Site Endpoint
18824
18825 • "s3.private.mil01.cloud-object-storage.appdomain.cloud"
18826
18827 • Milan Single Site Private Endpoint
18828
18829 • "s3.hkg02.cloud-object-storage.appdomain.cloud"
18830
18831 • Hong Kong Single Site Endpoint
18832
18833 • "s3.private.hkg02.cloud-object-storage.appdomain.cloud"
18834
18835 • Hong Kong Single Site Private Endpoint
18836
18837 • "s3.par01.cloud-object-storage.appdomain.cloud"
18838
18839 • Paris Single Site Endpoint
18840
18841 • "s3.private.par01.cloud-object-storage.appdomain.cloud"
18842
18843 • Paris Single Site Private Endpoint
18844
18845 • "s3.sng01.cloud-object-storage.appdomain.cloud"
18846
18847 • Singapore Single Site Endpoint
18848
18849 • "s3.private.sng01.cloud-object-storage.appdomain.cloud"
18850
18851 • Singapore Single Site Private Endpoint
18852
18853 --s3-endpoint
18854 Endpoint for IONOS S3 Object Storage.
18855
18856 Specify the endpoint from the same region.
18857
18858 Properties:
18859
18860 • Config: endpoint
18861
18862 • Env Var: RCLONE_S3_ENDPOINT
18863
18864 • Provider: IONOS
18865
18866 • Type: string
18867
18868 • Required: false
18869
18870 • Examples:
18871
18872 • "s3-eu-central-1.ionoscloud.com"
18873
18874 • Frankfurt, Germany
18875
18876 • "s3-eu-central-2.ionoscloud.com"
18877
18878 • Berlin, Germany
18879
18880 • "s3-eu-south-2.ionoscloud.com"
18881
18882 • Logrono, Spain
18883
18884 --s3-endpoint
18885 Endpoint for OSS API.
18886
18887 Properties:
18888
18889 • Config: endpoint
18890
18891 • Env Var: RCLONE_S3_ENDPOINT
18892
18893 • Provider: Alibaba
18894
18895 • Type: string
18896
18897 • Required: false
18898
18899 • Examples:
18900
18901 • "oss-accelerate.aliyuncs.com"
18902
18903 • Global Accelerate
18904
18905 • "oss-accelerate-overseas.aliyuncs.com"
18906
18907 • Global Accelerate (outside mainland China)
18908
18909 • "oss-cn-hangzhou.aliyuncs.com"
18910
18911 • East China 1 (Hangzhou)
18912
18913 • "oss-cn-shanghai.aliyuncs.com"
18914
18915 • East China 2 (Shanghai)
18916
18917 • "oss-cn-qingdao.aliyuncs.com"
18918
18919 • North China 1 (Qingdao)
18920
18921 • "oss-cn-beijing.aliyuncs.com"
18922
18923 • North China 2 (Beijing)
18924
18925 • "oss-cn-zhangjiakou.aliyuncs.com"
18926
18927 • North China 3 (Zhangjiakou)
18928
18929 • "oss-cn-huhehaote.aliyuncs.com"
18930
18931 • North China 5 (Hohhot)
18932
18933 • "oss-cn-wulanchabu.aliyuncs.com"
18934
18935 • North China 6 (Ulanqab)
18936
18937 • "oss-cn-shenzhen.aliyuncs.com"
18938
18939 • South China 1 (Shenzhen)
18940
18941 • "oss-cn-heyuan.aliyuncs.com"
18942
18943 • South China 2 (Heyuan)
18944
18945 • "oss-cn-guangzhou.aliyuncs.com"
18946
18947 • South China 3 (Guangzhou)
18948
18949 • "oss-cn-chengdu.aliyuncs.com"
18950
18951 • West China 1 (Chengdu)
18952
18953 • "oss-cn-hongkong.aliyuncs.com"
18954
18955 • Hong Kong (Hong Kong)
18956
18957 • "oss-us-west-1.aliyuncs.com"
18958
18959 • US West 1 (Silicon Valley)
18960
18961 • "oss-us-east-1.aliyuncs.com"
18962
18963 • US East 1 (Virginia)
18964
18965 • "oss-ap-southeast-1.aliyuncs.com"
18966
18967 • Southeast Asia Southeast 1 (Singapore)
18968
18969 • "oss-ap-southeast-2.aliyuncs.com"
18970
18971 • Asia Pacific Southeast 2 (Sydney)
18972
18973 • "oss-ap-southeast-3.aliyuncs.com"
18974
18975 • Southeast Asia Southeast 3 (Kuala Lumpur)
18976
18977 • "oss-ap-southeast-5.aliyuncs.com"
18978
18979 • Asia Pacific Southeast 5 (Jakarta)
18980
18981 • "oss-ap-northeast-1.aliyuncs.com"
18982
18983 • Asia Pacific Northeast 1 (Japan)
18984
18985 • "oss-ap-south-1.aliyuncs.com"
18986
18987 • Asia Pacific South 1 (Mumbai)
18988
18989 • "oss-eu-central-1.aliyuncs.com"
18990
18991 • Central Europe 1 (Frankfurt)
18992
18993 • "oss-eu-west-1.aliyuncs.com"
18994
18995 • West Europe (London)
18996
18997 • "oss-me-east-1.aliyuncs.com"
18998
18999 • Middle East 1 (Dubai)
19000
19001 --s3-endpoint
19002 Endpoint for OBS API.
19003
19004 Properties:
19005
19006 • Config: endpoint
19007
19008 • Env Var: RCLONE_S3_ENDPOINT
19009
19010 • Provider: HuaweiOBS
19011
19012 • Type: string
19013
19014 • Required: false
19015
19016 • Examples:
19017
19018 • "obs.af-south-1.myhuaweicloud.com"
19019
19020 • AF-Johannesburg
19021
19022 • "obs.ap-southeast-2.myhuaweicloud.com"
19023
19024 • AP-Bangkok
19025
19026 • "obs.ap-southeast-3.myhuaweicloud.com"
19027
19028 • AP-Singapore
19029
19030 • "obs.cn-east-3.myhuaweicloud.com"
19031
19032 • CN East-Shanghai1
19033
19034 • "obs.cn-east-2.myhuaweicloud.com"
19035
19036 • CN East-Shanghai2
19037
19038 • "obs.cn-north-1.myhuaweicloud.com"
19039
19040 • CN North-Beijing1
19041
19042 • "obs.cn-north-4.myhuaweicloud.com"
19043
19044 • CN North-Beijing4
19045
19046 • "obs.cn-south-1.myhuaweicloud.com"
19047
19048 • CN South-Guangzhou
19049
19050 • "obs.ap-southeast-1.myhuaweicloud.com"
19051
19052 • CN-Hong Kong
19053
19054 • "obs.sa-argentina-1.myhuaweicloud.com"
19055
19056 • LA-Buenos Aires1
19057
19058 • "obs.sa-peru-1.myhuaweicloud.com"
19059
19060 • LA-Lima1
19061
19062 • "obs.na-mexico-1.myhuaweicloud.com"
19063
19064 • LA-Mexico City1
19065
19066 • "obs.sa-chile-1.myhuaweicloud.com"
19067
19068 • LA-Santiago2
19069
19070 • "obs.sa-brazil-1.myhuaweicloud.com"
19071
19072 • LA-Sao Paulo1
19073
19074 • "obs.ru-northwest-2.myhuaweicloud.com"
19075
19076 • RU-Moscow2
19077
19078 --s3-endpoint
19079 Endpoint for Scaleway Object Storage.
19080
19081 Properties:
19082
19083 • Config: endpoint
19084
19085 • Env Var: RCLONE_S3_ENDPOINT
19086
19087 • Provider: Scaleway
19088
19089 • Type: string
19090
19091 • Required: false
19092
19093 • Examples:
19094
19095 • "s3.nl-ams.scw.cloud"
19096
19097 • Amsterdam Endpoint
19098
19099 • "s3.fr-par.scw.cloud"
19100
19101 • Paris Endpoint
19102
19103 • "s3.pl-waw.scw.cloud"
19104
19105 • Warsaw Endpoint
19106
19107 --s3-endpoint
19108 Endpoint for StackPath Object Storage.
19109
19110 Properties:
19111
19112 • Config: endpoint
19113
19114 • Env Var: RCLONE_S3_ENDPOINT
19115
19116 • Provider: StackPath
19117
19118 • Type: string
19119
19120 • Required: false
19121
19122 • Examples:
19123
19124 • "s3.us-east-2.stackpathstorage.com"
19125
19126 • US East Endpoint
19127
19128 • "s3.us-west-1.stackpathstorage.com"
19129
19130 • US West Endpoint
19131
19132 • "s3.eu-central-1.stackpathstorage.com"
19133
19134 • EU Endpoint
19135
19136 --s3-endpoint
19137 Endpoint of the Shared Gateway.
19138
19139 Properties:
19140
19141 • Config: endpoint
19142
19143 • Env Var: RCLONE_S3_ENDPOINT
19144
19145 • Provider: Storj
19146
19147 • Type: string
19148
19149 • Required: false
19150
19151 • Examples:
19152
19153 • "gateway.eu1.storjshare.io"
19154
19155 • EU1 Shared Gateway
19156
19157 • "gateway.us1.storjshare.io"
19158
19159 • US1 Shared Gateway
19160
19161 • "gateway.ap1.storjshare.io"
19162
19163 • Asia-Pacific Shared Gateway
19164
19165 --s3-endpoint
19166 Endpoint for Tencent COS API.
19167
19168 Properties:
19169
19170 • Config: endpoint
19171
19172 • Env Var: RCLONE_S3_ENDPOINT
19173
19174 • Provider: TencentCOS
19175
19176 • Type: string
19177
19178 • Required: false
19179
19180 • Examples:
19181
19182 • "cos.ap-beijing.myqcloud.com"
19183
19184 • Beijing Region
19185
19186 • "cos.ap-nanjing.myqcloud.com"
19187
19188 • Nanjing Region
19189
19190 • "cos.ap-shanghai.myqcloud.com"
19191
19192 • Shanghai Region
19193
19194 • "cos.ap-guangzhou.myqcloud.com"
19195
19196 • Guangzhou Region
19197
19198 • "cos.ap-nanjing.myqcloud.com"
19199
19200 • Nanjing Region
19201
19202 • "cos.ap-chengdu.myqcloud.com"
19203
19204 • Chengdu Region
19205
19206 • "cos.ap-chongqing.myqcloud.com"
19207
19208 • Chongqing Region
19209
19210 • "cos.ap-hongkong.myqcloud.com"
19211
19212 • Hong Kong (China) Region
19213
19214 • "cos.ap-singapore.myqcloud.com"
19215
19216 • Singapore Region
19217
19218 • "cos.ap-mumbai.myqcloud.com"
19219
19220 • Mumbai Region
19221
19222 • "cos.ap-seoul.myqcloud.com"
19223
19224 • Seoul Region
19225
19226 • "cos.ap-bangkok.myqcloud.com"
19227
19228 • Bangkok Region
19229
19230 • "cos.ap-tokyo.myqcloud.com"
19231
19232 • Tokyo Region
19233
19234 • "cos.na-siliconvalley.myqcloud.com"
19235
19236 • Silicon Valley Region
19237
19238 • "cos.na-ashburn.myqcloud.com"
19239
19240 • Virginia Region
19241
19242 • "cos.na-toronto.myqcloud.com"
19243
19244 • Toronto Region
19245
19246 • "cos.eu-frankfurt.myqcloud.com"
19247
19248 • Frankfurt Region
19249
19250 • "cos.eu-moscow.myqcloud.com"
19251
19252 • Moscow Region
19253
19254 • "cos.accelerate.myqcloud.com"
19255
19256 • Use Tencent COS Accelerate Endpoint
19257
19258 --s3-endpoint
19259 Endpoint for RackCorp Object Storage.
19260
19261 Properties:
19262
19263 • Config: endpoint
19264
19265 • Env Var: RCLONE_S3_ENDPOINT
19266
19267 • Provider: RackCorp
19268
19269 • Type: string
19270
19271 • Required: false
19272
19273 • Examples:
19274
19275 • "s3.rackcorp.com"
19276
19277 • Global (AnyCast) Endpoint
19278
19279 • "au.s3.rackcorp.com"
19280
19281 • Australia (Anycast) Endpoint
19282
19283 • "au-nsw.s3.rackcorp.com"
19284
19285 • Sydney (Australia) Endpoint
19286
19287 • "au-qld.s3.rackcorp.com"
19288
19289 • Brisbane (Australia) Endpoint
19290
19291 • "au-vic.s3.rackcorp.com"
19292
19293 • Melbourne (Australia) Endpoint
19294
19295 • "au-wa.s3.rackcorp.com"
19296
19297 • Perth (Australia) Endpoint
19298
19299 • "ph.s3.rackcorp.com"
19300
19301 • Manila (Philippines) Endpoint
19302
19303 • "th.s3.rackcorp.com"
19304
19305 • Bangkok (Thailand) Endpoint
19306
19307 • "hk.s3.rackcorp.com"
19308
19309 • HK (Hong Kong) Endpoint
19310
19311 • "mn.s3.rackcorp.com"
19312
19313 • Ulaanbaatar (Mongolia) Endpoint
19314
19315 • "kg.s3.rackcorp.com"
19316
19317 • Bishkek (Kyrgyzstan) Endpoint
19318
19319 • "id.s3.rackcorp.com"
19320
19321 • Jakarta (Indonesia) Endpoint
19322
19323 • "jp.s3.rackcorp.com"
19324
19325 • Tokyo (Japan) Endpoint
19326
19327 • "sg.s3.rackcorp.com"
19328
19329 • SG (Singapore) Endpoint
19330
19331 • "de.s3.rackcorp.com"
19332
19333 • Frankfurt (Germany) Endpoint
19334
19335 • "us.s3.rackcorp.com"
19336
19337 • USA (AnyCast) Endpoint
19338
19339 • "us-east-1.s3.rackcorp.com"
19340
19341 • New York (USA) Endpoint
19342
19343 • "us-west-1.s3.rackcorp.com"
19344
19345 • Freemont (USA) Endpoint
19346
19347 • "nz.s3.rackcorp.com"
19348
19349 • Auckland (New Zealand) Endpoint
19350
19351 --s3-endpoint
19352 Endpoint for Qiniu Object Storage.
19353
19354 Properties:
19355
19356 • Config: endpoint
19357
19358 • Env Var: RCLONE_S3_ENDPOINT
19359
19360 • Provider: Qiniu
19361
19362 • Type: string
19363
19364 • Required: false
19365
19366 • Examples:
19367
19368 • "s3-cn-east-1.qiniucs.com"
19369
19370 • East China Endpoint 1
19371
19372 • "s3-cn-east-2.qiniucs.com"
19373
19374 • East China Endpoint 2
19375
19376 • "s3-cn-north-1.qiniucs.com"
19377
19378 • North China Endpoint 1
19379
19380 • "s3-cn-south-1.qiniucs.com"
19381
19382 • South China Endpoint 1
19383
19384 • "s3-us-north-1.qiniucs.com"
19385
19386 • North America Endpoint 1
19387
19388 • "s3-ap-southeast-1.qiniucs.com"
19389
19390 • Southeast Asia Endpoint 1
19391
19392 • "s3-ap-northeast-1.qiniucs.com"
19393
19394 • Northeast Asia Endpoint 1
19395
19396 --s3-endpoint
19397 Endpoint for S3 API.
19398
19399 Required when using an S3 clone.
19400
19401 Properties:
19402
19403 • Config: endpoint
19404
19405 • Env Var: RCLONE_S3_ENDPOINT
19406
19407 • Provider: !AWS,IBMCOS,IDrive,IONOS,TencentCOS,HuaweiOBS,Alibaba,Chi‐
19408 naMobile,ArvanCloud,Scaleway,StackPath,Storj,RackCorp,Qiniu
19409
19410 • Type: string
19411
19412 • Required: false
19413
19414 • Examples:
19415
19416 • "objects-us-east-1.dream.io"
19417
19418 • Dream Objects endpoint
19419
19420 • "nyc3.digitaloceanspaces.com"
19421
19422 • Digital Ocean Spaces New York 3
19423
19424 • "ams3.digitaloceanspaces.com"
19425
19426 • Digital Ocean Spaces Amsterdam 3
19427
19428 • "sgp1.digitaloceanspaces.com"
19429
19430 • Digital Ocean Spaces Singapore 1
19431
19432 • "localhost:8333"
19433
19434 • SeaweedFS S3 localhost
19435
19436 • "s3.us-east-1.lyvecloud.seagate.com"
19437
19438 • Seagate Lyve Cloud US East 1 (Virginia)
19439
19440 • "s3.us-west-1.lyvecloud.seagate.com"
19441
19442 • Seagate Lyve Cloud US West 1 (California)
19443
19444 • "s3.ap-southeast-1.lyvecloud.seagate.com"
19445
19446 • Seagate Lyve Cloud AP Southeast 1 (Singapore)
19447
19448 • "s3.wasabisys.com"
19449
19450 • Wasabi US East 1 (N. Virginia)
19451
19452 • "s3.us-east-2.wasabisys.com"
19453
19454 • Wasabi US East 2 (N. Virginia)
19455
19456 • "s3.us-central-1.wasabisys.com"
19457
19458 • Wasabi US Central 1 (Texas)
19459
19460 • "s3.us-west-1.wasabisys.com"
19461
19462 • Wasabi US West 1 (Oregon)
19463
19464 • "s3.ca-central-1.wasabisys.com"
19465
19466 • Wasabi CA Central 1 (Toronto)
19467
19468 • "s3.eu-central-1.wasabisys.com"
19469
19470 • Wasabi EU Central 1 (Amsterdam)
19471
19472 • "s3.eu-central-2.wasabisys.com"
19473
19474 • Wasabi EU Central 2 (Frankfurt)
19475
19476 • "s3.eu-west-1.wasabisys.com"
19477
19478 • Wasabi EU West 1 (London)
19479
19480 • "s3.eu-west-2.wasabisys.com"
19481
19482 • Wasabi EU West 2 (Paris)
19483
19484 • "s3.ap-northeast-1.wasabisys.com"
19485
19486 • Wasabi AP Northeast 1 (Tokyo) endpoint
19487
19488 • "s3.ap-northeast-2.wasabisys.com"
19489
19490 • Wasabi AP Northeast 2 (Osaka) endpoint
19491
19492 • "s3.ap-southeast-1.wasabisys.com"
19493
19494 • Wasabi AP Southeast 1 (Singapore)
19495
19496 • "s3.ap-southeast-2.wasabisys.com"
19497
19498 • Wasabi AP Southeast 2 (Sydney)
19499
19500 • "s3.ir-thr-at1.arvanstorage.com"
19501
19502 • ArvanCloud Tehran Iran (Asiatech) endpoint
19503
19504 --s3-location-constraint
19505 Location constraint - must be set to match the Region.
19506
19507 Used when creating buckets only.
19508
19509 Properties:
19510
19511 • Config: location_constraint
19512
19513 • Env Var: RCLONE_S3_LOCATION_CONSTRAINT
19514
19515 • Provider: AWS
19516
19517 • Type: string
19518
19519 • Required: false
19520
19521 • Examples:
19522
19523 • ""
19524
19525 • Empty for US Region, Northern Virginia, or Pacific Northwest
19526
19527 • "us-east-2"
19528
19529 • US East (Ohio) Region
19530
19531 • "us-west-1"
19532
19533 • US West (Northern California) Region
19534
19535 • "us-west-2"
19536
19537 • US West (Oregon) Region
19538
19539 • "ca-central-1"
19540
19541 • Canada (Central) Region
19542
19543 • "eu-west-1"
19544
19545 • EU (Ireland) Region
19546
19547 • "eu-west-2"
19548
19549 • EU (London) Region
19550
19551 • "eu-west-3"
19552
19553 • EU (Paris) Region
19554
19555 • "eu-north-1"
19556
19557 • EU (Stockholm) Region
19558
19559 • "eu-south-1"
19560
19561 • EU (Milan) Region
19562
19563 • "EU"
19564
19565 • EU Region
19566
19567 • "ap-southeast-1"
19568
19569 • Asia Pacific (Singapore) Region
19570
19571 • "ap-southeast-2"
19572
19573 • Asia Pacific (Sydney) Region
19574
19575 • "ap-northeast-1"
19576
19577 • Asia Pacific (Tokyo) Region
19578
19579 • "ap-northeast-2"
19580
19581 • Asia Pacific (Seoul) Region
19582
19583 • "ap-northeast-3"
19584
19585 • Asia Pacific (Osaka-Local) Region
19586
19587 • "ap-south-1"
19588
19589 • Asia Pacific (Mumbai) Region
19590
19591 • "ap-east-1"
19592
19593 • Asia Pacific (Hong Kong) Region
19594
19595 • "sa-east-1"
19596
19597 • South America (Sao Paulo) Region
19598
19599 • "me-south-1"
19600
19601 • Middle East (Bahrain) Region
19602
19603 • "af-south-1"
19604
19605 • Africa (Cape Town) Region
19606
19607 • "cn-north-1"
19608
19609 • China (Beijing) Region
19610
19611 • "cn-northwest-1"
19612
19613 • China (Ningxia) Region
19614
19615 • "us-gov-east-1"
19616
19617 • AWS GovCloud (US-East) Region
19618
19619 • "us-gov-west-1"
19620
19621 • AWS GovCloud (US) Region
19622
19623 --s3-location-constraint
19624 Location constraint - must match endpoint.
19625
19626 Used when creating buckets only.
19627
19628 Properties:
19629
19630 • Config: location_constraint
19631
19632 • Env Var: RCLONE_S3_LOCATION_CONSTRAINT
19633
19634 • Provider: ChinaMobile
19635
19636 • Type: string
19637
19638 • Required: false
19639
19640 • Examples:
19641
19642 • "wuxi1"
19643
19644 • East China (Suzhou)
19645
19646 • "jinan1"
19647
19648 • East China (Jinan)
19649
19650 • "ningbo1"
19651
19652 • East China (Hangzhou)
19653
19654 • "shanghai1"
19655
19656 • East China (Shanghai-1)
19657
19658 • "zhengzhou1"
19659
19660 • Central China (Zhengzhou)
19661
19662 • "hunan1"
19663
19664 • Central China (Changsha-1)
19665
19666 • "zhuzhou1"
19667
19668 • Central China (Changsha-2)
19669
19670 • "guangzhou1"
19671
19672 • South China (Guangzhou-2)
19673
19674 • "dongguan1"
19675
19676 • South China (Guangzhou-3)
19677
19678 • "beijing1"
19679
19680 • North China (Beijing-1)
19681
19682 • "beijing2"
19683
19684 • North China (Beijing-2)
19685
19686 • "beijing4"
19687
19688 • North China (Beijing-3)
19689
19690 • "huhehaote1"
19691
19692 • North China (Huhehaote)
19693
19694 • "chengdu1"
19695
19696 • Southwest China (Chengdu)
19697
19698 • "chongqing1"
19699
19700 • Southwest China (Chongqing)
19701
19702 • "guiyang1"
19703
19704 • Southwest China (Guiyang)
19705
19706 • "xian1"
19707
19708 • Nouthwest China (Xian)
19709
19710 • "yunnan"
19711
19712 • Yunnan China (Kunming)
19713
19714 • "yunnan2"
19715
19716 • Yunnan China (Kunming-2)
19717
19718 • "tianjin1"
19719
19720 • Tianjin China (Tianjin)
19721
19722 • "jilin1"
19723
19724 • Jilin China (Changchun)
19725
19726 • "hubei1"
19727
19728 • Hubei China (Xiangyan)
19729
19730 • "jiangxi1"
19731
19732 • Jiangxi China (Nanchang)
19733
19734 • "gansu1"
19735
19736 • Gansu China (Lanzhou)
19737
19738 • "shanxi1"
19739
19740 • Shanxi China (Taiyuan)
19741
19742 • "liaoning1"
19743
19744 • Liaoning China (Shenyang)
19745
19746 • "hebei1"
19747
19748 • Hebei China (Shijiazhuang)
19749
19750 • "fujian1"
19751
19752 • Fujian China (Xiamen)
19753
19754 • "guangxi1"
19755
19756 • Guangxi China (Nanning)
19757
19758 • "anhui1"
19759
19760 • Anhui China (Huainan)
19761
19762 --s3-location-constraint
19763 Location constraint - must match endpoint.
19764
19765 Used when creating buckets only.
19766
19767 Properties:
19768
19769 • Config: location_constraint
19770
19771 • Env Var: RCLONE_S3_LOCATION_CONSTRAINT
19772
19773 • Provider: ArvanCloud
19774
19775 • Type: string
19776
19777 • Required: false
19778
19779 • Examples:
19780
19781 • "ir-thr-at1"
19782
19783 • Tehran Iran (Asiatech)
19784
19785 • "ir-tbz-sh1"
19786
19787 • Tabriz Iran (Shahriar)
19788
19789 --s3-location-constraint
19790 Location constraint - must match endpoint when using IBM Cloud Public.
19791
19792 For on-prem COS, do not make a selection from this list, hit enter.
19793
19794 Properties:
19795
19796 • Config: location_constraint
19797
19798 • Env Var: RCLONE_S3_LOCATION_CONSTRAINT
19799
19800 • Provider: IBMCOS
19801
19802 • Type: string
19803
19804 • Required: false
19805
19806 • Examples:
19807
19808 • "us-standard"
19809
19810 • US Cross Region Standard
19811
19812 • "us-vault"
19813
19814 • US Cross Region Vault
19815
19816 • "us-cold"
19817
19818 • US Cross Region Cold
19819
19820 • "us-flex"
19821
19822 • US Cross Region Flex
19823
19824 • "us-east-standard"
19825
19826 • US East Region Standard
19827
19828 • "us-east-vault"
19829
19830 • US East Region Vault
19831
19832 • "us-east-cold"
19833
19834 • US East Region Cold
19835
19836 • "us-east-flex"
19837
19838 • US East Region Flex
19839
19840 • "us-south-standard"
19841
19842 • US South Region Standard
19843
19844 • "us-south-vault"
19845
19846 • US South Region Vault
19847
19848 • "us-south-cold"
19849
19850 • US South Region Cold
19851
19852 • "us-south-flex"
19853
19854 • US South Region Flex
19855
19856 • "eu-standard"
19857
19858 • EU Cross Region Standard
19859
19860 • "eu-vault"
19861
19862 • EU Cross Region Vault
19863
19864 • "eu-cold"
19865
19866 • EU Cross Region Cold
19867
19868 • "eu-flex"
19869
19870 • EU Cross Region Flex
19871
19872 • "eu-gb-standard"
19873
19874 • Great Britain Standard
19875
19876 • "eu-gb-vault"
19877
19878 • Great Britain Vault
19879
19880 • "eu-gb-cold"
19881
19882 • Great Britain Cold
19883
19884 • "eu-gb-flex"
19885
19886 • Great Britain Flex
19887
19888 • "ap-standard"
19889
19890 • APAC Standard
19891
19892 • "ap-vault"
19893
19894 • APAC Vault
19895
19896 • "ap-cold"
19897
19898 • APAC Cold
19899
19900 • "ap-flex"
19901
19902 • APAC Flex
19903
19904 • "mel01-standard"
19905
19906 • Melbourne Standard
19907
19908 • "mel01-vault"
19909
19910 • Melbourne Vault
19911
19912 • "mel01-cold"
19913
19914 • Melbourne Cold
19915
19916 • "mel01-flex"
19917
19918 • Melbourne Flex
19919
19920 • "tor01-standard"
19921
19922 • Toronto Standard
19923
19924 • "tor01-vault"
19925
19926 • Toronto Vault
19927
19928 • "tor01-cold"
19929
19930 • Toronto Cold
19931
19932 • "tor01-flex"
19933
19934 • Toronto Flex
19935
19936 --s3-location-constraint
19937 Location constraint - the location where your bucket will be located
19938 and your data stored.
19939
19940 Properties:
19941
19942 • Config: location_constraint
19943
19944 • Env Var: RCLONE_S3_LOCATION_CONSTRAINT
19945
19946 • Provider: RackCorp
19947
19948 • Type: string
19949
19950 • Required: false
19951
19952 • Examples:
19953
19954 • "global"
19955
19956 • Global CDN Region
19957
19958 • "au"
19959
19960 • Australia (All locations)
19961
19962 • "au-nsw"
19963
19964 • NSW (Australia) Region
19965
19966 • "au-qld"
19967
19968 • QLD (Australia) Region
19969
19970 • "au-vic"
19971
19972 • VIC (Australia) Region
19973
19974 • "au-wa"
19975
19976 • Perth (Australia) Region
19977
19978 • "ph"
19979
19980 • Manila (Philippines) Region
19981
19982 • "th"
19983
19984 • Bangkok (Thailand) Region
19985
19986 • "hk"
19987
19988 • HK (Hong Kong) Region
19989
19990 • "mn"
19991
19992 • Ulaanbaatar (Mongolia) Region
19993
19994 • "kg"
19995
19996 • Bishkek (Kyrgyzstan) Region
19997
19998 • "id"
19999
20000 • Jakarta (Indonesia) Region
20001
20002 • "jp"
20003
20004 • Tokyo (Japan) Region
20005
20006 • "sg"
20007
20008 • SG (Singapore) Region
20009
20010 • "de"
20011
20012 • Frankfurt (Germany) Region
20013
20014 • "us"
20015
20016 • USA (AnyCast) Region
20017
20018 • "us-east-1"
20019
20020 • New York (USA) Region
20021
20022 • "us-west-1"
20023
20024 • Freemont (USA) Region
20025
20026 • "nz"
20027
20028 • Auckland (New Zealand) Region
20029
20030 --s3-location-constraint
20031 Location constraint - must be set to match the Region.
20032
20033 Used when creating buckets only.
20034
20035 Properties:
20036
20037 • Config: location_constraint
20038
20039 • Env Var: RCLONE_S3_LOCATION_CONSTRAINT
20040
20041 • Provider: Qiniu
20042
20043 • Type: string
20044
20045 • Required: false
20046
20047 • Examples:
20048
20049 • "cn-east-1"
20050
20051 • East China Region 1
20052
20053 • "cn-east-2"
20054
20055 • East China Region 2
20056
20057 • "cn-north-1"
20058
20059 • North China Region 1
20060
20061 • "cn-south-1"
20062
20063 • South China Region 1
20064
20065 • "us-north-1"
20066
20067 • North America Region 1
20068
20069 • "ap-southeast-1"
20070
20071 • Southeast Asia Region 1
20072
20073 • "ap-northeast-1"
20074
20075 • Northeast Asia Region 1
20076
20077 --s3-location-constraint
20078 Location constraint - must be set to match the Region.
20079
20080 Leave blank if not sure. Used when creating buckets only.
20081
20082 Properties:
20083
20084 • Config: location_constraint
20085
20086 • Env Var: RCLONE_S3_LOCATION_CONSTRAINT
20087
20088 • Provider: !AWS,Alibaba,HuaweiOBS,ChinaMobile,Cloudflare,IBM‐
20089 COS,IDrive,IONOS,ArvanCloud,Qiniu,RackCorp,Scaleway,Stack‐
20090 Path,Storj,TencentCOS
20091
20092 • Type: string
20093
20094 • Required: false
20095
20096 --s3-acl
20097 Canned ACL used when creating buckets and storing or copying objects.
20098
20099 This ACL is used for creating objects and if bucket_acl isn't set, for
20100 creating buckets too.
20101
20102 For more info visit https://docs.aws.amazon.com/AmazonS3/lat‐
20103 est/dev/acl-overview.html#canned-acl
20104
20105 Note that this ACL is applied when server-side copying objects as S3
20106 doesn't copy the ACL from the source but rather writes a fresh one.
20107
20108 Properties:
20109
20110 • Config: acl
20111
20112 • Env Var: RCLONE_S3_ACL
20113
20114 • Provider: !Storj,Cloudflare
20115
20116 • Type: string
20117
20118 • Required: false
20119
20120 • Examples:
20121
20122 • "default"
20123
20124 • Owner gets Full_CONTROL.
20125
20126 • No one else has access rights (default).
20127
20128 • "private"
20129
20130 • Owner gets FULL_CONTROL.
20131
20132 • No one else has access rights (default).
20133
20134 • "public-read"
20135
20136 • Owner gets FULL_CONTROL.
20137
20138 • The AllUsers group gets READ access.
20139
20140 • "public-read-write"
20141
20142 • Owner gets FULL_CONTROL.
20143
20144 • The AllUsers group gets READ and WRITE access.
20145
20146 • Granting this on a bucket is generally not recommended.
20147
20148 • "authenticated-read"
20149
20150 • Owner gets FULL_CONTROL.
20151
20152 • The AuthenticatedUsers group gets READ access.
20153
20154 • "bucket-owner-read"
20155
20156 • Object owner gets FULL_CONTROL.
20157
20158 • Bucket owner gets READ access.
20159
20160 • If you specify this canned ACL when creating a bucket, Amazon S3
20161 ignores it.
20162
20163 • "bucket-owner-full-control"
20164
20165 • Both the object owner and the bucket owner get FULL_CONTROL over
20166 the object.
20167
20168 • If you specify this canned ACL when creating a bucket, Amazon S3
20169 ignores it.
20170
20171 • "private"
20172
20173 • Owner gets FULL_CONTROL.
20174
20175 • No one else has access rights (default).
20176
20177 • This acl is available on IBM Cloud (Infra), IBM Cloud (Storage),
20178 On-Premise COS.
20179
20180 • "public-read"
20181
20182 • Owner gets FULL_CONTROL.
20183
20184 • The AllUsers group gets READ access.
20185
20186 • This acl is available on IBM Cloud (Infra), IBM Cloud (Storage),
20187 On-Premise IBM COS.
20188
20189 • "public-read-write"
20190
20191 • Owner gets FULL_CONTROL.
20192
20193 • The AllUsers group gets READ and WRITE access.
20194
20195 • This acl is available on IBM Cloud (Infra), On-Premise IBM COS.
20196
20197 • "authenticated-read"
20198
20199 • Owner gets FULL_CONTROL.
20200
20201 • The AuthenticatedUsers group gets READ access.
20202
20203 • Not supported on Buckets.
20204
20205 • This acl is available on IBM Cloud (Infra) and On-Premise IBM
20206 COS.
20207
20208 --s3-server-side-encryption
20209 The server-side encryption algorithm used when storing this object in
20210 S3.
20211
20212 Properties:
20213
20214 • Config: server_side_encryption
20215
20216 • Env Var: RCLONE_S3_SERVER_SIDE_ENCRYPTION
20217
20218 • Provider: AWS,Ceph,ChinaMobile,Minio
20219
20220 • Type: string
20221
20222 • Required: false
20223
20224 • Examples:
20225
20226 • ""
20227
20228 • None
20229
20230 • "AES256"
20231
20232 • AES256
20233
20234 • "aws:kms"
20235
20236 • aws:kms
20237
20238 --s3-sse-kms-key-id
20239 If using KMS ID you must provide the ARN of Key.
20240
20241 Properties:
20242
20243 • Config: sse_kms_key_id
20244
20245 • Env Var: RCLONE_S3_SSE_KMS_KEY_ID
20246
20247 • Provider: AWS,Ceph,Minio
20248
20249 • Type: string
20250
20251 • Required: false
20252
20253 • Examples:
20254
20255 • ""
20256
20257 • None
20258
20259 • "arn:aws:kms:us-east-1:*"
20260
20261 • arn:aws:kms:*
20262
20263 --s3-storage-class
20264 The storage class to use when storing new objects in S3.
20265
20266 Properties:
20267
20268 • Config: storage_class
20269
20270 • Env Var: RCLONE_S3_STORAGE_CLASS
20271
20272 • Provider: AWS
20273
20274 • Type: string
20275
20276 • Required: false
20277
20278 • Examples:
20279
20280 • ""
20281
20282 • Default
20283
20284 • "STANDARD"
20285
20286 • Standard storage class
20287
20288 • "REDUCED_REDUNDANCY"
20289
20290 • Reduced redundancy storage class
20291
20292 • "STANDARD_IA"
20293
20294 • Standard Infrequent Access storage class
20295
20296 • "ONEZONE_IA"
20297
20298 • One Zone Infrequent Access storage class
20299
20300 • "GLACIER"
20301
20302 • Glacier storage class
20303
20304 • "DEEP_ARCHIVE"
20305
20306 • Glacier Deep Archive storage class
20307
20308 • "INTELLIGENT_TIERING"
20309
20310 • Intelligent-Tiering storage class
20311
20312 • "GLACIER_IR"
20313
20314 • Glacier Instant Retrieval storage class
20315
20316 --s3-storage-class
20317 The storage class to use when storing new objects in OSS.
20318
20319 Properties:
20320
20321 • Config: storage_class
20322
20323 • Env Var: RCLONE_S3_STORAGE_CLASS
20324
20325 • Provider: Alibaba
20326
20327 • Type: string
20328
20329 • Required: false
20330
20331 • Examples:
20332
20333 • ""
20334
20335 • Default
20336
20337 • "STANDARD"
20338
20339 • Standard storage class
20340
20341 • "GLACIER"
20342
20343 • Archive storage mode
20344
20345 • "STANDARD_IA"
20346
20347 • Infrequent access storage mode
20348
20349 --s3-storage-class
20350 The storage class to use when storing new objects in ChinaMobile.
20351
20352 Properties:
20353
20354 • Config: storage_class
20355
20356 • Env Var: RCLONE_S3_STORAGE_CLASS
20357
20358 • Provider: ChinaMobile
20359
20360 • Type: string
20361
20362 • Required: false
20363
20364 • Examples:
20365
20366 • ""
20367
20368 • Default
20369
20370 • "STANDARD"
20371
20372 • Standard storage class
20373
20374 • "GLACIER"
20375
20376 • Archive storage mode
20377
20378 • "STANDARD_IA"
20379
20380 • Infrequent access storage mode
20381
20382 --s3-storage-class
20383 The storage class to use when storing new objects in ArvanCloud.
20384
20385 Properties:
20386
20387 • Config: storage_class
20388
20389 • Env Var: RCLONE_S3_STORAGE_CLASS
20390
20391 • Provider: ArvanCloud
20392
20393 • Type: string
20394
20395 • Required: false
20396
20397 • Examples:
20398
20399 • "STANDARD"
20400
20401 • Standard storage class
20402
20403 --s3-storage-class
20404 The storage class to use when storing new objects in Tencent COS.
20405
20406 Properties:
20407
20408 • Config: storage_class
20409
20410 • Env Var: RCLONE_S3_STORAGE_CLASS
20411
20412 • Provider: TencentCOS
20413
20414 • Type: string
20415
20416 • Required: false
20417
20418 • Examples:
20419
20420 • ""
20421
20422 • Default
20423
20424 • "STANDARD"
20425
20426 • Standard storage class
20427
20428 • "ARCHIVE"
20429
20430 • Archive storage mode
20431
20432 • "STANDARD_IA"
20433
20434 • Infrequent access storage mode
20435
20436 --s3-storage-class
20437 The storage class to use when storing new objects in S3.
20438
20439 Properties:
20440
20441 • Config: storage_class
20442
20443 • Env Var: RCLONE_S3_STORAGE_CLASS
20444
20445 • Provider: Scaleway
20446
20447 • Type: string
20448
20449 • Required: false
20450
20451 • Examples:
20452
20453 • ""
20454
20455 • Default.
20456
20457 • "STANDARD"
20458
20459 • The Standard class for any upload.
20460
20461 • Suitable for on-demand content like streaming or CDN.
20462
20463 • "GLACIER"
20464
20465 • Archived storage.
20466
20467 • Prices are lower, but it needs to be restored first to be ac‐
20468 cessed.
20469
20470 --s3-storage-class
20471 The storage class to use when storing new objects in Qiniu.
20472
20473 Properties:
20474
20475 • Config: storage_class
20476
20477 • Env Var: RCLONE_S3_STORAGE_CLASS
20478
20479 • Provider: Qiniu
20480
20481 • Type: string
20482
20483 • Required: false
20484
20485 • Examples:
20486
20487 • "STANDARD"
20488
20489 • Standard storage class
20490
20491 • "LINE"
20492
20493 • Infrequent access storage mode
20494
20495 • "GLACIER"
20496
20497 • Archive storage mode
20498
20499 • "DEEP_ARCHIVE"
20500
20501 • Deep archive storage mode
20502
20503 Advanced options
20504 Here are the Advanced options specific to s3 (Amazon S3 Compliant Stor‐
20505 age Providers including AWS, Alibaba, Ceph, China Mobile, Cloudflare,
20506 ArvanCloud, Digital Ocean, Dreamhost, Huawei OBS, IBM COS, IDrive e2,
20507 IONOS Cloud, Lyve Cloud, Minio, Netease, RackCorp, Scaleway, SeaweedFS,
20508 StackPath, Storj, Tencent COS, Qiniu and Wasabi).
20509
20510 --s3-bucket-acl
20511 Canned ACL used when creating buckets.
20512
20513 For more info visit https://docs.aws.amazon.com/AmazonS3/lat‐
20514 est/dev/acl-overview.html#canned-acl
20515
20516 Note that this ACL is applied when only when creating buckets. If it
20517 isn't set then "acl" is used instead.
20518
20519 Properties:
20520
20521 • Config: bucket_acl
20522
20523 • Env Var: RCLONE_S3_BUCKET_ACL
20524
20525 • Type: string
20526
20527 • Required: false
20528
20529 • Examples:
20530
20531 • "private"
20532
20533 • Owner gets FULL_CONTROL.
20534
20535 • No one else has access rights (default).
20536
20537 • "public-read"
20538
20539 • Owner gets FULL_CONTROL.
20540
20541 • The AllUsers group gets READ access.
20542
20543 • "public-read-write"
20544
20545 • Owner gets FULL_CONTROL.
20546
20547 • The AllUsers group gets READ and WRITE access.
20548
20549 • Granting this on a bucket is generally not recommended.
20550
20551 • "authenticated-read"
20552
20553 • Owner gets FULL_CONTROL.
20554
20555 • The AuthenticatedUsers group gets READ access.
20556
20557 --s3-requester-pays
20558 Enables requester pays option when interacting with S3 bucket.
20559
20560 Properties:
20561
20562 • Config: requester_pays
20563
20564 • Env Var: RCLONE_S3_REQUESTER_PAYS
20565
20566 • Provider: AWS
20567
20568 • Type: bool
20569
20570 • Default: false
20571
20572 --s3-sse-customer-algorithm
20573 If using SSE-C, the server-side encryption algorithm used when storing
20574 this object in S3.
20575
20576 Properties:
20577
20578 • Config: sse_customer_algorithm
20579
20580 • Env Var: RCLONE_S3_SSE_CUSTOMER_ALGORITHM
20581
20582 • Provider: AWS,Ceph,ChinaMobile,Minio
20583
20584 • Type: string
20585
20586 • Required: false
20587
20588 • Examples:
20589
20590 • ""
20591
20592 • None
20593
20594 • "AES256"
20595
20596 • AES256
20597
20598 --s3-sse-customer-key
20599 To use SSE-C you may provide the secret encryption key used to en‐
20600 crypt/decrypt your data.
20601
20602 Alternatively you can provide --sse-customer-key-base64.
20603
20604 Properties:
20605
20606 • Config: sse_customer_key
20607
20608 • Env Var: RCLONE_S3_SSE_CUSTOMER_KEY
20609
20610 • Provider: AWS,Ceph,ChinaMobile,Minio
20611
20612 • Type: string
20613
20614 • Required: false
20615
20616 • Examples:
20617
20618 • ""
20619
20620 • None
20621
20622 --s3-sse-customer-key-base64
20623 If using SSE-C you must provide the secret encryption key encoded in
20624 base64 format to encrypt/decrypt your data.
20625
20626 Alternatively you can provide --sse-customer-key.
20627
20628 Properties:
20629
20630 • Config: sse_customer_key_base64
20631
20632 • Env Var: RCLONE_S3_SSE_CUSTOMER_KEY_BASE64
20633
20634 • Provider: AWS,Ceph,ChinaMobile,Minio
20635
20636 • Type: string
20637
20638 • Required: false
20639
20640 • Examples:
20641
20642 • ""
20643
20644 • None
20645
20646 --s3-sse-customer-key-md5
20647 If using SSE-C you may provide the secret encryption key MD5 checksum
20648 (optional).
20649
20650 If you leave it blank, this is calculated automatically from the
20651 sse_customer_key provided.
20652
20653 Properties:
20654
20655 • Config: sse_customer_key_md5
20656
20657 • Env Var: RCLONE_S3_SSE_CUSTOMER_KEY_MD5
20658
20659 • Provider: AWS,Ceph,ChinaMobile,Minio
20660
20661 • Type: string
20662
20663 • Required: false
20664
20665 • Examples:
20666
20667 • ""
20668
20669 • None
20670
20671 --s3-upload-cutoff
20672 Cutoff for switching to chunked upload.
20673
20674 Any files larger than this will be uploaded in chunks of chunk_size.
20675 The minimum is 0 and the maximum is 5 GiB.
20676
20677 Properties:
20678
20679 • Config: upload_cutoff
20680
20681 • Env Var: RCLONE_S3_UPLOAD_CUTOFF
20682
20683 • Type: SizeSuffix
20684
20685 • Default: 200Mi
20686
20687 --s3-chunk-size
20688 Chunk size to use for uploading.
20689
20690 When uploading files larger than upload_cutoff or files with unknown
20691 size (e.g. from "rclone rcat" or uploaded with "rclone mount" or
20692 google photos or google docs) they will be uploaded as multipart up‐
20693 loads using this chunk size.
20694
20695 Note that "--s3-upload-concurrency" chunks of this size are buffered in
20696 memory per transfer.
20697
20698 If you are transferring large files over high-speed links and you have
20699 enough memory, then increasing this will speed up the transfers.
20700
20701 Rclone will automatically increase the chunk size when uploading a
20702 large file of known size to stay below the 10,000 chunks limit.
20703
20704 Files of unknown size are uploaded with the configured chunk_size.
20705 Since the default chunk size is 5 MiB and there can be at most 10,000
20706 chunks, this means that by default the maximum size of a file you can
20707 stream upload is 48 GiB. If you wish to stream upload larger files
20708 then you will need to increase chunk_size.
20709
20710 Increasing the chunk size decreases the accuracy of the progress sta‐
20711 tistics displayed with "-P" flag. Rclone treats chunk as sent when
20712 it's buffered by the AWS SDK, when in fact it may still be uploading.
20713 A bigger chunk size means a bigger AWS SDK buffer and progress report‐
20714 ing more deviating from the truth.
20715
20716 Properties:
20717
20718 • Config: chunk_size
20719
20720 • Env Var: RCLONE_S3_CHUNK_SIZE
20721
20722 • Type: SizeSuffix
20723
20724 • Default: 5Mi
20725
20726 --s3-max-upload-parts
20727 Maximum number of parts in a multipart upload.
20728
20729 This option defines the maximum number of multipart chunks to use when
20730 doing a multipart upload.
20731
20732 This can be useful if a service does not support the AWS S3 specifica‐
20733 tion of 10,000 chunks.
20734
20735 Rclone will automatically increase the chunk size when uploading a
20736 large file of a known size to stay below this number of chunks limit.
20737
20738 Properties:
20739
20740 • Config: max_upload_parts
20741
20742 • Env Var: RCLONE_S3_MAX_UPLOAD_PARTS
20743
20744 • Type: int
20745
20746 • Default: 10000
20747
20748 --s3-copy-cutoff
20749 Cutoff for switching to multipart copy.
20750
20751 Any files larger than this that need to be server-side copied will be
20752 copied in chunks of this size.
20753
20754 The minimum is 0 and the maximum is 5 GiB.
20755
20756 Properties:
20757
20758 • Config: copy_cutoff
20759
20760 • Env Var: RCLONE_S3_COPY_CUTOFF
20761
20762 • Type: SizeSuffix
20763
20764 • Default: 4.656Gi
20765
20766 --s3-disable-checksum
20767 Don't store MD5 checksum with object metadata.
20768
20769 Normally rclone will calculate the MD5 checksum of the input before up‐
20770 loading it so it can add it to metadata on the object. This is great
20771 for data integrity checking but can cause long delays for large files
20772 to start uploading.
20773
20774 Properties:
20775
20776 • Config: disable_checksum
20777
20778 • Env Var: RCLONE_S3_DISABLE_CHECKSUM
20779
20780 • Type: bool
20781
20782 • Default: false
20783
20784 --s3-shared-credentials-file
20785 Path to the shared credentials file.
20786
20787 If env_auth = true then rclone can use a shared credentials file.
20788
20789 If this variable is empty rclone will look for the "AWS_SHARED_CREDEN‐
20790 TIALS_FILE" env variable. If the env value is empty it will default to
20791 the current user's home directory.
20792
20793 Linux/OSX: "$HOME/.aws/credentials"
20794 Windows: "%USERPROFILE%\.aws\credentials"
20795
20796 Properties:
20797
20798 • Config: shared_credentials_file
20799
20800 • Env Var: RCLONE_S3_SHARED_CREDENTIALS_FILE
20801
20802 • Type: string
20803
20804 • Required: false
20805
20806 --s3-profile
20807 Profile to use in the shared credentials file.
20808
20809 If env_auth = true then rclone can use a shared credentials file. This
20810 variable controls which profile is used in that file.
20811
20812 If empty it will default to the environment variable "AWS_PROFILE" or
20813 "default" if that environment variable is also not set.
20814
20815 Properties:
20816
20817 • Config: profile
20818
20819 • Env Var: RCLONE_S3_PROFILE
20820
20821 • Type: string
20822
20823 • Required: false
20824
20825 --s3-session-token
20826 An AWS session token.
20827
20828 Properties:
20829
20830 • Config: session_token
20831
20832 • Env Var: RCLONE_S3_SESSION_TOKEN
20833
20834 • Type: string
20835
20836 • Required: false
20837
20838 --s3-upload-concurrency
20839 Concurrency for multipart uploads.
20840
20841 This is the number of chunks of the same file that are uploaded concur‐
20842 rently.
20843
20844 If you are uploading small numbers of large files over high-speed links
20845 and these uploads do not fully utilize your bandwidth, then increasing
20846 this may help to speed up the transfers.
20847
20848 Properties:
20849
20850 • Config: upload_concurrency
20851
20852 • Env Var: RCLONE_S3_UPLOAD_CONCURRENCY
20853
20854 • Type: int
20855
20856 • Default: 4
20857
20858 --s3-force-path-style
20859 If true use path style access if false use virtual hosted style.
20860
20861 If this is true (the default) then rclone will use path style access,
20862 if false then rclone will use virtual path style. See the AWS S3 docs
20863 (https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingBucket.html#ac‐
20864 cess-bucket-intro) for more info.
20865
20866 Some providers (e.g. AWS, Aliyun OSS, Netease COS, or Tencent COS) re‐
20867 quire this set to false - rclone will do this automatically based on
20868 the provider setting.
20869
20870 Properties:
20871
20872 • Config: force_path_style
20873
20874 • Env Var: RCLONE_S3_FORCE_PATH_STYLE
20875
20876 • Type: bool
20877
20878 • Default: true
20879
20880 --s3-v2-auth
20881 If true use v2 authentication.
20882
20883 If this is false (the default) then rclone will use v4 authentication.
20884 If it is set then rclone will use v2 authentication.
20885
20886 Use this only if v4 signatures don't work, e.g. pre Jewel/v10 CEPH.
20887
20888 Properties:
20889
20890 • Config: v2_auth
20891
20892 • Env Var: RCLONE_S3_V2_AUTH
20893
20894 • Type: bool
20895
20896 • Default: false
20897
20898 --s3-use-accelerate-endpoint
20899 If true use the AWS S3 accelerated endpoint.
20900
20901 See: AWS S3 Transfer acceleration (https://docs.aws.amazon.com/Ama‐
20902 zonS3/latest/dev/transfer-acceleration-examples.html)
20903
20904 Properties:
20905
20906 • Config: use_accelerate_endpoint
20907
20908 • Env Var: RCLONE_S3_USE_ACCELERATE_ENDPOINT
20909
20910 • Provider: AWS
20911
20912 • Type: bool
20913
20914 • Default: false
20915
20916 --s3-leave-parts-on-error
20917 If true avoid calling abort upload on a failure, leaving all success‐
20918 fully uploaded parts on S3 for manual recovery.
20919
20920 It should be set to true for resuming uploads across different ses‐
20921 sions.
20922
20923 WARNING: Storing parts of an incomplete multipart upload counts towards
20924 space usage on S3 and will add additional costs if not cleaned up.
20925
20926 Properties:
20927
20928 • Config: leave_parts_on_error
20929
20930 • Env Var: RCLONE_S3_LEAVE_PARTS_ON_ERROR
20931
20932 • Provider: AWS
20933
20934 • Type: bool
20935
20936 • Default: false
20937
20938 --s3-list-chunk
20939 Size of listing chunk (response list for each ListObject S3 request).
20940
20941 This option is also known as "MaxKeys", "max-items", or "page-size"
20942 from the AWS S3 specification. Most services truncate the response
20943 list to 1000 objects even if requested more than that. In AWS S3 this
20944 is a global maximum and cannot be changed, see AWS S3
20945 (https://docs.aws.amazon.com/cli/latest/reference/s3/ls.html). In
20946 Ceph, this can be increased with the "rgw list buckets max chunk" op‐
20947 tion.
20948
20949 Properties:
20950
20951 • Config: list_chunk
20952
20953 • Env Var: RCLONE_S3_LIST_CHUNK
20954
20955 • Type: int
20956
20957 • Default: 1000
20958
20959 --s3-list-version
20960 Version of ListObjects to use: 1,2 or 0 for auto.
20961
20962 When S3 originally launched it only provided the ListObjects call to
20963 enumerate objects in a bucket.
20964
20965 However in May 2016 the ListObjectsV2 call was introduced. This is
20966 much higher performance and should be used if at all possible.
20967
20968 If set to the default, 0, rclone will guess according to the provider
20969 set which list objects method to call. If it guesses wrong, then it
20970 may be set manually here.
20971
20972 Properties:
20973
20974 • Config: list_version
20975
20976 • Env Var: RCLONE_S3_LIST_VERSION
20977
20978 • Type: int
20979
20980 • Default: 0
20981
20982 --s3-list-url-encode
20983 Whether to url encode listings: true/false/unset
20984
20985 Some providers support URL encoding listings and where this is avail‐
20986 able this is more reliable when using control characters in file names.
20987 If this is set to unset (the default) then rclone will choose according
20988 to the provider setting what to apply, but you can override rclone's
20989 choice here.
20990
20991 Properties:
20992
20993 • Config: list_url_encode
20994
20995 • Env Var: RCLONE_S3_LIST_URL_ENCODE
20996
20997 • Type: Tristate
20998
20999 • Default: unset
21000
21001 --s3-no-check-bucket
21002 If set, don't attempt to check the bucket exists or create it.
21003
21004 This can be useful when trying to minimise the number of transactions
21005 rclone does if you know the bucket exists already.
21006
21007 It can also be needed if the user you are using does not have bucket
21008 creation permissions. Before v1.52.0 this would have passed silently
21009 due to a bug.
21010
21011 Properties:
21012
21013 • Config: no_check_bucket
21014
21015 • Env Var: RCLONE_S3_NO_CHECK_BUCKET
21016
21017 • Type: bool
21018
21019 • Default: false
21020
21021 --s3-no-head
21022 If set, don't HEAD uploaded objects to check integrity.
21023
21024 This can be useful when trying to minimise the number of transactions
21025 rclone does.
21026
21027 Setting it means that if rclone receives a 200 OK message after upload‐
21028 ing an object with PUT then it will assume that it got uploaded proper‐
21029 ly.
21030
21031 In particular it will assume:
21032
21033 • the metadata, including modtime, storage class and content type was
21034 as uploaded
21035
21036 • the size was as uploaded
21037
21038 It reads the following items from the response for a single part PUT:
21039
21040 • the MD5SUM
21041
21042 • The uploaded date
21043
21044 For multipart uploads these items aren't read.
21045
21046 If an source object of unknown length is uploaded then rclone will do a
21047 HEAD request.
21048
21049 Setting this flag increases the chance for undetected upload failures,
21050 in particular an incorrect size, so it isn't recommended for normal op‐
21051 eration. In practice the chance of an undetected upload failure is
21052 very small even with this flag.
21053
21054 Properties:
21055
21056 • Config: no_head
21057
21058 • Env Var: RCLONE_S3_NO_HEAD
21059
21060 • Type: bool
21061
21062 • Default: false
21063
21064 --s3-no-head-object
21065 If set, do not do HEAD before GET when getting objects.
21066
21067 Properties:
21068
21069 • Config: no_head_object
21070
21071 • Env Var: RCLONE_S3_NO_HEAD_OBJECT
21072
21073 • Type: bool
21074
21075 • Default: false
21076
21077 --s3-encoding
21078 The encoding for the backend.
21079
21080 See the encoding section in the overview (https://rclone.org/over‐
21081 view/#encoding) for more info.
21082
21083 Properties:
21084
21085 • Config: encoding
21086
21087 • Env Var: RCLONE_S3_ENCODING
21088
21089 • Type: MultiEncoder
21090
21091 • Default: Slash,InvalidUtf8,Dot
21092
21093 --s3-memory-pool-flush-time
21094 How often internal memory buffer pools will be flushed.
21095
21096 Uploads which requires additional buffers (f.e multipart) will use mem‐
21097 ory pool for allocations. This option controls how often unused buf‐
21098 fers will be removed from the pool.
21099
21100 Properties:
21101
21102 • Config: memory_pool_flush_time
21103
21104 • Env Var: RCLONE_S3_MEMORY_POOL_FLUSH_TIME
21105
21106 • Type: Duration
21107
21108 • Default: 1m0s
21109
21110 --s3-memory-pool-use-mmap
21111 Whether to use mmap buffers in internal memory pool.
21112
21113 Properties:
21114
21115 • Config: memory_pool_use_mmap
21116
21117 • Env Var: RCLONE_S3_MEMORY_POOL_USE_MMAP
21118
21119 • Type: bool
21120
21121 • Default: false
21122
21123 --s3-disable-http2
21124 Disable usage of http2 for S3 backends.
21125
21126 There is currently an unsolved issue with the s3 (specifically minio)
21127 backend and HTTP/2. HTTP/2 is enabled by default for the s3 backend
21128 but can be disabled here. When the issue is solved this flag will be
21129 removed.
21130
21131 See: https://github.com/rclone/rclone/issues/4673,
21132 https://github.com/rclone/rclone/issues/3631
21133
21134 Properties:
21135
21136 • Config: disable_http2
21137
21138 • Env Var: RCLONE_S3_DISABLE_HTTP2
21139
21140 • Type: bool
21141
21142 • Default: false
21143
21144 --s3-download-url
21145 Custom endpoint for downloads. This is usually set to a CloudFront CDN
21146 URL as AWS S3 offers cheaper egress for data downloaded through the
21147 CloudFront network.
21148
21149 Properties:
21150
21151 • Config: download_url
21152
21153 • Env Var: RCLONE_S3_DOWNLOAD_URL
21154
21155 • Type: string
21156
21157 • Required: false
21158
21159 --s3-use-multipart-etag
21160 Whether to use ETag in multipart uploads for verification
21161
21162 This should be true, false or left unset to use the default for the
21163 provider.
21164
21165 Properties:
21166
21167 • Config: use_multipart_etag
21168
21169 • Env Var: RCLONE_S3_USE_MULTIPART_ETAG
21170
21171 • Type: Tristate
21172
21173 • Default: unset
21174
21175 --s3-use-presigned-request
21176 Whether to use a presigned request or PutObject for single part uploads
21177
21178 If this is false rclone will use PutObject from the AWS SDK to upload
21179 an object.
21180
21181 Versions of rclone < 1.59 use presigned requests to upload a single
21182 part object and setting this flag to true will re-enable that function‐
21183 ality. This shouldn't be necessary except in exceptional circumstances
21184 or for testing.
21185
21186 Properties:
21187
21188 • Config: use_presigned_request
21189
21190 • Env Var: RCLONE_S3_USE_PRESIGNED_REQUEST
21191
21192 • Type: bool
21193
21194 • Default: false
21195
21196 --s3-versions
21197 Include old versions in directory listings.
21198
21199 Properties:
21200
21201 • Config: versions
21202
21203 • Env Var: RCLONE_S3_VERSIONS
21204
21205 • Type: bool
21206
21207 • Default: false
21208
21209 --s3-version-at
21210 Show file versions as they were at the specified time.
21211
21212 The parameter should be a date, "2006-01-02", datetime "2006-01-02
21213 15:04:05" or a duration for that long ago, eg "100d" or "1h".
21214
21215 Note that when using this no file write operations are permitted, so
21216 you can't upload files or delete them.
21217
21218 See the time option docs (https://rclone.org/docs/#time-option) for
21219 valid formats.
21220
21221 Properties:
21222
21223 • Config: version_at
21224
21225 • Env Var: RCLONE_S3_VERSION_AT
21226
21227 • Type: Time
21228
21229 • Default: off
21230
21231 --s3-decompress
21232 If set this will decompress gzip encoded objects.
21233
21234 It is possible to upload objects to S3 with "Content-Encoding: gzip"
21235 set. Normally rclone will download these files as compressed objects.
21236
21237 If this flag is set then rclone will decompress these files with "Con‐
21238 tent-Encoding: gzip" as they are received. This means that rclone
21239 can't check the size and hash but the file contents will be decom‐
21240 pressed.
21241
21242 Properties:
21243
21244 • Config: decompress
21245
21246 • Env Var: RCLONE_S3_DECOMPRESS
21247
21248 • Type: bool
21249
21250 • Default: false
21251
21252 --s3-might-gzip
21253 Set this if the backend might gzip objects.
21254
21255 Normally providers will not alter objects when they are downloaded. If
21256 an object was not uploaded with Content-Encoding: gzip then it won't be
21257 set on download.
21258
21259 However some providers may gzip objects even if they weren't uploaded
21260 with Content-Encoding: gzip (eg Cloudflare).
21261
21262 A symptom of this would be receiving errors like
21263
21264 ERROR corrupted on transfer: sizes differ NNN vs MMM
21265
21266 If you set this flag and rclone downloads an object with Content-Encod‐
21267 ing: gzip set and chunked transfer encoding, then rclone will decom‐
21268 press the object on the fly.
21269
21270 If this is set to unset (the default) then rclone will choose according
21271 to the provider setting what to apply, but you can override rclone's
21272 choice here.
21273
21274 Properties:
21275
21276 • Config: might_gzip
21277
21278 • Env Var: RCLONE_S3_MIGHT_GZIP
21279
21280 • Type: Tristate
21281
21282 • Default: unset
21283
21284 --s3-no-system-metadata
21285 Suppress setting and reading of system metadata
21286
21287 Properties:
21288
21289 • Config: no_system_metadata
21290
21291 • Env Var: RCLONE_S3_NO_SYSTEM_METADATA
21292
21293 • Type: bool
21294
21295 • Default: false
21296
21297 Metadata
21298 User metadata is stored as x-amz-meta- keys. S3 metadata keys are case
21299 insensitive and are always returned in lower case.
21300
21301 Here are the possible system metadata items for the s3 backend.
21302
21303 Name Help Type Example Read Only
21304 ──────────────────────────────────────────────────────────────────────────────────────────────────────
21305 btime Time of RFC 3339 2006-01-02T15:04:05.999999999Z07:00 Y
21306 file birth
21307 (creation)
21308 read from
21309 Last-Modi‐
21310 fied header
21311 cache-con‐ Cache-Con‐ string no-cache N
21312 trol trol header
21313 content- Content- string inline N
21314 disposition Disposition
21315 header
21316 content-en‐ Content-En‐ string gzip N
21317 coding coding
21318 header
21319 content- Content- string en-US N
21320 language Language
21321 header
21322 content- Content- string text/plain N
21323 type Type header
21324 mtime Time of RFC 3339 2006-01-02T15:04:05.999999999Z07:00 N
21325 last modi‐
21326 fication,
21327 read from
21328 rclone
21329 metadata
21330 tier Tier of the string GLACIER Y
21331 object
21332
21333 See the metadata (https://rclone.org/docs/#metadata) docs for more in‐
21334 fo.
21335
21336 Backend commands
21337 Here are the commands specific to the s3 backend.
21338
21339 Run them with
21340
21341 rclone backend COMMAND remote:
21342
21343 The help below will explain what arguments each command takes.
21344
21345 See the backend (https://rclone.org/commands/rclone_backend/) command
21346 for more info on how to pass options and arguments.
21347
21348 These can be run on a running backend using the rc command backend/com‐
21349 mand (https://rclone.org/rc/#backend-command).
21350
21351 restore
21352 Restore objects from GLACIER to normal storage
21353
21354 rclone backend restore remote: [options] [<arguments>+]
21355
21356 This command can be used to restore one or more objects from GLACIER to
21357 normal storage.
21358
21359 Usage Examples:
21360
21361 rclone backend restore s3:bucket/path/to/object [-o priority=PRIORITY] [-o lifetime=DAYS]
21362 rclone backend restore s3:bucket/path/to/directory [-o priority=PRIORITY] [-o lifetime=DAYS]
21363 rclone backend restore s3:bucket [-o priority=PRIORITY] [-o lifetime=DAYS]
21364
21365 This flag also obeys the filters. Test first with -i/--interactive or
21366 --dry-run flags
21367
21368 rclone -i backend restore --include "*.txt" s3:bucket/path -o priority=Standard
21369
21370 All the objects shown will be marked for restore, then
21371
21372 rclone backend restore --include "*.txt" s3:bucket/path -o priority=Standard
21373
21374 It returns a list of status dictionaries with Remote and Status keys.
21375 The Status will be OK if it was successful or an error message if not.
21376
21377 [
21378 {
21379 "Status": "OK",
21380 "Path": "test.txt"
21381 },
21382 {
21383 "Status": "OK",
21384 "Path": "test/file4.txt"
21385 }
21386 ]
21387
21388 Options:
21389
21390 • "description": The optional description for the job.
21391
21392 • "lifetime": Lifetime of the active copy in days
21393
21394 • "priority": Priority of restore: Standard|Expedited|Bulk
21395
21396 list-multipart-uploads
21397 List the unfinished multipart uploads
21398
21399 rclone backend list-multipart-uploads remote: [options] [<arguments>+]
21400
21401 This command lists the unfinished multipart uploads in JSON format.
21402
21403 rclone backend list-multipart s3:bucket/path/to/object
21404
21405 It returns a dictionary of buckets with values as lists of unfinished
21406 multipart uploads.
21407
21408 You can call it with no bucket in which case it lists all bucket, with
21409 a bucket or with a bucket and path.
21410
21411 {
21412 "rclone": [
21413 {
21414 "Initiated": "2020-06-26T14:20:36Z",
21415 "Initiator": {
21416 "DisplayName": "XXX",
21417 "ID": "arn:aws:iam::XXX:user/XXX"
21418 },
21419 "Key": "KEY",
21420 "Owner": {
21421 "DisplayName": null,
21422 "ID": "XXX"
21423 },
21424 "StorageClass": "STANDARD",
21425 "UploadId": "XXX"
21426 }
21427 ],
21428 "rclone-1000files": [],
21429 "rclone-dst": []
21430 }
21431
21432 cleanup
21433 Remove unfinished multipart uploads.
21434
21435 rclone backend cleanup remote: [options] [<arguments>+]
21436
21437 This command removes unfinished multipart uploads of age greater than
21438 max-age which defaults to 24 hours.
21439
21440 Note that you can use -i/--dry-run with this command to see what it
21441 would do.
21442
21443 rclone backend cleanup s3:bucket/path/to/object
21444 rclone backend cleanup -o max-age=7w s3:bucket/path/to/object
21445
21446 Durations are parsed as per the rest of rclone, 2h, 7d, 7w etc.
21447
21448 Options:
21449
21450 • "max-age": Max age of upload to delete
21451
21452 cleanup-hidden
21453 Remove old versions of files.
21454
21455 rclone backend cleanup-hidden remote: [options] [<arguments>+]
21456
21457 This command removes any old hidden versions of files on a versions en‐
21458 abled bucket.
21459
21460 Note that you can use -i/--dry-run with this command to see what it
21461 would do.
21462
21463 rclone backend cleanup-hidden s3:bucket/path/to/dir
21464
21465 versioning
21466 Set/get versioning support for a bucket.
21467
21468 rclone backend versioning remote: [options] [<arguments>+]
21469
21470 This command sets versioning support if a parameter is passed and then
21471 returns the current versioning status for the bucket supplied.
21472
21473 rclone backend versioning s3:bucket # read status only
21474 rclone backend versioning s3:bucket Enabled
21475 rclone backend versioning s3:bucket Suspended
21476
21477 It may return "Enabled", "Suspended" or "Unversioned". Note that once
21478 versioning has been enabled the status can't be set back to "Unver‐
21479 sioned".
21480
21481 Anonymous access to public buckets
21482 If you want to use rclone to access a public bucket, configure with a
21483 blank access_key_id and secret_access_key. Your config should end up
21484 looking like this:
21485
21486 [anons3]
21487 type = s3
21488 provider = AWS
21489 env_auth = false
21490 access_key_id =
21491 secret_access_key =
21492 region = us-east-1
21493 endpoint =
21494 location_constraint =
21495 acl = private
21496 server_side_encryption =
21497 storage_class =
21498
21499 Then use it as normal with the name of the public bucket, e.g.
21500
21501 rclone lsd anons3:1000genomes
21502
21503 You will be able to list and copy data but not upload it.
21504
21505 Providers
21506 AWS S3
21507 This is the provider used as main example and described in the configu‐
21508 ration section above.
21509
21510 AWS Snowball Edge
21511 AWS Snowball (https://aws.amazon.com/snowball/) is a hardware appliance
21512 used for transferring bulk data back to AWS. Its main software inter‐
21513 face is S3 object storage.
21514
21515 To use rclone with AWS Snowball Edge devices, configure as standard for
21516 an 'S3 Compatible Service'.
21517
21518 If using rclone pre v1.59 be sure to set upload_cutoff = 0 otherwise
21519 you will run into authentication header issues as the snowball device
21520 does not support query parameter based authentication.
21521
21522 With rclone v1.59 or later setting upload_cutoff should not be neces‐
21523 sary.
21524
21525 eg.
21526
21527 [snowball]
21528 type = s3
21529 provider = Other
21530 access_key_id = YOUR_ACCESS_KEY
21531 secret_access_key = YOUR_SECRET_KEY
21532 endpoint = http://[IP of Snowball]:8080
21533 upload_cutoff = 0
21534
21535 Ceph
21536 Ceph (https://ceph.com/) is an open-source, unified, distributed stor‐
21537 age system designed for excellent performance, reliability and scala‐
21538 bility. It has an S3 compatible object storage interface.
21539
21540 To use rclone with Ceph, configure as above but leave the region blank
21541 and set the endpoint. You should end up with something like this in
21542 your config:
21543
21544 [ceph]
21545 type = s3
21546 provider = Ceph
21547 env_auth = false
21548 access_key_id = XXX
21549 secret_access_key = YYY
21550 region =
21551 endpoint = https://ceph.endpoint.example.com
21552 location_constraint =
21553 acl =
21554 server_side_encryption =
21555 storage_class =
21556
21557 If you are using an older version of CEPH (e.g. 10.2.x Jewel) and a
21558 version of rclone before v1.59 then you may need to supply the parame‐
21559 ter --s3-upload-cutoff 0 or put this in the config file as upload_cut‐
21560 off 0 to work around a bug which causes uploading of small files to
21561 fail.
21562
21563 Note also that Ceph sometimes puts / in the passwords it gives users.
21564 If you read the secret access key using the command line tools you will
21565 get a JSON blob with the / escaped as \/. Make sure you only write /
21566 in the secret access key.
21567
21568 Eg the dump from Ceph looks something like this (irrelevant keys re‐
21569 moved).
21570
21571 {
21572 "user_id": "xxx",
21573 "display_name": "xxxx",
21574 "keys": [
21575 {
21576 "user": "xxx",
21577 "access_key": "xxxxxx",
21578 "secret_key": "xxxxxx\/xxxx"
21579 }
21580 ],
21581 }
21582
21583 Because this is a json dump, it is encoding the / as \/, so if you use
21584 the secret key as xxxxxx/xxxx it will work fine.
21585
21586 Cloudflare R2
21587 Cloudflare R2 (https://blog.cloudflare.com/r2-open-beta/) Storage al‐
21588 lows developers to store large amounts of unstructured data without the
21589 costly egress bandwidth fees associated with typical cloud storage ser‐
21590 vices.
21591
21592 Here is an example of making a Cloudflare R2 configuration. First run:
21593
21594 rclone config
21595
21596 This will guide you through an interactive setup process.
21597
21598 Note that all buckets are private, and all are stored in the same "au‐
21599 to" region. It is necessary to use Cloudflare workers to share the
21600 content of a bucket publicly.
21601
21602 No remotes found, make a new one?
21603 n) New remote
21604 s) Set configuration password
21605 q) Quit config
21606 n/s/q> n
21607 name> r2
21608 Option Storage.
21609 Type of storage to configure.
21610 Choose a number from below, or type in your own value.
21611 ...
21612 XX / Amazon S3 Compliant Storage Providers including AWS, Alibaba, Ceph, China Mobile, Cloudflare, ArvanCloud, Digital Ocean, Dreamhost, Huawei OBS, IBM COS, Lyve Cloud, Minio, Netease, RackCorp, Scaleway, SeaweedFS, StackPath, Storj, Tencent COS and Wasabi
21613 \ (s3)
21614 ...
21615 Storage> s3
21616 Option provider.
21617 Choose your S3 provider.
21618 Choose a number from below, or type in your own value.
21619 Press Enter to leave empty.
21620 ...
21621 XX / Cloudflare R2 Storage
21622 \ (Cloudflare)
21623 ...
21624 provider> Cloudflare
21625 Option env_auth.
21626 Get AWS credentials from runtime (environment variables or EC2/ECS meta data if no env vars).
21627 Only applies if access_key_id and secret_access_key is blank.
21628 Choose a number from below, or type in your own boolean value (true or false).
21629 Press Enter for the default (false).
21630 1 / Enter AWS credentials in the next step.
21631 \ (false)
21632 2 / Get AWS credentials from the environment (env vars or IAM).
21633 \ (true)
21634 env_auth> 1
21635 Option access_key_id.
21636 AWS Access Key ID.
21637 Leave blank for anonymous access or runtime credentials.
21638 Enter a value. Press Enter to leave empty.
21639 access_key_id> ACCESS_KEY
21640 Option secret_access_key.
21641 AWS Secret Access Key (password).
21642 Leave blank for anonymous access or runtime credentials.
21643 Enter a value. Press Enter to leave empty.
21644 secret_access_key> SECRET_ACCESS_KEY
21645 Option region.
21646 Region to connect to.
21647 Choose a number from below, or type in your own value.
21648 Press Enter to leave empty.
21649 1 / R2 buckets are automatically distributed across Cloudflare's data centers for low latency.
21650 \ (auto)
21651 region> 1
21652 Option endpoint.
21653 Endpoint for S3 API.
21654 Required when using an S3 clone.
21655 Enter a value. Press Enter to leave empty.
21656 endpoint> https://ACCOUNT_ID.r2.cloudflarestorage.com
21657 Edit advanced config?
21658 y) Yes
21659 n) No (default)
21660 y/n> n
21661 --------------------
21662 y) Yes this is OK (default)
21663 e) Edit this remote
21664 d) Delete this remote
21665 y/e/d> y
21666
21667 This will leave your config looking something like:
21668
21669 [r2]
21670 type = s3
21671 provider = Cloudflare
21672 access_key_id = ACCESS_KEY
21673 secret_access_key = SECRET_ACCESS_KEY
21674 region = auto
21675 endpoint = https://ACCOUNT_ID.r2.cloudflarestorage.com
21676 acl = private
21677
21678 Now run rclone lsf r2: to see your buckets and rclone lsf r2:bucket to
21679 look within a bucket.
21680
21681 Dreamhost
21682 Dreamhost DreamObjects (https://www.dreamhost.com/cloud/storage/) is an
21683 object storage system based on CEPH.
21684
21685 To use rclone with Dreamhost, configure as above but leave the region
21686 blank and set the endpoint. You should end up with something like this
21687 in your config:
21688
21689 [dreamobjects]
21690 type = s3
21691 provider = DreamHost
21692 env_auth = false
21693 access_key_id = your_access_key
21694 secret_access_key = your_secret_key
21695 region =
21696 endpoint = objects-us-west-1.dream.io
21697 location_constraint =
21698 acl = private
21699 server_side_encryption =
21700 storage_class =
21701
21702 DigitalOcean Spaces
21703 Spaces (https://www.digitalocean.com/products/object-storage/) is an
21704 S3-interoperable (https://developers.digitalocean.com/documenta‐
21705 tion/spaces/) object storage service from cloud provider DigitalOcean.
21706
21707 To connect to DigitalOcean Spaces you will need an access key and se‐
21708 cret key. These can be retrieved on the "Applications & API
21709 (https://cloud.digitalocean.com/settings/api/tokens)" page of the Digi‐
21710 talOcean control panel. They will be needed when prompted by rclone
21711 config for your access_key_id and secret_access_key.
21712
21713 When prompted for a region or location_constraint, press enter to use
21714 the default value. The region must be included in the endpoint setting
21715 (e.g. nyc3.digitaloceanspaces.com). The default values can be used
21716 for other settings.
21717
21718 Going through the whole process of creating a new remote by running
21719 rclone config, each prompt should be answered as shown below:
21720
21721 Storage> s3
21722 env_auth> 1
21723 access_key_id> YOUR_ACCESS_KEY
21724 secret_access_key> YOUR_SECRET_KEY
21725 region>
21726 endpoint> nyc3.digitaloceanspaces.com
21727 location_constraint>
21728 acl>
21729 storage_class>
21730
21731 The resulting configuration file should look like:
21732
21733 [spaces]
21734 type = s3
21735 provider = DigitalOcean
21736 env_auth = false
21737 access_key_id = YOUR_ACCESS_KEY
21738 secret_access_key = YOUR_SECRET_KEY
21739 region =
21740 endpoint = nyc3.digitaloceanspaces.com
21741 location_constraint =
21742 acl =
21743 server_side_encryption =
21744 storage_class =
21745
21746 Once configured, you can create a new Space and begin copying files.
21747 For example:
21748
21749 rclone mkdir spaces:my-new-space
21750 rclone copy /path/to/files spaces:my-new-space
21751
21752 Huawei OBS
21753 Object Storage Service (OBS) provides stable, secure, efficient, and
21754 easy-to-use cloud storage that lets you store virtually any volume of
21755 unstructured data in any format and access it from anywhere.
21756
21757 OBS provides an S3 interface, you can copy and modify the following
21758 configuration and add it to your rclone configuration file.
21759
21760 [obs]
21761 type = s3
21762 provider = HuaweiOBS
21763 access_key_id = your-access-key-id
21764 secret_access_key = your-secret-access-key
21765 region = af-south-1
21766 endpoint = obs.af-south-1.myhuaweicloud.com
21767 acl = private
21768
21769 Or you can also configure via the interactive command line:
21770
21771 No remotes found, make a new one?
21772 n) New remote
21773 s) Set configuration password
21774 q) Quit config
21775 n/s/q> n
21776 name> obs
21777 Option Storage.
21778 Type of storage to configure.
21779 Choose a number from below, or type in your own value.
21780 [snip]
21781 5 / Amazon S3 Compliant Storage Providers including AWS, Alibaba, Ceph, China Mobile, Cloudflare, ArvanCloud, Digital Ocean, Dreamhost, Huawei OBS, IBM COS, Lyve Cloud, Minio, Netease, RackCorp, Scaleway, SeaweedFS, StackPath, Storj, Tencent COS and Wasabi
21782 \ (s3)
21783 [snip]
21784 Storage> 5
21785 Option provider.
21786 Choose your S3 provider.
21787 Choose a number from below, or type in your own value.
21788 Press Enter to leave empty.
21789 [snip]
21790 9 / Huawei Object Storage Service
21791 \ (HuaweiOBS)
21792 [snip]
21793 provider> 9
21794 Option env_auth.
21795 Get AWS credentials from runtime (environment variables or EC2/ECS meta data if no env vars).
21796 Only applies if access_key_id and secret_access_key is blank.
21797 Choose a number from below, or type in your own boolean value (true or false).
21798 Press Enter for the default (false).
21799 1 / Enter AWS credentials in the next step.
21800 \ (false)
21801 2 / Get AWS credentials from the environment (env vars or IAM).
21802 \ (true)
21803 env_auth> 1
21804 Option access_key_id.
21805 AWS Access Key ID.
21806 Leave blank for anonymous access or runtime credentials.
21807 Enter a value. Press Enter to leave empty.
21808 access_key_id> your-access-key-id
21809 Option secret_access_key.
21810 AWS Secret Access Key (password).
21811 Leave blank for anonymous access or runtime credentials.
21812 Enter a value. Press Enter to leave empty.
21813 secret_access_key> your-secret-access-key
21814 Option region.
21815 Region to connect to.
21816 Choose a number from below, or type in your own value.
21817 Press Enter to leave empty.
21818 1 / AF-Johannesburg
21819 \ (af-south-1)
21820 2 / AP-Bangkok
21821 \ (ap-southeast-2)
21822 [snip]
21823 region> 1
21824 Option endpoint.
21825 Endpoint for OBS API.
21826 Choose a number from below, or type in your own value.
21827 Press Enter to leave empty.
21828 1 / AF-Johannesburg
21829 \ (obs.af-south-1.myhuaweicloud.com)
21830 2 / AP-Bangkok
21831 \ (obs.ap-southeast-2.myhuaweicloud.com)
21832 [snip]
21833 endpoint> 1
21834 Option acl.
21835 Canned ACL used when creating buckets and storing or copying objects.
21836 This ACL is used for creating objects and if bucket_acl isn't set, for creating buckets too.
21837 For more info visit https://docs.aws.amazon.com/AmazonS3/latest/dev/acl-overview.html#canned-acl
21838 Note that this ACL is applied when server-side copying objects as S3
21839 doesn't copy the ACL from the source but rather writes a fresh one.
21840 Choose a number from below, or type in your own value.
21841 Press Enter to leave empty.
21842 / Owner gets FULL_CONTROL.
21843 1 | No one else has access rights (default).
21844 \ (private)
21845 [snip]
21846 acl> 1
21847 Edit advanced config?
21848 y) Yes
21849 n) No (default)
21850 y/n>
21851 --------------------
21852 [obs]
21853 type = s3
21854 provider = HuaweiOBS
21855 access_key_id = your-access-key-id
21856 secret_access_key = your-secret-access-key
21857 region = af-south-1
21858 endpoint = obs.af-south-1.myhuaweicloud.com
21859 acl = private
21860 --------------------
21861 y) Yes this is OK (default)
21862 e) Edit this remote
21863 d) Delete this remote
21864 y/e/d> y
21865 Current remotes:
21866
21867 Name Type
21868 ==== ====
21869 obs s3
21870
21871 e) Edit existing remote
21872 n) New remote
21873 d) Delete remote
21874 r) Rename remote
21875 c) Copy remote
21876 s) Set configuration password
21877 q) Quit config
21878 e/n/d/r/c/s/q> q
21879
21880 IBM COS (S3)
21881 Information stored with IBM Cloud Object Storage is encrypted and dis‐
21882 persed across multiple geographic locations, and accessed through an
21883 implementation of the S3 API. This service makes use of the distrib‐
21884 uted storage technologies provided by IBM’s Cloud Object Storage System
21885 (formerly Cleversafe). For more information visit:
21886 (http://www.ibm.com/cloud/object-storage)
21887
21888 To configure access to IBM COS S3, follow the steps below:
21889
21890 1. Run rclone config and select n for a new remote.
21891
21892 2018/02/14 14:13:11 NOTICE: Config file "C:\\Users\\a\\.config\\rclone\\rclone.conf" not found - using defaults
21893 No remotes found, make a new one?
21894 n) New remote
21895 s) Set configuration password
21896 q) Quit config
21897 n/s/q> n
21898
21899 2. Enter the name for the configuration
21900
21901 name> <YOUR NAME>
21902
21903 3. Select "s3" storage.
21904
21905 Choose a number from below, or type in your own value
21906 1 / Alias for an existing remote
21907 \ "alias"
21908 2 / Amazon Drive
21909 \ "amazon cloud drive"
21910 3 / Amazon S3 Complaint Storage Providers (Dreamhost, Ceph, ChinaMobile, ArvanCloud, Minio, IBM COS)
21911 \ "s3"
21912 4 / Backblaze B2
21913 \ "b2"
21914 [snip]
21915 23 / HTTP
21916 \ "http"
21917 Storage> 3
21918
21919 4. Select IBM COS as the S3 Storage Provider.
21920
21921 Choose the S3 provider.
21922 Choose a number from below, or type in your own value
21923 1 / Choose this option to configure Storage to AWS S3
21924 \ "AWS"
21925 2 / Choose this option to configure Storage to Ceph Systems
21926 \ "Ceph"
21927 3 / Choose this option to configure Storage to Dreamhost
21928 \ "Dreamhost"
21929 4 / Choose this option to the configure Storage to IBM COS S3
21930 \ "IBMCOS"
21931 5 / Choose this option to the configure Storage to Minio
21932 \ "Minio"
21933 Provider>4
21934
21935 5. Enter the Access Key and Secret.
21936
21937 AWS Access Key ID - leave blank for anonymous access or runtime credentials.
21938 access_key_id> <>
21939 AWS Secret Access Key (password) - leave blank for anonymous access or runtime credentials.
21940 secret_access_key> <>
21941
21942 6. Specify the endpoint for IBM COS. For Public IBM COS, choose from
21943 the option below. For On Premise IBM COS, enter an endpoint ad‐
21944 dress.
21945
21946 Endpoint for IBM COS S3 API.
21947 Specify if using an IBM COS On Premise.
21948 Choose a number from below, or type in your own value
21949 1 / US Cross Region Endpoint
21950 \ "s3-api.us-geo.objectstorage.softlayer.net"
21951 2 / US Cross Region Dallas Endpoint
21952 \ "s3-api.dal.us-geo.objectstorage.softlayer.net"
21953 3 / US Cross Region Washington DC Endpoint
21954 \ "s3-api.wdc-us-geo.objectstorage.softlayer.net"
21955 4 / US Cross Region San Jose Endpoint
21956 \ "s3-api.sjc-us-geo.objectstorage.softlayer.net"
21957 5 / US Cross Region Private Endpoint
21958 \ "s3-api.us-geo.objectstorage.service.networklayer.com"
21959 6 / US Cross Region Dallas Private Endpoint
21960 \ "s3-api.dal-us-geo.objectstorage.service.networklayer.com"
21961 7 / US Cross Region Washington DC Private Endpoint
21962 \ "s3-api.wdc-us-geo.objectstorage.service.networklayer.com"
21963 8 / US Cross Region San Jose Private Endpoint
21964 \ "s3-api.sjc-us-geo.objectstorage.service.networklayer.com"
21965 9 / US Region East Endpoint
21966 \ "s3.us-east.objectstorage.softlayer.net"
21967 10 / US Region East Private Endpoint
21968 \ "s3.us-east.objectstorage.service.networklayer.com"
21969 11 / US Region South Endpoint
21970 [snip]
21971 34 / Toronto Single Site Private Endpoint
21972 \ "s3.tor01.objectstorage.service.networklayer.com"
21973 endpoint>1
21974
21975 7. Specify a IBM COS Location Constraint. The location constraint must
21976 match endpoint when using IBM Cloud Public. For on-prem COS, do not
21977 make a selection from this list, hit enter
21978
21979 1 / US Cross Region Standard
21980 \ "us-standard"
21981 2 / US Cross Region Vault
21982 \ "us-vault"
21983 3 / US Cross Region Cold
21984 \ "us-cold"
21985 4 / US Cross Region Flex
21986 \ "us-flex"
21987 5 / US East Region Standard
21988 \ "us-east-standard"
21989 6 / US East Region Vault
21990 \ "us-east-vault"
21991 7 / US East Region Cold
21992 \ "us-east-cold"
21993 8 / US East Region Flex
21994 \ "us-east-flex"
21995 9 / US South Region Standard
21996 \ "us-south-standard"
21997 10 / US South Region Vault
21998 \ "us-south-vault"
21999 [snip]
22000 32 / Toronto Flex
22001 \ "tor01-flex"
22002 location_constraint>1
22003
22004 9. Specify a canned ACL. IBM Cloud (Storage) supports "public-read"
22005 and "private". IBM Cloud(Infra) supports all the canned ACLs. On-
22006 Premise COS supports all the canned ACLs.
22007
22008 Canned ACL used when creating buckets and/or storing objects in S3.
22009 For more info visit https://docs.aws.amazon.com/AmazonS3/latest/dev/acl-overview.html#canned-acl
22010 Choose a number from below, or type in your own value
22011 1 / Owner gets FULL_CONTROL. No one else has access rights (default). This acl is available on IBM Cloud (Infra), IBM Cloud (Storage), On-Premise COS
22012 \ "private"
22013 2 / Owner gets FULL_CONTROL. The AllUsers group gets READ access. This acl is available on IBM Cloud (Infra), IBM Cloud (Storage), On-Premise IBM COS
22014 \ "public-read"
22015 3 / Owner gets FULL_CONTROL. The AllUsers group gets READ and WRITE access. This acl is available on IBM Cloud (Infra), On-Premise IBM COS
22016 \ "public-read-write"
22017 4 / Owner gets FULL_CONTROL. The AuthenticatedUsers group gets READ access. Not supported on Buckets. This acl is available on IBM Cloud (Infra) and On-Premise IBM COS
22018 \ "authenticated-read"
22019 acl> 1
22020
22021 12. Review the displayed configuration and accept to save the "remote"
22022 then quit. The config file should look like this
22023
22024 [xxx]
22025 type = s3
22026 Provider = IBMCOS
22027 access_key_id = xxx
22028 secret_access_key = yyy
22029 endpoint = s3-api.us-geo.objectstorage.softlayer.net
22030 location_constraint = us-standard
22031 acl = private
22032
22033 13. Execute rclone commands
22034
22035 1) Create a bucket.
22036 rclone mkdir IBM-COS-XREGION:newbucket
22037 2) List available buckets.
22038 rclone lsd IBM-COS-XREGION:
22039 -1 2017-11-08 21:16:22 -1 test
22040 -1 2018-02-14 20:16:39 -1 newbucket
22041 3) List contents of a bucket.
22042 rclone ls IBM-COS-XREGION:newbucket
22043 18685952 test.exe
22044 4) Copy a file from local to remote.
22045 rclone copy /Users/file.txt IBM-COS-XREGION:newbucket
22046 5) Copy a file from remote to local.
22047 rclone copy IBM-COS-XREGION:newbucket/file.txt .
22048 6) Delete a file on remote.
22049 rclone delete IBM-COS-XREGION:newbucket/file.txt
22050
22051 IDrive e2
22052 Here is an example of making an IDrive e2 (https://www.idrive.com/e2/)
22053 configuration. First run:
22054
22055 rclone config
22056
22057 This will guide you through an interactive setup process.
22058
22059 No remotes found, make a new one?
22060 n) New remote
22061 s) Set configuration password
22062 q) Quit config
22063 n/s/q> n
22064
22065 Enter name for new remote.
22066 name> e2
22067
22068 Option Storage.
22069 Type of storage to configure.
22070 Choose a number from below, or type in your own value.
22071 [snip]
22072 XX / Amazon S3 Compliant Storage Providers including AWS, Alibaba, Ceph, China Mobile, Cloudflare, ArvanCloud, Digital Ocean, Dreamhost, Huawei OBS, IBM COS, IDrive e2, Lyve Cloud, Minio, Netease, RackCorp, Scaleway, SeaweedFS, StackPath, Storj, Tencent COS and Wasabi
22073 \ (s3)
22074 [snip]
22075 Storage> s3
22076
22077 Option provider.
22078 Choose your S3 provider.
22079 Choose a number from below, or type in your own value.
22080 Press Enter to leave empty.
22081 [snip]
22082 XX / IDrive e2
22083 \ (IDrive)
22084 [snip]
22085 provider> IDrive
22086
22087 Option env_auth.
22088 Get AWS credentials from runtime (environment variables or EC2/ECS meta data if no env vars).
22089 Only applies if access_key_id and secret_access_key is blank.
22090 Choose a number from below, or type in your own boolean value (true or false).
22091 Press Enter for the default (false).
22092 1 / Enter AWS credentials in the next step.
22093 \ (false)
22094 2 / Get AWS credentials from the environment (env vars or IAM).
22095 \ (true)
22096 env_auth>
22097
22098 Option access_key_id.
22099 AWS Access Key ID.
22100 Leave blank for anonymous access or runtime credentials.
22101 Enter a value. Press Enter to leave empty.
22102 access_key_id> YOUR_ACCESS_KEY
22103
22104 Option secret_access_key.
22105 AWS Secret Access Key (password).
22106 Leave blank for anonymous access or runtime credentials.
22107 Enter a value. Press Enter to leave empty.
22108 secret_access_key> YOUR_SECRET_KEY
22109
22110 Option acl.
22111 Canned ACL used when creating buckets and storing or copying objects.
22112 This ACL is used for creating objects and if bucket_acl isn't set, for creating buckets too.
22113 For more info visit https://docs.aws.amazon.com/AmazonS3/latest/dev/acl-overview.html#canned-acl
22114 Note that this ACL is applied when server-side copying objects as S3
22115 doesn't copy the ACL from the source but rather writes a fresh one.
22116 Choose a number from below, or type in your own value.
22117 Press Enter to leave empty.
22118 / Owner gets FULL_CONTROL.
22119 1 | No one else has access rights (default).
22120 \ (private)
22121 / Owner gets FULL_CONTROL.
22122 2 | The AllUsers group gets READ access.
22123 \ (public-read)
22124 / Owner gets FULL_CONTROL.
22125 3 | The AllUsers group gets READ and WRITE access.
22126 | Granting this on a bucket is generally not recommended.
22127 \ (public-read-write)
22128 / Owner gets FULL_CONTROL.
22129 4 | The AuthenticatedUsers group gets READ access.
22130 \ (authenticated-read)
22131 / Object owner gets FULL_CONTROL.
22132 5 | Bucket owner gets READ access.
22133 | If you specify this canned ACL when creating a bucket, Amazon S3 ignores it.
22134 \ (bucket-owner-read)
22135 / Both the object owner and the bucket owner get FULL_CONTROL over the object.
22136 6 | If you specify this canned ACL when creating a bucket, Amazon S3 ignores it.
22137 \ (bucket-owner-full-control)
22138 acl>
22139
22140 Edit advanced config?
22141 y) Yes
22142 n) No (default)
22143 y/n>
22144
22145 Configuration complete.
22146 Options:
22147 - type: s3
22148 - provider: IDrive
22149 - access_key_id: YOUR_ACCESS_KEY
22150 - secret_access_key: YOUR_SECRET_KEY
22151 - endpoint: q9d9.la12.idrivee2-5.com
22152 Keep this "e2" remote?
22153 y) Yes this is OK (default)
22154 e) Edit this remote
22155 d) Delete this remote
22156 y/e/d> y
22157
22158 IONOS Cloud
22159 IONOS S3 Object Storage (https://cloud.ionos.com/storage/object-stor‐
22160 age) is a service offered by IONOS for storing and accessing unstruc‐
22161 tured data. To connect to the service, you will need an access key and
22162 a secret key. These can be found in the Data Center Designer
22163 (https://dcd.ionos.com/), by selecting Manager resources > Object Stor‐
22164 age Key Manager.
22165
22166 Here is an example of a configuration. First, run rclone config. This
22167 will walk you through an interactive setup process. Type n to add the
22168 new remote, and then enter a name:
22169
22170 Enter name for new remote.
22171 name> ionos-fra
22172
22173 Type s3 to choose the connection type:
22174
22175 Option Storage.
22176 Type of storage to configure.
22177 Choose a number from below, or type in your own value.
22178 [snip]
22179 XX / Amazon S3 Compliant Storage Providers including AWS, Alibaba, Ceph, China Mobile, Cloudflare, ArvanCloud, Digital Ocean, Dreamhost, Huawei OBS, IBM COS, IDrive e2, IONOS Cloud, Lyve Cloud, Minio, Netease, RackCorp, Scaleway, SeaweedFS, StackPath, Storj, Tencent COS and Wasabi
22180 \ (s3)
22181 [snip]
22182 Storage> s3
22183
22184 Type IONOS:
22185
22186 Option provider.
22187 Choose your S3 provider.
22188 Choose a number from below, or type in your own value.
22189 Press Enter to leave empty.
22190 [snip]
22191 XX / IONOS Cloud
22192 \ (IONOS)
22193 [snip]
22194 provider> IONOS
22195
22196 Press Enter to choose the default option Enter AWS credentials in the
22197 next step:
22198
22199 Option env_auth.
22200 Get AWS credentials from runtime (environment variables or EC2/ECS meta data if no env vars).
22201 Only applies if access_key_id and secret_access_key is blank.
22202 Choose a number from below, or type in your own boolean value (true or false).
22203 Press Enter for the default (false).
22204 1 / Enter AWS credentials in the next step.
22205 \ (false)
22206 2 / Get AWS credentials from the environment (env vars or IAM).
22207 \ (true)
22208 env_auth>
22209
22210 Enter your Access Key and Secret key. These can be retrieved in the
22211 Data Center Designer (https://dcd.ionos.com/), click on the menu “Man‐
22212 ager resources” / "Object Storage Key Manager".
22213
22214 Option access_key_id.
22215 AWS Access Key ID.
22216 Leave blank for anonymous access or runtime credentials.
22217 Enter a value. Press Enter to leave empty.
22218 access_key_id> YOUR_ACCESS_KEY
22219
22220 Option secret_access_key.
22221 AWS Secret Access Key (password).
22222 Leave blank for anonymous access or runtime credentials.
22223 Enter a value. Press Enter to leave empty.
22224 secret_access_key> YOUR_SECRET_KEY
22225
22226 Choose the region where your bucket is located:
22227
22228 Option region.
22229 Region where your bucket will be created and your data stored.
22230 Choose a number from below, or type in your own value.
22231 Press Enter to leave empty.
22232 1 / Frankfurt, Germany
22233 \ (de)
22234 2 / Berlin, Germany
22235 \ (eu-central-2)
22236 3 / Logrono, Spain
22237 \ (eu-south-2)
22238 region> 2
22239
22240 Choose the endpoint from the same region:
22241
22242 Option endpoint.
22243 Endpoint for IONOS S3 Object Storage.
22244 Specify the endpoint from the same region.
22245 Choose a number from below, or type in your own value.
22246 Press Enter to leave empty.
22247 1 / Frankfurt, Germany
22248 \ (s3-eu-central-1.ionoscloud.com)
22249 2 / Berlin, Germany
22250 \ (s3-eu-central-2.ionoscloud.com)
22251 3 / Logrono, Spain
22252 \ (s3-eu-south-2.ionoscloud.com)
22253 endpoint> 1
22254
22255 Press Enter to choose the default option or choose the desired ACL set‐
22256 ting:
22257
22258 Option acl.
22259 Canned ACL used when creating buckets and storing or copying objects.
22260 This ACL is used for creating objects and if bucket_acl isn't set, for creating buckets too.
22261 For more info visit https://docs.aws.amazon.com/AmazonS3/latest/dev/acl-overview.html#canned-acl
22262 Note that this ACL is applied when server-side copying objects as S3
22263 doesn't copy the ACL from the source but rather writes a fresh one.
22264 Choose a number from below, or type in your own value.
22265 Press Enter to leave empty.
22266 / Owner gets FULL_CONTROL.
22267 1 | No one else has access rights (default).
22268 \ (private)
22269 / Owner gets FULL_CONTROL.
22270 [snip]
22271 acl>
22272
22273 Press Enter to skip the advanced config:
22274
22275 Edit advanced config?
22276 y) Yes
22277 n) No (default)
22278 y/n>
22279
22280 Press Enter to save the configuration, and then q to quit the configu‐
22281 ration process:
22282
22283 Configuration complete.
22284 Options:
22285 - type: s3
22286 - provider: IONOS
22287 - access_key_id: YOUR_ACCESS_KEY
22288 - secret_access_key: YOUR_SECRET_KEY
22289 - endpoint: s3-eu-central-1.ionoscloud.com
22290 Keep this "ionos-fra" remote?
22291 y) Yes this is OK (default)
22292 e) Edit this remote
22293 d) Delete this remote
22294 y/e/d> y
22295
22296 Done! Now you can try some commands (for macOS, use ./rclone instead of
22297 rclone).
22298
22299 1) Create a bucket (the name must be unique within the whole IONOS S3)
22300
22301 rclone mkdir ionos-fra:my-bucket
22302
22303 2) List available buckets
22304
22305 rclone lsd ionos-fra:
22306
22307 4) Copy a file from local to remote
22308
22309 rclone copy /Users/file.txt ionos-fra:my-bucket
22310
22311 3) List contents of a bucket
22312
22313 rclone ls ionos-fra:my-bucket
22314
22315 5) Copy a file from remote to local
22316
22317 rclone copy ionos-fra:my-bucket/file.txt
22318
22319 Minio
22320 Minio (https://minio.io/) is an object storage server built for cloud
22321 application developers and devops.
22322
22323 It is very easy to install and provides an S3 compatible server which
22324 can be used by rclone.
22325
22326 To use it, install Minio following the instructions here
22327 (https://docs.minio.io/docs/minio-quickstart-guide).
22328
22329 When it configures itself Minio will print something like this
22330
22331 Endpoint: http://192.168.1.106:9000 http://172.23.0.1:9000
22332 AccessKey: USWUXHGYZQYFYFFIT3RE
22333 SecretKey: MOJRH0mkL1IPauahWITSVvyDrQbEEIwljvmxdq03
22334 Region: us-east-1
22335 SQS ARNs: arn:minio:sqs:us-east-1:1:redis arn:minio:sqs:us-east-1:2:redis
22336
22337 Browser Access:
22338 http://192.168.1.106:9000 http://172.23.0.1:9000
22339
22340 Command-line Access: https://docs.minio.io/docs/minio-client-quickstart-guide
22341 $ mc config host add myminio http://192.168.1.106:9000 USWUXHGYZQYFYFFIT3RE MOJRH0mkL1IPauahWITSVvyDrQbEEIwljvmxdq03
22342
22343 Object API (Amazon S3 compatible):
22344 Go: https://docs.minio.io/docs/golang-client-quickstart-guide
22345 Java: https://docs.minio.io/docs/java-client-quickstart-guide
22346 Python: https://docs.minio.io/docs/python-client-quickstart-guide
22347 JavaScript: https://docs.minio.io/docs/javascript-client-quickstart-guide
22348 .NET: https://docs.minio.io/docs/dotnet-client-quickstart-guide
22349
22350 Drive Capacity: 26 GiB Free, 165 GiB Total
22351
22352 These details need to go into rclone config like this. Note that it is
22353 important to put the region in as stated above.
22354
22355 env_auth> 1
22356 access_key_id> USWUXHGYZQYFYFFIT3RE
22357 secret_access_key> MOJRH0mkL1IPauahWITSVvyDrQbEEIwljvmxdq03
22358 region> us-east-1
22359 endpoint> http://192.168.1.106:9000
22360 location_constraint>
22361 server_side_encryption>
22362
22363 Which makes the config file look like this
22364
22365 [minio]
22366 type = s3
22367 provider = Minio
22368 env_auth = false
22369 access_key_id = USWUXHGYZQYFYFFIT3RE
22370 secret_access_key = MOJRH0mkL1IPauahWITSVvyDrQbEEIwljvmxdq03
22371 region = us-east-1
22372 endpoint = http://192.168.1.106:9000
22373 location_constraint =
22374 server_side_encryption =
22375
22376 So once set up, for example, to copy files into a bucket
22377
22378 rclone copy /path/to/files minio:bucket
22379
22380 Qiniu Cloud Object Storage (Kodo)
22381 Qiniu Cloud Object Storage (Kodo) (https://www.qiniu.com/en/prod‐
22382 ucts/kodo), a completely independent-researched core technology which
22383 is proven by repeated customer experience has occupied absolute leading
22384 market leader position. Kodo can be widely applied to mass data man‐
22385 agement.
22386
22387 To configure access to Qiniu Kodo, follow the steps below:
22388
22389 1. Run rclone config and select n for a new remote.
22390
22391 rclone config
22392 No remotes found, make a new one?
22393 n) New remote
22394 s) Set configuration password
22395 q) Quit config
22396 n/s/q> n
22397
22398 2. Give the name of the configuration. For example, name it 'qiniu'.
22399
22400 name> qiniu
22401
22402 3. Select s3 storage.
22403
22404 Choose a number from below, or type in your own value
22405 1 / 1Fichier
22406 \ (fichier)
22407 2 / Akamai NetStorage
22408 \ (netstorage)
22409 3 / Alias for an existing remote
22410 \ (alias)
22411 4 / Amazon Drive
22412 \ (amazon cloud drive)
22413 5 / Amazon S3 Compliant Storage Providers including AWS, Alibaba, Ceph, China Mobile, Cloudflare, ArvanCloud, Digital Ocean, Dreamhost, Huawei OBS, IBM COS, IDrive e2, Lyve Cloud, Minio, Netease, RackCorp, Scaleway, SeaweedFS, StackPath, Storj, Tencent COS, Qiniu and Wasabi
22414 \ (s3)
22415 [snip]
22416 Storage> s3
22417
22418 4. Select Qiniu provider.
22419
22420 Choose a number from below, or type in your own value
22421 1 / Amazon Web Services (AWS) S3
22422 \ "AWS"
22423 [snip]
22424 22 / Qiniu Object Storage (Kodo)
22425 \ (Qiniu)
22426 [snip]
22427 provider> Qiniu
22428
22429 5. Enter your SecretId and SecretKey of Qiniu Kodo.
22430
22431 Get AWS credentials from runtime (environment variables or EC2/ECS meta data if no env vars).
22432 Only applies if access_key_id and secret_access_key is blank.
22433 Enter a boolean value (true or false). Press Enter for the default ("false").
22434 Choose a number from below, or type in your own value
22435 1 / Enter AWS credentials in the next step
22436 \ "false"
22437 2 / Get AWS credentials from the environment (env vars or IAM)
22438 \ "true"
22439 env_auth> 1
22440 AWS Access Key ID.
22441 Leave blank for anonymous access or runtime credentials.
22442 Enter a string value. Press Enter for the default ("").
22443 access_key_id> AKIDxxxxxxxxxx
22444 AWS Secret Access Key (password)
22445 Leave blank for anonymous access or runtime credentials.
22446 Enter a string value. Press Enter for the default ("").
22447 secret_access_key> xxxxxxxxxxx
22448
22449 6. Select endpoint for Qiniu Kodo. This is the standard endpoint for
22450 different region.
22451
22452 / The default endpoint - a good choice if you are unsure.
22453 1 | East China Region 1.
22454 | Needs location constraint cn-east-1.
22455 \ (cn-east-1)
22456 / East China Region 2.
22457 2 | Needs location constraint cn-east-2.
22458 \ (cn-east-2)
22459 / North China Region 1.
22460 3 | Needs location constraint cn-north-1.
22461 \ (cn-north-1)
22462 / South China Region 1.
22463 4 | Needs location constraint cn-south-1.
22464 \ (cn-south-1)
22465 / North America Region.
22466 5 | Needs location constraint us-north-1.
22467 \ (us-north-1)
22468 / Southeast Asia Region 1.
22469 6 | Needs location constraint ap-southeast-1.
22470 \ (ap-southeast-1)
22471 / Northeast Asia Region 1.
22472 7 | Needs location constraint ap-northeast-1.
22473 \ (ap-northeast-1)
22474 [snip]
22475 endpoint> 1
22476
22477 Option endpoint.
22478 Endpoint for Qiniu Object Storage.
22479 Choose a number from below, or type in your own value.
22480 Press Enter to leave empty.
22481 1 / East China Endpoint 1
22482 \ (s3-cn-east-1.qiniucs.com)
22483 2 / East China Endpoint 2
22484 \ (s3-cn-east-2.qiniucs.com)
22485 3 / North China Endpoint 1
22486 \ (s3-cn-north-1.qiniucs.com)
22487 4 / South China Endpoint 1
22488 \ (s3-cn-south-1.qiniucs.com)
22489 5 / North America Endpoint 1
22490 \ (s3-us-north-1.qiniucs.com)
22491 6 / Southeast Asia Endpoint 1
22492 \ (s3-ap-southeast-1.qiniucs.com)
22493 7 / Northeast Asia Endpoint 1
22494 \ (s3-ap-northeast-1.qiniucs.com)
22495 endpoint> 1
22496
22497 Option location_constraint.
22498 Location constraint - must be set to match the Region.
22499 Used when creating buckets only.
22500 Choose a number from below, or type in your own value.
22501 Press Enter to leave empty.
22502 1 / East China Region 1
22503 \ (cn-east-1)
22504 2 / East China Region 2
22505 \ (cn-east-2)
22506 3 / North China Region 1
22507 \ (cn-north-1)
22508 4 / South China Region 1
22509 \ (cn-south-1)
22510 5 / North America Region 1
22511 \ (us-north-1)
22512 6 / Southeast Asia Region 1
22513 \ (ap-southeast-1)
22514 7 / Northeast Asia Region 1
22515 \ (ap-northeast-1)
22516 location_constraint> 1
22517
22518 7. Choose acl and storage class.
22519
22520 Note that this ACL is applied when server-side copying objects as S3
22521 doesn't copy the ACL from the source but rather writes a fresh one.
22522 Enter a string value. Press Enter for the default ("").
22523 Choose a number from below, or type in your own value
22524 / Owner gets FULL_CONTROL.
22525 1 | No one else has access rights (default).
22526 \ (private)
22527 / Owner gets FULL_CONTROL.
22528 2 | The AllUsers group gets READ access.
22529 \ (public-read)
22530 [snip]
22531 acl> 2
22532 The storage class to use when storing new objects in Tencent COS.
22533 Enter a string value. Press Enter for the default ("").
22534 Choose a number from below, or type in your own value
22535 1 / Standard storage class
22536 \ (STANDARD)
22537 2 / Infrequent access storage mode
22538 \ (LINE)
22539 3 / Archive storage mode
22540 \ (GLACIER)
22541 4 / Deep archive storage mode
22542 \ (DEEP_ARCHIVE)
22543 [snip]
22544 storage_class> 1
22545 Edit advanced config? (y/n)
22546 y) Yes
22547 n) No (default)
22548 y/n> n
22549 Remote config
22550 --------------------
22551 [qiniu]
22552 - type: s3
22553 - provider: Qiniu
22554 - access_key_id: xxx
22555 - secret_access_key: xxx
22556 - region: cn-east-1
22557 - endpoint: s3-cn-east-1.qiniucs.com
22558 - location_constraint: cn-east-1
22559 - acl: public-read
22560 - storage_class: STANDARD
22561 --------------------
22562 y) Yes this is OK (default)
22563 e) Edit this remote
22564 d) Delete this remote
22565 y/e/d> y
22566 Current remotes:
22567
22568 Name Type
22569 ==== ====
22570 qiniu s3
22571
22572 RackCorp
22573 RackCorp Object Storage (https://www.rackcorp.com/storage/s3storage) is
22574 an S3 compatible object storage platform from your friendly cloud
22575 provider RackCorp. The service is fast, reliable, well priced and lo‐
22576 cated in many strategic locations unserviced by others, to ensure you
22577 can maintain data sovereignty.
22578
22579 Before you can use RackCorp Object Storage, you'll need to "sign up
22580 (https://www.rackcorp.com/signup)" for an account on our "portal
22581 (https://portal.rackcorp.com)". Next you can create an access key, a
22582 secret key and buckets, in your location of choice with ease. These
22583 details are required for the next steps of configuration, when rclone
22584 config asks for your access_key_id and secret_access_key.
22585
22586 Your config should end up looking a bit like this:
22587
22588 [RCS3-demo-config]
22589 type = s3
22590 provider = RackCorp
22591 env_auth = true
22592 access_key_id = YOURACCESSKEY
22593 secret_access_key = YOURSECRETACCESSKEY
22594 region = au-nsw
22595 endpoint = s3.rackcorp.com
22596 location_constraint = au-nsw
22597
22598 Scaleway
22599 Scaleway (https://www.scaleway.com/object-storage/) The Object Storage
22600 platform allows you to store anything from backups, logs and web assets
22601 to documents and photos. Files can be dropped from the Scaleway con‐
22602 sole or transferred through our API and CLI or using any S3-compatible
22603 tool.
22604
22605 Scaleway provides an S3 interface which can be configured for use with
22606 rclone like this:
22607
22608 [scaleway]
22609 type = s3
22610 provider = Scaleway
22611 env_auth = false
22612 endpoint = s3.nl-ams.scw.cloud
22613 access_key_id = SCWXXXXXXXXXXXXXX
22614 secret_access_key = 1111111-2222-3333-44444-55555555555555
22615 region = nl-ams
22616 location_constraint =
22617 acl = private
22618 server_side_encryption =
22619 storage_class =
22620
22621 C14 Cold Storage (https://www.online.net/en/storage/c14-cold-storage)
22622 is the low-cost S3 Glacier alternative from Scaleway and it works the
22623 same way as on S3 by accepting the "GLACIER" storage_class. So you can
22624 configure your remote with the storage_class = GLACIER option to upload
22625 directly to C14. Don't forget that in this state you can't read files
22626 back after, you will need to restore them to "STANDARD" storage_class
22627 first before being able to read them (see "restore" section above)
22628
22629 Seagate Lyve Cloud
22630 Seagate Lyve Cloud (https://www.seagate.com/gb/en/services/cloud/stor‐
22631 age/) is an S3 compatible object storage platform from Seagate
22632 (https://seagate.com/) intended for enterprise use.
22633
22634 Here is a config run through for a remote called remote - you may
22635 choose a different name of course. Note that to create an access key
22636 and secret key you will need to create a service account first.
22637
22638 $ rclone config
22639 No remotes found, make a new one?
22640 n) New remote
22641 s) Set configuration password
22642 q) Quit config
22643 n/s/q> n
22644 name> remote
22645
22646 Choose s3 backend
22647
22648 Type of storage to configure.
22649 Choose a number from below, or type in your own value.
22650 [snip]
22651 XX / Amazon S3 Compliant Storage Providers including AWS, Alibaba, Ceph, ChinaMobile, ArvanCloud, Digital Ocean, Dreamhost, Huawei OBS, IBM COS, Lyve Cloud, Minio, RackCorp, SeaweedFS, and Tencent COS
22652 \ (s3)
22653 [snip]
22654 Storage> s3
22655
22656 Choose LyveCloud as S3 provider
22657
22658 Choose your S3 provider.
22659 Choose a number from below, or type in your own value.
22660 Press Enter to leave empty.
22661 [snip]
22662 XX / Seagate Lyve Cloud
22663 \ (LyveCloud)
22664 [snip]
22665 provider> LyveCloud
22666
22667 Take the default (just press enter) to enter access key and secret in
22668 the config file.
22669
22670 Get AWS credentials from runtime (environment variables or EC2/ECS meta data if no env vars).
22671 Only applies if access_key_id and secret_access_key is blank.
22672 Choose a number from below, or type in your own boolean value (true or false).
22673 Press Enter for the default (false).
22674 1 / Enter AWS credentials in the next step.
22675 \ (false)
22676 2 / Get AWS credentials from the environment (env vars or IAM).
22677 \ (true)
22678 env_auth>
22679
22680 AWS Access Key ID.
22681 Leave blank for anonymous access or runtime credentials.
22682 Enter a value. Press Enter to leave empty.
22683 access_key_id> XXX
22684
22685 AWS Secret Access Key (password).
22686 Leave blank for anonymous access or runtime credentials.
22687 Enter a value. Press Enter to leave empty.
22688 secret_access_key> YYY
22689
22690 Leave region blank
22691
22692 Region to connect to.
22693 Leave blank if you are using an S3 clone and you don't have a region.
22694 Choose a number from below, or type in your own value.
22695 Press Enter to leave empty.
22696 / Use this if unsure.
22697 1 | Will use v4 signatures and an empty region.
22698 \ ()
22699 / Use this only if v4 signatures don't work.
22700 2 | E.g. pre Jewel/v10 CEPH.
22701 \ (other-v2-signature)
22702 region>
22703
22704 Choose an endpoint from the list
22705
22706 Endpoint for S3 API.
22707 Required when using an S3 clone.
22708 Choose a number from below, or type in your own value.
22709 Press Enter to leave empty.
22710 1 / Seagate Lyve Cloud US East 1 (Virginia)
22711 \ (s3.us-east-1.lyvecloud.seagate.com)
22712 2 / Seagate Lyve Cloud US West 1 (California)
22713 \ (s3.us-west-1.lyvecloud.seagate.com)
22714 3 / Seagate Lyve Cloud AP Southeast 1 (Singapore)
22715 \ (s3.ap-southeast-1.lyvecloud.seagate.com)
22716 endpoint> 1
22717
22718 Leave location constraint blank
22719
22720 Location constraint - must be set to match the Region.
22721 Leave blank if not sure. Used when creating buckets only.
22722 Enter a value. Press Enter to leave empty.
22723 location_constraint>
22724
22725 Choose default ACL (private).
22726
22727 Canned ACL used when creating buckets and storing or copying objects.
22728 This ACL is used for creating objects and if bucket_acl isn't set, for creating buckets too.
22729 For more info visit https://docs.aws.amazon.com/AmazonS3/latest/dev/acl-overview.html#canned-acl
22730 Note that this ACL is applied when server-side copying objects as S3
22731 doesn't copy the ACL from the source but rather writes a fresh one.
22732 Choose a number from below, or type in your own value.
22733 Press Enter to leave empty.
22734 / Owner gets FULL_CONTROL.
22735 1 | No one else has access rights (default).
22736 \ (private)
22737 [snip]
22738 acl>
22739
22740 And the config file should end up looking like this:
22741
22742 [remote]
22743 type = s3
22744 provider = LyveCloud
22745 access_key_id = XXX
22746 secret_access_key = YYY
22747 endpoint = s3.us-east-1.lyvecloud.seagate.com
22748
22749 SeaweedFS
22750 SeaweedFS (https://github.com/chrislusf/seaweedfs/) is a distributed
22751 storage system for blobs, objects, files, and data lake, with O(1) disk
22752 seek and a scalable file metadata store. It has an S3 compatible ob‐
22753 ject storage interface. SeaweedFS can also act as a gateway to remote
22754 S3 compatible object store (https://github.com/chrislusf/seaweedfs/wi‐
22755 ki/Gateway-to-Remote-Object-Storage) to cache data and metadata with
22756 asynchronous write back, for fast local speed and minimize access cost.
22757
22758 Assuming the SeaweedFS are configured with weed shell as such:
22759
22760 > s3.bucket.create -name foo
22761 > s3.configure -access_key=any -secret_key=any -buckets=foo -user=me -actions=Read,Write,List,Tagging,Admin -apply
22762 {
22763 "identities": [
22764 {
22765 "name": "me",
22766 "credentials": [
22767 {
22768 "accessKey": "any",
22769 "secretKey": "any"
22770 }
22771 ],
22772 "actions": [
22773 "Read:foo",
22774 "Write:foo",
22775 "List:foo",
22776 "Tagging:foo",
22777 "Admin:foo"
22778 ]
22779 }
22780 ]
22781 }
22782
22783 To use rclone with SeaweedFS, above configuration should end up with
22784 something like this in your config:
22785
22786 [seaweedfs_s3]
22787 type = s3
22788 provider = SeaweedFS
22789 access_key_id = any
22790 secret_access_key = any
22791 endpoint = localhost:8333
22792
22793 So once set up, for example to copy files into a bucket
22794
22795 rclone copy /path/to/files seaweedfs_s3:foo
22796
22797 Wasabi
22798 Wasabi (https://wasabi.com) is a cloud-based object storage service for
22799 a broad range of applications and use cases. Wasabi is designed for
22800 individuals and organizations that require a high-performance, reli‐
22801 able, and secure data storage infrastructure at minimal cost.
22802
22803 Wasabi provides an S3 interface which can be configured for use with
22804 rclone like this.
22805
22806 No remotes found, make a new one?
22807 n) New remote
22808 s) Set configuration password
22809 n/s> n
22810 name> wasabi
22811 Type of storage to configure.
22812 Choose a number from below, or type in your own value
22813 [snip]
22814 XX / Amazon S3 (also Dreamhost, Ceph, ChinaMobile, ArvanCloud, Minio)
22815 \ "s3"
22816 [snip]
22817 Storage> s3
22818 Get AWS credentials from runtime (environment variables or EC2/ECS meta data if no env vars). Only applies if access_key_id and secret_access_key is blank.
22819 Choose a number from below, or type in your own value
22820 1 / Enter AWS credentials in the next step
22821 \ "false"
22822 2 / Get AWS credentials from the environment (env vars or IAM)
22823 \ "true"
22824 env_auth> 1
22825 AWS Access Key ID - leave blank for anonymous access or runtime credentials.
22826 access_key_id> YOURACCESSKEY
22827 AWS Secret Access Key (password) - leave blank for anonymous access or runtime credentials.
22828 secret_access_key> YOURSECRETACCESSKEY
22829 Region to connect to.
22830 Choose a number from below, or type in your own value
22831 / The default endpoint - a good choice if you are unsure.
22832 1 | US Region, Northern Virginia, or Pacific Northwest.
22833 | Leave location constraint empty.
22834 \ "us-east-1"
22835 [snip]
22836 region> us-east-1
22837 Endpoint for S3 API.
22838 Leave blank if using AWS to use the default endpoint for the region.
22839 Specify if using an S3 clone such as Ceph.
22840 endpoint> s3.wasabisys.com
22841 Location constraint - must be set to match the Region. Used when creating buckets only.
22842 Choose a number from below, or type in your own value
22843 1 / Empty for US Region, Northern Virginia, or Pacific Northwest.
22844 \ ""
22845 [snip]
22846 location_constraint>
22847 Canned ACL used when creating buckets and/or storing objects in S3.
22848 For more info visit https://docs.aws.amazon.com/AmazonS3/latest/dev/acl-overview.html#canned-acl
22849 Choose a number from below, or type in your own value
22850 1 / Owner gets FULL_CONTROL. No one else has access rights (default).
22851 \ "private"
22852 [snip]
22853 acl>
22854 The server-side encryption algorithm used when storing this object in S3.
22855 Choose a number from below, or type in your own value
22856 1 / None
22857 \ ""
22858 2 / AES256
22859 \ "AES256"
22860 server_side_encryption>
22861 The storage class to use when storing objects in S3.
22862 Choose a number from below, or type in your own value
22863 1 / Default
22864 \ ""
22865 2 / Standard storage class
22866 \ "STANDARD"
22867 3 / Reduced redundancy storage class
22868 \ "REDUCED_REDUNDANCY"
22869 4 / Standard Infrequent Access storage class
22870 \ "STANDARD_IA"
22871 storage_class>
22872 Remote config
22873 --------------------
22874 [wasabi]
22875 env_auth = false
22876 access_key_id = YOURACCESSKEY
22877 secret_access_key = YOURSECRETACCESSKEY
22878 region = us-east-1
22879 endpoint = s3.wasabisys.com
22880 location_constraint =
22881 acl =
22882 server_side_encryption =
22883 storage_class =
22884 --------------------
22885 y) Yes this is OK
22886 e) Edit this remote
22887 d) Delete this remote
22888 y/e/d> y
22889
22890 This will leave the config file looking like this.
22891
22892 [wasabi]
22893 type = s3
22894 provider = Wasabi
22895 env_auth = false
22896 access_key_id = YOURACCESSKEY
22897 secret_access_key = YOURSECRETACCESSKEY
22898 region =
22899 endpoint = s3.wasabisys.com
22900 location_constraint =
22901 acl =
22902 server_side_encryption =
22903 storage_class =
22904
22905 Alibaba OSS
22906 Here is an example of making an Alibaba Cloud (Aliyun) OSS
22907 (https://www.alibabacloud.com/product/oss/) configuration. First run:
22908
22909 rclone config
22910
22911 This will guide you through an interactive setup process.
22912
22913 No remotes found, make a new one?
22914 n) New remote
22915 s) Set configuration password
22916 q) Quit config
22917 n/s/q> n
22918 name> oss
22919 Type of storage to configure.
22920 Enter a string value. Press Enter for the default ("").
22921 Choose a number from below, or type in your own value
22922 [snip]
22923 4 / Amazon S3 Compliant Storage Providers including AWS, Alibaba, Ceph, ChinaMobile, ArvanCloud, Digital Ocean, Dreamhost, Huawei OBS, IBM COS, Minio, and Tencent COS
22924 \ "s3"
22925 [snip]
22926 Storage> s3
22927 Choose your S3 provider.
22928 Enter a string value. Press Enter for the default ("").
22929 Choose a number from below, or type in your own value
22930 1 / Amazon Web Services (AWS) S3
22931 \ "AWS"
22932 2 / Alibaba Cloud Object Storage System (OSS) formerly Aliyun
22933 \ "Alibaba"
22934 3 / Ceph Object Storage
22935 \ "Ceph"
22936 [snip]
22937 provider> Alibaba
22938 Get AWS credentials from runtime (environment variables or EC2/ECS meta data if no env vars).
22939 Only applies if access_key_id and secret_access_key is blank.
22940 Enter a boolean value (true or false). Press Enter for the default ("false").
22941 Choose a number from below, or type in your own value
22942 1 / Enter AWS credentials in the next step
22943 \ "false"
22944 2 / Get AWS credentials from the environment (env vars or IAM)
22945 \ "true"
22946 env_auth> 1
22947 AWS Access Key ID.
22948 Leave blank for anonymous access or runtime credentials.
22949 Enter a string value. Press Enter for the default ("").
22950 access_key_id> accesskeyid
22951 AWS Secret Access Key (password)
22952 Leave blank for anonymous access or runtime credentials.
22953 Enter a string value. Press Enter for the default ("").
22954 secret_access_key> secretaccesskey
22955 Endpoint for OSS API.
22956 Enter a string value. Press Enter for the default ("").
22957 Choose a number from below, or type in your own value
22958 1 / East China 1 (Hangzhou)
22959 \ "oss-cn-hangzhou.aliyuncs.com"
22960 2 / East China 2 (Shanghai)
22961 \ "oss-cn-shanghai.aliyuncs.com"
22962 3 / North China 1 (Qingdao)
22963 \ "oss-cn-qingdao.aliyuncs.com"
22964 [snip]
22965 endpoint> 1
22966 Canned ACL used when creating buckets and storing or copying objects.
22967
22968 Note that this ACL is applied when server-side copying objects as S3
22969 doesn't copy the ACL from the source but rather writes a fresh one.
22970 Enter a string value. Press Enter for the default ("").
22971 Choose a number from below, or type in your own value
22972 1 / Owner gets FULL_CONTROL. No one else has access rights (default).
22973 \ "private"
22974 2 / Owner gets FULL_CONTROL. The AllUsers group gets READ access.
22975 \ "public-read"
22976 / Owner gets FULL_CONTROL. The AllUsers group gets READ and WRITE access.
22977 [snip]
22978 acl> 1
22979 The storage class to use when storing new objects in OSS.
22980 Enter a string value. Press Enter for the default ("").
22981 Choose a number from below, or type in your own value
22982 1 / Default
22983 \ ""
22984 2 / Standard storage class
22985 \ "STANDARD"
22986 3 / Archive storage mode.
22987 \ "GLACIER"
22988 4 / Infrequent access storage mode.
22989 \ "STANDARD_IA"
22990 storage_class> 1
22991 Edit advanced config? (y/n)
22992 y) Yes
22993 n) No
22994 y/n> n
22995 Remote config
22996 --------------------
22997 [oss]
22998 type = s3
22999 provider = Alibaba
23000 env_auth = false
23001 access_key_id = accesskeyid
23002 secret_access_key = secretaccesskey
23003 endpoint = oss-cn-hangzhou.aliyuncs.com
23004 acl = private
23005 storage_class = Standard
23006 --------------------
23007 y) Yes this is OK
23008 e) Edit this remote
23009 d) Delete this remote
23010 y/e/d> y
23011
23012 China Mobile Ecloud Elastic Object Storage (EOS)
23013 Here is an example of making an China Mobile Ecloud Elastic Object
23014 Storage (EOS) (https:///ecloud.10086.cn/home/product-introduction/eos/)
23015 configuration. First run:
23016
23017 rclone config
23018
23019 This will guide you through an interactive setup process.
23020
23021 No remotes found, make a new one?
23022 n) New remote
23023 s) Set configuration password
23024 q) Quit config
23025 n/s/q> n
23026 name> ChinaMobile
23027 Option Storage.
23028 Type of storage to configure.
23029 Choose a number from below, or type in your own value.
23030 ...
23031 5 / Amazon S3 Compliant Storage Providers including AWS, Alibaba, Ceph, ChinaMobile, Digital Ocean, Dreamhost, Huawei OBS, IBM COS, Lyve Cloud, Minio, RackCorp, SeaweedFS, and Tencent COS
23032 \ (s3)
23033 ...
23034 Storage> s3
23035 Option provider.
23036 Choose your S3 provider.
23037 Choose a number from below, or type in your own value.
23038 Press Enter to leave empty.
23039 ...
23040 4 / China Mobile Ecloud Elastic Object Storage (EOS)
23041 \ (ChinaMobile)
23042 ...
23043 provider> ChinaMobile
23044 Option env_auth.
23045 Get AWS credentials from runtime (environment variables or EC2/ECS meta data if no env vars).
23046 Only applies if access_key_id and secret_access_key is blank.
23047 Choose a number from below, or type in your own boolean value (true or false).
23048 Press Enter for the default (false).
23049 1 / Enter AWS credentials in the next step.
23050 \ (false)
23051 2 / Get AWS credentials from the environment (env vars or IAM).
23052 \ (true)
23053 env_auth>
23054 Option access_key_id.
23055 AWS Access Key ID.
23056 Leave blank for anonymous access or runtime credentials.
23057 Enter a value. Press Enter to leave empty.
23058 access_key_id> accesskeyid
23059 Option secret_access_key.
23060 AWS Secret Access Key (password).
23061 Leave blank for anonymous access or runtime credentials.
23062 Enter a value. Press Enter to leave empty.
23063 secret_access_key> secretaccesskey
23064 Option endpoint.
23065 Endpoint for China Mobile Ecloud Elastic Object Storage (EOS) API.
23066 Choose a number from below, or type in your own value.
23067 Press Enter to leave empty.
23068 / The default endpoint - a good choice if you are unsure.
23069 1 | East China (Suzhou)
23070 \ (eos-wuxi-1.cmecloud.cn)
23071 2 / East China (Jinan)
23072 \ (eos-jinan-1.cmecloud.cn)
23073 3 / East China (Hangzhou)
23074 \ (eos-ningbo-1.cmecloud.cn)
23075 4 / East China (Shanghai-1)
23076 \ (eos-shanghai-1.cmecloud.cn)
23077 5 / Central China (Zhengzhou)
23078 \ (eos-zhengzhou-1.cmecloud.cn)
23079 6 / Central China (Changsha-1)
23080 \ (eos-hunan-1.cmecloud.cn)
23081 7 / Central China (Changsha-2)
23082 \ (eos-zhuzhou-1.cmecloud.cn)
23083 8 / South China (Guangzhou-2)
23084 \ (eos-guangzhou-1.cmecloud.cn)
23085 9 / South China (Guangzhou-3)
23086 \ (eos-dongguan-1.cmecloud.cn)
23087 10 / North China (Beijing-1)
23088 \ (eos-beijing-1.cmecloud.cn)
23089 11 / North China (Beijing-2)
23090 \ (eos-beijing-2.cmecloud.cn)
23091 12 / North China (Beijing-3)
23092 \ (eos-beijing-4.cmecloud.cn)
23093 13 / North China (Huhehaote)
23094 \ (eos-huhehaote-1.cmecloud.cn)
23095 14 / Southwest China (Chengdu)
23096 \ (eos-chengdu-1.cmecloud.cn)
23097 15 / Southwest China (Chongqing)
23098 \ (eos-chongqing-1.cmecloud.cn)
23099 16 / Southwest China (Guiyang)
23100 \ (eos-guiyang-1.cmecloud.cn)
23101 17 / Nouthwest China (Xian)
23102 \ (eos-xian-1.cmecloud.cn)
23103 18 / Yunnan China (Kunming)
23104 \ (eos-yunnan.cmecloud.cn)
23105 19 / Yunnan China (Kunming-2)
23106 \ (eos-yunnan-2.cmecloud.cn)
23107 20 / Tianjin China (Tianjin)
23108 \ (eos-tianjin-1.cmecloud.cn)
23109 21 / Jilin China (Changchun)
23110 \ (eos-jilin-1.cmecloud.cn)
23111 22 / Hubei China (Xiangyan)
23112 \ (eos-hubei-1.cmecloud.cn)
23113 23 / Jiangxi China (Nanchang)
23114 \ (eos-jiangxi-1.cmecloud.cn)
23115 24 / Gansu China (Lanzhou)
23116 \ (eos-gansu-1.cmecloud.cn)
23117 25 / Shanxi China (Taiyuan)
23118 \ (eos-shanxi-1.cmecloud.cn)
23119 26 / Liaoning China (Shenyang)
23120 \ (eos-liaoning-1.cmecloud.cn)
23121 27 / Hebei China (Shijiazhuang)
23122 \ (eos-hebei-1.cmecloud.cn)
23123 28 / Fujian China (Xiamen)
23124 \ (eos-fujian-1.cmecloud.cn)
23125 29 / Guangxi China (Nanning)
23126 \ (eos-guangxi-1.cmecloud.cn)
23127 30 / Anhui China (Huainan)
23128 \ (eos-anhui-1.cmecloud.cn)
23129 endpoint> 1
23130 Option location_constraint.
23131 Location constraint - must match endpoint.
23132 Used when creating buckets only.
23133 Choose a number from below, or type in your own value.
23134 Press Enter to leave empty.
23135 1 / East China (Suzhou)
23136 \ (wuxi1)
23137 2 / East China (Jinan)
23138 \ (jinan1)
23139 3 / East China (Hangzhou)
23140 \ (ningbo1)
23141 4 / East China (Shanghai-1)
23142 \ (shanghai1)
23143 5 / Central China (Zhengzhou)
23144 \ (zhengzhou1)
23145 6 / Central China (Changsha-1)
23146 \ (hunan1)
23147 7 / Central China (Changsha-2)
23148 \ (zhuzhou1)
23149 8 / South China (Guangzhou-2)
23150 \ (guangzhou1)
23151 9 / South China (Guangzhou-3)
23152 \ (dongguan1)
23153 10 / North China (Beijing-1)
23154 \ (beijing1)
23155 11 / North China (Beijing-2)
23156 \ (beijing2)
23157 12 / North China (Beijing-3)
23158 \ (beijing4)
23159 13 / North China (Huhehaote)
23160 \ (huhehaote1)
23161 14 / Southwest China (Chengdu)
23162 \ (chengdu1)
23163 15 / Southwest China (Chongqing)
23164 \ (chongqing1)
23165 16 / Southwest China (Guiyang)
23166 \ (guiyang1)
23167 17 / Nouthwest China (Xian)
23168 \ (xian1)
23169 18 / Yunnan China (Kunming)
23170 \ (yunnan)
23171 19 / Yunnan China (Kunming-2)
23172 \ (yunnan2)
23173 20 / Tianjin China (Tianjin)
23174 \ (tianjin1)
23175 21 / Jilin China (Changchun)
23176 \ (jilin1)
23177 22 / Hubei China (Xiangyan)
23178 \ (hubei1)
23179 23 / Jiangxi China (Nanchang)
23180 \ (jiangxi1)
23181 24 / Gansu China (Lanzhou)
23182 \ (gansu1)
23183 25 / Shanxi China (Taiyuan)
23184 \ (shanxi1)
23185 26 / Liaoning China (Shenyang)
23186 \ (liaoning1)
23187 27 / Hebei China (Shijiazhuang)
23188 \ (hebei1)
23189 28 / Fujian China (Xiamen)
23190 \ (fujian1)
23191 29 / Guangxi China (Nanning)
23192 \ (guangxi1)
23193 30 / Anhui China (Huainan)
23194 \ (anhui1)
23195 location_constraint> 1
23196 Option acl.
23197 Canned ACL used when creating buckets and storing or copying objects.
23198 This ACL is used for creating objects and if bucket_acl isn't set, for creating buckets too.
23199 For more info visit https://docs.aws.amazon.com/AmazonS3/latest/dev/acl-overview.html#canned-acl
23200 Note that this ACL is applied when server-side copying objects as S3
23201 doesn't copy the ACL from the source but rather writes a fresh one.
23202 Choose a number from below, or type in your own value.
23203 Press Enter to leave empty.
23204 / Owner gets FULL_CONTROL.
23205 1 | No one else has access rights (default).
23206 \ (private)
23207 / Owner gets FULL_CONTROL.
23208 2 | The AllUsers group gets READ access.
23209 \ (public-read)
23210 / Owner gets FULL_CONTROL.
23211 3 | The AllUsers group gets READ and WRITE access.
23212 | Granting this on a bucket is generally not recommended.
23213 \ (public-read-write)
23214 / Owner gets FULL_CONTROL.
23215 4 | The AuthenticatedUsers group gets READ access.
23216 \ (authenticated-read)
23217 / Object owner gets FULL_CONTROL.
23218 acl> private
23219 Option server_side_encryption.
23220 The server-side encryption algorithm used when storing this object in S3.
23221 Choose a number from below, or type in your own value.
23222 Press Enter to leave empty.
23223 1 / None
23224 \ ()
23225 2 / AES256
23226 \ (AES256)
23227 server_side_encryption>
23228 Option storage_class.
23229 The storage class to use when storing new objects in ChinaMobile.
23230 Choose a number from below, or type in your own value.
23231 Press Enter to leave empty.
23232 1 / Default
23233 \ ()
23234 2 / Standard storage class
23235 \ (STANDARD)
23236 3 / Archive storage mode
23237 \ (GLACIER)
23238 4 / Infrequent access storage mode
23239 \ (STANDARD_IA)
23240 storage_class>
23241 Edit advanced config?
23242 y) Yes
23243 n) No (default)
23244 y/n> n
23245 --------------------
23246 [ChinaMobile]
23247 type = s3
23248 provider = ChinaMobile
23249 access_key_id = accesskeyid
23250 secret_access_key = secretaccesskey
23251 endpoint = eos-wuxi-1.cmecloud.cn
23252 location_constraint = wuxi1
23253 acl = private
23254 --------------------
23255 y) Yes this is OK (default)
23256 e) Edit this remote
23257 d) Delete this remote
23258 y/e/d> y
23259
23260 ArvanCloud
23261 ArvanCloud (https://www.arvancloud.com/en/products/cloud-storage) Ar‐
23262 vanCloud Object Storage goes beyond the limited traditional file stor‐
23263 age. It gives you access to backup and archived files and allows shar‐
23264 ing. Files like profile image in the app, images sent by users or
23265 scanned documents can be stored securely and easily in our Object Stor‐
23266 age service.
23267
23268 ArvanCloud provides an S3 interface which can be configured for use
23269 with rclone like this.
23270
23271 No remotes found, make a new one?
23272 n) New remote
23273 s) Set configuration password
23274 n/s> n
23275 name> ArvanCloud
23276 Type of storage to configure.
23277 Choose a number from below, or type in your own value
23278 [snip]
23279 XX / Amazon S3 (also Dreamhost, Ceph, ChinaMobile, ArvanCloud, Minio)
23280 \ "s3"
23281 [snip]
23282 Storage> s3
23283 Get AWS credentials from runtime (environment variables or EC2/ECS meta data if no env vars). Only applies if access_key_id and secret_access_key is blank.
23284 Choose a number from below, or type in your own value
23285 1 / Enter AWS credentials in the next step
23286 \ "false"
23287 2 / Get AWS credentials from the environment (env vars or IAM)
23288 \ "true"
23289 env_auth> 1
23290 AWS Access Key ID - leave blank for anonymous access or runtime credentials.
23291 access_key_id> YOURACCESSKEY
23292 AWS Secret Access Key (password) - leave blank for anonymous access or runtime credentials.
23293 secret_access_key> YOURSECRETACCESSKEY
23294 Region to connect to.
23295 Choose a number from below, or type in your own value
23296 / The default endpoint - a good choice if you are unsure.
23297 1 | US Region, Northern Virginia, or Pacific Northwest.
23298 | Leave location constraint empty.
23299 \ "us-east-1"
23300 [snip]
23301 region>
23302 Endpoint for S3 API.
23303 Leave blank if using ArvanCloud to use the default endpoint for the region.
23304 Specify if using an S3 clone such as Ceph.
23305 endpoint> s3.arvanstorage.com
23306 Location constraint - must be set to match the Region. Used when creating buckets only.
23307 Choose a number from below, or type in your own value
23308 1 / Empty for Iran-Tehran Region.
23309 \ ""
23310 [snip]
23311 location_constraint>
23312 Canned ACL used when creating buckets and/or storing objects in S3.
23313 For more info visit https://docs.aws.amazon.com/AmazonS3/latest/dev/acl-overview.html#canned-acl
23314 Choose a number from below, or type in your own value
23315 1 / Owner gets FULL_CONTROL. No one else has access rights (default).
23316 \ "private"
23317 [snip]
23318 acl>
23319 The server-side encryption algorithm used when storing this object in S3.
23320 Choose a number from below, or type in your own value
23321 1 / None
23322 \ ""
23323 2 / AES256
23324 \ "AES256"
23325 server_side_encryption>
23326 The storage class to use when storing objects in S3.
23327 Choose a number from below, or type in your own value
23328 1 / Default
23329 \ ""
23330 2 / Standard storage class
23331 \ "STANDARD"
23332 storage_class>
23333 Remote config
23334 --------------------
23335 [ArvanCloud]
23336 env_auth = false
23337 access_key_id = YOURACCESSKEY
23338 secret_access_key = YOURSECRETACCESSKEY
23339 region = ir-thr-at1
23340 endpoint = s3.arvanstorage.com
23341 location_constraint =
23342 acl =
23343 server_side_encryption =
23344 storage_class =
23345 --------------------
23346 y) Yes this is OK
23347 e) Edit this remote
23348 d) Delete this remote
23349 y/e/d> y
23350
23351 This will leave the config file looking like this.
23352
23353 [ArvanCloud]
23354 type = s3
23355 provider = ArvanCloud
23356 env_auth = false
23357 access_key_id = YOURACCESSKEY
23358 secret_access_key = YOURSECRETACCESSKEY
23359 region =
23360 endpoint = s3.arvanstorage.com
23361 location_constraint =
23362 acl =
23363 server_side_encryption =
23364 storage_class =
23365
23366 Tencent COS
23367 Tencent Cloud Object Storage (COS) (https://intl.cloud.ten‐
23368 cent.com/product/cos) is a distributed storage service offered by Ten‐
23369 cent Cloud for unstructured data. It is secure, stable, massive, con‐
23370 venient, low-delay and low-cost.
23371
23372 To configure access to Tencent COS, follow the steps below:
23373
23374 1. Run rclone config and select n for a new remote.
23375
23376 rclone config
23377 No remotes found, make a new one?
23378 n) New remote
23379 s) Set configuration password
23380 q) Quit config
23381 n/s/q> n
23382
23383 2. Give the name of the configuration. For example, name it 'cos'.
23384
23385 name> cos
23386
23387 3. Select s3 storage.
23388
23389 Choose a number from below, or type in your own value
23390 1 / 1Fichier
23391 \ "fichier"
23392 2 / Alias for an existing remote
23393 \ "alias"
23394 3 / Amazon Drive
23395 \ "amazon cloud drive"
23396 4 / Amazon S3 Compliant Storage Providers including AWS, Alibaba, Ceph, ChinaMobile, ArvanCloud, Digital Ocean, Dreamhost, Huawei OBS, IBM COS, Minio, and Tencent COS
23397 \ "s3"
23398 [snip]
23399 Storage> s3
23400
23401 4. Select TencentCOS provider.
23402
23403 Choose a number from below, or type in your own value
23404 1 / Amazon Web Services (AWS) S3
23405 \ "AWS"
23406 [snip]
23407 11 / Tencent Cloud Object Storage (COS)
23408 \ "TencentCOS"
23409 [snip]
23410 provider> TencentCOS
23411
23412 5. Enter your SecretId and SecretKey of Tencent Cloud.
23413
23414 Get AWS credentials from runtime (environment variables or EC2/ECS meta data if no env vars).
23415 Only applies if access_key_id and secret_access_key is blank.
23416 Enter a boolean value (true or false). Press Enter for the default ("false").
23417 Choose a number from below, or type in your own value
23418 1 / Enter AWS credentials in the next step
23419 \ "false"
23420 2 / Get AWS credentials from the environment (env vars or IAM)
23421 \ "true"
23422 env_auth> 1
23423 AWS Access Key ID.
23424 Leave blank for anonymous access or runtime credentials.
23425 Enter a string value. Press Enter for the default ("").
23426 access_key_id> AKIDxxxxxxxxxx
23427 AWS Secret Access Key (password)
23428 Leave blank for anonymous access or runtime credentials.
23429 Enter a string value. Press Enter for the default ("").
23430 secret_access_key> xxxxxxxxxxx
23431
23432 6. Select endpoint for Tencent COS. This is the standard endpoint for
23433 different region.
23434
23435 1 / Beijing Region.
23436 \ "cos.ap-beijing.myqcloud.com"
23437 2 / Nanjing Region.
23438 \ "cos.ap-nanjing.myqcloud.com"
23439 3 / Shanghai Region.
23440 \ "cos.ap-shanghai.myqcloud.com"
23441 4 / Guangzhou Region.
23442 \ "cos.ap-guangzhou.myqcloud.com"
23443 [snip]
23444 endpoint> 4
23445
23446 7. Choose acl and storage class.
23447
23448 Note that this ACL is applied when server-side copying objects as S3
23449 doesn't copy the ACL from the source but rather writes a fresh one.
23450 Enter a string value. Press Enter for the default ("").
23451 Choose a number from below, or type in your own value
23452 1 / Owner gets Full_CONTROL. No one else has access rights (default).
23453 \ "default"
23454 [snip]
23455 acl> 1
23456 The storage class to use when storing new objects in Tencent COS.
23457 Enter a string value. Press Enter for the default ("").
23458 Choose a number from below, or type in your own value
23459 1 / Default
23460 \ ""
23461 [snip]
23462 storage_class> 1
23463 Edit advanced config? (y/n)
23464 y) Yes
23465 n) No (default)
23466 y/n> n
23467 Remote config
23468 --------------------
23469 [cos]
23470 type = s3
23471 provider = TencentCOS
23472 env_auth = false
23473 access_key_id = xxx
23474 secret_access_key = xxx
23475 endpoint = cos.ap-guangzhou.myqcloud.com
23476 acl = default
23477 --------------------
23478 y) Yes this is OK (default)
23479 e) Edit this remote
23480 d) Delete this remote
23481 y/e/d> y
23482 Current remotes:
23483
23484 Name Type
23485 ==== ====
23486 cos s3
23487
23488 Netease NOS
23489 For Netease NOS configure as per the configurator rclone config setting
23490 the provider Netease. This will automatically set force_path_style =
23491 false which is necessary for it to run properly.
23492
23493 Storj
23494 Storj is a decentralized cloud storage which can be used through its
23495 native protocol or an S3 compatible gateway.
23496
23497 The S3 compatible gateway is configured using rclone config with a type
23498 of s3 and with a provider name of Storj. Here is an example run of the
23499 configurator.
23500
23501 Type of storage to configure.
23502 Storage> s3
23503 Get AWS credentials from runtime (environment variables or EC2/ECS meta data if no env vars).
23504 Only applies if access_key_id and secret_access_key is blank.
23505 Choose a number from below, or type in your own boolean value (true or false).
23506 Press Enter for the default (false).
23507 1 / Enter AWS credentials in the next step.
23508 \ (false)
23509 2 / Get AWS credentials from the environment (env vars or IAM).
23510 \ (true)
23511 env_auth> 1
23512 Option access_key_id.
23513 AWS Access Key ID.
23514 Leave blank for anonymous access or runtime credentials.
23515 Enter a value. Press Enter to leave empty.
23516 access_key_id> XXXX (as shown when creating the access grant)
23517 Option secret_access_key.
23518 AWS Secret Access Key (password).
23519 Leave blank for anonymous access or runtime credentials.
23520 Enter a value. Press Enter to leave empty.
23521 secret_access_key> XXXX (as shown when creating the access grant)
23522 Option endpoint.
23523 Endpoint of the Shared Gateway.
23524 Choose a number from below, or type in your own value.
23525 Press Enter to leave empty.
23526 1 / EU1 Shared Gateway
23527 \ (gateway.eu1.storjshare.io)
23528 2 / US1 Shared Gateway
23529 \ (gateway.us1.storjshare.io)
23530 3 / Asia-Pacific Shared Gateway
23531 \ (gateway.ap1.storjshare.io)
23532 endpoint> 1 (as shown when creating the access grant)
23533 Edit advanced config?
23534 y) Yes
23535 n) No (default)
23536 y/n> n
23537
23538 Note that s3 credentials are generated when you create an access grant
23539 (https://docs.storj.io/dcs/api-reference/s3-compatible-gateway#usage).
23540
23541 Backend quirks
23542 • --chunk-size is forced to be 64 MiB or greater. This will use more
23543 memory than the default of 5 MiB.
23544
23545 • Server side copy is disabled as it isn't currently supported in the
23546 gateway.
23547
23548 • GetTier and SetTier are not supported.
23549
23550 Backend bugs
23551 Due to issue #39 (https://github.com/storj/gateway-mt/issues/39) up‐
23552 loading multipart files via the S3 gateway causes them to lose their
23553 metadata. For rclone's purpose this means that the modification time
23554 is not stored, nor is any MD5SUM (if one is available from the source).
23555
23556 This has the following consequences:
23557
23558 • Using rclone rcat will fail as the medatada doesn't match after up‐
23559 load
23560
23561 • Uploading files with rclone mount will fail for the same reason
23562
23563 • This can worked around by using --vfs-cache-mode writes or --vfs-
23564 cache-mode full or setting --s3-upload-cutoff large
23565
23566 • Files uploaded via a multipart upload won't have their modtimes
23567
23568 • This will mean that rclone sync will likely keep trying to upload
23569 files bigger than --s3-upload-cutoff
23570
23571 • This can be worked around with --checksum or --size-only or setting
23572 --s3-upload-cutoff large
23573
23574 • The maximum value for --s3-upload-cutoff is 5GiB though
23575
23576 One general purpose workaround is to set --s3-upload-cutoff 5G. This
23577 means that rclone will upload files smaller than 5GiB as single parts.
23578 Note that this can be set in the config file with upload_cutoff = 5G or
23579 configured in the advanced settings. If you regularly transfer files
23580 larger than 5G then using --checksum or --size-only in rclone sync is
23581 the recommended workaround.
23582
23583 Comparison with the native protocol
23584 Use the the native protocol to take advantage of client-side encryption
23585 as well as to achieve the best possible download performance. Uploads
23586 will be erasure-coded locally, thus a 1gb upload will result in 2.68gb
23587 of data being uploaded to storage nodes across the network.
23588
23589 Use this backend and the S3 compatible Hosted Gateway to increase up‐
23590 load performance and reduce the load on your systems and network. Up‐
23591 loads will be encrypted and erasure-coded server-side, thus a 1GB up‐
23592 load will result in only in 1GB of data being uploaded to storage nodes
23593 across the network.
23594
23595 For more detailed comparison please check the documentation of the
23596 storj backend.
23597
23598 Limitations
23599 rclone about is not supported by the S3 backend. Backends without this
23600 capability cannot determine free space for an rclone mount or use poli‐
23601 cy mfs (most free space) as a member of an rclone union remote.
23602
23603 See List of backends that do not support rclone about
23604 (https://rclone.org/overview/#optional-features) and rclone about
23605 (https://rclone.org/commands/rclone_about/)
23606
23608 B2 is Backblaze's cloud storage system (https://www.backblaze.com/b2/).
23609
23610 Paths are specified as remote:bucket (or remote: for the lsd command.)
23611 You may put subdirectories in too, e.g. remote:bucket/path/to/dir.
23612
23613 Configuration
23614 Here is an example of making a b2 configuration. First run
23615
23616 rclone config
23617
23618 This will guide you through an interactive setup process. To authenti‐
23619 cate you will either need your Account ID (a short hex number) and Mas‐
23620 ter Application Key (a long hex number) OR an Application Key, which is
23621 the recommended method. See below for further details on generating
23622 and using an Application Key.
23623
23624 No remotes found, make a new one?
23625 n) New remote
23626 q) Quit config
23627 n/q> n
23628 name> remote
23629 Type of storage to configure.
23630 Choose a number from below, or type in your own value
23631 [snip]
23632 XX / Backblaze B2
23633 \ "b2"
23634 [snip]
23635 Storage> b2
23636 Account ID or Application Key ID
23637 account> 123456789abc
23638 Application Key
23639 key> 0123456789abcdef0123456789abcdef0123456789
23640 Endpoint for the service - leave blank normally.
23641 endpoint>
23642 Remote config
23643 --------------------
23644 [remote]
23645 account = 123456789abc
23646 key = 0123456789abcdef0123456789abcdef0123456789
23647 endpoint =
23648 --------------------
23649 y) Yes this is OK
23650 e) Edit this remote
23651 d) Delete this remote
23652 y/e/d> y
23653
23654 This remote is called remote and can now be used like this
23655
23656 See all buckets
23657
23658 rclone lsd remote:
23659
23660 Create a new bucket
23661
23662 rclone mkdir remote:bucket
23663
23664 List the contents of a bucket
23665
23666 rclone ls remote:bucket
23667
23668 Sync /home/local/directory to the remote bucket, deleting any excess
23669 files in the bucket.
23670
23671 rclone sync -i /home/local/directory remote:bucket
23672
23673 Application Keys
23674 B2 supports multiple Application Keys for different access permission
23675 to B2 Buckets (https://www.backblaze.com/b2/docs/applica‐
23676 tion_keys.html).
23677
23678 You can use these with rclone too; you will need to use rclone version
23679 1.43 or later.
23680
23681 Follow Backblaze's docs to create an Application Key with the required
23682 permission and add the applicationKeyId as the account and the Applica‐
23683 tion Key itself as the key.
23684
23685 Note that you must put the applicationKeyId as the account – you can't
23686 use the master Account ID. If you try then B2 will return 401 errors.
23687
23688 --fast-list
23689 This remote supports --fast-list which allows you to use fewer transac‐
23690 tions in exchange for more memory. See the rclone docs
23691 (https://rclone.org/docs/#fast-list) for more details.
23692
23693 Modified time
23694 The modified time is stored as metadata on the object as X-Bz-Info-
23695 src_last_modified_millis as milliseconds since 1970-01-01 in the Back‐
23696 blaze standard. Other tools should be able to use this as a modified
23697 time.
23698
23699 Modified times are used in syncing and are fully supported. Note that
23700 if a modification time needs to be updated on an object then it will
23701 create a new version of the object.
23702
23703 Restricted filename characters
23704 In addition to the default restricted characters set
23705 (https://rclone.org/overview/#restricted-characters) the following
23706 characters are also replaced:
23707
23708 Character Value Replacement
23709 ────────────────────────────────
23710 \ 0x5C \
23711
23712 Invalid UTF-8 bytes will also be replaced (https://rclone.org/over‐
23713 view/#invalid-utf8), as they can't be used in JSON strings.
23714
23715 Note that in 2020-05 Backblaze started allowing characters in file
23716 names. Rclone hasn't changed its encoding as this could cause syncs to
23717 re-transfer files. If you want rclone not to replace then see the
23718 --b2-encoding flag below and remove the BackSlash from the string.
23719 This can be set in the config.
23720
23721 SHA1 checksums
23722 The SHA1 checksums of the files are checked on upload and download and
23723 will be used in the syncing process.
23724
23725 Large files (bigger than the limit in --b2-upload-cutoff) which are up‐
23726 loaded in chunks will store their SHA1 on the object as X-Bz-Info-
23727 large_file_sha1 as recommended by Backblaze.
23728
23729 For a large file to be uploaded with an SHA1 checksum, the source needs
23730 to support SHA1 checksums. The local disk supports SHA1 checksums so
23731 large file transfers from local disk will have an SHA1. See the over‐
23732 view (https://rclone.org/overview/#features) for exactly which remotes
23733 support SHA1.
23734
23735 Sources which don't support SHA1, in particular crypt will upload large
23736 files without SHA1 checksums. This may be fixed in the future (see
23737 #1767 (https://github.com/rclone/rclone/issues/1767)).
23738
23739 Files sizes below --b2-upload-cutoff will always have an SHA1 regard‐
23740 less of the source.
23741
23742 Transfers
23743 Backblaze recommends that you do lots of transfers simultaneously for
23744 maximum speed. In tests from my SSD equipped laptop the optimum set‐
23745 ting is about --transfers 32 though higher numbers may be used for a
23746 slight speed improvement. The optimum number for you may vary depend‐
23747 ing on your hardware, how big the files are, how much you want to load
23748 your computer, etc. The default of --transfers 4 is definitely too low
23749 for Backblaze B2 though.
23750
23751 Note that uploading big files (bigger than 200 MiB by default) will use
23752 a 96 MiB RAM buffer by default. There can be at most --transfers of
23753 these in use at any moment, so this sets the upper limit on the memory
23754 used.
23755
23756 Versions
23757 When rclone uploads a new version of a file it creates a new version of
23758 it (https://www.backblaze.com/b2/docs/file_versions.html). Likewise
23759 when you delete a file, the old version will be marked hidden and still
23760 be available. Conversely, you may opt in to a "hard delete" of files
23761 with the --b2-hard-delete flag which would permanently remove the file
23762 instead of hiding it.
23763
23764 Old versions of files, where available, are visible using the --b2-ver‐
23765 sions flag.
23766
23767 It is also possible to view a bucket as it was at a certain point in
23768 time, using the --b2-version-at flag. This will show the file versions
23769 as they were at that time, showing files that have been deleted after‐
23770 wards, and hiding files that were created since.
23771
23772 If you wish to remove all the old versions then you can use the rclone
23773 cleanup remote:bucket command which will delete all the old versions of
23774 files, leaving the current ones intact. You can also supply a path and
23775 only old versions under that path will be deleted, e.g. rclone cleanup
23776 remote:bucket/path/to/stuff.
23777
23778 Note that cleanup will remove partially uploaded files from the bucket
23779 if they are more than a day old.
23780
23781 When you purge a bucket, the current and the old versions will be
23782 deleted then the bucket will be deleted.
23783
23784 However delete will cause the current versions of the files to become
23785 hidden old versions.
23786
23787 Here is a session showing the listing and retrieval of an old version
23788 followed by a cleanup of the old versions.
23789
23790 Show current version and all the versions with --b2-versions flag.
23791
23792 $ rclone -q ls b2:cleanup-test
23793 9 one.txt
23794
23795 $ rclone -q --b2-versions ls b2:cleanup-test
23796 9 one.txt
23797 8 one-v2016-07-04-141032-000.txt
23798 16 one-v2016-07-04-141003-000.txt
23799 15 one-v2016-07-02-155621-000.txt
23800
23801 Retrieve an old version
23802
23803 $ rclone -q --b2-versions copy b2:cleanup-test/one-v2016-07-04-141003-000.txt /tmp
23804
23805 $ ls -l /tmp/one-v2016-07-04-141003-000.txt
23806 -rw-rw-r-- 1 ncw ncw 16 Jul 2 17:46 /tmp/one-v2016-07-04-141003-000.txt
23807
23808 Clean up all the old versions and show that they've gone.
23809
23810 $ rclone -q cleanup b2:cleanup-test
23811
23812 $ rclone -q ls b2:cleanup-test
23813 9 one.txt
23814
23815 $ rclone -q --b2-versions ls b2:cleanup-test
23816 9 one.txt
23817
23818 Data usage
23819 It is useful to know how many requests are sent to the server in dif‐
23820 ferent scenarios.
23821
23822 All copy commands send the following 4 requests:
23823
23824 /b2api/v1/b2_authorize_account
23825 /b2api/v1/b2_create_bucket
23826 /b2api/v1/b2_list_buckets
23827 /b2api/v1/b2_list_file_names
23828
23829 The b2_list_file_names request will be sent once for every 1k files in
23830 the remote path, providing the checksum and modification time of the
23831 listed files. As of version 1.33 issue #818
23832 (https://github.com/rclone/rclone/issues/818) causes extra requests to
23833 be sent when using B2 with Crypt. When a copy operation does not re‐
23834 quire any files to be uploaded, no more requests will be sent.
23835
23836 Uploading files that do not require chunking, will send 2 requests per
23837 file upload:
23838
23839 /b2api/v1/b2_get_upload_url
23840 /b2api/v1/b2_upload_file/
23841
23842 Uploading files requiring chunking, will send 2 requests (one each to
23843 start and finish the upload) and another 2 requests for each chunk:
23844
23845 /b2api/v1/b2_start_large_file
23846 /b2api/v1/b2_get_upload_part_url
23847 /b2api/v1/b2_upload_part/
23848 /b2api/v1/b2_finish_large_file
23849
23850 Versions
23851 Versions can be viewed with the --b2-versions flag. When it is set
23852 rclone will show and act on older versions of files. For example
23853
23854 Listing without --b2-versions
23855
23856 $ rclone -q ls b2:cleanup-test
23857 9 one.txt
23858
23859 And with
23860
23861 $ rclone -q --b2-versions ls b2:cleanup-test
23862 9 one.txt
23863 8 one-v2016-07-04-141032-000.txt
23864 16 one-v2016-07-04-141003-000.txt
23865 15 one-v2016-07-02-155621-000.txt
23866
23867 Showing that the current version is unchanged but older versions can be
23868 seen. These have the UTC date that they were uploaded to the server to
23869 the nearest millisecond appended to them.
23870
23871 Note that when using --b2-versions no file write operations are permit‐
23872 ted, so you can't upload files or delete them.
23873
23874 B2 and rclone link
23875 Rclone supports generating file share links for private B2 buckets.
23876 They can either be for a file for example:
23877
23878 ./rclone link B2:bucket/path/to/file.txt
23879 https://f002.backblazeb2.com/file/bucket/path/to/file.txt?Authorization=xxxxxxxx
23880
23881 or if run on a directory you will get:
23882
23883 ./rclone link B2:bucket/path
23884 https://f002.backblazeb2.com/file/bucket/path?Authorization=xxxxxxxx
23885
23886 you can then use the authorization token (the part of the url from the
23887 ?Authorization= on) on any file path under that directory. For exam‐
23888 ple:
23889
23890 https://f002.backblazeb2.com/file/bucket/path/to/file1?Authorization=xxxxxxxx
23891 https://f002.backblazeb2.com/file/bucket/path/file2?Authorization=xxxxxxxx
23892 https://f002.backblazeb2.com/file/bucket/path/folder/file3?Authorization=xxxxxxxx
23893
23894 Standard options
23895 Here are the Standard options specific to b2 (Backblaze B2).
23896
23897 --b2-account
23898 Account ID or Application Key ID.
23899
23900 Properties:
23901
23902 • Config: account
23903
23904 • Env Var: RCLONE_B2_ACCOUNT
23905
23906 • Type: string
23907
23908 • Required: true
23909
23910 --b2-key
23911 Application Key.
23912
23913 Properties:
23914
23915 • Config: key
23916
23917 • Env Var: RCLONE_B2_KEY
23918
23919 • Type: string
23920
23921 • Required: true
23922
23923 --b2-hard-delete
23924 Permanently delete files on remote removal, otherwise hide files.
23925
23926 Properties:
23927
23928 • Config: hard_delete
23929
23930 • Env Var: RCLONE_B2_HARD_DELETE
23931
23932 • Type: bool
23933
23934 • Default: false
23935
23936 Advanced options
23937 Here are the Advanced options specific to b2 (Backblaze B2).
23938
23939 --b2-endpoint
23940 Endpoint for the service.
23941
23942 Leave blank normally.
23943
23944 Properties:
23945
23946 • Config: endpoint
23947
23948 • Env Var: RCLONE_B2_ENDPOINT
23949
23950 • Type: string
23951
23952 • Required: false
23953
23954 --b2-test-mode
23955 A flag string for X-Bz-Test-Mode header for debugging.
23956
23957 This is for debugging purposes only. Setting it to one of the strings
23958 below will cause b2 to return specific errors:
23959
23960 • "fail_some_uploads"
23961
23962 • "expire_some_account_authorization_tokens"
23963
23964 • "force_cap_exceeded"
23965
23966 These will be set in the "X-Bz-Test-Mode" header which is documented in
23967 the b2 integrations checklist (https://www.backblaze.com/b2/docs/inte‐
23968 gration_checklist.html).
23969
23970 Properties:
23971
23972 • Config: test_mode
23973
23974 • Env Var: RCLONE_B2_TEST_MODE
23975
23976 • Type: string
23977
23978 • Required: false
23979
23980 --b2-versions
23981 Include old versions in directory listings.
23982
23983 Note that when using this no file write operations are permitted, so
23984 you can't upload files or delete them.
23985
23986 Properties:
23987
23988 • Config: versions
23989
23990 • Env Var: RCLONE_B2_VERSIONS
23991
23992 • Type: bool
23993
23994 • Default: false
23995
23996 --b2-version-at
23997 Show file versions as they were at the specified time.
23998
23999 Note that when using this no file write operations are permitted, so
24000 you can't upload files or delete them.
24001
24002 Properties:
24003
24004 • Config: version_at
24005
24006 • Env Var: RCLONE_B2_VERSION_AT
24007
24008 • Type: Time
24009
24010 • Default: off
24011
24012 --b2-upload-cutoff
24013 Cutoff for switching to chunked upload.
24014
24015 Files above this size will be uploaded in chunks of "--b2-chunk-size".
24016
24017 This value should be set no larger than 4.657 GiB (== 5 GB).
24018
24019 Properties:
24020
24021 • Config: upload_cutoff
24022
24023 • Env Var: RCLONE_B2_UPLOAD_CUTOFF
24024
24025 • Type: SizeSuffix
24026
24027 • Default: 200Mi
24028
24029 --b2-copy-cutoff
24030 Cutoff for switching to multipart copy.
24031
24032 Any files larger than this that need to be server-side copied will be
24033 copied in chunks of this size.
24034
24035 The minimum is 0 and the maximum is 4.6 GiB.
24036
24037 Properties:
24038
24039 • Config: copy_cutoff
24040
24041 • Env Var: RCLONE_B2_COPY_CUTOFF
24042
24043 • Type: SizeSuffix
24044
24045 • Default: 4Gi
24046
24047 --b2-chunk-size
24048 Upload chunk size.
24049
24050 When uploading large files, chunk the file into this size.
24051
24052 Must fit in memory. These chunks are buffered in memory and there
24053 might a maximum of "--transfers" chunks in progress at once.
24054
24055 5,000,000 Bytes is the minimum size.
24056
24057 Properties:
24058
24059 • Config: chunk_size
24060
24061 • Env Var: RCLONE_B2_CHUNK_SIZE
24062
24063 • Type: SizeSuffix
24064
24065 • Default: 96Mi
24066
24067 --b2-disable-checksum
24068 Disable checksums for large (> upload cutoff) files.
24069
24070 Normally rclone will calculate the SHA1 checksum of the input before
24071 uploading it so it can add it to metadata on the object. This is great
24072 for data integrity checking but can cause long delays for large files
24073 to start uploading.
24074
24075 Properties:
24076
24077 • Config: disable_checksum
24078
24079 • Env Var: RCLONE_B2_DISABLE_CHECKSUM
24080
24081 • Type: bool
24082
24083 • Default: false
24084
24085 --b2-download-url
24086 Custom endpoint for downloads.
24087
24088 This is usually set to a Cloudflare CDN URL as Backblaze offers free
24089 egress for data downloaded through the Cloudflare network. Rclone
24090 works with private buckets by sending an "Authorization" header. If
24091 the custom endpoint rewrites the requests for authentication, e.g., in
24092 Cloudflare Workers, this header needs to be handled properly. Leave
24093 blank if you want to use the endpoint provided by Backblaze.
24094
24095 The URL provided here SHOULD have the protocol and SHOULD NOT have a
24096 trailing slash or specify the /file/bucket subpath as rclone will re‐
24097 quest files with "{download_url}/file/{bucket_name}/{path}".
24098
24099 Example: > https://mysubdomain.mydomain.tld (No trailing "/", "file" or
24100 "bucket")
24101
24102 Properties:
24103
24104 • Config: download_url
24105
24106 • Env Var: RCLONE_B2_DOWNLOAD_URL
24107
24108 • Type: string
24109
24110 • Required: false
24111
24112 --b2-download-auth-duration
24113 Time before the authorization token will expire in s or suffix
24114 ms|s|m|h|d.
24115
24116 The duration before the download authorization token will expire. The
24117 minimum value is 1 second. The maximum value is one week.
24118
24119 Properties:
24120
24121 • Config: download_auth_duration
24122
24123 • Env Var: RCLONE_B2_DOWNLOAD_AUTH_DURATION
24124
24125 • Type: Duration
24126
24127 • Default: 1w
24128
24129 --b2-memory-pool-flush-time
24130 How often internal memory buffer pools will be flushed. Uploads which
24131 requires additional buffers (f.e multipart) will use memory pool for
24132 allocations. This option controls how often unused buffers will be re‐
24133 moved from the pool.
24134
24135 Properties:
24136
24137 • Config: memory_pool_flush_time
24138
24139 • Env Var: RCLONE_B2_MEMORY_POOL_FLUSH_TIME
24140
24141 • Type: Duration
24142
24143 • Default: 1m0s
24144
24145 --b2-memory-pool-use-mmap
24146 Whether to use mmap buffers in internal memory pool.
24147
24148 Properties:
24149
24150 • Config: memory_pool_use_mmap
24151
24152 • Env Var: RCLONE_B2_MEMORY_POOL_USE_MMAP
24153
24154 • Type: bool
24155
24156 • Default: false
24157
24158 --b2-encoding
24159 The encoding for the backend.
24160
24161 See the encoding section in the overview (https://rclone.org/over‐
24162 view/#encoding) for more info.
24163
24164 Properties:
24165
24166 • Config: encoding
24167
24168 • Env Var: RCLONE_B2_ENCODING
24169
24170 • Type: MultiEncoder
24171
24172 • Default: Slash,BackSlash,Del,Ctl,InvalidUtf8,Dot
24173
24174 Limitations
24175 rclone about is not supported by the B2 backend. Backends without this
24176 capability cannot determine free space for an rclone mount or use poli‐
24177 cy mfs (most free space) as a member of an rclone union remote.
24178
24179 See List of backends that do not support rclone about
24180 (https://rclone.org/overview/#optional-features) and rclone about
24181 (https://rclone.org/commands/rclone_about/)
24182
24184 Paths are specified as remote:path
24185
24186 Paths may be as deep as required, e.g. remote:directory/subdirectory.
24187
24188 The initial setup for Box involves getting a token from Box which you
24189 can do either in your browser, or with a config.json downloaded from
24190 Box to use JWT authentication. rclone config walks you through it.
24191
24192 Configuration
24193 Here is an example of how to make a remote called remote. First run:
24194
24195 rclone config
24196
24197 This will guide you through an interactive setup process:
24198
24199 No remotes found, make a new one?
24200 n) New remote
24201 s) Set configuration password
24202 q) Quit config
24203 n/s/q> n
24204 name> remote
24205 Type of storage to configure.
24206 Choose a number from below, or type in your own value
24207 [snip]
24208 XX / Box
24209 \ "box"
24210 [snip]
24211 Storage> box
24212 Box App Client Id - leave blank normally.
24213 client_id>
24214 Box App Client Secret - leave blank normally.
24215 client_secret>
24216 Box App config.json location
24217 Leave blank normally.
24218 Enter a string value. Press Enter for the default ("").
24219 box_config_file>
24220 Box App Primary Access Token
24221 Leave blank normally.
24222 Enter a string value. Press Enter for the default ("").
24223 access_token>
24224
24225 Enter a string value. Press Enter for the default ("user").
24226 Choose a number from below, or type in your own value
24227 1 / Rclone should act on behalf of a user
24228 \ "user"
24229 2 / Rclone should act on behalf of a service account
24230 \ "enterprise"
24231 box_sub_type>
24232 Remote config
24233 Use auto config?
24234 * Say Y if not sure
24235 * Say N if you are working on a remote or headless machine
24236 y) Yes
24237 n) No
24238 y/n> y
24239 If your browser doesn't open automatically go to the following link: http://127.0.0.1:53682/auth
24240 Log in and authorize rclone for access
24241 Waiting for code...
24242 Got code
24243 --------------------
24244 [remote]
24245 client_id =
24246 client_secret =
24247 token = {"access_token":"XXX","token_type":"bearer","refresh_token":"XXX","expiry":"XXX"}
24248 --------------------
24249 y) Yes this is OK
24250 e) Edit this remote
24251 d) Delete this remote
24252 y/e/d> y
24253
24254 See the remote setup docs (https://rclone.org/remote_setup/) for how to
24255 set it up on a machine with no Internet browser available.
24256
24257 Note that rclone runs a webserver on your local machine to collect the
24258 token as returned from Box. This only runs from the moment it opens
24259 your browser to the moment you get back the verification code. This is
24260 on http://127.0.0.1:53682/ and this it may require you to unblock it
24261 temporarily if you are running a host firewall.
24262
24263 Once configured you can then use rclone like this,
24264
24265 List directories in top level of your Box
24266
24267 rclone lsd remote:
24268
24269 List all the files in your Box
24270
24271 rclone ls remote:
24272
24273 To copy a local directory to an Box directory called backup
24274
24275 rclone copy /home/source remote:backup
24276
24277 Using rclone with an Enterprise account with SSO
24278 If you have an "Enterprise" account type with Box with single sign on
24279 (SSO), you need to create a password to use Box with rclone. This can
24280 be done at your Enterprise Box account by going to Settings, "Account"
24281 Tab, and then set the password in the "Authentication" field.
24282
24283 Once you have done this, you can setup your Enterprise Box account us‐
24284 ing the same procedure detailed above in the, using the password you
24285 have just set.
24286
24287 Invalid refresh token
24288 According to the box docs (https://develop‐
24289 er.box.com/v2.0/docs/oauth-20#section-6-using-the-access-and-refresh-
24290 tokens):
24291
24292 Each refresh_token is valid for one use in 60 days.
24293
24294 This means that if you
24295
24296 • Don't use the box remote for 60 days
24297
24298 • Copy the config file with a box refresh token in and use it in two
24299 places
24300
24301 • Get an error on a token refresh
24302
24303 then rclone will return an error which includes the text Invalid re‐
24304 fresh token.
24305
24306 To fix this you will need to use oauth2 again to update the refresh to‐
24307 ken. You can use the methods in the remote setup docs
24308 (https://rclone.org/remote_setup/), bearing in mind that if you use the
24309 copy the config file method, you should not use that remote on the com‐
24310 puter you did the authentication on.
24311
24312 Here is how to do it.
24313
24314 $ rclone config
24315 Current remotes:
24316
24317 Name Type
24318 ==== ====
24319 remote box
24320
24321 e) Edit existing remote
24322 n) New remote
24323 d) Delete remote
24324 r) Rename remote
24325 c) Copy remote
24326 s) Set configuration password
24327 q) Quit config
24328 e/n/d/r/c/s/q> e
24329 Choose a number from below, or type in an existing value
24330 1 > remote
24331 remote> remote
24332 --------------------
24333 [remote]
24334 type = box
24335 token = {"access_token":"XXX","token_type":"bearer","refresh_token":"XXX","expiry":"2017-07-08T23:40:08.059167677+01:00"}
24336 --------------------
24337 Edit remote
24338 Value "client_id" = ""
24339 Edit? (y/n)>
24340 y) Yes
24341 n) No
24342 y/n> n
24343 Value "client_secret" = ""
24344 Edit? (y/n)>
24345 y) Yes
24346 n) No
24347 y/n> n
24348 Remote config
24349 Already have a token - refresh?
24350 y) Yes
24351 n) No
24352 y/n> y
24353 Use auto config?
24354 * Say Y if not sure
24355 * Say N if you are working on a remote or headless machine
24356 y) Yes
24357 n) No
24358 y/n> y
24359 If your browser doesn't open automatically go to the following link: http://127.0.0.1:53682/auth
24360 Log in and authorize rclone for access
24361 Waiting for code...
24362 Got code
24363 --------------------
24364 [remote]
24365 type = box
24366 token = {"access_token":"YYY","token_type":"bearer","refresh_token":"YYY","expiry":"2017-07-23T12:22:29.259137901+01:00"}
24367 --------------------
24368 y) Yes this is OK
24369 e) Edit this remote
24370 d) Delete this remote
24371 y/e/d> y
24372
24373 Modified time and hashes
24374 Box allows modification times to be set on objects accurate to 1 sec‐
24375 ond. These will be used to detect whether objects need syncing or not.
24376
24377 Box supports SHA1 type hashes, so you can use the --checksum flag.
24378
24379 Restricted filename characters
24380 In addition to the default restricted characters set
24381 (https://rclone.org/overview/#restricted-characters) the following
24382 characters are also replaced:
24383
24384 Character Value Replacement
24385 ────────────────────────────────
24386 \ 0x5C \
24387
24388 File names can also not end with the following characters. These only
24389 get replaced if they are the last character in the name:
24390
24391 Character Value Replacement
24392 ────────────────────────────────
24393 SP 0x20 ␠
24394
24395 Invalid UTF-8 bytes will also be replaced (https://rclone.org/over‐
24396 view/#invalid-utf8), as they can't be used in JSON strings.
24397
24398 Transfers
24399 For files above 50 MiB rclone will use a chunked transfer. Rclone will
24400 upload up to --transfers chunks at the same time (shared among all the
24401 multipart uploads). Chunks are buffered in memory and are normally 8
24402 MiB so increasing --transfers will increase memory use.
24403
24404 Deleting files
24405 Depending on the enterprise settings for your user, the item will ei‐
24406 ther be actually deleted from Box or moved to the trash.
24407
24408 Emptying the trash is supported via the rclone however cleanup command
24409 however this deletes every trashed file and folder individually so it
24410 may take a very long time. Emptying the trash via the WebUI does not
24411 have this limitation so it is advised to empty the trash via the WebUI.
24412
24413 Root folder ID
24414 You can set the root_folder_id for rclone. This is the directory
24415 (identified by its Folder ID) that rclone considers to be the root of
24416 your Box drive.
24417
24418 Normally you will leave this blank and rclone will determine the cor‐
24419 rect root to use itself.
24420
24421 However you can set this to restrict rclone to a specific folder hier‐
24422 archy.
24423
24424 In order to do this you will have to find the Folder ID of the directo‐
24425 ry you wish rclone to display. This will be the last segment of the
24426 URL when you open the relevant folder in the Box web interface.
24427
24428 So if the folder you want rclone to use has a URL which looks like
24429 https://app.box.com/folder/11xxxxxxxxx8 in the browser, then you use
24430 11xxxxxxxxx8 as the root_folder_id in the config.
24431
24432 Standard options
24433 Here are the Standard options specific to box (Box).
24434
24435 --box-client-id
24436 OAuth Client Id.
24437
24438 Leave blank normally.
24439
24440 Properties:
24441
24442 • Config: client_id
24443
24444 • Env Var: RCLONE_BOX_CLIENT_ID
24445
24446 • Type: string
24447
24448 • Required: false
24449
24450 --box-client-secret
24451 OAuth Client Secret.
24452
24453 Leave blank normally.
24454
24455 Properties:
24456
24457 • Config: client_secret
24458
24459 • Env Var: RCLONE_BOX_CLIENT_SECRET
24460
24461 • Type: string
24462
24463 • Required: false
24464
24465 --box-box-config-file
24466 Box App config.json location
24467
24468 Leave blank normally.
24469
24470 Leading ~ will be expanded in the file name as will environment vari‐
24471 ables such as ${RCLONE_CONFIG_DIR}.
24472
24473 Properties:
24474
24475 • Config: box_config_file
24476
24477 • Env Var: RCLONE_BOX_BOX_CONFIG_FILE
24478
24479 • Type: string
24480
24481 • Required: false
24482
24483 --box-access-token
24484 Box App Primary Access Token
24485
24486 Leave blank normally.
24487
24488 Properties:
24489
24490 • Config: access_token
24491
24492 • Env Var: RCLONE_BOX_ACCESS_TOKEN
24493
24494 • Type: string
24495
24496 • Required: false
24497
24498 --box-box-sub-type
24499 Properties:
24500
24501 • Config: box_sub_type
24502
24503 • Env Var: RCLONE_BOX_BOX_SUB_TYPE
24504
24505 • Type: string
24506
24507 • Default: "user"
24508
24509 • Examples:
24510
24511 • "user"
24512
24513 • Rclone should act on behalf of a user.
24514
24515 • "enterprise"
24516
24517 • Rclone should act on behalf of a service account.
24518
24519 Advanced options
24520 Here are the Advanced options specific to box (Box).
24521
24522 --box-token
24523 OAuth Access Token as a JSON blob.
24524
24525 Properties:
24526
24527 • Config: token
24528
24529 • Env Var: RCLONE_BOX_TOKEN
24530
24531 • Type: string
24532
24533 • Required: false
24534
24535 --box-auth-url
24536 Auth server URL.
24537
24538 Leave blank to use the provider defaults.
24539
24540 Properties:
24541
24542 • Config: auth_url
24543
24544 • Env Var: RCLONE_BOX_AUTH_URL
24545
24546 • Type: string
24547
24548 • Required: false
24549
24550 --box-token-url
24551 Token server url.
24552
24553 Leave blank to use the provider defaults.
24554
24555 Properties:
24556
24557 • Config: token_url
24558
24559 • Env Var: RCLONE_BOX_TOKEN_URL
24560
24561 • Type: string
24562
24563 • Required: false
24564
24565 --box-root-folder-id
24566 Fill in for rclone to use a non root folder as its starting point.
24567
24568 Properties:
24569
24570 • Config: root_folder_id
24571
24572 • Env Var: RCLONE_BOX_ROOT_FOLDER_ID
24573
24574 • Type: string
24575
24576 • Default: "0"
24577
24578 --box-upload-cutoff
24579 Cutoff for switching to multipart upload (>= 50 MiB).
24580
24581 Properties:
24582
24583 • Config: upload_cutoff
24584
24585 • Env Var: RCLONE_BOX_UPLOAD_CUTOFF
24586
24587 • Type: SizeSuffix
24588
24589 • Default: 50Mi
24590
24591 --box-commit-retries
24592 Max number of times to try committing a multipart file.
24593
24594 Properties:
24595
24596 • Config: commit_retries
24597
24598 • Env Var: RCLONE_BOX_COMMIT_RETRIES
24599
24600 • Type: int
24601
24602 • Default: 100
24603
24604 --box-list-chunk
24605 Size of listing chunk 1-1000.
24606
24607 Properties:
24608
24609 • Config: list_chunk
24610
24611 • Env Var: RCLONE_BOX_LIST_CHUNK
24612
24613 • Type: int
24614
24615 • Default: 1000
24616
24617 --box-owned-by
24618 Only show items owned by the login (email address) passed in.
24619
24620 Properties:
24621
24622 • Config: owned_by
24623
24624 • Env Var: RCLONE_BOX_OWNED_BY
24625
24626 • Type: string
24627
24628 • Required: false
24629
24630 --box-encoding
24631 The encoding for the backend.
24632
24633 See the encoding section in the overview (https://rclone.org/over‐
24634 view/#encoding) for more info.
24635
24636 Properties:
24637
24638 • Config: encoding
24639
24640 • Env Var: RCLONE_BOX_ENCODING
24641
24642 • Type: MultiEncoder
24643
24644 • Default: Slash,BackSlash,Del,Ctl,RightSpace,InvalidUtf8,Dot
24645
24646 Limitations
24647 Note that Box is case insensitive so you can't have a file called "Hel‐
24648 lo.doc" and one called "hello.doc".
24649
24650 Box file names can't have the \ character in. rclone maps this to and
24651 from an identical looking unicode equivalent \ (U+FF3C Fullwidth Re‐
24652 verse Solidus).
24653
24654 Box only supports filenames up to 255 characters in length.
24655
24656 rclone about is not supported by the Box backend. Backends without
24657 this capability cannot determine free space for an rclone mount or use
24658 policy mfs (most free space) as a member of an rclone union remote.
24659
24660 See List of backends that do not support rclone about
24661 (https://rclone.org/overview/#optional-features) and rclone about
24662 (https://rclone.org/commands/rclone_about/)
24663
24665 The cache remote wraps another existing remote and stores file struc‐
24666 ture and its data for long running tasks like rclone mount.
24667
24668 Status
24669 The cache backend code is working but it currently doesn't have a main‐
24670 tainer so there are outstanding bugs
24671 (https://github.com/rclone/rclone/issues?q=is%3Aopen+is%3Aissue+la‐
24672 bel%3Abug+label%3A%22Remote%3A+Cache%22) which aren't getting fixed.
24673
24674 The cache backend is due to be phased out in favour of the VFS caching
24675 layer eventually which is more tightly integrated into rclone.
24676
24677 Until this happens we recommend only using the cache backend if you
24678 find you can't work without it. There are many docs online describing
24679 the use of the cache backend to minimize API hits and by-and-large
24680 these are out of date and the cache backend isn't needed in those sce‐
24681 narios any more.
24682
24683 Configuration
24684 To get started you just need to have an existing remote which can be
24685 configured with cache.
24686
24687 Here is an example of how to make a remote called test-cache. First
24688 run:
24689
24690 rclone config
24691
24692 This will guide you through an interactive setup process:
24693
24694 No remotes found, make a new one?
24695 n) New remote
24696 r) Rename remote
24697 c) Copy remote
24698 s) Set configuration password
24699 q) Quit config
24700 n/r/c/s/q> n
24701 name> test-cache
24702 Type of storage to configure.
24703 Choose a number from below, or type in your own value
24704 [snip]
24705 XX / Cache a remote
24706 \ "cache"
24707 [snip]
24708 Storage> cache
24709 Remote to cache.
24710 Normally should contain a ':' and a path, e.g. "myremote:path/to/dir",
24711 "myremote:bucket" or maybe "myremote:" (not recommended).
24712 remote> local:/test
24713 Optional: The URL of the Plex server
24714 plex_url> http://127.0.0.1:32400
24715 Optional: The username of the Plex user
24716 plex_username> dummyusername
24717 Optional: The password of the Plex user
24718 y) Yes type in my own password
24719 g) Generate random password
24720 n) No leave this optional password blank
24721 y/g/n> y
24722 Enter the password:
24723 password:
24724 Confirm the password:
24725 password:
24726 The size of a chunk. Lower value good for slow connections but can affect seamless reading.
24727 Default: 5M
24728 Choose a number from below, or type in your own value
24729 1 / 1 MiB
24730 \ "1M"
24731 2 / 5 MiB
24732 \ "5M"
24733 3 / 10 MiB
24734 \ "10M"
24735 chunk_size> 2
24736 How much time should object info (file size, file hashes, etc.) be stored in cache. Use a very high value if you don't plan on changing the source FS from outside the cache.
24737 Accepted units are: "s", "m", "h".
24738 Default: 5m
24739 Choose a number from below, or type in your own value
24740 1 / 1 hour
24741 \ "1h"
24742 2 / 24 hours
24743 \ "24h"
24744 3 / 24 hours
24745 \ "48h"
24746 info_age> 2
24747 The maximum size of stored chunks. When the storage grows beyond this size, the oldest chunks will be deleted.
24748 Default: 10G
24749 Choose a number from below, or type in your own value
24750 1 / 500 MiB
24751 \ "500M"
24752 2 / 1 GiB
24753 \ "1G"
24754 3 / 10 GiB
24755 \ "10G"
24756 chunk_total_size> 3
24757 Remote config
24758 --------------------
24759 [test-cache]
24760 remote = local:/test
24761 plex_url = http://127.0.0.1:32400
24762 plex_username = dummyusername
24763 plex_password = *** ENCRYPTED ***
24764 chunk_size = 5M
24765 info_age = 48h
24766 chunk_total_size = 10G
24767
24768 You can then use it like this,
24769
24770 List directories in top level of your drive
24771
24772 rclone lsd test-cache:
24773
24774 List all the files in your drive
24775
24776 rclone ls test-cache:
24777
24778 To start a cached mount
24779
24780 rclone mount --allow-other test-cache: /var/tmp/test-cache
24781
24782 Write Features
24783 Offline uploading
24784 In an effort to make writing through cache more reliable, the backend
24785 now supports this feature which can be activated by specifying a cache-
24786 tmp-upload-path.
24787
24788 A files goes through these states when using this feature:
24789
24790 1. An upload is started (usually by copying a file on the cache remote)
24791
24792 2. When the copy to the temporary location is complete the file is part
24793 of the cached remote and looks and behaves like any other file
24794 (reading included)
24795
24796 3. After cache-tmp-wait-time passes and the file is next in line,
24797 rclone move is used to move the file to the cloud provider
24798
24799 4. Reading the file still works during the upload but most modifica‐
24800 tions on it will be prohibited
24801
24802 5. Once the move is complete the file is unlocked for modifications as
24803 it becomes as any other regular file
24804
24805 6. If the file is being read through cache when it's actually deleted
24806 from the temporary path then cache will simply swap the source to
24807 the cloud provider without interrupting the reading (small blip can
24808 happen though)
24809
24810 Files are uploaded in sequence and only one file is uploaded at a time.
24811 Uploads will be stored in a queue and be processed based on the order
24812 they were added. The queue and the temporary storage is persistent
24813 across restarts but can be cleared on startup with the --cache-db-purge
24814 flag.
24815
24816 Write Support
24817 Writes are supported through cache. One caveat is that a mounted cache
24818 remote does not add any retry or fallback mechanism to the upload oper‐
24819 ation. This will depend on the implementation of the wrapped remote.
24820 Consider using Offline uploading for reliable writes.
24821
24822 One special case is covered with cache-writes which will cache the file
24823 data at the same time as the upload when it is enabled making it avail‐
24824 able from the cache store immediately once the upload is finished.
24825
24826 Read Features
24827 Multiple connections
24828 To counter the high latency between a local PC where rclone is running
24829 and cloud providers, the cache remote can split multiple requests to
24830 the cloud provider for smaller file chunks and combines them together
24831 locally where they can be available almost immediately before the read‐
24832 er usually needs them.
24833
24834 This is similar to buffering when media files are played online.
24835 Rclone will stay around the current marker but always try its best to
24836 stay ahead and prepare the data before.
24837
24838 Plex Integration
24839 There is a direct integration with Plex which allows cache to detect
24840 during reading if the file is in playback or not. This helps cache to
24841 adapt how it queries the cloud provider depending on what is needed
24842 for.
24843
24844 Scans will have a minimum amount of workers (1) while in a confirmed
24845 playback cache will deploy the configured number of workers.
24846
24847 This integration opens the doorway to additional performance improve‐
24848 ments which will be explored in the near future.
24849
24850 Note: If Plex options are not configured, cache will function with its
24851 configured options without adapting any of its settings.
24852
24853 How to enable? Run rclone config and add all the Plex options (end‐
24854 point, username and password) in your remote and it will be automati‐
24855 cally enabled.
24856
24857 Affected settings: - cache-workers: Configured value during confirmed
24858 playback or 1 all the other times
24859
24860 Certificate Validation
24861 When the Plex server is configured to only accept secure connections,
24862 it is possible to use .plex.direct URLs to ensure certificate valida‐
24863 tion succeeds. These URLs are used by Plex internally to connect to
24864 the Plex server securely.
24865
24866 The format for these URLs is the following:
24867
24868 https://ip-with-dots-replaced.server-hash.plex.direct:32400/
24869
24870 The ip-with-dots-replaced part can be any IPv4 address, where the dots
24871 have been replaced with dashes, e.g. 127.0.0.1 becomes 127-0-0-1.
24872
24873 To get the server-hash part, the easiest way is to visit
24874
24875 https://plex.tv/api/resources?includeHttps=1&X-Plex-Token=your-plex-to‐
24876 ken
24877
24878 This page will list all the available Plex servers for your account
24879 with at least one .plex.direct link for each. Copy one URL and replace
24880 the IP address with the desired address. This can be used as the
24881 plex_url value.
24882
24883 Known issues
24884 Mount and --dir-cache-time
24885 --dir-cache-time controls the first layer of directory caching which
24886 works at the mount layer. Being an independent caching mechanism from
24887 the cache backend, it will manage its own entries based on the config‐
24888 ured time.
24889
24890 To avoid getting in a scenario where dir cache has obsolete data and
24891 cache would have the correct one, try to set --dir-cache-time to a low‐
24892 er time than --cache-info-age. Default values are already configured
24893 in this way.
24894
24895 Windows support - Experimental
24896 There are a couple of issues with Windows mount functionality that
24897 still require some investigations. It should be considered as experi‐
24898 mental thus far as fixes come in for this OS.
24899
24900 Most of the issues seem to be related to the difference between
24901 filesystems on Linux flavors and Windows as cache is heavily dependent
24902 on them.
24903
24904 Any reports or feedback on how cache behaves on this OS is greatly ap‐
24905 preciated.
24906
24907 • https://github.com/rclone/rclone/issues/1935
24908
24909 • https://github.com/rclone/rclone/issues/1907
24910
24911 • https://github.com/rclone/rclone/issues/1834
24912
24913 Risk of throttling
24914 Future iterations of the cache backend will make use of the pooling
24915 functionality of the cloud provider to synchronize and at the same time
24916 make writing through it more tolerant to failures.
24917
24918 There are a couple of enhancements in track to add these but in the
24919 meantime there is a valid concern that the expiring cache listings can
24920 lead to cloud provider throttles or bans due to repeated queries on it
24921 for very large mounts.
24922
24923 Some recommendations: - don't use a very small interval for entry in‐
24924 formation (--cache-info-age) - while writes aren't yet optimised, you
24925 can still write through cache which gives you the advantage of adding
24926 the file in the cache at the same time if configured to do so.
24927
24928 Future enhancements:
24929
24930 • https://github.com/rclone/rclone/issues/1937
24931
24932 • https://github.com/rclone/rclone/issues/1936
24933
24934 cache and crypt
24935 One common scenario is to keep your data encrypted in the cloud
24936 provider using the crypt remote. crypt uses a similar technique to
24937 wrap around an existing remote and handles this translation in a seam‐
24938 less way.
24939
24940 There is an issue with wrapping the remotes in this order: cloud remote
24941 -> crypt -> cache
24942
24943 During testing, I experienced a lot of bans with the remotes in this
24944 order. I suspect it might be related to how crypt opens files on the
24945 cloud provider which makes it think we're downloading the full file in‐
24946 stead of small chunks. Organizing the remotes in this order yields
24947 better results: cloud remote -> cache -> crypt
24948
24949 absolute remote paths
24950 cache can not differentiate between relative and absolute paths for the
24951 wrapped remote. Any path given in the remote config setting and on the
24952 command line will be passed to the wrapped remote as is, but for stor‐
24953 ing the chunks on disk the path will be made relative by removing any
24954 leading / character.
24955
24956 This behavior is irrelevant for most backend types, but there are back‐
24957 ends where a leading / changes the effective directory, e.g. in the
24958 sftp backend paths starting with a / are relative to the root of the
24959 SSH server and paths without are relative to the user home directory.
24960 As a result sftp:bin and sftp:/bin will share the same cache folder,
24961 even if they represent a different directory on the SSH server.
24962
24963 Cache and Remote Control (--rc)
24964 Cache supports the new --rc mode in rclone and can be remote controlled
24965 through the following end points: By default, the listener is disabled
24966 if you do not add the flag.
24967
24968 rc cache/expire
24969 Purge a remote from the cache backend. Supports either a directory or
24970 a file. It supports both encrypted and unencrypted file names if cache
24971 is wrapped by crypt.
24972
24973 Params: - remote = path to remote (required) - withData = true/false to
24974 delete cached data (chunks) as well (optional, false by default)
24975
24976 Standard options
24977 Here are the Standard options specific to cache (Cache a remote).
24978
24979 --cache-remote
24980 Remote to cache.
24981
24982 Normally should contain a ':' and a path, e.g. "myremote:path/to/dir",
24983 "myremote:bucket" or maybe "myremote:" (not recommended).
24984
24985 Properties:
24986
24987 • Config: remote
24988
24989 • Env Var: RCLONE_CACHE_REMOTE
24990
24991 • Type: string
24992
24993 • Required: true
24994
24995 --cache-plex-url
24996 The URL of the Plex server.
24997
24998 Properties:
24999
25000 • Config: plex_url
25001
25002 • Env Var: RCLONE_CACHE_PLEX_URL
25003
25004 • Type: string
25005
25006 • Required: false
25007
25008 --cache-plex-username
25009 The username of the Plex user.
25010
25011 Properties:
25012
25013 • Config: plex_username
25014
25015 • Env Var: RCLONE_CACHE_PLEX_USERNAME
25016
25017 • Type: string
25018
25019 • Required: false
25020
25021 --cache-plex-password
25022 The password of the Plex user.
25023
25024 NB Input to this must be obscured - see rclone obscure
25025 (https://rclone.org/commands/rclone_obscure/).
25026
25027 Properties:
25028
25029 • Config: plex_password
25030
25031 • Env Var: RCLONE_CACHE_PLEX_PASSWORD
25032
25033 • Type: string
25034
25035 • Required: false
25036
25037 --cache-chunk-size
25038 The size of a chunk (partial file data).
25039
25040 Use lower numbers for slower connections. If the chunk size is
25041 changed, any downloaded chunks will be invalid and cache-chunk-path
25042 will need to be cleared or unexpected EOF errors will occur.
25043
25044 Properties:
25045
25046 • Config: chunk_size
25047
25048 • Env Var: RCLONE_CACHE_CHUNK_SIZE
25049
25050 • Type: SizeSuffix
25051
25052 • Default: 5Mi
25053
25054 • Examples:
25055
25056 • "1M"
25057
25058 • 1 MiB
25059
25060 • "5M"
25061
25062 • 5 MiB
25063
25064 • "10M"
25065
25066 • 10 MiB
25067
25068 --cache-info-age
25069 How long to cache file structure information (directory listings, file
25070 size, times, etc.). If all write operations are done through the cache
25071 then you can safely make this value very large as the cache store will
25072 also be updated in real time.
25073
25074 Properties:
25075
25076 • Config: info_age
25077
25078 • Env Var: RCLONE_CACHE_INFO_AGE
25079
25080 • Type: Duration
25081
25082 • Default: 6h0m0s
25083
25084 • Examples:
25085
25086 • "1h"
25087
25088 • 1 hour
25089
25090 • "24h"
25091
25092 • 24 hours
25093
25094 • "48h"
25095
25096 • 48 hours
25097
25098 --cache-chunk-total-size
25099 The total size that the chunks can take up on the local disk.
25100
25101 If the cache exceeds this value then it will start to delete the oldest
25102 chunks until it goes under this value.
25103
25104 Properties:
25105
25106 • Config: chunk_total_size
25107
25108 • Env Var: RCLONE_CACHE_CHUNK_TOTAL_SIZE
25109
25110 • Type: SizeSuffix
25111
25112 • Default: 10Gi
25113
25114 • Examples:
25115
25116 • "500M"
25117
25118 • 500 MiB
25119
25120 • "1G"
25121
25122 • 1 GiB
25123
25124 • "10G"
25125
25126 • 10 GiB
25127
25128 Advanced options
25129 Here are the Advanced options specific to cache (Cache a remote).
25130
25131 --cache-plex-token
25132 The plex token for authentication - auto set normally.
25133
25134 Properties:
25135
25136 • Config: plex_token
25137
25138 • Env Var: RCLONE_CACHE_PLEX_TOKEN
25139
25140 • Type: string
25141
25142 • Required: false
25143
25144 --cache-plex-insecure
25145 Skip all certificate verification when connecting to the Plex server.
25146
25147 Properties:
25148
25149 • Config: plex_insecure
25150
25151 • Env Var: RCLONE_CACHE_PLEX_INSECURE
25152
25153 • Type: string
25154
25155 • Required: false
25156
25157 --cache-db-path
25158 Directory to store file structure metadata DB.
25159
25160 The remote name is used as the DB file name.
25161
25162 Properties:
25163
25164 • Config: db_path
25165
25166 • Env Var: RCLONE_CACHE_DB_PATH
25167
25168 • Type: string
25169
25170 • Default: "$HOME/.cache/rclone/cache-backend"
25171
25172 --cache-chunk-path
25173 Directory to cache chunk files.
25174
25175 Path to where partial file data (chunks) are stored locally. The re‐
25176 mote name is appended to the final path.
25177
25178 This config follows the "--cache-db-path". If you specify a custom lo‐
25179 cation for "--cache-db-path" and don't specify one for "--cache-chunk-
25180 path" then "--cache-chunk-path" will use the same path as "--cache-db-
25181 path".
25182
25183 Properties:
25184
25185 • Config: chunk_path
25186
25187 • Env Var: RCLONE_CACHE_CHUNK_PATH
25188
25189 • Type: string
25190
25191 • Default: "$HOME/.cache/rclone/cache-backend"
25192
25193 --cache-db-purge
25194 Clear all the cached data for this remote on start.
25195
25196 Properties:
25197
25198 • Config: db_purge
25199
25200 • Env Var: RCLONE_CACHE_DB_PURGE
25201
25202 • Type: bool
25203
25204 • Default: false
25205
25206 --cache-chunk-clean-interval
25207 How often should the cache perform cleanups of the chunk storage.
25208
25209 The default value should be ok for most people. If you find that the
25210 cache goes over "cache-chunk-total-size" too often then try to lower
25211 this value to force it to perform cleanups more often.
25212
25213 Properties:
25214
25215 • Config: chunk_clean_interval
25216
25217 • Env Var: RCLONE_CACHE_CHUNK_CLEAN_INTERVAL
25218
25219 • Type: Duration
25220
25221 • Default: 1m0s
25222
25223 --cache-read-retries
25224 How many times to retry a read from a cache storage.
25225
25226 Since reading from a cache stream is independent from downloading file
25227 data, readers can get to a point where there's no more data in the
25228 cache. Most of the times this can indicate a connectivity issue if
25229 cache isn't able to provide file data anymore.
25230
25231 For really slow connections, increase this to a point where the stream
25232 is able to provide data but your experience will be very stuttering.
25233
25234 Properties:
25235
25236 • Config: read_retries
25237
25238 • Env Var: RCLONE_CACHE_READ_RETRIES
25239
25240 • Type: int
25241
25242 • Default: 10
25243
25244 --cache-workers
25245 How many workers should run in parallel to download chunks.
25246
25247 Higher values will mean more parallel processing (better CPU needed)
25248 and more concurrent requests on the cloud provider. This impacts sev‐
25249 eral aspects like the cloud provider API limits, more stress on the
25250 hardware that rclone runs on but it also means that streams will be
25251 more fluid and data will be available much more faster to readers.
25252
25253 Note: If the optional Plex integration is enabled then this setting
25254 will adapt to the type of reading performed and the value specified
25255 here will be used as a maximum number of workers to use.
25256
25257 Properties:
25258
25259 • Config: workers
25260
25261 • Env Var: RCLONE_CACHE_WORKERS
25262
25263 • Type: int
25264
25265 • Default: 4
25266
25267 --cache-chunk-no-memory
25268 Disable the in-memory cache for storing chunks during streaming.
25269
25270 By default, cache will keep file data during streaming in RAM as well
25271 to provide it to readers as fast as possible.
25272
25273 This transient data is evicted as soon as it is read and the number of
25274 chunks stored doesn't exceed the number of workers. However, depending
25275 on other settings like "cache-chunk-size" and "cache-workers" this
25276 footprint can increase if there are parallel streams too (multiple
25277 files being read at the same time).
25278
25279 If the hardware permits it, use this feature to provide an overall bet‐
25280 ter performance during streaming but it can also be disabled if RAM is
25281 not available on the local machine.
25282
25283 Properties:
25284
25285 • Config: chunk_no_memory
25286
25287 • Env Var: RCLONE_CACHE_CHUNK_NO_MEMORY
25288
25289 • Type: bool
25290
25291 • Default: false
25292
25293 --cache-rps
25294 Limits the number of requests per second to the source FS (-1 to dis‐
25295 able).
25296
25297 This setting places a hard limit on the number of requests per second
25298 that cache will be doing to the cloud provider remote and try to re‐
25299 spect that value by setting waits between reads.
25300
25301 If you find that you're getting banned or limited on the cloud provider
25302 through cache and know that a smaller number of requests per second
25303 will allow you to work with it then you can use this setting for that.
25304
25305 A good balance of all the other settings should make this setting use‐
25306 less but it is available to set for more special cases.
25307
25308 NOTE: This will limit the number of requests during streams but other
25309 API calls to the cloud provider like directory listings will still
25310 pass.
25311
25312 Properties:
25313
25314 • Config: rps
25315
25316 • Env Var: RCLONE_CACHE_RPS
25317
25318 • Type: int
25319
25320 • Default: -1
25321
25322 --cache-writes
25323 Cache file data on writes through the FS.
25324
25325 If you need to read files immediately after you upload them through
25326 cache you can enable this flag to have their data stored in the cache
25327 store at the same time during upload.
25328
25329 Properties:
25330
25331 • Config: writes
25332
25333 • Env Var: RCLONE_CACHE_WRITES
25334
25335 • Type: bool
25336
25337 • Default: false
25338
25339 --cache-tmp-upload-path
25340 Directory to keep temporary files until they are uploaded.
25341
25342 This is the path where cache will use as a temporary storage for new
25343 files that need to be uploaded to the cloud provider.
25344
25345 Specifying a value will enable this feature. Without it, it is com‐
25346 pletely disabled and files will be uploaded directly to the cloud
25347 provider
25348
25349 Properties:
25350
25351 • Config: tmp_upload_path
25352
25353 • Env Var: RCLONE_CACHE_TMP_UPLOAD_PATH
25354
25355 • Type: string
25356
25357 • Required: false
25358
25359 --cache-tmp-wait-time
25360 How long should files be stored in local cache before being uploaded.
25361
25362 This is the duration that a file must wait in the temporary location
25363 cache-tmp-upload-path before it is selected for upload.
25364
25365 Note that only one file is uploaded at a time and it can take longer to
25366 start the upload if a queue formed for this purpose.
25367
25368 Properties:
25369
25370 • Config: tmp_wait_time
25371
25372 • Env Var: RCLONE_CACHE_TMP_WAIT_TIME
25373
25374 • Type: Duration
25375
25376 • Default: 15s
25377
25378 --cache-db-wait-time
25379 How long to wait for the DB to be available - 0 is unlimited.
25380
25381 Only one process can have the DB open at any one time, so rclone waits
25382 for this duration for the DB to become available before it gives an er‐
25383 ror.
25384
25385 If you set it to 0 then it will wait forever.
25386
25387 Properties:
25388
25389 • Config: db_wait_time
25390
25391 • Env Var: RCLONE_CACHE_DB_WAIT_TIME
25392
25393 • Type: Duration
25394
25395 • Default: 1s
25396
25397 Backend commands
25398 Here are the commands specific to the cache backend.
25399
25400 Run them with
25401
25402 rclone backend COMMAND remote:
25403
25404 The help below will explain what arguments each command takes.
25405
25406 See the backend (https://rclone.org/commands/rclone_backend/) command
25407 for more info on how to pass options and arguments.
25408
25409 These can be run on a running backend using the rc command backend/com‐
25410 mand (https://rclone.org/rc/#backend-command).
25411
25412 stats
25413 Print stats on the cache backend in JSON format.
25414
25415 rclone backend stats remote: [options] [<arguments>+]
25416
25418 The chunker overlay transparently splits large files into smaller
25419 chunks during upload to wrapped remote and transparently assembles them
25420 back when the file is downloaded. This allows to effectively overcome
25421 size limits imposed by storage providers.
25422
25423 Configuration
25424 To use it, first set up the underlying remote following the configura‐
25425 tion instructions for that remote. You can also use a local pathname
25426 instead of a remote.
25427
25428 First check your chosen remote is working - we'll call it remote:path
25429 here. Note that anything inside remote:path will be chunked and any‐
25430 thing outside won't. This means that if you are using a bucket-based
25431 remote (e.g. S3, B2, swift) then you should probably put the bucket in
25432 the remote s3:bucket.
25433
25434 Now configure chunker using rclone config. We will call this one over‐
25435 lay to separate it from the remote itself.
25436
25437 No remotes found, make a new one?
25438 n) New remote
25439 s) Set configuration password
25440 q) Quit config
25441 n/s/q> n
25442 name> overlay
25443 Type of storage to configure.
25444 Choose a number from below, or type in your own value
25445 [snip]
25446 XX / Transparently chunk/split large files
25447 \ "chunker"
25448 [snip]
25449 Storage> chunker
25450 Remote to chunk/unchunk.
25451 Normally should contain a ':' and a path, e.g. "myremote:path/to/dir",
25452 "myremote:bucket" or maybe "myremote:" (not recommended).
25453 Enter a string value. Press Enter for the default ("").
25454 remote> remote:path
25455 Files larger than chunk size will be split in chunks.
25456 Enter a size with suffix K,M,G,T. Press Enter for the default ("2G").
25457 chunk_size> 100M
25458 Choose how chunker handles hash sums. All modes but "none" require metadata.
25459 Enter a string value. Press Enter for the default ("md5").
25460 Choose a number from below, or type in your own value
25461 1 / Pass any hash supported by wrapped remote for non-chunked files, return nothing otherwise
25462 \ "none"
25463 2 / MD5 for composite files
25464 \ "md5"
25465 3 / SHA1 for composite files
25466 \ "sha1"
25467 4 / MD5 for all files
25468 \ "md5all"
25469 5 / SHA1 for all files
25470 \ "sha1all"
25471 6 / Copying a file to chunker will request MD5 from the source falling back to SHA1 if unsupported
25472 \ "md5quick"
25473 7 / Similar to "md5quick" but prefers SHA1 over MD5
25474 \ "sha1quick"
25475 hash_type> md5
25476 Edit advanced config? (y/n)
25477 y) Yes
25478 n) No
25479 y/n> n
25480 Remote config
25481 --------------------
25482 [overlay]
25483 type = chunker
25484 remote = remote:bucket
25485 chunk_size = 100M
25486 hash_type = md5
25487 --------------------
25488 y) Yes this is OK
25489 e) Edit this remote
25490 d) Delete this remote
25491 y/e/d> y
25492
25493 Specifying the remote
25494 In normal use, make sure the remote has a : in. If you specify the re‐
25495 mote without a : then rclone will use a local directory of that name.
25496 So if you use a remote of /path/to/secret/files then rclone will chunk
25497 stuff in that directory. If you use a remote of name then rclone will
25498 put files in a directory called name in the current directory.
25499
25500 Chunking
25501 When rclone starts a file upload, chunker checks the file size. If it
25502 doesn't exceed the configured chunk size, chunker will just pass the
25503 file to the wrapped remote. If a file is large, chunker will transpar‐
25504 ently cut data in pieces with temporary names and stream them one by
25505 one, on the fly. Each data chunk will contain the specified number of
25506 bytes, except for the last one which may have less data. If file size
25507 is unknown in advance (this is called a streaming upload), chunker will
25508 internally create a temporary copy, record its size and repeat the
25509 above process.
25510
25511 When upload completes, temporary chunk files are finally renamed. This
25512 scheme guarantees that operations can be run in parallel and look from
25513 outside as atomic. A similar method with hidden temporary chunks is
25514 used for other operations (copy/move/rename, etc.). If an operation
25515 fails, hidden chunks are normally destroyed, and the target composite
25516 file stays intact.
25517
25518 When a composite file download is requested, chunker transparently as‐
25519 sembles it by concatenating data chunks in order. As the split is
25520 trivial one could even manually concatenate data chunks together to ob‐
25521 tain the original content.
25522
25523 When the list rclone command scans a directory on wrapped remote, the
25524 potential chunk files are accounted for, grouped and assembled into
25525 composite directory entries. Any temporary chunks are hidden.
25526
25527 List and other commands can sometimes come across composite files with
25528 missing or invalid chunks, e.g. shadowed by like-named directory or
25529 another file. This usually means that wrapped file system has been di‐
25530 rectly tampered with or damaged. If chunker detects a missing chunk it
25531 will by default print warning, skip the whole incomplete group of
25532 chunks but proceed with current command. You can set the --chunker-
25533 fail-hard flag to have commands abort with error message in such cases.
25534
25535 Chunk names
25536 The default chunk name format is *.rclone_chunk.###, hence by default
25537 chunk names are BIG_FILE_NAME.rclone_chunk.001,
25538 BIG_FILE_NAME.rclone_chunk.002 etc. You can configure another name
25539 format using the name_format configuration file option. The format us‐
25540 es asterisk * as a placeholder for the base file name and one or more
25541 consecutive hash characters # as a placeholder for sequential chunk
25542 number. There must be one and only one asterisk. The number of con‐
25543 secutive hash characters defines the minimum length of a string repre‐
25544 senting a chunk number. If decimal chunk number has less digits than
25545 the number of hashes, it is left-padded by zeros. If the decimal
25546 string is longer, it is left intact. By default numbering starts from
25547 1 but there is another option that allows user to start from 0, e.g.
25548 for compatibility with legacy software.
25549
25550 For example, if name format is big_*-##.part and original file name is
25551 data.txt and numbering starts from 0, then the first chunk will be
25552 named big_data.txt-00.part, the 99th chunk will be big_data.txt-98.part
25553 and the 302nd chunk will become big_data.txt-301.part.
25554
25555 Note that list assembles composite directory entries only when chunk
25556 names match the configured format and treats non-conforming file names
25557 as normal non-chunked files.
25558
25559 When using norename transactions, chunk names will additionally have a
25560 unique file version suffix. For example,
25561 BIG_FILE_NAME.rclone_chunk.001_bp562k.
25562
25563 Metadata
25564 Besides data chunks chunker will by default create metadata object for
25565 a composite file. The object is named after the original file. Chun‐
25566 ker allows user to disable metadata completely (the none format). Note
25567 that metadata is normally not created for files smaller than the con‐
25568 figured chunk size. This may change in future rclone releases.
25569
25570 Simple JSON metadata format
25571 This is the default format. It supports hash sums and chunk validation
25572 for composite files. Meta objects carry the following fields:
25573
25574 • ver - version of format, currently 1
25575
25576 • size - total size of composite file
25577
25578 • nchunks - number of data chunks in file
25579
25580 • md5 - MD5 hashsum of composite file (if present)
25581
25582 • sha1 - SHA1 hashsum (if present)
25583
25584 • txn - identifies current version of the file
25585
25586 There is no field for composite file name as it's simply equal to the
25587 name of meta object on the wrapped remote. Please refer to respective
25588 sections for details on hashsums and modified time handling.
25589
25590 No metadata
25591 You can disable meta objects by setting the meta format option to none.
25592 In this mode chunker will scan directory for all files that follow con‐
25593 figured chunk name format, group them by detecting chunks with the same
25594 base name and show group names as virtual composite files. This method
25595 is more prone to missing chunk errors (especially missing last chunk)
25596 than format with metadata enabled.
25597
25598 Hashsums
25599 Chunker supports hashsums only when a compatible metadata is present.
25600 Hence, if you choose metadata format of none, chunker will report hash‐
25601 sum as UNSUPPORTED.
25602
25603 Please note that by default metadata is stored only for composite
25604 files. If a file is smaller than configured chunk size, chunker will
25605 transparently redirect hash requests to wrapped remote, so support de‐
25606 pends on that. You will see the empty string as a hashsum of requested
25607 type for small files if the wrapped remote doesn't support it.
25608
25609 Many storage backends support MD5 and SHA1 hash types, so does chunker.
25610 With chunker you can choose one or another but not both. MD5 is set by
25611 default as the most supported type. Since chunker keeps hashes for
25612 composite files and falls back to the wrapped remote hash for non-chun‐
25613 ked ones, we advise you to choose the same hash type as supported by
25614 wrapped remote so that your file listings look coherent.
25615
25616 If your storage backend does not support MD5 or SHA1 but you need con‐
25617 sistent file hashing, configure chunker with md5all or sha1all. These
25618 two modes guarantee given hash for all files. If wrapped remote
25619 doesn't support it, chunker will then add metadata to all files, even
25620 small. However, this can double the amount of small files in storage
25621 and incur additional service charges. You can even use chunker to
25622 force md5/sha1 support in any other remote at expense of sidecar meta
25623 objects by setting e.g. chunk_type=sha1all to force hashsums and
25624 chunk_size=1P to effectively disable chunking.
25625
25626 Normally, when a file is copied to chunker controlled remote, chunker
25627 will ask the file source for compatible file hash and revert to on-the-
25628 fly calculation if none is found. This involves some CPU overhead but
25629 provides a guarantee that given hashsum is available. Also, chunker
25630 will reject a server-side copy or move operation if source and destina‐
25631 tion hashsum types are different resulting in the extra network band‐
25632 width, too. In some rare cases this may be undesired, so chunker pro‐
25633 vides two optional choices: sha1quick and md5quick. If the source does
25634 not support primary hash type and the quick mode is enabled, chunker
25635 will try to fall back to the secondary type. This will save CPU and
25636 bandwidth but can result in empty hashsums at destination. Beware of
25637 consequences: the sync command will revert (sometimes silently) to
25638 time/size comparison if compatible hashsums between source and target
25639 are not found.
25640
25641 Modified time
25642 Chunker stores modification times using the wrapped remote so support
25643 depends on that. For a small non-chunked file the chunker overlay sim‐
25644 ply manipulates modification time of the wrapped remote file. For a
25645 composite file with metadata chunker will get and set modification time
25646 of the metadata object on the wrapped remote. If file is chunked but
25647 metadata format is none then chunker will use modification time of the
25648 first data chunk.
25649
25650 Migrations
25651 The idiomatic way to migrate to a different chunk size, hash type,
25652 transaction style or chunk naming scheme is to:
25653
25654 • Collect all your chunked files under a directory and have your chun‐
25655 ker remote point to it.
25656
25657 • Create another directory (most probably on the same cloud storage)
25658 and configure a new remote with desired metadata format, hash type,
25659 chunk naming etc.
25660
25661 • Now run rclone sync -i oldchunks: newchunks: and all your data will
25662 be transparently converted in transfer. This may take some time, yet
25663 chunker will try server-side copy if possible.
25664
25665 • After checking data integrity you may remove configuration section of
25666 the old remote.
25667
25668 If rclone gets killed during a long operation on a big composite file,
25669 hidden temporary chunks may stay in the directory. They will not be
25670 shown by the list command but will eat up your account quota. Please
25671 note that the deletefile command deletes only active chunks of a file.
25672 As a workaround, you can use remote of the wrapped file system to see
25673 them. An easy way to get rid of hidden garbage is to copy littered di‐
25674 rectory somewhere using the chunker remote and purge the original di‐
25675 rectory. The copy command will copy only active chunks while the purge
25676 will remove everything including garbage.
25677
25678 Caveats and Limitations
25679 Chunker requires wrapped remote to support server-side move (or copy +
25680 delete) operations, otherwise it will explicitly refuse to start. This
25681 is because it internally renames temporary chunk files to their final
25682 names when an operation completes successfully.
25683
25684 Chunker encodes chunk number in file name, so with default name_format
25685 setting it adds 17 characters. Also chunker adds 7 characters of tem‐
25686 porary suffix during operations. Many file systems limit base file
25687 name without path by 255 characters. Using rclone's crypt remote as a
25688 base file system limits file name by 143 characters. Thus, maximum
25689 name length is 231 for most files and 119 for chunker-over-crypt. A
25690 user in need can change name format to e.g. *.rcc## and save 10 char‐
25691 acters (provided at most 99 chunks per file).
25692
25693 Note that a move implemented using the copy-and-delete method may incur
25694 double charging with some cloud storage providers.
25695
25696 Chunker will not automatically rename existing chunks when you run
25697 rclone config on a live remote and change the chunk name format. Be‐
25698 ware that in result of this some files which have been treated as
25699 chunks before the change can pop up in directory listings as normal
25700 files and vice versa. The same warning holds for the chunk size. If
25701 you desperately need to change critical chunking settings, you should
25702 run data migration as described above.
25703
25704 If wrapped remote is case insensitive, the chunker overlay will inherit
25705 that property (so you can't have a file called "Hello.doc" and "hel‐
25706 lo.doc" in the same directory).
25707
25708 Chunker included in rclone releases up to v1.54 can sometimes fail to
25709 detect metadata produced by recent versions of rclone. We recommend
25710 users to keep rclone up-to-date to avoid data corruption.
25711
25712 Changing transactions is dangerous and requires explicit migration.
25713
25714 Standard options
25715 Here are the Standard options specific to chunker (Transparently
25716 chunk/split large files).
25717
25718 --chunker-remote
25719 Remote to chunk/unchunk.
25720
25721 Normally should contain a ':' and a path, e.g. "myremote:path/to/dir",
25722 "myremote:bucket" or maybe "myremote:" (not recommended).
25723
25724 Properties:
25725
25726 • Config: remote
25727
25728 • Env Var: RCLONE_CHUNKER_REMOTE
25729
25730 • Type: string
25731
25732 • Required: true
25733
25734 --chunker-chunk-size
25735 Files larger than chunk size will be split in chunks.
25736
25737 Properties:
25738
25739 • Config: chunk_size
25740
25741 • Env Var: RCLONE_CHUNKER_CHUNK_SIZE
25742
25743 • Type: SizeSuffix
25744
25745 • Default: 2Gi
25746
25747 --chunker-hash-type
25748 Choose how chunker handles hash sums.
25749
25750 All modes but "none" require metadata.
25751
25752 Properties:
25753
25754 • Config: hash_type
25755
25756 • Env Var: RCLONE_CHUNKER_HASH_TYPE
25757
25758 • Type: string
25759
25760 • Default: "md5"
25761
25762 • Examples:
25763
25764 • "none"
25765
25766 • Pass any hash supported by wrapped remote for non-chunked files.
25767
25768 • Return nothing otherwise.
25769
25770 • "md5"
25771
25772 • MD5 for composite files.
25773
25774 • "sha1"
25775
25776 • SHA1 for composite files.
25777
25778 • "md5all"
25779
25780 • MD5 for all files.
25781
25782 • "sha1all"
25783
25784 • SHA1 for all files.
25785
25786 • "md5quick"
25787
25788 • Copying a file to chunker will request MD5 from the source.
25789
25790 • Falling back to SHA1 if unsupported.
25791
25792 • "sha1quick"
25793
25794 • Similar to "md5quick" but prefers SHA1 over MD5.
25795
25796 Advanced options
25797 Here are the Advanced options specific to chunker (Transparently
25798 chunk/split large files).
25799
25800 --chunker-name-format
25801 String format of chunk file names.
25802
25803 The two placeholders are: base file name (*) and chunk number (#...).
25804 There must be one and only one asterisk and one or more consecutive
25805 hash characters. If chunk number has less digits than the number of
25806 hashes, it is left-padded by zeros. If there are more digits in the
25807 number, they are left as is. Possible chunk files are ignored if their
25808 name does not match given format.
25809
25810 Properties:
25811
25812 • Config: name_format
25813
25814 • Env Var: RCLONE_CHUNKER_NAME_FORMAT
25815
25816 • Type: string
25817
25818 • Default: "*.rclone_chunk.###"
25819
25820 --chunker-start-from
25821 Minimum valid chunk number. Usually 0 or 1.
25822
25823 By default chunk numbers start from 1.
25824
25825 Properties:
25826
25827 • Config: start_from
25828
25829 • Env Var: RCLONE_CHUNKER_START_FROM
25830
25831 • Type: int
25832
25833 • Default: 1
25834
25835 --chunker-meta-format
25836 Format of the metadata object or "none".
25837
25838 By default "simplejson". Metadata is a small JSON file named after the
25839 composite file.
25840
25841 Properties:
25842
25843 • Config: meta_format
25844
25845 • Env Var: RCLONE_CHUNKER_META_FORMAT
25846
25847 • Type: string
25848
25849 • Default: "simplejson"
25850
25851 • Examples:
25852
25853 • "none"
25854
25855 • Do not use metadata files at all.
25856
25857 • Requires hash type "none".
25858
25859 • "simplejson"
25860
25861 • Simple JSON supports hash sums and chunk validation.
25862
25863 • It has the following fields: ver, size, nchunks, md5, sha1.
25864
25865 --chunker-fail-hard
25866 Choose how chunker should handle files with missing or invalid chunks.
25867
25868 Properties:
25869
25870 • Config: fail_hard
25871
25872 • Env Var: RCLONE_CHUNKER_FAIL_HARD
25873
25874 • Type: bool
25875
25876 • Default: false
25877
25878 • Examples:
25879
25880 • "true"
25881
25882 • Report errors and abort current command.
25883
25884 • "false"
25885
25886 • Warn user, skip incomplete file and proceed.
25887
25888 --chunker-transactions
25889 Choose how chunker should handle temporary files during transactions.
25890
25891 Properties:
25892
25893 • Config: transactions
25894
25895 • Env Var: RCLONE_CHUNKER_TRANSACTIONS
25896
25897 • Type: string
25898
25899 • Default: "rename"
25900
25901 • Examples:
25902
25903 • "rename"
25904
25905 • Rename temporary files after a successful transaction.
25906
25907 • "norename"
25908
25909 • Leave temporary file names and write transaction ID to metadata
25910 file.
25911
25912 • Metadata is required for no rename transactions (meta format can‐
25913 not be "none").
25914
25915 • If you are using norename transactions you should be careful not
25916 to downgrade Rclone
25917
25918 • as older versions of Rclone don't support this transaction style
25919 and will misinterpret
25920
25921 • files manipulated by norename transactions.
25922
25923 • This method is EXPERIMENTAL, don't use on production systems.
25924
25925 • "auto"
25926
25927 • Rename or norename will be used depending on capabilities of the
25928 backend.
25929
25930 • If meta format is set to "none", rename transactions will always
25931 be used.
25932
25933 • This method is EXPERIMENTAL, don't use on production systems.
25934
25936 Citrix ShareFile (https://sharefile.com) is a secure file sharing and
25937 transfer service aimed as business.
25938
25939 Configuration
25940 The initial setup for Citrix ShareFile involves getting a token from
25941 Citrix ShareFile which you can in your browser. rclone config walks
25942 you through it.
25943
25944 Here is an example of how to make a remote called remote. First run:
25945
25946 rclone config
25947
25948 This will guide you through an interactive setup process:
25949
25950 No remotes found, make a new one?
25951 n) New remote
25952 s) Set configuration password
25953 q) Quit config
25954 n/s/q> n
25955 name> remote
25956 Type of storage to configure.
25957 Enter a string value. Press Enter for the default ("").
25958 Choose a number from below, or type in your own value
25959 XX / Citrix Sharefile
25960 \ "sharefile"
25961 Storage> sharefile
25962 ** See help for sharefile backend at: https://rclone.org/sharefile/ **
25963
25964 ID of the root folder
25965
25966 Leave blank to access "Personal Folders". You can use one of the
25967 standard values here or any folder ID (long hex number ID).
25968 Enter a string value. Press Enter for the default ("").
25969 Choose a number from below, or type in your own value
25970 1 / Access the Personal Folders. (Default)
25971 \ ""
25972 2 / Access the Favorites folder.
25973 \ "favorites"
25974 3 / Access all the shared folders.
25975 \ "allshared"
25976 4 / Access all the individual connectors.
25977 \ "connectors"
25978 5 / Access the home, favorites, and shared folders as well as the connectors.
25979 \ "top"
25980 root_folder_id>
25981 Edit advanced config? (y/n)
25982 y) Yes
25983 n) No
25984 y/n> n
25985 Remote config
25986 Use auto config?
25987 * Say Y if not sure
25988 * Say N if you are working on a remote or headless machine
25989 y) Yes
25990 n) No
25991 y/n> y
25992 If your browser doesn't open automatically go to the following link: http://127.0.0.1:53682/auth?state=XXX
25993 Log in and authorize rclone for access
25994 Waiting for code...
25995 Got code
25996 --------------------
25997 [remote]
25998 type = sharefile
25999 endpoint = https://XXX.sharefile.com
26000 token = {"access_token":"XXX","token_type":"bearer","refresh_token":"XXX","expiry":"2019-09-30T19:41:45.878561877+01:00"}
26001 --------------------
26002 y) Yes this is OK
26003 e) Edit this remote
26004 d) Delete this remote
26005 y/e/d> y
26006
26007 See the remote setup docs (https://rclone.org/remote_setup/) for how to
26008 set it up on a machine with no Internet browser available.
26009
26010 Note that rclone runs a webserver on your local machine to collect the
26011 token as returned from Citrix ShareFile. This only runs from the mo‐
26012 ment it opens your browser to the moment you get back the verification
26013 code. This is on http://127.0.0.1:53682/ and this it may require you
26014 to unblock it temporarily if you are running a host firewall.
26015
26016 Once configured you can then use rclone like this,
26017
26018 List directories in top level of your ShareFile
26019
26020 rclone lsd remote:
26021
26022 List all the files in your ShareFile
26023
26024 rclone ls remote:
26025
26026 To copy a local directory to an ShareFile directory called backup
26027
26028 rclone copy /home/source remote:backup
26029
26030 Paths may be as deep as required, e.g. remote:directory/subdirectory.
26031
26032 Modified time and hashes
26033 ShareFile allows modification times to be set on objects accurate to 1
26034 second. These will be used to detect whether objects need syncing or
26035 not.
26036
26037 ShareFile supports MD5 type hashes, so you can use the --checksum flag.
26038
26039 Transfers
26040 For files above 128 MiB rclone will use a chunked transfer. Rclone
26041 will upload up to --transfers chunks at the same time (shared among all
26042 the multipart uploads). Chunks are buffered in memory and are normally
26043 64 MiB so increasing --transfers will increase memory use.
26044
26045 Restricted filename characters
26046 In addition to the default restricted characters set
26047 (https://rclone.org/overview/#restricted-characters) the following
26048 characters are also replaced:
26049
26050 Character Value Replacement
26051 ────────────────────────────────
26052 \ 0x5C \
26053 * 0x2A *
26054 < 0x3C <
26055 > 0x3E >
26056 ? 0x3F ?
26057 : 0x3A :
26058 | 0x7C |
26059 " 0x22 "
26060
26061 File names can also not start or end with the following characters.
26062 These only get replaced if they are the first or last character in the
26063 name:
26064
26065 Character Value Replacement
26066 ────────────────────────────────
26067 SP 0x20 ␠
26068 . 0x2E .
26069
26070 Invalid UTF-8 bytes will also be replaced (https://rclone.org/over‐
26071 view/#invalid-utf8), as they can't be used in JSON strings.
26072
26073 Standard options
26074 Here are the Standard options specific to sharefile (Citrix Sharefile).
26075
26076 --sharefile-root-folder-id
26077 ID of the root folder.
26078
26079 Leave blank to access "Personal Folders". You can use one of the stan‐
26080 dard values here or any folder ID (long hex number ID).
26081
26082 Properties:
26083
26084 • Config: root_folder_id
26085
26086 • Env Var: RCLONE_SHAREFILE_ROOT_FOLDER_ID
26087
26088 • Type: string
26089
26090 • Required: false
26091
26092 • Examples:
26093
26094 • ""
26095
26096 • Access the Personal Folders (default).
26097
26098 • "favorites"
26099
26100 • Access the Favorites folder.
26101
26102 • "allshared"
26103
26104 • Access all the shared folders.
26105
26106 • "connectors"
26107
26108 • Access all the individual connectors.
26109
26110 • "top"
26111
26112 • Access the home, favorites, and shared folders as well as the
26113 connectors.
26114
26115 Advanced options
26116 Here are the Advanced options specific to sharefile (Citrix Sharefile).
26117
26118 --sharefile-upload-cutoff
26119 Cutoff for switching to multipart upload.
26120
26121 Properties:
26122
26123 • Config: upload_cutoff
26124
26125 • Env Var: RCLONE_SHAREFILE_UPLOAD_CUTOFF
26126
26127 • Type: SizeSuffix
26128
26129 • Default: 128Mi
26130
26131 --sharefile-chunk-size
26132 Upload chunk size.
26133
26134 Must a power of 2 >= 256k.
26135
26136 Making this larger will improve performance, but note that each chunk
26137 is buffered in memory one per transfer.
26138
26139 Reducing this will reduce memory usage but decrease performance.
26140
26141 Properties:
26142
26143 • Config: chunk_size
26144
26145 • Env Var: RCLONE_SHAREFILE_CHUNK_SIZE
26146
26147 • Type: SizeSuffix
26148
26149 • Default: 64Mi
26150
26151 --sharefile-endpoint
26152 Endpoint for API calls.
26153
26154 This is usually auto discovered as part of the oauth process, but can
26155 be set manually to something like: https://XXX.sharefile.com
26156
26157 Properties:
26158
26159 • Config: endpoint
26160
26161 • Env Var: RCLONE_SHAREFILE_ENDPOINT
26162
26163 • Type: string
26164
26165 • Required: false
26166
26167 --sharefile-encoding
26168 The encoding for the backend.
26169
26170 See the encoding section in the overview (https://rclone.org/over‐
26171 view/#encoding) for more info.
26172
26173 Properties:
26174
26175 • Config: encoding
26176
26177 • Env Var: RCLONE_SHAREFILE_ENCODING
26178
26179 • Type: MultiEncoder
26180
26181 • Default: Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,Back‐
26182 Slash,Ctl,LeftSpace,LeftPeriod,RightSpace,RightPeriod,InvalidUtf8,Dot
26183
26184 Limitations
26185 Note that ShareFile is case insensitive so you can't have a file called
26186 "Hello.doc" and one called "hello.doc".
26187
26188 ShareFile only supports filenames up to 256 characters in length.
26189
26190 rclone about is not supported by the Citrix ShareFile backend. Back‐
26191 ends without this capability cannot determine free space for an rclone
26192 mount or use policy mfs (most free space) as a member of an rclone
26193 union remote.
26194
26195 See List of backends that do not support rclone about
26196 (https://rclone.org/overview/#optional-features) and rclone about
26197 (https://rclone.org/commands/rclone_about/)
26198
26200 Rclone crypt remotes encrypt and decrypt other remotes.
26201
26202 A remote of type crypt does not access a storage system
26203 (https://rclone.org/overview/) directly, but instead wraps another re‐
26204 mote, which in turn accesses the storage system. This is similar to
26205 how alias (https://rclone.org/alias/), union
26206 (https://rclone.org/union/), chunker (https://rclone.org/chunker/) and
26207 a few others work. It makes the usage very flexible, as you can add a
26208 layer, in this case an encryption layer, on top of any other backend,
26209 even in multiple layers. Rclone's functionality can be used as with
26210 any other remote, for example you can mount (https://rclone.org/com‐
26211 mands/rclone_mount/) a crypt remote.
26212
26213 Accessing a storage system through a crypt remote realizes client-side
26214 encryption, which makes it safe to keep your data in a location you do
26215 not trust will not get compromised. When working against the crypt re‐
26216 mote, rclone will automatically encrypt (before uploading) and decrypt
26217 (after downloading) on your local system as needed on the fly, leaving
26218 the data encrypted at rest in the wrapped remote. If you access the
26219 storage system using an application other than rclone, or access the
26220 wrapped remote directly using rclone, there will not be any encryp‐
26221 tion/decryption: Downloading existing content will just give you the
26222 encrypted (scrambled) format, and anything you upload will not become
26223 encrypted.
26224
26225 The encryption is a secret-key encryption (also called symmetric key
26226 encryption) algorithm, where a password (or pass phrase) is used to
26227 generate real encryption key. The password can be supplied by user, or
26228 you may chose to let rclone generate one. It will be stored in the
26229 configuration file, in a lightly obscured form. If you are in an envi‐
26230 ronment where you are not able to keep your configuration secured, you
26231 should add configuration encryption (https://rclone.org/docs/#configu‐
26232 ration-encryption) as protection. As long as you have this configura‐
26233 tion file, you will be able to decrypt your data. Without the configu‐
26234 ration file, as long as you remember the password (or keep it in a safe
26235 place), you can re-create the configuration and gain access to the ex‐
26236 isting data. You may also configure a corresponding remote in a dif‐
26237 ferent installation to access the same data. See below for guidance to
26238 changing password.
26239
26240 Encryption uses cryptographic salt (https://en.wikipedia.org/wi‐
26241 ki/Salt_(cryptography)), to permute the encryption key so that the same
26242 string may be encrypted in different ways. When configuring the crypt
26243 remote it is optional to enter a salt, or to let rclone generate a
26244 unique salt. If omitted, rclone uses a built-in unique string. Nor‐
26245 mally in cryptography, the salt is stored together with the encrypted
26246 content, and do not have to be memorized by the user. This is not the
26247 case in rclone, because rclone does not store any additional informa‐
26248 tion on the remotes. Use of custom salt is effectively a second pass‐
26249 word that must be memorized.
26250
26251 File content encryption is performed using NaCl SecretBox
26252 (https://godoc.org/golang.org/x/crypto/nacl/secretbox), based on XSal‐
26253 sa20 cipher and Poly1305 for integrity. Names (file- and directory
26254 names) are also encrypted by default, but this has some implications
26255 and is therefore possible to turned off.
26256
26257 Configuration
26258 Here is an example of how to make a remote called secret.
26259
26260 To use crypt, first set up the underlying remote. Follow the rclone
26261 config instructions for the specific backend.
26262
26263 Before configuring the crypt remote, check the underlying remote is
26264 working. In this example the underlying remote is called remote. We
26265 will configure a path path within this remote to contain the encrypted
26266 content. Anything inside remote:path will be encrypted and anything
26267 outside will not.
26268
26269 Configure crypt using rclone config. In this example the crypt remote
26270 is called secret, to differentiate it from the underlying remote.
26271
26272 When you are done you can use the crypt remote named secret just as you
26273 would with any other remote, e.g. rclone copy D:\docs secret:\docs,
26274 and rclone will encrypt and decrypt as needed on the fly. If you ac‐
26275 cess the wrapped remote remote:path directly you will bypass the en‐
26276 cryption, and anything you read will be in encrypted form, and anything
26277 you write will be unencrypted. To avoid issues it is best to configure
26278 a dedicated path for encrypted content, and access it exclusively
26279 through a crypt remote.
26280
26281 No remotes found, make a new one?
26282 n) New remote
26283 s) Set configuration password
26284 q) Quit config
26285 n/s/q> n
26286 name> secret
26287 Type of storage to configure.
26288 Enter a string value. Press Enter for the default ("").
26289 Choose a number from below, or type in your own value
26290 [snip]
26291 XX / Encrypt/Decrypt a remote
26292 \ "crypt"
26293 [snip]
26294 Storage> crypt
26295 ** See help for crypt backend at: https://rclone.org/crypt/ **
26296
26297 Remote to encrypt/decrypt.
26298 Normally should contain a ':' and a path, eg "myremote:path/to/dir",
26299 "myremote:bucket" or maybe "myremote:" (not recommended).
26300 Enter a string value. Press Enter for the default ("").
26301 remote> remote:path
26302 How to encrypt the filenames.
26303 Enter a string value. Press Enter for the default ("standard").
26304 Choose a number from below, or type in your own value.
26305 / Encrypt the filenames.
26306 1 | See the docs for the details.
26307 \ "standard"
26308 2 / Very simple filename obfuscation.
26309 \ "obfuscate"
26310 / Don't encrypt the file names.
26311 3 | Adds a ".bin" extension only.
26312 \ "off"
26313 filename_encryption>
26314 Option to either encrypt directory names or leave them intact.
26315
26316 NB If filename_encryption is "off" then this option will do nothing.
26317 Enter a boolean value (true or false). Press Enter for the default ("true").
26318 Choose a number from below, or type in your own value
26319 1 / Encrypt directory names.
26320 \ "true"
26321 2 / Don't encrypt directory names, leave them intact.
26322 \ "false"
26323 directory_name_encryption>
26324 Password or pass phrase for encryption.
26325 y) Yes type in my own password
26326 g) Generate random password
26327 y/g> y
26328 Enter the password:
26329 password:
26330 Confirm the password:
26331 password:
26332 Password or pass phrase for salt. Optional but recommended.
26333 Should be different to the previous password.
26334 y) Yes type in my own password
26335 g) Generate random password
26336 n) No leave this optional password blank (default)
26337 y/g/n> g
26338 Password strength in bits.
26339 64 is just about memorable
26340 128 is secure
26341 1024 is the maximum
26342 Bits> 128
26343 Your password is: JAsJvRcgR-_veXNfy_sGmQ
26344 Use this password? Please note that an obscured version of this
26345 password (and not the password itself) will be stored under your
26346 configuration file, so keep this generated password in a safe place.
26347 y) Yes (default)
26348 n) No
26349 y/n>
26350 Edit advanced config? (y/n)
26351 y) Yes
26352 n) No (default)
26353 y/n>
26354 Remote config
26355 --------------------
26356 [secret]
26357 type = crypt
26358 remote = remote:path
26359 password = *** ENCRYPTED ***
26360 password2 = *** ENCRYPTED ***
26361 --------------------
26362 y) Yes this is OK (default)
26363 e) Edit this remote
26364 d) Delete this remote
26365 y/e/d>
26366
26367 Important The crypt password stored in rclone.conf is lightly obscured.
26368 That only protects it from cursory inspection. It is not secure unless
26369 configuration encryption (https://rclone.org/docs/#configuration-en‐
26370 cryption) of rclone.conf is specified.
26371
26372 A long passphrase is recommended, or rclone config can generate a ran‐
26373 dom one.
26374
26375 The obscured password is created using AES-CTR with a static key. The
26376 salt is stored verbatim at the beginning of the obscured password.
26377 This static key is shared between all versions of rclone.
26378
26379 If you reconfigure rclone with the same passwords/passphrases elsewhere
26380 it will be compatible, but the obscured version will be different due
26381 to the different salt.
26382
26383 Rclone does not encrypt
26384
26385 • file length - this can be calculated within 16 bytes
26386
26387 • modification time - used for syncing
26388
26389 Specifying the remote
26390 When configuring the remote to encrypt/decrypt, you may specify any
26391 string that rclone accepts as a source/destination of other commands.
26392
26393 The primary use case is to specify the path into an already configured
26394 remote (e.g. remote:path/to/dir or remote:bucket), such that data in a
26395 remote untrusted location can be stored encrypted.
26396
26397 You may also specify a local filesystem path, such as /path/to/dir on
26398 Linux, C:\path\to\dir on Windows. By creating a crypt remote pointing
26399 to such a local filesystem path, you can use rclone as a utility for
26400 pure local file encryption, for example to keep encrypted files on a
26401 removable USB drive.
26402
26403 Note: A string which do not contain a : will by rclone be treated as a
26404 relative path in the local filesystem. For example, if you enter the
26405 name remote without the trailing :, it will be treated as a subdirecto‐
26406 ry of the current directory with name "remote".
26407
26408 If a path remote:path/to/dir is specified, rclone stores encrypted
26409 files in path/to/dir on the remote. With file name encryption, files
26410 saved to secret:subdir/subfile are stored in the unencrypted path
26411 path/to/dir but the subdir/subpath element is encrypted.
26412
26413 The path you specify does not have to exist, rclone will create it when
26414 needed.
26415
26416 If you intend to use the wrapped remote both directly for keeping unen‐
26417 crypted content, as well as through a crypt remote for encrypted con‐
26418 tent, it is recommended to point the crypt remote to a separate direc‐
26419 tory within the wrapped remote. If you use a bucket-based storage sys‐
26420 tem (e.g. Swift, S3, Google Compute Storage, B2) it is generally ad‐
26421 visable to wrap the crypt remote around a specific bucket (s3:bucket).
26422 If wrapping around the entire root of the storage (s3:), and use the
26423 optional file name encryption, rclone will encrypt the bucket name.
26424
26425 Changing password
26426 Should the password, or the configuration file containing a lightly ob‐
26427 scured form of the password, be compromised, you need to re-encrypt
26428 your data with a new password. Since rclone uses secret-key encryp‐
26429 tion, where the encryption key is generated directly from the password
26430 kept on the client, it is not possible to change the password/key of
26431 already encrypted content. Just changing the password configured for
26432 an existing crypt remote means you will no longer able to decrypt any
26433 of the previously encrypted content. The only possibility is to re-up‐
26434 load everything via a crypt remote configured with your new password.
26435
26436 Depending on the size of your data, your bandwidth, storage quota etc,
26437 there are different approaches you can take: - If you have everything
26438 in a different location, for example on your local system, you could
26439 remove all of the prior encrypted files, change the password for your
26440 configured crypt remote (or delete and re-create the crypt configura‐
26441 tion), and then re-upload everything from the alternative location. -
26442 If you have enough space on the storage system you can create a new
26443 crypt remote pointing to a separate directory on the same backend, and
26444 then use rclone to copy everything from the original crypt remote to
26445 the new, effectively decrypting everything on the fly using the old
26446 password and re-encrypting using the new password. When done, delete
26447 the original crypt remote directory and finally the rclone crypt con‐
26448 figuration with the old password. All data will be streamed from the
26449 storage system and back, so you will get half the bandwidth and be
26450 charged twice if you have upload and download quota on the storage sys‐
26451 tem.
26452
26453 Note: A security problem related to the random password generator was
26454 fixed in rclone version 1.53.3 (released 2020-11-19). Passwords gener‐
26455 ated by rclone config in version 1.49.0 (released 2019-08-26) to 1.53.2
26456 (released 2020-10-26) are not considered secure and should be changed.
26457 If you made up your own password, or used rclone version older than
26458 1.49.0 or newer than 1.53.2 to generate it, you are not affected by
26459 this issue. See issue #4783 (https://github.com/rclone/rclone/is‐
26460 sues/4783) for more details, and a tool you can use to check if you are
26461 affected.
26462
26463 Example
26464 Create the following file structure using "standard" file name encryp‐
26465 tion.
26466
26467 plaintext/
26468 ├── file0.txt
26469 ├── file1.txt
26470 └── subdir
26471 ├── file2.txt
26472 ├── file3.txt
26473 └── subsubdir
26474 └── file4.txt
26475
26476 Copy these to the remote, and list them
26477
26478 $ rclone -q copy plaintext secret:
26479 $ rclone -q ls secret:
26480 7 file1.txt
26481 6 file0.txt
26482 8 subdir/file2.txt
26483 10 subdir/subsubdir/file4.txt
26484 9 subdir/file3.txt
26485
26486 The crypt remote looks like
26487
26488 $ rclone -q ls remote:path
26489 55 hagjclgavj2mbiqm6u6cnjjqcg
26490 54 v05749mltvv1tf4onltun46gls
26491 57 86vhrsv86mpbtd3a0akjuqslj8/dlj7fkq4kdq72emafg7a7s41uo
26492 58 86vhrsv86mpbtd3a0akjuqslj8/7uu829995du6o42n32otfhjqp4/b9pausrfansjth5ob3jkdqd4lc
26493 56 86vhrsv86mpbtd3a0akjuqslj8/8njh1sk437gttmep3p70g81aps
26494
26495 The directory structure is preserved
26496
26497 $ rclone -q ls secret:subdir
26498 8 file2.txt
26499 9 file3.txt
26500 10 subsubdir/file4.txt
26501
26502 Without file name encryption .bin extensions are added to underlying
26503 names. This prevents the cloud provider attempting to interpret file
26504 content.
26505
26506 $ rclone -q ls remote:path
26507 54 file0.txt.bin
26508 57 subdir/file3.txt.bin
26509 56 subdir/file2.txt.bin
26510 58 subdir/subsubdir/file4.txt.bin
26511 55 file1.txt.bin
26512
26513 File name encryption modes
26514 Off
26515
26516 • doesn't hide file names or directory structure
26517
26518 • allows for longer file names (~246 characters)
26519
26520 • can use sub paths and copy single files
26521
26522 Standard
26523
26524 • file names encrypted
26525
26526 • file names can't be as long (~143 characters)
26527
26528 • can use sub paths and copy single files
26529
26530 • directory structure visible
26531
26532 • identical files names will have identical uploaded names
26533
26534 • can use shortcuts to shorten the directory recursion
26535
26536 Obfuscation
26537
26538 This is a simple "rotate" of the filename, with each file having a rot
26539 distance based on the filename. Rclone stores the distance at the be‐
26540 ginning of the filename. A file called "hello" may become "53.jgnnq".
26541
26542 Obfuscation is not a strong encryption of filenames, but hinders auto‐
26543 mated scanning tools picking up on filename patterns. It is an inter‐
26544 mediate between "off" and "standard" which allows for longer path seg‐
26545 ment names.
26546
26547 There is a possibility with some unicode based filenames that the ob‐
26548 fuscation is weak and may map lower case characters to upper case
26549 equivalents.
26550
26551 Obfuscation cannot be relied upon for strong protection.
26552
26553 • file names very lightly obfuscated
26554
26555 • file names can be longer than standard encryption
26556
26557 • can use sub paths and copy single files
26558
26559 • directory structure visible
26560
26561 • identical files names will have identical uploaded names
26562
26563 Cloud storage systems have limits on file name length and total path
26564 length which rclone is more likely to breach using "Standard" file name
26565 encryption. Where file names are less than 156 characters in length
26566 issues should not be encountered, irrespective of cloud storage
26567 provider.
26568
26569 An experimental advanced option filename_encoding is now provided to
26570 address this problem to a certain degree. For cloud storage systems
26571 with case sensitive file names (e.g. Google Drive), base64 can be used
26572 to reduce file name length. For cloud storage systems using UTF-16 to
26573 store file names internally (e.g. OneDrive), base32768 can be used to
26574 drastically reduce file name length.
26575
26576 An alternative, future rclone file name encryption mode may tolerate
26577 backend provider path length limits.
26578
26579 Directory name encryption
26580 Crypt offers the option of encrypting dir names or leaving them intact.
26581 There are two options:
26582
26583 True
26584
26585 Encrypts the whole file path including directory names Example:
26586 1/12/123.txt is encrypted to
26587 p0e52nreeaj0a5ea7s64m4j72s/l42g6771hnv3an9cgc8cr2n1ng/qgm4avr35m5loi1th53ato71v0
26588
26589 False
26590
26591 Only encrypts file names, skips directory names Example: 1/12/123.txt
26592 is encrypted to 1/12/qgm4avr35m5loi1th53ato71v0
26593
26594 Modified time and hashes
26595 Crypt stores modification times using the underlying remote so support
26596 depends on that.
26597
26598 Hashes are not stored for crypt. However the data integrity is pro‐
26599 tected by an extremely strong crypto authenticator.
26600
26601 Use the rclone cryptcheck command to check the integrity of a crypted
26602 remote instead of rclone check which can't check the checksums proper‐
26603 ly.
26604
26605 Standard options
26606 Here are the Standard options specific to crypt (Encrypt/Decrypt a re‐
26607 mote).
26608
26609 --crypt-remote
26610 Remote to encrypt/decrypt.
26611
26612 Normally should contain a ':' and a path, e.g. "myremote:path/to/dir",
26613 "myremote:bucket" or maybe "myremote:" (not recommended).
26614
26615 Properties:
26616
26617 • Config: remote
26618
26619 • Env Var: RCLONE_CRYPT_REMOTE
26620
26621 • Type: string
26622
26623 • Required: true
26624
26625 --crypt-filename-encryption
26626 How to encrypt the filenames.
26627
26628 Properties:
26629
26630 • Config: filename_encryption
26631
26632 • Env Var: RCLONE_CRYPT_FILENAME_ENCRYPTION
26633
26634 • Type: string
26635
26636 • Default: "standard"
26637
26638 • Examples:
26639
26640 • "standard"
26641
26642 • Encrypt the filenames.
26643
26644 • See the docs for the details.
26645
26646 • "obfuscate"
26647
26648 • Very simple filename obfuscation.
26649
26650 • "off"
26651
26652 • Don't encrypt the file names.
26653
26654 • Adds a ".bin" extension only.
26655
26656 --crypt-directory-name-encryption
26657 Option to either encrypt directory names or leave them intact.
26658
26659 NB If filename_encryption is "off" then this option will do nothing.
26660
26661 Properties:
26662
26663 • Config: directory_name_encryption
26664
26665 • Env Var: RCLONE_CRYPT_DIRECTORY_NAME_ENCRYPTION
26666
26667 • Type: bool
26668
26669 • Default: true
26670
26671 • Examples:
26672
26673 • "true"
26674
26675 • Encrypt directory names.
26676
26677 • "false"
26678
26679 • Don't encrypt directory names, leave them intact.
26680
26681 --crypt-password
26682 Password or pass phrase for encryption.
26683
26684 NB Input to this must be obscured - see rclone obscure
26685 (https://rclone.org/commands/rclone_obscure/).
26686
26687 Properties:
26688
26689 • Config: password
26690
26691 • Env Var: RCLONE_CRYPT_PASSWORD
26692
26693 • Type: string
26694
26695 • Required: true
26696
26697 --crypt-password2
26698 Password or pass phrase for salt.
26699
26700 Optional but recommended. Should be different to the previous pass‐
26701 word.
26702
26703 NB Input to this must be obscured - see rclone obscure
26704 (https://rclone.org/commands/rclone_obscure/).
26705
26706 Properties:
26707
26708 • Config: password2
26709
26710 • Env Var: RCLONE_CRYPT_PASSWORD2
26711
26712 • Type: string
26713
26714 • Required: false
26715
26716 Advanced options
26717 Here are the Advanced options specific to crypt (Encrypt/Decrypt a re‐
26718 mote).
26719
26720 --crypt-server-side-across-configs
26721 Allow server-side operations (e.g. copy) to work across different
26722 crypt configs.
26723
26724 Normally this option is not what you want, but if you have two crypts
26725 pointing to the same backend you can use it.
26726
26727 This can be used, for example, to change file name encryption type
26728 without re-uploading all the data. Just make two crypt backends point‐
26729 ing to two different directories with the single changed parameter and
26730 use rclone move to move the files between the crypt remotes.
26731
26732 Properties:
26733
26734 • Config: server_side_across_configs
26735
26736 • Env Var: RCLONE_CRYPT_SERVER_SIDE_ACROSS_CONFIGS
26737
26738 • Type: bool
26739
26740 • Default: false
26741
26742 --crypt-show-mapping
26743 For all files listed show how the names encrypt.
26744
26745 If this flag is set then for each file that the remote is asked to
26746 list, it will log (at level INFO) a line stating the decrypted file
26747 name and the encrypted file name.
26748
26749 This is so you can work out which encrypted names are which decrypted
26750 names just in case you need to do something with the encrypted file
26751 names, or for debugging purposes.
26752
26753 Properties:
26754
26755 • Config: show_mapping
26756
26757 • Env Var: RCLONE_CRYPT_SHOW_MAPPING
26758
26759 • Type: bool
26760
26761 • Default: false
26762
26763 --crypt-no-data-encryption
26764 Option to either encrypt file data or leave it unencrypted.
26765
26766 Properties:
26767
26768 • Config: no_data_encryption
26769
26770 • Env Var: RCLONE_CRYPT_NO_DATA_ENCRYPTION
26771
26772 • Type: bool
26773
26774 • Default: false
26775
26776 • Examples:
26777
26778 • "true"
26779
26780 • Don't encrypt file data, leave it unencrypted.
26781
26782 • "false"
26783
26784 • Encrypt file data.
26785
26786 --crypt-filename-encoding
26787 How to encode the encrypted filename to text string.
26788
26789 This option could help with shortening the encrypted filename. The
26790 suitable option would depend on the way your remote count the filename
26791 length and if it's case sensitive.
26792
26793 Properties:
26794
26795 • Config: filename_encoding
26796
26797 • Env Var: RCLONE_CRYPT_FILENAME_ENCODING
26798
26799 • Type: string
26800
26801 • Default: "base32"
26802
26803 • Examples:
26804
26805 • "base32"
26806
26807 • Encode using base32. Suitable for all remote.
26808
26809 • "base64"
26810
26811 • Encode using base64. Suitable for case sensitive remote.
26812
26813 • "base32768"
26814
26815 • Encode using base32768. Suitable if your remote counts UTF-16 or
26816
26817 • Unicode codepoint instead of UTF-8 byte length. (Eg. Onedrive)
26818
26819 Metadata
26820 Any metadata supported by the underlying remote is read and written.
26821
26822 See the metadata (https://rclone.org/docs/#metadata) docs for more in‐
26823 fo.
26824
26825 Backend commands
26826 Here are the commands specific to the crypt backend.
26827
26828 Run them with
26829
26830 rclone backend COMMAND remote:
26831
26832 The help below will explain what arguments each command takes.
26833
26834 See the backend (https://rclone.org/commands/rclone_backend/) command
26835 for more info on how to pass options and arguments.
26836
26837 These can be run on a running backend using the rc command backend/com‐
26838 mand (https://rclone.org/rc/#backend-command).
26839
26840 encode
26841 Encode the given filename(s)
26842
26843 rclone backend encode remote: [options] [<arguments>+]
26844
26845 This encodes the filenames given as arguments returning a list of
26846 strings of the encoded results.
26847
26848 Usage Example:
26849
26850 rclone backend encode crypt: file1 [file2...]
26851 rclone rc backend/command command=encode fs=crypt: file1 [file2...]
26852
26853 decode
26854 Decode the given filename(s)
26855
26856 rclone backend decode remote: [options] [<arguments>+]
26857
26858 This decodes the filenames given as arguments returning a list of
26859 strings of the decoded results. It will return an error if any of the
26860 inputs are invalid.
26861
26862 Usage Example:
26863
26864 rclone backend decode crypt: encryptedfile1 [encryptedfile2...]
26865 rclone rc backend/command command=decode fs=crypt: encryptedfile1 [encryptedfile2...]
26866
26867 Backing up a crypted remote
26868 If you wish to backup a crypted remote, it is recommended that you use
26869 rclone sync on the encrypted files, and make sure the passwords are the
26870 same in the new encrypted remote.
26871
26872 This will have the following advantages
26873
26874 • rclone sync will check the checksums while copying
26875
26876 • you can use rclone check between the encrypted remotes
26877
26878 • you don't decrypt and encrypt unnecessarily
26879
26880 For example, let's say you have your original remote at remote: with
26881 the encrypted version at eremote: with path remote:crypt. You would
26882 then set up the new remote remote2: and then the encrypted version ere‐
26883 mote2: with path remote2:crypt using the same passwords as eremote:.
26884
26885 To sync the two remotes you would do
26886
26887 rclone sync -i remote:crypt remote2:crypt
26888
26889 And to check the integrity you would do
26890
26891 rclone check remote:crypt remote2:crypt
26892
26893 File formats
26894 File encryption
26895 Files are encrypted 1:1 source file to destination object. The file
26896 has a header and is divided into chunks.
26897
26898 Header
26899 • 8 bytes magic string RCLONE\x00\x00
26900
26901 • 24 bytes Nonce (IV)
26902
26903 The initial nonce is generated from the operating systems crypto strong
26904 random number generator. The nonce is incremented for each chunk read
26905 making sure each nonce is unique for each block written. The chance of
26906 a nonce being re-used is minuscule. If you wrote an exabyte of data
26907 (10¹⁸ bytes) you would have a probability of approximately 2×10⁻³² of
26908 re-using a nonce.
26909
26910 Chunk
26911 Each chunk will contain 64 KiB of data, except for the last one which
26912 may have less data. The data chunk is in standard NaCl SecretBox for‐
26913 mat. SecretBox uses XSalsa20 and Poly1305 to encrypt and authenticate
26914 messages.
26915
26916 Each chunk contains:
26917
26918 • 16 Bytes of Poly1305 authenticator
26919
26920 • 1 - 65536 bytes XSalsa20 encrypted data
26921
26922 64k chunk size was chosen as the best performing chunk size (the au‐
26923 thenticator takes too much time below this and the performance drops
26924 off due to cache effects above this). Note that these chunks are
26925 buffered in memory so they can't be too big.
26926
26927 This uses a 32 byte (256 bit key) key derived from the user password.
26928
26929 Examples
26930 1 byte file will encrypt to
26931
26932 • 32 bytes header
26933
26934 • 17 bytes data chunk
26935
26936 49 bytes total
26937
26938 1 MiB (1048576 bytes) file will encrypt to
26939
26940 • 32 bytes header
26941
26942 • 16 chunks of 65568 bytes
26943
26944 1049120 bytes total (a 0.05% overhead). This is the overhead for big
26945 files.
26946
26947 Name encryption
26948 File names are encrypted segment by segment - the path is broken up in‐
26949 to / separated strings and these are encrypted individually.
26950
26951 File segments are padded using PKCS#7 to a multiple of 16 bytes before
26952 encryption.
26953
26954 They are then encrypted with EME using AES with 256 bit key. EME (ECB-
26955 Mix-ECB) is a wide-block encryption mode presented in the 2003 paper "A
26956 Parallelizable Enciphering Mode" by Halevi and Rogaway.
26957
26958 This makes for deterministic encryption which is what we want - the
26959 same filename must encrypt to the same thing otherwise we can't find it
26960 on the cloud storage system.
26961
26962 This means that
26963
26964 • filenames with the same name will encrypt the same
26965
26966 • filenames which start the same won't have a common prefix
26967
26968 This uses a 32 byte key (256 bits) and a 16 byte (128 bits) IV both of
26969 which are derived from the user password.
26970
26971 After encryption they are written out using a modified version of stan‐
26972 dard base32 encoding as described in RFC4648. The standard encoding is
26973 modified in two ways:
26974
26975 • it becomes lower case (no-one likes upper case filenames!)
26976
26977 • we strip the padding character =
26978
26979 base32 is used rather than the more efficient base64 so rclone can be
26980 used on case insensitive remotes (e.g. Windows, Amazon Drive).
26981
26982 Key derivation
26983 Rclone uses scrypt with parameters N=16384, r=8, p=1 with an optional
26984 user supplied salt (password2) to derive the 32+32+16 = 80 bytes of key
26985 material required. If the user doesn't supply a salt then rclone uses
26986 an internal one.
26987
26988 scrypt makes it impractical to mount a dictionary attack on rclone en‐
26989 crypted data. For full protection against this you should always use a
26990 salt.
26991
26992 SEE ALSO
26993 • rclone cryptdecode (https://rclone.org/commands/rclone_cryptdecode/)
26994 - Show forward/reverse mapping of encrypted filenames
26995
26997 Warning
26998 This remote is currently experimental. Things may break and data may
26999 be lost. Anything you do with this remote is at your own risk. Please
27000 understand the risks associated with using experimental code and don't
27001 use this remote in critical applications.
27002
27003 The Compress remote adds compression to another remote. It is best
27004 used with remotes containing many large compressible files.
27005
27006 Configuration
27007 To use this remote, all you need to do is specify another remote and a
27008 compression mode to use:
27009
27010 Current remotes:
27011
27012 Name Type
27013 ==== ====
27014 remote_to_press sometype
27015
27016 e) Edit existing remote
27017 $ rclone config
27018 n) New remote
27019 d) Delete remote
27020 r) Rename remote
27021 c) Copy remote
27022 s) Set configuration password
27023 q) Quit config
27024 e/n/d/r/c/s/q> n
27025 name> compress
27026 ...
27027 8 / Compress a remote
27028 \ "compress"
27029 ...
27030 Storage> compress
27031 ** See help for compress backend at: https://rclone.org/compress/ **
27032
27033 Remote to compress.
27034 Enter a string value. Press Enter for the default ("").
27035 remote> remote_to_press:subdir
27036 Compression mode.
27037 Enter a string value. Press Enter for the default ("gzip").
27038 Choose a number from below, or type in your own value
27039 1 / Gzip compression balanced for speed and compression strength.
27040 \ "gzip"
27041 compression_mode> gzip
27042 Edit advanced config? (y/n)
27043 y) Yes
27044 n) No (default)
27045 y/n> n
27046 Remote config
27047 --------------------
27048 [compress]
27049 type = compress
27050 remote = remote_to_press:subdir
27051 compression_mode = gzip
27052 --------------------
27053 y) Yes this is OK (default)
27054 e) Edit this remote
27055 d) Delete this remote
27056 y/e/d> y
27057
27058 Compression Modes
27059 Currently only gzip compression is supported. It provides a decent
27060 balance between speed and size and is well supported by other applica‐
27061 tions. Compression strength can further be configured via an advanced
27062 setting where 0 is no compression and 9 is strongest compression.
27063
27064 File types
27065 If you open a remote wrapped by compress, you will see that there are
27066 many files with an extension corresponding to the compression algorithm
27067 you chose. These files are standard files that can be opened by vari‐
27068 ous archive programs, but they have some hidden metadata that allows
27069 them to be used by rclone. While you may download and decompress these
27070 files at will, do not manually delete or rename files. Files without
27071 correct metadata files will not be recognized by rclone.
27072
27073 File names
27074 The compressed files will be named *.###########.gz where * is the base
27075 file and the # part is base64 encoded size of the uncompressed file.
27076 The file names should not be changed by anything other than the rclone
27077 compression backend.
27078
27079 Standard options
27080 Here are the Standard options specific to compress (Compress a remote).
27081
27082 --compress-remote
27083 Remote to compress.
27084
27085 Properties:
27086
27087 • Config: remote
27088
27089 • Env Var: RCLONE_COMPRESS_REMOTE
27090
27091 • Type: string
27092
27093 • Required: true
27094
27095 --compress-mode
27096 Compression mode.
27097
27098 Properties:
27099
27100 • Config: mode
27101
27102 • Env Var: RCLONE_COMPRESS_MODE
27103
27104 • Type: string
27105
27106 • Default: "gzip"
27107
27108 • Examples:
27109
27110 • "gzip"
27111
27112 • Standard gzip compression with fastest parameters.
27113
27114 Advanced options
27115 Here are the Advanced options specific to compress (Compress a remote).
27116
27117 --compress-level
27118 GZIP compression level (-2 to 9).
27119
27120 Generally -1 (default, equivalent to 5) is recommended. Levels 1 to 9
27121 increase compression at the cost of speed. Going past 6 generally of‐
27122 fers very little return.
27123
27124 Level -2 uses Huffman encoding only. Only use if you know what you are
27125 doing. Level 0 turns off compression.
27126
27127 Properties:
27128
27129 • Config: level
27130
27131 • Env Var: RCLONE_COMPRESS_LEVEL
27132
27133 • Type: int
27134
27135 • Default: -1
27136
27137 --compress-ram-cache-limit
27138 Some remotes don't allow the upload of files with unknown size. In
27139 this case the compressed file will need to be cached to determine it's
27140 size.
27141
27142 Files smaller than this limit will be cached in RAM, files larger than
27143 this limit will be cached on disk.
27144
27145 Properties:
27146
27147 • Config: ram_cache_limit
27148
27149 • Env Var: RCLONE_COMPRESS_RAM_CACHE_LIMIT
27150
27151 • Type: SizeSuffix
27152
27153 • Default: 20Mi
27154
27155 Metadata
27156 Any metadata supported by the underlying remote is read and written.
27157
27158 See the metadata (https://rclone.org/docs/#metadata) docs for more in‐
27159 fo.
27160
27162 The combine backend joins remotes together into a single directory
27163 tree.
27164
27165 For example you might have a remote for images on one provider:
27166
27167 $ rclone tree s3:imagesbucket
27168 /
27169 ├── image1.jpg
27170 └── image2.jpg
27171
27172 And a remote for files on another:
27173
27174 $ rclone tree drive:important/files
27175 /
27176 ├── file1.txt
27177 └── file2.txt
27178
27179 The combine backend can join these together into a synthetic directory
27180 structure like this:
27181
27182 $ rclone tree combined:
27183 /
27184 ├── files
27185 │ ├── file1.txt
27186 │ └── file2.txt
27187 └── images
27188 ├── image1.jpg
27189 └── image2.jpg
27190
27191 You'd do this by specifying an upstreams parameter in the config like
27192 this
27193
27194 upstreams = images=s3:imagesbucket files=drive:important/files
27195
27196 During the initial setup with rclone config you will specify the up‐
27197 streams remotes as a space separated list. The upstream remotes can
27198 either be a local paths or other remotes.
27199
27200 Configuration
27201 Here is an example of how to make a combine called remote for the exam‐
27202 ple above. First run:
27203
27204 rclone config
27205
27206 This will guide you through an interactive setup process:
27207
27208 No remotes found, make a new one?
27209 n) New remote
27210 s) Set configuration password
27211 q) Quit config
27212 n/s/q> n
27213 name> remote
27214 Option Storage.
27215 Type of storage to configure.
27216 Choose a number from below, or type in your own value.
27217 ...
27218 XX / Combine several remotes into one
27219 \ (combine)
27220 ...
27221 Storage> combine
27222 Option upstreams.
27223 Upstreams for combining
27224 These should be in the form
27225 dir=remote:path dir2=remote2:path
27226 Where before the = is specified the root directory and after is the remote to
27227 put there.
27228 Embedded spaces can be added using quotes
27229 "dir=remote:path with space" "dir2=remote2:path with space"
27230 Enter a fs.SpaceSepList value.
27231 upstreams> images=s3:imagesbucket files=drive:important/files
27232 --------------------
27233 [remote]
27234 type = combine
27235 upstreams = images=s3:imagesbucket files=drive:important/files
27236 --------------------
27237 y) Yes this is OK (default)
27238 e) Edit this remote
27239 d) Delete this remote
27240 y/e/d> y
27241
27242 Configuring for Google Drive Shared Drives
27243 Rclone has a convenience feature for making a combine backend for all
27244 the shared drives you have access to.
27245
27246 Assuming your main (non shared drive) Google drive remote is called
27247 drive: you would run
27248
27249 rclone backend -o config drives drive:
27250
27251 This would produce something like this:
27252
27253 [My Drive]
27254 type = alias
27255 remote = drive,team_drive=0ABCDEF-01234567890,root_folder_id=:
27256
27257 [Test Drive]
27258 type = alias
27259 remote = drive,team_drive=0ABCDEFabcdefghijkl,root_folder_id=:
27260
27261 [AllDrives]
27262 type = combine
27263 upstreams = "My Drive=My Drive:" "Test Drive=Test Drive:"
27264
27265 If you then add that config to your config file (find it with rclone
27266 config file) then you can access all the shared drives in one place
27267 with the AllDrives: remote.
27268
27269 See the Google Drive docs (https://rclone.org/drive/#drives) for full
27270 info.
27271
27272 Standard options
27273 Here are the Standard options specific to combine (Combine several re‐
27274 motes into one).
27275
27276 --combine-upstreams
27277 Upstreams for combining
27278
27279 These should be in the form
27280
27281 dir=remote:path dir2=remote2:path
27282
27283 Where before the = is specified the root directory and after is the re‐
27284 mote to put there.
27285
27286 Embedded spaces can be added using quotes
27287
27288 "dir=remote:path with space" "dir2=remote2:path with space"
27289
27290 Properties:
27291
27292 • Config: upstreams
27293
27294 • Env Var: RCLONE_COMBINE_UPSTREAMS
27295
27296 • Type: SpaceSepList
27297
27298 • Default:
27299
27300 Metadata
27301 Any metadata supported by the underlying remote is read and written.
27302
27303 See the metadata (https://rclone.org/docs/#metadata) docs for more in‐
27304 fo.
27305
27307 Paths are specified as remote:path
27308
27309 Dropbox paths may be as deep as required, e.g. remote:directory/subdi‐
27310 rectory.
27311
27312 Configuration
27313 The initial setup for dropbox involves getting a token from Dropbox
27314 which you need to do in your browser. rclone config walks you through
27315 it.
27316
27317 Here is an example of how to make a remote called remote. First run:
27318
27319 rclone config
27320
27321 This will guide you through an interactive setup process:
27322
27323 n) New remote
27324 d) Delete remote
27325 q) Quit config
27326 e/n/d/q> n
27327 name> remote
27328 Type of storage to configure.
27329 Choose a number from below, or type in your own value
27330 [snip]
27331 XX / Dropbox
27332 \ "dropbox"
27333 [snip]
27334 Storage> dropbox
27335 Dropbox App Key - leave blank normally.
27336 app_key>
27337 Dropbox App Secret - leave blank normally.
27338 app_secret>
27339 Remote config
27340 Please visit:
27341 https://www.dropbox.com/1/oauth2/authorize?client_id=XXXXXXXXXXXXXXX&response_type=code
27342 Enter the code: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX_XXXXXXXXXX
27343 --------------------
27344 [remote]
27345 app_key =
27346 app_secret =
27347 token = XXXXXXXXXXXXXXXXXXXXXXXXXXXXX_XXXX_XXXXXXXXXXXXXXXXXXXXXXXXXXXXX
27348 --------------------
27349 y) Yes this is OK
27350 e) Edit this remote
27351 d) Delete this remote
27352 y/e/d> y
27353
27354 You can then use it like this,
27355
27356 List directories in top level of your dropbox
27357
27358 rclone lsd remote:
27359
27360 List all the files in your dropbox
27361
27362 rclone ls remote:
27363
27364 To copy a local directory to a dropbox directory called backup
27365
27366 rclone copy /home/source remote:backup
27367
27368 Dropbox for business
27369 Rclone supports Dropbox for business and Team Folders.
27370
27371 When using Dropbox for business remote: and remote:path/to/file will
27372 refer to your personal folder.
27373
27374 If you wish to see Team Folders you must use a leading / in the path,
27375 so rclone lsd remote:/ will refer to the root and show you all Team
27376 Folders and your User Folder.
27377
27378 You can then use team folders like this remote:/TeamFolder and re‐
27379 mote:/TeamFolder/path/to/file.
27380
27381 A leading / for a Dropbox personal account will do nothing, but it will
27382 take an extra HTTP transaction so it should be avoided.
27383
27384 Modified time and Hashes
27385 Dropbox supports modified times, but the only way to set a modification
27386 time is to re-upload the file.
27387
27388 This means that if you uploaded your data with an older version of
27389 rclone which didn't support the v2 API and modified times, rclone will
27390 decide to upload all your old data to fix the modification times. If
27391 you don't want this to happen use --size-only or --checksum flag to
27392 stop it.
27393
27394 Dropbox supports its own hash type (https://www.dropbox.com/develop‐
27395 ers/reference/content-hash) which is checked for all transfers.
27396
27397 Restricted filename characters
27398 Character Value Replacement
27399 ────────────────────────────────
27400 NUL 0x00 ␀
27401 / 0x2F /
27402 DEL 0x7F ␡
27403 \ 0x5C \
27404
27405 File names can also not end with the following characters. These only
27406 get replaced if they are the last character in the name:
27407
27408 Character Value Replacement
27409 ────────────────────────────────
27410 SP 0x20 ␠
27411
27412 Invalid UTF-8 bytes will also be replaced (https://rclone.org/over‐
27413 view/#invalid-utf8), as they can't be used in JSON strings.
27414
27415 Batch mode uploads
27416 Using batch mode uploads is very important for performance when using
27417 the Dropbox API. See the dropbox performance guide (https://develop‐
27418 ers.dropbox.com/dbx-performance-guide) for more info.
27419
27420 There are 3 modes rclone can use for uploads.
27421
27422 --dropbox-batch-mode off
27423 In this mode rclone will not use upload batching. This was the default
27424 before rclone v1.55. It has the disadvantage that it is very likely to
27425 encounter too_many_requests errors like this
27426
27427 NOTICE: too_many_requests/.: Too many requests or write operations. Trying again in 15 seconds.
27428
27429 When rclone receives these it has to wait for 15s or sometimes 300s be‐
27430 fore continuing which really slows down transfers.
27431
27432 This will happen especially if --transfers is large, so this mode isn't
27433 recommended except for compatibility or investigating problems.
27434
27435 --dropbox-batch-mode sync
27436 In this mode rclone will batch up uploads to the size specified by
27437 --dropbox-batch-size and commit them together.
27438
27439 Using this mode means you can use a much higher --transfers parameter
27440 (32 or 64 works fine) without receiving too_many_requests errors.
27441
27442 This mode ensures full data integrity.
27443
27444 Note that there may be a pause when quitting rclone while rclone fin‐
27445 ishes up the last batch using this mode.
27446
27447 --dropbox-batch-mode async
27448 In this mode rclone will batch up uploads to the size specified by
27449 --dropbox-batch-size and commit them together.
27450
27451 However it will not wait for the status of the batch to be returned to
27452 the caller. This means rclone can use a much bigger batch size (much
27453 bigger than --transfers), at the cost of not being able to check the
27454 status of the upload.
27455
27456 This provides the maximum possible upload speed especially with lots of
27457 small files, however rclone can't check the file got uploaded properly
27458 using this mode.
27459
27460 If you are using this mode then using "rclone check" after the transfer
27461 completes is recommended. Or you could do an initial transfer with
27462 --dropbox-batch-mode async then do a final transfer with --dropbox-
27463 batch-mode sync (the default).
27464
27465 Note that there may be a pause when quitting rclone while rclone fin‐
27466 ishes up the last batch using this mode.
27467
27468 Standard options
27469 Here are the Standard options specific to dropbox (Dropbox).
27470
27471 --dropbox-client-id
27472 OAuth Client Id.
27473
27474 Leave blank normally.
27475
27476 Properties:
27477
27478 • Config: client_id
27479
27480 • Env Var: RCLONE_DROPBOX_CLIENT_ID
27481
27482 • Type: string
27483
27484 • Required: false
27485
27486 --dropbox-client-secret
27487 OAuth Client Secret.
27488
27489 Leave blank normally.
27490
27491 Properties:
27492
27493 • Config: client_secret
27494
27495 • Env Var: RCLONE_DROPBOX_CLIENT_SECRET
27496
27497 • Type: string
27498
27499 • Required: false
27500
27501 Advanced options
27502 Here are the Advanced options specific to dropbox (Dropbox).
27503
27504 --dropbox-token
27505 OAuth Access Token as a JSON blob.
27506
27507 Properties:
27508
27509 • Config: token
27510
27511 • Env Var: RCLONE_DROPBOX_TOKEN
27512
27513 • Type: string
27514
27515 • Required: false
27516
27517 --dropbox-auth-url
27518 Auth server URL.
27519
27520 Leave blank to use the provider defaults.
27521
27522 Properties:
27523
27524 • Config: auth_url
27525
27526 • Env Var: RCLONE_DROPBOX_AUTH_URL
27527
27528 • Type: string
27529
27530 • Required: false
27531
27532 --dropbox-token-url
27533 Token server url.
27534
27535 Leave blank to use the provider defaults.
27536
27537 Properties:
27538
27539 • Config: token_url
27540
27541 • Env Var: RCLONE_DROPBOX_TOKEN_URL
27542
27543 • Type: string
27544
27545 • Required: false
27546
27547 --dropbox-chunk-size
27548 Upload chunk size (< 150Mi).
27549
27550 Any files larger than this will be uploaded in chunks of this size.
27551
27552 Note that chunks are buffered in memory (one at a time) so rclone can
27553 deal with retries. Setting this larger will increase the speed slight‐
27554 ly (at most 10% for 128 MiB in tests) at the cost of using more memory.
27555 It can be set smaller if you are tight on memory.
27556
27557 Properties:
27558
27559 • Config: chunk_size
27560
27561 • Env Var: RCLONE_DROPBOX_CHUNK_SIZE
27562
27563 • Type: SizeSuffix
27564
27565 • Default: 48Mi
27566
27567 --dropbox-impersonate
27568 Impersonate this user when using a business account.
27569
27570 Note that if you want to use impersonate, you should make sure this
27571 flag is set when running "rclone config" as this will cause rclone to
27572 request the "members.read" scope which it won't normally. This is
27573 needed to lookup a members email address into the internal ID that
27574 dropbox uses in the API.
27575
27576 Using the "members.read" scope will require a Dropbox Team Admin to ap‐
27577 prove during the OAuth flow.
27578
27579 You will have to use your own App (setting your own client_id and
27580 client_secret) to use this option as currently rclone's default set of
27581 permissions doesn't include "members.read". This can be added once
27582 v1.55 or later is in use everywhere.
27583
27584 Properties:
27585
27586 • Config: impersonate
27587
27588 • Env Var: RCLONE_DROPBOX_IMPERSONATE
27589
27590 • Type: string
27591
27592 • Required: false
27593
27594 --dropbox-shared-files
27595 Instructs rclone to work on individual shared files.
27596
27597 In this mode rclone's features are extremely limited - only list (ls,
27598 lsl, etc.) operations and read operations (e.g. downloading) are sup‐
27599 ported in this mode. All other operations will be disabled.
27600
27601 Properties:
27602
27603 • Config: shared_files
27604
27605 • Env Var: RCLONE_DROPBOX_SHARED_FILES
27606
27607 • Type: bool
27608
27609 • Default: false
27610
27611 --dropbox-shared-folders
27612 Instructs rclone to work on shared folders.
27613
27614 When this flag is used with no path only the List operation is support‐
27615 ed and all available shared folders will be listed. If you specify a
27616 path the first part will be interpreted as the name of shared folder.
27617 Rclone will then try to mount this shared to the root namespace. On
27618 success shared folder rclone proceeds normally. The shared folder is
27619 now pretty much a normal folder and all normal operations are support‐
27620 ed.
27621
27622 Note that we don't unmount the shared folder afterwards so the --drop‐
27623 box-shared-folders can be omitted after the first use of a particular
27624 shared folder.
27625
27626 Properties:
27627
27628 • Config: shared_folders
27629
27630 • Env Var: RCLONE_DROPBOX_SHARED_FOLDERS
27631
27632 • Type: bool
27633
27634 • Default: false
27635
27636 --dropbox-batch-mode
27637 Upload file batching sync|async|off.
27638
27639 This sets the batch mode used by rclone.
27640
27641 For full info see the main docs (https://rclone.org/dropbox/#batch-
27642 mode)
27643
27644 This has 3 possible values
27645
27646 • off - no batching
27647
27648 • sync - batch uploads and check completion (default)
27649
27650 • async - batch upload and don't check completion
27651
27652 Rclone will close any outstanding batches when it exits which may make
27653 a delay on quit.
27654
27655 Properties:
27656
27657 • Config: batch_mode
27658
27659 • Env Var: RCLONE_DROPBOX_BATCH_MODE
27660
27661 • Type: string
27662
27663 • Default: "sync"
27664
27665 --dropbox-batch-size
27666 Max number of files in upload batch.
27667
27668 This sets the batch size of files to upload. It has to be less than
27669 1000.
27670
27671 By default this is 0 which means rclone which calculate the batch size
27672 depending on the setting of batch_mode.
27673
27674 • batch_mode: async - default batch_size is 100
27675
27676 • batch_mode: sync - default batch_size is the same as --transfers
27677
27678 • batch_mode: off - not in use
27679
27680 Rclone will close any outstanding batches when it exits which may make
27681 a delay on quit.
27682
27683 Setting this is a great idea if you are uploading lots of small files
27684 as it will make them a lot quicker. You can use --transfers 32 to max‐
27685 imise throughput.
27686
27687 Properties:
27688
27689 • Config: batch_size
27690
27691 • Env Var: RCLONE_DROPBOX_BATCH_SIZE
27692
27693 • Type: int
27694
27695 • Default: 0
27696
27697 --dropbox-batch-timeout
27698 Max time to allow an idle upload batch before uploading.
27699
27700 If an upload batch is idle for more than this long then it will be up‐
27701 loaded.
27702
27703 The default for this is 0 which means rclone will choose a sensible de‐
27704 fault based on the batch_mode in use.
27705
27706 • batch_mode: async - default batch_timeout is 500ms
27707
27708 • batch_mode: sync - default batch_timeout is 10s
27709
27710 • batch_mode: off - not in use
27711
27712 Properties:
27713
27714 • Config: batch_timeout
27715
27716 • Env Var: RCLONE_DROPBOX_BATCH_TIMEOUT
27717
27718 • Type: Duration
27719
27720 • Default: 0s
27721
27722 --dropbox-batch-commit-timeout
27723 Max time to wait for a batch to finish committing
27724
27725 Properties:
27726
27727 • Config: batch_commit_timeout
27728
27729 • Env Var: RCLONE_DROPBOX_BATCH_COMMIT_TIMEOUT
27730
27731 • Type: Duration
27732
27733 • Default: 10m0s
27734
27735 --dropbox-encoding
27736 The encoding for the backend.
27737
27738 See the encoding section in the overview (https://rclone.org/over‐
27739 view/#encoding) for more info.
27740
27741 Properties:
27742
27743 • Config: encoding
27744
27745 • Env Var: RCLONE_DROPBOX_ENCODING
27746
27747 • Type: MultiEncoder
27748
27749 • Default: Slash,BackSlash,Del,RightSpace,InvalidUtf8,Dot
27750
27751 Limitations
27752 Note that Dropbox is case insensitive so you can't have a file called
27753 "Hello.doc" and one called "hello.doc".
27754
27755 There are some file names such as thumbs.db which Dropbox can't store.
27756 There is a full list of them in the "Ignored Files" section of this
27757 document (https://www.dropbox.com/en/help/145). Rclone will issue an
27758 error message File name disallowed - not uploading if it attempts to
27759 upload one of those file names, but the sync won't fail.
27760
27761 Some errors may occur if you try to sync copyright-protected files be‐
27762 cause Dropbox has its own copyright detector
27763 (https://techcrunch.com/2014/03/30/how-dropbox-knows-when-youre-shar‐
27764 ing-copyrighted-stuff-without-actually-looking-at-your-stuff/) that
27765 prevents this sort of file being downloaded. This will return the er‐
27766 ror ERROR : /path/to/your/file: Failed to copy: failed to open source
27767 object: path/restricted_content/.
27768
27769 If you have more than 10,000 files in a directory then rclone purge
27770 dropbox:dir will return the error Failed to purge: There are too many
27771 files involved in this operation. As a work-around do an rclone delete
27772 dropbox:dir followed by an rclone rmdir dropbox:dir.
27773
27774 When using rclone link you'll need to set --expire if using a non-per‐
27775 sonal account otherwise the visibility may not be correct. (Note that
27776 --expire isn't supported on personal accounts). See the forum discus‐
27777 sion (https://forum.rclone.org/t/rclone-link-dropbox-permissions/23211)
27778 and the dropbox SDK issue (https://github.com/dropbox/dropbox-sdk-go-
27779 unofficial/issues/75).
27780
27781 Get your own Dropbox App ID
27782 When you use rclone with Dropbox in its default configuration you are
27783 using rclone's App ID. This is shared between all the rclone users.
27784
27785 Here is how to create your own Dropbox App ID for rclone:
27786
27787 1. Log into the Dropbox App console (https://www.dropbox.com/develop‐
27788 ers/apps/create) with your Dropbox Account (It need not to be the
27789 same account as the Dropbox you want to access)
27790
27791 2. Choose an API => Usually this should be Dropbox API
27792
27793 3. Choose the type of access you want to use => Full Dropbox or App
27794 Folder
27795
27796 4. Name your App. The app name is global, so you can't use rclone for
27797 example
27798
27799 5. Click the button Create App
27800
27801 6. Switch to the Permissions tab. Enable at least the following per‐
27802 missions: account_info.read, files.metadata.write, files.con‐
27803 tent.write, files.content.read, sharing.write. The files.metada‐
27804 ta.read and sharing.read checkboxes will be marked too. Click Sub‐
27805 mit
27806
27807 7. Switch to the Settings tab. Fill OAuth2 - Redirect URIs as
27808 http://localhost:53682/
27809
27810 8. Find the App key and App secret values on the Settings tab. Use
27811 these values in rclone config to add a new remote or edit an exist‐
27812 ing remote. The App key setting corresponds to client_id in rclone
27813 config, the App secret corresponds to client_secret
27814
27816 This backend supports Storage Made Easy's Enterprise File Fabric™
27817 (https://storagemadeeasy.com/about/) which provides a software solution
27818 to integrate and unify File and Object Storage accessible through a
27819 global file system.
27820
27821 Configuration
27822 The initial setup for the Enterprise File Fabric backend involves get‐
27823 ting a token from the Enterprise File Fabric which you need to do in
27824 your browser. rclone config walks you through it.
27825
27826 Here is an example of how to make a remote called remote. First run:
27827
27828 rclone config
27829
27830 This will guide you through an interactive setup process:
27831
27832 No remotes found, make a new one?
27833 n) New remote
27834 s) Set configuration password
27835 q) Quit config
27836 n/s/q> n
27837 name> remote
27838 Type of storage to configure.
27839 Enter a string value. Press Enter for the default ("").
27840 Choose a number from below, or type in your own value
27841 [snip]
27842 XX / Enterprise File Fabric
27843 \ "filefabric"
27844 [snip]
27845 Storage> filefabric
27846 ** See help for filefabric backend at: https://rclone.org/filefabric/ **
27847
27848 URL of the Enterprise File Fabric to connect to
27849 Enter a string value. Press Enter for the default ("").
27850 Choose a number from below, or type in your own value
27851 1 / Storage Made Easy US
27852 \ "https://storagemadeeasy.com"
27853 2 / Storage Made Easy EU
27854 \ "https://eu.storagemadeeasy.com"
27855 3 / Connect to your Enterprise File Fabric
27856 \ "https://yourfabric.smestorage.com"
27857 url> https://yourfabric.smestorage.com/
27858 ID of the root folder
27859 Leave blank normally.
27860
27861 Fill in to make rclone start with directory of a given ID.
27862
27863 Enter a string value. Press Enter for the default ("").
27864 root_folder_id>
27865 Permanent Authentication Token
27866
27867 A Permanent Authentication Token can be created in the Enterprise File
27868 Fabric, on the users Dashboard under Security, there is an entry
27869 you'll see called "My Authentication Tokens". Click the Manage button
27870 to create one.
27871
27872 These tokens are normally valid for several years.
27873
27874 For more info see: https://docs.storagemadeeasy.com/organisationcloud/api-tokens
27875
27876 Enter a string value. Press Enter for the default ("").
27877 permanent_token> xxxxxxxxxxxxxxx-xxxxxxxxxxxxxxxx
27878 Edit advanced config? (y/n)
27879 y) Yes
27880 n) No (default)
27881 y/n> n
27882 Remote config
27883 --------------------
27884 [remote]
27885 type = filefabric
27886 url = https://yourfabric.smestorage.com/
27887 permanent_token = xxxxxxxxxxxxxxx-xxxxxxxxxxxxxxxx
27888 --------------------
27889 y) Yes this is OK (default)
27890 e) Edit this remote
27891 d) Delete this remote
27892 y/e/d> y
27893
27894 Once configured you can then use rclone like this,
27895
27896 List directories in top level of your Enterprise File Fabric
27897
27898 rclone lsd remote:
27899
27900 List all the files in your Enterprise File Fabric
27901
27902 rclone ls remote:
27903
27904 To copy a local directory to an Enterprise File Fabric directory called
27905 backup
27906
27907 rclone copy /home/source remote:backup
27908
27909 Modified time and hashes
27910 The Enterprise File Fabric allows modification times to be set on files
27911 accurate to 1 second. These will be used to detect whether objects
27912 need syncing or not.
27913
27914 The Enterprise File Fabric does not support any data hashes at this
27915 time.
27916
27917 Restricted filename characters
27918 The default restricted characters set (https://rclone.org/overview/#re‐
27919 stricted-characters) will be replaced.
27920
27921 Invalid UTF-8 bytes will also be replaced (https://rclone.org/over‐
27922 view/#invalid-utf8), as they can't be used in JSON strings.
27923
27924 Empty files
27925 Empty files aren't supported by the Enterprise File Fabric. Rclone
27926 will therefore upload an empty file as a single space with a mime type
27927 of application/vnd.rclone.empty.file and files with that mime type are
27928 treated as empty.
27929
27930 Root folder ID
27931 You can set the root_folder_id for rclone. This is the directory
27932 (identified by its Folder ID) that rclone considers to be the root of
27933 your Enterprise File Fabric.
27934
27935 Normally you will leave this blank and rclone will determine the cor‐
27936 rect root to use itself.
27937
27938 However you can set this to restrict rclone to a specific folder hier‐
27939 archy.
27940
27941 In order to do this you will have to find the Folder ID of the directo‐
27942 ry you wish rclone to display. These aren't displayed in the web in‐
27943 terface, but you can use rclone lsf to find them, for example
27944
27945 $ rclone lsf --dirs-only -Fip --csv filefabric:
27946 120673758,Burnt PDFs/
27947 120673759,My Quick Uploads/
27948 120673755,My Syncs/
27949 120673756,My backups/
27950 120673757,My contacts/
27951 120673761,S3 Storage/
27952
27953 The ID for "S3 Storage" would be 120673761.
27954
27955 Standard options
27956 Here are the Standard options specific to filefabric (Enterprise File
27957 Fabric).
27958
27959 --filefabric-url
27960 URL of the Enterprise File Fabric to connect to.
27961
27962 Properties:
27963
27964 • Config: url
27965
27966 • Env Var: RCLONE_FILEFABRIC_URL
27967
27968 • Type: string
27969
27970 • Required: true
27971
27972 • Examples:
27973
27974 • "https://storagemadeeasy.com"
27975
27976 • Storage Made Easy US
27977
27978 • "https://eu.storagemadeeasy.com"
27979
27980 • Storage Made Easy EU
27981
27982 • "https://yourfabric.smestorage.com"
27983
27984 • Connect to your Enterprise File Fabric
27985
27986 --filefabric-root-folder-id
27987 ID of the root folder.
27988
27989 Leave blank normally.
27990
27991 Fill in to make rclone start with directory of a given ID.
27992
27993 Properties:
27994
27995 • Config: root_folder_id
27996
27997 • Env Var: RCLONE_FILEFABRIC_ROOT_FOLDER_ID
27998
27999 • Type: string
28000
28001 • Required: false
28002
28003 --filefabric-permanent-token
28004 Permanent Authentication Token.
28005
28006 A Permanent Authentication Token can be created in the Enterprise File
28007 Fabric, on the users Dashboard under Security, there is an entry you'll
28008 see called "My Authentication Tokens". Click the Manage button to cre‐
28009 ate one.
28010
28011 These tokens are normally valid for several years.
28012
28013 For more info see: https://docs.storagemadeeasy.com/organisation‐
28014 cloud/api-tokens
28015
28016 Properties:
28017
28018 • Config: permanent_token
28019
28020 • Env Var: RCLONE_FILEFABRIC_PERMANENT_TOKEN
28021
28022 • Type: string
28023
28024 • Required: false
28025
28026 Advanced options
28027 Here are the Advanced options specific to filefabric (Enterprise File
28028 Fabric).
28029
28030 --filefabric-token
28031 Session Token.
28032
28033 This is a session token which rclone caches in the config file. It is
28034 usually valid for 1 hour.
28035
28036 Don't set this value - rclone will set it automatically.
28037
28038 Properties:
28039
28040 • Config: token
28041
28042 • Env Var: RCLONE_FILEFABRIC_TOKEN
28043
28044 • Type: string
28045
28046 • Required: false
28047
28048 --filefabric-token-expiry
28049 Token expiry time.
28050
28051 Don't set this value - rclone will set it automatically.
28052
28053 Properties:
28054
28055 • Config: token_expiry
28056
28057 • Env Var: RCLONE_FILEFABRIC_TOKEN_EXPIRY
28058
28059 • Type: string
28060
28061 • Required: false
28062
28063 --filefabric-version
28064 Version read from the file fabric.
28065
28066 Don't set this value - rclone will set it automatically.
28067
28068 Properties:
28069
28070 • Config: version
28071
28072 • Env Var: RCLONE_FILEFABRIC_VERSION
28073
28074 • Type: string
28075
28076 • Required: false
28077
28078 --filefabric-encoding
28079 The encoding for the backend.
28080
28081 See the encoding section in the overview (https://rclone.org/over‐
28082 view/#encoding) for more info.
28083
28084 Properties:
28085
28086 • Config: encoding
28087
28088 • Env Var: RCLONE_FILEFABRIC_ENCODING
28089
28090 • Type: MultiEncoder
28091
28092 • Default: Slash,Del,Ctl,InvalidUtf8,Dot
28093
28095 FTP is the File Transfer Protocol. Rclone FTP support is provided us‐
28096 ing the github.com/jlaffaye/ftp (https://godoc.org/github.com/jlaf‐
28097 faye/ftp) package.
28098
28099 Limitations of Rclone's FTP backend
28100
28101 Paths are specified as remote:path. If the path does not begin with a
28102 / it is relative to the home directory of the user. An empty path re‐
28103 mote: refers to the user's home directory.
28104
28105 Configuration
28106 To create an FTP configuration named remote, run
28107
28108 rclone config
28109
28110 Rclone config guides you through an interactive setup process. A mini‐
28111 mal rclone FTP remote definition only requires host, username and pass‐
28112 word. For an anonymous FTP server, see below.
28113
28114 No remotes found, make a new one?
28115 n) New remote
28116 r) Rename remote
28117 c) Copy remote
28118 s) Set configuration password
28119 q) Quit config
28120 n/r/c/s/q> n
28121 name> remote
28122 Type of storage to configure.
28123 Enter a string value. Press Enter for the default ("").
28124 Choose a number from below, or type in your own value
28125 [snip]
28126 XX / FTP
28127 \ "ftp"
28128 [snip]
28129 Storage> ftp
28130 ** See help for ftp backend at: https://rclone.org/ftp/ **
28131
28132 FTP host to connect to
28133 Enter a string value. Press Enter for the default ("").
28134 Choose a number from below, or type in your own value
28135 1 / Connect to ftp.example.com
28136 \ "ftp.example.com"
28137 host> ftp.example.com
28138 FTP username
28139 Enter a string value. Press Enter for the default ("$USER").
28140 user>
28141 FTP port number
28142 Enter a signed integer. Press Enter for the default (21).
28143 port>
28144 FTP password
28145 y) Yes type in my own password
28146 g) Generate random password
28147 y/g> y
28148 Enter the password:
28149 password:
28150 Confirm the password:
28151 password:
28152 Use FTP over TLS (Implicit)
28153 Enter a boolean value (true or false). Press Enter for the default ("false").
28154 tls>
28155 Use FTP over TLS (Explicit)
28156 Enter a boolean value (true or false). Press Enter for the default ("false").
28157 explicit_tls>
28158 Remote config
28159 --------------------
28160 [remote]
28161 type = ftp
28162 host = ftp.example.com
28163 pass = *** ENCRYPTED ***
28164 --------------------
28165 y) Yes this is OK
28166 e) Edit this remote
28167 d) Delete this remote
28168 y/e/d> y
28169
28170 To see all directories in the home directory of remote
28171
28172 rclone lsd remote:
28173
28174 Make a new directory
28175
28176 rclone mkdir remote:path/to/directory
28177
28178 List the contents of a directory
28179
28180 rclone ls remote:path/to/directory
28181
28182 Sync /home/local/directory to the remote directory, deleting any excess
28183 files in the directory.
28184
28185 rclone sync -i /home/local/directory remote:directory
28186
28187 Anonymous FTP
28188 When connecting to a FTP server that allows anonymous login, you can
28189 use the special "anonymous" username. Traditionally, this user account
28190 accepts any string as a password, although it is common to use either
28191 the password "anonymous" or "guest". Some servers require the use of a
28192 valid e-mail address as password.
28193
28194 Using on-the-fly or connection string (https://rclone.org/docs/#connec‐
28195 tion-strings) remotes makes it easy to access such servers, without re‐
28196 quiring any configuration in advance. The following are examples of
28197 that:
28198
28199 rclone lsf :ftp: --ftp-host=speedtest.tele2.net --ftp-user=anonymous --ftp-pass=$(rclone obscure dummy)
28200 rclone lsf :ftp,host=speedtest.tele2.net,user=anonymous,pass=$(rclone obscure dummy):
28201
28202 The above examples work in Linux shells and in PowerShell, but not Win‐
28203 dows Command Prompt. They execute the rclone obscure
28204 (https://rclone.org/commands/rclone_obscure/) command to create a pass‐
28205 word string in the format required by the pass option. The following
28206 examples are exactly the same, except use an already obscured string
28207 representation of the same password "dummy", and therefore works even
28208 in Windows Command Prompt:
28209
28210 rclone lsf :ftp: --ftp-host=speedtest.tele2.net --ftp-user=anonymous --ftp-pass=IXs2wc8OJOz7SYLBk47Ji1rHTmxM
28211 rclone lsf :ftp,host=speedtest.tele2.net,user=anonymous,pass=IXs2wc8OJOz7SYLBk47Ji1rHTmxM:
28212
28213 Implicit TLS
28214 Rlone FTP supports implicit FTP over TLS servers (FTPS). This has to
28215 be enabled in the FTP backend config for the remote, or with --ftp-tls.
28216 The default FTPS port is 990, not 21 and can be set with --ftp-port.
28217
28218 Restricted filename characters
28219 In addition to the default restricted characters set
28220 (https://rclone.org/overview/#restricted-characters) the following
28221 characters are also replaced:
28222
28223 File names cannot end with the following characters. Replacement is
28224 limited to the last character in a file name:
28225
28226 Character Value Replacement
28227 ────────────────────────────────
28228 SP 0x20 ␠
28229
28230 Not all FTP servers can have all characters in file names, for example:
28231
28232 FTP Server Forbidden characters
28233 ──────────────────────────────────
28234 proftpd *
28235 pureftpd \ [ ]
28236
28237 This backend's interactive configuration wizard provides a selection of
28238 sensible encoding settings for major FTP servers: ProFTPd, PureFTPd,
28239 VsFTPd. Just hit a selection number when prompted.
28240
28241 Standard options
28242 Here are the Standard options specific to ftp (FTP).
28243
28244 --ftp-host
28245 FTP host to connect to.
28246
28247 E.g. "ftp.example.com".
28248
28249 Properties:
28250
28251 • Config: host
28252
28253 • Env Var: RCLONE_FTP_HOST
28254
28255 • Type: string
28256
28257 • Required: true
28258
28259 --ftp-user
28260 FTP username.
28261
28262 Properties:
28263
28264 • Config: user
28265
28266 • Env Var: RCLONE_FTP_USER
28267
28268 • Type: string
28269
28270 • Default: "$USER"
28271
28272 --ftp-port
28273 FTP port number.
28274
28275 Properties:
28276
28277 • Config: port
28278
28279 • Env Var: RCLONE_FTP_PORT
28280
28281 • Type: int
28282
28283 • Default: 21
28284
28285 --ftp-pass
28286 FTP password.
28287
28288 NB Input to this must be obscured - see rclone obscure
28289 (https://rclone.org/commands/rclone_obscure/).
28290
28291 Properties:
28292
28293 • Config: pass
28294
28295 • Env Var: RCLONE_FTP_PASS
28296
28297 • Type: string
28298
28299 • Required: false
28300
28301 --ftp-tls
28302 Use Implicit FTPS (FTP over TLS).
28303
28304 When using implicit FTP over TLS the client connects using TLS right
28305 from the start which breaks compatibility with non-TLS-aware servers.
28306 This is usually served over port 990 rather than port 21. Cannot be
28307 used in combination with explicit FTP.
28308
28309 Properties:
28310
28311 • Config: tls
28312
28313 • Env Var: RCLONE_FTP_TLS
28314
28315 • Type: bool
28316
28317 • Default: false
28318
28319 --ftp-explicit-tls
28320 Use Explicit FTPS (FTP over TLS).
28321
28322 When using explicit FTP over TLS the client explicitly requests securi‐
28323 ty from the server in order to upgrade a plain text connection to an
28324 encrypted one. Cannot be used in combination with implicit FTP.
28325
28326 Properties:
28327
28328 • Config: explicit_tls
28329
28330 • Env Var: RCLONE_FTP_EXPLICIT_TLS
28331
28332 • Type: bool
28333
28334 • Default: false
28335
28336 Advanced options
28337 Here are the Advanced options specific to ftp (FTP).
28338
28339 --ftp-concurrency
28340 Maximum number of FTP simultaneous connections, 0 for unlimited.
28341
28342 Note that setting this is very likely to cause deadlocks so it should
28343 be used with care.
28344
28345 If you are doing a sync or copy then make sure concurrency is one more
28346 than the sum of --transfers and --checkers.
28347
28348 If you use --check-first then it just needs to be one more than the
28349 maximum of --checkers and --transfers.
28350
28351 So for concurrency 3 you'd use --checkers 2 --transfers 2 --check-first
28352 or --checkers 1 --transfers 1.
28353
28354 Properties:
28355
28356 • Config: concurrency
28357
28358 • Env Var: RCLONE_FTP_CONCURRENCY
28359
28360 • Type: int
28361
28362 • Default: 0
28363
28364 --ftp-no-check-certificate
28365 Do not verify the TLS certificate of the server.
28366
28367 Properties:
28368
28369 • Config: no_check_certificate
28370
28371 • Env Var: RCLONE_FTP_NO_CHECK_CERTIFICATE
28372
28373 • Type: bool
28374
28375 • Default: false
28376
28377 --ftp-disable-epsv
28378 Disable using EPSV even if server advertises support.
28379
28380 Properties:
28381
28382 • Config: disable_epsv
28383
28384 • Env Var: RCLONE_FTP_DISABLE_EPSV
28385
28386 • Type: bool
28387
28388 • Default: false
28389
28390 --ftp-disable-mlsd
28391 Disable using MLSD even if server advertises support.
28392
28393 Properties:
28394
28395 • Config: disable_mlsd
28396
28397 • Env Var: RCLONE_FTP_DISABLE_MLSD
28398
28399 • Type: bool
28400
28401 • Default: false
28402
28403 --ftp-disable-utf8
28404 Disable using UTF-8 even if server advertises support.
28405
28406 Properties:
28407
28408 • Config: disable_utf8
28409
28410 • Env Var: RCLONE_FTP_DISABLE_UTF8
28411
28412 • Type: bool
28413
28414 • Default: false
28415
28416 --ftp-writing-mdtm
28417 Use MDTM to set modification time (VsFtpd quirk)
28418
28419 Properties:
28420
28421 • Config: writing_mdtm
28422
28423 • Env Var: RCLONE_FTP_WRITING_MDTM
28424
28425 • Type: bool
28426
28427 • Default: false
28428
28429 --ftp-force-list-hidden
28430 Use LIST -a to force listing of hidden files and folders. This will
28431 disable the use of MLSD.
28432
28433 Properties:
28434
28435 • Config: force_list_hidden
28436
28437 • Env Var: RCLONE_FTP_FORCE_LIST_HIDDEN
28438
28439 • Type: bool
28440
28441 • Default: false
28442
28443 --ftp-idle-timeout
28444 Max time before closing idle connections.
28445
28446 If no connections have been returned to the connection pool in the time
28447 given, rclone will empty the connection pool.
28448
28449 Set to 0 to keep connections indefinitely.
28450
28451 Properties:
28452
28453 • Config: idle_timeout
28454
28455 • Env Var: RCLONE_FTP_IDLE_TIMEOUT
28456
28457 • Type: Duration
28458
28459 • Default: 1m0s
28460
28461 --ftp-close-timeout
28462 Maximum time to wait for a response to close.
28463
28464 Properties:
28465
28466 • Config: close_timeout
28467
28468 • Env Var: RCLONE_FTP_CLOSE_TIMEOUT
28469
28470 • Type: Duration
28471
28472 • Default: 1m0s
28473
28474 --ftp-tls-cache-size
28475 Size of TLS session cache for all control and data connections.
28476
28477 TLS cache allows to resume TLS sessions and reuse PSK between connec‐
28478 tions. Increase if default size is not enough resulting in TLS resump‐
28479 tion errors. Enabled by default. Use 0 to disable.
28480
28481 Properties:
28482
28483 • Config: tls_cache_size
28484
28485 • Env Var: RCLONE_FTP_TLS_CACHE_SIZE
28486
28487 • Type: int
28488
28489 • Default: 32
28490
28491 --ftp-disable-tls13
28492 Disable TLS 1.3 (workaround for FTP servers with buggy TLS)
28493
28494 Properties:
28495
28496 • Config: disable_tls13
28497
28498 • Env Var: RCLONE_FTP_DISABLE_TLS13
28499
28500 • Type: bool
28501
28502 • Default: false
28503
28504 --ftp-shut-timeout
28505 Maximum time to wait for data connection closing status.
28506
28507 Properties:
28508
28509 • Config: shut_timeout
28510
28511 • Env Var: RCLONE_FTP_SHUT_TIMEOUT
28512
28513 • Type: Duration
28514
28515 • Default: 1m0s
28516
28517 --ftp-ask-password
28518 Allow asking for FTP password when needed.
28519
28520 If this is set and no password is supplied then rclone will ask for a
28521 password
28522
28523 Properties:
28524
28525 • Config: ask_password
28526
28527 • Env Var: RCLONE_FTP_ASK_PASSWORD
28528
28529 • Type: bool
28530
28531 • Default: false
28532
28533 --ftp-encoding
28534 The encoding for the backend.
28535
28536 See the encoding section in the overview (https://rclone.org/over‐
28537 view/#encoding) for more info.
28538
28539 Properties:
28540
28541 • Config: encoding
28542
28543 • Env Var: RCLONE_FTP_ENCODING
28544
28545 • Type: MultiEncoder
28546
28547 • Default: Slash,Del,Ctl,RightSpace,Dot
28548
28549 • Examples:
28550
28551 • "Asterisk,Ctl,Dot,Slash"
28552
28553 • ProFTPd can't handle '*' in file names
28554
28555 • "BackSlash,Ctl,Del,Dot,RightSpace,Slash,SquareBracket"
28556
28557 • PureFTPd can't handle '[]' or '*' in file names
28558
28559 • "Ctl,LeftPeriod,Slash"
28560
28561 • VsFTPd can't handle file names starting with dot
28562
28563 Limitations
28564 FTP servers acting as rclone remotes must support passive mode. The
28565 mode cannot be configured as passive is the only supported one.
28566 Rclone's FTP implementation is not compatible with active mode as the
28567 library it uses doesn't support it (https://github.com/jlaffaye/ftp/is‐
28568 sues/29). This will likely never be supported due to security con‐
28569 cerns.
28570
28571 Rclone's FTP backend does not support any checksums but can compare
28572 file sizes.
28573
28574 rclone about is not supported by the FTP backend. Backends without
28575 this capability cannot determine free space for an rclone mount or use
28576 policy mfs (most free space) as a member of an rclone union remote.
28577
28578 See List of backends that do not support rclone about
28579 (https://rclone.org/overview/#optional-features) and rclone about
28580 (https://rclone.org/commands/rclone_about/)
28581
28582 The implementation of : --dump headers, --dump bodies, --dump auth for
28583 debugging isn't the same as for rclone HTTP based backends - it has
28584 less fine grained control.
28585
28586 --timeout isn't supported (but --contimeout is).
28587
28588 --bind isn't supported.
28589
28590 Rclone's FTP backend could support server-side move but does not at
28591 present.
28592
28593 The ftp_proxy environment variable is not currently supported.
28594
28595 Modified time
28596 File modification time (timestamps) is supported to 1 second resolution
28597 for major FTP servers: ProFTPd, PureFTPd, VsFTPd, and FileZilla FTP
28598 server. The VsFTPd server has non-standard implementation of time re‐
28599 lated protocol commands and needs a special configuration setting:
28600 writing_mdtm = true.
28601
28602 Support for precise file time with other FTP servers varies depending
28603 on what protocol extensions they advertise. If all the MLSD, MDTM and
28604 MFTM extensions are present, rclone will use them together to provide
28605 precise time. Otherwise the times you see on the FTP server through
28606 rclone are those of the last file upload.
28607
28608 You can use the following command to check whether rclone can use pre‐
28609 cise time with your FTP server: rclone backend features your_ftp_re‐
28610 mote: (the trailing colon is important). Look for the number in the
28611 line tagged by Precision designating the remote time precision ex‐
28612 pressed as nanoseconds. A value of 1000000000 means that file time
28613 precision of 1 second is available. A value of 3153600000000000000 (or
28614 another large number) means "unsupported".
28615
28617 Paths are specified as remote:bucket (or remote: for the lsd command.)
28618 You may put subdirectories in too, e.g. remote:bucket/path/to/dir.
28619
28620 Configuration
28621 The initial setup for google cloud storage involves getting a token
28622 from Google Cloud Storage which you need to do in your browser. rclone
28623 config walks you through it.
28624
28625 Here is an example of how to make a remote called remote. First run:
28626
28627 rclone config
28628
28629 This will guide you through an interactive setup process:
28630
28631 n) New remote
28632 d) Delete remote
28633 q) Quit config
28634 e/n/d/q> n
28635 name> remote
28636 Type of storage to configure.
28637 Choose a number from below, or type in your own value
28638 [snip]
28639 XX / Google Cloud Storage (this is not Google Drive)
28640 \ "google cloud storage"
28641 [snip]
28642 Storage> google cloud storage
28643 Google Application Client Id - leave blank normally.
28644 client_id>
28645 Google Application Client Secret - leave blank normally.
28646 client_secret>
28647 Project number optional - needed only for list/create/delete buckets - see your developer console.
28648 project_number> 12345678
28649 Service Account Credentials JSON file path - needed only if you want use SA instead of interactive login.
28650 service_account_file>
28651 Access Control List for new objects.
28652 Choose a number from below, or type in your own value
28653 1 / Object owner gets OWNER access, and all Authenticated Users get READER access.
28654 \ "authenticatedRead"
28655 2 / Object owner gets OWNER access, and project team owners get OWNER access.
28656 \ "bucketOwnerFullControl"
28657 3 / Object owner gets OWNER access, and project team owners get READER access.
28658 \ "bucketOwnerRead"
28659 4 / Object owner gets OWNER access [default if left blank].
28660 \ "private"
28661 5 / Object owner gets OWNER access, and project team members get access according to their roles.
28662 \ "projectPrivate"
28663 6 / Object owner gets OWNER access, and all Users get READER access.
28664 \ "publicRead"
28665 object_acl> 4
28666 Access Control List for new buckets.
28667 Choose a number from below, or type in your own value
28668 1 / Project team owners get OWNER access, and all Authenticated Users get READER access.
28669 \ "authenticatedRead"
28670 2 / Project team owners get OWNER access [default if left blank].
28671 \ "private"
28672 3 / Project team members get access according to their roles.
28673 \ "projectPrivate"
28674 4 / Project team owners get OWNER access, and all Users get READER access.
28675 \ "publicRead"
28676 5 / Project team owners get OWNER access, and all Users get WRITER access.
28677 \ "publicReadWrite"
28678 bucket_acl> 2
28679 Location for the newly created buckets.
28680 Choose a number from below, or type in your own value
28681 1 / Empty for default location (US).
28682 \ ""
28683 2 / Multi-regional location for Asia.
28684 \ "asia"
28685 3 / Multi-regional location for Europe.
28686 \ "eu"
28687 4 / Multi-regional location for United States.
28688 \ "us"
28689 5 / Taiwan.
28690 \ "asia-east1"
28691 6 / Tokyo.
28692 \ "asia-northeast1"
28693 7 / Singapore.
28694 \ "asia-southeast1"
28695 8 / Sydney.
28696 \ "australia-southeast1"
28697 9 / Belgium.
28698 \ "europe-west1"
28699 10 / London.
28700 \ "europe-west2"
28701 11 / Iowa.
28702 \ "us-central1"
28703 12 / South Carolina.
28704 \ "us-east1"
28705 13 / Northern Virginia.
28706 \ "us-east4"
28707 14 / Oregon.
28708 \ "us-west1"
28709 location> 12
28710 The storage class to use when storing objects in Google Cloud Storage.
28711 Choose a number from below, or type in your own value
28712 1 / Default
28713 \ ""
28714 2 / Multi-regional storage class
28715 \ "MULTI_REGIONAL"
28716 3 / Regional storage class
28717 \ "REGIONAL"
28718 4 / Nearline storage class
28719 \ "NEARLINE"
28720 5 / Coldline storage class
28721 \ "COLDLINE"
28722 6 / Durable reduced availability storage class
28723 \ "DURABLE_REDUCED_AVAILABILITY"
28724 storage_class> 5
28725 Remote config
28726 Use auto config?
28727 * Say Y if not sure
28728 * Say N if you are working on a remote or headless machine or Y didn't work
28729 y) Yes
28730 n) No
28731 y/n> y
28732 If your browser doesn't open automatically go to the following link: http://127.0.0.1:53682/auth
28733 Log in and authorize rclone for access
28734 Waiting for code...
28735 Got code
28736 --------------------
28737 [remote]
28738 type = google cloud storage
28739 client_id =
28740 client_secret =
28741 token = {"AccessToken":"xxxx.xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx","RefreshToken":"x/xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx_xxxxxxxxx","Expiry":"2014-07-17T20:49:14.929208288+01:00","Extra":null}
28742 project_number = 12345678
28743 object_acl = private
28744 bucket_acl = private
28745 --------------------
28746 y) Yes this is OK
28747 e) Edit this remote
28748 d) Delete this remote
28749 y/e/d> y
28750
28751 Note that rclone runs a webserver on your local machine to collect the
28752 token as returned from Google if you use auto config mode. This only
28753 runs from the moment it opens your browser to the moment you get back
28754 the verification code. This is on http://127.0.0.1:53682/ and this it
28755 may require you to unblock it temporarily if you are running a host
28756 firewall, or use manual mode.
28757
28758 This remote is called remote and can now be used like this
28759
28760 See all the buckets in your project
28761
28762 rclone lsd remote:
28763
28764 Make a new bucket
28765
28766 rclone mkdir remote:bucket
28767
28768 List the contents of a bucket
28769
28770 rclone ls remote:bucket
28771
28772 Sync /home/local/directory to the remote bucket, deleting any excess
28773 files in the bucket.
28774
28775 rclone sync -i /home/local/directory remote:bucket
28776
28777 Service Account support
28778 You can set up rclone with Google Cloud Storage in an unattended mode,
28779 i.e. not tied to a specific end-user Google account. This is useful
28780 when you want to synchronise files onto machines that don't have ac‐
28781 tively logged-in users, for example build machines.
28782
28783 To get credentials for Google Cloud Platform IAM Service Accounts
28784 (https://cloud.google.com/iam/docs/service-accounts), please head to
28785 the Service Account (https://console.cloud.google.com/permissions/ser‐
28786 viceaccounts) section of the Google Developer Console. Service Ac‐
28787 counts behave just like normal User permissions in Google Cloud Storage
28788 ACLs (https://cloud.google.com/storage/docs/access-control), so you can
28789 limit their access (e.g. make them read only). After creating an ac‐
28790 count, a JSON file containing the Service Account's credentials will be
28791 downloaded onto your machines. These credentials are what rclone will
28792 use for authentication.
28793
28794 To use a Service Account instead of OAuth2 token flow, enter the path
28795 to your Service Account credentials at the service_account_file prompt
28796 and rclone won't use the browser based authentication flow. If you'd
28797 rather stuff the contents of the credentials file into the rclone con‐
28798 fig file, you can set service_account_credentials with the actual con‐
28799 tents of the file instead, or set the equivalent environment variable.
28800
28801 Anonymous Access
28802 For downloads of objects that permit public access you can configure
28803 rclone to use anonymous access by setting anonymous to true. With
28804 unauthorized access you can't write or create files but only read or
28805 list those buckets and objects that have public read access.
28806
28807 Application Default Credentials
28808 If no other source of credentials is provided, rclone will fall back to
28809 Application Default Credentials (https://cloud.google.com/video-intel‐
28810 ligence/docs/common/auth#authenticating_with_application_default_cre‐
28811 dentials) this is useful both when you already have configured authen‐
28812 tication for your developer account, or in production when running on a
28813 google compute host. Note that if running in docker, you may need to
28814 run additional commands on your google compute machine - see this page
28815 (https://cloud.google.com/container-registry/docs/advanced-authentica‐
28816 tion#gcloud_as_a_docker_credential_helper).
28817
28818 Note that in the case application default credentials are used, there
28819 is no need to explicitly configure a project number.
28820
28821 --fast-list
28822 This remote supports --fast-list which allows you to use fewer transac‐
28823 tions in exchange for more memory. See the rclone docs
28824 (https://rclone.org/docs/#fast-list) for more details.
28825
28826 Custom upload headers
28827 You can set custom upload headers with the --header-upload flag.
28828 Google Cloud Storage supports the headers as described in the working
28829 with metadata documentation (https://cloud.google.com/storage/docs/gsu‐
28830 til/addlhelp/WorkingWithObjectMetadata)
28831
28832 • Cache-Control
28833
28834 • Content-Disposition
28835
28836 • Content-Encoding
28837
28838 • Content-Language
28839
28840 • Content-Type
28841
28842 • X-Goog-Storage-Class
28843
28844 • X-Goog-Meta-
28845
28846 Eg --header-upload "Content-Type text/potato"
28847
28848 Note that the last of these is for setting custom metadata in the form
28849 --header-upload "x-goog-meta-key: value"
28850
28851 Modification time
28852 Google Cloud Storage stores md5sum natively. Google's gsutil
28853 (https://cloud.google.com/storage/docs/gsutil) tool stores modification
28854 time with one-second precision as goog-reserved-file-mtime in file
28855 metadata.
28856
28857 To ensure compatibility with gsutil, rclone stores modification time in
28858 2 separate metadata entries. mtime uses RFC3339 format with one-
28859 nanosecond precision. goog-reserved-file-mtime uses Unix timestamp
28860 format with one-second precision. To get modification time from object
28861 metadata, rclone reads the metadata in the following order: mtime,
28862 goog-reserved-file-mtime, object updated time.
28863
28864 Note that rclone's default modify window is 1ns. Files uploaded by
28865 gsutil only contain timestamps with one-second precision. If you use
28866 rclone to sync files previously uploaded by gsutil, rclone will attempt
28867 to update modification time for all these files. To avoid these possi‐
28868 bly unnecessary updates, use --modify-window 1s.
28869
28870 Restricted filename characters
28871 Character Value Replacement
28872 ────────────────────────────────
28873 NUL 0x00 ␀
28874 LF 0x0A ␊
28875 CR 0x0D ␍
28876 / 0x2F /
28877
28878 Invalid UTF-8 bytes will also be replaced (https://rclone.org/over‐
28879 view/#invalid-utf8), as they can't be used in JSON strings.
28880
28881 Standard options
28882 Here are the Standard options specific to google cloud storage (Google
28883 Cloud Storage (this is not Google Drive)).
28884
28885 --gcs-client-id
28886 OAuth Client Id.
28887
28888 Leave blank normally.
28889
28890 Properties:
28891
28892 • Config: client_id
28893
28894 • Env Var: RCLONE_GCS_CLIENT_ID
28895
28896 • Type: string
28897
28898 • Required: false
28899
28900 --gcs-client-secret
28901 OAuth Client Secret.
28902
28903 Leave blank normally.
28904
28905 Properties:
28906
28907 • Config: client_secret
28908
28909 • Env Var: RCLONE_GCS_CLIENT_SECRET
28910
28911 • Type: string
28912
28913 • Required: false
28914
28915 --gcs-project-number
28916 Project number.
28917
28918 Optional - needed only for list/create/delete buckets - see your devel‐
28919 oper console.
28920
28921 Properties:
28922
28923 • Config: project_number
28924
28925 • Env Var: RCLONE_GCS_PROJECT_NUMBER
28926
28927 • Type: string
28928
28929 • Required: false
28930
28931 --gcs-service-account-file
28932 Service Account Credentials JSON file path.
28933
28934 Leave blank normally. Needed only if you want use SA instead of inter‐
28935 active login.
28936
28937 Leading ~ will be expanded in the file name as will environment vari‐
28938 ables such as ${RCLONE_CONFIG_DIR}.
28939
28940 Properties:
28941
28942 • Config: service_account_file
28943
28944 • Env Var: RCLONE_GCS_SERVICE_ACCOUNT_FILE
28945
28946 • Type: string
28947
28948 • Required: false
28949
28950 --gcs-service-account-credentials
28951 Service Account Credentials JSON blob.
28952
28953 Leave blank normally. Needed only if you want use SA instead of inter‐
28954 active login.
28955
28956 Properties:
28957
28958 • Config: service_account_credentials
28959
28960 • Env Var: RCLONE_GCS_SERVICE_ACCOUNT_CREDENTIALS
28961
28962 • Type: string
28963
28964 • Required: false
28965
28966 --gcs-anonymous
28967 Access public buckets and objects without credentials.
28968
28969 Set to 'true' if you just want to download files and don't configure
28970 credentials.
28971
28972 Properties:
28973
28974 • Config: anonymous
28975
28976 • Env Var: RCLONE_GCS_ANONYMOUS
28977
28978 • Type: bool
28979
28980 • Default: false
28981
28982 --gcs-object-acl
28983 Access Control List for new objects.
28984
28985 Properties:
28986
28987 • Config: object_acl
28988
28989 • Env Var: RCLONE_GCS_OBJECT_ACL
28990
28991 • Type: string
28992
28993 • Required: false
28994
28995 • Examples:
28996
28997 • "authenticatedRead"
28998
28999 • Object owner gets OWNER access.
29000
29001 • All Authenticated Users get READER access.
29002
29003 • "bucketOwnerFullControl"
29004
29005 • Object owner gets OWNER access.
29006
29007 • Project team owners get OWNER access.
29008
29009 • "bucketOwnerRead"
29010
29011 • Object owner gets OWNER access.
29012
29013 • Project team owners get READER access.
29014
29015 • "private"
29016
29017 • Object owner gets OWNER access.
29018
29019 • Default if left blank.
29020
29021 • "projectPrivate"
29022
29023 • Object owner gets OWNER access.
29024
29025 • Project team members get access according to their roles.
29026
29027 • "publicRead"
29028
29029 • Object owner gets OWNER access.
29030
29031 • All Users get READER access.
29032
29033 --gcs-bucket-acl
29034 Access Control List for new buckets.
29035
29036 Properties:
29037
29038 • Config: bucket_acl
29039
29040 • Env Var: RCLONE_GCS_BUCKET_ACL
29041
29042 • Type: string
29043
29044 • Required: false
29045
29046 • Examples:
29047
29048 • "authenticatedRead"
29049
29050 • Project team owners get OWNER access.
29051
29052 • All Authenticated Users get READER access.
29053
29054 • "private"
29055
29056 • Project team owners get OWNER access.
29057
29058 • Default if left blank.
29059
29060 • "projectPrivate"
29061
29062 • Project team members get access according to their roles.
29063
29064 • "publicRead"
29065
29066 • Project team owners get OWNER access.
29067
29068 • All Users get READER access.
29069
29070 • "publicReadWrite"
29071
29072 • Project team owners get OWNER access.
29073
29074 • All Users get WRITER access.
29075
29076 --gcs-bucket-policy-only
29077 Access checks should use bucket-level IAM policies.
29078
29079 If you want to upload objects to a bucket with Bucket Policy Only set
29080 then you will need to set this.
29081
29082 When it is set, rclone:
29083
29084 • ignores ACLs set on buckets
29085
29086 • ignores ACLs set on objects
29087
29088 • creates buckets with Bucket Policy Only set
29089
29090 Docs: https://cloud.google.com/storage/docs/bucket-policy-only
29091
29092 Properties:
29093
29094 • Config: bucket_policy_only
29095
29096 • Env Var: RCLONE_GCS_BUCKET_POLICY_ONLY
29097
29098 • Type: bool
29099
29100 • Default: false
29101
29102 --gcs-location
29103 Location for the newly created buckets.
29104
29105 Properties:
29106
29107 • Config: location
29108
29109 • Env Var: RCLONE_GCS_LOCATION
29110
29111 • Type: string
29112
29113 • Required: false
29114
29115 • Examples:
29116
29117 • ""
29118
29119 • Empty for default location (US)
29120
29121 • "asia"
29122
29123 • Multi-regional location for Asia
29124
29125 • "eu"
29126
29127 • Multi-regional location for Europe
29128
29129 • "us"
29130
29131 • Multi-regional location for United States
29132
29133 • "asia-east1"
29134
29135 • Taiwan
29136
29137 • "asia-east2"
29138
29139 • Hong Kong
29140
29141 • "asia-northeast1"
29142
29143 • Tokyo
29144
29145 • "asia-northeast2"
29146
29147 • Osaka
29148
29149 • "asia-northeast3"
29150
29151 • Seoul
29152
29153 • "asia-south1"
29154
29155 • Mumbai
29156
29157 • "asia-south2"
29158
29159 • Delhi
29160
29161 • "asia-southeast1"
29162
29163 • Singapore
29164
29165 • "asia-southeast2"
29166
29167 • Jakarta
29168
29169 • "australia-southeast1"
29170
29171 • Sydney
29172
29173 • "australia-southeast2"
29174
29175 • Melbourne
29176
29177 • "europe-north1"
29178
29179 • Finland
29180
29181 • "europe-west1"
29182
29183 • Belgium
29184
29185 • "europe-west2"
29186
29187 • London
29188
29189 • "europe-west3"
29190
29191 • Frankfurt
29192
29193 • "europe-west4"
29194
29195 • Netherlands
29196
29197 • "europe-west6"
29198
29199 • Zürich
29200
29201 • "europe-central2"
29202
29203 • Warsaw
29204
29205 • "us-central1"
29206
29207 • Iowa
29208
29209 • "us-east1"
29210
29211 • South Carolina
29212
29213 • "us-east4"
29214
29215 • Northern Virginia
29216
29217 • "us-west1"
29218
29219 • Oregon
29220
29221 • "us-west2"
29222
29223 • California
29224
29225 • "us-west3"
29226
29227 • Salt Lake City
29228
29229 • "us-west4"
29230
29231 • Las Vegas
29232
29233 • "northamerica-northeast1"
29234
29235 • Montréal
29236
29237 • "northamerica-northeast2"
29238
29239 • Toronto
29240
29241 • "southamerica-east1"
29242
29243 • São Paulo
29244
29245 • "southamerica-west1"
29246
29247 • Santiago
29248
29249 • "asia1"
29250
29251 • Dual region: asia-northeast1 and asia-northeast2.
29252
29253 • "eur4"
29254
29255 • Dual region: europe-north1 and europe-west4.
29256
29257 • "nam4"
29258
29259 • Dual region: us-central1 and us-east1.
29260
29261 --gcs-storage-class
29262 The storage class to use when storing objects in Google Cloud Storage.
29263
29264 Properties:
29265
29266 • Config: storage_class
29267
29268 • Env Var: RCLONE_GCS_STORAGE_CLASS
29269
29270 • Type: string
29271
29272 • Required: false
29273
29274 • Examples:
29275
29276 • ""
29277
29278 • Default
29279
29280 • "MULTI_REGIONAL"
29281
29282 • Multi-regional storage class
29283
29284 • "REGIONAL"
29285
29286 • Regional storage class
29287
29288 • "NEARLINE"
29289
29290 • Nearline storage class
29291
29292 • "COLDLINE"
29293
29294 • Coldline storage class
29295
29296 • "ARCHIVE"
29297
29298 • Archive storage class
29299
29300 • "DURABLE_REDUCED_AVAILABILITY"
29301
29302 • Durable reduced availability storage class
29303
29304 Advanced options
29305 Here are the Advanced options specific to google cloud storage (Google
29306 Cloud Storage (this is not Google Drive)).
29307
29308 --gcs-token
29309 OAuth Access Token as a JSON blob.
29310
29311 Properties:
29312
29313 • Config: token
29314
29315 • Env Var: RCLONE_GCS_TOKEN
29316
29317 • Type: string
29318
29319 • Required: false
29320
29321 --gcs-auth-url
29322 Auth server URL.
29323
29324 Leave blank to use the provider defaults.
29325
29326 Properties:
29327
29328 • Config: auth_url
29329
29330 • Env Var: RCLONE_GCS_AUTH_URL
29331
29332 • Type: string
29333
29334 • Required: false
29335
29336 --gcs-token-url
29337 Token server url.
29338
29339 Leave blank to use the provider defaults.
29340
29341 Properties:
29342
29343 • Config: token_url
29344
29345 • Env Var: RCLONE_GCS_TOKEN_URL
29346
29347 • Type: string
29348
29349 • Required: false
29350
29351 --gcs-no-check-bucket
29352 If set, don't attempt to check the bucket exists or create it.
29353
29354 This can be useful when trying to minimise the number of transactions
29355 rclone does if you know the bucket exists already.
29356
29357 Properties:
29358
29359 • Config: no_check_bucket
29360
29361 • Env Var: RCLONE_GCS_NO_CHECK_BUCKET
29362
29363 • Type: bool
29364
29365 • Default: false
29366
29367 --gcs-decompress
29368 If set this will decompress gzip encoded objects.
29369
29370 It is possible to upload objects to GCS with "Content-Encoding: gzip"
29371 set. Normally rclone will download these files as compressed objects.
29372
29373 If this flag is set then rclone will decompress these files with "Con‐
29374 tent-Encoding: gzip" as they are received. This means that rclone
29375 can't check the size and hash but the file contents will be decom‐
29376 pressed.
29377
29378 Properties:
29379
29380 • Config: decompress
29381
29382 • Env Var: RCLONE_GCS_DECOMPRESS
29383
29384 • Type: bool
29385
29386 • Default: false
29387
29388 --gcs-endpoint
29389 Endpoint for the service.
29390
29391 Leave blank normally.
29392
29393 Properties:
29394
29395 • Config: endpoint
29396
29397 • Env Var: RCLONE_GCS_ENDPOINT
29398
29399 • Type: string
29400
29401 • Required: false
29402
29403 --gcs-encoding
29404 The encoding for the backend.
29405
29406 See the encoding section in the overview (https://rclone.org/over‐
29407 view/#encoding) for more info.
29408
29409 Properties:
29410
29411 • Config: encoding
29412
29413 • Env Var: RCLONE_GCS_ENCODING
29414
29415 • Type: MultiEncoder
29416
29417 • Default: Slash,CrLf,InvalidUtf8,Dot
29418
29419 Limitations
29420 rclone about is not supported by the Google Cloud Storage backend.
29421 Backends without this capability cannot determine free space for an
29422 rclone mount or use policy mfs (most free space) as a member of an
29423 rclone union remote.
29424
29425 See List of backends that do not support rclone about
29426 (https://rclone.org/overview/#optional-features) and rclone about
29427 (https://rclone.org/commands/rclone_about/)
29428
29430 Paths are specified as drive:path
29431
29432 Drive paths may be as deep as required, e.g. drive:directory/subdirec‐
29433 tory.
29434
29435 Configuration
29436 The initial setup for drive involves getting a token from Google drive
29437 which you need to do in your browser. rclone config walks you through
29438 it.
29439
29440 Here is an example of how to make a remote called remote. First run:
29441
29442 rclone config
29443
29444 This will guide you through an interactive setup process:
29445
29446 No remotes found, make a new one?
29447 n) New remote
29448 r) Rename remote
29449 c) Copy remote
29450 s) Set configuration password
29451 q) Quit config
29452 n/r/c/s/q> n
29453 name> remote
29454 Type of storage to configure.
29455 Choose a number from below, or type in your own value
29456 [snip]
29457 XX / Google Drive
29458 \ "drive"
29459 [snip]
29460 Storage> drive
29461 Google Application Client Id - leave blank normally.
29462 client_id>
29463 Google Application Client Secret - leave blank normally.
29464 client_secret>
29465 Scope that rclone should use when requesting access from drive.
29466 Choose a number from below, or type in your own value
29467 1 / Full access all files, excluding Application Data Folder.
29468 \ "drive"
29469 2 / Read-only access to file metadata and file contents.
29470 \ "drive.readonly"
29471 / Access to files created by rclone only.
29472 3 | These are visible in the drive website.
29473 | File authorization is revoked when the user deauthorizes the app.
29474 \ "drive.file"
29475 / Allows read and write access to the Application Data folder.
29476 4 | This is not visible in the drive website.
29477 \ "drive.appfolder"
29478 / Allows read-only access to file metadata but
29479 5 | does not allow any access to read or download file content.
29480 \ "drive.metadata.readonly"
29481 scope> 1
29482 Service Account Credentials JSON file path - needed only if you want use SA instead of interactive login.
29483 service_account_file>
29484 Remote config
29485 Use auto config?
29486 * Say Y if not sure
29487 * Say N if you are working on a remote or headless machine or Y didn't work
29488 y) Yes
29489 n) No
29490 y/n> y
29491 If your browser doesn't open automatically go to the following link: http://127.0.0.1:53682/auth
29492 Log in and authorize rclone for access
29493 Waiting for code...
29494 Got code
29495 Configure this as a Shared Drive (Team Drive)?
29496 y) Yes
29497 n) No
29498 y/n> n
29499 --------------------
29500 [remote]
29501 client_id =
29502 client_secret =
29503 scope = drive
29504 root_folder_id =
29505 service_account_file =
29506 token = {"access_token":"XXX","token_type":"Bearer","refresh_token":"XXX","expiry":"2014-03-16T13:57:58.955387075Z"}
29507 --------------------
29508 y) Yes this is OK
29509 e) Edit this remote
29510 d) Delete this remote
29511 y/e/d> y
29512
29513 Note that rclone runs a webserver on your local machine to collect the
29514 token as returned from Google if you use auto config mode. This only
29515 runs from the moment it opens your browser to the moment you get back
29516 the verification code. This is on http://127.0.0.1:53682/ and it may
29517 require you to unblock it temporarily if you are running a host fire‐
29518 wall, or use manual mode.
29519
29520 You can then use it like this,
29521
29522 List directories in top level of your drive
29523
29524 rclone lsd remote:
29525
29526 List all the files in your drive
29527
29528 rclone ls remote:
29529
29530 To copy a local directory to a drive directory called backup
29531
29532 rclone copy /home/source remote:backup
29533
29534 Scopes
29535 Rclone allows you to select which scope you would like for rclone to
29536 use. This changes what type of token is granted to rclone. The scopes
29537 are defined here (https://developers.google.com/drive/v3/web/about-
29538 auth).
29539
29540 The scope are
29541
29542 drive
29543 This is the default scope and allows full access to all files, except
29544 for the Application Data Folder (see below).
29545
29546 Choose this one if you aren't sure.
29547
29548 drive.readonly
29549 This allows read only access to all files. Files may be listed and
29550 downloaded but not uploaded, renamed or deleted.
29551
29552 drive.file
29553 With this scope rclone can read/view/modify only those files and fold‐
29554 ers it creates.
29555
29556 So if you uploaded files to drive via the web interface (or any other
29557 means) they will not be visible to rclone.
29558
29559 This can be useful if you are using rclone to backup data and you want
29560 to be sure confidential data on your drive is not visible to rclone.
29561
29562 Files created with this scope are visible in the web interface.
29563
29564 drive.appfolder
29565 This gives rclone its own private area to store files. Rclone will not
29566 be able to see any other files on your drive and you won't be able to
29567 see rclone's files from the web interface either.
29568
29569 drive.metadata.readonly
29570 This allows read only access to file names only. It does not allow
29571 rclone to download or upload data, or rename or delete files or direc‐
29572 tories.
29573
29574 Root folder ID
29575 This option has been moved to the advanced section. You can set the
29576 root_folder_id for rclone. This is the directory (identified by its
29577 Folder ID) that rclone considers to be the root of your drive.
29578
29579 Normally you will leave this blank and rclone will determine the cor‐
29580 rect root to use itself.
29581
29582 However you can set this to restrict rclone to a specific folder hier‐
29583 archy or to access data within the "Computers" tab on the drive web in‐
29584 terface (where files from Google's Backup and Sync desktop program go).
29585
29586 In order to do this you will have to find the Folder ID of the directo‐
29587 ry you wish rclone to display. This will be the last segment of the
29588 URL when you open the relevant folder in the drive web interface.
29589
29590 So if the folder you want rclone to use has a URL which looks like
29591 https://drive.google.com/drive/fold‐
29592 ers/1XyfxxxxxxxxxxxxxxxxxxxxxxxxxKHCh in the browser, then you use
29593 1XyfxxxxxxxxxxxxxxxxxxxxxxxxxKHCh as the root_folder_id in the config.
29594
29595 NB folders under the "Computers" tab seem to be read only (drive gives
29596 a 500 error) when using rclone.
29597
29598 There doesn't appear to be an API to discover the folder IDs of the
29599 "Computers" tab - please contact us if you know otherwise!
29600
29601 Note also that rclone can't access any data under the "Backups" tab on
29602 the google drive web interface yet.
29603
29604 Service Account support
29605 You can set up rclone with Google Drive in an unattended mode, i.e.
29606 not tied to a specific end-user Google account. This is useful when
29607 you want to synchronise files onto machines that don't have actively
29608 logged-in users, for example build machines.
29609
29610 To use a Service Account instead of OAuth2 token flow, enter the path
29611 to your Service Account credentials at the service_account_file prompt
29612 during rclone config and rclone won't use the browser based authentica‐
29613 tion flow. If you'd rather stuff the contents of the credentials file
29614 into the rclone config file, you can set service_account_credentials
29615 with the actual contents of the file instead, or set the equivalent en‐
29616 vironment variable.
29617
29618 Use case - Google Apps/G-suite account and individual Drive
29619 Let's say that you are the administrator of a Google Apps (old) or G-
29620 suite account. The goal is to store data on an individual's Drive ac‐
29621 count, who IS a member of the domain. We'll call the domain exam‐
29622 ple.com, and the user foo@example.com.
29623
29624 There's a few steps we need to go through to accomplish this:
29625
29626 1. Create a service account for example.com
29627 • To create a service account and obtain its credentials, go to the
29628 Google Developer Console (https://console.developers.google.com).
29629
29630 • You must have a project - create one if you don't.
29631
29632 • Then go to "IAM & admin" -> "Service Accounts".
29633
29634 • Use the "Create Credentials" button. Fill in "Service account name"
29635 with something that identifies your client. "Role" can be empty.
29636
29637 • Tick "Furnish a new private key" - select "Key type JSON".
29638
29639 • Tick "Enable G Suite Domain-wide Delegation". This option makes "im‐
29640 personation" possible, as documented here: Delegating domain-wide au‐
29641 thority to the service account (https://developers.google.com/identi‐
29642 ty/protocols/OAuth2ServiceAccount#delegatingauthority)
29643
29644 • These credentials are what rclone will use for authentication. If
29645 you ever need to remove access, press the "Delete service account
29646 key" button.
29647
29648 2. Allowing API access to example.com Google Drive
29649 • Go to example.com's admin console
29650
29651 • Go into "Security" (or use the search bar)
29652
29653 • Select "Show more" and then "Advanced settings"
29654
29655 • Select "Manage API client access" in the "Authentication" section
29656
29657 • In the "Client Name" field enter the service account's "Client ID" -
29658 this can be found in the Developer Console under "IAM & Admin" ->
29659 "Service Accounts", then "View Client ID" for the newly created ser‐
29660 vice account. It is a ~21 character numerical string.
29661
29662 • In the next field, "One or More API Scopes", enter
29663 https://www.googleapis.com/auth/drive to grant access to Google Drive
29664 specifically.
29665
29666 3. Configure rclone, assuming a new install
29667 rclone config
29668
29669 n/s/q> n # New
29670 name>gdrive # Gdrive is an example name
29671 Storage> # Select the number shown for Google Drive
29672 client_id> # Can be left blank
29673 client_secret> # Can be left blank
29674 scope> # Select your scope, 1 for example
29675 root_folder_id> # Can be left blank
29676 service_account_file> /home/foo/myJSONfile.json # This is where the JSON file goes!
29677 y/n> # Auto config, n
29678
29679 4. Verify that it's working
29680 • rclone -v --drive-impersonate foo@example.com lsf gdrive:backup
29681
29682 • The arguments do:
29683
29684 • -v - verbose logging
29685
29686 • --drive-impersonate foo@example.com - this is what does the magic,
29687 pretending to be user foo.
29688
29689 • lsf - list files in a parsing friendly way
29690
29691 • gdrive:backup - use the remote called gdrive, work in the folder
29692 named backup.
29693
29694 Note: in case you configured a specific root folder on gdrive and
29695 rclone is unable to access the contents of that folder when using
29696 --drive-impersonate, do this instead: - in the gdrive web interface,
29697 share your root folder with the user/email of the new Service Account
29698 you created/selected at step #1 - use rclone without specifying the
29699 --drive-impersonate option, like this: rclone -v lsf gdrive:backup
29700
29701 Shared drives (team drives)
29702 If you want to configure the remote to point to a Google Shared Drive
29703 (previously known as Team Drives) then answer y to the question Config‐
29704 ure this as a Shared Drive (Team Drive)?.
29705
29706 This will fetch the list of Shared Drives from google and allow you to
29707 configure which one you want to use. You can also type in a Shared
29708 Drive ID if you prefer.
29709
29710 For example:
29711
29712 Configure this as a Shared Drive (Team Drive)?
29713 y) Yes
29714 n) No
29715 y/n> y
29716 Fetching Shared Drive list...
29717 Choose a number from below, or type in your own value
29718 1 / Rclone Test
29719 \ "xxxxxxxxxxxxxxxxxxxx"
29720 2 / Rclone Test 2
29721 \ "yyyyyyyyyyyyyyyyyyyy"
29722 3 / Rclone Test 3
29723 \ "zzzzzzzzzzzzzzzzzzzz"
29724 Enter a Shared Drive ID> 1
29725 --------------------
29726 [remote]
29727 client_id =
29728 client_secret =
29729 token = {"AccessToken":"xxxx.x.xxxxx_xxxxxxxxxxx_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx","RefreshToken":"1/xxxxxxxxxxxxxxxx_xxxxxxxxxxxxxxxxxxxxxxxxxx","Expiry":"2014-03-16T13:57:58.955387075Z","Extra":null}
29730 team_drive = xxxxxxxxxxxxxxxxxxxx
29731 --------------------
29732 y) Yes this is OK
29733 e) Edit this remote
29734 d) Delete this remote
29735 y/e/d> y
29736
29737 --fast-list
29738 This remote supports --fast-list which allows you to use fewer transac‐
29739 tions in exchange for more memory. See the rclone docs
29740 (https://rclone.org/docs/#fast-list) for more details.
29741
29742 It does this by combining multiple list calls into a single API re‐
29743 quest.
29744
29745 This works by combining many '%s' in parents filters into one expres‐
29746 sion. To list the contents of directories a, b and c, the following
29747 requests will be send by the regular List function:
29748
29749 trashed=false and 'a' in parents
29750 trashed=false and 'b' in parents
29751 trashed=false and 'c' in parents
29752
29753 These can now be combined into a single request:
29754
29755 trashed=false and ('a' in parents or 'b' in parents or 'c' in parents)
29756
29757 The implementation of ListR will put up to 50 parents filters into one
29758 request. It will use the --checkers value to specify the number of re‐
29759 quests to run in parallel.
29760
29761 In tests, these batch requests were up to 20x faster than the regular
29762 method. Running the following command against different sized folders
29763 gives:
29764
29765 rclone lsjson -vv -R --checkers=6 gdrive:folder
29766
29767 small folder (220 directories, 700 files):
29768
29769 • without --fast-list: 38s
29770
29771 • with --fast-list: 10s
29772
29773 large folder (10600 directories, 39000 files):
29774
29775 • without --fast-list: 22:05 min
29776
29777 • with --fast-list: 58s
29778
29779 Modified time
29780 Google drive stores modification times accurate to 1 ms.
29781
29782 Restricted filename characters
29783 Only Invalid UTF-8 bytes will be replaced (https://rclone.org/over‐
29784 view/#invalid-utf8), as they can't be used in JSON strings.
29785
29786 In contrast to other backends, / can also be used in names and . or ..
29787 are valid names.
29788
29789 Revisions
29790 Google drive stores revisions of files. When you upload a change to an
29791 existing file to google drive using rclone it will create a new revi‐
29792 sion of that file.
29793
29794 Revisions follow the standard google policy which at time of writing
29795 was
29796
29797 • They are deleted after 30 days or 100 revisions (whatever comes
29798 first).
29799
29800 • They do not count towards a user storage quota.
29801
29802 Deleting files
29803 By default rclone will send all files to the trash when deleting files.
29804 If deleting them permanently is required then use the --drive-use-
29805 trash=false flag, or set the equivalent environment variable.
29806
29807 Shortcuts
29808 In March 2020 Google introduced a new feature in Google Drive called
29809 drive shortcuts (https://support.google.com/drive/answer/9700156) (API
29810 (https://developers.google.com/drive/api/v3/shortcuts)). These will
29811 (by September 2020) replace the ability for files or folders to be in
29812 multiple folders at once (https://cloud.google.com/blog/products/g-
29813 suite/simplifying-google-drives-folder-structure-and-sharing-models).
29814
29815 Shortcuts are files that link to other files on Google Drive somewhat
29816 like a symlink in unix, except they point to the underlying file data
29817 (e.g. the inode in unix terms) so they don't break if the source is
29818 renamed or moved about.
29819
29820 Be default rclone treats these as follows.
29821
29822 For shortcuts pointing to files:
29823
29824 • When listing a file shortcut appears as the destination file.
29825
29826 • When downloading the contents of the destination file is downloaded.
29827
29828 • When updating shortcut file with a non shortcut file, the shortcut is
29829 removed then a new file is uploaded in place of the shortcut.
29830
29831 • When server-side moving (renaming) the shortcut is renamed, not the
29832 destination file.
29833
29834 • When server-side copying the shortcut is copied, not the contents of
29835 the shortcut. (unless --drive-copy-shortcut-content is in use in
29836 which case the contents of the shortcut gets copied).
29837
29838 • When deleting the shortcut is deleted not the linked file.
29839
29840 • When setting the modification time, the modification time of the
29841 linked file will be set.
29842
29843 For shortcuts pointing to folders:
29844
29845 • When listing the shortcut appears as a folder and that folder will
29846 contain the contents of the linked folder appear (including any sub
29847 folders)
29848
29849 • When downloading the contents of the linked folder and sub contents
29850 are downloaded
29851
29852 • When uploading to a shortcut folder the file will be placed in the
29853 linked folder
29854
29855 • When server-side moving (renaming) the shortcut is renamed, not the
29856 destination folder
29857
29858 • When server-side copying the contents of the linked folder is copied,
29859 not the shortcut.
29860
29861 • When deleting with rclone rmdir or rclone purge the shortcut is
29862 deleted not the linked folder.
29863
29864 • NB When deleting with rclone remove or rclone mount the contents of
29865 the linked folder will be deleted.
29866
29867 The rclone backend (https://rclone.org/commands/rclone_backend/) com‐
29868 mand can be used to create shortcuts.
29869
29870 Shortcuts can be completely ignored with the --drive-skip-shortcuts
29871 flag or the corresponding skip_shortcuts configuration setting.
29872
29873 Emptying trash
29874 If you wish to empty your trash you can use the rclone cleanup remote:
29875 command which will permanently delete all your trashed files. This
29876 command does not take any path arguments.
29877
29878 Note that Google Drive takes some time (minutes to days) to empty the
29879 trash even though the command returns within a few seconds. No output
29880 is echoed, so there will be no confirmation even using -v or -vv.
29881
29882 Quota information
29883 To view your current quota you can use the rclone about remote: command
29884 which will display your usage limit (quota), the usage in Google Drive,
29885 the size of all files in the Trash and the space used by other Google
29886 services such as Gmail. This command does not take any path arguments.
29887
29888 Import/Export of google documents
29889 Google documents can be exported from and uploaded to Google Drive.
29890
29891 When rclone downloads a Google doc it chooses a format to download de‐
29892 pending upon the --drive-export-formats setting. By default the export
29893 formats are docx,xlsx,pptx,svg which are a sensible default for an ed‐
29894 itable document.
29895
29896 When choosing a format, rclone runs down the list provided in order and
29897 chooses the first file format the doc can be exported as from the list.
29898 If the file can't be exported to a format on the formats list, then
29899 rclone will choose a format from the default list.
29900
29901 If you prefer an archive copy then you might use --drive-export-formats
29902 pdf, or if you prefer openoffice/libreoffice formats you might use
29903 --drive-export-formats ods,odt,odp.
29904
29905 Note that rclone adds the extension to the google doc, so if it is
29906 called My Spreadsheet on google docs, it will be exported as My Spread‐
29907 sheet.xlsx or My Spreadsheet.pdf etc.
29908
29909 When importing files into Google Drive, rclone will convert all files
29910 with an extension in --drive-import-formats to their associated docu‐
29911 ment type. rclone will not convert any files by default, since the
29912 conversion is lossy process.
29913
29914 The conversion must result in a file with the same extension when the
29915 --drive-export-formats rules are applied to the uploaded document.
29916
29917 Here are some examples for allowed and prohibited conversions.
29918
29919 export-for‐ import-for‐ Upload Ext Document Ext Allowed
29920 mats mats
29921 ────────────────────────────────────────────────────────────────
29922 odt odt odt odt Yes
29923 odt docx,odt odt odt Yes
29924 docx docx docx Yes
29925 odt odt docx No
29926 odt,docx docx,odt docx odt No
29927 docx,odt docx,odt docx docx Yes
29928 docx,odt docx,odt odt docx No
29929
29930 This limitation can be disabled by specifying --drive-allow-import-
29931 name-change. When using this flag, rclone can convert multiple files
29932 types resulting in the same document type at once, e.g. with --drive-
29933 import-formats docx,odt,txt, all files having these extension would re‐
29934 sult in a document represented as a docx file. This brings the addi‐
29935 tional risk of overwriting a document, if multiple files have the same
29936 stem. Many rclone operations will not handle this name change in any
29937 way. They assume an equal name when copying files and might copy the
29938 file again or delete them when the name changes.
29939
29940 Here are the possible export extensions with their corresponding mime
29941 types. Most of these can also be used for importing, but there more
29942 that are not listed here. Some of these additional ones might only be
29943 available when the operating system provides the correct MIME type en‐
29944 tries.
29945
29946 This list can be changed by Google Drive at any time and might not rep‐
29947 resent the currently available conversions.
29948
29949 Extension Mime Type Description
29950 ─────────────────────────────────────────────────────────────────────────────
29951 bmp image/bmp Windows Bitmap format
29952 csv text/csv Standard CSV format for
29953 Spreadsheets
29954 doc application/msword Classic Word file
29955 docx application/vnd.openxml‐ Microsoft Office Document
29956 formats-officedocu‐
29957 ment.wordprocess‐
29958 ingml.document
29959 epub application/epub+zip E-book format
29960 html text/html An HTML Document
29961 jpg image/jpeg A JPEG Image File
29962 json application/vnd.google- JSON Text Format for
29963 apps.script+json Google Apps scripts
29964 odp application/vnd.oa‐ Openoffice Presentation
29965 sis.opendocument.presen‐
29966 tation
29967
29968
29969 ods application/vnd.oa‐ Openoffice Spreadsheet
29970 sis.opendocument.spread‐
29971 sheet
29972 ods application/x-vnd.oa‐ Openoffice Spreadsheet
29973 sis.opendocument.spread‐
29974 sheet
29975 odt application/vnd.oa‐ Openoffice Document
29976 sis.opendocument.text
29977 pdf application/pdf Adobe PDF Format
29978 pjpeg image/pjpeg Progressive JPEG Image
29979 png image/png PNG Image Format
29980 pptx application/vnd.openxml‐ Microsoft Office Power‐
29981 formats-officedocu‐ point
29982 ment.presentationml.pre‐
29983 sentation
29984 rtf application/rtf Rich Text Format
29985 svg image/svg+xml Scalable Vector Graphics
29986 Format
29987 tsv text/tab-separated-val‐ Standard TSV format for
29988 ues spreadsheets
29989 txt text/plain Plain Text
29990 wmf application/x-msmetafile Windows Meta File
29991 xls application/vnd.ms-excel Classic Excel file
29992 xlsx application/vnd.openxml‐ Microsoft Office Spread‐
29993 formats-officedocu‐ sheet
29994 ment.spreadsheetml.sheet
29995 zip application/zip A ZIP file of HTML, Images
29996 CSS
29997
29998 Google documents can also be exported as link files. These files will
29999 open a browser window for the Google Docs website of that document when
30000 opened. The link file extension has to be specified as a --drive-ex‐
30001 port-formats parameter. They will match all available Google Docu‐
30002 ments.
30003
30004 Extension Description OS Support
30005 ─────────────────────────────────────────────────
30006 desktop freedesktop.org Linux
30007 specified desktop
30008 entry
30009 link.html An HTML Document All
30010 with a redirect
30011 url INI style link file macOS, Windows
30012 webloc macOS specific XML macOS
30013 format
30014
30015 Standard options
30016 Here are the Standard options specific to drive (Google Drive).
30017
30018 --drive-client-id
30019 Google Application Client Id Setting your own is recommended. See
30020 https://rclone.org/drive/#making-your-own-client-id for how to create
30021 your own. If you leave this blank, it will use an internal key which
30022 is low performance.
30023
30024 Properties:
30025
30026 • Config: client_id
30027
30028 • Env Var: RCLONE_DRIVE_CLIENT_ID
30029
30030 • Type: string
30031
30032 • Required: false
30033
30034 --drive-client-secret
30035 OAuth Client Secret.
30036
30037 Leave blank normally.
30038
30039 Properties:
30040
30041 • Config: client_secret
30042
30043 • Env Var: RCLONE_DRIVE_CLIENT_SECRET
30044
30045 • Type: string
30046
30047 • Required: false
30048
30049 --drive-scope
30050 Scope that rclone should use when requesting access from drive.
30051
30052 Properties:
30053
30054 • Config: scope
30055
30056 • Env Var: RCLONE_DRIVE_SCOPE
30057
30058 • Type: string
30059
30060 • Required: false
30061
30062 • Examples:
30063
30064 • "drive"
30065
30066 • Full access all files, excluding Application Data Folder.
30067
30068 • "drive.readonly"
30069
30070 • Read-only access to file metadata and file contents.
30071
30072 • "drive.file"
30073
30074 • Access to files created by rclone only.
30075
30076 • These are visible in the drive website.
30077
30078 • File authorization is revoked when the user deauthorizes the app.
30079
30080 • "drive.appfolder"
30081
30082 • Allows read and write access to the Application Data folder.
30083
30084 • This is not visible in the drive website.
30085
30086 • "drive.metadata.readonly"
30087
30088 • Allows read-only access to file metadata but
30089
30090 • does not allow any access to read or download file content.
30091
30092 --drive-service-account-file
30093 Service Account Credentials JSON file path.
30094
30095 Leave blank normally. Needed only if you want use SA instead of inter‐
30096 active login.
30097
30098 Leading ~ will be expanded in the file name as will environment vari‐
30099 ables such as ${RCLONE_CONFIG_DIR}.
30100
30101 Properties:
30102
30103 • Config: service_account_file
30104
30105 • Env Var: RCLONE_DRIVE_SERVICE_ACCOUNT_FILE
30106
30107 • Type: string
30108
30109 • Required: false
30110
30111 --drive-alternate-export
30112 Deprecated: No longer needed.
30113
30114 Properties:
30115
30116 • Config: alternate_export
30117
30118 • Env Var: RCLONE_DRIVE_ALTERNATE_EXPORT
30119
30120 • Type: bool
30121
30122 • Default: false
30123
30124 Advanced options
30125 Here are the Advanced options specific to drive (Google Drive).
30126
30127 --drive-token
30128 OAuth Access Token as a JSON blob.
30129
30130 Properties:
30131
30132 • Config: token
30133
30134 • Env Var: RCLONE_DRIVE_TOKEN
30135
30136 • Type: string
30137
30138 • Required: false
30139
30140 --drive-auth-url
30141 Auth server URL.
30142
30143 Leave blank to use the provider defaults.
30144
30145 Properties:
30146
30147 • Config: auth_url
30148
30149 • Env Var: RCLONE_DRIVE_AUTH_URL
30150
30151 • Type: string
30152
30153 • Required: false
30154
30155 --drive-token-url
30156 Token server url.
30157
30158 Leave blank to use the provider defaults.
30159
30160 Properties:
30161
30162 • Config: token_url
30163
30164 • Env Var: RCLONE_DRIVE_TOKEN_URL
30165
30166 • Type: string
30167
30168 • Required: false
30169
30170 --drive-root-folder-id
30171 ID of the root folder. Leave blank normally.
30172
30173 Fill in to access "Computers" folders (see docs), or for rclone to use
30174 a non root folder as its starting point.
30175
30176 Properties:
30177
30178 • Config: root_folder_id
30179
30180 • Env Var: RCLONE_DRIVE_ROOT_FOLDER_ID
30181
30182 • Type: string
30183
30184 • Required: false
30185
30186 --drive-service-account-credentials
30187 Service Account Credentials JSON blob.
30188
30189 Leave blank normally. Needed only if you want use SA instead of inter‐
30190 active login.
30191
30192 Properties:
30193
30194 • Config: service_account_credentials
30195
30196 • Env Var: RCLONE_DRIVE_SERVICE_ACCOUNT_CREDENTIALS
30197
30198 • Type: string
30199
30200 • Required: false
30201
30202 --drive-team-drive
30203 ID of the Shared Drive (Team Drive).
30204
30205 Properties:
30206
30207 • Config: team_drive
30208
30209 • Env Var: RCLONE_DRIVE_TEAM_DRIVE
30210
30211 • Type: string
30212
30213 • Required: false
30214
30215 --drive-auth-owner-only
30216 Only consider files owned by the authenticated user.
30217
30218 Properties:
30219
30220 • Config: auth_owner_only
30221
30222 • Env Var: RCLONE_DRIVE_AUTH_OWNER_ONLY
30223
30224 • Type: bool
30225
30226 • Default: false
30227
30228 --drive-use-trash
30229 Send files to the trash instead of deleting permanently.
30230
30231 Defaults to true, namely sending files to the trash. Use --drive-use-
30232 trash=false to delete files permanently instead.
30233
30234 Properties:
30235
30236 • Config: use_trash
30237
30238 • Env Var: RCLONE_DRIVE_USE_TRASH
30239
30240 • Type: bool
30241
30242 • Default: true
30243
30244 --drive-copy-shortcut-content
30245 Server side copy contents of shortcuts instead of the shortcut.
30246
30247 When doing server side copies, normally rclone will copy shortcuts as
30248 shortcuts.
30249
30250 If this flag is used then rclone will copy the contents of shortcuts
30251 rather than shortcuts themselves when doing server side copies.
30252
30253 Properties:
30254
30255 • Config: copy_shortcut_content
30256
30257 • Env Var: RCLONE_DRIVE_COPY_SHORTCUT_CONTENT
30258
30259 • Type: bool
30260
30261 • Default: false
30262
30263 --drive-skip-gdocs
30264 Skip google documents in all listings.
30265
30266 If given, gdocs practically become invisible to rclone.
30267
30268 Properties:
30269
30270 • Config: skip_gdocs
30271
30272 • Env Var: RCLONE_DRIVE_SKIP_GDOCS
30273
30274 • Type: bool
30275
30276 • Default: false
30277
30278 --drive-skip-checksum-gphotos
30279 Skip MD5 checksum on Google photos and videos only.
30280
30281 Use this if you get checksum errors when transferring Google photos or
30282 videos.
30283
30284 Setting this flag will cause Google photos and videos to return a blank
30285 MD5 checksum.
30286
30287 Google photos are identified by being in the "photos" space.
30288
30289 Corrupted checksums are caused by Google modifying the image/video but
30290 not updating the checksum.
30291
30292 Properties:
30293
30294 • Config: skip_checksum_gphotos
30295
30296 • Env Var: RCLONE_DRIVE_SKIP_CHECKSUM_GPHOTOS
30297
30298 • Type: bool
30299
30300 • Default: false
30301
30302 --drive-shared-with-me
30303 Only show files that are shared with me.
30304
30305 Instructs rclone to operate on your "Shared with me" folder (where
30306 Google Drive lets you access the files and folders others have shared
30307 with you).
30308
30309 This works both with the "list" (lsd, lsl, etc.) and the "copy" com‐
30310 mands (copy, sync, etc.), and with all other commands too.
30311
30312 Properties:
30313
30314 • Config: shared_with_me
30315
30316 • Env Var: RCLONE_DRIVE_SHARED_WITH_ME
30317
30318 • Type: bool
30319
30320 • Default: false
30321
30322 --drive-trashed-only
30323 Only show files that are in the trash.
30324
30325 This will show trashed files in their original directory structure.
30326
30327 Properties:
30328
30329 • Config: trashed_only
30330
30331 • Env Var: RCLONE_DRIVE_TRASHED_ONLY
30332
30333 • Type: bool
30334
30335 • Default: false
30336
30337 --drive-starred-only
30338 Only show files that are starred.
30339
30340 Properties:
30341
30342 • Config: starred_only
30343
30344 • Env Var: RCLONE_DRIVE_STARRED_ONLY
30345
30346 • Type: bool
30347
30348 • Default: false
30349
30350 --drive-formats
30351 Deprecated: See export_formats.
30352
30353 Properties:
30354
30355 • Config: formats
30356
30357 • Env Var: RCLONE_DRIVE_FORMATS
30358
30359 • Type: string
30360
30361 • Required: false
30362
30363 --drive-export-formats
30364 Comma separated list of preferred formats for downloading Google docs.
30365
30366 Properties:
30367
30368 • Config: export_formats
30369
30370 • Env Var: RCLONE_DRIVE_EXPORT_FORMATS
30371
30372 • Type: string
30373
30374 • Default: "docx,xlsx,pptx,svg"
30375
30376 --drive-import-formats
30377 Comma separated list of preferred formats for uploading Google docs.
30378
30379 Properties:
30380
30381 • Config: import_formats
30382
30383 • Env Var: RCLONE_DRIVE_IMPORT_FORMATS
30384
30385 • Type: string
30386
30387 • Required: false
30388
30389 --drive-allow-import-name-change
30390 Allow the filetype to change when uploading Google docs.
30391
30392 E.g. file.doc to file.docx. This will confuse sync and reupload every
30393 time.
30394
30395 Properties:
30396
30397 • Config: allow_import_name_change
30398
30399 • Env Var: RCLONE_DRIVE_ALLOW_IMPORT_NAME_CHANGE
30400
30401 • Type: bool
30402
30403 • Default: false
30404
30405 --drive-use-created-date
30406 Use file created date instead of modified date.
30407
30408 Useful when downloading data and you want the creation date used in
30409 place of the last modified date.
30410
30411 WARNING: This flag may have some unexpected consequences.
30412
30413 When uploading to your drive all files will be overwritten unless they
30414 haven't been modified since their creation. And the inverse will occur
30415 while downloading. This side effect can be avoided by using the
30416 "--checksum" flag.
30417
30418 This feature was implemented to retain photos capture date as recorded
30419 by google photos. You will first need to check the "Create a Google
30420 Photos folder" option in your google drive settings. You can then copy
30421 or move the photos locally and use the date the image was taken (creat‐
30422 ed) set as the modification date.
30423
30424 Properties:
30425
30426 • Config: use_created_date
30427
30428 • Env Var: RCLONE_DRIVE_USE_CREATED_DATE
30429
30430 • Type: bool
30431
30432 • Default: false
30433
30434 --drive-use-shared-date
30435 Use date file was shared instead of modified date.
30436
30437 Note that, as with "--drive-use-created-date", this flag may have unex‐
30438 pected consequences when uploading/downloading files.
30439
30440 If both this flag and "--drive-use-created-date" are set, the created
30441 date is used.
30442
30443 Properties:
30444
30445 • Config: use_shared_date
30446
30447 • Env Var: RCLONE_DRIVE_USE_SHARED_DATE
30448
30449 • Type: bool
30450
30451 • Default: false
30452
30453 --drive-list-chunk
30454 Size of listing chunk 100-1000, 0 to disable.
30455
30456 Properties:
30457
30458 • Config: list_chunk
30459
30460 • Env Var: RCLONE_DRIVE_LIST_CHUNK
30461
30462 • Type: int
30463
30464 • Default: 1000
30465
30466 --drive-impersonate
30467 Impersonate this user when using a service account.
30468
30469 Properties:
30470
30471 • Config: impersonate
30472
30473 • Env Var: RCLONE_DRIVE_IMPERSONATE
30474
30475 • Type: string
30476
30477 • Required: false
30478
30479 --drive-upload-cutoff
30480 Cutoff for switching to chunked upload.
30481
30482 Properties:
30483
30484 • Config: upload_cutoff
30485
30486 • Env Var: RCLONE_DRIVE_UPLOAD_CUTOFF
30487
30488 • Type: SizeSuffix
30489
30490 • Default: 8Mi
30491
30492 --drive-chunk-size
30493 Upload chunk size.
30494
30495 Must a power of 2 >= 256k.
30496
30497 Making this larger will improve performance, but note that each chunk
30498 is buffered in memory one per transfer.
30499
30500 Reducing this will reduce memory usage but decrease performance.
30501
30502 Properties:
30503
30504 • Config: chunk_size
30505
30506 • Env Var: RCLONE_DRIVE_CHUNK_SIZE
30507
30508 • Type: SizeSuffix
30509
30510 • Default: 8Mi
30511
30512 --drive-acknowledge-abuse
30513 Set to allow files which return cannotDownloadAbusiveFile to be down‐
30514 loaded.
30515
30516 If downloading a file returns the error "This file has been identified
30517 as malware or spam and cannot be downloaded" with the error code "can‐
30518 notDownloadAbusiveFile" then supply this flag to rclone to indicate you
30519 acknowledge the risks of downloading the file and rclone will download
30520 it anyway.
30521
30522 Properties:
30523
30524 • Config: acknowledge_abuse
30525
30526 • Env Var: RCLONE_DRIVE_ACKNOWLEDGE_ABUSE
30527
30528 • Type: bool
30529
30530 • Default: false
30531
30532 --drive-keep-revision-forever
30533 Keep new head revision of each file forever.
30534
30535 Properties:
30536
30537 • Config: keep_revision_forever
30538
30539 • Env Var: RCLONE_DRIVE_KEEP_REVISION_FOREVER
30540
30541 • Type: bool
30542
30543 • Default: false
30544
30545 --drive-size-as-quota
30546 Show sizes as storage quota usage, not actual size.
30547
30548 Show the size of a file as the storage quota used. This is the current
30549 version plus any older versions that have been set to keep forever.
30550
30551 WARNING: This flag may have some unexpected consequences.
30552
30553 It is not recommended to set this flag in your config - the recommended
30554 usage is using the flag form --drive-size-as-quota when doing rclone
30555 ls/lsl/lsf/lsjson/etc only.
30556
30557 If you do use this flag for syncing (not recommended) then you will
30558 need to use --ignore size also.
30559
30560 Properties:
30561
30562 • Config: size_as_quota
30563
30564 • Env Var: RCLONE_DRIVE_SIZE_AS_QUOTA
30565
30566 • Type: bool
30567
30568 • Default: false
30569
30570 --drive-v2-download-min-size
30571 If Object's are greater, use drive v2 API to download.
30572
30573 Properties:
30574
30575 • Config: v2_download_min_size
30576
30577 • Env Var: RCLONE_DRIVE_V2_DOWNLOAD_MIN_SIZE
30578
30579 • Type: SizeSuffix
30580
30581 • Default: off
30582
30583 --drive-pacer-min-sleep
30584 Minimum time to sleep between API calls.
30585
30586 Properties:
30587
30588 • Config: pacer_min_sleep
30589
30590 • Env Var: RCLONE_DRIVE_PACER_MIN_SLEEP
30591
30592 • Type: Duration
30593
30594 • Default: 100ms
30595
30596 --drive-pacer-burst
30597 Number of API calls to allow without sleeping.
30598
30599 Properties:
30600
30601 • Config: pacer_burst
30602
30603 • Env Var: RCLONE_DRIVE_PACER_BURST
30604
30605 • Type: int
30606
30607 • Default: 100
30608
30609 --drive-server-side-across-configs
30610 Allow server-side operations (e.g. copy) to work across different
30611 drive configs.
30612
30613 This can be useful if you wish to do a server-side copy between two
30614 different Google drives. Note that this isn't enabled by default be‐
30615 cause it isn't easy to tell if it will work between any two configura‐
30616 tions.
30617
30618 Properties:
30619
30620 • Config: server_side_across_configs
30621
30622 • Env Var: RCLONE_DRIVE_SERVER_SIDE_ACROSS_CONFIGS
30623
30624 • Type: bool
30625
30626 • Default: false
30627
30628 --drive-disable-http2
30629 Disable drive using http2.
30630
30631 There is currently an unsolved issue with the google drive backend and
30632 HTTP/2. HTTP/2 is therefore disabled by default for the drive backend
30633 but can be re-enabled here. When the issue is solved this flag will be
30634 removed.
30635
30636 See: https://github.com/rclone/rclone/issues/3631
30637
30638 Properties:
30639
30640 • Config: disable_http2
30641
30642 • Env Var: RCLONE_DRIVE_DISABLE_HTTP2
30643
30644 • Type: bool
30645
30646 • Default: true
30647
30648 --drive-stop-on-upload-limit
30649 Make upload limit errors be fatal.
30650
30651 At the time of writing it is only possible to upload 750 GiB of data to
30652 Google Drive a day (this is an undocumented limit). When this limit is
30653 reached Google Drive produces a slightly different error message. When
30654 this flag is set it causes these errors to be fatal. These will stop
30655 the in-progress sync.
30656
30657 Note that this detection is relying on error message strings which
30658 Google don't document so it may break in the future.
30659
30660 See: https://github.com/rclone/rclone/issues/3857
30661
30662 Properties:
30663
30664 • Config: stop_on_upload_limit
30665
30666 • Env Var: RCLONE_DRIVE_STOP_ON_UPLOAD_LIMIT
30667
30668 • Type: bool
30669
30670 • Default: false
30671
30672 --drive-stop-on-download-limit
30673 Make download limit errors be fatal.
30674
30675 At the time of writing it is only possible to download 10 TiB of data
30676 from Google Drive a day (this is an undocumented limit). When this
30677 limit is reached Google Drive produces a slightly different error mes‐
30678 sage. When this flag is set it causes these errors to be fatal. These
30679 will stop the in-progress sync.
30680
30681 Note that this detection is relying on error message strings which
30682 Google don't document so it may break in the future.
30683
30684 Properties:
30685
30686 • Config: stop_on_download_limit
30687
30688 • Env Var: RCLONE_DRIVE_STOP_ON_DOWNLOAD_LIMIT
30689
30690 • Type: bool
30691
30692 • Default: false
30693
30694 --drive-skip-shortcuts
30695 If set skip shortcut files.
30696
30697 Normally rclone dereferences shortcut files making them appear as if
30698 they are the original file (see the shortcuts section). If this flag
30699 is set then rclone will ignore shortcut files completely.
30700
30701 Properties:
30702
30703 • Config: skip_shortcuts
30704
30705 • Env Var: RCLONE_DRIVE_SKIP_SHORTCUTS
30706
30707 • Type: bool
30708
30709 • Default: false
30710
30711 --drive-skip-dangling-shortcuts
30712 If set skip dangling shortcut files.
30713
30714 If this is set then rclone will not show any dangling shortcuts in
30715 listings.
30716
30717 Properties:
30718
30719 • Config: skip_dangling_shortcuts
30720
30721 • Env Var: RCLONE_DRIVE_SKIP_DANGLING_SHORTCUTS
30722
30723 • Type: bool
30724
30725 • Default: false
30726
30727 --drive-resource-key
30728 Resource key for accessing a link-shared file.
30729
30730 If you need to access files shared with a link like this
30731
30732 https://drive.google.com/drive/folders/XXX?resourcekey=YYY&usp=sharing
30733
30734 Then you will need to use the first part "XXX" as the "root_folder_id"
30735 and the second part "YYY" as the "resource_key" otherwise you will get
30736 404 not found errors when trying to access the directory.
30737
30738 See: https://developers.google.com/drive/api/guides/resource-keys
30739
30740 This resource key requirement only applies to a subset of old files.
30741
30742 Note also that opening the folder once in the web interface (with the
30743 user you've authenticated rclone with) seems to be enough so that the
30744 resource key is no needed.
30745
30746 Properties:
30747
30748 • Config: resource_key
30749
30750 • Env Var: RCLONE_DRIVE_RESOURCE_KEY
30751
30752 • Type: string
30753
30754 • Required: false
30755
30756 --drive-encoding
30757 The encoding for the backend.
30758
30759 See the encoding section in the overview (https://rclone.org/over‐
30760 view/#encoding) for more info.
30761
30762 Properties:
30763
30764 • Config: encoding
30765
30766 • Env Var: RCLONE_DRIVE_ENCODING
30767
30768 • Type: MultiEncoder
30769
30770 • Default: InvalidUtf8
30771
30772 Backend commands
30773 Here are the commands specific to the drive backend.
30774
30775 Run them with
30776
30777 rclone backend COMMAND remote:
30778
30779 The help below will explain what arguments each command takes.
30780
30781 See the backend (https://rclone.org/commands/rclone_backend/) command
30782 for more info on how to pass options and arguments.
30783
30784 These can be run on a running backend using the rc command backend/com‐
30785 mand (https://rclone.org/rc/#backend-command).
30786
30787 get
30788 Get command for fetching the drive config parameters
30789
30790 rclone backend get remote: [options] [<arguments>+]
30791
30792 This is a get command which will be used to fetch the various drive
30793 config parameters
30794
30795 Usage Examples:
30796
30797 rclone backend get drive: [-o service_account_file] [-o chunk_size]
30798 rclone rc backend/command command=get fs=drive: [-o service_account_file] [-o chunk_size]
30799
30800 Options:
30801
30802 • "chunk_size": show the current upload chunk size
30803
30804 • "service_account_file": show the current service account file
30805
30806 set
30807 Set command for updating the drive config parameters
30808
30809 rclone backend set remote: [options] [<arguments>+]
30810
30811 This is a set command which will be used to update the various drive
30812 config parameters
30813
30814 Usage Examples:
30815
30816 rclone backend set drive: [-o service_account_file=sa.json] [-o chunk_size=67108864]
30817 rclone rc backend/command command=set fs=drive: [-o service_account_file=sa.json] [-o chunk_size=67108864]
30818
30819 Options:
30820
30821 • "chunk_size": update the current upload chunk size
30822
30823 • "service_account_file": update the current service account file
30824
30825 shortcut
30826 Create shortcuts from files or directories
30827
30828 rclone backend shortcut remote: [options] [<arguments>+]
30829
30830 This command creates shortcuts from files or directories.
30831
30832 Usage:
30833
30834 rclone backend shortcut drive: source_item destination_shortcut
30835 rclone backend shortcut drive: source_item -o target=drive2: destination_shortcut
30836
30837 In the first example this creates a shortcut from the "source_item"
30838 which can be a file or a directory to the "destination_shortcut". The
30839 "source_item" and the "destination_shortcut" should be relative paths
30840 from "drive:"
30841
30842 In the second example this creates a shortcut from the "source_item"
30843 relative to "drive:" to the "destination_shortcut" relative to
30844 "drive2:". This may fail with a permission error if the user authenti‐
30845 cated with "drive2:" can't read files from "drive:".
30846
30847 Options:
30848
30849 • "target": optional target remote for the shortcut destination
30850
30851 drives
30852 List the Shared Drives available to this account
30853
30854 rclone backend drives remote: [options] [<arguments>+]
30855
30856 This command lists the Shared Drives (Team Drives) available to this
30857 account.
30858
30859 Usage:
30860
30861 rclone backend [-o config] drives drive:
30862
30863 This will return a JSON list of objects like this
30864
30865 [
30866 {
30867 "id": "0ABCDEF-01234567890",
30868 "kind": "drive#teamDrive",
30869 "name": "My Drive"
30870 },
30871 {
30872 "id": "0ABCDEFabcdefghijkl",
30873 "kind": "drive#teamDrive",
30874 "name": "Test Drive"
30875 }
30876 ]
30877
30878 With the -o config parameter it will output the list in a format suit‐
30879 able for adding to a config file to make aliases for all the drives
30880 found and a combined drive.
30881
30882 [My Drive]
30883 type = alias
30884 remote = drive,team_drive=0ABCDEF-01234567890,root_folder_id=:
30885
30886 [Test Drive]
30887 type = alias
30888 remote = drive,team_drive=0ABCDEFabcdefghijkl,root_folder_id=:
30889
30890 [AllDrives]
30891 type = combine
30892 upstreams = "My Drive=My Drive:" "Test Drive=Test Drive:"
30893
30894 Adding this to the rclone config file will cause those team drives to
30895 be accessible with the aliases shown. Any illegal characters will be
30896 substituted with "_" and duplicate names will have numbers suffixed.
30897 It will also add a remote called AllDrives which shows all the shared
30898 drives combined into one directory tree.
30899
30900 untrash
30901 Untrash files and directories
30902
30903 rclone backend untrash remote: [options] [<arguments>+]
30904
30905 This command untrashes all the files and directories in the directory
30906 passed in recursively.
30907
30908 Usage:
30909
30910 This takes an optional directory to trash which make this easier to use
30911 via the API.
30912
30913 rclone backend untrash drive:directory
30914 rclone backend -i untrash drive:directory subdir
30915
30916 Use the -i flag to see what would be restored before restoring it.
30917
30918 Result:
30919
30920 {
30921 "Untrashed": 17,
30922 "Errors": 0
30923 }
30924
30925 copyid
30926 Copy files by ID
30927
30928 rclone backend copyid remote: [options] [<arguments>+]
30929
30930 This command copies files by ID
30931
30932 Usage:
30933
30934 rclone backend copyid drive: ID path
30935 rclone backend copyid drive: ID1 path1 ID2 path2
30936
30937 It copies the drive file with ID given to the path (an rclone path
30938 which will be passed internally to rclone copyto). The ID and path
30939 pairs can be repeated.
30940
30941 The path should end with a / to indicate copy the file as named to this
30942 directory. If it doesn't end with a / then the last path component
30943 will be used as the file name.
30944
30945 If the destination is a drive backend then server-side copying will be
30946 attempted if possible.
30947
30948 Use the -i flag to see what would be copied before copying.
30949
30950 exportformats
30951 Dump the export formats for debug purposes
30952
30953 rclone backend exportformats remote: [options] [<arguments>+]
30954
30955 importformats
30956 Dump the import formats for debug purposes
30957
30958 rclone backend importformats remote: [options] [<arguments>+]
30959
30960 Limitations
30961 Drive has quite a lot of rate limiting. This causes rclone to be lim‐
30962 ited to transferring about 2 files per second only. Individual files
30963 may be transferred much faster at 100s of MiB/s but lots of small files
30964 can take a long time.
30965
30966 Server side copies are also subject to a separate rate limit. If you
30967 see User rate limit exceeded errors, wait at least 24 hours and retry.
30968 You can disable server-side copies with --disable copy to download and
30969 upload the files if you prefer.
30970
30971 Limitations of Google Docs
30972 Google docs will appear as size -1 in rclone ls, rclone ncdu etc, and
30973 as size 0 in anything which uses the VFS layer, e.g. rclone mount and
30974 rclone serve. When calculating directory totals, e.g. in rclone size
30975 and rclone ncdu, they will be counted in as empty files.
30976
30977 This is because rclone can't find out the size of the Google docs with‐
30978 out downloading them.
30979
30980 Google docs will transfer correctly with rclone sync, rclone copy etc
30981 as rclone knows to ignore the size when doing the transfer.
30982
30983 However an unfortunate consequence of this is that you may not be able
30984 to download Google docs using rclone mount. If it doesn't work you
30985 will get a 0 sized file. If you try again the doc may gain its correct
30986 size and be downloadable. Whether it will work on not depends on the
30987 application accessing the mount and the OS you are running - experiment
30988 to find out if it does work for you!
30989
30990 Duplicated files
30991 Sometimes, for no reason I've been able to track down, drive will du‐
30992 plicate a file that rclone uploads. Drive unlike all the other remotes
30993 can have duplicated files.
30994
30995 Duplicated files cause problems with the syncing and you will see mes‐
30996 sages in the log about duplicates.
30997
30998 Use rclone dedupe to fix duplicated files.
30999
31000 Note that this isn't just a problem with rclone, even Google Photos on
31001 Android duplicates files on drive sometimes.
31002
31003 Rclone appears to be re-copying files it shouldn't
31004 The most likely cause of this is the duplicated file issue above - run
31005 rclone dedupe and check your logs for duplicate object or directory
31006 messages.
31007
31008 This can also be caused by a delay/caching on google drive's end when
31009 comparing directory listings. Specifically with team drives used in
31010 combination with --fast-list. Files that were uploaded recently may
31011 not appear on the directory list sent to rclone when using --fast-list.
31012
31013 Waiting a moderate period of time between attempts (estimated to be ap‐
31014 proximately 1 hour) and/or not using --fast-list both seem to be effec‐
31015 tive in preventing the problem.
31016
31017 Making your own client_id
31018 When you use rclone with Google drive in its default configuration you
31019 are using rclone's client_id. This is shared between all the rclone
31020 users. There is a global rate limit on the number of queries per sec‐
31021 ond that each client_id can do set by Google. rclone already has a
31022 high quota and I will continue to make sure it is high enough by con‐
31023 tacting Google.
31024
31025 It is strongly recommended to use your own client ID as the default
31026 rclone ID is heavily used. If you have multiple services running, it
31027 is recommended to use an API key for each service. The default Google
31028 quota is 10 transactions per second so it is recommended to stay under
31029 that number as if you use more than that, it will cause rclone to rate
31030 limit and make things slower.
31031
31032 Here is how to create your own Google Drive client ID for rclone:
31033
31034 1. Log into the Google API Console (https://console.develop‐
31035 ers.google.com/) with your Google account. It doesn't matter what
31036 Google account you use. (It need not be the same account as the
31037 Google Drive you want to access)
31038
31039 2. Select a project or create a new project.
31040
31041 3. Under "ENABLE APIS AND SERVICES" search for "Drive", and enable the
31042 "Google Drive API".
31043
31044 4. Click "Credentials" in the left-side panel (not "Create creden‐
31045 tials", which opens the wizard), then "Create credentials"
31046
31047 5. If you already configured an "Oauth Consent Screen", then skip to
31048 the next step; if not, click on "CONFIGURE CONSENT SCREEN" button
31049 (near the top right corner of the right panel), then select "Exter‐
31050 nal" and click on "CREATE"; on the next screen, enter an "Applica‐
31051 tion name" ("rclone" is OK); enter "User Support Email" (your own
31052 email is OK); enter "Developer Contact Email" (your own email is
31053 OK); then click on "Save" (all other data is optional). Click
31054 again on "Credentials" on the left panel to go back to the "Creden‐
31055 tials" screen.
31056
31057 (PS: if you are a GSuite user, you could also select "Internal" in‐
31058 stead of "External" above, but this will restrict API use to Google
31059 Workspace users in your organisation).
31060
31061 6. Click on the "+ CREATE CREDENTIALS" button at the top of the
31062 screen, then select "OAuth client ID".
31063
31064 7. Choose an application type of "Desktop app" and click "Create".
31065 (the default name is fine)
31066
31067 8. It will show you a client ID and client secret. Make a note of
31068 these.
31069
31070 (If you selected "External" at Step 5 continue to "Publish App" in
31071 the Steps 9 and 10. If you chose "Internal" you don't need to pub‐
31072 lish and can skip straight to Step 11.)
31073
31074 9. Go to "Oauth consent screen" and press "Publish App"
31075
31076 10. Click "OAuth consent screen", then click "PUBLISH APP" button and
31077 confirm, or add your account under "Test users".
31078
31079 11. Provide the noted client ID and client secret to rclone.
31080
31081 Be aware that, due to the "enhanced security" recently introduced by
31082 Google, you are theoretically expected to "submit your app for verifi‐
31083 cation" and then wait a few weeks(!) for their response; in practice,
31084 you can go right ahead and use the client ID and client secret with
31085 rclone, the only issue will be a very scary confirmation screen shown
31086 when you connect via your browser for rclone to be able to get its to‐
31087 ken-id (but as this only happens during the remote configuration, it's
31088 not such a big deal).
31089
31090 (Thanks to @balazer on github for these instructions.)
31091
31092 Sometimes, creation of an OAuth consent in Google API Console fails due
31093 to an error message “The request failed because changes to one of the
31094 field of the resource is not supported”. As a convenient workaround,
31095 the necessary Google Drive API key can be created on the Python Quick‐
31096 start (https://developers.google.com/drive/api/v3/quickstart/python)
31097 page. Just push the Enable the Drive API button to receive the Client
31098 ID and Secret. Note that it will automatically create a new project in
31099 the API Console.
31100
31102 The rclone backend for Google Photos (https://www.google.com/pho‐
31103 tos/about/) is a specialized backend for transferring photos and videos
31104 to and from Google Photos.
31105
31106 NB The Google Photos API which rclone uses has quite a few limitations,
31107 so please read the limitations section carefully to make sure it is
31108 suitable for your use.
31109
31110 Configuration
31111 The initial setup for google cloud storage involves getting a token
31112 from Google Photos which you need to do in your browser. rclone config
31113 walks you through it.
31114
31115 Here is an example of how to make a remote called remote. First run:
31116
31117 rclone config
31118
31119 This will guide you through an interactive setup process:
31120
31121 No remotes found, make a new one?
31122 n) New remote
31123 s) Set configuration password
31124 q) Quit config
31125 n/s/q> n
31126 name> remote
31127 Type of storage to configure.
31128 Enter a string value. Press Enter for the default ("").
31129 Choose a number from below, or type in your own value
31130 [snip]
31131 XX / Google Photos
31132 \ "google photos"
31133 [snip]
31134 Storage> google photos
31135 ** See help for google photos backend at: https://rclone.org/googlephotos/ **
31136
31137 Google Application Client Id
31138 Leave blank normally.
31139 Enter a string value. Press Enter for the default ("").
31140 client_id>
31141 Google Application Client Secret
31142 Leave blank normally.
31143 Enter a string value. Press Enter for the default ("").
31144 client_secret>
31145 Set to make the Google Photos backend read only.
31146
31147 If you choose read only then rclone will only request read only access
31148 to your photos, otherwise rclone will request full access.
31149 Enter a boolean value (true or false). Press Enter for the default ("false").
31150 read_only>
31151 Edit advanced config? (y/n)
31152 y) Yes
31153 n) No
31154 y/n> n
31155 Remote config
31156 Use auto config?
31157 * Say Y if not sure
31158 * Say N if you are working on a remote or headless machine
31159 y) Yes
31160 n) No
31161 y/n> y
31162 If your browser doesn't open automatically go to the following link: http://127.0.0.1:53682/auth
31163 Log in and authorize rclone for access
31164 Waiting for code...
31165 Got code
31166
31167 *** IMPORTANT: All media items uploaded to Google Photos with rclone
31168 *** are stored in full resolution at original quality. These uploads
31169 *** will count towards storage in your Google Account.
31170
31171 --------------------
31172 [remote]
31173 type = google photos
31174 token = {"access_token":"XXX","token_type":"Bearer","refresh_token":"XXX","expiry":"2019-06-28T17:38:04.644930156+01:00"}
31175 --------------------
31176 y) Yes this is OK
31177 e) Edit this remote
31178 d) Delete this remote
31179 y/e/d> y
31180
31181 Note that rclone runs a webserver on your local machine to collect the
31182 token as returned from Google if you use auto config mode. This only
31183 runs from the moment it opens your browser to the moment you get back
31184 the verification code. This is on http://127.0.0.1:53682/ and this may
31185 require you to unblock it temporarily if you are running a host fire‐
31186 wall, or use manual mode.
31187
31188 This remote is called remote and can now be used like this
31189
31190 See all the albums in your photos
31191
31192 rclone lsd remote:album
31193
31194 Make a new album
31195
31196 rclone mkdir remote:album/newAlbum
31197
31198 List the contents of an album
31199
31200 rclone ls remote:album/newAlbum
31201
31202 Sync /home/local/images to the Google Photos, removing any excess files
31203 in the album.
31204
31205 rclone sync -i /home/local/image remote:album/newAlbum
31206
31207 Layout
31208 As Google Photos is not a general purpose cloud storage system the
31209 backend is laid out to help you navigate it.
31210
31211 The directories under media show different ways of categorizing the me‐
31212 dia. Each file will appear multiple times. So if you want to make a
31213 backup of your google photos you might choose to backup remote:me‐
31214 dia/by-month. (NB remote:media/by-day is rather slow at the moment so
31215 avoid for syncing.)
31216
31217 Note that all your photos and videos will appear somewhere under media,
31218 but they may not appear under album unless you've put them into albums.
31219
31220 /
31221 - upload
31222 - file1.jpg
31223 - file2.jpg
31224 - ...
31225 - media
31226 - all
31227 - file1.jpg
31228 - file2.jpg
31229 - ...
31230 - by-year
31231 - 2000
31232 - file1.jpg
31233 - ...
31234 - 2001
31235 - file2.jpg
31236 - ...
31237 - ...
31238 - by-month
31239 - 2000
31240 - 2000-01
31241 - file1.jpg
31242 - ...
31243 - 2000-02
31244 - file2.jpg
31245 - ...
31246 - ...
31247 - by-day
31248 - 2000
31249 - 2000-01-01
31250 - file1.jpg
31251 - ...
31252 - 2000-01-02
31253 - file2.jpg
31254 - ...
31255 - ...
31256 - album
31257 - album name
31258 - album name/sub
31259 - shared-album
31260 - album name
31261 - album name/sub
31262 - feature
31263 - favorites
31264 - file1.jpg
31265 - file2.jpg
31266
31267 There are two writable parts of the tree, the upload directory and sub
31268 directories of the album directory.
31269
31270 The upload directory is for uploading files you don't want to put into
31271 albums. This will be empty to start with and will contain the files
31272 you've uploaded for one rclone session only, becoming empty again when
31273 you restart rclone. The use case for this would be if you have a load
31274 of files you just want to once off dump into Google Photos. For re‐
31275 peated syncing, uploading to album will work better.
31276
31277 Directories within the album directory are also writeable and you may
31278 create new directories (albums) under album. If you copy files with a
31279 directory hierarchy in there then rclone will create albums with the /
31280 character in them. For example if you do
31281
31282 rclone copy /path/to/images remote:album/images
31283
31284 and the images directory contains
31285
31286 images
31287 - file1.jpg
31288 dir
31289 file2.jpg
31290 dir2
31291 dir3
31292 file3.jpg
31293
31294 Then rclone will create the following albums with the following files
31295 in
31296
31297 • images
31298
31299 • file1.jpg
31300
31301 • images/dir
31302
31303 • file2.jpg
31304
31305 • images/dir2/dir3
31306
31307 • file3.jpg
31308
31309 This means that you can use the album path pretty much like a normal
31310 filesystem and it is a good target for repeated syncing.
31311
31312 The shared-album directory shows albums shared with you or by you.
31313 This is similar to the Sharing tab in the Google Photos web interface.
31314
31315 Standard options
31316 Here are the Standard options specific to google photos (Google Pho‐
31317 tos).
31318
31319 --gphotos-client-id
31320 OAuth Client Id.
31321
31322 Leave blank normally.
31323
31324 Properties:
31325
31326 • Config: client_id
31327
31328 • Env Var: RCLONE_GPHOTOS_CLIENT_ID
31329
31330 • Type: string
31331
31332 • Required: false
31333
31334 --gphotos-client-secret
31335 OAuth Client Secret.
31336
31337 Leave blank normally.
31338
31339 Properties:
31340
31341 • Config: client_secret
31342
31343 • Env Var: RCLONE_GPHOTOS_CLIENT_SECRET
31344
31345 • Type: string
31346
31347 • Required: false
31348
31349 --gphotos-read-only
31350 Set to make the Google Photos backend read only.
31351
31352 If you choose read only then rclone will only request read only access
31353 to your photos, otherwise rclone will request full access.
31354
31355 Properties:
31356
31357 • Config: read_only
31358
31359 • Env Var: RCLONE_GPHOTOS_READ_ONLY
31360
31361 • Type: bool
31362
31363 • Default: false
31364
31365 Advanced options
31366 Here are the Advanced options specific to google photos (Google Pho‐
31367 tos).
31368
31369 --gphotos-token
31370 OAuth Access Token as a JSON blob.
31371
31372 Properties:
31373
31374 • Config: token
31375
31376 • Env Var: RCLONE_GPHOTOS_TOKEN
31377
31378 • Type: string
31379
31380 • Required: false
31381
31382 --gphotos-auth-url
31383 Auth server URL.
31384
31385 Leave blank to use the provider defaults.
31386
31387 Properties:
31388
31389 • Config: auth_url
31390
31391 • Env Var: RCLONE_GPHOTOS_AUTH_URL
31392
31393 • Type: string
31394
31395 • Required: false
31396
31397 --gphotos-token-url
31398 Token server url.
31399
31400 Leave blank to use the provider defaults.
31401
31402 Properties:
31403
31404 • Config: token_url
31405
31406 • Env Var: RCLONE_GPHOTOS_TOKEN_URL
31407
31408 • Type: string
31409
31410 • Required: false
31411
31412 --gphotos-read-size
31413 Set to read the size of media items.
31414
31415 Normally rclone does not read the size of media items since this takes
31416 another transaction. This isn't necessary for syncing. However rclone
31417 mount needs to know the size of files in advance of reading them, so
31418 setting this flag when using rclone mount is recommended if you want to
31419 read the media.
31420
31421 Properties:
31422
31423 • Config: read_size
31424
31425 • Env Var: RCLONE_GPHOTOS_READ_SIZE
31426
31427 • Type: bool
31428
31429 • Default: false
31430
31431 --gphotos-start-year
31432 Year limits the photos to be downloaded to those which are uploaded af‐
31433 ter the given year.
31434
31435 Properties:
31436
31437 • Config: start_year
31438
31439 • Env Var: RCLONE_GPHOTOS_START_YEAR
31440
31441 • Type: int
31442
31443 • Default: 2000
31444
31445 --gphotos-include-archived
31446 Also view and download archived media.
31447
31448 By default, rclone does not request archived media. Thus, when sync‐
31449 ing, archived media is not visible in directory listings or trans‐
31450 ferred.
31451
31452 Note that media in albums is always visible and synced, no matter their
31453 archive status.
31454
31455 With this flag, archived media are always visible in directory listings
31456 and transferred.
31457
31458 Without this flag, archived media will not be visible in directory
31459 listings and won't be transferred.
31460
31461 Properties:
31462
31463 • Config: include_archived
31464
31465 • Env Var: RCLONE_GPHOTOS_INCLUDE_ARCHIVED
31466
31467 • Type: bool
31468
31469 • Default: false
31470
31471 --gphotos-encoding
31472 The encoding for the backend.
31473
31474 See the encoding section in the overview (https://rclone.org/over‐
31475 view/#encoding) for more info.
31476
31477 Properties:
31478
31479 • Config: encoding
31480
31481 • Env Var: RCLONE_GPHOTOS_ENCODING
31482
31483 • Type: MultiEncoder
31484
31485 • Default: Slash,CrLf,InvalidUtf8,Dot
31486
31487 Limitations
31488 Only images and videos can be uploaded. If you attempt to upload non
31489 videos or images or formats that Google Photos doesn't understand,
31490 rclone will upload the file, then Google Photos will give an error when
31491 it is put turned into a media item.
31492
31493 Note that all media items uploaded to Google Photos through the API are
31494 stored in full resolution at "original quality" and will count towards
31495 your storage quota in your Google Account. The API does not offer a
31496 way to upload in "high quality" mode..
31497
31498 rclone about is not supported by the Google Photos backend. Backends
31499 without this capability cannot determine free space for an rclone mount
31500 or use policy mfs (most free space) as a member of an rclone union re‐
31501 mote.
31502
31503 See List of backends that do not support rclone about
31504 (https://rclone.org/overview/#optional-features) See rclone about
31505 (https://rclone.org/commands/rclone_about/)
31506
31507 Downloading Images
31508 When Images are downloaded this strips EXIF location (according to the
31509 docs and my tests). This is a limitation of the Google Photos API and
31510 is covered by bug #112096115 (https://issuetracker.google.com/is‐
31511 sues/112096115).
31512
31513 The current google API does not allow photos to be downloaded at origi‐
31514 nal resolution. This is very important if you are, for example, relying
31515 on "Google Photos" as a backup of your photos. You will not be able to
31516 use rclone to redownload original images. You could use 'google take‐
31517 out' to recover the original photos as a last resort
31518
31519 Downloading Videos
31520 When videos are downloaded they are downloaded in a really compressed
31521 version of the video compared to downloading it via the Google Photos
31522 web interface. This is covered by bug #113672044 (https://issuetrack‐
31523 er.google.com/issues/113672044).
31524
31525 Duplicates
31526 If a file name is duplicated in a directory then rclone will add the
31527 file ID into its name. So two files called file.jpg would then appear
31528 as file {123456}.jpg and file {ABCDEF}.jpg (the actual IDs are a lot
31529 longer alas!).
31530
31531 If you upload the same image (with the same binary data) twice then
31532 Google Photos will deduplicate it. However it will retain the filename
31533 from the first upload which may confuse rclone. For example if you up‐
31534 loaded an image to upload then uploaded the same image to album/my_al‐
31535 bum the filename of the image in album/my_album will be what it was up‐
31536 loaded with initially, not what you uploaded it with to album. In
31537 practise this shouldn't cause too many problems.
31538
31539 Modified time
31540 The date shown of media in Google Photos is the creation date as deter‐
31541 mined by the EXIF information, or the upload date if that is not known.
31542
31543 This is not changeable by rclone and is not the modification date of
31544 the media on local disk. This means that rclone cannot use the dates
31545 from Google Photos for syncing purposes.
31546
31547 Size
31548 The Google Photos API does not return the size of media. This means
31549 that when syncing to Google Photos, rclone can only do a file existence
31550 check.
31551
31552 It is possible to read the size of the media, but this needs an extra
31553 HTTP HEAD request per media item so is very slow and uses up a lot of
31554 transactions. This can be enabled with the --gphotos-read-size option
31555 or the read_size = true config parameter.
31556
31557 If you want to use the backend with rclone mount you may need to enable
31558 this flag (depending on your OS and application using the photos) oth‐
31559 erwise you may not be able to read media off the mount. You'll need to
31560 experiment to see if it works for you without the flag.
31561
31562 Albums
31563 Rclone can only upload files to albums it created. This is a limita‐
31564 tion of the Google Photos API (https://developers.google.com/photos/li‐
31565 brary/guides/manage-albums).
31566
31567 Rclone can remove files it uploaded from albums it created only.
31568
31569 Deleting files
31570 Rclone can remove files from albums it created, but note that the
31571 Google Photos API does not allow media to be deleted permanently so
31572 this media will still remain. See bug #109759781 (https://issuetrack‐
31573 er.google.com/issues/109759781).
31574
31575 Rclone cannot delete files anywhere except under album.
31576
31577 Deleting albums
31578 The Google Photos API does not support deleting albums - see bug
31579 #135714733 (https://issuetracker.google.com/issues/135714733).
31580
31582 Hasher is a special overlay backend to create remotes which handle
31583 checksums for other remotes. It's main functions include: - Emulate
31584 hash types unimplemented by backends - Cache checksums to help with
31585 slow hashing of large local or (S)FTP files - Warm up checksum cache
31586 from external SUM files
31587
31588 Getting started
31589 To use Hasher, first set up the underlying remote following the config‐
31590 uration instructions for that remote. You can also use a local path‐
31591 name instead of a remote. Check that your base remote is working.
31592
31593 Let's call the base remote myRemote:path here. Note that anything in‐
31594 side myRemote:path will be handled by hasher and anything outside
31595 won't. This means that if you are using a bucket based remote (S3, B2,
31596 Swift) then you should put the bucket in the remote s3:bucket.
31597
31598 Now proceed to interactive or manual configuration.
31599
31600 Interactive configuration
31601 Run rclone config:
31602
31603 No remotes found, make a new one?
31604 n) New remote
31605 s) Set configuration password
31606 q) Quit config
31607 n/s/q> n
31608 name> Hasher1
31609 Type of storage to configure.
31610 Choose a number from below, or type in your own value
31611 [snip]
31612 XX / Handle checksums for other remotes
31613 \ "hasher"
31614 [snip]
31615 Storage> hasher
31616 Remote to cache checksums for, like myremote:mypath.
31617 Enter a string value. Press Enter for the default ("").
31618 remote> myRemote:path
31619 Comma separated list of supported checksum types.
31620 Enter a string value. Press Enter for the default ("md5,sha1").
31621 hashsums> md5
31622 Maximum time to keep checksums in cache. 0 = no cache, off = cache forever.
31623 max_age> off
31624 Edit advanced config? (y/n)
31625 y) Yes
31626 n) No
31627 y/n> n
31628 Remote config
31629 --------------------
31630 [Hasher1]
31631 type = hasher
31632 remote = myRemote:path
31633 hashsums = md5
31634 max_age = off
31635 --------------------
31636 y) Yes this is OK
31637 e) Edit this remote
31638 d) Delete this remote
31639 y/e/d> y
31640
31641 Manual configuration
31642 Run rclone config path to see the path of current active config file,
31643 usually YOURHOME/.config/rclone/rclone.conf. Open it in your favorite
31644 text editor, find section for the base remote and create new section
31645 for hasher like in the following examples:
31646
31647 [Hasher1]
31648 type = hasher
31649 remote = myRemote:path
31650 hashes = md5
31651 max_age = off
31652
31653 [Hasher2]
31654 type = hasher
31655 remote = /local/path
31656 hashes = dropbox,sha1
31657 max_age = 24h
31658
31659 Hasher takes basically the following parameters: - remote is required,
31660 - hashes is a comma separated list of supported checksums (by default
31661 md5,sha1), - max_age - maximum time to keep a checksum value in the
31662 cache, 0 will disable caching completely, off will cache "forever"
31663 (that is until the files get changed).
31664
31665 Make sure the remote has : (colon) in. If you specify the remote with‐
31666 out a colon then rclone will use a local directory of that name. So if
31667 you use a remote of /local/path then rclone will handle hashes for that
31668 directory. If you use remote = name literally then rclone will put
31669 files in a directory called name located under current directory.
31670
31671 Usage
31672 Basic operations
31673 Now you can use it as Hasher2:subdir/file instead of base remote.
31674 Hasher will transparently update cache with new checksums when a file
31675 is fully read or overwritten, like:
31676
31677 rclone copy External:path/file Hasher:dest/path
31678
31679 rclone cat Hasher:path/to/file > /dev/null
31680
31681 The way to refresh all cached checksums (even unsupported by the base
31682 backend) for a subtree is to re-download all files in the subtree. For
31683 example, use hashsum --download using any supported hashsum on the com‐
31684 mand line (we just care to re-read):
31685
31686 rclone hashsum MD5 --download Hasher:path/to/subtree > /dev/null
31687
31688 rclone backend dump Hasher:path/to/subtree
31689
31690 You can print or drop hashsum cache using custom backend commands:
31691
31692 rclone backend dump Hasher:dir/subdir
31693
31694 rclone backend drop Hasher:
31695
31696 Pre-Seed from a SUM File
31697 Hasher supports two backend commands: generic SUM file import and
31698 faster but less consistent stickyimport.
31699
31700 rclone backend import Hasher:dir/subdir SHA1 /path/to/SHA1SUM [--checkers 4]
31701
31702 Instead of SHA1 it can be any hash supported by the remote. The last
31703 argument can point to either a local or an other-remote:path text file
31704 in SUM format. The command will parse the SUM file, then walk down the
31705 path given by the first argument, snapshot current fingerprints and
31706 fill in the cache entries correspondingly. - Paths in the SUM file are
31707 treated as relative to hasher:dir/subdir. - The command will not check
31708 that supplied values are correct. You must know what you are doing. -
31709 This is a one-time action. The SUM file will not get "attached" to the
31710 remote. Cache entries can still be overwritten later, should the ob‐
31711 ject's fingerprint change. - The tree walk can take long depending on
31712 the tree size. You can increase --checkers to make it faster. Or use
31713 stickyimport if you don't care about fingerprints and consistency.
31714
31715 rclone backend stickyimport hasher:path/to/data sha1 remote:/path/to/sum.sha1
31716
31717 stickyimport is similar to import but works much faster because it does
31718 not need to stat existing files and skips initial tree walk. Instead
31719 of binding cache entries to file fingerprints it creates sticky entries
31720 bound to the file name alone ignoring size, modification time etc.
31721 Such hash entries can be replaced only by purge, delete, backend drop
31722 or by full re-read/re-write of the files.
31723
31724 Configuration reference
31725 Standard options
31726 Here are the Standard options specific to hasher (Better checksums for
31727 other remotes).
31728
31729 --hasher-remote
31730 Remote to cache checksums for (e.g. myRemote:path).
31731
31732 Properties:
31733
31734 • Config: remote
31735
31736 • Env Var: RCLONE_HASHER_REMOTE
31737
31738 • Type: string
31739
31740 • Required: true
31741
31742 --hasher-hashes
31743 Comma separated list of supported checksum types.
31744
31745 Properties:
31746
31747 • Config: hashes
31748
31749 • Env Var: RCLONE_HASHER_HASHES
31750
31751 • Type: CommaSepList
31752
31753 • Default: md5,sha1
31754
31755 --hasher-max-age
31756 Maximum time to keep checksums in cache (0 = no cache, off = cache for‐
31757 ever).
31758
31759 Properties:
31760
31761 • Config: max_age
31762
31763 • Env Var: RCLONE_HASHER_MAX_AGE
31764
31765 • Type: Duration
31766
31767 • Default: off
31768
31769 Advanced options
31770 Here are the Advanced options specific to hasher (Better checksums for
31771 other remotes).
31772
31773 --hasher-auto-size
31774 Auto-update checksum for files smaller than this size (disabled by de‐
31775 fault).
31776
31777 Properties:
31778
31779 • Config: auto_size
31780
31781 • Env Var: RCLONE_HASHER_AUTO_SIZE
31782
31783 • Type: SizeSuffix
31784
31785 • Default: 0
31786
31787 Metadata
31788 Any metadata supported by the underlying remote is read and written.
31789
31790 See the metadata (https://rclone.org/docs/#metadata) docs for more in‐
31791 fo.
31792
31793 Backend commands
31794 Here are the commands specific to the hasher backend.
31795
31796 Run them with
31797
31798 rclone backend COMMAND remote:
31799
31800 The help below will explain what arguments each command takes.
31801
31802 See the backend (https://rclone.org/commands/rclone_backend/) command
31803 for more info on how to pass options and arguments.
31804
31805 These can be run on a running backend using the rc command backend/com‐
31806 mand (https://rclone.org/rc/#backend-command).
31807
31808 drop
31809 Drop cache
31810
31811 rclone backend drop remote: [options] [<arguments>+]
31812
31813 Completely drop checksum cache. Usage Example: rclone backend drop
31814 hasher:
31815
31816 dump
31817 Dump the database
31818
31819 rclone backend dump remote: [options] [<arguments>+]
31820
31821 Dump cache records covered by the current remote
31822
31823 fulldump
31824 Full dump of the database
31825
31826 rclone backend fulldump remote: [options] [<arguments>+]
31827
31828 Dump all cache records in the database
31829
31830 import
31831 Import a SUM file
31832
31833 rclone backend import remote: [options] [<arguments>+]
31834
31835 Amend hash cache from a SUM file and bind checksums to files by
31836 size/time. Usage Example: rclone backend import hasher:subdir md5
31837 /path/to/sum.md5
31838
31839 stickyimport
31840 Perform fast import of a SUM file
31841
31842 rclone backend stickyimport remote: [options] [<arguments>+]
31843
31844 Fill hash cache from a SUM file without verifying file fingerprints.
31845 Usage Example: rclone backend stickyimport hasher:subdir md5 re‐
31846 mote:path/to/sum.md5
31847
31848 Implementation details (advanced)
31849 This section explains how various rclone operations work on a hasher
31850 remote.
31851
31852 Disclaimer. This section describes current implementation which can
31853 change in future rclone versions!.
31854
31855 Hashsum command
31856 The rclone hashsum (or md5sum or sha1sum) command will:
31857
31858 1. if requested hash is supported by lower level, just pass it.
31859
31860 2. if object size is below auto_size then download object and calculate
31861 requested hashes on the fly.
31862
31863 3. if unsupported and the size is big enough, build object fingerprint
31864 (including size, modtime if supported, first-found other hash if
31865 any).
31866
31867 4. if the strict match is found in cache for the requested remote, re‐
31868 turn the stored hash.
31869
31870 5. if remote found but fingerprint mismatched, then purge the entry and
31871 proceed to step 6.
31872
31873 6. if remote not found or had no requested hash type or after step 5:
31874 download object, calculate all supported hashes on the fly and store
31875 in cache; return requested hash.
31876
31877 Other operations
31878 • whenever a file is uploaded or downloaded in full, capture the stream
31879 to calculate all supported hashes on the fly and update database
31880
31881 • server-side move will update keys of existing cache entries
31882
31883 • deletefile will remove a single cache entry
31884
31885 • purge will remove all cache entries under the purged path
31886
31887 Note that setting max_age = 0 will disable checksum caching completely.
31888
31889 If you set max_age = off, checksums in cache will never age, unless you
31890 fully rewrite or delete the file.
31891
31892 Cache storage
31893 Cached checksums are stored as bolt database files under rclone cache
31894 directory, usually ~/.cache/rclone/kv/. Databases are maintained one
31895 per base backend, named like BaseRemote~hasher.bolt. Checksums for
31896 multiple alias-es into a single base backend will be stored in the sin‐
31897 gle database. All local paths are treated as aliases into the local
31898 backend (unless crypted or chunked) and stored in
31899 ~/.cache/rclone/kv/local~hasher.bolt. Databases can be shared between
31900 multiple rclone processes.
31901
31903 HDFS (https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-
31904 hdfs/HdfsDesign.html) is a distributed file-system, part of the Apache
31905 Hadoop (https://hadoop.apache.org/) framework.
31906
31907 Paths are specified as remote: or remote:path/to/dir.
31908
31909 Configuration
31910 Here is an example of how to make a remote called remote. First run:
31911
31912 rclone config
31913
31914 This will guide you through an interactive setup process:
31915
31916 No remotes found, make a new one?
31917 n) New remote
31918 s) Set configuration password
31919 q) Quit config
31920 n/s/q> n
31921 name> remote
31922 Type of storage to configure.
31923 Enter a string value. Press Enter for the default ("").
31924 Choose a number from below, or type in your own value
31925 [skip]
31926 XX / Hadoop distributed file system
31927 \ "hdfs"
31928 [skip]
31929 Storage> hdfs
31930 ** See help for hdfs backend at: https://rclone.org/hdfs/ **
31931
31932 hadoop name node and port
31933 Enter a string value. Press Enter for the default ("").
31934 Choose a number from below, or type in your own value
31935 1 / Connect to host namenode at port 8020
31936 \ "namenode:8020"
31937 namenode> namenode.hadoop:8020
31938 hadoop user name
31939 Enter a string value. Press Enter for the default ("").
31940 Choose a number from below, or type in your own value
31941 1 / Connect to hdfs as root
31942 \ "root"
31943 username> root
31944 Edit advanced config? (y/n)
31945 y) Yes
31946 n) No (default)
31947 y/n> n
31948 Remote config
31949 --------------------
31950 [remote]
31951 type = hdfs
31952 namenode = namenode.hadoop:8020
31953 username = root
31954 --------------------
31955 y) Yes this is OK (default)
31956 e) Edit this remote
31957 d) Delete this remote
31958 y/e/d> y
31959 Current remotes:
31960
31961 Name Type
31962 ==== ====
31963 hadoop hdfs
31964
31965 e) Edit existing remote
31966 n) New remote
31967 d) Delete remote
31968 r) Rename remote
31969 c) Copy remote
31970 s) Set configuration password
31971 q) Quit config
31972 e/n/d/r/c/s/q> q
31973
31974 This remote is called remote and can now be used like this
31975
31976 See all the top level directories
31977
31978 rclone lsd remote:
31979
31980 List the contents of a directory
31981
31982 rclone ls remote:directory
31983
31984 Sync the remote directory to /home/local/directory, deleting any excess
31985 files.
31986
31987 rclone sync -i remote:directory /home/local/directory
31988
31989 Setting up your own HDFS instance for testing
31990 You may start with a manual setup (https://hadoop.apache.org/docs/sta‐
31991 ble/hadoop-project-dist/hadoop-common/SingleCluster.html) or use the
31992 docker image from the tests:
31993
31994 If you want to build the docker image
31995
31996 git clone https://github.com/rclone/rclone.git
31997 cd rclone/fstest/testserver/images/test-hdfs
31998 docker build --rm -t rclone/test-hdfs .
31999
32000 Or you can just use the latest one pushed
32001
32002 docker run --rm --name "rclone-hdfs" -p 127.0.0.1:9866:9866 -p 127.0.0.1:8020:8020 --hostname "rclone-hdfs" rclone/test-hdfs
32003
32004 NB it need few seconds to startup.
32005
32006 For this docker image the remote needs to be configured like this:
32007
32008 [remote]
32009 type = hdfs
32010 namenode = 127.0.0.1:8020
32011 username = root
32012
32013 You can stop this image with docker kill rclone-hdfs (NB it does not
32014 use volumes, so all data uploaded will be lost.)
32015
32016 Modified time
32017 Time accurate to 1 second is stored.
32018
32019 Checksum
32020 No checksums are implemented.
32021
32022 Usage information
32023 You can use the rclone about remote: command which will display
32024 filesystem size and current usage.
32025
32026 Restricted filename characters
32027 In addition to the default restricted characters set
32028 (https://rclone.org/overview/#restricted-characters) the following
32029 characters are also replaced:
32030
32031 Character Value Replacement
32032 ────────────────────────────────
32033 : 0x3A :
32034
32035 Invalid UTF-8 bytes will also be replaced (https://rclone.org/over‐
32036 view/#invalid-utf8).
32037
32038 Standard options
32039 Here are the Standard options specific to hdfs (Hadoop distributed file
32040 system).
32041
32042 --hdfs-namenode
32043 Hadoop name node and port.
32044
32045 E.g. "namenode:8020" to connect to host namenode at port 8020.
32046
32047 Properties:
32048
32049 • Config: namenode
32050
32051 • Env Var: RCLONE_HDFS_NAMENODE
32052
32053 • Type: string
32054
32055 • Required: true
32056
32057 --hdfs-username
32058 Hadoop user name.
32059
32060 Properties:
32061
32062 • Config: username
32063
32064 • Env Var: RCLONE_HDFS_USERNAME
32065
32066 • Type: string
32067
32068 • Required: false
32069
32070 • Examples:
32071
32072 • "root"
32073
32074 • Connect to hdfs as root.
32075
32076 Advanced options
32077 Here are the Advanced options specific to hdfs (Hadoop distributed file
32078 system).
32079
32080 --hdfs-service-principal-name
32081 Kerberos service principal name for the namenode.
32082
32083 Enables KERBEROS authentication. Specifies the Service Principal Name
32084 (SERVICE/FQDN) for the namenode. E.g. "hdfs/namenode.hadoop.docker"
32085 for namenode running as service 'hdfs' with FQDN 'namenode.hadoop.dock‐
32086 er'.
32087
32088 Properties:
32089
32090 • Config: service_principal_name
32091
32092 • Env Var: RCLONE_HDFS_SERVICE_PRINCIPAL_NAME
32093
32094 • Type: string
32095
32096 • Required: false
32097
32098 --hdfs-data-transfer-protection
32099 Kerberos data transfer protection: authentication|integrity|privacy.
32100
32101 Specifies whether or not authentication, data signature integrity
32102 checks, and wire encryption is required when communicating the the
32103 datanodes. Possible values are 'authentication', 'integrity' and 'pri‐
32104 vacy'. Used only with KERBEROS enabled.
32105
32106 Properties:
32107
32108 • Config: data_transfer_protection
32109
32110 • Env Var: RCLONE_HDFS_DATA_TRANSFER_PROTECTION
32111
32112 • Type: string
32113
32114 • Required: false
32115
32116 • Examples:
32117
32118 • "privacy"
32119
32120 • Ensure authentication, integrity and encryption enabled.
32121
32122 --hdfs-encoding
32123 The encoding for the backend.
32124
32125 See the encoding section in the overview (https://rclone.org/over‐
32126 view/#encoding) for more info.
32127
32128 Properties:
32129
32130 • Config: encoding
32131
32132 • Env Var: RCLONE_HDFS_ENCODING
32133
32134 • Type: MultiEncoder
32135
32136 • Default: Slash,Colon,Del,Ctl,InvalidUtf8,Dot
32137
32138 Limitations
32139 • No server-side Move or DirMove.
32140
32141 • Checksums not implemented.
32142
32144 Paths are specified as remote:path
32145
32146 Paths may be as deep as required, e.g. remote:directory/subdirectory.
32147
32148 The initial setup for hidrive involves getting a token from HiDrive
32149 which you need to do in your browser. rclone config walks you through
32150 it.
32151
32152 Configuration
32153 Here is an example of how to make a remote called remote. First run:
32154
32155 rclone config
32156
32157 This will guide you through an interactive setup process:
32158
32159 No remotes found - make a new one
32160 n) New remote
32161 s) Set configuration password
32162 q) Quit config
32163 n/s/q> n
32164 name> remote
32165 Type of storage to configure.
32166 Choose a number from below, or type in your own value
32167 [snip]
32168 XX / HiDrive
32169 \ "hidrive"
32170 [snip]
32171 Storage> hidrive
32172 OAuth Client Id - Leave blank normally.
32173 client_id>
32174 OAuth Client Secret - Leave blank normally.
32175 client_secret>
32176 Access permissions that rclone should use when requesting access from HiDrive.
32177 Leave blank normally.
32178 scope_access>
32179 Edit advanced config?
32180 y/n> n
32181 Use auto config?
32182 y/n> y
32183 If your browser doesn't open automatically go to the following link: http://127.0.0.1:53682/auth?state=xxxxxxxxxxxxxxxxxxxxxx
32184 Log in and authorize rclone for access
32185 Waiting for code...
32186 Got code
32187 --------------------
32188 [remote]
32189 type = hidrive
32190 token = {"access_token":"xxxxxxxxxxxxxxxxxxxx","token_type":"Bearer","refresh_token":"xxxxxxxxxxxxxxxxxxxxxxx","expiry":"xxxxxxxxxxxxxxxxxxxxxxx"}
32191 --------------------
32192 y) Yes this is OK (default)
32193 e) Edit this remote
32194 d) Delete this remote
32195 y/e/d> y
32196
32197 You should be aware that OAuth-tokens can be used to access your ac‐
32198 count and hence should not be shared with other persons. See the below
32199 section for more information.
32200
32201 See the remote setup docs (https://rclone.org/remote_setup/) for how to
32202 set it up on a machine with no Internet browser available.
32203
32204 Note that rclone runs a webserver on your local machine to collect the
32205 token as returned from HiDrive. This only runs from the moment it
32206 opens your browser to the moment you get back the verification code.
32207 The webserver runs on http://127.0.0.1:53682/. If local port 53682 is
32208 protected by a firewall you may need to temporarily unblock the fire‐
32209 wall to complete authorization.
32210
32211 Once configured you can then use rclone like this,
32212
32213 List directories in top level of your HiDrive root folder
32214
32215 rclone lsd remote:
32216
32217 List all the files in your HiDrive filesystem
32218
32219 rclone ls remote:
32220
32221 To copy a local directory to a HiDrive directory called backup
32222
32223 rclone copy /home/source remote:backup
32224
32225 Keeping your tokens safe
32226 Any OAuth-tokens will be stored by rclone in the remote's configuration
32227 file as unencrypted text. Anyone can use a valid refresh-token to ac‐
32228 cess your HiDrive filesystem without knowing your password. Therefore
32229 you should make sure no one else can access your configuration.
32230
32231 It is possible to encrypt rclone's configuration file. You can find
32232 information on securing your configuration file by viewing the configu‐
32233 ration encryption docs (https://rclone.org/docs/#configuration-encryp‐
32234 tion).
32235
32236 Invalid refresh token
32237 As can be verified here (https://developer.hidrive.com/basics-flows/),
32238 each refresh_token (for Native Applications) is valid for 60 days. If
32239 used to access HiDrivei, its validity will be automatically extended.
32240
32241 This means that if you
32242
32243 • Don't use the HiDrive remote for 60 days
32244
32245 then rclone will return an error which includes a text that implies the
32246 refresh token is invalid or expired.
32247
32248 To fix this you will need to authorize rclone to access your HiDrive
32249 account again.
32250
32251 Using
32252
32253 rclone config reconnect remote:
32254
32255 the process is very similar to the process of initial setup exemplified
32256 before.
32257
32258 Modified time and hashes
32259 HiDrive allows modification times to be set on objects accurate to 1
32260 second.
32261
32262 HiDrive supports its own hash type (https://stat‐
32263 ic.hidrive.com/dev/0001) which is used to verify the integrity of file
32264 contents after successful transfers.
32265
32266 Restricted filename characters
32267 HiDrive cannot store files or folders that include / (0x2F) or null-
32268 bytes (0x00) in their name. Any other characters can be used in the
32269 names of files or folders. Additionally, files or folders cannot be
32270 named either of the following: . or ..
32271
32272 Therefore rclone will automatically replace these characters, if files
32273 or folders are stored or accessed with such names.
32274
32275 You can read about how this filename encoding works in general here.
32276
32277 Keep in mind that HiDrive only supports file or folder names with a
32278 length of 255 characters or less.
32279
32280 Transfers
32281 HiDrive limits file sizes per single request to a maximum of 2 GiB. To
32282 allow storage of larger files and allow for better upload performance,
32283 the hidrive backend will use a chunked transfer for files larger than
32284 96 MiB. Rclone will upload multiple parts/chunks of the file at the
32285 same time. Chunks in the process of being uploaded are buffered in
32286 memory, so you may want to restrict this behaviour on systems with lim‐
32287 ited resources.
32288
32289 You can customize this behaviour using the following options:
32290
32291 • chunk_size: size of file parts
32292
32293 • upload_cutoff: files larger or equal to this in size will use a chun‐
32294 ked transfer
32295
32296 • upload_concurrency: number of file-parts to upload at the same time
32297
32298 See the below section about configuration options for more details.
32299
32300 Root folder
32301 You can set the root folder for rclone. This is the directory that
32302 rclone considers to be the root of your HiDrive.
32303
32304 Usually, you will leave this blank, and rclone will use the root of the
32305 account.
32306
32307 However, you can set this to restrict rclone to a specific folder hier‐
32308 archy.
32309
32310 This works by prepending the contents of the root_prefix option to any
32311 paths accessed by rclone. For example, the following two ways to ac‐
32312 cess the home directory are equivalent:
32313
32314 rclone lsd --hidrive-root-prefix="/users/test/" remote:path
32315
32316 rclone lsd remote:/users/test/path
32317
32318 See the below section about configuration options for more details.
32319
32320 Directory member count
32321 By default, rclone will know the number of directory members contained
32322 in a directory. For example, rclone lsd uses this information.
32323
32324 The acquisition of this information will result in additional time
32325 costs for HiDrive's API. When dealing with large directory structures,
32326 it may be desirable to circumvent this time cost, especially when this
32327 information is not explicitly needed. For this, the disable_fetch‐
32328 ing_member_count option can be used.
32329
32330 See the below section about configuration options for more details.
32331
32332 Standard options
32333 Here are the Standard options specific to hidrive (HiDrive).
32334
32335 --hidrive-client-id
32336 OAuth Client Id.
32337
32338 Leave blank normally.
32339
32340 Properties:
32341
32342 • Config: client_id
32343
32344 • Env Var: RCLONE_HIDRIVE_CLIENT_ID
32345
32346 • Type: string
32347
32348 • Required: false
32349
32350 --hidrive-client-secret
32351 OAuth Client Secret.
32352
32353 Leave blank normally.
32354
32355 Properties:
32356
32357 • Config: client_secret
32358
32359 • Env Var: RCLONE_HIDRIVE_CLIENT_SECRET
32360
32361 • Type: string
32362
32363 • Required: false
32364
32365 --hidrive-scope-access
32366 Access permissions that rclone should use when requesting access from
32367 HiDrive.
32368
32369 Properties:
32370
32371 • Config: scope_access
32372
32373 • Env Var: RCLONE_HIDRIVE_SCOPE_ACCESS
32374
32375 • Type: string
32376
32377 • Default: "rw"
32378
32379 • Examples:
32380
32381 • "rw"
32382
32383 • Read and write access to resources.
32384
32385 • "ro"
32386
32387 • Read-only access to resources.
32388
32389 Advanced options
32390 Here are the Advanced options specific to hidrive (HiDrive).
32391
32392 --hidrive-token
32393 OAuth Access Token as a JSON blob.
32394
32395 Properties:
32396
32397 • Config: token
32398
32399 • Env Var: RCLONE_HIDRIVE_TOKEN
32400
32401 • Type: string
32402
32403 • Required: false
32404
32405 --hidrive-auth-url
32406 Auth server URL.
32407
32408 Leave blank to use the provider defaults.
32409
32410 Properties:
32411
32412 • Config: auth_url
32413
32414 • Env Var: RCLONE_HIDRIVE_AUTH_URL
32415
32416 • Type: string
32417
32418 • Required: false
32419
32420 --hidrive-token-url
32421 Token server url.
32422
32423 Leave blank to use the provider defaults.
32424
32425 Properties:
32426
32427 • Config: token_url
32428
32429 • Env Var: RCLONE_HIDRIVE_TOKEN_URL
32430
32431 • Type: string
32432
32433 • Required: false
32434
32435 --hidrive-scope-role
32436 User-level that rclone should use when requesting access from HiDrive.
32437
32438 Properties:
32439
32440 • Config: scope_role
32441
32442 • Env Var: RCLONE_HIDRIVE_SCOPE_ROLE
32443
32444 • Type: string
32445
32446 • Default: "user"
32447
32448 • Examples:
32449
32450 • "user"
32451
32452 • User-level access to management permissions.
32453
32454 • This will be sufficient in most cases.
32455
32456 • "admin"
32457
32458 • Extensive access to management permissions.
32459
32460 • "owner"
32461
32462 • Full access to management permissions.
32463
32464 --hidrive-root-prefix
32465 The root/parent folder for all paths.
32466
32467 Fill in to use the specified folder as the parent for all paths given
32468 to the remote. This way rclone can use any folder as its starting
32469 point.
32470
32471 Properties:
32472
32473 • Config: root_prefix
32474
32475 • Env Var: RCLONE_HIDRIVE_ROOT_PREFIX
32476
32477 • Type: string
32478
32479 • Default: "/"
32480
32481 • Examples:
32482
32483 • "/"
32484
32485 • The topmost directory accessible by rclone.
32486
32487 • This will be equivalent with "root" if rclone uses a regular
32488 HiDrive user account.
32489
32490 • "root"
32491
32492 • The topmost directory of the HiDrive user account
32493
32494 • ""
32495
32496 • This specifies that there is no root-prefix for your paths.
32497
32498 • When using this you will always need to specify paths to this re‐
32499 mote with a valid parent e.g. "remote:/path/to/dir" or "re‐
32500 mote:root/path/to/dir".
32501
32502 --hidrive-endpoint
32503 Endpoint for the service.
32504
32505 This is the URL that API-calls will be made to.
32506
32507 Properties:
32508
32509 • Config: endpoint
32510
32511 • Env Var: RCLONE_HIDRIVE_ENDPOINT
32512
32513 • Type: string
32514
32515 • Default: "https://api.hidrive.strato.com/2.1"
32516
32517 --hidrive-disable-fetching-member-count
32518 Do not fetch number of objects in directories unless it is absolutely
32519 necessary.
32520
32521 Requests may be faster if the number of objects in subdirectories is
32522 not fetched.
32523
32524 Properties:
32525
32526 • Config: disable_fetching_member_count
32527
32528 • Env Var: RCLONE_HIDRIVE_DISABLE_FETCHING_MEMBER_COUNT
32529
32530 • Type: bool
32531
32532 • Default: false
32533
32534 --hidrive-chunk-size
32535 Chunksize for chunked uploads.
32536
32537 Any files larger than the configured cutoff (or files of unknown size)
32538 will be uploaded in chunks of this size.
32539
32540 The upper limit for this is 2147483647 bytes (about 2.000Gi). That is
32541 the maximum amount of bytes a single upload-operation will support.
32542 Setting this above the upper limit or to a negative value will cause
32543 uploads to fail.
32544
32545 Setting this to larger values may increase the upload speed at the cost
32546 of using more memory. It can be set to smaller values smaller to save
32547 on memory.
32548
32549 Properties:
32550
32551 • Config: chunk_size
32552
32553 • Env Var: RCLONE_HIDRIVE_CHUNK_SIZE
32554
32555 • Type: SizeSuffix
32556
32557 • Default: 48Mi
32558
32559 --hidrive-upload-cutoff
32560 Cutoff/Threshold for chunked uploads.
32561
32562 Any files larger than this will be uploaded in chunks of the configured
32563 chunksize.
32564
32565 The upper limit for this is 2147483647 bytes (about 2.000Gi). That is
32566 the maximum amount of bytes a single upload-operation will support.
32567 Setting this above the upper limit will cause uploads to fail.
32568
32569 Properties:
32570
32571 • Config: upload_cutoff
32572
32573 • Env Var: RCLONE_HIDRIVE_UPLOAD_CUTOFF
32574
32575 • Type: SizeSuffix
32576
32577 • Default: 96Mi
32578
32579 --hidrive-upload-concurrency
32580 Concurrency for chunked uploads.
32581
32582 This is the upper limit for how many transfers for the same file are
32583 running concurrently. Setting this above to a value smaller than 1
32584 will cause uploads to deadlock.
32585
32586 If you are uploading small numbers of large files over high-speed links
32587 and these uploads do not fully utilize your bandwidth, then increasing
32588 this may help to speed up the transfers.
32589
32590 Properties:
32591
32592 • Config: upload_concurrency
32593
32594 • Env Var: RCLONE_HIDRIVE_UPLOAD_CONCURRENCY
32595
32596 • Type: int
32597
32598 • Default: 4
32599
32600 --hidrive-encoding
32601 The encoding for the backend.
32602
32603 See the encoding section in the overview (https://rclone.org/over‐
32604 view/#encoding) for more info.
32605
32606 Properties:
32607
32608 • Config: encoding
32609
32610 • Env Var: RCLONE_HIDRIVE_ENCODING
32611
32612 • Type: MultiEncoder
32613
32614 • Default: Slash,Dot
32615
32616 Limitations
32617 Symbolic links
32618 HiDrive is able to store symbolic links (symlinks) by design, for exam‐
32619 ple, when unpacked from a zip archive.
32620
32621 There exists no direct mechanism to manage native symlinks in remotes.
32622 As such this implementation has chosen to ignore any native symlinks
32623 present in the remote. rclone will not be able to access or show any
32624 symlinks stored in the hidrive-remote. This means symlinks cannot be
32625 individually removed, copied, or moved, except when removing, copying,
32626 or moving the parent folder.
32627
32628 This does not affect the .rclonelink-files that rclone uses to encode
32629 and store symbolic links.
32630
32631 Sparse files
32632 It is possible to store sparse files in HiDrive.
32633
32634 Note that copying a sparse file will expand the holes into null-byte
32635 (0x00) regions that will then consume disk space. Likewise, when down‐
32636 loading a sparse file, the resulting file will have null-byte regions
32637 in the place of file holes.
32638
32640 The HTTP remote is a read only remote for reading files of a webserver.
32641 The webserver should provide file listings which rclone will read and
32642 turn into a remote. This has been tested with common webservers such
32643 as Apache/Nginx/Caddy and will likely work with file listings from most
32644 web servers. (If it doesn't then please file an issue, or send a pull
32645 request!)
32646
32647 Paths are specified as remote: or remote:path.
32648
32649 The remote: represents the configured url, and any path following it
32650 will be resolved relative to this url, according to the URL standard.
32651 This means with remote url https://beta.rclone.org/branch and path fix,
32652 the resolved URL will be https://beta.rclone.org/branch/fix, while with
32653 path /fix the resolved URL will be https://beta.rclone.org/fix as the
32654 absolute path is resolved from the root of the domain.
32655
32656 If the path following the remote: ends with / it will be assumed to
32657 point to a directory. If the path does not end with /, then a HEAD re‐
32658 quest is sent and the response used to decide if it it is treated as a
32659 file or a directory (run with -vv to see details). When --http-no-head
32660 is specified, a path without ending / is always assumed to be a file.
32661 If rclone incorrectly assumes the path is a file, the solution is to
32662 specify the path with ending /. When you know the path is a directory,
32663 ending it with / is always better as it avoids the initial HEAD re‐
32664 quest.
32665
32666 To just download a single file it is easier to use copyurl
32667 (https://rclone.org/commands/rclone_copyurl/).
32668
32669 Configuration
32670 Here is an example of how to make a remote called remote. First run:
32671
32672 rclone config
32673
32674 This will guide you through an interactive setup process:
32675
32676 No remotes found, make a new one?
32677 n) New remote
32678 s) Set configuration password
32679 q) Quit config
32680 n/s/q> n
32681 name> remote
32682 Type of storage to configure.
32683 Choose a number from below, or type in your own value
32684 [snip]
32685 XX / HTTP
32686 \ "http"
32687 [snip]
32688 Storage> http
32689 URL of http host to connect to
32690 Choose a number from below, or type in your own value
32691 1 / Connect to example.com
32692 \ "https://example.com"
32693 url> https://beta.rclone.org
32694 Remote config
32695 --------------------
32696 [remote]
32697 url = https://beta.rclone.org
32698 --------------------
32699 y) Yes this is OK
32700 e) Edit this remote
32701 d) Delete this remote
32702 y/e/d> y
32703 Current remotes:
32704
32705 Name Type
32706 ==== ====
32707 remote http
32708
32709 e) Edit existing remote
32710 n) New remote
32711 d) Delete remote
32712 r) Rename remote
32713 c) Copy remote
32714 s) Set configuration password
32715 q) Quit config
32716 e/n/d/r/c/s/q> q
32717
32718 This remote is called remote and can now be used like this
32719
32720 See all the top level directories
32721
32722 rclone lsd remote:
32723
32724 List the contents of a directory
32725
32726 rclone ls remote:directory
32727
32728 Sync the remote directory to /home/local/directory, deleting any excess
32729 files.
32730
32731 rclone sync -i remote:directory /home/local/directory
32732
32733 Read only
32734 This remote is read only - you can't upload files to an HTTP server.
32735
32736 Modified time
32737 Most HTTP servers store time accurate to 1 second.
32738
32739 Checksum
32740 No checksums are stored.
32741
32742 Usage without a config file
32743 Since the http remote only has one config parameter it is easy to use
32744 without a config file:
32745
32746 rclone lsd --http-url https://beta.rclone.org :http:
32747
32748 or:
32749
32750 rclone lsd :http,url='https://beta.rclone.org':
32751
32752 Standard options
32753 Here are the Standard options specific to http (HTTP).
32754
32755 --http-url
32756 URL of HTTP host to connect to.
32757
32758 E.g. "https://example.com", or "https://user:pass@example.com" to use
32759 a username and password.
32760
32761 Properties:
32762
32763 • Config: url
32764
32765 • Env Var: RCLONE_HTTP_URL
32766
32767 • Type: string
32768
32769 • Required: true
32770
32771 Advanced options
32772 Here are the Advanced options specific to http (HTTP).
32773
32774 --http-headers
32775 Set HTTP headers for all transactions.
32776
32777 Use this to set additional HTTP headers for all transactions.
32778
32779 The input format is comma separated list of key,value pairs. Standard
32780 CSV encoding (https://godoc.org/encoding/csv) may be used.
32781
32782 For example, to set a Cookie use 'Cookie,name=value', or '"Cook‐
32783 ie","name=value"'.
32784
32785 You can set multiple headers, e.g. '"Cookie","name=value","Authoriza‐
32786 tion","xxx"'.
32787
32788 Properties:
32789
32790 • Config: headers
32791
32792 • Env Var: RCLONE_HTTP_HEADERS
32793
32794 • Type: CommaSepList
32795
32796 • Default:
32797
32798 --http-no-slash
32799 Set this if the site doesn't end directories with /.
32800
32801 Use this if your target website does not use / on the end of directo‐
32802 ries.
32803
32804 A / on the end of a path is how rclone normally tells the difference
32805 between files and directories. If this flag is set, then rclone will
32806 treat all files with Content-Type: text/html as directories and read
32807 URLs from them rather than downloading them.
32808
32809 Note that this may cause rclone to confuse genuine HTML files with di‐
32810 rectories.
32811
32812 Properties:
32813
32814 • Config: no_slash
32815
32816 • Env Var: RCLONE_HTTP_NO_SLASH
32817
32818 • Type: bool
32819
32820 • Default: false
32821
32822 --http-no-head
32823 Don't use HEAD requests.
32824
32825 HEAD requests are mainly used to find file sizes in dir listing. If
32826 your site is being very slow to load then you can try this option.
32827 Normally rclone does a HEAD request for each potential file in a direc‐
32828 tory listing to:
32829
32830 • find its size
32831
32832 • check it really exists
32833
32834 • check to see if it is a directory
32835
32836 If you set this option, rclone will not do the HEAD request. This will
32837 mean that directory listings are much quicker, but rclone won't have
32838 the times or sizes of any files, and some files that don't exist may be
32839 in the listing.
32840
32841 Properties:
32842
32843 • Config: no_head
32844
32845 • Env Var: RCLONE_HTTP_NO_HEAD
32846
32847 • Type: bool
32848
32849 • Default: false
32850
32851 Limitations
32852 rclone about is not supported by the HTTP backend. Backends without
32853 this capability cannot determine free space for an rclone mount or use
32854 policy mfs (most free space) as a member of an rclone union remote.
32855
32856 See List of backends that do not support rclone about
32857 (https://rclone.org/overview/#optional-features) and rclone about
32858 (https://rclone.org/commands/rclone_about/)
32859
32861 The Internet Archive backend utilizes Items on archive.org (https://ar‐
32862 chive.org/)
32863
32864 Refer to IAS3 API documentation (https://archive.org/ser‐
32865 vices/docs/api/ias3.html) for the API this backend uses.
32866
32867 Paths are specified as remote:bucket (or remote: for the lsd command.)
32868 You may put subdirectories in too, e.g. remote:item/path/to/dir.
32869
32870 Unlike S3, listing up all items uploaded by you isn't supported.
32871
32872 Once you have made a remote, you can use it like this:
32873
32874 Make a new item
32875
32876 rclone mkdir remote:item
32877
32878 List the contents of a item
32879
32880 rclone ls remote:item
32881
32882 Sync /home/local/directory to the remote item, deleting any excess
32883 files in the item.
32884
32885 rclone sync -i /home/local/directory remote:item
32886
32887 Notes
32888 Because of Internet Archive's architecture, it enqueues write opera‐
32889 tions (and extra post-processings) in a per-item queue. You can check
32890 item's queue at https://catalogd.archive.org/history/item-name-here .
32891 Because of that, all uploads/deletes will not show up immediately and
32892 takes some time to be available. The per-item queue is enqueued to an
32893 another queue, Item Deriver Queue. You can check the status of Item
32894 Deriver Queue here. (https://catalogd.archive.org/catalog.php?wherea‐
32895 mi=1) This queue has a limit, and it may block you from uploading, or
32896 even deleting. You should avoid uploading a lot of small files for
32897 better behavior.
32898
32899 You can optionally wait for the server's processing to finish, by set‐
32900 ting non-zero value to wait_archive key. By making it wait, rclone can
32901 do normal file comparison. Make sure to set a large enough value (e.g.
32902 30m0s for smaller files) as it can take a long time depending on serv‐
32903 er's queue.
32904
32905 About metadata
32906 This backend supports setting, updating and reading metadata of each
32907 file. The metadata will appear as file metadata on Internet Archive.
32908 However, some fields are reserved by both Internet Archive and rclone.
32909
32910 The following are reserved by Internet Archive: - name - source - size
32911 - md5 - crc32 - sha1 - format - old_version - viruscheck - summation
32912
32913 Trying to set values to these keys is ignored with a warning. Only
32914 setting mtime is an exception. Doing so make it the identical behavior
32915 as setting ModTime.
32916
32917 rclone reserves all the keys starting with rclone-. Setting value for
32918 these keys will give you warnings, but values are set according to re‐
32919 quest.
32920
32921 If there are multiple values for a key, only the first one is returned.
32922 This is a limitation of rclone, that supports one value per one key.
32923 It can be triggered when you did a server-side copy.
32924
32925 Reading metadata will also provide custom (non-standard nor reserved)
32926 ones.
32927
32928 Configuration
32929 Here is an example of making an internetarchive configuration. Most
32930 applies to the other providers as well, any differences are described
32931 below.
32932
32933 First run
32934
32935 rclone config
32936
32937 This will guide you through an interactive setup process.
32938
32939 No remotes found, make a new one?
32940 n) New remote
32941 s) Set configuration password
32942 q) Quit config
32943 n/s/q> n
32944 name> remote
32945 Option Storage.
32946 Type of storage to configure.
32947 Choose a number from below, or type in your own value.
32948 XX / InternetArchive Items
32949 \ (internetarchive)
32950 Storage> internetarchive
32951 Option access_key_id.
32952 IAS3 Access Key.
32953 Leave blank for anonymous access.
32954 You can find one here: https://archive.org/account/s3.php
32955 Enter a value. Press Enter to leave empty.
32956 access_key_id> XXXX
32957 Option secret_access_key.
32958 IAS3 Secret Key (password).
32959 Leave blank for anonymous access.
32960 Enter a value. Press Enter to leave empty.
32961 secret_access_key> XXXX
32962 Edit advanced config?
32963 y) Yes
32964 n) No (default)
32965 y/n> y
32966 Option endpoint.
32967 IAS3 Endpoint.
32968 Leave blank for default value.
32969 Enter a string value. Press Enter for the default (https://s3.us.archive.org).
32970 endpoint>
32971 Option front_endpoint.
32972 Host of InternetArchive Frontend.
32973 Leave blank for default value.
32974 Enter a string value. Press Enter for the default (https://archive.org).
32975 front_endpoint>
32976 Option disable_checksum.
32977 Don't store MD5 checksum with object metadata.
32978 Normally rclone will calculate the MD5 checksum of the input before
32979 uploading it so it can ask the server to check the object against checksum.
32980 This is great for data integrity checking but can cause long delays for
32981 large files to start uploading.
32982 Enter a boolean value (true or false). Press Enter for the default (true).
32983 disable_checksum> true
32984 Option encoding.
32985 The encoding for the backend.
32986 See the [encoding section in the overview](https://rclone.org/overview/#encoding) for more info.
32987 Enter a encoder.MultiEncoder value. Press Enter for the default (Slash,Question,Hash,Percent,Del,Ctl,InvalidUtf8,Dot).
32988 encoding>
32989 Edit advanced config?
32990 y) Yes
32991 n) No (default)
32992 y/n> n
32993 --------------------
32994 [remote]
32995 type = internetarchive
32996 access_key_id = XXXX
32997 secret_access_key = XXXX
32998 --------------------
32999 y) Yes this is OK (default)
33000 e) Edit this remote
33001 d) Delete this remote
33002 y/e/d> y
33003
33004 Standard options
33005 Here are the Standard options specific to internetarchive (Internet Ar‐
33006 chive).
33007
33008 --internetarchive-access-key-id
33009 IAS3 Access Key.
33010
33011 Leave blank for anonymous access. You can find one here: https://ar‐
33012 chive.org/account/s3.php
33013
33014 Properties:
33015
33016 • Config: access_key_id
33017
33018 • Env Var: RCLONE_INTERNETARCHIVE_ACCESS_KEY_ID
33019
33020 • Type: string
33021
33022 • Required: false
33023
33024 --internetarchive-secret-access-key
33025 IAS3 Secret Key (password).
33026
33027 Leave blank for anonymous access.
33028
33029 Properties:
33030
33031 • Config: secret_access_key
33032
33033 • Env Var: RCLONE_INTERNETARCHIVE_SECRET_ACCESS_KEY
33034
33035 • Type: string
33036
33037 • Required: false
33038
33039 Advanced options
33040 Here are the Advanced options specific to internetarchive (Internet Ar‐
33041 chive).
33042
33043 --internetarchive-endpoint
33044 IAS3 Endpoint.
33045
33046 Leave blank for default value.
33047
33048 Properties:
33049
33050 • Config: endpoint
33051
33052 • Env Var: RCLONE_INTERNETARCHIVE_ENDPOINT
33053
33054 • Type: string
33055
33056 • Default: "https://s3.us.archive.org"
33057
33058 --internetarchive-front-endpoint
33059 Host of InternetArchive Frontend.
33060
33061 Leave blank for default value.
33062
33063 Properties:
33064
33065 • Config: front_endpoint
33066
33067 • Env Var: RCLONE_INTERNETARCHIVE_FRONT_ENDPOINT
33068
33069 • Type: string
33070
33071 • Default: "https://archive.org"
33072
33073 --internetarchive-disable-checksum
33074 Don't ask the server to test against MD5 checksum calculated by rclone.
33075 Normally rclone will calculate the MD5 checksum of the input before up‐
33076 loading it so it can ask the server to check the object against check‐
33077 sum. This is great for data integrity checking but can cause long de‐
33078 lays for large files to start uploading.
33079
33080 Properties:
33081
33082 • Config: disable_checksum
33083
33084 • Env Var: RCLONE_INTERNETARCHIVE_DISABLE_CHECKSUM
33085
33086 • Type: bool
33087
33088 • Default: true
33089
33090 --internetarchive-wait-archive
33091 Timeout for waiting the server's processing tasks (specifically archive
33092 and book_op) to finish. Only enable if you need to be guaranteed to be
33093 reflected after write operations. 0 to disable waiting. No errors to
33094 be thrown in case of timeout.
33095
33096 Properties:
33097
33098 • Config: wait_archive
33099
33100 • Env Var: RCLONE_INTERNETARCHIVE_WAIT_ARCHIVE
33101
33102 • Type: Duration
33103
33104 • Default: 0s
33105
33106 --internetarchive-encoding
33107 The encoding for the backend.
33108
33109 See the encoding section in the overview (https://rclone.org/over‐
33110 view/#encoding) for more info.
33111
33112 Properties:
33113
33114 • Config: encoding
33115
33116 • Env Var: RCLONE_INTERNETARCHIVE_ENCODING
33117
33118 • Type: MultiEncoder
33119
33120 • Default: Slash,LtGt,CrLf,Del,Ctl,InvalidUtf8,Dot
33121
33122 Metadata
33123 Metadata fields provided by Internet Archive. If there are multiple
33124 values for a key, only the first one is returned. This is a limitation
33125 of Rclone, that supports one value per one key.
33126
33127 Owner is able to add custom keys. Metadata feature grabs all the keys
33128 including them.
33129
33130 Here are the possible system metadata items for the internetarchive
33131 backend.
33132
33133 Name Help Type Example Read Only
33134 ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
33135 crc32 CRC32 cal‐ string 01234567 Y
33136 culated by
33137 Internet
33138 Archive
33139 format Name of string Comma-Separated Y
33140 format Values
33141 identified
33142 by Internet
33143 Archive
33144 md5 MD5 hash string 01234567012345670123456701234567 Y
33145 calculated
33146 by Internet
33147 Archive
33148 mtime Time of RFC 3339 2006-01-02T15:04:05.999999999Z Y
33149 last modi‐
33150 fication,
33151 managed by
33152 Rclone
33153 name Full file filename backend/internetarchive/inter‐ Y
33154 path, with‐ netarchive.go
33155 out the
33156 bucket part
33157
33158
33159
33160
33161 old_version Whether the boolean true Y
33162 file was
33163 replaced
33164 and moved
33165 by keep-
33166 old-version
33167 flag
33168 rclone-ia- Time of RFC 3339 2006-01-02T15:04:05.999999999Z N
33169 mtime last modi‐
33170 fication,
33171 managed by
33172 Internet
33173 Archive
33174 rclone- Time of RFC 3339 2006-01-02T15:04:05.999999999Z N
33175 mtime last modi‐
33176 fication,
33177 managed by
33178 Rclone
33179 rclone-up‐ Random val‐ string aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa N
33180 date-track ue used by
33181 Rclone for
33182 tracking
33183 changes in‐
33184 side Inter‐
33185 net Archive
33186 sha1 SHA1 hash string 0123456701234567012345670123456701234567 Y
33187 calculated
33188 by Internet
33189 Archive
33190 size File size decimal 123456 Y
33191 in bytes number
33192 source The source string original Y
33193 of the file
33194 summation Check string md5 Y
33195 https://fo‐
33196 rum.rclone.org/t/31922
33197 for how it
33198 is used
33199 viruscheck The last time vir‐ unixtime 1654191352 Y
33200 uscheck process was
33201 run for the file (?)
33202
33203 See the metadata (https://rclone.org/docs/#metadata) docs for more in‐
33204 fo.
33205
33207 Jottacloud is a cloud storage service provider from a Norwegian compa‐
33208 ny, using its own datacenters in Norway. In addition to the official
33209 service at jottacloud.com (https://www.jottacloud.com/), it also pro‐
33210 vides white-label solutions to different companies, such as: * Telia *
33211 Telia Cloud (cloud.telia.se) * Telia Sky (sky.telia.no) * Tele2 * Tele2
33212 Cloud (mittcloud.tele2.se) * Elkjøp (with subsidiaries): * Elkjøp Cloud
33213 (cloud.elkjop.no) * Elgiganten Sweden (cloud.elgiganten.se) * Elgigan‐
33214 ten Denmark (cloud.elgiganten.dk) * Giganti Cloud (cloud.gigantti.fi) *
33215 ELKO Cloud (cloud.elko.is)
33216
33217 Most of the white-label versions are supported by this backend, al‐
33218 though may require different authentication setup - described below.
33219
33220 Paths are specified as remote:path
33221
33222 Paths may be as deep as required, e.g. remote:directory/subdirectory.
33223
33224 Authentication types
33225 Some of the whitelabel versions uses a different authentication method
33226 than the official service, and you have to choose the correct one when
33227 setting up the remote.
33228
33229 Standard authentication
33230 The standard authentication method used by the official service (jotta‐
33231 cloud.com), as well as some of the whitelabel services, requires you to
33232 generate a single-use personal login token from the account security
33233 settings in the service's web interface. Log in to your account, go to
33234 "Settings" and then "Security", or use the direct link presented to you
33235 by rclone when configuring the remote: <https://www.jotta‐
33236 cloud.com/web/secure>. Scroll down to the section "Personal login to‐
33237 ken", and click the "Generate" button. Note that if you are using a
33238 whitelabel service you probably can't use the direct link, you need to
33239 find the same page in their dedicated web interface, and also it may be
33240 in a different location than described above.
33241
33242 To access your account from multiple instances of rclone, you need to
33243 configure each of them with a separate personal login token. E.g. you
33244 create a Jottacloud remote with rclone in one location, and copy the
33245 configuration file to a second location where you also want to run
33246 rclone and access the same remote. Then you need to replace the token
33247 for one of them, using the config reconnect (https://rclone.org/com‐
33248 mands/rclone_config_reconnect/) command, which requires you to generate
33249 a new personal login token and supply as input. If you do not do this,
33250 the token may easily end up being invalidated, resulting in both in‐
33251 stances failing with an error message something along the lines of:
33252
33253 oauth2: cannot fetch token: 400 Bad Request
33254 Response: {"error":"invalid_grant","error_description":"Stale token"}
33255
33256 When this happens, you need to replace the token as described above to
33257 be able to use your remote again.
33258
33259 All personal login tokens you have taken into use will be listed in the
33260 web interface under "My logged in devices", and from the right side of
33261 that list you can click the "X" button to revoke individual tokens.
33262
33263 Legacy authentication
33264 If you are using one of the whitelabel versions (e.g. from Elkjøp) you
33265 may not have the option to generate a CLI token. In this case you'll
33266 have to use the legacy authentication. To do this select yes when the
33267 setup asks for legacy authentication and enter your username and pass‐
33268 word. The rest of the setup is identical to the default setup.
33269
33270 Telia Cloud authentication
33271 Similar to other whitelabel versions Telia Cloud doesn't offer the op‐
33272 tion of creating a CLI token, and additionally uses a separate authen‐
33273 tication flow where the username is generated internally. To setup
33274 rclone to use Telia Cloud, choose Telia Cloud authentication in the
33275 setup. The rest of the setup is identical to the default setup.
33276
33277 Tele2 Cloud authentication
33278 As Tele2-Com Hem merger was completed this authentication can be used
33279 for former Com Hem Cloud and Tele2 Cloud customers as no support for
33280 creating a CLI token exists, and additionally uses a separate authenti‐
33281 cation flow where the username is generated internally. To setup
33282 rclone to use Tele2 Cloud, choose Tele2 Cloud authentication in the
33283 setup. The rest of the setup is identical to the default setup.
33284
33285 Configuration
33286 Here is an example of how to make a remote called remote with the de‐
33287 fault setup. First run:
33288
33289 rclone config
33290
33291 This will guide you through an interactive setup process:
33292
33293 No remotes found, make a new one?
33294 n) New remote
33295 s) Set configuration password
33296 q) Quit config
33297 n/s/q> n
33298 name> remote
33299 Option Storage.
33300 Type of storage to configure.
33301 Choose a number from below, or type in your own value.
33302 [snip]
33303 XX / Jottacloud
33304 \ (jottacloud)
33305 [snip]
33306 Storage> jottacloud
33307 Edit advanced config?
33308 y) Yes
33309 n) No (default)
33310 y/n> n
33311 Option config_type.
33312 Select authentication type.
33313 Choose a number from below, or type in an existing string value.
33314 Press Enter for the default (standard).
33315 / Standard authentication.
33316 1 | Use this if you're a normal Jottacloud user.
33317 \ (standard)
33318 / Legacy authentication.
33319 2 | This is only required for certain whitelabel versions of Jottacloud and not recommended for normal users.
33320 \ (legacy)
33321 / Telia Cloud authentication.
33322 3 | Use this if you are using Telia Cloud.
33323 \ (telia)
33324 / Tele2 Cloud authentication.
33325 4 | Use this if you are using Tele2 Cloud.
33326 \ (tele2)
33327 config_type> 1
33328 Personal login token.
33329 Generate here: https://www.jottacloud.com/web/secure
33330 Login Token> <your token here>
33331 Use a non-standard device/mountpoint?
33332 Choosing no, the default, will let you access the storage used for the archive
33333 section of the official Jottacloud client. If you instead want to access the
33334 sync or the backup section, for example, you must choose yes.
33335 y) Yes
33336 n) No (default)
33337 y/n> y
33338 Option config_device.
33339 The device to use. In standard setup the built-in Jotta device is used,
33340 which contains predefined mountpoints for archive, sync etc. All other devices
33341 are treated as backup devices by the official Jottacloud client. You may create
33342 a new by entering a unique name.
33343 Choose a number from below, or type in your own string value.
33344 Press Enter for the default (DESKTOP-3H31129).
33345 1 > DESKTOP-3H31129
33346 2 > Jotta
33347 config_device> 2
33348 Option config_mountpoint.
33349 The mountpoint to use for the built-in device Jotta.
33350 The standard setup is to use the Archive mountpoint. Most other mountpoints
33351 have very limited support in rclone and should generally be avoided.
33352 Choose a number from below, or type in an existing string value.
33353 Press Enter for the default (Archive).
33354 1 > Archive
33355 2 > Shared
33356 3 > Sync
33357 config_mountpoint> 1
33358 --------------------
33359 [remote]
33360 type = jottacloud
33361 configVersion = 1
33362 client_id = jottacli
33363 client_secret =
33364 tokenURL = https://id.jottacloud.com/auth/realms/jottacloud/protocol/openid-connect/token
33365 token = {........}
33366 username = 2940e57271a93d987d6f8a21
33367 device = Jotta
33368 mountpoint = Archive
33369 --------------------
33370 y) Yes this is OK (default)
33371 e) Edit this remote
33372 d) Delete this remote
33373 y/e/d> y
33374
33375 Once configured you can then use rclone like this,
33376
33377 List directories in top level of your Jottacloud
33378
33379 rclone lsd remote:
33380
33381 List all the files in your Jottacloud
33382
33383 rclone ls remote:
33384
33385 To copy a local directory to an Jottacloud directory called backup
33386
33387 rclone copy /home/source remote:backup
33388
33389 Devices and Mountpoints
33390 The official Jottacloud client registers a device for each computer you
33391 install it on, and shows them in the backup section of the user inter‐
33392 face. For each folder you select for backup it will create a mount‐
33393 point within this device. A built-in device called Jotta is special,
33394 and contains mountpoints Archive, Sync and some others, used for corre‐
33395 sponding features in official clients.
33396
33397 With rclone you'll want to use the standard Jotta/Archive device/mount‐
33398 point in most cases. However, you may for example want to access files
33399 from the sync or backup functionality provided by the official clients,
33400 and rclone therefore provides the option to select other devices and
33401 mountpoints during config.
33402
33403 You are allowed to create new devices and mountpoints. All devices ex‐
33404 cept the built-in Jotta device are treated as backup devices by offi‐
33405 cial Jottacloud clients, and the mountpoints on them are individual
33406 backup sets.
33407
33408 With the built-in Jotta device, only existing, built-in, mountpoints
33409 can be selected. In addition to the mentioned Archive and Sync, it may
33410 contain several other mountpoints such as: Latest, Links, Shared and
33411 Trash. All of these are special mountpoints with a different internal
33412 representation than the "regular" mountpoints. Rclone will only to a
33413 very limited degree support them. Generally you should avoid these,
33414 unless you know what you are doing.
33415
33416 --fast-list
33417 This remote supports --fast-list which allows you to use fewer transac‐
33418 tions in exchange for more memory. See the rclone docs
33419 (https://rclone.org/docs/#fast-list) for more details.
33420
33421 Note that the implementation in Jottacloud always uses only a single
33422 API request to get the entire list, so for large folders this could
33423 lead to long wait time before the first results are shown.
33424
33425 Note also that with rclone version 1.58 and newer information about
33426 MIME types (https://rclone.org/overview/#mime-type) are not available
33427 when using --fast-list.
33428
33429 Modified time and hashes
33430 Jottacloud allows modification times to be set on objects accurate to 1
33431 second. These will be used to detect whether objects need syncing or
33432 not.
33433
33434 Jottacloud supports MD5 type hashes, so you can use the --checksum
33435 flag.
33436
33437 Note that Jottacloud requires the MD5 hash before upload so if the
33438 source does not have an MD5 checksum then the file will be cached tem‐
33439 porarily on disk (in location given by --temp-dir
33440 (https://rclone.org/docs/#temp-dir-dir)) before it is uploaded. Small
33441 files will be cached in memory - see the --jottacloud-md5-memory-limit
33442 flag. When uploading from local disk the source checksum is always
33443 available, so this does not apply. Starting with rclone version 1.52
33444 the same is true for crypted remotes (in older versions the crypt back‐
33445 end would not calculate hashes for uploads from local disk, so the Jot‐
33446 tacloud backend had to do it as described above).
33447
33448 Restricted filename characters
33449 In addition to the default restricted characters set
33450 (https://rclone.org/overview/#restricted-characters) the following
33451 characters are also replaced:
33452
33453 Character Value Replacement
33454 ────────────────────────────────
33455 " 0x22 "
33456 * 0x2A *
33457 : 0x3A :
33458 < 0x3C <
33459 > 0x3E >
33460 ? 0x3F ?
33461 | 0x7C |
33462
33463 Invalid UTF-8 bytes will also be replaced (https://rclone.org/over‐
33464 view/#invalid-utf8), as they can't be used in XML strings.
33465
33466 Deleting files
33467 By default, rclone will send all files to the trash when deleting
33468 files. They will be permanently deleted automatically after 30 days.
33469 You may bypass the trash and permanently delete files immediately by
33470 using the --jottacloud-hard-delete flag, or set the equivalent environ‐
33471 ment variable. Emptying the trash is supported by the cleanup
33472 (https://rclone.org/commands/rclone_cleanup/) command.
33473
33474 Versions
33475 Jottacloud supports file versioning. When rclone uploads a new version
33476 of a file it creates a new version of it. Currently rclone only sup‐
33477 ports retrieving the current version but older versions can be accessed
33478 via the Jottacloud Website.
33479
33480 Versioning can be disabled by --jottacloud-no-versions option. This is
33481 achieved by deleting the remote file prior to uploading a new version.
33482 If the upload the fails no version of the file will be available in the
33483 remote.
33484
33485 Quota information
33486 To view your current quota you can use the rclone about remote: command
33487 which will display your usage limit (unless it is unlimited) and the
33488 current usage.
33489
33490 Advanced options
33491 Here are the Advanced options specific to jottacloud (Jottacloud).
33492
33493 --jottacloud-md5-memory-limit
33494 Files bigger than this will be cached on disk to calculate the MD5 if
33495 required.
33496
33497 Properties:
33498
33499 • Config: md5_memory_limit
33500
33501 • Env Var: RCLONE_JOTTACLOUD_MD5_MEMORY_LIMIT
33502
33503 • Type: SizeSuffix
33504
33505 • Default: 10Mi
33506
33507 --jottacloud-trashed-only
33508 Only show files that are in the trash.
33509
33510 This will show trashed files in their original directory structure.
33511
33512 Properties:
33513
33514 • Config: trashed_only
33515
33516 • Env Var: RCLONE_JOTTACLOUD_TRASHED_ONLY
33517
33518 • Type: bool
33519
33520 • Default: false
33521
33522 --jottacloud-hard-delete
33523 Delete files permanently rather than putting them into the trash.
33524
33525 Properties:
33526
33527 • Config: hard_delete
33528
33529 • Env Var: RCLONE_JOTTACLOUD_HARD_DELETE
33530
33531 • Type: bool
33532
33533 • Default: false
33534
33535 --jottacloud-upload-resume-limit
33536 Files bigger than this can be resumed if the upload fail's.
33537
33538 Properties:
33539
33540 • Config: upload_resume_limit
33541
33542 • Env Var: RCLONE_JOTTACLOUD_UPLOAD_RESUME_LIMIT
33543
33544 • Type: SizeSuffix
33545
33546 • Default: 10Mi
33547
33548 --jottacloud-no-versions
33549 Avoid server side versioning by deleting files and recreating files in‐
33550 stead of overwriting them.
33551
33552 Properties:
33553
33554 • Config: no_versions
33555
33556 • Env Var: RCLONE_JOTTACLOUD_NO_VERSIONS
33557
33558 • Type: bool
33559
33560 • Default: false
33561
33562 --jottacloud-encoding
33563 The encoding for the backend.
33564
33565 See the encoding section in the overview (https://rclone.org/over‐
33566 view/#encoding) for more info.
33567
33568 Properties:
33569
33570 • Config: encoding
33571
33572 • Env Var: RCLONE_JOTTACLOUD_ENCODING
33573
33574 • Type: MultiEncoder
33575
33576 • Default: Slash,LtGt,DoubleQuote,Colon,Question,Aster‐
33577 isk,Pipe,Del,Ctl,InvalidUtf8,Dot
33578
33579 Limitations
33580 Note that Jottacloud is case insensitive so you can't have a file
33581 called "Hello.doc" and one called "hello.doc".
33582
33583 There are quite a few characters that can't be in Jottacloud file
33584 names. Rclone will map these names to and from an identical looking
33585 unicode equivalent. For example if a file has a ? in it will be
33586 mapped to ? instead.
33587
33588 Jottacloud only supports filenames up to 255 characters in length.
33589
33590 Troubleshooting
33591 Jottacloud exhibits some inconsistent behaviours regarding deleted
33592 files and folders which may cause Copy, Move and DirMove operations to
33593 previously deleted paths to fail. Emptying the trash should help in
33594 such cases.
33595
33597 Paths are specified as remote:path
33598
33599 Paths may be as deep as required, e.g. remote:directory/subdirectory.
33600
33601 Configuration
33602 The initial setup for Koofr involves creating an application password
33603 for rclone. You can do that by opening the Koofr web application
33604 (https://app.koofr.net/app/admin/preferences/password), giving the
33605 password a nice name like rclone and clicking on generate.
33606
33607 Here is an example of how to make a remote called koofr. First run:
33608
33609 rclone config
33610
33611 This will guide you through an interactive setup process:
33612
33613 No remotes found, make a new one?
33614 n) New remote
33615 s) Set configuration password
33616 q) Quit config
33617 n/s/q> n
33618 name> koofr
33619 Option Storage.
33620 Type of storage to configure.
33621 Choose a number from below, or type in your own value.
33622 [snip]
33623 22 / Koofr, Digi Storage and other Koofr-compatible storage providers
33624 \ (koofr)
33625 [snip]
33626 Storage> koofr
33627 Option provider.
33628 Choose your storage provider.
33629 Choose a number from below, or type in your own value.
33630 Press Enter to leave empty.
33631 1 / Koofr, https://app.koofr.net/
33632 \ (koofr)
33633 2 / Digi Storage, https://storage.rcs-rds.ro/
33634 \ (digistorage)
33635 3 / Any other Koofr API compatible storage service
33636 \ (other)
33637 provider> 1
33638 Option user.
33639 Your user name.
33640 Enter a value.
33641 user> USERNAME
33642 Option password.
33643 Your password for rclone (generate one at https://app.koofr.net/app/admin/preferences/password).
33644 Choose an alternative below.
33645 y) Yes, type in my own password
33646 g) Generate random password
33647 y/g> y
33648 Enter the password:
33649 password:
33650 Confirm the password:
33651 password:
33652 Edit advanced config?
33653 y) Yes
33654 n) No (default)
33655 y/n> n
33656 Remote config
33657 --------------------
33658 [koofr]
33659 type = koofr
33660 provider = koofr
33661 user = USERNAME
33662 password = *** ENCRYPTED ***
33663 --------------------
33664 y) Yes this is OK (default)
33665 e) Edit this remote
33666 d) Delete this remote
33667 y/e/d> y
33668
33669 You can choose to edit advanced config in order to enter your own ser‐
33670 vice URL if you use an on-premise or white label Koofr instance, or
33671 choose an alternative mount instead of your primary storage.
33672
33673 Once configured you can then use rclone like this,
33674
33675 List directories in top level of your Koofr
33676
33677 rclone lsd koofr:
33678
33679 List all the files in your Koofr
33680
33681 rclone ls koofr:
33682
33683 To copy a local directory to an Koofr directory called backup
33684
33685 rclone copy /home/source koofr:backup
33686
33687 Restricted filename characters
33688 In addition to the default restricted characters set
33689 (https://rclone.org/overview/#restricted-characters) the following
33690 characters are also replaced:
33691
33692 Character Value Replacement
33693 ────────────────────────────────
33694 \ 0x5C \
33695
33696 Invalid UTF-8 bytes will also be replaced (https://rclone.org/over‐
33697 view/#invalid-utf8), as they can't be used in XML strings.
33698
33699 Standard options
33700 Here are the Standard options specific to koofr (Koofr, Digi Storage
33701 and other Koofr-compatible storage providers).
33702
33703 --koofr-provider
33704 Choose your storage provider.
33705
33706 Properties:
33707
33708 • Config: provider
33709
33710 • Env Var: RCLONE_KOOFR_PROVIDER
33711
33712 • Type: string
33713
33714 • Required: false
33715
33716 • Examples:
33717
33718 • "koofr"
33719
33720 • Koofr, https://app.koofr.net/
33721
33722 • "digistorage"
33723
33724 • Digi Storage, https://storage.rcs-rds.ro/
33725
33726 • "other"
33727
33728 • Any other Koofr API compatible storage service
33729
33730 --koofr-endpoint
33731 The Koofr API endpoint to use.
33732
33733 Properties:
33734
33735 • Config: endpoint
33736
33737 • Env Var: RCLONE_KOOFR_ENDPOINT
33738
33739 • Provider: other
33740
33741 • Type: string
33742
33743 • Required: true
33744
33745 --koofr-user
33746 Your user name.
33747
33748 Properties:
33749
33750 • Config: user
33751
33752 • Env Var: RCLONE_KOOFR_USER
33753
33754 • Type: string
33755
33756 • Required: true
33757
33758 --koofr-password
33759 Your password for rclone (generate one at https://app.koofr.net/app/ad‐
33760 min/preferences/password).
33761
33762 NB Input to this must be obscured - see rclone obscure
33763 (https://rclone.org/commands/rclone_obscure/).
33764
33765 Properties:
33766
33767 • Config: password
33768
33769 • Env Var: RCLONE_KOOFR_PASSWORD
33770
33771 • Provider: koofr
33772
33773 • Type: string
33774
33775 • Required: true
33776
33777 --koofr-password
33778 Your password for rclone (generate one at https://storage.rcs-
33779 rds.ro/app/admin/preferences/password).
33780
33781 NB Input to this must be obscured - see rclone obscure
33782 (https://rclone.org/commands/rclone_obscure/).
33783
33784 Properties:
33785
33786 • Config: password
33787
33788 • Env Var: RCLONE_KOOFR_PASSWORD
33789
33790 • Provider: digistorage
33791
33792 • Type: string
33793
33794 • Required: true
33795
33796 --koofr-password
33797 Your password for rclone (generate one at your service's settings
33798 page).
33799
33800 NB Input to this must be obscured - see rclone obscure
33801 (https://rclone.org/commands/rclone_obscure/).
33802
33803 Properties:
33804
33805 • Config: password
33806
33807 • Env Var: RCLONE_KOOFR_PASSWORD
33808
33809 • Provider: other
33810
33811 • Type: string
33812
33813 • Required: true
33814
33815 Advanced options
33816 Here are the Advanced options specific to koofr (Koofr, Digi Storage
33817 and other Koofr-compatible storage providers).
33818
33819 --koofr-mountid
33820 Mount ID of the mount to use.
33821
33822 If omitted, the primary mount is used.
33823
33824 Properties:
33825
33826 • Config: mountid
33827
33828 • Env Var: RCLONE_KOOFR_MOUNTID
33829
33830 • Type: string
33831
33832 • Required: false
33833
33834 --koofr-setmtime
33835 Does the backend support setting modification time.
33836
33837 Set this to false if you use a mount ID that points to a Dropbox or
33838 Amazon Drive backend.
33839
33840 Properties:
33841
33842 • Config: setmtime
33843
33844 • Env Var: RCLONE_KOOFR_SETMTIME
33845
33846 • Type: bool
33847
33848 • Default: true
33849
33850 --koofr-encoding
33851 The encoding for the backend.
33852
33853 See the encoding section in the overview (https://rclone.org/over‐
33854 view/#encoding) for more info.
33855
33856 Properties:
33857
33858 • Config: encoding
33859
33860 • Env Var: RCLONE_KOOFR_ENCODING
33861
33862 • Type: MultiEncoder
33863
33864 • Default: Slash,BackSlash,Del,Ctl,InvalidUtf8,Dot
33865
33866 Limitations
33867 Note that Koofr is case insensitive so you can't have a file called
33868 "Hello.doc" and one called "hello.doc".
33869
33870 Providers
33871 Koofr
33872 This is the original Koofr (https://koofr.eu) storage provider used as
33873 main example and described in the configuration section above.
33874
33875 Digi Storage
33876 Digi Storage (https://www.digi.ro/servicii/online/digi-storage) is a
33877 cloud storage service run by Digi.ro (https://www.digi.ro/) that pro‐
33878 vides a Koofr API.
33879
33880 Here is an example of how to make a remote called ds. First run:
33881
33882 rclone config
33883
33884 This will guide you through an interactive setup process:
33885
33886 No remotes found, make a new one?
33887 n) New remote
33888 s) Set configuration password
33889 q) Quit config
33890 n/s/q> n
33891 name> ds
33892 Option Storage.
33893 Type of storage to configure.
33894 Choose a number from below, or type in your own value.
33895 [snip]
33896 22 / Koofr, Digi Storage and other Koofr-compatible storage providers
33897 \ (koofr)
33898 [snip]
33899 Storage> koofr
33900 Option provider.
33901 Choose your storage provider.
33902 Choose a number from below, or type in your own value.
33903 Press Enter to leave empty.
33904 1 / Koofr, https://app.koofr.net/
33905 \ (koofr)
33906 2 / Digi Storage, https://storage.rcs-rds.ro/
33907 \ (digistorage)
33908 3 / Any other Koofr API compatible storage service
33909 \ (other)
33910 provider> 2
33911 Option user.
33912 Your user name.
33913 Enter a value.
33914 user> USERNAME
33915 Option password.
33916 Your password for rclone (generate one at https://storage.rcs-rds.ro/app/admin/preferences/password).
33917 Choose an alternative below.
33918 y) Yes, type in my own password
33919 g) Generate random password
33920 y/g> y
33921 Enter the password:
33922 password:
33923 Confirm the password:
33924 password:
33925 Edit advanced config?
33926 y) Yes
33927 n) No (default)
33928 y/n> n
33929 --------------------
33930 [ds]
33931 type = koofr
33932 provider = digistorage
33933 user = USERNAME
33934 password = *** ENCRYPTED ***
33935 --------------------
33936 y) Yes this is OK (default)
33937 e) Edit this remote
33938 d) Delete this remote
33939 y/e/d> y
33940
33941 Other
33942 You may also want to use another, public or private storage provider
33943 that runs a Koofr API compatible service, by simply providing the base
33944 URL to connect to.
33945
33946 Here is an example of how to make a remote called other. First run:
33947
33948 rclone config
33949
33950 This will guide you through an interactive setup process:
33951
33952 No remotes found, make a new one?
33953 n) New remote
33954 s) Set configuration password
33955 q) Quit config
33956 n/s/q> n
33957 name> other
33958 Option Storage.
33959 Type of storage to configure.
33960 Choose a number from below, or type in your own value.
33961 [snip]
33962 22 / Koofr, Digi Storage and other Koofr-compatible storage providers
33963 \ (koofr)
33964 [snip]
33965 Storage> koofr
33966 Option provider.
33967 Choose your storage provider.
33968 Choose a number from below, or type in your own value.
33969 Press Enter to leave empty.
33970 1 / Koofr, https://app.koofr.net/
33971 \ (koofr)
33972 2 / Digi Storage, https://storage.rcs-rds.ro/
33973 \ (digistorage)
33974 3 / Any other Koofr API compatible storage service
33975 \ (other)
33976 provider> 3
33977 Option endpoint.
33978 The Koofr API endpoint to use.
33979 Enter a value.
33980 endpoint> https://koofr.other.org
33981 Option user.
33982 Your user name.
33983 Enter a value.
33984 user> USERNAME
33985 Option password.
33986 Your password for rclone (generate one at your service's settings page).
33987 Choose an alternative below.
33988 y) Yes, type in my own password
33989 g) Generate random password
33990 y/g> y
33991 Enter the password:
33992 password:
33993 Confirm the password:
33994 password:
33995 Edit advanced config?
33996 y) Yes
33997 n) No (default)
33998 y/n> n
33999 --------------------
34000 [other]
34001 type = koofr
34002 provider = other
34003 endpoint = https://koofr.other.org
34004 user = USERNAME
34005 password = *** ENCRYPTED ***
34006 --------------------
34007 y) Yes this is OK (default)
34008 e) Edit this remote
34009 d) Delete this remote
34010 y/e/d> y
34011
34013 Mail.ru Cloud (https://cloud.mail.ru/) is a cloud storage provided by a
34014 Russian internet company Mail.Ru Group (https://mail.ru). The official
34015 desktop client is Disk-O: (https://disk-o.cloud/en), available on Win‐
34016 dows and Mac OS.
34017
34018 Currently it is recommended to disable 2FA on Mail.ru accounts intended
34019 for rclone until it gets eventually implemented.
34020
34021 Features highlights
34022 • Paths may be as deep as required, e.g. remote:directory/subdirectory
34023
34024 • Files have a last modified time property, directories don't
34025
34026 • Deleted files are by default moved to the trash
34027
34028 • Files and directories can be shared via public links
34029
34030 • Partial uploads or streaming are not supported, file size must be
34031 known before upload
34032
34033 • Maximum file size is limited to 2G for a free account, unlimited for
34034 paid accounts
34035
34036 • Storage keeps hash for all files and performs transparent deduplica‐
34037 tion, the hash algorithm is a modified SHA1
34038
34039 • If a particular file is already present in storage, one can quickly
34040 submit file hash instead of long file upload (this optimization is
34041 supported by rclone)
34042
34043 Configuration
34044 Here is an example of making a mailru configuration.
34045
34046 First create a Mail.ru Cloud account and choose a tariff.
34047
34048 You will need to log in and create an app password for rclone. Rclone
34049 will not work with your normal username and password - it will give an
34050 error like oauth2: server response missing access_token.
34051
34052 • Click on your user icon in the top right
34053
34054 • Go to Security / "Пароль и безопасность"
34055
34056 • Click password for apps / "Пароли для внешних приложений"
34057
34058 • Add the password - give it a name - eg "rclone"
34059
34060 • Copy the password and use this password below - your normal login
34061 password won't work.
34062
34063 Now run
34064
34065 rclone config
34066
34067 This will guide you through an interactive setup process:
34068
34069 No remotes found, make a new one?
34070 n) New remote
34071 s) Set configuration password
34072 q) Quit config
34073 n/s/q> n
34074 name> remote
34075 Type of storage to configure.
34076 Type of storage to configure.
34077 Enter a string value. Press Enter for the default ("").
34078 Choose a number from below, or type in your own value
34079 [snip]
34080 XX / Mail.ru Cloud
34081 \ "mailru"
34082 [snip]
34083 Storage> mailru
34084 User name (usually email)
34085 Enter a string value. Press Enter for the default ("").
34086 user> username@mail.ru
34087 Password
34088
34089 This must be an app password - rclone will not work with your normal
34090 password. See the Configuration section in the docs for how to make an
34091 app password.
34092 y) Yes type in my own password
34093 g) Generate random password
34094 y/g> y
34095 Enter the password:
34096 password:
34097 Confirm the password:
34098 password:
34099 Skip full upload if there is another file with same data hash.
34100 This feature is called "speedup" or "put by hash". It is especially efficient
34101 in case of generally available files like popular books, video or audio clips
34102 [snip]
34103 Enter a boolean value (true or false). Press Enter for the default ("true").
34104 Choose a number from below, or type in your own value
34105 1 / Enable
34106 \ "true"
34107 2 / Disable
34108 \ "false"
34109 speedup_enable> 1
34110 Edit advanced config? (y/n)
34111 y) Yes
34112 n) No
34113 y/n> n
34114 Remote config
34115 --------------------
34116 [remote]
34117 type = mailru
34118 user = username@mail.ru
34119 pass = *** ENCRYPTED ***
34120 speedup_enable = true
34121 --------------------
34122 y) Yes this is OK
34123 e) Edit this remote
34124 d) Delete this remote
34125 y/e/d> y
34126
34127 Configuration of this backend does not require a local web browser.
34128 You can use the configured backend as shown below:
34129
34130 See top level directories
34131
34132 rclone lsd remote:
34133
34134 Make a new directory
34135
34136 rclone mkdir remote:directory
34137
34138 List the contents of a directory
34139
34140 rclone ls remote:directory
34141
34142 Sync /home/local/directory to the remote path, deleting any excess
34143 files in the path.
34144
34145 rclone sync -i /home/local/directory remote:directory
34146
34147 Modified time
34148 Files support a modification time attribute with up to 1 second preci‐
34149 sion. Directories do not have a modification time, which is shown as
34150 "Jan 1 1970".
34151
34152 Hash checksums
34153 Hash sums use a custom Mail.ru algorithm based on SHA1. If file size
34154 is less than or equal to the SHA1 block size (20 bytes), its hash is
34155 simply its data right-padded with zero bytes. Hash sum of a larger
34156 file is computed as a SHA1 sum of the file data bytes concatenated with
34157 a decimal representation of the data length.
34158
34159 Emptying Trash
34160 Removing a file or directory actually moves it to the trash, which is
34161 not visible to rclone but can be seen in a web browser. The trashed
34162 file still occupies part of total quota. If you wish to empty your
34163 trash and free some quota, you can use the rclone cleanup remote: com‐
34164 mand, which will permanently delete all your trashed files. This com‐
34165 mand does not take any path arguments.
34166
34167 Quota information
34168 To view your current quota you can use the rclone about remote: command
34169 which will display your usage limit (quota) and the current usage.
34170
34171 Restricted filename characters
34172 In addition to the default restricted characters set
34173 (https://rclone.org/overview/#restricted-characters) the following
34174 characters are also replaced:
34175
34176 Character Value Replacement
34177 ────────────────────────────────
34178 " 0x22 "
34179 * 0x2A *
34180 : 0x3A :
34181 < 0x3C <
34182 > 0x3E >
34183 ? 0x3F ?
34184 \ 0x5C \
34185 | 0x7C |
34186
34187 Invalid UTF-8 bytes will also be replaced (https://rclone.org/over‐
34188 view/#invalid-utf8), as they can't be used in JSON strings.
34189
34190 Standard options
34191 Here are the Standard options specific to mailru (Mail.ru Cloud).
34192
34193 --mailru-user
34194 User name (usually email).
34195
34196 Properties:
34197
34198 • Config: user
34199
34200 • Env Var: RCLONE_MAILRU_USER
34201
34202 • Type: string
34203
34204 • Required: true
34205
34206 --mailru-pass
34207 Password.
34208
34209 This must be an app password - rclone will not work with your normal
34210 password. See the Configuration section in the docs for how to make an
34211 app password.
34212
34213 NB Input to this must be obscured - see rclone obscure
34214 (https://rclone.org/commands/rclone_obscure/).
34215
34216 Properties:
34217
34218 • Config: pass
34219
34220 • Env Var: RCLONE_MAILRU_PASS
34221
34222 • Type: string
34223
34224 • Required: true
34225
34226 --mailru-speedup-enable
34227 Skip full upload if there is another file with same data hash.
34228
34229 This feature is called "speedup" or "put by hash". It is especially
34230 efficient in case of generally available files like popular books,
34231 video or audio clips, because files are searched by hash in all ac‐
34232 counts of all mailru users. It is meaningless and ineffective if
34233 source file is unique or encrypted. Please note that rclone may need
34234 local memory and disk space to calculate content hash in advance and
34235 decide whether full upload is required. Also, if rclone does not know
34236 file size in advance (e.g. in case of streaming or partial uploads),
34237 it will not even try this optimization.
34238
34239 Properties:
34240
34241 • Config: speedup_enable
34242
34243 • Env Var: RCLONE_MAILRU_SPEEDUP_ENABLE
34244
34245 • Type: bool
34246
34247 • Default: true
34248
34249 • Examples:
34250
34251 • "true"
34252
34253 • Enable
34254
34255 • "false"
34256
34257 • Disable
34258
34259 Advanced options
34260 Here are the Advanced options specific to mailru (Mail.ru Cloud).
34261
34262 --mailru-speedup-file-patterns
34263 Comma separated list of file name patterns eligible for speedup (put by
34264 hash).
34265
34266 Patterns are case insensitive and can contain '*' or '?' meta charac‐
34267 ters.
34268
34269 Properties:
34270
34271 • Config: speedup_file_patterns
34272
34273 • Env Var: RCLONE_MAILRU_SPEEDUP_FILE_PATTERNS
34274
34275 • Type: string
34276
34277 • Default: ".mkv,.avi,.mp4,.mp3,.zip,.gz,.rar,.pdf"
34278
34279 • Examples:
34280
34281 • ""
34282
34283 • Empty list completely disables speedup (put by hash).
34284
34285 • "*"
34286
34287 • All files will be attempted for speedup.
34288
34289 • ".mkv,.avi,.mp4,.mp3"
34290
34291 • Only common audio/video files will be tried for put by hash.
34292
34293 • ".zip,.gz,.rar,.pdf"
34294
34295 • Only common archives or PDF books will be tried for speedup.
34296
34297 --mailru-speedup-max-disk
34298 This option allows you to disable speedup (put by hash) for large
34299 files.
34300
34301 Reason is that preliminary hashing can exhaust your RAM or disk space.
34302
34303 Properties:
34304
34305 • Config: speedup_max_disk
34306
34307 • Env Var: RCLONE_MAILRU_SPEEDUP_MAX_DISK
34308
34309 • Type: SizeSuffix
34310
34311 • Default: 3Gi
34312
34313 • Examples:
34314
34315 • "0"
34316
34317 • Completely disable speedup (put by hash).
34318
34319 • "1G"
34320
34321 • Files larger than 1Gb will be uploaded directly.
34322
34323 • "3G"
34324
34325 • Choose this option if you have less than 3Gb free on local disk.
34326
34327 --mailru-speedup-max-memory
34328 Files larger than the size given below will always be hashed on disk.
34329
34330 Properties:
34331
34332 • Config: speedup_max_memory
34333
34334 • Env Var: RCLONE_MAILRU_SPEEDUP_MAX_MEMORY
34335
34336 • Type: SizeSuffix
34337
34338 • Default: 32Mi
34339
34340 • Examples:
34341
34342 • "0"
34343
34344 • Preliminary hashing will always be done in a temporary disk loca‐
34345 tion.
34346
34347 • "32M"
34348
34349 • Do not dedicate more than 32Mb RAM for preliminary hashing.
34350
34351 • "256M"
34352
34353 • You have at most 256Mb RAM free for hash calculations.
34354
34355 --mailru-check-hash
34356 What should copy do if file checksum is mismatched or invalid.
34357
34358 Properties:
34359
34360 • Config: check_hash
34361
34362 • Env Var: RCLONE_MAILRU_CHECK_HASH
34363
34364 • Type: bool
34365
34366 • Default: true
34367
34368 • Examples:
34369
34370 • "true"
34371
34372 • Fail with error.
34373
34374 • "false"
34375
34376 • Ignore and continue.
34377
34378 --mailru-user-agent
34379 HTTP user agent used internally by client.
34380
34381 Defaults to "rclone/VERSION" or "--user-agent" provided on command
34382 line.
34383
34384 Properties:
34385
34386 • Config: user_agent
34387
34388 • Env Var: RCLONE_MAILRU_USER_AGENT
34389
34390 • Type: string
34391
34392 • Required: false
34393
34394 --mailru-quirks
34395 Comma separated list of internal maintenance flags.
34396
34397 This option must not be used by an ordinary user. It is intended only
34398 to facilitate remote troubleshooting of backend issues. Strict meaning
34399 of flags is not documented and not guaranteed to persist between re‐
34400 leases. Quirks will be removed when the backend grows stable. Sup‐
34401 ported quirks: atomicmkdir binlist unknowndirs
34402
34403 Properties:
34404
34405 • Config: quirks
34406
34407 • Env Var: RCLONE_MAILRU_QUIRKS
34408
34409 • Type: string
34410
34411 • Required: false
34412
34413 --mailru-encoding
34414 The encoding for the backend.
34415
34416 See the encoding section in the overview (https://rclone.org/over‐
34417 view/#encoding) for more info.
34418
34419 Properties:
34420
34421 • Config: encoding
34422
34423 • Env Var: RCLONE_MAILRU_ENCODING
34424
34425 • Type: MultiEncoder
34426
34427 • Default: Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,Back‐
34428 Slash,Del,Ctl,InvalidUtf8,Dot
34429
34430 Limitations
34431 File size limits depend on your account. A single file size is limited
34432 by 2G for a free account and unlimited for paid tariffs. Please refer
34433 to the Mail.ru site for the total uploaded size limits.
34434
34435 Note that Mailru is case insensitive so you can't have a file called
34436 "Hello.doc" and one called "hello.doc".
34437
34439 Mega (https://mega.nz/) is a cloud storage and file hosting service
34440 known for its security feature where all files are encrypted locally
34441 before they are uploaded. This prevents anyone (including employees of
34442 Mega) from accessing the files without knowledge of the key used for
34443 encryption.
34444
34445 This is an rclone backend for Mega which supports the file transfer
34446 features of Mega using the same client side encryption.
34447
34448 Paths are specified as remote:path
34449
34450 Paths may be as deep as required, e.g. remote:directory/subdirectory.
34451
34452 Configuration
34453 Here is an example of how to make a remote called remote. First run:
34454
34455 rclone config
34456
34457 This will guide you through an interactive setup process:
34458
34459 No remotes found, make a new one?
34460 n) New remote
34461 s) Set configuration password
34462 q) Quit config
34463 n/s/q> n
34464 name> remote
34465 Type of storage to configure.
34466 Choose a number from below, or type in your own value
34467 [snip]
34468 XX / Mega
34469 \ "mega"
34470 [snip]
34471 Storage> mega
34472 User name
34473 user> you@example.com
34474 Password.
34475 y) Yes type in my own password
34476 g) Generate random password
34477 n) No leave this optional password blank
34478 y/g/n> y
34479 Enter the password:
34480 password:
34481 Confirm the password:
34482 password:
34483 Remote config
34484 --------------------
34485 [remote]
34486 type = mega
34487 user = you@example.com
34488 pass = *** ENCRYPTED ***
34489 --------------------
34490 y) Yes this is OK
34491 e) Edit this remote
34492 d) Delete this remote
34493 y/e/d> y
34494
34495 NOTE: The encryption keys need to have been already generated after a
34496 regular login via the browser, otherwise attempting to use the creden‐
34497 tials in rclone will fail.
34498
34499 Once configured you can then use rclone like this,
34500
34501 List directories in top level of your Mega
34502
34503 rclone lsd remote:
34504
34505 List all the files in your Mega
34506
34507 rclone ls remote:
34508
34509 To copy a local directory to an Mega directory called backup
34510
34511 rclone copy /home/source remote:backup
34512
34513 Modified time and hashes
34514 Mega does not support modification times or hashes yet.
34515
34516 Restricted filename characters
34517 Character Value Replacement
34518 ────────────────────────────────
34519 NUL 0x00 ␀
34520 / 0x2F /
34521
34522 Invalid UTF-8 bytes will also be replaced (https://rclone.org/over‐
34523 view/#invalid-utf8), as they can't be used in JSON strings.
34524
34525 Duplicated files
34526 Mega can have two files with exactly the same name and path (unlike a
34527 normal file system).
34528
34529 Duplicated files cause problems with the syncing and you will see mes‐
34530 sages in the log about duplicates.
34531
34532 Use rclone dedupe to fix duplicated files.
34533
34534 Failure to log-in
34535 Object not found
34536 If you are connecting to your Mega remote for the first time, to test
34537 access and synchronization, you may receive an error such as
34538
34539 Failed to create file system for "my-mega-remote:":
34540 couldn't login: Object (typically, node or user) not found
34541
34542 The diagnostic steps often recommended in the rclone forum (https://fo‐
34543 rum.rclone.org/search?q=mega) start with the MEGAcmd utility. Note
34544 that this refers to the official C++ command from
34545 https://github.com/meganz/MEGAcmd and not the go language built command
34546 from t3rm1n4l/megacmd that is no longer maintained.
34547
34548 Follow the instructions for installing MEGAcmd and try accessing your
34549 remote as they recommend. You can establish whether or not you can log
34550 in using MEGAcmd, and obtain diagnostic information to help you, and
34551 search or work with others in the forum.
34552
34553 MEGA CMD> login me@example.com
34554 Password:
34555 Fetching nodes ...
34556 Loading transfers from local cache
34557 Login complete as me@example.com
34558 me@example.com:/$
34559
34560 Note that some have found issues with passwords containing special
34561 characters. If you can not log on with rclone, but MEGAcmd logs on
34562 just fine, then consider changing your password temporarily to pure al‐
34563 phanumeric characters, in case that helps.
34564
34565 Repeated commands blocks access
34566 Mega remotes seem to get blocked (reject logins) under "heavy use". We
34567 haven't worked out the exact blocking rules but it seems to be related
34568 to fast paced, successive rclone commands.
34569
34570 For example, executing this command 90 times in a row rclone link re‐
34571 mote:file will cause the remote to become "blocked". This is not an
34572 abnormal situation, for example if you wish to get the public links of
34573 a directory with hundred of files... After more or less a week, the
34574 remote will remote accept rclone logins normally again.
34575
34576 You can mitigate this issue by mounting the remote it with rclone
34577 mount. This will log-in when mounting and a log-out when unmounting
34578 only. You can also run rclone rcd and then use rclone rc to run the
34579 commands over the API to avoid logging in each time.
34580
34581 Rclone does not currently close mega sessions (you can see them in the
34582 web interface), however closing the sessions does not solve the issue.
34583
34584 If you space rclone commands by 3 seconds it will avoid blocking the
34585 remote. We haven't identified the exact blocking rules, so perhaps one
34586 could execute the command 80 times without waiting and avoid blocking
34587 by waiting 3 seconds, then continuing...
34588
34589 Note that this has been observed by trial and error and might not be
34590 set in stone.
34591
34592 Other tools seem not to produce this blocking effect, as they use a
34593 different working approach (state-based, using sessionIDs instead of
34594 log-in) which isn't compatible with the current stateless rclone ap‐
34595 proach.
34596
34597 Note that once blocked, the use of other tools (such as megacmd) is not
34598 a sure workaround: following megacmd login times have been observed in
34599 succession for blocked remote: 7 minutes, 20 min, 30min, 30 min, 30min.
34600 Web access looks unaffected though.
34601
34602 Investigation is continuing in relation to workarounds based on time‐
34603 outs, pacers, retrials and tpslimits - if you discover something rele‐
34604 vant, please post on the forum.
34605
34606 So, if rclone was working nicely and suddenly you are unable to log-in
34607 and you are sure the user and the password are correct, likely you have
34608 got the remote blocked for a while.
34609
34610 Standard options
34611 Here are the Standard options specific to mega (Mega).
34612
34613 --mega-user
34614 User name.
34615
34616 Properties:
34617
34618 • Config: user
34619
34620 • Env Var: RCLONE_MEGA_USER
34621
34622 • Type: string
34623
34624 • Required: true
34625
34626 --mega-pass
34627 Password.
34628
34629 NB Input to this must be obscured - see rclone obscure
34630 (https://rclone.org/commands/rclone_obscure/).
34631
34632 Properties:
34633
34634 • Config: pass
34635
34636 • Env Var: RCLONE_MEGA_PASS
34637
34638 • Type: string
34639
34640 • Required: true
34641
34642 Advanced options
34643 Here are the Advanced options specific to mega (Mega).
34644
34645 --mega-debug
34646 Output more debug from Mega.
34647
34648 If this flag is set (along with -vv) it will print further debugging
34649 information from the mega backend.
34650
34651 Properties:
34652
34653 • Config: debug
34654
34655 • Env Var: RCLONE_MEGA_DEBUG
34656
34657 • Type: bool
34658
34659 • Default: false
34660
34661 --mega-hard-delete
34662 Delete files permanently rather than putting them into the trash.
34663
34664 Normally the mega backend will put all deletions into the trash rather
34665 than permanently deleting them. If you specify this then rclone will
34666 permanently delete objects instead.
34667
34668 Properties:
34669
34670 • Config: hard_delete
34671
34672 • Env Var: RCLONE_MEGA_HARD_DELETE
34673
34674 • Type: bool
34675
34676 • Default: false
34677
34678 --mega-encoding
34679 The encoding for the backend.
34680
34681 See the encoding section in the overview (https://rclone.org/over‐
34682 view/#encoding) for more info.
34683
34684 Properties:
34685
34686 • Config: encoding
34687
34688 • Env Var: RCLONE_MEGA_ENCODING
34689
34690 • Type: MultiEncoder
34691
34692 • Default: Slash,InvalidUtf8,Dot
34693
34694 Limitations
34695 This backend uses the go-mega go library
34696 (https://github.com/t3rm1n4l/go-mega) which is an opensource go library
34697 implementing the Mega API. There doesn't appear to be any documenta‐
34698 tion for the mega protocol beyond the mega C++ SDK
34699 (https://github.com/meganz/sdk) source code so there are likely quite a
34700 few errors still remaining in this library.
34701
34702 Mega allows duplicate files which may confuse rclone.
34703
34705 The memory backend is an in RAM backend. It does not persist its data
34706 - use the local backend for that.
34707
34708 The memory backend behaves like a bucket-based remote (e.g. like s3).
34709 Because it has no parameters you can just use it with the :memory: re‐
34710 mote name.
34711
34712 Configuration
34713 You can configure it as a remote like this with rclone config too if
34714 you want to:
34715
34716 No remotes found, make a new one?
34717 n) New remote
34718 s) Set configuration password
34719 q) Quit config
34720 n/s/q> n
34721 name> remote
34722 Type of storage to configure.
34723 Enter a string value. Press Enter for the default ("").
34724 Choose a number from below, or type in your own value
34725 [snip]
34726 XX / Memory
34727 \ "memory"
34728 [snip]
34729 Storage> memory
34730 ** See help for memory backend at: https://rclone.org/memory/ **
34731
34732 Remote config
34733
34734 --------------------
34735 [remote]
34736 type = memory
34737 --------------------
34738 y) Yes this is OK (default)
34739 e) Edit this remote
34740 d) Delete this remote
34741 y/e/d> y
34742
34743 Because the memory backend isn't persistent it is most useful for test‐
34744 ing or with an rclone server or rclone mount, e.g.
34745
34746 rclone mount :memory: /mnt/tmp
34747 rclone serve webdav :memory:
34748 rclone serve sftp :memory:
34749
34750 Modified time and hashes
34751 The memory backend supports MD5 hashes and modification times accurate
34752 to 1 nS.
34753
34754 Restricted filename characters
34755 The memory backend replaces the default restricted characters set
34756 (https://rclone.org/overview/#restricted-characters).
34757
34759 Paths are specified as remote: You may put subdirectories in too, e.g.
34760 remote:/path/to/dir. If you have a CP code you can use that as the
34761 folder after the domain such as <domain>/<cpcode>/<internal directories
34762 within cpcode>.
34763
34764 For example, this is commonly configured with or without a CP code: *
34765 With a CP code. [your-domain-prefix]-nsu.akamaihd.net/123456/subdirec‐
34766 tory/ * Without a CP code. [your-domain-prefix]-nsu.akamaihd.net
34767
34768 See all buckets rclone lsd remote: The initial setup for Netstorage in‐
34769 volves getting an account and secret. Use rclone config to walk you
34770 through the setup process.
34771
34772 Configuration
34773 Here's an example of how to make a remote called ns1.
34774
34775 1. To begin the interactive configuration process, enter this command:
34776
34777 rclone config
34778
34779 2. Type n to create a new remote.
34780
34781 n) New remote
34782 d) Delete remote
34783 q) Quit config
34784 e/n/d/q> n
34785
34786 3. For this example, enter ns1 when you reach the name> prompt.
34787
34788 name> ns1
34789
34790 4. Enter netstorage as the type of storage to configure.
34791
34792 Type of storage to configure.
34793 Enter a string value. Press Enter for the default ("").
34794 Choose a number from below, or type in your own value
34795 XX / NetStorage
34796 \ "netstorage"
34797 Storage> netstorage
34798
34799 5. Select between the HTTP or HTTPS protocol. Most users should choose
34800 HTTPS, which is the default. HTTP is provided primarily for debug‐
34801 ging purposes.
34802
34803 Enter a string value. Press Enter for the default ("").
34804 Choose a number from below, or type in your own value
34805 1 / HTTP protocol
34806 \ "http"
34807 2 / HTTPS protocol
34808 \ "https"
34809 protocol> 1
34810
34811 6. Specify your NetStorage host, CP code, and any necessary content
34812 paths using this format: <domain>/<cpcode>/<content>/
34813
34814 Enter a string value. Press Enter for the default ("").
34815 host> baseball-nsu.akamaihd.net/123456/content/
34816
34817 7. Set the netstorage account name
34818
34819 Enter a string value. Press Enter for the default ("").
34820 account> username
34821
34822 8. Set the Netstorage account secret/G2O key which will be used for au‐
34823 thentication purposes. Select the y option to set your own password
34824 then enter your secret. Note: The secret is stored in the
34825 rclone.conf file with hex-encoded encryption.
34826
34827 y) Yes type in my own password
34828 g) Generate random password
34829 y/g> y
34830 Enter the password:
34831 password:
34832 Confirm the password:
34833 password:
34834
34835 9. View the summary and confirm your remote configuration.
34836
34837 [ns1]
34838 type = netstorage
34839 protocol = http
34840 host = baseball-nsu.akamaihd.net/123456/content/
34841 account = username
34842 secret = *** ENCRYPTED ***
34843 --------------------
34844 y) Yes this is OK (default)
34845 e) Edit this remote
34846 d) Delete this remote
34847 y/e/d> y
34848
34849 This remote is called ns1 and can now be used.
34850
34851 Example operations
34852 Get started with rclone and NetStorage with these examples. For addi‐
34853 tional rclone commands, visit https://rclone.org/commands/.
34854
34855 See contents of a directory in your project
34856 rclone lsd ns1:/974012/testing/
34857
34858 Sync the contents local with remote
34859 rclone sync . ns1:/974012/testing/
34860
34861 Upload local content to remote
34862 rclone copy notes.txt ns1:/974012/testing/
34863
34864 Delete content on remote
34865 rclone delete ns1:/974012/testing/notes.txt
34866
34867 Move or copy content between CP codes.
34868 Your credentials must have access to two CP codes on the same remote.
34869 You can't perform operations between different remotes.
34870
34871 rclone move ns1:/974012/testing/notes.txt ns1:/974450/testing2/
34872
34873 Features
34874 Symlink Support
34875 The Netstorage backend changes the rclone --links, -l behavior. When
34876 uploading, instead of creating the .rclonelink file, use the "symlink"
34877 API in order to create the corresponding symlink on the remote. The
34878 .rclonelink file will not be created, the upload will be intercepted
34879 and only the symlink file that matches the source file name with no
34880 suffix will be created on the remote.
34881
34882 This will effectively allow commands like copy/copyto, move/moveto and
34883 sync to upload from local to remote and download from remote to local
34884 directories with symlinks. Due to internal rclone limitations, it is
34885 not possible to upload an individual symlink file to any remote back‐
34886 end. You can always use the "backend symlink" command to create a sym‐
34887 link on the NetStorage server, refer to "symlink" section below.
34888
34889 Individual symlink files on the remote can be used with the commands
34890 like "cat" to print the destination name, or "delete" to delete sym‐
34891 link, or copy, copy/to and move/moveto to download from the remote to
34892 local. Note: individual symlink files on the remote should be speci‐
34893 fied including the suffix .rclonelink.
34894
34895 Note: No file with the suffix .rclonelink should ever exist on the
34896 server since it is not possible to actually upload/create a file with
34897 .rclonelink suffix with rclone, it can only exist if it is manually
34898 created through a non-rclone method on the remote.
34899
34900 Implicit vs. Explicit Directories
34901 With NetStorage, directories can exist in one of two forms:
34902
34903 1. Explicit Directory. This is an actual, physical directory that you
34904 have created in a storage group.
34905
34906 2. Implicit Directory. This refers to a directory within a path that
34907 has not been physically created. For example, during upload of a
34908 file, nonexistent subdirectories can be specified in the target
34909 path. NetStorage creates these as "implicit." While the directories
34910 aren't physically created, they exist implicitly and the noted path
34911 is connected with the uploaded file.
34912
34913 Rclone will intercept all file uploads and mkdir commands for the Net‐
34914 Storage remote and will explicitly issue the mkdir command for each di‐
34915 rectory in the uploading path. This will help with the interoperabili‐
34916 ty with the other Akamai services such as SFTP and the Content Manage‐
34917 ment Shell (CMShell). Rclone will not guarantee correctness of opera‐
34918 tions with implicit directories which might have been created as a re‐
34919 sult of using an upload API directly.
34920
34921 --fast-list / ListR support
34922 NetStorage remote supports the ListR feature by using the "list" Net‐
34923 Storage API action to return a lexicographical list of all objects
34924 within the specified CP code, recursing into subdirectories as they're
34925 encountered.
34926
34927 • Rclone will use the ListR method for some commands by default. Com‐
34928 mands such as lsf -R will use ListR by default. To disable this, in‐
34929 clude the --disable listR option to use the non-recursive method of
34930 listing objects.
34931
34932 • Rclone will not use the ListR method for some commands. Commands
34933 such as sync don't use ListR by default. To force using the ListR
34934 method, include the --fast-list option.
34935
34936 There are pros and cons of using the ListR method, refer to rclone doc‐
34937 umentation (https://rclone.org/docs/#fast-list). In general, the sync
34938 command over an existing deep tree on the remote will run faster with
34939 the "--fast-list" flag but with extra memory usage as a side effect.
34940 It might also result in higher CPU utilization but the whole task can
34941 be completed faster.
34942
34943 Note: There is a known limitation that "lsf -R" will display number of
34944 files in the directory and directory size as -1 when ListR method is
34945 used. The workaround is to pass "--disable listR" flag if these num‐
34946 bers are important in the output.
34947
34948 Purge
34949 NetStorage remote supports the purge feature by using the "quick-
34950 delete" NetStorage API action. The quick-delete action is disabled by
34951 default for security reasons and can be enabled for the account through
34952 the Akamai portal. Rclone will first try to use quick-delete action
34953 for the purge command and if this functionality is disabled then will
34954 fall back to a standard delete method.
34955
34956 Note: Read the NetStorage Usage API (https://learn.akamai.com/en-
34957 us/webhelp/netstorage/netstorage-http-api-developer-
34958 guide/GUID-15836617-9F50-405A-833C-EA2556756A30.html) for considera‐
34959 tions when using "quick-delete". In general, using quick-delete method
34960 will not delete the tree immediately and objects targeted for quick-
34961 delete may still be accessible.
34962
34963 Standard options
34964 Here are the Standard options specific to netstorage (Akamai NetStor‐
34965 age).
34966
34967 --netstorage-host
34968 Domain+path of NetStorage host to connect to.
34969
34970 Format should be <domain>/<internal folders>
34971
34972 Properties:
34973
34974 • Config: host
34975
34976 • Env Var: RCLONE_NETSTORAGE_HOST
34977
34978 • Type: string
34979
34980 • Required: true
34981
34982 --netstorage-account
34983 Set the NetStorage account name
34984
34985 Properties:
34986
34987 • Config: account
34988
34989 • Env Var: RCLONE_NETSTORAGE_ACCOUNT
34990
34991 • Type: string
34992
34993 • Required: true
34994
34995 --netstorage-secret
34996 Set the NetStorage account secret/G2O key for authentication.
34997
34998 Please choose the 'y' option to set your own password then enter your
34999 secret.
35000
35001 NB Input to this must be obscured - see rclone obscure
35002 (https://rclone.org/commands/rclone_obscure/).
35003
35004 Properties:
35005
35006 • Config: secret
35007
35008 • Env Var: RCLONE_NETSTORAGE_SECRET
35009
35010 • Type: string
35011
35012 • Required: true
35013
35014 Advanced options
35015 Here are the Advanced options specific to netstorage (Akamai NetStor‐
35016 age).
35017
35018 --netstorage-protocol
35019 Select between HTTP or HTTPS protocol.
35020
35021 Most users should choose HTTPS, which is the default. HTTP is provided
35022 primarily for debugging purposes.
35023
35024 Properties:
35025
35026 • Config: protocol
35027
35028 • Env Var: RCLONE_NETSTORAGE_PROTOCOL
35029
35030 • Type: string
35031
35032 • Default: "https"
35033
35034 • Examples:
35035
35036 • "http"
35037
35038 • HTTP protocol
35039
35040 • "https"
35041
35042 • HTTPS protocol
35043
35044 Backend commands
35045 Here are the commands specific to the netstorage backend.
35046
35047 Run them with
35048
35049 rclone backend COMMAND remote:
35050
35051 The help below will explain what arguments each command takes.
35052
35053 See the backend (https://rclone.org/commands/rclone_backend/) command
35054 for more info on how to pass options and arguments.
35055
35056 These can be run on a running backend using the rc command backend/com‐
35057 mand (https://rclone.org/rc/#backend-command).
35058
35059 du
35060 Return disk usage information for a specified directory
35061
35062 rclone backend du remote: [options] [<arguments>+]
35063
35064 The usage information returned, includes the targeted directory as well
35065 as all files stored in any sub-directories that may exist.
35066
35067 symlink
35068 You can create a symbolic link in ObjectStore with the symlink action.
35069
35070 rclone backend symlink remote: [options] [<arguments>+]
35071
35072 The desired path location (including applicable sub-directories) ending
35073 in the object that will be the target of the symlink (for example,
35074 /links/mylink). Include the file extension for the object, if applica‐
35075 ble. rclone backend symlink <src> <path>
35076
35078 Paths are specified as remote:container (or remote: for the lsd com‐
35079 mand.) You may put subdirectories in too, e.g. remote:contain‐
35080 er/path/to/dir.
35081
35082 Configuration
35083 Here is an example of making a Microsoft Azure Blob Storage configura‐
35084 tion. For a remote called remote. First run:
35085
35086 rclone config
35087
35088 This will guide you through an interactive setup process:
35089
35090 No remotes found, make a new one?
35091 n) New remote
35092 s) Set configuration password
35093 q) Quit config
35094 n/s/q> n
35095 name> remote
35096 Type of storage to configure.
35097 Choose a number from below, or type in your own value
35098 [snip]
35099 XX / Microsoft Azure Blob Storage
35100 \ "azureblob"
35101 [snip]
35102 Storage> azureblob
35103 Storage Account Name
35104 account> account_name
35105 Storage Account Key
35106 key> base64encodedkey==
35107 Endpoint for the service - leave blank normally.
35108 endpoint>
35109 Remote config
35110 --------------------
35111 [remote]
35112 account = account_name
35113 key = base64encodedkey==
35114 endpoint =
35115 --------------------
35116 y) Yes this is OK
35117 e) Edit this remote
35118 d) Delete this remote
35119 y/e/d> y
35120
35121 See all containers
35122
35123 rclone lsd remote:
35124
35125 Make a new container
35126
35127 rclone mkdir remote:container
35128
35129 List the contents of a container
35130
35131 rclone ls remote:container
35132
35133 Sync /home/local/directory to the remote container, deleting any excess
35134 files in the container.
35135
35136 rclone sync -i /home/local/directory remote:container
35137
35138 --fast-list
35139 This remote supports --fast-list which allows you to use fewer transac‐
35140 tions in exchange for more memory. See the rclone docs
35141 (https://rclone.org/docs/#fast-list) for more details.
35142
35143 Modified time
35144 The modified time is stored as metadata on the object with the mtime
35145 key. It is stored using RFC3339 Format time with nanosecond precision.
35146 The metadata is supplied during directory listings so there is no over‐
35147 head to using it.
35148
35149 Performance
35150 When uploading large files, increasing the value of --azureblob-upload-
35151 concurrency will increase performance at the cost of using more memory.
35152 The default of 16 is set quite conservatively to use less memory. It
35153 maybe be necessary raise it to 64 or higher to fully utilize a 1 GBit/s
35154 link with a single file transfer.
35155
35156 Restricted filename characters
35157 In addition to the default restricted characters set
35158 (https://rclone.org/overview/#restricted-characters) the following
35159 characters are also replaced:
35160
35161 Character Value Replacement
35162 ────────────────────────────────
35163 / 0x2F /
35164 \ 0x5C \
35165
35166 File names can also not end with the following characters. These only
35167 get replaced if they are the last character in the name:
35168
35169 Character Value Replacement
35170 ────────────────────────────────
35171 . 0x2E .
35172
35173 Invalid UTF-8 bytes will also be replaced (https://rclone.org/over‐
35174 view/#invalid-utf8), as they can't be used in JSON strings.
35175
35176 Hashes
35177 MD5 hashes are stored with blobs. However blobs that were uploaded in
35178 chunks only have an MD5 if the source remote was capable of MD5 hashes,
35179 e.g. the local disk.
35180
35181 Authenticating with Azure Blob Storage
35182 Rclone has 3 ways of authenticating with Azure Blob Storage:
35183
35184 Account and Key
35185 This is the most straight forward and least flexible way. Just fill in
35186 the account and key lines and leave the rest blank.
35187
35188 SAS URL
35189 This can be an account level SAS URL or container level SAS URL.
35190
35191 To use it leave account, key blank and fill in sas_url.
35192
35193 An account level SAS URL or container level SAS URL can be obtained
35194 from the Azure portal or the Azure Storage Explorer. To get a contain‐
35195 er level SAS URL right click on a container in the Azure Blob explorer
35196 in the Azure portal.
35197
35198 If you use a container level SAS URL, rclone operations are permitted
35199 only on a particular container, e.g.
35200
35201 rclone ls azureblob:container
35202
35203 You can also list the single container from the root. This will only
35204 show the container specified by the SAS URL.
35205
35206 $ rclone lsd azureblob:
35207 container/
35208
35209 Note that you can't see or access any other containers - this will fail
35210
35211 rclone ls azureblob:othercontainer
35212
35213 Container level SAS URLs are useful for temporarily allowing third par‐
35214 ties access to a single container or putting credentials into an un‐
35215 trusted environment such as a CI build server.
35216
35217 Standard options
35218 Here are the Standard options specific to azureblob (Microsoft Azure
35219 Blob Storage).
35220
35221 --azureblob-account
35222 Storage Account Name.
35223
35224 Leave blank to use SAS URL or Emulator.
35225
35226 Properties:
35227
35228 • Config: account
35229
35230 • Env Var: RCLONE_AZUREBLOB_ACCOUNT
35231
35232 • Type: string
35233
35234 • Required: false
35235
35236 --azureblob-service-principal-file
35237 Path to file containing credentials for use with a service principal.
35238
35239 Leave blank normally. Needed only if you want to use a service princi‐
35240 pal instead of interactive login.
35241
35242 $ az ad sp create-for-rbac --name "<name>" \
35243 --role "Storage Blob Data Owner" \
35244 --scopes "/subscriptions/<subscription>/resourceGroups/<resource-group>/providers/Microsoft.Storage/storageAccounts/<storage-account>/blobServices/default/containers/<container>" \
35245 > azure-principal.json
35246
35247 See "Create an Azure service principal" (https://docs.microsoft.com/en-
35248 us/cli/azure/create-an-azure-service-principal-azure-cli) and "Assign
35249 an Azure role for access to blob data" (https://docs.microsoft.com/en-
35250 us/azure/storage/common/storage-auth-aad-rbac-cli) pages for more de‐
35251 tails.
35252
35253 Properties:
35254
35255 • Config: service_principal_file
35256
35257 • Env Var: RCLONE_AZUREBLOB_SERVICE_PRINCIPAL_FILE
35258
35259 • Type: string
35260
35261 • Required: false
35262
35263 --azureblob-key
35264 Storage Account Key.
35265
35266 Leave blank to use SAS URL or Emulator.
35267
35268 Properties:
35269
35270 • Config: key
35271
35272 • Env Var: RCLONE_AZUREBLOB_KEY
35273
35274 • Type: string
35275
35276 • Required: false
35277
35278 --azureblob-sas-url
35279 SAS URL for container level access only.
35280
35281 Leave blank if using account/key or Emulator.
35282
35283 Properties:
35284
35285 • Config: sas_url
35286
35287 • Env Var: RCLONE_AZUREBLOB_SAS_URL
35288
35289 • Type: string
35290
35291 • Required: false
35292
35293 --azureblob-use-msi
35294 Use a managed service identity to authenticate (only works in Azure).
35295
35296 When true, use a managed service identity (https://docs.micro‐
35297 soft.com/en-us/azure/active-directory/managed-identities-azure-re‐
35298 sources/) to authenticate to Azure Storage instead of a SAS token or
35299 account key.
35300
35301 If the VM(SS) on which this program is running has a system-assigned
35302 identity, it will be used by default. If the resource has no system-
35303 assigned but exactly one user-assigned identity, the user-assigned
35304 identity will be used by default. If the resource has multiple user-
35305 assigned identities, the identity to use must be explicitly specified
35306 using exactly one of the msi_object_id, msi_client_id, or msi_mi_res_id
35307 parameters.
35308
35309 Properties:
35310
35311 • Config: use_msi
35312
35313 • Env Var: RCLONE_AZUREBLOB_USE_MSI
35314
35315 • Type: bool
35316
35317 • Default: false
35318
35319 --azureblob-use-emulator
35320 Uses local storage emulator if provided as 'true'.
35321
35322 Leave blank if using real azure storage endpoint.
35323
35324 Properties:
35325
35326 • Config: use_emulator
35327
35328 • Env Var: RCLONE_AZUREBLOB_USE_EMULATOR
35329
35330 • Type: bool
35331
35332 • Default: false
35333
35334 Advanced options
35335 Here are the Advanced options specific to azureblob (Microsoft Azure
35336 Blob Storage).
35337
35338 --azureblob-msi-object-id
35339 Object ID of the user-assigned MSI to use, if any.
35340
35341 Leave blank if msi_client_id or msi_mi_res_id specified.
35342
35343 Properties:
35344
35345 • Config: msi_object_id
35346
35347 • Env Var: RCLONE_AZUREBLOB_MSI_OBJECT_ID
35348
35349 • Type: string
35350
35351 • Required: false
35352
35353 --azureblob-msi-client-id
35354 Object ID of the user-assigned MSI to use, if any.
35355
35356 Leave blank if msi_object_id or msi_mi_res_id specified.
35357
35358 Properties:
35359
35360 • Config: msi_client_id
35361
35362 • Env Var: RCLONE_AZUREBLOB_MSI_CLIENT_ID
35363
35364 • Type: string
35365
35366 • Required: false
35367
35368 --azureblob-msi-mi-res-id
35369 Azure resource ID of the user-assigned MSI to use, if any.
35370
35371 Leave blank if msi_client_id or msi_object_id specified.
35372
35373 Properties:
35374
35375 • Config: msi_mi_res_id
35376
35377 • Env Var: RCLONE_AZUREBLOB_MSI_MI_RES_ID
35378
35379 • Type: string
35380
35381 • Required: false
35382
35383 --azureblob-endpoint
35384 Endpoint for the service.
35385
35386 Leave blank normally.
35387
35388 Properties:
35389
35390 • Config: endpoint
35391
35392 • Env Var: RCLONE_AZUREBLOB_ENDPOINT
35393
35394 • Type: string
35395
35396 • Required: false
35397
35398 --azureblob-upload-cutoff
35399 Cutoff for switching to chunked upload (<= 256 MiB) (deprecated).
35400
35401 Properties:
35402
35403 • Config: upload_cutoff
35404
35405 • Env Var: RCLONE_AZUREBLOB_UPLOAD_CUTOFF
35406
35407 • Type: string
35408
35409 • Required: false
35410
35411 --azureblob-chunk-size
35412 Upload chunk size.
35413
35414 Note that this is stored in memory and there may be up to "--transfers"
35415 * "--azureblob-upload-concurrency" chunks stored at once in memory.
35416
35417 Properties:
35418
35419 • Config: chunk_size
35420
35421 • Env Var: RCLONE_AZUREBLOB_CHUNK_SIZE
35422
35423 • Type: SizeSuffix
35424
35425 • Default: 4Mi
35426
35427 --azureblob-upload-concurrency
35428 Concurrency for multipart uploads.
35429
35430 This is the number of chunks of the same file that are uploaded concur‐
35431 rently.
35432
35433 If you are uploading small numbers of large files over high-speed links
35434 and these uploads do not fully utilize your bandwidth, then increasing
35435 this may help to speed up the transfers.
35436
35437 In tests, upload speed increases almost linearly with upload concurren‐
35438 cy. For example to fill a gigabit pipe it may be necessary to raise
35439 this to 64. Note that this will use more memory.
35440
35441 Note that chunks are stored in memory and there may be up to "--trans‐
35442 fers" * "--azureblob-upload-concurrency" chunks stored at once in memo‐
35443 ry.
35444
35445 Properties:
35446
35447 • Config: upload_concurrency
35448
35449 • Env Var: RCLONE_AZUREBLOB_UPLOAD_CONCURRENCY
35450
35451 • Type: int
35452
35453 • Default: 16
35454
35455 --azureblob-list-chunk
35456 Size of blob list.
35457
35458 This sets the number of blobs requested in each listing chunk. Default
35459 is the maximum, 5000. "List blobs" requests are permitted 2 minutes
35460 per megabyte to complete. If an operation is taking longer than 2 min‐
35461 utes per megabyte on average, it will time out ( source
35462 (https://docs.microsoft.com/en-us/rest/api/storageservices/setting-
35463 timeouts-for-blob-service-operations#exceptions-to-default-timeout-in‐
35464 terval) ). This can be used to limit the number of blobs items to re‐
35465 turn, to avoid the time out.
35466
35467 Properties:
35468
35469 • Config: list_chunk
35470
35471 • Env Var: RCLONE_AZUREBLOB_LIST_CHUNK
35472
35473 • Type: int
35474
35475 • Default: 5000
35476
35477 --azureblob-access-tier
35478 Access tier of blob: hot, cool or archive.
35479
35480 Archived blobs can be restored by setting access tier to hot or cool.
35481 Leave blank if you intend to use default access tier, which is set at
35482 account level
35483
35484 If there is no "access tier" specified, rclone doesn't apply any tier.
35485 rclone performs "Set Tier" operation on blobs while uploading, if ob‐
35486 jects are not modified, specifying "access tier" to new one will have
35487 no effect. If blobs are in "archive tier" at remote, trying to perform
35488 data transfer operations from remote will not be allowed. User should
35489 first restore by tiering blob to "Hot" or "Cool".
35490
35491 Properties:
35492
35493 • Config: access_tier
35494
35495 • Env Var: RCLONE_AZUREBLOB_ACCESS_TIER
35496
35497 • Type: string
35498
35499 • Required: false
35500
35501 --azureblob-archive-tier-delete
35502 Delete archive tier blobs before overwriting.
35503
35504 Archive tier blobs cannot be updated. So without this flag, if you at‐
35505 tempt to update an archive tier blob, then rclone will produce the er‐
35506 ror:
35507
35508 can't update archive tier blob without --azureblob-archive-tier-delete
35509
35510 With this flag set then before rclone attempts to overwrite an archive
35511 tier blob, it will delete the existing blob before uploading its re‐
35512 placement. This has the potential for data loss if the upload fails
35513 (unlike updating a normal blob) and also may cost more since deleting
35514 archive tier blobs early may be chargable.
35515
35516 Properties:
35517
35518 • Config: archive_tier_delete
35519
35520 • Env Var: RCLONE_AZUREBLOB_ARCHIVE_TIER_DELETE
35521
35522 • Type: bool
35523
35524 • Default: false
35525
35526 --azureblob-disable-checksum
35527 Don't store MD5 checksum with object metadata.
35528
35529 Normally rclone will calculate the MD5 checksum of the input before up‐
35530 loading it so it can add it to metadata on the object. This is great
35531 for data integrity checking but can cause long delays for large files
35532 to start uploading.
35533
35534 Properties:
35535
35536 • Config: disable_checksum
35537
35538 • Env Var: RCLONE_AZUREBLOB_DISABLE_CHECKSUM
35539
35540 • Type: bool
35541
35542 • Default: false
35543
35544 --azureblob-memory-pool-flush-time
35545 How often internal memory buffer pools will be flushed.
35546
35547 Uploads which requires additional buffers (f.e multipart) will use mem‐
35548 ory pool for allocations. This option controls how often unused buf‐
35549 fers will be removed from the pool.
35550
35551 Properties:
35552
35553 • Config: memory_pool_flush_time
35554
35555 • Env Var: RCLONE_AZUREBLOB_MEMORY_POOL_FLUSH_TIME
35556
35557 • Type: Duration
35558
35559 • Default: 1m0s
35560
35561 --azureblob-memory-pool-use-mmap
35562 Whether to use mmap buffers in internal memory pool.
35563
35564 Properties:
35565
35566 • Config: memory_pool_use_mmap
35567
35568 • Env Var: RCLONE_AZUREBLOB_MEMORY_POOL_USE_MMAP
35569
35570 • Type: bool
35571
35572 • Default: false
35573
35574 --azureblob-encoding
35575 The encoding for the backend.
35576
35577 See the encoding section in the overview (https://rclone.org/over‐
35578 view/#encoding) for more info.
35579
35580 Properties:
35581
35582 • Config: encoding
35583
35584 • Env Var: RCLONE_AZUREBLOB_ENCODING
35585
35586 • Type: MultiEncoder
35587
35588 • Default: Slash,BackSlash,Del,Ctl,RightPeriod,InvalidUtf8
35589
35590 --azureblob-public-access
35591 Public access level of a container: blob or container.
35592
35593 Properties:
35594
35595 • Config: public_access
35596
35597 • Env Var: RCLONE_AZUREBLOB_PUBLIC_ACCESS
35598
35599 • Type: string
35600
35601 • Required: false
35602
35603 • Examples:
35604
35605 • ""
35606
35607 • The container and its blobs can be accessed only with an autho‐
35608 rized request.
35609
35610 • It's a default value.
35611
35612 • "blob"
35613
35614 • Blob data within this container can be read via anonymous re‐
35615 quest.
35616
35617 • "container"
35618
35619 • Allow full public read access for container and blob data.
35620
35621 --azureblob-no-head-object
35622 If set, do not do HEAD before GET when getting objects.
35623
35624 Properties:
35625
35626 • Config: no_head_object
35627
35628 • Env Var: RCLONE_AZUREBLOB_NO_HEAD_OBJECT
35629
35630 • Type: bool
35631
35632 • Default: false
35633
35634 Limitations
35635 MD5 sums are only uploaded with chunked files if the source has an MD5
35636 sum. This will always be the case for a local to azure copy.
35637
35638 rclone about is not supported by the Microsoft Azure Blob storage back‐
35639 end. Backends without this capability cannot determine free space for
35640 an rclone mount or use policy mfs (most free space) as a member of an
35641 rclone union remote.
35642
35643 See List of backends that do not support rclone about
35644 (https://rclone.org/overview/#optional-features) and rclone about
35645 (https://rclone.org/commands/rclone_about/)
35646
35647 Azure Storage Emulator Support
35648 You can run rclone with storage emulator (usually azurite).
35649
35650 To do this, just set up a new remote with rclone config following in‐
35651 structions described in introduction and set use_emulator config as
35652 true. You do not need to provide default account name neither an ac‐
35653 count key.
35654
35655 Also, if you want to access a storage emulator instance running on a
35656 different machine, you can override Endpoint parameter in advanced set‐
35657 tings, setting it to http(s)://<host>:<port>/devstoreaccount1 (e.g.
35658 http://10.254.2.5:10000/devstoreaccount1).
35659
35661 Paths are specified as remote:path
35662
35663 Paths may be as deep as required, e.g. remote:directory/subdirectory.
35664
35665 Configuration
35666 The initial setup for OneDrive involves getting a token from Microsoft
35667 which you need to do in your browser. rclone config walks you through
35668 it.
35669
35670 Here is an example of how to make a remote called remote. First run:
35671
35672 rclone config
35673
35674 This will guide you through an interactive setup process:
35675
35676 e) Edit existing remote
35677 n) New remote
35678 d) Delete remote
35679 r) Rename remote
35680 c) Copy remote
35681 s) Set configuration password
35682 q) Quit config
35683 e/n/d/r/c/s/q> n
35684 name> remote
35685 Type of storage to configure.
35686 Enter a string value. Press Enter for the default ("").
35687 Choose a number from below, or type in your own value
35688 [snip]
35689 XX / Microsoft OneDrive
35690 \ "onedrive"
35691 [snip]
35692 Storage> onedrive
35693 Microsoft App Client Id
35694 Leave blank normally.
35695 Enter a string value. Press Enter for the default ("").
35696 client_id>
35697 Microsoft App Client Secret
35698 Leave blank normally.
35699 Enter a string value. Press Enter for the default ("").
35700 client_secret>
35701 Edit advanced config? (y/n)
35702 y) Yes
35703 n) No
35704 y/n> n
35705 Remote config
35706 Use auto config?
35707 * Say Y if not sure
35708 * Say N if you are working on a remote or headless machine
35709 y) Yes
35710 n) No
35711 y/n> y
35712 If your browser doesn't open automatically go to the following link: http://127.0.0.1:53682/auth
35713 Log in and authorize rclone for access
35714 Waiting for code...
35715 Got code
35716 Choose a number from below, or type in an existing value
35717 1 / OneDrive Personal or Business
35718 \ "onedrive"
35719 2 / Sharepoint site
35720 \ "sharepoint"
35721 3 / Type in driveID
35722 \ "driveid"
35723 4 / Type in SiteID
35724 \ "siteid"
35725 5 / Search a Sharepoint site
35726 \ "search"
35727 Your choice> 1
35728 Found 1 drives, please select the one you want to use:
35729 0: OneDrive (business) id=b!Eqwertyuiopasdfghjklzxcvbnm-7mnbvcxzlkjhgfdsapoiuytrewqk
35730 Chose drive to use:> 0
35731 Found drive 'root' of type 'business', URL: https://org-my.sharepoint.com/personal/you/Documents
35732 Is that okay?
35733 y) Yes
35734 n) No
35735 y/n> y
35736 --------------------
35737 [remote]
35738 type = onedrive
35739 token = {"access_token":"youraccesstoken","token_type":"Bearer","refresh_token":"yourrefreshtoken","expiry":"2018-08-26T22:39:52.486512262+08:00"}
35740 drive_id = b!Eqwertyuiopasdfghjklzxcvbnm-7mnbvcxzlkjhgfdsapoiuytrewqk
35741 drive_type = business
35742 --------------------
35743 y) Yes this is OK
35744 e) Edit this remote
35745 d) Delete this remote
35746 y/e/d> y
35747
35748 See the remote setup docs (https://rclone.org/remote_setup/) for how to
35749 set it up on a machine with no Internet browser available.
35750
35751 Note that rclone runs a webserver on your local machine to collect the
35752 token as returned from Microsoft. This only runs from the moment it
35753 opens your browser to the moment you get back the verification code.
35754 This is on http://127.0.0.1:53682/ and this it may require you to un‐
35755 block it temporarily if you are running a host firewall.
35756
35757 Once configured you can then use rclone like this,
35758
35759 List directories in top level of your OneDrive
35760
35761 rclone lsd remote:
35762
35763 List all the files in your OneDrive
35764
35765 rclone ls remote:
35766
35767 To copy a local directory to an OneDrive directory called backup
35768
35769 rclone copy /home/source remote:backup
35770
35771 Getting your own Client ID and Key
35772 rclone uses a default Client ID when talking to OneDrive, unless a cus‐
35773 tom client_id is specified in the config. The default Client ID and
35774 Key are shared by all rclone users when performing requests.
35775
35776 You may choose to create and use your own Client ID, in case the de‐
35777 fault one does not work well for you. For example, you might see
35778 throttling.
35779
35780 Creating Client ID for OneDrive Personal
35781 To create your own Client ID, please follow these steps:
35782
35783 1. Open https://portal.azure.com/#blade/Microsoft_AAD_Regis‐
35784 teredApps/ApplicationsListBlade and then click New registration.
35785
35786 2. Enter a name for your app, choose account type Accounts in any orga‐
35787 nizational directory (Any Azure AD directory - Multitenant) and per‐
35788 sonal Microsoft accounts (e.g. Skype, Xbox), select Web in Redirect
35789 URI, then type (do not copy and paste) http://localhost:53682/ and
35790 click Register. Copy and keep the Application (client) ID under the
35791 app name for later use.
35792
35793 3. Under manage select Certificates & secrets, click New client secret.
35794 Enter a description (can be anything) and set Expires to 24 months.
35795 Copy and keep that secret Value for later use (you won't be able to
35796 see this value afterwards).
35797
35798 4. Under manage select API permissions, click Add a permission and se‐
35799 lect Microsoft Graph then select delegated permissions.
35800
35801 5. Search and select the following permissions: Files.Read, Files.Read‐
35802 Write, Files.Read.All, Files.ReadWrite.All, offline_access, Us‐
35803 er.Read and Sites.Read.All (if custom access scopes are configured,
35804 select the permissions accordingly). Once selected click Add per‐
35805 missions at the bottom.
35806
35807 Now the application is complete. Run rclone config to create or edit a
35808 OneDrive remote. Supply the app ID and password as Client ID and Se‐
35809 cret, respectively. rclone will walk you through the remaining steps.
35810
35811 The access_scopes option allows you to configure the permissions re‐
35812 quested by rclone. See Microsoft Docs (https://docs.microsoft.com/en-
35813 us/graph/permissions-reference#files-permissions) for more information
35814 about the different scopes.
35815
35816 The Sites.Read.All permission is required if you need to search Share‐
35817 Point sites when configuring the remote
35818 (https://github.com/rclone/rclone/pull/5883). However, if that permis‐
35819 sion is not assigned, you need to exclude Sites.Read.All from your ac‐
35820 cess scopes or set disable_site_permission option to true in the ad‐
35821 vanced options.
35822
35823 Creating Client ID for OneDrive Business
35824 The steps for OneDrive Personal may or may not work for OneDrive Busi‐
35825 ness, depending on the security settings of the organization. A common
35826 error is that the publisher of the App is not verified.
35827
35828 You may try to verify you account (https://docs.microsoft.com/en-
35829 us/azure/active-directory/develop/publisher-verification-overview), or
35830 try to limit the App to your organization only, as shown below.
35831
35832 1. Make sure to create the App with your business account.
35833
35834 2. Follow the steps above to create an App. However, we need a differ‐
35835 ent account type here: Accounts in this organizational directory on‐
35836 ly (*** - Single tenant). Note that you can also change the account
35837 type after creating the App.
35838
35839 3. Find the tenant ID (https://docs.microsoft.com/en-us/azure/active-
35840 directory/fundamentals/active-directory-how-to-find-tenant) of your
35841 organization.
35842
35843 4. In the rclone config, set auth_url to https://login.microsofton‐
35844 line.com/YOUR_TENANT_ID/oauth2/v2.0/authorize.
35845
35846 5. In the rclone config, set token_url to https://login.microsofton‐
35847 line.com/YOUR_TENANT_ID/oauth2/v2.0/token.
35848
35849 Note: If you have a special region, you may need a different host in
35850 step 4 and 5. Here are some hints
35851 (https://github.com/rclone/rclone/blob/bc23bf11db1c78c6ebbf8ea538fbebf7058b4176/back‐
35852 end/onedrive/onedrive.go#L86).
35853
35854 Modification time and hashes
35855 OneDrive allows modification times to be set on objects accurate to 1
35856 second. These will be used to detect whether objects need syncing or
35857 not.
35858
35859 OneDrive personal supports SHA1 type hashes. OneDrive for business and
35860 Sharepoint Server support QuickXorHash (https://docs.microsoft.com/en-
35861 us/onedrive/developer/code-snippets/quickxorhash).
35862
35863 For all types of OneDrive you can use the --checksum flag.
35864
35865 Restricted filename characters
35866 In addition to the default restricted characters set
35867 (https://rclone.org/overview/#restricted-characters) the following
35868 characters are also replaced:
35869
35870 Character Value Replacement
35871 ────────────────────────────────
35872 " 0x22 "
35873 * 0x2A *
35874 : 0x3A :
35875
35876 < 0x3C <
35877 > 0x3E >
35878 ? 0x3F ?
35879 \ 0x5C \
35880 | 0x7C |
35881
35882 File names can also not end with the following characters. These only
35883 get replaced if they are the last character in the name:
35884
35885 Character Value Replacement
35886 ────────────────────────────────
35887 SP 0x20 ␠
35888 . 0x2E .
35889
35890 File names can also not begin with the following characters. These on‐
35891 ly get replaced if they are the first character in the name:
35892
35893 Character Value Replacement
35894 ────────────────────────────────
35895 SP 0x20 ␠
35896 ~ 0x7E ~
35897
35898 Invalid UTF-8 bytes will also be replaced (https://rclone.org/over‐
35899 view/#invalid-utf8), as they can't be used in JSON strings.
35900
35901 Deleting files
35902 Any files you delete with rclone will end up in the trash. Microsoft
35903 doesn't provide an API to permanently delete files, nor to empty the
35904 trash, so you will have to do that with one of Microsoft's apps or via
35905 the OneDrive website.
35906
35907 Standard options
35908 Here are the Standard options specific to onedrive (Microsoft
35909 OneDrive).
35910
35911 --onedrive-client-id
35912 OAuth Client Id.
35913
35914 Leave blank normally.
35915
35916 Properties:
35917
35918 • Config: client_id
35919
35920 • Env Var: RCLONE_ONEDRIVE_CLIENT_ID
35921
35922 • Type: string
35923
35924 • Required: false
35925
35926 --onedrive-client-secret
35927 OAuth Client Secret.
35928
35929 Leave blank normally.
35930
35931 Properties:
35932
35933 • Config: client_secret
35934
35935 • Env Var: RCLONE_ONEDRIVE_CLIENT_SECRET
35936
35937 • Type: string
35938
35939 • Required: false
35940
35941 --onedrive-region
35942 Choose national cloud region for OneDrive.
35943
35944 Properties:
35945
35946 • Config: region
35947
35948 • Env Var: RCLONE_ONEDRIVE_REGION
35949
35950 • Type: string
35951
35952 • Default: "global"
35953
35954 • Examples:
35955
35956 • "global"
35957
35958 • Microsoft Cloud Global
35959
35960 • "us"
35961
35962 • Microsoft Cloud for US Government
35963
35964 • "de"
35965
35966 • Microsoft Cloud Germany
35967
35968 • "cn"
35969
35970 • Azure and Office 365 operated by Vnet Group in China
35971
35972 Advanced options
35973 Here are the Advanced options specific to onedrive (Microsoft
35974 OneDrive).
35975
35976 --onedrive-token
35977 OAuth Access Token as a JSON blob.
35978
35979 Properties:
35980
35981 • Config: token
35982
35983 • Env Var: RCLONE_ONEDRIVE_TOKEN
35984
35985 • Type: string
35986
35987 • Required: false
35988
35989 --onedrive-auth-url
35990 Auth server URL.
35991
35992 Leave blank to use the provider defaults.
35993
35994 Properties:
35995
35996 • Config: auth_url
35997
35998 • Env Var: RCLONE_ONEDRIVE_AUTH_URL
35999
36000 • Type: string
36001
36002 • Required: false
36003
36004 --onedrive-token-url
36005 Token server url.
36006
36007 Leave blank to use the provider defaults.
36008
36009 Properties:
36010
36011 • Config: token_url
36012
36013 • Env Var: RCLONE_ONEDRIVE_TOKEN_URL
36014
36015 • Type: string
36016
36017 • Required: false
36018
36019 --onedrive-chunk-size
36020 Chunk size to upload files with - must be multiple of 320k (327,680
36021 bytes).
36022
36023 Above this size files will be chunked - must be multiple of 320k
36024 (327,680 bytes) and should not exceed 250M (262,144,000 bytes) else you
36025 may encounter "Microsoft.SharePoint.Client.InvalidClientQueryException:
36026 The request message is too big." Note that the chunks will be buffered
36027 into memory.
36028
36029 Properties:
36030
36031 • Config: chunk_size
36032
36033 • Env Var: RCLONE_ONEDRIVE_CHUNK_SIZE
36034
36035 • Type: SizeSuffix
36036
36037 • Default: 10Mi
36038
36039 --onedrive-drive-id
36040 The ID of the drive to use.
36041
36042 Properties:
36043
36044 • Config: drive_id
36045
36046 • Env Var: RCLONE_ONEDRIVE_DRIVE_ID
36047
36048 • Type: string
36049
36050 • Required: false
36051
36052 --onedrive-drive-type
36053 The type of the drive (personal | business | documentLibrary).
36054
36055 Properties:
36056
36057 • Config: drive_type
36058
36059 • Env Var: RCLONE_ONEDRIVE_DRIVE_TYPE
36060
36061 • Type: string
36062
36063 • Required: false
36064
36065 --onedrive-root-folder-id
36066 ID of the root folder.
36067
36068 This isn't normally needed, but in special circumstances you might know
36069 the folder ID that you wish to access but not be able to get there
36070 through a path traversal.
36071
36072 Properties:
36073
36074 • Config: root_folder_id
36075
36076 • Env Var: RCLONE_ONEDRIVE_ROOT_FOLDER_ID
36077
36078 • Type: string
36079
36080 • Required: false
36081
36082 --onedrive-access-scopes
36083 Set scopes to be requested by rclone.
36084
36085 Choose or manually enter a custom space separated list with all scopes,
36086 that rclone should request.
36087
36088 Properties:
36089
36090 • Config: access_scopes
36091
36092 • Env Var: RCLONE_ONEDRIVE_ACCESS_SCOPES
36093
36094 • Type: SpaceSepList
36095
36096 • Default: Files.Read Files.ReadWrite Files.Read.All Files.Read‐
36097 Write.All Sites.Read.All offline_access
36098
36099 • Examples:
36100
36101 • "Files.Read Files.ReadWrite Files.Read.All Files.ReadWrite.All
36102 Sites.Read.All offline_access"
36103
36104 • Read and write access to all resources
36105
36106 • "Files.Read Files.Read.All Sites.Read.All offline_access"
36107
36108 • Read only access to all resources
36109
36110 • "Files.Read Files.ReadWrite Files.Read.All Files.ReadWrite.All off‐
36111 line_access"
36112
36113 • Read and write access to all resources, without the ability to
36114 browse SharePoint sites.
36115
36116 • Same as if disable_site_permission was set to true
36117
36118 --onedrive-disable-site-permission
36119 Disable the request for Sites.Read.All permission.
36120
36121 If set to true, you will no longer be able to search for a SharePoint
36122 site when configuring drive ID, because rclone will not request
36123 Sites.Read.All permission. Set it to true if your organization didn't
36124 assign Sites.Read.All permission to the application, and your organiza‐
36125 tion disallows users to consent app permission request on their own.
36126
36127 Properties:
36128
36129 • Config: disable_site_permission
36130
36131 • Env Var: RCLONE_ONEDRIVE_DISABLE_SITE_PERMISSION
36132
36133 • Type: bool
36134
36135 • Default: false
36136
36137 --onedrive-expose-onenote-files
36138 Set to make OneNote files show up in directory listings.
36139
36140 By default, rclone will hide OneNote files in directory listings be‐
36141 cause operations like "Open" and "Update" won't work on them. But this
36142 behaviour may also prevent you from deleting them. If you want to
36143 delete OneNote files or otherwise want them to show up in directory
36144 listing, set this option.
36145
36146 Properties:
36147
36148 • Config: expose_onenote_files
36149
36150 • Env Var: RCLONE_ONEDRIVE_EXPOSE_ONENOTE_FILES
36151
36152 • Type: bool
36153
36154 • Default: false
36155
36156 --onedrive-server-side-across-configs
36157 Allow server-side operations (e.g. copy) to work across different
36158 onedrive configs.
36159
36160 This will only work if you are copying between two OneDrive Personal
36161 drives AND the files to copy are already shared between them. In other
36162 cases, rclone will fall back to normal copy (which will be slightly
36163 slower).
36164
36165 Properties:
36166
36167 • Config: server_side_across_configs
36168
36169 • Env Var: RCLONE_ONEDRIVE_SERVER_SIDE_ACROSS_CONFIGS
36170
36171 • Type: bool
36172
36173 • Default: false
36174
36175 --onedrive-list-chunk
36176 Size of listing chunk.
36177
36178 Properties:
36179
36180 • Config: list_chunk
36181
36182 • Env Var: RCLONE_ONEDRIVE_LIST_CHUNK
36183
36184 • Type: int
36185
36186 • Default: 1000
36187
36188 --onedrive-no-versions
36189 Remove all versions on modifying operations.
36190
36191 Onedrive for business creates versions when rclone uploads new files
36192 overwriting an existing one and when it sets the modification time.
36193
36194 These versions take up space out of the quota.
36195
36196 This flag checks for versions after file upload and setting modifica‐
36197 tion time and removes all but the last version.
36198
36199 NB Onedrive personal can't currently delete versions so don't use this
36200 flag there.
36201
36202 Properties:
36203
36204 • Config: no_versions
36205
36206 • Env Var: RCLONE_ONEDRIVE_NO_VERSIONS
36207
36208 • Type: bool
36209
36210 • Default: false
36211
36212 --onedrive-link-scope
36213 Set the scope of the links created by the link command.
36214
36215 Properties:
36216
36217 • Config: link_scope
36218
36219 • Env Var: RCLONE_ONEDRIVE_LINK_SCOPE
36220
36221 • Type: string
36222
36223 • Default: "anonymous"
36224
36225 • Examples:
36226
36227 • "anonymous"
36228
36229 • Anyone with the link has access, without needing to sign in.
36230
36231 • This may include people outside of your organization.
36232
36233 • Anonymous link support may be disabled by an administrator.
36234
36235 • "organization"
36236
36237 • Anyone signed into your organization (tenant) can use the link to
36238 get access.
36239
36240 • Only available in OneDrive for Business and SharePoint.
36241
36242 --onedrive-link-type
36243 Set the type of the links created by the link command.
36244
36245 Properties:
36246
36247 • Config: link_type
36248
36249 • Env Var: RCLONE_ONEDRIVE_LINK_TYPE
36250
36251 • Type: string
36252
36253 • Default: "view"
36254
36255 • Examples:
36256
36257 • "view"
36258
36259 • Creates a read-only link to the item.
36260
36261 • "edit"
36262
36263 • Creates a read-write link to the item.
36264
36265 • "embed"
36266
36267 • Creates an embeddable link to the item.
36268
36269 --onedrive-link-password
36270 Set the password for links created by the link command.
36271
36272 At the time of writing this only works with OneDrive personal paid ac‐
36273 counts.
36274
36275 Properties:
36276
36277 • Config: link_password
36278
36279 • Env Var: RCLONE_ONEDRIVE_LINK_PASSWORD
36280
36281 • Type: string
36282
36283 • Required: false
36284
36285 --onedrive-encoding
36286 The encoding for the backend.
36287
36288 See the encoding section in the overview (https://rclone.org/over‐
36289 view/#encoding) for more info.
36290
36291 Properties:
36292
36293 • Config: encoding
36294
36295 • Env Var: RCLONE_ONEDRIVE_ENCODING
36296
36297 • Type: MultiEncoder
36298
36299 • Default: Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,Back‐
36300 Slash,Del,Ctl,LeftSpace,LeftTilde,RightSpace,RightPeriod,In‐
36301 validUtf8,Dot
36302
36303 Limitations
36304 If you don't use rclone for 90 days the refresh token will expire.
36305 This will result in authorization problems. This is easy to fix by
36306 running the rclone config reconnect remote: command to get a new token
36307 and refresh token.
36308
36309 Naming
36310 Note that OneDrive is case insensitive so you can't have a file called
36311 "Hello.doc" and one called "hello.doc".
36312
36313 There are quite a few characters that can't be in OneDrive file names.
36314 These can't occur on Windows platforms, but on non-Windows platforms
36315 they are common. Rclone will map these names to and from an identical
36316 looking unicode equivalent. For example if a file has a ? in it will
36317 be mapped to ? instead.
36318
36319 File sizes
36320 The largest allowed file size is 250 GiB for both OneDrive Personal and
36321 OneDrive for Business (Updated 13 Jan 2021) (https://support.micro‐
36322 soft.com/en-us/office/invalid-file-names-and-file-types-in-onedrive-
36323 and-sharepoint-64883a5d-228e-48f5-b3d2-eb39e07630fa?ui=en-us&rs=en-
36324 us&ad=us#individualfilesize).
36325
36326 Path length
36327 The entire path, including the file name, must contain fewer than 400
36328 characters for OneDrive, OneDrive for Business and SharePoint Online.
36329 If you are encrypting file and folder names with rclone, you may want
36330 to pay attention to this limitation because the encrypted names are
36331 typically longer than the original ones.
36332
36333 Number of files
36334 OneDrive seems to be OK with at least 50,000 files in a folder, but at
36335 100,000 rclone will get errors listing the directory like couldn’t list
36336 files: UnknownError:. See #2707 (https://github.com/rclone/rclone/is‐
36337 sues/2707) for more info.
36338
36339 An official document about the limitations for different types of
36340 OneDrive can be found here (https://support.office.com/en-us/arti‐
36341 cle/invalid-file-names-and-file-types-in-onedrive-onedrive-for-busi‐
36342 ness-and-sharepoint-64883a5d-228e-48f5-b3d2-eb39e07630fa).
36343
36344 Versions
36345 Every change in a file OneDrive causes the service to create a new ver‐
36346 sion of the file. This counts against a users quota. For example
36347 changing the modification time of a file creates a second version, so
36348 the file apparently uses twice the space.
36349
36350 For example the copy command is affected by this as rclone copies the
36351 file and then afterwards sets the modification time to match the source
36352 file which uses another version.
36353
36354 You can use the rclone cleanup command (see below) to remove all old
36355 versions.
36356
36357 Or you can set the no_versions parameter to true and rclone will remove
36358 versions after operations which create new versions. This takes extra
36359 transactions so only enable it if you need it.
36360
36361 Note At the time of writing Onedrive Personal creates versions (but not
36362 for setting the modification time) but the API for removing them re‐
36363 turns "API not found" so cleanup and no_versions should not be used on
36364 Onedrive Personal.
36365
36366 Disabling versioning
36367 Starting October 2018, users will no longer be able to disable version‐
36368 ing by default. This is because Microsoft has brought an update
36369 (https://techcommunity.microsoft.com/t5/Microsoft-OneDrive-Blog/New-Up‐
36370 dates-to-OneDrive-and-SharePoint-Team-Site-Versioning/ba-p/204390) to
36371 the mechanism. To change this new default setting, a PowerShell com‐
36372 mand is required to be run by a SharePoint admin. If you are an admin,
36373 you can run these commands in PowerShell to change that setting:
36374
36375 1. Install-Module -Name Microsoft.Online.SharePoint.PowerShell (in case
36376 you haven't installed this already)
36377
36378 2. Import-Module Microsoft.Online.SharePoint.PowerShell -Disable‐
36379 NameChecking
36380
36381 3. Connect-SPOService -Url https://YOURSITE-admin.sharepoint.com -Cre‐
36382 dential YOU@YOURSITE.COM (replacing YOURSITE, YOU, YOURSITE.COM with
36383 the actual values; this will prompt for your credentials)
36384
36385 4. Set-SPOTenant -EnableMinimumVersionRequirement $False
36386
36387 5. Disconnect-SPOService (to disconnect from the server)
36388
36389 Below are the steps for normal users to disable versioning. If you
36390 don't see the "No Versioning" option, make sure the above requirements
36391 are met.
36392
36393 User Weropol (https://github.com/Weropol) has found a method to disable
36394 versioning on OneDrive
36395
36396 1. Open the settings menu by clicking on the gear symbol at the top of
36397 the OneDrive Business page.
36398
36399 2. Click Site settings.
36400
36401 3. Once on the Site settings page, navigate to Site Administration >
36402 Site libraries and lists.
36403
36404 4. Click Customize "Documents".
36405
36406 5. Click General Settings > Versioning Settings.
36407
36408 6. Under Document Version History select the option No versioning.
36409 Note: This will disable the creation of new file versions, but will
36410 not remove any previous versions. Your documents are safe.
36411
36412 7. Apply the changes by clicking OK.
36413
36414 8. Use rclone to upload or modify files. (I also use the --no-update-
36415 modtime flag)
36416
36417 9. Restore the versioning settings after using rclone. (Optional)
36418
36419 Cleanup
36420 OneDrive supports rclone cleanup which causes rclone to look through
36421 every file under the path supplied and delete all version but the cur‐
36422 rent version. Because this involves traversing all the files, then
36423 querying each file for versions it can be quite slow. Rclone does
36424 --checkers tests in parallel. The command also supports -i which is a
36425 great way to see what it would do.
36426
36427 rclone cleanup -i remote:path/subdir # interactively remove all old version for path/subdir
36428 rclone cleanup remote:path/subdir # unconditionally remove all old version for path/subdir
36429
36430 NB Onedrive personal can't currently delete versions
36431
36432 Troubleshooting
36433 Excessive throttling or blocked on SharePoint
36434 If you experience excessive throttling or is being blocked on Share‐
36435 Point then it may help to set the user agent explicitly with a flag
36436 like this: --user-agent "ISV|rclone.org|rclone/v1.55.1"
36437
36438 The specific details can be found in the Microsoft document: Avoid get‐
36439 ting throttled or blocked in SharePoint Online (https://docs.micro‐
36440 soft.com/en-us/sharepoint/dev/general-development/how-to-avoid-getting-
36441 throttled-or-blocked-in-sharepoint-online#how-to-decorate-your-http-
36442 traffic-to-avoid-throttling)
36443
36444 Unexpected file size/hash differences on Sharepoint
36445 It is a known (https://github.com/OneDrive/onedrive-api-docs/is‐
36446 sues/935#issuecomment-441741631) issue that Sharepoint (not OneDrive or
36447 OneDrive for Business) silently modifies uploaded files, mainly Office
36448 files (.docx, .xlsx, etc.), causing file size and hash checks to fail.
36449 There are also other situations that will cause OneDrive to report in‐
36450 consistent file sizes. To use rclone with such affected files on
36451 Sharepoint, you may disable these checks with the following command
36452 line arguments:
36453
36454 --ignore-checksum --ignore-size
36455
36456 Alternatively, if you have write access to the OneDrive files, it may
36457 be possible to fix this problem for certain files, by attempting the
36458 steps below. Open the web interface for OneDrive
36459 (https://onedrive.live.com) and find the affected files (which will be
36460 in the error messages/log for rclone). Simply click on each of these
36461 files, causing OneDrive to open them on the web. This will cause each
36462 file to be converted in place to a format that is functionally equiva‐
36463 lent but which will no longer trigger the size discrepancy. Once all
36464 problematic files are converted you will no longer need the ignore op‐
36465 tions above.
36466
36467 Replacing/deleting existing files on Sharepoint gets "item not found"
36468 It is a known (https://github.com/OneDrive/onedrive-api-docs/is‐
36469 sues/1068) issue that Sharepoint (not OneDrive or OneDrive for Busi‐
36470 ness) may return "item not found" errors when users try to replace or
36471 delete uploaded files; this seems to mainly affect Office files (.docx,
36472 .xlsx, etc.) and web files (.html, .aspx, etc.). As a workaround, you
36473 may use the --backup-dir <BACKUP_DIR> command line argument so rclone
36474 moves the files to be replaced/deleted into a given backup directory
36475 (instead of directly replacing/deleting them). For example, to in‐
36476 struct rclone to move the files into the directory rclone-backup-dir on
36477 backend mysharepoint, you may use:
36478
36479 --backup-dir mysharepoint:rclone-backup-dir
36480
36481 access_denied (AADSTS65005)
36482 Error: access_denied
36483 Code: AADSTS65005
36484 Description: Using application 'rclone' is currently not supported for your organization [YOUR_ORGANIZATION] because it is in an unmanaged state. An administrator needs to claim ownership of the company by DNS validation of [YOUR_ORGANIZATION] before the application rclone can be provisioned.
36485
36486 This means that rclone can't use the OneDrive for Business API with
36487 your account. You can't do much about it, maybe write an email to your
36488 admins.
36489
36490 However, there are other ways to interact with your OneDrive account.
36491 Have a look at the WebDAV backend: https://rclone.org/webdav/#share‐
36492 point
36493
36494 invalid_grant (AADSTS50076)
36495 Error: invalid_grant
36496 Code: AADSTS50076
36497 Description: Due to a configuration change made by your administrator, or because you moved to a new location, you must use multi-factor authentication to access '...'.
36498
36499 If you see the error above after enabling multi-factor authentication
36500 for your account, you can fix it by refreshing your OAuth refresh to‐
36501 ken. To do that, run rclone config, and choose to edit your OneDrive
36502 backend. Then, you don't need to actually make any changes until you
36503 reach this question: Already have a token - refresh?. For this ques‐
36504 tion, answer y and go through the process to refresh your token, just
36505 like the first time the backend is configured. After this, rclone
36506 should work again for this backend.
36507
36508 Invalid request when making public links
36509 On Sharepoint and OneDrive for Business, rclone link may return an "In‐
36510 valid request" error. A possible cause is that the organisation admin
36511 didn't allow public links to be made for the organisation/sharepoint
36512 library. To fix the permissions as an admin, take a look at the docs:
36513 1 (https://docs.microsoft.com/en-us/sharepoint/turn-external-sharing-
36514 on-or-off), 2 (https://support.microsoft.com/en-us/office/set-up-and-
36515 manage-access-requests-94b26e0b-2822-49d4-929a-8455698654b3).
36516
36518 Paths are specified as remote:path
36519
36520 Paths may be as deep as required, e.g. remote:directory/subdirectory.
36521
36522 Configuration
36523 Here is an example of how to make a remote called remote. First run:
36524
36525 rclone config
36526
36527 This will guide you through an interactive setup process:
36528
36529 n) New remote
36530 d) Delete remote
36531 q) Quit config
36532 e/n/d/q> n
36533 name> remote
36534 Type of storage to configure.
36535 Choose a number from below, or type in your own value
36536 [snip]
36537 XX / OpenDrive
36538 \ "opendrive"
36539 [snip]
36540 Storage> opendrive
36541 Username
36542 username>
36543 Password
36544 y) Yes type in my own password
36545 g) Generate random password
36546 y/g> y
36547 Enter the password:
36548 password:
36549 Confirm the password:
36550 password:
36551 --------------------
36552 [remote]
36553 username =
36554 password = *** ENCRYPTED ***
36555 --------------------
36556 y) Yes this is OK
36557 e) Edit this remote
36558 d) Delete this remote
36559 y/e/d> y
36560
36561 List directories in top level of your OpenDrive
36562
36563 rclone lsd remote:
36564
36565 List all the files in your OpenDrive
36566
36567 rclone ls remote:
36568
36569 To copy a local directory to an OpenDrive directory called backup
36570
36571 rclone copy /home/source remote:backup
36572
36573 Modified time and MD5SUMs
36574 OpenDrive allows modification times to be set on objects accurate to 1
36575 second. These will be used to detect whether objects need syncing or
36576 not.
36577
36578 Restricted filename characters
36579 Character Value Replacement
36580 ────────────────────────────────
36581 NUL 0x00 ␀
36582 / 0x2F /
36583 " 0x22 "
36584 * 0x2A *
36585 : 0x3A :
36586 < 0x3C <
36587 > 0x3E >
36588 ? 0x3F ?
36589 \ 0x5C \
36590 | 0x7C |
36591
36592 File names can also not begin or end with the following characters.
36593 These only get replaced if they are the first or last character in the
36594 name:
36595
36596 Character Value Replacement
36597 ────────────────────────────────
36598 SP 0x20 ␠
36599 HT 0x09 ␉
36600 LF 0x0A ␊
36601 VT 0x0B ␋
36602 CR 0x0D ␍
36603
36604 Invalid UTF-8 bytes will also be replaced (https://rclone.org/over‐
36605 view/#invalid-utf8), as they can't be used in JSON strings.
36606
36607 Standard options
36608 Here are the Standard options specific to opendrive (OpenDrive).
36609
36610 --opendrive-username
36611 Username.
36612
36613 Properties:
36614
36615 • Config: username
36616
36617 • Env Var: RCLONE_OPENDRIVE_USERNAME
36618
36619 • Type: string
36620
36621 • Required: true
36622
36623 --opendrive-password
36624 Password.
36625
36626 NB Input to this must be obscured - see rclone obscure
36627 (https://rclone.org/commands/rclone_obscure/).
36628
36629 Properties:
36630
36631 • Config: password
36632
36633 • Env Var: RCLONE_OPENDRIVE_PASSWORD
36634
36635 • Type: string
36636
36637 • Required: true
36638
36639 Advanced options
36640 Here are the Advanced options specific to opendrive (OpenDrive).
36641
36642 --opendrive-encoding
36643 The encoding for the backend.
36644
36645 See the encoding section in the overview (https://rclone.org/over‐
36646 view/#encoding) for more info.
36647
36648 Properties:
36649
36650 • Config: encoding
36651
36652 • Env Var: RCLONE_OPENDRIVE_ENCODING
36653
36654 • Type: MultiEncoder
36655
36656 • Default: Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,Back‐
36657 Slash,LeftSpace,LeftCrLfHtVt,RightSpace,RightCrLfHtVt,InvalidUtf8,Dot
36658
36659 --opendrive-chunk-size
36660 Files will be uploaded in chunks this size.
36661
36662 Note that these chunks are buffered in memory so increasing them will
36663 increase memory use.
36664
36665 Properties:
36666
36667 • Config: chunk_size
36668
36669 • Env Var: RCLONE_OPENDRIVE_CHUNK_SIZE
36670
36671 • Type: SizeSuffix
36672
36673 • Default: 10Mi
36674
36675 Limitations
36676 Note that OpenDrive is case insensitive so you can't have a file called
36677 "Hello.doc" and one called "hello.doc".
36678
36679 There are quite a few characters that can't be in OpenDrive file names.
36680 These can't occur on Windows platforms, but on non-Windows platforms
36681 they are common. Rclone will map these names to and from an identical
36682 looking unicode equivalent. For example if a file has a ? in it will
36683 be mapped to ? instead.
36684
36685 rclone about is not supported by the OpenDrive backend. Backends with‐
36686 out this capability cannot determine free space for an rclone mount or
36687 use policy mfs (most free space) as a member of an rclone union remote.
36688
36689 See List of backends that do not support rclone about
36690 (https://rclone.org/overview/#optional-features) and rclone about
36691 (https://rclone.org/commands/rclone_about/)
36692
36694 Oracle Object Storage Overview (https://docs.oracle.com/en-us/iaas/Con‐
36695 tent/Object/Concepts/objectstorageoverview.htm)
36696
36697 Oracle Object Storage FAQ (https://www.oracle.com/cloud/storage/object-
36698 storage/faq/)
36699
36700 Paths are specified as remote:bucket (or remote: for the lsd command.)
36701 You may put subdirectories in too, e.g. remote:bucket/path/to/dir.
36702
36703 Configuration
36704 Here is an example of making an oracle object storage configuration.
36705 rclone config walks you through it.
36706
36707 Here is an example of how to make a remote called remote. First run:
36708
36709 rclone config
36710
36711 This will guide you through an interactive setup process:
36712
36713 n) New remote
36714 d) Delete remote
36715 r) Rename remote
36716 c) Copy remote
36717 s) Set configuration password
36718 q) Quit config
36719 e/n/d/r/c/s/q> n
36720
36721 Enter name for new remote.
36722 name> remote
36723
36724 Option Storage.
36725 Type of storage to configure.
36726 Choose a number from below, or type in your own value.
36727 [snip]
36728 XX / Oracle Cloud Infrastructure Object Storage
36729 \ (oracleobjectstorage)
36730 Storage> oracleobjectstorage
36731
36732 Option provider.
36733 Choose your Auth Provider
36734 Choose a number from below, or type in your own string value.
36735 Press Enter for the default (env_auth).
36736 1 / automatically pickup the credentials from runtime(env), first one to provide auth wins
36737 \ (env_auth)
36738 / use an OCI user and an API key for authentication.
36739 2 | you’ll need to put in a config file your tenancy OCID, user OCID, region, the path, fingerprint to an API key.
36740 | https://docs.oracle.com/en-us/iaas/Content/API/Concepts/sdkconfig.htm
36741 \ (user_principal_auth)
36742 / use instance principals to authorize an instance to make API calls.
36743 3 | each instance has its own identity, and authenticates using the certificates that are read from instance metadata.
36744 | https://docs.oracle.com/en-us/iaas/Content/Identity/Tasks/callingservicesfrominstances.htm
36745 \ (instance_principal_auth)
36746 4 / use resource principals to make API calls
36747 \ (resource_principal_auth)
36748 5 / no credentials needed, this is typically for reading public buckets
36749 \ (no_auth)
36750 provider> 2
36751
36752 Option namespace.
36753 Object storage namespace
36754 Enter a value.
36755 namespace> idbamagbg734
36756
36757 Option compartment.
36758 Object storage compartment OCID
36759 Enter a value.
36760 compartment> ocid1.compartment.oc1..aaaaaaaapufkxc7ame3sthry5i7ujrwfc7ejnthhu6bhanm5oqfjpyasjkba
36761
36762 Option region.
36763 Object storage Region
36764 Enter a value.
36765 region> us-ashburn-1
36766
36767 Option endpoint.
36768 Endpoint for Object storage API.
36769 Leave blank to use the default endpoint for the region.
36770 Enter a value. Press Enter to leave empty.
36771 endpoint>
36772
36773 Option config_file.
36774 Path to OCI config file
36775 Choose a number from below, or type in your own string value.
36776 Press Enter for the default (~/.oci/config).
36777 1 / oci configuration file location
36778 \ (~/.oci/config)
36779 config_file> /etc/oci/dev.conf
36780
36781 Option config_profile.
36782 Profile name inside OCI config file
36783 Choose a number from below, or type in your own string value.
36784 Press Enter for the default (Default).
36785 1 / Use the default profile
36786 \ (Default)
36787 config_profile> Test
36788
36789 Edit advanced config?
36790 y) Yes
36791 n) No (default)
36792 y/n> n
36793
36794 Configuration complete.
36795 Options:
36796 - type: oracleobjectstorage
36797 - namespace: idbamagbg734
36798 - compartment: ocid1.compartment.oc1..aaaaaaaapufkxc7ame3sthry5i7ujrwfc7ejnthhu6bhanm5oqfjpyasjkba
36799 - region: us-ashburn-1
36800 - provider: user_principal_auth
36801 - config_file: /etc/oci/dev.conf
36802 - config_profile: Test
36803 Keep this "remote" remote?
36804 y) Yes this is OK (default)
36805 e) Edit this remote
36806 d) Delete this remote
36807 y/e/d> y
36808
36809 See all buckets
36810
36811 rclone lsd remote:
36812
36813 Create a new bucket
36814
36815 rclone mkdir remote:bucket
36816
36817 List the contents of a bucket
36818
36819 rclone ls remote:bucket
36820 rclone ls remote:bucket --max-depth 1
36821
36822 Modified time
36823 The modified time is stored as metadata on the object as opc-meta-mtime
36824 as floating point since the epoch, accurate to 1 ns.
36825
36826 If the modification time needs to be updated rclone will attempt to
36827 perform a server side copy to update the modification if the object can
36828 be copied in a single part. In the case the object is larger than 5Gb,
36829 the object will be uploaded rather than copied.
36830
36831 Note that reading this from the object takes an additional HEAD request
36832 as the metadata isn't returned in object listings.
36833
36834 Multipart uploads
36835 rclone supports multipart uploads with OOS which means that it can up‐
36836 load files bigger than 5 GiB.
36837
36838 Note that files uploaded both with multipart upload and through crypt
36839 remotes do not have MD5 sums.
36840
36841 rclone switches from single part uploads to multipart uploads at the
36842 point specified by --oos-upload-cutoff. This can be a maximum of 5 GiB
36843 and a minimum of 0 (ie always upload multipart files).
36844
36845 The chunk sizes used in the multipart upload are specified by --oos-
36846 chunk-size and the number of chunks uploaded concurrently is specified
36847 by --oos-upload-concurrency.
36848
36849 Multipart uploads will use --transfers * --oos-upload-concurrency *
36850 --oos-chunk-size extra memory. Single part uploads to not use extra
36851 memory.
36852
36853 Single part transfers can be faster than multipart transfers or slower
36854 depending on your latency from oos - the more latency, the more likely
36855 single part transfers will be faster.
36856
36857 Increasing --oos-upload-concurrency will increase throughput (8 would
36858 be a sensible value) and increasing --oos-chunk-size also increases
36859 throughput (16M would be sensible). Increasing either of these will
36860 use more memory. The default values are high enough to gain most of
36861 the possible performance without using too much memory.
36862
36863 Standard options
36864 Here are the Standard options specific to oracleobjectstorage (Oracle
36865 Cloud Infrastructure Object Storage).
36866
36867 --oos-provider
36868 Choose your Auth Provider
36869
36870 Properties:
36871
36872 • Config: provider
36873
36874 • Env Var: RCLONE_OOS_PROVIDER
36875
36876 • Type: string
36877
36878 • Default: "env_auth"
36879
36880 • Examples:
36881
36882 • "env_auth"
36883
36884 • automatically pickup the credentials from runtime(env), first one
36885 to provide auth wins
36886
36887 • "user_principal_auth"
36888
36889 • use an OCI user and an API key for authentication.
36890
36891 • you’ll need to put in a config file your tenancy OCID, user OCID,
36892 region, the path, fingerprint to an API key.
36893
36894 • https://docs.oracle.com/en-us/iaas/Content/API/Concepts/sdkcon‐
36895 fig.htm
36896
36897 • "instance_principal_auth"
36898
36899 • use instance principals to authorize an instance to make API
36900 calls.
36901
36902 • each instance has its own identity, and authenticates using the
36903 certificates that are read from instance metadata.
36904
36905 • https://docs.oracle.com/en-us/iaas/Content/Identity/Tasks/call‐
36906 ingservicesfrominstances.htm
36907
36908 • "resource_principal_auth"
36909
36910 • use resource principals to make API calls
36911
36912 • "no_auth"
36913
36914 • no credentials needed, this is typically for reading public buck‐
36915 ets
36916
36917 --oos-namespace
36918 Object storage namespace
36919
36920 Properties:
36921
36922 • Config: namespace
36923
36924 • Env Var: RCLONE_OOS_NAMESPACE
36925
36926 • Type: string
36927
36928 • Required: true
36929
36930 --oos-compartment
36931 Object storage compartment OCID
36932
36933 Properties:
36934
36935 • Config: compartment
36936
36937 • Env Var: RCLONE_OOS_COMPARTMENT
36938
36939 • Provider: !no_auth
36940
36941 • Type: string
36942
36943 • Required: true
36944
36945 --oos-region
36946 Object storage Region
36947
36948 Properties:
36949
36950 • Config: region
36951
36952 • Env Var: RCLONE_OOS_REGION
36953
36954 • Type: string
36955
36956 • Required: true
36957
36958 --oos-endpoint
36959 Endpoint for Object storage API.
36960
36961 Leave blank to use the default endpoint for the region.
36962
36963 Properties:
36964
36965 • Config: endpoint
36966
36967 • Env Var: RCLONE_OOS_ENDPOINT
36968
36969 • Type: string
36970
36971 • Required: false
36972
36973 --oos-config-file
36974 Path to OCI config file
36975
36976 Properties:
36977
36978 • Config: config_file
36979
36980 • Env Var: RCLONE_OOS_CONFIG_FILE
36981
36982 • Provider: user_principal_auth
36983
36984 • Type: string
36985
36986 • Default: "~/.oci/config"
36987
36988 • Examples:
36989
36990 • "~/.oci/config"
36991
36992 • oci configuration file location
36993
36994 --oos-config-profile
36995 Profile name inside the oci config file
36996
36997 Properties:
36998
36999 • Config: config_profile
37000
37001 • Env Var: RCLONE_OOS_CONFIG_PROFILE
37002
37003 • Provider: user_principal_auth
37004
37005 • Type: string
37006
37007 • Default: "Default"
37008
37009 • Examples:
37010
37011 • "Default"
37012
37013 • Use the default profile
37014
37015 Advanced options
37016 Here are the Advanced options specific to oracleobjectstorage (Oracle
37017 Cloud Infrastructure Object Storage).
37018
37019 --oos-upload-cutoff
37020 Cutoff for switching to chunked upload.
37021
37022 Any files larger than this will be uploaded in chunks of chunk_size.
37023 The minimum is 0 and the maximum is 5 GiB.
37024
37025 Properties:
37026
37027 • Config: upload_cutoff
37028
37029 • Env Var: RCLONE_OOS_UPLOAD_CUTOFF
37030
37031 • Type: SizeSuffix
37032
37033 • Default: 200Mi
37034
37035 --oos-chunk-size
37036 Chunk size to use for uploading.
37037
37038 When uploading files larger than upload_cutoff or files with unknown
37039 size (e.g. from "rclone rcat" or uploaded with "rclone mount" or
37040 google photos or google docs) they will be uploaded as multipart up‐
37041 loads using this chunk size.
37042
37043 Note that "upload_concurrency" chunks of this size are buffered in mem‐
37044 ory per transfer.
37045
37046 If you are transferring large files over high-speed links and you have
37047 enough memory, then increasing this will speed up the transfers.
37048
37049 Rclone will automatically increase the chunk size when uploading a
37050 large file of known size to stay below the 10,000 chunks limit.
37051
37052 Files of unknown size are uploaded with the configured chunk_size.
37053 Since the default chunk size is 5 MiB and there can be at most 10,000
37054 chunks, this means that by default the maximum size of a file you can
37055 stream upload is 48 GiB. If you wish to stream upload larger files
37056 then you will need to increase chunk_size.
37057
37058 Increasing the chunk size decreases the accuracy of the progress sta‐
37059 tistics displayed with "-P" flag.
37060
37061 Properties:
37062
37063 • Config: chunk_size
37064
37065 • Env Var: RCLONE_OOS_CHUNK_SIZE
37066
37067 • Type: SizeSuffix
37068
37069 • Default: 5Mi
37070
37071 --oos-upload-concurrency
37072 Concurrency for multipart uploads.
37073
37074 This is the number of chunks of the same file that are uploaded concur‐
37075 rently.
37076
37077 If you are uploading small numbers of large files over high-speed links
37078 and these uploads do not fully utilize your bandwidth, then increasing
37079 this may help to speed up the transfers.
37080
37081 Properties:
37082
37083 • Config: upload_concurrency
37084
37085 • Env Var: RCLONE_OOS_UPLOAD_CONCURRENCY
37086
37087 • Type: int
37088
37089 • Default: 10
37090
37091 --oos-copy-cutoff
37092 Cutoff for switching to multipart copy.
37093
37094 Any files larger than this that need to be server-side copied will be
37095 copied in chunks of this size.
37096
37097 The minimum is 0 and the maximum is 5 GiB.
37098
37099 Properties:
37100
37101 • Config: copy_cutoff
37102
37103 • Env Var: RCLONE_OOS_COPY_CUTOFF
37104
37105 • Type: SizeSuffix
37106
37107 • Default: 4.656Gi
37108
37109 --oos-copy-timeout
37110 Timeout for copy.
37111
37112 Copy is an asynchronous operation, specify timeout to wait for copy to
37113 succeed
37114
37115 Properties:
37116
37117 • Config: copy_timeout
37118
37119 • Env Var: RCLONE_OOS_COPY_TIMEOUT
37120
37121 • Type: Duration
37122
37123 • Default: 1m0s
37124
37125 --oos-disable-checksum
37126 Don't store MD5 checksum with object metadata.
37127
37128 Normally rclone will calculate the MD5 checksum of the input before up‐
37129 loading it so it can add it to metadata on the object. This is great
37130 for data integrity checking but can cause long delays for large files
37131 to start uploading.
37132
37133 Properties:
37134
37135 • Config: disable_checksum
37136
37137 • Env Var: RCLONE_OOS_DISABLE_CHECKSUM
37138
37139 • Type: bool
37140
37141 • Default: false
37142
37143 --oos-encoding
37144 The encoding for the backend.
37145
37146 See the encoding section in the overview (https://rclone.org/over‐
37147 view/#encoding) for more info.
37148
37149 Properties:
37150
37151 • Config: encoding
37152
37153 • Env Var: RCLONE_OOS_ENCODING
37154
37155 • Type: MultiEncoder
37156
37157 • Default: Slash,InvalidUtf8,Dot
37158
37159 --oos-leave-parts-on-error
37160 If true avoid calling abort upload on a failure, leaving all success‐
37161 fully uploaded parts on S3 for manual recovery.
37162
37163 It should be set to true for resuming uploads across different ses‐
37164 sions.
37165
37166 WARNING: Storing parts of an incomplete multipart upload counts towards
37167 space usage on object storage and will add additional costs if not
37168 cleaned up.
37169
37170 Properties:
37171
37172 • Config: leave_parts_on_error
37173
37174 • Env Var: RCLONE_OOS_LEAVE_PARTS_ON_ERROR
37175
37176 • Type: bool
37177
37178 • Default: false
37179
37180 --oos-no-check-bucket
37181 If set, don't attempt to check the bucket exists or create it.
37182
37183 This can be useful when trying to minimise the number of transactions
37184 rclone does if you know the bucket exists already.
37185
37186 It can also be needed if the user you are using does not have bucket
37187 creation permissions.
37188
37189 Properties:
37190
37191 • Config: no_check_bucket
37192
37193 • Env Var: RCLONE_OOS_NO_CHECK_BUCKET
37194
37195 • Type: bool
37196
37197 • Default: false
37198
37199 Backend commands
37200 Here are the commands specific to the oracleobjectstorage backend.
37201
37202 Run them with
37203
37204 rclone backend COMMAND remote:
37205
37206 The help below will explain what arguments each command takes.
37207
37208 See the backend (https://rclone.org/commands/rclone_backend/) command
37209 for more info on how to pass options and arguments.
37210
37211 These can be run on a running backend using the rc command backend/com‐
37212 mand (https://rclone.org/rc/#backend-command).
37213
37214 rename
37215 change the name of an object
37216
37217 rclone backend rename remote: [options] [<arguments>+]
37218
37219 This command can be used to rename a object.
37220
37221 Usage Examples:
37222
37223 rclone backend rename oos:bucket relative-object-path-under-bucket object-new-name
37224
37225 list-multipart-uploads
37226 List the unfinished multipart uploads
37227
37228 rclone backend list-multipart-uploads remote: [options] [<arguments>+]
37229
37230 This command lists the unfinished multipart uploads in JSON format.
37231
37232 rclone backend list-multipart-uploads oos:bucket/path/to/object
37233
37234 It returns a dictionary of buckets with values as lists of unfinished
37235 multipart uploads.
37236
37237 You can call it with no bucket in which case it lists all bucket, with
37238 a bucket or with a bucket and path.
37239
37240 {
37241 "test-bucket": [
37242 {
37243 "namespace": "test-namespace",
37244 "bucket": "test-bucket",
37245 "object": "600m.bin",
37246 "uploadId": "51dd8114-52a4-b2f2-c42f-5291f05eb3c8",
37247 "timeCreated": "2022-07-29T06:21:16.595Z",
37248 "storageTier": "Standard"
37249 }
37250 ]
37251
37252 cleanup
37253 Remove unfinished multipart uploads.
37254
37255 rclone backend cleanup remote: [options] [<arguments>+]
37256
37257 This command removes unfinished multipart uploads of age greater than
37258 max-age which defaults to 24 hours.
37259
37260 Note that you can use -i/--dry-run with this command to see what it
37261 would do.
37262
37263 rclone backend cleanup oos:bucket/path/to/object
37264 rclone backend cleanup -o max-age=7w oos:bucket/path/to/object
37265
37266 Durations are parsed as per the rest of rclone, 2h, 7d, 7w etc.
37267
37268 Options:
37269
37270 • "max-age": Max age of upload to delete
37271
37273 Paths are specified as remote:bucket (or remote: for the lsd command.)
37274 You may put subdirectories in too, e.g. remote:bucket/path/to/dir.
37275
37276 Configuration
37277 Here is an example of making an QingStor configuration. First run
37278
37279 rclone config
37280
37281 This will guide you through an interactive setup process.
37282
37283 No remotes found, make a new one?
37284 n) New remote
37285 r) Rename remote
37286 c) Copy remote
37287 s) Set configuration password
37288 q) Quit config
37289 n/r/c/s/q> n
37290 name> remote
37291 Type of storage to configure.
37292 Choose a number from below, or type in your own value
37293 [snip]
37294 XX / QingStor Object Storage
37295 \ "qingstor"
37296 [snip]
37297 Storage> qingstor
37298 Get QingStor credentials from runtime. Only applies if access_key_id and secret_access_key is blank.
37299 Choose a number from below, or type in your own value
37300 1 / Enter QingStor credentials in the next step
37301 \ "false"
37302 2 / Get QingStor credentials from the environment (env vars or IAM)
37303 \ "true"
37304 env_auth> 1
37305 QingStor Access Key ID - leave blank for anonymous access or runtime credentials.
37306 access_key_id> access_key
37307 QingStor Secret Access Key (password) - leave blank for anonymous access or runtime credentials.
37308 secret_access_key> secret_key
37309 Enter an endpoint URL to connection QingStor API.
37310 Leave blank will use the default value "https://qingstor.com:443"
37311 endpoint>
37312 Zone connect to. Default is "pek3a".
37313 Choose a number from below, or type in your own value
37314 / The Beijing (China) Three Zone
37315 1 | Needs location constraint pek3a.
37316 \ "pek3a"
37317 / The Shanghai (China) First Zone
37318 2 | Needs location constraint sh1a.
37319 \ "sh1a"
37320 zone> 1
37321 Number of connection retry.
37322 Leave blank will use the default value "3".
37323 connection_retries>
37324 Remote config
37325 --------------------
37326 [remote]
37327 env_auth = false
37328 access_key_id = access_key
37329 secret_access_key = secret_key
37330 endpoint =
37331 zone = pek3a
37332 connection_retries =
37333 --------------------
37334 y) Yes this is OK
37335 e) Edit this remote
37336 d) Delete this remote
37337 y/e/d> y
37338
37339 This remote is called remote and can now be used like this
37340
37341 See all buckets
37342
37343 rclone lsd remote:
37344
37345 Make a new bucket
37346
37347 rclone mkdir remote:bucket
37348
37349 List the contents of a bucket
37350
37351 rclone ls remote:bucket
37352
37353 Sync /home/local/directory to the remote bucket, deleting any excess
37354 files in the bucket.
37355
37356 rclone sync -i /home/local/directory remote:bucket
37357
37358 --fast-list
37359 This remote supports --fast-list which allows you to use fewer transac‐
37360 tions in exchange for more memory. See the rclone docs
37361 (https://rclone.org/docs/#fast-list) for more details.
37362
37363 Multipart uploads
37364 rclone supports multipart uploads with QingStor which means that it can
37365 upload files bigger than 5 GiB. Note that files uploaded with multi‐
37366 part upload don't have an MD5SUM.
37367
37368 Note that incomplete multipart uploads older than 24 hours can be re‐
37369 moved with rclone cleanup remote:bucket just for one bucket rclone
37370 cleanup remote: for all buckets. QingStor does not ever remove incom‐
37371 plete multipart uploads so it may be necessary to run this from time to
37372 time.
37373
37374 Buckets and Zone
37375 With QingStor you can list buckets (rclone lsd) using any zone, but you
37376 can only access the content of a bucket from the zone it was created
37377 in. If you attempt to access a bucket from the wrong zone, you will
37378 get an error, incorrect zone, the bucket is not in 'XXX' zone.
37379
37380 Authentication
37381 There are two ways to supply rclone with a set of QingStor credentials.
37382 In order of precedence:
37383
37384 • Directly in the rclone configuration file (as configured by rclone
37385 config)
37386
37387 • set access_key_id and secret_access_key
37388
37389 • Runtime configuration:
37390
37391 • set env_auth to true in the config file
37392
37393 • Exporting the following environment variables before running rclone
37394
37395 • Access Key ID: QS_ACCESS_KEY_ID or QS_ACCESS_KEY
37396
37397 • Secret Access Key: QS_SECRET_ACCESS_KEY or QS_SECRET_KEY
37398
37399 Restricted filename characters
37400 The control characters 0x00-0x1F and / are replaced as in the default
37401 restricted characters set (https://rclone.org/overview/#restricted-
37402 characters). Note that 0x7F is not replaced.
37403
37404 Invalid UTF-8 bytes will also be replaced (https://rclone.org/over‐
37405 view/#invalid-utf8), as they can't be used in JSON strings.
37406
37407 Standard options
37408 Here are the Standard options specific to qingstor (QingCloud Object
37409 Storage).
37410
37411 --qingstor-env-auth
37412 Get QingStor credentials from runtime.
37413
37414 Only applies if access_key_id and secret_access_key is blank.
37415
37416 Properties:
37417
37418 • Config: env_auth
37419
37420 • Env Var: RCLONE_QINGSTOR_ENV_AUTH
37421
37422 • Type: bool
37423
37424 • Default: false
37425
37426 • Examples:
37427
37428 • "false"
37429
37430 • Enter QingStor credentials in the next step.
37431
37432 • "true"
37433
37434 • Get QingStor credentials from the environment (env vars or IAM).
37435
37436 --qingstor-access-key-id
37437 QingStor Access Key ID.
37438
37439 Leave blank for anonymous access or runtime credentials.
37440
37441 Properties:
37442
37443 • Config: access_key_id
37444
37445 • Env Var: RCLONE_QINGSTOR_ACCESS_KEY_ID
37446
37447 • Type: string
37448
37449 • Required: false
37450
37451 --qingstor-secret-access-key
37452 QingStor Secret Access Key (password).
37453
37454 Leave blank for anonymous access or runtime credentials.
37455
37456 Properties:
37457
37458 • Config: secret_access_key
37459
37460 • Env Var: RCLONE_QINGSTOR_SECRET_ACCESS_KEY
37461
37462 • Type: string
37463
37464 • Required: false
37465
37466 --qingstor-endpoint
37467 Enter an endpoint URL to connection QingStor API.
37468
37469 Leave blank will use the default value "https://qingstor.com:443".
37470
37471 Properties:
37472
37473 • Config: endpoint
37474
37475 • Env Var: RCLONE_QINGSTOR_ENDPOINT
37476
37477 • Type: string
37478
37479 • Required: false
37480
37481 --qingstor-zone
37482 Zone to connect to.
37483
37484 Default is "pek3a".
37485
37486 Properties:
37487
37488 • Config: zone
37489
37490 • Env Var: RCLONE_QINGSTOR_ZONE
37491
37492 • Type: string
37493
37494 • Required: false
37495
37496 • Examples:
37497
37498 • "pek3a"
37499
37500 • The Beijing (China) Three Zone.
37501
37502 • Needs location constraint pek3a.
37503
37504 • "sh1a"
37505
37506 • The Shanghai (China) First Zone.
37507
37508 • Needs location constraint sh1a.
37509
37510 • "gd2a"
37511
37512 • The Guangdong (China) Second Zone.
37513
37514 • Needs location constraint gd2a.
37515
37516 Advanced options
37517 Here are the Advanced options specific to qingstor (QingCloud Object
37518 Storage).
37519
37520 --qingstor-connection-retries
37521 Number of connection retries.
37522
37523 Properties:
37524
37525 • Config: connection_retries
37526
37527 • Env Var: RCLONE_QINGSTOR_CONNECTION_RETRIES
37528
37529 • Type: int
37530
37531 • Default: 3
37532
37533 --qingstor-upload-cutoff
37534 Cutoff for switching to chunked upload.
37535
37536 Any files larger than this will be uploaded in chunks of chunk_size.
37537 The minimum is 0 and the maximum is 5 GiB.
37538
37539 Properties:
37540
37541 • Config: upload_cutoff
37542
37543 • Env Var: RCLONE_QINGSTOR_UPLOAD_CUTOFF
37544
37545 • Type: SizeSuffix
37546
37547 • Default: 200Mi
37548
37549 --qingstor-chunk-size
37550 Chunk size to use for uploading.
37551
37552 When uploading files larger than upload_cutoff they will be uploaded as
37553 multipart uploads using this chunk size.
37554
37555 Note that "--qingstor-upload-concurrency" chunks of this size are
37556 buffered in memory per transfer.
37557
37558 If you are transferring large files over high-speed links and you have
37559 enough memory, then increasing this will speed up the transfers.
37560
37561 Properties:
37562
37563 • Config: chunk_size
37564
37565 • Env Var: RCLONE_QINGSTOR_CHUNK_SIZE
37566
37567 • Type: SizeSuffix
37568
37569 • Default: 4Mi
37570
37571 --qingstor-upload-concurrency
37572 Concurrency for multipart uploads.
37573
37574 This is the number of chunks of the same file that are uploaded concur‐
37575 rently.
37576
37577 NB if you set this to > 1 then the checksums of multipart uploads be‐
37578 come corrupted (the uploads themselves are not corrupted though).
37579
37580 If you are uploading small numbers of large files over high-speed links
37581 and these uploads do not fully utilize your bandwidth, then increasing
37582 this may help to speed up the transfers.
37583
37584 Properties:
37585
37586 • Config: upload_concurrency
37587
37588 • Env Var: RCLONE_QINGSTOR_UPLOAD_CONCURRENCY
37589
37590 • Type: int
37591
37592 • Default: 1
37593
37594 --qingstor-encoding
37595 The encoding for the backend.
37596
37597 See the encoding section in the overview (https://rclone.org/over‐
37598 view/#encoding) for more info.
37599
37600 Properties:
37601
37602 • Config: encoding
37603
37604 • Env Var: RCLONE_QINGSTOR_ENCODING
37605
37606 • Type: MultiEncoder
37607
37608 • Default: Slash,Ctl,InvalidUtf8
37609
37610 Limitations
37611 rclone about is not supported by the qingstor backend. Backends with‐
37612 out this capability cannot determine free space for an rclone mount or
37613 use policy mfs (most free space) as a member of an rclone union remote.
37614
37615 See List of backends that do not support rclone about
37616 (https://rclone.org/overview/#optional-features) and rclone about
37617 (https://rclone.org/commands/rclone_about/)
37618
37620 Sia (sia.tech (https://sia.tech/)) is a decentralized cloud storage
37621 platform based on the blockchain (https://wikipedia.org/wi‐
37622 ki/Blockchain) technology. With rclone you can use it like any other
37623 remote filesystem or mount Sia folders locally. The technology behind
37624 it involves a number of new concepts such as Siacoins and Wallet,
37625 Blockchain and Consensus, Renting and Hosting, and so on. If you are
37626 new to it, you'd better first familiarize yourself using their excel‐
37627 lent support documentation (https://support.sia.tech/).
37628
37629 Introduction
37630 Before you can use rclone with Sia, you will need to have a running
37631 copy of Sia-UI or siad (the Sia daemon) locally on your computer or on
37632 local network (e.g. a NAS). Please follow the Get started
37633 (https://sia.tech/get-started) guide and install one.
37634
37635 rclone interacts with Sia network by talking to the Sia daemon via HTTP
37636 API (https://sia.tech/docs/) which is usually available on port 9980.
37637 By default you will run the daemon locally on the same computer so it's
37638 safe to leave the API password blank (the API URL will be
37639 http://127.0.0.1:9980 making external access impossible).
37640
37641 However, if you want to access Sia daemon running on another node, for
37642 example due to memory constraints or because you want to share single
37643 daemon between several rclone and Sia-UI instances, you'll need to make
37644 a few more provisions: - Ensure you have Sia daemon installed directly
37645 or in a docker container (https://github.com/SiaFounda‐
37646 tion/siad/pkgs/container/siad) because Sia-UI does not support this
37647 mode natively. - Run it on externally accessible port, for example
37648 provide --api-addr :9980 and --disable-api-security arguments on the
37649 daemon command line. - Enforce API password for the siad daemon via
37650 environment variable SIA_API_PASSWORD or text file named apipassword in
37651 the daemon directory. - Set rclone backend option api_password taking
37652 it from above locations.
37653
37654 Notes: 1. If your wallet is locked, rclone cannot unlock it automati‐
37655 cally. You should either unlock it in advance by using Sia-UI or via
37656 command line siac wallet unlock. Alternatively you can make siad un‐
37657 lock your wallet automatically upon startup by running it with environ‐
37658 ment variable SIA_WALLET_PASSWORD. 2. If siad cannot find the
37659 SIA_API_PASSWORD variable or the apipassword file in the SIA_DIR direc‐
37660 tory, it will generate a random password and store in the text file
37661 named apipassword under YOUR_HOME/.sia/ directory on Unix or
37662 C:\Users\YOUR_HOME\AppData\Local\Sia\apipassword on Windows. Remember
37663 this when you configure password in rclone. 3. The only way to use
37664 siad without API password is to run it on localhost with command line
37665 argument --authorize-api=false, but this is insecure and strongly dis‐
37666 couraged.
37667
37668 Configuration
37669 Here is an example of how to make a sia remote called mySia. First,
37670 run:
37671
37672 rclone config
37673
37674 This will guide you through an interactive setup process:
37675
37676 No remotes found, make a new one?
37677 n) New remote
37678 s) Set configuration password
37679 q) Quit config
37680 n/s/q> n
37681 name> mySia
37682 Type of storage to configure.
37683 Enter a string value. Press Enter for the default ("").
37684 Choose a number from below, or type in your own value
37685 ...
37686 29 / Sia Decentralized Cloud
37687 \ "sia"
37688 ...
37689 Storage> sia
37690 Sia daemon API URL, like http://sia.daemon.host:9980.
37691 Note that siad must run with --disable-api-security to open API port for other hosts (not recommended).
37692 Keep default if Sia daemon runs on localhost.
37693 Enter a string value. Press Enter for the default ("http://127.0.0.1:9980").
37694 api_url> http://127.0.0.1:9980
37695 Sia Daemon API Password.
37696 Can be found in the apipassword file located in HOME/.sia/ or in the daemon directory.
37697 y) Yes type in my own password
37698 g) Generate random password
37699 n) No leave this optional password blank (default)
37700 y/g/n> y
37701 Enter the password:
37702 password:
37703 Confirm the password:
37704 password:
37705 Edit advanced config?
37706 y) Yes
37707 n) No (default)
37708 y/n> n
37709 --------------------
37710 [mySia]
37711 type = sia
37712 api_url = http://127.0.0.1:9980
37713 api_password = *** ENCRYPTED ***
37714 --------------------
37715 y) Yes this is OK (default)
37716 e) Edit this remote
37717 d) Delete this remote
37718 y/e/d> y
37719
37720 Once configured, you can then use rclone like this:
37721
37722 • List directories in top level of your Sia storage
37723
37724 rclone lsd mySia:
37725
37726 • List all the files in your Sia storage
37727
37728 rclone ls mySia:
37729
37730 • Upload a local directory to the Sia directory called backup
37731
37732 rclone copy /home/source mySia:backup
37733
37734 Standard options
37735 Here are the Standard options specific to sia (Sia Decentralized
37736 Cloud).
37737
37738 --sia-api-url
37739 Sia daemon API URL, like http://sia.daemon.host:9980.
37740
37741 Note that siad must run with --disable-api-security to open API port
37742 for other hosts (not recommended). Keep default if Sia daemon runs on
37743 localhost.
37744
37745 Properties:
37746
37747 • Config: api_url
37748
37749 • Env Var: RCLONE_SIA_API_URL
37750
37751 • Type: string
37752
37753 • Default: "http://127.0.0.1:9980"
37754
37755 --sia-api-password
37756 Sia Daemon API Password.
37757
37758 Can be found in the apipassword file located in HOME/.sia/ or in the
37759 daemon directory.
37760
37761 NB Input to this must be obscured - see rclone obscure
37762 (https://rclone.org/commands/rclone_obscure/).
37763
37764 Properties:
37765
37766 • Config: api_password
37767
37768 • Env Var: RCLONE_SIA_API_PASSWORD
37769
37770 • Type: string
37771
37772 • Required: false
37773
37774 Advanced options
37775 Here are the Advanced options specific to sia (Sia Decentralized
37776 Cloud).
37777
37778 --sia-user-agent
37779 Siad User Agent
37780
37781 Sia daemon requires the 'Sia-Agent' user agent by default for security
37782
37783 Properties:
37784
37785 • Config: user_agent
37786
37787 • Env Var: RCLONE_SIA_USER_AGENT
37788
37789 • Type: string
37790
37791 • Default: "Sia-Agent"
37792
37793 --sia-encoding
37794 The encoding for the backend.
37795
37796 See the encoding section in the overview (https://rclone.org/over‐
37797 view/#encoding) for more info.
37798
37799 Properties:
37800
37801 • Config: encoding
37802
37803 • Env Var: RCLONE_SIA_ENCODING
37804
37805 • Type: MultiEncoder
37806
37807 • Default: Slash,Question,Hash,Percent,Del,Ctl,InvalidUtf8,Dot
37808
37809 Limitations
37810 • Modification times not supported
37811
37812 • Checksums not supported
37813
37814 • rclone about not supported
37815
37816 • rclone can work only with Siad or Sia-UI at the moment, the SkyNet
37817 daemon is not supported yet.
37818
37819 • Sia does not allow control characters or symbols like question and
37820 pound signs in file names. rclone will transparently encode
37821 (https://rclone.org/overview/#encoding) them for you, but you'd bet‐
37822 ter be aware
37823
37825 Swift refers to OpenStack Object Storage (https://docs.open‐
37826 stack.org/swift/latest/). Commercial implementations of that being:
37827
37828 • Rackspace Cloud Files (https://www.rackspace.com/cloud/files/)
37829
37830 • Memset Memstore (https://www.memset.com/cloud/storage/)
37831
37832 • OVH Object Storage (https://www.ovh.co.uk/public-cloud/storage/ob‐
37833 ject-storage/)
37834
37835 • Oracle Cloud Storage (https://docs.oracle.com/en-us/iaas/integra‐
37836 tion/doc/configure-object-storage.html)
37837
37838 • IBM Bluemix Cloud ObjectStorage Swift (https://con‐
37839 sole.bluemix.net/docs/infrastructure/objectstorage-swift/index.html)
37840
37841 Paths are specified as remote:container (or remote: for the lsd com‐
37842 mand.) You may put subdirectories in too, e.g. remote:contain‐
37843 er/path/to/dir.
37844
37845 Configuration
37846 Here is an example of making a swift configuration. First run
37847
37848 rclone config
37849
37850 This will guide you through an interactive setup process.
37851
37852 No remotes found, make a new one?
37853 n) New remote
37854 s) Set configuration password
37855 q) Quit config
37856 n/s/q> n
37857 name> remote
37858 Type of storage to configure.
37859 Choose a number from below, or type in your own value
37860 [snip]
37861 XX / OpenStack Swift (Rackspace Cloud Files, Memset Memstore, OVH)
37862 \ "swift"
37863 [snip]
37864 Storage> swift
37865 Get swift credentials from environment variables in standard OpenStack form.
37866 Choose a number from below, or type in your own value
37867 1 / Enter swift credentials in the next step
37868 \ "false"
37869 2 / Get swift credentials from environment vars. Leave other fields blank if using this.
37870 \ "true"
37871 env_auth> true
37872 User name to log in (OS_USERNAME).
37873 user>
37874 API key or password (OS_PASSWORD).
37875 key>
37876 Authentication URL for server (OS_AUTH_URL).
37877 Choose a number from below, or type in your own value
37878 1 / Rackspace US
37879 \ "https://auth.api.rackspacecloud.com/v1.0"
37880 2 / Rackspace UK
37881 \ "https://lon.auth.api.rackspacecloud.com/v1.0"
37882 3 / Rackspace v2
37883 \ "https://identity.api.rackspacecloud.com/v2.0"
37884 4 / Memset Memstore UK
37885 \ "https://auth.storage.memset.com/v1.0"
37886 5 / Memset Memstore UK v2
37887 \ "https://auth.storage.memset.com/v2.0"
37888 6 / OVH
37889 \ "https://auth.cloud.ovh.net/v3"
37890 auth>
37891 User ID to log in - optional - most swift systems use user and leave this blank (v3 auth) (OS_USER_ID).
37892 user_id>
37893 User domain - optional (v3 auth) (OS_USER_DOMAIN_NAME)
37894 domain>
37895 Tenant name - optional for v1 auth, this or tenant_id required otherwise (OS_TENANT_NAME or OS_PROJECT_NAME)
37896 tenant>
37897 Tenant ID - optional for v1 auth, this or tenant required otherwise (OS_TENANT_ID)
37898 tenant_id>
37899 Tenant domain - optional (v3 auth) (OS_PROJECT_DOMAIN_NAME)
37900 tenant_domain>
37901 Region name - optional (OS_REGION_NAME)
37902 region>
37903 Storage URL - optional (OS_STORAGE_URL)
37904 storage_url>
37905 Auth Token from alternate authentication - optional (OS_AUTH_TOKEN)
37906 auth_token>
37907 AuthVersion - optional - set to (1,2,3) if your auth URL has no version (ST_AUTH_VERSION)
37908 auth_version>
37909 Endpoint type to choose from the service catalogue (OS_ENDPOINT_TYPE)
37910 Choose a number from below, or type in your own value
37911 1 / Public (default, choose this if not sure)
37912 \ "public"
37913 2 / Internal (use internal service net)
37914 \ "internal"
37915 3 / Admin
37916 \ "admin"
37917 endpoint_type>
37918 Remote config
37919 --------------------
37920 [test]
37921 env_auth = true
37922 user =
37923 key =
37924 auth =
37925 user_id =
37926 domain =
37927 tenant =
37928 tenant_id =
37929 tenant_domain =
37930 region =
37931 storage_url =
37932 auth_token =
37933 auth_version =
37934 endpoint_type =
37935 --------------------
37936 y) Yes this is OK
37937 e) Edit this remote
37938 d) Delete this remote
37939 y/e/d> y
37940
37941 This remote is called remote and can now be used like this
37942
37943 See all containers
37944
37945 rclone lsd remote:
37946
37947 Make a new container
37948
37949 rclone mkdir remote:container
37950
37951 List the contents of a container
37952
37953 rclone ls remote:container
37954
37955 Sync /home/local/directory to the remote container, deleting any excess
37956 files in the container.
37957
37958 rclone sync -i /home/local/directory remote:container
37959
37960 Configuration from an OpenStack credentials file
37961 An OpenStack credentials file typically looks something something like
37962 this (without the comments)
37963
37964 export OS_AUTH_URL=https://a.provider.net/v2.0
37965 export OS_TENANT_ID=ffffffffffffffffffffffffffffffff
37966 export OS_TENANT_NAME="1234567890123456"
37967 export OS_USERNAME="123abc567xy"
37968 echo "Please enter your OpenStack Password: "
37969 read -sr OS_PASSWORD_INPUT
37970 export OS_PASSWORD=$OS_PASSWORD_INPUT
37971 export OS_REGION_NAME="SBG1"
37972 if [ -z "$OS_REGION_NAME" ]; then unset OS_REGION_NAME; fi
37973
37974 The config file needs to look something like this where $OS_USERNAME
37975 represents the value of the OS_USERNAME variable - 123abc567xy in the
37976 example above.
37977
37978 [remote]
37979 type = swift
37980 user = $OS_USERNAME
37981 key = $OS_PASSWORD
37982 auth = $OS_AUTH_URL
37983 tenant = $OS_TENANT_NAME
37984
37985 Note that you may (or may not) need to set region too - try without
37986 first.
37987
37988 Configuration from the environment
37989 If you prefer you can configure rclone to use swift using a standard
37990 set of OpenStack environment variables.
37991
37992 When you run through the config, make sure you choose true for env_auth
37993 and leave everything else blank.
37994
37995 rclone will then set any empty config parameters from the environment
37996 using standard OpenStack environment variables. There is a list of the
37997 variables (https://godoc.org/github.com/ncw/swift#Connection.ApplyEnvi‐
37998 ronment) in the docs for the swift library.
37999
38000 Using an alternate authentication method
38001 If your OpenStack installation uses a non-standard authentication meth‐
38002 od that might not be yet supported by rclone or the underlying swift
38003 library, you can authenticate externally (e.g. calling manually the
38004 openstack commands to get a token). Then, you just need to pass the
38005 two configuration variables auth_token and storage_url. If they are
38006 both provided, the other variables are ignored. rclone will not try to
38007 authenticate but instead assume it is already authenticated and use
38008 these two variables to access the OpenStack installation.
38009
38010 Using rclone without a config file
38011 You can use rclone with swift without a config file, if desired, like
38012 this:
38013
38014 source openstack-credentials-file
38015 export RCLONE_CONFIG_MYREMOTE_TYPE=swift
38016 export RCLONE_CONFIG_MYREMOTE_ENV_AUTH=true
38017 rclone lsd myremote:
38018
38019 --fast-list
38020 This remote supports --fast-list which allows you to use fewer transac‐
38021 tions in exchange for more memory. See the rclone docs
38022 (https://rclone.org/docs/#fast-list) for more details.
38023
38024 --update and --use-server-modtime
38025 As noted below, the modified time is stored on metadata on the object.
38026 It is used by default for all operations that require checking the time
38027 a file was last updated. It allows rclone to treat the remote more
38028 like a true filesystem, but it is inefficient because it requires an
38029 extra API call to retrieve the metadata.
38030
38031 For many operations, the time the object was last uploaded to the re‐
38032 mote is sufficient to determine if it is "dirty". By using --update
38033 along with --use-server-modtime, you can avoid the extra API call and
38034 simply upload files whose local modtime is newer than the time it was
38035 last uploaded.
38036
38037 Modified time
38038 The modified time is stored as metadata on the object as X-Object-Meta-
38039 Mtime as floating point since the epoch accurate to 1 ns.
38040
38041 This is a de facto standard (used in the official python-swiftclient
38042 amongst others) for storing the modification time for an object.
38043
38044 Restricted filename characters
38045 Character Value Replacement
38046 ────────────────────────────────
38047 NUL 0x00 ␀
38048 / 0x2F /
38049
38050 Invalid UTF-8 bytes will also be replaced (https://rclone.org/over‐
38051 view/#invalid-utf8), as they can't be used in JSON strings.
38052
38053 Standard options
38054 Here are the Standard options specific to swift (OpenStack Swift
38055 (Rackspace Cloud Files, Memset Memstore, OVH)).
38056
38057 --swift-env-auth
38058 Get swift credentials from environment variables in standard OpenStack
38059 form.
38060
38061 Properties:
38062
38063 • Config: env_auth
38064
38065 • Env Var: RCLONE_SWIFT_ENV_AUTH
38066
38067 • Type: bool
38068
38069 • Default: false
38070
38071 • Examples:
38072
38073 • "false"
38074
38075 • Enter swift credentials in the next step.
38076
38077 • "true"
38078
38079 • Get swift credentials from environment vars.
38080
38081 • Leave other fields blank if using this.
38082
38083 --swift-user
38084 User name to log in (OS_USERNAME).
38085
38086 Properties:
38087
38088 • Config: user
38089
38090 • Env Var: RCLONE_SWIFT_USER
38091
38092 • Type: string
38093
38094 • Required: false
38095
38096 --swift-key
38097 API key or password (OS_PASSWORD).
38098
38099 Properties:
38100
38101 • Config: key
38102
38103 • Env Var: RCLONE_SWIFT_KEY
38104
38105 • Type: string
38106
38107 • Required: false
38108
38109 --swift-auth
38110 Authentication URL for server (OS_AUTH_URL).
38111
38112 Properties:
38113
38114 • Config: auth
38115
38116 • Env Var: RCLONE_SWIFT_AUTH
38117
38118 • Type: string
38119
38120 • Required: false
38121
38122 • Examples:
38123
38124 • "https://auth.api.rackspacecloud.com/v1.0"
38125
38126 • Rackspace US
38127
38128 • "https://lon.auth.api.rackspacecloud.com/v1.0"
38129
38130 • Rackspace UK
38131
38132 • "https://identity.api.rackspacecloud.com/v2.0"
38133
38134 • Rackspace v2
38135
38136 • "https://auth.storage.memset.com/v1.0"
38137
38138 • Memset Memstore UK
38139
38140 • "https://auth.storage.memset.com/v2.0"
38141
38142 • Memset Memstore UK v2
38143
38144 • "https://auth.cloud.ovh.net/v3"
38145
38146 • OVH
38147
38148 --swift-user-id
38149 User ID to log in - optional - most swift systems use user and leave
38150 this blank (v3 auth) (OS_USER_ID).
38151
38152 Properties:
38153
38154 • Config: user_id
38155
38156 • Env Var: RCLONE_SWIFT_USER_ID
38157
38158 • Type: string
38159
38160 • Required: false
38161
38162 --swift-domain
38163 User domain - optional (v3 auth) (OS_USER_DOMAIN_NAME)
38164
38165 Properties:
38166
38167 • Config: domain
38168
38169 • Env Var: RCLONE_SWIFT_DOMAIN
38170
38171 • Type: string
38172
38173 • Required: false
38174
38175 --swift-tenant
38176 Tenant name - optional for v1 auth, this or tenant_id required other‐
38177 wise (OS_TENANT_NAME or OS_PROJECT_NAME).
38178
38179 Properties:
38180
38181 • Config: tenant
38182
38183 • Env Var: RCLONE_SWIFT_TENANT
38184
38185 • Type: string
38186
38187 • Required: false
38188
38189 --swift-tenant-id
38190 Tenant ID - optional for v1 auth, this or tenant required otherwise
38191 (OS_TENANT_ID).
38192
38193 Properties:
38194
38195 • Config: tenant_id
38196
38197 • Env Var: RCLONE_SWIFT_TENANT_ID
38198
38199 • Type: string
38200
38201 • Required: false
38202
38203 --swift-tenant-domain
38204 Tenant domain - optional (v3 auth) (OS_PROJECT_DOMAIN_NAME).
38205
38206 Properties:
38207
38208 • Config: tenant_domain
38209
38210 • Env Var: RCLONE_SWIFT_TENANT_DOMAIN
38211
38212 • Type: string
38213
38214 • Required: false
38215
38216 --swift-region
38217 Region name - optional (OS_REGION_NAME).
38218
38219 Properties:
38220
38221 • Config: region
38222
38223 • Env Var: RCLONE_SWIFT_REGION
38224
38225 • Type: string
38226
38227 • Required: false
38228
38229 --swift-storage-url
38230 Storage URL - optional (OS_STORAGE_URL).
38231
38232 Properties:
38233
38234 • Config: storage_url
38235
38236 • Env Var: RCLONE_SWIFT_STORAGE_URL
38237
38238 • Type: string
38239
38240 • Required: false
38241
38242 --swift-auth-token
38243 Auth Token from alternate authentication - optional (OS_AUTH_TOKEN).
38244
38245 Properties:
38246
38247 • Config: auth_token
38248
38249 • Env Var: RCLONE_SWIFT_AUTH_TOKEN
38250
38251 • Type: string
38252
38253 • Required: false
38254
38255 --swift-application-credential-id
38256 Application Credential ID (OS_APPLICATION_CREDENTIAL_ID).
38257
38258 Properties:
38259
38260 • Config: application_credential_id
38261
38262 • Env Var: RCLONE_SWIFT_APPLICATION_CREDENTIAL_ID
38263
38264 • Type: string
38265
38266 • Required: false
38267
38268 --swift-application-credential-name
38269 Application Credential Name (OS_APPLICATION_CREDENTIAL_NAME).
38270
38271 Properties:
38272
38273 • Config: application_credential_name
38274
38275 • Env Var: RCLONE_SWIFT_APPLICATION_CREDENTIAL_NAME
38276
38277 • Type: string
38278
38279 • Required: false
38280
38281 --swift-application-credential-secret
38282 Application Credential Secret (OS_APPLICATION_CREDENTIAL_SECRET).
38283
38284 Properties:
38285
38286 • Config: application_credential_secret
38287
38288 • Env Var: RCLONE_SWIFT_APPLICATION_CREDENTIAL_SECRET
38289
38290 • Type: string
38291
38292 • Required: false
38293
38294 --swift-auth-version
38295 AuthVersion - optional - set to (1,2,3) if your auth URL has no version
38296 (ST_AUTH_VERSION).
38297
38298 Properties:
38299
38300 • Config: auth_version
38301
38302 • Env Var: RCLONE_SWIFT_AUTH_VERSION
38303
38304 • Type: int
38305
38306 • Default: 0
38307
38308 --swift-endpoint-type
38309 Endpoint type to choose from the service catalogue (OS_ENDPOINT_TYPE).
38310
38311 Properties:
38312
38313 • Config: endpoint_type
38314
38315 • Env Var: RCLONE_SWIFT_ENDPOINT_TYPE
38316
38317 • Type: string
38318
38319 • Default: "public"
38320
38321 • Examples:
38322
38323 • "public"
38324
38325 • Public (default, choose this if not sure)
38326
38327 • "internal"
38328
38329 • Internal (use internal service net)
38330
38331 • "admin"
38332
38333 • Admin
38334
38335 --swift-storage-policy
38336 The storage policy to use when creating a new container.
38337
38338 This applies the specified storage policy when creating a new contain‐
38339 er. The policy cannot be changed afterwards. The allowed configura‐
38340 tion values and their meaning depend on your Swift storage provider.
38341
38342 Properties:
38343
38344 • Config: storage_policy
38345
38346 • Env Var: RCLONE_SWIFT_STORAGE_POLICY
38347
38348 • Type: string
38349
38350 • Required: false
38351
38352 • Examples:
38353
38354 • ""
38355
38356 • Default
38357
38358 • "pcs"
38359
38360 • OVH Public Cloud Storage
38361
38362 • "pca"
38363
38364 • OVH Public Cloud Archive
38365
38366 Advanced options
38367 Here are the Advanced options specific to swift (OpenStack Swift
38368 (Rackspace Cloud Files, Memset Memstore, OVH)).
38369
38370 --swift-leave-parts-on-error
38371 If true avoid calling abort upload on a failure.
38372
38373 It should be set to true for resuming uploads across different ses‐
38374 sions.
38375
38376 Properties:
38377
38378 • Config: leave_parts_on_error
38379
38380 • Env Var: RCLONE_SWIFT_LEAVE_PARTS_ON_ERROR
38381
38382 • Type: bool
38383
38384 • Default: false
38385
38386 --swift-chunk-size
38387 Above this size files will be chunked into a _segments container.
38388
38389 Above this size files will be chunked into a _segments container. The
38390 default for this is 5 GiB which is its maximum value.
38391
38392 Properties:
38393
38394 • Config: chunk_size
38395
38396 • Env Var: RCLONE_SWIFT_CHUNK_SIZE
38397
38398 • Type: SizeSuffix
38399
38400 • Default: 5Gi
38401
38402 --swift-no-chunk
38403 Don't chunk files during streaming upload.
38404
38405 When doing streaming uploads (e.g. using rcat or mount) setting this
38406 flag will cause the swift backend to not upload chunked files.
38407
38408 This will limit the maximum upload size to 5 GiB. However non chunked
38409 files are easier to deal with and have an MD5SUM.
38410
38411 Rclone will still chunk files bigger than chunk_size when doing normal
38412 copy operations.
38413
38414 Properties:
38415
38416 • Config: no_chunk
38417
38418 • Env Var: RCLONE_SWIFT_NO_CHUNK
38419
38420 • Type: bool
38421
38422 • Default: false
38423
38424 --swift-no-large-objects
38425 Disable support for static and dynamic large objects
38426
38427 Swift cannot transparently store files bigger than 5 GiB. There are
38428 two schemes for doing that, static or dynamic large objects, and the
38429 API does not allow rclone to determine whether a file is a static or
38430 dynamic large object without doing a HEAD on the object. Since these
38431 need to be treated differently, this means rclone has to issue HEAD re‐
38432 quests for objects for example when reading checksums.
38433
38434 When no_large_objects is set, rclone will assume that there are no
38435 static or dynamic large objects stored. This means it can stop doing
38436 the extra HEAD calls which in turn increases performance greatly espe‐
38437 cially when doing a swift to swift transfer with --checksum set.
38438
38439 Setting this option implies no_chunk and also that no files will be up‐
38440 loaded in chunks, so files bigger than 5 GiB will just fail on upload.
38441
38442 If you set this option and there are static or dynamic large objects,
38443 then this will give incorrect hashes for them. Downloads will succeed,
38444 but other operations such as Remove and Copy will fail.
38445
38446 Properties:
38447
38448 • Config: no_large_objects
38449
38450 • Env Var: RCLONE_SWIFT_NO_LARGE_OBJECTS
38451
38452 • Type: bool
38453
38454 • Default: false
38455
38456 --swift-encoding
38457 The encoding for the backend.
38458
38459 See the encoding section in the overview (https://rclone.org/over‐
38460 view/#encoding) for more info.
38461
38462 Properties:
38463
38464 • Config: encoding
38465
38466 • Env Var: RCLONE_SWIFT_ENCODING
38467
38468 • Type: MultiEncoder
38469
38470 • Default: Slash,InvalidUtf8
38471
38472 Limitations
38473 The Swift API doesn't return a correct MD5SUM for segmented files (Dy‐
38474 namic or Static Large Objects) so rclone won't check or use the MD5SUM
38475 for these.
38476
38477 Troubleshooting
38478 Rclone gives Failed to create file system for "remote:": Bad Request
38479 Due to an oddity of the underlying swift library, it gives a "Bad Re‐
38480 quest" error rather than a more sensible error when the authentication
38481 fails for Swift.
38482
38483 So this most likely means your username / password is wrong. You can
38484 investigate further with the --dump-bodies flag.
38485
38486 This may also be caused by specifying the region when you shouldn't
38487 have (e.g. OVH).
38488
38489 Rclone gives Failed to create file system: Response didn't have storage url
38490 and auth token
38491 This is most likely caused by forgetting to specify your tenant when
38492 setting up a swift remote.
38493
38494 OVH Cloud Archive
38495 To use rclone with OVH cloud archive, first use rclone config to set up
38496 a swift backend with OVH, choosing pca as the storage_policy.
38497
38498 Uploading Objects
38499 Uploading objects to OVH cloud archive is no different to object stor‐
38500 age, you just simply run the command you like (move, copy or sync) to
38501 upload the objects. Once uploaded the objects will show in a "Frozen"
38502 state within the OVH control panel.
38503
38504 Retrieving Objects
38505 To retrieve objects use rclone copy as normal. If the objects are in a
38506 frozen state then rclone will ask for them all to be unfrozen and it
38507 will wait at the end of the output with a message like the following:
38508
38509 2019/03/23 13:06:33 NOTICE: Received retry after error - sleeping until
38510 2019-03-23T13:16:33.481657164+01:00 (9m59.99985121s)
38511
38512 Rclone will wait for the time specified then retry the copy.
38513
38515 Paths are specified as remote:path
38516
38517 Paths may be as deep as required, e.g. remote:directory/subdirectory.
38518
38519 Configuration
38520 The initial setup for pCloud involves getting a token from pCloud which
38521 you need to do in your browser. rclone config walks you through it.
38522
38523 Here is an example of how to make a remote called remote. First run:
38524
38525 rclone config
38526
38527 This will guide you through an interactive setup process:
38528
38529 No remotes found, make a new one?
38530 n) New remote
38531 s) Set configuration password
38532 q) Quit config
38533 n/s/q> n
38534 name> remote
38535 Type of storage to configure.
38536 Choose a number from below, or type in your own value
38537 [snip]
38538 XX / Pcloud
38539 \ "pcloud"
38540 [snip]
38541 Storage> pcloud
38542 Pcloud App Client Id - leave blank normally.
38543 client_id>
38544 Pcloud App Client Secret - leave blank normally.
38545 client_secret>
38546 Remote config
38547 Use auto config?
38548 * Say Y if not sure
38549 * Say N if you are working on a remote or headless machine
38550 y) Yes
38551 n) No
38552 y/n> y
38553 If your browser doesn't open automatically go to the following link: http://127.0.0.1:53682/auth
38554 Log in and authorize rclone for access
38555 Waiting for code...
38556 Got code
38557 --------------------
38558 [remote]
38559 client_id =
38560 client_secret =
38561 token = {"access_token":"XXX","token_type":"bearer","expiry":"0001-01-01T00:00:00Z"}
38562 --------------------
38563 y) Yes this is OK
38564 e) Edit this remote
38565 d) Delete this remote
38566 y/e/d> y
38567
38568 See the remote setup docs (https://rclone.org/remote_setup/) for how to
38569 set it up on a machine with no Internet browser available.
38570
38571 Note that rclone runs a webserver on your local machine to collect the
38572 token as returned from pCloud. This only runs from the moment it opens
38573 your browser to the moment you get back the verification code. This is
38574 on http://127.0.0.1:53682/ and this it may require you to unblock it
38575 temporarily if you are running a host firewall.
38576
38577 Once configured you can then use rclone like this,
38578
38579 List directories in top level of your pCloud
38580
38581 rclone lsd remote:
38582
38583 List all the files in your pCloud
38584
38585 rclone ls remote:
38586
38587 To copy a local directory to a pCloud directory called backup
38588
38589 rclone copy /home/source remote:backup
38590
38591 Modified time and hashes
38592 pCloud allows modification times to be set on objects accurate to 1
38593 second. These will be used to detect whether objects need syncing or
38594 not. In order to set a Modification time pCloud requires the object be
38595 re-uploaded.
38596
38597 pCloud supports MD5 and SHA1 hashes in the US region, and SHA1 and
38598 SHA256 hashes in the EU region, so you can use the --checksum flag.
38599
38600 Restricted filename characters
38601 In addition to the default restricted characters set
38602 (https://rclone.org/overview/#restricted-characters) the following
38603 characters are also replaced:
38604
38605 Character Value Replacement
38606 ────────────────────────────────
38607 \ 0x5C \
38608
38609 Invalid UTF-8 bytes will also be replaced (https://rclone.org/over‐
38610 view/#invalid-utf8), as they can't be used in JSON strings.
38611
38612 Deleting files
38613 Deleted files will be moved to the trash. Your subscription level will
38614 determine how long items stay in the trash. rclone cleanup can be used
38615 to empty the trash.
38616
38617 Emptying the trash
38618 Due to an API limitation, the rclone cleanup command will only work if
38619 you set your username and password in the advanced options for this
38620 backend. Since we generally want to avoid storing user passwords in
38621 the rclone config file, we advise you to only set this up if you need
38622 the rclone cleanup command to work.
38623
38624 Root folder ID
38625 You can set the root_folder_id for rclone. This is the directory
38626 (identified by its Folder ID) that rclone considers to be the root of
38627 your pCloud drive.
38628
38629 Normally you will leave this blank and rclone will determine the cor‐
38630 rect root to use itself.
38631
38632 However you can set this to restrict rclone to a specific folder hier‐
38633 archy.
38634
38635 In order to do this you will have to find the Folder ID of the directo‐
38636 ry you wish rclone to display. This will be the folder field of the
38637 URL when you open the relevant folder in the pCloud web interface.
38638
38639 So if the folder you want rclone to use has a URL which looks like
38640 https://my.pcloud.com/#page=filemanager&folder=5xxxxxxxx8&tpl=folder‐
38641 grid in the browser, then you use 5xxxxxxxx8 as the root_folder_id in
38642 the config.
38643
38644 Standard options
38645 Here are the Standard options specific to pcloud (Pcloud).
38646
38647 --pcloud-client-id
38648 OAuth Client Id.
38649
38650 Leave blank normally.
38651
38652 Properties:
38653
38654 • Config: client_id
38655
38656 • Env Var: RCLONE_PCLOUD_CLIENT_ID
38657
38658 • Type: string
38659
38660 • Required: false
38661
38662 --pcloud-client-secret
38663 OAuth Client Secret.
38664
38665 Leave blank normally.
38666
38667 Properties:
38668
38669 • Config: client_secret
38670
38671 • Env Var: RCLONE_PCLOUD_CLIENT_SECRET
38672
38673 • Type: string
38674
38675 • Required: false
38676
38677 Advanced options
38678 Here are the Advanced options specific to pcloud (Pcloud).
38679
38680 --pcloud-token
38681 OAuth Access Token as a JSON blob.
38682
38683 Properties:
38684
38685 • Config: token
38686
38687 • Env Var: RCLONE_PCLOUD_TOKEN
38688
38689 • Type: string
38690
38691 • Required: false
38692
38693 --pcloud-auth-url
38694 Auth server URL.
38695
38696 Leave blank to use the provider defaults.
38697
38698 Properties:
38699
38700 • Config: auth_url
38701
38702 • Env Var: RCLONE_PCLOUD_AUTH_URL
38703
38704 • Type: string
38705
38706 • Required: false
38707
38708 --pcloud-token-url
38709 Token server url.
38710
38711 Leave blank to use the provider defaults.
38712
38713 Properties:
38714
38715 • Config: token_url
38716
38717 • Env Var: RCLONE_PCLOUD_TOKEN_URL
38718
38719 • Type: string
38720
38721 • Required: false
38722
38723 --pcloud-encoding
38724 The encoding for the backend.
38725
38726 See the encoding section in the overview (https://rclone.org/over‐
38727 view/#encoding) for more info.
38728
38729 Properties:
38730
38731 • Config: encoding
38732
38733 • Env Var: RCLONE_PCLOUD_ENCODING
38734
38735 • Type: MultiEncoder
38736
38737 • Default: Slash,BackSlash,Del,Ctl,InvalidUtf8,Dot
38738
38739 --pcloud-root-folder-id
38740 Fill in for rclone to use a non root folder as its starting point.
38741
38742 Properties:
38743
38744 • Config: root_folder_id
38745
38746 • Env Var: RCLONE_PCLOUD_ROOT_FOLDER_ID
38747
38748 • Type: string
38749
38750 • Default: "d0"
38751
38752 --pcloud-hostname
38753 Hostname to connect to.
38754
38755 This is normally set when rclone initially does the oauth connection,
38756 however you will need to set it by hand if you are using remote config
38757 with rclone authorize.
38758
38759 Properties:
38760
38761 • Config: hostname
38762
38763 • Env Var: RCLONE_PCLOUD_HOSTNAME
38764
38765 • Type: string
38766
38767 • Default: "api.pcloud.com"
38768
38769 • Examples:
38770
38771 • "api.pcloud.com"
38772
38773 • Original/US region
38774
38775 • "eapi.pcloud.com"
38776
38777 • EU region
38778
38779 --pcloud-username
38780 Your pcloud username.
38781
38782 This is only required when you want to use the cleanup command. Due to
38783 a bug in the pcloud API the required API does not support OAuth authen‐
38784 tication so we have to rely on user password authentication for it.
38785
38786 Properties:
38787
38788 • Config: username
38789
38790 • Env Var: RCLONE_PCLOUD_USERNAME
38791
38792 • Type: string
38793
38794 • Required: false
38795
38796 --pcloud-password
38797 Your pcloud password.
38798
38799 NB Input to this must be obscured - see rclone obscure
38800 (https://rclone.org/commands/rclone_obscure/).
38801
38802 Properties:
38803
38804 • Config: password
38805
38806 • Env Var: RCLONE_PCLOUD_PASSWORD
38807
38808 • Type: string
38809
38810 • Required: false
38811
38813 Paths are specified as remote:path
38814
38815 Paths may be as deep as required, e.g. remote:directory/subdirectory.
38816
38817 Configuration
38818 The initial setup for premiumize.me (https://premiumize.me/) involves
38819 getting a token from premiumize.me which you need to do in your brows‐
38820 er. rclone config walks you through it.
38821
38822 Here is an example of how to make a remote called remote. First run:
38823
38824 rclone config
38825
38826 This will guide you through an interactive setup process:
38827
38828 No remotes found, make a new one?
38829 n) New remote
38830 s) Set configuration password
38831 q) Quit config
38832 n/s/q> n
38833 name> remote
38834 Type of storage to configure.
38835 Enter a string value. Press Enter for the default ("").
38836 Choose a number from below, or type in your own value
38837 [snip]
38838 XX / premiumize.me
38839 \ "premiumizeme"
38840 [snip]
38841 Storage> premiumizeme
38842 ** See help for premiumizeme backend at: https://rclone.org/premiumizeme/ **
38843
38844 Remote config
38845 Use auto config?
38846 * Say Y if not sure
38847 * Say N if you are working on a remote or headless machine
38848 y) Yes
38849 n) No
38850 y/n> y
38851 If your browser doesn't open automatically go to the following link: http://127.0.0.1:53682/auth
38852 Log in and authorize rclone for access
38853 Waiting for code...
38854 Got code
38855 --------------------
38856 [remote]
38857 type = premiumizeme
38858 token = {"access_token":"XXX","token_type":"Bearer","refresh_token":"XXX","expiry":"2029-08-07T18:44:15.548915378+01:00"}
38859 --------------------
38860 y) Yes this is OK
38861 e) Edit this remote
38862 d) Delete this remote
38863 y/e/d>
38864
38865 See the remote setup docs (https://rclone.org/remote_setup/) for how to
38866 set it up on a machine with no Internet browser available.
38867
38868 Note that rclone runs a webserver on your local machine to collect the
38869 token as returned from premiumize.me. This only runs from the moment
38870 it opens your browser to the moment you get back the verification code.
38871 This is on http://127.0.0.1:53682/ and this it may require you to un‐
38872 block it temporarily if you are running a host firewall.
38873
38874 Once configured you can then use rclone like this,
38875
38876 List directories in top level of your premiumize.me
38877
38878 rclone lsd remote:
38879
38880 List all the files in your premiumize.me
38881
38882 rclone ls remote:
38883
38884 To copy a local directory to an premiumize.me directory called backup
38885
38886 rclone copy /home/source remote:backup
38887
38888 Modified time and hashes
38889 premiumize.me does not support modification times or hashes, therefore
38890 syncing will default to --size-only checking. Note that using --update
38891 will work.
38892
38893 Restricted filename characters
38894 In addition to the default restricted characters set
38895 (https://rclone.org/overview/#restricted-characters) the following
38896 characters are also replaced:
38897
38898 Character Value Replacement
38899 ────────────────────────────────
38900 \ 0x5C \
38901 " 0x22 "
38902
38903 Invalid UTF-8 bytes will also be replaced (https://rclone.org/over‐
38904 view/#invalid-utf8), as they can't be used in JSON strings.
38905
38906 Standard options
38907 Here are the Standard options specific to premiumizeme (premiumize.me).
38908
38909 --premiumizeme-api-key
38910 API Key.
38911
38912 This is not normally used - use oauth instead.
38913
38914 Properties:
38915
38916 • Config: api_key
38917
38918 • Env Var: RCLONE_PREMIUMIZEME_API_KEY
38919
38920 • Type: string
38921
38922 • Required: false
38923
38924 Advanced options
38925 Here are the Advanced options specific to premiumizeme (premiumize.me).
38926
38927 --premiumizeme-encoding
38928 The encoding for the backend.
38929
38930 See the encoding section in the overview (https://rclone.org/over‐
38931 view/#encoding) for more info.
38932
38933 Properties:
38934
38935 • Config: encoding
38936
38937 • Env Var: RCLONE_PREMIUMIZEME_ENCODING
38938
38939 • Type: MultiEncoder
38940
38941 • Default: Slash,DoubleQuote,BackSlash,Del,Ctl,InvalidUtf8,Dot
38942
38943 Limitations
38944 Note that premiumize.me is case insensitive so you can't have a file
38945 called "Hello.doc" and one called "hello.doc".
38946
38947 premiumize.me file names can't have the \ or " characters in. rclone
38948 maps these to and from an identical looking unicode equivalents \ and
38949 "
38950
38951 premiumize.me only supports filenames up to 255 characters in length.
38952
38954 Paths are specified as remote:path
38955
38956 put.io paths may be as deep as required, e.g. remote:directory/subdi‐
38957 rectory.
38958
38959 Configuration
38960 The initial setup for put.io involves getting a token from put.io which
38961 you need to do in your browser. rclone config walks you through it.
38962
38963 Here is an example of how to make a remote called remote. First run:
38964
38965 rclone config
38966
38967 This will guide you through an interactive setup process:
38968
38969 No remotes found, make a new one?
38970 n) New remote
38971 s) Set configuration password
38972 q) Quit config
38973 n/s/q> n
38974 name> putio
38975 Type of storage to configure.
38976 Enter a string value. Press Enter for the default ("").
38977 Choose a number from below, or type in your own value
38978 [snip]
38979 XX / Put.io
38980 \ "putio"
38981 [snip]
38982 Storage> putio
38983 ** See help for putio backend at: https://rclone.org/putio/ **
38984
38985 Remote config
38986 Use auto config?
38987 * Say Y if not sure
38988 * Say N if you are working on a remote or headless machine
38989 y) Yes
38990 n) No
38991 y/n> y
38992 If your browser doesn't open automatically go to the following link: http://127.0.0.1:53682/auth
38993 Log in and authorize rclone for access
38994 Waiting for code...
38995 Got code
38996 --------------------
38997 [putio]
38998 type = putio
38999 token = {"access_token":"XXXXXXXX","expiry":"0001-01-01T00:00:00Z"}
39000 --------------------
39001 y) Yes this is OK
39002 e) Edit this remote
39003 d) Delete this remote
39004 y/e/d> y
39005 Current remotes:
39006
39007 Name Type
39008 ==== ====
39009 putio putio
39010
39011 e) Edit existing remote
39012 n) New remote
39013 d) Delete remote
39014 r) Rename remote
39015 c) Copy remote
39016 s) Set configuration password
39017 q) Quit config
39018 e/n/d/r/c/s/q> q
39019
39020 Note that rclone runs a webserver on your local machine to collect the
39021 token as returned from Google if you use auto config mode. This only
39022 runs from the moment it opens your browser to the moment you get back
39023 the verification code. This is on http://127.0.0.1:53682/ and this it
39024 may require you to unblock it temporarily if you are running a host
39025 firewall, or use manual mode.
39026
39027 You can then use it like this,
39028
39029 List directories in top level of your put.io
39030
39031 rclone lsd remote:
39032
39033 List all the files in your put.io
39034
39035 rclone ls remote:
39036
39037 To copy a local directory to a put.io directory called backup
39038
39039 rclone copy /home/source remote:backup
39040
39041 Restricted filename characters
39042 In addition to the default restricted characters set
39043 (https://rclone.org/overview/#restricted-characters) the following
39044 characters are also replaced:
39045
39046 Character Value Replacement
39047 ────────────────────────────────
39048 \ 0x5C \
39049
39050 Invalid UTF-8 bytes will also be replaced (https://rclone.org/over‐
39051 view/#invalid-utf8), as they can't be used in JSON strings.
39052
39053 Advanced options
39054 Here are the Advanced options specific to putio (Put.io).
39055
39056 --putio-encoding
39057 The encoding for the backend.
39058
39059 See the encoding section in the overview (https://rclone.org/over‐
39060 view/#encoding) for more info.
39061
39062 Properties:
39063
39064 • Config: encoding
39065
39066 • Env Var: RCLONE_PUTIO_ENCODING
39067
39068 • Type: MultiEncoder
39069
39070 • Default: Slash,BackSlash,Del,Ctl,InvalidUtf8,Dot
39071
39072 Limitations
39073 put.io has rate limiting. When you hit a limit, rclone automatically
39074 retries after waiting the amount of time requested by the server.
39075
39076 If you want to avoid ever hitting these limits, you may use the --tp‐
39077 slimit flag with a low number. Note that the imposed limits may be
39078 different for different operations, and may change over time.
39079
39081 This is a backend for the Seafile (https://www.seafile.com/) storage
39082 service: - It works with both the free community edition or the profes‐
39083 sional edition. - Seafile versions 6.x and 7.x are all supported. -
39084 Encrypted libraries are also supported. - It supports 2FA enabled
39085 users
39086
39087 Configuration
39088 There are two distinct modes you can setup your remote: - you point
39089 your remote to the root of the server, meaning you don't specify a li‐
39090 brary during the configuration: Paths are specified as remote:library.
39091 You may put subdirectories in too, e.g. remote:library/path/to/dir. -
39092 you point your remote to a specific library during the configuration:
39093 Paths are specified as remote:path/to/dir. This is the recommended
39094 mode when using encrypted libraries. (This mode is possibly slightly
39095 faster than the root mode)
39096
39097 Configuration in root mode
39098 Here is an example of making a seafile configuration for a user with no
39099 two-factor authentication. First run
39100
39101 rclone config
39102
39103 This will guide you through an interactive setup process. To authenti‐
39104 cate you will need the URL of your server, your email (or username) and
39105 your password.
39106
39107 No remotes found, make a new one?
39108 n) New remote
39109 s) Set configuration password
39110 q) Quit config
39111 n/s/q> n
39112 name> seafile
39113 Type of storage to configure.
39114 Enter a string value. Press Enter for the default ("").
39115 Choose a number from below, or type in your own value
39116 [snip]
39117 XX / Seafile
39118 \ "seafile"
39119 [snip]
39120 Storage> seafile
39121 ** See help for seafile backend at: https://rclone.org/seafile/ **
39122
39123 URL of seafile host to connect to
39124 Enter a string value. Press Enter for the default ("").
39125 Choose a number from below, or type in your own value
39126 1 / Connect to cloud.seafile.com
39127 \ "https://cloud.seafile.com/"
39128 url> http://my.seafile.server/
39129 User name (usually email address)
39130 Enter a string value. Press Enter for the default ("").
39131 user> me@example.com
39132 Password
39133 y) Yes type in my own password
39134 g) Generate random password
39135 n) No leave this optional password blank (default)
39136 y/g> y
39137 Enter the password:
39138 password:
39139 Confirm the password:
39140 password:
39141 Two-factor authentication ('true' if the account has 2FA enabled)
39142 Enter a boolean value (true or false). Press Enter for the default ("false").
39143 2fa> false
39144 Name of the library. Leave blank to access all non-encrypted libraries.
39145 Enter a string value. Press Enter for the default ("").
39146 library>
39147 Library password (for encrypted libraries only). Leave blank if you pass it through the command line.
39148 y) Yes type in my own password
39149 g) Generate random password
39150 n) No leave this optional password blank (default)
39151 y/g/n> n
39152 Edit advanced config? (y/n)
39153 y) Yes
39154 n) No (default)
39155 y/n> n
39156 Remote config
39157 Two-factor authentication is not enabled on this account.
39158 --------------------
39159 [seafile]
39160 type = seafile
39161 url = http://my.seafile.server/
39162 user = me@example.com
39163 pass = *** ENCRYPTED ***
39164 2fa = false
39165 --------------------
39166 y) Yes this is OK (default)
39167 e) Edit this remote
39168 d) Delete this remote
39169 y/e/d> y
39170
39171 This remote is called seafile. It's pointing to the root of your
39172 seafile server and can now be used like this:
39173
39174 See all libraries
39175
39176 rclone lsd seafile:
39177
39178 Create a new library
39179
39180 rclone mkdir seafile:library
39181
39182 List the contents of a library
39183
39184 rclone ls seafile:library
39185
39186 Sync /home/local/directory to the remote library, deleting any excess
39187 files in the library.
39188
39189 rclone sync -i /home/local/directory seafile:library
39190
39191 Configuration in library mode
39192 Here's an example of a configuration in library mode with a user that
39193 has the two-factor authentication enabled. Your 2FA code will be asked
39194 at the end of the configuration, and will attempt to authenticate you:
39195
39196 No remotes found, make a new one?
39197 n) New remote
39198 s) Set configuration password
39199 q) Quit config
39200 n/s/q> n
39201 name> seafile
39202 Type of storage to configure.
39203 Enter a string value. Press Enter for the default ("").
39204 Choose a number from below, or type in your own value
39205 [snip]
39206 XX / Seafile
39207 \ "seafile"
39208 [snip]
39209 Storage> seafile
39210 ** See help for seafile backend at: https://rclone.org/seafile/ **
39211
39212 URL of seafile host to connect to
39213 Enter a string value. Press Enter for the default ("").
39214 Choose a number from below, or type in your own value
39215 1 / Connect to cloud.seafile.com
39216 \ "https://cloud.seafile.com/"
39217 url> http://my.seafile.server/
39218 User name (usually email address)
39219 Enter a string value. Press Enter for the default ("").
39220 user> me@example.com
39221 Password
39222 y) Yes type in my own password
39223 g) Generate random password
39224 n) No leave this optional password blank (default)
39225 y/g> y
39226 Enter the password:
39227 password:
39228 Confirm the password:
39229 password:
39230 Two-factor authentication ('true' if the account has 2FA enabled)
39231 Enter a boolean value (true or false). Press Enter for the default ("false").
39232 2fa> true
39233 Name of the library. Leave blank to access all non-encrypted libraries.
39234 Enter a string value. Press Enter for the default ("").
39235 library> My Library
39236 Library password (for encrypted libraries only). Leave blank if you pass it through the command line.
39237 y) Yes type in my own password
39238 g) Generate random password
39239 n) No leave this optional password blank (default)
39240 y/g/n> n
39241 Edit advanced config? (y/n)
39242 y) Yes
39243 n) No (default)
39244 y/n> n
39245 Remote config
39246 Two-factor authentication: please enter your 2FA code
39247 2fa code> 123456
39248 Authenticating...
39249 Success!
39250 --------------------
39251 [seafile]
39252 type = seafile
39253 url = http://my.seafile.server/
39254 user = me@example.com
39255 pass =
39256 2fa = true
39257 library = My Library
39258 --------------------
39259 y) Yes this is OK (default)
39260 e) Edit this remote
39261 d) Delete this remote
39262 y/e/d> y
39263
39264 You'll notice your password is blank in the configuration. It's be‐
39265 cause we only need the password to authenticate you once.
39266
39267 You specified My Library during the configuration. The root of the re‐
39268 mote is pointing at the root of the library My Library:
39269
39270 See all files in the library:
39271
39272 rclone lsd seafile:
39273
39274 Create a new directory inside the library
39275
39276 rclone mkdir seafile:directory
39277
39278 List the contents of a directory
39279
39280 rclone ls seafile:directory
39281
39282 Sync /home/local/directory to the remote library, deleting any excess
39283 files in the library.
39284
39285 rclone sync -i /home/local/directory seafile:
39286
39287 --fast-list
39288 Seafile version 7+ supports --fast-list which allows you to use fewer
39289 transactions in exchange for more memory. See the rclone docs
39290 (https://rclone.org/docs/#fast-list) for more details. Please note
39291 this is not supported on seafile server version 6.x
39292
39293 Restricted filename characters
39294 In addition to the default restricted characters set
39295 (https://rclone.org/overview/#restricted-characters) the following
39296 characters are also replaced:
39297
39298 Character Value Replacement
39299 ────────────────────────────────
39300 / 0x2F /
39301 " 0x22 "
39302 \ 0x5C \
39303
39304 Invalid UTF-8 bytes will also be replaced (https://rclone.org/over‐
39305 view/#invalid-utf8), as they can't be used in JSON strings.
39306
39307 Seafile and rclone link
39308 Rclone supports generating share links for non-encrypted libraries on‐
39309 ly. They can either be for a file or a directory:
39310
39311 rclone link seafile:seafile-tutorial.doc
39312 http://my.seafile.server/f/fdcd8a2f93f84b8b90f4/
39313
39314 or if run on a directory you will get:
39315
39316 rclone link seafile:dir
39317 http://my.seafile.server/d/9ea2455f6f55478bbb0d/
39318
39319 Please note a share link is unique for each file or directory. If you
39320 run a link command on a file/dir that has already been shared, you will
39321 get the exact same link.
39322
39323 Compatibility
39324 It has been actively tested using the seafile docker image
39325 (https://github.com/haiwen/seafile-docker) of these versions: - 6.3.4
39326 community edition - 7.0.5 community edition - 7.1.3 community edition
39327
39328 Versions below 6.0 are not supported. Versions between 6.0 and 6.3
39329 haven't been tested and might not work properly.
39330
39331 Standard options
39332 Here are the Standard options specific to seafile (seafile).
39333
39334 --seafile-url
39335 URL of seafile host to connect to.
39336
39337 Properties:
39338
39339 • Config: url
39340
39341 • Env Var: RCLONE_SEAFILE_URL
39342
39343 • Type: string
39344
39345 • Required: true
39346
39347 • Examples:
39348
39349 • "https://cloud.seafile.com/"
39350
39351 • Connect to cloud.seafile.com.
39352
39353 --seafile-user
39354 User name (usually email address).
39355
39356 Properties:
39357
39358 • Config: user
39359
39360 • Env Var: RCLONE_SEAFILE_USER
39361
39362 • Type: string
39363
39364 • Required: true
39365
39366 --seafile-pass
39367 Password.
39368
39369 NB Input to this must be obscured - see rclone obscure
39370 (https://rclone.org/commands/rclone_obscure/).
39371
39372 Properties:
39373
39374 • Config: pass
39375
39376 • Env Var: RCLONE_SEAFILE_PASS
39377
39378 • Type: string
39379
39380 • Required: false
39381
39382 --seafile-2fa
39383 Two-factor authentication ('true' if the account has 2FA enabled).
39384
39385 Properties:
39386
39387 • Config: 2fa
39388
39389 • Env Var: RCLONE_SEAFILE_2FA
39390
39391 • Type: bool
39392
39393 • Default: false
39394
39395 --seafile-library
39396 Name of the library.
39397
39398 Leave blank to access all non-encrypted libraries.
39399
39400 Properties:
39401
39402 • Config: library
39403
39404 • Env Var: RCLONE_SEAFILE_LIBRARY
39405
39406 • Type: string
39407
39408 • Required: false
39409
39410 --seafile-library-key
39411 Library password (for encrypted libraries only).
39412
39413 Leave blank if you pass it through the command line.
39414
39415 NB Input to this must be obscured - see rclone obscure
39416 (https://rclone.org/commands/rclone_obscure/).
39417
39418 Properties:
39419
39420 • Config: library_key
39421
39422 • Env Var: RCLONE_SEAFILE_LIBRARY_KEY
39423
39424 • Type: string
39425
39426 • Required: false
39427
39428 --seafile-auth-token
39429 Authentication token.
39430
39431 Properties:
39432
39433 • Config: auth_token
39434
39435 • Env Var: RCLONE_SEAFILE_AUTH_TOKEN
39436
39437 • Type: string
39438
39439 • Required: false
39440
39441 Advanced options
39442 Here are the Advanced options specific to seafile (seafile).
39443
39444 --seafile-create-library
39445 Should rclone create a library if it doesn't exist.
39446
39447 Properties:
39448
39449 • Config: create_library
39450
39451 • Env Var: RCLONE_SEAFILE_CREATE_LIBRARY
39452
39453 • Type: bool
39454
39455 • Default: false
39456
39457 --seafile-encoding
39458 The encoding for the backend.
39459
39460 See the encoding section in the overview (https://rclone.org/over‐
39461 view/#encoding) for more info.
39462
39463 Properties:
39464
39465 • Config: encoding
39466
39467 • Env Var: RCLONE_SEAFILE_ENCODING
39468
39469 • Type: MultiEncoder
39470
39471 • Default: Slash,DoubleQuote,BackSlash,Ctl,InvalidUtf8
39472
39474 SFTP is the Secure (or SSH) File Transfer Protocol
39475 (https://en.wikipedia.org/wiki/SSH_File_Transfer_Protocol).
39476
39477 The SFTP backend can be used with a number of different providers:
39478
39479 • Hetzner Storage Box
39480
39481 • rsync.net
39482
39483 SFTP runs over SSH v2 and is installed as standard with most modern SSH
39484 installations.
39485
39486 Paths are specified as remote:path. If the path does not begin with a
39487 / it is relative to the home directory of the user. An empty path re‐
39488 mote: refers to the user's home directory. For example, rclone lsd re‐
39489 mote: would list the home directory of the user configured in the
39490 rclone remote config (i.e /home/sftpuser). However, rclone lsd re‐
39491 mote:/ would list the root directory for remote machine (i.e. /)
39492
39493 Note that some SFTP servers will need the leading / - Synology is a
39494 good example of this. rsync.net and Hetzner, on the other hand, re‐
39495 quires users to OMIT the leading /.
39496
39497 Note that by default rclone will try to execute shell commands on the
39498 server, see shell access considerations.
39499
39500 Configuration
39501 Here is an example of making an SFTP configuration. First run
39502
39503 rclone config
39504
39505 This will guide you through an interactive setup process.
39506
39507 No remotes found, make a new one?
39508 n) New remote
39509 s) Set configuration password
39510 q) Quit config
39511 n/s/q> n
39512 name> remote
39513 Type of storage to configure.
39514 Choose a number from below, or type in your own value
39515 [snip]
39516 XX / SSH/SFTP
39517 \ "sftp"
39518 [snip]
39519 Storage> sftp
39520 SSH host to connect to
39521 Choose a number from below, or type in your own value
39522 1 / Connect to example.com
39523 \ "example.com"
39524 host> example.com
39525 SSH username
39526 Enter a string value. Press Enter for the default ("$USER").
39527 user> sftpuser
39528 SSH port number
39529 Enter a signed integer. Press Enter for the default (22).
39530 port>
39531 SSH password, leave blank to use ssh-agent.
39532 y) Yes type in my own password
39533 g) Generate random password
39534 n) No leave this optional password blank
39535 y/g/n> n
39536 Path to unencrypted PEM-encoded private key file, leave blank to use ssh-agent.
39537 key_file>
39538 Remote config
39539 --------------------
39540 [remote]
39541 host = example.com
39542 user = sftpuser
39543 port =
39544 pass =
39545 key_file =
39546 --------------------
39547 y) Yes this is OK
39548 e) Edit this remote
39549 d) Delete this remote
39550 y/e/d> y
39551
39552 This remote is called remote and can now be used like this:
39553
39554 See all directories in the home directory
39555
39556 rclone lsd remote:
39557
39558 See all directories in the root directory
39559
39560 rclone lsd remote:/
39561
39562 Make a new directory
39563
39564 rclone mkdir remote:path/to/directory
39565
39566 List the contents of a directory
39567
39568 rclone ls remote:path/to/directory
39569
39570 Sync /home/local/directory to the remote directory, deleting any excess
39571 files in the directory.
39572
39573 rclone sync -i /home/local/directory remote:directory
39574
39575 Mount the remote path /srv/www-data/ to the local path /mnt/www-data
39576
39577 rclone mount remote:/srv/www-data/ /mnt/www-data
39578
39579 SSH Authentication
39580 The SFTP remote supports three authentication methods:
39581
39582 • Password
39583
39584 • Key file, including certificate signed keys
39585
39586 • ssh-agent
39587
39588 Key files should be PEM-encoded private key files. For instance
39589 /home/$USER/.ssh/id_rsa. Only unencrypted OpenSSH or PEM encrypted
39590 files are supported.
39591
39592 The key file can be specified in either an external file (key_file) or
39593 contained within the rclone config file (key_pem). If using key_pem in
39594 the config file, the entry should be on a single line with new line (''
39595 or '') separating lines. i.e.
39596
39597 key_pem = -----BEGIN RSA PRIVATE KEY-----\nMaMbaIXtE\n0gAMbMbaSsd\nMbaass\n-----END RSA PRIVATE KEY-----
39598
39599 This will generate it correctly for key_pem for use in the config:
39600
39601 awk '{printf "%s\\n", $0}' < ~/.ssh/id_rsa
39602
39603 If you don't specify pass, key_file, or key_pem or ask_password then
39604 rclone will attempt to contact an ssh-agent. You can also specify
39605 key_use_agent to force the usage of an ssh-agent. In this case
39606 key_file or key_pem can also be specified to force the usage of a spe‐
39607 cific key in the ssh-agent.
39608
39609 Using an ssh-agent is the only way to load encrypted OpenSSH keys at
39610 the moment.
39611
39612 If you set the ask_password option, rclone will prompt for a password
39613 when needed and no password has been configured.
39614
39615 Certificate-signed keys
39616 With traditional key-based authentication, you configure your private
39617 key only, and the public key built into it will be used during the au‐
39618 thentication process.
39619
39620 If you have a certificate you may use it to sign your public key, cre‐
39621 ating a separate SSH user certificate that should be used instead of
39622 the plain public key extracted from the private key. Then you must
39623 provide the path to the user certificate public key file in pub‐
39624 key_file.
39625
39626 Note: This is not the traditional public key paired with your private
39627 key, typically saved as /home/$USER/.ssh/id_rsa.pub. Setting this path
39628 in pubkey_file will not work.
39629
39630 Example:
39631
39632 [remote]
39633 type = sftp
39634 host = example.com
39635 user = sftpuser
39636 key_file = ~/id_rsa
39637 pubkey_file = ~/id_rsa-cert.pub
39638
39639 If you concatenate a cert with a private key then you can specify the
39640 merged file in both places.
39641
39642 Note: the cert must come first in the file. e.g.
39643
39644 cat id_rsa-cert.pub id_rsa > merged_key
39645
39646 Host key validation
39647 By default rclone will not check the server's host key for validation.
39648 This can allow an attacker to replace a server with their own and if
39649 you use password authentication then this can lead to that password be‐
39650 ing exposed.
39651
39652 Host key matching, using standard known_hosts files can be turned on by
39653 enabling the known_hosts_file option. This can point to the file main‐
39654 tained by OpenSSH or can point to a unique file.
39655
39656 e.g. using the OpenSSH known_hosts file:
39657
39658 [remote]
39659 type = sftp
39660 host = example.com
39661 user = sftpuser
39662 pass =
39663 known_hosts_file = ~/.ssh/known_hosts
39664
39665 Alternatively you can create your own known hosts file like this:
39666
39667 ssh-keyscan -t dsa,rsa,ecdsa,ed25519 example.com >> known_hosts
39668
39669 There are some limitations:
39670
39671 • rclone will not manage this file for you. If the key is missing or
39672 wrong then the connection will be refused.
39673
39674 • If the server is set up for a certificate host key then the entry in
39675 the known_hosts file must be the @cert-authority entry for the CA
39676
39677 If the host key provided by the server does not match the one in the
39678 file (or is missing) then the connection will be aborted and an error
39679 returned such as
39680
39681 NewFs: couldn't connect SSH: ssh: handshake failed: knownhosts: key mismatch
39682
39683 or
39684
39685 NewFs: couldn't connect SSH: ssh: handshake failed: knownhosts: key is unknown
39686
39687 If you see an error such as
39688
39689 NewFs: couldn't connect SSH: ssh: handshake failed: ssh: no authorities for hostname: example.com:22
39690
39691 then it is likely the server has presented a CA signed host certificate
39692 and you will need to add the appropriate @cert-authority entry.
39693
39694 The known_hosts_file setting can be set during rclone config as an ad‐
39695 vanced option.
39696
39697 ssh-agent on macOS
39698 Note that there seem to be various problems with using an ssh-agent on
39699 macOS due to recent changes in the OS. The most effective work-around
39700 seems to be to start an ssh-agent in each session, e.g.
39701
39702 eval `ssh-agent -s` && ssh-add -A
39703
39704 And then at the end of the session
39705
39706 eval `ssh-agent -k`
39707
39708 These commands can be used in scripts of course.
39709
39710 Shell access
39711 Some functionality of the SFTP backend relies on remote shell access,
39712 and the possibility to execute commands. This includes checksum, and
39713 in some cases also about. The shell commands that must be executed may
39714 be different on different type of shells, and also quoting/escaping of
39715 file path arguments containing special characters may be different.
39716 Rclone therefore needs to know what type of shell it is, and if shell
39717 access is available at all.
39718
39719 Most servers run on some version of Unix, and then a basic Unix shell
39720 can be assumed, without further distinction. Windows 10, Server 2019,
39721 and later can also run a SSH server, which is a port of OpenSSH (see
39722 official installation guide (https://docs.microsoft.com/en-us/windows-
39723 server/administration/openssh/openssh_install_firstuse)). On a Windows
39724 server the shell handling is different: Although it can also be set up
39725 to use a Unix type shell, e.g. Cygwin bash, the default is to use Win‐
39726 dows Command Prompt (cmd.exe), and PowerShell is a recommended alterna‐
39727 tive. All of these have behave differently, which rclone must handle.
39728
39729 Rclone tries to auto-detect what type of shell is used on the server,
39730 first time you access the SFTP remote. If a remote shell session is
39731 successfully created, it will look for indications that it is CMD or
39732 PowerShell, with fall-back to Unix if not something else is detected.
39733 If unable to even create a remote shell session, then shell command ex‐
39734 ecution will be disabled entirely. The result is stored in the SFTP
39735 remote configuration, in option shell_type, so that the auto-detection
39736 only have to be performed once. If you manually set a value for this
39737 option before first run, the auto-detection will be skipped, and if you
39738 set a different value later this will override any existing. Value
39739 none can be set to avoid any attempts at executing shell commands, e.g.
39740 if this is not allowed on the server.
39741
39742 When the server is rclone serve sftp (https://rclone.org/com‐
39743 mands/rclone_serve_sftp/), the rclone SFTP remote will detect this as a
39744 Unix type shell - even if it is running on Windows. This server does
39745 not actually have a shell, but it accepts input commands matching the
39746 specific ones that the SFTP backend relies on for Unix shells, e.g.
39747 md5sum and df. Also it handles the string escape rules used for Unix
39748 shell. Treating it as a Unix type shell from a SFTP remote will there‐
39749 fore always be correct, and support all features.
39750
39751 Shell access considerations
39752 The shell type auto-detection logic, described above, means that by de‐
39753 fault rclone will try to run a shell command the first time a new sftp
39754 remote is accessed. If you configure a sftp remote without a config
39755 file, e.g. an on the fly (https://rclone.org/docs/#backend-path-to-
39756 dir%5D) remote, rclone will have nowhere to store the result, and it
39757 will re-run the command on every access. To avoid this you should ex‐
39758 plicitly set the shell_type option to the correct value, or to none if
39759 you want to prevent rclone from executing any remote shell commands.
39760
39761 It is also important to note that, since the shell type decides how
39762 quoting and escaping of file paths used as command-line arguments are
39763 performed, configuring the wrong shell type may leave you exposed to
39764 command injection exploits. Make sure to confirm the auto-detected
39765 shell type, or explicitly set the shell type you know is correct, or
39766 disable shell access until you know.
39767
39768 Checksum
39769 SFTP does not natively support checksums (file hash), but rclone is
39770 able to use checksumming if the same login has shell access, and can
39771 execute remote commands. If there is a command that can calculate com‐
39772 patible checksums on the remote system, Rclone can then be configured
39773 to execute this whenever a checksum is needed, and read back the re‐
39774 sults. Currently MD5 and SHA-1 are supported.
39775
39776 Normally this requires an external utility being available on the serv‐
39777 er. By default rclone will try commands md5sum, md5 and rclone md5sum
39778 for MD5 checksums, and the first one found usable will be picked. Same
39779 with sha1sum, sha1 and rclone sha1sum commands for SHA-1 checksums.
39780 These utilities normally need to be in the remote's PATH to be found.
39781
39782 In some cases the shell itself is capable of calculating checksums.
39783 PowerShell is an example of such a shell. If rclone detects that the
39784 remote shell is PowerShell, which means it most probably is a Windows
39785 OpenSSH server, rclone will use a predefined script block to produce
39786 the checksums when no external checksum commands are found (see shell
39787 access). This assumes PowerShell version 4.0 or newer.
39788
39789 The options md5sum_command and sha1_command can be used to customize
39790 the command to be executed for calculation of checksums. You can for
39791 example set a specific path to where md5sum and sha1sum executables are
39792 located, or use them to specify some other tools that print checksums
39793 in compatible format. The value can include command-line arguments, or
39794 even shell script blocks as with PowerShell. Rclone has subcommands
39795 md5sum (https://rclone.org/commands/rclone_md5sum/) and sha1sum
39796 (https://rclone.org/commands/rclone_sha1sum/) that use compatible for‐
39797 mat, which means if you have an rclone executable on the server it can
39798 be used. As mentioned above, they will be automatically picked up if
39799 found in PATH, but if not you can set something like /path/to/rclone
39800 md5sum as the value of option md5sum_command to make sure a specific
39801 executable is used.
39802
39803 Remote checksumming is recommended and enabled by default. First time
39804 rclone is using a SFTP remote, if options md5sum_command or sha1_com‐
39805 mand are not set, it will check if any of the default commands for each
39806 of them, as described above, can be used. The result will be saved in
39807 the remote configuration, so next time it will use the same. Value
39808 none will be set if none of the default commands could be used for a
39809 specific algorithm, and this algorithm will not be supported by the re‐
39810 mote.
39811
39812 Disabling the checksumming may be required if you are connecting to
39813 SFTP servers which are not under your control, and to which the execu‐
39814 tion of remote shell commands is prohibited. Set the configuration op‐
39815 tion disable_hashcheck to true to disable checksumming entirely, or set
39816 shell_type to none to disable all functionality based on remote shell
39817 command execution.
39818
39819 Modified time
39820 Modified times are stored on the server to 1 second precision.
39821
39822 Modified times are used in syncing and are fully supported.
39823
39824 Some SFTP servers disable setting/modifying the file modification time
39825 after upload (for example, certain configurations of ProFTPd with
39826 mod_sftp). If you are using one of these servers, you can set the op‐
39827 tion set_modtime = false in your RClone backend configuration to dis‐
39828 able this behaviour.
39829
39830 About command
39831 The about command returns the total space, free space, and used space
39832 on the remote for the disk of the specified path on the remote or, if
39833 not set, the disk of the root on the remote.
39834
39835 SFTP usually supports the about (https://rclone.org/com‐
39836 mands/rclone_about/) command, but it depends on the server. If the
39837 server implements the vendor-specific VFS statistics extension, which
39838 is normally the case with OpenSSH instances, it will be used. If not,
39839 but the same login has access to a Unix shell, where the df command is
39840 available (e.g. in the remote's PATH), then this will be used instead.
39841 If the server shell is PowerShell, probably with a Windows OpenSSH
39842 server, rclone will use a built-in shell command (see shell access).
39843 If none of the above is applicable, about will fail.
39844
39845 Standard options
39846 Here are the Standard options specific to sftp (SSH/SFTP).
39847
39848 --sftp-host
39849 SSH host to connect to.
39850
39851 E.g. "example.com".
39852
39853 Properties:
39854
39855 • Config: host
39856
39857 • Env Var: RCLONE_SFTP_HOST
39858
39859 • Type: string
39860
39861 • Required: true
39862
39863 --sftp-user
39864 SSH username.
39865
39866 Properties:
39867
39868 • Config: user
39869
39870 • Env Var: RCLONE_SFTP_USER
39871
39872 • Type: string
39873
39874 • Default: "$USER"
39875
39876 --sftp-port
39877 SSH port number.
39878
39879 Properties:
39880
39881 • Config: port
39882
39883 • Env Var: RCLONE_SFTP_PORT
39884
39885 • Type: int
39886
39887 • Default: 22
39888
39889 --sftp-pass
39890 SSH password, leave blank to use ssh-agent.
39891
39892 NB Input to this must be obscured - see rclone obscure
39893 (https://rclone.org/commands/rclone_obscure/).
39894
39895 Properties:
39896
39897 • Config: pass
39898
39899 • Env Var: RCLONE_SFTP_PASS
39900
39901 • Type: string
39902
39903 • Required: false
39904
39905 --sftp-key-pem
39906 Raw PEM-encoded private key.
39907
39908 If specified, will override key_file parameter.
39909
39910 Properties:
39911
39912 • Config: key_pem
39913
39914 • Env Var: RCLONE_SFTP_KEY_PEM
39915
39916 • Type: string
39917
39918 • Required: false
39919
39920 --sftp-key-file
39921 Path to PEM-encoded private key file.
39922
39923 Leave blank or set key-use-agent to use ssh-agent.
39924
39925 Leading ~ will be expanded in the file name as will environment vari‐
39926 ables such as ${RCLONE_CONFIG_DIR}.
39927
39928 Properties:
39929
39930 • Config: key_file
39931
39932 • Env Var: RCLONE_SFTP_KEY_FILE
39933
39934 • Type: string
39935
39936 • Required: false
39937
39938 --sftp-key-file-pass
39939 The passphrase to decrypt the PEM-encoded private key file.
39940
39941 Only PEM encrypted key files (old OpenSSH format) are supported. En‐
39942 crypted keys in the new OpenSSH format can't be used.
39943
39944 NB Input to this must be obscured - see rclone obscure
39945 (https://rclone.org/commands/rclone_obscure/).
39946
39947 Properties:
39948
39949 • Config: key_file_pass
39950
39951 • Env Var: RCLONE_SFTP_KEY_FILE_PASS
39952
39953 • Type: string
39954
39955 • Required: false
39956
39957 --sftp-pubkey-file
39958 Optional path to public key file.
39959
39960 Set this if you have a signed certificate you want to use for authenti‐
39961 cation.
39962
39963 Leading ~ will be expanded in the file name as will environment vari‐
39964 ables such as ${RCLONE_CONFIG_DIR}.
39965
39966 Properties:
39967
39968 • Config: pubkey_file
39969
39970 • Env Var: RCLONE_SFTP_PUBKEY_FILE
39971
39972 • Type: string
39973
39974 • Required: false
39975
39976 --sftp-key-use-agent
39977 When set forces the usage of the ssh-agent.
39978
39979 When key-file is also set, the ".pub" file of the specified key-file is
39980 read and only the associated key is requested from the ssh-agent. This
39981 allows to avoid Too many authentication failures for *username* errors
39982 when the ssh-agent contains many keys.
39983
39984 Properties:
39985
39986 • Config: key_use_agent
39987
39988 • Env Var: RCLONE_SFTP_KEY_USE_AGENT
39989
39990 • Type: bool
39991
39992 • Default: false
39993
39994 --sftp-use-insecure-cipher
39995 Enable the use of insecure ciphers and key exchange methods.
39996
39997 This enables the use of the following insecure ciphers and key exchange
39998 methods:
39999
40000 • aes128-cbc
40001
40002 • aes192-cbc
40003
40004 • aes256-cbc
40005
40006 • 3des-cbc
40007
40008 • diffie-hellman-group-exchange-sha256
40009
40010 • diffie-hellman-group-exchange-sha1
40011
40012 Those algorithms are insecure and may allow plaintext data to be recov‐
40013 ered by an attacker.
40014
40015 Properties:
40016
40017 • Config: use_insecure_cipher
40018
40019 • Env Var: RCLONE_SFTP_USE_INSECURE_CIPHER
40020
40021 • Type: bool
40022
40023 • Default: false
40024
40025 • Examples:
40026
40027 • "false"
40028
40029 • Use default Cipher list.
40030
40031 • "true"
40032
40033 • Enables the use of the aes128-cbc cipher and diffie-hellman-
40034 group-exchange-sha256, diffie-hellman-group-exchange-sha1 key ex‐
40035 change.
40036
40037 --sftp-disable-hashcheck
40038 Disable the execution of SSH commands to determine if remote file hash‐
40039 ing is available.
40040
40041 Leave blank or set to false to enable hashing (recommended), set to
40042 true to disable hashing.
40043
40044 Properties:
40045
40046 • Config: disable_hashcheck
40047
40048 • Env Var: RCLONE_SFTP_DISABLE_HASHCHECK
40049
40050 • Type: bool
40051
40052 • Default: false
40053
40054 Advanced options
40055 Here are the Advanced options specific to sftp (SSH/SFTP).
40056
40057 --sftp-known-hosts-file
40058 Optional path to known_hosts file.
40059
40060 Set this value to enable server host key validation.
40061
40062 Leading ~ will be expanded in the file name as will environment vari‐
40063 ables such as ${RCLONE_CONFIG_DIR}.
40064
40065 Properties:
40066
40067 • Config: known_hosts_file
40068
40069 • Env Var: RCLONE_SFTP_KNOWN_HOSTS_FILE
40070
40071 • Type: string
40072
40073 • Required: false
40074
40075 • Examples:
40076
40077 • "~/.ssh/known_hosts"
40078
40079 • Use OpenSSH's known_hosts file.
40080
40081 --sftp-ask-password
40082 Allow asking for SFTP password when needed.
40083
40084 If this is set and no password is supplied then rclone will: - ask for
40085 a password - not contact the ssh agent
40086
40087 Properties:
40088
40089 • Config: ask_password
40090
40091 • Env Var: RCLONE_SFTP_ASK_PASSWORD
40092
40093 • Type: bool
40094
40095 • Default: false
40096
40097 --sftp-path-override
40098 Override path used by SSH shell commands.
40099
40100 This allows checksum calculation when SFTP and SSH paths are different.
40101 This issue affects among others Synology NAS boxes.
40102
40103 E.g. if shared folders can be found in directories representing vol‐
40104 umes:
40105
40106 rclone sync /home/local/directory remote:/directory --sftp-path-override /volume2/directory
40107
40108 E.g. if home directory can be found in a shared folder called "home":
40109
40110 rclone sync /home/local/directory remote:/home/directory --sftp-path-override /volume1/homes/USER/directory
40111
40112 Properties:
40113
40114 • Config: path_override
40115
40116 • Env Var: RCLONE_SFTP_PATH_OVERRIDE
40117
40118 • Type: string
40119
40120 • Required: false
40121
40122 --sftp-set-modtime
40123 Set the modified time on the remote if set.
40124
40125 Properties:
40126
40127 • Config: set_modtime
40128
40129 • Env Var: RCLONE_SFTP_SET_MODTIME
40130
40131 • Type: bool
40132
40133 • Default: true
40134
40135 --sftp-shell-type
40136 The type of SSH shell on remote server, if any.
40137
40138 Leave blank for autodetect.
40139
40140 Properties:
40141
40142 • Config: shell_type
40143
40144 • Env Var: RCLONE_SFTP_SHELL_TYPE
40145
40146 • Type: string
40147
40148 • Required: false
40149
40150 • Examples:
40151
40152 • "none"
40153
40154 • No shell access
40155
40156 • "unix"
40157
40158 • Unix shell
40159
40160 • "powershell"
40161
40162 • PowerShell
40163
40164 • "cmd"
40165
40166 • Windows Command Prompt
40167
40168 --sftp-md5sum-command
40169 The command used to read md5 hashes.
40170
40171 Leave blank for autodetect.
40172
40173 Properties:
40174
40175 • Config: md5sum_command
40176
40177 • Env Var: RCLONE_SFTP_MD5SUM_COMMAND
40178
40179 • Type: string
40180
40181 • Required: false
40182
40183 --sftp-sha1sum-command
40184 The command used to read sha1 hashes.
40185
40186 Leave blank for autodetect.
40187
40188 Properties:
40189
40190 • Config: sha1sum_command
40191
40192 • Env Var: RCLONE_SFTP_SHA1SUM_COMMAND
40193
40194 • Type: string
40195
40196 • Required: false
40197
40198 --sftp-skip-links
40199 Set to skip any symlinks and any other non regular files.
40200
40201 Properties:
40202
40203 • Config: skip_links
40204
40205 • Env Var: RCLONE_SFTP_SKIP_LINKS
40206
40207 • Type: bool
40208
40209 • Default: false
40210
40211 --sftp-subsystem
40212 Specifies the SSH2 subsystem on the remote host.
40213
40214 Properties:
40215
40216 • Config: subsystem
40217
40218 • Env Var: RCLONE_SFTP_SUBSYSTEM
40219
40220 • Type: string
40221
40222 • Default: "sftp"
40223
40224 --sftp-server-command
40225 Specifies the path or command to run a sftp server on the remote host.
40226
40227 The subsystem option is ignored when server_command is defined.
40228
40229 Properties:
40230
40231 • Config: server_command
40232
40233 • Env Var: RCLONE_SFTP_SERVER_COMMAND
40234
40235 • Type: string
40236
40237 • Required: false
40238
40239 --sftp-use-fstat
40240 If set use fstat instead of stat.
40241
40242 Some servers limit the amount of open files and calling Stat after
40243 opening the file will throw an error from the server. Setting this
40244 flag will call Fstat instead of Stat which is called on an already open
40245 file handle.
40246
40247 It has been found that this helps with IBM Sterling SFTP servers which
40248 have "extractability" level set to 1 which means only 1 file can be
40249 opened at any given time.
40250
40251 Properties:
40252
40253 • Config: use_fstat
40254
40255 • Env Var: RCLONE_SFTP_USE_FSTAT
40256
40257 • Type: bool
40258
40259 • Default: false
40260
40261 --sftp-disable-concurrent-reads
40262 If set don't use concurrent reads.
40263
40264 Normally concurrent reads are safe to use and not using them will de‐
40265 grade performance, so this option is disabled by default.
40266
40267 Some servers limit the amount number of times a file can be downloaded.
40268 Using concurrent reads can trigger this limit, so if you have a server
40269 which returns
40270
40271 Failed to copy: file does not exist
40272
40273 Then you may need to enable this flag.
40274
40275 If concurrent reads are disabled, the use_fstat option is ignored.
40276
40277 Properties:
40278
40279 • Config: disable_concurrent_reads
40280
40281 • Env Var: RCLONE_SFTP_DISABLE_CONCURRENT_READS
40282
40283 • Type: bool
40284
40285 • Default: false
40286
40287 --sftp-disable-concurrent-writes
40288 If set don't use concurrent writes.
40289
40290 Normally rclone uses concurrent writes to upload files. This improves
40291 the performance greatly, especially for distant servers.
40292
40293 This option disables concurrent writes should that be necessary.
40294
40295 Properties:
40296
40297 • Config: disable_concurrent_writes
40298
40299 • Env Var: RCLONE_SFTP_DISABLE_CONCURRENT_WRITES
40300
40301 • Type: bool
40302
40303 • Default: false
40304
40305 --sftp-idle-timeout
40306 Max time before closing idle connections.
40307
40308 If no connections have been returned to the connection pool in the time
40309 given, rclone will empty the connection pool.
40310
40311 Set to 0 to keep connections indefinitely.
40312
40313 Properties:
40314
40315 • Config: idle_timeout
40316
40317 • Env Var: RCLONE_SFTP_IDLE_TIMEOUT
40318
40319 • Type: Duration
40320
40321 • Default: 1m0s
40322
40323 --sftp-chunk-size
40324 Upload and download chunk size.
40325
40326 This controls the maximum size of payload in SFTP protocol packets.
40327 The RFC limits this to 32768 bytes (32k), which is the default. Howev‐
40328 er, a lot of servers support larger sizes, typically limited to a maxi‐
40329 mum total package size of 256k, and setting it larger will increase
40330 transfer speed dramatically on high latency links. This includes
40331 OpenSSH, and, for example, using the value of 255k works well, leaving
40332 plenty of room for overhead while still being within a total packet
40333 size of 256k.
40334
40335 Make sure to test thoroughly before using a value higher than 32k, and
40336 only use it if you always connect to the same server or after suffi‐
40337 ciently broad testing. If you get errors such as "failed to send pack‐
40338 et payload: EOF", lots of "connection lost", or "corrupted on trans‐
40339 fer", when copying a larger file, try lowering the value. The server
40340 run by rclone serve sftp sends packets with standard 32k maximum pay‐
40341 load so you must not set a different chunk_size when downloading files,
40342 but it accepts packets up to the 256k total size, so for uploads the
40343 chunk_size can be set as for the OpenSSH example above.
40344
40345 Properties:
40346
40347 • Config: chunk_size
40348
40349 • Env Var: RCLONE_SFTP_CHUNK_SIZE
40350
40351 • Type: SizeSuffix
40352
40353 • Default: 32Ki
40354
40355 --sftp-concurrency
40356 The maximum number of outstanding requests for one file
40357
40358 This controls the maximum number of outstanding requests for one file.
40359 Increasing it will increase throughput on high latency links at the
40360 cost of using more memory.
40361
40362 Properties:
40363
40364 • Config: concurrency
40365
40366 • Env Var: RCLONE_SFTP_CONCURRENCY
40367
40368 • Type: int
40369
40370 • Default: 64
40371
40372 --sftp-set-env
40373 Environment variables to pass to sftp and commands
40374
40375 Set environment variables in the form:
40376
40377 VAR=value
40378
40379 to be passed to the sftp client and to any commands run (eg md5sum).
40380
40381 Pass multiple variables space separated, eg
40382
40383 VAR1=value VAR2=value
40384
40385 and pass variables with spaces in in quotes, eg
40386
40387 "VAR3=value with space" "VAR4=value with space" VAR5=nospacehere
40388
40389 Properties:
40390
40391 • Config: set_env
40392
40393 • Env Var: RCLONE_SFTP_SET_ENV
40394
40395 • Type: SpaceSepList
40396
40397 • Default:
40398
40399 Limitations
40400 On some SFTP servers (e.g. Synology) the paths are different for SSH
40401 and SFTP so the hashes can't be calculated properly. For them using
40402 disable_hashcheck is a good idea.
40403
40404 The only ssh agent supported under Windows is Putty's pageant.
40405
40406 The Go SSH library disables the use of the aes128-cbc cipher by de‐
40407 fault, due to security concerns. This can be re-enabled on a per-con‐
40408 nection basis by setting the use_insecure_cipher setting in the config‐
40409 uration file to true. Further details on the insecurity of this cipher
40410 can be found in this paper (http://www.isg.rhul.ac.uk/~kp/SandPfi‐
40411 nal.pdf).
40412
40413 SFTP isn't supported under plan9 until this issue
40414 (https://github.com/pkg/sftp/issues/156) is fixed.
40415
40416 Note that since SFTP isn't HTTP based the following flags don't work
40417 with it: --dump-headers, --dump-bodies, --dump-auth.
40418
40419 Note that --timeout and --contimeout are both supported.
40420
40421 rsync.net
40422 rsync.net is supported through the SFTP backend.
40423
40424 See rsync.net's documentation of rclone examples
40425 (https://www.rsync.net/products/rclone.html).
40426
40427 Hetzner Storage Box
40428 Hetzner Storage Boxes are supported through the SFTP backend on port
40429 23.
40430
40431 See Hetzner's documentation for details (https://docs.hetzner.com/ro‐
40432 bot/storage-box/access/access-ssh-rsync-borg#rclone)
40433
40435 SMB is a communication protocol to share files over network
40436 (https://en.wikipedia.org/wiki/Server_Message_Block).
40437
40438 This relies on go-smb2 library (https://github.com/hirochachacha/go-
40439 smb2/) for communication with SMB protocol.
40440
40441 Paths are specified as remote:sharename (or remote: for the lsd com‐
40442 mand.) You may put subdirectories in too, e.g. re‐
40443 mote:item/path/to/dir.
40444
40445 Notes
40446 The first path segment must be the name of the share, which you entered
40447 when you started to share on Windows. On smbd, it's the section title
40448 in smb.conf (usually in /etc/samba/) file. You can find shares by
40449 quering the root if you're unsure (e.g. rclone lsd remote:).
40450
40451 You can't access to the shared printers from rclone, obviously.
40452
40453 You can't use Anonymous access for logging in. You have to use the
40454 guest user with an empty password instead. The rclone client tries to
40455 avoid 8.3 names when uploading files by encoding trailing spaces and
40456 periods. Alternatively, the local backend (https://rclone.org/lo‐
40457 cal/#paths-on-windows) on Windows can access SMB servers using UNC
40458 paths, by \\server\share. This doesn't apply to non-Windows OSes, such
40459 as Linux and macOS.
40460
40461 Configuration
40462 Here is an example of making a SMB configuration.
40463
40464 First run
40465
40466 rclone config
40467
40468 This will guide you through an interactive setup process.
40469
40470 No remotes found, make a new one?
40471 n) New remote
40472 s) Set configuration password
40473 q) Quit config
40474 n/s/q> n
40475 name> remote
40476 Option Storage.
40477 Type of storage to configure.
40478 Choose a number from below, or type in your own value.
40479 XX / SMB / CIFS
40480 \ (smb)
40481 Storage> smb
40482
40483 Option host.
40484 Samba hostname to connect to.
40485 E.g. "example.com".
40486 Enter a value.
40487 host> localhost
40488
40489 Option user.
40490 Samba username.
40491 Enter a string value. Press Enter for the default (lesmi).
40492 user> guest
40493
40494 Option port.
40495 Samba port number.
40496 Enter a signed integer. Press Enter for the default (445).
40497 port>
40498
40499 Option pass.
40500 Samba password.
40501 Choose an alternative below. Press Enter for the default (n).
40502 y) Yes, type in my own password
40503 g) Generate random password
40504 n) No, leave this optional password blank (default)
40505 y/g/n> g
40506 Password strength in bits.
40507 64 is just about memorable
40508 128 is secure
40509 1024 is the maximum
40510 Bits> 64
40511 Your password is: XXXX
40512 Use this password? Please note that an obscured version of this
40513 password (and not the password itself) will be stored under your
40514 configuration file, so keep this generated password in a safe place.
40515 y) Yes (default)
40516 n) No
40517 y/n> y
40518
40519 Option domain.
40520 Domain name for NTLM authentication.
40521 Enter a string value. Press Enter for the default (WORKGROUP).
40522 domain>
40523
40524 Edit advanced config?
40525 y) Yes
40526 n) No (default)
40527 y/n> n
40528
40529 Configuration complete.
40530 Options:
40531 - type: samba
40532 - host: localhost
40533 - user: guest
40534 - pass: *** ENCRYPTED ***
40535 Keep this "remote" remote?
40536 y) Yes this is OK (default)
40537 e) Edit this remote
40538 d) Delete this remote
40539 y/e/d> d
40540
40541 Standard options
40542 Here are the Standard options specific to smb (SMB / CIFS).
40543
40544 --smb-host
40545 SMB server hostname to connect to.
40546
40547 E.g. "example.com".
40548
40549 Properties:
40550
40551 • Config: host
40552
40553 • Env Var: RCLONE_SMB_HOST
40554
40555 • Type: string
40556
40557 • Required: true
40558
40559 --smb-user
40560 SMB username.
40561
40562 Properties:
40563
40564 • Config: user
40565
40566 • Env Var: RCLONE_SMB_USER
40567
40568 • Type: string
40569
40570 • Default: "$USER"
40571
40572 --smb-port
40573 SMB port number.
40574
40575 Properties:
40576
40577 • Config: port
40578
40579 • Env Var: RCLONE_SMB_PORT
40580
40581 • Type: int
40582
40583 • Default: 445
40584
40585 --smb-pass
40586 SMB password.
40587
40588 NB Input to this must be obscured - see rclone obscure
40589 (https://rclone.org/commands/rclone_obscure/).
40590
40591 Properties:
40592
40593 • Config: pass
40594
40595 • Env Var: RCLONE_SMB_PASS
40596
40597 • Type: string
40598
40599 • Required: false
40600
40601 --smb-domain
40602 Domain name for NTLM authentication.
40603
40604 Properties:
40605
40606 • Config: domain
40607
40608 • Env Var: RCLONE_SMB_DOMAIN
40609
40610 • Type: string
40611
40612 • Default: "WORKGROUP"
40613
40614 Advanced options
40615 Here are the Advanced options specific to smb (SMB / CIFS).
40616
40617 --smb-idle-timeout
40618 Max time before closing idle connections.
40619
40620 If no connections have been returned to the connection pool in the time
40621 given, rclone will empty the connection pool.
40622
40623 Set to 0 to keep connections indefinitely.
40624
40625 Properties:
40626
40627 • Config: idle_timeout
40628
40629 • Env Var: RCLONE_SMB_IDLE_TIMEOUT
40630
40631 • Type: Duration
40632
40633 • Default: 1m0s
40634
40635 --smb-hide-special-share
40636 Hide special shares (e.g. print$) which users aren't supposed to ac‐
40637 cess.
40638
40639 Properties:
40640
40641 • Config: hide_special_share
40642
40643 • Env Var: RCLONE_SMB_HIDE_SPECIAL_SHARE
40644
40645 • Type: bool
40646
40647 • Default: true
40648
40649 --smb-case-insensitive
40650 Whether the server is configured to be case-insensitive.
40651
40652 Always true on Windows shares.
40653
40654 Properties:
40655
40656 • Config: case_insensitive
40657
40658 • Env Var: RCLONE_SMB_CASE_INSENSITIVE
40659
40660 • Type: bool
40661
40662 • Default: true
40663
40664 --smb-encoding
40665 The encoding for the backend.
40666
40667 See the encoding section in the overview (https://rclone.org/over‐
40668 view/#encoding) for more info.
40669
40670 Properties:
40671
40672 • Config: encoding
40673
40674 • Env Var: RCLONE_SMB_ENCODING
40675
40676 • Type: MultiEncoder
40677
40678 • Default: Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,Back‐
40679 Slash,Ctl,RightSpace,RightPeriod,InvalidUtf8,Dot
40680
40682 Storj (https://storj.io) is an encrypted, secure, and cost-effective
40683 object storage service that enables you to store, back up, and archive
40684 large amounts of data in a decentralized manner.
40685
40686 Backend options
40687 Storj can be used both with this native backend and with the s3 backend
40688 using the Storj S3 compatible gateway (https://rclone.org/s3/#storj)
40689 (shared or private).
40690
40691 Use this backend to take advantage of client-side encryption as well as
40692 to achieve the best possible download performance. Uploads will be
40693 erasure-coded locally, thus a 1gb upload will result in 2.68gb of data
40694 being uploaded to storage nodes across the network.
40695
40696 Use the s3 backend and one of the S3 compatible Hosted Gateways to in‐
40697 crease upload performance and reduce the load on your systems and net‐
40698 work. Uploads will be encrypted and erasure-coded server-side, thus a
40699 1GB upload will result in only in 1GB of data being uploaded to storage
40700 nodes across the network.
40701
40702 Side by side comparison with more details:
40703
40704 • Characteristics:
40705
40706 • Storj backend: Uses native RPC protocol, connects directly to the
40707 storage nodes which hosts the data. Requires more CPU resource of
40708 encoding/decoding and has network amplification (especially during
40709 the upload), uses lots of TCP connections
40710
40711 • S3 backend: Uses S3 compatible HTTP Rest API via the shared gate‐
40712 ways. There is no network amplification, but performance depends
40713 on the shared gateways and the secret encryption key is shared with
40714 the gateway.
40715
40716 • Typical usage:
40717
40718 • Storj backend: Server environments and desktops with enough re‐
40719 sources, internet speed and connectivity - and applications where
40720 storjs client-side encryption is required.
40721
40722 • S3 backend: Desktops and similar with limited resources, internet
40723 speed or connectivity.
40724
40725 • Security:
40726
40727 • Storj backend: strong. Private encryption key doesn't need to
40728 leave the local computer.
40729
40730 • S3 backend: weaker. Private encryption key is shared with
40731 (https://docs.storj.io/dcs/api-reference/s3-compatible-gateway#se‐
40732 curity-and-encryption) the authentication service of the hosted
40733 gateway, where it's stored encrypted. It can be stronger when com‐
40734 bining with the rclone crypt backend.
40735
40736 • Bandwidth usage (upload):
40737
40738 • Storj backend: higher. As data is erasure coded on the client side
40739 both the original data and the parities should be uploaded. About
40740 ~2.7 times more data is required to be uploaded. Client may start
40741 to upload with even higher number of nodes (~3.7 times more) and
40742 abandon/stop the slow uploads.
40743
40744 • S3 backend: normal. Only the raw data is uploaded, erasure coding
40745 happens on the gateway.
40746
40747 • Bandwidth usage (download)
40748
40749 • Storj backend: almost normal. Only the minimal number of data is
40750 required, but to avoid very slow data providers a few more sources
40751 are used and the slowest are ignored (max 1.2x overhead).
40752
40753 • S3 backend: normal. Only the raw data is downloaded, erasure cod‐
40754 ing happens on the shared gateway.
40755
40756 • CPU usage:
40757
40758 • Storj backend: higher, but more predictable. Erasure code and en‐
40759 cryption/decryption happens locally which requires significant CPU
40760 usage.
40761
40762 • S3 backend: less. Erasure code and encryption/decryption happens
40763 on shared s3 gateways (and as is, it depends on the current load on
40764 the gateways)
40765
40766 • TCP connection usage:
40767
40768 • Storj backend: high. A direct connection is required to each of
40769 the Storj nodes resulting in 110 connections on upload and 35 on
40770 download per 64 MB segment. Not all the connections are actively
40771 used (slow ones are pruned), but they are all opened. Adjusting
40772 the max open file limit (https://rclone.org/storj/#known-issues)
40773 may be required.
40774
40775 • S3 backend: normal. Only one connection per download/upload thread
40776 is required to the shared gateway.
40777
40778 • Overall performance:
40779
40780 • Storj backend: with enough resources (CPU and bandwidth) storj
40781 backend can provide even 2x better performance. Data is directly
40782 downloaded to / uploaded from to the client instead of the gateway.
40783
40784 • S3 backend: Can be faster on edge devices where CPU and network
40785 bandwidth is limited as the shared S3 compatible gateways take care
40786 about the encrypting/decryption and erasure coding and no down‐
40787 load/upload amplification.
40788
40789 • Decentralization:
40790
40791 • Storj backend: high. Data is downloaded directly from the distrib‐
40792 uted cloud of storage providers.
40793
40794 • S3 backend: low. Requires a running S3 gateway (either self-hosted
40795 or Storj-hosted).
40796
40797 • Limitations:
40798
40799 • Storj backend: rclone checksum is not possible without download, as
40800 checksum metadata is not calculated during upload
40801
40802 • S3 backend: secret encryption key is shared with the gateway
40803
40804 Configuration
40805 To make a new Storj configuration you need one of the following: * Ac‐
40806 cess Grant that someone else shared with you. * API Key (https://docu‐
40807 mentation.storj.io/getting-started/uploading-your-first-object/create-
40808 an-api-key) of a Storj project you are a member of.
40809
40810 Here is an example of how to make a remote called remote. First run:
40811
40812 rclone config
40813
40814 This will guide you through an interactive setup process:
40815
40816 Setup with access grant
40817 No remotes found, make a new one?
40818 n) New remote
40819 s) Set configuration password
40820 q) Quit config
40821 n/s/q> n
40822 name> remote
40823 Type of storage to configure.
40824 Enter a string value. Press Enter for the default ("").
40825 Choose a number from below, or type in your own value
40826 [snip]
40827 XX / Storj Decentralized Cloud Storage
40828 \ "storj"
40829 [snip]
40830 Storage> storj
40831 ** See help for storj backend at: https://rclone.org/storj/ **
40832
40833 Choose an authentication method.
40834 Enter a string value. Press Enter for the default ("existing").
40835 Choose a number from below, or type in your own value
40836 1 / Use an existing access grant.
40837 \ "existing"
40838 2 / Create a new access grant from satellite address, API key, and passphrase.
40839 \ "new"
40840 provider> existing
40841 Access Grant.
40842 Enter a string value. Press Enter for the default ("").
40843 access_grant> your-access-grant-received-by-someone-else
40844 Remote config
40845 --------------------
40846 [remote]
40847 type = storj
40848 access_grant = your-access-grant-received-by-someone-else
40849 --------------------
40850 y) Yes this is OK (default)
40851 e) Edit this remote
40852 d) Delete this remote
40853 y/e/d> y
40854
40855 Setup with API key and passphrase
40856 No remotes found, make a new one?
40857 n) New remote
40858 s) Set configuration password
40859 q) Quit config
40860 n/s/q> n
40861 name> remote
40862 Type of storage to configure.
40863 Enter a string value. Press Enter for the default ("").
40864 Choose a number from below, or type in your own value
40865 [snip]
40866 XX / Storj Decentralized Cloud Storage
40867 \ "storj"
40868 [snip]
40869 Storage> storj
40870 ** See help for storj backend at: https://rclone.org/storj/ **
40871
40872 Choose an authentication method.
40873 Enter a string value. Press Enter for the default ("existing").
40874 Choose a number from below, or type in your own value
40875 1 / Use an existing access grant.
40876 \ "existing"
40877 2 / Create a new access grant from satellite address, API key, and passphrase.
40878 \ "new"
40879 provider> new
40880 Satellite Address. Custom satellite address should match the format: `<nodeid>@<address>:<port>`.
40881 Enter a string value. Press Enter for the default ("us-central-1.storj.io").
40882 Choose a number from below, or type in your own value
40883 1 / US Central 1
40884 \ "us-central-1.storj.io"
40885 2 / Europe West 1
40886 \ "europe-west-1.storj.io"
40887 3 / Asia East 1
40888 \ "asia-east-1.storj.io"
40889 satellite_address> 1
40890 API Key.
40891 Enter a string value. Press Enter for the default ("").
40892 api_key> your-api-key-for-your-storj-project
40893 Encryption Passphrase. To access existing objects enter passphrase used for uploading.
40894 Enter a string value. Press Enter for the default ("").
40895 passphrase> your-human-readable-encryption-passphrase
40896 Remote config
40897 --------------------
40898 [remote]
40899 type = storj
40900 satellite_address = 12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us-central-1.tardigrade.io:7777
40901 api_key = your-api-key-for-your-storj-project
40902 passphrase = your-human-readable-encryption-passphrase
40903 access_grant = the-access-grant-generated-from-the-api-key-and-passphrase
40904 --------------------
40905 y) Yes this is OK (default)
40906 e) Edit this remote
40907 d) Delete this remote
40908 y/e/d> y
40909
40910 Standard options
40911 Here are the Standard options specific to storj (Storj Decentralized
40912 Cloud Storage).
40913
40914 --storj-provider
40915 Choose an authentication method.
40916
40917 Properties:
40918
40919 • Config: provider
40920
40921 • Env Var: RCLONE_STORJ_PROVIDER
40922
40923 • Type: string
40924
40925 • Default: "existing"
40926
40927 • Examples:
40928
40929 • "existing"
40930
40931 • Use an existing access grant.
40932
40933 • "new"
40934
40935 • Create a new access grant from satellite address, API key, and
40936 passphrase.
40937
40938 --storj-access-grant
40939 Access grant.
40940
40941 Properties:
40942
40943 • Config: access_grant
40944
40945 • Env Var: RCLONE_STORJ_ACCESS_GRANT
40946
40947 • Provider: existing
40948
40949 • Type: string
40950
40951 • Required: false
40952
40953 --storj-satellite-address
40954 Satellite address.
40955
40956 Custom satellite address should match the format: <nodeid>@<ad‐
40957 dress>:<port>.
40958
40959 Properties:
40960
40961 • Config: satellite_address
40962
40963 • Env Var: RCLONE_STORJ_SATELLITE_ADDRESS
40964
40965 • Provider: new
40966
40967 • Type: string
40968
40969 • Default: "us-central-1.storj.io"
40970
40971 • Examples:
40972
40973 • "us-central-1.storj.io"
40974
40975 • US Central 1
40976
40977 • "europe-west-1.storj.io"
40978
40979 • Europe West 1
40980
40981 • "asia-east-1.storj.io"
40982
40983 • Asia East 1
40984
40985 --storj-api-key
40986 API key.
40987
40988 Properties:
40989
40990 • Config: api_key
40991
40992 • Env Var: RCLONE_STORJ_API_KEY
40993
40994 • Provider: new
40995
40996 • Type: string
40997
40998 • Required: false
40999
41000 --storj-passphrase
41001 Encryption passphrase.
41002
41003 To access existing objects enter passphrase used for uploading.
41004
41005 Properties:
41006
41007 • Config: passphrase
41008
41009 • Env Var: RCLONE_STORJ_PASSPHRASE
41010
41011 • Provider: new
41012
41013 • Type: string
41014
41015 • Required: false
41016
41017 Usage
41018 Paths are specified as remote:bucket (or remote: for the lsf command.)
41019 You may put subdirectories in too, e.g. remote:bucket/path/to/dir.
41020
41021 Once configured you can then use rclone like this.
41022
41023 Create a new bucket
41024 Use the mkdir command to create new bucket, e.g. bucket.
41025
41026 rclone mkdir remote:bucket
41027
41028 List all buckets
41029 Use the lsf command to list all buckets.
41030
41031 rclone lsf remote:
41032
41033 Note the colon (:) character at the end of the command line.
41034
41035 Delete a bucket
41036 Use the rmdir command to delete an empty bucket.
41037
41038 rclone rmdir remote:bucket
41039
41040 Use the purge command to delete a non-empty bucket with all its con‐
41041 tent.
41042
41043 rclone purge remote:bucket
41044
41045 Upload objects
41046 Use the copy command to upload an object.
41047
41048 rclone copy --progress /home/local/directory/file.ext remote:bucket/path/to/dir/
41049
41050 The --progress flag is for displaying progress information. Remove it
41051 if you don't need this information.
41052
41053 Use a folder in the local path to upload all its objects.
41054
41055 rclone copy --progress /home/local/directory/ remote:bucket/path/to/dir/
41056
41057 Only modified files will be copied.
41058
41059 List objects
41060 Use the ls command to list recursively all objects in a bucket.
41061
41062 rclone ls remote:bucket
41063
41064 Add the folder to the remote path to list recursively all objects in
41065 this folder.
41066
41067 rclone ls remote:bucket/path/to/dir/
41068
41069 Use the lsf command to list non-recursively all objects in a bucket or
41070 a folder.
41071
41072 rclone lsf remote:bucket/path/to/dir/
41073
41074 Download objects
41075 Use the copy command to download an object.
41076
41077 rclone copy --progress remote:bucket/path/to/dir/file.ext /home/local/directory/
41078
41079 The --progress flag is for displaying progress information. Remove it
41080 if you don't need this information.
41081
41082 Use a folder in the remote path to download all its objects.
41083
41084 rclone copy --progress remote:bucket/path/to/dir/ /home/local/directory/
41085
41086 Delete objects
41087 Use the deletefile command to delete a single object.
41088
41089 rclone deletefile remote:bucket/path/to/dir/file.ext
41090
41091 Use the delete command to delete all object in a folder.
41092
41093 rclone delete remote:bucket/path/to/dir/
41094
41095 Print the total size of objects
41096 Use the size command to print the total size of objects in a bucket or
41097 a folder.
41098
41099 rclone size remote:bucket/path/to/dir/
41100
41101 Sync two Locations
41102 Use the sync command to sync the source to the destination, changing
41103 the destination only, deleting any excess files.
41104
41105 rclone sync -i --progress /home/local/directory/ remote:bucket/path/to/dir/
41106
41107 The --progress flag is for displaying progress information. Remove it
41108 if you don't need this information.
41109
41110 Since this can cause data loss, test first with the --dry-run flag to
41111 see exactly what would be copied and deleted.
41112
41113 The sync can be done also from Storj to the local file system.
41114
41115 rclone sync -i --progress remote:bucket/path/to/dir/ /home/local/directory/
41116
41117 Or between two Storj buckets.
41118
41119 rclone sync -i --progress remote-us:bucket/path/to/dir/ remote-europe:bucket/path/to/dir/
41120
41121 Or even between another cloud storage and Storj.
41122
41123 rclone sync -i --progress s3:bucket/path/to/dir/ storj:bucket/path/to/dir/
41124
41125 Limitations
41126 rclone about is not supported by the rclone Storj backend. Backends
41127 without this capability cannot determine free space for an rclone mount
41128 or use policy mfs (most free space) as a member of an rclone union re‐
41129 mote.
41130
41131 See List of backends that do not support rclone about
41132 (https://rclone.org/overview/#optional-features) and rclone about
41133 (https://rclone.org/commands/rclone_about/)
41134
41135 Known issues
41136 If you get errors like too many open files this usually happens when
41137 the default ulimit for system max open files is exceeded. Native Storj
41138 protocol opens a large number of TCP connections (each of which is
41139 counted as an open file). For a single upload stream you can expect
41140 110 TCP connections to be opened. For a single download stream you can
41141 expect 35. This batch of connections will be opened for every 64 MiB
41142 segment and you should also expect TCP connections to be reused. If
41143 you do many transfers you eventually open a connection to most storage
41144 nodes (thousands of nodes).
41145
41146 To fix these, please raise your system limits. You can do this issuing
41147 a ulimit -n 65536 just before you run rclone. To change the limits
41148 more permanently you can add this to your shell startup script, e.g.
41149 $HOME/.bashrc, or change the system-wide configuration, usually
41150 /etc/sysctl.conf and/or /etc/security/limits.conf, but please refer to
41151 your operating system manual.
41152
41154 SugarSync (https://sugarsync.com) is a cloud service that enables ac‐
41155 tive synchronization of files across computers and other devices for
41156 file backup, access, syncing, and sharing.
41157
41158 Configuration
41159 The initial setup for SugarSync involves getting a token from SugarSync
41160 which you can do with rclone. rclone config walks you through it.
41161
41162 Here is an example of how to make a remote called remote. First run:
41163
41164 rclone config
41165
41166 This will guide you through an interactive setup process:
41167
41168 No remotes found, make a new one?
41169 n) New remote
41170 s) Set configuration password
41171 q) Quit config
41172 n/s/q> n
41173 name> remote
41174 Type of storage to configure.
41175 Enter a string value. Press Enter for the default ("").
41176 Choose a number from below, or type in your own value
41177 [snip]
41178 XX / Sugarsync
41179 \ "sugarsync"
41180 [snip]
41181 Storage> sugarsync
41182 ** See help for sugarsync backend at: https://rclone.org/sugarsync/ **
41183
41184 Sugarsync App ID.
41185 Leave blank to use rclone's.
41186 Enter a string value. Press Enter for the default ("").
41187 app_id>
41188 Sugarsync Access Key ID.
41189 Leave blank to use rclone's.
41190 Enter a string value. Press Enter for the default ("").
41191 access_key_id>
41192 Sugarsync Private Access Key
41193 Leave blank to use rclone's.
41194 Enter a string value. Press Enter for the default ("").
41195 private_access_key>
41196 Permanently delete files if true
41197 otherwise put them in the deleted files.
41198 Enter a boolean value (true or false). Press Enter for the default ("false").
41199 hard_delete>
41200 Edit advanced config? (y/n)
41201 y) Yes
41202 n) No (default)
41203 y/n> n
41204 Remote config
41205 Username (email address)> nick@craig-wood.com
41206 Your Sugarsync password is only required during setup and will not be stored.
41207 password:
41208 --------------------
41209 [remote]
41210 type = sugarsync
41211 refresh_token = https://api.sugarsync.com/app-authorization/XXXXXXXXXXXXXXXXXX
41212 --------------------
41213 y) Yes this is OK (default)
41214 e) Edit this remote
41215 d) Delete this remote
41216 y/e/d> y
41217
41218 Note that the config asks for your email and password but doesn't store
41219 them, it only uses them to get the initial token.
41220
41221 Once configured you can then use rclone like this,
41222
41223 List directories (sync folders) in top level of your SugarSync
41224
41225 rclone lsd remote:
41226
41227 List all the files in your SugarSync folder "Test"
41228
41229 rclone ls remote:Test
41230
41231 To copy a local directory to an SugarSync folder called backup
41232
41233 rclone copy /home/source remote:backup
41234
41235 Paths are specified as remote:path
41236
41237 Paths may be as deep as required, e.g. remote:directory/subdirectory.
41238
41239 NB you can't create files in the top level folder you have to create a
41240 folder, which rclone will create as a "Sync Folder" with SugarSync.
41241
41242 Modified time and hashes
41243 SugarSync does not support modification times or hashes, therefore
41244 syncing will default to --size-only checking. Note that using --update
41245 will work as rclone can read the time files were uploaded.
41246
41247 Restricted filename characters
41248 SugarSync replaces the default restricted characters set
41249 (https://rclone.org/overview/#restricted-characters) except for DEL.
41250
41251 Invalid UTF-8 bytes will also be replaced (https://rclone.org/over‐
41252 view/#invalid-utf8), as they can't be used in XML strings.
41253
41254 Deleting files
41255 Deleted files will be moved to the "Deleted items" folder by default.
41256
41257 However you can supply the flag --sugarsync-hard-delete or set the con‐
41258 fig parameter hard_delete = true if you would like files to be deleted
41259 straight away.
41260
41261 Standard options
41262 Here are the Standard options specific to sugarsync (Sugarsync).
41263
41264 --sugarsync-app-id
41265 Sugarsync App ID.
41266
41267 Leave blank to use rclone's.
41268
41269 Properties:
41270
41271 • Config: app_id
41272
41273 • Env Var: RCLONE_SUGARSYNC_APP_ID
41274
41275 • Type: string
41276
41277 • Required: false
41278
41279 --sugarsync-access-key-id
41280 Sugarsync Access Key ID.
41281
41282 Leave blank to use rclone's.
41283
41284 Properties:
41285
41286 • Config: access_key_id
41287
41288 • Env Var: RCLONE_SUGARSYNC_ACCESS_KEY_ID
41289
41290 • Type: string
41291
41292 • Required: false
41293
41294 --sugarsync-private-access-key
41295 Sugarsync Private Access Key.
41296
41297 Leave blank to use rclone's.
41298
41299 Properties:
41300
41301 • Config: private_access_key
41302
41303 • Env Var: RCLONE_SUGARSYNC_PRIVATE_ACCESS_KEY
41304
41305 • Type: string
41306
41307 • Required: false
41308
41309 --sugarsync-hard-delete
41310 Permanently delete files if true otherwise put them in the deleted
41311 files.
41312
41313 Properties:
41314
41315 • Config: hard_delete
41316
41317 • Env Var: RCLONE_SUGARSYNC_HARD_DELETE
41318
41319 • Type: bool
41320
41321 • Default: false
41322
41323 Advanced options
41324 Here are the Advanced options specific to sugarsync (Sugarsync).
41325
41326 --sugarsync-refresh-token
41327 Sugarsync refresh token.
41328
41329 Leave blank normally, will be auto configured by rclone.
41330
41331 Properties:
41332
41333 • Config: refresh_token
41334
41335 • Env Var: RCLONE_SUGARSYNC_REFRESH_TOKEN
41336
41337 • Type: string
41338
41339 • Required: false
41340
41341 --sugarsync-authorization
41342 Sugarsync authorization.
41343
41344 Leave blank normally, will be auto configured by rclone.
41345
41346 Properties:
41347
41348 • Config: authorization
41349
41350 • Env Var: RCLONE_SUGARSYNC_AUTHORIZATION
41351
41352 • Type: string
41353
41354 • Required: false
41355
41356 --sugarsync-authorization-expiry
41357 Sugarsync authorization expiry.
41358
41359 Leave blank normally, will be auto configured by rclone.
41360
41361 Properties:
41362
41363 • Config: authorization_expiry
41364
41365 • Env Var: RCLONE_SUGARSYNC_AUTHORIZATION_EXPIRY
41366
41367 • Type: string
41368
41369 • Required: false
41370
41371 --sugarsync-user
41372 Sugarsync user.
41373
41374 Leave blank normally, will be auto configured by rclone.
41375
41376 Properties:
41377
41378 • Config: user
41379
41380 • Env Var: RCLONE_SUGARSYNC_USER
41381
41382 • Type: string
41383
41384 • Required: false
41385
41386 --sugarsync-root-id
41387 Sugarsync root id.
41388
41389 Leave blank normally, will be auto configured by rclone.
41390
41391 Properties:
41392
41393 • Config: root_id
41394
41395 • Env Var: RCLONE_SUGARSYNC_ROOT_ID
41396
41397 • Type: string
41398
41399 • Required: false
41400
41401 --sugarsync-deleted-id
41402 Sugarsync deleted folder id.
41403
41404 Leave blank normally, will be auto configured by rclone.
41405
41406 Properties:
41407
41408 • Config: deleted_id
41409
41410 • Env Var: RCLONE_SUGARSYNC_DELETED_ID
41411
41412 • Type: string
41413
41414 • Required: false
41415
41416 --sugarsync-encoding
41417 The encoding for the backend.
41418
41419 See the encoding section in the overview (https://rclone.org/over‐
41420 view/#encoding) for more info.
41421
41422 Properties:
41423
41424 • Config: encoding
41425
41426 • Env Var: RCLONE_SUGARSYNC_ENCODING
41427
41428 • Type: MultiEncoder
41429
41430 • Default: Slash,Ctl,InvalidUtf8,Dot
41431
41432 Limitations
41433 rclone about is not supported by the SugarSync backend. Backends with‐
41434 out this capability cannot determine free space for an rclone mount or
41435 use policy mfs (most free space) as a member of an rclone union remote.
41436
41437 See List of backends that do not support rclone about
41438 (https://rclone.org/overview/#optional-features) and rclone about
41439 (https://rclone.org/commands/rclone_about/)
41440
41442 The Tardigrade backend has been renamed to be the Storj backend
41443 (https://rclone.org/storj/). Old configuration files will continue to
41444 work.
41445
41447 This is a Backend for Uptobox file storage service. Uptobox is closer
41448 to a one-click hoster than a traditional cloud storage provider and
41449 therefore not suitable for long term storage.
41450
41451 Paths are specified as remote:path
41452
41453 Paths may be as deep as required, e.g. remote:directory/subdirectory.
41454
41455 Configuration
41456 To configure an Uptobox backend you'll need your personal api token.
41457 You'll find it in your account settings (https://uptobox.com/my_ac‐
41458 count)
41459
41460 Here is an example of how to make a remote called remote with the de‐
41461 fault setup. First run:
41462
41463 rclone config
41464
41465 This will guide you through an interactive setup process:
41466
41467 Current remotes:
41468
41469 Name Type
41470 ==== ====
41471 TestUptobox uptobox
41472
41473 e) Edit existing remote
41474 n) New remote
41475 d) Delete remote
41476 r) Rename remote
41477 c) Copy remote
41478 s) Set configuration password
41479 q) Quit config
41480 e/n/d/r/c/s/q> n
41481 name> uptobox
41482 Type of storage to configure.
41483 Enter a string value. Press Enter for the default ("").
41484 Choose a number from below, or type in your own value
41485 [...]
41486 37 / Uptobox
41487 \ "uptobox"
41488 [...]
41489 Storage> uptobox
41490 ** See help for uptobox backend at: https://rclone.org/uptobox/ **
41491
41492 Your API Key, get it from https://uptobox.com/my_account
41493 Enter a string value. Press Enter for the default ("").
41494 api_key> xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
41495 Edit advanced config? (y/n)
41496 y) Yes
41497 n) No (default)
41498 y/n> n
41499 Remote config
41500 --------------------
41501 [uptobox]
41502 type = uptobox
41503 api_key = xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
41504 --------------------
41505 y) Yes this is OK (default)
41506 e) Edit this remote
41507 d) Delete this remote
41508 y/e/d>
41509
41510 Once configured you can then use rclone like this,
41511
41512 List directories in top level of your Uptobox
41513
41514 rclone lsd remote:
41515
41516 List all the files in your Uptobox
41517
41518 rclone ls remote:
41519
41520 To copy a local directory to an Uptobox directory called backup
41521
41522 rclone copy /home/source remote:backup
41523
41524 Modified time and hashes
41525 Uptobox supports neither modified times nor checksums.
41526
41527 Restricted filename characters
41528 In addition to the default restricted characters set
41529 (https://rclone.org/overview/#restricted-characters) the following
41530 characters are also replaced:
41531
41532 Character Value Replacement
41533 ────────────────────────────────
41534 " 0x22 "
41535 ` 0x41 `
41536
41537 Invalid UTF-8 bytes will also be replaced (https://rclone.org/over‐
41538 view/#invalid-utf8), as they can't be used in XML strings.
41539
41540 Standard options
41541 Here are the Standard options specific to uptobox (Uptobox).
41542
41543 --uptobox-access-token
41544 Your access token.
41545
41546 Get it from https://uptobox.com/my_account.
41547
41548 Properties:
41549
41550 • Config: access_token
41551
41552 • Env Var: RCLONE_UPTOBOX_ACCESS_TOKEN
41553
41554 • Type: string
41555
41556 • Required: false
41557
41558 Advanced options
41559 Here are the Advanced options specific to uptobox (Uptobox).
41560
41561 --uptobox-encoding
41562 The encoding for the backend.
41563
41564 See the encoding section in the overview (https://rclone.org/over‐
41565 view/#encoding) for more info.
41566
41567 Properties:
41568
41569 • Config: encoding
41570
41571 • Env Var: RCLONE_UPTOBOX_ENCODING
41572
41573 • Type: MultiEncoder
41574
41575 • Default: Slash,LtGt,DoubleQuote,BackQuote,Del,Ctl,LeftSpace,In‐
41576 validUtf8,Dot
41577
41578 Limitations
41579 Uptobox will delete inactive files that have not been accessed in 60
41580 days.
41581
41582 rclone about is not supported by this backend an overview of used space
41583 can however been seen in the uptobox web interface.
41584
41586 The union remote provides a unification similar to UnionFS using other
41587 remotes.
41588
41589 Paths may be as deep as required or a local path, e.g. remote:directo‐
41590 ry/subdirectory or /directory/subdirectory.
41591
41592 During the initial setup with rclone config you will specify the up‐
41593 stream remotes as a space separated list. The upstream remotes can ei‐
41594 ther be a local paths or other remotes.
41595
41596 Attribute :ro and :nc can be attach to the end of path to tag the re‐
41597 mote as read only or no create, e.g. remote:directory/subdirectory:ro
41598 or remote:directory/subdirectory:nc.
41599
41600 Subfolders can be used in upstream remotes. Assume a union remote
41601 named backup with the remotes mydrive:private/backup. Invoking rclone
41602 mkdir backup:desktop is exactly the same as invoking rclone mkdir my‐
41603 drive:private/backup/desktop.
41604
41605 There will be no special handling of paths containing .. segments.
41606 Invoking rclone mkdir backup:../desktop is exactly the same as invoking
41607 rclone mkdir mydrive:private/backup/../desktop.
41608
41609 Configuration
41610 Here is an example of how to make a union called remote for local fold‐
41611 ers. First run:
41612
41613 rclone config
41614
41615 This will guide you through an interactive setup process:
41616
41617 No remotes found, make a new one?
41618 n) New remote
41619 s) Set configuration password
41620 q) Quit config
41621 n/s/q> n
41622 name> remote
41623 Type of storage to configure.
41624 Choose a number from below, or type in your own value
41625 [snip]
41626 XX / Union merges the contents of several remotes
41627 \ "union"
41628 [snip]
41629 Storage> union
41630 List of space separated upstreams.
41631 Can be 'upstreama:test/dir upstreamb:', '\"upstreama:test/space:ro dir\" upstreamb:', etc.
41632 Enter a string value. Press Enter for the default ("").
41633 upstreams> remote1:dir1 remote2:dir2 remote3:dir3
41634 Policy to choose upstream on ACTION class.
41635 Enter a string value. Press Enter for the default ("epall").
41636 action_policy>
41637 Policy to choose upstream on CREATE class.
41638 Enter a string value. Press Enter for the default ("epmfs").
41639 create_policy>
41640 Policy to choose upstream on SEARCH class.
41641 Enter a string value. Press Enter for the default ("ff").
41642 search_policy>
41643 Cache time of usage and free space (in seconds). This option is only useful when a path preserving policy is used.
41644 Enter a signed integer. Press Enter for the default ("120").
41645 cache_time>
41646 Remote config
41647 --------------------
41648 [remote]
41649 type = union
41650 upstreams = remote1:dir1 remote2:dir2 remote3:dir3
41651 --------------------
41652 y) Yes this is OK
41653 e) Edit this remote
41654 d) Delete this remote
41655 y/e/d> y
41656 Current remotes:
41657
41658 Name Type
41659 ==== ====
41660 remote union
41661
41662 e) Edit existing remote
41663 n) New remote
41664 d) Delete remote
41665 r) Rename remote
41666 c) Copy remote
41667 s) Set configuration password
41668 q) Quit config
41669 e/n/d/r/c/s/q> q
41670
41671 Once configured you can then use rclone like this,
41672
41673 List directories in top level in remote1:dir1, remote2:dir2 and re‐
41674 mote3:dir3
41675
41676 rclone lsd remote:
41677
41678 List all the files in remote1:dir1, remote2:dir2 and remote3:dir3
41679
41680 rclone ls remote:
41681
41682 Copy another local directory to the union directory called source,
41683 which will be placed into remote3:dir3
41684
41685 rclone copy C:\source remote:source
41686
41687 Behavior / Policies
41688 The behavior of union backend is inspired by trapexit/mergerfs
41689 (https://github.com/trapexit/mergerfs). All functions are grouped into
41690 3 categories: action, create and search. These functions and cate‐
41691 gories can be assigned a policy which dictates what file or directory
41692 is chosen when performing that behavior. Any policy can be assigned to
41693 a function or category though some may not be very useful in practice.
41694 For instance: rand (random) may be useful for file creation (create)
41695 but could lead to very odd behavior if used for delete if there were
41696 more than one copy of the file.
41697
41698 Function / Category classifications
41699 Cate‐ Description Functions
41700 gory
41701 ─────────────────────────────────────────────────────────────────────────────
41702 action Writing Exist‐ move, rmdir, rmdirs, delete, purge and copy, sync
41703 ing file (as destination when file exist)
41704 create Create non-ex‐ copy, sync (as destination when file not exist)
41705 isting file
41706 search Reading and ls, lsd, lsl, cat, md5sum, sha1sum and copy, sync
41707 listing file (as source)
41708 N/A size, about
41709
41710 Path Preservation
41711 Policies, as described below, are of two basic types. path preserving
41712 and non-path preserving.
41713
41714 All policies which start with ep (epff, eplfs, eplus, epmfs, eprand)
41715 are path preserving. ep stands for existing path.
41716
41717 A path preserving policy will only consider upstreams where the rela‐
41718 tive path being accessed already exists.
41719
41720 When using non-path preserving policies paths will be created in target
41721 upstreams as necessary.
41722
41723 Quota Relevant Policies
41724 Some policies rely on quota information. These policies should be used
41725 only if your upstreams support the respective quota fields.
41726
41727 Policy Required Field
41728 ────────────────────────────
41729 lfs, eplfs Free
41730 mfs, epmfs Free
41731 lus, eplus Used
41732 lno, eplno Objects
41733
41734 To check if your upstream supports the field, run rclone about remote:
41735 [flags] and see if the required field exists.
41736
41737 Filters
41738 Policies basically search upstream remotes and create a list of files /
41739 paths for functions to work on. The policy is responsible for filter‐
41740 ing and sorting. The policy type defines the sorting but filtering is
41741 mostly uniform as described below.
41742
41743 • No search policies filter.
41744
41745 • All action policies will filter out remotes which are tagged as read-
41746 only.
41747
41748 • All create policies will filter out remotes which are tagged read-on‐
41749 ly or no-create.
41750
41751 If all remotes are filtered an error will be returned.
41752
41753 Policy descriptions
41754 The policies definition are inspired by trapexit/mergerfs
41755 (https://github.com/trapexit/mergerfs) but not exactly the same. Some
41756 policy definition could be different due to the much larger latency of
41757 remote file systems.
41758
41759 Policy Description
41760 ──────────────────────────────────────────────────────────────────────────
41761 all Search category: same as epall. Action category: same
41762 as epall. Create category: act on all upstreams.
41763 epall (existing Search category: Given this order configured, act on
41764 path, all) the first one found where the relative path exists.
41765 Action category: apply to all found. Create category:
41766 act on all upstreams where the relative path exists.
41767 epff (existing Act on the first one found, by the time upstreams re‐
41768 path, first ply, where the relative path exists.
41769 found)
41770 eplfs (existing Of all the upstreams on which the relative path exists
41771 path, least free choose the one with the least free space.
41772 space)
41773 eplus (existing Of all the upstreams on which the relative path exists
41774 path, least used choose the one with the least used space.
41775 space)
41776 eplno (existing Of all the upstreams on which the relative path exists
41777 path, least num‐ choose the one with the least number of objects.
41778 ber of objects)
41779 epmfs (existing Of all the upstreams on which the relative path exists
41780 path, most free choose the one with the most free space.
41781 space)
41782 eprand (existing Calls epall and then randomizes. Returns only one up‐
41783 path, random) stream.
41784 ff (first found) Search category: same as epff. Action category: same
41785 as epff. Create category: Act on the first one found
41786 by the time upstreams reply.
41787 lfs (least free Search category: same as eplfs. Action category: same
41788 space) as eplfs. Create category: Pick the upstream with the
41789 least available free space.
41790 lus (least used Search category: same as eplus. Action category: same
41791 space) as eplus. Create category: Pick the upstream with the
41792 least used space.
41793 lno (least num‐ Search category: same as eplno. Action category: same
41794 ber of objects) as eplno. Create category: Pick the upstream with the
41795 least number of objects.
41796 mfs (most free Search category: same as epmfs. Action category: same
41797 space) as epmfs. Create category: Pick the upstream with the
41798 most available free space.
41799 newest Pick the file / directory with the largest mtime.
41800 rand (random) Calls all and then randomizes. Returns only one up‐
41801 stream.
41802
41803 Standard options
41804 Here are the Standard options specific to union (Union merges the con‐
41805 tents of several upstream fs).
41806
41807 --union-upstreams
41808 List of space separated upstreams.
41809
41810 Can be 'upstreama:test/dir upstreamb:', '"upstreama:test/space:ro dir"
41811 upstreamb:', etc.
41812
41813 Properties:
41814
41815 • Config: upstreams
41816
41817 • Env Var: RCLONE_UNION_UPSTREAMS
41818
41819 • Type: string
41820
41821 • Required: true
41822
41823 --union-action-policy
41824 Policy to choose upstream on ACTION category.
41825
41826 Properties:
41827
41828 • Config: action_policy
41829
41830 • Env Var: RCLONE_UNION_ACTION_POLICY
41831
41832 • Type: string
41833
41834 • Default: "epall"
41835
41836 --union-create-policy
41837 Policy to choose upstream on CREATE category.
41838
41839 Properties:
41840
41841 • Config: create_policy
41842
41843 • Env Var: RCLONE_UNION_CREATE_POLICY
41844
41845 • Type: string
41846
41847 • Default: "epmfs"
41848
41849 --union-search-policy
41850 Policy to choose upstream on SEARCH category.
41851
41852 Properties:
41853
41854 • Config: search_policy
41855
41856 • Env Var: RCLONE_UNION_SEARCH_POLICY
41857
41858 • Type: string
41859
41860 • Default: "ff"
41861
41862 --union-cache-time
41863 Cache time of usage and free space (in seconds).
41864
41865 This option is only useful when a path preserving policy is used.
41866
41867 Properties:
41868
41869 • Config: cache_time
41870
41871 • Env Var: RCLONE_UNION_CACHE_TIME
41872
41873 • Type: int
41874
41875 • Default: 120
41876
41877 Advanced options
41878 Here are the Advanced options specific to union (Union merges the con‐
41879 tents of several upstream fs).
41880
41881 --union-min-free-space
41882 Minimum viable free space for lfs/eplfs policies.
41883
41884 If a remote has less than this much free space then it won't be consid‐
41885 ered for use in lfs or eplfs policies.
41886
41887 Properties:
41888
41889 • Config: min_free_space
41890
41891 • Env Var: RCLONE_UNION_MIN_FREE_SPACE
41892
41893 • Type: SizeSuffix
41894
41895 • Default: 1Gi
41896
41897 Metadata
41898 Any metadata supported by the underlying remote is read and written.
41899
41900 See the metadata (https://rclone.org/docs/#metadata) docs for more in‐
41901 fo.
41902
41904 Paths are specified as remote:path
41905
41906 Paths may be as deep as required, e.g. remote:directory/subdirectory.
41907
41908 Configuration
41909 To configure the WebDAV remote you will need to have a URL for it, and
41910 a username and password. If you know what kind of system you are con‐
41911 necting to then rclone can enable extra features.
41912
41913 Here is an example of how to make a remote called remote. First run:
41914
41915 rclone config
41916
41917 This will guide you through an interactive setup process:
41918
41919 No remotes found, make a new one?
41920 n) New remote
41921 s) Set configuration password
41922 q) Quit config
41923 n/s/q> n
41924 name> remote
41925 Type of storage to configure.
41926 Choose a number from below, or type in your own value
41927 [snip]
41928 XX / WebDAV
41929 \ "webdav"
41930 [snip]
41931 Storage> webdav
41932 URL of http host to connect to
41933 Choose a number from below, or type in your own value
41934 1 / Connect to example.com
41935 \ "https://example.com"
41936 url> https://example.com/remote.php/webdav/
41937 Name of the WebDAV site/service/software you are using
41938 Choose a number from below, or type in your own value
41939 1 / Nextcloud
41940 \ "nextcloud"
41941 2 / Owncloud
41942 \ "owncloud"
41943 3 / Sharepoint Online, authenticated by Microsoft account.
41944 \ "sharepoint"
41945 4 / Sharepoint with NTLM authentication. Usually self-hosted or on-premises.
41946 \ "sharepoint-ntlm"
41947 5 / Other site/service or software
41948 \ "other"
41949 vendor> 1
41950 User name
41951 user> user
41952 Password.
41953 y) Yes type in my own password
41954 g) Generate random password
41955 n) No leave this optional password blank
41956 y/g/n> y
41957 Enter the password:
41958 password:
41959 Confirm the password:
41960 password:
41961 Bearer token instead of user/pass (e.g. a Macaroon)
41962 bearer_token>
41963 Remote config
41964 --------------------
41965 [remote]
41966 type = webdav
41967 url = https://example.com/remote.php/webdav/
41968 vendor = nextcloud
41969 user = user
41970 pass = *** ENCRYPTED ***
41971 bearer_token =
41972 --------------------
41973 y) Yes this is OK
41974 e) Edit this remote
41975 d) Delete this remote
41976 y/e/d> y
41977
41978 Once configured you can then use rclone like this,
41979
41980 List directories in top level of your WebDAV
41981
41982 rclone lsd remote:
41983
41984 List all the files in your WebDAV
41985
41986 rclone ls remote:
41987
41988 To copy a local directory to an WebDAV directory called backup
41989
41990 rclone copy /home/source remote:backup
41991
41992 Modified time and hashes
41993 Plain WebDAV does not support modified times. However when used with
41994 Owncloud or Nextcloud rclone will support modified times.
41995
41996 Likewise plain WebDAV does not support hashes, however when used with
41997 Owncloud or Nextcloud rclone will support SHA1 and MD5 hashes. Depend‐
41998 ing on the exact version of Owncloud or Nextcloud hashes may appear on
41999 all objects, or only on objects which had a hash uploaded with them.
42000
42001 Standard options
42002 Here are the Standard options specific to webdav (WebDAV).
42003
42004 --webdav-url
42005 URL of http host to connect to.
42006
42007 E.g. https://example.com.
42008
42009 Properties:
42010
42011 • Config: url
42012
42013 • Env Var: RCLONE_WEBDAV_URL
42014
42015 • Type: string
42016
42017 • Required: true
42018
42019 --webdav-vendor
42020 Name of the WebDAV site/service/software you are using.
42021
42022 Properties:
42023
42024 • Config: vendor
42025
42026 • Env Var: RCLONE_WEBDAV_VENDOR
42027
42028 • Type: string
42029
42030 • Required: false
42031
42032 • Examples:
42033
42034 • "nextcloud"
42035
42036 • Nextcloud
42037
42038 • "owncloud"
42039
42040 • Owncloud
42041
42042 • "sharepoint"
42043
42044 • Sharepoint Online, authenticated by Microsoft account
42045
42046 • "sharepoint-ntlm"
42047
42048 • Sharepoint with NTLM authentication, usually self-hosted or on-
42049 premises
42050
42051 • "other"
42052
42053 • Other site/service or software
42054
42055 --webdav-user
42056 User name.
42057
42058 In case NTLM authentication is used, the username should be in the for‐
42059 mat 'Domain'.
42060
42061 Properties:
42062
42063 • Config: user
42064
42065 • Env Var: RCLONE_WEBDAV_USER
42066
42067 • Type: string
42068
42069 • Required: false
42070
42071 --webdav-pass
42072 Password.
42073
42074 NB Input to this must be obscured - see rclone obscure
42075 (https://rclone.org/commands/rclone_obscure/).
42076
42077 Properties:
42078
42079 • Config: pass
42080
42081 • Env Var: RCLONE_WEBDAV_PASS
42082
42083 • Type: string
42084
42085 • Required: false
42086
42087 --webdav-bearer-token
42088 Bearer token instead of user/pass (e.g. a Macaroon).
42089
42090 Properties:
42091
42092 • Config: bearer_token
42093
42094 • Env Var: RCLONE_WEBDAV_BEARER_TOKEN
42095
42096 • Type: string
42097
42098 • Required: false
42099
42100 Advanced options
42101 Here are the Advanced options specific to webdav (WebDAV).
42102
42103 --webdav-bearer-token-command
42104 Command to run to get a bearer token.
42105
42106 Properties:
42107
42108 • Config: bearer_token_command
42109
42110 • Env Var: RCLONE_WEBDAV_BEARER_TOKEN_COMMAND
42111
42112 • Type: string
42113
42114 • Required: false
42115
42116 --webdav-encoding
42117 The encoding for the backend.
42118
42119 See the encoding section in the overview (https://rclone.org/over‐
42120 view/#encoding) for more info.
42121
42122 Default encoding is Slash,LtGt,DoubleQuote,Colon,Question,Aster‐
42123 isk,Pipe,Hash,Percent,BackSlash,Del,Ctl,LeftSpace,Left‐
42124 Tilde,RightSpace,RightPeriod,InvalidUtf8 for sharepoint-ntlm or identi‐
42125 ty otherwise.
42126
42127 Properties:
42128
42129 • Config: encoding
42130
42131 • Env Var: RCLONE_WEBDAV_ENCODING
42132
42133 • Type: string
42134
42135 • Required: false
42136
42137 --webdav-headers
42138 Set HTTP headers for all transactions.
42139
42140 Use this to set additional HTTP headers for all transactions
42141
42142 The input format is comma separated list of key,value pairs. Standard
42143 CSV encoding (https://godoc.org/encoding/csv) may be used.
42144
42145 For example, to set a Cookie use 'Cookie,name=value', or '"Cook‐
42146 ie","name=value"'.
42147
42148 You can set multiple headers, e.g. '"Cookie","name=value","Authoriza‐
42149 tion","xxx"'.
42150
42151 Properties:
42152
42153 • Config: headers
42154
42155 • Env Var: RCLONE_WEBDAV_HEADERS
42156
42157 • Type: CommaSepList
42158
42159 • Default:
42160
42161 Provider notes
42162 See below for notes on specific providers.
42163
42164 Owncloud
42165 Click on the settings cog in the bottom right of the page and this will
42166 show the WebDAV URL that rclone needs in the config step. It will look
42167 something like https://example.com/remote.php/webdav/.
42168
42169 Owncloud supports modified times using the X-OC-Mtime header.
42170
42171 Nextcloud
42172 This is configured in an identical way to Owncloud. Note that
42173 Nextcloud initially did not support streaming of files (rcat) whereas
42174 Owncloud did, but this (https://github.com/nextcloud/nextcloud-snap/is‐
42175 sues/365) seems to be fixed as of 2020-11-27 (tested with rclone
42176 v1.53.1 and Nextcloud Server v19).
42177
42178 Sharepoint Online
42179 Rclone can be used with Sharepoint provided by OneDrive for Business or
42180 Office365 Education Accounts. This feature is only needed for a few of
42181 these Accounts, mostly Office365 Education ones. These accounts are
42182 sometimes not verified by the domain owner github#1975
42183 (https://github.com/rclone/rclone/issues/1975)
42184
42185 This means that these accounts can't be added using the official API
42186 (other Accounts should work with the "onedrive" option). However, it
42187 is possible to access them using webdav.
42188
42189 To use a sharepoint remote with rclone, add it like this: First, you
42190 need to get your remote's URL:
42191
42192 • Go here (https://onedrive.live.com/about/en-us/signin/) to open your
42193 OneDrive or to sign in
42194
42195 • Now take a look at your address bar, the URL should look like this:
42196 https://[YOUR-DOMAIN]-my.sharepoint.com/personal/[YOUR-EMAIL]/_lay‐
42197 outs/15/onedrive.aspx
42198
42199 You'll only need this URL up to the email address. After that, you'll
42200 most likely want to add "/Documents". That subdirectory contains the
42201 actual data stored on your OneDrive.
42202
42203 Add the remote to rclone like this: Configure the url as https://[YOUR-
42204 DOMAIN]-my.sharepoint.com/personal/[YOUR-EMAIL]/Documents and use your
42205 normal account email and password for user and pass. If you have 2FA
42206 enabled, you have to generate an app password. Set the vendor to
42207 sharepoint.
42208
42209 Your config file should look like this:
42210
42211 [sharepoint]
42212 type = webdav
42213 url = https://[YOUR-DOMAIN]-my.sharepoint.com/personal/[YOUR-EMAIL]/Documents
42214 vendor = sharepoint
42215 user = YourEmailAddress
42216 pass = encryptedpassword
42217
42218 Sharepoint with NTLM Authentication
42219 Use this option in case your (hosted) Sharepoint is not tied to
42220 OneDrive accounts and uses NTLM authentication.
42221
42222 To get the url configuration, similarly to the above, first navigate to
42223 the desired directory in your browser to get the URL, then strip every‐
42224 thing after the name of the opened directory.
42225
42226 Example: If the URL is: https://example.sharepoint.com/sites/12345/Doc‐
42227 uments/Forms/AllItems.aspx
42228
42229 The configuration to use would be: https://example.share‐
42230 point.com/sites/12345/Documents
42231
42232 Set the vendor to sharepoint-ntlm.
42233
42234 NTLM uses domain and user name combination for authentication, set user
42235 to DOMAIN\username.
42236
42237 Your config file should look like this:
42238
42239 [sharepoint]
42240 type = webdav
42241 url = https://[YOUR-DOMAIN]/some-path-to/Documents
42242 vendor = sharepoint-ntlm
42243 user = DOMAIN\user
42244 pass = encryptedpassword
42245
42246 Required Flags for SharePoint
42247 As SharePoint does some special things with uploaded documents, you
42248 won't be able to use the documents size or the documents hash to com‐
42249 pare if a file has been changed since the upload / which file is newer.
42250
42251 For Rclone calls copying files (especially Office files such as .docx,
42252 .xlsx, etc.) from/to SharePoint (like copy, sync, etc.), you should ap‐
42253 pend these flags to ensure Rclone uses the "Last Modified" datetime
42254 property to compare your documents:
42255
42256 --ignore-size --ignore-checksum --update
42257
42258 dCache
42259 dCache is a storage system that supports many protocols and authentica‐
42260 tion/authorisation schemes. For WebDAV clients, it allows users to au‐
42261 thenticate with username and password (BASIC), X.509, Kerberos, and
42262 various bearer tokens, including Macaroons (https://www.dcache.org/man‐
42263 uals/workshop-2017-05-29-Umea/000-Final/anupam_macaroons_v02.pdf) and
42264 OpenID-Connect (https://en.wikipedia.org/wiki/OpenID_Connect) access
42265 tokens.
42266
42267 Configure as normal using the other type. Don't enter a username or
42268 password, instead enter your Macaroon as the bearer_token.
42269
42270 The config will end up looking something like this.
42271
42272 [dcache]
42273 type = webdav
42274 url = https://dcache...
42275 vendor = other
42276 user =
42277 pass =
42278 bearer_token = your-macaroon
42279
42280 There is a script (https://github.com/sara-nl/GridScripts/blob/mas‐
42281 ter/get-macaroon) that obtains a Macaroon from a dCache WebDAV end‐
42282 point, and creates an rclone config file.
42283
42284 Macaroons may also be obtained from the dCacheView web-browser/Java‐
42285 Script client that comes with dCache.
42286
42287 OpenID-Connect
42288 dCache also supports authenticating with OpenID-Connect access tokens.
42289 OpenID-Connect is a protocol (based on OAuth 2.0) that allows services
42290 to identify users who have authenticated with some central service.
42291
42292 Support for OpenID-Connect in rclone is currently achieved using anoth‐
42293 er software package called oidc-agent (https://github.com/indigo-
42294 dc/oidc-agent). This is a command-line tool that facilitates obtaining
42295 an access token. Once installed and configured, an access token is ob‐
42296 tained by running the oidc-token command. The following example shows
42297 a (shortened) access token obtained from the XDC OIDC Provider.
42298
42299 paul@celebrimbor:~$ oidc-token XDC
42300 eyJraWQ[...]QFXDt0
42301 paul@celebrimbor:~$
42302
42303 Note Before the oidc-token command will work, the refresh token must be
42304 loaded into the oidc agent. This is done with the oidc-add command
42305 (e.g., oidc-add XDC). This is typically done once per login session.
42306 Full details on this and how to register oidc-agent with your OIDC
42307 Provider are provided in the oidc-agent documentation (https://indigo-
42308 dc.gitbooks.io/oidc-agent/).
42309
42310 The rclone bearer_token_command configuration option is used to fetch
42311 the access token from oidc-agent.
42312
42313 Configure as a normal WebDAV endpoint, using the 'other' vendor, leav‐
42314 ing the username and password empty. When prompted, choose to edit the
42315 advanced config and enter the command to get a bearer token (e.g.,
42316 oidc-agent XDC).
42317
42318 The following example config shows a WebDAV endpoint that uses oidc-
42319 agent to supply an access token from the XDC OIDC Provider.
42320
42321 [dcache]
42322 type = webdav
42323 url = https://dcache.example.org/
42324 vendor = other
42325 bearer_token_command = oidc-token XDC
42326
42328 Yandex Disk (https://disk.yandex.com) is a cloud storage solution cre‐
42329 ated by Yandex (https://yandex.com).
42330
42331 Configuration
42332 Here is an example of making a yandex configuration. First run
42333
42334 rclone config
42335
42336 This will guide you through an interactive setup process:
42337
42338 No remotes found, make a new one?
42339 n) New remote
42340 s) Set configuration password
42341 n/s> n
42342 name> remote
42343 Type of storage to configure.
42344 Choose a number from below, or type in your own value
42345 [snip]
42346 XX / Yandex Disk
42347 \ "yandex"
42348 [snip]
42349 Storage> yandex
42350 Yandex Client Id - leave blank normally.
42351 client_id>
42352 Yandex Client Secret - leave blank normally.
42353 client_secret>
42354 Remote config
42355 Use auto config?
42356 * Say Y if not sure
42357 * Say N if you are working on a remote or headless machine
42358 y) Yes
42359 n) No
42360 y/n> y
42361 If your browser doesn't open automatically go to the following link: http://127.0.0.1:53682/auth
42362 Log in and authorize rclone for access
42363 Waiting for code...
42364 Got code
42365 --------------------
42366 [remote]
42367 client_id =
42368 client_secret =
42369 token = {"access_token":"xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx","token_type":"OAuth","expiry":"2016-12-29T12:27:11.362788025Z"}
42370 --------------------
42371 y) Yes this is OK
42372 e) Edit this remote
42373 d) Delete this remote
42374 y/e/d> y
42375
42376 See the remote setup docs (https://rclone.org/remote_setup/) for how to
42377 set it up on a machine with no Internet browser available.
42378
42379 Note that rclone runs a webserver on your local machine to collect the
42380 token as returned from Yandex Disk. This only runs from the moment it
42381 opens your browser to the moment you get back the verification code.
42382 This is on http://127.0.0.1:53682/ and this it may require you to un‐
42383 block it temporarily if you are running a host firewall.
42384
42385 Once configured you can then use rclone like this,
42386
42387 See top level directories
42388
42389 rclone lsd remote:
42390
42391 Make a new directory
42392
42393 rclone mkdir remote:directory
42394
42395 List the contents of a directory
42396
42397 rclone ls remote:directory
42398
42399 Sync /home/local/directory to the remote path, deleting any excess
42400 files in the path.
42401
42402 rclone sync -i /home/local/directory remote:directory
42403
42404 Yandex paths may be as deep as required, e.g. remote:directory/subdi‐
42405 rectory.
42406
42407 Modified time
42408 Modified times are supported and are stored accurate to 1 ns in custom
42409 metadata called rclone_modified in RFC3339 with nanoseconds format.
42410
42411 MD5 checksums
42412 MD5 checksums are natively supported by Yandex Disk.
42413
42414 Emptying Trash
42415 If you wish to empty your trash you can use the rclone cleanup remote:
42416 command which will permanently delete all your trashed files. This
42417 command does not take any path arguments.
42418
42419 Quota information
42420 To view your current quota you can use the rclone about remote: command
42421 which will display your usage limit (quota) and the current usage.
42422
42423 Restricted filename characters
42424 The default restricted characters set (https://rclone.org/overview/#re‐
42425 stricted-characters) are replaced.
42426
42427 Invalid UTF-8 bytes will also be replaced (https://rclone.org/over‐
42428 view/#invalid-utf8), as they can't be used in JSON strings.
42429
42430 Standard options
42431 Here are the Standard options specific to yandex (Yandex Disk).
42432
42433 --yandex-client-id
42434 OAuth Client Id.
42435
42436 Leave blank normally.
42437
42438 Properties:
42439
42440 • Config: client_id
42441
42442 • Env Var: RCLONE_YANDEX_CLIENT_ID
42443
42444 • Type: string
42445
42446 • Required: false
42447
42448 --yandex-client-secret
42449 OAuth Client Secret.
42450
42451 Leave blank normally.
42452
42453 Properties:
42454
42455 • Config: client_secret
42456
42457 • Env Var: RCLONE_YANDEX_CLIENT_SECRET
42458
42459 • Type: string
42460
42461 • Required: false
42462
42463 Advanced options
42464 Here are the Advanced options specific to yandex (Yandex Disk).
42465
42466 --yandex-token
42467 OAuth Access Token as a JSON blob.
42468
42469 Properties:
42470
42471 • Config: token
42472
42473 • Env Var: RCLONE_YANDEX_TOKEN
42474
42475 • Type: string
42476
42477 • Required: false
42478
42479 --yandex-auth-url
42480 Auth server URL.
42481
42482 Leave blank to use the provider defaults.
42483
42484 Properties:
42485
42486 • Config: auth_url
42487
42488 • Env Var: RCLONE_YANDEX_AUTH_URL
42489
42490 • Type: string
42491
42492 • Required: false
42493
42494 --yandex-token-url
42495 Token server url.
42496
42497 Leave blank to use the provider defaults.
42498
42499 Properties:
42500
42501 • Config: token_url
42502
42503 • Env Var: RCLONE_YANDEX_TOKEN_URL
42504
42505 • Type: string
42506
42507 • Required: false
42508
42509 --yandex-hard-delete
42510 Delete files permanently rather than putting them into the trash.
42511
42512 Properties:
42513
42514 • Config: hard_delete
42515
42516 • Env Var: RCLONE_YANDEX_HARD_DELETE
42517
42518 • Type: bool
42519
42520 • Default: false
42521
42522 --yandex-encoding
42523 The encoding for the backend.
42524
42525 See the encoding section in the overview (https://rclone.org/over‐
42526 view/#encoding) for more info.
42527
42528 Properties:
42529
42530 • Config: encoding
42531
42532 • Env Var: RCLONE_YANDEX_ENCODING
42533
42534 • Type: MultiEncoder
42535
42536 • Default: Slash,Del,Ctl,InvalidUtf8,Dot
42537
42538 Limitations
42539 When uploading very large files (bigger than about 5 GiB) you will need
42540 to increase the --timeout parameter. This is because Yandex pauses
42541 (perhaps to calculate the MD5SUM for the entire file) before returning
42542 confirmation that the file has been uploaded. The default handling of
42543 timeouts in rclone is to assume a 5 minute pause is an error and close
42544 the connection - you'll see net/http: timeout awaiting response headers
42545 errors in the logs if this is happening. Setting the timeout to twice
42546 the max size of file in GiB should be enough, so if you want to upload
42547 a 30 GiB file set a timeout of 2 * 30 = 60m, that is --timeout 60m.
42548
42549 Having a Yandex Mail account is mandatory to use the Yandex.Disk sub‐
42550 scription. Token generation will work without a mail account, but
42551 Rclone won't be able to complete any actions.
42552
42553 [403 - DiskUnsupportedUserAccountTypeError] User account type is not supported.
42554
42556 Zoho WorkDrive (https://www.zoho.com/workdrive/) is a cloud storage so‐
42557 lution created by Zoho (https://zoho.com).
42558
42559 Configuration
42560 Here is an example of making a zoho configuration. First run
42561
42562 rclone config
42563
42564 This will guide you through an interactive setup process:
42565
42566 No remotes found, make a new one?
42567 n) New remote
42568 s) Set configuration password
42569 n/s> n
42570 name> remote
42571 Type of storage to configure.
42572 Enter a string value. Press Enter for the default ("").
42573 Choose a number from below, or type in your own value
42574 [snip]
42575 XX / Zoho
42576 \ "zoho"
42577 [snip]
42578 Storage> zoho
42579 ** See help for zoho backend at: https://rclone.org/zoho/ **
42580
42581 OAuth Client Id
42582 Leave blank normally.
42583 Enter a string value. Press Enter for the default ("").
42584 client_id>
42585 OAuth Client Secret
42586 Leave blank normally.
42587 Enter a string value. Press Enter for the default ("").
42588 client_secret>
42589 Edit advanced config? (y/n)
42590 y) Yes
42591 n) No (default)
42592 y/n> n
42593 Remote config
42594 Use auto config?
42595 * Say Y if not sure
42596 * Say N if you are working on a remote or headless machine
42597 y) Yes (default)
42598 n) No
42599 y/n>
42600 If your browser doesn't open automatically go to the following link: http://127.0.0.1:53682/auth?state=LVn0IHzxej1ZkmQw31d0wQ
42601 Log in and authorize rclone for access
42602 Waiting for code...
42603 Got code
42604 Choose a number from below, or type in your own value
42605 1 / MyTeam
42606 \ "4u28602177065ff22426787a6745dba8954eb"
42607 Enter a Team ID> 1
42608 Choose a number from below, or type in your own value
42609 1 / General
42610 \ "4u2869d2aa6fca04f4f2f896b6539243b85b1"
42611 Enter a Workspace ID> 1
42612 --------------------
42613 [remote]
42614 type = zoho
42615 token = {"access_token":"xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx","token_type":"Zoho-oauthtoken","refresh_token":"xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx","expiry":"2020-10-12T00:54:52.370275223+02:00"}
42616 root_folder_id = xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
42617 --------------------
42618 y) Yes this is OK (default)
42619 e) Edit this remote
42620 d) Delete this remote
42621 y/e/d>
42622
42623 See the remote setup docs (https://rclone.org/remote_setup/) for how to
42624 set it up on a machine with no Internet browser available.
42625
42626 Rclone runs a webserver on your local computer to collect the autho‐
42627 rization token from Zoho Workdrive. This is only from the moment your
42628 browser is opened until the token is returned. The webserver runs on
42629 http://127.0.0.1:53682/. If local port 53682 is protected by a fire‐
42630 wall you may need to temporarily unblock the firewall to complete au‐
42631 thorization.
42632
42633 Once configured you can then use rclone like this,
42634
42635 See top level directories
42636
42637 rclone lsd remote:
42638
42639 Make a new directory
42640
42641 rclone mkdir remote:directory
42642
42643 List the contents of a directory
42644
42645 rclone ls remote:directory
42646
42647 Sync /home/local/directory to the remote path, deleting any excess
42648 files in the path.
42649
42650 rclone sync -i /home/local/directory remote:directory
42651
42652 Zoho paths may be as deep as required, eg remote:directory/subdirecto‐
42653 ry.
42654
42655 Modified time
42656 Modified times are currently not supported for Zoho Workdrive
42657
42658 Checksums
42659 No checksums are supported.
42660
42661 Usage information
42662 To view your current quota you can use the rclone about remote: command
42663 which will display your current usage.
42664
42665 Restricted filename characters
42666 Only control characters and invalid UTF-8 are replaced. In addition
42667 most Unicode full-width characters are not supported at all and will be
42668 removed from filenames during upload.
42669
42670 Standard options
42671 Here are the Standard options specific to zoho (Zoho).
42672
42673 --zoho-client-id
42674 OAuth Client Id.
42675
42676 Leave blank normally.
42677
42678 Properties:
42679
42680 • Config: client_id
42681
42682 • Env Var: RCLONE_ZOHO_CLIENT_ID
42683
42684 • Type: string
42685
42686 • Required: false
42687
42688 --zoho-client-secret
42689 OAuth Client Secret.
42690
42691 Leave blank normally.
42692
42693 Properties:
42694
42695 • Config: client_secret
42696
42697 • Env Var: RCLONE_ZOHO_CLIENT_SECRET
42698
42699 • Type: string
42700
42701 • Required: false
42702
42703 --zoho-region
42704 Zoho region to connect to.
42705
42706 You'll have to use the region your organization is registered in. If
42707 not sure use the same top level domain as you connect to in your brows‐
42708 er.
42709
42710 Properties:
42711
42712 • Config: region
42713
42714 • Env Var: RCLONE_ZOHO_REGION
42715
42716 • Type: string
42717
42718 • Required: false
42719
42720 • Examples:
42721
42722 • "com"
42723
42724 • United states / Global
42725
42726 • "eu"
42727
42728 • Europe
42729
42730 • "in"
42731
42732 • India
42733
42734 • "jp"
42735
42736 • Japan
42737
42738 • "com.cn"
42739
42740 • China
42741
42742 • "com.au"
42743
42744 • Australia
42745
42746 Advanced options
42747 Here are the Advanced options specific to zoho (Zoho).
42748
42749 --zoho-token
42750 OAuth Access Token as a JSON blob.
42751
42752 Properties:
42753
42754 • Config: token
42755
42756 • Env Var: RCLONE_ZOHO_TOKEN
42757
42758 • Type: string
42759
42760 • Required: false
42761
42762 --zoho-auth-url
42763 Auth server URL.
42764
42765 Leave blank to use the provider defaults.
42766
42767 Properties:
42768
42769 • Config: auth_url
42770
42771 • Env Var: RCLONE_ZOHO_AUTH_URL
42772
42773 • Type: string
42774
42775 • Required: false
42776
42777 --zoho-token-url
42778 Token server url.
42779
42780 Leave blank to use the provider defaults.
42781
42782 Properties:
42783
42784 • Config: token_url
42785
42786 • Env Var: RCLONE_ZOHO_TOKEN_URL
42787
42788 • Type: string
42789
42790 • Required: false
42791
42792 --zoho-encoding
42793 The encoding for the backend.
42794
42795 See the encoding section in the overview (https://rclone.org/over‐
42796 view/#encoding) for more info.
42797
42798 Properties:
42799
42800 • Config: encoding
42801
42802 • Env Var: RCLONE_ZOHO_ENCODING
42803
42804 • Type: MultiEncoder
42805
42806 • Default: Del,Ctl,InvalidUtf8
42807
42808 Setting up your own client_id
42809 For Zoho we advise you to set up your own client_id. To do so you have
42810 to complete the following steps.
42811
42812 1. Log in to the Zoho API Console (https://api-console.zoho.com)
42813
42814 2. Create a new client of type "Server-based Application". The name
42815 and website don't matter, but you must add the redirect URL
42816 http://localhost:53682/.
42817
42818 3. Once the client is created, you can go to the settings tab and en‐
42819 able it in other regions.
42820
42821 The client id and client secret can now be used with rclone.
42822
42824 Local paths are specified as normal filesystem paths, e.g.
42825 /path/to/wherever, so
42826
42827 rclone sync -i /home/source /tmp/destination
42828
42829 Will sync /home/source to /tmp/destination.
42830
42831 Configuration
42832 For consistencies sake one can also configure a remote of type local in
42833 the config file, and access the local filesystem using rclone remote
42834 paths, e.g. remote:path/to/wherever, but it is probably easier not to.
42835
42836 Modified time
42837 Rclone reads and writes the modified time using an accuracy determined
42838 by the OS. Typically this is 1ns on Linux, 10 ns on Windows and 1 Sec‐
42839 ond on OS X.
42840
42841 Filenames
42842 Filenames should be encoded in UTF-8 on disk. This is the normal case
42843 for Windows and OS X.
42844
42845 There is a bit more uncertainty in the Linux world, but new distribu‐
42846 tions will have UTF-8 encoded files names. If you are using an old
42847 Linux filesystem with non UTF-8 file names (e.g. latin1) then you can
42848 use the convmv tool to convert the filesystem to UTF-8. This tool is
42849 available in most distributions' package managers.
42850
42851 If an invalid (non-UTF8) filename is read, the invalid characters will
42852 be replaced with a quoted representation of the invalid bytes. The
42853 name gro\xdf will be transferred as gro‛DF. rclone will emit a debug
42854 message in this case (use -v to see), e.g.
42855
42856 Local file system at .: Replacing invalid UTF-8 characters in "gro\xdf"
42857
42858 Restricted characters
42859 With the local backend, restrictions on the characters that are usable
42860 in file or directory names depend on the operating system. To check
42861 what rclone will replace by default on your system, run rclone help
42862 flags local-encoding.
42863
42864 On non Windows platforms the following characters are replaced when
42865 handling file names.
42866
42867 Character Value Replacement
42868 ────────────────────────────────
42869 NUL 0x00 ␀
42870 / 0x2F /
42871
42872 When running on Windows the following characters are replaced. This
42873 list is based on the Windows file naming conventions (https://docs.mi‐
42874 crosoft.com/de-de/windows/desktop/FileIO/naming-a-file#naming-conven‐
42875 tions).
42876
42877 Character Value Replacement
42878 ────────────────────────────────
42879 NUL 0x00 ␀
42880 SOH 0x01 ␁
42881 STX 0x02 ␂
42882 ETX 0x03 ␃
42883 EOT 0x04 ␄
42884 ENQ 0x05 ␅
42885 ACK 0x06 ␆
42886 BEL 0x07 ␇
42887 BS 0x08 ␈
42888 HT 0x09 ␉
42889 LF 0x0A ␊
42890 VT 0x0B ␋
42891 FF 0x0C ␌
42892 CR 0x0D ␍
42893 SO 0x0E ␎
42894 SI 0x0F ␏
42895 DLE 0x10 ␐
42896 DC1 0x11 ␑
42897 DC2 0x12 ␒
42898 DC3 0x13 ␓
42899 DC4 0x14 ␔
42900 NAK 0x15 ␕
42901 SYN 0x16 ␖
42902 ETB 0x17 ␗
42903 CAN 0x18 ␘
42904 EM 0x19 ␙
42905 SUB 0x1A ␚
42906 ESC 0x1B ␛
42907 FS 0x1C ␜
42908 GS 0x1D ␝
42909 RS 0x1E ␞
42910 US 0x1F ␟
42911 / 0x2F /
42912 " 0x22 "
42913 * 0x2A *
42914 : 0x3A :
42915 < 0x3C <
42916 > 0x3E >
42917 ? 0x3F ?
42918 \ 0x5C \
42919 | 0x7C |
42920
42921 File names on Windows can also not end with the following characters.
42922 These only get replaced if they are the last character in the name:
42923
42924 Character Value Replacement
42925 ────────────────────────────────
42926 SP 0x20 ␠
42927 . 0x2E .
42928
42929 Invalid UTF-8 bytes will also be replaced (https://rclone.org/over‐
42930 view/#invalid-utf8), as they can't be converted to UTF-16.
42931
42932 Paths on Windows
42933 On Windows there are many ways of specifying a path to a file system
42934 resource. Local paths can be absolute, like C:\path\to\wherever, or
42935 relative, like ..\wherever. Network paths in UNC format, \\serv‐
42936 er\share, are also supported. Path separator can be either \ (as in
42937 C:\path\to\wherever) or / (as in C:/path/to/wherever). Length of these
42938 paths are limited to 259 characters for files and 247 characters for
42939 directories, but there is an alternative extended-length path format
42940 increasing the limit to (approximately) 32,767 characters. This format
42941 requires absolute paths and the use of prefix \\?\, e.g.
42942 \\?\D:\some\very\long\path. For convenience rclone will automatically
42943 convert regular paths into the corresponding extended-length paths, so
42944 in most cases you do not have to worry about this (read more below).
42945
42946 Note that Windows supports using the same prefix \\?\ to specify path
42947 to volumes identified by their GUID, e.g. \\?\Vol‐
42948 ume{b75e2c83-0000-0000-0000-602f00000000}\some\path. This is not sup‐
42949 ported in rclone, due to an issue (https://github.com/golang/go/is‐
42950 sues/39785) in go.
42951
42952 Long paths
42953 Rclone handles long paths automatically, by converting all paths to ex‐
42954 tended-length path format (https://docs.microsoft.com/en-us/win‐
42955 dows/win32/fileio/maximum-file-path-limitation), which allows paths up
42956 to 32,767 characters.
42957
42958 This conversion will ensure paths are absolute and prefix them with the
42959 \\?\. This is why you will see that your paths, for instance .\files
42960 is shown as path \\?\C:\files in the output, and \\server\share as
42961 \\?\UNC\server\share.
42962
42963 However, in rare cases this may cause problems with buggy file system
42964 drivers like EncFS (https://github.com/rclone/rclone/issues/261). To
42965 disable UNC conversion globally, add this to your .rclone.conf file:
42966
42967 [local]
42968 nounc = true
42969
42970 If you want to selectively disable UNC, you can add it to a separate
42971 entry like this:
42972
42973 [nounc]
42974 type = local
42975 nounc = true
42976
42977 And use rclone like this:
42978
42979 rclone copy c:\src nounc:z:\dst
42980
42981 This will use UNC paths on c:\src but not on z:\dst. Of course this
42982 will cause problems if the absolute path length of a file exceeds 259
42983 characters on z, so only use this option if you have to.
42984
42985 Symlinks / Junction points
42986 Normally rclone will ignore symlinks or junction points (which behave
42987 like symlinks under Windows).
42988
42989 If you supply --copy-links or -L then rclone will follow the symlink
42990 and copy the pointed to file or directory. Note that this flag is in‐
42991 compatible with --links / -l.
42992
42993 This flag applies to all commands.
42994
42995 For example, supposing you have a directory structure like this
42996
42997 $ tree /tmp/a
42998 /tmp/a
42999 ├── b -> ../b
43000 ├── expected -> ../expected
43001 ├── one
43002 └── two
43003 └── three
43004
43005 Then you can see the difference with and without the flag like this
43006
43007 $ rclone ls /tmp/a
43008 6 one
43009 6 two/three
43010
43011 and
43012
43013 $ rclone -L ls /tmp/a
43014 4174 expected
43015 6 one
43016 6 two/three
43017 6 b/two
43018 6 b/one
43019
43020 --links, -l
43021 Normally rclone will ignore symlinks or junction points (which behave
43022 like symlinks under Windows).
43023
43024 If you supply this flag then rclone will copy symbolic links from the
43025 local storage, and store them as text files, with a '.rclonelink' suf‐
43026 fix in the remote storage.
43027
43028 The text file will contain the target of the symbolic link (see exam‐
43029 ple).
43030
43031 This flag applies to all commands.
43032
43033 For example, supposing you have a directory structure like this
43034
43035 $ tree /tmp/a
43036 /tmp/a
43037 ├── file1 -> ./file4
43038 └── file2 -> /home/user/file3
43039
43040 Copying the entire directory with '-l'
43041
43042 $ rclone copyto -l /tmp/a/file1 remote:/tmp/a/
43043
43044 The remote files are created with a '.rclonelink' suffix
43045
43046 $ rclone ls remote:/tmp/a
43047 5 file1.rclonelink
43048 14 file2.rclonelink
43049
43050 The remote files will contain the target of the symbolic links
43051
43052 $ rclone cat remote:/tmp/a/file1.rclonelink
43053 ./file4
43054
43055 $ rclone cat remote:/tmp/a/file2.rclonelink
43056 /home/user/file3
43057
43058 Copying them back with '-l'
43059
43060 $ rclone copyto -l remote:/tmp/a/ /tmp/b/
43061
43062 $ tree /tmp/b
43063 /tmp/b
43064 ├── file1 -> ./file4
43065 └── file2 -> /home/user/file3
43066
43067 However, if copied back without '-l'
43068
43069 $ rclone copyto remote:/tmp/a/ /tmp/b/
43070
43071 $ tree /tmp/b
43072 /tmp/b
43073 ├── file1.rclonelink
43074 └── file2.rclonelink
43075
43076 Note that this flag is incompatible with -copy-links / -L.
43077
43078 Restricting filesystems with --one-file-system
43079 Normally rclone will recurse through filesystems as mounted.
43080
43081 However if you set --one-file-system or -x this tells rclone to stay in
43082 the filesystem specified by the root and not to recurse into different
43083 file systems.
43084
43085 For example if you have a directory hierarchy like this
43086
43087 root
43088 ├── disk1 - disk1 mounted on the root
43089 │ └── file3 - stored on disk1
43090 ├── disk2 - disk2 mounted on the root
43091 │ └── file4 - stored on disk12
43092 ├── file1 - stored on the root disk
43093 └── file2 - stored on the root disk
43094
43095 Using rclone --one-file-system copy root remote: will only copy file1
43096 and file2. Eg
43097
43098 $ rclone -q --one-file-system ls root
43099 0 file1
43100 0 file2
43101
43102 $ rclone -q ls root
43103 0 disk1/file3
43104 0 disk2/file4
43105 0 file1
43106 0 file2
43107
43108 NB Rclone (like most unix tools such as du, rsync and tar) treats a
43109 bind mount to the same device as being on the same filesystem.
43110
43111 NB This flag is only available on Unix based systems. On systems where
43112 it isn't supported (e.g. Windows) it will be ignored.
43113
43114 Advanced options
43115 Here are the Advanced options specific to local (Local Disk).
43116
43117 --local-nounc
43118 Disable UNC (long path names) conversion on Windows.
43119
43120 Properties:
43121
43122 • Config: nounc
43123
43124 • Env Var: RCLONE_LOCAL_NOUNC
43125
43126 • Type: bool
43127
43128 • Default: false
43129
43130 • Examples:
43131
43132 • "true"
43133
43134 • Disables long file names.
43135
43136 --copy-links / -L
43137 Follow symlinks and copy the pointed to item.
43138
43139 Properties:
43140
43141 • Config: copy_links
43142
43143 • Env Var: RCLONE_LOCAL_COPY_LINKS
43144
43145 • Type: bool
43146
43147 • Default: false
43148
43149 --links / -l
43150 Translate symlinks to/from regular files with a '.rclonelink' exten‐
43151 sion.
43152
43153 Properties:
43154
43155 • Config: links
43156
43157 • Env Var: RCLONE_LOCAL_LINKS
43158
43159 • Type: bool
43160
43161 • Default: false
43162
43163 --skip-links
43164 Don't warn about skipped symlinks.
43165
43166 This flag disables warning messages on skipped symlinks or junction
43167 points, as you explicitly acknowledge that they should be skipped.
43168
43169 Properties:
43170
43171 • Config: skip_links
43172
43173 • Env Var: RCLONE_LOCAL_SKIP_LINKS
43174
43175 • Type: bool
43176
43177 • Default: false
43178
43179 --local-zero-size-links
43180 Assume the Stat size of links is zero (and read them instead) (depre‐
43181 cated).
43182
43183 Rclone used to use the Stat size of links as the link size, but this
43184 fails in quite a few places:
43185
43186 • Windows
43187
43188 • On some virtual filesystems (such ash LucidLink)
43189
43190 • Android
43191
43192 So rclone now always reads the link.
43193
43194 Properties:
43195
43196 • Config: zero_size_links
43197
43198 • Env Var: RCLONE_LOCAL_ZERO_SIZE_LINKS
43199
43200 • Type: bool
43201
43202 • Default: false
43203
43204 --local-unicode-normalization
43205 Apply unicode NFC normalization to paths and filenames.
43206
43207 This flag can be used to normalize file names into unicode NFC form
43208 that are read from the local filesystem.
43209
43210 Rclone does not normally touch the encoding of file names it reads from
43211 the file system.
43212
43213 This can be useful when using macOS as it normally provides decomposed
43214 (NFD) unicode which in some language (eg Korean) doesn't display prop‐
43215 erly on some OSes.
43216
43217 Note that rclone compares filenames with unicode normalization in the
43218 sync routine so this flag shouldn't normally be used.
43219
43220 Properties:
43221
43222 • Config: unicode_normalization
43223
43224 • Env Var: RCLONE_LOCAL_UNICODE_NORMALIZATION
43225
43226 • Type: bool
43227
43228 • Default: false
43229
43230 --local-no-check-updated
43231 Don't check to see if the files change during upload.
43232
43233 Normally rclone checks the size and modification time of files as they
43234 are being uploaded and aborts with a message which starts "can't copy -
43235 source file is being updated" if the file changes during upload.
43236
43237 However on some file systems this modification time check may fail
43238 (e.g. Glusterfs #2206 (https://github.com/rclone/rclone/issues/2206))
43239 so this check can be disabled with this flag.
43240
43241 If this flag is set, rclone will use its best efforts to transfer a
43242 file which is being updated. If the file is only having things append‐
43243 ed to it (e.g. a log) then rclone will transfer the log file with the
43244 size it had the first time rclone saw it.
43245
43246 If the file is being modified throughout (not just appended to) then
43247 the transfer may fail with a hash check failure.
43248
43249 In detail, once the file has had stat() called on it for the first time
43250 we:
43251
43252 • Only transfer the size that stat gave
43253
43254 • Only checksum the size that stat gave
43255
43256 • Don't update the stat info for the file
43257
43258 Properties:
43259
43260 • Config: no_check_updated
43261
43262 • Env Var: RCLONE_LOCAL_NO_CHECK_UPDATED
43263
43264 • Type: bool
43265
43266 • Default: false
43267
43268 --one-file-system / -x
43269 Don't cross filesystem boundaries (unix/macOS only).
43270
43271 Properties:
43272
43273 • Config: one_file_system
43274
43275 • Env Var: RCLONE_LOCAL_ONE_FILE_SYSTEM
43276
43277 • Type: bool
43278
43279 • Default: false
43280
43281 --local-case-sensitive
43282 Force the filesystem to report itself as case sensitive.
43283
43284 Normally the local backend declares itself as case insensitive on Win‐
43285 dows/macOS and case sensitive for everything else. Use this flag to
43286 override the default choice.
43287
43288 Properties:
43289
43290 • Config: case_sensitive
43291
43292 • Env Var: RCLONE_LOCAL_CASE_SENSITIVE
43293
43294 • Type: bool
43295
43296 • Default: false
43297
43298 --local-case-insensitive
43299 Force the filesystem to report itself as case insensitive.
43300
43301 Normally the local backend declares itself as case insensitive on Win‐
43302 dows/macOS and case sensitive for everything else. Use this flag to
43303 override the default choice.
43304
43305 Properties:
43306
43307 • Config: case_insensitive
43308
43309 • Env Var: RCLONE_LOCAL_CASE_INSENSITIVE
43310
43311 • Type: bool
43312
43313 • Default: false
43314
43315 --local-no-preallocate
43316 Disable preallocation of disk space for transferred files.
43317
43318 Preallocation of disk space helps prevent filesystem fragmentation.
43319 However, some virtual filesystem layers (such as Google Drive File
43320 Stream) may incorrectly set the actual file size equal to the preallo‐
43321 cated space, causing checksum and file size checks to fail. Use this
43322 flag to disable preallocation.
43323
43324 Properties:
43325
43326 • Config: no_preallocate
43327
43328 • Env Var: RCLONE_LOCAL_NO_PREALLOCATE
43329
43330 • Type: bool
43331
43332 • Default: false
43333
43334 --local-no-sparse
43335 Disable sparse files for multi-thread downloads.
43336
43337 On Windows platforms rclone will make sparse files when doing multi-
43338 thread downloads. This avoids long pauses on large files where the OS
43339 zeros the file. However sparse files may be undesirable as they cause
43340 disk fragmentation and can be slow to work with.
43341
43342 Properties:
43343
43344 • Config: no_sparse
43345
43346 • Env Var: RCLONE_LOCAL_NO_SPARSE
43347
43348 • Type: bool
43349
43350 • Default: false
43351
43352 --local-no-set-modtime
43353 Disable setting modtime.
43354
43355 Normally rclone updates modification time of files after they are done
43356 uploading. This can cause permissions issues on Linux platforms when
43357 the user rclone is running as does not own the file uploaded, such as
43358 when copying to a CIFS mount owned by another user. If this option is
43359 enabled, rclone will no longer update the modtime after copying a file.
43360
43361 Properties:
43362
43363 • Config: no_set_modtime
43364
43365 • Env Var: RCLONE_LOCAL_NO_SET_MODTIME
43366
43367 • Type: bool
43368
43369 • Default: false
43370
43371 --local-encoding
43372 The encoding for the backend.
43373
43374 See the encoding section in the overview (https://rclone.org/over‐
43375 view/#encoding) for more info.
43376
43377 Properties:
43378
43379 • Config: encoding
43380
43381 • Env Var: RCLONE_LOCAL_ENCODING
43382
43383 • Type: MultiEncoder
43384
43385 • Default: Slash,Dot
43386
43387 Metadata
43388 Depending on which OS is in use the local backend may return only some
43389 of the system metadata. Setting system metadata is supported on all
43390 OSes but setting user metadata is only supported on linux, freebsd,
43391 netbsd, macOS and Solaris. It is not supported on Windows yet (see
43392 pkg/attrs#47 (https://github.com/pkg/xattr/issues/47)).
43393
43394 User metadata is stored as extended attributes (which may not be sup‐
43395 ported by all file systems) under the "user.*" prefix.
43396
43397 Here are the possible system metadata items for the local backend.
43398
43399 Name Help Type Example Read Only
43400 ──────────────────────────────────────────────────────────────────────────────────────────────────────
43401 atime Time of RFC 3339 2006-01-02T15:04:05.999999999Z07:00 N
43402 last access
43403 btime Time of RFC 3339 2006-01-02T15:04:05.999999999Z07:00 N
43404 file birth
43405 (creation)
43406 gid Group ID of decimal 500 N
43407 owner number
43408 mode File type octal, unix 0100664 N
43409 and mode style
43410 mtime Time of RFC 3339 2006-01-02T15:04:05.999999999Z07:00 N
43411 last modi‐
43412 fication
43413 rdev Device ID hexadecimal 1abc N
43414 (if special
43415 file)
43416 uid User ID of decimal 500 N
43417 owner number
43418
43419 See the metadata (https://rclone.org/docs/#metadata) docs for more in‐
43420 fo.
43421
43422 Backend commands
43423 Here are the commands specific to the local backend.
43424
43425 Run them with
43426
43427 rclone backend COMMAND remote:
43428
43429 The help below will explain what arguments each command takes.
43430
43431 See the backend (https://rclone.org/commands/rclone_backend/) command
43432 for more info on how to pass options and arguments.
43433
43434 These can be run on a running backend using the rc command backend/com‐
43435 mand (https://rclone.org/rc/#backend-command).
43436
43437 noop
43438 A null operation for testing backend commands
43439
43440 rclone backend noop remote: [options] [<arguments>+]
43441
43442 This is a test command which has some options you can try to change the
43443 output.
43444
43445 Options:
43446
43447 • "echo": echo the input arguments
43448
43449 • "error": return an error based on option value
43450
43452 v1.60.1 - 2022-11-17
43453 See commits (https://github.com/rclone/rclone/com‐
43454 pare/v1.60.0...v1.60.1)
43455
43456 • Bug Fixes
43457
43458 • lib/cache: Fix alias backend shutting down too soon (Nick Craig-
43459 Wood)
43460
43461 • wasm: Fix walltime link error by adding up-to-date wasm_exec.js
43462 (João Henrique Franco)
43463
43464 • docs
43465
43466 • Update faq.md with bisync (Samuel Johnson)
43467
43468 • Corrected download links in windows install docs (coultonluke)
43469
43470 • Add direct download link for windows arm64 (albertony)
43471
43472 • Remove link to rclone slack as it is no longer supported (Nick
43473 Craig-Wood)
43474
43475 • Faq: how to use a proxy server that requires a username and pass‐
43476 word (asdffdsazqqq)
43477
43478 • Oracle-object-storage: doc fix (Manoj Ghosh)
43479
43480 • Fix typo remove in rclone_serve_restic command (Joda Stößer)
43481
43482 • Fix character that was incorrectly interpreted as markdown
43483 (Clément Notin)
43484
43485 • VFS
43486
43487 • Fix deadlock caused by cache cleaner and upload finishing (Nick
43488 Craig-Wood)
43489
43490 • Local
43491
43492 • Clean absolute paths (albertony)
43493
43494 • Fix -L/--copy-links with filters missing directories (Nick Craig-
43495 Wood)
43496
43497 • Mailru
43498
43499 • Note that an app password is now needed (Nick Craig-Wood)
43500
43501 • Allow timestamps to be before the epoch 1970-01-01 (Nick Craig-
43502 Wood)
43503
43504 • S3
43505
43506 • Add provider quirk --s3-might-gzip to fix corrupted on transfer:
43507 sizes differ (Nick Craig-Wood)
43508
43509 • Allow Storj to server side copy since it seems to work now (Nick
43510 Craig-Wood)
43511
43512 • Fix for unchecked err value in s3 listv2 (Aaron Gokaslan)
43513
43514 • Add additional Wasabi locations (techknowlogick)
43515
43516 • Smb
43517
43518 • Fix Failed to sync: context canceled at the end of syncs (Nick
43519 Craig-Wood)
43520
43521 • WebDAV
43522
43523 • Fix Move/Copy/DirMove when using -server-side-across-configs (Nick
43524 Craig-Wood)
43525
43526 v1.60.0 - 2022-10-21
43527 See commits (https://github.com/rclone/rclone/com‐
43528 pare/v1.59.0...v1.60.0)
43529
43530 • New backends
43531
43532 • Oracle object storage (https://rclone.org/oracleobjectstorage/)
43533 (Manoj Ghosh)
43534
43535 • SMB (https://rclone.org/smb/) / CIFS (Windows file sharing)
43536 (Lesmiscore)
43537
43538 • New S3 providers
43539
43540 • IONOS Cloud Storage (https://rclone.org/s3/#ionos) (Dmitry
43541 Deniskin)
43542
43543 • Qiniu KODO (https://rclone.org/s3/#qiniu) (Bachue Zhou)
43544
43545 • New Features
43546
43547 • build
43548
43549 • Update to go1.19 and make go1.17 the minimum required version
43550 (Nick Craig-Wood)
43551
43552 • Install.sh: fix arm-v7 download (Ole Frost)
43553
43554 • fs: Warn the user when using an existing remote name without a
43555 colon (Nick Craig-Wood)
43556
43557 • httplib: Add --xxx-min-tls-version option to select minimum TLS
43558 version for HTTP servers (Robert Newson)
43559
43560 • librclone: Add PHP bindings and test program (Jordi Gonzalez Muñoz)
43561
43562 • operations
43563
43564 • Add --server-side-across-configs global flag for any backend
43565 (Nick Craig-Wood)
43566
43567 • Optimise --copy-dest and --compare-dest (Nick Craig-Wood)
43568
43569 • rc: add job/stopgroup to stop group (Evan Spensley)
43570
43571 • serve dlna
43572
43573 • Add --announce-interval to control SSDP Announce Interval (Yanc‐
43574 eyChiew)
43575
43576 • Add --interface to Specify SSDP interface names line (Simon Bos)
43577
43578 • Add support for more external subtitles (YanceyChiew)
43579
43580 • Add verification of addresses (YanceyChiew)
43581
43582 • sync: Optimise --copy-dest and --compare-dest (Nick Craig-Wood)
43583
43584 • doc updates (albertony, Alexander Knorr, anonion, João Henrique
43585 Franco, Josh Soref, Lorenzo Milesi, Marco Molteni, Mark Trolley,
43586 Ole Frost, partev, Ryan Morey, Tom Mombourquette, YFdyh000)
43587
43588 • Bug Fixes
43589
43590 • filter
43591
43592 • Fix incorrect filtering with UseFilter context flag and wrapping
43593 backends (Nick Craig-Wood)
43594
43595 • Make sure we check --files-from when looking for a single file
43596 (Nick Craig-Wood)
43597
43598 • rc
43599
43600 • Fix mount/listmounts not returning the full Fs entered in
43601 mount/mount (Tom Mombourquette)
43602
43603 • Handle external unmount when mounting (Isaac Aymerich)
43604
43605 • Validate Daemon option is not set when mounting a volume via RC
43606 (Isaac Aymerich)
43607
43608 • sync: Update docs and error messages to reflect fixes to overlap
43609 checks (Nick Naumann)
43610
43611 • VFS
43612
43613 • Reduce memory use by embedding sync.Cond (Nick Craig-Wood)
43614
43615 • Reduce memory usage by re-ordering commonly used structures (Nick
43616 Craig-Wood)
43617
43618 • Fix excess CPU used by VFS cache cleaner looping (Nick Craig-Wood)
43619
43620 • Local
43621
43622 • Obey file filters in listing to fix errors on excluded files (Nick
43623 Craig-Wood)
43624
43625 • Fix "Failed to read metadata: function not implemented" on old Lin‐
43626 ux kernels (Nick Craig-Wood)
43627
43628 • Compress
43629
43630 • Fix crash due to nil metadata (Nick Craig-Wood)
43631
43632 • Fix error handling to not use or return nil objects (Nick Craig-
43633 Wood)
43634
43635 • Drive
43636
43637 • Make --drive-stop-on-upload-limit obey quota exceeded error (Steve
43638 Kowalik)
43639
43640 • FTP
43641
43642 • Add --ftp-force-list-hidden option to show hidden items (Øyvind
43643 Heddeland Instefjord)
43644
43645 • Fix hang when using ExplicitTLS to certain servers. (Nick Craig-
43646 Wood)
43647
43648 • Google Cloud Storage
43649
43650 • Add --gcs-endpoint flag and config parameter (Nick Craig-Wood)
43651
43652 • Hubic
43653
43654 • Remove backend as service has now shut down (Nick Craig-Wood)
43655
43656 • Onedrive
43657
43658 • Rename Onedrive(cn) 21Vianet to Vnet Group (Yen Hu)
43659
43660 • Disable change notify in China region since it is not supported
43661 (Nick Craig-Wood)
43662
43663 • S3
43664
43665 • Implement --s3-versions flag to show old versions of objects if en‐
43666 abled (Nick Craig-Wood)
43667
43668 • Implement --s3-version-at flag to show versions of objects at a
43669 particular time (Nick Craig-Wood)
43670
43671 • Implement backend versioning command to get/set bucket versioning
43672 (Nick Craig-Wood)
43673
43674 • Implement Purge to purge versions and backend cleanup-hidden (Nick
43675 Craig-Wood)
43676
43677 • Add --s3-decompress flag to decompress gzip-encoded files (Nick
43678 Craig-Wood)
43679
43680 • Add --s3-sse-customer-key-base64 to supply keys with binary data
43681 (Richard Bateman)
43682
43683 • Try to keep the maximum precision in ModTime with --user-server-
43684 modtime (Nick Craig-Wood)
43685
43686 • Drop binary metadata with an ERROR message as it can't be stored
43687 (Nick Craig-Wood)
43688
43689 • Add --s3-no-system-metadata to suppress read and write of system
43690 metadata (Nick Craig-Wood)
43691
43692 • SFTP
43693
43694 • Fix directory creation races (Lesmiscore)
43695
43696 • Swift
43697
43698 • Add --swift-no-large-objects to reduce HEAD requests (Nick Craig-
43699 Wood)
43700
43701 • Union
43702
43703 • Propagate SlowHash feature to fix hasher interaction (Lesmiscore)
43704
43705 v1.59.2 - 2022-09-15
43706 See commits (https://github.com/rclone/rclone/com‐
43707 pare/v1.59.1...v1.59.2)
43708
43709 • Bug Fixes
43710
43711 • config: Move locking to fix fatal error: concurrent map read and
43712 map write (Nick Craig-Wood)
43713
43714 • Local
43715
43716 • Disable xattr support if the filesystems indicates it is not sup‐
43717 ported (Nick Craig-Wood)
43718
43719 • Azure Blob
43720
43721 • Fix chunksize calculations producing too many parts (Nick Craig-
43722 Wood)
43723
43724 • B2
43725
43726 • Fix chunksize calculations producing too many parts (Nick Craig-
43727 Wood)
43728
43729 • S3
43730
43731 • Fix chunksize calculations producing too many parts (Nick Craig-
43732 Wood)
43733
43734 v1.59.1 - 2022-08-08
43735 See commits (https://github.com/rclone/rclone/com‐
43736 pare/v1.59.0...v1.59.1)
43737
43738 • Bug Fixes
43739
43740 • accounting: Fix panic in core/stats-reset with unknown group (Nick
43741 Craig-Wood)
43742
43743 • build: Fix android build after GitHub actions change (Nick Craig-
43744 Wood)
43745
43746 • dlna: Fix SOAP action header parsing (Joram Schrijver)
43747
43748 • docs: Fix links to mount command from install docs (albertony)
43749
43750 • dropbox: Fix ChangeNotify was unable to decrypt errors (Nick Craig-
43751 Wood)
43752
43753 • fs: Fix parsing of times and durations of the form "YYYY-MM-DD
43754 HH:MM:SS" (Nick Craig-Wood)
43755
43756 • serve sftp: Fix checksum detection (Nick Craig-Wood)
43757
43758 • sync: Add accidentally missed filter-sensitivity to --backup-dir
43759 option (Nick Naumann)
43760
43761 • Combine
43762
43763 • Fix docs showing remote= instead of upstreams= (Nick Craig-Wood)
43764
43765 • Throw error if duplicate directory name is specified (Nick Craig-
43766 Wood)
43767
43768 • Fix errors with backends shutting down while in use (Nick Craig-
43769 Wood)
43770
43771 • Dropbox
43772
43773 • Fix hang on quit with --dropbox-batch-mode off (Nick Craig-Wood)
43774
43775 • Fix infinite loop on uploading a corrupted file (Nick Craig-Wood)
43776
43777 • Internetarchive
43778
43779 • Ignore checksums for files using the different method (Lesmiscore)
43780
43781 • Handle hash symbol in the middle of filename (Lesmiscore)
43782
43783 • Jottacloud
43784
43785 • Fix working with whitelabel Elgiganten Cloud
43786
43787 • Do not store username in config when using standard auth (al‐
43788 bertony)
43789
43790 • Mega
43791
43792 • Fix nil pointer exception when bad node received (Nick Craig-Wood)
43793
43794 • S3
43795
43796 • Fix --s3-no-head panic: reflect: Elem of invalid type s3.PutObject‐
43797 Input (Nick Craig-Wood)
43798
43799 • SFTP
43800
43801 • Fix issue with WS_FTP by working around failing RealPath (al‐
43802 bertony)
43803
43804 • Union
43805
43806 • Fix duplicated files when using directories with leading / (Nick
43807 Craig-Wood)
43808
43809 • Fix multiple files being uploaded when roots don't exist (Nick
43810 Craig-Wood)
43811
43812 • Fix panic due to misalignment of struct field in 32 bit architec‐
43813 tures (r-ricci)
43814
43815 v1.59.0 - 2022-07-09
43816 See commits (https://github.com/rclone/rclone/com‐
43817 pare/v1.58.0...v1.59.0)
43818
43819 • New backends
43820
43821 • Combine multiple remotes in one directory tree (Nick Craig-Wood)
43822
43823 • Hidrive (https://rclone.org/hidrive/) (Ovidiu Victor Tatar)
43824
43825 • Internet Archive (https://rclone.org/internetarchive/) (Lesmiscore
43826 (Naoya Ozaki))
43827
43828 • New S3 providers
43829
43830 • ArvanCloud AOS (https://rclone.org/s3/#arvan-cloud) (ehsantdy)
43831
43832 • Cloudflare R2 (https://rclone.org/s3/#cloudflare-r2) (Nick Craig-
43833 Wood)
43834
43835 • Huawei OBS (https://rclone.org/s3/#huawei-obs) (m00594701)
43836
43837 • IDrive e2 (https://rclone.org/s3/#idrive-e2) (vyloy)
43838
43839 • New commands
43840
43841 • test makefile (https://rclone.org/commands/rclone_test_makefile/):
43842 Create a single file for testing (Nick Craig-Wood)
43843
43844 • New Features
43845
43846 • Metadata framework (https://rclone.org/docs/#metadata) to read and
43847 write system and user metadata on backends (Nick Craig-Wood)
43848
43849 • Implemented initially for local, s3 and internetarchive backends
43850
43851 • --metadata/-M flag to control whether metadata is copied
43852
43853 • --metadata-set flag to specify metadata for uploads
43854
43855 • Thanks to Manz Solutions (https://manz-solutions.at/) for spon‐
43856 soring this work.
43857
43858 • build
43859
43860 • Update to go1.18 and make go1.16 the minimum required version
43861 (Nick Craig-Wood)
43862
43863 • Update android go build to 1.18.x and NDK to 23.1.7779620 (Nick
43864 Craig-Wood)
43865
43866 • All windows binaries now no longer CGO (Nick Craig-Wood)
43867
43868 • Add linux/arm/v6 to docker images (Nick Craig-Wood)
43869
43870 • A huge number of fixes found with staticcheck (https://static‐
43871 check.io/) (albertony)
43872
43873 • Configurable version suffix independent of version number (al‐
43874 bertony)
43875
43876 • check: Implement --no-traverse and --no-unicode-normalization (Nick
43877 Craig-Wood)
43878
43879 • config: Readability improvements (albertony)
43880
43881 • copyurl: Add --header-filename to honor the HTTP header filename
43882 directive (J-P Treen)
43883
43884 • filter: Allow multiple --exclude-if-present flags (albertony)
43885
43886 • fshttp: Add --disable-http-keep-alives to disable HTTP Keep Alives
43887 (Nick Craig-Wood)
43888
43889 • install.sh
43890
43891 • Set the modes on the files and/or directories on macOS (Michael C
43892 Tiernan - MIT-Research Computing Project)
43893
43894 • Pre verify sudo authorization -v before calling curl. (Michael C
43895 Tiernan - MIT-Research Computing Project)
43896
43897 • lib/encoder: Add Semicolon encoding (Nick Craig-Wood)
43898
43899 • lsf: Add metadata support with M flag (Nick Craig-Wood)
43900
43901 • lsjson: Add --metadata/-M flag (Nick Craig-Wood)
43902
43903 • ncdu
43904
43905 • Implement multi selection (CrossR)
43906
43907 • Replace termbox with tcell's termbox wrapper (eNV25)
43908
43909 • Display correct path in delete confirmation dialog (Roberto Ric‐
43910 ci)
43911
43912 • operations
43913
43914 • Speed up hash checking by aborting the other hash if first re‐
43915 turns nothing (Nick Craig-Wood)
43916
43917 • Use correct src/dst in some log messages (zzr93)
43918
43919 • rcat: Check checksums by default like copy does (Nick Craig-Wood)
43920
43921 • selfupdate: Replace deprecated x/crypto/openpgp package with Pro‐
43922 tonMail/go-crypto (albertony)
43923
43924 • serve ftp: Check --passive-port arguments are correct (Nick Craig-
43925 Wood)
43926
43927 • size: Warn about inaccurate results when objects with unknown size
43928 (albertony)
43929
43930 • sync: Overlap check is now filter-sensitive so --backup-dir can be
43931 in the root provided it is filtered (Nick)
43932
43933 • test info: Check file name lengths using 1,2,3,4 byte unicode char‐
43934 acters (Nick Craig-Wood)
43935
43936 • test makefile(s): --sparse, --zero, --pattern, --ascii, --chargen
43937 flags to control file contents (Nick Craig-Wood)
43938
43939 • Make sure we call the Shutdown method on backends (Martin Czygan)
43940
43941 • Bug Fixes
43942
43943 • accounting: Fix unknown length file transfers counting 3 transfers
43944 each (buda)
43945
43946 • ncdu: Fix issue where dir size is summed when file sizes are -1
43947 (albertony)
43948
43949 • sync/copy/move
43950
43951 • Fix --fast-list --create-empty-src-dirs and --exclude (Nick
43952 Craig-Wood)
43953
43954 • Fix --max-duration and --cutoff-mode soft (Nick Craig-Wood)
43955
43956 • Fix fs cache unpin (Martin Czygan)
43957
43958 • Set proper exit code for errors that are not low-level retried
43959 (e.g. size/timestamp changing) (albertony)
43960
43961 • Mount
43962
43963 • Support windows/arm64 (may still be problems - see #5828
43964 (https://github.com/rclone/rclone/issues/5828)) (Nick Craig-Wood)
43965
43966 • Log IO errors at ERROR level (Nick Craig-Wood)
43967
43968 • Ignore _netdev mount argument (Hugal31)
43969
43970 • VFS
43971
43972 • Add --vfs-fast-fingerprint for less accurate but faster finger‐
43973 prints (Nick Craig-Wood)
43974
43975 • Add --vfs-disk-space-total-size option to manually set the total
43976 disk space (Claudio Maradonna)
43977
43978 • vfscache: Fix fatal error: sync: unlock of unlocked mutex error
43979 (Nick Craig-Wood)
43980
43981 • Local
43982
43983 • Fix parsing of --local-nounc flag (Nick Craig-Wood)
43984
43985 • Add Metadata support (Nick Craig-Wood)
43986
43987 • Crypt
43988
43989 • Support metadata (Nick Craig-Wood)
43990
43991 • Azure Blob
43992
43993 • Calculate Chunksize/blocksize to stay below maxUploadParts (Leroy
43994 van Logchem)
43995
43996 • Use chunksize lib to determine chunksize dynamically (Derek Bat‐
43997 tams)
43998
43999 • Case insensitive access tier (Rob Pickerill)
44000
44001 • Allow remote emulator (azurite) (Lorenzo Maiorfi)
44002
44003 • B2
44004
44005 • Add --b2-version-at flag to show file versions at time specified
44006 (SwazRGB)
44007
44008 • Use chunksize lib to determine chunksize dynamically (Derek Bat‐
44009 tams)
44010
44011 • Chunker
44012
44013 • Mark as not supporting metadata (Nick Craig-Wood)
44014
44015 • Compress
44016
44017 • Support metadata (Nick Craig-Wood)
44018
44019 • Drive
44020
44021 • Make backend config -o config add a combined AllDrives: remote
44022 (Nick Craig-Wood)
44023
44024 • Make --drive-shared-with-me work with shared drives (Nick Craig-
44025 Wood)
44026
44027 • Add --drive-resource-key for accessing link-shared files (Nick
44028 Craig-Wood)
44029
44030 • Add backend commands exportformats and importformats for debugging
44031 (Nick Craig-Wood)
44032
44033 • Fix 404 errors on copy/server side copy objects from public folder
44034 (Nick Craig-Wood)
44035
44036 • Update Internal OAuth consent screen docs (Phil Shackleton)
44037
44038 • Moved root_folder_id to advanced section (Abhiraj)
44039
44040 • Dropbox
44041
44042 • Migrate from deprecated api (m8rge)
44043
44044 • Add logs to show when poll interval limits are exceeded (Nick
44045 Craig-Wood)
44046
44047 • Fix nil pointer exception on dropbox impersonate user not found
44048 (Nick Craig-Wood)
44049
44050 • Fichier
44051
44052 • Parse api error codes and them accordingly (buengese)
44053
44054 • FTP
44055
44056 • Add support for disable_utf8 option (Jason Zheng)
44057
44058 • Revert to upstream github.com/jlaffaye/ftp from our fork (Nick
44059 Craig-Wood)
44060
44061 • Google Cloud Storage
44062
44063 • Add --gcs-no-check-bucket to minimise transactions and perms (Nick
44064 Gooding)
44065
44066 • Add --gcs-decompress flag to decompress gzip-encoded files (Nick
44067 Craig-Wood)
44068
44069 • by default these will be downloaded compressed (which previously
44070 failed)
44071
44072 • Hasher
44073
44074 • Support metadata (Nick Craig-Wood)
44075
44076 • HTTP
44077
44078 • Fix missing response when using custom auth handler (albertony)
44079
44080 • Jottacloud
44081
44082 • Add support for upload to custom device and mountpoint (albertony)
44083
44084 • Always store username in config and use it to avoid initial API re‐
44085 quest (albertony)
44086
44087 • Fix issue with server-side copy when destination is in trash (al‐
44088 bertony)
44089
44090 • Fix listing output of remote with special characters (albertony)
44091
44092 • Mailru
44093
44094 • Fix timeout by using int instead of time.Duration for keeping num‐
44095 ber of seconds (albertony)
44096
44097 • Mega
44098
44099 • Document using MEGAcmd to help with login failures (Art M. Gal‐
44100 lagher)
44101
44102 • Onedrive
44103
44104 • Implement --poll-interval for onedrive (Hugo Laloge)
44105
44106 • Add access scopes option (Sven Gerber)
44107
44108 • Opendrive
44109
44110 • Resolve lag and truncate bugs (Scott Grimes)
44111
44112 • Pcloud
44113
44114 • Fix about with no free space left (buengese)
44115
44116 • Fix cleanup (buengese)
44117
44118 • S3
44119
44120 • Use PUT Object instead of presigned URLs to upload single part ob‐
44121 jects (Nick Craig-Wood)
44122
44123 • Backend restore command to skip non-GLACIER objects (Vincent Mur‐
44124 phy)
44125
44126 • Use chunksize lib to determine chunksize dynamically (Derek Bat‐
44127 tams)
44128
44129 • Retry RequestTimeout errors (Nick Craig-Wood)
44130
44131 • Implement reading and writing of metadata (Nick Craig-Wood)
44132
44133 • SFTP
44134
44135 • Add support for about and hashsum on windows server (albertony)
44136
44137 • Use vendor-specific VFS statistics extension for about if available
44138 (albertony)
44139
44140 • Add --sftp-chunk-size to control packets sizes for high latency
44141 links (Nick Craig-Wood)
44142
44143 • Add --sftp-concurrency to improve high latency transfers (Nick
44144 Craig-Wood)
44145
44146 • Add --sftp-set-env option to set environment variables (Nick Craig-
44147 Wood)
44148
44149 • Add Hetzner Storage Boxes to supported sftp backends (Anthrazz)
44150
44151 • Storj
44152
44153 • Fix put which lead to the file being unreadable when using mount
44154 (Erik van Velzen)
44155
44156 • Union
44157
44158 • Add min_free_space option for lfs/eplfs policies (Nick Craig-Wood)
44159
44160 • Fix uploading files to union of all bucket based remotes (Nick
44161 Craig-Wood)
44162
44163 • Fix get free space for remotes which don't support it (Nick Craig-
44164 Wood)
44165
44166 • Fix eplus policy to select correct entry for existing files (Nick
44167 Craig-Wood)
44168
44169 • Support metadata (Nick Craig-Wood)
44170
44171 • Uptobox
44172
44173 • Fix root path handling (buengese)
44174
44175 • WebDAV
44176
44177 • Add SharePoint in other specific regions support (Noah Hsu)
44178
44179 • Yandex
44180
44181 • Handle api error on server-side move (albertony)
44182
44183 • Zoho
44184
44185 • Add Japan and China regions (buengese)
44186
44187 v1.58.1 - 2022-04-29
44188 See commits (https://github.com/rclone/rclone/com‐
44189 pare/v1.58.0...v1.58.1)
44190
44191 • Bug Fixes
44192
44193 • build: Update github.com/billziss-gh to github.com/winfsp (Nick
44194 Craig-Wood)
44195
44196 • filter: Fix timezone of --min-age/-max-age from UTC to local as
44197 documented (Nick Craig-Wood)
44198
44199 • rc/js: Correct RC method names (Sơn Trần-Nguyễn)
44200
44201 • docs
44202
44203 • Fix some links to command pages (albertony)
44204
44205 • Add --multi-thread-streams note to --transfers. (Zsolt Ero)
44206
44207 • Mount
44208
44209 • Fix --devname and fusermount: unknown option 'fsname' when mounting
44210 via rc (Nick Craig-Wood)
44211
44212 • VFS
44213
44214 • Remove wording which suggests VFS is only for mounting (Nick Craig-
44215 Wood)
44216
44217 • Dropbox
44218
44219 • Fix retries of multipart uploads with incorrect_offset error (Nick
44220 Craig-Wood)
44221
44222 • Google Cloud Storage
44223
44224 • Use the s3 pacer to speed up transactions (Nick Craig-Wood)
44225
44226 • pacer: Default the Google pacer to a burst of 100 to fix gcs pacing
44227 (Nick Craig-Wood)
44228
44229 • Jottacloud
44230
44231 • Fix scope in token request (albertony)
44232
44233 • Netstorage
44234
44235 • Fix unescaped HTML in documentation (Nick Craig-Wood)
44236
44237 • Make levels of headings consistent (Nick Craig-Wood)
44238
44239 • Add support contacts to netstorage doc (Nil Alexandrov)
44240
44241 • Onedrive
44242
44243 • Note that sharepoint also changes web files (.html, .aspx) (GH)
44244
44245 • Putio
44246
44247 • Handle rate limit errors (Berkan Teber)
44248
44249 • Fix multithread download and other ranged requests (rafma0)
44250
44251 • S3
44252
44253 • Add ChinaMobile EOS to provider list (GuoXingbin)
44254
44255 • Sync providers in config description with providers (Nick Craig-
44256 Wood)
44257
44258 • SFTP
44259
44260 • Fix OpenSSH 8.8+ RSA keys incompatibility (KARBOWSKI Piotr)
44261
44262 • Note that Scaleway C14 is deprecating SFTP in favor of S3 (Adrien
44263 Rey-Jarthon)
44264
44265 • Storj
44266
44267 • Fix bucket creation on Move (Nick Craig-Wood)
44268
44269 • WebDAV
44270
44271 • Don't override Referer if user sets it (Nick Craig-Wood)
44272
44273 v1.58.0 - 2022-03-18
44274 See commits (https://github.com/rclone/rclone/com‐
44275 pare/v1.57.0...v1.58.0)
44276
44277 • New backends
44278
44279 • Akamai Netstorage (Nil Alexandrov)
44280
44281 • Seagate Lyve (https://rclone.org/s3/#lyve), SeaweedFS, Storj, Rack‐
44282 Corp via s3 backend
44283
44284 • Storj (https://rclone.org/storj/) (renamed from Tardigrade - your
44285 old config files will continue working)
44286
44287 • New commands
44288
44289 • bisync (https://rclone.org/bisync/) - experimental bidirectional
44290 cloud sync (Ivan Andreev, Chris Nelson)
44291
44292 • New Features
44293
44294 • build
44295
44296 • Add windows/arm64 build (rclone mount not supported yet) (Nick
44297 Craig-Wood)
44298
44299 • Raise minimum go version to go1.15 (Nick Craig-Wood)
44300
44301 • config: Allow dot in remote names and improve config editing (al‐
44302 bertony)
44303
44304 • dedupe: Add quit as a choice in interactive mode (albertony)
44305
44306 • dlna: Change icons to the newest ones. (Alain Nussbaumer)
44307
44308 • filter: Add {{ regexp }} syntax (https://rclone.org/filtering/#reg‐
44309 exp) to pattern matches (Nick Craig-Wood)
44310
44311 • fshttp: Add prometheus metrics for HTTP status code (Michał
44312 Matczuk)
44313
44314 • hashsum: Support creating hash from data received on stdin (al‐
44315 bertony)
44316
44317 • librclone
44318
44319 • Allow empty string or null input instead of empty json object
44320 (albertony)
44321
44322 • Add support for mount commands (albertony)
44323
44324 • operations: Add server-side moves to stats (Ole Frost)
44325
44326 • rc: Allow user to disable authentication for web gui (negative0)
44327
44328 • tree: Remove obsolete --human replaced by global --human-readable
44329 (albertony)
44330
44331 • version: Report correct friendly-name for newer Windows 10/11 ver‐
44332 sions (albertony)
44333
44334 • Bug Fixes
44335
44336 • build
44337
44338 • Fix ARM architecture version in .deb packages after nfpm change
44339 (Nick Craig-Wood)
44340
44341 • Hard fork github.com/jlaffaye/ftp to fix go get
44342 github.com/rclone/rclone (Nick Craig-Wood)
44343
44344 • oauthutil: Fix crash when webbrowser requests /robots.txt (Nick
44345 Craig-Wood)
44346
44347 • operations: Fix goroutine leak in case of copy retry (Ankur Gupta)
44348
44349 • rc:
44350
44351 • Fix operations/publiclink default for expires parameter (Nick
44352 Craig-Wood)
44353
44354 • Fix missing computation of transferQueueSize when summing up sta‐
44355 tistics group (Carlo Mion)
44356
44357 • Fix missing StatsInfo fields in the computation of the group sum
44358 (Carlo Mion)
44359
44360 • sync: Fix --max-duration so it doesn't retry when the duration is
44361 exceeded (Nick Craig-Wood)
44362
44363 • touch: Fix issue where a directory is created instead of a file
44364 (albertony)
44365
44366 • Mount
44367
44368 • Add --devname to set the device name sent to FUSE for mount display
44369 (Nick Craig-Wood)
44370
44371 • VFS
44372
44373 • Add vfs/stats remote control to show statistics (Nick Craig-Wood)
44374
44375 • Fix failed to _ensure cache internal error: downloaders is nil er‐
44376 ror (Nick Craig-Wood)
44377
44378 • Fix handling of special characters in file names (Bumsu Hyeon)
44379
44380 • Local
44381
44382 • Fix hash invalidation which caused errors with local crypt mount
44383 (Nick Craig-Wood)
44384
44385 • Crypt
44386
44387 • Add base64 and base32768 filename encoding options (Max Sum, Sinan
44388 Tan)
44389
44390 • Azure Blob
44391
44392 • Implement --azureblob-upload-concurrency parameter to speed uploads
44393 (Nick Craig-Wood)
44394
44395 • Remove 100MB upper limit on chunk_size as it is no longer needed
44396 (Nick Craig-Wood)
44397
44398 • Raise --azureblob-upload-concurrency to 16 by default (Nick Craig-
44399 Wood)
44400
44401 • Fix crash with SAS URL and no container (Nick Craig-Wood)
44402
44403 • Compress
44404
44405 • Fix crash if metadata upload failed (Nick Craig-Wood)
44406
44407 • Fix memory leak (Nick Craig-Wood)
44408
44409 • Drive
44410
44411 • Added --drive-copy-shortcut-content (Abhiraj)
44412
44413 • Disable OAuth OOB flow (copy a token) due to Google deprecation
44414 (Nick Craig-Wood)
44415
44416 • See the deprecation note (https://developers.google‐
44417 blog.com/2022/02/making-oauth-flows-safer.html#disallowed-oob).
44418
44419 • Add --drive-skip-dangling-shortcuts flag (Nick Craig-Wood)
44420
44421 • When using a link type --drive-export-formats shows all doc types
44422 (Nick Craig-Wood)
44423
44424 • Dropbox
44425
44426 • Speed up directory listings by specifying 1000 items in a chunk
44427 (Nick Craig-Wood)
44428
44429 • Save an API request when at the root (Nick Craig-Wood)
44430
44431 • Fichier
44432
44433 • Implemented About functionality (Gourav T)
44434
44435 • FTP
44436
44437 • Add --ftp-ask-password to prompt for password when needed (Borna
44438 Butkovic)
44439
44440 • Google Cloud Storage
44441
44442 • Add missing regions (Nick Craig-Wood)
44443
44444 • Disable OAuth OOB flow (copy a token) due to Google deprecation
44445 (Nick Craig-Wood)
44446
44447 • See the deprecation note (https://developers.google‐
44448 blog.com/2022/02/making-oauth-flows-safer.html#disallowed-oob).
44449
44450 • Googlephotos
44451
44452 • Disable OAuth OOB flow (copy a token) due to Google deprecation
44453 (Nick Craig-Wood)
44454
44455 • See the deprecation note (https://developers.google‐
44456 blog.com/2022/02/making-oauth-flows-safer.html#disallowed-oob).
44457
44458 • Hasher
44459
44460 • Fix crash on object not found (Nick Craig-Wood)
44461
44462 • Hdfs
44463
44464 • Add file (Move) and directory move (DirMove) support (Andy Jackson)
44465
44466 • HTTP
44467
44468 • Improved recognition of URL pointing to a single file (albertony)
44469
44470 • Jottacloud
44471
44472 • Change API used by recursive list (ListR) (Kim)
44473
44474 • Add support for Tele2 Cloud (Fredric Arklid)
44475
44476 • Koofr
44477
44478 • Add Digistorage service as a Koofr provider. (jaKa)
44479
44480 • Mailru
44481
44482 • Fix int32 overflow on arm32 (Ivan Andreev)
44483
44484 • Onedrive
44485
44486 • Add config option for oauth scope Sites.Read.All (Charlie Jiang)
44487
44488 • Minor optimization of quickxorhash (Isaac Levy)
44489
44490 • Add --onedrive-root-folder-id flag (Nick Craig-Wood)
44491
44492 • Do not retry on 400 pathIsTooLong error (ctrl-q)
44493
44494 • Pcloud
44495
44496 • Add support for recursive list (ListR) (Niels van de Weem)
44497
44498 • Fix pre-1970 time stamps (Nick Craig-Wood)
44499
44500 • S3
44501
44502 • Use ListObjectsV2 for faster listings (Felix Bünemann)
44503
44504 • Fallback to ListObject v1 on unsupported providers (Nick Craig-
44505 Wood)
44506
44507 • Use the ETag on multipart transfers to verify the transfer was OK
44508 (Nick Craig-Wood)
44509
44510 • Add --s3-use-multipart-etag provider quirk to disable this on un‐
44511 supported providers (Nick Craig-Wood)
44512
44513 • New Providers
44514
44515 • RackCorp object storage (bbabich)
44516
44517 • Seagate Lyve Cloud storage (Nick Craig-Wood)
44518
44519 • SeaweedFS (Chris Lu)
44520
44521 • Storj Shared gateways (Márton Elek, Nick Craig-Wood)
44522
44523 • Add Wasabi AP Northeast 2 endpoint info (lindwurm)
44524
44525 • Add GLACIER_IR storage class (Yunhai Luo)
44526
44527 • Document Content-MD5 workaround for object-lock enabled buckets
44528 (Paulo Martins)
44529
44530 • Fix multipart upload with --no-head flag (Nick Craig-Wood)
44531
44532 • Simplify content length processing in s3 with download url (Logesh‐
44533 waran Murugesan)
44534
44535 • SFTP
44536
44537 • Add rclone to list of supported md5sum/sha1sum commands to look for
44538 (albertony)
44539
44540 • Refactor so we only have one way of running remote commands (Nick
44541 Craig-Wood)
44542
44543 • Fix timeout on hashing large files by sending keepalives (Nick
44544 Craig-Wood)
44545
44546 • Fix unnecessary seeking when uploading and downloading files (Nick
44547 Craig-Wood)
44548
44549 • Update docs on how to create known_hosts file (Nick Craig-Wood)
44550
44551 • Storj
44552
44553 • Rename tardigrade backend to storj backend (Nick Craig-Wood)
44554
44555 • Implement server side Move for files (Nick Craig-Wood)
44556
44557 • Update docs to explain differences between s3 and this backend
44558 (Elek, Márton)
44559
44560 • Swift
44561
44562 • Fix About so it shows info about the current container only (Nick
44563 Craig-Wood)
44564
44565 • Union
44566
44567 • Fix treatment of remotes with // in (Nick Craig-Wood)
44568
44569 • Fix deadlock when one part of a multi-upload fails (Nick Craig-
44570 Wood)
44571
44572 • Fix eplus policy returned nil (Vitor Arruda)
44573
44574 • Yandex
44575
44576 • Add permanent deletion support (deinferno)
44577
44578 v1.57.0 - 2021-11-01
44579 See commits (https://github.com/rclone/rclone/com‐
44580 pare/v1.56.0...v1.57.0)
44581
44582 • New backends
44583
44584 • Sia: for Sia decentralized cloud (Ian Levesque, Matthew Sevey, Ivan
44585 Andreev)
44586
44587 • Hasher: caches hashes and enable hashes for backends that don't
44588 support them (Ivan Andreev)
44589
44590 • New commands
44591
44592 • lsjson --stat: to get info about a single file/dir and opera‐
44593 tions/stat api (Nick Craig-Wood)
44594
44595 • config paths: show configured paths (albertony)
44596
44597 • New Features
44598
44599 • about: Make human-readable output more consistent with other com‐
44600 mands (albertony)
44601
44602 • build
44603
44604 • Use go1.17 for building and make go1.14 the minimum supported
44605 (Nick Craig-Wood)
44606
44607 • Update Go to 1.16 and NDK to 22b for Android builds (x0b)
44608
44609 • config
44610
44611 • Support hyphen in remote name from environment variable (al‐
44612 bertony)
44613
44614 • Make temporary directory user-configurable (albertony)
44615
44616 • Convert --cache-dir value to an absolute path (albertony)
44617
44618 • Do not override MIME types from OS defaults (albertony)
44619
44620 • docs
44621
44622 • Toc styling and header levels cleanup (albertony)
44623
44624 • Extend documentation on valid remote names (albertony)
44625
44626 • Mention make for building and cmount tag for macos (Alex Chen)
44627
44628 • ...and many more contributions to numerous to mention!
44629
44630 • fs: Move with --ignore-existing will not delete skipped files
44631 (Nathan Collins)
44632
44633 • hashsum
44634
44635 • Treat hash values in sum file as case insensitive (Ivan Andreev)
44636
44637 • Don't put ERROR or UNSUPPORTED in output (Ivan Andreev)
44638
44639 • lib/encoder: Add encoding of square brackets (Ivan Andreev)
44640
44641 • lib/file: Improve error message when attempting to create dir on
44642 nonexistent drive on windows (albertony)
44643
44644 • lib/http: Factor password hash salt into options with default
44645 (Nolan Woods)
44646
44647 • lib/kv: Add key-value database api (Ivan Andreev)
44648
44649 • librclone
44650
44651 • Add RcloneFreeString function (albertony)
44652
44653 • Free strings in python example (albertony)
44654
44655 • log: Optionally print pid in logs (Ivan Andreev)
44656
44657 • ls: Introduce --human-readable global option to print human-read‐
44658 able sizes (albertony)
44659
44660 • ncdu: Introduce key u to toggle human-readable (albertony)
44661
44662 • operations: Add rmdirs -v output (Justin Winokur)
44663
44664 • serve sftp
44665
44666 • Generate an ECDSA server key as well as RSA (Nick Craig-Wood)
44667
44668 • Generate an Ed25519 server key as well as ECDSA and RSA (al‐
44669 bertony)
44670
44671 • serve docker
44672
44673 • Allow to customize proxy settings of docker plugin (Ivan Andreev)
44674
44675 • Build docker plugin for multiple platforms (Thomas Stachl)
44676
44677 • size: Include human-readable count (albertony)
44678
44679 • touch: Add support for touching files in directory, with recursive
44680 option, filtering and --dry-run/-i (albertony)
44681
44682 • tree: Option to print human-readable sizes removed in favor of
44683 global option (albertony)
44684
44685 • Bug Fixes
44686
44687 • lib/http
44688
44689 • Fix bad username check in single auth secret provider (Nolan
44690 Woods)
44691
44692 • Fix handling of SSL credentials (Nolan Woods)
44693
44694 • serve ftp: Ensure modtime is passed as UTC always to fix timezone
44695 oddities (Nick Craig-Wood)
44696
44697 • serve sftp: Fix generation of server keys on windows (albertony)
44698
44699 • serve docker: Fix octal umask (Ivan Andreev)
44700
44701 • Mount
44702
44703 • Enable rclone to be run as mount helper direct from the fstab (Ivan
44704 Andreev)
44705
44706 • Use procfs to validate mount on linux (Ivan Andreev)
44707
44708 • Correctly daemonize for compatibility with automount (Ivan Andreev)
44709
44710 • VFS
44711
44712 • Ensure names used in cache path are legal on current OS (albertony)
44713
44714 • Ignore ECLOSED when truncating file handles to fix intermittent bad
44715 file descriptor error (Nick Craig-Wood)
44716
44717 • Local
44718
44719 • Refactor default OS encoding out from local backend into shared en‐
44720 coder lib (albertony)
44721
44722 • Crypt
44723
44724 • Return wrapped object even with --crypt-no-data-encryption (Ivan
44725 Andreev)
44726
44727 • Fix uploads with --crypt-no-data-encryption (Nick Craig-Wood)
44728
44729 • Azure Blob
44730
44731 • Add --azureblob-no-head-object (Tatsuya Noyori)
44732
44733 • Box
44734
44735 • Make listings of heavily used directories more reliable (Nick
44736 Craig-Wood)
44737
44738 • When doing cleanup delete as much as possible (Nick Craig-Wood)
44739
44740 • Add --box-list-chunk to control listing chunk size (Nick Craig-
44741 Wood)
44742
44743 • Delete items in parallel in cleanup using --checkers threads (Nick
44744 Craig-Wood)
44745
44746 • Add --box-owned-by to only show items owned by the login passed
44747 (Nick Craig-Wood)
44748
44749 • Retry operation_blocked_temporary errors (Nick Craig-Wood)
44750
44751 • Chunker
44752
44753 • Md5all must create metadata if base hash is slow (Ivan Andreev)
44754
44755 • Drive
44756
44757 • Speed up directory listings by constraining the API listing using
44758 the current filters (fotile96, Ivan Andreev)
44759
44760 • Fix buffering for single request upload for files smaller than
44761 --drive-upload-cutoff (YenForYang)
44762
44763 • Add -o config option to backend drives to make config for all
44764 shared drives (Nick Craig-Wood)
44765
44766 • Dropbox
44767
44768 • Add --dropbox-batch-commit-timeout to control batch timeout (Nick
44769 Craig-Wood)
44770
44771 • Filefabric
44772
44773 • Make backoff exponential for error_background to fix errors (Nick
44774 Craig-Wood)
44775
44776 • Fix directory move after API change (Nick Craig-Wood)
44777
44778 • FTP
44779
44780 • Enable tls session cache by default (Ivan Andreev)
44781
44782 • Add option to disable tls13 (Ivan Andreev)
44783
44784 • Fix timeout after long uploads (Ivan Andreev)
44785
44786 • Add support for precise time (Ivan Andreev)
44787
44788 • Enable CI for ProFtpd, PureFtpd, VsFtpd (Ivan Andreev)
44789
44790 • Googlephotos
44791
44792 • Use encoder for album names to fix albums with control characters
44793 (Parth Shukla)
44794
44795 • Jottacloud
44796
44797 • Implement SetModTime to support modtime-only changes (albertony)
44798
44799 • Improved error handling with SetModTime and corrupt files in gener‐
44800 al (albertony)
44801
44802 • Add support for UserInfo (rclone config userinfo) feature (al‐
44803 bertony)
44804
44805 • Return direct download link from rclone link command (albertony)
44806
44807 • Koofr
44808
44809 • Create direct share link (Dmitry Bogatov)
44810
44811 • Pcloud
44812
44813 • Add sha256 support (Ken Enrique Morel)
44814
44815 • Premiumizeme
44816
44817 • Fix directory listing after API changes (Nick Craig-Wood)
44818
44819 • Fix server side move after API change (Nick Craig-Wood)
44820
44821 • Fix server side directory move after API changes (Nick Craig-Wood)
44822
44823 • S3
44824
44825 • Add support to use CDN URL to download the file (Logeshwaran)
44826
44827 • Add AWS Snowball Edge to providers examples (r0kk3rz)
44828
44829 • Use a combination of SDK retries and rclone retries (Nick Craig-
44830 Wood)
44831
44832 • Fix IAM Role for Service Account not working and other auth prob‐
44833 lems (Nick Craig-Wood)
44834
44835 • Fix shared_credentials_file auth after reverting incorrect fix
44836 (Nick Craig-Wood)
44837
44838 • Fix corrupted on transfer: sizes differ 0 vs xxxx with Ceph (Nick
44839 Craig-Wood)
44840
44841 • Seafile
44842
44843 • Fix error when not configured for 2fa (Fred)
44844
44845 • SFTP
44846
44847 • Fix timeout when doing MD5SUM of large file (Nick Craig-Wood)
44848
44849 • Swift
44850
44851 • Update OCI URL (David Liu)
44852
44853 • Document OVH Cloud Archive (HNGamingUK)
44854
44855 • Union
44856
44857 • Fix rename not working with union of local disk and bucket based
44858 remote (Nick Craig-Wood)
44859
44860 v1.56.2 - 2021-10-01
44861 See commits (https://github.com/rclone/rclone/com‐
44862 pare/v1.56.1...v1.56.2)
44863
44864 • Bug Fixes
44865
44866 • serve http: Re-add missing auth to http service (Nolan Woods)
44867
44868 • build: Update golang.org/x/sys to fix crash on macOS when compiled
44869 with go1.17 (Herby Gillot)
44870
44871 • FTP
44872
44873 • Fix deadlock after failed update when concurrency=1 (Ivan Andreev)
44874
44875 v1.56.1 - 2021-09-19
44876 See commits (https://github.com/rclone/rclone/com‐
44877 pare/v1.56.0...v1.56.1)
44878
44879 • Bug Fixes
44880
44881 • accounting: Fix maximum bwlimit by scaling scale max token bucket
44882 size (Nick Craig-Wood)
44883
44884 • rc: Fix speed does not update in core/stats (negative0)
44885
44886 • selfupdate: Fix --quiet option, not quite quiet (yedamo)
44887
44888 • serve http: Fix serve http exiting directly after starting (Cnly)
44889
44890 • build
44891
44892 • Apply gofmt from golang 1.17 (Ivan Andreev)
44893
44894 • Update Go to 1.16 and NDK to 22b for android/any (x0b)
44895
44896 • Mount
44897
44898 • Fix --daemon mode (Ivan Andreev)
44899
44900 • VFS
44901
44902 • Fix duplicates on rename (Nick Craig-Wood)
44903
44904 • Fix crash when truncating a just uploaded object (Nick Craig-Wood)
44905
44906 • Fix issue where empty dirs would build up in cache meta dir (al‐
44907 bertony)
44908
44909 • Drive
44910
44911 • Fix instructions for auto config (Greg Sadetsky)
44912
44913 • Fix lsf example without drive-impersonate (Greg Sadetsky)
44914
44915 • Onedrive
44916
44917 • Handle HTTP 400 better in PublicLink (Alex Chen)
44918
44919 • Clarification of the process for creating custom client_id (Mariano
44920 Absatz)
44921
44922 • Pcloud
44923
44924 • Return an early error when Put is called with an unknown size (Nick
44925 Craig-Wood)
44926
44927 • Try harder to delete a failed upload (Nick Craig-Wood)
44928
44929 • S3
44930
44931 • Add Wasabi's AP-Northeast endpoint info (hota)
44932
44933 • Fix typo in s3 documentation (Greg Sadetsky)
44934
44935 • Seafile
44936
44937 • Fix 2fa config state machine (Fred)
44938
44939 • SFTP
44940
44941 • Remove spurious error message on --sftp-disable-concurrent-reads
44942 (Nick Craig-Wood)
44943
44944 • Sugarsync
44945
44946 • Fix initial connection after config re-arrangement (Nick Craig-
44947 Wood)
44948
44949 v1.56.0 - 2021-07-20
44950 See commits (https://github.com/rclone/rclone/com‐
44951 pare/v1.55.0...v1.56.0)
44952
44953 • New backends
44954
44955 • Uptobox (https://rclone.org/uptobox/) (buengese)
44956
44957 • New commands
44958
44959 • serve docker (https://rclone.org/commands/rclone_serve_docker/)
44960 (Antoine GIRARD) (Ivan Andreev)
44961
44962 • and accompanying docker volume plugin (https://rclone.org/dock‐
44963 er/)
44964
44965 • checksum (https://rclone.org/commands/rclone_checksum/) to check
44966 files against a file of checksums (Ivan Andreev)
44967
44968 • this is also available as rclone md5sum -C etc
44969
44970 • config touch (https://rclone.org/commands/rclone_config_touch/):
44971 ensure config exists at configured location (albertony)
44972
44973 • test changenotify (https://rclone.org/com‐
44974 mands/rclone_test_changenotify/): command to help debugging
44975 changenotify (Nick Craig-Wood)
44976
44977 • Deprecations
44978
44979 • dbhashsum: Remove command deprecated a year ago (Ivan Andreev)
44980
44981 • cache: Deprecate cache backend (Ivan Andreev)
44982
44983 • New Features
44984
44985 • rework config system so it can be used non-interactively via cli
44986 and rc API.
44987
44988 • See docs in config create (https://rclone.org/com‐
44989 mands/rclone_config_create/)
44990
44991 • This is a very big change to all the backends so may cause break‐
44992 ages - please file bugs!
44993
44994 • librclone - export the rclone RC as a C library (lewisxy) (Nick
44995 Craig-Wood)
44996
44997 • Link a C-API rclone shared object into your project
44998
44999 • Use the RC as an in memory interface
45000
45001 • Python example supplied
45002
45003 • Also supports Android and gomobile
45004
45005 • fs
45006
45007 • Add --disable-http2 for global http2 disable (Nick Craig-Wood)
45008
45009 • Make --dump imply -vv (Alex Chen)
45010
45011 • Use binary prefixes for size and rate units (albertony)
45012
45013 • Use decimal prefixes for counts (albertony)
45014
45015 • Add google search widget to rclone.org (Ivan Andreev)
45016
45017 • accounting: Calculate rolling average speed (Haochen Tong)
45018
45019 • atexit: Terminate with non-zero status after receiving signal
45020 (Michael Hanselmann)
45021
45022 • build
45023
45024 • Only run event-based workflow scripts under rclone repo with man‐
45025 ual override (Mathieu Carbou)
45026
45027 • Add Android build with gomobile (x0b)
45028
45029 • check: Log the hash in use like cryptcheck does (Nick Craig-Wood)
45030
45031 • version: Print os/version, kernel and bitness (Ivan Andreev)
45032
45033 • config
45034
45035 • Prevent use of Windows reserved names in config file name (al‐
45036 bertony)
45037
45038 • Create config file in windows appdata directory by default (al‐
45039 bertony)
45040
45041 • Treat any config file paths with filename notfound as memory-only
45042 config (albertony)
45043
45044 • Delay load config file (albertony)
45045
45046 • Replace defaultConfig with a thread-safe in-memory implementation
45047 (Chris Macklin)
45048
45049 • Allow config create and friends to take key=value parameters
45050 (Nick Craig-Wood)
45051
45052 • Fixed issues with flags/options set by environment vars. (Ole
45053 Frost)
45054
45055 • fshttp: Implement graceful DSCP error handling (Tyson Moore)
45056
45057 • lib/http - provides an abstraction for a central http server that
45058 services can bind routes to (Nolan Woods)
45059
45060 • Add --template config and flags to serve/data (Nolan Woods)
45061
45062 • Add default 404 handler (Nolan Woods)
45063
45064 • link: Use "off" value for unset expiry (Nick Craig-Wood)
45065
45066 • oauthutil: Raise fatal error if token expired without refresh token
45067 (Alex Chen)
45068
45069 • rcat: Add --size flag for more efficient uploads of known size
45070 (Nazar Mishturak)
45071
45072 • serve sftp: Add --stdio flag to serve via stdio (Tom)
45073
45074 • sync: Don't warn about --no-traverse when --files-from is set (Nick
45075 Gaya)
45076
45077 • test makefiles
45078
45079 • Add --seed flag and make data generated repeatable (Nick Craig-
45080 Wood)
45081
45082 • Add log levels and speed summary (Nick Craig-Wood)
45083
45084 • Bug Fixes
45085
45086 • accounting: Fix startTime of statsGroups.sum (Haochen Tong)
45087
45088 • cmd/ncdu: Fix out of range panic in delete (buengese)
45089
45090 • config
45091
45092 • Fix issues with memory-only config file paths (albertony)
45093
45094 • Fix in memory config not saving on the fly backend config (Nick
45095 Craig-Wood)
45096
45097 • fshttp: Fix address parsing for DSCP (Tyson Moore)
45098
45099 • ncdu: Update termbox-go library to fix crash (Nick Craig-Wood)
45100
45101 • oauthutil: Fix old authorize result not recognised (Cnly)
45102
45103 • operations: Don't update timestamps of files in --compare-dest
45104 (Nick Gaya)
45105
45106 • selfupdate: fix archive name on macos (Ivan Andreev)
45107
45108 • Mount
45109
45110 • Refactor before adding serve docker (Antoine GIRARD)
45111
45112 • VFS
45113
45114 • Add cache reset for --vfs-cache-max-size handling at cache poll in‐
45115 terval (Leo Luan)
45116
45117 • Fix modtime changing when reading file into cache (Nick Craig-Wood)
45118
45119 • Avoid unnecessary subdir in cache path (albertony)
45120
45121 • Fix that umask option cannot be set as environment variable (al‐
45122 bertony)
45123
45124 • Do not print notice about missing poll-interval support when set to
45125 0 (albertony)
45126
45127 • Local
45128
45129 • Always use readlink to read symlink size for better compatibility
45130 (Nick Craig-Wood)
45131
45132 • Add --local-unicode-normalization (and remove --local-no-unicode-
45133 normalization) (Nick Craig-Wood)
45134
45135 • Skip entries removed concurrently with List() (Ivan Andreev)
45136
45137 • Crypt
45138
45139 • Support timestamped filenames from --b2-versions (Dominik Mydlil)
45140
45141 • B2
45142
45143 • Don't include the bucket name in public link file prefixes (Jeffrey
45144 Tolar)
45145
45146 • Fix versions and .files with no extension (Nick Craig-Wood)
45147
45148 • Factor version handling into lib/version (Dominik Mydlil)
45149
45150 • Box
45151
45152 • Use upload preflight check to avoid listings in file uploads (Nick
45153 Craig-Wood)
45154
45155 • Return errors instead of calling log.Fatal with them (Nick Craig-
45156 Wood)
45157
45158 • Drive
45159
45160 • Switch to the Drives API for looking up shared drives (Nick Craig-
45161 Wood)
45162
45163 • Fix some google docs being treated as files (Nick Craig-Wood)
45164
45165 • Dropbox
45166
45167 • Add --dropbox-batch-mode flag to speed up uploading (Nick Craig-
45168 Wood)
45169
45170 • Read the batch mode (https://rclone.org/dropbox/#batch-mode) docs
45171 for more info
45172
45173 • Set visibility in link sharing when --expire is set (Nick Craig-
45174 Wood)
45175
45176 • Simplify chunked uploads (Alexey Ivanov)
45177
45178 • Improve "own App IP" instructions (Ivan Andreev)
45179
45180 • Fichier
45181
45182 • Check if more than one upload link is returned (Nick Craig-Wood)
45183
45184 • Support downloading password protected files and folders (Florian
45185 Penzkofer)
45186
45187 • Make error messages report text from the API (Nick Craig-Wood)
45188
45189 • Fix move of files in the same directory (Nick Craig-Wood)
45190
45191 • Check that we actually got a download token and retry if we didn't
45192 (buengese)
45193
45194 • Filefabric
45195
45196 • Fix listing after change of from field from "int" to int. (Nick
45197 Craig-Wood)
45198
45199 • FTP
45200
45201 • Make upload error 250 indicate success (Nick Craig-Wood)
45202
45203 • GCS
45204
45205 • Make compatible with gsutil's mtime metadata (database64128)
45206
45207 • Clean up time format constants (database64128)
45208
45209 • Google Photos
45210
45211 • Fix read only scope not being used properly (Nick Craig-Wood)
45212
45213 • HTTP
45214
45215 • Replace httplib with lib/http (Nolan Woods)
45216
45217 • Clean up Bind to better use middleware (Nolan Woods)
45218
45219 • Jottacloud
45220
45221 • Fix legacy auth with state based config system (buengese)
45222
45223 • Fix invalid url in output from link command (albertony)
45224
45225 • Add no versions option (buengese)
45226
45227 • Onedrive
45228
45229 • Add list_chunk option (Nick Gaya)
45230
45231 • Also report root error if unable to cancel multipart upload (Cnly)
45232
45233 • Fix failed to configure: empty token found error (Nick Craig-Wood)
45234
45235 • Make link return direct download link (Xuanchen Wu)
45236
45237 • S3
45238
45239 • Add --s3-no-head-object (Tatsuya Noyori)
45240
45241 • Remove WebIdentityRoleProvider to fix crash on auth (Nick Craig-
45242 Wood)
45243
45244 • Don't check to see if remote is object if it ends with / (Nick
45245 Craig-Wood)
45246
45247 • Add SeaweedFS (Chris Lu)
45248
45249 • Update Alibaba OSS endpoints (Chuan Zh)
45250
45251 • SFTP
45252
45253 • Fix performance regression by re-enabling concurrent writes (Nick
45254 Craig-Wood)
45255
45256 • Expand tilde and environment variables in configured
45257 known_hosts_file (albertony)
45258
45259 • Tardigrade
45260
45261 • Upgrade to uplink v1.4.6 (Caleb Case)
45262
45263 • Use negative offset (Caleb Case)
45264
45265 • Add warning about too many open files (acsfer)
45266
45267 • WebDAV
45268
45269 • Fix sharepoint auth over http (Nick Craig-Wood)
45270
45271 • Add headers option (Antoon Prins)
45272
45273 v1.55.1 - 2021-04-26
45274 See commits (https://github.com/rclone/rclone/com‐
45275 pare/v1.55.0...v1.55.1)
45276
45277 • Bug Fixes
45278
45279 • selfupdate
45280
45281 • Dont detect FUSE if build is static (Ivan Andreev)
45282
45283 • Add build tag noselfupdate (Ivan Andreev)
45284
45285 • sync: Fix incorrect error reported by graceful cutoff (Nick Craig-
45286 Wood)
45287
45288 • install.sh: fix macOS arm64 download (Nick Craig-Wood)
45289
45290 • build: Fix version numbers in android branch builds (Nick Craig-
45291 Wood)
45292
45293 • docs
45294
45295 • Contributing.md: update setup instructions for go1.16 (Nick Gaya)
45296
45297 • WinFsp 2021 is out of beta (albertony)
45298
45299 • Minor cleanup of space around code section (albertony)
45300
45301 • Fixed some typos (albertony)
45302
45303 • VFS
45304
45305 • Fix a code path which allows dirty data to be removed causing data
45306 loss (Nick Craig-Wood)
45307
45308 • Compress
45309
45310 • Fix compressed name regexp (buengese)
45311
45312 • Drive
45313
45314 • Fix backend copyid of google doc to directory (Nick Craig-Wood)
45315
45316 • Don't open browser when service account... (Ansh Mittal)
45317
45318 • Dropbox
45319
45320 • Add missing team_data.member scope for use with --impersonate (Nick
45321 Craig-Wood)
45322
45323 • Fix About after scopes changes - rclone config reconnect needed
45324 (Nick Craig-Wood)
45325
45326 • Fix Unable to decrypt returned paths from changeNotify (Nick Craig-
45327 Wood)
45328
45329 • FTP
45330
45331 • Fix implicit TLS (Ivan Andreev)
45332
45333 • Onedrive
45334
45335 • Work around for random "Unable to initialize RPS" errors (OleFrost)
45336
45337 • SFTP
45338
45339 • Revert sftp library to v1.12.0 from v1.13.0 to fix performance re‐
45340 gression (Nick Craig-Wood)
45341
45342 • Fix Update ReadFrom failed: failed to send packet: EOF errors (Nick
45343 Craig-Wood)
45344
45345 • Zoho
45346
45347 • Fix error when region isn't set (buengese)
45348
45349 • Do not ask for mountpoint twice when using headless setup
45350 (buengese)
45351
45352 v1.55.0 - 2021-03-31
45353 See commits (https://github.com/rclone/rclone/com‐
45354 pare/v1.54.0...v1.55.0)
45355
45356 • New commands
45357
45358 • selfupdate (https://rclone.org/commands/rclone_selfupdate/) (Ivan
45359 Andreev)
45360
45361 • Allows rclone to update itself in-place or via a package (using
45362 --package flag)
45363
45364 • Reads cryptographically signed signatures for non beta releases
45365
45366 • Works on all OSes.
45367
45368 • test (https://rclone.org/commands/rclone_test/) - these are test
45369 commands - use with care!
45370
45371 • histogram - Makes a histogram of file name characters.
45372
45373 • info - Discovers file name or other limitations for paths.
45374
45375 • makefiles - Make a random file hierarchy for testing.
45376
45377 • memory - Load all the objects at remote:path into memory and re‐
45378 port memory stats.
45379
45380 • New Features
45381
45382 • Connection strings (https://rclone.org/docs/#connection-strings)
45383
45384 • Config parameters can now be passed as part of the remote name as
45385 a connection string.
45386
45387 • For example, to do the equivalent of --drive-shared-with-me use
45388 drive,shared_with_me:
45389
45390 • Make sure we don't save on the fly remote config to the config
45391 file (Nick Craig-Wood)
45392
45393 • Make sure backends with additional config have a different name
45394 for caching (Nick Craig-Wood)
45395
45396 • This work was sponsored by CERN, through the CS3MESH4EOSC Project
45397 (https://cs3mesh4eosc.eu/).
45398
45399 • CS3MESH4EOSC has received funding from the European Union’s
45400 Horizon 2020
45401
45402 • research and innovation programme under Grant Agreement no.
45403 863353.
45404
45405 • build
45406
45407 • Update go build version to go1.16 and raise minimum go version to
45408 go1.13 (Nick Craig-Wood)
45409
45410 • Make a macOS ARM64 build to support Apple Silicon (Nick Craig-
45411 Wood)
45412
45413 • Install macfuse 4.x instead of osxfuse 3.x (Nick Craig-Wood)
45414
45415 • Use GO386=softfloat instead of deprecated GO386=387 for 386
45416 builds (Nick Craig-Wood)
45417
45418 • Disable IOS builds for the time being (Nick Craig-Wood)
45419
45420 • Androids builds made with up to date NDK (x0b)
45421
45422 • Add an rclone user to the Docker image but don't use it by de‐
45423 fault (cynthia kwok)
45424
45425 • dedupe: Make largest directory primary to minimize data moved (Sak‐
45426 sham Khanna)
45427
45428 • config
45429
45430 • Wrap config library in an interface (Fionera)
45431
45432 • Make config file system pluggable (Nick Craig-Wood)
45433
45434 • --config "" or "/notfound" for in memory config only (Nick Craig-
45435 Wood)
45436
45437 • Clear fs cache of stale entries when altering config (Nick Craig-
45438 Wood)
45439
45440 • copyurl: Add option to print resulting auto-filename (albertony)
45441
45442 • delete: Make --rmdirs obey the filters (Nick Craig-Wood)
45443
45444 • docs - many fixes and reworks from edwardxml, albertony, pvalls,
45445 Ivan Andreev, Evan Harris, buengese, Alexey Tabakman
45446
45447 • encoder/filename - add SCSU as tables (Klaus Post)
45448
45449 • Add multiple paths support to --compare-dest and --copy-dest flag
45450 (K265)
45451
45452 • filter: Make --exclude "dir/" equivalent to --exclude "dir/**"
45453 (Nick Craig-Wood)
45454
45455 • fshttp: Add DSCP support with --dscp for QoS with differentiated
45456 services (Max Sum)
45457
45458 • lib/cache: Add Delete and DeletePrefix methods (Nick Craig-Wood)
45459
45460 • lib/file
45461
45462 • Make pre-allocate detect disk full errors and return them (Nick
45463 Craig-Wood)
45464
45465 • Don't run preallocate concurrently (Nick Craig-Wood)
45466
45467 • Retry preallocate on EINTR (Nick Craig-Wood)
45468
45469 • operations: Made copy and sync operations obey a RetryAfterError
45470 (Ankur Gupta)
45471
45472 • rc
45473
45474 • Add string alternatives for setting options over the rc (Nick
45475 Craig-Wood)
45476
45477 • Add options/local to see the options configured in the context
45478 (Nick Craig-Wood)
45479
45480 • Add _config parameter to set global config for just this rc call
45481 (Nick Craig-Wood)
45482
45483 • Implement passing filter config with _filter parameter (Nick
45484 Craig-Wood)
45485
45486 • Add fscache/clear and fscache/entries to control the fs cache
45487 (Nick Craig-Wood)
45488
45489 • Avoid +Inf value for speed in core/stats (albertony)
45490
45491 • Add a full set of stats to core/stats (Nick Craig-Wood)
45492
45493 • Allow fs= params to be a JSON blob (Nick Craig-Wood)
45494
45495 • rcd: Added systemd notification during the rclone rcd command.
45496 (Naveen Honest Raj)
45497
45498 • rmdirs: Make --rmdirs obey the filters (Nick Craig-Wood)
45499
45500 • version: Show build tags and type of executable (Ivan Andreev)
45501
45502 • Bug Fixes
45503
45504 • install.sh: make it fail on download errors (Ivan Andreev)
45505
45506 • Fix excessive retries missing --max-duration timeout (Nick Craig-
45507 Wood)
45508
45509 • Fix crash when --low-level-retries=0 (Nick Craig-Wood)
45510
45511 • Fix failed token refresh on mounts created via the rc (Nick Craig-
45512 Wood)
45513
45514 • fshttp: Fix bandwidth limiting after bad merge (Nick Craig-Wood)
45515
45516 • lib/atexit
45517
45518 • Unregister interrupt handler once it has fired so users can in‐
45519 terrupt again (Nick Craig-Wood)
45520
45521 • Fix occasional failure to unmount with CTRL-C (Nick Craig-Wood)
45522
45523 • Fix deadlock calling Finalise while Run is running (Nick Craig-
45524 Wood)
45525
45526 • lib/rest: Fix multipart uploads not stopping on context cancel
45527 (Nick Craig-Wood)
45528
45529 • Mount
45530
45531 • Allow mounting to root directory on windows (albertony)
45532
45533 • Improved handling of relative paths on windows (albertony)
45534
45535 • Fix unicode issues with accented characters on macOS (Nick Craig-
45536 Wood)
45537
45538 • Docs: document the new FileSecurity option in WinFsp 2021 (al‐
45539 bertony)
45540
45541 • Docs: add note about volume path syntax on windows (albertony)
45542
45543 • Fix caching of old directories after renaming them (Nick Craig-
45544 Wood)
45545
45546 • Update cgofuse to the latest version to bring in macfuse 4 fix
45547 (Nick Craig-Wood)
45548
45549 • VFS
45550
45551 • --vfs-used-is-size to report used space using recursive scan
45552 (tYYGH)
45553
45554 • Don't set modification time if it was already correct (Nick Craig-
45555 Wood)
45556
45557 • Fix Create causing windows explorer to truncate files on CTRL-C
45558 CTRL-V (Nick Craig-Wood)
45559
45560 • Fix modtimes not updating when writing via cache (Nick Craig-Wood)
45561
45562 • Fix modtimes changing by fractional seconds after upload (Nick
45563 Craig-Wood)
45564
45565 • Fix modtime set if --vfs-cache-mode writes/full and no write (Nick
45566 Craig-Wood)
45567
45568 • Rename files in cache and cancel uploads on directory rename (Nick
45569 Craig-Wood)
45570
45571 • Fix directory renaming by renaming dirs cached in memory (Nick
45572 Craig-Wood)
45573
45574 • Local
45575
45576 • Add flag --local-no-preallocate (David Sze)
45577
45578 • Make nounc an advanced option except on Windows (albertony)
45579
45580 • Don't ignore preallocate disk full errors (Nick Craig-Wood)
45581
45582 • Cache
45583
45584 • Add --fs-cache-expire-duration to control the fs cache (Nick Craig-
45585 Wood)
45586
45587 • Crypt
45588
45589 • Add option to not encrypt data (Vesnyx)
45590
45591 • Log hash ok on upload (albertony)
45592
45593 • Azure Blob
45594
45595 • Add container public access level support. (Manish Kumar)
45596
45597 • B2
45598
45599 • Fix HTML files downloaded via cloudflare (Nick Craig-Wood)
45600
45601 • Box
45602
45603 • Fix transfers getting stuck on token expiry after API change (Nick
45604 Craig-Wood)
45605
45606 • Chunker
45607
45608 • Partially implement no-rename transactions (Maxwell Calman)
45609
45610 • Drive
45611
45612 • Don't stop server side copy if couldn't read description (Nick
45613 Craig-Wood)
45614
45615 • Pass context on to drive SDK - to help with cancellation (Nick
45616 Craig-Wood)
45617
45618 • Dropbox
45619
45620 • Add polling for changes support (Robert Thomas)
45621
45622 • Make --timeout 0 work properly (Nick Craig-Wood)
45623
45624 • Raise priority of rate limited message to INFO to make it more no‐
45625 ticeable (Nick Craig-Wood)
45626
45627 • Fichier
45628
45629 • Implement copy & move (buengese)
45630
45631 • Implement public link (buengese)
45632
45633 • FTP
45634
45635 • Implement Shutdown method (Nick Craig-Wood)
45636
45637 • Close idle connections after --ftp-idle-timeout (1m by default)
45638 (Nick Craig-Wood)
45639
45640 • Make --timeout 0 work properly (Nick Craig-Wood)
45641
45642 • Add --ftp-close-timeout flag for use with awkward ftp servers (Nick
45643 Craig-Wood)
45644
45645 • Retry connections and logins on 421 errors (Nick Craig-Wood)
45646
45647 • Hdfs
45648
45649 • Fix permissions for when directory is created (Lucas Messenger)
45650
45651 • Onedrive
45652
45653 • Make --timeout 0 work properly (Nick Craig-Wood)
45654
45655 • S3
45656
45657 • Fix --s3-profile which wasn't working (Nick Craig-Wood)
45658
45659 • SFTP
45660
45661 • Close idle connections after --sftp-idle-timeout (1m by default)
45662 (Nick Craig-Wood)
45663
45664 • Fix "file not found" errors for read once servers (Nick Craig-Wood)
45665
45666 • Fix SetModTime stat failed: object not found with --sftp-set-mod‐
45667 time=false (Nick Craig-Wood)
45668
45669 • Swift
45670
45671 • Update github.com/ncw/swift to v2.0.0 (Nick Craig-Wood)
45672
45673 • Implement copying large objects (nguyenhuuluan434)
45674
45675 • Union
45676
45677 • Fix crash when using epff policy (Nick Craig-Wood)
45678
45679 • Fix union attempting to update files on a read only file system
45680 (Nick Craig-Wood)
45681
45682 • Refactor to use fspath.SplitFs instead of fs.ParseRemote (Nick
45683 Craig-Wood)
45684
45685 • Fix initialisation broken in refactor (Nick Craig-Wood)
45686
45687 • WebDAV
45688
45689 • Add support for sharepoint with NTLM authentication (Rauno Ots)
45690
45691 • Make sharepoint-ntlm docs more consistent (Alex Chen)
45692
45693 • Improve terminology in sharepoint-ntlm docs (Ivan Andreev)
45694
45695 • Disable HTTP/2 for NTLM authentication (georne)
45696
45697 • Fix sharepoint-ntlm error 401 for parallel actions (Ivan Andreev)
45698
45699 • Check that purged directory really exists (Ivan Andreev)
45700
45701 • Yandex
45702
45703 • Make --timeout 0 work properly (Nick Craig-Wood)
45704
45705 • Zoho
45706
45707 • Replace client id - you will need to rclone config reconnect after
45708 this (buengese)
45709
45710 • Add forgotten setupRegion() to NewFs - this finally fixes regions
45711 other than EU (buengese)
45712
45713 v1.54.1 - 2021-03-08
45714 See commits (https://github.com/rclone/rclone/com‐
45715 pare/v1.54.0...v1.54.1)
45716
45717 • Bug Fixes
45718
45719 • accounting: Fix --bwlimit when up or down is off (Nick Craig-Wood)
45720
45721 • docs
45722
45723 • Fix nesting of brackets and backticks in ftp docs (edwardxml)
45724
45725 • Fix broken link in sftp page (edwardxml)
45726
45727 • Fix typo in crypt.md (Romeo Kienzler)
45728
45729 • Changelog: Correct link to digitalis.io (Alex JOST)
45730
45731 • Replace #file-caching with #vfs-file-caching (Miron Veryanskiy)
45732
45733 • Convert bogus example link to code (edwardxml)
45734
45735 • Remove dead link from rc.md (edwardxml)
45736
45737 • rc: Sync,copy,move: document createEmptySrcDirs parameter (Nick
45738 Craig-Wood)
45739
45740 • lsjson: Fix unterminated JSON in the presence of errors (Nick
45741 Craig-Wood)
45742
45743 • Mount
45744
45745 • Fix mount dropping on macOS by setting --daemon-timeout 10m (Nick
45746 Craig-Wood)
45747
45748 • VFS
45749
45750 • Document simultaneous usage with the same cache shouldn't be used
45751 (Nick Craig-Wood)
45752
45753 • B2
45754
45755 • Automatically raise upload cutoff to avoid spurious error (Nick
45756 Craig-Wood)
45757
45758 • Fix failed to create file system with application key limited to a
45759 prefix (Nick Craig-Wood)
45760
45761 • Drive
45762
45763 • Refer to Shared Drives instead of Team Drives (Nick Craig-Wood)
45764
45765 • Dropbox
45766
45767 • Add scopes to oauth request and optionally "members.read" (Nick
45768 Craig-Wood)
45769
45770 • S3
45771
45772 • Fix failed to create file system with folder level permissions pol‐
45773 icy (Nick Craig-Wood)
45774
45775 • Fix Wasabi HEAD requests returning stale data by using only 1
45776 transport (Nick Craig-Wood)
45777
45778 • Fix shared_credentials_file auth (Dmitry Chepurovskiy)
45779
45780 • Add --s3-no-head to reducing costs docs (Nick Craig-Wood)
45781
45782 • Union
45783
45784 • Fix mkdir at root with remote:/ (Nick Craig-Wood)
45785
45786 • Zoho
45787
45788 • Fix custom client id's (buengese)
45789
45790 v1.54.0 - 2021-02-02
45791 See commits (https://github.com/rclone/rclone/com‐
45792 pare/v1.53.0...v1.54.0)
45793
45794 • New backends
45795
45796 • Compression remote (experimental) (buengese)
45797
45798 • Enterprise File Fabric (Nick Craig-Wood)
45799
45800 • This work was sponsored by Storage Made Easy (https://storage‐
45801 madeeasy.com/)
45802
45803 • HDFS (Hadoop Distributed File System) (Yury Stankevich)
45804
45805 • Zoho workdrive (buengese)
45806
45807 • New Features
45808
45809 • Deglobalise the config (Nick Craig-Wood)
45810
45811 • Global config now read from the context
45812
45813 • This will enable passing of global config via the rc
45814
45815 • This work was sponsored by Digitalis (https://digitalis.io/)
45816
45817 • Add --bwlimit for upload and download (Nick Craig-Wood)
45818
45819 • Obey bwlimit in http Transport for better limiting
45820
45821 • Enhance systemd integration (Hekmon)
45822
45823 • log level identification, manual activation with flag, automatic
45824 systemd launch detection
45825
45826 • Don't compile systemd log integration for non unix systems (Ben‐
45827 jamin Gustin)
45828
45829 • Add a --download flag to md5sum/sha1sum/hashsum to force rclone to
45830 download and hash files locally (lostheli)
45831
45832 • Add --progress-terminal-title to print ETA to terminal title (La‐
45833 Sombra)
45834
45835 • Make backend env vars show in help as the defaults for backend
45836 flags (Nick Craig-Wood)
45837
45838 • build
45839
45840 • Raise minimum go version to go1.12 (Nick Craig-Wood)
45841
45842 • dedupe
45843
45844 • Add --by-hash to dedupe on content hash not file name (Nick
45845 Craig-Wood)
45846
45847 • Add --dedupe-mode list to just list dupes, changing nothing (Nick
45848 Craig-Wood)
45849
45850 • Add warning if used on a remote which can't have duplicate names
45851 (Nick Craig-Wood)
45852
45853 • fs
45854
45855 • Add Shutdown optional method for backends (Nick Craig-Wood)
45856
45857 • When using --files-from check files concurrently (zhucan)
45858
45859 • Accumulate stats when using --dry-run (Ingo Weiss)
45860
45861 • Always show stats when using --dry-run or --interactive (Nick
45862 Craig-Wood)
45863
45864 • Add support for flag --no-console on windows to hide the console
45865 window (albertony)
45866
45867 • genautocomplete: Add support to output to stdout (Ingo)
45868
45869 • ncdu
45870
45871 • Highlight read errors instead of aborting (Claudio Bantaloukas)
45872
45873 • Add sort by average size in directory (Adam Plánský)
45874
45875 • Add toggle option for average s3ize in directory - key 'a' (Adam
45876 Plánský)
45877
45878 • Add empty folder flag into ncdu browser (Adam Plánský)
45879
45880 • Add ! (error) and . (unreadable) file flags to go with e (empty)
45881 (Nick Craig-Wood)
45882
45883 • obscure: Make rclone obscure - ignore newline at end of line (Nick
45884 Craig-Wood)
45885
45886 • operations
45887
45888 • Add logs when need to upload files to set mod times (Nick Craig-
45889 Wood)
45890
45891 • Move and copy log name of the destination object in verbose (Adam
45892 Plánský)
45893
45894 • Add size if known to skipped items and JSON log (Nick Craig-Wood)
45895
45896 • rc
45897
45898 • Prefer actual listener address if using ":port" or "addr:0" only
45899 (Nick Craig-Wood)
45900
45901 • Add listener for finished jobs (Aleksandar Jankovic)
45902
45903 • serve ftp: Add options to enable TLS (Deepak Sah)
45904
45905 • serve http/webdav: Redirect requests to the base url without the /
45906 (Nick Craig-Wood)
45907
45908 • serve restic: Implement object cache (Nick Craig-Wood)
45909
45910 • stats: Add counter for deleted directories (Nick Craig-Wood)
45911
45912 • sync: Only print "There was nothing to transfer" if no errors (Nick
45913 Craig-Wood)
45914
45915 • webui
45916
45917 • Prompt user for updating webui if an update is available (Chai‐
45918 tanya Bankanhal)
45919
45920 • Fix plugins initialization (negative0)
45921
45922 • Bug Fixes
45923
45924 • fs
45925
45926 • Fix nil pointer on copy & move operations directly to remote
45927 (Anagh Kumar Baranwal)
45928
45929 • Fix parsing of .. when joining remotes (Nick Craig-Wood)
45930
45931 • log: Fix enabling systemd logging when using --log-file (Nick
45932 Craig-Wood)
45933
45934 • check
45935
45936 • Make the error count match up in the log message (Nick Craig-
45937 Wood)
45938
45939 • move: Fix data loss when source and destination are the same object
45940 (Nick Craig-Wood)
45941
45942 • operations
45943
45944 • Fix --cutoff-mode hard not cutting off immediately (Nick Craig-
45945 Wood)
45946
45947 • Fix --immutable error message (Nick Craig-Wood)
45948
45949 • sync
45950
45951 • Fix --cutoff-mode soft & cautious so it doesn't end the transfer
45952 early (Nick Craig-Wood)
45953
45954 • Fix --immutable errors retrying many times (Nick Craig-Wood)
45955
45956 • Docs
45957
45958 • Many fixes and a rewrite of the filtering docs (edwardxml)
45959
45960 • Many spelling and grammar fixes (Josh Soref)
45961
45962 • Doc fixes for commands delete, purge, rmdir, rmdirs and mount (al‐
45963 bertony)
45964
45965 • And thanks to these people for many doc fixes too numerous to list
45966
45967 • Ameer Dawood, Antoine GIRARD, Bob Bagwill, Christopher Stewart
45968
45969 • CokeMine, David, Dov Murik, Durval Menezes, Evan Harris, gtorelly
45970
45971 • Ilyess Bachiri, Janne Johansson, Kerry Su, Marcin Zelent,
45972
45973 • Martin Michlmayr, Milly, Sơn Trần-Nguyễn
45974
45975 • Mount
45976
45977 • Update systemd status with cache stats (Hekmon)
45978
45979 • Disable bazil/fuse based mount on macOS (Nick Craig-Wood)
45980
45981 • Make rclone mount actually run rclone cmount under macOS (Nick
45982 Craig-Wood)
45983
45984 • Implement mknod to make NFS file creation work (Nick Craig-Wood)
45985
45986 • Make sure we don't call umount more than once (Nick Craig-Wood)
45987
45988 • More user friendly mounting as network drive on windows (albertony)
45989
45990 • Detect if uid or gid are set in same option string: -o
45991 uid=123,gid=456 (albertony)
45992
45993 • Don't attempt to unmount if fs has been destroyed already (Nick
45994 Craig-Wood)
45995
45996 • VFS
45997
45998 • Fix virtual entries causing deleted files to still appear (Nick
45999 Craig-Wood)
46000
46001 • Fix "file already exists" error for stale cache files (Nick Craig-
46002 Wood)
46003
46004 • Fix file leaks with --vfs-cache-mode full and --buffer-size 0 (Nick
46005 Craig-Wood)
46006
46007 • Fix invalid cache path on windows when using :backend: as remote
46008 (albertony)
46009
46010 • Local
46011
46012 • Continue listing files/folders when a circular symlink is detected
46013 (Manish Gupta)
46014
46015 • New flag --local-zero-size-links to fix sync on some virtual
46016 filesystems (Riccardo Iaconelli)
46017
46018 • Azure Blob
46019
46020 • Add support for service principals (James Lim)
46021
46022 • Add support for managed identities (Brad Ackerman)
46023
46024 • Add examples for access tier (Bob Pusateri)
46025
46026 • Utilize the streaming capabilities from the SDK for multipart up‐
46027 loads (Denis Neuling)
46028
46029 • Fix setting of mime types (Nick Craig-Wood)
46030
46031 • Fix crash when listing outside a SAS URL's root (Nick Craig-Wood)
46032
46033 • Delete archive tier blobs before update if --azureblob-archive-
46034 tier-delete (Nick Craig-Wood)
46035
46036 • Fix crash on startup (Nick Craig-Wood)
46037
46038 • Fix memory usage by upgrading the SDK to v0.13.0 and implementing a
46039 TransferManager (Nick Craig-Wood)
46040
46041 • Require go1.14+ to compile due to SDK changes (Nick Craig-Wood)
46042
46043 • B2
46044
46045 • Make NewObject use less expensive API calls (Nick Craig-Wood)
46046
46047 • This will improve --files-from and restic serve in particular
46048
46049 • Fixed crash on an empty file name (lluuaapp)
46050
46051 • Box
46052
46053 • Fix NewObject for files that differ in case (Nick Craig-Wood)
46054
46055 • Fix finding directories in a case insensitive way (Nick Craig-Wood)
46056
46057 • Chunker
46058
46059 • Skip long local hashing, hash in-transit (fixes) (Ivan Andreev)
46060
46061 • Set Features ReadMimeType to false as Object.MimeType not supported
46062 (Nick Craig-Wood)
46063
46064 • Fix case-insensitive NewObject, test metadata detection (Ivan An‐
46065 dreev)
46066
46067 • Drive
46068
46069 • Implement rclone backend copyid command for copying files by ID
46070 (Nick Craig-Wood)
46071
46072 • Added flag --drive-stop-on-download-limit to stop transfers when
46073 the download limit is exceeded (Anagh Kumar Baranwal)
46074
46075 • Implement CleanUp workaround for team drives (buengese)
46076
46077 • Allow shortcut resolution and creation to be retried (Nick Craig-
46078 Wood)
46079
46080 • Log that emptying the trash can take some time (Nick Craig-Wood)
46081
46082 • Add xdg office icons to xdg desktop files (Pau Rodriguez-Estivill)
46083
46084 • Dropbox
46085
46086 • Add support for viewing shared files and folders (buengese)
46087
46088 • Enable short lived access tokens (Nick Craig-Wood)
46089
46090 • Implement IDer on Objects so rclone lsf etc can read the IDs
46091 (buengese)
46092
46093 • Set Features ReadMimeType to false as Object.MimeType not supported
46094 (Nick Craig-Wood)
46095
46096 • Make malformed_path errors from too long files not retriable (Nick
46097 Craig-Wood)
46098
46099 • Test file name length before upload to fix upload loop (Nick Craig-
46100 Wood)
46101
46102 • Fichier
46103
46104 • Set Features ReadMimeType to true as Object.MimeType is supported
46105 (Nick Craig-Wood)
46106
46107 • FTP
46108
46109 • Add --ftp-disable-msld option to ignore MLSD for really old servers
46110 (Nick Craig-Wood)
46111
46112 • Make --tpslimit apply (Nick Craig-Wood)
46113
46114 • Google Cloud Storage
46115
46116 • Storage class object header support (Laurens Janssen)
46117
46118 • Fix anonymous client to use rclone's HTTP client (Nick Craig-Wood)
46119
46120 • Fix Entry doesn't belong in directory "" (same as directory) - ig‐
46121 noring (Nick Craig-Wood)
46122
46123 • Googlephotos
46124
46125 • New flag --gphotos-include-archived to show archived photos as well
46126 (Nicolas Rueff)
46127
46128 • Jottacloud
46129
46130 • Don't erroneously report support for writing mime types (buengese)
46131
46132 • Add support for Telia Cloud (Patrik Nordlén)
46133
46134 • Mailru
46135
46136 • Accept special folders eg camera-upload (Ivan Andreev)
46137
46138 • Avoid prehashing of large local files (Ivan Andreev)
46139
46140 • Fix uploads after recent changes on server (Ivan Andreev)
46141
46142 • Fix range requests after June 2020 changes on server (Ivan Andreev)
46143
46144 • Fix invalid timestamp on corrupted files (fixes) (Ivan Andreev)
46145
46146 • Remove deprecated protocol quirks (Ivan Andreev)
46147
46148 • Memory
46149
46150 • Fix setting of mime types (Nick Craig-Wood)
46151
46152 • Onedrive
46153
46154 • Add support for China region operated by 21vianet and other region‐
46155 al suppliers (NyaMisty)
46156
46157 • Warn on gateway timeout errors (Nick Craig-Wood)
46158
46159 • Fall back to normal copy if server-side copy unavailable (Alex
46160 Chen)
46161
46162 • Fix server-side copy completely disabled on OneDrive for Business
46163 (Cnly)
46164
46165 • (business only) workaround to replace existing file on server-side
46166 copy (Alex Chen)
46167
46168 • Enhance link creation with expiry, scope, type and password (Nick
46169 Craig-Wood)
46170
46171 • Remove % and # from the set of encoded characters (Alex Chen)
46172
46173 • Support addressing site by server-relative URL (kice)
46174
46175 • Opendrive
46176
46177 • Fix finding directories in a case insensitive way (Nick Craig-Wood)
46178
46179 • Pcloud
46180
46181 • Fix setting of mime types (Nick Craig-Wood)
46182
46183 • Premiumizeme
46184
46185 • Fix finding directories in a case insensitive way (Nick Craig-Wood)
46186
46187 • Qingstor
46188
46189 • Fix error propagation in CleanUp (Nick Craig-Wood)
46190
46191 • Fix rclone cleanup (Nick Craig-Wood)
46192
46193 • S3
46194
46195 • Added --s3-disable-http2 to disable http/2 (Anagh Kumar Baranwal)
46196
46197 • Complete SSE-C implementation (Nick Craig-Wood)
46198
46199 • Fix hashes on small files with AWS:KMS and SSE-C (Nick Craig-
46200 Wood)
46201
46202 • Add MD5 metadata to objects uploaded with SSE-AWS/SSE-C (Nick
46203 Craig-Wood)
46204
46205 • Add --s3-no-head parameter to minimise transactions on upload (Nick
46206 Craig-Wood)
46207
46208 • Update docs with a Reducing Costs section (Nick Craig-Wood)
46209
46210 • Added error handling for error code 429 indicating too many re‐
46211 quests (Anagh Kumar Baranwal)
46212
46213 • Add requester pays option (kelv)
46214
46215 • Fix copy multipart with v2 auth failing with 'SignatureDoesNot‐
46216 Match' (Louis Koo)
46217
46218 • SFTP
46219
46220 • Allow cert based auth via optional pubkey (Stephen Harris)
46221
46222 • Allow user to optionally check server hosts key to add security
46223 (Stephen Harris)
46224
46225 • Defer asking for user passwords until the SSH connection succeeds
46226 (Stephen Harris)
46227
46228 • Remember entered password in AskPass mode (Stephen Harris)
46229
46230 • Implement Shutdown method (Nick Craig-Wood)
46231
46232 • Implement keyboard interactive authentication (Nick Craig-Wood)
46233
46234 • Make --tpslimit apply (Nick Craig-Wood)
46235
46236 • Implement --sftp-use-fstat for unusual SFTP servers (Nick Craig-
46237 Wood)
46238
46239 • Sugarsync
46240
46241 • Fix NewObject for files that differ in case (Nick Craig-Wood)
46242
46243 • Fix finding directories in a case insensitive way (Nick Craig-Wood)
46244
46245 • Swift
46246
46247 • Fix deletion of parts of Static Large Object (SLO) (Nguyễn Hữu
46248 Luân)
46249
46250 • Ensure partially uploaded large files are uploaded unless --swift-
46251 leave-parts-on-error (Nguyễn Hữu Luân)
46252
46253 • Tardigrade
46254
46255 • Upgrade to uplink v1.4.1 (Caleb Case)
46256
46257 • WebDAV
46258
46259 • Updated docs to show streaming to nextcloud is working (Durval
46260 Menezes)
46261
46262 • Yandex
46263
46264 • Set Features WriteMimeType to false as Yandex ignores mime types
46265 (Nick Craig-Wood)
46266
46267 v1.53.4 - 2021-01-20
46268 See commits (https://github.com/rclone/rclone/com‐
46269 pare/v1.53.3...v1.53.4)
46270
46271 • Bug Fixes
46272
46273 • accounting: Fix data race in Transferred() (Maciej Zimnoch)
46274
46275 • build
46276
46277 • Stop tagged releases making a current beta (Nick Craig-Wood)
46278
46279 • Upgrade docker buildx action (Matteo Pietro Dazzi)
46280
46281 • Add -buildmode to cross-compile.go (Nick Craig-Wood)
46282
46283 • Fix docker build by upgrading ilteoood/docker_buildx (Nick Craig-
46284 Wood)
46285
46286 • Revert GitHub actions brew fix since this is now fixed (Nick
46287 Craig-Wood)
46288
46289 • Fix brew install --cask syntax for macOS build (Nick Craig-Wood)
46290
46291 • Update nfpm syntax to fix build of .deb/.rpm packages (Nick
46292 Craig-Wood)
46293
46294 • Fix for Windows build errors (Ivan Andreev)
46295
46296 • fs: Parseduration: fixed tests to use UTC time (Ankur Gupta)
46297
46298 • fshttp: Prevent overlap of HTTP headers in logs (Nathan Collins)
46299
46300 • rc
46301
46302 • Fix core/command giving 500 internal error (Nick Craig-Wood)
46303
46304 • Add Copy method to rc.Params (Nick Craig-Wood)
46305
46306 • Fix 500 error when marshalling errors from core/command (Nick
46307 Craig-Wood)
46308
46309 • plugins: Create plugins files only if webui is enabled. (nega‐
46310 tive0)
46311
46312 • serve http: Fix serving files of unknown length (Nick Craig-Wood)
46313
46314 • serve sftp: Fix authentication on one connection blocking others
46315 (Nick Craig-Wood)
46316
46317 • Mount
46318
46319 • Add optional brew tag to throw an error when using mount in the bi‐
46320 naries installed via Homebrew (Anagh Kumar Baranwal)
46321
46322 • Add "." and ".." to directories to match cmount and expectations
46323 (Nick Craig-Wood)
46324
46325 • VFS
46326
46327 • Make cache dir absolute before using it to fix path too long errors
46328 (Nick Craig-Wood)
46329
46330 • Chunker
46331
46332 • Improve detection of incompatible metadata (Ivan Andreev)
46333
46334 • Google Cloud Storage
46335
46336 • Fix server side copy of large objects (Nick Craig-Wood)
46337
46338 • Jottacloud
46339
46340 • Fix token renewer to fix long uploads (Nick Craig-Wood)
46341
46342 • Fix token refresh failed: is not a regular file error (Nick Craig-
46343 Wood)
46344
46345 • Pcloud
46346
46347 • Only use SHA1 hashes in EU region (Nick Craig-Wood)
46348
46349 • Sharefile
46350
46351 • Undo Fix backend due to API swapping integers for strings (Nick
46352 Craig-Wood)
46353
46354 • WebDAV
46355
46356 • Fix Open Range requests to fix 4shared mount (Nick Craig-Wood)
46357
46358 • Add "Depth: 0" to GET requests to fix bitrix (Nick Craig-Wood)
46359
46360 v1.53.3 - 2020-11-19
46361 See commits (https://github.com/rclone/rclone/com‐
46362 pare/v1.53.2...v1.53.3)
46363
46364 • Bug Fixes
46365
46366 • random: Fix incorrect use of math/rand instead of crypto/rand
46367 CVE-2020-28924 (Nick Craig-Wood)
46368
46369 • Passwords you have generated with rclone config may be insecure
46370
46371 • See issue #4783 (https://github.com/rclone/rclone/issues/4783)
46372 for more details and a checking tool
46373
46374 • random: Seed math/rand in one place with crypto strong seed (Nick
46375 Craig-Wood)
46376
46377 • VFS
46378
46379 • Fix vfs/refresh calls with fs= parameter (Nick Craig-Wood)
46380
46381 • Sharefile
46382
46383 • Fix backend due to API swapping integers for strings (Nick Craig-
46384 Wood)
46385
46386 v1.53.2 - 2020-10-26
46387 See commits (https://github.com/rclone/rclone/com‐
46388 pare/v1.53.1...v1.53.2)
46389
46390 • Bug Fixes
46391
46392 • accounting
46393
46394 • Fix incorrect speed and transferTime in core/stats (Nick Craig-
46395 Wood)
46396
46397 • Stabilize display order of transfers on Windows (Nick Craig-Wood)
46398
46399 • operations
46400
46401 • Fix use of --suffix without --backup-dir (Nick Craig-Wood)
46402
46403 • Fix spurious "--checksum is in use but the source and destination
46404 have no hashes in common" (Nick Craig-Wood)
46405
46406 • build
46407
46408 • Work around GitHub actions brew problem (Nick Craig-Wood)
46409
46410 • Stop using set-env and set-path in the GitHub actions (Nick
46411 Craig-Wood)
46412
46413 • Mount
46414
46415 • mount2: Fix the swapped UID / GID values (Russell Cattelan)
46416
46417 • VFS
46418
46419 • Detect and recover from a file being removed externally from the
46420 cache (Nick Craig-Wood)
46421
46422 • Fix a deadlock vulnerability in downloaders.Close (Leo Luan)
46423
46424 • Fix a race condition in retryFailedResets (Leo Luan)
46425
46426 • Fix missed concurrency control between some item operations and re‐
46427 set (Leo Luan)
46428
46429 • Add exponential backoff during ENOSPC retries (Leo Luan)
46430
46431 • Add a missed update of used cache space (Leo Luan)
46432
46433 • Fix --no-modtime to not attempt to set modtimes (as documented)
46434 (Nick Craig-Wood)
46435
46436 • Local
46437
46438 • Fix sizes and syncing with --links option on Windows (Nick Craig-
46439 Wood)
46440
46441 • Chunker
46442
46443 • Disable ListR to fix missing files on GDrive (workaround) (Ivan An‐
46444 dreev)
46445
46446 • Fix upload over crypt (Ivan Andreev)
46447
46448 • Fichier
46449
46450 • Increase maximum file size from 100GB to 300GB (gyutw)
46451
46452 • Jottacloud
46453
46454 • Remove clientSecret from config when upgrading to token based au‐
46455 thentication (buengese)
46456
46457 • Avoid double url escaping of device/mountpoint (albertony)
46458
46459 • Remove DirMove workaround as it's not required anymore - also
46460 (buengese)
46461
46462 • Mailru
46463
46464 • Fix uploads after recent changes on server (Ivan Andreev)
46465
46466 • Fix range requests after june changes on server (Ivan Andreev)
46467
46468 • Fix invalid timestamp on corrupted files (fixes) (Ivan Andreev)
46469
46470 • Onedrive
46471
46472 • Fix disk usage for sharepoint (Nick Craig-Wood)
46473
46474 • S3
46475
46476 • Add missing regions for AWS (Anagh Kumar Baranwal)
46477
46478 • Seafile
46479
46480 • Fix accessing libraries > 2GB on 32 bit systems (Muffin King)
46481
46482 • SFTP
46483
46484 • Always convert the checksum to lower case (buengese)
46485
46486 • Union
46487
46488 • Create root directories if none exist (Nick Craig-Wood)
46489
46490 v1.53.1 - 2020-09-13
46491 See commits (https://github.com/rclone/rclone/com‐
46492 pare/v1.53.0...v1.53.1)
46493
46494 • Bug Fixes
46495
46496 • accounting: Remove new line from end of --stats-one-line display
46497 (Nick Craig-Wood)
46498
46499 • check
46500
46501 • Add back missing --download flag (Nick Craig-Wood)
46502
46503 • Fix docs (Nick Craig-Wood)
46504
46505 • docs
46506
46507 • Note --log-file does append (Nick Craig-Wood)
46508
46509 • Add full stops for consistency in rclone --help (edwardxml)
46510
46511 • Add Tencent COS to s3 provider list (wjielai)
46512
46513 • Updated mount command to reflect that it requires Go 1.13 or new‐
46514 er (Evan Harris)
46515
46516 • jottacloud: Mention that uploads from local disk will not need to
46517 cache files to disk for md5 calculation (albertony)
46518
46519 • Fix formatting of rc docs page (Nick Craig-Wood)
46520
46521 • build
46522
46523 • Include vendor tar ball in release and fix startdev (Nick Craig-
46524 Wood)
46525
46526 • Fix "Illegal instruction" error for ARMv6 builds (Nick Craig-
46527 Wood)
46528
46529 • Fix architecture name in ARMv7 build (Nick Craig-Wood)
46530
46531 • VFS
46532
46533 • Fix spurious error "vfs cache: failed to _ensure cache EOF" (Nick
46534 Craig-Wood)
46535
46536 • Log an ERROR if we fail to set the file to be sparse (Nick Craig-
46537 Wood)
46538
46539 • Local
46540
46541 • Log an ERROR if we fail to set the file to be sparse (Nick Craig-
46542 Wood)
46543
46544 • Drive
46545
46546 • Re-adds special oauth help text (Tim Gallant)
46547
46548 • Opendrive
46549
46550 • Do not retry 400 errors (Evan Harris)
46551
46552 v1.53.0 - 2020-09-02
46553 See commits (https://github.com/rclone/rclone/com‐
46554 pare/v1.52.0...v1.53.0)
46555
46556 • New Features
46557
46558 • The VFS layer (https://rclone.org/commands/rclone_mount/#vfs-virtu‐
46559 al-file-system) was heavily reworked for this release - see below
46560 for more details
46561
46562 • Interactive mode -i/--interactive (https://rclone.org/docs/#inter‐
46563 active) for destructive operations (fishbullet)
46564
46565 • Add --bwlimit-file (https://rclone.org/docs/#bwlimit-file-band‐
46566 width-spec) flag to limit speeds of individual file transfers (Nick
46567 Craig-Wood)
46568
46569 • Transfers are sorted by start time in the stats and progress output
46570 (Max Sum)
46571
46572 • Make sure backends expand ~ and environment vars in file names they
46573 use (Nick Craig-Wood)
46574
46575 • Add --refresh-times (https://rclone.org/docs/#refresh-times) flag
46576 to set modtimes on hashless backends (Nick Craig-Wood)
46577
46578 • build
46579
46580 • Remove vendor directory in favour of Go modules (Nick Craig-Wood)
46581
46582 • Build with go1.15.x by default (Nick Craig-Wood)
46583
46584 • Drop macOS 386 build as it is no longer supported by go1.15 (Nick
46585 Craig-Wood)
46586
46587 • Add ARMv7 to the supported builds (Nick Craig-Wood)
46588
46589 • Enable rclone cmount on macOS (Nick Craig-Wood)
46590
46591 • Make rclone build with gccgo (Nick Craig-Wood)
46592
46593 • Make rclone build with wasm (Nick Craig-Wood)
46594
46595 • Change beta numbering to be semver compatible (Nick Craig-Wood)
46596
46597 • Add file properties and icon to Windows executable (albertony)
46598
46599 • Add experimental interface for integrating rclone into browsers
46600 (Nick Craig-Wood)
46601
46602 • lib: Add file name compression (Klaus Post)
46603
46604 • rc
46605
46606 • Allow installation and use of plugins and test plugins with
46607 rclone-webui (Chaitanya Bankanhal)
46608
46609 • Add reverse proxy pluginsHandler for serving plugins (Chaitanya
46610 Bankanhal)
46611
46612 • Add mount/listmounts option for listing current mounts (Chaitanya
46613 Bankanhal)
46614
46615 • Add operations/uploadfile to upload a file through rc using en‐
46616 coding multipart/form-data (Chaitanya Bankanhal)
46617
46618 • Add core/command to execute rclone terminal commands. (Chaitanya
46619 Bankanhal)
46620
46621 • rclone check
46622
46623 • Add reporting of filenames for same/missing/changed (Nick Craig-
46624 Wood)
46625
46626 • Make check command obey --dry-run/-i/--interactive (Nick Craig-
46627 Wood)
46628
46629 • Make check do --checkers files concurrently (Nick Craig-Wood)
46630
46631 • Retry downloads if they fail when using the --download flag (Nick
46632 Craig-Wood)
46633
46634 • Make it show stats by default (Nick Craig-Wood)
46635
46636 • rclone obscure: Allow obscure command to accept password on STDIN
46637 (David Ibarra)
46638
46639 • rclone config
46640
46641 • Set RCLONE_CONFIG_DIR for use in config files and subprocesses
46642 (Nick Craig-Wood)
46643
46644 • Reject remote names starting with a dash. (jtagcat)
46645
46646 • rclone cryptcheck: Add reporting of filenames for same/miss‐
46647 ing/changed (Nick Craig-Wood)
46648
46649 • rclone dedupe: Make it obey the --size-only flag for duplicate de‐
46650 tection (Nick Craig-Wood)
46651
46652 • rclone link: Add --expire and --unlink flags (Roman Kredentser)
46653
46654 • rclone mkdir: Warn when using mkdir on remotes which can't have
46655 empty directories (Nick Craig-Wood)
46656
46657 • rclone rc: Allow JSON parameters to simplify command line usage
46658 (Nick Craig-Wood)
46659
46660 • rclone serve ftp
46661
46662 • Don't compile on < go1.13 after dependency update (Nick Craig-
46663 Wood)
46664
46665 • Add error message if auth proxy fails (Nick Craig-Wood)
46666
46667 • Use refactored goftp.io/server library for binary shrink (Nick
46668 Craig-Wood)
46669
46670 • rclone serve restic: Expose interfaces so that rclone can be used
46671 as a library from within restic (Jack)
46672
46673 • rclone sync: Add --track-renames-strategy leaf (Nick Craig-Wood)
46674
46675 • rclone touch: Add ability to set nanosecond resolution times (Nick
46676 Craig-Wood)
46677
46678 • rclone tree: Remove -i shorthand for --noindent as it conflicts
46679 with -i/--interactive (Nick Craig-Wood)
46680
46681 • Bug Fixes
46682
46683 • accounting
46684
46685 • Fix documentation for speed/speedAvg (Nick Craig-Wood)
46686
46687 • Fix elapsed time not show actual time since beginning (Chaitanya
46688 Bankanhal)
46689
46690 • Fix deadlock in stats printing (Nick Craig-Wood)
46691
46692 • build
46693
46694 • Fix file handle leak in GitHub release tool (Garrett Squire)
46695
46696 • rclone check: Fix successful retries with --download counting er‐
46697 rors (Nick Craig-Wood)
46698
46699 • rclone dedupe: Fix logging to be easier to understand (Nick Craig-
46700 Wood)
46701
46702 • Mount
46703
46704 • Warn macOS users that mount implementation is changing (Nick Craig-
46705 Wood)
46706
46707 • to test the new implementation use rclone cmount instead of
46708 rclone mount
46709
46710 • this is because the library rclone uses has dropped macOS support
46711
46712 • rc interface
46713
46714 • Add call for unmount all (Chaitanya Bankanhal)
46715
46716 • Make mount/mount remote control take vfsOpt option (Nick Craig-
46717 Wood)
46718
46719 • Add mountOpt to mount/mount (Nick Craig-Wood)
46720
46721 • Add VFS and Mount options to mount/listmounts (Nick Craig-Wood)
46722
46723 • Catch panics in cgofuse initialization and turn into error messages
46724 (Nick Craig-Wood)
46725
46726 • Always supply stat information in Readdir (Nick Craig-Wood)
46727
46728 • Add support for reading unknown length files using direct IO (Win‐
46729 dows) (Nick Craig-Wood)
46730
46731 • Fix On Windows don't add -o uid/gid=-1 if user supplies -o uid/gid.
46732 (Nick Craig-Wood)
46733
46734 • Fix macOS losing directory contents in cmount (Nick Craig-Wood)
46735
46736 • Fix volume name broken in recent refactor (Nick Craig-Wood)
46737
46738 • VFS
46739
46740 • Implement partial reads for --vfs-cache-mode full (Nick Craig-Wood)
46741
46742 • Add --vfs-writeback option to delay writes back to cloud storage
46743 (Nick Craig-Wood)
46744
46745 • Add --vfs-read-ahead parameter for use with --vfs-cache-mode full
46746 (Nick Craig-Wood)
46747
46748 • Restart pending uploads on restart of the cache (Nick Craig-Wood)
46749
46750 • Support synchronous cache space recovery upon ENOSPC (Leo Luan)
46751
46752 • Allow ReadAt and WriteAt to run concurrently with themselves (Nick
46753 Craig-Wood)
46754
46755 • Change modtime of file before upload to current (Rob Calistri)
46756
46757 • Recommend --vfs-cache-modes writes on backends which can't stream
46758 (Nick Craig-Wood)
46759
46760 • Add an optional fs parameter to vfs rc methods (Nick Craig-Wood)
46761
46762 • Fix errors when using > 260 char files in the cache in Windows
46763 (Nick Craig-Wood)
46764
46765 • Fix renaming of items while they are being uploaded (Nick Craig-
46766 Wood)
46767
46768 • Fix very high load caused by slow directory listings (Nick Craig-
46769 Wood)
46770
46771 • Fix renamed files not being uploaded with --vfs-cache-mode minimal
46772 (Nick Craig-Wood)
46773
46774 • Fix directory locking caused by slow directory listings (Nick
46775 Craig-Wood)
46776
46777 • Fix saving from chrome without --vfs-cache-mode writes (Nick Craig-
46778 Wood)
46779
46780 • Local
46781
46782 • Add --local-no-updated to provide a consistent view of changing ob‐
46783 jects (Nick Craig-Wood)
46784
46785 • Add --local-no-set-modtime option to prevent modtime changes (tyhu‐
46786 ber1)
46787
46788 • Fix race conditions updating and reading Object metadata (Nick
46789 Craig-Wood)
46790
46791 • Cache
46792
46793 • Make any created backends be cached to fix rc problems (Nick Craig-
46794 Wood)
46795
46796 • Fix dedupe on caches wrapping drives (Nick Craig-Wood)
46797
46798 • Crypt
46799
46800 • Add --crypt-server-side-across-configs flag (Nick Craig-Wood)
46801
46802 • Make any created backends be cached to fix rc problems (Nick Craig-
46803 Wood)
46804
46805 • Alias
46806
46807 • Make any created backends be cached to fix rc problems (Nick Craig-
46808 Wood)
46809
46810 • Azure Blob
46811
46812 • Don't compile on < go1.13 after dependency update (Nick Craig-Wood)
46813
46814 • B2
46815
46816 • Implement server-side copy for files > 5GB (Nick Craig-Wood)
46817
46818 • Cancel in progress multipart uploads and copies on rclone exit
46819 (Nick Craig-Wood)
46820
46821 • Note that b2's encoding now allows but rclone's hasn't changed
46822 (Nick Craig-Wood)
46823
46824 • Fix transfers when using download_url (Nick Craig-Wood)
46825
46826 • Box
46827
46828 • Implement rclone cleanup (buengese)
46829
46830 • Cancel in progress multipart uploads and copies on rclone exit
46831 (Nick Craig-Wood)
46832
46833 • Allow authentication with access token (David)
46834
46835 • Chunker
46836
46837 • Make any created backends be cached to fix rc problems (Nick Craig-
46838 Wood)
46839
46840 • Drive
46841
46842 • Add rclone backend drives to list shared drives (teamdrives) (Nick
46843 Craig-Wood)
46844
46845 • Implement rclone backend untrash (Nick Craig-Wood)
46846
46847 • Work around drive bug which didn't set modtime of copied docs (Nick
46848 Craig-Wood)
46849
46850 • Added --drive-starred-only to only show starred files (Jay McEn‐
46851 tire)
46852
46853 • Deprecate --drive-alternate-export as it is no longer needed (the‐
46854 mylogin)
46855
46856 • Fix duplication of Google docs on server-side copy (Nick Craig-
46857 Wood)
46858
46859 • Fix "panic: send on closed channel" when recycling dir entries
46860 (Nick Craig-Wood)
46861
46862 • Dropbox
46863
46864 • Add copyright detector info in limitations section in the docs
46865 (Alex Guerrero)
46866
46867 • Fix rclone link by removing expires parameter (Nick Craig-Wood)
46868
46869 • Fichier
46870
46871 • Detect Flood detected: IP Locked error and sleep for 30s (Nick
46872 Craig-Wood)
46873
46874 • FTP
46875
46876 • Add explicit TLS support (Heiko Bornholdt)
46877
46878 • Add support for --dump bodies and --dump auth for debugging (Nick
46879 Craig-Wood)
46880
46881 • Fix interoperation with pure-ftpd (Nick Craig-Wood)
46882
46883 • Google Cloud Storage
46884
46885 • Add support for anonymous access (Kai Lüke)
46886
46887 • Jottacloud
46888
46889 • Bring back legacy authentication for use with whitelabel versions
46890 (buengese)
46891
46892 • Switch to new api root - also implement a very ugly workaround for
46893 the DirMove failures (buengese)
46894
46895 • Onedrive
46896
46897 • Rework cancel of multipart uploads on rclone exit (Nick Craig-Wood)
46898
46899 • Implement rclone cleanup (Nick Craig-Wood)
46900
46901 • Add --onedrive-no-versions flag to remove old versions (Nick Craig-
46902 Wood)
46903
46904 • Pcloud
46905
46906 • Implement rclone link for public link creation (buengese)
46907
46908 • Qingstor
46909
46910 • Cancel in progress multipart uploads on rclone exit (Nick Craig-
46911 Wood)
46912
46913 • S3
46914
46915 • Preserve metadata when doing multipart copy (Nick Craig-Wood)
46916
46917 • Cancel in progress multipart uploads and copies on rclone exit
46918 (Nick Craig-Wood)
46919
46920 • Add rclone link for public link sharing (Roman Kredentser)
46921
46922 • Add rclone backend restore command to restore objects from GLACIER
46923 (Nick Craig-Wood)
46924
46925 • Add rclone cleanup and rclone backend cleanup to clean unfinished
46926 multipart uploads (Nick Craig-Wood)
46927
46928 • Add rclone backend list-multipart-uploads to list unfinished multi‐
46929 part uploads (Nick Craig-Wood)
46930
46931 • Add --s3-max-upload-parts support (Kamil Trzciński)
46932
46933 • Add --s3-no-check-bucket for minimising rclone transactions and
46934 perms (Nick Craig-Wood)
46935
46936 • Add --s3-profile and --s3-shared-credentials-file options (Nick
46937 Craig-Wood)
46938
46939 • Use regional s3 us-east-1 endpoint (David)
46940
46941 • Add Scaleway provider (Vincent Feltz)
46942
46943 • Update IBM COS endpoints (Egor Margineanu)
46944
46945 • Reduce the default --s3-copy-cutoff to < 5GB for Backblaze S3 com‐
46946 patibility (Nick Craig-Wood)
46947
46948 • Fix detection of bucket existing (Nick Craig-Wood)
46949
46950 • SFTP
46951
46952 • Use the absolute path instead of the relative path for listing for
46953 improved compatibility (Nick Craig-Wood)
46954
46955 • Add --sftp-subsystem and --sftp-server-command options (aus)
46956
46957 • Swift
46958
46959 • Fix dangling large objects breaking the listing (Nick Craig-Wood)
46960
46961 • Fix purge not deleting directory markers (Nick Craig-Wood)
46962
46963 • Fix update multipart object removing all of its own parts (Nick
46964 Craig-Wood)
46965
46966 • Fix missing hash from object returned from upload (Nick Craig-Wood)
46967
46968 • Tardigrade
46969
46970 • Upgrade to uplink v1.2.0 (Kaloyan Raev)
46971
46972 • Union
46973
46974 • Fix writing with the all policy (Nick Craig-Wood)
46975
46976 • WebDAV
46977
46978 • Fix directory creation with 4shared (Nick Craig-Wood)
46979
46980 v1.52.3 - 2020-08-07
46981 See commits (https://github.com/rclone/rclone/com‐
46982 pare/v1.52.2...v1.52.3)
46983
46984 • Bug Fixes
46985
46986 • docs
46987
46988 • Disable smart typography (e.g. en-dash) in MANUAL.* and man page
46989 (Nick Craig-Wood)
46990
46991 • Update install.md to reflect minimum Go version (Evan Harris)
46992
46993 • Update install from source instructions (Nick Craig-Wood)
46994
46995 • make_manual: Support SOURCE_DATE_EPOCH (Morten Linderud)
46996
46997 • log: Fix --use-json-log going to stderr not --log-file on Windows
46998 (Nick Craig-Wood)
46999
47000 • serve dlna: Fix file list on Samsung Series 6+ TVs (Matteo Pietro
47001 Dazzi)
47002
47003 • sync: Fix deadlock with --track-renames-strategy modtime (Nick
47004 Craig-Wood)
47005
47006 • Cache
47007
47008 • Fix moveto/copyto remote:file remote:file2 (Nick Craig-Wood)
47009
47010 • Drive
47011
47012 • Stop using root_folder_id as a cache (Nick Craig-Wood)
47013
47014 • Make dangling shortcuts appear in listings (Nick Craig-Wood)
47015
47016 • Drop "Disabling ListR" messages down to debug (Nick Craig-Wood)
47017
47018 • Workaround and policy for Google Drive API (Dmitry Ustalov)
47019
47020 • FTP
47021
47022 • Add note to docs about home vs root directory selection (Nick
47023 Craig-Wood)
47024
47025 • Onedrive
47026
47027 • Fix reverting to Copy when Move would have worked (Nick Craig-Wood)
47028
47029 • Avoid comma rendered in URL in onedrive.md (Kevin)
47030
47031 • Pcloud
47032
47033 • Fix oauth on European region "eapi.pcloud.com" (Nick Craig-Wood)
47034
47035 • S3
47036
47037 • Fix bucket Region auto detection when Region unset in config (Nick
47038 Craig-Wood)
47039
47040 v1.52.2 - 2020-06-24
47041 See commits (https://github.com/rclone/rclone/com‐
47042 pare/v1.52.1...v1.52.2)
47043
47044 • Bug Fixes
47045
47046 • build
47047
47048 • Fix docker release build action (Nick Craig-Wood)
47049
47050 • Fix custom timezone in Docker image (NoLooseEnds)
47051
47052 • check: Fix misleading message which printed errors instead of dif‐
47053 ferences (Nick Craig-Wood)
47054
47055 • errors: Add WSAECONNREFUSED and more to the list of retriable Win‐
47056 dows errors (Nick Craig-Wood)
47057
47058 • rcd: Fix incorrect prometheus metrics (Gary Kim)
47059
47060 • serve restic: Fix flags so they use environment variables (Nick
47061 Craig-Wood)
47062
47063 • serve webdav: Fix flags so they use environment variables (Nick
47064 Craig-Wood)
47065
47066 • sync: Fix --track-renames-strategy modtime (Nick Craig-Wood)
47067
47068 • Drive
47069
47070 • Fix not being able to delete a directory with a trashed shortcut
47071 (Nick Craig-Wood)
47072
47073 • Fix creating a directory inside a shortcut (Nick Craig-Wood)
47074
47075 • Fix --drive-impersonate with cached root_folder_id (Nick Craig-
47076 Wood)
47077
47078 • SFTP
47079
47080 • Fix SSH key PEM loading (Zac Rubin)
47081
47082 • Swift
47083
47084 • Speed up deletes by not retrying segment container deletes (Nick
47085 Craig-Wood)
47086
47087 • Tardigrade
47088
47089 • Upgrade to uplink v1.1.1 (Caleb Case)
47090
47091 • WebDAV
47092
47093 • Fix free/used display for rclone about/df for certain backends
47094 (Nick Craig-Wood)
47095
47096 v1.52.1 - 2020-06-10
47097 See commits (https://github.com/rclone/rclone/com‐
47098 pare/v1.52.0...v1.52.1)
47099
47100 • Bug Fixes
47101
47102 • lib/file: Fix SetSparse on Windows 7 which fixes downloads of files
47103 > 250MB (Nick Craig-Wood)
47104
47105 • build
47106
47107 • Update go.mod to go1.14 to enable -mod=vendor build (Nick Craig-
47108 Wood)
47109
47110 • Remove quicktest from Dockerfile (Nick Craig-Wood)
47111
47112 • Build Docker images with GitHub actions (Matteo Pietro Dazzi)
47113
47114 • Update Docker build workflows (Nick Craig-Wood)
47115
47116 • Set user_allow_other in /etc/fuse.conf in the Docker image (Nick
47117 Craig-Wood)
47118
47119 • Fix xgo build after go1.14 go.mod update (Nick Craig-Wood)
47120
47121 • docs
47122
47123 • Add link to source and modified time to footer of every page
47124 (Nick Craig-Wood)
47125
47126 • Remove manually set dates and use git dates instead (Nick Craig-
47127 Wood)
47128
47129 • Minor tense, punctuation, brevity and positivity changes for the
47130 home page (edwardxml)
47131
47132 • Remove leading slash in page reference in footer when present
47133 (Nick Craig-Wood)
47134
47135 • Note commands which need obscured input in the docs (Nick Craig-
47136 Wood)
47137
47138 • obscure: Write more help as we are referencing it elsewhere (Nick
47139 Craig-Wood)
47140
47141 • VFS
47142
47143 • Fix OS vs Unix path confusion - fixes ChangeNotify on Windows (Nick
47144 Craig-Wood)
47145
47146 • Drive
47147
47148 • Fix missing items when listing using --fast-list / ListR (Nick
47149 Craig-Wood)
47150
47151 • Putio
47152
47153 • Fix panic on Object.Open (Cenk Alti)
47154
47155 • S3
47156
47157 • Fix upload of single files into buckets without create permission
47158 (Nick Craig-Wood)
47159
47160 • Fix --header-upload (Nick Craig-Wood)
47161
47162 • Tardigrade
47163
47164 • Fix listing bug by upgrading to v1.0.7
47165
47166 • Set UserAgent to rclone (Caleb Case)
47167
47168 v1.52.0 - 2020-05-27
47169 Special thanks to Martin Michlmayr for proof reading and correcting all
47170 the docs and Edward Barker for helping re-write the front page.
47171
47172 See commits (https://github.com/rclone/rclone/com‐
47173 pare/v1.51.0...v1.52.0)
47174
47175 • New backends
47176
47177 • Tardigrade (https://rclone.org/tardigrade/) backend for use with
47178 storj.io (Caleb Case)
47179
47180 • Union (https://rclone.org/union/) re-write to have multiple
47181 writable remotes (Max Sum)
47182
47183 • Seafile for Seafile server (Fred @creativeprojects)
47184
47185 • New commands
47186
47187 • backend: command for backend-specific commands (see backends) (Nick
47188 Craig-Wood)
47189
47190 • cachestats: Deprecate in favour of rclone backend stats cache:
47191 (Nick Craig-Wood)
47192
47193 • dbhashsum: Deprecate in favour of rclone hashsum DropboxHash (Nick
47194 Craig-Wood)
47195
47196 • New Features
47197
47198 • Add --header-download and --header-upload flags for setting HTTP
47199 headers when uploading/downloading (Tim Gallant)
47200
47201 • Add --header flag to add HTTP headers to every HTTP transaction
47202 (Nick Craig-Wood)
47203
47204 • Add --check-first to do all checking before starting transfers
47205 (Nick Craig-Wood)
47206
47207 • Add --track-renames-strategy for configurable matching criteria for
47208 --track-renames (Bernd Schoolmann)
47209
47210 • Add --cutoff-mode hard,soft,cautious (Shing Kit Chan & Franklyn
47211 Tackitt)
47212
47213 • Filter flags (e.g. --files-from -) can read from stdin (fishbul‐
47214 let)
47215
47216 • Add --error-on-no-transfer option (Jon Fautley)
47217
47218 • Implement --order-by xxx,mixed for copying some small and some big
47219 files (Nick Craig-Wood)
47220
47221 • Allow --max-backlog to be negative meaning as large as possible
47222 (Nick Craig-Wood)
47223
47224 • Added --no-unicode-normalization flag to allow Unicode filenames to
47225 remain unique (Ben Zenker)
47226
47227 • Allow --min-age/--max-age to take a date as well as a duration
47228 (Nick Craig-Wood)
47229
47230 • Add rename statistics for file and directory renames (Nick Craig-
47231 Wood)
47232
47233 • Add statistics output to JSON log (reddi)
47234
47235 • Make stats be printed on non-zero exit code (Nick Craig-Wood)
47236
47237 • When running --password-command allow use of stdin (Sébastien
47238 Gross)
47239
47240 • Stop empty strings being a valid remote path (Nick Craig-Wood)
47241
47242 • accounting: support WriterTo for less memory copying (Nick Craig-
47243 Wood)
47244
47245 • build
47246
47247 • Update to use go1.14 for the build (Nick Craig-Wood)
47248
47249 • Add -trimpath to release build for reproduceable builds (Nick
47250 Craig-Wood)
47251
47252 • Remove GOOS and GOARCH from Dockerfile (Brandon Philips)
47253
47254 • config
47255
47256 • Fsync the config file after writing to save more reliably (Nick
47257 Craig-Wood)
47258
47259 • Add --obscure and --no-obscure flags to config create/update
47260 (Nick Craig-Wood)
47261
47262 • Make config show take remote: as well as remote (Nick Craig-Wood)
47263
47264 • copyurl: Add --no-clobber flag (Denis)
47265
47266 • delete: Added --rmdirs flag to delete directories as well (Kush)
47267
47268 • filter: Added --files-from-raw flag (Ankur Gupta)
47269
47270 • genautocomplete: Add support for fish shell (Matan Rosenberg)
47271
47272 • log: Add support for syslog LOCAL facilities (Patryk Jakuszew)
47273
47274 • lsjson: Add --hash-type parameter and use it in lsf to speed up
47275 hashing (Nick Craig-Wood)
47276
47277 • rc
47278
47279 • Add -o/--opt and -a/--arg for more structured input (Nick Craig-
47280 Wood)
47281
47282 • Implement backend/command for running backend-specific commands
47283 remotely (Nick Craig-Wood)
47284
47285 • Add mount/mount command for starting rclone mount via the API
47286 (Chaitanya)
47287
47288 • rcd: Add Prometheus metrics support (Gary Kim)
47289
47290 • serve http
47291
47292 • Added a --template flag for user defined markup (calistri)
47293
47294 • Add Last-Modified headers to files and directories (Nick Craig-
47295 Wood)
47296
47297 • serve sftp: Add support for multiple host keys by repeating --key
47298 flag (Maxime Suret)
47299
47300 • touch: Add --localtime flag to make --timestamp localtime not UTC
47301 (Nick Craig-Wood)
47302
47303 • Bug Fixes
47304
47305 • accounting
47306
47307 • Restore "Max number of stats groups reached" log line (Michał
47308 Matczuk)
47309
47310 • Correct exitcode on Transfer Limit Exceeded flag. (Anuar
47311 Serdaliyev)
47312
47313 • Reset bytes read during copy retry (Ankur Gupta)
47314
47315 • Fix race clearing stats (Nick Craig-Wood)
47316
47317 • copy: Only create empty directories when they don't exist on the
47318 remote (Ishuah Kariuki)
47319
47320 • dedupe: Stop dedupe deleting files with identical IDs (Nick Craig-
47321 Wood)
47322
47323 • oauth
47324
47325 • Use custom http client so that --no-check-certificate is honored
47326 by oauth token fetch (Mark Spieth)
47327
47328 • Replace deprecated oauth2.NoContext (Lars Lehtonen)
47329
47330 • operations
47331
47332 • Fix setting the timestamp on Windows for multithread copy (Nick
47333 Craig-Wood)
47334
47335 • Make rcat obey --ignore-checksum (Nick Craig-Wood)
47336
47337 • Make --max-transfer more accurate (Nick Craig-Wood)
47338
47339 • rc
47340
47341 • Fix dropped error (Lars Lehtonen)
47342
47343 • Fix misplaced http server config (Xiaoxing Ye)
47344
47345 • Disable duplicate log (ElonH)
47346
47347 • serve dlna
47348
47349 • Cds: don't specify childCount at all when unknown (Dan Walters)
47350
47351 • Cds: use modification time as date in dlna metadata (Dan Walters)
47352
47353 • serve restic: Fix tests after restic project removed vendoring
47354 (Nick Craig-Wood)
47355
47356 • sync
47357
47358 • Fix incorrect "nothing to transfer" message using --delete-before
47359 (Nick Craig-Wood)
47360
47361 • Only create empty directories when they don't exist on the remote
47362 (Ishuah Kariuki)
47363
47364 • Mount
47365
47366 • Add --async-read flag to disable asynchronous reads (Nick Craig-
47367 Wood)
47368
47369 • Ignore --allow-root flag with a warning as it has been removed up‐
47370 stream (Nick Craig-Wood)
47371
47372 • Warn if --allow-non-empty used on Windows and clarify docs (Nick
47373 Craig-Wood)
47374
47375 • Constrain to go1.13 or above otherwise bazil.org/fuse fails to com‐
47376 pile (Nick Craig-Wood)
47377
47378 • Fix fail because of too long volume name (evileye)
47379
47380 • Report 1PB free for unknown disk sizes (Nick Craig-Wood)
47381
47382 • Map more rclone errors into file systems errors (Nick Craig-Wood)
47383
47384 • Fix disappearing cwd problem (Nick Craig-Wood)
47385
47386 • Use ReaddirPlus on Windows to improve directory listing performance
47387 (Nick Craig-Wood)
47388
47389 • Send a hint as to whether the filesystem is case insensitive or not
47390 (Nick Craig-Wood)
47391
47392 • Add rc command mount/types (Nick Craig-Wood)
47393
47394 • Change maximum leaf name length to 1024 bytes (Nick Craig-Wood)
47395
47396 • VFS
47397
47398 • Add --vfs-read-wait and --vfs-write-wait flags to control time
47399 waiting for a sequential read/write (Nick Craig-Wood)
47400
47401 • Change default --vfs-read-wait to 20ms (it was 5ms and not config‐
47402 urable) (Nick Craig-Wood)
47403
47404 • Make df output more consistent on a rclone mount. (Yves G)
47405
47406 • Report 1PB free for unknown disk sizes (Nick Craig-Wood)
47407
47408 • Fix race condition caused by unlocked reading of Dir.path (Nick
47409 Craig-Wood)
47410
47411 • Make File lock and Dir lock not overlap to avoid deadlock (Nick
47412 Craig-Wood)
47413
47414 • Implement lock ordering between File and Dir to eliminate deadlocks
47415 (Nick Craig-Wood)
47416
47417 • Factor the vfs cache into its own package (Nick Craig-Wood)
47418
47419 • Pin the Fs in use in the Fs cache (Nick Craig-Wood)
47420
47421 • Add SetSys() methods to Node to allow caching stuff on a node (Nick
47422 Craig-Wood)
47423
47424 • Ignore file not found errors from Hash in Read.Release (Nick Craig-
47425 Wood)
47426
47427 • Fix hang in read wait code (Nick Craig-Wood)
47428
47429 • Local
47430
47431 • Speed up multi thread downloads by using sparse files on Windows
47432 (Nick Craig-Wood)
47433
47434 • Implement --local-no-sparse flag for disabling sparse files (Nick
47435 Craig-Wood)
47436
47437 • Implement rclone backend noop for testing purposes (Nick Craig-
47438 Wood)
47439
47440 • Fix "file not found" errors on post transfer Hash calculation (Nick
47441 Craig-Wood)
47442
47443 • Cache
47444
47445 • Implement rclone backend stats command (Nick Craig-Wood)
47446
47447 • Fix Server Side Copy with Temp Upload (Brandon McNama)
47448
47449 • Remove Unused Functions (Lars Lehtonen)
47450
47451 • Disable race tests until bbolt is fixed (Nick Craig-Wood)
47452
47453 • Move methods used for testing into test file (greatroar)
47454
47455 • Add Pin and Unpin and canonicalised lookup (Nick Craig-Wood)
47456
47457 • Use proper import path go.etcd.io/bbolt (Robert-André Mauchin)
47458
47459 • Crypt
47460
47461 • Calculate hashes for uploads from local disk (Nick Craig-Wood)
47462
47463 • This allows crypted Jottacloud uploads without using local disk
47464
47465 • This means crypted s3/b2 uploads will now have hashes
47466
47467 • Added rclone backend decode/encode commands to replicate function‐
47468 ality of cryptdecode (Anagh Kumar Baranwal)
47469
47470 • Get rid of the unused Cipher interface as it obfuscated the code
47471 (Nick Craig-Wood)
47472
47473 • Azure Blob
47474
47475 • Implement streaming of unknown sized files so rcat is now supported
47476 (Nick Craig-Wood)
47477
47478 • Implement memory pooling to control memory use (Nick Craig-Wood)
47479
47480 • Add --azureblob-disable-checksum flag (Nick Craig-Wood)
47481
47482 • Retry InvalidBlobOrBlock error as it may indicate block concurrency
47483 problems (Nick Craig-Wood)
47484
47485 • Remove unused Object.parseTimeString() (Lars Lehtonen)
47486
47487 • Fix permission error on SAS URL limited to container (Nick Craig-
47488 Wood)
47489
47490 • B2
47491
47492 • Add support for --header-upload and --header-download (Tim Gallant)
47493
47494 • Ignore directory markers at the root also (Nick Craig-Wood)
47495
47496 • Force the case of the SHA1 to lowercase (Nick Craig-Wood)
47497
47498 • Remove unused largeUpload.clearUploadURL() (Lars Lehtonen)
47499
47500 • Box
47501
47502 • Add support for --header-upload and --header-download (Tim Gallant)
47503
47504 • Implement About to read size used (Nick Craig-Wood)
47505
47506 • Add token renew function for jwt auth (David Bramwell)
47507
47508 • Added support for interchangeable root folder for Box backend
47509 (Sunil Patra)
47510
47511 • Remove unnecessary iat from jws claims (David)
47512
47513 • Drive
47514
47515 • Follow shortcuts by default, skip with --drive-skip-shortcuts (Nick
47516 Craig-Wood)
47517
47518 • Implement rclone backend shortcut command for creating shortcuts
47519 (Nick Craig-Wood)
47520
47521 • Added rclone backend command to change service_account_file and
47522 chunk_size (Anagh Kumar Baranwal)
47523
47524 • Fix missing files when using --fast-list and --drive-shared-with-me
47525 (Nick Craig-Wood)
47526
47527 • Fix duplicate items when using --drive-shared-with-me (Nick Craig-
47528 Wood)
47529
47530 • Extend --drive-stop-on-upload-limit to respond to teamDriveFileLim‐
47531 itExceeded. (harry)
47532
47533 • Don't delete files with multiple parents to avoid data loss (Nick
47534 Craig-Wood)
47535
47536 • Server side copy docs use default description if empty (Nick Craig-
47537 Wood)
47538
47539 • Dropbox
47540
47541 • Make error insufficient space to be fatal (harry)
47542
47543 • Add info about required redirect url (Elan Ruusamäe)
47544
47545 • Fichier
47546
47547 • Add support for --header-upload and --header-download (Tim Gallant)
47548
47549 • Implement custom pacer to deal with the new rate limiting
47550 (buengese)
47551
47552 • FTP
47553
47554 • Fix lockup when using concurrency limit on failed connections (Nick
47555 Craig-Wood)
47556
47557 • Fix lockup on failed upload when using concurrency limit (Nick
47558 Craig-Wood)
47559
47560 • Fix lockup on Close failures when using concurrency limit (Nick
47561 Craig-Wood)
47562
47563 • Work around pureftp sending spurious 150 messages (Nick Craig-Wood)
47564
47565 • Google Cloud Storage
47566
47567 • Add support for --header-upload and --header-download (Nick Craig-
47568 Wood)
47569
47570 • Add ARCHIVE storage class to help (Adam Stroud)
47571
47572 • Ignore directory markers at the root (Nick Craig-Wood)
47573
47574 • Googlephotos
47575
47576 • Make the start year configurable (Daven)
47577
47578 • Add support for --header-upload and --header-download (Tim Gallant)
47579
47580 • Create feature/favorites directory (Brandon Philips)
47581
47582 • Fix "concurrent map write" error (Nick Craig-Wood)
47583
47584 • Don't put an image in error message (Nick Craig-Wood)
47585
47586 • HTTP
47587
47588 • Improved directory listing with new template from Caddy project
47589 (calisro)
47590
47591 • Jottacloud
47592
47593 • Implement --jottacloud-trashed-only (buengese)
47594
47595 • Add support for --header-upload and --header-download (Tim Gallant)
47596
47597 • Use RawURLEncoding when decoding base64 encoded login token
47598 (buengese)
47599
47600 • Implement cleanup (buengese)
47601
47602 • Update docs regarding cleanup, removed remains from old auth, and
47603 added warning about special mountpoints. (albertony)
47604
47605 • Mailru
47606
47607 • Describe 2FA requirements (valery1707)
47608
47609 • Onedrive
47610
47611 • Implement --onedrive-server-side-across-configs (Nick Craig-Wood)
47612
47613 • Add support for --header-upload and --header-download (Tim Gallant)
47614
47615 • Fix occasional 416 errors on multipart uploads (Nick Craig-Wood)
47616
47617 • Added maximum chunk size limit warning in the docs (Harry)
47618
47619 • Fix missing drive on config (Nick Craig-Wood)
47620
47621 • Make error quotaLimitReached to be fatal (harry)
47622
47623 • Opendrive
47624
47625 • Add support for --header-upload and --header-download (Tim Gallant)
47626
47627 • Pcloud
47628
47629 • Added support for interchangeable root folder for pCloud backend
47630 (Sunil Patra)
47631
47632 • Add support for --header-upload and --header-download (Tim Gallant)
47633
47634 • Fix initial config "Auth state doesn't match" message (Nick Craig-
47635 Wood)
47636
47637 • Premiumizeme
47638
47639 • Add support for --header-upload and --header-download (Tim Gallant)
47640
47641 • Prune unused functions (Lars Lehtonen)
47642
47643 • Putio
47644
47645 • Add support for --header-upload and --header-download (Nick Craig-
47646 Wood)
47647
47648 • Make downloading files use the rclone http Client (Nick Craig-Wood)
47649
47650 • Fix parsing of remotes with leading and trailing / (Nick Craig-
47651 Wood)
47652
47653 • Qingstor
47654
47655 • Make rclone cleanup remove pending multipart uploads older than 24h
47656 (Nick Craig-Wood)
47657
47658 • Try harder to cancel failed multipart uploads (Nick Craig-Wood)
47659
47660 • Prune multiUploader.list() (Lars Lehtonen)
47661
47662 • Lint fix (Lars Lehtonen)
47663
47664 • S3
47665
47666 • Add support for --header-upload and --header-download (Tim Gallant)
47667
47668 • Use memory pool for buffer allocations (Maciej Zimnoch)
47669
47670 • Add SSE-C support for AWS, Ceph, and MinIO (Jack Anderson)
47671
47672 • Fail fast multipart upload (Michał Matczuk)
47673
47674 • Report errors on bucket creation (mkdir) correctly (Nick Craig-
47675 Wood)
47676
47677 • Specify that Minio supports URL encoding in listings (Nick Craig-
47678 Wood)
47679
47680 • Added 500 as retryErrorCode (Michał Matczuk)
47681
47682 • Use --low-level-retries as the number of SDK retries (Aleksandar
47683 Janković)
47684
47685 • Fix multipart abort context (Aleksandar Jankovic)
47686
47687 • Replace deprecated session.New() with session.NewSession() (Lars
47688 Lehtonen)
47689
47690 • Use the provided size parameter when allocating a new memory pool
47691 (Joachim Brandon LeBlanc)
47692
47693 • Use rclone's low level retries instead of AWS SDK to fix listing
47694 retries (Nick Craig-Wood)
47695
47696 • Ignore directory markers at the root also (Nick Craig-Wood)
47697
47698 • Use single memory pool (Michał Matczuk)
47699
47700 • Do not resize buf on put to memBuf (Michał Matczuk)
47701
47702 • Improve docs for --s3-disable-checksum (Nick Craig-Wood)
47703
47704 • Don't leak memory or tokens in edge cases for multipart upload
47705 (Nick Craig-Wood)
47706
47707 • Seafile
47708
47709 • Implement 2FA (Fred)
47710
47711 • SFTP
47712
47713 • Added --sftp-pem-key to support inline key files (calisro)
47714
47715 • Fix post transfer copies failing with 0 size when using set_mod‐
47716 time=false (Nick Craig-Wood)
47717
47718 • Sharefile
47719
47720 • Add support for --header-upload and --header-download (Tim Gallant)
47721
47722 • Sugarsync
47723
47724 • Add support for --header-upload and --header-download (Tim Gallant)
47725
47726 • Swift
47727
47728 • Add support for --header-upload and --header-download (Nick Craig-
47729 Wood)
47730
47731 • Fix cosmetic issue in error message (Martin Michlmayr)
47732
47733 • Union
47734
47735 • Implement multiple writable remotes (Max Sum)
47736
47737 • Fix server-side copy (Max Sum)
47738
47739 • Implement ListR (Max Sum)
47740
47741 • Enable ListR when upstreams contain local (Max Sum)
47742
47743 • WebDAV
47744
47745 • Add support for --header-upload and --header-download (Tim Gallant)
47746
47747 • Fix X-OC-Mtime header for Transip compatibility (Nick Craig-Wood)
47748
47749 • Report full and consistent usage with about (Yves G)
47750
47751 • Yandex
47752
47753 • Add support for --header-upload and --header-download (Tim Gallant)
47754
47755 v1.51.0 - 2020-02-01
47756 • New backends
47757
47758 • Memory (https://rclone.org/memory/) (Nick Craig-Wood)
47759
47760 • Sugarsync (https://rclone.org/sugarsync/) (Nick Craig-Wood)
47761
47762 • New Features
47763
47764 • Adjust all backends to have --backend-encoding parameter (Nick
47765 Craig-Wood)
47766
47767 • this enables the encoding for special characters to be adjusted
47768 or disabled
47769
47770 • Add --max-duration flag to control the maximum duration of a trans‐
47771 fer session (boosh)
47772
47773 • Add --expect-continue-timeout flag, default 1s (Nick Craig-Wood)
47774
47775 • Add --no-check-dest flag for copying without testing the destina‐
47776 tion (Nick Craig-Wood)
47777
47778 • Implement --order-by flag to order transfers (Nick Craig-Wood)
47779
47780 • accounting
47781
47782 • Don't show entries in both transferring and checking (Nick Craig-
47783 Wood)
47784
47785 • Add option to delete stats (Aleksandar Jankovic)
47786
47787 • build
47788
47789 • Compress the test builds with gzip (Nick Craig-Wood)
47790
47791 • Implement a framework for starting test servers during tests
47792 (Nick Craig-Wood)
47793
47794 • cmd: Always print elapsed time to tenth place seconds in progress
47795 (Gary Kim)
47796
47797 • config
47798
47799 • Add --password-command to allow dynamic config password (Damon
47800 Permezel)
47801
47802 • Give config questions default values (Nick Craig-Wood)
47803
47804 • Check a remote exists when creating a new one (Nick Craig-Wood)
47805
47806 • copyurl: Add --stdout flag to write to stdout (Nick Craig-Wood)
47807
47808 • dedupe: Implement keep smallest too (Nick Craig-Wood)
47809
47810 • hashsum: Add flag --base64 flag (landall)
47811
47812 • lsf: Speed up on s3/swift/etc by not reading mimetype by default
47813 (Nick Craig-Wood)
47814
47815 • lsjson: Add --no-mimetype flag (Nick Craig-Wood)
47816
47817 • rc: Add methods to turn on blocking and mutex profiling (Nick
47818 Craig-Wood)
47819
47820 • rcd
47821
47822 • Adding group parameter to stats (Chaitanya)
47823
47824 • Move webgui apart; option to disable browser (Xiaoxing Ye)
47825
47826 • serve sftp: Add support for public key with auth proxy (Paul Tins‐
47827 ley)
47828
47829 • stats: Show deletes in stats and hide zero stats (anuar45)
47830
47831 • Bug Fixes
47832
47833 • accounting
47834
47835 • Fix error counter counting multiple times (Ankur Gupta)
47836
47837 • Fix error count shown as checks (Cnly)
47838
47839 • Clear finished transfer in stats-reset (Maciej Zimnoch)
47840
47841 • Added StatsInfo locking in statsGroups sum function (Michał
47842 Matczuk)
47843
47844 • asyncreader: Fix EOF error (buengese)
47845
47846 • check: Fix --one-way recursing more directories than it needs to
47847 (Nick Craig-Wood)
47848
47849 • chunkedreader: Disable hash calculation for first segment (Nick
47850 Craig-Wood)
47851
47852 • config
47853
47854 • Do not open browser on headless on drive/gcs/google photos (Xi‐
47855 aoxing Ye)
47856
47857 • SetValueAndSave ignore error if config section does not exist yet
47858 (buengese)
47859
47860 • cmd: Fix completion with an encrypted config (Danil Semelenov)
47861
47862 • dbhashsum: Stop it returning UNSUPPORTED on dropbox (Nick Craig-
47863 Wood)
47864
47865 • dedupe: Add missing modes to help string (Nick Craig-Wood)
47866
47867 • operations
47868
47869 • Fix dedupe continuing on errors like insufficientFilePersimmon
47870 (SezalAgrawal)
47871
47872 • Clear accounting before low level retry (Maciej Zimnoch)
47873
47874 • Write debug message when hashes could not be checked (Ole Schütt)
47875
47876 • Move interface assertion to tests to remove pflag dependency
47877 (Nick Craig-Wood)
47878
47879 • Make NewOverrideObjectInfo public and factor uses (Nick Craig-
47880 Wood)
47881
47882 • proxy: Replace use of bcrypt with sha256 (Nick Craig-Wood)
47883
47884 • vendor
47885
47886 • Update bazil.org/fuse to fix FreeBSD 12.1 (Nick Craig-Wood)
47887
47888 • Update github.com/t3rm1n4l/go-mega to fix mega "illegal base64
47889 data at input byte 22" (Nick Craig-Wood)
47890
47891 • Update termbox-go to fix ncdu command on FreeBSD (Kuang-che Wu)
47892
47893 • Update t3rm1n4l/go-mega - fixes mega: couldn't login: crypto/aes:
47894 invalid key size 0 (Nick Craig-Wood)
47895
47896 • Mount
47897
47898 • Enable async reads for a 20% speedup (Nick Craig-Wood)
47899
47900 • Replace use of WriteAt with Write for cache mode >= writes and
47901 O_APPEND (Brett Dutro)
47902
47903 • Make sure we call unmount when exiting (Nick Craig-Wood)
47904
47905 • Don't build on go1.10 as bazil/fuse no longer supports it (Nick
47906 Craig-Wood)
47907
47908 • When setting dates discard out of range dates (Nick Craig-Wood)
47909
47910 • VFS
47911
47912 • Add a newly created file straight into the directory (Nick Craig-
47913 Wood)
47914
47915 • Only calculate one hash for reads for a speedup (Nick Craig-Wood)
47916
47917 • Make ReadAt for non cached files work better with non-sequential
47918 reads (Nick Craig-Wood)
47919
47920 • Fix edge cases when reading ModTime from file (Nick Craig-Wood)
47921
47922 • Make sure existing files opened for write show correct size (Nick
47923 Craig-Wood)
47924
47925 • Don't cache the path in RW file objects to fix renaming (Nick
47926 Craig-Wood)
47927
47928 • Fix rename of open files when using the VFS cache (Nick Craig-Wood)
47929
47930 • When renaming files in the cache, rename the cache item in memory
47931 too (Nick Craig-Wood)
47932
47933 • Fix open file renaming on drive when using --vfs-cache-mode writes
47934 (Nick Craig-Wood)
47935
47936 • Fix incorrect modtime for mv into mount with --vfs-cache-modes
47937 writes (Nick Craig-Wood)
47938
47939 • On rename, rename in cache too if the file exists (Anagh Kumar
47940 Baranwal)
47941
47942 • Local
47943
47944 • Make source file being updated errors be NoLowLevelRetry errors
47945 (Nick Craig-Wood)
47946
47947 • Fix update of hidden files on Windows (Nick Craig-Wood)
47948
47949 • Cache
47950
47951 • Follow move of upstream library github.com/coreos/bbolt
47952 github.com/etcd-io/bbolt (Nick Craig-Wood)
47953
47954 • Fix fatal error: concurrent map writes (Nick Craig-Wood)
47955
47956 • Crypt
47957
47958 • Reorder the filename encryption options (Thomas Eales)
47959
47960 • Correctly handle trailing dot (buengese)
47961
47962 • Chunker
47963
47964 • Reduce length of temporary suffix (Ivan Andreev)
47965
47966 • Drive
47967
47968 • Add --drive-stop-on-upload-limit flag to stop syncs when upload
47969 limit reached (Nick Craig-Wood)
47970
47971 • Add --drive-use-shared-date to use date file was shared instead of
47972 modified date (Garry McNulty)
47973
47974 • Make sure invalid auth for teamdrives always reports an error (Nick
47975 Craig-Wood)
47976
47977 • Fix --fast-list when using appDataFolder (Nick Craig-Wood)
47978
47979 • Use multipart resumable uploads for streaming and uploads in mount
47980 (Nick Craig-Wood)
47981
47982 • Log an ERROR if an incomplete search is returned (Nick Craig-Wood)
47983
47984 • Hide dangerous config from the configurator (Nick Craig-Wood)
47985
47986 • Dropbox
47987
47988 • Treat insufficient_space errors as non retriable errors (Nick
47989 Craig-Wood)
47990
47991 • Jottacloud
47992
47993 • Use new auth method used by official client (buengese)
47994
47995 • Add URL to generate Login Token to config wizard (Nick Craig-Wood)
47996
47997 • Add support whitelabel versions (buengese)
47998
47999 • Koofr
48000
48001 • Use rclone HTTP client. (jaKa)
48002
48003 • Onedrive
48004
48005 • Add Sites.Read.All permission (Benjamin Richter)
48006
48007 • Add support "Retry-After" header (Motonori IWAMURO)
48008
48009 • Opendrive
48010
48011 • Implement --opendrive-chunk-size (Nick Craig-Wood)
48012
48013 • S3
48014
48015 • Re-implement multipart upload to fix memory issues (Nick Craig-
48016 Wood)
48017
48018 • Add --s3-copy-cutoff for size to switch to multipart copy (Nick
48019 Craig-Wood)
48020
48021 • Add new region Asia Pacific (Hong Kong) (Outvi V)
48022
48023 • Reduce memory usage streaming files by reducing max stream upload
48024 size (Nick Craig-Wood)
48025
48026 • Add --s3-list-chunk option for bucket listing (Thomas Kriechbaumer)
48027
48028 • Force path style bucket access to off for AWS deprecation (Nick
48029 Craig-Wood)
48030
48031 • Use AWS web identity role provider if available (Tennix)
48032
48033 • Add StackPath Object Storage Support (Dave Koston)
48034
48035 • Fix ExpiryWindow value (Aleksandar Jankovic)
48036
48037 • Fix DisableChecksum condition (Aleksandar Janković)
48038
48039 • Fix URL decoding of NextMarker (Nick Craig-Wood)
48040
48041 • SFTP
48042
48043 • Add --sftp-skip-links to skip symlinks and non regular files (Nick
48044 Craig-Wood)
48045
48046 • Retry Creation of Connection (Sebastian Brandt)
48047
48048 • Fix "failed to parse private key file: ssh: not an encrypted key"
48049 error (Nick Craig-Wood)
48050
48051 • Open files for update write only to fix AWS SFTP interop (Nick
48052 Craig-Wood)
48053
48054 • Swift
48055
48056 • Reserve segments of dynamic large object when delete objects in
48057 container what was enabled versioning. (Nguyễn Hữu Luân)
48058
48059 • Fix parsing of X-Object-Manifest (Nick Craig-Wood)
48060
48061 • Update OVH API endpoint (unbelauscht)
48062
48063 • WebDAV
48064
48065 • Make nextcloud only upload SHA1 checksums (Nick Craig-Wood)
48066
48067 • Fix case of "Bearer" in Authorization: header to agree with RFC
48068 (Nick Craig-Wood)
48069
48070 • Add Referer header to fix problems with WAFs (Nick Craig-Wood)
48071
48072 v1.50.2 - 2019-11-19
48073 • Bug Fixes
48074
48075 • accounting: Fix memory leak on retries operations (Nick Craig-Wood)
48076
48077 • Drive
48078
48079 • Fix listing of the root directory with drive.files scope (Nick
48080 Craig-Wood)
48081
48082 • Fix --drive-root-folder-id with team/shared drives (Nick Craig-
48083 Wood)
48084
48085 v1.50.1 - 2019-11-02
48086 • Bug Fixes
48087
48088 • hash: Fix accidentally changed hash names for DropboxHash and
48089 CRC-32 (Nick Craig-Wood)
48090
48091 • fshttp: Fix error reporting on tpslimit token bucket errors (Nick
48092 Craig-Wood)
48093
48094 • fshttp: Don't print token bucket errors on context cancelled (Nick
48095 Craig-Wood)
48096
48097 • Local
48098
48099 • Fix listings of . on Windows (Nick Craig-Wood)
48100
48101 • Onedrive
48102
48103 • Fix DirMove/Move after Onedrive change (Xiaoxing Ye)
48104
48105 v1.50.0 - 2019-10-26
48106 • New backends
48107
48108 • Citrix Sharefile (https://rclone.org/sharefile/) (Nick Craig-Wood)
48109
48110 • Chunker (https://rclone.org/chunker/) - an overlay backend to split
48111 files into smaller parts (Ivan Andreev)
48112
48113 • Mail.ru Cloud (https://rclone.org/mailru/) (Ivan Andreev)
48114
48115 • New Features
48116
48117 • encodings (Fabian Möller & Nick Craig-Wood)
48118
48119 • All backends now use file name encoding to ensure any file name
48120 can be written to any backend.
48121
48122 • See the restricted file name docs (https://rclone.org/over‐
48123 view/#restricted-filenames) for more info and the local backend
48124 docs.
48125
48126 • Some file names may look different in rclone if you are using any
48127 control characters in names or unicode FULLWIDTH symbols
48128 (https://en.wikipedia.org/wiki/Halfwidth_and_Full‐
48129 width_Forms_(Unicode_block)).
48130
48131 • build
48132
48133 • Update to use go1.13 for the build (Nick Craig-Wood)
48134
48135 • Drop support for go1.9 (Nick Craig-Wood)
48136
48137 • Build rclone with GitHub actions (Nick Craig-Wood)
48138
48139 • Convert python scripts to python3 (Nick Craig-Wood)
48140
48141 • Swap Azure/go-ansiterm for mattn/go-colorable (Nick Craig-Wood)
48142
48143 • Dockerfile fixes (Matei David)
48144
48145 • Add plugin support (https://github.com/rclone/rclone/blob/mas‐
48146 ter/CONTRIBUTING.md#writing-a-plugin) for backends and commands
48147 (Richard Patel)
48148
48149 • config
48150
48151 • Use alternating Red/Green in config to make more obvious (Nick
48152 Craig-Wood)
48153
48154 • contrib
48155
48156 • Add sample DLNA server Docker Compose manifest. (pataquets)
48157
48158 • Add sample WebDAV server Docker Compose manifest. (pataquets)
48159
48160 • copyurl
48161
48162 • Add --auto-filename flag for using file name from URL in destina‐
48163 tion path (Denis)
48164
48165 • serve dlna:
48166
48167 • Many compatibility improvements (Dan Walters)
48168
48169 • Support for external srt subtitles (Dan Walters)
48170
48171 • rc
48172
48173 • Added command core/quit (Saksham Khanna)
48174
48175 • Bug Fixes
48176
48177 • sync
48178
48179 • Make --update/-u not transfer files that haven't changed (Nick
48180 Craig-Wood)
48181
48182 • Free objects after they come out of the transfer pipe to save
48183 memory (Nick Craig-Wood)
48184
48185 • Fix --files-from without --no-traverse doing a recursive scan
48186 (Nick Craig-Wood)
48187
48188 • operations
48189
48190 • Fix accounting for server-side copies (Nick Craig-Wood)
48191
48192 • Display 'All duplicates removed' only if dedupe successful (Sezal
48193 Agrawal)
48194
48195 • Display 'Deleted X extra copies' only if dedupe successful (Sezal
48196 Agrawal)
48197
48198 • accounting
48199
48200 • Only allow up to 100 completed transfers in the accounting list
48201 to save memory (Nick Craig-Wood)
48202
48203 • Cull the old time ranges when possible to save memory (Nick
48204 Craig-Wood)
48205
48206 • Fix panic due to server-side copy fallback (Ivan Andreev)
48207
48208 • Fix memory leak noticeable for transfers of large numbers of ob‐
48209 jects (Nick Craig-Wood)
48210
48211 • Fix total duration calculation (Nick Craig-Wood)
48212
48213 • cmd
48214
48215 • Fix environment variables not setting command line flags (Nick
48216 Craig-Wood)
48217
48218 • Make autocomplete compatible with bash's posix mode for macOS
48219 (Danil Semelenov)
48220
48221 • Make --progress work in git bash on Windows (Nick Craig-Wood)
48222
48223 • Fix 'compopt: command not found' on autocomplete on macOS (Danil
48224 Semelenov)
48225
48226 • config
48227
48228 • Fix setting of non top level flags from environment variables
48229 (Nick Craig-Wood)
48230
48231 • Check config names more carefully and report errors (Nick Craig-
48232 Wood)
48233
48234 • Remove error: can't use --size-only and --ignore-size together.
48235 (Nick Craig-Wood)
48236
48237 • filter: Prevent mixing options when --files-from is in use (Michele
48238 Caci)
48239
48240 • serve sftp: Fix crash on unsupported operations (e.g. Readlink)
48241 (Nick Craig-Wood)
48242
48243 • Mount
48244
48245 • Allow files of unknown size to be read properly (Nick Craig-Wood)
48246
48247 • Skip tests on <= 2 CPUs to avoid lockup (Nick Craig-Wood)
48248
48249 • Fix panic on File.Open (Nick Craig-Wood)
48250
48251 • Fix "mount_fusefs: -o timeout=: option not supported" on FreeBSD
48252 (Nick Craig-Wood)
48253
48254 • Don't pass huge filenames (>4k) to FUSE as it can't cope (Nick
48255 Craig-Wood)
48256
48257 • VFS
48258
48259 • Add flag --vfs-case-insensitive for windows/macOS mounts (Ivan An‐
48260 dreev)
48261
48262 • Make objects of unknown size readable through the VFS (Nick Craig-
48263 Wood)
48264
48265 • Move writeback of dirty data out of close() method into its own
48266 method (FlushWrites) and remove close() call from Flush() (Brett
48267 Dutro)
48268
48269 • Stop empty dirs disappearing when renamed on bucket-based remotes
48270 (Nick Craig-Wood)
48271
48272 • Stop change notify polling clearing so much of the directory cache
48273 (Nick Craig-Wood)
48274
48275 • Azure Blob
48276
48277 • Disable logging to the Windows event log (Nick Craig-Wood)
48278
48279 • B2
48280
48281 • Remove unverified: prefix on sha1 to improve interop (e.g. with
48282 CyberDuck) (Nick Craig-Wood)
48283
48284 • Box
48285
48286 • Add options to get access token via JWT auth (David)
48287
48288 • Drive
48289
48290 • Disable HTTP/2 by default to work around INTERNAL_ERROR problems
48291 (Nick Craig-Wood)
48292
48293 • Make sure that drive root ID is always canonical (Nick Craig-Wood)
48294
48295 • Fix --drive-shared-with-me from the root with lsand --fast-list
48296 (Nick Craig-Wood)
48297
48298 • Fix ChangeNotify polling for shared drives (Nick Craig-Wood)
48299
48300 • Fix change notify polling when using appDataFolder (Nick Craig-
48301 Wood)
48302
48303 • Dropbox
48304
48305 • Make disallowed filenames errors not retry (Nick Craig-Wood)
48306
48307 • Fix nil pointer exception on restricted files (Nick Craig-Wood)
48308
48309 • Fichier
48310
48311 • Fix accessing files > 2GB on 32 bit systems (Nick Craig-Wood)
48312
48313 • FTP
48314
48315 • Allow disabling EPSV mode (Jon Fautley)
48316
48317 • HTTP
48318
48319 • HEAD directory entries in parallel to speedup (Nick Craig-Wood)
48320
48321 • Add --http-no-head to stop rclone doing HEAD in listings (Nick
48322 Craig-Wood)
48323
48324 • Putio
48325
48326 • Add ability to resume uploads (Cenk Alti)
48327
48328 • S3
48329
48330 • Fix signature v2_auth headers (Anthony Rusdi)
48331
48332 • Fix encoding for control characters (Nick Craig-Wood)
48333
48334 • Only ask for URL encoded directory listings if we need them on Ceph
48335 (Nick Craig-Wood)
48336
48337 • Add option for multipart failure behaviour (Aleksandar Jankovic)
48338
48339 • Support for multipart copy (庄天翼)
48340
48341 • Fix nil pointer reference if no metadata returned for object (Nick
48342 Craig-Wood)
48343
48344 • SFTP
48345
48346 • Fix --sftp-ask-password trying to contact the ssh agent (Nick
48347 Craig-Wood)
48348
48349 • Fix hashes of files with backslashes (Nick Craig-Wood)
48350
48351 • Include more ciphers with --sftp-use-insecure-cipher (Carlos Fer‐
48352 reyra)
48353
48354 • WebDAV
48355
48356 • Parse and return Sharepoint error response (Henning Surmeier)
48357
48358 v1.49.5 - 2019-10-05
48359 • Bug Fixes
48360
48361 • Revert back to go1.12.x for the v1.49.x builds as go1.13.x was
48362 causing issues (Nick Craig-Wood)
48363
48364 • Fix rpm packages by using master builds of nfpm (Nick Craig-Wood)
48365
48366 • Fix macOS build after brew changes (Nick Craig-Wood)
48367
48368 v1.49.4 - 2019-09-29
48369 • Bug Fixes
48370
48371 • cmd/rcd: Address ZipSlip vulnerability (Richard Patel)
48372
48373 • accounting: Fix file handle leak on errors (Nick Craig-Wood)
48374
48375 • oauthutil: Fix security problem when running with two users on the
48376 same machine (Nick Craig-Wood)
48377
48378 • FTP
48379
48380 • Fix listing of an empty root returning: error dir not found (Nick
48381 Craig-Wood)
48382
48383 • S3
48384
48385 • Fix SetModTime on GLACIER/ARCHIVE objects and implement set/get
48386 tier (Nick Craig-Wood)
48387
48388 v1.49.3 - 2019-09-15
48389 • Bug Fixes
48390
48391 • accounting
48392
48393 • Fix total duration calculation (Aleksandar Jankovic)
48394
48395 • Fix "file already closed" on transfer retries (Nick Craig-Wood)
48396
48397 v1.49.2 - 2019-09-08
48398 • New Features
48399
48400 • build: Add Docker workflow support (Alfonso Montero)
48401
48402 • Bug Fixes
48403
48404 • accounting: Fix locking in Transfer to avoid deadlock with
48405 --progress (Nick Craig-Wood)
48406
48407 • docs: Fix template argument for mktemp in install.sh (Cnly)
48408
48409 • operations: Fix -u/--update with google photos / files of unknown
48410 size (Nick Craig-Wood)
48411
48412 • rc: Fix docs for config/create /update /password (Nick Craig-Wood)
48413
48414 • Google Cloud Storage
48415
48416 • Fix need for elevated permissions on SetModTime (Nick Craig-Wood)
48417
48418 v1.49.1 - 2019-08-28
48419 • Bug Fixes
48420
48421 • config: Fix generated passwords being stored as empty password
48422 (Nick Craig-Wood)
48423
48424 • rcd: Added missing parameter for web-gui info logs. (Chaitanya)
48425
48426 • Googlephotos
48427
48428 • Fix crash on error response (Nick Craig-Wood)
48429
48430 • Onedrive
48431
48432 • Fix crash on error response (Nick Craig-Wood)
48433
48434 v1.49.0 - 2019-08-26
48435 • New backends
48436
48437 • 1fichier (https://rclone.org/fichier/) (Laura Hausmann)
48438
48439 • Google Photos (https://rclone.org/googlephotos/) (Nick Craig-Wood)
48440
48441 • Putio (https://rclone.org/putio/) (Cenk Alti)
48442
48443 • premiumize.me (https://rclone.org/premiumizeme/) (Nick Craig-Wood)
48444
48445 • New Features
48446
48447 • Experimental web GUI (https://rclone.org/gui/) (Chaitanya Bankan‐
48448 hal)
48449
48450 • Implement --compare-dest & --copy-dest (yparitcher)
48451
48452 • Implement --suffix without --backup-dir for backup to current dir
48453 (yparitcher)
48454
48455 • config reconnect to re-login (re-run the oauth login) for the back‐
48456 end. (Nick Craig-Wood)
48457
48458 • config userinfo to discover which user you are logged in as. (Nick
48459 Craig-Wood)
48460
48461 • config disconnect to disconnect you (log out) from the backend.
48462 (Nick Craig-Wood)
48463
48464 • Add --use-json-log for JSON logging (justinalin)
48465
48466 • Add context propagation to rclone (Aleksandar Jankovic)
48467
48468 • Reworking internal statistics interfaces so they work with rc jobs
48469 (Aleksandar Jankovic)
48470
48471 • Add Higher units for ETA (AbelThar)
48472
48473 • Update rclone logos to new design (Andreas Chlupka)
48474
48475 • hash: Add CRC-32 support (Cenk Alti)
48476
48477 • help showbackend: Fixed advanced option category when there are no
48478 standard options (buengese)
48479
48480 • ncdu: Display/Copy to Clipboard Current Path (Gary Kim)
48481
48482 • operations:
48483
48484 • Run hashing operations in parallel (Nick Craig-Wood)
48485
48486 • Don't calculate checksums when using --ignore-checksum (Nick
48487 Craig-Wood)
48488
48489 • Check transfer hashes when using --size-only mode (Nick Craig-
48490 Wood)
48491
48492 • Disable multi thread copy for local to local copies (Nick Craig-
48493 Wood)
48494
48495 • Debug successful hashes as well as failures (Nick Craig-Wood)
48496
48497 • rc
48498
48499 • Add ability to stop async jobs (Aleksandar Jankovic)
48500
48501 • Return current settings if core/bwlimit called without parameters
48502 (Nick Craig-Wood)
48503
48504 • Rclone-WebUI integration with rclone (Chaitanya Bankanhal)
48505
48506 • Added command line parameter to control the cross origin resource
48507 sharing (CORS) in the rcd. (Security Improvement) (Chaitanya
48508 Bankanhal)
48509
48510 • Add anchor tags to the docs so links are consistent (Nick Craig-
48511 Wood)
48512
48513 • Remove _async key from input parameters after parsing so later
48514 operations won't get confused (buengese)
48515
48516 • Add call to clear stats (Aleksandar Jankovic)
48517
48518 • rcd
48519
48520 • Auto-login for web-gui (Chaitanya Bankanhal)
48521
48522 • Implement --baseurl for rcd and web-gui (Chaitanya Bankanhal)
48523
48524 • serve dlna
48525
48526 • Only select interfaces which can multicast for SSDP (Nick Craig-
48527 Wood)
48528
48529 • Add more builtin mime types to cover standard audio/video (Nick
48530 Craig-Wood)
48531
48532 • Fix missing mime types on Android causing missing videos (Nick
48533 Craig-Wood)
48534
48535 • serve ftp
48536
48537 • Refactor to bring into line with other serve commands (Nick
48538 Craig-Wood)
48539
48540 • Implement --auth-proxy (Nick Craig-Wood)
48541
48542 • serve http: Implement --baseurl (Nick Craig-Wood)
48543
48544 • serve restic: Implement --baseurl (Nick Craig-Wood)
48545
48546 • serve sftp
48547
48548 • Implement auth proxy (Nick Craig-Wood)
48549
48550 • Fix detection of whether server is authorized (Nick Craig-Wood)
48551
48552 • serve webdav
48553
48554 • Implement --baseurl (Nick Craig-Wood)
48555
48556 • Support --auth-proxy (Nick Craig-Wood)
48557
48558 • Bug Fixes
48559
48560 • Make "bad record MAC" a retriable error (Nick Craig-Wood)
48561
48562 • copyurl: Fix copying files that return HTTP errors (Nick Craig-
48563 Wood)
48564
48565 • march: Fix checking sub-directories when using --no-traverse
48566 (buengese)
48567
48568 • rc
48569
48570 • Fix unmarshalable http.AuthFn in options and put in test for mar‐
48571 shalability (Nick Craig-Wood)
48572
48573 • Move job expire flags to rc to fix initialization problem (Nick
48574 Craig-Wood)
48575
48576 • Fix --loopback with rc/list and others (Nick Craig-Wood)
48577
48578 • rcat: Fix slowdown on systems with multiple hashes (Nick Craig-
48579 Wood)
48580
48581 • rcd: Fix permissions problems on cache directory with web gui down‐
48582 load (Nick Craig-Wood)
48583
48584 • Mount
48585
48586 • Default --daemon-timeout to 15 minutes on macOS and FreeBSD (Nick
48587 Craig-Wood)
48588
48589 • Update docs to show mounting from root OK for bucket-based (Nick
48590 Craig-Wood)
48591
48592 • Remove nonseekable flag from write files (Nick Craig-Wood)
48593
48594 • VFS
48595
48596 • Make write without cache more efficient (Nick Craig-Wood)
48597
48598 • Fix --vfs-cache-mode minimal and writes ignoring cached files (Nick
48599 Craig-Wood)
48600
48601 • Local
48602
48603 • Add --local-case-sensitive and --local-case-insensitive (Nick
48604 Craig-Wood)
48605
48606 • Avoid polluting page cache when uploading local files to remote
48607 backends (Michał Matczuk)
48608
48609 • Don't calculate any hashes by default (Nick Craig-Wood)
48610
48611 • Fadvise run syscall on a dedicated go routine (Michał Matczuk)
48612
48613 • Azure Blob
48614
48615 • Azure Storage Emulator support (Sandeep)
48616
48617 • Updated config help details to remove connection string references
48618 (Sandeep)
48619
48620 • Make all operations work from the root (Nick Craig-Wood)
48621
48622 • B2
48623
48624 • Implement link sharing (yparitcher)
48625
48626 • Enable server-side copy to copy between buckets (Nick Craig-Wood)
48627
48628 • Make all operations work from the root (Nick Craig-Wood)
48629
48630 • Drive
48631
48632 • Fix server-side copy of big files (Nick Craig-Wood)
48633
48634 • Update API for teamdrive use (Nick Craig-Wood)
48635
48636 • Add error for purge with --drive-trashed-only (ginvine)
48637
48638 • Fichier
48639
48640 • Make FolderID int and adjust related code (buengese)
48641
48642 • Google Cloud Storage
48643
48644 • Reduce oauth scope requested as suggested by Google (Nick Craig-
48645 Wood)
48646
48647 • Make all operations work from the root (Nick Craig-Wood)
48648
48649 • HTTP
48650
48651 • Add --http-headers flag for setting arbitrary headers (Nick Craig-
48652 Wood)
48653
48654 • Jottacloud
48655
48656 • Use new api for retrieving internal username (buengese)
48657
48658 • Refactor configuration and minor cleanup (buengese)
48659
48660 • Koofr
48661
48662 • Support setting modification times on Koofr backend. (jaKa)
48663
48664 • Opendrive
48665
48666 • Refactor to use existing lib/rest facilities for uploads (Nick
48667 Craig-Wood)
48668
48669 • Qingstor
48670
48671 • Upgrade to v3 SDK and fix listing loop (Nick Craig-Wood)
48672
48673 • Make all operations work from the root (Nick Craig-Wood)
48674
48675 • S3
48676
48677 • Add INTELLIGENT_TIERING storage class (Matti Niemenmaa)
48678
48679 • Make all operations work from the root (Nick Craig-Wood)
48680
48681 • SFTP
48682
48683 • Add missing interface check and fix About (Nick Craig-Wood)
48684
48685 • Completely ignore all modtime checks if SetModTime=false (Jon Faut‐
48686 ley)
48687
48688 • Support md5/sha1 with rsync.net (Nick Craig-Wood)
48689
48690 • Save the md5/sha1 command in use to the config file for efficiency
48691 (Nick Craig-Wood)
48692
48693 • Opt-in support for diffie-hellman-group-exchange-sha256 diffie-
48694 hellman-group-exchange-sha1 (Yi FU)
48695
48696 • Swift
48697
48698 • Use FixRangeOption to fix 0 length files via the VFS (Nick Craig-
48699 Wood)
48700
48701 • Fix upload when using no_chunk to return the correct size (Nick
48702 Craig-Wood)
48703
48704 • Make all operations work from the root (Nick Craig-Wood)
48705
48706 • Fix segments leak during failed large file uploads. (nguyenhuulu‐
48707 an434)
48708
48709 • WebDAV
48710
48711 • Add --webdav-bearer-token-command (Nick Craig-Wood)
48712
48713 • Refresh token when it expires with --webdav-bearer-token-command
48714 (Nick Craig-Wood)
48715
48716 • Add docs for using bearer_token_command with oidc-agent (Paul Mil‐
48717 lar)
48718
48719 v1.48.0 - 2019-06-15
48720 • New commands
48721
48722 • serve sftp: Serve an rclone remote over SFTP (Nick Craig-Wood)
48723
48724 • New Features
48725
48726 • Multi threaded downloads to local storage (Nick Craig-Wood)
48727
48728 • controlled with --multi-thread-cutoff and --multi-thread-streams
48729
48730 • Use rclone.conf from rclone executable directory to enable portable
48731 use (albertony)
48732
48733 • Allow sync of a file and a directory with the same name (forgems)
48734
48735 • this is common on bucket-based remotes, e.g. s3, gcs
48736
48737 • Add --ignore-case-sync for forced case insensitivity (garry415)
48738
48739 • Implement --stats-one-line-date and --stats-one-line-date-format
48740 (Peter Berbec)
48741
48742 • Log an ERROR for all commands which exit with non-zero status (Nick
48743 Craig-Wood)
48744
48745 • Use go-homedir to read the home directory more reliably (Nick
48746 Craig-Wood)
48747
48748 • Enable creating encrypted config through external script invocation
48749 (Wojciech Smigielski)
48750
48751 • build: Drop support for go1.8 (Nick Craig-Wood)
48752
48753 • config: Make config create/update encrypt passwords where necessary
48754 (Nick Craig-Wood)
48755
48756 • copyurl: Honor --no-check-certificate (Stefan Breunig)
48757
48758 • install: Linux skip man pages if no mandb (didil)
48759
48760 • lsf: Support showing the Tier of the object (Nick Craig-Wood)
48761
48762 • lsjson
48763
48764 • Added EncryptedPath to output (calisro)
48765
48766 • Support showing the Tier of the object (Nick Craig-Wood)
48767
48768 • Add IsBucket field for bucket-based remote listing of the root
48769 (Nick Craig-Wood)
48770
48771 • rc
48772
48773 • Add --loopback flag to run commands directly without a server
48774 (Nick Craig-Wood)
48775
48776 • Add operations/fsinfo: Return information about the remote (Nick
48777 Craig-Wood)
48778
48779 • Skip auth for OPTIONS request (Nick Craig-Wood)
48780
48781 • cmd/providers: Add DefaultStr, ValueStr and Type fields (Nick
48782 Craig-Wood)
48783
48784 • jobs: Make job expiry timeouts configurable (Aleksandar Jankovic)
48785
48786 • serve dlna reworked and improved (Dan Walters)
48787
48788 • serve ftp: add --ftp-public-ip flag to specify public IP (calistri)
48789
48790 • serve restic: Add support for --private-repos in serve restic (Flo‐
48791 rian Apolloner)
48792
48793 • serve webdav: Combine serve webdav and serve http (Gary Kim)
48794
48795 • size: Ignore negative sizes when calculating total (Garry McNulty)
48796
48797 • Bug Fixes
48798
48799 • Make move and copy individual files obey --backup-dir (Nick Craig-
48800 Wood)
48801
48802 • If --ignore-checksum is in effect, don't calculate checksum (Nick
48803 Craig-Wood)
48804
48805 • moveto: Fix case-insensitive same remote move (Gary Kim)
48806
48807 • rc: Fix serving bucket-based objects with --rc-serve (Nick Craig-
48808 Wood)
48809
48810 • serve webdav: Fix serveDir not being updated with changes from web‐
48811 dav (Gary Kim)
48812
48813 • Mount
48814
48815 • Fix poll interval documentation (Animosity022)
48816
48817 • VFS
48818
48819 • Make WriteAt for non cached files work with non-sequential writes
48820 (Nick Craig-Wood)
48821
48822 • Local
48823
48824 • Only calculate the required hashes for big speedup (Nick Craig-
48825 Wood)
48826
48827 • Log errors when listing instead of returning an error (Nick Craig-
48828 Wood)
48829
48830 • Fix preallocate warning on Linux with ZFS (Nick Craig-Wood)
48831
48832 • Crypt
48833
48834 • Make rclone dedupe work through crypt (Nick Craig-Wood)
48835
48836 • Fix wrapping of ChangeNotify to decrypt directories properly (Nick
48837 Craig-Wood)
48838
48839 • Support PublicLink (rclone link) of underlying backend (Nick Craig-
48840 Wood)
48841
48842 • Implement Optional methods SetTier, GetTier (Nick Craig-Wood)
48843
48844 • B2
48845
48846 • Implement server-side copy (Nick Craig-Wood)
48847
48848 • Implement SetModTime (Nick Craig-Wood)
48849
48850 • Drive
48851
48852 • Fix move and copy from TeamDrive to GDrive (Fionera)
48853
48854 • Add notes that cleanup works in the background on drive (Nick
48855 Craig-Wood)
48856
48857 • Add --drive-server-side-across-configs to default back to old serv‐
48858 er-side copy semantics by default (Nick Craig-Wood)
48859
48860 • Add --drive-size-as-quota to show storage quota usage for file size
48861 (Garry McNulty)
48862
48863 • FTP
48864
48865 • Add FTP List timeout (Jeff Quinn)
48866
48867 • Add FTP over TLS support (Gary Kim)
48868
48869 • Add --ftp-no-check-certificate option for FTPS (Gary Kim)
48870
48871 • Google Cloud Storage
48872
48873 • Fix upload errors when uploading pre 1970 files (Nick Craig-Wood)
48874
48875 • Jottacloud
48876
48877 • Add support for selecting device and mountpoint. (buengese)
48878
48879 • Mega
48880
48881 • Add cleanup support (Gary Kim)
48882
48883 • Onedrive
48884
48885 • More accurately check if root is found (Cnly)
48886
48887 • S3
48888
48889 • Support S3 Accelerated endpoints with --s3-use-accelerate-endpoint
48890 (Nick Craig-Wood)
48891
48892 • Add config info for Wasabi's EU Central endpoint (Robert Marko)
48893
48894 • Make SetModTime work for GLACIER while syncing (Philip Harvey)
48895
48896 • SFTP
48897
48898 • Add About support (Gary Kim)
48899
48900 • Fix about parsing of df results so it can cope with -ve results
48901 (Nick Craig-Wood)
48902
48903 • Send custom client version and debug server version (Nick Craig-
48904 Wood)
48905
48906 • WebDAV
48907
48908 • Retry on 423 Locked errors (Nick Craig-Wood)
48909
48910 v1.47.0 - 2019-04-13
48911 • New backends
48912
48913 • Backend for Koofr cloud storage service. (jaKa)
48914
48915 • New Features
48916
48917 • Resume downloads if the reader fails in copy (Nick Craig-Wood)
48918
48919 • this means rclone will restart transfers if the source has an er‐
48920 ror
48921
48922 • this is most useful for downloads or cloud to cloud copies
48923
48924 • Use --fast-list for listing operations where it won't use more mem‐
48925 ory (Nick Craig-Wood)
48926
48927 • this should speed up the following operations on remotes which
48928 support ListR
48929
48930 • dedupe, serve restic lsf, ls, lsl, lsjson, lsd, md5sum, sha1sum,
48931 hashsum, size, delete, cat, settier
48932
48933 • use --disable ListR to get old behaviour if required
48934
48935 • Make --files-from traverse the destination unless --no-traverse is
48936 set (Nick Craig-Wood)
48937
48938 • this fixes --files-from with Google drive and excessive API use
48939 in general.
48940
48941 • Make server-side copy account bytes and obey --max-transfer (Nick
48942 Craig-Wood)
48943
48944 • Add --create-empty-src-dirs flag and default to not creating empty
48945 dirs (ishuah)
48946
48947 • Add client side TLS/SSL flags --ca-cert/--client-cert/--client-key
48948 (Nick Craig-Wood)
48949
48950 • Implement --suffix-keep-extension for use with --suffix (Nick
48951 Craig-Wood)
48952
48953 • build:
48954
48955 • Switch to semver compliant version tags to be go modules compli‐
48956 ant (Nick Craig-Wood)
48957
48958 • Update to use go1.12.x for the build (Nick Craig-Wood)
48959
48960 • serve dlna: Add connection manager service description to improve
48961 compatibility (Dan Walters)
48962
48963 • lsf: Add 'e' format to show encrypted names and 'o' for original
48964 IDs (Nick Craig-Wood)
48965
48966 • lsjson: Added --files-only and --dirs-only flags (calistri)
48967
48968 • rc: Implement operations/publiclink the equivalent of rclone link
48969 (Nick Craig-Wood)
48970
48971 • Bug Fixes
48972
48973 • accounting: Fix total ETA when --stats-unit bits is in effect (Nick
48974 Craig-Wood)
48975
48976 • Bash TAB completion
48977
48978 • Use private custom func to fix clash between rclone and kubectl
48979 (Nick Craig-Wood)
48980
48981 • Fix for remotes with underscores in their names (Six)
48982
48983 • Fix completion of remotes (Florian Gamböck)
48984
48985 • Fix autocompletion of remote paths with spaces (Danil Semelenov)
48986
48987 • serve dlna: Fix root XML service descriptor (Dan Walters)
48988
48989 • ncdu: Fix display corruption with Chinese characters (Nick Craig-
48990 Wood)
48991
48992 • Add SIGTERM to signals which run the exit handlers on unix (Nick
48993 Craig-Wood)
48994
48995 • rc: Reload filter when the options are set via the rc (Nick Craig-
48996 Wood)
48997
48998 • VFS / Mount
48999
49000 • Fix FreeBSD: Ignore Truncate if called with no readers and already
49001 the correct size (Nick Craig-Wood)
49002
49003 • Read directory and check for a file before mkdir (Nick Craig-Wood)
49004
49005 • Shorten the locking window for vfs/refresh (Nick Craig-Wood)
49006
49007 • Azure Blob
49008
49009 • Enable MD5 checksums when uploading files bigger than the "Cutoff"
49010 (Dr.Rx)
49011
49012 • Fix SAS URL support (Nick Craig-Wood)
49013
49014 • B2
49015
49016 • Allow manual configuration of backblaze downloadUrl (Vince)
49017
49018 • Ignore already_hidden error on remove (Nick Craig-Wood)
49019
49020 • Ignore malformed src_last_modified_millis (Nick Craig-Wood)
49021
49022 • Drive
49023
49024 • Add --skip-checksum-gphotos to ignore incorrect checksums on Google
49025 Photos (Nick Craig-Wood)
49026
49027 • Allow server-side move/copy between different remotes. (Fionera)
49028
49029 • Add docs on team drives and --fast-list eventual consistency (Nes‐
49030 tar47)
49031
49032 • Fix imports of text files (Nick Craig-Wood)
49033
49034 • Fix range requests on 0 length files (Nick Craig-Wood)
49035
49036 • Fix creation of duplicates with server-side copy (Nick Craig-Wood)
49037
49038 • Dropbox
49039
49040 • Retry blank errors to fix long listings (Nick Craig-Wood)
49041
49042 • FTP
49043
49044 • Add --ftp-concurrency to limit maximum number of connections (Nick
49045 Craig-Wood)
49046
49047 • Google Cloud Storage
49048
49049 • Fall back to default application credentials (marcintustin)
49050
49051 • Allow bucket policy only buckets (Nick Craig-Wood)
49052
49053 • HTTP
49054
49055 • Add --http-no-slash for websites with directories with no slashes
49056 (Nick Craig-Wood)
49057
49058 • Remove duplicates from listings (Nick Craig-Wood)
49059
49060 • Fix socket leak on 404 errors (Nick Craig-Wood)
49061
49062 • Jottacloud
49063
49064 • Fix token refresh (Sebastian Bünger)
49065
49066 • Add device registration (Oliver Heyme)
49067
49068 • Onedrive
49069
49070 • Implement graceful cancel of multipart uploads if rclone is inter‐
49071 rupted (Cnly)
49072
49073 • Always add trailing colon to path when addressing items, (Cnly)
49074
49075 • Return errors instead of panic for invalid uploads (Fabian Möller)
49076
49077 • S3
49078
49079 • Add support for "Glacier Deep Archive" storage class (Manu)
49080
49081 • Update Dreamhost endpoint (Nick Craig-Wood)
49082
49083 • Note incompatibility with CEPH Jewel (Nick Craig-Wood)
49084
49085 • SFTP
49086
49087 • Allow custom ssh client config (Alexandru Bumbacea)
49088
49089 • Swift
49090
49091 • Obey Retry-After to enable OVH restore from cold storage (Nick
49092 Craig-Wood)
49093
49094 • Work around token expiry on CEPH (Nick Craig-Wood)
49095
49096 • WebDAV
49097
49098 • Allow IsCollection property to be integer or boolean (Nick Craig-
49099 Wood)
49100
49101 • Fix race when creating directories (Nick Craig-Wood)
49102
49103 • Fix About/df when reading the available/total returns 0 (Nick
49104 Craig-Wood)
49105
49106 v1.46 - 2019-02-09
49107 • New backends
49108
49109 • Support Alibaba Cloud (Aliyun) OSS via the s3 backend (Nick Craig-
49110 Wood)
49111
49112 • New commands
49113
49114 • serve dlna: serves a remove via DLNA for the local network (ni‐
49115 colov)
49116
49117 • New Features
49118
49119 • copy, move: Restore deprecated --no-traverse flag (Nick Craig-Wood)
49120
49121 • This is useful for when transferring a small number of files into
49122 a large destination
49123
49124 • genautocomplete: Add remote path completion for bash completion
49125 (Christopher Peterson & Danil Semelenov)
49126
49127 • Buffer memory handling reworked to return memory to the OS better
49128 (Nick Craig-Wood)
49129
49130 • Buffer recycling library to replace sync.Pool
49131
49132 • Optionally use memory mapped memory for better memory shrinking
49133
49134 • Enable with --use-mmap if having memory problems - not default
49135 yet
49136
49137 • Parallelise reading of files specified by --files-from (Nick Craig-
49138 Wood)
49139
49140 • check: Add stats showing total files matched. (Dario Guzik)
49141
49142 • Allow rename/delete open files under Windows (Nick Craig-Wood)
49143
49144 • lsjson: Use exactly the correct number of decimal places in the
49145 seconds (Nick Craig-Wood)
49146
49147 • Add cookie support with cmdline switch --use-cookies for all HTTP
49148 based remotes (qip)
49149
49150 • Warn if --checksum is set but there are no hashes available (Nick
49151 Craig-Wood)
49152
49153 • Rework rate limiting (pacer) to be more accurate and allow bursting
49154 (Nick Craig-Wood)
49155
49156 • Improve error reporting for too many/few arguments in commands
49157 (Nick Craig-Wood)
49158
49159 • listremotes: Remove -l short flag as it conflicts with the new
49160 global flag (weetmuts)
49161
49162 • Make http serving with auth generate INFO messages on auth fail
49163 (Nick Craig-Wood)
49164
49165 • Bug Fixes
49166
49167 • Fix layout of stats (Nick Craig-Wood)
49168
49169 • Fix --progress crash under Windows Jenkins (Nick Craig-Wood)
49170
49171 • Fix transfer of google/onedrive docs by calling Rcat in Copy when
49172 size is -1 (Cnly)
49173
49174 • copyurl: Fix checking of --dry-run (Denis Skovpen)
49175
49176 • Mount
49177
49178 • Check that mountpoint and local directory to mount don't overlap
49179 (Nick Craig-Wood)
49180
49181 • Fix mount size under 32 bit Windows (Nick Craig-Wood)
49182
49183 • VFS
49184
49185 • Implement renaming of directories for backends without DirMove
49186 (Nick Craig-Wood)
49187
49188 • now all backends except b2 support renaming directories
49189
49190 • Implement --vfs-cache-max-size to limit the total size of the cache
49191 (Nick Craig-Wood)
49192
49193 • Add --dir-perms and --file-perms flags to set default permissions
49194 (Nick Craig-Wood)
49195
49196 • Fix deadlock on concurrent operations on a directory (Nick Craig-
49197 Wood)
49198
49199 • Fix deadlock between RWFileHandle.close and File.Remove (Nick
49200 Craig-Wood)
49201
49202 • Fix renaming/deleting open files with cache mode "writes" under
49203 Windows (Nick Craig-Wood)
49204
49205 • Fix panic on rename with --dry-run set (Nick Craig-Wood)
49206
49207 • Fix vfs/refresh with recurse=true needing the --fast-list flag
49208
49209 • Local
49210
49211 • Add support for -l/--links (symbolic link translation) (yair@uni‐
49212 corn)
49213
49214 • this works by showing links as link.rclonelink - see local back‐
49215 end docs for more info
49216
49217 • this errors if used with -L/--copy-links
49218
49219 • Fix renaming/deleting open files on Windows (Nick Craig-Wood)
49220
49221 • Crypt
49222
49223 • Check for maximum length before decrypting filename to fix panic
49224 (Garry McNulty)
49225
49226 • Azure Blob
49227
49228 • Allow building azureblob backend on *BSD (themylogin)
49229
49230 • Use the rclone HTTP client to support --dump headers, --tpslimit,
49231 etc. (Nick Craig-Wood)
49232
49233 • Use the s3 pacer for 0 delay in non error conditions (Nick Craig-
49234 Wood)
49235
49236 • Ignore directory markers (Nick Craig-Wood)
49237
49238 • Stop Mkdir attempting to create existing containers (Nick Craig-
49239 Wood)
49240
49241 • B2
49242
49243 • cleanup: will remove unfinished large files >24hrs old (Garry Mc‐
49244 Nulty)
49245
49246 • For a bucket limited application key check the bucket name (Nick
49247 Craig-Wood)
49248
49249 • before this, rclone would use the authorised bucket regardless of
49250 what you put on the command line
49251
49252 • Added --b2-disable-checksum flag (Wojciech Smigielski)
49253
49254 • this enables large files to be uploaded without a SHA-1 hash for
49255 speed reasons
49256
49257 • Drive
49258
49259 • Set default pacer to 100ms for 10 tps (Nick Craig-Wood)
49260
49261 • This fits the Google defaults much better and reduces the 403 er‐
49262 rors massively
49263
49264 • Add --drive-pacer-min-sleep and --drive-pacer-burst to control
49265 the pacer
49266
49267 • Improve ChangeNotify support for items with multiple parents (Fabi‐
49268 an Möller)
49269
49270 • Fix ListR for items with multiple parents - this fixes oddities
49271 with vfs/refresh (Fabian Möller)
49272
49273 • Fix using --drive-impersonate and appfolders (Nick Craig-Wood)
49274
49275 • Fix google docs in rclone mount for some (not all) applications
49276 (Nick Craig-Wood)
49277
49278 • Dropbox
49279
49280 • Retry-After support for Dropbox backend (Mathieu Carbou)
49281
49282 • FTP
49283
49284 • Wait for 60 seconds for a connection to Close then declare it dead
49285 (Nick Craig-Wood)
49286
49287 • helps with indefinite hangs on some FTP servers
49288
49289 • Google Cloud Storage
49290
49291 • Update google cloud storage endpoints (weetmuts)
49292
49293 • HTTP
49294
49295 • Add an example with username and password which is supported but
49296 wasn't documented (Nick Craig-Wood)
49297
49298 • Fix backend with --files-from and nonexistent files (Nick Craig-
49299 Wood)
49300
49301 • Hubic
49302
49303 • Make error message more informative if authentication fails (Nick
49304 Craig-Wood)
49305
49306 • Jottacloud
49307
49308 • Resume and deduplication support (Oliver Heyme)
49309
49310 • Use token auth for all API requests Don't store password anymore
49311 (Sebastian Bünger)
49312
49313 • Add support for 2-factor authentication (Sebastian Bünger)
49314
49315 • Mega
49316
49317 • Implement v2 account login which fixes logins for newer Mega ac‐
49318 counts (Nick Craig-Wood)
49319
49320 • Return error if an unknown length file is attempted to be uploaded
49321 (Nick Craig-Wood)
49322
49323 • Add new error codes for better error reporting (Nick Craig-Wood)
49324
49325 • Onedrive
49326
49327 • Fix broken support for "shared with me" folders (Alex Chen)
49328
49329 • Fix root ID not normalised (Cnly)
49330
49331 • Return err instead of panic on unknown-sized uploads (Cnly)
49332
49333 • Qingstor
49334
49335 • Fix go routine leak on multipart upload errors (Nick Craig-Wood)
49336
49337 • Add upload chunk size/concurrency/cutoff control (Nick Craig-Wood)
49338
49339 • Default --qingstor-upload-concurrency to 1 to work around bug (Nick
49340 Craig-Wood)
49341
49342 • S3
49343
49344 • Implement --s3-upload-cutoff for single part uploads below this
49345 (Nick Craig-Wood)
49346
49347 • Change --s3-upload-concurrency default to 4 to increase performance
49348 (Nick Craig-Wood)
49349
49350 • Add --s3-bucket-acl to control bucket ACL (Nick Craig-Wood)
49351
49352 • Auto detect region for buckets on operation failure (Nick Craig-
49353 Wood)
49354
49355 • Add GLACIER storage class (William Cocker)
49356
49357 • Add Scaleway to s3 documentation (Rémy Léone)
49358
49359 • Add AWS endpoint eu-north-1 (weetmuts)
49360
49361 • SFTP
49362
49363 • Add support for PEM encrypted private keys (Fabian Möller)
49364
49365 • Add option to force the usage of an ssh-agent (Fabian Möller)
49366
49367 • Perform environment variable expansion on key-file (Fabian Möller)
49368
49369 • Fix rmdir on Windows based servers (e.g. CrushFTP) (Nick Craig-
49370 Wood)
49371
49372 • Fix rmdir deleting directory contents on some SFTP servers (Nick
49373 Craig-Wood)
49374
49375 • Fix error on dangling symlinks (Nick Craig-Wood)
49376
49377 • Swift
49378
49379 • Add --swift-no-chunk to disable segmented uploads in rcat/mount
49380 (Nick Craig-Wood)
49381
49382 • Introduce application credential auth support (kayrus)
49383
49384 • Fix memory usage by slimming Object (Nick Craig-Wood)
49385
49386 • Fix extra requests on upload (Nick Craig-Wood)
49387
49388 • Fix reauth on big files (Nick Craig-Wood)
49389
49390 • Union
49391
49392 • Fix poll-interval not working (Nick Craig-Wood)
49393
49394 • WebDAV
49395
49396 • Support About which means rclone mount will show the correct disk
49397 size (Nick Craig-Wood)
49398
49399 • Support MD5 and SHA1 hashes with Owncloud and Nextcloud (Nick
49400 Craig-Wood)
49401
49402 • Fail soft on time parsing errors (Nick Craig-Wood)
49403
49404 • Fix infinite loop on failed directory creation (Nick Craig-Wood)
49405
49406 • Fix identification of directories for Bitrix Site Manager (Nick
49407 Craig-Wood)
49408
49409 • Fix upload of 0 length files on some servers (Nick Craig-Wood)
49410
49411 • Fix if MKCOL fails with 423 Locked assume the directory exists
49412 (Nick Craig-Wood)
49413
49414 v1.45 - 2018-11-24
49415 • New backends
49416
49417 • The Yandex backend was re-written - see below for details (Sebas‐
49418 tian Bünger)
49419
49420 • New commands
49421
49422 • rcd: New command just to serve the remote control API (Nick Craig-
49423 Wood)
49424
49425 • New Features
49426
49427 • The remote control API (rc) was greatly expanded to allow full con‐
49428 trol over rclone (Nick Craig-Wood)
49429
49430 • sensitive operations require authorization or the --rc-no-auth
49431 flag
49432
49433 • config/* operations to configure rclone
49434
49435 • options/* for reading/setting command line flags
49436
49437 • operations/* for all low level operations, e.g. copy file, list
49438 directory
49439
49440 • sync/* for sync, copy and move
49441
49442 • --rc-files flag to serve files on the rc http server
49443
49444 • this is for building web native GUIs for rclone
49445
49446 • Optionally serving objects on the rc http server
49447
49448 • Ensure rclone fails to start up if the --rc port is in use al‐
49449 ready
49450
49451 • See the rc docs (https://rclone.org/rc/) for more info
49452
49453 • sync/copy/move
49454
49455 • Make --files-from only read the objects specified and don't scan
49456 directories (Nick Craig-Wood)
49457
49458 • This is a huge speed improvement for destinations with lots of
49459 files
49460
49461 • filter: Add --ignore-case flag (Nick Craig-Wood)
49462
49463 • ncdu: Add remove function ('d' key) (Henning Surmeier)
49464
49465 • rc command
49466
49467 • Add --json flag for structured JSON input (Nick Craig-Wood)
49468
49469 • Add --user and --pass flags and interpret --rc-user, --rc-pass,
49470 --rc-addr (Nick Craig-Wood)
49471
49472 • build
49473
49474 • Require go1.8 or later for compilation (Nick Craig-Wood)
49475
49476 • Enable softfloat on MIPS arch (Scott Edlund)
49477
49478 • Integration test framework revamped with a better report and bet‐
49479 ter retries (Nick Craig-Wood)
49480
49481 • Bug Fixes
49482
49483 • cmd: Make --progress update the stats correctly at the end (Nick
49484 Craig-Wood)
49485
49486 • config: Create config directory on save if it is missing (Nick
49487 Craig-Wood)
49488
49489 • dedupe: Check for existing filename before renaming a dupe file
49490 (ssaqua)
49491
49492 • move: Don't create directories with --dry-run (Nick Craig-Wood)
49493
49494 • operations: Fix Purge and Rmdirs when dir is not the root (Nick
49495 Craig-Wood)
49496
49497 • serve http/webdav/restic: Ensure rclone exits if the port is in use
49498 (Nick Craig-Wood)
49499
49500 • Mount
49501
49502 • Make --volname work for Windows and macOS (Nick Craig-Wood)
49503
49504 • Azure Blob
49505
49506 • Avoid context deadline exceeded error by setting a large TryTimeout
49507 value (brused27)
49508
49509 • Fix erroneous Rmdir error "directory not empty" (Nick Craig-Wood)
49510
49511 • Wait for up to 60s to create a just deleted container (Nick Craig-
49512 Wood)
49513
49514 • Dropbox
49515
49516 • Add dropbox impersonate support (Jake Coggiano)
49517
49518 • Jottacloud
49519
49520 • Fix bug in --fast-list handing of empty folders (albertony)
49521
49522 • Opendrive
49523
49524 • Fix transfer of files with + and & in (Nick Craig-Wood)
49525
49526 • Fix retries of upload chunks (Nick Craig-Wood)
49527
49528 • S3
49529
49530 • Set ACL for server-side copies to that provided by the user (Nick
49531 Craig-Wood)
49532
49533 • Fix role_arn, credential_source, ... (Erik Swanson)
49534
49535 • Add config info for Wasabi's US-West endpoint (Henry Ptasinski)
49536
49537 • SFTP
49538
49539 • Ensure file hash checking is really disabled (Jon Fautley)
49540
49541 • Swift
49542
49543 • Add pacer for retries to make swift more reliable (Nick Craig-Wood)
49544
49545 • WebDAV
49546
49547 • Add Content-Type to PUT requests (Nick Craig-Wood)
49548
49549 • Fix config parsing so --webdav-user and --webdav-pass flags work
49550 (Nick Craig-Wood)
49551
49552 • Add RFC3339 date format (Ralf Hemberger)
49553
49554 • Yandex
49555
49556 • The yandex backend was re-written (Sebastian Bünger)
49557
49558 • This implements low level retries (Sebastian Bünger)
49559
49560 • Copy, Move, DirMove, PublicLink and About optional interfaces
49561 (Sebastian Bünger)
49562
49563 • Improved general error handling (Sebastian Bünger)
49564
49565 • Removed ListR for now due to inconsistent behaviour (Sebastian
49566 Bünger)
49567
49568 v1.44 - 2018-10-15
49569 • New commands
49570
49571 • serve ftp: Add ftp server (Antoine GIRARD)
49572
49573 • settier: perform storage tier changes on supported remotes
49574 (sandeepkru)
49575
49576 • New Features
49577
49578 • Reworked command line help
49579
49580 • Make default help less verbose (Nick Craig-Wood)
49581
49582 • Split flags up into global and backend flags (Nick Craig-Wood)
49583
49584 • Implement specialised help for flags and backends (Nick Craig-
49585 Wood)
49586
49587 • Show URL of backend help page when starting config (Nick Craig-
49588 Wood)
49589
49590 • stats: Long names now split in center (Joanna Marek)
49591
49592 • Add --log-format flag for more control over log output (dcpu)
49593
49594 • rc: Add support for OPTIONS and basic CORS (frenos)
49595
49596 • stats: show FatalErrors and NoRetryErrors in stats (Cédric Connes)
49597
49598 • Bug Fixes
49599
49600 • Fix -P not ending with a new line (Nick Craig-Wood)
49601
49602 • config: don't create default config dir when user supplies --config
49603 (albertony)
49604
49605 • Don't print non-ASCII characters with --progress on windows (Nick
49606 Craig-Wood)
49607
49608 • Correct logs for excluded items (ssaqua)
49609
49610 • Mount
49611
49612 • Remove EXPERIMENTAL tags (Nick Craig-Wood)
49613
49614 • VFS
49615
49616 • Fix race condition detected by serve ftp tests (Nick Craig-Wood)
49617
49618 • Add vfs/poll-interval rc command (Fabian Möller)
49619
49620 • Enable rename for nearly all remotes using server-side Move or Copy
49621 (Nick Craig-Wood)
49622
49623 • Reduce directory cache cleared by poll-interval (Fabian Möller)
49624
49625 • Remove EXPERIMENTAL tags (Nick Craig-Wood)
49626
49627 • Local
49628
49629 • Skip bad symlinks in dir listing with -L enabled (Cédric Connes)
49630
49631 • Preallocate files on Windows to reduce fragmentation (Nick Craig-
49632 Wood)
49633
49634 • Preallocate files on linux with fallocate(2) (Nick Craig-Wood)
49635
49636 • Cache
49637
49638 • Add cache/fetch rc function (Fabian Möller)
49639
49640 • Fix worker scale down (Fabian Möller)
49641
49642 • Improve performance by not sending info requests for cached chunks
49643 (dcpu)
49644
49645 • Fix error return value of cache/fetch rc method (Fabian Möller)
49646
49647 • Documentation fix for cache-chunk-total-size (Anagh Kumar Baranwal)
49648
49649 • Preserve leading / in wrapped remote path (Fabian Möller)
49650
49651 • Add plex_insecure option to skip certificate validation (Fabian
49652 Möller)
49653
49654 • Remove entries that no longer exist in the source (dcpu)
49655
49656 • Crypt
49657
49658 • Preserve leading / in wrapped remote path (Fabian Möller)
49659
49660 • Alias
49661
49662 • Fix handling of Windows network paths (Nick Craig-Wood)
49663
49664 • Azure Blob
49665
49666 • Add --azureblob-list-chunk parameter (Santiago Rodríguez)
49667
49668 • Implemented settier command support on azureblob remote. (sandeep‐
49669 kru)
49670
49671 • Work around SDK bug which causes errors for chunk-sized files (Nick
49672 Craig-Wood)
49673
49674 • Box
49675
49676 • Implement link sharing. (Sebastian Bünger)
49677
49678 • Drive
49679
49680 • Add --drive-import-formats - google docs can now be imported (Fabi‐
49681 an Möller)
49682
49683 • Rewrite mime type and extension handling (Fabian Möller)
49684
49685 • Add document links (Fabian Möller)
49686
49687 • Add support for multipart document extensions (Fabian Möller)
49688
49689 • Add support for apps-script to json export (Fabian Möller)
49690
49691 • Fix escaped chars in documents during list (Fabian Möller)
49692
49693 • Add --drive-v2-download-min-size a workaround for slow downloads
49694 (Fabian Möller)
49695
49696 • Improve directory notifications in ChangeNotify (Fabian Möller)
49697
49698 • When listing team drives in config, continue on failure (Nick
49699 Craig-Wood)
49700
49701 • FTP
49702
49703 • Add a small pause after failed upload before deleting file (Nick
49704 Craig-Wood)
49705
49706 • Google Cloud Storage
49707
49708 • Fix service_account_file being ignored (Fabian Möller)
49709
49710 • Jottacloud
49711
49712 • Minor improvement in quota info (omit if unlimited) (albertony)
49713
49714 • Add --fast-list support (albertony)
49715
49716 • Add permanent delete support: --jottacloud-hard-delete (albertony)
49717
49718 • Add link sharing support (albertony)
49719
49720 • Fix handling of reserved characters. (Sebastian Bünger)
49721
49722 • Fix socket leak on Object.Remove (Nick Craig-Wood)
49723
49724 • Onedrive
49725
49726 • Rework to support Microsoft Graph (Cnly)
49727
49728 • NB this will require re-authenticating the remote
49729
49730 • Removed upload cutoff and always do session uploads (Oliver Heyme)
49731
49732 • Use single-part upload for empty files (Cnly)
49733
49734 • Fix new fields not saved when editing old config (Alex Chen)
49735
49736 • Fix sometimes special chars in filenames not replaced (Alex Chen)
49737
49738 • Ignore OneNote files by default (Alex Chen)
49739
49740 • Add link sharing support (jackyzy823)
49741
49742 • S3
49743
49744 • Use custom pacer, to retry operations when reasonable (Craig
49745 Miskell)
49746
49747 • Use configured server-side-encryption and storage class options
49748 when calling CopyObject() (Paul Kohout)
49749
49750 • Make --s3-v2-auth flag (Nick Craig-Wood)
49751
49752 • Fix v2 auth on files with spaces (Nick Craig-Wood)
49753
49754 • Union
49755
49756 • Implement union backend which reads from multiple backends (Felix
49757 Brucker)
49758
49759 • Implement optional interfaces (Move, DirMove, Copy, etc.) (Nick
49760 Craig-Wood)
49761
49762 • Fix ChangeNotify to support multiple remotes (Fabian Möller)
49763
49764 • Fix --backup-dir on union backend (Nick Craig-Wood)
49765
49766 • WebDAV
49767
49768 • Add another time format (Nick Craig-Wood)
49769
49770 • Add a small pause after failed upload before deleting file (Nick
49771 Craig-Wood)
49772
49773 • Add workaround for missing mtime (buergi)
49774
49775 • Sharepoint: Renew cookies after 12hrs (Henning Surmeier)
49776
49777 • Yandex
49778
49779 • Remove redundant nil checks (teresy)
49780
49781 v1.43.1 - 2018-09-07
49782 Point release to fix hubic and azureblob backends.
49783
49784 • Bug Fixes
49785
49786 • ncdu: Return error instead of log.Fatal in Show (Fabian Möller)
49787
49788 • cmd: Fix crash with --progress and --stats 0 (Nick Craig-Wood)
49789
49790 • docs: Tidy website display (Anagh Kumar Baranwal)
49791
49792 • Azure Blob:
49793
49794 • Fix multi-part uploads. (sandeepkru)
49795
49796 • Hubic
49797
49798 • Fix uploads (Nick Craig-Wood)
49799
49800 • Retry auth fetching if it fails to make hubic more reliable (Nick
49801 Craig-Wood)
49802
49803 v1.43 - 2018-09-01
49804 • New backends
49805
49806 • Jottacloud (Sebastian Bünger)
49807
49808 • New commands
49809
49810 • copyurl: copies a URL to a remote (Denis)
49811
49812 • New Features
49813
49814 • Reworked config for backends (Nick Craig-Wood)
49815
49816 • All backend config can now be supplied by command line, env var
49817 or config file
49818
49819 • Advanced section in the config wizard for the optional items
49820
49821 • A large step towards rclone backends being usable in other go
49822 software
49823
49824 • Allow on the fly remotes with :backend: syntax
49825
49826 • Stats revamp
49827
49828 • Add --progress/-P flag to show interactive progress (Nick Craig-
49829 Wood)
49830
49831 • Show the total progress of the sync in the stats (Nick Craig-
49832 Wood)
49833
49834 • Add --stats-one-line flag for single line stats (Nick Craig-Wood)
49835
49836 • Added weekday schedule into --bwlimit (Mateusz)
49837
49838 • lsjson: Add option to show the original object IDs (Fabian Möller)
49839
49840 • serve webdav: Make Content-Type without reading the file and add
49841 --etag-hash (Nick Craig-Wood)
49842
49843 • build
49844
49845 • Build macOS with native compiler (Nick Craig-Wood)
49846
49847 • Update to use go1.11 for the build (Nick Craig-Wood)
49848
49849 • rc
49850
49851 • Added core/stats to return the stats (reddi1)
49852
49853 • version --check: Prints the current release and beta versions (Nick
49854 Craig-Wood)
49855
49856 • Bug Fixes
49857
49858 • accounting
49859
49860 • Fix time to completion estimates (Nick Craig-Wood)
49861
49862 • Fix moving average speed for file stats (Nick Craig-Wood)
49863
49864 • config: Fix error reading password from piped input (Nick Craig-
49865 Wood)
49866
49867 • move: Fix --delete-empty-src-dirs flag to delete all empty dirs on
49868 move (ishuah)
49869
49870 • Mount
49871
49872 • Implement --daemon-timeout flag for OSXFUSE (Nick Craig-Wood)
49873
49874 • Fix mount --daemon not working with encrypted config (Alex Chen)
49875
49876 • Clip the number of blocks to 2^32-1 on macOS - fixes borg backup
49877 (Nick Craig-Wood)
49878
49879 • VFS
49880
49881 • Enable vfs-read-chunk-size by default (Fabian Möller)
49882
49883 • Add the vfs/refresh rc command (Fabian Möller)
49884
49885 • Add non recursive mode to vfs/refresh rc command (Fabian Möller)
49886
49887 • Try to seek buffer on read only files (Fabian Möller)
49888
49889 • Local
49890
49891 • Fix crash when deprecated --local-no-unicode-normalization is sup‐
49892 plied (Nick Craig-Wood)
49893
49894 • Fix mkdir error when trying to copy files to the root of a drive on
49895 windows (Nick Craig-Wood)
49896
49897 • Cache
49898
49899 • Fix nil pointer deref when using lsjson on cached directory (Nick
49900 Craig-Wood)
49901
49902 • Fix nil pointer deref for occasional crash on playback (Nick Craig-
49903 Wood)
49904
49905 • Crypt
49906
49907 • Fix accounting when checking hashes on upload (Nick Craig-Wood)
49908
49909 • Amazon Cloud Drive
49910
49911 • Make very clear in the docs that rclone has no ACD keys (Nick
49912 Craig-Wood)
49913
49914 • Azure Blob
49915
49916 • Add connection string and SAS URL auth (Nick Craig-Wood)
49917
49918 • List the container to see if it exists (Nick Craig-Wood)
49919
49920 • Port new Azure Blob Storage SDK (sandeepkru)
49921
49922 • Added blob tier, tier between Hot, Cool and Archive. (sandeepkru)
49923
49924 • Remove leading / from paths (Nick Craig-Wood)
49925
49926 • B2
49927
49928 • Support Application Keys (Nick Craig-Wood)
49929
49930 • Remove leading / from paths (Nick Craig-Wood)
49931
49932 • Box
49933
49934 • Fix upload of > 2GB files on 32 bit platforms (Nick Craig-Wood)
49935
49936 • Make --box-commit-retries flag defaulting to 100 to fix large up‐
49937 loads (Nick Craig-Wood)
49938
49939 • Drive
49940
49941 • Add --drive-keep-revision-forever flag (lewapm)
49942
49943 • Handle gdocs when filtering file names in list (Fabian Möller)
49944
49945 • Support using --fast-list for large speedups (Fabian Möller)
49946
49947 • FTP
49948
49949 • Fix Put mkParentDir failed: 521 for BunnyCDN (Nick Craig-Wood)
49950
49951 • Google Cloud Storage
49952
49953 • Fix index out of range error with --fast-list (Nick Craig-Wood)
49954
49955 • Jottacloud
49956
49957 • Fix MD5 error check (Oliver Heyme)
49958
49959 • Handle empty time values (Martin Polden)
49960
49961 • Calculate missing MD5s (Oliver Heyme)
49962
49963 • Docs, fixes and tests for MD5 calculation (Nick Craig-Wood)
49964
49965 • Add optional MimeTyper interface. (Sebastian Bünger)
49966
49967 • Implement optional About interface (for df support). (Sebastian
49968 Bünger)
49969
49970 • Mega
49971
49972 • Wait for events instead of arbitrary sleeping (Nick Craig-Wood)
49973
49974 • Add --mega-hard-delete flag (Nick Craig-Wood)
49975
49976 • Fix failed logins with upper case chars in email (Nick Craig-Wood)
49977
49978 • Onedrive
49979
49980 • Shared folder support (Yoni Jah)
49981
49982 • Implement DirMove (Cnly)
49983
49984 • Fix rmdir sometimes deleting directories with contents (Nick Craig-
49985 Wood)
49986
49987 • Pcloud
49988
49989 • Delete half uploaded files on upload error (Nick Craig-Wood)
49990
49991 • Qingstor
49992
49993 • Remove leading / from paths (Nick Craig-Wood)
49994
49995 • S3
49996
49997 • Fix index out of range error with --fast-list (Nick Craig-Wood)
49998
49999 • Add --s3-force-path-style (Nick Craig-Wood)
50000
50001 • Add support for KMS Key ID (bsteiss)
50002
50003 • Remove leading / from paths (Nick Craig-Wood)
50004
50005 • Swift
50006
50007 • Add storage_policy (Ruben Vandamme)
50008
50009 • Make it so just storage_url or auth_token can be overridden (Nick
50010 Craig-Wood)
50011
50012 • Fix server-side copy bug for unusual file names (Nick Craig-Wood)
50013
50014 • Remove leading / from paths (Nick Craig-Wood)
50015
50016 • WebDAV
50017
50018 • Ensure we call MKCOL with a URL with a trailing / for QNAP interop
50019 (Nick Craig-Wood)
50020
50021 • If root ends with / then don't check if it is a file (Nick Craig-
50022 Wood)
50023
50024 • Don't accept redirects when reading metadata (Nick Craig-Wood)
50025
50026 • Add bearer token (Macaroon) support for dCache (Nick Craig-Wood)
50027
50028 • Document dCache and Macaroons (Onno Zweers)
50029
50030 • Sharepoint recursion with different depth (Henning)
50031
50032 • Attempt to remove failed uploads (Nick Craig-Wood)
50033
50034 • Yandex
50035
50036 • Fix listing/deleting files in the root (Nick Craig-Wood)
50037
50038 v1.42 - 2018-06-16
50039 • New backends
50040
50041 • OpenDrive (Oliver Heyme, Jakub Karlicek, ncw)
50042
50043 • New commands
50044
50045 • deletefile command (Filip Bartodziej)
50046
50047 • New Features
50048
50049 • copy, move: Copy single files directly, don't use --files-from
50050 work-around
50051
50052 • this makes them much more efficient
50053
50054 • Implement --max-transfer flag to quit transferring at a limit
50055
50056 • make exit code 8 for --max-transfer exceeded
50057
50058 • copy: copy empty source directories to destination (Ishuah Kariuki)
50059
50060 • check: Add --one-way flag (Kasper Byrdal Nielsen)
50061
50062 • Add siginfo handler for macOS for ctrl-T stats (kubatasiemski)
50063
50064 • rc
50065
50066 • add core/gc to run a garbage collection on demand
50067
50068 • enable go profiling by default on the --rc port
50069
50070 • return error from remote on failure
50071
50072 • lsf
50073
50074 • Add --absolute flag to add a leading / onto path names
50075
50076 • Add --csv flag for compliant CSV output
50077
50078 • Add 'm' format specifier to show the MimeType
50079
50080 • Implement 'i' format for showing object ID
50081
50082 • lsjson
50083
50084 • Add MimeType to the output
50085
50086 • Add ID field to output to show Object ID
50087
50088 • Add --retries-sleep flag (Benjamin Joseph Dag)
50089
50090 • Oauth tidy up web page and error handling (Henning Surmeier)
50091
50092 • Bug Fixes
50093
50094 • Password prompt output with --log-file fixed for unix (Filip Bar‐
50095 todziej)
50096
50097 • Calculate ModifyWindow each time on the fly to fix various problems
50098 (Stefan Breunig)
50099
50100 • Mount
50101
50102 • Only print "File.rename error" if there actually is an error (Ste‐
50103 fan Breunig)
50104
50105 • Delay rename if file has open writers instead of failing outright
50106 (Stefan Breunig)
50107
50108 • Ensure atexit gets run on interrupt
50109
50110 • macOS enhancements
50111
50112 • Make --noappledouble --noapplexattr
50113
50114 • Add --volname flag and remove special chars from it
50115
50116 • Make Get/List/Set/Remove xattr return ENOSYS for efficiency
50117
50118 • Make --daemon work for macOS without CGO
50119
50120 • VFS
50121
50122 • Add --vfs-read-chunk-size and --vfs-read-chunk-size-limit (Fabian
50123 Möller)
50124
50125 • Fix ChangeNotify for new or changed folders (Fabian Möller)
50126
50127 • Local
50128
50129 • Fix symlink/junction point directory handling under Windows
50130
50131 • NB you will need to add -L to your command line to copy files
50132 with reparse points
50133
50134 • Cache
50135
50136 • Add non cached dirs on notifications (Remus Bunduc)
50137
50138 • Allow root to be expired from rc (Remus Bunduc)
50139
50140 • Clean remaining empty folders from temp upload path (Remus Bunduc)
50141
50142 • Cache lists using batch writes (Remus Bunduc)
50143
50144 • Use secure websockets for HTTPS Plex addresses (John Clayton)
50145
50146 • Reconnect plex websocket on failures (Remus Bunduc)
50147
50148 • Fix panic when running without plex configs (Remus Bunduc)
50149
50150 • Fix root folder caching (Remus Bunduc)
50151
50152 • Crypt
50153
50154 • Check the crypted hash of files when uploading for extra data secu‐
50155 rity
50156
50157 • Dropbox
50158
50159 • Make Dropbox for business folders accessible using an initial / in
50160 the path
50161
50162 • Google Cloud Storage
50163
50164 • Low level retry all operations if necessary
50165
50166 • Google Drive
50167
50168 • Add --drive-acknowledge-abuse to download flagged files
50169
50170 • Add --drive-alternate-export to fix large doc export
50171
50172 • Don't attempt to choose Team Drives when using rclone config create
50173
50174 • Fix change list polling with team drives
50175
50176 • Fix ChangeNotify for folders (Fabian Möller)
50177
50178 • Fix about (and df on a mount) for team drives
50179
50180 • Onedrive
50181
50182 • Errorhandler for onedrive for business requests (Henning Surmeier)
50183
50184 • S3
50185
50186 • Adjust upload concurrency with --s3-upload-concurrency (themylogin)
50187
50188 • Fix --s3-chunk-size which was always using the minimum
50189
50190 • SFTP
50191
50192 • Add --ssh-path-override flag (Piotr Oleszczyk)
50193
50194 • Fix slow downloads for long latency connections
50195
50196 • Webdav
50197
50198 • Add workarounds for biz.mail.ru
50199
50200 • Ignore Reason-Phrase in status line to fix 4shared (Rodrigo)
50201
50202 • Better error message generation
50203
50204 v1.41 - 2018-04-28
50205 • New backends
50206
50207 • Mega support added
50208
50209 • Webdav now supports SharePoint cookie authentication (hensur)
50210
50211 • New commands
50212
50213 • link: create public link to files and folders (Stefan Breunig)
50214
50215 • about: gets quota info from a remote (a-roussos, ncw)
50216
50217 • hashsum: a generic tool for any hash to produce md5sum like output
50218
50219 • New Features
50220
50221 • lsd: Add -R flag and fix and update docs for all ls commands
50222
50223 • ncdu: added a "refresh" key - CTRL-L (Keith Goldfarb)
50224
50225 • serve restic: Add append-only mode (Steve Kriss)
50226
50227 • serve restic: Disallow overwriting files in append-only mode (Alex‐
50228 ander Neumann)
50229
50230 • serve restic: Print actual listener address (Matt Holt)
50231
50232 • size: Add --json flag (Matthew Holt)
50233
50234 • sync: implement --ignore-errors (Mateusz Pabian)
50235
50236 • dedupe: Add dedupe largest functionality (Richard Yang)
50237
50238 • fs: Extend SizeSuffix to include TB and PB for rclone about
50239
50240 • fs: add --dump goroutines and --dump openfiles for debugging
50241
50242 • rc: implement core/memstats to print internal memory usage info
50243
50244 • rc: new call rc/pid (Michael P. Dubner)
50245
50246 • Compile
50247
50248 • Drop support for go1.6
50249
50250 • Release
50251
50252 • Fix make tarball (Chih-Hsuan Yen)
50253
50254 • Bug Fixes
50255
50256 • filter: fix --min-age and --max-age together check
50257
50258 • fs: limit MaxIdleConns and MaxIdleConnsPerHost in transport
50259
50260 • lsd,lsf: make sure all times we output are in local time
50261
50262 • rc: fix setting bwlimit to unlimited
50263
50264 • rc: take note of the --rc-addr flag too as per the docs
50265
50266 • Mount
50267
50268 • Use About to return the correct disk total/used/free (e.g. in df)
50269
50270 • Set --attr-timeout default to 1s - fixes:
50271
50272 • rclone using too much memory
50273
50274 • rclone not serving files to samba
50275
50276 • excessive time listing directories
50277
50278 • Fix df -i (upstream fix)
50279
50280 • VFS
50281
50282 • Filter files . and .. from directory listing
50283
50284 • Only make the VFS cache if --vfs-cache-mode > Off
50285
50286 • Local
50287
50288 • Add --local-no-check-updated to disable updated file checks
50289
50290 • Retry remove on Windows sharing violation error
50291
50292 • Cache
50293
50294 • Flush the memory cache after close
50295
50296 • Purge file data on notification
50297
50298 • Always forget parent dir for notifications
50299
50300 • Integrate with Plex websocket
50301
50302 • Add rc cache/stats (seuffert)
50303
50304 • Add info log on notification
50305
50306 • Box
50307
50308 • Fix failure reading large directories - parse file/directory size
50309 as float
50310
50311 • Dropbox
50312
50313 • Fix crypt+obfuscate on dropbox
50314
50315 • Fix repeatedly uploading the same files
50316
50317 • FTP
50318
50319 • Work around strange response from box FTP server
50320
50321 • More workarounds for FTP servers to fix mkParentDir error
50322
50323 • Fix no error on listing nonexistent directory
50324
50325 • Google Cloud Storage
50326
50327 • Add service_account_credentials (Matt Holt)
50328
50329 • Detect bucket presence by listing it - minimises permissions needed
50330
50331 • Ignore zero length directory markers
50332
50333 • Google Drive
50334
50335 • Add service_account_credentials (Matt Holt)
50336
50337 • Fix directory move leaving a hardlinked directory behind
50338
50339 • Return proper google errors when Opening files
50340
50341 • When initialized with a filepath, optional features used incorrect
50342 root path (Stefan Breunig)
50343
50344 • HTTP
50345
50346 • Fix sync for servers which don't return Content-Length in HEAD
50347
50348 • Onedrive
50349
50350 • Add QuickXorHash support for OneDrive for business
50351
50352 • Fix socket leak in multipart session upload
50353
50354 • S3
50355
50356 • Look in S3 named profile files for credentials
50357
50358 • Add --s3-disable-checksum to disable checksum uploading (Chris Re‐
50359 dekop)
50360
50361 • Hierarchical configuration support (Giri Badanahatti)
50362
50363 • Add in config for all the supported S3 providers
50364
50365 • Add One Zone Infrequent Access storage class (Craig Rachel)
50366
50367 • Add --use-server-modtime support (Peter Baumgartner)
50368
50369 • Add --s3-chunk-size option to control multipart uploads
50370
50371 • Ignore zero length directory markers
50372
50373 • SFTP
50374
50375 • Update docs to match code, fix typos and clarify disable_hashcheck
50376 prompt (Michael G. Noll)
50377
50378 • Update docs with Synology quirks
50379
50380 • Fail soft with a debug on hash failure
50381
50382 • Swift
50383
50384 • Add --use-server-modtime support (Peter Baumgartner)
50385
50386 • Webdav
50387
50388 • Support SharePoint cookie authentication (hensur)
50389
50390 • Strip leading and trailing / off root
50391
50392 v1.40 - 2018-03-19
50393 • New backends
50394
50395 • Alias backend to create aliases for existing remote names (Fabian
50396 Möller)
50397
50398 • New commands
50399
50400 • lsf: list for parsing purposes (Jakub Tasiemski)
50401
50402 • by default this is a simple non recursive list of files and di‐
50403 rectories
50404
50405 • it can be configured to add more info in an easy to parse way
50406
50407 • serve restic: for serving a remote as a Restic REST endpoint
50408
50409 • This enables restic to use any backends that rclone can access
50410
50411 • Thanks Alexander Neumann for help, patches and review
50412
50413 • rc: enable the remote control of a running rclone
50414
50415 • The running rclone must be started with --rc and related flags.
50416
50417 • Currently there is support for bwlimit, and flushing for mount
50418 and cache.
50419
50420 • New Features
50421
50422 • --max-delete flag to add a delete threshold (Bjørn Erik Pedersen)
50423
50424 • All backends now support RangeOption for ranged Open
50425
50426 • cat: Use RangeOption for limited fetches to make more efficient
50427
50428 • cryptcheck: make reading of nonce more efficient with RangeOption
50429
50430 • serve http/webdav/restic
50431
50432 • support SSL/TLS
50433
50434 • add --user --pass and --htpasswd for authentication
50435
50436 • copy/move: detect file size change during copy/move and abort
50437 transfer (ishuah)
50438
50439 • cryptdecode: added option to return encrypted file names. (ishuah)
50440
50441 • lsjson: add --encrypted to show encrypted name (Jakub Tasiemski)
50442
50443 • Add --stats-file-name-length to specify the printed file name
50444 length for stats (Will Gunn)
50445
50446 • Compile
50447
50448 • Code base was shuffled and factored
50449
50450 • backends moved into a backend directory
50451
50452 • large packages split up
50453
50454 • See the CONTRIBUTING.md doc for info as to what lives where now
50455
50456 • Update to using go1.10 as the default go version
50457
50458 • Implement daily full integration tests (https://pub.rclone.org/in‐
50459 tegration-tests/)
50460
50461 • Release
50462
50463 • Include a source tarball and sign it and the binaries
50464
50465 • Sign the git tags as part of the release process
50466
50467 • Add .deb and .rpm packages as part of the build
50468
50469 • Make a beta release for all branches on the main repo (but not pull
50470 requests)
50471
50472 • Bug Fixes
50473
50474 • config: fixes errors on nonexistent config by loading config file
50475 only on first access
50476
50477 • config: retry saving the config after failure (Mateusz)
50478
50479 • sync: when using --backup-dir don't delete files if we can't set
50480 their modtime
50481
50482 • this fixes odd behaviour with Dropbox and --backup-dir
50483
50484 • fshttp: fix idle timeouts for HTTP connections
50485
50486 • serve http: fix serving files with : in - fixes
50487
50488 • Fix --exclude-if-present to ignore directories which it doesn't
50489 have permission for (Iakov Davydov)
50490
50491 • Make accounting work properly with crypt and b2
50492
50493 • remove --no-traverse flag because it is obsolete
50494
50495 • Mount
50496
50497 • Add --attr-timeout flag to control attribute caching in kernel
50498
50499 • this now defaults to 0 which is correct but less efficient
50500
50501 • see the mount docs (https://rclone.org/commands/rclone_mount/#at‐
50502 tribute-caching) for more info
50503
50504 • Add --daemon flag to allow mount to run in the background (ishuah)
50505
50506 • Fix: Return ENOSYS rather than EIO on attempted link
50507
50508 • This fixes FileZilla accessing an rclone mount served over sftp.
50509
50510 • Fix setting modtime twice
50511
50512 • Mount tests now run on CI for Linux (mount & cmount)/Mac/Windows
50513
50514 • Many bugs fixed in the VFS layer - see below
50515
50516 • VFS
50517
50518 • Many fixes for --vfs-cache-mode writes and above
50519
50520 • Update cached copy if we know it has changed (fixes stale data)
50521
50522 • Clean path names before using them in the cache
50523
50524 • Disable cache cleaner if --vfs-cache-poll-interval=0
50525
50526 • Fill and clean the cache immediately on startup
50527
50528 • Fix Windows opening every file when it stats the file
50529
50530 • Fix applying modtime for an open Write Handle
50531
50532 • Fix creation of files when truncating
50533
50534 • Write 0 bytes when flushing unwritten handles to avoid race condi‐
50535 tions in FUSE
50536
50537 • Downgrade "poll-interval is not supported" message to Info
50538
50539 • Make OpenFile and friends return EINVAL if O_RDONLY and O_TRUNC
50540
50541 • Local
50542
50543 • Downgrade "invalid cross-device link: trying copy" to debug
50544
50545 • Make DirMove return fs.ErrorCantDirMove to allow fallback to Copy
50546 for cross device
50547
50548 • Fix race conditions updating the hashes
50549
50550 • Cache
50551
50552 • Add support for polling - cache will update when remote changes on
50553 supported backends
50554
50555 • Reduce log level for Plex api
50556
50557 • Fix dir cache issue
50558
50559 • Implement --cache-db-wait-time flag
50560
50561 • Improve efficiency with RangeOption and RangeSeek
50562
50563 • Fix dirmove with temp fs enabled
50564
50565 • Notify vfs when using temp fs
50566
50567 • Offline uploading
50568
50569 • Remote control support for path flushing
50570
50571 • Amazon cloud drive
50572
50573 • Rclone no longer has any working keys - disable integration tests
50574
50575 • Implement DirChangeNotify to notify cache/vfs/mount of changes
50576
50577 • Azureblob
50578
50579 • Don't check for bucket/container presence if listing was OK
50580
50581 • this makes rclone do one less request per invocation
50582
50583 • Improve accounting for chunked uploads
50584
50585 • Backblaze B2
50586
50587 • Don't check for bucket/container presence if listing was OK
50588
50589 • this makes rclone do one less request per invocation
50590
50591 • Box
50592
50593 • Improve accounting for chunked uploads
50594
50595 • Dropbox
50596
50597 • Fix custom oauth client parameters
50598
50599 • Google Cloud Storage
50600
50601 • Don't check for bucket/container presence if listing was OK
50602
50603 • this makes rclone do one less request per invocation
50604
50605 • Google Drive
50606
50607 • Migrate to api v3 (Fabian Möller)
50608
50609 • Add scope configuration and root folder selection
50610
50611 • Add --drive-impersonate for service accounts
50612
50613 • thanks to everyone who tested, explored and contributed docs
50614
50615 • Add --drive-use-created-date to use created date as modified date
50616 (nbuchanan)
50617
50618 • Request the export formats only when required
50619
50620 • This makes rclone quicker when there are no google docs
50621
50622 • Fix finding paths with latin1 chars (a workaround for a drive bug)
50623
50624 • Fix copying of a single Google doc file
50625
50626 • Fix --drive-auth-owner-only to look in all directories
50627
50628 • HTTP
50629
50630 • Fix handling of directories with & in
50631
50632 • Onedrive
50633
50634 • Removed upload cutoff and always do session uploads
50635
50636 • this stops the creation of multiple versions on business onedrive
50637
50638 • Overwrite object size value with real size when reading file.
50639 (Victor)
50640
50641 • this fixes oddities when onedrive misreports the size of images
50642
50643 • Pcloud
50644
50645 • Remove unused chunked upload flag and code
50646
50647 • Qingstor
50648
50649 • Don't check for bucket/container presence if listing was OK
50650
50651 • this makes rclone do one less request per invocation
50652
50653 • S3
50654
50655 • Support hashes for multipart files (Chris Redekop)
50656
50657 • Initial support for IBM COS (S3) (Giri Badanahatti)
50658
50659 • Update docs to discourage use of v2 auth with CEPH and others
50660
50661 • Don't check for bucket/container presence if listing was OK
50662
50663 • this makes rclone do one less request per invocation
50664
50665 • Fix server-side copy and set modtime on files with + in
50666
50667 • SFTP
50668
50669 • Add option to disable remote hash check command execution (Jon
50670 Fautley)
50671
50672 • Add --sftp-ask-password flag to prompt for password when needed
50673 (Leo R. Lundgren)
50674
50675 • Add set_modtime configuration option
50676
50677 • Fix following of symlinks
50678
50679 • Fix reading config file outside of Fs setup
50680
50681 • Fix reading $USER in username fallback not $HOME
50682
50683 • Fix running under crontab - Use correct OS way of reading username
50684
50685 • Swift
50686
50687 • Fix refresh of authentication token
50688
50689 • in v1.39 a bug was introduced which ignored new tokens - this
50690 fixes it
50691
50692 • Fix extra HEAD transaction when uploading a new file
50693
50694 • Don't check for bucket/container presence if listing was OK
50695
50696 • this makes rclone do one less request per invocation
50697
50698 • Webdav
50699
50700 • Add new time formats to support mydrive.ch and others
50701
50702 v1.39 - 2017-12-23
50703 • New backends
50704
50705 • WebDAV
50706
50707 • tested with nextcloud, owncloud, put.io and others!
50708
50709 • Pcloud
50710
50711 • cache - wraps a cache around other backends (Remus Bunduc)
50712
50713 • useful in combination with mount
50714
50715 • NB this feature is in beta so use with care
50716
50717 • New commands
50718
50719 • serve command with subcommands:
50720
50721 • serve webdav: this implements a webdav server for any rclone re‐
50722 mote.
50723
50724 • serve http: command to serve a remote over HTTP
50725
50726 • config: add sub commands for full config file management
50727
50728 • create/delete/dump/edit/file/password/providers/show/update
50729
50730 • touch: to create or update the timestamp of a file (Jakub Tasiems‐
50731 ki)
50732
50733 • New Features
50734
50735 • curl install for rclone (Filip Bartodziej)
50736
50737 • --stats now shows percentage, size, rate and ETA in condensed form
50738 (Ishuah Kariuki)
50739
50740 • --exclude-if-present to exclude a directory if a file is present
50741 (Iakov Davydov)
50742
50743 • rmdirs: add --leave-root flag (lewapm)
50744
50745 • move: add --delete-empty-src-dirs flag to remove dirs after move
50746 (Ishuah Kariuki)
50747
50748 • Add --dump flag, introduce --dump requests, responses and remove
50749 --dump-auth, --dump-filters
50750
50751 • Obscure X-Auth-Token: from headers when dumping too
50752
50753 • Document and implement exit codes for different failure modes
50754 (Ishuah Kariuki)
50755
50756 • Compile
50757
50758 • Bug Fixes
50759
50760 • Retry lots more different types of errors to make multipart trans‐
50761 fers more reliable
50762
50763 • Save the config before asking for a token, fixes disappearing oauth
50764 config
50765
50766 • Warn the user if --include and --exclude are used together (Ernest
50767 Borowski)
50768
50769 • Fix duplicate files (e.g. on Google drive) causing spurious copies
50770
50771 • Allow trailing and leading whitespace for passwords (Jason Rose)
50772
50773 • ncdu: fix crashes on empty directories
50774
50775 • rcat: fix goroutine leak
50776
50777 • moveto/copyto: Fix to allow copying to the same name
50778
50779 • Mount
50780
50781 • --vfs-cache mode to make writes into mounts more reliable.
50782
50783 • this requires caching files on the disk (see --cache-dir)
50784
50785 • As this is a new feature, use with care
50786
50787 • Use sdnotify to signal systemd the mount is ready (Fabian Möller)
50788
50789 • Check if directory is not empty before mounting (Ernest Borowski)
50790
50791 • Local
50792
50793 • Add error message for cross file system moves
50794
50795 • Fix equality check for times
50796
50797 • Dropbox
50798
50799 • Rework multipart upload
50800
50801 • buffer the chunks when uploading large files so they can be re‐
50802 tried
50803
50804 • change default chunk size to 48MB now we are buffering them in
50805 memory
50806
50807 • retry every error after the first chunk is done successfully
50808
50809 • Fix error when renaming directories
50810
50811 • Swift
50812
50813 • Fix crash on bad authentication
50814
50815 • Google Drive
50816
50817 • Add service account support (Tim Cooijmans)
50818
50819 • S3
50820
50821 • Make it work properly with Digital Ocean Spaces (Andrew Starr-
50822 Bochicchio)
50823
50824 • Fix crash if a bad listing is received
50825
50826 • Add support for ECS task IAM roles (David Minor)
50827
50828 • Backblaze B2
50829
50830 • Fix multipart upload retries
50831
50832 • Fix --hard-delete to make it work 100% of the time
50833
50834 • Swift
50835
50836 • Allow authentication with storage URL and auth key (Giovanni Pizzi)
50837
50838 • Add new fields for swift configuration to support IBM Bluemix Swift
50839 (Pierre Carlson)
50840
50841 • Add OS_TENANT_ID and OS_USER_ID to config
50842
50843 • Allow configs with user id instead of user name
50844
50845 • Check if swift segments container exists before creating (John
50846 Leach)
50847
50848 • Fix memory leak in swift transfers (upstream fix)
50849
50850 • SFTP
50851
50852 • Add option to enable the use of aes128-cbc cipher (Jon Fautley)
50853
50854 • Amazon cloud drive
50855
50856 • Fix download of large files failing with "Only one auth mechanism
50857 allowed"
50858
50859 • crypt
50860
50861 • Option to encrypt directory names or leave them intact
50862
50863 • Implement DirChangeNotify (Fabian Möller)
50864
50865 • onedrive
50866
50867 • Add option to choose resourceURL during setup of OneDrive Business
50868 account if more than one is available for user
50869
50870 v1.38 - 2017-09-30
50871 • New backends
50872
50873 • Azure Blob Storage (thanks Andrei Dragomir)
50874
50875 • Box
50876
50877 • Onedrive for Business (thanks Oliver Heyme)
50878
50879 • QingStor from QingCloud (thanks wuyu)
50880
50881 • New commands
50882
50883 • rcat - read from standard input and stream upload
50884
50885 • tree - shows a nicely formatted recursive listing
50886
50887 • cryptdecode - decode crypted file names (thanks ishuah)
50888
50889 • config show - print the config file
50890
50891 • config file - print the config file location
50892
50893 • New Features
50894
50895 • Empty directories are deleted on sync
50896
50897 • dedupe - implement merging of duplicate directories
50898
50899 • check and cryptcheck made more consistent and use less memory
50900
50901 • cleanup for remaining remotes (thanks ishuah)
50902
50903 • --immutable for ensuring that files don't change (thanks Jacob Mc‐
50904 Namee)
50905
50906 • --user-agent option (thanks Alex McGrath Kraak)
50907
50908 • --disable flag to disable optional features
50909
50910 • --bind flag for choosing the local addr on outgoing connections
50911
50912 • Support for zsh auto-completion (thanks bpicode)
50913
50914 • Stop normalizing file names but do a normalized compare in sync
50915
50916 • Compile
50917
50918 • Update to using go1.9 as the default go version
50919
50920 • Remove snapd build due to maintenance problems
50921
50922 • Bug Fixes
50923
50924 • Improve retriable error detection which makes multipart uploads
50925 better
50926
50927 • Make check obey --ignore-size
50928
50929 • Fix bwlimit toggle in conjunction with schedules (thanks cbruegg)
50930
50931 • config ensures newly written config is on the same mount
50932
50933 • Local
50934
50935 • Revert to copy when moving file across file system boundaries
50936
50937 • --skip-links to suppress symlink warnings (thanks Zhiming Wang)
50938
50939 • Mount
50940
50941 • Re-use rcat internals to support uploads from all remotes
50942
50943 • Dropbox
50944
50945 • Fix "entry doesn't belong in directory" error
50946
50947 • Stop using deprecated API methods
50948
50949 • Swift
50950
50951 • Fix server-side copy to empty container with --fast-list
50952
50953 • Google Drive
50954
50955 • Change the default for --drive-use-trash to true
50956
50957 • S3
50958
50959 • Set session token when using STS (thanks Girish Ramakrishnan)
50960
50961 • Glacier docs and error messages (thanks Jan Varho)
50962
50963 • Read 1000 (not 1024) items in dir listings to fix Wasabi
50964
50965 • Backblaze B2
50966
50967 • Fix SHA1 mismatch when downloading files with no SHA1
50968
50969 • Calculate missing hashes on the fly instead of spooling
50970
50971 • --b2-hard-delete to permanently delete (not hide) files (thanks
50972 John Papandriopoulos)
50973
50974 • Hubic
50975
50976 • Fix creating containers - no longer have to use the default con‐
50977 tainer
50978
50979 • Swift
50980
50981 • Optionally configure from a standard set of OpenStack environment
50982 vars
50983
50984 • Add endpoint_type config
50985
50986 • Google Cloud Storage
50987
50988 • Fix bucket creation to work with limited permission users
50989
50990 • SFTP
50991
50992 • Implement connection pooling for multiple ssh connections
50993
50994 • Limit new connections per second
50995
50996 • Add support for MD5 and SHA1 hashes where available (thanks Chris‐
50997 tian Brüggemann)
50998
50999 • HTTP
51000
51001 • Fix URL encoding issues
51002
51003 • Fix directories with : in
51004
51005 • Fix panic with URL encoded content
51006
51007 v1.37 - 2017-07-22
51008 • New backends
51009
51010 • FTP - thanks to Antonio Messina
51011
51012 • HTTP - thanks to Vasiliy Tolstov
51013
51014 • New commands
51015
51016 • rclone ncdu - for exploring a remote with a text based user inter‐
51017 face.
51018
51019 • rclone lsjson - for listing with a machine-readable output
51020
51021 • rclone dbhashsum - to show Dropbox style hashes of files (local or
51022 Dropbox)
51023
51024 • New Features
51025
51026 • Implement --fast-list flag
51027
51028 • This allows remotes to list recursively if they can
51029
51030 • This uses less transactions (important if you pay for them)
51031
51032 • This may or may not be quicker
51033
51034 • This will use more memory as it has to hold the listing in memory
51035
51036 • --old-sync-method deprecated - the remaining uses are covered by
51037 --fast-list
51038
51039 • This involved a major re-write of all the listing code
51040
51041 • Add --tpslimit and --tpslimit-burst to limit transactions per sec‐
51042 ond
51043
51044 • this is useful in conjunction with rclone mount to limit external
51045 apps
51046
51047 • Add --stats-log-level so can see --stats without -v
51048
51049 • Print password prompts to stderr - Hraban Luyat
51050
51051 • Warn about duplicate files when syncing
51052
51053 • Oauth improvements
51054
51055 • allow auth_url and token_url to be set in the config file
51056
51057 • Print redirection URI if using own credentials.
51058
51059 • Don't Mkdir at the start of sync to save transactions
51060
51061 • Compile
51062
51063 • Update build to go1.8.3
51064
51065 • Require go1.6 for building rclone
51066
51067 • Compile 386 builds with "GO386=387" for maximum compatibility
51068
51069 • Bug Fixes
51070
51071 • Fix menu selection when no remotes
51072
51073 • Config saving reworked to not kill the file if disk gets full
51074
51075 • Don't delete remote if name does not change while renaming
51076
51077 • moveto, copyto: report transfers and checks as per move and copy
51078
51079 • Local
51080
51081 • Add --local-no-unicode-normalization flag - Bob Potter
51082
51083 • Mount
51084
51085 • Now supported on Windows using cgofuse and WinFsp - thanks to Bill
51086 Zissimopoulos for much help
51087
51088 • Compare checksums on upload/download via FUSE
51089
51090 • Unmount when program ends with SIGINT (Ctrl+C) or SIGTERM - Jérôme
51091 Vizcaino
51092
51093 • On read only open of file, make open pending until first read
51094
51095 • Make --read-only reject modify operations
51096
51097 • Implement ModTime via FUSE for remotes that support it
51098
51099 • Allow modTime to be changed even before all writers are closed
51100
51101 • Fix panic on renames
51102
51103 • Fix hang on errored upload
51104
51105 • Crypt
51106
51107 • Report the name:root as specified by the user
51108
51109 • Add an "obfuscate" option for filename encryption - Stephen Harris
51110
51111 • Amazon Drive
51112
51113 • Fix initialization order for token renewer
51114
51115 • Remove revoked credentials, allow oauth proxy config and update
51116 docs
51117
51118 • B2
51119
51120 • Reduce minimum chunk size to 5MB
51121
51122 • Drive
51123
51124 • Add team drive support
51125
51126 • Reduce bandwidth by adding fields for partial responses - Martin
51127 Kristensen
51128
51129 • Implement --drive-shared-with-me flag to view shared with me files
51130 - Danny Tsai
51131
51132 • Add --drive-trashed-only to read only the files in the trash
51133
51134 • Remove obsolete --drive-full-list
51135
51136 • Add missing seek to start on retries of chunked uploads
51137
51138 • Fix stats accounting for upload
51139
51140 • Convert / in names to a unicode equivalent (/)
51141
51142 • Poll for Google Drive changes when mounted
51143
51144 • OneDrive
51145
51146 • Fix the uploading of files with spaces
51147
51148 • Fix initialization order for token renewer
51149
51150 • Display speeds accurately when uploading - Yoni Jah
51151
51152 • Swap to using http://localhost:53682/ as redirect URL - Michael
51153 Ledin
51154
51155 • Retry on token expired error, reset upload body on retry - Yoni Jah
51156
51157 • Google Cloud Storage
51158
51159 • Add ability to specify location and storage class via config and
51160 command line - thanks gdm85
51161
51162 • Create container if necessary on server-side copy
51163
51164 • Increase directory listing chunk to 1000 to increase performance
51165
51166 • Obtain a refresh token for GCS - Steven Lu
51167
51168 • Yandex
51169
51170 • Fix the name reported in log messages (was empty)
51171
51172 • Correct error return for listing empty directory
51173
51174 • Dropbox
51175
51176 • Rewritten to use the v2 API
51177
51178 • Now supports ModTime
51179
51180 • Can only set by uploading the file again
51181
51182 • If you uploaded with an old rclone, rclone may upload every‐
51183 thing again
51184
51185 • Use --size-only or --checksum to avoid this
51186
51187 • Now supports the Dropbox content hashing scheme
51188
51189 • Now supports low level retries
51190
51191 • S3
51192
51193 • Work around eventual consistency in bucket creation
51194
51195 • Create container if necessary on server-side copy
51196
51197 • Add us-east-2 (Ohio) and eu-west-2 (London) S3 regions - Zahiar
51198 Ahmed
51199
51200 • Swift, Hubic
51201
51202 • Fix zero length directory markers showing in the subdirectory list‐
51203 ing
51204
51205 • this caused lots of duplicate transfers
51206
51207 • Fix paged directory listings
51208
51209 • this caused duplicate directory errors
51210
51211 • Create container if necessary on server-side copy
51212
51213 • Increase directory listing chunk to 1000 to increase performance
51214
51215 • Make sensible error if the user forgets the container
51216
51217 • SFTP
51218
51219 • Add support for using ssh key files
51220
51221 • Fix under Windows
51222
51223 • Fix ssh agent on Windows
51224
51225 • Adapt to latest version of library - Igor Kharin
51226
51227 v1.36 - 2017-03-18
51228 • New Features
51229
51230 • SFTP remote (Jack Schmidt)
51231
51232 • Re-implement sync routine to work a directory at a time reducing
51233 memory usage
51234
51235 • Logging revamped to be more inline with rsync - now much quieter *
51236 -v only shows transfers * -vv is for full debug * --syslog to log
51237 to syslog on capable platforms
51238
51239 • Implement --backup-dir and --suffix
51240
51241 • Implement --track-renames (initial implementation by Bjørn Erik
51242 Pedersen)
51243
51244 • Add time-based bandwidth limits (Lukas Loesche)
51245
51246 • rclone cryptcheck: checks integrity of crypt remotes
51247
51248 • Allow all config file variables and options to be set from environ‐
51249 ment variables
51250
51251 • Add --buffer-size parameter to control buffer size for copy
51252
51253 • Make --delete-after the default
51254
51255 • Add --ignore-checksum flag (fixed by Hisham Zarka)
51256
51257 • rclone check: Add --download flag to check all the data, not just
51258 hashes
51259
51260 • rclone cat: add --head, --tail, --offset, --count and --discard
51261
51262 • rclone config: when choosing from a list, allow the value to be en‐
51263 tered too
51264
51265 • rclone config: allow rename and copy of remotes
51266
51267 • rclone obscure: for generating encrypted passwords for rclone's
51268 config (T.C. Ferguson)
51269
51270 • Comply with XDG Base Directory specification (Dario Giovannetti)
51271
51272 • this moves the default location of the config file in a backwards
51273 compatible way
51274
51275 • Release changes
51276
51277 • Ubuntu snap support (Dedsec1)
51278
51279 • Compile with go 1.8
51280
51281 • MIPS/Linux big and little endian support
51282
51283 • Bug Fixes
51284
51285 • Fix copyto copying things to the wrong place if the destination dir
51286 didn't exist
51287
51288 • Fix parsing of remotes in moveto and copyto
51289
51290 • Fix --delete-before deleting files on copy
51291
51292 • Fix --files-from with an empty file copying everything
51293
51294 • Fix sync: don't update mod times if --dry-run set
51295
51296 • Fix MimeType propagation
51297
51298 • Fix filters to add ** rules to directory rules
51299
51300 • Local
51301
51302 • Implement -L, --copy-links flag to allow rclone to follow symlinks
51303
51304 • Open files in write only mode so rclone can write to an rclone
51305 mount
51306
51307 • Fix unnormalised unicode causing problems reading directories
51308
51309 • Fix interaction between -x flag and --max-depth
51310
51311 • Mount
51312
51313 • Implement proper directory handling (mkdir, rmdir, renaming)
51314
51315 • Make include and exclude filters apply to mount
51316
51317 • Implement read and write async buffers - control with --buffer-size
51318
51319 • Fix fsync on for directories
51320
51321 • Fix retry on network failure when reading off crypt
51322
51323 • Crypt
51324
51325 • Add --crypt-show-mapping to show encrypted file mapping
51326
51327 • Fix crypt writer getting stuck in a loop
51328
51329 • IMPORTANT this bug had the potential to cause data corruption
51330 when
51331
51332 • reading data from a network based remote and
51333
51334 • writing to a crypt on Google Drive
51335
51336 • Use the cryptcheck command to validate your data if you are con‐
51337 cerned
51338
51339 • If syncing two crypt remotes, sync the unencrypted remote
51340
51341 • Amazon Drive
51342
51343 • Fix panics on Move (rename)
51344
51345 • Fix panic on token expiry
51346
51347 • B2
51348
51349 • Fix inconsistent listings and rclone check
51350
51351 • Fix uploading empty files with go1.8
51352
51353 • Constrain memory usage when doing multipart uploads
51354
51355 • Fix upload url not being refreshed properly
51356
51357 • Drive
51358
51359 • Fix Rmdir on directories with trashed files
51360
51361 • Fix "Ignoring unknown object" when downloading
51362
51363 • Add --drive-list-chunk
51364
51365 • Add --drive-skip-gdocs (Károly Oláh)
51366
51367 • OneDrive
51368
51369 • Implement Move
51370
51371 • Fix Copy
51372
51373 • Fix overwrite detection in Copy
51374
51375 • Fix waitForJob to parse errors correctly
51376
51377 • Use token renewer to stop auth errors on long uploads
51378
51379 • Fix uploading empty files with go1.8
51380
51381 • Google Cloud Storage
51382
51383 • Fix depth 1 directory listings
51384
51385 • Yandex
51386
51387 • Fix single level directory listing
51388
51389 • Dropbox
51390
51391 • Normalise the case for single level directory listings
51392
51393 • Fix depth 1 listing
51394
51395 • S3
51396
51397 • Added ca-central-1 region (Jon Yergatian)
51398
51399 v1.35 - 2017-01-02
51400 • New Features
51401
51402 • moveto and copyto commands for choosing a destination name on
51403 copy/move
51404
51405 • rmdirs command to recursively delete empty directories
51406
51407 • Allow repeated --include/--exclude/--filter options
51408
51409 • Only show transfer stats on commands which transfer stuff
51410
51411 • show stats on any command using the --stats flag
51412
51413 • Allow overlapping directories in move when server-side dir move is
51414 supported
51415
51416 • Add --stats-unit option - thanks Scott McGillivray
51417
51418 • Bug Fixes
51419
51420 • Fix the config file being overwritten when two rclone instances are
51421 running
51422
51423 • Make rclone lsd obey the filters properly
51424
51425 • Fix compilation on mips
51426
51427 • Fix not transferring files that don't differ in size
51428
51429 • Fix panic on nil retry/fatal error
51430
51431 • Mount
51432
51433 • Retry reads on error - should help with reliability a lot
51434
51435 • Report the modification times for directories from the remote
51436
51437 • Add bandwidth accounting and limiting (fixes --bwlimit)
51438
51439 • If --stats provided will show stats and which files are transfer‐
51440 ring
51441
51442 • Support R/W files if truncate is set.
51443
51444 • Implement statfs interface so df works
51445
51446 • Note that write is now supported on Amazon Drive
51447
51448 • Report number of blocks in a file - thanks Stefan Breunig
51449
51450 • Crypt
51451
51452 • Prevent the user pointing crypt at itself
51453
51454 • Fix failed to authenticate decrypted block errors
51455
51456 • these will now return the underlying unexpected EOF instead
51457
51458 • Amazon Drive
51459
51460 • Add support for server-side move and directory move - thanks Stefan
51461 Breunig
51462
51463 • Fix nil pointer deref on size attribute
51464
51465 • B2
51466
51467 • Use new prefix and delimiter parameters in directory listings
51468
51469 • This makes --max-depth 1 dir listings as used in mount much
51470 faster
51471
51472 • Reauth the account while doing uploads too - should help with token
51473 expiry
51474
51475 • Drive
51476
51477 • Make DirMove more efficient and complain about moving the root
51478
51479 • Create destination directory on Move()
51480
51481 v1.34 - 2016-11-06
51482 • New Features
51483
51484 • Stop single file and --files-from operations iterating through the
51485 source bucket.
51486
51487 • Stop removing failed upload to cloud storage remotes
51488
51489 • Make ContentType be preserved for cloud to cloud copies
51490
51491 • Add support to toggle bandwidth limits via SIGUSR2 - thanks Marco
51492 Paganini
51493
51494 • rclone check shows count of hashes that couldn't be checked
51495
51496 • rclone listremotes command
51497
51498 • Support linux/arm64 build - thanks Fredrik Fornwall
51499
51500 • Remove Authorization: lines from --dump-headers output
51501
51502 • Bug Fixes
51503
51504 • Ignore files with control characters in the names
51505
51506 • Fix rclone move command
51507
51508 • Delete src files which already existed in dst
51509
51510 • Fix deletion of src file when dst file older
51511
51512 • Fix rclone check on crypted file systems
51513
51514 • Make failed uploads not count as "Transferred"
51515
51516 • Make sure high level retries show with -q
51517
51518 • Use a vendor directory with godep for repeatable builds
51519
51520 • rclone mount - FUSE
51521
51522 • Implement FUSE mount options
51523
51524 • --no-modtime, --debug-fuse, --read-only, --allow-non-empty, --al‐
51525 low-root, --allow-other
51526
51527 • --default-permissions, --write-back-cache, --max-read-ahead,
51528 --umask, --uid, --gid
51529
51530 • Add --dir-cache-time to control caching of directory entries
51531
51532 • Implement seek for files opened for read (useful for video players)
51533
51534 • with -no-seek flag to disable
51535
51536 • Fix crash on 32 bit ARM (alignment of 64 bit counter)
51537
51538 • ...and many more internal fixes and improvements!
51539
51540 • Crypt
51541
51542 • Don't show encrypted password in configurator to stop confusion
51543
51544 • Amazon Drive
51545
51546 • New wait for upload option --acd-upload-wait-per-gb
51547
51548 • upload timeouts scale by file size and can be disabled
51549
51550 • Add 502 Bad Gateway to list of errors we retry
51551
51552 • Fix overwriting a file with a zero length file
51553
51554 • Fix ACD file size warning limit - thanks Felix Bünemann
51555
51556 • Local
51557
51558 • Unix: implement -x/--one-file-system to stay on a single file sys‐
51559 tem
51560
51561 • thanks Durval Menezes and Luiz Carlos Rumbelsperger Viana
51562
51563 • Windows: ignore the symlink bit on files
51564
51565 • Windows: Ignore directory-based junction points
51566
51567 • B2
51568
51569 • Make sure each upload has at least one upload slot - fixes strange
51570 upload stats
51571
51572 • Fix uploads when using crypt
51573
51574 • Fix download of large files (sha1 mismatch)
51575
51576 • Return error when we try to create a bucket which someone else owns
51577
51578 • Update B2 docs with Data usage, and Crypt section - thanks Tomasz
51579 Mazur
51580
51581 • S3
51582
51583 • Command line and config file support for
51584
51585 • Setting/overriding ACL - thanks Radek Šenfeld
51586
51587 • Setting storage class - thanks Asko Tamm
51588
51589 • Drive
51590
51591 • Make exponential backoff work exactly as per Google specification
51592
51593 • add .epub, .odp and .tsv as export formats.
51594
51595 • Swift
51596
51597 • Don't read metadata for directory marker objects
51598
51599 v1.33 - 2016-08-24
51600 • New Features
51601
51602 • Implement encryption
51603
51604 • data encrypted in NACL secretbox format
51605
51606 • with optional file name encryption
51607
51608 • New commands
51609
51610 • rclone mount - implements FUSE mounting of remotes (EXPERIMENTAL)
51611
51612 • works on Linux, FreeBSD and OS X (need testers for the last 2!)
51613
51614 • rclone cat - outputs remote file or files to the terminal
51615
51616 • rclone genautocomplete - command to make a bash completion script
51617 for rclone
51618
51619 • Editing a remote using rclone config now goes through the wizard
51620
51621 • Compile with go 1.7 - this fixes rclone on macOS Sierra and on 386
51622 processors
51623
51624 • Use cobra for sub commands and docs generation
51625
51626 • drive
51627
51628 • Document how to make your own client_id
51629
51630 • s3
51631
51632 • User-configurable Amazon S3 ACL (thanks Radek Šenfeld)
51633
51634 • b2
51635
51636 • Fix stats accounting for upload - no more jumping to 100% done
51637
51638 • On cleanup delete hide marker if it is the current file
51639
51640 • New B2 API endpoint (thanks Per Cederberg)
51641
51642 • Set maximum backoff to 5 Minutes
51643
51644 • onedrive
51645
51646 • Fix URL escaping in file names - e.g. uploading files with + in
51647 them.
51648
51649 • amazon cloud drive
51650
51651 • Fix token expiry during large uploads
51652
51653 • Work around 408 REQUEST_TIMEOUT and 504 GATEWAY_TIMEOUT errors
51654
51655 • local
51656
51657 • Fix filenames with invalid UTF-8 not being uploaded
51658
51659 • Fix problem with some UTF-8 characters on OS X
51660
51661 v1.32 - 2016-07-13
51662 • Backblaze B2
51663
51664 • Fix upload of files large files not in root
51665
51666 v1.31 - 2016-07-13
51667 • New Features
51668
51669 • Reduce memory on sync by about 50%
51670
51671 • Implement --no-traverse flag to stop copy traversing the destina‐
51672 tion remote.
51673
51674 • This can be used to reduce memory usage down to the smallest pos‐
51675 sible.
51676
51677 • Useful to copy a small number of files into a large destination
51678 folder.
51679
51680 • Implement cleanup command for emptying trash / removing old ver‐
51681 sions of files
51682
51683 • Currently B2 only
51684
51685 • Single file handling improved
51686
51687 • Now copied with --files-from
51688
51689 • Automatically sets --no-traverse when copying a single file
51690
51691 • Info on using installing with ansible - thanks Stefan Weichinger
51692
51693 • Implement --no-update-modtime flag to stop rclone fixing the remote
51694 modified times.
51695
51696 • Bug Fixes
51697
51698 • Fix move command - stop it running for overlapping Fses - this was
51699 causing data loss.
51700
51701 • Local
51702
51703 • Fix incomplete hashes - this was causing problems for B2.
51704
51705 • Amazon Drive
51706
51707 • Rename Amazon Cloud Drive to Amazon Drive - no changes to config
51708 file needed.
51709
51710 • Swift
51711
51712 • Add support for non-default project domain - thanks Antonio Messi‐
51713 na.
51714
51715 • S3
51716
51717 • Add instructions on how to use rclone with minio.
51718
51719 • Add ap-northeast-2 (Seoul) and ap-south-1 (Mumbai) regions.
51720
51721 • Skip setting the modified time for objects > 5GB as it isn't possi‐
51722 ble.
51723
51724 • Backblaze B2
51725
51726 • Add --b2-versions flag so old versions can be listed and retrieved.
51727
51728 • Treat 403 errors (e.g. cap exceeded) as fatal.
51729
51730 • Implement cleanup command for deleting old file versions.
51731
51732 • Make error handling compliant with B2 integrations notes.
51733
51734 • Fix handling of token expiry.
51735
51736 • Implement --b2-test-mode to set X-Bz-Test-Mode header.
51737
51738 • Set cutoff for chunked upload to 200MB as per B2 guidelines.
51739
51740 • Make upload multi-threaded.
51741
51742 • Dropbox
51743
51744 • Don't retry 461 errors.
51745
51746 v1.30 - 2016-06-18
51747 • New Features
51748
51749 • Directory listing code reworked for more features and better error
51750 reporting (thanks to Klaus Post for help). This enables
51751
51752 • Directory include filtering for efficiency
51753
51754 • --max-depth parameter
51755
51756 • Better error reporting
51757
51758 • More to come
51759
51760 • Retry more errors
51761
51762 • Add --ignore-size flag - for uploading images to onedrive
51763
51764 • Log -v output to stdout by default
51765
51766 • Display the transfer stats in more human-readable form
51767
51768 • Make 0 size files specifiable with --max-size 0b
51769
51770 • Add b suffix so we can specify bytes in --bwlimit, --min-size, etc.
51771
51772 • Use "password:" instead of "password>" prompt - thanks Klaus Post
51773 and Leigh Klotz
51774
51775 • Bug Fixes
51776
51777 • Fix retry doing one too many retries
51778
51779 • Local
51780
51781 • Fix problems with OS X and UTF-8 characters
51782
51783 • Amazon Drive
51784
51785 • Check a file exists before uploading to help with 408 Conflict er‐
51786 rors
51787
51788 • Reauth on 401 errors - this has been causing a lot of problems
51789
51790 • Work around spurious 403 errors
51791
51792 • Restart directory listings on error
51793
51794 • Google Drive
51795
51796 • Check a file exists before uploading to help with duplicates
51797
51798 • Fix retry of multipart uploads
51799
51800 • Backblaze B2
51801
51802 • Implement large file uploading
51803
51804 • S3
51805
51806 • Add AES256 server-side encryption for - thanks Justin R. Wilson
51807
51808 • Google Cloud Storage
51809
51810 • Make sure we don't use conflicting content types on upload
51811
51812 • Add service account support - thanks Michal Witkowski
51813
51814 • Swift
51815
51816 • Add auth version parameter
51817
51818 • Add domain option for openstack (v3 auth) - thanks Fabian Ruff
51819
51820 v1.29 - 2016-04-18
51821 • New Features
51822
51823 • Implement -I, --ignore-times for unconditional upload
51824
51825 • Improve dedupecommand
51826
51827 • Now removes identical copies without asking
51828
51829 • Now obeys --dry-run
51830
51831 • Implement --dedupe-mode for non interactive running
51832
51833 • --dedupe-mode interactive - interactive the default.
51834
51835 • --dedupe-mode skip - removes identical files then skips any‐
51836 thing left.
51837
51838 • --dedupe-mode first - removes identical files then keeps the
51839 first one.
51840
51841 • --dedupe-mode newest - removes identical files then keeps the
51842 newest one.
51843
51844 • --dedupe-mode oldest - removes identical files then keeps the
51845 oldest one.
51846
51847 • --dedupe-mode rename - removes identical files then renames the
51848 rest to be different.
51849
51850 • Bug fixes
51851
51852 • Make rclone check obey the --size-only flag.
51853
51854 • Use "application/octet-stream" if discovered mime type is invalid.
51855
51856 • Fix missing "quit" option when there are no remotes.
51857
51858 • Google Drive
51859
51860 • Increase default chunk size to 8 MB - increases upload speed of big
51861 files
51862
51863 • Speed up directory listings and make more reliable
51864
51865 • Add missing retries for Move and DirMove - increases reliability
51866
51867 • Preserve mime type on file update
51868
51869 • Backblaze B2
51870
51871 • Enable mod time syncing
51872
51873 • This means that B2 will now check modification times
51874
51875 • It will upload new files to update the modification times
51876
51877 • (there isn't an API to just set the mod time.)
51878
51879 • If you want the old behaviour use --size-only.
51880
51881 • Update API to new version
51882
51883 • Fix parsing of mod time when not in metadata
51884
51885 • Swift/Hubic
51886
51887 • Don't return an MD5SUM for static large objects
51888
51889 • S3
51890
51891 • Fix uploading files bigger than 50GB
51892
51893 v1.28 - 2016-03-01
51894 • New Features
51895
51896 • Configuration file encryption - thanks Klaus Post
51897
51898 • Improve rclone config adding more help and making it easier to un‐
51899 derstand
51900
51901 • Implement -u/--update so creation times can be used on all remotes
51902
51903 • Implement --low-level-retries flag
51904
51905 • Optionally disable gzip compression on downloads with --no-gzip-en‐
51906 coding
51907
51908 • Bug fixes
51909
51910 • Don't make directories if --dry-run set
51911
51912 • Fix and document the move command
51913
51914 • Fix redirecting stderr on unix-like OSes when using --log-file
51915
51916 • Fix delete command to wait until all finished - fixes missing
51917 deletes.
51918
51919 • Backblaze B2
51920
51921 • Use one upload URL per go routine fixes more than one upload using
51922 auth token
51923
51924 • Add pacing, retries and reauthentication - fixes token expiry prob‐
51925 lems
51926
51927 • Upload without using a temporary file from local (and remotes which
51928 support SHA1)
51929
51930 • Fix reading metadata for all files when it shouldn't have been
51931
51932 • Drive
51933
51934 • Fix listing drive documents at root
51935
51936 • Disable copy and move for Google docs
51937
51938 • Swift
51939
51940 • Fix uploading of chunked files with non ASCII characters
51941
51942 • Allow setting of storage_url in the config - thanks Xavier Lucas
51943
51944 • S3
51945
51946 • Allow IAM role and credentials from environment variables - thanks
51947 Brian Stengaard
51948
51949 • Allow low privilege users to use S3 (check if directory exists dur‐
51950 ing Mkdir) - thanks Jakub Gedeon
51951
51952 • Amazon Drive
51953
51954 • Retry on more things to make directory listings more reliable
51955
51956 v1.27 - 2016-01-31
51957 • New Features
51958
51959 • Easier headless configuration with rclone authorize
51960
51961 • Add support for multiple hash types - we now check SHA1 as well as
51962 MD5 hashes.
51963
51964 • delete command which does obey the filters (unlike purge)
51965
51966 • dedupe command to deduplicate a remote. Useful with Google Drive.
51967
51968 • Add --ignore-existing flag to skip all files that exist on destina‐
51969 tion.
51970
51971 • Add --delete-before, --delete-during, --delete-after flags.
51972
51973 • Add --memprofile flag to debug memory use.
51974
51975 • Warn the user about files with same name but different case
51976
51977 • Make --include rules add their implicit exclude * at the end of the
51978 filter list
51979
51980 • Deprecate compiling with go1.3
51981
51982 • Amazon Drive
51983
51984 • Fix download of files > 10 GB
51985
51986 • Fix directory traversal ("Next token is expired") for large direc‐
51987 tory listings
51988
51989 • Remove 409 conflict from error codes we will retry - stops very
51990 long pauses
51991
51992 • Backblaze B2
51993
51994 • SHA1 hashes now checked by rclone core
51995
51996 • Drive
51997
51998 • Add --drive-auth-owner-only to only consider files owned by the us‐
51999 er - thanks Björn Harrtell
52000
52001 • Export Google documents
52002
52003 • Dropbox
52004
52005 • Make file exclusion error controllable with -q
52006
52007 • Swift
52008
52009 • Fix upload from unprivileged user.
52010
52011 • S3
52012
52013 • Fix updating of mod times of files with + in.
52014
52015 • Local
52016
52017 • Add local file system option to disable UNC on Windows.
52018
52019 v1.26 - 2016-01-02
52020 • New Features
52021
52022 • Yandex storage backend - thank you Dmitry Burdeev ("dibu")
52023
52024 • Implement Backblaze B2 storage backend
52025
52026 • Add --min-age and --max-age flags - thank you Adriano Aurélio
52027 Meirelles
52028
52029 • Make ls/lsl/md5sum/size/check obey includes and excludes
52030
52031 • Fixes
52032
52033 • Fix crash in http logging
52034
52035 • Upload releases to github too
52036
52037 • Swift
52038
52039 • Fix sync for chunked files
52040
52041 • OneDrive
52042
52043 • Re-enable server-side copy
52044
52045 • Don't mask HTTP error codes with JSON decode error
52046
52047 • S3
52048
52049 • Fix corrupting Content-Type on mod time update (thanks Joseph
52050 Spurrier)
52051
52052 v1.25 - 2015-11-14
52053 • New features
52054
52055 • Implement Hubic storage system
52056
52057 • Fixes
52058
52059 • Fix deletion of some excluded files without --delete-excluded
52060
52061 • This could have deleted files unexpectedly on sync
52062
52063 • Always check first with --dry-run!
52064
52065 • Swift
52066
52067 • Stop SetModTime losing metadata (e.g. X-Object-Manifest)
52068
52069 • This could have caused data loss for files > 5GB in size
52070
52071 • Use ContentType from Object to avoid lookups in listings
52072
52073 • OneDrive
52074
52075 • disable server-side copy as it seems to be broken at Microsoft
52076
52077 v1.24 - 2015-11-07
52078 • New features
52079
52080 • Add support for Microsoft OneDrive
52081
52082 • Add --no-check-certificate option to disable server certificate
52083 verification
52084
52085 • Add async readahead buffer for faster transfer of big files
52086
52087 • Fixes
52088
52089 • Allow spaces in remotes and check remote names for validity at cre‐
52090 ation time
52091
52092 • Allow '&' and disallow ':' in Windows filenames.
52093
52094 • Swift
52095
52096 • Ignore directory marker objects where appropriate - allows working
52097 with Hubic
52098
52099 • Don't delete the container if fs wasn't at root
52100
52101 • S3
52102
52103 • Don't delete the bucket if fs wasn't at root
52104
52105 • Google Cloud Storage
52106
52107 • Don't delete the bucket if fs wasn't at root
52108
52109 v1.23 - 2015-10-03
52110 • New features
52111
52112 • Implement rclone size for measuring remotes
52113
52114 • Fixes
52115
52116 • Fix headless config for drive and gcs
52117
52118 • Tell the user they should try again if the webserver method failed
52119
52120 • Improve output of --dump-headers
52121
52122 • S3
52123
52124 • Allow anonymous access to public buckets
52125
52126 • Swift
52127
52128 • Stop chunked operations logging "Failed to read info: Object Not
52129 Found"
52130
52131 • Use Content-Length on uploads for extra reliability
52132
52133 v1.22 - 2015-09-28
52134 • Implement rsync like include and exclude flags
52135
52136 • swift
52137
52138 • Support files > 5GB - thanks Sergey Tolmachev
52139
52140 v1.21 - 2015-09-22
52141 • New features
52142
52143 • Display individual transfer progress
52144
52145 • Make lsl output times in localtime
52146
52147 • Fixes
52148
52149 • Fix allowing user to override credentials again in Drive, GCS and
52150 ACD
52151
52152 • Amazon Drive
52153
52154 • Implement compliant pacing scheme
52155
52156 • Google Drive
52157
52158 • Make directory reads concurrent for increased speed.
52159
52160 v1.20 - 2015-09-15
52161 • New features
52162
52163 • Amazon Drive support
52164
52165 • Oauth support redone - fix many bugs and improve usability
52166
52167 • Use "golang.org/x/oauth2" as oauth library of choice
52168
52169 • Improve oauth usability for smoother initial signup
52170
52171 • drive, googlecloudstorage: optionally use auto config for the
52172 oauth token
52173
52174 • Implement --dump-headers and --dump-bodies debug flags
52175
52176 • Show multiple matched commands if abbreviation too short
52177
52178 • Implement server-side move where possible
52179
52180 • local
52181
52182 • Always use UNC paths internally on Windows - fixes a lot of bugs
52183
52184 • dropbox
52185
52186 • force use of our custom transport which makes timeouts work
52187
52188 • Thanks to Klaus Post for lots of help with this release
52189
52190 v1.19 - 2015-08-28
52191 • New features
52192
52193 • Server side copies for s3/swift/drive/dropbox/gcs
52194
52195 • Move command - uses server-side copies if it can
52196
52197 • Implement --retries flag - tries 3 times by default
52198
52199 • Build for plan9/amd64 and solaris/amd64 too
52200
52201 • Fixes
52202
52203 • Make a current version download with a fixed URL for scripting
52204
52205 • Ignore rmdir in limited fs rather than throwing error
52206
52207 • dropbox
52208
52209 • Increase chunk size to improve upload speeds massively
52210
52211 • Issue an error message when trying to upload bad file name
52212
52213 v1.18 - 2015-08-17
52214 • drive
52215
52216 • Add --drive-use-trash flag so rclone trashes instead of deletes
52217
52218 • Add "Forbidden to download" message for files with no downloadURL
52219
52220 • dropbox
52221
52222 • Remove datastore
52223
52224 • This was deprecated and it caused a lot of problems
52225
52226 • Modification times and MD5SUMs no longer stored
52227
52228 • Fix uploading files > 2GB
52229
52230 • s3
52231
52232 • use official AWS SDK from github.com/aws/aws-sdk-go
52233
52234 • NB will most likely require you to delete and recreate remote
52235
52236 • enable multipart upload which enables files > 5GB
52237
52238 • tested with Ceph / RadosGW / S3 emulation
52239
52240 • many thanks to Sam Liston and Brian Haymore at the Utah Center for
52241 High Performance Computing (https://www.chpc.utah.edu/) for a Ceph
52242 test account
52243
52244 • misc
52245
52246 • Show errors when reading the config file
52247
52248 • Do not print stats in quiet mode - thanks Leonid Shalupov
52249
52250 • Add FAQ
52251
52252 • Fix created directories not obeying umask
52253
52254 • Linux installation instructions - thanks Shimon Doodkin
52255
52256 v1.17 - 2015-06-14
52257 • dropbox: fix case insensitivity issues - thanks Leonid Shalupov
52258
52259 v1.16 - 2015-06-09
52260 • Fix uploading big files which was causing timeouts or panics
52261
52262 • Don't check md5sum after download with --size-only
52263
52264 v1.15 - 2015-06-06
52265 • Add --checksum flag to only discard transfers by MD5SUM - thanks Alex
52266 Couper
52267
52268 • Implement --size-only flag to sync on size not checksum & modtime
52269
52270 • Expand docs and remove duplicated information
52271
52272 • Document rclone's limitations with directories
52273
52274 • dropbox: update docs about case insensitivity
52275
52276 v1.14 - 2015-05-21
52277 • local: fix encoding of non utf-8 file names - fixes a duplicate file
52278 problem
52279
52280 • drive: docs about rate limiting
52281
52282 • google cloud storage: Fix compile after API change in
52283 "google.golang.org/api/storage/v1"
52284
52285 v1.13 - 2015-05-10
52286 • Revise documentation (especially sync)
52287
52288 • Implement --timeout and --conntimeout
52289
52290 • s3: ignore etags from multipart uploads which aren't md5sums
52291
52292 v1.12 - 2015-03-15
52293 • drive: Use chunked upload for files above a certain size
52294
52295 • drive: add --drive-chunk-size and --drive-upload-cutoff parameters
52296
52297 • drive: switch to insert from update when a failed copy deletes the
52298 upload
52299
52300 • core: Log duplicate files if they are detected
52301
52302 v1.11 - 2015-03-04
52303 • swift: add region parameter
52304
52305 • drive: fix crash on failed to update remote mtime
52306
52307 • In remote paths, change native directory separators to /
52308
52309 • Add synchronization to ls/lsl/lsd output to stop corruptions
52310
52311 • Ensure all stats/log messages to go stderr
52312
52313 • Add --log-file flag to log everything (including panics) to file
52314
52315 • Make it possible to disable stats printing with --stats=0
52316
52317 • Implement --bwlimit to limit data transfer bandwidth
52318
52319 v1.10 - 2015-02-12
52320 • s3: list an unlimited number of items
52321
52322 • Fix getting stuck in the configurator
52323
52324 v1.09 - 2015-02-07
52325 • windows: Stop drive letters (e.g. C:) getting mixed up with remotes
52326 (e.g. drive:)
52327
52328 • local: Fix directory separators on Windows
52329
52330 • drive: fix rate limit exceeded errors
52331
52332 v1.08 - 2015-02-04
52333 • drive: fix subdirectory listing to not list entire drive
52334
52335 • drive: Fix SetModTime
52336
52337 • dropbox: adapt code to recent library changes
52338
52339 v1.07 - 2014-12-23
52340 • google cloud storage: fix memory leak
52341
52342 v1.06 - 2014-12-12
52343 • Fix "Couldn't find home directory" on OSX
52344
52345 • swift: Add tenant parameter
52346
52347 • Use new location of Google API packages
52348
52349 v1.05 - 2014-08-09
52350 • Improved tests and consequently lots of minor fixes
52351
52352 • core: Fix race detected by go race detector
52353
52354 • core: Fixes after running errcheck
52355
52356 • drive: reset root directory on Rmdir and Purge
52357
52358 • fs: Document that Purger returns error on empty directory, test and
52359 fix
52360
52361 • google cloud storage: fix ListDir on subdirectory
52362
52363 • google cloud storage: re-read metadata in SetModTime
52364
52365 • s3: make reading metadata more reliable to work around eventual con‐
52366 sistency problems
52367
52368 • s3: strip trailing / from ListDir()
52369
52370 • swift: return directories without / in ListDir
52371
52372 v1.04 - 2014-07-21
52373 • google cloud storage: Fix crash on Update
52374
52375 v1.03 - 2014-07-20
52376 • swift, s3, dropbox: fix updated files being marked as corrupted
52377
52378 • Make compile with go 1.1 again
52379
52380 v1.02 - 2014-07-19
52381 • Implement Dropbox remote
52382
52383 • Implement Google Cloud Storage remote
52384
52385 • Verify Md5sums and Sizes after copies
52386
52387 • Remove times from "ls" command - lists sizes only
52388
52389 • Add add "lsl" - lists times and sizes
52390
52391 • Add "md5sum" command
52392
52393 v1.01 - 2014-07-04
52394 • drive: fix transfer of big files using up lots of memory
52395
52396 v1.00 - 2014-07-03
52397 • drive: fix whole second dates
52398
52399 v0.99 - 2014-06-26
52400 • Fix --dry-run not working
52401
52402 • Make compatible with go 1.1
52403
52404 v0.98 - 2014-05-30
52405 • s3: Treat missing Content-Length as 0 for some ceph installations
52406
52407 • rclonetest: add file with a space in
52408
52409 v0.97 - 2014-05-05
52410 • Implement copying of single files
52411
52412 • s3 & swift: support paths inside containers/buckets
52413
52414 v0.96 - 2014-04-24
52415 • drive: Fix multiple files of same name being created
52416
52417 • drive: Use o.Update and fs.Put to optimise transfers
52418
52419 • Add version number, -V and --version
52420
52421 v0.95 - 2014-03-28
52422 • rclone.org: website, docs and graphics
52423
52424 • drive: fix path parsing
52425
52426 v0.94 - 2014-03-27
52427 • Change remote format one last time
52428
52429 • GNU style flags
52430
52431 v0.93 - 2014-03-16
52432 • drive: store token in config file
52433
52434 • cross compile other versions
52435
52436 • set strict permissions on config file
52437
52438 v0.92 - 2014-03-15
52439 • Config fixes and --config option
52440
52441 v0.91 - 2014-03-15
52442 • Make config file
52443
52444 v0.90 - 2013-06-27
52445 • Project named rclone
52446
52447 v0.00 - 2012-11-18
52448 • Project started
52449
52451 Limitations
52452 Directory timestamps aren't preserved
52453 Rclone doesn't currently preserve the timestamps of directories. This
52454 is because rclone only really considers objects when syncing.
52455
52456 Rclone struggles with millions of files in a directory/bucket
52457 Currently rclone loads each directory/bucket entirely into memory be‐
52458 fore using it. Since each rclone object takes 0.5k-1k of memory this
52459 can take a very long time and use a large amount of memory.
52460
52461 Millions of files in a directory tends to occur on bucket-based remotes
52462 (e.g. S3 buckets) since those remotes do not segregate subdirectories
52463 within the bucket.
52464
52465 Bucket-based remotes and folders
52466 Bucket-based remotes (e.g. S3/GCS/Swift/B2) do not have a concept of
52467 directories. Rclone therefore cannot create directories in them which
52468 means that empty directories on a bucket-based remote will tend to dis‐
52469 appear.
52470
52471 Some software creates empty keys ending in / as directory markers.
52472 Rclone doesn't do this as it potentially creates more objects and costs
52473 more. This ability may be added in the future (probably via a flag/op‐
52474 tion).
52475
52476 Bugs
52477 Bugs are stored in rclone's GitHub project:
52478
52479 • Reported bugs (https://github.com/rclone/rclone/is‐
52480 sues?q=is%3Aopen+is%3Aissue+label%3Abug)
52481
52482 • Known issues (https://github.com/rclone/rclone/is‐
52483 sues?q=is%3Aopen+is%3Aissue+milestone%3A%22Known+Problem%22)
52484
52486 Do all cloud storage systems support all rclone commands
52487 Yes they do. All the rclone commands (e.g. sync, copy, etc.) will
52488 work on all the remote storage systems.
52489
52490 Can I copy the config from one machine to another
52491 Sure! Rclone stores all of its config in a single file. If you want to
52492 find this file, run rclone config file which will tell you where it is.
52493
52494 See the remote setup docs (https://rclone.org/remote_setup/) for more
52495 info.
52496
52497 How do I configure rclone on a remote / headless box with no browser?
52498 This has now been documented in its own remote setup page
52499 (https://rclone.org/remote_setup/).
52500
52501 Can rclone sync directly from drive to s3
52502 Rclone can sync between two remote cloud storage systems just fine.
52503
52504 Note that it effectively downloads the file and uploads it again, so
52505 the node running rclone would need to have lots of bandwidth.
52506
52507 The syncs would be incremental (on a file by file basis).
52508
52509 e.g.
52510
52511 rclone sync -i drive:Folder s3:bucket
52512
52513 Using rclone from multiple locations at the same time
52514 You can use rclone from multiple places at the same time if you choose
52515 different subdirectory for the output, e.g.
52516
52517 Server A> rclone sync -i /tmp/whatever remote:ServerA
52518 Server B> rclone sync -i /tmp/whatever remote:ServerB
52519
52520 If you sync to the same directory then you should use rclone copy oth‐
52521 erwise the two instances of rclone may delete each other's files, e.g.
52522
52523 Server A> rclone copy /tmp/whatever remote:Backup
52524 Server B> rclone copy /tmp/whatever remote:Backup
52525
52526 The file names you upload from Server A and Server B should be differ‐
52527 ent in this case, otherwise some file systems (e.g. Drive) may make
52528 duplicates.
52529
52530 Why doesn't rclone support partial transfers / binary diffs like rsync?
52531 Rclone stores each file you transfer as a native object on the remote
52532 cloud storage system. This means that you can see the files you upload
52533 as expected using alternative access methods (e.g. using the Google
52534 Drive web interface). There is a 1:1 mapping between files on your
52535 hard disk and objects created in the cloud storage system.
52536
52537 Cloud storage systems (at least none I've come across yet) don't sup‐
52538 port partially uploading an object. You can't take an existing object,
52539 and change some bytes in the middle of it.
52540
52541 It would be possible to make a sync system which stored binary diffs
52542 instead of whole objects like rclone does, but that would break the 1:1
52543 mapping of files on your hard disk to objects in the remote cloud stor‐
52544 age system.
52545
52546 All the cloud storage systems support partial downloads of content, so
52547 it would be possible to make partial downloads work. However to make
52548 this work efficiently this would require storing a significant amount
52549 of metadata, which breaks the desired 1:1 mapping of files to objects.
52550
52551 Can rclone do bi-directional sync?
52552 Yes, since rclone v1.58.0, bidirectional cloud sync
52553 (https://rclone.org/bisync/) is available.
52554
52555 Can I use rclone with an HTTP proxy?
52556 Yes. rclone will follow the standard environment variables for prox‐
52557 ies, similar to cURL and other programs.
52558
52559 In general the variables are called http_proxy (for services reached
52560 over http) and https_proxy (for services reached over https). Most
52561 public services will be using https, but you may wish to set both.
52562
52563 The content of the variable is protocol://server:port. The protocol
52564 value is the one used to talk to the proxy server, itself, and is com‐
52565 monly either http or socks5.
52566
52567 Slightly annoyingly, there is no standard for the name; some applica‐
52568 tions may use http_proxy but another one HTTP_PROXY. The Go libraries
52569 used by rclone will try both variations, but you may wish to set all
52570 possibilities. So, on Linux, you may end up with code similar to
52571
52572 export http_proxy=http://proxyserver:12345
52573 export https_proxy=$http_proxy
52574 export HTTP_PROXY=$http_proxy
52575 export HTTPS_PROXY=$http_proxy
52576
52577 Note: If the proxy server requires a username and password, then use
52578
52579 export http_proxy=http://username:password@proxyserver:12345
52580 export https_proxy=$http_proxy
52581 export HTTP_PROXY=$http_proxy
52582 export HTTPS_PROXY=$http_proxy
52583
52584 The NO_PROXY allows you to disable the proxy for specific hosts. Hosts
52585 must be comma separated, and can contain domains or parts. For in‐
52586 stance "foo.com" also matches "bar.foo.com".
52587
52588 e.g.
52589
52590 export no_proxy=localhost,127.0.0.0/8,my.host.name
52591 export NO_PROXY=$no_proxy
52592
52593 Note that the FTP backend does not support ftp_proxy yet.
52594
52595 Rclone gives x509: failed to load system roots and no roots provided error
52596 This means that rclone can't find the SSL root certificates. Likely
52597 you are running rclone on a NAS with a cut-down Linux OS, or possibly
52598 on Solaris.
52599
52600 Rclone (via the Go runtime) tries to load the root certificates from
52601 these places on Linux.
52602
52603 "/etc/ssl/certs/ca-certificates.crt", // Debian/Ubuntu/Gentoo etc.
52604 "/etc/pki/tls/certs/ca-bundle.crt", // Fedora/RHEL
52605 "/etc/ssl/ca-bundle.pem", // OpenSUSE
52606 "/etc/pki/tls/cacert.pem", // OpenELEC
52607
52608 So doing something like this should fix the problem. It also sets the
52609 time which is important for SSL to work properly.
52610
52611 mkdir -p /etc/ssl/certs/
52612 curl -o /etc/ssl/certs/ca-certificates.crt https://raw.githubusercontent.com/bagder/ca-bundle/master/ca-bundle.crt
52613 ntpclient -s -h pool.ntp.org
52614
52615 The two environment variables SSL_CERT_FILE and SSL_CERT_DIR, mentioned
52616 in the x509 package (https://godoc.org/crypto/x509), provide an addi‐
52617 tional way to provide the SSL root certificates.
52618
52619 Note that you may need to add the --insecure option to the curl command
52620 line if it doesn't work without.
52621
52622 curl --insecure -o /etc/ssl/certs/ca-certificates.crt https://raw.githubusercontent.com/bagder/ca-bundle/master/ca-bundle.crt
52623
52624 Rclone gives Failed to load config file: function not implemented error
52625 Likely this means that you are running rclone on Linux version not sup‐
52626 ported by the go runtime, ie earlier than version 2.6.23.
52627
52628 See the system requirements section in the go install docs
52629 (https://golang.org/doc/install) for full details.
52630
52631 All my uploaded docx/xlsx/pptx files appear as archive/zip
52632 This is caused by uploading these files from a Windows computer which
52633 hasn't got the Microsoft Office suite installed. The easiest way to
52634 fix is to install the Word viewer and the Microsoft Office Compatibili‐
52635 ty Pack for Word, Excel, and PowerPoint 2007 and later versions' file
52636 formats
52637
52638 tcp lookup some.domain.com no such host
52639 This happens when rclone cannot resolve a domain. Please check that
52640 your DNS setup is generally working, e.g.
52641
52642 # both should print a long list of possible IP addresses
52643 dig www.googleapis.com # resolve using your default DNS
52644 dig www.googleapis.com @8.8.8.8 # resolve with Google's DNS server
52645
52646 If you are using systemd-resolved (default on Arch Linux), ensure it is
52647 at version 233 or higher. Previous releases contain a bug which causes
52648 not all domains to be resolved properly.
52649
52650 Additionally with the GODEBUG=netdns= environment variable the Go re‐
52651 solver decision can be influenced. This also allows to resolve certain
52652 issues with DNS resolution. See the name resolution section in the go
52653 docs (https://golang.org/pkg/net/#hdr-Name_Resolution).
52654
52655 The total size reported in the stats for a sync is wrong and keeps changing
52656 It is likely you have more than 10,000 files that need to be synced.
52657 By default, rclone only gets 10,000 files ahead in a sync so as not to
52658 use up too much memory. You can change this default with the --max-
52659 backlog (https://rclone.org/docs/#max-backlog-n) flag.
52660
52661 Rclone is using too much memory or appears to have a memory leak
52662 Rclone is written in Go which uses a garbage collector. The default
52663 settings for the garbage collector mean that it runs when the heap size
52664 has doubled.
52665
52666 However it is possible to tune the garbage collector to use less memory
52667 by setting GOGC (https://dave.cheney.net/tag/gogc) to a lower value,
52668 say export GOGC=20. This will make the garbage collector work harder,
52669 reducing memory size at the expense of CPU usage.
52670
52671 The most common cause of rclone using lots of memory is a single direc‐
52672 tory with thousands or millions of files in. Rclone has to load this
52673 entirely into memory as rclone objects. Each rclone object takes
52674 0.5k-1k of memory.
52675
52676 Rclone changes fullwidth Unicode punctuation marks in file names
52677 For example: On a Windows system, you have a file with name
52678 Test:1.jpg, where : is the Unicode fullwidth colon symbol. When us‐
52679 ing rclone to copy this to your Google Drive, you will notice that the
52680 file gets renamed to Test:1.jpg, where : is the regular (halfwidth)
52681 colon.
52682
52683 The reason for such renames is the way rclone handles different re‐
52684 stricted filenames (https://rclone.org/overview/#restricted-filenames)
52685 on different cloud storage systems. It tries to avoid ambiguous file
52686 names as much and allow moving files between many cloud storage systems
52687 transparently, by replacing invalid characters with similar looking
52688 Unicode characters when transferring to one storage system, and replac‐
52689 ing back again when transferring to a different storage system where
52690 the original characters are supported. When the same Unicode charac‐
52691 ters are intentionally used in file names, this replacement strategy
52692 leads to unwanted renames. Read more here (https://rclone.org/over‐
52693 view/#restricted-filenames-caveats).
52694
52696 This is free software under the terms of the MIT license (check the
52697 COPYING file included with the source code).
52698
52699 Copyright (C) 2019 by Nick Craig-Wood https://www.craig-wood.com/nick/
52700
52701 Permission is hereby granted, free of charge, to any person obtaining a copy
52702 of this software and associated documentation files (the "Software"), to deal
52703 in the Software without restriction, including without limitation the rights
52704 to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
52705 copies of the Software, and to permit persons to whom the Software is
52706 furnished to do so, subject to the following conditions:
52707
52708 The above copyright notice and this permission notice shall be included in
52709 all copies or substantial portions of the Software.
52710
52711 THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
52712 IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
52713 FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
52714 AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
52715 LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
52716 OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
52717 THE SOFTWARE.
52718
52720 Authors
52721 • Nick Craig-Wood <nick@craig-wood.com>
52722
52723 Contributors
52724 {{< rem email addresses removed from here need to be addeed to bin/.ig‐
52725 nore-emails to make sure update-authors.py doesn't immediately put them
52726 back in again. >}}
52727
52728 • Alex Couper <amcouper@gmail.com>
52729
52730 • Leonid Shalupov <leonid@shalupov.com> <shalupov@diverse.org.ru>
52731
52732 • Shimon Doodkin <helpmepro1@gmail.com>
52733
52734 • Colin Nicholson <colin@colinn.com>
52735
52736 • Klaus Post <klauspost@gmail.com>
52737
52738 • Sergey Tolmachev <tolsi.ru@gmail.com>
52739
52740 • Adriano Aurélio Meirelles <adriano@atinge.com>
52741
52742 • C. Bess <cbess@users.noreply.github.com>
52743
52744 • Dmitry Burdeev <dibu28@gmail.com>
52745
52746 • Joseph Spurrier <github@josephspurrier.com>
52747
52748 • Björn Harrtell <bjorn@wololo.org>
52749
52750 • Xavier Lucas <xavier.lucas@corp.ovh.com>
52751
52752 • Werner Beroux <werner@beroux.com>
52753
52754 • Brian Stengaard <brian@stengaard.eu>
52755
52756 • Jakub Gedeon <jgedeon@sofi.com>
52757
52758 • Jim Tittsler <jwt@onjapan.net>
52759
52760 • Michal Witkowski <michal@improbable.io>
52761
52762 • Fabian Ruff <fabian.ruff@sap.com>
52763
52764 • Leigh Klotz <klotz@quixey.com>
52765
52766 • Romain Lapray <lapray.romain@gmail.com>
52767
52768 • Justin R. Wilson <jrw972@gmail.com>
52769
52770 • Antonio Messina <antonio.s.messina@gmail.com>
52771
52772 • Stefan G. Weichinger <office@oops.co.at>
52773
52774 • Per Cederberg <cederberg@gmail.com>
52775
52776 • Radek Šenfeld <rush@logic.cz>
52777
52778 • Fredrik Fornwall <fredrik@fornwall.net>
52779
52780 • Asko Tamm <asko@deekit.net>
52781
52782 • xor-zz <xor@gstocco.com>
52783
52784 • Tomasz Mazur <tmazur90@gmail.com>
52785
52786 • Marco Paganini <paganini@paganini.net>
52787
52788 • Felix Bünemann <buenemann@louis.info>
52789
52790 • Durval Menezes <jmrclone@durval.com>
52791
52792 • Luiz Carlos Rumbelsperger Viana <maxd13_luiz_carlos@hotmail.com>
52793
52794 • Stefan Breunig <stefan-github@yrden.de>
52795
52796 • Alishan Ladhani <ali-l@users.noreply.github.com>
52797
52798 • 0xJAKE <0xJAKE@users.noreply.github.com>
52799
52800 • Thibault Molleman <thibaultmol@users.noreply.github.com>
52801
52802 • Scott McGillivray <scott.mcgillivray@gmail.com>
52803
52804 • Bjørn Erik Pedersen <bjorn.erik.pedersen@gmail.com>
52805
52806 • Lukas Loesche <lukas@mesosphere.io>
52807
52808 • emyarod <allllaboutyou@gmail.com>
52809
52810 • T.C. Ferguson <tcf909@gmail.com>
52811
52812 • Brandur <brandur@mutelight.org>
52813
52814 • Dario Giovannetti <dev@dariogiovannetti.net>
52815
52816 • Károly Oláh <okaresz@aol.com>
52817
52818 • Jon Yergatian <jon@macfanatic.ca>
52819
52820 • Jack Schmidt <github@mowsey.org>
52821
52822 • Dedsec1 <Dedsec1@users.noreply.github.com>
52823
52824 • Hisham Zarka <hzarka@gmail.com>
52825
52826 • Jérôme Vizcaino <jerome.vizcaino@gmail.com>
52827
52828 • Mike Tesch <mjt6129@rit.edu>
52829
52830 • Marvin Watson <marvwatson@users.noreply.github.com>
52831
52832 • Danny Tsai <danny8376@gmail.com>
52833
52834 • Yoni Jah <yonjah+git@gmail.com> <yonjah+github@gmail.com>
52835
52836 • Stephen Harris <github@spuddy.org> <sweharris@users.nore‐
52837 ply.github.com>
52838
52839 • Ihor Dvoretskyi <ihor.dvoretskyi@gmail.com>
52840
52841 • Jon Craton <jncraton@gmail.com>
52842
52843 • Hraban Luyat <hraban@0brg.net>
52844
52845 • Michael Ledin <mledin89@gmail.com>
52846
52847 • Martin Kristensen <me@azgul.com>
52848
52849 • Too Much IO <toomuchio@users.noreply.github.com>
52850
52851 • Anisse Astier <anisse@astier.eu>
52852
52853 • Zahiar Ahmed <zahiar@live.com>
52854
52855 • Igor Kharin <igorkharin@gmail.com>
52856
52857 • Bill Zissimopoulos <billziss@navimatics.com>
52858
52859 • Bob Potter <bobby.potter@gmail.com>
52860
52861 • Steven Lu <tacticalazn@gmail.com>
52862
52863 • Sjur Fredriksen <sjurtf@ifi.uio.no>
52864
52865 • Ruwbin <hubus12345@gmail.com>
52866
52867 • Fabian Möller <fabianm88@gmail.com> <f.moeller@nynex.de>
52868
52869 • Edward Q. Bridges <github@eqbridges.com>
52870
52871 • Vasiliy Tolstov <v.tolstov@selfip.ru>
52872
52873 • Harshavardhana <harsha@minio.io>
52874
52875 • sainaen <sainaen@gmail.com>
52876
52877 • gdm85 <gdm85@users.noreply.github.com>
52878
52879 • Yaroslav Halchenko <debian@onerussian.com>
52880
52881 • John Papandriopoulos <jpap@users.noreply.github.com>
52882
52883 • Zhiming Wang <zmwangx@gmail.com>
52884
52885 • Andy Pilate <cubox@cubox.me>
52886
52887 • Oliver Heyme <olihey@googlemail.com> <olihey@users.nore‐
52888 ply.github.com> <de8olihe@lego.com>
52889
52890 • wuyu <wuyu@yunify.com>
52891
52892 • Andrei Dragomir <adragomi@adobe.com>
52893
52894 • Christian Brüggemann <mail@cbruegg.com>
52895
52896 • Alex McGrath Kraak <amkdude@gmail.com>
52897
52898 • bpicode <bjoern.pirnay@googlemail.com>
52899
52900 • Daniel Jagszent <daniel@jagszent.de>
52901
52902 • Josiah White <thegenius2009@gmail.com>
52903
52904 • Ishuah Kariuki <kariuki@ishuah.com> <ishuah91@gmail.com>
52905
52906 • Jan Varho <jan@varho.org>
52907
52908 • Girish Ramakrishnan <girish@cloudron.io>
52909
52910 • LingMan <LingMan@users.noreply.github.com>
52911
52912 • Jacob McNamee <jacobmcnamee@gmail.com>
52913
52914 • jersou <jertux@gmail.com>
52915
52916 • thierry <thierry@substantiel.fr>
52917
52918 • Simon Leinen <simon.leinen@gmail.com> <ubuntu@s3-test.novalocal>
52919
52920 • Dan Dascalescu <ddascalescu+github@gmail.com>
52921
52922 • Jason Rose <jason@jro.io>
52923
52924 • Andrew Starr-Bochicchio <a.starr.b@gmail.com>
52925
52926 • John Leach <john@johnleach.co.uk>
52927
52928 • Corban Raun <craun@instructure.com>
52929
52930 • Pierre Carlson <mpcarl@us.ibm.com>
52931
52932 • Ernest Borowski <er.borowski@gmail.com>
52933
52934 • Remus Bunduc <remus.bunduc@gmail.com>
52935
52936 • Iakov Davydov <iakov.davydov@unil.ch> <dav05.gith@myths.ru>
52937
52938 • Jakub Tasiemski <tasiemski@gmail.com>
52939
52940 • David Minor <dminor@saymedia.com>
52941
52942 • Tim Cooijmans <cooijmans.tim@gmail.com>
52943
52944 • Laurence <liuxy6@gmail.com>
52945
52946 • Giovanni Pizzi <gio.piz@gmail.com>
52947
52948 • Filip Bartodziej <filipbartodziej@gmail.com>
52949
52950 • Jon Fautley <jon@dead.li>
52951
52952 • lewapm <32110057+lewapm@users.noreply.github.com>
52953
52954 • Yassine Imounachen <yassine256@gmail.com>
52955
52956 • Chris Redekop <chris-redekop@users.noreply.github.com> <chris.re‐
52957 dekop@gmail.com>
52958
52959 • Jon Fautley <jon@adenoid.appstal.co.uk>
52960
52961 • Will Gunn <WillGunn@users.noreply.github.com>
52962
52963 • Lucas Bremgartner <lucas@bremis.ch>
52964
52965 • Jody Frankowski <jody.frankowski@gmail.com>
52966
52967 • Andreas Roussos <arouss1980@gmail.com>
52968
52969 • nbuchanan <nbuchanan@utah.gov>
52970
52971 • Durval Menezes <rclone@durval.com>
52972
52973 • Victor <vb-github@viblo.se>
52974
52975 • Mateusz <pabian.mateusz@gmail.com>
52976
52977 • Daniel Loader <spicypixel@gmail.com>
52978
52979 • David0rk <davidork@gmail.com>
52980
52981 • Alexander Neumann <alexander@bumpern.de>
52982
52983 • Giri Badanahatti <gbadanahatti@us.ibm.com@Giris-MacBook-Pro.local>
52984
52985 • Leo R. Lundgren <leo@finalresort.org>
52986
52987 • wolfv <wolfv6@users.noreply.github.com>
52988
52989 • Dave Pedu <dave@davepedu.com>
52990
52991 • Stefan Lindblom <lindblom@spotify.com>
52992
52993 • seuffert <oliver@seuffert.biz>
52994
52995 • gbadanahatti <37121690+gbadanahatti@users.noreply.github.com>
52996
52997 • Keith Goldfarb <barkofdelight@gmail.com>
52998
52999 • Steve Kriss <steve@heptio.com>
53000
53001 • Chih-Hsuan Yen <yan12125@gmail.com>
53002
53003 • Alexander Neumann <fd0@users.noreply.github.com>
53004
53005 • Matt Holt <mholt@users.noreply.github.com>
53006
53007 • Eri Bastos <bastos.eri@gmail.com>
53008
53009 • Michael P. Dubner <pywebmail@list.ru>
53010
53011 • Antoine GIRARD <sapk@users.noreply.github.com>
53012
53013 • Mateusz Piotrowski <mpp302@gmail.com>
53014
53015 • Animosity022 <animosity22@users.noreply.github.com> <earl.tex‐
53016 ter@gmail.com>
53017
53018 • Peter Baumgartner <pete@lincolnloop.com>
53019
53020 • Craig Rachel <craig@craigrachel.com>
53021
53022 • Michael G. Noll <miguno@users.noreply.github.com>
53023
53024 • hensur <me@hensur.de>
53025
53026 • Oliver Heyme <de8olihe@lego.com>
53027
53028 • Richard Yang <richard@yenforyang.com>
53029
53030 • Piotr Oleszczyk <piotr.oleszczyk@gmail.com>
53031
53032 • Rodrigo <rodarima@gmail.com>
53033
53034 • NoLooseEnds <NoLooseEnds@users.noreply.github.com>
53035
53036 • Jakub Karlicek <jakub@karlicek.me>
53037
53038 • John Clayton <john@codemonkeylabs.com>
53039
53040 • Kasper Byrdal Nielsen <byrdal76@gmail.com>
53041
53042 • Benjamin Joseph Dag <bjdag1234@users.noreply.github.com>
53043
53044 • themylogin <themylogin@gmail.com>
53045
53046 • Onno Zweers <onno.zweers@surfsara.nl>
53047
53048 • Jasper Lievisse Adriaanse <jasper@humppa.nl>
53049
53050 • sandeepkru <sandeep.ummadi@gmail.com> <sandeepkru@users.nore‐
53051 ply.github.com>
53052
53053 • HerrH <atomtigerzoo@users.noreply.github.com>
53054
53055 • Andrew <4030760+sparkyman215@users.noreply.github.com>
53056
53057 • dan smith <XX1011@gmail.com>
53058
53059 • Oleg Kovalov <iamolegkovalov@gmail.com>
53060
53061 • Ruben Vandamme <github-com-00ff86@vandamme.email>
53062
53063 • Cnly <minecnly@gmail.com>
53064
53065 • Andres Alvarez <1671935+kir4h@users.noreply.github.com>
53066
53067 • reddi1 <xreddi@gmail.com>
53068
53069 • Matt Tucker <matthewtckr@gmail.com>
53070
53071 • Sebastian Bünger <buengese@gmail.com> <buengese@protonmail.com>
53072
53073 • Martin Polden <mpolden@mpolden.no>
53074
53075 • Alex Chen <Cnly@users.noreply.github.com>
53076
53077 • Denis <deniskovpen@gmail.com>
53078
53079 • bsteiss <35940619+bsteiss@users.noreply.github.com>
53080
53081 • Cédric Connes <cedric.connes@gmail.com>
53082
53083 • Dr. Tobias Quathamer <toddy15@users.noreply.github.com>
53084
53085 • dcpu <42736967+dcpu@users.noreply.github.com>
53086
53087 • Sheldon Rupp <me@shel.io>
53088
53089 • albertony <12441419+albertony@users.noreply.github.com>
53090
53091 • cron410 <cron410@gmail.com>
53092
53093 • Anagh Kumar Baranwal <6824881+darthShadow@users.noreply.github.com>
53094
53095 • Felix Brucker <felix@felixbrucker.com>
53096
53097 • Santiago Rodríguez <scollazo@users.noreply.github.com>
53098
53099 • Craig Miskell <craig.miskell@fluxfederation.com>
53100
53101 • Antoine GIRARD <sapk@sapk.fr>
53102
53103 • Joanna Marek <joanna.marek@u2i.com>
53104
53105 • frenos <frenos@users.noreply.github.com>
53106
53107 • ssaqua <ssaqua@users.noreply.github.com>
53108
53109 • xnaas <me@xnaas.info>
53110
53111 • Frantisek Fuka <fuka@fuxoft.cz>
53112
53113 • Paul Kohout <pauljkohout@yahoo.com>
53114
53115 • dcpu <43330287+dcpu@users.noreply.github.com>
53116
53117 • jackyzy823 <jackyzy823@gmail.com>
53118
53119 • David Haguenauer <ml@kurokatta.org>
53120
53121 • teresy <hi.teresy@gmail.com>
53122
53123 • buergi <patbuergi@gmx.de>
53124
53125 • Florian Gamboeck <mail@floga.de>
53126
53127 • Ralf Hemberger <10364191+rhemberger@users.noreply.github.com>
53128
53129 • Scott Edlund <sedlund@users.noreply.github.com>
53130
53131 • Erik Swanson <erik@retailnext.net>
53132
53133 • Jake Coggiano <jake@stripe.com>
53134
53135 • brused27 <brused27@noemailaddress>
53136
53137 • Peter Kaminski <kaminski@istori.com>
53138
53139 • Henry Ptasinski <henry@logout.com>
53140
53141 • Alexander <kharkovalexander@gmail.com>
53142
53143 • Garry McNulty <garrmcnu@gmail.com>
53144
53145 • Mathieu Carbou <mathieu.carbou@gmail.com>
53146
53147 • Mark Otway <mark@otway.com>
53148
53149 • William Cocker <37018962+WilliamCocker@users.noreply.github.com>
53150
53151 • François Leurent <131.js@cloudyks.org>
53152
53153 • Arkadius Stefanski <arkste@gmail.com>
53154
53155 • Jay <dev@jaygoel.com>
53156
53157 • andrea rota <a@xelera.eu>
53158
53159 • nicolov <nicolov@users.noreply.github.com>
53160
53161 • Dario Guzik <dario@guzik.com.ar>
53162
53163 • qip <qip@users.noreply.github.com>
53164
53165 • yair@unicorn <yair@unicorn>
53166
53167 • Matt Robinson <brimstone@the.narro.ws>
53168
53169 • kayrus <kay.diam@gmail.com>
53170
53171 • Rémy Léone <remy.leone@gmail.com>
53172
53173 • Wojciech Smigielski <wojciech.hieronim.smigielski@gmail.com>
53174
53175 • weetmuts <oehrstroem@gmail.com>
53176
53177 • Jonathan <vanillajonathan@users.noreply.github.com>
53178
53179 • James Carpenter <orbsmiv@users.noreply.github.com>
53180
53181 • Vince <vince0villamora@gmail.com>
53182
53183 • Nestar47 <47841759+Nestar47@users.noreply.github.com>
53184
53185 • Six <brbsix@gmail.com>
53186
53187 • Alexandru Bumbacea <alexandru.bumbacea@booking.com>
53188
53189 • calisro <robert.calistri@gmail.com>
53190
53191 • Dr.Rx <david.rey@nventive.com>
53192
53193 • marcintustin <marcintustin@users.noreply.github.com>
53194
53195 • jaKa Močnik <jaka@koofr.net>
53196
53197 • Fionera <fionera@fionera.de>
53198
53199 • Dan Walters <dan@walters.io>
53200
53201 • Danil Semelenov <sgtpep@users.noreply.github.com>
53202
53203 • xopez <28950736+xopez@users.noreply.github.com>
53204
53205 • Ben Boeckel <mathstuf@gmail.com>
53206
53207 • Manu <manu@snapdragon.cc>
53208
53209 • Kyle E. Mitchell <kyle@kemitchell.com>
53210
53211 • Gary Kim <gary@garykim.dev>
53212
53213 • Jon <jonathn@github.com>
53214
53215 • Jeff Quinn <jeffrey.quinn@bluevoyant.com>
53216
53217 • Peter Berbec <peter@berbec.com>
53218
53219 • didil <1284255+didil@users.noreply.github.com>
53220
53221 • id01 <gaviniboom@gmail.com>
53222
53223 • Robert Marko <robimarko@gmail.com>
53224
53225 • Philip Harvey <32467456+pharveybattelle@users.noreply.github.com>
53226
53227 • JorisE <JorisE@users.noreply.github.com>
53228
53229 • garry415 <garry.415@gmail.com>
53230
53231 • forgems <forgems@gmail.com>
53232
53233 • Florian Apolloner <florian@apolloner.eu>
53234
53235 • Aleksandar Janković <office@ajankovic.com> <ajankovic@users.nore‐
53236 ply.github.com>
53237
53238 • Maran <maran@protonmail.com>
53239
53240 • nguyenhuuluan434 <nguyenhuuluan434@gmail.com>
53241
53242 • Laura Hausmann <zotan@zotan.pw> <laura@hausmann.dev>
53243
53244 • yparitcher <y@paritcher.com>
53245
53246 • AbelThar <abela.tharen@gmail.com>
53247
53248 • Matti Niemenmaa <matti.niemenmaa+git@iki.fi>
53249
53250 • Russell Davis <russelldavis@users.noreply.github.com>
53251
53252 • Yi FU <yi.fu@tink.se>
53253
53254 • Paul Millar <paul.millar@desy.de>
53255
53256 • justinalin <justinalin@qnap.com>
53257
53258 • EliEron <subanimehd@gmail.com>
53259
53260 • justina777 <chiahuei.lin@gmail.com>
53261
53262 • Chaitanya Bankanhal <bchaitanya15@gmail.com>
53263
53264 • Michał Matczuk <michal@scylladb.com>
53265
53266 • Macavirus <macavirus@zoho.com>
53267
53268 • Abhinav Sharma <abhi18av@outlook.com>
53269
53270 • ginvine <34869051+ginvine@users.noreply.github.com>
53271
53272 • Patrick Wang <mail6543210@yahoo.com.tw>
53273
53274 • Cenk Alti <cenkalti@gmail.com>
53275
53276 • Andreas Chlupka <andy@chlupka.com>
53277
53278 • Alfonso Montero <amontero@tinet.org>
53279
53280 • Ivan Andreev <ivandeex@gmail.com>
53281
53282 • David Baumgold <david@davidbaumgold.com>
53283
53284 • Lars Lehtonen <lars.lehtonen@gmail.com>
53285
53286 • Matei David <matei.david@gmail.com>
53287
53288 • David <david.bramwell@endemolshine.com>
53289
53290 • Anthony Rusdi <33247310+antrusd@users.noreply.github.com>
53291
53292 • Richard Patel <me@terorie.dev>
53293
53294 • 庄天翼 <zty0826@gmail.com>
53295
53296 • SwitchJS <dev@switchjs.com>
53297
53298 • Raphael <PowershellNinja@users.noreply.github.com>
53299
53300 • Sezal Agrawal <sezalagrawal@gmail.com>
53301
53302 • Tyler <TylerNakamura@users.noreply.github.com>
53303
53304 • Brett Dutro <brett.dutro@gmail.com>
53305
53306 • Vighnesh SK <booterror99@gmail.com>
53307
53308 • Arijit Biswas <dibbyo456@gmail.com>
53309
53310 • Michele Caci <michele.caci@gmail.com>
53311
53312 • AlexandrBoltris <ua2fgb@gmail.com>
53313
53314 • Bryce Larson <blarson@saltstack.com>
53315
53316 • Carlos Ferreyra <crypticmind@gmail.com>
53317
53318 • Saksham Khanna <sakshamkhanna@outlook.com>
53319
53320 • dausruddin <5763466+dausruddin@users.noreply.github.com>
53321
53322 • zero-24 <zero-24@users.noreply.github.com>
53323
53324 • Xiaoxing Ye <ye@xiaoxing.us>
53325
53326 • Barry Muldrey <barry@muldrey.net>
53327
53328 • Sebastian Brandt <sebastian.brandt@friday.de>
53329
53330 • Marco Molteni <marco.molteni@mailbox.org>
53331
53332 • Ankur Gupta <7876747+ankur0493@users.noreply.github.com>
53333
53334 • Maciej Zimnoch <maciej@scylladb.com>
53335
53336 • anuar45 <serdaliyev.anuar@gmail.com>
53337
53338 • Fernando <ferferga@users.noreply.github.com>
53339
53340 • David Cole <david.cole@sohonet.com>
53341
53342 • Wei He <git@weispot.com>
53343
53344 • Outvi V <19144373+outloudvi@users.noreply.github.com>
53345
53346 • Thomas Kriechbaumer <thomas@kriechbaumer.name>
53347
53348 • Tennix <tennix@users.noreply.github.com>
53349
53350 • Ole Schütt <ole@schuett.name>
53351
53352 • Kuang-che Wu <kcwu@csie.org>
53353
53354 • Thomas Eales <wingsuit@users.noreply.github.com>
53355
53356 • Paul Tinsley <paul.tinsley@vitalsource.com>
53357
53358 • Felix Hungenberg <git@shiftgeist.com>
53359
53360 • Benjamin Richter <github@dev.telepath.de>
53361
53362 • landall <cst_zf@qq.com>
53363
53364 • thestigma <thestigma@gmail.com>
53365
53366 • jtagcat <38327267+jtagcat@users.noreply.github.com>
53367
53368 • Damon Permezel <permezel@me.com>
53369
53370 • boosh <boosh@users.noreply.github.com>
53371
53372 • unbelauscht <58393353+unbelauscht@users.noreply.github.com>
53373
53374 • Motonori IWAMURO <vmi@nifty.com>
53375
53376 • Benjapol Worakan <benwrk@live.com>
53377
53378 • Dave Koston <dave.koston@stackpath.com>
53379
53380 • Durval Menezes <DurvalMenezes@users.noreply.github.com>
53381
53382 • Tim Gallant <me@timgallant.us>
53383
53384 • Frederick Zhang <frederick888@tsundere.moe>
53385
53386 • valery1707 <valery1707@gmail.com>
53387
53388 • Yves G <theYinYeti@yalis.fr>
53389
53390 • Shing Kit Chan <chanshingkit@gmail.com>
53391
53392 • Franklyn Tackitt <franklyn@tackitt.net>
53393
53394 • Robert-André Mauchin <zebob.m@gmail.com>
53395
53396 • evileye <48332831+ibiruai@users.noreply.github.com>
53397
53398 • Joachim Brandon LeBlanc <brandon@leblanc.codes>
53399
53400 • Patryk Jakuszew <patryk.jakuszew@gmail.com>
53401
53402 • fishbullet <shindu666@gmail.com>
53403
53404 • greatroar <@>
53405
53406 • Bernd Schoolmann <mail@quexten.com>
53407
53408 • Elan Ruusamäe <glen@pld-linux.org>
53409
53410 • Max Sum <max@lolyculture.com>
53411
53412 • Mark Spieth <mspieth@users.noreply.github.com>
53413
53414 • harry <me@harry.plus>
53415
53416 • Samantha McVey <samantham@posteo.net>
53417
53418 • Jack Anderson <jack.anderson@metaswitch.com>
53419
53420 • Michael G <draget@speciesm.net>
53421
53422 • Brandon Philips <brandon@ifup.org>
53423
53424 • Daven <dooven@users.noreply.github.com>
53425
53426 • Martin Stone <martin@d7415.co.uk>
53427
53428 • David Bramwell <13053834+dbramwell@users.noreply.github.com>
53429
53430 • Sunil Patra <snl_su@live.com>
53431
53432 • Adam Stroud <adam.stroud@gmail.com>
53433
53434 • Kush <kushsharma@users.noreply.github.com>
53435
53436 • Matan Rosenberg <matan129@gmail.com>
53437
53438 • gitch1 <63495046+gitch1@users.noreply.github.com>
53439
53440 • ElonH <elonhhuang@gmail.com>
53441
53442 • Fred <fred@creativeprojects.tech>
53443
53444 • Sébastien Gross <renard@users.noreply.github.com>
53445
53446 • Maxime Suret <11944422+msuret@users.noreply.github.com>
53447
53448 • Caleb Case <caleb@storj.io> <calebcase@gmail.com>
53449
53450 • Ben Zenker <imbenzenker@gmail.com>
53451
53452 • Martin Michlmayr <tbm@cyrius.com>
53453
53454 • Brandon McNama <bmcnama@pagerduty.com>
53455
53456 • Daniel Slyman <github@skylayer.eu>
53457
53458 • Alex Guerrero <guerrero@users.noreply.github.com>
53459
53460 • Matteo Pietro Dazzi <matteopietro.dazzi@gft.com>
53461
53462 • edwardxml <56691903+edwardxml@users.noreply.github.com>
53463
53464 • Roman Kredentser <shareed2k@gmail.com>
53465
53466 • Kamil Trzciński <ayufan@ayufan.eu>
53467
53468 • Zac Rubin <z-0@users.noreply.github.com>
53469
53470 • Vincent Feltz <psycho@feltzv.fr>
53471
53472 • Heiko Bornholdt <bornholdt@informatik.uni-hamburg.de>
53473
53474 • Matteo Pietro Dazzi <matteopietro.dazzi@gmail.com>
53475
53476 • jtagcat <gitlab@c7.ee>
53477
53478 • Petri Salminen <petri@salminen.dev>
53479
53480 • Tim Burke <tim.burke@gmail.com>
53481
53482 • Kai Lüke <kai@kinvolk.io>
53483
53484 • Garrett Squire <github@garrettsquire.com>
53485
53486 • Evan Harris <eharris@puremagic.com>
53487
53488 • Kevin <keyam@microsoft.com>
53489
53490 • Morten Linderud <morten@linderud.pw>
53491
53492 • Dmitry Ustalov <dmitry.ustalov@gmail.com>
53493
53494 • Jack <196648+jdeng@users.noreply.github.com>
53495
53496 • kcris <cristian.tarsoaga@gmail.com>
53497
53498 • tyhuber1 <68970760+tyhuber1@users.noreply.github.com>
53499
53500 • David Ibarra <david.ibarra@realty.com>
53501
53502 • Tim Gallant <tim@lilt.com>
53503
53504 • Kaloyan Raev <kaloyan@storj.io>
53505
53506 • Jay McEntire <jay.mcentire@gmail.com>
53507
53508 • Leo Luan <leoluan@us.ibm.com>
53509
53510 • aus <549081+aus@users.noreply.github.com>
53511
53512 • Aaron Gokaslan <agokaslan@fb.com>
53513
53514 • Egor Margineanu <egmar@users.noreply.github.com>
53515
53516 • Lucas Kanashiro <lucas.kanashiro@canonical.com>
53517
53518 • WarpedPixel <WarpedPixel@users.noreply.github.com>
53519
53520 • Sam Edwards <sam@samedwards.ca>
53521
53522 • wjielai <gouki0123@gmail.com>
53523
53524 • Muffin King <jinxz_k@live.com>
53525
53526 • Christopher Stewart <6573710+1f47a@users.noreply.github.com>
53527
53528 • Russell Cattelan <cattelan@digitalelves.com>
53529
53530 • gyutw <30371241+gyutw@users.noreply.github.com>
53531
53532 • Hekmon <edouardhur@gmail.com>
53533
53534 • LaSombra <lasombra@users.noreply.github.com>
53535
53536 • Dov Murik <dov.murik@gmail.com>
53537
53538 • Ameer Dawood <ameer1234567890@gmail.com>
53539
53540 • Dan Hipschman <dan.hipschman@opendoor.com>
53541
53542 • Josh Soref <jsoref@users.noreply.github.com>
53543
53544 • David <david@staron.nl>
53545
53546 • Ingo <ingo@hoffmann.cx>
53547
53548 • Adam Plánský <adamplansky@users.noreply.github.com> <adamplan‐
53549 sky@gmail.com>
53550
53551 • Manish Gupta <manishgupta.ait@gmail.com>
53552
53553 • Deepak Sah <sah.sslpu@gmail.com>
53554
53555 • Marcin Zelent <marcin@zelent.net>
53556
53557 • zhucan <zhucan.k8s@gmail.com>
53558
53559 • James Lim <james.lim@samsara.com>
53560
53561 • Laurens Janssen <BD69BM@insim.biz>
53562
53563 • Bob Bagwill <bobbagwill@gmail.com>
53564
53565 • Nathan Collins <colli372@msu.edu>
53566
53567 • lostheli
53568
53569 • kelv <kelvin@acks.org>
53570
53571 • Milly <milly.ca@gmail.com>
53572
53573 • gtorelly <gtorelly@gmail.com>
53574
53575 • Brad Ackerman <brad@facefault.org>
53576
53577 • Mitsuo Heijo <mitsuo.heijo@gmail.com>
53578
53579 • Claudio Bantaloukas <rockdreamer@gmail.com>
53580
53581 • Benjamin Gustin <gustin.ben@gmail.com>
53582
53583 • Ingo Weiss <ingo@redhat.com>
53584
53585 • Kerry Su <me@sshockwave.net>
53586
53587 • Ilyess Bachiri <ilyess.bachiri@sonder.com>
53588
53589 • Yury Stankevich <urykhy@gmail.com>
53590
53591 • kice <wslikerqs@gmail.com>
53592
53593 • Denis Neuling <denisneuling@gmail.com>
53594
53595 • Janne Johansson <icepic.dz@gmail.com>
53596
53597 • Patrik Nordlén <patriki@gmail.com>
53598
53599 • CokeMine <aptx4561@gmail.com>
53600
53601 • Sơn Trần-Nguyễn <github@sntran.com>
53602
53603 • lluuaapp <266615+lluuaapp@users.noreply.github.com>
53604
53605 • Zach Kipp <kipp.zach@gmail.com>
53606
53607 • Riccardo Iaconelli <riccardo@kde.org>
53608
53609 • Sakuragawa Misty <gyc990326@gmail.com>
53610
53611 • Nicolas Rueff <nicolas@rueff.fr>
53612
53613 • Pau Rodriguez-Estivill <prodrigestivill@gmail.com>
53614
53615 • Bob Pusateri <BobPusateri@users.noreply.github.com>
53616
53617 • Alex JOST <25005220+dimejo@users.noreply.github.com>
53618
53619 • Alexey Tabakman <samosad.ru@gmail.com>
53620
53621 • David Sze <sze.david@gmail.com>
53622
53623 • cynthia kwok <cynthia.m.kwok@gmail.com>
53624
53625 • Miron Veryanskiy <MironVeryanskiy@gmail.com>
53626
53627 • K265 <k.265@qq.com>
53628
53629 • Vesnyx <Vesnyx@users.noreply.github.com>
53630
53631 • Dmitry Chepurovskiy <me@dm3ch.net>
53632
53633 • Rauno Ots <rauno.ots@cgi.com>
53634
53635 • Georg Neugschwandtner <georg.neugschwandtner@gmx.net>
53636
53637 • pvalls <polvallsrue@gmail.com>
53638
53639 • Robert Thomas <31854736+wolveix@users.noreply.github.com>
53640
53641 • Romeo Kienzler <romeo.kienzler@gmail.com>
53642
53643 • tYYGH <tYYGH@users.noreply.github.com>
53644
53645 • georne <77802995+georne@users.noreply.github.com>
53646
53647 • Maxwell Calman <mcalman@MacBook-Pro.local>
53648
53649 • Naveen Honest Raj <naveendurai19@gmail.com>
53650
53651 • Lucas Messenger <lmesseng@cisco.com>
53652
53653 • Manish Kumar <krmanish260@gmail.com>
53654
53655 • x0b <x0bdev@gmail.com>
53656
53657 • CERN through the CS3MESH4EOSC Project
53658
53659 • Nick Gaya <nicholasgaya+github@gmail.com>
53660
53661 • Ashok Gelal <401055+ashokgelal@users.noreply.github.com>
53662
53663 • Dominik Mydlil <dominik.mydlil@outlook.com>
53664
53665 • Nazar Mishturak <nazarmx@gmail.com>
53666
53667 • Ansh Mittal <iamAnshMittal@gmail.com>
53668
53669 • noabody <noabody@yahoo.com>
53670
53671 • OleFrost <82263101+olefrost@users.noreply.github.com>
53672
53673 • Kenny Parsons <kennyparsons93@gmail.com>
53674
53675 • Jeffrey Tolar <tolar.jeffrey@gmail.com>
53676
53677 • jtagcat <git-514635f7@jtag.cat>
53678
53679 • Tatsuya Noyori <63089076+public-tatsuya-noyori@users.nore‐
53680 ply.github.com>
53681
53682 • lewisxy <lewisxy@users.noreply.github.com>
53683
53684 • Nolan Woods <nolan_w@sfu.ca>
53685
53686 • Gautam Kumar <25435568+gautamajay52@users.noreply.github.com>
53687
53688 • Chris Macklin <chris.macklin@10xgenomics.com>
53689
53690 • Antoon Prins <antoon.prins@surfsara.nl>
53691
53692 • Alexey Ivanov <rbtz@dropbox.com>
53693
53694 • Serge Pouliquen <sp31415@free.fr>
53695
53696 • acsfer <carlos@reendex.com>
53697
53698 • Tom <tom@tom-fitzhenry.me.uk>
53699
53700 • Tyson Moore <tyson@tyson.me>
53701
53702 • database64128 <free122448@hotmail.com>
53703
53704 • Chris Lu <chrislusf@users.noreply.github.com>
53705
53706 • Reid Buzby <reid@rethink.software>
53707
53708 • darrenrhs <darrenrhs@gmail.com>
53709
53710 • Florian Penzkofer <fp@nullptr.de>
53711
53712 • Xuanchen Wu <117010292@link.cuhk.edu.cn>
53713
53714 • partev <petrosyan@gmail.com>
53715
53716 • Dmitry Sitnikov <fo2@inbox.ru>
53717
53718 • Haochen Tong <i@hexchain.org>
53719
53720 • Michael Hanselmann <public@hansmi.ch>
53721
53722 • Chuan Zh <zhchuan7@gmail.com>
53723
53724 • Antoine GIRARD <antoine.girard@sapk.fr>
53725
53726 • Justin Winokur (Jwink3101) <Jwink3101@users.noreply.github.com>
53727
53728 • Mariano Absatz (git) <scm@baby.com.ar>
53729
53730 • Greg Sadetsky <lepetitg@gmail.com>
53731
53732 • yedamo <logindaveye@gmail.com>
53733
53734 • hota <lindwurm.q@gmail.com>
53735
53736 • vinibali <vinibali1@gmail.com>
53737
53738 • Ken Enrique Morel <ken.morel.santana@gmail.com>
53739
53740 • Justin Hellings <justin.hellings@gmail.com>
53741
53742 • Parth Shukla <pparth@pparth.net>
53743
53744 • wzl <wangzl31@outlook.com>
53745
53746 • HNGamingUK <connor@earnshawhome.co.uk>
53747
53748 • Jonta <359397+Jonta@users.noreply.github.com>
53749
53750 • YenForYang <YenForYang@users.noreply.github.com>
53751
53752 • Joda Stößer <stoesser@yay-digital.de> <services+github@simjo.st>
53753
53754 • Logeshwaran <waranlogesh@gmail.com>
53755
53756 • Rajat Goel <rajat@dropbox.com>
53757
53758 • r0kk3rz <r0kk3rz@gmail.com>
53759
53760 • Matthew Sevey <mjsevey@gmail.com>
53761
53762 • Filip Rysavy <fil@siasky.net>
53763
53764 • Ian Levesque <ian@ianlevesque.org>
53765
53766 • Thomas Stachl <thomas@stachl.me>
53767
53768 • Dmitry Bogatov <git#v1@kaction.cc>
53769
53770 • thomae <4493560+thomae@users.noreply.github.com>
53771
53772 • trevyn <trevyn-git@protonmail.com>
53773
53774 • David Liu <david.yx.liu@oracle.com>
53775
53776 • Chris Nelson <stuff@cjnaz.com>
53777
53778 • Felix Bünemann <felix.buenemann@gmail.com>
53779
53780 • Atílio Antônio <atiliodadalto@hotmail.com>
53781
53782 • Roberto Ricci <ricci@disroot.org>
53783
53784 • Carlo Mion <mion00@gmail.com>
53785
53786 • Chris Lu <chris.lu@gmail.com>
53787
53788 • Vitor Arruda <vitor.pimenta.arruda@gmail.com>
53789
53790 • bbabich <bbabich@datamossa.com>
53791
53792 • David <dp.davide.palma@gmail.com>
53793
53794 • Borna Butkovic <borna@favicode.net>
53795
53796 • Fredric Arklid <fredric.arklid@consid.se>
53797
53798 • Andy Jackson <Andrew.Jackson@bl.uk>
53799
53800 • Sinan Tan <i@tinytangent.com>
53801
53802 • deinferno <14363193+deinferno@users.noreply.github.com>
53803
53804 • rsapkf <rsapkfff@pm.me>
53805
53806 • Will Holtz <wholtz@gmail.com>
53807
53808 • GGG KILLER <gggkiller2@gmail.com>
53809
53810 • Logeshwaran Murugesan <logeshwaran@testpress.in>
53811
53812 • Lu Wang <coolwanglu@gmail.com>
53813
53814 • Bumsu Hyeon <ksitht@gmail.com>
53815
53816 • Shmz Ozggrn <98463324+ShmzOzggrn@users.noreply.github.com>
53817
53818 • Kim <kim@jotta.no>
53819
53820 • Niels van de Weem <n.van.de.weem@smile.nl>
53821
53822 • Koopa <codingkoopa@gmail.com>
53823
53824 • Yunhai Luo <yunhai-luo@hotmail.com>
53825
53826 • Charlie Jiang <w@chariri.moe>
53827
53828 • Alain Nussbaumer <alain.nussbaumer@alleluia.ch>
53829
53830 • Vanessasaurus <814322+vsoch@users.noreply.github.com>
53831
53832 • Isaac Levy <isaac.r.levy@gmail.com>
53833
53834 • Gourav T <workflowautomation@protonmail.com>
53835
53836 • Paulo Martins <paulo.pontes.m@gmail.com>
53837
53838 • viveknathani <viveknathani2402@gmail.com>
53839
53840 • Eng Zer Jun <engzerjun@gmail.com>
53841
53842 • Abhiraj <abhiraj.official15@gmail.com>
53843
53844 • Márton Elek <elek@apache.org> <elek@users.noreply.github.com>
53845
53846 • Vincent Murphy <vdm@vdm.ie>
53847
53848 • ctrl-q <34975747+ctrl-q@users.noreply.github.com>
53849
53850 • Nil Alexandrov <nalexand@akamai.com>
53851
53852 • GuoXingbin <101376330+guoxingbin@users.noreply.github.com>
53853
53854 • Berkan Teber <berkan@berkanteber.com>
53855
53856 • Tobias Klauser <tklauser@distanz.ch>
53857
53858 • KARBOWSKI Piotr <piotr.karbowski@gmail.com>
53859
53860 • GH <geeklihui@foxmail.com>
53861
53862 • rafma0 <int.main@gmail.com>
53863
53864 • Adrien Rey-Jarthon <jobs@adrienjarthon.com>
53865
53866 • Nick Gooding <73336146+nickgooding@users.noreply.github.com>
53867
53868 • Leroy van Logchem <lr.vanlogchem@gmail.com>
53869
53870 • Zsolt Ero <zsolt.ero@gmail.com>
53871
53872 • Lesmiscore <nao20010128@gmail.com>
53873
53874 • ehsantdy <ehsan.tadayon@arvancloud.com>
53875
53876 • SwazRGB <65694696+swazrgb@users.noreply.github.com>
53877
53878 • Mateusz Puczyński <mati6095@gmail.com>
53879
53880 • Michael C Tiernan - MIT-Research Computing Project <mtiernan@mit.edu>
53881
53882 • Kaspian <34658474+KaspianDev@users.noreply.github.com>
53883
53884 • Werner <EvilOlaf@users.noreply.github.com>
53885
53886 • Hugal31 <hugo.laloge@gmail.com>
53887
53888 • Christian Galo <36752715+cgalo5758@users.noreply.github.com>
53889
53890 • Erik van Velzen <erik@evanv.nl>
53891
53892 • Derek Battams <derek@battams.ca>
53893
53894 • SimonLiu <simonliu009@users.noreply.github.com>
53895
53896 • Hugo Laloge <hla@lescompanions.com>
53897
53898 • Mr-Kanister <68117355+Mr-Kanister@users.noreply.github.com>
53899
53900 • Rob Pickerill <r.pickerill@gmail.com>
53901
53902 • Andrey <to.merge@gmail.com>
53903
53904 • Eric Wolf <19wolf@gmail.com>
53905
53906 • Nick <nick.naumann@mailbox.tu-dresden.de>
53907
53908 • Jason Zheng <jszheng17@gmail.com>
53909
53910 • Matthew Vernon <mvernon@wikimedia.org>
53911
53912 • Noah Hsu <i@nn.ci>
53913
53914 • m00594701 <mengpengbo@huawei.com>
53915
53916 • Art M. Gallagher <artmg50@gmail.com>
53917
53918 • Sven Gerber <49589423+svengerber@users.noreply.github.com>
53919
53920 • CrossR <r.cross@lancaster.ac.uk>
53921
53922 • Maciej Radzikowski <maciej@radzikowski.com.pl>
53923
53924 • Scott Grimes <scott.grimes@spaciq.com>
53925
53926 • Phil Shackleton <71221528+philshacks@users.noreply.github.com>
53927
53928 • eNV25 <env252525@gmail.com>
53929
53930 • Caleb <inventor96@users.noreply.github.com>
53931
53932 • J-P Treen <jp@wraptious.com>
53933
53934 • Martin Czygan <53705+miku@users.noreply.github.com>
53935
53936 • buda <sandrojijavadze@protonmail.com>
53937
53938 • mirekphd <36706320+mirekphd@users.noreply.github.com>
53939
53940 • vyloy <vyloy@qq.com>
53941
53942 • Anthrazz <25553648+Anthrazz@users.noreply.github.com>
53943
53944 • zzr93 <34027824+zzr93@users.noreply.github.com>
53945
53946 • Paul Norman <penorman@mac.com>
53947
53948 • Lorenzo Maiorfi <maiorfi@gmail.com>
53949
53950 • Claudio Maradonna <penguyman@stronzi.org>
53951
53952 • Ovidiu Victor Tatar <ovi.tatar@googlemail.com>
53953
53954 • Evan Spensley <epspensley@gmail.com>
53955
53956 • Yen Hu <61753151+0x59656e@users.noreply.github.com>
53957
53958 • Steve Kowalik <steven@wedontsleep.org>
53959
53960 • Jordi Gonzalez Muñoz <jordigonzm@gmail.com>
53961
53962 • Joram Schrijver <i@joram.io>
53963
53964 • Mark Trolley <marktrolley@gmail.com>
53965
53966 • João Henrique Franco <joaohenrique.franco@gmail.com>
53967
53968 • anonion <aman207@users.noreply.github.com>
53969
53970 • Ryan Morey <4590343+rmorey@users.noreply.github.com>
53971
53972 • Simon Bos <simonbos9@gmail.com>
53973
53974 • YFdyh000 <yfdyh000@gmail.com> * Josh Soref
53975 <2119212+jsoref@users.noreply.github.com>
53976
53977 • Øyvind Heddeland Instefjord <instefjord@outlook.com>
53978
53979 • Dmitry Deniskin <110819396+ddeniskin@users.noreply.github.com>
53980
53981 • Alexander Knorr <106825+opexxx@users.noreply.github.com>
53982
53983 • Richard Bateman <richard@batemansr.us>
53984
53985 • Dimitri Papadopoulos Orfanos <3234522+DimitriPapadopoulos@users.nore‐
53986 ply.github.com>
53987
53988 • Lorenzo Milesi <lorenzo.milesi@yetopen.com>
53989
53990 • Isaac Aymerich <isaac.aymerich@gmail.com>
53991
53992 • YanceyChiew <35898533+YanceyChiew@users.noreply.github.com>
53993
53994 • Manoj Ghosh <msays2000@gmail.com>
53995
53996 • Bachue Zhou <bachue.shu@gmail.com>
53997
53998 • Manoj Ghosh <manoj.ghosh@oracle.com>
53999
54000 • Tom Mombourquette <tom@devnode.com>
54001
54002 • Robert Newson <rnewson@apache.org>
54003
54005 Forum
54006 Forum for questions and general discussion:
54007
54008 • https://forum.rclone.org
54009
54010 GitHub repository
54011 The project's repository is located at:
54012
54013 • https://github.com/rclone/rclone
54014
54015 There you can file bug reports or contribute with pull requests.
54016
54017 Twitter
54018 You can also follow me on twitter for rclone announcements:
54019
54020 • [@njcw](https://twitter.com/njcw)
54021
54022 Email
54023 Or if all else fails or you want to ask something private or confiden‐
54024 tial email Nick Craig-Wood (mailto:nick@craig-wood.com). Please don't
54025 email me requests for help - those are better directed to the forum.
54026 Thanks!
54027
54029 Nick Craig-Wood.
54030
54031
54032
54033User Manual Nov 17, 2022 rclone(1)