1rclone(1) rclone(1)
2
6 • About rclone
7 • What can rclone do for you?
9 • What features does rclone have?
11 • What providers does rclone support?
13 • Download (https://rclone.org/downloads/)
15 • Install (https://rclone.org/install/)
17 • Donate. (https://rclone.org/donate/)
19 About rclone
21 Rclone is a command-line program to manage files on cloud storage. It
22 is a feature-rich alternative to cloud vendors' web storage interfaces.
23 Over 40 cloud storage products support rclone including S3 object
24 stores, business & consumer file storage services, as well as standard
25 transfer protocols.
26 Rclone has powerful cloud equivalents to the unix commands rsync, cp,
28 mv, mount, ls, ncdu, tree, rm, and cat. Rclone's familiar syntax in‐
29 cludes shell pipeline support, and --dry-run protection. It is used at
30 the command line, in scripts or via its API.
31 Users call rclone "The Swiss army knife of cloud storage", and "Tech‐
33 nology indistinguishable from magic".
34 Rclone really looks after your data. It preserves timestamps and veri‐
36 fies checksums at all times. Transfers over limited bandwidth; inter‐
37 mittent connections, or subject to quota can be restarted, from the
38 last good file transferred. You can check (https://rclone.org/com‐
39 mands/rclone_check/) the integrity of your files. Where possible,
40 rclone employs server-side transfers to minimise local bandwidth use
41 and transfers from one provider to another without using local disk.
42 Virtual backends wrap local and cloud file systems to apply encryption
44 (https://rclone.org/crypt/), compression (https://rclone.org/com‐
45 press/), chunking (https://rclone.org/chunker/), hashing
46 (https://rclone.org/hasher/) and joining (https://rclone.org/union/).
47 Rclone mounts (https://rclone.org/commands/rclone_mount/) any local,
49 cloud or virtual filesystem as a disk on Windows, macOS, linux and
50 FreeBSD, and also serves these over SFTP (https://rclone.org/com‐
51 mands/rclone_serve_sftp/), HTTP (https://rclone.org/com‐
52 mands/rclone_serve_http/), WebDAV (https://rclone.org/com‐
53 mands/rclone_serve_webdav/), FTP (https://rclone.org/com‐
54 mands/rclone_serve_ftp/) and DLNA (https://rclone.org/com‐
55 mands/rclone_serve_dlna/).
56 Rclone is mature, open-source software originally inspired by rsync and
58 written in Go (https://golang.org). The friendly support community is
59 familiar with varied use cases. Official Ubuntu, Debian, Fedora, Brew
60 and Chocolatey repos. include rclone. For the latest version down‐
61 loading from rclone.org (https://rclone.org/downloads/) is recommended.
62 Rclone is widely used on Linux, Windows and Mac. Third-party develop‐
64 ers create innovative backup, restore, GUI and business process solu‐
65 tions using the rclone command line or API.
66 Rclone does the heavy lifting of communicating with cloud storage.
68 What can rclone do for you?
70 Rclone helps you:
71 • Backup (and encrypt) files to cloud storage
73 • Restore (and decrypt) files from cloud storage
75 • Mirror cloud data to other cloud services or locally
77 • Migrate data to the cloud, or between cloud storage vendors
79 • Mount multiple, encrypted, cached or diverse cloud storage as a disk
81 • Analyse and account for data held on cloud storage using lsf
83 (https://rclone.org/commands/rclone_lsf/), ljson
84 (https://rclone.org/commands/rclone_lsjson/), size
85 (https://rclone.org/commands/rclone_size/), ncdu
86 (https://rclone.org/commands/rclone_ncdu/)
87 • Union (https://rclone.org/union/) file systems together to present
89 multiple local and/or cloud file systems as one
90 Features
92 • Transfers
93 • MD5, SHA1 hashes are checked at all times for file integrity
95 • Timestamps are preserved on files
97 • Operations can be restarted at any time
99 • Can be to and from network, e.g. two different cloud providers
101 • Can use multi-threaded downloads to local disk
103 • Copy (https://rclone.org/commands/rclone_copy/) new or changed files
105 to cloud storage
106 • Sync (https://rclone.org/commands/rclone_sync/) (one way) to make a
108 directory identical
109 • Move (https://rclone.org/commands/rclone_move/) files to cloud stor‐
111 age deleting the local after verification
112 • Check (https://rclone.org/commands/rclone_check/) hashes and for
114 missing/extra files
115 • Mount (https://rclone.org/commands/rclone_mount/) your cloud storage
117 as a network disk
118 • Serve (https://rclone.org/commands/rclone_serve/) local or remote
120 files over HTTP (https://rclone.org/commands/rclone_serve_http/)/Web‐
121 Dav (https://rclone.org/commands/rclone_serve_webdav/)/FTP
122 (https://rclone.org/commands/rclone_serve_ftp/)/SFTP
123 (https://rclone.org/commands/rclone_serve_sftp/)/DLNA
124 (https://rclone.org/commands/rclone_serve_dlna/)
125 • Experimental Web based GUI (https://rclone.org/gui/)
127 Supported providers
129 (There are many others, built on standard protocols such as WebDAV or
130 S3, that work out of the box.)
131 • 1Fichier
133 • Akamai Netstorage
135 • Alibaba Cloud (Aliyun) Object Storage System (OSS)
137 • Amazon Drive
139 • Amazon S3
141 • Backblaze B2
143 • Box
145 • Ceph
147 • China Mobile Ecloud Elastic Object Storage (EOS)
149 • Arvan Cloud Object Storage (AOS)
151 • Citrix ShareFile
153 • Cloudflare R2
155 • DigitalOcean Spaces
157 • Digi Storage
159 • Dreamhost
161 • Dropbox
163 • Enterprise File Fabric
165 • FTP
167 • Google Cloud Storage
169 • Google Drive
171 • Google Photos
173 • HDFS
175 • Hetzner Storage Box
177 • HiDrive
179 • HTTP
181 • Internet Archive
183 • Jottacloud
185 • IBM COS S3
187 • IDrive e2
189 • IONOS Cloud
191 • Koofr
193 • Mail.ru Cloud
195 • Memset Memstore
197 • Mega
199 • Memory
201 • Microsoft Azure Blob Storage
203 • Microsoft OneDrive
205 • Minio
207 • Nextcloud
209 • OVH
211 • OpenDrive
213 • OpenStack Swift
215 • Oracle Cloud Storage Swift
217 • Oracle Object Storage
219 • ownCloud
221 • pCloud
223 • premiumize.me
225 • put.io
227 • QingStor
229 • Qiniu Cloud Object Storage (Kodo)
231 • Rackspace Cloud Files
233 • rsync.net
235 • Scaleway
237 • Seafile
239 • Seagate Lyve Cloud
241 • SeaweedFS
243 • SFTP
245 • Sia
247 • SMB / CIFS
249 • StackPath
251 • Storj
253 • SugarSync
255 • Tencent Cloud Object Storage (COS)
257 • Uptobox
259 • Wasabi
261 • WebDAV
263 • Yandex Disk
265 • Zoho WorkDrive
267 • The local filesystem
269 Virtual providers
271 These backends adapt or modify other storage providers:
272 • Alias: Rename existing remotes
274 • Cache: Cache remotes (DEPRECATED)
276 • Chunker: Split large files
278 • Combine: Combine multiple remotes into a directory tree
280 • Compress: Compress files
282 • Crypt: Encrypt files
284 • Hasher: Hash files
286 • Union: Join multiple remotes to work together
288 Links
290 • Home page (https://rclone.org/)
291 • GitHub project page for source and bug tracker
293 (https://github.com/rclone/rclone)
294 • Rclone Forum (https://forum.rclone.org)
296 • Downloads (https://rclone.org/downloads/)
298
300 Rclone is a Go program and comes as a single binary file.
301 Quickstart
303 • Download (https://rclone.org/downloads/) the relevant binary.
304 • Extract the rclone executable, rclone.exe on Windows, from the ar‐
306 chive.
307 • Run rclone config to setup. See rclone config docs
309 (https://rclone.org/docs/) for more details.
310 • Optionally configure automatic execution.
312 See below for some expanded Linux / macOS / Windows instructions.
314 See the usage (https://rclone.org/docs/) docs for how to use rclone, or
316 run rclone -h.
317 Already installed rclone can be easily updated to the latest version
319 using the rclone selfupdate (https://rclone.org/commands/rclone_selfup‐
320 date/) command.
321 Script installation
323 To install rclone on Linux/macOS/BSD systems, run:
324 sudo -v ; curl https://rclone.org/install.sh | sudo bash
326 For beta installation, run:
328 sudo -v ; curl https://rclone.org/install.sh | sudo bash -s beta
330 Note that this script checks the version of rclone installed first and
332 won't re-download if not needed.
333 Linux installation
335 Precompiled binary
336 Fetch and unpack
337 curl -O https://downloads.rclone.org/rclone-current-linux-amd64.zip
339 unzip rclone-current-linux-amd64.zip
340 cd rclone-*-linux-amd64
341 Copy binary file
343 sudo cp rclone /usr/bin/
345 sudo chown root:root /usr/bin/rclone
346 sudo chmod 755 /usr/bin/rclone
347 Install manpage
349 sudo mkdir -p /usr/local/share/man/man1
351 sudo cp rclone.1 /usr/local/share/man/man1/
352 sudo mandb
353 Run rclone config to setup. See rclone config docs
355 (https://rclone.org/docs/) for more details.
356 rclone config
358 macOS installation
360 Installation with brew
361 brew install rclone
362 NOTE: This version of rclone will not support mount any more (see #5373
364 (https://github.com/rclone/rclone/issues/5373)). If mounting is wanted
365 on macOS, either install a precompiled binary or enable the relevant
366 option when installing from source.
367 Note that this is a third party installer not controlled by the rclone
369 developers so it may be out of date. Its current version is as below.
370 [IMAGE: Homebrew package (https://repology.org/badge/version-for-re‐
372 po/homebrew/rclone.svg)] (https://repology.org/project/rclone/versions)
373 Precompiled binary, using curl
375 To avoid problems with macOS gatekeeper enforcing the binary to be
376 signed and notarized it is enough to download with curl.
377 Download the latest version of rclone.
379 cd && curl -O https://downloads.rclone.org/rclone-current-osx-amd64.zip
381 Unzip the download and cd to the extracted folder.
383 unzip -a rclone-current-osx-amd64.zip && cd rclone-*-osx-amd64
385 Move rclone to your $PATH. You will be prompted for your password.
387 sudo mkdir -p /usr/local/bin
389 sudo mv rclone /usr/local/bin/
390 (the mkdir command is safe to run, even if the directory already ex‐
392 ists).
393 Remove the leftover files.
395 cd .. && rm -rf rclone-*-osx-amd64 rclone-current-osx-amd64.zip
397 Run rclone config to setup. See rclone config docs
399 (https://rclone.org/docs/) for more details.
400 rclone config
402 Precompiled binary, using a web browser
404 When downloading a binary with a web browser, the browser will set the
405 macOS gatekeeper quarantine attribute. Starting from Catalina, when
406 attempting to run rclone, a pop-up will appear saying:
407 "rclone" cannot be opened because the developer cannot be verified.
409 macOS cannot verify that this app is free from malware.
410 The simplest fix is to run
412 xattr -d com.apple.quarantine rclone
414 Windows installation
416 Precompiled binary
417 Fetch the correct binary for your processor type by clicking on these
418 links. If not sure, use the first link.
419 • Intel/AMD - 64 Bit (https://downloads.rclone.org/rclone-current-win‐
421 dows-amd64.zip)
422 • Intel/AMD - 32 Bit (https://downloads.rclone.org/rclone-current-win‐
424 dows-386.zip)
425 • ARM - 64 Bit (https://downloads.rclone.org/rclone-current-windows-
427 arm64.zip)
428 Open this file in the Explorer and extract rclone.exe. Rclone is a
430 portable executable so you can place it wherever is convenient.
431 Open a CMD window (or powershell) and run the binary. Note that rclone
433 does not launch a GUI by default, it runs in the CMD Window.
434 • Run rclone.exe config to setup. See rclone config docs
436 (https://rclone.org/docs/) for more details.
437 • Optionally configure automatic execution.
439 If you are planning to use the rclone mount (https://rclone.org/com‐
441 mands/rclone_mount/) feature then you will need to install the third
442 party utility WinFsp (https://winfsp.dev/) also.
443 Chocolatey package manager
445 Make sure you have Choco (https://chocolatey.org/) installed
446 choco search rclone
448 choco install rclone
449 This will install rclone on your Windows machine. If you are planning
451 to use rclone mount (https://rclone.org/commands/rclone_mount/) then
452 choco install winfsp
454 will install that too.
456 Note that this is a third party installer not controlled by the rclone
458 developers so it may be out of date. Its current version is as below.
459 [IMAGE: Chocolatey package (https://repology.org/badge/version-for-re‐
461 po/chocolatey/rclone.svg)] (https://repology.org/project/rclone/ver‐
462 sions)
463 Package manager installation
465 Many Linux, Windows, macOS and other OS distributions package and dis‐
466 tribute rclone.
467 The distributed versions of rclone are often quite out of date and for
469 this reason we recommend one of the other installation methods if pos‐
470 sible.
471 You can get an idea of how up to date or not your OS distribution's
473 package is here.
474 [IMAGE: Packaging status (https://repology.org/badge/vertical-allre‐
476 pos/rclone.svg?columns=3)] (https://repology.org/project/rclone/ver‐
477 sions)
478 Docker installation
480 The rclone developers maintain a docker image for rclone
481 (https://hub.docker.com/r/rclone/rclone).
482 These images are built as part of the release process based on a mini‐
484 mal Alpine Linux.
485 The :latest tag will always point to the latest stable release. You
487 can use the :beta tag to get the latest build from master. You can al‐
488 so use version tags, e.g. :1.49.1, :1.49 or :1.
489 $ docker pull rclone/rclone:latest
491 latest: Pulling from rclone/rclone
492 Digest: sha256:0e0ced72671989bb837fea8e88578b3fc48371aa45d209663683e24cfdaa0e11
493 ...
494 $ docker run --rm rclone/rclone:latest version
495 rclone v1.49.1
496 - os/arch: linux/amd64
497 - go version: go1.12.9
498 There are a few command line options to consider when starting an
500 rclone Docker container from the rclone image.
501 • You need to mount the host rclone config dir at /config/rclone into
503 the Docker container. Due to the fact that rclone updates tokens in‐
504 side its config file, and that the update process involves a file re‐
505 name, you need to mount the whole host rclone config dir, not just
506 the single host rclone config file.
507 • You need to mount a host data dir at /data into the Docker container.
509 • By default, the rclone binary inside a Docker container runs with
511 UID=0 (root). As a result, all files created in a run will have
512 UID=0. If your config and data files reside on the host with a non-
513 root UID:GID, you need to pass these on the container start command
514 line.
515 • If you want to access the RC interface (either via the API or the Web
517 UI), it is required to set the --rc-addr to :5572 in order to connect
518 to it from outside the container. An explanation about why this is
519 necessary is present here (https://web.ar‐
520 chive.org/web/20200808071950/https://pythonspeed.com/articles/docker-
521 connection-refused/).
522 • NOTE: Users running this container with the docker network set to
524 host should probably set it to listen to localhost only, with
525 127.0.0.1:5572 as the value for --rc-addr
526 • It is possible to use rclone mount inside a userspace Docker contain‐
528 er, and expose the resulting fuse mount to the host. The exact dock‐
529 er run options to do that might vary slightly between hosts. See,
530 e.g. the discussion in this thread (https://github.com/moby/moby/is‐
531 sues/9448).
532 You also need to mount the host /etc/passwd and /etc/group for fuse
534 to work inside the container.
535 Here are some commands tested on an Ubuntu 18.04.3 host:
537 # config on host at ~/.config/rclone/rclone.conf
539 # data on host at ~/data
540 # make sure the config is ok by listing the remotes
542 docker run --rm \
543 --volume ~/.config/rclone:/config/rclone \
544 --volume ~/data:/data:shared \
545 --user $(id -u):$(id -g) \
546 rclone/rclone \
547 listremotes
548 # perform mount inside Docker container, expose result to host
550 mkdir -p ~/data/mount
551 docker run --rm \
552 --volume ~/.config/rclone:/config/rclone \
553 --volume ~/data:/data:shared \
554 --user $(id -u):$(id -g) \
555 --volume /etc/passwd:/etc/passwd:ro --volume /etc/group:/etc/group:ro \
556 --device /dev/fuse --cap-add SYS_ADMIN --security-opt apparmor:unconfined \
557 rclone/rclone \
558 mount dropbox:Photos /data/mount &
559 ls ~/data/mount
560 kill %1
561 Source installation
563 Make sure you have git and Go (https://golang.org/) installed. Go ver‐
564 sion 1.17 or newer is required, latest release is recommended. You can
565 get it from your package manager, or download it from golang.org/dl
566 (https://golang.org/dl/). Then you can run the following:
567 git clone https://github.com/rclone/rclone.git
569 cd rclone
570 go build
571 This will check out the rclone source in subfolder rclone, which you
573 can later modify and send pull requests with. Then it will build the
574 rclone executable in the same folder. As an initial check you can now
575 run ./rclone version (.\rclone version on Windows).
576 Note that on macOS and Windows the mount (https://rclone.org/com‐
578 mands/rclone_mount/) command will not be available unless you specify
579 an additional build tag cmount.
580 go build -tags cmount
582 This assumes you have a GCC compatible C compiler (GCC or Clang) in
584 your PATH, as it uses cgo (https://pkg.go.dev/cmd/cgo). But on Win‐
585 dows, the cgofuse (https://github.com/winfsp/cgofuse) library that the
586 cmount implementation is based on, also supports building without cgo
587 (https://github.com/golang/go/wiki/WindowsDLLs), i.e. by setting envi‐
588 ronment variable CGO_ENABLED to value 0 (static linking). This is how
589 the official Windows release of rclone is being built, starting with
590 version 1.59. It is still possible to build with cgo on Windows as
591 well, by using the MinGW port of GCC, e.g. by installing it in a MSYS2
592 (https://www.msys2.org) distribution (make sure you install it in the
593 classic mingw64 subsystem, the ucrt64 version is not compatible).
594 Additionally, on Windows, you must install the third party utility Win‐
596 Fsp (https://winfsp.dev/), with the "Developer" feature selected. If
597 building with cgo, you must also set environment variable CPATH point‐
598 ing to the fuse include directory within the WinFsp installation (nor‐
599 mally C:\Program Files (x86)\WinFsp\inc\fuse).
600 You may also add arguments -ldflags -s (with or without -tags cmount),
602 to omit symbol table and debug information, making the executable file
603 smaller, and -trimpath to remove references to local file system paths.
604 This is how the official rclone releases are built.
605 go build -trimpath -ldflags -s -tags cmount
607 Instead of executing the go build command directly, you can run it via
609 the Makefile. It changes the version number suffix from "-DEV" to
610 "-beta" and appends commit details. It also copies the resulting
611 rclone executable into your GOPATH bin folder ($(go env GOPATH)/bin,
612 which corresponds to ~/go/bin/rclone by default).
613 make
615 To include mount command on macOS and Windows with Makefile build:
617 make GOTAGS=cmount
619 There are other make targets that can be used for more advanced builds,
621 such as cross-compiling for all supported os/architectures, embedding
622 icon and version info resources into windows executable, and packaging
623 results into release artifacts. See Makefile
624 (https://github.com/rclone/rclone/blob/master/Makefile) and cross-com‐
625 pile.go (https://github.com/rclone/rclone/blob/master/bin/cross-com‐
626 pile.go) for details.
627 Another alternative is to download the source, build and install rclone
629 in one operation, as a regular Go package. The source will be stored
630 it in the Go module cache, and the resulting executable will be in your
631 GOPATH bin folder ($(go env GOPATH)/bin, which corresponds to
632 ~/go/bin/rclone by default).
633 With Go version 1.17 or newer:
635 go install github.com/rclone/rclone@latest
637 With Go versions older than 1.17 (do not use the -u flag, it causes Go
639 to try to update the dependencies that rclone uses and sometimes these
640 don't work with the current version):
641 go get github.com/rclone/rclone
643 Ansible installation
645 This can be done with Stefan Weichinger's ansible role
646 (https://github.com/stefangweichinger/ansible-rclone).
647 Instructions
649 1. git clone https://github.com/stefangweichinger/ansible-rclone.git
651 into your local roles-directory
652 2. add the role to the hosts you want rclone installed to:
654 - hosts: rclone-hosts
656 roles:
657 - rclone
658 Portable installation
660 As mentioned above (https://rclone.org/install/#quickstart), rclone is
661 single executable (rclone, or rclone.exe on Windows) that you can down‐
662 load as a zip archive and extract into a location of your choosing.
663 When executing different commands, it may create files in different lo‐
664 cations, such as a configuration file and various temporary files. By
665 default the locations for these are according to your operating system,
666 e.g. configuration file in your user profile directory and temporary
667 files in the standard temporary directory, but you can customize all of
668 them, e.g. to make a completely self-contained, portable installation.
669 Run the config paths (https://rclone.org/commands/rclone_config_paths/)
671 command to see the locations that rclone will use.
672 To override them set the corresponding options (as command-line argu‐
674 ments, or as environment variables (https://rclone.org/docs/#environ‐
675 ment-variables)): - --config (https://rclone.org/docs/#config-config-
676 file) - --cache-dir (https://rclone.org/docs/#cache-dir-dir) - --temp-
677 dir (https://rclone.org/docs/#temp-dir-dir)
678 Autostart
680 After installing and configuring rclone, as described above, you are
681 ready to use rclone as an interactive command line utility. If your
682 goal is to perform periodic operations, such as a regular sync
683 (https://rclone.org/commands/rclone_sync/), you will probably want to
684 configure your rclone command in your operating system's scheduler. If
685 you need to expose service-like features, such as remote control
686 (https://rclone.org/rc/), GUI (https://rclone.org/gui/), serve
687 (https://rclone.org/commands/rclone_serve/) or mount
688 (https://rclone.org/commands/rclone_mount/), you will often want an
689 rclone command always running in the background, and configuring it to
690 run in a service infrastructure may be a better option. Below are some
691 alternatives on how to achieve this on different operating systems.
692 NOTE: Before setting up autorun it is highly recommended that you have
694 tested your command manually from a Command Prompt first.
695 Autostart on Windows
697 The most relevant alternatives for autostart on Windows are: - Run at
698 user log on using the Startup folder - Run at user log on, at system
699 startup or at schedule using Task Scheduler - Run at system startup us‐
700 ing Windows service
701 Running in background
703 Rclone is a console application, so if not starting from an existing
704 Command Prompt, e.g. when starting rclone.exe from a shortcut, it will
705 open a Command Prompt window. When configuring rclone to run from task
706 scheduler and windows service you are able to set it to run hidden in
707 background. From rclone version 1.54 you can also make it run hidden
708 from anywhere by adding option --no-console (it may still flash briefly
709 when the program starts). Since rclone normally writes information and
710 any error messages to the console, you must redirect this to a file to
711 be able to see it. Rclone has a built-in option --log-file for that.
712 Example command to run a sync in background:
714 c:\rclone\rclone.exe sync c:\files remote:/files --no-console --log-file c:\rclone\logs\sync_files.txt
716 User account
718 As mentioned in the mount (https://rclone.org/commands/rclone_mount/)
719 documentation, mounted drives created as Administrator are not visible
720 to other accounts, not even the account that was elevated as Adminis‐
721 trator. By running the mount command as the built-in SYSTEM user ac‐
722 count, it will create drives accessible for everyone on the system.
723 Both scheduled task and Windows service can be used to achieve this.
724 NOTE: Remember that when rclone runs as the SYSTEM user, the user pro‐
726 file that it sees will not be yours. This means that if you normally
727 run rclone with configuration file in the default location, to be able
728 to use the same configuration when running as the system user you must
729 explicitly tell rclone where to find it with the --config
730 (https://rclone.org/docs/#config-config-file) option, or else it will
731 look in the system users profile path (C:\Windows\System32\config\sys‐
732 temprofile). To test your command manually from a Command Prompt, you
733 can run it with the PsExec (https://docs.microsoft.com/en-us/sysinter‐
734 nals/downloads/psexec) utility from Microsoft's Sysinternals suite,
735 which takes option -s to execute commands as the SYSTEM user.
736 Start from Startup folder
738 To quickly execute an rclone command you can simply create a standard
739 Windows Explorer shortcut for the complete rclone command you want to
740 run. If you store this shortcut in the special "Startup" start-menu
741 folder, Windows will automatically run it at login. To open this fold‐
742 er in Windows Explorer, enter path %APPDATA%\Microsoft\Windows\Start
743 Menu\Programs\Startup, or C:\ProgramData\Microsoft\Windows\Start
744 Menu\Programs\StartUp if you want the command to start for every user
745 that logs in.
746 This is the easiest approach to autostarting of rclone, but it offers
748 no functionality to set it to run as different user, or to set condi‐
749 tions or actions on certain events. Setting up a scheduled task as de‐
750 scribed below will often give you better results.
751 Start from Task Scheduler
753 Task Scheduler is an administrative tool built into Windows, and it can
754 be used to configure rclone to be started automatically in a highly
755 configurable way, e.g. periodically on a schedule, on user log on, or
756 at system startup. It can run be configured to run as the current us‐
757 er, or for a mount command that needs to be available to all users it
758 can run as the SYSTEM user. For technical information, see
759 https://docs.microsoft.com/windows/win32/taskschd/task-scheduler-start-
760 page.
761 Run as service
763 For running rclone at system startup, you can create a Windows service
764 that executes your rclone command, as an alternative to scheduled task
765 configured to run at startup.
766 Mount command built-in service integration
768 For mount commands, rclone has a built-in Windows service integration
769 via the third-party WinFsp library it uses. Registering as a regular
770 Windows service easy, as you just have to execute the built-in Power‐
771 Shell command New-Service (requires administrative privileges).
772 Example of a PowerShell command that creates a Windows service for
774 mounting some remote:/files as drive letter X:, for all users (service
775 will be running as the local system account):
776 New-Service -Name Rclone -BinaryPathName 'c:\rclone\rclone.exe mount remote:/files X: --config c:\rclone\config\rclone.conf --log-file c:\rclone\logs\mount.txt'
778 The WinFsp service infrastructure (https://github.com/billziss-gh/winf‐
780 sp/wiki/WinFsp-Service-Architecture) supports incorporating services
781 for file system implementations, such as rclone, into its own launcher
782 service, as kind of "child services". This has the additional advan‐
783 tage that it also implements a network provider that integrates into
784 Windows standard methods for managing network drives. This is current‐
785 ly not officially supported by Rclone, but with WinFsp version 2019.3
786 B2 / v1.5B2 or later it should be possible through path rewriting as
787 described here (https://github.com/rclone/rclone/issues/3340).
788 Third-party service integration
790 To Windows service running any rclone command, the excellent third-par‐
791 ty utility NSSM (http://nssm.cc), the "Non-Sucking Service Manager",
792 can be used. It includes some advanced features such as adjusting
793 process priority, defining process environment variables, redirect to
794 file anything written to stdout, and customized response to different
795 exit codes, with a GUI to configure everything from (although it can
796 also be used from command line ).
797 There are also several other alternatives. To mention one more, WinSW
799 (https://github.com/winsw/winsw), "Windows Service Wrapper", is worth
800 checking out. It requires .NET Framework, but it is preinstalled on
801 newer versions of Windows, and it also provides alternative standalone
802 distributions which includes necessary runtime (.NET 5). WinSW is a
803 command-line only utility, where you have to manually create an XML
804 file with service configuration. This may be a drawback for some, but
805 it can also be an advantage as it is easy to back up and re-use the
806 configuration settings, without having go through manual steps in a
807 GUI. One thing to note is that by default it does not restart the ser‐
808 vice on error, one have to explicit enable this in the configuration
809 file (via the "onfailure" parameter).
810 Autostart on Linux
812 Start as a service
813 To always run rclone in background, relevant for mount commands etc,
814 you can use systemd to set up rclone as a system or user service. Run‐
815 ning as a system service ensures that it is run at startup even if the
816 user it is running as has no active session. Running rclone as a user
817 service ensures that it only starts after the configured user has
818 logged into the system.
819 Run periodically from cron
821 To run a periodic command, such as a copy/sync, you can set up a cron
822 job.
823
825 Rclone is a command line program to manage files on cloud storage. Af‐
826 ter download (https://rclone.org/downloads/) and install, continue here
827 to learn how to use it: Initial configuration, what the basic syntax
828 looks like, describes the various subcommands, the various options, and
829 more.
830 Configure
832 First, you'll need to configure rclone. As the object storage systems
833 have quite complicated authentication these are kept in a config file.
834 (See the --config entry for how to find the config file and choose its
835 location.)
836 The easiest way to make the config is to run rclone with the config op‐
838 tion:
839 rclone config
841 See the following for detailed instructions for
843 • 1Fichier (https://rclone.org/fichier/)
845 • Akamai Netstorage (https://rclone.org/netstorage/)
847 • Alias (https://rclone.org/alias/)
849 • Amazon Drive (https://rclone.org/amazonclouddrive/)
851 • Amazon S3 (https://rclone.org/s3/)
853 • Backblaze B2 (https://rclone.org/b2/)
855 • Box (https://rclone.org/box/)
857 • Chunker (https://rclone.org/chunker/) - transparently splits large
859 files for other remotes
860 • Citrix ShareFile (https://rclone.org/sharefile/)
862 • Compress (https://rclone.org/compress/)
864 • Combine (https://rclone.org/combine/)
866 • Crypt (https://rclone.org/crypt/) - to encrypt other remotes
868 • DigitalOcean Spaces (https://rclone.org/s3/#digitalocean-spaces)
870 • Digi Storage (https://rclone.org/koofr/#digi-storage)
872 • Dropbox (https://rclone.org/dropbox/)
874 • Enterprise File Fabric (https://rclone.org/filefabric/)
876 • FTP (https://rclone.org/ftp/)
878 • Google Cloud Storage (https://rclone.org/googlecloudstorage/)
880 • Google Drive (https://rclone.org/drive/)
882 • Google Photos (https://rclone.org/googlephotos/)
884 • Hasher (https://rclone.org/hasher/) - to handle checksums for other
886 remotes
887 • HDFS (https://rclone.org/hdfs/)
889 • HiDrive (https://rclone.org/hidrive/)
891 • HTTP (https://rclone.org/http/)
893 • Internet Archive (https://rclone.org/internetarchive/)
895 • Jottacloud (https://rclone.org/jottacloud/)
897 • Koofr (https://rclone.org/koofr/)
899 • Mail.ru Cloud (https://rclone.org/mailru/)
901 • Mega (https://rclone.org/mega/)
903 • Memory (https://rclone.org/memory/)
905 • Microsoft Azure Blob Storage (https://rclone.org/azureblob/)
907 • Microsoft OneDrive (https://rclone.org/onedrive/)
909 • OpenStack Swift / Rackspace Cloudfiles / Memset Memstore
911 (https://rclone.org/swift/)
912 • OpenDrive (https://rclone.org/opendrive/)
914 • Oracle Object Storage (https://rclone.org/oracleobjectstorage/)
916 • Pcloud (https://rclone.org/pcloud/)
918 • premiumize.me (https://rclone.org/premiumizeme/)
920 • put.io (https://rclone.org/putio/)
922 • QingStor (https://rclone.org/qingstor/)
924 • Seafile (https://rclone.org/seafile/)
926 • SFTP (https://rclone.org/sftp/)
928 • Sia (https://rclone.org/sia/)
930 • SMB (https://rclone.org/smb/)
932 • Storj (https://rclone.org/storj/)
934 • SugarSync (https://rclone.org/sugarsync/)
936 • Union (https://rclone.org/union/)
938 • Uptobox (https://rclone.org/uptobox/)
940 • WebDAV (https://rclone.org/webdav/)
942 • Yandex Disk (https://rclone.org/yandex/)
944 • Zoho WorkDrive (https://rclone.org/zoho/)
946 • The local filesystem (https://rclone.org/local/)
948 Basic syntax
950 Rclone syncs a directory tree from one storage system to another.
951 Its syntax is like this
953 Syntax: [options] subcommand <parameters> <parameters...>
955 Source and destination paths are specified by the name you gave the
957 storage system in the config file then the sub path, e.g. "drive:my‐
958 folder" to look at "myfolder" in Google drive.
959 You can define as many storage paths as you like in the config file.
961 Please use the -i / --interactive flag while learning rclone to avoid
963 accidental data loss.
964 Subcommands
966 rclone uses a system of subcommands. For example
967 rclone ls remote:path # lists a remote
969 rclone copy /local/path remote:path # copies /local/path to the remote
970 rclone sync -i /local/path remote:path # syncs /local/path to the remote
971
973 Enter an interactive configuration session.
974 Synopsis
976 Enter an interactive configuration session where you can setup new re‐
977 motes and manage existing ones. You may also set or remove a password
978 to protect your configuration.
979 rclone config [flags]
981 Options
983 -h, --help help for config
984 See the global flags page (https://rclone.org/flags/) for global op‐
986 tions not listed here.
987 SEE ALSO
989 • rclone (https://rclone.org/commands/rclone/) - Show help for rclone
990 commands, flags and backends.
991 • rclone config create (https://rclone.org/commands/rclone_config_cre‐
993 ate/) - Create a new remote with name, type and options.
994 • rclone config delete (https://rclone.org/commands/rclone_con‐
996 fig_delete/) - Delete an existing remote.
997 • rclone config disconnect (https://rclone.org/commands/rclone_con‐
999 fig_disconnect/) - Disconnects user from remote
1000 • rclone config dump (https://rclone.org/commands/rclone_config_dump/)
1002 - Dump the config file as JSON.
1003 • rclone config file (https://rclone.org/commands/rclone_config_file/)
1005 - Show path of configuration file in use.
1006 • rclone config password (https://rclone.org/commands/rclone_con‐
1008 fig_password/) - Update password in an existing remote.
1009 • rclone config paths (https://rclone.org/commands/rclone_con‐
1011 fig_paths/) - Show paths used for configuration, cache, temp etc.
1012 • rclone config providers (https://rclone.org/commands/rclone_con‐
1014 fig_providers/) - List in JSON format all the providers and options.
1015 • rclone config reconnect (https://rclone.org/commands/rclone_con‐
1017 fig_reconnect/) - Re-authenticates user with remote.
1018 • rclone config show (https://rclone.org/commands/rclone_config_show/)
1020 - Print (decrypted) config file, or the config for a single remote.
1021 • rclone config touch (https://rclone.org/commands/rclone_con‐
1023 fig_touch/) - Ensure configuration file exists.
1024 • rclone config update (https://rclone.org/commands/rclone_config_up‐
1026 date/) - Update options in an existing remote.
1027 • rclone config userinfo (https://rclone.org/commands/rclone_con‐
1029 fig_userinfo/) - Prints info about logged in user of remote.
1030
1032 Copy files from source to dest, skipping identical files.
1033 Synopsis
1035 Copy the source to the destination. Does not transfer files that are
1036 identical on source and destination, testing by size and modification
1037 time or MD5SUM. Doesn't delete files from the destination. If you
1038 want to also delete files from destination, to make it match source,
1039 use the sync (https://rclone.org/commands/rclone_sync/) command in‐
1040 stead.
1041 Note that it is always the contents of the directory that is synced,
1043 not the directory itself. So when source:path is a directory, it's the
1044 contents of source:path that are copied, not the directory name and
1045 contents.
1046 To copy single files, use the copyto (https://rclone.org/com‐
1048 mands/rclone_copyto/) command instead.
1049 If dest:path doesn't exist, it is created and the source:path contents
1051 go there.
1052 For example
1054 rclone copy source:sourcepath dest:destpath
1056 Let's say there are two files in sourcepath
1058 sourcepath/one.txt
1060 sourcepath/two.txt
1061 This copies them to
1063 destpath/one.txt
1065 destpath/two.txt
1066 Not to
1068 destpath/sourcepath/one.txt
1070 destpath/sourcepath/two.txt
1071 If you are familiar with rsync, rclone always works as if you had writ‐
1073 ten a trailing / - meaning "copy the contents of this directory". This
1074 applies to all commands and whether you are talking about the source or
1075 destination.
1076 See the --no-traverse (https://rclone.org/docs/#no-traverse) option for
1078 controlling whether rclone lists the destination directory or not.
1079 Supplying this option when copying a small number of files into a large
1080 destination can speed transfers up greatly.
1081 For example, if you have many files in /path/to/src but only a few of
1083 them change every day, you can copy all the files which have changed
1084 recently very efficiently like this:
1085 rclone copy --max-age 24h --no-traverse /path/to/src remote:
1087 Note: Use the -P/--progress flag to view real-time transfer statistics.
1089 Note: Use the --dry-run or the --interactive/-i flag to test without
1091 copying anything.
1092 rclone copy source:path dest:path [flags]
1094 Options
1096 --create-empty-src-dirs Create empty source dirs on destination after copy
1097 -h, --help help for copy
1098 See the global flags page (https://rclone.org/flags/) for global op‐
1100 tions not listed here.
1101 SEE ALSO
1103 • rclone (https://rclone.org/commands/rclone/) - Show help for rclone
1104 commands, flags and backends.
1105
1107 Make source and dest identical, modifying destination only.
1108 Synopsis
1110 Sync the source to the destination, changing the destination only.
1111 Doesn't transfer files that are identical on source and destination,
1112 testing by size and modification time or MD5SUM. Destination is updat‐
1113 ed to match source, including deleting files if necessary (except du‐
1114 plicate objects, see below). If you don't want to delete files from
1115 destination, use the copy (https://rclone.org/commands/rclone_copy/)
1116 command instead.
1117 Important: Since this can cause data loss, test first with the --dry-
1119 run or the --interactive/-i flag.
1120 rclone sync -i SOURCE remote:DESTINATION
1122 Note that files in the destination won't be deleted if there were any
1124 errors at any point. Duplicate objects (files with the same name, on
1125 those providers that support it) are also not yet handled.
1126 It is always the contents of the directory that is synced, not the di‐
1128 rectory itself. So when source:path is a directory, it's the contents
1129 of source:path that are copied, not the directory name and contents.
1130 See extended explanation in the copy (https://rclone.org/com‐
1131 mands/rclone_copy/) command if unsure.
1132 If dest:path doesn't exist, it is created and the source:path contents
1134 go there.
1135 It is not possible to sync overlapping remotes. However, you may ex‐
1137 clude the destination from the sync with a filter rule or by putting an
1138 exclude-if-present file inside the destination directory and sync to a
1139 destination that is inside the source directory.
1140 Note: Use the -P/--progress flag to view real-time transfer statistics
1142 Note: Use the rclone dedupe command to deal with "Duplicate object/di‐
1144 rectory found in source/destination - ignoring" errors. See this forum
1145 post (https://forum.rclone.org/t/sync-not-clearing-duplicates/14372)
1146 for more info.
1147 rclone sync source:path dest:path [flags]
1149 Options
1151 --create-empty-src-dirs Create empty source dirs on destination after sync
1152 -h, --help help for sync
1153 See the global flags page (https://rclone.org/flags/) for global op‐
1155 tions not listed here.
1156 SEE ALSO
1158 • rclone (https://rclone.org/commands/rclone/) - Show help for rclone
1159 commands, flags and backends.
1160
1162 Move files from source to dest.
1163 Synopsis
1165 Moves the contents of the source directory to the destination directo‐
1166 ry. Rclone will error if the source and destination overlap and the
1167 remote does not support a server-side directory move operation.
1168 To move single files, use the moveto (https://rclone.org/com‐
1170 mands/rclone_moveto/) command instead.
1171 If no filters are in use and if possible this will server-side move
1173 source:path into dest:path. After this source:path will no longer ex‐
1174 ist.
1175 Otherwise for each file in source:path selected by the filters (if any)
1177 this will move it into dest:path. If possible a server-side move will
1178 be used, otherwise it will copy it (server-side if possible) into
1179 dest:path then delete the original (if no errors on copy) in
1180 source:path.
1181 If you want to delete empty source directories after move, use the
1183 --delete-empty-src-dirs flag.
1184 See the --no-traverse (https://rclone.org/docs/#no-traverse) option for
1186 controlling whether rclone lists the destination directory or not.
1187 Supplying this option when moving a small number of files into a large
1188 destination can speed transfers up greatly.
1189 Important: Since this can cause data loss, test first with the --dry-
1191 run or the --interactive/-i flag.
1192 Note: Use the -P/--progress flag to view real-time transfer statistics.
1194 rclone move source:path dest:path [flags]
1196 Options
1198 --create-empty-src-dirs Create empty source dirs on destination after move
1199 --delete-empty-src-dirs Delete empty source dirs after move
1200 -h, --help help for move
1201 See the global flags page (https://rclone.org/flags/) for global op‐
1203 tions not listed here.
1204 SEE ALSO
1206 • rclone (https://rclone.org/commands/rclone/) - Show help for rclone
1207 commands, flags and backends.
1208
1210 Remove the files in path.
1211 Synopsis
1213 Remove the files in path. Unlike purge (https://rclone.org/com‐
1214 mands/rclone_purge/) it obeys include/exclude filters so can be used to
1215 selectively delete files.
1216 rclone delete only deletes files but leaves the directory structure
1218 alone. If you want to delete a directory and all of its contents use
1219 the purge (https://rclone.org/commands/rclone_purge/) command.
1220 If you supply the --rmdirs flag, it will remove all empty directories
1222 along with it. You can also use the separate command rmdir
1223 (https://rclone.org/commands/rclone_rmdir/) or rmdirs
1224 (https://rclone.org/commands/rclone_rmdirs/) to delete empty directo‐
1225 ries only.
1226 For example, to delete all files bigger than 100 MiB, you may first
1228 want to check what would be deleted (use either):
1229 rclone --min-size 100M lsl remote:path
1231 rclone --dry-run --min-size 100M delete remote:path
1232 Then proceed with the actual delete:
1234 rclone --min-size 100M delete remote:path
1236 That reads "delete everything with a minimum size of 100 MiB", hence
1238 delete all files bigger than 100 MiB.
1239 Important: Since this can cause data loss, test first with the --dry-
1241 run or the --interactive/-i flag.
1242 rclone delete remote:path [flags]
1244 Options
1246 -h, --help help for delete
1247 --rmdirs rmdirs removes empty directories but leaves root intact
1248 See the global flags page (https://rclone.org/flags/) for global op‐
1250 tions not listed here.
1251 SEE ALSO
1253 • rclone (https://rclone.org/commands/rclone/) - Show help for rclone
1254 commands, flags and backends.
1255
1257 Remove the path and all of its contents.
1258 Synopsis
1260 Remove the path and all of its contents. Note that this does not obey
1261 include/exclude filters - everything will be removed. Use the delete
1262 (https://rclone.org/commands/rclone_delete/) command if you want to se‐
1263 lectively delete files. To delete empty directories only, use command
1264 rmdir (https://rclone.org/commands/rclone_rmdir/) or rmdirs
1265 (https://rclone.org/commands/rclone_rmdirs/).
1266 Important: Since this can cause data loss, test first with the --dry-
1268 run or the --interactive/-i flag.
1269 rclone purge remote:path [flags]
1271 Options
1273 -h, --help help for purge
1274 See the global flags page (https://rclone.org/flags/) for global op‐
1276 tions not listed here.
1277 SEE ALSO
1279 • rclone (https://rclone.org/commands/rclone/) - Show help for rclone
1280 commands, flags and backends.
1281
1283 Make the path if it doesn't already exist.
1284 rclone mkdir remote:path [flags]
1286 Options
1288 -h, --help help for mkdir
1289 See the global flags page (https://rclone.org/flags/) for global op‐
1291 tions not listed here.
1292 SEE ALSO
1294 • rclone (https://rclone.org/commands/rclone/) - Show help for rclone
1295 commands, flags and backends.
1296
1298 Remove the empty directory at path.
1299 Synopsis
1301 This removes empty directory given by path. Will not remove the path
1302 if it has any objects in it, not even empty subdirectories. Use com‐
1303 mand rmdirs (https://rclone.org/commands/rclone_rmdirs/) (or delete
1304 (https://rclone.org/commands/rclone_delete/) with option --rmdirs) to
1305 do that.
1306 To delete a path and any objects in it, use purge
1308 (https://rclone.org/commands/rclone_purge/) command.
1309 rclone rmdir remote:path [flags]
1311 Options
1313 -h, --help help for rmdir
1314 See the global flags page (https://rclone.org/flags/) for global op‐
1316 tions not listed here.
1317 SEE ALSO
1319 • rclone (https://rclone.org/commands/rclone/) - Show help for rclone
1320 commands, flags and backends.
1321
1323 Checks the files in the source and destination match.
1324 Synopsis
1326 Checks the files in the source and destination match. It compares
1327 sizes and hashes (MD5 or SHA1) and logs a report of files that don't
1328 match. It doesn't alter the source or destination.
1329 For the crypt (https://rclone.org/crypt/) remote there is a dedicated
1331 command, cryptcheck (https://rclone.org/commands/rclone_cryptcheck/),
1332 that are able to check the checksums of the crypted files.
1333 If you supply the --size-only flag, it will only compare the sizes not
1335 the hashes as well. Use this for a quick check.
1336 If you supply the --download flag, it will download the data from both
1338 remotes and check them against each other on the fly. This can be use‐
1339 ful for remotes that don't support hashes or if you really want to
1340 check all the data.
1341 If you supply the --checkfile HASH flag with a valid hash name, the
1343 source:path must point to a text file in the SUM format.
1344 If you supply the --one-way flag, it will only check that files in the
1346 source match the files in the destination, not the other way around.
1347 This means that extra files in the destination that are not in the
1348 source will not be detected.
1349 The --differ, --missing-on-dst, --missing-on-src, --match and --error
1351 flags write paths, one per line, to the file name (or stdout if it is
1352 -) supplied. What they write is described in the help below. For ex‐
1353 ample --differ will write all paths which are present on both the
1354 source and destination but different.
1355 The --combined flag will write a file (or stdout) which contains all
1357 file paths with a symbol and then a space and then the path to tell you
1358 what happened to it. These are reminiscent of diff files.
1359 • = path means path was found in source and destination and was identi‐
1361 cal
1362 • `- path` means path was missing on the source, so only in the desti‐
1364 nation
1365 • `+ path` means path was missing on the destination, so only in the
1367 source
1368 • `* path` means path was present in source and destination but differ‐
1370 ent.
1371 • ! path means there was an error reading or hashing the source or
1373 dest.
1374 rclone check source:path dest:path [flags]
1376 Options
1378 -C, --checkfile string Treat source:path as a SUM file with hashes of given type
1379 --combined string Make a combined report of changes to this file
1380 --differ string Report all non-matching files to this file
1381 --download Check by downloading rather than with hash
1382 --error string Report all files with errors (hashing or reading) to this file
1383 -h, --help help for check
1384 --match string Report all matching files to this file
1385 --missing-on-dst string Report all files missing from the destination to this file
1386 --missing-on-src string Report all files missing from the source to this file
1387 --one-way Check one way only, source files must exist on remote
1388 See the global flags page (https://rclone.org/flags/) for global op‐
1390 tions not listed here.
1391 SEE ALSO
1393 • rclone (https://rclone.org/commands/rclone/) - Show help for rclone
1394 commands, flags and backends.
1395
1397 List the objects in the path with size and path.
1398 Synopsis
1400 Lists the objects in the source path to standard output in a human
1401 readable format with size and path. Recurses by default.
1402 Eg
1404 $ rclone ls swift:bucket
1406 60295 bevajer5jef
1407 90613 canole
1408 94467 diwogej7
1409 37600 fubuwic
1410 Any of the filtering options can be applied to this command.
1412 There are several related list commands
1414 • ls to list size and path of objects only
1416 • lsl to list modification time, size and path of objects only
1418 • lsd to list directories only
1420 • lsf to list objects and directories in easy to parse format
1422 • lsjson to list objects and directories in JSON format
1424 ls,lsl,lsd are designed to be human-readable. lsf is designed to be
1426 human and machine-readable. lsjson is designed to be machine-readable.
1427 Note that ls and lsl recurse by default - use --max-depth 1 to stop the
1429 recursion.
1430 The other list commands lsd,lsf,lsjson do not recurse by default - use
1432 -R to make them recurse.
1433 Listing a nonexistent directory will produce an error except for re‐
1435 motes which can't have empty directories (e.g. s3, swift, or gcs - the
1436 bucket-based remotes).
1437 rclone ls remote:path [flags]
1439 Options
1441 -h, --help help for ls
1442 See the global flags page (https://rclone.org/flags/) for global op‐
1444 tions not listed here.
1445 SEE ALSO
1447 • rclone (https://rclone.org/commands/rclone/) - Show help for rclone
1448 commands, flags and backends.
1449
1451 List all directories/containers/buckets in the path.
1452 Synopsis
1454 Lists the directories in the source path to standard output. Does not
1455 recurse by default. Use the -R flag to recurse.
1456 This command lists the total size of the directory (if known, -1 if
1458 not), the modification time (if known, the current time if not), the
1459 number of objects in the directory (if known, -1 if not) and the name
1460 of the directory, Eg
1461 $ rclone lsd swift:
1463 494000 2018-04-26 08:43:20 10000 10000files
1464 65 2018-04-26 08:43:20 1 1File
1465 Or
1467 $ rclone lsd drive:test
1469 -1 2016-10-17 17:41:53 -1 1000files
1470 -1 2017-01-03 14:40:54 -1 2500files
1471 -1 2017-07-08 14:39:28 -1 4000files
1472 If you just want the directory names use rclone lsf --dirs-only.
1474 Any of the filtering options can be applied to this command.
1476 There are several related list commands
1478 • ls to list size and path of objects only
1480 • lsl to list modification time, size and path of objects only
1482 • lsd to list directories only
1484 • lsf to list objects and directories in easy to parse format
1486 • lsjson to list objects and directories in JSON format
1488 ls,lsl,lsd are designed to be human-readable. lsf is designed to be
1490 human and machine-readable. lsjson is designed to be machine-readable.
1491 Note that ls and lsl recurse by default - use --max-depth 1 to stop the
1493 recursion.
1494 The other list commands lsd,lsf,lsjson do not recurse by default - use
1496 -R to make them recurse.
1497 Listing a nonexistent directory will produce an error except for re‐
1499 motes which can't have empty directories (e.g. s3, swift, or gcs - the
1500 bucket-based remotes).
1501 rclone lsd remote:path [flags]
1503 Options
1505 -h, --help help for lsd
1506 -R, --recursive Recurse into the listing
1507 See the global flags page (https://rclone.org/flags/) for global op‐
1509 tions not listed here.
1510 SEE ALSO
1512 • rclone (https://rclone.org/commands/rclone/) - Show help for rclone
1513 commands, flags and backends.
1514
1516 List the objects in path with modification time, size and path.
1517 Synopsis
1519 Lists the objects in the source path to standard output in a human
1520 readable format with modification time, size and path. Recurses by de‐
1521 fault.
1522 Eg
1524 $ rclone lsl swift:bucket
1526 60295 2016-06-25 18:55:41.062626927 bevajer5jef
1527 90613 2016-06-25 18:55:43.302607074 canole
1528 94467 2016-06-25 18:55:43.046609333 diwogej7
1529 37600 2016-06-25 18:55:40.814629136 fubuwic
1530 Any of the filtering options can be applied to this command.
1532 There are several related list commands
1534 • ls to list size and path of objects only
1536 • lsl to list modification time, size and path of objects only
1538 • lsd to list directories only
1540 • lsf to list objects and directories in easy to parse format
1542 • lsjson to list objects and directories in JSON format
1544 ls,lsl,lsd are designed to be human-readable. lsf is designed to be
1546 human and machine-readable. lsjson is designed to be machine-readable.
1547 Note that ls and lsl recurse by default - use --max-depth 1 to stop the
1549 recursion.
1550 The other list commands lsd,lsf,lsjson do not recurse by default - use
1552 -R to make them recurse.
1553 Listing a nonexistent directory will produce an error except for re‐
1555 motes which can't have empty directories (e.g. s3, swift, or gcs - the
1556 bucket-based remotes).
1557 rclone lsl remote:path [flags]
1559 Options
1561 -h, --help help for lsl
1562 See the global flags page (https://rclone.org/flags/) for global op‐
1564 tions not listed here.
1565 SEE ALSO
1567 • rclone (https://rclone.org/commands/rclone/) - Show help for rclone
1568 commands, flags and backends.
1569
1571 Produces an md5sum file for all the objects in the path.
1572 Synopsis
1574 Produces an md5sum file for all the objects in the path. This is in
1575 the same format as the standard md5sum tool produces.
1576 By default, the hash is requested from the remote. If MD5 is not sup‐
1578 ported by the remote, no hash will be returned. With the download
1579 flag, the file will be downloaded from the remote and hashed locally
1580 enabling MD5 for any remote.
1581 For other algorithms, see the hashsum (https://rclone.org/com‐
1583 mands/rclone_hashsum/) command. Running rclone md5sum remote:path is
1584 equivalent to running rclone hashsum MD5 remote:path.
1585 This command can also hash data received on standard input (stdin), by
1587 not passing a remote:path, or by passing a hyphen as remote:path when
1588 there is data to read (if not, the hyphen will be treated literally, as
1589 a relative path).
1590 rclone md5sum remote:path [flags]
1592 Options
1594 --base64 Output base64 encoded hashsum
1595 -C, --checkfile string Validate hashes against a given SUM file instead of printing them
1596 --download Download the file and hash it locally; if this flag is not specified, the hash is requested from the remote
1597 -h, --help help for md5sum
1598 --output-file string Output hashsums to a file rather than the terminal
1599 See the global flags page (https://rclone.org/flags/) for global op‐
1601 tions not listed here.
1602 SEE ALSO
1604 • rclone (https://rclone.org/commands/rclone/) - Show help for rclone
1605 commands, flags and backends.
1606
1608 Produces an sha1sum file for all the objects in the path.
1609 Synopsis
1611 Produces an sha1sum file for all the objects in the path. This is in
1612 the same format as the standard sha1sum tool produces.
1613 By default, the hash is requested from the remote. If SHA-1 is not
1615 supported by the remote, no hash will be returned. With the download
1616 flag, the file will be downloaded from the remote and hashed locally
1617 enabling SHA-1 for any remote.
1618 For other algorithms, see the hashsum (https://rclone.org/com‐
1620 mands/rclone_hashsum/) command. Running rclone sha1sum remote:path is
1621 equivalent to running rclone hashsum SHA1 remote:path.
1622 This command can also hash data received on standard input (stdin), by
1624 not passing a remote:path, or by passing a hyphen as remote:path when
1625 there is data to read (if not, the hyphen will be treated literally, as
1626 a relative path).
1627 This command can also hash data received on STDIN, if not passing a re‐
1629 mote:path.
1630 rclone sha1sum remote:path [flags]
1632 Options
1634 --base64 Output base64 encoded hashsum
1635 -C, --checkfile string Validate hashes against a given SUM file instead of printing them
1636 --download Download the file and hash it locally; if this flag is not specified, the hash is requested from the remote
1637 -h, --help help for sha1sum
1638 --output-file string Output hashsums to a file rather than the terminal
1639 See the global flags page (https://rclone.org/flags/) for global op‐
1641 tions not listed here.
1642 SEE ALSO
1644 • rclone (https://rclone.org/commands/rclone/) - Show help for rclone
1645 commands, flags and backends.
1646
1648 Prints the total size and number of objects in remote:path.
1649 Synopsis
1651 Counts objects in the path and calculates the total size. Prints the
1652 result to standard output.
1653 By default the output is in human-readable format, but shows values in
1655 both human-readable format as well as the raw numbers (global option
1656 --human-readable is not considered). Use option --json to format out‐
1657 put as JSON instead.
1658 Recurses by default, use --max-depth 1 to stop the recursion.
1660 Some backends do not always provide file sizes, see for example Google
1662 Photos (https://rclone.org/googlephotos/#size) and Google Drive
1663 (https://rclone.org/drive/#limitations-of-google-docs). Rclone will
1664 then show a notice in the log indicating how many such files were en‐
1665 countered, and count them in as empty files in the output of the size
1666 command.
1667 rclone size remote:path [flags]
1669 Options
1671 -h, --help help for size
1672 --json Format output as JSON
1673 See the global flags page (https://rclone.org/flags/) for global op‐
1675 tions not listed here.
1676 SEE ALSO
1678 • rclone (https://rclone.org/commands/rclone/) - Show help for rclone
1679 commands, flags and backends.
1680
1682 Show the version number.
1683 Synopsis
1685 Show the rclone version number, the go version, the build target OS and
1686 architecture, the runtime OS and kernel version and bitness, build tags
1687 and the type of executable (static or dynamic).
1688 For example:
1690 $ rclone version
1692 rclone v1.55.0
1693 - os/version: ubuntu 18.04 (64 bit)
1694 - os/kernel: 4.15.0-136-generic (x86_64)
1695 - os/type: linux
1696 - os/arch: amd64
1697 - go/version: go1.16
1698 - go/linking: static
1699 - go/tags: none
1700 Note: before rclone version 1.55 the os/type and os/arch lines were
1702 merged, and the "go/version" line was tagged as "go version".
1703 If you supply the --check flag, then it will do an online check to com‐
1705 pare your version with the latest release and the latest beta.
1706 $ rclone version --check
1708 yours: 1.42.0.6
1709 latest: 1.42 (released 2018-06-16)
1710 beta: 1.42.0.5 (released 2018-06-17)
1711 Or
1713 $ rclone version --check
1715 yours: 1.41
1716 latest: 1.42 (released 2018-06-16)
1717 upgrade: https://downloads.rclone.org/v1.42
1718 beta: 1.42.0.5 (released 2018-06-17)
1719 upgrade: https://beta.rclone.org/v1.42-005-g56e1e820
1720 rclone version [flags]
1722 Options
1724 --check Check for new version
1725 -h, --help help for version
1726 See the global flags page (https://rclone.org/flags/) for global op‐
1728 tions not listed here.
1729 SEE ALSO
1731 • rclone (https://rclone.org/commands/rclone/) - Show help for rclone
1732 commands, flags and backends.
1733
1735 Clean up the remote if possible.
1736 Synopsis
1738 Clean up the remote if possible. Empty the trash or delete old file
1739 versions. Not supported by all remotes.
1740 rclone cleanup remote:path [flags]
1742 Options
1744 -h, --help help for cleanup
1745 See the global flags page (https://rclone.org/flags/) for global op‐
1747 tions not listed here.
1748 SEE ALSO
1750 • rclone (https://rclone.org/commands/rclone/) - Show help for rclone
1751 commands, flags and backends.
1752
1754 Interactively find duplicate filenames and delete/rename them.
1755 Synopsis
1757 By default dedupe interactively finds files with duplicate names and
1758 offers to delete all but one or rename them to be different. This is
1759 known as deduping by name.
1760 Deduping by name is only useful with a small group of backends (e.g.
1762 Google Drive, Opendrive) that can have duplicate file names. It can be
1763 run on wrapping backends (e.g. crypt) if they wrap a backend which
1764 supports duplicate file names.
1765 However if --by-hash is passed in then dedupe will find files with du‐
1767 plicate hashes instead which will work on any backend which supports at
1768 least one hash. This can be used to find files with duplicate content.
1769 This is known as deduping by hash.
1770 If deduping by name, first rclone will merge directories with the same
1772 name. It will do this iteratively until all the identically named di‐
1773 rectories have been merged.
1774 Next, if deduping by name, for every group of duplicate file names /
1776 hashes, it will delete all but one identical file it finds without con‐
1777 firmation. This means that for most duplicated files the dedupe com‐
1778 mand will not be interactive.
1779 dedupe considers files to be identical if they have the same file path
1781 and the same hash. If the backend does not support hashes (e.g. crypt
1782 wrapping Google Drive) then they will never be found to be identical.
1783 If you use the --size-only flag then files will be considered identical
1784 if they have the same size (any hash will be ignored). This can be
1785 useful on crypt backends which do not support hashes.
1786 Next rclone will resolve the remaining duplicates. Exactly which ac‐
1788 tion is taken depends on the dedupe mode. By default, rclone will in‐
1789 teractively query the user for each one.
1790 Important: Since this can cause data loss, test first with the --dry-
1792 run or the --interactive/-i flag.
1793 Here is an example run.
1795 Before - with duplicates
1797 $ rclone lsl drive:dupes
1799 6048320 2016-03-05 16:23:16.798000000 one.txt
1800 6048320 2016-03-05 16:23:11.775000000 one.txt
1801 564374 2016-03-05 16:23:06.731000000 one.txt
1802 6048320 2016-03-05 16:18:26.092000000 one.txt
1803 6048320 2016-03-05 16:22:46.185000000 two.txt
1804 1744073 2016-03-05 16:22:38.104000000 two.txt
1805 564374 2016-03-05 16:22:52.118000000 two.txt
1806 Now the dedupe session
1808 $ rclone dedupe drive:dupes
1810 2016/03/05 16:24:37 Google drive root 'dupes': Looking for duplicates using interactive mode.
1811 one.txt: Found 4 files with duplicate names
1812 one.txt: Deleting 2/3 identical duplicates (MD5 "1eedaa9fe86fd4b8632e2ac549403b36")
1813 one.txt: 2 duplicates remain
1814 1: 6048320 bytes, 2016-03-05 16:23:16.798000000, MD5 1eedaa9fe86fd4b8632e2ac549403b36
1815 2: 564374 bytes, 2016-03-05 16:23:06.731000000, MD5 7594e7dc9fc28f727c42ee3e0749de81
1816 s) Skip and do nothing
1817 k) Keep just one (choose which in next step)
1818 r) Rename all to be different (by changing file.jpg to file-1.jpg)
1819 s/k/r> k
1820 Enter the number of the file to keep> 1
1821 one.txt: Deleted 1 extra copies
1822 two.txt: Found 3 files with duplicate names
1823 two.txt: 3 duplicates remain
1824 1: 564374 bytes, 2016-03-05 16:22:52.118000000, MD5 7594e7dc9fc28f727c42ee3e0749de81
1825 2: 6048320 bytes, 2016-03-05 16:22:46.185000000, MD5 1eedaa9fe86fd4b8632e2ac549403b36
1826 3: 1744073 bytes, 2016-03-05 16:22:38.104000000, MD5 851957f7fb6f0bc4ce76be966d336802
1827 s) Skip and do nothing
1828 k) Keep just one (choose which in next step)
1829 r) Rename all to be different (by changing file.jpg to file-1.jpg)
1830 s/k/r> r
1831 two-1.txt: renamed from: two.txt
1832 two-2.txt: renamed from: two.txt
1833 two-3.txt: renamed from: two.txt
1834 The result being
1836 $ rclone lsl drive:dupes
1838 6048320 2016-03-05 16:23:16.798000000 one.txt
1839 564374 2016-03-05 16:22:52.118000000 two-1.txt
1840 6048320 2016-03-05 16:22:46.185000000 two-2.txt
1841 1744073 2016-03-05 16:22:38.104000000 two-3.txt
1842 Dedupe can be run non interactively using the --dedupe-mode flag or by
1844 using an extra parameter with the same value
1845 • --dedupe-mode interactive - interactive as above.
1847 • --dedupe-mode skip - removes identical files then skips anything
1849 left.
1850 • --dedupe-mode first - removes identical files then keeps the first
1852 one.
1853 • --dedupe-mode newest - removes identical files then keeps the newest
1855 one.
1856 • --dedupe-mode oldest - removes identical files then keeps the oldest
1858 one.
1859 • --dedupe-mode largest - removes identical files then keeps the
1861 largest one.
1862 • --dedupe-mode smallest - removes identical files then keeps the
1864 smallest one.
1865 • --dedupe-mode rename - removes identical files then renames the rest
1867 to be different.
1868 • --dedupe-mode list - lists duplicate dirs and files only and changes
1870 nothing.
1871 For example, to rename all the identically named photos in your Google
1873 Photos directory, do
1874 rclone dedupe --dedupe-mode rename "drive:Google Photos"
1876 Or
1878 rclone dedupe rename "drive:Google Photos"
1880 rclone dedupe [mode] remote:path [flags]
1882 Options
1884 --by-hash Find identical hashes rather than names
1885 --dedupe-mode string Dedupe mode interactive|skip|first|newest|oldest|largest|smallest|rename (default "interactive")
1886 -h, --help help for dedupe
1887 See the global flags page (https://rclone.org/flags/) for global op‐
1889 tions not listed here.
1890 SEE ALSO
1892 • rclone (https://rclone.org/commands/rclone/) - Show help for rclone
1893 commands, flags and backends.
1894
1896 Get quota information from the remote.
1897 Synopsis
1899 rclone about prints quota information about a remote to standard out‐
1900 put. The output is typically used, free, quota and trash contents.
1901 E.g. Typical output from rclone about remote: is:
1903 Total: 17 GiB
1905 Used: 7.444 GiB
1906 Free: 1.315 GiB
1907 Trashed: 100.000 MiB
1908 Other: 8.241 GiB
1909 Where the fields are:
1911 • Total: Total size available.
1913 • Used: Total size used.
1915 • Free: Total space available to this user.
1917 • Trashed: Total space used by trash.
1919 • Other: Total amount in other storage (e.g. Gmail, Google Photos).
1921 • Objects: Total number of objects in the storage.
1923 All sizes are in number of bytes.
1925 Applying a --full flag to the command prints the bytes in full, e.g.
1927 Total: 18253611008
1929 Used: 7993453766
1930 Free: 1411001220
1931 Trashed: 104857602
1932 Other: 8849156022
1933 A --json flag generates conveniently machine-readable output, e.g.
1935 {
1937 "total": 18253611008,
1938 "used": 7993453766,
1939 "trashed": 104857602,
1940 "other": 8849156022,
1941 "free": 1411001220
1942 }
1943 Not all backends print all fields. Information is not included if it
1945 is not provided by a backend. Where the value is unlimited it is omit‐
1946 ted.
1947 Some backends does not support the rclone about command at all, see
1949 complete list in documentation (https://rclone.org/overview/#optional-
1950 features).
1951 rclone about remote: [flags]
1953 Options
1955 --full Full numbers instead of human-readable
1956 -h, --help help for about
1957 --json Format output as JSON
1958 See the global flags page (https://rclone.org/flags/) for global op‐
1960 tions not listed here.
1961 SEE ALSO
1963 • rclone (https://rclone.org/commands/rclone/) - Show help for rclone
1964 commands, flags and backends.
1965
1967 Remote authorization.
1968 Synopsis
1970 Remote authorization. Used to authorize a remote or headless rclone
1971 from a machine with a browser - use as instructed by rclone config.
1972 Use the --auth-no-open-browser to prevent rclone to open auth link in
1974 default browser automatically.
1975 rclone authorize [flags]
1977 Options
1979 --auth-no-open-browser Do not automatically open auth link in default browser
1980 -h, --help help for authorize
1981 See the global flags page (https://rclone.org/flags/) for global op‐
1983 tions not listed here.
1984 SEE ALSO
1986 • rclone (https://rclone.org/commands/rclone/) - Show help for rclone
1987 commands, flags and backends.
1988
1990 Run a backend-specific command.
1991 Synopsis
1993 This runs a backend-specific command. The commands themselves (except
1994 for "help" and "features") are defined by the backends and you should
1995 see the backend docs for definitions.
1996 You can discover what commands a backend implements by using
1998 rclone backend help remote:
2000 rclone backend help <backendname>
2001 You can also discover information about the backend using (see opera‐
2003 tions/fsinfo (https://rclone.org/rc/#operations-fsinfo) in the remote
2004 control docs for more info).
2005 rclone backend features remote:
2007 Pass options to the backend command with -o. This should be key=value
2009 or key, e.g.:
2010 rclone backend stats remote:path stats -o format=json -o long
2012 Pass arguments to the backend by placing them on the end of the line
2014 rclone backend cleanup remote:path file1 file2 file3
2016 Note to run these commands on a running backend then see backend/com‐
2018 mand (https://rclone.org/rc/#backend-command) in the rc docs.
2019 rclone backend <command> remote:path [opts] <args> [flags]
2021 Options
2023 -h, --help help for backend
2024 --json Always output in JSON format
2025 -o, --option stringArray Option in the form name=value or name
2026 See the global flags page (https://rclone.org/flags/) for global op‐
2028 tions not listed here.
2029 SEE ALSO
2031 • rclone (https://rclone.org/commands/rclone/) - Show help for rclone
2032 commands, flags and backends.
2033
2035 Perform bidirectional synchronization between two paths.
2036 Synopsis
2038 Perform bidirectional synchronization between two paths.
2039 Bisync (https://rclone.org/bisync/) provides a bidirectional cloud sync
2041 solution in rclone. It retains the Path1 and Path2 filesystem listings
2042 from the prior run. On each successive run it will: - list files on
2043 Path1 and Path2, and check for changes on each side. Changes include
2044 New, Newer, Older, and Deleted files. - Propagate changes on Path1 to
2045 Path2, and vice-versa.
2046 See full bisync description (https://rclone.org/bisync/) for details.
2048 rclone bisync remote1:path1 remote2:path2 [flags]
2050 Options
2052 --check-access Ensure expected RCLONE_TEST files are found on both Path1 and Path2 filesystems, else abort.
2053 --check-filename string Filename for --check-access (default: RCLONE_TEST)
2054 --check-sync string Controls comparison of final listings: true|false|only (default: true) (default "true")
2055 --filters-file string Read filtering patterns from a file
2056 --force Bypass --max-delete safety check and run the sync. Consider using with --verbose
2057 -h, --help help for bisync
2058 --localtime Use local time in listings (default: UTC)
2059 --no-cleanup Retain working files (useful for troubleshooting and testing).
2060 --remove-empty-dirs Remove empty directories at the final cleanup step.
2061 -1, --resync Performs the resync run. Path1 files may overwrite Path2 versions. Consider using --verbose or --dry-run first.
2062 --workdir string Use custom working dir - useful for testing. (default: $HOME/.cache/rclone/bisync)
2063 See the global flags page (https://rclone.org/flags/) for global op‐
2065 tions not listed here.
2066 SEE ALSO
2068 • rclone (https://rclone.org/commands/rclone/) - Show help for rclone
2069 commands, flags and backends.
2070
2072 Concatenates any files and sends them to stdout.
2073 Synopsis
2075 rclone cat sends any files to standard output.
2076 You can use it like this to output a single file
2078 rclone cat remote:path/to/file
2080 Or like this to output any file in dir or its subdirectories.
2082 rclone cat remote:path/to/dir
2084 Or like this to output any .txt files in dir or its subdirectories.
2086 rclone --include "*.txt" cat remote:path/to/dir
2088 Use the --head flag to print characters only at the start, --tail for
2090 the end and --offset and --count to print a section in the middle.
2091 Note that if offset is negative it will count from the end, so --offset
2092 -1 --count 1 is equivalent to --tail 1.
2093 rclone cat remote:path [flags]
2095 Options
2097 --count int Only print N characters (default -1)
2098 --discard Discard the output instead of printing
2099 --head int Only print the first N characters
2100 -h, --help help for cat
2101 --offset int Start printing at offset N (or from end if -ve)
2102 --tail int Only print the last N characters
2103 See the global flags page (https://rclone.org/flags/) for global op‐
2105 tions not listed here.
2106 SEE ALSO
2108 • rclone (https://rclone.org/commands/rclone/) - Show help for rclone
2109 commands, flags and backends.
2110
2112 Checks the files in the source against a SUM file.
2113 Synopsis
2115 Checks that hashsums of source files match the SUM file. It compares
2116 hashes (MD5, SHA1, etc) and logs a report of files which don't match.
2117 It doesn't alter the file system.
2118 If you supply the --download flag, it will download the data from re‐
2120 mote and calculate the contents hash on the fly. This can be useful
2121 for remotes that don't support hashes or if you really want to check
2122 all the data.
2123 Note that hash values in the SUM file are treated as case insensitive.
2125 If you supply the --one-way flag, it will only check that files in the
2127 source match the files in the destination, not the other way around.
2128 This means that extra files in the destination that are not in the
2129 source will not be detected.
2130 The --differ, --missing-on-dst, --missing-on-src, --match and --error
2132 flags write paths, one per line, to the file name (or stdout if it is
2133 -) supplied. What they write is described in the help below. For ex‐
2134 ample --differ will write all paths which are present on both the
2135 source and destination but different.
2136 The --combined flag will write a file (or stdout) which contains all
2138 file paths with a symbol and then a space and then the path to tell you
2139 what happened to it. These are reminiscent of diff files.
2140 • = path means path was found in source and destination and was identi‐
2142 cal
2143 • `- path` means path was missing on the source, so only in the desti‐
2145 nation
2146 • `+ path` means path was missing on the destination, so only in the
2148 source
2149 • `* path` means path was present in source and destination but differ‐
2151 ent.
2152 • ! path means there was an error reading or hashing the source or
2154 dest.
2155 rclone checksum <hash> sumfile src:path [flags]
2157 Options
2159 --combined string Make a combined report of changes to this file
2160 --differ string Report all non-matching files to this file
2161 --download Check by hashing the contents
2162 --error string Report all files with errors (hashing or reading) to this file
2163 -h, --help help for checksum
2164 --match string Report all matching files to this file
2165 --missing-on-dst string Report all files missing from the destination to this file
2166 --missing-on-src string Report all files missing from the source to this file
2167 --one-way Check one way only, source files must exist on remote
2168 See the global flags page (https://rclone.org/flags/) for global op‐
2170 tions not listed here.
2171 SEE ALSO
2173 • rclone (https://rclone.org/commands/rclone/) - Show help for rclone
2174 commands, flags and backends.
2175
2177 Generate the autocompletion script for the specified shell
2178 Synopsis
2180 Generate the autocompletion script for rclone for the specified shell.
2181 See each sub-command's help for details on how to use the generated
2182 script.
2183 Options
2185 -h, --help help for completion
2186 See the global flags page (https://rclone.org/flags/) for global op‐
2188 tions not listed here.
2189 SEE ALSO
2191 • rclone (https://rclone.org/commands/rclone/) - Show help for rclone
2192 commands, flags and backends.
2193 • rclone completion bash (https://rclone.org/commands/rclone_comple‐
2195 tion_bash/) - Generate the autocompletion script for bash
2196 • rclone completion fish (https://rclone.org/commands/rclone_comple‐
2198 tion_fish/) - Generate the autocompletion script for fish
2199 • rclone completion powershell (https://rclone.org/commands/rclone_com‐
2201 pletion_powershell/) - Generate the autocompletion script for power‐
2202 shell
2203 • rclone completion zsh (https://rclone.org/commands/rclone_comple‐
2205 tion_zsh/) - Generate the autocompletion script for zsh
2206
2208 Generate the autocompletion script for bash
2209 Synopsis
2211 Generate the autocompletion script for the bash shell.
2212 This script depends on the 'bash-completion' package. If it is not in‐
2214 stalled already, you can install it via your OS's package manager.
2215 To load completions in your current shell session:
2217 source <(rclone completion bash)
2219 To load completions for every new session, execute once:
2221 Linux:
2223 rclone completion bash > /etc/bash_completion.d/rclone
2224 macOS:
2226 rclone completion bash > $(brew --prefix)/etc/bash_completion.d/rclone
2227 You will need to start a new shell for this setup to take effect.
2229 rclone completion bash
2231 Options
2233 -h, --help help for bash
2234 --no-descriptions disable completion descriptions
2235 See the global flags page (https://rclone.org/flags/) for global op‐
2237 tions not listed here.
2238 SEE ALSO
2240 • rclone completion (https://rclone.org/commands/rclone_completion/) -
2241 Generate the autocompletion script for the specified shell
2242
2244 Generate the autocompletion script for fish
2245 Synopsis
2247 Generate the autocompletion script for the fish shell.
2248 To load completions in your current shell session:
2250 rclone completion fish | source
2252 To load completions for every new session, execute once:
2254 rclone completion fish > ~/.config/fish/completions/rclone.fish
2256 You will need to start a new shell for this setup to take effect.
2258 rclone completion fish [flags]
2260 Options
2262 -h, --help help for fish
2263 --no-descriptions disable completion descriptions
2264 See the global flags page (https://rclone.org/flags/) for global op‐
2266 tions not listed here.
2267 SEE ALSO
2269 • rclone completion (https://rclone.org/commands/rclone_completion/) -
2270 Generate the autocompletion script for the specified shell
2271
2273 Generate the autocompletion script for powershell
2274 Synopsis
2276 Generate the autocompletion script for powershell.
2277 To load completions in your current shell session:
2279 rclone completion powershell | Out-String | Invoke-Expression
2281 To load completions for every new session, add the output of the above
2283 command to your powershell profile.
2284 rclone completion powershell [flags]
2286 Options
2288 -h, --help help for powershell
2289 --no-descriptions disable completion descriptions
2290 See the global flags page (https://rclone.org/flags/) for global op‐
2292 tions not listed here.
2293 SEE ALSO
2295 • rclone completion (https://rclone.org/commands/rclone_completion/) -
2296 Generate the autocompletion script for the specified shell
2297
2299 Generate the autocompletion script for zsh
2300 Synopsis
2302 Generate the autocompletion script for the zsh shell.
2303 If shell completion is not already enabled in your environment you will
2305 need to enable it. You can execute the following once:
2306 echo "autoload -U compinit; compinit" >> ~/.zshrc
2308 To load completions in your current shell session:
2310 source <(rclone completion zsh); compdef _rclone rclone
2312 To load completions for every new session, execute once:
2314 Linux:
2316 rclone completion zsh > "${fpath[1]}/_rclone"
2317 macOS:
2319 rclone completion zsh > $(brew --prefix)/share/zsh/site-functions/_rclone
2320 You will need to start a new shell for this setup to take effect.
2322 rclone completion zsh [flags]
2324 Options
2326 -h, --help help for zsh
2327 --no-descriptions disable completion descriptions
2328 See the global flags page (https://rclone.org/flags/) for global op‐
2330 tions not listed here.
2331 SEE ALSO
2333 • rclone completion (https://rclone.org/commands/rclone_completion/) -
2334 Generate the autocompletion script for the specified shell
2335
2337 Create a new remote with name, type and options.
2338 Synopsis
2340 Create a new remote of name with type and options. The options should
2341 be passed in pairs of key value or as key=value.
2342 For example, to make a swift remote of name myremote using auto config
2344 you would do:
2345 rclone config create myremote swift env_auth true
2347 rclone config create myremote swift env_auth=true
2348 So for example if you wanted to configure a Google Drive remote but us‐
2350 ing remote authorization you would do this:
2351 rclone config create mydrive drive config_is_local=false
2353 Note that if the config process would normally ask a question the de‐
2355 fault is taken (unless --non-interactive is used). Each time that hap‐
2356 pens rclone will print or DEBUG a message saying how to affect the val‐
2357 ue taken.
2358 If any of the parameters passed is a password field, then rclone will
2360 automatically obscure them if they aren't already obscured before
2361 putting them in the config file.
2362 NB If the password parameter is 22 characters or longer and consists
2364 only of base64 characters then rclone can get confused about whether
2365 the password is already obscured or not and put unobscured passwords
2366 into the config file. If you want to be 100% certain that the pass‐
2367 words get obscured then use the --obscure flag, or if you are 100% cer‐
2368 tain you are already passing obscured passwords then use --no-obscure.
2369 You can also set obscured passwords using the rclone config password
2370 command.
2371 The flag --non-interactive is for use by applications that wish to con‐
2373 figure rclone themselves, rather than using rclone's text based config‐
2374 uration questions. If this flag is set, and rclone needs to ask the
2375 user a question, a JSON blob will be returned with the question in it.
2376 This will look something like (some irrelevant detail removed):
2378 {
2380 "State": "*oauth-islocal,teamdrive,,",
2381 "Option": {
2382 "Name": "config_is_local",
2383 "Help": "Use auto config?\n * Say Y if not sure\n * Say N if you are working on a remote or headless machine\n",
2384 "Default": true,
2385 "Examples": [
2386 {
2387 "Value": "true",
2388 "Help": "Yes"
2389 },
2390 {
2391 "Value": "false",
2392 "Help": "No"
2393 }
2394 ],
2395 "Required": false,
2396 "IsPassword": false,
2397 "Type": "bool",
2398 "Exclusive": true,
2399 },
2400 "Error": "",
2401 }
2402 The format of Option is the same as returned by rclone config
2404 providers. The question should be asked to the user and returned to
2405 rclone as the --result option along with the --state parameter.
2406 The keys of Option are used as follows:
2408 • Name - name of variable - show to user
2410 • Help - help text. Hard wrapped at 80 chars. Any URLs should be
2412 clicky.
2413 • Default - default value - return this if the user just wants the de‐
2415 fault.
2416 • Examples - the user should be able to choose one of these
2418 • Required - the value should be non-empty
2420 • IsPassword - the value is a password and should be edited as such
2422 • Type - type of value, eg bool, string, int and others
2424 • Exclusive - if set no free-form entry allowed only the Examples
2426 • Irrelevant keys Provider, ShortOpt, Hide, NoPrefix, Advanced
2428 If Error is set then it should be shown to the user at the same time as
2430 the question.
2431 rclone config update name --continue --state "*oauth-islocal,teamdrive,," --result "true"
2433 Note that when using --continue all passwords should be passed in the
2435 clear (not obscured). Any default config values should be passed in
2436 with each invocation of --continue.
2437 At the end of the non interactive process, rclone will return a result
2439 with State as empty string.
2440 If --all is passed then rclone will ask all the config questions, not
2442 just the post config questions. Any parameters are used as defaults
2443 for questions as usual.
2444 Note that bin/config.py in the rclone source implements this protocol
2446 as a readable demonstration.
2447 rclone config create name type [key value]* [flags]
2449 Options
2451 --all Ask the full set of config questions
2452 --continue Continue the configuration process with an answer
2453 -h, --help help for create
2454 --no-obscure Force any passwords not to be obscured
2455 --non-interactive Don't interact with user and return questions
2456 --obscure Force any passwords to be obscured
2457 --result string Result - use with --continue
2458 --state string State - use with --continue
2459 See the global flags page (https://rclone.org/flags/) for global op‐
2461 tions not listed here.
2462 SEE ALSO
2464 • rclone config (https://rclone.org/commands/rclone_config/) - Enter an
2465 interactive configuration session.
2466
2468 Delete an existing remote.
2469 rclone config delete name [flags]
2471 Options
2473 -h, --help help for delete
2474 See the global flags page (https://rclone.org/flags/) for global op‐
2476 tions not listed here.
2477 SEE ALSO
2479 • rclone config (https://rclone.org/commands/rclone_config/) - Enter an
2480 interactive configuration session.
2481
2483 Disconnects user from remote
2484 Synopsis
2486 This disconnects the remote: passed in to the cloud storage system.
2487 This normally means revoking the oauth token.
2489 To reconnect use "rclone config reconnect".
2491 rclone config disconnect remote: [flags]
2493 Options
2495 -h, --help help for disconnect
2496 See the global flags page (https://rclone.org/flags/) for global op‐
2498 tions not listed here.
2499 SEE ALSO
2501 • rclone config (https://rclone.org/commands/rclone_config/) - Enter an
2502 interactive configuration session.
2503
2505 Dump the config file as JSON.
2506 rclone config dump [flags]
2508 Options
2510 -h, --help help for dump
2511 See the global flags page (https://rclone.org/flags/) for global op‐
2513 tions not listed here.
2514 SEE ALSO
2516 • rclone config (https://rclone.org/commands/rclone_config/) - Enter an
2517 interactive configuration session.
2518
2520 Enter an interactive configuration session.
2521
2523 Enter an interactive configuration session where you can setup new re‐
2524 motes and manage existing ones. You may also set or remove a password
2525 to protect your configuration.
2526 rclone config edit [flags]
2528
2530 -h, --help help for edit
2531 See the global flags page (https://rclone.org/flags/) for global op‐
2533 tions not listed here.
2534
2536 • rclone config (https://rclone.org/commands/rclone_config/) - Enter an
2537 interactive configuration session.
2538
2540 Show path of configuration file in use.
2541 rclone config file [flags]
2543 Options
2545 -h, --help help for file
2546 See the global flags page (https://rclone.org/flags/) for global op‐
2548 tions not listed here.
2549 SEE ALSO
2551 • rclone config (https://rclone.org/commands/rclone_config/) - Enter an
2552 interactive configuration session.
2553
2555 Update password in an existing remote.
2556 Synopsis
2558 Update an existing remote's password. The password should be passed in
2559 pairs of key password or as key=password. The password should be
2560 passed in in clear (unobscured).
2561 For example, to set password of a remote of name myremote you would do:
2563 rclone config password myremote fieldname mypassword
2565 rclone config password myremote fieldname=mypassword
2566 This command is obsolete now that "config update" and "config create"
2568 both support obscuring passwords directly.
2569 rclone config password name [key value]+ [flags]
2571 Options
2573 -h, --help help for password
2574 See the global flags page (https://rclone.org/flags/) for global op‐
2576 tions not listed here.
2577 SEE ALSO
2579 • rclone config (https://rclone.org/commands/rclone_config/) - Enter an
2580 interactive configuration session.
2581
2583 Show paths used for configuration, cache, temp etc.
2584 rclone config paths [flags]
2586 Options
2588 -h, --help help for paths
2589 See the global flags page (https://rclone.org/flags/) for global op‐
2591 tions not listed here.
2592 SEE ALSO
2594 • rclone config (https://rclone.org/commands/rclone_config/) - Enter an
2595 interactive configuration session.
2596
2598 List in JSON format all the providers and options.
2599 rclone config providers [flags]
2601 Options
2603 -h, --help help for providers
2604 See the global flags page (https://rclone.org/flags/) for global op‐
2606 tions not listed here.
2607 SEE ALSO
2609 • rclone config (https://rclone.org/commands/rclone_config/) - Enter an
2610 interactive configuration session.
2611
2613 Re-authenticates user with remote.
2614 Synopsis
2616 This reconnects remote: passed in to the cloud storage system.
2617 To disconnect the remote use "rclone config disconnect".
2619 This normally means going through the interactive oauth flow again.
2621 rclone config reconnect remote: [flags]
2623 Options
2625 -h, --help help for reconnect
2626 See the global flags page (https://rclone.org/flags/) for global op‐
2628 tions not listed here.
2629 SEE ALSO
2631 • rclone config (https://rclone.org/commands/rclone_config/) - Enter an
2632 interactive configuration session.
2633
2635 Print (decrypted) config file, or the config for a single remote.
2636 rclone config show [<remote>] [flags]
2638 Options
2640 -h, --help help for show
2641 See the global flags page (https://rclone.org/flags/) for global op‐
2643 tions not listed here.
2644 SEE ALSO
2646 • rclone config (https://rclone.org/commands/rclone_config/) - Enter an
2647 interactive configuration session.
2648
2650 Ensure configuration file exists.
2651 rclone config touch [flags]
2653 Options
2655 -h, --help help for touch
2656 See the global flags page (https://rclone.org/flags/) for global op‐
2658 tions not listed here.
2659 SEE ALSO
2661 • rclone config (https://rclone.org/commands/rclone_config/) - Enter an
2662 interactive configuration session.
2663
2665 Update options in an existing remote.
2666 Synopsis
2668 Update an existing remote's options. The options should be passed in
2669 pairs of key value or as key=value.
2670 For example, to update the env_auth field of a remote of name myremote
2672 you would do:
2673 rclone config update myremote env_auth true
2675 rclone config update myremote env_auth=true
2676 If the remote uses OAuth the token will be updated, if you don't re‐
2678 quire this add an extra parameter thus:
2679 rclone config update myremote env_auth=true config_refresh_token=false
2681 Note that if the config process would normally ask a question the de‐
2683 fault is taken (unless --non-interactive is used). Each time that hap‐
2684 pens rclone will print or DEBUG a message saying how to affect the val‐
2685 ue taken.
2686 If any of the parameters passed is a password field, then rclone will
2688 automatically obscure them if they aren't already obscured before
2689 putting them in the config file.
2690 NB If the password parameter is 22 characters or longer and consists
2692 only of base64 characters then rclone can get confused about whether
2693 the password is already obscured or not and put unobscured passwords
2694 into the config file. If you want to be 100% certain that the pass‐
2695 words get obscured then use the --obscure flag, or if you are 100% cer‐
2696 tain you are already passing obscured passwords then use --no-obscure.
2697 You can also set obscured passwords using the rclone config password
2698 command.
2699 The flag --non-interactive is for use by applications that wish to con‐
2701 figure rclone themselves, rather than using rclone's text based config‐
2702 uration questions. If this flag is set, and rclone needs to ask the
2703 user a question, a JSON blob will be returned with the question in it.
2704 This will look something like (some irrelevant detail removed):
2706 {
2708 "State": "*oauth-islocal,teamdrive,,",
2709 "Option": {
2710 "Name": "config_is_local",
2711 "Help": "Use auto config?\n * Say Y if not sure\n * Say N if you are working on a remote or headless machine\n",
2712 "Default": true,
2713 "Examples": [
2714 {
2715 "Value": "true",
2716 "Help": "Yes"
2717 },
2718 {
2719 "Value": "false",
2720 "Help": "No"
2721 }
2722 ],
2723 "Required": false,
2724 "IsPassword": false,
2725 "Type": "bool",
2726 "Exclusive": true,
2727 },
2728 "Error": "",
2729 }
2730 The format of Option is the same as returned by rclone config
2732 providers. The question should be asked to the user and returned to
2733 rclone as the --result option along with the --state parameter.
2734 The keys of Option are used as follows:
2736 • Name - name of variable - show to user
2738 • Help - help text. Hard wrapped at 80 chars. Any URLs should be
2740 clicky.
2741 • Default - default value - return this if the user just wants the de‐
2743 fault.
2744 • Examples - the user should be able to choose one of these
2746 • Required - the value should be non-empty
2748 • IsPassword - the value is a password and should be edited as such
2750 • Type - type of value, eg bool, string, int and others
2752 • Exclusive - if set no free-form entry allowed only the Examples
2754 • Irrelevant keys Provider, ShortOpt, Hide, NoPrefix, Advanced
2756 If Error is set then it should be shown to the user at the same time as
2758 the question.
2759 rclone config update name --continue --state "*oauth-islocal,teamdrive,," --result "true"
2761 Note that when using --continue all passwords should be passed in the
2763 clear (not obscured). Any default config values should be passed in
2764 with each invocation of --continue.
2765 At the end of the non interactive process, rclone will return a result
2767 with State as empty string.
2768 If --all is passed then rclone will ask all the config questions, not
2770 just the post config questions. Any parameters are used as defaults
2771 for questions as usual.
2772 Note that bin/config.py in the rclone source implements this protocol
2774 as a readable demonstration.
2775 rclone config update name [key value]+ [flags]
2777 Options
2779 --all Ask the full set of config questions
2780 --continue Continue the configuration process with an answer
2781 -h, --help help for update
2782 --no-obscure Force any passwords not to be obscured
2783 --non-interactive Don't interact with user and return questions
2784 --obscure Force any passwords to be obscured
2785 --result string Result - use with --continue
2786 --state string State - use with --continue
2787 See the global flags page (https://rclone.org/flags/) for global op‐
2789 tions not listed here.
2790 SEE ALSO
2792 • rclone config (https://rclone.org/commands/rclone_config/) - Enter an
2793 interactive configuration session.
2794
2796 Prints info about logged in user of remote.
2797 Synopsis
2799 This prints the details of the person logged in to the cloud storage
2800 system.
2801 rclone config userinfo remote: [flags]
2803 Options
2805 -h, --help help for userinfo
2806 --json Format output as JSON
2807 See the global flags page (https://rclone.org/flags/) for global op‐
2809 tions not listed here.
2810 SEE ALSO
2812 • rclone config (https://rclone.org/commands/rclone_config/) - Enter an
2813 interactive configuration session.
2814
2816 Copy files from source to dest, skipping identical files.
2817 Synopsis
2819 If source:path is a file or directory then it copies it to a file or
2820 directory named dest:path.
2821 This can be used to upload single files to other than their current
2823 name. If the source is a directory then it acts exactly like the copy
2824 (https://rclone.org/commands/rclone_copy/) command.
2825 So
2827 rclone copyto src dst
2829 where src and dst are rclone paths, either remote:path or /path/to/lo‐
2831 cal or C:.
2832 This will:
2834 if src is file
2836 copy it to dst, overwriting an existing file if it exists
2837 if src is directory
2838 copy it to dst, overwriting existing files if they exist
2839 see copy command for full details
2840 This doesn't transfer files that are identical on src and dst, testing
2842 by size and modification time or MD5SUM. It doesn't delete files from
2843 the destination.
2844 Note: Use the -P/--progress flag to view real-time transfer statistics
2846 rclone copyto source:path dest:path [flags]
2848 Options
2850 -h, --help help for copyto
2851 See the global flags page (https://rclone.org/flags/) for global op‐
2853 tions not listed here.
2854 SEE ALSO
2856 • rclone (https://rclone.org/commands/rclone/) - Show help for rclone
2857 commands, flags and backends.
2858
2860 Copy url content to dest.
2861 Synopsis
2863 Download a URL's content and copy it to the destination without saving
2864 it in temporary storage.
2865 Setting --auto-filename will attempt to automatically determine the
2867 filename from the URL (after any redirections) and used in the destina‐
2868 tion path. With --auto-filename-header in addition, if a specific
2869 filename is set in HTTP headers, it will be used instead of the name
2870 from the URL. With --print-filename in addition, the resulting file
2871 name will be printed.
2872 Setting --no-clobber will prevent overwriting file on the destination
2874 if there is one with the same name.
2875 Setting --stdout or making the output file name - will cause the output
2877 to be written to standard output.
2878 rclone copyurl https://example.com dest:path [flags]
2880 Options
2882 -a, --auto-filename Get the file name from the URL and use it for destination file path
2883 --header-filename Get the file name from the Content-Disposition header
2884 -h, --help help for copyurl
2885 --no-clobber Prevent overwriting file with same name
2886 -p, --print-filename Print the resulting name from --auto-filename
2887 --stdout Write the output to stdout rather than a file
2888 See the global flags page (https://rclone.org/flags/) for global op‐
2890 tions not listed here.
2891 SEE ALSO
2893 • rclone (https://rclone.org/commands/rclone/) - Show help for rclone
2894 commands, flags and backends.
2895
2897 Cryptcheck checks the integrity of a crypted remote.
2898 Synopsis
2900 rclone cryptcheck checks a remote against a crypted
2901 (https://rclone.org/crypt/) remote. This is the equivalent of running
2902 rclone check (https://rclone.org/commands/rclone_check/), but able to
2903 check the checksums of the crypted remote.
2904 For it to work the underlying remote of the cryptedremote must support
2906 some kind of checksum.
2907 It works by reading the nonce from each file on the cryptedremote: and
2909 using that to encrypt each file on the remote:. It then checks the
2910 checksum of the underlying file on the cryptedremote: against the
2911 checksum of the file it has just encrypted.
2912 Use it like this
2914 rclone cryptcheck /path/to/files encryptedremote:path
2916 You can use it like this also, but that will involve downloading all
2918 the files in remote:path.
2919 rclone cryptcheck remote:path encryptedremote:path
2921 After it has run it will log the status of the encryptedremote:.
2923 If you supply the --one-way flag, it will only check that files in the
2925 source match the files in the destination, not the other way around.
2926 This means that extra files in the destination that are not in the
2927 source will not be detected.
2928 The --differ, --missing-on-dst, --missing-on-src, --match and --error
2930 flags write paths, one per line, to the file name (or stdout if it is
2931 -) supplied. What they write is described in the help below. For ex‐
2932 ample --differ will write all paths which are present on both the
2933 source and destination but different.
2934 The --combined flag will write a file (or stdout) which contains all
2936 file paths with a symbol and then a space and then the path to tell you
2937 what happened to it. These are reminiscent of diff files.
2938 • = path means path was found in source and destination and was identi‐
2940 cal
2941 • `- path` means path was missing on the source, so only in the desti‐
2943 nation
2944 • `+ path` means path was missing on the destination, so only in the
2946 source
2947 • `* path` means path was present in source and destination but differ‐
2949 ent.
2950 • ! path means there was an error reading or hashing the source or
2952 dest.
2953 rclone cryptcheck remote:path cryptedremote:path [flags]
2955 Options
2957 --combined string Make a combined report of changes to this file
2958 --differ string Report all non-matching files to this file
2959 --error string Report all files with errors (hashing or reading) to this file
2960 -h, --help help for cryptcheck
2961 --match string Report all matching files to this file
2962 --missing-on-dst string Report all files missing from the destination to this file
2963 --missing-on-src string Report all files missing from the source to this file
2964 --one-way Check one way only, source files must exist on remote
2965 See the global flags page (https://rclone.org/flags/) for global op‐
2967 tions not listed here.
2968 SEE ALSO
2970 • rclone (https://rclone.org/commands/rclone/) - Show help for rclone
2971 commands, flags and backends.
2972
2974 Cryptdecode returns unencrypted file names.
2975 Synopsis
2977 rclone cryptdecode returns unencrypted file names when provided with a
2978 list of encrypted file names. List limit is 10 items.
2979 If you supply the --reverse flag, it will return encrypted file names.
2981 use it like this
2983 rclone cryptdecode encryptedremote: encryptedfilename1 encryptedfilename2
2985 rclone cryptdecode --reverse encryptedremote: filename1 filename2
2987 Another way to accomplish this is by using the rclone backend encode
2989 (or decode) command. See the documentation on the crypt
2990 (https://rclone.org/crypt/) overlay for more info.
2991 rclone cryptdecode encryptedremote: encryptedfilename [flags]
2993 Options
2995 -h, --help help for cryptdecode
2996 --reverse Reverse cryptdecode, encrypts filenames
2997 See the global flags page (https://rclone.org/flags/) for global op‐
2999 tions not listed here.
3000 SEE ALSO
3002 • rclone (https://rclone.org/commands/rclone/) - Show help for rclone
3003 commands, flags and backends.
3004
3006 Remove a single file from remote.
3007 Synopsis
3009 Remove a single file from remote. Unlike delete it cannot be used to
3010 remove a directory and it doesn't obey include/exclude filters - if the
3011 specified file exists, it will always be removed.
3012 rclone deletefile remote:path [flags]
3014 Options
3016 -h, --help help for deletefile
3017 See the global flags page (https://rclone.org/flags/) for global op‐
3019 tions not listed here.
3020 SEE ALSO
3022 • rclone (https://rclone.org/commands/rclone/) - Show help for rclone
3023 commands, flags and backends.
3024
3026 Output completion script for a given shell.
3027 Synopsis
3029 Generates a shell completion script for rclone. Run with --help to
3030 list the supported shells.
3031 Options
3033 -h, --help help for genautocomplete
3034 See the global flags page (https://rclone.org/flags/) for global op‐
3036 tions not listed here.
3037 SEE ALSO
3039 • rclone (https://rclone.org/commands/rclone/) - Show help for rclone
3040 commands, flags and backends.
3041 • rclone genautocomplete bash (https://rclone.org/commands/rclone_ge‐
3043 nautocomplete_bash/) - Output bash completion script for rclone.
3044 • rclone genautocomplete fish (https://rclone.org/commands/rclone_ge‐
3046 nautocomplete_fish/) - Output fish completion script for rclone.
3047 • rclone genautocomplete zsh (https://rclone.org/commands/rclone_genau‐
3049 tocomplete_zsh/) - Output zsh completion script for rclone.
3050
3052 Output bash completion script for rclone.
3053 Synopsis
3055 Generates a bash shell autocompletion script for rclone.
3056 This writes to /etc/bash_completion.d/rclone by default so will proba‐
3058 bly need to be run with sudo or as root, e.g.
3059 sudo rclone genautocomplete bash
3061 Logout and login again to use the autocompletion scripts, or source
3063 them directly
3064 . /etc/bash_completion
3066 If you supply a command line argument the script will be written there.
3068 If output_file is "-", then the output will be written to stdout.
3070 rclone genautocomplete bash [output_file] [flags]
3072 Options
3074 -h, --help help for bash
3075 See the global flags page (https://rclone.org/flags/) for global op‐
3077 tions not listed here.
3078 SEE ALSO
3080 • rclone genautocomplete (https://rclone.org/commands/rclone_genauto‐
3081 complete/) - Output completion script for a given shell.
3082
3084 Output fish completion script for rclone.
3085 Synopsis
3087 Generates a fish autocompletion script for rclone.
3088 This writes to /etc/fish/completions/rclone.fish by default so will
3090 probably need to be run with sudo or as root, e.g.
3091 sudo rclone genautocomplete fish
3093 Logout and login again to use the autocompletion scripts, or source
3095 them directly
3096 . /etc/fish/completions/rclone.fish
3098 If you supply a command line argument the script will be written there.
3100 If output_file is "-", then the output will be written to stdout.
3102 rclone genautocomplete fish [output_file] [flags]
3104 Options
3106 -h, --help help for fish
3107 See the global flags page (https://rclone.org/flags/) for global op‐
3109 tions not listed here.
3110 SEE ALSO
3112 • rclone genautocomplete (https://rclone.org/commands/rclone_genauto‐
3113 complete/) - Output completion script for a given shell.
3114
3116 Output zsh completion script for rclone.
3117 Synopsis
3119 Generates a zsh autocompletion script for rclone.
3120 This writes to /usr/share/zsh/vendor-completions/_rclone by default so
3122 will probably need to be run with sudo or as root, e.g.
3123 sudo rclone genautocomplete zsh
3125 Logout and login again to use the autocompletion scripts, or source
3127 them directly
3128 autoload -U compinit && compinit
3130 If you supply a command line argument the script will be written there.
3132 If output_file is "-", then the output will be written to stdout.
3134 rclone genautocomplete zsh [output_file] [flags]
3136 Options
3138 -h, --help help for zsh
3139 See the global flags page (https://rclone.org/flags/) for global op‐
3141 tions not listed here.
3142 SEE ALSO
3144 • rclone genautocomplete (https://rclone.org/commands/rclone_genauto‐
3145 complete/) - Output completion script for a given shell.
3146
3148 Output markdown docs for rclone to the directory supplied.
3149 Synopsis
3151 This produces markdown docs for the rclone commands to the directory
3152 supplied. These are in a format suitable for hugo to render into the
3153 rclone.org website.
3154 rclone gendocs output_directory [flags]
3156 Options
3158 -h, --help help for gendocs
3159 See the global flags page (https://rclone.org/flags/) for global op‐
3161 tions not listed here.
3162 SEE ALSO
3164 • rclone (https://rclone.org/commands/rclone/) - Show help for rclone
3165 commands, flags and backends.
3166
3168 Produces a hashsum file for all the objects in the path.
3169 Synopsis
3171 Produces a hash file for all the objects in the path using the hash
3172 named. The output is in the same format as the standard md5sum/sha1sum
3173 tool.
3174 By default, the hash is requested from the remote. If the hash is not
3176 supported by the remote, no hash will be returned. With the download
3177 flag, the file will be downloaded from the remote and hashed locally
3178 enabling any hash for any remote.
3179 For the MD5 and SHA1 algorithms there are also dedicated commands,
3181 md5sum (https://rclone.org/commands/rclone_md5sum/) and sha1sum
3182 (https://rclone.org/commands/rclone_sha1sum/).
3183 This command can also hash data received on standard input (stdin), by
3185 not passing a remote:path, or by passing a hyphen as remote:path when
3186 there is data to read (if not, the hyphen will be treated literally, as
3187 a relative path).
3188 Run without a hash to see the list of all supported hashes, e.g.
3190 $ rclone hashsum
3192 Supported hashes are:
3193 * md5
3194 * sha1
3195 * whirlpool
3196 * crc32
3197 * sha256
3198 * dropbox
3199 * hidrive
3200 * mailru
3201 * quickxor
3202 Then
3204 $ rclone hashsum MD5 remote:path
3206 Note that hash names are case insensitive and values are output in low‐
3208 er case.
3209 rclone hashsum <hash> remote:path [flags]
3211 Options
3213 --base64 Output base64 encoded hashsum
3214 -C, --checkfile string Validate hashes against a given SUM file instead of printing them
3215 --download Download the file and hash it locally; if this flag is not specified, the hash is requested from the remote
3216 -h, --help help for hashsum
3217 --output-file string Output hashsums to a file rather than the terminal
3218 See the global flags page (https://rclone.org/flags/) for global op‐
3220 tions not listed here.
3221 SEE ALSO
3223 • rclone (https://rclone.org/commands/rclone/) - Show help for rclone
3224 commands, flags and backends.
3225
3227 Generate public link to file/folder.
3228 Synopsis
3230 rclone link will create, retrieve or remove a public link to the given
3231 file or folder.
3232 rclone link remote:path/to/file
3234 rclone link remote:path/to/folder/
3235 rclone link --unlink remote:path/to/folder/
3236 rclone link --expire 1d remote:path/to/file
3237 If you supply the --expire flag, it will set the expiration time other‐
3239 wise it will use the default (100 years). Note not all backends sup‐
3240 port the --expire flag - if the backend doesn't support it then the
3241 link returned won't expire.
3242 Use the --unlink flag to remove existing public links to the file or
3244 folder. Note not all backends support "--unlink" flag - those that
3245 don't will just ignore it.
3246 If successful, the last line of the output will contain the link. Ex‐
3248 act capabilities depend on the remote, but the link will always by de‐
3249 fault be created with the least constraints – e.g. no expiry, no pass‐
3250 word protection, accessible without account.
3251 rclone link remote:path [flags]
3253 Options
3255 --expire Duration The amount of time that the link will be valid (default off)
3256 -h, --help help for link
3257 --unlink Remove existing public link to file/folder
3258 See the global flags page (https://rclone.org/flags/) for global op‐
3260 tions not listed here.
3261 SEE ALSO
3263 • rclone (https://rclone.org/commands/rclone/) - Show help for rclone
3264 commands, flags and backends.
3265
3267 List all the remotes in the config file.
3268 Synopsis
3270 rclone listremotes lists all the available remotes from the config
3271 file.
3272 When used with the --long flag it lists the types too.
3274 rclone listremotes [flags]
3276 Options
3278 -h, --help help for listremotes
3279 --long Show the type as well as names
3280 See the global flags page (https://rclone.org/flags/) for global op‐
3282 tions not listed here.
3283 SEE ALSO
3285 • rclone (https://rclone.org/commands/rclone/) - Show help for rclone
3286 commands, flags and backends.
3287
3289 List directories and objects in remote:path formatted for parsing.
3290 Synopsis
3292 List the contents of the source path (directories and objects) to stan‐
3293 dard output in a form which is easy to parse by scripts. By default
3294 this will just be the names of the objects and directories, one per
3295 line. The directories will have a / suffix.
3296 Eg
3298 $ rclone lsf swift:bucket
3300 bevajer5jef
3301 canole
3302 diwogej7
3303 ferejej3gux/
3304 fubuwic
3305 Use the --format option to control what gets listed. By default this
3307 is just the path, but you can use these parameters to control the out‐
3308 put:
3309 p - path
3311 s - size
3312 t - modification time
3313 h - hash
3314 i - ID of object
3315 o - Original ID of underlying object
3316 m - MimeType of object if known
3317 e - encrypted name
3318 T - tier of storage if known, e.g. "Hot" or "Cool"
3319 M - Metadata of object in JSON blob format, eg {"key":"value"}
3320 So if you wanted the path, size and modification time, you would use
3322 --format "pst", or maybe --format "tsp" to put the path last.
3323 Eg
3325 $ rclone lsf --format "tsp" swift:bucket
3327 2016-06-25 18:55:41;60295;bevajer5jef
3328 2016-06-25 18:55:43;90613;canole
3329 2016-06-25 18:55:43;94467;diwogej7
3330 2018-04-26 08:50:45;0;ferejej3gux/
3331 2016-06-25 18:55:40;37600;fubuwic
3332 If you specify "h" in the format you will get the MD5 hash by default,
3334 use the --hash flag to change which hash you want. Note that this can
3335 be returned as an empty string if it isn't available on the object (and
3336 for directories), "ERROR" if there was an error reading it from the ob‐
3337 ject and "UNSUPPORTED" if that object does not support that hash type.
3338 For example, to emulate the md5sum command you can use
3340 rclone lsf -R --hash MD5 --format hp --separator " " --files-only .
3342 Eg
3344 $ rclone lsf -R --hash MD5 --format hp --separator " " --files-only swift:bucket
3346 7908e352297f0f530b84a756f188baa3 bevajer5jef
3347 cd65ac234e6fea5925974a51cdd865cc canole
3348 03b5341b4f234b9d984d03ad076bae91 diwogej7
3349 8fd37c3810dd660778137ac3a66cc06d fubuwic
3350 99713e14a4c4ff553acaf1930fad985b gixacuh7ku
3351 (Though "rclone md5sum ." is an easier way of typing this.)
3353 By default the separator is ";" this can be changed with the --separa‐
3355 tor flag. Note that separators aren't escaped in the path so putting
3356 it last is a good strategy.
3357 Eg
3359 $ rclone lsf --separator "," --format "tshp" swift:bucket
3361 2016-06-25 18:55:41,60295,7908e352297f0f530b84a756f188baa3,bevajer5jef
3362 2016-06-25 18:55:43,90613,cd65ac234e6fea5925974a51cdd865cc,canole
3363 2016-06-25 18:55:43,94467,03b5341b4f234b9d984d03ad076bae91,diwogej7
3364 2018-04-26 08:52:53,0,,ferejej3gux/
3365 2016-06-25 18:55:40,37600,8fd37c3810dd660778137ac3a66cc06d,fubuwic
3366 You can output in CSV standard format. This will escape things in " if
3368 they contain ,
3369 Eg
3371 $ rclone lsf --csv --files-only --format ps remote:path
3373 test.log,22355
3374 test.sh,449
3375 "this file contains a comma, in the file name.txt",6
3376 Note that the --absolute parameter is useful for making lists of files
3378 to pass to an rclone copy with the --files-from-raw flag.
3379 For example, to find all the files modified within one day and copy
3381 those only (without traversing the whole directory structure):
3382 rclone lsf --absolute --files-only --max-age 1d /path/to/local > new_files
3384 rclone copy --files-from-raw new_files /path/to/local remote:path
3385 Any of the filtering options can be applied to this command.
3387 There are several related list commands
3389 • ls to list size and path of objects only
3391 • lsl to list modification time, size and path of objects only
3393 • lsd to list directories only
3395 • lsf to list objects and directories in easy to parse format
3397 • lsjson to list objects and directories in JSON format
3399 ls,lsl,lsd are designed to be human-readable. lsf is designed to be
3401 human and machine-readable. lsjson is designed to be machine-readable.
3402 Note that ls and lsl recurse by default - use --max-depth 1 to stop the
3404 recursion.
3405 The other list commands lsd,lsf,lsjson do not recurse by default - use
3407 -R to make them recurse.
3408 Listing a nonexistent directory will produce an error except for re‐
3410 motes which can't have empty directories (e.g. s3, swift, or gcs - the
3411 bucket-based remotes).
3412 rclone lsf remote:path [flags]
3414 Options
3416 --absolute Put a leading / in front of path names
3417 --csv Output in CSV format
3418 -d, --dir-slash Append a slash to directory names (default true)
3419 --dirs-only Only list directories
3420 --files-only Only list files
3421 -F, --format string Output format - see help for details (default "p")
3422 --hash h Use this hash when h is used in the format MD5|SHA-1|DropboxHash (default "md5")
3423 -h, --help help for lsf
3424 -R, --recursive Recurse into the listing
3425 -s, --separator string Separator for the items in the format (default ";")
3426 See the global flags page (https://rclone.org/flags/) for global op‐
3428 tions not listed here.
3429 SEE ALSO
3431 • rclone (https://rclone.org/commands/rclone/) - Show help for rclone
3432 commands, flags and backends.
3433
3435 List directories and objects in the path in JSON format.
3436 Synopsis
3438 List directories and objects in the path in JSON format.
3439 The output is an array of Items, where each Item looks like this
3441 {
3443 "Hashes" : {
3444 "SHA-1" : "f572d396fae9206628714fb2ce00f72e94f2258f",
3445 "MD5" : "b1946ac92492d2347c6235b4d2611184",
3446 "DropboxHash" : "ecb65bb98f9d905b70458986c39fcbad7715e5f2fcc3b1f07767d7c83e2438cc"
3447 },
3448 "ID": "y2djkhiujf83u33",
3449 "OrigID": "UYOJVTUW00Q1RzTDA",
3450 "IsBucket" : false,
3451 "IsDir" : false,
3452 "MimeType" : "application/octet-stream",
3453 "ModTime" : "2017-05-31T16:15:57.034468261+01:00",
3454 "Name" : "file.txt",
3455 "Encrypted" : "v0qpsdq8anpci8n929v3uu9338",
3456 "EncryptedPath" : "kja9098349023498/v0qpsdq8anpci8n929v3uu9338",
3457 "Path" : "full/path/goes/here/file.txt",
3458 "Size" : 6,
3459 "Tier" : "hot",
3460 }
3461 If --hash is not specified the Hashes property won't be emitted. The
3463 types of hash can be specified with the --hash-type parameter (which
3464 may be repeated). If --hash-type is set then it implies --hash.
3465 If --no-modtime is specified then ModTime will be blank. This can
3467 speed things up on remotes where reading the ModTime takes an extra re‐
3468 quest (e.g. s3, swift).
3469 If --no-mimetype is specified then MimeType will be blank. This can
3471 speed things up on remotes where reading the MimeType takes an extra
3472 request (e.g. s3, swift).
3473 If --encrypted is not specified the Encrypted won't be emitted.
3475 If --dirs-only is not specified files in addition to directories are
3477 returned
3478 If --files-only is not specified directories in addition to the files
3480 will be returned.
3481 If --metadata is set then an additional Metadata key will be returned.
3483 This will have metadata in rclone standard format as a JSON object.
3484 if --stat is set then a single JSON blob will be returned about the
3486 item pointed to. This will return an error if the item isn't found.
3487 However on bucket based backends (like s3, gcs, b2, azureblob etc) if
3488 the item isn't found it will return an empty directory as it isn't pos‐
3489 sible to tell empty directories from missing directories there.
3490 The Path field will only show folders below the remote path being list‐
3492 ed. If "remote:path" contains the file "subfolder/file.txt", the Path
3493 for "file.txt" will be "subfolder/file.txt", not "remote:path/subfold‐
3494 er/file.txt". When used without --recursive the Path will always be
3495 the same as Name.
3496 If the directory is a bucket in a bucket-based backend, then "IsBucket"
3498 will be set to true. This key won't be present unless it is "true".
3499 The time is in RFC3339 format with up to nanosecond precision. The
3501 number of decimal digits in the seconds will depend on the precision
3502 that the remote can hold the times, so if times are accurate to the
3503 nearest millisecond (e.g. Google Drive) then 3 digits will always be
3504 shown ("2017-05-31T16:15:57.034+01:00") whereas if the times are accu‐
3505 rate to the nearest second (Dropbox, Box, WebDav, etc.) no digits will
3506 be shown ("2017-05-31T16:15:57+01:00").
3507 The whole output can be processed as a JSON blob, or alternatively it
3509 can be processed line by line as each item is written one to a line.
3510 Any of the filtering options can be applied to this command.
3512 There are several related list commands
3514 • ls to list size and path of objects only
3516 • lsl to list modification time, size and path of objects only
3518 • lsd to list directories only
3520 • lsf to list objects and directories in easy to parse format
3522 • lsjson to list objects and directories in JSON format
3524 ls,lsl,lsd are designed to be human-readable. lsf is designed to be
3526 human and machine-readable. lsjson is designed to be machine-readable.
3527 Note that ls and lsl recurse by default - use --max-depth 1 to stop the
3529 recursion.
3530 The other list commands lsd,lsf,lsjson do not recurse by default - use
3532 -R to make them recurse.
3533 Listing a nonexistent directory will produce an error except for re‐
3535 motes which can't have empty directories (e.g. s3, swift, or gcs - the
3536 bucket-based remotes).
3537 rclone lsjson remote:path [flags]
3539 Options
3541 --dirs-only Show only directories in the listing
3542 --encrypted Show the encrypted names
3543 --files-only Show only files in the listing
3544 --hash Include hashes in the output (may take longer)
3545 --hash-type stringArray Show only this hash type (may be repeated)
3546 -h, --help help for lsjson
3547 --no-mimetype Don't read the mime type (can speed things up)
3548 --no-modtime Don't read the modification time (can speed things up)
3549 --original Show the ID of the underlying Object
3550 -R, --recursive Recurse into the listing
3551 --stat Just return the info for the pointed to file
3552 See the global flags page (https://rclone.org/flags/) for global op‐
3554 tions not listed here.
3555 SEE ALSO
3557 • rclone (https://rclone.org/commands/rclone/) - Show help for rclone
3558 commands, flags and backends.
3559
3561 Mount the remote as file system on a mountpoint.
3562 Synopsis
3564 rclone mount allows Linux, FreeBSD, macOS and Windows to mount any of
3565 Rclone's cloud storage systems as a file system with FUSE.
3566 First set up your remote using rclone config. Check it works with
3568 rclone ls etc.
3569 On Linux and macOS, you can run mount in either foreground or back‐
3571 ground (aka daemon) mode. Mount runs in foreground mode by default.
3572 Use the --daemon flag to force background mode. On Windows you can run
3573 mount in foreground only, the flag is ignored.
3574 In background mode rclone acts as a generic Unix mount program: the
3576 main program starts, spawns background rclone process to setup and
3577 maintain the mount, waits until success or timeout and exits with ap‐
3578 propriate code (killing the child process if it fails).
3579 On Linux/macOS/FreeBSD start the mount like this, where /path/to/lo‐
3581 cal/mount is an empty existing directory:
3582 rclone mount remote:path/to/files /path/to/local/mount
3584 On Windows you can start a mount in different ways. See below for de‐
3586 tails. If foreground mount is used interactively from a console win‐
3587 dow, rclone will serve the mount and occupy the console so another win‐
3588 dow should be used to work with the mount until rclone is interrupted
3589 e.g. by pressing Ctrl-C.
3590 The following examples will mount to an automatically assigned drive,
3592 to specific drive letter X:, to path C:\path\parent\mount (where parent
3593 directory or drive must exist, and mount must not exist, and is not
3594 supported when mounting as a network drive), and the last example will
3595 mount as network share \\cloud\remote and map it to an automatically
3596 assigned drive:
3597 rclone mount remote:path/to/files *
3599 rclone mount remote:path/to/files X:
3600 rclone mount remote:path/to/files C:\path\parent\mount
3601 rclone mount remote:path/to/files \\cloud\remote
3602 When the program ends while in foreground mode, either via Ctrl+C or
3604 receiving a SIGINT or SIGTERM signal, the mount should be automatically
3605 stopped.
3606 When running in background mode the user will have to stop the mount
3608 manually:
3609 # Linux
3611 fusermount -u /path/to/local/mount
3612 # OS X
3613 umount /path/to/local/mount
3614 The umount operation can fail, for example when the mountpoint is busy.
3616 When that happens, it is the user's responsibility to stop the mount
3617 manually.
3618 The size of the mounted file system will be set according to informa‐
3620 tion retrieved from the remote, the same as returned by the rclone
3621 about (https://rclone.org/commands/rclone_about/) command. Remotes
3622 with unlimited storage may report the used size only, then an addition‐
3623 al 1 PiB of free space is assumed. If the remote does not support
3624 (https://rclone.org/overview/#optional-features) the about feature at
3625 all, then 1 PiB is set as both the total and the free size.
3626 Installing on Windows
3628 To run rclone mount on Windows, you will need to download and install
3629 WinFsp (http://www.secfs.net/winfsp/).
3630 WinFsp (https://github.com/winfsp/winfsp) is an open-source Windows
3632 File System Proxy which makes it easy to write user space file systems
3633 for Windows. It provides a FUSE emulation layer which rclone uses com‐
3634 bination with cgofuse (https://github.com/winfsp/cgofuse). Both of
3635 these packages are by Bill Zissimopoulos who was very helpful during
3636 the implementation of rclone mount for Windows.
3637 Mounting modes on windows
3639 Unlike other operating systems, Microsoft Windows provides a different
3640 filesystem type for network and fixed drives. It optimises access on
3641 the assumption fixed disk drives are fast and reliable, while network
3642 drives have relatively high latency and less reliability. Some set‐
3643 tings can also be differentiated between the two types, for example
3644 that Windows Explorer should just display icons and not create preview
3645 thumbnails for image and video files on network drives.
3646 In most cases, rclone will mount the remote as a normal, fixed disk
3648 drive by default. However, you can also choose to mount it as a remote
3649 network drive, often described as a network share. If you mount an
3650 rclone remote using the default, fixed drive mode and experience unex‐
3651 pected program errors, freezes or other issues, consider mounting as a
3652 network drive instead.
3653 When mounting as a fixed disk drive you can either mount to an unused
3655 drive letter, or to a path representing a nonexistent subdirectory of
3656 an existing parent directory or drive. Using the special value * will
3657 tell rclone to automatically assign the next available drive letter,
3658 starting with Z: and moving backward. Examples:
3659 rclone mount remote:path/to/files *
3661 rclone mount remote:path/to/files X:
3662 rclone mount remote:path/to/files C:\path\parent\mount
3663 rclone mount remote:path/to/files X:
3664 Option --volname can be used to set a custom volume name for the mount‐
3666 ed file system. The default is to use the remote name and path.
3667 To mount as network drive, you can add option --network-mode to your
3669 mount command. Mounting to a directory path is not supported in this
3670 mode, it is a limitation Windows imposes on junctions, so the remote
3671 must always be mounted to a drive letter.
3672 rclone mount remote:path/to/files X: --network-mode
3674 A volume name specified with --volname will be used to create the net‐
3676 work share path. A complete UNC path, such as \\cloud\remote, option‐
3677 ally with path \\cloud\remote\madeup\path, will be used as is. Any
3678 other string will be used as the share part, after a default prefix
3679 \\server\. If no volume name is specified then \\server\share will be
3680 used. You must make sure the volume name is unique when you are mount‐
3681 ing more than one drive, or else the mount command will fail. The
3682 share name will treated as the volume label for the mapped drive, shown
3683 in Windows Explorer etc, while the complete \\server\share will be re‐
3684 ported as the remote UNC path by net use etc, just like a normal net‐
3685 work drive mapping.
3686 If you specify a full network share UNC path with --volname, this will
3688 implicitly set the --network-mode option, so the following two examples
3689 have same result:
3690 rclone mount remote:path/to/files X: --network-mode
3692 rclone mount remote:path/to/files X: --volname \\server\share
3693 You may also specify the network share UNC path as the mountpoint it‐
3695 self. Then rclone will automatically assign a drive letter, same as
3696 with * and use that as mountpoint, and instead use the UNC path speci‐
3697 fied as the volume name, as if it were specified with the --volname op‐
3698 tion. This will also implicitly set the --network-mode option. This
3699 means the following two examples have same result:
3700 rclone mount remote:path/to/files \\cloud\remote
3702 rclone mount remote:path/to/files * --volname \\cloud\remote
3703 There is yet another way to enable network mode, and to set the share
3705 path, and that is to pass the "native" libfuse/WinFsp option directly:
3706 --fuse-flag --VolumePrefix=\server\share. Note that the path must be
3707 with just a single backslash prefix in this case.
3708 Note: In previous versions of rclone this was the only supported meth‐
3710 od.
3711 Read more about drive mapping (https://en.wikipedia.org/wiki/Drive_map‐
3713 ping)
3714 See also Limitations section below.
3716 Windows filesystem permissions
3718 The FUSE emulation layer on Windows must convert between the POSIX-
3719 based permission model used in FUSE, and the permission model used in
3720 Windows, based on access-control lists (ACL).
3721 The mounted filesystem will normally get three entries in its access-
3723 control list (ACL), representing permissions for the POSIX permission
3724 scopes: Owner, group and others. By default, the owner and group will
3725 be taken from the current user, and the built-in group "Everyone" will
3726 be used to represent others. The user/group can be customized with
3727 FUSE options "UserName" and "GroupName", e.g. -o UserName=user123 -o
3728 GroupName="Authenticated Users". The permissions on each entry will be
3729 set according to options --dir-perms and --file-perms, which takes a
3730 value in traditional numeric notation (https://en.wikipedia.org/wi‐
3731 ki/File-system_permissions#Numeric_notation).
3732 The default permissions corresponds to --file-perms 0666 --dir-perms
3734 0777, i.e. read and write permissions to everyone. This means you
3735 will not be able to start any programs from the mount. To be able to
3736 do that you must add execute permissions, e.g. --file-perms 0777
3737 --dir-perms 0777 to add it to everyone. If the program needs to write
3738 files, chances are you will have to enable VFS File Caching as well
3739 (see also limitations).
3740 Note that the mapping of permissions is not always trivial, and the re‐
3742 sult you see in Windows Explorer may not be exactly like you expected.
3743 For example, when setting a value that includes write access, this will
3744 be mapped to individual permissions "write attributes", "write data"
3745 and "append data", but not "write extended attributes". Windows will
3746 then show this as basic permission "Special" instead of "Write", be‐
3747 cause "Write" includes the "write extended attributes" permission.
3748 If you set POSIX permissions for only allowing access to the owner, us‐
3750 ing --file-perms 0600 --dir-perms 0700, the user group and the built-in
3751 "Everyone" group will still be given some special permissions, such as
3752 "read attributes" and "read permissions", in Windows. This is done for
3753 compatibility reasons, e.g. to allow users without additional permis‐
3754 sions to be able to read basic metadata about files like in UNIX. One
3755 case that may arise is that other programs (incorrectly) interprets
3756 this as the file being accessible by everyone. For example an SSH
3757 client may warn about "unprotected private key file".
3758 WinFsp 2021 (version 1.9) introduces a new FUSE option "FileSecurity",
3760 that allows the complete specification of file security descriptors us‐
3761 ing SDDL (https://docs.microsoft.com/en-us/windows/win32/secauthz/secu‐
3762 rity-descriptor-string-format). With this you can work around issues
3763 such as the mentioned "unprotected private key file" by specifying -o
3764 FileSecurity="D:P(A;;FA;;;OW)", for file all access (FA) to the owner
3765 (OW).
3766 Windows caveats
3768 Drives created as Administrator are not visible to other accounts, not
3769 even an account that was elevated to Administrator with the User Ac‐
3770 count Control (UAC) feature. A result of this is that if you mount to
3771 a drive letter from a Command Prompt run as Administrator, and then try
3772 to access the same drive from Windows Explorer (which does not run as
3773 Administrator), you will not be able to see the mounted drive.
3774 If you don't need to access the drive from applications running with
3776 administrative privileges, the easiest way around this is to always
3777 create the mount from a non-elevated command prompt.
3778 To make mapped drives available to the user account that created them
3780 regardless if elevated or not, there is a special Windows setting
3781 called linked connections (https://docs.microsoft.com/en-us/trou‐
3782 bleshoot/windows-client/networking/mapped-drives-not-available-from-el‐
3783 evated-command#detail-to-configure-the-enablelinkedconnections-reg‐
3784 istry-entry) that can be enabled.
3785 It is also possible to make a drive mount available to everyone on the
3787 system, by running the process creating it as the built-in SYSTEM ac‐
3788 count. There are several ways to do this: One is to use the command-
3789 line utility PsExec (https://docs.microsoft.com/en-us/sysinter‐
3790 nals/downloads/psexec), from Microsoft's Sysinternals suite, which has
3791 option -s to start processes as the SYSTEM account. Another alterna‐
3792 tive is to run the mount command from a Windows Scheduled Task, or a
3793 Windows Service, configured to run as the SYSTEM account. A third al‐
3794 ternative is to use the WinFsp.Launcher infrastructure
3795 (https://github.com/winfsp/winfsp/wiki/WinFsp-Service-Architecture)).
3796 Note that when running rclone as another user, it will not use the con‐
3797 figuration file from your profile unless you tell it to with the --con‐
3798 fig (https://rclone.org/docs/#config-config-file) option. Read more in
3799 the install documentation (https://rclone.org/install/).
3800 Note that mapping to a directory path, instead of a drive letter, does
3802 not suffer from the same limitations.
3803 Limitations
3805 Without the use of --vfs-cache-mode this can only write files sequen‐
3806 tially, it can only seek when reading. This means that many applica‐
3807 tions won't work with their files on an rclone mount without --vfs-
3808 cache-mode writes or --vfs-cache-mode full. See the VFS File Caching
3809 section for more info.
3810 The bucket-based remotes (e.g. Swift, S3, Google Compute Storage, B2)
3812 do not support the concept of empty directories, so empty directories
3813 will have a tendency to disappear once they fall out of the directory
3814 cache.
3815 When rclone mount is invoked on Unix with --daemon flag, the main
3817 rclone program will wait for the background mount to become ready or
3818 until the timeout specified by the --daemon-wait flag. On Linux it can
3819 check mount status using ProcFS so the flag in fact sets maximum time
3820 to wait, while the real wait can be less. On macOS / BSD the time to
3821 wait is constant and the check is performed only at the end. We advise
3822 you to set wait time on macOS reasonably.
3823 Only supported on Linux, FreeBSD, OS X and Windows at the moment.
3825 rclone mount vs rclone sync/copy
3827 File systems expect things to be 100% reliable, whereas cloud storage
3828 systems are a long way from 100% reliable. The rclone sync/copy com‐
3829 mands cope with this with lots of retries. However rclone mount can't
3830 use retries in the same way without making local copies of the uploads.
3831 Look at the VFS File Caching for solutions to make mount more reliable.
3832 Attribute caching
3834 You can use the flag --attr-timeout to set the time the kernel caches
3835 the attributes (size, modification time, etc.) for directory entries.
3836 The default is 1s which caches files just long enough to avoid too many
3838 callbacks to rclone from the kernel.
3839 In theory 0s should be the correct value for filesystems which can
3841 change outside the control of the kernel. However this causes quite a
3842 few problems such as rclone using too much memory
3843 (https://github.com/rclone/rclone/issues/2157), rclone not serving
3844 files to samba (https://forum.rclone.org/t/rclone-1-39-vs-1-40-mount-
3845 issue/5112) and excessive time listing directories
3846 (https://github.com/rclone/rclone/issues/2095#issuecomment-371141147).
3847 The kernel can cache the info about a file for the time given by --at‐
3849 tr-timeout. You may see corruption if the remote file changes length
3850 during this window. It will show up as either a truncated file or a
3851 file with garbage on the end. With --attr-timeout 1s this is very un‐
3852 likely but not impossible. The higher you set --attr-timeout the more
3853 likely it is. The default setting of "1s" is the lowest setting which
3854 mitigates the problems above.
3855 If you set it higher (10s or 1m say) then the kernel will call back to
3857 rclone less often making it more efficient, however there is more
3858 chance of the corruption issue above.
3859 If files don't change on the remote outside of the control of rclone
3861 then there is no chance of corruption.
3862 This is the same as setting the attr_timeout option in mount.fuse.
3864 Filters
3866 Note that all the rclone filters can be used to select a subset of the
3867 files to be visible in the mount.
3868 systemd
3870 When running rclone mount as a systemd service, it is possible to use
3871 Type=notify. In this case the service will enter the started state af‐
3872 ter the mountpoint has been successfully set up. Units having the
3873 rclone mount service specified as a requirement will see all files and
3874 folders immediately in this mode.
3875 Note that systemd runs mount units without any environment variables
3877 including PATH or HOME. This means that tilde (~) expansion will not
3878 work and you should provide --config and --cache-dir explicitly as ab‐
3879 solute paths via rclone arguments. Since mounting requires the fuser‐
3880 mount program, rclone will use the fallback PATH of /bin:/usr/bin in
3881 this scenario. Please ensure that fusermount is present on this PATH.
3882 Rclone as Unix mount helper
3884 The core Unix program /bin/mount normally takes the -t FSTYPE argument
3885 then runs the /sbin/mount.FSTYPE helper program passing it mount op‐
3886 tions as -o key=val,... or --opt=.... Automount (classic or systemd)
3887 behaves in a similar way.
3888 rclone by default expects GNU-style flags --key val. To run it as a
3890 mount helper you should symlink rclone binary to /sbin/mount.rclone and
3891 optionally /usr/bin/rclonefs, e.g. ln -s /usr/bin/rclone
3892 /sbin/mount.rclone. rclone will detect it and translate command-line
3893 arguments appropriately.
3894 Now you can run classic mounts like this:
3896 mount sftp1:subdir /mnt/data -t rclone -o vfs_cache_mode=writes,sftp_key_file=/path/to/pem
3898 or create systemd mount units:
3900 # /etc/systemd/system/mnt-data.mount
3902 [Unit]
3903 After=network-online.target
3904 [Mount]
3905 Type=rclone
3906 What=sftp1:subdir
3907 Where=/mnt/data
3908 Options=rw,allow_other,args2env,vfs-cache-mode=writes,config=/etc/rclone.conf,cache-dir=/var/rclone
3909 optionally accompanied by systemd automount unit
3911 # /etc/systemd/system/mnt-data.automount
3913 [Unit]
3914 After=network-online.target
3915 Before=remote-fs.target
3916 [Automount]
3917 Where=/mnt/data
3918 TimeoutIdleSec=600
3919 [Install]
3920 WantedBy=multi-user.target
3921 or add in /etc/fstab a line like
3923 sftp1:subdir /mnt/data rclone rw,noauto,nofail,_netdev,x-systemd.automount,args2env,vfs_cache_mode=writes,config=/etc/rclone.conf,cache_dir=/var/cache/rclone 0 0
3925 or use classic Automountd. Remember to provide explicit con‐
3927 fig=...,cache-dir=... as a workaround for mount units being run without
3928 HOME.
3929 Rclone in the mount helper mode will split -o argument(s) by comma, re‐
3931 place _ by - and prepend -- to get the command-line flags. Options
3932 containing commas or spaces can be wrapped in single or double quotes.
3933 Any inner quotes inside outer quotes of the same type should be dou‐
3934 bled.
3935 Mount option syntax includes a few extra options treated specially:
3937 • env.NAME=VALUE will set an environment variable for the mount
3939 process. This helps with Automountd and Systemd.mount which don't
3940 allow setting custom environment for mount helpers. Typically you
3941 will use env.HTTPS_PROXY=proxy.host:3128 or env.HOME=/root
3942 • command=cmount can be used to run cmount or any other rclone command
3944 rather than the default mount.
3945 • args2env will pass mount options to the mount helper running in back‐
3947 ground via environment variables instead of command line arguments.
3948 This allows to hide secrets from such commands as ps or pgrep.
3949 • vv... will be transformed into appropriate --verbose=N
3951 • standard mount options like x-systemd.automount, _netdev, nosuid and
3953 alike are intended only for Automountd and ignored by rclone.
3954 VFS - Virtual File System
3956 This command uses the VFS layer. This adapts the cloud storage objects
3957 that rclone uses into something which looks much more like a disk fil‐
3958 ing system.
3959 Cloud storage objects have lots of properties which aren't like disk
3961 files - you can't extend them or write to the middle of them, so the
3962 VFS layer has to deal with that. Because there is no one right way of
3963 doing this there are various options explained below.
3964 The VFS layer also implements a directory cache - this caches info
3966 about files and directories (but not the data) in memory.
3967 VFS Directory Cache
3969 Using the --dir-cache-time flag, you can control how long a directory
3970 should be considered up to date and not refreshed from the backend.
3971 Changes made through the VFS will appear immediately or invalidate the
3972 cache.
3973 --dir-cache-time duration Time to cache directory entries for (default 5m0s)
3975 --poll-interval duration Time to wait between polling for changes. Must be smaller than dir-cache-time. Only on supported remotes. Set to 0 to disable (default 1m0s)
3976 However, changes made directly on the cloud storage by the web inter‐
3978 face or a different copy of rclone will only be picked up once the di‐
3979 rectory cache expires if the backend configured does not support
3980 polling for changes. If the backend supports polling, changes will be
3981 picked up within the polling interval.
3982 You can send a SIGHUP signal to rclone for it to flush all directory
3984 caches, regardless of how old they are. Assuming only one rclone in‐
3985 stance is running, you can reset the cache like this:
3986 kill -SIGHUP $(pidof rclone)
3988 If you configure rclone with a remote control then you can use rclone
3990 rc to flush the whole directory cache:
3991 rclone rc vfs/forget
3993 Or individual files or directories:
3995 rclone rc vfs/forget file=path/to/file dir=path/to/dir
3997 VFS File Buffering
3999 The --buffer-size flag determines the amount of memory, that will be
4000 used to buffer data in advance.
4001 Each open file will try to keep the specified amount of data in memory
4003 at all times. The buffered data is bound to one open file and won't be
4004 shared.
4005 This flag is a upper limit for the used memory per open file. The buf‐
4007 fer will only use memory for data that is downloaded but not not yet
4008 read. If the buffer is empty, only a small amount of memory will be
4009 used.
4010 The maximum memory used by rclone for buffering can be up to --buffer-
4012 size * open files.
4013 VFS File Caching
4015 These flags control the VFS file caching options. File caching is nec‐
4016 essary to make the VFS layer appear compatible with a normal file sys‐
4017 tem. It can be disabled at the cost of some compatibility.
4018 For example you'll need to enable VFS caching if you want to read and
4020 write simultaneously to a file. See below for more details.
4021 Note that the VFS cache is separate from the cache backend and you may
4023 find that you need one or the other or both.
4024 --cache-dir string Directory rclone will use for caching.
4026 --vfs-cache-mode CacheMode Cache mode off|minimal|writes|full (default off)
4027 --vfs-cache-max-age duration Max age of objects in the cache (default 1h0m0s)
4028 --vfs-cache-max-size SizeSuffix Max total size of objects in the cache (default off)
4029 --vfs-cache-poll-interval duration Interval to poll the cache for stale objects (default 1m0s)
4030 --vfs-write-back duration Time to writeback files after last use when using cache (default 5s)
4031 If run with -vv rclone will print the location of the file cache. The
4033 files are stored in the user cache file area which is OS dependent but
4034 can be controlled with --cache-dir or setting the appropriate environ‐
4035 ment variable.
4036 The cache has 4 different modes selected by --vfs-cache-mode. The
4038 higher the cache mode the more compatible rclone becomes at the cost of
4039 using disk space.
4040 Note that files are written back to the remote only when they are
4042 closed and if they haven't been accessed for --vfs-write-back seconds.
4043 If rclone is quit or dies with files that haven't been uploaded, these
4044 will be uploaded next time rclone is run with the same flags.
4045 If using --vfs-cache-max-size note that the cache may exceed this size
4047 for two reasons. Firstly because it is only checked every --vfs-cache-
4048 poll-interval. Secondly because open files cannot be evicted from the
4049 cache.
4050 You should not run two copies of rclone using the same VFS cache with
4052 the same or overlapping remotes if using --vfs-cache-mode > off. This
4053 can potentially cause data corruption if you do. You can work around
4054 this by giving each rclone its own cache hierarchy with --cache-dir.
4055 You don't need to worry about this if the remotes in use don't overlap.
4056 --vfs-cache-mode off
4058 In this mode (the default) the cache will read directly from the remote
4059 and write directly to the remote without caching anything on disk.
4060 This will mean some operations are not possible
4062 • Files can't be opened for both read AND write
4064 • Files opened for write can't be seeked
4066 • Existing files opened for write must have O_TRUNC set
4068 • Files open for read with O_TRUNC will be opened write only
4070 • Files open for write only will behave as if O_TRUNC was supplied
4072 • Open modes O_APPEND, O_TRUNC are ignored
4074 • If an upload fails it can't be retried
4076 --vfs-cache-mode minimal
4078 This is very similar to "off" except that files opened for read AND
4079 write will be buffered to disk. This means that files opened for write
4080 will be a lot more compatible, but uses the minimal disk space.
4081 These operations are not possible
4083 • Files opened for write only can't be seeked
4085 • Existing files opened for write must have O_TRUNC set
4087 • Files opened for write only will ignore O_APPEND, O_TRUNC
4089 • If an upload fails it can't be retried
4091 --vfs-cache-mode writes
4093 In this mode files opened for read only are still read directly from
4094 the remote, write only and read/write files are buffered to disk first.
4095 This mode should support all normal file system operations.
4097 If an upload fails it will be retried at exponentially increasing in‐
4099 tervals up to 1 minute.
4100 --vfs-cache-mode full
4102 In this mode all reads and writes are buffered to and from disk. When
4103 data is read from the remote this is buffered to disk as well.
4104 In this mode the files in the cache will be sparse files and rclone
4106 will keep track of which bits of the files it has downloaded.
4107 So if an application only reads the starts of each file, then rclone
4109 will only buffer the start of the file. These files will appear to be
4110 their full size in the cache, but they will be sparse files with only
4111 the data that has been downloaded present in them.
4112 This mode should support all normal file system operations and is oth‐
4114 erwise identical to --vfs-cache-mode writes.
4115 When reading a file rclone will read --buffer-size plus --vfs-read-
4117 ahead bytes ahead. The --buffer-size is buffered in memory whereas the
4118 --vfs-read-ahead is buffered on disk.
4119 When using this mode it is recommended that --buffer-size is not set
4121 too large and --vfs-read-ahead is set large if required.
4122 IMPORTANT not all file systems support sparse files. In particular
4124 FAT/exFAT do not. Rclone will perform very badly if the cache directo‐
4125 ry is on a filesystem which doesn't support sparse files and it will
4126 log an ERROR message if one is detected.
4127 Fingerprinting
4129 Various parts of the VFS use fingerprinting to see if a local file copy
4130 has changed relative to a remote file. Fingerprints are made from:
4131 • size
4133 • modification time
4135 • hash
4137 where available on an object.
4139 On some backends some of these attributes are slow to read (they take
4141 an extra API call per object, or extra work per object).
4142 For example hash is slow with the local and sftp backends as they have
4144 to read the entire file and hash it, and modtime is slow with the s3,
4145 swift, ftp and qinqstor backends because they need to do an extra API
4146 call to fetch it.
4147 If you use the --vfs-fast-fingerprint flag then rclone will not include
4149 the slow operations in the fingerprint. This makes the fingerprinting
4150 less accurate but much faster and will improve the opening time of
4151 cached files.
4152 If you are running a vfs cache over local, s3 or swift backends then
4154 using this flag is recommended.
4155 Note that if you change the value of this flag, the fingerprints of the
4157 files in the cache may be invalidated and the files will need to be
4158 downloaded again.
4159 VFS Chunked Reading
4161 When rclone reads files from a remote it reads them in chunks. This
4162 means that rather than requesting the whole file rclone reads the chunk
4163 specified. This can reduce the used download quota for some remotes by
4164 requesting only chunks from the remote that are actually read, at the
4165 cost of an increased number of requests.
4166 These flags control the chunking:
4168 --vfs-read-chunk-size SizeSuffix Read the source objects in chunks (default 128M)
4170 --vfs-read-chunk-size-limit SizeSuffix Max chunk doubling size (default off)
4171 Rclone will start reading a chunk of size --vfs-read-chunk-size, and
4173 then double the size for each read. When --vfs-read-chunk-size-limit
4174 is specified, and greater than --vfs-read-chunk-size, the chunk size
4175 for each open file will get doubled only until the specified value is
4176 reached. If the value is "off", which is the default, the limit is
4177 disabled and the chunk size will grow indefinitely.
4178 With --vfs-read-chunk-size 100M and --vfs-read-chunk-size-limit 0 the
4180 following parts will be downloaded: 0-100M, 100M-200M, 200M-300M,
4181 300M-400M and so on. When --vfs-read-chunk-size-limit 500M is speci‐
4182 fied, the result would be 0-100M, 100M-300M, 300M-700M, 700M-1200M,
4183 1200M-1700M and so on.
4184 Setting --vfs-read-chunk-size to 0 or "off" disables chunked reading.
4186 VFS Performance
4188 These flags may be used to enable/disable features of the VFS for per‐
4189 formance or other reasons. See also the chunked reading feature.
4190 In particular S3 and Swift benefit hugely from the --no-modtime flag
4192 (or use --use-server-modtime for a slightly different effect) as each
4193 read of the modification time takes a transaction.
4194 --no-checksum Don't compare checksums on up/download.
4196 --no-modtime Don't read/write the modification time (can speed things up).
4197 --no-seek Don't allow seeking in files.
4198 --read-only Only allow read-only access.
4199 Sometimes rclone is delivered reads or writes out of order. Rather
4201 than seeking rclone will wait a short time for the in sequence read or
4202 write to come in. These flags only come into effect when not using an
4203 on disk cache file.
4204 --vfs-read-wait duration Time to wait for in-sequence read before seeking (default 20ms)
4206 --vfs-write-wait duration Time to wait for in-sequence write before giving error (default 1s)
4207 When using VFS write caching (--vfs-cache-mode with value writes or
4209 full), the global flag --transfers can be set to adjust the number of
4210 parallel uploads of modified files from the cache (the related global
4211 flag --checkers has no effect on the VFS).
4212 --transfers int Number of file transfers to run in parallel (default 4)
4214 VFS Case Sensitivity
4216 Linux file systems are case-sensitive: two files can differ only by
4217 case, and the exact case must be used when opening a file.
4218 File systems in modern Windows are case-insensitive but case-preserv‐
4220 ing: although existing files can be opened using any case, the exact
4221 case used to create the file is preserved and available for programs to
4222 query. It is not allowed for two files in the same directory to differ
4223 only by case.
4224 Usually file systems on macOS are case-insensitive. It is possible to
4226 make macOS file systems case-sensitive but that is not the default.
4227 The --vfs-case-insensitive VFS flag controls how rclone handles these
4229 two cases. If its value is "false", rclone passes file names to the
4230 remote as-is. If the flag is "true" (or appears without a value on the
4231 command line), rclone may perform a "fixup" as explained below.
4232 The user may specify a file name to open/delete/rename/etc with a case
4234 different than what is stored on the remote. If an argument refers to
4235 an existing file with exactly the same name, then the case of the ex‐
4236 isting file on the disk will be used. However, if a file name with ex‐
4237 actly the same name is not found but a name differing only by case ex‐
4238 ists, rclone will transparently fixup the name. This fixup happens on‐
4239 ly when an existing file is requested. Case sensitivity of file names
4240 created anew by rclone is controlled by the underlying remote.
4241 Note that case sensitivity of the operating system running rclone (the
4243 target) may differ from case sensitivity of a file system presented by
4244 rclone (the source). The flag controls whether "fixup" is performed to
4245 satisfy the target.
4246 If the flag is not provided on the command line, then its default value
4248 depends on the operating system where rclone runs: "true" on Windows
4249 and macOS, "false" otherwise. If the flag is provided without a value,
4250 then it is "true".
4251 VFS Disk Options
4253 This flag allows you to manually set the statistics about the filing
4254 system. It can be useful when those statistics cannot be read correct‐
4255 ly automatically.
4256 --vfs-disk-space-total-size Manually set the total disk space size (example: 256G, default: -1)
4258 Alternate report of used bytes
4260 Some backends, most notably S3, do not report the amount of bytes used.
4261 If you need this information to be available when running df on the
4262 filesystem, then pass the flag --vfs-used-is-size to rclone. With this
4263 flag set, instead of relying on the backend to report this information,
4264 rclone will scan the whole remote similar to rclone size and compute
4265 the total used space itself.
4266 WARNING. Contrary to rclone size, this flag ignores filters so that the
4268 result is accurate. However, this is very inefficient and may cost
4269 lots of API calls resulting in extra charges. Use it as a last resort
4270 and only with caching.
4271 rclone mount remote:path /path/to/mountpoint [flags]
4273 Options
4275 --allow-non-empty Allow mounting over a non-empty directory (not supported on Windows)
4276 --allow-other Allow access to other users (not supported on Windows)
4277 --allow-root Allow access to root user (not supported on Windows)
4278 --async-read Use asynchronous reads (not supported on Windows) (default true)
4279 --attr-timeout duration Time for which file/directory attributes are cached (default 1s)
4280 --daemon Run mount in background and exit parent process (as background output is suppressed, use --log-file with --log-format=pid,... to monitor) (not supported on Windows)
4281 --daemon-timeout duration Time limit for rclone to respond to kernel (not supported on Windows)
4282 --daemon-wait duration Time to wait for ready mount from daemon (maximum time on Linux, constant sleep time on OSX/BSD) (not supported on Windows) (default 1m0s)
4283 --debug-fuse Debug the FUSE internals - needs -v
4284 --default-permissions Makes kernel enforce access control based on the file mode (not supported on Windows)
4285 --devname string Set the device name - default is remote:path
4286 --dir-cache-time duration Time to cache directory entries for (default 5m0s)
4287 --dir-perms FileMode Directory permissions (default 0777)
4288 --file-perms FileMode File permissions (default 0666)
4289 --fuse-flag stringArray Flags or arguments to be passed direct to libfuse/WinFsp (repeat if required)
4290 --gid uint32 Override the gid field set by the filesystem (not supported on Windows) (default 1000)
4291 -h, --help help for mount
4292 --max-read-ahead SizeSuffix The number of bytes that can be prefetched for sequential reads (not supported on Windows) (default 128Ki)
4293 --network-mode Mount as remote network drive, instead of fixed disk drive (supported on Windows only)
4294 --no-checksum Don't compare checksums on up/download
4295 --no-modtime Don't read/write the modification time (can speed things up)
4296 --no-seek Don't allow seeking in files
4297 --noappledouble Ignore Apple Double (._) and .DS_Store files (supported on OSX only) (default true)
4298 --noapplexattr Ignore all "com.apple.*" extended attributes (supported on OSX only)
4299 -o, --option stringArray Option for libfuse/WinFsp (repeat if required)
4300 --poll-interval duration Time to wait between polling for changes, must be smaller than dir-cache-time and only on supported remotes (set 0 to disable) (default 1m0s)
4301 --read-only Only allow read-only access
4302 --uid uint32 Override the uid field set by the filesystem (not supported on Windows) (default 1000)
4303 --umask int Override the permission bits set by the filesystem (not supported on Windows) (default 2)
4304 --vfs-cache-max-age duration Max age of objects in the cache (default 1h0m0s)
4305 --vfs-cache-max-size SizeSuffix Max total size of objects in the cache (default off)
4306 --vfs-cache-mode CacheMode Cache mode off|minimal|writes|full (default off)
4307 --vfs-cache-poll-interval duration Interval to poll the cache for stale objects (default 1m0s)
4308 --vfs-case-insensitive If a file name not found, find a case insensitive match
4309 --vfs-disk-space-total-size SizeSuffix Specify the total space of disk (default off)
4310 --vfs-fast-fingerprint Use fast (less accurate) fingerprints for change detection
4311 --vfs-read-ahead SizeSuffix Extra read ahead over --buffer-size when using cache-mode full
4312 --vfs-read-chunk-size SizeSuffix Read the source objects in chunks (default 128Mi)
4313 --vfs-read-chunk-size-limit SizeSuffix If greater than --vfs-read-chunk-size, double the chunk size after each chunk read, until the limit is reached ('off' is unlimited) (default off)
4314 --vfs-read-wait duration Time to wait for in-sequence read before seeking (default 20ms)
4315 --vfs-used-is-size rclone size Use the rclone size algorithm for Used size
4316 --vfs-write-back duration Time to writeback files after last use when using cache (default 5s)
4317 --vfs-write-wait duration Time to wait for in-sequence write before giving error (default 1s)
4318 --volname string Set the volume name (supported on Windows and OSX only)
4319 --write-back-cache Makes kernel buffer writes before sending them to rclone (without this, writethrough caching is used) (not supported on Windows)
4320 See the global flags page (https://rclone.org/flags/) for global op‐
4322 tions not listed here.
4323 SEE ALSO
4325 • rclone (https://rclone.org/commands/rclone/) - Show help for rclone
4326 commands, flags and backends.
4327
4329 Move file or directory from source to dest.
4330 Synopsis
4332 If source:path is a file or directory then it moves it to a file or di‐
4333 rectory named dest:path.
4334 This can be used to rename files or upload single files to other than
4336 their existing name. If the source is a directory then it acts exactly
4337 like the move (https://rclone.org/commands/rclone_move/) command.
4338 So
4340 rclone moveto src dst
4342 where src and dst are rclone paths, either remote:path or /path/to/lo‐
4344 cal or C:.
4345 This will:
4347 if src is file
4349 move it to dst, overwriting an existing file if it exists
4350 if src is directory
4351 move it to dst, overwriting existing files if they exist
4352 see move command for full details
4353 This doesn't transfer files that are identical on src and dst, testing
4355 by size and modification time or MD5SUM. src will be deleted on suc‐
4356 cessful transfer.
4357 Important: Since this can cause data loss, test first with the --dry-
4359 run or the --interactive/-i flag.
4360 Note: Use the -P/--progress flag to view real-time transfer statistics.
4362 rclone moveto source:path dest:path [flags]
4364 Options
4366 -h, --help help for moveto
4367 See the global flags page (https://rclone.org/flags/) for global op‐
4369 tions not listed here.
4370 SEE ALSO
4372 • rclone (https://rclone.org/commands/rclone/) - Show help for rclone
4373 commands, flags and backends.
4374
4376 Explore a remote with a text based user interface.
4377 Synopsis
4379 This displays a text based user interface allowing the navigation of a
4380 remote. It is most useful for answering the question - "What is using
4381 all my disk space?".
4382 To make the user interface it first scans the entire remote given and
4384 builds an in memory representation. rclone ncdu can be used during
4385 this scanning phase and you will see it building up the directory
4386 structure as it goes along.
4387 You can interact with the user interface using key presses, press '?'
4389 to toggle the help on and off. The supported keys are:
4390 ↑,↓ or k,j to Move
4392 →,l to enter
4393 ←,h to return
4394 c toggle counts
4395 g toggle graph
4396 a toggle average size in directory
4397 u toggle human-readable format
4398 n,s,C,A sort by name,size,count,average size
4399 d delete file/directory
4400 v select file/directory
4401 V enter visual select mode
4402 D delete selected files/directories
4403 y copy current path to clipboard
4404 Y display current path
4405 ^L refresh screen (fix screen corruption)
4406 ? to toggle help on and off
4407 q/ESC/^c to quit
4408 Listed files/directories may be prefixed by a one-character flag, some
4410 of them combined with a description in brackets at end of line. These
4411 flags have the following meaning:
4412 e means this is an empty directory, i.e. contains no files (but
4414 may contain empty subdirectories)
4415 ~ means this is a directory where some of the files (possibly in
4416 subdirectories) have unknown size, and therefore the directory
4417 size may be underestimated (and average size inaccurate, as it
4418 is average of the files with known sizes).
4419 . means an error occurred while reading a subdirectory, and
4420 therefore the directory size may be underestimated (and average
4421 size inaccurate)
4422 ! means an error occurred while reading this directory
4423 This an homage to the ncdu tool (https://dev.yorhel.nl/ncdu) but for
4425 rclone remotes. It is missing lots of features at the moment but is
4426 useful as it stands.
4427 Note that it might take some time to delete big files/directories. The
4429 UI won't respond in the meantime since the deletion is done syn‐
4430 chronously.
4431 For a non-interactive listing of the remote, see the tree
4433 (https://rclone.org/commands/rclone_tree/) command. To just get the
4434 total size of the remote you can also use the size
4435 (https://rclone.org/commands/rclone_size/) command.
4436 rclone ncdu remote:path [flags]
4438 Options
4440 -h, --help help for ncdu
4441 See the global flags page (https://rclone.org/flags/) for global op‐
4443 tions not listed here.
4444 SEE ALSO
4446 • rclone (https://rclone.org/commands/rclone/) - Show help for rclone
4447 commands, flags and backends.
4448
4450 Obscure password for use in the rclone config file.
4451 Synopsis
4453 In the rclone config file, human-readable passwords are obscured. Ob‐
4454 scuring them is done by encrypting them and writing them out in base64.
4455 This is not a secure way of encrypting these passwords as rclone can
4456 decrypt them - it is to prevent "eyedropping" - namely someone seeing a
4457 password in the rclone config file by accident.
4458 Many equally important things (like access tokens) are not obscured in
4460 the config file. However it is very hard to shoulder surf a 64 charac‐
4461 ter hex token.
4462 This command can also accept a password through STDIN instead of an ar‐
4464 gument by passing a hyphen as an argument. This will use the first
4465 line of STDIN as the password not including the trailing newline.
4466 echo "secretpassword" | rclone obscure -
4468 If there is no data on STDIN to read, rclone obscure will default to
4470 obfuscating the hyphen itself.
4471 If you want to encrypt the config file then please use config file en‐
4473 cryption - see rclone config (https://rclone.org/commands/rclone_con‐
4474 fig/) for more info.
4475 rclone obscure password [flags]
4477 Options
4479 -h, --help help for obscure
4480 See the global flags page (https://rclone.org/flags/) for global op‐
4482 tions not listed here.
4483 SEE ALSO
4485 • rclone (https://rclone.org/commands/rclone/) - Show help for rclone
4486 commands, flags and backends.
4487
4489 Run a command against a running rclone.
4490 Synopsis
4492 This runs a command against a running rclone. Use the --url flag to
4493 specify an non default URL to connect on. This can be either a ":port"
4494 which is taken to mean "http://localhost:port" or a "host:port" which
4495 is taken to mean "http://host:port"
4496 A username and password can be passed in with --user and --pass.
4498 Note that --rc-addr, --rc-user, --rc-pass will be read also for --url,
4500 --user, --pass.
4501 Arguments should be passed in as parameter=value.
4503 The result will be returned as a JSON object by default.
4505 The --json parameter can be used to pass in a JSON blob as an input in‐
4507 stead of key=value arguments. This is the only way of passing in more
4508 complicated values.
4509 The -o/--opt option can be used to set a key "opt" with key, value op‐
4511 tions in the form -o key=value or -o key. It can be repeated as many
4512 times as required. This is useful for rc commands which take the "opt"
4513 parameter which by convention is a dictionary of strings.
4514 -o key=value -o key2
4516 Will place this in the "opt" value
4518 {"key":"value", "key2","")
4520 The -a/--arg option can be used to set strings in the "arg" value. It
4522 can be repeated as many times as required. This is useful for rc com‐
4523 mands which take the "arg" parameter which by convention is a list of
4524 strings.
4525 -a value -a value2
4527 Will place this in the "arg" value
4529 ["value", "value2"]
4531 Use --loopback to connect to the rclone instance running rclone rc.
4533 This is very useful for testing commands without having to run an
4534 rclone rc server, e.g.:
4535 rclone rc --loopback operations/about fs=/
4537 Use rclone rc to see a list of all possible commands.
4539 rclone rc commands parameter [flags]
4541 Options
4543 -a, --arg stringArray Argument placed in the "arg" array
4544 -h, --help help for rc
4545 --json string Input JSON - use instead of key=value args
4546 --loopback If set connect to this rclone instance not via HTTP
4547 --no-output If set, don't output the JSON result
4548 -o, --opt stringArray Option in the form name=value or name placed in the "opt" array
4549 --pass string Password to use to connect to rclone remote control
4550 --url string URL to connect to rclone remote control (default "http://localhost:5572/")
4551 --user string Username to use to rclone remote control
4552 See the global flags page (https://rclone.org/flags/) for global op‐
4554 tions not listed here.
4555 SEE ALSO
4557 • rclone (https://rclone.org/commands/rclone/) - Show help for rclone
4558 commands, flags and backends.
4559
4561 Copies standard input to file on remote.
4562 Synopsis
4564 rclone rcat reads from standard input (stdin) and copies it to a single
4565 remote file.
4566 echo "hello world" | rclone rcat remote:path/to/file
4568 ffmpeg - | rclone rcat remote:path/to/file
4569 If the remote file already exists, it will be overwritten.
4571 rcat will try to upload small files in a single request, which is usu‐
4573 ally more efficient than the streaming/chunked upload endpoints, which
4574 use multiple requests. Exact behaviour depends on the remote. What is
4575 considered a small file may be set through --streaming-upload-cutoff.
4576 Uploading only starts after the cutoff is reached or if the file ends
4577 before that. The data must fit into RAM. The cutoff needs to be small
4578 enough to adhere the limits of your remote, please see there. General‐
4579 ly speaking, setting this cutoff too high will decrease your perfor‐
4580 mance.
4581 Use the --size flag to preallocate the file in advance at the remote
4583 end and actually stream it, even if remote backend doesn't support
4584 streaming.
4585 --size should be the exact size of the input stream in bytes. If the
4587 size of the stream is different in length to the --size passed in then
4588 the transfer will likely fail.
4589 Note that the upload can also not be retried because the data is not
4591 kept around until the upload succeeds. If you need to transfer a lot
4592 of data, you're better off caching locally and then rclone move it to
4593 the destination.
4594 rclone rcat remote:path [flags]
4596 Options
4598 -h, --help help for rcat
4599 --size int File size hint to preallocate (default -1)
4600 See the global flags page (https://rclone.org/flags/) for global op‐
4602 tions not listed here.
4603 SEE ALSO
4605 • rclone (https://rclone.org/commands/rclone/) - Show help for rclone
4606 commands, flags and backends.
4607
4609 Run rclone listening to remote control commands only.
4610 Synopsis
4612 This runs rclone so that it only listens to remote control commands.
4613 This is useful if you are controlling rclone via the rc API.
4615 If you pass in a path to a directory, rclone will serve that directory
4617 for GET requests on the URL passed in. It will also open the URL in
4618 the browser when rclone is run.
4619 See the rc documentation (https://rclone.org/rc/) for more info on the
4621 rc flags.
4622 rclone rcd <path to files to serve>* [flags]
4624 Options
4626 -h, --help help for rcd
4627 See the global flags page (https://rclone.org/flags/) for global op‐
4629 tions not listed here.
4630 SEE ALSO
4632 • rclone (https://rclone.org/commands/rclone/) - Show help for rclone
4633 commands, flags and backends.
4634
4636 Remove empty directories under the path.
4637 Synopsis
4639 This recursively removes any empty directories (including directories
4640 that only contain empty directories), that it finds under the path.
4641 The root path itself will also be removed if it is empty, unless you
4642 supply the --leave-root flag.
4643 Use command rmdir (https://rclone.org/commands/rclone_rmdir/) to delete
4645 just the empty directory given by path, not recurse.
4646 This is useful for tidying up remotes that rclone has left a lot of
4648 empty directories in. For example the delete (https://rclone.org/com‐
4649 mands/rclone_delete/) command will delete files but leave the directory
4650 structure (unless used with option --rmdirs).
4651 To delete a path and any objects in it, use purge
4653 (https://rclone.org/commands/rclone_purge/) command.
4654 rclone rmdirs remote:path [flags]
4656 Options
4658 -h, --help help for rmdirs
4659 --leave-root Do not remove root directory if empty
4660 See the global flags page (https://rclone.org/flags/) for global op‐
4662 tions not listed here.
4663 SEE ALSO
4665 • rclone (https://rclone.org/commands/rclone/) - Show help for rclone
4666 commands, flags and backends.
4667
4669 Update the rclone binary.
4670 Synopsis
4672 This command downloads the latest release of rclone and replaces the
4673 currently running binary. The download is verified with a hashsum and
4674 cryptographically signed signature.
4675 If used without flags (or with implied --stable flag), this command
4677 will install the latest stable release. However, some issues may be
4678 fixed (or features added) only in the latest beta release. In such
4679 cases you should run the command with the --beta flag, i.e. rclone
4680 selfupdate --beta. You can check in advance what version would be in‐
4681 stalled by adding the --check flag, then repeat the command without it
4682 when you are satisfied.
4683 Sometimes the rclone team may recommend you a concrete beta or stable
4685 rclone release to troubleshoot your issue or add a bleeding edge fea‐
4686 ture. The --version VER flag, if given, will update to the concrete
4687 version instead of the latest one. If you omit micro version from VER
4688 (for example 1.53), the latest matching micro version will be used.
4689 Upon successful update rclone will print a message that contains a pre‐
4691 vious version number. You will need it if you later decide to revert
4692 your update for some reason. Then you'll have to note the previous
4693 version and run the following command: rclone selfupdate [--beta] OLD‐
4694 VER. If the old version contains only dots and digits (for example
4695 v1.54.0) then it's a stable release so you won't need the --beta flag.
4696 Beta releases have an additional information similar to v1.54.0-be‐
4697 ta.5111.06f1c0c61. (if you are a developer and use a locally built
4698 rclone, the version number will end with -DEV, you will have to rebuild
4699 it as it obviously can't be distributed).
4700 If you previously installed rclone via a package manager, the package
4702 may include local documentation or configure services. You may wish to
4703 update with the flag --package deb or --package rpm (whichever is cor‐
4704 rect for your OS) to update these too. This command with the default
4705 --package zip will update only the rclone executable so the local manu‐
4706 al may become inaccurate after it.
4707 The rclone mount command (https://rclone.org/commands/rclone_mount/)
4709 may or may not support extended FUSE options depending on the build and
4710 OS. selfupdate will refuse to update if the capability would be dis‐
4711 carded.
4712 Note: Windows forbids deletion of a currently running executable so
4714 this command will rename the old executable to 'rclone.old.exe' upon
4715 success.
4716 Please note that this command was not available before rclone version
4718 1.55. If it fails for you with the message unknown command "selfup‐
4719 date" then you will need to update manually following the install in‐
4720 structions located at https://rclone.org/install/
4721 rclone selfupdate [flags]
4723 Options
4725 --beta Install beta release
4726 --check Check for latest release, do not download
4727 -h, --help help for selfupdate
4728 --output string Save the downloaded binary at a given path (default: replace running binary)
4729 --package string Package format: zip|deb|rpm (default: zip)
4730 --stable Install stable release (this is the default)
4731 --version string Install the given rclone version (default: latest)
4732 See the global flags page (https://rclone.org/flags/) for global op‐
4734 tions not listed here.
4735 SEE ALSO
4737 • rclone (https://rclone.org/commands/rclone/) - Show help for rclone
4738 commands, flags and backends.
4739
4741 Serve a remote over a protocol.
4742 Synopsis
4744 Serve a remote over a given protocol. Requires the use of a subcommand
4745 to specify the protocol, e.g.
4746 rclone serve http remote:
4748 Each subcommand has its own options which you can see in their help.
4750 rclone serve <protocol> [opts] <remote> [flags]
4752 Options
4754 -h, --help help for serve
4755 See the global flags page (https://rclone.org/flags/) for global op‐
4757 tions not listed here.
4758 SEE ALSO
4760 • rclone (https://rclone.org/commands/rclone/) - Show help for rclone
4761 commands, flags and backends.
4762 • rclone serve dlna (https://rclone.org/commands/rclone_serve_dlna/) -
4764 Serve remote:path over DLNA
4765 • rclone serve docker (https://rclone.org/commands/rclone_serve_dock‐
4767 er/) - Serve any remote on docker's volume plugin API.
4768 • rclone serve ftp (https://rclone.org/commands/rclone_serve_ftp/) -
4770 Serve remote:path over FTP.
4771 • rclone serve http (https://rclone.org/commands/rclone_serve_http/) -
4773 Serve the remote over HTTP.
4774 • rclone serve restic (https://rclone.org/com‐
4776 mands/rclone_serve_restic/) - Serve the remote for restic's REST API.
4777 • rclone serve sftp (https://rclone.org/commands/rclone_serve_sftp/) -
4779 Serve the remote over SFTP.
4780 • rclone serve webdav (https://rclone.org/commands/rclone_serve_web‐
4782 dav/) - Serve remote:path over WebDAV.
4783
4785 Serve remote:path over DLNA
4786 Synopsis
4788 Run a DLNA media server for media stored in an rclone remote. Many de‐
4789 vices, such as the Xbox and PlayStation, can automatically discover
4790 this server in the LAN and play audio/video from it. VLC is also sup‐
4791 ported. Service discovery uses UDP multicast packets (SSDP) and will
4792 thus only work on LANs.
4793 Rclone will list all files present in the remote, without filtering
4795 based on media formats or file extensions. Additionally, there is no
4796 media transcoding support. This means that some players might show
4797 files that they are not able to play back correctly.
4798 Server options
4800 Use --addr to specify which IP address and port the server should lis‐
4801 ten on, e.g. --addr 1.2.3.4:8000 or --addr :8080 to listen to all IPs.
4802 Use --name to choose the friendly server name, which is by default
4804 "rclone (hostname)".
4805 Use --log-trace in conjunction with -vv to enable additional debug log‐
4807 ging of all UPNP traffic.
4808 VFS - Virtual File System
4810 This command uses the VFS layer. This adapts the cloud storage objects
4811 that rclone uses into something which looks much more like a disk fil‐
4812 ing system.
4813 Cloud storage objects have lots of properties which aren't like disk
4815 files - you can't extend them or write to the middle of them, so the
4816 VFS layer has to deal with that. Because there is no one right way of
4817 doing this there are various options explained below.
4818 The VFS layer also implements a directory cache - this caches info
4820 about files and directories (but not the data) in memory.
4821 VFS Directory Cache
4823 Using the --dir-cache-time flag, you can control how long a directory
4824 should be considered up to date and not refreshed from the backend.
4825 Changes made through the VFS will appear immediately or invalidate the
4826 cache.
4827 --dir-cache-time duration Time to cache directory entries for (default 5m0s)
4829 --poll-interval duration Time to wait between polling for changes. Must be smaller than dir-cache-time. Only on supported remotes. Set to 0 to disable (default 1m0s)
4830 However, changes made directly on the cloud storage by the web inter‐
4832 face or a different copy of rclone will only be picked up once the di‐
4833 rectory cache expires if the backend configured does not support
4834 polling for changes. If the backend supports polling, changes will be
4835 picked up within the polling interval.
4836 You can send a SIGHUP signal to rclone for it to flush all directory
4838 caches, regardless of how old they are. Assuming only one rclone in‐
4839 stance is running, you can reset the cache like this:
4840 kill -SIGHUP $(pidof rclone)
4842 If you configure rclone with a remote control then you can use rclone
4844 rc to flush the whole directory cache:
4845 rclone rc vfs/forget
4847 Or individual files or directories:
4849 rclone rc vfs/forget file=path/to/file dir=path/to/dir
4851 VFS File Buffering
4853 The --buffer-size flag determines the amount of memory, that will be
4854 used to buffer data in advance.
4855 Each open file will try to keep the specified amount of data in memory
4857 at all times. The buffered data is bound to one open file and won't be
4858 shared.
4859 This flag is a upper limit for the used memory per open file. The buf‐
4861 fer will only use memory for data that is downloaded but not not yet
4862 read. If the buffer is empty, only a small amount of memory will be
4863 used.
4864 The maximum memory used by rclone for buffering can be up to --buffer-
4866 size * open files.
4867 VFS File Caching
4869 These flags control the VFS file caching options. File caching is nec‐
4870 essary to make the VFS layer appear compatible with a normal file sys‐
4871 tem. It can be disabled at the cost of some compatibility.
4872 For example you'll need to enable VFS caching if you want to read and
4874 write simultaneously to a file. See below for more details.
4875 Note that the VFS cache is separate from the cache backend and you may
4877 find that you need one or the other or both.
4878 --cache-dir string Directory rclone will use for caching.
4880 --vfs-cache-mode CacheMode Cache mode off|minimal|writes|full (default off)
4881 --vfs-cache-max-age duration Max age of objects in the cache (default 1h0m0s)
4882 --vfs-cache-max-size SizeSuffix Max total size of objects in the cache (default off)
4883 --vfs-cache-poll-interval duration Interval to poll the cache for stale objects (default 1m0s)
4884 --vfs-write-back duration Time to writeback files after last use when using cache (default 5s)
4885 If run with -vv rclone will print the location of the file cache. The
4887 files are stored in the user cache file area which is OS dependent but
4888 can be controlled with --cache-dir or setting the appropriate environ‐
4889 ment variable.
4890 The cache has 4 different modes selected by --vfs-cache-mode. The
4892 higher the cache mode the more compatible rclone becomes at the cost of
4893 using disk space.
4894 Note that files are written back to the remote only when they are
4896 closed and if they haven't been accessed for --vfs-write-back seconds.
4897 If rclone is quit or dies with files that haven't been uploaded, these
4898 will be uploaded next time rclone is run with the same flags.
4899 If using --vfs-cache-max-size note that the cache may exceed this size
4901 for two reasons. Firstly because it is only checked every --vfs-cache-
4902 poll-interval. Secondly because open files cannot be evicted from the
4903 cache.
4904 You should not run two copies of rclone using the same VFS cache with
4906 the same or overlapping remotes if using --vfs-cache-mode > off. This
4907 can potentially cause data corruption if you do. You can work around
4908 this by giving each rclone its own cache hierarchy with --cache-dir.
4909 You don't need to worry about this if the remotes in use don't overlap.
4910 --vfs-cache-mode off
4912 In this mode (the default) the cache will read directly from the remote
4913 and write directly to the remote without caching anything on disk.
4914 This will mean some operations are not possible
4916 • Files can't be opened for both read AND write
4918 • Files opened for write can't be seeked
4920 • Existing files opened for write must have O_TRUNC set
4922 • Files open for read with O_TRUNC will be opened write only
4924 • Files open for write only will behave as if O_TRUNC was supplied
4926 • Open modes O_APPEND, O_TRUNC are ignored
4928 • If an upload fails it can't be retried
4930 --vfs-cache-mode minimal
4932 This is very similar to "off" except that files opened for read AND
4933 write will be buffered to disk. This means that files opened for write
4934 will be a lot more compatible, but uses the minimal disk space.
4935 These operations are not possible
4937 • Files opened for write only can't be seeked
4939 • Existing files opened for write must have O_TRUNC set
4941 • Files opened for write only will ignore O_APPEND, O_TRUNC
4943 • If an upload fails it can't be retried
4945 --vfs-cache-mode writes
4947 In this mode files opened for read only are still read directly from
4948 the remote, write only and read/write files are buffered to disk first.
4949 This mode should support all normal file system operations.
4951 If an upload fails it will be retried at exponentially increasing in‐
4953 tervals up to 1 minute.
4954 --vfs-cache-mode full
4956 In this mode all reads and writes are buffered to and from disk. When
4957 data is read from the remote this is buffered to disk as well.
4958 In this mode the files in the cache will be sparse files and rclone
4960 will keep track of which bits of the files it has downloaded.
4961 So if an application only reads the starts of each file, then rclone
4963 will only buffer the start of the file. These files will appear to be
4964 their full size in the cache, but they will be sparse files with only
4965 the data that has been downloaded present in them.
4966 This mode should support all normal file system operations and is oth‐
4968 erwise identical to --vfs-cache-mode writes.
4969 When reading a file rclone will read --buffer-size plus --vfs-read-
4971 ahead bytes ahead. The --buffer-size is buffered in memory whereas the
4972 --vfs-read-ahead is buffered on disk.
4973 When using this mode it is recommended that --buffer-size is not set
4975 too large and --vfs-read-ahead is set large if required.
4976 IMPORTANT not all file systems support sparse files. In particular
4978 FAT/exFAT do not. Rclone will perform very badly if the cache directo‐
4979 ry is on a filesystem which doesn't support sparse files and it will
4980 log an ERROR message if one is detected.
4981 Fingerprinting
4983 Various parts of the VFS use fingerprinting to see if a local file copy
4984 has changed relative to a remote file. Fingerprints are made from:
4985 • size
4987 • modification time
4989 • hash
4991 where available on an object.
4993 On some backends some of these attributes are slow to read (they take
4995 an extra API call per object, or extra work per object).
4996 For example hash is slow with the local and sftp backends as they have
4998 to read the entire file and hash it, and modtime is slow with the s3,
4999 swift, ftp and qinqstor backends because they need to do an extra API
5000 call to fetch it.
5001 If you use the --vfs-fast-fingerprint flag then rclone will not include
5003 the slow operations in the fingerprint. This makes the fingerprinting
5004 less accurate but much faster and will improve the opening time of
5005 cached files.
5006 If you are running a vfs cache over local, s3 or swift backends then
5008 using this flag is recommended.
5009 Note that if you change the value of this flag, the fingerprints of the
5011 files in the cache may be invalidated and the files will need to be
5012 downloaded again.
5013 VFS Chunked Reading
5015 When rclone reads files from a remote it reads them in chunks. This
5016 means that rather than requesting the whole file rclone reads the chunk
5017 specified. This can reduce the used download quota for some remotes by
5018 requesting only chunks from the remote that are actually read, at the
5019 cost of an increased number of requests.
5020 These flags control the chunking:
5022 --vfs-read-chunk-size SizeSuffix Read the source objects in chunks (default 128M)
5024 --vfs-read-chunk-size-limit SizeSuffix Max chunk doubling size (default off)
5025 Rclone will start reading a chunk of size --vfs-read-chunk-size, and
5027 then double the size for each read. When --vfs-read-chunk-size-limit
5028 is specified, and greater than --vfs-read-chunk-size, the chunk size
5029 for each open file will get doubled only until the specified value is
5030 reached. If the value is "off", which is the default, the limit is
5031 disabled and the chunk size will grow indefinitely.
5032 With --vfs-read-chunk-size 100M and --vfs-read-chunk-size-limit 0 the
5034 following parts will be downloaded: 0-100M, 100M-200M, 200M-300M,
5035 300M-400M and so on. When --vfs-read-chunk-size-limit 500M is speci‐
5036 fied, the result would be 0-100M, 100M-300M, 300M-700M, 700M-1200M,
5037 1200M-1700M and so on.
5038 Setting --vfs-read-chunk-size to 0 or "off" disables chunked reading.
5040 VFS Performance
5042 These flags may be used to enable/disable features of the VFS for per‐
5043 formance or other reasons. See also the chunked reading feature.
5044 In particular S3 and Swift benefit hugely from the --no-modtime flag
5046 (or use --use-server-modtime for a slightly different effect) as each
5047 read of the modification time takes a transaction.
5048 --no-checksum Don't compare checksums on up/download.
5050 --no-modtime Don't read/write the modification time (can speed things up).
5051 --no-seek Don't allow seeking in files.
5052 --read-only Only allow read-only access.
5053 Sometimes rclone is delivered reads or writes out of order. Rather
5055 than seeking rclone will wait a short time for the in sequence read or
5056 write to come in. These flags only come into effect when not using an
5057 on disk cache file.
5058 --vfs-read-wait duration Time to wait for in-sequence read before seeking (default 20ms)
5060 --vfs-write-wait duration Time to wait for in-sequence write before giving error (default 1s)
5061 When using VFS write caching (--vfs-cache-mode with value writes or
5063 full), the global flag --transfers can be set to adjust the number of
5064 parallel uploads of modified files from the cache (the related global
5065 flag --checkers has no effect on the VFS).
5066 --transfers int Number of file transfers to run in parallel (default 4)
5068 VFS Case Sensitivity
5070 Linux file systems are case-sensitive: two files can differ only by
5071 case, and the exact case must be used when opening a file.
5072 File systems in modern Windows are case-insensitive but case-preserv‐
5074 ing: although existing files can be opened using any case, the exact
5075 case used to create the file is preserved and available for programs to
5076 query. It is not allowed for two files in the same directory to differ
5077 only by case.
5078 Usually file systems on macOS are case-insensitive. It is possible to
5080 make macOS file systems case-sensitive but that is not the default.
5081 The --vfs-case-insensitive VFS flag controls how rclone handles these
5083 two cases. If its value is "false", rclone passes file names to the
5084 remote as-is. If the flag is "true" (or appears without a value on the
5085 command line), rclone may perform a "fixup" as explained below.
5086 The user may specify a file name to open/delete/rename/etc with a case
5088 different than what is stored on the remote. If an argument refers to
5089 an existing file with exactly the same name, then the case of the ex‐
5090 isting file on the disk will be used. However, if a file name with ex‐
5091 actly the same name is not found but a name differing only by case ex‐
5092 ists, rclone will transparently fixup the name. This fixup happens on‐
5093 ly when an existing file is requested. Case sensitivity of file names
5094 created anew by rclone is controlled by the underlying remote.
5095 Note that case sensitivity of the operating system running rclone (the
5097 target) may differ from case sensitivity of a file system presented by
5098 rclone (the source). The flag controls whether "fixup" is performed to
5099 satisfy the target.
5100 If the flag is not provided on the command line, then its default value
5102 depends on the operating system where rclone runs: "true" on Windows
5103 and macOS, "false" otherwise. If the flag is provided without a value,
5104 then it is "true".
5105 VFS Disk Options
5107 This flag allows you to manually set the statistics about the filing
5108 system. It can be useful when those statistics cannot be read correct‐
5109 ly automatically.
5110 --vfs-disk-space-total-size Manually set the total disk space size (example: 256G, default: -1)
5112 Alternate report of used bytes
5114 Some backends, most notably S3, do not report the amount of bytes used.
5115 If you need this information to be available when running df on the
5116 filesystem, then pass the flag --vfs-used-is-size to rclone. With this
5117 flag set, instead of relying on the backend to report this information,
5118 rclone will scan the whole remote similar to rclone size and compute
5119 the total used space itself.
5120 WARNING. Contrary to rclone size, this flag ignores filters so that the
5122 result is accurate. However, this is very inefficient and may cost
5123 lots of API calls resulting in extra charges. Use it as a last resort
5124 and only with caching.
5125 rclone serve dlna remote:path [flags]
5127 Options
5129 --addr string The ip:port or :port to bind the DLNA http server to (default ":7879")
5130 --announce-interval duration The interval between SSDP announcements (default 12m0s)
5131 --dir-cache-time duration Time to cache directory entries for (default 5m0s)
5132 --dir-perms FileMode Directory permissions (default 0777)
5133 --file-perms FileMode File permissions (default 0666)
5134 --gid uint32 Override the gid field set by the filesystem (not supported on Windows) (default 1000)
5135 -h, --help help for dlna
5136 --interface stringArray The interface to use for SSDP (repeat as necessary)
5137 --log-trace Enable trace logging of SOAP traffic
5138 --name string Name of DLNA server
5139 --no-checksum Don't compare checksums on up/download
5140 --no-modtime Don't read/write the modification time (can speed things up)
5141 --no-seek Don't allow seeking in files
5142 --poll-interval duration Time to wait between polling for changes, must be smaller than dir-cache-time and only on supported remotes (set 0 to disable) (default 1m0s)
5143 --read-only Only allow read-only access
5144 --uid uint32 Override the uid field set by the filesystem (not supported on Windows) (default 1000)
5145 --umask int Override the permission bits set by the filesystem (not supported on Windows) (default 2)
5146 --vfs-cache-max-age duration Max age of objects in the cache (default 1h0m0s)
5147 --vfs-cache-max-size SizeSuffix Max total size of objects in the cache (default off)
5148 --vfs-cache-mode CacheMode Cache mode off|minimal|writes|full (default off)
5149 --vfs-cache-poll-interval duration Interval to poll the cache for stale objects (default 1m0s)
5150 --vfs-case-insensitive If a file name not found, find a case insensitive match
5151 --vfs-disk-space-total-size SizeSuffix Specify the total space of disk (default off)
5152 --vfs-fast-fingerprint Use fast (less accurate) fingerprints for change detection
5153 --vfs-read-ahead SizeSuffix Extra read ahead over --buffer-size when using cache-mode full
5154 --vfs-read-chunk-size SizeSuffix Read the source objects in chunks (default 128Mi)
5155 --vfs-read-chunk-size-limit SizeSuffix If greater than --vfs-read-chunk-size, double the chunk size after each chunk read, until the limit is reached ('off' is unlimited) (default off)
5156 --vfs-read-wait duration Time to wait for in-sequence read before seeking (default 20ms)
5157 --vfs-used-is-size rclone size Use the rclone size algorithm for Used size
5158 --vfs-write-back duration Time to writeback files after last use when using cache (default 5s)
5159 --vfs-write-wait duration Time to wait for in-sequence write before giving error (default 1s)
5160 See the global flags page (https://rclone.org/flags/) for global op‐
5162 tions not listed here.
5163 SEE ALSO
5165 • rclone serve (https://rclone.org/commands/rclone_serve/) - Serve a
5166 remote over a protocol.
5167
5169 Serve any remote on docker's volume plugin API.
5170 Synopsis
5172 This command implements the Docker volume plugin API allowing docker to
5173 use rclone as a data storage mechanism for various cloud providers.
5174 rclone provides docker volume plugin based on it.
5175 To create a docker plugin, one must create a Unix or TCP socket that
5177 Docker will look for when you use the plugin and then it listens for
5178 commands from docker daemon and runs the corresponding code when neces‐
5179 sary. Docker plugins can run as a managed plugin under control of the
5180 docker daemon or as an independent native service. For testing, you
5181 can just run it directly from the command line, for example:
5182 sudo rclone serve docker --base-dir /tmp/rclone-volumes --socket-addr localhost:8787 -vv
5184 Running rclone serve docker will create the said socket, listening for
5186 commands from Docker to create the necessary Volumes. Normally you
5187 need not give the --socket-addr flag. The API will listen on the unix
5188 domain socket at /run/docker/plugins/rclone.sock. In the example above
5189 rclone will create a TCP socket and a small file /etc/docker/plug‐
5190 ins/rclone.spec containing the socket address. We use sudo because
5191 both paths are writeable only by the root user.
5192 If you later decide to change listening socket, the docker daemon must
5194 be restarted to reconnect to /run/docker/plugins/rclone.sock or parse
5195 new /etc/docker/plugins/rclone.spec. Until you restart, any volume re‐
5196 lated docker commands will timeout trying to access the old socket.
5197 Running directly is supported on Linux only, not on Windows or MacOS.
5198 This is not a problem with managed plugin mode described in details in
5199 the full documentation (https://rclone.org/docker).
5200 The command will create volume mounts under the path given by --base-
5202 dir (by default /var/lib/docker-volumes/rclone available only to root)
5203 and maintain the JSON formatted file docker-plugin.state in the rclone
5204 cache directory with book-keeping records of created and mounted vol‐
5205 umes.
5206 All mount and VFS options are submitted by the docker daemon via API,
5208 but you can also provide defaults on the command line as well as set
5209 path to the config file and cache directory or adjust logging verbosi‐
5210 ty.
5211 VFS - Virtual File System
5213 This command uses the VFS layer. This adapts the cloud storage objects
5214 that rclone uses into something which looks much more like a disk fil‐
5215 ing system.
5216 Cloud storage objects have lots of properties which aren't like disk
5218 files - you can't extend them or write to the middle of them, so the
5219 VFS layer has to deal with that. Because there is no one right way of
5220 doing this there are various options explained below.
5221 The VFS layer also implements a directory cache - this caches info
5223 about files and directories (but not the data) in memory.
5224 VFS Directory Cache
5226 Using the --dir-cache-time flag, you can control how long a directory
5227 should be considered up to date and not refreshed from the backend.
5228 Changes made through the VFS will appear immediately or invalidate the
5229 cache.
5230 --dir-cache-time duration Time to cache directory entries for (default 5m0s)
5232 --poll-interval duration Time to wait between polling for changes. Must be smaller than dir-cache-time. Only on supported remotes. Set to 0 to disable (default 1m0s)
5233 However, changes made directly on the cloud storage by the web inter‐
5235 face or a different copy of rclone will only be picked up once the di‐
5236 rectory cache expires if the backend configured does not support
5237 polling for changes. If the backend supports polling, changes will be
5238 picked up within the polling interval.
5239 You can send a SIGHUP signal to rclone for it to flush all directory
5241 caches, regardless of how old they are. Assuming only one rclone in‐
5242 stance is running, you can reset the cache like this:
5243 kill -SIGHUP $(pidof rclone)
5245 If you configure rclone with a remote control then you can use rclone
5247 rc to flush the whole directory cache:
5248 rclone rc vfs/forget
5250 Or individual files or directories:
5252 rclone rc vfs/forget file=path/to/file dir=path/to/dir
5254 VFS File Buffering
5256 The --buffer-size flag determines the amount of memory, that will be
5257 used to buffer data in advance.
5258 Each open file will try to keep the specified amount of data in memory
5260 at all times. The buffered data is bound to one open file and won't be
5261 shared.
5262 This flag is a upper limit for the used memory per open file. The buf‐
5264 fer will only use memory for data that is downloaded but not not yet
5265 read. If the buffer is empty, only a small amount of memory will be
5266 used.
5267 The maximum memory used by rclone for buffering can be up to --buffer-
5269 size * open files.
5270 VFS File Caching
5272 These flags control the VFS file caching options. File caching is nec‐
5273 essary to make the VFS layer appear compatible with a normal file sys‐
5274 tem. It can be disabled at the cost of some compatibility.
5275 For example you'll need to enable VFS caching if you want to read and
5277 write simultaneously to a file. See below for more details.
5278 Note that the VFS cache is separate from the cache backend and you may
5280 find that you need one or the other or both.
5281 --cache-dir string Directory rclone will use for caching.
5283 --vfs-cache-mode CacheMode Cache mode off|minimal|writes|full (default off)
5284 --vfs-cache-max-age duration Max age of objects in the cache (default 1h0m0s)
5285 --vfs-cache-max-size SizeSuffix Max total size of objects in the cache (default off)
5286 --vfs-cache-poll-interval duration Interval to poll the cache for stale objects (default 1m0s)
5287 --vfs-write-back duration Time to writeback files after last use when using cache (default 5s)
5288 If run with -vv rclone will print the location of the file cache. The
5290 files are stored in the user cache file area which is OS dependent but
5291 can be controlled with --cache-dir or setting the appropriate environ‐
5292 ment variable.
5293 The cache has 4 different modes selected by --vfs-cache-mode. The
5295 higher the cache mode the more compatible rclone becomes at the cost of
5296 using disk space.
5297 Note that files are written back to the remote only when they are
5299 closed and if they haven't been accessed for --vfs-write-back seconds.
5300 If rclone is quit or dies with files that haven't been uploaded, these
5301 will be uploaded next time rclone is run with the same flags.
5302 If using --vfs-cache-max-size note that the cache may exceed this size
5304 for two reasons. Firstly because it is only checked every --vfs-cache-
5305 poll-interval. Secondly because open files cannot be evicted from the
5306 cache.
5307 You should not run two copies of rclone using the same VFS cache with
5309 the same or overlapping remotes if using --vfs-cache-mode > off. This
5310 can potentially cause data corruption if you do. You can work around
5311 this by giving each rclone its own cache hierarchy with --cache-dir.
5312 You don't need to worry about this if the remotes in use don't overlap.
5313 --vfs-cache-mode off
5315 In this mode (the default) the cache will read directly from the remote
5316 and write directly to the remote without caching anything on disk.
5317 This will mean some operations are not possible
5319 • Files can't be opened for both read AND write
5321 • Files opened for write can't be seeked
5323 • Existing files opened for write must have O_TRUNC set
5325 • Files open for read with O_TRUNC will be opened write only
5327 • Files open for write only will behave as if O_TRUNC was supplied
5329 • Open modes O_APPEND, O_TRUNC are ignored
5331 • If an upload fails it can't be retried
5333 --vfs-cache-mode minimal
5335 This is very similar to "off" except that files opened for read AND
5336 write will be buffered to disk. This means that files opened for write
5337 will be a lot more compatible, but uses the minimal disk space.
5338 These operations are not possible
5340 • Files opened for write only can't be seeked
5342 • Existing files opened for write must have O_TRUNC set
5344 • Files opened for write only will ignore O_APPEND, O_TRUNC
5346 • If an upload fails it can't be retried
5348 --vfs-cache-mode writes
5350 In this mode files opened for read only are still read directly from
5351 the remote, write only and read/write files are buffered to disk first.
5352 This mode should support all normal file system operations.
5354 If an upload fails it will be retried at exponentially increasing in‐
5356 tervals up to 1 minute.
5357 --vfs-cache-mode full
5359 In this mode all reads and writes are buffered to and from disk. When
5360 data is read from the remote this is buffered to disk as well.
5361 In this mode the files in the cache will be sparse files and rclone
5363 will keep track of which bits of the files it has downloaded.
5364 So if an application only reads the starts of each file, then rclone
5366 will only buffer the start of the file. These files will appear to be
5367 their full size in the cache, but they will be sparse files with only
5368 the data that has been downloaded present in them.
5369 This mode should support all normal file system operations and is oth‐
5371 erwise identical to --vfs-cache-mode writes.
5372 When reading a file rclone will read --buffer-size plus --vfs-read-
5374 ahead bytes ahead. The --buffer-size is buffered in memory whereas the
5375 --vfs-read-ahead is buffered on disk.
5376 When using this mode it is recommended that --buffer-size is not set
5378 too large and --vfs-read-ahead is set large if required.
5379 IMPORTANT not all file systems support sparse files. In particular
5381 FAT/exFAT do not. Rclone will perform very badly if the cache directo‐
5382 ry is on a filesystem which doesn't support sparse files and it will
5383 log an ERROR message if one is detected.
5384 Fingerprinting
5386 Various parts of the VFS use fingerprinting to see if a local file copy
5387 has changed relative to a remote file. Fingerprints are made from:
5388 • size
5390 • modification time
5392 • hash
5394 where available on an object.
5396 On some backends some of these attributes are slow to read (they take
5398 an extra API call per object, or extra work per object).
5399 For example hash is slow with the local and sftp backends as they have
5401 to read the entire file and hash it, and modtime is slow with the s3,
5402 swift, ftp and qinqstor backends because they need to do an extra API
5403 call to fetch it.
5404 If you use the --vfs-fast-fingerprint flag then rclone will not include
5406 the slow operations in the fingerprint. This makes the fingerprinting
5407 less accurate but much faster and will improve the opening time of
5408 cached files.
5409 If you are running a vfs cache over local, s3 or swift backends then
5411 using this flag is recommended.
5412 Note that if you change the value of this flag, the fingerprints of the
5414 files in the cache may be invalidated and the files will need to be
5415 downloaded again.
5416 VFS Chunked Reading
5418 When rclone reads files from a remote it reads them in chunks. This
5419 means that rather than requesting the whole file rclone reads the chunk
5420 specified. This can reduce the used download quota for some remotes by
5421 requesting only chunks from the remote that are actually read, at the
5422 cost of an increased number of requests.
5423 These flags control the chunking:
5425 --vfs-read-chunk-size SizeSuffix Read the source objects in chunks (default 128M)
5427 --vfs-read-chunk-size-limit SizeSuffix Max chunk doubling size (default off)
5428 Rclone will start reading a chunk of size --vfs-read-chunk-size, and
5430 then double the size for each read. When --vfs-read-chunk-size-limit
5431 is specified, and greater than --vfs-read-chunk-size, the chunk size
5432 for each open file will get doubled only until the specified value is
5433 reached. If the value is "off", which is the default, the limit is
5434 disabled and the chunk size will grow indefinitely.
5435 With --vfs-read-chunk-size 100M and --vfs-read-chunk-size-limit 0 the
5437 following parts will be downloaded: 0-100M, 100M-200M, 200M-300M,
5438 300M-400M and so on. When --vfs-read-chunk-size-limit 500M is speci‐
5439 fied, the result would be 0-100M, 100M-300M, 300M-700M, 700M-1200M,
5440 1200M-1700M and so on.
5441 Setting --vfs-read-chunk-size to 0 or "off" disables chunked reading.
5443 VFS Performance
5445 These flags may be used to enable/disable features of the VFS for per‐
5446 formance or other reasons. See also the chunked reading feature.
5447 In particular S3 and Swift benefit hugely from the --no-modtime flag
5449 (or use --use-server-modtime for a slightly different effect) as each
5450 read of the modification time takes a transaction.
5451 --no-checksum Don't compare checksums on up/download.
5453 --no-modtime Don't read/write the modification time (can speed things up).
5454 --no-seek Don't allow seeking in files.
5455 --read-only Only allow read-only access.
5456 Sometimes rclone is delivered reads or writes out of order. Rather
5458 than seeking rclone will wait a short time for the in sequence read or
5459 write to come in. These flags only come into effect when not using an
5460 on disk cache file.
5461 --vfs-read-wait duration Time to wait for in-sequence read before seeking (default 20ms)
5463 --vfs-write-wait duration Time to wait for in-sequence write before giving error (default 1s)
5464 When using VFS write caching (--vfs-cache-mode with value writes or
5466 full), the global flag --transfers can be set to adjust the number of
5467 parallel uploads of modified files from the cache (the related global
5468 flag --checkers has no effect on the VFS).
5469 --transfers int Number of file transfers to run in parallel (default 4)
5471 VFS Case Sensitivity
5473 Linux file systems are case-sensitive: two files can differ only by
5474 case, and the exact case must be used when opening a file.
5475 File systems in modern Windows are case-insensitive but case-preserv‐
5477 ing: although existing files can be opened using any case, the exact
5478 case used to create the file is preserved and available for programs to
5479 query. It is not allowed for two files in the same directory to differ
5480 only by case.
5481 Usually file systems on macOS are case-insensitive. It is possible to
5483 make macOS file systems case-sensitive but that is not the default.
5484 The --vfs-case-insensitive VFS flag controls how rclone handles these
5486 two cases. If its value is "false", rclone passes file names to the
5487 remote as-is. If the flag is "true" (or appears without a value on the
5488 command line), rclone may perform a "fixup" as explained below.
5489 The user may specify a file name to open/delete/rename/etc with a case
5491 different than what is stored on the remote. If an argument refers to
5492 an existing file with exactly the same name, then the case of the ex‐
5493 isting file on the disk will be used. However, if a file name with ex‐
5494 actly the same name is not found but a name differing only by case ex‐
5495 ists, rclone will transparently fixup the name. This fixup happens on‐
5496 ly when an existing file is requested. Case sensitivity of file names
5497 created anew by rclone is controlled by the underlying remote.
5498 Note that case sensitivity of the operating system running rclone (the
5500 target) may differ from case sensitivity of a file system presented by
5501 rclone (the source). The flag controls whether "fixup" is performed to
5502 satisfy the target.
5503 If the flag is not provided on the command line, then its default value
5505 depends on the operating system where rclone runs: "true" on Windows
5506 and macOS, "false" otherwise. If the flag is provided without a value,
5507 then it is "true".
5508 VFS Disk Options
5510 This flag allows you to manually set the statistics about the filing
5511 system. It can be useful when those statistics cannot be read correct‐
5512 ly automatically.
5513 --vfs-disk-space-total-size Manually set the total disk space size (example: 256G, default: -1)
5515 Alternate report of used bytes
5517 Some backends, most notably S3, do not report the amount of bytes used.
5518 If you need this information to be available when running df on the
5519 filesystem, then pass the flag --vfs-used-is-size to rclone. With this
5520 flag set, instead of relying on the backend to report this information,
5521 rclone will scan the whole remote similar to rclone size and compute
5522 the total used space itself.
5523 WARNING. Contrary to rclone size, this flag ignores filters so that the
5525 result is accurate. However, this is very inefficient and may cost
5526 lots of API calls resulting in extra charges. Use it as a last resort
5527 and only with caching.
5528 rclone serve docker [flags]
5530 Options
5532 --allow-non-empty Allow mounting over a non-empty directory (not supported on Windows)
5533 --allow-other Allow access to other users (not supported on Windows)
5534 --allow-root Allow access to root user (not supported on Windows)
5535 --async-read Use asynchronous reads (not supported on Windows) (default true)
5536 --attr-timeout duration Time for which file/directory attributes are cached (default 1s)
5537 --base-dir string Base directory for volumes (default "/var/lib/docker-volumes/rclone")
5538 --daemon Run mount in background and exit parent process (as background output is suppressed, use --log-file with --log-format=pid,... to monitor) (not supported on Windows)
5539 --daemon-timeout duration Time limit for rclone to respond to kernel (not supported on Windows)
5540 --daemon-wait duration Time to wait for ready mount from daemon (maximum time on Linux, constant sleep time on OSX/BSD) (not supported on Windows) (default 1m0s)
5541 --debug-fuse Debug the FUSE internals - needs -v
5542 --default-permissions Makes kernel enforce access control based on the file mode (not supported on Windows)
5543 --devname string Set the device name - default is remote:path
5544 --dir-cache-time duration Time to cache directory entries for (default 5m0s)
5545 --dir-perms FileMode Directory permissions (default 0777)
5546 --file-perms FileMode File permissions (default 0666)
5547 --forget-state Skip restoring previous state
5548 --fuse-flag stringArray Flags or arguments to be passed direct to libfuse/WinFsp (repeat if required)
5549 --gid uint32 Override the gid field set by the filesystem (not supported on Windows) (default 1000)
5550 -h, --help help for docker
5551 --max-read-ahead SizeSuffix The number of bytes that can be prefetched for sequential reads (not supported on Windows) (default 128Ki)
5552 --network-mode Mount as remote network drive, instead of fixed disk drive (supported on Windows only)
5553 --no-checksum Don't compare checksums on up/download
5554 --no-modtime Don't read/write the modification time (can speed things up)
5555 --no-seek Don't allow seeking in files
5556 --no-spec Do not write spec file
5557 --noappledouble Ignore Apple Double (._) and .DS_Store files (supported on OSX only) (default true)
5558 --noapplexattr Ignore all "com.apple.*" extended attributes (supported on OSX only)
5559 -o, --option stringArray Option for libfuse/WinFsp (repeat if required)
5560 --poll-interval duration Time to wait between polling for changes, must be smaller than dir-cache-time and only on supported remotes (set 0 to disable) (default 1m0s)
5561 --read-only Only allow read-only access
5562 --socket-addr string Address <host:port> or absolute path (default: /run/docker/plugins/rclone.sock)
5563 --socket-gid int GID for unix socket (default: current process GID) (default 1000)
5564 --uid uint32 Override the uid field set by the filesystem (not supported on Windows) (default 1000)
5565 --umask int Override the permission bits set by the filesystem (not supported on Windows) (default 2)
5566 --vfs-cache-max-age duration Max age of objects in the cache (default 1h0m0s)
5567 --vfs-cache-max-size SizeSuffix Max total size of objects in the cache (default off)
5568 --vfs-cache-mode CacheMode Cache mode off|minimal|writes|full (default off)
5569 --vfs-cache-poll-interval duration Interval to poll the cache for stale objects (default 1m0s)
5570 --vfs-case-insensitive If a file name not found, find a case insensitive match
5571 --vfs-disk-space-total-size SizeSuffix Specify the total space of disk (default off)
5572 --vfs-fast-fingerprint Use fast (less accurate) fingerprints for change detection
5573 --vfs-read-ahead SizeSuffix Extra read ahead over --buffer-size when using cache-mode full
5574 --vfs-read-chunk-size SizeSuffix Read the source objects in chunks (default 128Mi)
5575 --vfs-read-chunk-size-limit SizeSuffix If greater than --vfs-read-chunk-size, double the chunk size after each chunk read, until the limit is reached ('off' is unlimited) (default off)
5576 --vfs-read-wait duration Time to wait for in-sequence read before seeking (default 20ms)
5577 --vfs-used-is-size rclone size Use the rclone size algorithm for Used size
5578 --vfs-write-back duration Time to writeback files after last use when using cache (default 5s)
5579 --vfs-write-wait duration Time to wait for in-sequence write before giving error (default 1s)
5580 --volname string Set the volume name (supported on Windows and OSX only)
5581 --write-back-cache Makes kernel buffer writes before sending them to rclone (without this, writethrough caching is used) (not supported on Windows)
5582 See the global flags page (https://rclone.org/flags/) for global op‐
5584 tions not listed here.
5585 SEE ALSO
5587 • rclone serve (https://rclone.org/commands/rclone_serve/) - Serve a
5588 remote over a protocol.
5589
5591 Serve remote:path over FTP.
5592 Synopsis
5594 Run a basic FTP server to serve a remote over FTP protocol. This can
5595 be viewed with a FTP client or you can make a remote of type FTP to
5596 read and write it.
5597 Server options
5599 Use --addr to specify which IP address and port the server should lis‐
5600 ten on, e.g. --addr 1.2.3.4:8000 or --addr :8080 to listen to all IPs.
5601 By default it only listens on localhost. You can use port :0 to let
5602 the OS choose an available port.
5603 If you set --addr to listen on a public or LAN accessible IP address
5605 then using Authentication is advised - see the next section for info.
5606 Authentication
5608 By default this will serve files without needing a login.
5609 You can set a single username and password with the --user and --pass
5611 flags.
5612 VFS - Virtual File System
5614 This command uses the VFS layer. This adapts the cloud storage objects
5615 that rclone uses into something which looks much more like a disk fil‐
5616 ing system.
5617 Cloud storage objects have lots of properties which aren't like disk
5619 files - you can't extend them or write to the middle of them, so the
5620 VFS layer has to deal with that. Because there is no one right way of
5621 doing this there are various options explained below.
5622 The VFS layer also implements a directory cache - this caches info
5624 about files and directories (but not the data) in memory.
5625 VFS Directory Cache
5627 Using the --dir-cache-time flag, you can control how long a directory
5628 should be considered up to date and not refreshed from the backend.
5629 Changes made through the VFS will appear immediately or invalidate the
5630 cache.
5631 --dir-cache-time duration Time to cache directory entries for (default 5m0s)
5633 --poll-interval duration Time to wait between polling for changes. Must be smaller than dir-cache-time. Only on supported remotes. Set to 0 to disable (default 1m0s)
5634 However, changes made directly on the cloud storage by the web inter‐
5636 face or a different copy of rclone will only be picked up once the di‐
5637 rectory cache expires if the backend configured does not support
5638 polling for changes. If the backend supports polling, changes will be
5639 picked up within the polling interval.
5640 You can send a SIGHUP signal to rclone for it to flush all directory
5642 caches, regardless of how old they are. Assuming only one rclone in‐
5643 stance is running, you can reset the cache like this:
5644 kill -SIGHUP $(pidof rclone)
5646 If you configure rclone with a remote control then you can use rclone
5648 rc to flush the whole directory cache:
5649 rclone rc vfs/forget
5651 Or individual files or directories:
5653 rclone rc vfs/forget file=path/to/file dir=path/to/dir
5655 VFS File Buffering
5657 The --buffer-size flag determines the amount of memory, that will be
5658 used to buffer data in advance.
5659 Each open file will try to keep the specified amount of data in memory
5661 at all times. The buffered data is bound to one open file and won't be
5662 shared.
5663 This flag is a upper limit for the used memory per open file. The buf‐
5665 fer will only use memory for data that is downloaded but not not yet
5666 read. If the buffer is empty, only a small amount of memory will be
5667 used.
5668 The maximum memory used by rclone for buffering can be up to --buffer-
5670 size * open files.
5671 VFS File Caching
5673 These flags control the VFS file caching options. File caching is nec‐
5674 essary to make the VFS layer appear compatible with a normal file sys‐
5675 tem. It can be disabled at the cost of some compatibility.
5676 For example you'll need to enable VFS caching if you want to read and
5678 write simultaneously to a file. See below for more details.
5679 Note that the VFS cache is separate from the cache backend and you may
5681 find that you need one or the other or both.
5682 --cache-dir string Directory rclone will use for caching.
5684 --vfs-cache-mode CacheMode Cache mode off|minimal|writes|full (default off)
5685 --vfs-cache-max-age duration Max age of objects in the cache (default 1h0m0s)
5686 --vfs-cache-max-size SizeSuffix Max total size of objects in the cache (default off)
5687 --vfs-cache-poll-interval duration Interval to poll the cache for stale objects (default 1m0s)
5688 --vfs-write-back duration Time to writeback files after last use when using cache (default 5s)
5689 If run with -vv rclone will print the location of the file cache. The
5691 files are stored in the user cache file area which is OS dependent but
5692 can be controlled with --cache-dir or setting the appropriate environ‐
5693 ment variable.
5694 The cache has 4 different modes selected by --vfs-cache-mode. The
5696 higher the cache mode the more compatible rclone becomes at the cost of
5697 using disk space.
5698 Note that files are written back to the remote only when they are
5700 closed and if they haven't been accessed for --vfs-write-back seconds.
5701 If rclone is quit or dies with files that haven't been uploaded, these
5702 will be uploaded next time rclone is run with the same flags.
5703 If using --vfs-cache-max-size note that the cache may exceed this size
5705 for two reasons. Firstly because it is only checked every --vfs-cache-
5706 poll-interval. Secondly because open files cannot be evicted from the
5707 cache.
5708 You should not run two copies of rclone using the same VFS cache with
5710 the same or overlapping remotes if using --vfs-cache-mode > off. This
5711 can potentially cause data corruption if you do. You can work around
5712 this by giving each rclone its own cache hierarchy with --cache-dir.
5713 You don't need to worry about this if the remotes in use don't overlap.
5714 --vfs-cache-mode off
5716 In this mode (the default) the cache will read directly from the remote
5717 and write directly to the remote without caching anything on disk.
5718 This will mean some operations are not possible
5720 • Files can't be opened for both read AND write
5722 • Files opened for write can't be seeked
5724 • Existing files opened for write must have O_TRUNC set
5726 • Files open for read with O_TRUNC will be opened write only
5728 • Files open for write only will behave as if O_TRUNC was supplied
5730 • Open modes O_APPEND, O_TRUNC are ignored
5732 • If an upload fails it can't be retried
5734 --vfs-cache-mode minimal
5736 This is very similar to "off" except that files opened for read AND
5737 write will be buffered to disk. This means that files opened for write
5738 will be a lot more compatible, but uses the minimal disk space.
5739 These operations are not possible
5741 • Files opened for write only can't be seeked
5743 • Existing files opened for write must have O_TRUNC set
5745 • Files opened for write only will ignore O_APPEND, O_TRUNC
5747 • If an upload fails it can't be retried
5749 --vfs-cache-mode writes
5751 In this mode files opened for read only are still read directly from
5752 the remote, write only and read/write files are buffered to disk first.
5753 This mode should support all normal file system operations.
5755 If an upload fails it will be retried at exponentially increasing in‐
5757 tervals up to 1 minute.
5758 --vfs-cache-mode full
5760 In this mode all reads and writes are buffered to and from disk. When
5761 data is read from the remote this is buffered to disk as well.
5762 In this mode the files in the cache will be sparse files and rclone
5764 will keep track of which bits of the files it has downloaded.
5765 So if an application only reads the starts of each file, then rclone
5767 will only buffer the start of the file. These files will appear to be
5768 their full size in the cache, but they will be sparse files with only
5769 the data that has been downloaded present in them.
5770 This mode should support all normal file system operations and is oth‐
5772 erwise identical to --vfs-cache-mode writes.
5773 When reading a file rclone will read --buffer-size plus --vfs-read-
5775 ahead bytes ahead. The --buffer-size is buffered in memory whereas the
5776 --vfs-read-ahead is buffered on disk.
5777 When using this mode it is recommended that --buffer-size is not set
5779 too large and --vfs-read-ahead is set large if required.
5780 IMPORTANT not all file systems support sparse files. In particular
5782 FAT/exFAT do not. Rclone will perform very badly if the cache directo‐
5783 ry is on a filesystem which doesn't support sparse files and it will
5784 log an ERROR message if one is detected.
5785 Fingerprinting
5787 Various parts of the VFS use fingerprinting to see if a local file copy
5788 has changed relative to a remote file. Fingerprints are made from:
5789 • size
5791 • modification time
5793 • hash
5795 where available on an object.
5797 On some backends some of these attributes are slow to read (they take
5799 an extra API call per object, or extra work per object).
5800 For example hash is slow with the local and sftp backends as they have
5802 to read the entire file and hash it, and modtime is slow with the s3,
5803 swift, ftp and qinqstor backends because they need to do an extra API
5804 call to fetch it.
5805 If you use the --vfs-fast-fingerprint flag then rclone will not include
5807 the slow operations in the fingerprint. This makes the fingerprinting
5808 less accurate but much faster and will improve the opening time of
5809 cached files.
5810 If you are running a vfs cache over local, s3 or swift backends then
5812 using this flag is recommended.
5813 Note that if you change the value of this flag, the fingerprints of the
5815 files in the cache may be invalidated and the files will need to be
5816 downloaded again.
5817 VFS Chunked Reading
5819 When rclone reads files from a remote it reads them in chunks. This
5820 means that rather than requesting the whole file rclone reads the chunk
5821 specified. This can reduce the used download quota for some remotes by
5822 requesting only chunks from the remote that are actually read, at the
5823 cost of an increased number of requests.
5824 These flags control the chunking:
5826 --vfs-read-chunk-size SizeSuffix Read the source objects in chunks (default 128M)
5828 --vfs-read-chunk-size-limit SizeSuffix Max chunk doubling size (default off)
5829 Rclone will start reading a chunk of size --vfs-read-chunk-size, and
5831 then double the size for each read. When --vfs-read-chunk-size-limit
5832 is specified, and greater than --vfs-read-chunk-size, the chunk size
5833 for each open file will get doubled only until the specified value is
5834 reached. If the value is "off", which is the default, the limit is
5835 disabled and the chunk size will grow indefinitely.
5836 With --vfs-read-chunk-size 100M and --vfs-read-chunk-size-limit 0 the
5838 following parts will be downloaded: 0-100M, 100M-200M, 200M-300M,
5839 300M-400M and so on. When --vfs-read-chunk-size-limit 500M is speci‐
5840 fied, the result would be 0-100M, 100M-300M, 300M-700M, 700M-1200M,
5841 1200M-1700M and so on.
5842 Setting --vfs-read-chunk-size to 0 or "off" disables chunked reading.
5844 VFS Performance
5846 These flags may be used to enable/disable features of the VFS for per‐
5847 formance or other reasons. See also the chunked reading feature.
5848 In particular S3 and Swift benefit hugely from the --no-modtime flag
5850 (or use --use-server-modtime for a slightly different effect) as each
5851 read of the modification time takes a transaction.
5852 --no-checksum Don't compare checksums on up/download.
5854 --no-modtime Don't read/write the modification time (can speed things up).
5855 --no-seek Don't allow seeking in files.
5856 --read-only Only allow read-only access.
5857 Sometimes rclone is delivered reads or writes out of order. Rather
5859 than seeking rclone will wait a short time for the in sequence read or
5860 write to come in. These flags only come into effect when not using an
5861 on disk cache file.
5862 --vfs-read-wait duration Time to wait for in-sequence read before seeking (default 20ms)
5864 --vfs-write-wait duration Time to wait for in-sequence write before giving error (default 1s)
5865 When using VFS write caching (--vfs-cache-mode with value writes or
5867 full), the global flag --transfers can be set to adjust the number of
5868 parallel uploads of modified files from the cache (the related global
5869 flag --checkers has no effect on the VFS).
5870 --transfers int Number of file transfers to run in parallel (default 4)
5872 VFS Case Sensitivity
5874 Linux file systems are case-sensitive: two files can differ only by
5875 case, and the exact case must be used when opening a file.
5876 File systems in modern Windows are case-insensitive but case-preserv‐
5878 ing: although existing files can be opened using any case, the exact
5879 case used to create the file is preserved and available for programs to
5880 query. It is not allowed for two files in the same directory to differ
5881 only by case.
5882 Usually file systems on macOS are case-insensitive. It is possible to
5884 make macOS file systems case-sensitive but that is not the default.
5885 The --vfs-case-insensitive VFS flag controls how rclone handles these
5887 two cases. If its value is "false", rclone passes file names to the
5888 remote as-is. If the flag is "true" (or appears without a value on the
5889 command line), rclone may perform a "fixup" as explained below.
5890 The user may specify a file name to open/delete/rename/etc with a case
5892 different than what is stored on the remote. If an argument refers to
5893 an existing file with exactly the same name, then the case of the ex‐
5894 isting file on the disk will be used. However, if a file name with ex‐
5895 actly the same name is not found but a name differing only by case ex‐
5896 ists, rclone will transparently fixup the name. This fixup happens on‐
5897 ly when an existing file is requested. Case sensitivity of file names
5898 created anew by rclone is controlled by the underlying remote.
5899 Note that case sensitivity of the operating system running rclone (the
5901 target) may differ from case sensitivity of a file system presented by
5902 rclone (the source). The flag controls whether "fixup" is performed to
5903 satisfy the target.
5904 If the flag is not provided on the command line, then its default value
5906 depends on the operating system where rclone runs: "true" on Windows
5907 and macOS, "false" otherwise. If the flag is provided without a value,
5908 then it is "true".
5909 VFS Disk Options
5911 This flag allows you to manually set the statistics about the filing
5912 system. It can be useful when those statistics cannot be read correct‐
5913 ly automatically.
5914 --vfs-disk-space-total-size Manually set the total disk space size (example: 256G, default: -1)
5916 Alternate report of used bytes
5918 Some backends, most notably S3, do not report the amount of bytes used.
5919 If you need this information to be available when running df on the
5920 filesystem, then pass the flag --vfs-used-is-size to rclone. With this
5921 flag set, instead of relying on the backend to report this information,
5922 rclone will scan the whole remote similar to rclone size and compute
5923 the total used space itself.
5924 WARNING. Contrary to rclone size, this flag ignores filters so that the
5926 result is accurate. However, this is very inefficient and may cost
5927 lots of API calls resulting in extra charges. Use it as a last resort
5928 and only with caching.
5929 Auth Proxy
5931 If you supply the parameter --auth-proxy /path/to/program then rclone
5932 will use that program to generate backends on the fly which then are
5933 used to authenticate incoming requests. This uses a simple JSON based
5934 protocol with input on STDIN and output on STDOUT.
5935 PLEASE NOTE: --auth-proxy and --authorized-keys cannot be used togeth‐
5937 er, if --auth-proxy is set the authorized keys option will be ignored.
5938 There is an example program bin/test_proxy.py
5940 (https://github.com/rclone/rclone/blob/master/test_proxy.py) in the
5941 rclone source code.
5942 The program's job is to take a user and pass on the input and turn
5944 those into the config for a backend on STDOUT in JSON format. This
5945 config will have any default parameters for the backend added, but it
5946 won't use configuration from environment variables or command line op‐
5947 tions - it is the job of the proxy program to make a complete config.
5948 This config generated must have this extra parameter - _root - root to
5950 use for the backend
5951 And it may have this parameter - _obscure - comma separated strings for
5953 parameters to obscure
5954 If password authentication was used by the client, input to the proxy
5956 process (on STDIN) would look similar to this:
5957 {
5959 "user": "me",
5960 "pass": "mypassword"
5961 }
5962 If public-key authentication was used by the client, input to the proxy
5964 process (on STDIN) would look similar to this:
5965 {
5967 "user": "me",
5968 "public_key": "AAAAB3NzaC1yc2EAAAADAQABAAABAQDuwESFdAe14hVS6omeyX7edc...JQdf"
5969 }
5970 And as an example return this on STDOUT
5972 {
5974 "type": "sftp",
5975 "_root": "",
5976 "_obscure": "pass",
5977 "user": "me",
5978 "pass": "mypassword",
5979 "host": "sftp.example.com"
5980 }
5981 This would mean that an SFTP backend would be created on the fly for
5983 the user and pass/public_key returned in the output to the host given.
5984 Note that since _obscure is set to pass, rclone will obscure the pass
5985 parameter before creating the backend (which is required for sftp back‐
5986 ends).
5987 The program can manipulate the supplied user in any way, for example to
5989 make proxy to many different sftp backends, you could make the user be
5990 user@example.com and then set the host to example.com in the output and
5991 the user to user. For security you'd probably want to restrict the
5992 host to a limited list.
5993 Note that an internal cache is keyed on user so only use that for con‐
5995 figuration, don't use pass or public_key. This also means that if a
5996 user's password or public-key is changed the cache will need to expire
5997 (which takes 5 mins) before it takes effect.
5998 This can be used to build general purpose proxies to any kind of back‐
6000 end that rclone supports.
6001 rclone serve ftp remote:path [flags]
6003 Options
6005 --addr string IPaddress:Port or :Port to bind server to (default "localhost:2121")
6006 --auth-proxy string A program to use to create the backend from the auth
6007 --cert string TLS PEM key (concatenation of certificate and CA certificate)
6008 --dir-cache-time duration Time to cache directory entries for (default 5m0s)
6009 --dir-perms FileMode Directory permissions (default 0777)
6010 --file-perms FileMode File permissions (default 0666)
6011 --gid uint32 Override the gid field set by the filesystem (not supported on Windows) (default 1000)
6012 -h, --help help for ftp
6013 --key string TLS PEM Private key
6014 --no-checksum Don't compare checksums on up/download
6015 --no-modtime Don't read/write the modification time (can speed things up)
6016 --no-seek Don't allow seeking in files
6017 --pass string Password for authentication (empty value allow every password)
6018 --passive-port string Passive port range to use (default "30000-32000")
6019 --poll-interval duration Time to wait between polling for changes, must be smaller than dir-cache-time and only on supported remotes (set 0 to disable) (default 1m0s)
6020 --public-ip string Public IP address to advertise for passive connections
6021 --read-only Only allow read-only access
6022 --uid uint32 Override the uid field set by the filesystem (not supported on Windows) (default 1000)
6023 --umask int Override the permission bits set by the filesystem (not supported on Windows) (default 2)
6024 --user string User name for authentication (default "anonymous")
6025 --vfs-cache-max-age duration Max age of objects in the cache (default 1h0m0s)
6026 --vfs-cache-max-size SizeSuffix Max total size of objects in the cache (default off)
6027 --vfs-cache-mode CacheMode Cache mode off|minimal|writes|full (default off)
6028 --vfs-cache-poll-interval duration Interval to poll the cache for stale objects (default 1m0s)
6029 --vfs-case-insensitive If a file name not found, find a case insensitive match
6030 --vfs-disk-space-total-size SizeSuffix Specify the total space of disk (default off)
6031 --vfs-fast-fingerprint Use fast (less accurate) fingerprints for change detection
6032 --vfs-read-ahead SizeSuffix Extra read ahead over --buffer-size when using cache-mode full
6033 --vfs-read-chunk-size SizeSuffix Read the source objects in chunks (default 128Mi)
6034 --vfs-read-chunk-size-limit SizeSuffix If greater than --vfs-read-chunk-size, double the chunk size after each chunk read, until the limit is reached ('off' is unlimited) (default off)
6035 --vfs-read-wait duration Time to wait for in-sequence read before seeking (default 20ms)
6036 --vfs-used-is-size rclone size Use the rclone size algorithm for Used size
6037 --vfs-write-back duration Time to writeback files after last use when using cache (default 5s)
6038 --vfs-write-wait duration Time to wait for in-sequence write before giving error (default 1s)
6039 See the global flags page (https://rclone.org/flags/) for global op‐
6041 tions not listed here.
6042 SEE ALSO
6044 • rclone serve (https://rclone.org/commands/rclone_serve/) - Serve a
6045 remote over a protocol.
6046
6048 Serve the remote over HTTP.
6049 Synopsis
6051 Run a basic web server to serve a remote over HTTP. This can be viewed
6052 in a web browser or you can make a remote of type http read from it.
6053 You can use the filter flags (e.g. --include, --exclude) to control
6055 what is served.
6056 The server will log errors. Use -v to see access logs.
6058 --bwlimit will be respected for file transfers. Use --stats to control
6060 the stats printing.
6061 Server options
6063 Use --addr to specify which IP address and port the server should lis‐
6064 ten on, eg --addr 1.2.3.4:8000 or --addr :8080 to listen to all IPs.
6065 By default it only listens on localhost. You can use port :0 to let
6066 the OS choose an available port.
6067 If you set --addr to listen on a public or LAN accessible IP address
6069 then using Authentication is advised - see the next section for info.
6070 --server-read-timeout and --server-write-timeout can be used to control
6072 the timeouts on the server. Note that this is the total time for a
6073 transfer.
6074 --max-header-bytes controls the maximum number of bytes the server will
6076 accept in the HTTP header.
6077 --baseurl controls the URL prefix that rclone serves from. By default
6079 rclone will serve from the root. If you used --baseurl "/rclone" then
6080 rclone would serve from a URL starting with "/rclone/". This is useful
6081 if you wish to proxy rclone serve. Rclone automatically inserts lead‐
6082 ing and trailing "/" on --baseurl, so --baseurl "rclone", --baseurl
6083 "/rclone" and --baseurl "/rclone/" are all treated identically.
6084 SSL/TLS
6086 By default this will serve over http. If you want you can serve over
6087 https. You will need to supply the --cert and --key flags. If you
6088 wish to do client side certificate validation then you will need to
6089 supply --client-ca also.
6090 --cert should be a either a PEM encoded certificate or a concatenation
6092 of that with the CA certificate. --key should be the PEM encoded pri‐
6093 vate key and --client-ca should be the PEM encoded client certificate
6094 authority certificate.
6095 --min-tls-version is minimum TLS version that is acceptable. Valid
6097 values are "tls1.0", "tls1.1", "tls1.2" and "tls1.3" (default
6098 "tls1.0").
6099 Template
6101 --template allows a user to specify a custom markup template for HTTP
6102 and WebDAV serve functions. The server exports the following markup to
6103 be used within the template to server pages:
6104 Parameter Description
6106 ──────────────────────────────────────────────────────────────────────────
6107 .Name The full path of a file/directory.
6108 .Title Directory listing of .Name
6109 .Sort The current sort used. This is
6110 changeable via ?sort= parameter
6111 Sort Options:
6112 namedirfirst,name,size,time (de‐
6113 fault namedirfirst)
6114 .Order The current ordering used. This is
6115 changeable via ?order= parameter
6116 Order Options: asc,desc (default
6117 asc)
6118 .Query Currently unused.
6119 .Breadcrumb Allows for creating a relative nav‐
6120 igation
6121 -- .Link The relative to the root link of
6122 the Text.
6123 -- .Text The Name of the directory.
6124 .Entries Information about a specific
6125 file/directory.
6126 -- .URL The 'url' of an entry.
6127 -- .Leaf Currently same as 'URL' but intend‐
6128 ed to be 'just' the name.
6129 -- .IsDir Boolean for if an entry is a direc‐
6131 tory or not.
6132 -- .Size Size in Bytes of the entry.
6133 -- .ModTime The UTC timestamp of an entry.
6134 Authentication
6136 By default this will serve files without needing a login.
6137 You can either use an htpasswd file which can take lots of users, or
6139 set a single username and password with the --user and --pass flags.
6140 Use --htpasswd /path/to/htpasswd to provide an htpasswd file. This is
6142 in standard apache format and supports MD5, SHA1 and BCrypt for basic
6143 authentication. Bcrypt is recommended.
6144 To create an htpasswd file:
6146 touch htpasswd
6148 htpasswd -B htpasswd user
6149 htpasswd -B htpasswd anotherUser
6150 The password file can be updated while rclone is running.
6152 Use --realm to set the authentication realm.
6154 Use --salt to change the password hashing salt from the default.
6156 VFS - Virtual File System
6158 This command uses the VFS layer. This adapts the cloud storage objects
6159 that rclone uses into something which looks much more like a disk fil‐
6160 ing system.
6161 Cloud storage objects have lots of properties which aren't like disk
6163 files - you can't extend them or write to the middle of them, so the
6164 VFS layer has to deal with that. Because there is no one right way of
6165 doing this there are various options explained below.
6166 The VFS layer also implements a directory cache - this caches info
6168 about files and directories (but not the data) in memory.
6169 VFS Directory Cache
6171 Using the --dir-cache-time flag, you can control how long a directory
6172 should be considered up to date and not refreshed from the backend.
6173 Changes made through the VFS will appear immediately or invalidate the
6174 cache.
6175 --dir-cache-time duration Time to cache directory entries for (default 5m0s)
6177 --poll-interval duration Time to wait between polling for changes. Must be smaller than dir-cache-time. Only on supported remotes. Set to 0 to disable (default 1m0s)
6178 However, changes made directly on the cloud storage by the web inter‐
6180 face or a different copy of rclone will only be picked up once the di‐
6181 rectory cache expires if the backend configured does not support
6182 polling for changes. If the backend supports polling, changes will be
6183 picked up within the polling interval.
6184 You can send a SIGHUP signal to rclone for it to flush all directory
6186 caches, regardless of how old they are. Assuming only one rclone in‐
6187 stance is running, you can reset the cache like this:
6188 kill -SIGHUP $(pidof rclone)
6190 If you configure rclone with a remote control then you can use rclone
6192 rc to flush the whole directory cache:
6193 rclone rc vfs/forget
6195 Or individual files or directories:
6197 rclone rc vfs/forget file=path/to/file dir=path/to/dir
6199 VFS File Buffering
6201 The --buffer-size flag determines the amount of memory, that will be
6202 used to buffer data in advance.
6203 Each open file will try to keep the specified amount of data in memory
6205 at all times. The buffered data is bound to one open file and won't be
6206 shared.
6207 This flag is a upper limit for the used memory per open file. The buf‐
6209 fer will only use memory for data that is downloaded but not not yet
6210 read. If the buffer is empty, only a small amount of memory will be
6211 used.
6212 The maximum memory used by rclone for buffering can be up to --buffer-
6214 size * open files.
6215 VFS File Caching
6217 These flags control the VFS file caching options. File caching is nec‐
6218 essary to make the VFS layer appear compatible with a normal file sys‐
6219 tem. It can be disabled at the cost of some compatibility.
6220 For example you'll need to enable VFS caching if you want to read and
6222 write simultaneously to a file. See below for more details.
6223 Note that the VFS cache is separate from the cache backend and you may
6225 find that you need one or the other or both.
6226 --cache-dir string Directory rclone will use for caching.
6228 --vfs-cache-mode CacheMode Cache mode off|minimal|writes|full (default off)
6229 --vfs-cache-max-age duration Max age of objects in the cache (default 1h0m0s)
6230 --vfs-cache-max-size SizeSuffix Max total size of objects in the cache (default off)
6231 --vfs-cache-poll-interval duration Interval to poll the cache for stale objects (default 1m0s)
6232 --vfs-write-back duration Time to writeback files after last use when using cache (default 5s)
6233 If run with -vv rclone will print the location of the file cache. The
6235 files are stored in the user cache file area which is OS dependent but
6236 can be controlled with --cache-dir or setting the appropriate environ‐
6237 ment variable.
6238 The cache has 4 different modes selected by --vfs-cache-mode. The
6240 higher the cache mode the more compatible rclone becomes at the cost of
6241 using disk space.
6242 Note that files are written back to the remote only when they are
6244 closed and if they haven't been accessed for --vfs-write-back seconds.
6245 If rclone is quit or dies with files that haven't been uploaded, these
6246 will be uploaded next time rclone is run with the same flags.
6247 If using --vfs-cache-max-size note that the cache may exceed this size
6249 for two reasons. Firstly because it is only checked every --vfs-cache-
6250 poll-interval. Secondly because open files cannot be evicted from the
6251 cache.
6252 You should not run two copies of rclone using the same VFS cache with
6254 the same or overlapping remotes if using --vfs-cache-mode > off. This
6255 can potentially cause data corruption if you do. You can work around
6256 this by giving each rclone its own cache hierarchy with --cache-dir.
6257 You don't need to worry about this if the remotes in use don't overlap.
6258 --vfs-cache-mode off
6260 In this mode (the default) the cache will read directly from the remote
6261 and write directly to the remote without caching anything on disk.
6262 This will mean some operations are not possible
6264 • Files can't be opened for both read AND write
6266 • Files opened for write can't be seeked
6268 • Existing files opened for write must have O_TRUNC set
6270 • Files open for read with O_TRUNC will be opened write only
6272 • Files open for write only will behave as if O_TRUNC was supplied
6274 • Open modes O_APPEND, O_TRUNC are ignored
6276 • If an upload fails it can't be retried
6278 --vfs-cache-mode minimal
6280 This is very similar to "off" except that files opened for read AND
6281 write will be buffered to disk. This means that files opened for write
6282 will be a lot more compatible, but uses the minimal disk space.
6283 These operations are not possible
6285 • Files opened for write only can't be seeked
6287 • Existing files opened for write must have O_TRUNC set
6289 • Files opened for write only will ignore O_APPEND, O_TRUNC
6291 • If an upload fails it can't be retried
6293 --vfs-cache-mode writes
6295 In this mode files opened for read only are still read directly from
6296 the remote, write only and read/write files are buffered to disk first.
6297 This mode should support all normal file system operations.
6299 If an upload fails it will be retried at exponentially increasing in‐
6301 tervals up to 1 minute.
6302 --vfs-cache-mode full
6304 In this mode all reads and writes are buffered to and from disk. When
6305 data is read from the remote this is buffered to disk as well.
6306 In this mode the files in the cache will be sparse files and rclone
6308 will keep track of which bits of the files it has downloaded.
6309 So if an application only reads the starts of each file, then rclone
6311 will only buffer the start of the file. These files will appear to be
6312 their full size in the cache, but they will be sparse files with only
6313 the data that has been downloaded present in them.
6314 This mode should support all normal file system operations and is oth‐
6316 erwise identical to --vfs-cache-mode writes.
6317 When reading a file rclone will read --buffer-size plus --vfs-read-
6319 ahead bytes ahead. The --buffer-size is buffered in memory whereas the
6320 --vfs-read-ahead is buffered on disk.
6321 When using this mode it is recommended that --buffer-size is not set
6323 too large and --vfs-read-ahead is set large if required.
6324 IMPORTANT not all file systems support sparse files. In particular
6326 FAT/exFAT do not. Rclone will perform very badly if the cache directo‐
6327 ry is on a filesystem which doesn't support sparse files and it will
6328 log an ERROR message if one is detected.
6329 Fingerprinting
6331 Various parts of the VFS use fingerprinting to see if a local file copy
6332 has changed relative to a remote file. Fingerprints are made from:
6333 • size
6335 • modification time
6337 • hash
6339 where available on an object.
6341 On some backends some of these attributes are slow to read (they take
6343 an extra API call per object, or extra work per object).
6344 For example hash is slow with the local and sftp backends as they have
6346 to read the entire file and hash it, and modtime is slow with the s3,
6347 swift, ftp and qinqstor backends because they need to do an extra API
6348 call to fetch it.
6349 If you use the --vfs-fast-fingerprint flag then rclone will not include
6351 the slow operations in the fingerprint. This makes the fingerprinting
6352 less accurate but much faster and will improve the opening time of
6353 cached files.
6354 If you are running a vfs cache over local, s3 or swift backends then
6356 using this flag is recommended.
6357 Note that if you change the value of this flag, the fingerprints of the
6359 files in the cache may be invalidated and the files will need to be
6360 downloaded again.
6361 VFS Chunked Reading
6363 When rclone reads files from a remote it reads them in chunks. This
6364 means that rather than requesting the whole file rclone reads the chunk
6365 specified. This can reduce the used download quota for some remotes by
6366 requesting only chunks from the remote that are actually read, at the
6367 cost of an increased number of requests.
6368 These flags control the chunking:
6370 --vfs-read-chunk-size SizeSuffix Read the source objects in chunks (default 128M)
6372 --vfs-read-chunk-size-limit SizeSuffix Max chunk doubling size (default off)
6373 Rclone will start reading a chunk of size --vfs-read-chunk-size, and
6375 then double the size for each read. When --vfs-read-chunk-size-limit
6376 is specified, and greater than --vfs-read-chunk-size, the chunk size
6377 for each open file will get doubled only until the specified value is
6378 reached. If the value is "off", which is the default, the limit is
6379 disabled and the chunk size will grow indefinitely.
6380 With --vfs-read-chunk-size 100M and --vfs-read-chunk-size-limit 0 the
6382 following parts will be downloaded: 0-100M, 100M-200M, 200M-300M,
6383 300M-400M and so on. When --vfs-read-chunk-size-limit 500M is speci‐
6384 fied, the result would be 0-100M, 100M-300M, 300M-700M, 700M-1200M,
6385 1200M-1700M and so on.
6386 Setting --vfs-read-chunk-size to 0 or "off" disables chunked reading.
6388 VFS Performance
6390 These flags may be used to enable/disable features of the VFS for per‐
6391 formance or other reasons. See also the chunked reading feature.
6392 In particular S3 and Swift benefit hugely from the --no-modtime flag
6394 (or use --use-server-modtime for a slightly different effect) as each
6395 read of the modification time takes a transaction.
6396 --no-checksum Don't compare checksums on up/download.
6398 --no-modtime Don't read/write the modification time (can speed things up).
6399 --no-seek Don't allow seeking in files.
6400 --read-only Only allow read-only access.
6401 Sometimes rclone is delivered reads or writes out of order. Rather
6403 than seeking rclone will wait a short time for the in sequence read or
6404 write to come in. These flags only come into effect when not using an
6405 on disk cache file.
6406 --vfs-read-wait duration Time to wait for in-sequence read before seeking (default 20ms)
6408 --vfs-write-wait duration Time to wait for in-sequence write before giving error (default 1s)
6409 When using VFS write caching (--vfs-cache-mode with value writes or
6411 full), the global flag --transfers can be set to adjust the number of
6412 parallel uploads of modified files from the cache (the related global
6413 flag --checkers has no effect on the VFS).
6414 --transfers int Number of file transfers to run in parallel (default 4)
6416 VFS Case Sensitivity
6418 Linux file systems are case-sensitive: two files can differ only by
6419 case, and the exact case must be used when opening a file.
6420 File systems in modern Windows are case-insensitive but case-preserv‐
6422 ing: although existing files can be opened using any case, the exact
6423 case used to create the file is preserved and available for programs to
6424 query. It is not allowed for two files in the same directory to differ
6425 only by case.
6426 Usually file systems on macOS are case-insensitive. It is possible to
6428 make macOS file systems case-sensitive but that is not the default.
6429 The --vfs-case-insensitive VFS flag controls how rclone handles these
6431 two cases. If its value is "false", rclone passes file names to the
6432 remote as-is. If the flag is "true" (or appears without a value on the
6433 command line), rclone may perform a "fixup" as explained below.
6434 The user may specify a file name to open/delete/rename/etc with a case
6436 different than what is stored on the remote. If an argument refers to
6437 an existing file with exactly the same name, then the case of the ex‐
6438 isting file on the disk will be used. However, if a file name with ex‐
6439 actly the same name is not found but a name differing only by case ex‐
6440 ists, rclone will transparently fixup the name. This fixup happens on‐
6441 ly when an existing file is requested. Case sensitivity of file names
6442 created anew by rclone is controlled by the underlying remote.
6443 Note that case sensitivity of the operating system running rclone (the
6445 target) may differ from case sensitivity of a file system presented by
6446 rclone (the source). The flag controls whether "fixup" is performed to
6447 satisfy the target.
6448 If the flag is not provided on the command line, then its default value
6450 depends on the operating system where rclone runs: "true" on Windows
6451 and macOS, "false" otherwise. If the flag is provided without a value,
6452 then it is "true".
6453 VFS Disk Options
6455 This flag allows you to manually set the statistics about the filing
6456 system. It can be useful when those statistics cannot be read correct‐
6457 ly automatically.
6458 --vfs-disk-space-total-size Manually set the total disk space size (example: 256G, default: -1)
6460 Alternate report of used bytes
6462 Some backends, most notably S3, do not report the amount of bytes used.
6463 If you need this information to be available when running df on the
6464 filesystem, then pass the flag --vfs-used-is-size to rclone. With this
6465 flag set, instead of relying on the backend to report this information,
6466 rclone will scan the whole remote similar to rclone size and compute
6467 the total used space itself.
6468 WARNING. Contrary to rclone size, this flag ignores filters so that the
6470 result is accurate. However, this is very inefficient and may cost
6471 lots of API calls resulting in extra charges. Use it as a last resort
6472 and only with caching.
6473 rclone serve http remote:path [flags]
6475 Options
6477 --addr string IPaddress:Port or :Port to bind server to (default "127.0.0.1:8080")
6478 --baseurl string Prefix for URLs - leave blank for root
6479 --cert string SSL PEM key (concatenation of certificate and CA certificate)
6480 --client-ca string Client certificate authority to verify clients with
6481 --dir-cache-time duration Time to cache directory entries for (default 5m0s)
6482 --dir-perms FileMode Directory permissions (default 0777)
6483 --file-perms FileMode File permissions (default 0666)
6484 --gid uint32 Override the gid field set by the filesystem (not supported on Windows) (default 1000)
6485 -h, --help help for http
6486 --htpasswd string A htpasswd file - if not provided no authentication is done
6487 --key string SSL PEM Private key
6488 --max-header-bytes int Maximum size of request header (default 4096)
6489 --min-tls-version string Minimum TLS version that is acceptable (default "tls1.0")
6490 --no-checksum Don't compare checksums on up/download
6491 --no-modtime Don't read/write the modification time (can speed things up)
6492 --no-seek Don't allow seeking in files
6493 --pass string Password for authentication
6494 --poll-interval duration Time to wait between polling for changes, must be smaller than dir-cache-time and only on supported remotes (set 0 to disable) (default 1m0s)
6495 --read-only Only allow read-only access
6496 --realm string Realm for authentication
6497 --salt string Password hashing salt (default "dlPL2MqE")
6498 --server-read-timeout duration Timeout for server reading data (default 1h0m0s)
6499 --server-write-timeout duration Timeout for server writing data (default 1h0m0s)
6500 --template string User-specified template
6501 --uid uint32 Override the uid field set by the filesystem (not supported on Windows) (default 1000)
6502 --umask int Override the permission bits set by the filesystem (not supported on Windows) (default 2)
6503 --user string User name for authentication
6504 --vfs-cache-max-age duration Max age of objects in the cache (default 1h0m0s)
6505 --vfs-cache-max-size SizeSuffix Max total size of objects in the cache (default off)
6506 --vfs-cache-mode CacheMode Cache mode off|minimal|writes|full (default off)
6507 --vfs-cache-poll-interval duration Interval to poll the cache for stale objects (default 1m0s)
6508 --vfs-case-insensitive If a file name not found, find a case insensitive match
6509 --vfs-disk-space-total-size SizeSuffix Specify the total space of disk (default off)
6510 --vfs-fast-fingerprint Use fast (less accurate) fingerprints for change detection
6511 --vfs-read-ahead SizeSuffix Extra read ahead over --buffer-size when using cache-mode full
6512 --vfs-read-chunk-size SizeSuffix Read the source objects in chunks (default 128Mi)
6513 --vfs-read-chunk-size-limit SizeSuffix If greater than --vfs-read-chunk-size, double the chunk size after each chunk read, until the limit is reached ('off' is unlimited) (default off)
6514 --vfs-read-wait duration Time to wait for in-sequence read before seeking (default 20ms)
6515 --vfs-used-is-size rclone size Use the rclone size algorithm for Used size
6516 --vfs-write-back duration Time to writeback files after last use when using cache (default 5s)
6517 --vfs-write-wait duration Time to wait for in-sequence write before giving error (default 1s)
6518 See the global flags page (https://rclone.org/flags/) for global op‐
6520 tions not listed here.
6521 SEE ALSO
6523 • rclone serve (https://rclone.org/commands/rclone_serve/) - Serve a
6524 remote over a protocol.
6525
6527 Serve the remote for restic's REST API.
6528 Synopsis
6530 Run a basic web server to serve a remove over restic's REST backend API
6531 over HTTP. This allows restic to use rclone as a data storage mecha‐
6532 nism for cloud providers that restic does not support directly.
6533 Restic (https://restic.net/) is a command-line program for doing back‐
6535 ups.
6536 The server will log errors. Use -v to see access logs.
6538 --bwlimit will be respected for file transfers. Use --stats to control
6540 the stats printing.
6541 Setting up rclone for use by restic
6543 First set up a remote for your chosen cloud provider
6544 (https://rclone.org/docs/#configure).
6545 Once you have set up the remote, check it is working with, for example
6547 "rclone lsd remote:". You may have called the remote something other
6548 than "remote:" - just substitute whatever you called it in the follow‐
6549 ing instructions.
6550 Now start the rclone restic server
6552 rclone serve restic -v remote:backup
6554 Where you can replace "backup" in the above by whatever path in the re‐
6556 mote you wish to use.
6557 By default this will serve on "localhost:8080" you can change this with
6559 use of the --addr flag.
6560 You might wish to start this server on boot.
6562 Adding --cache-objects=false will cause rclone to stop caching objects
6564 returned from the List call. Caching is normally desirable as it
6565 speeds up downloading objects, saves transactions and uses very little
6566 memory.
6567 Setting up restic to use rclone
6569 Now you can follow the restic instructions (http://restic.readthedo‐
6570 cs.io/en/latest/030_preparing_a_new_repo.html#rest-server) on setting
6571 up restic.
6572 Note that you will need restic 0.8.2 or later to interoperate with
6574 rclone.
6575 For the example above you will want to use "http://localhost:8080/" as
6577 the URL for the REST server.
6578 For example:
6580 $ export RESTIC_REPOSITORY=rest:http://localhost:8080/
6582 $ export RESTIC_PASSWORD=yourpassword
6583 $ restic init
6584 created restic backend 8b1a4b56ae at rest:http://localhost:8080/
6585 Please note that knowledge of your password is required to access
6587 the repository. Losing your password means that your data is
6588 irrecoverably lost.
6589 $ restic backup /path/to/files/to/backup
6590 scan [/path/to/files/to/backup]
6591 scanned 189 directories, 312 files in 0:00
6592 [0:00] 100.00% 38.128 MiB / 38.128 MiB 501 / 501 items 0 errors ETA 0:00
6593 duration: 0:00
6594 snapshot 45c8fdd8 saved
6595 Multiple repositories
6597 Note that you can use the endpoint to host multiple repositories. Do
6598 this by adding a directory name or path after the URL. Note that these
6599 must end with /. Eg
6600 $ export RESTIC_REPOSITORY=rest:http://localhost:8080/user1repo/
6602 # backup user1 stuff
6603 $ export RESTIC_REPOSITORY=rest:http://localhost:8080/user2repo/
6604 # backup user2 stuff
6605 Private repositories
6607 The--private-repos flag can be used to limit users to repositories
6608 starting with a path of /<username>/.
6609 Server options
6611 Use --addr to specify which IP address and port the server should lis‐
6612 ten on, e.g. --addr 1.2.3.4:8000 or --addr :8080 to listen to all IPs.
6613 By default it only listens on localhost. You can use port :0 to let
6614 the OS choose an available port.
6615 If you set --addr to listen on a public or LAN accessible IP address
6617 then using Authentication is advised - see the next section for info.
6618 --server-read-timeout and --server-write-timeout can be used to control
6620 the timeouts on the server. Note that this is the total time for a
6621 transfer.
6622 --max-header-bytes controls the maximum number of bytes the server will
6624 accept in the HTTP header.
6625 --baseurl controls the URL prefix that rclone serves from. By default
6627 rclone will serve from the root. If you used --baseurl "/rclone" then
6628 rclone would serve from a URL starting with "/rclone/". This is useful
6629 if you wish to proxy rclone serve. Rclone automatically inserts lead‐
6630 ing and trailing "/" on --baseurl, so --baseurl "rclone", --baseurl
6631 "/rclone" and --baseurl "/rclone/" are all treated identically.
6632 --template allows a user to specify a custom markup template for HTTP
6634 and WebDAV serve functions. The server exports the following markup to
6635 be used within the template to server pages:
6636 Parameter Description
6638 ──────────────────────────────────────────────────────────────────────────
6639 .Name The full path of a file/directory.
6640 .Title Directory listing of .Name
6641 .Sort The current sort used. This is
6642 changeable via ?sort= parameter
6643 Sort Options:
6644 namedirfirst,name,size,time (de‐
6645 fault namedirfirst)
6646 .Order The current ordering used. This is
6647 changeable via ?order= parameter
6648 Order Options: asc,desc (default
6649 asc)
6650 .Query Currently unused.
6651 .Breadcrumb Allows for creating a relative nav‐
6652 igation
6653 -- .Link The relative to the root link of
6654 the Text.
6655 -- .Text The Name of the directory.
6656 .Entries Information about a specific
6657 file/directory.
6658 -- .URL The 'url' of an entry.
6659 -- .Leaf Currently same as 'URL' but intend‐
6660 ed to be 'just' the name.
6661 -- .IsDir Boolean for if an entry is a direc‐
6662 tory or not.
6663 -- .Size Size in Bytes of the entry.
6664 -- .ModTime The UTC timestamp of an entry.
6665 Authentication
6667 By default this will serve files without needing a login.
6668 You can either use an htpasswd file which can take lots of users, or
6670 set a single username and password with the --user and --pass flags.
6671 Use --htpasswd /path/to/htpasswd to provide an htpasswd file. This is
6673 in standard apache format and supports MD5, SHA1 and BCrypt for basic
6674 authentication. Bcrypt is recommended.
6675 To create an htpasswd file:
6677 touch htpasswd
6679 htpasswd -B htpasswd user
6680 htpasswd -B htpasswd anotherUser
6681 The password file can be updated while rclone is running.
6683 Use --realm to set the authentication realm.
6685 SSL/TLS
6687 By default this will serve over HTTP. If you want you can serve over
6688 HTTPS. You will need to supply the --cert and --key flags. If you
6689 wish to do client side certificate validation then you will need to
6690 supply --client-ca also.
6691 --cert should be either a PEM encoded certificate or a concatenation of
6693 that with the CA certificate. --key should be the PEM encoded private
6694 key and --client-ca should be the PEM encoded client certificate au‐
6695 thority certificate.
6696 --min-tls-version is minimum TLS version that is acceptable. Valid
6698 values are "tls1.0", "tls1.1", "tls1.2" and "tls1.3" (default
6699 "tls1.0").
6700 rclone serve restic remote:path [flags]
6702 Options
6704 --addr string IPaddress:Port or :Port to bind server to (default "localhost:8080")
6705 --append-only Disallow deletion of repository data
6706 --baseurl string Prefix for URLs - leave blank for root
6707 --cache-objects Cache listed objects (default true)
6708 --cert string SSL PEM key (concatenation of certificate and CA certificate)
6709 --client-ca string Client certificate authority to verify clients with
6710 -h, --help help for restic
6711 --htpasswd string htpasswd file - if not provided no authentication is done
6712 --key string SSL PEM Private key
6713 --max-header-bytes int Maximum size of request header (default 4096)
6714 --min-tls-version string Minimum TLS version that is acceptable (default "tls1.0")
6715 --pass string Password for authentication
6716 --private-repos Users can only access their private repo
6717 --realm string Realm for authentication (default "rclone")
6718 --server-read-timeout duration Timeout for server reading data (default 1h0m0s)
6719 --server-write-timeout duration Timeout for server writing data (default 1h0m0s)
6720 --stdio Run an HTTP2 server on stdin/stdout
6721 --template string User-specified template
6722 --user string User name for authentication
6723 See the global flags page (https://rclone.org/flags/) for global op‐
6725 tions not listed here.
6726 SEE ALSO
6728 • rclone serve (https://rclone.org/commands/rclone_serve/) - Serve a
6729 remote over a protocol.
6730
6732 Serve the remote over SFTP.
6733 Synopsis
6735 Run an SFTP server to serve a remote over SFTP. This can be used with
6736 an SFTP client or you can make a remote of type sftp to use with it.
6737 You can use the filter flags (e.g. --include, --exclude) to control
6739 what is served.
6740 The server will respond to a small number of shell commands, mainly
6742 md5sum, sha1sum and df, which enable it to provide support for check‐
6743 sums and the about feature when accessed from an sftp remote.
6744 Note that this server uses standard 32 KiB packet payload size, which
6746 means you must not configure the client to expect anything else, e.g.
6747 with the chunk_size (https://rclone.org/sftp/#sftp-chunk-size) option
6748 on an sftp remote.
6749 The server will log errors. Use -v to see access logs.
6751 --bwlimit will be respected for file transfers. Use --stats to control
6753 the stats printing.
6754 You must provide some means of authentication, either with --us‐
6756 er/--pass, an authorized keys file (specify location with --authorized-
6757 keys - the default is the same as ssh), an --auth-proxy, or set the
6758 --no-auth flag for no authentication when logging in.
6759 If you don't supply a host --key then rclone will generate rsa, ecdsa
6761 and ed25519 variants, and cache them for later use in rclone's cache
6762 directory (see rclone help flags cache-dir) in the "serve-sftp" direc‐
6763 tory.
6764 By default the server binds to localhost:2022 - if you want it to be
6766 reachable externally then supply --addr :2022 for example.
6767 Note that the default of --vfs-cache-mode off is fine for the rclone
6769 sftp backend, but it may not be with other SFTP clients.
6770 If --stdio is specified, rclone will serve SFTP over stdio, which can
6772 be used with sshd via ~/.ssh/authorized_keys, for example:
6773 restrict,command="rclone serve sftp --stdio ./photos" ssh-rsa ...
6775 On the client you need to set --transfers 1 when using --stdio. Other‐
6777 wise multiple instances of the rclone server are started by OpenSSH
6778 which can lead to "corrupted on transfer" errors. This is the case be‐
6779 cause the client chooses indiscriminately which server to send commands
6780 to while the servers all have different views of the state of the fil‐
6781 ing system.
6782 The "restrict" in authorized_keys prevents SHA1SUMs and MD5SUMs from
6784 beeing used. Omitting "restrict" and using --sftp-path-override to en‐
6785 able checksumming is possible but less secure and you could use the
6786 SFTP server provided by OpenSSH in this case.
6787 VFS - Virtual File System
6789 This command uses the VFS layer. This adapts the cloud storage objects
6790 that rclone uses into something which looks much more like a disk fil‐
6791 ing system.
6792 Cloud storage objects have lots of properties which aren't like disk
6794 files - you can't extend them or write to the middle of them, so the
6795 VFS layer has to deal with that. Because there is no one right way of
6796 doing this there are various options explained below.
6797 The VFS layer also implements a directory cache - this caches info
6799 about files and directories (but not the data) in memory.
6800 VFS Directory Cache
6802 Using the --dir-cache-time flag, you can control how long a directory
6803 should be considered up to date and not refreshed from the backend.
6804 Changes made through the VFS will appear immediately or invalidate the
6805 cache.
6806 --dir-cache-time duration Time to cache directory entries for (default 5m0s)
6808 --poll-interval duration Time to wait between polling for changes. Must be smaller than dir-cache-time. Only on supported remotes. Set to 0 to disable (default 1m0s)
6809 However, changes made directly on the cloud storage by the web inter‐
6811 face or a different copy of rclone will only be picked up once the di‐
6812 rectory cache expires if the backend configured does not support
6813 polling for changes. If the backend supports polling, changes will be
6814 picked up within the polling interval.
6815 You can send a SIGHUP signal to rclone for it to flush all directory
6817 caches, regardless of how old they are. Assuming only one rclone in‐
6818 stance is running, you can reset the cache like this:
6819 kill -SIGHUP $(pidof rclone)
6821 If you configure rclone with a remote control then you can use rclone
6823 rc to flush the whole directory cache:
6824 rclone rc vfs/forget
6826 Or individual files or directories:
6828 rclone rc vfs/forget file=path/to/file dir=path/to/dir
6830 VFS File Buffering
6832 The --buffer-size flag determines the amount of memory, that will be
6833 used to buffer data in advance.
6834 Each open file will try to keep the specified amount of data in memory
6836 at all times. The buffered data is bound to one open file and won't be
6837 shared.
6838 This flag is a upper limit for the used memory per open file. The buf‐
6840 fer will only use memory for data that is downloaded but not not yet
6841 read. If the buffer is empty, only a small amount of memory will be
6842 used.
6843 The maximum memory used by rclone for buffering can be up to --buffer-
6845 size * open files.
6846 VFS File Caching
6848 These flags control the VFS file caching options. File caching is nec‐
6849 essary to make the VFS layer appear compatible with a normal file sys‐
6850 tem. It can be disabled at the cost of some compatibility.
6851 For example you'll need to enable VFS caching if you want to read and
6853 write simultaneously to a file. See below for more details.
6854 Note that the VFS cache is separate from the cache backend and you may
6856 find that you need one or the other or both.
6857 --cache-dir string Directory rclone will use for caching.
6859 --vfs-cache-mode CacheMode Cache mode off|minimal|writes|full (default off)
6860 --vfs-cache-max-age duration Max age of objects in the cache (default 1h0m0s)
6861 --vfs-cache-max-size SizeSuffix Max total size of objects in the cache (default off)
6862 --vfs-cache-poll-interval duration Interval to poll the cache for stale objects (default 1m0s)
6863 --vfs-write-back duration Time to writeback files after last use when using cache (default 5s)
6864 If run with -vv rclone will print the location of the file cache. The
6866 files are stored in the user cache file area which is OS dependent but
6867 can be controlled with --cache-dir or setting the appropriate environ‐
6868 ment variable.
6869 The cache has 4 different modes selected by --vfs-cache-mode. The
6871 higher the cache mode the more compatible rclone becomes at the cost of
6872 using disk space.
6873 Note that files are written back to the remote only when they are
6875 closed and if they haven't been accessed for --vfs-write-back seconds.
6876 If rclone is quit or dies with files that haven't been uploaded, these
6877 will be uploaded next time rclone is run with the same flags.
6878 If using --vfs-cache-max-size note that the cache may exceed this size
6880 for two reasons. Firstly because it is only checked every --vfs-cache-
6881 poll-interval. Secondly because open files cannot be evicted from the
6882 cache.
6883 You should not run two copies of rclone using the same VFS cache with
6885 the same or overlapping remotes if using --vfs-cache-mode > off. This
6886 can potentially cause data corruption if you do. You can work around
6887 this by giving each rclone its own cache hierarchy with --cache-dir.
6888 You don't need to worry about this if the remotes in use don't overlap.
6889 --vfs-cache-mode off
6891 In this mode (the default) the cache will read directly from the remote
6892 and write directly to the remote without caching anything on disk.
6893 This will mean some operations are not possible
6895 • Files can't be opened for both read AND write
6897 • Files opened for write can't be seeked
6899 • Existing files opened for write must have O_TRUNC set
6901 • Files open for read with O_TRUNC will be opened write only
6903 • Files open for write only will behave as if O_TRUNC was supplied
6905 • Open modes O_APPEND, O_TRUNC are ignored
6907 • If an upload fails it can't be retried
6909 --vfs-cache-mode minimal
6911 This is very similar to "off" except that files opened for read AND
6912 write will be buffered to disk. This means that files opened for write
6913 will be a lot more compatible, but uses the minimal disk space.
6914 These operations are not possible
6916 • Files opened for write only can't be seeked
6918 • Existing files opened for write must have O_TRUNC set
6920 • Files opened for write only will ignore O_APPEND, O_TRUNC
6922 • If an upload fails it can't be retried
6924 --vfs-cache-mode writes
6926 In this mode files opened for read only are still read directly from
6927 the remote, write only and read/write files are buffered to disk first.
6928 This mode should support all normal file system operations.
6930 If an upload fails it will be retried at exponentially increasing in‐
6932 tervals up to 1 minute.
6933 --vfs-cache-mode full
6935 In this mode all reads and writes are buffered to and from disk. When
6936 data is read from the remote this is buffered to disk as well.
6937 In this mode the files in the cache will be sparse files and rclone
6939 will keep track of which bits of the files it has downloaded.
6940 So if an application only reads the starts of each file, then rclone
6942 will only buffer the start of the file. These files will appear to be
6943 their full size in the cache, but they will be sparse files with only
6944 the data that has been downloaded present in them.
6945 This mode should support all normal file system operations and is oth‐
6947 erwise identical to --vfs-cache-mode writes.
6948 When reading a file rclone will read --buffer-size plus --vfs-read-
6950 ahead bytes ahead. The --buffer-size is buffered in memory whereas the
6951 --vfs-read-ahead is buffered on disk.
6952 When using this mode it is recommended that --buffer-size is not set
6954 too large and --vfs-read-ahead is set large if required.
6955 IMPORTANT not all file systems support sparse files. In particular
6957 FAT/exFAT do not. Rclone will perform very badly if the cache directo‐
6958 ry is on a filesystem which doesn't support sparse files and it will
6959 log an ERROR message if one is detected.
6960 Fingerprinting
6962 Various parts of the VFS use fingerprinting to see if a local file copy
6963 has changed relative to a remote file. Fingerprints are made from:
6964 • size
6966 • modification time
6968 • hash
6970 where available on an object.
6972 On some backends some of these attributes are slow to read (they take
6974 an extra API call per object, or extra work per object).
6975 For example hash is slow with the local and sftp backends as they have
6977 to read the entire file and hash it, and modtime is slow with the s3,
6978 swift, ftp and qinqstor backends because they need to do an extra API
6979 call to fetch it.
6980 If you use the --vfs-fast-fingerprint flag then rclone will not include
6982 the slow operations in the fingerprint. This makes the fingerprinting
6983 less accurate but much faster and will improve the opening time of
6984 cached files.
6985 If you are running a vfs cache over local, s3 or swift backends then
6987 using this flag is recommended.
6988 Note that if you change the value of this flag, the fingerprints of the
6990 files in the cache may be invalidated and the files will need to be
6991 downloaded again.
6992 VFS Chunked Reading
6994 When rclone reads files from a remote it reads them in chunks. This
6995 means that rather than requesting the whole file rclone reads the chunk
6996 specified. This can reduce the used download quota for some remotes by
6997 requesting only chunks from the remote that are actually read, at the
6998 cost of an increased number of requests.
6999 These flags control the chunking:
7001 --vfs-read-chunk-size SizeSuffix Read the source objects in chunks (default 128M)
7003 --vfs-read-chunk-size-limit SizeSuffix Max chunk doubling size (default off)
7004 Rclone will start reading a chunk of size --vfs-read-chunk-size, and
7006 then double the size for each read. When --vfs-read-chunk-size-limit
7007 is specified, and greater than --vfs-read-chunk-size, the chunk size
7008 for each open file will get doubled only until the specified value is
7009 reached. If the value is "off", which is the default, the limit is
7010 disabled and the chunk size will grow indefinitely.
7011 With --vfs-read-chunk-size 100M and --vfs-read-chunk-size-limit 0 the
7013 following parts will be downloaded: 0-100M, 100M-200M, 200M-300M,
7014 300M-400M and so on. When --vfs-read-chunk-size-limit 500M is speci‐
7015 fied, the result would be 0-100M, 100M-300M, 300M-700M, 700M-1200M,
7016 1200M-1700M and so on.
7017 Setting --vfs-read-chunk-size to 0 or "off" disables chunked reading.
7019 VFS Performance
7021 These flags may be used to enable/disable features of the VFS for per‐
7022 formance or other reasons. See also the chunked reading feature.
7023 In particular S3 and Swift benefit hugely from the --no-modtime flag
7025 (or use --use-server-modtime for a slightly different effect) as each
7026 read of the modification time takes a transaction.
7027 --no-checksum Don't compare checksums on up/download.
7029 --no-modtime Don't read/write the modification time (can speed things up).
7030 --no-seek Don't allow seeking in files.
7031 --read-only Only allow read-only access.
7032 Sometimes rclone is delivered reads or writes out of order. Rather
7034 than seeking rclone will wait a short time for the in sequence read or
7035 write to come in. These flags only come into effect when not using an
7036 on disk cache file.
7037 --vfs-read-wait duration Time to wait for in-sequence read before seeking (default 20ms)
7039 --vfs-write-wait duration Time to wait for in-sequence write before giving error (default 1s)
7040 When using VFS write caching (--vfs-cache-mode with value writes or
7042 full), the global flag --transfers can be set to adjust the number of
7043 parallel uploads of modified files from the cache (the related global
7044 flag --checkers has no effect on the VFS).
7045 --transfers int Number of file transfers to run in parallel (default 4)
7047 VFS Case Sensitivity
7049 Linux file systems are case-sensitive: two files can differ only by
7050 case, and the exact case must be used when opening a file.
7051 File systems in modern Windows are case-insensitive but case-preserv‐
7053 ing: although existing files can be opened using any case, the exact
7054 case used to create the file is preserved and available for programs to
7055 query. It is not allowed for two files in the same directory to differ
7056 only by case.
7057 Usually file systems on macOS are case-insensitive. It is possible to
7059 make macOS file systems case-sensitive but that is not the default.
7060 The --vfs-case-insensitive VFS flag controls how rclone handles these
7062 two cases. If its value is "false", rclone passes file names to the
7063 remote as-is. If the flag is "true" (or appears without a value on the
7064 command line), rclone may perform a "fixup" as explained below.
7065 The user may specify a file name to open/delete/rename/etc with a case
7067 different than what is stored on the remote. If an argument refers to
7068 an existing file with exactly the same name, then the case of the ex‐
7069 isting file on the disk will be used. However, if a file name with ex‐
7070 actly the same name is not found but a name differing only by case ex‐
7071 ists, rclone will transparently fixup the name. This fixup happens on‐
7072 ly when an existing file is requested. Case sensitivity of file names
7073 created anew by rclone is controlled by the underlying remote.
7074 Note that case sensitivity of the operating system running rclone (the
7076 target) may differ from case sensitivity of a file system presented by
7077 rclone (the source). The flag controls whether "fixup" is performed to
7078 satisfy the target.
7079 If the flag is not provided on the command line, then its default value
7081 depends on the operating system where rclone runs: "true" on Windows
7082 and macOS, "false" otherwise. If the flag is provided without a value,
7083 then it is "true".
7084 VFS Disk Options
7086 This flag allows you to manually set the statistics about the filing
7087 system. It can be useful when those statistics cannot be read correct‐
7088 ly automatically.
7089 --vfs-disk-space-total-size Manually set the total disk space size (example: 256G, default: -1)
7091 Alternate report of used bytes
7093 Some backends, most notably S3, do not report the amount of bytes used.
7094 If you need this information to be available when running df on the
7095 filesystem, then pass the flag --vfs-used-is-size to rclone. With this
7096 flag set, instead of relying on the backend to report this information,
7097 rclone will scan the whole remote similar to rclone size and compute
7098 the total used space itself.
7099 WARNING. Contrary to rclone size, this flag ignores filters so that the
7101 result is accurate. However, this is very inefficient and may cost
7102 lots of API calls resulting in extra charges. Use it as a last resort
7103 and only with caching.
7104 Auth Proxy
7106 If you supply the parameter --auth-proxy /path/to/program then rclone
7107 will use that program to generate backends on the fly which then are
7108 used to authenticate incoming requests. This uses a simple JSON based
7109 protocol with input on STDIN and output on STDOUT.
7110 PLEASE NOTE: --auth-proxy and --authorized-keys cannot be used togeth‐
7112 er, if --auth-proxy is set the authorized keys option will be ignored.
7113 There is an example program bin/test_proxy.py
7115 (https://github.com/rclone/rclone/blob/master/test_proxy.py) in the
7116 rclone source code.
7117 The program's job is to take a user and pass on the input and turn
7119 those into the config for a backend on STDOUT in JSON format. This
7120 config will have any default parameters for the backend added, but it
7121 won't use configuration from environment variables or command line op‐
7122 tions - it is the job of the proxy program to make a complete config.
7123 This config generated must have this extra parameter - _root - root to
7125 use for the backend
7126 And it may have this parameter - _obscure - comma separated strings for
7128 parameters to obscure
7129 If password authentication was used by the client, input to the proxy
7131 process (on STDIN) would look similar to this:
7132 {
7134 "user": "me",
7135 "pass": "mypassword"
7136 }
7137 If public-key authentication was used by the client, input to the proxy
7139 process (on STDIN) would look similar to this:
7140 {
7142 "user": "me",
7143 "public_key": "AAAAB3NzaC1yc2EAAAADAQABAAABAQDuwESFdAe14hVS6omeyX7edc...JQdf"
7144 }
7145 And as an example return this on STDOUT
7147 {
7149 "type": "sftp",
7150 "_root": "",
7151 "_obscure": "pass",
7152 "user": "me",
7153 "pass": "mypassword",
7154 "host": "sftp.example.com"
7155 }
7156 This would mean that an SFTP backend would be created on the fly for
7158 the user and pass/public_key returned in the output to the host given.
7159 Note that since _obscure is set to pass, rclone will obscure the pass
7160 parameter before creating the backend (which is required for sftp back‐
7161 ends).
7162 The program can manipulate the supplied user in any way, for example to
7164 make proxy to many different sftp backends, you could make the user be
7165 user@example.com and then set the host to example.com in the output and
7166 the user to user. For security you'd probably want to restrict the
7167 host to a limited list.
7168 Note that an internal cache is keyed on user so only use that for con‐
7170 figuration, don't use pass or public_key. This also means that if a
7171 user's password or public-key is changed the cache will need to expire
7172 (which takes 5 mins) before it takes effect.
7173 This can be used to build general purpose proxies to any kind of back‐
7175 end that rclone supports.
7176 rclone serve sftp remote:path [flags]
7178 Options
7180 --addr string IPaddress:Port or :Port to bind server to (default "localhost:2022")
7181 --auth-proxy string A program to use to create the backend from the auth
7182 --authorized-keys string Authorized keys file (default "~/.ssh/authorized_keys")
7183 --dir-cache-time duration Time to cache directory entries for (default 5m0s)
7184 --dir-perms FileMode Directory permissions (default 0777)
7185 --file-perms FileMode File permissions (default 0666)
7186 --gid uint32 Override the gid field set by the filesystem (not supported on Windows) (default 1000)
7187 -h, --help help for sftp
7188 --key stringArray SSH private host key file (Can be multi-valued, leave blank to auto generate)
7189 --no-auth Allow connections with no authentication if set
7190 --no-checksum Don't compare checksums on up/download
7191 --no-modtime Don't read/write the modification time (can speed things up)
7192 --no-seek Don't allow seeking in files
7193 --pass string Password for authentication
7194 --poll-interval duration Time to wait between polling for changes, must be smaller than dir-cache-time and only on supported remotes (set 0 to disable) (default 1m0s)
7195 --read-only Only allow read-only access
7196 --stdio Run an sftp server on stdin/stdout
7197 --uid uint32 Override the uid field set by the filesystem (not supported on Windows) (default 1000)
7198 --umask int Override the permission bits set by the filesystem (not supported on Windows) (default 2)
7199 --user string User name for authentication
7200 --vfs-cache-max-age duration Max age of objects in the cache (default 1h0m0s)
7201 --vfs-cache-max-size SizeSuffix Max total size of objects in the cache (default off)
7202 --vfs-cache-mode CacheMode Cache mode off|minimal|writes|full (default off)
7203 --vfs-cache-poll-interval duration Interval to poll the cache for stale objects (default 1m0s)
7204 --vfs-case-insensitive If a file name not found, find a case insensitive match
7205 --vfs-disk-space-total-size SizeSuffix Specify the total space of disk (default off)
7206 --vfs-fast-fingerprint Use fast (less accurate) fingerprints for change detection
7207 --vfs-read-ahead SizeSuffix Extra read ahead over --buffer-size when using cache-mode full
7208 --vfs-read-chunk-size SizeSuffix Read the source objects in chunks (default 128Mi)
7209 --vfs-read-chunk-size-limit SizeSuffix If greater than --vfs-read-chunk-size, double the chunk size after each chunk read, until the limit is reached ('off' is unlimited) (default off)
7210 --vfs-read-wait duration Time to wait for in-sequence read before seeking (default 20ms)
7211 --vfs-used-is-size rclone size Use the rclone size algorithm for Used size
7212 --vfs-write-back duration Time to writeback files after last use when using cache (default 5s)
7213 --vfs-write-wait duration Time to wait for in-sequence write before giving error (default 1s)
7214 See the global flags page (https://rclone.org/flags/) for global op‐
7216 tions not listed here.
7217 SEE ALSO
7219 • rclone serve (https://rclone.org/commands/rclone_serve/) - Serve a
7220 remote over a protocol.
7221
7223 Serve remote:path over WebDAV.
7224 Synopsis
7226 Run a basic WebDAV server to serve a remote over HTTP via the WebDAV
7227 protocol. This can be viewed with a WebDAV client, through a web
7228 browser, or you can make a remote of type WebDAV to read and write it.
7229 WebDAV options
7231 --etag-hash
7232 This controls the ETag header. Without this flag the ETag will be
7233 based on the ModTime and Size of the object.
7234 If this flag is set to "auto" then rclone will choose the first sup‐
7236 ported hash on the backend or you can use a named hash such as "MD5" or
7237 "SHA-1". Use the hashsum (https://rclone.org/commands/rclone_hashsum/)
7238 command to see the full list.
7239 Server options
7241 Use --addr to specify which IP address and port the server should lis‐
7242 ten on, e.g. --addr 1.2.3.4:8000 or --addr :8080 to listen to all IPs.
7243 By default it only listens on localhost. You can use port :0 to let
7244 the OS choose an available port.
7245 If you set --addr to listen on a public or LAN accessible IP address
7247 then using Authentication is advised - see the next section for info.
7248 --server-read-timeout and --server-write-timeout can be used to control
7250 the timeouts on the server. Note that this is the total time for a
7251 transfer.
7252 --max-header-bytes controls the maximum number of bytes the server will
7254 accept in the HTTP header.
7255 --baseurl controls the URL prefix that rclone serves from. By default
7257 rclone will serve from the root. If you used --baseurl "/rclone" then
7258 rclone would serve from a URL starting with "/rclone/". This is useful
7259 if you wish to proxy rclone serve. Rclone automatically inserts lead‐
7260 ing and trailing "/" on --baseurl, so --baseurl "rclone", --baseurl
7261 "/rclone" and --baseurl "/rclone/" are all treated identically.
7262 --template allows a user to specify a custom markup template for HTTP
7264 and WebDAV serve functions. The server exports the following markup to
7265 be used within the template to server pages:
7266 Parameter Description
7268 ──────────────────────────────────────────────────────────────────────────
7269 .Name The full path of a file/directory.
7270 .Title Directory listing of .Name
7271 .Sort The current sort used. This is
7272 changeable via ?sort= parameter
7273 Sort Options:
7274 namedirfirst,name,size,time (de‐
7275 fault namedirfirst)
7276 .Order The current ordering used. This is
7277 changeable via ?order= parameter
7278 Order Options: asc,desc (default
7279 asc)
7280 .Query Currently unused.
7281 .Breadcrumb Allows for creating a relative nav‐
7282 igation
7283 -- .Link The relative to the root link of
7284 the Text.
7285 -- .Text The Name of the directory.
7286 .Entries Information about a specific
7287 file/directory.
7288 -- .URL The 'url' of an entry.
7289 -- .Leaf Currently same as 'URL' but intend‐
7290 ed to be 'just' the name.
7291 -- .IsDir Boolean for if an entry is a direc‐
7292 tory or not.
7293 -- .Size Size in Bytes of the entry.
7294 -- .ModTime The UTC timestamp of an entry.
7295 Authentication
7297 By default this will serve files without needing a login.
7298 You can either use an htpasswd file which can take lots of users, or
7300 set a single username and password with the --user and --pass flags.
7301 Use --htpasswd /path/to/htpasswd to provide an htpasswd file. This is
7303 in standard apache format and supports MD5, SHA1 and BCrypt for basic
7304 authentication. Bcrypt is recommended.
7305 To create an htpasswd file:
7307 touch htpasswd
7309 htpasswd -B htpasswd user
7310 htpasswd -B htpasswd anotherUser
7311 The password file can be updated while rclone is running.
7313 Use --realm to set the authentication realm.
7315 SSL/TLS
7317 By default this will serve over HTTP. If you want you can serve over
7318 HTTPS. You will need to supply the --cert and --key flags. If you
7319 wish to do client side certificate validation then you will need to
7320 supply --client-ca also.
7321 --cert should be either a PEM encoded certificate or a concatenation of
7323 that with the CA certificate. --key should be the PEM encoded private
7324 key and --client-ca should be the PEM encoded client certificate au‐
7325 thority certificate.
7326 --min-tls-version is minimum TLS version that is acceptable. Valid
7328 values are "tls1.0", "tls1.1", "tls1.2" and "tls1.3" (default
7329 "tls1.0").
7330 VFS - Virtual File System
7332 This command uses the VFS layer. This adapts the cloud storage objects
7333 that rclone uses into something which looks much more like a disk fil‐
7334 ing system.
7335 Cloud storage objects have lots of properties which aren't like disk
7337 files - you can't extend them or write to the middle of them, so the
7338 VFS layer has to deal with that. Because there is no one right way of
7339 doing this there are various options explained below.
7340 The VFS layer also implements a directory cache - this caches info
7342 about files and directories (but not the data) in memory.
7343 VFS Directory Cache
7345 Using the --dir-cache-time flag, you can control how long a directory
7346 should be considered up to date and not refreshed from the backend.
7347 Changes made through the VFS will appear immediately or invalidate the
7348 cache.
7349 --dir-cache-time duration Time to cache directory entries for (default 5m0s)
7351 --poll-interval duration Time to wait between polling for changes. Must be smaller than dir-cache-time. Only on supported remotes. Set to 0 to disable (default 1m0s)
7352 However, changes made directly on the cloud storage by the web inter‐
7354 face or a different copy of rclone will only be picked up once the di‐
7355 rectory cache expires if the backend configured does not support
7356 polling for changes. If the backend supports polling, changes will be
7357 picked up within the polling interval.
7358 You can send a SIGHUP signal to rclone for it to flush all directory
7360 caches, regardless of how old they are. Assuming only one rclone in‐
7361 stance is running, you can reset the cache like this:
7362 kill -SIGHUP $(pidof rclone)
7364 If you configure rclone with a remote control then you can use rclone
7366 rc to flush the whole directory cache:
7367 rclone rc vfs/forget
7369 Or individual files or directories:
7371 rclone rc vfs/forget file=path/to/file dir=path/to/dir
7373 VFS File Buffering
7375 The --buffer-size flag determines the amount of memory, that will be
7376 used to buffer data in advance.
7377 Each open file will try to keep the specified amount of data in memory
7379 at all times. The buffered data is bound to one open file and won't be
7380 shared.
7381 This flag is a upper limit for the used memory per open file. The buf‐
7383 fer will only use memory for data that is downloaded but not not yet
7384 read. If the buffer is empty, only a small amount of memory will be
7385 used.
7386 The maximum memory used by rclone for buffering can be up to --buffer-
7388 size * open files.
7389 VFS File Caching
7391 These flags control the VFS file caching options. File caching is nec‐
7392 essary to make the VFS layer appear compatible with a normal file sys‐
7393 tem. It can be disabled at the cost of some compatibility.
7394 For example you'll need to enable VFS caching if you want to read and
7396 write simultaneously to a file. See below for more details.
7397 Note that the VFS cache is separate from the cache backend and you may
7399 find that you need one or the other or both.
7400 --cache-dir string Directory rclone will use for caching.
7402 --vfs-cache-mode CacheMode Cache mode off|minimal|writes|full (default off)
7403 --vfs-cache-max-age duration Max age of objects in the cache (default 1h0m0s)
7404 --vfs-cache-max-size SizeSuffix Max total size of objects in the cache (default off)
7405 --vfs-cache-poll-interval duration Interval to poll the cache for stale objects (default 1m0s)
7406 --vfs-write-back duration Time to writeback files after last use when using cache (default 5s)
7407 If run with -vv rclone will print the location of the file cache. The
7409 files are stored in the user cache file area which is OS dependent but
7410 can be controlled with --cache-dir or setting the appropriate environ‐
7411 ment variable.
7412 The cache has 4 different modes selected by --vfs-cache-mode. The
7414 higher the cache mode the more compatible rclone becomes at the cost of
7415 using disk space.
7416 Note that files are written back to the remote only when they are
7418 closed and if they haven't been accessed for --vfs-write-back seconds.
7419 If rclone is quit or dies with files that haven't been uploaded, these
7420 will be uploaded next time rclone is run with the same flags.
7421 If using --vfs-cache-max-size note that the cache may exceed this size
7423 for two reasons. Firstly because it is only checked every --vfs-cache-
7424 poll-interval. Secondly because open files cannot be evicted from the
7425 cache.
7426 You should not run two copies of rclone using the same VFS cache with
7428 the same or overlapping remotes if using --vfs-cache-mode > off. This
7429 can potentially cause data corruption if you do. You can work around
7430 this by giving each rclone its own cache hierarchy with --cache-dir.
7431 You don't need to worry about this if the remotes in use don't overlap.
7432 --vfs-cache-mode off
7434 In this mode (the default) the cache will read directly from the remote
7435 and write directly to the remote without caching anything on disk.
7436 This will mean some operations are not possible
7438 • Files can't be opened for both read AND write
7440 • Files opened for write can't be seeked
7442 • Existing files opened for write must have O_TRUNC set
7444 • Files open for read with O_TRUNC will be opened write only
7446 • Files open for write only will behave as if O_TRUNC was supplied
7448 • Open modes O_APPEND, O_TRUNC are ignored
7450 • If an upload fails it can't be retried
7452 --vfs-cache-mode minimal
7454 This is very similar to "off" except that files opened for read AND
7455 write will be buffered to disk. This means that files opened for write
7456 will be a lot more compatible, but uses the minimal disk space.
7457 These operations are not possible
7459 • Files opened for write only can't be seeked
7461 • Existing files opened for write must have O_TRUNC set
7463 • Files opened for write only will ignore O_APPEND, O_TRUNC
7465 • If an upload fails it can't be retried
7467 --vfs-cache-mode writes
7469 In this mode files opened for read only are still read directly from
7470 the remote, write only and read/write files are buffered to disk first.
7471 This mode should support all normal file system operations.
7473 If an upload fails it will be retried at exponentially increasing in‐
7475 tervals up to 1 minute.
7476 --vfs-cache-mode full
7478 In this mode all reads and writes are buffered to and from disk. When
7479 data is read from the remote this is buffered to disk as well.
7480 In this mode the files in the cache will be sparse files and rclone
7482 will keep track of which bits of the files it has downloaded.
7483 So if an application only reads the starts of each file, then rclone
7485 will only buffer the start of the file. These files will appear to be
7486 their full size in the cache, but they will be sparse files with only
7487 the data that has been downloaded present in them.
7488 This mode should support all normal file system operations and is oth‐
7490 erwise identical to --vfs-cache-mode writes.
7491 When reading a file rclone will read --buffer-size plus --vfs-read-
7493 ahead bytes ahead. The --buffer-size is buffered in memory whereas the
7494 --vfs-read-ahead is buffered on disk.
7495 When using this mode it is recommended that --buffer-size is not set
7497 too large and --vfs-read-ahead is set large if required.
7498 IMPORTANT not all file systems support sparse files. In particular
7500 FAT/exFAT do not. Rclone will perform very badly if the cache directo‐
7501 ry is on a filesystem which doesn't support sparse files and it will
7502 log an ERROR message if one is detected.
7503 Fingerprinting
7505 Various parts of the VFS use fingerprinting to see if a local file copy
7506 has changed relative to a remote file. Fingerprints are made from:
7507 • size
7509 • modification time
7511 • hash
7513 where available on an object.
7515 On some backends some of these attributes are slow to read (they take
7517 an extra API call per object, or extra work per object).
7518 For example hash is slow with the local and sftp backends as they have
7520 to read the entire file and hash it, and modtime is slow with the s3,
7521 swift, ftp and qinqstor backends because they need to do an extra API
7522 call to fetch it.
7523 If you use the --vfs-fast-fingerprint flag then rclone will not include
7525 the slow operations in the fingerprint. This makes the fingerprinting
7526 less accurate but much faster and will improve the opening time of
7527 cached files.
7528 If you are running a vfs cache over local, s3 or swift backends then
7530 using this flag is recommended.
7531 Note that if you change the value of this flag, the fingerprints of the
7533 files in the cache may be invalidated and the files will need to be
7534 downloaded again.
7535 VFS Chunked Reading
7537 When rclone reads files from a remote it reads them in chunks. This
7538 means that rather than requesting the whole file rclone reads the chunk
7539 specified. This can reduce the used download quota for some remotes by
7540 requesting only chunks from the remote that are actually read, at the
7541 cost of an increased number of requests.
7542 These flags control the chunking:
7544 --vfs-read-chunk-size SizeSuffix Read the source objects in chunks (default 128M)
7546 --vfs-read-chunk-size-limit SizeSuffix Max chunk doubling size (default off)
7547 Rclone will start reading a chunk of size --vfs-read-chunk-size, and
7549 then double the size for each read. When --vfs-read-chunk-size-limit
7550 is specified, and greater than --vfs-read-chunk-size, the chunk size
7551 for each open file will get doubled only until the specified value is
7552 reached. If the value is "off", which is the default, the limit is
7553 disabled and the chunk size will grow indefinitely.
7554 With --vfs-read-chunk-size 100M and --vfs-read-chunk-size-limit 0 the
7556 following parts will be downloaded: 0-100M, 100M-200M, 200M-300M,
7557 300M-400M and so on. When --vfs-read-chunk-size-limit 500M is speci‐
7558 fied, the result would be 0-100M, 100M-300M, 300M-700M, 700M-1200M,
7559 1200M-1700M and so on.
7560 Setting --vfs-read-chunk-size to 0 or "off" disables chunked reading.
7562 VFS Performance
7564 These flags may be used to enable/disable features of the VFS for per‐
7565 formance or other reasons. See also the chunked reading feature.
7566 In particular S3 and Swift benefit hugely from the --no-modtime flag
7568 (or use --use-server-modtime for a slightly different effect) as each
7569 read of the modification time takes a transaction.
7570 --no-checksum Don't compare checksums on up/download.
7572 --no-modtime Don't read/write the modification time (can speed things up).
7573 --no-seek Don't allow seeking in files.
7574 --read-only Only allow read-only access.
7575 Sometimes rclone is delivered reads or writes out of order. Rather
7577 than seeking rclone will wait a short time for the in sequence read or
7578 write to come in. These flags only come into effect when not using an
7579 on disk cache file.
7580 --vfs-read-wait duration Time to wait for in-sequence read before seeking (default 20ms)
7582 --vfs-write-wait duration Time to wait for in-sequence write before giving error (default 1s)
7583 When using VFS write caching (--vfs-cache-mode with value writes or
7585 full), the global flag --transfers can be set to adjust the number of
7586 parallel uploads of modified files from the cache (the related global
7587 flag --checkers has no effect on the VFS).
7588 --transfers int Number of file transfers to run in parallel (default 4)
7590 VFS Case Sensitivity
7592 Linux file systems are case-sensitive: two files can differ only by
7593 case, and the exact case must be used when opening a file.
7594 File systems in modern Windows are case-insensitive but case-preserv‐
7596 ing: although existing files can be opened using any case, the exact
7597 case used to create the file is preserved and available for programs to
7598 query. It is not allowed for two files in the same directory to differ
7599 only by case.
7600 Usually file systems on macOS are case-insensitive. It is possible to
7602 make macOS file systems case-sensitive but that is not the default.
7603 The --vfs-case-insensitive VFS flag controls how rclone handles these
7605 two cases. If its value is "false", rclone passes file names to the
7606 remote as-is. If the flag is "true" (or appears without a value on the
7607 command line), rclone may perform a "fixup" as explained below.
7608 The user may specify a file name to open/delete/rename/etc with a case
7610 different than what is stored on the remote. If an argument refers to
7611 an existing file with exactly the same name, then the case of the ex‐
7612 isting file on the disk will be used. However, if a file name with ex‐
7613 actly the same name is not found but a name differing only by case ex‐
7614 ists, rclone will transparently fixup the name. This fixup happens on‐
7615 ly when an existing file is requested. Case sensitivity of file names
7616 created anew by rclone is controlled by the underlying remote.
7617 Note that case sensitivity of the operating system running rclone (the
7619 target) may differ from case sensitivity of a file system presented by
7620 rclone (the source). The flag controls whether "fixup" is performed to
7621 satisfy the target.
7622 If the flag is not provided on the command line, then its default value
7624 depends on the operating system where rclone runs: "true" on Windows
7625 and macOS, "false" otherwise. If the flag is provided without a value,
7626 then it is "true".
7627 VFS Disk Options
7629 This flag allows you to manually set the statistics about the filing
7630 system. It can be useful when those statistics cannot be read correct‐
7631 ly automatically.
7632 --vfs-disk-space-total-size Manually set the total disk space size (example: 256G, default: -1)
7634 Alternate report of used bytes
7636 Some backends, most notably S3, do not report the amount of bytes used.
7637 If you need this information to be available when running df on the
7638 filesystem, then pass the flag --vfs-used-is-size to rclone. With this
7639 flag set, instead of relying on the backend to report this information,
7640 rclone will scan the whole remote similar to rclone size and compute
7641 the total used space itself.
7642 WARNING. Contrary to rclone size, this flag ignores filters so that the
7644 result is accurate. However, this is very inefficient and may cost
7645 lots of API calls resulting in extra charges. Use it as a last resort
7646 and only with caching.
7647 Auth Proxy
7649 If you supply the parameter --auth-proxy /path/to/program then rclone
7650 will use that program to generate backends on the fly which then are
7651 used to authenticate incoming requests. This uses a simple JSON based
7652 protocol with input on STDIN and output on STDOUT.
7653 PLEASE NOTE: --auth-proxy and --authorized-keys cannot be used togeth‐
7655 er, if --auth-proxy is set the authorized keys option will be ignored.
7656 There is an example program bin/test_proxy.py
7658 (https://github.com/rclone/rclone/blob/master/test_proxy.py) in the
7659 rclone source code.
7660 The program's job is to take a user and pass on the input and turn
7662 those into the config for a backend on STDOUT in JSON format. This
7663 config will have any default parameters for the backend added, but it
7664 won't use configuration from environment variables or command line op‐
7665 tions - it is the job of the proxy program to make a complete config.
7666 This config generated must have this extra parameter - _root - root to
7668 use for the backend
7669 And it may have this parameter - _obscure - comma separated strings for
7671 parameters to obscure
7672 If password authentication was used by the client, input to the proxy
7674 process (on STDIN) would look similar to this:
7675 {
7677 "user": "me",
7678 "pass": "mypassword"
7679 }
7680 If public-key authentication was used by the client, input to the proxy
7682 process (on STDIN) would look similar to this:
7683 {
7685 "user": "me",
7686 "public_key": "AAAAB3NzaC1yc2EAAAADAQABAAABAQDuwESFdAe14hVS6omeyX7edc...JQdf"
7687 }
7688 And as an example return this on STDOUT
7690 {
7692 "type": "sftp",
7693 "_root": "",
7694 "_obscure": "pass",
7695 "user": "me",
7696 "pass": "mypassword",
7697 "host": "sftp.example.com"
7698 }
7699 This would mean that an SFTP backend would be created on the fly for
7701 the user and pass/public_key returned in the output to the host given.
7702 Note that since _obscure is set to pass, rclone will obscure the pass
7703 parameter before creating the backend (which is required for sftp back‐
7704 ends).
7705 The program can manipulate the supplied user in any way, for example to
7707 make proxy to many different sftp backends, you could make the user be
7708 user@example.com and then set the host to example.com in the output and
7709 the user to user. For security you'd probably want to restrict the
7710 host to a limited list.
7711 Note that an internal cache is keyed on user so only use that for con‐
7713 figuration, don't use pass or public_key. This also means that if a
7714 user's password or public-key is changed the cache will need to expire
7715 (which takes 5 mins) before it takes effect.
7716 This can be used to build general purpose proxies to any kind of back‐
7718 end that rclone supports.
7719 rclone serve webdav remote:path [flags]
7721 Options
7723 --addr string IPaddress:Port or :Port to bind server to (default "localhost:8080")
7724 --auth-proxy string A program to use to create the backend from the auth
7725 --baseurl string Prefix for URLs - leave blank for root
7726 --cert string SSL PEM key (concatenation of certificate and CA certificate)
7727 --client-ca string Client certificate authority to verify clients with
7728 --dir-cache-time duration Time to cache directory entries for (default 5m0s)
7729 --dir-perms FileMode Directory permissions (default 0777)
7730 --disable-dir-list Disable HTML directory list on GET request for a directory
7731 --etag-hash string Which hash to use for the ETag, or auto or blank for off
7732 --file-perms FileMode File permissions (default 0666)
7733 --gid uint32 Override the gid field set by the filesystem (not supported on Windows) (default 1000)
7734 -h, --help help for webdav
7735 --htpasswd string htpasswd file - if not provided no authentication is done
7736 --key string SSL PEM Private key
7737 --max-header-bytes int Maximum size of request header (default 4096)
7738 --min-tls-version string Minimum TLS version that is acceptable (default "tls1.0")
7739 --no-checksum Don't compare checksums on up/download
7740 --no-modtime Don't read/write the modification time (can speed things up)
7741 --no-seek Don't allow seeking in files
7742 --pass string Password for authentication
7743 --poll-interval duration Time to wait between polling for changes, must be smaller than dir-cache-time and only on supported remotes (set 0 to disable) (default 1m0s)
7744 --read-only Only allow read-only access
7745 --realm string Realm for authentication (default "rclone")
7746 --server-read-timeout duration Timeout for server reading data (default 1h0m0s)
7747 --server-write-timeout duration Timeout for server writing data (default 1h0m0s)
7748 --template string User-specified template
7749 --uid uint32 Override the uid field set by the filesystem (not supported on Windows) (default 1000)
7750 --umask int Override the permission bits set by the filesystem (not supported on Windows) (default 2)
7751 --user string User name for authentication
7752 --vfs-cache-max-age duration Max age of objects in the cache (default 1h0m0s)
7753 --vfs-cache-max-size SizeSuffix Max total size of objects in the cache (default off)
7754 --vfs-cache-mode CacheMode Cache mode off|minimal|writes|full (default off)
7755 --vfs-cache-poll-interval duration Interval to poll the cache for stale objects (default 1m0s)
7756 --vfs-case-insensitive If a file name not found, find a case insensitive match
7757 --vfs-disk-space-total-size SizeSuffix Specify the total space of disk (default off)
7758 --vfs-fast-fingerprint Use fast (less accurate) fingerprints for change detection
7759 --vfs-read-ahead SizeSuffix Extra read ahead over --buffer-size when using cache-mode full
7760 --vfs-read-chunk-size SizeSuffix Read the source objects in chunks (default 128Mi)
7761 --vfs-read-chunk-size-limit SizeSuffix If greater than --vfs-read-chunk-size, double the chunk size after each chunk read, until the limit is reached ('off' is unlimited) (default off)
7762 --vfs-read-wait duration Time to wait for in-sequence read before seeking (default 20ms)
7763 --vfs-used-is-size rclone size Use the rclone size algorithm for Used size
7764 --vfs-write-back duration Time to writeback files after last use when using cache (default 5s)
7765 --vfs-write-wait duration Time to wait for in-sequence write before giving error (default 1s)
7766 See the global flags page (https://rclone.org/flags/) for global op‐
7768 tions not listed here.
7769 SEE ALSO
7771 • rclone serve (https://rclone.org/commands/rclone_serve/) - Serve a
7772 remote over a protocol.
7773
7775 Changes storage class/tier of objects in remote.
7776 Synopsis
7778 rclone settier changes storage tier or class at remote if supported.
7779 Few cloud storage services provides different storage classes on ob‐
7780 jects, for example AWS S3 and Glacier, Azure Blob storage - Hot, Cool
7781 and Archive, Google Cloud Storage, Regional Storage, Nearline, Coldline
7782 etc.
7783 Note that, certain tier changes make objects not available to access
7785 immediately. For example tiering to archive in azure blob storage
7786 makes objects in frozen state, user can restore by setting tier to
7787 Hot/Cool, similarly S3 to Glacier makes object inaccessible.true
7788 You can use it to tier single object
7790 rclone settier Cool remote:path/file
7792 Or use rclone filters to set tier on only specific files
7794 rclone --include "*.txt" settier Hot remote:path/dir
7796 Or just provide remote directory and all files in directory will be
7798 tiered
7799 rclone settier tier remote:path/dir
7801 rclone settier tier remote:path [flags]
7803 Options
7805 -h, --help help for settier
7806 See the global flags page (https://rclone.org/flags/) for global op‐
7808 tions not listed here.
7809 SEE ALSO
7811 • rclone (https://rclone.org/commands/rclone/) - Show help for rclone
7812 commands, flags and backends.
7813
7815 Run a test command
7816 Synopsis
7818 Rclone test is used to run test commands.
7819 Select which test comand you want with the subcommand, eg
7821 rclone test memory remote:
7823 Each subcommand has its own options which you can see in their help.
7825 NB Be careful running these commands, they may do strange things so
7827 reading their documentation first is recommended.
7828 Options
7830 -h, --help help for test
7831 See the global flags page (https://rclone.org/flags/) for global op‐
7833 tions not listed here.
7834 SEE ALSO
7836 • rclone (https://rclone.org/commands/rclone/) - Show help for rclone
7837 commands, flags and backends.
7838 • rclone test changenotify (https://rclone.org/com‐
7840 mands/rclone_test_changenotify/) - Log any change notify requests for
7841 the remote passed in.
7842 • rclone test histogram (https://rclone.org/commands/rclone_test_his‐
7844 togram/) - Makes a histogram of file name characters.
7845 • rclone test info (https://rclone.org/commands/rclone_test_info/) -
7847 Discovers file name or other limitations for paths.
7848 • rclone test makefile (https://rclone.org/commands/rclone_test_make‐
7850 file/) - Make files with random contents of the size given
7851 • rclone test makefiles (https://rclone.org/commands/rclone_test_make‐
7853 files/) - Make a random file hierarchy in a directory
7854 • rclone test memory (https://rclone.org/commands/rclone_test_memory/)
7856 - Load all the objects at remote:path into memory and report memory
7857 stats.
7858
7860 Log any change notify requests for the remote passed in.
7861 rclone test changenotify remote: [flags]
7863 Options
7865 -h, --help help for changenotify
7866 --poll-interval duration Time to wait between polling for changes (default 10s)
7867 See the global flags page (https://rclone.org/flags/) for global op‐
7869 tions not listed here.
7870 SEE ALSO
7872 • rclone test (https://rclone.org/commands/rclone_test/) - Run a test
7873 command
7874
7876 Makes a histogram of file name characters.
7877 Synopsis
7879 This command outputs JSON which shows the histogram of characters used
7880 in filenames in the remote:path specified.
7881 The data doesn't contain any identifying information but is useful for
7883 the rclone developers when developing filename compression.
7884 rclone test histogram [remote:path] [flags]
7886 Options
7888 -h, --help help for histogram
7889 See the global flags page (https://rclone.org/flags/) for global op‐
7891 tions not listed here.
7892 SEE ALSO
7894 • rclone test (https://rclone.org/commands/rclone_test/) - Run a test
7895 command
7896
7898 Discovers file name or other limitations for paths.
7899 Synopsis
7901 rclone info discovers what filenames and upload methods are possible to
7902 write to the paths passed in and how long they can be. It can take
7903 some time. It will write test files into the remote:path passed in.
7904 It outputs a bit of go code for each one.
7905 NB this can create undeletable files and other hazards - use with care
7907 rclone test info [remote:path]+ [flags]
7909 Options
7911 --all Run all tests
7912 --check-control Check control characters
7913 --check-length Check max filename length
7914 --check-normalization Check UTF-8 Normalization
7915 --check-streaming Check uploads with indeterminate file size
7916 -h, --help help for info
7917 --upload-wait duration Wait after writing a file
7918 --write-json string Write results to file
7919 See the global flags page (https://rclone.org/flags/) for global op‐
7921 tions not listed here.
7922 SEE ALSO
7924 • rclone test (https://rclone.org/commands/rclone_test/) - Run a test
7925 command
7926
7928 Make files with random contents of the size given
7929 rclone test makefile <size> [<file>]+ [flags]
7931 Options
7933 --ascii Fill files with random ASCII printable bytes only
7934 --chargen Fill files with a ASCII chargen pattern
7935 -h, --help help for makefile
7936 --pattern Fill files with a periodic pattern
7937 --seed int Seed for the random number generator (0 for random) (default 1)
7938 --sparse Make the files sparse (appear to be filled with ASCII 0x00)
7939 --zero Fill files with ASCII 0x00
7940 See the global flags page (https://rclone.org/flags/) for global op‐
7942 tions not listed here.
7943 SEE ALSO
7945 • rclone test (https://rclone.org/commands/rclone_test/) - Run a test
7946 command
7947
7949 Make a random file hierarchy in a directory
7950 rclone test makefiles <dir> [flags]
7952 Options
7954 --ascii Fill files with random ASCII printable bytes only
7955 --chargen Fill files with a ASCII chargen pattern
7956 --files int Number of files to create (default 1000)
7957 --files-per-directory int Average number of files per directory (default 10)
7958 -h, --help help for makefiles
7959 --max-file-size SizeSuffix Maximum size of files to create (default 100)
7960 --max-name-length int Maximum size of file names (default 12)
7961 --min-file-size SizeSuffix Minimum size of file to create
7962 --min-name-length int Minimum size of file names (default 4)
7963 --pattern Fill files with a periodic pattern
7964 --seed int Seed for the random number generator (0 for random) (default 1)
7965 --sparse Make the files sparse (appear to be filled with ASCII 0x00)
7966 --zero Fill files with ASCII 0x00
7967 See the global flags page (https://rclone.org/flags/) for global op‐
7969 tions not listed here.
7970 SEE ALSO
7972 • rclone test (https://rclone.org/commands/rclone_test/) - Run a test
7973 command
7974
7976 Load all the objects at remote:path into memory and report memory
7977 stats.
7978 rclone test memory remote:path [flags]
7980 Options
7982 -h, --help help for memory
7983 See the global flags page (https://rclone.org/flags/) for global op‐
7985 tions not listed here.
7986 SEE ALSO
7988 • rclone test (https://rclone.org/commands/rclone_test/) - Run a test
7989 command
7990
7992 Create new file or change file modification time.
7993 Synopsis
7995 Set the modification time on file(s) as specified by remote:path to
7996 have the current time.
7997 If remote:path does not exist then a zero sized file will be created,
7999 unless --no-create or --recursive is provided.
8000 If --recursive is used then recursively sets the modification time on
8002 all existing files that is found under the path. Filters are support‐
8003 ed, and you can test with the --dry-run or the --interactive flag.
8004 If --timestamp is used then sets the modification time to that time in‐
8006 stead of the current time. Times may be specified as one of:
8007 • 'YYMMDD' - e.g. 17.10.30
8009 • 'YYYY-MM-DDTHH:MM:SS' - e.g. 2006-01-02T15:04:05
8011 • 'YYYY-MM-DDTHH:MM:SS.SSS' - e.g. 2006-01-02T15:04:05.123456789
8013 Note that value of --timestamp is in UTC. If you want local time then
8015 add the --localtime flag.
8016 rclone touch remote:path [flags]
8018 Options
8020 -h, --help help for touch
8021 --localtime Use localtime for timestamp, not UTC
8022 -C, --no-create Do not create the file if it does not exist (implied with --recursive)
8023 -R, --recursive Recursively touch all files
8024 -t, --timestamp string Use specified time instead of the current time of day
8025 See the global flags page (https://rclone.org/flags/) for global op‐
8027 tions not listed here.
8028 SEE ALSO
8030 • rclone (https://rclone.org/commands/rclone/) - Show help for rclone
8031 commands, flags and backends.
8032
8034 List the contents of the remote in a tree like fashion.
8035 Synopsis
8037 rclone tree lists the contents of a remote in a similar way to the unix
8038 tree command.
8039 For example
8041 $ rclone tree remote:path
8043 /
8044 ├── file1
8045 ├── file2
8046 ├── file3
8047 └── subdir
8048 ├── file4
8049 └── file5
8050 1 directories, 5 files
8052 You can use any of the filtering options with the tree command (e.g.
8054 --include and --exclude. You can also use --fast-list.
8055 The tree command has many options for controlling the listing which are
8057 compatible with the tree command, for example you can include file
8058 sizes with --size. Note that not all of them have short options as
8059 they conflict with rclone's short options.
8060 For a more interactive navigation of the remote see the ncdu
8062 (https://rclone.org/commands/rclone_ncdu/) command.
8063 rclone tree remote:path [flags]
8065 Options
8067 -a, --all All files are listed (list . files too)
8068 -C, --color Turn colorization on always
8069 -d, --dirs-only List directories only
8070 --dirsfirst List directories before files (-U disables)
8071 --full-path Print the full path prefix for each file
8072 -h, --help help for tree
8073 --level int Descend only level directories deep
8074 -D, --modtime Print the date of last modification.
8075 --noindent Don't print indentation lines
8076 --noreport Turn off file/directory count at end of tree listing
8077 -o, --output string Output to file instead of stdout
8078 -p, --protections Print the protections for each file.
8079 -Q, --quote Quote filenames with double quotes.
8080 -s, --size Print the size in bytes of each file.
8081 --sort string Select sort: name,version,size,mtime,ctime
8082 --sort-ctime Sort files by last status change time
8083 -t, --sort-modtime Sort files by last modification time
8084 -r, --sort-reverse Reverse the order of the sort
8085 -U, --unsorted Leave files unsorted
8086 --version Sort files alphanumerically by version
8087 See the global flags page (https://rclone.org/flags/) for global op‐
8089 tions not listed here.
8090 SEE ALSO
8092 • rclone (https://rclone.org/commands/rclone/) - Show help for rclone
8093 commands, flags and backends.
8094 Copying single files
8096 rclone normally syncs or copies directories. However, if the source
8097 remote points to a file, rclone will just copy that file. The destina‐
8098 tion remote must point to a directory - rclone will give the error
8099 Failed to create file system for "remote:file": is a file not a direc‐
8100 tory if it isn't.
8101 For example, suppose you have a remote with a file in called test.jpg,
8103 then you could copy just that file like this
8104 rclone copy remote:test.jpg /tmp/download
8106 The file test.jpg will be placed inside /tmp/download.
8108 This is equivalent to specifying
8110 rclone copy --files-from /tmp/files remote: /tmp/download
8112 Where /tmp/files contains the single line
8114 test.jpg
8116 It is recommended to use copy when copying individual files, not sync.
8118 They have pretty much the same effect but copy will use a lot less mem‐
8119 ory.
8120 Syntax of remote paths
8122 The syntax of the paths passed to the rclone command are as follows.
8123 /path/to/dir
8125 This refers to the local file system.
8126 On Windows \ may be used instead of / in local paths only, non local
8128 paths must use /. See local filesystem (https://rclone.org/lo‐
8129 cal/#paths-on-windows) documentation for more about Windows-specific
8130 paths.
8131 These paths needn't start with a leading / - if they don't then they
8133 will be relative to the current directory.
8134 remote:path/to/dir
8136 This refers to a directory path/to/dir on remote: as defined in the
8137 config file (configured with rclone config).
8138 remote:/path/to/dir
8140 On most backends this is refers to the same directory as re‐
8141 mote:path/to/dir and that format should be preferred. On a very small
8142 number of remotes (FTP, SFTP, Dropbox for business) this will refer to
8143 a different directory. On these, paths without a leading / will refer
8144 to your "home" directory and paths with a leading / will refer to the
8145 root.
8146 :backend:path/to/dir
8148 This is an advanced form for creating remotes on the fly. backend
8149 should be the name or prefix of a backend (the type in the config file)
8150 and all the configuration for the backend should be provided on the
8151 command line (or in environment variables).
8152 Here are some examples:
8154 rclone lsd --http-url https://pub.rclone.org :http:
8156 To list all the directories in the root of https://pub.rclone.org/.
8158 rclone lsf --http-url https://example.com :http:path/to/dir
8160 To list files and directories in https://example.com/path/to/dir/
8162 rclone copy --http-url https://example.com :http:path/to/dir /tmp/dir
8164 To copy files and directories in https://example.com/path/to/dir to
8166 /tmp/dir.
8167 rclone copy --sftp-host example.com :sftp:path/to/dir /tmp/dir
8169 To copy files and directories from example.com in the relative directo‐
8171 ry path/to/dir to /tmp/dir using sftp.
8172 Connection strings
8174 The above examples can also be written using a connection string syn‐
8175 tax, so instead of providing the arguments as command line parameters
8176 --http-url https://pub.rclone.org they are provided as part of the re‐
8177 mote specification as a kind of connection string.
8178 rclone lsd ":http,url='https://pub.rclone.org':"
8180 rclone lsf ":http,url='https://example.com':path/to/dir"
8181 rclone copy ":http,url='https://example.com':path/to/dir" /tmp/dir
8182 rclone copy :sftp,host=example.com:path/to/dir /tmp/dir
8183 These can apply to modify existing remotes as well as create new re‐
8185 motes with the on the fly syntax. This example is equivalent to adding
8186 the --drive-shared-with-me parameter to the remote gdrive:.
8187 rclone lsf "gdrive,shared_with_me:path/to/dir"
8189 The major advantage to using the connection string style syntax is that
8191 it only applies to the remote, not to all the remotes of that type of
8192 the command line. A common confusion is this attempt to copy a file
8193 shared on google drive to the normal drive which does not work because
8194 the --drive-shared-with-me flag applies to both the source and the des‐
8195 tination.
8196 rclone copy --drive-shared-with-me gdrive:shared-file.txt gdrive:
8198 However using the connection string syntax, this does work.
8200 rclone copy "gdrive,shared_with_me:shared-file.txt" gdrive:
8202 Note that the connection string only affects the options of the immedi‐
8204 ate backend. If for example gdriveCrypt is a crypt based on gdrive,
8205 then the following command will not work as intended, because
8206 shared_with_me is ignored by the crypt backend:
8207 rclone copy "gdriveCrypt,shared_with_me:shared-file.txt" gdriveCrypt:
8209 The connection strings have the following syntax
8211 remote,parameter=value,parameter2=value2:path/to/dir
8213 :backend,parameter=value,parameter2=value2:path/to/dir
8214 If the parameter has a : or , then it must be placed in quotes " or ',
8216 so
8217 remote,parameter="colon:value",parameter2="comma,value":path/to/dir
8219 :backend,parameter='colon:value',parameter2='comma,value':path/to/dir
8220 If a quoted value needs to include that quote, then it should be dou‐
8222 bled, so
8223 remote,parameter="with""quote",parameter2='with''quote':path/to/dir
8225 This will make parameter be with"quote and parameter2 be with'quote.
8227 If you leave off the =parameter then rclone will substitute =true which
8229 works very well with flags. For example, to use s3 configured in the
8230 environment you could use:
8231 rclone lsd :s3,env_auth:
8233 Which is equivalent to
8235 rclone lsd :s3,env_auth=true:
8237 Note that on the command line you might need to surround these connec‐
8239 tion strings with " or ' to stop the shell interpreting any special
8240 characters within them.
8241 If you are a shell master then you'll know which strings are OK and
8243 which aren't, but if you aren't sure then enclose them in " and use '
8244 as the inside quote. This syntax works on all OSes.
8245 rclone copy ":http,url='https://example.com':path/to/dir" /tmp/dir
8247 On Linux/macOS some characters are still interpreted inside " strings
8249 in the shell (notably \ and $ and ") so if your strings contain those
8250 you can swap the roles of " and ' thus. (This syntax does not work on
8251 Windows.)
8252 rclone copy ':http,url="https://example.com":path/to/dir' /tmp/dir
8254 Connection strings, config and logging
8256 If you supply extra configuration to a backend by command line flag,
8257 environment variable or connection string then rclone will add a suffix
8258 based on the hash of the config to the name of the remote, eg
8259 rclone -vv lsf --s3-chunk-size 20M s3:
8261 Has the log message
8263 DEBUG : s3: detected overridden config - adding "{Srj1p}" suffix to name
8265 This is so rclone can tell the modified remote apart from the unmodi‐
8267 fied remote when caching the backends.
8268 This should only be noticeable in the logs.
8270 This means that on the fly backends such as
8272 rclone -vv lsf :s3,env_auth:
8274 Will get their own names
8276 DEBUG : :s3: detected overridden config - adding "{YTu53}" suffix to name
8278 Valid remote names
8280 Remote names are case sensitive, and must adhere to the following
8281 rules: - May only contain 0-9, A-Z, a-z, _, -, . and space. - May not
8282 start with - or space.
8283 Quoting and the shell
8285 When you are typing commands to your computer you are using something
8286 called the command line shell. This interprets various characters in
8287 an OS specific way.
8288 Here are some gotchas which may help users unfamiliar with the shell
8290 rules
8291 Linux / OSX
8293 If your names have spaces or shell metacharacters (e.g. *, ?, $, ', ",
8294 etc.) then you must quote them. Use single quotes ' by default.
8295 rclone copy 'Important files?' remote:backup
8297 If you want to send a ' you will need to use ", e.g.
8299 rclone copy "O'Reilly Reviews" remote:backup
8301 The rules for quoting metacharacters are complicated and if you want
8303 the full details you'll have to consult the manual page for your shell.
8304 Windows
8306 If your names have spaces in you need to put them in ", e.g.
8307 rclone copy "E:\folder name\folder name\folder name" remote:backup
8309 If you are using the root directory on its own then don't quote it (see
8311 #464 (https://github.com/rclone/rclone/issues/464) for why), e.g.
8312 rclone copy E:\ remote:backup
8314 Copying files or directories with : in the names
8316 rclone uses : to mark a remote name. This is, however, a valid file‐
8317 name component in non-Windows OSes. The remote name parser will only
8318 search for a : up to the first / so if you need to act on a file or di‐
8319 rectory like this then use the full path starting with a /, or use ./
8320 as a current directory prefix.
8321 So to sync a directory called sync:me to a remote called remote: use
8323 rclone sync -i ./sync:me remote:path
8325 or
8327 rclone sync -i /full/path/to/sync:me remote:path
8329 Server Side Copy
8331 Most remotes (but not all - see the overview (https://rclone.org/over‐
8332 view/#optional-features)) support server-side copy.
8333 This means if you want to copy one folder to another then rclone won't
8335 download all the files and re-upload them; it will instruct the server
8336 to copy them in place.
8337 Eg
8339 rclone copy s3:oldbucket s3:newbucket
8341 Will copy the contents of oldbucket to newbucket without downloading
8343 and re-uploading.
8344 Remotes which don't support server-side copy will download and re-up‐
8346 load in this case.
8347 Server side copies are used with sync and copy and will be identified
8349 in the log when using the -v flag. The move command may also use them
8350 if remote doesn't support server-side move directly. This is done by
8351 issuing a server-side copy then a delete which is much quicker than a
8352 download and re-upload.
8353 Server side copies will only be attempted if the remote names are the
8355 same.
8356 This can be used when scripting to make aged backups efficiently, e.g.
8358 rclone sync -i remote:current-backup remote:previous-backup
8360 rclone sync -i /path/to/files remote:current-backup
8361 Metadata support
8363 Metadata is data about a file which isn't the contents of the file.
8364 Normally rclone only preserves the modification time and the content
8365 (MIME) type where possible.
8366 Rclone supports preserving all the available metadata on files (not di‐
8368 rectories) when using the --metadata or -M flag.
8369 Exactly what metadata is supported and what that support means depends
8371 on the backend. Backends that support metadata have a metadata section
8372 in their docs and are listed in the features table
8373 (https://rclone.org/overview/#features) (Eg local
8374 (https://rclone.org/local/#metadata), s3)
8375 Rclone only supports a one-time sync of metadata. This means that
8377 metadata will be synced from the source object to the destination ob‐
8378 ject only when the source object has changed and needs to be re-upload‐
8379 ed. If the metadata subsequently changes on the source object without
8380 changing the object itself then it won't be synced to the destination
8381 object. This is in line with the way rclone syncs Content-Type without
8382 the --metadata flag.
8383 Using --metadata when syncing from local to local will preserve file
8385 attributes such as file mode, owner, extended attributes (not Windows).
8386 Note that arbitrary metadata may be added to objects using the --meta‐
8388 data-set key=value flag when the object is first uploaded. This flag
8389 can be repeated as many times as necessary.
8390 Types of metadata
8392 Metadata is divided into two type. System metadata and User metadata.
8393 Metadata which the backend uses itself is called system metadata. For
8395 example on the local backend the system metadata uid will store the us‐
8396 er ID of the file when used on a unix based platform.
8397 Arbitrary metadata is called user metadata and this can be set however
8399 is desired.
8400 When objects are copied from backend to backend, they will attempt to
8402 interpret system metadata if it is supplied. Metadata may change from
8403 being user metadata to system metadata as objects are copied between
8404 different backends. For example copying an object from s3 sets the
8405 content-type metadata. In a backend which understands this (like
8406 azureblob) this will become the Content-Type of the object. In a back‐
8407 end which doesn't understand this (like the local backend) this will
8408 become user metadata. However should the local object be copied back
8409 to s3, the Content-Type will be set correctly.
8410 Metadata framework
8412 Rclone implements a metadata framework which can read metadata from an
8413 object and write it to the object when (and only when) it is being up‐
8414 loaded.
8415 This metadata is stored as a dictionary with string keys and string
8417 values.
8418 There are some limits on the names of the keys (these may be clarified
8420 further in the future).
8421 • must be lower case
8423 • may be a-z 0-9 containing . - or _
8425 • length is backend dependent
8427 Each backend can provide system metadata that it understands. Some
8429 backends can also store arbitrary user metadata.
8430 Where possible the key names are standardized, so, for example, it is
8432 possible to copy object metadata from s3 to azureblob for example and
8433 metadata will be translated appropriately.
8434 Some backends have limits on the size of the metadata and rclone will
8436 give errors on upload if they are exceeded.
8437 Metadata preservation
8439 The goal of the implementation is to
8440 1. Preserve metadata if at all possible
8442 2. Interpret metadata if at all possible
8444 The consequences of 1 is that you can copy an S3 object to a local disk
8446 then back to S3 losslessly. Likewise you can copy a local file with
8447 file attributes and xattrs from local disk to s3 and back again loss‐
8448 lessly.
8449 The consequence of 2 is that you can copy an S3 object with metadata to
8451 Azureblob (say) and have the metadata appear on the Azureblob object
8452 also.
8453 Standard system metadata
8455 Here is a table of standard system metadata which, if appropriate, a
8456 backend may implement.
8457 key description example
8459 ─────────────────────────────────────────────────────────────────────────────────────────────────
8460 mode File type and mode: 0100664
8461 octal, unix style
8462 uid User ID of owner: 500
8463 decimal number
8464 gid Group ID of owner: 500
8465 decimal number
8466 rdev Device ID (if special 0
8467 file) => hexadecimal
8468 atime Time of last access: 2006-01-02T15:04:05.999999999Z07:00
8469 RFC 3339
8470 mtime Time of last modifi‐ 2006-01-02T15:04:05.999999999Z07:00
8471 cation: RFC 3339
8472 btime Time of file creation 2006-01-02T15:04:05.999999999Z07:00
8473 (birth): RFC 3339
8474 cache-control Cache-Control header no-cache
8475 content-disposition Content-Disposition inline
8476 header
8477 content-encoding Content-Encoding gzip
8478 header
8479 content-language Content-Language en-US
8480 header
8481 content-type Content-Type header text/plain
8482 The metadata keys mtime and content-type will take precedence if sup‐
8484 plied in the metadata over reading the Content-Type or modification
8485 time of the source object.
8486 Hashes are not included in system metadata as there is a well defined
8488 way of reading those already.
8489 Options
8491 Rclone has a number of options to control its behaviour.
8492 Options that take parameters can have the values passed in two ways,
8494 --option=value or --option value. However boolean (true/false) options
8495 behave slightly differently to the other options in that --boolean sets
8496 the option to true and the absence of the flag sets it to false. It is
8497 also possible to specify --boolean=false or --boolean=true. Note that
8498 --boolean false is not valid - this is parsed as --boolean and the
8499 false is parsed as an extra command line argument for rclone.
8500 Time or duration options
8502 TIME or DURATION options can be specified as a duration string or a
8503 time string.
8504 A duration string is a possibly signed sequence of decimal numbers,
8506 each with optional fraction and a unit suffix, such as "300ms", "-1.5h"
8507 or "2h45m". Default units are seconds or the following abbreviations
8508 are valid:
8509 • ms - Milliseconds
8511 • s - Seconds
8513 • m - Minutes
8515 • h - Hours
8517 • d - Days
8519 • w - Weeks
8521 • M - Months
8523 • y - Years
8525 These can also be specified as an absolute time in the following for‐
8527 mats:
8528 • RFC3339 - e.g. 2006-01-02T15:04:05Z or 2006-01-02T15:04:05+07:00
8530 • ISO8601 Date and time, local timezone - 2006-01-02T15:04:05
8532 • ISO8601 Date and time, local timezone - 2006-01-02 15:04:05
8534 • ISO8601 Date - 2006-01-02 (YYYY-MM-DD)
8536 Size options
8538 Options which use SIZE use KiB (multiples of 1024 bytes) by default.
8539 However, a suffix of B for Byte, K for KiB, M for MiB, G for GiB, T for
8540 TiB and P for PiB may be used. These are the binary units, e.g. 1,
8541 2**10, 2**20, 2**30 respectively.
8542 --backup-dir=DIR
8544 When using sync, copy or move any files which would have been overwrit‐
8545 ten or deleted are moved in their original hierarchy into this directo‐
8546 ry.
8547 If --suffix is set, then the moved files will have the suffix added to
8549 them. If there is a file with the same path (after the suffix has been
8550 added) in DIR, then it will be overwritten.
8551 The remote in use must support server-side move or copy and you must
8553 use the same remote as the destination of the sync. The backup direc‐
8554 tory must not overlap the destination directory without it being ex‐
8555 cluded by a filter rule.
8556 For example
8558 rclone sync -i /path/to/local remote:current --backup-dir remote:old
8560 will sync /path/to/local to remote:current, but for any files which
8562 would have been updated or deleted will be stored in remote:old.
8563 If running rclone from a script you might want to use today's date as
8565 the directory name passed to --backup-dir to store the old files, or
8566 you might want to pass --suffix with today's date.
8567 See --compare-dest and --copy-dest.
8569 --bind string
8571 Local address to bind to for outgoing connections. This can be an IPv4
8572 address (1.2.3.4), an IPv6 address (1234::789A) or host name. If the
8573 host name doesn't resolve or resolves to more than one IP address it
8574 will give an error.
8575 --bwlimit=BANDWIDTH_SPEC
8577 This option controls the bandwidth limit. For example
8578 --bwlimit 10M
8580 would mean limit the upload and download bandwidth to 10 MiB/s. NB
8582 this is bytes per second not bits per second. To use a single limit,
8583 specify the desired bandwidth in KiB/s, or use a suffix B|K|M|G|T|P.
8584 The default is 0 which means to not limit bandwidth.
8585 The upload and download bandwidth can be specified separately, as
8587 --bwlimit UP:DOWN, so
8588 --bwlimit 10M:100k
8590 would mean limit the upload bandwidth to 10 MiB/s and the download
8592 bandwidth to 100 KiB/s. Either limit can be "off" meaning no limit, so
8593 to just limit the upload bandwidth you would use
8594 --bwlimit 10M:off
8596 this would limit the upload bandwidth to 10 MiB/s but the download
8598 bandwidth would be unlimited.
8599 When specified as above the bandwidth limits last for the duration of
8601 run of the rclone binary.
8602 It is also possible to specify a "timetable" of limits, which will
8604 cause certain limits to be applied at certain times. To specify a
8605 timetable, format your entries as WEEKDAY-HH:MM,BANDWIDTH WEEKDAY-
8606 HH:MM,BANDWIDTH... where: WEEKDAY is optional element.
8607 • BANDWIDTH can be a single number, e.g.100k or a pair of numbers for
8609 upload:download, e.g.10M:1M.
8610 • WEEKDAY can be written as the whole word or only using the first 3
8612 characters. It is optional.
8613 • HH:MM is an hour from 00:00 to 23:59.
8615 An example of a typical timetable to avoid link saturation during day‐
8617 time working hours could be:
8618 --bwlimit "08:00,512k 12:00,10M 13:00,512k 18:00,30M 23:00,off"
8620 In this example, the transfer bandwidth will be set to 512 KiB/s at 8am
8622 every day. At noon, it will rise to 10 MiB/s, and drop back to 512
8623 KiB/sec at 1pm. At 6pm, the bandwidth limit will be set to 30 MiB/s,
8624 and at 11pm it will be completely disabled (full speed). Anything be‐
8625 tween 11pm and 8am will remain unlimited.
8626 An example of timetable with WEEKDAY could be:
8628 --bwlimit "Mon-00:00,512 Fri-23:59,10M Sat-10:00,1M Sun-20:00,off"
8630 It means that, the transfer bandwidth will be set to 512 KiB/s on Mon‐
8632 day. It will rise to 10 MiB/s before the end of Friday. At 10:00 on
8633 Saturday it will be set to 1 MiB/s. From 20:00 on Sunday it will be
8634 unlimited.
8635 Timeslots without WEEKDAY are extended to the whole week. So this ex‐
8637 ample:
8638 --bwlimit "Mon-00:00,512 12:00,1M Sun-20:00,off"
8640 Is equivalent to this:
8642 --bwlimit "Mon-00:00,512Mon-12:00,1M Tue-12:00,1M Wed-12:00,1M
8644 Thu-12:00,1M Fri-12:00,1M Sat-12:00,1M Sun-12:00,1M Sun-20:00,off"
8645 Bandwidth limit apply to the data transfer for all backends. For most
8647 backends the directory listing bandwidth is also included (exceptions
8648 being the non HTTP backends, ftp, sftp and storj).
8649 Note that the units are Byte/s, not bit/s. Typically connections are
8651 measured in bit/s - to convert divide by 8. For example, let's say you
8652 have a 10 Mbit/s connection and you wish rclone to use half of it - 5
8653 Mbit/s. This is 5/8 = 0.625 MiB/s so you would use a --bwlimit 0.625M
8654 parameter for rclone.
8655 On Unix systems (Linux, macOS, ...) the bandwidth limiter can be tog‐
8657 gled by sending a SIGUSR2 signal to rclone. This allows to remove the
8658 limitations of a long running rclone transfer and to restore it back to
8659 the value specified with --bwlimit quickly when needed. Assuming there
8660 is only one rclone instance running, you can toggle the limiter like
8661 this:
8662 kill -SIGUSR2 $(pidof rclone)
8664 If you configure rclone with a remote control then you can use change
8666 the bwlimit dynamically:
8667 rclone rc core/bwlimit rate=1M
8669 --bwlimit-file=BANDWIDTH_SPEC
8671 This option controls per file bandwidth limit. For the options see the
8672 --bwlimit flag.
8673 For example use this to allow no transfers to be faster than 1 MiB/s
8675 --bwlimit-file 1M
8677 This can be used in conjunction with --bwlimit.
8679 Note that if a schedule is provided the file will use the schedule in
8681 effect at the start of the transfer.
8682 --buffer-size=SIZE
8684 Use this sized buffer to speed up file transfers. Each --transfer will
8685 use this much memory for buffering.
8686 When using mount or cmount each open file descriptor will use this much
8688 memory for buffering. See the mount (https://rclone.org/com‐
8689 mands/rclone_mount/#file-buffering) documentation for more details.
8690 Set to 0 to disable the buffering for the minimum memory usage.
8692 Note that the memory allocation of the buffers is influenced by the
8694 --use-mmap flag.
8695 --cache-dir=DIR
8697 Specify the directory rclone will use for caching, to override the de‐
8698 fault.
8699 Default value is depending on operating system: - Windows %LocalAppDa‐
8701 ta%\rclone, if LocalAppData is defined. - macOS $HOME/Li‐
8702 brary/Caches/rclone if HOME is defined. - Unix $XDG_CACHE_HOME/rclone
8703 if XDG_CACHE_HOME is defined, else $HOME/.cache/rclone if HOME is de‐
8704 fined. - Fallback (on all OS) to $TMPDIR/rclone, where TMPDIR is the
8705 value from --temp-dir.
8706 You can use the config paths (https://rclone.org/commands/rclone_con‐
8708 fig_paths/) command to see the current value.
8709 Cache directory is heavily used by the VFS File Caching
8711 (https://rclone.org/commands/rclone_mount/#vfs-file-caching) mount fea‐
8712 ture, but also by serve (https://rclone.org/commands/rclone_serve/),
8713 GUI and other parts of rclone.
8714 --check-first
8716 If this flag is set then in a sync, copy or move, rclone will do all
8717 the checks to see whether files need to be transferred before doing any
8718 of the transfers. Normally rclone would start running transfers as
8719 soon as possible.
8720 This flag can be useful on IO limited systems where transfers interfere
8722 with checking.
8723 It can also be useful to ensure perfect ordering when using --order-by.
8725 Using this flag can use more memory as it effectively sets --max-back‐
8727 log to infinite. This means that all the info on the objects to trans‐
8728 fer is held in memory before the transfers start.
8729 --checkers=N
8731 Originally controlling just the number of file checkers to run in par‐
8732 allel, e.g. by rclone copy. Now a fairly universal parallelism con‐
8733 trol used by rclone in several places.
8734 Note: checkers do the equality checking of files during a sync. For
8736 some storage systems (e.g. S3, Swift, Dropbox) this can take a signif‐
8737 icant amount of time so they are run in parallel.
8738 The default is to run 8 checkers in parallel. However, in case of
8740 slow-reacting backends you may need to lower (rather than increase)
8741 this default by setting --checkers to 4 or less threads. This is espe‐
8742 cially advised if you are experiencing backend server crashes during
8743 file checking phase (e.g. on subsequent or top-up backups where little
8744 or no file copying is done and checking takes up most of the time).
8745 Increase this setting only with utmost care, while monitoring your
8746 server health and file checking throughput.
8747 -c, --checksum
8749 Normally rclone will look at modification time and size of files to see
8750 if they are equal. If you set this flag then rclone will check the
8751 file hash and size to determine if files are equal.
8752 This is useful when the remote doesn't support setting modified time
8754 and a more accurate sync is desired than just checking the file size.
8755 This is very useful when transferring between remotes which store the
8757 same hash type on the object, e.g. Drive and Swift. For details of
8758 which remotes support which hash type see the table in the overview
8759 section (https://rclone.org/overview/).
8760 Eg rclone --checksum sync s3:/bucket swift:/bucket would run much
8762 quicker than without the --checksum flag.
8763 When using this flag, rclone won't update mtimes of remote files if
8765 they are incorrect as it would normally.
8766 --compare-dest=DIR
8768 When using sync, copy or move DIR is checked in addition to the desti‐
8769 nation for files. If a file identical to the source is found that file
8770 is NOT copied from source. This is useful to copy just files that have
8771 changed since the last backup.
8772 You must use the same remote as the destination of the sync. The com‐
8774 pare directory must not overlap the destination directory.
8775 See --copy-dest and --backup-dir.
8777 --config=CONFIG_FILE
8779 Specify the location of the rclone configuration file, to override the
8780 default. E.g. rclone config --config="rclone.conf".
8781 The exact default is a bit complex to describe, due to changes intro‐
8783 duced through different versions of rclone while preserving backwards
8784 compatibility, but in most cases it is as simple as:
8785 • %APPDATA%/rclone/rclone.conf on Windows
8787 • ~/.config/rclone/rclone.conf on other
8789 The complete logic is as follows: Rclone will look for an existing con‐
8791 figuration file in any of the following locations, in priority order:
8792 1. rclone.conf (in program directory, where rclone executable is)
8794 2. %APPDATA%/rclone/rclone.conf (only on Windows)
8796 3. $XDG_CONFIG_HOME/rclone/rclone.conf (on all systems, including Win‐
8798 dows)
8799 4. ~/.config/rclone/rclone.conf (see below for explanation of ~ symbol)
8801 5. ~/.rclone.conf
8803 If no existing configuration file is found, then a new one will be cre‐
8805 ated in the following location:
8806 • On Windows: Location 2 listed above, except in the unlikely event
8808 that APPDATA is not defined, then location 4 is used instead.
8809 • On Unix: Location 3 if XDG_CONFIG_HOME is defined, else location 4.
8811 • Fallback to location 5 (on all OS), when the rclone directory cannot
8813 be created, but if also a home directory was not found then path
8814 .rclone.conf relative to current working directory will be used as a
8815 final resort.
8816 The ~ symbol in paths above represent the home directory of the current
8818 user on any OS, and the value is defined as following:
8819 • On Windows: %HOME% if defined, else %USERPROFILE%, or else
8821 %HOMEDRIVE%\%HOMEPATH%.
8822 • On Unix: $HOME if defined, else by looking up current user in OS-spe‐
8824 cific user database (e.g. passwd file), or else use the result from
8825 shell command cd && pwd.
8826 If you run rclone config file you will see where the default location
8828 is for you.
8829 The fact that an existing file rclone.conf in the same directory as the
8831 rclone executable is always preferred, means that it is easy to run in
8832 "portable" mode by downloading rclone executable to a writable directo‐
8833 ry and then create an empty file rclone.conf in the same directory.
8834 If the location is set to empty string "" or path to a file with name
8836 notfound, or the os null device represented by value NUL on Windows and
8837 /dev/null on Unix systems, then rclone will keep the config file in
8838 memory only.
8839 The file format is basic INI (https://en.wikipedia.org/wi‐
8841 ki/INI_file#Format): Sections of text, led by a [section] header and
8842 followed by key=value entries on separate lines. In rclone each remote
8843 is represented by its own section, where the section name defines the
8844 name of the remote. Options are specified as the key=value entries,
8845 where the key is the option name without the --backend- prefix, in low‐
8846 ercase and with _ instead of -. E.g. option --mega-hard-delete corre‐
8847 sponds to key hard_delete. Only backend options can be specified. A
8848 special, and required, key type identifies the storage system
8849 (https://rclone.org/overview/), where the value is the internal lower‐
8850 case name as returned by command rclone help backends. Comments are
8851 indicated by ; or # at the beginning of a line.
8852 Example:
8854 [megaremote]
8856 type = mega
8857 user = you@example.com
8858 pass = PDPcQVVjVtzFY-GTdDFozqBhTdsPg3qH
8859 Note that passwords are in obscured (https://rclone.org/com‐
8861 mands/rclone_obscure/) form. Also, many storage systems uses token-
8862 based authentication instead of passwords, and this requires additional
8863 steps. It is easier, and safer, to use the interactive command rclone
8864 config instead of manually editing the configuration file.
8865 The configuration file will typically contain login information, and
8867 should therefore have restricted permissions so that only the current
8868 user can read it. Rclone tries to ensure this when it writes the file.
8869 You may also choose to encrypt the file.
8870 When token-based authentication are used, the configuration file must
8872 be writable, because rclone needs to update the tokens inside it.
8873 --contimeout=TIME
8875 Set the connection timeout. This should be in go time format which
8876 looks like 5s for 5 seconds, 10m for 10 minutes, or 3h30m.
8877 The connection timeout is the amount of time rclone will wait for a
8879 connection to go through to a remote object storage system. It is 1m
8880 by default.
8881 --copy-dest=DIR
8883 When using sync, copy or move DIR is checked in addition to the desti‐
8884 nation for files. If a file identical to the source is found that file
8885 is server-side copied from DIR to the destination. This is useful for
8886 incremental backup.
8887 The remote in use must support server-side copy and you must use the
8889 same remote as the destination of the sync. The compare directory must
8890 not overlap the destination directory.
8891 See --compare-dest and --backup-dir.
8893 --dedupe-mode MODE
8895 Mode to run dedupe command in. One of interactive, skip, first, new‐
8896 est, oldest, rename. The default is interactive.
8897 See the dedupe command for more information as to what these options
8898 mean.
8899 --disable FEATURE,FEATURE,...
8901 This disables a comma separated list of optional features. For example
8902 to disable server-side move and server-side copy use:
8903 --disable move,copy
8905 The features can be put in any case.
8907 To see a list of which features can be disabled use:
8909 --disable help
8911 See the overview features (https://rclone.org/overview/#features) and
8913 optional features (https://rclone.org/overview/#optional-features) to
8914 get an idea of which feature does what.
8915 This flag can be useful for debugging and in exceptional circumstances
8917 (e.g. Google Drive limiting the total volume of Server Side Copies to
8918 100 GiB/day).
8919 --disable-http2
8921 This stops rclone from trying to use HTTP/2 if available. This can
8922 sometimes speed up transfers due to a problem in the Go standard li‐
8923 brary (https://github.com/golang/go/issues/37373).
8924 --dscp VALUE
8926 Specify a DSCP value or name to use in connections. This could help
8927 QoS system to identify traffic class. BE, EF, DF, LE, CSx and AFxx are
8928 allowed.
8929 See the description of differentiated services
8931 (https://en.wikipedia.org/wiki/Differentiated_services) to get an idea
8932 of this field. Setting this to 1 (LE) to identify the flow to SCAV‐
8933 ENGER class can avoid occupying too much bandwidth in a network with
8934 DiffServ support (RFC 8622 (https://tools.ietf.org/html/rfc8622)).
8935 For example, if you configured QoS on router to handle LE properly.
8937 Running:
8938 rclone copy --dscp LE from:/from to:/to
8940 would make the priority lower than usual internet flows.
8942 This option has no effect on Windows (see golang/go#42728
8944 (https://github.com/golang/go/issues/42728)).
8945 -n, --dry-run
8947 Do a trial run with no permanent changes. Use this to see what rclone
8948 would do without actually doing it. Useful when setting up the sync
8949 command which deletes files in the destination.
8950 --expect-continue-timeout=TIME
8952 This specifies the amount of time to wait for a server's first response
8953 headers after fully writing the request headers if the request has an
8954 "Expect: 100-continue" header. Not all backends support using this.
8955 Zero means no timeout and causes the body to be sent immediately, with‐
8957 out waiting for the server to approve. This time does not include the
8958 time to send the request header.
8959 The default is 1s. Set to 0 to disable.
8961 --error-on-no-transfer
8963 By default, rclone will exit with return code 0 if there were no er‐
8964 rors.
8965 This option allows rclone to return exit code 9 if no files were trans‐
8967 ferred between the source and destination. This allows using rclone in
8968 scripts, and triggering follow-on actions if data was copied, or skip‐
8969 ping if not.
8970 NB: Enabling this option turns a usually non-fatal error into a poten‐
8972 tially fatal one - please check and adjust your scripts accordingly!
8973 --fs-cache-expire-duration=TIME
8975 When using rclone via the API rclone caches created remotes for 5 min‐
8976 utes by default in the "fs cache". This means that if you do repeated
8977 actions on the same remote then rclone won't have to build it again
8978 from scratch, which makes it more efficient.
8979 This flag sets the time that the remotes are cached for. If you set it
8981 to 0 (or negative) then rclone won't cache the remotes at all.
8982 Note that if you use some flags, eg --backup-dir and if this is set to
8984 0 rclone may build two remotes (one for the source or destination and
8985 one for the --backup-dir where it may have only built one before.
8986 --fs-cache-expire-interval=TIME
8988 This controls how often rclone checks for cached remotes to expire.
8989 See the --fs-cache-expire-duration documentation above for more info.
8990 The default is 60s, set to 0 to disable expiry.
8991 --header
8993 Add an HTTP header for all transactions. The flag can be repeated to
8994 add multiple headers.
8995 If you want to add headers only for uploads use --header-upload and if
8997 you want to add headers only for downloads use --header-download.
8998 This flag is supported for all HTTP based backends even those not sup‐
9000 ported by --header-upload and --header-download so may be used as a
9001 workaround for those with care.
9002 rclone ls remote:test --header "X-Rclone: Foo" --header "X-LetMeIn: Yes"
9004 --header-download
9006 Add an HTTP header for all download transactions. The flag can be re‐
9007 peated to add multiple headers.
9008 rclone sync -i s3:test/src ~/dst --header-download "X-Amz-Meta-Test: Foo" --header-download "X-Amz-Meta-Test2: Bar"
9010 See the GitHub issue here (https://github.com/rclone/rclone/issues/59)
9012 for currently supported backends.
9013 --header-upload
9015 Add an HTTP header for all upload transactions. The flag can be re‐
9016 peated to add multiple headers.
9017 rclone sync -i ~/src s3:test/dst --header-upload "Content-Disposition: attachment; filename='cool.html'" --header-upload "X-Amz-Meta-Test: FooBar"
9019 See the GitHub issue here (https://github.com/rclone/rclone/issues/59)
9021 for currently supported backends.
9022 --human-readable
9024 Rclone commands output values for sizes (e.g. number of bytes) and
9025 counts (e.g. number of files) either as raw numbers, or in human-read‐
9026 able format.
9027 In human-readable format the values are scaled to larger units, indi‐
9029 cated with a suffix shown after the value, and rounded to three deci‐
9030 mals. Rclone consistently uses binary units (powers of 2) for sizes
9031 and decimal units (powers of 10) for counts. The unit prefix for size
9032 is according to IEC standard notation, e.g. Ki for kibi. Used with
9033 byte unit, 1 KiB means 1024 Byte. In list type of output, only the
9034 unit prefix appended to the value (e.g. 9.762Ki), while in more textu‐
9035 al output the full unit is shown (e.g. 9.762 KiB). For counts the SI
9036 standard notation is used, e.g. prefix k for kilo. Used with file
9037 counts, 1k means 1000 files.
9038 The various list (https://rclone.org/commands/rclone_ls/) commands out‐
9040 put raw numbers by default. Option --human-readable will make them
9041 output values in human-readable format instead (with the short unit
9042 prefix).
9043 The about (https://rclone.org/commands/rclone_about/) command outputs
9045 human-readable by default, with a command-specific option --full to
9046 output the raw numbers instead.
9047 Command size (https://rclone.org/commands/rclone_size/) outputs both
9049 human-readable and raw numbers in the same output.
9050 The tree (https://rclone.org/commands/rclone_tree/) command also con‐
9052 siders --human-readable, but it will not use the exact same notation as
9053 the other commands: It rounds to one decimal, and uses single letter
9054 suffix, e.g. K instead of Ki. The reason for this is that it relies
9055 on an external library.
9056 The interactive command ncdu (https://rclone.org/commands/rclone_ncdu/)
9058 shows human-readable by default, and responds to key u for toggling hu‐
9059 man-readable format.
9060 --ignore-case-sync
9062 Using this option will cause rclone to ignore the case of the files
9063 when synchronizing so files will not be copied/synced when the existing
9064 filenames are the same, even if the casing is different.
9065 --ignore-checksum
9067 Normally rclone will check that the checksums of transferred files
9068 match, and give an error "corrupted on transfer" if they don't.
9069 You can use this option to skip that check. You should only use it if
9071 you have had the "corrupted on transfer" error message and you are sure
9072 you might want to transfer potentially corrupted data.
9073 --ignore-existing
9075 Using this option will make rclone unconditionally skip all files that
9076 exist on the destination, no matter the content of these files.
9077 While this isn't a generally recommended option, it can be useful in
9079 cases where your files change due to encryption. However, it cannot
9080 correct partial transfers in case a transfer was interrupted.
9081 When performing a move/moveto command, this flag will leave skipped
9083 files in the source location unchanged when a file with the same name
9084 exists on the destination.
9085 --ignore-size
9087 Normally rclone will look at modification time and size of files to see
9088 if they are equal. If you set this flag then rclone will check only
9089 the modification time. If --checksum is set then it only checks the
9090 checksum.
9091 It will also cause rclone to skip verifying the sizes are the same af‐
9093 ter transfer.
9094 This can be useful for transferring files to and from OneDrive which
9096 occasionally misreports the size of image files (see #399
9097 (https://github.com/rclone/rclone/issues/399) for more info).
9098 -I, --ignore-times
9100 Using this option will cause rclone to unconditionally upload all files
9101 regardless of the state of files on the destination.
9102 Normally rclone would skip any files that have the same modification
9104 time and are the same size (or have the same checksum if using --check‐
9105 sum).
9106 --immutable
9108 Treat source and destination files as immutable and disallow modifica‐
9109 tion.
9110 With this option set, files will be created and deleted as requested,
9112 but existing files will never be updated. If an existing file does not
9113 match between the source and destination, rclone will give the error
9114 Source and destination exist but do not match: immutable file modified.
9115 Note that only commands which transfer files (e.g. sync, copy, move)
9117 are affected by this behavior, and only modification is disallowed.
9118 Files may still be deleted explicitly (e.g. delete, purge) or implic‐
9119 itly (e.g. sync, move). Use copy --immutable if it is desired to
9120 avoid deletion as well as modification.
9121 This can be useful as an additional layer of protection for immutable
9123 or append-only data sets (notably backup archives), where modification
9124 implies corruption and should not be propagated.
9125 -i / --interactive
9127 This flag can be used to tell rclone that you wish a manual confirma‐
9128 tion before destructive operations.
9129 It is recommended that you use this flag while learning rclone espe‐
9131 cially with rclone sync.
9132 For example
9134 $ rclone delete -i /tmp/dir
9136 rclone: delete "important-file.txt"?
9137 y) Yes, this is OK (default)
9138 n) No, skip this
9139 s) Skip all delete operations with no more questions
9140 !) Do all delete operations with no more questions
9141 q) Exit rclone now.
9142 y/n/s/!/q> n
9143 The options mean
9145 • y: Yes, this operation should go ahead. You can also press Return
9147 for this to happen. You'll be asked every time unless you choose s
9148 or !.
9149 • n: No, do not do this operation. You'll be asked every time unless
9151 you choose s or !.
9152 • s: Skip all the following operations of this type with no more ques‐
9154 tions. This takes effect until rclone exits. If there are any dif‐
9155 ferent kind of operations you'll be prompted for them.
9156 • !: Do all the following operations with no more questions. Useful if
9158 you've decided that you don't mind rclone doing that kind of opera‐
9159 tion. This takes effect until rclone exits . If there are any dif‐
9160 ferent kind of operations you'll be prompted for them.
9161 • q: Quit rclone now, just in case!
9163 --leave-root
9165 During rmdirs it will not remove root directory, even if it's empty.
9166 --log-file=FILE
9168 Log all of rclone's output to FILE. This is not active by default.
9169 This can be useful for tracking down problems with syncs in combination
9170 with the -v flag. See the Logging section for more info.
9171 If FILE exists then rclone will append to it.
9173 Note that if you are using the logrotate program to manage rclone's
9175 logs, then you should use the copytruncate option as rclone doesn't
9176 have a signal to rotate logs.
9177 --log-format LIST
9179 Comma separated list of log format options. Accepted options are date,
9180 time, microseconds, pid, longfile, shortfile, UTC. Any other keywords
9181 will be silently ignored. pid will tag log messages with process iden‐
9182 tifier which useful with rclone mount --daemon. Other accepted options
9183 are explained in the go documentation (https://pkg.go.dev/log#pkg-con‐
9184 stants). The default log format is "date,time".
9185 --log-level LEVEL
9187 This sets the log level for rclone. The default log level is NOTICE.
9188 DEBUG is equivalent to -vv. It outputs lots of debug info - useful for
9190 bug reports and really finding out what rclone is doing.
9191 INFO is equivalent to -v. It outputs information about each transfer
9193 and prints stats once a minute by default.
9194 NOTICE is the default log level if no logging flags are supplied. It
9196 outputs very little when things are working normally. It outputs warn‐
9197 ings and significant events.
9198 ERROR is equivalent to -q. It only outputs error messages.
9200 --use-json-log
9202 This switches the log format to JSON for rclone. The fields of json
9203 log are level, msg, source, time.
9204 --low-level-retries NUMBER
9206 This controls the number of low level retries rclone does.
9207 A low level retry is used to retry a failing operation - typically one
9209 HTTP request. This might be uploading a chunk of a big file for exam‐
9210 ple. You will see low level retries in the log with the -v flag.
9211 This shouldn't need to be changed from the default in normal opera‐
9213 tions. However, if you get a lot of low level retries you may wish to
9214 reduce the value so rclone moves on to a high level retry (see the
9215 --retries flag) quicker.
9216 Disable low level retries with --low-level-retries 1.
9218 --max-backlog=N
9220 This is the maximum allowable backlog of files in a sync/copy/move
9221 queued for being checked or transferred.
9222 This can be set arbitrarily large. It will only use memory when the
9224 queue is in use. Note that it will use in the order of N KiB of memory
9225 when the backlog is in use.
9226 Setting this large allows rclone to calculate how many files are pend‐
9228 ing more accurately, give a more accurate estimated finish time and
9229 make --order-by work more accurately.
9230 Setting this small will make rclone more synchronous to the listings of
9232 the remote which may be desirable.
9233 Setting this to a negative number will make the backlog as large as
9235 possible.
9236 --max-delete=N
9238 This tells rclone not to delete more than N files. If that limit is
9239 exceeded then a fatal error will be generated and rclone will stop the
9240 operation in progress.
9241 --max-depth=N
9243 This modifies the recursion depth for all the commands except purge.
9244 So if you do rclone --max-depth 1 ls remote:path you will see only the
9246 files in the top level directory. Using --max-depth 2 means you will
9247 see all the files in first two directory levels and so on.
9248 For historical reasons the lsd command defaults to using a --max-depth
9250 of 1 - you can override this with the command line flag.
9251 You can use this command to disable recursion (with --max-depth 1).
9253 Note that if you use this with sync and --delete-excluded the files not
9255 recursed through are considered excluded and will be deleted on the
9256 destination. Test first with --dry-run if you are not sure what will
9257 happen.
9258 --max-duration=TIME
9260 Rclone will stop scheduling new transfers when it has run for the dura‐
9261 tion specified.
9262 Defaults to off.
9264 When the limit is reached any existing transfers will complete.
9266 Rclone won't exit with an error if the transfer limit is reached.
9268 --max-transfer=SIZE
9270 Rclone will stop transferring when it has reached the size specified.
9271 Defaults to off.
9272 When the limit is reached all transfers will stop immediately.
9274 Rclone will exit with exit code 8 if the transfer limit is reached.
9276 --metadata / -M
9278 Setting this flag enables rclone to copy the metadata from the source
9279 to the destination. For local backends this is ownership, permissions,
9280 xattr etc. See the #metadata for more info.
9281 --metadata-set key=value
9283 Add metadata key = value when uploading. This can be repeated as many
9284 times as required. See the #metadata for more info.
9285 --cutoff-mode=hard|soft|cautious
9287 This modifies the behavior of --max-transfer Defaults to --cutoff-
9288 mode=hard.
9289 Specifying --cutoff-mode=hard will stop transferring immediately when
9291 Rclone reaches the limit.
9292 Specifying --cutoff-mode=soft will stop starting new transfers when
9294 Rclone reaches the limit.
9295 Specifying --cutoff-mode=cautious will try to prevent Rclone from
9297 reaching the limit.
9298 --modify-window=TIME
9300 When checking whether a file has been modified, this is the maximum al‐
9301 lowed time difference that a file can have and still be considered
9302 equivalent.
9303 The default is 1ns unless this is overridden by a remote. For example
9305 OS X only stores modification times to the nearest second so if you are
9306 reading and writing to an OS X filing system this will be 1s by de‐
9307 fault.
9308 This command line flag allows you to override that computed default.
9310 --multi-thread-cutoff=SIZE
9312 When downloading files to the local backend above this size, rclone
9313 will use multiple threads to download the file (default 250M).
9314 Rclone preallocates the file (using fallocate(FALLOC_FL_KEEP_SIZE) on
9316 unix or NTSetInformationFile on Windows both of which takes no time)
9317 then each thread writes directly into the file at the correct place.
9318 This means that rclone won't create fragmented or sparse files and
9319 there won't be any assembly time at the end of the transfer.
9320 The number of threads used to download is controlled by --multi-thread-
9322 streams.
9323 Use -vv if you wish to see info about the threads.
9325 This will work with the sync/copy/move commands and friends copy‐
9327 to/moveto. Multi thread downloads will be used with rclone mount and
9328 rclone serve if --vfs-cache-mode is set to writes or above.
9329 NB that this only works for a local destination but will work with any
9331 source.
9332 NB that multi thread copies are disabled for local to local copies as
9334 they are faster without unless --multi-thread-streams is set explicit‐
9335 ly.
9336 NB on Windows using multi-thread downloads will cause the resulting
9338 files to be sparse (https://en.wikipedia.org/wiki/Sparse_file). Use
9339 --local-no-sparse to disable sparse files (which may cause long delays
9340 at the start of downloads) or disable multi-thread downloads with
9341 --multi-thread-streams 0
9342 --multi-thread-streams=N
9344 When using multi thread downloads (see above --multi-thread-cutoff)
9345 this sets the maximum number of streams to use. Set to 0 to disable
9346 multi thread downloads (Default 4).
9347 Exactly how many streams rclone uses for the download depends on the
9349 size of the file. To calculate the number of download streams Rclone
9350 divides the size of the file by the --multi-thread-cutoff and rounds
9351 up, up to the maximum set with --multi-thread-streams.
9352 So if --multi-thread-cutoff 250M and --multi-thread-streams 4 are in
9354 effect (the defaults):
9355 • 0..250 MiB files will be downloaded with 1 stream
9357 • 250..500 MiB files will be downloaded with 2 streams
9359 • 500..750 MiB files will be downloaded with 3 streams
9361 • 750+ MiB files will be downloaded with 4 streams
9363 --no-check-dest
9365 The --no-check-dest can be used with move or copy and it causes rclone
9366 not to check the destination at all when copying files.
9367 This means that:
9369 • the destination is not listed minimising the API calls
9371 • files are always transferred
9373 • this can cause duplicates on remotes which allow it (e.g. Google
9375 Drive)
9376 • --retries 1 is recommended otherwise you'll transfer everything again
9378 on a retry
9379 This flag is useful to minimise the transactions if you know that none
9381 of the files are on the destination.
9382 This is a specialized flag which should be ignored by most users!
9384 --no-gzip-encoding
9386 Don't set Accept-Encoding: gzip. This means that rclone won't ask the
9387 server for compressed files automatically. Useful if you've set the
9388 server to return files with Content-Encoding: gzip but you uploaded
9389 compressed files.
9390 There is no need to set this in normal operation, and doing so will de‐
9392 crease the network transfer efficiency of rclone.
9393 --no-traverse
9395 The --no-traverse flag controls whether the destination file system is
9396 traversed when using the copy or move commands. --no-traverse is not
9397 compatible with sync and will be ignored if you supply it with sync.
9398 If you are only copying a small number of files (or are filtering most
9400 of the files) and/or have a large number of files on the destination
9401 then --no-traverse will stop rclone listing the destination and save
9402 time.
9403 However, if you are copying a large number of files, especially if you
9405 are doing a copy where lots of the files under consideration haven't
9406 changed and won't need copying then you shouldn't use --no-traverse.
9407 See rclone copy (https://rclone.org/commands/rclone_copy/) for an exam‐
9409 ple of how to use it.
9410 --no-unicode-normalization
9412 Don't normalize unicode characters in filenames during the sync rou‐
9413 tine.
9414 Sometimes, an operating system will store filenames containing unicode
9416 parts in their decomposed form (particularly macOS). Some cloud stor‐
9417 age systems will then recompose the unicode, resulting in duplicate
9418 files if the data is ever copied back to a local filesystem.
9419 Using this flag will disable that functionality, treating each unicode
9421 character as unique. For example, by default é and é will be normal‐
9422 ized into the same character. With --no-unicode-normalization they
9423 will be treated as unique characters.
9424 --no-update-modtime
9426 When using this flag, rclone won't update modification times of remote
9427 files if they are incorrect as it would normally.
9428 This can be used if the remote is being synced with another tool also
9430 (e.g. the Google Drive client).
9431 --order-by string
9433 The --order-by flag controls the order in which files in the backlog
9434 are processed in rclone sync, rclone copy and rclone move.
9435 The order by string is constructed like this. The first part describes
9437 what aspect is being measured:
9438 • size - order by the size of the files
9440 • name - order by the full path of the files
9442 • modtime - order by the modification date of the files
9444 This can have a modifier appended with a comma:
9446 • ascending or asc - order so that the smallest (or oldest) is pro‐
9448 cessed first
9449 • descending or desc - order so that the largest (or newest) is pro‐
9451 cessed first
9452 • mixed - order so that the smallest is processed first for some
9454 threads and the largest for others
9455 If the modifier is mixed then it can have an optional percentage (which
9457 defaults to 50), e.g. size,mixed,25 which means that 25% of the
9458 threads should be taking the smallest items and 75% the largest. The
9459 threads which take the smallest first will always take the smallest
9460 first and likewise the largest first threads. The mixed mode can be
9461 useful to minimise the transfer time when you are transferring a mix‐
9462 ture of large and small files - the large files are guaranteed upload
9463 threads and bandwidth and the small files will be processed continuous‐
9464 ly.
9465 If no modifier is supplied then the order is ascending.
9467 For example
9469 • --order-by size,desc - send the largest files first
9471 • --order-by modtime,ascending - send the oldest files first
9473 • --order-by name - send the files with alphabetically by path first
9475 If the --order-by flag is not supplied or it is supplied with an empty
9477 string then the default ordering will be used which is as scanned.
9478 With --checkers 1 this is mostly alphabetical, however with the default
9479 --checkers 8 it is somewhat random.
9480 Limitations
9482 The --order-by flag does not do a separate pass over the data. This
9483 means that it may transfer some files out of the order specified if
9484 • there are no files in the backlog or the source has not been fully
9486 scanned yet
9487 • there are more than --max-backlog files in the backlog
9489 Rclone will do its best to transfer the best file it has so in practice
9491 this should not cause a problem. Think of --order-by as being more of
9492 a best efforts flag rather than a perfect ordering.
9493 If you want perfect ordering then you will need to specify --check-
9495 first which will find all the files which need transferring first be‐
9496 fore transferring any.
9497 --password-command SpaceSepList
9499 This flag supplies a program which should supply the config password
9500 when run. This is an alternative to rclone prompting for the password
9501 or setting the RCLONE_CONFIG_PASS variable.
9502 The argument to this should be a command with a space separated list of
9504 arguments. If one of the arguments has a space in then enclose it in
9505 ", if you want a literal " in an argument then enclose the argument in
9506 " and double the ". See CSV encoding (https://godoc.org/encoding/csv)
9507 for more info.
9508 Eg
9510 --password-command echo hello
9512 --password-command echo "hello with space"
9513 --password-command echo "hello with ""quotes"" and space"
9514 See the Configuration Encryption for more info.
9516 See a Windows PowerShell example on the Wiki
9518 (https://github.com/rclone/rclone/wiki/Windows-Powershell-use-rclone-
9519 password-command-for-Config-file-password).
9520 -P, --progress
9522 This flag makes rclone update the stats in a static block in the termi‐
9523 nal providing a realtime overview of the transfer.
9524 Any log messages will scroll above the static block. Log messages will
9526 push the static block down to the bottom of the terminal where it will
9527 stay.
9528 Normally this is updated every 500mS but this period can be overridden
9530 with the --stats flag.
9531 This can be used with the --stats-one-line flag for a simpler display.
9533 Note: On Windows until this bug (https://github.com/Azure/go-an‐
9535 siterm/issues/26) is fixed all non-ASCII characters will be replaced
9536 with . when --progress is in use.
9537 --progress-terminal-title
9539 This flag, when used with -P/--progress, will print the string ETA: %s
9540 to the terminal title.
9541 -q, --quiet
9543 This flag will limit rclone's output to error messages only.
9544 --refresh-times
9546 The --refresh-times flag can be used to update modification times of
9547 existing files when they are out of sync on backends which don't sup‐
9548 port hashes.
9549 This is useful if you uploaded files with the incorrect timestamps and
9551 you now wish to correct them.
9552 This flag is only useful for destinations which don't support hashes
9554 (e.g. crypt).
9555 This can be used any of the sync commands sync, copy or move.
9557 To use this flag you will need to be doing a modification time sync (so
9559 not using --size-only or --checksum). The flag will have no effect
9560 when using --size-only or --checksum.
9561 If this flag is used when rclone comes to upload a file it will check
9563 to see if there is an existing file on the destination. If this file
9564 matches the source with size (and checksum if available) but has a dif‐
9565 fering timestamp then instead of re-uploading it, rclone will update
9566 the timestamp on the destination file. If the checksum does not match
9567 rclone will upload the new file. If the checksum is absent (e.g. on a
9568 crypt backend) then rclone will update the timestamp.
9569 Note that some remotes can't set the modification time without re-up‐
9571 loading the file so this flag is less useful on them.
9572 Normally if you are doing a modification time sync rclone will update
9574 modification times without --refresh-times provided that the remote
9575 supports checksums and the checksums match on the file. However if the
9576 checksums are absent then rclone will upload the file rather than set‐
9577 ting the timestamp as this is the safe behaviour.
9578 --retries int
9580 Retry the entire sync if it fails this many times it fails (default 3).
9581 Some remotes can be unreliable and a few retries help pick up the files
9583 which didn't get transferred because of errors.
9584 Disable retries with --retries 1.
9586 --retries-sleep=TIME
9588 This sets the interval between each retry specified by --retries
9589 The default is 0. Use 0 to disable.
9591 --server-side-across-configs
9593 Allow server-side operations (e.g. copy or move) to work across dif‐
9594 ferent configurations.
9595 This can be useful if you wish to do a server-side copy or move between
9597 two remotes which use the same backend but are configured differently.
9598 Note that this isn't enabled by default because it isn't easy for
9600 rclone to tell if it will work between any two configurations.
9601 --size-only
9603 Normally rclone will look at modification time and size of files to see
9604 if they are equal. If you set this flag then rclone will check only
9605 the size.
9606 This can be useful transferring files from Dropbox which have been mod‐
9608 ified by the desktop sync client which doesn't set checksums of modifi‐
9609 cation times in the same way as rclone.
9610 --stats=TIME
9612 Commands which transfer data (sync, copy, copyto, move, moveto) will
9613 print data transfer stats at regular intervals to show their progress.
9614 This sets the interval.
9616 The default is 1m. Use 0 to disable.
9618 If you set the stats interval then all commands can show stats. This
9620 can be useful when running other commands, check or mount for example.
9621 Stats are logged at INFO level by default which means they won't show
9623 at default log level NOTICE. Use --stats-log-level NOTICE or -v to
9624 make them show. See the Logging section for more info on log levels.
9625 Note that on macOS you can send a SIGINFO (which is normally ctrl-T in
9627 the terminal) to make the stats print immediately.
9628 --stats-file-name-length integer
9630 By default, the --stats output will truncate file names and paths
9631 longer than 40 characters. This is equivalent to providing --stats-
9632 file-name-length 40. Use --stats-file-name-length 0 to disable any
9633 truncation of file names printed by stats.
9634 --stats-log-level string
9636 Log level to show --stats output at. This can be DEBUG, INFO, NOTICE,
9637 or ERROR. The default is INFO. This means at the default level of
9638 logging which is NOTICE the stats won't show - if you want them to then
9639 use --stats-log-level NOTICE. See the Logging section for more info on
9640 log levels.
9641 --stats-one-line
9643 When this is specified, rclone condenses the stats into a single line
9644 showing the most important stats only.
9645 --stats-one-line-date
9647 When this is specified, rclone enables the single-line stats and
9648 prepends the display with a date string. The default is 2006/01/02
9649 15:04:05 -
9650 --stats-one-line-date-format
9652 When this is specified, rclone enables the single-line stats and
9653 prepends the display with a user-supplied date string. The date string
9654 MUST be enclosed in quotes. Follow golang specs
9655 (https://golang.org/pkg/time/#Time.Format) for date formatting syntax.
9656 --stats-unit=bits|bytes
9658 By default, data transfer rates will be printed in bytes per second.
9659 This option allows the data rate to be printed in bits per second.
9661 Data transfer volume will still be reported in bytes.
9663 The rate is reported as a binary unit, not SI unit. So 1 Mbit/s equals
9665 1,048,576 bit/s and not 1,000,000 bit/s.
9666 The default is bytes.
9668 --suffix=SUFFIX
9670 When using sync, copy or move any files which would have been overwrit‐
9671 ten or deleted will have the suffix added to them. If there is a file
9672 with the same path (after the suffix has been added), then it will be
9673 overwritten.
9674 The remote in use must support server-side move or copy and you must
9676 use the same remote as the destination of the sync.
9677 This is for use with files to add the suffix in the current directory
9679 or with --backup-dir. See --backup-dir for more info.
9680 For example
9682 rclone copy -i /path/to/local/file remote:current --suffix .bak
9684 will copy /path/to/local to remote:current, but for any files which
9686 would have been updated or deleted have .bak added.
9687 If using rclone sync with --suffix and without --backup-dir then it is
9689 recommended to put a filter rule in excluding the suffix otherwise the
9690 sync will delete the backup files.
9691 rclone sync -i /path/to/local/file remote:current --suffix .bak --exclude "*.bak"
9693 --suffix-keep-extension
9695 When using --suffix, setting this causes rclone put the SUFFIX before
9696 the extension of the files that it backs up rather than after.
9697 So let's say we had --suffix -2019-01-01, without the flag file.txt
9699 would be backed up to file.txt-2019-01-01 and with the flag it would be
9700 backed up to file-2019-01-01.txt. This can be helpful to make sure the
9701 suffixed files can still be opened.
9702 --syslog
9704 On capable OSes (not Windows or Plan9) send all log output to syslog.
9705 This can be useful for running rclone in a script or rclone mount.
9707 --syslog-facility string
9709 If using --syslog this sets the syslog facility (e.g. KERN, USER).
9710 See man syslog for a list of possible facilities. The default facility
9711 is DAEMON.
9712 --temp-dir=DIR
9714 Specify the directory rclone will use for temporary files, to override
9715 the default. Make sure the directory exists and have accessible per‐
9716 missions.
9717 By default the operating system's temp directory will be used: - On
9719 Unix systems, $TMPDIR if non-empty, else /tmp. - On Windows, the first
9720 non-empty value from %TMP%, %TEMP%, %USERPROFILE%, or the Windows di‐
9721 rectory.
9722 When overriding the default with this option, the specified path will
9724 be set as value of environment variable TMPDIR on Unix systems and TMP
9725 and TEMP on Windows.
9726 You can use the config paths (https://rclone.org/commands/rclone_con‐
9728 fig_paths/) command to see the current value.
9729 --tpslimit float
9731 Limit transactions per second to this number. Default is 0 which is
9732 used to mean unlimited transactions per second.
9733 A transaction is roughly defined as an API call; its exact meaning will
9735 depend on the backend. For HTTP based backends it is an HTTP
9736 PUT/GET/POST/etc and its response. For FTP/SFTP it is a round trip
9737 transaction over TCP.
9738 For example, to limit rclone to 10 transactions per second use --tp‐
9740 slimit 10, or to 1 transaction every 2 seconds use --tpslimit 0.5.
9741 Use this when the number of transactions per second from rclone is
9743 causing a problem with the cloud storage provider (e.g. getting you
9744 banned or rate limited).
9745 This can be very useful for rclone mount to control the behaviour of
9747 applications using it.
9748 This limit applies to all HTTP based backends and to the FTP and SFTP
9750 backends. It does not apply to the local backend or the Storj backend.
9751 See also --tpslimit-burst.
9753 --tpslimit-burst int
9755 Max burst of transactions for --tpslimit (default 1).
9756 Normally --tpslimit will do exactly the number of transaction per sec‐
9758 ond specified. However if you supply --tps-burst then rclone can save
9759 up some transactions from when it was idle giving a burst of up to the
9760 parameter supplied.
9761 For example if you provide --tpslimit-burst 10 then if rclone has been
9763 idle for more than 10*--tpslimit then it can do 10 transactions very
9764 quickly before they are limited again.
9765 This may be used to increase performance of --tpslimit without changing
9767 the long term average number of transactions per second.
9768 --track-renames
9770 By default, rclone doesn't keep track of renamed files, so if you re‐
9771 name a file locally then sync it to a remote, rclone will delete the
9772 old file on the remote and upload a new copy.
9773 An rclone sync with --track-renames runs like a normal sync, but keeps
9775 track of objects which exist in the destination but not in the source
9776 (which would normally be deleted), and which objects exist in the
9777 source but not the destination (which would normally be transferred).
9778 These objects are then candidates for renaming.
9779 After the sync, rclone matches up the source only and destination only
9781 objects using the --track-renames-strategy specified and either renames
9782 the destination object or transfers the source and deletes the destina‐
9783 tion object. --track-renames is stateless like all of rclone's syncs.
9784 To use this flag the destination must support server-side copy or serv‐
9786 er-side move, and to use a hash based --track-renames-strategy (the de‐
9787 fault) the source and the destination must have a compatible hash.
9788 If the destination does not support server-side copy or move, rclone
9790 will fall back to the default behaviour and log an error level message
9791 to the console.
9792 Encrypted destinations are not currently supported by --track-renames
9794 if --track-renames-strategy includes hash.
9795 Note that --track-renames is incompatible with --no-traverse and that
9797 it uses extra memory to keep track of all the rename candidates.
9798 Note also that --track-renames is incompatible with --delete-before and
9800 will select --delete-after instead of --delete-during.
9801 --track-renames-strategy (hash,modtime,leaf,size)
9803 This option changes the file matching criteria for --track-renames.
9804 The matching is controlled by a comma separated selection of these to‐
9806 kens:
9807 • modtime - the modification time of the file - not supported on all
9809 backends
9810 • hash - the hash of the file contents - not supported on all backends
9812 • leaf - the name of the file not including its directory name
9814 • size - the size of the file (this is always enabled)
9816 The default option is hash.
9818 Using --track-renames-strategy modtime,leaf would match files based on
9820 modification time, the leaf of the file name and the size only.
9821 Using --track-renames-strategy modtime or leaf can enable --track-re‐
9823 names support for encrypted destinations.
9824 Note that the hash strategy is not supported with encrypted destina‐
9826 tions.
9827 --delete-(before,during,after)
9829 This option allows you to specify when files on your destination are
9830 deleted when you sync folders.
9831 Specifying the value --delete-before will delete all files present on
9833 the destination, but not on the source before starting the transfer of
9834 any new or updated files. This uses two passes through the file sys‐
9835 tems, one for the deletions and one for the copies.
9836 Specifying --delete-during will delete files while checking and upload‐
9838 ing files. This is the fastest option and uses the least memory.
9839 Specifying --delete-after (the default value) will delay deletion of
9841 files until all new/updated files have been successfully transferred.
9842 The files to be deleted are collected in the copy pass then deleted af‐
9843 ter the copy pass has completed successfully. The files to be deleted
9844 are held in memory so this mode may use more memory. This is the
9845 safest mode as it will only delete files if there have been no errors
9846 subsequent to that. If there have been errors before the deletions
9847 start then you will get the message not deleting files as there were IO
9848 errors.
9849 --fast-list
9851 When doing anything which involves a directory listing (e.g. sync,
9852 copy, ls - in fact nearly every command), rclone normally lists a di‐
9853 rectory and processes it before using more directory lists to process
9854 any subdirectories. This can be parallelised and works very quickly
9855 using the least amount of memory.
9856 However, some remotes have a way of listing all files beneath a direc‐
9858 tory in one (or a small number) of transactions. These tend to be the
9859 bucket-based remotes (e.g. S3, B2, GCS, Swift).
9860 If you use the --fast-list flag then rclone will use this method for
9862 listing directories. This will have the following consequences for the
9863 listing:
9864 • It will use fewer transactions (important if you pay for them)
9866 • It will use more memory. Rclone has to load the whole listing into
9868 memory.
9869 • It may be faster because it uses fewer transactions
9871 • It may be slower because it can't be parallelized
9873 rclone should always give identical results with and without --fast-
9875 list.
9876 If you pay for transactions and can fit your entire sync listing into
9878 memory then --fast-list is recommended. If you have a very big sync to
9879 do then don't use --fast-list otherwise you will run out of memory.
9880 If you use --fast-list on a remote which doesn't support it, then
9882 rclone will just ignore it.
9883 --timeout=TIME
9885 This sets the IO idle timeout. If a transfer has started but then be‐
9886 comes idle for this long it is considered broken and disconnected.
9887 The default is 5m. Set to 0 to disable.
9889 --transfers=N
9891 The number of file transfers to run in parallel. It can sometimes be
9892 useful to set this to a smaller number if the remote is giving a lot of
9893 timeouts or bigger if you have lots of bandwidth and a fast remote.
9894 The default is to run 4 file transfers in parallel.
9896 Look at --multi-thread-streams if you would like to control single file
9898 transfers.
9899 -u, --update
9901 This forces rclone to skip any files which exist on the destination and
9902 have a modified time that is newer than the source file.
9903 This can be useful in avoiding needless transfers when transferring to
9905 a remote which doesn't support modification times directly (or when us‐
9906 ing --use-server-modtime to avoid extra API calls) as it is more accu‐
9907 rate than a --size-only check and faster than using --checksum. On
9908 such remotes (or when using --use-server-modtime) the time checked will
9909 be the uploaded time.
9910 If an existing destination file has a modification time older than the
9912 source file's, it will be updated if the sizes are different. If the
9913 sizes are the same, it will be updated if the checksum is different or
9914 not available.
9915 If an existing destination file has a modification time equal (within
9917 the computed modify window) to the source file's, it will be updated if
9918 the sizes are different. The checksum will not be checked in this case
9919 unless the --checksum flag is provided.
9920 In all other cases the file will not be updated.
9922 Consider using the --modify-window flag to compensate for time skews
9924 between the source and the backend, for backends that do not support
9925 mod times, and instead use uploaded times. However, if the backend
9926 does not support checksums, note that syncing or copying within the
9927 time skew window may still result in additional transfers for safety.
9928 --use-mmap
9930 If this flag is set then rclone will use anonymous memory allocated by
9931 mmap on Unix based platforms and VirtualAlloc on Windows for its trans‐
9932 fer buffers (size controlled by --buffer-size). Memory allocated like
9933 this does not go on the Go heap and can be returned to the OS immedi‐
9934 ately when it is finished with.
9935 If this flag is not set then rclone will allocate and free the buffers
9937 using the Go memory allocator which may use more memory as memory pages
9938 are returned less aggressively to the OS.
9939 It is possible this does not work well on all platforms so it is dis‐
9941 abled by default; in the future it may be enabled by default.
9942 --use-server-modtime
9944 Some object-store backends (e.g, Swift, S3) do not preserve file modi‐
9945 fication times (modtime). On these backends, rclone stores the origi‐
9946 nal modtime as additional metadata on the object. By default it will
9947 make an API call to retrieve the metadata when the modtime is needed by
9948 an operation.
9949 Use this flag to disable the extra API call and rely instead on the
9951 server's modified time. In cases such as a local to remote sync using
9952 --update, knowing the local file is newer than the time it was last up‐
9953 loaded to the remote is sufficient. In those cases, this flag can
9954 speed up the process and reduce the number of API calls necessary.
9955 Using this flag on a sync operation without also using --update would
9957 cause all files modified at any time other than the last upload time to
9958 be uploaded again, which is probably not what you want.
9959 -v, -vv, --verbose
9961 With -v rclone will tell you about each file that is transferred and a
9962 small number of significant events.
9963 With -vv rclone will become very verbose telling you about every file
9965 it considers and transfers. Please send bug reports with a log with
9966 this setting.
9967 When setting verbosity as an environment variable, use RCLONE_VERBOSE=1
9969 or RCLONE_VERBOSE=2 for -v and -vv respectively.
9970 -V, --version
9972 Prints the version number
9973 SSL/TLS options
9975 The outgoing SSL/TLS connections rclone makes can be controlled with
9976 these options. For example this can be very useful with the HTTP or
9977 WebDAV backends. Rclone HTTP servers have their own set of configura‐
9978 tion for SSL/TLS which you can find in their documentation.
9979 --ca-cert string
9981 This loads the PEM encoded certificate authority certificate and uses
9982 it to verify the certificates of the servers rclone connects to.
9983 If you have generated certificates signed with a local CA then you will
9985 need this flag to connect to servers using those certificates.
9986 --client-cert string
9988 This loads the PEM encoded client side certificate.
9989 This is used for mutual TLS authentication
9991 (https://en.wikipedia.org/wiki/Mutual_authentication).
9992 The --client-key flag is required too when using this.
9994 --client-key string
9996 This loads the PEM encoded client side private key used for mutual TLS
9997 authentication. Used in conjunction with --client-cert.
9998 --no-check-certificate=true/false
10000 --no-check-certificate controls whether a client verifies the server's
10001 certificate chain and host name. If --no-check-certificate is true,
10002 TLS accepts any certificate presented by the server and any host name
10003 in that certificate. In this mode, TLS is susceptible to man-in-the-
10004 middle attacks.
10005 This option defaults to false.
10007 This should be used only for testing.
10009 Configuration Encryption
10011 Your configuration file contains information for logging in to your
10012 cloud services. This means that you should keep your rclone.conf file
10013 in a secure location.
10014 If you are in an environment where that isn't possible, you can add a
10016 password to your configuration. This means that you will have to sup‐
10017 ply the password every time you start rclone.
10018 To add a password to your rclone configuration, execute rclone config.
10020 >rclone config
10022 Current remotes:
10023 e) Edit existing remote
10025 n) New remote
10026 d) Delete remote
10027 s) Set configuration password
10028 q) Quit config
10029 e/n/d/s/q>
10030 Go into s, Set configuration password:
10032 e/n/d/s/q> s
10034 Your configuration is not encrypted.
10035 If you add a password, you will protect your login information to cloud services.
10036 a) Add Password
10037 q) Quit to main menu
10038 a/q> a
10039 Enter NEW configuration password:
10040 password:
10041 Confirm NEW password:
10042 password:
10043 Password set
10044 Your configuration is encrypted.
10045 c) Change Password
10046 u) Unencrypt configuration
10047 q) Quit to main menu
10048 c/u/q>
10049 Your configuration is now encrypted, and every time you start rclone
10051 you will have to supply the password. See below for details. In the
10052 same menu, you can change the password or completely remove encryption
10053 from your configuration.
10054 There is no way to recover the configuration if you lose your password.
10056 rclone uses nacl secretbox (https://godoc.org/golang.org/x/crypto/na‐
10058 cl/secretbox) which in turn uses XSalsa20 and Poly1305 to encrypt and
10059 authenticate your configuration with secret-key cryptography. The
10060 password is SHA-256 hashed, which produces the key for secretbox. The
10061 hashed password is not stored.
10062 While this provides very good security, we do not recommend storing
10064 your encrypted rclone configuration in public if it contains sensitive
10065 information, maybe except if you use a very strong password.
10066 If it is safe in your environment, you can set the RCLONE_CONFIG_PASS
10068 environment variable to contain your password, in which case it will be
10069 used for decrypting the configuration.
10070 You can set this for a session from a script. For unix like systems
10072 save this to a file called set-rclone-password:
10073 #!/bin/echo Source this file don't run it
10075 read -s RCLONE_CONFIG_PASS
10077 export RCLONE_CONFIG_PASS
10078 Then source the file when you want to use it. From the shell you would
10080 do source set-rclone-password. It will then ask you for the password
10081 and set it in the environment variable.
10082 An alternate means of supplying the password is to provide a script
10084 which will retrieve the password and print on standard output. This
10085 script should have a fully specified path name and not rely on any en‐
10086 vironment variables. The script is supplied either via --password-com‐
10087 mand="..." command line argument or via the RCLONE_PASSWORD_COMMAND en‐
10088 vironment variable.
10089 One useful example of this is using the passwordstore application to
10091 retrieve the password:
10092 export RCLONE_PASSWORD_COMMAND="pass rclone/config"
10094 If the passwordstore password manager holds the password for the rclone
10096 configuration, using the script method means the password is primarily
10097 protected by the passwordstore system, and is never embedded in the
10098 clear in scripts, nor available for examination using the standard com‐
10099 mands available. It is quite possible with long running rclone ses‐
10100 sions for copies of passwords to be innocently captured in log files or
10101 terminal scroll buffers, etc. Using the script method of supplying the
10102 password enhances the security of the config password considerably.
10103 If you are running rclone inside a script, unless you are using the
10105 --password-command method, you might want to disable password prompts.
10106 To do that, pass the parameter --ask-password=false to rclone. This
10107 will make rclone fail instead of asking for a password if RCLONE_CON‐
10108 FIG_PASS doesn't contain a valid password, and --password-command has
10109 not been supplied.
10110 Whenever running commands that may be affected by options in a configu‐
10112 ration file, rclone will look for an existing file according to the
10113 rules described above, and load any it finds. If an encrypted file is
10114 found, this includes decrypting it, with the possible consequence of a
10115 password prompt. When executing a command line that you know are not
10116 actually using anything from such a configuration file, you can avoid
10117 it being loaded by overriding the location, e.g. with one of the docu‐
10118 mented special values for memory-only configuration. Since only back‐
10119 end options can be stored in configuration files, this is normally un‐
10120 necessary for commands that do not operate on backends, e.g. genauto‐
10121 complete. However, it will be relevant for commands that do operate on
10122 backends in general, but are used without referencing a stored remote,
10123 e.g. listing local filesystem paths, or connection strings: rclone
10124 --config="" ls .
10125 Developer options
10127 These options are useful when developing or debugging rclone. There
10128 are also some more remote specific options which aren't documented here
10129 which are used for testing. These start with remote name e.g.
10130 --drive-test-option - see the docs for the remote in question.
10131 --cpuprofile=FILE
10133 Write CPU profile to file. This can be analysed with go tool pprof.
10134 --dump flag,flag,flag
10136 The --dump flag takes a comma separated list of flags to dump info
10137 about.
10138 Note that some headers including Accept-Encoding as shown may not be
10140 correct in the request and the response may not show Content-Encoding
10141 if the go standard libraries auto gzip encoding was in effect. In this
10142 case the body of the request will be gunzipped before showing it.
10143 The available flags are:
10145 --dump headers
10147 Dump HTTP headers with Authorization: lines removed. May still contain
10148 sensitive info. Can be very verbose. Useful for debugging only.
10149 Use --dump auth if you do want the Authorization: headers.
10151 --dump bodies
10153 Dump HTTP headers and bodies - may contain sensitive info. Can be very
10154 verbose. Useful for debugging only.
10155 Note that the bodies are buffered in memory so don't use this for enor‐
10157 mous files.
10158 --dump requests
10160 Like --dump bodies but dumps the request bodies and the response head‐
10161 ers. Useful for debugging download problems.
10162 --dump responses
10164 Like --dump bodies but dumps the response bodies and the request head‐
10165 ers. Useful for debugging upload problems.
10166 --dump auth
10168 Dump HTTP headers - will contain sensitive info such as Authorization:
10169 headers - use --dump headers to dump without Authorization: headers.
10170 Can be very verbose. Useful for debugging only.
10171 --dump filters
10173 Dump the filters to the output. Useful to see exactly what include and
10174 exclude options are filtering on.
10175 --dump goroutines
10177 This dumps a list of the running go-routines at the end of the command
10178 to standard output.
10179 --dump openfiles
10181 This dumps a list of the open files at the end of the command. It uses
10182 the lsof command to do that so you'll need that installed to use it.
10183 --memprofile=FILE
10185 Write memory profile to file. This can be analysed with go tool pprof.
10186 Filtering
10188 For the filtering options
10189 • --delete-excluded
10191 • --filter
10193 • --filter-from
10195 • --exclude
10197 • --exclude-from
10199 • --exclude-if-present
10201 • --include
10203 • --include-from
10205 • --files-from
10207 • --files-from-raw
10209 • --min-size
10211 • --max-size
10213 • --min-age
10215 • --max-age
10217 • --dump filters
10219 See the filtering section (https://rclone.org/filtering/).
10221 Remote control
10223 For the remote control options and for instructions on how to remote
10224 control rclone
10225 • --rc
10227 • and anything starting with --rc-
10229 See the remote control section (https://rclone.org/rc/).
10231 Logging
10233 rclone has 4 levels of logging, ERROR, NOTICE, INFO and DEBUG.
10234 By default, rclone logs to standard error. This means you can redirect
10236 standard error and still see the normal output of rclone commands (e.g.
10237 rclone ls).
10238 By default, rclone will produce Error and Notice level messages.
10240 If you use the -q flag, rclone will only produce Error messages.
10242 If you use the -v flag, rclone will produce Error, Notice and Info mes‐
10244 sages.
10245 If you use the -vv flag, rclone will produce Error, Notice, Info and
10247 Debug messages.
10248 You can also control the log levels with the --log-level flag.
10250 If you use the --log-file=FILE option, rclone will redirect Error, Info
10252 and Debug messages along with standard error to FILE.
10253 If you use the --syslog flag then rclone will log to syslog and the
10255 --syslog-facility control which facility it uses.
10256 Rclone prefixes all log messages with their level in capitals, e.g.
10258 INFO which makes it easy to grep the log file for different kinds of
10259 information.
10260 Exit Code
10262 If any errors occur during the command execution, rclone will exit with
10263 a non-zero exit code. This allows scripts to detect when rclone opera‐
10264 tions have failed.
10265 During the startup phase, rclone will exit immediately if an error is
10267 detected in the configuration. There will always be a log message im‐
10268 mediately before exiting.
10269 When rclone is running it will accumulate errors as it goes along, and
10271 only exit with a non-zero exit code if (after retries) there were still
10272 failed transfers. For every error counted there will be a high priori‐
10273 ty log message (visible with -q) showing the message and which file
10274 caused the problem. A high priority message is also shown when start‐
10275 ing a retry so the user can see that any previous error messages may
10276 not be valid after the retry. If rclone has done a retry it will log a
10277 high priority message if the retry was successful.
10278 List of exit codes
10280 • 0 - success
10281 • 1 - Syntax or usage error
10283 • 2 - Error not otherwise categorised
10285 • 3 - Directory not found
10287 • 4 - File not found
10289 • 5 - Temporary error (one that more retries might fix) (Retry errors)
10291 • 6 - Less serious errors (like 461 errors from dropbox) (NoRetry er‐
10293 rors)
10294 • 7 - Fatal error (one that more retries won't fix, like account sus‐
10296 pended) (Fatal errors)
10297 • 8 - Transfer exceeded - limit set by --max-transfer reached
10299 • 9 - Operation successful, but no files transferred
10301 Environment Variables
10303 Rclone can be configured entirely using environment variables. These
10304 can be used to set defaults for options or config file entries.
10305 Options
10307 Every option in rclone can have its default set by environment vari‐
10308 able.
10309 To find the name of the environment variable, first, take the long op‐
10311 tion name, strip the leading --, change - to _, make upper case and
10312 prepend RCLONE_.
10313 For example, to always set --stats 5s, set the environment variable
10315 RCLONE_STATS=5s. If you set stats on the command line this will over‐
10316 ride the environment variable setting.
10317 Or to always use the trash in drive --drive-use-trash, set
10319 RCLONE_DRIVE_USE_TRASH=true.
10320 Verbosity is slightly different, the environment variable equivalent of
10322 --verbose or -v is RCLONE_VERBOSE=1, or for -vv, RCLONE_VERBOSE=2.
10323 The same parser is used for the options and the environment variables
10325 so they take exactly the same form.
10326 The options set by environment variables can be seen with the -vv flag,
10328 e.g. rclone version -vv.
10329 Config file
10331 You can set defaults for values in the config file on an individual re‐
10332 mote basis. The names of the config items are documented in the page
10333 for each backend.
10334 To find the name of the environment variable, you need to set, take
10336 RCLONE_CONFIG_ + name of remote + _ + name of config file option and
10337 make it all uppercase.
10338 For example, to configure an S3 remote named mys3: without a config
10340 file (using unix ways of setting environment variables):
10341 $ export RCLONE_CONFIG_MYS3_TYPE=s3
10343 $ export RCLONE_CONFIG_MYS3_ACCESS_KEY_ID=XXX
10344 $ export RCLONE_CONFIG_MYS3_SECRET_ACCESS_KEY=XXX
10345 $ rclone lsd mys3:
10346 -1 2016-09-21 12:54:21 -1 my-bucket
10347 $ rclone listremotes | grep mys3
10348 mys3:
10349 Note that if you want to create a remote using environment variables
10351 you must create the ..._TYPE variable as above.
10352 Note that the name of a remote created using environment variable is
10354 case insensitive, in contrast to regular remotes stored in config file
10355 as documented above. You must write the name in uppercase in the envi‐
10356 ronment variable, but as seen from example above it will be listed and
10357 can be accessed in lowercase, while you can also refer to the same re‐
10358 mote in uppercase:
10359 $ rclone lsd mys3:
10361 -1 2016-09-21 12:54:21 -1 my-bucket
10362 $ rclone lsd MYS3:
10363 -1 2016-09-21 12:54:21 -1 my-bucket
10364 Note that you can only set the options of the immediate backend, so
10366 RCLONE_CONFIG_MYS3CRYPT_ACCESS_KEY_ID has no effect, if myS3Crypt is a
10367 crypt remote based on an S3 remote. However RCLONE_S3_ACCESS_KEY_ID
10368 will set the access key of all remotes using S3, including myS3Crypt.
10369 Note also that now rclone has connection strings, it is probably easier
10371 to use those instead which makes the above example
10372 rclone lsd :s3,access_key_id=XXX,secret_access_key=XXX:
10374 Precedence
10376 The various different methods of backend configuration are read in this
10377 order and the first one with a value is used.
10378 • Parameters in connection strings, e.g. myRemote,skip_links:
10380 • Flag values as supplied on the command line, e.g. --skip-links
10382 • Remote specific environment vars, e.g. RCLONE_CONFIG_MYRE‐
10384 MOTE_SKIP_LINKS (see above).
10385 • Backend-specific environment vars, e.g. RCLONE_LOCAL_SKIP_LINKS.
10387 • Backend generic environment vars, e.g. RCLONE_SKIP_LINKS.
10389 • Config file, e.g. skip_links = true.
10391 • Default values, e.g. false - these can't be changed.
10393 So if both --skip-links is supplied on the command line and an environ‐
10395 ment variable RCLONE_LOCAL_SKIP_LINKS is set, the command line flag
10396 will take preference.
10397 The backend configurations set by environment variables can be seen
10399 with the -vv flag, e.g. rclone about myRemote: -vv.
10400 For non backend configuration the order is as follows:
10402 • Flag values as supplied on the command line, e.g. --stats 5s.
10404 • Environment vars, e.g. RCLONE_STATS=5s.
10406 • Default values, e.g. 1m - these can't be changed.
10408 Other environment variables
10410 • RCLONE_CONFIG_PASS set to contain your config file password (see Con‐
10411 figuration Encryption section)
10412 • HTTP_PROXY, HTTPS_PROXY and NO_PROXY (or the lowercase versions
10414 thereof).
10415 • HTTPS_PROXY takes precedence over HTTP_PROXY for https requests.
10417 • The environment values may be either a complete URL or a
10419 "host[:port]" for, in which case the "http" scheme is assumed.
10420 • USER and LOGNAME values are used as fallbacks for current username.
10422 The primary method for looking up username is OS-specific: Windows
10423 API on Windows, real user ID in /etc/passwd on Unix systems. In the
10424 documentation the current username is simply referred to as $USER.
10425 • RCLONE_CONFIG_DIR - rclone sets this variable for use in config files
10427 and sub processes to point to the directory holding the config file.
10428 The options set by environment variables can be seen with the -vv and
10430 --log-level=DEBUG flags, e.g. rclone version -vv.
10431
10433 Some of the configurations (those involving oauth2) require an Internet
10434 connected web browser.
10435 If you are trying to set rclone up on a remote or headless box with no
10437 browser available on it (e.g. a NAS or a server in a datacenter) then
10438 you will need to use an alternative means of configuration. There are
10439 two ways of doing it, described below.
10440 Configuring using rclone authorize
10442 On the headless box run rclone config but answer N to the Use auto con‐
10443 fig? question.
10444 ...
10446 Remote config
10447 Use auto config?
10448 * Say Y if not sure
10449 * Say N if you are working on a remote or headless machine
10450 y) Yes (default)
10451 n) No
10452 y/n> n
10453 For this to work, you will need rclone available on a machine that has
10454 a web browser available.
10455 For more help and alternate methods see: https://rclone.org/remote_setup/
10457 Execute the following on the machine with the web browser (same rclone
10459 version recommended):
10460 rclone authorize "amazon cloud drive"
10462 Then paste the result below:
10464 result>
10465 Then on your main desktop machine
10467 rclone authorize "amazon cloud drive"
10469 If your browser doesn't open automatically go to the following link: http://127.0.0.1:53682/auth
10470 Log in and authorize rclone for access
10471 Waiting for code...
10472 Got code
10473 Paste the following into your remote machine --->
10474 SECRET_TOKEN
10475 <---End paste
10476 Then back to the headless box, paste in the code
10478 result> SECRET_TOKEN
10480 --------------------
10481 [acd12]
10482 client_id =
10483 client_secret =
10484 token = SECRET_TOKEN
10485 --------------------
10486 y) Yes this is OK
10487 e) Edit this remote
10488 d) Delete this remote
10489 y/e/d>
10490 Configuring by copying the config file
10492 Rclone stores all of its config in a single configuration file. This
10493 can easily be copied to configure a remote rclone.
10494 So first configure rclone on your desktop machine with
10496 rclone config
10498 to set up the config file.
10500 Find the config file by running rclone config file, for example
10502 $ rclone config file
10504 Configuration file is stored at:
10505 /home/user/.rclone.conf
10506 Now transfer it to the remote box (scp, cut paste, ftp, sftp, etc.) and
10508 place it in the correct place (use rclone config file on the remote box
10509 to find out where).
10510 Configuring using SSH Tunnel
10512 Linux and MacOS users can utilize SSH Tunnel to redirect the headless
10513 box port 53682 to local machine by using the following command:
10514 ssh -L localhost:53682:localhost:53682 username@remote_server
10516 Then on the headless box run rclone config and answer Y to the Use auto
10518 config? question.
10519 ...
10521 Remote config
10522 Use auto config?
10523 * Say Y if not sure
10524 * Say N if you are working on a remote or headless machine
10525 y) Yes (default)
10526 n) No
10527 y/n> y
10528 Then copy and paste the auth url
10530 http://127.0.0.1:53682/auth?state=xxxxxxxxxxxx to the browser on your
10531 local machine, complete the auth and it is done.
10532
10534 Filter flags determine which files rclone sync, move, ls, lsl, md5sum,
10535 sha1sum, size, delete, check and similar commands apply to.
10536 They are specified in terms of path/file name patterns; path/file
10538 lists; file age and size, or presence of a file in a directory. Bucket
10539 based remotes without the concept of directory apply filters to object
10540 key, age and size in an analogous way.
10541 Rclone purge does not obey filters.
10543 To test filters without risk of damage to data, apply them to rclone
10545 ls, or with the --dry-run and -vv flags.
10546 Rclone filter patterns can only be used in filter command line options,
10548 not in the specification of a remote.
10549 E.g. rclone copy "remote:dir*.jpg" /path/to/dir does not have a filter
10551 effect. rclone copy remote:dir /path/to/dir --include "*.jpg" does.
10552 Important Avoid mixing any two of --include..., --exclude... or --fil‐
10554 ter... flags in an rclone command. The results may not be what you ex‐
10555 pect. Instead use a --filter... flag.
10556 Patterns for matching path/file names
10558 Pattern syntax
10559 Here is a formal definition of the pattern syntax, examples are below.
10560 Rclone matching rules follow a glob style:
10562 * matches any sequence of non-separator (/) characters
10564 ** matches any sequence of characters including / separators
10565 ? matches any single non-separator (/) character
10566 [ [ ! ] { character-range } ]
10567 character class (must be non-empty)
10568 { pattern-list }
10569 pattern alternatives
10570 {{ regexp }}
10571 regular expression to match
10572 c matches character c (c != *, **, ?, \, [, {, })
10573 \c matches reserved character c (c = *, **, ?, \, [, {, }) or character class
10574 character-range:
10576 c matches character c (c != \, -, ])
10578 \c matches reserved character c (c = \, -, ])
10579 lo - hi matches character c for lo <= c <= hi
10580 pattern-list:
10582 pattern { , pattern }
10584 comma-separated (without spaces) patterns
10585 character classes (see Go regular expression reference
10587 (https://golang.org/pkg/regexp/syntax/)) include:
10588 Named character classes (e.g. [\d], [^\d], [\D], [^\D])
10590 Perl character classes (e.g. \s, \S, \w, \W)
10591 ASCII character classes (e.g. [[:alnum:]], [[:alpha:]], [[:punct:]], [[:xdigit:]])
10592 regexp for advanced users to insert a regular expression - see below
10594 for more info:
10595 Any re2 regular expression not containing `}}`
10597 If the filter pattern starts with a / then it only matches at the top
10599 level of the directory tree, relative to the root of the remote (not
10600 necessarily the root of the drive). If it does not start with / then
10601 it is matched starting at the end of the path/file name but it only
10602 matches a complete path element - it must match from a / separator or
10603 the beginning of the path/file.
10604 file.jpg - matches "file.jpg"
10606 - matches "directory/file.jpg"
10607 - doesn't match "afile.jpg"
10608 - doesn't match "directory/afile.jpg"
10609 /file.jpg - matches "file.jpg" in the root directory of the remote
10610 - doesn't match "afile.jpg"
10611 - doesn't match "directory/file.jpg"
10612 The top level of the remote may not be the top level of the drive.
10614 E.g. for a Microsoft Windows local directory structure
10616 F:
10618 ├── bkp
10619 ├── data
10620 │ ├── excl
10621 │ │ ├── 123.jpg
10622 │ │ └── 456.jpg
10623 │ ├── incl
10624 │ │ └── document.pdf
10625 To copy the contents of folder data into folder bkp excluding the con‐
10627 tents of subfolder exclthe following command treats F:\data and F:\bkp
10628 as top level for filtering.
10629 rclone copy F:\data\ F:\bkp\ --exclude=/excl/**
10631 Important Use / in path/file name patterns and not \ even if running on
10633 Microsoft Windows.
10634 Simple patterns are case sensitive unless the --ignore-case flag is
10636 used.
10637 Without --ignore-case (default)
10639 potato - matches "potato"
10641 - doesn't match "POTATO"
10642 With --ignore-case
10644 potato - matches "potato"
10646 - matches "POTATO"
10647 Using regular expressions in filter patterns
10649 The syntax of filter patterns is glob style matching (like bash uses)
10650 to make things easy for users. However this does not provide absolute
10651 control over the matching, so for advanced users rclone also provides a
10652 regular expression syntax.
10653 The regular expressions used are as defined in the Go regular expres‐
10655 sion reference (https://golang.org/pkg/regexp/syntax/). Regular ex‐
10656 pressions should be enclosed in {{ }}. They will match only the last
10657 path segment if the glob doesn't start with / or the whole path name if
10658 it does. Note that rclone does not attempt to parse the supplied regu‐
10659 lar expression, meaning that using any regular expression filter will
10660 prevent rclone from using directory filter rules, as it will instead
10661 check every path against the supplied regular expression(s).
10662 Here is how the {{regexp}} is transformed into an full regular expres‐
10664 sion to match the entire path:
10665 {{regexp}} becomes (^|/)(regexp)$
10667 /{{regexp}} becomes ^(regexp)$
10668 Regexp syntax can be mixed with glob syntax, for example
10670 *.{{jpe?g}} to match file.jpg, file.jpeg but not file.png
10672 You can also use regexp flags - to set case insensitive, for example
10674 *.{{(?i)jpg}} to match file.jpg, file.JPG but not file.png
10676 Be careful with wildcards in regular expressions - you don't want them
10678 to match path separators normally. To match any file name starting
10679 with start and ending with end write
10680 {{start[^/]*end\.jpg}}
10682 Not
10684 {{start.*end\.jpg}}
10686 Which will match a directory called start with a file called end.jpg in
10688 it as the .* will match / characters.
10689 Note that you can use -vv --dump filters to show the filter patterns in
10691 regexp format - rclone implements the glob patters by transforming them
10692 into regular expressions.
10693 Filter pattern examples
10695 Description Pattern Matches Does not match
10696 ─────────────────────────────────────────────────────────────────────
10697 Wildcard *.jpg /file.jpg /file.png
10698 /dir/file.jpg /dir/file.png
10699 Rooted /*.jpg /file.jpg /file.png
10700 /file2.jpg /dir/file.jpg
10701 Alternates *.{jpg,png} /file.jpg /file.gif
10702 /dir/file.png /dir/file.gif
10703 Path Wildcard dir/** /dir/anyfile file.png
10704 /subdir/dir/sub‐ /subdir/file.png
10705 subdir/anyfile
10706 Any Char *.t?t /file.txt /file.qxt
10707 /dir/file.tzt /dir/file.png
10708 Range *.[a-z] /file.a /file.0
10709 /dir/file.b /dir/file.1
10710 Escape *.\?\?\? /file.??? /file.abc
10711 /dir/file.??? /dir/file.def
10712 Class *.\d\d\d /file.012 /file.abc
10713 /dir/file.345 /dir/file.def
10714 Regexp *.{{jpe?g}} /file.jpeg /file.png
10715 /dir/file.jpg /dir/file.jpeeg
10716 Rooted Regexp /{{.*\.jpe?g}} /file.jpeg /file.png
10717 /file.jpg /dir/file.jpg
10718 How filter rules are applied to files
10720 Rclone path/file name filters are made up of one or more of the follow‐
10721 ing flags:
10722 • --include
10724 • --include-from
10726 • --exclude
10728 • --exclude-from
10730 • --filter
10732 • --filter-from
10734 There can be more than one instance of individual flags.
10736 Rclone internally uses a combined list of all the include and exclude
10738 rules. The order in which rules are processed can influence the result
10739 of the filter.
10740 All flags of the same type are processed together in the order above,
10742 regardless of what order the different types of flags are included on
10743 the command line.
10744 Multiple instances of the same flag are processed from left to right
10746 according to their position in the command line.
10747 To mix up the order of processing includes and excludes use --filter...
10749 flags.
10750 Within --include-from, --exclude-from and --filter-from flags rules are
10752 processed from top to bottom of the referenced file.
10753 If there is an --include or --include-from flag specified, rclone im‐
10755 plies a - ** rule which it adds to the bottom of the internal rule
10756 list. Specifying a + rule with a --filter... flag does not imply that
10757 rule.
10758 Each path/file name passed through rclone is matched against the com‐
10760 bined filter list. At first match to a rule the path/file name is in‐
10761 cluded or excluded and no further filter rules are processed for that
10762 path/file.
10763 If rclone does not find a match, after testing against all rules (in‐
10765 cluding the implied rule if appropriate), the path/file name is includ‐
10766 ed.
10767 Any path/file included at that stage is processed by the rclone com‐
10769 mand.
10770 --files-from and --files-from-raw flags over-ride and cannot be com‐
10772 bined with other filter options.
10773 To see the internal combined rule list, in regular expression form, for
10775 a command add the --dump filters flag. Running an rclone command with
10776 --dump filters and -vv flags lists the internal filter elements and
10777 shows how they are applied to each source path/file. There is not cur‐
10778 rently a means provided to pass regular expression filter options into
10779 rclone directly though character class filter rules contain character
10780 classes. Go regular expression reference (https://golang.org/pkg/reg‐
10781 exp/syntax/)
10782 How filter rules are applied to directories
10784 Rclone commands are applied to path/file names not directories. The
10785 entire contents of a directory can be matched to a filter by the pat‐
10786 tern directory/* or recursively by directory/**.
10787 Directory filter rules are defined with a closing / separator.
10789 E.g. /directory/subdirectory/ is an rclone directory filter rule.
10791 Rclone commands can use directory filter rules to determine whether
10793 they recurse into subdirectories. This potentially optimises access to
10794 a remote by avoiding listing unnecessary directories. Whether optimi‐
10795 sation is desirable depends on the specific filter rules and source re‐
10796 mote content.
10797 If any regular expression filters are in use, then no directory recur‐
10799 sion optimisation is possible, as rclone must check every path against
10800 the supplied regular expression(s).
10801 Directory recursion optimisation occurs if either:
10803 • A source remote does not support the rclone ListR primitive. local,
10805 sftp, Microsoft OneDrive and WebDAV do not support ListR. Google
10806 Drive and most bucket type storage do. Full list
10807 (https://rclone.org/overview/#optional-features)
10808 • On other remotes (those that support ListR), if the rclone command is
10810 not naturally recursive, and provided it is not run with the --fast-
10811 list flag. ls, lsf -R and size are naturally recursive but sync,
10812 copy and move are not.
10813 • Whenever the --disable ListR flag is applied to an rclone command.
10815 Rclone commands imply directory filter rules from path/file filter
10817 rules. To view the directory filter rules rclone has implied for a
10818 command specify the --dump filters flag.
10819 E.g. for an include rule
10821 /a/*.jpg
10823 Rclone implies the directory include rule
10825 /a/
10827 Directory filter rules specified in an rclone command can limit the
10829 scope of an rclone command but path/file filters still have to be spec‐
10830 ified.
10831 E.g. rclone ls remote: --include /directory/ will not match any files.
10833 Because it is an --include option the --exclude ** rule is implied, and
10834 the /directory/ pattern serves only to optimise access to the remote by
10835 ignoring everything outside of that directory.
10836 E.g. rclone ls remote: --filter-from filter-list.txt with a file fil‐
10838 ter-list.txt:
10839 - /dir1/
10841 - /dir2/
10842 + *.pdf
10843 - **
10844 All files in directories dir1 or dir2 or their subdirectories are com‐
10846 pletely excluded from the listing. Only files of suffix pdf in the
10847 root of remote: or its subdirectories are listed. The - ** rule pre‐
10848 vents listing of any path/files not previously matched by the rules
10849 above.
10850 Option exclude-if-present creates a directory exclude rule based on the
10852 presence of a file in a directory and takes precedence over other
10853 rclone directory filter rules.
10854 When using pattern list syntax, if a pattern item contains either / or
10856 **, then rclone will not able to imply a directory filter rule from
10857 this pattern list.
10858 E.g. for an include rule
10860 {dir1/**,dir2/**}
10862 Rclone will match files below directories dir1 or dir2 only, but will
10864 not be able to use this filter to exclude a directory dir3 from being
10865 traversed.
10866 Directory recursion optimisation may affect performance, but normally
10868 not the result. One exception to this is sync operations with option
10869 --create-empty-src-dirs, where any traversed empty directories will be
10870 created. With the pattern list example {dir1/**,dir2/**} above, this
10871 would create an empty directory dir3 on destination (when it exists on
10872 source). Changing the filter to {dir1,dir2}/**, or splitting it into
10873 two include rules --include dir1/** --include dir2/**, will match the
10874 same files while also filtering directories, with the result that an
10875 empty directory dir3 will no longer be created.
10876 --exclude - Exclude files matching pattern
10878 Excludes path/file names from an rclone command based on a single ex‐
10879 clude rule.
10880 This flag can be repeated. See above for the order filter flags are
10882 processed in.
10883 --exclude should not be used with --include, --include-from, --filter
10885 or --filter-from flags.
10886 --exclude has no effect when combined with --files-from or --files-
10888 from-raw flags.
10889 E.g. rclone ls remote: --exclude *.bak excludes all .bak files from
10891 listing.
10892 E.g. rclone size remote: "--exclude /dir/**" returns the total size of
10894 all files on remote: excluding those in root directory dir and sub di‐
10895 rectories.
10896 E.g. on Microsoft Windows rclone ls remote: --exclude
10898 "*\[{JP,KR,HK}\]*" lists the files in remote: with [JP] or [KR] or [HK]
10899 in their name. Quotes prevent the shell from interpreting the \ char‐
10900 acters.\ characters escape the [ and ] so an rclone filter treats them
10901 literally rather than as a character-range. The { and } define an
10902 rclone pattern list. For other operating systems single quotes are re‐
10903 quired ie rclone ls remote: --exclude '*\[{JP,KR,HK}\]*'
10904 --exclude-from - Read exclude patterns from file
10906 Excludes path/file names from an rclone command based on rules in a
10907 named file. The file contains a list of remarks and pattern rules.
10908 For an example exclude-file.txt:
10910 # a sample exclude rule file
10912 *.bak
10913 file2.jpg
10914 rclone ls remote: --exclude-from exclude-file.txt lists the files on
10916 remote: except those named file2.jpg or with a suffix .bak. That is
10917 equivalent to rclone ls remote: --exclude file2.jpg --exclude "*.bak".
10918 This flag can be repeated. See above for the order filter flags are
10920 processed in.
10921 The --exclude-from flag is useful where multiple exclude filter rules
10923 are applied to an rclone command.
10924 --exclude-from should not be used with --include, --include-from,
10926 --filter or --filter-from flags.
10927 --exclude-from has no effect when combined with --files-from or
10929 --files-from-raw flags.
10930 --exclude-from followed by - reads filter rules from standard input.
10932 --include - Include files matching pattern
10934 Adds a single include rule based on path/file names to an rclone com‐
10935 mand.
10936 This flag can be repeated. See above for the order filter flags are
10938 processed in.
10939 --include has no effect when combined with --files-from or --files-
10941 from-raw flags.
10942 --include implies --exclude ** at the end of an rclone internal filter
10944 list. Therefore if you mix --include and --include-from flags with
10945 --exclude, --exclude-from, --filter or --filter-from, you must use in‐
10946 clude rules for all the files you want in the include statement. For
10947 more flexibility use the --filter-from flag.
10948 E.g. rclone ls remote: --include "*.{png,jpg}" lists the files on re‐
10950 mote: with suffix .png and .jpg. All other files are excluded.
10951 E.g. multiple rclone copy commands can be combined with --include and
10953 a pattern-list.
10954 rclone copy /vol1/A remote:A
10956 rclone copy /vol1/B remote:B
10957 is equivalent to:
10959 rclone copy /vol1 remote: --include "{A,B}/**"
10961 E.g. rclone ls remote:/wheat --include "??[^[:punct:]]*" lists the
10963 files remote: directory wheat (and subdirectories) whose third charac‐
10964 ter is not punctuation. This example uses an ASCII character class
10965 (https://golang.org/pkg/regexp/syntax/).
10966 --include-from - Read include patterns from file
10968 Adds path/file names to an rclone command based on rules in a named
10969 file. The file contains a list of remarks and pattern rules.
10970 For an example include-file.txt:
10972 # a sample include rule file
10974 *.jpg
10975 file2.avi
10976 rclone ls remote: --include-from include-file.txt lists the files on
10978 remote: with name file2.avi or suffix .jpg. That is equivalent to
10979 rclone ls remote: --include file2.avi --include "*.jpg".
10980 This flag can be repeated. See above for the order filter flags are
10982 processed in.
10983 The --include-from flag is useful where multiple include filter rules
10985 are applied to an rclone command.
10986 --include-from implies --exclude ** at the end of an rclone internal
10988 filter list. Therefore if you mix --include and --include-from flags
10989 with --exclude, --exclude-from, --filter or --filter-from, you must use
10990 include rules for all the files you want in the include statement. For
10991 more flexibility use the --filter-from flag.
10992 --exclude-from has no effect when combined with --files-from or
10994 --files-from-raw flags.
10995 --exclude-from followed by - reads filter rules from standard input.
10997 --filter - Add a file-filtering rule
10999 Specifies path/file names to an rclone command, based on a single in‐
11000 clude or exclude rule, in + or - format.
11001 This flag can be repeated. See above for the order filter flags are
11003 processed in.
11004 --filter + differs from --include. In the case of --include rclone im‐
11006 plies an --exclude * rule which it adds to the bottom of the internal
11007 rule list. --filter...+ does not imply that rule.
11008 --filter has no effect when combined with --files-from or --files-from-
11010 raw flags.
11011 --filter should not be used with --include, --include-from, --exclude
11013 or --exclude-from flags.
11014 E.g. rclone ls remote: --filter "- *.bak" excludes all .bak files from
11016 a list of remote:.
11017 --filter-from - Read filtering patterns from a file
11019 Adds path/file names to an rclone command based on rules in a named
11020 file. The file contains a list of remarks and pattern rules. Include
11021 rules start with + and exclude rules with -. ! clears existing rules.
11022 Rules are processed in the order they are defined.
11023 This flag can be repeated. See above for the order filter flags are
11025 processed in.
11026 Arrange the order of filter rules with the most restrictive first and
11028 work down.
11029 E.g. for filter-file.txt:
11031 # a sample filter rule file
11033 - secret*.jpg
11034 + *.jpg
11035 + *.png
11036 + file2.avi
11037 - /dir/Trash/**
11038 + /dir/**
11039 # exclude everything else
11040 - *
11041 rclone ls remote: --filter-from filter-file.txt lists the path/files on
11043 remote: including all jpg and png files, excluding any matching se‐
11044 cret*.jpg and including file2.avi. It also includes everything in the
11045 directory dir at the root of remote, except remote:dir/Trash which it
11046 excludes. Everything else is excluded.
11047 E.g. for an alternative filter-file.txt:
11049 - secret*.jpg
11051 + *.jpg
11052 + *.png
11053 + file2.avi
11054 - *
11055 Files file1.jpg, file3.png and file2.avi are listed whilst secret17.jpg
11057 and files without the suffix .jpgor.png` are excluded.
11058 E.g. for an alternative filter-file.txt:
11060 + *.jpg
11062 + *.gif
11063 !
11064 + 42.doc
11065 - *
11066 Only file 42.doc is listed. Prior rules are cleared by the !.
11068 --files-from - Read list of source-file names
11070 Adds path/files to an rclone command from a list in a named file.
11071 Rclone processes the path/file names in the order of the list, and no
11072 others.
11073 Other filter flags (--include, --include-from, --exclude, --exclude-
11075 from, --filter and --filter-from) are ignored when --files-from is
11076 used.
11077 --files-from expects a list of files as its input. Leading or trailing
11079 whitespace is stripped from the input lines. Lines starting with # or
11080 ; are ignored.
11081 Rclone commands with a --files-from flag traverse the remote, treating
11083 the names in --files-from as a set of filters.
11084 If the --no-traverse and --files-from flags are used together an rclone
11086 command does not traverse the remote. Instead it addresses each
11087 path/file named in the file individually. For each path/file name,
11088 that requires typically 1 API call. This can be efficient for a short
11089 --files-from list and a remote containing many files.
11090 Rclone commands do not error if any names in the --files-from file are
11092 missing from the source remote.
11093 The --files-from flag can be repeated in a single rclone command to
11095 read path/file names from more than one file. The files are read from
11096 left to right along the command line.
11097 Paths within the --files-from file are interpreted as starting with the
11099 root specified in the rclone command. Leading / separators are ig‐
11100 nored. See --files-from-raw if you need the input to be processed in a
11101 raw manner.
11102 E.g. for a file files-from.txt:
11104 # comment
11106 file1.jpg
11107 subdir/file2.jpg
11108 rclone copy --files-from files-from.txt /home/me/pics remote:pics
11110 copies the following, if they exist, and only those files.
11111 /home/me/pics/file1.jpg → remote:pics/file1.jpg
11113 /home/me/pics/subdir/file2.jpg → remote:pics/subdir/file2.jpg
11114 E.g. to copy the following files referenced by their absolute paths:
11116 /home/user1/42
11118 /home/user1/dir/ford
11119 /home/user2/prefect
11120 First find a common subdirectory - in this case /home and put the re‐
11122 maining files in files-from.txt with or without leading /, e.g.
11123 user1/42
11125 user1/dir/ford
11126 user2/prefect
11127 Then copy these to a remote:
11129 rclone copy --files-from files-from.txt /home remote:backup
11131 The three files are transferred as follows:
11133 /home/user1/42 → remote:backup/user1/important
11135 /home/user1/dir/ford → remote:backup/user1/dir/file
11136 /home/user2/prefect → remote:backup/user2/stuff
11137 Alternatively if / is chosen as root files-from.txt will be:
11139 /home/user1/42
11141 /home/user1/dir/ford
11142 /home/user2/prefect
11143 The copy command will be:
11145 rclone copy --files-from files-from.txt / remote:backup
11147 Then there will be an extra home directory on the remote:
11149 /home/user1/42 → remote:backup/home/user1/42
11151 /home/user1/dir/ford → remote:backup/home/user1/dir/ford
11152 /home/user2/prefect → remote:backup/home/user2/prefect
11153 --files-from-raw - Read list of source-file names without any processing
11155 This flag is the same as --files-from except that input is read in a
11156 raw manner. Lines with leading / trailing whitespace, and lines start‐
11157 ing with ; or # are read without any processing. rclone lsf
11158 (https://rclone.org/commands/rclone_lsf/) has a compatible format that
11159 can be used to export file lists from remotes for input to --files-
11160 from-raw.
11161 --ignore-case - make searches case insensitive
11163 By default, rclone filter patterns are case sensitive. The --ignore-
11164 case flag makes all of the filters patterns on the command line case
11165 insensitive.
11166 E.g. --include "zaphod.txt" does not match a file Zaphod.txt. With
11168 --ignore-case a match is made.
11169 Quoting shell metacharacters
11171 Rclone commands with filter patterns containing shell metacharacters
11172 may not as work as expected in your shell and may require quoting.
11173 E.g. linux, OSX (* metacharacter)
11175 • --include \*.jpg
11177 • --include '*.jpg'
11179 • --include='*.jpg'
11181 Microsoft Windows expansion is done by the command, not shell, so --in‐
11183 clude *.jpg does not require quoting.
11184 If the rclone error Command .... needs .... arguments maximum: you pro‐
11186 vided .... non flag arguments: is encountered, the cause is commonly
11187 spaces within the name of a remote or flag value. The fix then is to
11188 quote values containing spaces.
11189 Other filters
11191 --min-size - Don't transfer any file smaller than this
11192 Controls the minimum size file within the scope of an rclone command.
11193 Default units are KiB but abbreviations K, M, G, T or P are valid.
11194 E.g. rclone ls remote: --min-size 50k lists files on remote: of 50 KiB
11196 size or larger.
11197 See the size option docs (https://rclone.org/docs/#size-option) for
11199 more info.
11200 --max-size - Don't transfer any file larger than this
11202 Controls the maximum size file within the scope of an rclone command.
11203 Default units are KiB but abbreviations K, M, G, T or P are valid.
11204 E.g. rclone ls remote: --max-size 1G lists files on remote: of 1 GiB
11206 size or smaller.
11207 See the size option docs (https://rclone.org/docs/#size-option) for
11209 more info.
11210 --max-age - Don't transfer any file older than this
11212 Controls the maximum age of files within the scope of an rclone com‐
11213 mand.
11214 --max-age applies only to files and not to directories.
11216 E.g. rclone ls remote: --max-age 2d lists files on remote: of 2 days
11218 old or less.
11219 See the time option docs (https://rclone.org/docs/#time-option) for
11221 valid formats.
11222 --min-age - Don't transfer any file younger than this
11224 Controls the minimum age of files within the scope of an rclone com‐
11225 mand. (see --max-age for valid formats)
11226 --min-age applies only to files and not to directories.
11228 E.g. rclone ls remote: --min-age 2d lists files on remote: of 2 days
11230 old or more.
11231 See the time option docs (https://rclone.org/docs/#time-option) for
11233 valid formats.
11234 Other flags
11236 --delete-excluded - Delete files on dest excluded from sync
11237 Important this flag is dangerous to your data - use with --dry-run and
11238 -v first.
11239 In conjunction with rclone sync, --delete-excluded deletes any files on
11241 the destination which are excluded from the command.
11242 E.g. the scope of rclone sync -i A: B: can be restricted:
11244 rclone --min-size 50k --delete-excluded sync A: B:
11246 All files on B: which are less than 50 KiB are deleted because they are
11248 excluded from the rclone sync command.
11249 --dump filters - dump the filters to the output
11251 Dumps the defined filters to standard output in regular expression for‐
11252 mat.
11253 Useful for debugging.
11255 Exclude directory based on a file
11257 The --exclude-if-present flag controls whether a directory is within
11258 the scope of an rclone command based on the presence of a named file
11259 within it. The flag can be repeated to check for multiple file names,
11260 presence of any of them will exclude the directory.
11261 This flag has a priority over other filter flags.
11263 E.g. for the following directory structure:
11265 dir1/file1
11267 dir1/dir2/file2
11268 dir1/dir2/dir3/file3
11269 dir1/dir2/dir3/.ignore
11270 The command rclone ls --exclude-if-present .ignore dir1 does not list
11272 dir3, file3 or .ignore.
11273 Common pitfalls
11275 The most frequent filter support issues on the rclone forum
11276 (https://forum.rclone.org/) are:
11277 • Not using paths relative to the root of the remote
11279 • Not using / to match from the root of a remote
11281 • Not using ** to match the contents of a directory
11283
11285 Rclone can serve a web based GUI (graphical user interface). This is
11286 somewhat experimental at the moment so things may be subject to change.
11287 Run this command in a terminal and rclone will download and then dis‐
11289 play the GUI in a web browser.
11290 rclone rcd --rc-web-gui
11292 This will produce logs like this and rclone needs to continue to run to
11294 serve the GUI:
11295 2019/08/25 11:40:14 NOTICE: A new release for gui is present at https://github.com/rclone/rclone-webui-react/releases/download/v0.0.6/currentbuild.zip
11297 2019/08/25 11:40:14 NOTICE: Downloading webgui binary. Please wait. [Size: 3813937, Path : /home/USER/.cache/rclone/webgui/v0.0.6.zip]
11298 2019/08/25 11:40:16 NOTICE: Unzipping
11299 2019/08/25 11:40:16 NOTICE: Serving remote control on http://127.0.0.1:5572/
11300 This assumes you are running rclone locally on your machine. It is
11302 possible to separate the rclone and the GUI - see below for details.
11303 If you wish to check for updates then you can add --rc-web-gui-update
11305 to the command line.
11306 If you find your GUI broken, you may force it to update by add --rc-
11308 web-gui-force-update.
11309 By default, rclone will open your browser. Add --rc-web-gui-no-open-
11311 browser to disable this feature.
11312 Using the GUI
11314 Once the GUI opens, you will be looking at the dashboard which has an
11315 overall overview.
11316 On the left hand side you will see a series of view buttons you can
11318 click on:
11319 • Dashboard - main overview
11321 • Configs - examine and create new configurations
11323 • Explorer - view, download and upload files to the cloud storage sys‐
11325 tems
11326 • Backend - view or alter the backend config
11328 • Log out
11330 (More docs and walkthrough video to come!)
11332 How it works
11334 When you run the rclone rcd --rc-web-gui this is what happens
11335 • Rclone starts but only runs the remote control API ("rc").
11337 • The API is bound to localhost with an auto-generated username and
11339 password.
11340 • If the API bundle is missing then rclone will download it.
11342 • rclone will start serving the files from the API bundle over the same
11344 port as the API
11345 • rclone will open the browser with a login_token so it can log
11347 straight in.
11348 Advanced use
11350 The rclone rcd may use any of the flags documented on the rc page
11351 (https://rclone.org/rc/#supported-parameters).
11352 The flag --rc-web-gui is shorthand for
11354 • Download the web GUI if necessary
11356 • Check we are using some authentication
11358 • --rc-user gui
11360 • --rc-pass <random password>
11362 • --rc-serve
11364 These flags can be overridden as desired.
11366 See also the rclone rcd documentation (https://rclone.org/com‐
11368 mands/rclone_rcd/).
11369 Example: Running a public GUI
11371 For example the GUI could be served on a public port over SSL using an
11372 htpasswd file using the following flags:
11373 • --rc-web-gui
11375 • --rc-addr :443
11377 • --rc-htpasswd /path/to/htpasswd
11379 • --rc-cert /path/to/ssl.crt
11381 • --rc-key /path/to/ssl.key
11383 Example: Running a GUI behind a proxy
11385 If you want to run the GUI behind a proxy at /rclone you could use
11386 these flags:
11387 • --rc-web-gui
11389 • --rc-baseurl rclone
11391 • --rc-htpasswd /path/to/htpasswd
11393 Or instead of htpasswd if you just want a single user and password:
11395 • --rc-user me
11397 • --rc-pass mypassword
11399 Project
11401 The GUI is being developed in the: rclone/rclone-webui-react repository
11402 (https://github.com/rclone/rclone-webui-react).
11403 Bug reports and contributions are very welcome :-)
11405 If you have questions then please ask them on the rclone forum
11407 (https://forum.rclone.org/).
11408
11410 If rclone is run with the --rc flag then it starts an HTTP server which
11411 can be used to remote control rclone using its API.
11412 You can either use the rc command to access the API or use HTTP direct‐
11414 ly.
11415 If you just want to run a remote control then see the rcd
11417 (https://rclone.org/commands/rclone_rcd/) command.
11418 Supported parameters
11420 --rc
11421 Flag to start the http server listen on remote requests
11422 --rc-addr=IP
11424 IPaddress:Port or :Port to bind server to. (default "localhost:5572")
11425 --rc-cert=KEY
11427 SSL PEM key (concatenation of certificate and CA certificate)
11428 --rc-client-ca=PATH
11430 Client certificate authority to verify clients with
11431 --rc-htpasswd=PATH
11433 htpasswd file - if not provided no authentication is done
11434 --rc-key=PATH
11436 SSL PEM Private key
11437 --rc-max-header-bytes=VALUE
11439 Maximum size of request header (default 4096)
11440 --rc-min-tls-version=VALUE
11442 The minimum TLS version that is acceptable. Valid values are "tls1.0",
11443 "tls1.1", "tls1.2" and "tls1.3" (default "tls1.0").
11444 --rc-user=VALUE
11446 User name for authentication.
11447 --rc-pass=VALUE
11449 Password for authentication.
11450 --rc-realm=VALUE
11452 Realm for authentication (default "rclone")
11453 --rc-server-read-timeout=DURATION
11455 Timeout for server reading data (default 1h0m0s)
11456 --rc-server-write-timeout=DURATION
11458 Timeout for server writing data (default 1h0m0s)
11459 --rc-serve
11461 Enable the serving of remote objects via the HTTP interface. This
11462 means objects will be accessible at http://127.0.0.1:5572/ by default,
11463 so you can browse to http://127.0.0.1:5572/ or http://127.0.0.1:5572/*
11464 to see a listing of the remotes. Objects may be requested from remotes
11465 using this syntax http://127.0.0.1:5572/[remote:path]/path/to/object
11466 Default Off.
11468 --rc-files /path/to/directory
11470 Path to local files to serve on the HTTP server.
11471 If this is set then rclone will serve the files in that directory. It
11473 will also open the root in the web browser if specified. This is for
11474 implementing browser based GUIs for rclone functions.
11475 If --rc-user or --rc-pass is set then the URL that is opened will have
11477 the authorization in the URL in the http://user:pass@localhost/ style.
11478 Default Off.
11480 --rc-enable-metrics
11482 Enable OpenMetrics/Prometheus compatible endpoint at /metrics.
11483 Default Off.
11485 --rc-web-gui
11487 Set this flag to serve the default web gui on the same port as rclone.
11488 Default Off.
11490 --rc-allow-origin
11492 Set the allowed Access-Control-Allow-Origin for rc requests.
11493 Can be used with --rc-web-gui if the rclone is running on different IP
11495 than the web-gui.
11496 Default is IP address on which rc is running.
11498 --rc-web-fetch-url
11500 Set the URL to fetch the rclone-web-gui files from.
11501 Default https://api.github.com/repos/rclone/rclone-webui-react/releas‐
11503 es/latest.
11504 --rc-web-gui-update
11506 Set this flag to check and update rclone-webui-react from the rc-web-
11507 fetch-url.
11508 Default Off.
11510 --rc-web-gui-force-update
11512 Set this flag to force update rclone-webui-react from the rc-web-fetch-
11513 url.
11514 Default Off.
11516 --rc-web-gui-no-open-browser
11518 Set this flag to disable opening browser automatically when using web-
11519 gui.
11520 Default Off.
11522 --rc-job-expire-duration=DURATION
11524 Expire finished async jobs older than DURATION (default 60s).
11525 --rc-job-expire-interval=DURATION
11527 Interval duration to check for expired async jobs (default 10s).
11528 --rc-no-auth
11530 By default rclone will require authorisation to have been set up on the
11531 rc interface in order to use any methods which access any rclone re‐
11532 motes. Eg operations/list is denied as it involved creating a remote
11533 as is sync/copy.
11534 If this is set then no authorisation will be required on the server to
11536 use these methods. The alternative is to use --rc-user and --rc-pass
11537 and use these credentials in the request.
11538 Default Off.
11540 --rc-baseurl
11542 Prefix for URLs.
11543 Default is root
11545 --rc-template
11547 User-specified template.
11548 Accessing the remote control via the rclone rc command
11550 Rclone itself implements the remote control protocol in its rclone rc
11551 command.
11552 You can use it like this
11554 $ rclone rc rc/noop param1=one param2=two
11556 {
11557 "param1": "one",
11558 "param2": "two"
11559 }
11560 Run rclone rc on its own to see the help for the installed remote con‐
11562 trol commands.
11563 JSON input
11565 rclone rc also supports a --json flag which can be used to send more
11566 complicated input parameters.
11567 $ rclone rc --json '{ "p1": [1,"2",null,4], "p2": { "a":1, "b":2 } }' rc/noop
11569 {
11570 "p1": [
11571 1,
11572 "2",
11573 null,
11574 4
11575 ],
11576 "p2": {
11577 "a": 1,
11578 "b": 2
11579 }
11580 }
11581 If the parameter being passed is an object then it can be passed as a
11583 JSON string rather than using the --json flag which simplifies the com‐
11584 mand line.
11585 rclone rc operations/list fs=/tmp remote=test opt='{"showHash": true}'
11587 Rather than
11589 rclone rc operations/list --json '{"fs": "/tmp", "remote": "test", "opt": {"showHash": true}}'
11591 Special parameters
11593 The rc interface supports some special parameters which apply to all
11594 commands. These start with _ to show they are different.
11595 Running asynchronous jobs with _async = true
11597 Each rc call is classified as a job and it is assigned its own id. By
11598 default jobs are executed immediately as they are created or syn‐
11599 chronously.
11600 If _async has a true value when supplied to an rc call then it will re‐
11602 turn immediately with a job id and the task will be run in the back‐
11603 ground. The job/status call can be used to get information of the
11604 background job. The job can be queried for up to 1 minute after it has
11605 finished.
11606 It is recommended that potentially long running jobs, e.g. sync/sync,
11608 sync/copy, sync/move, operations/purge are run with the _async flag to
11609 avoid any potential problems with the HTTP request and response timing
11610 out.
11611 Starting a job with the _async flag:
11613 $ rclone rc --json '{ "p1": [1,"2",null,4], "p2": { "a":1, "b":2 }, "_async": true }' rc/noop
11615 {
11616 "jobid": 2
11617 }
11618 Query the status to see if the job has finished. For more information
11620 on the meaning of these return parameters see the job/status call.
11621 $ rclone rc --json '{ "jobid":2 }' job/status
11623 {
11624 "duration": 0.000124163,
11625 "endTime": "2018-10-27T11:38:07.911245881+01:00",
11626 "error": "",
11627 "finished": true,
11628 "id": 2,
11629 "output": {
11630 "_async": true,
11631 "p1": [
11632 1,
11633 "2",
11634 null,
11635 4
11636 ],
11637 "p2": {
11638 "a": 1,
11639 "b": 2
11640 }
11641 },
11642 "startTime": "2018-10-27T11:38:07.911121728+01:00",
11643 "success": true
11644 }
11645 job/list can be used to show the running or recently completed jobs
11647 $ rclone rc job/list
11649 {
11650 "jobids": [
11651 2
11652 ]
11653 }
11654 Setting config flags with _config
11656 If you wish to set config (the equivalent of the global flags) for the
11657 duration of an rc call only then pass in the _config parameter.
11658 This should be in the same format as the config key returned by op‐
11660 tions/get.
11661 For example, if you wished to run a sync with the --checksum parameter,
11663 you would pass this parameter in your JSON blob.
11664 "_config":{"CheckSum": true}
11666 If using rclone rc this could be passed as
11668 rclone rc operations/sync ... _config='{"CheckSum": true}'
11670 Any config parameters you don't set will inherit the global defaults
11672 which were set with command line flags or environment variables.
11673 Note that it is possible to set some values as strings or integers -
11675 see data types for more info. Here is an example setting the equiva‐
11676 lent of --buffer-size in string or integer format.
11677 "_config":{"BufferSize": "42M"}
11679 "_config":{"BufferSize": 44040192}
11680 If you wish to check the _config assignment has worked properly then
11682 calling options/local will show what the value got set to.
11683 Setting filter flags with _filter
11685 If you wish to set filters for the duration of an rc call only then
11686 pass in the _filter parameter.
11687 This should be in the same format as the filter key returned by op‐
11689 tions/get.
11690 For example, if you wished to run a sync with these flags
11692 --max-size 1M --max-age 42s --include "a" --include "b"
11694 you would pass this parameter in your JSON blob.
11696 "_filter":{"MaxSize":"1M", "IncludeRule":["a","b"], "MaxAge":"42s"}
11698 If using rclone rc this could be passed as
11700 rclone rc ... _filter='{"MaxSize":"1M", "IncludeRule":["a","b"], "MaxAge":"42s"}'
11702 Any filter parameters you don't set will inherit the global defaults
11704 which were set with command line flags or environment variables.
11705 Note that it is possible to set some values as strings or integers -
11707 see data types for more info. Here is an example setting the equiva‐
11708 lent of --buffer-size in string or integer format.
11709 "_filter":{"MinSize": "42M"}
11711 "_filter":{"MinSize": 44040192}
11712 If you wish to check the _filter assignment has worked properly then
11714 calling options/local will show what the value got set to.
11715 Assigning operations to groups with _group = value
11717 Each rc call has its own stats group for tracking its metrics. By de‐
11718 fault grouping is done by the composite group name from prefix job/ and
11719 id of the job like so job/1.
11720 If _group has a value then stats for that request will be grouped under
11722 that value. This allows caller to group stats under their own name.
11723 Stats for specific group can be accessed by passing group to
11725 core/stats:
11726 $ rclone rc --json '{ "group": "job/1" }' core/stats
11728 {
11729 "speed": 12345
11730 ...
11731 }
11732 Data types
11734 When the API returns types, these will mostly be straight forward inte‐
11735 ger, string or boolean types.
11736 However some of the types returned by the options/get call and taken by
11738 the options/set calls as well as the vfsOpt, mountOpt and the _config
11739 parameters.
11740 • Duration - these are returned as an integer duration in nanoseconds.
11742 They may be set as an integer, or they may be set with time string,
11743 eg "5s". See the options section (https://rclone.org/docs/#options)
11744 for more info.
11745 • Size - these are returned as an integer number of bytes. They may be
11747 set as an integer or they may be set with a size suffix string, eg
11748 "10M". See the options section (https://rclone.org/docs/#options)
11749 for more info.
11750 • Enumerated type (such as CutoffMode, DumpFlags, LogLevel, Vfs‐
11752 CacheMode - these will be returned as an integer and may be set as an
11753 integer but more conveniently they can be set as a string, eg "HARD"
11754 for CutoffMode or DEBUG for LogLevel.
11755 • BandwidthSpec - this will be set and returned as a string, eg "1M".
11757 Specifying remotes to work on
11759 Remotes are specified with the fs=, srcFs=, dstFs= parameters depending
11760 on the command being used.
11761 The parameters can be a string as per the rest of rclone, eg s3:buck‐
11763 et/path or :sftp:/my/dir. They can also be specified as JSON blobs.
11764 If specifying a JSON blob it should be a object mapping strings to
11766 strings. These values will be used to configure the remote. There are
11767 3 special values which may be set:
11768 • type - set to type to specify a remote called :type:
11770 • _name - set to name to specify a remote called name:
11772 • _root - sets the root of the remote - may be empty
11774 One of _name or type should normally be set. If the local backend is
11776 desired then type should be set to local. If _root isn't specified
11777 then it defaults to the root of the remote.
11778 For example this JSON is equivalent to remote:/tmp
11780 {
11782 "_name": "remote",
11783 "_path": "/tmp"
11784 }
11785 And this is equivalent to :sftp,host='example.com':/tmp
11787 {
11789 "type": "sftp",
11790 "host": "example.com",
11791 "_path": "/tmp"
11792 }
11793 And this is equivalent to /tmp/dir
11795 {
11797 type = "local",
11798 _ path = "/tmp/dir"
11799 }
11800 Supported commands
11802 backend/command: Runs a backend command.
11803 This takes the following parameters:
11804 • command - a string with the command name
11806 • fs - a remote name string e.g. "drive:"
11808 • arg - a list of arguments for the backend command
11810 • opt - a map of string to string of options
11812 Returns:
11814 • result - result from the backend command
11816 Example:
11818 rclone rc backend/command command=noop fs=. -o echo=yes -o blue -a path1 -a path2
11820 Returns
11822 {
11824 "result": {
11825 "arg": [
11826 "path1",
11827 "path2"
11828 ],
11829 "name": "noop",
11830 "opt": {
11831 "blue": "",
11832 "echo": "yes"
11833 }
11834 }
11835 }
11836 Note that this is the direct equivalent of using this "backend" com‐
11838 mand:
11839 rclone backend noop . -o echo=yes -o blue path1 path2
11841 Note that arguments must be preceded by the "-a" flag
11843 See the backend (https://rclone.org/commands/rclone_backend/) command
11845 for more information.
11846 Authentication is required for this call.
11848 cache/expire: Purge a remote from cache
11850 Purge a remote from the cache backend. Supports either a directory or
11851 a file. Params: - remote = path to remote (required) - withData =
11852 true/false to delete cached data (chunks) as well (optional)
11853 Eg
11855 rclone rc cache/expire remote=path/to/sub/folder/
11857 rclone rc cache/expire remote=/ withData=true
11858 cache/fetch: Fetch file chunks
11860 Ensure the specified file chunks are cached on disk.
11861 The chunks= parameter specifies the file chunks to check. It takes a
11863 comma separated list of array slice indices. The slice indices are
11864 similar to Python slices: start[:end]
11865 start is the 0 based chunk number from the beginning of the file to
11867 fetch inclusive. end is 0 based chunk number from the beginning of the
11868 file to fetch exclusive. Both values can be negative, in which case
11869 they count from the back of the file. The value "-5:" represents the
11870 last 5 chunks of a file.
11871 Some valid examples are: ":5,-5:" -> the first and last five chunks
11873 "0,-2" -> the first and the second last chunk "0:10" -> the first ten
11874 chunks
11875 Any parameter with a key that starts with "file" can be used to specify
11877 files to fetch, e.g.
11878 rclone rc cache/fetch chunks=0 file=hello file2=home/goodbye
11880 File names will automatically be encrypted when the a crypt remote is
11882 used on top of the cache.
11883 cache/stats: Get cache stats
11885 Show statistics for the cache remote.
11886 config/create: create the config for a remote.
11888 This takes the following parameters:
11889 • name - name of remote
11891 • parameters - a map of { "key": "value" } pairs
11893 • type - type of the new remote
11895 • opt - a dictionary of options to control the configuration
11897 • obscure - declare passwords are plain and need obscuring
11899 • noObscure - declare passwords are already obscured and don't need
11901 obscuring
11902 • nonInteractive - don't interact with a user, return questions
11904 • continue - continue the config process with an answer
11906 • all - ask all the config questions not just the post config ones
11908 • state - state to restart with - used with continue
11910 • result - result to restart with - used with continue
11912 See the config create (https://rclone.org/commands/rclone_config_cre‐
11914 ate/) command for more information on the above.
11915 Authentication is required for this call.
11917 config/delete: Delete a remote in the config file.
11919 Parameters:
11920 • name - name of remote to delete
11922 See the config delete (https://rclone.org/commands/rclone_con‐
11924 fig_delete/) command for more information on the above.
11925 Authentication is required for this call.
11927 config/dump: Dumps the config file.
11929 Returns a JSON object: - key: value
11930 Where keys are remote names and values are the config parameters.
11932 See the config dump (https://rclone.org/commands/rclone_config_dump/)
11934 command for more information on the above.
11935 Authentication is required for this call.
11937 config/get: Get a remote in the config file.
11939 Parameters:
11940 • name - name of remote to get
11942 See the config dump (https://rclone.org/commands/rclone_config_dump/)
11944 command for more information on the above.
11945 Authentication is required for this call.
11947 config/listremotes: Lists the remotes in the config file.
11949 Returns - remotes - array of remote names
11950 See the listremotes (https://rclone.org/commands/rclone_listremotes/)
11952 command for more information on the above.
11953 Authentication is required for this call.
11955 config/password: password the config for a remote.
11957 This takes the following parameters:
11958 • name - name of remote
11960 • parameters - a map of { "key": "value" } pairs
11962 See the config password (https://rclone.org/commands/rclone_con‐
11964 fig_password/) command for more information on the above.
11965 Authentication is required for this call.
11967 config/providers: Shows how providers are configured in the config file.
11969 Returns a JSON object: - providers - array of objects
11970 See the config providers (https://rclone.org/commands/rclone_con‐
11972 fig_providers/) command for more information on the above.
11973 Authentication is required for this call.
11975 config/update: update the config for a remote.
11977 This takes the following parameters:
11978 • name - name of remote
11980 • parameters - a map of { "key": "value" } pairs
11982 • opt - a dictionary of options to control the configuration
11984 • obscure - declare passwords are plain and need obscuring
11986 • noObscure - declare passwords are already obscured and don't need
11988 obscuring
11989 • nonInteractive - don't interact with a user, return questions
11991 • continue - continue the config process with an answer
11993 • all - ask all the config questions not just the post config ones
11995 • state - state to restart with - used with continue
11997 • result - result to restart with - used with continue
11999 See the config update (https://rclone.org/commands/rclone_config_up‐
12001 date/) command for more information on the above.
12002 Authentication is required for this call.
12004 core/bwlimit: Set the bandwidth limit.
12006 This sets the bandwidth limit to the string passed in. This should be
12007 a single bandwidth limit entry or a pair of upload:download bandwidth.
12008 Eg
12010 rclone rc core/bwlimit rate=off
12012 {
12013 "bytesPerSecond": -1,
12014 "bytesPerSecondTx": -1,
12015 "bytesPerSecondRx": -1,
12016 "rate": "off"
12017 }
12018 rclone rc core/bwlimit rate=1M
12019 {
12020 "bytesPerSecond": 1048576,
12021 "bytesPerSecondTx": 1048576,
12022 "bytesPerSecondRx": 1048576,
12023 "rate": "1M"
12024 }
12025 rclone rc core/bwlimit rate=1M:100k
12026 {
12027 "bytesPerSecond": 1048576,
12028 "bytesPerSecondTx": 1048576,
12029 "bytesPerSecondRx": 131072,
12030 "rate": "1M"
12031 }
12032 If the rate parameter is not supplied then the bandwidth is queried
12034 rclone rc core/bwlimit
12036 {
12037 "bytesPerSecond": 1048576,
12038 "bytesPerSecondTx": 1048576,
12039 "bytesPerSecondRx": 1048576,
12040 "rate": "1M"
12041 }
12042 The format of the parameter is exactly the same as passed to --bwlimit
12044 except only one bandwidth may be specified.
12045 In either case "rate" is returned as a human-readable string, and
12047 "bytesPerSecond" is returned as a number.
12048 core/command: Run a rclone terminal command over rc.
12050 This takes the following parameters:
12051 • command - a string with the command name.
12053 • arg - a list of arguments for the backend command.
12055 • opt - a map of string to string of options.
12057 • returnType - one of ("COMBINED_OUTPUT", "STREAM", "STREAM_ONLY_STD‐
12059 OUT", "STREAM_ONLY_STDERR").
12060 • Defaults to "COMBINED_OUTPUT" if not set.
12062 • The STREAM returnTypes will write the output to the body of the
12064 HTTP message.
12065 • The COMBINED_OUTPUT will write the output to the "result" parame‐
12067 ter.
12068 Returns:
12070 • result - result from the backend command.
12072 • Only set when using returnType "COMBINED_OUTPUT".
12074 • error - set if rclone exits with an error code.
12076 • returnType - one of ("COMBINED_OUTPUT", "STREAM", "STREAM_ONLY_STD‐
12078 OUT", "STREAM_ONLY_STDERR").
12079 Example:
12081 rclone rc core/command command=ls -a mydrive:/ -o max-depth=1
12083 rclone rc core/command -a ls -a mydrive:/ -o max-depth=1
12084 Returns:
12086 {
12088 "error": false,
12089 "result": "<Raw command line output>"
12090 }
12091 OR
12093 {
12094 "error": true,
12095 "result": "<Raw command line output>"
12096 }
12097 Authentication is required for this call.
12099 core/gc: Runs a garbage collection.
12101 This tells the go runtime to do a garbage collection run. It isn't
12102 necessary to call this normally, but it can be useful for debugging
12103 memory problems.
12104 core/group-list: Returns list of stats.
12106 This returns list of stats groups currently in memory.
12107 Returns the following values:
12109 {
12111 "groups": an array of group names:
12112 [
12113 "group1",
12114 "group2",
12115 ...
12116 ]
12117 }
12118 core/memstats: Returns the memory statistics
12120 This returns the memory statistics of the running program. What the
12121 values mean are explained in the go docs: https://golang.org/pkg/run‐
12122 time/#MemStats
12123 The most interesting values for most people are:
12125 • HeapAlloc - this is the amount of memory rclone is actually using
12127 • HeapSys - this is the amount of memory rclone has obtained from the
12129 OS
12130 • Sys - this is the total amount of memory requested from the OS
12132 • It is virtual memory so may include unused memory
12134 core/obscure: Obscures a string passed in.
12136 Pass a clear string and rclone will obscure it for the config file: -
12137 clear - string
12138 Returns: - obscured - string
12140 core/pid: Return PID of current process
12142 This returns PID of current process. Useful for stopping rclone
12143 process.
12144 core/quit: Terminates the app.
12146 (Optional) Pass an exit code to be used for terminating the app: - ex‐
12147 itCode - int
12148 core/stats: Returns stats about current transfers.
12150 This returns all available stats:
12151 rclone rc core/stats
12153 If group is not provided then summed up stats for all groups will be
12155 returned.
12156 Parameters
12158 • group - name of the stats group (string)
12160 Returns the following values:
12162 {
12164 "bytes": total transferred bytes since the start of the group,
12165 "checks": number of files checked,
12166 "deletes" : number of files deleted,
12167 "elapsedTime": time in floating point seconds since rclone was started,
12168 "errors": number of errors,
12169 "eta": estimated time in seconds until the group completes,
12170 "fatalError": boolean whether there has been at least one fatal error,
12171 "lastError": last error string,
12172 "renames" : number of files renamed,
12173 "retryError": boolean showing whether there has been at least one non-NoRetryError,
12174 "speed": average speed in bytes per second since start of the group,
12175 "totalBytes": total number of bytes in the group,
12176 "totalChecks": total number of checks in the group,
12177 "totalTransfers": total number of transfers in the group,
12178 "transferTime" : total time spent on running jobs,
12179 "transfers": number of transferred files,
12180 "transferring": an array of currently active file transfers:
12181 [
12182 {
12183 "bytes": total transferred bytes for this file,
12184 "eta": estimated time in seconds until file transfer completion
12185 "name": name of the file,
12186 "percentage": progress of the file transfer in percent,
12187 "speed": average speed over the whole transfer in bytes per second,
12188 "speedAvg": current speed in bytes per second as an exponentially weighted moving average,
12189 "size": size of the file in bytes
12190 }
12191 ],
12192 "checking": an array of names of currently active file checks
12193 []
12194 }
12195 Values for "transferring", "checking" and "lastError" are only assigned
12197 if data is available. The value for "eta" is null if an eta cannot be
12198 determined.
12199 core/stats-delete: Delete stats group.
12201 This deletes entire stats group.
12202 Parameters
12204 • group - name of the stats group (string)
12206 core/stats-reset: Reset stats.
12208 This clears counters, errors and finished transfers for all stats or
12209 specific stats group if group is provided.
12210 Parameters
12212 • group - name of the stats group (string)
12214 core/transferred: Returns stats about completed transfers.
12216 This returns stats about completed transfers:
12217 rclone rc core/transferred
12219 If group is not provided then completed transfers for all groups will
12221 be returned.
12222 Note only the last 100 completed transfers are returned.
12224 Parameters
12226 • group - name of the stats group (string)
12228 Returns the following values:
12230 {
12232 "transferred": an array of completed transfers (including failed ones):
12233 [
12234 {
12235 "name": name of the file,
12236 "size": size of the file in bytes,
12237 "bytes": total transferred bytes for this file,
12238 "checked": if the transfer is only checked (skipped, deleted),
12239 "timestamp": integer representing millisecond unix epoch,
12240 "error": string description of the error (empty if successful),
12241 "jobid": id of the job that this transfer belongs to
12242 }
12243 ]
12244 }
12245 core/version: Shows the current version of rclone and the go runtime.
12247 This shows the current version of go and the go runtime:
12248 • version - rclone version, e.g. "v1.53.0"
12250 • decomposed - version number as [major, minor, patch]
12252 • isGit - boolean - true if this was compiled from the git version
12254 • isBeta - boolean - true if this is a beta version
12256 • os - OS in use as according to Go
12258 • arch - cpu architecture in use according to Go
12260 • goVersion - version of Go runtime in use
12262 • linking - type of rclone executable (static or dynamic)
12264 • goTags - space separated build tags or "none"
12266 debug/set-block-profile-rate: Set runtime.SetBlockProfileRate for blocking
12268 profiling.
12269 SetBlockProfileRate controls the fraction of goroutine blocking events
12270 that are reported in the blocking profile. The profiler aims to sample
12271 an average of one blocking event per rate nanoseconds spent blocked.
12272 To include every blocking event in the profile, pass rate = 1. To turn
12274 off profiling entirely, pass rate <= 0.
12275 After calling this you can use this to see the blocking profile:
12277 go tool pprof http://localhost:5572/debug/pprof/block
12279 Parameters:
12281 • rate - int
12283 debug/set-mutex-profile-fraction: Set runtime.SetMutexProfileFraction for
12285 mutex profiling.
12286 SetMutexProfileFraction controls the fraction of mutex contention
12287 events that are reported in the mutex profile. On average 1/rate
12288 events are reported. The previous rate is returned.
12289 To turn off profiling entirely, pass rate 0. To just read the current
12291 rate, pass rate < 0. (For n>1 the details of sampling may change.)
12292 Once this is set you can look use this to profile the mutex contention:
12294 go tool pprof http://localhost:5572/debug/pprof/mutex
12296 Parameters:
12298 • rate - int
12300 Results:
12302 • previousRate - int
12304 fscache/clear: Clear the Fs cache.
12306 This clears the fs cache. This is where remotes created from backends
12307 are cached for a short while to make repeated rc calls more efficient.
12308 If you change the parameters of a backend then you may want to call
12310 this to clear an existing remote out of the cache before re-creating
12311 it.
12312 Authentication is required for this call.
12314 fscache/entries: Returns the number of entries in the fs cache.
12316 This returns the number of entries in the fs cache.
12317 Returns - entries - number of items in the cache
12319 Authentication is required for this call.
12321 job/list: Lists the IDs of the running jobs
12323 Parameters: None.
12324 Results:
12326 • jobids - array of integer job ids.
12328 job/status: Reads the status of the job ID
12330 Parameters:
12331 • jobid - id of the job (integer).
12333 Results:
12335 • finished - boolean
12337 • duration - time in seconds that the job ran for
12339 • endTime - time the job finished (e.g.
12341 "2018-10-26T18:50:20.528746884+01:00")
12342 • error - error from the job or empty string for no error
12344 • finished - boolean whether the job has finished or not
12346 • id - as passed in above
12348 • startTime - time the job started (e.g.
12350 "2018-10-26T18:50:20.528336039+01:00")
12351 • success - boolean - true for success false otherwise
12353 • output - output of the job as would have been returned if called syn‐
12355 chronously
12356 • progress - output of the progress related to the underlying job
12358 job/stop: Stop the running job
12360 Parameters:
12361 • jobid - id of the job (integer).
12363 job/stopgroup: Stop all running jobs in a group
12365 Parameters:
12366 • group - name of the group (string).
12368 mount/listmounts: Show current mount points
12370 This shows currently mounted points, which can be used for performing
12371 an unmount.
12372 This takes no parameters and returns
12374 • mountPoints: list of current mount points
12376 Eg
12378 rclone rc mount/listmounts
12380 Authentication is required for this call.
12382 mount/mount: Create a new mount point
12384 rclone allows Linux, FreeBSD, macOS and Windows to mount any of
12385 Rclone's cloud storage systems as a file system with FUSE.
12386 If no mountType is provided, the priority is given as follows: 1.
12388 mount 2.cmount 3.mount2
12389 This takes the following parameters:
12391 • fs - a remote path to be mounted (required)
12393 • mountPoint: valid path on the local machine (required)
12395 • mountType: one of the values (mount, cmount, mount2) specifies the
12397 mount implementation to use
12398 • mountOpt: a JSON object with Mount options in.
12400 • vfsOpt: a JSON object with VFS options in.
12402 Example:
12404 rclone rc mount/mount fs=mydrive: mountPoint=/home/<user>/mountPoint
12406 rclone rc mount/mount fs=mydrive: mountPoint=/home/<user>/mountPoint mountType=mount
12407 rclone rc mount/mount fs=TestDrive: mountPoint=/mnt/tmp vfsOpt='{"CacheMode": 2}' mountOpt='{"AllowOther": true}'
12408 The vfsOpt are as described in options/get and can be seen in the the
12410 "vfs" section when running and the mountOpt can be seen in the "mount"
12411 section:
12412 rclone rc options/get
12414 Authentication is required for this call.
12416 mount/types: Show all possible mount types
12418 This shows all possible mount types and returns them as a list.
12419 This takes no parameters and returns
12421 • mountTypes: list of mount types
12423 The mount types are strings like "mount", "mount2", "cmount" and can be
12425 passed to mount/mount as the mountType parameter.
12426 Eg
12428 rclone rc mount/types
12430 Authentication is required for this call.
12432 mount/unmount: Unmount selected active mount
12434 rclone allows Linux, FreeBSD, macOS and Windows to mount any of
12435 Rclone's cloud storage systems as a file system with FUSE.
12436 This takes the following parameters:
12438 • mountPoint: valid path on the local machine where the mount was cre‐
12440 ated (required)
12441 Example:
12443 rclone rc mount/unmount mountPoint=/home/<user>/mountPoint
12445 Authentication is required for this call.
12447 mount/unmountall: Unmount all active mounts
12449 rclone allows Linux, FreeBSD, macOS and Windows to mount any of
12450 Rclone's cloud storage systems as a file system with FUSE.
12451 This takes no parameters and returns error if unmount does not succeed.
12453 Eg
12455 rclone rc mount/unmountall
12457 Authentication is required for this call.
12459 operations/about: Return the space used on the remote
12461 This takes the following parameters:
12462 • fs - a remote name string e.g. "drive:"
12464 The result is as returned from rclone about --json
12466 See the about (https://rclone.org/commands/rclone_about/) command for
12468 more information on the above.
12469 Authentication is required for this call.
12471 operations/cleanup: Remove trashed files in the remote or path
12473 This takes the following parameters:
12474 • fs - a remote name string e.g. "drive:"
12476 See the cleanup (https://rclone.org/commands/rclone_cleanup/) command
12478 for more information on the above.
12479 Authentication is required for this call.
12481 operations/copyfile: Copy a file from source remote to destination remote
12483 This takes the following parameters:
12484 • srcFs - a remote name string e.g. "drive:" for the source
12486 • srcRemote - a path within that remote e.g. "file.txt" for the source
12488 • dstFs - a remote name string e.g. "drive2:" for the destination
12490 • dstRemote - a path within that remote e.g. "file2.txt" for the des‐
12492 tination
12493 Authentication is required for this call.
12495 operations/copyurl: Copy the URL to the object
12497 This takes the following parameters:
12498 • fs - a remote name string e.g. "drive:"
12500 • remote - a path within that remote e.g. "dir"
12502 • url - string, URL to read from
12504 • autoFilename - boolean, set to true to retrieve destination file name
12506 from url
12507 See the copyurl (https://rclone.org/commands/rclone_copyurl/) command
12509 for more information on the above.
12510 Authentication is required for this call.
12512 operations/delete: Remove files in the path
12514 This takes the following parameters:
12515 • fs - a remote name string e.g. "drive:"
12517 See the delete (https://rclone.org/commands/rclone_delete/) command for
12519 more information on the above.
12520 Authentication is required for this call.
12522 operations/deletefile: Remove the single file pointed to
12524 This takes the following parameters:
12525 • fs - a remote name string e.g. "drive:"
12527 • remote - a path within that remote e.g. "dir"
12529 See the deletefile (https://rclone.org/commands/rclone_deletefile/)
12531 command for more information on the above.
12532 Authentication is required for this call.
12534 operations/fsinfo: Return information about the remote
12536 This takes the following parameters:
12537 • fs - a remote name string e.g. "drive:"
12539 This returns info about the remote passed in;
12541 {
12543 // optional features and whether they are available or not
12544 "Features": {
12545 "About": true,
12546 "BucketBased": false,
12547 "BucketBasedRootOK": false,
12548 "CanHaveEmptyDirectories": true,
12549 "CaseInsensitive": false,
12550 "ChangeNotify": false,
12551 "CleanUp": false,
12552 "Command": true,
12553 "Copy": false,
12554 "DirCacheFlush": false,
12555 "DirMove": true,
12556 "Disconnect": false,
12557 "DuplicateFiles": false,
12558 "GetTier": false,
12559 "IsLocal": true,
12560 "ListR": false,
12561 "MergeDirs": false,
12562 "MetadataInfo": true,
12563 "Move": true,
12564 "OpenWriterAt": true,
12565 "PublicLink": false,
12566 "Purge": true,
12567 "PutStream": true,
12568 "PutUnchecked": false,
12569 "ReadMetadata": true,
12570 "ReadMimeType": false,
12571 "ServerSideAcrossConfigs": false,
12572 "SetTier": false,
12573 "SetWrapper": false,
12574 "Shutdown": false,
12575 "SlowHash": true,
12576 "SlowModTime": false,
12577 "UnWrap": false,
12578 "UserInfo": false,
12579 "UserMetadata": true,
12580 "WrapFs": false,
12581 "WriteMetadata": true,
12582 "WriteMimeType": false
12583 },
12584 // Names of hashes available
12585 "Hashes": [
12586 "md5",
12587 "sha1",
12588 "whirlpool",
12589 "crc32",
12590 "sha256",
12591 "dropbox",
12592 "mailru",
12593 "quickxor"
12594 ],
12595 "Name": "local", // Name as created
12596 "Precision": 1, // Precision of timestamps in ns
12597 "Root": "/", // Path as created
12598 "String": "Local file system at /", // how the remote will appear in logs
12599 // Information about the system metadata for this backend
12600 "MetadataInfo": {
12601 "System": {
12602 "atime": {
12603 "Help": "Time of last access",
12604 "Type": "RFC 3339",
12605 "Example": "2006-01-02T15:04:05.999999999Z07:00"
12606 },
12607 "btime": {
12608 "Help": "Time of file birth (creation)",
12609 "Type": "RFC 3339",
12610 "Example": "2006-01-02T15:04:05.999999999Z07:00"
12611 },
12612 "gid": {
12613 "Help": "Group ID of owner",
12614 "Type": "decimal number",
12615 "Example": "500"
12616 },
12617 "mode": {
12618 "Help": "File type and mode",
12619 "Type": "octal, unix style",
12620 "Example": "0100664"
12621 },
12622 "mtime": {
12623 "Help": "Time of last modification",
12624 "Type": "RFC 3339",
12625 "Example": "2006-01-02T15:04:05.999999999Z07:00"
12626 },
12627 "rdev": {
12628 "Help": "Device ID (if special file)",
12629 "Type": "hexadecimal",
12630 "Example": "1abc"
12631 },
12632 "uid": {
12633 "Help": "User ID of owner",
12634 "Type": "decimal number",
12635 "Example": "500"
12636 }
12637 },
12638 "Help": "Textual help string\n"
12639 }
12640 }
12641 This command does not have a command line equivalent so use this in‐
12643 stead:
12644 rclone rc --loopback operations/fsinfo fs=remote:
12646 operations/list: List the given remote and path in JSON format
12648 This takes the following parameters:
12649 • fs - a remote name string e.g. "drive:"
12651 • remote - a path within that remote e.g. "dir"
12653 • opt - a dictionary of options to control the listing (optional)
12655 • recurse - If set recurse directories
12657 • noModTime - If set return modification time
12659 • showEncrypted - If set show decrypted names
12661 • showOrigIDs - If set show the IDs for each item if known
12663 • showHash - If set return a dictionary of hashes
12665 • noMimeType - If set don't show mime types
12667 • dirsOnly - If set only show directories
12669 • filesOnly - If set only show files
12671 • metadata - If set return metadata of objects also
12673 • hashTypes - array of strings of hash types to show if showHash set
12675 Returns:
12677 • list
12679 • This is an array of objects as described in the lsjson command
12681 See the lsjson (https://rclone.org/commands/rclone_lsjson/) command for
12683 more information on the above and examples.
12684 Authentication is required for this call.
12686 operations/mkdir: Make a destination directory or container
12688 This takes the following parameters:
12689 • fs - a remote name string e.g. "drive:"
12691 • remote - a path within that remote e.g. "dir"
12693 See the mkdir (https://rclone.org/commands/rclone_mkdir/) command for
12695 more information on the above.
12696 Authentication is required for this call.
12698 operations/movefile: Move a file from source remote to destination remote
12700 This takes the following parameters:
12701 • srcFs - a remote name string e.g. "drive:" for the source
12703 • srcRemote - a path within that remote e.g. "file.txt" for the source
12705 • dstFs - a remote name string e.g. "drive2:" for the destination
12707 • dstRemote - a path within that remote e.g. "file2.txt" for the des‐
12709 tination
12710 Authentication is required for this call.
12712 operations/publiclink: Create or retrieve a public link to the given file
12714 or folder.
12715 This takes the following parameters:
12716 • fs - a remote name string e.g. "drive:"
12718 • remote - a path within that remote e.g. "dir"
12720 • unlink - boolean - if set removes the link rather than adding it (op‐
12722 tional)
12723 • expire - string - the expiry time of the link e.g. "1d" (optional)
12725 Returns:
12727 • url - URL of the resource
12729 See the link (https://rclone.org/commands/rclone_link/) command for
12731 more information on the above.
12732 Authentication is required for this call.
12734 operations/purge: Remove a directory or container and all of its contents
12736 This takes the following parameters:
12737 • fs - a remote name string e.g. "drive:"
12739 • remote - a path within that remote e.g. "dir"
12741 See the purge (https://rclone.org/commands/rclone_purge/) command for
12743 more information on the above.
12744 Authentication is required for this call.
12746 operations/rmdir: Remove an empty directory or container
12748 This takes the following parameters:
12749 • fs - a remote name string e.g. "drive:"
12751 • remote - a path within that remote e.g. "dir"
12753 See the rmdir (https://rclone.org/commands/rclone_rmdir/) command for
12755 more information on the above.
12756 Authentication is required for this call.
12758 operations/rmdirs: Remove all the empty directories in the path
12760 This takes the following parameters:
12761 • fs - a remote name string e.g. "drive:"
12763 • remote - a path within that remote e.g. "dir"
12765 • leaveRoot - boolean, set to true not to delete the root
12767 See the rmdirs (https://rclone.org/commands/rclone_rmdirs/) command for
12769 more information on the above.
12770 Authentication is required for this call.
12772 operations/size: Count the number of bytes and files in remote
12774 This takes the following parameters:
12775 • fs - a remote name string e.g. "drive:path/to/dir"
12777 Returns:
12779 • count - number of files
12781 • bytes - number of bytes in those files
12783 See the size (https://rclone.org/commands/rclone_size/) command for
12785 more information on the above.
12786 Authentication is required for this call.
12788 operations/stat: Give information about the supplied file or directory
12790 This takes the following parameters
12791 • fs - a remote name string eg "drive:"
12793 • remote - a path within that remote eg "dir"
12795 • opt - a dictionary of options to control the listing (optional)
12797 • see operations/list for the options
12799 The result is
12801 • item - an object as described in the lsjson command. Will be null if
12803 not found.
12804 Note that if you are only interested in files then it is much more ef‐
12806 ficient to set the filesOnly flag in the options.
12807 See the lsjson (https://rclone.org/commands/rclone_lsjson/) command for
12809 more information on the above and examples.
12810 Authentication is required for this call.
12812 operations/uploadfile: Upload file using multiform/form-data
12814 This takes the following parameters:
12815 • fs - a remote name string e.g. "drive:"
12817 • remote - a path within that remote e.g. "dir"
12819 • each part in body represents a file to be uploaded
12821 See the uploadfile (https://rclone.org/commands/rclone_uploadfile/)
12823 command for more information on the above.
12824 Authentication is required for this call.
12826 options/blocks: List all the option blocks
12828 Returns: - options - a list of the options block names
12829 options/get: Get all the global options
12831 Returns an object where keys are option block names and values are an
12832 object with the current option values in.
12833 Note that these are the global options which are unaffected by use of
12835 the _config and _filter parameters. If you wish to read the parameters
12836 set in _config then use options/config and for _filter use options/fil‐
12837 ter.
12838 This shows the internal names of the option within rclone which should
12840 map to the external options very easily with a few exceptions.
12841 options/local: Get the currently active config for this call
12843 Returns an object with the keys "config" and "filter". The "config"
12844 key contains the local config and the "filter" key contains the local
12845 filters.
12846 Note that these are the local options specific to this rc call. If
12848 _config was not supplied then they will be the global options. Like‐
12849 wise with "_filter".
12850 This call is mostly useful for seeing if _config and _filter passing is
12852 working.
12853 This shows the internal names of the option within rclone which should
12855 map to the external options very easily with a few exceptions.
12856 options/set: Set an option
12858 Parameters:
12859 • option block name containing an object with
12861 • key: value
12863 Repeated as often as required.
12865 Only supply the options you wish to change. If an option is unknown it
12867 will be silently ignored. Not all options will have an effect when
12868 changed like this.
12869 For example:
12871 This sets DEBUG level logs (-vv) (these can be set by number or string)
12873 rclone rc options/set --json '{"main": {"LogLevel": "DEBUG"}}'
12875 rclone rc options/set --json '{"main": {"LogLevel": 8}}'
12876 And this sets INFO level logs (-v)
12878 rclone rc options/set --json '{"main": {"LogLevel": "INFO"}}'
12880 And this sets NOTICE level logs (normal without -v)
12882 rclone rc options/set --json '{"main": {"LogLevel": "NOTICE"}}'
12884 pluginsctl/addPlugin: Add a plugin using url
12886 Used for adding a plugin to the webgui.
12887 This takes the following parameters:
12889 • url - http url of the github repo where the plugin is hosted
12891 (http://github.com/rclone/rclone-webui-react).
12892 Example:
12894 rclone rc pluginsctl/addPlugin
12896 Authentication is required for this call.
12898 pluginsctl/getPluginsForType: Get plugins with type criteria
12900 This shows all possible plugins by a mime type.
12901 This takes the following parameters:
12903 • type - supported mime type by a loaded plugin e.g. (video/mp4, au‐
12905 dio/mp3).
12906 • pluginType - filter plugins based on their type e.g. (DASHBOARD,
12908 FILE_HANDLER, TERMINAL).
12909 Returns:
12911 • loadedPlugins - list of current production plugins.
12913 • testPlugins - list of temporarily loaded development plugins, usually
12915 running on a different server.
12916 Example:
12918 rclone rc pluginsctl/getPluginsForType type=video/mp4
12920 Authentication is required for this call.
12922 pluginsctl/listPlugins: Get the list of currently loaded plugins
12924 This allows you to get the currently enabled plugins and their details.
12925 This takes no parameters and returns:
12927 • loadedPlugins - list of current production plugins.
12929 • testPlugins - list of temporarily loaded development plugins, usually
12931 running on a different server.
12932 E.g.
12934 rclone rc pluginsctl/listPlugins
12936 Authentication is required for this call.
12938 pluginsctl/listTestPlugins: Show currently loaded test plugins
12940 Allows listing of test plugins with the rclone.test set to true in
12941 package.json of the plugin.
12942 This takes no parameters and returns:
12944 • loadedTestPlugins - list of currently available test plugins.
12946 E.g.
12948 rclone rc pluginsctl/listTestPlugins
12950 Authentication is required for this call.
12952 pluginsctl/removePlugin: Remove a loaded plugin
12954 This allows you to remove a plugin using it's name.
12955 This takes parameters:
12957 • name - name of the plugin in the format author/plugin_name.
12959 E.g.
12961 rclone rc pluginsctl/removePlugin name=rclone/video-plugin
12963 Authentication is required for this call.
12965 pluginsctl/removeTestPlugin: Remove a test plugin
12967 This allows you to remove a plugin using it's name.
12968 This takes the following parameters:
12970 • name - name of the plugin in the format author/plugin_name.
12972 Example:
12974 rclone rc pluginsctl/removeTestPlugin name=rclone/rclone-webui-react
12976 Authentication is required for this call.
12978 rc/error: This returns an error
12980 This returns an error with the input as part of its error string. Use‐
12981 ful for testing error handling.
12982 rc/list: List all the registered remote control commands
12984 This lists all the registered remote control commands as a JSON map in
12985 the commands response.
12986 rc/noop: Echo the input to the output parameters
12988 This echoes the input parameters to the output parameters for testing
12989 purposes. It can be used to check that rclone is still alive and to
12990 check that parameter passing is working properly.
12991 rc/noopauth: Echo the input to the output parameters requiring auth
12993 This echoes the input parameters to the output parameters for testing
12994 purposes. It can be used to check that rclone is still alive and to
12995 check that parameter passing is working properly.
12996 Authentication is required for this call.
12998 sync/bisync: Perform bidirectional synchronization between two paths.
13000 This takes the following parameters
13001 • path1 - a remote directory string e.g. drive:path1
13003 • path2 - a remote directory string e.g. drive:path2
13005 • dryRun - dry-run mode
13007 • resync - performs the resync run
13009 • checkAccess - abort if RCLONE_TEST files are not found on both
13011 filesystems
13012 • checkFilename - file name for checkAccess (default: RCLONE_TEST)
13014 • maxDelete - abort sync if percentage of deleted files is above this
13016 threshold (default: 50)
13017 • force - maxDelete safety check and run the sync
13019 • checkSync - true by default, false disables comparison of final list‐
13021 ings, only will skip sync, only compare listings from the last run
13022 • removeEmptyDirs - remove empty directories at the final cleanup step
13024 • filtersFile - read filtering patterns from a file
13026 • workdir - server directory for history files (default:
13028 /home/ncw/.cache/rclone/bisync)
13029 • noCleanup - retain working files
13031 See bisync command help (https://rclone.org/commands/rclone_bisync/)
13033 and full bisync description (https://rclone.org/bisync/) for more in‐
13034 formation.
13035 Authentication is required for this call.
13037 sync/copy: copy a directory from source remote to destination remote
13039 This takes the following parameters:
13040 • srcFs - a remote name string e.g. "drive:src" for the source
13042 • dstFs - a remote name string e.g. "drive:dst" for the destination
13044 • createEmptySrcDirs - create empty src directories on destination if
13046 set
13047 See the copy (https://rclone.org/commands/rclone_copy/) command for
13049 more information on the above.
13050 Authentication is required for this call.
13052 sync/move: move a directory from source remote to destination remote
13054 This takes the following parameters:
13055 • srcFs - a remote name string e.g. "drive:src" for the source
13057 • dstFs - a remote name string e.g. "drive:dst" for the destination
13059 • createEmptySrcDirs - create empty src directories on destination if
13061 set
13062 • deleteEmptySrcDirs - delete empty src directories if set
13064 See the move (https://rclone.org/commands/rclone_move/) command for
13066 more information on the above.
13067 Authentication is required for this call.
13069 sync/sync: sync a directory from source remote to destination remote
13071 This takes the following parameters:
13072 • srcFs - a remote name string e.g. "drive:src" for the source
13074 • dstFs - a remote name string e.g. "drive:dst" for the destination
13076 • createEmptySrcDirs - create empty src directories on destination if
13078 set
13079 See the sync (https://rclone.org/commands/rclone_sync/) command for
13081 more information on the above.
13082 Authentication is required for this call.
13084 vfs/forget: Forget files or directories in the directory cache.
13086 This forgets the paths in the directory cache causing them to be re-
13087 read from the remote when needed.
13088 If no paths are passed in then it will forget all the paths in the di‐
13090 rectory cache.
13091 rclone rc vfs/forget
13093 Otherwise pass files or dirs in as file=path or dir=path. Any parame‐
13095 ter key starting with file will forget that file and any starting with
13096 dir will forget that dir, e.g.
13097 rclone rc vfs/forget file=hello file2=goodbye dir=home/junk
13099 This command takes an "fs" parameter. If this parameter is not sup‐
13101 plied and if there is only one VFS in use then that VFS will be used.
13102 If there is more than one VFS in use then the "fs" parameter must be
13103 supplied.
13104 vfs/list: List active VFSes.
13106 This lists the active VFSes.
13107 It returns a list under the key "vfses" where the values are the VFS
13109 names that could be passed to the other VFS commands in the "fs" param‐
13110 eter.
13111 vfs/poll-interval: Get the status or update the value of the poll-interval
13113 option.
13114 Without any parameter given this returns the current status of the
13115 poll-interval setting.
13116 When the interval=duration parameter is set, the poll-interval value is
13118 updated and the polling function is notified. Setting interval=0 dis‐
13119 ables poll-interval.
13120 rclone rc vfs/poll-interval interval=5m
13122 The timeout=duration parameter can be used to specify a time to wait
13124 for the current poll function to apply the new value. If timeout is
13125 less or equal 0, which is the default, wait indefinitely.
13126 The new poll-interval value will only be active when the timeout is not
13128 reached.
13129 If poll-interval is updated or disabled temporarily, some changes might
13131 not get picked up by the polling function, depending on the used re‐
13132 mote.
13133 This command takes an "fs" parameter. If this parameter is not sup‐
13135 plied and if there is only one VFS in use then that VFS will be used.
13136 If there is more than one VFS in use then the "fs" parameter must be
13137 supplied.
13138 vfs/refresh: Refresh the directory cache.
13140 This reads the directories for the specified paths and freshens the di‐
13141 rectory cache.
13142 If no paths are passed in then it will refresh the root directory.
13144 rclone rc vfs/refresh
13146 Otherwise pass directories in as dir=path. Any parameter key starting
13148 with dir will refresh that directory, e.g.
13149 rclone rc vfs/refresh dir=home/junk dir2=data/misc
13151 If the parameter recursive=true is given the whole directory tree will
13153 get refreshed. This refresh will use --fast-list if enabled.
13154 This command takes an "fs" parameter. If this parameter is not sup‐
13156 plied and if there is only one VFS in use then that VFS will be used.
13157 If there is more than one VFS in use then the "fs" parameter must be
13158 supplied.
13159 vfs/stats: Stats for a VFS.
13161 This returns stats for the selected VFS.
13162 {
13164 // Status of the disk cache - only present if --vfs-cache-mode > off
13165 "diskCache": {
13166 "bytesUsed": 0,
13167 "erroredFiles": 0,
13168 "files": 0,
13169 "hashType": 1,
13170 "outOfSpace": false,
13171 "path": "/home/user/.cache/rclone/vfs/local/mnt/a",
13172 "pathMeta": "/home/user/.cache/rclone/vfsMeta/local/mnt/a",
13173 "uploadsInProgress": 0,
13174 "uploadsQueued": 0
13175 },
13176 "fs": "/mnt/a",
13177 "inUse": 1,
13178 // Status of the in memory metadata cache
13179 "metadataCache": {
13180 "dirs": 1,
13181 "files": 0
13182 },
13183 // Options as returned by options/get
13184 "opt": {
13185 "CacheMaxAge": 3600000000000,
13186 // ...
13187 "WriteWait": 1000000000
13188 }
13189 }
13190 This command takes an "fs" parameter. If this parameter is not sup‐
13192 plied and if there is only one VFS in use then that VFS will be used.
13193 If there is more than one VFS in use then the "fs" parameter must be
13194 supplied.
13195 Accessing the remote control via HTTP
13197 Rclone implements a simple HTTP based protocol.
13198 Each endpoint takes an JSON object and returns a JSON object or an er‐
13200 ror. The JSON objects are essentially a map of string names to values.
13201 All calls must made using POST.
13203 The input objects can be supplied using URL parameters, POST parameters
13205 or by supplying "Content-Type: application/json" and a JSON blob in the
13206 body. There are examples of these below using curl.
13207 The response will be a JSON blob in the body of the response. This is
13209 formatted to be reasonably human-readable.
13210 Error returns
13212 If an error occurs then there will be an HTTP error status (e.g. 500)
13213 and the body of the response will contain a JSON encoded error object,
13214 e.g.
13215 {
13217 "error": "Expecting string value for key \"remote\" (was float64)",
13218 "input": {
13219 "fs": "/tmp",
13220 "remote": 3
13221 },
13222 "status": 400
13223 "path": "operations/rmdir",
13224 }
13225 The keys in the error response are - error - error string - input - the
13227 input parameters to the call - status - the HTTP status code - path -
13228 the path of the call
13229 CORS
13231 The sever implements basic CORS support and allows all origins for
13232 that. The response to a preflight OPTIONS request will echo the re‐
13233 quested "Access-Control-Request-Headers" back.
13234 Using POST with URL parameters only
13236 curl -X POST 'http://localhost:5572/rc/noop?potato=1&sausage=2'
13237 Response
13239 {
13241 "potato": "1",
13242 "sausage": "2"
13243 }
13244 Here is what an error response looks like:
13246 curl -X POST 'http://localhost:5572/rc/error?potato=1&sausage=2'
13248 {
13250 "error": "arbitrary error on input map[potato:1 sausage:2]",
13251 "input": {
13252 "potato": "1",
13253 "sausage": "2"
13254 }
13255 }
13256 Note that curl doesn't return errors to the shell unless you use the -f
13258 option
13259 $ curl -f -X POST 'http://localhost:5572/rc/error?potato=1&sausage=2'
13261 curl: (22) The requested URL returned error: 400 Bad Request
13262 $ echo $?
13263 22
13264 Using POST with a form
13266 curl --data "potato=1" --data "sausage=2" http://localhost:5572/rc/noop
13267 Response
13269 {
13271 "potato": "1",
13272 "sausage": "2"
13273 }
13274 Note that you can combine these with URL parameters too with the POST
13276 parameters taking precedence.
13277 curl --data "potato=1" --data "sausage=2" "http://localhost:5572/rc/noop?rutabaga=3&sausage=4"
13279 Response
13281 {
13283 "potato": "1",
13284 "rutabaga": "3",
13285 "sausage": "4"
13286 }
13287 Using POST with a JSON blob
13289 curl -H "Content-Type: application/json" -X POST -d '{"potato":2,"sausage":1}' http://localhost:5572/rc/noop
13290 response
13292 {
13294 "password": "xyz",
13295 "username": "xyz"
13296 }
13297 This can be combined with URL parameters too if required. The JSON
13299 blob takes precedence.
13300 curl -H "Content-Type: application/json" -X POST -d '{"potato":2,"sausage":1}' 'http://localhost:5572/rc/noop?rutabaga=3&potato=4'
13302 {
13304 "potato": 2,
13305 "rutabaga": "3",
13306 "sausage": 1
13307 }
13308 Debugging rclone with pprof
13310 If you use the --rc flag this will also enable the use of the go pro‐
13311 filing tools on the same port.
13312 To use these, first install go (https://golang.org/doc/install).
13314 Debugging memory use
13316 To profile rclone's memory use you can run:
13317 go tool pprof -web http://localhost:5572/debug/pprof/heap
13319 This should open a page in your browser showing what is using what mem‐
13321 ory.
13322 You can also use the -text flag to produce a textual summary
13324 $ go tool pprof -text http://localhost:5572/debug/pprof/heap
13326 Showing nodes accounting for 1537.03kB, 100% of 1537.03kB total
13327 flat flat% sum% cum cum%
13328 1024.03kB 66.62% 66.62% 1024.03kB 66.62% github.com/rclone/rclone/vendor/golang.org/x/net/http2/hpack.addDecoderNode
13329 513kB 33.38% 100% 513kB 33.38% net/http.newBufioWriterSize
13330 0 0% 100% 1024.03kB 66.62% github.com/rclone/rclone/cmd/all.init
13331 0 0% 100% 1024.03kB 66.62% github.com/rclone/rclone/cmd/serve.init
13332 0 0% 100% 1024.03kB 66.62% github.com/rclone/rclone/cmd/serve/restic.init
13333 0 0% 100% 1024.03kB 66.62% github.com/rclone/rclone/vendor/golang.org/x/net/http2.init
13334 0 0% 100% 1024.03kB 66.62% github.com/rclone/rclone/vendor/golang.org/x/net/http2/hpack.init
13335 0 0% 100% 1024.03kB 66.62% github.com/rclone/rclone/vendor/golang.org/x/net/http2/hpack.init.0
13336 0 0% 100% 1024.03kB 66.62% main.init
13337 0 0% 100% 513kB 33.38% net/http.(*conn).readRequest
13338 0 0% 100% 513kB 33.38% net/http.(*conn).serve
13339 0 0% 100% 1024.03kB 66.62% runtime.main
13340 Debugging go routine leaks
13342 Memory leaks are most often caused by go routine leaks keeping memory
13343 alive which should have been garbage collected.
13344 See all active go routines using
13346 curl http://localhost:5572/debug/pprof/goroutine?debug=1
13348 Or go to http://localhost:5572/debug/pprof/goroutine?debug=1 in your
13350 browser.
13351 Other profiles to look at
13353 You can see a summary of profiles available at http://local‐
13354 host:5572/debug/pprof/
13355 Here is how to use some of them:
13357 • Memory: go tool pprof http://localhost:5572/debug/pprof/heap
13359 • Go routines: curl http://localhost:5572/debug/pprof/goroutine?debug=1
13361 • 30-second CPU profile: go tool pprof http://localhost:5572/de‐
13363 bug/pprof/profile
13364 • 5-second execution trace: wget http://localhost:5572/de‐
13366 bug/pprof/trace?seconds=5
13367 • Goroutine blocking profile
13369 • Enable first with: rclone rc debug/set-block-profile-rate rate=1
13371 (docs)
13372 • go tool pprof http://localhost:5572/debug/pprof/block
13374 • Contended mutexes:
13376 • Enable first with: rclone rc debug/set-mutex-profile-fraction
13378 rate=1 (docs)
13379 • go tool pprof http://localhost:5572/debug/pprof/mutex
13381 See the net/http/pprof docs (https://golang.org/pkg/net/http/pprof/)
13383 for more info on how to use the profiling and for a general overview
13384 see the Go team's blog post on profiling go programs
13385 (https://blog.golang.org/profiling-go-programs).
13386 The profiling hook is zero overhead unless it is used (https://stack‐
13388 overflow.com/q/26545159/164234).
13389
13391 Each cloud storage system is slightly different. Rclone attempts to
13392 provide a unified interface to them, but some underlying differences
13393 show through.
13394 Features
13396 Here is an overview of the major features of each cloud storage system.
13397 Name Hash ModTime Case In‐ Duplicate MIME Type Metadata
13399 sensitive Files
13400 ──────────────────────────────────────────────────────────────────────────────────
13401 1Fichier Whirlpool - No Yes R -
13402 Akamai MD5, R/W No No R -
13403 Netstorage SHA256
13404 Amazon MD5 - Yes No R -
13405 Drive
13406 Amazon S3 MD5 R/W No No R/W RWU
13407 (or S3
13408 compati‐
13409 ble)
13410 Backblaze SHA1 R/W No No R/W -
13411 B2
13412 Box SHA1 R/W Yes No - -
13413 Citrix MD5 R/W Yes No - -
13414 ShareFile
13415 Dropbox DBHASH ¹ R Yes No - -
13416 Enterprise - R/W Yes No R/W -
13417 File Fab‐
13418 ric
13419 FTP - R/W ¹⁰ No No - -
13420 Google MD5 R/W No No R/W -
13421 Cloud
13422 Storage
13423 Google MD5 R/W No Yes R/W -
13424 Drive
13425 Google - - No Yes R -
13426 Photos
13427 HDFS - R/W No No - -
13428 HiDrive HiDrive ¹² R/W No No - -
13429 HTTP - R No No R -
13430 Internet MD5, SHA1, R/W ¹¹ No No - RWU
13431 Archive CRC32
13432 Jottacloud MD5 R/W Yes No R -
13433 Koofr MD5 - Yes No - -
13434 Mail.ru Mailru ⁶ R/W Yes No - -
13435 Cloud
13436 Mega - - No Yes - -
13437 Memory MD5 R/W No No - -
13438 Microsoft MD5 R/W No No R/W -
13439 Azure Blob
13440 Storage
13441 Microsoft SHA1 ⁵ R/W Yes No R -
13442 OneDrive
13443 OpenDrive MD5 R/W Yes Partial ⁸ - -
13444 OpenStack MD5 R/W No No R/W -
13445 Swift
13446 Oracle Ob‐ MD5 R/W No No R/W -
13447 ject Stor‐
13448 age
13449 pCloud MD5, SHA1 R No No W -
13450 ⁷
13451 premiu‐ - - Yes No R -
13452 mize.me
13453 put.io CRC-32 R/W No Yes R -
13454 QingStor MD5 - ⁹ No No R/W -
13455 Seafile - - No No - -
13456 SFTP MD5, SHA1 R/W Depends No - -
13457 ²
13458 Sia - - No No - -
13459 SMB - - Yes No - -
13460 SugarSync - - No No - -
13461 Storj - R No No - -
13462 Uptobox - - No Yes - -
13463 WebDAV MD5, SHA1 R ⁴ Depends No - -
13464 ³
13465 Yandex MD5 R/W No No R -
13466 Disk
13467 Zoho Work‐ - - No No - -
13468 Drive
13469 The local All R/W Depends No - RWU
13470 filesystem
13471 Notes
13473 ¹ Dropbox supports its own custom hash (https://www.dropbox.com/devel‐
13474 opers/reference/content-hash). This is an SHA256 sum of all the 4 MiB
13475 block SHA256s.
13476 ² SFTP supports checksums if the same login has shell access and md5sum
13478 or sha1sum as well as echo are in the remote's PATH.
13479 ³ WebDAV supports hashes when used with Owncloud and Nextcloud only.
13481 ⁴ WebDAV supports modtimes when used with Owncloud and Nextcloud only.
13483 ⁵ Microsoft OneDrive Personal supports SHA1 hashes, whereas OneDrive
13485 for business and SharePoint server support Microsoft's own QuickXorHash
13486 (https://docs.microsoft.com/en-us/onedrive/developer/code-snip‐
13487 pets/quickxorhash).
13488 ⁶ Mail.ru uses its own modified SHA1 hash
13490 ⁷ pCloud only supports SHA1 (not MD5) in its EU region
13492 ⁸ Opendrive does not support creation of duplicate files using their
13494 web client interface or other stock clients, but the underlying storage
13495 platform has been determined to allow duplicate files, and it is possi‐
13496 ble to create them with rclone. It may be that this is a mistake or an
13497 unsupported feature.
13498 ⁹ QingStor does not support SetModTime for objects bigger than 5 GiB.
13500 ¹⁰ FTP supports modtimes for the major FTP servers, and also others if
13502 they advertised required protocol extensions. See this
13503 (https://rclone.org/ftp/#modified-time) for more details.
13504 ¹¹ Internet Archive requires option wait_archive to be set to a non-ze‐
13506 ro value for full modtime support.
13507 ¹² HiDrive supports its own custom hash (https://stat‐
13509 ic.hidrive.com/dev/0001). It combines SHA1 sums for each 4 KiB block
13510 hierarchically to a single top-level sum.
13511 Hash
13513 The cloud storage system supports various hash types of the objects.
13514 The hashes are used when transferring data as an integrity check and
13515 can be specifically used with the --checksum flag in syncs and in the
13516 check command.
13517 To use the verify checksums when transferring between cloud storage
13519 systems they must support a common hash type.
13520 ModTime
13522 Almost all cloud storage systems store some sort of timestamp on ob‐
13523 jects, but several of them not something that is appropriate to use for
13524 syncing. E.g. some backends will only write a timestamp that repre‐
13525 sent the time of the upload. To be relevant for syncing it should be
13526 able to store the modification time of the source object. If this is
13527 not the case, rclone will only check the file size by default, though
13528 can be configured to check the file hash (with the --checksum flag).
13529 Ideally it should also be possible to change the timestamp of an exist‐
13530 ing file without having to re-upload it.
13531 Storage systems with a - in the ModTime column, means the modification
13533 read on objects is not the modification time of the file when uploaded.
13534 It is most likely the time the file was uploaded, or possibly something
13535 else (like the time the picture was taken in Google Photos).
13536 Storage systems with a R (for read-only) in the ModTime column, means
13538 the it keeps modification times on objects, and updates them when up‐
13539 loading objects, but it does not support changing only the modification
13540 time (SetModTime operation) without re-uploading, possibly not even
13541 without deleting existing first. Some operations in rclone, such as
13542 copy and sync commands, will automatically check for SetModTime support
13543 and re-upload if necessary to keep the modification times in sync.
13544 Other commands will not work without SetModTime support, e.g. touch
13545 command on an existing file will fail, and changes to modification time
13546 only on a files in a mount will be silently ignored.
13547 Storage systems with R/W (for read/write) in the ModTime column, means
13549 they do also support modtime-only operations.
13550 Case Insensitive
13552 If a cloud storage systems is case sensitive then it is possible to
13553 have two files which differ only in case, e.g. file.txt and FILE.txt.
13554 If a cloud storage system is case insensitive then that isn't possible.
13555 This can cause problems when syncing between a case insensitive system
13557 and a case sensitive system. The symptom of this is that no matter how
13558 many times you run the sync it never completes fully.
13559 The local filesystem and SFTP may or may not be case sensitive depend‐
13561 ing on OS.
13562 • Windows - usually case insensitive, though case is preserved
13564 • OSX - usually case insensitive, though it is possible to format case
13566 sensitive
13567 • Linux - usually case sensitive, but there are case insensitive file
13569 systems (e.g. FAT formatted USB keys)
13570 Most of the time this doesn't cause any problems as people tend to
13572 avoid files whose name differs only by case even on case sensitive sys‐
13573 tems.
13574 Duplicate files
13576 If a cloud storage system allows duplicate files then it can have two
13577 objects with the same name.
13578 This confuses rclone greatly when syncing - use the rclone dedupe com‐
13580 mand to rename or remove duplicates.
13581 Restricted filenames
13583 Some cloud storage systems might have restrictions on the characters
13584 that are usable in file or directory names. When rclone detects such a
13585 name during a file upload, it will transparently replace the restricted
13586 characters with similar looking Unicode characters. To handle the dif‐
13587 ferent sets of restricted characters for different backends, rclone us‐
13588 es something it calls encoding.
13589 This process is designed to avoid ambiguous file names as much as pos‐
13591 sible and allow to move files between many cloud storage systems trans‐
13592 parently.
13593 The name shown by rclone to the user or during log output will only
13595 contain a minimal set of replaced characters to ensure correct format‐
13596 ting and not necessarily the actual name used on the cloud storage.
13597 This transformation is reversed when downloading a file or parsing
13599 rclone arguments. For example, when uploading a file named my
13600 file?.txt to Onedrive, it will be displayed as my file?.txt on the con‐
13601 sole, but stored as my file？.txt to Onedrive (the ? gets replaced by
13602 the similar looking ？ character, the so-called "fullwidth question
13603 mark"). The reverse transformation allows to read a file unusu‐
13604 al/name.txt from Google Drive, by passing the name unusual／name.txt on
13605 the command line (the / needs to be replaced by the similar looking ／
13606 character).
13607 Caveats
13609 The filename encoding system works well in most cases, at least where
13610 file names are written in English or similar languages. You might not
13611 even notice it: It just works. In some cases it may lead to issues,
13612 though. E.g. when file names are written in Chinese, or Japanese,
13613 where it is always the Unicode fullwidth variants of the punctuation
13614 marks that are used.
13615 On Windows, the characters :, * and ? are examples of restricted char‐
13617 acters. If these are used in filenames on a remote that supports it,
13618 Rclone will transparently convert them to their fullwidth Unicode vari‐
13619 ants ＊, ？ and ： when downloading to Windows, and back again when up‐
13620 loading. This way files with names that are not allowed on Windows can
13621 still be stored.
13622 However, if you have files on your Windows system originally with these
13624 same Unicode characters in their names, they will be included in the
13625 same conversion process. E.g. if you create a file in your Windows
13626 filesystem with name Test：1.jpg, where ： is the Unicode fullwidth
13627 colon symbol, and use rclone to upload it to Google Drive, which sup‐
13628 ports regular : (halfwidth question mark), rclone will replace the
13629 fullwidth : with the halfwidth : and store the file as Test:1.jpg in
13630 Google Drive. Since both Windows and Google Drive allows the name
13631 Test：1.jpg, it would probably be better if rclone just kept the name
13632 as is in this case.
13633 With the opposite situation; if you have a file named Test:1.jpg, in
13635 your Google Drive, e.g. uploaded from a Linux system where : is valid
13636 in file names. Then later use rclone to copy this file to your Windows
13637 computer you will notice that on your local disk it gets renamed to
13638 Test：1.jpg. The original filename is not legal on Windows, due to the
13639 :, and rclone therefore renames it to make the copy possible. That is
13640 all good. However, this can also lead to an issue: If you already had
13641 a different file named Test：1.jpg on Windows, and then use rclone to
13642 copy either way. Rclone will then treat the file originally named
13643 Test:1.jpg on Google Drive and the file originally named Test：1.jpg on
13644 Windows as the same file, and replace the contents from one with the
13645 other.
13646 Its virtually impossible to handle all cases like these correctly in
13648 all situations, but by customizing the encoding option, changing the
13649 set of characters that rclone should convert, you should be able to
13650 create a configuration that works well for your specific situation.
13651 See also the example (https://rclone.org/overview/#encoding-example-
13652 windows) below.
13653 (Windows was used as an example of a file system with many restricted
13655 characters, and Google drive a storage system with few.)
13656 Default restricted characters
13658 The table below shows the characters that are replaced by default.
13659 When a replacement character is found in a filename, this character
13661 will be escaped with the ‛ character to avoid ambiguous file names.
13662 (e.g. a file named ␀.txt would shown as ‛␀.txt)
13663 Each cloud storage backend can use a different set of characters, which
13665 will be specified in the documentation for each backend.
13666 Character Value Replacement
13668 ────────────────────────────────
13669 NUL 0x00 ␀
13670 SOH 0x01 ␁
13671 STX 0x02 ␂
13672 ETX 0x03 ␃
13673 EOT 0x04 ␄
13674 ENQ 0x05 ␅
13675 ACK 0x06 ␆
13676 BEL 0x07 ␇
13677 BS 0x08 ␈
13678 HT 0x09 ␉
13679 LF 0x0A ␊
13680 VT 0x0B ␋
13681 FF 0x0C ␌
13682 CR 0x0D ␍
13683 SO 0x0E ␎
13684 SI 0x0F ␏
13685 DLE 0x10 ␐
13686 DC1 0x11 ␑
13687 DC2 0x12 ␒
13688 DC3 0x13 ␓
13689 DC4 0x14 ␔
13690 NAK 0x15 ␕
13691 SYN 0x16 ␖
13692 ETB 0x17 ␗
13693 CAN 0x18 ␘
13694 EM 0x19 ␙
13695 SUB 0x1A ␚
13696 ESC 0x1B ␛
13698 FS 0x1C ␜
13699 GS 0x1D ␝
13700 RS 0x1E ␞
13701 US 0x1F ␟
13702 / 0x2F ／
13703 DEL 0x7F ␡
13704 The default encoding will also encode these file names as they are
13706 problematic with many cloud storage systems.
13707 File name Replacement
13709 ────────────────────────
13710 . ．
13711 .. ．．
13712 Invalid UTF-8 bytes
13714 Some backends only support a sequence of well formed UTF-8 bytes as
13715 file or directory names.
13716 In this case all invalid UTF-8 bytes will be replaced with a quoted
13718 representation of the byte value to allow uploading a file to such a
13719 backend. For example, the invalid byte 0xFE will be encoded as ‛FE.
13720 A common source of invalid UTF-8 bytes are local filesystems, that
13722 store names in a different encoding than UTF-8 or UTF-16, like latin1.
13723 See the local filenames (https://rclone.org/local/#filenames) section
13724 for details.
13725 Encoding option
13727 Most backends have an encoding option, specified as a flag --backend-
13728 encoding where backend is the name of the backend, or as a config pa‐
13729 rameter encoding (you'll need to select the Advanced config in rclone
13730 config to see it).
13731 This will have default value which encodes and decodes characters in
13733 such a way as to preserve the maximum number of characters (see above).
13734 However this can be incorrect in some scenarios, for example if you
13736 have a Windows file system with Unicode fullwidth characters ＊, ？ or
13737 ：, that you want to remain as those characters on the remote rather
13738 than being translated to regular (halfwidth) *, ? and :.
13739 The --backend-encoding flags allow you to change that. You can disable
13741 the encoding completely with --backend-encoding None or set encoding =
13742 None in the config file.
13743 Encoding takes a comma separated list of encodings. You can see the
13745 list of all possible values by passing an invalid value to this flag,
13746 e.g. --local-encoding "help". The command rclone help flags encoding
13747 will show you the defaults for the backends.
13748 Encoding Characters Encoded as
13750 ─────────────────────────────────────────────────────────────────────────────────────
13751 Asterisk * ＊
13752 BackQuote ` ｀
13753 BackSlash \ ＼
13754 Colon : ：
13755 CrLf CR 0x0D, LF 0x0A ␍, ␊
13756 Ctl All control characters ␀␁␂␃␄␅␆␇␈␉␊␋␌␍␎␏␐␑␒␓␔␕␖␗␘␙␚␛␜␝␞␟
13757 0x00-0x1F
13758 Del DEL 0x7F ␡
13759 Dollar $ ＄
13760 Dot . or .. as entire string ．, ．．
13761 DoubleQuote " ＂
13762 Hash # ＃
13763 InvalidUtf8 An invalid UTF-8 charac‐ �
13764 ter (e.g. latin1)
13765 LeftCrLfHtVt CR 0x0D, LF 0x0A, HT ␍, ␊, ␉, ␋
13766 0x09, VT 0x0B on the
13767 left of a string
13768 LeftPeriod . on the left of a .
13769 string
13770 LeftSpace SPACE on the left of a ␠
13771 string
13772 LeftTilde ~ on the left of a ～
13773 string
13774 LtGt <, > ＜, ＞
13775 None No characters are encod‐
13776 ed
13777 Percent % ％
13778 Pipe | ｜
13779 Question ? ？
13780 RightCrLfHtVt CR 0x0D, LF 0x0A, HT ␍, ␊, ␉, ␋
13781 0x09, VT 0x0B on the
13782 right of a string
13783 RightPeriod . on the right of a .
13784 string
13785 RightSpace SPACE on the right of a ␠
13786 string
13787 Semicolon ; ；
13788 SingleQuote ' ＇
13789 Slash / ／
13790 SquareBracket [, ] ［, ］
13791 Encoding example: FTP
13793 To take a specific example, the FTP backend's default encoding is
13794 --ftp-encoding "Slash,Del,Ctl,RightSpace,Dot"
13796 However, let's say the FTP server is running on Windows and can't have
13798 any of the invalid Windows characters in file names. You are backing
13799 up Linux servers to this FTP server which do have those characters in
13800 file names. So you would add the Windows set which are
13801 Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,BackSlash,Ctl,RightSpace,RightPeriod,InvalidUtf8,Dot
13803 to the existing ones, giving:
13805 Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,BackSlash,Ctl,RightSpace,RightPeriod,InvalidUtf8,Dot,Del,RightSpace
13807 This can be specified using the --ftp-encoding flag or using an encod‐
13809 ing parameter in the config file.
13810 Encoding example: Windows
13812 As a nother example, take a Windows system where there is a file with
13813 name Test：1.jpg, where ： is the Unicode fullwidth colon symbol. When
13814 using rclone to copy this to a remote which supports :, the regular
13815 (halfwidth) colon (such as Google Drive), you will notice that the file
13816 gets renamed to Test:1.jpg.
13817 To avoid this you can change the set of characters rclone should con‐
13819 vert for the local filesystem, using command-line argument --local-en‐
13820 coding. Rclone's default behavior on Windows corresponds to
13821 --local-encoding "Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,BackSlash,Ctl,RightSpace,RightPeriod,InvalidUtf8,Dot"
13823 If you want to use fullwidth characters ：, ＊ and ？ in your filenames
13825 without rclone changing them when uploading to a remote, then set the
13826 same as the default value but without Colon,Question,Asterisk:
13827 --local-encoding "Slash,LtGt,DoubleQuote,Pipe,BackSlash,Ctl,RightSpace,RightPeriod,InvalidUtf8,Dot"
13829 Alternatively, you can disable the conversion of any characters with
13831 --local-encoding None.
13832 Instead of using command-line argument --local-encoding, you may also
13834 set it as environment variable (https://rclone.org/docs/#environment-
13835 variables) RCLONE_LOCAL_ENCODING, or configure
13836 (https://rclone.org/docs/#configure) a remote of type local in your
13837 config, and set the encoding option there.
13838 The risk by doing this is that if you have a filename with the regular
13840 (halfwidth) :, * and ? in your cloud storage, and you try to download
13841 it to your Windows filesystem, this will fail. These characters are
13842 not valid in filenames on Windows, and you have told rclone not to work
13843 around this by converting them to valid fullwidth variants.
13844 MIME Type
13846 MIME types (also known as media types) classify types of documents us‐
13847 ing a simple text classification, e.g. text/html or application/pdf.
13848 Some cloud storage systems support reading (R) the MIME type of objects
13850 and some support writing (W) the MIME type of objects.
13851 The MIME type can be important if you are serving files directly to
13853 HTTP from the storage system.
13854 If you are copying from a remote which supports reading (R) to a remote
13856 which supports writing (W) then rclone will preserve the MIME types.
13857 Otherwise they will be guessed from the extension, or the remote itself
13858 may assign the MIME type.
13859 Metadata
13861 Backends may or may support reading or writing metadata. They may sup‐
13862 port reading and writing system metadata (metadata intrinsic to that
13863 backend) and/or user metadata (general purpose metadata).
13864 The levels of metadata support are
13866 Key Explanation
13868 ─────────────────────────────────
13869 R Read only System Metadata
13870 RW Read and write System
13871 Metadata
13872 RWU Read and write System
13873 Metadata and read and
13874 write User Metadata
13875 See the metadata docs (https://rclone.org/docs/#metadata) for more in‐
13877 fo.
13878 Optional Features
13880 All rclone remotes support a base command set. Other features depend
13881 upon backend-specific capabilities.
13882 Name Purge Copy Move DirMove CleanUp ListR StreamU‐ LinkShar‐ About Empty‐
13884 pload ing Dir
13885 ───────────────────────────────────────────────────────────────────────────────────────────────────
13886 1Fichi‐ No Yes Yes No No No No Yes No Yes
13887 er
13888 Akamai Yes No No No No Yes Yes No No Yes
13889 Net‐
13890 storage
13891 Amazon Yes No Yes Yes No No No No No Yes
13892 Drive
13893 Amazon No Yes No No Yes Yes Yes Yes No No
13894 S3 (or
13895 S3 com‐
13896 pati‐
13897 ble)
13898 Back‐ No Yes No No Yes Yes Yes Yes No No
13899 blaze
13900 B2
13901 Box Yes Yes Yes Yes Yes ‡‡ No Yes Yes Yes Yes
13902 Citrix Yes Yes Yes Yes No No Yes No No Yes
13903 Share‐
13904 File
13905 Dropbox Yes Yes Yes Yes No No Yes Yes Yes Yes
13906 Enter‐ Yes Yes Yes Yes Yes No No No No Yes
13908 prise
13909 File
13910 Fabric
13911 FTP No No Yes Yes No No Yes No No Yes
13912 Google Yes Yes No No No Yes Yes No No No
13913 Cloud
13914 Storage
13915 Google Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes
13916 Drive
13917 Google No No No No No No No No No No
13918 Photos
13919 HDFS Yes No Yes Yes No No Yes No Yes Yes
13920 HiDrive Yes Yes Yes Yes No No Yes No No Yes
13921 HTTP No No No No No No No No No Yes
13922 Inter‐ No Yes No No Yes Yes No Yes Yes No
13923 net Ar‐
13924 chive
13925 Jotta‐ Yes Yes Yes Yes Yes Yes No Yes Yes Yes
13926 cloud
13927 Koofr Yes Yes Yes Yes No No Yes Yes Yes Yes
13928 Mail.ru Yes Yes Yes Yes Yes No No Yes Yes Yes
13929 Cloud
13930 Mega Yes No Yes Yes Yes No No Yes Yes Yes
13931 Memory No Yes No No No Yes Yes No No No
13932 Micro‐ Yes Yes No No No Yes Yes No No No
13933 soft
13934 Azure
13935 Blob
13936 Storage
13937 Micro‐ Yes Yes Yes Yes Yes No No Yes Yes Yes
13938 soft
13939 OneDrive
13940 Open‐ Yes Yes Yes Yes No No No No No Yes
13941 Drive
13942 Open‐ Yes † Yes No No No Yes Yes No Yes No
13943 Stack
13944 Swift
13945 Oracle No Yes No No Yes Yes Yes No No No
13946 Object
13947 Storage
13948 pCloud Yes Yes Yes Yes Yes No No Yes Yes Yes
13949 premiu‐ Yes No Yes Yes No No No Yes Yes Yes
13950 mize.me
13951 put.io Yes No Yes Yes Yes No Yes No Yes Yes
13952 QingStor No Yes No No Yes Yes No No No No
13953 Seafile Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes
13954 SFTP No No Yes Yes No No Yes No Yes Yes
13955 Sia No No No No No No Yes No No Yes
13956 SMB No No Yes Yes No No Yes No No Yes
13957 Sug‐ Yes Yes Yes Yes No No Yes Yes No Yes
13958 arSync
13959 Storj Yes † No Yes No No Yes Yes No No No
13960 Uptobox No Yes Yes Yes No No No No No No
13961 WebDAV Yes Yes Yes Yes No No Yes ‡ No Yes Yes
13962 Yandex Yes Yes Yes Yes Yes No Yes Yes Yes Yes
13963 Disk
13964 Zoho Yes Yes Yes Yes No No No No Yes Yes
13965 Work‐
13966 Drive
13967 The lo‐ Yes No Yes Yes No No Yes No Yes Yes
13968 cal
13969 filesys‐
13970 tem
13971 Purge
13973 This deletes a directory quicker than just deleting all the files in
13974 the directory.
13975 † Note Swift and Storj implement this in order to delete directory
13977 markers but they don't actually have a quicker way of deleting files
13978 other than deleting them individually.
13979 ‡ StreamUpload is not supported with Nextcloud
13981 Copy
13983 Used when copying an object to and from the same remote. This known as
13984 a server-side copy so you can copy a file without downloading it and
13985 uploading it again. It is used if you use rclone copy or rclone move
13986 if the remote doesn't support Move directly.
13987 If the server doesn't support Copy directly then for copy operations
13989 the file is downloaded then re-uploaded.
13990 Move
13992 Used when moving/renaming an object on the same remote. This is known
13993 as a server-side move of a file. This is used in rclone move if the
13994 server doesn't support DirMove.
13995 If the server isn't capable of Move then rclone simulates it with Copy
13997 then delete. If the server doesn't support Copy then rclone will down‐
13998 load the file and re-upload it.
13999 DirMove
14001 This is used to implement rclone move to move a directory if possible.
14002 If it isn't then it will use Move on each file (which falls back to
14003 Copy then download and upload - see Move section).
14004 CleanUp
14006 This is used for emptying the trash for a remote by rclone cleanup.
14007 If the server can't do CleanUp then rclone cleanup will return an er‐
14009 ror.
14010 ‡‡ Note that while Box implements this it has to delete every file in‐
14012 dividually so it will be slower than emptying the trash via the WebUI
14013 ListR
14015 The remote supports a recursive list to list all the contents beneath a
14016 directory quickly. This enables the --fast-list flag to work. See the
14017 rclone docs (https://rclone.org/docs/#fast-list) for more details.
14018 StreamUpload
14020 Some remotes allow files to be uploaded without knowing the file size
14021 in advance. This allows certain operations to work without spooling
14022 the file to local disk first, e.g. rclone rcat.
14023 LinkSharing
14025 Sets the necessary permissions on a file or folder and prints a link
14026 that allows others to access them, even if they don't have an account
14027 on the particular cloud provider.
14028 About
14030 Rclone about prints quota information for a remote. Typical output in‐
14031 cludes bytes used, free, quota and in trash.
14032 If a remote lacks about capability rclone about remote:returns an er‐
14034 ror.
14035 Backends without about capability cannot determine free space for an
14037 rclone mount, or use policy mfs (most free space) as a member of an
14038 rclone union remote.
14039 See rclone about command (https://rclone.org/commands/rclone_about/)
14041 EmptyDir
14043 The remote supports empty directories. See Limitations
14044 (https://rclone.org/bugs/#limitations) for details. Most Object/Buck‐
14045 et-based remotes do not support this.
14046
14048 This describes the global flags available to every rclone command split
14049 into two groups, non backend and backend flags.
14050 Non Backend Flags
14052 These flags are available for every command.
14053 --ask-password Allow prompt for password for encrypted configuration (default true)
14055 --auto-confirm If enabled, do not request console confirmation
14056 --backup-dir string Make backups into hierarchy based in DIR
14057 --bind string Local address to bind to for outgoing connections, IPv4, IPv6 or name
14058 --buffer-size SizeSuffix In memory buffer size when reading files for each --transfer (default 16Mi)
14059 --bwlimit BwTimetable Bandwidth limit in KiB/s, or use suffix B|K|M|G|T|P or a full timetable
14060 --bwlimit-file BwTimetable Bandwidth limit per file in KiB/s, or use suffix B|K|M|G|T|P or a full timetable
14061 --ca-cert string CA certificate used to verify servers
14062 --cache-dir string Directory rclone will use for caching (default "$HOME/.cache/rclone")
14063 --check-first Do all the checks before starting transfers
14064 --checkers int Number of checkers to run in parallel (default 8)
14065 -c, --checksum Skip based on checksum (if available) & size, not mod-time & size
14066 --client-cert string Client SSL certificate (PEM) for mutual TLS auth
14067 --client-key string Client SSL private key (PEM) for mutual TLS auth
14068 --compare-dest stringArray Include additional comma separated server-side paths during comparison
14069 --config string Config file (default "$HOME/.config/rclone/rclone.conf")
14070 --contimeout duration Connect timeout (default 1m0s)
14071 --copy-dest stringArray Implies --compare-dest but also copies files from paths into destination
14072 --cpuprofile string Write cpu profile to file
14073 --cutoff-mode string Mode to stop transfers when reaching the max transfer limit HARD|SOFT|CAUTIOUS (default "HARD")
14074 --delete-after When synchronizing, delete files on destination after transferring (default)
14075 --delete-before When synchronizing, delete files on destination before transferring
14076 --delete-during When synchronizing, delete files during transfer
14077 --delete-excluded Delete files on dest excluded from sync
14078 --disable string Disable a comma separated list of features (use --disable help to see a list)
14079 --disable-http-keep-alives Disable HTTP keep-alives and use each connection once.
14080 --disable-http2 Disable HTTP/2 in the global transport
14081 -n, --dry-run Do a trial run with no permanent changes
14082 --dscp string Set DSCP value to connections, value or name, e.g. CS1, LE, DF, AF21
14083 --dump DumpFlags List of items to dump from: headers,bodies,requests,responses,auth,filters,goroutines,openfiles
14084 --dump-bodies Dump HTTP headers and bodies - may contain sensitive info
14085 --dump-headers Dump HTTP headers - may contain sensitive info
14086 --error-on-no-transfer Sets exit code 9 if no files are transferred, useful in scripts
14087 --exclude stringArray Exclude files matching pattern
14088 --exclude-from stringArray Read exclude patterns from file (use - to read from stdin)
14089 --exclude-if-present stringArray Exclude directories if filename is present
14090 --expect-continue-timeout duration Timeout when using expect / 100-continue in HTTP (default 1s)
14091 --fast-list Use recursive list if available; uses more memory but fewer transactions
14092 --files-from stringArray Read list of source-file names from file (use - to read from stdin)
14093 --files-from-raw stringArray Read list of source-file names from file without any processing of lines (use - to read from stdin)
14094 -f, --filter stringArray Add a file-filtering rule
14095 --filter-from stringArray Read filtering patterns from a file (use - to read from stdin)
14096 --fs-cache-expire-duration duration Cache remotes for this long (0 to disable caching) (default 5m0s)
14097 --fs-cache-expire-interval duration Interval to check for expired remotes (default 1m0s)
14098 --header stringArray Set HTTP header for all transactions
14099 --header-download stringArray Set HTTP header for download transactions
14100 --header-upload stringArray Set HTTP header for upload transactions
14101 --human-readable Print numbers in a human-readable format, sizes with suffix Ki|Mi|Gi|Ti|Pi
14102 --ignore-case Ignore case in filters (case insensitive)
14103 --ignore-case-sync Ignore case when synchronizing
14104 --ignore-checksum Skip post copy check of checksums
14105 --ignore-errors Delete even if there are I/O errors
14106 --ignore-existing Skip all files that exist on destination
14107 --ignore-size Ignore size when skipping use mod-time or checksum
14108 -I, --ignore-times Don't skip files that match size and time - transfer all files
14109 --immutable Do not modify files, fail if existing files have been modified
14110 --include stringArray Include files matching pattern
14111 --include-from stringArray Read include patterns from file (use - to read from stdin)
14112 -i, --interactive Enable interactive mode
14113 --kv-lock-time duration Maximum time to keep key-value database locked by process (default 1s)
14114 --log-file string Log everything to this file
14115 --log-format string Comma separated list of log format options (default "date,time")
14116 --log-level string Log level DEBUG|INFO|NOTICE|ERROR (default "NOTICE")
14117 --log-systemd Activate systemd integration for the logger
14118 --low-level-retries int Number of low level retries to do (default 10)
14119 --max-age Duration Only transfer files younger than this in s or suffix ms|s|m|h|d|w|M|y (default off)
14120 --max-backlog int Maximum number of objects in sync or check backlog (default 10000)
14121 --max-delete int When synchronizing, limit the number of deletes (default -1)
14122 --max-depth int If set limits the recursion depth to this (default -1)
14123 --max-duration duration Maximum duration rclone will transfer data for
14124 --max-size SizeSuffix Only transfer files smaller than this in KiB or suffix B|K|M|G|T|P (default off)
14125 --max-stats-groups int Maximum number of stats groups to keep in memory, on max oldest is discarded (default 1000)
14126 --max-transfer SizeSuffix Maximum size of data to transfer (default off)
14127 --memprofile string Write memory profile to file
14128 -M, --metadata If set, preserve metadata when copying objects
14129 --metadata-set stringArray Add metadata key=value when uploading
14130 --min-age Duration Only transfer files older than this in s or suffix ms|s|m|h|d|w|M|y (default off)
14131 --min-size SizeSuffix Only transfer files bigger than this in KiB or suffix B|K|M|G|T|P (default off)
14132 --modify-window duration Max time diff to be considered the same (default 1ns)
14133 --multi-thread-cutoff SizeSuffix Use multi-thread downloads for files above this size (default 250Mi)
14134 --multi-thread-streams int Max number of streams to use for multi-thread downloads (default 4)
14135 --no-check-certificate Do not verify the server SSL certificate (insecure)
14136 --no-check-dest Don't check the destination, copy regardless
14137 --no-console Hide console window (supported on Windows only)
14138 --no-gzip-encoding Don't set Accept-Encoding: gzip
14139 --no-traverse Don't traverse destination file system on copy
14140 --no-unicode-normalization Don't normalize unicode characters in filenames
14141 --no-update-modtime Don't update destination mod-time if files identical
14142 --order-by string Instructions on how to order the transfers, e.g. 'size,descending'
14143 --password-command SpaceSepList Command for supplying password for encrypted configuration
14144 -P, --progress Show progress during transfer
14145 --progress-terminal-title Show progress on the terminal title (requires -P/--progress)
14146 -q, --quiet Print as little stuff as possible
14147 --rc Enable the remote control server
14148 --rc-addr string IPaddress:Port or :Port to bind server to (default "localhost:5572")
14149 --rc-allow-origin string Set the allowed origin for CORS
14150 --rc-baseurl string Prefix for URLs - leave blank for root
14151 --rc-cert string SSL PEM key (concatenation of certificate and CA certificate)
14152 --rc-client-ca string Client certificate authority to verify clients with
14153 --rc-enable-metrics Enable prometheus metrics on /metrics
14154 --rc-files string Path to local files to serve on the HTTP server
14155 --rc-htpasswd string htpasswd file - if not provided no authentication is done
14156 --rc-job-expire-duration duration Expire finished async jobs older than this value (default 1m0s)
14157 --rc-job-expire-interval duration Interval to check for expired async jobs (default 10s)
14158 --rc-key string SSL PEM Private key
14159 --rc-max-header-bytes int Maximum size of request header (default 4096)
14160 --rc-min-tls-version string Minimum TLS version that is acceptable (default "tls1.0")
14161 --rc-no-auth Don't require auth for certain methods
14162 --rc-pass string Password for authentication
14163 --rc-realm string Realm for authentication (default "rclone")
14164 --rc-serve Enable the serving of remote objects
14165 --rc-server-read-timeout duration Timeout for server reading data (default 1h0m0s)
14166 --rc-server-write-timeout duration Timeout for server writing data (default 1h0m0s)
14167 --rc-template string User-specified template
14168 --rc-user string User name for authentication
14169 --rc-web-fetch-url string URL to fetch the releases for webgui (default "https://api.github.com/repos/rclone/rclone-webui-react/releases/latest")
14170 --rc-web-gui Launch WebGUI on localhost
14171 --rc-web-gui-force-update Force update to latest version of web gui
14172 --rc-web-gui-no-open-browser Don't open the browser automatically
14173 --rc-web-gui-update Check and update to latest version of web gui
14174 --refresh-times Refresh the modtime of remote files
14175 --retries int Retry operations this many times if they fail (default 3)
14176 --retries-sleep duration Interval between retrying operations if they fail, e.g. 500ms, 60s, 5m (0 to disable)
14177 --server-side-across-configs Allow server-side operations (e.g. copy) to work across different configs
14178 --size-only Skip based on size only, not mod-time or checksum
14179 --stats duration Interval between printing stats, e.g. 500ms, 60s, 5m (0 to disable) (default 1m0s)
14180 --stats-file-name-length int Max file name length in stats (0 for no limit) (default 45)
14181 --stats-log-level string Log level to show --stats output DEBUG|INFO|NOTICE|ERROR (default "INFO")
14182 --stats-one-line Make the stats fit on one line
14183 --stats-one-line-date Enable --stats-one-line and add current date/time prefix
14184 --stats-one-line-date-format string Enable --stats-one-line-date and use custom formatted date: Enclose date string in double quotes ("), see https://golang.org/pkg/time/#Time.Format
14185 --stats-unit string Show data rate in stats as either 'bits' or 'bytes' per second (default "bytes")
14186 --streaming-upload-cutoff SizeSuffix Cutoff for switching to chunked upload if file size is unknown, upload starts after reaching cutoff or when file ends (default 100Ki)
14187 --suffix string Suffix to add to changed files
14188 --suffix-keep-extension Preserve the extension when using --suffix
14189 --syslog Use Syslog for logging
14190 --syslog-facility string Facility for syslog, e.g. KERN,USER,... (default "DAEMON")
14191 --temp-dir string Directory rclone will use for temporary files (default "/tmp")
14192 --timeout duration IO idle timeout (default 5m0s)
14193 --tpslimit float Limit HTTP transactions per second to this
14194 --tpslimit-burst int Max burst of transactions for --tpslimit (default 1)
14195 --track-renames When synchronizing, track file renames and do a server-side move if possible
14196 --track-renames-strategy string Strategies to use when synchronizing using track-renames hash|modtime|leaf (default "hash")
14197 --transfers int Number of file transfers to run in parallel (default 4)
14198 -u, --update Skip files that are newer on the destination
14199 --use-cookies Enable session cookiejar
14200 --use-json-log Use json log format
14201 --use-mmap Use mmap allocator (see docs)
14202 --use-server-modtime Use server modified time instead of object metadata
14203 --user-agent string Set the user-agent to a specified string (default "rclone/v1.60.1")
14204 -v, --verbose count Print lots more stuff (repeat for more)
14205 Backend Flags
14207 These flags are available for every command. They control the backends
14208 and may be set in the config file.
14209 --acd-auth-url string Auth server URL
14211 --acd-client-id string OAuth Client Id
14212 --acd-client-secret string OAuth Client Secret
14213 --acd-encoding MultiEncoder The encoding for the backend (default Slash,InvalidUtf8,Dot)
14214 --acd-templink-threshold SizeSuffix Files >= this size will be downloaded via their tempLink (default 9Gi)
14215 --acd-token string OAuth Access Token as a JSON blob
14216 --acd-token-url string Token server url
14217 --acd-upload-wait-per-gb Duration Additional time per GiB to wait after a failed complete upload to see if it appears (default 3m0s)
14218 --alias-remote string Remote or path to alias
14219 --azureblob-access-tier string Access tier of blob: hot, cool or archive
14220 --azureblob-account string Storage Account Name
14221 --azureblob-archive-tier-delete Delete archive tier blobs before overwriting
14222 --azureblob-chunk-size SizeSuffix Upload chunk size (default 4Mi)
14223 --azureblob-disable-checksum Don't store MD5 checksum with object metadata
14224 --azureblob-encoding MultiEncoder The encoding for the backend (default Slash,BackSlash,Del,Ctl,RightPeriod,InvalidUtf8)
14225 --azureblob-endpoint string Endpoint for the service
14226 --azureblob-key string Storage Account Key
14227 --azureblob-list-chunk int Size of blob list (default 5000)
14228 --azureblob-memory-pool-flush-time Duration How often internal memory buffer pools will be flushed (default 1m0s)
14229 --azureblob-memory-pool-use-mmap Whether to use mmap buffers in internal memory pool
14230 --azureblob-msi-client-id string Object ID of the user-assigned MSI to use, if any
14231 --azureblob-msi-mi-res-id string Azure resource ID of the user-assigned MSI to use, if any
14232 --azureblob-msi-object-id string Object ID of the user-assigned MSI to use, if any
14233 --azureblob-no-head-object If set, do not do HEAD before GET when getting objects
14234 --azureblob-public-access string Public access level of a container: blob or container
14235 --azureblob-sas-url string SAS URL for container level access only
14236 --azureblob-service-principal-file string Path to file containing credentials for use with a service principal
14237 --azureblob-upload-concurrency int Concurrency for multipart uploads (default 16)
14238 --azureblob-upload-cutoff string Cutoff for switching to chunked upload (<= 256 MiB) (deprecated)
14239 --azureblob-use-emulator Uses local storage emulator if provided as 'true'
14240 --azureblob-use-msi Use a managed service identity to authenticate (only works in Azure)
14241 --b2-account string Account ID or Application Key ID
14242 --b2-chunk-size SizeSuffix Upload chunk size (default 96Mi)
14243 --b2-copy-cutoff SizeSuffix Cutoff for switching to multipart copy (default 4Gi)
14244 --b2-disable-checksum Disable checksums for large (> upload cutoff) files
14245 --b2-download-auth-duration Duration Time before the authorization token will expire in s or suffix ms|s|m|h|d (default 1w)
14246 --b2-download-url string Custom endpoint for downloads
14247 --b2-encoding MultiEncoder The encoding for the backend (default Slash,BackSlash,Del,Ctl,InvalidUtf8,Dot)
14248 --b2-endpoint string Endpoint for the service
14249 --b2-hard-delete Permanently delete files on remote removal, otherwise hide files
14250 --b2-key string Application Key
14251 --b2-memory-pool-flush-time Duration How often internal memory buffer pools will be flushed (default 1m0s)
14252 --b2-memory-pool-use-mmap Whether to use mmap buffers in internal memory pool
14253 --b2-test-mode string A flag string for X-Bz-Test-Mode header for debugging
14254 --b2-upload-cutoff SizeSuffix Cutoff for switching to chunked upload (default 200Mi)
14255 --b2-version-at Time Show file versions as they were at the specified time (default off)
14256 --b2-versions Include old versions in directory listings
14257 --box-access-token string Box App Primary Access Token
14258 --box-auth-url string Auth server URL
14259 --box-box-config-file string Box App config.json location
14260 --box-box-sub-type string (default "user")
14261 --box-client-id string OAuth Client Id
14262 --box-client-secret string OAuth Client Secret
14263 --box-commit-retries int Max number of times to try committing a multipart file (default 100)
14264 --box-encoding MultiEncoder The encoding for the backend (default Slash,BackSlash,Del,Ctl,RightSpace,InvalidUtf8,Dot)
14265 --box-list-chunk int Size of listing chunk 1-1000 (default 1000)
14266 --box-owned-by string Only show items owned by the login (email address) passed in
14267 --box-root-folder-id string Fill in for rclone to use a non root folder as its starting point
14268 --box-token string OAuth Access Token as a JSON blob
14269 --box-token-url string Token server url
14270 --box-upload-cutoff SizeSuffix Cutoff for switching to multipart upload (>= 50 MiB) (default 50Mi)
14271 --cache-chunk-clean-interval Duration How often should the cache perform cleanups of the chunk storage (default 1m0s)
14272 --cache-chunk-no-memory Disable the in-memory cache for storing chunks during streaming
14273 --cache-chunk-path string Directory to cache chunk files (default "$HOME/.cache/rclone/cache-backend")
14274 --cache-chunk-size SizeSuffix The size of a chunk (partial file data) (default 5Mi)
14275 --cache-chunk-total-size SizeSuffix The total size that the chunks can take up on the local disk (default 10Gi)
14276 --cache-db-path string Directory to store file structure metadata DB (default "$HOME/.cache/rclone/cache-backend")
14277 --cache-db-purge Clear all the cached data for this remote on start
14278 --cache-db-wait-time Duration How long to wait for the DB to be available - 0 is unlimited (default 1s)
14279 --cache-info-age Duration How long to cache file structure information (directory listings, file size, times, etc.) (default 6h0m0s)
14280 --cache-plex-insecure string Skip all certificate verification when connecting to the Plex server
14281 --cache-plex-password string The password of the Plex user (obscured)
14282 --cache-plex-url string The URL of the Plex server
14283 --cache-plex-username string The username of the Plex user
14284 --cache-read-retries int How many times to retry a read from a cache storage (default 10)
14285 --cache-remote string Remote to cache
14286 --cache-rps int Limits the number of requests per second to the source FS (-1 to disable) (default -1)
14287 --cache-tmp-upload-path string Directory to keep temporary files until they are uploaded
14288 --cache-tmp-wait-time Duration How long should files be stored in local cache before being uploaded (default 15s)
14289 --cache-workers int How many workers should run in parallel to download chunks (default 4)
14290 --cache-writes Cache file data on writes through the FS
14291 --chunker-chunk-size SizeSuffix Files larger than chunk size will be split in chunks (default 2Gi)
14292 --chunker-fail-hard Choose how chunker should handle files with missing or invalid chunks
14293 --chunker-hash-type string Choose how chunker handles hash sums (default "md5")
14294 --chunker-remote string Remote to chunk/unchunk
14295 --combine-upstreams SpaceSepList Upstreams for combining
14296 --compress-level int GZIP compression level (-2 to 9) (default -1)
14297 --compress-mode string Compression mode (default "gzip")
14298 --compress-ram-cache-limit SizeSuffix Some remotes don't allow the upload of files with unknown size (default 20Mi)
14299 --compress-remote string Remote to compress
14300 -L, --copy-links Follow symlinks and copy the pointed to item
14301 --crypt-directory-name-encryption Option to either encrypt directory names or leave them intact (default true)
14302 --crypt-filename-encoding string How to encode the encrypted filename to text string (default "base32")
14303 --crypt-filename-encryption string How to encrypt the filenames (default "standard")
14304 --crypt-no-data-encryption Option to either encrypt file data or leave it unencrypted
14305 --crypt-password string Password or pass phrase for encryption (obscured)
14306 --crypt-password2 string Password or pass phrase for salt (obscured)
14307 --crypt-remote string Remote to encrypt/decrypt
14308 --crypt-server-side-across-configs Allow server-side operations (e.g. copy) to work across different crypt configs
14309 --crypt-show-mapping For all files listed show how the names encrypt
14310 --drive-acknowledge-abuse Set to allow files which return cannotDownloadAbusiveFile to be downloaded
14311 --drive-allow-import-name-change Allow the filetype to change when uploading Google docs
14312 --drive-auth-owner-only Only consider files owned by the authenticated user
14313 --drive-auth-url string Auth server URL
14314 --drive-chunk-size SizeSuffix Upload chunk size (default 8Mi)
14315 --drive-client-id string Google Application Client Id
14316 --drive-client-secret string OAuth Client Secret
14317 --drive-copy-shortcut-content Server side copy contents of shortcuts instead of the shortcut
14318 --drive-disable-http2 Disable drive using http2 (default true)
14319 --drive-encoding MultiEncoder The encoding for the backend (default InvalidUtf8)
14320 --drive-export-formats string Comma separated list of preferred formats for downloading Google docs (default "docx,xlsx,pptx,svg")
14321 --drive-formats string Deprecated: See export_formats
14322 --drive-impersonate string Impersonate this user when using a service account
14323 --drive-import-formats string Comma separated list of preferred formats for uploading Google docs
14324 --drive-keep-revision-forever Keep new head revision of each file forever
14325 --drive-list-chunk int Size of listing chunk 100-1000, 0 to disable (default 1000)
14326 --drive-pacer-burst int Number of API calls to allow without sleeping (default 100)
14327 --drive-pacer-min-sleep Duration Minimum time to sleep between API calls (default 100ms)
14328 --drive-resource-key string Resource key for accessing a link-shared file
14329 --drive-root-folder-id string ID of the root folder
14330 --drive-scope string Scope that rclone should use when requesting access from drive
14331 --drive-server-side-across-configs Allow server-side operations (e.g. copy) to work across different drive configs
14332 --drive-service-account-credentials string Service Account Credentials JSON blob
14333 --drive-service-account-file string Service Account Credentials JSON file path
14334 --drive-shared-with-me Only show files that are shared with me
14335 --drive-size-as-quota Show sizes as storage quota usage, not actual size
14336 --drive-skip-checksum-gphotos Skip MD5 checksum on Google photos and videos only
14337 --drive-skip-dangling-shortcuts If set skip dangling shortcut files
14338 --drive-skip-gdocs Skip google documents in all listings
14339 --drive-skip-shortcuts If set skip shortcut files
14340 --drive-starred-only Only show files that are starred
14341 --drive-stop-on-download-limit Make download limit errors be fatal
14342 --drive-stop-on-upload-limit Make upload limit errors be fatal
14343 --drive-team-drive string ID of the Shared Drive (Team Drive)
14344 --drive-token string OAuth Access Token as a JSON blob
14345 --drive-token-url string Token server url
14346 --drive-trashed-only Only show files that are in the trash
14347 --drive-upload-cutoff SizeSuffix Cutoff for switching to chunked upload (default 8Mi)
14348 --drive-use-created-date Use file created date instead of modified date
14349 --drive-use-shared-date Use date file was shared instead of modified date
14350 --drive-use-trash Send files to the trash instead of deleting permanently (default true)
14351 --drive-v2-download-min-size SizeSuffix If Object's are greater, use drive v2 API to download (default off)
14352 --dropbox-auth-url string Auth server URL
14353 --dropbox-batch-commit-timeout Duration Max time to wait for a batch to finish committing (default 10m0s)
14354 --dropbox-batch-mode string Upload file batching sync|async|off (default "sync")
14355 --dropbox-batch-size int Max number of files in upload batch
14356 --dropbox-batch-timeout Duration Max time to allow an idle upload batch before uploading (default 0s)
14357 --dropbox-chunk-size SizeSuffix Upload chunk size (< 150Mi) (default 48Mi)
14358 --dropbox-client-id string OAuth Client Id
14359 --dropbox-client-secret string OAuth Client Secret
14360 --dropbox-encoding MultiEncoder The encoding for the backend (default Slash,BackSlash,Del,RightSpace,InvalidUtf8,Dot)
14361 --dropbox-impersonate string Impersonate this user when using a business account
14362 --dropbox-shared-files Instructs rclone to work on individual shared files
14363 --dropbox-shared-folders Instructs rclone to work on shared folders
14364 --dropbox-token string OAuth Access Token as a JSON blob
14365 --dropbox-token-url string Token server url
14366 --fichier-api-key string Your API Key, get it from https://1fichier.com/console/params.pl
14367 --fichier-encoding MultiEncoder The encoding for the backend (default Slash,LtGt,DoubleQuote,SingleQuote,BackQuote,Dollar,BackSlash,Del,Ctl,LeftSpace,RightSpace,InvalidUtf8,Dot)
14368 --fichier-file-password string If you want to download a shared file that is password protected, add this parameter (obscured)
14369 --fichier-folder-password string If you want to list the files in a shared folder that is password protected, add this parameter (obscured)
14370 --fichier-shared-folder string If you want to download a shared folder, add this parameter
14371 --filefabric-encoding MultiEncoder The encoding for the backend (default Slash,Del,Ctl,InvalidUtf8,Dot)
14372 --filefabric-permanent-token string Permanent Authentication Token
14373 --filefabric-root-folder-id string ID of the root folder
14374 --filefabric-token string Session Token
14375 --filefabric-token-expiry string Token expiry time
14376 --filefabric-url string URL of the Enterprise File Fabric to connect to
14377 --filefabric-version string Version read from the file fabric
14378 --ftp-ask-password Allow asking for FTP password when needed
14379 --ftp-close-timeout Duration Maximum time to wait for a response to close (default 1m0s)
14380 --ftp-concurrency int Maximum number of FTP simultaneous connections, 0 for unlimited
14381 --ftp-disable-epsv Disable using EPSV even if server advertises support
14382 --ftp-disable-mlsd Disable using MLSD even if server advertises support
14383 --ftp-disable-tls13 Disable TLS 1.3 (workaround for FTP servers with buggy TLS)
14384 --ftp-disable-utf8 Disable using UTF-8 even if server advertises support
14385 --ftp-encoding MultiEncoder The encoding for the backend (default Slash,Del,Ctl,RightSpace,Dot)
14386 --ftp-explicit-tls Use Explicit FTPS (FTP over TLS)
14387 --ftp-force-list-hidden Use LIST -a to force listing of hidden files and folders. This will disable the use of MLSD
14388 --ftp-host string FTP host to connect to
14389 --ftp-idle-timeout Duration Max time before closing idle connections (default 1m0s)
14390 --ftp-no-check-certificate Do not verify the TLS certificate of the server
14391 --ftp-pass string FTP password (obscured)
14392 --ftp-port int FTP port number (default 21)
14393 --ftp-shut-timeout Duration Maximum time to wait for data connection closing status (default 1m0s)
14394 --ftp-tls Use Implicit FTPS (FTP over TLS)
14395 --ftp-tls-cache-size int Size of TLS session cache for all control and data connections (default 32)
14396 --ftp-user string FTP username (default "$USER")
14397 --ftp-writing-mdtm Use MDTM to set modification time (VsFtpd quirk)
14398 --gcs-anonymous Access public buckets and objects without credentials
14399 --gcs-auth-url string Auth server URL
14400 --gcs-bucket-acl string Access Control List for new buckets
14401 --gcs-bucket-policy-only Access checks should use bucket-level IAM policies
14402 --gcs-client-id string OAuth Client Id
14403 --gcs-client-secret string OAuth Client Secret
14404 --gcs-decompress If set this will decompress gzip encoded objects
14405 --gcs-encoding MultiEncoder The encoding for the backend (default Slash,CrLf,InvalidUtf8,Dot)
14406 --gcs-endpoint string Endpoint for the service
14407 --gcs-location string Location for the newly created buckets
14408 --gcs-no-check-bucket If set, don't attempt to check the bucket exists or create it
14409 --gcs-object-acl string Access Control List for new objects
14410 --gcs-project-number string Project number
14411 --gcs-service-account-file string Service Account Credentials JSON file path
14412 --gcs-storage-class string The storage class to use when storing objects in Google Cloud Storage
14413 --gcs-token string OAuth Access Token as a JSON blob
14414 --gcs-token-url string Token server url
14415 --gphotos-auth-url string Auth server URL
14416 --gphotos-client-id string OAuth Client Id
14417 --gphotos-client-secret string OAuth Client Secret
14418 --gphotos-encoding MultiEncoder The encoding for the backend (default Slash,CrLf,InvalidUtf8,Dot)
14419 --gphotos-include-archived Also view and download archived media
14420 --gphotos-read-only Set to make the Google Photos backend read only
14421 --gphotos-read-size Set to read the size of media items
14422 --gphotos-start-year int Year limits the photos to be downloaded to those which are uploaded after the given year (default 2000)
14423 --gphotos-token string OAuth Access Token as a JSON blob
14424 --gphotos-token-url string Token server url
14425 --hasher-auto-size SizeSuffix Auto-update checksum for files smaller than this size (disabled by default)
14426 --hasher-hashes CommaSepList Comma separated list of supported checksum types (default md5,sha1)
14427 --hasher-max-age Duration Maximum time to keep checksums in cache (0 = no cache, off = cache forever) (default off)
14428 --hasher-remote string Remote to cache checksums for (e.g. myRemote:path)
14429 --hdfs-data-transfer-protection string Kerberos data transfer protection: authentication|integrity|privacy
14430 --hdfs-encoding MultiEncoder The encoding for the backend (default Slash,Colon,Del,Ctl,InvalidUtf8,Dot)
14431 --hdfs-namenode string Hadoop name node and port
14432 --hdfs-service-principal-name string Kerberos service principal name for the namenode
14433 --hdfs-username string Hadoop user name
14434 --hidrive-auth-url string Auth server URL
14435 --hidrive-chunk-size SizeSuffix Chunksize for chunked uploads (default 48Mi)
14436 --hidrive-client-id string OAuth Client Id
14437 --hidrive-client-secret string OAuth Client Secret
14438 --hidrive-disable-fetching-member-count Do not fetch number of objects in directories unless it is absolutely necessary
14439 --hidrive-encoding MultiEncoder The encoding for the backend (default Slash,Dot)
14440 --hidrive-endpoint string Endpoint for the service (default "https://api.hidrive.strato.com/2.1")
14441 --hidrive-root-prefix string The root/parent folder for all paths (default "/")
14442 --hidrive-scope-access string Access permissions that rclone should use when requesting access from HiDrive (default "rw")
14443 --hidrive-scope-role string User-level that rclone should use when requesting access from HiDrive (default "user")
14444 --hidrive-token string OAuth Access Token as a JSON blob
14445 --hidrive-token-url string Token server url
14446 --hidrive-upload-concurrency int Concurrency for chunked uploads (default 4)
14447 --hidrive-upload-cutoff SizeSuffix Cutoff/Threshold for chunked uploads (default 96Mi)
14448 --http-headers CommaSepList Set HTTP headers for all transactions
14449 --http-no-head Don't use HEAD requests
14450 --http-no-slash Set this if the site doesn't end directories with /
14451 --http-url string URL of HTTP host to connect to
14452 --internetarchive-access-key-id string IAS3 Access Key
14453 --internetarchive-disable-checksum Don't ask the server to test against MD5 checksum calculated by rclone (default true)
14454 --internetarchive-encoding MultiEncoder The encoding for the backend (default Slash,LtGt,CrLf,Del,Ctl,InvalidUtf8,Dot)
14455 --internetarchive-endpoint string IAS3 Endpoint (default "https://s3.us.archive.org")
14456 --internetarchive-front-endpoint string Host of InternetArchive Frontend (default "https://archive.org")
14457 --internetarchive-secret-access-key string IAS3 Secret Key (password)
14458 --internetarchive-wait-archive Duration Timeout for waiting the server's processing tasks (specifically archive and book_op) to finish (default 0s)
14459 --jottacloud-encoding MultiEncoder The encoding for the backend (default Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,Del,Ctl,InvalidUtf8,Dot)
14460 --jottacloud-hard-delete Delete files permanently rather than putting them into the trash
14461 --jottacloud-md5-memory-limit SizeSuffix Files bigger than this will be cached on disk to calculate the MD5 if required (default 10Mi)
14462 --jottacloud-no-versions Avoid server side versioning by deleting files and recreating files instead of overwriting them
14463 --jottacloud-trashed-only Only show files that are in the trash
14464 --jottacloud-upload-resume-limit SizeSuffix Files bigger than this can be resumed if the upload fail's (default 10Mi)
14465 --koofr-encoding MultiEncoder The encoding for the backend (default Slash,BackSlash,Del,Ctl,InvalidUtf8,Dot)
14466 --koofr-endpoint string The Koofr API endpoint to use
14467 --koofr-mountid string Mount ID of the mount to use
14468 --koofr-password string Your password for rclone (generate one at https://app.koofr.net/app/admin/preferences/password) (obscured)
14469 --koofr-provider string Choose your storage provider
14470 --koofr-setmtime Does the backend support setting modification time (default true)
14471 --koofr-user string Your user name
14472 -l, --links Translate symlinks to/from regular files with a '.rclonelink' extension
14473 --local-case-insensitive Force the filesystem to report itself as case insensitive
14474 --local-case-sensitive Force the filesystem to report itself as case sensitive
14475 --local-encoding MultiEncoder The encoding for the backend (default Slash,Dot)
14476 --local-no-check-updated Don't check to see if the files change during upload
14477 --local-no-preallocate Disable preallocation of disk space for transferred files
14478 --local-no-set-modtime Disable setting modtime
14479 --local-no-sparse Disable sparse files for multi-thread downloads
14480 --local-nounc Disable UNC (long path names) conversion on Windows
14481 --local-unicode-normalization Apply unicode NFC normalization to paths and filenames
14482 --local-zero-size-links Assume the Stat size of links is zero (and read them instead) (deprecated)
14483 --mailru-check-hash What should copy do if file checksum is mismatched or invalid (default true)
14484 --mailru-encoding MultiEncoder The encoding for the backend (default Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,BackSlash,Del,Ctl,InvalidUtf8,Dot)
14485 --mailru-pass string Password (obscured)
14486 --mailru-speedup-enable Skip full upload if there is another file with same data hash (default true)
14487 --mailru-speedup-file-patterns string Comma separated list of file name patterns eligible for speedup (put by hash) (default "*.mkv,*.avi,*.mp4,*.mp3,*.zip,*.gz,*.rar,*.pdf")
14488 --mailru-speedup-max-disk SizeSuffix This option allows you to disable speedup (put by hash) for large files (default 3Gi)
14489 --mailru-speedup-max-memory SizeSuffix Files larger than the size given below will always be hashed on disk (default 32Mi)
14490 --mailru-user string User name (usually email)
14491 --mega-debug Output more debug from Mega
14492 --mega-encoding MultiEncoder The encoding for the backend (default Slash,InvalidUtf8,Dot)
14493 --mega-hard-delete Delete files permanently rather than putting them into the trash
14494 --mega-pass string Password (obscured)
14495 --mega-user string User name
14496 --netstorage-account string Set the NetStorage account name
14497 --netstorage-host string Domain+path of NetStorage host to connect to
14498 --netstorage-protocol string Select between HTTP or HTTPS protocol (default "https")
14499 --netstorage-secret string Set the NetStorage account secret/G2O key for authentication (obscured)
14500 -x, --one-file-system Don't cross filesystem boundaries (unix/macOS only)
14501 --onedrive-access-scopes SpaceSepList Set scopes to be requested by rclone (default Files.Read Files.ReadWrite Files.Read.All Files.ReadWrite.All Sites.Read.All offline_access)
14502 --onedrive-auth-url string Auth server URL
14503 --onedrive-chunk-size SizeSuffix Chunk size to upload files with - must be multiple of 320k (327,680 bytes) (default 10Mi)
14504 --onedrive-client-id string OAuth Client Id
14505 --onedrive-client-secret string OAuth Client Secret
14506 --onedrive-drive-id string The ID of the drive to use
14507 --onedrive-drive-type string The type of the drive (personal | business | documentLibrary)
14508 --onedrive-encoding MultiEncoder The encoding for the backend (default Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,BackSlash,Del,Ctl,LeftSpace,LeftTilde,RightSpace,RightPeriod,InvalidUtf8,Dot)
14509 --onedrive-expose-onenote-files Set to make OneNote files show up in directory listings
14510 --onedrive-link-password string Set the password for links created by the link command
14511 --onedrive-link-scope string Set the scope of the links created by the link command (default "anonymous")
14512 --onedrive-link-type string Set the type of the links created by the link command (default "view")
14513 --onedrive-list-chunk int Size of listing chunk (default 1000)
14514 --onedrive-no-versions Remove all versions on modifying operations
14515 --onedrive-region string Choose national cloud region for OneDrive (default "global")
14516 --onedrive-root-folder-id string ID of the root folder
14517 --onedrive-server-side-across-configs Allow server-side operations (e.g. copy) to work across different onedrive configs
14518 --onedrive-token string OAuth Access Token as a JSON blob
14519 --onedrive-token-url string Token server url
14520 --oos-chunk-size SizeSuffix Chunk size to use for uploading (default 5Mi)
14521 --oos-compartment string Object storage compartment OCID
14522 --oos-config-file string Path to OCI config file (default "~/.oci/config")
14523 --oos-config-profile string Profile name inside the oci config file (default "Default")
14524 --oos-copy-cutoff SizeSuffix Cutoff for switching to multipart copy (default 4.656Gi)
14525 --oos-copy-timeout Duration Timeout for copy (default 1m0s)
14526 --oos-disable-checksum Don't store MD5 checksum with object metadata
14527 --oos-encoding MultiEncoder The encoding for the backend (default Slash,InvalidUtf8,Dot)
14528 --oos-endpoint string Endpoint for Object storage API
14529 --oos-leave-parts-on-error If true avoid calling abort upload on a failure, leaving all successfully uploaded parts on S3 for manual recovery
14530 --oos-namespace string Object storage namespace
14531 --oos-no-check-bucket If set, don't attempt to check the bucket exists or create it
14532 --oos-provider string Choose your Auth Provider (default "env_auth")
14533 --oos-region string Object storage Region
14534 --oos-upload-concurrency int Concurrency for multipart uploads (default 10)
14535 --oos-upload-cutoff SizeSuffix Cutoff for switching to chunked upload (default 200Mi)
14536 --opendrive-chunk-size SizeSuffix Files will be uploaded in chunks this size (default 10Mi)
14537 --opendrive-encoding MultiEncoder The encoding for the backend (default Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,BackSlash,LeftSpace,LeftCrLfHtVt,RightSpace,RightCrLfHtVt,InvalidUtf8,Dot)
14538 --opendrive-password string Password (obscured)
14539 --opendrive-username string Username
14540 --pcloud-auth-url string Auth server URL
14541 --pcloud-client-id string OAuth Client Id
14542 --pcloud-client-secret string OAuth Client Secret
14543 --pcloud-encoding MultiEncoder The encoding for the backend (default Slash,BackSlash,Del,Ctl,InvalidUtf8,Dot)
14544 --pcloud-hostname string Hostname to connect to (default "api.pcloud.com")
14545 --pcloud-password string Your pcloud password (obscured)
14546 --pcloud-root-folder-id string Fill in for rclone to use a non root folder as its starting point (default "d0")
14547 --pcloud-token string OAuth Access Token as a JSON blob
14548 --pcloud-token-url string Token server url
14549 --pcloud-username string Your pcloud username
14550 --premiumizeme-encoding MultiEncoder The encoding for the backend (default Slash,DoubleQuote,BackSlash,Del,Ctl,InvalidUtf8,Dot)
14551 --putio-encoding MultiEncoder The encoding for the backend (default Slash,BackSlash,Del,Ctl,InvalidUtf8,Dot)
14552 --qingstor-access-key-id string QingStor Access Key ID
14553 --qingstor-chunk-size SizeSuffix Chunk size to use for uploading (default 4Mi)
14554 --qingstor-connection-retries int Number of connection retries (default 3)
14555 --qingstor-encoding MultiEncoder The encoding for the backend (default Slash,Ctl,InvalidUtf8)
14556 --qingstor-endpoint string Enter an endpoint URL to connection QingStor API
14557 --qingstor-env-auth Get QingStor credentials from runtime
14558 --qingstor-secret-access-key string QingStor Secret Access Key (password)
14559 --qingstor-upload-concurrency int Concurrency for multipart uploads (default 1)
14560 --qingstor-upload-cutoff SizeSuffix Cutoff for switching to chunked upload (default 200Mi)
14561 --qingstor-zone string Zone to connect to
14562 --s3-access-key-id string AWS Access Key ID
14563 --s3-acl string Canned ACL used when creating buckets and storing or copying objects
14564 --s3-bucket-acl string Canned ACL used when creating buckets
14565 --s3-chunk-size SizeSuffix Chunk size to use for uploading (default 5Mi)
14566 --s3-copy-cutoff SizeSuffix Cutoff for switching to multipart copy (default 4.656Gi)
14567 --s3-decompress If set this will decompress gzip encoded objects
14568 --s3-disable-checksum Don't store MD5 checksum with object metadata
14569 --s3-disable-http2 Disable usage of http2 for S3 backends
14570 --s3-download-url string Custom endpoint for downloads
14571 --s3-encoding MultiEncoder The encoding for the backend (default Slash,InvalidUtf8,Dot)
14572 --s3-endpoint string Endpoint for S3 API
14573 --s3-env-auth Get AWS credentials from runtime (environment variables or EC2/ECS meta data if no env vars)
14574 --s3-force-path-style If true use path style access if false use virtual hosted style (default true)
14575 --s3-leave-parts-on-error If true avoid calling abort upload on a failure, leaving all successfully uploaded parts on S3 for manual recovery
14576 --s3-list-chunk int Size of listing chunk (response list for each ListObject S3 request) (default 1000)
14577 --s3-list-url-encode Tristate Whether to url encode listings: true/false/unset (default unset)
14578 --s3-list-version int Version of ListObjects to use: 1,2 or 0 for auto
14579 --s3-location-constraint string Location constraint - must be set to match the Region
14580 --s3-max-upload-parts int Maximum number of parts in a multipart upload (default 10000)
14581 --s3-memory-pool-flush-time Duration How often internal memory buffer pools will be flushed (default 1m0s)
14582 --s3-memory-pool-use-mmap Whether to use mmap buffers in internal memory pool
14583 --s3-might-gzip Tristate Set this if the backend might gzip objects (default unset)
14584 --s3-no-check-bucket If set, don't attempt to check the bucket exists or create it
14585 --s3-no-head If set, don't HEAD uploaded objects to check integrity
14586 --s3-no-head-object If set, do not do HEAD before GET when getting objects
14587 --s3-no-system-metadata Suppress setting and reading of system metadata
14588 --s3-profile string Profile to use in the shared credentials file
14589 --s3-provider string Choose your S3 provider
14590 --s3-region string Region to connect to
14591 --s3-requester-pays Enables requester pays option when interacting with S3 bucket
14592 --s3-secret-access-key string AWS Secret Access Key (password)
14593 --s3-server-side-encryption string The server-side encryption algorithm used when storing this object in S3
14594 --s3-session-token string An AWS session token
14595 --s3-shared-credentials-file string Path to the shared credentials file
14596 --s3-sse-customer-algorithm string If using SSE-C, the server-side encryption algorithm used when storing this object in S3
14597 --s3-sse-customer-key string To use SSE-C you may provide the secret encryption key used to encrypt/decrypt your data
14598 --s3-sse-customer-key-base64 string If using SSE-C you must provide the secret encryption key encoded in base64 format to encrypt/decrypt your data
14599 --s3-sse-customer-key-md5 string If using SSE-C you may provide the secret encryption key MD5 checksum (optional)
14600 --s3-sse-kms-key-id string If using KMS ID you must provide the ARN of Key
14601 --s3-storage-class string The storage class to use when storing new objects in S3
14602 --s3-upload-concurrency int Concurrency for multipart uploads (default 4)
14603 --s3-upload-cutoff SizeSuffix Cutoff for switching to chunked upload (default 200Mi)
14604 --s3-use-accelerate-endpoint If true use the AWS S3 accelerated endpoint
14605 --s3-use-multipart-etag Tristate Whether to use ETag in multipart uploads for verification (default unset)
14606 --s3-use-presigned-request Whether to use a presigned request or PutObject for single part uploads
14607 --s3-v2-auth If true use v2 authentication
14608 --s3-version-at Time Show file versions as they were at the specified time (default off)
14609 --s3-versions Include old versions in directory listings
14610 --seafile-2fa Two-factor authentication ('true' if the account has 2FA enabled)
14611 --seafile-create-library Should rclone create a library if it doesn't exist
14612 --seafile-encoding MultiEncoder The encoding for the backend (default Slash,DoubleQuote,BackSlash,Ctl,InvalidUtf8)
14613 --seafile-library string Name of the library
14614 --seafile-library-key string Library password (for encrypted libraries only) (obscured)
14615 --seafile-pass string Password (obscured)
14616 --seafile-url string URL of seafile host to connect to
14617 --seafile-user string User name (usually email address)
14618 --sftp-ask-password Allow asking for SFTP password when needed
14619 --sftp-chunk-size SizeSuffix Upload and download chunk size (default 32Ki)
14620 --sftp-concurrency int The maximum number of outstanding requests for one file (default 64)
14621 --sftp-disable-concurrent-reads If set don't use concurrent reads
14622 --sftp-disable-concurrent-writes If set don't use concurrent writes
14623 --sftp-disable-hashcheck Disable the execution of SSH commands to determine if remote file hashing is available
14624 --sftp-host string SSH host to connect to
14625 --sftp-idle-timeout Duration Max time before closing idle connections (default 1m0s)
14626 --sftp-key-file string Path to PEM-encoded private key file
14627 --sftp-key-file-pass string The passphrase to decrypt the PEM-encoded private key file (obscured)
14628 --sftp-key-pem string Raw PEM-encoded private key
14629 --sftp-key-use-agent When set forces the usage of the ssh-agent
14630 --sftp-known-hosts-file string Optional path to known_hosts file
14631 --sftp-md5sum-command string The command used to read md5 hashes
14632 --sftp-pass string SSH password, leave blank to use ssh-agent (obscured)
14633 --sftp-path-override string Override path used by SSH shell commands
14634 --sftp-port int SSH port number (default 22)
14635 --sftp-pubkey-file string Optional path to public key file
14636 --sftp-server-command string Specifies the path or command to run a sftp server on the remote host
14637 --sftp-set-env SpaceSepList Environment variables to pass to sftp and commands
14638 --sftp-set-modtime Set the modified time on the remote if set (default true)
14639 --sftp-sha1sum-command string The command used to read sha1 hashes
14640 --sftp-shell-type string The type of SSH shell on remote server, if any
14641 --sftp-skip-links Set to skip any symlinks and any other non regular files
14642 --sftp-subsystem string Specifies the SSH2 subsystem on the remote host (default "sftp")
14643 --sftp-use-fstat If set use fstat instead of stat
14644 --sftp-use-insecure-cipher Enable the use of insecure ciphers and key exchange methods
14645 --sftp-user string SSH username (default "$USER")
14646 --sharefile-chunk-size SizeSuffix Upload chunk size (default 64Mi)
14647 --sharefile-encoding MultiEncoder The encoding for the backend (default Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,BackSlash,Ctl,LeftSpace,LeftPeriod,RightSpace,RightPeriod,InvalidUtf8,Dot)
14648 --sharefile-endpoint string Endpoint for API calls
14649 --sharefile-root-folder-id string ID of the root folder
14650 --sharefile-upload-cutoff SizeSuffix Cutoff for switching to multipart upload (default 128Mi)
14651 --sia-api-password string Sia Daemon API Password (obscured)
14652 --sia-api-url string Sia daemon API URL, like http://sia.daemon.host:9980 (default "http://127.0.0.1:9980")
14653 --sia-encoding MultiEncoder The encoding for the backend (default Slash,Question,Hash,Percent,Del,Ctl,InvalidUtf8,Dot)
14654 --sia-user-agent string Siad User Agent (default "Sia-Agent")
14655 --skip-links Don't warn about skipped symlinks
14656 --smb-case-insensitive Whether the server is configured to be case-insensitive (default true)
14657 --smb-domain string Domain name for NTLM authentication (default "WORKGROUP")
14658 --smb-encoding MultiEncoder The encoding for the backend (default Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,BackSlash,Ctl,RightSpace,RightPeriod,InvalidUtf8,Dot)
14659 --smb-hide-special-share Hide special shares (e.g. print$) which users aren't supposed to access (default true)
14660 --smb-host string SMB server hostname to connect to
14661 --smb-idle-timeout Duration Max time before closing idle connections (default 1m0s)
14662 --smb-pass string SMB password (obscured)
14663 --smb-port int SMB port number (default 445)
14664 --smb-user string SMB username (default "$USER")
14665 --storj-access-grant string Access grant
14666 --storj-api-key string API key
14667 --storj-passphrase string Encryption passphrase
14668 --storj-provider string Choose an authentication method (default "existing")
14669 --storj-satellite-address string Satellite address (default "us-central-1.storj.io")
14670 --sugarsync-access-key-id string Sugarsync Access Key ID
14671 --sugarsync-app-id string Sugarsync App ID
14672 --sugarsync-authorization string Sugarsync authorization
14673 --sugarsync-authorization-expiry string Sugarsync authorization expiry
14674 --sugarsync-deleted-id string Sugarsync deleted folder id
14675 --sugarsync-encoding MultiEncoder The encoding for the backend (default Slash,Ctl,InvalidUtf8,Dot)
14676 --sugarsync-hard-delete Permanently delete files if true
14677 --sugarsync-private-access-key string Sugarsync Private Access Key
14678 --sugarsync-refresh-token string Sugarsync refresh token
14679 --sugarsync-root-id string Sugarsync root id
14680 --sugarsync-user string Sugarsync user
14681 --swift-application-credential-id string Application Credential ID (OS_APPLICATION_CREDENTIAL_ID)
14682 --swift-application-credential-name string Application Credential Name (OS_APPLICATION_CREDENTIAL_NAME)
14683 --swift-application-credential-secret string Application Credential Secret (OS_APPLICATION_CREDENTIAL_SECRET)
14684 --swift-auth string Authentication URL for server (OS_AUTH_URL)
14685 --swift-auth-token string Auth Token from alternate authentication - optional (OS_AUTH_TOKEN)
14686 --swift-auth-version int AuthVersion - optional - set to (1,2,3) if your auth URL has no version (ST_AUTH_VERSION)
14687 --swift-chunk-size SizeSuffix Above this size files will be chunked into a _segments container (default 5Gi)
14688 --swift-domain string User domain - optional (v3 auth) (OS_USER_DOMAIN_NAME)
14689 --swift-encoding MultiEncoder The encoding for the backend (default Slash,InvalidUtf8)
14690 --swift-endpoint-type string Endpoint type to choose from the service catalogue (OS_ENDPOINT_TYPE) (default "public")
14691 --swift-env-auth Get swift credentials from environment variables in standard OpenStack form
14692 --swift-key string API key or password (OS_PASSWORD)
14693 --swift-leave-parts-on-error If true avoid calling abort upload on a failure
14694 --swift-no-chunk Don't chunk files during streaming upload
14695 --swift-no-large-objects Disable support for static and dynamic large objects
14696 --swift-region string Region name - optional (OS_REGION_NAME)
14697 --swift-storage-policy string The storage policy to use when creating a new container
14698 --swift-storage-url string Storage URL - optional (OS_STORAGE_URL)
14699 --swift-tenant string Tenant name - optional for v1 auth, this or tenant_id required otherwise (OS_TENANT_NAME or OS_PROJECT_NAME)
14700 --swift-tenant-domain string Tenant domain - optional (v3 auth) (OS_PROJECT_DOMAIN_NAME)
14701 --swift-tenant-id string Tenant ID - optional for v1 auth, this or tenant required otherwise (OS_TENANT_ID)
14702 --swift-user string User name to log in (OS_USERNAME)
14703 --swift-user-id string User ID to log in - optional - most swift systems use user and leave this blank (v3 auth) (OS_USER_ID)
14704 --union-action-policy string Policy to choose upstream on ACTION category (default "epall")
14705 --union-cache-time int Cache time of usage and free space (in seconds) (default 120)
14706 --union-create-policy string Policy to choose upstream on CREATE category (default "epmfs")
14707 --union-min-free-space SizeSuffix Minimum viable free space for lfs/eplfs policies (default 1Gi)
14708 --union-search-policy string Policy to choose upstream on SEARCH category (default "ff")
14709 --union-upstreams string List of space separated upstreams
14710 --uptobox-access-token string Your access token
14711 --uptobox-encoding MultiEncoder The encoding for the backend (default Slash,LtGt,DoubleQuote,BackQuote,Del,Ctl,LeftSpace,InvalidUtf8,Dot)
14712 --webdav-bearer-token string Bearer token instead of user/pass (e.g. a Macaroon)
14713 --webdav-bearer-token-command string Command to run to get a bearer token
14714 --webdav-encoding string The encoding for the backend
14715 --webdav-headers CommaSepList Set HTTP headers for all transactions
14716 --webdav-pass string Password (obscured)
14717 --webdav-url string URL of http host to connect to
14718 --webdav-user string User name
14719 --webdav-vendor string Name of the WebDAV site/service/software you are using
14720 --yandex-auth-url string Auth server URL
14721 --yandex-client-id string OAuth Client Id
14722 --yandex-client-secret string OAuth Client Secret
14723 --yandex-encoding MultiEncoder The encoding for the backend (default Slash,Del,Ctl,InvalidUtf8,Dot)
14724 --yandex-hard-delete Delete files permanently rather than putting them into the trash
14725 --yandex-token string OAuth Access Token as a JSON blob
14726 --yandex-token-url string Token server url
14727 --zoho-auth-url string Auth server URL
14728 --zoho-client-id string OAuth Client Id
14729 --zoho-client-secret string OAuth Client Secret
14730 --zoho-encoding MultiEncoder The encoding for the backend (default Del,Ctl,InvalidUtf8)
14731 --zoho-region string Zoho region to connect to
14732 --zoho-token string OAuth Access Token as a JSON blob
14733 --zoho-token-url string Token server url
14734
14736 Introduction
14737 Docker 1.9 has added support for creating named volumes
14738 (https://docs.docker.com/storage/volumes/) via command-line interface
14739 (https://docs.docker.com/engine/reference/commandline/volume_create/)
14740 and mounting them in containers as a way to share data between them.
14741 Since Docker 1.10 you can create named volumes with Docker Compose
14742 (https://docs.docker.com/compose/) by descriptions in docker-com‐
14743 pose.yml (https://docs.docker.com/compose/compose-file/compose-file-
14744 v2/#volume-configuration-reference) files for use by container groups
14745 on a single host. As of Docker 1.12 volumes are supported by Docker
14746 Swarm (https://docs.docker.com/engine/swarm/key-concepts/) included
14747 with Docker Engine and created from descriptions in swarm compose v3
14748 (https://docs.docker.com/compose/compose-file/compose-file-v3/#volume-
14749 configuration-reference) files for use with swarm stacks across multi‐
14750 ple cluster nodes.
14751 Docker Volume Plugins (https://docs.docker.com/engine/extend/plug‐
14753 ins_volume/) augment the default local volume driver included in Docker
14754 with stateful volumes shared across containers and hosts. Unlike local
14755 volumes, your data will not be deleted when such volume is removed.
14756 Plugins can run managed by the docker daemon, as a native system ser‐
14757 vice (under systemd, sysv or upstart) or as a standalone executable.
14758 Rclone can run as docker volume plugin in all these modes. It inter‐
14759 acts with the local docker daemon via plugin API (https://docs.dock‐
14760 er.com/engine/extend/plugin_api/) and handles mounting of remote file
14761 systems into docker containers so it must run on the same host as the
14762 docker daemon or on every Swarm node.
14763 Getting started
14765 In the first example we will use the SFTP (https://rclone.org/sftp/)
14766 rclone volume with Docker engine on a standalone Ubuntu machine.
14767 Start from installing Docker (https://docs.docker.com/engine/install/)
14769 on the host.
14770 The FUSE driver is a prerequisite for rclone mounting and should be in‐
14772 stalled on host:
14773 sudo apt-get -y install fuse
14775 Create two directories required by rclone docker plugin:
14777 sudo mkdir -p /var/lib/docker-plugins/rclone/config
14779 sudo mkdir -p /var/lib/docker-plugins/rclone/cache
14780 Install the managed rclone docker plugin for your architecture (here
14782 amd64):
14783 docker plugin install rclone/docker-volume-rclone:amd64 args="-v" --alias rclone --grant-all-permissions
14785 docker plugin list
14786 Create your SFTP volume (https://rclone.org/sftp/#standard-options):
14788 docker volume create firstvolume -d rclone -o type=sftp -o sftp-host=_hostname_ -o sftp-user=_username_ -o sftp-pass=_password_ -o allow-other=true
14790 Note that since all options are static, you don't even have to run
14792 rclone config or create the rclone.conf file (but the config directory
14793 should still be present). In the simplest case you can use localhost
14794 as hostname and your SSH credentials as username and password. You can
14795 also change the remote path to your home directory on the host, for ex‐
14796 ample -o path=/home/username.
14797 Time to create a test container and mount the volume into it:
14799 docker run --rm -it -v firstvolume:/mnt --workdir /mnt ubuntu:latest bash
14801 If all goes well, you will enter the new container and change right to
14803 the mounted SFTP remote. You can type ls to list the mounted directory
14804 or otherwise play with it. Type exit when you are done. The container
14805 will stop but the volume will stay, ready to be reused. When it's not
14806 needed anymore, remove it:
14807 docker volume list
14809 docker volume remove firstvolume
14810 Now let us try something more elaborate: Google Drive
14812 (https://rclone.org/drive/) volume on multi-node Docker Swarm.
14813 You should start from installing Docker and FUSE, creating plugin di‐
14815 rectories and installing rclone plugin on every swarm node. Then setup
14816 the Swarm (https://docs.docker.com/engine/swarm/swarm-mode/).
14817 Google Drive volumes need an access token which can be setup via web
14819 browser and will be periodically renewed by rclone. The managed plugin
14820 cannot run a browser so we will use a technique similar to the rclone
14821 setup on a headless box (https://rclone.org/remote_setup/).
14822 Run rclone config (https://rclone.org/commands/rclone_config_create/)
14824 on another machine equipped with web browser and graphical user inter‐
14825 face. Create the Google Drive remote (https://rclone.org/drive/#stan‐
14826 dard-options). When done, transfer the resulting rclone.conf to the
14827 Swarm cluster and save as /var/lib/docker-plugins/rclone/con‐
14828 fig/rclone.conf on every node. By default this location is accessible
14829 only to the root user so you will need appropriate privileges. The re‐
14830 sulting config will look like this:
14831 [gdrive]
14833 type = drive
14834 scope = drive
14835 drive_id = 1234567...
14836 root_folder_id = 0Abcd...
14837 token = {"access_token":...}
14838 Now create the file named example.yml with a swarm stack description
14840 like this:
14841 version: '3'
14843 services:
14844 heimdall:
14845 image: linuxserver/heimdall:latest
14846 ports: [8080:80]
14847 volumes: [configdata:/config]
14848 volumes:
14849 configdata:
14850 driver: rclone
14851 driver_opts:
14852 remote: 'gdrive:heimdall'
14853 allow_other: 'true'
14854 vfs_cache_mode: full
14855 poll_interval: 0
14856 and run the stack:
14858 docker stack deploy example -c ./example.yml
14860 After a few seconds docker will spread the parsed stack description
14862 over cluster, create the example_heimdall service on port 8080, run
14863 service containers on one or more cluster nodes and request the exam‐
14864 ple_configdata volume from rclone plugins on the node hosts. You can
14865 use the following commands to confirm results:
14866 docker service ls
14868 docker service ps example_heimdall
14869 docker volume ls
14870 Point your browser to http://cluster.host.address:8080 and play with
14872 the service. Stop it with docker stack remove example when you are
14873 done. Note that the example_configdata volume(s) created on demand at
14874 the cluster nodes will not be automatically removed together with the
14875 stack but stay for future reuse. You can remove them manually by in‐
14876 voking the docker volume remove example_configdata command on every
14877 node.
14878 Creating Volumes via CLI
14880 Volumes can be created with docker volume create (https://docs.dock‐
14881 er.com/engine/reference/commandline/volume_create/). Here are a few
14882 examples:
14883 docker volume create vol1 -d rclone -o remote=storj: -o vfs-cache-mode=full
14885 docker volume create vol2 -d rclone -o remote=:storj,access_grant=xxx:heimdall
14886 docker volume create vol3 -d rclone -o type=storj -o path=heimdall -o storj-access-grant=xxx -o poll-interval=0
14887 Note the -d rclone flag that tells docker to request volume from the
14889 rclone driver. This works even if you installed managed driver by its
14890 full name rclone/docker-volume-rclone because you provided the --alias
14891 rclone option.
14892 Volumes can be inspected as follows:
14894 docker volume list
14896 docker volume inspect vol1
14897 Volume Configuration
14899 Rclone flags and volume options are set via the -o flag to the docker
14900 volume create command. They include backend-specific parameters as
14901 well as mount and VFS options. Also there are a few special -o op‐
14902 tions: remote, fs, type, path, mount-type and persist.
14903 remote determines an existing remote name from the config file, with
14905 trailing colon and optionally with a remote path. See the full syntax
14906 in the rclone documentation (https://rclone.org/docs/#syntax-of-remote-
14907 paths). This option can be aliased as fs to prevent confusion with the
14908 remote parameter of such backends as crypt or alias.
14909 The remote=:backend:dir/subdir syntax can be used to create on-the-fly
14911 (config-less) remotes (https://rclone.org/docs/#backend-path-to-dir),
14912 while the type and path options provide a simpler alternative for this.
14913 Using two split options
14914 -o type=backend -o path=dir/subdir
14916 is equivalent to the combined syntax
14918 -o remote=:backend:dir/subdir
14920 but is arguably easier to parameterize in scripts. The path part is
14922 optional.
14923 Mount and VFS options (https://rclone.org/commands/rclone_serve_dock‐
14925 er/#options) as well as backend parameters
14926 (https://rclone.org/flags/#backend-flags) are named like their twin
14927 command-line flags without the -- CLI prefix. Optionally you can use
14928 underscores instead of dashes in option names. For example, --vfs-
14929 cache-mode full becomes -o vfs-cache-mode=full or -o
14930 vfs_cache_mode=full. Boolean CLI flags without value will gain the
14931 true value, e.g. --allow-other becomes -o allow-other=true or -o al‐
14932 low_other=true.
14933 Please note that you can provide parameters only for the backend imme‐
14935 diately referenced by the backend type of mounted remote. If this is a
14936 wrapping backend like alias, chunker or crypt, you cannot provide op‐
14937 tions for the referred to remote or backend. This limitation is im‐
14938 posed by the rclone connection string parser. The only workaround is
14939 to feed plugin with rclone.conf or configure plugin arguments (see be‐
14940 low).
14941 Special Volume Options
14943 mount-type determines the mount method and in general can be one of:
14944 mount, cmount, or mount2. This can be aliased as mount_type. It
14945 should be noted that the managed rclone docker plugin currently does
14946 not support the cmount method and mount2 is rarely needed. This option
14947 defaults to the first found method, which is usually mount so you gen‐
14948 erally won't need it.
14949 persist is a reserved boolean (true/false) option. In future it will
14951 allow to persist on-the-fly remotes in the plugin rclone.conf file.
14952 Connection Strings
14954 The remote value can be extended with connection strings
14955 (https://rclone.org/docs/#connection-strings) as an alternative way to
14956 supply backend parameters. This is equivalent to the -o backend op‐
14957 tions with one syntactic difference. Inside connection string the
14958 backend prefix must be dropped from parameter names but in the -o
14959 param=value array it must be present. For instance, compare the fol‐
14960 lowing option array
14961 -o remote=:sftp:/home -o sftp-host=localhost
14963 with equivalent connection string:
14965 -o remote=:sftp,host=localhost:/home
14967 This difference exists because flag options -o key=val include not only
14969 backend parameters but also mount/VFS flags and possibly other set‐
14970 tings. Also it allows to discriminate the remote option from the
14971 crypt-remote (or similarly named backend parameters) and arguably sim‐
14972 plifies scripting due to clearer value substitution.
14973 Using with Swarm or Compose
14975 Both Docker Swarm and Docker Compose use YAML
14976 (http://yaml.org/spec/1.2/spec.html)-formatted text files to describe
14977 groups (stacks) of containers, their properties, networks and volumes.
14978 Compose uses the compose v2 (https://docs.docker.com/compose/compose-
14979 file/compose-file-v2/#volume-configuration-reference) format, Swarm us‐
14980 es the compose v3 (https://docs.docker.com/compose/compose-file/com‐
14981 pose-file-v3/#volume-configuration-reference) format. They are mostly
14982 similar, differences are explained in the docker documentation
14983 (https://docs.docker.com/compose/compose-file/compose-versioning/#up‐
14984 grading).
14985 Volumes are described by the children of the top-level volumes: node.
14987 Each of them should be named after its volume and have at least two el‐
14988 ements, the self-explanatory driver: rclone value and the driver_opts:
14989 structure playing the same role as -o key=val CLI flags:
14990 volumes:
14992 volume_name_1:
14993 driver: rclone
14994 driver_opts:
14995 remote: 'gdrive:'
14996 allow_other: 'true'
14997 vfs_cache_mode: full
14998 token: '{"type": "borrower", "expires": "2021-12-31"}'
14999 poll_interval: 0
15000 Notice a few important details: - YAML prefers _ in option names in‐
15002 stead of -. - YAML treats single and double quotes interchangeably.
15003 Simple strings and integers can be left unquoted. - Boolean values
15004 must be quoted like 'true' or "false" because these two words are re‐
15005 served by YAML. - The filesystem string is keyed with remote (or with
15006 fs). Normally you can omit quotes here, but if the string ends with
15007 colon, you must quote it like remote: "storage_box:". - YAML is picky
15008 about surrounding braces in values as this is in fact another syntax
15009 for key/value mappings (http://yaml.org/spec/1.2/spec.html#id2790832).
15010 For example, JSON access tokens usually contain double quotes and sur‐
15011 rounding braces, so you must put them in single quotes.
15012 Installing as Managed Plugin
15014 Docker daemon can install plugins from an image registry and run them
15015 managed. We maintain the docker-volume-rclone (https://hub.dock‐
15016 er.com/p/rclone/docker-volume-rclone/) plugin image on Docker Hub
15017 (https://hub.docker.com).
15018 Rclone volume plugin requires Docker Engine >= 19.03.15
15020 The plugin requires presence of two directories on the host before it
15022 can be installed. Note that plugin will not create them automatically.
15023 By default they must exist on host at the following locations (though
15024 you can tweak the paths): - /var/lib/docker-plugins/rclone/config is
15025 reserved for the rclone.conf config file and must exist even if it's
15026 empty and the config file is not present. - /var/lib/docker-plug‐
15027 ins/rclone/cache holds the plugin state file as well as optional VFS
15028 caches.
15029 You can install managed plugin (https://docs.docker.com/engine/refer‐
15031 ence/commandline/plugin_install/) with default settings as follows:
15032 docker plugin install rclone/docker-volume-rclone:amd64 --grant-all-permissions --alias rclone
15034 The :amd64 part of the image specification after colon is called a tag.
15036 Usually you will want to install the latest plugin for your architec‐
15037 ture. In this case the tag will just name it, like amd64 above. The
15038 following plugin architectures are currently available: - amd64 - arm64
15039 - arm-v7
15040 Sometimes you might want a concrete plugin version, not the latest one.
15042 Then you should use image tag in the form :ARCHITECTURE-VERSION. For
15043 example, to install plugin version v1.56.2 on architecture arm64 you
15044 will use tag arm64-1.56.2 (note the removed v) so the full image speci‐
15045 fication becomes rclone/docker-volume-rclone:arm64-1.56.2.
15046 We also provide the latest plugin tag, but since docker does not sup‐
15048 port multi-architecture plugins as of the time of this writing, this
15049 tag is currently an alias for amd64. By convention the latest tag is
15050 the default one and can be omitted, thus both rclone/docker-volume-
15051 rclone:latest and just rclone/docker-volume-rclone will refer to the
15052 latest plugin release for the amd64 platform.
15053 Also the amd64 part can be omitted from the versioned rclone plugin
15055 tags. For example, rclone image reference rclone/docker-volume-
15056 rclone:amd64-1.56.2 can be abbreviated as rclone/docker-volume-
15057 rclone:1.56.2 for convenience. However, for non-intel architectures
15058 you still have to use the full tag as amd64 or latest will fail to
15059 start.
15060 Managed plugin is in fact a special container running in a namespace
15062 separate from normal docker containers. Inside it runs the rclone
15063 serve docker command. The config and cache directories are bind-mount‐
15064 ed into the container at start. The docker daemon connects to a unix
15065 socket created by the command inside the container. The command cre‐
15066 ates on-demand remote mounts right inside, then docker machinery propa‐
15067 gates them through kernel mount namespaces and bind-mounts into re‐
15068 questing user containers.
15069 You can tweak a few plugin settings after installation when it's dis‐
15071 abled (not in use), for instance:
15072 docker plugin disable rclone
15074 docker plugin set rclone RCLONE_VERBOSE=2 config=/etc/rclone args="--vfs-cache-mode=writes --allow-other"
15075 docker plugin enable rclone
15076 docker plugin inspect rclone
15077 Note that if docker refuses to disable the plugin, you should find and
15079 remove all active volumes connected with it as well as containers and
15080 swarm services that use them. This is rather tedious so please care‐
15081 fully plan in advance.
15082 You can tweak the following settings: args, config, cache, HTTP_PROXY,
15084 HTTPS_PROXY, NO_PROXY and RCLONE_VERBOSE. It's your task to keep plug‐
15085 in settings in sync across swarm cluster nodes.
15086 args sets command-line arguments for the rclone serve docker command
15088 (none by default). Arguments should be separated by space so you will
15089 normally want to put them in quotes on the docker plugin set
15090 (https://docs.docker.com/engine/reference/commandline/plugin_set/) com‐
15091 mand line. Both serve docker flags (https://rclone.org/com‐
15092 mands/rclone_serve_docker/#options) and generic rclone flags
15093 (https://rclone.org/flags/) are supported, including backend parameters
15094 that will be used as defaults for volume creation. Note that plugin
15095 will fail (due to this docker bug (https://github.com/moby/mo‐
15096 by/blob/v20.10.7/plugin/v2/plugin.go#L195)) if the args value is empty.
15097 Use e.g. args="-v" as a workaround.
15098 config=/host/dir sets alternative host location for the config directo‐
15100 ry. Plugin will look for rclone.conf here. It's not an error if the
15101 config file is not present but the directory must exist. Please note
15102 that plugin can periodically rewrite the config file, for example when
15103 it renews storage access tokens. Keep this in mind and try to avoid
15104 races between the plugin and other instances of rclone on the host that
15105 might try to change the config simultaneously resulting in corrupted
15106 rclone.conf. You can also put stuff like private key files for SFTP
15107 remotes in this directory. Just note that it's bind-mounted inside the
15108 plugin container at the predefined path /data/config. For example, if
15109 your key file is named sftp-box1.key on the host, the corresponding
15110 volume config option should read -o sftp-key-file=/data/config/sftp-
15111 box1.key.
15112 cache=/host/dir sets alternative host location for the cache directory.
15114 The plugin will keep VFS caches here. Also it will create and maintain
15115 the docker-plugin.state file in this directory. When the plugin is
15116 restarted or reinstalled, it will look in this file to recreate any
15117 volumes that existed previously. However, they will not be re-mounted
15118 into consuming containers after restart. Usually this is not a problem
15119 as the docker daemon normally will restart affected user containers af‐
15120 ter failures, daemon restarts or host reboots.
15121 RCLONE_VERBOSE sets plugin verbosity from 0 (errors only, by default)
15123 to 2 (debugging). Verbosity can be also tweaked via args="-v [-v]
15124 ...". Since arguments are more generic, you will rarely need this set‐
15125 ting. The plugin output by default feeds the docker daemon log on lo‐
15126 cal host. Log entries are reflected as errors in the docker log but
15127 retain their actual level assigned by rclone in the encapsulated mes‐
15128 sage string.
15129 HTTP_PROXY, HTTPS_PROXY, NO_PROXY customize the plugin proxy settings.
15131 You can set custom plugin options right when you install it, in one go:
15133 docker plugin remove rclone
15135 docker plugin install rclone/docker-volume-rclone:amd64 \
15136 --alias rclone --grant-all-permissions \
15137 args="-v --allow-other" config=/etc/rclone
15138 docker plugin inspect rclone
15139 Healthchecks
15141 The docker plugin volume protocol doesn't provide a way for plugins to
15142 inform the docker daemon that a volume is (un-)available. As a work‐
15143 around you can setup a healthcheck to verify that the mount is respond‐
15144 ing, for example:
15145 services:
15147 my_service:
15148 image: my_image
15149 healthcheck:
15150 test: ls /path/to/rclone/mount || exit 1
15151 interval: 1m
15152 timeout: 15s
15153 retries: 3
15154 start_period: 15s
15155 Running Plugin under Systemd
15157 In most cases you should prefer managed mode. Moreover, MacOS and Win‐
15158 dows do not support native Docker plugins. Please use managed mode on
15159 these systems. Proceed further only if you are on Linux.
15160 First, install rclone (https://rclone.org/install/). You can just run
15162 it (type rclone serve docker and hit enter) for the test.
15163 Install FUSE:
15165 sudo apt-get -y install fuse
15167 Download two systemd configuration files: docker-volume-rclone.service
15169 (https://raw.githubusercontent.com/rclone/rclone/master/contrib/docker-
15170 plugin/systemd/docker-volume-rclone.service) and docker-volume-
15171 rclone.socket (https://raw.githubusercontent.com/rclone/rclone/mas‐
15172 ter/contrib/docker-plugin/systemd/docker-volume-rclone.socket).
15173 Put them to the /etc/systemd/system/ directory:
15175 cp docker-volume-plugin.service /etc/systemd/system/
15177 cp docker-volume-plugin.socket /etc/systemd/system/
15178 Please note that all commands in this section must be run as root but
15180 we omit sudo prefix for brevity. Now create directories required by
15181 the service:
15182 mkdir -p /var/lib/docker-volumes/rclone
15184 mkdir -p /var/lib/docker-plugins/rclone/config
15185 mkdir -p /var/lib/docker-plugins/rclone/cache
15186 Run the docker plugin service in the socket activated mode:
15188 systemctl daemon-reload
15190 systemctl start docker-volume-rclone.service
15191 systemctl enable docker-volume-rclone.socket
15192 systemctl start docker-volume-rclone.socket
15193 systemctl restart docker
15194 Or run the service directly: - run systemctl daemon-reload to let sys‐
15196 temd pick up new config - run systemctl enable docker-volume-
15197 rclone.service to make the new service start automatically when you
15198 power on your machine. - run systemctl start docker-volume-rclone.ser‐
15199 vice to start the service now. - run systemctl restart docker to
15200 restart docker daemon and let it detect the new plugin socket. Note
15201 that this step is not needed in managed mode where docker knows about
15202 plugin state changes.
15203 The two methods are equivalent from the user perspective, but I person‐
15205 ally prefer socket activation.
15206 Troubleshooting
15208 You can see managed plugin settings (https://docs.docker.com/engine/ex‐
15209 tend/#debugging-plugins) with
15210 docker plugin list
15212 docker plugin inspect rclone
15213 Note that docker (including latest 20.10.7) will not show actual values
15215 of args, just the defaults.
15216 Use journalctl --unit docker to see managed plugin output as part of
15218 the docker daemon log. Note that docker reflects plugin lines as er‐
15219 rors but their actual level can be seen from encapsulated message
15220 string.
15221 You will usually install the latest version of managed plugin for your
15223 platform. Use the following commands to print the actual installed
15224 version:
15225 PLUGID=$(docker plugin list --no-trunc | awk '/rclone/{print$1}')
15227 sudo runc --root /run/docker/runtime-runc/plugins.moby exec $PLUGID rclone version
15228 You can even use runc to run shell inside the plugin container:
15230 sudo runc --root /run/docker/runtime-runc/plugins.moby exec --tty $PLUGID bash
15232 Also you can use curl to check the plugin socket connectivity:
15234 docker plugin list --no-trunc
15236 PLUGID=123abc...
15237 sudo curl -H Content-Type:application/json -XPOST -d {} --unix-socket /run/docker/plugins/$PLUGID/rclone.sock http://localhost/Plugin.Activate
15238 though this is rarely needed.
15240 Caveats
15242 Finally I'd like to mention a caveat with updating volume settings.
15243 Docker CLI does not have a dedicated command like docker volume update.
15244 It may be tempting to invoke docker volume create with updated options
15245 on existing volume, but there is a gotcha. The command will do noth‐
15246 ing, it won't even return an error. I hope that docker maintainers
15247 will fix this some day. In the meantime be aware that you must remove
15248 your volume before recreating it with new settings:
15249 docker volume remove my_vol
15251 docker volume create my_vol -d rclone -o opt1=new_val1 ...
15252 and verify that settings did update:
15254 docker volume list
15256 docker volume inspect my_vol
15257 If docker refuses to remove the volume, you should find containers or
15259 swarm services that use it and stop them first.
15260 Getting started
15262 • Install rclone (https://rclone.org/install/) and setup your remotes.
15263 • Bisync will create its working directory at ~/.cache/rclone/bisync on
15265 Linux or C:\Users\MyLogin\AppData\Local\rclone\bisync on Windows.
15266 Make sure that this location is writable.
15267 • Run bisync with the --resync flag, specifying the paths to the local
15269 and remote sync directory roots.
15270 • For successive sync runs, leave off the --resync flag.
15272 • Consider using a filters file for excluding unnecessary files and di‐
15274 rectories from the sync.
15275 • Consider setting up the --check-access feature for safety.
15277 • On Linux, consider setting up a crontab entry. bisync can safely run
15279 in concurrent cron jobs thanks to lock files it maintains.
15280 Here is a typical run log (with timestamps removed for clarity):
15282 rclone bisync /testdir/path1/ /testdir/path2/ --verbose
15284 INFO : Synching Path1 "/testdir/path1/" with Path2 "/testdir/path2/"
15285 INFO : Path1 checking for diffs
15286 INFO : - Path1 File is new - file11.txt
15287 INFO : - Path1 File is newer - file2.txt
15288 INFO : - Path1 File is newer - file5.txt
15289 INFO : - Path1 File is newer - file7.txt
15290 INFO : - Path1 File was deleted - file4.txt
15291 INFO : - Path1 File was deleted - file6.txt
15292 INFO : - Path1 File was deleted - file8.txt
15293 INFO : Path1: 7 changes: 1 new, 3 newer, 0 older, 3 deleted
15294 INFO : Path2 checking for diffs
15295 INFO : - Path2 File is new - file10.txt
15296 INFO : - Path2 File is newer - file1.txt
15297 INFO : - Path2 File is newer - file5.txt
15298 INFO : - Path2 File is newer - file6.txt
15299 INFO : - Path2 File was deleted - file3.txt
15300 INFO : - Path2 File was deleted - file7.txt
15301 INFO : - Path2 File was deleted - file8.txt
15302 INFO : Path2: 7 changes: 1 new, 3 newer, 0 older, 3 deleted
15303 INFO : Applying changes
15304 INFO : - Path1 Queue copy to Path2 - /testdir/path2/file11.txt
15305 INFO : - Path1 Queue copy to Path2 - /testdir/path2/file2.txt
15306 INFO : - Path2 Queue delete - /testdir/path2/file4.txt
15307 NOTICE: - WARNING New or changed in both paths - file5.txt
15308 NOTICE: - Path1 Renaming Path1 copy - /testdir/path1/file5.txt..path1
15309 NOTICE: - Path1 Queue copy to Path2 - /testdir/path2/file5.txt..path1
15310 NOTICE: - Path2 Renaming Path2 copy - /testdir/path2/file5.txt..path2
15311 NOTICE: - Path2 Queue copy to Path1 - /testdir/path1/file5.txt..path2
15312 INFO : - Path2 Queue copy to Path1 - /testdir/path1/file6.txt
15313 INFO : - Path1 Queue copy to Path2 - /testdir/path2/file7.txt
15314 INFO : - Path2 Queue copy to Path1 - /testdir/path1/file1.txt
15315 INFO : - Path2 Queue copy to Path1 - /testdir/path1/file10.txt
15316 INFO : - Path1 Queue delete - /testdir/path1/file3.txt
15317 INFO : - Path2 Do queued copies to - Path1
15318 INFO : - Path1 Do queued copies to - Path2
15319 INFO : - Do queued deletes on - Path1
15320 INFO : - Do queued deletes on - Path2
15321 INFO : Updating listings
15322 INFO : Validating listings for Path1 "/testdir/path1/" vs Path2 "/testdir/path2/"
15323 INFO : Bisync successful
15324 Command line syntax
15326 $ rclone bisync --help
15327 Usage:
15328 rclone bisync remote1:path1 remote2:path2 [flags]
15329 Positional arguments:
15331 Path1, Path2 Local path, or remote storage with ':' plus optional path.
15332 Type 'rclone listremotes' for list of configured remotes.
15333 Optional Flags:
15335 --check-access Ensure expected `RCLONE_TEST` files are found on
15336 both Path1 and Path2 filesystems, else abort.
15337 --check-filename FILENAME Filename for `--check-access` (default: `RCLONE_TEST`)
15338 --check-sync CHOICE Controls comparison of final listings:
15339 `true | false | only` (default: true)
15340 If set to `only`, bisync will only compare listings
15341 from the last run but skip actual sync.
15342 --filters-file PATH Read filtering patterns from a file
15343 --max-delete PERCENT Safety check on maximum percentage of deleted files allowed.
15344 If exceeded, the bisync run will abort. (default: 50%)
15345 --force Bypass `--max-delete` safety check and run the sync.
15346 Consider using with `--verbose`
15347 --remove-empty-dirs Remove empty directories at the final cleanup step.
15348 -1, --resync Performs the resync run.
15349 Warning: Path1 files may overwrite Path2 versions.
15350 Consider using `--verbose` or `--dry-run` first.
15351 --localtime Use local time in listings (default: UTC)
15352 --no-cleanup Retain working files (useful for troubleshooting and testing).
15353 --workdir PATH Use custom working directory (useful for testing).
15354 (default: `~/.cache/rclone/bisync`)
15355 -n, --dry-run Go through the motions - No files are copied/deleted.
15356 -v, --verbose Increases logging verbosity.
15357 May be specified more than once for more details.
15358 -h, --help help for bisync
15359 Arbitrary rclone flags may be specified on the bisync command line
15361 (https://rclone.org/commands/rclone_bisync/), for example rclone bisync
15362 ./testdir/path1/ gdrive:testdir/path2/ --drive-skip-gdocs -v -v --time‐
15363 out 10s Note that interactions of various rclone flags with bisync
15364 process flow has not been fully tested yet.
15365 Paths
15367 Path1 and Path2 arguments may be references to any mix of local direc‐
15368 tory paths (absolute or relative), UNC paths (//server/share/path),
15369 Windows drive paths (with a drive letter and :) or configured remotes
15370 (https://rclone.org/docs/#syntax-of-remote-paths) with optional subdi‐
15371 rectory paths. Cloud references are distinguished by having a : in the
15372 argument (see Windows support below).
15373 Path1 and Path2 are treated equally, in that neither has priority for
15375 file changes, and access efficiency does not change whether a remote is
15376 on Path1 or Path2.
15377 The listings in bisync working directory (default:
15379 ~/.cache/rclone/bisync) are named based on the Path1 and Path2 argu‐
15380 ments so that separate syncs to individual directories within the tree
15381 may be set up, e.g.: path_to_local_tree..dropbox_subdir.lst.
15382 Any empty directories after the sync on both the Path1 and Path2
15384 filesystems are not deleted by default. If the --remove-empty-dirs
15385 flag is specified, then both paths will have any empty directories
15386 purged as the last step in the process.
15387 Command-line flags
15389 --resync
15390 This will effectively make both Path1 and Path2 filesystems contain a
15391 matching superset of all files. Path2 files that do not exist in Path1
15392 will be copied to Path1, and the process will then sync the Path1 tree
15393 to Path2.
15394 The base directories on the both Path1 and Path2 filesystems must exist
15396 or bisync will fail. This is required for safety - that bisync can
15397 verify that both paths are valid.
15398 When using --resync a newer version of a file on the Path2 filesystem
15400 will be overwritten by the Path1 filesystem version. Carefully evalu‐
15401 ate deltas using --dry-run (https://rclone.org/flags/#non-backend-
15402 flags).
15403 For a resync run, one of the paths may be empty (no files in the path
15405 tree). The resync run should result in files on both paths, else a
15406 normal non-resync run will fail.
15407 For a non-resync run, either path being empty (no files in the tree)
15409 fails with Empty current PathN listing. Cannot sync to an empty direc‐
15410 tory: X.pathN.lst This is a safety check that an unexpected empty path
15411 does not result in deleting everything in the other path.
15412 --check-access
15414 Access check files are an additional safety measure against data loss.
15415 bisync will ensure it can find matching RCLONE_TEST files in the same
15416 places in the Path1 and Path2 filesystems. Time stamps and file con‐
15417 tents are not important, just the names and locations. Place one or
15418 more RCLONE_TEST files in the Path1 or Path2 filesystem and then do ei‐
15419 ther a run without --check-access or a --resync to set matching files
15420 on both filesystems. If you have symbolic links in your sync tree it
15421 is recommended to place RCLONE_TEST files in the linked-to directory
15422 tree to protect against bisync assuming a bunch of deleted files if the
15423 linked-to tree should not be accessible. Also see the --check-filename
15424 flag.
15425 --max-delete
15427 As a safety check, if greater than the --max-delete percent of files
15428 were deleted on either the Path1 or Path2 filesystem, then bisync will
15429 abort with a warning message, without making any changes. The default
15430 --max-delete is 50%. One way to trigger this limit is to rename a di‐
15431 rectory that contains more than half of your files. This will appear
15432 to bisync as a bunch of deleted files and a bunch of new files. This
15433 safety check is intended to block bisync from deleting all of the files
15434 on both filesystems due to a temporary network access issue, or if the
15435 user had inadvertently deleted the files on one side or the other. To
15436 force the sync either set a different delete percentage limit, e.g.
15437 --max-delete 75 (allows up to 75% deletion), or use --force to bypass
15438 the check.
15439 Also see the all files changed check.
15441 --filters-file
15443 By using rclone filter features you can exclude file types or directory
15444 sub-trees from the sync. See the bisync filters section and generic
15445 --filter-from (https://rclone.org/filtering/#filter-from-read-filter‐
15446 ing-patterns-from-a-file) documentation. An example filters file con‐
15447 tains filters for non-allowed files for synching with Dropbox.
15448 If you make changes to your filters file then bisync requires a run
15450 with --resync. This is a safety feature, which avoids existing files
15451 on the Path1 and/or Path2 side from seeming to disappear from view
15452 (since they are excluded in the new listings), which would fool bisync
15453 into seeing them as deleted (as compared to the prior run listings),
15454 and then bisync would proceed to delete them for real.
15455 To block this from happening bisync calculates an MD5 hash of the fil‐
15457 ters file and stores the hash in a .md5 file in the same place as your
15458 filters file. On the next runs with --filters-file set, bisync re-cal‐
15459 culates the MD5 hash of the current filters file and compares it to the
15460 hash stored in .md5 file. If they don't match the run aborts with a
15461 critical error and thus forces you to do a --resync, likely avoiding a
15462 disaster.
15463 --check-sync
15465 Enabled by default, the check-sync function checks that all of the same
15466 files exist in both the Path1 and Path2 history listings. This check-
15467 sync integrity check is performed at the end of the sync run by de‐
15468 fault. Any untrapped failing copy/deletes between the two paths might
15469 result in differences between the two listings and in the untracked
15470 file content differences between the two paths. A resync run would
15471 correct the error.
15472 Note that the default-enabled integrity check locally executes a load
15474 of both the final Path1 and Path2 listings, and thus adds to the run
15475 time of a sync. Using --check-sync=false will disable it and may sig‐
15476 nificantly reduce the sync run times for very large numbers of files.
15477 The check may be run manually with --check-sync=only. It runs only the
15479 integrity check and terminates without actually synching.
15480 Operation
15482 Runtime flow details
15483 bisync retains the listings of the Path1 and Path2 filesystems from the
15484 prior run. On each successive run it will:
15485 • list files on path1 and path2, and check for changes on each side.
15487 Changes include New, Newer, Older, and Deleted files.
15488 • Propagate changes on path1 to path2, and vice-versa.
15490 Safety measures
15492 • Lock file prevents multiple simultaneous runs when taking a while.
15493 This can be particularly useful if bisync is run by cron scheduler.
15494 • Handle change conflicts non-destructively by creating ..path1 and
15496 ..path2 file versions.
15497 • File system access health check using RCLONE_TEST files (see the
15499 --check-access flag).
15500 • Abort on excessive deletes - protects against a failed listing being
15502 interpreted as all the files were deleted. See the --max-delete and
15503 --force flags.
15504 • If something evil happens, bisync goes into a safe state to block
15506 damage by later runs. (See Error Handling)
15507 Normal sync checks
15509 Type Description Result Implementation
15510 ────────────────────────────────────────────────────────────────────────────────
15511 Path2 File is new on Path2, does Path2 version rclone copy Path2
15512 new not exist on Path1 survives to Path1
15513 Path2 File is newer on Path2, un‐ Path2 version rclone copy Path2
15514 newer changed on Path1 survives to Path1
15515 Path2 File is deleted on Path2, File is deleted rclone delete
15516 deleted unchanged on Path1 Path1
15517 Path1 File is new on Path1, does Path1 version rclone copy Path1
15518 new not exist on Path2 survives to Path2
15519 Path1 File is newer on Path1, un‐ Path1 version rclone copy Path1
15520 newer changed on Path2 survives to Path2
15521 Path1 File is older on Path1, un‐ Path1 version rclone copy Path1
15522 older changed on Path2 survives to Path2
15523 Path2 File is older on Path2, un‐ Path2 version rclone copy Path2
15524 older changed on Path1 survives to Path1
15525 Path1 File no longer exists on File is deleted rclone delete
15526 deleted Path1 Path2
15527 Unusual sync checks
15529 Type Description Result Implementa‐
15530 tion
15531 ────────────────────────────────────────────────────────────────────────────────
15532 Path1 new AND File is new on Path1 Files renamed to rclone copy
15533 Path2 new AND new on Path2 _Path1 and _Path2 _Path2 file
15534 to Path1,
15535 rclone copy
15536 _Path1 file
15537 to Path2
15538 Path2 newer AND File is newer on Files renamed to rclone copy
15539 Path1 changed Path2 AND also _Path1 and _Path2 _Path2 file
15540 changed (newer/old‐ to Path1,
15541 er/size) on Path1 rclone copy
15542 _Path1 file
15543 to Path2
15544 Path2 newer AND File is newer on Path2 version sur‐ rclone copy
15545 Path1 deleted Path2 AND also delet‐ vives Path2 to
15546 ed on Path1 Path1
15547 Path2 deleted AND File is deleted on Path1 version sur‐ rclone copy
15548 Path1 changed Path2 AND changed vives Path1 to
15549 (newer/older/size) on Path2
15550 Path1
15551 Path1 deleted AND File is deleted on Path2 version sur‐ rclone copy
15552 Path2 changed Path1 AND changed vives Path2 to
15553 (newer/older/size) on Path1
15554 Path2
15555 All files changed check
15557 if all prior existing files on either of the filesystems have changed
15558 (e.g. timestamps have changed due to changing the system's timezone)
15559 then bisync will abort without making any changes. Any new files are
15560 not considered for this check. You could use --force to force the sync
15561 (whichever side has the changed timestamp files wins). Alternately, a
15562 --resync may be used (Path1 versions will be pushed to Path2). Consid‐
15563 er the situation carefully and perhaps use --dry-run before you commit
15564 to the changes.
15565 Modification time
15567 Bisync relies on file timestamps to identify changed files and will
15568 refuse to operate if backend lacks the modification time support.
15569 If you or your application should change the content of a file without
15571 changing the modification time then bisync will not notice the change,
15572 and thus will not copy it to the other side.
15573 Note that on some cloud storage systems it is not possible to have file
15575 timestamps that match precisely between the local and other filesys‐
15576 tems.
15577 Bisync's approach to this problem is by tracking the changes on each
15579 side separately over time with a local database of files in that side
15580 then applying the resulting changes on the other side.
15581 Error handling
15583 Certain bisync critical errors, such as file copy/move failing, will
15584 result in a bisync lockout of following runs. The lockout is asserted
15585 because the sync status and history of the Path1 and Path2 filesystems
15586 cannot be trusted, so it is safer to block any further changes until
15587 someone checks things out. The recovery is to do a --resync again.
15588 It is recommended to use --resync --dry-run --verbose initially and
15590 carefully review what changes will be made before running the --resync
15591 without --dry-run.
15592 Most of these events come up due to a error status from an internal
15594 call. On such a critical error the {...}.path1.lst and {...}.path2.lst
15595 listing files are renamed to extension .lst-err, which blocks any fu‐
15596 ture bisync runs (since the normal .lst files are not found). Bisync
15597 keeps them under bisync subdirectory of the rclone cache directory,
15598 typically at ${HOME}/.cache/rclone/bisync/ on Linux.
15599 Some errors are considered temporary and re-running the bisync is not
15601 blocked. The critical return blocks further bisync runs.
15602 Lock file
15604 When bisync is running, a lock file is created in the bisync working
15605 directory, typically at ~/.cache/rclone/bisync/PATH1..PATH2.lck on Lin‐
15606 ux. If bisync should crash or hang, the lock file will remain in place
15607 and block any further runs of bisync for the same paths. Delete the
15608 lock file as part of debugging the situation. The lock file effective‐
15609 ly blocks follow-on (e.g., scheduled by cron) runs when the prior invo‐
15610 cation is taking a long time. The lock file contains PID of the block‐
15611 ing process, which may help in debug.
15612 Note that while concurrent bisync runs are allowed, be very cautious
15614 that there is no overlap in the trees being synched between concurrent
15615 runs, lest there be replicated files, deleted files and general mayhem.
15616 Return codes
15618 rclone bisync returns the following codes to calling program: - 0 on a
15619 successful run, - 1 for a non-critical failing run (a rerun may be suc‐
15620 cessful), - 2 for a critically aborted run (requires a --resync to re‐
15621 cover).
15622 Limitations
15624 Supported backends
15625 Bisync is considered BETA and has been tested with the following back‐
15626 ends: - Local filesystem - Google Drive - Dropbox - OneDrive - S3 -
15627 SFTP - Yandex Disk
15628 It has not been fully tested with other services yet. If it works, or
15630 sorta works, please let us know and we'll update the list. Run the
15631 test suite to check for proper operation as described below.
15632 First release of rclone bisync requires that underlying backend sup‐
15634 ported the modification time feature and will refuse to run otherwise.
15635 This limitation will be lifted in a future rclone bisync release.
15636 Concurrent modifications
15638 When using Local, FTP or SFTP remotes rclone does not create temporary
15639 files at the destination when copying, and thus if the connection is
15640 lost the created file may be corrupt, which will likely propagate back
15641 to the original path on the next sync, resulting in data loss. This
15642 will be solved in a future release, there is no workaround at the mo‐
15643 ment.
15644 Files that change during a bisync run may result in data loss. This
15646 has been seen in a highly dynamic environment, where the filesystem is
15647 getting hammered by running processes during the sync. The solution is
15648 to sync at quiet times or filter out unnecessary directories and files.
15649 Empty directories
15651 New empty directories on one path are not propagated to the other side.
15652 This is because bisync (and rclone) natively works on files not direc‐
15653 tories. The following sequence is a workaround but will not propagate
15654 the delete of an empty directory to the other side:
15655 rclone bisync PATH1 PATH2
15657 rclone copy PATH1 PATH2 --filter "+ */" --filter "- **" --create-empty-src-dirs
15658 rclone copy PATH2 PATH2 --filter "+ */" --filter "- **" --create-empty-src-dirs
15659 Renamed directories
15661 Renaming a folder on the Path1 side results is deleting all files on
15662 the Path2 side and then copying all files again from Path1 to Path2.
15663 Bisync sees this as all files in the old directory name as deleted and
15664 all files in the new directory name as new. Similarly, renaming a di‐
15665 rectory on both sides to the same name will result in creating ..path1
15666 and ..path2 files on both sides. Currently the most effective and ef‐
15667 ficient method of renaming a directory is to rename it on both sides,
15668 then do a --resync.
15669 Case sensitivity
15671 Synching with case-insensitive filesystems, such as Windows or Box, can
15672 result in file name conflicts. This will be fixed in a future release.
15673 The near term workaround is to make sure that files on both sides don't
15674 have spelling case differences (Smile.jpg vs. smile.jpg).
15675 Windows support
15677 Bisync has been tested on Windows 8.1, Windows 10 Pro 64-bit and on
15678 Windows GitHub runners.
15679 Drive letters are allowed, including drive letters mapped to network
15681 drives (rclone bisync J:\localsync GDrive:). If a drive letter is
15682 omitted, the shell current drive is the default. Drive letters are a
15683 single character follows by :, so cloud names must be more than one
15684 character long.
15685 Absolute paths (with or without a drive letter), and relative paths
15687 (with or without a drive letter) are supported.
15688 Working directory is created at C:\Users\MyLogin\AppData\Lo‐
15690 cal\rclone\bisync.
15691 Note that bisync output may show a mix of forward / and back \ slashes.
15693 Be careful of case independent directory and file naming on Windows vs.
15695 case dependent Linux
15696 Filtering
15698 See filtering documentation (https://rclone.org/filtering/) for how
15699 filter rules are written and interpreted.
15700 Bisync's --filters-file flag slightly extends the rclone's --filter-
15702 from (https://rclone.org/filtering/#filter-from-read-filtering-pat‐
15703 terns-from-a-file) filtering mechanism. For a given bisync run you may
15704 provide only one --filters-file. The --include*, --exclude*, and
15705 --filter flags are also supported.
15706 How to filter directories
15708 Filtering portions of the directory tree is a critical feature for
15709 synching.
15710 Examples of directory trees (always beneath the Path1/Path2 root level)
15712 you may want to exclude from your sync: - Directory trees containing
15713 only software build intermediate files. - Directory trees containing
15714 application temporary files and data such as the Windows C:\Users\MyLo‐
15715 gin\AppData\ tree. - Directory trees containing files that are large,
15716 less important, or are getting thrashed continuously by ongoing pro‐
15717 cesses.
15718 On the other hand, there may be only select directories that you actu‐
15720 ally want to sync, and exclude all others. See the Example include-
15721 style filters for Windows user directories below.
15722 Filters file writing guidelines
15724 1. Begin with excluding directory trees:
15725 • e.g. `- /AppData/`
15727 • ** on the end is not necessary. Once a given directory level is
15729 excluded then everything beneath it won't be looked at by rclone.
15730 • Exclude such directories that are unneeded, are big, dynamically
15732 thrashed, or where there may be access permission issues.
15733 • Excluding such dirs first will make rclone operations (much)
15735 faster.
15736 • Specific files may also be excluded, as with the Dropbox exclu‐
15738 sions example below.
15739 2. Decide if its easier (or cleaner) to:
15741 • Include select directories and therefore exclude everything else
15743 -- or --
15744 • Exclude select directories and therefore include everything else
15746 3. Include select directories:
15748 • Add lines like: `+ /Documents/PersonalFiles/**` to select which
15750 directories to include in the sync.
15751 • ** on the end specifies to include the full depth of the speci‐
15753 fied tree.
15754 • With Include-style filters, files at the Path1/Path2 root are not
15756 included. They may be included with `+ /*`.
15757 • Place RCLONE_TEST files within these included directory trees.
15759 They will only be looked for in these directory trees.
15760 • Finish by excluding everything else by adding `- **` at the end
15762 of the filters file.
15763 • Disregard step 4.
15765 4. Exclude select directories:
15767 • Add more lines like in step 1. For example: -/Desktop/temp‐
15769 files/, or `- /testdir/. Again, a**` on the end is not neces‐
15770 sary.
15771 • Do not add a `- **` in the file. Without this line, everything
15773 will be included that has not be explicitly excluded.
15774 • Disregard step 3.
15776 A few rules for the syntax of a filter file expanding on filtering doc‐
15778 umentation (https://rclone.org/filtering/):
15779 • Lines may start with spaces and tabs - rclone strips leading white‐
15781 space.
15782 • If the first non-whitespace character is a # then the line is a com‐
15784 ment and will be ignored.
15785 • Blank lines are ignored.
15787 • The first non-whitespace character on a filter line must be a + or -.
15789 • Exactly 1 space is allowed between the +/- and the path term.
15791 • Only forward slashes (/) are used in path terms, even on Windows.
15793 • The rest of the line is taken as the path term. Trailing whitespace
15795 is taken literally, and probably is an error.
15796 Example include-style filters for Windows user directories
15798 This Windows include-style example is based on the sync root (Path1)
15799 set to C:\Users\MyLogin. The strategy is to select specific directo‐
15800 ries to be synched with a network drive (Path2).
15801 • `- /AppData/` excludes an entire tree of Windows stored stuff that
15803 need not be synched. In my case, AppData has >11 GB of stuff I don't
15804 care about, and there are some subdirectories beneath AppData that
15805 are not accessible to my user login, resulting in bisync critical
15806 aborts.
15807 • Windows creates cache files starting with both upper and lowercase
15809 NTUSER at C:\Users\MyLogin. These files may be dynamic, locked, and
15810 are generally don't care.
15811 • There are just a few directories with my data that I do want synched,
15813 in the form of `+ /. By selecting only the directory trees I want
15814 to avoid the dozen plus directories that various apps make atC:`.
15815 • Include files in the root of the sync point, C:\Users\MyLogin, by
15817 adding the `+ /*` line.
15818 • This is an Include-style filters file, therefore it ends with `- **`
15820 which excludes everything not explicitly included.
15821 - /AppData/
15823 - NTUSER*
15824 - ntuser*
15825 + /Documents/Family/**
15826 + /Documents/Sketchup/**
15827 + /Documents/Microcapture_Photo/**
15828 + /Documents/Microcapture_Video/**
15829 + /Desktop/**
15830 + /Pictures/**
15831 + /*
15832 - **
15833 Note also that Windows implements several "library" links such as
15835 C:\Users\MyLogin\My Documents\My Music pointing to C:\Users\MyLogin\Mu‐
15836 sic. rclone sees these as links, so you must add --links to the bisync
15837 command line if you which to follow these links. I find that I get
15838 permission errors in trying to follow the links, so I don't include the
15839 rclone --links flag, but then you get lots of Can't follow symlink...
15840 noise from rclone about not following the links. This noise can be
15841 quashed by adding --quiet to the bisync command line.
15842 Example exclude-style filters files for use with Dropbox
15844 • Dropbox disallows synching the listed temporary and configuration/da‐
15845 ta files. The `- ` filters exclude these files where ever they may
15846 occur in the sync tree. Consider adding similar exclusions for file
15847 types you don't need to sync, such as core dump and software build
15848 files.
15849 • bisync testing creates /testdir/ at the top level of the sync tree,
15851 and usually deletes the tree after the test. If a normal sync should
15852 run while the /testdir/ tree exists the --check-access phase may fail
15853 due to unbalanced RCLONE_TEST files. The `- /testdir/` filter blocks
15854 this tree from being synched. You don't need this exclusion if you
15855 are not doing bisync development testing.
15856 • Everything else beneath the Path1/Path2 root will be synched.
15858 • RCLONE_TEST files may be placed anywhere within the tree, including
15860 the root.
15861 Example filters file for Dropbox
15863 # Filter file for use with bisync
15864 # See https://rclone.org/filtering/ for filtering rules
15865 # NOTICE: If you make changes to this file you MUST do a --resync run.
15866 # Run with --dry-run to see what changes will be made.
15867 # Dropbox wont sync some files so filter them away here.
15869 # See https://help.dropbox.com/installs-integrations/sync-uploads/files-not-syncing
15870 - .dropbox.attr
15871 - ~*.tmp
15872 - ~$*
15873 - .~*
15874 - desktop.ini
15875 - .dropbox
15876 # Used for bisync testing, so excluded from normal runs
15878 - /testdir/
15879 # Other example filters
15881 #- /TiBU/
15882 #- /Photos/
15883 How --check-access handles filters
15885 At the start of a bisync run, listings are gathered for Path1 and Path2
15886 while using the user's --filters-file. During the check access phase,
15887 bisync scans these listings for RCLONE_TEST files. Any RCLONE_TEST
15888 files hidden by the --filters-file are not in the listings and thus not
15889 checked during the check access phase.
15890 Troubleshooting
15892 Reading bisync logs
15893 Here are two normal runs. The first one has a newer file on the re‐
15894 mote. The second has no deltas between local and remote.
15895 2021/05/16 00:24:38 INFO : Synching Path1 "/path/to/local/tree/" with Path2 "dropbox:/"
15897 2021/05/16 00:24:38 INFO : Path1 checking for diffs
15898 2021/05/16 00:24:38 INFO : - Path1 File is new - file.txt
15899 2021/05/16 00:24:38 INFO : Path1: 1 changes: 1 new, 0 newer, 0 older, 0 deleted
15900 2021/05/16 00:24:38 INFO : Path2 checking for diffs
15901 2021/05/16 00:24:38 INFO : Applying changes
15902 2021/05/16 00:24:38 INFO : - Path1 Queue copy to Path2 - dropbox:/file.txt
15903 2021/05/16 00:24:38 INFO : - Path1 Do queued copies to - Path2
15904 2021/05/16 00:24:38 INFO : Updating listings
15905 2021/05/16 00:24:38 INFO : Validating listings for Path1 "/path/to/local/tree/" vs Path2 "dropbox:/"
15906 2021/05/16 00:24:38 INFO : Bisync successful
15907 2021/05/16 00:36:52 INFO : Synching Path1 "/path/to/local/tree/" with Path2 "dropbox:/"
15909 2021/05/16 00:36:52 INFO : Path1 checking for diffs
15910 2021/05/16 00:36:52 INFO : Path2 checking for diffs
15911 2021/05/16 00:36:52 INFO : No changes found
15912 2021/05/16 00:36:52 INFO : Updating listings
15913 2021/05/16 00:36:52 INFO : Validating listings for Path1 "/path/to/local/tree/" vs Path2 "dropbox:/"
15914 2021/05/16 00:36:52 INFO : Bisync successful
15915 Dry run oddity
15917 The --dry-run messages may indicate that it would try to delete some
15918 files. For example, if a file is new on Path2 and does not exist on
15919 Path1 then it would normally be copied to Path1, but with --dry-run en‐
15920 abled those copies don't happen, which leads to the attempted delete on
15921 the Path2, blocked again by --dry-run: ... Not deleting as --dry-run.
15922 This whole confusing situation is an artifact of the --dry-run flag.
15924 Scrutinize the proposed deletes carefully, and if the files would have
15925 been copied to Path1 then the threatened deletes on Path2 may be disre‐
15926 garded.
15927 Retries
15929 Rclone has built in retries. If you run with --verbose you'll see er‐
15930 ror and retry messages such as shown below. This is usually not a bug.
15931 If at the end of the run you see Bisync successful and not Bisync crit‐
15932 ical error or Bisync aborted then the run was successful, and you can
15933 ignore the error messages.
15934 The following run shows an intermittent fail. Lines 5 and _6- are low
15936 level messages. Line 6 is a bubbled-up warning message, conveying the
15937 error. Rclone normally retries failing commands, so there may be nu‐
15938 merous such messages in the log.
15939 Since there are no final error/warning messages on line 7, rclone has
15941 recovered from failure after a retry, and the overall sync was success‐
15942 ful.
15943 1: 2021/05/14 00:44:12 INFO : Synching Path1 "/path/to/local/tree" with Path2 "dropbox:"
15945 2: 2021/05/14 00:44:12 INFO : Path1 checking for diffs
15946 3: 2021/05/14 00:44:12 INFO : Path2 checking for diffs
15947 4: 2021/05/14 00:44:12 INFO : Path2: 113 changes: 22 new, 0 newer, 0 older, 91 deleted
15948 5: 2021/05/14 00:44:12 ERROR : /path/to/local/tree/objects/af: error listing: unexpected end of JSON input
15949 6: 2021/05/14 00:44:12 NOTICE: WARNING listing try 1 failed. - dropbox:
15950 7: 2021/05/14 00:44:12 INFO : Bisync successful
15951 This log shows a Critical failure which requires a --resync to recover
15953 from. See the Runtime Error Handling section.
15954 2021/05/12 00:49:40 INFO : Google drive root '': Waiting for checks to finish
15956 2021/05/12 00:49:40 INFO : Google drive root '': Waiting for transfers to finish
15957 2021/05/12 00:49:40 INFO : Google drive root '': not deleting files as there were IO errors
15958 2021/05/12 00:49:40 ERROR : Attempt 3/3 failed with 3 errors and: not deleting files as there were IO errors
15959 2021/05/12 00:49:40 ERROR : Failed to sync: not deleting files as there were IO errors
15960 2021/05/12 00:49:40 NOTICE: WARNING rclone sync try 3 failed. - /path/to/local/tree/
15961 2021/05/12 00:49:40 ERROR : Bisync aborted. Must run --resync to recover.
15962 Denied downloads of "infected" or "abusive" files
15964 Google Drive has a filter for certain file types (.exe, .apk, et
15965 cetera) that by default cannot be copied from Google Drive to the local
15966 filesystem. If you are having problems, run with --verbose to see
15967 specifically which files are generating complaints. If the error is
15968 This file has been identified as malware or spam and cannot be down‐
15969 loaded, consider using the flag --drive-acknowledge-abuse
15970 (https://rclone.org/drive/#drive-acknowledge-abuse).
15971 Google Doc files
15973 Google docs exist as virtual files on Google Drive and cannot be trans‐
15974 ferred to other filesystems natively. While it is possible to export a
15975 Google doc to a normal file (with .xlsx extension, for example), it is
15976 not possible to import a normal file back into a Google document.
15977 Bisync's handling of Google Doc files is to flag them in the run log
15979 output for user's attention and ignore them for any file transfers,
15980 deletes, or syncs. They will show up with a length of -1 in the list‐
15981 ings. This bisync run is otherwise successful:
15982 2021/05/11 08:23:15 INFO : Synching Path1 "/path/to/local/tree/base/" with Path2 "GDrive:"
15984 2021/05/11 08:23:15 INFO : ...path2.lst-new: Ignoring incorrect line: "- -1 - - 2018-07-29T08:49:30.136000000+0000 GoogleDoc.docx"
15985 2021/05/11 08:23:15 INFO : Bisync successful
15986 Usage examples
15988 Cron
15989 Rclone does not yet have a built-in capability to monitor the local
15990 file system for changes and must be blindly run periodically. On Win‐
15991 dows this can be done using a Task Scheduler, on Linux you can use Cron
15992 which is described below.
15993 The 1st example runs a sync every 5 minutes between a local directory
15995 and an OwnCloud server, with output logged to a runlog file:
15996 # Minute (0-59)
15998 # Hour (0-23)
15999 # Day of Month (1-31)
16000 # Month (1-12 or Jan-Dec)
16001 # Day of Week (0-6 or Sun-Sat)
16002 # Command
16003 */5 * * * * /path/to/rclone bisync /local/files MyCloud: --check-access --filters-file /path/to/bysync-filters.txt --log-file /path/to//bisync.log
16004 See crontab syntax (https://www.man7.org/linux/man-
16006 pages/man1/crontab.1p.html#INPUT_FILES)). for the details of crontab
16007 time interval expressions.
16008 If you run rclone bisync as a cron job, redirect stdout/stderr to a
16010 file. The 2nd example runs a sync to Dropbox every hour and logs all
16011 stdout (via the >>) and stderr (via 2>&1) to a log file.
16012 0 * * * * /path/to/rclone bisync /path/to/local/dropbox Dropbox: --check-access --filters-file /home/user/filters.txt >> /path/to/logs/dropbox-run.log 2>&1
16014 Sharing an encrypted folder tree between hosts
16016 bisync can keep a local folder in sync with a cloud service, but what
16017 if you have some highly sensitive files to be synched?
16018 Usage of a cloud service is for exchanging both routine and sensitive
16020 personal files between one's home network, one's personal notebook when
16021 on the road, and with one's work computer. The routine data is not
16022 sensitive. For the sensitive data, configure an rclone crypt remote
16023 (https://rclone.org/crypt/) to point to a subdirectory within the local
16024 disk tree that is bisync'd to Dropbox, and then set up an bisync for
16025 this local crypt directory to a directory outside of the main sync
16026 tree.
16027 Linux server setup
16029 • /path/to/DBoxroot is the root of my local sync tree. There are nu‐
16030 merous subdirectories.
16031 • /path/to/DBoxroot/crypt is the root subdirectory for files that are
16033 encrypted. This local directory target is setup as an rclone crypt
16034 remote named Dropcrypt:. See rclone.conf snippet below.
16035 • /path/to/my/unencrypted/files is the root of my sensitive files - not
16037 encrypted, not within the tree synched to Dropbox.
16038 • To sync my local unencrypted files with the encrypted Dropbox ver‐
16040 sions I manually run bisync /path/to/my/unencrypted/files DropCrypt:.
16041 This step could be bundled into a script to run before and after the
16042 full Dropbox tree sync in the last step, thus actively keeping the
16043 sensitive files in sync.
16044 • bisync /path/to/DBoxroot Dropbox: runs periodically via cron, keeping
16046 my full local sync tree in sync with Dropbox.
16047 Windows notebook setup
16049 • The Dropbox client runs keeping the local tree C:\Users\MyLogin\Drop‐
16050 box always in sync with Dropbox. I could have used rclone bisync in‐
16051 stead.
16052 • A separate directory tree at C:\Users\MyLogin\Documents\DropLocal
16054 hosts the tree of unencrypted files/folders.
16055 • To sync my local unencrypted files with the encrypted Dropbox ver‐
16057 sions I manually run the following command: rclone bisync
16058 C:\Users\MyLogin\Documents\DropLocal Dropcrypt:.
16059 • The Dropbox client then syncs the changes with Dropbox.
16061 rclone.conf snippet
16063 [Dropbox]
16064 type = dropbox
16065 ...
16066 [Dropcrypt]
16068 type = crypt
16069 remote = /path/to/DBoxroot/crypt # on the Linux server
16070 remote = C:\Users\MyLogin\Dropbox\crypt # on the Windows notebook
16071 filename_encryption = standard
16072 directory_name_encryption = true
16073 password = ...
16074 ...
16075 Testing
16077 You should read this section only if you are developing for rclone.
16078 You need to have rclone source code locally to work with bisync tests.
16079 Bisync has a dedicated test framework implemented in the bisync_test.go
16081 file located in the rclone source tree. The test suite is based on the
16082 go test command. Series of tests are stored in subdirectories below
16083 the cmd/bisync/testdata directory. Individual tests can be invoked by
16084 their directory name, e.g. go test . -case basic -remote local -re‐
16085 mote2 gdrive: -v
16086 Tests will make a temporary folder on remote and purge it afterwards.
16088 If during test run there are intermittent errors and rclone retries,
16089 these errors will be captured and flagged as invalid MISCOMPAREs. Re‐
16090 running the test will let it pass. Consider such failures as noise.
16091 Test command syntax
16093 usage: go test ./cmd/bisync [options...]
16094 Options:
16096 -case NAME Name(s) of the test case(s) to run. Multiple names should
16097 be separated by commas. You can remove the `test_` prefix
16098 and replace `_` by `-` in test name for convenience.
16099 If not `all`, the name(s) should map to a directory under
16100 `./cmd/bisync/testdata`.
16101 Use `all` to run all tests (default: all)
16102 -remote PATH1 `local` or name of cloud service with `:` (default: local)
16103 -remote2 PATH2 `local` or name of cloud service with `:` (default: local)
16104 -no-compare Disable comparing test results with the golden directory
16105 (default: compare)
16106 -no-cleanup Disable cleanup of Path1 and Path2 testdirs.
16107 Useful for troubleshooting. (default: cleanup)
16108 -golden Store results in the golden directory (default: false)
16109 This flag can be used with multiple tests.
16110 -debug Print debug messages
16111 -stop-at NUM Stop test after given step number. (default: run to the end)
16112 Implies `-no-compare` and `-no-cleanup`, if the test really
16113 ends prematurely. Only meaningful for a single test case.
16114 -refresh-times Force refreshing the target modtime, useful for Dropbox
16115 (default: false)
16116 -verbose Run tests verbosely
16117 Note: unlike rclone flags which must be prefixed by double dash (--),
16119 the test command flags can be equally prefixed by a single - or double
16120 dash.
16121 Running tests
16123 • go test . -case basic -remote local -remote2 local runs the test_ba‐
16124 sic test case using only the local filesystem, synching one local di‐
16125 rectory with another local directory. Test script output is to the
16126 console, while commands within scenario.txt have their output sent to
16127 the .../workdir/test.log file, which is finally compared to the gold‐
16128 en copy.
16129 • The first argument after go test should be a relative name of the di‐
16131 rectory containing bisync source code. If you run tests right from
16132 there, the argument will be . (current directory) as in most exam‐
16133 ples below. If you run bisync tests from the rclone source directo‐
16134 ry, the command should be go test ./cmd/bisync ....
16135 • The test engine will mangle rclone output to ensure comparability
16137 with golden listings and logs.
16138 • Test scenarios are located in ./cmd/bisync/testdata. The test -case
16140 argument should match the full name of a subdirectory under that di‐
16141 rectory. Every test subdirectory name on disk must start with test_,
16142 this prefix can be omitted on command line for brevity. Also, under‐
16143 scores in the name can be replaced by dashes for convenience.
16144 • go test . -remote local -remote2 local -case all runs all tests.
16146 • Path1 and Path2 may either be the keyword local or may be names of
16148 configured cloud services. go test . -remote gdrive: -remote2 drop‐
16149 box: -case basic will run the test between these two services, with‐
16150 out transferring any files to the local filesystem.
16151 • Test run stdout and stderr console output may be directed to a file,
16153 e.g. go test . -remote gdrive: -remote2 local -case all > runlog.txt
16154 2>&1
16155 Test execution flow
16157 1. The base setup in the initial directory of the testcase is applied
16158 on the Path1 and Path2 filesystems (via rclone copy the initial di‐
16159 rectory to Path1, then rclone sync Path1 to Path2).
16160 2. The commands in the scenario.txt file are applied, with output di‐
16162 rected to the test.log file in the test working directory. Typical‐
16163 ly, the first actual command in the scenario.txt file is to do a
16164 --resync, which establishes the baseline {...}.path1.lst and
16165 {...}.path2.lst files in the test working directory (.../workdir/
16166 relative to the temporary test directory). Various commands and
16167 listing snapshots are done within the test.
16168 3. Finally, the contents of the test working directory are compared to
16170 the contents of the testcase's golden directory.
16171 Notes about testing
16173 • Test cases are in individual directories beneath ./cmd/bisync/testda‐
16174 ta. A command line reference to a test is understood to reference a
16175 directory beneath testdata. For example, go test ./cmd/bisync -case
16176 dry-run -remote gdrive: -remote2 local refers to the test case in
16177 ./cmd/bisync/testdata/test_dry_run.
16178 • The test working directory is located at .../workdir relative to a
16180 temporary test directory, usually under /tmp on Linux.
16181 • The local test sync tree is created at a temporary directory named
16183 like bisync.XXX under system temporary directory.
16184 • The remote test sync tree is located at a temporary directory under
16186 <remote:>/bisync.XXX/.
16187 • path1 and/or path2 subdirectories are created in a temporary directo‐
16189 ry under the respective local or cloud test remote.
16190 • By default, the Path1 and Path2 test dirs and workdir will be deleted
16192 after each test run. The -no-cleanup flag disables purging these di‐
16193 rectories when validating and debugging a given test. These directo‐
16194 ries will be flushed before running another test, independent of the
16195 -no-cleanup usage.
16196 • You will likely want to add `- /testdir/to your normal bisync--fil‐
16198 ters-fileso that normal syncs do not attempt to sync the test tem‐
16199 porary directories, which may haveRCLONE_TESTmiscompares in some
16200 testcases which would otherwise trip the--check-accesssystem.
16201 The--check-accessmechanism is hard-coded to ignoreRCLONE_TESTfiles
16202 beneathbisync/testdata`, so the test cases may reside on the synched
16203 tree even if there are check file mismatches in the test tree.
16204 • Some Dropbox tests can fail, notably printing the following message:
16206 src and dst identical but can't set mod time without deleting and re-
16207 uploading This is expected and happens due a way Dropbox handles mod‐
16208 ification times. You should use the -refresh-times test flag to make
16209 up for this.
16210 • If Dropbox tests hit request limit for you and print error message
16212 too_many_requests/...: Too many requests or write operations. then
16213 follow the Dropbox App ID instructions (https://rclone.org/drop‐
16214 box/#get-your-own-dropbox-app-id).
16215 Updating golden results
16217 Sometimes even a slight change in the bisync source can cause little
16218 changes spread around many log files. Updating them manually would be
16219 a nightmare.
16220 The -golden flag will store the test.log and *.lst listings from each
16222 test case into respective golden directories. Golden results will au‐
16223 tomatically contain generic strings instead of local or cloud paths
16224 which means that they should match when run with a different cloud ser‐
16225 vice.
16226 Your normal workflow might be as follows: 1. Git-clone the rclone
16228 sources locally 2. Modify bisync source and check that it builds 3.
16229 Run the whole test suite go test ./cmd/bisync -remote local 4. If some
16230 tests show log difference, recheck them individually, e.g.: go test
16231 ./cmd/bisync -remote local -case basic 5. If you are convinced with
16232 the difference, goldenize all tests at once: go test ./cmd/bisync -re‐
16233 mote local -golden 6. Use word diff: git diff --word-diff
16234 ./cmd/bisync/testdata/. Please note that normal line-level diff is
16235 generally useless here. 7. Check the difference carefully! 8. Commit
16236 the change (git commit) only if you are sure. If unsure, save your
16237 code changes then wipe the log diffs from git: git reset [--hard].
16238 Structure of test scenarios
16240 • <testname>/initial/ contains a tree of files that will be set as the
16241 initial condition on both Path1 and Path2 testdirs.
16242 • <testname>/modfiles/ contains files that will be used to modify the
16244 Path1 and/or Path2 filesystems.
16245 • <testname>/golden/ contains the expected content of the test working
16247 directory (workdir) at the completion of the testcase.
16248 • <testname>/scenario.txt contains the body of the test, in the form of
16250 various commands to modify files, run bisync, and snapshot listings.
16251 Output from these commands is captured to .../workdir/test.log for
16252 comparison to the golden files.
16253 Supported test commands
16255 • test <some message> Print the line to the console and to the
16256 test.log: test sync is working correctly with options x, y, z
16257 • copy-listings <prefix> Save a copy of all .lst listings in the test
16259 working directory with the specified prefix: save-listings exclude-
16260 pass-run
16261 • move-listings <prefix> Similar to copy-listings but removes the
16263 source
16264 • purge-children <dir> This will delete all child files and purge all
16266 child subdirs under given directory but keep the parent intact. This
16267 behavior is important for tests with Google Drive because removing
16268 and re-creating the parent would change its ID.
16269 • delete-file <file> Delete a single file.
16271 • delete-glob <dir> <pattern> Delete a group of files located one level
16273 deep in the given directory with names maching a given glob pattern.
16274 • touch-glob YYYY-MM-DD <dir> <pattern> Change modification time on a
16276 group of files.
16277 • touch-copy YYYY-MM-DD <source-file> <dest-dir> Change file modifica‐
16279 tion time then copy it to destination.
16280 • copy-file <source-file> <dest-dir> Copy a single file to given direc‐
16282 tory.
16283 • copy-as <source-file> <dest-file> Similar to above but destination
16285 must include both directory and the new file name at destination.
16286 • copy-dir <src> <dst> and sync-dir <src> <dst> Copy/sync a directory.
16288 Equivalent of rclone copy and rclone sync.
16289 • list-dirs <dir> Equivalent to rclone lsf -R --dirs-only <dir>
16291 • bisync [options] Runs bisync against -remote and -remote2.
16293 Supported substitution terms
16295 • {testdir/} - the root dir of the testcase
16296 • {datadir/} - the modfiles dir under the testcase root
16298 • {workdir/} - the temporary test working directory
16300 • {path1/} - the root of the Path1 test directory tree
16302 • {path2/} - the root of the Path2 test directory tree
16304 • {session} - base name of the test listings
16306 • {/} - OS-specific path separator
16308 • {spc}, {tab}, {eol} - whitespace
16310 • {chr:HH} - raw byte with given hexadecimal code
16312 Substitution results of the terms named like {dir/} will end with / (or
16314 backslash on Windows), so it is not necessary to include slash in the
16315 usage, for example delete-file {path1/}file1.txt.
16316 Benchmarks
16318 This section is work in progress.
16319 Here are a few data points for scale, execution times, and memory us‐
16321 age.
16322 The first set of data was taken between a local disk to Dropbox. The
16324 speedtest.net (https://speedtest.net) download speed was ~170 Mbps, and
16325 upload speed was ~10 Mbps. 500 files (~9.5 MB each) had been already
16326 synched. 50 files were added in a new directory, each ~9.5 MB, ~475 MB
16327 total.
16328 Change Operations and times Overall run
16330 time
16331 ─────────────────────────────────────────────────────────────────────────────
16332 500 files synched (noth‐ 1x listings for Path1 & Path2 1.5 sec
16333 ing to move)
16334 500 files synched with 1x listings for Path1 & Path2 1.5 sec
16335 --check-access
16336 50 new files on remote Queued 50 copies down: 27 sec 29 sec
16337 Moved local dir Queued 50 copies up: 410 sec, 50 421 sec
16338 deletes up: 9 sec
16339 Moved remote dir Queued 50 copies down: 31 sec, 50 33 sec
16340 deletes down: <1 sec
16341 Delete local dir Queued 50 deletes up: 9 sec 13 sec
16342 This next data is from a user's application. They had ~400GB of data
16344 over 1.96 million files being sync'ed between a Windows local disk and
16345 some remote cloud. The file full path length was on average 35 charac‐
16346 ters (which factors into load time and RAM required).
16347 • Loading the prior listing into memory (1.96 million files, listing
16349 file size 140 MB) took ~30 sec and occupied about 1 GB of RAM.
16350 • Getting a fresh listing of the local file system (producing the 140
16352 MB output file) took about XXX sec.
16353 • Getting a fresh listing of the remote file system (producing the 140
16355 MB output file) took about XXX sec. The network download speed was
16356 measured at XXX Mb/s.
16357 • Once the prior and current Path1 and Path2 listings were loaded (a
16359 total of four to be loaded, two at a time), determining the deltas
16360 was pretty quick (a few seconds for this test case), and the transfer
16361 time for any files to be copied was dominated by the network band‐
16362 width.
16363 References
16365 rclone's bisync implementation was derived from the rclonesync-V2
16366 (https://github.com/cjnaz/rclonesync-V2) project, including documenta‐
16367 tion and test mechanisms, with [@cjnaz](https://github.com/cjnaz)'s
16368 full support and encouragement.
16369 rclone bisync is similar in nature to a range of other projects:
16371 • unison (https://github.com/bcpierce00/unison)
16373 • syncthing (https://github.com/syncthing/syncthing)
16375 • cjnaz/rclonesync (https://github.com/cjnaz/rclonesync-V2)
16377 • ConorWilliams/rsinc (https://github.com/ConorWilliams/rsinc)
16379 • jwink3101/syncrclone (https://github.com/Jwink3101/syncrclone)
16381 • DavideRossi/upback (https://github.com/DavideRossi/upback)
16383 Bisync adopts the differential synchronization technique, which is
16385 based on keeping history of changes performed by both synchronizing
16386 sides. See the Dual Shadow Method section in the Neil Fraser's article
16387 (https://neil.fraser.name/writing/sync/).
16388 Also note a number of academic publications by Benjamin Pierce
16390 (http://www.cis.upenn.edu/%7Ebcpierce/papers/index.shtml#File%20Syn‐
16391 chronization) about Unison and synchronization in general.
16392
16394 This is a backend for the 1fichier (https://1fichier.com) cloud storage
16395 service. Note that a Premium subscription is required to use the API.
16396 Paths are specified as remote:path
16398 Paths may be as deep as required, e.g. remote:directory/subdirectory.
16400 Configuration
16402 The initial setup for 1Fichier involves getting the API key from the
16403 website which you need to do in your browser.
16404 Here is an example of how to make a remote called remote. First run:
16406 rclone config
16408 This will guide you through an interactive setup process:
16410 No remotes found, make a new one?
16412 n) New remote
16413 s) Set configuration password
16414 q) Quit config
16415 n/s/q> n
16416 name> remote
16417 Type of storage to configure.
16418 Enter a string value. Press Enter for the default ("").
16419 Choose a number from below, or type in your own value
16420 [snip]
16421 XX / 1Fichier
16422 \ "fichier"
16423 [snip]
16424 Storage> fichier
16425 ** See help for fichier backend at: https://rclone.org/fichier/ **
16426 Your API Key, get it from https://1fichier.com/console/params.pl
16428 Enter a string value. Press Enter for the default ("").
16429 api_key> example_key
16430 Edit advanced config? (y/n)
16432 y) Yes
16433 n) No
16434 y/n>
16435 Remote config
16436 --------------------
16437 [remote]
16438 type = fichier
16439 api_key = example_key
16440 --------------------
16441 y) Yes this is OK
16442 e) Edit this remote
16443 d) Delete this remote
16444 y/e/d> y
16445 Once configured you can then use rclone like this,
16447 List directories in top level of your 1Fichier account
16449 rclone lsd remote:
16451 List all the files in your 1Fichier account
16453 rclone ls remote:
16455 To copy a local directory to a 1Fichier directory called backup
16457 rclone copy /home/source remote:backup
16459 Modified time and hashes
16461 1Fichier does not support modification times. It supports the
16462 Whirlpool hash algorithm.
16463 Duplicated files
16465 1Fichier can have two files with exactly the same name and path (unlike
16466 a normal file system).
16467 Duplicated files cause problems with the syncing and you will see mes‐
16469 sages in the log about duplicates.
16470 Restricted filename characters
16472 In addition to the default restricted characters set
16473 (https://rclone.org/overview/#restricted-characters) the following
16474 characters are also replaced:
16475 Character Value Replacement
16477 ────────────────────────────────
16478 \ 0x5C ＼
16479 < 0x3C ＜
16480 > 0x3E ＞
16481 " 0x22 ＂
16482 $ 0x24 ＄
16483 ` 0x60 ｀
16484 ' 0x27 ＇
16485 File names can also not start or end with the following characters.
16487 These only get replaced if they are the first or last character in the
16488 name:
16489 Character Value Replacement
16491 ────────────────────────────────
16492 SP 0x20 ␠
16493 Invalid UTF-8 bytes will also be replaced (https://rclone.org/over‐
16495 view/#invalid-utf8), as they can't be used in JSON strings.
16496 Standard options
16498 Here are the Standard options specific to fichier (1Fichier).
16499 --fichier-api-key
16501 Your API Key, get it from https://1fichier.com/console/params.pl.
16502 Properties:
16504 • Config: api_key
16506 • Env Var: RCLONE_FICHIER_API_KEY
16508 • Type: string
16510 • Required: false
16512 Advanced options
16514 Here are the Advanced options specific to fichier (1Fichier).
16515 --fichier-shared-folder
16517 If you want to download a shared folder, add this parameter.
16518 Properties:
16520 • Config: shared_folder
16522 • Env Var: RCLONE_FICHIER_SHARED_FOLDER
16524 • Type: string
16526 • Required: false
16528 --fichier-file-password
16530 If you want to download a shared file that is password protected, add
16531 this parameter.
16532 NB Input to this must be obscured - see rclone obscure
16534 (https://rclone.org/commands/rclone_obscure/).
16535 Properties:
16537 • Config: file_password
16539 • Env Var: RCLONE_FICHIER_FILE_PASSWORD
16541 • Type: string
16543 • Required: false
16545 --fichier-folder-password
16547 If you want to list the files in a shared folder that is password pro‐
16548 tected, add this parameter.
16549 NB Input to this must be obscured - see rclone obscure
16551 (https://rclone.org/commands/rclone_obscure/).
16552 Properties:
16554 • Config: folder_password
16556 • Env Var: RCLONE_FICHIER_FOLDER_PASSWORD
16558 • Type: string
16560 • Required: false
16562 --fichier-encoding
16564 The encoding for the backend.
16565 See the encoding section in the overview (https://rclone.org/over‐
16567 view/#encoding) for more info.
16568 Properties:
16570 • Config: encoding
16572 • Env Var: RCLONE_FICHIER_ENCODING
16574 • Type: MultiEncoder
16576 • Default: Slash,LtGt,DoubleQuote,SingleQuote,BackQuote,Dollar,Back‐
16578 Slash,Del,Ctl,LeftSpace,RightSpace,InvalidUtf8,Dot
16579 Limitations
16581 rclone about is not supported by the 1Fichier backend. Backends with‐
16582 out this capability cannot determine free space for an rclone mount or
16583 use policy mfs (most free space) as a member of an rclone union remote.
16584 See List of backends that do not support rclone about
16586 (https://rclone.org/overview/#optional-features) and rclone about
16587 (https://rclone.org/commands/rclone_about/)
16588
16590 The alias remote provides a new name for another remote.
16591 Paths may be as deep as required or a local path, e.g. remote:directo‐
16593 ry/subdirectory or /directory/subdirectory.
16594 During the initial setup with rclone config you will specify the target
16596 remote. The target remote can either be a local path or another re‐
16597 mote.
16598 Subfolders can be used in target remote. Assume an alias remote named
16600 backup with the target mydrive:private/backup. Invoking rclone mkdir
16601 backup:desktop is exactly the same as invoking rclone mkdir my‐
16602 drive:private/backup/desktop.
16603 There will be no special handling of paths containing .. segments.
16605 Invoking rclone mkdir backup:../desktop is exactly the same as invoking
16606 rclone mkdir mydrive:private/backup/../desktop. The empty path is not
16607 allowed as a remote. To alias the current directory use . instead.
16608 The target remote can also be a connection string
16610 (https://rclone.org/docs/#connection-strings). This can be used to
16611 modify the config of a remote for different uses, e.g. the alias my‐
16612 DriveTrash with the target remote myDrive,trashed_only: can be used to
16613 only show the trashed files in myDrive.
16614 Configuration
16616 Here is an example of how to make an alias called remote for local
16617 folder. First run:
16618 rclone config
16620 This will guide you through an interactive setup process:
16622 No remotes found, make a new one?
16624 n) New remote
16625 s) Set configuration password
16626 q) Quit config
16627 n/s/q> n
16628 name> remote
16629 Type of storage to configure.
16630 Choose a number from below, or type in your own value
16631 [snip]
16632 XX / Alias for an existing remote
16633 \ "alias"
16634 [snip]
16635 Storage> alias
16636 Remote or path to alias.
16637 Can be "myremote:path/to/dir", "myremote:bucket", "myremote:" or "/local/path".
16638 remote> /mnt/storage/backup
16639 Remote config
16640 --------------------
16641 [remote]
16642 remote = /mnt/storage/backup
16643 --------------------
16644 y) Yes this is OK
16645 e) Edit this remote
16646 d) Delete this remote
16647 y/e/d> y
16648 Current remotes:
16649 Name Type
16651 ==== ====
16652 remote alias
16653 e) Edit existing remote
16655 n) New remote
16656 d) Delete remote
16657 r) Rename remote
16658 c) Copy remote
16659 s) Set configuration password
16660 q) Quit config
16661 e/n/d/r/c/s/q> q
16662 Once configured you can then use rclone like this,
16664 List directories in top level in /mnt/storage/backup
16666 rclone lsd remote:
16668 List all the files in /mnt/storage/backup
16670 rclone ls remote:
16672 Copy another local directory to the alias directory called source
16674 rclone copy /home/source remote:source
16676 Standard options
16678 Here are the Standard options specific to alias (Alias for an existing
16679 remote).
16680 --alias-remote
16682 Remote or path to alias.
16683 Can be "myremote:path/to/dir", "myremote:bucket", "myremote:" or "/lo‐
16685 cal/path".
16686 Properties:
16688 • Config: remote
16690 • Env Var: RCLONE_ALIAS_REMOTE
16692 • Type: string
16694 • Required: true
16696
16698 Amazon Drive, formerly known as Amazon Cloud Drive, is a cloud storage
16699 service run by Amazon for consumers.
16700 Status
16702 Important: rclone supports Amazon Drive only if you have your own set
16703 of API keys. Unfortunately the Amazon Drive developer program
16704 (https://developer.amazon.com/amazon-drive) is now closed to new en‐
16705 tries so if you don't already have your own set of keys you will not be
16706 able to use rclone with Amazon Drive.
16707 For the history on why rclone no longer has a set of Amazon Drive API
16709 keys see the forum (https://forum.rclone.org/t/rclone-has-been-banned-
16710 from-amazon-drive/2314).
16711 If you happen to know anyone who works at Amazon then please ask them
16713 to re-instate rclone into the Amazon Drive developer program - thanks!
16714 Configuration
16716 The initial setup for Amazon Drive involves getting a token from Amazon
16717 which you need to do in your browser. rclone config walks you through
16718 it.
16719 The configuration process for Amazon Drive may involve using an oauth
16721 proxy (https://github.com/ncw/oauthproxy). This is used to keep the
16722 Amazon credentials out of the source code. The proxy runs in Google's
16723 very secure App Engine environment and doesn't store any credentials
16724 which pass through it.
16725 Since rclone doesn't currently have its own Amazon Drive credentials so
16727 you will either need to have your own client_id and client_secret with
16728 Amazon Drive, or use a third-party oauth proxy in which case you will
16729 need to enter client_id, client_secret, auth_url and token_url.
16730 Note also if you are not using Amazon's auth_url and token_url, (ie you
16732 filled in something for those) then if setting up on a remote machine
16733 you can only use the copying the config method of configuration
16734 (https://rclone.org/remote_setup/#configuring-by-copying-the-config-
16735 file) - rclone authorize will not work.
16736 Here is an example of how to make a remote called remote. First run:
16738 rclone config
16740 This will guide you through an interactive setup process:
16742 No remotes found, make a new one?
16744 n) New remote
16745 r) Rename remote
16746 c) Copy remote
16747 s) Set configuration password
16748 q) Quit config
16749 n/r/c/s/q> n
16750 name> remote
16751 Type of storage to configure.
16752 Choose a number from below, or type in your own value
16753 [snip]
16754 XX / Amazon Drive
16755 \ "amazon cloud drive"
16756 [snip]
16757 Storage> amazon cloud drive
16758 Amazon Application Client Id - required.
16759 client_id> your client ID goes here
16760 Amazon Application Client Secret - required.
16761 client_secret> your client secret goes here
16762 Auth server URL - leave blank to use Amazon's.
16763 auth_url> Optional auth URL
16764 Token server url - leave blank to use Amazon's.
16765 token_url> Optional token URL
16766 Remote config
16767 Make sure your Redirect URL is set to "http://127.0.0.1:53682/" in your custom config.
16768 Use auto config?
16769 * Say Y if not sure
16770 * Say N if you are working on a remote or headless machine
16771 y) Yes
16772 n) No
16773 y/n> y
16774 If your browser doesn't open automatically go to the following link: http://127.0.0.1:53682/auth
16775 Log in and authorize rclone for access
16776 Waiting for code...
16777 Got code
16778 --------------------
16779 [remote]
16780 client_id = your client ID goes here
16781 client_secret = your client secret goes here
16782 auth_url = Optional auth URL
16783 token_url = Optional token URL
16784 token = {"access_token":"xxxxxxxxxxxxxxxxxxxxxxx","token_type":"bearer","refresh_token":"xxxxxxxxxxxxxxxxxx","expiry":"2015-09-06T16:07:39.658438471+01:00"}
16785 --------------------
16786 y) Yes this is OK
16787 e) Edit this remote
16788 d) Delete this remote
16789 y/e/d> y
16790 See the remote setup docs (https://rclone.org/remote_setup/) for how to
16792 set it up on a machine with no Internet browser available.
16793 Note that rclone runs a webserver on your local machine to collect the
16795 token as returned from Amazon. This only runs from the moment it opens
16796 your browser to the moment you get back the verification code. This is
16797 on http://127.0.0.1:53682/ and this it may require you to unblock it
16798 temporarily if you are running a host firewall.
16799 Once configured you can then use rclone like this,
16801 List directories in top level of your Amazon Drive
16803 rclone lsd remote:
16805 List all the files in your Amazon Drive
16807 rclone ls remote:
16809 To copy a local directory to an Amazon Drive directory called backup
16811 rclone copy /home/source remote:backup
16813 Modified time and MD5SUMs
16815 Amazon Drive doesn't allow modification times to be changed via the API
16816 so these won't be accurate or used for syncing.
16817 It does store MD5SUMs so for a more accurate sync, you can use the
16819 --checksum flag.
16820 Restricted filename characters
16822 Character Value Replacement
16823 ────────────────────────────────
16824 NUL 0x00 ␀
16825 / 0x2F ／
16826 Invalid UTF-8 bytes will also be replaced (https://rclone.org/over‐
16828 view/#invalid-utf8), as they can't be used in JSON strings.
16829 Deleting files
16831 Any files you delete with rclone will end up in the trash. Amazon
16832 don't provide an API to permanently delete files, nor to empty the
16833 trash, so you will have to do that with one of Amazon's apps or via the
16834 Amazon Drive website. As of November 17, 2016, files are automatically
16835 deleted by Amazon from the trash after 30 days.
16836 Using with non .com Amazon accounts
16838 Let's say you usually use amazon.co.uk. When you authenticate with
16839 rclone it will take you to an amazon.com page to log in. Your ama‐
16840 zon.co.uk email and password should work here just fine.
16841 Standard options
16843 Here are the Standard options specific to amazon cloud drive (Amazon
16844 Drive).
16845 --acd-client-id
16847 OAuth Client Id.
16848 Leave blank normally.
16850 Properties:
16852 • Config: client_id
16854 • Env Var: RCLONE_ACD_CLIENT_ID
16856 • Type: string
16858 • Required: false
16860 --acd-client-secret
16862 OAuth Client Secret.
16863 Leave blank normally.
16865 Properties:
16867 • Config: client_secret
16869 • Env Var: RCLONE_ACD_CLIENT_SECRET
16871 • Type: string
16873 • Required: false
16875 Advanced options
16877 Here are the Advanced options specific to amazon cloud drive (Amazon
16878 Drive).
16879 --acd-token
16881 OAuth Access Token as a JSON blob.
16882 Properties:
16884 • Config: token
16886 • Env Var: RCLONE_ACD_TOKEN
16888 • Type: string
16890 • Required: false
16892 --acd-auth-url
16894 Auth server URL.
16895 Leave blank to use the provider defaults.
16897 Properties:
16899 • Config: auth_url
16901 • Env Var: RCLONE_ACD_AUTH_URL
16903 • Type: string
16905 • Required: false
16907 --acd-token-url
16909 Token server url.
16910 Leave blank to use the provider defaults.
16912 Properties:
16914 • Config: token_url
16916 • Env Var: RCLONE_ACD_TOKEN_URL
16918 • Type: string
16920 • Required: false
16922 --acd-checkpoint
16924 Checkpoint for internal polling (debug).
16925 Properties:
16927 • Config: checkpoint
16929 • Env Var: RCLONE_ACD_CHECKPOINT
16931 • Type: string
16933 • Required: false
16935 --acd-upload-wait-per-gb
16937 Additional time per GiB to wait after a failed complete upload to see
16938 if it appears.
16939 Sometimes Amazon Drive gives an error when a file has been fully up‐
16941 loaded but the file appears anyway after a little while. This happens
16942 sometimes for files over 1 GiB in size and nearly every time for files
16943 bigger than 10 GiB. This parameter controls the time rclone waits for
16944 the file to appear.
16945 The default value for this parameter is 3 minutes per GiB, so by de‐
16947 fault it will wait 3 minutes for every GiB uploaded to see if the file
16948 appears.
16949 You can disable this feature by setting it to 0. This may cause con‐
16951 flict errors as rclone retries the failed upload but the file will most
16952 likely appear correctly eventually.
16953 These values were determined empirically by observing lots of uploads
16955 of big files for a range of file sizes.
16956 Upload with the "-v" flag to see more info about what rclone is doing
16958 in this situation.
16959 Properties:
16961 • Config: upload_wait_per_gb
16963 • Env Var: RCLONE_ACD_UPLOAD_WAIT_PER_GB
16965 • Type: Duration
16967 • Default: 3m0s
16969 --acd-templink-threshold
16971 Files >= this size will be downloaded via their tempLink.
16972 Files this size or more will be downloaded via their "tempLink". This
16974 is to work around a problem with Amazon Drive which blocks downloads of
16975 files bigger than about 10 GiB. The default for this is 9 GiB which
16976 shouldn't need to be changed.
16977 To download files above this threshold, rclone requests a "tempLink"
16979 which downloads the file through a temporary URL directly from the un‐
16980 derlying S3 storage.
16981 Properties:
16983 • Config: templink_threshold
16985 • Env Var: RCLONE_ACD_TEMPLINK_THRESHOLD
16987 • Type: SizeSuffix
16989 • Default: 9Gi
16991 --acd-encoding
16993 The encoding for the backend.
16994 See the encoding section in the overview (https://rclone.org/over‐
16996 view/#encoding) for more info.
16997 Properties:
16999 • Config: encoding
17001 • Env Var: RCLONE_ACD_ENCODING
17003 • Type: MultiEncoder
17005 • Default: Slash,InvalidUtf8,Dot
17007 Limitations
17009 Note that Amazon Drive is case insensitive so you can't have a file
17010 called "Hello.doc" and one called "hello.doc".
17011 Amazon Drive has rate limiting so you may notice errors in the sync
17013 (429 errors). rclone will automatically retry the sync up to 3 times
17014 by default (see --retries flag) which should hopefully work around this
17015 problem.
17016 Amazon Drive has an internal limit of file sizes that can be uploaded
17018 to the service. This limit is not officially published, but all files
17019 larger than this will fail.
17020 At the time of writing (Jan 2016) is in the area of 50 GiB per file.
17022 This means that larger files are likely to fail.
17023 Unfortunately there is no way for rclone to see that this failure is
17025 because of file size, so it will retry the operation, as any other
17026 failure. To avoid this problem, use --max-size 50000M option to limit
17027 the maximum size of uploaded files. Note that --max-size does not
17028 split files into segments, it only ignores files over this size.
17029 rclone about is not supported by the Amazon Drive backend. Backends
17031 without this capability cannot determine free space for an rclone mount
17032 or use policy mfs (most free space) as a member of an rclone union re‐
17033 mote.
17034 See List of backends that do not support rclone about
17036 (https://rclone.org/overview/#optional-features) and rclone about
17037 (https://rclone.org/commands/rclone_about/)
17038
17040 The S3 backend can be used with a number of different providers:
17041 • AWS S3
17043 • Alibaba Cloud (Aliyun) Object Storage System (OSS)
17045 • Ceph
17047 • China Mobile Ecloud Elastic Object Storage (EOS)
17049 • Cloudflare R2
17051 • Arvan Cloud Object Storage (AOS)
17053 • DigitalOcean Spaces
17055 • Dreamhost
17057 • Huawei OBS
17059 • IBM COS S3
17061 • IDrive e2
17063 • IONOS Cloud
17065 • Minio
17067 • Qiniu Cloud Object Storage (Kodo)
17069 • RackCorp Object Storage
17071 • Scaleway
17073 • Seagate Lyve Cloud
17075 • SeaweedFS
17077 • StackPath
17079 • Storj
17081 • Tencent Cloud Object Storage (COS)
17083 • Wasabi
17085 Paths are specified as remote:bucket (or remote: for the lsd command.)
17087 You may put subdirectories in too, e.g. remote:bucket/path/to/dir.
17088 Once you have made a remote (see the provider specific section above)
17090 you can use it like this:
17091 See all buckets
17093 rclone lsd remote:
17095 Make a new bucket
17097 rclone mkdir remote:bucket
17099 List the contents of a bucket
17101 rclone ls remote:bucket
17103 Sync /home/local/directory to the remote bucket, deleting any excess
17105 files in the bucket.
17106 rclone sync -i /home/local/directory remote:bucket
17108 Configuration
17110 Here is an example of making an s3 configuration for the AWS S3
17111 provider. Most applies to the other providers as well, any differences
17112 are described below.
17113 First run
17115 rclone config
17117 This will guide you through an interactive setup process.
17119 No remotes found, make a new one?
17121 n) New remote
17122 s) Set configuration password
17123 q) Quit config
17124 n/s/q> n
17125 name> remote
17126 Type of storage to configure.
17127 Choose a number from below, or type in your own value
17128 [snip]
17129 XX / Amazon S3 Compliant Storage Providers including AWS, Ceph, ChinaMobile, ArvanCloud, Dreamhost, IBM COS, Minio, and Tencent COS
17130 \ "s3"
17131 [snip]
17132 Storage> s3
17133 Choose your S3 provider.
17134 Choose a number from below, or type in your own value
17135 1 / Amazon Web Services (AWS) S3
17136 \ "AWS"
17137 2 / Ceph Object Storage
17138 \ "Ceph"
17139 3 / Digital Ocean Spaces
17140 \ "DigitalOcean"
17141 4 / Dreamhost DreamObjects
17142 \ "Dreamhost"
17143 5 / IBM COS S3
17144 \ "IBMCOS"
17145 6 / Minio Object Storage
17146 \ "Minio"
17147 7 / Wasabi Object Storage
17148 \ "Wasabi"
17149 8 / Any other S3 compatible provider
17150 \ "Other"
17151 provider> 1
17152 Get AWS credentials from runtime (environment variables or EC2/ECS meta data if no env vars). Only applies if access_key_id and secret_access_key is blank.
17153 Choose a number from below, or type in your own value
17154 1 / Enter AWS credentials in the next step
17155 \ "false"
17156 2 / Get AWS credentials from the environment (env vars or IAM)
17157 \ "true"
17158 env_auth> 1
17159 AWS Access Key ID - leave blank for anonymous access or runtime credentials.
17160 access_key_id> XXX
17161 AWS Secret Access Key (password) - leave blank for anonymous access or runtime credentials.
17162 secret_access_key> YYY
17163 Region to connect to.
17164 Choose a number from below, or type in your own value
17165 / The default endpoint - a good choice if you are unsure.
17166 1 | US Region, Northern Virginia, or Pacific Northwest.
17167 | Leave location constraint empty.
17168 \ "us-east-1"
17169 / US East (Ohio) Region
17170 2 | Needs location constraint us-east-2.
17171 \ "us-east-2"
17172 / US West (Oregon) Region
17173 3 | Needs location constraint us-west-2.
17174 \ "us-west-2"
17175 / US West (Northern California) Region
17176 4 | Needs location constraint us-west-1.
17177 \ "us-west-1"
17178 / Canada (Central) Region
17179 5 | Needs location constraint ca-central-1.
17180 \ "ca-central-1"
17181 / EU (Ireland) Region
17182 6 | Needs location constraint EU or eu-west-1.
17183 \ "eu-west-1"
17184 / EU (London) Region
17185 7 | Needs location constraint eu-west-2.
17186 \ "eu-west-2"
17187 / EU (Frankfurt) Region
17188 8 | Needs location constraint eu-central-1.
17189 \ "eu-central-1"
17190 / Asia Pacific (Singapore) Region
17191 9 | Needs location constraint ap-southeast-1.
17192 \ "ap-southeast-1"
17193 / Asia Pacific (Sydney) Region
17194 10 | Needs location constraint ap-southeast-2.
17195 \ "ap-southeast-2"
17196 / Asia Pacific (Tokyo) Region
17197 11 | Needs location constraint ap-northeast-1.
17198 \ "ap-northeast-1"
17199 / Asia Pacific (Seoul)
17200 12 | Needs location constraint ap-northeast-2.
17201 \ "ap-northeast-2"
17202 / Asia Pacific (Mumbai)
17203 13 | Needs location constraint ap-south-1.
17204 \ "ap-south-1"
17205 / Asia Pacific (Hong Kong) Region
17206 14 | Needs location constraint ap-east-1.
17207 \ "ap-east-1"
17208 / South America (Sao Paulo) Region
17209 15 | Needs location constraint sa-east-1.
17210 \ "sa-east-1"
17211 region> 1
17212 Endpoint for S3 API.
17213 Leave blank if using AWS to use the default endpoint for the region.
17214 endpoint>
17215 Location constraint - must be set to match the Region. Used when creating buckets only.
17216 Choose a number from below, or type in your own value
17217 1 / Empty for US Region, Northern Virginia, or Pacific Northwest.
17218 \ ""
17219 2 / US East (Ohio) Region.
17220 \ "us-east-2"
17221 3 / US West (Oregon) Region.
17222 \ "us-west-2"
17223 4 / US West (Northern California) Region.
17224 \ "us-west-1"
17225 5 / Canada (Central) Region.
17226 \ "ca-central-1"
17227 6 / EU (Ireland) Region.
17228 \ "eu-west-1"
17229 7 / EU (London) Region.
17230 \ "eu-west-2"
17231 8 / EU Region.
17232 \ "EU"
17233 9 / Asia Pacific (Singapore) Region.
17234 \ "ap-southeast-1"
17235 10 / Asia Pacific (Sydney) Region.
17236 \ "ap-southeast-2"
17237 11 / Asia Pacific (Tokyo) Region.
17238 \ "ap-northeast-1"
17239 12 / Asia Pacific (Seoul)
17240 \ "ap-northeast-2"
17241 13 / Asia Pacific (Mumbai)
17242 \ "ap-south-1"
17243 14 / Asia Pacific (Hong Kong)
17244 \ "ap-east-1"
17245 15 / South America (Sao Paulo) Region.
17246 \ "sa-east-1"
17247 location_constraint> 1
17248 Canned ACL used when creating buckets and/or storing objects in S3.
17249 For more info visit https://docs.aws.amazon.com/AmazonS3/latest/dev/acl-overview.html#canned-acl
17250 Choose a number from below, or type in your own value
17251 1 / Owner gets FULL_CONTROL. No one else has access rights (default).
17252 \ "private"
17253 2 / Owner gets FULL_CONTROL. The AllUsers group gets READ access.
17254 \ "public-read"
17255 / Owner gets FULL_CONTROL. The AllUsers group gets READ and WRITE access.
17256 3 | Granting this on a bucket is generally not recommended.
17257 \ "public-read-write"
17258 4 / Owner gets FULL_CONTROL. The AuthenticatedUsers group gets READ access.
17259 \ "authenticated-read"
17260 / Object owner gets FULL_CONTROL. Bucket owner gets READ access.
17261 5 | If you specify this canned ACL when creating a bucket, Amazon S3 ignores it.
17262 \ "bucket-owner-read"
17263 / Both the object owner and the bucket owner get FULL_CONTROL over the object.
17264 6 | If you specify this canned ACL when creating a bucket, Amazon S3 ignores it.
17265 \ "bucket-owner-full-control"
17266 acl> 1
17267 The server-side encryption algorithm used when storing this object in S3.
17268 Choose a number from below, or type in your own value
17269 1 / None
17270 \ ""
17271 2 / AES256
17272 \ "AES256"
17273 server_side_encryption> 1
17274 The storage class to use when storing objects in S3.
17275 Choose a number from below, or type in your own value
17276 1 / Default
17277 \ ""
17278 2 / Standard storage class
17279 \ "STANDARD"
17280 3 / Reduced redundancy storage class
17281 \ "REDUCED_REDUNDANCY"
17282 4 / Standard Infrequent Access storage class
17283 \ "STANDARD_IA"
17284 5 / One Zone Infrequent Access storage class
17285 \ "ONEZONE_IA"
17286 6 / Glacier storage class
17287 \ "GLACIER"
17288 7 / Glacier Deep Archive storage class
17289 \ "DEEP_ARCHIVE"
17290 8 / Intelligent-Tiering storage class
17291 \ "INTELLIGENT_TIERING"
17292 9 / Glacier Instant Retrieval storage class
17293 \ "GLACIER_IR"
17294 storage_class> 1
17295 Remote config
17296 --------------------
17297 [remote]
17298 type = s3
17299 provider = AWS
17300 env_auth = false
17301 access_key_id = XXX
17302 secret_access_key = YYY
17303 region = us-east-1
17304 endpoint =
17305 location_constraint =
17306 acl = private
17307 server_side_encryption =
17308 storage_class =
17309 --------------------
17310 y) Yes this is OK
17311 e) Edit this remote
17312 d) Delete this remote
17313 y/e/d>
17314 Modified time
17316 The modified time is stored as metadata on the object as X-Amz-Meta-
17317 Mtime as floating point since the epoch, accurate to 1 ns.
17318 If the modification time needs to be updated rclone will attempt to
17320 perform a server side copy to update the modification if the object can
17321 be copied in a single part. In the case the object is larger than 5Gb
17322 or is in Glacier or Glacier Deep Archive storage the object will be up‐
17323 loaded rather than copied.
17324 Note that reading this from the object takes an additional HEAD request
17326 as the metadata isn't returned in object listings.
17327 Reducing costs
17329 Avoiding HEAD requests to read the modification time
17330 By default, rclone will use the modification time of objects stored in
17331 S3 for syncing. This is stored in object metadata which unfortunately
17332 takes an extra HEAD request to read which can be expensive (in time and
17333 money).
17334 The modification time is used by default for all operations that re‐
17336 quire checking the time a file was last updated. It allows rclone to
17337 treat the remote more like a true filesystem, but it is inefficient on
17338 S3 because it requires an extra API call to retrieve the metadata.
17339 The extra API calls can be avoided when syncing (using rclone sync or
17341 rclone copy) in a few different ways, each with its own tradeoffs.
17342 • --size-only
17344 • Only checks the size of files.
17346 • Uses no extra transactions.
17348 • If the file doesn't change size then rclone won't detect it has
17350 changed.
17351 • rclone sync --size-only /path/to/source s3:bucket
17353 • --checksum
17355 • Checks the size and MD5 checksum of files.
17357 • Uses no extra transactions.
17359 • The most accurate detection of changes possible.
17361 • Will cause the source to read an MD5 checksum which, if it is a lo‐
17363 cal disk, will cause lots of disk activity.
17364 • If the source and destination are both S3 this is the recommended
17366 flag to use for maximum efficiency.
17367 • rclone sync --checksum /path/to/source s3:bucket
17369 • --update --use-server-modtime
17371 • Uses no extra transactions.
17373 • Modification time becomes the time the object was uploaded.
17375 • For many operations this is sufficient to determine if it needs up‐
17377 loading.
17378 • Using --update along with --use-server-modtime, avoids the extra
17380 API call and uploads files whose local modification time is newer
17381 than the time it was last uploaded.
17382 • Files created with timestamps in the past will be missed by the
17384 sync.
17385 • rclone sync --update --use-server-modtime /path/to/source s3:bucket
17387 These flags can and should be used in combination with --fast-list -
17389 see below.
17390 If using rclone mount or any command using the VFS (eg rclone serve)
17392 commands then you might want to consider using the VFS flag --no-mod‐
17393 time which will stop rclone reading the modification time for every ob‐
17394 ject. You could also use --use-server-modtime if you are happy with
17395 the modification times of the objects being the time of upload.
17396 Avoiding GET requests to read directory listings
17398 Rclone's default directory traversal is to process each directory indi‐
17399 vidually. This takes one API call per directory. Using the --fast-
17400 list flag will read all info about the objects into memory first using
17401 a smaller number of API calls (one per 1000 objects). See the rclone
17402 docs (https://rclone.org/docs/#fast-list) for more details.
17403 rclone sync --fast-list --checksum /path/to/source s3:bucket
17405 --fast-list trades off API transactions for memory use. As a rough
17407 guide rclone uses 1k of memory per object stored, so using --fast-list
17408 on a sync of a million objects will use roughly 1 GiB of RAM.
17409 If you are only copying a small number of files into a big repository
17411 then using --no-traverse is a good idea. This finds objects directly
17412 instead of through directory listings. You can do a "top-up" sync very
17413 cheaply by using --max-age and --no-traverse to copy only recent files,
17414 eg
17415 rclone copy --max-age 24h --no-traverse /path/to/source s3:bucket
17417 You'd then do a full rclone sync less often.
17419 Note that --fast-list isn't required in the top-up sync.
17421 Avoiding HEAD requests after PUT
17423 By default, rclone will HEAD every object it uploads. It does this to
17424 check the object got uploaded correctly.
17425 You can disable this with the --s3-no-head option - see there for more
17427 details.
17428 Setting this flag increases the chance for undetected upload failures.
17430 Hashes
17432 For small objects which weren't uploaded as multipart uploads (objects
17433 sized below --s3-upload-cutoff if uploaded with rclone) rclone uses the
17434 ETag: header as an MD5 checksum.
17435 However for objects which were uploaded as multipart uploads or with
17437 server side encryption (SSE-AWS or SSE-C) the ETag header is no longer
17438 the MD5 sum of the data, so rclone adds an additional piece of metadata
17439 X-Amz-Meta-Md5chksum which is a base64 encoded MD5 hash (in the same
17440 format as is required for Content-MD5).
17441 For large objects, calculating this hash can take some time so the ad‐
17443 dition of this hash can be disabled with --s3-disable-checksum. This
17444 will mean that these objects do not have an MD5 checksum.
17445 Note that reading this from the object takes an additional HEAD request
17447 as the metadata isn't returned in object listings.
17448 Versions
17450 When bucket versioning is enabled (this can be done with rclone with
17451 the rclone backend versioning command) when rclone uploads a new ver‐
17452 sion of a file it creates a new version of it (https://docs.aws.ama‐
17453 zon.com/AmazonS3/latest/userguide/Versioning.html) Likewise when you
17454 delete a file, the old version will be marked hidden and still be
17455 available.
17456 Old versions of files, where available, are visible using the --s3-ver‐
17458 sions flag.
17459 It is also possible to view a bucket as it was at a certain point in
17461 time, using the --s3-version-at flag. This will show the file versions
17462 as they were at that time, showing files that have been deleted after‐
17463 wards, and hiding files that were created since.
17464 If you wish to remove all the old versions then you can use the rclone
17466 backend cleanup-hidden remote:bucket command which will delete all the
17467 old hidden versions of files, leaving the current ones intact. You can
17468 also supply a path and only old versions under that path will be delet‐
17469 ed, e.g. rclone backend cleanup-hidden remote:bucket/path/to/stuff.
17470 When you purge a bucket, the current and the old versions will be
17472 deleted then the bucket will be deleted.
17473 However delete will cause the current versions of the files to become
17475 hidden old versions.
17476 Here is a session showing the listing and retrieval of an old version
17478 followed by a cleanup of the old versions.
17479 Show current version and all the versions with --s3-versions flag.
17481 $ rclone -q ls s3:cleanup-test
17483 9 one.txt
17484 $ rclone -q --s3-versions ls s3:cleanup-test
17486 9 one.txt
17487 8 one-v2016-07-04-141032-000.txt
17488 16 one-v2016-07-04-141003-000.txt
17489 15 one-v2016-07-02-155621-000.txt
17490 Retrieve an old version
17492 $ rclone -q --s3-versions copy s3:cleanup-test/one-v2016-07-04-141003-000.txt /tmp
17494 $ ls -l /tmp/one-v2016-07-04-141003-000.txt
17496 -rw-rw-r-- 1 ncw ncw 16 Jul 2 17:46 /tmp/one-v2016-07-04-141003-000.txt
17497 Clean up all the old versions and show that they've gone.
17499 $ rclone -q backend cleanup-hidden s3:cleanup-test
17501 $ rclone -q ls s3:cleanup-test
17503 9 one.txt
17504 $ rclone -q --s3-versions ls s3:cleanup-test
17506 9 one.txt
17507 Cleanup
17509 If you run rclone cleanup s3:bucket then it will remove all pending
17510 multipart uploads older than 24 hours. You can use the -i flag to see
17511 exactly what it will do. If you want more control over the expiry date
17512 then run rclone backend cleanup s3:bucket -o max-age=1h to expire all
17513 uploads older than one hour. You can use rclone backend list-multi‐
17514 part-uploads s3:bucket to see the pending multipart uploads.
17515 Restricted filename characters
17517 S3 allows any valid UTF-8 string as a key.
17518 Invalid UTF-8 bytes will be replaced (https://rclone.org/overview/#in‐
17520 valid-utf8), as they can't be used in XML.
17521 The following characters are replaced since these are problematic when
17523 dealing with the REST API:
17524 Character Value Replacement
17526 ────────────────────────────────
17527 NUL 0x00 ␀
17528 / 0x2F ／
17529 The encoding will also encode these file names as they don't seem to
17531 work with the SDK properly:
17532 File name Replacement
17534 ────────────────────────
17535 . ．
17536 .. ．．
17537 Multipart uploads
17539 rclone supports multipart uploads with S3 which means that it can up‐
17540 load files bigger than 5 GiB.
17541 Note that files uploaded both with multipart upload and through crypt
17543 remotes do not have MD5 sums.
17544 rclone switches from single part uploads to multipart uploads at the
17546 point specified by --s3-upload-cutoff. This can be a maximum of 5 GiB
17547 and a minimum of 0 (ie always upload multipart files).
17548 The chunk sizes used in the multipart upload are specified by
17550 --s3-chunk-size and the number of chunks uploaded concurrently is spec‐
17551 ified by --s3-upload-concurrency.
17552 Multipart uploads will use --transfers * --s3-upload-concurrency *
17554 --s3-chunk-size extra memory. Single part uploads to not use extra
17555 memory.
17556 Single part transfers can be faster than multipart transfers or slower
17558 depending on your latency from S3 - the more latency, the more likely
17559 single part transfers will be faster.
17560 Increasing --s3-upload-concurrency will increase throughput (8 would be
17562 a sensible value) and increasing --s3-chunk-size also increases
17563 throughput (16M would be sensible). Increasing either of these will
17564 use more memory. The default values are high enough to gain most of
17565 the possible performance without using too much memory.
17566 Buckets and Regions
17568 With Amazon S3 you can list buckets (rclone lsd) using any region, but
17569 you can only access the content of a bucket from the region it was cre‐
17570 ated in. If you attempt to access a bucket from the wrong region, you
17571 will get an error, incorrect region, the bucket is not in 'XXX' region.
17572 Authentication
17574 There are a number of ways to supply rclone with a set of AWS creden‐
17575 tials, with and without using the environment.
17576 The different authentication methods are tried in this order:
17578 • Directly in the rclone configuration file (env_auth = false in the
17580 config file):
17581 • access_key_id and secret_access_key are required.
17583 • session_token can be optionally set when using AWS STS.
17585 • Runtime configuration (env_auth = true in the config file):
17587 • Export the following environment variables before running rclone:
17589 • Access Key ID: AWS_ACCESS_KEY_ID or AWS_ACCESS_KEY
17591 • Secret Access Key: AWS_SECRET_ACCESS_KEY or AWS_SECRET_KEY
17593 • Session Token: AWS_SESSION_TOKEN (optional)
17595 • Or, use a named profile (https://docs.aws.amazon.com/cli/lat‐
17597 est/userguide/cli-multiple-profiles.html):
17598 • Profile files are standard files used by AWS CLI tools
17600 • By default it will use the profile in your home directory (e.g.
17602 ~/.aws/credentials on unix based systems) file and the "default"
17603 profile, to change set these environment variables:
17604 • AWS_SHARED_CREDENTIALS_FILE to control which file.
17606 • AWS_PROFILE to control which profile to use.
17608 • Or, run rclone in an ECS task with an IAM role (AWS only).
17610 • Or, run rclone on an EC2 instance with an IAM role (AWS only).
17612 • Or, run rclone in an EKS pod with an IAM role that is associated
17614 with a service account (AWS only).
17615 If none of these option actually end up providing rclone with AWS cre‐
17617 dentials then S3 interaction will be non-authenticated (see below).
17618 S3 Permissions
17620 When using the sync subcommand of rclone the following minimum permis‐
17621 sions are required to be available on the bucket being written to:
17622 • ListBucket
17624 • DeleteObject
17626 • GetObject
17628 • PutObject
17630 • PutObjectACL
17632 When using the lsd subcommand, the ListAllMyBuckets permission is re‐
17634 quired.
17635 Example policy:
17637 {
17639 "Version": "2012-10-17",
17640 "Statement": [
17641 {
17642 "Effect": "Allow",
17643 "Principal": {
17644 "AWS": "arn:aws:iam::USER_SID:user/USER_NAME"
17645 },
17646 "Action": [
17647 "s3:ListBucket",
17648 "s3:DeleteObject",
17649 "s3:GetObject",
17650 "s3:PutObject",
17651 "s3:PutObjectAcl"
17652 ],
17653 "Resource": [
17654 "arn:aws:s3:::BUCKET_NAME/*",
17655 "arn:aws:s3:::BUCKET_NAME"
17656 ]
17657 },
17658 {
17659 "Effect": "Allow",
17660 "Action": "s3:ListAllMyBuckets",
17661 "Resource": "arn:aws:s3:::*"
17662 }
17663 ]
17664 }
17665 Notes on above:
17667 1. This is a policy that can be used when creating bucket. It assumes
17669 that USER_NAME has been created.
17670 2. The Resource entry must include both resource ARNs, as one implies
17672 the bucket and the other implies the bucket's objects.
17673 For reference, here's an Ansible script
17675 (https://gist.github.com/ebridges/ebfc9042dd7c756cd101cfa807b7ae2b)
17676 that will generate one or more buckets that will work with rclone sync.
17677 Key Management System (KMS)
17679 If you are using server-side encryption with KMS then you must make
17680 sure rclone is configured with server_side_encryption = aws:kms other‐
17681 wise you will find you can't transfer small objects - these will create
17682 checksum errors.
17683 Glacier and Glacier Deep Archive
17685 You can upload objects using the glacier storage class or transition
17686 them to glacier using a lifecycle policy (http://docs.aws.ama‐
17687 zon.com/AmazonS3/latest/user-guide/create-lifecycle.html). The bucket
17688 can still be synced or copied into normally, but if rclone tries to ac‐
17689 cess data from the glacier storage class you will see an error like be‐
17690 low.
17691 2017/09/11 19:07:43 Failed to sync: failed to open source object: Object in GLACIER, restore first: path/to/file
17693 In this case you need to restore (http://docs.aws.amazon.com/Ama‐
17695 zonS3/latest/user-guide/restore-archived-objects.html) the object(s) in
17696 question before using rclone.
17697 Note that rclone only speaks the S3 API it does not speak the Glacier
17699 Vault API, so rclone cannot directly access Glacier Vaults.
17700 Object-lock enabled S3 bucket
17702 According to AWS's documentation on S3 Object Lock
17703 (https://docs.aws.amazon.com/AmazonS3/latest/userguide/object-lock-
17704 overview.html#object-lock-permission):
17705 If you configure a default retention period on a bucket, re‐
17707 quests to upload objects in such a bucket must include the Con‐
17708 tent-MD5 header.
17709 As mentioned in the Hashes section, small files that are not uploaded
17711 as multipart, use a different tag, causing the upload to fail. A sim‐
17712 ple solution is to set the --s3-upload-cutoff 0 and force all the files
17713 to be uploaded as multipart.
17714 Standard options
17716 Here are the Standard options specific to s3 (Amazon S3 Compliant Stor‐
17717 age Providers including AWS, Alibaba, Ceph, China Mobile, Cloudflare,
17718 ArvanCloud, Digital Ocean, Dreamhost, Huawei OBS, IBM COS, IDrive e2,
17719 IONOS Cloud, Lyve Cloud, Minio, Netease, RackCorp, Scaleway, SeaweedFS,
17720 StackPath, Storj, Tencent COS, Qiniu and Wasabi).
17721 --s3-provider
17723 Choose your S3 provider.
17724 Properties:
17726 • Config: provider
17728 • Env Var: RCLONE_S3_PROVIDER
17730 • Type: string
17732 • Required: false
17734 • Examples:
17736 • "AWS"
17738 • Amazon Web Services (AWS) S3
17740 • "Alibaba"
17742 • Alibaba Cloud Object Storage System (OSS) formerly Aliyun
17744 • "Ceph"
17746 • Ceph Object Storage
17748 • "ChinaMobile"
17750 • China Mobile Ecloud Elastic Object Storage (EOS)
17752 • "Cloudflare"
17754 • Cloudflare R2 Storage
17756 • "ArvanCloud"
17758 • Arvan Cloud Object Storage (AOS)
17760 • "DigitalOcean"
17762 • Digital Ocean Spaces
17764 • "Dreamhost"
17766 • Dreamhost DreamObjects
17768 • "HuaweiOBS"
17770 • Huawei Object Storage Service
17772 • "IBMCOS"
17774 • IBM COS S3
17776 • "IDrive"
17778 • IDrive e2
17780 • "IONOS"
17782 • IONOS Cloud
17784 • "LyveCloud"
17786 • Seagate Lyve Cloud
17788 • "Minio"
17790 • Minio Object Storage
17792 • "Netease"
17794 • Netease Object Storage (NOS)
17796 • "RackCorp"
17798 • RackCorp Object Storage
17800 • "Scaleway"
17802 • Scaleway Object Storage
17804 • "SeaweedFS"
17806 • SeaweedFS S3
17808 • "StackPath"
17810 • StackPath Object Storage
17812 • "Storj"
17814 • Storj (S3 Compatible Gateway)
17816 • "TencentCOS"
17818 • Tencent Cloud Object Storage (COS)
17820 • "Wasabi"
17822 • Wasabi Object Storage
17824 • "Qiniu"
17826 • Qiniu Object Storage (Kodo)
17828 • "Other"
17830 • Any other S3 compatible provider
17832 --s3-env-auth
17834 Get AWS credentials from runtime (environment variables or EC2/ECS meta
17835 data if no env vars).
17836 Only applies if access_key_id and secret_access_key is blank.
17838 Properties:
17840 • Config: env_auth
17842 • Env Var: RCLONE_S3_ENV_AUTH
17844 • Type: bool
17846 • Default: false
17848 • Examples:
17850 • "false"
17852 • Enter AWS credentials in the next step.
17854 • "true"
17856 • Get AWS credentials from the environment (env vars or IAM).
17858 --s3-access-key-id
17860 AWS Access Key ID.
17861 Leave blank for anonymous access or runtime credentials.
17863 Properties:
17865 • Config: access_key_id
17867 • Env Var: RCLONE_S3_ACCESS_KEY_ID
17869 • Type: string
17871 • Required: false
17873 --s3-secret-access-key
17875 AWS Secret Access Key (password).
17876 Leave blank for anonymous access or runtime credentials.
17878 Properties:
17880 • Config: secret_access_key
17882 • Env Var: RCLONE_S3_SECRET_ACCESS_KEY
17884 • Type: string
17886 • Required: false
17888 --s3-region
17890 Region to connect to.
17891 Properties:
17893 • Config: region
17895 • Env Var: RCLONE_S3_REGION
17897 • Provider: AWS
17899 • Type: string
17901 • Required: false
17903 • Examples:
17905 • "us-east-1"
17907 • The default endpoint - a good choice if you are unsure.
17909 • US Region, Northern Virginia, or Pacific Northwest.
17911 • Leave location constraint empty.
17913 • "us-east-2"
17915 • US East (Ohio) Region.
17917 • Needs location constraint us-east-2.
17919 • "us-west-1"
17921 • US West (Northern California) Region.
17923 • Needs location constraint us-west-1.
17925 • "us-west-2"
17927 • US West (Oregon) Region.
17929 • Needs location constraint us-west-2.
17931 • "ca-central-1"
17933 • Canada (Central) Region.
17935 • Needs location constraint ca-central-1.
17937 • "eu-west-1"
17939 • EU (Ireland) Region.
17941 • Needs location constraint EU or eu-west-1.
17943 • "eu-west-2"
17945 • EU (London) Region.
17947 • Needs location constraint eu-west-2.
17949 • "eu-west-3"
17951 • EU (Paris) Region.
17953 • Needs location constraint eu-west-3.
17955 • "eu-north-1"
17957 • EU (Stockholm) Region.
17959 • Needs location constraint eu-north-1.
17961 • "eu-south-1"
17963 • EU (Milan) Region.
17965 • Needs location constraint eu-south-1.
17967 • "eu-central-1"
17969 • EU (Frankfurt) Region.
17971 • Needs location constraint eu-central-1.
17973 • "ap-southeast-1"
17975 • Asia Pacific (Singapore) Region.
17977 • Needs location constraint ap-southeast-1.
17979 • "ap-southeast-2"
17981 • Asia Pacific (Sydney) Region.
17983 • Needs location constraint ap-southeast-2.
17985 • "ap-northeast-1"
17987 • Asia Pacific (Tokyo) Region.
17989 • Needs location constraint ap-northeast-1.
17991 • "ap-northeast-2"
17993 • Asia Pacific (Seoul).
17995 • Needs location constraint ap-northeast-2.
17997 • "ap-northeast-3"
17999 • Asia Pacific (Osaka-Local).
18001 • Needs location constraint ap-northeast-3.
18003 • "ap-south-1"
18005 • Asia Pacific (Mumbai).
18007 • Needs location constraint ap-south-1.
18009 • "ap-east-1"
18011 • Asia Pacific (Hong Kong) Region.
18013 • Needs location constraint ap-east-1.
18015 • "sa-east-1"
18017 • South America (Sao Paulo) Region.
18019 • Needs location constraint sa-east-1.
18021 • "me-south-1"
18023 • Middle East (Bahrain) Region.
18025 • Needs location constraint me-south-1.
18027 • "af-south-1"
18029 • Africa (Cape Town) Region.
18031 • Needs location constraint af-south-1.
18033 • "cn-north-1"
18035 • China (Beijing) Region.
18037 • Needs location constraint cn-north-1.
18039 • "cn-northwest-1"
18041 • China (Ningxia) Region.
18043 • Needs location constraint cn-northwest-1.
18045 • "us-gov-east-1"
18047 • AWS GovCloud (US-East) Region.
18049 • Needs location constraint us-gov-east-1.
18051 • "us-gov-west-1"
18053 • AWS GovCloud (US) Region.
18055 • Needs location constraint us-gov-west-1.
18057 --s3-region
18059 region - the location where your bucket will be created and your data
18060 stored.
18061 Properties:
18063 • Config: region
18065 • Env Var: RCLONE_S3_REGION
18067 • Provider: RackCorp
18069 • Type: string
18071 • Required: false
18073 • Examples:
18075 • "global"
18077 • Global CDN (All locations) Region
18079 • "au"
18081 • Australia (All states)
18083 • "au-nsw"
18085 • NSW (Australia) Region
18087 • "au-qld"
18089 • QLD (Australia) Region
18091 • "au-vic"
18093 • VIC (Australia) Region
18095 • "au-wa"
18097 • Perth (Australia) Region
18099 • "ph"
18101 • Manila (Philippines) Region
18103 • "th"
18105 • Bangkok (Thailand) Region
18107 • "hk"
18109 • HK (Hong Kong) Region
18111 • "mn"
18113 • Ulaanbaatar (Mongolia) Region
18115 • "kg"
18117 • Bishkek (Kyrgyzstan) Region
18119 • "id"
18121 • Jakarta (Indonesia) Region
18123 • "jp"
18125 • Tokyo (Japan) Region
18127 • "sg"
18129 • SG (Singapore) Region
18131 • "de"
18133 • Frankfurt (Germany) Region
18135 • "us"
18137 • USA (AnyCast) Region
18139 • "us-east-1"
18141 • New York (USA) Region
18143 • "us-west-1"
18145 • Freemont (USA) Region
18147 • "nz"
18149 • Auckland (New Zealand) Region
18151 --s3-region
18153 Region to connect to.
18154 Properties:
18156 • Config: region
18158 • Env Var: RCLONE_S3_REGION
18160 • Provider: Scaleway
18162 • Type: string
18164 • Required: false
18166 • Examples:
18168 • "nl-ams"
18170 • Amsterdam, The Netherlands
18172 • "fr-par"
18174 • Paris, France
18176 • "pl-waw"
18178 • Warsaw, Poland
18180 --s3-region
18182 Region to connect to. - the location where your bucket will be created
18183 and your data stored. Need bo be same with your endpoint.
18184 Properties:
18186 • Config: region
18188 • Env Var: RCLONE_S3_REGION
18190 • Provider: HuaweiOBS
18192 • Type: string
18194 • Required: false
18196 • Examples:
18198 • "af-south-1"
18200 • AF-Johannesburg
18202 • "ap-southeast-2"
18204 • AP-Bangkok
18206 • "ap-southeast-3"
18208 • AP-Singapore
18210 • "cn-east-3"
18212 • CN East-Shanghai1
18214 • "cn-east-2"
18216 • CN East-Shanghai2
18218 • "cn-north-1"
18220 • CN North-Beijing1
18222 • "cn-north-4"
18224 • CN North-Beijing4
18226 • "cn-south-1"
18228 • CN South-Guangzhou
18230 • "ap-southeast-1"
18232 • CN-Hong Kong
18234 • "sa-argentina-1"
18236 • LA-Buenos Aires1
18238 • "sa-peru-1"
18240 • LA-Lima1
18242 • "na-mexico-1"
18244 • LA-Mexico City1
18246 • "sa-chile-1"
18248 • LA-Santiago2
18250 • "sa-brazil-1"
18252 • LA-Sao Paulo1
18254 • "ru-northwest-2"
18256 • RU-Moscow2
18258 --s3-region
18260 Region to connect to.
18261 Properties:
18263 • Config: region
18265 • Env Var: RCLONE_S3_REGION
18267 • Provider: Cloudflare
18269 • Type: string
18271 • Required: false
18273 • Examples:
18275 • "auto"
18277 • R2 buckets are automatically distributed across Cloudflare's data
18279 centers for low latency.
18280 --s3-region
18282 Region to connect to.
18283 Properties:
18285 • Config: region
18287 • Env Var: RCLONE_S3_REGION
18289 • Provider: Qiniu
18291 • Type: string
18293 • Required: false
18295 • Examples:
18297 • "cn-east-1"
18299 • The default endpoint - a good choice if you are unsure.
18301 • East China Region 1.
18303 • Needs location constraint cn-east-1.
18305 • "cn-east-2"
18307 • East China Region 2.
18309 • Needs location constraint cn-east-2.
18311 • "cn-north-1"
18313 • North China Region 1.
18315 • Needs location constraint cn-north-1.
18317 • "cn-south-1"
18319 • South China Region 1.
18321 • Needs location constraint cn-south-1.
18323 • "us-north-1"
18325 • North America Region.
18327 • Needs location constraint us-north-1.
18329 • "ap-southeast-1"
18331 • Southeast Asia Region 1.
18333 • Needs location constraint ap-southeast-1.
18335 • "ap-northeast-1"
18337 • Northeast Asia Region 1.
18339 • Needs location constraint ap-northeast-1.
18341 --s3-region
18343 Region where your bucket will be created and your data stored.
18344 Properties:
18346 • Config: region
18348 • Env Var: RCLONE_S3_REGION
18350 • Provider: IONOS
18352 • Type: string
18354 • Required: false
18356 • Examples:
18358 • "de"
18360 • Frankfurt, Germany
18362 • "eu-central-2"
18364 • Berlin, Germany
18366 • "eu-south-2"
18368 • Logrono, Spain
18370 --s3-region
18372 Region to connect to.
18373 Leave blank if you are using an S3 clone and you don't have a region.
18375 Properties:
18377 • Config: region
18379 • Env Var: RCLONE_S3_REGION
18381 • Provider: !AWS,Alibaba,ChinaMobile,Cloudflare,IONOS,Arvan‐
18383 Cloud,Qiniu,RackCorp,Scaleway,Storj,TencentCOS,HuaweiOBS,IDrive
18384 • Type: string
18386 • Required: false
18388 • Examples:
18390 • ""
18392 • Use this if unsure.
18394 • Will use v4 signatures and an empty region.
18396 • "other-v2-signature"
18398 • Use this only if v4 signatures don't work.
18400 • E.g. pre Jewel/v10 CEPH.
18402 --s3-endpoint
18404 Endpoint for S3 API.
18405 Leave blank if using AWS to use the default endpoint for the region.
18407 Properties:
18409 • Config: endpoint
18411 • Env Var: RCLONE_S3_ENDPOINT
18413 • Provider: AWS
18415 • Type: string
18417 • Required: false
18419 --s3-endpoint
18421 Endpoint for China Mobile Ecloud Elastic Object Storage (EOS) API.
18422 Properties:
18424 • Config: endpoint
18426 • Env Var: RCLONE_S3_ENDPOINT
18428 • Provider: ChinaMobile
18430 • Type: string
18432 • Required: false
18434 • Examples:
18436 • "eos-wuxi-1.cmecloud.cn"
18438 • The default endpoint - a good choice if you are unsure.
18440 • East China (Suzhou)
18442 • "eos-jinan-1.cmecloud.cn"
18444 • East China (Jinan)
18446 • "eos-ningbo-1.cmecloud.cn"
18448 • East China (Hangzhou)
18450 • "eos-shanghai-1.cmecloud.cn"
18452 • East China (Shanghai-1)
18454 • "eos-zhengzhou-1.cmecloud.cn"
18456 • Central China (Zhengzhou)
18458 • "eos-hunan-1.cmecloud.cn"
18460 • Central China (Changsha-1)
18462 • "eos-zhuzhou-1.cmecloud.cn"
18464 • Central China (Changsha-2)
18466 • "eos-guangzhou-1.cmecloud.cn"
18468 • South China (Guangzhou-2)
18470 • "eos-dongguan-1.cmecloud.cn"
18472 • South China (Guangzhou-3)
18474 • "eos-beijing-1.cmecloud.cn"
18476 • North China (Beijing-1)
18478 • "eos-beijing-2.cmecloud.cn"
18480 • North China (Beijing-2)
18482 • "eos-beijing-4.cmecloud.cn"
18484 • North China (Beijing-3)
18486 • "eos-huhehaote-1.cmecloud.cn"
18488 • North China (Huhehaote)
18490 • "eos-chengdu-1.cmecloud.cn"
18492 • Southwest China (Chengdu)
18494 • "eos-chongqing-1.cmecloud.cn"
18496 • Southwest China (Chongqing)
18498 • "eos-guiyang-1.cmecloud.cn"
18500 • Southwest China (Guiyang)
18502 • "eos-xian-1.cmecloud.cn"
18504 • Nouthwest China (Xian)
18506 • "eos-yunnan.cmecloud.cn"
18508 • Yunnan China (Kunming)
18510 • "eos-yunnan-2.cmecloud.cn"
18512 • Yunnan China (Kunming-2)
18514 • "eos-tianjin-1.cmecloud.cn"
18516 • Tianjin China (Tianjin)
18518 • "eos-jilin-1.cmecloud.cn"
18520 • Jilin China (Changchun)
18522 • "eos-hubei-1.cmecloud.cn"
18524 • Hubei China (Xiangyan)
18526 • "eos-jiangxi-1.cmecloud.cn"
18528 • Jiangxi China (Nanchang)
18530 • "eos-gansu-1.cmecloud.cn"
18532 • Gansu China (Lanzhou)
18534 • "eos-shanxi-1.cmecloud.cn"
18536 • Shanxi China (Taiyuan)
18538 • "eos-liaoning-1.cmecloud.cn"
18540 • Liaoning China (Shenyang)
18542 • "eos-hebei-1.cmecloud.cn"
18544 • Hebei China (Shijiazhuang)
18546 • "eos-fujian-1.cmecloud.cn"
18548 • Fujian China (Xiamen)
18550 • "eos-guangxi-1.cmecloud.cn"
18552 • Guangxi China (Nanning)
18554 • "eos-anhui-1.cmecloud.cn"
18556 • Anhui China (Huainan)
18558 --s3-endpoint
18560 Endpoint for Arvan Cloud Object Storage (AOS) API.
18561 Properties:
18563 • Config: endpoint
18565 • Env Var: RCLONE_S3_ENDPOINT
18567 • Provider: ArvanCloud
18569 • Type: string
18571 • Required: false
18573 • Examples:
18575 • "s3.ir-thr-at1.arvanstorage.com"
18577 • The default endpoint - a good choice if you are unsure.
18579 • Tehran Iran (Asiatech)
18581 • "s3.ir-tbz-sh1.arvanstorage.com"
18583 • Tabriz Iran (Shahriar)
18585 --s3-endpoint
18587 Endpoint for IBM COS S3 API.
18588 Specify if using an IBM COS On Premise.
18590 Properties:
18592 • Config: endpoint
18594 • Env Var: RCLONE_S3_ENDPOINT
18596 • Provider: IBMCOS
18598 • Type: string
18600 • Required: false
18602 • Examples:
18604 • "s3.us.cloud-object-storage.appdomain.cloud"
18606 • US Cross Region Endpoint
18608 • "s3.dal.us.cloud-object-storage.appdomain.cloud"
18610 • US Cross Region Dallas Endpoint
18612 • "s3.wdc.us.cloud-object-storage.appdomain.cloud"
18614 • US Cross Region Washington DC Endpoint
18616 • "s3.sjc.us.cloud-object-storage.appdomain.cloud"
18618 • US Cross Region San Jose Endpoint
18620 • "s3.private.us.cloud-object-storage.appdomain.cloud"
18622 • US Cross Region Private Endpoint
18624 • "s3.private.dal.us.cloud-object-storage.appdomain.cloud"
18626 • US Cross Region Dallas Private Endpoint
18628 • "s3.private.wdc.us.cloud-object-storage.appdomain.cloud"
18630 • US Cross Region Washington DC Private Endpoint
18632 • "s3.private.sjc.us.cloud-object-storage.appdomain.cloud"
18634 • US Cross Region San Jose Private Endpoint
18636 • "s3.us-east.cloud-object-storage.appdomain.cloud"
18638 • US Region East Endpoint
18640 • "s3.private.us-east.cloud-object-storage.appdomain.cloud"
18642 • US Region East Private Endpoint
18644 • "s3.us-south.cloud-object-storage.appdomain.cloud"
18646 • US Region South Endpoint
18648 • "s3.private.us-south.cloud-object-storage.appdomain.cloud"
18650 • US Region South Private Endpoint
18652 • "s3.eu.cloud-object-storage.appdomain.cloud"
18654 • EU Cross Region Endpoint
18656 • "s3.fra.eu.cloud-object-storage.appdomain.cloud"
18658 • EU Cross Region Frankfurt Endpoint
18660 • "s3.mil.eu.cloud-object-storage.appdomain.cloud"
18662 • EU Cross Region Milan Endpoint
18664 • "s3.ams.eu.cloud-object-storage.appdomain.cloud"
18666 • EU Cross Region Amsterdam Endpoint
18668 • "s3.private.eu.cloud-object-storage.appdomain.cloud"
18670 • EU Cross Region Private Endpoint
18672 • "s3.private.fra.eu.cloud-object-storage.appdomain.cloud"
18674 • EU Cross Region Frankfurt Private Endpoint
18676 • "s3.private.mil.eu.cloud-object-storage.appdomain.cloud"
18678 • EU Cross Region Milan Private Endpoint
18680 • "s3.private.ams.eu.cloud-object-storage.appdomain.cloud"
18682 • EU Cross Region Amsterdam Private Endpoint
18684 • "s3.eu-gb.cloud-object-storage.appdomain.cloud"
18686 • Great Britain Endpoint
18688 • "s3.private.eu-gb.cloud-object-storage.appdomain.cloud"
18690 • Great Britain Private Endpoint
18692 • "s3.eu-de.cloud-object-storage.appdomain.cloud"
18694 • EU Region DE Endpoint
18696 • "s3.private.eu-de.cloud-object-storage.appdomain.cloud"
18698 • EU Region DE Private Endpoint
18700 • "s3.ap.cloud-object-storage.appdomain.cloud"
18702 • APAC Cross Regional Endpoint
18704 • "s3.tok.ap.cloud-object-storage.appdomain.cloud"
18706 • APAC Cross Regional Tokyo Endpoint
18708 • "s3.hkg.ap.cloud-object-storage.appdomain.cloud"
18710 • APAC Cross Regional HongKong Endpoint
18712 • "s3.seo.ap.cloud-object-storage.appdomain.cloud"
18714 • APAC Cross Regional Seoul Endpoint
18716 • "s3.private.ap.cloud-object-storage.appdomain.cloud"
18718 • APAC Cross Regional Private Endpoint
18720 • "s3.private.tok.ap.cloud-object-storage.appdomain.cloud"
18722 • APAC Cross Regional Tokyo Private Endpoint
18724 • "s3.private.hkg.ap.cloud-object-storage.appdomain.cloud"
18726 • APAC Cross Regional HongKong Private Endpoint
18728 • "s3.private.seo.ap.cloud-object-storage.appdomain.cloud"
18730 • APAC Cross Regional Seoul Private Endpoint
18732 • "s3.jp-tok.cloud-object-storage.appdomain.cloud"
18734 • APAC Region Japan Endpoint
18736 • "s3.private.jp-tok.cloud-object-storage.appdomain.cloud"
18738 • APAC Region Japan Private Endpoint
18740 • "s3.au-syd.cloud-object-storage.appdomain.cloud"
18742 • APAC Region Australia Endpoint
18744 • "s3.private.au-syd.cloud-object-storage.appdomain.cloud"
18746 • APAC Region Australia Private Endpoint
18748 • "s3.ams03.cloud-object-storage.appdomain.cloud"
18750 • Amsterdam Single Site Endpoint
18752 • "s3.private.ams03.cloud-object-storage.appdomain.cloud"
18754 • Amsterdam Single Site Private Endpoint
18756 • "s3.che01.cloud-object-storage.appdomain.cloud"
18758 • Chennai Single Site Endpoint
18760 • "s3.private.che01.cloud-object-storage.appdomain.cloud"
18762 • Chennai Single Site Private Endpoint
18764 • "s3.mel01.cloud-object-storage.appdomain.cloud"
18766 • Melbourne Single Site Endpoint
18768 • "s3.private.mel01.cloud-object-storage.appdomain.cloud"
18770 • Melbourne Single Site Private Endpoint
18772 • "s3.osl01.cloud-object-storage.appdomain.cloud"
18774 • Oslo Single Site Endpoint
18776 • "s3.private.osl01.cloud-object-storage.appdomain.cloud"
18778 • Oslo Single Site Private Endpoint
18780 • "s3.tor01.cloud-object-storage.appdomain.cloud"
18782 • Toronto Single Site Endpoint
18784 • "s3.private.tor01.cloud-object-storage.appdomain.cloud"
18786 • Toronto Single Site Private Endpoint
18788 • "s3.seo01.cloud-object-storage.appdomain.cloud"
18790 • Seoul Single Site Endpoint
18792 • "s3.private.seo01.cloud-object-storage.appdomain.cloud"
18794 • Seoul Single Site Private Endpoint
18796 • "s3.mon01.cloud-object-storage.appdomain.cloud"
18798 • Montreal Single Site Endpoint
18800 • "s3.private.mon01.cloud-object-storage.appdomain.cloud"
18802 • Montreal Single Site Private Endpoint
18804 • "s3.mex01.cloud-object-storage.appdomain.cloud"
18806 • Mexico Single Site Endpoint
18808 • "s3.private.mex01.cloud-object-storage.appdomain.cloud"
18810 • Mexico Single Site Private Endpoint
18812 • "s3.sjc04.cloud-object-storage.appdomain.cloud"
18814 • San Jose Single Site Endpoint
18816 • "s3.private.sjc04.cloud-object-storage.appdomain.cloud"
18818 • San Jose Single Site Private Endpoint
18820 • "s3.mil01.cloud-object-storage.appdomain.cloud"
18822 • Milan Single Site Endpoint
18824 • "s3.private.mil01.cloud-object-storage.appdomain.cloud"
18826 • Milan Single Site Private Endpoint
18828 • "s3.hkg02.cloud-object-storage.appdomain.cloud"
18830 • Hong Kong Single Site Endpoint
18832 • "s3.private.hkg02.cloud-object-storage.appdomain.cloud"
18834 • Hong Kong Single Site Private Endpoint
18836 • "s3.par01.cloud-object-storage.appdomain.cloud"
18838 • Paris Single Site Endpoint
18840 • "s3.private.par01.cloud-object-storage.appdomain.cloud"
18842 • Paris Single Site Private Endpoint
18844 • "s3.sng01.cloud-object-storage.appdomain.cloud"
18846 • Singapore Single Site Endpoint
18848 • "s3.private.sng01.cloud-object-storage.appdomain.cloud"
18850 • Singapore Single Site Private Endpoint
18852 --s3-endpoint
18854 Endpoint for IONOS S3 Object Storage.
18855 Specify the endpoint from the same region.
18857 Properties:
18859 • Config: endpoint
18861 • Env Var: RCLONE_S3_ENDPOINT
18863 • Provider: IONOS
18865 • Type: string
18867 • Required: false
18869 • Examples:
18871 • "s3-eu-central-1.ionoscloud.com"
18873 • Frankfurt, Germany
18875 • "s3-eu-central-2.ionoscloud.com"
18877 • Berlin, Germany
18879 • "s3-eu-south-2.ionoscloud.com"
18881 • Logrono, Spain
18883 --s3-endpoint
18885 Endpoint for OSS API.
18886 Properties:
18888 • Config: endpoint
18890 • Env Var: RCLONE_S3_ENDPOINT
18892 • Provider: Alibaba
18894 • Type: string
18896 • Required: false
18898 • Examples:
18900 • "oss-accelerate.aliyuncs.com"
18902 • Global Accelerate
18904 • "oss-accelerate-overseas.aliyuncs.com"
18906 • Global Accelerate (outside mainland China)
18908 • "oss-cn-hangzhou.aliyuncs.com"
18910 • East China 1 (Hangzhou)
18912 • "oss-cn-shanghai.aliyuncs.com"
18914 • East China 2 (Shanghai)
18916 • "oss-cn-qingdao.aliyuncs.com"
18918 • North China 1 (Qingdao)
18920 • "oss-cn-beijing.aliyuncs.com"
18922 • North China 2 (Beijing)
18924 • "oss-cn-zhangjiakou.aliyuncs.com"
18926 • North China 3 (Zhangjiakou)
18928 • "oss-cn-huhehaote.aliyuncs.com"
18930 • North China 5 (Hohhot)
18932 • "oss-cn-wulanchabu.aliyuncs.com"
18934 • North China 6 (Ulanqab)
18936 • "oss-cn-shenzhen.aliyuncs.com"
18938 • South China 1 (Shenzhen)
18940 • "oss-cn-heyuan.aliyuncs.com"
18942 • South China 2 (Heyuan)
18944 • "oss-cn-guangzhou.aliyuncs.com"
18946 • South China 3 (Guangzhou)
18948 • "oss-cn-chengdu.aliyuncs.com"
18950 • West China 1 (Chengdu)
18952 • "oss-cn-hongkong.aliyuncs.com"
18954 • Hong Kong (Hong Kong)
18956 • "oss-us-west-1.aliyuncs.com"
18958 • US West 1 (Silicon Valley)
18960 • "oss-us-east-1.aliyuncs.com"
18962 • US East 1 (Virginia)
18964 • "oss-ap-southeast-1.aliyuncs.com"
18966 • Southeast Asia Southeast 1 (Singapore)
18968 • "oss-ap-southeast-2.aliyuncs.com"
18970 • Asia Pacific Southeast 2 (Sydney)
18972 • "oss-ap-southeast-3.aliyuncs.com"
18974 • Southeast Asia Southeast 3 (Kuala Lumpur)
18976 • "oss-ap-southeast-5.aliyuncs.com"
18978 • Asia Pacific Southeast 5 (Jakarta)
18980 • "oss-ap-northeast-1.aliyuncs.com"
18982 • Asia Pacific Northeast 1 (Japan)
18984 • "oss-ap-south-1.aliyuncs.com"
18986 • Asia Pacific South 1 (Mumbai)
18988 • "oss-eu-central-1.aliyuncs.com"
18990 • Central Europe 1 (Frankfurt)
18992 • "oss-eu-west-1.aliyuncs.com"
18994 • West Europe (London)
18996 • "oss-me-east-1.aliyuncs.com"
18998 • Middle East 1 (Dubai)
19000 --s3-endpoint
19002 Endpoint for OBS API.
19003 Properties:
19005 • Config: endpoint
19007 • Env Var: RCLONE_S3_ENDPOINT
19009 • Provider: HuaweiOBS
19011 • Type: string
19013 • Required: false
19015 • Examples:
19017 • "obs.af-south-1.myhuaweicloud.com"
19019 • AF-Johannesburg
19021 • "obs.ap-southeast-2.myhuaweicloud.com"
19023 • AP-Bangkok
19025 • "obs.ap-southeast-3.myhuaweicloud.com"
19027 • AP-Singapore
19029 • "obs.cn-east-3.myhuaweicloud.com"
19031 • CN East-Shanghai1
19033 • "obs.cn-east-2.myhuaweicloud.com"
19035 • CN East-Shanghai2
19037 • "obs.cn-north-1.myhuaweicloud.com"
19039 • CN North-Beijing1
19041 • "obs.cn-north-4.myhuaweicloud.com"
19043 • CN North-Beijing4
19045 • "obs.cn-south-1.myhuaweicloud.com"
19047 • CN South-Guangzhou
19049 • "obs.ap-southeast-1.myhuaweicloud.com"
19051 • CN-Hong Kong
19053 • "obs.sa-argentina-1.myhuaweicloud.com"
19055 • LA-Buenos Aires1
19057 • "obs.sa-peru-1.myhuaweicloud.com"
19059 • LA-Lima1
19061 • "obs.na-mexico-1.myhuaweicloud.com"
19063 • LA-Mexico City1
19065 • "obs.sa-chile-1.myhuaweicloud.com"
19067 • LA-Santiago2
19069 • "obs.sa-brazil-1.myhuaweicloud.com"
19071 • LA-Sao Paulo1
19073 • "obs.ru-northwest-2.myhuaweicloud.com"
19075 • RU-Moscow2
19077 --s3-endpoint
19079 Endpoint for Scaleway Object Storage.
19080 Properties:
19082 • Config: endpoint
19084 • Env Var: RCLONE_S3_ENDPOINT
19086 • Provider: Scaleway
19088 • Type: string
19090 • Required: false
19092 • Examples:
19094 • "s3.nl-ams.scw.cloud"
19096 • Amsterdam Endpoint
19098 • "s3.fr-par.scw.cloud"
19100 • Paris Endpoint
19102 • "s3.pl-waw.scw.cloud"
19104 • Warsaw Endpoint
19106 --s3-endpoint
19108 Endpoint for StackPath Object Storage.
19109 Properties:
19111 • Config: endpoint
19113 • Env Var: RCLONE_S3_ENDPOINT
19115 • Provider: StackPath
19117 • Type: string
19119 • Required: false
19121 • Examples:
19123 • "s3.us-east-2.stackpathstorage.com"
19125 • US East Endpoint
19127 • "s3.us-west-1.stackpathstorage.com"
19129 • US West Endpoint
19131 • "s3.eu-central-1.stackpathstorage.com"
19133 • EU Endpoint
19135 --s3-endpoint
19137 Endpoint of the Shared Gateway.
19138 Properties:
19140 • Config: endpoint
19142 • Env Var: RCLONE_S3_ENDPOINT
19144 • Provider: Storj
19146 • Type: string
19148 • Required: false
19150 • Examples:
19152 • "gateway.eu1.storjshare.io"
19154 • EU1 Shared Gateway
19156 • "gateway.us1.storjshare.io"
19158 • US1 Shared Gateway
19160 • "gateway.ap1.storjshare.io"
19162 • Asia-Pacific Shared Gateway
19164 --s3-endpoint
19166 Endpoint for Tencent COS API.
19167 Properties:
19169 • Config: endpoint
19171 • Env Var: RCLONE_S3_ENDPOINT
19173 • Provider: TencentCOS
19175 • Type: string
19177 • Required: false
19179 • Examples:
19181 • "cos.ap-beijing.myqcloud.com"
19183 • Beijing Region
19185 • "cos.ap-nanjing.myqcloud.com"
19187 • Nanjing Region
19189 • "cos.ap-shanghai.myqcloud.com"
19191 • Shanghai Region
19193 • "cos.ap-guangzhou.myqcloud.com"
19195 • Guangzhou Region
19197 • "cos.ap-nanjing.myqcloud.com"
19199 • Nanjing Region
19201 • "cos.ap-chengdu.myqcloud.com"
19203 • Chengdu Region
19205 • "cos.ap-chongqing.myqcloud.com"
19207 • Chongqing Region
19209 • "cos.ap-hongkong.myqcloud.com"
19211 • Hong Kong (China) Region
19213 • "cos.ap-singapore.myqcloud.com"
19215 • Singapore Region
19217 • "cos.ap-mumbai.myqcloud.com"
19219 • Mumbai Region
19221 • "cos.ap-seoul.myqcloud.com"
19223 • Seoul Region
19225 • "cos.ap-bangkok.myqcloud.com"
19227 • Bangkok Region
19229 • "cos.ap-tokyo.myqcloud.com"
19231 • Tokyo Region
19233 • "cos.na-siliconvalley.myqcloud.com"
19235 • Silicon Valley Region
19237 • "cos.na-ashburn.myqcloud.com"
19239 • Virginia Region
19241 • "cos.na-toronto.myqcloud.com"
19243 • Toronto Region
19245 • "cos.eu-frankfurt.myqcloud.com"
19247 • Frankfurt Region
19249 • "cos.eu-moscow.myqcloud.com"
19251 • Moscow Region
19253 • "cos.accelerate.myqcloud.com"
19255 • Use Tencent COS Accelerate Endpoint
19257 --s3-endpoint
19259 Endpoint for RackCorp Object Storage.
19260 Properties:
19262 • Config: endpoint
19264 • Env Var: RCLONE_S3_ENDPOINT
19266 • Provider: RackCorp
19268 • Type: string
19270 • Required: false
19272 • Examples:
19274 • "s3.rackcorp.com"
19276 • Global (AnyCast) Endpoint
19278 • "au.s3.rackcorp.com"
19280 • Australia (Anycast) Endpoint
19282 • "au-nsw.s3.rackcorp.com"
19284 • Sydney (Australia) Endpoint
19286 • "au-qld.s3.rackcorp.com"
19288 • Brisbane (Australia) Endpoint
19290 • "au-vic.s3.rackcorp.com"
19292 • Melbourne (Australia) Endpoint
19294 • "au-wa.s3.rackcorp.com"
19296 • Perth (Australia) Endpoint
19298 • "ph.s3.rackcorp.com"
19300 • Manila (Philippines) Endpoint
19302 • "th.s3.rackcorp.com"
19304 • Bangkok (Thailand) Endpoint
19306 • "hk.s3.rackcorp.com"
19308 • HK (Hong Kong) Endpoint
19310 • "mn.s3.rackcorp.com"
19312 • Ulaanbaatar (Mongolia) Endpoint
19314 • "kg.s3.rackcorp.com"
19316 • Bishkek (Kyrgyzstan) Endpoint
19318 • "id.s3.rackcorp.com"
19320 • Jakarta (Indonesia) Endpoint
19322 • "jp.s3.rackcorp.com"
19324 • Tokyo (Japan) Endpoint
19326 • "sg.s3.rackcorp.com"
19328 • SG (Singapore) Endpoint
19330 • "de.s3.rackcorp.com"
19332 • Frankfurt (Germany) Endpoint
19334 • "us.s3.rackcorp.com"
19336 • USA (AnyCast) Endpoint
19338 • "us-east-1.s3.rackcorp.com"
19340 • New York (USA) Endpoint
19342 • "us-west-1.s3.rackcorp.com"
19344 • Freemont (USA) Endpoint
19346 • "nz.s3.rackcorp.com"
19348 • Auckland (New Zealand) Endpoint
19350 --s3-endpoint
19352 Endpoint for Qiniu Object Storage.
19353 Properties:
19355 • Config: endpoint
19357 • Env Var: RCLONE_S3_ENDPOINT
19359 • Provider: Qiniu
19361 • Type: string
19363 • Required: false
19365 • Examples:
19367 • "s3-cn-east-1.qiniucs.com"
19369 • East China Endpoint 1
19371 • "s3-cn-east-2.qiniucs.com"
19373 • East China Endpoint 2
19375 • "s3-cn-north-1.qiniucs.com"
19377 • North China Endpoint 1
19379 • "s3-cn-south-1.qiniucs.com"
19381 • South China Endpoint 1
19383 • "s3-us-north-1.qiniucs.com"
19385 • North America Endpoint 1
19387 • "s3-ap-southeast-1.qiniucs.com"
19389 • Southeast Asia Endpoint 1
19391 • "s3-ap-northeast-1.qiniucs.com"
19393 • Northeast Asia Endpoint 1
19395 --s3-endpoint
19397 Endpoint for S3 API.
19398 Required when using an S3 clone.
19400 Properties:
19402 • Config: endpoint
19404 • Env Var: RCLONE_S3_ENDPOINT
19406 • Provider: !AWS,IBMCOS,IDrive,IONOS,TencentCOS,HuaweiOBS,Alibaba,Chi‐
19408 naMobile,ArvanCloud,Scaleway,StackPath,Storj,RackCorp,Qiniu
19409 • Type: string
19411 • Required: false
19413 • Examples:
19415 • "objects-us-east-1.dream.io"
19417 • Dream Objects endpoint
19419 • "nyc3.digitaloceanspaces.com"
19421 • Digital Ocean Spaces New York 3
19423 • "ams3.digitaloceanspaces.com"
19425 • Digital Ocean Spaces Amsterdam 3
19427 • "sgp1.digitaloceanspaces.com"
19429 • Digital Ocean Spaces Singapore 1
19431 • "localhost:8333"
19433 • SeaweedFS S3 localhost
19435 • "s3.us-east-1.lyvecloud.seagate.com"
19437 • Seagate Lyve Cloud US East 1 (Virginia)
19439 • "s3.us-west-1.lyvecloud.seagate.com"
19441 • Seagate Lyve Cloud US West 1 (California)
19443 • "s3.ap-southeast-1.lyvecloud.seagate.com"
19445 • Seagate Lyve Cloud AP Southeast 1 (Singapore)
19447 • "s3.wasabisys.com"
19449 • Wasabi US East 1 (N. Virginia)
19451 • "s3.us-east-2.wasabisys.com"
19453 • Wasabi US East 2 (N. Virginia)
19455 • "s3.us-central-1.wasabisys.com"
19457 • Wasabi US Central 1 (Texas)
19459 • "s3.us-west-1.wasabisys.com"
19461 • Wasabi US West 1 (Oregon)
19463 • "s3.ca-central-1.wasabisys.com"
19465 • Wasabi CA Central 1 (Toronto)
19467 • "s3.eu-central-1.wasabisys.com"
19469 • Wasabi EU Central 1 (Amsterdam)
19471 • "s3.eu-central-2.wasabisys.com"
19473 • Wasabi EU Central 2 (Frankfurt)
19475 • "s3.eu-west-1.wasabisys.com"
19477 • Wasabi EU West 1 (London)
19479 • "s3.eu-west-2.wasabisys.com"
19481 • Wasabi EU West 2 (Paris)
19483 • "s3.ap-northeast-1.wasabisys.com"
19485 • Wasabi AP Northeast 1 (Tokyo) endpoint
19487 • "s3.ap-northeast-2.wasabisys.com"
19489 • Wasabi AP Northeast 2 (Osaka) endpoint
19491 • "s3.ap-southeast-1.wasabisys.com"
19493 • Wasabi AP Southeast 1 (Singapore)
19495 • "s3.ap-southeast-2.wasabisys.com"
19497 • Wasabi AP Southeast 2 (Sydney)
19499 • "s3.ir-thr-at1.arvanstorage.com"
19501 • ArvanCloud Tehran Iran (Asiatech) endpoint
19503 --s3-location-constraint
19505 Location constraint - must be set to match the Region.
19506 Used when creating buckets only.
19508 Properties:
19510 • Config: location_constraint
19512 • Env Var: RCLONE_S3_LOCATION_CONSTRAINT
19514 • Provider: AWS
19516 • Type: string
19518 • Required: false
19520 • Examples:
19522 • ""
19524 • Empty for US Region, Northern Virginia, or Pacific Northwest
19526 • "us-east-2"
19528 • US East (Ohio) Region
19530 • "us-west-1"
19532 • US West (Northern California) Region
19534 • "us-west-2"
19536 • US West (Oregon) Region
19538 • "ca-central-1"
19540 • Canada (Central) Region
19542 • "eu-west-1"
19544 • EU (Ireland) Region
19546 • "eu-west-2"
19548 • EU (London) Region
19550 • "eu-west-3"
19552 • EU (Paris) Region
19554 • "eu-north-1"
19556 • EU (Stockholm) Region
19558 • "eu-south-1"
19560 • EU (Milan) Region
19562 • "EU"
19564 • EU Region
19566 • "ap-southeast-1"
19568 • Asia Pacific (Singapore) Region
19570 • "ap-southeast-2"
19572 • Asia Pacific (Sydney) Region
19574 • "ap-northeast-1"
19576 • Asia Pacific (Tokyo) Region
19578 • "ap-northeast-2"
19580 • Asia Pacific (Seoul) Region
19582 • "ap-northeast-3"
19584 • Asia Pacific (Osaka-Local) Region
19586 • "ap-south-1"
19588 • Asia Pacific (Mumbai) Region
19590 • "ap-east-1"
19592 • Asia Pacific (Hong Kong) Region
19594 • "sa-east-1"
19596 • South America (Sao Paulo) Region
19598 • "me-south-1"
19600 • Middle East (Bahrain) Region
19602 • "af-south-1"
19604 • Africa (Cape Town) Region
19606 • "cn-north-1"
19608 • China (Beijing) Region
19610 • "cn-northwest-1"
19612 • China (Ningxia) Region
19614 • "us-gov-east-1"
19616 • AWS GovCloud (US-East) Region
19618 • "us-gov-west-1"
19620 • AWS GovCloud (US) Region
19622 --s3-location-constraint
19624 Location constraint - must match endpoint.
19625 Used when creating buckets only.
19627 Properties:
19629 • Config: location_constraint
19631 • Env Var: RCLONE_S3_LOCATION_CONSTRAINT
19633 • Provider: ChinaMobile
19635 • Type: string
19637 • Required: false
19639 • Examples:
19641 • "wuxi1"
19643 • East China (Suzhou)
19645 • "jinan1"
19647 • East China (Jinan)
19649 • "ningbo1"
19651 • East China (Hangzhou)
19653 • "shanghai1"
19655 • East China (Shanghai-1)
19657 • "zhengzhou1"
19659 • Central China (Zhengzhou)
19661 • "hunan1"
19663 • Central China (Changsha-1)
19665 • "zhuzhou1"
19667 • Central China (Changsha-2)
19669 • "guangzhou1"
19671 • South China (Guangzhou-2)
19673 • "dongguan1"
19675 • South China (Guangzhou-3)
19677 • "beijing1"
19679 • North China (Beijing-1)
19681 • "beijing2"
19683 • North China (Beijing-2)
19685 • "beijing4"
19687 • North China (Beijing-3)
19689 • "huhehaote1"
19691 • North China (Huhehaote)
19693 • "chengdu1"
19695 • Southwest China (Chengdu)
19697 • "chongqing1"
19699 • Southwest China (Chongqing)
19701 • "guiyang1"
19703 • Southwest China (Guiyang)
19705 • "xian1"
19707 • Nouthwest China (Xian)
19709 • "yunnan"
19711 • Yunnan China (Kunming)
19713 • "yunnan2"
19715 • Yunnan China (Kunming-2)
19717 • "tianjin1"
19719 • Tianjin China (Tianjin)
19721 • "jilin1"
19723 • Jilin China (Changchun)
19725 • "hubei1"
19727 • Hubei China (Xiangyan)
19729 • "jiangxi1"
19731 • Jiangxi China (Nanchang)
19733 • "gansu1"
19735 • Gansu China (Lanzhou)
19737 • "shanxi1"
19739 • Shanxi China (Taiyuan)
19741 • "liaoning1"
19743 • Liaoning China (Shenyang)
19745 • "hebei1"
19747 • Hebei China (Shijiazhuang)
19749 • "fujian1"
19751 • Fujian China (Xiamen)
19753 • "guangxi1"
19755 • Guangxi China (Nanning)
19757 • "anhui1"
19759 • Anhui China (Huainan)
19761 --s3-location-constraint
19763 Location constraint - must match endpoint.
19764 Used when creating buckets only.
19766 Properties:
19768 • Config: location_constraint
19770 • Env Var: RCLONE_S3_LOCATION_CONSTRAINT
19772 • Provider: ArvanCloud
19774 • Type: string
19776 • Required: false
19778 • Examples:
19780 • "ir-thr-at1"
19782 • Tehran Iran (Asiatech)
19784 • "ir-tbz-sh1"
19786 • Tabriz Iran (Shahriar)
19788 --s3-location-constraint
19790 Location constraint - must match endpoint when using IBM Cloud Public.
19791 For on-prem COS, do not make a selection from this list, hit enter.
19793 Properties:
19795 • Config: location_constraint
19797 • Env Var: RCLONE_S3_LOCATION_CONSTRAINT
19799 • Provider: IBMCOS
19801 • Type: string
19803 • Required: false
19805 • Examples:
19807 • "us-standard"
19809 • US Cross Region Standard
19811 • "us-vault"
19813 • US Cross Region Vault
19815 • "us-cold"
19817 • US Cross Region Cold
19819 • "us-flex"
19821 • US Cross Region Flex
19823 • "us-east-standard"
19825 • US East Region Standard
19827 • "us-east-vault"
19829 • US East Region Vault
19831 • "us-east-cold"
19833 • US East Region Cold
19835 • "us-east-flex"
19837 • US East Region Flex
19839 • "us-south-standard"
19841 • US South Region Standard
19843 • "us-south-vault"
19845 • US South Region Vault
19847 • "us-south-cold"
19849 • US South Region Cold
19851 • "us-south-flex"
19853 • US South Region Flex
19855 • "eu-standard"
19857 • EU Cross Region Standard
19859 • "eu-vault"
19861 • EU Cross Region Vault
19863 • "eu-cold"
19865 • EU Cross Region Cold
19867 • "eu-flex"
19869 • EU Cross Region Flex
19871 • "eu-gb-standard"
19873 • Great Britain Standard
19875 • "eu-gb-vault"
19877 • Great Britain Vault
19879 • "eu-gb-cold"
19881 • Great Britain Cold
19883 • "eu-gb-flex"
19885 • Great Britain Flex
19887 • "ap-standard"
19889 • APAC Standard
19891 • "ap-vault"
19893 • APAC Vault
19895 • "ap-cold"
19897 • APAC Cold
19899 • "ap-flex"
19901 • APAC Flex
19903 • "mel01-standard"
19905 • Melbourne Standard
19907 • "mel01-vault"
19909 • Melbourne Vault
19911 • "mel01-cold"
19913 • Melbourne Cold
19915 • "mel01-flex"
19917 • Melbourne Flex
19919 • "tor01-standard"
19921 • Toronto Standard
19923 • "tor01-vault"
19925 • Toronto Vault
19927 • "tor01-cold"
19929 • Toronto Cold
19931 • "tor01-flex"
19933 • Toronto Flex
19935 --s3-location-constraint
19937 Location constraint - the location where your bucket will be located
19938 and your data stored.
19939 Properties:
19941 • Config: location_constraint
19943 • Env Var: RCLONE_S3_LOCATION_CONSTRAINT
19945 • Provider: RackCorp
19947 • Type: string
19949 • Required: false
19951 • Examples:
19953 • "global"
19955 • Global CDN Region
19957 • "au"
19959 • Australia (All locations)
19961 • "au-nsw"
19963 • NSW (Australia) Region
19965 • "au-qld"
19967 • QLD (Australia) Region
19969 • "au-vic"
19971 • VIC (Australia) Region
19973 • "au-wa"
19975 • Perth (Australia) Region
19977 • "ph"
19979 • Manila (Philippines) Region
19981 • "th"
19983 • Bangkok (Thailand) Region
19985 • "hk"
19987 • HK (Hong Kong) Region
19989 • "mn"
19991 • Ulaanbaatar (Mongolia) Region
19993 • "kg"
19995 • Bishkek (Kyrgyzstan) Region
19997 • "id"
19999 • Jakarta (Indonesia) Region
20001 • "jp"
20003 • Tokyo (Japan) Region
20005 • "sg"
20007 • SG (Singapore) Region
20009 • "de"
20011 • Frankfurt (Germany) Region
20013 • "us"
20015 • USA (AnyCast) Region
20017 • "us-east-1"
20019 • New York (USA) Region
20021 • "us-west-1"
20023 • Freemont (USA) Region
20025 • "nz"
20027 • Auckland (New Zealand) Region
20029 --s3-location-constraint
20031 Location constraint - must be set to match the Region.
20032 Used when creating buckets only.
20034 Properties:
20036 • Config: location_constraint
20038 • Env Var: RCLONE_S3_LOCATION_CONSTRAINT
20040 • Provider: Qiniu
20042 • Type: string
20044 • Required: false
20046 • Examples:
20048 • "cn-east-1"
20050 • East China Region 1
20052 • "cn-east-2"
20054 • East China Region 2
20056 • "cn-north-1"
20058 • North China Region 1
20060 • "cn-south-1"
20062 • South China Region 1
20064 • "us-north-1"
20066 • North America Region 1
20068 • "ap-southeast-1"
20070 • Southeast Asia Region 1
20072 • "ap-northeast-1"
20074 • Northeast Asia Region 1
20076 --s3-location-constraint
20078 Location constraint - must be set to match the Region.
20079 Leave blank if not sure. Used when creating buckets only.
20081 Properties:
20083 • Config: location_constraint
20085 • Env Var: RCLONE_S3_LOCATION_CONSTRAINT
20087 • Provider: !AWS,Alibaba,HuaweiOBS,ChinaMobile,Cloudflare,IBM‐
20089 COS,IDrive,IONOS,ArvanCloud,Qiniu,RackCorp,Scaleway,Stack‐
20090 Path,Storj,TencentCOS
20091 • Type: string
20093 • Required: false
20095 --s3-acl
20097 Canned ACL used when creating buckets and storing or copying objects.
20098 This ACL is used for creating objects and if bucket_acl isn't set, for
20100 creating buckets too.
20101 For more info visit https://docs.aws.amazon.com/AmazonS3/lat‐
20103 est/dev/acl-overview.html#canned-acl
20104 Note that this ACL is applied when server-side copying objects as S3
20106 doesn't copy the ACL from the source but rather writes a fresh one.
20107 Properties:
20109 • Config: acl
20111 • Env Var: RCLONE_S3_ACL
20113 • Provider: !Storj,Cloudflare
20115 • Type: string
20117 • Required: false
20119 • Examples:
20121 • "default"
20123 • Owner gets Full_CONTROL.
20125 • No one else has access rights (default).
20127 • "private"
20129 • Owner gets FULL_CONTROL.
20131 • No one else has access rights (default).
20133 • "public-read"
20135 • Owner gets FULL_CONTROL.
20137 • The AllUsers group gets READ access.
20139 • "public-read-write"
20141 • Owner gets FULL_CONTROL.
20143 • The AllUsers group gets READ and WRITE access.
20145 • Granting this on a bucket is generally not recommended.
20147 • "authenticated-read"
20149 • Owner gets FULL_CONTROL.
20151 • The AuthenticatedUsers group gets READ access.
20153 • "bucket-owner-read"
20155 • Object owner gets FULL_CONTROL.
20157 • Bucket owner gets READ access.
20159 • If you specify this canned ACL when creating a bucket, Amazon S3
20161 ignores it.
20162 • "bucket-owner-full-control"
20164 • Both the object owner and the bucket owner get FULL_CONTROL over
20166 the object.
20167 • If you specify this canned ACL when creating a bucket, Amazon S3
20169 ignores it.
20170 • "private"
20172 • Owner gets FULL_CONTROL.
20174 • No one else has access rights (default).
20176 • This acl is available on IBM Cloud (Infra), IBM Cloud (Storage),
20178 On-Premise COS.
20179 • "public-read"
20181 • Owner gets FULL_CONTROL.
20183 • The AllUsers group gets READ access.
20185 • This acl is available on IBM Cloud (Infra), IBM Cloud (Storage),
20187 On-Premise IBM COS.
20188 • "public-read-write"
20190 • Owner gets FULL_CONTROL.
20192 • The AllUsers group gets READ and WRITE access.
20194 • This acl is available on IBM Cloud (Infra), On-Premise IBM COS.
20196 • "authenticated-read"
20198 • Owner gets FULL_CONTROL.
20200 • The AuthenticatedUsers group gets READ access.
20202 • Not supported on Buckets.
20204 • This acl is available on IBM Cloud (Infra) and On-Premise IBM
20206 COS.
20207 --s3-server-side-encryption
20209 The server-side encryption algorithm used when storing this object in
20210 S3.
20211 Properties:
20213 • Config: server_side_encryption
20215 • Env Var: RCLONE_S3_SERVER_SIDE_ENCRYPTION
20217 • Provider: AWS,Ceph,ChinaMobile,Minio
20219 • Type: string
20221 • Required: false
20223 • Examples:
20225 • ""
20227 • None
20229 • "AES256"
20231 • AES256
20233 • "aws:kms"
20235 • aws:kms
20237 --s3-sse-kms-key-id
20239 If using KMS ID you must provide the ARN of Key.
20240 Properties:
20242 • Config: sse_kms_key_id
20244 • Env Var: RCLONE_S3_SSE_KMS_KEY_ID
20246 • Provider: AWS,Ceph,Minio
20248 • Type: string
20250 • Required: false
20252 • Examples:
20254 • ""
20256 • None
20258 • "arn:aws:kms:us-east-1:*"
20260 • arn:aws:kms:*
20262 --s3-storage-class
20264 The storage class to use when storing new objects in S3.
20265 Properties:
20267 • Config: storage_class
20269 • Env Var: RCLONE_S3_STORAGE_CLASS
20271 • Provider: AWS
20273 • Type: string
20275 • Required: false
20277 • Examples:
20279 • ""
20281 • Default
20283 • "STANDARD"
20285 • Standard storage class
20287 • "REDUCED_REDUNDANCY"
20289 • Reduced redundancy storage class
20291 • "STANDARD_IA"
20293 • Standard Infrequent Access storage class
20295 • "ONEZONE_IA"
20297 • One Zone Infrequent Access storage class
20299 • "GLACIER"
20301 • Glacier storage class
20303 • "DEEP_ARCHIVE"
20305 • Glacier Deep Archive storage class
20307 • "INTELLIGENT_TIERING"
20309 • Intelligent-Tiering storage class
20311 • "GLACIER_IR"
20313 • Glacier Instant Retrieval storage class
20315 --s3-storage-class
20317 The storage class to use when storing new objects in OSS.
20318 Properties:
20320 • Config: storage_class
20322 • Env Var: RCLONE_S3_STORAGE_CLASS
20324 • Provider: Alibaba
20326 • Type: string
20328 • Required: false
20330 • Examples:
20332 • ""
20334 • Default
20336 • "STANDARD"
20338 • Standard storage class
20340 • "GLACIER"
20342 • Archive storage mode
20344 • "STANDARD_IA"
20346 • Infrequent access storage mode
20348 --s3-storage-class
20350 The storage class to use when storing new objects in ChinaMobile.
20351 Properties:
20353 • Config: storage_class
20355 • Env Var: RCLONE_S3_STORAGE_CLASS
20357 • Provider: ChinaMobile
20359 • Type: string
20361 • Required: false
20363 • Examples:
20365 • ""
20367 • Default
20369 • "STANDARD"
20371 • Standard storage class
20373 • "GLACIER"
20375 • Archive storage mode
20377 • "STANDARD_IA"
20379 • Infrequent access storage mode
20381 --s3-storage-class
20383 The storage class to use when storing new objects in ArvanCloud.
20384 Properties:
20386 • Config: storage_class
20388 • Env Var: RCLONE_S3_STORAGE_CLASS
20390 • Provider: ArvanCloud
20392 • Type: string
20394 • Required: false
20396 • Examples:
20398 • "STANDARD"
20400 • Standard storage class
20402 --s3-storage-class
20404 The storage class to use when storing new objects in Tencent COS.
20405 Properties:
20407 • Config: storage_class
20409 • Env Var: RCLONE_S3_STORAGE_CLASS
20411 • Provider: TencentCOS
20413 • Type: string
20415 • Required: false
20417 • Examples:
20419 • ""
20421 • Default
20423 • "STANDARD"
20425 • Standard storage class
20427 • "ARCHIVE"
20429 • Archive storage mode
20431 • "STANDARD_IA"
20433 • Infrequent access storage mode
20435 --s3-storage-class
20437 The storage class to use when storing new objects in S3.
20438 Properties:
20440 • Config: storage_class
20442 • Env Var: RCLONE_S3_STORAGE_CLASS
20444 • Provider: Scaleway
20446 • Type: string
20448 • Required: false
20450 • Examples:
20452 • ""
20454 • Default.
20456 • "STANDARD"
20458 • The Standard class for any upload.
20460 • Suitable for on-demand content like streaming or CDN.
20462 • "GLACIER"
20464 • Archived storage.
20466 • Prices are lower, but it needs to be restored first to be ac‐
20468 cessed.
20469 --s3-storage-class
20471 The storage class to use when storing new objects in Qiniu.
20472 Properties:
20474 • Config: storage_class
20476 • Env Var: RCLONE_S3_STORAGE_CLASS
20478 • Provider: Qiniu
20480 • Type: string
20482 • Required: false
20484 • Examples:
20486 • "STANDARD"
20488 • Standard storage class
20490 • "LINE"
20492 • Infrequent access storage mode
20494 • "GLACIER"
20496 • Archive storage mode
20498 • "DEEP_ARCHIVE"
20500 • Deep archive storage mode
20502 Advanced options
20504 Here are the Advanced options specific to s3 (Amazon S3 Compliant Stor‐
20505 age Providers including AWS, Alibaba, Ceph, China Mobile, Cloudflare,
20506 ArvanCloud, Digital Ocean, Dreamhost, Huawei OBS, IBM COS, IDrive e2,
20507 IONOS Cloud, Lyve Cloud, Minio, Netease, RackCorp, Scaleway, SeaweedFS,
20508 StackPath, Storj, Tencent COS, Qiniu and Wasabi).
20509 --s3-bucket-acl
20511 Canned ACL used when creating buckets.
20512 For more info visit https://docs.aws.amazon.com/AmazonS3/lat‐
20514 est/dev/acl-overview.html#canned-acl
20515 Note that this ACL is applied when only when creating buckets. If it
20517 isn't set then "acl" is used instead.
20518 Properties:
20520 • Config: bucket_acl
20522 • Env Var: RCLONE_S3_BUCKET_ACL
20524 • Type: string
20526 • Required: false
20528 • Examples:
20530 • "private"
20532 • Owner gets FULL_CONTROL.
20534 • No one else has access rights (default).
20536 • "public-read"
20538 • Owner gets FULL_CONTROL.
20540 • The AllUsers group gets READ access.
20542 • "public-read-write"
20544 • Owner gets FULL_CONTROL.
20546 • The AllUsers group gets READ and WRITE access.
20548 • Granting this on a bucket is generally not recommended.
20550 • "authenticated-read"
20552 • Owner gets FULL_CONTROL.
20554 • The AuthenticatedUsers group gets READ access.
20556 --s3-requester-pays
20558 Enables requester pays option when interacting with S3 bucket.
20559 Properties:
20561 • Config: requester_pays
20563 • Env Var: RCLONE_S3_REQUESTER_PAYS
20565 • Provider: AWS
20567 • Type: bool
20569 • Default: false
20571 --s3-sse-customer-algorithm
20573 If using SSE-C, the server-side encryption algorithm used when storing
20574 this object in S3.
20575 Properties:
20577 • Config: sse_customer_algorithm
20579 • Env Var: RCLONE_S3_SSE_CUSTOMER_ALGORITHM
20581 • Provider: AWS,Ceph,ChinaMobile,Minio
20583 • Type: string
20585 • Required: false
20587 • Examples:
20589 • ""
20591 • None
20593 • "AES256"
20595 • AES256
20597 --s3-sse-customer-key
20599 To use SSE-C you may provide the secret encryption key used to en‐
20600 crypt/decrypt your data.
20601 Alternatively you can provide --sse-customer-key-base64.
20603 Properties:
20605 • Config: sse_customer_key
20607 • Env Var: RCLONE_S3_SSE_CUSTOMER_KEY
20609 • Provider: AWS,Ceph,ChinaMobile,Minio
20611 • Type: string
20613 • Required: false
20615 • Examples:
20617 • ""
20619 • None
20621 --s3-sse-customer-key-base64
20623 If using SSE-C you must provide the secret encryption key encoded in
20624 base64 format to encrypt/decrypt your data.
20625 Alternatively you can provide --sse-customer-key.
20627 Properties:
20629 • Config: sse_customer_key_base64
20631 • Env Var: RCLONE_S3_SSE_CUSTOMER_KEY_BASE64
20633 • Provider: AWS,Ceph,ChinaMobile,Minio
20635 • Type: string
20637 • Required: false
20639 • Examples:
20641 • ""
20643 • None
20645 --s3-sse-customer-key-md5
20647 If using SSE-C you may provide the secret encryption key MD5 checksum
20648 (optional).
20649 If you leave it blank, this is calculated automatically from the
20651 sse_customer_key provided.
20652 Properties:
20654 • Config: sse_customer_key_md5
20656 • Env Var: RCLONE_S3_SSE_CUSTOMER_KEY_MD5
20658 • Provider: AWS,Ceph,ChinaMobile,Minio
20660 • Type: string
20662 • Required: false
20664 • Examples:
20666 • ""
20668 • None
20670 --s3-upload-cutoff
20672 Cutoff for switching to chunked upload.
20673 Any files larger than this will be uploaded in chunks of chunk_size.
20675 The minimum is 0 and the maximum is 5 GiB.
20676 Properties:
20678 • Config: upload_cutoff
20680 • Env Var: RCLONE_S3_UPLOAD_CUTOFF
20682 • Type: SizeSuffix
20684 • Default: 200Mi
20686 --s3-chunk-size
20688 Chunk size to use for uploading.
20689 When uploading files larger than upload_cutoff or files with unknown
20691 size (e.g. from "rclone rcat" or uploaded with "rclone mount" or
20692 google photos or google docs) they will be uploaded as multipart up‐
20693 loads using this chunk size.
20694 Note that "--s3-upload-concurrency" chunks of this size are buffered in
20696 memory per transfer.
20697 If you are transferring large files over high-speed links and you have
20699 enough memory, then increasing this will speed up the transfers.
20700 Rclone will automatically increase the chunk size when uploading a
20702 large file of known size to stay below the 10,000 chunks limit.
20703 Files of unknown size are uploaded with the configured chunk_size.
20705 Since the default chunk size is 5 MiB and there can be at most 10,000
20706 chunks, this means that by default the maximum size of a file you can
20707 stream upload is 48 GiB. If you wish to stream upload larger files
20708 then you will need to increase chunk_size.
20709 Increasing the chunk size decreases the accuracy of the progress sta‐
20711 tistics displayed with "-P" flag. Rclone treats chunk as sent when
20712 it's buffered by the AWS SDK, when in fact it may still be uploading.
20713 A bigger chunk size means a bigger AWS SDK buffer and progress report‐
20714 ing more deviating from the truth.
20715 Properties:
20717 • Config: chunk_size
20719 • Env Var: RCLONE_S3_CHUNK_SIZE
20721 • Type: SizeSuffix
20723 • Default: 5Mi
20725 --s3-max-upload-parts
20727 Maximum number of parts in a multipart upload.
20728 This option defines the maximum number of multipart chunks to use when
20730 doing a multipart upload.
20731 This can be useful if a service does not support the AWS S3 specifica‐
20733 tion of 10,000 chunks.
20734 Rclone will automatically increase the chunk size when uploading a
20736 large file of a known size to stay below this number of chunks limit.
20737 Properties:
20739 • Config: max_upload_parts
20741 • Env Var: RCLONE_S3_MAX_UPLOAD_PARTS
20743 • Type: int
20745 • Default: 10000
20747 --s3-copy-cutoff
20749 Cutoff for switching to multipart copy.
20750 Any files larger than this that need to be server-side copied will be
20752 copied in chunks of this size.
20753 The minimum is 0 and the maximum is 5 GiB.
20755 Properties:
20757 • Config: copy_cutoff
20759 • Env Var: RCLONE_S3_COPY_CUTOFF
20761 • Type: SizeSuffix
20763 • Default: 4.656Gi
20765 --s3-disable-checksum
20767 Don't store MD5 checksum with object metadata.
20768 Normally rclone will calculate the MD5 checksum of the input before up‐
20770 loading it so it can add it to metadata on the object. This is great
20771 for data integrity checking but can cause long delays for large files
20772 to start uploading.
20773 Properties:
20775 • Config: disable_checksum
20777 • Env Var: RCLONE_S3_DISABLE_CHECKSUM
20779 • Type: bool
20781 • Default: false
20783 --s3-shared-credentials-file
20785 Path to the shared credentials file.
20786 If env_auth = true then rclone can use a shared credentials file.
20788 If this variable is empty rclone will look for the "AWS_SHARED_CREDEN‐
20790 TIALS_FILE" env variable. If the env value is empty it will default to
20791 the current user's home directory.
20792 Linux/OSX: "$HOME/.aws/credentials"
20794 Windows: "%USERPROFILE%\.aws\credentials"
20795 Properties:
20797 • Config: shared_credentials_file
20799 • Env Var: RCLONE_S3_SHARED_CREDENTIALS_FILE
20801 • Type: string
20803 • Required: false
20805 --s3-profile
20807 Profile to use in the shared credentials file.
20808 If env_auth = true then rclone can use a shared credentials file. This
20810 variable controls which profile is used in that file.
20811 If empty it will default to the environment variable "AWS_PROFILE" or
20813 "default" if that environment variable is also not set.
20814 Properties:
20816 • Config: profile
20818 • Env Var: RCLONE_S3_PROFILE
20820 • Type: string
20822 • Required: false
20824 --s3-session-token
20826 An AWS session token.
20827 Properties:
20829 • Config: session_token
20831 • Env Var: RCLONE_S3_SESSION_TOKEN
20833 • Type: string
20835 • Required: false
20837 --s3-upload-concurrency
20839 Concurrency for multipart uploads.
20840 This is the number of chunks of the same file that are uploaded concur‐
20842 rently.
20843 If you are uploading small numbers of large files over high-speed links
20845 and these uploads do not fully utilize your bandwidth, then increasing
20846 this may help to speed up the transfers.
20847 Properties:
20849 • Config: upload_concurrency
20851 • Env Var: RCLONE_S3_UPLOAD_CONCURRENCY
20853 • Type: int
20855 • Default: 4
20857 --s3-force-path-style
20859 If true use path style access if false use virtual hosted style.
20860 If this is true (the default) then rclone will use path style access,
20862 if false then rclone will use virtual path style. See the AWS S3 docs
20863 (https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingBucket.html#ac‐
20864 cess-bucket-intro) for more info.
20865 Some providers (e.g. AWS, Aliyun OSS, Netease COS, or Tencent COS) re‐
20867 quire this set to false - rclone will do this automatically based on
20868 the provider setting.
20869 Properties:
20871 • Config: force_path_style
20873 • Env Var: RCLONE_S3_FORCE_PATH_STYLE
20875 • Type: bool
20877 • Default: true
20879 --s3-v2-auth
20881 If true use v2 authentication.
20882 If this is false (the default) then rclone will use v4 authentication.
20884 If it is set then rclone will use v2 authentication.
20885 Use this only if v4 signatures don't work, e.g. pre Jewel/v10 CEPH.
20887 Properties:
20889 • Config: v2_auth
20891 • Env Var: RCLONE_S3_V2_AUTH
20893 • Type: bool
20895 • Default: false
20897 --s3-use-accelerate-endpoint
20899 If true use the AWS S3 accelerated endpoint.
20900 See: AWS S3 Transfer acceleration (https://docs.aws.amazon.com/Ama‐
20902 zonS3/latest/dev/transfer-acceleration-examples.html)
20903 Properties:
20905 • Config: use_accelerate_endpoint
20907 • Env Var: RCLONE_S3_USE_ACCELERATE_ENDPOINT
20909 • Provider: AWS
20911 • Type: bool
20913 • Default: false
20915 --s3-leave-parts-on-error
20917 If true avoid calling abort upload on a failure, leaving all success‐
20918 fully uploaded parts on S3 for manual recovery.
20919 It should be set to true for resuming uploads across different ses‐
20921 sions.
20922 WARNING: Storing parts of an incomplete multipart upload counts towards
20924 space usage on S3 and will add additional costs if not cleaned up.
20925 Properties:
20927 • Config: leave_parts_on_error
20929 • Env Var: RCLONE_S3_LEAVE_PARTS_ON_ERROR
20931 • Provider: AWS
20933 • Type: bool
20935 • Default: false
20937 --s3-list-chunk
20939 Size of listing chunk (response list for each ListObject S3 request).
20940 This option is also known as "MaxKeys", "max-items", or "page-size"
20942 from the AWS S3 specification. Most services truncate the response
20943 list to 1000 objects even if requested more than that. In AWS S3 this
20944 is a global maximum and cannot be changed, see AWS S3
20945 (https://docs.aws.amazon.com/cli/latest/reference/s3/ls.html). In
20946 Ceph, this can be increased with the "rgw list buckets max chunk" op‐
20947 tion.
20948 Properties:
20950 • Config: list_chunk
20952 • Env Var: RCLONE_S3_LIST_CHUNK
20954 • Type: int
20956 • Default: 1000
20958 --s3-list-version
20960 Version of ListObjects to use: 1,2 or 0 for auto.
20961 When S3 originally launched it only provided the ListObjects call to
20963 enumerate objects in a bucket.
20964 However in May 2016 the ListObjectsV2 call was introduced. This is
20966 much higher performance and should be used if at all possible.
20967 If set to the default, 0, rclone will guess according to the provider
20969 set which list objects method to call. If it guesses wrong, then it
20970 may be set manually here.
20971 Properties:
20973 • Config: list_version
20975 • Env Var: RCLONE_S3_LIST_VERSION
20977 • Type: int
20979 • Default: 0
20981 --s3-list-url-encode
20983 Whether to url encode listings: true/false/unset
20984 Some providers support URL encoding listings and where this is avail‐
20986 able this is more reliable when using control characters in file names.
20987 If this is set to unset (the default) then rclone will choose according
20988 to the provider setting what to apply, but you can override rclone's
20989 choice here.
20990 Properties:
20992 • Config: list_url_encode
20994 • Env Var: RCLONE_S3_LIST_URL_ENCODE
20996 • Type: Tristate
20998 • Default: unset
21000 --s3-no-check-bucket
21002 If set, don't attempt to check the bucket exists or create it.
21003 This can be useful when trying to minimise the number of transactions
21005 rclone does if you know the bucket exists already.
21006 It can also be needed if the user you are using does not have bucket
21008 creation permissions. Before v1.52.0 this would have passed silently
21009 due to a bug.
21010 Properties:
21012 • Config: no_check_bucket
21014 • Env Var: RCLONE_S3_NO_CHECK_BUCKET
21016 • Type: bool
21018 • Default: false
21020 --s3-no-head
21022 If set, don't HEAD uploaded objects to check integrity.
21023 This can be useful when trying to minimise the number of transactions
21025 rclone does.
21026 Setting it means that if rclone receives a 200 OK message after upload‐
21028 ing an object with PUT then it will assume that it got uploaded proper‐
21029 ly.
21030 In particular it will assume:
21032 • the metadata, including modtime, storage class and content type was
21034 as uploaded
21035 • the size was as uploaded
21037 It reads the following items from the response for a single part PUT:
21039 • the MD5SUM
21041 • The uploaded date
21043 For multipart uploads these items aren't read.
21045 If an source object of unknown length is uploaded then rclone will do a
21047 HEAD request.
21048 Setting this flag increases the chance for undetected upload failures,
21050 in particular an incorrect size, so it isn't recommended for normal op‐
21051 eration. In practice the chance of an undetected upload failure is
21052 very small even with this flag.
21053 Properties:
21055 • Config: no_head
21057 • Env Var: RCLONE_S3_NO_HEAD
21059 • Type: bool
21061 • Default: false
21063 --s3-no-head-object
21065 If set, do not do HEAD before GET when getting objects.
21066 Properties:
21068 • Config: no_head_object
21070 • Env Var: RCLONE_S3_NO_HEAD_OBJECT
21072 • Type: bool
21074 • Default: false
21076 --s3-encoding
21078 The encoding for the backend.
21079 See the encoding section in the overview (https://rclone.org/over‐
21081 view/#encoding) for more info.
21082 Properties:
21084 • Config: encoding
21086 • Env Var: RCLONE_S3_ENCODING
21088 • Type: MultiEncoder
21090 • Default: Slash,InvalidUtf8,Dot
21092 --s3-memory-pool-flush-time
21094 How often internal memory buffer pools will be flushed.
21095 Uploads which requires additional buffers (f.e multipart) will use mem‐
21097 ory pool for allocations. This option controls how often unused buf‐
21098 fers will be removed from the pool.
21099 Properties:
21101 • Config: memory_pool_flush_time
21103 • Env Var: RCLONE_S3_MEMORY_POOL_FLUSH_TIME
21105 • Type: Duration
21107 • Default: 1m0s
21109 --s3-memory-pool-use-mmap
21111 Whether to use mmap buffers in internal memory pool.
21112 Properties:
21114 • Config: memory_pool_use_mmap
21116 • Env Var: RCLONE_S3_MEMORY_POOL_USE_MMAP
21118 • Type: bool
21120 • Default: false
21122 --s3-disable-http2
21124 Disable usage of http2 for S3 backends.
21125 There is currently an unsolved issue with the s3 (specifically minio)
21127 backend and HTTP/2. HTTP/2 is enabled by default for the s3 backend
21128 but can be disabled here. When the issue is solved this flag will be
21129 removed.
21130 See: https://github.com/rclone/rclone/issues/4673,
21132 https://github.com/rclone/rclone/issues/3631
21133 Properties:
21135 • Config: disable_http2
21137 • Env Var: RCLONE_S3_DISABLE_HTTP2
21139 • Type: bool
21141 • Default: false
21143 --s3-download-url
21145 Custom endpoint for downloads. This is usually set to a CloudFront CDN
21146 URL as AWS S3 offers cheaper egress for data downloaded through the
21147 CloudFront network.
21148 Properties:
21150 • Config: download_url
21152 • Env Var: RCLONE_S3_DOWNLOAD_URL
21154 • Type: string
21156 • Required: false
21158 --s3-use-multipart-etag
21160 Whether to use ETag in multipart uploads for verification
21161 This should be true, false or left unset to use the default for the
21163 provider.
21164 Properties:
21166 • Config: use_multipart_etag
21168 • Env Var: RCLONE_S3_USE_MULTIPART_ETAG
21170 • Type: Tristate
21172 • Default: unset
21174 --s3-use-presigned-request
21176 Whether to use a presigned request or PutObject for single part uploads
21177 If this is false rclone will use PutObject from the AWS SDK to upload
21179 an object.
21180 Versions of rclone < 1.59 use presigned requests to upload a single
21182 part object and setting this flag to true will re-enable that function‐
21183 ality. This shouldn't be necessary except in exceptional circumstances
21184 or for testing.
21185 Properties:
21187 • Config: use_presigned_request
21189 • Env Var: RCLONE_S3_USE_PRESIGNED_REQUEST
21191 • Type: bool
21193 • Default: false
21195 --s3-versions
21197 Include old versions in directory listings.
21198 Properties:
21200 • Config: versions
21202 • Env Var: RCLONE_S3_VERSIONS
21204 • Type: bool
21206 • Default: false
21208 --s3-version-at
21210 Show file versions as they were at the specified time.
21211 The parameter should be a date, "2006-01-02", datetime "2006-01-02
21213 15:04:05" or a duration for that long ago, eg "100d" or "1h".
21214 Note that when using this no file write operations are permitted, so
21216 you can't upload files or delete them.
21217 See the time option docs (https://rclone.org/docs/#time-option) for
21219 valid formats.
21220 Properties:
21222 • Config: version_at
21224 • Env Var: RCLONE_S3_VERSION_AT
21226 • Type: Time
21228 • Default: off
21230 --s3-decompress
21232 If set this will decompress gzip encoded objects.
21233 It is possible to upload objects to S3 with "Content-Encoding: gzip"
21235 set. Normally rclone will download these files as compressed objects.
21236 If this flag is set then rclone will decompress these files with "Con‐
21238 tent-Encoding: gzip" as they are received. This means that rclone
21239 can't check the size and hash but the file contents will be decom‐
21240 pressed.
21241 Properties:
21243 • Config: decompress
21245 • Env Var: RCLONE_S3_DECOMPRESS
21247 • Type: bool
21249 • Default: false
21251 --s3-might-gzip
21253 Set this if the backend might gzip objects.
21254 Normally providers will not alter objects when they are downloaded. If
21256 an object was not uploaded with Content-Encoding: gzip then it won't be
21257 set on download.
21258 However some providers may gzip objects even if they weren't uploaded
21260 with Content-Encoding: gzip (eg Cloudflare).
21261 A symptom of this would be receiving errors like
21263 ERROR corrupted on transfer: sizes differ NNN vs MMM
21265 If you set this flag and rclone downloads an object with Content-Encod‐
21267 ing: gzip set and chunked transfer encoding, then rclone will decom‐
21268 press the object on the fly.
21269 If this is set to unset (the default) then rclone will choose according
21271 to the provider setting what to apply, but you can override rclone's
21272 choice here.
21273 Properties:
21275 • Config: might_gzip
21277 • Env Var: RCLONE_S3_MIGHT_GZIP
21279 • Type: Tristate
21281 • Default: unset
21283 --s3-no-system-metadata
21285 Suppress setting and reading of system metadata
21286 Properties:
21288 • Config: no_system_metadata
21290 • Env Var: RCLONE_S3_NO_SYSTEM_METADATA
21292 • Type: bool
21294 • Default: false
21296 Metadata
21298 User metadata is stored as x-amz-meta- keys. S3 metadata keys are case
21299 insensitive and are always returned in lower case.
21300 Here are the possible system metadata items for the s3 backend.
21302 Name Help Type Example Read Only
21304 ──────────────────────────────────────────────────────────────────────────────────────────────────────
21305 btime Time of RFC 3339 2006-01-02T15:04:05.999999999Z07:00 Y
21306 file birth
21307 (creation)
21308 read from
21309 Last-Modi‐
21310 fied header
21311 cache-con‐ Cache-Con‐ string no-cache N
21312 trol trol header
21313 content- Content- string inline N
21314 disposition Disposition
21315 header
21316 content-en‐ Content-En‐ string gzip N
21317 coding coding
21318 header
21319 content- Content- string en-US N
21320 language Language
21321 header
21322 content- Content- string text/plain N
21323 type Type header
21324 mtime Time of RFC 3339 2006-01-02T15:04:05.999999999Z07:00 N
21325 last modi‐
21326 fication,
21327 read from
21328 rclone
21329 metadata
21330 tier Tier of the string GLACIER Y
21331 object
21332 See the metadata (https://rclone.org/docs/#metadata) docs for more in‐
21334 fo.
21335 Backend commands
21337 Here are the commands specific to the s3 backend.
21338 Run them with
21340 rclone backend COMMAND remote:
21342 The help below will explain what arguments each command takes.
21344 See the backend (https://rclone.org/commands/rclone_backend/) command
21346 for more info on how to pass options and arguments.
21347 These can be run on a running backend using the rc command backend/com‐
21349 mand (https://rclone.org/rc/#backend-command).
21350 restore
21352 Restore objects from GLACIER to normal storage
21353 rclone backend restore remote: [options] [<arguments>+]
21355 This command can be used to restore one or more objects from GLACIER to
21357 normal storage.
21358 Usage Examples:
21360 rclone backend restore s3:bucket/path/to/object [-o priority=PRIORITY] [-o lifetime=DAYS]
21362 rclone backend restore s3:bucket/path/to/directory [-o priority=PRIORITY] [-o lifetime=DAYS]
21363 rclone backend restore s3:bucket [-o priority=PRIORITY] [-o lifetime=DAYS]
21364 This flag also obeys the filters. Test first with -i/--interactive or
21366 --dry-run flags
21367 rclone -i backend restore --include "*.txt" s3:bucket/path -o priority=Standard
21369 All the objects shown will be marked for restore, then
21371 rclone backend restore --include "*.txt" s3:bucket/path -o priority=Standard
21373 It returns a list of status dictionaries with Remote and Status keys.
21375 The Status will be OK if it was successful or an error message if not.
21376 [
21378 {
21379 "Status": "OK",
21380 "Path": "test.txt"
21381 },
21382 {
21383 "Status": "OK",
21384 "Path": "test/file4.txt"
21385 }
21386 ]
21387 Options:
21389 • "description": The optional description for the job.
21391 • "lifetime": Lifetime of the active copy in days
21393 • "priority": Priority of restore: Standard|Expedited|Bulk
21395 list-multipart-uploads
21397 List the unfinished multipart uploads
21398 rclone backend list-multipart-uploads remote: [options] [<arguments>+]
21400 This command lists the unfinished multipart uploads in JSON format.
21402 rclone backend list-multipart s3:bucket/path/to/object
21404 It returns a dictionary of buckets with values as lists of unfinished
21406 multipart uploads.
21407 You can call it with no bucket in which case it lists all bucket, with
21409 a bucket or with a bucket and path.
21410 {
21412 "rclone": [
21413 {
21414 "Initiated": "2020-06-26T14:20:36Z",
21415 "Initiator": {
21416 "DisplayName": "XXX",
21417 "ID": "arn:aws:iam::XXX:user/XXX"
21418 },
21419 "Key": "KEY",
21420 "Owner": {
21421 "DisplayName": null,
21422 "ID": "XXX"
21423 },
21424 "StorageClass": "STANDARD",
21425 "UploadId": "XXX"
21426 }
21427 ],
21428 "rclone-1000files": [],
21429 "rclone-dst": []
21430 }
21431 cleanup
21433 Remove unfinished multipart uploads.
21434 rclone backend cleanup remote: [options] [<arguments>+]
21436 This command removes unfinished multipart uploads of age greater than
21438 max-age which defaults to 24 hours.
21439 Note that you can use -i/--dry-run with this command to see what it
21441 would do.
21442 rclone backend cleanup s3:bucket/path/to/object
21444 rclone backend cleanup -o max-age=7w s3:bucket/path/to/object
21445 Durations are parsed as per the rest of rclone, 2h, 7d, 7w etc.
21447 Options:
21449 • "max-age": Max age of upload to delete
21451 cleanup-hidden
21453 Remove old versions of files.
21454 rclone backend cleanup-hidden remote: [options] [<arguments>+]
21456 This command removes any old hidden versions of files on a versions en‐
21458 abled bucket.
21459 Note that you can use -i/--dry-run with this command to see what it
21461 would do.
21462 rclone backend cleanup-hidden s3:bucket/path/to/dir
21464 versioning
21466 Set/get versioning support for a bucket.
21467 rclone backend versioning remote: [options] [<arguments>+]
21469 This command sets versioning support if a parameter is passed and then
21471 returns the current versioning status for the bucket supplied.
21472 rclone backend versioning s3:bucket # read status only
21474 rclone backend versioning s3:bucket Enabled
21475 rclone backend versioning s3:bucket Suspended
21476 It may return "Enabled", "Suspended" or "Unversioned". Note that once
21478 versioning has been enabled the status can't be set back to "Unver‐
21479 sioned".
21480 Anonymous access to public buckets
21482 If you want to use rclone to access a public bucket, configure with a
21483 blank access_key_id and secret_access_key. Your config should end up
21484 looking like this:
21485 [anons3]
21487 type = s3
21488 provider = AWS
21489 env_auth = false
21490 access_key_id =
21491 secret_access_key =
21492 region = us-east-1
21493 endpoint =
21494 location_constraint =
21495 acl = private
21496 server_side_encryption =
21497 storage_class =
21498 Then use it as normal with the name of the public bucket, e.g.
21500 rclone lsd anons3:1000genomes
21502 You will be able to list and copy data but not upload it.
21504 Providers
21506 AWS S3
21507 This is the provider used as main example and described in the configu‐
21508 ration section above.
21509 AWS Snowball Edge
21511 AWS Snowball (https://aws.amazon.com/snowball/) is a hardware appliance
21512 used for transferring bulk data back to AWS. Its main software inter‐
21513 face is S3 object storage.
21514 To use rclone with AWS Snowball Edge devices, configure as standard for
21516 an 'S3 Compatible Service'.
21517 If using rclone pre v1.59 be sure to set upload_cutoff = 0 otherwise
21519 you will run into authentication header issues as the snowball device
21520 does not support query parameter based authentication.
21521 With rclone v1.59 or later setting upload_cutoff should not be neces‐
21523 sary.
21524 eg.
21526 [snowball]
21528 type = s3
21529 provider = Other
21530 access_key_id = YOUR_ACCESS_KEY
21531 secret_access_key = YOUR_SECRET_KEY
21532 endpoint = http://[IP of Snowball]:8080
21533 upload_cutoff = 0
21534 Ceph
21536 Ceph (https://ceph.com/) is an open-source, unified, distributed stor‐
21537 age system designed for excellent performance, reliability and scala‐
21538 bility. It has an S3 compatible object storage interface.
21539 To use rclone with Ceph, configure as above but leave the region blank
21541 and set the endpoint. You should end up with something like this in
21542 your config:
21543 [ceph]
21545 type = s3
21546 provider = Ceph
21547 env_auth = false
21548 access_key_id = XXX
21549 secret_access_key = YYY
21550 region =
21551 endpoint = https://ceph.endpoint.example.com
21552 location_constraint =
21553 acl =
21554 server_side_encryption =
21555 storage_class =
21556 If you are using an older version of CEPH (e.g. 10.2.x Jewel) and a
21558 version of rclone before v1.59 then you may need to supply the parame‐
21559 ter --s3-upload-cutoff 0 or put this in the config file as upload_cut‐
21560 off 0 to work around a bug which causes uploading of small files to
21561 fail.
21562 Note also that Ceph sometimes puts / in the passwords it gives users.
21564 If you read the secret access key using the command line tools you will
21565 get a JSON blob with the / escaped as \/. Make sure you only write /
21566 in the secret access key.
21567 Eg the dump from Ceph looks something like this (irrelevant keys re‐
21569 moved).
21570 {
21572 "user_id": "xxx",
21573 "display_name": "xxxx",
21574 "keys": [
21575 {
21576 "user": "xxx",
21577 "access_key": "xxxxxx",
21578 "secret_key": "xxxxxx\/xxxx"
21579 }
21580 ],
21581 }
21582 Because this is a json dump, it is encoding the / as \/, so if you use
21584 the secret key as xxxxxx/xxxx it will work fine.
21585 Cloudflare R2
21587 Cloudflare R2 (https://blog.cloudflare.com/r2-open-beta/) Storage al‐
21588 lows developers to store large amounts of unstructured data without the
21589 costly egress bandwidth fees associated with typical cloud storage ser‐
21590 vices.
21591 Here is an example of making a Cloudflare R2 configuration. First run:
21593 rclone config
21595 This will guide you through an interactive setup process.
21597 Note that all buckets are private, and all are stored in the same "au‐
21599 to" region. It is necessary to use Cloudflare workers to share the
21600 content of a bucket publicly.
21601 No remotes found, make a new one?
21603 n) New remote
21604 s) Set configuration password
21605 q) Quit config
21606 n/s/q> n
21607 name> r2
21608 Option Storage.
21609 Type of storage to configure.
21610 Choose a number from below, or type in your own value.
21611 ...
21612 XX / Amazon S3 Compliant Storage Providers including AWS, Alibaba, Ceph, China Mobile, Cloudflare, ArvanCloud, Digital Ocean, Dreamhost, Huawei OBS, IBM COS, Lyve Cloud, Minio, Netease, RackCorp, Scaleway, SeaweedFS, StackPath, Storj, Tencent COS and Wasabi
21613 \ (s3)
21614 ...
21615 Storage> s3
21616 Option provider.
21617 Choose your S3 provider.
21618 Choose a number from below, or type in your own value.
21619 Press Enter to leave empty.
21620 ...
21621 XX / Cloudflare R2 Storage
21622 \ (Cloudflare)
21623 ...
21624 provider> Cloudflare
21625 Option env_auth.
21626 Get AWS credentials from runtime (environment variables or EC2/ECS meta data if no env vars).
21627 Only applies if access_key_id and secret_access_key is blank.
21628 Choose a number from below, or type in your own boolean value (true or false).
21629 Press Enter for the default (false).
21630 1 / Enter AWS credentials in the next step.
21631 \ (false)
21632 2 / Get AWS credentials from the environment (env vars or IAM).
21633 \ (true)
21634 env_auth> 1
21635 Option access_key_id.
21636 AWS Access Key ID.
21637 Leave blank for anonymous access or runtime credentials.
21638 Enter a value. Press Enter to leave empty.
21639 access_key_id> ACCESS_KEY
21640 Option secret_access_key.
21641 AWS Secret Access Key (password).
21642 Leave blank for anonymous access or runtime credentials.
21643 Enter a value. Press Enter to leave empty.
21644 secret_access_key> SECRET_ACCESS_KEY
21645 Option region.
21646 Region to connect to.
21647 Choose a number from below, or type in your own value.
21648 Press Enter to leave empty.
21649 1 / R2 buckets are automatically distributed across Cloudflare's data centers for low latency.
21650 \ (auto)
21651 region> 1
21652 Option endpoint.
21653 Endpoint for S3 API.
21654 Required when using an S3 clone.
21655 Enter a value. Press Enter to leave empty.
21656 endpoint> https://ACCOUNT_ID.r2.cloudflarestorage.com
21657 Edit advanced config?
21658 y) Yes
21659 n) No (default)
21660 y/n> n
21661 --------------------
21662 y) Yes this is OK (default)
21663 e) Edit this remote
21664 d) Delete this remote
21665 y/e/d> y
21666 This will leave your config looking something like:
21668 [r2]
21670 type = s3
21671 provider = Cloudflare
21672 access_key_id = ACCESS_KEY
21673 secret_access_key = SECRET_ACCESS_KEY
21674 region = auto
21675 endpoint = https://ACCOUNT_ID.r2.cloudflarestorage.com
21676 acl = private
21677 Now run rclone lsf r2: to see your buckets and rclone lsf r2:bucket to
21679 look within a bucket.
21680 Dreamhost
21682 Dreamhost DreamObjects (https://www.dreamhost.com/cloud/storage/) is an
21683 object storage system based on CEPH.
21684 To use rclone with Dreamhost, configure as above but leave the region
21686 blank and set the endpoint. You should end up with something like this
21687 in your config:
21688 [dreamobjects]
21690 type = s3
21691 provider = DreamHost
21692 env_auth = false
21693 access_key_id = your_access_key
21694 secret_access_key = your_secret_key
21695 region =
21696 endpoint = objects-us-west-1.dream.io
21697 location_constraint =
21698 acl = private
21699 server_side_encryption =
21700 storage_class =
21701 DigitalOcean Spaces
21703 Spaces (https://www.digitalocean.com/products/object-storage/) is an
21704 S3-interoperable (https://developers.digitalocean.com/documenta‐
21705 tion/spaces/) object storage service from cloud provider DigitalOcean.
21706 To connect to DigitalOcean Spaces you will need an access key and se‐
21708 cret key. These can be retrieved on the "Applications & API
21709 (https://cloud.digitalocean.com/settings/api/tokens)" page of the Digi‐
21710 talOcean control panel. They will be needed when prompted by rclone
21711 config for your access_key_id and secret_access_key.
21712 When prompted for a region or location_constraint, press enter to use
21714 the default value. The region must be included in the endpoint setting
21715 (e.g. nyc3.digitaloceanspaces.com). The default values can be used
21716 for other settings.
21717 Going through the whole process of creating a new remote by running
21719 rclone config, each prompt should be answered as shown below:
21720 Storage> s3
21722 env_auth> 1
21723 access_key_id> YOUR_ACCESS_KEY
21724 secret_access_key> YOUR_SECRET_KEY
21725 region>
21726 endpoint> nyc3.digitaloceanspaces.com
21727 location_constraint>
21728 acl>
21729 storage_class>
21730 The resulting configuration file should look like:
21732 [spaces]
21734 type = s3
21735 provider = DigitalOcean
21736 env_auth = false
21737 access_key_id = YOUR_ACCESS_KEY
21738 secret_access_key = YOUR_SECRET_KEY
21739 region =
21740 endpoint = nyc3.digitaloceanspaces.com
21741 location_constraint =
21742 acl =
21743 server_side_encryption =
21744 storage_class =
21745 Once configured, you can create a new Space and begin copying files.
21747 For example:
21748 rclone mkdir spaces:my-new-space
21750 rclone copy /path/to/files spaces:my-new-space
21751 Huawei OBS
21753 Object Storage Service (OBS) provides stable, secure, efficient, and
21754 easy-to-use cloud storage that lets you store virtually any volume of
21755 unstructured data in any format and access it from anywhere.
21756 OBS provides an S3 interface, you can copy and modify the following
21758 configuration and add it to your rclone configuration file.
21759 [obs]
21761 type = s3
21762 provider = HuaweiOBS
21763 access_key_id = your-access-key-id
21764 secret_access_key = your-secret-access-key
21765 region = af-south-1
21766 endpoint = obs.af-south-1.myhuaweicloud.com
21767 acl = private
21768 Or you can also configure via the interactive command line:
21770 No remotes found, make a new one?
21772 n) New remote
21773 s) Set configuration password
21774 q) Quit config
21775 n/s/q> n
21776 name> obs
21777 Option Storage.
21778 Type of storage to configure.
21779 Choose a number from below, or type in your own value.
21780 [snip]
21781 5 / Amazon S3 Compliant Storage Providers including AWS, Alibaba, Ceph, China Mobile, Cloudflare, ArvanCloud, Digital Ocean, Dreamhost, Huawei OBS, IBM COS, Lyve Cloud, Minio, Netease, RackCorp, Scaleway, SeaweedFS, StackPath, Storj, Tencent COS and Wasabi
21782 \ (s3)
21783 [snip]
21784 Storage> 5
21785 Option provider.
21786 Choose your S3 provider.
21787 Choose a number from below, or type in your own value.
21788 Press Enter to leave empty.
21789 [snip]
21790 9 / Huawei Object Storage Service
21791 \ (HuaweiOBS)
21792 [snip]
21793 provider> 9
21794 Option env_auth.
21795 Get AWS credentials from runtime (environment variables or EC2/ECS meta data if no env vars).
21796 Only applies if access_key_id and secret_access_key is blank.
21797 Choose a number from below, or type in your own boolean value (true or false).
21798 Press Enter for the default (false).
21799 1 / Enter AWS credentials in the next step.
21800 \ (false)
21801 2 / Get AWS credentials from the environment (env vars or IAM).
21802 \ (true)
21803 env_auth> 1
21804 Option access_key_id.
21805 AWS Access Key ID.
21806 Leave blank for anonymous access or runtime credentials.
21807 Enter a value. Press Enter to leave empty.
21808 access_key_id> your-access-key-id
21809 Option secret_access_key.
21810 AWS Secret Access Key (password).
21811 Leave blank for anonymous access or runtime credentials.
21812 Enter a value. Press Enter to leave empty.
21813 secret_access_key> your-secret-access-key
21814 Option region.
21815 Region to connect to.
21816 Choose a number from below, or type in your own value.
21817 Press Enter to leave empty.
21818 1 / AF-Johannesburg
21819 \ (af-south-1)
21820 2 / AP-Bangkok
21821 \ (ap-southeast-2)
21822 [snip]
21823 region> 1
21824 Option endpoint.
21825 Endpoint for OBS API.
21826 Choose a number from below, or type in your own value.
21827 Press Enter to leave empty.
21828 1 / AF-Johannesburg
21829 \ (obs.af-south-1.myhuaweicloud.com)
21830 2 / AP-Bangkok
21831 \ (obs.ap-southeast-2.myhuaweicloud.com)
21832 [snip]
21833 endpoint> 1
21834 Option acl.
21835 Canned ACL used when creating buckets and storing or copying objects.
21836 This ACL is used for creating objects and if bucket_acl isn't set, for creating buckets too.
21837 For more info visit https://docs.aws.amazon.com/AmazonS3/latest/dev/acl-overview.html#canned-acl
21838 Note that this ACL is applied when server-side copying objects as S3
21839 doesn't copy the ACL from the source but rather writes a fresh one.
21840 Choose a number from below, or type in your own value.
21841 Press Enter to leave empty.
21842 / Owner gets FULL_CONTROL.
21843 1 | No one else has access rights (default).
21844 \ (private)
21845 [snip]
21846 acl> 1
21847 Edit advanced config?
21848 y) Yes
21849 n) No (default)
21850 y/n>
21851 --------------------
21852 [obs]
21853 type = s3
21854 provider = HuaweiOBS
21855 access_key_id = your-access-key-id
21856 secret_access_key = your-secret-access-key
21857 region = af-south-1
21858 endpoint = obs.af-south-1.myhuaweicloud.com
21859 acl = private
21860 --------------------
21861 y) Yes this is OK (default)
21862 e) Edit this remote
21863 d) Delete this remote
21864 y/e/d> y
21865 Current remotes:
21866 Name Type
21868 ==== ====
21869 obs s3
21870 e) Edit existing remote
21872 n) New remote
21873 d) Delete remote
21874 r) Rename remote
21875 c) Copy remote
21876 s) Set configuration password
21877 q) Quit config
21878 e/n/d/r/c/s/q> q
21879 IBM COS (S3)
21881 Information stored with IBM Cloud Object Storage is encrypted and dis‐
21882 persed across multiple geographic locations, and accessed through an
21883 implementation of the S3 API. This service makes use of the distrib‐
21884 uted storage technologies provided by IBM’s Cloud Object Storage System
21885 (formerly Cleversafe). For more information visit:
21886 (http://www.ibm.com/cloud/object-storage)
21887 To configure access to IBM COS S3, follow the steps below:
21889 1. Run rclone config and select n for a new remote.
21891 2018/02/14 14:13:11 NOTICE: Config file "C:\\Users\\a\\.config\\rclone\\rclone.conf" not found - using defaults
21893 No remotes found, make a new one?
21894 n) New remote
21895 s) Set configuration password
21896 q) Quit config
21897 n/s/q> n
21898 2. Enter the name for the configuration
21900 name> <YOUR NAME>
21902 3. Select "s3" storage.
21904 Choose a number from below, or type in your own value
21906 1 / Alias for an existing remote
21907 \ "alias"
21908 2 / Amazon Drive
21909 \ "amazon cloud drive"
21910 3 / Amazon S3 Complaint Storage Providers (Dreamhost, Ceph, ChinaMobile, ArvanCloud, Minio, IBM COS)
21911 \ "s3"
21912 4 / Backblaze B2
21913 \ "b2"
21914 [snip]
21915 23 / HTTP
21916 \ "http"
21917 Storage> 3
21918 4. Select IBM COS as the S3 Storage Provider.
21920 Choose the S3 provider.
21922 Choose a number from below, or type in your own value
21923 1 / Choose this option to configure Storage to AWS S3
21924 \ "AWS"
21925 2 / Choose this option to configure Storage to Ceph Systems
21926 \ "Ceph"
21927 3 / Choose this option to configure Storage to Dreamhost
21928 \ "Dreamhost"
21929 4 / Choose this option to the configure Storage to IBM COS S3
21930 \ "IBMCOS"
21931 5 / Choose this option to the configure Storage to Minio
21932 \ "Minio"
21933 Provider>4
21934 5. Enter the Access Key and Secret.
21936 AWS Access Key ID - leave blank for anonymous access or runtime credentials.
21938 access_key_id> <>
21939 AWS Secret Access Key (password) - leave blank for anonymous access or runtime credentials.
21940 secret_access_key> <>
21941 6. Specify the endpoint for IBM COS. For Public IBM COS, choose from
21943 the option below. For On Premise IBM COS, enter an endpoint ad‐
21944 dress.
21945 Endpoint for IBM COS S3 API.
21947 Specify if using an IBM COS On Premise.
21948 Choose a number from below, or type in your own value
21949 1 / US Cross Region Endpoint
21950 \ "s3-api.us-geo.objectstorage.softlayer.net"
21951 2 / US Cross Region Dallas Endpoint
21952 \ "s3-api.dal.us-geo.objectstorage.softlayer.net"
21953 3 / US Cross Region Washington DC Endpoint
21954 \ "s3-api.wdc-us-geo.objectstorage.softlayer.net"
21955 4 / US Cross Region San Jose Endpoint
21956 \ "s3-api.sjc-us-geo.objectstorage.softlayer.net"
21957 5 / US Cross Region Private Endpoint
21958 \ "s3-api.us-geo.objectstorage.service.networklayer.com"
21959 6 / US Cross Region Dallas Private Endpoint
21960 \ "s3-api.dal-us-geo.objectstorage.service.networklayer.com"
21961 7 / US Cross Region Washington DC Private Endpoint
21962 \ "s3-api.wdc-us-geo.objectstorage.service.networklayer.com"
21963 8 / US Cross Region San Jose Private Endpoint
21964 \ "s3-api.sjc-us-geo.objectstorage.service.networklayer.com"
21965 9 / US Region East Endpoint
21966 \ "s3.us-east.objectstorage.softlayer.net"
21967 10 / US Region East Private Endpoint
21968 \ "s3.us-east.objectstorage.service.networklayer.com"
21969 11 / US Region South Endpoint
21970 [snip]
21971 34 / Toronto Single Site Private Endpoint
21972 \ "s3.tor01.objectstorage.service.networklayer.com"
21973 endpoint>1
21974 7. Specify a IBM COS Location Constraint. The location constraint must
21976 match endpoint when using IBM Cloud Public. For on-prem COS, do not
21977 make a selection from this list, hit enter
21978 1 / US Cross Region Standard
21980 \ "us-standard"
21981 2 / US Cross Region Vault
21982 \ "us-vault"
21983 3 / US Cross Region Cold
21984 \ "us-cold"
21985 4 / US Cross Region Flex
21986 \ "us-flex"
21987 5 / US East Region Standard
21988 \ "us-east-standard"
21989 6 / US East Region Vault
21990 \ "us-east-vault"
21991 7 / US East Region Cold
21992 \ "us-east-cold"
21993 8 / US East Region Flex
21994 \ "us-east-flex"
21995 9 / US South Region Standard
21996 \ "us-south-standard"
21997 10 / US South Region Vault
21998 \ "us-south-vault"
21999 [snip]
22000 32 / Toronto Flex
22001 \ "tor01-flex"
22002 location_constraint>1
22003 9. Specify a canned ACL. IBM Cloud (Storage) supports "public-read"
22005 and "private". IBM Cloud(Infra) supports all the canned ACLs. On-
22006 Premise COS supports all the canned ACLs.
22007 Canned ACL used when creating buckets and/or storing objects in S3.
22009 For more info visit https://docs.aws.amazon.com/AmazonS3/latest/dev/acl-overview.html#canned-acl
22010 Choose a number from below, or type in your own value
22011 1 / Owner gets FULL_CONTROL. No one else has access rights (default). This acl is available on IBM Cloud (Infra), IBM Cloud (Storage), On-Premise COS
22012 \ "private"
22013 2 / Owner gets FULL_CONTROL. The AllUsers group gets READ access. This acl is available on IBM Cloud (Infra), IBM Cloud (Storage), On-Premise IBM COS
22014 \ "public-read"
22015 3 / Owner gets FULL_CONTROL. The AllUsers group gets READ and WRITE access. This acl is available on IBM Cloud (Infra), On-Premise IBM COS
22016 \ "public-read-write"
22017 4 / Owner gets FULL_CONTROL. The AuthenticatedUsers group gets READ access. Not supported on Buckets. This acl is available on IBM Cloud (Infra) and On-Premise IBM COS
22018 \ "authenticated-read"
22019 acl> 1
22020 12. Review the displayed configuration and accept to save the "remote"
22022 then quit. The config file should look like this
22023 [xxx]
22025 type = s3
22026 Provider = IBMCOS
22027 access_key_id = xxx
22028 secret_access_key = yyy
22029 endpoint = s3-api.us-geo.objectstorage.softlayer.net
22030 location_constraint = us-standard
22031 acl = private
22032 13. Execute rclone commands
22034 1) Create a bucket.
22036 rclone mkdir IBM-COS-XREGION:newbucket
22037 2) List available buckets.
22038 rclone lsd IBM-COS-XREGION:
22039 -1 2017-11-08 21:16:22 -1 test
22040 -1 2018-02-14 20:16:39 -1 newbucket
22041 3) List contents of a bucket.
22042 rclone ls IBM-COS-XREGION:newbucket
22043 18685952 test.exe
22044 4) Copy a file from local to remote.
22045 rclone copy /Users/file.txt IBM-COS-XREGION:newbucket
22046 5) Copy a file from remote to local.
22047 rclone copy IBM-COS-XREGION:newbucket/file.txt .
22048 6) Delete a file on remote.
22049 rclone delete IBM-COS-XREGION:newbucket/file.txt
22050 IDrive e2
22052 Here is an example of making an IDrive e2 (https://www.idrive.com/e2/)
22053 configuration. First run:
22054 rclone config
22056 This will guide you through an interactive setup process.
22058 No remotes found, make a new one?
22060 n) New remote
22061 s) Set configuration password
22062 q) Quit config
22063 n/s/q> n
22064 Enter name for new remote.
22066 name> e2
22067 Option Storage.
22069 Type of storage to configure.
22070 Choose a number from below, or type in your own value.
22071 [snip]
22072 XX / Amazon S3 Compliant Storage Providers including AWS, Alibaba, Ceph, China Mobile, Cloudflare, ArvanCloud, Digital Ocean, Dreamhost, Huawei OBS, IBM COS, IDrive e2, Lyve Cloud, Minio, Netease, RackCorp, Scaleway, SeaweedFS, StackPath, Storj, Tencent COS and Wasabi
22073 \ (s3)
22074 [snip]
22075 Storage> s3
22076 Option provider.
22078 Choose your S3 provider.
22079 Choose a number from below, or type in your own value.
22080 Press Enter to leave empty.
22081 [snip]
22082 XX / IDrive e2
22083 \ (IDrive)
22084 [snip]
22085 provider> IDrive
22086 Option env_auth.
22088 Get AWS credentials from runtime (environment variables or EC2/ECS meta data if no env vars).
22089 Only applies if access_key_id and secret_access_key is blank.
22090 Choose a number from below, or type in your own boolean value (true or false).
22091 Press Enter for the default (false).
22092 1 / Enter AWS credentials in the next step.
22093 \ (false)
22094 2 / Get AWS credentials from the environment (env vars or IAM).
22095 \ (true)
22096 env_auth>
22097 Option access_key_id.
22099 AWS Access Key ID.
22100 Leave blank for anonymous access or runtime credentials.
22101 Enter a value. Press Enter to leave empty.
22102 access_key_id> YOUR_ACCESS_KEY
22103 Option secret_access_key.
22105 AWS Secret Access Key (password).
22106 Leave blank for anonymous access or runtime credentials.
22107 Enter a value. Press Enter to leave empty.
22108 secret_access_key> YOUR_SECRET_KEY
22109 Option acl.
22111 Canned ACL used when creating buckets and storing or copying objects.
22112 This ACL is used for creating objects and if bucket_acl isn't set, for creating buckets too.
22113 For more info visit https://docs.aws.amazon.com/AmazonS3/latest/dev/acl-overview.html#canned-acl
22114 Note that this ACL is applied when server-side copying objects as S3
22115 doesn't copy the ACL from the source but rather writes a fresh one.
22116 Choose a number from below, or type in your own value.
22117 Press Enter to leave empty.
22118 / Owner gets FULL_CONTROL.
22119 1 | No one else has access rights (default).
22120 \ (private)
22121 / Owner gets FULL_CONTROL.
22122 2 | The AllUsers group gets READ access.
22123 \ (public-read)
22124 / Owner gets FULL_CONTROL.
22125 3 | The AllUsers group gets READ and WRITE access.
22126 | Granting this on a bucket is generally not recommended.
22127 \ (public-read-write)
22128 / Owner gets FULL_CONTROL.
22129 4 | The AuthenticatedUsers group gets READ access.
22130 \ (authenticated-read)
22131 / Object owner gets FULL_CONTROL.
22132 5 | Bucket owner gets READ access.
22133 | If you specify this canned ACL when creating a bucket, Amazon S3 ignores it.
22134 \ (bucket-owner-read)
22135 / Both the object owner and the bucket owner get FULL_CONTROL over the object.
22136 6 | If you specify this canned ACL when creating a bucket, Amazon S3 ignores it.
22137 \ (bucket-owner-full-control)
22138 acl>
22139 Edit advanced config?
22141 y) Yes
22142 n) No (default)
22143 y/n>
22144 Configuration complete.
22146 Options:
22147 - type: s3
22148 - provider: IDrive
22149 - access_key_id: YOUR_ACCESS_KEY
22150 - secret_access_key: YOUR_SECRET_KEY
22151 - endpoint: q9d9.la12.idrivee2-5.com
22152 Keep this "e2" remote?
22153 y) Yes this is OK (default)
22154 e) Edit this remote
22155 d) Delete this remote
22156 y/e/d> y
22157 IONOS Cloud
22159 IONOS S3 Object Storage (https://cloud.ionos.com/storage/object-stor‐
22160 age) is a service offered by IONOS for storing and accessing unstruc‐
22161 tured data. To connect to the service, you will need an access key and
22162 a secret key. These can be found in the Data Center Designer
22163 (https://dcd.ionos.com/), by selecting Manager resources > Object Stor‐
22164 age Key Manager.
22165 Here is an example of a configuration. First, run rclone config. This
22167 will walk you through an interactive setup process. Type n to add the
22168 new remote, and then enter a name:
22169 Enter name for new remote.
22171 name> ionos-fra
22172 Type s3 to choose the connection type:
22174 Option Storage.
22176 Type of storage to configure.
22177 Choose a number from below, or type in your own value.
22178 [snip]
22179 XX / Amazon S3 Compliant Storage Providers including AWS, Alibaba, Ceph, China Mobile, Cloudflare, ArvanCloud, Digital Ocean, Dreamhost, Huawei OBS, IBM COS, IDrive e2, IONOS Cloud, Lyve Cloud, Minio, Netease, RackCorp, Scaleway, SeaweedFS, StackPath, Storj, Tencent COS and Wasabi
22180 \ (s3)
22181 [snip]
22182 Storage> s3
22183 Type IONOS:
22185 Option provider.
22187 Choose your S3 provider.
22188 Choose a number from below, or type in your own value.
22189 Press Enter to leave empty.
22190 [snip]
22191 XX / IONOS Cloud
22192 \ (IONOS)
22193 [snip]
22194 provider> IONOS
22195 Press Enter to choose the default option Enter AWS credentials in the
22197 next step:
22198 Option env_auth.
22200 Get AWS credentials from runtime (environment variables or EC2/ECS meta data if no env vars).
22201 Only applies if access_key_id and secret_access_key is blank.
22202 Choose a number from below, or type in your own boolean value (true or false).
22203 Press Enter for the default (false).
22204 1 / Enter AWS credentials in the next step.
22205 \ (false)
22206 2 / Get AWS credentials from the environment (env vars or IAM).
22207 \ (true)
22208 env_auth>
22209 Enter your Access Key and Secret key. These can be retrieved in the
22211 Data Center Designer (https://dcd.ionos.com/), click on the menu “Man‐
22212 ager resources” / "Object Storage Key Manager".
22213 Option access_key_id.
22215 AWS Access Key ID.
22216 Leave blank for anonymous access or runtime credentials.
22217 Enter a value. Press Enter to leave empty.
22218 access_key_id> YOUR_ACCESS_KEY
22219 Option secret_access_key.
22221 AWS Secret Access Key (password).
22222 Leave blank for anonymous access or runtime credentials.
22223 Enter a value. Press Enter to leave empty.
22224 secret_access_key> YOUR_SECRET_KEY
22225 Choose the region where your bucket is located:
22227 Option region.
22229 Region where your bucket will be created and your data stored.
22230 Choose a number from below, or type in your own value.
22231 Press Enter to leave empty.
22232 1 / Frankfurt, Germany
22233 \ (de)
22234 2 / Berlin, Germany
22235 \ (eu-central-2)
22236 3 / Logrono, Spain
22237 \ (eu-south-2)
22238 region> 2
22239 Choose the endpoint from the same region:
22241 Option endpoint.
22243 Endpoint for IONOS S3 Object Storage.
22244 Specify the endpoint from the same region.
22245 Choose a number from below, or type in your own value.
22246 Press Enter to leave empty.
22247 1 / Frankfurt, Germany
22248 \ (s3-eu-central-1.ionoscloud.com)
22249 2 / Berlin, Germany
22250 \ (s3-eu-central-2.ionoscloud.com)
22251 3 / Logrono, Spain
22252 \ (s3-eu-south-2.ionoscloud.com)
22253 endpoint> 1
22254 Press Enter to choose the default option or choose the desired ACL set‐
22256 ting:
22257 Option acl.
22259 Canned ACL used when creating buckets and storing or copying objects.
22260 This ACL is used for creating objects and if bucket_acl isn't set, for creating buckets too.
22261 For more info visit https://docs.aws.amazon.com/AmazonS3/latest/dev/acl-overview.html#canned-acl
22262 Note that this ACL is applied when server-side copying objects as S3
22263 doesn't copy the ACL from the source but rather writes a fresh one.
22264 Choose a number from below, or type in your own value.
22265 Press Enter to leave empty.
22266 / Owner gets FULL_CONTROL.
22267 1 | No one else has access rights (default).
22268 \ (private)
22269 / Owner gets FULL_CONTROL.
22270 [snip]
22271 acl>
22272 Press Enter to skip the advanced config:
22274 Edit advanced config?
22276 y) Yes
22277 n) No (default)
22278 y/n>
22279 Press Enter to save the configuration, and then q to quit the configu‐
22281 ration process:
22282 Configuration complete.
22284 Options:
22285 - type: s3
22286 - provider: IONOS
22287 - access_key_id: YOUR_ACCESS_KEY
22288 - secret_access_key: YOUR_SECRET_KEY
22289 - endpoint: s3-eu-central-1.ionoscloud.com
22290 Keep this "ionos-fra" remote?
22291 y) Yes this is OK (default)
22292 e) Edit this remote
22293 d) Delete this remote
22294 y/e/d> y
22295 Done! Now you can try some commands (for macOS, use ./rclone instead of
22297 rclone).
22298 1) Create a bucket (the name must be unique within the whole IONOS S3)
22300 rclone mkdir ionos-fra:my-bucket
22302 2) List available buckets
22304 rclone lsd ionos-fra:
22306 4) Copy a file from local to remote
22308 rclone copy /Users/file.txt ionos-fra:my-bucket
22310 3) List contents of a bucket
22312 rclone ls ionos-fra:my-bucket
22314 5) Copy a file from remote to local
22316 rclone copy ionos-fra:my-bucket/file.txt
22318 Minio
22320 Minio (https://minio.io/) is an object storage server built for cloud
22321 application developers and devops.
22322 It is very easy to install and provides an S3 compatible server which
22324 can be used by rclone.
22325 To use it, install Minio following the instructions here
22327 (https://docs.minio.io/docs/minio-quickstart-guide).
22328 When it configures itself Minio will print something like this
22330 Endpoint: http://192.168.1.106:9000 http://172.23.0.1:9000
22332 AccessKey: USWUXHGYZQYFYFFIT3RE
22333 SecretKey: MOJRH0mkL1IPauahWITSVvyDrQbEEIwljvmxdq03
22334 Region: us-east-1
22335 SQS ARNs: arn:minio:sqs:us-east-1:1:redis arn:minio:sqs:us-east-1:2:redis
22336 Browser Access:
22338 http://192.168.1.106:9000 http://172.23.0.1:9000
22339 Command-line Access: https://docs.minio.io/docs/minio-client-quickstart-guide
22341 $ mc config host add myminio http://192.168.1.106:9000 USWUXHGYZQYFYFFIT3RE MOJRH0mkL1IPauahWITSVvyDrQbEEIwljvmxdq03
22342 Object API (Amazon S3 compatible):
22344 Go: https://docs.minio.io/docs/golang-client-quickstart-guide
22345 Java: https://docs.minio.io/docs/java-client-quickstart-guide
22346 Python: https://docs.minio.io/docs/python-client-quickstart-guide
22347 JavaScript: https://docs.minio.io/docs/javascript-client-quickstart-guide
22348 .NET: https://docs.minio.io/docs/dotnet-client-quickstart-guide
22349 Drive Capacity: 26 GiB Free, 165 GiB Total
22351 These details need to go into rclone config like this. Note that it is
22353 important to put the region in as stated above.
22354 env_auth> 1
22356 access_key_id> USWUXHGYZQYFYFFIT3RE
22357 secret_access_key> MOJRH0mkL1IPauahWITSVvyDrQbEEIwljvmxdq03
22358 region> us-east-1
22359 endpoint> http://192.168.1.106:9000
22360 location_constraint>
22361 server_side_encryption>
22362 Which makes the config file look like this
22364 [minio]
22366 type = s3
22367 provider = Minio
22368 env_auth = false
22369 access_key_id = USWUXHGYZQYFYFFIT3RE
22370 secret_access_key = MOJRH0mkL1IPauahWITSVvyDrQbEEIwljvmxdq03
22371 region = us-east-1
22372 endpoint = http://192.168.1.106:9000
22373 location_constraint =
22374 server_side_encryption =
22375 So once set up, for example, to copy files into a bucket
22377 rclone copy /path/to/files minio:bucket
22379 Qiniu Cloud Object Storage (Kodo)
22381 Qiniu Cloud Object Storage (Kodo) (https://www.qiniu.com/en/prod‐
22382 ucts/kodo), a completely independent-researched core technology which
22383 is proven by repeated customer experience has occupied absolute leading
22384 market leader position. Kodo can be widely applied to mass data man‐
22385 agement.
22386 To configure access to Qiniu Kodo, follow the steps below:
22388 1. Run rclone config and select n for a new remote.
22390 rclone config
22392 No remotes found, make a new one?
22393 n) New remote
22394 s) Set configuration password
22395 q) Quit config
22396 n/s/q> n
22397 2. Give the name of the configuration. For example, name it 'qiniu'.
22399 name> qiniu
22401 3. Select s3 storage.
22403 Choose a number from below, or type in your own value
22405 1 / 1Fichier
22406 \ (fichier)
22407 2 / Akamai NetStorage
22408 \ (netstorage)
22409 3 / Alias for an existing remote
22410 \ (alias)
22411 4 / Amazon Drive
22412 \ (amazon cloud drive)
22413 5 / Amazon S3 Compliant Storage Providers including AWS, Alibaba, Ceph, China Mobile, Cloudflare, ArvanCloud, Digital Ocean, Dreamhost, Huawei OBS, IBM COS, IDrive e2, Lyve Cloud, Minio, Netease, RackCorp, Scaleway, SeaweedFS, StackPath, Storj, Tencent COS, Qiniu and Wasabi
22414 \ (s3)
22415 [snip]
22416 Storage> s3
22417 4. Select Qiniu provider.
22419 Choose a number from below, or type in your own value
22421 1 / Amazon Web Services (AWS) S3
22422 \ "AWS"
22423 [snip]
22424 22 / Qiniu Object Storage (Kodo)
22425 \ (Qiniu)
22426 [snip]
22427 provider> Qiniu
22428 5. Enter your SecretId and SecretKey of Qiniu Kodo.
22430 Get AWS credentials from runtime (environment variables or EC2/ECS meta data if no env vars).
22432 Only applies if access_key_id and secret_access_key is blank.
22433 Enter a boolean value (true or false). Press Enter for the default ("false").
22434 Choose a number from below, or type in your own value
22435 1 / Enter AWS credentials in the next step
22436 \ "false"
22437 2 / Get AWS credentials from the environment (env vars or IAM)
22438 \ "true"
22439 env_auth> 1
22440 AWS Access Key ID.
22441 Leave blank for anonymous access or runtime credentials.
22442 Enter a string value. Press Enter for the default ("").
22443 access_key_id> AKIDxxxxxxxxxx
22444 AWS Secret Access Key (password)
22445 Leave blank for anonymous access or runtime credentials.
22446 Enter a string value. Press Enter for the default ("").
22447 secret_access_key> xxxxxxxxxxx
22448 6. Select endpoint for Qiniu Kodo. This is the standard endpoint for
22450 different region.
22451 / The default endpoint - a good choice if you are unsure.
22453 1 | East China Region 1.
22454 | Needs location constraint cn-east-1.
22455 \ (cn-east-1)
22456 / East China Region 2.
22457 2 | Needs location constraint cn-east-2.
22458 \ (cn-east-2)
22459 / North China Region 1.
22460 3 | Needs location constraint cn-north-1.
22461 \ (cn-north-1)
22462 / South China Region 1.
22463 4 | Needs location constraint cn-south-1.
22464 \ (cn-south-1)
22465 / North America Region.
22466 5 | Needs location constraint us-north-1.
22467 \ (us-north-1)
22468 / Southeast Asia Region 1.
22469 6 | Needs location constraint ap-southeast-1.
22470 \ (ap-southeast-1)
22471 / Northeast Asia Region 1.
22472 7 | Needs location constraint ap-northeast-1.
22473 \ (ap-northeast-1)
22474 [snip]
22475 endpoint> 1
22476 Option endpoint.
22478 Endpoint for Qiniu Object Storage.
22479 Choose a number from below, or type in your own value.
22480 Press Enter to leave empty.
22481 1 / East China Endpoint 1
22482 \ (s3-cn-east-1.qiniucs.com)
22483 2 / East China Endpoint 2
22484 \ (s3-cn-east-2.qiniucs.com)
22485 3 / North China Endpoint 1
22486 \ (s3-cn-north-1.qiniucs.com)
22487 4 / South China Endpoint 1
22488 \ (s3-cn-south-1.qiniucs.com)
22489 5 / North America Endpoint 1
22490 \ (s3-us-north-1.qiniucs.com)
22491 6 / Southeast Asia Endpoint 1
22492 \ (s3-ap-southeast-1.qiniucs.com)
22493 7 / Northeast Asia Endpoint 1
22494 \ (s3-ap-northeast-1.qiniucs.com)
22495 endpoint> 1
22496 Option location_constraint.
22498 Location constraint - must be set to match the Region.
22499 Used when creating buckets only.
22500 Choose a number from below, or type in your own value.
22501 Press Enter to leave empty.
22502 1 / East China Region 1
22503 \ (cn-east-1)
22504 2 / East China Region 2
22505 \ (cn-east-2)
22506 3 / North China Region 1
22507 \ (cn-north-1)
22508 4 / South China Region 1
22509 \ (cn-south-1)
22510 5 / North America Region 1
22511 \ (us-north-1)
22512 6 / Southeast Asia Region 1
22513 \ (ap-southeast-1)
22514 7 / Northeast Asia Region 1
22515 \ (ap-northeast-1)
22516 location_constraint> 1
22517 7. Choose acl and storage class.
22519 Note that this ACL is applied when server-side copying objects as S3
22521 doesn't copy the ACL from the source but rather writes a fresh one.
22522 Enter a string value. Press Enter for the default ("").
22523 Choose a number from below, or type in your own value
22524 / Owner gets FULL_CONTROL.
22525 1 | No one else has access rights (default).
22526 \ (private)
22527 / Owner gets FULL_CONTROL.
22528 2 | The AllUsers group gets READ access.
22529 \ (public-read)
22530 [snip]
22531 acl> 2
22532 The storage class to use when storing new objects in Tencent COS.
22533 Enter a string value. Press Enter for the default ("").
22534 Choose a number from below, or type in your own value
22535 1 / Standard storage class
22536 \ (STANDARD)
22537 2 / Infrequent access storage mode
22538 \ (LINE)
22539 3 / Archive storage mode
22540 \ (GLACIER)
22541 4 / Deep archive storage mode
22542 \ (DEEP_ARCHIVE)
22543 [snip]
22544 storage_class> 1
22545 Edit advanced config? (y/n)
22546 y) Yes
22547 n) No (default)
22548 y/n> n
22549 Remote config
22550 --------------------
22551 [qiniu]
22552 - type: s3
22553 - provider: Qiniu
22554 - access_key_id: xxx
22555 - secret_access_key: xxx
22556 - region: cn-east-1
22557 - endpoint: s3-cn-east-1.qiniucs.com
22558 - location_constraint: cn-east-1
22559 - acl: public-read
22560 - storage_class: STANDARD
22561 --------------------
22562 y) Yes this is OK (default)
22563 e) Edit this remote
22564 d) Delete this remote
22565 y/e/d> y
22566 Current remotes:
22567 Name Type
22569 ==== ====
22570 qiniu s3
22571 RackCorp
22573 RackCorp Object Storage (https://www.rackcorp.com/storage/s3storage) is
22574 an S3 compatible object storage platform from your friendly cloud
22575 provider RackCorp. The service is fast, reliable, well priced and lo‐
22576 cated in many strategic locations unserviced by others, to ensure you
22577 can maintain data sovereignty.
22578 Before you can use RackCorp Object Storage, you'll need to "sign up
22580 (https://www.rackcorp.com/signup)" for an account on our "portal
22581 (https://portal.rackcorp.com)". Next you can create an access key, a
22582 secret key and buckets, in your location of choice with ease. These
22583 details are required for the next steps of configuration, when rclone
22584 config asks for your access_key_id and secret_access_key.
22585 Your config should end up looking a bit like this:
22587 [RCS3-demo-config]
22589 type = s3
22590 provider = RackCorp
22591 env_auth = true
22592 access_key_id = YOURACCESSKEY
22593 secret_access_key = YOURSECRETACCESSKEY
22594 region = au-nsw
22595 endpoint = s3.rackcorp.com
22596 location_constraint = au-nsw
22597 Scaleway
22599 Scaleway (https://www.scaleway.com/object-storage/) The Object Storage
22600 platform allows you to store anything from backups, logs and web assets
22601 to documents and photos. Files can be dropped from the Scaleway con‐
22602 sole or transferred through our API and CLI or using any S3-compatible
22603 tool.
22604 Scaleway provides an S3 interface which can be configured for use with
22606 rclone like this:
22607 [scaleway]
22609 type = s3
22610 provider = Scaleway
22611 env_auth = false
22612 endpoint = s3.nl-ams.scw.cloud
22613 access_key_id = SCWXXXXXXXXXXXXXX
22614 secret_access_key = 1111111-2222-3333-44444-55555555555555
22615 region = nl-ams
22616 location_constraint =
22617 acl = private
22618 server_side_encryption =
22619 storage_class =
22620 C14 Cold Storage (https://www.online.net/en/storage/c14-cold-storage)
22622 is the low-cost S3 Glacier alternative from Scaleway and it works the
22623 same way as on S3 by accepting the "GLACIER" storage_class. So you can
22624 configure your remote with the storage_class = GLACIER option to upload
22625 directly to C14. Don't forget that in this state you can't read files
22626 back after, you will need to restore them to "STANDARD" storage_class
22627 first before being able to read them (see "restore" section above)
22628 Seagate Lyve Cloud
22630 Seagate Lyve Cloud (https://www.seagate.com/gb/en/services/cloud/stor‐
22631 age/) is an S3 compatible object storage platform from Seagate
22632 (https://seagate.com/) intended for enterprise use.
22633 Here is a config run through for a remote called remote - you may
22635 choose a different name of course. Note that to create an access key
22636 and secret key you will need to create a service account first.
22637 $ rclone config
22639 No remotes found, make a new one?
22640 n) New remote
22641 s) Set configuration password
22642 q) Quit config
22643 n/s/q> n
22644 name> remote
22645 Choose s3 backend
22647 Type of storage to configure.
22649 Choose a number from below, or type in your own value.
22650 [snip]
22651 XX / Amazon S3 Compliant Storage Providers including AWS, Alibaba, Ceph, ChinaMobile, ArvanCloud, Digital Ocean, Dreamhost, Huawei OBS, IBM COS, Lyve Cloud, Minio, RackCorp, SeaweedFS, and Tencent COS
22652 \ (s3)
22653 [snip]
22654 Storage> s3
22655 Choose LyveCloud as S3 provider
22657 Choose your S3 provider.
22659 Choose a number from below, or type in your own value.
22660 Press Enter to leave empty.
22661 [snip]
22662 XX / Seagate Lyve Cloud
22663 \ (LyveCloud)
22664 [snip]
22665 provider> LyveCloud
22666 Take the default (just press enter) to enter access key and secret in
22668 the config file.
22669 Get AWS credentials from runtime (environment variables or EC2/ECS meta data if no env vars).
22671 Only applies if access_key_id and secret_access_key is blank.
22672 Choose a number from below, or type in your own boolean value (true or false).
22673 Press Enter for the default (false).
22674 1 / Enter AWS credentials in the next step.
22675 \ (false)
22676 2 / Get AWS credentials from the environment (env vars or IAM).
22677 \ (true)
22678 env_auth>
22679 AWS Access Key ID.
22681 Leave blank for anonymous access or runtime credentials.
22682 Enter a value. Press Enter to leave empty.
22683 access_key_id> XXX
22684 AWS Secret Access Key (password).
22686 Leave blank for anonymous access or runtime credentials.
22687 Enter a value. Press Enter to leave empty.
22688 secret_access_key> YYY
22689 Leave region blank
22691 Region to connect to.
22693 Leave blank if you are using an S3 clone and you don't have a region.
22694 Choose a number from below, or type in your own value.
22695 Press Enter to leave empty.
22696 / Use this if unsure.
22697 1 | Will use v4 signatures and an empty region.
22698 \ ()
22699 / Use this only if v4 signatures don't work.
22700 2 | E.g. pre Jewel/v10 CEPH.
22701 \ (other-v2-signature)
22702 region>
22703 Choose an endpoint from the list
22705 Endpoint for S3 API.
22707 Required when using an S3 clone.
22708 Choose a number from below, or type in your own value.
22709 Press Enter to leave empty.
22710 1 / Seagate Lyve Cloud US East 1 (Virginia)
22711 \ (s3.us-east-1.lyvecloud.seagate.com)
22712 2 / Seagate Lyve Cloud US West 1 (California)
22713 \ (s3.us-west-1.lyvecloud.seagate.com)
22714 3 / Seagate Lyve Cloud AP Southeast 1 (Singapore)
22715 \ (s3.ap-southeast-1.lyvecloud.seagate.com)
22716 endpoint> 1
22717 Leave location constraint blank
22719 Location constraint - must be set to match the Region.
22721 Leave blank if not sure. Used when creating buckets only.
22722 Enter a value. Press Enter to leave empty.
22723 location_constraint>
22724 Choose default ACL (private).
22726 Canned ACL used when creating buckets and storing or copying objects.
22728 This ACL is used for creating objects and if bucket_acl isn't set, for creating buckets too.
22729 For more info visit https://docs.aws.amazon.com/AmazonS3/latest/dev/acl-overview.html#canned-acl
22730 Note that this ACL is applied when server-side copying objects as S3
22731 doesn't copy the ACL from the source but rather writes a fresh one.
22732 Choose a number from below, or type in your own value.
22733 Press Enter to leave empty.
22734 / Owner gets FULL_CONTROL.
22735 1 | No one else has access rights (default).
22736 \ (private)
22737 [snip]
22738 acl>
22739 And the config file should end up looking like this:
22741 [remote]
22743 type = s3
22744 provider = LyveCloud
22745 access_key_id = XXX
22746 secret_access_key = YYY
22747 endpoint = s3.us-east-1.lyvecloud.seagate.com
22748 SeaweedFS
22750 SeaweedFS (https://github.com/chrislusf/seaweedfs/) is a distributed
22751 storage system for blobs, objects, files, and data lake, with O(1) disk
22752 seek and a scalable file metadata store. It has an S3 compatible ob‐
22753 ject storage interface. SeaweedFS can also act as a gateway to remote
22754 S3 compatible object store (https://github.com/chrislusf/seaweedfs/wi‐
22755 ki/Gateway-to-Remote-Object-Storage) to cache data and metadata with
22756 asynchronous write back, for fast local speed and minimize access cost.
22757 Assuming the SeaweedFS are configured with weed shell as such:
22759 > s3.bucket.create -name foo
22761 > s3.configure -access_key=any -secret_key=any -buckets=foo -user=me -actions=Read,Write,List,Tagging,Admin -apply
22762 {
22763 "identities": [
22764 {
22765 "name": "me",
22766 "credentials": [
22767 {
22768 "accessKey": "any",
22769 "secretKey": "any"
22770 }
22771 ],
22772 "actions": [
22773 "Read:foo",
22774 "Write:foo",
22775 "List:foo",
22776 "Tagging:foo",
22777 "Admin:foo"
22778 ]
22779 }
22780 ]
22781 }
22782 To use rclone with SeaweedFS, above configuration should end up with
22784 something like this in your config:
22785 [seaweedfs_s3]
22787 type = s3
22788 provider = SeaweedFS
22789 access_key_id = any
22790 secret_access_key = any
22791 endpoint = localhost:8333
22792 So once set up, for example to copy files into a bucket
22794 rclone copy /path/to/files seaweedfs_s3:foo
22796 Wasabi
22798 Wasabi (https://wasabi.com) is a cloud-based object storage service for
22799 a broad range of applications and use cases. Wasabi is designed for
22800 individuals and organizations that require a high-performance, reli‐
22801 able, and secure data storage infrastructure at minimal cost.
22802 Wasabi provides an S3 interface which can be configured for use with
22804 rclone like this.
22805 No remotes found, make a new one?
22807 n) New remote
22808 s) Set configuration password
22809 n/s> n
22810 name> wasabi
22811 Type of storage to configure.
22812 Choose a number from below, or type in your own value
22813 [snip]
22814 XX / Amazon S3 (also Dreamhost, Ceph, ChinaMobile, ArvanCloud, Minio)
22815 \ "s3"
22816 [snip]
22817 Storage> s3
22818 Get AWS credentials from runtime (environment variables or EC2/ECS meta data if no env vars). Only applies if access_key_id and secret_access_key is blank.
22819 Choose a number from below, or type in your own value
22820 1 / Enter AWS credentials in the next step
22821 \ "false"
22822 2 / Get AWS credentials from the environment (env vars or IAM)
22823 \ "true"
22824 env_auth> 1
22825 AWS Access Key ID - leave blank for anonymous access or runtime credentials.
22826 access_key_id> YOURACCESSKEY
22827 AWS Secret Access Key (password) - leave blank for anonymous access or runtime credentials.
22828 secret_access_key> YOURSECRETACCESSKEY
22829 Region to connect to.
22830 Choose a number from below, or type in your own value
22831 / The default endpoint - a good choice if you are unsure.
22832 1 | US Region, Northern Virginia, or Pacific Northwest.
22833 | Leave location constraint empty.
22834 \ "us-east-1"
22835 [snip]
22836 region> us-east-1
22837 Endpoint for S3 API.
22838 Leave blank if using AWS to use the default endpoint for the region.
22839 Specify if using an S3 clone such as Ceph.
22840 endpoint> s3.wasabisys.com
22841 Location constraint - must be set to match the Region. Used when creating buckets only.
22842 Choose a number from below, or type in your own value
22843 1 / Empty for US Region, Northern Virginia, or Pacific Northwest.
22844 \ ""
22845 [snip]
22846 location_constraint>
22847 Canned ACL used when creating buckets and/or storing objects in S3.
22848 For more info visit https://docs.aws.amazon.com/AmazonS3/latest/dev/acl-overview.html#canned-acl
22849 Choose a number from below, or type in your own value
22850 1 / Owner gets FULL_CONTROL. No one else has access rights (default).
22851 \ "private"
22852 [snip]
22853 acl>
22854 The server-side encryption algorithm used when storing this object in S3.
22855 Choose a number from below, or type in your own value
22856 1 / None
22857 \ ""
22858 2 / AES256
22859 \ "AES256"
22860 server_side_encryption>
22861 The storage class to use when storing objects in S3.
22862 Choose a number from below, or type in your own value
22863 1 / Default
22864 \ ""
22865 2 / Standard storage class
22866 \ "STANDARD"
22867 3 / Reduced redundancy storage class
22868 \ "REDUCED_REDUNDANCY"
22869 4 / Standard Infrequent Access storage class
22870 \ "STANDARD_IA"
22871 storage_class>
22872 Remote config
22873 --------------------
22874 [wasabi]
22875 env_auth = false
22876 access_key_id = YOURACCESSKEY
22877 secret_access_key = YOURSECRETACCESSKEY
22878 region = us-east-1
22879 endpoint = s3.wasabisys.com
22880 location_constraint =
22881 acl =
22882 server_side_encryption =
22883 storage_class =
22884 --------------------
22885 y) Yes this is OK
22886 e) Edit this remote
22887 d) Delete this remote
22888 y/e/d> y
22889 This will leave the config file looking like this.
22891 [wasabi]
22893 type = s3
22894 provider = Wasabi
22895 env_auth = false
22896 access_key_id = YOURACCESSKEY
22897 secret_access_key = YOURSECRETACCESSKEY
22898 region =
22899 endpoint = s3.wasabisys.com
22900 location_constraint =
22901 acl =
22902 server_side_encryption =
22903 storage_class =
22904 Alibaba OSS
22906 Here is an example of making an Alibaba Cloud (Aliyun) OSS
22907 (https://www.alibabacloud.com/product/oss/) configuration. First run:
22908 rclone config
22910 This will guide you through an interactive setup process.
22912 No remotes found, make a new one?
22914 n) New remote
22915 s) Set configuration password
22916 q) Quit config
22917 n/s/q> n
22918 name> oss
22919 Type of storage to configure.
22920 Enter a string value. Press Enter for the default ("").
22921 Choose a number from below, or type in your own value
22922 [snip]
22923 4 / Amazon S3 Compliant Storage Providers including AWS, Alibaba, Ceph, ChinaMobile, ArvanCloud, Digital Ocean, Dreamhost, Huawei OBS, IBM COS, Minio, and Tencent COS
22924 \ "s3"
22925 [snip]
22926 Storage> s3
22927 Choose your S3 provider.
22928 Enter a string value. Press Enter for the default ("").
22929 Choose a number from below, or type in your own value
22930 1 / Amazon Web Services (AWS) S3
22931 \ "AWS"
22932 2 / Alibaba Cloud Object Storage System (OSS) formerly Aliyun
22933 \ "Alibaba"
22934 3 / Ceph Object Storage
22935 \ "Ceph"
22936 [snip]
22937 provider> Alibaba
22938 Get AWS credentials from runtime (environment variables or EC2/ECS meta data if no env vars).
22939 Only applies if access_key_id and secret_access_key is blank.
22940 Enter a boolean value (true or false). Press Enter for the default ("false").
22941 Choose a number from below, or type in your own value
22942 1 / Enter AWS credentials in the next step
22943 \ "false"
22944 2 / Get AWS credentials from the environment (env vars or IAM)
22945 \ "true"
22946 env_auth> 1
22947 AWS Access Key ID.
22948 Leave blank for anonymous access or runtime credentials.
22949 Enter a string value. Press Enter for the default ("").
22950 access_key_id> accesskeyid
22951 AWS Secret Access Key (password)
22952 Leave blank for anonymous access or runtime credentials.
22953 Enter a string value. Press Enter for the default ("").
22954 secret_access_key> secretaccesskey
22955 Endpoint for OSS API.
22956 Enter a string value. Press Enter for the default ("").
22957 Choose a number from below, or type in your own value
22958 1 / East China 1 (Hangzhou)
22959 \ "oss-cn-hangzhou.aliyuncs.com"
22960 2 / East China 2 (Shanghai)
22961 \ "oss-cn-shanghai.aliyuncs.com"
22962 3 / North China 1 (Qingdao)
22963 \ "oss-cn-qingdao.aliyuncs.com"
22964 [snip]
22965 endpoint> 1
22966 Canned ACL used when creating buckets and storing or copying objects.
22967 Note that this ACL is applied when server-side copying objects as S3
22969 doesn't copy the ACL from the source but rather writes a fresh one.
22970 Enter a string value. Press Enter for the default ("").
22971 Choose a number from below, or type in your own value
22972 1 / Owner gets FULL_CONTROL. No one else has access rights (default).
22973 \ "private"
22974 2 / Owner gets FULL_CONTROL. The AllUsers group gets READ access.
22975 \ "public-read"
22976 / Owner gets FULL_CONTROL. The AllUsers group gets READ and WRITE access.
22977 [snip]
22978 acl> 1
22979 The storage class to use when storing new objects in OSS.
22980 Enter a string value. Press Enter for the default ("").
22981 Choose a number from below, or type in your own value
22982 1 / Default
22983 \ ""
22984 2 / Standard storage class
22985 \ "STANDARD"
22986 3 / Archive storage mode.
22987 \ "GLACIER"
22988 4 / Infrequent access storage mode.
22989 \ "STANDARD_IA"
22990 storage_class> 1
22991 Edit advanced config? (y/n)
22992 y) Yes
22993 n) No
22994 y/n> n
22995 Remote config
22996 --------------------
22997 [oss]
22998 type = s3
22999 provider = Alibaba
23000 env_auth = false
23001 access_key_id = accesskeyid
23002 secret_access_key = secretaccesskey
23003 endpoint = oss-cn-hangzhou.aliyuncs.com
23004 acl = private
23005 storage_class = Standard
23006 --------------------
23007 y) Yes this is OK
23008 e) Edit this remote
23009 d) Delete this remote
23010 y/e/d> y
23011 China Mobile Ecloud Elastic Object Storage (EOS)
23013 Here is an example of making an China Mobile Ecloud Elastic Object
23014 Storage (EOS) (https:///ecloud.10086.cn/home/product-introduction/eos/)
23015 configuration. First run:
23016 rclone config
23018 This will guide you through an interactive setup process.
23020 No remotes found, make a new one?
23022 n) New remote
23023 s) Set configuration password
23024 q) Quit config
23025 n/s/q> n
23026 name> ChinaMobile
23027 Option Storage.
23028 Type of storage to configure.
23029 Choose a number from below, or type in your own value.
23030 ...
23031 5 / Amazon S3 Compliant Storage Providers including AWS, Alibaba, Ceph, ChinaMobile, Digital Ocean, Dreamhost, Huawei OBS, IBM COS, Lyve Cloud, Minio, RackCorp, SeaweedFS, and Tencent COS
23032 \ (s3)
23033 ...
23034 Storage> s3
23035 Option provider.
23036 Choose your S3 provider.
23037 Choose a number from below, or type in your own value.
23038 Press Enter to leave empty.
23039 ...
23040 4 / China Mobile Ecloud Elastic Object Storage (EOS)
23041 \ (ChinaMobile)
23042 ...
23043 provider> ChinaMobile
23044 Option env_auth.
23045 Get AWS credentials from runtime (environment variables or EC2/ECS meta data if no env vars).
23046 Only applies if access_key_id and secret_access_key is blank.
23047 Choose a number from below, or type in your own boolean value (true or false).
23048 Press Enter for the default (false).
23049 1 / Enter AWS credentials in the next step.
23050 \ (false)
23051 2 / Get AWS credentials from the environment (env vars or IAM).
23052 \ (true)
23053 env_auth>
23054 Option access_key_id.
23055 AWS Access Key ID.
23056 Leave blank for anonymous access or runtime credentials.
23057 Enter a value. Press Enter to leave empty.
23058 access_key_id> accesskeyid
23059 Option secret_access_key.
23060 AWS Secret Access Key (password).
23061 Leave blank for anonymous access or runtime credentials.
23062 Enter a value. Press Enter to leave empty.
23063 secret_access_key> secretaccesskey
23064 Option endpoint.
23065 Endpoint for China Mobile Ecloud Elastic Object Storage (EOS) API.
23066 Choose a number from below, or type in your own value.
23067 Press Enter to leave empty.
23068 / The default endpoint - a good choice if you are unsure.
23069 1 | East China (Suzhou)
23070 \ (eos-wuxi-1.cmecloud.cn)
23071 2 / East China (Jinan)
23072 \ (eos-jinan-1.cmecloud.cn)
23073 3 / East China (Hangzhou)
23074 \ (eos-ningbo-1.cmecloud.cn)
23075 4 / East China (Shanghai-1)
23076 \ (eos-shanghai-1.cmecloud.cn)
23077 5 / Central China (Zhengzhou)
23078 \ (eos-zhengzhou-1.cmecloud.cn)
23079 6 / Central China (Changsha-1)
23080 \ (eos-hunan-1.cmecloud.cn)
23081 7 / Central China (Changsha-2)
23082 \ (eos-zhuzhou-1.cmecloud.cn)
23083 8 / South China (Guangzhou-2)
23084 \ (eos-guangzhou-1.cmecloud.cn)
23085 9 / South China (Guangzhou-3)
23086 \ (eos-dongguan-1.cmecloud.cn)
23087 10 / North China (Beijing-1)
23088 \ (eos-beijing-1.cmecloud.cn)
23089 11 / North China (Beijing-2)
23090 \ (eos-beijing-2.cmecloud.cn)
23091 12 / North China (Beijing-3)
23092 \ (eos-beijing-4.cmecloud.cn)
23093 13 / North China (Huhehaote)
23094 \ (eos-huhehaote-1.cmecloud.cn)
23095 14 / Southwest China (Chengdu)
23096 \ (eos-chengdu-1.cmecloud.cn)
23097 15 / Southwest China (Chongqing)
23098 \ (eos-chongqing-1.cmecloud.cn)
23099 16 / Southwest China (Guiyang)
23100 \ (eos-guiyang-1.cmecloud.cn)
23101 17 / Nouthwest China (Xian)
23102 \ (eos-xian-1.cmecloud.cn)
23103 18 / Yunnan China (Kunming)
23104 \ (eos-yunnan.cmecloud.cn)
23105 19 / Yunnan China (Kunming-2)
23106 \ (eos-yunnan-2.cmecloud.cn)
23107 20 / Tianjin China (Tianjin)
23108 \ (eos-tianjin-1.cmecloud.cn)
23109 21 / Jilin China (Changchun)
23110 \ (eos-jilin-1.cmecloud.cn)
23111 22 / Hubei China (Xiangyan)
23112 \ (eos-hubei-1.cmecloud.cn)
23113 23 / Jiangxi China (Nanchang)
23114 \ (eos-jiangxi-1.cmecloud.cn)
23115 24 / Gansu China (Lanzhou)
23116 \ (eos-gansu-1.cmecloud.cn)
23117 25 / Shanxi China (Taiyuan)
23118 \ (eos-shanxi-1.cmecloud.cn)
23119 26 / Liaoning China (Shenyang)
23120 \ (eos-liaoning-1.cmecloud.cn)
23121 27 / Hebei China (Shijiazhuang)
23122 \ (eos-hebei-1.cmecloud.cn)
23123 28 / Fujian China (Xiamen)
23124 \ (eos-fujian-1.cmecloud.cn)
23125 29 / Guangxi China (Nanning)
23126 \ (eos-guangxi-1.cmecloud.cn)
23127 30 / Anhui China (Huainan)
23128 \ (eos-anhui-1.cmecloud.cn)
23129 endpoint> 1
23130 Option location_constraint.
23131 Location constraint - must match endpoint.
23132 Used when creating buckets only.
23133 Choose a number from below, or type in your own value.
23134 Press Enter to leave empty.
23135 1 / East China (Suzhou)
23136 \ (wuxi1)
23137 2 / East China (Jinan)
23138 \ (jinan1)
23139 3 / East China (Hangzhou)
23140 \ (ningbo1)
23141 4 / East China (Shanghai-1)
23142 \ (shanghai1)
23143 5 / Central China (Zhengzhou)
23144 \ (zhengzhou1)
23145 6 / Central China (Changsha-1)
23146 \ (hunan1)
23147 7 / Central China (Changsha-2)
23148 \ (zhuzhou1)
23149 8 / South China (Guangzhou-2)
23150 \ (guangzhou1)
23151 9 / South China (Guangzhou-3)
23152 \ (dongguan1)
23153 10 / North China (Beijing-1)
23154 \ (beijing1)
23155 11 / North China (Beijing-2)
23156 \ (beijing2)
23157 12 / North China (Beijing-3)
23158 \ (beijing4)
23159 13 / North China (Huhehaote)
23160 \ (huhehaote1)
23161 14 / Southwest China (Chengdu)
23162 \ (chengdu1)
23163 15 / Southwest China (Chongqing)
23164 \ (chongqing1)
23165 16 / Southwest China (Guiyang)
23166 \ (guiyang1)
23167 17 / Nouthwest China (Xian)
23168 \ (xian1)
23169 18 / Yunnan China (Kunming)
23170 \ (yunnan)
23171 19 / Yunnan China (Kunming-2)
23172 \ (yunnan2)
23173 20 / Tianjin China (Tianjin)
23174 \ (tianjin1)
23175 21 / Jilin China (Changchun)
23176 \ (jilin1)
23177 22 / Hubei China (Xiangyan)
23178 \ (hubei1)
23179 23 / Jiangxi China (Nanchang)
23180 \ (jiangxi1)
23181 24 / Gansu China (Lanzhou)
23182 \ (gansu1)
23183 25 / Shanxi China (Taiyuan)
23184 \ (shanxi1)
23185 26 / Liaoning China (Shenyang)
23186 \ (liaoning1)
23187 27 / Hebei China (Shijiazhuang)
23188 \ (hebei1)
23189 28 / Fujian China (Xiamen)
23190 \ (fujian1)
23191 29 / Guangxi China (Nanning)
23192 \ (guangxi1)
23193 30 / Anhui China (Huainan)
23194 \ (anhui1)
23195 location_constraint> 1
23196 Option acl.
23197 Canned ACL used when creating buckets and storing or copying objects.
23198 This ACL is used for creating objects and if bucket_acl isn't set, for creating buckets too.
23199 For more info visit https://docs.aws.amazon.com/AmazonS3/latest/dev/acl-overview.html#canned-acl
23200 Note that this ACL is applied when server-side copying objects as S3
23201 doesn't copy the ACL from the source but rather writes a fresh one.
23202 Choose a number from below, or type in your own value.
23203 Press Enter to leave empty.
23204 / Owner gets FULL_CONTROL.
23205 1 | No one else has access rights (default).
23206 \ (private)
23207 / Owner gets FULL_CONTROL.
23208 2 | The AllUsers group gets READ access.
23209 \ (public-read)
23210 / Owner gets FULL_CONTROL.
23211 3 | The AllUsers group gets READ and WRITE access.
23212 | Granting this on a bucket is generally not recommended.
23213 \ (public-read-write)
23214 / Owner gets FULL_CONTROL.
23215 4 | The AuthenticatedUsers group gets READ access.
23216 \ (authenticated-read)
23217 / Object owner gets FULL_CONTROL.
23218 acl> private
23219 Option server_side_encryption.
23220 The server-side encryption algorithm used when storing this object in S3.
23221 Choose a number from below, or type in your own value.
23222 Press Enter to leave empty.
23223 1 / None
23224 \ ()
23225 2 / AES256
23226 \ (AES256)
23227 server_side_encryption>
23228 Option storage_class.
23229 The storage class to use when storing new objects in ChinaMobile.
23230 Choose a number from below, or type in your own value.
23231 Press Enter to leave empty.
23232 1 / Default
23233 \ ()
23234 2 / Standard storage class
23235 \ (STANDARD)
23236 3 / Archive storage mode
23237 \ (GLACIER)
23238 4 / Infrequent access storage mode
23239 \ (STANDARD_IA)
23240 storage_class>
23241 Edit advanced config?
23242 y) Yes
23243 n) No (default)
23244 y/n> n
23245 --------------------
23246 [ChinaMobile]
23247 type = s3
23248 provider = ChinaMobile
23249 access_key_id = accesskeyid
23250 secret_access_key = secretaccesskey
23251 endpoint = eos-wuxi-1.cmecloud.cn
23252 location_constraint = wuxi1
23253 acl = private
23254 --------------------
23255 y) Yes this is OK (default)
23256 e) Edit this remote
23257 d) Delete this remote
23258 y/e/d> y
23259 ArvanCloud
23261 ArvanCloud (https://www.arvancloud.com/en/products/cloud-storage) Ar‐
23262 vanCloud Object Storage goes beyond the limited traditional file stor‐
23263 age. It gives you access to backup and archived files and allows shar‐
23264 ing. Files like profile image in the app, images sent by users or
23265 scanned documents can be stored securely and easily in our Object Stor‐
23266 age service.
23267 ArvanCloud provides an S3 interface which can be configured for use
23269 with rclone like this.
23270 No remotes found, make a new one?
23272 n) New remote
23273 s) Set configuration password
23274 n/s> n
23275 name> ArvanCloud
23276 Type of storage to configure.
23277 Choose a number from below, or type in your own value
23278 [snip]
23279 XX / Amazon S3 (also Dreamhost, Ceph, ChinaMobile, ArvanCloud, Minio)
23280 \ "s3"
23281 [snip]
23282 Storage> s3
23283 Get AWS credentials from runtime (environment variables or EC2/ECS meta data if no env vars). Only applies if access_key_id and secret_access_key is blank.
23284 Choose a number from below, or type in your own value
23285 1 / Enter AWS credentials in the next step
23286 \ "false"
23287 2 / Get AWS credentials from the environment (env vars or IAM)
23288 \ "true"
23289 env_auth> 1
23290 AWS Access Key ID - leave blank for anonymous access or runtime credentials.
23291 access_key_id> YOURACCESSKEY
23292 AWS Secret Access Key (password) - leave blank for anonymous access or runtime credentials.
23293 secret_access_key> YOURSECRETACCESSKEY
23294 Region to connect to.
23295 Choose a number from below, or type in your own value
23296 / The default endpoint - a good choice if you are unsure.
23297 1 | US Region, Northern Virginia, or Pacific Northwest.
23298 | Leave location constraint empty.
23299 \ "us-east-1"
23300 [snip]
23301 region>
23302 Endpoint for S3 API.
23303 Leave blank if using ArvanCloud to use the default endpoint for the region.
23304 Specify if using an S3 clone such as Ceph.
23305 endpoint> s3.arvanstorage.com
23306 Location constraint - must be set to match the Region. Used when creating buckets only.
23307 Choose a number from below, or type in your own value
23308 1 / Empty for Iran-Tehran Region.
23309 \ ""
23310 [snip]
23311 location_constraint>
23312 Canned ACL used when creating buckets and/or storing objects in S3.
23313 For more info visit https://docs.aws.amazon.com/AmazonS3/latest/dev/acl-overview.html#canned-acl
23314 Choose a number from below, or type in your own value
23315 1 / Owner gets FULL_CONTROL. No one else has access rights (default).
23316 \ "private"
23317 [snip]
23318 acl>
23319 The server-side encryption algorithm used when storing this object in S3.
23320 Choose a number from below, or type in your own value
23321 1 / None
23322 \ ""
23323 2 / AES256
23324 \ "AES256"
23325 server_side_encryption>
23326 The storage class to use when storing objects in S3.
23327 Choose a number from below, or type in your own value
23328 1 / Default
23329 \ ""
23330 2 / Standard storage class
23331 \ "STANDARD"
23332 storage_class>
23333 Remote config
23334 --------------------
23335 [ArvanCloud]
23336 env_auth = false
23337 access_key_id = YOURACCESSKEY
23338 secret_access_key = YOURSECRETACCESSKEY
23339 region = ir-thr-at1
23340 endpoint = s3.arvanstorage.com
23341 location_constraint =
23342 acl =
23343 server_side_encryption =
23344 storage_class =
23345 --------------------
23346 y) Yes this is OK
23347 e) Edit this remote
23348 d) Delete this remote
23349 y/e/d> y
23350 This will leave the config file looking like this.
23352 [ArvanCloud]
23354 type = s3
23355 provider = ArvanCloud
23356 env_auth = false
23357 access_key_id = YOURACCESSKEY
23358 secret_access_key = YOURSECRETACCESSKEY
23359 region =
23360 endpoint = s3.arvanstorage.com
23361 location_constraint =
23362 acl =
23363 server_side_encryption =
23364 storage_class =
23365 Tencent COS
23367 Tencent Cloud Object Storage (COS) (https://intl.cloud.ten‐
23368 cent.com/product/cos) is a distributed storage service offered by Ten‐
23369 cent Cloud for unstructured data. It is secure, stable, massive, con‐
23370 venient, low-delay and low-cost.
23371 To configure access to Tencent COS, follow the steps below:
23373 1. Run rclone config and select n for a new remote.
23375 rclone config
23377 No remotes found, make a new one?
23378 n) New remote
23379 s) Set configuration password
23380 q) Quit config
23381 n/s/q> n
23382 2. Give the name of the configuration. For example, name it 'cos'.
23384 name> cos
23386 3. Select s3 storage.
23388 Choose a number from below, or type in your own value
23390 1 / 1Fichier
23391 \ "fichier"
23392 2 / Alias for an existing remote
23393 \ "alias"
23394 3 / Amazon Drive
23395 \ "amazon cloud drive"
23396 4 / Amazon S3 Compliant Storage Providers including AWS, Alibaba, Ceph, ChinaMobile, ArvanCloud, Digital Ocean, Dreamhost, Huawei OBS, IBM COS, Minio, and Tencent COS
23397 \ "s3"
23398 [snip]
23399 Storage> s3
23400 4. Select TencentCOS provider.
23402 Choose a number from below, or type in your own value
23404 1 / Amazon Web Services (AWS) S3
23405 \ "AWS"
23406 [snip]
23407 11 / Tencent Cloud Object Storage (COS)
23408 \ "TencentCOS"
23409 [snip]
23410 provider> TencentCOS
23411 5. Enter your SecretId and SecretKey of Tencent Cloud.
23413 Get AWS credentials from runtime (environment variables or EC2/ECS meta data if no env vars).
23415 Only applies if access_key_id and secret_access_key is blank.
23416 Enter a boolean value (true or false). Press Enter for the default ("false").
23417 Choose a number from below, or type in your own value
23418 1 / Enter AWS credentials in the next step
23419 \ "false"
23420 2 / Get AWS credentials from the environment (env vars or IAM)
23421 \ "true"
23422 env_auth> 1
23423 AWS Access Key ID.
23424 Leave blank for anonymous access or runtime credentials.
23425 Enter a string value. Press Enter for the default ("").
23426 access_key_id> AKIDxxxxxxxxxx
23427 AWS Secret Access Key (password)
23428 Leave blank for anonymous access or runtime credentials.
23429 Enter a string value. Press Enter for the default ("").
23430 secret_access_key> xxxxxxxxxxx
23431 6. Select endpoint for Tencent COS. This is the standard endpoint for
23433 different region.
23434 1 / Beijing Region.
23436 \ "cos.ap-beijing.myqcloud.com"
23437 2 / Nanjing Region.
23438 \ "cos.ap-nanjing.myqcloud.com"
23439 3 / Shanghai Region.
23440 \ "cos.ap-shanghai.myqcloud.com"
23441 4 / Guangzhou Region.
23442 \ "cos.ap-guangzhou.myqcloud.com"
23443 [snip]
23444 endpoint> 4
23445 7. Choose acl and storage class.
23447 Note that this ACL is applied when server-side copying objects as S3
23449 doesn't copy the ACL from the source but rather writes a fresh one.
23450 Enter a string value. Press Enter for the default ("").
23451 Choose a number from below, or type in your own value
23452 1 / Owner gets Full_CONTROL. No one else has access rights (default).
23453 \ "default"
23454 [snip]
23455 acl> 1
23456 The storage class to use when storing new objects in Tencent COS.
23457 Enter a string value. Press Enter for the default ("").
23458 Choose a number from below, or type in your own value
23459 1 / Default
23460 \ ""
23461 [snip]
23462 storage_class> 1
23463 Edit advanced config? (y/n)
23464 y) Yes
23465 n) No (default)
23466 y/n> n
23467 Remote config
23468 --------------------
23469 [cos]
23470 type = s3
23471 provider = TencentCOS
23472 env_auth = false
23473 access_key_id = xxx
23474 secret_access_key = xxx
23475 endpoint = cos.ap-guangzhou.myqcloud.com
23476 acl = default
23477 --------------------
23478 y) Yes this is OK (default)
23479 e) Edit this remote
23480 d) Delete this remote
23481 y/e/d> y
23482 Current remotes:
23483 Name Type
23485 ==== ====
23486 cos s3
23487 Netease NOS
23489 For Netease NOS configure as per the configurator rclone config setting
23490 the provider Netease. This will automatically set force_path_style =
23491 false which is necessary for it to run properly.
23492 Storj
23494 Storj is a decentralized cloud storage which can be used through its
23495 native protocol or an S3 compatible gateway.
23496 The S3 compatible gateway is configured using rclone config with a type
23498 of s3 and with a provider name of Storj. Here is an example run of the
23499 configurator.
23500 Type of storage to configure.
23502 Storage> s3
23503 Get AWS credentials from runtime (environment variables or EC2/ECS meta data if no env vars).
23504 Only applies if access_key_id and secret_access_key is blank.
23505 Choose a number from below, or type in your own boolean value (true or false).
23506 Press Enter for the default (false).
23507 1 / Enter AWS credentials in the next step.
23508 \ (false)
23509 2 / Get AWS credentials from the environment (env vars or IAM).
23510 \ (true)
23511 env_auth> 1
23512 Option access_key_id.
23513 AWS Access Key ID.
23514 Leave blank for anonymous access or runtime credentials.
23515 Enter a value. Press Enter to leave empty.
23516 access_key_id> XXXX (as shown when creating the access grant)
23517 Option secret_access_key.
23518 AWS Secret Access Key (password).
23519 Leave blank for anonymous access or runtime credentials.
23520 Enter a value. Press Enter to leave empty.
23521 secret_access_key> XXXX (as shown when creating the access grant)
23522 Option endpoint.
23523 Endpoint of the Shared Gateway.
23524 Choose a number from below, or type in your own value.
23525 Press Enter to leave empty.
23526 1 / EU1 Shared Gateway
23527 \ (gateway.eu1.storjshare.io)
23528 2 / US1 Shared Gateway
23529 \ (gateway.us1.storjshare.io)
23530 3 / Asia-Pacific Shared Gateway
23531 \ (gateway.ap1.storjshare.io)
23532 endpoint> 1 (as shown when creating the access grant)
23533 Edit advanced config?
23534 y) Yes
23535 n) No (default)
23536 y/n> n
23537 Note that s3 credentials are generated when you create an access grant
23539 (https://docs.storj.io/dcs/api-reference/s3-compatible-gateway#usage).
23540 Backend quirks
23542 • --chunk-size is forced to be 64 MiB or greater. This will use more
23543 memory than the default of 5 MiB.
23544 • Server side copy is disabled as it isn't currently supported in the
23546 gateway.
23547 • GetTier and SetTier are not supported.
23549 Backend bugs
23551 Due to issue #39 (https://github.com/storj/gateway-mt/issues/39) up‐
23552 loading multipart files via the S3 gateway causes them to lose their
23553 metadata. For rclone's purpose this means that the modification time
23554 is not stored, nor is any MD5SUM (if one is available from the source).
23555 This has the following consequences:
23557 • Using rclone rcat will fail as the medatada doesn't match after up‐
23559 load
23560 • Uploading files with rclone mount will fail for the same reason
23562 • This can worked around by using --vfs-cache-mode writes or --vfs-
23564 cache-mode full or setting --s3-upload-cutoff large
23565 • Files uploaded via a multipart upload won't have their modtimes
23567 • This will mean that rclone sync will likely keep trying to upload
23569 files bigger than --s3-upload-cutoff
23570 • This can be worked around with --checksum or --size-only or setting
23572 --s3-upload-cutoff large
23573 • The maximum value for --s3-upload-cutoff is 5GiB though
23575 One general purpose workaround is to set --s3-upload-cutoff 5G. This
23577 means that rclone will upload files smaller than 5GiB as single parts.
23578 Note that this can be set in the config file with upload_cutoff = 5G or
23579 configured in the advanced settings. If you regularly transfer files
23580 larger than 5G then using --checksum or --size-only in rclone sync is
23581 the recommended workaround.
23582 Comparison with the native protocol
23584 Use the the native protocol to take advantage of client-side encryption
23585 as well as to achieve the best possible download performance. Uploads
23586 will be erasure-coded locally, thus a 1gb upload will result in 2.68gb
23587 of data being uploaded to storage nodes across the network.
23588 Use this backend and the S3 compatible Hosted Gateway to increase up‐
23590 load performance and reduce the load on your systems and network. Up‐
23591 loads will be encrypted and erasure-coded server-side, thus a 1GB up‐
23592 load will result in only in 1GB of data being uploaded to storage nodes
23593 across the network.
23594 For more detailed comparison please check the documentation of the
23596 storj backend.
23597 Limitations
23599 rclone about is not supported by the S3 backend. Backends without this
23600 capability cannot determine free space for an rclone mount or use poli‐
23601 cy mfs (most free space) as a member of an rclone union remote.
23602 See List of backends that do not support rclone about
23604 (https://rclone.org/overview/#optional-features) and rclone about
23605 (https://rclone.org/commands/rclone_about/)
23606
23608 B2 is Backblaze's cloud storage system (https://www.backblaze.com/b2/).
23609 Paths are specified as remote:bucket (or remote: for the lsd command.)
23611 You may put subdirectories in too, e.g. remote:bucket/path/to/dir.
23612 Configuration
23614 Here is an example of making a b2 configuration. First run
23615 rclone config
23617 This will guide you through an interactive setup process. To authenti‐
23619 cate you will either need your Account ID (a short hex number) and Mas‐
23620 ter Application Key (a long hex number) OR an Application Key, which is
23621 the recommended method. See below for further details on generating
23622 and using an Application Key.
23623 No remotes found, make a new one?
23625 n) New remote
23626 q) Quit config
23627 n/q> n
23628 name> remote
23629 Type of storage to configure.
23630 Choose a number from below, or type in your own value
23631 [snip]
23632 XX / Backblaze B2
23633 \ "b2"
23634 [snip]
23635 Storage> b2
23636 Account ID or Application Key ID
23637 account> 123456789abc
23638 Application Key
23639 key> 0123456789abcdef0123456789abcdef0123456789
23640 Endpoint for the service - leave blank normally.
23641 endpoint>
23642 Remote config
23643 --------------------
23644 [remote]
23645 account = 123456789abc
23646 key = 0123456789abcdef0123456789abcdef0123456789
23647 endpoint =
23648 --------------------
23649 y) Yes this is OK
23650 e) Edit this remote
23651 d) Delete this remote
23652 y/e/d> y
23653 This remote is called remote and can now be used like this
23655 See all buckets
23657 rclone lsd remote:
23659 Create a new bucket
23661 rclone mkdir remote:bucket
23663 List the contents of a bucket
23665 rclone ls remote:bucket
23667 Sync /home/local/directory to the remote bucket, deleting any excess
23669 files in the bucket.
23670 rclone sync -i /home/local/directory remote:bucket
23672 Application Keys
23674 B2 supports multiple Application Keys for different access permission
23675 to B2 Buckets (https://www.backblaze.com/b2/docs/applica‐
23676 tion_keys.html).
23677 You can use these with rclone too; you will need to use rclone version
23679 1.43 or later.
23680 Follow Backblaze's docs to create an Application Key with the required
23682 permission and add the applicationKeyId as the account and the Applica‐
23683 tion Key itself as the key.
23684 Note that you must put the applicationKeyId as the account – you can't
23686 use the master Account ID. If you try then B2 will return 401 errors.
23687 --fast-list
23689 This remote supports --fast-list which allows you to use fewer transac‐
23690 tions in exchange for more memory. See the rclone docs
23691 (https://rclone.org/docs/#fast-list) for more details.
23692 Modified time
23694 The modified time is stored as metadata on the object as X-Bz-Info-
23695 src_last_modified_millis as milliseconds since 1970-01-01 in the Back‐
23696 blaze standard. Other tools should be able to use this as a modified
23697 time.
23698 Modified times are used in syncing and are fully supported. Note that
23700 if a modification time needs to be updated on an object then it will
23701 create a new version of the object.
23702 Restricted filename characters
23704 In addition to the default restricted characters set
23705 (https://rclone.org/overview/#restricted-characters) the following
23706 characters are also replaced:
23707 Character Value Replacement
23709 ────────────────────────────────
23710 \ 0x5C ＼
23711 Invalid UTF-8 bytes will also be replaced (https://rclone.org/over‐
23713 view/#invalid-utf8), as they can't be used in JSON strings.
23714 Note that in 2020-05 Backblaze started allowing characters in file
23716 names. Rclone hasn't changed its encoding as this could cause syncs to
23717 re-transfer files. If you want rclone not to replace then see the
23718 --b2-encoding flag below and remove the BackSlash from the string.
23719 This can be set in the config.
23720 SHA1 checksums
23722 The SHA1 checksums of the files are checked on upload and download and
23723 will be used in the syncing process.
23724 Large files (bigger than the limit in --b2-upload-cutoff) which are up‐
23726 loaded in chunks will store their SHA1 on the object as X-Bz-Info-
23727 large_file_sha1 as recommended by Backblaze.
23728 For a large file to be uploaded with an SHA1 checksum, the source needs
23730 to support SHA1 checksums. The local disk supports SHA1 checksums so
23731 large file transfers from local disk will have an SHA1. See the over‐
23732 view (https://rclone.org/overview/#features) for exactly which remotes
23733 support SHA1.
23734 Sources which don't support SHA1, in particular crypt will upload large
23736 files without SHA1 checksums. This may be fixed in the future (see
23737 #1767 (https://github.com/rclone/rclone/issues/1767)).
23738 Files sizes below --b2-upload-cutoff will always have an SHA1 regard‐
23740 less of the source.
23741 Transfers
23743 Backblaze recommends that you do lots of transfers simultaneously for
23744 maximum speed. In tests from my SSD equipped laptop the optimum set‐
23745 ting is about --transfers 32 though higher numbers may be used for a
23746 slight speed improvement. The optimum number for you may vary depend‐
23747 ing on your hardware, how big the files are, how much you want to load
23748 your computer, etc. The default of --transfers 4 is definitely too low
23749 for Backblaze B2 though.
23750 Note that uploading big files (bigger than 200 MiB by default) will use
23752 a 96 MiB RAM buffer by default. There can be at most --transfers of
23753 these in use at any moment, so this sets the upper limit on the memory
23754 used.
23755 Versions
23757 When rclone uploads a new version of a file it creates a new version of
23758 it (https://www.backblaze.com/b2/docs/file_versions.html). Likewise
23759 when you delete a file, the old version will be marked hidden and still
23760 be available. Conversely, you may opt in to a "hard delete" of files
23761 with the --b2-hard-delete flag which would permanently remove the file
23762 instead of hiding it.
23763 Old versions of files, where available, are visible using the --b2-ver‐
23765 sions flag.
23766 It is also possible to view a bucket as it was at a certain point in
23768 time, using the --b2-version-at flag. This will show the file versions
23769 as they were at that time, showing files that have been deleted after‐
23770 wards, and hiding files that were created since.
23771 If you wish to remove all the old versions then you can use the rclone
23773 cleanup remote:bucket command which will delete all the old versions of
23774 files, leaving the current ones intact. You can also supply a path and
23775 only old versions under that path will be deleted, e.g. rclone cleanup
23776 remote:bucket/path/to/stuff.
23777 Note that cleanup will remove partially uploaded files from the bucket
23779 if they are more than a day old.
23780 When you purge a bucket, the current and the old versions will be
23782 deleted then the bucket will be deleted.
23783 However delete will cause the current versions of the files to become
23785 hidden old versions.
23786 Here is a session showing the listing and retrieval of an old version
23788 followed by a cleanup of the old versions.
23789 Show current version and all the versions with --b2-versions flag.
23791 $ rclone -q ls b2:cleanup-test
23793 9 one.txt
23794 $ rclone -q --b2-versions ls b2:cleanup-test
23796 9 one.txt
23797 8 one-v2016-07-04-141032-000.txt
23798 16 one-v2016-07-04-141003-000.txt
23799 15 one-v2016-07-02-155621-000.txt
23800 Retrieve an old version
23802 $ rclone -q --b2-versions copy b2:cleanup-test/one-v2016-07-04-141003-000.txt /tmp
23804 $ ls -l /tmp/one-v2016-07-04-141003-000.txt
23806 -rw-rw-r-- 1 ncw ncw 16 Jul 2 17:46 /tmp/one-v2016-07-04-141003-000.txt
23807 Clean up all the old versions and show that they've gone.
23809 $ rclone -q cleanup b2:cleanup-test
23811 $ rclone -q ls b2:cleanup-test
23813 9 one.txt
23814 $ rclone -q --b2-versions ls b2:cleanup-test
23816 9 one.txt
23817 Data usage
23819 It is useful to know how many requests are sent to the server in dif‐
23820 ferent scenarios.
23821 All copy commands send the following 4 requests:
23823 /b2api/v1/b2_authorize_account
23825 /b2api/v1/b2_create_bucket
23826 /b2api/v1/b2_list_buckets
23827 /b2api/v1/b2_list_file_names
23828 The b2_list_file_names request will be sent once for every 1k files in
23830 the remote path, providing the checksum and modification time of the
23831 listed files. As of version 1.33 issue #818
23832 (https://github.com/rclone/rclone/issues/818) causes extra requests to
23833 be sent when using B2 with Crypt. When a copy operation does not re‐
23834 quire any files to be uploaded, no more requests will be sent.
23835 Uploading files that do not require chunking, will send 2 requests per
23837 file upload:
23838 /b2api/v1/b2_get_upload_url
23840 /b2api/v1/b2_upload_file/
23841 Uploading files requiring chunking, will send 2 requests (one each to
23843 start and finish the upload) and another 2 requests for each chunk:
23844 /b2api/v1/b2_start_large_file
23846 /b2api/v1/b2_get_upload_part_url
23847 /b2api/v1/b2_upload_part/
23848 /b2api/v1/b2_finish_large_file
23849 Versions
23851 Versions can be viewed with the --b2-versions flag. When it is set
23852 rclone will show and act on older versions of files. For example
23853 Listing without --b2-versions
23855 $ rclone -q ls b2:cleanup-test
23857 9 one.txt
23858 And with
23860 $ rclone -q --b2-versions ls b2:cleanup-test
23862 9 one.txt
23863 8 one-v2016-07-04-141032-000.txt
23864 16 one-v2016-07-04-141003-000.txt
23865 15 one-v2016-07-02-155621-000.txt
23866 Showing that the current version is unchanged but older versions can be
23868 seen. These have the UTC date that they were uploaded to the server to
23869 the nearest millisecond appended to them.
23870 Note that when using --b2-versions no file write operations are permit‐
23872 ted, so you can't upload files or delete them.
23873 B2 and rclone link
23875 Rclone supports generating file share links for private B2 buckets.
23876 They can either be for a file for example:
23877 ./rclone link B2:bucket/path/to/file.txt
23879 https://f002.backblazeb2.com/file/bucket/path/to/file.txt?Authorization=xxxxxxxx
23880 or if run on a directory you will get:
23882 ./rclone link B2:bucket/path
23884 https://f002.backblazeb2.com/file/bucket/path?Authorization=xxxxxxxx
23885 you can then use the authorization token (the part of the url from the
23887 ?Authorization= on) on any file path under that directory. For exam‐
23888 ple:
23889 https://f002.backblazeb2.com/file/bucket/path/to/file1?Authorization=xxxxxxxx
23891 https://f002.backblazeb2.com/file/bucket/path/file2?Authorization=xxxxxxxx
23892 https://f002.backblazeb2.com/file/bucket/path/folder/file3?Authorization=xxxxxxxx
23893 Standard options
23895 Here are the Standard options specific to b2 (Backblaze B2).
23896 --b2-account
23898 Account ID or Application Key ID.
23899 Properties:
23901 • Config: account
23903 • Env Var: RCLONE_B2_ACCOUNT
23905 • Type: string
23907 • Required: true
23909 --b2-key
23911 Application Key.
23912 Properties:
23914 • Config: key
23916 • Env Var: RCLONE_B2_KEY
23918 • Type: string
23920 • Required: true
23922 --b2-hard-delete
23924 Permanently delete files on remote removal, otherwise hide files.
23925 Properties:
23927 • Config: hard_delete
23929 • Env Var: RCLONE_B2_HARD_DELETE
23931 • Type: bool
23933 • Default: false
23935 Advanced options
23937 Here are the Advanced options specific to b2 (Backblaze B2).
23938 --b2-endpoint
23940 Endpoint for the service.
23941 Leave blank normally.
23943 Properties:
23945 • Config: endpoint
23947 • Env Var: RCLONE_B2_ENDPOINT
23949 • Type: string
23951 • Required: false
23953 --b2-test-mode
23955 A flag string for X-Bz-Test-Mode header for debugging.
23956 This is for debugging purposes only. Setting it to one of the strings
23958 below will cause b2 to return specific errors:
23959 • "fail_some_uploads"
23961 • "expire_some_account_authorization_tokens"
23963 • "force_cap_exceeded"
23965 These will be set in the "X-Bz-Test-Mode" header which is documented in
23967 the b2 integrations checklist (https://www.backblaze.com/b2/docs/inte‐
23968 gration_checklist.html).
23969 Properties:
23971 • Config: test_mode
23973 • Env Var: RCLONE_B2_TEST_MODE
23975 • Type: string
23977 • Required: false
23979 --b2-versions
23981 Include old versions in directory listings.
23982 Note that when using this no file write operations are permitted, so
23984 you can't upload files or delete them.
23985 Properties:
23987 • Config: versions
23989 • Env Var: RCLONE_B2_VERSIONS
23991 • Type: bool
23993 • Default: false
23995 --b2-version-at
23997 Show file versions as they were at the specified time.
23998 Note that when using this no file write operations are permitted, so
24000 you can't upload files or delete them.
24001 Properties:
24003 • Config: version_at
24005 • Env Var: RCLONE_B2_VERSION_AT
24007 • Type: Time
24009 • Default: off
24011 --b2-upload-cutoff
24013 Cutoff for switching to chunked upload.
24014 Files above this size will be uploaded in chunks of "--b2-chunk-size".
24016 This value should be set no larger than 4.657 GiB (== 5 GB).
24018 Properties:
24020 • Config: upload_cutoff
24022 • Env Var: RCLONE_B2_UPLOAD_CUTOFF
24024 • Type: SizeSuffix
24026 • Default: 200Mi
24028 --b2-copy-cutoff
24030 Cutoff for switching to multipart copy.
24031 Any files larger than this that need to be server-side copied will be
24033 copied in chunks of this size.
24034 The minimum is 0 and the maximum is 4.6 GiB.
24036 Properties:
24038 • Config: copy_cutoff
24040 • Env Var: RCLONE_B2_COPY_CUTOFF
24042 • Type: SizeSuffix
24044 • Default: 4Gi
24046 --b2-chunk-size
24048 Upload chunk size.
24049 When uploading large files, chunk the file into this size.
24051 Must fit in memory. These chunks are buffered in memory and there
24053 might a maximum of "--transfers" chunks in progress at once.
24054 5,000,000 Bytes is the minimum size.
24056 Properties:
24058 • Config: chunk_size
24060 • Env Var: RCLONE_B2_CHUNK_SIZE
24062 • Type: SizeSuffix
24064 • Default: 96Mi
24066 --b2-disable-checksum
24068 Disable checksums for large (> upload cutoff) files.
24069 Normally rclone will calculate the SHA1 checksum of the input before
24071 uploading it so it can add it to metadata on the object. This is great
24072 for data integrity checking but can cause long delays for large files
24073 to start uploading.
24074 Properties:
24076 • Config: disable_checksum
24078 • Env Var: RCLONE_B2_DISABLE_CHECKSUM
24080 • Type: bool
24082 • Default: false
24084 --b2-download-url
24086 Custom endpoint for downloads.
24087 This is usually set to a Cloudflare CDN URL as Backblaze offers free
24089 egress for data downloaded through the Cloudflare network. Rclone
24090 works with private buckets by sending an "Authorization" header. If
24091 the custom endpoint rewrites the requests for authentication, e.g., in
24092 Cloudflare Workers, this header needs to be handled properly. Leave
24093 blank if you want to use the endpoint provided by Backblaze.
24094 The URL provided here SHOULD have the protocol and SHOULD NOT have a
24096 trailing slash or specify the /file/bucket subpath as rclone will re‐
24097 quest files with "{download_url}/file/{bucket_name}/{path}".
24098 Example: > https://mysubdomain.mydomain.tld (No trailing "/", "file" or
24100 "bucket")
24101 Properties:
24103 • Config: download_url
24105 • Env Var: RCLONE_B2_DOWNLOAD_URL
24107 • Type: string
24109 • Required: false
24111 --b2-download-auth-duration
24113 Time before the authorization token will expire in s or suffix
24114 ms|s|m|h|d.
24115 The duration before the download authorization token will expire. The
24117 minimum value is 1 second. The maximum value is one week.
24118 Properties:
24120 • Config: download_auth_duration
24122 • Env Var: RCLONE_B2_DOWNLOAD_AUTH_DURATION
24124 • Type: Duration
24126 • Default: 1w
24128 --b2-memory-pool-flush-time
24130 How often internal memory buffer pools will be flushed. Uploads which
24131 requires additional buffers (f.e multipart) will use memory pool for
24132 allocations. This option controls how often unused buffers will be re‐
24133 moved from the pool.
24134 Properties:
24136 • Config: memory_pool_flush_time
24138 • Env Var: RCLONE_B2_MEMORY_POOL_FLUSH_TIME
24140 • Type: Duration
24142 • Default: 1m0s
24144 --b2-memory-pool-use-mmap
24146 Whether to use mmap buffers in internal memory pool.
24147 Properties:
24149 • Config: memory_pool_use_mmap
24151 • Env Var: RCLONE_B2_MEMORY_POOL_USE_MMAP
24153 • Type: bool
24155 • Default: false
24157 --b2-encoding
24159 The encoding for the backend.
24160 See the encoding section in the overview (https://rclone.org/over‐
24162 view/#encoding) for more info.
24163 Properties:
24165 • Config: encoding
24167 • Env Var: RCLONE_B2_ENCODING
24169 • Type: MultiEncoder
24171 • Default: Slash,BackSlash,Del,Ctl,InvalidUtf8,Dot
24173 Limitations
24175 rclone about is not supported by the B2 backend. Backends without this
24176 capability cannot determine free space for an rclone mount or use poli‐
24177 cy mfs (most free space) as a member of an rclone union remote.
24178 See List of backends that do not support rclone about
24180 (https://rclone.org/overview/#optional-features) and rclone about
24181 (https://rclone.org/commands/rclone_about/)
24182
24184 Paths are specified as remote:path
24185 Paths may be as deep as required, e.g. remote:directory/subdirectory.
24187 The initial setup for Box involves getting a token from Box which you
24189 can do either in your browser, or with a config.json downloaded from
24190 Box to use JWT authentication. rclone config walks you through it.
24191 Configuration
24193 Here is an example of how to make a remote called remote. First run:
24194 rclone config
24196 This will guide you through an interactive setup process:
24198 No remotes found, make a new one?
24200 n) New remote
24201 s) Set configuration password
24202 q) Quit config
24203 n/s/q> n
24204 name> remote
24205 Type of storage to configure.
24206 Choose a number from below, or type in your own value
24207 [snip]
24208 XX / Box
24209 \ "box"
24210 [snip]
24211 Storage> box
24212 Box App Client Id - leave blank normally.
24213 client_id>
24214 Box App Client Secret - leave blank normally.
24215 client_secret>
24216 Box App config.json location
24217 Leave blank normally.
24218 Enter a string value. Press Enter for the default ("").
24219 box_config_file>
24220 Box App Primary Access Token
24221 Leave blank normally.
24222 Enter a string value. Press Enter for the default ("").
24223 access_token>
24224 Enter a string value. Press Enter for the default ("user").
24226 Choose a number from below, or type in your own value
24227 1 / Rclone should act on behalf of a user
24228 \ "user"
24229 2 / Rclone should act on behalf of a service account
24230 \ "enterprise"
24231 box_sub_type>
24232 Remote config
24233 Use auto config?
24234 * Say Y if not sure
24235 * Say N if you are working on a remote or headless machine
24236 y) Yes
24237 n) No
24238 y/n> y
24239 If your browser doesn't open automatically go to the following link: http://127.0.0.1:53682/auth
24240 Log in and authorize rclone for access
24241 Waiting for code...
24242 Got code
24243 --------------------
24244 [remote]
24245 client_id =
24246 client_secret =
24247 token = {"access_token":"XXX","token_type":"bearer","refresh_token":"XXX","expiry":"XXX"}
24248 --------------------
24249 y) Yes this is OK
24250 e) Edit this remote
24251 d) Delete this remote
24252 y/e/d> y
24253 See the remote setup docs (https://rclone.org/remote_setup/) for how to
24255 set it up on a machine with no Internet browser available.
24256 Note that rclone runs a webserver on your local machine to collect the
24258 token as returned from Box. This only runs from the moment it opens
24259 your browser to the moment you get back the verification code. This is
24260 on http://127.0.0.1:53682/ and this it may require you to unblock it
24261 temporarily if you are running a host firewall.
24262 Once configured you can then use rclone like this,
24264 List directories in top level of your Box
24266 rclone lsd remote:
24268 List all the files in your Box
24270 rclone ls remote:
24272 To copy a local directory to an Box directory called backup
24274 rclone copy /home/source remote:backup
24276 Using rclone with an Enterprise account with SSO
24278 If you have an "Enterprise" account type with Box with single sign on
24279 (SSO), you need to create a password to use Box with rclone. This can
24280 be done at your Enterprise Box account by going to Settings, "Account"
24281 Tab, and then set the password in the "Authentication" field.
24282 Once you have done this, you can setup your Enterprise Box account us‐
24284 ing the same procedure detailed above in the, using the password you
24285 have just set.
24286 Invalid refresh token
24288 According to the box docs (https://develop‐
24289 er.box.com/v2.0/docs/oauth-20#section-6-using-the-access-and-refresh-
24290 tokens):
24291 Each refresh_token is valid for one use in 60 days.
24293 This means that if you
24295 • Don't use the box remote for 60 days
24297 • Copy the config file with a box refresh token in and use it in two
24299 places
24300 • Get an error on a token refresh
24302 then rclone will return an error which includes the text Invalid re‐
24304 fresh token.
24305 To fix this you will need to use oauth2 again to update the refresh to‐
24307 ken. You can use the methods in the remote setup docs
24308 (https://rclone.org/remote_setup/), bearing in mind that if you use the
24309 copy the config file method, you should not use that remote on the com‐
24310 puter you did the authentication on.
24311 Here is how to do it.
24313 $ rclone config
24315 Current remotes:
24316 Name Type
24318 ==== ====
24319 remote box
24320 e) Edit existing remote
24322 n) New remote
24323 d) Delete remote
24324 r) Rename remote
24325 c) Copy remote
24326 s) Set configuration password
24327 q) Quit config
24328 e/n/d/r/c/s/q> e
24329 Choose a number from below, or type in an existing value
24330 1 > remote
24331 remote> remote
24332 --------------------
24333 [remote]
24334 type = box
24335 token = {"access_token":"XXX","token_type":"bearer","refresh_token":"XXX","expiry":"2017-07-08T23:40:08.059167677+01:00"}
24336 --------------------
24337 Edit remote
24338 Value "client_id" = ""
24339 Edit? (y/n)>
24340 y) Yes
24341 n) No
24342 y/n> n
24343 Value "client_secret" = ""
24344 Edit? (y/n)>
24345 y) Yes
24346 n) No
24347 y/n> n
24348 Remote config
24349 Already have a token - refresh?
24350 y) Yes
24351 n) No
24352 y/n> y
24353 Use auto config?
24354 * Say Y if not sure
24355 * Say N if you are working on a remote or headless machine
24356 y) Yes
24357 n) No
24358 y/n> y
24359 If your browser doesn't open automatically go to the following link: http://127.0.0.1:53682/auth
24360 Log in and authorize rclone for access
24361 Waiting for code...
24362 Got code
24363 --------------------
24364 [remote]
24365 type = box
24366 token = {"access_token":"YYY","token_type":"bearer","refresh_token":"YYY","expiry":"2017-07-23T12:22:29.259137901+01:00"}
24367 --------------------
24368 y) Yes this is OK
24369 e) Edit this remote
24370 d) Delete this remote
24371 y/e/d> y
24372 Modified time and hashes
24374 Box allows modification times to be set on objects accurate to 1 sec‐
24375 ond. These will be used to detect whether objects need syncing or not.
24376 Box supports SHA1 type hashes, so you can use the --checksum flag.
24378 Restricted filename characters
24380 In addition to the default restricted characters set
24381 (https://rclone.org/overview/#restricted-characters) the following
24382 characters are also replaced:
24383 Character Value Replacement
24385 ────────────────────────────────
24386 \ 0x5C ＼
24387 File names can also not end with the following characters. These only
24389 get replaced if they are the last character in the name:
24390 Character Value Replacement
24392 ────────────────────────────────
24393 SP 0x20 ␠
24394 Invalid UTF-8 bytes will also be replaced (https://rclone.org/over‐
24396 view/#invalid-utf8), as they can't be used in JSON strings.
24397 Transfers
24399 For files above 50 MiB rclone will use a chunked transfer. Rclone will
24400 upload up to --transfers chunks at the same time (shared among all the
24401 multipart uploads). Chunks are buffered in memory and are normally 8
24402 MiB so increasing --transfers will increase memory use.
24403 Deleting files
24405 Depending on the enterprise settings for your user, the item will ei‐
24406 ther be actually deleted from Box or moved to the trash.
24407 Emptying the trash is supported via the rclone however cleanup command
24409 however this deletes every trashed file and folder individually so it
24410 may take a very long time. Emptying the trash via the WebUI does not
24411 have this limitation so it is advised to empty the trash via the WebUI.
24412 Root folder ID
24414 You can set the root_folder_id for rclone. This is the directory
24415 (identified by its Folder ID) that rclone considers to be the root of
24416 your Box drive.
24417 Normally you will leave this blank and rclone will determine the cor‐
24419 rect root to use itself.
24420 However you can set this to restrict rclone to a specific folder hier‐
24422 archy.
24423 In order to do this you will have to find the Folder ID of the directo‐
24425 ry you wish rclone to display. This will be the last segment of the
24426 URL when you open the relevant folder in the Box web interface.
24427 So if the folder you want rclone to use has a URL which looks like
24429 https://app.box.com/folder/11xxxxxxxxx8 in the browser, then you use
24430 11xxxxxxxxx8 as the root_folder_id in the config.
24431 Standard options
24433 Here are the Standard options specific to box (Box).
24434 --box-client-id
24436 OAuth Client Id.
24437 Leave blank normally.
24439 Properties:
24441 • Config: client_id
24443 • Env Var: RCLONE_BOX_CLIENT_ID
24445 • Type: string
24447 • Required: false
24449 --box-client-secret
24451 OAuth Client Secret.
24452 Leave blank normally.
24454 Properties:
24456 • Config: client_secret
24458 • Env Var: RCLONE_BOX_CLIENT_SECRET
24460 • Type: string
24462 • Required: false
24464 --box-box-config-file
24466 Box App config.json location
24467 Leave blank normally.
24469 Leading ~ will be expanded in the file name as will environment vari‐
24471 ables such as ${RCLONE_CONFIG_DIR}.
24472 Properties:
24474 • Config: box_config_file
24476 • Env Var: RCLONE_BOX_BOX_CONFIG_FILE
24478 • Type: string
24480 • Required: false
24482 --box-access-token
24484 Box App Primary Access Token
24485 Leave blank normally.
24487 Properties:
24489 • Config: access_token
24491 • Env Var: RCLONE_BOX_ACCESS_TOKEN
24493 • Type: string
24495 • Required: false
24497 --box-box-sub-type
24499 Properties:
24500 • Config: box_sub_type
24502 • Env Var: RCLONE_BOX_BOX_SUB_TYPE
24504 • Type: string
24506 • Default: "user"
24508 • Examples:
24510 • "user"
24512 • Rclone should act on behalf of a user.
24514 • "enterprise"
24516 • Rclone should act on behalf of a service account.
24518 Advanced options
24520 Here are the Advanced options specific to box (Box).
24521 --box-token
24523 OAuth Access Token as a JSON blob.
24524 Properties:
24526 • Config: token
24528 • Env Var: RCLONE_BOX_TOKEN
24530 • Type: string
24532 • Required: false
24534 --box-auth-url
24536 Auth server URL.
24537 Leave blank to use the provider defaults.
24539 Properties:
24541 • Config: auth_url
24543 • Env Var: RCLONE_BOX_AUTH_URL
24545 • Type: string
24547 • Required: false
24549 --box-token-url
24551 Token server url.
24552 Leave blank to use the provider defaults.
24554 Properties:
24556 • Config: token_url
24558 • Env Var: RCLONE_BOX_TOKEN_URL
24560 • Type: string
24562 • Required: false
24564 --box-root-folder-id
24566 Fill in for rclone to use a non root folder as its starting point.
24567 Properties:
24569 • Config: root_folder_id
24571 • Env Var: RCLONE_BOX_ROOT_FOLDER_ID
24573 • Type: string
24575 • Default: "0"
24577 --box-upload-cutoff
24579 Cutoff for switching to multipart upload (>= 50 MiB).
24580 Properties:
24582 • Config: upload_cutoff
24584 • Env Var: RCLONE_BOX_UPLOAD_CUTOFF
24586 • Type: SizeSuffix
24588 • Default: 50Mi
24590 --box-commit-retries
24592 Max number of times to try committing a multipart file.
24593 Properties:
24595 • Config: commit_retries
24597 • Env Var: RCLONE_BOX_COMMIT_RETRIES
24599 • Type: int
24601 • Default: 100
24603 --box-list-chunk
24605 Size of listing chunk 1-1000.
24606 Properties:
24608 • Config: list_chunk
24610 • Env Var: RCLONE_BOX_LIST_CHUNK
24612 • Type: int
24614 • Default: 1000
24616 --box-owned-by
24618 Only show items owned by the login (email address) passed in.
24619 Properties:
24621 • Config: owned_by
24623 • Env Var: RCLONE_BOX_OWNED_BY
24625 • Type: string
24627 • Required: false
24629 --box-encoding
24631 The encoding for the backend.
24632 See the encoding section in the overview (https://rclone.org/over‐
24634 view/#encoding) for more info.
24635 Properties:
24637 • Config: encoding
24639 • Env Var: RCLONE_BOX_ENCODING
24641 • Type: MultiEncoder
24643 • Default: Slash,BackSlash,Del,Ctl,RightSpace,InvalidUtf8,Dot
24645 Limitations
24647 Note that Box is case insensitive so you can't have a file called "Hel‐
24648 lo.doc" and one called "hello.doc".
24649 Box file names can't have the \ character in. rclone maps this to and
24651 from an identical looking unicode equivalent ＼ (U+FF3C Fullwidth Re‐
24652 verse Solidus).
24653 Box only supports filenames up to 255 characters in length.
24655 rclone about is not supported by the Box backend. Backends without
24657 this capability cannot determine free space for an rclone mount or use
24658 policy mfs (most free space) as a member of an rclone union remote.
24659 See List of backends that do not support rclone about
24661 (https://rclone.org/overview/#optional-features) and rclone about
24662 (https://rclone.org/commands/rclone_about/)
24663
24665 The cache remote wraps another existing remote and stores file struc‐
24666 ture and its data for long running tasks like rclone mount.
24667 Status
24669 The cache backend code is working but it currently doesn't have a main‐
24670 tainer so there are outstanding bugs
24671 (https://github.com/rclone/rclone/issues?q=is%3Aopen+is%3Aissue+la‐
24672 bel%3Abug+label%3A%22Remote%3A+Cache%22) which aren't getting fixed.
24673 The cache backend is due to be phased out in favour of the VFS caching
24675 layer eventually which is more tightly integrated into rclone.
24676 Until this happens we recommend only using the cache backend if you
24678 find you can't work without it. There are many docs online describing
24679 the use of the cache backend to minimize API hits and by-and-large
24680 these are out of date and the cache backend isn't needed in those sce‐
24681 narios any more.
24682 Configuration
24684 To get started you just need to have an existing remote which can be
24685 configured with cache.
24686 Here is an example of how to make a remote called test-cache. First
24688 run:
24689 rclone config
24691 This will guide you through an interactive setup process:
24693 No remotes found, make a new one?
24695 n) New remote
24696 r) Rename remote
24697 c) Copy remote
24698 s) Set configuration password
24699 q) Quit config
24700 n/r/c/s/q> n
24701 name> test-cache
24702 Type of storage to configure.
24703 Choose a number from below, or type in your own value
24704 [snip]
24705 XX / Cache a remote
24706 \ "cache"
24707 [snip]
24708 Storage> cache
24709 Remote to cache.
24710 Normally should contain a ':' and a path, e.g. "myremote:path/to/dir",
24711 "myremote:bucket" or maybe "myremote:" (not recommended).
24712 remote> local:/test
24713 Optional: The URL of the Plex server
24714 plex_url> http://127.0.0.1:32400
24715 Optional: The username of the Plex user
24716 plex_username> dummyusername
24717 Optional: The password of the Plex user
24718 y) Yes type in my own password
24719 g) Generate random password
24720 n) No leave this optional password blank
24721 y/g/n> y
24722 Enter the password:
24723 password:
24724 Confirm the password:
24725 password:
24726 The size of a chunk. Lower value good for slow connections but can affect seamless reading.
24727 Default: 5M
24728 Choose a number from below, or type in your own value
24729 1 / 1 MiB
24730 \ "1M"
24731 2 / 5 MiB
24732 \ "5M"
24733 3 / 10 MiB
24734 \ "10M"
24735 chunk_size> 2
24736 How much time should object info (file size, file hashes, etc.) be stored in cache. Use a very high value if you don't plan on changing the source FS from outside the cache.
24737 Accepted units are: "s", "m", "h".
24738 Default: 5m
24739 Choose a number from below, or type in your own value
24740 1 / 1 hour
24741 \ "1h"
24742 2 / 24 hours
24743 \ "24h"
24744 3 / 24 hours
24745 \ "48h"
24746 info_age> 2
24747 The maximum size of stored chunks. When the storage grows beyond this size, the oldest chunks will be deleted.
24748 Default: 10G
24749 Choose a number from below, or type in your own value
24750 1 / 500 MiB
24751 \ "500M"
24752 2 / 1 GiB
24753 \ "1G"
24754 3 / 10 GiB
24755 \ "10G"
24756 chunk_total_size> 3
24757 Remote config
24758 --------------------
24759 [test-cache]
24760 remote = local:/test
24761 plex_url = http://127.0.0.1:32400
24762 plex_username = dummyusername
24763 plex_password = *** ENCRYPTED ***
24764 chunk_size = 5M
24765 info_age = 48h
24766 chunk_total_size = 10G
24767 You can then use it like this,
24769 List directories in top level of your drive
24771 rclone lsd test-cache:
24773 List all the files in your drive
24775 rclone ls test-cache:
24777 To start a cached mount
24779 rclone mount --allow-other test-cache: /var/tmp/test-cache
24781 Write Features
24783 Offline uploading
24784 In an effort to make writing through cache more reliable, the backend
24785 now supports this feature which can be activated by specifying a cache-
24786 tmp-upload-path.
24787 A files goes through these states when using this feature:
24789 1. An upload is started (usually by copying a file on the cache remote)
24791 2. When the copy to the temporary location is complete the file is part
24793 of the cached remote and looks and behaves like any other file
24794 (reading included)
24795 3. After cache-tmp-wait-time passes and the file is next in line,
24797 rclone move is used to move the file to the cloud provider
24798 4. Reading the file still works during the upload but most modifica‐
24800 tions on it will be prohibited
24801 5. Once the move is complete the file is unlocked for modifications as
24803 it becomes as any other regular file
24804 6. If the file is being read through cache when it's actually deleted
24806 from the temporary path then cache will simply swap the source to
24807 the cloud provider without interrupting the reading (small blip can
24808 happen though)
24809 Files are uploaded in sequence and only one file is uploaded at a time.
24811 Uploads will be stored in a queue and be processed based on the order
24812 they were added. The queue and the temporary storage is persistent
24813 across restarts but can be cleared on startup with the --cache-db-purge
24814 flag.
24815 Write Support
24817 Writes are supported through cache. One caveat is that a mounted cache
24818 remote does not add any retry or fallback mechanism to the upload oper‐
24819 ation. This will depend on the implementation of the wrapped remote.
24820 Consider using Offline uploading for reliable writes.
24821 One special case is covered with cache-writes which will cache the file
24823 data at the same time as the upload when it is enabled making it avail‐
24824 able from the cache store immediately once the upload is finished.
24825 Read Features
24827 Multiple connections
24828 To counter the high latency between a local PC where rclone is running
24829 and cloud providers, the cache remote can split multiple requests to
24830 the cloud provider for smaller file chunks and combines them together
24831 locally where they can be available almost immediately before the read‐
24832 er usually needs them.
24833 This is similar to buffering when media files are played online.
24835 Rclone will stay around the current marker but always try its best to
24836 stay ahead and prepare the data before.
24837 Plex Integration
24839 There is a direct integration with Plex which allows cache to detect
24840 during reading if the file is in playback or not. This helps cache to
24841 adapt how it queries the cloud provider depending on what is needed
24842 for.
24843 Scans will have a minimum amount of workers (1) while in a confirmed
24845 playback cache will deploy the configured number of workers.
24846 This integration opens the doorway to additional performance improve‐
24848 ments which will be explored in the near future.
24849 Note: If Plex options are not configured, cache will function with its
24851 configured options without adapting any of its settings.
24852 How to enable? Run rclone config and add all the Plex options (end‐
24854 point, username and password) in your remote and it will be automati‐
24855 cally enabled.
24856 Affected settings: - cache-workers: Configured value during confirmed
24858 playback or 1 all the other times
24859 Certificate Validation
24861 When the Plex server is configured to only accept secure connections,
24862 it is possible to use .plex.direct URLs to ensure certificate valida‐
24863 tion succeeds. These URLs are used by Plex internally to connect to
24864 the Plex server securely.
24865 The format for these URLs is the following:
24867 https://ip-with-dots-replaced.server-hash.plex.direct:32400/
24869 The ip-with-dots-replaced part can be any IPv4 address, where the dots
24871 have been replaced with dashes, e.g. 127.0.0.1 becomes 127-0-0-1.
24872 To get the server-hash part, the easiest way is to visit
24874 https://plex.tv/api/resources?includeHttps=1&X-Plex-Token=your-plex-to‐
24876 ken
24877 This page will list all the available Plex servers for your account
24879 with at least one .plex.direct link for each. Copy one URL and replace
24880 the IP address with the desired address. This can be used as the
24881 plex_url value.
24882 Known issues
24884 Mount and --dir-cache-time
24885 --dir-cache-time controls the first layer of directory caching which
24886 works at the mount layer. Being an independent caching mechanism from
24887 the cache backend, it will manage its own entries based on the config‐
24888 ured time.
24889 To avoid getting in a scenario where dir cache has obsolete data and
24891 cache would have the correct one, try to set --dir-cache-time to a low‐
24892 er time than --cache-info-age. Default values are already configured
24893 in this way.
24894 Windows support - Experimental
24896 There are a couple of issues with Windows mount functionality that
24897 still require some investigations. It should be considered as experi‐
24898 mental thus far as fixes come in for this OS.
24899 Most of the issues seem to be related to the difference between
24901 filesystems on Linux flavors and Windows as cache is heavily dependent
24902 on them.
24903 Any reports or feedback on how cache behaves on this OS is greatly ap‐
24905 preciated.
24906 • https://github.com/rclone/rclone/issues/1935
24908 • https://github.com/rclone/rclone/issues/1907
24910 • https://github.com/rclone/rclone/issues/1834
24912 Risk of throttling
24914 Future iterations of the cache backend will make use of the pooling
24915 functionality of the cloud provider to synchronize and at the same time
24916 make writing through it more tolerant to failures.
24917 There are a couple of enhancements in track to add these but in the
24919 meantime there is a valid concern that the expiring cache listings can
24920 lead to cloud provider throttles or bans due to repeated queries on it
24921 for very large mounts.
24922 Some recommendations: - don't use a very small interval for entry in‐
24924 formation (--cache-info-age) - while writes aren't yet optimised, you
24925 can still write through cache which gives you the advantage of adding
24926 the file in the cache at the same time if configured to do so.
24927 Future enhancements:
24929 • https://github.com/rclone/rclone/issues/1937
24931 • https://github.com/rclone/rclone/issues/1936
24933 cache and crypt
24935 One common scenario is to keep your data encrypted in the cloud
24936 provider using the crypt remote. crypt uses a similar technique to
24937 wrap around an existing remote and handles this translation in a seam‐
24938 less way.
24939 There is an issue with wrapping the remotes in this order: cloud remote
24941 -> crypt -> cache
24942 During testing, I experienced a lot of bans with the remotes in this
24944 order. I suspect it might be related to how crypt opens files on the
24945 cloud provider which makes it think we're downloading the full file in‐
24946 stead of small chunks. Organizing the remotes in this order yields
24947 better results: cloud remote -> cache -> crypt
24948 absolute remote paths
24950 cache can not differentiate between relative and absolute paths for the
24951 wrapped remote. Any path given in the remote config setting and on the
24952 command line will be passed to the wrapped remote as is, but for stor‐
24953 ing the chunks on disk the path will be made relative by removing any
24954 leading / character.
24955 This behavior is irrelevant for most backend types, but there are back‐
24957 ends where a leading / changes the effective directory, e.g. in the
24958 sftp backend paths starting with a / are relative to the root of the
24959 SSH server and paths without are relative to the user home directory.
24960 As a result sftp:bin and sftp:/bin will share the same cache folder,
24961 even if they represent a different directory on the SSH server.
24962 Cache and Remote Control (--rc)
24964 Cache supports the new --rc mode in rclone and can be remote controlled
24965 through the following end points: By default, the listener is disabled
24966 if you do not add the flag.
24967 rc cache/expire
24969 Purge a remote from the cache backend. Supports either a directory or
24970 a file. It supports both encrypted and unencrypted file names if cache
24971 is wrapped by crypt.
24972 Params: - remote = path to remote (required) - withData = true/false to
24974 delete cached data (chunks) as well (optional, false by default)
24975 Standard options
24977 Here are the Standard options specific to cache (Cache a remote).
24978 --cache-remote
24980 Remote to cache.
24981 Normally should contain a ':' and a path, e.g. "myremote:path/to/dir",
24983 "myremote:bucket" or maybe "myremote:" (not recommended).
24984 Properties:
24986 • Config: remote
24988 • Env Var: RCLONE_CACHE_REMOTE
24990 • Type: string
24992 • Required: true
24994 --cache-plex-url
24996 The URL of the Plex server.
24997 Properties:
24999 • Config: plex_url
25001 • Env Var: RCLONE_CACHE_PLEX_URL
25003 • Type: string
25005 • Required: false
25007 --cache-plex-username
25009 The username of the Plex user.
25010 Properties:
25012 • Config: plex_username
25014 • Env Var: RCLONE_CACHE_PLEX_USERNAME
25016 • Type: string
25018 • Required: false
25020 --cache-plex-password
25022 The password of the Plex user.
25023 NB Input to this must be obscured - see rclone obscure
25025 (https://rclone.org/commands/rclone_obscure/).
25026 Properties:
25028 • Config: plex_password
25030 • Env Var: RCLONE_CACHE_PLEX_PASSWORD
25032 • Type: string
25034 • Required: false
25036 --cache-chunk-size
25038 The size of a chunk (partial file data).
25039 Use lower numbers for slower connections. If the chunk size is
25041 changed, any downloaded chunks will be invalid and cache-chunk-path
25042 will need to be cleared or unexpected EOF errors will occur.
25043 Properties:
25045 • Config: chunk_size
25047 • Env Var: RCLONE_CACHE_CHUNK_SIZE
25049 • Type: SizeSuffix
25051 • Default: 5Mi
25053 • Examples:
25055 • "1M"
25057 • 1 MiB
25059 • "5M"
25061 • 5 MiB
25063 • "10M"
25065 • 10 MiB
25067 --cache-info-age
25069 How long to cache file structure information (directory listings, file
25070 size, times, etc.). If all write operations are done through the cache
25071 then you can safely make this value very large as the cache store will
25072 also be updated in real time.
25073 Properties:
25075 • Config: info_age
25077 • Env Var: RCLONE_CACHE_INFO_AGE
25079 • Type: Duration
25081 • Default: 6h0m0s
25083 • Examples:
25085 • "1h"
25087 • 1 hour
25089 • "24h"
25091 • 24 hours
25093 • "48h"
25095 • 48 hours
25097 --cache-chunk-total-size
25099 The total size that the chunks can take up on the local disk.
25100 If the cache exceeds this value then it will start to delete the oldest
25102 chunks until it goes under this value.
25103 Properties:
25105 • Config: chunk_total_size
25107 • Env Var: RCLONE_CACHE_CHUNK_TOTAL_SIZE
25109 • Type: SizeSuffix
25111 • Default: 10Gi
25113 • Examples:
25115 • "500M"
25117 • 500 MiB
25119 • "1G"
25121 • 1 GiB
25123 • "10G"
25125 • 10 GiB
25127 Advanced options
25129 Here are the Advanced options specific to cache (Cache a remote).
25130 --cache-plex-token
25132 The plex token for authentication - auto set normally.
25133 Properties:
25135 • Config: plex_token
25137 • Env Var: RCLONE_CACHE_PLEX_TOKEN
25139 • Type: string
25141 • Required: false
25143 --cache-plex-insecure
25145 Skip all certificate verification when connecting to the Plex server.
25146 Properties:
25148 • Config: plex_insecure
25150 • Env Var: RCLONE_CACHE_PLEX_INSECURE
25152 • Type: string
25154 • Required: false
25156 --cache-db-path
25158 Directory to store file structure metadata DB.
25159 The remote name is used as the DB file name.
25161 Properties:
25163 • Config: db_path
25165 • Env Var: RCLONE_CACHE_DB_PATH
25167 • Type: string
25169 • Default: "$HOME/.cache/rclone/cache-backend"
25171 --cache-chunk-path
25173 Directory to cache chunk files.
25174 Path to where partial file data (chunks) are stored locally. The re‐
25176 mote name is appended to the final path.
25177 This config follows the "--cache-db-path". If you specify a custom lo‐
25179 cation for "--cache-db-path" and don't specify one for "--cache-chunk-
25180 path" then "--cache-chunk-path" will use the same path as "--cache-db-
25181 path".
25182 Properties:
25184 • Config: chunk_path
25186 • Env Var: RCLONE_CACHE_CHUNK_PATH
25188 • Type: string
25190 • Default: "$HOME/.cache/rclone/cache-backend"
25192 --cache-db-purge
25194 Clear all the cached data for this remote on start.
25195 Properties:
25197 • Config: db_purge
25199 • Env Var: RCLONE_CACHE_DB_PURGE
25201 • Type: bool
25203 • Default: false
25205 --cache-chunk-clean-interval
25207 How often should the cache perform cleanups of the chunk storage.
25208 The default value should be ok for most people. If you find that the
25210 cache goes over "cache-chunk-total-size" too often then try to lower
25211 this value to force it to perform cleanups more often.
25212 Properties:
25214 • Config: chunk_clean_interval
25216 • Env Var: RCLONE_CACHE_CHUNK_CLEAN_INTERVAL
25218 • Type: Duration
25220 • Default: 1m0s
25222 --cache-read-retries
25224 How many times to retry a read from a cache storage.
25225 Since reading from a cache stream is independent from downloading file
25227 data, readers can get to a point where there's no more data in the
25228 cache. Most of the times this can indicate a connectivity issue if
25229 cache isn't able to provide file data anymore.
25230 For really slow connections, increase this to a point where the stream
25232 is able to provide data but your experience will be very stuttering.
25233 Properties:
25235 • Config: read_retries
25237 • Env Var: RCLONE_CACHE_READ_RETRIES
25239 • Type: int
25241 • Default: 10
25243 --cache-workers
25245 How many workers should run in parallel to download chunks.
25246 Higher values will mean more parallel processing (better CPU needed)
25248 and more concurrent requests on the cloud provider. This impacts sev‐
25249 eral aspects like the cloud provider API limits, more stress on the
25250 hardware that rclone runs on but it also means that streams will be
25251 more fluid and data will be available much more faster to readers.
25252 Note: If the optional Plex integration is enabled then this setting
25254 will adapt to the type of reading performed and the value specified
25255 here will be used as a maximum number of workers to use.
25256 Properties:
25258 • Config: workers
25260 • Env Var: RCLONE_CACHE_WORKERS
25262 • Type: int
25264 • Default: 4
25266 --cache-chunk-no-memory
25268 Disable the in-memory cache for storing chunks during streaming.
25269 By default, cache will keep file data during streaming in RAM as well
25271 to provide it to readers as fast as possible.
25272 This transient data is evicted as soon as it is read and the number of
25274 chunks stored doesn't exceed the number of workers. However, depending
25275 on other settings like "cache-chunk-size" and "cache-workers" this
25276 footprint can increase if there are parallel streams too (multiple
25277 files being read at the same time).
25278 If the hardware permits it, use this feature to provide an overall bet‐
25280 ter performance during streaming but it can also be disabled if RAM is
25281 not available on the local machine.
25282 Properties:
25284 • Config: chunk_no_memory
25286 • Env Var: RCLONE_CACHE_CHUNK_NO_MEMORY
25288 • Type: bool
25290 • Default: false
25292 --cache-rps
25294 Limits the number of requests per second to the source FS (-1 to dis‐
25295 able).
25296 This setting places a hard limit on the number of requests per second
25298 that cache will be doing to the cloud provider remote and try to re‐
25299 spect that value by setting waits between reads.
25300 If you find that you're getting banned or limited on the cloud provider
25302 through cache and know that a smaller number of requests per second
25303 will allow you to work with it then you can use this setting for that.
25304 A good balance of all the other settings should make this setting use‐
25306 less but it is available to set for more special cases.
25307 NOTE: This will limit the number of requests during streams but other
25309 API calls to the cloud provider like directory listings will still
25310 pass.
25311 Properties:
25313 • Config: rps
25315 • Env Var: RCLONE_CACHE_RPS
25317 • Type: int
25319 • Default: -1
25321 --cache-writes
25323 Cache file data on writes through the FS.
25324 If you need to read files immediately after you upload them through
25326 cache you can enable this flag to have their data stored in the cache
25327 store at the same time during upload.
25328 Properties:
25330 • Config: writes
25332 • Env Var: RCLONE_CACHE_WRITES
25334 • Type: bool
25336 • Default: false
25338 --cache-tmp-upload-path
25340 Directory to keep temporary files until they are uploaded.
25341 This is the path where cache will use as a temporary storage for new
25343 files that need to be uploaded to the cloud provider.
25344 Specifying a value will enable this feature. Without it, it is com‐
25346 pletely disabled and files will be uploaded directly to the cloud
25347 provider
25348 Properties:
25350 • Config: tmp_upload_path
25352 • Env Var: RCLONE_CACHE_TMP_UPLOAD_PATH
25354 • Type: string
25356 • Required: false
25358 --cache-tmp-wait-time
25360 How long should files be stored in local cache before being uploaded.
25361 This is the duration that a file must wait in the temporary location
25363 cache-tmp-upload-path before it is selected for upload.
25364 Note that only one file is uploaded at a time and it can take longer to
25366 start the upload if a queue formed for this purpose.
25367 Properties:
25369 • Config: tmp_wait_time
25371 • Env Var: RCLONE_CACHE_TMP_WAIT_TIME
25373 • Type: Duration
25375 • Default: 15s
25377 --cache-db-wait-time
25379 How long to wait for the DB to be available - 0 is unlimited.
25380 Only one process can have the DB open at any one time, so rclone waits
25382 for this duration for the DB to become available before it gives an er‐
25383 ror.
25384 If you set it to 0 then it will wait forever.
25386 Properties:
25388 • Config: db_wait_time
25390 • Env Var: RCLONE_CACHE_DB_WAIT_TIME
25392 • Type: Duration
25394 • Default: 1s
25396 Backend commands
25398 Here are the commands specific to the cache backend.
25399 Run them with
25401 rclone backend COMMAND remote:
25403 The help below will explain what arguments each command takes.
25405 See the backend (https://rclone.org/commands/rclone_backend/) command
25407 for more info on how to pass options and arguments.
25408 These can be run on a running backend using the rc command backend/com‐
25410 mand (https://rclone.org/rc/#backend-command).
25411 stats
25413 Print stats on the cache backend in JSON format.
25414 rclone backend stats remote: [options] [<arguments>+]
25416
25418 The chunker overlay transparently splits large files into smaller
25419 chunks during upload to wrapped remote and transparently assembles them
25420 back when the file is downloaded. This allows to effectively overcome
25421 size limits imposed by storage providers.
25422 Configuration
25424 To use it, first set up the underlying remote following the configura‐
25425 tion instructions for that remote. You can also use a local pathname
25426 instead of a remote.
25427 First check your chosen remote is working - we'll call it remote:path
25429 here. Note that anything inside remote:path will be chunked and any‐
25430 thing outside won't. This means that if you are using a bucket-based
25431 remote (e.g. S3, B2, swift) then you should probably put the bucket in
25432 the remote s3:bucket.
25433 Now configure chunker using rclone config. We will call this one over‐
25435 lay to separate it from the remote itself.
25436 No remotes found, make a new one?
25438 n) New remote
25439 s) Set configuration password
25440 q) Quit config
25441 n/s/q> n
25442 name> overlay
25443 Type of storage to configure.
25444 Choose a number from below, or type in your own value
25445 [snip]
25446 XX / Transparently chunk/split large files
25447 \ "chunker"
25448 [snip]
25449 Storage> chunker
25450 Remote to chunk/unchunk.
25451 Normally should contain a ':' and a path, e.g. "myremote:path/to/dir",
25452 "myremote:bucket" or maybe "myremote:" (not recommended).
25453 Enter a string value. Press Enter for the default ("").
25454 remote> remote:path
25455 Files larger than chunk size will be split in chunks.
25456 Enter a size with suffix K,M,G,T. Press Enter for the default ("2G").
25457 chunk_size> 100M
25458 Choose how chunker handles hash sums. All modes but "none" require metadata.
25459 Enter a string value. Press Enter for the default ("md5").
25460 Choose a number from below, or type in your own value
25461 1 / Pass any hash supported by wrapped remote for non-chunked files, return nothing otherwise
25462 \ "none"
25463 2 / MD5 for composite files
25464 \ "md5"
25465 3 / SHA1 for composite files
25466 \ "sha1"
25467 4 / MD5 for all files
25468 \ "md5all"
25469 5 / SHA1 for all files
25470 \ "sha1all"
25471 6 / Copying a file to chunker will request MD5 from the source falling back to SHA1 if unsupported
25472 \ "md5quick"
25473 7 / Similar to "md5quick" but prefers SHA1 over MD5
25474 \ "sha1quick"
25475 hash_type> md5
25476 Edit advanced config? (y/n)
25477 y) Yes
25478 n) No
25479 y/n> n
25480 Remote config
25481 --------------------
25482 [overlay]
25483 type = chunker
25484 remote = remote:bucket
25485 chunk_size = 100M
25486 hash_type = md5
25487 --------------------
25488 y) Yes this is OK
25489 e) Edit this remote
25490 d) Delete this remote
25491 y/e/d> y
25492 Specifying the remote
25494 In normal use, make sure the remote has a : in. If you specify the re‐
25495 mote without a : then rclone will use a local directory of that name.
25496 So if you use a remote of /path/to/secret/files then rclone will chunk
25497 stuff in that directory. If you use a remote of name then rclone will
25498 put files in a directory called name in the current directory.
25499 Chunking
25501 When rclone starts a file upload, chunker checks the file size. If it
25502 doesn't exceed the configured chunk size, chunker will just pass the
25503 file to the wrapped remote. If a file is large, chunker will transpar‐
25504 ently cut data in pieces with temporary names and stream them one by
25505 one, on the fly. Each data chunk will contain the specified number of
25506 bytes, except for the last one which may have less data. If file size
25507 is unknown in advance (this is called a streaming upload), chunker will
25508 internally create a temporary copy, record its size and repeat the
25509 above process.
25510 When upload completes, temporary chunk files are finally renamed. This
25512 scheme guarantees that operations can be run in parallel and look from
25513 outside as atomic. A similar method with hidden temporary chunks is
25514 used for other operations (copy/move/rename, etc.). If an operation
25515 fails, hidden chunks are normally destroyed, and the target composite
25516 file stays intact.
25517 When a composite file download is requested, chunker transparently as‐
25519 sembles it by concatenating data chunks in order. As the split is
25520 trivial one could even manually concatenate data chunks together to ob‐
25521 tain the original content.
25522 When the list rclone command scans a directory on wrapped remote, the
25524 potential chunk files are accounted for, grouped and assembled into
25525 composite directory entries. Any temporary chunks are hidden.
25526 List and other commands can sometimes come across composite files with
25528 missing or invalid chunks, e.g. shadowed by like-named directory or
25529 another file. This usually means that wrapped file system has been di‐
25530 rectly tampered with or damaged. If chunker detects a missing chunk it
25531 will by default print warning, skip the whole incomplete group of
25532 chunks but proceed with current command. You can set the --chunker-
25533 fail-hard flag to have commands abort with error message in such cases.
25534 Chunk names
25536 The default chunk name format is *.rclone_chunk.###, hence by default
25537 chunk names are BIG_FILE_NAME.rclone_chunk.001,
25538 BIG_FILE_NAME.rclone_chunk.002 etc. You can configure another name
25539 format using the name_format configuration file option. The format us‐
25540 es asterisk * as a placeholder for the base file name and one or more
25541 consecutive hash characters # as a placeholder for sequential chunk
25542 number. There must be one and only one asterisk. The number of con‐
25543 secutive hash characters defines the minimum length of a string repre‐
25544 senting a chunk number. If decimal chunk number has less digits than
25545 the number of hashes, it is left-padded by zeros. If the decimal
25546 string is longer, it is left intact. By default numbering starts from
25547 1 but there is another option that allows user to start from 0, e.g.
25548 for compatibility with legacy software.
25549 For example, if name format is big_*-##.part and original file name is
25551 data.txt and numbering starts from 0, then the first chunk will be
25552 named big_data.txt-00.part, the 99th chunk will be big_data.txt-98.part
25553 and the 302nd chunk will become big_data.txt-301.part.
25554 Note that list assembles composite directory entries only when chunk
25556 names match the configured format and treats non-conforming file names
25557 as normal non-chunked files.
25558 When using norename transactions, chunk names will additionally have a
25560 unique file version suffix. For example,
25561 BIG_FILE_NAME.rclone_chunk.001_bp562k.
25562 Metadata
25564 Besides data chunks chunker will by default create metadata object for
25565 a composite file. The object is named after the original file. Chun‐
25566 ker allows user to disable metadata completely (the none format). Note
25567 that metadata is normally not created for files smaller than the con‐
25568 figured chunk size. This may change in future rclone releases.
25569 Simple JSON metadata format
25571 This is the default format. It supports hash sums and chunk validation
25572 for composite files. Meta objects carry the following fields:
25573 • ver - version of format, currently 1
25575 • size - total size of composite file
25577 • nchunks - number of data chunks in file
25579 • md5 - MD5 hashsum of composite file (if present)
25581 • sha1 - SHA1 hashsum (if present)
25583 • txn - identifies current version of the file
25585 There is no field for composite file name as it's simply equal to the
25587 name of meta object on the wrapped remote. Please refer to respective
25588 sections for details on hashsums and modified time handling.
25589 No metadata
25591 You can disable meta objects by setting the meta format option to none.
25592 In this mode chunker will scan directory for all files that follow con‐
25593 figured chunk name format, group them by detecting chunks with the same
25594 base name and show group names as virtual composite files. This method
25595 is more prone to missing chunk errors (especially missing last chunk)
25596 than format with metadata enabled.
25597 Hashsums
25599 Chunker supports hashsums only when a compatible metadata is present.
25600 Hence, if you choose metadata format of none, chunker will report hash‐
25601 sum as UNSUPPORTED.
25602 Please note that by default metadata is stored only for composite
25604 files. If a file is smaller than configured chunk size, chunker will
25605 transparently redirect hash requests to wrapped remote, so support de‐
25606 pends on that. You will see the empty string as a hashsum of requested
25607 type for small files if the wrapped remote doesn't support it.
25608 Many storage backends support MD5 and SHA1 hash types, so does chunker.
25610 With chunker you can choose one or another but not both. MD5 is set by
25611 default as the most supported type. Since chunker keeps hashes for
25612 composite files and falls back to the wrapped remote hash for non-chun‐
25613 ked ones, we advise you to choose the same hash type as supported by
25614 wrapped remote so that your file listings look coherent.
25615 If your storage backend does not support MD5 or SHA1 but you need con‐
25617 sistent file hashing, configure chunker with md5all or sha1all. These
25618 two modes guarantee given hash for all files. If wrapped remote
25619 doesn't support it, chunker will then add metadata to all files, even
25620 small. However, this can double the amount of small files in storage
25621 and incur additional service charges. You can even use chunker to
25622 force md5/sha1 support in any other remote at expense of sidecar meta
25623 objects by setting e.g. chunk_type=sha1all to force hashsums and
25624 chunk_size=1P to effectively disable chunking.
25625 Normally, when a file is copied to chunker controlled remote, chunker
25627 will ask the file source for compatible file hash and revert to on-the-
25628 fly calculation if none is found. This involves some CPU overhead but
25629 provides a guarantee that given hashsum is available. Also, chunker
25630 will reject a server-side copy or move operation if source and destina‐
25631 tion hashsum types are different resulting in the extra network band‐
25632 width, too. In some rare cases this may be undesired, so chunker pro‐
25633 vides two optional choices: sha1quick and md5quick. If the source does
25634 not support primary hash type and the quick mode is enabled, chunker
25635 will try to fall back to the secondary type. This will save CPU and
25636 bandwidth but can result in empty hashsums at destination. Beware of
25637 consequences: the sync command will revert (sometimes silently) to
25638 time/size comparison if compatible hashsums between source and target
25639 are not found.
25640 Modified time
25642 Chunker stores modification times using the wrapped remote so support
25643 depends on that. For a small non-chunked file the chunker overlay sim‐
25644 ply manipulates modification time of the wrapped remote file. For a
25645 composite file with metadata chunker will get and set modification time
25646 of the metadata object on the wrapped remote. If file is chunked but
25647 metadata format is none then chunker will use modification time of the
25648 first data chunk.
25649 Migrations
25651 The idiomatic way to migrate to a different chunk size, hash type,
25652 transaction style or chunk naming scheme is to:
25653 • Collect all your chunked files under a directory and have your chun‐
25655 ker remote point to it.
25656 • Create another directory (most probably on the same cloud storage)
25658 and configure a new remote with desired metadata format, hash type,
25659 chunk naming etc.
25660 • Now run rclone sync -i oldchunks: newchunks: and all your data will
25662 be transparently converted in transfer. This may take some time, yet
25663 chunker will try server-side copy if possible.
25664 • After checking data integrity you may remove configuration section of
25666 the old remote.
25667 If rclone gets killed during a long operation on a big composite file,
25669 hidden temporary chunks may stay in the directory. They will not be
25670 shown by the list command but will eat up your account quota. Please
25671 note that the deletefile command deletes only active chunks of a file.
25672 As a workaround, you can use remote of the wrapped file system to see
25673 them. An easy way to get rid of hidden garbage is to copy littered di‐
25674 rectory somewhere using the chunker remote and purge the original di‐
25675 rectory. The copy command will copy only active chunks while the purge
25676 will remove everything including garbage.
25677 Caveats and Limitations
25679 Chunker requires wrapped remote to support server-side move (or copy +
25680 delete) operations, otherwise it will explicitly refuse to start. This
25681 is because it internally renames temporary chunk files to their final
25682 names when an operation completes successfully.
25683 Chunker encodes chunk number in file name, so with default name_format
25685 setting it adds 17 characters. Also chunker adds 7 characters of tem‐
25686 porary suffix during operations. Many file systems limit base file
25687 name without path by 255 characters. Using rclone's crypt remote as a
25688 base file system limits file name by 143 characters. Thus, maximum
25689 name length is 231 for most files and 119 for chunker-over-crypt. A
25690 user in need can change name format to e.g. *.rcc## and save 10 char‐
25691 acters (provided at most 99 chunks per file).
25692 Note that a move implemented using the copy-and-delete method may incur
25694 double charging with some cloud storage providers.
25695 Chunker will not automatically rename existing chunks when you run
25697 rclone config on a live remote and change the chunk name format. Be‐
25698 ware that in result of this some files which have been treated as
25699 chunks before the change can pop up in directory listings as normal
25700 files and vice versa. The same warning holds for the chunk size. If
25701 you desperately need to change critical chunking settings, you should
25702 run data migration as described above.
25703 If wrapped remote is case insensitive, the chunker overlay will inherit
25705 that property (so you can't have a file called "Hello.doc" and "hel‐
25706 lo.doc" in the same directory).
25707 Chunker included in rclone releases up to v1.54 can sometimes fail to
25709 detect metadata produced by recent versions of rclone. We recommend
25710 users to keep rclone up-to-date to avoid data corruption.
25711 Changing transactions is dangerous and requires explicit migration.
25713 Standard options
25715 Here are the Standard options specific to chunker (Transparently
25716 chunk/split large files).
25717 --chunker-remote
25719 Remote to chunk/unchunk.
25720 Normally should contain a ':' and a path, e.g. "myremote:path/to/dir",
25722 "myremote:bucket" or maybe "myremote:" (not recommended).
25723 Properties:
25725 • Config: remote
25727 • Env Var: RCLONE_CHUNKER_REMOTE
25729 • Type: string
25731 • Required: true
25733 --chunker-chunk-size
25735 Files larger than chunk size will be split in chunks.
25736 Properties:
25738 • Config: chunk_size
25740 • Env Var: RCLONE_CHUNKER_CHUNK_SIZE
25742 • Type: SizeSuffix
25744 • Default: 2Gi
25746 --chunker-hash-type
25748 Choose how chunker handles hash sums.
25749 All modes but "none" require metadata.
25751 Properties:
25753 • Config: hash_type
25755 • Env Var: RCLONE_CHUNKER_HASH_TYPE
25757 • Type: string
25759 • Default: "md5"
25761 • Examples:
25763 • "none"
25765 • Pass any hash supported by wrapped remote for non-chunked files.
25767 • Return nothing otherwise.
25769 • "md5"
25771 • MD5 for composite files.
25773 • "sha1"
25775 • SHA1 for composite files.
25777 • "md5all"
25779 • MD5 for all files.
25781 • "sha1all"
25783 • SHA1 for all files.
25785 • "md5quick"
25787 • Copying a file to chunker will request MD5 from the source.
25789 • Falling back to SHA1 if unsupported.
25791 • "sha1quick"
25793 • Similar to "md5quick" but prefers SHA1 over MD5.
25795 Advanced options
25797 Here are the Advanced options specific to chunker (Transparently
25798 chunk/split large files).
25799 --chunker-name-format
25801 String format of chunk file names.
25802 The two placeholders are: base file name (*) and chunk number (#...).
25804 There must be one and only one asterisk and one or more consecutive
25805 hash characters. If chunk number has less digits than the number of
25806 hashes, it is left-padded by zeros. If there are more digits in the
25807 number, they are left as is. Possible chunk files are ignored if their
25808 name does not match given format.
25809 Properties:
25811 • Config: name_format
25813 • Env Var: RCLONE_CHUNKER_NAME_FORMAT
25815 • Type: string
25817 • Default: "*.rclone_chunk.###"
25819 --chunker-start-from
25821 Minimum valid chunk number. Usually 0 or 1.
25822 By default chunk numbers start from 1.
25824 Properties:
25826 • Config: start_from
25828 • Env Var: RCLONE_CHUNKER_START_FROM
25830 • Type: int
25832 • Default: 1
25834 --chunker-meta-format
25836 Format of the metadata object or "none".
25837 By default "simplejson". Metadata is a small JSON file named after the
25839 composite file.
25840 Properties:
25842 • Config: meta_format
25844 • Env Var: RCLONE_CHUNKER_META_FORMAT
25846 • Type: string
25848 • Default: "simplejson"
25850 • Examples:
25852 • "none"
25854 • Do not use metadata files at all.
25856 • Requires hash type "none".
25858 • "simplejson"
25860 • Simple JSON supports hash sums and chunk validation.
25862 • It has the following fields: ver, size, nchunks, md5, sha1.
25864 --chunker-fail-hard
25866 Choose how chunker should handle files with missing or invalid chunks.
25867 Properties:
25869 • Config: fail_hard
25871 • Env Var: RCLONE_CHUNKER_FAIL_HARD
25873 • Type: bool
25875 • Default: false
25877 • Examples:
25879 • "true"
25881 • Report errors and abort current command.
25883 • "false"
25885 • Warn user, skip incomplete file and proceed.
25887 --chunker-transactions
25889 Choose how chunker should handle temporary files during transactions.
25890 Properties:
25892 • Config: transactions
25894 • Env Var: RCLONE_CHUNKER_TRANSACTIONS
25896 • Type: string
25898 • Default: "rename"
25900 • Examples:
25902 • "rename"
25904 • Rename temporary files after a successful transaction.
25906 • "norename"
25908 • Leave temporary file names and write transaction ID to metadata
25910 file.
25911 • Metadata is required for no rename transactions (meta format can‐
25913 not be "none").
25914 • If you are using norename transactions you should be careful not
25916 to downgrade Rclone
25917 • as older versions of Rclone don't support this transaction style
25919 and will misinterpret
25920 • files manipulated by norename transactions.
25922 • This method is EXPERIMENTAL, don't use on production systems.
25924 • "auto"
25926 • Rename or norename will be used depending on capabilities of the
25928 backend.
25929 • If meta format is set to "none", rename transactions will always
25931 be used.
25932 • This method is EXPERIMENTAL, don't use on production systems.
25934
25936 Citrix ShareFile (https://sharefile.com) is a secure file sharing and
25937 transfer service aimed as business.
25938 Configuration
25940 The initial setup for Citrix ShareFile involves getting a token from
25941 Citrix ShareFile which you can in your browser. rclone config walks
25942 you through it.
25943 Here is an example of how to make a remote called remote. First run:
25945 rclone config
25947 This will guide you through an interactive setup process:
25949 No remotes found, make a new one?
25951 n) New remote
25952 s) Set configuration password
25953 q) Quit config
25954 n/s/q> n
25955 name> remote
25956 Type of storage to configure.
25957 Enter a string value. Press Enter for the default ("").
25958 Choose a number from below, or type in your own value
25959 XX / Citrix Sharefile
25960 \ "sharefile"
25961 Storage> sharefile
25962 ** See help for sharefile backend at: https://rclone.org/sharefile/ **
25963 ID of the root folder
25965 Leave blank to access "Personal Folders". You can use one of the
25967 standard values here or any folder ID (long hex number ID).
25968 Enter a string value. Press Enter for the default ("").
25969 Choose a number from below, or type in your own value
25970 1 / Access the Personal Folders. (Default)
25971 \ ""
25972 2 / Access the Favorites folder.
25973 \ "favorites"
25974 3 / Access all the shared folders.
25975 \ "allshared"
25976 4 / Access all the individual connectors.
25977 \ "connectors"
25978 5 / Access the home, favorites, and shared folders as well as the connectors.
25979 \ "top"
25980 root_folder_id>
25981 Edit advanced config? (y/n)
25982 y) Yes
25983 n) No
25984 y/n> n
25985 Remote config
25986 Use auto config?
25987 * Say Y if not sure
25988 * Say N if you are working on a remote or headless machine
25989 y) Yes
25990 n) No
25991 y/n> y
25992 If your browser doesn't open automatically go to the following link: http://127.0.0.1:53682/auth?state=XXX
25993 Log in and authorize rclone for access
25994 Waiting for code...
25995 Got code
25996 --------------------
25997 [remote]
25998 type = sharefile
25999 endpoint = https://XXX.sharefile.com
26000 token = {"access_token":"XXX","token_type":"bearer","refresh_token":"XXX","expiry":"2019-09-30T19:41:45.878561877+01:00"}
26001 --------------------
26002 y) Yes this is OK
26003 e) Edit this remote
26004 d) Delete this remote
26005 y/e/d> y
26006 See the remote setup docs (https://rclone.org/remote_setup/) for how to
26008 set it up on a machine with no Internet browser available.
26009 Note that rclone runs a webserver on your local machine to collect the
26011 token as returned from Citrix ShareFile. This only runs from the mo‐
26012 ment it opens your browser to the moment you get back the verification
26013 code. This is on http://127.0.0.1:53682/ and this it may require you
26014 to unblock it temporarily if you are running a host firewall.
26015 Once configured you can then use rclone like this,
26017 List directories in top level of your ShareFile
26019 rclone lsd remote:
26021 List all the files in your ShareFile
26023 rclone ls remote:
26025 To copy a local directory to an ShareFile directory called backup
26027 rclone copy /home/source remote:backup
26029 Paths may be as deep as required, e.g. remote:directory/subdirectory.
26031 Modified time and hashes
26033 ShareFile allows modification times to be set on objects accurate to 1
26034 second. These will be used to detect whether objects need syncing or
26035 not.
26036 ShareFile supports MD5 type hashes, so you can use the --checksum flag.
26038 Transfers
26040 For files above 128 MiB rclone will use a chunked transfer. Rclone
26041 will upload up to --transfers chunks at the same time (shared among all
26042 the multipart uploads). Chunks are buffered in memory and are normally
26043 64 MiB so increasing --transfers will increase memory use.
26044 Restricted filename characters
26046 In addition to the default restricted characters set
26047 (https://rclone.org/overview/#restricted-characters) the following
26048 characters are also replaced:
26049 Character Value Replacement
26051 ────────────────────────────────
26052 \ 0x5C ＼
26053 * 0x2A ＊
26054 < 0x3C ＜
26055 > 0x3E ＞
26056 ? 0x3F ？
26057 : 0x3A ：
26058 | 0x7C ｜
26059 " 0x22 ＂
26060 File names can also not start or end with the following characters.
26062 These only get replaced if they are the first or last character in the
26063 name:
26064 Character Value Replacement
26066 ────────────────────────────────
26067 SP 0x20 ␠
26068 . 0x2E ．
26069 Invalid UTF-8 bytes will also be replaced (https://rclone.org/over‐
26071 view/#invalid-utf8), as they can't be used in JSON strings.
26072 Standard options
26074 Here are the Standard options specific to sharefile (Citrix Sharefile).
26075 --sharefile-root-folder-id
26077 ID of the root folder.
26078 Leave blank to access "Personal Folders". You can use one of the stan‐
26080 dard values here or any folder ID (long hex number ID).
26081 Properties:
26083 • Config: root_folder_id
26085 • Env Var: RCLONE_SHAREFILE_ROOT_FOLDER_ID
26087 • Type: string
26089 • Required: false
26091 • Examples:
26093 • ""
26095 • Access the Personal Folders (default).
26097 • "favorites"
26099 • Access the Favorites folder.
26101 • "allshared"
26103 • Access all the shared folders.
26105 • "connectors"
26107 • Access all the individual connectors.
26109 • "top"
26111 • Access the home, favorites, and shared folders as well as the
26113 connectors.
26114 Advanced options
26116 Here are the Advanced options specific to sharefile (Citrix Sharefile).
26117 --sharefile-upload-cutoff
26119 Cutoff for switching to multipart upload.
26120 Properties:
26122 • Config: upload_cutoff
26124 • Env Var: RCLONE_SHAREFILE_UPLOAD_CUTOFF
26126 • Type: SizeSuffix
26128 • Default: 128Mi
26130 --sharefile-chunk-size
26132 Upload chunk size.
26133 Must a power of 2 >= 256k.
26135 Making this larger will improve performance, but note that each chunk
26137 is buffered in memory one per transfer.
26138 Reducing this will reduce memory usage but decrease performance.
26140 Properties:
26142 • Config: chunk_size
26144 • Env Var: RCLONE_SHAREFILE_CHUNK_SIZE
26146 • Type: SizeSuffix
26148 • Default: 64Mi
26150 --sharefile-endpoint
26152 Endpoint for API calls.
26153 This is usually auto discovered as part of the oauth process, but can
26155 be set manually to something like: https://XXX.sharefile.com
26156 Properties:
26158 • Config: endpoint
26160 • Env Var: RCLONE_SHAREFILE_ENDPOINT
26162 • Type: string
26164 • Required: false
26166 --sharefile-encoding
26168 The encoding for the backend.
26169 See the encoding section in the overview (https://rclone.org/over‐
26171 view/#encoding) for more info.
26172 Properties:
26174 • Config: encoding
26176 • Env Var: RCLONE_SHAREFILE_ENCODING
26178 • Type: MultiEncoder
26180 • Default: Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,Back‐
26182 Slash,Ctl,LeftSpace,LeftPeriod,RightSpace,RightPeriod,InvalidUtf8,Dot
26183 Limitations
26185 Note that ShareFile is case insensitive so you can't have a file called
26186 "Hello.doc" and one called "hello.doc".
26187 ShareFile only supports filenames up to 256 characters in length.
26189 rclone about is not supported by the Citrix ShareFile backend. Back‐
26191 ends without this capability cannot determine free space for an rclone
26192 mount or use policy mfs (most free space) as a member of an rclone
26193 union remote.
26194 See List of backends that do not support rclone about
26196 (https://rclone.org/overview/#optional-features) and rclone about
26197 (https://rclone.org/commands/rclone_about/)
26198
26200 Rclone crypt remotes encrypt and decrypt other remotes.
26201 A remote of type crypt does not access a storage system
26203 (https://rclone.org/overview/) directly, but instead wraps another re‐
26204 mote, which in turn accesses the storage system. This is similar to
26205 how alias (https://rclone.org/alias/), union
26206 (https://rclone.org/union/), chunker (https://rclone.org/chunker/) and
26207 a few others work. It makes the usage very flexible, as you can add a
26208 layer, in this case an encryption layer, on top of any other backend,
26209 even in multiple layers. Rclone's functionality can be used as with
26210 any other remote, for example you can mount (https://rclone.org/com‐
26211 mands/rclone_mount/) a crypt remote.
26212 Accessing a storage system through a crypt remote realizes client-side
26214 encryption, which makes it safe to keep your data in a location you do
26215 not trust will not get compromised. When working against the crypt re‐
26216 mote, rclone will automatically encrypt (before uploading) and decrypt
26217 (after downloading) on your local system as needed on the fly, leaving
26218 the data encrypted at rest in the wrapped remote. If you access the
26219 storage system using an application other than rclone, or access the
26220 wrapped remote directly using rclone, there will not be any encryp‐
26221 tion/decryption: Downloading existing content will just give you the
26222 encrypted (scrambled) format, and anything you upload will not become
26223 encrypted.
26224 The encryption is a secret-key encryption (also called symmetric key
26226 encryption) algorithm, where a password (or pass phrase) is used to
26227 generate real encryption key. The password can be supplied by user, or
26228 you may chose to let rclone generate one. It will be stored in the
26229 configuration file, in a lightly obscured form. If you are in an envi‐
26230 ronment where you are not able to keep your configuration secured, you
26231 should add configuration encryption (https://rclone.org/docs/#configu‐
26232 ration-encryption) as protection. As long as you have this configura‐
26233 tion file, you will be able to decrypt your data. Without the configu‐
26234 ration file, as long as you remember the password (or keep it in a safe
26235 place), you can re-create the configuration and gain access to the ex‐
26236 isting data. You may also configure a corresponding remote in a dif‐
26237 ferent installation to access the same data. See below for guidance to
26238 changing password.
26239 Encryption uses cryptographic salt (https://en.wikipedia.org/wi‐
26241 ki/Salt_(cryptography)), to permute the encryption key so that the same
26242 string may be encrypted in different ways. When configuring the crypt
26243 remote it is optional to enter a salt, or to let rclone generate a
26244 unique salt. If omitted, rclone uses a built-in unique string. Nor‐
26245 mally in cryptography, the salt is stored together with the encrypted
26246 content, and do not have to be memorized by the user. This is not the
26247 case in rclone, because rclone does not store any additional informa‐
26248 tion on the remotes. Use of custom salt is effectively a second pass‐
26249 word that must be memorized.
26250 File content encryption is performed using NaCl SecretBox
26252 (https://godoc.org/golang.org/x/crypto/nacl/secretbox), based on XSal‐
26253 sa20 cipher and Poly1305 for integrity. Names (file- and directory
26254 names) are also encrypted by default, but this has some implications
26255 and is therefore possible to turned off.
26256 Configuration
26258 Here is an example of how to make a remote called secret.
26259 To use crypt, first set up the underlying remote. Follow the rclone
26261 config instructions for the specific backend.
26262 Before configuring the crypt remote, check the underlying remote is
26264 working. In this example the underlying remote is called remote. We
26265 will configure a path path within this remote to contain the encrypted
26266 content. Anything inside remote:path will be encrypted and anything
26267 outside will not.
26268 Configure crypt using rclone config. In this example the crypt remote
26270 is called secret, to differentiate it from the underlying remote.
26271 When you are done you can use the crypt remote named secret just as you
26273 would with any other remote, e.g. rclone copy D:\docs secret:\docs,
26274 and rclone will encrypt and decrypt as needed on the fly. If you ac‐
26275 cess the wrapped remote remote:path directly you will bypass the en‐
26276 cryption, and anything you read will be in encrypted form, and anything
26277 you write will be unencrypted. To avoid issues it is best to configure
26278 a dedicated path for encrypted content, and access it exclusively
26279 through a crypt remote.
26280 No remotes found, make a new one?
26282 n) New remote
26283 s) Set configuration password
26284 q) Quit config
26285 n/s/q> n
26286 name> secret
26287 Type of storage to configure.
26288 Enter a string value. Press Enter for the default ("").
26289 Choose a number from below, or type in your own value
26290 [snip]
26291 XX / Encrypt/Decrypt a remote
26292 \ "crypt"
26293 [snip]
26294 Storage> crypt
26295 ** See help for crypt backend at: https://rclone.org/crypt/ **
26296 Remote to encrypt/decrypt.
26298 Normally should contain a ':' and a path, eg "myremote:path/to/dir",
26299 "myremote:bucket" or maybe "myremote:" (not recommended).
26300 Enter a string value. Press Enter for the default ("").
26301 remote> remote:path
26302 How to encrypt the filenames.
26303 Enter a string value. Press Enter for the default ("standard").
26304 Choose a number from below, or type in your own value.
26305 / Encrypt the filenames.
26306 1 | See the docs for the details.
26307 \ "standard"
26308 2 / Very simple filename obfuscation.
26309 \ "obfuscate"
26310 / Don't encrypt the file names.
26311 3 | Adds a ".bin" extension only.
26312 \ "off"
26313 filename_encryption>
26314 Option to either encrypt directory names or leave them intact.
26315 NB If filename_encryption is "off" then this option will do nothing.
26317 Enter a boolean value (true or false). Press Enter for the default ("true").
26318 Choose a number from below, or type in your own value
26319 1 / Encrypt directory names.
26320 \ "true"
26321 2 / Don't encrypt directory names, leave them intact.
26322 \ "false"
26323 directory_name_encryption>
26324 Password or pass phrase for encryption.
26325 y) Yes type in my own password
26326 g) Generate random password
26327 y/g> y
26328 Enter the password:
26329 password:
26330 Confirm the password:
26331 password:
26332 Password or pass phrase for salt. Optional but recommended.
26333 Should be different to the previous password.
26334 y) Yes type in my own password
26335 g) Generate random password
26336 n) No leave this optional password blank (default)
26337 y/g/n> g
26338 Password strength in bits.
26339 64 is just about memorable
26340 128 is secure
26341 1024 is the maximum
26342 Bits> 128
26343 Your password is: JAsJvRcgR-_veXNfy_sGmQ
26344 Use this password? Please note that an obscured version of this
26345 password (and not the password itself) will be stored under your
26346 configuration file, so keep this generated password in a safe place.
26347 y) Yes (default)
26348 n) No
26349 y/n>
26350 Edit advanced config? (y/n)
26351 y) Yes
26352 n) No (default)
26353 y/n>
26354 Remote config
26355 --------------------
26356 [secret]
26357 type = crypt
26358 remote = remote:path
26359 password = *** ENCRYPTED ***
26360 password2 = *** ENCRYPTED ***
26361 --------------------
26362 y) Yes this is OK (default)
26363 e) Edit this remote
26364 d) Delete this remote
26365 y/e/d>
26366 Important The crypt password stored in rclone.conf is lightly obscured.
26368 That only protects it from cursory inspection. It is not secure unless
26369 configuration encryption (https://rclone.org/docs/#configuration-en‐
26370 cryption) of rclone.conf is specified.
26371 A long passphrase is recommended, or rclone config can generate a ran‐
26373 dom one.
26374 The obscured password is created using AES-CTR with a static key. The
26376 salt is stored verbatim at the beginning of the obscured password.
26377 This static key is shared between all versions of rclone.
26378 If you reconfigure rclone with the same passwords/passphrases elsewhere
26380 it will be compatible, but the obscured version will be different due
26381 to the different salt.
26382 Rclone does not encrypt
26384 • file length - this can be calculated within 16 bytes
26386 • modification time - used for syncing
26388 Specifying the remote
26390 When configuring the remote to encrypt/decrypt, you may specify any
26391 string that rclone accepts as a source/destination of other commands.
26392 The primary use case is to specify the path into an already configured
26394 remote (e.g. remote:path/to/dir or remote:bucket), such that data in a
26395 remote untrusted location can be stored encrypted.
26396 You may also specify a local filesystem path, such as /path/to/dir on
26398 Linux, C:\path\to\dir on Windows. By creating a crypt remote pointing
26399 to such a local filesystem path, you can use rclone as a utility for
26400 pure local file encryption, for example to keep encrypted files on a
26401 removable USB drive.
26402 Note: A string which do not contain a : will by rclone be treated as a
26404 relative path in the local filesystem. For example, if you enter the
26405 name remote without the trailing :, it will be treated as a subdirecto‐
26406 ry of the current directory with name "remote".
26407 If a path remote:path/to/dir is specified, rclone stores encrypted
26409 files in path/to/dir on the remote. With file name encryption, files
26410 saved to secret:subdir/subfile are stored in the unencrypted path
26411 path/to/dir but the subdir/subpath element is encrypted.
26412 The path you specify does not have to exist, rclone will create it when
26414 needed.
26415 If you intend to use the wrapped remote both directly for keeping unen‐
26417 crypted content, as well as through a crypt remote for encrypted con‐
26418 tent, it is recommended to point the crypt remote to a separate direc‐
26419 tory within the wrapped remote. If you use a bucket-based storage sys‐
26420 tem (e.g. Swift, S3, Google Compute Storage, B2) it is generally ad‐
26421 visable to wrap the crypt remote around a specific bucket (s3:bucket).
26422 If wrapping around the entire root of the storage (s3:), and use the
26423 optional file name encryption, rclone will encrypt the bucket name.
26424 Changing password
26426 Should the password, or the configuration file containing a lightly ob‐
26427 scured form of the password, be compromised, you need to re-encrypt
26428 your data with a new password. Since rclone uses secret-key encryp‐
26429 tion, where the encryption key is generated directly from the password
26430 kept on the client, it is not possible to change the password/key of
26431 already encrypted content. Just changing the password configured for
26432 an existing crypt remote means you will no longer able to decrypt any
26433 of the previously encrypted content. The only possibility is to re-up‐
26434 load everything via a crypt remote configured with your new password.
26435 Depending on the size of your data, your bandwidth, storage quota etc,
26437 there are different approaches you can take: - If you have everything
26438 in a different location, for example on your local system, you could
26439 remove all of the prior encrypted files, change the password for your
26440 configured crypt remote (or delete and re-create the crypt configura‐
26441 tion), and then re-upload everything from the alternative location. -
26442 If you have enough space on the storage system you can create a new
26443 crypt remote pointing to a separate directory on the same backend, and
26444 then use rclone to copy everything from the original crypt remote to
26445 the new, effectively decrypting everything on the fly using the old
26446 password and re-encrypting using the new password. When done, delete
26447 the original crypt remote directory and finally the rclone crypt con‐
26448 figuration with the old password. All data will be streamed from the
26449 storage system and back, so you will get half the bandwidth and be
26450 charged twice if you have upload and download quota on the storage sys‐
26451 tem.
26452 Note: A security problem related to the random password generator was
26454 fixed in rclone version 1.53.3 (released 2020-11-19). Passwords gener‐
26455 ated by rclone config in version 1.49.0 (released 2019-08-26) to 1.53.2
26456 (released 2020-10-26) are not considered secure and should be changed.
26457 If you made up your own password, or used rclone version older than
26458 1.49.0 or newer than 1.53.2 to generate it, you are not affected by
26459 this issue. See issue #4783 (https://github.com/rclone/rclone/is‐
26460 sues/4783) for more details, and a tool you can use to check if you are
26461 affected.
26462 Example
26464 Create the following file structure using "standard" file name encryp‐
26465 tion.
26466 plaintext/
26468 ├── file0.txt
26469 ├── file1.txt
26470 └── subdir
26471 ├── file2.txt
26472 ├── file3.txt
26473 └── subsubdir
26474 └── file4.txt
26475 Copy these to the remote, and list them
26477 $ rclone -q copy plaintext secret:
26479 $ rclone -q ls secret:
26480 7 file1.txt
26481 6 file0.txt
26482 8 subdir/file2.txt
26483 10 subdir/subsubdir/file4.txt
26484 9 subdir/file3.txt
26485 The crypt remote looks like
26487 $ rclone -q ls remote:path
26489 55 hagjclgavj2mbiqm6u6cnjjqcg
26490 54 v05749mltvv1tf4onltun46gls
26491 57 86vhrsv86mpbtd3a0akjuqslj8/dlj7fkq4kdq72emafg7a7s41uo
26492 58 86vhrsv86mpbtd3a0akjuqslj8/7uu829995du6o42n32otfhjqp4/b9pausrfansjth5ob3jkdqd4lc
26493 56 86vhrsv86mpbtd3a0akjuqslj8/8njh1sk437gttmep3p70g81aps
26494 The directory structure is preserved
26496 $ rclone -q ls secret:subdir
26498 8 file2.txt
26499 9 file3.txt
26500 10 subsubdir/file4.txt
26501 Without file name encryption .bin extensions are added to underlying
26503 names. This prevents the cloud provider attempting to interpret file
26504 content.
26505 $ rclone -q ls remote:path
26507 54 file0.txt.bin
26508 57 subdir/file3.txt.bin
26509 56 subdir/file2.txt.bin
26510 58 subdir/subsubdir/file4.txt.bin
26511 55 file1.txt.bin
26512 File name encryption modes
26514 Off
26515 • doesn't hide file names or directory structure
26517 • allows for longer file names (~246 characters)
26519 • can use sub paths and copy single files
26521 Standard
26523 • file names encrypted
26525 • file names can't be as long (~143 characters)
26527 • can use sub paths and copy single files
26529 • directory structure visible
26531 • identical files names will have identical uploaded names
26533 • can use shortcuts to shorten the directory recursion
26535 Obfuscation
26537 This is a simple "rotate" of the filename, with each file having a rot
26539 distance based on the filename. Rclone stores the distance at the be‐
26540 ginning of the filename. A file called "hello" may become "53.jgnnq".
26541 Obfuscation is not a strong encryption of filenames, but hinders auto‐
26543 mated scanning tools picking up on filename patterns. It is an inter‐
26544 mediate between "off" and "standard" which allows for longer path seg‐
26545 ment names.
26546 There is a possibility with some unicode based filenames that the ob‐
26548 fuscation is weak and may map lower case characters to upper case
26549 equivalents.
26550 Obfuscation cannot be relied upon for strong protection.
26552 • file names very lightly obfuscated
26554 • file names can be longer than standard encryption
26556 • can use sub paths and copy single files
26558 • directory structure visible
26560 • identical files names will have identical uploaded names
26562 Cloud storage systems have limits on file name length and total path
26564 length which rclone is more likely to breach using "Standard" file name
26565 encryption. Where file names are less than 156 characters in length
26566 issues should not be encountered, irrespective of cloud storage
26567 provider.
26568 An experimental advanced option filename_encoding is now provided to
26570 address this problem to a certain degree. For cloud storage systems
26571 with case sensitive file names (e.g. Google Drive), base64 can be used
26572 to reduce file name length. For cloud storage systems using UTF-16 to
26573 store file names internally (e.g. OneDrive), base32768 can be used to
26574 drastically reduce file name length.
26575 An alternative, future rclone file name encryption mode may tolerate
26577 backend provider path length limits.
26578 Directory name encryption
26580 Crypt offers the option of encrypting dir names or leaving them intact.
26581 There are two options:
26582 True
26584 Encrypts the whole file path including directory names Example:
26586 1/12/123.txt is encrypted to
26587 p0e52nreeaj0a5ea7s64m4j72s/l42g6771hnv3an9cgc8cr2n1ng/qgm4avr35m5loi1th53ato71v0
26588 False
26590 Only encrypts file names, skips directory names Example: 1/12/123.txt
26592 is encrypted to 1/12/qgm4avr35m5loi1th53ato71v0
26593 Modified time and hashes
26595 Crypt stores modification times using the underlying remote so support
26596 depends on that.
26597 Hashes are not stored for crypt. However the data integrity is pro‐
26599 tected by an extremely strong crypto authenticator.
26600 Use the rclone cryptcheck command to check the integrity of a crypted
26602 remote instead of rclone check which can't check the checksums proper‐
26603 ly.
26604 Standard options
26606 Here are the Standard options specific to crypt (Encrypt/Decrypt a re‐
26607 mote).
26608 --crypt-remote
26610 Remote to encrypt/decrypt.
26611 Normally should contain a ':' and a path, e.g. "myremote:path/to/dir",
26613 "myremote:bucket" or maybe "myremote:" (not recommended).
26614 Properties:
26616 • Config: remote
26618 • Env Var: RCLONE_CRYPT_REMOTE
26620 • Type: string
26622 • Required: true
26624 --crypt-filename-encryption
26626 How to encrypt the filenames.
26627 Properties:
26629 • Config: filename_encryption
26631 • Env Var: RCLONE_CRYPT_FILENAME_ENCRYPTION
26633 • Type: string
26635 • Default: "standard"
26637 • Examples:
26639 • "standard"
26641 • Encrypt the filenames.
26643 • See the docs for the details.
26645 • "obfuscate"
26647 • Very simple filename obfuscation.
26649 • "off"
26651 • Don't encrypt the file names.
26653 • Adds a ".bin" extension only.
26655 --crypt-directory-name-encryption
26657 Option to either encrypt directory names or leave them intact.
26658 NB If filename_encryption is "off" then this option will do nothing.
26660 Properties:
26662 • Config: directory_name_encryption
26664 • Env Var: RCLONE_CRYPT_DIRECTORY_NAME_ENCRYPTION
26666 • Type: bool
26668 • Default: true
26670 • Examples:
26672 • "true"
26674 • Encrypt directory names.
26676 • "false"
26678 • Don't encrypt directory names, leave them intact.
26680 --crypt-password
26682 Password or pass phrase for encryption.
26683 NB Input to this must be obscured - see rclone obscure
26685 (https://rclone.org/commands/rclone_obscure/).
26686 Properties:
26688 • Config: password
26690 • Env Var: RCLONE_CRYPT_PASSWORD
26692 • Type: string
26694 • Required: true
26696 --crypt-password2
26698 Password or pass phrase for salt.
26699 Optional but recommended. Should be different to the previous pass‐
26701 word.
26702 NB Input to this must be obscured - see rclone obscure
26704 (https://rclone.org/commands/rclone_obscure/).
26705 Properties:
26707 • Config: password2
26709 • Env Var: RCLONE_CRYPT_PASSWORD2
26711 • Type: string
26713 • Required: false
26715 Advanced options
26717 Here are the Advanced options specific to crypt (Encrypt/Decrypt a re‐
26718 mote).
26719 --crypt-server-side-across-configs
26721 Allow server-side operations (e.g. copy) to work across different
26722 crypt configs.
26723 Normally this option is not what you want, but if you have two crypts
26725 pointing to the same backend you can use it.
26726 This can be used, for example, to change file name encryption type
26728 without re-uploading all the data. Just make two crypt backends point‐
26729 ing to two different directories with the single changed parameter and
26730 use rclone move to move the files between the crypt remotes.
26731 Properties:
26733 • Config: server_side_across_configs
26735 • Env Var: RCLONE_CRYPT_SERVER_SIDE_ACROSS_CONFIGS
26737 • Type: bool
26739 • Default: false
26741 --crypt-show-mapping
26743 For all files listed show how the names encrypt.
26744 If this flag is set then for each file that the remote is asked to
26746 list, it will log (at level INFO) a line stating the decrypted file
26747 name and the encrypted file name.
26748 This is so you can work out which encrypted names are which decrypted
26750 names just in case you need to do something with the encrypted file
26751 names, or for debugging purposes.
26752 Properties:
26754 • Config: show_mapping
26756 • Env Var: RCLONE_CRYPT_SHOW_MAPPING
26758 • Type: bool
26760 • Default: false
26762 --crypt-no-data-encryption
26764 Option to either encrypt file data or leave it unencrypted.
26765 Properties:
26767 • Config: no_data_encryption
26769 • Env Var: RCLONE_CRYPT_NO_DATA_ENCRYPTION
26771 • Type: bool
26773 • Default: false
26775 • Examples:
26777 • "true"
26779 • Don't encrypt file data, leave it unencrypted.
26781 • "false"
26783 • Encrypt file data.
26785 --crypt-filename-encoding
26787 How to encode the encrypted filename to text string.
26788 This option could help with shortening the encrypted filename. The
26790 suitable option would depend on the way your remote count the filename
26791 length and if it's case sensitive.
26792 Properties:
26794 • Config: filename_encoding
26796 • Env Var: RCLONE_CRYPT_FILENAME_ENCODING
26798 • Type: string
26800 • Default: "base32"
26802 • Examples:
26804 • "base32"
26806 • Encode using base32. Suitable for all remote.
26808 • "base64"
26810 • Encode using base64. Suitable for case sensitive remote.
26812 • "base32768"
26814 • Encode using base32768. Suitable if your remote counts UTF-16 or
26816 • Unicode codepoint instead of UTF-8 byte length. (Eg. Onedrive)
26818 Metadata
26820 Any metadata supported by the underlying remote is read and written.
26821 See the metadata (https://rclone.org/docs/#metadata) docs for more in‐
26823 fo.
26824 Backend commands
26826 Here are the commands specific to the crypt backend.
26827 Run them with
26829 rclone backend COMMAND remote:
26831 The help below will explain what arguments each command takes.
26833 See the backend (https://rclone.org/commands/rclone_backend/) command
26835 for more info on how to pass options and arguments.
26836 These can be run on a running backend using the rc command backend/com‐
26838 mand (https://rclone.org/rc/#backend-command).
26839 encode
26841 Encode the given filename(s)
26842 rclone backend encode remote: [options] [<arguments>+]
26844 This encodes the filenames given as arguments returning a list of
26846 strings of the encoded results.
26847 Usage Example:
26849 rclone backend encode crypt: file1 [file2...]
26851 rclone rc backend/command command=encode fs=crypt: file1 [file2...]
26852 decode
26854 Decode the given filename(s)
26855 rclone backend decode remote: [options] [<arguments>+]
26857 This decodes the filenames given as arguments returning a list of
26859 strings of the decoded results. It will return an error if any of the
26860 inputs are invalid.
26861 Usage Example:
26863 rclone backend decode crypt: encryptedfile1 [encryptedfile2...]
26865 rclone rc backend/command command=decode fs=crypt: encryptedfile1 [encryptedfile2...]
26866 Backing up a crypted remote
26868 If you wish to backup a crypted remote, it is recommended that you use
26869 rclone sync on the encrypted files, and make sure the passwords are the
26870 same in the new encrypted remote.
26871 This will have the following advantages
26873 • rclone sync will check the checksums while copying
26875 • you can use rclone check between the encrypted remotes
26877 • you don't decrypt and encrypt unnecessarily
26879 For example, let's say you have your original remote at remote: with
26881 the encrypted version at eremote: with path remote:crypt. You would
26882 then set up the new remote remote2: and then the encrypted version ere‐
26883 mote2: with path remote2:crypt using the same passwords as eremote:.
26884 To sync the two remotes you would do
26886 rclone sync -i remote:crypt remote2:crypt
26888 And to check the integrity you would do
26890 rclone check remote:crypt remote2:crypt
26892 File formats
26894 File encryption
26895 Files are encrypted 1:1 source file to destination object. The file
26896 has a header and is divided into chunks.
26897 Header
26899 • 8 bytes magic string RCLONE\x00\x00
26900 • 24 bytes Nonce (IV)
26902 The initial nonce is generated from the operating systems crypto strong
26904 random number generator. The nonce is incremented for each chunk read
26905 making sure each nonce is unique for each block written. The chance of
26906 a nonce being re-used is minuscule. If you wrote an exabyte of data
26907 (10¹⁸ bytes) you would have a probability of approximately 2×10⁻³² of
26908 re-using a nonce.
26909 Chunk
26911 Each chunk will contain 64 KiB of data, except for the last one which
26912 may have less data. The data chunk is in standard NaCl SecretBox for‐
26913 mat. SecretBox uses XSalsa20 and Poly1305 to encrypt and authenticate
26914 messages.
26915 Each chunk contains:
26917 • 16 Bytes of Poly1305 authenticator
26919 • 1 - 65536 bytes XSalsa20 encrypted data
26921 64k chunk size was chosen as the best performing chunk size (the au‐
26923 thenticator takes too much time below this and the performance drops
26924 off due to cache effects above this). Note that these chunks are
26925 buffered in memory so they can't be too big.
26926 This uses a 32 byte (256 bit key) key derived from the user password.
26928 Examples
26930 1 byte file will encrypt to
26931 • 32 bytes header
26933 • 17 bytes data chunk
26935 49 bytes total
26937 1 MiB (1048576 bytes) file will encrypt to
26939 • 32 bytes header
26941 • 16 chunks of 65568 bytes
26943 1049120 bytes total (a 0.05% overhead). This is the overhead for big
26945 files.
26946 Name encryption
26948 File names are encrypted segment by segment - the path is broken up in‐
26949 to / separated strings and these are encrypted individually.
26950 File segments are padded using PKCS#7 to a multiple of 16 bytes before
26952 encryption.
26953 They are then encrypted with EME using AES with 256 bit key. EME (ECB-
26955 Mix-ECB) is a wide-block encryption mode presented in the 2003 paper "A
26956 Parallelizable Enciphering Mode" by Halevi and Rogaway.
26957 This makes for deterministic encryption which is what we want - the
26959 same filename must encrypt to the same thing otherwise we can't find it
26960 on the cloud storage system.
26961 This means that
26963 • filenames with the same name will encrypt the same
26965 • filenames which start the same won't have a common prefix
26967 This uses a 32 byte key (256 bits) and a 16 byte (128 bits) IV both of
26969 which are derived from the user password.
26970 After encryption they are written out using a modified version of stan‐
26972 dard base32 encoding as described in RFC4648. The standard encoding is
26973 modified in two ways:
26974 • it becomes lower case (no-one likes upper case filenames!)
26976 • we strip the padding character =
26978 base32 is used rather than the more efficient base64 so rclone can be
26980 used on case insensitive remotes (e.g. Windows, Amazon Drive).
26981 Key derivation
26983 Rclone uses scrypt with parameters N=16384, r=8, p=1 with an optional
26984 user supplied salt (password2) to derive the 32+32+16 = 80 bytes of key
26985 material required. If the user doesn't supply a salt then rclone uses
26986 an internal one.
26987 scrypt makes it impractical to mount a dictionary attack on rclone en‐
26989 crypted data. For full protection against this you should always use a
26990 salt.
26991 SEE ALSO
26993 • rclone cryptdecode (https://rclone.org/commands/rclone_cryptdecode/)
26994 - Show forward/reverse mapping of encrypted filenames
26995
26997 Warning
26998 This remote is currently experimental. Things may break and data may
26999 be lost. Anything you do with this remote is at your own risk. Please
27000 understand the risks associated with using experimental code and don't
27001 use this remote in critical applications.
27002 The Compress remote adds compression to another remote. It is best
27004 used with remotes containing many large compressible files.
27005 Configuration
27007 To use this remote, all you need to do is specify another remote and a
27008 compression mode to use:
27009 Current remotes:
27011 Name Type
27013 ==== ====
27014 remote_to_press sometype
27015 e) Edit existing remote
27017 $ rclone config
27018 n) New remote
27019 d) Delete remote
27020 r) Rename remote
27021 c) Copy remote
27022 s) Set configuration password
27023 q) Quit config
27024 e/n/d/r/c/s/q> n
27025 name> compress
27026 ...
27027 8 / Compress a remote
27028 \ "compress"
27029 ...
27030 Storage> compress
27031 ** See help for compress backend at: https://rclone.org/compress/ **
27032 Remote to compress.
27034 Enter a string value. Press Enter for the default ("").
27035 remote> remote_to_press:subdir
27036 Compression mode.
27037 Enter a string value. Press Enter for the default ("gzip").
27038 Choose a number from below, or type in your own value
27039 1 / Gzip compression balanced for speed and compression strength.
27040 \ "gzip"
27041 compression_mode> gzip
27042 Edit advanced config? (y/n)
27043 y) Yes
27044 n) No (default)
27045 y/n> n
27046 Remote config
27047 --------------------
27048 [compress]
27049 type = compress
27050 remote = remote_to_press:subdir
27051 compression_mode = gzip
27052 --------------------
27053 y) Yes this is OK (default)
27054 e) Edit this remote
27055 d) Delete this remote
27056 y/e/d> y
27057 Compression Modes
27059 Currently only gzip compression is supported. It provides a decent
27060 balance between speed and size and is well supported by other applica‐
27061 tions. Compression strength can further be configured via an advanced
27062 setting where 0 is no compression and 9 is strongest compression.
27063 File types
27065 If you open a remote wrapped by compress, you will see that there are
27066 many files with an extension corresponding to the compression algorithm
27067 you chose. These files are standard files that can be opened by vari‐
27068 ous archive programs, but they have some hidden metadata that allows
27069 them to be used by rclone. While you may download and decompress these
27070 files at will, do not manually delete or rename files. Files without
27071 correct metadata files will not be recognized by rclone.
27072 File names
27074 The compressed files will be named *.###########.gz where * is the base
27075 file and the # part is base64 encoded size of the uncompressed file.
27076 The file names should not be changed by anything other than the rclone
27077 compression backend.
27078 Standard options
27080 Here are the Standard options specific to compress (Compress a remote).
27081 --compress-remote
27083 Remote to compress.
27084 Properties:
27086 • Config: remote
27088 • Env Var: RCLONE_COMPRESS_REMOTE
27090 • Type: string
27092 • Required: true
27094 --compress-mode
27096 Compression mode.
27097 Properties:
27099 • Config: mode
27101 • Env Var: RCLONE_COMPRESS_MODE
27103 • Type: string
27105 • Default: "gzip"
27107 • Examples:
27109 • "gzip"
27111 • Standard gzip compression with fastest parameters.
27113 Advanced options
27115 Here are the Advanced options specific to compress (Compress a remote).
27116 --compress-level
27118 GZIP compression level (-2 to 9).
27119 Generally -1 (default, equivalent to 5) is recommended. Levels 1 to 9
27121 increase compression at the cost of speed. Going past 6 generally of‐
27122 fers very little return.
27123 Level -2 uses Huffman encoding only. Only use if you know what you are
27125 doing. Level 0 turns off compression.
27126 Properties:
27128 • Config: level
27130 • Env Var: RCLONE_COMPRESS_LEVEL
27132 • Type: int
27134 • Default: -1
27136 --compress-ram-cache-limit
27138 Some remotes don't allow the upload of files with unknown size. In
27139 this case the compressed file will need to be cached to determine it's
27140 size.
27141 Files smaller than this limit will be cached in RAM, files larger than
27143 this limit will be cached on disk.
27144 Properties:
27146 • Config: ram_cache_limit
27148 • Env Var: RCLONE_COMPRESS_RAM_CACHE_LIMIT
27150 • Type: SizeSuffix
27152 • Default: 20Mi
27154 Metadata
27156 Any metadata supported by the underlying remote is read and written.
27157 See the metadata (https://rclone.org/docs/#metadata) docs for more in‐
27159 fo.
27160
27162 The combine backend joins remotes together into a single directory
27163 tree.
27164 For example you might have a remote for images on one provider:
27166 $ rclone tree s3:imagesbucket
27168 /
27169 ├── image1.jpg
27170 └── image2.jpg
27171 And a remote for files on another:
27173 $ rclone tree drive:important/files
27175 /
27176 ├── file1.txt
27177 └── file2.txt
27178 The combine backend can join these together into a synthetic directory
27180 structure like this:
27181 $ rclone tree combined:
27183 /
27184 ├── files
27185 │ ├── file1.txt
27186 │ └── file2.txt
27187 └── images
27188 ├── image1.jpg
27189 └── image2.jpg
27190 You'd do this by specifying an upstreams parameter in the config like
27192 this
27193 upstreams = images=s3:imagesbucket files=drive:important/files
27195 During the initial setup with rclone config you will specify the up‐
27197 streams remotes as a space separated list. The upstream remotes can
27198 either be a local paths or other remotes.
27199 Configuration
27201 Here is an example of how to make a combine called remote for the exam‐
27202 ple above. First run:
27203 rclone config
27205 This will guide you through an interactive setup process:
27207 No remotes found, make a new one?
27209 n) New remote
27210 s) Set configuration password
27211 q) Quit config
27212 n/s/q> n
27213 name> remote
27214 Option Storage.
27215 Type of storage to configure.
27216 Choose a number from below, or type in your own value.
27217 ...
27218 XX / Combine several remotes into one
27219 \ (combine)
27220 ...
27221 Storage> combine
27222 Option upstreams.
27223 Upstreams for combining
27224 These should be in the form
27225 dir=remote:path dir2=remote2:path
27226 Where before the = is specified the root directory and after is the remote to
27227 put there.
27228 Embedded spaces can be added using quotes
27229 "dir=remote:path with space" "dir2=remote2:path with space"
27230 Enter a fs.SpaceSepList value.
27231 upstreams> images=s3:imagesbucket files=drive:important/files
27232 --------------------
27233 [remote]
27234 type = combine
27235 upstreams = images=s3:imagesbucket files=drive:important/files
27236 --------------------
27237 y) Yes this is OK (default)
27238 e) Edit this remote
27239 d) Delete this remote
27240 y/e/d> y
27241 Configuring for Google Drive Shared Drives
27243 Rclone has a convenience feature for making a combine backend for all
27244 the shared drives you have access to.
27245 Assuming your main (non shared drive) Google drive remote is called
27247 drive: you would run
27248 rclone backend -o config drives drive:
27250 This would produce something like this:
27252 [My Drive]
27254 type = alias
27255 remote = drive,team_drive=0ABCDEF-01234567890,root_folder_id=:
27256 [Test Drive]
27258 type = alias
27259 remote = drive,team_drive=0ABCDEFabcdefghijkl,root_folder_id=:
27260 [AllDrives]
27262 type = combine
27263 upstreams = "My Drive=My Drive:" "Test Drive=Test Drive:"
27264 If you then add that config to your config file (find it with rclone
27266 config file) then you can access all the shared drives in one place
27267 with the AllDrives: remote.
27268 See the Google Drive docs (https://rclone.org/drive/#drives) for full
27270 info.
27271 Standard options
27273 Here are the Standard options specific to combine (Combine several re‐
27274 motes into one).
27275 --combine-upstreams
27277 Upstreams for combining
27278 These should be in the form
27280 dir=remote:path dir2=remote2:path
27282 Where before the = is specified the root directory and after is the re‐
27284 mote to put there.
27285 Embedded spaces can be added using quotes
27287 "dir=remote:path with space" "dir2=remote2:path with space"
27289 Properties:
27291 • Config: upstreams
27293 • Env Var: RCLONE_COMBINE_UPSTREAMS
27295 • Type: SpaceSepList
27297 • Default:
27299 Metadata
27301 Any metadata supported by the underlying remote is read and written.
27302 See the metadata (https://rclone.org/docs/#metadata) docs for more in‐
27304 fo.
27305
27307 Paths are specified as remote:path
27308 Dropbox paths may be as deep as required, e.g. remote:directory/subdi‐
27310 rectory.
27311 Configuration
27313 The initial setup for dropbox involves getting a token from Dropbox
27314 which you need to do in your browser. rclone config walks you through
27315 it.
27316 Here is an example of how to make a remote called remote. First run:
27318 rclone config
27320 This will guide you through an interactive setup process:
27322 n) New remote
27324 d) Delete remote
27325 q) Quit config
27326 e/n/d/q> n
27327 name> remote
27328 Type of storage to configure.
27329 Choose a number from below, or type in your own value
27330 [snip]
27331 XX / Dropbox
27332 \ "dropbox"
27333 [snip]
27334 Storage> dropbox
27335 Dropbox App Key - leave blank normally.
27336 app_key>
27337 Dropbox App Secret - leave blank normally.
27338 app_secret>
27339 Remote config
27340 Please visit:
27341 https://www.dropbox.com/1/oauth2/authorize?client_id=XXXXXXXXXXXXXXX&response_type=code
27342 Enter the code: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX_XXXXXXXXXX
27343 --------------------
27344 [remote]
27345 app_key =
27346 app_secret =
27347 token = XXXXXXXXXXXXXXXXXXXXXXXXXXXXX_XXXX_XXXXXXXXXXXXXXXXXXXXXXXXXXXXX
27348 --------------------
27349 y) Yes this is OK
27350 e) Edit this remote
27351 d) Delete this remote
27352 y/e/d> y
27353 You can then use it like this,
27355 List directories in top level of your dropbox
27357 rclone lsd remote:
27359 List all the files in your dropbox
27361 rclone ls remote:
27363 To copy a local directory to a dropbox directory called backup
27365 rclone copy /home/source remote:backup
27367 Dropbox for business
27369 Rclone supports Dropbox for business and Team Folders.
27370 When using Dropbox for business remote: and remote:path/to/file will
27372 refer to your personal folder.
27373 If you wish to see Team Folders you must use a leading / in the path,
27375 so rclone lsd remote:/ will refer to the root and show you all Team
27376 Folders and your User Folder.
27377 You can then use team folders like this remote:/TeamFolder and re‐
27379 mote:/TeamFolder/path/to/file.
27380 A leading / for a Dropbox personal account will do nothing, but it will
27382 take an extra HTTP transaction so it should be avoided.
27383 Modified time and Hashes
27385 Dropbox supports modified times, but the only way to set a modification
27386 time is to re-upload the file.
27387 This means that if you uploaded your data with an older version of
27389 rclone which didn't support the v2 API and modified times, rclone will
27390 decide to upload all your old data to fix the modification times. If
27391 you don't want this to happen use --size-only or --checksum flag to
27392 stop it.
27393 Dropbox supports its own hash type (https://www.dropbox.com/develop‐
27395 ers/reference/content-hash) which is checked for all transfers.
27396 Restricted filename characters
27398 Character Value Replacement
27399 ────────────────────────────────
27400 NUL 0x00 ␀
27401 / 0x2F ／
27402 DEL 0x7F ␡
27403 \ 0x5C ＼
27404 File names can also not end with the following characters. These only
27406 get replaced if they are the last character in the name:
27407 Character Value Replacement
27409 ────────────────────────────────
27410 SP 0x20 ␠
27411 Invalid UTF-8 bytes will also be replaced (https://rclone.org/over‐
27413 view/#invalid-utf8), as they can't be used in JSON strings.
27414 Batch mode uploads
27416 Using batch mode uploads is very important for performance when using
27417 the Dropbox API. See the dropbox performance guide (https://develop‐
27418 ers.dropbox.com/dbx-performance-guide) for more info.
27419 There are 3 modes rclone can use for uploads.
27421 --dropbox-batch-mode off
27423 In this mode rclone will not use upload batching. This was the default
27424 before rclone v1.55. It has the disadvantage that it is very likely to
27425 encounter too_many_requests errors like this
27426 NOTICE: too_many_requests/.: Too many requests or write operations. Trying again in 15 seconds.
27428 When rclone receives these it has to wait for 15s or sometimes 300s be‐
27430 fore continuing which really slows down transfers.
27431 This will happen especially if --transfers is large, so this mode isn't
27433 recommended except for compatibility or investigating problems.
27434 --dropbox-batch-mode sync
27436 In this mode rclone will batch up uploads to the size specified by
27437 --dropbox-batch-size and commit them together.
27438 Using this mode means you can use a much higher --transfers parameter
27440 (32 or 64 works fine) without receiving too_many_requests errors.
27441 This mode ensures full data integrity.
27443 Note that there may be a pause when quitting rclone while rclone fin‐
27445 ishes up the last batch using this mode.
27446 --dropbox-batch-mode async
27448 In this mode rclone will batch up uploads to the size specified by
27449 --dropbox-batch-size and commit them together.
27450 However it will not wait for the status of the batch to be returned to
27452 the caller. This means rclone can use a much bigger batch size (much
27453 bigger than --transfers), at the cost of not being able to check the
27454 status of the upload.
27455 This provides the maximum possible upload speed especially with lots of
27457 small files, however rclone can't check the file got uploaded properly
27458 using this mode.
27459 If you are using this mode then using "rclone check" after the transfer
27461 completes is recommended. Or you could do an initial transfer with
27462 --dropbox-batch-mode async then do a final transfer with --dropbox-
27463 batch-mode sync (the default).
27464 Note that there may be a pause when quitting rclone while rclone fin‐
27466 ishes up the last batch using this mode.
27467 Standard options
27469 Here are the Standard options specific to dropbox (Dropbox).
27470 --dropbox-client-id
27472 OAuth Client Id.
27473 Leave blank normally.
27475 Properties:
27477 • Config: client_id
27479 • Env Var: RCLONE_DROPBOX_CLIENT_ID
27481 • Type: string
27483 • Required: false
27485 --dropbox-client-secret
27487 OAuth Client Secret.
27488 Leave blank normally.
27490 Properties:
27492 • Config: client_secret
27494 • Env Var: RCLONE_DROPBOX_CLIENT_SECRET
27496 • Type: string
27498 • Required: false
27500 Advanced options
27502 Here are the Advanced options specific to dropbox (Dropbox).
27503 --dropbox-token
27505 OAuth Access Token as a JSON blob.
27506 Properties:
27508 • Config: token
27510 • Env Var: RCLONE_DROPBOX_TOKEN
27512 • Type: string
27514 • Required: false
27516 --dropbox-auth-url
27518 Auth server URL.
27519 Leave blank to use the provider defaults.
27521 Properties:
27523 • Config: auth_url
27525 • Env Var: RCLONE_DROPBOX_AUTH_URL
27527 • Type: string
27529 • Required: false
27531 --dropbox-token-url
27533 Token server url.
27534 Leave blank to use the provider defaults.
27536 Properties:
27538 • Config: token_url
27540 • Env Var: RCLONE_DROPBOX_TOKEN_URL
27542 • Type: string
27544 • Required: false
27546 --dropbox-chunk-size
27548 Upload chunk size (< 150Mi).
27549 Any files larger than this will be uploaded in chunks of this size.
27551 Note that chunks are buffered in memory (one at a time) so rclone can
27553 deal with retries. Setting this larger will increase the speed slight‐
27554 ly (at most 10% for 128 MiB in tests) at the cost of using more memory.
27555 It can be set smaller if you are tight on memory.
27556 Properties:
27558 • Config: chunk_size
27560 • Env Var: RCLONE_DROPBOX_CHUNK_SIZE
27562 • Type: SizeSuffix
27564 • Default: 48Mi
27566 --dropbox-impersonate
27568 Impersonate this user when using a business account.
27569 Note that if you want to use impersonate, you should make sure this
27571 flag is set when running "rclone config" as this will cause rclone to
27572 request the "members.read" scope which it won't normally. This is
27573 needed to lookup a members email address into the internal ID that
27574 dropbox uses in the API.
27575 Using the "members.read" scope will require a Dropbox Team Admin to ap‐
27577 prove during the OAuth flow.
27578 You will have to use your own App (setting your own client_id and
27580 client_secret) to use this option as currently rclone's default set of
27581 permissions doesn't include "members.read". This can be added once
27582 v1.55 or later is in use everywhere.
27583 Properties:
27585 • Config: impersonate
27587 • Env Var: RCLONE_DROPBOX_IMPERSONATE
27589 • Type: string
27591 • Required: false
27593 --dropbox-shared-files
27595 Instructs rclone to work on individual shared files.
27596 In this mode rclone's features are extremely limited - only list (ls,
27598 lsl, etc.) operations and read operations (e.g. downloading) are sup‐
27599 ported in this mode. All other operations will be disabled.
27600 Properties:
27602 • Config: shared_files
27604 • Env Var: RCLONE_DROPBOX_SHARED_FILES
27606 • Type: bool
27608 • Default: false
27610 --dropbox-shared-folders
27612 Instructs rclone to work on shared folders.
27613 When this flag is used with no path only the List operation is support‐
27615 ed and all available shared folders will be listed. If you specify a
27616 path the first part will be interpreted as the name of shared folder.
27617 Rclone will then try to mount this shared to the root namespace. On
27618 success shared folder rclone proceeds normally. The shared folder is
27619 now pretty much a normal folder and all normal operations are support‐
27620 ed.
27621 Note that we don't unmount the shared folder afterwards so the --drop‐
27623 box-shared-folders can be omitted after the first use of a particular
27624 shared folder.
27625 Properties:
27627 • Config: shared_folders
27629 • Env Var: RCLONE_DROPBOX_SHARED_FOLDERS
27631 • Type: bool
27633 • Default: false
27635 --dropbox-batch-mode
27637 Upload file batching sync|async|off.
27638 This sets the batch mode used by rclone.
27640 For full info see the main docs (https://rclone.org/dropbox/#batch-
27642 mode)
27643 This has 3 possible values
27645 • off - no batching
27647 • sync - batch uploads and check completion (default)
27649 • async - batch upload and don't check completion
27651 Rclone will close any outstanding batches when it exits which may make
27653 a delay on quit.
27654 Properties:
27656 • Config: batch_mode
27658 • Env Var: RCLONE_DROPBOX_BATCH_MODE
27660 • Type: string
27662 • Default: "sync"
27664 --dropbox-batch-size
27666 Max number of files in upload batch.
27667 This sets the batch size of files to upload. It has to be less than
27669 1000.
27670 By default this is 0 which means rclone which calculate the batch size
27672 depending on the setting of batch_mode.
27673 • batch_mode: async - default batch_size is 100
27675 • batch_mode: sync - default batch_size is the same as --transfers
27677 • batch_mode: off - not in use
27679 Rclone will close any outstanding batches when it exits which may make
27681 a delay on quit.
27682 Setting this is a great idea if you are uploading lots of small files
27684 as it will make them a lot quicker. You can use --transfers 32 to max‐
27685 imise throughput.
27686 Properties:
27688 • Config: batch_size
27690 • Env Var: RCLONE_DROPBOX_BATCH_SIZE
27692 • Type: int
27694 • Default: 0
27696 --dropbox-batch-timeout
27698 Max time to allow an idle upload batch before uploading.
27699 If an upload batch is idle for more than this long then it will be up‐
27701 loaded.
27702 The default for this is 0 which means rclone will choose a sensible de‐
27704 fault based on the batch_mode in use.
27705 • batch_mode: async - default batch_timeout is 500ms
27707 • batch_mode: sync - default batch_timeout is 10s
27709 • batch_mode: off - not in use
27711 Properties:
27713 • Config: batch_timeout
27715 • Env Var: RCLONE_DROPBOX_BATCH_TIMEOUT
27717 • Type: Duration
27719 • Default: 0s
27721 --dropbox-batch-commit-timeout
27723 Max time to wait for a batch to finish committing
27724 Properties:
27726 • Config: batch_commit_timeout
27728 • Env Var: RCLONE_DROPBOX_BATCH_COMMIT_TIMEOUT
27730 • Type: Duration
27732 • Default: 10m0s
27734 --dropbox-encoding
27736 The encoding for the backend.
27737 See the encoding section in the overview (https://rclone.org/over‐
27739 view/#encoding) for more info.
27740 Properties:
27742 • Config: encoding
27744 • Env Var: RCLONE_DROPBOX_ENCODING
27746 • Type: MultiEncoder
27748 • Default: Slash,BackSlash,Del,RightSpace,InvalidUtf8,Dot
27750 Limitations
27752 Note that Dropbox is case insensitive so you can't have a file called
27753 "Hello.doc" and one called "hello.doc".
27754 There are some file names such as thumbs.db which Dropbox can't store.
27756 There is a full list of them in the "Ignored Files" section of this
27757 document (https://www.dropbox.com/en/help/145). Rclone will issue an
27758 error message File name disallowed - not uploading if it attempts to
27759 upload one of those file names, but the sync won't fail.
27760 Some errors may occur if you try to sync copyright-protected files be‐
27762 cause Dropbox has its own copyright detector
27763 (https://techcrunch.com/2014/03/30/how-dropbox-knows-when-youre-shar‐
27764 ing-copyrighted-stuff-without-actually-looking-at-your-stuff/) that
27765 prevents this sort of file being downloaded. This will return the er‐
27766 ror ERROR : /path/to/your/file: Failed to copy: failed to open source
27767 object: path/restricted_content/.
27768 If you have more than 10,000 files in a directory then rclone purge
27770 dropbox:dir will return the error Failed to purge: There are too many
27771 files involved in this operation. As a work-around do an rclone delete
27772 dropbox:dir followed by an rclone rmdir dropbox:dir.
27773 When using rclone link you'll need to set --expire if using a non-per‐
27775 sonal account otherwise the visibility may not be correct. (Note that
27776 --expire isn't supported on personal accounts). See the forum discus‐
27777 sion (https://forum.rclone.org/t/rclone-link-dropbox-permissions/23211)
27778 and the dropbox SDK issue (https://github.com/dropbox/dropbox-sdk-go-
27779 unofficial/issues/75).
27780 Get your own Dropbox App ID
27782 When you use rclone with Dropbox in its default configuration you are
27783 using rclone's App ID. This is shared between all the rclone users.
27784 Here is how to create your own Dropbox App ID for rclone:
27786 1. Log into the Dropbox App console (https://www.dropbox.com/develop‐
27788 ers/apps/create) with your Dropbox Account (It need not to be the
27789 same account as the Dropbox you want to access)
27790 2. Choose an API => Usually this should be Dropbox API
27792 3. Choose the type of access you want to use => Full Dropbox or App
27794 Folder
27795 4. Name your App. The app name is global, so you can't use rclone for
27797 example
27798 5. Click the button Create App
27800 6. Switch to the Permissions tab. Enable at least the following per‐
27802 missions: account_info.read, files.metadata.write, files.con‐
27803 tent.write, files.content.read, sharing.write. The files.metada‐
27804 ta.read and sharing.read checkboxes will be marked too. Click Sub‐
27805 mit
27806 7. Switch to the Settings tab. Fill OAuth2 - Redirect URIs as
27808 http://localhost:53682/
27809 8. Find the App key and App secret values on the Settings tab. Use
27811 these values in rclone config to add a new remote or edit an exist‐
27812 ing remote. The App key setting corresponds to client_id in rclone
27813 config, the App secret corresponds to client_secret
27814
27816 This backend supports Storage Made Easy's Enterprise File Fabric™
27817 (https://storagemadeeasy.com/about/) which provides a software solution
27818 to integrate and unify File and Object Storage accessible through a
27819 global file system.
27820 Configuration
27822 The initial setup for the Enterprise File Fabric backend involves get‐
27823 ting a token from the Enterprise File Fabric which you need to do in
27824 your browser. rclone config walks you through it.
27825 Here is an example of how to make a remote called remote. First run:
27827 rclone config
27829 This will guide you through an interactive setup process:
27831 No remotes found, make a new one?
27833 n) New remote
27834 s) Set configuration password
27835 q) Quit config
27836 n/s/q> n
27837 name> remote
27838 Type of storage to configure.
27839 Enter a string value. Press Enter for the default ("").
27840 Choose a number from below, or type in your own value
27841 [snip]
27842 XX / Enterprise File Fabric
27843 \ "filefabric"
27844 [snip]
27845 Storage> filefabric
27846 ** See help for filefabric backend at: https://rclone.org/filefabric/ **
27847 URL of the Enterprise File Fabric to connect to
27849 Enter a string value. Press Enter for the default ("").
27850 Choose a number from below, or type in your own value
27851 1 / Storage Made Easy US
27852 \ "https://storagemadeeasy.com"
27853 2 / Storage Made Easy EU
27854 \ "https://eu.storagemadeeasy.com"
27855 3 / Connect to your Enterprise File Fabric
27856 \ "https://yourfabric.smestorage.com"
27857 url> https://yourfabric.smestorage.com/
27858 ID of the root folder
27859 Leave blank normally.
27860 Fill in to make rclone start with directory of a given ID.
27862 Enter a string value. Press Enter for the default ("").
27864 root_folder_id>
27865 Permanent Authentication Token
27866 A Permanent Authentication Token can be created in the Enterprise File
27868 Fabric, on the users Dashboard under Security, there is an entry
27869 you'll see called "My Authentication Tokens". Click the Manage button
27870 to create one.
27871 These tokens are normally valid for several years.
27873 For more info see: https://docs.storagemadeeasy.com/organisationcloud/api-tokens
27875 Enter a string value. Press Enter for the default ("").
27877 permanent_token> xxxxxxxxxxxxxxx-xxxxxxxxxxxxxxxx
27878 Edit advanced config? (y/n)
27879 y) Yes
27880 n) No (default)
27881 y/n> n
27882 Remote config
27883 --------------------
27884 [remote]
27885 type = filefabric
27886 url = https://yourfabric.smestorage.com/
27887 permanent_token = xxxxxxxxxxxxxxx-xxxxxxxxxxxxxxxx
27888 --------------------
27889 y) Yes this is OK (default)
27890 e) Edit this remote
27891 d) Delete this remote
27892 y/e/d> y
27893 Once configured you can then use rclone like this,
27895 List directories in top level of your Enterprise File Fabric
27897 rclone lsd remote:
27899 List all the files in your Enterprise File Fabric
27901 rclone ls remote:
27903 To copy a local directory to an Enterprise File Fabric directory called
27905 backup
27906 rclone copy /home/source remote:backup
27908 Modified time and hashes
27910 The Enterprise File Fabric allows modification times to be set on files
27911 accurate to 1 second. These will be used to detect whether objects
27912 need syncing or not.
27913 The Enterprise File Fabric does not support any data hashes at this
27915 time.
27916 Restricted filename characters
27918 The default restricted characters set (https://rclone.org/overview/#re‐
27919 stricted-characters) will be replaced.
27920 Invalid UTF-8 bytes will also be replaced (https://rclone.org/over‐
27922 view/#invalid-utf8), as they can't be used in JSON strings.
27923 Empty files
27925 Empty files aren't supported by the Enterprise File Fabric. Rclone
27926 will therefore upload an empty file as a single space with a mime type
27927 of application/vnd.rclone.empty.file and files with that mime type are
27928 treated as empty.
27929 Root folder ID
27931 You can set the root_folder_id for rclone. This is the directory
27932 (identified by its Folder ID) that rclone considers to be the root of
27933 your Enterprise File Fabric.
27934 Normally you will leave this blank and rclone will determine the cor‐
27936 rect root to use itself.
27937 However you can set this to restrict rclone to a specific folder hier‐
27939 archy.
27940 In order to do this you will have to find the Folder ID of the directo‐
27942 ry you wish rclone to display. These aren't displayed in the web in‐
27943 terface, but you can use rclone lsf to find them, for example
27944 $ rclone lsf --dirs-only -Fip --csv filefabric:
27946 120673758,Burnt PDFs/
27947 120673759,My Quick Uploads/
27948 120673755,My Syncs/
27949 120673756,My backups/
27950 120673757,My contacts/
27951 120673761,S3 Storage/
27952 The ID for "S3 Storage" would be 120673761.
27954 Standard options
27956 Here are the Standard options specific to filefabric (Enterprise File
27957 Fabric).
27958 --filefabric-url
27960 URL of the Enterprise File Fabric to connect to.
27961 Properties:
27963 • Config: url
27965 • Env Var: RCLONE_FILEFABRIC_URL
27967 • Type: string
27969 • Required: true
27971 • Examples:
27973 • "https://storagemadeeasy.com"
27975 • Storage Made Easy US
27977 • "https://eu.storagemadeeasy.com"
27979 • Storage Made Easy EU
27981 • "https://yourfabric.smestorage.com"
27983 • Connect to your Enterprise File Fabric
27985 --filefabric-root-folder-id
27987 ID of the root folder.
27988 Leave blank normally.
27990 Fill in to make rclone start with directory of a given ID.
27992 Properties:
27994 • Config: root_folder_id
27996 • Env Var: RCLONE_FILEFABRIC_ROOT_FOLDER_ID
27998 • Type: string
28000 • Required: false
28002 --filefabric-permanent-token
28004 Permanent Authentication Token.
28005 A Permanent Authentication Token can be created in the Enterprise File
28007 Fabric, on the users Dashboard under Security, there is an entry you'll
28008 see called "My Authentication Tokens". Click the Manage button to cre‐
28009 ate one.
28010 These tokens are normally valid for several years.
28012 For more info see: https://docs.storagemadeeasy.com/organisation‐
28014 cloud/api-tokens
28015 Properties:
28017 • Config: permanent_token
28019 • Env Var: RCLONE_FILEFABRIC_PERMANENT_TOKEN
28021 • Type: string
28023 • Required: false
28025 Advanced options
28027 Here are the Advanced options specific to filefabric (Enterprise File
28028 Fabric).
28029 --filefabric-token
28031 Session Token.
28032 This is a session token which rclone caches in the config file. It is
28034 usually valid for 1 hour.
28035 Don't set this value - rclone will set it automatically.
28037 Properties:
28039 • Config: token
28041 • Env Var: RCLONE_FILEFABRIC_TOKEN
28043 • Type: string
28045 • Required: false
28047 --filefabric-token-expiry
28049 Token expiry time.
28050 Don't set this value - rclone will set it automatically.
28052 Properties:
28054 • Config: token_expiry
28056 • Env Var: RCLONE_FILEFABRIC_TOKEN_EXPIRY
28058 • Type: string
28060 • Required: false
28062 --filefabric-version
28064 Version read from the file fabric.
28065 Don't set this value - rclone will set it automatically.
28067 Properties:
28069 • Config: version
28071 • Env Var: RCLONE_FILEFABRIC_VERSION
28073 • Type: string
28075 • Required: false
28077 --filefabric-encoding
28079 The encoding for the backend.
28080 See the encoding section in the overview (https://rclone.org/over‐
28082 view/#encoding) for more info.
28083 Properties:
28085 • Config: encoding
28087 • Env Var: RCLONE_FILEFABRIC_ENCODING
28089 • Type: MultiEncoder
28091 • Default: Slash,Del,Ctl,InvalidUtf8,Dot
28093
28095 FTP is the File Transfer Protocol. Rclone FTP support is provided us‐
28096 ing the github.com/jlaffaye/ftp (https://godoc.org/github.com/jlaf‐
28097 faye/ftp) package.
28098 Limitations of Rclone's FTP backend
28100 Paths are specified as remote:path. If the path does not begin with a
28102 / it is relative to the home directory of the user. An empty path re‐
28103 mote: refers to the user's home directory.
28104 Configuration
28106 To create an FTP configuration named remote, run
28107 rclone config
28109 Rclone config guides you through an interactive setup process. A mini‐
28111 mal rclone FTP remote definition only requires host, username and pass‐
28112 word. For an anonymous FTP server, see below.
28113 No remotes found, make a new one?
28115 n) New remote
28116 r) Rename remote
28117 c) Copy remote
28118 s) Set configuration password
28119 q) Quit config
28120 n/r/c/s/q> n
28121 name> remote
28122 Type of storage to configure.
28123 Enter a string value. Press Enter for the default ("").
28124 Choose a number from below, or type in your own value
28125 [snip]
28126 XX / FTP
28127 \ "ftp"
28128 [snip]
28129 Storage> ftp
28130 ** See help for ftp backend at: https://rclone.org/ftp/ **
28131 FTP host to connect to
28133 Enter a string value. Press Enter for the default ("").
28134 Choose a number from below, or type in your own value
28135 1 / Connect to ftp.example.com
28136 \ "ftp.example.com"
28137 host> ftp.example.com
28138 FTP username
28139 Enter a string value. Press Enter for the default ("$USER").
28140 user>
28141 FTP port number
28142 Enter a signed integer. Press Enter for the default (21).
28143 port>
28144 FTP password
28145 y) Yes type in my own password
28146 g) Generate random password
28147 y/g> y
28148 Enter the password:
28149 password:
28150 Confirm the password:
28151 password:
28152 Use FTP over TLS (Implicit)
28153 Enter a boolean value (true or false). Press Enter for the default ("false").
28154 tls>
28155 Use FTP over TLS (Explicit)
28156 Enter a boolean value (true or false). Press Enter for the default ("false").
28157 explicit_tls>
28158 Remote config
28159 --------------------
28160 [remote]
28161 type = ftp
28162 host = ftp.example.com
28163 pass = *** ENCRYPTED ***
28164 --------------------
28165 y) Yes this is OK
28166 e) Edit this remote
28167 d) Delete this remote
28168 y/e/d> y
28169 To see all directories in the home directory of remote
28171 rclone lsd remote:
28173 Make a new directory
28175 rclone mkdir remote:path/to/directory
28177 List the contents of a directory
28179 rclone ls remote:path/to/directory
28181 Sync /home/local/directory to the remote directory, deleting any excess
28183 files in the directory.
28184 rclone sync -i /home/local/directory remote:directory
28186 Anonymous FTP
28188 When connecting to a FTP server that allows anonymous login, you can
28189 use the special "anonymous" username. Traditionally, this user account
28190 accepts any string as a password, although it is common to use either
28191 the password "anonymous" or "guest". Some servers require the use of a
28192 valid e-mail address as password.
28193 Using on-the-fly or connection string (https://rclone.org/docs/#connec‐
28195 tion-strings) remotes makes it easy to access such servers, without re‐
28196 quiring any configuration in advance. The following are examples of
28197 that:
28198 rclone lsf :ftp: --ftp-host=speedtest.tele2.net --ftp-user=anonymous --ftp-pass=$(rclone obscure dummy)
28200 rclone lsf :ftp,host=speedtest.tele2.net,user=anonymous,pass=$(rclone obscure dummy):
28201 The above examples work in Linux shells and in PowerShell, but not Win‐
28203 dows Command Prompt. They execute the rclone obscure
28204 (https://rclone.org/commands/rclone_obscure/) command to create a pass‐
28205 word string in the format required by the pass option. The following
28206 examples are exactly the same, except use an already obscured string
28207 representation of the same password "dummy", and therefore works even
28208 in Windows Command Prompt:
28209 rclone lsf :ftp: --ftp-host=speedtest.tele2.net --ftp-user=anonymous --ftp-pass=IXs2wc8OJOz7SYLBk47Ji1rHTmxM
28211 rclone lsf :ftp,host=speedtest.tele2.net,user=anonymous,pass=IXs2wc8OJOz7SYLBk47Ji1rHTmxM:
28212 Implicit TLS
28214 Rlone FTP supports implicit FTP over TLS servers (FTPS). This has to
28215 be enabled in the FTP backend config for the remote, or with --ftp-tls.
28216 The default FTPS port is 990, not 21 and can be set with --ftp-port.
28217 Restricted filename characters
28219 In addition to the default restricted characters set
28220 (https://rclone.org/overview/#restricted-characters) the following
28221 characters are also replaced:
28222 File names cannot end with the following characters. Replacement is
28224 limited to the last character in a file name:
28225 Character Value Replacement
28227 ────────────────────────────────
28228 SP 0x20 ␠
28229 Not all FTP servers can have all characters in file names, for example:
28231 FTP Server Forbidden characters
28233 ──────────────────────────────────
28234 proftpd *
28235 pureftpd \ [ ]
28236 This backend's interactive configuration wizard provides a selection of
28238 sensible encoding settings for major FTP servers: ProFTPd, PureFTPd,
28239 VsFTPd. Just hit a selection number when prompted.
28240 Standard options
28242 Here are the Standard options specific to ftp (FTP).
28243 --ftp-host
28245 FTP host to connect to.
28246 E.g. "ftp.example.com".
28248 Properties:
28250 • Config: host
28252 • Env Var: RCLONE_FTP_HOST
28254 • Type: string
28256 • Required: true
28258 --ftp-user
28260 FTP username.
28261 Properties:
28263 • Config: user
28265 • Env Var: RCLONE_FTP_USER
28267 • Type: string
28269 • Default: "$USER"
28271 --ftp-port
28273 FTP port number.
28274 Properties:
28276 • Config: port
28278 • Env Var: RCLONE_FTP_PORT
28280 • Type: int
28282 • Default: 21
28284 --ftp-pass
28286 FTP password.
28287 NB Input to this must be obscured - see rclone obscure
28289 (https://rclone.org/commands/rclone_obscure/).
28290 Properties:
28292 • Config: pass
28294 • Env Var: RCLONE_FTP_PASS
28296 • Type: string
28298 • Required: false
28300 --ftp-tls
28302 Use Implicit FTPS (FTP over TLS).
28303 When using implicit FTP over TLS the client connects using TLS right
28305 from the start which breaks compatibility with non-TLS-aware servers.
28306 This is usually served over port 990 rather than port 21. Cannot be
28307 used in combination with explicit FTP.
28308 Properties:
28310 • Config: tls
28312 • Env Var: RCLONE_FTP_TLS
28314 • Type: bool
28316 • Default: false
28318 --ftp-explicit-tls
28320 Use Explicit FTPS (FTP over TLS).
28321 When using explicit FTP over TLS the client explicitly requests securi‐
28323 ty from the server in order to upgrade a plain text connection to an
28324 encrypted one. Cannot be used in combination with implicit FTP.
28325 Properties:
28327 • Config: explicit_tls
28329 • Env Var: RCLONE_FTP_EXPLICIT_TLS
28331 • Type: bool
28333 • Default: false
28335 Advanced options
28337 Here are the Advanced options specific to ftp (FTP).
28338 --ftp-concurrency
28340 Maximum number of FTP simultaneous connections, 0 for unlimited.
28341 Note that setting this is very likely to cause deadlocks so it should
28343 be used with care.
28344 If you are doing a sync or copy then make sure concurrency is one more
28346 than the sum of --transfers and --checkers.
28347 If you use --check-first then it just needs to be one more than the
28349 maximum of --checkers and --transfers.
28350 So for concurrency 3 you'd use --checkers 2 --transfers 2 --check-first
28352 or --checkers 1 --transfers 1.
28353 Properties:
28355 • Config: concurrency
28357 • Env Var: RCLONE_FTP_CONCURRENCY
28359 • Type: int
28361 • Default: 0
28363 --ftp-no-check-certificate
28365 Do not verify the TLS certificate of the server.
28366 Properties:
28368 • Config: no_check_certificate
28370 • Env Var: RCLONE_FTP_NO_CHECK_CERTIFICATE
28372 • Type: bool
28374 • Default: false
28376 --ftp-disable-epsv
28378 Disable using EPSV even if server advertises support.
28379 Properties:
28381 • Config: disable_epsv
28383 • Env Var: RCLONE_FTP_DISABLE_EPSV
28385 • Type: bool
28387 • Default: false
28389 --ftp-disable-mlsd
28391 Disable using MLSD even if server advertises support.
28392 Properties:
28394 • Config: disable_mlsd
28396 • Env Var: RCLONE_FTP_DISABLE_MLSD
28398 • Type: bool
28400 • Default: false
28402 --ftp-disable-utf8
28404 Disable using UTF-8 even if server advertises support.
28405 Properties:
28407 • Config: disable_utf8
28409 • Env Var: RCLONE_FTP_DISABLE_UTF8
28411 • Type: bool
28413 • Default: false
28415 --ftp-writing-mdtm
28417 Use MDTM to set modification time (VsFtpd quirk)
28418 Properties:
28420 • Config: writing_mdtm
28422 • Env Var: RCLONE_FTP_WRITING_MDTM
28424 • Type: bool
28426 • Default: false
28428 --ftp-force-list-hidden
28430 Use LIST -a to force listing of hidden files and folders. This will
28431 disable the use of MLSD.
28432 Properties:
28434 • Config: force_list_hidden
28436 • Env Var: RCLONE_FTP_FORCE_LIST_HIDDEN
28438 • Type: bool
28440 • Default: false
28442 --ftp-idle-timeout
28444 Max time before closing idle connections.
28445 If no connections have been returned to the connection pool in the time
28447 given, rclone will empty the connection pool.
28448 Set to 0 to keep connections indefinitely.
28450 Properties:
28452 • Config: idle_timeout
28454 • Env Var: RCLONE_FTP_IDLE_TIMEOUT
28456 • Type: Duration
28458 • Default: 1m0s
28460 --ftp-close-timeout
28462 Maximum time to wait for a response to close.
28463 Properties:
28465 • Config: close_timeout
28467 • Env Var: RCLONE_FTP_CLOSE_TIMEOUT
28469 • Type: Duration
28471 • Default: 1m0s
28473 --ftp-tls-cache-size
28475 Size of TLS session cache for all control and data connections.
28476 TLS cache allows to resume TLS sessions and reuse PSK between connec‐
28478 tions. Increase if default size is not enough resulting in TLS resump‐
28479 tion errors. Enabled by default. Use 0 to disable.
28480 Properties:
28482 • Config: tls_cache_size
28484 • Env Var: RCLONE_FTP_TLS_CACHE_SIZE
28486 • Type: int
28488 • Default: 32
28490 --ftp-disable-tls13
28492 Disable TLS 1.3 (workaround for FTP servers with buggy TLS)
28493 Properties:
28495 • Config: disable_tls13
28497 • Env Var: RCLONE_FTP_DISABLE_TLS13
28499 • Type: bool
28501 • Default: false
28503 --ftp-shut-timeout
28505 Maximum time to wait for data connection closing status.
28506 Properties:
28508 • Config: shut_timeout
28510 • Env Var: RCLONE_FTP_SHUT_TIMEOUT
28512 • Type: Duration
28514 • Default: 1m0s
28516 --ftp-ask-password
28518 Allow asking for FTP password when needed.
28519 If this is set and no password is supplied then rclone will ask for a
28521 password
28522 Properties:
28524 • Config: ask_password
28526 • Env Var: RCLONE_FTP_ASK_PASSWORD
28528 • Type: bool
28530 • Default: false
28532 --ftp-encoding
28534 The encoding for the backend.
28535 See the encoding section in the overview (https://rclone.org/over‐
28537 view/#encoding) for more info.
28538 Properties:
28540 • Config: encoding
28542 • Env Var: RCLONE_FTP_ENCODING
28544 • Type: MultiEncoder
28546 • Default: Slash,Del,Ctl,RightSpace,Dot
28548 • Examples:
28550 • "Asterisk,Ctl,Dot,Slash"
28552 • ProFTPd can't handle '*' in file names
28554 • "BackSlash,Ctl,Del,Dot,RightSpace,Slash,SquareBracket"
28556 • PureFTPd can't handle '[]' or '*' in file names
28558 • "Ctl,LeftPeriod,Slash"
28560 • VsFTPd can't handle file names starting with dot
28562 Limitations
28564 FTP servers acting as rclone remotes must support passive mode. The
28565 mode cannot be configured as passive is the only supported one.
28566 Rclone's FTP implementation is not compatible with active mode as the
28567 library it uses doesn't support it (https://github.com/jlaffaye/ftp/is‐
28568 sues/29). This will likely never be supported due to security con‐
28569 cerns.
28570 Rclone's FTP backend does not support any checksums but can compare
28572 file sizes.
28573 rclone about is not supported by the FTP backend. Backends without
28575 this capability cannot determine free space for an rclone mount or use
28576 policy mfs (most free space) as a member of an rclone union remote.
28577 See List of backends that do not support rclone about
28579 (https://rclone.org/overview/#optional-features) and rclone about
28580 (https://rclone.org/commands/rclone_about/)
28581 The implementation of : --dump headers, --dump bodies, --dump auth for
28583 debugging isn't the same as for rclone HTTP based backends - it has
28584 less fine grained control.
28585 --timeout isn't supported (but --contimeout is).
28587 --bind isn't supported.
28589 Rclone's FTP backend could support server-side move but does not at
28591 present.
28592 The ftp_proxy environment variable is not currently supported.
28594 Modified time
28596 File modification time (timestamps) is supported to 1 second resolution
28597 for major FTP servers: ProFTPd, PureFTPd, VsFTPd, and FileZilla FTP
28598 server. The VsFTPd server has non-standard implementation of time re‐
28599 lated protocol commands and needs a special configuration setting:
28600 writing_mdtm = true.
28601 Support for precise file time with other FTP servers varies depending
28603 on what protocol extensions they advertise. If all the MLSD, MDTM and
28604 MFTM extensions are present, rclone will use them together to provide
28605 precise time. Otherwise the times you see on the FTP server through
28606 rclone are those of the last file upload.
28607 You can use the following command to check whether rclone can use pre‐
28609 cise time with your FTP server: rclone backend features your_ftp_re‐
28610 mote: (the trailing colon is important). Look for the number in the
28611 line tagged by Precision designating the remote time precision ex‐
28612 pressed as nanoseconds. A value of 1000000000 means that file time
28613 precision of 1 second is available. A value of 3153600000000000000 (or
28614 another large number) means "unsupported".
28615
28617 Paths are specified as remote:bucket (or remote: for the lsd command.)
28618 You may put subdirectories in too, e.g. remote:bucket/path/to/dir.
28619 Configuration
28621 The initial setup for google cloud storage involves getting a token
28622 from Google Cloud Storage which you need to do in your browser. rclone
28623 config walks you through it.
28624 Here is an example of how to make a remote called remote. First run:
28626 rclone config
28628 This will guide you through an interactive setup process:
28630 n) New remote
28632 d) Delete remote
28633 q) Quit config
28634 e/n/d/q> n
28635 name> remote
28636 Type of storage to configure.
28637 Choose a number from below, or type in your own value
28638 [snip]
28639 XX / Google Cloud Storage (this is not Google Drive)
28640 \ "google cloud storage"
28641 [snip]
28642 Storage> google cloud storage
28643 Google Application Client Id - leave blank normally.
28644 client_id>
28645 Google Application Client Secret - leave blank normally.
28646 client_secret>
28647 Project number optional - needed only for list/create/delete buckets - see your developer console.
28648 project_number> 12345678
28649 Service Account Credentials JSON file path - needed only if you want use SA instead of interactive login.
28650 service_account_file>
28651 Access Control List for new objects.
28652 Choose a number from below, or type in your own value
28653 1 / Object owner gets OWNER access, and all Authenticated Users get READER access.
28654 \ "authenticatedRead"
28655 2 / Object owner gets OWNER access, and project team owners get OWNER access.
28656 \ "bucketOwnerFullControl"
28657 3 / Object owner gets OWNER access, and project team owners get READER access.
28658 \ "bucketOwnerRead"
28659 4 / Object owner gets OWNER access [default if left blank].
28660 \ "private"
28661 5 / Object owner gets OWNER access, and project team members get access according to their roles.
28662 \ "projectPrivate"
28663 6 / Object owner gets OWNER access, and all Users get READER access.
28664 \ "publicRead"
28665 object_acl> 4
28666 Access Control List for new buckets.
28667 Choose a number from below, or type in your own value
28668 1 / Project team owners get OWNER access, and all Authenticated Users get READER access.
28669 \ "authenticatedRead"
28670 2 / Project team owners get OWNER access [default if left blank].
28671 \ "private"
28672 3 / Project team members get access according to their roles.
28673 \ "projectPrivate"
28674 4 / Project team owners get OWNER access, and all Users get READER access.
28675 \ "publicRead"
28676 5 / Project team owners get OWNER access, and all Users get WRITER access.
28677 \ "publicReadWrite"
28678 bucket_acl> 2
28679 Location for the newly created buckets.
28680 Choose a number from below, or type in your own value
28681 1 / Empty for default location (US).
28682 \ ""
28683 2 / Multi-regional location for Asia.
28684 \ "asia"
28685 3 / Multi-regional location for Europe.
28686 \ "eu"
28687 4 / Multi-regional location for United States.
28688 \ "us"
28689 5 / Taiwan.
28690 \ "asia-east1"
28691 6 / Tokyo.
28692 \ "asia-northeast1"
28693 7 / Singapore.
28694 \ "asia-southeast1"
28695 8 / Sydney.
28696 \ "australia-southeast1"
28697 9 / Belgium.
28698 \ "europe-west1"
28699 10 / London.
28700 \ "europe-west2"
28701 11 / Iowa.
28702 \ "us-central1"
28703 12 / South Carolina.
28704 \ "us-east1"
28705 13 / Northern Virginia.
28706 \ "us-east4"
28707 14 / Oregon.
28708 \ "us-west1"
28709 location> 12
28710 The storage class to use when storing objects in Google Cloud Storage.
28711 Choose a number from below, or type in your own value
28712 1 / Default
28713 \ ""
28714 2 / Multi-regional storage class
28715 \ "MULTI_REGIONAL"
28716 3 / Regional storage class
28717 \ "REGIONAL"
28718 4 / Nearline storage class
28719 \ "NEARLINE"
28720 5 / Coldline storage class
28721 \ "COLDLINE"
28722 6 / Durable reduced availability storage class
28723 \ "DURABLE_REDUCED_AVAILABILITY"
28724 storage_class> 5
28725 Remote config
28726 Use auto config?
28727 * Say Y if not sure
28728 * Say N if you are working on a remote or headless machine or Y didn't work
28729 y) Yes
28730 n) No
28731 y/n> y
28732 If your browser doesn't open automatically go to the following link: http://127.0.0.1:53682/auth
28733 Log in and authorize rclone for access
28734 Waiting for code...
28735 Got code
28736 --------------------
28737 [remote]
28738 type = google cloud storage
28739 client_id =
28740 client_secret =
28741 token = {"AccessToken":"xxxx.xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx","RefreshToken":"x/xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx_xxxxxxxxx","Expiry":"2014-07-17T20:49:14.929208288+01:00","Extra":null}
28742 project_number = 12345678
28743 object_acl = private
28744 bucket_acl = private
28745 --------------------
28746 y) Yes this is OK
28747 e) Edit this remote
28748 d) Delete this remote
28749 y/e/d> y
28750 Note that rclone runs a webserver on your local machine to collect the
28752 token as returned from Google if you use auto config mode. This only
28753 runs from the moment it opens your browser to the moment you get back
28754 the verification code. This is on http://127.0.0.1:53682/ and this it
28755 may require you to unblock it temporarily if you are running a host
28756 firewall, or use manual mode.
28757 This remote is called remote and can now be used like this
28759 See all the buckets in your project
28761 rclone lsd remote:
28763 Make a new bucket
28765 rclone mkdir remote:bucket
28767 List the contents of a bucket
28769 rclone ls remote:bucket
28771 Sync /home/local/directory to the remote bucket, deleting any excess
28773 files in the bucket.
28774 rclone sync -i /home/local/directory remote:bucket
28776 Service Account support
28778 You can set up rclone with Google Cloud Storage in an unattended mode,
28779 i.e. not tied to a specific end-user Google account. This is useful
28780 when you want to synchronise files onto machines that don't have ac‐
28781 tively logged-in users, for example build machines.
28782 To get credentials for Google Cloud Platform IAM Service Accounts
28784 (https://cloud.google.com/iam/docs/service-accounts), please head to
28785 the Service Account (https://console.cloud.google.com/permissions/ser‐
28786 viceaccounts) section of the Google Developer Console. Service Ac‐
28787 counts behave just like normal User permissions in Google Cloud Storage
28788 ACLs (https://cloud.google.com/storage/docs/access-control), so you can
28789 limit their access (e.g. make them read only). After creating an ac‐
28790 count, a JSON file containing the Service Account's credentials will be
28791 downloaded onto your machines. These credentials are what rclone will
28792 use for authentication.
28793 To use a Service Account instead of OAuth2 token flow, enter the path
28795 to your Service Account credentials at the service_account_file prompt
28796 and rclone won't use the browser based authentication flow. If you'd
28797 rather stuff the contents of the credentials file into the rclone con‐
28798 fig file, you can set service_account_credentials with the actual con‐
28799 tents of the file instead, or set the equivalent environment variable.
28800 Anonymous Access
28802 For downloads of objects that permit public access you can configure
28803 rclone to use anonymous access by setting anonymous to true. With
28804 unauthorized access you can't write or create files but only read or
28805 list those buckets and objects that have public read access.
28806 Application Default Credentials
28808 If no other source of credentials is provided, rclone will fall back to
28809 Application Default Credentials (https://cloud.google.com/video-intel‐
28810 ligence/docs/common/auth#authenticating_with_application_default_cre‐
28811 dentials) this is useful both when you already have configured authen‐
28812 tication for your developer account, or in production when running on a
28813 google compute host. Note that if running in docker, you may need to
28814 run additional commands on your google compute machine - see this page
28815 (https://cloud.google.com/container-registry/docs/advanced-authentica‐
28816 tion#gcloud_as_a_docker_credential_helper).
28817 Note that in the case application default credentials are used, there
28819 is no need to explicitly configure a project number.
28820 --fast-list
28822 This remote supports --fast-list which allows you to use fewer transac‐
28823 tions in exchange for more memory. See the rclone docs
28824 (https://rclone.org/docs/#fast-list) for more details.
28825 Custom upload headers
28827 You can set custom upload headers with the --header-upload flag.
28828 Google Cloud Storage supports the headers as described in the working
28829 with metadata documentation (https://cloud.google.com/storage/docs/gsu‐
28830 til/addlhelp/WorkingWithObjectMetadata)
28831 • Cache-Control
28833 • Content-Disposition
28835 • Content-Encoding
28837 • Content-Language
28839 • Content-Type
28841 • X-Goog-Storage-Class
28843 • X-Goog-Meta-
28845 Eg --header-upload "Content-Type text/potato"
28847 Note that the last of these is for setting custom metadata in the form
28849 --header-upload "x-goog-meta-key: value"
28850 Modification time
28852 Google Cloud Storage stores md5sum natively. Google's gsutil
28853 (https://cloud.google.com/storage/docs/gsutil) tool stores modification
28854 time with one-second precision as goog-reserved-file-mtime in file
28855 metadata.
28856 To ensure compatibility with gsutil, rclone stores modification time in
28858 2 separate metadata entries. mtime uses RFC3339 format with one-
28859 nanosecond precision. goog-reserved-file-mtime uses Unix timestamp
28860 format with one-second precision. To get modification time from object
28861 metadata, rclone reads the metadata in the following order: mtime,
28862 goog-reserved-file-mtime, object updated time.
28863 Note that rclone's default modify window is 1ns. Files uploaded by
28865 gsutil only contain timestamps with one-second precision. If you use
28866 rclone to sync files previously uploaded by gsutil, rclone will attempt
28867 to update modification time for all these files. To avoid these possi‐
28868 bly unnecessary updates, use --modify-window 1s.
28869 Restricted filename characters
28871 Character Value Replacement
28872 ────────────────────────────────
28873 NUL 0x00 ␀
28874 LF 0x0A ␊
28875 CR 0x0D ␍
28876 / 0x2F ／
28877 Invalid UTF-8 bytes will also be replaced (https://rclone.org/over‐
28879 view/#invalid-utf8), as they can't be used in JSON strings.
28880 Standard options
28882 Here are the Standard options specific to google cloud storage (Google
28883 Cloud Storage (this is not Google Drive)).
28884 --gcs-client-id
28886 OAuth Client Id.
28887 Leave blank normally.
28889 Properties:
28891 • Config: client_id
28893 • Env Var: RCLONE_GCS_CLIENT_ID
28895 • Type: string
28897 • Required: false
28899 --gcs-client-secret
28901 OAuth Client Secret.
28902 Leave blank normally.
28904 Properties:
28906 • Config: client_secret
28908 • Env Var: RCLONE_GCS_CLIENT_SECRET
28910 • Type: string
28912 • Required: false
28914 --gcs-project-number
28916 Project number.
28917 Optional - needed only for list/create/delete buckets - see your devel‐
28919 oper console.
28920 Properties:
28922 • Config: project_number
28924 • Env Var: RCLONE_GCS_PROJECT_NUMBER
28926 • Type: string
28928 • Required: false
28930 --gcs-service-account-file
28932 Service Account Credentials JSON file path.
28933 Leave blank normally. Needed only if you want use SA instead of inter‐
28935 active login.
28936 Leading ~ will be expanded in the file name as will environment vari‐
28938 ables such as ${RCLONE_CONFIG_DIR}.
28939 Properties:
28941 • Config: service_account_file
28943 • Env Var: RCLONE_GCS_SERVICE_ACCOUNT_FILE
28945 • Type: string
28947 • Required: false
28949 --gcs-service-account-credentials
28951 Service Account Credentials JSON blob.
28952 Leave blank normally. Needed only if you want use SA instead of inter‐
28954 active login.
28955 Properties:
28957 • Config: service_account_credentials
28959 • Env Var: RCLONE_GCS_SERVICE_ACCOUNT_CREDENTIALS
28961 • Type: string
28963 • Required: false
28965 --gcs-anonymous
28967 Access public buckets and objects without credentials.
28968 Set to 'true' if you just want to download files and don't configure
28970 credentials.
28971 Properties:
28973 • Config: anonymous
28975 • Env Var: RCLONE_GCS_ANONYMOUS
28977 • Type: bool
28979 • Default: false
28981 --gcs-object-acl
28983 Access Control List for new objects.
28984 Properties:
28986 • Config: object_acl
28988 • Env Var: RCLONE_GCS_OBJECT_ACL
28990 • Type: string
28992 • Required: false
28994 • Examples:
28996 • "authenticatedRead"
28998 • Object owner gets OWNER access.
29000 • All Authenticated Users get READER access.
29002 • "bucketOwnerFullControl"
29004 • Object owner gets OWNER access.
29006 • Project team owners get OWNER access.
29008 • "bucketOwnerRead"
29010 • Object owner gets OWNER access.
29012 • Project team owners get READER access.
29014 • "private"
29016 • Object owner gets OWNER access.
29018 • Default if left blank.
29020 • "projectPrivate"
29022 • Object owner gets OWNER access.
29024 • Project team members get access according to their roles.
29026 • "publicRead"
29028 • Object owner gets OWNER access.
29030 • All Users get READER access.
29032 --gcs-bucket-acl
29034 Access Control List for new buckets.
29035 Properties:
29037 • Config: bucket_acl
29039 • Env Var: RCLONE_GCS_BUCKET_ACL
29041 • Type: string
29043 • Required: false
29045 • Examples:
29047 • "authenticatedRead"
29049 • Project team owners get OWNER access.
29051 • All Authenticated Users get READER access.
29053 • "private"
29055 • Project team owners get OWNER access.
29057 • Default if left blank.
29059 • "projectPrivate"
29061 • Project team members get access according to their roles.
29063 • "publicRead"
29065 • Project team owners get OWNER access.
29067 • All Users get READER access.
29069 • "publicReadWrite"
29071 • Project team owners get OWNER access.
29073 • All Users get WRITER access.
29075 --gcs-bucket-policy-only
29077 Access checks should use bucket-level IAM policies.
29078 If you want to upload objects to a bucket with Bucket Policy Only set
29080 then you will need to set this.
29081 When it is set, rclone:
29083 • ignores ACLs set on buckets
29085 • ignores ACLs set on objects
29087 • creates buckets with Bucket Policy Only set
29089 Docs: https://cloud.google.com/storage/docs/bucket-policy-only
29091 Properties:
29093 • Config: bucket_policy_only
29095 • Env Var: RCLONE_GCS_BUCKET_POLICY_ONLY
29097 • Type: bool
29099 • Default: false
29101 --gcs-location
29103 Location for the newly created buckets.
29104 Properties:
29106 • Config: location
29108 • Env Var: RCLONE_GCS_LOCATION
29110 • Type: string
29112 • Required: false
29114 • Examples:
29116 • ""
29118 • Empty for default location (US)
29120 • "asia"
29122 • Multi-regional location for Asia
29124 • "eu"
29126 • Multi-regional location for Europe
29128 • "us"
29130 • Multi-regional location for United States
29132 • "asia-east1"
29134 • Taiwan
29136 • "asia-east2"
29138 • Hong Kong
29140 • "asia-northeast1"
29142 • Tokyo
29144 • "asia-northeast2"
29146 • Osaka
29148 • "asia-northeast3"
29150 • Seoul
29152 • "asia-south1"
29154 • Mumbai
29156 • "asia-south2"
29158 • Delhi
29160 • "asia-southeast1"
29162 • Singapore
29164 • "asia-southeast2"
29166 • Jakarta
29168 • "australia-southeast1"
29170 • Sydney
29172 • "australia-southeast2"
29174 • Melbourne
29176 • "europe-north1"
29178 • Finland
29180 • "europe-west1"
29182 • Belgium
29184 • "europe-west2"
29186 • London
29188 • "europe-west3"
29190 • Frankfurt
29192 • "europe-west4"
29194 • Netherlands
29196 • "europe-west6"
29198 • Zürich
29200 • "europe-central2"
29202 • Warsaw
29204 • "us-central1"
29206 • Iowa
29208 • "us-east1"
29210 • South Carolina
29212 • "us-east4"
29214 • Northern Virginia
29216 • "us-west1"
29218 • Oregon
29220 • "us-west2"
29222 • California
29224 • "us-west3"
29226 • Salt Lake City
29228 • "us-west4"
29230 • Las Vegas
29232 • "northamerica-northeast1"
29234 • Montréal
29236 • "northamerica-northeast2"
29238 • Toronto
29240 • "southamerica-east1"
29242 • São Paulo
29244 • "southamerica-west1"
29246 • Santiago
29248 • "asia1"
29250 • Dual region: asia-northeast1 and asia-northeast2.
29252 • "eur4"
29254 • Dual region: europe-north1 and europe-west4.
29256 • "nam4"
29258 • Dual region: us-central1 and us-east1.
29260 --gcs-storage-class
29262 The storage class to use when storing objects in Google Cloud Storage.
29263 Properties:
29265 • Config: storage_class
29267 • Env Var: RCLONE_GCS_STORAGE_CLASS
29269 • Type: string
29271 • Required: false
29273 • Examples:
29275 • ""
29277 • Default
29279 • "MULTI_REGIONAL"
29281 • Multi-regional storage class
29283 • "REGIONAL"
29285 • Regional storage class
29287 • "NEARLINE"
29289 • Nearline storage class
29291 • "COLDLINE"
29293 • Coldline storage class
29295 • "ARCHIVE"
29297 • Archive storage class
29299 • "DURABLE_REDUCED_AVAILABILITY"
29301 • Durable reduced availability storage class
29303 Advanced options
29305 Here are the Advanced options specific to google cloud storage (Google
29306 Cloud Storage (this is not Google Drive)).
29307 --gcs-token
29309 OAuth Access Token as a JSON blob.
29310 Properties:
29312 • Config: token
29314 • Env Var: RCLONE_GCS_TOKEN
29316 • Type: string
29318 • Required: false
29320 --gcs-auth-url
29322 Auth server URL.
29323 Leave blank to use the provider defaults.
29325 Properties:
29327 • Config: auth_url
29329 • Env Var: RCLONE_GCS_AUTH_URL
29331 • Type: string
29333 • Required: false
29335 --gcs-token-url
29337 Token server url.
29338 Leave blank to use the provider defaults.
29340 Properties:
29342 • Config: token_url
29344 • Env Var: RCLONE_GCS_TOKEN_URL
29346 • Type: string
29348 • Required: false
29350 --gcs-no-check-bucket
29352 If set, don't attempt to check the bucket exists or create it.
29353 This can be useful when trying to minimise the number of transactions
29355 rclone does if you know the bucket exists already.
29356 Properties:
29358 • Config: no_check_bucket
29360 • Env Var: RCLONE_GCS_NO_CHECK_BUCKET
29362 • Type: bool
29364 • Default: false
29366 --gcs-decompress
29368 If set this will decompress gzip encoded objects.
29369 It is possible to upload objects to GCS with "Content-Encoding: gzip"
29371 set. Normally rclone will download these files as compressed objects.
29372 If this flag is set then rclone will decompress these files with "Con‐
29374 tent-Encoding: gzip" as they are received. This means that rclone
29375 can't check the size and hash but the file contents will be decom‐
29376 pressed.
29377 Properties:
29379 • Config: decompress
29381 • Env Var: RCLONE_GCS_DECOMPRESS
29383 • Type: bool
29385 • Default: false
29387 --gcs-endpoint
29389 Endpoint for the service.
29390 Leave blank normally.
29392 Properties:
29394 • Config: endpoint
29396 • Env Var: RCLONE_GCS_ENDPOINT
29398 • Type: string
29400 • Required: false
29402 --gcs-encoding
29404 The encoding for the backend.
29405 See the encoding section in the overview (https://rclone.org/over‐
29407 view/#encoding) for more info.
29408 Properties:
29410 • Config: encoding
29412 • Env Var: RCLONE_GCS_ENCODING
29414 • Type: MultiEncoder
29416 • Default: Slash,CrLf,InvalidUtf8,Dot
29418 Limitations
29420 rclone about is not supported by the Google Cloud Storage backend.
29421 Backends without this capability cannot determine free space for an
29422 rclone mount or use policy mfs (most free space) as a member of an
29423 rclone union remote.
29424 See List of backends that do not support rclone about
29426 (https://rclone.org/overview/#optional-features) and rclone about
29427 (https://rclone.org/commands/rclone_about/)
29428
29430 Paths are specified as drive:path
29431 Drive paths may be as deep as required, e.g. drive:directory/subdirec‐
29433 tory.
29434 Configuration
29436 The initial setup for drive involves getting a token from Google drive
29437 which you need to do in your browser. rclone config walks you through
29438 it.
29439 Here is an example of how to make a remote called remote. First run:
29441 rclone config
29443 This will guide you through an interactive setup process:
29445 No remotes found, make a new one?
29447 n) New remote
29448 r) Rename remote
29449 c) Copy remote
29450 s) Set configuration password
29451 q) Quit config
29452 n/r/c/s/q> n
29453 name> remote
29454 Type of storage to configure.
29455 Choose a number from below, or type in your own value
29456 [snip]
29457 XX / Google Drive
29458 \ "drive"
29459 [snip]
29460 Storage> drive
29461 Google Application Client Id - leave blank normally.
29462 client_id>
29463 Google Application Client Secret - leave blank normally.
29464 client_secret>
29465 Scope that rclone should use when requesting access from drive.
29466 Choose a number from below, or type in your own value
29467 1 / Full access all files, excluding Application Data Folder.
29468 \ "drive"
29469 2 / Read-only access to file metadata and file contents.
29470 \ "drive.readonly"
29471 / Access to files created by rclone only.
29472 3 | These are visible in the drive website.
29473 | File authorization is revoked when the user deauthorizes the app.
29474 \ "drive.file"
29475 / Allows read and write access to the Application Data folder.
29476 4 | This is not visible in the drive website.
29477 \ "drive.appfolder"
29478 / Allows read-only access to file metadata but
29479 5 | does not allow any access to read or download file content.
29480 \ "drive.metadata.readonly"
29481 scope> 1
29482 Service Account Credentials JSON file path - needed only if you want use SA instead of interactive login.
29483 service_account_file>
29484 Remote config
29485 Use auto config?
29486 * Say Y if not sure
29487 * Say N if you are working on a remote or headless machine or Y didn't work
29488 y) Yes
29489 n) No
29490 y/n> y
29491 If your browser doesn't open automatically go to the following link: http://127.0.0.1:53682/auth
29492 Log in and authorize rclone for access
29493 Waiting for code...
29494 Got code
29495 Configure this as a Shared Drive (Team Drive)?
29496 y) Yes
29497 n) No
29498 y/n> n
29499 --------------------
29500 [remote]
29501 client_id =
29502 client_secret =
29503 scope = drive
29504 root_folder_id =
29505 service_account_file =
29506 token = {"access_token":"XXX","token_type":"Bearer","refresh_token":"XXX","expiry":"2014-03-16T13:57:58.955387075Z"}
29507 --------------------
29508 y) Yes this is OK
29509 e) Edit this remote
29510 d) Delete this remote
29511 y/e/d> y
29512 Note that rclone runs a webserver on your local machine to collect the
29514 token as returned from Google if you use auto config mode. This only
29515 runs from the moment it opens your browser to the moment you get back
29516 the verification code. This is on http://127.0.0.1:53682/ and it may
29517 require you to unblock it temporarily if you are running a host fire‐
29518 wall, or use manual mode.
29519 You can then use it like this,
29521 List directories in top level of your drive
29523 rclone lsd remote:
29525 List all the files in your drive
29527 rclone ls remote:
29529 To copy a local directory to a drive directory called backup
29531 rclone copy /home/source remote:backup
29533 Scopes
29535 Rclone allows you to select which scope you would like for rclone to
29536 use. This changes what type of token is granted to rclone. The scopes
29537 are defined here (https://developers.google.com/drive/v3/web/about-
29538 auth).
29539 The scope are
29541 drive
29543 This is the default scope and allows full access to all files, except
29544 for the Application Data Folder (see below).
29545 Choose this one if you aren't sure.
29547 drive.readonly
29549 This allows read only access to all files. Files may be listed and
29550 downloaded but not uploaded, renamed or deleted.
29551 drive.file
29553 With this scope rclone can read/view/modify only those files and fold‐
29554 ers it creates.
29555 So if you uploaded files to drive via the web interface (or any other
29557 means) they will not be visible to rclone.
29558 This can be useful if you are using rclone to backup data and you want
29560 to be sure confidential data on your drive is not visible to rclone.
29561 Files created with this scope are visible in the web interface.
29563 drive.appfolder
29565 This gives rclone its own private area to store files. Rclone will not
29566 be able to see any other files on your drive and you won't be able to
29567 see rclone's files from the web interface either.
29568 drive.metadata.readonly
29570 This allows read only access to file names only. It does not allow
29571 rclone to download or upload data, or rename or delete files or direc‐
29572 tories.
29573 Root folder ID
29575 This option has been moved to the advanced section. You can set the
29576 root_folder_id for rclone. This is the directory (identified by its
29577 Folder ID) that rclone considers to be the root of your drive.
29578 Normally you will leave this blank and rclone will determine the cor‐
29580 rect root to use itself.
29581 However you can set this to restrict rclone to a specific folder hier‐
29583 archy or to access data within the "Computers" tab on the drive web in‐
29584 terface (where files from Google's Backup and Sync desktop program go).
29585 In order to do this you will have to find the Folder ID of the directo‐
29587 ry you wish rclone to display. This will be the last segment of the
29588 URL when you open the relevant folder in the drive web interface.
29589 So if the folder you want rclone to use has a URL which looks like
29591 https://drive.google.com/drive/fold‐
29592 ers/1XyfxxxxxxxxxxxxxxxxxxxxxxxxxKHCh in the browser, then you use
29593 1XyfxxxxxxxxxxxxxxxxxxxxxxxxxKHCh as the root_folder_id in the config.
29594 NB folders under the "Computers" tab seem to be read only (drive gives
29596 a 500 error) when using rclone.
29597 There doesn't appear to be an API to discover the folder IDs of the
29599 "Computers" tab - please contact us if you know otherwise!
29600 Note also that rclone can't access any data under the "Backups" tab on
29602 the google drive web interface yet.
29603 Service Account support
29605 You can set up rclone with Google Drive in an unattended mode, i.e.
29606 not tied to a specific end-user Google account. This is useful when
29607 you want to synchronise files onto machines that don't have actively
29608 logged-in users, for example build machines.
29609 To use a Service Account instead of OAuth2 token flow, enter the path
29611 to your Service Account credentials at the service_account_file prompt
29612 during rclone config and rclone won't use the browser based authentica‐
29613 tion flow. If you'd rather stuff the contents of the credentials file
29614 into the rclone config file, you can set service_account_credentials
29615 with the actual contents of the file instead, or set the equivalent en‐
29616 vironment variable.
29617 Use case - Google Apps/G-suite account and individual Drive
29619 Let's say that you are the administrator of a Google Apps (old) or G-
29620 suite account. The goal is to store data on an individual's Drive ac‐
29621 count, who IS a member of the domain. We'll call the domain exam‐
29622 ple.com, and the user foo@example.com.
29623 There's a few steps we need to go through to accomplish this:
29625 1. Create a service account for example.com
29627 • To create a service account and obtain its credentials, go to the
29628 Google Developer Console (https://console.developers.google.com).
29629 • You must have a project - create one if you don't.
29631 • Then go to "IAM & admin" -> "Service Accounts".
29633 • Use the "Create Credentials" button. Fill in "Service account name"
29635 with something that identifies your client. "Role" can be empty.
29636 • Tick "Furnish a new private key" - select "Key type JSON".
29638 • Tick "Enable G Suite Domain-wide Delegation". This option makes "im‐
29640 personation" possible, as documented here: Delegating domain-wide au‐
29641 thority to the service account (https://developers.google.com/identi‐
29642 ty/protocols/OAuth2ServiceAccount#delegatingauthority)
29643 • These credentials are what rclone will use for authentication. If
29645 you ever need to remove access, press the "Delete service account
29646 key" button.
29647 2. Allowing API access to example.com Google Drive
29649 • Go to example.com's admin console
29650 • Go into "Security" (or use the search bar)
29652 • Select "Show more" and then "Advanced settings"
29654 • Select "Manage API client access" in the "Authentication" section
29656 • In the "Client Name" field enter the service account's "Client ID" -
29658 this can be found in the Developer Console under "IAM & Admin" ->
29659 "Service Accounts", then "View Client ID" for the newly created ser‐
29660 vice account. It is a ~21 character numerical string.
29661 • In the next field, "One or More API Scopes", enter
29663 https://www.googleapis.com/auth/drive to grant access to Google Drive
29664 specifically.
29665 3. Configure rclone, assuming a new install
29667 rclone config
29668 n/s/q> n # New
29670 name>gdrive # Gdrive is an example name
29671 Storage> # Select the number shown for Google Drive
29672 client_id> # Can be left blank
29673 client_secret> # Can be left blank
29674 scope> # Select your scope, 1 for example
29675 root_folder_id> # Can be left blank
29676 service_account_file> /home/foo/myJSONfile.json # This is where the JSON file goes!
29677 y/n> # Auto config, n
29678 4. Verify that it's working
29680 • rclone -v --drive-impersonate foo@example.com lsf gdrive:backup
29681 • The arguments do:
29683 • -v - verbose logging
29685 • --drive-impersonate foo@example.com - this is what does the magic,
29687 pretending to be user foo.
29688 • lsf - list files in a parsing friendly way
29690 • gdrive:backup - use the remote called gdrive, work in the folder
29692 named backup.
29693 Note: in case you configured a specific root folder on gdrive and
29695 rclone is unable to access the contents of that folder when using
29696 --drive-impersonate, do this instead: - in the gdrive web interface,
29697 share your root folder with the user/email of the new Service Account
29698 you created/selected at step #1 - use rclone without specifying the
29699 --drive-impersonate option, like this: rclone -v lsf gdrive:backup
29700 Shared drives (team drives)
29702 If you want to configure the remote to point to a Google Shared Drive
29703 (previously known as Team Drives) then answer y to the question Config‐
29704 ure this as a Shared Drive (Team Drive)?.
29705 This will fetch the list of Shared Drives from google and allow you to
29707 configure which one you want to use. You can also type in a Shared
29708 Drive ID if you prefer.
29709 For example:
29711 Configure this as a Shared Drive (Team Drive)?
29713 y) Yes
29714 n) No
29715 y/n> y
29716 Fetching Shared Drive list...
29717 Choose a number from below, or type in your own value
29718 1 / Rclone Test
29719 \ "xxxxxxxxxxxxxxxxxxxx"
29720 2 / Rclone Test 2
29721 \ "yyyyyyyyyyyyyyyyyyyy"
29722 3 / Rclone Test 3
29723 \ "zzzzzzzzzzzzzzzzzzzz"
29724 Enter a Shared Drive ID> 1
29725 --------------------
29726 [remote]
29727 client_id =
29728 client_secret =
29729 token = {"AccessToken":"xxxx.x.xxxxx_xxxxxxxxxxx_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx","RefreshToken":"1/xxxxxxxxxxxxxxxx_xxxxxxxxxxxxxxxxxxxxxxxxxx","Expiry":"2014-03-16T13:57:58.955387075Z","Extra":null}
29730 team_drive = xxxxxxxxxxxxxxxxxxxx
29731 --------------------
29732 y) Yes this is OK
29733 e) Edit this remote
29734 d) Delete this remote
29735 y/e/d> y
29736 --fast-list
29738 This remote supports --fast-list which allows you to use fewer transac‐
29739 tions in exchange for more memory. See the rclone docs
29740 (https://rclone.org/docs/#fast-list) for more details.
29741 It does this by combining multiple list calls into a single API re‐
29743 quest.
29744 This works by combining many '%s' in parents filters into one expres‐
29746 sion. To list the contents of directories a, b and c, the following
29747 requests will be send by the regular List function:
29748 trashed=false and 'a' in parents
29750 trashed=false and 'b' in parents
29751 trashed=false and 'c' in parents
29752 These can now be combined into a single request:
29754 trashed=false and ('a' in parents or 'b' in parents or 'c' in parents)
29756 The implementation of ListR will put up to 50 parents filters into one
29758 request. It will use the --checkers value to specify the number of re‐
29759 quests to run in parallel.
29760 In tests, these batch requests were up to 20x faster than the regular
29762 method. Running the following command against different sized folders
29763 gives:
29764 rclone lsjson -vv -R --checkers=6 gdrive:folder
29766 small folder (220 directories, 700 files):
29768 • without --fast-list: 38s
29770 • with --fast-list: 10s
29772 large folder (10600 directories, 39000 files):
29774 • without --fast-list: 22:05 min
29776 • with --fast-list: 58s
29778 Modified time
29780 Google drive stores modification times accurate to 1 ms.
29781 Restricted filename characters
29783 Only Invalid UTF-8 bytes will be replaced (https://rclone.org/over‐
29784 view/#invalid-utf8), as they can't be used in JSON strings.
29785 In contrast to other backends, / can also be used in names and . or ..
29787 are valid names.
29788 Revisions
29790 Google drive stores revisions of files. When you upload a change to an
29791 existing file to google drive using rclone it will create a new revi‐
29792 sion of that file.
29793 Revisions follow the standard google policy which at time of writing
29795 was
29796 • They are deleted after 30 days or 100 revisions (whatever comes
29798 first).
29799 • They do not count towards a user storage quota.
29801 Deleting files
29803 By default rclone will send all files to the trash when deleting files.
29804 If deleting them permanently is required then use the --drive-use-
29805 trash=false flag, or set the equivalent environment variable.
29806 Shortcuts
29808 In March 2020 Google introduced a new feature in Google Drive called
29809 drive shortcuts (https://support.google.com/drive/answer/9700156) (API
29810 (https://developers.google.com/drive/api/v3/shortcuts)). These will
29811 (by September 2020) replace the ability for files or folders to be in
29812 multiple folders at once (https://cloud.google.com/blog/products/g-
29813 suite/simplifying-google-drives-folder-structure-and-sharing-models).
29814 Shortcuts are files that link to other files on Google Drive somewhat
29816 like a symlink in unix, except they point to the underlying file data
29817 (e.g. the inode in unix terms) so they don't break if the source is
29818 renamed or moved about.
29819 Be default rclone treats these as follows.
29821 For shortcuts pointing to files:
29823 • When listing a file shortcut appears as the destination file.
29825 • When downloading the contents of the destination file is downloaded.
29827 • When updating shortcut file with a non shortcut file, the shortcut is
29829 removed then a new file is uploaded in place of the shortcut.
29830 • When server-side moving (renaming) the shortcut is renamed, not the
29832 destination file.
29833 • When server-side copying the shortcut is copied, not the contents of
29835 the shortcut. (unless --drive-copy-shortcut-content is in use in
29836 which case the contents of the shortcut gets copied).
29837 • When deleting the shortcut is deleted not the linked file.
29839 • When setting the modification time, the modification time of the
29841 linked file will be set.
29842 For shortcuts pointing to folders:
29844 • When listing the shortcut appears as a folder and that folder will
29846 contain the contents of the linked folder appear (including any sub
29847 folders)
29848 • When downloading the contents of the linked folder and sub contents
29850 are downloaded
29851 • When uploading to a shortcut folder the file will be placed in the
29853 linked folder
29854 • When server-side moving (renaming) the shortcut is renamed, not the
29856 destination folder
29857 • When server-side copying the contents of the linked folder is copied,
29859 not the shortcut.
29860 • When deleting with rclone rmdir or rclone purge the shortcut is
29862 deleted not the linked folder.
29863 • NB When deleting with rclone remove or rclone mount the contents of
29865 the linked folder will be deleted.
29866 The rclone backend (https://rclone.org/commands/rclone_backend/) com‐
29868 mand can be used to create shortcuts.
29869 Shortcuts can be completely ignored with the --drive-skip-shortcuts
29871 flag or the corresponding skip_shortcuts configuration setting.
29872 Emptying trash
29874 If you wish to empty your trash you can use the rclone cleanup remote:
29875 command which will permanently delete all your trashed files. This
29876 command does not take any path arguments.
29877 Note that Google Drive takes some time (minutes to days) to empty the
29879 trash even though the command returns within a few seconds. No output
29880 is echoed, so there will be no confirmation even using -v or -vv.
29881 Quota information
29883 To view your current quota you can use the rclone about remote: command
29884 which will display your usage limit (quota), the usage in Google Drive,
29885 the size of all files in the Trash and the space used by other Google
29886 services such as Gmail. This command does not take any path arguments.
29887 Import/Export of google documents
29889 Google documents can be exported from and uploaded to Google Drive.
29890 When rclone downloads a Google doc it chooses a format to download de‐
29892 pending upon the --drive-export-formats setting. By default the export
29893 formats are docx,xlsx,pptx,svg which are a sensible default for an ed‐
29894 itable document.
29895 When choosing a format, rclone runs down the list provided in order and
29897 chooses the first file format the doc can be exported as from the list.
29898 If the file can't be exported to a format on the formats list, then
29899 rclone will choose a format from the default list.
29900 If you prefer an archive copy then you might use --drive-export-formats
29902 pdf, or if you prefer openoffice/libreoffice formats you might use
29903 --drive-export-formats ods,odt,odp.
29904 Note that rclone adds the extension to the google doc, so if it is
29906 called My Spreadsheet on google docs, it will be exported as My Spread‐
29907 sheet.xlsx or My Spreadsheet.pdf etc.
29908 When importing files into Google Drive, rclone will convert all files
29910 with an extension in --drive-import-formats to their associated docu‐
29911 ment type. rclone will not convert any files by default, since the
29912 conversion is lossy process.
29913 The conversion must result in a file with the same extension when the
29915 --drive-export-formats rules are applied to the uploaded document.
29916 Here are some examples for allowed and prohibited conversions.
29918 export-for‐ import-for‐ Upload Ext Document Ext Allowed
29920 mats mats
29921 ────────────────────────────────────────────────────────────────
29922 odt odt odt odt Yes
29923 odt docx,odt odt odt Yes
29924 docx docx docx Yes
29925 odt odt docx No
29926 odt,docx docx,odt docx odt No
29927 docx,odt docx,odt docx docx Yes
29928 docx,odt docx,odt odt docx No
29929 This limitation can be disabled by specifying --drive-allow-import-
29931 name-change. When using this flag, rclone can convert multiple files
29932 types resulting in the same document type at once, e.g. with --drive-
29933 import-formats docx,odt,txt, all files having these extension would re‐
29934 sult in a document represented as a docx file. This brings the addi‐
29935 tional risk of overwriting a document, if multiple files have the same
29936 stem. Many rclone operations will not handle this name change in any
29937 way. They assume an equal name when copying files and might copy the
29938 file again or delete them when the name changes.
29939 Here are the possible export extensions with their corresponding mime
29941 types. Most of these can also be used for importing, but there more
29942 that are not listed here. Some of these additional ones might only be
29943 available when the operating system provides the correct MIME type en‐
29944 tries.
29945 This list can be changed by Google Drive at any time and might not rep‐
29947 resent the currently available conversions.
29948 Extension Mime Type Description
29950 ─────────────────────────────────────────────────────────────────────────────
29951 bmp image/bmp Windows Bitmap format
29952 csv text/csv Standard CSV format for
29953 Spreadsheets
29954 doc application/msword Classic Word file
29955 docx application/vnd.openxml‐ Microsoft Office Document
29956 formats-officedocu‐
29957 ment.wordprocess‐
29958 ingml.document
29959 epub application/epub+zip E-book format
29960 html text/html An HTML Document
29961 jpg image/jpeg A JPEG Image File
29962 json application/vnd.google- JSON Text Format for
29963 apps.script+json Google Apps scripts
29964 odp application/vnd.oa‐ Openoffice Presentation
29965 sis.opendocument.presen‐
29966 tation
29967 ods application/vnd.oa‐ Openoffice Spreadsheet
29970 sis.opendocument.spread‐
29971 sheet
29972 ods application/x-vnd.oa‐ Openoffice Spreadsheet
29973 sis.opendocument.spread‐
29974 sheet
29975 odt application/vnd.oa‐ Openoffice Document
29976 sis.opendocument.text
29977 pdf application/pdf Adobe PDF Format
29978 pjpeg image/pjpeg Progressive JPEG Image
29979 png image/png PNG Image Format
29980 pptx application/vnd.openxml‐ Microsoft Office Power‐
29981 formats-officedocu‐ point
29982 ment.presentationml.pre‐
29983 sentation
29984 rtf application/rtf Rich Text Format
29985 svg image/svg+xml Scalable Vector Graphics
29986 Format
29987 tsv text/tab-separated-val‐ Standard TSV format for
29988 ues spreadsheets
29989 txt text/plain Plain Text
29990 wmf application/x-msmetafile Windows Meta File
29991 xls application/vnd.ms-excel Classic Excel file
29992 xlsx application/vnd.openxml‐ Microsoft Office Spread‐
29993 formats-officedocu‐ sheet
29994 ment.spreadsheetml.sheet
29995 zip application/zip A ZIP file of HTML, Images
29996 CSS
29997 Google documents can also be exported as link files. These files will
29999 open a browser window for the Google Docs website of that document when
30000 opened. The link file extension has to be specified as a --drive-ex‐
30001 port-formats parameter. They will match all available Google Docu‐
30002 ments.
30003 Extension Description OS Support
30005 ─────────────────────────────────────────────────
30006 desktop freedesktop.org Linux
30007 specified desktop
30008 entry
30009 link.html An HTML Document All
30010 with a redirect
30011 url INI style link file macOS, Windows
30012 webloc macOS specific XML macOS
30013 format
30014 Standard options
30016 Here are the Standard options specific to drive (Google Drive).
30017 --drive-client-id
30019 Google Application Client Id Setting your own is recommended. See
30020 https://rclone.org/drive/#making-your-own-client-id for how to create
30021 your own. If you leave this blank, it will use an internal key which
30022 is low performance.
30023 Properties:
30025 • Config: client_id
30027 • Env Var: RCLONE_DRIVE_CLIENT_ID
30029 • Type: string
30031 • Required: false
30033 --drive-client-secret
30035 OAuth Client Secret.
30036 Leave blank normally.
30038 Properties:
30040 • Config: client_secret
30042 • Env Var: RCLONE_DRIVE_CLIENT_SECRET
30044 • Type: string
30046 • Required: false
30048 --drive-scope
30050 Scope that rclone should use when requesting access from drive.
30051 Properties:
30053 • Config: scope
30055 • Env Var: RCLONE_DRIVE_SCOPE
30057 • Type: string
30059 • Required: false
30061 • Examples:
30063 • "drive"
30065 • Full access all files, excluding Application Data Folder.
30067 • "drive.readonly"
30069 • Read-only access to file metadata and file contents.
30071 • "drive.file"
30073 • Access to files created by rclone only.
30075 • These are visible in the drive website.
30077 • File authorization is revoked when the user deauthorizes the app.
30079 • "drive.appfolder"
30081 • Allows read and write access to the Application Data folder.
30083 • This is not visible in the drive website.
30085 • "drive.metadata.readonly"
30087 • Allows read-only access to file metadata but
30089 • does not allow any access to read or download file content.
30091 --drive-service-account-file
30093 Service Account Credentials JSON file path.
30094 Leave blank normally. Needed only if you want use SA instead of inter‐
30096 active login.
30097 Leading ~ will be expanded in the file name as will environment vari‐
30099 ables such as ${RCLONE_CONFIG_DIR}.
30100 Properties:
30102 • Config: service_account_file
30104 • Env Var: RCLONE_DRIVE_SERVICE_ACCOUNT_FILE
30106 • Type: string
30108 • Required: false
30110 --drive-alternate-export
30112 Deprecated: No longer needed.
30113 Properties:
30115 • Config: alternate_export
30117 • Env Var: RCLONE_DRIVE_ALTERNATE_EXPORT
30119 • Type: bool
30121 • Default: false
30123 Advanced options
30125 Here are the Advanced options specific to drive (Google Drive).
30126 --drive-token
30128 OAuth Access Token as a JSON blob.
30129 Properties:
30131 • Config: token
30133 • Env Var: RCLONE_DRIVE_TOKEN
30135 • Type: string
30137 • Required: false
30139 --drive-auth-url
30141 Auth server URL.
30142 Leave blank to use the provider defaults.
30144 Properties:
30146 • Config: auth_url
30148 • Env Var: RCLONE_DRIVE_AUTH_URL
30150 • Type: string
30152 • Required: false
30154 --drive-token-url
30156 Token server url.
30157 Leave blank to use the provider defaults.
30159 Properties:
30161 • Config: token_url
30163 • Env Var: RCLONE_DRIVE_TOKEN_URL
30165 • Type: string
30167 • Required: false
30169 --drive-root-folder-id
30171 ID of the root folder. Leave blank normally.
30172 Fill in to access "Computers" folders (see docs), or for rclone to use
30174 a non root folder as its starting point.
30175 Properties:
30177 • Config: root_folder_id
30179 • Env Var: RCLONE_DRIVE_ROOT_FOLDER_ID
30181 • Type: string
30183 • Required: false
30185 --drive-service-account-credentials
30187 Service Account Credentials JSON blob.
30188 Leave blank normally. Needed only if you want use SA instead of inter‐
30190 active login.
30191 Properties:
30193 • Config: service_account_credentials
30195 • Env Var: RCLONE_DRIVE_SERVICE_ACCOUNT_CREDENTIALS
30197 • Type: string
30199 • Required: false
30201 --drive-team-drive
30203 ID of the Shared Drive (Team Drive).
30204 Properties:
30206 • Config: team_drive
30208 • Env Var: RCLONE_DRIVE_TEAM_DRIVE
30210 • Type: string
30212 • Required: false
30214 --drive-auth-owner-only
30216 Only consider files owned by the authenticated user.
30217 Properties:
30219 • Config: auth_owner_only
30221 • Env Var: RCLONE_DRIVE_AUTH_OWNER_ONLY
30223 • Type: bool
30225 • Default: false
30227 --drive-use-trash
30229 Send files to the trash instead of deleting permanently.
30230 Defaults to true, namely sending files to the trash. Use --drive-use-
30232 trash=false to delete files permanently instead.
30233 Properties:
30235 • Config: use_trash
30237 • Env Var: RCLONE_DRIVE_USE_TRASH
30239 • Type: bool
30241 • Default: true
30243 --drive-copy-shortcut-content
30245 Server side copy contents of shortcuts instead of the shortcut.
30246 When doing server side copies, normally rclone will copy shortcuts as
30248 shortcuts.
30249 If this flag is used then rclone will copy the contents of shortcuts
30251 rather than shortcuts themselves when doing server side copies.
30252 Properties:
30254 • Config: copy_shortcut_content
30256 • Env Var: RCLONE_DRIVE_COPY_SHORTCUT_CONTENT
30258 • Type: bool
30260 • Default: false
30262 --drive-skip-gdocs
30264 Skip google documents in all listings.
30265 If given, gdocs practically become invisible to rclone.
30267 Properties:
30269 • Config: skip_gdocs
30271 • Env Var: RCLONE_DRIVE_SKIP_GDOCS
30273 • Type: bool
30275 • Default: false
30277 --drive-skip-checksum-gphotos
30279 Skip MD5 checksum on Google photos and videos only.
30280 Use this if you get checksum errors when transferring Google photos or
30282 videos.
30283 Setting this flag will cause Google photos and videos to return a blank
30285 MD5 checksum.
30286 Google photos are identified by being in the "photos" space.
30288 Corrupted checksums are caused by Google modifying the image/video but
30290 not updating the checksum.
30291 Properties:
30293 • Config: skip_checksum_gphotos
30295 • Env Var: RCLONE_DRIVE_SKIP_CHECKSUM_GPHOTOS
30297 • Type: bool
30299 • Default: false
30301 --drive-shared-with-me
30303 Only show files that are shared with me.
30304 Instructs rclone to operate on your "Shared with me" folder (where
30306 Google Drive lets you access the files and folders others have shared
30307 with you).
30308 This works both with the "list" (lsd, lsl, etc.) and the "copy" com‐
30310 mands (copy, sync, etc.), and with all other commands too.
30311 Properties:
30313 • Config: shared_with_me
30315 • Env Var: RCLONE_DRIVE_SHARED_WITH_ME
30317 • Type: bool
30319 • Default: false
30321 --drive-trashed-only
30323 Only show files that are in the trash.
30324 This will show trashed files in their original directory structure.
30326 Properties:
30328 • Config: trashed_only
30330 • Env Var: RCLONE_DRIVE_TRASHED_ONLY
30332 • Type: bool
30334 • Default: false
30336 --drive-starred-only
30338 Only show files that are starred.
30339 Properties:
30341 • Config: starred_only
30343 • Env Var: RCLONE_DRIVE_STARRED_ONLY
30345 • Type: bool
30347 • Default: false
30349 --drive-formats
30351 Deprecated: See export_formats.
30352 Properties:
30354 • Config: formats
30356 • Env Var: RCLONE_DRIVE_FORMATS
30358 • Type: string
30360 • Required: false
30362 --drive-export-formats
30364 Comma separated list of preferred formats for downloading Google docs.
30365 Properties:
30367 • Config: export_formats
30369 • Env Var: RCLONE_DRIVE_EXPORT_FORMATS
30371 • Type: string
30373 • Default: "docx,xlsx,pptx,svg"
30375 --drive-import-formats
30377 Comma separated list of preferred formats for uploading Google docs.
30378 Properties:
30380 • Config: import_formats
30382 • Env Var: RCLONE_DRIVE_IMPORT_FORMATS
30384 • Type: string
30386 • Required: false
30388 --drive-allow-import-name-change
30390 Allow the filetype to change when uploading Google docs.
30391 E.g. file.doc to file.docx. This will confuse sync and reupload every
30393 time.
30394 Properties:
30396 • Config: allow_import_name_change
30398 • Env Var: RCLONE_DRIVE_ALLOW_IMPORT_NAME_CHANGE
30400 • Type: bool
30402 • Default: false
30404 --drive-use-created-date
30406 Use file created date instead of modified date.
30407 Useful when downloading data and you want the creation date used in
30409 place of the last modified date.
30410 WARNING: This flag may have some unexpected consequences.
30412 When uploading to your drive all files will be overwritten unless they
30414 haven't been modified since their creation. And the inverse will occur
30415 while downloading. This side effect can be avoided by using the
30416 "--checksum" flag.
30417 This feature was implemented to retain photos capture date as recorded
30419 by google photos. You will first need to check the "Create a Google
30420 Photos folder" option in your google drive settings. You can then copy
30421 or move the photos locally and use the date the image was taken (creat‐
30422 ed) set as the modification date.
30423 Properties:
30425 • Config: use_created_date
30427 • Env Var: RCLONE_DRIVE_USE_CREATED_DATE
30429 • Type: bool
30431 • Default: false
30433 --drive-use-shared-date
30435 Use date file was shared instead of modified date.
30436 Note that, as with "--drive-use-created-date", this flag may have unex‐
30438 pected consequences when uploading/downloading files.
30439 If both this flag and "--drive-use-created-date" are set, the created
30441 date is used.
30442 Properties:
30444 • Config: use_shared_date
30446 • Env Var: RCLONE_DRIVE_USE_SHARED_DATE
30448 • Type: bool
30450 • Default: false
30452 --drive-list-chunk
30454 Size of listing chunk 100-1000, 0 to disable.
30455 Properties:
30457 • Config: list_chunk
30459 • Env Var: RCLONE_DRIVE_LIST_CHUNK
30461 • Type: int
30463 • Default: 1000
30465 --drive-impersonate
30467 Impersonate this user when using a service account.
30468 Properties:
30470 • Config: impersonate
30472 • Env Var: RCLONE_DRIVE_IMPERSONATE
30474 • Type: string
30476 • Required: false
30478 --drive-upload-cutoff
30480 Cutoff for switching to chunked upload.
30481 Properties:
30483 • Config: upload_cutoff
30485 • Env Var: RCLONE_DRIVE_UPLOAD_CUTOFF
30487 • Type: SizeSuffix
30489 • Default: 8Mi
30491 --drive-chunk-size
30493 Upload chunk size.
30494 Must a power of 2 >= 256k.
30496 Making this larger will improve performance, but note that each chunk
30498 is buffered in memory one per transfer.
30499 Reducing this will reduce memory usage but decrease performance.
30501 Properties:
30503 • Config: chunk_size
30505 • Env Var: RCLONE_DRIVE_CHUNK_SIZE
30507 • Type: SizeSuffix
30509 • Default: 8Mi
30511 --drive-acknowledge-abuse
30513 Set to allow files which return cannotDownloadAbusiveFile to be down‐
30514 loaded.
30515 If downloading a file returns the error "This file has been identified
30517 as malware or spam and cannot be downloaded" with the error code "can‐
30518 notDownloadAbusiveFile" then supply this flag to rclone to indicate you
30519 acknowledge the risks of downloading the file and rclone will download
30520 it anyway.
30521 Properties:
30523 • Config: acknowledge_abuse
30525 • Env Var: RCLONE_DRIVE_ACKNOWLEDGE_ABUSE
30527 • Type: bool
30529 • Default: false
30531 --drive-keep-revision-forever
30533 Keep new head revision of each file forever.
30534 Properties:
30536 • Config: keep_revision_forever
30538 • Env Var: RCLONE_DRIVE_KEEP_REVISION_FOREVER
30540 • Type: bool
30542 • Default: false
30544 --drive-size-as-quota
30546 Show sizes as storage quota usage, not actual size.
30547 Show the size of a file as the storage quota used. This is the current
30549 version plus any older versions that have been set to keep forever.
30550 WARNING: This flag may have some unexpected consequences.
30552 It is not recommended to set this flag in your config - the recommended
30554 usage is using the flag form --drive-size-as-quota when doing rclone
30555 ls/lsl/lsf/lsjson/etc only.
30556 If you do use this flag for syncing (not recommended) then you will
30558 need to use --ignore size also.
30559 Properties:
30561 • Config: size_as_quota
30563 • Env Var: RCLONE_DRIVE_SIZE_AS_QUOTA
30565 • Type: bool
30567 • Default: false
30569 --drive-v2-download-min-size
30571 If Object's are greater, use drive v2 API to download.
30572 Properties:
30574 • Config: v2_download_min_size
30576 • Env Var: RCLONE_DRIVE_V2_DOWNLOAD_MIN_SIZE
30578 • Type: SizeSuffix
30580 • Default: off
30582 --drive-pacer-min-sleep
30584 Minimum time to sleep between API calls.
30585 Properties:
30587 • Config: pacer_min_sleep
30589 • Env Var: RCLONE_DRIVE_PACER_MIN_SLEEP
30591 • Type: Duration
30593 • Default: 100ms
30595 --drive-pacer-burst
30597 Number of API calls to allow without sleeping.
30598 Properties:
30600 • Config: pacer_burst
30602 • Env Var: RCLONE_DRIVE_PACER_BURST
30604 • Type: int
30606 • Default: 100
30608 --drive-server-side-across-configs
30610 Allow server-side operations (e.g. copy) to work across different
30611 drive configs.
30612 This can be useful if you wish to do a server-side copy between two
30614 different Google drives. Note that this isn't enabled by default be‐
30615 cause it isn't easy to tell if it will work between any two configura‐
30616 tions.
30617 Properties:
30619 • Config: server_side_across_configs
30621 • Env Var: RCLONE_DRIVE_SERVER_SIDE_ACROSS_CONFIGS
30623 • Type: bool
30625 • Default: false
30627 --drive-disable-http2
30629 Disable drive using http2.
30630 There is currently an unsolved issue with the google drive backend and
30632 HTTP/2. HTTP/2 is therefore disabled by default for the drive backend
30633 but can be re-enabled here. When the issue is solved this flag will be
30634 removed.
30635 See: https://github.com/rclone/rclone/issues/3631
30637 Properties:
30639 • Config: disable_http2
30641 • Env Var: RCLONE_DRIVE_DISABLE_HTTP2
30643 • Type: bool
30645 • Default: true
30647 --drive-stop-on-upload-limit
30649 Make upload limit errors be fatal.
30650 At the time of writing it is only possible to upload 750 GiB of data to
30652 Google Drive a day (this is an undocumented limit). When this limit is
30653 reached Google Drive produces a slightly different error message. When
30654 this flag is set it causes these errors to be fatal. These will stop
30655 the in-progress sync.
30656 Note that this detection is relying on error message strings which
30658 Google don't document so it may break in the future.
30659 See: https://github.com/rclone/rclone/issues/3857
30661 Properties:
30663 • Config: stop_on_upload_limit
30665 • Env Var: RCLONE_DRIVE_STOP_ON_UPLOAD_LIMIT
30667 • Type: bool
30669 • Default: false
30671 --drive-stop-on-download-limit
30673 Make download limit errors be fatal.
30674 At the time of writing it is only possible to download 10 TiB of data
30676 from Google Drive a day (this is an undocumented limit). When this
30677 limit is reached Google Drive produces a slightly different error mes‐
30678 sage. When this flag is set it causes these errors to be fatal. These
30679 will stop the in-progress sync.
30680 Note that this detection is relying on error message strings which
30682 Google don't document so it may break in the future.
30683 Properties:
30685 • Config: stop_on_download_limit
30687 • Env Var: RCLONE_DRIVE_STOP_ON_DOWNLOAD_LIMIT
30689 • Type: bool
30691 • Default: false
30693 --drive-skip-shortcuts
30695 If set skip shortcut files.
30696 Normally rclone dereferences shortcut files making them appear as if
30698 they are the original file (see the shortcuts section). If this flag
30699 is set then rclone will ignore shortcut files completely.
30700 Properties:
30702 • Config: skip_shortcuts
30704 • Env Var: RCLONE_DRIVE_SKIP_SHORTCUTS
30706 • Type: bool
30708 • Default: false
30710 --drive-skip-dangling-shortcuts
30712 If set skip dangling shortcut files.
30713 If this is set then rclone will not show any dangling shortcuts in
30715 listings.
30716 Properties:
30718 • Config: skip_dangling_shortcuts
30720 • Env Var: RCLONE_DRIVE_SKIP_DANGLING_SHORTCUTS
30722 • Type: bool
30724 • Default: false
30726 --drive-resource-key
30728 Resource key for accessing a link-shared file.
30729 If you need to access files shared with a link like this
30731 https://drive.google.com/drive/folders/XXX?resourcekey=YYY&usp=sharing
30733 Then you will need to use the first part "XXX" as the "root_folder_id"
30735 and the second part "YYY" as the "resource_key" otherwise you will get
30736 404 not found errors when trying to access the directory.
30737 See: https://developers.google.com/drive/api/guides/resource-keys
30739 This resource key requirement only applies to a subset of old files.
30741 Note also that opening the folder once in the web interface (with the
30743 user you've authenticated rclone with) seems to be enough so that the
30744 resource key is no needed.
30745 Properties:
30747 • Config: resource_key
30749 • Env Var: RCLONE_DRIVE_RESOURCE_KEY
30751 • Type: string
30753 • Required: false
30755 --drive-encoding
30757 The encoding for the backend.
30758 See the encoding section in the overview (https://rclone.org/over‐
30760 view/#encoding) for more info.
30761 Properties:
30763 • Config: encoding
30765 • Env Var: RCLONE_DRIVE_ENCODING
30767 • Type: MultiEncoder
30769 • Default: InvalidUtf8
30771 Backend commands
30773 Here are the commands specific to the drive backend.
30774 Run them with
30776 rclone backend COMMAND remote:
30778 The help below will explain what arguments each command takes.
30780 See the backend (https://rclone.org/commands/rclone_backend/) command
30782 for more info on how to pass options and arguments.
30783 These can be run on a running backend using the rc command backend/com‐
30785 mand (https://rclone.org/rc/#backend-command).
30786 get
30788 Get command for fetching the drive config parameters
30789 rclone backend get remote: [options] [<arguments>+]
30791 This is a get command which will be used to fetch the various drive
30793 config parameters
30794 Usage Examples:
30796 rclone backend get drive: [-o service_account_file] [-o chunk_size]
30798 rclone rc backend/command command=get fs=drive: [-o service_account_file] [-o chunk_size]
30799 Options:
30801 • "chunk_size": show the current upload chunk size
30803 • "service_account_file": show the current service account file
30805 set
30807 Set command for updating the drive config parameters
30808 rclone backend set remote: [options] [<arguments>+]
30810 This is a set command which will be used to update the various drive
30812 config parameters
30813 Usage Examples:
30815 rclone backend set drive: [-o service_account_file=sa.json] [-o chunk_size=67108864]
30817 rclone rc backend/command command=set fs=drive: [-o service_account_file=sa.json] [-o chunk_size=67108864]
30818 Options:
30820 • "chunk_size": update the current upload chunk size
30822 • "service_account_file": update the current service account file
30824 shortcut
30826 Create shortcuts from files or directories
30827 rclone backend shortcut remote: [options] [<arguments>+]
30829 This command creates shortcuts from files or directories.
30831 Usage:
30833 rclone backend shortcut drive: source_item destination_shortcut
30835 rclone backend shortcut drive: source_item -o target=drive2: destination_shortcut
30836 In the first example this creates a shortcut from the "source_item"
30838 which can be a file or a directory to the "destination_shortcut". The
30839 "source_item" and the "destination_shortcut" should be relative paths
30840 from "drive:"
30841 In the second example this creates a shortcut from the "source_item"
30843 relative to "drive:" to the "destination_shortcut" relative to
30844 "drive2:". This may fail with a permission error if the user authenti‐
30845 cated with "drive2:" can't read files from "drive:".
30846 Options:
30848 • "target": optional target remote for the shortcut destination
30850 drives
30852 List the Shared Drives available to this account
30853 rclone backend drives remote: [options] [<arguments>+]
30855 This command lists the Shared Drives (Team Drives) available to this
30857 account.
30858 Usage:
30860 rclone backend [-o config] drives drive:
30862 This will return a JSON list of objects like this
30864 [
30866 {
30867 "id": "0ABCDEF-01234567890",
30868 "kind": "drive#teamDrive",
30869 "name": "My Drive"
30870 },
30871 {
30872 "id": "0ABCDEFabcdefghijkl",
30873 "kind": "drive#teamDrive",
30874 "name": "Test Drive"
30875 }
30876 ]
30877 With the -o config parameter it will output the list in a format suit‐
30879 able for adding to a config file to make aliases for all the drives
30880 found and a combined drive.
30881 [My Drive]
30883 type = alias
30884 remote = drive,team_drive=0ABCDEF-01234567890,root_folder_id=:
30885 [Test Drive]
30887 type = alias
30888 remote = drive,team_drive=0ABCDEFabcdefghijkl,root_folder_id=:
30889 [AllDrives]
30891 type = combine
30892 upstreams = "My Drive=My Drive:" "Test Drive=Test Drive:"
30893 Adding this to the rclone config file will cause those team drives to
30895 be accessible with the aliases shown. Any illegal characters will be
30896 substituted with "_" and duplicate names will have numbers suffixed.
30897 It will also add a remote called AllDrives which shows all the shared
30898 drives combined into one directory tree.
30899 untrash
30901 Untrash files and directories
30902 rclone backend untrash remote: [options] [<arguments>+]
30904 This command untrashes all the files and directories in the directory
30906 passed in recursively.
30907 Usage:
30909 This takes an optional directory to trash which make this easier to use
30911 via the API.
30912 rclone backend untrash drive:directory
30914 rclone backend -i untrash drive:directory subdir
30915 Use the -i flag to see what would be restored before restoring it.
30917 Result:
30919 {
30921 "Untrashed": 17,
30922 "Errors": 0
30923 }
30924 copyid
30926 Copy files by ID
30927 rclone backend copyid remote: [options] [<arguments>+]
30929 This command copies files by ID
30931 Usage:
30933 rclone backend copyid drive: ID path
30935 rclone backend copyid drive: ID1 path1 ID2 path2
30936 It copies the drive file with ID given to the path (an rclone path
30938 which will be passed internally to rclone copyto). The ID and path
30939 pairs can be repeated.
30940 The path should end with a / to indicate copy the file as named to this
30942 directory. If it doesn't end with a / then the last path component
30943 will be used as the file name.
30944 If the destination is a drive backend then server-side copying will be
30946 attempted if possible.
30947 Use the -i flag to see what would be copied before copying.
30949 exportformats
30951 Dump the export formats for debug purposes
30952 rclone backend exportformats remote: [options] [<arguments>+]
30954 importformats
30956 Dump the import formats for debug purposes
30957 rclone backend importformats remote: [options] [<arguments>+]
30959 Limitations
30961 Drive has quite a lot of rate limiting. This causes rclone to be lim‐
30962 ited to transferring about 2 files per second only. Individual files
30963 may be transferred much faster at 100s of MiB/s but lots of small files
30964 can take a long time.
30965 Server side copies are also subject to a separate rate limit. If you
30967 see User rate limit exceeded errors, wait at least 24 hours and retry.
30968 You can disable server-side copies with --disable copy to download and
30969 upload the files if you prefer.
30970 Limitations of Google Docs
30972 Google docs will appear as size -1 in rclone ls, rclone ncdu etc, and
30973 as size 0 in anything which uses the VFS layer, e.g. rclone mount and
30974 rclone serve. When calculating directory totals, e.g. in rclone size
30975 and rclone ncdu, they will be counted in as empty files.
30976 This is because rclone can't find out the size of the Google docs with‐
30978 out downloading them.
30979 Google docs will transfer correctly with rclone sync, rclone copy etc
30981 as rclone knows to ignore the size when doing the transfer.
30982 However an unfortunate consequence of this is that you may not be able
30984 to download Google docs using rclone mount. If it doesn't work you
30985 will get a 0 sized file. If you try again the doc may gain its correct
30986 size and be downloadable. Whether it will work on not depends on the
30987 application accessing the mount and the OS you are running - experiment
30988 to find out if it does work for you!
30989 Duplicated files
30991 Sometimes, for no reason I've been able to track down, drive will du‐
30992 plicate a file that rclone uploads. Drive unlike all the other remotes
30993 can have duplicated files.
30994 Duplicated files cause problems with the syncing and you will see mes‐
30996 sages in the log about duplicates.
30997 Use rclone dedupe to fix duplicated files.
30999 Note that this isn't just a problem with rclone, even Google Photos on
31001 Android duplicates files on drive sometimes.
31002 Rclone appears to be re-copying files it shouldn't
31004 The most likely cause of this is the duplicated file issue above - run
31005 rclone dedupe and check your logs for duplicate object or directory
31006 messages.
31007 This can also be caused by a delay/caching on google drive's end when
31009 comparing directory listings. Specifically with team drives used in
31010 combination with --fast-list. Files that were uploaded recently may
31011 not appear on the directory list sent to rclone when using --fast-list.
31012 Waiting a moderate period of time between attempts (estimated to be ap‐
31014 proximately 1 hour) and/or not using --fast-list both seem to be effec‐
31015 tive in preventing the problem.
31016 Making your own client_id
31018 When you use rclone with Google drive in its default configuration you
31019 are using rclone's client_id. This is shared between all the rclone
31020 users. There is a global rate limit on the number of queries per sec‐
31021 ond that each client_id can do set by Google. rclone already has a
31022 high quota and I will continue to make sure it is high enough by con‐
31023 tacting Google.
31024 It is strongly recommended to use your own client ID as the default
31026 rclone ID is heavily used. If you have multiple services running, it
31027 is recommended to use an API key for each service. The default Google
31028 quota is 10 transactions per second so it is recommended to stay under
31029 that number as if you use more than that, it will cause rclone to rate
31030 limit and make things slower.
31031 Here is how to create your own Google Drive client ID for rclone:
31033 1. Log into the Google API Console (https://console.develop‐
31035 ers.google.com/) with your Google account. It doesn't matter what
31036 Google account you use. (It need not be the same account as the
31037 Google Drive you want to access)
31038 2. Select a project or create a new project.
31040 3. Under "ENABLE APIS AND SERVICES" search for "Drive", and enable the
31042 "Google Drive API".
31043 4. Click "Credentials" in the left-side panel (not "Create creden‐
31045 tials", which opens the wizard), then "Create credentials"
31046 5. If you already configured an "Oauth Consent Screen", then skip to
31048 the next step; if not, click on "CONFIGURE CONSENT SCREEN" button
31049 (near the top right corner of the right panel), then select "Exter‐
31050 nal" and click on "CREATE"; on the next screen, enter an "Applica‐
31051 tion name" ("rclone" is OK); enter "User Support Email" (your own
31052 email is OK); enter "Developer Contact Email" (your own email is
31053 OK); then click on "Save" (all other data is optional). Click
31054 again on "Credentials" on the left panel to go back to the "Creden‐
31055 tials" screen.
31056 (PS: if you are a GSuite user, you could also select "Internal" in‐
31058 stead of "External" above, but this will restrict API use to Google
31059 Workspace users in your organisation).
31060 6. Click on the "+ CREATE CREDENTIALS" button at the top of the
31062 screen, then select "OAuth client ID".
31063 7. Choose an application type of "Desktop app" and click "Create".
31065 (the default name is fine)
31066 8. It will show you a client ID and client secret. Make a note of
31068 these.
31069 (If you selected "External" at Step 5 continue to "Publish App" in
31071 the Steps 9 and 10. If you chose "Internal" you don't need to pub‐
31072 lish and can skip straight to Step 11.)
31073 9. Go to "Oauth consent screen" and press "Publish App"
31075 10. Click "OAuth consent screen", then click "PUBLISH APP" button and
31077 confirm, or add your account under "Test users".
31078 11. Provide the noted client ID and client secret to rclone.
31080 Be aware that, due to the "enhanced security" recently introduced by
31082 Google, you are theoretically expected to "submit your app for verifi‐
31083 cation" and then wait a few weeks(!) for their response; in practice,
31084 you can go right ahead and use the client ID and client secret with
31085 rclone, the only issue will be a very scary confirmation screen shown
31086 when you connect via your browser for rclone to be able to get its to‐
31087 ken-id (but as this only happens during the remote configuration, it's
31088 not such a big deal).
31089 (Thanks to @balazer on github for these instructions.)
31091 Sometimes, creation of an OAuth consent in Google API Console fails due
31093 to an error message “The request failed because changes to one of the
31094 field of the resource is not supported”. As a convenient workaround,
31095 the necessary Google Drive API key can be created on the Python Quick‐
31096 start (https://developers.google.com/drive/api/v3/quickstart/python)
31097 page. Just push the Enable the Drive API button to receive the Client
31098 ID and Secret. Note that it will automatically create a new project in
31099 the API Console.
31100
31102 The rclone backend for Google Photos (https://www.google.com/pho‐
31103 tos/about/) is a specialized backend for transferring photos and videos
31104 to and from Google Photos.
31105 NB The Google Photos API which rclone uses has quite a few limitations,
31107 so please read the limitations section carefully to make sure it is
31108 suitable for your use.
31109 Configuration
31111 The initial setup for google cloud storage involves getting a token
31112 from Google Photos which you need to do in your browser. rclone config
31113 walks you through it.
31114 Here is an example of how to make a remote called remote. First run:
31116 rclone config
31118 This will guide you through an interactive setup process:
31120 No remotes found, make a new one?
31122 n) New remote
31123 s) Set configuration password
31124 q) Quit config
31125 n/s/q> n
31126 name> remote
31127 Type of storage to configure.
31128 Enter a string value. Press Enter for the default ("").
31129 Choose a number from below, or type in your own value
31130 [snip]
31131 XX / Google Photos
31132 \ "google photos"
31133 [snip]
31134 Storage> google photos
31135 ** See help for google photos backend at: https://rclone.org/googlephotos/ **
31136 Google Application Client Id
31138 Leave blank normally.
31139 Enter a string value. Press Enter for the default ("").
31140 client_id>
31141 Google Application Client Secret
31142 Leave blank normally.
31143 Enter a string value. Press Enter for the default ("").
31144 client_secret>
31145 Set to make the Google Photos backend read only.
31146 If you choose read only then rclone will only request read only access
31148 to your photos, otherwise rclone will request full access.
31149 Enter a boolean value (true or false). Press Enter for the default ("false").
31150 read_only>
31151 Edit advanced config? (y/n)
31152 y) Yes
31153 n) No
31154 y/n> n
31155 Remote config
31156 Use auto config?
31157 * Say Y if not sure
31158 * Say N if you are working on a remote or headless machine
31159 y) Yes
31160 n) No
31161 y/n> y
31162 If your browser doesn't open automatically go to the following link: http://127.0.0.1:53682/auth
31163 Log in and authorize rclone for access
31164 Waiting for code...
31165 Got code
31166 *** IMPORTANT: All media items uploaded to Google Photos with rclone
31168 *** are stored in full resolution at original quality. These uploads
31169 *** will count towards storage in your Google Account.
31170 --------------------
31172 [remote]
31173 type = google photos
31174 token = {"access_token":"XXX","token_type":"Bearer","refresh_token":"XXX","expiry":"2019-06-28T17:38:04.644930156+01:00"}
31175 --------------------
31176 y) Yes this is OK
31177 e) Edit this remote
31178 d) Delete this remote
31179 y/e/d> y
31180 Note that rclone runs a webserver on your local machine to collect the
31182 token as returned from Google if you use auto config mode. This only
31183 runs from the moment it opens your browser to the moment you get back
31184 the verification code. This is on http://127.0.0.1:53682/ and this may
31185 require you to unblock it temporarily if you are running a host fire‐
31186 wall, or use manual mode.
31187 This remote is called remote and can now be used like this
31189 See all the albums in your photos
31191 rclone lsd remote:album
31193 Make a new album
31195 rclone mkdir remote:album/newAlbum
31197 List the contents of an album
31199 rclone ls remote:album/newAlbum
31201 Sync /home/local/images to the Google Photos, removing any excess files
31203 in the album.
31204 rclone sync -i /home/local/image remote:album/newAlbum
31206 Layout
31208 As Google Photos is not a general purpose cloud storage system the
31209 backend is laid out to help you navigate it.
31210 The directories under media show different ways of categorizing the me‐
31212 dia. Each file will appear multiple times. So if you want to make a
31213 backup of your google photos you might choose to backup remote:me‐
31214 dia/by-month. (NB remote:media/by-day is rather slow at the moment so
31215 avoid for syncing.)
31216 Note that all your photos and videos will appear somewhere under media,
31218 but they may not appear under album unless you've put them into albums.
31219 /
31221 - upload
31222 - file1.jpg
31223 - file2.jpg
31224 - ...
31225 - media
31226 - all
31227 - file1.jpg
31228 - file2.jpg
31229 - ...
31230 - by-year
31231 - 2000
31232 - file1.jpg
31233 - ...
31234 - 2001
31235 - file2.jpg
31236 - ...
31237 - ...
31238 - by-month
31239 - 2000
31240 - 2000-01
31241 - file1.jpg
31242 - ...
31243 - 2000-02
31244 - file2.jpg
31245 - ...
31246 - ...
31247 - by-day
31248 - 2000
31249 - 2000-01-01
31250 - file1.jpg
31251 - ...
31252 - 2000-01-02
31253 - file2.jpg
31254 - ...
31255 - ...
31256 - album
31257 - album name
31258 - album name/sub
31259 - shared-album
31260 - album name
31261 - album name/sub
31262 - feature
31263 - favorites
31264 - file1.jpg
31265 - file2.jpg
31266 There are two writable parts of the tree, the upload directory and sub
31268 directories of the album directory.
31269 The upload directory is for uploading files you don't want to put into
31271 albums. This will be empty to start with and will contain the files
31272 you've uploaded for one rclone session only, becoming empty again when
31273 you restart rclone. The use case for this would be if you have a load
31274 of files you just want to once off dump into Google Photos. For re‐
31275 peated syncing, uploading to album will work better.
31276 Directories within the album directory are also writeable and you may
31278 create new directories (albums) under album. If you copy files with a
31279 directory hierarchy in there then rclone will create albums with the /
31280 character in them. For example if you do
31281 rclone copy /path/to/images remote:album/images
31283 and the images directory contains
31285 images
31287 - file1.jpg
31288 dir
31289 file2.jpg
31290 dir2
31291 dir3
31292 file3.jpg
31293 Then rclone will create the following albums with the following files
31295 in
31296 • images
31298 • file1.jpg
31300 • images/dir
31302 • file2.jpg
31304 • images/dir2/dir3
31306 • file3.jpg
31308 This means that you can use the album path pretty much like a normal
31310 filesystem and it is a good target for repeated syncing.
31311 The shared-album directory shows albums shared with you or by you.
31313 This is similar to the Sharing tab in the Google Photos web interface.
31314 Standard options
31316 Here are the Standard options specific to google photos (Google Pho‐
31317 tos).
31318 --gphotos-client-id
31320 OAuth Client Id.
31321 Leave blank normally.
31323 Properties:
31325 • Config: client_id
31327 • Env Var: RCLONE_GPHOTOS_CLIENT_ID
31329 • Type: string
31331 • Required: false
31333 --gphotos-client-secret
31335 OAuth Client Secret.
31336 Leave blank normally.
31338 Properties:
31340 • Config: client_secret
31342 • Env Var: RCLONE_GPHOTOS_CLIENT_SECRET
31344 • Type: string
31346 • Required: false
31348 --gphotos-read-only
31350 Set to make the Google Photos backend read only.
31351 If you choose read only then rclone will only request read only access
31353 to your photos, otherwise rclone will request full access.
31354 Properties:
31356 • Config: read_only
31358 • Env Var: RCLONE_GPHOTOS_READ_ONLY
31360 • Type: bool
31362 • Default: false
31364 Advanced options
31366 Here are the Advanced options specific to google photos (Google Pho‐
31367 tos).
31368 --gphotos-token
31370 OAuth Access Token as a JSON blob.
31371 Properties:
31373 • Config: token
31375 • Env Var: RCLONE_GPHOTOS_TOKEN
31377 • Type: string
31379 • Required: false
31381 --gphotos-auth-url
31383 Auth server URL.
31384 Leave blank to use the provider defaults.
31386 Properties:
31388 • Config: auth_url
31390 • Env Var: RCLONE_GPHOTOS_AUTH_URL
31392 • Type: string
31394 • Required: false
31396 --gphotos-token-url
31398 Token server url.
31399 Leave blank to use the provider defaults.
31401 Properties:
31403 • Config: token_url
31405 • Env Var: RCLONE_GPHOTOS_TOKEN_URL
31407 • Type: string
31409 • Required: false
31411 --gphotos-read-size
31413 Set to read the size of media items.
31414 Normally rclone does not read the size of media items since this takes
31416 another transaction. This isn't necessary for syncing. However rclone
31417 mount needs to know the size of files in advance of reading them, so
31418 setting this flag when using rclone mount is recommended if you want to
31419 read the media.
31420 Properties:
31422 • Config: read_size
31424 • Env Var: RCLONE_GPHOTOS_READ_SIZE
31426 • Type: bool
31428 • Default: false
31430 --gphotos-start-year
31432 Year limits the photos to be downloaded to those which are uploaded af‐
31433 ter the given year.
31434 Properties:
31436 • Config: start_year
31438 • Env Var: RCLONE_GPHOTOS_START_YEAR
31440 • Type: int
31442 • Default: 2000
31444 --gphotos-include-archived
31446 Also view and download archived media.
31447 By default, rclone does not request archived media. Thus, when sync‐
31449 ing, archived media is not visible in directory listings or trans‐
31450 ferred.
31451 Note that media in albums is always visible and synced, no matter their
31453 archive status.
31454 With this flag, archived media are always visible in directory listings
31456 and transferred.
31457 Without this flag, archived media will not be visible in directory
31459 listings and won't be transferred.
31460 Properties:
31462 • Config: include_archived
31464 • Env Var: RCLONE_GPHOTOS_INCLUDE_ARCHIVED
31466 • Type: bool
31468 • Default: false
31470 --gphotos-encoding
31472 The encoding for the backend.
31473 See the encoding section in the overview (https://rclone.org/over‐
31475 view/#encoding) for more info.
31476 Properties:
31478 • Config: encoding
31480 • Env Var: RCLONE_GPHOTOS_ENCODING
31482 • Type: MultiEncoder
31484 • Default: Slash,CrLf,InvalidUtf8,Dot
31486 Limitations
31488 Only images and videos can be uploaded. If you attempt to upload non
31489 videos or images or formats that Google Photos doesn't understand,
31490 rclone will upload the file, then Google Photos will give an error when
31491 it is put turned into a media item.
31492 Note that all media items uploaded to Google Photos through the API are
31494 stored in full resolution at "original quality" and will count towards
31495 your storage quota in your Google Account. The API does not offer a
31496 way to upload in "high quality" mode..
31497 rclone about is not supported by the Google Photos backend. Backends
31499 without this capability cannot determine free space for an rclone mount
31500 or use policy mfs (most free space) as a member of an rclone union re‐
31501 mote.
31502 See List of backends that do not support rclone about
31504 (https://rclone.org/overview/#optional-features) See rclone about
31505 (https://rclone.org/commands/rclone_about/)
31506 Downloading Images
31508 When Images are downloaded this strips EXIF location (according to the
31509 docs and my tests). This is a limitation of the Google Photos API and
31510 is covered by bug #112096115 (https://issuetracker.google.com/is‐
31511 sues/112096115).
31512 The current google API does not allow photos to be downloaded at origi‐
31514 nal resolution. This is very important if you are, for example, relying
31515 on "Google Photos" as a backup of your photos. You will not be able to
31516 use rclone to redownload original images. You could use 'google take‐
31517 out' to recover the original photos as a last resort
31518 Downloading Videos
31520 When videos are downloaded they are downloaded in a really compressed
31521 version of the video compared to downloading it via the Google Photos
31522 web interface. This is covered by bug #113672044 (https://issuetrack‐
31523 er.google.com/issues/113672044).
31524 Duplicates
31526 If a file name is duplicated in a directory then rclone will add the
31527 file ID into its name. So two files called file.jpg would then appear
31528 as file {123456}.jpg and file {ABCDEF}.jpg (the actual IDs are a lot
31529 longer alas!).
31530 If you upload the same image (with the same binary data) twice then
31532 Google Photos will deduplicate it. However it will retain the filename
31533 from the first upload which may confuse rclone. For example if you up‐
31534 loaded an image to upload then uploaded the same image to album/my_al‐
31535 bum the filename of the image in album/my_album will be what it was up‐
31536 loaded with initially, not what you uploaded it with to album. In
31537 practise this shouldn't cause too many problems.
31538 Modified time
31540 The date shown of media in Google Photos is the creation date as deter‐
31541 mined by the EXIF information, or the upload date if that is not known.
31542 This is not changeable by rclone and is not the modification date of
31544 the media on local disk. This means that rclone cannot use the dates
31545 from Google Photos for syncing purposes.
31546 Size
31548 The Google Photos API does not return the size of media. This means
31549 that when syncing to Google Photos, rclone can only do a file existence
31550 check.
31551 It is possible to read the size of the media, but this needs an extra
31553 HTTP HEAD request per media item so is very slow and uses up a lot of
31554 transactions. This can be enabled with the --gphotos-read-size option
31555 or the read_size = true config parameter.
31556 If you want to use the backend with rclone mount you may need to enable
31558 this flag (depending on your OS and application using the photos) oth‐
31559 erwise you may not be able to read media off the mount. You'll need to
31560 experiment to see if it works for you without the flag.
31561 Albums
31563 Rclone can only upload files to albums it created. This is a limita‐
31564 tion of the Google Photos API (https://developers.google.com/photos/li‐
31565 brary/guides/manage-albums).
31566 Rclone can remove files it uploaded from albums it created only.
31568 Deleting files
31570 Rclone can remove files from albums it created, but note that the
31571 Google Photos API does not allow media to be deleted permanently so
31572 this media will still remain. See bug #109759781 (https://issuetrack‐
31573 er.google.com/issues/109759781).
31574 Rclone cannot delete files anywhere except under album.
31576 Deleting albums
31578 The Google Photos API does not support deleting albums - see bug
31579 #135714733 (https://issuetracker.google.com/issues/135714733).
31580
31582 Hasher is a special overlay backend to create remotes which handle
31583 checksums for other remotes. It's main functions include: - Emulate
31584 hash types unimplemented by backends - Cache checksums to help with
31585 slow hashing of large local or (S)FTP files - Warm up checksum cache
31586 from external SUM files
31587 Getting started
31589 To use Hasher, first set up the underlying remote following the config‐
31590 uration instructions for that remote. You can also use a local path‐
31591 name instead of a remote. Check that your base remote is working.
31592 Let's call the base remote myRemote:path here. Note that anything in‐
31594 side myRemote:path will be handled by hasher and anything outside
31595 won't. This means that if you are using a bucket based remote (S3, B2,
31596 Swift) then you should put the bucket in the remote s3:bucket.
31597 Now proceed to interactive or manual configuration.
31599 Interactive configuration
31601 Run rclone config:
31602 No remotes found, make a new one?
31604 n) New remote
31605 s) Set configuration password
31606 q) Quit config
31607 n/s/q> n
31608 name> Hasher1
31609 Type of storage to configure.
31610 Choose a number from below, or type in your own value
31611 [snip]
31612 XX / Handle checksums for other remotes
31613 \ "hasher"
31614 [snip]
31615 Storage> hasher
31616 Remote to cache checksums for, like myremote:mypath.
31617 Enter a string value. Press Enter for the default ("").
31618 remote> myRemote:path
31619 Comma separated list of supported checksum types.
31620 Enter a string value. Press Enter for the default ("md5,sha1").
31621 hashsums> md5
31622 Maximum time to keep checksums in cache. 0 = no cache, off = cache forever.
31623 max_age> off
31624 Edit advanced config? (y/n)
31625 y) Yes
31626 n) No
31627 y/n> n
31628 Remote config
31629 --------------------
31630 [Hasher1]
31631 type = hasher
31632 remote = myRemote:path
31633 hashsums = md5
31634 max_age = off
31635 --------------------
31636 y) Yes this is OK
31637 e) Edit this remote
31638 d) Delete this remote
31639 y/e/d> y
31640 Manual configuration
31642 Run rclone config path to see the path of current active config file,
31643 usually YOURHOME/.config/rclone/rclone.conf. Open it in your favorite
31644 text editor, find section for the base remote and create new section
31645 for hasher like in the following examples:
31646 [Hasher1]
31648 type = hasher
31649 remote = myRemote:path
31650 hashes = md5
31651 max_age = off
31652 [Hasher2]
31654 type = hasher
31655 remote = /local/path
31656 hashes = dropbox,sha1
31657 max_age = 24h
31658 Hasher takes basically the following parameters: - remote is required,
31660 - hashes is a comma separated list of supported checksums (by default
31661 md5,sha1), - max_age - maximum time to keep a checksum value in the
31662 cache, 0 will disable caching completely, off will cache "forever"
31663 (that is until the files get changed).
31664 Make sure the remote has : (colon) in. If you specify the remote with‐
31666 out a colon then rclone will use a local directory of that name. So if
31667 you use a remote of /local/path then rclone will handle hashes for that
31668 directory. If you use remote = name literally then rclone will put
31669 files in a directory called name located under current directory.
31670 Usage
31672 Basic operations
31673 Now you can use it as Hasher2:subdir/file instead of base remote.
31674 Hasher will transparently update cache with new checksums when a file
31675 is fully read or overwritten, like:
31676 rclone copy External:path/file Hasher:dest/path
31678 rclone cat Hasher:path/to/file > /dev/null
31680 The way to refresh all cached checksums (even unsupported by the base
31682 backend) for a subtree is to re-download all files in the subtree. For
31683 example, use hashsum --download using any supported hashsum on the com‐
31684 mand line (we just care to re-read):
31685 rclone hashsum MD5 --download Hasher:path/to/subtree > /dev/null
31687 rclone backend dump Hasher:path/to/subtree
31689 You can print or drop hashsum cache using custom backend commands:
31691 rclone backend dump Hasher:dir/subdir
31693 rclone backend drop Hasher:
31695 Pre-Seed from a SUM File
31697 Hasher supports two backend commands: generic SUM file import and
31698 faster but less consistent stickyimport.
31699 rclone backend import Hasher:dir/subdir SHA1 /path/to/SHA1SUM [--checkers 4]
31701 Instead of SHA1 it can be any hash supported by the remote. The last
31703 argument can point to either a local or an other-remote:path text file
31704 in SUM format. The command will parse the SUM file, then walk down the
31705 path given by the first argument, snapshot current fingerprints and
31706 fill in the cache entries correspondingly. - Paths in the SUM file are
31707 treated as relative to hasher:dir/subdir. - The command will not check
31708 that supplied values are correct. You must know what you are doing. -
31709 This is a one-time action. The SUM file will not get "attached" to the
31710 remote. Cache entries can still be overwritten later, should the ob‐
31711 ject's fingerprint change. - The tree walk can take long depending on
31712 the tree size. You can increase --checkers to make it faster. Or use
31713 stickyimport if you don't care about fingerprints and consistency.
31714 rclone backend stickyimport hasher:path/to/data sha1 remote:/path/to/sum.sha1
31716 stickyimport is similar to import but works much faster because it does
31718 not need to stat existing files and skips initial tree walk. Instead
31719 of binding cache entries to file fingerprints it creates sticky entries
31720 bound to the file name alone ignoring size, modification time etc.
31721 Such hash entries can be replaced only by purge, delete, backend drop
31722 or by full re-read/re-write of the files.
31723 Configuration reference
31725 Standard options
31726 Here are the Standard options specific to hasher (Better checksums for
31727 other remotes).
31728 --hasher-remote
31730 Remote to cache checksums for (e.g. myRemote:path).
31731 Properties:
31733 • Config: remote
31735 • Env Var: RCLONE_HASHER_REMOTE
31737 • Type: string
31739 • Required: true
31741 --hasher-hashes
31743 Comma separated list of supported checksum types.
31744 Properties:
31746 • Config: hashes
31748 • Env Var: RCLONE_HASHER_HASHES
31750 • Type: CommaSepList
31752 • Default: md5,sha1
31754 --hasher-max-age
31756 Maximum time to keep checksums in cache (0 = no cache, off = cache for‐
31757 ever).
31758 Properties:
31760 • Config: max_age
31762 • Env Var: RCLONE_HASHER_MAX_AGE
31764 • Type: Duration
31766 • Default: off
31768 Advanced options
31770 Here are the Advanced options specific to hasher (Better checksums for
31771 other remotes).
31772 --hasher-auto-size
31774 Auto-update checksum for files smaller than this size (disabled by de‐
31775 fault).
31776 Properties:
31778 • Config: auto_size
31780 • Env Var: RCLONE_HASHER_AUTO_SIZE
31782 • Type: SizeSuffix
31784 • Default: 0
31786 Metadata
31788 Any metadata supported by the underlying remote is read and written.
31789 See the metadata (https://rclone.org/docs/#metadata) docs for more in‐
31791 fo.
31792 Backend commands
31794 Here are the commands specific to the hasher backend.
31795 Run them with
31797 rclone backend COMMAND remote:
31799 The help below will explain what arguments each command takes.
31801 See the backend (https://rclone.org/commands/rclone_backend/) command
31803 for more info on how to pass options and arguments.
31804 These can be run on a running backend using the rc command backend/com‐
31806 mand (https://rclone.org/rc/#backend-command).
31807 drop
31809 Drop cache
31810 rclone backend drop remote: [options] [<arguments>+]
31812 Completely drop checksum cache. Usage Example: rclone backend drop
31814 hasher:
31815 dump
31817 Dump the database
31818 rclone backend dump remote: [options] [<arguments>+]
31820 Dump cache records covered by the current remote
31822 fulldump
31824 Full dump of the database
31825 rclone backend fulldump remote: [options] [<arguments>+]
31827 Dump all cache records in the database
31829 import
31831 Import a SUM file
31832 rclone backend import remote: [options] [<arguments>+]
31834 Amend hash cache from a SUM file and bind checksums to files by
31836 size/time. Usage Example: rclone backend import hasher:subdir md5
31837 /path/to/sum.md5
31838 stickyimport
31840 Perform fast import of a SUM file
31841 rclone backend stickyimport remote: [options] [<arguments>+]
31843 Fill hash cache from a SUM file without verifying file fingerprints.
31845 Usage Example: rclone backend stickyimport hasher:subdir md5 re‐
31846 mote:path/to/sum.md5
31847 Implementation details (advanced)
31849 This section explains how various rclone operations work on a hasher
31850 remote.
31851 Disclaimer. This section describes current implementation which can
31853 change in future rclone versions!.
31854 Hashsum command
31856 The rclone hashsum (or md5sum or sha1sum) command will:
31857 1. if requested hash is supported by lower level, just pass it.
31859 2. if object size is below auto_size then download object and calculate
31861 requested hashes on the fly.
31862 3. if unsupported and the size is big enough, build object fingerprint
31864 (including size, modtime if supported, first-found other hash if
31865 any).
31866 4. if the strict match is found in cache for the requested remote, re‐
31868 turn the stored hash.
31869 5. if remote found but fingerprint mismatched, then purge the entry and
31871 proceed to step 6.
31872 6. if remote not found or had no requested hash type or after step 5:
31874 download object, calculate all supported hashes on the fly and store
31875 in cache; return requested hash.
31876 Other operations
31878 • whenever a file is uploaded or downloaded in full, capture the stream
31879 to calculate all supported hashes on the fly and update database
31880 • server-side move will update keys of existing cache entries
31882 • deletefile will remove a single cache entry
31884 • purge will remove all cache entries under the purged path
31886 Note that setting max_age = 0 will disable checksum caching completely.
31888 If you set max_age = off, checksums in cache will never age, unless you
31890 fully rewrite or delete the file.
31891 Cache storage
31893 Cached checksums are stored as bolt database files under rclone cache
31894 directory, usually ~/.cache/rclone/kv/. Databases are maintained one
31895 per base backend, named like BaseRemote~hasher.bolt. Checksums for
31896 multiple alias-es into a single base backend will be stored in the sin‐
31897 gle database. All local paths are treated as aliases into the local
31898 backend (unless crypted or chunked) and stored in
31899 ~/.cache/rclone/kv/local~hasher.bolt. Databases can be shared between
31900 multiple rclone processes.
31901
31903 HDFS (https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-
31904 hdfs/HdfsDesign.html) is a distributed file-system, part of the Apache
31905 Hadoop (https://hadoop.apache.org/) framework.
31906 Paths are specified as remote: or remote:path/to/dir.
31908 Configuration
31910 Here is an example of how to make a remote called remote. First run:
31911 rclone config
31913 This will guide you through an interactive setup process:
31915 No remotes found, make a new one?
31917 n) New remote
31918 s) Set configuration password
31919 q) Quit config
31920 n/s/q> n
31921 name> remote
31922 Type of storage to configure.
31923 Enter a string value. Press Enter for the default ("").
31924 Choose a number from below, or type in your own value
31925 [skip]
31926 XX / Hadoop distributed file system
31927 \ "hdfs"
31928 [skip]
31929 Storage> hdfs
31930 ** See help for hdfs backend at: https://rclone.org/hdfs/ **
31931 hadoop name node and port
31933 Enter a string value. Press Enter for the default ("").
31934 Choose a number from below, or type in your own value
31935 1 / Connect to host namenode at port 8020
31936 \ "namenode:8020"
31937 namenode> namenode.hadoop:8020
31938 hadoop user name
31939 Enter a string value. Press Enter for the default ("").
31940 Choose a number from below, or type in your own value
31941 1 / Connect to hdfs as root
31942 \ "root"
31943 username> root
31944 Edit advanced config? (y/n)
31945 y) Yes
31946 n) No (default)
31947 y/n> n
31948 Remote config
31949 --------------------
31950 [remote]
31951 type = hdfs
31952 namenode = namenode.hadoop:8020
31953 username = root
31954 --------------------
31955 y) Yes this is OK (default)
31956 e) Edit this remote
31957 d) Delete this remote
31958 y/e/d> y
31959 Current remotes:
31960 Name Type
31962 ==== ====
31963 hadoop hdfs
31964 e) Edit existing remote
31966 n) New remote
31967 d) Delete remote
31968 r) Rename remote
31969 c) Copy remote
31970 s) Set configuration password
31971 q) Quit config
31972 e/n/d/r/c/s/q> q
31973 This remote is called remote and can now be used like this
31975 See all the top level directories
31977 rclone lsd remote:
31979 List the contents of a directory
31981 rclone ls remote:directory
31983 Sync the remote directory to /home/local/directory, deleting any excess
31985 files.
31986 rclone sync -i remote:directory /home/local/directory
31988 Setting up your own HDFS instance for testing
31990 You may start with a manual setup (https://hadoop.apache.org/docs/sta‐
31991 ble/hadoop-project-dist/hadoop-common/SingleCluster.html) or use the
31992 docker image from the tests:
31993 If you want to build the docker image
31995 git clone https://github.com/rclone/rclone.git
31997 cd rclone/fstest/testserver/images/test-hdfs
31998 docker build --rm -t rclone/test-hdfs .
31999 Or you can just use the latest one pushed
32001 docker run --rm --name "rclone-hdfs" -p 127.0.0.1:9866:9866 -p 127.0.0.1:8020:8020 --hostname "rclone-hdfs" rclone/test-hdfs
32003 NB it need few seconds to startup.
32005 For this docker image the remote needs to be configured like this:
32007 [remote]
32009 type = hdfs
32010 namenode = 127.0.0.1:8020
32011 username = root
32012 You can stop this image with docker kill rclone-hdfs (NB it does not
32014 use volumes, so all data uploaded will be lost.)
32015 Modified time
32017 Time accurate to 1 second is stored.
32018 Checksum
32020 No checksums are implemented.
32021 Usage information
32023 You can use the rclone about remote: command which will display
32024 filesystem size and current usage.
32025 Restricted filename characters
32027 In addition to the default restricted characters set
32028 (https://rclone.org/overview/#restricted-characters) the following
32029 characters are also replaced:
32030 Character Value Replacement
32032 ────────────────────────────────
32033 : 0x3A ：
32034 Invalid UTF-8 bytes will also be replaced (https://rclone.org/over‐
32036 view/#invalid-utf8).
32037 Standard options
32039 Here are the Standard options specific to hdfs (Hadoop distributed file
32040 system).
32041 --hdfs-namenode
32043 Hadoop name node and port.
32044 E.g. "namenode:8020" to connect to host namenode at port 8020.
32046 Properties:
32048 • Config: namenode
32050 • Env Var: RCLONE_HDFS_NAMENODE
32052 • Type: string
32054 • Required: true
32056 --hdfs-username
32058 Hadoop user name.
32059 Properties:
32061 • Config: username
32063 • Env Var: RCLONE_HDFS_USERNAME
32065 • Type: string
32067 • Required: false
32069 • Examples:
32071 • "root"
32073 • Connect to hdfs as root.
32075 Advanced options
32077 Here are the Advanced options specific to hdfs (Hadoop distributed file
32078 system).
32079 --hdfs-service-principal-name
32081 Kerberos service principal name for the namenode.
32082 Enables KERBEROS authentication. Specifies the Service Principal Name
32084 (SERVICE/FQDN) for the namenode. E.g. "hdfs/namenode.hadoop.docker"
32085 for namenode running as service 'hdfs' with FQDN 'namenode.hadoop.dock‐
32086 er'.
32087 Properties:
32089 • Config: service_principal_name
32091 • Env Var: RCLONE_HDFS_SERVICE_PRINCIPAL_NAME
32093 • Type: string
32095 • Required: false
32097 --hdfs-data-transfer-protection
32099 Kerberos data transfer protection: authentication|integrity|privacy.
32100 Specifies whether or not authentication, data signature integrity
32102 checks, and wire encryption is required when communicating the the
32103 datanodes. Possible values are 'authentication', 'integrity' and 'pri‐
32104 vacy'. Used only with KERBEROS enabled.
32105 Properties:
32107 • Config: data_transfer_protection
32109 • Env Var: RCLONE_HDFS_DATA_TRANSFER_PROTECTION
32111 • Type: string
32113 • Required: false
32115 • Examples:
32117 • "privacy"
32119 • Ensure authentication, integrity and encryption enabled.
32121 --hdfs-encoding
32123 The encoding for the backend.
32124 See the encoding section in the overview (https://rclone.org/over‐
32126 view/#encoding) for more info.
32127 Properties:
32129 • Config: encoding
32131 • Env Var: RCLONE_HDFS_ENCODING
32133 • Type: MultiEncoder
32135 • Default: Slash,Colon,Del,Ctl,InvalidUtf8,Dot
32137 Limitations
32139 • No server-side Move or DirMove.
32140 • Checksums not implemented.
32142
32144 Paths are specified as remote:path
32145 Paths may be as deep as required, e.g. remote:directory/subdirectory.
32147 The initial setup for hidrive involves getting a token from HiDrive
32149 which you need to do in your browser. rclone config walks you through
32150 it.
32151 Configuration
32153 Here is an example of how to make a remote called remote. First run:
32154 rclone config
32156 This will guide you through an interactive setup process:
32158 No remotes found - make a new one
32160 n) New remote
32161 s) Set configuration password
32162 q) Quit config
32163 n/s/q> n
32164 name> remote
32165 Type of storage to configure.
32166 Choose a number from below, or type in your own value
32167 [snip]
32168 XX / HiDrive
32169 \ "hidrive"
32170 [snip]
32171 Storage> hidrive
32172 OAuth Client Id - Leave blank normally.
32173 client_id>
32174 OAuth Client Secret - Leave blank normally.
32175 client_secret>
32176 Access permissions that rclone should use when requesting access from HiDrive.
32177 Leave blank normally.
32178 scope_access>
32179 Edit advanced config?
32180 y/n> n
32181 Use auto config?
32182 y/n> y
32183 If your browser doesn't open automatically go to the following link: http://127.0.0.1:53682/auth?state=xxxxxxxxxxxxxxxxxxxxxx
32184 Log in and authorize rclone for access
32185 Waiting for code...
32186 Got code
32187 --------------------
32188 [remote]
32189 type = hidrive
32190 token = {"access_token":"xxxxxxxxxxxxxxxxxxxx","token_type":"Bearer","refresh_token":"xxxxxxxxxxxxxxxxxxxxxxx","expiry":"xxxxxxxxxxxxxxxxxxxxxxx"}
32191 --------------------
32192 y) Yes this is OK (default)
32193 e) Edit this remote
32194 d) Delete this remote
32195 y/e/d> y
32196 You should be aware that OAuth-tokens can be used to access your ac‐
32198 count and hence should not be shared with other persons. See the below
32199 section for more information.
32200 See the remote setup docs (https://rclone.org/remote_setup/) for how to
32202 set it up on a machine with no Internet browser available.
32203 Note that rclone runs a webserver on your local machine to collect the
32205 token as returned from HiDrive. This only runs from the moment it
32206 opens your browser to the moment you get back the verification code.
32207 The webserver runs on http://127.0.0.1:53682/. If local port 53682 is
32208 protected by a firewall you may need to temporarily unblock the fire‐
32209 wall to complete authorization.
32210 Once configured you can then use rclone like this,
32212 List directories in top level of your HiDrive root folder
32214 rclone lsd remote:
32216 List all the files in your HiDrive filesystem
32218 rclone ls remote:
32220 To copy a local directory to a HiDrive directory called backup
32222 rclone copy /home/source remote:backup
32224 Keeping your tokens safe
32226 Any OAuth-tokens will be stored by rclone in the remote's configuration
32227 file as unencrypted text. Anyone can use a valid refresh-token to ac‐
32228 cess your HiDrive filesystem without knowing your password. Therefore
32229 you should make sure no one else can access your configuration.
32230 It is possible to encrypt rclone's configuration file. You can find
32232 information on securing your configuration file by viewing the configu‐
32233 ration encryption docs (https://rclone.org/docs/#configuration-encryp‐
32234 tion).
32235 Invalid refresh token
32237 As can be verified here (https://developer.hidrive.com/basics-flows/),
32238 each refresh_token (for Native Applications) is valid for 60 days. If
32239 used to access HiDrivei, its validity will be automatically extended.
32240 This means that if you
32242 • Don't use the HiDrive remote for 60 days
32244 then rclone will return an error which includes a text that implies the
32246 refresh token is invalid or expired.
32247 To fix this you will need to authorize rclone to access your HiDrive
32249 account again.
32250 Using
32252 rclone config reconnect remote:
32254 the process is very similar to the process of initial setup exemplified
32256 before.
32257 Modified time and hashes
32259 HiDrive allows modification times to be set on objects accurate to 1
32260 second.
32261 HiDrive supports its own hash type (https://stat‐
32263 ic.hidrive.com/dev/0001) which is used to verify the integrity of file
32264 contents after successful transfers.
32265 Restricted filename characters
32267 HiDrive cannot store files or folders that include / (0x2F) or null-
32268 bytes (0x00) in their name. Any other characters can be used in the
32269 names of files or folders. Additionally, files or folders cannot be
32270 named either of the following: . or ..
32271 Therefore rclone will automatically replace these characters, if files
32273 or folders are stored or accessed with such names.
32274 You can read about how this filename encoding works in general here.
32276 Keep in mind that HiDrive only supports file or folder names with a
32278 length of 255 characters or less.
32279 Transfers
32281 HiDrive limits file sizes per single request to a maximum of 2 GiB. To
32282 allow storage of larger files and allow for better upload performance,
32283 the hidrive backend will use a chunked transfer for files larger than
32284 96 MiB. Rclone will upload multiple parts/chunks of the file at the
32285 same time. Chunks in the process of being uploaded are buffered in
32286 memory, so you may want to restrict this behaviour on systems with lim‐
32287 ited resources.
32288 You can customize this behaviour using the following options:
32290 • chunk_size: size of file parts
32292 • upload_cutoff: files larger or equal to this in size will use a chun‐
32294 ked transfer
32295 • upload_concurrency: number of file-parts to upload at the same time
32297 See the below section about configuration options for more details.
32299 Root folder
32301 You can set the root folder for rclone. This is the directory that
32302 rclone considers to be the root of your HiDrive.
32303 Usually, you will leave this blank, and rclone will use the root of the
32305 account.
32306 However, you can set this to restrict rclone to a specific folder hier‐
32308 archy.
32309 This works by prepending the contents of the root_prefix option to any
32311 paths accessed by rclone. For example, the following two ways to ac‐
32312 cess the home directory are equivalent:
32313 rclone lsd --hidrive-root-prefix="/users/test/" remote:path
32315 rclone lsd remote:/users/test/path
32317 See the below section about configuration options for more details.
32319 Directory member count
32321 By default, rclone will know the number of directory members contained
32322 in a directory. For example, rclone lsd uses this information.
32323 The acquisition of this information will result in additional time
32325 costs for HiDrive's API. When dealing with large directory structures,
32326 it may be desirable to circumvent this time cost, especially when this
32327 information is not explicitly needed. For this, the disable_fetch‐
32328 ing_member_count option can be used.
32329 See the below section about configuration options for more details.
32331 Standard options
32333 Here are the Standard options specific to hidrive (HiDrive).
32334 --hidrive-client-id
32336 OAuth Client Id.
32337 Leave blank normally.
32339 Properties:
32341 • Config: client_id
32343 • Env Var: RCLONE_HIDRIVE_CLIENT_ID
32345 • Type: string
32347 • Required: false
32349 --hidrive-client-secret
32351 OAuth Client Secret.
32352 Leave blank normally.
32354 Properties:
32356 • Config: client_secret
32358 • Env Var: RCLONE_HIDRIVE_CLIENT_SECRET
32360 • Type: string
32362 • Required: false
32364 --hidrive-scope-access
32366 Access permissions that rclone should use when requesting access from
32367 HiDrive.
32368 Properties:
32370 • Config: scope_access
32372 • Env Var: RCLONE_HIDRIVE_SCOPE_ACCESS
32374 • Type: string
32376 • Default: "rw"
32378 • Examples:
32380 • "rw"
32382 • Read and write access to resources.
32384 • "ro"
32386 • Read-only access to resources.
32388 Advanced options
32390 Here are the Advanced options specific to hidrive (HiDrive).
32391 --hidrive-token
32393 OAuth Access Token as a JSON blob.
32394 Properties:
32396 • Config: token
32398 • Env Var: RCLONE_HIDRIVE_TOKEN
32400 • Type: string
32402 • Required: false
32404 --hidrive-auth-url
32406 Auth server URL.
32407 Leave blank to use the provider defaults.
32409 Properties:
32411 • Config: auth_url
32413 • Env Var: RCLONE_HIDRIVE_AUTH_URL
32415 • Type: string
32417 • Required: false
32419 --hidrive-token-url
32421 Token server url.
32422 Leave blank to use the provider defaults.
32424 Properties:
32426 • Config: token_url
32428 • Env Var: RCLONE_HIDRIVE_TOKEN_URL
32430 • Type: string
32432 • Required: false
32434 --hidrive-scope-role
32436 User-level that rclone should use when requesting access from HiDrive.
32437 Properties:
32439 • Config: scope_role
32441 • Env Var: RCLONE_HIDRIVE_SCOPE_ROLE
32443 • Type: string
32445 • Default: "user"
32447 • Examples:
32449 • "user"
32451 • User-level access to management permissions.
32453 • This will be sufficient in most cases.
32455 • "admin"
32457 • Extensive access to management permissions.
32459 • "owner"
32461 • Full access to management permissions.
32463 --hidrive-root-prefix
32465 The root/parent folder for all paths.
32466 Fill in to use the specified folder as the parent for all paths given
32468 to the remote. This way rclone can use any folder as its starting
32469 point.
32470 Properties:
32472 • Config: root_prefix
32474 • Env Var: RCLONE_HIDRIVE_ROOT_PREFIX
32476 • Type: string
32478 • Default: "/"
32480 • Examples:
32482 • "/"
32484 • The topmost directory accessible by rclone.
32486 • This will be equivalent with "root" if rclone uses a regular
32488 HiDrive user account.
32489 • "root"
32491 • The topmost directory of the HiDrive user account
32493 • ""
32495 • This specifies that there is no root-prefix for your paths.
32497 • When using this you will always need to specify paths to this re‐
32499 mote with a valid parent e.g. "remote:/path/to/dir" or "re‐
32500 mote:root/path/to/dir".
32501 --hidrive-endpoint
32503 Endpoint for the service.
32504 This is the URL that API-calls will be made to.
32506 Properties:
32508 • Config: endpoint
32510 • Env Var: RCLONE_HIDRIVE_ENDPOINT
32512 • Type: string
32514 • Default: "https://api.hidrive.strato.com/2.1"
32516 --hidrive-disable-fetching-member-count
32518 Do not fetch number of objects in directories unless it is absolutely
32519 necessary.
32520 Requests may be faster if the number of objects in subdirectories is
32522 not fetched.
32523 Properties:
32525 • Config: disable_fetching_member_count
32527 • Env Var: RCLONE_HIDRIVE_DISABLE_FETCHING_MEMBER_COUNT
32529 • Type: bool
32531 • Default: false
32533 --hidrive-chunk-size
32535 Chunksize for chunked uploads.
32536 Any files larger than the configured cutoff (or files of unknown size)
32538 will be uploaded in chunks of this size.
32539 The upper limit for this is 2147483647 bytes (about 2.000Gi). That is
32541 the maximum amount of bytes a single upload-operation will support.
32542 Setting this above the upper limit or to a negative value will cause
32543 uploads to fail.
32544 Setting this to larger values may increase the upload speed at the cost
32546 of using more memory. It can be set to smaller values smaller to save
32547 on memory.
32548 Properties:
32550 • Config: chunk_size
32552 • Env Var: RCLONE_HIDRIVE_CHUNK_SIZE
32554 • Type: SizeSuffix
32556 • Default: 48Mi
32558 --hidrive-upload-cutoff
32560 Cutoff/Threshold for chunked uploads.
32561 Any files larger than this will be uploaded in chunks of the configured
32563 chunksize.
32564 The upper limit for this is 2147483647 bytes (about 2.000Gi). That is
32566 the maximum amount of bytes a single upload-operation will support.
32567 Setting this above the upper limit will cause uploads to fail.
32568 Properties:
32570 • Config: upload_cutoff
32572 • Env Var: RCLONE_HIDRIVE_UPLOAD_CUTOFF
32574 • Type: SizeSuffix
32576 • Default: 96Mi
32578 --hidrive-upload-concurrency
32580 Concurrency for chunked uploads.
32581 This is the upper limit for how many transfers for the same file are
32583 running concurrently. Setting this above to a value smaller than 1
32584 will cause uploads to deadlock.
32585 If you are uploading small numbers of large files over high-speed links
32587 and these uploads do not fully utilize your bandwidth, then increasing
32588 this may help to speed up the transfers.
32589 Properties:
32591 • Config: upload_concurrency
32593 • Env Var: RCLONE_HIDRIVE_UPLOAD_CONCURRENCY
32595 • Type: int
32597 • Default: 4
32599 --hidrive-encoding
32601 The encoding for the backend.
32602 See the encoding section in the overview (https://rclone.org/over‐
32604 view/#encoding) for more info.
32605 Properties:
32607 • Config: encoding
32609 • Env Var: RCLONE_HIDRIVE_ENCODING
32611 • Type: MultiEncoder
32613 • Default: Slash,Dot
32615 Limitations
32617 Symbolic links
32618 HiDrive is able to store symbolic links (symlinks) by design, for exam‐
32619 ple, when unpacked from a zip archive.
32620 There exists no direct mechanism to manage native symlinks in remotes.
32622 As such this implementation has chosen to ignore any native symlinks
32623 present in the remote. rclone will not be able to access or show any
32624 symlinks stored in the hidrive-remote. This means symlinks cannot be
32625 individually removed, copied, or moved, except when removing, copying,
32626 or moving the parent folder.
32627 This does not affect the .rclonelink-files that rclone uses to encode
32629 and store symbolic links.
32630 Sparse files
32632 It is possible to store sparse files in HiDrive.
32633 Note that copying a sparse file will expand the holes into null-byte
32635 (0x00) regions that will then consume disk space. Likewise, when down‐
32636 loading a sparse file, the resulting file will have null-byte regions
32637 in the place of file holes.
32638
32640 The HTTP remote is a read only remote for reading files of a webserver.
32641 The webserver should provide file listings which rclone will read and
32642 turn into a remote. This has been tested with common webservers such
32643 as Apache/Nginx/Caddy and will likely work with file listings from most
32644 web servers. (If it doesn't then please file an issue, or send a pull
32645 request!)
32646 Paths are specified as remote: or remote:path.
32648 The remote: represents the configured url, and any path following it
32650 will be resolved relative to this url, according to the URL standard.
32651 This means with remote url https://beta.rclone.org/branch and path fix,
32652 the resolved URL will be https://beta.rclone.org/branch/fix, while with
32653 path /fix the resolved URL will be https://beta.rclone.org/fix as the
32654 absolute path is resolved from the root of the domain.
32655 If the path following the remote: ends with / it will be assumed to
32657 point to a directory. If the path does not end with /, then a HEAD re‐
32658 quest is sent and the response used to decide if it it is treated as a
32659 file or a directory (run with -vv to see details). When --http-no-head
32660 is specified, a path without ending / is always assumed to be a file.
32661 If rclone incorrectly assumes the path is a file, the solution is to
32662 specify the path with ending /. When you know the path is a directory,
32663 ending it with / is always better as it avoids the initial HEAD re‐
32664 quest.
32665 To just download a single file it is easier to use copyurl
32667 (https://rclone.org/commands/rclone_copyurl/).
32668 Configuration
32670 Here is an example of how to make a remote called remote. First run:
32671 rclone config
32673 This will guide you through an interactive setup process:
32675 No remotes found, make a new one?
32677 n) New remote
32678 s) Set configuration password
32679 q) Quit config
32680 n/s/q> n
32681 name> remote
32682 Type of storage to configure.
32683 Choose a number from below, or type in your own value
32684 [snip]
32685 XX / HTTP
32686 \ "http"
32687 [snip]
32688 Storage> http
32689 URL of http host to connect to
32690 Choose a number from below, or type in your own value
32691 1 / Connect to example.com
32692 \ "https://example.com"
32693 url> https://beta.rclone.org
32694 Remote config
32695 --------------------
32696 [remote]
32697 url = https://beta.rclone.org
32698 --------------------
32699 y) Yes this is OK
32700 e) Edit this remote
32701 d) Delete this remote
32702 y/e/d> y
32703 Current remotes:
32704 Name Type
32706 ==== ====
32707 remote http
32708 e) Edit existing remote
32710 n) New remote
32711 d) Delete remote
32712 r) Rename remote
32713 c) Copy remote
32714 s) Set configuration password
32715 q) Quit config
32716 e/n/d/r/c/s/q> q
32717 This remote is called remote and can now be used like this
32719 See all the top level directories
32721 rclone lsd remote:
32723 List the contents of a directory
32725 rclone ls remote:directory
32727 Sync the remote directory to /home/local/directory, deleting any excess
32729 files.
32730 rclone sync -i remote:directory /home/local/directory
32732 Read only
32734 This remote is read only - you can't upload files to an HTTP server.
32735 Modified time
32737 Most HTTP servers store time accurate to 1 second.
32738 Checksum
32740 No checksums are stored.
32741 Usage without a config file
32743 Since the http remote only has one config parameter it is easy to use
32744 without a config file:
32745 rclone lsd --http-url https://beta.rclone.org :http:
32747 or:
32749 rclone lsd :http,url='https://beta.rclone.org':
32751 Standard options
32753 Here are the Standard options specific to http (HTTP).
32754 --http-url
32756 URL of HTTP host to connect to.
32757 E.g. "https://example.com", or "https://user:pass@example.com" to use
32759 a username and password.
32760 Properties:
32762 • Config: url
32764 • Env Var: RCLONE_HTTP_URL
32766 • Type: string
32768 • Required: true
32770 Advanced options
32772 Here are the Advanced options specific to http (HTTP).
32773 --http-headers
32775 Set HTTP headers for all transactions.
32776 Use this to set additional HTTP headers for all transactions.
32778 The input format is comma separated list of key,value pairs. Standard
32780 CSV encoding (https://godoc.org/encoding/csv) may be used.
32781 For example, to set a Cookie use 'Cookie,name=value', or '"Cook‐
32783 ie","name=value"'.
32784 You can set multiple headers, e.g. '"Cookie","name=value","Authoriza‐
32786 tion","xxx"'.
32787 Properties:
32789 • Config: headers
32791 • Env Var: RCLONE_HTTP_HEADERS
32793 • Type: CommaSepList
32795 • Default:
32797 --http-no-slash
32799 Set this if the site doesn't end directories with /.
32800 Use this if your target website does not use / on the end of directo‐
32802 ries.
32803 A / on the end of a path is how rclone normally tells the difference
32805 between files and directories. If this flag is set, then rclone will
32806 treat all files with Content-Type: text/html as directories and read
32807 URLs from them rather than downloading them.
32808 Note that this may cause rclone to confuse genuine HTML files with di‐
32810 rectories.
32811 Properties:
32813 • Config: no_slash
32815 • Env Var: RCLONE_HTTP_NO_SLASH
32817 • Type: bool
32819 • Default: false
32821 --http-no-head
32823 Don't use HEAD requests.
32824 HEAD requests are mainly used to find file sizes in dir listing. If
32826 your site is being very slow to load then you can try this option.
32827 Normally rclone does a HEAD request for each potential file in a direc‐
32828 tory listing to:
32829 • find its size
32831 • check it really exists
32833 • check to see if it is a directory
32835 If you set this option, rclone will not do the HEAD request. This will
32837 mean that directory listings are much quicker, but rclone won't have
32838 the times or sizes of any files, and some files that don't exist may be
32839 in the listing.
32840 Properties:
32842 • Config: no_head
32844 • Env Var: RCLONE_HTTP_NO_HEAD
32846 • Type: bool
32848 • Default: false
32850 Limitations
32852 rclone about is not supported by the HTTP backend. Backends without
32853 this capability cannot determine free space for an rclone mount or use
32854 policy mfs (most free space) as a member of an rclone union remote.
32855 See List of backends that do not support rclone about
32857 (https://rclone.org/overview/#optional-features) and rclone about
32858 (https://rclone.org/commands/rclone_about/)
32859
32861 The Internet Archive backend utilizes Items on archive.org (https://ar‐
32862 chive.org/)
32863 Refer to IAS3 API documentation (https://archive.org/ser‐
32865 vices/docs/api/ias3.html) for the API this backend uses.
32866 Paths are specified as remote:bucket (or remote: for the lsd command.)
32868 You may put subdirectories in too, e.g. remote:item/path/to/dir.
32869 Unlike S3, listing up all items uploaded by you isn't supported.
32871 Once you have made a remote, you can use it like this:
32873 Make a new item
32875 rclone mkdir remote:item
32877 List the contents of a item
32879 rclone ls remote:item
32881 Sync /home/local/directory to the remote item, deleting any excess
32883 files in the item.
32884 rclone sync -i /home/local/directory remote:item
32886 Notes
32888 Because of Internet Archive's architecture, it enqueues write opera‐
32889 tions (and extra post-processings) in a per-item queue. You can check
32890 item's queue at https://catalogd.archive.org/history/item-name-here .
32891 Because of that, all uploads/deletes will not show up immediately and
32892 takes some time to be available. The per-item queue is enqueued to an
32893 another queue, Item Deriver Queue. You can check the status of Item
32894 Deriver Queue here. (https://catalogd.archive.org/catalog.php?wherea‐
32895 mi=1) This queue has a limit, and it may block you from uploading, or
32896 even deleting. You should avoid uploading a lot of small files for
32897 better behavior.
32898 You can optionally wait for the server's processing to finish, by set‐
32900 ting non-zero value to wait_archive key. By making it wait, rclone can
32901 do normal file comparison. Make sure to set a large enough value (e.g.
32902 30m0s for smaller files) as it can take a long time depending on serv‐
32903 er's queue.
32904 About metadata
32906 This backend supports setting, updating and reading metadata of each
32907 file. The metadata will appear as file metadata on Internet Archive.
32908 However, some fields are reserved by both Internet Archive and rclone.
32909 The following are reserved by Internet Archive: - name - source - size
32911 - md5 - crc32 - sha1 - format - old_version - viruscheck - summation
32912 Trying to set values to these keys is ignored with a warning. Only
32914 setting mtime is an exception. Doing so make it the identical behavior
32915 as setting ModTime.
32916 rclone reserves all the keys starting with rclone-. Setting value for
32918 these keys will give you warnings, but values are set according to re‐
32919 quest.
32920 If there are multiple values for a key, only the first one is returned.
32922 This is a limitation of rclone, that supports one value per one key.
32923 It can be triggered when you did a server-side copy.
32924 Reading metadata will also provide custom (non-standard nor reserved)
32926 ones.
32927 Configuration
32929 Here is an example of making an internetarchive configuration. Most
32930 applies to the other providers as well, any differences are described
32931 below.
32932 First run
32934 rclone config
32936 This will guide you through an interactive setup process.
32938 No remotes found, make a new one?
32940 n) New remote
32941 s) Set configuration password
32942 q) Quit config
32943 n/s/q> n
32944 name> remote
32945 Option Storage.
32946 Type of storage to configure.
32947 Choose a number from below, or type in your own value.
32948 XX / InternetArchive Items
32949 \ (internetarchive)
32950 Storage> internetarchive
32951 Option access_key_id.
32952 IAS3 Access Key.
32953 Leave blank for anonymous access.
32954 You can find one here: https://archive.org/account/s3.php
32955 Enter a value. Press Enter to leave empty.
32956 access_key_id> XXXX
32957 Option secret_access_key.
32958 IAS3 Secret Key (password).
32959 Leave blank for anonymous access.
32960 Enter a value. Press Enter to leave empty.
32961 secret_access_key> XXXX
32962 Edit advanced config?
32963 y) Yes
32964 n) No (default)
32965 y/n> y
32966 Option endpoint.
32967 IAS3 Endpoint.
32968 Leave blank for default value.
32969 Enter a string value. Press Enter for the default (https://s3.us.archive.org).
32970 endpoint>
32971 Option front_endpoint.
32972 Host of InternetArchive Frontend.
32973 Leave blank for default value.
32974 Enter a string value. Press Enter for the default (https://archive.org).
32975 front_endpoint>
32976 Option disable_checksum.
32977 Don't store MD5 checksum with object metadata.
32978 Normally rclone will calculate the MD5 checksum of the input before
32979 uploading it so it can ask the server to check the object against checksum.
32980 This is great for data integrity checking but can cause long delays for
32981 large files to start uploading.
32982 Enter a boolean value (true or false). Press Enter for the default (true).
32983 disable_checksum> true
32984 Option encoding.
32985 The encoding for the backend.
32986 See the [encoding section in the overview](https://rclone.org/overview/#encoding) for more info.
32987 Enter a encoder.MultiEncoder value. Press Enter for the default (Slash,Question,Hash,Percent,Del,Ctl,InvalidUtf8,Dot).
32988 encoding>
32989 Edit advanced config?
32990 y) Yes
32991 n) No (default)
32992 y/n> n
32993 --------------------
32994 [remote]
32995 type = internetarchive
32996 access_key_id = XXXX
32997 secret_access_key = XXXX
32998 --------------------
32999 y) Yes this is OK (default)
33000 e) Edit this remote
33001 d) Delete this remote
33002 y/e/d> y
33003 Standard options
33005 Here are the Standard options specific to internetarchive (Internet Ar‐
33006 chive).
33007 --internetarchive-access-key-id
33009 IAS3 Access Key.
33010 Leave blank for anonymous access. You can find one here: https://ar‐
33012 chive.org/account/s3.php
33013 Properties:
33015 • Config: access_key_id
33017 • Env Var: RCLONE_INTERNETARCHIVE_ACCESS_KEY_ID
33019 • Type: string
33021 • Required: false
33023 --internetarchive-secret-access-key
33025 IAS3 Secret Key (password).
33026 Leave blank for anonymous access.
33028 Properties:
33030 • Config: secret_access_key
33032 • Env Var: RCLONE_INTERNETARCHIVE_SECRET_ACCESS_KEY
33034 • Type: string
33036 • Required: false
33038 Advanced options
33040 Here are the Advanced options specific to internetarchive (Internet Ar‐
33041 chive).
33042 --internetarchive-endpoint
33044 IAS3 Endpoint.
33045 Leave blank for default value.
33047 Properties:
33049 • Config: endpoint
33051 • Env Var: RCLONE_INTERNETARCHIVE_ENDPOINT
33053 • Type: string
33055 • Default: "https://s3.us.archive.org"
33057 --internetarchive-front-endpoint
33059 Host of InternetArchive Frontend.
33060 Leave blank for default value.
33062 Properties:
33064 • Config: front_endpoint
33066 • Env Var: RCLONE_INTERNETARCHIVE_FRONT_ENDPOINT
33068 • Type: string
33070 • Default: "https://archive.org"
33072 --internetarchive-disable-checksum
33074 Don't ask the server to test against MD5 checksum calculated by rclone.
33075 Normally rclone will calculate the MD5 checksum of the input before up‐
33076 loading it so it can ask the server to check the object against check‐
33077 sum. This is great for data integrity checking but can cause long de‐
33078 lays for large files to start uploading.
33079 Properties:
33081 • Config: disable_checksum
33083 • Env Var: RCLONE_INTERNETARCHIVE_DISABLE_CHECKSUM
33085 • Type: bool
33087 • Default: true
33089 --internetarchive-wait-archive
33091 Timeout for waiting the server's processing tasks (specifically archive
33092 and book_op) to finish. Only enable if you need to be guaranteed to be
33093 reflected after write operations. 0 to disable waiting. No errors to
33094 be thrown in case of timeout.
33095 Properties:
33097 • Config: wait_archive
33099 • Env Var: RCLONE_INTERNETARCHIVE_WAIT_ARCHIVE
33101 • Type: Duration
33103 • Default: 0s
33105 --internetarchive-encoding
33107 The encoding for the backend.
33108 See the encoding section in the overview (https://rclone.org/over‐
33110 view/#encoding) for more info.
33111 Properties:
33113 • Config: encoding
33115 • Env Var: RCLONE_INTERNETARCHIVE_ENCODING
33117 • Type: MultiEncoder
33119 • Default: Slash,LtGt,CrLf,Del,Ctl,InvalidUtf8,Dot
33121 Metadata
33123 Metadata fields provided by Internet Archive. If there are multiple
33124 values for a key, only the first one is returned. This is a limitation
33125 of Rclone, that supports one value per one key.
33126 Owner is able to add custom keys. Metadata feature grabs all the keys
33128 including them.
33129 Here are the possible system metadata items for the internetarchive
33131 backend.
33132 Name Help Type Example Read Only
33134 ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
33135 crc32 CRC32 cal‐ string 01234567 Y
33136 culated by
33137 Internet
33138 Archive
33139 format Name of string Comma-Separated Y
33140 format Values
33141 identified
33142 by Internet
33143 Archive
33144 md5 MD5 hash string 01234567012345670123456701234567 Y
33145 calculated
33146 by Internet
33147 Archive
33148 mtime Time of RFC 3339 2006-01-02T15:04:05.999999999Z Y
33149 last modi‐
33150 fication,
33151 managed by
33152 Rclone
33153 name Full file filename backend/internetarchive/inter‐ Y
33154 path, with‐ netarchive.go
33155 out the
33156 bucket part
33157 old_version Whether the boolean true Y
33162 file was
33163 replaced
33164 and moved
33165 by keep-
33166 old-version
33167 flag
33168 rclone-ia- Time of RFC 3339 2006-01-02T15:04:05.999999999Z N
33169 mtime last modi‐
33170 fication,
33171 managed by
33172 Internet
33173 Archive
33174 rclone- Time of RFC 3339 2006-01-02T15:04:05.999999999Z N
33175 mtime last modi‐
33176 fication,
33177 managed by
33178 Rclone
33179 rclone-up‐ Random val‐ string aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa N
33180 date-track ue used by
33181 Rclone for
33182 tracking
33183 changes in‐
33184 side Inter‐
33185 net Archive
33186 sha1 SHA1 hash string 0123456701234567012345670123456701234567 Y
33187 calculated
33188 by Internet
33189 Archive
33190 size File size decimal 123456 Y
33191 in bytes number
33192 source The source string original Y
33193 of the file
33194 summation Check string md5 Y
33195 https://fo‐
33196 rum.rclone.org/t/31922
33197 for how it
33198 is used
33199 viruscheck The last time vir‐ unixtime 1654191352 Y
33200 uscheck process was
33201 run for the file (?)
33202 See the metadata (https://rclone.org/docs/#metadata) docs for more in‐
33204 fo.
33205
33207 Jottacloud is a cloud storage service provider from a Norwegian compa‐
33208 ny, using its own datacenters in Norway. In addition to the official
33209 service at jottacloud.com (https://www.jottacloud.com/), it also pro‐
33210 vides white-label solutions to different companies, such as: * Telia *
33211 Telia Cloud (cloud.telia.se) * Telia Sky (sky.telia.no) * Tele2 * Tele2
33212 Cloud (mittcloud.tele2.se) * Elkjøp (with subsidiaries): * Elkjøp Cloud
33213 (cloud.elkjop.no) * Elgiganten Sweden (cloud.elgiganten.se) * Elgigan‐
33214 ten Denmark (cloud.elgiganten.dk) * Giganti Cloud (cloud.gigantti.fi) *
33215 ELKO Cloud (cloud.elko.is)
33216 Most of the white-label versions are supported by this backend, al‐
33218 though may require different authentication setup - described below.
33219 Paths are specified as remote:path
33221 Paths may be as deep as required, e.g. remote:directory/subdirectory.
33223 Authentication types
33225 Some of the whitelabel versions uses a different authentication method
33226 than the official service, and you have to choose the correct one when
33227 setting up the remote.
33228 Standard authentication
33230 The standard authentication method used by the official service (jotta‐
33231 cloud.com), as well as some of the whitelabel services, requires you to
33232 generate a single-use personal login token from the account security
33233 settings in the service's web interface. Log in to your account, go to
33234 "Settings" and then "Security", or use the direct link presented to you
33235 by rclone when configuring the remote: <https://www.jotta‐
33236 cloud.com/web/secure>. Scroll down to the section "Personal login to‐
33237 ken", and click the "Generate" button. Note that if you are using a
33238 whitelabel service you probably can't use the direct link, you need to
33239 find the same page in their dedicated web interface, and also it may be
33240 in a different location than described above.
33241 To access your account from multiple instances of rclone, you need to
33243 configure each of them with a separate personal login token. E.g. you
33244 create a Jottacloud remote with rclone in one location, and copy the
33245 configuration file to a second location where you also want to run
33246 rclone and access the same remote. Then you need to replace the token
33247 for one of them, using the config reconnect (https://rclone.org/com‐
33248 mands/rclone_config_reconnect/) command, which requires you to generate
33249 a new personal login token and supply as input. If you do not do this,
33250 the token may easily end up being invalidated, resulting in both in‐
33251 stances failing with an error message something along the lines of:
33252 oauth2: cannot fetch token: 400 Bad Request
33254 Response: {"error":"invalid_grant","error_description":"Stale token"}
33255 When this happens, you need to replace the token as described above to
33257 be able to use your remote again.
33258 All personal login tokens you have taken into use will be listed in the
33260 web interface under "My logged in devices", and from the right side of
33261 that list you can click the "X" button to revoke individual tokens.
33262 Legacy authentication
33264 If you are using one of the whitelabel versions (e.g. from Elkjøp) you
33265 may not have the option to generate a CLI token. In this case you'll
33266 have to use the legacy authentication. To do this select yes when the
33267 setup asks for legacy authentication and enter your username and pass‐
33268 word. The rest of the setup is identical to the default setup.
33269 Telia Cloud authentication
33271 Similar to other whitelabel versions Telia Cloud doesn't offer the op‐
33272 tion of creating a CLI token, and additionally uses a separate authen‐
33273 tication flow where the username is generated internally. To setup
33274 rclone to use Telia Cloud, choose Telia Cloud authentication in the
33275 setup. The rest of the setup is identical to the default setup.
33276 Tele2 Cloud authentication
33278 As Tele2-Com Hem merger was completed this authentication can be used
33279 for former Com Hem Cloud and Tele2 Cloud customers as no support for
33280 creating a CLI token exists, and additionally uses a separate authenti‐
33281 cation flow where the username is generated internally. To setup
33282 rclone to use Tele2 Cloud, choose Tele2 Cloud authentication in the
33283 setup. The rest of the setup is identical to the default setup.
33284 Configuration
33286 Here is an example of how to make a remote called remote with the de‐
33287 fault setup. First run:
33288 rclone config
33290 This will guide you through an interactive setup process:
33292 No remotes found, make a new one?
33294 n) New remote
33295 s) Set configuration password
33296 q) Quit config
33297 n/s/q> n
33298 name> remote
33299 Option Storage.
33300 Type of storage to configure.
33301 Choose a number from below, or type in your own value.
33302 [snip]
33303 XX / Jottacloud
33304 \ (jottacloud)
33305 [snip]
33306 Storage> jottacloud
33307 Edit advanced config?
33308 y) Yes
33309 n) No (default)
33310 y/n> n
33311 Option config_type.
33312 Select authentication type.
33313 Choose a number from below, or type in an existing string value.
33314 Press Enter for the default (standard).
33315 / Standard authentication.
33316 1 | Use this if you're a normal Jottacloud user.
33317 \ (standard)
33318 / Legacy authentication.
33319 2 | This is only required for certain whitelabel versions of Jottacloud and not recommended for normal users.
33320 \ (legacy)
33321 / Telia Cloud authentication.
33322 3 | Use this if you are using Telia Cloud.
33323 \ (telia)
33324 / Tele2 Cloud authentication.
33325 4 | Use this if you are using Tele2 Cloud.
33326 \ (tele2)
33327 config_type> 1
33328 Personal login token.
33329 Generate here: https://www.jottacloud.com/web/secure
33330 Login Token> <your token here>
33331 Use a non-standard device/mountpoint?
33332 Choosing no, the default, will let you access the storage used for the archive
33333 section of the official Jottacloud client. If you instead want to access the
33334 sync or the backup section, for example, you must choose yes.
33335 y) Yes
33336 n) No (default)
33337 y/n> y
33338 Option config_device.
33339 The device to use. In standard setup the built-in Jotta device is used,
33340 which contains predefined mountpoints for archive, sync etc. All other devices
33341 are treated as backup devices by the official Jottacloud client. You may create
33342 a new by entering a unique name.
33343 Choose a number from below, or type in your own string value.
33344 Press Enter for the default (DESKTOP-3H31129).
33345 1 > DESKTOP-3H31129
33346 2 > Jotta
33347 config_device> 2
33348 Option config_mountpoint.
33349 The mountpoint to use for the built-in device Jotta.
33350 The standard setup is to use the Archive mountpoint. Most other mountpoints
33351 have very limited support in rclone and should generally be avoided.
33352 Choose a number from below, or type in an existing string value.
33353 Press Enter for the default (Archive).
33354 1 > Archive
33355 2 > Shared
33356 3 > Sync
33357 config_mountpoint> 1
33358 --------------------
33359 [remote]
33360 type = jottacloud
33361 configVersion = 1
33362 client_id = jottacli
33363 client_secret =
33364 tokenURL = https://id.jottacloud.com/auth/realms/jottacloud/protocol/openid-connect/token
33365 token = {........}
33366 username = 2940e57271a93d987d6f8a21
33367 device = Jotta
33368 mountpoint = Archive
33369 --------------------
33370 y) Yes this is OK (default)
33371 e) Edit this remote
33372 d) Delete this remote
33373 y/e/d> y
33374 Once configured you can then use rclone like this,
33376 List directories in top level of your Jottacloud
33378 rclone lsd remote:
33380 List all the files in your Jottacloud
33382 rclone ls remote:
33384 To copy a local directory to an Jottacloud directory called backup
33386 rclone copy /home/source remote:backup
33388 Devices and Mountpoints
33390 The official Jottacloud client registers a device for each computer you
33391 install it on, and shows them in the backup section of the user inter‐
33392 face. For each folder you select for backup it will create a mount‐
33393 point within this device. A built-in device called Jotta is special,
33394 and contains mountpoints Archive, Sync and some others, used for corre‐
33395 sponding features in official clients.
33396 With rclone you'll want to use the standard Jotta/Archive device/mount‐
33398 point in most cases. However, you may for example want to access files
33399 from the sync or backup functionality provided by the official clients,
33400 and rclone therefore provides the option to select other devices and
33401 mountpoints during config.
33402 You are allowed to create new devices and mountpoints. All devices ex‐
33404 cept the built-in Jotta device are treated as backup devices by offi‐
33405 cial Jottacloud clients, and the mountpoints on them are individual
33406 backup sets.
33407 With the built-in Jotta device, only existing, built-in, mountpoints
33409 can be selected. In addition to the mentioned Archive and Sync, it may
33410 contain several other mountpoints such as: Latest, Links, Shared and
33411 Trash. All of these are special mountpoints with a different internal
33412 representation than the "regular" mountpoints. Rclone will only to a
33413 very limited degree support them. Generally you should avoid these,
33414 unless you know what you are doing.
33415 --fast-list
33417 This remote supports --fast-list which allows you to use fewer transac‐
33418 tions in exchange for more memory. See the rclone docs
33419 (https://rclone.org/docs/#fast-list) for more details.
33420 Note that the implementation in Jottacloud always uses only a single
33422 API request to get the entire list, so for large folders this could
33423 lead to long wait time before the first results are shown.
33424 Note also that with rclone version 1.58 and newer information about
33426 MIME types (https://rclone.org/overview/#mime-type) are not available
33427 when using --fast-list.
33428 Modified time and hashes
33430 Jottacloud allows modification times to be set on objects accurate to 1
33431 second. These will be used to detect whether objects need syncing or
33432 not.
33433 Jottacloud supports MD5 type hashes, so you can use the --checksum
33435 flag.
33436 Note that Jottacloud requires the MD5 hash before upload so if the
33438 source does not have an MD5 checksum then the file will be cached tem‐
33439 porarily on disk (in location given by --temp-dir
33440 (https://rclone.org/docs/#temp-dir-dir)) before it is uploaded. Small
33441 files will be cached in memory - see the --jottacloud-md5-memory-limit
33442 flag. When uploading from local disk the source checksum is always
33443 available, so this does not apply. Starting with rclone version 1.52
33444 the same is true for crypted remotes (in older versions the crypt back‐
33445 end would not calculate hashes for uploads from local disk, so the Jot‐
33446 tacloud backend had to do it as described above).
33447 Restricted filename characters
33449 In addition to the default restricted characters set
33450 (https://rclone.org/overview/#restricted-characters) the following
33451 characters are also replaced:
33452 Character Value Replacement
33454 ────────────────────────────────
33455 " 0x22 ＂
33456 * 0x2A ＊
33457 : 0x3A ：
33458 < 0x3C ＜
33459 > 0x3E ＞
33460 ? 0x3F ？
33461 | 0x7C ｜
33462 Invalid UTF-8 bytes will also be replaced (https://rclone.org/over‐
33464 view/#invalid-utf8), as they can't be used in XML strings.
33465 Deleting files
33467 By default, rclone will send all files to the trash when deleting
33468 files. They will be permanently deleted automatically after 30 days.
33469 You may bypass the trash and permanently delete files immediately by
33470 using the --jottacloud-hard-delete flag, or set the equivalent environ‐
33471 ment variable. Emptying the trash is supported by the cleanup
33472 (https://rclone.org/commands/rclone_cleanup/) command.
33473 Versions
33475 Jottacloud supports file versioning. When rclone uploads a new version
33476 of a file it creates a new version of it. Currently rclone only sup‐
33477 ports retrieving the current version but older versions can be accessed
33478 via the Jottacloud Website.
33479 Versioning can be disabled by --jottacloud-no-versions option. This is
33481 achieved by deleting the remote file prior to uploading a new version.
33482 If the upload the fails no version of the file will be available in the
33483 remote.
33484 Quota information
33486 To view your current quota you can use the rclone about remote: command
33487 which will display your usage limit (unless it is unlimited) and the
33488 current usage.
33489 Advanced options
33491 Here are the Advanced options specific to jottacloud (Jottacloud).
33492 --jottacloud-md5-memory-limit
33494 Files bigger than this will be cached on disk to calculate the MD5 if
33495 required.
33496 Properties:
33498 • Config: md5_memory_limit
33500 • Env Var: RCLONE_JOTTACLOUD_MD5_MEMORY_LIMIT
33502 • Type: SizeSuffix
33504 • Default: 10Mi
33506 --jottacloud-trashed-only
33508 Only show files that are in the trash.
33509 This will show trashed files in their original directory structure.
33511 Properties:
33513 • Config: trashed_only
33515 • Env Var: RCLONE_JOTTACLOUD_TRASHED_ONLY
33517 • Type: bool
33519 • Default: false
33521 --jottacloud-hard-delete
33523 Delete files permanently rather than putting them into the trash.
33524 Properties:
33526 • Config: hard_delete
33528 • Env Var: RCLONE_JOTTACLOUD_HARD_DELETE
33530 • Type: bool
33532 • Default: false
33534 --jottacloud-upload-resume-limit
33536 Files bigger than this can be resumed if the upload fail's.
33537 Properties:
33539 • Config: upload_resume_limit
33541 • Env Var: RCLONE_JOTTACLOUD_UPLOAD_RESUME_LIMIT
33543 • Type: SizeSuffix
33545 • Default: 10Mi
33547 --jottacloud-no-versions
33549 Avoid server side versioning by deleting files and recreating files in‐
33550 stead of overwriting them.
33551 Properties:
33553 • Config: no_versions
33555 • Env Var: RCLONE_JOTTACLOUD_NO_VERSIONS
33557 • Type: bool
33559 • Default: false
33561 --jottacloud-encoding
33563 The encoding for the backend.
33564 See the encoding section in the overview (https://rclone.org/over‐
33566 view/#encoding) for more info.
33567 Properties:
33569 • Config: encoding
33571 • Env Var: RCLONE_JOTTACLOUD_ENCODING
33573 • Type: MultiEncoder
33575 • Default: Slash,LtGt,DoubleQuote,Colon,Question,Aster‐
33577 isk,Pipe,Del,Ctl,InvalidUtf8,Dot
33578 Limitations
33580 Note that Jottacloud is case insensitive so you can't have a file
33581 called "Hello.doc" and one called "hello.doc".
33582 There are quite a few characters that can't be in Jottacloud file
33584 names. Rclone will map these names to and from an identical looking
33585 unicode equivalent. For example if a file has a ? in it will be
33586 mapped to ？ instead.
33587 Jottacloud only supports filenames up to 255 characters in length.
33589 Troubleshooting
33591 Jottacloud exhibits some inconsistent behaviours regarding deleted
33592 files and folders which may cause Copy, Move and DirMove operations to
33593 previously deleted paths to fail. Emptying the trash should help in
33594 such cases.
33595
33597 Paths are specified as remote:path
33598 Paths may be as deep as required, e.g. remote:directory/subdirectory.
33600 Configuration
33602 The initial setup for Koofr involves creating an application password
33603 for rclone. You can do that by opening the Koofr web application
33604 (https://app.koofr.net/app/admin/preferences/password), giving the
33605 password a nice name like rclone and clicking on generate.
33606 Here is an example of how to make a remote called koofr. First run:
33608 rclone config
33610 This will guide you through an interactive setup process:
33612 No remotes found, make a new one?
33614 n) New remote
33615 s) Set configuration password
33616 q) Quit config
33617 n/s/q> n
33618 name> koofr
33619 Option Storage.
33620 Type of storage to configure.
33621 Choose a number from below, or type in your own value.
33622 [snip]
33623 22 / Koofr, Digi Storage and other Koofr-compatible storage providers
33624 \ (koofr)
33625 [snip]
33626 Storage> koofr
33627 Option provider.
33628 Choose your storage provider.
33629 Choose a number from below, or type in your own value.
33630 Press Enter to leave empty.
33631 1 / Koofr, https://app.koofr.net/
33632 \ (koofr)
33633 2 / Digi Storage, https://storage.rcs-rds.ro/
33634 \ (digistorage)
33635 3 / Any other Koofr API compatible storage service
33636 \ (other)
33637 provider> 1
33638 Option user.
33639 Your user name.
33640 Enter a value.
33641 user> USERNAME
33642 Option password.
33643 Your password for rclone (generate one at https://app.koofr.net/app/admin/preferences/password).
33644 Choose an alternative below.
33645 y) Yes, type in my own password
33646 g) Generate random password
33647 y/g> y
33648 Enter the password:
33649 password:
33650 Confirm the password:
33651 password:
33652 Edit advanced config?
33653 y) Yes
33654 n) No (default)
33655 y/n> n
33656 Remote config
33657 --------------------
33658 [koofr]
33659 type = koofr
33660 provider = koofr
33661 user = USERNAME
33662 password = *** ENCRYPTED ***
33663 --------------------
33664 y) Yes this is OK (default)
33665 e) Edit this remote
33666 d) Delete this remote
33667 y/e/d> y
33668 You can choose to edit advanced config in order to enter your own ser‐
33670 vice URL if you use an on-premise or white label Koofr instance, or
33671 choose an alternative mount instead of your primary storage.
33672 Once configured you can then use rclone like this,
33674 List directories in top level of your Koofr
33676 rclone lsd koofr:
33678 List all the files in your Koofr
33680 rclone ls koofr:
33682 To copy a local directory to an Koofr directory called backup
33684 rclone copy /home/source koofr:backup
33686 Restricted filename characters
33688 In addition to the default restricted characters set
33689 (https://rclone.org/overview/#restricted-characters) the following
33690 characters are also replaced:
33691 Character Value Replacement
33693 ────────────────────────────────
33694 \ 0x5C ＼
33695 Invalid UTF-8 bytes will also be replaced (https://rclone.org/over‐
33697 view/#invalid-utf8), as they can't be used in XML strings.
33698 Standard options
33700 Here are the Standard options specific to koofr (Koofr, Digi Storage
33701 and other Koofr-compatible storage providers).
33702 --koofr-provider
33704 Choose your storage provider.
33705 Properties:
33707 • Config: provider
33709 • Env Var: RCLONE_KOOFR_PROVIDER
33711 • Type: string
33713 • Required: false
33715 • Examples:
33717 • "koofr"
33719 • Koofr, https://app.koofr.net/
33721 • "digistorage"
33723 • Digi Storage, https://storage.rcs-rds.ro/
33725 • "other"
33727 • Any other Koofr API compatible storage service
33729 --koofr-endpoint
33731 The Koofr API endpoint to use.
33732 Properties:
33734 • Config: endpoint
33736 • Env Var: RCLONE_KOOFR_ENDPOINT
33738 • Provider: other
33740 • Type: string
33742 • Required: true
33744 --koofr-user
33746 Your user name.
33747 Properties:
33749 • Config: user
33751 • Env Var: RCLONE_KOOFR_USER
33753 • Type: string
33755 • Required: true
33757 --koofr-password
33759 Your password for rclone (generate one at https://app.koofr.net/app/ad‐
33760 min/preferences/password).
33761 NB Input to this must be obscured - see rclone obscure
33763 (https://rclone.org/commands/rclone_obscure/).
33764 Properties:
33766 • Config: password
33768 • Env Var: RCLONE_KOOFR_PASSWORD
33770 • Provider: koofr
33772 • Type: string
33774 • Required: true
33776 --koofr-password
33778 Your password for rclone (generate one at https://storage.rcs-
33779 rds.ro/app/admin/preferences/password).
33780 NB Input to this must be obscured - see rclone obscure
33782 (https://rclone.org/commands/rclone_obscure/).
33783 Properties:
33785 • Config: password
33787 • Env Var: RCLONE_KOOFR_PASSWORD
33789 • Provider: digistorage
33791 • Type: string
33793 • Required: true
33795 --koofr-password
33797 Your password for rclone (generate one at your service's settings
33798 page).
33799 NB Input to this must be obscured - see rclone obscure
33801 (https://rclone.org/commands/rclone_obscure/).
33802 Properties:
33804 • Config: password
33806 • Env Var: RCLONE_KOOFR_PASSWORD
33808 • Provider: other
33810 • Type: string
33812 • Required: true
33814 Advanced options
33816 Here are the Advanced options specific to koofr (Koofr, Digi Storage
33817 and other Koofr-compatible storage providers).
33818 --koofr-mountid
33820 Mount ID of the mount to use.
33821 If omitted, the primary mount is used.
33823 Properties:
33825 • Config: mountid
33827 • Env Var: RCLONE_KOOFR_MOUNTID
33829 • Type: string
33831 • Required: false
33833 --koofr-setmtime
33835 Does the backend support setting modification time.
33836 Set this to false if you use a mount ID that points to a Dropbox or
33838 Amazon Drive backend.
33839 Properties:
33841 • Config: setmtime
33843 • Env Var: RCLONE_KOOFR_SETMTIME
33845 • Type: bool
33847 • Default: true
33849 --koofr-encoding
33851 The encoding for the backend.
33852 See the encoding section in the overview (https://rclone.org/over‐
33854 view/#encoding) for more info.
33855 Properties:
33857 • Config: encoding
33859 • Env Var: RCLONE_KOOFR_ENCODING
33861 • Type: MultiEncoder
33863 • Default: Slash,BackSlash,Del,Ctl,InvalidUtf8,Dot
33865 Limitations
33867 Note that Koofr is case insensitive so you can't have a file called
33868 "Hello.doc" and one called "hello.doc".
33869 Providers
33871 Koofr
33872 This is the original Koofr (https://koofr.eu) storage provider used as
33873 main example and described in the configuration section above.
33874 Digi Storage
33876 Digi Storage (https://www.digi.ro/servicii/online/digi-storage) is a
33877 cloud storage service run by Digi.ro (https://www.digi.ro/) that pro‐
33878 vides a Koofr API.
33879 Here is an example of how to make a remote called ds. First run:
33881 rclone config
33883 This will guide you through an interactive setup process:
33885 No remotes found, make a new one?
33887 n) New remote
33888 s) Set configuration password
33889 q) Quit config
33890 n/s/q> n
33891 name> ds
33892 Option Storage.
33893 Type of storage to configure.
33894 Choose a number from below, or type in your own value.
33895 [snip]
33896 22 / Koofr, Digi Storage and other Koofr-compatible storage providers
33897 \ (koofr)
33898 [snip]
33899 Storage> koofr
33900 Option provider.
33901 Choose your storage provider.
33902 Choose a number from below, or type in your own value.
33903 Press Enter to leave empty.
33904 1 / Koofr, https://app.koofr.net/
33905 \ (koofr)
33906 2 / Digi Storage, https://storage.rcs-rds.ro/
33907 \ (digistorage)
33908 3 / Any other Koofr API compatible storage service
33909 \ (other)
33910 provider> 2
33911 Option user.
33912 Your user name.
33913 Enter a value.
33914 user> USERNAME
33915 Option password.
33916 Your password for rclone (generate one at https://storage.rcs-rds.ro/app/admin/preferences/password).
33917 Choose an alternative below.
33918 y) Yes, type in my own password
33919 g) Generate random password
33920 y/g> y
33921 Enter the password:
33922 password:
33923 Confirm the password:
33924 password:
33925 Edit advanced config?
33926 y) Yes
33927 n) No (default)
33928 y/n> n
33929 --------------------
33930 [ds]
33931 type = koofr
33932 provider = digistorage
33933 user = USERNAME
33934 password = *** ENCRYPTED ***
33935 --------------------
33936 y) Yes this is OK (default)
33937 e) Edit this remote
33938 d) Delete this remote
33939 y/e/d> y
33940 Other
33942 You may also want to use another, public or private storage provider
33943 that runs a Koofr API compatible service, by simply providing the base
33944 URL to connect to.
33945 Here is an example of how to make a remote called other. First run:
33947 rclone config
33949 This will guide you through an interactive setup process:
33951 No remotes found, make a new one?
33953 n) New remote
33954 s) Set configuration password
33955 q) Quit config
33956 n/s/q> n
33957 name> other
33958 Option Storage.
33959 Type of storage to configure.
33960 Choose a number from below, or type in your own value.
33961 [snip]
33962 22 / Koofr, Digi Storage and other Koofr-compatible storage providers
33963 \ (koofr)
33964 [snip]
33965 Storage> koofr
33966 Option provider.
33967 Choose your storage provider.
33968 Choose a number from below, or type in your own value.
33969 Press Enter to leave empty.
33970 1 / Koofr, https://app.koofr.net/
33971 \ (koofr)
33972 2 / Digi Storage, https://storage.rcs-rds.ro/
33973 \ (digistorage)
33974 3 / Any other Koofr API compatible storage service
33975 \ (other)
33976 provider> 3
33977 Option endpoint.
33978 The Koofr API endpoint to use.
33979 Enter a value.
33980 endpoint> https://koofr.other.org
33981 Option user.
33982 Your user name.
33983 Enter a value.
33984 user> USERNAME
33985 Option password.
33986 Your password for rclone (generate one at your service's settings page).
33987 Choose an alternative below.
33988 y) Yes, type in my own password
33989 g) Generate random password
33990 y/g> y
33991 Enter the password:
33992 password:
33993 Confirm the password:
33994 password:
33995 Edit advanced config?
33996 y) Yes
33997 n) No (default)
33998 y/n> n
33999 --------------------
34000 [other]
34001 type = koofr
34002 provider = other
34003 endpoint = https://koofr.other.org
34004 user = USERNAME
34005 password = *** ENCRYPTED ***
34006 --------------------
34007 y) Yes this is OK (default)
34008 e) Edit this remote
34009 d) Delete this remote
34010 y/e/d> y
34011
34013 Mail.ru Cloud (https://cloud.mail.ru/) is a cloud storage provided by a
34014 Russian internet company Mail.Ru Group (https://mail.ru). The official
34015 desktop client is Disk-O: (https://disk-o.cloud/en), available on Win‐
34016 dows and Mac OS.
34017 Currently it is recommended to disable 2FA on Mail.ru accounts intended
34019 for rclone until it gets eventually implemented.
34020 Features highlights
34022 • Paths may be as deep as required, e.g. remote:directory/subdirectory
34023 • Files have a last modified time property, directories don't
34025 • Deleted files are by default moved to the trash
34027 • Files and directories can be shared via public links
34029 • Partial uploads or streaming are not supported, file size must be
34031 known before upload
34032 • Maximum file size is limited to 2G for a free account, unlimited for
34034 paid accounts
34035 • Storage keeps hash for all files and performs transparent deduplica‐
34037 tion, the hash algorithm is a modified SHA1
34038 • If a particular file is already present in storage, one can quickly
34040 submit file hash instead of long file upload (this optimization is
34041 supported by rclone)
34042 Configuration
34044 Here is an example of making a mailru configuration.
34045 First create a Mail.ru Cloud account and choose a tariff.
34047 You will need to log in and create an app password for rclone. Rclone
34049 will not work with your normal username and password - it will give an
34050 error like oauth2: server response missing access_token.
34051 • Click on your user icon in the top right
34053 • Go to Security / "Пароль и безопасность"
34055 • Click password for apps / "Пароли для внешних приложений"
34057 • Add the password - give it a name - eg "rclone"
34059 • Copy the password and use this password below - your normal login
34061 password won't work.
34062 Now run
34064 rclone config
34066 This will guide you through an interactive setup process:
34068 No remotes found, make a new one?
34070 n) New remote
34071 s) Set configuration password
34072 q) Quit config
34073 n/s/q> n
34074 name> remote
34075 Type of storage to configure.
34076 Type of storage to configure.
34077 Enter a string value. Press Enter for the default ("").
34078 Choose a number from below, or type in your own value
34079 [snip]
34080 XX / Mail.ru Cloud
34081 \ "mailru"
34082 [snip]
34083 Storage> mailru
34084 User name (usually email)
34085 Enter a string value. Press Enter for the default ("").
34086 user> username@mail.ru
34087 Password
34088 This must be an app password - rclone will not work with your normal
34090 password. See the Configuration section in the docs for how to make an
34091 app password.
34092 y) Yes type in my own password
34093 g) Generate random password
34094 y/g> y
34095 Enter the password:
34096 password:
34097 Confirm the password:
34098 password:
34099 Skip full upload if there is another file with same data hash.
34100 This feature is called "speedup" or "put by hash". It is especially efficient
34101 in case of generally available files like popular books, video or audio clips
34102 [snip]
34103 Enter a boolean value (true or false). Press Enter for the default ("true").
34104 Choose a number from below, or type in your own value
34105 1 / Enable
34106 \ "true"
34107 2 / Disable
34108 \ "false"
34109 speedup_enable> 1
34110 Edit advanced config? (y/n)
34111 y) Yes
34112 n) No
34113 y/n> n
34114 Remote config
34115 --------------------
34116 [remote]
34117 type = mailru
34118 user = username@mail.ru
34119 pass = *** ENCRYPTED ***
34120 speedup_enable = true
34121 --------------------
34122 y) Yes this is OK
34123 e) Edit this remote
34124 d) Delete this remote
34125 y/e/d> y
34126 Configuration of this backend does not require a local web browser.
34128 You can use the configured backend as shown below:
34129 See top level directories
34131 rclone lsd remote:
34133 Make a new directory
34135 rclone mkdir remote:directory
34137 List the contents of a directory
34139 rclone ls remote:directory
34141 Sync /home/local/directory to the remote path, deleting any excess
34143 files in the path.
34144 rclone sync -i /home/local/directory remote:directory
34146 Modified time
34148 Files support a modification time attribute with up to 1 second preci‐
34149 sion. Directories do not have a modification time, which is shown as
34150 "Jan 1 1970".
34151 Hash checksums
34153 Hash sums use a custom Mail.ru algorithm based on SHA1. If file size
34154 is less than or equal to the SHA1 block size (20 bytes), its hash is
34155 simply its data right-padded with zero bytes. Hash sum of a larger
34156 file is computed as a SHA1 sum of the file data bytes concatenated with
34157 a decimal representation of the data length.
34158 Emptying Trash
34160 Removing a file or directory actually moves it to the trash, which is
34161 not visible to rclone but can be seen in a web browser. The trashed
34162 file still occupies part of total quota. If you wish to empty your
34163 trash and free some quota, you can use the rclone cleanup remote: com‐
34164 mand, which will permanently delete all your trashed files. This com‐
34165 mand does not take any path arguments.
34166 Quota information
34168 To view your current quota you can use the rclone about remote: command
34169 which will display your usage limit (quota) and the current usage.
34170 Restricted filename characters
34172 In addition to the default restricted characters set
34173 (https://rclone.org/overview/#restricted-characters) the following
34174 characters are also replaced:
34175 Character Value Replacement
34177 ────────────────────────────────
34178 " 0x22 ＂
34179 * 0x2A ＊
34180 : 0x3A ：
34181 < 0x3C ＜
34182 > 0x3E ＞
34183 ? 0x3F ？
34184 \ 0x5C ＼
34185 | 0x7C ｜
34186 Invalid UTF-8 bytes will also be replaced (https://rclone.org/over‐
34188 view/#invalid-utf8), as they can't be used in JSON strings.
34189 Standard options
34191 Here are the Standard options specific to mailru (Mail.ru Cloud).
34192 --mailru-user
34194 User name (usually email).
34195 Properties:
34197 • Config: user
34199 • Env Var: RCLONE_MAILRU_USER
34201 • Type: string
34203 • Required: true
34205 --mailru-pass
34207 Password.
34208 This must be an app password - rclone will not work with your normal
34210 password. See the Configuration section in the docs for how to make an
34211 app password.
34212 NB Input to this must be obscured - see rclone obscure
34214 (https://rclone.org/commands/rclone_obscure/).
34215 Properties:
34217 • Config: pass
34219 • Env Var: RCLONE_MAILRU_PASS
34221 • Type: string
34223 • Required: true
34225 --mailru-speedup-enable
34227 Skip full upload if there is another file with same data hash.
34228 This feature is called "speedup" or "put by hash". It is especially
34230 efficient in case of generally available files like popular books,
34231 video or audio clips, because files are searched by hash in all ac‐
34232 counts of all mailru users. It is meaningless and ineffective if
34233 source file is unique or encrypted. Please note that rclone may need
34234 local memory and disk space to calculate content hash in advance and
34235 decide whether full upload is required. Also, if rclone does not know
34236 file size in advance (e.g. in case of streaming or partial uploads),
34237 it will not even try this optimization.
34238 Properties:
34240 • Config: speedup_enable
34242 • Env Var: RCLONE_MAILRU_SPEEDUP_ENABLE
34244 • Type: bool
34246 • Default: true
34248 • Examples:
34250 • "true"
34252 • Enable
34254 • "false"
34256 • Disable
34258 Advanced options
34260 Here are the Advanced options specific to mailru (Mail.ru Cloud).
34261 --mailru-speedup-file-patterns
34263 Comma separated list of file name patterns eligible for speedup (put by
34264 hash).
34265 Patterns are case insensitive and can contain '*' or '?' meta charac‐
34267 ters.
34268 Properties:
34270 • Config: speedup_file_patterns
34272 • Env Var: RCLONE_MAILRU_SPEEDUP_FILE_PATTERNS
34274 • Type: string
34276 • Default: ".mkv,.avi,.mp4,.mp3,.zip,.gz,.rar,.pdf"
34278 • Examples:
34280 • ""
34282 • Empty list completely disables speedup (put by hash).
34284 • "*"
34286 • All files will be attempted for speedup.
34288 • ".mkv,.avi,.mp4,.mp3"
34290 • Only common audio/video files will be tried for put by hash.
34292 • ".zip,.gz,.rar,.pdf"
34294 • Only common archives or PDF books will be tried for speedup.
34296 --mailru-speedup-max-disk
34298 This option allows you to disable speedup (put by hash) for large
34299 files.
34300 Reason is that preliminary hashing can exhaust your RAM or disk space.
34302 Properties:
34304 • Config: speedup_max_disk
34306 • Env Var: RCLONE_MAILRU_SPEEDUP_MAX_DISK
34308 • Type: SizeSuffix
34310 • Default: 3Gi
34312 • Examples:
34314 • "0"
34316 • Completely disable speedup (put by hash).
34318 • "1G"
34320 • Files larger than 1Gb will be uploaded directly.
34322 • "3G"
34324 • Choose this option if you have less than 3Gb free on local disk.
34326 --mailru-speedup-max-memory
34328 Files larger than the size given below will always be hashed on disk.
34329 Properties:
34331 • Config: speedup_max_memory
34333 • Env Var: RCLONE_MAILRU_SPEEDUP_MAX_MEMORY
34335 • Type: SizeSuffix
34337 • Default: 32Mi
34339 • Examples:
34341 • "0"
34343 • Preliminary hashing will always be done in a temporary disk loca‐
34345 tion.
34346 • "32M"
34348 • Do not dedicate more than 32Mb RAM for preliminary hashing.
34350 • "256M"
34352 • You have at most 256Mb RAM free for hash calculations.
34354 --mailru-check-hash
34356 What should copy do if file checksum is mismatched or invalid.
34357 Properties:
34359 • Config: check_hash
34361 • Env Var: RCLONE_MAILRU_CHECK_HASH
34363 • Type: bool
34365 • Default: true
34367 • Examples:
34369 • "true"
34371 • Fail with error.
34373 • "false"
34375 • Ignore and continue.
34377 --mailru-user-agent
34379 HTTP user agent used internally by client.
34380 Defaults to "rclone/VERSION" or "--user-agent" provided on command
34382 line.
34383 Properties:
34385 • Config: user_agent
34387 • Env Var: RCLONE_MAILRU_USER_AGENT
34389 • Type: string
34391 • Required: false
34393 --mailru-quirks
34395 Comma separated list of internal maintenance flags.
34396 This option must not be used by an ordinary user. It is intended only
34398 to facilitate remote troubleshooting of backend issues. Strict meaning
34399 of flags is not documented and not guaranteed to persist between re‐
34400 leases. Quirks will be removed when the backend grows stable. Sup‐
34401 ported quirks: atomicmkdir binlist unknowndirs
34402 Properties:
34404 • Config: quirks
34406 • Env Var: RCLONE_MAILRU_QUIRKS
34408 • Type: string
34410 • Required: false
34412 --mailru-encoding
34414 The encoding for the backend.
34415 See the encoding section in the overview (https://rclone.org/over‐
34417 view/#encoding) for more info.
34418 Properties:
34420 • Config: encoding
34422 • Env Var: RCLONE_MAILRU_ENCODING
34424 • Type: MultiEncoder
34426 • Default: Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,Back‐
34428 Slash,Del,Ctl,InvalidUtf8,Dot
34429 Limitations
34431 File size limits depend on your account. A single file size is limited
34432 by 2G for a free account and unlimited for paid tariffs. Please refer
34433 to the Mail.ru site for the total uploaded size limits.
34434 Note that Mailru is case insensitive so you can't have a file called
34436 "Hello.doc" and one called "hello.doc".
34437
34439 Mega (https://mega.nz/) is a cloud storage and file hosting service
34440 known for its security feature where all files are encrypted locally
34441 before they are uploaded. This prevents anyone (including employees of
34442 Mega) from accessing the files without knowledge of the key used for
34443 encryption.
34444 This is an rclone backend for Mega which supports the file transfer
34446 features of Mega using the same client side encryption.
34447 Paths are specified as remote:path
34449 Paths may be as deep as required, e.g. remote:directory/subdirectory.
34451 Configuration
34453 Here is an example of how to make a remote called remote. First run:
34454 rclone config
34456 This will guide you through an interactive setup process:
34458 No remotes found, make a new one?
34460 n) New remote
34461 s) Set configuration password
34462 q) Quit config
34463 n/s/q> n
34464 name> remote
34465 Type of storage to configure.
34466 Choose a number from below, or type in your own value
34467 [snip]
34468 XX / Mega
34469 \ "mega"
34470 [snip]
34471 Storage> mega
34472 User name
34473 user> you@example.com
34474 Password.
34475 y) Yes type in my own password
34476 g) Generate random password
34477 n) No leave this optional password blank
34478 y/g/n> y
34479 Enter the password:
34480 password:
34481 Confirm the password:
34482 password:
34483 Remote config
34484 --------------------
34485 [remote]
34486 type = mega
34487 user = you@example.com
34488 pass = *** ENCRYPTED ***
34489 --------------------
34490 y) Yes this is OK
34491 e) Edit this remote
34492 d) Delete this remote
34493 y/e/d> y
34494 NOTE: The encryption keys need to have been already generated after a
34496 regular login via the browser, otherwise attempting to use the creden‐
34497 tials in rclone will fail.
34498 Once configured you can then use rclone like this,
34500 List directories in top level of your Mega
34502 rclone lsd remote:
34504 List all the files in your Mega
34506 rclone ls remote:
34508 To copy a local directory to an Mega directory called backup
34510 rclone copy /home/source remote:backup
34512 Modified time and hashes
34514 Mega does not support modification times or hashes yet.
34515 Restricted filename characters
34517 Character Value Replacement
34518 ────────────────────────────────
34519 NUL 0x00 ␀
34520 / 0x2F ／
34521 Invalid UTF-8 bytes will also be replaced (https://rclone.org/over‐
34523 view/#invalid-utf8), as they can't be used in JSON strings.
34524 Duplicated files
34526 Mega can have two files with exactly the same name and path (unlike a
34527 normal file system).
34528 Duplicated files cause problems with the syncing and you will see mes‐
34530 sages in the log about duplicates.
34531 Use rclone dedupe to fix duplicated files.
34533 Failure to log-in
34535 Object not found
34536 If you are connecting to your Mega remote for the first time, to test
34537 access and synchronization, you may receive an error such as
34538 Failed to create file system for "my-mega-remote:":
34540 couldn't login: Object (typically, node or user) not found
34541 The diagnostic steps often recommended in the rclone forum (https://fo‐
34543 rum.rclone.org/search?q=mega) start with the MEGAcmd utility. Note
34544 that this refers to the official C++ command from
34545 https://github.com/meganz/MEGAcmd and not the go language built command
34546 from t3rm1n4l/megacmd that is no longer maintained.
34547 Follow the instructions for installing MEGAcmd and try accessing your
34549 remote as they recommend. You can establish whether or not you can log
34550 in using MEGAcmd, and obtain diagnostic information to help you, and
34551 search or work with others in the forum.
34552 MEGA CMD> login me@example.com
34554 Password:
34555 Fetching nodes ...
34556 Loading transfers from local cache
34557 Login complete as me@example.com
34558 me@example.com:/$
34559 Note that some have found issues with passwords containing special
34561 characters. If you can not log on with rclone, but MEGAcmd logs on
34562 just fine, then consider changing your password temporarily to pure al‐
34563 phanumeric characters, in case that helps.
34564 Repeated commands blocks access
34566 Mega remotes seem to get blocked (reject logins) under "heavy use". We
34567 haven't worked out the exact blocking rules but it seems to be related
34568 to fast paced, successive rclone commands.
34569 For example, executing this command 90 times in a row rclone link re‐
34571 mote:file will cause the remote to become "blocked". This is not an
34572 abnormal situation, for example if you wish to get the public links of
34573 a directory with hundred of files... After more or less a week, the
34574 remote will remote accept rclone logins normally again.
34575 You can mitigate this issue by mounting the remote it with rclone
34577 mount. This will log-in when mounting and a log-out when unmounting
34578 only. You can also run rclone rcd and then use rclone rc to run the
34579 commands over the API to avoid logging in each time.
34580 Rclone does not currently close mega sessions (you can see them in the
34582 web interface), however closing the sessions does not solve the issue.
34583 If you space rclone commands by 3 seconds it will avoid blocking the
34585 remote. We haven't identified the exact blocking rules, so perhaps one
34586 could execute the command 80 times without waiting and avoid blocking
34587 by waiting 3 seconds, then continuing...
34588 Note that this has been observed by trial and error and might not be
34590 set in stone.
34591 Other tools seem not to produce this blocking effect, as they use a
34593 different working approach (state-based, using sessionIDs instead of
34594 log-in) which isn't compatible with the current stateless rclone ap‐
34595 proach.
34596 Note that once blocked, the use of other tools (such as megacmd) is not
34598 a sure workaround: following megacmd login times have been observed in
34599 succession for blocked remote: 7 minutes, 20 min, 30min, 30 min, 30min.
34600 Web access looks unaffected though.
34601 Investigation is continuing in relation to workarounds based on time‐
34603 outs, pacers, retrials and tpslimits - if you discover something rele‐
34604 vant, please post on the forum.
34605 So, if rclone was working nicely and suddenly you are unable to log-in
34607 and you are sure the user and the password are correct, likely you have
34608 got the remote blocked for a while.
34609 Standard options
34611 Here are the Standard options specific to mega (Mega).
34612 --mega-user
34614 User name.
34615 Properties:
34617 • Config: user
34619 • Env Var: RCLONE_MEGA_USER
34621 • Type: string
34623 • Required: true
34625 --mega-pass
34627 Password.
34628 NB Input to this must be obscured - see rclone obscure
34630 (https://rclone.org/commands/rclone_obscure/).
34631 Properties:
34633 • Config: pass
34635 • Env Var: RCLONE_MEGA_PASS
34637 • Type: string
34639 • Required: true
34641 Advanced options
34643 Here are the Advanced options specific to mega (Mega).
34644 --mega-debug
34646 Output more debug from Mega.
34647 If this flag is set (along with -vv) it will print further debugging
34649 information from the mega backend.
34650 Properties:
34652 • Config: debug
34654 • Env Var: RCLONE_MEGA_DEBUG
34656 • Type: bool
34658 • Default: false
34660 --mega-hard-delete
34662 Delete files permanently rather than putting them into the trash.
34663 Normally the mega backend will put all deletions into the trash rather
34665 than permanently deleting them. If you specify this then rclone will
34666 permanently delete objects instead.
34667 Properties:
34669 • Config: hard_delete
34671 • Env Var: RCLONE_MEGA_HARD_DELETE
34673 • Type: bool
34675 • Default: false
34677 --mega-encoding
34679 The encoding for the backend.
34680 See the encoding section in the overview (https://rclone.org/over‐
34682 view/#encoding) for more info.
34683 Properties:
34685 • Config: encoding
34687 • Env Var: RCLONE_MEGA_ENCODING
34689 • Type: MultiEncoder
34691 • Default: Slash,InvalidUtf8,Dot
34693 Limitations
34695 This backend uses the go-mega go library
34696 (https://github.com/t3rm1n4l/go-mega) which is an opensource go library
34697 implementing the Mega API. There doesn't appear to be any documenta‐
34698 tion for the mega protocol beyond the mega C++ SDK
34699 (https://github.com/meganz/sdk) source code so there are likely quite a
34700 few errors still remaining in this library.
34701 Mega allows duplicate files which may confuse rclone.
34703
34705 The memory backend is an in RAM backend. It does not persist its data
34706 - use the local backend for that.
34707 The memory backend behaves like a bucket-based remote (e.g. like s3).
34709 Because it has no parameters you can just use it with the :memory: re‐
34710 mote name.
34711 Configuration
34713 You can configure it as a remote like this with rclone config too if
34714 you want to:
34715 No remotes found, make a new one?
34717 n) New remote
34718 s) Set configuration password
34719 q) Quit config
34720 n/s/q> n
34721 name> remote
34722 Type of storage to configure.
34723 Enter a string value. Press Enter for the default ("").
34724 Choose a number from below, or type in your own value
34725 [snip]
34726 XX / Memory
34727 \ "memory"
34728 [snip]
34729 Storage> memory
34730 ** See help for memory backend at: https://rclone.org/memory/ **
34731 Remote config
34733 --------------------
34735 [remote]
34736 type = memory
34737 --------------------
34738 y) Yes this is OK (default)
34739 e) Edit this remote
34740 d) Delete this remote
34741 y/e/d> y
34742 Because the memory backend isn't persistent it is most useful for test‐
34744 ing or with an rclone server or rclone mount, e.g.
34745 rclone mount :memory: /mnt/tmp
34747 rclone serve webdav :memory:
34748 rclone serve sftp :memory:
34749 Modified time and hashes
34751 The memory backend supports MD5 hashes and modification times accurate
34752 to 1 nS.
34753 Restricted filename characters
34755 The memory backend replaces the default restricted characters set
34756 (https://rclone.org/overview/#restricted-characters).
34757
34759 Paths are specified as remote: You may put subdirectories in too, e.g.
34760 remote:/path/to/dir. If you have a CP code you can use that as the
34761 folder after the domain such as <domain>/<cpcode>/<internal directories
34762 within cpcode>.
34763 For example, this is commonly configured with or without a CP code: *
34765 With a CP code. [your-domain-prefix]-nsu.akamaihd.net/123456/subdirec‐
34766 tory/ * Without a CP code. [your-domain-prefix]-nsu.akamaihd.net
34767 See all buckets rclone lsd remote: The initial setup for Netstorage in‐
34769 volves getting an account and secret. Use rclone config to walk you
34770 through the setup process.
34771 Configuration
34773 Here's an example of how to make a remote called ns1.
34774 1. To begin the interactive configuration process, enter this command:
34776 rclone config
34778 2. Type n to create a new remote.
34780 n) New remote
34782 d) Delete remote
34783 q) Quit config
34784 e/n/d/q> n
34785 3. For this example, enter ns1 when you reach the name> prompt.
34787 name> ns1
34789 4. Enter netstorage as the type of storage to configure.
34791 Type of storage to configure.
34793 Enter a string value. Press Enter for the default ("").
34794 Choose a number from below, or type in your own value
34795 XX / NetStorage
34796 \ "netstorage"
34797 Storage> netstorage
34798 5. Select between the HTTP or HTTPS protocol. Most users should choose
34800 HTTPS, which is the default. HTTP is provided primarily for debug‐
34801 ging purposes.
34802 Enter a string value. Press Enter for the default ("").
34804 Choose a number from below, or type in your own value
34805 1 / HTTP protocol
34806 \ "http"
34807 2 / HTTPS protocol
34808 \ "https"
34809 protocol> 1
34810 6. Specify your NetStorage host, CP code, and any necessary content
34812 paths using this format: <domain>/<cpcode>/<content>/
34813 Enter a string value. Press Enter for the default ("").
34815 host> baseball-nsu.akamaihd.net/123456/content/
34816 7. Set the netstorage account name
34818 Enter a string value. Press Enter for the default ("").
34820 account> username
34821 8. Set the Netstorage account secret/G2O key which will be used for au‐
34823 thentication purposes. Select the y option to set your own password
34824 then enter your secret. Note: The secret is stored in the
34825 rclone.conf file with hex-encoded encryption.
34826 y) Yes type in my own password
34828 g) Generate random password
34829 y/g> y
34830 Enter the password:
34831 password:
34832 Confirm the password:
34833 password:
34834 9. View the summary and confirm your remote configuration.
34836 [ns1]
34838 type = netstorage
34839 protocol = http
34840 host = baseball-nsu.akamaihd.net/123456/content/
34841 account = username
34842 secret = *** ENCRYPTED ***
34843 --------------------
34844 y) Yes this is OK (default)
34845 e) Edit this remote
34846 d) Delete this remote
34847 y/e/d> y
34848 This remote is called ns1 and can now be used.
34850 Example operations
34852 Get started with rclone and NetStorage with these examples. For addi‐
34853 tional rclone commands, visit https://rclone.org/commands/.
34854 See contents of a directory in your project
34856 rclone lsd ns1:/974012/testing/
34857 Sync the contents local with remote
34859 rclone sync . ns1:/974012/testing/
34860 Upload local content to remote
34862 rclone copy notes.txt ns1:/974012/testing/
34863 Delete content on remote
34865 rclone delete ns1:/974012/testing/notes.txt
34866 Move or copy content between CP codes.
34868 Your credentials must have access to two CP codes on the same remote.
34869 You can't perform operations between different remotes.
34870 rclone move ns1:/974012/testing/notes.txt ns1:/974450/testing2/
34872 Features
34874 Symlink Support
34875 The Netstorage backend changes the rclone --links, -l behavior. When
34876 uploading, instead of creating the .rclonelink file, use the "symlink"
34877 API in order to create the corresponding symlink on the remote. The
34878 .rclonelink file will not be created, the upload will be intercepted
34879 and only the symlink file that matches the source file name with no
34880 suffix will be created on the remote.
34881 This will effectively allow commands like copy/copyto, move/moveto and
34883 sync to upload from local to remote and download from remote to local
34884 directories with symlinks. Due to internal rclone limitations, it is
34885 not possible to upload an individual symlink file to any remote back‐
34886 end. You can always use the "backend symlink" command to create a sym‐
34887 link on the NetStorage server, refer to "symlink" section below.
34888 Individual symlink files on the remote can be used with the commands
34890 like "cat" to print the destination name, or "delete" to delete sym‐
34891 link, or copy, copy/to and move/moveto to download from the remote to
34892 local. Note: individual symlink files on the remote should be speci‐
34893 fied including the suffix .rclonelink.
34894 Note: No file with the suffix .rclonelink should ever exist on the
34896 server since it is not possible to actually upload/create a file with
34897 .rclonelink suffix with rclone, it can only exist if it is manually
34898 created through a non-rclone method on the remote.
34899 Implicit vs. Explicit Directories
34901 With NetStorage, directories can exist in one of two forms:
34902 1. Explicit Directory. This is an actual, physical directory that you
34904 have created in a storage group.
34905 2. Implicit Directory. This refers to a directory within a path that
34907 has not been physically created. For example, during upload of a
34908 file, nonexistent subdirectories can be specified in the target
34909 path. NetStorage creates these as "implicit." While the directories
34910 aren't physically created, they exist implicitly and the noted path
34911 is connected with the uploaded file.
34912 Rclone will intercept all file uploads and mkdir commands for the Net‐
34914 Storage remote and will explicitly issue the mkdir command for each di‐
34915 rectory in the uploading path. This will help with the interoperabili‐
34916 ty with the other Akamai services such as SFTP and the Content Manage‐
34917 ment Shell (CMShell). Rclone will not guarantee correctness of opera‐
34918 tions with implicit directories which might have been created as a re‐
34919 sult of using an upload API directly.
34920 --fast-list / ListR support
34922 NetStorage remote supports the ListR feature by using the "list" Net‐
34923 Storage API action to return a lexicographical list of all objects
34924 within the specified CP code, recursing into subdirectories as they're
34925 encountered.
34926 • Rclone will use the ListR method for some commands by default. Com‐
34928 mands such as lsf -R will use ListR by default. To disable this, in‐
34929 clude the --disable listR option to use the non-recursive method of
34930 listing objects.
34931 • Rclone will not use the ListR method for some commands. Commands
34933 such as sync don't use ListR by default. To force using the ListR
34934 method, include the --fast-list option.
34935 There are pros and cons of using the ListR method, refer to rclone doc‐
34937 umentation (https://rclone.org/docs/#fast-list). In general, the sync
34938 command over an existing deep tree on the remote will run faster with
34939 the "--fast-list" flag but with extra memory usage as a side effect.
34940 It might also result in higher CPU utilization but the whole task can
34941 be completed faster.
34942 Note: There is a known limitation that "lsf -R" will display number of
34944 files in the directory and directory size as -1 when ListR method is
34945 used. The workaround is to pass "--disable listR" flag if these num‐
34946 bers are important in the output.
34947 Purge
34949 NetStorage remote supports the purge feature by using the "quick-
34950 delete" NetStorage API action. The quick-delete action is disabled by
34951 default for security reasons and can be enabled for the account through
34952 the Akamai portal. Rclone will first try to use quick-delete action
34953 for the purge command and if this functionality is disabled then will
34954 fall back to a standard delete method.
34955 Note: Read the NetStorage Usage API (https://learn.akamai.com/en-
34957 us/webhelp/netstorage/netstorage-http-api-developer-
34958 guide/GUID-15836617-9F50-405A-833C-EA2556756A30.html) for considera‐
34959 tions when using "quick-delete". In general, using quick-delete method
34960 will not delete the tree immediately and objects targeted for quick-
34961 delete may still be accessible.
34962 Standard options
34964 Here are the Standard options specific to netstorage (Akamai NetStor‐
34965 age).
34966 --netstorage-host
34968 Domain+path of NetStorage host to connect to.
34969 Format should be <domain>/<internal folders>
34971 Properties:
34973 • Config: host
34975 • Env Var: RCLONE_NETSTORAGE_HOST
34977 • Type: string
34979 • Required: true
34981 --netstorage-account
34983 Set the NetStorage account name
34984 Properties:
34986 • Config: account
34988 • Env Var: RCLONE_NETSTORAGE_ACCOUNT
34990 • Type: string
34992 • Required: true
34994 --netstorage-secret
34996 Set the NetStorage account secret/G2O key for authentication.
34997 Please choose the 'y' option to set your own password then enter your
34999 secret.
35000 NB Input to this must be obscured - see rclone obscure
35002 (https://rclone.org/commands/rclone_obscure/).
35003 Properties:
35005 • Config: secret
35007 • Env Var: RCLONE_NETSTORAGE_SECRET
35009 • Type: string
35011 • Required: true
35013 Advanced options
35015 Here are the Advanced options specific to netstorage (Akamai NetStor‐
35016 age).
35017 --netstorage-protocol
35019 Select between HTTP or HTTPS protocol.
35020 Most users should choose HTTPS, which is the default. HTTP is provided
35022 primarily for debugging purposes.
35023 Properties:
35025 • Config: protocol
35027 • Env Var: RCLONE_NETSTORAGE_PROTOCOL
35029 • Type: string
35031 • Default: "https"
35033 • Examples:
35035 • "http"
35037 • HTTP protocol
35039 • "https"
35041 • HTTPS protocol
35043 Backend commands
35045 Here are the commands specific to the netstorage backend.
35046 Run them with
35048 rclone backend COMMAND remote:
35050 The help below will explain what arguments each command takes.
35052 See the backend (https://rclone.org/commands/rclone_backend/) command
35054 for more info on how to pass options and arguments.
35055 These can be run on a running backend using the rc command backend/com‐
35057 mand (https://rclone.org/rc/#backend-command).
35058 du
35060 Return disk usage information for a specified directory
35061 rclone backend du remote: [options] [<arguments>+]
35063 The usage information returned, includes the targeted directory as well
35065 as all files stored in any sub-directories that may exist.
35066 symlink
35068 You can create a symbolic link in ObjectStore with the symlink action.
35069 rclone backend symlink remote: [options] [<arguments>+]
35071 The desired path location (including applicable sub-directories) ending
35073 in the object that will be the target of the symlink (for example,
35074 /links/mylink). Include the file extension for the object, if applica‐
35075 ble. rclone backend symlink <src> <path>
35076
35078 Paths are specified as remote:container (or remote: for the lsd com‐
35079 mand.) You may put subdirectories in too, e.g. remote:contain‐
35080 er/path/to/dir.
35081 Configuration
35083 Here is an example of making a Microsoft Azure Blob Storage configura‐
35084 tion. For a remote called remote. First run:
35085 rclone config
35087 This will guide you through an interactive setup process:
35089 No remotes found, make a new one?
35091 n) New remote
35092 s) Set configuration password
35093 q) Quit config
35094 n/s/q> n
35095 name> remote
35096 Type of storage to configure.
35097 Choose a number from below, or type in your own value
35098 [snip]
35099 XX / Microsoft Azure Blob Storage
35100 \ "azureblob"
35101 [snip]
35102 Storage> azureblob
35103 Storage Account Name
35104 account> account_name
35105 Storage Account Key
35106 key> base64encodedkey==
35107 Endpoint for the service - leave blank normally.
35108 endpoint>
35109 Remote config
35110 --------------------
35111 [remote]
35112 account = account_name
35113 key = base64encodedkey==
35114 endpoint =
35115 --------------------
35116 y) Yes this is OK
35117 e) Edit this remote
35118 d) Delete this remote
35119 y/e/d> y
35120 See all containers
35122 rclone lsd remote:
35124 Make a new container
35126 rclone mkdir remote:container
35128 List the contents of a container
35130 rclone ls remote:container
35132 Sync /home/local/directory to the remote container, deleting any excess
35134 files in the container.
35135 rclone sync -i /home/local/directory remote:container
35137 --fast-list
35139 This remote supports --fast-list which allows you to use fewer transac‐
35140 tions in exchange for more memory. See the rclone docs
35141 (https://rclone.org/docs/#fast-list) for more details.
35142 Modified time
35144 The modified time is stored as metadata on the object with the mtime
35145 key. It is stored using RFC3339 Format time with nanosecond precision.
35146 The metadata is supplied during directory listings so there is no over‐
35147 head to using it.
35148 Performance
35150 When uploading large files, increasing the value of --azureblob-upload-
35151 concurrency will increase performance at the cost of using more memory.
35152 The default of 16 is set quite conservatively to use less memory. It
35153 maybe be necessary raise it to 64 or higher to fully utilize a 1 GBit/s
35154 link with a single file transfer.
35155 Restricted filename characters
35157 In addition to the default restricted characters set
35158 (https://rclone.org/overview/#restricted-characters) the following
35159 characters are also replaced:
35160 Character Value Replacement
35162 ────────────────────────────────
35163 / 0x2F ／
35164 \ 0x5C ＼
35165 File names can also not end with the following characters. These only
35167 get replaced if they are the last character in the name:
35168 Character Value Replacement
35170 ────────────────────────────────
35171 . 0x2E ．
35172 Invalid UTF-8 bytes will also be replaced (https://rclone.org/over‐
35174 view/#invalid-utf8), as they can't be used in JSON strings.
35175 Hashes
35177 MD5 hashes are stored with blobs. However blobs that were uploaded in
35178 chunks only have an MD5 if the source remote was capable of MD5 hashes,
35179 e.g. the local disk.
35180 Authenticating with Azure Blob Storage
35182 Rclone has 3 ways of authenticating with Azure Blob Storage:
35183 Account and Key
35185 This is the most straight forward and least flexible way. Just fill in
35186 the account and key lines and leave the rest blank.
35187 SAS URL
35189 This can be an account level SAS URL or container level SAS URL.
35190 To use it leave account, key blank and fill in sas_url.
35192 An account level SAS URL or container level SAS URL can be obtained
35194 from the Azure portal or the Azure Storage Explorer. To get a contain‐
35195 er level SAS URL right click on a container in the Azure Blob explorer
35196 in the Azure portal.
35197 If you use a container level SAS URL, rclone operations are permitted
35199 only on a particular container, e.g.
35200 rclone ls azureblob:container
35202 You can also list the single container from the root. This will only
35204 show the container specified by the SAS URL.
35205 $ rclone lsd azureblob:
35207 container/
35208 Note that you can't see or access any other containers - this will fail
35210 rclone ls azureblob:othercontainer
35212 Container level SAS URLs are useful for temporarily allowing third par‐
35214 ties access to a single container or putting credentials into an un‐
35215 trusted environment such as a CI build server.
35216 Standard options
35218 Here are the Standard options specific to azureblob (Microsoft Azure
35219 Blob Storage).
35220 --azureblob-account
35222 Storage Account Name.
35223 Leave blank to use SAS URL or Emulator.
35225 Properties:
35227 • Config: account
35229 • Env Var: RCLONE_AZUREBLOB_ACCOUNT
35231 • Type: string
35233 • Required: false
35235 --azureblob-service-principal-file
35237 Path to file containing credentials for use with a service principal.
35238 Leave blank normally. Needed only if you want to use a service princi‐
35240 pal instead of interactive login.
35241 $ az ad sp create-for-rbac --name "<name>" \
35243 --role "Storage Blob Data Owner" \
35244 --scopes "/subscriptions/<subscription>/resourceGroups/<resource-group>/providers/Microsoft.Storage/storageAccounts/<storage-account>/blobServices/default/containers/<container>" \
35245 > azure-principal.json
35246 See "Create an Azure service principal" (https://docs.microsoft.com/en-
35248 us/cli/azure/create-an-azure-service-principal-azure-cli) and "Assign
35249 an Azure role for access to blob data" (https://docs.microsoft.com/en-
35250 us/azure/storage/common/storage-auth-aad-rbac-cli) pages for more de‐
35251 tails.
35252 Properties:
35254 • Config: service_principal_file
35256 • Env Var: RCLONE_AZUREBLOB_SERVICE_PRINCIPAL_FILE
35258 • Type: string
35260 • Required: false
35262 --azureblob-key
35264 Storage Account Key.
35265 Leave blank to use SAS URL or Emulator.
35267 Properties:
35269 • Config: key
35271 • Env Var: RCLONE_AZUREBLOB_KEY
35273 • Type: string
35275 • Required: false
35277 --azureblob-sas-url
35279 SAS URL for container level access only.
35280 Leave blank if using account/key or Emulator.
35282 Properties:
35284 • Config: sas_url
35286 • Env Var: RCLONE_AZUREBLOB_SAS_URL
35288 • Type: string
35290 • Required: false
35292 --azureblob-use-msi
35294 Use a managed service identity to authenticate (only works in Azure).
35295 When true, use a managed service identity (https://docs.micro‐
35297 soft.com/en-us/azure/active-directory/managed-identities-azure-re‐
35298 sources/) to authenticate to Azure Storage instead of a SAS token or
35299 account key.
35300 If the VM(SS) on which this program is running has a system-assigned
35302 identity, it will be used by default. If the resource has no system-
35303 assigned but exactly one user-assigned identity, the user-assigned
35304 identity will be used by default. If the resource has multiple user-
35305 assigned identities, the identity to use must be explicitly specified
35306 using exactly one of the msi_object_id, msi_client_id, or msi_mi_res_id
35307 parameters.
35308 Properties:
35310 • Config: use_msi
35312 • Env Var: RCLONE_AZUREBLOB_USE_MSI
35314 • Type: bool
35316 • Default: false
35318 --azureblob-use-emulator
35320 Uses local storage emulator if provided as 'true'.
35321 Leave blank if using real azure storage endpoint.
35323 Properties:
35325 • Config: use_emulator
35327 • Env Var: RCLONE_AZUREBLOB_USE_EMULATOR
35329 • Type: bool
35331 • Default: false
35333 Advanced options
35335 Here are the Advanced options specific to azureblob (Microsoft Azure
35336 Blob Storage).
35337 --azureblob-msi-object-id
35339 Object ID of the user-assigned MSI to use, if any.
35340 Leave blank if msi_client_id or msi_mi_res_id specified.
35342 Properties:
35344 • Config: msi_object_id
35346 • Env Var: RCLONE_AZUREBLOB_MSI_OBJECT_ID
35348 • Type: string
35350 • Required: false
35352 --azureblob-msi-client-id
35354 Object ID of the user-assigned MSI to use, if any.
35355 Leave blank if msi_object_id or msi_mi_res_id specified.
35357 Properties:
35359 • Config: msi_client_id
35361 • Env Var: RCLONE_AZUREBLOB_MSI_CLIENT_ID
35363 • Type: string
35365 • Required: false
35367 --azureblob-msi-mi-res-id
35369 Azure resource ID of the user-assigned MSI to use, if any.
35370 Leave blank if msi_client_id or msi_object_id specified.
35372 Properties:
35374 • Config: msi_mi_res_id
35376 • Env Var: RCLONE_AZUREBLOB_MSI_MI_RES_ID
35378 • Type: string
35380 • Required: false
35382 --azureblob-endpoint
35384 Endpoint for the service.
35385 Leave blank normally.
35387 Properties:
35389 • Config: endpoint
35391 • Env Var: RCLONE_AZUREBLOB_ENDPOINT
35393 • Type: string
35395 • Required: false
35397 --azureblob-upload-cutoff
35399 Cutoff for switching to chunked upload (<= 256 MiB) (deprecated).
35400 Properties:
35402 • Config: upload_cutoff
35404 • Env Var: RCLONE_AZUREBLOB_UPLOAD_CUTOFF
35406 • Type: string
35408 • Required: false
35410 --azureblob-chunk-size
35412 Upload chunk size.
35413 Note that this is stored in memory and there may be up to "--transfers"
35415 * "--azureblob-upload-concurrency" chunks stored at once in memory.
35416 Properties:
35418 • Config: chunk_size
35420 • Env Var: RCLONE_AZUREBLOB_CHUNK_SIZE
35422 • Type: SizeSuffix
35424 • Default: 4Mi
35426 --azureblob-upload-concurrency
35428 Concurrency for multipart uploads.
35429 This is the number of chunks of the same file that are uploaded concur‐
35431 rently.
35432 If you are uploading small numbers of large files over high-speed links
35434 and these uploads do not fully utilize your bandwidth, then increasing
35435 this may help to speed up the transfers.
35436 In tests, upload speed increases almost linearly with upload concurren‐
35438 cy. For example to fill a gigabit pipe it may be necessary to raise
35439 this to 64. Note that this will use more memory.
35440 Note that chunks are stored in memory and there may be up to "--trans‐
35442 fers" * "--azureblob-upload-concurrency" chunks stored at once in memo‐
35443 ry.
35444 Properties:
35446 • Config: upload_concurrency
35448 • Env Var: RCLONE_AZUREBLOB_UPLOAD_CONCURRENCY
35450 • Type: int
35452 • Default: 16
35454 --azureblob-list-chunk
35456 Size of blob list.
35457 This sets the number of blobs requested in each listing chunk. Default
35459 is the maximum, 5000. "List blobs" requests are permitted 2 minutes
35460 per megabyte to complete. If an operation is taking longer than 2 min‐
35461 utes per megabyte on average, it will time out ( source
35462 (https://docs.microsoft.com/en-us/rest/api/storageservices/setting-
35463 timeouts-for-blob-service-operations#exceptions-to-default-timeout-in‐
35464 terval) ). This can be used to limit the number of blobs items to re‐
35465 turn, to avoid the time out.
35466 Properties:
35468 • Config: list_chunk
35470 • Env Var: RCLONE_AZUREBLOB_LIST_CHUNK
35472 • Type: int
35474 • Default: 5000
35476 --azureblob-access-tier
35478 Access tier of blob: hot, cool or archive.
35479 Archived blobs can be restored by setting access tier to hot or cool.
35481 Leave blank if you intend to use default access tier, which is set at
35482 account level
35483 If there is no "access tier" specified, rclone doesn't apply any tier.
35485 rclone performs "Set Tier" operation on blobs while uploading, if ob‐
35486 jects are not modified, specifying "access tier" to new one will have
35487 no effect. If blobs are in "archive tier" at remote, trying to perform
35488 data transfer operations from remote will not be allowed. User should
35489 first restore by tiering blob to "Hot" or "Cool".
35490 Properties:
35492 • Config: access_tier
35494 • Env Var: RCLONE_AZUREBLOB_ACCESS_TIER
35496 • Type: string
35498 • Required: false
35500 --azureblob-archive-tier-delete
35502 Delete archive tier blobs before overwriting.
35503 Archive tier blobs cannot be updated. So without this flag, if you at‐
35505 tempt to update an archive tier blob, then rclone will produce the er‐
35506 ror:
35507 can't update archive tier blob without --azureblob-archive-tier-delete
35509 With this flag set then before rclone attempts to overwrite an archive
35511 tier blob, it will delete the existing blob before uploading its re‐
35512 placement. This has the potential for data loss if the upload fails
35513 (unlike updating a normal blob) and also may cost more since deleting
35514 archive tier blobs early may be chargable.
35515 Properties:
35517 • Config: archive_tier_delete
35519 • Env Var: RCLONE_AZUREBLOB_ARCHIVE_TIER_DELETE
35521 • Type: bool
35523 • Default: false
35525 --azureblob-disable-checksum
35527 Don't store MD5 checksum with object metadata.
35528 Normally rclone will calculate the MD5 checksum of the input before up‐
35530 loading it so it can add it to metadata on the object. This is great
35531 for data integrity checking but can cause long delays for large files
35532 to start uploading.
35533 Properties:
35535 • Config: disable_checksum
35537 • Env Var: RCLONE_AZUREBLOB_DISABLE_CHECKSUM
35539 • Type: bool
35541 • Default: false
35543 --azureblob-memory-pool-flush-time
35545 How often internal memory buffer pools will be flushed.
35546 Uploads which requires additional buffers (f.e multipart) will use mem‐
35548 ory pool for allocations. This option controls how often unused buf‐
35549 fers will be removed from the pool.
35550 Properties:
35552 • Config: memory_pool_flush_time
35554 • Env Var: RCLONE_AZUREBLOB_MEMORY_POOL_FLUSH_TIME
35556 • Type: Duration
35558 • Default: 1m0s
35560 --azureblob-memory-pool-use-mmap
35562 Whether to use mmap buffers in internal memory pool.
35563 Properties:
35565 • Config: memory_pool_use_mmap
35567 • Env Var: RCLONE_AZUREBLOB_MEMORY_POOL_USE_MMAP
35569 • Type: bool
35571 • Default: false
35573 --azureblob-encoding
35575 The encoding for the backend.
35576 See the encoding section in the overview (https://rclone.org/over‐
35578 view/#encoding) for more info.
35579 Properties:
35581 • Config: encoding
35583 • Env Var: RCLONE_AZUREBLOB_ENCODING
35585 • Type: MultiEncoder
35587 • Default: Slash,BackSlash,Del,Ctl,RightPeriod,InvalidUtf8
35589 --azureblob-public-access
35591 Public access level of a container: blob or container.
35592 Properties:
35594 • Config: public_access
35596 • Env Var: RCLONE_AZUREBLOB_PUBLIC_ACCESS
35598 • Type: string
35600 • Required: false
35602 • Examples:
35604 • ""
35606 • The container and its blobs can be accessed only with an autho‐
35608 rized request.
35609 • It's a default value.
35611 • "blob"
35613 • Blob data within this container can be read via anonymous re‐
35615 quest.
35616 • "container"
35618 • Allow full public read access for container and blob data.
35620 --azureblob-no-head-object
35622 If set, do not do HEAD before GET when getting objects.
35623 Properties:
35625 • Config: no_head_object
35627 • Env Var: RCLONE_AZUREBLOB_NO_HEAD_OBJECT
35629 • Type: bool
35631 • Default: false
35633 Limitations
35635 MD5 sums are only uploaded with chunked files if the source has an MD5
35636 sum. This will always be the case for a local to azure copy.
35637 rclone about is not supported by the Microsoft Azure Blob storage back‐
35639 end. Backends without this capability cannot determine free space for
35640 an rclone mount or use policy mfs (most free space) as a member of an
35641 rclone union remote.
35642 See List of backends that do not support rclone about
35644 (https://rclone.org/overview/#optional-features) and rclone about
35645 (https://rclone.org/commands/rclone_about/)
35646 Azure Storage Emulator Support
35648 You can run rclone with storage emulator (usually azurite).
35649 To do this, just set up a new remote with rclone config following in‐
35651 structions described in introduction and set use_emulator config as
35652 true. You do not need to provide default account name neither an ac‐
35653 count key.
35654 Also, if you want to access a storage emulator instance running on a
35656 different machine, you can override Endpoint parameter in advanced set‐
35657 tings, setting it to http(s)://<host>:<port>/devstoreaccount1 (e.g.
35658 http://10.254.2.5:10000/devstoreaccount1).
35659
35661 Paths are specified as remote:path
35662 Paths may be as deep as required, e.g. remote:directory/subdirectory.
35664 Configuration
35666 The initial setup for OneDrive involves getting a token from Microsoft
35667 which you need to do in your browser. rclone config walks you through
35668 it.
35669 Here is an example of how to make a remote called remote. First run:
35671 rclone config
35673 This will guide you through an interactive setup process:
35675 e) Edit existing remote
35677 n) New remote
35678 d) Delete remote
35679 r) Rename remote
35680 c) Copy remote
35681 s) Set configuration password
35682 q) Quit config
35683 e/n/d/r/c/s/q> n
35684 name> remote
35685 Type of storage to configure.
35686 Enter a string value. Press Enter for the default ("").
35687 Choose a number from below, or type in your own value
35688 [snip]
35689 XX / Microsoft OneDrive
35690 \ "onedrive"
35691 [snip]
35692 Storage> onedrive
35693 Microsoft App Client Id
35694 Leave blank normally.
35695 Enter a string value. Press Enter for the default ("").
35696 client_id>
35697 Microsoft App Client Secret
35698 Leave blank normally.
35699 Enter a string value. Press Enter for the default ("").
35700 client_secret>
35701 Edit advanced config? (y/n)
35702 y) Yes
35703 n) No
35704 y/n> n
35705 Remote config
35706 Use auto config?
35707 * Say Y if not sure
35708 * Say N if you are working on a remote or headless machine
35709 y) Yes
35710 n) No
35711 y/n> y
35712 If your browser doesn't open automatically go to the following link: http://127.0.0.1:53682/auth
35713 Log in and authorize rclone for access
35714 Waiting for code...
35715 Got code
35716 Choose a number from below, or type in an existing value
35717 1 / OneDrive Personal or Business
35718 \ "onedrive"
35719 2 / Sharepoint site
35720 \ "sharepoint"
35721 3 / Type in driveID
35722 \ "driveid"
35723 4 / Type in SiteID
35724 \ "siteid"
35725 5 / Search a Sharepoint site
35726 \ "search"
35727 Your choice> 1
35728 Found 1 drives, please select the one you want to use:
35729 0: OneDrive (business) id=b!Eqwertyuiopasdfghjklzxcvbnm-7mnbvcxzlkjhgfdsapoiuytrewqk
35730 Chose drive to use:> 0
35731 Found drive 'root' of type 'business', URL: https://org-my.sharepoint.com/personal/you/Documents
35732 Is that okay?
35733 y) Yes
35734 n) No
35735 y/n> y
35736 --------------------
35737 [remote]
35738 type = onedrive
35739 token = {"access_token":"youraccesstoken","token_type":"Bearer","refresh_token":"yourrefreshtoken","expiry":"2018-08-26T22:39:52.486512262+08:00"}
35740 drive_id = b!Eqwertyuiopasdfghjklzxcvbnm-7mnbvcxzlkjhgfdsapoiuytrewqk
35741 drive_type = business
35742 --------------------
35743 y) Yes this is OK
35744 e) Edit this remote
35745 d) Delete this remote
35746 y/e/d> y
35747 See the remote setup docs (https://rclone.org/remote_setup/) for how to
35749 set it up on a machine with no Internet browser available.
35750 Note that rclone runs a webserver on your local machine to collect the
35752 token as returned from Microsoft. This only runs from the moment it
35753 opens your browser to the moment you get back the verification code.
35754 This is on http://127.0.0.1:53682/ and this it may require you to un‐
35755 block it temporarily if you are running a host firewall.
35756 Once configured you can then use rclone like this,
35758 List directories in top level of your OneDrive
35760 rclone lsd remote:
35762 List all the files in your OneDrive
35764 rclone ls remote:
35766 To copy a local directory to an OneDrive directory called backup
35768 rclone copy /home/source remote:backup
35770 Getting your own Client ID and Key
35772 rclone uses a default Client ID when talking to OneDrive, unless a cus‐
35773 tom client_id is specified in the config. The default Client ID and
35774 Key are shared by all rclone users when performing requests.
35775 You may choose to create and use your own Client ID, in case the de‐
35777 fault one does not work well for you. For example, you might see
35778 throttling.
35779 Creating Client ID for OneDrive Personal
35781 To create your own Client ID, please follow these steps:
35782 1. Open https://portal.azure.com/#blade/Microsoft_AAD_Regis‐
35784 teredApps/ApplicationsListBlade and then click New registration.
35785 2. Enter a name for your app, choose account type Accounts in any orga‐
35787 nizational directory (Any Azure AD directory - Multitenant) and per‐
35788 sonal Microsoft accounts (e.g. Skype, Xbox), select Web in Redirect
35789 URI, then type (do not copy and paste) http://localhost:53682/ and
35790 click Register. Copy and keep the Application (client) ID under the
35791 app name for later use.
35792 3. Under manage select Certificates & secrets, click New client secret.
35794 Enter a description (can be anything) and set Expires to 24 months.
35795 Copy and keep that secret Value for later use (you won't be able to
35796 see this value afterwards).
35797 4. Under manage select API permissions, click Add a permission and se‐
35799 lect Microsoft Graph then select delegated permissions.
35800 5. Search and select the following permissions: Files.Read, Files.Read‐
35802 Write, Files.Read.All, Files.ReadWrite.All, offline_access, Us‐
35803 er.Read and Sites.Read.All (if custom access scopes are configured,
35804 select the permissions accordingly). Once selected click Add per‐
35805 missions at the bottom.
35806 Now the application is complete. Run rclone config to create or edit a
35808 OneDrive remote. Supply the app ID and password as Client ID and Se‐
35809 cret, respectively. rclone will walk you through the remaining steps.
35810 The access_scopes option allows you to configure the permissions re‐
35812 quested by rclone. See Microsoft Docs (https://docs.microsoft.com/en-
35813 us/graph/permissions-reference#files-permissions) for more information
35814 about the different scopes.
35815 The Sites.Read.All permission is required if you need to search Share‐
35817 Point sites when configuring the remote
35818 (https://github.com/rclone/rclone/pull/5883). However, if that permis‐
35819 sion is not assigned, you need to exclude Sites.Read.All from your ac‐
35820 cess scopes or set disable_site_permission option to true in the ad‐
35821 vanced options.
35822 Creating Client ID for OneDrive Business
35824 The steps for OneDrive Personal may or may not work for OneDrive Busi‐
35825 ness, depending on the security settings of the organization. A common
35826 error is that the publisher of the App is not verified.
35827 You may try to verify you account (https://docs.microsoft.com/en-
35829 us/azure/active-directory/develop/publisher-verification-overview), or
35830 try to limit the App to your organization only, as shown below.
35831 1. Make sure to create the App with your business account.
35833 2. Follow the steps above to create an App. However, we need a differ‐
35835 ent account type here: Accounts in this organizational directory on‐
35836 ly (*** - Single tenant). Note that you can also change the account
35837 type after creating the App.
35838 3. Find the tenant ID (https://docs.microsoft.com/en-us/azure/active-
35840 directory/fundamentals/active-directory-how-to-find-tenant) of your
35841 organization.
35842 4. In the rclone config, set auth_url to https://login.microsofton‐
35844 line.com/YOUR_TENANT_ID/oauth2/v2.0/authorize.
35845 5. In the rclone config, set token_url to https://login.microsofton‐
35847 line.com/YOUR_TENANT_ID/oauth2/v2.0/token.
35848 Note: If you have a special region, you may need a different host in
35850 step 4 and 5. Here are some hints
35851 (https://github.com/rclone/rclone/blob/bc23bf11db1c78c6ebbf8ea538fbebf7058b4176/back‐
35852 end/onedrive/onedrive.go#L86).
35853 Modification time and hashes
35855 OneDrive allows modification times to be set on objects accurate to 1
35856 second. These will be used to detect whether objects need syncing or
35857 not.
35858 OneDrive personal supports SHA1 type hashes. OneDrive for business and
35860 Sharepoint Server support QuickXorHash (https://docs.microsoft.com/en-
35861 us/onedrive/developer/code-snippets/quickxorhash).
35862 For all types of OneDrive you can use the --checksum flag.
35864 Restricted filename characters
35866 In addition to the default restricted characters set
35867 (https://rclone.org/overview/#restricted-characters) the following
35868 characters are also replaced:
35869 Character Value Replacement
35871 ────────────────────────────────
35872 " 0x22 ＂
35873 * 0x2A ＊
35874 : 0x3A ：
35875 < 0x3C ＜
35877 > 0x3E ＞
35878 ? 0x3F ？
35879 \ 0x5C ＼
35880 | 0x7C ｜
35881 File names can also not end with the following characters. These only
35883 get replaced if they are the last character in the name:
35884 Character Value Replacement
35886 ────────────────────────────────
35887 SP 0x20 ␠
35888 . 0x2E ．
35889 File names can also not begin with the following characters. These on‐
35891 ly get replaced if they are the first character in the name:
35892 Character Value Replacement
35894 ────────────────────────────────
35895 SP 0x20 ␠
35896 ~ 0x7E ～
35897 Invalid UTF-8 bytes will also be replaced (https://rclone.org/over‐
35899 view/#invalid-utf8), as they can't be used in JSON strings.
35900 Deleting files
35902 Any files you delete with rclone will end up in the trash. Microsoft
35903 doesn't provide an API to permanently delete files, nor to empty the
35904 trash, so you will have to do that with one of Microsoft's apps or via
35905 the OneDrive website.
35906 Standard options
35908 Here are the Standard options specific to onedrive (Microsoft
35909 OneDrive).
35910 --onedrive-client-id
35912 OAuth Client Id.
35913 Leave blank normally.
35915 Properties:
35917 • Config: client_id
35919 • Env Var: RCLONE_ONEDRIVE_CLIENT_ID
35921 • Type: string
35923 • Required: false
35925 --onedrive-client-secret
35927 OAuth Client Secret.
35928 Leave blank normally.
35930 Properties:
35932 • Config: client_secret
35934 • Env Var: RCLONE_ONEDRIVE_CLIENT_SECRET
35936 • Type: string
35938 • Required: false
35940 --onedrive-region
35942 Choose national cloud region for OneDrive.
35943 Properties:
35945 • Config: region
35947 • Env Var: RCLONE_ONEDRIVE_REGION
35949 • Type: string
35951 • Default: "global"
35953 • Examples:
35955 • "global"
35957 • Microsoft Cloud Global
35959 • "us"
35961 • Microsoft Cloud for US Government
35963 • "de"
35965 • Microsoft Cloud Germany
35967 • "cn"
35969 • Azure and Office 365 operated by Vnet Group in China
35971 Advanced options
35973 Here are the Advanced options specific to onedrive (Microsoft
35974 OneDrive).
35975 --onedrive-token
35977 OAuth Access Token as a JSON blob.
35978 Properties:
35980 • Config: token
35982 • Env Var: RCLONE_ONEDRIVE_TOKEN
35984 • Type: string
35986 • Required: false
35988 --onedrive-auth-url
35990 Auth server URL.
35991 Leave blank to use the provider defaults.
35993 Properties:
35995 • Config: auth_url
35997 • Env Var: RCLONE_ONEDRIVE_AUTH_URL
35999 • Type: string
36001 • Required: false
36003 --onedrive-token-url
36005 Token server url.
36006 Leave blank to use the provider defaults.
36008 Properties:
36010 • Config: token_url
36012 • Env Var: RCLONE_ONEDRIVE_TOKEN_URL
36014 • Type: string
36016 • Required: false
36018 --onedrive-chunk-size
36020 Chunk size to upload files with - must be multiple of 320k (327,680
36021 bytes).
36022 Above this size files will be chunked - must be multiple of 320k
36024 (327,680 bytes) and should not exceed 250M (262,144,000 bytes) else you
36025 may encounter "Microsoft.SharePoint.Client.InvalidClientQueryException:
36026 The request message is too big." Note that the chunks will be buffered
36027 into memory.
36028 Properties:
36030 • Config: chunk_size
36032 • Env Var: RCLONE_ONEDRIVE_CHUNK_SIZE
36034 • Type: SizeSuffix
36036 • Default: 10Mi
36038 --onedrive-drive-id
36040 The ID of the drive to use.
36041 Properties:
36043 • Config: drive_id
36045 • Env Var: RCLONE_ONEDRIVE_DRIVE_ID
36047 • Type: string
36049 • Required: false
36051 --onedrive-drive-type
36053 The type of the drive (personal | business | documentLibrary).
36054 Properties:
36056 • Config: drive_type
36058 • Env Var: RCLONE_ONEDRIVE_DRIVE_TYPE
36060 • Type: string
36062 • Required: false
36064 --onedrive-root-folder-id
36066 ID of the root folder.
36067 This isn't normally needed, but in special circumstances you might know
36069 the folder ID that you wish to access but not be able to get there
36070 through a path traversal.
36071 Properties:
36073 • Config: root_folder_id
36075 • Env Var: RCLONE_ONEDRIVE_ROOT_FOLDER_ID
36077 • Type: string
36079 • Required: false
36081 --onedrive-access-scopes
36083 Set scopes to be requested by rclone.
36084 Choose or manually enter a custom space separated list with all scopes,
36086 that rclone should request.
36087 Properties:
36089 • Config: access_scopes
36091 • Env Var: RCLONE_ONEDRIVE_ACCESS_SCOPES
36093 • Type: SpaceSepList
36095 • Default: Files.Read Files.ReadWrite Files.Read.All Files.Read‐
36097 Write.All Sites.Read.All offline_access
36098 • Examples:
36100 • "Files.Read Files.ReadWrite Files.Read.All Files.ReadWrite.All
36102 Sites.Read.All offline_access"
36103 • Read and write access to all resources
36105 • "Files.Read Files.Read.All Sites.Read.All offline_access"
36107 • Read only access to all resources
36109 • "Files.Read Files.ReadWrite Files.Read.All Files.ReadWrite.All off‐
36111 line_access"
36112 • Read and write access to all resources, without the ability to
36114 browse SharePoint sites.
36115 • Same as if disable_site_permission was set to true
36117 --onedrive-disable-site-permission
36119 Disable the request for Sites.Read.All permission.
36120 If set to true, you will no longer be able to search for a SharePoint
36122 site when configuring drive ID, because rclone will not request
36123 Sites.Read.All permission. Set it to true if your organization didn't
36124 assign Sites.Read.All permission to the application, and your organiza‐
36125 tion disallows users to consent app permission request on their own.
36126 Properties:
36128 • Config: disable_site_permission
36130 • Env Var: RCLONE_ONEDRIVE_DISABLE_SITE_PERMISSION
36132 • Type: bool
36134 • Default: false
36136 --onedrive-expose-onenote-files
36138 Set to make OneNote files show up in directory listings.
36139 By default, rclone will hide OneNote files in directory listings be‐
36141 cause operations like "Open" and "Update" won't work on them. But this
36142 behaviour may also prevent you from deleting them. If you want to
36143 delete OneNote files or otherwise want them to show up in directory
36144 listing, set this option.
36145 Properties:
36147 • Config: expose_onenote_files
36149 • Env Var: RCLONE_ONEDRIVE_EXPOSE_ONENOTE_FILES
36151 • Type: bool
36153 • Default: false
36155 --onedrive-server-side-across-configs
36157 Allow server-side operations (e.g. copy) to work across different
36158 onedrive configs.
36159 This will only work if you are copying between two OneDrive Personal
36161 drives AND the files to copy are already shared between them. In other
36162 cases, rclone will fall back to normal copy (which will be slightly
36163 slower).
36164 Properties:
36166 • Config: server_side_across_configs
36168 • Env Var: RCLONE_ONEDRIVE_SERVER_SIDE_ACROSS_CONFIGS
36170 • Type: bool
36172 • Default: false
36174 --onedrive-list-chunk
36176 Size of listing chunk.
36177 Properties:
36179 • Config: list_chunk
36181 • Env Var: RCLONE_ONEDRIVE_LIST_CHUNK
36183 • Type: int
36185 • Default: 1000
36187 --onedrive-no-versions
36189 Remove all versions on modifying operations.
36190 Onedrive for business creates versions when rclone uploads new files
36192 overwriting an existing one and when it sets the modification time.
36193 These versions take up space out of the quota.
36195 This flag checks for versions after file upload and setting modifica‐
36197 tion time and removes all but the last version.
36198 NB Onedrive personal can't currently delete versions so don't use this
36200 flag there.
36201 Properties:
36203 • Config: no_versions
36205 • Env Var: RCLONE_ONEDRIVE_NO_VERSIONS
36207 • Type: bool
36209 • Default: false
36211 --onedrive-link-scope
36213 Set the scope of the links created by the link command.
36214 Properties:
36216 • Config: link_scope
36218 • Env Var: RCLONE_ONEDRIVE_LINK_SCOPE
36220 • Type: string
36222 • Default: "anonymous"
36224 • Examples:
36226 • "anonymous"
36228 • Anyone with the link has access, without needing to sign in.
36230 • This may include people outside of your organization.
36232 • Anonymous link support may be disabled by an administrator.
36234 • "organization"
36236 • Anyone signed into your organization (tenant) can use the link to
36238 get access.
36239 • Only available in OneDrive for Business and SharePoint.
36241 --onedrive-link-type
36243 Set the type of the links created by the link command.
36244 Properties:
36246 • Config: link_type
36248 • Env Var: RCLONE_ONEDRIVE_LINK_TYPE
36250 • Type: string
36252 • Default: "view"
36254 • Examples:
36256 • "view"
36258 • Creates a read-only link to the item.
36260 • "edit"
36262 • Creates a read-write link to the item.
36264 • "embed"
36266 • Creates an embeddable link to the item.
36268 --onedrive-link-password
36270 Set the password for links created by the link command.
36271 At the time of writing this only works with OneDrive personal paid ac‐
36273 counts.
36274 Properties:
36276 • Config: link_password
36278 • Env Var: RCLONE_ONEDRIVE_LINK_PASSWORD
36280 • Type: string
36282 • Required: false
36284 --onedrive-encoding
36286 The encoding for the backend.
36287 See the encoding section in the overview (https://rclone.org/over‐
36289 view/#encoding) for more info.
36290 Properties:
36292 • Config: encoding
36294 • Env Var: RCLONE_ONEDRIVE_ENCODING
36296 • Type: MultiEncoder
36298 • Default: Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,Back‐
36300 Slash,Del,Ctl,LeftSpace,LeftTilde,RightSpace,RightPeriod,In‐
36301 validUtf8,Dot
36302 Limitations
36304 If you don't use rclone for 90 days the refresh token will expire.
36305 This will result in authorization problems. This is easy to fix by
36306 running the rclone config reconnect remote: command to get a new token
36307 and refresh token.
36308 Naming
36310 Note that OneDrive is case insensitive so you can't have a file called
36311 "Hello.doc" and one called "hello.doc".
36312 There are quite a few characters that can't be in OneDrive file names.
36314 These can't occur on Windows platforms, but on non-Windows platforms
36315 they are common. Rclone will map these names to and from an identical
36316 looking unicode equivalent. For example if a file has a ? in it will
36317 be mapped to ？ instead.
36318 File sizes
36320 The largest allowed file size is 250 GiB for both OneDrive Personal and
36321 OneDrive for Business (Updated 13 Jan 2021) (https://support.micro‐
36322 soft.com/en-us/office/invalid-file-names-and-file-types-in-onedrive-
36323 and-sharepoint-64883a5d-228e-48f5-b3d2-eb39e07630fa?ui=en-us&rs=en-
36324 us&ad=us#individualfilesize).
36325 Path length
36327 The entire path, including the file name, must contain fewer than 400
36328 characters for OneDrive, OneDrive for Business and SharePoint Online.
36329 If you are encrypting file and folder names with rclone, you may want
36330 to pay attention to this limitation because the encrypted names are
36331 typically longer than the original ones.
36332 Number of files
36334 OneDrive seems to be OK with at least 50,000 files in a folder, but at
36335 100,000 rclone will get errors listing the directory like couldn’t list
36336 files: UnknownError:. See #2707 (https://github.com/rclone/rclone/is‐
36337 sues/2707) for more info.
36338 An official document about the limitations for different types of
36340 OneDrive can be found here (https://support.office.com/en-us/arti‐
36341 cle/invalid-file-names-and-file-types-in-onedrive-onedrive-for-busi‐
36342 ness-and-sharepoint-64883a5d-228e-48f5-b3d2-eb39e07630fa).
36343 Versions
36345 Every change in a file OneDrive causes the service to create a new ver‐
36346 sion of the file. This counts against a users quota. For example
36347 changing the modification time of a file creates a second version, so
36348 the file apparently uses twice the space.
36349 For example the copy command is affected by this as rclone copies the
36351 file and then afterwards sets the modification time to match the source
36352 file which uses another version.
36353 You can use the rclone cleanup command (see below) to remove all old
36355 versions.
36356 Or you can set the no_versions parameter to true and rclone will remove
36358 versions after operations which create new versions. This takes extra
36359 transactions so only enable it if you need it.
36360 Note At the time of writing Onedrive Personal creates versions (but not
36362 for setting the modification time) but the API for removing them re‐
36363 turns "API not found" so cleanup and no_versions should not be used on
36364 Onedrive Personal.
36365 Disabling versioning
36367 Starting October 2018, users will no longer be able to disable version‐
36368 ing by default. This is because Microsoft has brought an update
36369 (https://techcommunity.microsoft.com/t5/Microsoft-OneDrive-Blog/New-Up‐
36370 dates-to-OneDrive-and-SharePoint-Team-Site-Versioning/ba-p/204390) to
36371 the mechanism. To change this new default setting, a PowerShell com‐
36372 mand is required to be run by a SharePoint admin. If you are an admin,
36373 you can run these commands in PowerShell to change that setting:
36374 1. Install-Module -Name Microsoft.Online.SharePoint.PowerShell (in case
36376 you haven't installed this already)
36377 2. Import-Module Microsoft.Online.SharePoint.PowerShell -Disable‐
36379 NameChecking
36380 3. Connect-SPOService -Url https://YOURSITE-admin.sharepoint.com -Cre‐
36382 dential YOU@YOURSITE.COM (replacing YOURSITE, YOU, YOURSITE.COM with
36383 the actual values; this will prompt for your credentials)
36384 4. Set-SPOTenant -EnableMinimumVersionRequirement $False
36386 5. Disconnect-SPOService (to disconnect from the server)
36388 Below are the steps for normal users to disable versioning. If you
36390 don't see the "No Versioning" option, make sure the above requirements
36391 are met.
36392 User Weropol (https://github.com/Weropol) has found a method to disable
36394 versioning on OneDrive
36395 1. Open the settings menu by clicking on the gear symbol at the top of
36397 the OneDrive Business page.
36398 2. Click Site settings.
36400 3. Once on the Site settings page, navigate to Site Administration >
36402 Site libraries and lists.
36403 4. Click Customize "Documents".
36405 5. Click General Settings > Versioning Settings.
36407 6. Under Document Version History select the option No versioning.
36409 Note: This will disable the creation of new file versions, but will
36410 not remove any previous versions. Your documents are safe.
36411 7. Apply the changes by clicking OK.
36413 8. Use rclone to upload or modify files. (I also use the --no-update-
36415 modtime flag)
36416 9. Restore the versioning settings after using rclone. (Optional)
36418 Cleanup
36420 OneDrive supports rclone cleanup which causes rclone to look through
36421 every file under the path supplied and delete all version but the cur‐
36422 rent version. Because this involves traversing all the files, then
36423 querying each file for versions it can be quite slow. Rclone does
36424 --checkers tests in parallel. The command also supports -i which is a
36425 great way to see what it would do.
36426 rclone cleanup -i remote:path/subdir # interactively remove all old version for path/subdir
36428 rclone cleanup remote:path/subdir # unconditionally remove all old version for path/subdir
36429 NB Onedrive personal can't currently delete versions
36431 Troubleshooting
36433 Excessive throttling or blocked on SharePoint
36434 If you experience excessive throttling or is being blocked on Share‐
36435 Point then it may help to set the user agent explicitly with a flag
36436 like this: --user-agent "ISV|rclone.org|rclone/v1.55.1"
36437 The specific details can be found in the Microsoft document: Avoid get‐
36439 ting throttled or blocked in SharePoint Online (https://docs.micro‐
36440 soft.com/en-us/sharepoint/dev/general-development/how-to-avoid-getting-
36441 throttled-or-blocked-in-sharepoint-online#how-to-decorate-your-http-
36442 traffic-to-avoid-throttling)
36443 Unexpected file size/hash differences on Sharepoint
36445 It is a known (https://github.com/OneDrive/onedrive-api-docs/is‐
36446 sues/935#issuecomment-441741631) issue that Sharepoint (not OneDrive or
36447 OneDrive for Business) silently modifies uploaded files, mainly Office
36448 files (.docx, .xlsx, etc.), causing file size and hash checks to fail.
36449 There are also other situations that will cause OneDrive to report in‐
36450 consistent file sizes. To use rclone with such affected files on
36451 Sharepoint, you may disable these checks with the following command
36452 line arguments:
36453 --ignore-checksum --ignore-size
36455 Alternatively, if you have write access to the OneDrive files, it may
36457 be possible to fix this problem for certain files, by attempting the
36458 steps below. Open the web interface for OneDrive
36459 (https://onedrive.live.com) and find the affected files (which will be
36460 in the error messages/log for rclone). Simply click on each of these
36461 files, causing OneDrive to open them on the web. This will cause each
36462 file to be converted in place to a format that is functionally equiva‐
36463 lent but which will no longer trigger the size discrepancy. Once all
36464 problematic files are converted you will no longer need the ignore op‐
36465 tions above.
36466 Replacing/deleting existing files on Sharepoint gets "item not found"
36468 It is a known (https://github.com/OneDrive/onedrive-api-docs/is‐
36469 sues/1068) issue that Sharepoint (not OneDrive or OneDrive for Busi‐
36470 ness) may return "item not found" errors when users try to replace or
36471 delete uploaded files; this seems to mainly affect Office files (.docx,
36472 .xlsx, etc.) and web files (.html, .aspx, etc.). As a workaround, you
36473 may use the --backup-dir <BACKUP_DIR> command line argument so rclone
36474 moves the files to be replaced/deleted into a given backup directory
36475 (instead of directly replacing/deleting them). For example, to in‐
36476 struct rclone to move the files into the directory rclone-backup-dir on
36477 backend mysharepoint, you may use:
36478 --backup-dir mysharepoint:rclone-backup-dir
36480 access_denied (AADSTS65005)
36482 Error: access_denied
36483 Code: AADSTS65005
36484 Description: Using application 'rclone' is currently not supported for your organization [YOUR_ORGANIZATION] because it is in an unmanaged state. An administrator needs to claim ownership of the company by DNS validation of [YOUR_ORGANIZATION] before the application rclone can be provisioned.
36485 This means that rclone can't use the OneDrive for Business API with
36487 your account. You can't do much about it, maybe write an email to your
36488 admins.
36489 However, there are other ways to interact with your OneDrive account.
36491 Have a look at the WebDAV backend: https://rclone.org/webdav/#share‐
36492 point
36493 invalid_grant (AADSTS50076)
36495 Error: invalid_grant
36496 Code: AADSTS50076
36497 Description: Due to a configuration change made by your administrator, or because you moved to a new location, you must use multi-factor authentication to access '...'.
36498 If you see the error above after enabling multi-factor authentication
36500 for your account, you can fix it by refreshing your OAuth refresh to‐
36501 ken. To do that, run rclone config, and choose to edit your OneDrive
36502 backend. Then, you don't need to actually make any changes until you
36503 reach this question: Already have a token - refresh?. For this ques‐
36504 tion, answer y and go through the process to refresh your token, just
36505 like the first time the backend is configured. After this, rclone
36506 should work again for this backend.
36507 Invalid request when making public links
36509 On Sharepoint and OneDrive for Business, rclone link may return an "In‐
36510 valid request" error. A possible cause is that the organisation admin
36511 didn't allow public links to be made for the organisation/sharepoint
36512 library. To fix the permissions as an admin, take a look at the docs:
36513 1 (https://docs.microsoft.com/en-us/sharepoint/turn-external-sharing-
36514 on-or-off), 2 (https://support.microsoft.com/en-us/office/set-up-and-
36515 manage-access-requests-94b26e0b-2822-49d4-929a-8455698654b3).
36516
36518 Paths are specified as remote:path
36519 Paths may be as deep as required, e.g. remote:directory/subdirectory.
36521 Configuration
36523 Here is an example of how to make a remote called remote. First run:
36524 rclone config
36526 This will guide you through an interactive setup process:
36528 n) New remote
36530 d) Delete remote
36531 q) Quit config
36532 e/n/d/q> n
36533 name> remote
36534 Type of storage to configure.
36535 Choose a number from below, or type in your own value
36536 [snip]
36537 XX / OpenDrive
36538 \ "opendrive"
36539 [snip]
36540 Storage> opendrive
36541 Username
36542 username>
36543 Password
36544 y) Yes type in my own password
36545 g) Generate random password
36546 y/g> y
36547 Enter the password:
36548 password:
36549 Confirm the password:
36550 password:
36551 --------------------
36552 [remote]
36553 username =
36554 password = *** ENCRYPTED ***
36555 --------------------
36556 y) Yes this is OK
36557 e) Edit this remote
36558 d) Delete this remote
36559 y/e/d> y
36560 List directories in top level of your OpenDrive
36562 rclone lsd remote:
36564 List all the files in your OpenDrive
36566 rclone ls remote:
36568 To copy a local directory to an OpenDrive directory called backup
36570 rclone copy /home/source remote:backup
36572 Modified time and MD5SUMs
36574 OpenDrive allows modification times to be set on objects accurate to 1
36575 second. These will be used to detect whether objects need syncing or
36576 not.
36577 Restricted filename characters
36579 Character Value Replacement
36580 ────────────────────────────────
36581 NUL 0x00 ␀
36582 / 0x2F ／
36583 " 0x22 ＂
36584 * 0x2A ＊
36585 : 0x3A ：
36586 < 0x3C ＜
36587 > 0x3E ＞
36588 ? 0x3F ？
36589 \ 0x5C ＼
36590 | 0x7C ｜
36591 File names can also not begin or end with the following characters.
36593 These only get replaced if they are the first or last character in the
36594 name:
36595 Character Value Replacement
36597 ────────────────────────────────
36598 SP 0x20 ␠
36599 HT 0x09 ␉
36600 LF 0x0A ␊
36601 VT 0x0B ␋
36602 CR 0x0D ␍
36603 Invalid UTF-8 bytes will also be replaced (https://rclone.org/over‐
36605 view/#invalid-utf8), as they can't be used in JSON strings.
36606 Standard options
36608 Here are the Standard options specific to opendrive (OpenDrive).
36609 --opendrive-username
36611 Username.
36612 Properties:
36614 • Config: username
36616 • Env Var: RCLONE_OPENDRIVE_USERNAME
36618 • Type: string
36620 • Required: true
36622 --opendrive-password
36624 Password.
36625 NB Input to this must be obscured - see rclone obscure
36627 (https://rclone.org/commands/rclone_obscure/).
36628 Properties:
36630 • Config: password
36632 • Env Var: RCLONE_OPENDRIVE_PASSWORD
36634 • Type: string
36636 • Required: true
36638 Advanced options
36640 Here are the Advanced options specific to opendrive (OpenDrive).
36641 --opendrive-encoding
36643 The encoding for the backend.
36644 See the encoding section in the overview (https://rclone.org/over‐
36646 view/#encoding) for more info.
36647 Properties:
36649 • Config: encoding
36651 • Env Var: RCLONE_OPENDRIVE_ENCODING
36653 • Type: MultiEncoder
36655 • Default: Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,Back‐
36657 Slash,LeftSpace,LeftCrLfHtVt,RightSpace,RightCrLfHtVt,InvalidUtf8,Dot
36658 --opendrive-chunk-size
36660 Files will be uploaded in chunks this size.
36661 Note that these chunks are buffered in memory so increasing them will
36663 increase memory use.
36664 Properties:
36666 • Config: chunk_size
36668 • Env Var: RCLONE_OPENDRIVE_CHUNK_SIZE
36670 • Type: SizeSuffix
36672 • Default: 10Mi
36674 Limitations
36676 Note that OpenDrive is case insensitive so you can't have a file called
36677 "Hello.doc" and one called "hello.doc".
36678 There are quite a few characters that can't be in OpenDrive file names.
36680 These can't occur on Windows platforms, but on non-Windows platforms
36681 they are common. Rclone will map these names to and from an identical
36682 looking unicode equivalent. For example if a file has a ? in it will
36683 be mapped to ？ instead.
36684 rclone about is not supported by the OpenDrive backend. Backends with‐
36686 out this capability cannot determine free space for an rclone mount or
36687 use policy mfs (most free space) as a member of an rclone union remote.
36688 See List of backends that do not support rclone about
36690 (https://rclone.org/overview/#optional-features) and rclone about
36691 (https://rclone.org/commands/rclone_about/)
36692
36694 Oracle Object Storage Overview (https://docs.oracle.com/en-us/iaas/Con‐
36695 tent/Object/Concepts/objectstorageoverview.htm)
36696 Oracle Object Storage FAQ (https://www.oracle.com/cloud/storage/object-
36698 storage/faq/)
36699 Paths are specified as remote:bucket (or remote: for the lsd command.)
36701 You may put subdirectories in too, e.g. remote:bucket/path/to/dir.
36702 Configuration
36704 Here is an example of making an oracle object storage configuration.
36705 rclone config walks you through it.
36706 Here is an example of how to make a remote called remote. First run:
36708 rclone config
36710 This will guide you through an interactive setup process:
36712 n) New remote
36714 d) Delete remote
36715 r) Rename remote
36716 c) Copy remote
36717 s) Set configuration password
36718 q) Quit config
36719 e/n/d/r/c/s/q> n
36720 Enter name for new remote.
36722 name> remote
36723 Option Storage.
36725 Type of storage to configure.
36726 Choose a number from below, or type in your own value.
36727 [snip]
36728 XX / Oracle Cloud Infrastructure Object Storage
36729 \ (oracleobjectstorage)
36730 Storage> oracleobjectstorage
36731 Option provider.
36733 Choose your Auth Provider
36734 Choose a number from below, or type in your own string value.
36735 Press Enter for the default (env_auth).
36736 1 / automatically pickup the credentials from runtime(env), first one to provide auth wins
36737 \ (env_auth)
36738 / use an OCI user and an API key for authentication.
36739 2 | you’ll need to put in a config file your tenancy OCID, user OCID, region, the path, fingerprint to an API key.
36740 | https://docs.oracle.com/en-us/iaas/Content/API/Concepts/sdkconfig.htm
36741 \ (user_principal_auth)
36742 / use instance principals to authorize an instance to make API calls.
36743 3 | each instance has its own identity, and authenticates using the certificates that are read from instance metadata.
36744 | https://docs.oracle.com/en-us/iaas/Content/Identity/Tasks/callingservicesfrominstances.htm
36745 \ (instance_principal_auth)
36746 4 / use resource principals to make API calls
36747 \ (resource_principal_auth)
36748 5 / no credentials needed, this is typically for reading public buckets
36749 \ (no_auth)
36750 provider> 2
36751 Option namespace.
36753 Object storage namespace
36754 Enter a value.
36755 namespace> idbamagbg734
36756 Option compartment.
36758 Object storage compartment OCID
36759 Enter a value.
36760 compartment> ocid1.compartment.oc1..aaaaaaaapufkxc7ame3sthry5i7ujrwfc7ejnthhu6bhanm5oqfjpyasjkba
36761 Option region.
36763 Object storage Region
36764 Enter a value.
36765 region> us-ashburn-1
36766 Option endpoint.
36768 Endpoint for Object storage API.
36769 Leave blank to use the default endpoint for the region.
36770 Enter a value. Press Enter to leave empty.
36771 endpoint>
36772 Option config_file.
36774 Path to OCI config file
36775 Choose a number from below, or type in your own string value.
36776 Press Enter for the default (~/.oci/config).
36777 1 / oci configuration file location
36778 \ (~/.oci/config)
36779 config_file> /etc/oci/dev.conf
36780 Option config_profile.
36782 Profile name inside OCI config file
36783 Choose a number from below, or type in your own string value.
36784 Press Enter for the default (Default).
36785 1 / Use the default profile
36786 \ (Default)
36787 config_profile> Test
36788 Edit advanced config?
36790 y) Yes
36791 n) No (default)
36792 y/n> n
36793 Configuration complete.
36795 Options:
36796 - type: oracleobjectstorage
36797 - namespace: idbamagbg734
36798 - compartment: ocid1.compartment.oc1..aaaaaaaapufkxc7ame3sthry5i7ujrwfc7ejnthhu6bhanm5oqfjpyasjkba
36799 - region: us-ashburn-1
36800 - provider: user_principal_auth
36801 - config_file: /etc/oci/dev.conf
36802 - config_profile: Test
36803 Keep this "remote" remote?
36804 y) Yes this is OK (default)
36805 e) Edit this remote
36806 d) Delete this remote
36807 y/e/d> y
36808 See all buckets
36810 rclone lsd remote:
36812 Create a new bucket
36814 rclone mkdir remote:bucket
36816 List the contents of a bucket
36818 rclone ls remote:bucket
36820 rclone ls remote:bucket --max-depth 1
36821 Modified time
36823 The modified time is stored as metadata on the object as opc-meta-mtime
36824 as floating point since the epoch, accurate to 1 ns.
36825 If the modification time needs to be updated rclone will attempt to
36827 perform a server side copy to update the modification if the object can
36828 be copied in a single part. In the case the object is larger than 5Gb,
36829 the object will be uploaded rather than copied.
36830 Note that reading this from the object takes an additional HEAD request
36832 as the metadata isn't returned in object listings.
36833 Multipart uploads
36835 rclone supports multipart uploads with OOS which means that it can up‐
36836 load files bigger than 5 GiB.
36837 Note that files uploaded both with multipart upload and through crypt
36839 remotes do not have MD5 sums.
36840 rclone switches from single part uploads to multipart uploads at the
36842 point specified by --oos-upload-cutoff. This can be a maximum of 5 GiB
36843 and a minimum of 0 (ie always upload multipart files).
36844 The chunk sizes used in the multipart upload are specified by --oos-
36846 chunk-size and the number of chunks uploaded concurrently is specified
36847 by --oos-upload-concurrency.
36848 Multipart uploads will use --transfers * --oos-upload-concurrency *
36850 --oos-chunk-size extra memory. Single part uploads to not use extra
36851 memory.
36852 Single part transfers can be faster than multipart transfers or slower
36854 depending on your latency from oos - the more latency, the more likely
36855 single part transfers will be faster.
36856 Increasing --oos-upload-concurrency will increase throughput (8 would
36858 be a sensible value) and increasing --oos-chunk-size also increases
36859 throughput (16M would be sensible). Increasing either of these will
36860 use more memory. The default values are high enough to gain most of
36861 the possible performance without using too much memory.
36862 Standard options
36864 Here are the Standard options specific to oracleobjectstorage (Oracle
36865 Cloud Infrastructure Object Storage).
36866 --oos-provider
36868 Choose your Auth Provider
36869 Properties:
36871 • Config: provider
36873 • Env Var: RCLONE_OOS_PROVIDER
36875 • Type: string
36877 • Default: "env_auth"
36879 • Examples:
36881 • "env_auth"
36883 • automatically pickup the credentials from runtime(env), first one
36885 to provide auth wins
36886 • "user_principal_auth"
36888 • use an OCI user and an API key for authentication.
36890 • you’ll need to put in a config file your tenancy OCID, user OCID,
36892 region, the path, fingerprint to an API key.
36893 • https://docs.oracle.com/en-us/iaas/Content/API/Concepts/sdkcon‐
36895 fig.htm
36896 • "instance_principal_auth"
36898 • use instance principals to authorize an instance to make API
36900 calls.
36901 • each instance has its own identity, and authenticates using the
36903 certificates that are read from instance metadata.
36904 • https://docs.oracle.com/en-us/iaas/Content/Identity/Tasks/call‐
36906 ingservicesfrominstances.htm
36907 • "resource_principal_auth"
36909 • use resource principals to make API calls
36911 • "no_auth"
36913 • no credentials needed, this is typically for reading public buck‐
36915 ets
36916 --oos-namespace
36918 Object storage namespace
36919 Properties:
36921 • Config: namespace
36923 • Env Var: RCLONE_OOS_NAMESPACE
36925 • Type: string
36927 • Required: true
36929 --oos-compartment
36931 Object storage compartment OCID
36932 Properties:
36934 • Config: compartment
36936 • Env Var: RCLONE_OOS_COMPARTMENT
36938 • Provider: !no_auth
36940 • Type: string
36942 • Required: true
36944 --oos-region
36946 Object storage Region
36947 Properties:
36949 • Config: region
36951 • Env Var: RCLONE_OOS_REGION
36953 • Type: string
36955 • Required: true
36957 --oos-endpoint
36959 Endpoint for Object storage API.
36960 Leave blank to use the default endpoint for the region.
36962 Properties:
36964 • Config: endpoint
36966 • Env Var: RCLONE_OOS_ENDPOINT
36968 • Type: string
36970 • Required: false
36972 --oos-config-file
36974 Path to OCI config file
36975 Properties:
36977 • Config: config_file
36979 • Env Var: RCLONE_OOS_CONFIG_FILE
36981 • Provider: user_principal_auth
36983 • Type: string
36985 • Default: "~/.oci/config"
36987 • Examples:
36989 • "~/.oci/config"
36991 • oci configuration file location
36993 --oos-config-profile
36995 Profile name inside the oci config file
36996 Properties:
36998 • Config: config_profile
37000 • Env Var: RCLONE_OOS_CONFIG_PROFILE
37002 • Provider: user_principal_auth
37004 • Type: string
37006 • Default: "Default"
37008 • Examples:
37010 • "Default"
37012 • Use the default profile
37014 Advanced options
37016 Here are the Advanced options specific to oracleobjectstorage (Oracle
37017 Cloud Infrastructure Object Storage).
37018 --oos-upload-cutoff
37020 Cutoff for switching to chunked upload.
37021 Any files larger than this will be uploaded in chunks of chunk_size.
37023 The minimum is 0 and the maximum is 5 GiB.
37024 Properties:
37026 • Config: upload_cutoff
37028 • Env Var: RCLONE_OOS_UPLOAD_CUTOFF
37030 • Type: SizeSuffix
37032 • Default: 200Mi
37034 --oos-chunk-size
37036 Chunk size to use for uploading.
37037 When uploading files larger than upload_cutoff or files with unknown
37039 size (e.g. from "rclone rcat" or uploaded with "rclone mount" or
37040 google photos or google docs) they will be uploaded as multipart up‐
37041 loads using this chunk size.
37042 Note that "upload_concurrency" chunks of this size are buffered in mem‐
37044 ory per transfer.
37045 If you are transferring large files over high-speed links and you have
37047 enough memory, then increasing this will speed up the transfers.
37048 Rclone will automatically increase the chunk size when uploading a
37050 large file of known size to stay below the 10,000 chunks limit.
37051 Files of unknown size are uploaded with the configured chunk_size.
37053 Since the default chunk size is 5 MiB and there can be at most 10,000
37054 chunks, this means that by default the maximum size of a file you can
37055 stream upload is 48 GiB. If you wish to stream upload larger files
37056 then you will need to increase chunk_size.
37057 Increasing the chunk size decreases the accuracy of the progress sta‐
37059 tistics displayed with "-P" flag.
37060 Properties:
37062 • Config: chunk_size
37064 • Env Var: RCLONE_OOS_CHUNK_SIZE
37066 • Type: SizeSuffix
37068 • Default: 5Mi
37070 --oos-upload-concurrency
37072 Concurrency for multipart uploads.
37073 This is the number of chunks of the same file that are uploaded concur‐
37075 rently.
37076 If you are uploading small numbers of large files over high-speed links
37078 and these uploads do not fully utilize your bandwidth, then increasing
37079 this may help to speed up the transfers.
37080 Properties:
37082 • Config: upload_concurrency
37084 • Env Var: RCLONE_OOS_UPLOAD_CONCURRENCY
37086 • Type: int
37088 • Default: 10
37090 --oos-copy-cutoff
37092 Cutoff for switching to multipart copy.
37093 Any files larger than this that need to be server-side copied will be
37095 copied in chunks of this size.
37096 The minimum is 0 and the maximum is 5 GiB.
37098 Properties:
37100 • Config: copy_cutoff
37102 • Env Var: RCLONE_OOS_COPY_CUTOFF
37104 • Type: SizeSuffix
37106 • Default: 4.656Gi
37108 --oos-copy-timeout
37110 Timeout for copy.
37111 Copy is an asynchronous operation, specify timeout to wait for copy to
37113 succeed
37114 Properties:
37116 • Config: copy_timeout
37118 • Env Var: RCLONE_OOS_COPY_TIMEOUT
37120 • Type: Duration
37122 • Default: 1m0s
37124 --oos-disable-checksum
37126 Don't store MD5 checksum with object metadata.
37127 Normally rclone will calculate the MD5 checksum of the input before up‐
37129 loading it so it can add it to metadata on the object. This is great
37130 for data integrity checking but can cause long delays for large files
37131 to start uploading.
37132 Properties:
37134 • Config: disable_checksum
37136 • Env Var: RCLONE_OOS_DISABLE_CHECKSUM
37138 • Type: bool
37140 • Default: false
37142 --oos-encoding
37144 The encoding for the backend.
37145 See the encoding section in the overview (https://rclone.org/over‐
37147 view/#encoding) for more info.
37148 Properties:
37150 • Config: encoding
37152 • Env Var: RCLONE_OOS_ENCODING
37154 • Type: MultiEncoder
37156 • Default: Slash,InvalidUtf8,Dot
37158 --oos-leave-parts-on-error
37160 If true avoid calling abort upload on a failure, leaving all success‐
37161 fully uploaded parts on S3 for manual recovery.
37162 It should be set to true for resuming uploads across different ses‐
37164 sions.
37165 WARNING: Storing parts of an incomplete multipart upload counts towards
37167 space usage on object storage and will add additional costs if not
37168 cleaned up.
37169 Properties:
37171 • Config: leave_parts_on_error
37173 • Env Var: RCLONE_OOS_LEAVE_PARTS_ON_ERROR
37175 • Type: bool
37177 • Default: false
37179 --oos-no-check-bucket
37181 If set, don't attempt to check the bucket exists or create it.
37182 This can be useful when trying to minimise the number of transactions
37184 rclone does if you know the bucket exists already.
37185 It can also be needed if the user you are using does not have bucket
37187 creation permissions.
37188 Properties:
37190 • Config: no_check_bucket
37192 • Env Var: RCLONE_OOS_NO_CHECK_BUCKET
37194 • Type: bool
37196 • Default: false
37198 Backend commands
37200 Here are the commands specific to the oracleobjectstorage backend.
37201 Run them with
37203 rclone backend COMMAND remote:
37205 The help below will explain what arguments each command takes.
37207 See the backend (https://rclone.org/commands/rclone_backend/) command
37209 for more info on how to pass options and arguments.
37210 These can be run on a running backend using the rc command backend/com‐
37212 mand (https://rclone.org/rc/#backend-command).
37213 rename
37215 change the name of an object
37216 rclone backend rename remote: [options] [<arguments>+]
37218 This command can be used to rename a object.
37220 Usage Examples:
37222 rclone backend rename oos:bucket relative-object-path-under-bucket object-new-name
37224 list-multipart-uploads
37226 List the unfinished multipart uploads
37227 rclone backend list-multipart-uploads remote: [options] [<arguments>+]
37229 This command lists the unfinished multipart uploads in JSON format.
37231 rclone backend list-multipart-uploads oos:bucket/path/to/object
37233 It returns a dictionary of buckets with values as lists of unfinished
37235 multipart uploads.
37236 You can call it with no bucket in which case it lists all bucket, with
37238 a bucket or with a bucket and path.
37239 {
37241 "test-bucket": [
37242 {
37243 "namespace": "test-namespace",
37244 "bucket": "test-bucket",
37245 "object": "600m.bin",
37246 "uploadId": "51dd8114-52a4-b2f2-c42f-5291f05eb3c8",
37247 "timeCreated": "2022-07-29T06:21:16.595Z",
37248 "storageTier": "Standard"
37249 }
37250 ]
37251 cleanup
37253 Remove unfinished multipart uploads.
37254 rclone backend cleanup remote: [options] [<arguments>+]
37256 This command removes unfinished multipart uploads of age greater than
37258 max-age which defaults to 24 hours.
37259 Note that you can use -i/--dry-run with this command to see what it
37261 would do.
37262 rclone backend cleanup oos:bucket/path/to/object
37264 rclone backend cleanup -o max-age=7w oos:bucket/path/to/object
37265 Durations are parsed as per the rest of rclone, 2h, 7d, 7w etc.
37267 Options:
37269 • "max-age": Max age of upload to delete
37271
37273 Paths are specified as remote:bucket (or remote: for the lsd command.)
37274 You may put subdirectories in too, e.g. remote:bucket/path/to/dir.
37275 Configuration
37277 Here is an example of making an QingStor configuration. First run
37278 rclone config
37280 This will guide you through an interactive setup process.
37282 No remotes found, make a new one?
37284 n) New remote
37285 r) Rename remote
37286 c) Copy remote
37287 s) Set configuration password
37288 q) Quit config
37289 n/r/c/s/q> n
37290 name> remote
37291 Type of storage to configure.
37292 Choose a number from below, or type in your own value
37293 [snip]
37294 XX / QingStor Object Storage
37295 \ "qingstor"
37296 [snip]
37297 Storage> qingstor
37298 Get QingStor credentials from runtime. Only applies if access_key_id and secret_access_key is blank.
37299 Choose a number from below, or type in your own value
37300 1 / Enter QingStor credentials in the next step
37301 \ "false"
37302 2 / Get QingStor credentials from the environment (env vars or IAM)
37303 \ "true"
37304 env_auth> 1
37305 QingStor Access Key ID - leave blank for anonymous access or runtime credentials.
37306 access_key_id> access_key
37307 QingStor Secret Access Key (password) - leave blank for anonymous access or runtime credentials.
37308 secret_access_key> secret_key
37309 Enter an endpoint URL to connection QingStor API.
37310 Leave blank will use the default value "https://qingstor.com:443"
37311 endpoint>
37312 Zone connect to. Default is "pek3a".
37313 Choose a number from below, or type in your own value
37314 / The Beijing (China) Three Zone
37315 1 | Needs location constraint pek3a.
37316 \ "pek3a"
37317 / The Shanghai (China) First Zone
37318 2 | Needs location constraint sh1a.
37319 \ "sh1a"
37320 zone> 1
37321 Number of connection retry.
37322 Leave blank will use the default value "3".
37323 connection_retries>
37324 Remote config
37325 --------------------
37326 [remote]
37327 env_auth = false
37328 access_key_id = access_key
37329 secret_access_key = secret_key
37330 endpoint =
37331 zone = pek3a
37332 connection_retries =
37333 --------------------
37334 y) Yes this is OK
37335 e) Edit this remote
37336 d) Delete this remote
37337 y/e/d> y
37338 This remote is called remote and can now be used like this
37340 See all buckets
37342 rclone lsd remote:
37344 Make a new bucket
37346 rclone mkdir remote:bucket
37348 List the contents of a bucket
37350 rclone ls remote:bucket
37352 Sync /home/local/directory to the remote bucket, deleting any excess
37354 files in the bucket.
37355 rclone sync -i /home/local/directory remote:bucket
37357 --fast-list
37359 This remote supports --fast-list which allows you to use fewer transac‐
37360 tions in exchange for more memory. See the rclone docs
37361 (https://rclone.org/docs/#fast-list) for more details.
37362 Multipart uploads
37364 rclone supports multipart uploads with QingStor which means that it can
37365 upload files bigger than 5 GiB. Note that files uploaded with multi‐
37366 part upload don't have an MD5SUM.
37367 Note that incomplete multipart uploads older than 24 hours can be re‐
37369 moved with rclone cleanup remote:bucket just for one bucket rclone
37370 cleanup remote: for all buckets. QingStor does not ever remove incom‐
37371 plete multipart uploads so it may be necessary to run this from time to
37372 time.
37373 Buckets and Zone
37375 With QingStor you can list buckets (rclone lsd) using any zone, but you
37376 can only access the content of a bucket from the zone it was created
37377 in. If you attempt to access a bucket from the wrong zone, you will
37378 get an error, incorrect zone, the bucket is not in 'XXX' zone.
37379 Authentication
37381 There are two ways to supply rclone with a set of QingStor credentials.
37382 In order of precedence:
37383 • Directly in the rclone configuration file (as configured by rclone
37385 config)
37386 • set access_key_id and secret_access_key
37388 • Runtime configuration:
37390 • set env_auth to true in the config file
37392 • Exporting the following environment variables before running rclone
37394 • Access Key ID: QS_ACCESS_KEY_ID or QS_ACCESS_KEY
37396 • Secret Access Key: QS_SECRET_ACCESS_KEY or QS_SECRET_KEY
37398 Restricted filename characters
37400 The control characters 0x00-0x1F and / are replaced as in the default
37401 restricted characters set (https://rclone.org/overview/#restricted-
37402 characters). Note that 0x7F is not replaced.
37403 Invalid UTF-8 bytes will also be replaced (https://rclone.org/over‐
37405 view/#invalid-utf8), as they can't be used in JSON strings.
37406 Standard options
37408 Here are the Standard options specific to qingstor (QingCloud Object
37409 Storage).
37410 --qingstor-env-auth
37412 Get QingStor credentials from runtime.
37413 Only applies if access_key_id and secret_access_key is blank.
37415 Properties:
37417 • Config: env_auth
37419 • Env Var: RCLONE_QINGSTOR_ENV_AUTH
37421 • Type: bool
37423 • Default: false
37425 • Examples:
37427 • "false"
37429 • Enter QingStor credentials in the next step.
37431 • "true"
37433 • Get QingStor credentials from the environment (env vars or IAM).
37435 --qingstor-access-key-id
37437 QingStor Access Key ID.
37438 Leave blank for anonymous access or runtime credentials.
37440 Properties:
37442 • Config: access_key_id
37444 • Env Var: RCLONE_QINGSTOR_ACCESS_KEY_ID
37446 • Type: string
37448 • Required: false
37450 --qingstor-secret-access-key
37452 QingStor Secret Access Key (password).
37453 Leave blank for anonymous access or runtime credentials.
37455 Properties:
37457 • Config: secret_access_key
37459 • Env Var: RCLONE_QINGSTOR_SECRET_ACCESS_KEY
37461 • Type: string
37463 • Required: false
37465 --qingstor-endpoint
37467 Enter an endpoint URL to connection QingStor API.
37468 Leave blank will use the default value "https://qingstor.com:443".
37470 Properties:
37472 • Config: endpoint
37474 • Env Var: RCLONE_QINGSTOR_ENDPOINT
37476 • Type: string
37478 • Required: false
37480 --qingstor-zone
37482 Zone to connect to.
37483 Default is "pek3a".
37485 Properties:
37487 • Config: zone
37489 • Env Var: RCLONE_QINGSTOR_ZONE
37491 • Type: string
37493 • Required: false
37495 • Examples:
37497 • "pek3a"
37499 • The Beijing (China) Three Zone.
37501 • Needs location constraint pek3a.
37503 • "sh1a"
37505 • The Shanghai (China) First Zone.
37507 • Needs location constraint sh1a.
37509 • "gd2a"
37511 • The Guangdong (China) Second Zone.
37513 • Needs location constraint gd2a.
37515 Advanced options
37517 Here are the Advanced options specific to qingstor (QingCloud Object
37518 Storage).
37519 --qingstor-connection-retries
37521 Number of connection retries.
37522 Properties:
37524 • Config: connection_retries
37526 • Env Var: RCLONE_QINGSTOR_CONNECTION_RETRIES
37528 • Type: int
37530 • Default: 3
37532 --qingstor-upload-cutoff
37534 Cutoff for switching to chunked upload.
37535 Any files larger than this will be uploaded in chunks of chunk_size.
37537 The minimum is 0 and the maximum is 5 GiB.
37538 Properties:
37540 • Config: upload_cutoff
37542 • Env Var: RCLONE_QINGSTOR_UPLOAD_CUTOFF
37544 • Type: SizeSuffix
37546 • Default: 200Mi
37548 --qingstor-chunk-size
37550 Chunk size to use for uploading.
37551 When uploading files larger than upload_cutoff they will be uploaded as
37553 multipart uploads using this chunk size.
37554 Note that "--qingstor-upload-concurrency" chunks of this size are
37556 buffered in memory per transfer.
37557 If you are transferring large files over high-speed links and you have
37559 enough memory, then increasing this will speed up the transfers.
37560 Properties:
37562 • Config: chunk_size
37564 • Env Var: RCLONE_QINGSTOR_CHUNK_SIZE
37566 • Type: SizeSuffix
37568 • Default: 4Mi
37570 --qingstor-upload-concurrency
37572 Concurrency for multipart uploads.
37573 This is the number of chunks of the same file that are uploaded concur‐
37575 rently.
37576 NB if you set this to > 1 then the checksums of multipart uploads be‐
37578 come corrupted (the uploads themselves are not corrupted though).
37579 If you are uploading small numbers of large files over high-speed links
37581 and these uploads do not fully utilize your bandwidth, then increasing
37582 this may help to speed up the transfers.
37583 Properties:
37585 • Config: upload_concurrency
37587 • Env Var: RCLONE_QINGSTOR_UPLOAD_CONCURRENCY
37589 • Type: int
37591 • Default: 1
37593 --qingstor-encoding
37595 The encoding for the backend.
37596 See the encoding section in the overview (https://rclone.org/over‐
37598 view/#encoding) for more info.
37599 Properties:
37601 • Config: encoding
37603 • Env Var: RCLONE_QINGSTOR_ENCODING
37605 • Type: MultiEncoder
37607 • Default: Slash,Ctl,InvalidUtf8
37609 Limitations
37611 rclone about is not supported by the qingstor backend. Backends with‐
37612 out this capability cannot determine free space for an rclone mount or
37613 use policy mfs (most free space) as a member of an rclone union remote.
37614 See List of backends that do not support rclone about
37616 (https://rclone.org/overview/#optional-features) and rclone about
37617 (https://rclone.org/commands/rclone_about/)
37618
37620 Sia (sia.tech (https://sia.tech/)) is a decentralized cloud storage
37621 platform based on the blockchain (https://wikipedia.org/wi‐
37622 ki/Blockchain) technology. With rclone you can use it like any other
37623 remote filesystem or mount Sia folders locally. The technology behind
37624 it involves a number of new concepts such as Siacoins and Wallet,
37625 Blockchain and Consensus, Renting and Hosting, and so on. If you are
37626 new to it, you'd better first familiarize yourself using their excel‐
37627 lent support documentation (https://support.sia.tech/).
37628 Introduction
37630 Before you can use rclone with Sia, you will need to have a running
37631 copy of Sia-UI or siad (the Sia daemon) locally on your computer or on
37632 local network (e.g. a NAS). Please follow the Get started
37633 (https://sia.tech/get-started) guide and install one.
37634 rclone interacts with Sia network by talking to the Sia daemon via HTTP
37636 API (https://sia.tech/docs/) which is usually available on port 9980.
37637 By default you will run the daemon locally on the same computer so it's
37638 safe to leave the API password blank (the API URL will be
37639 http://127.0.0.1:9980 making external access impossible).
37640 However, if you want to access Sia daemon running on another node, for
37642 example due to memory constraints or because you want to share single
37643 daemon between several rclone and Sia-UI instances, you'll need to make
37644 a few more provisions: - Ensure you have Sia daemon installed directly
37645 or in a docker container (https://github.com/SiaFounda‐
37646 tion/siad/pkgs/container/siad) because Sia-UI does not support this
37647 mode natively. - Run it on externally accessible port, for example
37648 provide --api-addr :9980 and --disable-api-security arguments on the
37649 daemon command line. - Enforce API password for the siad daemon via
37650 environment variable SIA_API_PASSWORD or text file named apipassword in
37651 the daemon directory. - Set rclone backend option api_password taking
37652 it from above locations.
37653 Notes: 1. If your wallet is locked, rclone cannot unlock it automati‐
37655 cally. You should either unlock it in advance by using Sia-UI or via
37656 command line siac wallet unlock. Alternatively you can make siad un‐
37657 lock your wallet automatically upon startup by running it with environ‐
37658 ment variable SIA_WALLET_PASSWORD. 2. If siad cannot find the
37659 SIA_API_PASSWORD variable or the apipassword file in the SIA_DIR direc‐
37660 tory, it will generate a random password and store in the text file
37661 named apipassword under YOUR_HOME/.sia/ directory on Unix or
37662 C:\Users\YOUR_HOME\AppData\Local\Sia\apipassword on Windows. Remember
37663 this when you configure password in rclone. 3. The only way to use
37664 siad without API password is to run it on localhost with command line
37665 argument --authorize-api=false, but this is insecure and strongly dis‐
37666 couraged.
37667 Configuration
37669 Here is an example of how to make a sia remote called mySia. First,
37670 run:
37671 rclone config
37673 This will guide you through an interactive setup process:
37675 No remotes found, make a new one?
37677 n) New remote
37678 s) Set configuration password
37679 q) Quit config
37680 n/s/q> n
37681 name> mySia
37682 Type of storage to configure.
37683 Enter a string value. Press Enter for the default ("").
37684 Choose a number from below, or type in your own value
37685 ...
37686 29 / Sia Decentralized Cloud
37687 \ "sia"
37688 ...
37689 Storage> sia
37690 Sia daemon API URL, like http://sia.daemon.host:9980.
37691 Note that siad must run with --disable-api-security to open API port for other hosts (not recommended).
37692 Keep default if Sia daemon runs on localhost.
37693 Enter a string value. Press Enter for the default ("http://127.0.0.1:9980").
37694 api_url> http://127.0.0.1:9980
37695 Sia Daemon API Password.
37696 Can be found in the apipassword file located in HOME/.sia/ or in the daemon directory.
37697 y) Yes type in my own password
37698 g) Generate random password
37699 n) No leave this optional password blank (default)
37700 y/g/n> y
37701 Enter the password:
37702 password:
37703 Confirm the password:
37704 password:
37705 Edit advanced config?
37706 y) Yes
37707 n) No (default)
37708 y/n> n
37709 --------------------
37710 [mySia]
37711 type = sia
37712 api_url = http://127.0.0.1:9980
37713 api_password = *** ENCRYPTED ***
37714 --------------------
37715 y) Yes this is OK (default)
37716 e) Edit this remote
37717 d) Delete this remote
37718 y/e/d> y
37719 Once configured, you can then use rclone like this:
37721 • List directories in top level of your Sia storage
37723 rclone lsd mySia:
37725 • List all the files in your Sia storage
37727 rclone ls mySia:
37729 • Upload a local directory to the Sia directory called backup
37731 rclone copy /home/source mySia:backup
37733 Standard options
37735 Here are the Standard options specific to sia (Sia Decentralized
37736 Cloud).
37737 --sia-api-url
37739 Sia daemon API URL, like http://sia.daemon.host:9980.
37740 Note that siad must run with --disable-api-security to open API port
37742 for other hosts (not recommended). Keep default if Sia daemon runs on
37743 localhost.
37744 Properties:
37746 • Config: api_url
37748 • Env Var: RCLONE_SIA_API_URL
37750 • Type: string
37752 • Default: "http://127.0.0.1:9980"
37754 --sia-api-password
37756 Sia Daemon API Password.
37757 Can be found in the apipassword file located in HOME/.sia/ or in the
37759 daemon directory.
37760 NB Input to this must be obscured - see rclone obscure
37762 (https://rclone.org/commands/rclone_obscure/).
37763 Properties:
37765 • Config: api_password
37767 • Env Var: RCLONE_SIA_API_PASSWORD
37769 • Type: string
37771 • Required: false
37773 Advanced options
37775 Here are the Advanced options specific to sia (Sia Decentralized
37776 Cloud).
37777 --sia-user-agent
37779 Siad User Agent
37780 Sia daemon requires the 'Sia-Agent' user agent by default for security
37782 Properties:
37784 • Config: user_agent
37786 • Env Var: RCLONE_SIA_USER_AGENT
37788 • Type: string
37790 • Default: "Sia-Agent"
37792 --sia-encoding
37794 The encoding for the backend.
37795 See the encoding section in the overview (https://rclone.org/over‐
37797 view/#encoding) for more info.
37798 Properties:
37800 • Config: encoding
37802 • Env Var: RCLONE_SIA_ENCODING
37804 • Type: MultiEncoder
37806 • Default: Slash,Question,Hash,Percent,Del,Ctl,InvalidUtf8,Dot
37808 Limitations
37810 • Modification times not supported
37811 • Checksums not supported
37813 • rclone about not supported
37815 • rclone can work only with Siad or Sia-UI at the moment, the SkyNet
37817 daemon is not supported yet.
37818 • Sia does not allow control characters or symbols like question and
37820 pound signs in file names. rclone will transparently encode
37821 (https://rclone.org/overview/#encoding) them for you, but you'd bet‐
37822 ter be aware
37823
37825 Swift refers to OpenStack Object Storage (https://docs.open‐
37826 stack.org/swift/latest/). Commercial implementations of that being:
37827 • Rackspace Cloud Files (https://www.rackspace.com/cloud/files/)
37829 • Memset Memstore (https://www.memset.com/cloud/storage/)
37831 • OVH Object Storage (https://www.ovh.co.uk/public-cloud/storage/ob‐
37833 ject-storage/)
37834 • Oracle Cloud Storage (https://docs.oracle.com/en-us/iaas/integra‐
37836 tion/doc/configure-object-storage.html)
37837 • IBM Bluemix Cloud ObjectStorage Swift (https://con‐
37839 sole.bluemix.net/docs/infrastructure/objectstorage-swift/index.html)
37840 Paths are specified as remote:container (or remote: for the lsd com‐
37842 mand.) You may put subdirectories in too, e.g. remote:contain‐
37843 er/path/to/dir.
37844 Configuration
37846 Here is an example of making a swift configuration. First run
37847 rclone config
37849 This will guide you through an interactive setup process.
37851 No remotes found, make a new one?
37853 n) New remote
37854 s) Set configuration password
37855 q) Quit config
37856 n/s/q> n
37857 name> remote
37858 Type of storage to configure.
37859 Choose a number from below, or type in your own value
37860 [snip]
37861 XX / OpenStack Swift (Rackspace Cloud Files, Memset Memstore, OVH)
37862 \ "swift"
37863 [snip]
37864 Storage> swift
37865 Get swift credentials from environment variables in standard OpenStack form.
37866 Choose a number from below, or type in your own value
37867 1 / Enter swift credentials in the next step
37868 \ "false"
37869 2 / Get swift credentials from environment vars. Leave other fields blank if using this.
37870 \ "true"
37871 env_auth> true
37872 User name to log in (OS_USERNAME).
37873 user>
37874 API key or password (OS_PASSWORD).
37875 key>
37876 Authentication URL for server (OS_AUTH_URL).
37877 Choose a number from below, or type in your own value
37878 1 / Rackspace US
37879 \ "https://auth.api.rackspacecloud.com/v1.0"
37880 2 / Rackspace UK
37881 \ "https://lon.auth.api.rackspacecloud.com/v1.0"
37882 3 / Rackspace v2
37883 \ "https://identity.api.rackspacecloud.com/v2.0"
37884 4 / Memset Memstore UK
37885 \ "https://auth.storage.memset.com/v1.0"
37886 5 / Memset Memstore UK v2
37887 \ "https://auth.storage.memset.com/v2.0"
37888 6 / OVH
37889 \ "https://auth.cloud.ovh.net/v3"
37890 auth>
37891 User ID to log in - optional - most swift systems use user and leave this blank (v3 auth) (OS_USER_ID).
37892 user_id>
37893 User domain - optional (v3 auth) (OS_USER_DOMAIN_NAME)
37894 domain>
37895 Tenant name - optional for v1 auth, this or tenant_id required otherwise (OS_TENANT_NAME or OS_PROJECT_NAME)
37896 tenant>
37897 Tenant ID - optional for v1 auth, this or tenant required otherwise (OS_TENANT_ID)
37898 tenant_id>
37899 Tenant domain - optional (v3 auth) (OS_PROJECT_DOMAIN_NAME)
37900 tenant_domain>
37901 Region name - optional (OS_REGION_NAME)
37902 region>
37903 Storage URL - optional (OS_STORAGE_URL)
37904 storage_url>
37905 Auth Token from alternate authentication - optional (OS_AUTH_TOKEN)
37906 auth_token>
37907 AuthVersion - optional - set to (1,2,3) if your auth URL has no version (ST_AUTH_VERSION)
37908 auth_version>
37909 Endpoint type to choose from the service catalogue (OS_ENDPOINT_TYPE)
37910 Choose a number from below, or type in your own value
37911 1 / Public (default, choose this if not sure)
37912 \ "public"
37913 2 / Internal (use internal service net)
37914 \ "internal"
37915 3 / Admin
37916 \ "admin"
37917 endpoint_type>
37918 Remote config
37919 --------------------
37920 [test]
37921 env_auth = true
37922 user =
37923 key =
37924 auth =
37925 user_id =
37926 domain =
37927 tenant =
37928 tenant_id =
37929 tenant_domain =
37930 region =
37931 storage_url =
37932 auth_token =
37933 auth_version =
37934 endpoint_type =
37935 --------------------
37936 y) Yes this is OK
37937 e) Edit this remote
37938 d) Delete this remote
37939 y/e/d> y
37940 This remote is called remote and can now be used like this
37942 See all containers
37944 rclone lsd remote:
37946 Make a new container
37948 rclone mkdir remote:container
37950 List the contents of a container
37952 rclone ls remote:container
37954 Sync /home/local/directory to the remote container, deleting any excess
37956 files in the container.
37957 rclone sync -i /home/local/directory remote:container
37959 Configuration from an OpenStack credentials file
37961 An OpenStack credentials file typically looks something something like
37962 this (without the comments)
37963 export OS_AUTH_URL=https://a.provider.net/v2.0
37965 export OS_TENANT_ID=ffffffffffffffffffffffffffffffff
37966 export OS_TENANT_NAME="1234567890123456"
37967 export OS_USERNAME="123abc567xy"
37968 echo "Please enter your OpenStack Password: "
37969 read -sr OS_PASSWORD_INPUT
37970 export OS_PASSWORD=$OS_PASSWORD_INPUT
37971 export OS_REGION_NAME="SBG1"
37972 if [ -z "$OS_REGION_NAME" ]; then unset OS_REGION_NAME; fi
37973 The config file needs to look something like this where $OS_USERNAME
37975 represents the value of the OS_USERNAME variable - 123abc567xy in the
37976 example above.
37977 [remote]
37979 type = swift
37980 user = $OS_USERNAME
37981 key = $OS_PASSWORD
37982 auth = $OS_AUTH_URL
37983 tenant = $OS_TENANT_NAME
37984 Note that you may (or may not) need to set region too - try without
37986 first.
37987 Configuration from the environment
37989 If you prefer you can configure rclone to use swift using a standard
37990 set of OpenStack environment variables.
37991 When you run through the config, make sure you choose true for env_auth
37993 and leave everything else blank.
37994 rclone will then set any empty config parameters from the environment
37996 using standard OpenStack environment variables. There is a list of the
37997 variables (https://godoc.org/github.com/ncw/swift#Connection.ApplyEnvi‐
37998 ronment) in the docs for the swift library.
37999 Using an alternate authentication method
38001 If your OpenStack installation uses a non-standard authentication meth‐
38002 od that might not be yet supported by rclone or the underlying swift
38003 library, you can authenticate externally (e.g. calling manually the
38004 openstack commands to get a token). Then, you just need to pass the
38005 two configuration variables auth_token and storage_url. If they are
38006 both provided, the other variables are ignored. rclone will not try to
38007 authenticate but instead assume it is already authenticated and use
38008 these two variables to access the OpenStack installation.
38009 Using rclone without a config file
38011 You can use rclone with swift without a config file, if desired, like
38012 this:
38013 source openstack-credentials-file
38015 export RCLONE_CONFIG_MYREMOTE_TYPE=swift
38016 export RCLONE_CONFIG_MYREMOTE_ENV_AUTH=true
38017 rclone lsd myremote:
38018 --fast-list
38020 This remote supports --fast-list which allows you to use fewer transac‐
38021 tions in exchange for more memory. See the rclone docs
38022 (https://rclone.org/docs/#fast-list) for more details.
38023 --update and --use-server-modtime
38025 As noted below, the modified time is stored on metadata on the object.
38026 It is used by default for all operations that require checking the time
38027 a file was last updated. It allows rclone to treat the remote more
38028 like a true filesystem, but it is inefficient because it requires an
38029 extra API call to retrieve the metadata.
38030 For many operations, the time the object was last uploaded to the re‐
38032 mote is sufficient to determine if it is "dirty". By using --update
38033 along with --use-server-modtime, you can avoid the extra API call and
38034 simply upload files whose local modtime is newer than the time it was
38035 last uploaded.
38036 Modified time
38038 The modified time is stored as metadata on the object as X-Object-Meta-
38039 Mtime as floating point since the epoch accurate to 1 ns.
38040 This is a de facto standard (used in the official python-swiftclient
38042 amongst others) for storing the modification time for an object.
38043 Restricted filename characters
38045 Character Value Replacement
38046 ────────────────────────────────
38047 NUL 0x00 ␀
38048 / 0x2F ／
38049 Invalid UTF-8 bytes will also be replaced (https://rclone.org/over‐
38051 view/#invalid-utf8), as they can't be used in JSON strings.
38052 Standard options
38054 Here are the Standard options specific to swift (OpenStack Swift
38055 (Rackspace Cloud Files, Memset Memstore, OVH)).
38056 --swift-env-auth
38058 Get swift credentials from environment variables in standard OpenStack
38059 form.
38060 Properties:
38062 • Config: env_auth
38064 • Env Var: RCLONE_SWIFT_ENV_AUTH
38066 • Type: bool
38068 • Default: false
38070 • Examples:
38072 • "false"
38074 • Enter swift credentials in the next step.
38076 • "true"
38078 • Get swift credentials from environment vars.
38080 • Leave other fields blank if using this.
38082 --swift-user
38084 User name to log in (OS_USERNAME).
38085 Properties:
38087 • Config: user
38089 • Env Var: RCLONE_SWIFT_USER
38091 • Type: string
38093 • Required: false
38095 --swift-key
38097 API key or password (OS_PASSWORD).
38098 Properties:
38100 • Config: key
38102 • Env Var: RCLONE_SWIFT_KEY
38104 • Type: string
38106 • Required: false
38108 --swift-auth
38110 Authentication URL for server (OS_AUTH_URL).
38111 Properties:
38113 • Config: auth
38115 • Env Var: RCLONE_SWIFT_AUTH
38117 • Type: string
38119 • Required: false
38121 • Examples:
38123 • "https://auth.api.rackspacecloud.com/v1.0"
38125 • Rackspace US
38127 • "https://lon.auth.api.rackspacecloud.com/v1.0"
38129 • Rackspace UK
38131 • "https://identity.api.rackspacecloud.com/v2.0"
38133 • Rackspace v2
38135 • "https://auth.storage.memset.com/v1.0"
38137 • Memset Memstore UK
38139 • "https://auth.storage.memset.com/v2.0"
38141 • Memset Memstore UK v2
38143 • "https://auth.cloud.ovh.net/v3"
38145 • OVH
38147 --swift-user-id
38149 User ID to log in - optional - most swift systems use user and leave
38150 this blank (v3 auth) (OS_USER_ID).
38151 Properties:
38153 • Config: user_id
38155 • Env Var: RCLONE_SWIFT_USER_ID
38157 • Type: string
38159 • Required: false
38161 --swift-domain
38163 User domain - optional (v3 auth) (OS_USER_DOMAIN_NAME)
38164 Properties:
38166 • Config: domain
38168 • Env Var: RCLONE_SWIFT_DOMAIN
38170 • Type: string
38172 • Required: false
38174 --swift-tenant
38176 Tenant name - optional for v1 auth, this or tenant_id required other‐
38177 wise (OS_TENANT_NAME or OS_PROJECT_NAME).
38178 Properties:
38180 • Config: tenant
38182 • Env Var: RCLONE_SWIFT_TENANT
38184 • Type: string
38186 • Required: false
38188 --swift-tenant-id
38190 Tenant ID - optional for v1 auth, this or tenant required otherwise
38191 (OS_TENANT_ID).
38192 Properties:
38194 • Config: tenant_id
38196 • Env Var: RCLONE_SWIFT_TENANT_ID
38198 • Type: string
38200 • Required: false
38202 --swift-tenant-domain
38204 Tenant domain - optional (v3 auth) (OS_PROJECT_DOMAIN_NAME).
38205 Properties:
38207 • Config: tenant_domain
38209 • Env Var: RCLONE_SWIFT_TENANT_DOMAIN
38211 • Type: string
38213 • Required: false
38215 --swift-region
38217 Region name - optional (OS_REGION_NAME).
38218 Properties:
38220 • Config: region
38222 • Env Var: RCLONE_SWIFT_REGION
38224 • Type: string
38226 • Required: false
38228 --swift-storage-url
38230 Storage URL - optional (OS_STORAGE_URL).
38231 Properties:
38233 • Config: storage_url
38235 • Env Var: RCLONE_SWIFT_STORAGE_URL
38237 • Type: string
38239 • Required: false
38241 --swift-auth-token
38243 Auth Token from alternate authentication - optional (OS_AUTH_TOKEN).
38244 Properties:
38246 • Config: auth_token
38248 • Env Var: RCLONE_SWIFT_AUTH_TOKEN
38250 • Type: string
38252 • Required: false
38254 --swift-application-credential-id
38256 Application Credential ID (OS_APPLICATION_CREDENTIAL_ID).
38257 Properties:
38259 • Config: application_credential_id
38261 • Env Var: RCLONE_SWIFT_APPLICATION_CREDENTIAL_ID
38263 • Type: string
38265 • Required: false
38267 --swift-application-credential-name
38269 Application Credential Name (OS_APPLICATION_CREDENTIAL_NAME).
38270 Properties:
38272 • Config: application_credential_name
38274 • Env Var: RCLONE_SWIFT_APPLICATION_CREDENTIAL_NAME
38276 • Type: string
38278 • Required: false
38280 --swift-application-credential-secret
38282 Application Credential Secret (OS_APPLICATION_CREDENTIAL_SECRET).
38283 Properties:
38285 • Config: application_credential_secret
38287 • Env Var: RCLONE_SWIFT_APPLICATION_CREDENTIAL_SECRET
38289 • Type: string
38291 • Required: false
38293 --swift-auth-version
38295 AuthVersion - optional - set to (1,2,3) if your auth URL has no version
38296 (ST_AUTH_VERSION).
38297 Properties:
38299 • Config: auth_version
38301 • Env Var: RCLONE_SWIFT_AUTH_VERSION
38303 • Type: int
38305 • Default: 0
38307 --swift-endpoint-type
38309 Endpoint type to choose from the service catalogue (OS_ENDPOINT_TYPE).
38310 Properties:
38312 • Config: endpoint_type
38314 • Env Var: RCLONE_SWIFT_ENDPOINT_TYPE
38316 • Type: string
38318 • Default: "public"
38320 • Examples:
38322 • "public"
38324 • Public (default, choose this if not sure)
38326 • "internal"
38328 • Internal (use internal service net)
38330 • "admin"
38332 • Admin
38334 --swift-storage-policy
38336 The storage policy to use when creating a new container.
38337 This applies the specified storage policy when creating a new contain‐
38339 er. The policy cannot be changed afterwards. The allowed configura‐
38340 tion values and their meaning depend on your Swift storage provider.
38341 Properties:
38343 • Config: storage_policy
38345 • Env Var: RCLONE_SWIFT_STORAGE_POLICY
38347 • Type: string
38349 • Required: false
38351 • Examples:
38353 • ""
38355 • Default
38357 • "pcs"
38359 • OVH Public Cloud Storage
38361 • "pca"
38363 • OVH Public Cloud Archive
38365 Advanced options
38367 Here are the Advanced options specific to swift (OpenStack Swift
38368 (Rackspace Cloud Files, Memset Memstore, OVH)).
38369 --swift-leave-parts-on-error
38371 If true avoid calling abort upload on a failure.
38372 It should be set to true for resuming uploads across different ses‐
38374 sions.
38375 Properties:
38377 • Config: leave_parts_on_error
38379 • Env Var: RCLONE_SWIFT_LEAVE_PARTS_ON_ERROR
38381 • Type: bool
38383 • Default: false
38385 --swift-chunk-size
38387 Above this size files will be chunked into a _segments container.
38388 Above this size files will be chunked into a _segments container. The
38390 default for this is 5 GiB which is its maximum value.
38391 Properties:
38393 • Config: chunk_size
38395 • Env Var: RCLONE_SWIFT_CHUNK_SIZE
38397 • Type: SizeSuffix
38399 • Default: 5Gi
38401 --swift-no-chunk
38403 Don't chunk files during streaming upload.
38404 When doing streaming uploads (e.g. using rcat or mount) setting this
38406 flag will cause the swift backend to not upload chunked files.
38407 This will limit the maximum upload size to 5 GiB. However non chunked
38409 files are easier to deal with and have an MD5SUM.
38410 Rclone will still chunk files bigger than chunk_size when doing normal
38412 copy operations.
38413 Properties:
38415 • Config: no_chunk
38417 • Env Var: RCLONE_SWIFT_NO_CHUNK
38419 • Type: bool
38421 • Default: false
38423 --swift-no-large-objects
38425 Disable support for static and dynamic large objects
38426 Swift cannot transparently store files bigger than 5 GiB. There are
38428 two schemes for doing that, static or dynamic large objects, and the
38429 API does not allow rclone to determine whether a file is a static or
38430 dynamic large object without doing a HEAD on the object. Since these
38431 need to be treated differently, this means rclone has to issue HEAD re‐
38432 quests for objects for example when reading checksums.
38433 When no_large_objects is set, rclone will assume that there are no
38435 static or dynamic large objects stored. This means it can stop doing
38436 the extra HEAD calls which in turn increases performance greatly espe‐
38437 cially when doing a swift to swift transfer with --checksum set.
38438 Setting this option implies no_chunk and also that no files will be up‐
38440 loaded in chunks, so files bigger than 5 GiB will just fail on upload.
38441 If you set this option and there are static or dynamic large objects,
38443 then this will give incorrect hashes for them. Downloads will succeed,
38444 but other operations such as Remove and Copy will fail.
38445 Properties:
38447 • Config: no_large_objects
38449 • Env Var: RCLONE_SWIFT_NO_LARGE_OBJECTS
38451 • Type: bool
38453 • Default: false
38455 --swift-encoding
38457 The encoding for the backend.
38458 See the encoding section in the overview (https://rclone.org/over‐
38460 view/#encoding) for more info.
38461 Properties:
38463 • Config: encoding
38465 • Env Var: RCLONE_SWIFT_ENCODING
38467 • Type: MultiEncoder
38469 • Default: Slash,InvalidUtf8
38471 Limitations
38473 The Swift API doesn't return a correct MD5SUM for segmented files (Dy‐
38474 namic or Static Large Objects) so rclone won't check or use the MD5SUM
38475 for these.
38476 Troubleshooting
38478 Rclone gives Failed to create file system for "remote:": Bad Request
38479 Due to an oddity of the underlying swift library, it gives a "Bad Re‐
38480 quest" error rather than a more sensible error when the authentication
38481 fails for Swift.
38482 So this most likely means your username / password is wrong. You can
38484 investigate further with the --dump-bodies flag.
38485 This may also be caused by specifying the region when you shouldn't
38487 have (e.g. OVH).
38488 Rclone gives Failed to create file system: Response didn't have storage url
38490 and auth token
38491 This is most likely caused by forgetting to specify your tenant when
38492 setting up a swift remote.
38493 OVH Cloud Archive
38495 To use rclone with OVH cloud archive, first use rclone config to set up
38496 a swift backend with OVH, choosing pca as the storage_policy.
38497 Uploading Objects
38499 Uploading objects to OVH cloud archive is no different to object stor‐
38500 age, you just simply run the command you like (move, copy or sync) to
38501 upload the objects. Once uploaded the objects will show in a "Frozen"
38502 state within the OVH control panel.
38503 Retrieving Objects
38505 To retrieve objects use rclone copy as normal. If the objects are in a
38506 frozen state then rclone will ask for them all to be unfrozen and it
38507 will wait at the end of the output with a message like the following:
38508 2019/03/23 13:06:33 NOTICE: Received retry after error - sleeping until
38510 2019-03-23T13:16:33.481657164+01:00 (9m59.99985121s)
38511 Rclone will wait for the time specified then retry the copy.
38513
38515 Paths are specified as remote:path
38516 Paths may be as deep as required, e.g. remote:directory/subdirectory.
38518 Configuration
38520 The initial setup for pCloud involves getting a token from pCloud which
38521 you need to do in your browser. rclone config walks you through it.
38522 Here is an example of how to make a remote called remote. First run:
38524 rclone config
38526 This will guide you through an interactive setup process:
38528 No remotes found, make a new one?
38530 n) New remote
38531 s) Set configuration password
38532 q) Quit config
38533 n/s/q> n
38534 name> remote
38535 Type of storage to configure.
38536 Choose a number from below, or type in your own value
38537 [snip]
38538 XX / Pcloud
38539 \ "pcloud"
38540 [snip]
38541 Storage> pcloud
38542 Pcloud App Client Id - leave blank normally.
38543 client_id>
38544 Pcloud App Client Secret - leave blank normally.
38545 client_secret>
38546 Remote config
38547 Use auto config?
38548 * Say Y if not sure
38549 * Say N if you are working on a remote or headless machine
38550 y) Yes
38551 n) No
38552 y/n> y
38553 If your browser doesn't open automatically go to the following link: http://127.0.0.1:53682/auth
38554 Log in and authorize rclone for access
38555 Waiting for code...
38556 Got code
38557 --------------------
38558 [remote]
38559 client_id =
38560 client_secret =
38561 token = {"access_token":"XXX","token_type":"bearer","expiry":"0001-01-01T00:00:00Z"}
38562 --------------------
38563 y) Yes this is OK
38564 e) Edit this remote
38565 d) Delete this remote
38566 y/e/d> y
38567 See the remote setup docs (https://rclone.org/remote_setup/) for how to
38569 set it up on a machine with no Internet browser available.
38570 Note that rclone runs a webserver on your local machine to collect the
38572 token as returned from pCloud. This only runs from the moment it opens
38573 your browser to the moment you get back the verification code. This is
38574 on http://127.0.0.1:53682/ and this it may require you to unblock it
38575 temporarily if you are running a host firewall.
38576 Once configured you can then use rclone like this,
38578 List directories in top level of your pCloud
38580 rclone lsd remote:
38582 List all the files in your pCloud
38584 rclone ls remote:
38586 To copy a local directory to a pCloud directory called backup
38588 rclone copy /home/source remote:backup
38590 Modified time and hashes
38592 pCloud allows modification times to be set on objects accurate to 1
38593 second. These will be used to detect whether objects need syncing or
38594 not. In order to set a Modification time pCloud requires the object be
38595 re-uploaded.
38596 pCloud supports MD5 and SHA1 hashes in the US region, and SHA1 and
38598 SHA256 hashes in the EU region, so you can use the --checksum flag.
38599 Restricted filename characters
38601 In addition to the default restricted characters set
38602 (https://rclone.org/overview/#restricted-characters) the following
38603 characters are also replaced:
38604 Character Value Replacement
38606 ────────────────────────────────
38607 \ 0x5C ＼
38608 Invalid UTF-8 bytes will also be replaced (https://rclone.org/over‐
38610 view/#invalid-utf8), as they can't be used in JSON strings.
38611 Deleting files
38613 Deleted files will be moved to the trash. Your subscription level will
38614 determine how long items stay in the trash. rclone cleanup can be used
38615 to empty the trash.
38616 Emptying the trash
38618 Due to an API limitation, the rclone cleanup command will only work if
38619 you set your username and password in the advanced options for this
38620 backend. Since we generally want to avoid storing user passwords in
38621 the rclone config file, we advise you to only set this up if you need
38622 the rclone cleanup command to work.
38623 Root folder ID
38625 You can set the root_folder_id for rclone. This is the directory
38626 (identified by its Folder ID) that rclone considers to be the root of
38627 your pCloud drive.
38628 Normally you will leave this blank and rclone will determine the cor‐
38630 rect root to use itself.
38631 However you can set this to restrict rclone to a specific folder hier‐
38633 archy.
38634 In order to do this you will have to find the Folder ID of the directo‐
38636 ry you wish rclone to display. This will be the folder field of the
38637 URL when you open the relevant folder in the pCloud web interface.
38638 So if the folder you want rclone to use has a URL which looks like
38640 https://my.pcloud.com/#page=filemanager&folder=5xxxxxxxx8&tpl=folder‐
38641 grid in the browser, then you use 5xxxxxxxx8 as the root_folder_id in
38642 the config.
38643 Standard options
38645 Here are the Standard options specific to pcloud (Pcloud).
38646 --pcloud-client-id
38648 OAuth Client Id.
38649 Leave blank normally.
38651 Properties:
38653 • Config: client_id
38655 • Env Var: RCLONE_PCLOUD_CLIENT_ID
38657 • Type: string
38659 • Required: false
38661 --pcloud-client-secret
38663 OAuth Client Secret.
38664 Leave blank normally.
38666 Properties:
38668 • Config: client_secret
38670 • Env Var: RCLONE_PCLOUD_CLIENT_SECRET
38672 • Type: string
38674 • Required: false
38676 Advanced options
38678 Here are the Advanced options specific to pcloud (Pcloud).
38679 --pcloud-token
38681 OAuth Access Token as a JSON blob.
38682 Properties:
38684 • Config: token
38686 • Env Var: RCLONE_PCLOUD_TOKEN
38688 • Type: string
38690 • Required: false
38692 --pcloud-auth-url
38694 Auth server URL.
38695 Leave blank to use the provider defaults.
38697 Properties:
38699 • Config: auth_url
38701 • Env Var: RCLONE_PCLOUD_AUTH_URL
38703 • Type: string
38705 • Required: false
38707 --pcloud-token-url
38709 Token server url.
38710 Leave blank to use the provider defaults.
38712 Properties:
38714 • Config: token_url
38716 • Env Var: RCLONE_PCLOUD_TOKEN_URL
38718 • Type: string
38720 • Required: false
38722 --pcloud-encoding
38724 The encoding for the backend.
38725 See the encoding section in the overview (https://rclone.org/over‐
38727 view/#encoding) for more info.
38728 Properties:
38730 • Config: encoding
38732 • Env Var: RCLONE_PCLOUD_ENCODING
38734 • Type: MultiEncoder
38736 • Default: Slash,BackSlash,Del,Ctl,InvalidUtf8,Dot
38738 --pcloud-root-folder-id
38740 Fill in for rclone to use a non root folder as its starting point.
38741 Properties:
38743 • Config: root_folder_id
38745 • Env Var: RCLONE_PCLOUD_ROOT_FOLDER_ID
38747 • Type: string
38749 • Default: "d0"
38751 --pcloud-hostname
38753 Hostname to connect to.
38754 This is normally set when rclone initially does the oauth connection,
38756 however you will need to set it by hand if you are using remote config
38757 with rclone authorize.
38758 Properties:
38760 • Config: hostname
38762 • Env Var: RCLONE_PCLOUD_HOSTNAME
38764 • Type: string
38766 • Default: "api.pcloud.com"
38768 • Examples:
38770 • "api.pcloud.com"
38772 • Original/US region
38774 • "eapi.pcloud.com"
38776 • EU region
38778 --pcloud-username
38780 Your pcloud username.
38781 This is only required when you want to use the cleanup command. Due to
38783 a bug in the pcloud API the required API does not support OAuth authen‐
38784 tication so we have to rely on user password authentication for it.
38785 Properties:
38787 • Config: username
38789 • Env Var: RCLONE_PCLOUD_USERNAME
38791 • Type: string
38793 • Required: false
38795 --pcloud-password
38797 Your pcloud password.
38798 NB Input to this must be obscured - see rclone obscure
38800 (https://rclone.org/commands/rclone_obscure/).
38801 Properties:
38803 • Config: password
38805 • Env Var: RCLONE_PCLOUD_PASSWORD
38807 • Type: string
38809 • Required: false
38811
38813 Paths are specified as remote:path
38814 Paths may be as deep as required, e.g. remote:directory/subdirectory.
38816 Configuration
38818 The initial setup for premiumize.me (https://premiumize.me/) involves
38819 getting a token from premiumize.me which you need to do in your brows‐
38820 er. rclone config walks you through it.
38821 Here is an example of how to make a remote called remote. First run:
38823 rclone config
38825 This will guide you through an interactive setup process:
38827 No remotes found, make a new one?
38829 n) New remote
38830 s) Set configuration password
38831 q) Quit config
38832 n/s/q> n
38833 name> remote
38834 Type of storage to configure.
38835 Enter a string value. Press Enter for the default ("").
38836 Choose a number from below, or type in your own value
38837 [snip]
38838 XX / premiumize.me
38839 \ "premiumizeme"
38840 [snip]
38841 Storage> premiumizeme
38842 ** See help for premiumizeme backend at: https://rclone.org/premiumizeme/ **
38843 Remote config
38845 Use auto config?
38846 * Say Y if not sure
38847 * Say N if you are working on a remote or headless machine
38848 y) Yes
38849 n) No
38850 y/n> y
38851 If your browser doesn't open automatically go to the following link: http://127.0.0.1:53682/auth
38852 Log in and authorize rclone for access
38853 Waiting for code...
38854 Got code
38855 --------------------
38856 [remote]
38857 type = premiumizeme
38858 token = {"access_token":"XXX","token_type":"Bearer","refresh_token":"XXX","expiry":"2029-08-07T18:44:15.548915378+01:00"}
38859 --------------------
38860 y) Yes this is OK
38861 e) Edit this remote
38862 d) Delete this remote
38863 y/e/d>
38864 See the remote setup docs (https://rclone.org/remote_setup/) for how to
38866 set it up on a machine with no Internet browser available.
38867 Note that rclone runs a webserver on your local machine to collect the
38869 token as returned from premiumize.me. This only runs from the moment
38870 it opens your browser to the moment you get back the verification code.
38871 This is on http://127.0.0.1:53682/ and this it may require you to un‐
38872 block it temporarily if you are running a host firewall.
38873 Once configured you can then use rclone like this,
38875 List directories in top level of your premiumize.me
38877 rclone lsd remote:
38879 List all the files in your premiumize.me
38881 rclone ls remote:
38883 To copy a local directory to an premiumize.me directory called backup
38885 rclone copy /home/source remote:backup
38887 Modified time and hashes
38889 premiumize.me does not support modification times or hashes, therefore
38890 syncing will default to --size-only checking. Note that using --update
38891 will work.
38892 Restricted filename characters
38894 In addition to the default restricted characters set
38895 (https://rclone.org/overview/#restricted-characters) the following
38896 characters are also replaced:
38897 Character Value Replacement
38899 ────────────────────────────────
38900 \ 0x5C ＼
38901 " 0x22 ＂
38902 Invalid UTF-8 bytes will also be replaced (https://rclone.org/over‐
38904 view/#invalid-utf8), as they can't be used in JSON strings.
38905 Standard options
38907 Here are the Standard options specific to premiumizeme (premiumize.me).
38908 --premiumizeme-api-key
38910 API Key.
38911 This is not normally used - use oauth instead.
38913 Properties:
38915 • Config: api_key
38917 • Env Var: RCLONE_PREMIUMIZEME_API_KEY
38919 • Type: string
38921 • Required: false
38923 Advanced options
38925 Here are the Advanced options specific to premiumizeme (premiumize.me).
38926 --premiumizeme-encoding
38928 The encoding for the backend.
38929 See the encoding section in the overview (https://rclone.org/over‐
38931 view/#encoding) for more info.
38932 Properties:
38934 • Config: encoding
38936 • Env Var: RCLONE_PREMIUMIZEME_ENCODING
38938 • Type: MultiEncoder
38940 • Default: Slash,DoubleQuote,BackSlash,Del,Ctl,InvalidUtf8,Dot
38942 Limitations
38944 Note that premiumize.me is case insensitive so you can't have a file
38945 called "Hello.doc" and one called "hello.doc".
38946 premiumize.me file names can't have the \ or " characters in. rclone
38948 maps these to and from an identical looking unicode equivalents ＼ and
38949 ＂
38950 premiumize.me only supports filenames up to 255 characters in length.
38952
38954 Paths are specified as remote:path
38955 put.io paths may be as deep as required, e.g. remote:directory/subdi‐
38957 rectory.
38958 Configuration
38960 The initial setup for put.io involves getting a token from put.io which
38961 you need to do in your browser. rclone config walks you through it.
38962 Here is an example of how to make a remote called remote. First run:
38964 rclone config
38966 This will guide you through an interactive setup process:
38968 No remotes found, make a new one?
38970 n) New remote
38971 s) Set configuration password
38972 q) Quit config
38973 n/s/q> n
38974 name> putio
38975 Type of storage to configure.
38976 Enter a string value. Press Enter for the default ("").
38977 Choose a number from below, or type in your own value
38978 [snip]
38979 XX / Put.io
38980 \ "putio"
38981 [snip]
38982 Storage> putio
38983 ** See help for putio backend at: https://rclone.org/putio/ **
38984 Remote config
38986 Use auto config?
38987 * Say Y if not sure
38988 * Say N if you are working on a remote or headless machine
38989 y) Yes
38990 n) No
38991 y/n> y
38992 If your browser doesn't open automatically go to the following link: http://127.0.0.1:53682/auth
38993 Log in and authorize rclone for access
38994 Waiting for code...
38995 Got code
38996 --------------------
38997 [putio]
38998 type = putio
38999 token = {"access_token":"XXXXXXXX","expiry":"0001-01-01T00:00:00Z"}
39000 --------------------
39001 y) Yes this is OK
39002 e) Edit this remote
39003 d) Delete this remote
39004 y/e/d> y
39005 Current remotes:
39006 Name Type
39008 ==== ====
39009 putio putio
39010 e) Edit existing remote
39012 n) New remote
39013 d) Delete remote
39014 r) Rename remote
39015 c) Copy remote
39016 s) Set configuration password
39017 q) Quit config
39018 e/n/d/r/c/s/q> q
39019 Note that rclone runs a webserver on your local machine to collect the
39021 token as returned from Google if you use auto config mode. This only
39022 runs from the moment it opens your browser to the moment you get back
39023 the verification code. This is on http://127.0.0.1:53682/ and this it
39024 may require you to unblock it temporarily if you are running a host
39025 firewall, or use manual mode.
39026 You can then use it like this,
39028 List directories in top level of your put.io
39030 rclone lsd remote:
39032 List all the files in your put.io
39034 rclone ls remote:
39036 To copy a local directory to a put.io directory called backup
39038 rclone copy /home/source remote:backup
39040 Restricted filename characters
39042 In addition to the default restricted characters set
39043 (https://rclone.org/overview/#restricted-characters) the following
39044 characters are also replaced:
39045 Character Value Replacement
39047 ────────────────────────────────
39048 \ 0x5C ＼
39049 Invalid UTF-8 bytes will also be replaced (https://rclone.org/over‐
39051 view/#invalid-utf8), as they can't be used in JSON strings.
39052 Advanced options
39054 Here are the Advanced options specific to putio (Put.io).
39055 --putio-encoding
39057 The encoding for the backend.
39058 See the encoding section in the overview (https://rclone.org/over‐
39060 view/#encoding) for more info.
39061 Properties:
39063 • Config: encoding
39065 • Env Var: RCLONE_PUTIO_ENCODING
39067 • Type: MultiEncoder
39069 • Default: Slash,BackSlash,Del,Ctl,InvalidUtf8,Dot
39071 Limitations
39073 put.io has rate limiting. When you hit a limit, rclone automatically
39074 retries after waiting the amount of time requested by the server.
39075 If you want to avoid ever hitting these limits, you may use the --tp‐
39077 slimit flag with a low number. Note that the imposed limits may be
39078 different for different operations, and may change over time.
39079
39081 This is a backend for the Seafile (https://www.seafile.com/) storage
39082 service: - It works with both the free community edition or the profes‐
39083 sional edition. - Seafile versions 6.x and 7.x are all supported. -
39084 Encrypted libraries are also supported. - It supports 2FA enabled
39085 users
39086 Configuration
39088 There are two distinct modes you can setup your remote: - you point
39089 your remote to the root of the server, meaning you don't specify a li‐
39090 brary during the configuration: Paths are specified as remote:library.
39091 You may put subdirectories in too, e.g. remote:library/path/to/dir. -
39092 you point your remote to a specific library during the configuration:
39093 Paths are specified as remote:path/to/dir. This is the recommended
39094 mode when using encrypted libraries. (This mode is possibly slightly
39095 faster than the root mode)
39096 Configuration in root mode
39098 Here is an example of making a seafile configuration for a user with no
39099 two-factor authentication. First run
39100 rclone config
39102 This will guide you through an interactive setup process. To authenti‐
39104 cate you will need the URL of your server, your email (or username) and
39105 your password.
39106 No remotes found, make a new one?
39108 n) New remote
39109 s) Set configuration password
39110 q) Quit config
39111 n/s/q> n
39112 name> seafile
39113 Type of storage to configure.
39114 Enter a string value. Press Enter for the default ("").
39115 Choose a number from below, or type in your own value
39116 [snip]
39117 XX / Seafile
39118 \ "seafile"
39119 [snip]
39120 Storage> seafile
39121 ** See help for seafile backend at: https://rclone.org/seafile/ **
39122 URL of seafile host to connect to
39124 Enter a string value. Press Enter for the default ("").
39125 Choose a number from below, or type in your own value
39126 1 / Connect to cloud.seafile.com
39127 \ "https://cloud.seafile.com/"
39128 url> http://my.seafile.server/
39129 User name (usually email address)
39130 Enter a string value. Press Enter for the default ("").
39131 user> me@example.com
39132 Password
39133 y) Yes type in my own password
39134 g) Generate random password
39135 n) No leave this optional password blank (default)
39136 y/g> y
39137 Enter the password:
39138 password:
39139 Confirm the password:
39140 password:
39141 Two-factor authentication ('true' if the account has 2FA enabled)
39142 Enter a boolean value (true or false). Press Enter for the default ("false").
39143 2fa> false
39144 Name of the library. Leave blank to access all non-encrypted libraries.
39145 Enter a string value. Press Enter for the default ("").
39146 library>
39147 Library password (for encrypted libraries only). Leave blank if you pass it through the command line.
39148 y) Yes type in my own password
39149 g) Generate random password
39150 n) No leave this optional password blank (default)
39151 y/g/n> n
39152 Edit advanced config? (y/n)
39153 y) Yes
39154 n) No (default)
39155 y/n> n
39156 Remote config
39157 Two-factor authentication is not enabled on this account.
39158 --------------------
39159 [seafile]
39160 type = seafile
39161 url = http://my.seafile.server/
39162 user = me@example.com
39163 pass = *** ENCRYPTED ***
39164 2fa = false
39165 --------------------
39166 y) Yes this is OK (default)
39167 e) Edit this remote
39168 d) Delete this remote
39169 y/e/d> y
39170 This remote is called seafile. It's pointing to the root of your
39172 seafile server and can now be used like this:
39173 See all libraries
39175 rclone lsd seafile:
39177 Create a new library
39179 rclone mkdir seafile:library
39181 List the contents of a library
39183 rclone ls seafile:library
39185 Sync /home/local/directory to the remote library, deleting any excess
39187 files in the library.
39188 rclone sync -i /home/local/directory seafile:library
39190 Configuration in library mode
39192 Here's an example of a configuration in library mode with a user that
39193 has the two-factor authentication enabled. Your 2FA code will be asked
39194 at the end of the configuration, and will attempt to authenticate you:
39195 No remotes found, make a new one?
39197 n) New remote
39198 s) Set configuration password
39199 q) Quit config
39200 n/s/q> n
39201 name> seafile
39202 Type of storage to configure.
39203 Enter a string value. Press Enter for the default ("").
39204 Choose a number from below, or type in your own value
39205 [snip]
39206 XX / Seafile
39207 \ "seafile"
39208 [snip]
39209 Storage> seafile
39210 ** See help for seafile backend at: https://rclone.org/seafile/ **
39211 URL of seafile host to connect to
39213 Enter a string value. Press Enter for the default ("").
39214 Choose a number from below, or type in your own value
39215 1 / Connect to cloud.seafile.com
39216 \ "https://cloud.seafile.com/"
39217 url> http://my.seafile.server/
39218 User name (usually email address)
39219 Enter a string value. Press Enter for the default ("").
39220 user> me@example.com
39221 Password
39222 y) Yes type in my own password
39223 g) Generate random password
39224 n) No leave this optional password blank (default)
39225 y/g> y
39226 Enter the password:
39227 password:
39228 Confirm the password:
39229 password:
39230 Two-factor authentication ('true' if the account has 2FA enabled)
39231 Enter a boolean value (true or false). Press Enter for the default ("false").
39232 2fa> true
39233 Name of the library. Leave blank to access all non-encrypted libraries.
39234 Enter a string value. Press Enter for the default ("").
39235 library> My Library
39236 Library password (for encrypted libraries only). Leave blank if you pass it through the command line.
39237 y) Yes type in my own password
39238 g) Generate random password
39239 n) No leave this optional password blank (default)
39240 y/g/n> n
39241 Edit advanced config? (y/n)
39242 y) Yes
39243 n) No (default)
39244 y/n> n
39245 Remote config
39246 Two-factor authentication: please enter your 2FA code
39247 2fa code> 123456
39248 Authenticating...
39249 Success!
39250 --------------------
39251 [seafile]
39252 type = seafile
39253 url = http://my.seafile.server/
39254 user = me@example.com
39255 pass =
39256 2fa = true
39257 library = My Library
39258 --------------------
39259 y) Yes this is OK (default)
39260 e) Edit this remote
39261 d) Delete this remote
39262 y/e/d> y
39263 You'll notice your password is blank in the configuration. It's be‐
39265 cause we only need the password to authenticate you once.
39266 You specified My Library during the configuration. The root of the re‐
39268 mote is pointing at the root of the library My Library:
39269 See all files in the library:
39271 rclone lsd seafile:
39273 Create a new directory inside the library
39275 rclone mkdir seafile:directory
39277 List the contents of a directory
39279 rclone ls seafile:directory
39281 Sync /home/local/directory to the remote library, deleting any excess
39283 files in the library.
39284 rclone sync -i /home/local/directory seafile:
39286 --fast-list
39288 Seafile version 7+ supports --fast-list which allows you to use fewer
39289 transactions in exchange for more memory. See the rclone docs
39290 (https://rclone.org/docs/#fast-list) for more details. Please note
39291 this is not supported on seafile server version 6.x
39292 Restricted filename characters
39294 In addition to the default restricted characters set
39295 (https://rclone.org/overview/#restricted-characters) the following
39296 characters are also replaced:
39297 Character Value Replacement
39299 ────────────────────────────────
39300 / 0x2F ／
39301 " 0x22 ＂
39302 \ 0x5C ＼
39303 Invalid UTF-8 bytes will also be replaced (https://rclone.org/over‐
39305 view/#invalid-utf8), as they can't be used in JSON strings.
39306 Seafile and rclone link
39308 Rclone supports generating share links for non-encrypted libraries on‐
39309 ly. They can either be for a file or a directory:
39310 rclone link seafile:seafile-tutorial.doc
39312 http://my.seafile.server/f/fdcd8a2f93f84b8b90f4/
39313 or if run on a directory you will get:
39315 rclone link seafile:dir
39317 http://my.seafile.server/d/9ea2455f6f55478bbb0d/
39318 Please note a share link is unique for each file or directory. If you
39320 run a link command on a file/dir that has already been shared, you will
39321 get the exact same link.
39322 Compatibility
39324 It has been actively tested using the seafile docker image
39325 (https://github.com/haiwen/seafile-docker) of these versions: - 6.3.4
39326 community edition - 7.0.5 community edition - 7.1.3 community edition
39327 Versions below 6.0 are not supported. Versions between 6.0 and 6.3
39329 haven't been tested and might not work properly.
39330 Standard options
39332 Here are the Standard options specific to seafile (seafile).
39333 --seafile-url
39335 URL of seafile host to connect to.
39336 Properties:
39338 • Config: url
39340 • Env Var: RCLONE_SEAFILE_URL
39342 • Type: string
39344 • Required: true
39346 • Examples:
39348 • "https://cloud.seafile.com/"
39350 • Connect to cloud.seafile.com.
39352 --seafile-user
39354 User name (usually email address).
39355 Properties:
39357 • Config: user
39359 • Env Var: RCLONE_SEAFILE_USER
39361 • Type: string
39363 • Required: true
39365 --seafile-pass
39367 Password.
39368 NB Input to this must be obscured - see rclone obscure
39370 (https://rclone.org/commands/rclone_obscure/).
39371 Properties:
39373 • Config: pass
39375 • Env Var: RCLONE_SEAFILE_PASS
39377 • Type: string
39379 • Required: false
39381 --seafile-2fa
39383 Two-factor authentication ('true' if the account has 2FA enabled).
39384 Properties:
39386 • Config: 2fa
39388 • Env Var: RCLONE_SEAFILE_2FA
39390 • Type: bool
39392 • Default: false
39394 --seafile-library
39396 Name of the library.
39397 Leave blank to access all non-encrypted libraries.
39399 Properties:
39401 • Config: library
39403 • Env Var: RCLONE_SEAFILE_LIBRARY
39405 • Type: string
39407 • Required: false
39409 --seafile-library-key
39411 Library password (for encrypted libraries only).
39412 Leave blank if you pass it through the command line.
39414 NB Input to this must be obscured - see rclone obscure
39416 (https://rclone.org/commands/rclone_obscure/).
39417 Properties:
39419 • Config: library_key
39421 • Env Var: RCLONE_SEAFILE_LIBRARY_KEY
39423 • Type: string
39425 • Required: false
39427 --seafile-auth-token
39429 Authentication token.
39430 Properties:
39432 • Config: auth_token
39434 • Env Var: RCLONE_SEAFILE_AUTH_TOKEN
39436 • Type: string
39438 • Required: false
39440 Advanced options
39442 Here are the Advanced options specific to seafile (seafile).
39443 --seafile-create-library
39445 Should rclone create a library if it doesn't exist.
39446 Properties:
39448 • Config: create_library
39450 • Env Var: RCLONE_SEAFILE_CREATE_LIBRARY
39452 • Type: bool
39454 • Default: false
39456 --seafile-encoding
39458 The encoding for the backend.
39459 See the encoding section in the overview (https://rclone.org/over‐
39461 view/#encoding) for more info.
39462 Properties:
39464 • Config: encoding
39466 • Env Var: RCLONE_SEAFILE_ENCODING
39468 • Type: MultiEncoder
39470 • Default: Slash,DoubleQuote,BackSlash,Ctl,InvalidUtf8
39472
39474 SFTP is the Secure (or SSH) File Transfer Protocol
39475 (https://en.wikipedia.org/wiki/SSH_File_Transfer_Protocol).
39476 The SFTP backend can be used with a number of different providers:
39478 • Hetzner Storage Box
39480 • rsync.net
39482 SFTP runs over SSH v2 and is installed as standard with most modern SSH
39484 installations.
39485 Paths are specified as remote:path. If the path does not begin with a
39487 / it is relative to the home directory of the user. An empty path re‐
39488 mote: refers to the user's home directory. For example, rclone lsd re‐
39489 mote: would list the home directory of the user configured in the
39490 rclone remote config (i.e /home/sftpuser). However, rclone lsd re‐
39491 mote:/ would list the root directory for remote machine (i.e. /)
39492 Note that some SFTP servers will need the leading / - Synology is a
39494 good example of this. rsync.net and Hetzner, on the other hand, re‐
39495 quires users to OMIT the leading /.
39496 Note that by default rclone will try to execute shell commands on the
39498 server, see shell access considerations.
39499 Configuration
39501 Here is an example of making an SFTP configuration. First run
39502 rclone config
39504 This will guide you through an interactive setup process.
39506 No remotes found, make a new one?
39508 n) New remote
39509 s) Set configuration password
39510 q) Quit config
39511 n/s/q> n
39512 name> remote
39513 Type of storage to configure.
39514 Choose a number from below, or type in your own value
39515 [snip]
39516 XX / SSH/SFTP
39517 \ "sftp"
39518 [snip]
39519 Storage> sftp
39520 SSH host to connect to
39521 Choose a number from below, or type in your own value
39522 1 / Connect to example.com
39523 \ "example.com"
39524 host> example.com
39525 SSH username
39526 Enter a string value. Press Enter for the default ("$USER").
39527 user> sftpuser
39528 SSH port number
39529 Enter a signed integer. Press Enter for the default (22).
39530 port>
39531 SSH password, leave blank to use ssh-agent.
39532 y) Yes type in my own password
39533 g) Generate random password
39534 n) No leave this optional password blank
39535 y/g/n> n
39536 Path to unencrypted PEM-encoded private key file, leave blank to use ssh-agent.
39537 key_file>
39538 Remote config
39539 --------------------
39540 [remote]
39541 host = example.com
39542 user = sftpuser
39543 port =
39544 pass =
39545 key_file =
39546 --------------------
39547 y) Yes this is OK
39548 e) Edit this remote
39549 d) Delete this remote
39550 y/e/d> y
39551 This remote is called remote and can now be used like this:
39553 See all directories in the home directory
39555 rclone lsd remote:
39557 See all directories in the root directory
39559 rclone lsd remote:/
39561 Make a new directory
39563 rclone mkdir remote:path/to/directory
39565 List the contents of a directory
39567 rclone ls remote:path/to/directory
39569 Sync /home/local/directory to the remote directory, deleting any excess
39571 files in the directory.
39572 rclone sync -i /home/local/directory remote:directory
39574 Mount the remote path /srv/www-data/ to the local path /mnt/www-data
39576 rclone mount remote:/srv/www-data/ /mnt/www-data
39578 SSH Authentication
39580 The SFTP remote supports three authentication methods:
39581 • Password
39583 • Key file, including certificate signed keys
39585 • ssh-agent
39587 Key files should be PEM-encoded private key files. For instance
39589 /home/$USER/.ssh/id_rsa. Only unencrypted OpenSSH or PEM encrypted
39590 files are supported.
39591 The key file can be specified in either an external file (key_file) or
39593 contained within the rclone config file (key_pem). If using key_pem in
39594 the config file, the entry should be on a single line with new line (''
39595 or '') separating lines. i.e.
39596 key_pem = -----BEGIN RSA PRIVATE KEY-----\nMaMbaIXtE\n0gAMbMbaSsd\nMbaass\n-----END RSA PRIVATE KEY-----
39598 This will generate it correctly for key_pem for use in the config:
39600 awk '{printf "%s\\n", $0}' < ~/.ssh/id_rsa
39602 If you don't specify pass, key_file, or key_pem or ask_password then
39604 rclone will attempt to contact an ssh-agent. You can also specify
39605 key_use_agent to force the usage of an ssh-agent. In this case
39606 key_file or key_pem can also be specified to force the usage of a spe‐
39607 cific key in the ssh-agent.
39608 Using an ssh-agent is the only way to load encrypted OpenSSH keys at
39610 the moment.
39611 If you set the ask_password option, rclone will prompt for a password
39613 when needed and no password has been configured.
39614 Certificate-signed keys
39616 With traditional key-based authentication, you configure your private
39617 key only, and the public key built into it will be used during the au‐
39618 thentication process.
39619 If you have a certificate you may use it to sign your public key, cre‐
39621 ating a separate SSH user certificate that should be used instead of
39622 the plain public key extracted from the private key. Then you must
39623 provide the path to the user certificate public key file in pub‐
39624 key_file.
39625 Note: This is not the traditional public key paired with your private
39627 key, typically saved as /home/$USER/.ssh/id_rsa.pub. Setting this path
39628 in pubkey_file will not work.
39629 Example:
39631 [remote]
39633 type = sftp
39634 host = example.com
39635 user = sftpuser
39636 key_file = ~/id_rsa
39637 pubkey_file = ~/id_rsa-cert.pub
39638 If you concatenate a cert with a private key then you can specify the
39640 merged file in both places.
39641 Note: the cert must come first in the file. e.g.
39643 cat id_rsa-cert.pub id_rsa > merged_key
39645 Host key validation
39647 By default rclone will not check the server's host key for validation.
39648 This can allow an attacker to replace a server with their own and if
39649 you use password authentication then this can lead to that password be‐
39650 ing exposed.
39651 Host key matching, using standard known_hosts files can be turned on by
39653 enabling the known_hosts_file option. This can point to the file main‐
39654 tained by OpenSSH or can point to a unique file.
39655 e.g. using the OpenSSH known_hosts file:
39657 [remote]
39659 type = sftp
39660 host = example.com
39661 user = sftpuser
39662 pass =
39663 known_hosts_file = ~/.ssh/known_hosts
39664 Alternatively you can create your own known hosts file like this:
39666 ssh-keyscan -t dsa,rsa,ecdsa,ed25519 example.com >> known_hosts
39668 There are some limitations:
39670 • rclone will not manage this file for you. If the key is missing or
39672 wrong then the connection will be refused.
39673 • If the server is set up for a certificate host key then the entry in
39675 the known_hosts file must be the @cert-authority entry for the CA
39676 If the host key provided by the server does not match the one in the
39678 file (or is missing) then the connection will be aborted and an error
39679 returned such as
39680 NewFs: couldn't connect SSH: ssh: handshake failed: knownhosts: key mismatch
39682 or
39684 NewFs: couldn't connect SSH: ssh: handshake failed: knownhosts: key is unknown
39686 If you see an error such as
39688 NewFs: couldn't connect SSH: ssh: handshake failed: ssh: no authorities for hostname: example.com:22
39690 then it is likely the server has presented a CA signed host certificate
39692 and you will need to add the appropriate @cert-authority entry.
39693 The known_hosts_file setting can be set during rclone config as an ad‐
39695 vanced option.
39696 ssh-agent on macOS
39698 Note that there seem to be various problems with using an ssh-agent on
39699 macOS due to recent changes in the OS. The most effective work-around
39700 seems to be to start an ssh-agent in each session, e.g.
39701 eval `ssh-agent -s` && ssh-add -A
39703 And then at the end of the session
39705 eval `ssh-agent -k`
39707 These commands can be used in scripts of course.
39709 Shell access
39711 Some functionality of the SFTP backend relies on remote shell access,
39712 and the possibility to execute commands. This includes checksum, and
39713 in some cases also about. The shell commands that must be executed may
39714 be different on different type of shells, and also quoting/escaping of
39715 file path arguments containing special characters may be different.
39716 Rclone therefore needs to know what type of shell it is, and if shell
39717 access is available at all.
39718 Most servers run on some version of Unix, and then a basic Unix shell
39720 can be assumed, without further distinction. Windows 10, Server 2019,
39721 and later can also run a SSH server, which is a port of OpenSSH (see
39722 official installation guide (https://docs.microsoft.com/en-us/windows-
39723 server/administration/openssh/openssh_install_firstuse)). On a Windows
39724 server the shell handling is different: Although it can also be set up
39725 to use a Unix type shell, e.g. Cygwin bash, the default is to use Win‐
39726 dows Command Prompt (cmd.exe), and PowerShell is a recommended alterna‐
39727 tive. All of these have behave differently, which rclone must handle.
39728 Rclone tries to auto-detect what type of shell is used on the server,
39730 first time you access the SFTP remote. If a remote shell session is
39731 successfully created, it will look for indications that it is CMD or
39732 PowerShell, with fall-back to Unix if not something else is detected.
39733 If unable to even create a remote shell session, then shell command ex‐
39734 ecution will be disabled entirely. The result is stored in the SFTP
39735 remote configuration, in option shell_type, so that the auto-detection
39736 only have to be performed once. If you manually set a value for this
39737 option before first run, the auto-detection will be skipped, and if you
39738 set a different value later this will override any existing. Value
39739 none can be set to avoid any attempts at executing shell commands, e.g.
39740 if this is not allowed on the server.
39741 When the server is rclone serve sftp (https://rclone.org/com‐
39743 mands/rclone_serve_sftp/), the rclone SFTP remote will detect this as a
39744 Unix type shell - even if it is running on Windows. This server does
39745 not actually have a shell, but it accepts input commands matching the
39746 specific ones that the SFTP backend relies on for Unix shells, e.g.
39747 md5sum and df. Also it handles the string escape rules used for Unix
39748 shell. Treating it as a Unix type shell from a SFTP remote will there‐
39749 fore always be correct, and support all features.
39750 Shell access considerations
39752 The shell type auto-detection logic, described above, means that by de‐
39753 fault rclone will try to run a shell command the first time a new sftp
39754 remote is accessed. If you configure a sftp remote without a config
39755 file, e.g. an on the fly (https://rclone.org/docs/#backend-path-to-
39756 dir%5D) remote, rclone will have nowhere to store the result, and it
39757 will re-run the command on every access. To avoid this you should ex‐
39758 plicitly set the shell_type option to the correct value, or to none if
39759 you want to prevent rclone from executing any remote shell commands.
39760 It is also important to note that, since the shell type decides how
39762 quoting and escaping of file paths used as command-line arguments are
39763 performed, configuring the wrong shell type may leave you exposed to
39764 command injection exploits. Make sure to confirm the auto-detected
39765 shell type, or explicitly set the shell type you know is correct, or
39766 disable shell access until you know.
39767 Checksum
39769 SFTP does not natively support checksums (file hash), but rclone is
39770 able to use checksumming if the same login has shell access, and can
39771 execute remote commands. If there is a command that can calculate com‐
39772 patible checksums on the remote system, Rclone can then be configured
39773 to execute this whenever a checksum is needed, and read back the re‐
39774 sults. Currently MD5 and SHA-1 are supported.
39775 Normally this requires an external utility being available on the serv‐
39777 er. By default rclone will try commands md5sum, md5 and rclone md5sum
39778 for MD5 checksums, and the first one found usable will be picked. Same
39779 with sha1sum, sha1 and rclone sha1sum commands for SHA-1 checksums.
39780 These utilities normally need to be in the remote's PATH to be found.
39781 In some cases the shell itself is capable of calculating checksums.
39783 PowerShell is an example of such a shell. If rclone detects that the
39784 remote shell is PowerShell, which means it most probably is a Windows
39785 OpenSSH server, rclone will use a predefined script block to produce
39786 the checksums when no external checksum commands are found (see shell
39787 access). This assumes PowerShell version 4.0 or newer.
39788 The options md5sum_command and sha1_command can be used to customize
39790 the command to be executed for calculation of checksums. You can for
39791 example set a specific path to where md5sum and sha1sum executables are
39792 located, or use them to specify some other tools that print checksums
39793 in compatible format. The value can include command-line arguments, or
39794 even shell script blocks as with PowerShell. Rclone has subcommands
39795 md5sum (https://rclone.org/commands/rclone_md5sum/) and sha1sum
39796 (https://rclone.org/commands/rclone_sha1sum/) that use compatible for‐
39797 mat, which means if you have an rclone executable on the server it can
39798 be used. As mentioned above, they will be automatically picked up if
39799 found in PATH, but if not you can set something like /path/to/rclone
39800 md5sum as the value of option md5sum_command to make sure a specific
39801 executable is used.
39802 Remote checksumming is recommended and enabled by default. First time
39804 rclone is using a SFTP remote, if options md5sum_command or sha1_com‐
39805 mand are not set, it will check if any of the default commands for each
39806 of them, as described above, can be used. The result will be saved in
39807 the remote configuration, so next time it will use the same. Value
39808 none will be set if none of the default commands could be used for a
39809 specific algorithm, and this algorithm will not be supported by the re‐
39810 mote.
39811 Disabling the checksumming may be required if you are connecting to
39813 SFTP servers which are not under your control, and to which the execu‐
39814 tion of remote shell commands is prohibited. Set the configuration op‐
39815 tion disable_hashcheck to true to disable checksumming entirely, or set
39816 shell_type to none to disable all functionality based on remote shell
39817 command execution.
39818 Modified time
39820 Modified times are stored on the server to 1 second precision.
39821 Modified times are used in syncing and are fully supported.
39823 Some SFTP servers disable setting/modifying the file modification time
39825 after upload (for example, certain configurations of ProFTPd with
39826 mod_sftp). If you are using one of these servers, you can set the op‐
39827 tion set_modtime = false in your RClone backend configuration to dis‐
39828 able this behaviour.
39829 About command
39831 The about command returns the total space, free space, and used space
39832 on the remote for the disk of the specified path on the remote or, if
39833 not set, the disk of the root on the remote.
39834 SFTP usually supports the about (https://rclone.org/com‐
39836 mands/rclone_about/) command, but it depends on the server. If the
39837 server implements the vendor-specific VFS statistics extension, which
39838 is normally the case with OpenSSH instances, it will be used. If not,
39839 but the same login has access to a Unix shell, where the df command is
39840 available (e.g. in the remote's PATH), then this will be used instead.
39841 If the server shell is PowerShell, probably with a Windows OpenSSH
39842 server, rclone will use a built-in shell command (see shell access).
39843 If none of the above is applicable, about will fail.
39844 Standard options
39846 Here are the Standard options specific to sftp (SSH/SFTP).
39847 --sftp-host
39849 SSH host to connect to.
39850 E.g. "example.com".
39852 Properties:
39854 • Config: host
39856 • Env Var: RCLONE_SFTP_HOST
39858 • Type: string
39860 • Required: true
39862 --sftp-user
39864 SSH username.
39865 Properties:
39867 • Config: user
39869 • Env Var: RCLONE_SFTP_USER
39871 • Type: string
39873 • Default: "$USER"
39875 --sftp-port
39877 SSH port number.
39878 Properties:
39880 • Config: port
39882 • Env Var: RCLONE_SFTP_PORT
39884 • Type: int
39886 • Default: 22
39888 --sftp-pass
39890 SSH password, leave blank to use ssh-agent.
39891 NB Input to this must be obscured - see rclone obscure
39893 (https://rclone.org/commands/rclone_obscure/).
39894 Properties:
39896 • Config: pass
39898 • Env Var: RCLONE_SFTP_PASS
39900 • Type: string
39902 • Required: false
39904 --sftp-key-pem
39906 Raw PEM-encoded private key.
39907 If specified, will override key_file parameter.
39909 Properties:
39911 • Config: key_pem
39913 • Env Var: RCLONE_SFTP_KEY_PEM
39915 • Type: string
39917 • Required: false
39919 --sftp-key-file
39921 Path to PEM-encoded private key file.
39922 Leave blank or set key-use-agent to use ssh-agent.
39924 Leading ~ will be expanded in the file name as will environment vari‐
39926 ables such as ${RCLONE_CONFIG_DIR}.
39927 Properties:
39929 • Config: key_file
39931 • Env Var: RCLONE_SFTP_KEY_FILE
39933 • Type: string
39935 • Required: false
39937 --sftp-key-file-pass
39939 The passphrase to decrypt the PEM-encoded private key file.
39940 Only PEM encrypted key files (old OpenSSH format) are supported. En‐
39942 crypted keys in the new OpenSSH format can't be used.
39943 NB Input to this must be obscured - see rclone obscure
39945 (https://rclone.org/commands/rclone_obscure/).
39946 Properties:
39948 • Config: key_file_pass
39950 • Env Var: RCLONE_SFTP_KEY_FILE_PASS
39952 • Type: string
39954 • Required: false
39956 --sftp-pubkey-file
39958 Optional path to public key file.
39959 Set this if you have a signed certificate you want to use for authenti‐
39961 cation.
39962 Leading ~ will be expanded in the file name as will environment vari‐
39964 ables such as ${RCLONE_CONFIG_DIR}.
39965 Properties:
39967 • Config: pubkey_file
39969 • Env Var: RCLONE_SFTP_PUBKEY_FILE
39971 • Type: string
39973 • Required: false
39975 --sftp-key-use-agent
39977 When set forces the usage of the ssh-agent.
39978 When key-file is also set, the ".pub" file of the specified key-file is
39980 read and only the associated key is requested from the ssh-agent. This
39981 allows to avoid Too many authentication failures for *username* errors
39982 when the ssh-agent contains many keys.
39983 Properties:
39985 • Config: key_use_agent
39987 • Env Var: RCLONE_SFTP_KEY_USE_AGENT
39989 • Type: bool
39991 • Default: false
39993 --sftp-use-insecure-cipher
39995 Enable the use of insecure ciphers and key exchange methods.
39996 This enables the use of the following insecure ciphers and key exchange
39998 methods:
39999 • aes128-cbc
40001 • aes192-cbc
40003 • aes256-cbc
40005 • 3des-cbc
40007 • diffie-hellman-group-exchange-sha256
40009 • diffie-hellman-group-exchange-sha1
40011 Those algorithms are insecure and may allow plaintext data to be recov‐
40013 ered by an attacker.
40014 Properties:
40016 • Config: use_insecure_cipher
40018 • Env Var: RCLONE_SFTP_USE_INSECURE_CIPHER
40020 • Type: bool
40022 • Default: false
40024 • Examples:
40026 • "false"
40028 • Use default Cipher list.
40030 • "true"
40032 • Enables the use of the aes128-cbc cipher and diffie-hellman-
40034 group-exchange-sha256, diffie-hellman-group-exchange-sha1 key ex‐
40035 change.
40036 --sftp-disable-hashcheck
40038 Disable the execution of SSH commands to determine if remote file hash‐
40039 ing is available.
40040 Leave blank or set to false to enable hashing (recommended), set to
40042 true to disable hashing.
40043 Properties:
40045 • Config: disable_hashcheck
40047 • Env Var: RCLONE_SFTP_DISABLE_HASHCHECK
40049 • Type: bool
40051 • Default: false
40053 Advanced options
40055 Here are the Advanced options specific to sftp (SSH/SFTP).
40056 --sftp-known-hosts-file
40058 Optional path to known_hosts file.
40059 Set this value to enable server host key validation.
40061 Leading ~ will be expanded in the file name as will environment vari‐
40063 ables such as ${RCLONE_CONFIG_DIR}.
40064 Properties:
40066 • Config: known_hosts_file
40068 • Env Var: RCLONE_SFTP_KNOWN_HOSTS_FILE
40070 • Type: string
40072 • Required: false
40074 • Examples:
40076 • "~/.ssh/known_hosts"
40078 • Use OpenSSH's known_hosts file.
40080 --sftp-ask-password
40082 Allow asking for SFTP password when needed.
40083 If this is set and no password is supplied then rclone will: - ask for
40085 a password - not contact the ssh agent
40086 Properties:
40088 • Config: ask_password
40090 • Env Var: RCLONE_SFTP_ASK_PASSWORD
40092 • Type: bool
40094 • Default: false
40096 --sftp-path-override
40098 Override path used by SSH shell commands.
40099 This allows checksum calculation when SFTP and SSH paths are different.
40101 This issue affects among others Synology NAS boxes.
40102 E.g. if shared folders can be found in directories representing vol‐
40104 umes:
40105 rclone sync /home/local/directory remote:/directory --sftp-path-override /volume2/directory
40107 E.g. if home directory can be found in a shared folder called "home":
40109 rclone sync /home/local/directory remote:/home/directory --sftp-path-override /volume1/homes/USER/directory
40111 Properties:
40113 • Config: path_override
40115 • Env Var: RCLONE_SFTP_PATH_OVERRIDE
40117 • Type: string
40119 • Required: false
40121 --sftp-set-modtime
40123 Set the modified time on the remote if set.
40124 Properties:
40126 • Config: set_modtime
40128 • Env Var: RCLONE_SFTP_SET_MODTIME
40130 • Type: bool
40132 • Default: true
40134 --sftp-shell-type
40136 The type of SSH shell on remote server, if any.
40137 Leave blank for autodetect.
40139 Properties:
40141 • Config: shell_type
40143 • Env Var: RCLONE_SFTP_SHELL_TYPE
40145 • Type: string
40147 • Required: false
40149 • Examples:
40151 • "none"
40153 • No shell access
40155 • "unix"
40157 • Unix shell
40159 • "powershell"
40161 • PowerShell
40163 • "cmd"
40165 • Windows Command Prompt
40167 --sftp-md5sum-command
40169 The command used to read md5 hashes.
40170 Leave blank for autodetect.
40172 Properties:
40174 • Config: md5sum_command
40176 • Env Var: RCLONE_SFTP_MD5SUM_COMMAND
40178 • Type: string
40180 • Required: false
40182 --sftp-sha1sum-command
40184 The command used to read sha1 hashes.
40185 Leave blank for autodetect.
40187 Properties:
40189 • Config: sha1sum_command
40191 • Env Var: RCLONE_SFTP_SHA1SUM_COMMAND
40193 • Type: string
40195 • Required: false
40197 --sftp-skip-links
40199 Set to skip any symlinks and any other non regular files.
40200 Properties:
40202 • Config: skip_links
40204 • Env Var: RCLONE_SFTP_SKIP_LINKS
40206 • Type: bool
40208 • Default: false
40210 --sftp-subsystem
40212 Specifies the SSH2 subsystem on the remote host.
40213 Properties:
40215 • Config: subsystem
40217 • Env Var: RCLONE_SFTP_SUBSYSTEM
40219 • Type: string
40221 • Default: "sftp"
40223 --sftp-server-command
40225 Specifies the path or command to run a sftp server on the remote host.
40226 The subsystem option is ignored when server_command is defined.
40228 Properties:
40230 • Config: server_command
40232 • Env Var: RCLONE_SFTP_SERVER_COMMAND
40234 • Type: string
40236 • Required: false
40238 --sftp-use-fstat
40240 If set use fstat instead of stat.
40241 Some servers limit the amount of open files and calling Stat after
40243 opening the file will throw an error from the server. Setting this
40244 flag will call Fstat instead of Stat which is called on an already open
40245 file handle.
40246 It has been found that this helps with IBM Sterling SFTP servers which
40248 have "extractability" level set to 1 which means only 1 file can be
40249 opened at any given time.
40250 Properties:
40252 • Config: use_fstat
40254 • Env Var: RCLONE_SFTP_USE_FSTAT
40256 • Type: bool
40258 • Default: false
40260 --sftp-disable-concurrent-reads
40262 If set don't use concurrent reads.
40263 Normally concurrent reads are safe to use and not using them will de‐
40265 grade performance, so this option is disabled by default.
40266 Some servers limit the amount number of times a file can be downloaded.
40268 Using concurrent reads can trigger this limit, so if you have a server
40269 which returns
40270 Failed to copy: file does not exist
40272 Then you may need to enable this flag.
40274 If concurrent reads are disabled, the use_fstat option is ignored.
40276 Properties:
40278 • Config: disable_concurrent_reads
40280 • Env Var: RCLONE_SFTP_DISABLE_CONCURRENT_READS
40282 • Type: bool
40284 • Default: false
40286 --sftp-disable-concurrent-writes
40288 If set don't use concurrent writes.
40289 Normally rclone uses concurrent writes to upload files. This improves
40291 the performance greatly, especially for distant servers.
40292 This option disables concurrent writes should that be necessary.
40294 Properties:
40296 • Config: disable_concurrent_writes
40298 • Env Var: RCLONE_SFTP_DISABLE_CONCURRENT_WRITES
40300 • Type: bool
40302 • Default: false
40304 --sftp-idle-timeout
40306 Max time before closing idle connections.
40307 If no connections have been returned to the connection pool in the time
40309 given, rclone will empty the connection pool.
40310 Set to 0 to keep connections indefinitely.
40312 Properties:
40314 • Config: idle_timeout
40316 • Env Var: RCLONE_SFTP_IDLE_TIMEOUT
40318 • Type: Duration
40320 • Default: 1m0s
40322 --sftp-chunk-size
40324 Upload and download chunk size.
40325 This controls the maximum size of payload in SFTP protocol packets.
40327 The RFC limits this to 32768 bytes (32k), which is the default. Howev‐
40328 er, a lot of servers support larger sizes, typically limited to a maxi‐
40329 mum total package size of 256k, and setting it larger will increase
40330 transfer speed dramatically on high latency links. This includes
40331 OpenSSH, and, for example, using the value of 255k works well, leaving
40332 plenty of room for overhead while still being within a total packet
40333 size of 256k.
40334 Make sure to test thoroughly before using a value higher than 32k, and
40336 only use it if you always connect to the same server or after suffi‐
40337 ciently broad testing. If you get errors such as "failed to send pack‐
40338 et payload: EOF", lots of "connection lost", or "corrupted on trans‐
40339 fer", when copying a larger file, try lowering the value. The server
40340 run by rclone serve sftp sends packets with standard 32k maximum pay‐
40341 load so you must not set a different chunk_size when downloading files,
40342 but it accepts packets up to the 256k total size, so for uploads the
40343 chunk_size can be set as for the OpenSSH example above.
40344 Properties:
40346 • Config: chunk_size
40348 • Env Var: RCLONE_SFTP_CHUNK_SIZE
40350 • Type: SizeSuffix
40352 • Default: 32Ki
40354 --sftp-concurrency
40356 The maximum number of outstanding requests for one file
40357 This controls the maximum number of outstanding requests for one file.
40359 Increasing it will increase throughput on high latency links at the
40360 cost of using more memory.
40361 Properties:
40363 • Config: concurrency
40365 • Env Var: RCLONE_SFTP_CONCURRENCY
40367 • Type: int
40369 • Default: 64
40371 --sftp-set-env
40373 Environment variables to pass to sftp and commands
40374 Set environment variables in the form:
40376 VAR=value
40378 to be passed to the sftp client and to any commands run (eg md5sum).
40380 Pass multiple variables space separated, eg
40382 VAR1=value VAR2=value
40384 and pass variables with spaces in in quotes, eg
40386 "VAR3=value with space" "VAR4=value with space" VAR5=nospacehere
40388 Properties:
40390 • Config: set_env
40392 • Env Var: RCLONE_SFTP_SET_ENV
40394 • Type: SpaceSepList
40396 • Default:
40398 Limitations
40400 On some SFTP servers (e.g. Synology) the paths are different for SSH
40401 and SFTP so the hashes can't be calculated properly. For them using
40402 disable_hashcheck is a good idea.
40403 The only ssh agent supported under Windows is Putty's pageant.
40405 The Go SSH library disables the use of the aes128-cbc cipher by de‐
40407 fault, due to security concerns. This can be re-enabled on a per-con‐
40408 nection basis by setting the use_insecure_cipher setting in the config‐
40409 uration file to true. Further details on the insecurity of this cipher
40410 can be found in this paper (http://www.isg.rhul.ac.uk/~kp/SandPfi‐
40411 nal.pdf).
40412 SFTP isn't supported under plan9 until this issue
40414 (https://github.com/pkg/sftp/issues/156) is fixed.
40415 Note that since SFTP isn't HTTP based the following flags don't work
40417 with it: --dump-headers, --dump-bodies, --dump-auth.
40418 Note that --timeout and --contimeout are both supported.
40420 rsync.net
40422 rsync.net is supported through the SFTP backend.
40423 See rsync.net's documentation of rclone examples
40425 (https://www.rsync.net/products/rclone.html).
40426 Hetzner Storage Box
40428 Hetzner Storage Boxes are supported through the SFTP backend on port
40429 23.
40430 See Hetzner's documentation for details (https://docs.hetzner.com/ro‐
40432 bot/storage-box/access/access-ssh-rsync-borg#rclone)
40433
40435 SMB is a communication protocol to share files over network
40436 (https://en.wikipedia.org/wiki/Server_Message_Block).
40437 This relies on go-smb2 library (https://github.com/hirochachacha/go-
40439 smb2/) for communication with SMB protocol.
40440 Paths are specified as remote:sharename (or remote: for the lsd com‐
40442 mand.) You may put subdirectories in too, e.g. re‐
40443 mote:item/path/to/dir.
40444 Notes
40446 The first path segment must be the name of the share, which you entered
40447 when you started to share on Windows. On smbd, it's the section title
40448 in smb.conf (usually in /etc/samba/) file. You can find shares by
40449 quering the root if you're unsure (e.g. rclone lsd remote:).
40450 You can't access to the shared printers from rclone, obviously.
40452 You can't use Anonymous access for logging in. You have to use the
40454 guest user with an empty password instead. The rclone client tries to
40455 avoid 8.3 names when uploading files by encoding trailing spaces and
40456 periods. Alternatively, the local backend (https://rclone.org/lo‐
40457 cal/#paths-on-windows) on Windows can access SMB servers using UNC
40458 paths, by \\server\share. This doesn't apply to non-Windows OSes, such
40459 as Linux and macOS.
40460 Configuration
40462 Here is an example of making a SMB configuration.
40463 First run
40465 rclone config
40467 This will guide you through an interactive setup process.
40469 No remotes found, make a new one?
40471 n) New remote
40472 s) Set configuration password
40473 q) Quit config
40474 n/s/q> n
40475 name> remote
40476 Option Storage.
40477 Type of storage to configure.
40478 Choose a number from below, or type in your own value.
40479 XX / SMB / CIFS
40480 \ (smb)
40481 Storage> smb
40482 Option host.
40484 Samba hostname to connect to.
40485 E.g. "example.com".
40486 Enter a value.
40487 host> localhost
40488 Option user.
40490 Samba username.
40491 Enter a string value. Press Enter for the default (lesmi).
40492 user> guest
40493 Option port.
40495 Samba port number.
40496 Enter a signed integer. Press Enter for the default (445).
40497 port>
40498 Option pass.
40500 Samba password.
40501 Choose an alternative below. Press Enter for the default (n).
40502 y) Yes, type in my own password
40503 g) Generate random password
40504 n) No, leave this optional password blank (default)
40505 y/g/n> g
40506 Password strength in bits.
40507 64 is just about memorable
40508 128 is secure
40509 1024 is the maximum
40510 Bits> 64
40511 Your password is: XXXX
40512 Use this password? Please note that an obscured version of this
40513 password (and not the password itself) will be stored under your
40514 configuration file, so keep this generated password in a safe place.
40515 y) Yes (default)
40516 n) No
40517 y/n> y
40518 Option domain.
40520 Domain name for NTLM authentication.
40521 Enter a string value. Press Enter for the default (WORKGROUP).
40522 domain>
40523 Edit advanced config?
40525 y) Yes
40526 n) No (default)
40527 y/n> n
40528 Configuration complete.
40530 Options:
40531 - type: samba
40532 - host: localhost
40533 - user: guest
40534 - pass: *** ENCRYPTED ***
40535 Keep this "remote" remote?
40536 y) Yes this is OK (default)
40537 e) Edit this remote
40538 d) Delete this remote
40539 y/e/d> d
40540 Standard options
40542 Here are the Standard options specific to smb (SMB / CIFS).
40543 --smb-host
40545 SMB server hostname to connect to.
40546 E.g. "example.com".
40548 Properties:
40550 • Config: host
40552 • Env Var: RCLONE_SMB_HOST
40554 • Type: string
40556 • Required: true
40558 --smb-user
40560 SMB username.
40561 Properties:
40563 • Config: user
40565 • Env Var: RCLONE_SMB_USER
40567 • Type: string
40569 • Default: "$USER"
40571 --smb-port
40573 SMB port number.
40574 Properties:
40576 • Config: port
40578 • Env Var: RCLONE_SMB_PORT
40580 • Type: int
40582 • Default: 445
40584 --smb-pass
40586 SMB password.
40587 NB Input to this must be obscured - see rclone obscure
40589 (https://rclone.org/commands/rclone_obscure/).
40590 Properties:
40592 • Config: pass
40594 • Env Var: RCLONE_SMB_PASS
40596 • Type: string
40598 • Required: false
40600 --smb-domain
40602 Domain name for NTLM authentication.
40603 Properties:
40605 • Config: domain
40607 • Env Var: RCLONE_SMB_DOMAIN
40609 • Type: string
40611 • Default: "WORKGROUP"
40613 Advanced options
40615 Here are the Advanced options specific to smb (SMB / CIFS).
40616 --smb-idle-timeout
40618 Max time before closing idle connections.
40619 If no connections have been returned to the connection pool in the time
40621 given, rclone will empty the connection pool.
40622 Set to 0 to keep connections indefinitely.
40624 Properties:
40626 • Config: idle_timeout
40628 • Env Var: RCLONE_SMB_IDLE_TIMEOUT
40630 • Type: Duration
40632 • Default: 1m0s
40634 --smb-hide-special-share
40636 Hide special shares (e.g. print$) which users aren't supposed to ac‐
40637 cess.
40638 Properties:
40640 • Config: hide_special_share
40642 • Env Var: RCLONE_SMB_HIDE_SPECIAL_SHARE
40644 • Type: bool
40646 • Default: true
40648 --smb-case-insensitive
40650 Whether the server is configured to be case-insensitive.
40651 Always true on Windows shares.
40653 Properties:
40655 • Config: case_insensitive
40657 • Env Var: RCLONE_SMB_CASE_INSENSITIVE
40659 • Type: bool
40661 • Default: true
40663 --smb-encoding
40665 The encoding for the backend.
40666 See the encoding section in the overview (https://rclone.org/over‐
40668 view/#encoding) for more info.
40669 Properties:
40671 • Config: encoding
40673 • Env Var: RCLONE_SMB_ENCODING
40675 • Type: MultiEncoder
40677 • Default: Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,Back‐
40679 Slash,Ctl,RightSpace,RightPeriod,InvalidUtf8,Dot
40680
40682 Storj (https://storj.io) is an encrypted, secure, and cost-effective
40683 object storage service that enables you to store, back up, and archive
40684 large amounts of data in a decentralized manner.
40685 Backend options
40687 Storj can be used both with this native backend and with the s3 backend
40688 using the Storj S3 compatible gateway (https://rclone.org/s3/#storj)
40689 (shared or private).
40690 Use this backend to take advantage of client-side encryption as well as
40692 to achieve the best possible download performance. Uploads will be
40693 erasure-coded locally, thus a 1gb upload will result in 2.68gb of data
40694 being uploaded to storage nodes across the network.
40695 Use the s3 backend and one of the S3 compatible Hosted Gateways to in‐
40697 crease upload performance and reduce the load on your systems and net‐
40698 work. Uploads will be encrypted and erasure-coded server-side, thus a
40699 1GB upload will result in only in 1GB of data being uploaded to storage
40700 nodes across the network.
40701 Side by side comparison with more details:
40703 • Characteristics:
40705 • Storj backend: Uses native RPC protocol, connects directly to the
40707 storage nodes which hosts the data. Requires more CPU resource of
40708 encoding/decoding and has network amplification (especially during
40709 the upload), uses lots of TCP connections
40710 • S3 backend: Uses S3 compatible HTTP Rest API via the shared gate‐
40712 ways. There is no network amplification, but performance depends
40713 on the shared gateways and the secret encryption key is shared with
40714 the gateway.
40715 • Typical usage:
40717 • Storj backend: Server environments and desktops with enough re‐
40719 sources, internet speed and connectivity - and applications where
40720 storjs client-side encryption is required.
40721 • S3 backend: Desktops and similar with limited resources, internet
40723 speed or connectivity.
40724 • Security:
40726 • Storj backend: strong. Private encryption key doesn't need to
40728 leave the local computer.
40729 • S3 backend: weaker. Private encryption key is shared with
40731 (https://docs.storj.io/dcs/api-reference/s3-compatible-gateway#se‐
40732 curity-and-encryption) the authentication service of the hosted
40733 gateway, where it's stored encrypted. It can be stronger when com‐
40734 bining with the rclone crypt backend.
40735 • Bandwidth usage (upload):
40737 • Storj backend: higher. As data is erasure coded on the client side
40739 both the original data and the parities should be uploaded. About
40740 ~2.7 times more data is required to be uploaded. Client may start
40741 to upload with even higher number of nodes (~3.7 times more) and
40742 abandon/stop the slow uploads.
40743 • S3 backend: normal. Only the raw data is uploaded, erasure coding
40745 happens on the gateway.
40746 • Bandwidth usage (download)
40748 • Storj backend: almost normal. Only the minimal number of data is
40750 required, but to avoid very slow data providers a few more sources
40751 are used and the slowest are ignored (max 1.2x overhead).
40752 • S3 backend: normal. Only the raw data is downloaded, erasure cod‐
40754 ing happens on the shared gateway.
40755 • CPU usage:
40757 • Storj backend: higher, but more predictable. Erasure code and en‐
40759 cryption/decryption happens locally which requires significant CPU
40760 usage.
40761 • S3 backend: less. Erasure code and encryption/decryption happens
40763 on shared s3 gateways (and as is, it depends on the current load on
40764 the gateways)
40765 • TCP connection usage:
40767 • Storj backend: high. A direct connection is required to each of
40769 the Storj nodes resulting in 110 connections on upload and 35 on
40770 download per 64 MB segment. Not all the connections are actively
40771 used (slow ones are pruned), but they are all opened. Adjusting
40772 the max open file limit (https://rclone.org/storj/#known-issues)
40773 may be required.
40774 • S3 backend: normal. Only one connection per download/upload thread
40776 is required to the shared gateway.
40777 • Overall performance:
40779 • Storj backend: with enough resources (CPU and bandwidth) storj
40781 backend can provide even 2x better performance. Data is directly
40782 downloaded to / uploaded from to the client instead of the gateway.
40783 • S3 backend: Can be faster on edge devices where CPU and network
40785 bandwidth is limited as the shared S3 compatible gateways take care
40786 about the encrypting/decryption and erasure coding and no down‐
40787 load/upload amplification.
40788 • Decentralization:
40790 • Storj backend: high. Data is downloaded directly from the distrib‐
40792 uted cloud of storage providers.
40793 • S3 backend: low. Requires a running S3 gateway (either self-hosted
40795 or Storj-hosted).
40796 • Limitations:
40798 • Storj backend: rclone checksum is not possible without download, as
40800 checksum metadata is not calculated during upload
40801 • S3 backend: secret encryption key is shared with the gateway
40803 Configuration
40805 To make a new Storj configuration you need one of the following: * Ac‐
40806 cess Grant that someone else shared with you. * API Key (https://docu‐
40807 mentation.storj.io/getting-started/uploading-your-first-object/create-
40808 an-api-key) of a Storj project you are a member of.
40809 Here is an example of how to make a remote called remote. First run:
40811 rclone config
40813 This will guide you through an interactive setup process:
40815 Setup with access grant
40817 No remotes found, make a new one?
40818 n) New remote
40819 s) Set configuration password
40820 q) Quit config
40821 n/s/q> n
40822 name> remote
40823 Type of storage to configure.
40824 Enter a string value. Press Enter for the default ("").
40825 Choose a number from below, or type in your own value
40826 [snip]
40827 XX / Storj Decentralized Cloud Storage
40828 \ "storj"
40829 [snip]
40830 Storage> storj
40831 ** See help for storj backend at: https://rclone.org/storj/ **
40832 Choose an authentication method.
40834 Enter a string value. Press Enter for the default ("existing").
40835 Choose a number from below, or type in your own value
40836 1 / Use an existing access grant.
40837 \ "existing"
40838 2 / Create a new access grant from satellite address, API key, and passphrase.
40839 \ "new"
40840 provider> existing
40841 Access Grant.
40842 Enter a string value. Press Enter for the default ("").
40843 access_grant> your-access-grant-received-by-someone-else
40844 Remote config
40845 --------------------
40846 [remote]
40847 type = storj
40848 access_grant = your-access-grant-received-by-someone-else
40849 --------------------
40850 y) Yes this is OK (default)
40851 e) Edit this remote
40852 d) Delete this remote
40853 y/e/d> y
40854 Setup with API key and passphrase
40856 No remotes found, make a new one?
40857 n) New remote
40858 s) Set configuration password
40859 q) Quit config
40860 n/s/q> n
40861 name> remote
40862 Type of storage to configure.
40863 Enter a string value. Press Enter for the default ("").
40864 Choose a number from below, or type in your own value
40865 [snip]
40866 XX / Storj Decentralized Cloud Storage
40867 \ "storj"
40868 [snip]
40869 Storage> storj
40870 ** See help for storj backend at: https://rclone.org/storj/ **
40871 Choose an authentication method.
40873 Enter a string value. Press Enter for the default ("existing").
40874 Choose a number from below, or type in your own value
40875 1 / Use an existing access grant.
40876 \ "existing"
40877 2 / Create a new access grant from satellite address, API key, and passphrase.
40878 \ "new"
40879 provider> new
40880 Satellite Address. Custom satellite address should match the format: `<nodeid>@<address>:<port>`.
40881 Enter a string value. Press Enter for the default ("us-central-1.storj.io").
40882 Choose a number from below, or type in your own value
40883 1 / US Central 1
40884 \ "us-central-1.storj.io"
40885 2 / Europe West 1
40886 \ "europe-west-1.storj.io"
40887 3 / Asia East 1
40888 \ "asia-east-1.storj.io"
40889 satellite_address> 1
40890 API Key.
40891 Enter a string value. Press Enter for the default ("").
40892 api_key> your-api-key-for-your-storj-project
40893 Encryption Passphrase. To access existing objects enter passphrase used for uploading.
40894 Enter a string value. Press Enter for the default ("").
40895 passphrase> your-human-readable-encryption-passphrase
40896 Remote config
40897 --------------------
40898 [remote]
40899 type = storj
40900 satellite_address = 12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us-central-1.tardigrade.io:7777
40901 api_key = your-api-key-for-your-storj-project
40902 passphrase = your-human-readable-encryption-passphrase
40903 access_grant = the-access-grant-generated-from-the-api-key-and-passphrase
40904 --------------------
40905 y) Yes this is OK (default)
40906 e) Edit this remote
40907 d) Delete this remote
40908 y/e/d> y
40909 Standard options
40911 Here are the Standard options specific to storj (Storj Decentralized
40912 Cloud Storage).
40913 --storj-provider
40915 Choose an authentication method.
40916 Properties:
40918 • Config: provider
40920 • Env Var: RCLONE_STORJ_PROVIDER
40922 • Type: string
40924 • Default: "existing"
40926 • Examples:
40928 • "existing"
40930 • Use an existing access grant.
40932 • "new"
40934 • Create a new access grant from satellite address, API key, and
40936 passphrase.
40937 --storj-access-grant
40939 Access grant.
40940 Properties:
40942 • Config: access_grant
40944 • Env Var: RCLONE_STORJ_ACCESS_GRANT
40946 • Provider: existing
40948 • Type: string
40950 • Required: false
40952 --storj-satellite-address
40954 Satellite address.
40955 Custom satellite address should match the format: <nodeid>@<ad‐
40957 dress>:<port>.
40958 Properties:
40960 • Config: satellite_address
40962 • Env Var: RCLONE_STORJ_SATELLITE_ADDRESS
40964 • Provider: new
40966 • Type: string
40968 • Default: "us-central-1.storj.io"
40970 • Examples:
40972 • "us-central-1.storj.io"
40974 • US Central 1
40976 • "europe-west-1.storj.io"
40978 • Europe West 1
40980 • "asia-east-1.storj.io"
40982 • Asia East 1
40984 --storj-api-key
40986 API key.
40987 Properties:
40989 • Config: api_key
40991 • Env Var: RCLONE_STORJ_API_KEY
40993 • Provider: new
40995 • Type: string
40997 • Required: false
40999 --storj-passphrase
41001 Encryption passphrase.
41002 To access existing objects enter passphrase used for uploading.
41004 Properties:
41006 • Config: passphrase
41008 • Env Var: RCLONE_STORJ_PASSPHRASE
41010 • Provider: new
41012 • Type: string
41014 • Required: false
41016 Usage
41018 Paths are specified as remote:bucket (or remote: for the lsf command.)
41019 You may put subdirectories in too, e.g. remote:bucket/path/to/dir.
41020 Once configured you can then use rclone like this.
41022 Create a new bucket
41024 Use the mkdir command to create new bucket, e.g. bucket.
41025 rclone mkdir remote:bucket
41027 List all buckets
41029 Use the lsf command to list all buckets.
41030 rclone lsf remote:
41032 Note the colon (:) character at the end of the command line.
41034 Delete a bucket
41036 Use the rmdir command to delete an empty bucket.
41037 rclone rmdir remote:bucket
41039 Use the purge command to delete a non-empty bucket with all its con‐
41041 tent.
41042 rclone purge remote:bucket
41044 Upload objects
41046 Use the copy command to upload an object.
41047 rclone copy --progress /home/local/directory/file.ext remote:bucket/path/to/dir/
41049 The --progress flag is for displaying progress information. Remove it
41051 if you don't need this information.
41052 Use a folder in the local path to upload all its objects.
41054 rclone copy --progress /home/local/directory/ remote:bucket/path/to/dir/
41056 Only modified files will be copied.
41058 List objects
41060 Use the ls command to list recursively all objects in a bucket.
41061 rclone ls remote:bucket
41063 Add the folder to the remote path to list recursively all objects in
41065 this folder.
41066 rclone ls remote:bucket/path/to/dir/
41068 Use the lsf command to list non-recursively all objects in a bucket or
41070 a folder.
41071 rclone lsf remote:bucket/path/to/dir/
41073 Download objects
41075 Use the copy command to download an object.
41076 rclone copy --progress remote:bucket/path/to/dir/file.ext /home/local/directory/
41078 The --progress flag is for displaying progress information. Remove it
41080 if you don't need this information.
41081 Use a folder in the remote path to download all its objects.
41083 rclone copy --progress remote:bucket/path/to/dir/ /home/local/directory/
41085 Delete objects
41087 Use the deletefile command to delete a single object.
41088 rclone deletefile remote:bucket/path/to/dir/file.ext
41090 Use the delete command to delete all object in a folder.
41092 rclone delete remote:bucket/path/to/dir/
41094 Print the total size of objects
41096 Use the size command to print the total size of objects in a bucket or
41097 a folder.
41098 rclone size remote:bucket/path/to/dir/
41100 Sync two Locations
41102 Use the sync command to sync the source to the destination, changing
41103 the destination only, deleting any excess files.
41104 rclone sync -i --progress /home/local/directory/ remote:bucket/path/to/dir/
41106 The --progress flag is for displaying progress information. Remove it
41108 if you don't need this information.
41109 Since this can cause data loss, test first with the --dry-run flag to
41111 see exactly what would be copied and deleted.
41112 The sync can be done also from Storj to the local file system.
41114 rclone sync -i --progress remote:bucket/path/to/dir/ /home/local/directory/
41116 Or between two Storj buckets.
41118 rclone sync -i --progress remote-us:bucket/path/to/dir/ remote-europe:bucket/path/to/dir/
41120 Or even between another cloud storage and Storj.
41122 rclone sync -i --progress s3:bucket/path/to/dir/ storj:bucket/path/to/dir/
41124 Limitations
41126 rclone about is not supported by the rclone Storj backend. Backends
41127 without this capability cannot determine free space for an rclone mount
41128 or use policy mfs (most free space) as a member of an rclone union re‐
41129 mote.
41130 See List of backends that do not support rclone about
41132 (https://rclone.org/overview/#optional-features) and rclone about
41133 (https://rclone.org/commands/rclone_about/)
41134 Known issues
41136 If you get errors like too many open files this usually happens when
41137 the default ulimit for system max open files is exceeded. Native Storj
41138 protocol opens a large number of TCP connections (each of which is
41139 counted as an open file). For a single upload stream you can expect
41140 110 TCP connections to be opened. For a single download stream you can
41141 expect 35. This batch of connections will be opened for every 64 MiB
41142 segment and you should also expect TCP connections to be reused. If
41143 you do many transfers you eventually open a connection to most storage
41144 nodes (thousands of nodes).
41145 To fix these, please raise your system limits. You can do this issuing
41147 a ulimit -n 65536 just before you run rclone. To change the limits
41148 more permanently you can add this to your shell startup script, e.g.
41149 $HOME/.bashrc, or change the system-wide configuration, usually
41150 /etc/sysctl.conf and/or /etc/security/limits.conf, but please refer to
41151 your operating system manual.
41152
41154 SugarSync (https://sugarsync.com) is a cloud service that enables ac‐
41155 tive synchronization of files across computers and other devices for
41156 file backup, access, syncing, and sharing.
41157 Configuration
41159 The initial setup for SugarSync involves getting a token from SugarSync
41160 which you can do with rclone. rclone config walks you through it.
41161 Here is an example of how to make a remote called remote. First run:
41163 rclone config
41165 This will guide you through an interactive setup process:
41167 No remotes found, make a new one?
41169 n) New remote
41170 s) Set configuration password
41171 q) Quit config
41172 n/s/q> n
41173 name> remote
41174 Type of storage to configure.
41175 Enter a string value. Press Enter for the default ("").
41176 Choose a number from below, or type in your own value
41177 [snip]
41178 XX / Sugarsync
41179 \ "sugarsync"
41180 [snip]
41181 Storage> sugarsync
41182 ** See help for sugarsync backend at: https://rclone.org/sugarsync/ **
41183 Sugarsync App ID.
41185 Leave blank to use rclone's.
41186 Enter a string value. Press Enter for the default ("").
41187 app_id>
41188 Sugarsync Access Key ID.
41189 Leave blank to use rclone's.
41190 Enter a string value. Press Enter for the default ("").
41191 access_key_id>
41192 Sugarsync Private Access Key
41193 Leave blank to use rclone's.
41194 Enter a string value. Press Enter for the default ("").
41195 private_access_key>
41196 Permanently delete files if true
41197 otherwise put them in the deleted files.
41198 Enter a boolean value (true or false). Press Enter for the default ("false").
41199 hard_delete>
41200 Edit advanced config? (y/n)
41201 y) Yes
41202 n) No (default)
41203 y/n> n
41204 Remote config
41205 Username (email address)> nick@craig-wood.com
41206 Your Sugarsync password is only required during setup and will not be stored.
41207 password:
41208 --------------------
41209 [remote]
41210 type = sugarsync
41211 refresh_token = https://api.sugarsync.com/app-authorization/XXXXXXXXXXXXXXXXXX
41212 --------------------
41213 y) Yes this is OK (default)
41214 e) Edit this remote
41215 d) Delete this remote
41216 y/e/d> y
41217 Note that the config asks for your email and password but doesn't store
41219 them, it only uses them to get the initial token.
41220 Once configured you can then use rclone like this,
41222 List directories (sync folders) in top level of your SugarSync
41224 rclone lsd remote:
41226 List all the files in your SugarSync folder "Test"
41228 rclone ls remote:Test
41230 To copy a local directory to an SugarSync folder called backup
41232 rclone copy /home/source remote:backup
41234 Paths are specified as remote:path
41236 Paths may be as deep as required, e.g. remote:directory/subdirectory.
41238 NB you can't create files in the top level folder you have to create a
41240 folder, which rclone will create as a "Sync Folder" with SugarSync.
41241 Modified time and hashes
41243 SugarSync does not support modification times or hashes, therefore
41244 syncing will default to --size-only checking. Note that using --update
41245 will work as rclone can read the time files were uploaded.
41246 Restricted filename characters
41248 SugarSync replaces the default restricted characters set
41249 (https://rclone.org/overview/#restricted-characters) except for DEL.
41250 Invalid UTF-8 bytes will also be replaced (https://rclone.org/over‐
41252 view/#invalid-utf8), as they can't be used in XML strings.
41253 Deleting files
41255 Deleted files will be moved to the "Deleted items" folder by default.
41256 However you can supply the flag --sugarsync-hard-delete or set the con‐
41258 fig parameter hard_delete = true if you would like files to be deleted
41259 straight away.
41260 Standard options
41262 Here are the Standard options specific to sugarsync (Sugarsync).
41263 --sugarsync-app-id
41265 Sugarsync App ID.
41266 Leave blank to use rclone's.
41268 Properties:
41270 • Config: app_id
41272 • Env Var: RCLONE_SUGARSYNC_APP_ID
41274 • Type: string
41276 • Required: false
41278 --sugarsync-access-key-id
41280 Sugarsync Access Key ID.
41281 Leave blank to use rclone's.
41283 Properties:
41285 • Config: access_key_id
41287 • Env Var: RCLONE_SUGARSYNC_ACCESS_KEY_ID
41289 • Type: string
41291 • Required: false
41293 --sugarsync-private-access-key
41295 Sugarsync Private Access Key.
41296 Leave blank to use rclone's.
41298 Properties:
41300 • Config: private_access_key
41302 • Env Var: RCLONE_SUGARSYNC_PRIVATE_ACCESS_KEY
41304 • Type: string
41306 • Required: false
41308 --sugarsync-hard-delete
41310 Permanently delete files if true otherwise put them in the deleted
41311 files.
41312 Properties:
41314 • Config: hard_delete
41316 • Env Var: RCLONE_SUGARSYNC_HARD_DELETE
41318 • Type: bool
41320 • Default: false
41322 Advanced options
41324 Here are the Advanced options specific to sugarsync (Sugarsync).
41325 --sugarsync-refresh-token
41327 Sugarsync refresh token.
41328 Leave blank normally, will be auto configured by rclone.
41330 Properties:
41332 • Config: refresh_token
41334 • Env Var: RCLONE_SUGARSYNC_REFRESH_TOKEN
41336 • Type: string
41338 • Required: false
41340 --sugarsync-authorization
41342 Sugarsync authorization.
41343 Leave blank normally, will be auto configured by rclone.
41345 Properties:
41347 • Config: authorization
41349 • Env Var: RCLONE_SUGARSYNC_AUTHORIZATION
41351 • Type: string
41353 • Required: false
41355 --sugarsync-authorization-expiry
41357 Sugarsync authorization expiry.
41358 Leave blank normally, will be auto configured by rclone.
41360 Properties:
41362 • Config: authorization_expiry
41364 • Env Var: RCLONE_SUGARSYNC_AUTHORIZATION_EXPIRY
41366 • Type: string
41368 • Required: false
41370 --sugarsync-user
41372 Sugarsync user.
41373 Leave blank normally, will be auto configured by rclone.
41375 Properties:
41377 • Config: user
41379 • Env Var: RCLONE_SUGARSYNC_USER
41381 • Type: string
41383 • Required: false
41385 --sugarsync-root-id
41387 Sugarsync root id.
41388 Leave blank normally, will be auto configured by rclone.
41390 Properties:
41392 • Config: root_id
41394 • Env Var: RCLONE_SUGARSYNC_ROOT_ID
41396 • Type: string
41398 • Required: false
41400 --sugarsync-deleted-id
41402 Sugarsync deleted folder id.
41403 Leave blank normally, will be auto configured by rclone.
41405 Properties:
41407 • Config: deleted_id
41409 • Env Var: RCLONE_SUGARSYNC_DELETED_ID
41411 • Type: string
41413 • Required: false
41415 --sugarsync-encoding
41417 The encoding for the backend.
41418 See the encoding section in the overview (https://rclone.org/over‐
41420 view/#encoding) for more info.
41421 Properties:
41423 • Config: encoding
41425 • Env Var: RCLONE_SUGARSYNC_ENCODING
41427 • Type: MultiEncoder
41429 • Default: Slash,Ctl,InvalidUtf8,Dot
41431 Limitations
41433 rclone about is not supported by the SugarSync backend. Backends with‐
41434 out this capability cannot determine free space for an rclone mount or
41435 use policy mfs (most free space) as a member of an rclone union remote.
41436 See List of backends that do not support rclone about
41438 (https://rclone.org/overview/#optional-features) and rclone about
41439 (https://rclone.org/commands/rclone_about/)
41440
41442 The Tardigrade backend has been renamed to be the Storj backend
41443 (https://rclone.org/storj/). Old configuration files will continue to
41444 work.
41445
41447 This is a Backend for Uptobox file storage service. Uptobox is closer
41448 to a one-click hoster than a traditional cloud storage provider and
41449 therefore not suitable for long term storage.
41450 Paths are specified as remote:path
41452 Paths may be as deep as required, e.g. remote:directory/subdirectory.
41454 Configuration
41456 To configure an Uptobox backend you'll need your personal api token.
41457 You'll find it in your account settings (https://uptobox.com/my_ac‐
41458 count)
41459 Here is an example of how to make a remote called remote with the de‐
41461 fault setup. First run:
41462 rclone config
41464 This will guide you through an interactive setup process:
41466 Current remotes:
41468 Name Type
41470 ==== ====
41471 TestUptobox uptobox
41472 e) Edit existing remote
41474 n) New remote
41475 d) Delete remote
41476 r) Rename remote
41477 c) Copy remote
41478 s) Set configuration password
41479 q) Quit config
41480 e/n/d/r/c/s/q> n
41481 name> uptobox
41482 Type of storage to configure.
41483 Enter a string value. Press Enter for the default ("").
41484 Choose a number from below, or type in your own value
41485 [...]
41486 37 / Uptobox
41487 \ "uptobox"
41488 [...]
41489 Storage> uptobox
41490 ** See help for uptobox backend at: https://rclone.org/uptobox/ **
41491 Your API Key, get it from https://uptobox.com/my_account
41493 Enter a string value. Press Enter for the default ("").
41494 api_key> xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
41495 Edit advanced config? (y/n)
41496 y) Yes
41497 n) No (default)
41498 y/n> n
41499 Remote config
41500 --------------------
41501 [uptobox]
41502 type = uptobox
41503 api_key = xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
41504 --------------------
41505 y) Yes this is OK (default)
41506 e) Edit this remote
41507 d) Delete this remote
41508 y/e/d>
41509 Once configured you can then use rclone like this,
41511 List directories in top level of your Uptobox
41513 rclone lsd remote:
41515 List all the files in your Uptobox
41517 rclone ls remote:
41519 To copy a local directory to an Uptobox directory called backup
41521 rclone copy /home/source remote:backup
41523 Modified time and hashes
41525 Uptobox supports neither modified times nor checksums.
41526 Restricted filename characters
41528 In addition to the default restricted characters set
41529 (https://rclone.org/overview/#restricted-characters) the following
41530 characters are also replaced:
41531 Character Value Replacement
41533 ────────────────────────────────
41534 " 0x22 ＂
41535 ` 0x41 ｀
41536 Invalid UTF-8 bytes will also be replaced (https://rclone.org/over‐
41538 view/#invalid-utf8), as they can't be used in XML strings.
41539 Standard options
41541 Here are the Standard options specific to uptobox (Uptobox).
41542 --uptobox-access-token
41544 Your access token.
41545 Get it from https://uptobox.com/my_account.
41547 Properties:
41549 • Config: access_token
41551 • Env Var: RCLONE_UPTOBOX_ACCESS_TOKEN
41553 • Type: string
41555 • Required: false
41557 Advanced options
41559 Here are the Advanced options specific to uptobox (Uptobox).
41560 --uptobox-encoding
41562 The encoding for the backend.
41563 See the encoding section in the overview (https://rclone.org/over‐
41565 view/#encoding) for more info.
41566 Properties:
41568 • Config: encoding
41570 • Env Var: RCLONE_UPTOBOX_ENCODING
41572 • Type: MultiEncoder
41574 • Default: Slash,LtGt,DoubleQuote,BackQuote,Del,Ctl,LeftSpace,In‐
41576 validUtf8,Dot
41577 Limitations
41579 Uptobox will delete inactive files that have not been accessed in 60
41580 days.
41581 rclone about is not supported by this backend an overview of used space
41583 can however been seen in the uptobox web interface.
41584
41586 The union remote provides a unification similar to UnionFS using other
41587 remotes.
41588 Paths may be as deep as required or a local path, e.g. remote:directo‐
41590 ry/subdirectory or /directory/subdirectory.
41591 During the initial setup with rclone config you will specify the up‐
41593 stream remotes as a space separated list. The upstream remotes can ei‐
41594 ther be a local paths or other remotes.
41595 Attribute :ro and :nc can be attach to the end of path to tag the re‐
41597 mote as read only or no create, e.g. remote:directory/subdirectory:ro
41598 or remote:directory/subdirectory:nc.
41599 Subfolders can be used in upstream remotes. Assume a union remote
41601 named backup with the remotes mydrive:private/backup. Invoking rclone
41602 mkdir backup:desktop is exactly the same as invoking rclone mkdir my‐
41603 drive:private/backup/desktop.
41604 There will be no special handling of paths containing .. segments.
41606 Invoking rclone mkdir backup:../desktop is exactly the same as invoking
41607 rclone mkdir mydrive:private/backup/../desktop.
41608 Configuration
41610 Here is an example of how to make a union called remote for local fold‐
41611 ers. First run:
41612 rclone config
41614 This will guide you through an interactive setup process:
41616 No remotes found, make a new one?
41618 n) New remote
41619 s) Set configuration password
41620 q) Quit config
41621 n/s/q> n
41622 name> remote
41623 Type of storage to configure.
41624 Choose a number from below, or type in your own value
41625 [snip]
41626 XX / Union merges the contents of several remotes
41627 \ "union"
41628 [snip]
41629 Storage> union
41630 List of space separated upstreams.
41631 Can be 'upstreama:test/dir upstreamb:', '\"upstreama:test/space:ro dir\" upstreamb:', etc.
41632 Enter a string value. Press Enter for the default ("").
41633 upstreams> remote1:dir1 remote2:dir2 remote3:dir3
41634 Policy to choose upstream on ACTION class.
41635 Enter a string value. Press Enter for the default ("epall").
41636 action_policy>
41637 Policy to choose upstream on CREATE class.
41638 Enter a string value. Press Enter for the default ("epmfs").
41639 create_policy>
41640 Policy to choose upstream on SEARCH class.
41641 Enter a string value. Press Enter for the default ("ff").
41642 search_policy>
41643 Cache time of usage and free space (in seconds). This option is only useful when a path preserving policy is used.
41644 Enter a signed integer. Press Enter for the default ("120").
41645 cache_time>
41646 Remote config
41647 --------------------
41648 [remote]
41649 type = union
41650 upstreams = remote1:dir1 remote2:dir2 remote3:dir3
41651 --------------------
41652 y) Yes this is OK
41653 e) Edit this remote
41654 d) Delete this remote
41655 y/e/d> y
41656 Current remotes:
41657 Name Type
41659 ==== ====
41660 remote union
41661 e) Edit existing remote
41663 n) New remote
41664 d) Delete remote
41665 r) Rename remote
41666 c) Copy remote
41667 s) Set configuration password
41668 q) Quit config
41669 e/n/d/r/c/s/q> q
41670 Once configured you can then use rclone like this,
41672 List directories in top level in remote1:dir1, remote2:dir2 and re‐
41674 mote3:dir3
41675 rclone lsd remote:
41677 List all the files in remote1:dir1, remote2:dir2 and remote3:dir3
41679 rclone ls remote:
41681 Copy another local directory to the union directory called source,
41683 which will be placed into remote3:dir3
41684 rclone copy C:\source remote:source
41686 Behavior / Policies
41688 The behavior of union backend is inspired by trapexit/mergerfs
41689 (https://github.com/trapexit/mergerfs). All functions are grouped into
41690 3 categories: action, create and search. These functions and cate‐
41691 gories can be assigned a policy which dictates what file or directory
41692 is chosen when performing that behavior. Any policy can be assigned to
41693 a function or category though some may not be very useful in practice.
41694 For instance: rand (random) may be useful for file creation (create)
41695 but could lead to very odd behavior if used for delete if there were
41696 more than one copy of the file.
41697 Function / Category classifications
41699 Cate‐ Description Functions
41700 gory
41701 ─────────────────────────────────────────────────────────────────────────────
41702 action Writing Exist‐ move, rmdir, rmdirs, delete, purge and copy, sync
41703 ing file (as destination when file exist)
41704 create Create non-ex‐ copy, sync (as destination when file not exist)
41705 isting file
41706 search Reading and ls, lsd, lsl, cat, md5sum, sha1sum and copy, sync
41707 listing file (as source)
41708 N/A size, about
41709 Path Preservation
41711 Policies, as described below, are of two basic types. path preserving
41712 and non-path preserving.
41713 All policies which start with ep (epff, eplfs, eplus, epmfs, eprand)
41715 are path preserving. ep stands for existing path.
41716 A path preserving policy will only consider upstreams where the rela‐
41718 tive path being accessed already exists.
41719 When using non-path preserving policies paths will be created in target
41721 upstreams as necessary.
41722 Quota Relevant Policies
41724 Some policies rely on quota information. These policies should be used
41725 only if your upstreams support the respective quota fields.
41726 Policy Required Field
41728 ────────────────────────────
41729 lfs, eplfs Free
41730 mfs, epmfs Free
41731 lus, eplus Used
41732 lno, eplno Objects
41733 To check if your upstream supports the field, run rclone about remote:
41735 [flags] and see if the required field exists.
41736 Filters
41738 Policies basically search upstream remotes and create a list of files /
41739 paths for functions to work on. The policy is responsible for filter‐
41740 ing and sorting. The policy type defines the sorting but filtering is
41741 mostly uniform as described below.
41742 • No search policies filter.
41744 • All action policies will filter out remotes which are tagged as read-
41746 only.
41747 • All create policies will filter out remotes which are tagged read-on‐
41749 ly or no-create.
41750 If all remotes are filtered an error will be returned.
41752 Policy descriptions
41754 The policies definition are inspired by trapexit/mergerfs
41755 (https://github.com/trapexit/mergerfs) but not exactly the same. Some
41756 policy definition could be different due to the much larger latency of
41757 remote file systems.
41758 Policy Description
41760 ──────────────────────────────────────────────────────────────────────────
41761 all Search category: same as epall. Action category: same
41762 as epall. Create category: act on all upstreams.
41763 epall (existing Search category: Given this order configured, act on
41764 path, all) the first one found where the relative path exists.
41765 Action category: apply to all found. Create category:
41766 act on all upstreams where the relative path exists.
41767 epff (existing Act on the first one found, by the time upstreams re‐
41768 path, first ply, where the relative path exists.
41769 found)
41770 eplfs (existing Of all the upstreams on which the relative path exists
41771 path, least free choose the one with the least free space.
41772 space)
41773 eplus (existing Of all the upstreams on which the relative path exists
41774 path, least used choose the one with the least used space.
41775 space)
41776 eplno (existing Of all the upstreams on which the relative path exists
41777 path, least num‐ choose the one with the least number of objects.
41778 ber of objects)
41779 epmfs (existing Of all the upstreams on which the relative path exists
41780 path, most free choose the one with the most free space.
41781 space)
41782 eprand (existing Calls epall and then randomizes. Returns only one up‐
41783 path, random) stream.
41784 ff (first found) Search category: same as epff. Action category: same
41785 as epff. Create category: Act on the first one found
41786 by the time upstreams reply.
41787 lfs (least free Search category: same as eplfs. Action category: same
41788 space) as eplfs. Create category: Pick the upstream with the
41789 least available free space.
41790 lus (least used Search category: same as eplus. Action category: same
41791 space) as eplus. Create category: Pick the upstream with the
41792 least used space.
41793 lno (least num‐ Search category: same as eplno. Action category: same
41794 ber of objects) as eplno. Create category: Pick the upstream with the
41795 least number of objects.
41796 mfs (most free Search category: same as epmfs. Action category: same
41797 space) as epmfs. Create category: Pick the upstream with the
41798 most available free space.
41799 newest Pick the file / directory with the largest mtime.
41800 rand (random) Calls all and then randomizes. Returns only one up‐
41801 stream.
41802 Standard options
41804 Here are the Standard options specific to union (Union merges the con‐
41805 tents of several upstream fs).
41806 --union-upstreams
41808 List of space separated upstreams.
41809 Can be 'upstreama:test/dir upstreamb:', '"upstreama:test/space:ro dir"
41811 upstreamb:', etc.
41812 Properties:
41814 • Config: upstreams
41816 • Env Var: RCLONE_UNION_UPSTREAMS
41818 • Type: string
41820 • Required: true
41822 --union-action-policy
41824 Policy to choose upstream on ACTION category.
41825 Properties:
41827 • Config: action_policy
41829 • Env Var: RCLONE_UNION_ACTION_POLICY
41831 • Type: string
41833 • Default: "epall"
41835 --union-create-policy
41837 Policy to choose upstream on CREATE category.
41838 Properties:
41840 • Config: create_policy
41842 • Env Var: RCLONE_UNION_CREATE_POLICY
41844 • Type: string
41846 • Default: "epmfs"
41848 --union-search-policy
41850 Policy to choose upstream on SEARCH category.
41851 Properties:
41853 • Config: search_policy
41855 • Env Var: RCLONE_UNION_SEARCH_POLICY
41857 • Type: string
41859 • Default: "ff"
41861 --union-cache-time
41863 Cache time of usage and free space (in seconds).
41864 This option is only useful when a path preserving policy is used.
41866 Properties:
41868 • Config: cache_time
41870 • Env Var: RCLONE_UNION_CACHE_TIME
41872 • Type: int
41874 • Default: 120
41876 Advanced options
41878 Here are the Advanced options specific to union (Union merges the con‐
41879 tents of several upstream fs).
41880 --union-min-free-space
41882 Minimum viable free space for lfs/eplfs policies.
41883 If a remote has less than this much free space then it won't be consid‐
41885 ered for use in lfs or eplfs policies.
41886 Properties:
41888 • Config: min_free_space
41890 • Env Var: RCLONE_UNION_MIN_FREE_SPACE
41892 • Type: SizeSuffix
41894 • Default: 1Gi
41896 Metadata
41898 Any metadata supported by the underlying remote is read and written.
41899 See the metadata (https://rclone.org/docs/#metadata) docs for more in‐
41901 fo.
41902
41904 Paths are specified as remote:path
41905 Paths may be as deep as required, e.g. remote:directory/subdirectory.
41907 Configuration
41909 To configure the WebDAV remote you will need to have a URL for it, and
41910 a username and password. If you know what kind of system you are con‐
41911 necting to then rclone can enable extra features.
41912 Here is an example of how to make a remote called remote. First run:
41914 rclone config
41916 This will guide you through an interactive setup process:
41918 No remotes found, make a new one?
41920 n) New remote
41921 s) Set configuration password
41922 q) Quit config
41923 n/s/q> n
41924 name> remote
41925 Type of storage to configure.
41926 Choose a number from below, or type in your own value
41927 [snip]
41928 XX / WebDAV
41929 \ "webdav"
41930 [snip]
41931 Storage> webdav
41932 URL of http host to connect to
41933 Choose a number from below, or type in your own value
41934 1 / Connect to example.com
41935 \ "https://example.com"
41936 url> https://example.com/remote.php/webdav/
41937 Name of the WebDAV site/service/software you are using
41938 Choose a number from below, or type in your own value
41939 1 / Nextcloud
41940 \ "nextcloud"
41941 2 / Owncloud
41942 \ "owncloud"
41943 3 / Sharepoint Online, authenticated by Microsoft account.
41944 \ "sharepoint"
41945 4 / Sharepoint with NTLM authentication. Usually self-hosted or on-premises.
41946 \ "sharepoint-ntlm"
41947 5 / Other site/service or software
41948 \ "other"
41949 vendor> 1
41950 User name
41951 user> user
41952 Password.
41953 y) Yes type in my own password
41954 g) Generate random password
41955 n) No leave this optional password blank
41956 y/g/n> y
41957 Enter the password:
41958 password:
41959 Confirm the password:
41960 password:
41961 Bearer token instead of user/pass (e.g. a Macaroon)
41962 bearer_token>
41963 Remote config
41964 --------------------
41965 [remote]
41966 type = webdav
41967 url = https://example.com/remote.php/webdav/
41968 vendor = nextcloud
41969 user = user
41970 pass = *** ENCRYPTED ***
41971 bearer_token =
41972 --------------------
41973 y) Yes this is OK
41974 e) Edit this remote
41975 d) Delete this remote
41976 y/e/d> y
41977 Once configured you can then use rclone like this,
41979 List directories in top level of your WebDAV
41981 rclone lsd remote:
41983 List all the files in your WebDAV
41985 rclone ls remote:
41987 To copy a local directory to an WebDAV directory called backup
41989 rclone copy /home/source remote:backup
41991 Modified time and hashes
41993 Plain WebDAV does not support modified times. However when used with
41994 Owncloud or Nextcloud rclone will support modified times.
41995 Likewise plain WebDAV does not support hashes, however when used with
41997 Owncloud or Nextcloud rclone will support SHA1 and MD5 hashes. Depend‐
41998 ing on the exact version of Owncloud or Nextcloud hashes may appear on
41999 all objects, or only on objects which had a hash uploaded with them.
42000 Standard options
42002 Here are the Standard options specific to webdav (WebDAV).
42003 --webdav-url
42005 URL of http host to connect to.
42006 E.g. https://example.com.
42008 Properties:
42010 • Config: url
42012 • Env Var: RCLONE_WEBDAV_URL
42014 • Type: string
42016 • Required: true
42018 --webdav-vendor
42020 Name of the WebDAV site/service/software you are using.
42021 Properties:
42023 • Config: vendor
42025 • Env Var: RCLONE_WEBDAV_VENDOR
42027 • Type: string
42029 • Required: false
42031 • Examples:
42033 • "nextcloud"
42035 • Nextcloud
42037 • "owncloud"
42039 • Owncloud
42041 • "sharepoint"
42043 • Sharepoint Online, authenticated by Microsoft account
42045 • "sharepoint-ntlm"
42047 • Sharepoint with NTLM authentication, usually self-hosted or on-
42049 premises
42050 • "other"
42052 • Other site/service or software
42054 --webdav-user
42056 User name.
42057 In case NTLM authentication is used, the username should be in the for‐
42059 mat 'Domain'.
42060 Properties:
42062 • Config: user
42064 • Env Var: RCLONE_WEBDAV_USER
42066 • Type: string
42068 • Required: false
42070 --webdav-pass
42072 Password.
42073 NB Input to this must be obscured - see rclone obscure
42075 (https://rclone.org/commands/rclone_obscure/).
42076 Properties:
42078 • Config: pass
42080 • Env Var: RCLONE_WEBDAV_PASS
42082 • Type: string
42084 • Required: false
42086 --webdav-bearer-token
42088 Bearer token instead of user/pass (e.g. a Macaroon).
42089 Properties:
42091 • Config: bearer_token
42093 • Env Var: RCLONE_WEBDAV_BEARER_TOKEN
42095 • Type: string
42097 • Required: false
42099 Advanced options
42101 Here are the Advanced options specific to webdav (WebDAV).
42102 --webdav-bearer-token-command
42104 Command to run to get a bearer token.
42105 Properties:
42107 • Config: bearer_token_command
42109 • Env Var: RCLONE_WEBDAV_BEARER_TOKEN_COMMAND
42111 • Type: string
42113 • Required: false
42115 --webdav-encoding
42117 The encoding for the backend.
42118 See the encoding section in the overview (https://rclone.org/over‐
42120 view/#encoding) for more info.
42121 Default encoding is Slash,LtGt,DoubleQuote,Colon,Question,Aster‐
42123 isk,Pipe,Hash,Percent,BackSlash,Del,Ctl,LeftSpace,Left‐
42124 Tilde,RightSpace,RightPeriod,InvalidUtf8 for sharepoint-ntlm or identi‐
42125 ty otherwise.
42126 Properties:
42128 • Config: encoding
42130 • Env Var: RCLONE_WEBDAV_ENCODING
42132 • Type: string
42134 • Required: false
42136 --webdav-headers
42138 Set HTTP headers for all transactions.
42139 Use this to set additional HTTP headers for all transactions
42141 The input format is comma separated list of key,value pairs. Standard
42143 CSV encoding (https://godoc.org/encoding/csv) may be used.
42144 For example, to set a Cookie use 'Cookie,name=value', or '"Cook‐
42146 ie","name=value"'.
42147 You can set multiple headers, e.g. '"Cookie","name=value","Authoriza‐
42149 tion","xxx"'.
42150 Properties:
42152 • Config: headers
42154 • Env Var: RCLONE_WEBDAV_HEADERS
42156 • Type: CommaSepList
42158 • Default:
42160 Provider notes
42162 See below for notes on specific providers.
42163 Owncloud
42165 Click on the settings cog in the bottom right of the page and this will
42166 show the WebDAV URL that rclone needs in the config step. It will look
42167 something like https://example.com/remote.php/webdav/.
42168 Owncloud supports modified times using the X-OC-Mtime header.
42170 Nextcloud
42172 This is configured in an identical way to Owncloud. Note that
42173 Nextcloud initially did not support streaming of files (rcat) whereas
42174 Owncloud did, but this (https://github.com/nextcloud/nextcloud-snap/is‐
42175 sues/365) seems to be fixed as of 2020-11-27 (tested with rclone
42176 v1.53.1 and Nextcloud Server v19).
42177 Sharepoint Online
42179 Rclone can be used with Sharepoint provided by OneDrive for Business or
42180 Office365 Education Accounts. This feature is only needed for a few of
42181 these Accounts, mostly Office365 Education ones. These accounts are
42182 sometimes not verified by the domain owner github#1975
42183 (https://github.com/rclone/rclone/issues/1975)
42184 This means that these accounts can't be added using the official API
42186 (other Accounts should work with the "onedrive" option). However, it
42187 is possible to access them using webdav.
42188 To use a sharepoint remote with rclone, add it like this: First, you
42190 need to get your remote's URL:
42191 • Go here (https://onedrive.live.com/about/en-us/signin/) to open your
42193 OneDrive or to sign in
42194 • Now take a look at your address bar, the URL should look like this:
42196 https://[YOUR-DOMAIN]-my.sharepoint.com/personal/[YOUR-EMAIL]/_lay‐
42197 outs/15/onedrive.aspx
42198 You'll only need this URL up to the email address. After that, you'll
42200 most likely want to add "/Documents". That subdirectory contains the
42201 actual data stored on your OneDrive.
42202 Add the remote to rclone like this: Configure the url as https://[YOUR-
42204 DOMAIN]-my.sharepoint.com/personal/[YOUR-EMAIL]/Documents and use your
42205 normal account email and password for user and pass. If you have 2FA
42206 enabled, you have to generate an app password. Set the vendor to
42207 sharepoint.
42208 Your config file should look like this:
42210 [sharepoint]
42212 type = webdav
42213 url = https://[YOUR-DOMAIN]-my.sharepoint.com/personal/[YOUR-EMAIL]/Documents
42214 vendor = sharepoint
42215 user = YourEmailAddress
42216 pass = encryptedpassword
42217 Sharepoint with NTLM Authentication
42219 Use this option in case your (hosted) Sharepoint is not tied to
42220 OneDrive accounts and uses NTLM authentication.
42221 To get the url configuration, similarly to the above, first navigate to
42223 the desired directory in your browser to get the URL, then strip every‐
42224 thing after the name of the opened directory.
42225 Example: If the URL is: https://example.sharepoint.com/sites/12345/Doc‐
42227 uments/Forms/AllItems.aspx
42228 The configuration to use would be: https://example.share‐
42230 point.com/sites/12345/Documents
42231 Set the vendor to sharepoint-ntlm.
42233 NTLM uses domain and user name combination for authentication, set user
42235 to DOMAIN\username.
42236 Your config file should look like this:
42238 [sharepoint]
42240 type = webdav
42241 url = https://[YOUR-DOMAIN]/some-path-to/Documents
42242 vendor = sharepoint-ntlm
42243 user = DOMAIN\user
42244 pass = encryptedpassword
42245 Required Flags for SharePoint
42247 As SharePoint does some special things with uploaded documents, you
42248 won't be able to use the documents size or the documents hash to com‐
42249 pare if a file has been changed since the upload / which file is newer.
42250 For Rclone calls copying files (especially Office files such as .docx,
42252 .xlsx, etc.) from/to SharePoint (like copy, sync, etc.), you should ap‐
42253 pend these flags to ensure Rclone uses the "Last Modified" datetime
42254 property to compare your documents:
42255 --ignore-size --ignore-checksum --update
42257 dCache
42259 dCache is a storage system that supports many protocols and authentica‐
42260 tion/authorisation schemes. For WebDAV clients, it allows users to au‐
42261 thenticate with username and password (BASIC), X.509, Kerberos, and
42262 various bearer tokens, including Macaroons (https://www.dcache.org/man‐
42263 uals/workshop-2017-05-29-Umea/000-Final/anupam_macaroons_v02.pdf) and
42264 OpenID-Connect (https://en.wikipedia.org/wiki/OpenID_Connect) access
42265 tokens.
42266 Configure as normal using the other type. Don't enter a username or
42268 password, instead enter your Macaroon as the bearer_token.
42269 The config will end up looking something like this.
42271 [dcache]
42273 type = webdav
42274 url = https://dcache...
42275 vendor = other
42276 user =
42277 pass =
42278 bearer_token = your-macaroon
42279 There is a script (https://github.com/sara-nl/GridScripts/blob/mas‐
42281 ter/get-macaroon) that obtains a Macaroon from a dCache WebDAV end‐
42282 point, and creates an rclone config file.
42283 Macaroons may also be obtained from the dCacheView web-browser/Java‐
42285 Script client that comes with dCache.
42286 OpenID-Connect
42288 dCache also supports authenticating with OpenID-Connect access tokens.
42289 OpenID-Connect is a protocol (based on OAuth 2.0) that allows services
42290 to identify users who have authenticated with some central service.
42291 Support for OpenID-Connect in rclone is currently achieved using anoth‐
42293 er software package called oidc-agent (https://github.com/indigo-
42294 dc/oidc-agent). This is a command-line tool that facilitates obtaining
42295 an access token. Once installed and configured, an access token is ob‐
42296 tained by running the oidc-token command. The following example shows
42297 a (shortened) access token obtained from the XDC OIDC Provider.
42298 paul@celebrimbor:~$ oidc-token XDC
42300 eyJraWQ[...]QFXDt0
42301 paul@celebrimbor:~$
42302 Note Before the oidc-token command will work, the refresh token must be
42304 loaded into the oidc agent. This is done with the oidc-add command
42305 (e.g., oidc-add XDC). This is typically done once per login session.
42306 Full details on this and how to register oidc-agent with your OIDC
42307 Provider are provided in the oidc-agent documentation (https://indigo-
42308 dc.gitbooks.io/oidc-agent/).
42309 The rclone bearer_token_command configuration option is used to fetch
42311 the access token from oidc-agent.
42312 Configure as a normal WebDAV endpoint, using the 'other' vendor, leav‐
42314 ing the username and password empty. When prompted, choose to edit the
42315 advanced config and enter the command to get a bearer token (e.g.,
42316 oidc-agent XDC).
42317 The following example config shows a WebDAV endpoint that uses oidc-
42319 agent to supply an access token from the XDC OIDC Provider.
42320 [dcache]
42322 type = webdav
42323 url = https://dcache.example.org/
42324 vendor = other
42325 bearer_token_command = oidc-token XDC
42326
42328 Yandex Disk (https://disk.yandex.com) is a cloud storage solution cre‐
42329 ated by Yandex (https://yandex.com).
42330 Configuration
42332 Here is an example of making a yandex configuration. First run
42333 rclone config
42335 This will guide you through an interactive setup process:
42337 No remotes found, make a new one?
42339 n) New remote
42340 s) Set configuration password
42341 n/s> n
42342 name> remote
42343 Type of storage to configure.
42344 Choose a number from below, or type in your own value
42345 [snip]
42346 XX / Yandex Disk
42347 \ "yandex"
42348 [snip]
42349 Storage> yandex
42350 Yandex Client Id - leave blank normally.
42351 client_id>
42352 Yandex Client Secret - leave blank normally.
42353 client_secret>
42354 Remote config
42355 Use auto config?
42356 * Say Y if not sure
42357 * Say N if you are working on a remote or headless machine
42358 y) Yes
42359 n) No
42360 y/n> y
42361 If your browser doesn't open automatically go to the following link: http://127.0.0.1:53682/auth
42362 Log in and authorize rclone for access
42363 Waiting for code...
42364 Got code
42365 --------------------
42366 [remote]
42367 client_id =
42368 client_secret =
42369 token = {"access_token":"xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx","token_type":"OAuth","expiry":"2016-12-29T12:27:11.362788025Z"}
42370 --------------------
42371 y) Yes this is OK
42372 e) Edit this remote
42373 d) Delete this remote
42374 y/e/d> y
42375 See the remote setup docs (https://rclone.org/remote_setup/) for how to
42377 set it up on a machine with no Internet browser available.
42378 Note that rclone runs a webserver on your local machine to collect the
42380 token as returned from Yandex Disk. This only runs from the moment it
42381 opens your browser to the moment you get back the verification code.
42382 This is on http://127.0.0.1:53682/ and this it may require you to un‐
42383 block it temporarily if you are running a host firewall.
42384 Once configured you can then use rclone like this,
42386 See top level directories
42388 rclone lsd remote:
42390 Make a new directory
42392 rclone mkdir remote:directory
42394 List the contents of a directory
42396 rclone ls remote:directory
42398 Sync /home/local/directory to the remote path, deleting any excess
42400 files in the path.
42401 rclone sync -i /home/local/directory remote:directory
42403 Yandex paths may be as deep as required, e.g. remote:directory/subdi‐
42405 rectory.
42406 Modified time
42408 Modified times are supported and are stored accurate to 1 ns in custom
42409 metadata called rclone_modified in RFC3339 with nanoseconds format.
42410 MD5 checksums
42412 MD5 checksums are natively supported by Yandex Disk.
42413 Emptying Trash
42415 If you wish to empty your trash you can use the rclone cleanup remote:
42416 command which will permanently delete all your trashed files. This
42417 command does not take any path arguments.
42418 Quota information
42420 To view your current quota you can use the rclone about remote: command
42421 which will display your usage limit (quota) and the current usage.
42422 Restricted filename characters
42424 The default restricted characters set (https://rclone.org/overview/#re‐
42425 stricted-characters) are replaced.
42426 Invalid UTF-8 bytes will also be replaced (https://rclone.org/over‐
42428 view/#invalid-utf8), as they can't be used in JSON strings.
42429 Standard options
42431 Here are the Standard options specific to yandex (Yandex Disk).
42432 --yandex-client-id
42434 OAuth Client Id.
42435 Leave blank normally.
42437 Properties:
42439 • Config: client_id
42441 • Env Var: RCLONE_YANDEX_CLIENT_ID
42443 • Type: string
42445 • Required: false
42447 --yandex-client-secret
42449 OAuth Client Secret.
42450 Leave blank normally.
42452 Properties:
42454 • Config: client_secret
42456 • Env Var: RCLONE_YANDEX_CLIENT_SECRET
42458 • Type: string
42460 • Required: false
42462 Advanced options
42464 Here are the Advanced options specific to yandex (Yandex Disk).
42465 --yandex-token
42467 OAuth Access Token as a JSON blob.
42468 Properties:
42470 • Config: token
42472 • Env Var: RCLONE_YANDEX_TOKEN
42474 • Type: string
42476 • Required: false
42478 --yandex-auth-url
42480 Auth server URL.
42481 Leave blank to use the provider defaults.
42483 Properties:
42485 • Config: auth_url
42487 • Env Var: RCLONE_YANDEX_AUTH_URL
42489 • Type: string
42491 • Required: false
42493 --yandex-token-url
42495 Token server url.
42496 Leave blank to use the provider defaults.
42498 Properties:
42500 • Config: token_url
42502 • Env Var: RCLONE_YANDEX_TOKEN_URL
42504 • Type: string
42506 • Required: false
42508 --yandex-hard-delete
42510 Delete files permanently rather than putting them into the trash.
42511 Properties:
42513 • Config: hard_delete
42515 • Env Var: RCLONE_YANDEX_HARD_DELETE
42517 • Type: bool
42519 • Default: false
42521 --yandex-encoding
42523 The encoding for the backend.
42524 See the encoding section in the overview (https://rclone.org/over‐
42526 view/#encoding) for more info.
42527 Properties:
42529 • Config: encoding
42531 • Env Var: RCLONE_YANDEX_ENCODING
42533 • Type: MultiEncoder
42535 • Default: Slash,Del,Ctl,InvalidUtf8,Dot
42537 Limitations
42539 When uploading very large files (bigger than about 5 GiB) you will need
42540 to increase the --timeout parameter. This is because Yandex pauses
42541 (perhaps to calculate the MD5SUM for the entire file) before returning
42542 confirmation that the file has been uploaded. The default handling of
42543 timeouts in rclone is to assume a 5 minute pause is an error and close
42544 the connection - you'll see net/http: timeout awaiting response headers
42545 errors in the logs if this is happening. Setting the timeout to twice
42546 the max size of file in GiB should be enough, so if you want to upload
42547 a 30 GiB file set a timeout of 2 * 30 = 60m, that is --timeout 60m.
42548 Having a Yandex Mail account is mandatory to use the Yandex.Disk sub‐
42550 scription. Token generation will work without a mail account, but
42551 Rclone won't be able to complete any actions.
42552 [403 - DiskUnsupportedUserAccountTypeError] User account type is not supported.
42554
42556 Zoho WorkDrive (https://www.zoho.com/workdrive/) is a cloud storage so‐
42557 lution created by Zoho (https://zoho.com).
42558 Configuration
42560 Here is an example of making a zoho configuration. First run
42561 rclone config
42563 This will guide you through an interactive setup process:
42565 No remotes found, make a new one?
42567 n) New remote
42568 s) Set configuration password
42569 n/s> n
42570 name> remote
42571 Type of storage to configure.
42572 Enter a string value. Press Enter for the default ("").
42573 Choose a number from below, or type in your own value
42574 [snip]
42575 XX / Zoho
42576 \ "zoho"
42577 [snip]
42578 Storage> zoho
42579 ** See help for zoho backend at: https://rclone.org/zoho/ **
42580 OAuth Client Id
42582 Leave blank normally.
42583 Enter a string value. Press Enter for the default ("").
42584 client_id>
42585 OAuth Client Secret
42586 Leave blank normally.
42587 Enter a string value. Press Enter for the default ("").
42588 client_secret>
42589 Edit advanced config? (y/n)
42590 y) Yes
42591 n) No (default)
42592 y/n> n
42593 Remote config
42594 Use auto config?
42595 * Say Y if not sure
42596 * Say N if you are working on a remote or headless machine
42597 y) Yes (default)
42598 n) No
42599 y/n>
42600 If your browser doesn't open automatically go to the following link: http://127.0.0.1:53682/auth?state=LVn0IHzxej1ZkmQw31d0wQ
42601 Log in and authorize rclone for access
42602 Waiting for code...
42603 Got code
42604 Choose a number from below, or type in your own value
42605 1 / MyTeam
42606 \ "4u28602177065ff22426787a6745dba8954eb"
42607 Enter a Team ID> 1
42608 Choose a number from below, or type in your own value
42609 1 / General
42610 \ "4u2869d2aa6fca04f4f2f896b6539243b85b1"
42611 Enter a Workspace ID> 1
42612 --------------------
42613 [remote]
42614 type = zoho
42615 token = {"access_token":"xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx","token_type":"Zoho-oauthtoken","refresh_token":"xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx","expiry":"2020-10-12T00:54:52.370275223+02:00"}
42616 root_folder_id = xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
42617 --------------------
42618 y) Yes this is OK (default)
42619 e) Edit this remote
42620 d) Delete this remote
42621 y/e/d>
42622 See the remote setup docs (https://rclone.org/remote_setup/) for how to
42624 set it up on a machine with no Internet browser available.
42625 Rclone runs a webserver on your local computer to collect the autho‐
42627 rization token from Zoho Workdrive. This is only from the moment your
42628 browser is opened until the token is returned. The webserver runs on
42629 http://127.0.0.1:53682/. If local port 53682 is protected by a fire‐
42630 wall you may need to temporarily unblock the firewall to complete au‐
42631 thorization.
42632 Once configured you can then use rclone like this,
42634 See top level directories
42636 rclone lsd remote:
42638 Make a new directory
42640 rclone mkdir remote:directory
42642 List the contents of a directory
42644 rclone ls remote:directory
42646 Sync /home/local/directory to the remote path, deleting any excess
42648 files in the path.
42649 rclone sync -i /home/local/directory remote:directory
42651 Zoho paths may be as deep as required, eg remote:directory/subdirecto‐
42653 ry.
42654 Modified time
42656 Modified times are currently not supported for Zoho Workdrive
42657 Checksums
42659 No checksums are supported.
42660 Usage information
42662 To view your current quota you can use the rclone about remote: command
42663 which will display your current usage.
42664 Restricted filename characters
42666 Only control characters and invalid UTF-8 are replaced. In addition
42667 most Unicode full-width characters are not supported at all and will be
42668 removed from filenames during upload.
42669 Standard options
42671 Here are the Standard options specific to zoho (Zoho).
42672 --zoho-client-id
42674 OAuth Client Id.
42675 Leave blank normally.
42677 Properties:
42679 • Config: client_id
42681 • Env Var: RCLONE_ZOHO_CLIENT_ID
42683 • Type: string
42685 • Required: false
42687 --zoho-client-secret
42689 OAuth Client Secret.
42690 Leave blank normally.
42692 Properties:
42694 • Config: client_secret
42696 • Env Var: RCLONE_ZOHO_CLIENT_SECRET
42698 • Type: string
42700 • Required: false
42702 --zoho-region
42704 Zoho region to connect to.
42705 You'll have to use the region your organization is registered in. If
42707 not sure use the same top level domain as you connect to in your brows‐
42708 er.
42709 Properties:
42711 • Config: region
42713 • Env Var: RCLONE_ZOHO_REGION
42715 • Type: string
42717 • Required: false
42719 • Examples:
42721 • "com"
42723 • United states / Global
42725 • "eu"
42727 • Europe
42729 • "in"
42731 • India
42733 • "jp"
42735 • Japan
42737 • "com.cn"
42739 • China
42741 • "com.au"
42743 • Australia
42745 Advanced options
42747 Here are the Advanced options specific to zoho (Zoho).
42748 --zoho-token
42750 OAuth Access Token as a JSON blob.
42751 Properties:
42753 • Config: token
42755 • Env Var: RCLONE_ZOHO_TOKEN
42757 • Type: string
42759 • Required: false
42761 --zoho-auth-url
42763 Auth server URL.
42764 Leave blank to use the provider defaults.
42766 Properties:
42768 • Config: auth_url
42770 • Env Var: RCLONE_ZOHO_AUTH_URL
42772 • Type: string
42774 • Required: false
42776 --zoho-token-url
42778 Token server url.
42779 Leave blank to use the provider defaults.
42781 Properties:
42783 • Config: token_url
42785 • Env Var: RCLONE_ZOHO_TOKEN_URL
42787 • Type: string
42789 • Required: false
42791 --zoho-encoding
42793 The encoding for the backend.
42794 See the encoding section in the overview (https://rclone.org/over‐
42796 view/#encoding) for more info.
42797 Properties:
42799 • Config: encoding
42801 • Env Var: RCLONE_ZOHO_ENCODING
42803 • Type: MultiEncoder
42805 • Default: Del,Ctl,InvalidUtf8
42807 Setting up your own client_id
42809 For Zoho we advise you to set up your own client_id. To do so you have
42810 to complete the following steps.
42811 1. Log in to the Zoho API Console (https://api-console.zoho.com)
42813 2. Create a new client of type "Server-based Application". The name
42815 and website don't matter, but you must add the redirect URL
42816 http://localhost:53682/.
42817 3. Once the client is created, you can go to the settings tab and en‐
42819 able it in other regions.
42820 The client id and client secret can now be used with rclone.
42822
42824 Local paths are specified as normal filesystem paths, e.g.
42825 /path/to/wherever, so
42826 rclone sync -i /home/source /tmp/destination
42828 Will sync /home/source to /tmp/destination.
42830 Configuration
42832 For consistencies sake one can also configure a remote of type local in
42833 the config file, and access the local filesystem using rclone remote
42834 paths, e.g. remote:path/to/wherever, but it is probably easier not to.
42835 Modified time
42837 Rclone reads and writes the modified time using an accuracy determined
42838 by the OS. Typically this is 1ns on Linux, 10 ns on Windows and 1 Sec‐
42839 ond on OS X.
42840 Filenames
42842 Filenames should be encoded in UTF-8 on disk. This is the normal case
42843 for Windows and OS X.
42844 There is a bit more uncertainty in the Linux world, but new distribu‐
42846 tions will have UTF-8 encoded files names. If you are using an old
42847 Linux filesystem with non UTF-8 file names (e.g. latin1) then you can
42848 use the convmv tool to convert the filesystem to UTF-8. This tool is
42849 available in most distributions' package managers.
42850 If an invalid (non-UTF8) filename is read, the invalid characters will
42852 be replaced with a quoted representation of the invalid bytes. The
42853 name gro\xdf will be transferred as gro‛DF. rclone will emit a debug
42854 message in this case (use -v to see), e.g.
42855 Local file system at .: Replacing invalid UTF-8 characters in "gro\xdf"
42857 Restricted characters
42859 With the local backend, restrictions on the characters that are usable
42860 in file or directory names depend on the operating system. To check
42861 what rclone will replace by default on your system, run rclone help
42862 flags local-encoding.
42863 On non Windows platforms the following characters are replaced when
42865 handling file names.
42866 Character Value Replacement
42868 ────────────────────────────────
42869 NUL 0x00 ␀
42870 / 0x2F ／
42871 When running on Windows the following characters are replaced. This
42873 list is based on the Windows file naming conventions (https://docs.mi‐
42874 crosoft.com/de-de/windows/desktop/FileIO/naming-a-file#naming-conven‐
42875 tions).
42876 Character Value Replacement
42878 ────────────────────────────────
42879 NUL 0x00 ␀
42880 SOH 0x01 ␁
42881 STX 0x02 ␂
42882 ETX 0x03 ␃
42883 EOT 0x04 ␄
42884 ENQ 0x05 ␅
42885 ACK 0x06 ␆
42886 BEL 0x07 ␇
42887 BS 0x08 ␈
42888 HT 0x09 ␉
42889 LF 0x0A ␊
42890 VT 0x0B ␋
42891 FF 0x0C ␌
42892 CR 0x0D ␍
42893 SO 0x0E ␎
42894 SI 0x0F ␏
42895 DLE 0x10 ␐
42896 DC1 0x11 ␑
42897 DC2 0x12 ␒
42898 DC3 0x13 ␓
42899 DC4 0x14 ␔
42900 NAK 0x15 ␕
42901 SYN 0x16 ␖
42902 ETB 0x17 ␗
42903 CAN 0x18 ␘
42904 EM 0x19 ␙
42905 SUB 0x1A ␚
42906 ESC 0x1B ␛
42907 FS 0x1C ␜
42908 GS 0x1D ␝
42909 RS 0x1E ␞
42910 US 0x1F ␟
42911 / 0x2F ／
42912 " 0x22 ＂
42913 * 0x2A ＊
42914 : 0x3A ：
42915 < 0x3C ＜
42916 > 0x3E ＞
42917 ? 0x3F ？
42918 \ 0x5C ＼
42919 | 0x7C ｜
42920 File names on Windows can also not end with the following characters.
42922 These only get replaced if they are the last character in the name:
42923 Character Value Replacement
42925 ────────────────────────────────
42926 SP 0x20 ␠
42927 . 0x2E ．
42928 Invalid UTF-8 bytes will also be replaced (https://rclone.org/over‐
42930 view/#invalid-utf8), as they can't be converted to UTF-16.
42931 Paths on Windows
42933 On Windows there are many ways of specifying a path to a file system
42934 resource. Local paths can be absolute, like C:\path\to\wherever, or
42935 relative, like ..\wherever. Network paths in UNC format, \\serv‐
42936 er\share, are also supported. Path separator can be either \ (as in
42937 C:\path\to\wherever) or / (as in C:/path/to/wherever). Length of these
42938 paths are limited to 259 characters for files and 247 characters for
42939 directories, but there is an alternative extended-length path format
42940 increasing the limit to (approximately) 32,767 characters. This format
42941 requires absolute paths and the use of prefix \\?\, e.g.
42942 \\?\D:\some\very\long\path. For convenience rclone will automatically
42943 convert regular paths into the corresponding extended-length paths, so
42944 in most cases you do not have to worry about this (read more below).
42945 Note that Windows supports using the same prefix \\?\ to specify path
42947 to volumes identified by their GUID, e.g. \\?\Vol‐
42948 ume{b75e2c83-0000-0000-0000-602f00000000}\some\path. This is not sup‐
42949 ported in rclone, due to an issue (https://github.com/golang/go/is‐
42950 sues/39785) in go.
42951 Long paths
42953 Rclone handles long paths automatically, by converting all paths to ex‐
42954 tended-length path format (https://docs.microsoft.com/en-us/win‐
42955 dows/win32/fileio/maximum-file-path-limitation), which allows paths up
42956 to 32,767 characters.
42957 This conversion will ensure paths are absolute and prefix them with the
42959 \\?\. This is why you will see that your paths, for instance .\files
42960 is shown as path \\?\C:\files in the output, and \\server\share as
42961 \\?\UNC\server\share.
42962 However, in rare cases this may cause problems with buggy file system
42964 drivers like EncFS (https://github.com/rclone/rclone/issues/261). To
42965 disable UNC conversion globally, add this to your .rclone.conf file:
42966 [local]
42968 nounc = true
42969 If you want to selectively disable UNC, you can add it to a separate
42971 entry like this:
42972 [nounc]
42974 type = local
42975 nounc = true
42976 And use rclone like this:
42978 rclone copy c:\src nounc:z:\dst
42980 This will use UNC paths on c:\src but not on z:\dst. Of course this
42982 will cause problems if the absolute path length of a file exceeds 259
42983 characters on z, so only use this option if you have to.
42984 Symlinks / Junction points
42986 Normally rclone will ignore symlinks or junction points (which behave
42987 like symlinks under Windows).
42988 If you supply --copy-links or -L then rclone will follow the symlink
42990 and copy the pointed to file or directory. Note that this flag is in‐
42991 compatible with --links / -l.
42992 This flag applies to all commands.
42994 For example, supposing you have a directory structure like this
42996 $ tree /tmp/a
42998 /tmp/a
42999 ├── b -> ../b
43000 ├── expected -> ../expected
43001 ├── one
43002 └── two
43003 └── three
43004 Then you can see the difference with and without the flag like this
43006 $ rclone ls /tmp/a
43008 6 one
43009 6 two/three
43010 and
43012 $ rclone -L ls /tmp/a
43014 4174 expected
43015 6 one
43016 6 two/three
43017 6 b/two
43018 6 b/one
43019 --links, -l
43021 Normally rclone will ignore symlinks or junction points (which behave
43022 like symlinks under Windows).
43023 If you supply this flag then rclone will copy symbolic links from the
43025 local storage, and store them as text files, with a '.rclonelink' suf‐
43026 fix in the remote storage.
43027 The text file will contain the target of the symbolic link (see exam‐
43029 ple).
43030 This flag applies to all commands.
43032 For example, supposing you have a directory structure like this
43034 $ tree /tmp/a
43036 /tmp/a
43037 ├── file1 -> ./file4
43038 └── file2 -> /home/user/file3
43039 Copying the entire directory with '-l'
43041 $ rclone copyto -l /tmp/a/file1 remote:/tmp/a/
43043 The remote files are created with a '.rclonelink' suffix
43045 $ rclone ls remote:/tmp/a
43047 5 file1.rclonelink
43048 14 file2.rclonelink
43049 The remote files will contain the target of the symbolic links
43051 $ rclone cat remote:/tmp/a/file1.rclonelink
43053 ./file4
43054 $ rclone cat remote:/tmp/a/file2.rclonelink
43056 /home/user/file3
43057 Copying them back with '-l'
43059 $ rclone copyto -l remote:/tmp/a/ /tmp/b/
43061 $ tree /tmp/b
43063 /tmp/b
43064 ├── file1 -> ./file4
43065 └── file2 -> /home/user/file3
43066 However, if copied back without '-l'
43068 $ rclone copyto remote:/tmp/a/ /tmp/b/
43070 $ tree /tmp/b
43072 /tmp/b
43073 ├── file1.rclonelink
43074 └── file2.rclonelink
43075 Note that this flag is incompatible with -copy-links / -L.
43077 Restricting filesystems with --one-file-system
43079 Normally rclone will recurse through filesystems as mounted.
43080 However if you set --one-file-system or -x this tells rclone to stay in
43082 the filesystem specified by the root and not to recurse into different
43083 file systems.
43084 For example if you have a directory hierarchy like this
43086 root
43088 ├── disk1 - disk1 mounted on the root
43089 │ └── file3 - stored on disk1
43090 ├── disk2 - disk2 mounted on the root
43091 │ └── file4 - stored on disk12
43092 ├── file1 - stored on the root disk
43093 └── file2 - stored on the root disk
43094 Using rclone --one-file-system copy root remote: will only copy file1
43096 and file2. Eg
43097 $ rclone -q --one-file-system ls root
43099 0 file1
43100 0 file2
43101 $ rclone -q ls root
43103 0 disk1/file3
43104 0 disk2/file4
43105 0 file1
43106 0 file2
43107 NB Rclone (like most unix tools such as du, rsync and tar) treats a
43109 bind mount to the same device as being on the same filesystem.
43110 NB This flag is only available on Unix based systems. On systems where
43112 it isn't supported (e.g. Windows) it will be ignored.
43113 Advanced options
43115 Here are the Advanced options specific to local (Local Disk).
43116 --local-nounc
43118 Disable UNC (long path names) conversion on Windows.
43119 Properties:
43121 • Config: nounc
43123 • Env Var: RCLONE_LOCAL_NOUNC
43125 • Type: bool
43127 • Default: false
43129 • Examples:
43131 • "true"
43133 • Disables long file names.
43135 --copy-links / -L
43137 Follow symlinks and copy the pointed to item.
43138 Properties:
43140 • Config: copy_links
43142 • Env Var: RCLONE_LOCAL_COPY_LINKS
43144 • Type: bool
43146 • Default: false
43148 --links / -l
43150 Translate symlinks to/from regular files with a '.rclonelink' exten‐
43151 sion.
43152 Properties:
43154 • Config: links
43156 • Env Var: RCLONE_LOCAL_LINKS
43158 • Type: bool
43160 • Default: false
43162 --skip-links
43164 Don't warn about skipped symlinks.
43165 This flag disables warning messages on skipped symlinks or junction
43167 points, as you explicitly acknowledge that they should be skipped.
43168 Properties:
43170 • Config: skip_links
43172 • Env Var: RCLONE_LOCAL_SKIP_LINKS
43174 • Type: bool
43176 • Default: false
43178 --local-zero-size-links
43180 Assume the Stat size of links is zero (and read them instead) (depre‐
43181 cated).
43182 Rclone used to use the Stat size of links as the link size, but this
43184 fails in quite a few places:
43185 • Windows
43187 • On some virtual filesystems (such ash LucidLink)
43189 • Android
43191 So rclone now always reads the link.
43193 Properties:
43195 • Config: zero_size_links
43197 • Env Var: RCLONE_LOCAL_ZERO_SIZE_LINKS
43199 • Type: bool
43201 • Default: false
43203 --local-unicode-normalization
43205 Apply unicode NFC normalization to paths and filenames.
43206 This flag can be used to normalize file names into unicode NFC form
43208 that are read from the local filesystem.
43209 Rclone does not normally touch the encoding of file names it reads from
43211 the file system.
43212 This can be useful when using macOS as it normally provides decomposed
43214 (NFD) unicode which in some language (eg Korean) doesn't display prop‐
43215 erly on some OSes.
43216 Note that rclone compares filenames with unicode normalization in the
43218 sync routine so this flag shouldn't normally be used.
43219 Properties:
43221 • Config: unicode_normalization
43223 • Env Var: RCLONE_LOCAL_UNICODE_NORMALIZATION
43225 • Type: bool
43227 • Default: false
43229 --local-no-check-updated
43231 Don't check to see if the files change during upload.
43232 Normally rclone checks the size and modification time of files as they
43234 are being uploaded and aborts with a message which starts "can't copy -
43235 source file is being updated" if the file changes during upload.
43236 However on some file systems this modification time check may fail
43238 (e.g. Glusterfs #2206 (https://github.com/rclone/rclone/issues/2206))
43239 so this check can be disabled with this flag.
43240 If this flag is set, rclone will use its best efforts to transfer a
43242 file which is being updated. If the file is only having things append‐
43243 ed to it (e.g. a log) then rclone will transfer the log file with the
43244 size it had the first time rclone saw it.
43245 If the file is being modified throughout (not just appended to) then
43247 the transfer may fail with a hash check failure.
43248 In detail, once the file has had stat() called on it for the first time
43250 we:
43251 • Only transfer the size that stat gave
43253 • Only checksum the size that stat gave
43255 • Don't update the stat info for the file
43257 Properties:
43259 • Config: no_check_updated
43261 • Env Var: RCLONE_LOCAL_NO_CHECK_UPDATED
43263 • Type: bool
43265 • Default: false
43267 --one-file-system / -x
43269 Don't cross filesystem boundaries (unix/macOS only).
43270 Properties:
43272 • Config: one_file_system
43274 • Env Var: RCLONE_LOCAL_ONE_FILE_SYSTEM
43276 • Type: bool
43278 • Default: false
43280 --local-case-sensitive
43282 Force the filesystem to report itself as case sensitive.
43283 Normally the local backend declares itself as case insensitive on Win‐
43285 dows/macOS and case sensitive for everything else. Use this flag to
43286 override the default choice.
43287 Properties:
43289 • Config: case_sensitive
43291 • Env Var: RCLONE_LOCAL_CASE_SENSITIVE
43293 • Type: bool
43295 • Default: false
43297 --local-case-insensitive
43299 Force the filesystem to report itself as case insensitive.
43300 Normally the local backend declares itself as case insensitive on Win‐
43302 dows/macOS and case sensitive for everything else. Use this flag to
43303 override the default choice.
43304 Properties:
43306 • Config: case_insensitive
43308 • Env Var: RCLONE_LOCAL_CASE_INSENSITIVE
43310 • Type: bool
43312 • Default: false
43314 --local-no-preallocate
43316 Disable preallocation of disk space for transferred files.
43317 Preallocation of disk space helps prevent filesystem fragmentation.
43319 However, some virtual filesystem layers (such as Google Drive File
43320 Stream) may incorrectly set the actual file size equal to the preallo‐
43321 cated space, causing checksum and file size checks to fail. Use this
43322 flag to disable preallocation.
43323 Properties:
43325 • Config: no_preallocate
43327 • Env Var: RCLONE_LOCAL_NO_PREALLOCATE
43329 • Type: bool
43331 • Default: false
43333 --local-no-sparse
43335 Disable sparse files for multi-thread downloads.
43336 On Windows platforms rclone will make sparse files when doing multi-
43338 thread downloads. This avoids long pauses on large files where the OS
43339 zeros the file. However sparse files may be undesirable as they cause
43340 disk fragmentation and can be slow to work with.
43341 Properties:
43343 • Config: no_sparse
43345 • Env Var: RCLONE_LOCAL_NO_SPARSE
43347 • Type: bool
43349 • Default: false
43351 --local-no-set-modtime
43353 Disable setting modtime.
43354 Normally rclone updates modification time of files after they are done
43356 uploading. This can cause permissions issues on Linux platforms when
43357 the user rclone is running as does not own the file uploaded, such as
43358 when copying to a CIFS mount owned by another user. If this option is
43359 enabled, rclone will no longer update the modtime after copying a file.
43360 Properties:
43362 • Config: no_set_modtime
43364 • Env Var: RCLONE_LOCAL_NO_SET_MODTIME
43366 • Type: bool
43368 • Default: false
43370 --local-encoding
43372 The encoding for the backend.
43373 See the encoding section in the overview (https://rclone.org/over‐
43375 view/#encoding) for more info.
43376 Properties:
43378 • Config: encoding
43380 • Env Var: RCLONE_LOCAL_ENCODING
43382 • Type: MultiEncoder
43384 • Default: Slash,Dot
43386 Metadata
43388 Depending on which OS is in use the local backend may return only some
43389 of the system metadata. Setting system metadata is supported on all
43390 OSes but setting user metadata is only supported on linux, freebsd,
43391 netbsd, macOS and Solaris. It is not supported on Windows yet (see
43392 pkg/attrs#47 (https://github.com/pkg/xattr/issues/47)).
43393 User metadata is stored as extended attributes (which may not be sup‐
43395 ported by all file systems) under the "user.*" prefix.
43396 Here are the possible system metadata items for the local backend.
43398 Name Help Type Example Read Only
43400 ──────────────────────────────────────────────────────────────────────────────────────────────────────
43401 atime Time of RFC 3339 2006-01-02T15:04:05.999999999Z07:00 N
43402 last access
43403 btime Time of RFC 3339 2006-01-02T15:04:05.999999999Z07:00 N
43404 file birth
43405 (creation)
43406 gid Group ID of decimal 500 N
43407 owner number
43408 mode File type octal, unix 0100664 N
43409 and mode style
43410 mtime Time of RFC 3339 2006-01-02T15:04:05.999999999Z07:00 N
43411 last modi‐
43412 fication
43413 rdev Device ID hexadecimal 1abc N
43414 (if special
43415 file)
43416 uid User ID of decimal 500 N
43417 owner number
43418 See the metadata (https://rclone.org/docs/#metadata) docs for more in‐
43420 fo.
43421 Backend commands
43423 Here are the commands specific to the local backend.
43424 Run them with
43426 rclone backend COMMAND remote:
43428 The help below will explain what arguments each command takes.
43430 See the backend (https://rclone.org/commands/rclone_backend/) command
43432 for more info on how to pass options and arguments.
43433 These can be run on a running backend using the rc command backend/com‐
43435 mand (https://rclone.org/rc/#backend-command).
43436 noop
43438 A null operation for testing backend commands
43439 rclone backend noop remote: [options] [<arguments>+]
43441 This is a test command which has some options you can try to change the
43443 output.
43444 Options:
43446 • "echo": echo the input arguments
43448 • "error": return an error based on option value
43450
43452 v1.60.1 - 2022-11-17
43453 See commits (https://github.com/rclone/rclone/com‐
43454 pare/v1.60.0...v1.60.1)
43455 • Bug Fixes
43457 • lib/cache: Fix alias backend shutting down too soon (Nick Craig-
43459 Wood)
43460 • wasm: Fix walltime link error by adding up-to-date wasm_exec.js
43462 (João Henrique Franco)
43463 • docs
43465 • Update faq.md with bisync (Samuel Johnson)
43467 • Corrected download links in windows install docs (coultonluke)
43469 • Add direct download link for windows arm64 (albertony)
43471 • Remove link to rclone slack as it is no longer supported (Nick
43473 Craig-Wood)
43474 • Faq: how to use a proxy server that requires a username and pass‐
43476 word (asdffdsazqqq)
43477 • Oracle-object-storage: doc fix (Manoj Ghosh)
43479 • Fix typo remove in rclone_serve_restic command (Joda Stößer)
43481 • Fix character that was incorrectly interpreted as markdown
43483 (Clément Notin)
43484 • VFS
43486 • Fix deadlock caused by cache cleaner and upload finishing (Nick
43488 Craig-Wood)
43489 • Local
43491 • Clean absolute paths (albertony)
43493 • Fix -L/--copy-links with filters missing directories (Nick Craig-
43495 Wood)
43496 • Mailru
43498 • Note that an app password is now needed (Nick Craig-Wood)
43500 • Allow timestamps to be before the epoch 1970-01-01 (Nick Craig-
43502 Wood)
43503 • S3
43505 • Add provider quirk --s3-might-gzip to fix corrupted on transfer:
43507 sizes differ (Nick Craig-Wood)
43508 • Allow Storj to server side copy since it seems to work now (Nick
43510 Craig-Wood)
43511 • Fix for unchecked err value in s3 listv2 (Aaron Gokaslan)
43513 • Add additional Wasabi locations (techknowlogick)
43515 • Smb
43517 • Fix Failed to sync: context canceled at the end of syncs (Nick
43519 Craig-Wood)
43520 • WebDAV
43522 • Fix Move/Copy/DirMove when using -server-side-across-configs (Nick
43524 Craig-Wood)
43525 v1.60.0 - 2022-10-21
43527 See commits (https://github.com/rclone/rclone/com‐
43528 pare/v1.59.0...v1.60.0)
43529 • New backends
43531 • Oracle object storage (https://rclone.org/oracleobjectstorage/)
43533 (Manoj Ghosh)
43534 • SMB (https://rclone.org/smb/) / CIFS (Windows file sharing)
43536 (Lesmiscore)
43537 • New S3 providers
43539 • IONOS Cloud Storage (https://rclone.org/s3/#ionos) (Dmitry
43541 Deniskin)
43542 • Qiniu KODO (https://rclone.org/s3/#qiniu) (Bachue Zhou)
43544 • New Features
43546 • build
43548 • Update to go1.19 and make go1.17 the minimum required version
43550 (Nick Craig-Wood)
43551 • Install.sh: fix arm-v7 download (Ole Frost)
43553 • fs: Warn the user when using an existing remote name without a
43555 colon (Nick Craig-Wood)
43556 • httplib: Add --xxx-min-tls-version option to select minimum TLS
43558 version for HTTP servers (Robert Newson)
43559 • librclone: Add PHP bindings and test program (Jordi Gonzalez Muñoz)
43561 • operations
43563 • Add --server-side-across-configs global flag for any backend
43565 (Nick Craig-Wood)
43566 • Optimise --copy-dest and --compare-dest (Nick Craig-Wood)
43568 • rc: add job/stopgroup to stop group (Evan Spensley)
43570 • serve dlna
43572 • Add --announce-interval to control SSDP Announce Interval (Yanc‐
43574 eyChiew)
43575 • Add --interface to Specify SSDP interface names line (Simon Bos)
43577 • Add support for more external subtitles (YanceyChiew)
43579 • Add verification of addresses (YanceyChiew)
43581 • sync: Optimise --copy-dest and --compare-dest (Nick Craig-Wood)
43583 • doc updates (albertony, Alexander Knorr, anonion, João Henrique
43585 Franco, Josh Soref, Lorenzo Milesi, Marco Molteni, Mark Trolley,
43586 Ole Frost, partev, Ryan Morey, Tom Mombourquette, YFdyh000)
43587 • Bug Fixes
43589 • filter
43591 • Fix incorrect filtering with UseFilter context flag and wrapping
43593 backends (Nick Craig-Wood)
43594 • Make sure we check --files-from when looking for a single file
43596 (Nick Craig-Wood)
43597 • rc
43599 • Fix mount/listmounts not returning the full Fs entered in
43601 mount/mount (Tom Mombourquette)
43602 • Handle external unmount when mounting (Isaac Aymerich)
43604 • Validate Daemon option is not set when mounting a volume via RC
43606 (Isaac Aymerich)
43607 • sync: Update docs and error messages to reflect fixes to overlap
43609 checks (Nick Naumann)
43610 • VFS
43612 • Reduce memory use by embedding sync.Cond (Nick Craig-Wood)
43614 • Reduce memory usage by re-ordering commonly used structures (Nick
43616 Craig-Wood)
43617 • Fix excess CPU used by VFS cache cleaner looping (Nick Craig-Wood)
43619 • Local
43621 • Obey file filters in listing to fix errors on excluded files (Nick
43623 Craig-Wood)
43624 • Fix "Failed to read metadata: function not implemented" on old Lin‐
43626 ux kernels (Nick Craig-Wood)
43627 • Compress
43629 • Fix crash due to nil metadata (Nick Craig-Wood)
43631 • Fix error handling to not use or return nil objects (Nick Craig-
43633 Wood)
43634 • Drive
43636 • Make --drive-stop-on-upload-limit obey quota exceeded error (Steve
43638 Kowalik)
43639 • FTP
43641 • Add --ftp-force-list-hidden option to show hidden items (Øyvind
43643 Heddeland Instefjord)
43644 • Fix hang when using ExplicitTLS to certain servers. (Nick Craig-
43646 Wood)
43647 • Google Cloud Storage
43649 • Add --gcs-endpoint flag and config parameter (Nick Craig-Wood)
43651 • Hubic
43653 • Remove backend as service has now shut down (Nick Craig-Wood)
43655 • Onedrive
43657 • Rename Onedrive(cn) 21Vianet to Vnet Group (Yen Hu)
43659 • Disable change notify in China region since it is not supported
43661 (Nick Craig-Wood)
43662 • S3
43664 • Implement --s3-versions flag to show old versions of objects if en‐
43666 abled (Nick Craig-Wood)
43667 • Implement --s3-version-at flag to show versions of objects at a
43669 particular time (Nick Craig-Wood)
43670 • Implement backend versioning command to get/set bucket versioning
43672 (Nick Craig-Wood)
43673 • Implement Purge to purge versions and backend cleanup-hidden (Nick
43675 Craig-Wood)
43676 • Add --s3-decompress flag to decompress gzip-encoded files (Nick
43678 Craig-Wood)
43679 • Add --s3-sse-customer-key-base64 to supply keys with binary data
43681 (Richard Bateman)
43682 • Try to keep the maximum precision in ModTime with --user-server-
43684 modtime (Nick Craig-Wood)
43685 • Drop binary metadata with an ERROR message as it can't be stored
43687 (Nick Craig-Wood)
43688 • Add --s3-no-system-metadata to suppress read and write of system
43690 metadata (Nick Craig-Wood)
43691 • SFTP
43693 • Fix directory creation races (Lesmiscore)
43695 • Swift
43697 • Add --swift-no-large-objects to reduce HEAD requests (Nick Craig-
43699 Wood)
43700 • Union
43702 • Propagate SlowHash feature to fix hasher interaction (Lesmiscore)
43704 v1.59.2 - 2022-09-15
43706 See commits (https://github.com/rclone/rclone/com‐
43707 pare/v1.59.1...v1.59.2)
43708 • Bug Fixes
43710 • config: Move locking to fix fatal error: concurrent map read and
43712 map write (Nick Craig-Wood)
43713 • Local
43715 • Disable xattr support if the filesystems indicates it is not sup‐
43717 ported (Nick Craig-Wood)
43718 • Azure Blob
43720 • Fix chunksize calculations producing too many parts (Nick Craig-
43722 Wood)
43723 • B2
43725 • Fix chunksize calculations producing too many parts (Nick Craig-
43727 Wood)
43728 • S3
43730 • Fix chunksize calculations producing too many parts (Nick Craig-
43732 Wood)
43733 v1.59.1 - 2022-08-08
43735 See commits (https://github.com/rclone/rclone/com‐
43736 pare/v1.59.0...v1.59.1)
43737 • Bug Fixes
43739 • accounting: Fix panic in core/stats-reset with unknown group (Nick
43741 Craig-Wood)
43742 • build: Fix android build after GitHub actions change (Nick Craig-
43744 Wood)
43745 • dlna: Fix SOAP action header parsing (Joram Schrijver)
43747 • docs: Fix links to mount command from install docs (albertony)
43749 • dropbox: Fix ChangeNotify was unable to decrypt errors (Nick Craig-
43751 Wood)
43752 • fs: Fix parsing of times and durations of the form "YYYY-MM-DD
43754 HH:MM:SS" (Nick Craig-Wood)
43755 • serve sftp: Fix checksum detection (Nick Craig-Wood)
43757 • sync: Add accidentally missed filter-sensitivity to --backup-dir
43759 option (Nick Naumann)
43760 • Combine
43762 • Fix docs showing remote= instead of upstreams= (Nick Craig-Wood)
43764 • Throw error if duplicate directory name is specified (Nick Craig-
43766 Wood)
43767 • Fix errors with backends shutting down while in use (Nick Craig-
43769 Wood)
43770 • Dropbox
43772 • Fix hang on quit with --dropbox-batch-mode off (Nick Craig-Wood)
43774 • Fix infinite loop on uploading a corrupted file (Nick Craig-Wood)
43776 • Internetarchive
43778 • Ignore checksums for files using the different method (Lesmiscore)
43780 • Handle hash symbol in the middle of filename (Lesmiscore)
43782 • Jottacloud
43784 • Fix working with whitelabel Elgiganten Cloud
43786 • Do not store username in config when using standard auth (al‐
43788 bertony)
43789 • Mega
43791 • Fix nil pointer exception when bad node received (Nick Craig-Wood)
43793 • S3
43795 • Fix --s3-no-head panic: reflect: Elem of invalid type s3.PutObject‐
43797 Input (Nick Craig-Wood)
43798 • SFTP
43800 • Fix issue with WS_FTP by working around failing RealPath (al‐
43802 bertony)
43803 • Union
43805 • Fix duplicated files when using directories with leading / (Nick
43807 Craig-Wood)
43808 • Fix multiple files being uploaded when roots don't exist (Nick
43810 Craig-Wood)
43811 • Fix panic due to misalignment of struct field in 32 bit architec‐
43813 tures (r-ricci)
43814 v1.59.0 - 2022-07-09
43816 See commits (https://github.com/rclone/rclone/com‐
43817 pare/v1.58.0...v1.59.0)
43818 • New backends
43820 • Combine multiple remotes in one directory tree (Nick Craig-Wood)
43822 • Hidrive (https://rclone.org/hidrive/) (Ovidiu Victor Tatar)
43824 • Internet Archive (https://rclone.org/internetarchive/) (Lesmiscore
43826 (Naoya Ozaki))
43827 • New S3 providers
43829 • ArvanCloud AOS (https://rclone.org/s3/#arvan-cloud) (ehsantdy)
43831 • Cloudflare R2 (https://rclone.org/s3/#cloudflare-r2) (Nick Craig-
43833 Wood)
43834 • Huawei OBS (https://rclone.org/s3/#huawei-obs) (m00594701)
43836 • IDrive e2 (https://rclone.org/s3/#idrive-e2) (vyloy)
43838 • New commands
43840 • test makefile (https://rclone.org/commands/rclone_test_makefile/):
43842 Create a single file for testing (Nick Craig-Wood)
43843 • New Features
43845 • Metadata framework (https://rclone.org/docs/#metadata) to read and
43847 write system and user metadata on backends (Nick Craig-Wood)
43848 • Implemented initially for local, s3 and internetarchive backends
43850 • --metadata/-M flag to control whether metadata is copied
43852 • --metadata-set flag to specify metadata for uploads
43854 • Thanks to Manz Solutions (https://manz-solutions.at/) for spon‐
43856 soring this work.
43857 • build
43859 • Update to go1.18 and make go1.16 the minimum required version
43861 (Nick Craig-Wood)
43862 • Update android go build to 1.18.x and NDK to 23.1.7779620 (Nick
43864 Craig-Wood)
43865 • All windows binaries now no longer CGO (Nick Craig-Wood)
43867 • Add linux/arm/v6 to docker images (Nick Craig-Wood)
43869 • A huge number of fixes found with staticcheck (https://static‐
43871 check.io/) (albertony)
43872 • Configurable version suffix independent of version number (al‐
43874 bertony)
43875 • check: Implement --no-traverse and --no-unicode-normalization (Nick
43877 Craig-Wood)
43878 • config: Readability improvements (albertony)
43880 • copyurl: Add --header-filename to honor the HTTP header filename
43882 directive (J-P Treen)
43883 • filter: Allow multiple --exclude-if-present flags (albertony)
43885 • fshttp: Add --disable-http-keep-alives to disable HTTP Keep Alives
43887 (Nick Craig-Wood)
43888 • install.sh
43890 • Set the modes on the files and/or directories on macOS (Michael C
43892 Tiernan - MIT-Research Computing Project)
43893 • Pre verify sudo authorization -v before calling curl. (Michael C
43895 Tiernan - MIT-Research Computing Project)
43896 • lib/encoder: Add Semicolon encoding (Nick Craig-Wood)
43898 • lsf: Add metadata support with M flag (Nick Craig-Wood)
43900 • lsjson: Add --metadata/-M flag (Nick Craig-Wood)
43902 • ncdu
43904 • Implement multi selection (CrossR)
43906 • Replace termbox with tcell's termbox wrapper (eNV25)
43908 • Display correct path in delete confirmation dialog (Roberto Ric‐
43910 ci)
43911 • operations
43913 • Speed up hash checking by aborting the other hash if first re‐
43915 turns nothing (Nick Craig-Wood)
43916 • Use correct src/dst in some log messages (zzr93)
43918 • rcat: Check checksums by default like copy does (Nick Craig-Wood)
43920 • selfupdate: Replace deprecated x/crypto/openpgp package with Pro‐
43922 tonMail/go-crypto (albertony)
43923 • serve ftp: Check --passive-port arguments are correct (Nick Craig-
43925 Wood)
43926 • size: Warn about inaccurate results when objects with unknown size
43928 (albertony)
43929 • sync: Overlap check is now filter-sensitive so --backup-dir can be
43931 in the root provided it is filtered (Nick)
43932 • test info: Check file name lengths using 1,2,3,4 byte unicode char‐
43934 acters (Nick Craig-Wood)
43935 • test makefile(s): --sparse, --zero, --pattern, --ascii, --chargen
43937 flags to control file contents (Nick Craig-Wood)
43938 • Make sure we call the Shutdown method on backends (Martin Czygan)
43940 • Bug Fixes
43942 • accounting: Fix unknown length file transfers counting 3 transfers
43944 each (buda)
43945 • ncdu: Fix issue where dir size is summed when file sizes are -1
43947 (albertony)
43948 • sync/copy/move
43950 • Fix --fast-list --create-empty-src-dirs and --exclude (Nick
43952 Craig-Wood)
43953 • Fix --max-duration and --cutoff-mode soft (Nick Craig-Wood)
43955 • Fix fs cache unpin (Martin Czygan)
43957 • Set proper exit code for errors that are not low-level retried
43959 (e.g. size/timestamp changing) (albertony)
43960 • Mount
43962 • Support windows/arm64 (may still be problems - see #5828
43964 (https://github.com/rclone/rclone/issues/5828)) (Nick Craig-Wood)
43965 • Log IO errors at ERROR level (Nick Craig-Wood)
43967 • Ignore _netdev mount argument (Hugal31)
43969 • VFS
43971 • Add --vfs-fast-fingerprint for less accurate but faster finger‐
43973 prints (Nick Craig-Wood)
43974 • Add --vfs-disk-space-total-size option to manually set the total
43976 disk space (Claudio Maradonna)
43977 • vfscache: Fix fatal error: sync: unlock of unlocked mutex error
43979 (Nick Craig-Wood)
43980 • Local
43982 • Fix parsing of --local-nounc flag (Nick Craig-Wood)
43984 • Add Metadata support (Nick Craig-Wood)
43986 • Crypt
43988 • Support metadata (Nick Craig-Wood)
43990 • Azure Blob
43992 • Calculate Chunksize/blocksize to stay below maxUploadParts (Leroy
43994 van Logchem)
43995 • Use chunksize lib to determine chunksize dynamically (Derek Bat‐
43997 tams)
43998 • Case insensitive access tier (Rob Pickerill)
44000 • Allow remote emulator (azurite) (Lorenzo Maiorfi)
44002 • B2
44004 • Add --b2-version-at flag to show file versions at time specified
44006 (SwazRGB)
44007 • Use chunksize lib to determine chunksize dynamically (Derek Bat‐
44009 tams)
44010 • Chunker
44012 • Mark as not supporting metadata (Nick Craig-Wood)
44014 • Compress
44016 • Support metadata (Nick Craig-Wood)
44018 • Drive
44020 • Make backend config -o config add a combined AllDrives: remote
44022 (Nick Craig-Wood)
44023 • Make --drive-shared-with-me work with shared drives (Nick Craig-
44025 Wood)
44026 • Add --drive-resource-key for accessing link-shared files (Nick
44028 Craig-Wood)
44029 • Add backend commands exportformats and importformats for debugging
44031 (Nick Craig-Wood)
44032 • Fix 404 errors on copy/server side copy objects from public folder
44034 (Nick Craig-Wood)
44035 • Update Internal OAuth consent screen docs (Phil Shackleton)
44037 • Moved root_folder_id to advanced section (Abhiraj)
44039 • Dropbox
44041 • Migrate from deprecated api (m8rge)
44043 • Add logs to show when poll interval limits are exceeded (Nick
44045 Craig-Wood)
44046 • Fix nil pointer exception on dropbox impersonate user not found
44048 (Nick Craig-Wood)
44049 • Fichier
44051 • Parse api error codes and them accordingly (buengese)
44053 • FTP
44055 • Add support for disable_utf8 option (Jason Zheng)
44057 • Revert to upstream github.com/jlaffaye/ftp from our fork (Nick
44059 Craig-Wood)
44060 • Google Cloud Storage
44062 • Add --gcs-no-check-bucket to minimise transactions and perms (Nick
44064 Gooding)
44065 • Add --gcs-decompress flag to decompress gzip-encoded files (Nick
44067 Craig-Wood)
44068 • by default these will be downloaded compressed (which previously
44070 failed)
44071 • Hasher
44073 • Support metadata (Nick Craig-Wood)
44075 • HTTP
44077 • Fix missing response when using custom auth handler (albertony)
44079 • Jottacloud
44081 • Add support for upload to custom device and mountpoint (albertony)
44083 • Always store username in config and use it to avoid initial API re‐
44085 quest (albertony)
44086 • Fix issue with server-side copy when destination is in trash (al‐
44088 bertony)
44089 • Fix listing output of remote with special characters (albertony)
44091 • Mailru
44093 • Fix timeout by using int instead of time.Duration for keeping num‐
44095 ber of seconds (albertony)
44096 • Mega
44098 • Document using MEGAcmd to help with login failures (Art M. Gal‐
44100 lagher)
44101 • Onedrive
44103 • Implement --poll-interval for onedrive (Hugo Laloge)
44105 • Add access scopes option (Sven Gerber)
44107 • Opendrive
44109 • Resolve lag and truncate bugs (Scott Grimes)
44111 • Pcloud
44113 • Fix about with no free space left (buengese)
44115 • Fix cleanup (buengese)
44117 • S3
44119 • Use PUT Object instead of presigned URLs to upload single part ob‐
44121 jects (Nick Craig-Wood)
44122 • Backend restore command to skip non-GLACIER objects (Vincent Mur‐
44124 phy)
44125 • Use chunksize lib to determine chunksize dynamically (Derek Bat‐
44127 tams)
44128 • Retry RequestTimeout errors (Nick Craig-Wood)
44130 • Implement reading and writing of metadata (Nick Craig-Wood)
44132 • SFTP
44134 • Add support for about and hashsum on windows server (albertony)
44136 • Use vendor-specific VFS statistics extension for about if available
44138 (albertony)
44139 • Add --sftp-chunk-size to control packets sizes for high latency
44141 links (Nick Craig-Wood)
44142 • Add --sftp-concurrency to improve high latency transfers (Nick
44144 Craig-Wood)
44145 • Add --sftp-set-env option to set environment variables (Nick Craig-
44147 Wood)
44148 • Add Hetzner Storage Boxes to supported sftp backends (Anthrazz)
44150 • Storj
44152 • Fix put which lead to the file being unreadable when using mount
44154 (Erik van Velzen)
44155 • Union
44157 • Add min_free_space option for lfs/eplfs policies (Nick Craig-Wood)
44159 • Fix uploading files to union of all bucket based remotes (Nick
44161 Craig-Wood)
44162 • Fix get free space for remotes which don't support it (Nick Craig-
44164 Wood)
44165 • Fix eplus policy to select correct entry for existing files (Nick
44167 Craig-Wood)
44168 • Support metadata (Nick Craig-Wood)
44170 • Uptobox
44172 • Fix root path handling (buengese)
44174 • WebDAV
44176 • Add SharePoint in other specific regions support (Noah Hsu)
44178 • Yandex
44180 • Handle api error on server-side move (albertony)
44182 • Zoho
44184 • Add Japan and China regions (buengese)
44186 v1.58.1 - 2022-04-29
44188 See commits (https://github.com/rclone/rclone/com‐
44189 pare/v1.58.0...v1.58.1)
44190 • Bug Fixes
44192 • build: Update github.com/billziss-gh to github.com/winfsp (Nick
44194 Craig-Wood)
44195 • filter: Fix timezone of --min-age/-max-age from UTC to local as
44197 documented (Nick Craig-Wood)
44198 • rc/js: Correct RC method names (Sơn Trần-Nguyễn)
44200 • docs
44202 • Fix some links to command pages (albertony)
44204 • Add --multi-thread-streams note to --transfers. (Zsolt Ero)
44206 • Mount
44208 • Fix --devname and fusermount: unknown option 'fsname' when mounting
44210 via rc (Nick Craig-Wood)
44211 • VFS
44213 • Remove wording which suggests VFS is only for mounting (Nick Craig-
44215 Wood)
44216 • Dropbox
44218 • Fix retries of multipart uploads with incorrect_offset error (Nick
44220 Craig-Wood)
44221 • Google Cloud Storage
44223 • Use the s3 pacer to speed up transactions (Nick Craig-Wood)
44225 • pacer: Default the Google pacer to a burst of 100 to fix gcs pacing
44227 (Nick Craig-Wood)
44228 • Jottacloud
44230 • Fix scope in token request (albertony)
44232 • Netstorage
44234 • Fix unescaped HTML in documentation (Nick Craig-Wood)
44236 • Make levels of headings consistent (Nick Craig-Wood)
44238 • Add support contacts to netstorage doc (Nil Alexandrov)
44240 • Onedrive
44242 • Note that sharepoint also changes web files (.html, .aspx) (GH)
44244 • Putio
44246 • Handle rate limit errors (Berkan Teber)
44248 • Fix multithread download and other ranged requests (rafma0)
44250 • S3
44252 • Add ChinaMobile EOS to provider list (GuoXingbin)
44254 • Sync providers in config description with providers (Nick Craig-
44256 Wood)
44257 • SFTP
44259 • Fix OpenSSH 8.8+ RSA keys incompatibility (KARBOWSKI Piotr)
44261 • Note that Scaleway C14 is deprecating SFTP in favor of S3 (Adrien
44263 Rey-Jarthon)
44264 • Storj
44266 • Fix bucket creation on Move (Nick Craig-Wood)
44268 • WebDAV
44270 • Don't override Referer if user sets it (Nick Craig-Wood)
44272 v1.58.0 - 2022-03-18
44274 See commits (https://github.com/rclone/rclone/com‐
44275 pare/v1.57.0...v1.58.0)
44276 • New backends
44278 • Akamai Netstorage (Nil Alexandrov)
44280 • Seagate Lyve (https://rclone.org/s3/#lyve), SeaweedFS, Storj, Rack‐
44282 Corp via s3 backend
44283 • Storj (https://rclone.org/storj/) (renamed from Tardigrade - your
44285 old config files will continue working)
44286 • New commands
44288 • bisync (https://rclone.org/bisync/) - experimental bidirectional
44290 cloud sync (Ivan Andreev, Chris Nelson)
44291 • New Features
44293 • build
44295 • Add windows/arm64 build (rclone mount not supported yet) (Nick
44297 Craig-Wood)
44298 • Raise minimum go version to go1.15 (Nick Craig-Wood)
44300 • config: Allow dot in remote names and improve config editing (al‐
44302 bertony)
44303 • dedupe: Add quit as a choice in interactive mode (albertony)
44305 • dlna: Change icons to the newest ones. (Alain Nussbaumer)
44307 • filter: Add {{ regexp }} syntax (https://rclone.org/filtering/#reg‐
44309 exp) to pattern matches (Nick Craig-Wood)
44310 • fshttp: Add prometheus metrics for HTTP status code (Michał
44312 Matczuk)
44313 • hashsum: Support creating hash from data received on stdin (al‐
44315 bertony)
44316 • librclone
44318 • Allow empty string or null input instead of empty json object
44320 (albertony)
44321 • Add support for mount commands (albertony)
44323 • operations: Add server-side moves to stats (Ole Frost)
44325 • rc: Allow user to disable authentication for web gui (negative0)
44327 • tree: Remove obsolete --human replaced by global --human-readable
44329 (albertony)
44330 • version: Report correct friendly-name for newer Windows 10/11 ver‐
44332 sions (albertony)
44333 • Bug Fixes
44335 • build
44337 • Fix ARM architecture version in .deb packages after nfpm change
44339 (Nick Craig-Wood)
44340 • Hard fork github.com/jlaffaye/ftp to fix go get
44342 github.com/rclone/rclone (Nick Craig-Wood)
44343 • oauthutil: Fix crash when webbrowser requests /robots.txt (Nick
44345 Craig-Wood)
44346 • operations: Fix goroutine leak in case of copy retry (Ankur Gupta)
44348 • rc:
44350 • Fix operations/publiclink default for expires parameter (Nick
44352 Craig-Wood)
44353 • Fix missing computation of transferQueueSize when summing up sta‐
44355 tistics group (Carlo Mion)
44356 • Fix missing StatsInfo fields in the computation of the group sum
44358 (Carlo Mion)
44359 • sync: Fix --max-duration so it doesn't retry when the duration is
44361 exceeded (Nick Craig-Wood)
44362 • touch: Fix issue where a directory is created instead of a file
44364 (albertony)
44365 • Mount
44367 • Add --devname to set the device name sent to FUSE for mount display
44369 (Nick Craig-Wood)
44370 • VFS
44372 • Add vfs/stats remote control to show statistics (Nick Craig-Wood)
44374 • Fix failed to _ensure cache internal error: downloaders is nil er‐
44376 ror (Nick Craig-Wood)
44377 • Fix handling of special characters in file names (Bumsu Hyeon)
44379 • Local
44381 • Fix hash invalidation which caused errors with local crypt mount
44383 (Nick Craig-Wood)
44384 • Crypt
44386 • Add base64 and base32768 filename encoding options (Max Sum, Sinan
44388 Tan)
44389 • Azure Blob
44391 • Implement --azureblob-upload-concurrency parameter to speed uploads
44393 (Nick Craig-Wood)
44394 • Remove 100MB upper limit on chunk_size as it is no longer needed
44396 (Nick Craig-Wood)
44397 • Raise --azureblob-upload-concurrency to 16 by default (Nick Craig-
44399 Wood)
44400 • Fix crash with SAS URL and no container (Nick Craig-Wood)
44402 • Compress
44404 • Fix crash if metadata upload failed (Nick Craig-Wood)
44406 • Fix memory leak (Nick Craig-Wood)
44408 • Drive
44410 • Added --drive-copy-shortcut-content (Abhiraj)
44412 • Disable OAuth OOB flow (copy a token) due to Google deprecation
44414 (Nick Craig-Wood)
44415 • See the deprecation note (https://developers.google‐
44417 blog.com/2022/02/making-oauth-flows-safer.html#disallowed-oob).
44418 • Add --drive-skip-dangling-shortcuts flag (Nick Craig-Wood)
44420 • When using a link type --drive-export-formats shows all doc types
44422 (Nick Craig-Wood)
44423 • Dropbox
44425 • Speed up directory listings by specifying 1000 items in a chunk
44427 (Nick Craig-Wood)
44428 • Save an API request when at the root (Nick Craig-Wood)
44430 • Fichier
44432 • Implemented About functionality (Gourav T)
44434 • FTP
44436 • Add --ftp-ask-password to prompt for password when needed (Borna
44438 Butkovic)
44439 • Google Cloud Storage
44441 • Add missing regions (Nick Craig-Wood)
44443 • Disable OAuth OOB flow (copy a token) due to Google deprecation
44445 (Nick Craig-Wood)
44446 • See the deprecation note (https://developers.google‐
44448 blog.com/2022/02/making-oauth-flows-safer.html#disallowed-oob).
44449 • Googlephotos
44451 • Disable OAuth OOB flow (copy a token) due to Google deprecation
44453 (Nick Craig-Wood)
44454 • See the deprecation note (https://developers.google‐
44456 blog.com/2022/02/making-oauth-flows-safer.html#disallowed-oob).
44457 • Hasher
44459 • Fix crash on object not found (Nick Craig-Wood)
44461 • Hdfs
44463 • Add file (Move) and directory move (DirMove) support (Andy Jackson)
44465 • HTTP
44467 • Improved recognition of URL pointing to a single file (albertony)
44469 • Jottacloud
44471 • Change API used by recursive list (ListR) (Kim)
44473 • Add support for Tele2 Cloud (Fredric Arklid)
44475 • Koofr
44477 • Add Digistorage service as a Koofr provider. (jaKa)
44479 • Mailru
44481 • Fix int32 overflow on arm32 (Ivan Andreev)
44483 • Onedrive
44485 • Add config option for oauth scope Sites.Read.All (Charlie Jiang)
44487 • Minor optimization of quickxorhash (Isaac Levy)
44489 • Add --onedrive-root-folder-id flag (Nick Craig-Wood)
44491 • Do not retry on 400 pathIsTooLong error (ctrl-q)
44493 • Pcloud
44495 • Add support for recursive list (ListR) (Niels van de Weem)
44497 • Fix pre-1970 time stamps (Nick Craig-Wood)
44499 • S3
44501 • Use ListObjectsV2 for faster listings (Felix Bünemann)
44503 • Fallback to ListObject v1 on unsupported providers (Nick Craig-
44505 Wood)
44506 • Use the ETag on multipart transfers to verify the transfer was OK
44508 (Nick Craig-Wood)
44509 • Add --s3-use-multipart-etag provider quirk to disable this on un‐
44511 supported providers (Nick Craig-Wood)
44512 • New Providers
44514 • RackCorp object storage (bbabich)
44516 • Seagate Lyve Cloud storage (Nick Craig-Wood)
44518 • SeaweedFS (Chris Lu)
44520 • Storj Shared gateways (Márton Elek, Nick Craig-Wood)
44522 • Add Wasabi AP Northeast 2 endpoint info (lindwurm)
44524 • Add GLACIER_IR storage class (Yunhai Luo)
44526 • Document Content-MD5 workaround for object-lock enabled buckets
44528 (Paulo Martins)
44529 • Fix multipart upload with --no-head flag (Nick Craig-Wood)
44531 • Simplify content length processing in s3 with download url (Logesh‐
44533 waran Murugesan)
44534 • SFTP
44536 • Add rclone to list of supported md5sum/sha1sum commands to look for
44538 (albertony)
44539 • Refactor so we only have one way of running remote commands (Nick
44541 Craig-Wood)
44542 • Fix timeout on hashing large files by sending keepalives (Nick
44544 Craig-Wood)
44545 • Fix unnecessary seeking when uploading and downloading files (Nick
44547 Craig-Wood)
44548 • Update docs on how to create known_hosts file (Nick Craig-Wood)
44550 • Storj
44552 • Rename tardigrade backend to storj backend (Nick Craig-Wood)
44554 • Implement server side Move for files (Nick Craig-Wood)
44556 • Update docs to explain differences between s3 and this backend
44558 (Elek, Márton)
44559 • Swift
44561 • Fix About so it shows info about the current container only (Nick
44563 Craig-Wood)
44564 • Union
44566 • Fix treatment of remotes with // in (Nick Craig-Wood)
44568 • Fix deadlock when one part of a multi-upload fails (Nick Craig-
44570 Wood)
44571 • Fix eplus policy returned nil (Vitor Arruda)
44573 • Yandex
44575 • Add permanent deletion support (deinferno)
44577 v1.57.0 - 2021-11-01
44579 See commits (https://github.com/rclone/rclone/com‐
44580 pare/v1.56.0...v1.57.0)
44581 • New backends
44583 • Sia: for Sia decentralized cloud (Ian Levesque, Matthew Sevey, Ivan
44585 Andreev)
44586 • Hasher: caches hashes and enable hashes for backends that don't
44588 support them (Ivan Andreev)
44589 • New commands
44591 • lsjson --stat: to get info about a single file/dir and opera‐
44593 tions/stat api (Nick Craig-Wood)
44594 • config paths: show configured paths (albertony)
44596 • New Features
44598 • about: Make human-readable output more consistent with other com‐
44600 mands (albertony)
44601 • build
44603 • Use go1.17 for building and make go1.14 the minimum supported
44605 (Nick Craig-Wood)
44606 • Update Go to 1.16 and NDK to 22b for Android builds (x0b)
44608 • config
44610 • Support hyphen in remote name from environment variable (al‐
44612 bertony)
44613 • Make temporary directory user-configurable (albertony)
44615 • Convert --cache-dir value to an absolute path (albertony)
44617 • Do not override MIME types from OS defaults (albertony)
44619 • docs
44621 • Toc styling and header levels cleanup (albertony)
44623 • Extend documentation on valid remote names (albertony)
44625 • Mention make for building and cmount tag for macos (Alex Chen)
44627 • ...and many more contributions to numerous to mention!
44629 • fs: Move with --ignore-existing will not delete skipped files
44631 (Nathan Collins)
44632 • hashsum
44634 • Treat hash values in sum file as case insensitive (Ivan Andreev)
44636 • Don't put ERROR or UNSUPPORTED in output (Ivan Andreev)
44638 • lib/encoder: Add encoding of square brackets (Ivan Andreev)
44640 • lib/file: Improve error message when attempting to create dir on
44642 nonexistent drive on windows (albertony)
44643 • lib/http: Factor password hash salt into options with default
44645 (Nolan Woods)
44646 • lib/kv: Add key-value database api (Ivan Andreev)
44648 • librclone
44650 • Add RcloneFreeString function (albertony)
44652 • Free strings in python example (albertony)
44654 • log: Optionally print pid in logs (Ivan Andreev)
44656 • ls: Introduce --human-readable global option to print human-read‐
44658 able sizes (albertony)
44659 • ncdu: Introduce key u to toggle human-readable (albertony)
44661 • operations: Add rmdirs -v output (Justin Winokur)
44663 • serve sftp
44665 • Generate an ECDSA server key as well as RSA (Nick Craig-Wood)
44667 • Generate an Ed25519 server key as well as ECDSA and RSA (al‐
44669 bertony)
44670 • serve docker
44672 • Allow to customize proxy settings of docker plugin (Ivan Andreev)
44674 • Build docker plugin for multiple platforms (Thomas Stachl)
44676 • size: Include human-readable count (albertony)
44678 • touch: Add support for touching files in directory, with recursive
44680 option, filtering and --dry-run/-i (albertony)
44681 • tree: Option to print human-readable sizes removed in favor of
44683 global option (albertony)
44684 • Bug Fixes
44686 • lib/http
44688 • Fix bad username check in single auth secret provider (Nolan
44690 Woods)
44691 • Fix handling of SSL credentials (Nolan Woods)
44693 • serve ftp: Ensure modtime is passed as UTC always to fix timezone
44695 oddities (Nick Craig-Wood)
44696 • serve sftp: Fix generation of server keys on windows (albertony)
44698 • serve docker: Fix octal umask (Ivan Andreev)
44700 • Mount
44702 • Enable rclone to be run as mount helper direct from the fstab (Ivan
44704 Andreev)
44705 • Use procfs to validate mount on linux (Ivan Andreev)
44707 • Correctly daemonize for compatibility with automount (Ivan Andreev)
44709 • VFS
44711 • Ensure names used in cache path are legal on current OS (albertony)
44713 • Ignore ECLOSED when truncating file handles to fix intermittent bad
44715 file descriptor error (Nick Craig-Wood)
44716 • Local
44718 • Refactor default OS encoding out from local backend into shared en‐
44720 coder lib (albertony)
44721 • Crypt
44723 • Return wrapped object even with --crypt-no-data-encryption (Ivan
44725 Andreev)
44726 • Fix uploads with --crypt-no-data-encryption (Nick Craig-Wood)
44728 • Azure Blob
44730 • Add --azureblob-no-head-object (Tatsuya Noyori)
44732 • Box
44734 • Make listings of heavily used directories more reliable (Nick
44736 Craig-Wood)
44737 • When doing cleanup delete as much as possible (Nick Craig-Wood)
44739 • Add --box-list-chunk to control listing chunk size (Nick Craig-
44741 Wood)
44742 • Delete items in parallel in cleanup using --checkers threads (Nick
44744 Craig-Wood)
44745 • Add --box-owned-by to only show items owned by the login passed
44747 (Nick Craig-Wood)
44748 • Retry operation_blocked_temporary errors (Nick Craig-Wood)
44750 • Chunker
44752 • Md5all must create metadata if base hash is slow (Ivan Andreev)
44754 • Drive
44756 • Speed up directory listings by constraining the API listing using
44758 the current filters (fotile96, Ivan Andreev)
44759 • Fix buffering for single request upload for files smaller than
44761 --drive-upload-cutoff (YenForYang)
44762 • Add -o config option to backend drives to make config for all
44764 shared drives (Nick Craig-Wood)
44765 • Dropbox
44767 • Add --dropbox-batch-commit-timeout to control batch timeout (Nick
44769 Craig-Wood)
44770 • Filefabric
44772 • Make backoff exponential for error_background to fix errors (Nick
44774 Craig-Wood)
44775 • Fix directory move after API change (Nick Craig-Wood)
44777 • FTP
44779 • Enable tls session cache by default (Ivan Andreev)
44781 • Add option to disable tls13 (Ivan Andreev)
44783 • Fix timeout after long uploads (Ivan Andreev)
44785 • Add support for precise time (Ivan Andreev)
44787 • Enable CI for ProFtpd, PureFtpd, VsFtpd (Ivan Andreev)
44789 • Googlephotos
44791 • Use encoder for album names to fix albums with control characters
44793 (Parth Shukla)
44794 • Jottacloud
44796 • Implement SetModTime to support modtime-only changes (albertony)
44798 • Improved error handling with SetModTime and corrupt files in gener‐
44800 al (albertony)
44801 • Add support for UserInfo (rclone config userinfo) feature (al‐
44803 bertony)
44804 • Return direct download link from rclone link command (albertony)
44806 • Koofr
44808 • Create direct share link (Dmitry Bogatov)
44810 • Pcloud
44812 • Add sha256 support (Ken Enrique Morel)
44814 • Premiumizeme
44816 • Fix directory listing after API changes (Nick Craig-Wood)
44818 • Fix server side move after API change (Nick Craig-Wood)
44820 • Fix server side directory move after API changes (Nick Craig-Wood)
44822 • S3
44824 • Add support to use CDN URL to download the file (Logeshwaran)
44826 • Add AWS Snowball Edge to providers examples (r0kk3rz)
44828 • Use a combination of SDK retries and rclone retries (Nick Craig-
44830 Wood)
44831 • Fix IAM Role for Service Account not working and other auth prob‐
44833 lems (Nick Craig-Wood)
44834 • Fix shared_credentials_file auth after reverting incorrect fix
44836 (Nick Craig-Wood)
44837 • Fix corrupted on transfer: sizes differ 0 vs xxxx with Ceph (Nick
44839 Craig-Wood)
44840 • Seafile
44842 • Fix error when not configured for 2fa (Fred)
44844 • SFTP
44846 • Fix timeout when doing MD5SUM of large file (Nick Craig-Wood)
44848 • Swift
44850 • Update OCI URL (David Liu)
44852 • Document OVH Cloud Archive (HNGamingUK)
44854 • Union
44856 • Fix rename not working with union of local disk and bucket based
44858 remote (Nick Craig-Wood)
44859 v1.56.2 - 2021-10-01
44861 See commits (https://github.com/rclone/rclone/com‐
44862 pare/v1.56.1...v1.56.2)
44863 • Bug Fixes
44865 • serve http: Re-add missing auth to http service (Nolan Woods)
44867 • build: Update golang.org/x/sys to fix crash on macOS when compiled
44869 with go1.17 (Herby Gillot)
44870 • FTP
44872 • Fix deadlock after failed update when concurrency=1 (Ivan Andreev)
44874 v1.56.1 - 2021-09-19
44876 See commits (https://github.com/rclone/rclone/com‐
44877 pare/v1.56.0...v1.56.1)
44878 • Bug Fixes
44880 • accounting: Fix maximum bwlimit by scaling scale max token bucket
44882 size (Nick Craig-Wood)
44883 • rc: Fix speed does not update in core/stats (negative0)
44885 • selfupdate: Fix --quiet option, not quite quiet (yedamo)
44887 • serve http: Fix serve http exiting directly after starting (Cnly)
44889 • build
44891 • Apply gofmt from golang 1.17 (Ivan Andreev)
44893 • Update Go to 1.16 and NDK to 22b for android/any (x0b)
44895 • Mount
44897 • Fix --daemon mode (Ivan Andreev)
44899 • VFS
44901 • Fix duplicates on rename (Nick Craig-Wood)
44903 • Fix crash when truncating a just uploaded object (Nick Craig-Wood)
44905 • Fix issue where empty dirs would build up in cache meta dir (al‐
44907 bertony)
44908 • Drive
44910 • Fix instructions for auto config (Greg Sadetsky)
44912 • Fix lsf example without drive-impersonate (Greg Sadetsky)
44914 • Onedrive
44916 • Handle HTTP 400 better in PublicLink (Alex Chen)
44918 • Clarification of the process for creating custom client_id (Mariano
44920 Absatz)
44921 • Pcloud
44923 • Return an early error when Put is called with an unknown size (Nick
44925 Craig-Wood)
44926 • Try harder to delete a failed upload (Nick Craig-Wood)
44928 • S3
44930 • Add Wasabi's AP-Northeast endpoint info (hota)
44932 • Fix typo in s3 documentation (Greg Sadetsky)
44934 • Seafile
44936 • Fix 2fa config state machine (Fred)
44938 • SFTP
44940 • Remove spurious error message on --sftp-disable-concurrent-reads
44942 (Nick Craig-Wood)
44943 • Sugarsync
44945 • Fix initial connection after config re-arrangement (Nick Craig-
44947 Wood)
44948 v1.56.0 - 2021-07-20
44950 See commits (https://github.com/rclone/rclone/com‐
44951 pare/v1.55.0...v1.56.0)
44952 • New backends
44954 • Uptobox (https://rclone.org/uptobox/) (buengese)
44956 • New commands
44958 • serve docker (https://rclone.org/commands/rclone_serve_docker/)
44960 (Antoine GIRARD) (Ivan Andreev)
44961 • and accompanying docker volume plugin (https://rclone.org/dock‐
44963 er/)
44964 • checksum (https://rclone.org/commands/rclone_checksum/) to check
44966 files against a file of checksums (Ivan Andreev)
44967 • this is also available as rclone md5sum -C etc
44969 • config touch (https://rclone.org/commands/rclone_config_touch/):
44971 ensure config exists at configured location (albertony)
44972 • test changenotify (https://rclone.org/com‐
44974 mands/rclone_test_changenotify/): command to help debugging
44975 changenotify (Nick Craig-Wood)
44976 • Deprecations
44978 • dbhashsum: Remove command deprecated a year ago (Ivan Andreev)
44980 • cache: Deprecate cache backend (Ivan Andreev)
44982 • New Features
44984 • rework config system so it can be used non-interactively via cli
44986 and rc API.
44987 • See docs in config create (https://rclone.org/com‐
44989 mands/rclone_config_create/)
44990 • This is a very big change to all the backends so may cause break‐
44992 ages - please file bugs!
44993 • librclone - export the rclone RC as a C library (lewisxy) (Nick
44995 Craig-Wood)
44996 • Link a C-API rclone shared object into your project
44998 • Use the RC as an in memory interface
45000 • Python example supplied
45002 • Also supports Android and gomobile
45004 • fs
45006 • Add --disable-http2 for global http2 disable (Nick Craig-Wood)
45008 • Make --dump imply -vv (Alex Chen)
45010 • Use binary prefixes for size and rate units (albertony)
45012 • Use decimal prefixes for counts (albertony)
45014 • Add google search widget to rclone.org (Ivan Andreev)
45016 • accounting: Calculate rolling average speed (Haochen Tong)
45018 • atexit: Terminate with non-zero status after receiving signal
45020 (Michael Hanselmann)
45021 • build
45023 • Only run event-based workflow scripts under rclone repo with man‐
45025 ual override (Mathieu Carbou)
45026 • Add Android build with gomobile (x0b)
45028 • check: Log the hash in use like cryptcheck does (Nick Craig-Wood)
45030 • version: Print os/version, kernel and bitness (Ivan Andreev)
45032 • config
45034 • Prevent use of Windows reserved names in config file name (al‐
45036 bertony)
45037 • Create config file in windows appdata directory by default (al‐
45039 bertony)
45040 • Treat any config file paths with filename notfound as memory-only
45042 config (albertony)
45043 • Delay load config file (albertony)
45045 • Replace defaultConfig with a thread-safe in-memory implementation
45047 (Chris Macklin)
45048 • Allow config create and friends to take key=value parameters
45050 (Nick Craig-Wood)
45051 • Fixed issues with flags/options set by environment vars. (Ole
45053 Frost)
45054 • fshttp: Implement graceful DSCP error handling (Tyson Moore)
45056 • lib/http - provides an abstraction for a central http server that
45058 services can bind routes to (Nolan Woods)
45059 • Add --template config and flags to serve/data (Nolan Woods)
45061 • Add default 404 handler (Nolan Woods)
45063 • link: Use "off" value for unset expiry (Nick Craig-Wood)
45065 • oauthutil: Raise fatal error if token expired without refresh token
45067 (Alex Chen)
45068 • rcat: Add --size flag for more efficient uploads of known size
45070 (Nazar Mishturak)
45071 • serve sftp: Add --stdio flag to serve via stdio (Tom)
45073 • sync: Don't warn about --no-traverse when --files-from is set (Nick
45075 Gaya)
45076 • test makefiles
45078 • Add --seed flag and make data generated repeatable (Nick Craig-
45080 Wood)
45081 • Add log levels and speed summary (Nick Craig-Wood)
45083 • Bug Fixes
45085 • accounting: Fix startTime of statsGroups.sum (Haochen Tong)
45087 • cmd/ncdu: Fix out of range panic in delete (buengese)
45089 • config
45091 • Fix issues with memory-only config file paths (albertony)
45093 • Fix in memory config not saving on the fly backend config (Nick
45095 Craig-Wood)
45096 • fshttp: Fix address parsing for DSCP (Tyson Moore)
45098 • ncdu: Update termbox-go library to fix crash (Nick Craig-Wood)
45100 • oauthutil: Fix old authorize result not recognised (Cnly)
45102 • operations: Don't update timestamps of files in --compare-dest
45104 (Nick Gaya)
45105 • selfupdate: fix archive name on macos (Ivan Andreev)
45107 • Mount
45109 • Refactor before adding serve docker (Antoine GIRARD)
45111 • VFS
45113 • Add cache reset for --vfs-cache-max-size handling at cache poll in‐
45115 terval (Leo Luan)
45116 • Fix modtime changing when reading file into cache (Nick Craig-Wood)
45118 • Avoid unnecessary subdir in cache path (albertony)
45120 • Fix that umask option cannot be set as environment variable (al‐
45122 bertony)
45123 • Do not print notice about missing poll-interval support when set to
45125 0 (albertony)
45126 • Local
45128 • Always use readlink to read symlink size for better compatibility
45130 (Nick Craig-Wood)
45131 • Add --local-unicode-normalization (and remove --local-no-unicode-
45133 normalization) (Nick Craig-Wood)
45134 • Skip entries removed concurrently with List() (Ivan Andreev)
45136 • Crypt
45138 • Support timestamped filenames from --b2-versions (Dominik Mydlil)
45140 • B2
45142 • Don't include the bucket name in public link file prefixes (Jeffrey
45144 Tolar)
45145 • Fix versions and .files with no extension (Nick Craig-Wood)
45147 • Factor version handling into lib/version (Dominik Mydlil)
45149 • Box
45151 • Use upload preflight check to avoid listings in file uploads (Nick
45153 Craig-Wood)
45154 • Return errors instead of calling log.Fatal with them (Nick Craig-
45156 Wood)
45157 • Drive
45159 • Switch to the Drives API for looking up shared drives (Nick Craig-
45161 Wood)
45162 • Fix some google docs being treated as files (Nick Craig-Wood)
45164 • Dropbox
45166 • Add --dropbox-batch-mode flag to speed up uploading (Nick Craig-
45168 Wood)
45169 • Read the batch mode (https://rclone.org/dropbox/#batch-mode) docs
45171 for more info
45172 • Set visibility in link sharing when --expire is set (Nick Craig-
45174 Wood)
45175 • Simplify chunked uploads (Alexey Ivanov)
45177 • Improve "own App IP" instructions (Ivan Andreev)
45179 • Fichier
45181 • Check if more than one upload link is returned (Nick Craig-Wood)
45183 • Support downloading password protected files and folders (Florian
45185 Penzkofer)
45186 • Make error messages report text from the API (Nick Craig-Wood)
45188 • Fix move of files in the same directory (Nick Craig-Wood)
45190 • Check that we actually got a download token and retry if we didn't
45192 (buengese)
45193 • Filefabric
45195 • Fix listing after change of from field from "int" to int. (Nick
45197 Craig-Wood)
45198 • FTP
45200 • Make upload error 250 indicate success (Nick Craig-Wood)
45202 • GCS
45204 • Make compatible with gsutil's mtime metadata (database64128)
45206 • Clean up time format constants (database64128)
45208 • Google Photos
45210 • Fix read only scope not being used properly (Nick Craig-Wood)
45212 • HTTP
45214 • Replace httplib with lib/http (Nolan Woods)
45216 • Clean up Bind to better use middleware (Nolan Woods)
45218 • Jottacloud
45220 • Fix legacy auth with state based config system (buengese)
45222 • Fix invalid url in output from link command (albertony)
45224 • Add no versions option (buengese)
45226 • Onedrive
45228 • Add list_chunk option (Nick Gaya)
45230 • Also report root error if unable to cancel multipart upload (Cnly)
45232 • Fix failed to configure: empty token found error (Nick Craig-Wood)
45234 • Make link return direct download link (Xuanchen Wu)
45236 • S3
45238 • Add --s3-no-head-object (Tatsuya Noyori)
45240 • Remove WebIdentityRoleProvider to fix crash on auth (Nick Craig-
45242 Wood)
45243 • Don't check to see if remote is object if it ends with / (Nick
45245 Craig-Wood)
45246 • Add SeaweedFS (Chris Lu)
45248 • Update Alibaba OSS endpoints (Chuan Zh)
45250 • SFTP
45252 • Fix performance regression by re-enabling concurrent writes (Nick
45254 Craig-Wood)
45255 • Expand tilde and environment variables in configured
45257 known_hosts_file (albertony)
45258 • Tardigrade
45260 • Upgrade to uplink v1.4.6 (Caleb Case)
45262 • Use negative offset (Caleb Case)
45264 • Add warning about too many open files (acsfer)
45266 • WebDAV
45268 • Fix sharepoint auth over http (Nick Craig-Wood)
45270 • Add headers option (Antoon Prins)
45272 v1.55.1 - 2021-04-26
45274 See commits (https://github.com/rclone/rclone/com‐
45275 pare/v1.55.0...v1.55.1)
45276 • Bug Fixes
45278 • selfupdate
45280 • Dont detect FUSE if build is static (Ivan Andreev)
45282 • Add build tag noselfupdate (Ivan Andreev)
45284 • sync: Fix incorrect error reported by graceful cutoff (Nick Craig-
45286 Wood)
45287 • install.sh: fix macOS arm64 download (Nick Craig-Wood)
45289 • build: Fix version numbers in android branch builds (Nick Craig-
45291 Wood)
45292 • docs
45294 • Contributing.md: update setup instructions for go1.16 (Nick Gaya)
45296 • WinFsp 2021 is out of beta (albertony)
45298 • Minor cleanup of space around code section (albertony)
45300 • Fixed some typos (albertony)
45302 • VFS
45304 • Fix a code path which allows dirty data to be removed causing data
45306 loss (Nick Craig-Wood)
45307 • Compress
45309 • Fix compressed name regexp (buengese)
45311 • Drive
45313 • Fix backend copyid of google doc to directory (Nick Craig-Wood)
45315 • Don't open browser when service account... (Ansh Mittal)
45317 • Dropbox
45319 • Add missing team_data.member scope for use with --impersonate (Nick
45321 Craig-Wood)
45322 • Fix About after scopes changes - rclone config reconnect needed
45324 (Nick Craig-Wood)
45325 • Fix Unable to decrypt returned paths from changeNotify (Nick Craig-
45327 Wood)
45328 • FTP
45330 • Fix implicit TLS (Ivan Andreev)
45332 • Onedrive
45334 • Work around for random "Unable to initialize RPS" errors (OleFrost)
45336 • SFTP
45338 • Revert sftp library to v1.12.0 from v1.13.0 to fix performance re‐
45340 gression (Nick Craig-Wood)
45341 • Fix Update ReadFrom failed: failed to send packet: EOF errors (Nick
45343 Craig-Wood)
45344 • Zoho
45346 • Fix error when region isn't set (buengese)
45348 • Do not ask for mountpoint twice when using headless setup
45350 (buengese)
45351 v1.55.0 - 2021-03-31
45353 See commits (https://github.com/rclone/rclone/com‐
45354 pare/v1.54.0...v1.55.0)
45355 • New commands
45357 • selfupdate (https://rclone.org/commands/rclone_selfupdate/) (Ivan
45359 Andreev)
45360 • Allows rclone to update itself in-place or via a package (using
45362 --package flag)
45363 • Reads cryptographically signed signatures for non beta releases
45365 • Works on all OSes.
45367 • test (https://rclone.org/commands/rclone_test/) - these are test
45369 commands - use with care!
45370 • histogram - Makes a histogram of file name characters.
45372 • info - Discovers file name or other limitations for paths.
45374 • makefiles - Make a random file hierarchy for testing.
45376 • memory - Load all the objects at remote:path into memory and re‐
45378 port memory stats.
45379 • New Features
45381 • Connection strings (https://rclone.org/docs/#connection-strings)
45383 • Config parameters can now be passed as part of the remote name as
45385 a connection string.
45386 • For example, to do the equivalent of --drive-shared-with-me use
45388 drive,shared_with_me:
45389 • Make sure we don't save on the fly remote config to the config
45391 file (Nick Craig-Wood)
45392 • Make sure backends with additional config have a different name
45394 for caching (Nick Craig-Wood)
45395 • This work was sponsored by CERN, through the CS3MESH4EOSC Project
45397 (https://cs3mesh4eosc.eu/).
45398 • CS3MESH4EOSC has received funding from the European Union’s
45400 Horizon 2020
45401 • research and innovation programme under Grant Agreement no.
45403 863353.
45404 • build
45406 • Update go build version to go1.16 and raise minimum go version to
45408 go1.13 (Nick Craig-Wood)
45409 • Make a macOS ARM64 build to support Apple Silicon (Nick Craig-
45411 Wood)
45412 • Install macfuse 4.x instead of osxfuse 3.x (Nick Craig-Wood)
45414 • Use GO386=softfloat instead of deprecated GO386=387 for 386
45416 builds (Nick Craig-Wood)
45417 • Disable IOS builds for the time being (Nick Craig-Wood)
45419 • Androids builds made with up to date NDK (x0b)
45421 • Add an rclone user to the Docker image but don't use it by de‐
45423 fault (cynthia kwok)
45424 • dedupe: Make largest directory primary to minimize data moved (Sak‐
45426 sham Khanna)
45427 • config
45429 • Wrap config library in an interface (Fionera)
45431 • Make config file system pluggable (Nick Craig-Wood)
45433 • --config "" or "/notfound" for in memory config only (Nick Craig-
45435 Wood)
45436 • Clear fs cache of stale entries when altering config (Nick Craig-
45438 Wood)
45439 • copyurl: Add option to print resulting auto-filename (albertony)
45441 • delete: Make --rmdirs obey the filters (Nick Craig-Wood)
45443 • docs - many fixes and reworks from edwardxml, albertony, pvalls,
45445 Ivan Andreev, Evan Harris, buengese, Alexey Tabakman
45446 • encoder/filename - add SCSU as tables (Klaus Post)
45448 • Add multiple paths support to --compare-dest and --copy-dest flag
45450 (K265)
45451 • filter: Make --exclude "dir/" equivalent to --exclude "dir/**"
45453 (Nick Craig-Wood)
45454 • fshttp: Add DSCP support with --dscp for QoS with differentiated
45456 services (Max Sum)
45457 • lib/cache: Add Delete and DeletePrefix methods (Nick Craig-Wood)
45459 • lib/file
45461 • Make pre-allocate detect disk full errors and return them (Nick
45463 Craig-Wood)
45464 • Don't run preallocate concurrently (Nick Craig-Wood)
45466 • Retry preallocate on EINTR (Nick Craig-Wood)
45468 • operations: Made copy and sync operations obey a RetryAfterError
45470 (Ankur Gupta)
45471 • rc
45473 • Add string alternatives for setting options over the rc (Nick
45475 Craig-Wood)
45476 • Add options/local to see the options configured in the context
45478 (Nick Craig-Wood)
45479 • Add _config parameter to set global config for just this rc call
45481 (Nick Craig-Wood)
45482 • Implement passing filter config with _filter parameter (Nick
45484 Craig-Wood)
45485 • Add fscache/clear and fscache/entries to control the fs cache
45487 (Nick Craig-Wood)
45488 • Avoid +Inf value for speed in core/stats (albertony)
45490 • Add a full set of stats to core/stats (Nick Craig-Wood)
45492 • Allow fs= params to be a JSON blob (Nick Craig-Wood)
45494 • rcd: Added systemd notification during the rclone rcd command.
45496 (Naveen Honest Raj)
45497 • rmdirs: Make --rmdirs obey the filters (Nick Craig-Wood)
45499 • version: Show build tags and type of executable (Ivan Andreev)
45501 • Bug Fixes
45503 • install.sh: make it fail on download errors (Ivan Andreev)
45505 • Fix excessive retries missing --max-duration timeout (Nick Craig-
45507 Wood)
45508 • Fix crash when --low-level-retries=0 (Nick Craig-Wood)
45510 • Fix failed token refresh on mounts created via the rc (Nick Craig-
45512 Wood)
45513 • fshttp: Fix bandwidth limiting after bad merge (Nick Craig-Wood)
45515 • lib/atexit
45517 • Unregister interrupt handler once it has fired so users can in‐
45519 terrupt again (Nick Craig-Wood)
45520 • Fix occasional failure to unmount with CTRL-C (Nick Craig-Wood)
45522 • Fix deadlock calling Finalise while Run is running (Nick Craig-
45524 Wood)
45525 • lib/rest: Fix multipart uploads not stopping on context cancel
45527 (Nick Craig-Wood)
45528 • Mount
45530 • Allow mounting to root directory on windows (albertony)
45532 • Improved handling of relative paths on windows (albertony)
45534 • Fix unicode issues with accented characters on macOS (Nick Craig-
45536 Wood)
45537 • Docs: document the new FileSecurity option in WinFsp 2021 (al‐
45539 bertony)
45540 • Docs: add note about volume path syntax on windows (albertony)
45542 • Fix caching of old directories after renaming them (Nick Craig-
45544 Wood)
45545 • Update cgofuse to the latest version to bring in macfuse 4 fix
45547 (Nick Craig-Wood)
45548 • VFS
45550 • --vfs-used-is-size to report used space using recursive scan
45552 (tYYGH)
45553 • Don't set modification time if it was already correct (Nick Craig-
45555 Wood)
45556 • Fix Create causing windows explorer to truncate files on CTRL-C
45558 CTRL-V (Nick Craig-Wood)
45559 • Fix modtimes not updating when writing via cache (Nick Craig-Wood)
45561 • Fix modtimes changing by fractional seconds after upload (Nick
45563 Craig-Wood)
45564 • Fix modtime set if --vfs-cache-mode writes/full and no write (Nick
45566 Craig-Wood)
45567 • Rename files in cache and cancel uploads on directory rename (Nick
45569 Craig-Wood)
45570 • Fix directory renaming by renaming dirs cached in memory (Nick
45572 Craig-Wood)
45573 • Local
45575 • Add flag --local-no-preallocate (David Sze)
45577 • Make nounc an advanced option except on Windows (albertony)
45579 • Don't ignore preallocate disk full errors (Nick Craig-Wood)
45581 • Cache
45583 • Add --fs-cache-expire-duration to control the fs cache (Nick Craig-
45585 Wood)
45586 • Crypt
45588 • Add option to not encrypt data (Vesnyx)
45590 • Log hash ok on upload (albertony)
45592 • Azure Blob
45594 • Add container public access level support. (Manish Kumar)
45596 • B2
45598 • Fix HTML files downloaded via cloudflare (Nick Craig-Wood)
45600 • Box
45602 • Fix transfers getting stuck on token expiry after API change (Nick
45604 Craig-Wood)
45605 • Chunker
45607 • Partially implement no-rename transactions (Maxwell Calman)
45609 • Drive
45611 • Don't stop server side copy if couldn't read description (Nick
45613 Craig-Wood)
45614 • Pass context on to drive SDK - to help with cancellation (Nick
45616 Craig-Wood)
45617 • Dropbox
45619 • Add polling for changes support (Robert Thomas)
45621 • Make --timeout 0 work properly (Nick Craig-Wood)
45623 • Raise priority of rate limited message to INFO to make it more no‐
45625 ticeable (Nick Craig-Wood)
45626 • Fichier
45628 • Implement copy & move (buengese)
45630 • Implement public link (buengese)
45632 • FTP
45634 • Implement Shutdown method (Nick Craig-Wood)
45636 • Close idle connections after --ftp-idle-timeout (1m by default)
45638 (Nick Craig-Wood)
45639 • Make --timeout 0 work properly (Nick Craig-Wood)
45641 • Add --ftp-close-timeout flag for use with awkward ftp servers (Nick
45643 Craig-Wood)
45644 • Retry connections and logins on 421 errors (Nick Craig-Wood)
45646 • Hdfs
45648 • Fix permissions for when directory is created (Lucas Messenger)
45650 • Onedrive
45652 • Make --timeout 0 work properly (Nick Craig-Wood)
45654 • S3
45656 • Fix --s3-profile which wasn't working (Nick Craig-Wood)
45658 • SFTP
45660 • Close idle connections after --sftp-idle-timeout (1m by default)
45662 (Nick Craig-Wood)
45663 • Fix "file not found" errors for read once servers (Nick Craig-Wood)
45665 • Fix SetModTime stat failed: object not found with --sftp-set-mod‐
45667 time=false (Nick Craig-Wood)
45668 • Swift
45670 • Update github.com/ncw/swift to v2.0.0 (Nick Craig-Wood)
45672 • Implement copying large objects (nguyenhuuluan434)
45674 • Union
45676 • Fix crash when using epff policy (Nick Craig-Wood)
45678 • Fix union attempting to update files on a read only file system
45680 (Nick Craig-Wood)
45681 • Refactor to use fspath.SplitFs instead of fs.ParseRemote (Nick
45683 Craig-Wood)
45684 • Fix initialisation broken in refactor (Nick Craig-Wood)
45686 • WebDAV
45688 • Add support for sharepoint with NTLM authentication (Rauno Ots)
45690 • Make sharepoint-ntlm docs more consistent (Alex Chen)
45692 • Improve terminology in sharepoint-ntlm docs (Ivan Andreev)
45694 • Disable HTTP/2 for NTLM authentication (georne)
45696 • Fix sharepoint-ntlm error 401 for parallel actions (Ivan Andreev)
45698 • Check that purged directory really exists (Ivan Andreev)
45700 • Yandex
45702 • Make --timeout 0 work properly (Nick Craig-Wood)
45704 • Zoho
45706 • Replace client id - you will need to rclone config reconnect after
45708 this (buengese)
45709 • Add forgotten setupRegion() to NewFs - this finally fixes regions
45711 other than EU (buengese)
45712 v1.54.1 - 2021-03-08
45714 See commits (https://github.com/rclone/rclone/com‐
45715 pare/v1.54.0...v1.54.1)
45716 • Bug Fixes
45718 • accounting: Fix --bwlimit when up or down is off (Nick Craig-Wood)
45720 • docs
45722 • Fix nesting of brackets and backticks in ftp docs (edwardxml)
45724 • Fix broken link in sftp page (edwardxml)
45726 • Fix typo in crypt.md (Romeo Kienzler)
45728 • Changelog: Correct link to digitalis.io (Alex JOST)
45730 • Replace #file-caching with #vfs-file-caching (Miron Veryanskiy)
45732 • Convert bogus example link to code (edwardxml)
45734 • Remove dead link from rc.md (edwardxml)
45736 • rc: Sync,copy,move: document createEmptySrcDirs parameter (Nick
45738 Craig-Wood)
45739 • lsjson: Fix unterminated JSON in the presence of errors (Nick
45741 Craig-Wood)
45742 • Mount
45744 • Fix mount dropping on macOS by setting --daemon-timeout 10m (Nick
45746 Craig-Wood)
45747 • VFS
45749 • Document simultaneous usage with the same cache shouldn't be used
45751 (Nick Craig-Wood)
45752 • B2
45754 • Automatically raise upload cutoff to avoid spurious error (Nick
45756 Craig-Wood)
45757 • Fix failed to create file system with application key limited to a
45759 prefix (Nick Craig-Wood)
45760 • Drive
45762 • Refer to Shared Drives instead of Team Drives (Nick Craig-Wood)
45764 • Dropbox
45766 • Add scopes to oauth request and optionally "members.read" (Nick
45768 Craig-Wood)
45769 • S3
45771 • Fix failed to create file system with folder level permissions pol‐
45773 icy (Nick Craig-Wood)
45774 • Fix Wasabi HEAD requests returning stale data by using only 1
45776 transport (Nick Craig-Wood)
45777 • Fix shared_credentials_file auth (Dmitry Chepurovskiy)
45779 • Add --s3-no-head to reducing costs docs (Nick Craig-Wood)
45781 • Union
45783 • Fix mkdir at root with remote:/ (Nick Craig-Wood)
45785 • Zoho
45787 • Fix custom client id's (buengese)
45789 v1.54.0 - 2021-02-02
45791 See commits (https://github.com/rclone/rclone/com‐
45792 pare/v1.53.0...v1.54.0)
45793 • New backends
45795 • Compression remote (experimental) (buengese)
45797 • Enterprise File Fabric (Nick Craig-Wood)
45799 • This work was sponsored by Storage Made Easy (https://storage‐
45801 madeeasy.com/)
45802 • HDFS (Hadoop Distributed File System) (Yury Stankevich)
45804 • Zoho workdrive (buengese)
45806 • New Features
45808 • Deglobalise the config (Nick Craig-Wood)
45810 • Global config now read from the context
45812 • This will enable passing of global config via the rc
45814 • This work was sponsored by Digitalis (https://digitalis.io/)
45816 • Add --bwlimit for upload and download (Nick Craig-Wood)
45818 • Obey bwlimit in http Transport for better limiting
45820 • Enhance systemd integration (Hekmon)
45822 • log level identification, manual activation with flag, automatic
45824 systemd launch detection
45825 • Don't compile systemd log integration for non unix systems (Ben‐
45827 jamin Gustin)
45828 • Add a --download flag to md5sum/sha1sum/hashsum to force rclone to
45830 download and hash files locally (lostheli)
45831 • Add --progress-terminal-title to print ETA to terminal title (La‐
45833 Sombra)
45834 • Make backend env vars show in help as the defaults for backend
45836 flags (Nick Craig-Wood)
45837 • build
45839 • Raise minimum go version to go1.12 (Nick Craig-Wood)
45841 • dedupe
45843 • Add --by-hash to dedupe on content hash not file name (Nick
45845 Craig-Wood)
45846 • Add --dedupe-mode list to just list dupes, changing nothing (Nick
45848 Craig-Wood)
45849 • Add warning if used on a remote which can't have duplicate names
45851 (Nick Craig-Wood)
45852 • fs
45854 • Add Shutdown optional method for backends (Nick Craig-Wood)
45856 • When using --files-from check files concurrently (zhucan)
45858 • Accumulate stats when using --dry-run (Ingo Weiss)
45860 • Always show stats when using --dry-run or --interactive (Nick
45862 Craig-Wood)
45863 • Add support for flag --no-console on windows to hide the console
45865 window (albertony)
45866 • genautocomplete: Add support to output to stdout (Ingo)
45868 • ncdu
45870 • Highlight read errors instead of aborting (Claudio Bantaloukas)
45872 • Add sort by average size in directory (Adam Plánský)
45874 • Add toggle option for average s3ize in directory - key 'a' (Adam
45876 Plánský)
45877 • Add empty folder flag into ncdu browser (Adam Plánský)
45879 • Add ! (error) and . (unreadable) file flags to go with e (empty)
45881 (Nick Craig-Wood)
45882 • obscure: Make rclone obscure - ignore newline at end of line (Nick
45884 Craig-Wood)
45885 • operations
45887 • Add logs when need to upload files to set mod times (Nick Craig-
45889 Wood)
45890 • Move and copy log name of the destination object in verbose (Adam
45892 Plánský)
45893 • Add size if known to skipped items and JSON log (Nick Craig-Wood)
45895 • rc
45897 • Prefer actual listener address if using ":port" or "addr:0" only
45899 (Nick Craig-Wood)
45900 • Add listener for finished jobs (Aleksandar Jankovic)
45902 • serve ftp: Add options to enable TLS (Deepak Sah)
45904 • serve http/webdav: Redirect requests to the base url without the /
45906 (Nick Craig-Wood)
45907 • serve restic: Implement object cache (Nick Craig-Wood)
45909 • stats: Add counter for deleted directories (Nick Craig-Wood)
45911 • sync: Only print "There was nothing to transfer" if no errors (Nick
45913 Craig-Wood)
45914 • webui
45916 • Prompt user for updating webui if an update is available (Chai‐
45918 tanya Bankanhal)
45919 • Fix plugins initialization (negative0)
45921 • Bug Fixes
45923 • fs
45925 • Fix nil pointer on copy & move operations directly to remote
45927 (Anagh Kumar Baranwal)
45928 • Fix parsing of .. when joining remotes (Nick Craig-Wood)
45930 • log: Fix enabling systemd logging when using --log-file (Nick
45932 Craig-Wood)
45933 • check
45935 • Make the error count match up in the log message (Nick Craig-
45937 Wood)
45938 • move: Fix data loss when source and destination are the same object
45940 (Nick Craig-Wood)
45941 • operations
45943 • Fix --cutoff-mode hard not cutting off immediately (Nick Craig-
45945 Wood)
45946 • Fix --immutable error message (Nick Craig-Wood)
45948 • sync
45950 • Fix --cutoff-mode soft & cautious so it doesn't end the transfer
45952 early (Nick Craig-Wood)
45953 • Fix --immutable errors retrying many times (Nick Craig-Wood)
45955 • Docs
45957 • Many fixes and a rewrite of the filtering docs (edwardxml)
45959 • Many spelling and grammar fixes (Josh Soref)
45961 • Doc fixes for commands delete, purge, rmdir, rmdirs and mount (al‐
45963 bertony)
45964 • And thanks to these people for many doc fixes too numerous to list
45966 • Ameer Dawood, Antoine GIRARD, Bob Bagwill, Christopher Stewart
45968 • CokeMine, David, Dov Murik, Durval Menezes, Evan Harris, gtorelly
45970 • Ilyess Bachiri, Janne Johansson, Kerry Su, Marcin Zelent,
45972 • Martin Michlmayr, Milly, Sơn Trần-Nguyễn
45974 • Mount
45976 • Update systemd status with cache stats (Hekmon)
45978 • Disable bazil/fuse based mount on macOS (Nick Craig-Wood)
45980 • Make rclone mount actually run rclone cmount under macOS (Nick
45982 Craig-Wood)
45983 • Implement mknod to make NFS file creation work (Nick Craig-Wood)
45985 • Make sure we don't call umount more than once (Nick Craig-Wood)
45987 • More user friendly mounting as network drive on windows (albertony)
45989 • Detect if uid or gid are set in same option string: -o
45991 uid=123,gid=456 (albertony)
45992 • Don't attempt to unmount if fs has been destroyed already (Nick
45994 Craig-Wood)
45995 • VFS
45997 • Fix virtual entries causing deleted files to still appear (Nick
45999 Craig-Wood)
46000 • Fix "file already exists" error for stale cache files (Nick Craig-
46002 Wood)
46003 • Fix file leaks with --vfs-cache-mode full and --buffer-size 0 (Nick
46005 Craig-Wood)
46006 • Fix invalid cache path on windows when using :backend: as remote
46008 (albertony)
46009 • Local
46011 • Continue listing files/folders when a circular symlink is detected
46013 (Manish Gupta)
46014 • New flag --local-zero-size-links to fix sync on some virtual
46016 filesystems (Riccardo Iaconelli)
46017 • Azure Blob
46019 • Add support for service principals (James Lim)
46021 • Add support for managed identities (Brad Ackerman)
46023 • Add examples for access tier (Bob Pusateri)
46025 • Utilize the streaming capabilities from the SDK for multipart up‐
46027 loads (Denis Neuling)
46028 • Fix setting of mime types (Nick Craig-Wood)
46030 • Fix crash when listing outside a SAS URL's root (Nick Craig-Wood)
46032 • Delete archive tier blobs before update if --azureblob-archive-
46034 tier-delete (Nick Craig-Wood)
46035 • Fix crash on startup (Nick Craig-Wood)
46037 • Fix memory usage by upgrading the SDK to v0.13.0 and implementing a
46039 TransferManager (Nick Craig-Wood)
46040 • Require go1.14+ to compile due to SDK changes (Nick Craig-Wood)
46042 • B2
46044 • Make NewObject use less expensive API calls (Nick Craig-Wood)
46046 • This will improve --files-from and restic serve in particular
46048 • Fixed crash on an empty file name (lluuaapp)
46050 • Box
46052 • Fix NewObject for files that differ in case (Nick Craig-Wood)
46054 • Fix finding directories in a case insensitive way (Nick Craig-Wood)
46056 • Chunker
46058 • Skip long local hashing, hash in-transit (fixes) (Ivan Andreev)
46060 • Set Features ReadMimeType to false as Object.MimeType not supported
46062 (Nick Craig-Wood)
46063 • Fix case-insensitive NewObject, test metadata detection (Ivan An‐
46065 dreev)
46066 • Drive
46068 • Implement rclone backend copyid command for copying files by ID
46070 (Nick Craig-Wood)
46071 • Added flag --drive-stop-on-download-limit to stop transfers when
46073 the download limit is exceeded (Anagh Kumar Baranwal)
46074 • Implement CleanUp workaround for team drives (buengese)
46076 • Allow shortcut resolution and creation to be retried (Nick Craig-
46078 Wood)
46079 • Log that emptying the trash can take some time (Nick Craig-Wood)
46081 • Add xdg office icons to xdg desktop files (Pau Rodriguez-Estivill)
46083 • Dropbox
46085 • Add support for viewing shared files and folders (buengese)
46087 • Enable short lived access tokens (Nick Craig-Wood)
46089 • Implement IDer on Objects so rclone lsf etc can read the IDs
46091 (buengese)
46092 • Set Features ReadMimeType to false as Object.MimeType not supported
46094 (Nick Craig-Wood)
46095 • Make malformed_path errors from too long files not retriable (Nick
46097 Craig-Wood)
46098 • Test file name length before upload to fix upload loop (Nick Craig-
46100 Wood)
46101 • Fichier
46103 • Set Features ReadMimeType to true as Object.MimeType is supported
46105 (Nick Craig-Wood)
46106 • FTP
46108 • Add --ftp-disable-msld option to ignore MLSD for really old servers
46110 (Nick Craig-Wood)
46111 • Make --tpslimit apply (Nick Craig-Wood)
46113 • Google Cloud Storage
46115 • Storage class object header support (Laurens Janssen)
46117 • Fix anonymous client to use rclone's HTTP client (Nick Craig-Wood)
46119 • Fix Entry doesn't belong in directory "" (same as directory) - ig‐
46121 noring (Nick Craig-Wood)
46122 • Googlephotos
46124 • New flag --gphotos-include-archived to show archived photos as well
46126 (Nicolas Rueff)
46127 • Jottacloud
46129 • Don't erroneously report support for writing mime types (buengese)
46131 • Add support for Telia Cloud (Patrik Nordlén)
46133 • Mailru
46135 • Accept special folders eg camera-upload (Ivan Andreev)
46137 • Avoid prehashing of large local files (Ivan Andreev)
46139 • Fix uploads after recent changes on server (Ivan Andreev)
46141 • Fix range requests after June 2020 changes on server (Ivan Andreev)
46143 • Fix invalid timestamp on corrupted files (fixes) (Ivan Andreev)
46145 • Remove deprecated protocol quirks (Ivan Andreev)
46147 • Memory
46149 • Fix setting of mime types (Nick Craig-Wood)
46151 • Onedrive
46153 • Add support for China region operated by 21vianet and other region‐
46155 al suppliers (NyaMisty)
46156 • Warn on gateway timeout errors (Nick Craig-Wood)
46158 • Fall back to normal copy if server-side copy unavailable (Alex
46160 Chen)
46161 • Fix server-side copy completely disabled on OneDrive for Business
46163 (Cnly)
46164 • (business only) workaround to replace existing file on server-side
46166 copy (Alex Chen)
46167 • Enhance link creation with expiry, scope, type and password (Nick
46169 Craig-Wood)
46170 • Remove % and # from the set of encoded characters (Alex Chen)
46172 • Support addressing site by server-relative URL (kice)
46174 • Opendrive
46176 • Fix finding directories in a case insensitive way (Nick Craig-Wood)
46178 • Pcloud
46180 • Fix setting of mime types (Nick Craig-Wood)
46182 • Premiumizeme
46184 • Fix finding directories in a case insensitive way (Nick Craig-Wood)
46186 • Qingstor
46188 • Fix error propagation in CleanUp (Nick Craig-Wood)
46190 • Fix rclone cleanup (Nick Craig-Wood)
46192 • S3
46194 • Added --s3-disable-http2 to disable http/2 (Anagh Kumar Baranwal)
46196 • Complete SSE-C implementation (Nick Craig-Wood)
46198 • Fix hashes on small files with AWS:KMS and SSE-C (Nick Craig-
46200 Wood)
46201 • Add MD5 metadata to objects uploaded with SSE-AWS/SSE-C (Nick
46203 Craig-Wood)
46204 • Add --s3-no-head parameter to minimise transactions on upload (Nick
46206 Craig-Wood)
46207 • Update docs with a Reducing Costs section (Nick Craig-Wood)
46209 • Added error handling for error code 429 indicating too many re‐
46211 quests (Anagh Kumar Baranwal)
46212 • Add requester pays option (kelv)
46214 • Fix copy multipart with v2 auth failing with 'SignatureDoesNot‐
46216 Match' (Louis Koo)
46217 • SFTP
46219 • Allow cert based auth via optional pubkey (Stephen Harris)
46221 • Allow user to optionally check server hosts key to add security
46223 (Stephen Harris)
46224 • Defer asking for user passwords until the SSH connection succeeds
46226 (Stephen Harris)
46227 • Remember entered password in AskPass mode (Stephen Harris)
46229 • Implement Shutdown method (Nick Craig-Wood)
46231 • Implement keyboard interactive authentication (Nick Craig-Wood)
46233 • Make --tpslimit apply (Nick Craig-Wood)
46235 • Implement --sftp-use-fstat for unusual SFTP servers (Nick Craig-
46237 Wood)
46238 • Sugarsync
46240 • Fix NewObject for files that differ in case (Nick Craig-Wood)
46242 • Fix finding directories in a case insensitive way (Nick Craig-Wood)
46244 • Swift
46246 • Fix deletion of parts of Static Large Object (SLO) (Nguyễn Hữu
46248 Luân)
46249 • Ensure partially uploaded large files are uploaded unless --swift-
46251 leave-parts-on-error (Nguyễn Hữu Luân)
46252 • Tardigrade
46254 • Upgrade to uplink v1.4.1 (Caleb Case)
46256 • WebDAV
46258 • Updated docs to show streaming to nextcloud is working (Durval
46260 Menezes)
46261 • Yandex
46263 • Set Features WriteMimeType to false as Yandex ignores mime types
46265 (Nick Craig-Wood)
46266 v1.53.4 - 2021-01-20
46268 See commits (https://github.com/rclone/rclone/com‐
46269 pare/v1.53.3...v1.53.4)
46270 • Bug Fixes
46272 • accounting: Fix data race in Transferred() (Maciej Zimnoch)
46274 • build
46276 • Stop tagged releases making a current beta (Nick Craig-Wood)
46278 • Upgrade docker buildx action (Matteo Pietro Dazzi)
46280 • Add -buildmode to cross-compile.go (Nick Craig-Wood)
46282 • Fix docker build by upgrading ilteoood/docker_buildx (Nick Craig-
46284 Wood)
46285 • Revert GitHub actions brew fix since this is now fixed (Nick
46287 Craig-Wood)
46288 • Fix brew install --cask syntax for macOS build (Nick Craig-Wood)
46290 • Update nfpm syntax to fix build of .deb/.rpm packages (Nick
46292 Craig-Wood)
46293 • Fix for Windows build errors (Ivan Andreev)
46295 • fs: Parseduration: fixed tests to use UTC time (Ankur Gupta)
46297 • fshttp: Prevent overlap of HTTP headers in logs (Nathan Collins)
46299 • rc
46301 • Fix core/command giving 500 internal error (Nick Craig-Wood)
46303 • Add Copy method to rc.Params (Nick Craig-Wood)
46305 • Fix 500 error when marshalling errors from core/command (Nick
46307 Craig-Wood)
46308 • plugins: Create plugins files only if webui is enabled. (nega‐
46310 tive0)
46311 • serve http: Fix serving files of unknown length (Nick Craig-Wood)
46313 • serve sftp: Fix authentication on one connection blocking others
46315 (Nick Craig-Wood)
46316 • Mount
46318 • Add optional brew tag to throw an error when using mount in the bi‐
46320 naries installed via Homebrew (Anagh Kumar Baranwal)
46321 • Add "." and ".." to directories to match cmount and expectations
46323 (Nick Craig-Wood)
46324 • VFS
46326 • Make cache dir absolute before using it to fix path too long errors
46328 (Nick Craig-Wood)
46329 • Chunker
46331 • Improve detection of incompatible metadata (Ivan Andreev)
46333 • Google Cloud Storage
46335 • Fix server side copy of large objects (Nick Craig-Wood)
46337 • Jottacloud
46339 • Fix token renewer to fix long uploads (Nick Craig-Wood)
46341 • Fix token refresh failed: is not a regular file error (Nick Craig-
46343 Wood)
46344 • Pcloud
46346 • Only use SHA1 hashes in EU region (Nick Craig-Wood)
46348 • Sharefile
46350 • Undo Fix backend due to API swapping integers for strings (Nick
46352 Craig-Wood)
46353 • WebDAV
46355 • Fix Open Range requests to fix 4shared mount (Nick Craig-Wood)
46357 • Add "Depth: 0" to GET requests to fix bitrix (Nick Craig-Wood)
46359 v1.53.3 - 2020-11-19
46361 See commits (https://github.com/rclone/rclone/com‐
46362 pare/v1.53.2...v1.53.3)
46363 • Bug Fixes
46365 • random: Fix incorrect use of math/rand instead of crypto/rand
46367 CVE-2020-28924 (Nick Craig-Wood)
46368 • Passwords you have generated with rclone config may be insecure
46370 • See issue #4783 (https://github.com/rclone/rclone/issues/4783)
46372 for more details and a checking tool
46373 • random: Seed math/rand in one place with crypto strong seed (Nick
46375 Craig-Wood)
46376 • VFS
46378 • Fix vfs/refresh calls with fs= parameter (Nick Craig-Wood)
46380 • Sharefile
46382 • Fix backend due to API swapping integers for strings (Nick Craig-
46384 Wood)
46385 v1.53.2 - 2020-10-26
46387 See commits (https://github.com/rclone/rclone/com‐
46388 pare/v1.53.1...v1.53.2)
46389 • Bug Fixes
46391 • accounting
46393 • Fix incorrect speed and transferTime in core/stats (Nick Craig-
46395 Wood)
46396 • Stabilize display order of transfers on Windows (Nick Craig-Wood)
46398 • operations
46400 • Fix use of --suffix without --backup-dir (Nick Craig-Wood)
46402 • Fix spurious "--checksum is in use but the source and destination
46404 have no hashes in common" (Nick Craig-Wood)
46405 • build
46407 • Work around GitHub actions brew problem (Nick Craig-Wood)
46409 • Stop using set-env and set-path in the GitHub actions (Nick
46411 Craig-Wood)
46412 • Mount
46414 • mount2: Fix the swapped UID / GID values (Russell Cattelan)
46416 • VFS
46418 • Detect and recover from a file being removed externally from the
46420 cache (Nick Craig-Wood)
46421 • Fix a deadlock vulnerability in downloaders.Close (Leo Luan)
46423 • Fix a race condition in retryFailedResets (Leo Luan)
46425 • Fix missed concurrency control between some item operations and re‐
46427 set (Leo Luan)
46428 • Add exponential backoff during ENOSPC retries (Leo Luan)
46430 • Add a missed update of used cache space (Leo Luan)
46432 • Fix --no-modtime to not attempt to set modtimes (as documented)
46434 (Nick Craig-Wood)
46435 • Local
46437 • Fix sizes and syncing with --links option on Windows (Nick Craig-
46439 Wood)
46440 • Chunker
46442 • Disable ListR to fix missing files on GDrive (workaround) (Ivan An‐
46444 dreev)
46445 • Fix upload over crypt (Ivan Andreev)
46447 • Fichier
46449 • Increase maximum file size from 100GB to 300GB (gyutw)
46451 • Jottacloud
46453 • Remove clientSecret from config when upgrading to token based au‐
46455 thentication (buengese)
46456 • Avoid double url escaping of device/mountpoint (albertony)
46458 • Remove DirMove workaround as it's not required anymore - also
46460 (buengese)
46461 • Mailru
46463 • Fix uploads after recent changes on server (Ivan Andreev)
46465 • Fix range requests after june changes on server (Ivan Andreev)
46467 • Fix invalid timestamp on corrupted files (fixes) (Ivan Andreev)
46469 • Onedrive
46471 • Fix disk usage for sharepoint (Nick Craig-Wood)
46473 • S3
46475 • Add missing regions for AWS (Anagh Kumar Baranwal)
46477 • Seafile
46479 • Fix accessing libraries > 2GB on 32 bit systems (Muffin King)
46481 • SFTP
46483 • Always convert the checksum to lower case (buengese)
46485 • Union
46487 • Create root directories if none exist (Nick Craig-Wood)
46489 v1.53.1 - 2020-09-13
46491 See commits (https://github.com/rclone/rclone/com‐
46492 pare/v1.53.0...v1.53.1)
46493 • Bug Fixes
46495 • accounting: Remove new line from end of --stats-one-line display
46497 (Nick Craig-Wood)
46498 • check
46500 • Add back missing --download flag (Nick Craig-Wood)
46502 • Fix docs (Nick Craig-Wood)
46504 • docs
46506 • Note --log-file does append (Nick Craig-Wood)
46508 • Add full stops for consistency in rclone --help (edwardxml)
46510 • Add Tencent COS to s3 provider list (wjielai)
46512 • Updated mount command to reflect that it requires Go 1.13 or new‐
46514 er (Evan Harris)
46515 • jottacloud: Mention that uploads from local disk will not need to
46517 cache files to disk for md5 calculation (albertony)
46518 • Fix formatting of rc docs page (Nick Craig-Wood)
46520 • build
46522 • Include vendor tar ball in release and fix startdev (Nick Craig-
46524 Wood)
46525 • Fix "Illegal instruction" error for ARMv6 builds (Nick Craig-
46527 Wood)
46528 • Fix architecture name in ARMv7 build (Nick Craig-Wood)
46530 • VFS
46532 • Fix spurious error "vfs cache: failed to _ensure cache EOF" (Nick
46534 Craig-Wood)
46535 • Log an ERROR if we fail to set the file to be sparse (Nick Craig-
46537 Wood)
46538 • Local
46540 • Log an ERROR if we fail to set the file to be sparse (Nick Craig-
46542 Wood)
46543 • Drive
46545 • Re-adds special oauth help text (Tim Gallant)
46547 • Opendrive
46549 • Do not retry 400 errors (Evan Harris)
46551 v1.53.0 - 2020-09-02
46553 See commits (https://github.com/rclone/rclone/com‐
46554 pare/v1.52.0...v1.53.0)
46555 • New Features
46557 • The VFS layer (https://rclone.org/commands/rclone_mount/#vfs-virtu‐
46559 al-file-system) was heavily reworked for this release - see below
46560 for more details
46561 • Interactive mode -i/--interactive (https://rclone.org/docs/#inter‐
46563 active) for destructive operations (fishbullet)
46564 • Add --bwlimit-file (https://rclone.org/docs/#bwlimit-file-band‐
46566 width-spec) flag to limit speeds of individual file transfers (Nick
46567 Craig-Wood)
46568 • Transfers are sorted by start time in the stats and progress output
46570 (Max Sum)
46571 • Make sure backends expand ~ and environment vars in file names they
46573 use (Nick Craig-Wood)
46574 • Add --refresh-times (https://rclone.org/docs/#refresh-times) flag
46576 to set modtimes on hashless backends (Nick Craig-Wood)
46577 • build
46579 • Remove vendor directory in favour of Go modules (Nick Craig-Wood)
46581 • Build with go1.15.x by default (Nick Craig-Wood)
46583 • Drop macOS 386 build as it is no longer supported by go1.15 (Nick
46585 Craig-Wood)
46586 • Add ARMv7 to the supported builds (Nick Craig-Wood)
46588 • Enable rclone cmount on macOS (Nick Craig-Wood)
46590 • Make rclone build with gccgo (Nick Craig-Wood)
46592 • Make rclone build with wasm (Nick Craig-Wood)
46594 • Change beta numbering to be semver compatible (Nick Craig-Wood)
46596 • Add file properties and icon to Windows executable (albertony)
46598 • Add experimental interface for integrating rclone into browsers
46600 (Nick Craig-Wood)
46601 • lib: Add file name compression (Klaus Post)
46603 • rc
46605 • Allow installation and use of plugins and test plugins with
46607 rclone-webui (Chaitanya Bankanhal)
46608 • Add reverse proxy pluginsHandler for serving plugins (Chaitanya
46610 Bankanhal)
46611 • Add mount/listmounts option for listing current mounts (Chaitanya
46613 Bankanhal)
46614 • Add operations/uploadfile to upload a file through rc using en‐
46616 coding multipart/form-data (Chaitanya Bankanhal)
46617 • Add core/command to execute rclone terminal commands. (Chaitanya
46619 Bankanhal)
46620 • rclone check
46622 • Add reporting of filenames for same/missing/changed (Nick Craig-
46624 Wood)
46625 • Make check command obey --dry-run/-i/--interactive (Nick Craig-
46627 Wood)
46628 • Make check do --checkers files concurrently (Nick Craig-Wood)
46630 • Retry downloads if they fail when using the --download flag (Nick
46632 Craig-Wood)
46633 • Make it show stats by default (Nick Craig-Wood)
46635 • rclone obscure: Allow obscure command to accept password on STDIN
46637 (David Ibarra)
46638 • rclone config
46640 • Set RCLONE_CONFIG_DIR for use in config files and subprocesses
46642 (Nick Craig-Wood)
46643 • Reject remote names starting with a dash. (jtagcat)
46645 • rclone cryptcheck: Add reporting of filenames for same/miss‐
46647 ing/changed (Nick Craig-Wood)
46648 • rclone dedupe: Make it obey the --size-only flag for duplicate de‐
46650 tection (Nick Craig-Wood)
46651 • rclone link: Add --expire and --unlink flags (Roman Kredentser)
46653 • rclone mkdir: Warn when using mkdir on remotes which can't have
46655 empty directories (Nick Craig-Wood)
46656 • rclone rc: Allow JSON parameters to simplify command line usage
46658 (Nick Craig-Wood)
46659 • rclone serve ftp
46661 • Don't compile on < go1.13 after dependency update (Nick Craig-
46663 Wood)
46664 • Add error message if auth proxy fails (Nick Craig-Wood)
46666 • Use refactored goftp.io/server library for binary shrink (Nick
46668 Craig-Wood)
46669 • rclone serve restic: Expose interfaces so that rclone can be used
46671 as a library from within restic (Jack)
46672 • rclone sync: Add --track-renames-strategy leaf (Nick Craig-Wood)
46674 • rclone touch: Add ability to set nanosecond resolution times (Nick
46676 Craig-Wood)
46677 • rclone tree: Remove -i shorthand for --noindent as it conflicts
46679 with -i/--interactive (Nick Craig-Wood)
46680 • Bug Fixes
46682 • accounting
46684 • Fix documentation for speed/speedAvg (Nick Craig-Wood)
46686 • Fix elapsed time not show actual time since beginning (Chaitanya
46688 Bankanhal)
46689 • Fix deadlock in stats printing (Nick Craig-Wood)
46691 • build
46693 • Fix file handle leak in GitHub release tool (Garrett Squire)
46695 • rclone check: Fix successful retries with --download counting er‐
46697 rors (Nick Craig-Wood)
46698 • rclone dedupe: Fix logging to be easier to understand (Nick Craig-
46700 Wood)
46701 • Mount
46703 • Warn macOS users that mount implementation is changing (Nick Craig-
46705 Wood)
46706 • to test the new implementation use rclone cmount instead of
46708 rclone mount
46709 • this is because the library rclone uses has dropped macOS support
46711 • rc interface
46713 • Add call for unmount all (Chaitanya Bankanhal)
46715 • Make mount/mount remote control take vfsOpt option (Nick Craig-
46717 Wood)
46718 • Add mountOpt to mount/mount (Nick Craig-Wood)
46720 • Add VFS and Mount options to mount/listmounts (Nick Craig-Wood)
46722 • Catch panics in cgofuse initialization and turn into error messages
46724 (Nick Craig-Wood)
46725 • Always supply stat information in Readdir (Nick Craig-Wood)
46727 • Add support for reading unknown length files using direct IO (Win‐
46729 dows) (Nick Craig-Wood)
46730 • Fix On Windows don't add -o uid/gid=-1 if user supplies -o uid/gid.
46732 (Nick Craig-Wood)
46733 • Fix macOS losing directory contents in cmount (Nick Craig-Wood)
46735 • Fix volume name broken in recent refactor (Nick Craig-Wood)
46737 • VFS
46739 • Implement partial reads for --vfs-cache-mode full (Nick Craig-Wood)
46741 • Add --vfs-writeback option to delay writes back to cloud storage
46743 (Nick Craig-Wood)
46744 • Add --vfs-read-ahead parameter for use with --vfs-cache-mode full
46746 (Nick Craig-Wood)
46747 • Restart pending uploads on restart of the cache (Nick Craig-Wood)
46749 • Support synchronous cache space recovery upon ENOSPC (Leo Luan)
46751 • Allow ReadAt and WriteAt to run concurrently with themselves (Nick
46753 Craig-Wood)
46754 • Change modtime of file before upload to current (Rob Calistri)
46756 • Recommend --vfs-cache-modes writes on backends which can't stream
46758 (Nick Craig-Wood)
46759 • Add an optional fs parameter to vfs rc methods (Nick Craig-Wood)
46761 • Fix errors when using > 260 char files in the cache in Windows
46763 (Nick Craig-Wood)
46764 • Fix renaming of items while they are being uploaded (Nick Craig-
46766 Wood)
46767 • Fix very high load caused by slow directory listings (Nick Craig-
46769 Wood)
46770 • Fix renamed files not being uploaded with --vfs-cache-mode minimal
46772 (Nick Craig-Wood)
46773 • Fix directory locking caused by slow directory listings (Nick
46775 Craig-Wood)
46776 • Fix saving from chrome without --vfs-cache-mode writes (Nick Craig-
46778 Wood)
46779 • Local
46781 • Add --local-no-updated to provide a consistent view of changing ob‐
46783 jects (Nick Craig-Wood)
46784 • Add --local-no-set-modtime option to prevent modtime changes (tyhu‐
46786 ber1)
46787 • Fix race conditions updating and reading Object metadata (Nick
46789 Craig-Wood)
46790 • Cache
46792 • Make any created backends be cached to fix rc problems (Nick Craig-
46794 Wood)
46795 • Fix dedupe on caches wrapping drives (Nick Craig-Wood)
46797 • Crypt
46799 • Add --crypt-server-side-across-configs flag (Nick Craig-Wood)
46801 • Make any created backends be cached to fix rc problems (Nick Craig-
46803 Wood)
46804 • Alias
46806 • Make any created backends be cached to fix rc problems (Nick Craig-
46808 Wood)
46809 • Azure Blob
46811 • Don't compile on < go1.13 after dependency update (Nick Craig-Wood)
46813 • B2
46815 • Implement server-side copy for files > 5GB (Nick Craig-Wood)
46817 • Cancel in progress multipart uploads and copies on rclone exit
46819 (Nick Craig-Wood)
46820 • Note that b2's encoding now allows but rclone's hasn't changed
46822 (Nick Craig-Wood)
46823 • Fix transfers when using download_url (Nick Craig-Wood)
46825 • Box
46827 • Implement rclone cleanup (buengese)
46829 • Cancel in progress multipart uploads and copies on rclone exit
46831 (Nick Craig-Wood)
46832 • Allow authentication with access token (David)
46834 • Chunker
46836 • Make any created backends be cached to fix rc problems (Nick Craig-
46838 Wood)
46839 • Drive
46841 • Add rclone backend drives to list shared drives (teamdrives) (Nick
46843 Craig-Wood)
46844 • Implement rclone backend untrash (Nick Craig-Wood)
46846 • Work around drive bug which didn't set modtime of copied docs (Nick
46848 Craig-Wood)
46849 • Added --drive-starred-only to only show starred files (Jay McEn‐
46851 tire)
46852 • Deprecate --drive-alternate-export as it is no longer needed (the‐
46854 mylogin)
46855 • Fix duplication of Google docs on server-side copy (Nick Craig-
46857 Wood)
46858 • Fix "panic: send on closed channel" when recycling dir entries
46860 (Nick Craig-Wood)
46861 • Dropbox
46863 • Add copyright detector info in limitations section in the docs
46865 (Alex Guerrero)
46866 • Fix rclone link by removing expires parameter (Nick Craig-Wood)
46868 • Fichier
46870 • Detect Flood detected: IP Locked error and sleep for 30s (Nick
46872 Craig-Wood)
46873 • FTP
46875 • Add explicit TLS support (Heiko Bornholdt)
46877 • Add support for --dump bodies and --dump auth for debugging (Nick
46879 Craig-Wood)
46880 • Fix interoperation with pure-ftpd (Nick Craig-Wood)
46882 • Google Cloud Storage
46884 • Add support for anonymous access (Kai Lüke)
46886 • Jottacloud
46888 • Bring back legacy authentication for use with whitelabel versions
46890 (buengese)
46891 • Switch to new api root - also implement a very ugly workaround for
46893 the DirMove failures (buengese)
46894 • Onedrive
46896 • Rework cancel of multipart uploads on rclone exit (Nick Craig-Wood)
46898 • Implement rclone cleanup (Nick Craig-Wood)
46900 • Add --onedrive-no-versions flag to remove old versions (Nick Craig-
46902 Wood)
46903 • Pcloud
46905 • Implement rclone link for public link creation (buengese)
46907 • Qingstor
46909 • Cancel in progress multipart uploads on rclone exit (Nick Craig-
46911 Wood)
46912 • S3
46914 • Preserve metadata when doing multipart copy (Nick Craig-Wood)
46916 • Cancel in progress multipart uploads and copies on rclone exit
46918 (Nick Craig-Wood)
46919 • Add rclone link for public link sharing (Roman Kredentser)
46921 • Add rclone backend restore command to restore objects from GLACIER
46923 (Nick Craig-Wood)
46924 • Add rclone cleanup and rclone backend cleanup to clean unfinished
46926 multipart uploads (Nick Craig-Wood)
46927 • Add rclone backend list-multipart-uploads to list unfinished multi‐
46929 part uploads (Nick Craig-Wood)
46930 • Add --s3-max-upload-parts support (Kamil Trzciński)
46932 • Add --s3-no-check-bucket for minimising rclone transactions and
46934 perms (Nick Craig-Wood)
46935 • Add --s3-profile and --s3-shared-credentials-file options (Nick
46937 Craig-Wood)
46938 • Use regional s3 us-east-1 endpoint (David)
46940 • Add Scaleway provider (Vincent Feltz)
46942 • Update IBM COS endpoints (Egor Margineanu)
46944 • Reduce the default --s3-copy-cutoff to < 5GB for Backblaze S3 com‐
46946 patibility (Nick Craig-Wood)
46947 • Fix detection of bucket existing (Nick Craig-Wood)
46949 • SFTP
46951 • Use the absolute path instead of the relative path for listing for
46953 improved compatibility (Nick Craig-Wood)
46954 • Add --sftp-subsystem and --sftp-server-command options (aus)
46956 • Swift
46958 • Fix dangling large objects breaking the listing (Nick Craig-Wood)
46960 • Fix purge not deleting directory markers (Nick Craig-Wood)
46962 • Fix update multipart object removing all of its own parts (Nick
46964 Craig-Wood)
46965 • Fix missing hash from object returned from upload (Nick Craig-Wood)
46967 • Tardigrade
46969 • Upgrade to uplink v1.2.0 (Kaloyan Raev)
46971 • Union
46973 • Fix writing with the all policy (Nick Craig-Wood)
46975 • WebDAV
46977 • Fix directory creation with 4shared (Nick Craig-Wood)
46979 v1.52.3 - 2020-08-07
46981 See commits (https://github.com/rclone/rclone/com‐
46982 pare/v1.52.2...v1.52.3)
46983 • Bug Fixes
46985 • docs
46987 • Disable smart typography (e.g. en-dash) in MANUAL.* and man page
46989 (Nick Craig-Wood)
46990 • Update install.md to reflect minimum Go version (Evan Harris)
46992 • Update install from source instructions (Nick Craig-Wood)
46994 • make_manual: Support SOURCE_DATE_EPOCH (Morten Linderud)
46996 • log: Fix --use-json-log going to stderr not --log-file on Windows
46998 (Nick Craig-Wood)
46999 • serve dlna: Fix file list on Samsung Series 6+ TVs (Matteo Pietro
47001 Dazzi)
47002 • sync: Fix deadlock with --track-renames-strategy modtime (Nick
47004 Craig-Wood)
47005 • Cache
47007 • Fix moveto/copyto remote:file remote:file2 (Nick Craig-Wood)
47009 • Drive
47011 • Stop using root_folder_id as a cache (Nick Craig-Wood)
47013 • Make dangling shortcuts appear in listings (Nick Craig-Wood)
47015 • Drop "Disabling ListR" messages down to debug (Nick Craig-Wood)
47017 • Workaround and policy for Google Drive API (Dmitry Ustalov)
47019 • FTP
47021 • Add note to docs about home vs root directory selection (Nick
47023 Craig-Wood)
47024 • Onedrive
47026 • Fix reverting to Copy when Move would have worked (Nick Craig-Wood)
47028 • Avoid comma rendered in URL in onedrive.md (Kevin)
47030 • Pcloud
47032 • Fix oauth on European region "eapi.pcloud.com" (Nick Craig-Wood)
47034 • S3
47036 • Fix bucket Region auto detection when Region unset in config (Nick
47038 Craig-Wood)
47039 v1.52.2 - 2020-06-24
47041 See commits (https://github.com/rclone/rclone/com‐
47042 pare/v1.52.1...v1.52.2)
47043 • Bug Fixes
47045 • build
47047 • Fix docker release build action (Nick Craig-Wood)
47049 • Fix custom timezone in Docker image (NoLooseEnds)
47051 • check: Fix misleading message which printed errors instead of dif‐
47053 ferences (Nick Craig-Wood)
47054 • errors: Add WSAECONNREFUSED and more to the list of retriable Win‐
47056 dows errors (Nick Craig-Wood)
47057 • rcd: Fix incorrect prometheus metrics (Gary Kim)
47059 • serve restic: Fix flags so they use environment variables (Nick
47061 Craig-Wood)
47062 • serve webdav: Fix flags so they use environment variables (Nick
47064 Craig-Wood)
47065 • sync: Fix --track-renames-strategy modtime (Nick Craig-Wood)
47067 • Drive
47069 • Fix not being able to delete a directory with a trashed shortcut
47071 (Nick Craig-Wood)
47072 • Fix creating a directory inside a shortcut (Nick Craig-Wood)
47074 • Fix --drive-impersonate with cached root_folder_id (Nick Craig-
47076 Wood)
47077 • SFTP
47079 • Fix SSH key PEM loading (Zac Rubin)
47081 • Swift
47083 • Speed up deletes by not retrying segment container deletes (Nick
47085 Craig-Wood)
47086 • Tardigrade
47088 • Upgrade to uplink v1.1.1 (Caleb Case)
47090 • WebDAV
47092 • Fix free/used display for rclone about/df for certain backends
47094 (Nick Craig-Wood)
47095 v1.52.1 - 2020-06-10
47097 See commits (https://github.com/rclone/rclone/com‐
47098 pare/v1.52.0...v1.52.1)
47099 • Bug Fixes
47101 • lib/file: Fix SetSparse on Windows 7 which fixes downloads of files
47103 > 250MB (Nick Craig-Wood)
47104 • build
47106 • Update go.mod to go1.14 to enable -mod=vendor build (Nick Craig-
47108 Wood)
47109 • Remove quicktest from Dockerfile (Nick Craig-Wood)
47111 • Build Docker images with GitHub actions (Matteo Pietro Dazzi)
47113 • Update Docker build workflows (Nick Craig-Wood)
47115 • Set user_allow_other in /etc/fuse.conf in the Docker image (Nick
47117 Craig-Wood)
47118 • Fix xgo build after go1.14 go.mod update (Nick Craig-Wood)
47120 • docs
47122 • Add link to source and modified time to footer of every page
47124 (Nick Craig-Wood)
47125 • Remove manually set dates and use git dates instead (Nick Craig-
47127 Wood)
47128 • Minor tense, punctuation, brevity and positivity changes for the
47130 home page (edwardxml)
47131 • Remove leading slash in page reference in footer when present
47133 (Nick Craig-Wood)
47134 • Note commands which need obscured input in the docs (Nick Craig-
47136 Wood)
47137 • obscure: Write more help as we are referencing it elsewhere (Nick
47139 Craig-Wood)
47140 • VFS
47142 • Fix OS vs Unix path confusion - fixes ChangeNotify on Windows (Nick
47144 Craig-Wood)
47145 • Drive
47147 • Fix missing items when listing using --fast-list / ListR (Nick
47149 Craig-Wood)
47150 • Putio
47152 • Fix panic on Object.Open (Cenk Alti)
47154 • S3
47156 • Fix upload of single files into buckets without create permission
47158 (Nick Craig-Wood)
47159 • Fix --header-upload (Nick Craig-Wood)
47161 • Tardigrade
47163 • Fix listing bug by upgrading to v1.0.7
47165 • Set UserAgent to rclone (Caleb Case)
47167 v1.52.0 - 2020-05-27
47169 Special thanks to Martin Michlmayr for proof reading and correcting all
47170 the docs and Edward Barker for helping re-write the front page.
47171 See commits (https://github.com/rclone/rclone/com‐
47173 pare/v1.51.0...v1.52.0)
47174 • New backends
47176 • Tardigrade (https://rclone.org/tardigrade/) backend for use with
47178 storj.io (Caleb Case)
47179 • Union (https://rclone.org/union/) re-write to have multiple
47181 writable remotes (Max Sum)
47182 • Seafile for Seafile server (Fred @creativeprojects)
47184 • New commands
47186 • backend: command for backend-specific commands (see backends) (Nick
47188 Craig-Wood)
47189 • cachestats: Deprecate in favour of rclone backend stats cache:
47191 (Nick Craig-Wood)
47192 • dbhashsum: Deprecate in favour of rclone hashsum DropboxHash (Nick
47194 Craig-Wood)
47195 • New Features
47197 • Add --header-download and --header-upload flags for setting HTTP
47199 headers when uploading/downloading (Tim Gallant)
47200 • Add --header flag to add HTTP headers to every HTTP transaction
47202 (Nick Craig-Wood)
47203 • Add --check-first to do all checking before starting transfers
47205 (Nick Craig-Wood)
47206 • Add --track-renames-strategy for configurable matching criteria for
47208 --track-renames (Bernd Schoolmann)
47209 • Add --cutoff-mode hard,soft,cautious (Shing Kit Chan & Franklyn
47211 Tackitt)
47212 • Filter flags (e.g. --files-from -) can read from stdin (fishbul‐
47214 let)
47215 • Add --error-on-no-transfer option (Jon Fautley)
47217 • Implement --order-by xxx,mixed for copying some small and some big
47219 files (Nick Craig-Wood)
47220 • Allow --max-backlog to be negative meaning as large as possible
47222 (Nick Craig-Wood)
47223 • Added --no-unicode-normalization flag to allow Unicode filenames to
47225 remain unique (Ben Zenker)
47226 • Allow --min-age/--max-age to take a date as well as a duration
47228 (Nick Craig-Wood)
47229 • Add rename statistics for file and directory renames (Nick Craig-
47231 Wood)
47232 • Add statistics output to JSON log (reddi)
47234 • Make stats be printed on non-zero exit code (Nick Craig-Wood)
47236 • When running --password-command allow use of stdin (Sébastien
47238 Gross)
47239 • Stop empty strings being a valid remote path (Nick Craig-Wood)
47241 • accounting: support WriterTo for less memory copying (Nick Craig-
47243 Wood)
47244 • build
47246 • Update to use go1.14 for the build (Nick Craig-Wood)
47248 • Add -trimpath to release build for reproduceable builds (Nick
47250 Craig-Wood)
47251 • Remove GOOS and GOARCH from Dockerfile (Brandon Philips)
47253 • config
47255 • Fsync the config file after writing to save more reliably (Nick
47257 Craig-Wood)
47258 • Add --obscure and --no-obscure flags to config create/update
47260 (Nick Craig-Wood)
47261 • Make config show take remote: as well as remote (Nick Craig-Wood)
47263 • copyurl: Add --no-clobber flag (Denis)
47265 • delete: Added --rmdirs flag to delete directories as well (Kush)
47267 • filter: Added --files-from-raw flag (Ankur Gupta)
47269 • genautocomplete: Add support for fish shell (Matan Rosenberg)
47271 • log: Add support for syslog LOCAL facilities (Patryk Jakuszew)
47273 • lsjson: Add --hash-type parameter and use it in lsf to speed up
47275 hashing (Nick Craig-Wood)
47276 • rc
47278 • Add -o/--opt and -a/--arg for more structured input (Nick Craig-
47280 Wood)
47281 • Implement backend/command for running backend-specific commands
47283 remotely (Nick Craig-Wood)
47284 • Add mount/mount command for starting rclone mount via the API
47286 (Chaitanya)
47287 • rcd: Add Prometheus metrics support (Gary Kim)
47289 • serve http
47291 • Added a --template flag for user defined markup (calistri)
47293 • Add Last-Modified headers to files and directories (Nick Craig-
47295 Wood)
47296 • serve sftp: Add support for multiple host keys by repeating --key
47298 flag (Maxime Suret)
47299 • touch: Add --localtime flag to make --timestamp localtime not UTC
47301 (Nick Craig-Wood)
47302 • Bug Fixes
47304 • accounting
47306 • Restore "Max number of stats groups reached" log line (Michał
47308 Matczuk)
47309 • Correct exitcode on Transfer Limit Exceeded flag. (Anuar
47311 Serdaliyev)
47312 • Reset bytes read during copy retry (Ankur Gupta)
47314 • Fix race clearing stats (Nick Craig-Wood)
47316 • copy: Only create empty directories when they don't exist on the
47318 remote (Ishuah Kariuki)
47319 • dedupe: Stop dedupe deleting files with identical IDs (Nick Craig-
47321 Wood)
47322 • oauth
47324 • Use custom http client so that --no-check-certificate is honored
47326 by oauth token fetch (Mark Spieth)
47327 • Replace deprecated oauth2.NoContext (Lars Lehtonen)
47329 • operations
47331 • Fix setting the timestamp on Windows for multithread copy (Nick
47333 Craig-Wood)
47334 • Make rcat obey --ignore-checksum (Nick Craig-Wood)
47336 • Make --max-transfer more accurate (Nick Craig-Wood)
47338 • rc
47340 • Fix dropped error (Lars Lehtonen)
47342 • Fix misplaced http server config (Xiaoxing Ye)
47344 • Disable duplicate log (ElonH)
47346 • serve dlna
47348 • Cds: don't specify childCount at all when unknown (Dan Walters)
47350 • Cds: use modification time as date in dlna metadata (Dan Walters)
47352 • serve restic: Fix tests after restic project removed vendoring
47354 (Nick Craig-Wood)
47355 • sync
47357 • Fix incorrect "nothing to transfer" message using --delete-before
47359 (Nick Craig-Wood)
47360 • Only create empty directories when they don't exist on the remote
47362 (Ishuah Kariuki)
47363 • Mount
47365 • Add --async-read flag to disable asynchronous reads (Nick Craig-
47367 Wood)
47368 • Ignore --allow-root flag with a warning as it has been removed up‐
47370 stream (Nick Craig-Wood)
47371 • Warn if --allow-non-empty used on Windows and clarify docs (Nick
47373 Craig-Wood)
47374 • Constrain to go1.13 or above otherwise bazil.org/fuse fails to com‐
47376 pile (Nick Craig-Wood)
47377 • Fix fail because of too long volume name (evileye)
47379 • Report 1PB free for unknown disk sizes (Nick Craig-Wood)
47381 • Map more rclone errors into file systems errors (Nick Craig-Wood)
47383 • Fix disappearing cwd problem (Nick Craig-Wood)
47385 • Use ReaddirPlus on Windows to improve directory listing performance
47387 (Nick Craig-Wood)
47388 • Send a hint as to whether the filesystem is case insensitive or not
47390 (Nick Craig-Wood)
47391 • Add rc command mount/types (Nick Craig-Wood)
47393 • Change maximum leaf name length to 1024 bytes (Nick Craig-Wood)
47395 • VFS
47397 • Add --vfs-read-wait and --vfs-write-wait flags to control time
47399 waiting for a sequential read/write (Nick Craig-Wood)
47400 • Change default --vfs-read-wait to 20ms (it was 5ms and not config‐
47402 urable) (Nick Craig-Wood)
47403 • Make df output more consistent on a rclone mount. (Yves G)
47405 • Report 1PB free for unknown disk sizes (Nick Craig-Wood)
47407 • Fix race condition caused by unlocked reading of Dir.path (Nick
47409 Craig-Wood)
47410 • Make File lock and Dir lock not overlap to avoid deadlock (Nick
47412 Craig-Wood)
47413 • Implement lock ordering between File and Dir to eliminate deadlocks
47415 (Nick Craig-Wood)
47416 • Factor the vfs cache into its own package (Nick Craig-Wood)
47418 • Pin the Fs in use in the Fs cache (Nick Craig-Wood)
47420 • Add SetSys() methods to Node to allow caching stuff on a node (Nick
47422 Craig-Wood)
47423 • Ignore file not found errors from Hash in Read.Release (Nick Craig-
47425 Wood)
47426 • Fix hang in read wait code (Nick Craig-Wood)
47428 • Local
47430 • Speed up multi thread downloads by using sparse files on Windows
47432 (Nick Craig-Wood)
47433 • Implement --local-no-sparse flag for disabling sparse files (Nick
47435 Craig-Wood)
47436 • Implement rclone backend noop for testing purposes (Nick Craig-
47438 Wood)
47439 • Fix "file not found" errors on post transfer Hash calculation (Nick
47441 Craig-Wood)
47442 • Cache
47444 • Implement rclone backend stats command (Nick Craig-Wood)
47446 • Fix Server Side Copy with Temp Upload (Brandon McNama)
47448 • Remove Unused Functions (Lars Lehtonen)
47450 • Disable race tests until bbolt is fixed (Nick Craig-Wood)
47452 • Move methods used for testing into test file (greatroar)
47454 • Add Pin and Unpin and canonicalised lookup (Nick Craig-Wood)
47456 • Use proper import path go.etcd.io/bbolt (Robert-André Mauchin)
47458 • Crypt
47460 • Calculate hashes for uploads from local disk (Nick Craig-Wood)
47462 • This allows crypted Jottacloud uploads without using local disk
47464 • This means crypted s3/b2 uploads will now have hashes
47466 • Added rclone backend decode/encode commands to replicate function‐
47468 ality of cryptdecode (Anagh Kumar Baranwal)
47469 • Get rid of the unused Cipher interface as it obfuscated the code
47471 (Nick Craig-Wood)
47472 • Azure Blob
47474 • Implement streaming of unknown sized files so rcat is now supported
47476 (Nick Craig-Wood)
47477 • Implement memory pooling to control memory use (Nick Craig-Wood)
47479 • Add --azureblob-disable-checksum flag (Nick Craig-Wood)
47481 • Retry InvalidBlobOrBlock error as it may indicate block concurrency
47483 problems (Nick Craig-Wood)
47484 • Remove unused Object.parseTimeString() (Lars Lehtonen)
47486 • Fix permission error on SAS URL limited to container (Nick Craig-
47488 Wood)
47489 • B2
47491 • Add support for --header-upload and --header-download (Tim Gallant)
47493 • Ignore directory markers at the root also (Nick Craig-Wood)
47495 • Force the case of the SHA1 to lowercase (Nick Craig-Wood)
47497 • Remove unused largeUpload.clearUploadURL() (Lars Lehtonen)
47499 • Box
47501 • Add support for --header-upload and --header-download (Tim Gallant)
47503 • Implement About to read size used (Nick Craig-Wood)
47505 • Add token renew function for jwt auth (David Bramwell)
47507 • Added support for interchangeable root folder for Box backend
47509 (Sunil Patra)
47510 • Remove unnecessary iat from jws claims (David)
47512 • Drive
47514 • Follow shortcuts by default, skip with --drive-skip-shortcuts (Nick
47516 Craig-Wood)
47517 • Implement rclone backend shortcut command for creating shortcuts
47519 (Nick Craig-Wood)
47520 • Added rclone backend command to change service_account_file and
47522 chunk_size (Anagh Kumar Baranwal)
47523 • Fix missing files when using --fast-list and --drive-shared-with-me
47525 (Nick Craig-Wood)
47526 • Fix duplicate items when using --drive-shared-with-me (Nick Craig-
47528 Wood)
47529 • Extend --drive-stop-on-upload-limit to respond to teamDriveFileLim‐
47531 itExceeded. (harry)
47532 • Don't delete files with multiple parents to avoid data loss (Nick
47534 Craig-Wood)
47535 • Server side copy docs use default description if empty (Nick Craig-
47537 Wood)
47538 • Dropbox
47540 • Make error insufficient space to be fatal (harry)
47542 • Add info about required redirect url (Elan Ruusamäe)
47544 • Fichier
47546 • Add support for --header-upload and --header-download (Tim Gallant)
47548 • Implement custom pacer to deal with the new rate limiting
47550 (buengese)
47551 • FTP
47553 • Fix lockup when using concurrency limit on failed connections (Nick
47555 Craig-Wood)
47556 • Fix lockup on failed upload when using concurrency limit (Nick
47558 Craig-Wood)
47559 • Fix lockup on Close failures when using concurrency limit (Nick
47561 Craig-Wood)
47562 • Work around pureftp sending spurious 150 messages (Nick Craig-Wood)
47564 • Google Cloud Storage
47566 • Add support for --header-upload and --header-download (Nick Craig-
47568 Wood)
47569 • Add ARCHIVE storage class to help (Adam Stroud)
47571 • Ignore directory markers at the root (Nick Craig-Wood)
47573 • Googlephotos
47575 • Make the start year configurable (Daven)
47577 • Add support for --header-upload and --header-download (Tim Gallant)
47579 • Create feature/favorites directory (Brandon Philips)
47581 • Fix "concurrent map write" error (Nick Craig-Wood)
47583 • Don't put an image in error message (Nick Craig-Wood)
47585 • HTTP
47587 • Improved directory listing with new template from Caddy project
47589 (calisro)
47590 • Jottacloud
47592 • Implement --jottacloud-trashed-only (buengese)
47594 • Add support for --header-upload and --header-download (Tim Gallant)
47596 • Use RawURLEncoding when decoding base64 encoded login token
47598 (buengese)
47599 • Implement cleanup (buengese)
47601 • Update docs regarding cleanup, removed remains from old auth, and
47603 added warning about special mountpoints. (albertony)
47604 • Mailru
47606 • Describe 2FA requirements (valery1707)
47608 • Onedrive
47610 • Implement --onedrive-server-side-across-configs (Nick Craig-Wood)
47612 • Add support for --header-upload and --header-download (Tim Gallant)
47614 • Fix occasional 416 errors on multipart uploads (Nick Craig-Wood)
47616 • Added maximum chunk size limit warning in the docs (Harry)
47618 • Fix missing drive on config (Nick Craig-Wood)
47620 • Make error quotaLimitReached to be fatal (harry)
47622 • Opendrive
47624 • Add support for --header-upload and --header-download (Tim Gallant)
47626 • Pcloud
47628 • Added support for interchangeable root folder for pCloud backend
47630 (Sunil Patra)
47631 • Add support for --header-upload and --header-download (Tim Gallant)
47633 • Fix initial config "Auth state doesn't match" message (Nick Craig-
47635 Wood)
47636 • Premiumizeme
47638 • Add support for --header-upload and --header-download (Tim Gallant)
47640 • Prune unused functions (Lars Lehtonen)
47642 • Putio
47644 • Add support for --header-upload and --header-download (Nick Craig-
47646 Wood)
47647 • Make downloading files use the rclone http Client (Nick Craig-Wood)
47649 • Fix parsing of remotes with leading and trailing / (Nick Craig-
47651 Wood)
47652 • Qingstor
47654 • Make rclone cleanup remove pending multipart uploads older than 24h
47656 (Nick Craig-Wood)
47657 • Try harder to cancel failed multipart uploads (Nick Craig-Wood)
47659 • Prune multiUploader.list() (Lars Lehtonen)
47661 • Lint fix (Lars Lehtonen)
47663 • S3
47665 • Add support for --header-upload and --header-download (Tim Gallant)
47667 • Use memory pool for buffer allocations (Maciej Zimnoch)
47669 • Add SSE-C support for AWS, Ceph, and MinIO (Jack Anderson)
47671 • Fail fast multipart upload (Michał Matczuk)
47673 • Report errors on bucket creation (mkdir) correctly (Nick Craig-
47675 Wood)
47676 • Specify that Minio supports URL encoding in listings (Nick Craig-
47678 Wood)
47679 • Added 500 as retryErrorCode (Michał Matczuk)
47681 • Use --low-level-retries as the number of SDK retries (Aleksandar
47683 Janković)
47684 • Fix multipart abort context (Aleksandar Jankovic)
47686 • Replace deprecated session.New() with session.NewSession() (Lars
47688 Lehtonen)
47689 • Use the provided size parameter when allocating a new memory pool
47691 (Joachim Brandon LeBlanc)
47692 • Use rclone's low level retries instead of AWS SDK to fix listing
47694 retries (Nick Craig-Wood)
47695 • Ignore directory markers at the root also (Nick Craig-Wood)
47697 • Use single memory pool (Michał Matczuk)
47699 • Do not resize buf on put to memBuf (Michał Matczuk)
47701 • Improve docs for --s3-disable-checksum (Nick Craig-Wood)
47703 • Don't leak memory or tokens in edge cases for multipart upload
47705 (Nick Craig-Wood)
47706 • Seafile
47708 • Implement 2FA (Fred)
47710 • SFTP
47712 • Added --sftp-pem-key to support inline key files (calisro)
47714 • Fix post transfer copies failing with 0 size when using set_mod‐
47716 time=false (Nick Craig-Wood)
47717 • Sharefile
47719 • Add support for --header-upload and --header-download (Tim Gallant)
47721 • Sugarsync
47723 • Add support for --header-upload and --header-download (Tim Gallant)
47725 • Swift
47727 • Add support for --header-upload and --header-download (Nick Craig-
47729 Wood)
47730 • Fix cosmetic issue in error message (Martin Michlmayr)
47732 • Union
47734 • Implement multiple writable remotes (Max Sum)
47736 • Fix server-side copy (Max Sum)
47738 • Implement ListR (Max Sum)
47740 • Enable ListR when upstreams contain local (Max Sum)
47742 • WebDAV
47744 • Add support for --header-upload and --header-download (Tim Gallant)
47746 • Fix X-OC-Mtime header for Transip compatibility (Nick Craig-Wood)
47748 • Report full and consistent usage with about (Yves G)
47750 • Yandex
47752 • Add support for --header-upload and --header-download (Tim Gallant)
47754 v1.51.0 - 2020-02-01
47756 • New backends
47757 • Memory (https://rclone.org/memory/) (Nick Craig-Wood)
47759 • Sugarsync (https://rclone.org/sugarsync/) (Nick Craig-Wood)
47761 • New Features
47763 • Adjust all backends to have --backend-encoding parameter (Nick
47765 Craig-Wood)
47766 • this enables the encoding for special characters to be adjusted
47768 or disabled
47769 • Add --max-duration flag to control the maximum duration of a trans‐
47771 fer session (boosh)
47772 • Add --expect-continue-timeout flag, default 1s (Nick Craig-Wood)
47774 • Add --no-check-dest flag for copying without testing the destina‐
47776 tion (Nick Craig-Wood)
47777 • Implement --order-by flag to order transfers (Nick Craig-Wood)
47779 • accounting
47781 • Don't show entries in both transferring and checking (Nick Craig-
47783 Wood)
47784 • Add option to delete stats (Aleksandar Jankovic)
47786 • build
47788 • Compress the test builds with gzip (Nick Craig-Wood)
47790 • Implement a framework for starting test servers during tests
47792 (Nick Craig-Wood)
47793 • cmd: Always print elapsed time to tenth place seconds in progress
47795 (Gary Kim)
47796 • config
47798 • Add --password-command to allow dynamic config password (Damon
47800 Permezel)
47801 • Give config questions default values (Nick Craig-Wood)
47803 • Check a remote exists when creating a new one (Nick Craig-Wood)
47805 • copyurl: Add --stdout flag to write to stdout (Nick Craig-Wood)
47807 • dedupe: Implement keep smallest too (Nick Craig-Wood)
47809 • hashsum: Add flag --base64 flag (landall)
47811 • lsf: Speed up on s3/swift/etc by not reading mimetype by default
47813 (Nick Craig-Wood)
47814 • lsjson: Add --no-mimetype flag (Nick Craig-Wood)
47816 • rc: Add methods to turn on blocking and mutex profiling (Nick
47818 Craig-Wood)
47819 • rcd
47821 • Adding group parameter to stats (Chaitanya)
47823 • Move webgui apart; option to disable browser (Xiaoxing Ye)
47825 • serve sftp: Add support for public key with auth proxy (Paul Tins‐
47827 ley)
47828 • stats: Show deletes in stats and hide zero stats (anuar45)
47830 • Bug Fixes
47832 • accounting
47834 • Fix error counter counting multiple times (Ankur Gupta)
47836 • Fix error count shown as checks (Cnly)
47838 • Clear finished transfer in stats-reset (Maciej Zimnoch)
47840 • Added StatsInfo locking in statsGroups sum function (Michał
47842 Matczuk)
47843 • asyncreader: Fix EOF error (buengese)
47845 • check: Fix --one-way recursing more directories than it needs to
47847 (Nick Craig-Wood)
47848 • chunkedreader: Disable hash calculation for first segment (Nick
47850 Craig-Wood)
47851 • config
47853 • Do not open browser on headless on drive/gcs/google photos (Xi‐
47855 aoxing Ye)
47856 • SetValueAndSave ignore error if config section does not exist yet
47858 (buengese)
47859 • cmd: Fix completion with an encrypted config (Danil Semelenov)
47861 • dbhashsum: Stop it returning UNSUPPORTED on dropbox (Nick Craig-
47863 Wood)
47864 • dedupe: Add missing modes to help string (Nick Craig-Wood)
47866 • operations
47868 • Fix dedupe continuing on errors like insufficientFilePersimmon
47870 (SezalAgrawal)
47871 • Clear accounting before low level retry (Maciej Zimnoch)
47873 • Write debug message when hashes could not be checked (Ole Schütt)
47875 • Move interface assertion to tests to remove pflag dependency
47877 (Nick Craig-Wood)
47878 • Make NewOverrideObjectInfo public and factor uses (Nick Craig-
47880 Wood)
47881 • proxy: Replace use of bcrypt with sha256 (Nick Craig-Wood)
47883 • vendor
47885 • Update bazil.org/fuse to fix FreeBSD 12.1 (Nick Craig-Wood)
47887 • Update github.com/t3rm1n4l/go-mega to fix mega "illegal base64
47889 data at input byte 22" (Nick Craig-Wood)
47890 • Update termbox-go to fix ncdu command on FreeBSD (Kuang-che Wu)
47892 • Update t3rm1n4l/go-mega - fixes mega: couldn't login: crypto/aes:
47894 invalid key size 0 (Nick Craig-Wood)
47895 • Mount
47897 • Enable async reads for a 20% speedup (Nick Craig-Wood)
47899 • Replace use of WriteAt with Write for cache mode >= writes and
47901 O_APPEND (Brett Dutro)
47902 • Make sure we call unmount when exiting (Nick Craig-Wood)
47904 • Don't build on go1.10 as bazil/fuse no longer supports it (Nick
47906 Craig-Wood)
47907 • When setting dates discard out of range dates (Nick Craig-Wood)
47909 • VFS
47911 • Add a newly created file straight into the directory (Nick Craig-
47913 Wood)
47914 • Only calculate one hash for reads for a speedup (Nick Craig-Wood)
47916 • Make ReadAt for non cached files work better with non-sequential
47918 reads (Nick Craig-Wood)
47919 • Fix edge cases when reading ModTime from file (Nick Craig-Wood)
47921 • Make sure existing files opened for write show correct size (Nick
47923 Craig-Wood)
47924 • Don't cache the path in RW file objects to fix renaming (Nick
47926 Craig-Wood)
47927 • Fix rename of open files when using the VFS cache (Nick Craig-Wood)
47929 • When renaming files in the cache, rename the cache item in memory
47931 too (Nick Craig-Wood)
47932 • Fix open file renaming on drive when using --vfs-cache-mode writes
47934 (Nick Craig-Wood)
47935 • Fix incorrect modtime for mv into mount with --vfs-cache-modes
47937 writes (Nick Craig-Wood)
47938 • On rename, rename in cache too if the file exists (Anagh Kumar
47940 Baranwal)
47941 • Local
47943 • Make source file being updated errors be NoLowLevelRetry errors
47945 (Nick Craig-Wood)
47946 • Fix update of hidden files on Windows (Nick Craig-Wood)
47948 • Cache
47950 • Follow move of upstream library github.com/coreos/bbolt
47952 github.com/etcd-io/bbolt (Nick Craig-Wood)
47953 • Fix fatal error: concurrent map writes (Nick Craig-Wood)
47955 • Crypt
47957 • Reorder the filename encryption options (Thomas Eales)
47959 • Correctly handle trailing dot (buengese)
47961 • Chunker
47963 • Reduce length of temporary suffix (Ivan Andreev)
47965 • Drive
47967 • Add --drive-stop-on-upload-limit flag to stop syncs when upload
47969 limit reached (Nick Craig-Wood)
47970 • Add --drive-use-shared-date to use date file was shared instead of
47972 modified date (Garry McNulty)
47973 • Make sure invalid auth for teamdrives always reports an error (Nick
47975 Craig-Wood)
47976 • Fix --fast-list when using appDataFolder (Nick Craig-Wood)
47978 • Use multipart resumable uploads for streaming and uploads in mount
47980 (Nick Craig-Wood)
47981 • Log an ERROR if an incomplete search is returned (Nick Craig-Wood)
47983 • Hide dangerous config from the configurator (Nick Craig-Wood)
47985 • Dropbox
47987 • Treat insufficient_space errors as non retriable errors (Nick
47989 Craig-Wood)
47990 • Jottacloud
47992 • Use new auth method used by official client (buengese)
47994 • Add URL to generate Login Token to config wizard (Nick Craig-Wood)
47996 • Add support whitelabel versions (buengese)
47998 • Koofr
48000 • Use rclone HTTP client. (jaKa)
48002 • Onedrive
48004 • Add Sites.Read.All permission (Benjamin Richter)
48006 • Add support "Retry-After" header (Motonori IWAMURO)
48008 • Opendrive
48010 • Implement --opendrive-chunk-size (Nick Craig-Wood)
48012 • S3
48014 • Re-implement multipart upload to fix memory issues (Nick Craig-
48016 Wood)
48017 • Add --s3-copy-cutoff for size to switch to multipart copy (Nick
48019 Craig-Wood)
48020 • Add new region Asia Pacific (Hong Kong) (Outvi V)
48022 • Reduce memory usage streaming files by reducing max stream upload
48024 size (Nick Craig-Wood)
48025 • Add --s3-list-chunk option for bucket listing (Thomas Kriechbaumer)
48027 • Force path style bucket access to off for AWS deprecation (Nick
48029 Craig-Wood)
48030 • Use AWS web identity role provider if available (Tennix)
48032 • Add StackPath Object Storage Support (Dave Koston)
48034 • Fix ExpiryWindow value (Aleksandar Jankovic)
48036 • Fix DisableChecksum condition (Aleksandar Janković)
48038 • Fix URL decoding of NextMarker (Nick Craig-Wood)
48040 • SFTP
48042 • Add --sftp-skip-links to skip symlinks and non regular files (Nick
48044 Craig-Wood)
48045 • Retry Creation of Connection (Sebastian Brandt)
48047 • Fix "failed to parse private key file: ssh: not an encrypted key"
48049 error (Nick Craig-Wood)
48050 • Open files for update write only to fix AWS SFTP interop (Nick
48052 Craig-Wood)
48053 • Swift
48055 • Reserve segments of dynamic large object when delete objects in
48057 container what was enabled versioning. (Nguyễn Hữu Luân)
48058 • Fix parsing of X-Object-Manifest (Nick Craig-Wood)
48060 • Update OVH API endpoint (unbelauscht)
48062 • WebDAV
48064 • Make nextcloud only upload SHA1 checksums (Nick Craig-Wood)
48066 • Fix case of "Bearer" in Authorization: header to agree with RFC
48068 (Nick Craig-Wood)
48069 • Add Referer header to fix problems with WAFs (Nick Craig-Wood)
48071 v1.50.2 - 2019-11-19
48073 • Bug Fixes
48074 • accounting: Fix memory leak on retries operations (Nick Craig-Wood)
48076 • Drive
48078 • Fix listing of the root directory with drive.files scope (Nick
48080 Craig-Wood)
48081 • Fix --drive-root-folder-id with team/shared drives (Nick Craig-
48083 Wood)
48084 v1.50.1 - 2019-11-02
48086 • Bug Fixes
48087 • hash: Fix accidentally changed hash names for DropboxHash and
48089 CRC-32 (Nick Craig-Wood)
48090 • fshttp: Fix error reporting on tpslimit token bucket errors (Nick
48092 Craig-Wood)
48093 • fshttp: Don't print token bucket errors on context cancelled (Nick
48095 Craig-Wood)
48096 • Local
48098 • Fix listings of . on Windows (Nick Craig-Wood)
48100 • Onedrive
48102 • Fix DirMove/Move after Onedrive change (Xiaoxing Ye)
48104 v1.50.0 - 2019-10-26
48106 • New backends
48107 • Citrix Sharefile (https://rclone.org/sharefile/) (Nick Craig-Wood)
48109 • Chunker (https://rclone.org/chunker/) - an overlay backend to split
48111 files into smaller parts (Ivan Andreev)
48112 • Mail.ru Cloud (https://rclone.org/mailru/) (Ivan Andreev)
48114 • New Features
48116 • encodings (Fabian Möller & Nick Craig-Wood)
48118 • All backends now use file name encoding to ensure any file name
48120 can be written to any backend.
48121 • See the restricted file name docs (https://rclone.org/over‐
48123 view/#restricted-filenames) for more info and the local backend
48124 docs.
48125 • Some file names may look different in rclone if you are using any
48127 control characters in names or unicode FULLWIDTH symbols
48128 (https://en.wikipedia.org/wiki/Halfwidth_and_Full‐
48129 width_Forms_(Unicode_block)).
48130 • build
48132 • Update to use go1.13 for the build (Nick Craig-Wood)
48134 • Drop support for go1.9 (Nick Craig-Wood)
48136 • Build rclone with GitHub actions (Nick Craig-Wood)
48138 • Convert python scripts to python3 (Nick Craig-Wood)
48140 • Swap Azure/go-ansiterm for mattn/go-colorable (Nick Craig-Wood)
48142 • Dockerfile fixes (Matei David)
48144 • Add plugin support (https://github.com/rclone/rclone/blob/mas‐
48146 ter/CONTRIBUTING.md#writing-a-plugin) for backends and commands
48147 (Richard Patel)
48148 • config
48150 • Use alternating Red/Green in config to make more obvious (Nick
48152 Craig-Wood)
48153 • contrib
48155 • Add sample DLNA server Docker Compose manifest. (pataquets)
48157 • Add sample WebDAV server Docker Compose manifest. (pataquets)
48159 • copyurl
48161 • Add --auto-filename flag for using file name from URL in destina‐
48163 tion path (Denis)
48164 • serve dlna:
48166 • Many compatibility improvements (Dan Walters)
48168 • Support for external srt subtitles (Dan Walters)
48170 • rc
48172 • Added command core/quit (Saksham Khanna)
48174 • Bug Fixes
48176 • sync
48178 • Make --update/-u not transfer files that haven't changed (Nick
48180 Craig-Wood)
48181 • Free objects after they come out of the transfer pipe to save
48183 memory (Nick Craig-Wood)
48184 • Fix --files-from without --no-traverse doing a recursive scan
48186 (Nick Craig-Wood)
48187 • operations
48189 • Fix accounting for server-side copies (Nick Craig-Wood)
48191 • Display 'All duplicates removed' only if dedupe successful (Sezal
48193 Agrawal)
48194 • Display 'Deleted X extra copies' only if dedupe successful (Sezal
48196 Agrawal)
48197 • accounting
48199 • Only allow up to 100 completed transfers in the accounting list
48201 to save memory (Nick Craig-Wood)
48202 • Cull the old time ranges when possible to save memory (Nick
48204 Craig-Wood)
48205 • Fix panic due to server-side copy fallback (Ivan Andreev)
48207 • Fix memory leak noticeable for transfers of large numbers of ob‐
48209 jects (Nick Craig-Wood)
48210 • Fix total duration calculation (Nick Craig-Wood)
48212 • cmd
48214 • Fix environment variables not setting command line flags (Nick
48216 Craig-Wood)
48217 • Make autocomplete compatible with bash's posix mode for macOS
48219 (Danil Semelenov)
48220 • Make --progress work in git bash on Windows (Nick Craig-Wood)
48222 • Fix 'compopt: command not found' on autocomplete on macOS (Danil
48224 Semelenov)
48225 • config
48227 • Fix setting of non top level flags from environment variables
48229 (Nick Craig-Wood)
48230 • Check config names more carefully and report errors (Nick Craig-
48232 Wood)
48233 • Remove error: can't use --size-only and --ignore-size together.
48235 (Nick Craig-Wood)
48236 • filter: Prevent mixing options when --files-from is in use (Michele
48238 Caci)
48239 • serve sftp: Fix crash on unsupported operations (e.g. Readlink)
48241 (Nick Craig-Wood)
48242 • Mount
48244 • Allow files of unknown size to be read properly (Nick Craig-Wood)
48246 • Skip tests on <= 2 CPUs to avoid lockup (Nick Craig-Wood)
48248 • Fix panic on File.Open (Nick Craig-Wood)
48250 • Fix "mount_fusefs: -o timeout=: option not supported" on FreeBSD
48252 (Nick Craig-Wood)
48253 • Don't pass huge filenames (>4k) to FUSE as it can't cope (Nick
48255 Craig-Wood)
48256 • VFS
48258 • Add flag --vfs-case-insensitive for windows/macOS mounts (Ivan An‐
48260 dreev)
48261 • Make objects of unknown size readable through the VFS (Nick Craig-
48263 Wood)
48264 • Move writeback of dirty data out of close() method into its own
48266 method (FlushWrites) and remove close() call from Flush() (Brett
48267 Dutro)
48268 • Stop empty dirs disappearing when renamed on bucket-based remotes
48270 (Nick Craig-Wood)
48271 • Stop change notify polling clearing so much of the directory cache
48273 (Nick Craig-Wood)
48274 • Azure Blob
48276 • Disable logging to the Windows event log (Nick Craig-Wood)
48278 • B2
48280 • Remove unverified: prefix on sha1 to improve interop (e.g. with
48282 CyberDuck) (Nick Craig-Wood)
48283 • Box
48285 • Add options to get access token via JWT auth (David)
48287 • Drive
48289 • Disable HTTP/2 by default to work around INTERNAL_ERROR problems
48291 (Nick Craig-Wood)
48292 • Make sure that drive root ID is always canonical (Nick Craig-Wood)
48294 • Fix --drive-shared-with-me from the root with lsand --fast-list
48296 (Nick Craig-Wood)
48297 • Fix ChangeNotify polling for shared drives (Nick Craig-Wood)
48299 • Fix change notify polling when using appDataFolder (Nick Craig-
48301 Wood)
48302 • Dropbox
48304 • Make disallowed filenames errors not retry (Nick Craig-Wood)
48306 • Fix nil pointer exception on restricted files (Nick Craig-Wood)
48308 • Fichier
48310 • Fix accessing files > 2GB on 32 bit systems (Nick Craig-Wood)
48312 • FTP
48314 • Allow disabling EPSV mode (Jon Fautley)
48316 • HTTP
48318 • HEAD directory entries in parallel to speedup (Nick Craig-Wood)
48320 • Add --http-no-head to stop rclone doing HEAD in listings (Nick
48322 Craig-Wood)
48323 • Putio
48325 • Add ability to resume uploads (Cenk Alti)
48327 • S3
48329 • Fix signature v2_auth headers (Anthony Rusdi)
48331 • Fix encoding for control characters (Nick Craig-Wood)
48333 • Only ask for URL encoded directory listings if we need them on Ceph
48335 (Nick Craig-Wood)
48336 • Add option for multipart failure behaviour (Aleksandar Jankovic)
48338 • Support for multipart copy (庄天翼)
48340 • Fix nil pointer reference if no metadata returned for object (Nick
48342 Craig-Wood)
48343 • SFTP
48345 • Fix --sftp-ask-password trying to contact the ssh agent (Nick
48347 Craig-Wood)
48348 • Fix hashes of files with backslashes (Nick Craig-Wood)
48350 • Include more ciphers with --sftp-use-insecure-cipher (Carlos Fer‐
48352 reyra)
48353 • WebDAV
48355 • Parse and return Sharepoint error response (Henning Surmeier)
48357 v1.49.5 - 2019-10-05
48359 • Bug Fixes
48360 • Revert back to go1.12.x for the v1.49.x builds as go1.13.x was
48362 causing issues (Nick Craig-Wood)
48363 • Fix rpm packages by using master builds of nfpm (Nick Craig-Wood)
48365 • Fix macOS build after brew changes (Nick Craig-Wood)
48367 v1.49.4 - 2019-09-29
48369 • Bug Fixes
48370 • cmd/rcd: Address ZipSlip vulnerability (Richard Patel)
48372 • accounting: Fix file handle leak on errors (Nick Craig-Wood)
48374 • oauthutil: Fix security problem when running with two users on the
48376 same machine (Nick Craig-Wood)
48377 • FTP
48379 • Fix listing of an empty root returning: error dir not found (Nick
48381 Craig-Wood)
48382 • S3
48384 • Fix SetModTime on GLACIER/ARCHIVE objects and implement set/get
48386 tier (Nick Craig-Wood)
48387 v1.49.3 - 2019-09-15
48389 • Bug Fixes
48390 • accounting
48392 • Fix total duration calculation (Aleksandar Jankovic)
48394 • Fix "file already closed" on transfer retries (Nick Craig-Wood)
48396 v1.49.2 - 2019-09-08
48398 • New Features
48399 • build: Add Docker workflow support (Alfonso Montero)
48401 • Bug Fixes
48403 • accounting: Fix locking in Transfer to avoid deadlock with
48405 --progress (Nick Craig-Wood)
48406 • docs: Fix template argument for mktemp in install.sh (Cnly)
48408 • operations: Fix -u/--update with google photos / files of unknown
48410 size (Nick Craig-Wood)
48411 • rc: Fix docs for config/create /update /password (Nick Craig-Wood)
48413 • Google Cloud Storage
48415 • Fix need for elevated permissions on SetModTime (Nick Craig-Wood)
48417 v1.49.1 - 2019-08-28
48419 • Bug Fixes
48420 • config: Fix generated passwords being stored as empty password
48422 (Nick Craig-Wood)
48423 • rcd: Added missing parameter for web-gui info logs. (Chaitanya)
48425 • Googlephotos
48427 • Fix crash on error response (Nick Craig-Wood)
48429 • Onedrive
48431 • Fix crash on error response (Nick Craig-Wood)
48433 v1.49.0 - 2019-08-26
48435 • New backends
48436 • 1fichier (https://rclone.org/fichier/) (Laura Hausmann)
48438 • Google Photos (https://rclone.org/googlephotos/) (Nick Craig-Wood)
48440 • Putio (https://rclone.org/putio/) (Cenk Alti)
48442 • premiumize.me (https://rclone.org/premiumizeme/) (Nick Craig-Wood)
48444 • New Features
48446 • Experimental web GUI (https://rclone.org/gui/) (Chaitanya Bankan‐
48448 hal)
48449 • Implement --compare-dest & --copy-dest (yparitcher)
48451 • Implement --suffix without --backup-dir for backup to current dir
48453 (yparitcher)
48454 • config reconnect to re-login (re-run the oauth login) for the back‐
48456 end. (Nick Craig-Wood)
48457 • config userinfo to discover which user you are logged in as. (Nick
48459 Craig-Wood)
48460 • config disconnect to disconnect you (log out) from the backend.
48462 (Nick Craig-Wood)
48463 • Add --use-json-log for JSON logging (justinalin)
48465 • Add context propagation to rclone (Aleksandar Jankovic)
48467 • Reworking internal statistics interfaces so they work with rc jobs
48469 (Aleksandar Jankovic)
48470 • Add Higher units for ETA (AbelThar)
48472 • Update rclone logos to new design (Andreas Chlupka)
48474 • hash: Add CRC-32 support (Cenk Alti)
48476 • help showbackend: Fixed advanced option category when there are no
48478 standard options (buengese)
48479 • ncdu: Display/Copy to Clipboard Current Path (Gary Kim)
48481 • operations:
48483 • Run hashing operations in parallel (Nick Craig-Wood)
48485 • Don't calculate checksums when using --ignore-checksum (Nick
48487 Craig-Wood)
48488 • Check transfer hashes when using --size-only mode (Nick Craig-
48490 Wood)
48491 • Disable multi thread copy for local to local copies (Nick Craig-
48493 Wood)
48494 • Debug successful hashes as well as failures (Nick Craig-Wood)
48496 • rc
48498 • Add ability to stop async jobs (Aleksandar Jankovic)
48500 • Return current settings if core/bwlimit called without parameters
48502 (Nick Craig-Wood)
48503 • Rclone-WebUI integration with rclone (Chaitanya Bankanhal)
48505 • Added command line parameter to control the cross origin resource
48507 sharing (CORS) in the rcd. (Security Improvement) (Chaitanya
48508 Bankanhal)
48509 • Add anchor tags to the docs so links are consistent (Nick Craig-
48511 Wood)
48512 • Remove _async key from input parameters after parsing so later
48514 operations won't get confused (buengese)
48515 • Add call to clear stats (Aleksandar Jankovic)
48517 • rcd
48519 • Auto-login for web-gui (Chaitanya Bankanhal)
48521 • Implement --baseurl for rcd and web-gui (Chaitanya Bankanhal)
48523 • serve dlna
48525 • Only select interfaces which can multicast for SSDP (Nick Craig-
48527 Wood)
48528 • Add more builtin mime types to cover standard audio/video (Nick
48530 Craig-Wood)
48531 • Fix missing mime types on Android causing missing videos (Nick
48533 Craig-Wood)
48534 • serve ftp
48536 • Refactor to bring into line with other serve commands (Nick
48538 Craig-Wood)
48539 • Implement --auth-proxy (Nick Craig-Wood)
48541 • serve http: Implement --baseurl (Nick Craig-Wood)
48543 • serve restic: Implement --baseurl (Nick Craig-Wood)
48545 • serve sftp
48547 • Implement auth proxy (Nick Craig-Wood)
48549 • Fix detection of whether server is authorized (Nick Craig-Wood)
48551 • serve webdav
48553 • Implement --baseurl (Nick Craig-Wood)
48555 • Support --auth-proxy (Nick Craig-Wood)
48557 • Bug Fixes
48559 • Make "bad record MAC" a retriable error (Nick Craig-Wood)
48561 • copyurl: Fix copying files that return HTTP errors (Nick Craig-
48563 Wood)
48564 • march: Fix checking sub-directories when using --no-traverse
48566 (buengese)
48567 • rc
48569 • Fix unmarshalable http.AuthFn in options and put in test for mar‐
48571 shalability (Nick Craig-Wood)
48572 • Move job expire flags to rc to fix initialization problem (Nick
48574 Craig-Wood)
48575 • Fix --loopback with rc/list and others (Nick Craig-Wood)
48577 • rcat: Fix slowdown on systems with multiple hashes (Nick Craig-
48579 Wood)
48580 • rcd: Fix permissions problems on cache directory with web gui down‐
48582 load (Nick Craig-Wood)
48583 • Mount
48585 • Default --daemon-timeout to 15 minutes on macOS and FreeBSD (Nick
48587 Craig-Wood)
48588 • Update docs to show mounting from root OK for bucket-based (Nick
48590 Craig-Wood)
48591 • Remove nonseekable flag from write files (Nick Craig-Wood)
48593 • VFS
48595 • Make write without cache more efficient (Nick Craig-Wood)
48597 • Fix --vfs-cache-mode minimal and writes ignoring cached files (Nick
48599 Craig-Wood)
48600 • Local
48602 • Add --local-case-sensitive and --local-case-insensitive (Nick
48604 Craig-Wood)
48605 • Avoid polluting page cache when uploading local files to remote
48607 backends (Michał Matczuk)
48608 • Don't calculate any hashes by default (Nick Craig-Wood)
48610 • Fadvise run syscall on a dedicated go routine (Michał Matczuk)
48612 • Azure Blob
48614 • Azure Storage Emulator support (Sandeep)
48616 • Updated config help details to remove connection string references
48618 (Sandeep)
48619 • Make all operations work from the root (Nick Craig-Wood)
48621 • B2
48623 • Implement link sharing (yparitcher)
48625 • Enable server-side copy to copy between buckets (Nick Craig-Wood)
48627 • Make all operations work from the root (Nick Craig-Wood)
48629 • Drive
48631 • Fix server-side copy of big files (Nick Craig-Wood)
48633 • Update API for teamdrive use (Nick Craig-Wood)
48635 • Add error for purge with --drive-trashed-only (ginvine)
48637 • Fichier
48639 • Make FolderID int and adjust related code (buengese)
48641 • Google Cloud Storage
48643 • Reduce oauth scope requested as suggested by Google (Nick Craig-
48645 Wood)
48646 • Make all operations work from the root (Nick Craig-Wood)
48648 • HTTP
48650 • Add --http-headers flag for setting arbitrary headers (Nick Craig-
48652 Wood)
48653 • Jottacloud
48655 • Use new api for retrieving internal username (buengese)
48657 • Refactor configuration and minor cleanup (buengese)
48659 • Koofr
48661 • Support setting modification times on Koofr backend. (jaKa)
48663 • Opendrive
48665 • Refactor to use existing lib/rest facilities for uploads (Nick
48667 Craig-Wood)
48668 • Qingstor
48670 • Upgrade to v3 SDK and fix listing loop (Nick Craig-Wood)
48672 • Make all operations work from the root (Nick Craig-Wood)
48674 • S3
48676 • Add INTELLIGENT_TIERING storage class (Matti Niemenmaa)
48678 • Make all operations work from the root (Nick Craig-Wood)
48680 • SFTP
48682 • Add missing interface check and fix About (Nick Craig-Wood)
48684 • Completely ignore all modtime checks if SetModTime=false (Jon Faut‐
48686 ley)
48687 • Support md5/sha1 with rsync.net (Nick Craig-Wood)
48689 • Save the md5/sha1 command in use to the config file for efficiency
48691 (Nick Craig-Wood)
48692 • Opt-in support for diffie-hellman-group-exchange-sha256 diffie-
48694 hellman-group-exchange-sha1 (Yi FU)
48695 • Swift
48697 • Use FixRangeOption to fix 0 length files via the VFS (Nick Craig-
48699 Wood)
48700 • Fix upload when using no_chunk to return the correct size (Nick
48702 Craig-Wood)
48703 • Make all operations work from the root (Nick Craig-Wood)
48705 • Fix segments leak during failed large file uploads. (nguyenhuulu‐
48707 an434)
48708 • WebDAV
48710 • Add --webdav-bearer-token-command (Nick Craig-Wood)
48712 • Refresh token when it expires with --webdav-bearer-token-command
48714 (Nick Craig-Wood)
48715 • Add docs for using bearer_token_command with oidc-agent (Paul Mil‐
48717 lar)
48718 v1.48.0 - 2019-06-15
48720 • New commands
48721 • serve sftp: Serve an rclone remote over SFTP (Nick Craig-Wood)
48723 • New Features
48725 • Multi threaded downloads to local storage (Nick Craig-Wood)
48727 • controlled with --multi-thread-cutoff and --multi-thread-streams
48729 • Use rclone.conf from rclone executable directory to enable portable
48731 use (albertony)
48732 • Allow sync of a file and a directory with the same name (forgems)
48734 • this is common on bucket-based remotes, e.g. s3, gcs
48736 • Add --ignore-case-sync for forced case insensitivity (garry415)
48738 • Implement --stats-one-line-date and --stats-one-line-date-format
48740 (Peter Berbec)
48741 • Log an ERROR for all commands which exit with non-zero status (Nick
48743 Craig-Wood)
48744 • Use go-homedir to read the home directory more reliably (Nick
48746 Craig-Wood)
48747 • Enable creating encrypted config through external script invocation
48749 (Wojciech Smigielski)
48750 • build: Drop support for go1.8 (Nick Craig-Wood)
48752 • config: Make config create/update encrypt passwords where necessary
48754 (Nick Craig-Wood)
48755 • copyurl: Honor --no-check-certificate (Stefan Breunig)
48757 • install: Linux skip man pages if no mandb (didil)
48759 • lsf: Support showing the Tier of the object (Nick Craig-Wood)
48761 • lsjson
48763 • Added EncryptedPath to output (calisro)
48765 • Support showing the Tier of the object (Nick Craig-Wood)
48767 • Add IsBucket field for bucket-based remote listing of the root
48769 (Nick Craig-Wood)
48770 • rc
48772 • Add --loopback flag to run commands directly without a server
48774 (Nick Craig-Wood)
48775 • Add operations/fsinfo: Return information about the remote (Nick
48777 Craig-Wood)
48778 • Skip auth for OPTIONS request (Nick Craig-Wood)
48780 • cmd/providers: Add DefaultStr, ValueStr and Type fields (Nick
48782 Craig-Wood)
48783 • jobs: Make job expiry timeouts configurable (Aleksandar Jankovic)
48785 • serve dlna reworked and improved (Dan Walters)
48787 • serve ftp: add --ftp-public-ip flag to specify public IP (calistri)
48789 • serve restic: Add support for --private-repos in serve restic (Flo‐
48791 rian Apolloner)
48792 • serve webdav: Combine serve webdav and serve http (Gary Kim)
48794 • size: Ignore negative sizes when calculating total (Garry McNulty)
48796 • Bug Fixes
48798 • Make move and copy individual files obey --backup-dir (Nick Craig-
48800 Wood)
48801 • If --ignore-checksum is in effect, don't calculate checksum (Nick
48803 Craig-Wood)
48804 • moveto: Fix case-insensitive same remote move (Gary Kim)
48806 • rc: Fix serving bucket-based objects with --rc-serve (Nick Craig-
48808 Wood)
48809 • serve webdav: Fix serveDir not being updated with changes from web‐
48811 dav (Gary Kim)
48812 • Mount
48814 • Fix poll interval documentation (Animosity022)
48816 • VFS
48818 • Make WriteAt for non cached files work with non-sequential writes
48820 (Nick Craig-Wood)
48821 • Local
48823 • Only calculate the required hashes for big speedup (Nick Craig-
48825 Wood)
48826 • Log errors when listing instead of returning an error (Nick Craig-
48828 Wood)
48829 • Fix preallocate warning on Linux with ZFS (Nick Craig-Wood)
48831 • Crypt
48833 • Make rclone dedupe work through crypt (Nick Craig-Wood)
48835 • Fix wrapping of ChangeNotify to decrypt directories properly (Nick
48837 Craig-Wood)
48838 • Support PublicLink (rclone link) of underlying backend (Nick Craig-
48840 Wood)
48841 • Implement Optional methods SetTier, GetTier (Nick Craig-Wood)
48843 • B2
48845 • Implement server-side copy (Nick Craig-Wood)
48847 • Implement SetModTime (Nick Craig-Wood)
48849 • Drive
48851 • Fix move and copy from TeamDrive to GDrive (Fionera)
48853 • Add notes that cleanup works in the background on drive (Nick
48855 Craig-Wood)
48856 • Add --drive-server-side-across-configs to default back to old serv‐
48858 er-side copy semantics by default (Nick Craig-Wood)
48859 • Add --drive-size-as-quota to show storage quota usage for file size
48861 (Garry McNulty)
48862 • FTP
48864 • Add FTP List timeout (Jeff Quinn)
48866 • Add FTP over TLS support (Gary Kim)
48868 • Add --ftp-no-check-certificate option for FTPS (Gary Kim)
48870 • Google Cloud Storage
48872 • Fix upload errors when uploading pre 1970 files (Nick Craig-Wood)
48874 • Jottacloud
48876 • Add support for selecting device and mountpoint. (buengese)
48878 • Mega
48880 • Add cleanup support (Gary Kim)
48882 • Onedrive
48884 • More accurately check if root is found (Cnly)
48886 • S3
48888 • Support S3 Accelerated endpoints with --s3-use-accelerate-endpoint
48890 (Nick Craig-Wood)
48891 • Add config info for Wasabi's EU Central endpoint (Robert Marko)
48893 • Make SetModTime work for GLACIER while syncing (Philip Harvey)
48895 • SFTP
48897 • Add About support (Gary Kim)
48899 • Fix about parsing of df results so it can cope with -ve results
48901 (Nick Craig-Wood)
48902 • Send custom client version and debug server version (Nick Craig-
48904 Wood)
48905 • WebDAV
48907 • Retry on 423 Locked errors (Nick Craig-Wood)
48909 v1.47.0 - 2019-04-13
48911 • New backends
48912 • Backend for Koofr cloud storage service. (jaKa)
48914 • New Features
48916 • Resume downloads if the reader fails in copy (Nick Craig-Wood)
48918 • this means rclone will restart transfers if the source has an er‐
48920 ror
48921 • this is most useful for downloads or cloud to cloud copies
48923 • Use --fast-list for listing operations where it won't use more mem‐
48925 ory (Nick Craig-Wood)
48926 • this should speed up the following operations on remotes which
48928 support ListR
48929 • dedupe, serve restic lsf, ls, lsl, lsjson, lsd, md5sum, sha1sum,
48931 hashsum, size, delete, cat, settier
48932 • use --disable ListR to get old behaviour if required
48934 • Make --files-from traverse the destination unless --no-traverse is
48936 set (Nick Craig-Wood)
48937 • this fixes --files-from with Google drive and excessive API use
48939 in general.
48940 • Make server-side copy account bytes and obey --max-transfer (Nick
48942 Craig-Wood)
48943 • Add --create-empty-src-dirs flag and default to not creating empty
48945 dirs (ishuah)
48946 • Add client side TLS/SSL flags --ca-cert/--client-cert/--client-key
48948 (Nick Craig-Wood)
48949 • Implement --suffix-keep-extension for use with --suffix (Nick
48951 Craig-Wood)
48952 • build:
48954 • Switch to semver compliant version tags to be go modules compli‐
48956 ant (Nick Craig-Wood)
48957 • Update to use go1.12.x for the build (Nick Craig-Wood)
48959 • serve dlna: Add connection manager service description to improve
48961 compatibility (Dan Walters)
48962 • lsf: Add 'e' format to show encrypted names and 'o' for original
48964 IDs (Nick Craig-Wood)
48965 • lsjson: Added --files-only and --dirs-only flags (calistri)
48967 • rc: Implement operations/publiclink the equivalent of rclone link
48969 (Nick Craig-Wood)
48970 • Bug Fixes
48972 • accounting: Fix total ETA when --stats-unit bits is in effect (Nick
48974 Craig-Wood)
48975 • Bash TAB completion
48977 • Use private custom func to fix clash between rclone and kubectl
48979 (Nick Craig-Wood)
48980 • Fix for remotes with underscores in their names (Six)
48982 • Fix completion of remotes (Florian Gamböck)
48984 • Fix autocompletion of remote paths with spaces (Danil Semelenov)
48986 • serve dlna: Fix root XML service descriptor (Dan Walters)
48988 • ncdu: Fix display corruption with Chinese characters (Nick Craig-
48990 Wood)
48991 • Add SIGTERM to signals which run the exit handlers on unix (Nick
48993 Craig-Wood)
48994 • rc: Reload filter when the options are set via the rc (Nick Craig-
48996 Wood)
48997 • VFS / Mount
48999 • Fix FreeBSD: Ignore Truncate if called with no readers and already
49001 the correct size (Nick Craig-Wood)
49002 • Read directory and check for a file before mkdir (Nick Craig-Wood)
49004 • Shorten the locking window for vfs/refresh (Nick Craig-Wood)
49006 • Azure Blob
49008 • Enable MD5 checksums when uploading files bigger than the "Cutoff"
49010 (Dr.Rx)
49011 • Fix SAS URL support (Nick Craig-Wood)
49013 • B2
49015 • Allow manual configuration of backblaze downloadUrl (Vince)
49017 • Ignore already_hidden error on remove (Nick Craig-Wood)
49019 • Ignore malformed src_last_modified_millis (Nick Craig-Wood)
49021 • Drive
49023 • Add --skip-checksum-gphotos to ignore incorrect checksums on Google
49025 Photos (Nick Craig-Wood)
49026 • Allow server-side move/copy between different remotes. (Fionera)
49028 • Add docs on team drives and --fast-list eventual consistency (Nes‐
49030 tar47)
49031 • Fix imports of text files (Nick Craig-Wood)
49033 • Fix range requests on 0 length files (Nick Craig-Wood)
49035 • Fix creation of duplicates with server-side copy (Nick Craig-Wood)
49037 • Dropbox
49039 • Retry blank errors to fix long listings (Nick Craig-Wood)
49041 • FTP
49043 • Add --ftp-concurrency to limit maximum number of connections (Nick
49045 Craig-Wood)
49046 • Google Cloud Storage
49048 • Fall back to default application credentials (marcintustin)
49050 • Allow bucket policy only buckets (Nick Craig-Wood)
49052 • HTTP
49054 • Add --http-no-slash for websites with directories with no slashes
49056 (Nick Craig-Wood)
49057 • Remove duplicates from listings (Nick Craig-Wood)
49059 • Fix socket leak on 404 errors (Nick Craig-Wood)
49061 • Jottacloud
49063 • Fix token refresh (Sebastian Bünger)
49065 • Add device registration (Oliver Heyme)
49067 • Onedrive
49069 • Implement graceful cancel of multipart uploads if rclone is inter‐
49071 rupted (Cnly)
49072 • Always add trailing colon to path when addressing items, (Cnly)
49074 • Return errors instead of panic for invalid uploads (Fabian Möller)
49076 • S3
49078 • Add support for "Glacier Deep Archive" storage class (Manu)
49080 • Update Dreamhost endpoint (Nick Craig-Wood)
49082 • Note incompatibility with CEPH Jewel (Nick Craig-Wood)
49084 • SFTP
49086 • Allow custom ssh client config (Alexandru Bumbacea)
49088 • Swift
49090 • Obey Retry-After to enable OVH restore from cold storage (Nick
49092 Craig-Wood)
49093 • Work around token expiry on CEPH (Nick Craig-Wood)
49095 • WebDAV
49097 • Allow IsCollection property to be integer or boolean (Nick Craig-
49099 Wood)
49100 • Fix race when creating directories (Nick Craig-Wood)
49102 • Fix About/df when reading the available/total returns 0 (Nick
49104 Craig-Wood)
49105 v1.46 - 2019-02-09
49107 • New backends
49108 • Support Alibaba Cloud (Aliyun) OSS via the s3 backend (Nick Craig-
49110 Wood)
49111 • New commands
49113 • serve dlna: serves a remove via DLNA for the local network (ni‐
49115 colov)
49116 • New Features
49118 • copy, move: Restore deprecated --no-traverse flag (Nick Craig-Wood)
49120 • This is useful for when transferring a small number of files into
49122 a large destination
49123 • genautocomplete: Add remote path completion for bash completion
49125 (Christopher Peterson & Danil Semelenov)
49126 • Buffer memory handling reworked to return memory to the OS better
49128 (Nick Craig-Wood)
49129 • Buffer recycling library to replace sync.Pool
49131 • Optionally use memory mapped memory for better memory shrinking
49133 • Enable with --use-mmap if having memory problems - not default
49135 yet
49136 • Parallelise reading of files specified by --files-from (Nick Craig-
49138 Wood)
49139 • check: Add stats showing total files matched. (Dario Guzik)
49141 • Allow rename/delete open files under Windows (Nick Craig-Wood)
49143 • lsjson: Use exactly the correct number of decimal places in the
49145 seconds (Nick Craig-Wood)
49146 • Add cookie support with cmdline switch --use-cookies for all HTTP
49148 based remotes (qip)
49149 • Warn if --checksum is set but there are no hashes available (Nick
49151 Craig-Wood)
49152 • Rework rate limiting (pacer) to be more accurate and allow bursting
49154 (Nick Craig-Wood)
49155 • Improve error reporting for too many/few arguments in commands
49157 (Nick Craig-Wood)
49158 • listremotes: Remove -l short flag as it conflicts with the new
49160 global flag (weetmuts)
49161 • Make http serving with auth generate INFO messages on auth fail
49163 (Nick Craig-Wood)
49164 • Bug Fixes
49166 • Fix layout of stats (Nick Craig-Wood)
49168 • Fix --progress crash under Windows Jenkins (Nick Craig-Wood)
49170 • Fix transfer of google/onedrive docs by calling Rcat in Copy when
49172 size is -1 (Cnly)
49173 • copyurl: Fix checking of --dry-run (Denis Skovpen)
49175 • Mount
49177 • Check that mountpoint and local directory to mount don't overlap
49179 (Nick Craig-Wood)
49180 • Fix mount size under 32 bit Windows (Nick Craig-Wood)
49182 • VFS
49184 • Implement renaming of directories for backends without DirMove
49186 (Nick Craig-Wood)
49187 • now all backends except b2 support renaming directories
49189 • Implement --vfs-cache-max-size to limit the total size of the cache
49191 (Nick Craig-Wood)
49192 • Add --dir-perms and --file-perms flags to set default permissions
49194 (Nick Craig-Wood)
49195 • Fix deadlock on concurrent operations on a directory (Nick Craig-
49197 Wood)
49198 • Fix deadlock between RWFileHandle.close and File.Remove (Nick
49200 Craig-Wood)
49201 • Fix renaming/deleting open files with cache mode "writes" under
49203 Windows (Nick Craig-Wood)
49204 • Fix panic on rename with --dry-run set (Nick Craig-Wood)
49206 • Fix vfs/refresh with recurse=true needing the --fast-list flag
49208 • Local
49210 • Add support for -l/--links (symbolic link translation) (yair@uni‐
49212 corn)
49213 • this works by showing links as link.rclonelink - see local back‐
49215 end docs for more info
49216 • this errors if used with -L/--copy-links
49218 • Fix renaming/deleting open files on Windows (Nick Craig-Wood)
49220 • Crypt
49222 • Check for maximum length before decrypting filename to fix panic
49224 (Garry McNulty)
49225 • Azure Blob
49227 • Allow building azureblob backend on *BSD (themylogin)
49229 • Use the rclone HTTP client to support --dump headers, --tpslimit,
49231 etc. (Nick Craig-Wood)
49232 • Use the s3 pacer for 0 delay in non error conditions (Nick Craig-
49234 Wood)
49235 • Ignore directory markers (Nick Craig-Wood)
49237 • Stop Mkdir attempting to create existing containers (Nick Craig-
49239 Wood)
49240 • B2
49242 • cleanup: will remove unfinished large files >24hrs old (Garry Mc‐
49244 Nulty)
49245 • For a bucket limited application key check the bucket name (Nick
49247 Craig-Wood)
49248 • before this, rclone would use the authorised bucket regardless of
49250 what you put on the command line
49251 • Added --b2-disable-checksum flag (Wojciech Smigielski)
49253 • this enables large files to be uploaded without a SHA-1 hash for
49255 speed reasons
49256 • Drive
49258 • Set default pacer to 100ms for 10 tps (Nick Craig-Wood)
49260 • This fits the Google defaults much better and reduces the 403 er‐
49262 rors massively
49263 • Add --drive-pacer-min-sleep and --drive-pacer-burst to control
49265 the pacer
49266 • Improve ChangeNotify support for items with multiple parents (Fabi‐
49268 an Möller)
49269 • Fix ListR for items with multiple parents - this fixes oddities
49271 with vfs/refresh (Fabian Möller)
49272 • Fix using --drive-impersonate and appfolders (Nick Craig-Wood)
49274 • Fix google docs in rclone mount for some (not all) applications
49276 (Nick Craig-Wood)
49277 • Dropbox
49279 • Retry-After support for Dropbox backend (Mathieu Carbou)
49281 • FTP
49283 • Wait for 60 seconds for a connection to Close then declare it dead
49285 (Nick Craig-Wood)
49286 • helps with indefinite hangs on some FTP servers
49288 • Google Cloud Storage
49290 • Update google cloud storage endpoints (weetmuts)
49292 • HTTP
49294 • Add an example with username and password which is supported but
49296 wasn't documented (Nick Craig-Wood)
49297 • Fix backend with --files-from and nonexistent files (Nick Craig-
49299 Wood)
49300 • Hubic
49302 • Make error message more informative if authentication fails (Nick
49304 Craig-Wood)
49305 • Jottacloud
49307 • Resume and deduplication support (Oliver Heyme)
49309 • Use token auth for all API requests Don't store password anymore
49311 (Sebastian Bünger)
49312 • Add support for 2-factor authentication (Sebastian Bünger)
49314 • Mega
49316 • Implement v2 account login which fixes logins for newer Mega ac‐
49318 counts (Nick Craig-Wood)
49319 • Return error if an unknown length file is attempted to be uploaded
49321 (Nick Craig-Wood)
49322 • Add new error codes for better error reporting (Nick Craig-Wood)
49324 • Onedrive
49326 • Fix broken support for "shared with me" folders (Alex Chen)
49328 • Fix root ID not normalised (Cnly)
49330 • Return err instead of panic on unknown-sized uploads (Cnly)
49332 • Qingstor
49334 • Fix go routine leak on multipart upload errors (Nick Craig-Wood)
49336 • Add upload chunk size/concurrency/cutoff control (Nick Craig-Wood)
49338 • Default --qingstor-upload-concurrency to 1 to work around bug (Nick
49340 Craig-Wood)
49341 • S3
49343 • Implement --s3-upload-cutoff for single part uploads below this
49345 (Nick Craig-Wood)
49346 • Change --s3-upload-concurrency default to 4 to increase performance
49348 (Nick Craig-Wood)
49349 • Add --s3-bucket-acl to control bucket ACL (Nick Craig-Wood)
49351 • Auto detect region for buckets on operation failure (Nick Craig-
49353 Wood)
49354 • Add GLACIER storage class (William Cocker)
49356 • Add Scaleway to s3 documentation (Rémy Léone)
49358 • Add AWS endpoint eu-north-1 (weetmuts)
49360 • SFTP
49362 • Add support for PEM encrypted private keys (Fabian Möller)
49364 • Add option to force the usage of an ssh-agent (Fabian Möller)
49366 • Perform environment variable expansion on key-file (Fabian Möller)
49368 • Fix rmdir on Windows based servers (e.g. CrushFTP) (Nick Craig-
49370 Wood)
49371 • Fix rmdir deleting directory contents on some SFTP servers (Nick
49373 Craig-Wood)
49374 • Fix error on dangling symlinks (Nick Craig-Wood)
49376 • Swift
49378 • Add --swift-no-chunk to disable segmented uploads in rcat/mount
49380 (Nick Craig-Wood)
49381 • Introduce application credential auth support (kayrus)
49383 • Fix memory usage by slimming Object (Nick Craig-Wood)
49385 • Fix extra requests on upload (Nick Craig-Wood)
49387 • Fix reauth on big files (Nick Craig-Wood)
49389 • Union
49391 • Fix poll-interval not working (Nick Craig-Wood)
49393 • WebDAV
49395 • Support About which means rclone mount will show the correct disk
49397 size (Nick Craig-Wood)
49398 • Support MD5 and SHA1 hashes with Owncloud and Nextcloud (Nick
49400 Craig-Wood)
49401 • Fail soft on time parsing errors (Nick Craig-Wood)
49403 • Fix infinite loop on failed directory creation (Nick Craig-Wood)
49405 • Fix identification of directories for Bitrix Site Manager (Nick
49407 Craig-Wood)
49408 • Fix upload of 0 length files on some servers (Nick Craig-Wood)
49410 • Fix if MKCOL fails with 423 Locked assume the directory exists
49412 (Nick Craig-Wood)
49413 v1.45 - 2018-11-24
49415 • New backends
49416 • The Yandex backend was re-written - see below for details (Sebas‐
49418 tian Bünger)
49419 • New commands
49421 • rcd: New command just to serve the remote control API (Nick Craig-
49423 Wood)
49424 • New Features
49426 • The remote control API (rc) was greatly expanded to allow full con‐
49428 trol over rclone (Nick Craig-Wood)
49429 • sensitive operations require authorization or the --rc-no-auth
49431 flag
49432 • config/* operations to configure rclone
49434 • options/* for reading/setting command line flags
49436 • operations/* for all low level operations, e.g. copy file, list
49438 directory
49439 • sync/* for sync, copy and move
49441 • --rc-files flag to serve files on the rc http server
49443 • this is for building web native GUIs for rclone
49445 • Optionally serving objects on the rc http server
49447 • Ensure rclone fails to start up if the --rc port is in use al‐
49449 ready
49450 • See the rc docs (https://rclone.org/rc/) for more info
49452 • sync/copy/move
49454 • Make --files-from only read the objects specified and don't scan
49456 directories (Nick Craig-Wood)
49457 • This is a huge speed improvement for destinations with lots of
49459 files
49460 • filter: Add --ignore-case flag (Nick Craig-Wood)
49462 • ncdu: Add remove function ('d' key) (Henning Surmeier)
49464 • rc command
49466 • Add --json flag for structured JSON input (Nick Craig-Wood)
49468 • Add --user and --pass flags and interpret --rc-user, --rc-pass,
49470 --rc-addr (Nick Craig-Wood)
49471 • build
49473 • Require go1.8 or later for compilation (Nick Craig-Wood)
49475 • Enable softfloat on MIPS arch (Scott Edlund)
49477 • Integration test framework revamped with a better report and bet‐
49479 ter retries (Nick Craig-Wood)
49480 • Bug Fixes
49482 • cmd: Make --progress update the stats correctly at the end (Nick
49484 Craig-Wood)
49485 • config: Create config directory on save if it is missing (Nick
49487 Craig-Wood)
49488 • dedupe: Check for existing filename before renaming a dupe file
49490 (ssaqua)
49491 • move: Don't create directories with --dry-run (Nick Craig-Wood)
49493 • operations: Fix Purge and Rmdirs when dir is not the root (Nick
49495 Craig-Wood)
49496 • serve http/webdav/restic: Ensure rclone exits if the port is in use
49498 (Nick Craig-Wood)
49499 • Mount
49501 • Make --volname work for Windows and macOS (Nick Craig-Wood)
49503 • Azure Blob
49505 • Avoid context deadline exceeded error by setting a large TryTimeout
49507 value (brused27)
49508 • Fix erroneous Rmdir error "directory not empty" (Nick Craig-Wood)
49510 • Wait for up to 60s to create a just deleted container (Nick Craig-
49512 Wood)
49513 • Dropbox
49515 • Add dropbox impersonate support (Jake Coggiano)
49517 • Jottacloud
49519 • Fix bug in --fast-list handing of empty folders (albertony)
49521 • Opendrive
49523 • Fix transfer of files with + and & in (Nick Craig-Wood)
49525 • Fix retries of upload chunks (Nick Craig-Wood)
49527 • S3
49529 • Set ACL for server-side copies to that provided by the user (Nick
49531 Craig-Wood)
49532 • Fix role_arn, credential_source, ... (Erik Swanson)
49534 • Add config info for Wasabi's US-West endpoint (Henry Ptasinski)
49536 • SFTP
49538 • Ensure file hash checking is really disabled (Jon Fautley)
49540 • Swift
49542 • Add pacer for retries to make swift more reliable (Nick Craig-Wood)
49544 • WebDAV
49546 • Add Content-Type to PUT requests (Nick Craig-Wood)
49548 • Fix config parsing so --webdav-user and --webdav-pass flags work
49550 (Nick Craig-Wood)
49551 • Add RFC3339 date format (Ralf Hemberger)
49553 • Yandex
49555 • The yandex backend was re-written (Sebastian Bünger)
49557 • This implements low level retries (Sebastian Bünger)
49559 • Copy, Move, DirMove, PublicLink and About optional interfaces
49561 (Sebastian Bünger)
49562 • Improved general error handling (Sebastian Bünger)
49564 • Removed ListR for now due to inconsistent behaviour (Sebastian
49566 Bünger)
49567 v1.44 - 2018-10-15
49569 • New commands
49570 • serve ftp: Add ftp server (Antoine GIRARD)
49572 • settier: perform storage tier changes on supported remotes
49574 (sandeepkru)
49575 • New Features
49577 • Reworked command line help
49579 • Make default help less verbose (Nick Craig-Wood)
49581 • Split flags up into global and backend flags (Nick Craig-Wood)
49583 • Implement specialised help for flags and backends (Nick Craig-
49585 Wood)
49586 • Show URL of backend help page when starting config (Nick Craig-
49588 Wood)
49589 • stats: Long names now split in center (Joanna Marek)
49591 • Add --log-format flag for more control over log output (dcpu)
49593 • rc: Add support for OPTIONS and basic CORS (frenos)
49595 • stats: show FatalErrors and NoRetryErrors in stats (Cédric Connes)
49597 • Bug Fixes
49599 • Fix -P not ending with a new line (Nick Craig-Wood)
49601 • config: don't create default config dir when user supplies --config
49603 (albertony)
49604 • Don't print non-ASCII characters with --progress on windows (Nick
49606 Craig-Wood)
49607 • Correct logs for excluded items (ssaqua)
49609 • Mount
49611 • Remove EXPERIMENTAL tags (Nick Craig-Wood)
49613 • VFS
49615 • Fix race condition detected by serve ftp tests (Nick Craig-Wood)
49617 • Add vfs/poll-interval rc command (Fabian Möller)
49619 • Enable rename for nearly all remotes using server-side Move or Copy
49621 (Nick Craig-Wood)
49622 • Reduce directory cache cleared by poll-interval (Fabian Möller)
49624 • Remove EXPERIMENTAL tags (Nick Craig-Wood)
49626 • Local
49628 • Skip bad symlinks in dir listing with -L enabled (Cédric Connes)
49630 • Preallocate files on Windows to reduce fragmentation (Nick Craig-
49632 Wood)
49633 • Preallocate files on linux with fallocate(2) (Nick Craig-Wood)
49635 • Cache
49637 • Add cache/fetch rc function (Fabian Möller)
49639 • Fix worker scale down (Fabian Möller)
49641 • Improve performance by not sending info requests for cached chunks
49643 (dcpu)
49644 • Fix error return value of cache/fetch rc method (Fabian Möller)
49646 • Documentation fix for cache-chunk-total-size (Anagh Kumar Baranwal)
49648 • Preserve leading / in wrapped remote path (Fabian Möller)
49650 • Add plex_insecure option to skip certificate validation (Fabian
49652 Möller)
49653 • Remove entries that no longer exist in the source (dcpu)
49655 • Crypt
49657 • Preserve leading / in wrapped remote path (Fabian Möller)
49659 • Alias
49661 • Fix handling of Windows network paths (Nick Craig-Wood)
49663 • Azure Blob
49665 • Add --azureblob-list-chunk parameter (Santiago Rodríguez)
49667 • Implemented settier command support on azureblob remote. (sandeep‐
49669 kru)
49670 • Work around SDK bug which causes errors for chunk-sized files (Nick
49672 Craig-Wood)
49673 • Box
49675 • Implement link sharing. (Sebastian Bünger)
49677 • Drive
49679 • Add --drive-import-formats - google docs can now be imported (Fabi‐
49681 an Möller)
49682 • Rewrite mime type and extension handling (Fabian Möller)
49684 • Add document links (Fabian Möller)
49686 • Add support for multipart document extensions (Fabian Möller)
49688 • Add support for apps-script to json export (Fabian Möller)
49690 • Fix escaped chars in documents during list (Fabian Möller)
49692 • Add --drive-v2-download-min-size a workaround for slow downloads
49694 (Fabian Möller)
49695 • Improve directory notifications in ChangeNotify (Fabian Möller)
49697 • When listing team drives in config, continue on failure (Nick
49699 Craig-Wood)
49700 • FTP
49702 • Add a small pause after failed upload before deleting file (Nick
49704 Craig-Wood)
49705 • Google Cloud Storage
49707 • Fix service_account_file being ignored (Fabian Möller)
49709 • Jottacloud
49711 • Minor improvement in quota info (omit if unlimited) (albertony)
49713 • Add --fast-list support (albertony)
49715 • Add permanent delete support: --jottacloud-hard-delete (albertony)
49717 • Add link sharing support (albertony)
49719 • Fix handling of reserved characters. (Sebastian Bünger)
49721 • Fix socket leak on Object.Remove (Nick Craig-Wood)
49723 • Onedrive
49725 • Rework to support Microsoft Graph (Cnly)
49727 • NB this will require re-authenticating the remote
49729 • Removed upload cutoff and always do session uploads (Oliver Heyme)
49731 • Use single-part upload for empty files (Cnly)
49733 • Fix new fields not saved when editing old config (Alex Chen)
49735 • Fix sometimes special chars in filenames not replaced (Alex Chen)
49737 • Ignore OneNote files by default (Alex Chen)
49739 • Add link sharing support (jackyzy823)
49741 • S3
49743 • Use custom pacer, to retry operations when reasonable (Craig
49745 Miskell)
49746 • Use configured server-side-encryption and storage class options
49748 when calling CopyObject() (Paul Kohout)
49749 • Make --s3-v2-auth flag (Nick Craig-Wood)
49751 • Fix v2 auth on files with spaces (Nick Craig-Wood)
49753 • Union
49755 • Implement union backend which reads from multiple backends (Felix
49757 Brucker)
49758 • Implement optional interfaces (Move, DirMove, Copy, etc.) (Nick
49760 Craig-Wood)
49761 • Fix ChangeNotify to support multiple remotes (Fabian Möller)
49763 • Fix --backup-dir on union backend (Nick Craig-Wood)
49765 • WebDAV
49767 • Add another time format (Nick Craig-Wood)
49769 • Add a small pause after failed upload before deleting file (Nick
49771 Craig-Wood)
49772 • Add workaround for missing mtime (buergi)
49774 • Sharepoint: Renew cookies after 12hrs (Henning Surmeier)
49776 • Yandex
49778 • Remove redundant nil checks (teresy)
49780 v1.43.1 - 2018-09-07
49782 Point release to fix hubic and azureblob backends.
49783 • Bug Fixes
49785 • ncdu: Return error instead of log.Fatal in Show (Fabian Möller)
49787 • cmd: Fix crash with --progress and --stats 0 (Nick Craig-Wood)
49789 • docs: Tidy website display (Anagh Kumar Baranwal)
49791 • Azure Blob:
49793 • Fix multi-part uploads. (sandeepkru)
49795 • Hubic
49797 • Fix uploads (Nick Craig-Wood)
49799 • Retry auth fetching if it fails to make hubic more reliable (Nick
49801 Craig-Wood)
49802 v1.43 - 2018-09-01
49804 • New backends
49805 • Jottacloud (Sebastian Bünger)
49807 • New commands
49809 • copyurl: copies a URL to a remote (Denis)
49811 • New Features
49813 • Reworked config for backends (Nick Craig-Wood)
49815 • All backend config can now be supplied by command line, env var
49817 or config file
49818 • Advanced section in the config wizard for the optional items
49820 • A large step towards rclone backends being usable in other go
49822 software
49823 • Allow on the fly remotes with :backend: syntax
49825 • Stats revamp
49827 • Add --progress/-P flag to show interactive progress (Nick Craig-
49829 Wood)
49830 • Show the total progress of the sync in the stats (Nick Craig-
49832 Wood)
49833 • Add --stats-one-line flag for single line stats (Nick Craig-Wood)
49835 • Added weekday schedule into --bwlimit (Mateusz)
49837 • lsjson: Add option to show the original object IDs (Fabian Möller)
49839 • serve webdav: Make Content-Type without reading the file and add
49841 --etag-hash (Nick Craig-Wood)
49842 • build
49844 • Build macOS with native compiler (Nick Craig-Wood)
49846 • Update to use go1.11 for the build (Nick Craig-Wood)
49848 • rc
49850 • Added core/stats to return the stats (reddi1)
49852 • version --check: Prints the current release and beta versions (Nick
49854 Craig-Wood)
49855 • Bug Fixes
49857 • accounting
49859 • Fix time to completion estimates (Nick Craig-Wood)
49861 • Fix moving average speed for file stats (Nick Craig-Wood)
49863 • config: Fix error reading password from piped input (Nick Craig-
49865 Wood)
49866 • move: Fix --delete-empty-src-dirs flag to delete all empty dirs on
49868 move (ishuah)
49869 • Mount
49871 • Implement --daemon-timeout flag for OSXFUSE (Nick Craig-Wood)
49873 • Fix mount --daemon not working with encrypted config (Alex Chen)
49875 • Clip the number of blocks to 2^32-1 on macOS - fixes borg backup
49877 (Nick Craig-Wood)
49878 • VFS
49880 • Enable vfs-read-chunk-size by default (Fabian Möller)
49882 • Add the vfs/refresh rc command (Fabian Möller)
49884 • Add non recursive mode to vfs/refresh rc command (Fabian Möller)
49886 • Try to seek buffer on read only files (Fabian Möller)
49888 • Local
49890 • Fix crash when deprecated --local-no-unicode-normalization is sup‐
49892 plied (Nick Craig-Wood)
49893 • Fix mkdir error when trying to copy files to the root of a drive on
49895 windows (Nick Craig-Wood)
49896 • Cache
49898 • Fix nil pointer deref when using lsjson on cached directory (Nick
49900 Craig-Wood)
49901 • Fix nil pointer deref for occasional crash on playback (Nick Craig-
49903 Wood)
49904 • Crypt
49906 • Fix accounting when checking hashes on upload (Nick Craig-Wood)
49908 • Amazon Cloud Drive
49910 • Make very clear in the docs that rclone has no ACD keys (Nick
49912 Craig-Wood)
49913 • Azure Blob
49915 • Add connection string and SAS URL auth (Nick Craig-Wood)
49917 • List the container to see if it exists (Nick Craig-Wood)
49919 • Port new Azure Blob Storage SDK (sandeepkru)
49921 • Added blob tier, tier between Hot, Cool and Archive. (sandeepkru)
49923 • Remove leading / from paths (Nick Craig-Wood)
49925 • B2
49927 • Support Application Keys (Nick Craig-Wood)
49929 • Remove leading / from paths (Nick Craig-Wood)
49931 • Box
49933 • Fix upload of > 2GB files on 32 bit platforms (Nick Craig-Wood)
49935 • Make --box-commit-retries flag defaulting to 100 to fix large up‐
49937 loads (Nick Craig-Wood)
49938 • Drive
49940 • Add --drive-keep-revision-forever flag (lewapm)
49942 • Handle gdocs when filtering file names in list (Fabian Möller)
49944 • Support using --fast-list for large speedups (Fabian Möller)
49946 • FTP
49948 • Fix Put mkParentDir failed: 521 for BunnyCDN (Nick Craig-Wood)
49950 • Google Cloud Storage
49952 • Fix index out of range error with --fast-list (Nick Craig-Wood)
49954 • Jottacloud
49956 • Fix MD5 error check (Oliver Heyme)
49958 • Handle empty time values (Martin Polden)
49960 • Calculate missing MD5s (Oliver Heyme)
49962 • Docs, fixes and tests for MD5 calculation (Nick Craig-Wood)
49964 • Add optional MimeTyper interface. (Sebastian Bünger)
49966 • Implement optional About interface (for df support). (Sebastian
49968 Bünger)
49969 • Mega
49971 • Wait for events instead of arbitrary sleeping (Nick Craig-Wood)
49973 • Add --mega-hard-delete flag (Nick Craig-Wood)
49975 • Fix failed logins with upper case chars in email (Nick Craig-Wood)
49977 • Onedrive
49979 • Shared folder support (Yoni Jah)
49981 • Implement DirMove (Cnly)
49983 • Fix rmdir sometimes deleting directories with contents (Nick Craig-
49985 Wood)
49986 • Pcloud
49988 • Delete half uploaded files on upload error (Nick Craig-Wood)
49990 • Qingstor
49992 • Remove leading / from paths (Nick Craig-Wood)
49994 • S3
49996 • Fix index out of range error with --fast-list (Nick Craig-Wood)
49998 • Add --s3-force-path-style (Nick Craig-Wood)
50000 • Add support for KMS Key ID (bsteiss)
50002 • Remove leading / from paths (Nick Craig-Wood)
50004 • Swift
50006 • Add storage_policy (Ruben Vandamme)
50008 • Make it so just storage_url or auth_token can be overridden (Nick
50010 Craig-Wood)
50011 • Fix server-side copy bug for unusual file names (Nick Craig-Wood)
50013 • Remove leading / from paths (Nick Craig-Wood)
50015 • WebDAV
50017 • Ensure we call MKCOL with a URL with a trailing / for QNAP interop
50019 (Nick Craig-Wood)
50020 • If root ends with / then don't check if it is a file (Nick Craig-
50022 Wood)
50023 • Don't accept redirects when reading metadata (Nick Craig-Wood)
50025 • Add bearer token (Macaroon) support for dCache (Nick Craig-Wood)
50027 • Document dCache and Macaroons (Onno Zweers)
50029 • Sharepoint recursion with different depth (Henning)
50031 • Attempt to remove failed uploads (Nick Craig-Wood)
50033 • Yandex
50035 • Fix listing/deleting files in the root (Nick Craig-Wood)
50037 v1.42 - 2018-06-16
50039 • New backends
50040 • OpenDrive (Oliver Heyme, Jakub Karlicek, ncw)
50042 • New commands
50044 • deletefile command (Filip Bartodziej)
50046 • New Features
50048 • copy, move: Copy single files directly, don't use --files-from
50050 work-around
50051 • this makes them much more efficient
50053 • Implement --max-transfer flag to quit transferring at a limit
50055 • make exit code 8 for --max-transfer exceeded
50057 • copy: copy empty source directories to destination (Ishuah Kariuki)
50059 • check: Add --one-way flag (Kasper Byrdal Nielsen)
50061 • Add siginfo handler for macOS for ctrl-T stats (kubatasiemski)
50063 • rc
50065 • add core/gc to run a garbage collection on demand
50067 • enable go profiling by default on the --rc port
50069 • return error from remote on failure
50071 • lsf
50073 • Add --absolute flag to add a leading / onto path names
50075 • Add --csv flag for compliant CSV output
50077 • Add 'm' format specifier to show the MimeType
50079 • Implement 'i' format for showing object ID
50081 • lsjson
50083 • Add MimeType to the output
50085 • Add ID field to output to show Object ID
50087 • Add --retries-sleep flag (Benjamin Joseph Dag)
50089 • Oauth tidy up web page and error handling (Henning Surmeier)
50091 • Bug Fixes
50093 • Password prompt output with --log-file fixed for unix (Filip Bar‐
50095 todziej)
50096 • Calculate ModifyWindow each time on the fly to fix various problems
50098 (Stefan Breunig)
50099 • Mount
50101 • Only print "File.rename error" if there actually is an error (Ste‐
50103 fan Breunig)
50104 • Delay rename if file has open writers instead of failing outright
50106 (Stefan Breunig)
50107 • Ensure atexit gets run on interrupt
50109 • macOS enhancements
50111 • Make --noappledouble --noapplexattr
50113 • Add --volname flag and remove special chars from it
50115 • Make Get/List/Set/Remove xattr return ENOSYS for efficiency
50117 • Make --daemon work for macOS without CGO
50119 • VFS
50121 • Add --vfs-read-chunk-size and --vfs-read-chunk-size-limit (Fabian
50123 Möller)
50124 • Fix ChangeNotify for new or changed folders (Fabian Möller)
50126 • Local
50128 • Fix symlink/junction point directory handling under Windows
50130 • NB you will need to add -L to your command line to copy files
50132 with reparse points
50133 • Cache
50135 • Add non cached dirs on notifications (Remus Bunduc)
50137 • Allow root to be expired from rc (Remus Bunduc)
50139 • Clean remaining empty folders from temp upload path (Remus Bunduc)
50141 • Cache lists using batch writes (Remus Bunduc)
50143 • Use secure websockets for HTTPS Plex addresses (John Clayton)
50145 • Reconnect plex websocket on failures (Remus Bunduc)
50147 • Fix panic when running without plex configs (Remus Bunduc)
50149 • Fix root folder caching (Remus Bunduc)
50151 • Crypt
50153 • Check the crypted hash of files when uploading for extra data secu‐
50155 rity
50156 • Dropbox
50158 • Make Dropbox for business folders accessible using an initial / in
50160 the path
50161 • Google Cloud Storage
50163 • Low level retry all operations if necessary
50165 • Google Drive
50167 • Add --drive-acknowledge-abuse to download flagged files
50169 • Add --drive-alternate-export to fix large doc export
50171 • Don't attempt to choose Team Drives when using rclone config create
50173 • Fix change list polling with team drives
50175 • Fix ChangeNotify for folders (Fabian Möller)
50177 • Fix about (and df on a mount) for team drives
50179 • Onedrive
50181 • Errorhandler for onedrive for business requests (Henning Surmeier)
50183 • S3
50185 • Adjust upload concurrency with --s3-upload-concurrency (themylogin)
50187 • Fix --s3-chunk-size which was always using the minimum
50189 • SFTP
50191 • Add --ssh-path-override flag (Piotr Oleszczyk)
50193 • Fix slow downloads for long latency connections
50195 • Webdav
50197 • Add workarounds for biz.mail.ru
50199 • Ignore Reason-Phrase in status line to fix 4shared (Rodrigo)
50201 • Better error message generation
50203 v1.41 - 2018-04-28
50205 • New backends
50206 • Mega support added
50208 • Webdav now supports SharePoint cookie authentication (hensur)
50210 • New commands
50212 • link: create public link to files and folders (Stefan Breunig)
50214 • about: gets quota info from a remote (a-roussos, ncw)
50216 • hashsum: a generic tool for any hash to produce md5sum like output
50218 • New Features
50220 • lsd: Add -R flag and fix and update docs for all ls commands
50222 • ncdu: added a "refresh" key - CTRL-L (Keith Goldfarb)
50224 • serve restic: Add append-only mode (Steve Kriss)
50226 • serve restic: Disallow overwriting files in append-only mode (Alex‐
50228 ander Neumann)
50229 • serve restic: Print actual listener address (Matt Holt)
50231 • size: Add --json flag (Matthew Holt)
50233 • sync: implement --ignore-errors (Mateusz Pabian)
50235 • dedupe: Add dedupe largest functionality (Richard Yang)
50237 • fs: Extend SizeSuffix to include TB and PB for rclone about
50239 • fs: add --dump goroutines and --dump openfiles for debugging
50241 • rc: implement core/memstats to print internal memory usage info
50243 • rc: new call rc/pid (Michael P. Dubner)
50245 • Compile
50247 • Drop support for go1.6
50249 • Release
50251 • Fix make tarball (Chih-Hsuan Yen)
50253 • Bug Fixes
50255 • filter: fix --min-age and --max-age together check
50257 • fs: limit MaxIdleConns and MaxIdleConnsPerHost in transport
50259 • lsd,lsf: make sure all times we output are in local time
50261 • rc: fix setting bwlimit to unlimited
50263 • rc: take note of the --rc-addr flag too as per the docs
50265 • Mount
50267 • Use About to return the correct disk total/used/free (e.g. in df)
50269 • Set --attr-timeout default to 1s - fixes:
50271 • rclone using too much memory
50273 • rclone not serving files to samba
50275 • excessive time listing directories
50277 • Fix df -i (upstream fix)
50279 • VFS
50281 • Filter files . and .. from directory listing
50283 • Only make the VFS cache if --vfs-cache-mode > Off
50285 • Local
50287 • Add --local-no-check-updated to disable updated file checks
50289 • Retry remove on Windows sharing violation error
50291 • Cache
50293 • Flush the memory cache after close
50295 • Purge file data on notification
50297 • Always forget parent dir for notifications
50299 • Integrate with Plex websocket
50301 • Add rc cache/stats (seuffert)
50303 • Add info log on notification
50305 • Box
50307 • Fix failure reading large directories - parse file/directory size
50309 as float
50310 • Dropbox
50312 • Fix crypt+obfuscate on dropbox
50314 • Fix repeatedly uploading the same files
50316 • FTP
50318 • Work around strange response from box FTP server
50320 • More workarounds for FTP servers to fix mkParentDir error
50322 • Fix no error on listing nonexistent directory
50324 • Google Cloud Storage
50326 • Add service_account_credentials (Matt Holt)
50328 • Detect bucket presence by listing it - minimises permissions needed
50330 • Ignore zero length directory markers
50332 • Google Drive
50334 • Add service_account_credentials (Matt Holt)
50336 • Fix directory move leaving a hardlinked directory behind
50338 • Return proper google errors when Opening files
50340 • When initialized with a filepath, optional features used incorrect
50342 root path (Stefan Breunig)
50343 • HTTP
50345 • Fix sync for servers which don't return Content-Length in HEAD
50347 • Onedrive
50349 • Add QuickXorHash support for OneDrive for business
50351 • Fix socket leak in multipart session upload
50353 • S3
50355 • Look in S3 named profile files for credentials
50357 • Add --s3-disable-checksum to disable checksum uploading (Chris Re‐
50359 dekop)
50360 • Hierarchical configuration support (Giri Badanahatti)
50362 • Add in config for all the supported S3 providers
50364 • Add One Zone Infrequent Access storage class (Craig Rachel)
50366 • Add --use-server-modtime support (Peter Baumgartner)
50368 • Add --s3-chunk-size option to control multipart uploads
50370 • Ignore zero length directory markers
50372 • SFTP
50374 • Update docs to match code, fix typos and clarify disable_hashcheck
50376 prompt (Michael G. Noll)
50377 • Update docs with Synology quirks
50379 • Fail soft with a debug on hash failure
50381 • Swift
50383 • Add --use-server-modtime support (Peter Baumgartner)
50385 • Webdav
50387 • Support SharePoint cookie authentication (hensur)
50389 • Strip leading and trailing / off root
50391 v1.40 - 2018-03-19
50393 • New backends
50394 • Alias backend to create aliases for existing remote names (Fabian
50396 Möller)
50397 • New commands
50399 • lsf: list for parsing purposes (Jakub Tasiemski)
50401 • by default this is a simple non recursive list of files and di‐
50403 rectories
50404 • it can be configured to add more info in an easy to parse way
50406 • serve restic: for serving a remote as a Restic REST endpoint
50408 • This enables restic to use any backends that rclone can access
50410 • Thanks Alexander Neumann for help, patches and review
50412 • rc: enable the remote control of a running rclone
50414 • The running rclone must be started with --rc and related flags.
50416 • Currently there is support for bwlimit, and flushing for mount
50418 and cache.
50419 • New Features
50421 • --max-delete flag to add a delete threshold (Bjørn Erik Pedersen)
50423 • All backends now support RangeOption for ranged Open
50425 • cat: Use RangeOption for limited fetches to make more efficient
50427 • cryptcheck: make reading of nonce more efficient with RangeOption
50429 • serve http/webdav/restic
50431 • support SSL/TLS
50433 • add --user --pass and --htpasswd for authentication
50435 • copy/move: detect file size change during copy/move and abort
50437 transfer (ishuah)
50438 • cryptdecode: added option to return encrypted file names. (ishuah)
50440 • lsjson: add --encrypted to show encrypted name (Jakub Tasiemski)
50442 • Add --stats-file-name-length to specify the printed file name
50444 length for stats (Will Gunn)
50445 • Compile
50447 • Code base was shuffled and factored
50449 • backends moved into a backend directory
50451 • large packages split up
50453 • See the CONTRIBUTING.md doc for info as to what lives where now
50455 • Update to using go1.10 as the default go version
50457 • Implement daily full integration tests (https://pub.rclone.org/in‐
50459 tegration-tests/)
50460 • Release
50462 • Include a source tarball and sign it and the binaries
50464 • Sign the git tags as part of the release process
50466 • Add .deb and .rpm packages as part of the build
50468 • Make a beta release for all branches on the main repo (but not pull
50470 requests)
50471 • Bug Fixes
50473 • config: fixes errors on nonexistent config by loading config file
50475 only on first access
50476 • config: retry saving the config after failure (Mateusz)
50478 • sync: when using --backup-dir don't delete files if we can't set
50480 their modtime
50481 • this fixes odd behaviour with Dropbox and --backup-dir
50483 • fshttp: fix idle timeouts for HTTP connections
50485 • serve http: fix serving files with : in - fixes
50487 • Fix --exclude-if-present to ignore directories which it doesn't
50489 have permission for (Iakov Davydov)
50490 • Make accounting work properly with crypt and b2
50492 • remove --no-traverse flag because it is obsolete
50494 • Mount
50496 • Add --attr-timeout flag to control attribute caching in kernel
50498 • this now defaults to 0 which is correct but less efficient
50500 • see the mount docs (https://rclone.org/commands/rclone_mount/#at‐
50502 tribute-caching) for more info
50503 • Add --daemon flag to allow mount to run in the background (ishuah)
50505 • Fix: Return ENOSYS rather than EIO on attempted link
50507 • This fixes FileZilla accessing an rclone mount served over sftp.
50509 • Fix setting modtime twice
50511 • Mount tests now run on CI for Linux (mount & cmount)/Mac/Windows
50513 • Many bugs fixed in the VFS layer - see below
50515 • VFS
50517 • Many fixes for --vfs-cache-mode writes and above
50519 • Update cached copy if we know it has changed (fixes stale data)
50521 • Clean path names before using them in the cache
50523 • Disable cache cleaner if --vfs-cache-poll-interval=0
50525 • Fill and clean the cache immediately on startup
50527 • Fix Windows opening every file when it stats the file
50529 • Fix applying modtime for an open Write Handle
50531 • Fix creation of files when truncating
50533 • Write 0 bytes when flushing unwritten handles to avoid race condi‐
50535 tions in FUSE
50536 • Downgrade "poll-interval is not supported" message to Info
50538 • Make OpenFile and friends return EINVAL if O_RDONLY and O_TRUNC
50540 • Local
50542 • Downgrade "invalid cross-device link: trying copy" to debug
50544 • Make DirMove return fs.ErrorCantDirMove to allow fallback to Copy
50546 for cross device
50547 • Fix race conditions updating the hashes
50549 • Cache
50551 • Add support for polling - cache will update when remote changes on
50553 supported backends
50554 • Reduce log level for Plex api
50556 • Fix dir cache issue
50558 • Implement --cache-db-wait-time flag
50560 • Improve efficiency with RangeOption and RangeSeek
50562 • Fix dirmove with temp fs enabled
50564 • Notify vfs when using temp fs
50566 • Offline uploading
50568 • Remote control support for path flushing
50570 • Amazon cloud drive
50572 • Rclone no longer has any working keys - disable integration tests
50574 • Implement DirChangeNotify to notify cache/vfs/mount of changes
50576 • Azureblob
50578 • Don't check for bucket/container presence if listing was OK
50580 • this makes rclone do one less request per invocation
50582 • Improve accounting for chunked uploads
50584 • Backblaze B2
50586 • Don't check for bucket/container presence if listing was OK
50588 • this makes rclone do one less request per invocation
50590 • Box
50592 • Improve accounting for chunked uploads
50594 • Dropbox
50596 • Fix custom oauth client parameters
50598 • Google Cloud Storage
50600 • Don't check for bucket/container presence if listing was OK
50602 • this makes rclone do one less request per invocation
50604 • Google Drive
50606 • Migrate to api v3 (Fabian Möller)
50608 • Add scope configuration and root folder selection
50610 • Add --drive-impersonate for service accounts
50612 • thanks to everyone who tested, explored and contributed docs
50614 • Add --drive-use-created-date to use created date as modified date
50616 (nbuchanan)
50617 • Request the export formats only when required
50619 • This makes rclone quicker when there are no google docs
50621 • Fix finding paths with latin1 chars (a workaround for a drive bug)
50623 • Fix copying of a single Google doc file
50625 • Fix --drive-auth-owner-only to look in all directories
50627 • HTTP
50629 • Fix handling of directories with & in
50631 • Onedrive
50633 • Removed upload cutoff and always do session uploads
50635 • this stops the creation of multiple versions on business onedrive
50637 • Overwrite object size value with real size when reading file.
50639 (Victor)
50640 • this fixes oddities when onedrive misreports the size of images
50642 • Pcloud
50644 • Remove unused chunked upload flag and code
50646 • Qingstor
50648 • Don't check for bucket/container presence if listing was OK
50650 • this makes rclone do one less request per invocation
50652 • S3
50654 • Support hashes for multipart files (Chris Redekop)
50656 • Initial support for IBM COS (S3) (Giri Badanahatti)
50658 • Update docs to discourage use of v2 auth with CEPH and others
50660 • Don't check for bucket/container presence if listing was OK
50662 • this makes rclone do one less request per invocation
50664 • Fix server-side copy and set modtime on files with + in
50666 • SFTP
50668 • Add option to disable remote hash check command execution (Jon
50670 Fautley)
50671 • Add --sftp-ask-password flag to prompt for password when needed
50673 (Leo R. Lundgren)
50674 • Add set_modtime configuration option
50676 • Fix following of symlinks
50678 • Fix reading config file outside of Fs setup
50680 • Fix reading $USER in username fallback not $HOME
50682 • Fix running under crontab - Use correct OS way of reading username
50684 • Swift
50686 • Fix refresh of authentication token
50688 • in v1.39 a bug was introduced which ignored new tokens - this
50690 fixes it
50691 • Fix extra HEAD transaction when uploading a new file
50693 • Don't check for bucket/container presence if listing was OK
50695 • this makes rclone do one less request per invocation
50697 • Webdav
50699 • Add new time formats to support mydrive.ch and others
50701 v1.39 - 2017-12-23
50703 • New backends
50704 • WebDAV
50706 • tested with nextcloud, owncloud, put.io and others!
50708 • Pcloud
50710 • cache - wraps a cache around other backends (Remus Bunduc)
50712 • useful in combination with mount
50714 • NB this feature is in beta so use with care
50716 • New commands
50718 • serve command with subcommands:
50720 • serve webdav: this implements a webdav server for any rclone re‐
50722 mote.
50723 • serve http: command to serve a remote over HTTP
50725 • config: add sub commands for full config file management
50727 • create/delete/dump/edit/file/password/providers/show/update
50729 • touch: to create or update the timestamp of a file (Jakub Tasiems‐
50731 ki)
50732 • New Features
50734 • curl install for rclone (Filip Bartodziej)
50736 • --stats now shows percentage, size, rate and ETA in condensed form
50738 (Ishuah Kariuki)
50739 • --exclude-if-present to exclude a directory if a file is present
50741 (Iakov Davydov)
50742 • rmdirs: add --leave-root flag (lewapm)
50744 • move: add --delete-empty-src-dirs flag to remove dirs after move
50746 (Ishuah Kariuki)
50747 • Add --dump flag, introduce --dump requests, responses and remove
50749 --dump-auth, --dump-filters
50750 • Obscure X-Auth-Token: from headers when dumping too
50752 • Document and implement exit codes for different failure modes
50754 (Ishuah Kariuki)
50755 • Compile
50757 • Bug Fixes
50759 • Retry lots more different types of errors to make multipart trans‐
50761 fers more reliable
50762 • Save the config before asking for a token, fixes disappearing oauth
50764 config
50765 • Warn the user if --include and --exclude are used together (Ernest
50767 Borowski)
50768 • Fix duplicate files (e.g. on Google drive) causing spurious copies
50770 • Allow trailing and leading whitespace for passwords (Jason Rose)
50772 • ncdu: fix crashes on empty directories
50774 • rcat: fix goroutine leak
50776 • moveto/copyto: Fix to allow copying to the same name
50778 • Mount
50780 • --vfs-cache mode to make writes into mounts more reliable.
50782 • this requires caching files on the disk (see --cache-dir)
50784 • As this is a new feature, use with care
50786 • Use sdnotify to signal systemd the mount is ready (Fabian Möller)
50788 • Check if directory is not empty before mounting (Ernest Borowski)
50790 • Local
50792 • Add error message for cross file system moves
50794 • Fix equality check for times
50796 • Dropbox
50798 • Rework multipart upload
50800 • buffer the chunks when uploading large files so they can be re‐
50802 tried
50803 • change default chunk size to 48MB now we are buffering them in
50805 memory
50806 • retry every error after the first chunk is done successfully
50808 • Fix error when renaming directories
50810 • Swift
50812 • Fix crash on bad authentication
50814 • Google Drive
50816 • Add service account support (Tim Cooijmans)
50818 • S3
50820 • Make it work properly with Digital Ocean Spaces (Andrew Starr-
50822 Bochicchio)
50823 • Fix crash if a bad listing is received
50825 • Add support for ECS task IAM roles (David Minor)
50827 • Backblaze B2
50829 • Fix multipart upload retries
50831 • Fix --hard-delete to make it work 100% of the time
50833 • Swift
50835 • Allow authentication with storage URL and auth key (Giovanni Pizzi)
50837 • Add new fields for swift configuration to support IBM Bluemix Swift
50839 (Pierre Carlson)
50840 • Add OS_TENANT_ID and OS_USER_ID to config
50842 • Allow configs with user id instead of user name
50844 • Check if swift segments container exists before creating (John
50846 Leach)
50847 • Fix memory leak in swift transfers (upstream fix)
50849 • SFTP
50851 • Add option to enable the use of aes128-cbc cipher (Jon Fautley)
50853 • Amazon cloud drive
50855 • Fix download of large files failing with "Only one auth mechanism
50857 allowed"
50858 • crypt
50860 • Option to encrypt directory names or leave them intact
50862 • Implement DirChangeNotify (Fabian Möller)
50864 • onedrive
50866 • Add option to choose resourceURL during setup of OneDrive Business
50868 account if more than one is available for user
50869 v1.38 - 2017-09-30
50871 • New backends
50872 • Azure Blob Storage (thanks Andrei Dragomir)
50874 • Box
50876 • Onedrive for Business (thanks Oliver Heyme)
50878 • QingStor from QingCloud (thanks wuyu)
50880 • New commands
50882 • rcat - read from standard input and stream upload
50884 • tree - shows a nicely formatted recursive listing
50886 • cryptdecode - decode crypted file names (thanks ishuah)
50888 • config show - print the config file
50890 • config file - print the config file location
50892 • New Features
50894 • Empty directories are deleted on sync
50896 • dedupe - implement merging of duplicate directories
50898 • check and cryptcheck made more consistent and use less memory
50900 • cleanup for remaining remotes (thanks ishuah)
50902 • --immutable for ensuring that files don't change (thanks Jacob Mc‐
50904 Namee)
50905 • --user-agent option (thanks Alex McGrath Kraak)
50907 • --disable flag to disable optional features
50909 • --bind flag for choosing the local addr on outgoing connections
50911 • Support for zsh auto-completion (thanks bpicode)
50913 • Stop normalizing file names but do a normalized compare in sync
50915 • Compile
50917 • Update to using go1.9 as the default go version
50919 • Remove snapd build due to maintenance problems
50921 • Bug Fixes
50923 • Improve retriable error detection which makes multipart uploads
50925 better
50926 • Make check obey --ignore-size
50928 • Fix bwlimit toggle in conjunction with schedules (thanks cbruegg)
50930 • config ensures newly written config is on the same mount
50932 • Local
50934 • Revert to copy when moving file across file system boundaries
50936 • --skip-links to suppress symlink warnings (thanks Zhiming Wang)
50938 • Mount
50940 • Re-use rcat internals to support uploads from all remotes
50942 • Dropbox
50944 • Fix "entry doesn't belong in directory" error
50946 • Stop using deprecated API methods
50948 • Swift
50950 • Fix server-side copy to empty container with --fast-list
50952 • Google Drive
50954 • Change the default for --drive-use-trash to true
50956 • S3
50958 • Set session token when using STS (thanks Girish Ramakrishnan)
50960 • Glacier docs and error messages (thanks Jan Varho)
50962 • Read 1000 (not 1024) items in dir listings to fix Wasabi
50964 • Backblaze B2
50966 • Fix SHA1 mismatch when downloading files with no SHA1
50968 • Calculate missing hashes on the fly instead of spooling
50970 • --b2-hard-delete to permanently delete (not hide) files (thanks
50972 John Papandriopoulos)
50973 • Hubic
50975 • Fix creating containers - no longer have to use the default con‐
50977 tainer
50978 • Swift
50980 • Optionally configure from a standard set of OpenStack environment
50982 vars
50983 • Add endpoint_type config
50985 • Google Cloud Storage
50987 • Fix bucket creation to work with limited permission users
50989 • SFTP
50991 • Implement connection pooling for multiple ssh connections
50993 • Limit new connections per second
50995 • Add support for MD5 and SHA1 hashes where available (thanks Chris‐
50997 tian Brüggemann)
50998 • HTTP
51000 • Fix URL encoding issues
51002 • Fix directories with : in
51004 • Fix panic with URL encoded content
51006 v1.37 - 2017-07-22
51008 • New backends
51009 • FTP - thanks to Antonio Messina
51011 • HTTP - thanks to Vasiliy Tolstov
51013 • New commands
51015 • rclone ncdu - for exploring a remote with a text based user inter‐
51017 face.
51018 • rclone lsjson - for listing with a machine-readable output
51020 • rclone dbhashsum - to show Dropbox style hashes of files (local or
51022 Dropbox)
51023 • New Features
51025 • Implement --fast-list flag
51027 • This allows remotes to list recursively if they can
51029 • This uses less transactions (important if you pay for them)
51031 • This may or may not be quicker
51033 • This will use more memory as it has to hold the listing in memory
51035 • --old-sync-method deprecated - the remaining uses are covered by
51037 --fast-list
51038 • This involved a major re-write of all the listing code
51040 • Add --tpslimit and --tpslimit-burst to limit transactions per sec‐
51042 ond
51043 • this is useful in conjunction with rclone mount to limit external
51045 apps
51046 • Add --stats-log-level so can see --stats without -v
51048 • Print password prompts to stderr - Hraban Luyat
51050 • Warn about duplicate files when syncing
51052 • Oauth improvements
51054 • allow auth_url and token_url to be set in the config file
51056 • Print redirection URI if using own credentials.
51058 • Don't Mkdir at the start of sync to save transactions
51060 • Compile
51062 • Update build to go1.8.3
51064 • Require go1.6 for building rclone
51066 • Compile 386 builds with "GO386=387" for maximum compatibility
51068 • Bug Fixes
51070 • Fix menu selection when no remotes
51072 • Config saving reworked to not kill the file if disk gets full
51074 • Don't delete remote if name does not change while renaming
51076 • moveto, copyto: report transfers and checks as per move and copy
51078 • Local
51080 • Add --local-no-unicode-normalization flag - Bob Potter
51082 • Mount
51084 • Now supported on Windows using cgofuse and WinFsp - thanks to Bill
51086 Zissimopoulos for much help
51087 • Compare checksums on upload/download via FUSE
51089 • Unmount when program ends with SIGINT (Ctrl+C) or SIGTERM - Jérôme
51091 Vizcaino
51092 • On read only open of file, make open pending until first read
51094 • Make --read-only reject modify operations
51096 • Implement ModTime via FUSE for remotes that support it
51098 • Allow modTime to be changed even before all writers are closed
51100 • Fix panic on renames
51102 • Fix hang on errored upload
51104 • Crypt
51106 • Report the name:root as specified by the user
51108 • Add an "obfuscate" option for filename encryption - Stephen Harris
51110 • Amazon Drive
51112 • Fix initialization order for token renewer
51114 • Remove revoked credentials, allow oauth proxy config and update
51116 docs
51117 • B2
51119 • Reduce minimum chunk size to 5MB
51121 • Drive
51123 • Add team drive support
51125 • Reduce bandwidth by adding fields for partial responses - Martin
51127 Kristensen
51128 • Implement --drive-shared-with-me flag to view shared with me files
51130 - Danny Tsai
51131 • Add --drive-trashed-only to read only the files in the trash
51133 • Remove obsolete --drive-full-list
51135 • Add missing seek to start on retries of chunked uploads
51137 • Fix stats accounting for upload
51139 • Convert / in names to a unicode equivalent (／)
51141 • Poll for Google Drive changes when mounted
51143 • OneDrive
51145 • Fix the uploading of files with spaces
51147 • Fix initialization order for token renewer
51149 • Display speeds accurately when uploading - Yoni Jah
51151 • Swap to using http://localhost:53682/ as redirect URL - Michael
51153 Ledin
51154 • Retry on token expired error, reset upload body on retry - Yoni Jah
51156 • Google Cloud Storage
51158 • Add ability to specify location and storage class via config and
51160 command line - thanks gdm85
51161 • Create container if necessary on server-side copy
51163 • Increase directory listing chunk to 1000 to increase performance
51165 • Obtain a refresh token for GCS - Steven Lu
51167 • Yandex
51169 • Fix the name reported in log messages (was empty)
51171 • Correct error return for listing empty directory
51173 • Dropbox
51175 • Rewritten to use the v2 API
51177 • Now supports ModTime
51179 • Can only set by uploading the file again
51181 • If you uploaded with an old rclone, rclone may upload every‐
51183 thing again
51184 • Use --size-only or --checksum to avoid this
51186 • Now supports the Dropbox content hashing scheme
51188 • Now supports low level retries
51190 • S3
51192 • Work around eventual consistency in bucket creation
51194 • Create container if necessary on server-side copy
51196 • Add us-east-2 (Ohio) and eu-west-2 (London) S3 regions - Zahiar
51198 Ahmed
51199 • Swift, Hubic
51201 • Fix zero length directory markers showing in the subdirectory list‐
51203 ing
51204 • this caused lots of duplicate transfers
51206 • Fix paged directory listings
51208 • this caused duplicate directory errors
51210 • Create container if necessary on server-side copy
51212 • Increase directory listing chunk to 1000 to increase performance
51214 • Make sensible error if the user forgets the container
51216 • SFTP
51218 • Add support for using ssh key files
51220 • Fix under Windows
51222 • Fix ssh agent on Windows
51224 • Adapt to latest version of library - Igor Kharin
51226 v1.36 - 2017-03-18
51228 • New Features
51229 • SFTP remote (Jack Schmidt)
51231 • Re-implement sync routine to work a directory at a time reducing
51233 memory usage
51234 • Logging revamped to be more inline with rsync - now much quieter *
51236 -v only shows transfers * -vv is for full debug * --syslog to log
51237 to syslog on capable platforms
51238 • Implement --backup-dir and --suffix
51240 • Implement --track-renames (initial implementation by Bjørn Erik
51242 Pedersen)
51243 • Add time-based bandwidth limits (Lukas Loesche)
51245 • rclone cryptcheck: checks integrity of crypt remotes
51247 • Allow all config file variables and options to be set from environ‐
51249 ment variables
51250 • Add --buffer-size parameter to control buffer size for copy
51252 • Make --delete-after the default
51254 • Add --ignore-checksum flag (fixed by Hisham Zarka)
51256 • rclone check: Add --download flag to check all the data, not just
51258 hashes
51259 • rclone cat: add --head, --tail, --offset, --count and --discard
51261 • rclone config: when choosing from a list, allow the value to be en‐
51263 tered too
51264 • rclone config: allow rename and copy of remotes
51266 • rclone obscure: for generating encrypted passwords for rclone's
51268 config (T.C. Ferguson)
51269 • Comply with XDG Base Directory specification (Dario Giovannetti)
51271 • this moves the default location of the config file in a backwards
51273 compatible way
51274 • Release changes
51276 • Ubuntu snap support (Dedsec1)
51278 • Compile with go 1.8
51280 • MIPS/Linux big and little endian support
51282 • Bug Fixes
51284 • Fix copyto copying things to the wrong place if the destination dir
51286 didn't exist
51287 • Fix parsing of remotes in moveto and copyto
51289 • Fix --delete-before deleting files on copy
51291 • Fix --files-from with an empty file copying everything
51293 • Fix sync: don't update mod times if --dry-run set
51295 • Fix MimeType propagation
51297 • Fix filters to add ** rules to directory rules
51299 • Local
51301 • Implement -L, --copy-links flag to allow rclone to follow symlinks
51303 • Open files in write only mode so rclone can write to an rclone
51305 mount
51306 • Fix unnormalised unicode causing problems reading directories
51308 • Fix interaction between -x flag and --max-depth
51310 • Mount
51312 • Implement proper directory handling (mkdir, rmdir, renaming)
51314 • Make include and exclude filters apply to mount
51316 • Implement read and write async buffers - control with --buffer-size
51318 • Fix fsync on for directories
51320 • Fix retry on network failure when reading off crypt
51322 • Crypt
51324 • Add --crypt-show-mapping to show encrypted file mapping
51326 • Fix crypt writer getting stuck in a loop
51328 • IMPORTANT this bug had the potential to cause data corruption
51330 when
51331 • reading data from a network based remote and
51333 • writing to a crypt on Google Drive
51335 • Use the cryptcheck command to validate your data if you are con‐
51337 cerned
51338 • If syncing two crypt remotes, sync the unencrypted remote
51340 • Amazon Drive
51342 • Fix panics on Move (rename)
51344 • Fix panic on token expiry
51346 • B2
51348 • Fix inconsistent listings and rclone check
51350 • Fix uploading empty files with go1.8
51352 • Constrain memory usage when doing multipart uploads
51354 • Fix upload url not being refreshed properly
51356 • Drive
51358 • Fix Rmdir on directories with trashed files
51360 • Fix "Ignoring unknown object" when downloading
51362 • Add --drive-list-chunk
51364 • Add --drive-skip-gdocs (Károly Oláh)
51366 • OneDrive
51368 • Implement Move
51370 • Fix Copy
51372 • Fix overwrite detection in Copy
51374 • Fix waitForJob to parse errors correctly
51376 • Use token renewer to stop auth errors on long uploads
51378 • Fix uploading empty files with go1.8
51380 • Google Cloud Storage
51382 • Fix depth 1 directory listings
51384 • Yandex
51386 • Fix single level directory listing
51388 • Dropbox
51390 • Normalise the case for single level directory listings
51392 • Fix depth 1 listing
51394 • S3
51396 • Added ca-central-1 region (Jon Yergatian)
51398 v1.35 - 2017-01-02
51400 • New Features
51401 • moveto and copyto commands for choosing a destination name on
51403 copy/move
51404 • rmdirs command to recursively delete empty directories
51406 • Allow repeated --include/--exclude/--filter options
51408 • Only show transfer stats on commands which transfer stuff
51410 • show stats on any command using the --stats flag
51412 • Allow overlapping directories in move when server-side dir move is
51414 supported
51415 • Add --stats-unit option - thanks Scott McGillivray
51417 • Bug Fixes
51419 • Fix the config file being overwritten when two rclone instances are
51421 running
51422 • Make rclone lsd obey the filters properly
51424 • Fix compilation on mips
51426 • Fix not transferring files that don't differ in size
51428 • Fix panic on nil retry/fatal error
51430 • Mount
51432 • Retry reads on error - should help with reliability a lot
51434 • Report the modification times for directories from the remote
51436 • Add bandwidth accounting and limiting (fixes --bwlimit)
51438 • If --stats provided will show stats and which files are transfer‐
51440 ring
51441 • Support R/W files if truncate is set.
51443 • Implement statfs interface so df works
51445 • Note that write is now supported on Amazon Drive
51447 • Report number of blocks in a file - thanks Stefan Breunig
51449 • Crypt
51451 • Prevent the user pointing crypt at itself
51453 • Fix failed to authenticate decrypted block errors
51455 • these will now return the underlying unexpected EOF instead
51457 • Amazon Drive
51459 • Add support for server-side move and directory move - thanks Stefan
51461 Breunig
51462 • Fix nil pointer deref on size attribute
51464 • B2
51466 • Use new prefix and delimiter parameters in directory listings
51468 • This makes --max-depth 1 dir listings as used in mount much
51470 faster
51471 • Reauth the account while doing uploads too - should help with token
51473 expiry
51474 • Drive
51476 • Make DirMove more efficient and complain about moving the root
51478 • Create destination directory on Move()
51480 v1.34 - 2016-11-06
51482 • New Features
51483 • Stop single file and --files-from operations iterating through the
51485 source bucket.
51486 • Stop removing failed upload to cloud storage remotes
51488 • Make ContentType be preserved for cloud to cloud copies
51490 • Add support to toggle bandwidth limits via SIGUSR2 - thanks Marco
51492 Paganini
51493 • rclone check shows count of hashes that couldn't be checked
51495 • rclone listremotes command
51497 • Support linux/arm64 build - thanks Fredrik Fornwall
51499 • Remove Authorization: lines from --dump-headers output
51501 • Bug Fixes
51503 • Ignore files with control characters in the names
51505 • Fix rclone move command
51507 • Delete src files which already existed in dst
51509 • Fix deletion of src file when dst file older
51511 • Fix rclone check on crypted file systems
51513 • Make failed uploads not count as "Transferred"
51515 • Make sure high level retries show with -q
51517 • Use a vendor directory with godep for repeatable builds
51519 • rclone mount - FUSE
51521 • Implement FUSE mount options
51523 • --no-modtime, --debug-fuse, --read-only, --allow-non-empty, --al‐
51525 low-root, --allow-other
51526 • --default-permissions, --write-back-cache, --max-read-ahead,
51528 --umask, --uid, --gid
51529 • Add --dir-cache-time to control caching of directory entries
51531 • Implement seek for files opened for read (useful for video players)
51533 • with -no-seek flag to disable
51535 • Fix crash on 32 bit ARM (alignment of 64 bit counter)
51537 • ...and many more internal fixes and improvements!
51539 • Crypt
51541 • Don't show encrypted password in configurator to stop confusion
51543 • Amazon Drive
51545 • New wait for upload option --acd-upload-wait-per-gb
51547 • upload timeouts scale by file size and can be disabled
51549 • Add 502 Bad Gateway to list of errors we retry
51551 • Fix overwriting a file with a zero length file
51553 • Fix ACD file size warning limit - thanks Felix Bünemann
51555 • Local
51557 • Unix: implement -x/--one-file-system to stay on a single file sys‐
51559 tem
51560 • thanks Durval Menezes and Luiz Carlos Rumbelsperger Viana
51562 • Windows: ignore the symlink bit on files
51564 • Windows: Ignore directory-based junction points
51566 • B2
51568 • Make sure each upload has at least one upload slot - fixes strange
51570 upload stats
51571 • Fix uploads when using crypt
51573 • Fix download of large files (sha1 mismatch)
51575 • Return error when we try to create a bucket which someone else owns
51577 • Update B2 docs with Data usage, and Crypt section - thanks Tomasz
51579 Mazur
51580 • S3
51582 • Command line and config file support for
51584 • Setting/overriding ACL - thanks Radek Šenfeld
51586 • Setting storage class - thanks Asko Tamm
51588 • Drive
51590 • Make exponential backoff work exactly as per Google specification
51592 • add .epub, .odp and .tsv as export formats.
51594 • Swift
51596 • Don't read metadata for directory marker objects
51598 v1.33 - 2016-08-24
51600 • New Features
51601 • Implement encryption
51603 • data encrypted in NACL secretbox format
51605 • with optional file name encryption
51607 • New commands
51609 • rclone mount - implements FUSE mounting of remotes (EXPERIMENTAL)
51611 • works on Linux, FreeBSD and OS X (need testers for the last 2!)
51613 • rclone cat - outputs remote file or files to the terminal
51615 • rclone genautocomplete - command to make a bash completion script
51617 for rclone
51618 • Editing a remote using rclone config now goes through the wizard
51620 • Compile with go 1.7 - this fixes rclone on macOS Sierra and on 386
51622 processors
51623 • Use cobra for sub commands and docs generation
51625 • drive
51627 • Document how to make your own client_id
51629 • s3
51631 • User-configurable Amazon S3 ACL (thanks Radek Šenfeld)
51633 • b2
51635 • Fix stats accounting for upload - no more jumping to 100% done
51637 • On cleanup delete hide marker if it is the current file
51639 • New B2 API endpoint (thanks Per Cederberg)
51641 • Set maximum backoff to 5 Minutes
51643 • onedrive
51645 • Fix URL escaping in file names - e.g. uploading files with + in
51647 them.
51648 • amazon cloud drive
51650 • Fix token expiry during large uploads
51652 • Work around 408 REQUEST_TIMEOUT and 504 GATEWAY_TIMEOUT errors
51654 • local
51656 • Fix filenames with invalid UTF-8 not being uploaded
51658 • Fix problem with some UTF-8 characters on OS X
51660 v1.32 - 2016-07-13
51662 • Backblaze B2
51663 • Fix upload of files large files not in root
51665 v1.31 - 2016-07-13
51667 • New Features
51668 • Reduce memory on sync by about 50%
51670 • Implement --no-traverse flag to stop copy traversing the destina‐
51672 tion remote.
51673 • This can be used to reduce memory usage down to the smallest pos‐
51675 sible.
51676 • Useful to copy a small number of files into a large destination
51678 folder.
51679 • Implement cleanup command for emptying trash / removing old ver‐
51681 sions of files
51682 • Currently B2 only
51684 • Single file handling improved
51686 • Now copied with --files-from
51688 • Automatically sets --no-traverse when copying a single file
51690 • Info on using installing with ansible - thanks Stefan Weichinger
51692 • Implement --no-update-modtime flag to stop rclone fixing the remote
51694 modified times.
51695 • Bug Fixes
51697 • Fix move command - stop it running for overlapping Fses - this was
51699 causing data loss.
51700 • Local
51702 • Fix incomplete hashes - this was causing problems for B2.
51704 • Amazon Drive
51706 • Rename Amazon Cloud Drive to Amazon Drive - no changes to config
51708 file needed.
51709 • Swift
51711 • Add support for non-default project domain - thanks Antonio Messi‐
51713 na.
51714 • S3
51716 • Add instructions on how to use rclone with minio.
51718 • Add ap-northeast-2 (Seoul) and ap-south-1 (Mumbai) regions.
51720 • Skip setting the modified time for objects > 5GB as it isn't possi‐
51722 ble.
51723 • Backblaze B2
51725 • Add --b2-versions flag so old versions can be listed and retrieved.
51727 • Treat 403 errors (e.g. cap exceeded) as fatal.
51729 • Implement cleanup command for deleting old file versions.
51731 • Make error handling compliant with B2 integrations notes.
51733 • Fix handling of token expiry.
51735 • Implement --b2-test-mode to set X-Bz-Test-Mode header.
51737 • Set cutoff for chunked upload to 200MB as per B2 guidelines.
51739 • Make upload multi-threaded.
51741 • Dropbox
51743 • Don't retry 461 errors.
51745 v1.30 - 2016-06-18
51747 • New Features
51748 • Directory listing code reworked for more features and better error
51750 reporting (thanks to Klaus Post for help). This enables
51751 • Directory include filtering for efficiency
51753 • --max-depth parameter
51755 • Better error reporting
51757 • More to come
51759 • Retry more errors
51761 • Add --ignore-size flag - for uploading images to onedrive
51763 • Log -v output to stdout by default
51765 • Display the transfer stats in more human-readable form
51767 • Make 0 size files specifiable with --max-size 0b
51769 • Add b suffix so we can specify bytes in --bwlimit, --min-size, etc.
51771 • Use "password:" instead of "password>" prompt - thanks Klaus Post
51773 and Leigh Klotz
51774 • Bug Fixes
51776 • Fix retry doing one too many retries
51778 • Local
51780 • Fix problems with OS X and UTF-8 characters
51782 • Amazon Drive
51784 • Check a file exists before uploading to help with 408 Conflict er‐
51786 rors
51787 • Reauth on 401 errors - this has been causing a lot of problems
51789 • Work around spurious 403 errors
51791 • Restart directory listings on error
51793 • Google Drive
51795 • Check a file exists before uploading to help with duplicates
51797 • Fix retry of multipart uploads
51799 • Backblaze B2
51801 • Implement large file uploading
51803 • S3
51805 • Add AES256 server-side encryption for - thanks Justin R. Wilson
51807 • Google Cloud Storage
51809 • Make sure we don't use conflicting content types on upload
51811 • Add service account support - thanks Michal Witkowski
51813 • Swift
51815 • Add auth version parameter
51817 • Add domain option for openstack (v3 auth) - thanks Fabian Ruff
51819 v1.29 - 2016-04-18
51821 • New Features
51822 • Implement -I, --ignore-times for unconditional upload
51824 • Improve dedupecommand
51826 • Now removes identical copies without asking
51828 • Now obeys --dry-run
51830 • Implement --dedupe-mode for non interactive running
51832 • --dedupe-mode interactive - interactive the default.
51834 • --dedupe-mode skip - removes identical files then skips any‐
51836 thing left.
51837 • --dedupe-mode first - removes identical files then keeps the
51839 first one.
51840 • --dedupe-mode newest - removes identical files then keeps the
51842 newest one.
51843 • --dedupe-mode oldest - removes identical files then keeps the
51845 oldest one.
51846 • --dedupe-mode rename - removes identical files then renames the
51848 rest to be different.
51849 • Bug fixes
51851 • Make rclone check obey the --size-only flag.
51853 • Use "application/octet-stream" if discovered mime type is invalid.
51855 • Fix missing "quit" option when there are no remotes.
51857 • Google Drive
51859 • Increase default chunk size to 8 MB - increases upload speed of big
51861 files
51862 • Speed up directory listings and make more reliable
51864 • Add missing retries for Move and DirMove - increases reliability
51866 • Preserve mime type on file update
51868 • Backblaze B2
51870 • Enable mod time syncing
51872 • This means that B2 will now check modification times
51874 • It will upload new files to update the modification times
51876 • (there isn't an API to just set the mod time.)
51878 • If you want the old behaviour use --size-only.
51880 • Update API to new version
51882 • Fix parsing of mod time when not in metadata
51884 • Swift/Hubic
51886 • Don't return an MD5SUM for static large objects
51888 • S3
51890 • Fix uploading files bigger than 50GB
51892 v1.28 - 2016-03-01
51894 • New Features
51895 • Configuration file encryption - thanks Klaus Post
51897 • Improve rclone config adding more help and making it easier to un‐
51899 derstand
51900 • Implement -u/--update so creation times can be used on all remotes
51902 • Implement --low-level-retries flag
51904 • Optionally disable gzip compression on downloads with --no-gzip-en‐
51906 coding
51907 • Bug fixes
51909 • Don't make directories if --dry-run set
51911 • Fix and document the move command
51913 • Fix redirecting stderr on unix-like OSes when using --log-file
51915 • Fix delete command to wait until all finished - fixes missing
51917 deletes.
51918 • Backblaze B2
51920 • Use one upload URL per go routine fixes more than one upload using
51922 auth token
51923 • Add pacing, retries and reauthentication - fixes token expiry prob‐
51925 lems
51926 • Upload without using a temporary file from local (and remotes which
51928 support SHA1)
51929 • Fix reading metadata for all files when it shouldn't have been
51931 • Drive
51933 • Fix listing drive documents at root
51935 • Disable copy and move for Google docs
51937 • Swift
51939 • Fix uploading of chunked files with non ASCII characters
51941 • Allow setting of storage_url in the config - thanks Xavier Lucas
51943 • S3
51945 • Allow IAM role and credentials from environment variables - thanks
51947 Brian Stengaard
51948 • Allow low privilege users to use S3 (check if directory exists dur‐
51950 ing Mkdir) - thanks Jakub Gedeon
51951 • Amazon Drive
51953 • Retry on more things to make directory listings more reliable
51955 v1.27 - 2016-01-31
51957 • New Features
51958 • Easier headless configuration with rclone authorize
51960 • Add support for multiple hash types - we now check SHA1 as well as
51962 MD5 hashes.
51963 • delete command which does obey the filters (unlike purge)
51965 • dedupe command to deduplicate a remote. Useful with Google Drive.
51967 • Add --ignore-existing flag to skip all files that exist on destina‐
51969 tion.
51970 • Add --delete-before, --delete-during, --delete-after flags.
51972 • Add --memprofile flag to debug memory use.
51974 • Warn the user about files with same name but different case
51976 • Make --include rules add their implicit exclude * at the end of the
51978 filter list
51979 • Deprecate compiling with go1.3
51981 • Amazon Drive
51983 • Fix download of files > 10 GB
51985 • Fix directory traversal ("Next token is expired") for large direc‐
51987 tory listings
51988 • Remove 409 conflict from error codes we will retry - stops very
51990 long pauses
51991 • Backblaze B2
51993 • SHA1 hashes now checked by rclone core
51995 • Drive
51997 • Add --drive-auth-owner-only to only consider files owned by the us‐
51999 er - thanks Björn Harrtell
52000 • Export Google documents
52002 • Dropbox
52004 • Make file exclusion error controllable with -q
52006 • Swift
52008 • Fix upload from unprivileged user.
52010 • S3
52012 • Fix updating of mod times of files with + in.
52014 • Local
52016 • Add local file system option to disable UNC on Windows.
52018 v1.26 - 2016-01-02
52020 • New Features
52021 • Yandex storage backend - thank you Dmitry Burdeev ("dibu")
52023 • Implement Backblaze B2 storage backend
52025 • Add --min-age and --max-age flags - thank you Adriano Aurélio
52027 Meirelles
52028 • Make ls/lsl/md5sum/size/check obey includes and excludes
52030 • Fixes
52032 • Fix crash in http logging
52034 • Upload releases to github too
52036 • Swift
52038 • Fix sync for chunked files
52040 • OneDrive
52042 • Re-enable server-side copy
52044 • Don't mask HTTP error codes with JSON decode error
52046 • S3
52048 • Fix corrupting Content-Type on mod time update (thanks Joseph
52050 Spurrier)
52051 v1.25 - 2015-11-14
52053 • New features
52054 • Implement Hubic storage system
52056 • Fixes
52058 • Fix deletion of some excluded files without --delete-excluded
52060 • This could have deleted files unexpectedly on sync
52062 • Always check first with --dry-run!
52064 • Swift
52066 • Stop SetModTime losing metadata (e.g. X-Object-Manifest)
52068 • This could have caused data loss for files > 5GB in size
52070 • Use ContentType from Object to avoid lookups in listings
52072 • OneDrive
52074 • disable server-side copy as it seems to be broken at Microsoft
52076 v1.24 - 2015-11-07
52078 • New features
52079 • Add support for Microsoft OneDrive
52081 • Add --no-check-certificate option to disable server certificate
52083 verification
52084 • Add async readahead buffer for faster transfer of big files
52086 • Fixes
52088 • Allow spaces in remotes and check remote names for validity at cre‐
52090 ation time
52091 • Allow '&' and disallow ':' in Windows filenames.
52093 • Swift
52095 • Ignore directory marker objects where appropriate - allows working
52097 with Hubic
52098 • Don't delete the container if fs wasn't at root
52100 • S3
52102 • Don't delete the bucket if fs wasn't at root
52104 • Google Cloud Storage
52106 • Don't delete the bucket if fs wasn't at root
52108 v1.23 - 2015-10-03
52110 • New features
52111 • Implement rclone size for measuring remotes
52113 • Fixes
52115 • Fix headless config for drive and gcs
52117 • Tell the user they should try again if the webserver method failed
52119 • Improve output of --dump-headers
52121 • S3
52123 • Allow anonymous access to public buckets
52125 • Swift
52127 • Stop chunked operations logging "Failed to read info: Object Not
52129 Found"
52130 • Use Content-Length on uploads for extra reliability
52132 v1.22 - 2015-09-28
52134 • Implement rsync like include and exclude flags
52135 • swift
52137 • Support files > 5GB - thanks Sergey Tolmachev
52139 v1.21 - 2015-09-22
52141 • New features
52142 • Display individual transfer progress
52144 • Make lsl output times in localtime
52146 • Fixes
52148 • Fix allowing user to override credentials again in Drive, GCS and
52150 ACD
52151 • Amazon Drive
52153 • Implement compliant pacing scheme
52155 • Google Drive
52157 • Make directory reads concurrent for increased speed.
52159 v1.20 - 2015-09-15
52161 • New features
52162 • Amazon Drive support
52164 • Oauth support redone - fix many bugs and improve usability
52166 • Use "golang.org/x/oauth2" as oauth library of choice
52168 • Improve oauth usability for smoother initial signup
52170 • drive, googlecloudstorage: optionally use auto config for the
52172 oauth token
52173 • Implement --dump-headers and --dump-bodies debug flags
52175 • Show multiple matched commands if abbreviation too short
52177 • Implement server-side move where possible
52179 • local
52181 • Always use UNC paths internally on Windows - fixes a lot of bugs
52183 • dropbox
52185 • force use of our custom transport which makes timeouts work
52187 • Thanks to Klaus Post for lots of help with this release
52189 v1.19 - 2015-08-28
52191 • New features
52192 • Server side copies for s3/swift/drive/dropbox/gcs
52194 • Move command - uses server-side copies if it can
52196 • Implement --retries flag - tries 3 times by default
52198 • Build for plan9/amd64 and solaris/amd64 too
52200 • Fixes
52202 • Make a current version download with a fixed URL for scripting
52204 • Ignore rmdir in limited fs rather than throwing error
52206 • dropbox
52208 • Increase chunk size to improve upload speeds massively
52210 • Issue an error message when trying to upload bad file name
52212 v1.18 - 2015-08-17
52214 • drive
52215 • Add --drive-use-trash flag so rclone trashes instead of deletes
52217 • Add "Forbidden to download" message for files with no downloadURL
52219 • dropbox
52221 • Remove datastore
52223 • This was deprecated and it caused a lot of problems
52225 • Modification times and MD5SUMs no longer stored
52227 • Fix uploading files > 2GB
52229 • s3
52231 • use official AWS SDK from github.com/aws/aws-sdk-go
52233 • NB will most likely require you to delete and recreate remote
52235 • enable multipart upload which enables files > 5GB
52237 • tested with Ceph / RadosGW / S3 emulation
52239 • many thanks to Sam Liston and Brian Haymore at the Utah Center for
52241 High Performance Computing (https://www.chpc.utah.edu/) for a Ceph
52242 test account
52243 • misc
52245 • Show errors when reading the config file
52247 • Do not print stats in quiet mode - thanks Leonid Shalupov
52249 • Add FAQ
52251 • Fix created directories not obeying umask
52253 • Linux installation instructions - thanks Shimon Doodkin
52255 v1.17 - 2015-06-14
52257 • dropbox: fix case insensitivity issues - thanks Leonid Shalupov
52258 v1.16 - 2015-06-09
52260 • Fix uploading big files which was causing timeouts or panics
52261 • Don't check md5sum after download with --size-only
52263 v1.15 - 2015-06-06
52265 • Add --checksum flag to only discard transfers by MD5SUM - thanks Alex
52266 Couper
52267 • Implement --size-only flag to sync on size not checksum & modtime
52269 • Expand docs and remove duplicated information
52271 • Document rclone's limitations with directories
52273 • dropbox: update docs about case insensitivity
52275 v1.14 - 2015-05-21
52277 • local: fix encoding of non utf-8 file names - fixes a duplicate file
52278 problem
52279 • drive: docs about rate limiting
52281 • google cloud storage: Fix compile after API change in
52283 "google.golang.org/api/storage/v1"
52284 v1.13 - 2015-05-10
52286 • Revise documentation (especially sync)
52287 • Implement --timeout and --conntimeout
52289 • s3: ignore etags from multipart uploads which aren't md5sums
52291 v1.12 - 2015-03-15
52293 • drive: Use chunked upload for files above a certain size
52294 • drive: add --drive-chunk-size and --drive-upload-cutoff parameters
52296 • drive: switch to insert from update when a failed copy deletes the
52298 upload
52299 • core: Log duplicate files if they are detected
52301 v1.11 - 2015-03-04
52303 • swift: add region parameter
52304 • drive: fix crash on failed to update remote mtime
52306 • In remote paths, change native directory separators to /
52308 • Add synchronization to ls/lsl/lsd output to stop corruptions
52310 • Ensure all stats/log messages to go stderr
52312 • Add --log-file flag to log everything (including panics) to file
52314 • Make it possible to disable stats printing with --stats=0
52316 • Implement --bwlimit to limit data transfer bandwidth
52318 v1.10 - 2015-02-12
52320 • s3: list an unlimited number of items
52321 • Fix getting stuck in the configurator
52323 v1.09 - 2015-02-07
52325 • windows: Stop drive letters (e.g. C:) getting mixed up with remotes
52326 (e.g. drive:)
52327 • local: Fix directory separators on Windows
52329 • drive: fix rate limit exceeded errors
52331 v1.08 - 2015-02-04
52333 • drive: fix subdirectory listing to not list entire drive
52334 • drive: Fix SetModTime
52336 • dropbox: adapt code to recent library changes
52338 v1.07 - 2014-12-23
52340 • google cloud storage: fix memory leak
52341 v1.06 - 2014-12-12
52343 • Fix "Couldn't find home directory" on OSX
52344 • swift: Add tenant parameter
52346 • Use new location of Google API packages
52348 v1.05 - 2014-08-09
52350 • Improved tests and consequently lots of minor fixes
52351 • core: Fix race detected by go race detector
52353 • core: Fixes after running errcheck
52355 • drive: reset root directory on Rmdir and Purge
52357 • fs: Document that Purger returns error on empty directory, test and
52359 fix
52360 • google cloud storage: fix ListDir on subdirectory
52362 • google cloud storage: re-read metadata in SetModTime
52364 • s3: make reading metadata more reliable to work around eventual con‐
52366 sistency problems
52367 • s3: strip trailing / from ListDir()
52369 • swift: return directories without / in ListDir
52371 v1.04 - 2014-07-21
52373 • google cloud storage: Fix crash on Update
52374 v1.03 - 2014-07-20
52376 • swift, s3, dropbox: fix updated files being marked as corrupted
52377 • Make compile with go 1.1 again
52379 v1.02 - 2014-07-19
52381 • Implement Dropbox remote
52382 • Implement Google Cloud Storage remote
52384 • Verify Md5sums and Sizes after copies
52386 • Remove times from "ls" command - lists sizes only
52388 • Add add "lsl" - lists times and sizes
52390 • Add "md5sum" command
52392 v1.01 - 2014-07-04
52394 • drive: fix transfer of big files using up lots of memory
52395 v1.00 - 2014-07-03
52397 • drive: fix whole second dates
52398 v0.99 - 2014-06-26
52400 • Fix --dry-run not working
52401 • Make compatible with go 1.1
52403 v0.98 - 2014-05-30
52405 • s3: Treat missing Content-Length as 0 for some ceph installations
52406 • rclonetest: add file with a space in
52408 v0.97 - 2014-05-05
52410 • Implement copying of single files
52411 • s3 & swift: support paths inside containers/buckets
52413 v0.96 - 2014-04-24
52415 • drive: Fix multiple files of same name being created
52416 • drive: Use o.Update and fs.Put to optimise transfers
52418 • Add version number, -V and --version
52420 v0.95 - 2014-03-28
52422 • rclone.org: website, docs and graphics
52423 • drive: fix path parsing
52425 v0.94 - 2014-03-27
52427 • Change remote format one last time
52428 • GNU style flags
52430 v0.93 - 2014-03-16
52432 • drive: store token in config file
52433 • cross compile other versions
52435 • set strict permissions on config file
52437 v0.92 - 2014-03-15
52439 • Config fixes and --config option
52440 v0.91 - 2014-03-15
52442 • Make config file
52443 v0.90 - 2013-06-27
52445 • Project named rclone
52446 v0.00 - 2012-11-18
52448 • Project started
52449
52451 Limitations
52452 Directory timestamps aren't preserved
52453 Rclone doesn't currently preserve the timestamps of directories. This
52454 is because rclone only really considers objects when syncing.
52455 Rclone struggles with millions of files in a directory/bucket
52457 Currently rclone loads each directory/bucket entirely into memory be‐
52458 fore using it. Since each rclone object takes 0.5k-1k of memory this
52459 can take a very long time and use a large amount of memory.
52460 Millions of files in a directory tends to occur on bucket-based remotes
52462 (e.g. S3 buckets) since those remotes do not segregate subdirectories
52463 within the bucket.
52464 Bucket-based remotes and folders
52466 Bucket-based remotes (e.g. S3/GCS/Swift/B2) do not have a concept of
52467 directories. Rclone therefore cannot create directories in them which
52468 means that empty directories on a bucket-based remote will tend to dis‐
52469 appear.
52470 Some software creates empty keys ending in / as directory markers.
52472 Rclone doesn't do this as it potentially creates more objects and costs
52473 more. This ability may be added in the future (probably via a flag/op‐
52474 tion).
52475 Bugs
52477 Bugs are stored in rclone's GitHub project:
52478 • Reported bugs (https://github.com/rclone/rclone/is‐
52480 sues?q=is%3Aopen+is%3Aissue+label%3Abug)
52481 • Known issues (https://github.com/rclone/rclone/is‐
52483 sues?q=is%3Aopen+is%3Aissue+milestone%3A%22Known+Problem%22)
52484
52486 Do all cloud storage systems support all rclone commands
52487 Yes they do. All the rclone commands (e.g. sync, copy, etc.) will
52488 work on all the remote storage systems.
52489 Can I copy the config from one machine to another
52491 Sure! Rclone stores all of its config in a single file. If you want to
52492 find this file, run rclone config file which will tell you where it is.
52493 See the remote setup docs (https://rclone.org/remote_setup/) for more
52495 info.
52496 How do I configure rclone on a remote / headless box with no browser?
52498 This has now been documented in its own remote setup page
52499 (https://rclone.org/remote_setup/).
52500 Can rclone sync directly from drive to s3
52502 Rclone can sync between two remote cloud storage systems just fine.
52503 Note that it effectively downloads the file and uploads it again, so
52505 the node running rclone would need to have lots of bandwidth.
52506 The syncs would be incremental (on a file by file basis).
52508 e.g.
52510 rclone sync -i drive:Folder s3:bucket
52512 Using rclone from multiple locations at the same time
52514 You can use rclone from multiple places at the same time if you choose
52515 different subdirectory for the output, e.g.
52516 Server A> rclone sync -i /tmp/whatever remote:ServerA
52518 Server B> rclone sync -i /tmp/whatever remote:ServerB
52519 If you sync to the same directory then you should use rclone copy oth‐
52521 erwise the two instances of rclone may delete each other's files, e.g.
52522 Server A> rclone copy /tmp/whatever remote:Backup
52524 Server B> rclone copy /tmp/whatever remote:Backup
52525 The file names you upload from Server A and Server B should be differ‐
52527 ent in this case, otherwise some file systems (e.g. Drive) may make
52528 duplicates.
52529 Why doesn't rclone support partial transfers / binary diffs like rsync?
52531 Rclone stores each file you transfer as a native object on the remote
52532 cloud storage system. This means that you can see the files you upload
52533 as expected using alternative access methods (e.g. using the Google
52534 Drive web interface). There is a 1:1 mapping between files on your
52535 hard disk and objects created in the cloud storage system.
52536 Cloud storage systems (at least none I've come across yet) don't sup‐
52538 port partially uploading an object. You can't take an existing object,
52539 and change some bytes in the middle of it.
52540 It would be possible to make a sync system which stored binary diffs
52542 instead of whole objects like rclone does, but that would break the 1:1
52543 mapping of files on your hard disk to objects in the remote cloud stor‐
52544 age system.
52545 All the cloud storage systems support partial downloads of content, so
52547 it would be possible to make partial downloads work. However to make
52548 this work efficiently this would require storing a significant amount
52549 of metadata, which breaks the desired 1:1 mapping of files to objects.
52550 Can rclone do bi-directional sync?
52552 Yes, since rclone v1.58.0, bidirectional cloud sync
52553 (https://rclone.org/bisync/) is available.
52554 Can I use rclone with an HTTP proxy?
52556 Yes. rclone will follow the standard environment variables for prox‐
52557 ies, similar to cURL and other programs.
52558 In general the variables are called http_proxy (for services reached
52560 over http) and https_proxy (for services reached over https). Most
52561 public services will be using https, but you may wish to set both.
52562 The content of the variable is protocol://server:port. The protocol
52564 value is the one used to talk to the proxy server, itself, and is com‐
52565 monly either http or socks5.
52566 Slightly annoyingly, there is no standard for the name; some applica‐
52568 tions may use http_proxy but another one HTTP_PROXY. The Go libraries
52569 used by rclone will try both variations, but you may wish to set all
52570 possibilities. So, on Linux, you may end up with code similar to
52571 export http_proxy=http://proxyserver:12345
52573 export https_proxy=$http_proxy
52574 export HTTP_PROXY=$http_proxy
52575 export HTTPS_PROXY=$http_proxy
52576 Note: If the proxy server requires a username and password, then use
52578 export http_proxy=http://username:password@proxyserver:12345
52580 export https_proxy=$http_proxy
52581 export HTTP_PROXY=$http_proxy
52582 export HTTPS_PROXY=$http_proxy
52583 The NO_PROXY allows you to disable the proxy for specific hosts. Hosts
52585 must be comma separated, and can contain domains or parts. For in‐
52586 stance "foo.com" also matches "bar.foo.com".
52587 e.g.
52589 export no_proxy=localhost,127.0.0.0/8,my.host.name
52591 export NO_PROXY=$no_proxy
52592 Note that the FTP backend does not support ftp_proxy yet.
52594 Rclone gives x509: failed to load system roots and no roots provided error
52596 This means that rclone can't find the SSL root certificates. Likely
52597 you are running rclone on a NAS with a cut-down Linux OS, or possibly
52598 on Solaris.
52599 Rclone (via the Go runtime) tries to load the root certificates from
52601 these places on Linux.
52602 "/etc/ssl/certs/ca-certificates.crt", // Debian/Ubuntu/Gentoo etc.
52604 "/etc/pki/tls/certs/ca-bundle.crt", // Fedora/RHEL
52605 "/etc/ssl/ca-bundle.pem", // OpenSUSE
52606 "/etc/pki/tls/cacert.pem", // OpenELEC
52607 So doing something like this should fix the problem. It also sets the
52609 time which is important for SSL to work properly.
52610 mkdir -p /etc/ssl/certs/
52612 curl -o /etc/ssl/certs/ca-certificates.crt https://raw.githubusercontent.com/bagder/ca-bundle/master/ca-bundle.crt
52613 ntpclient -s -h pool.ntp.org
52614 The two environment variables SSL_CERT_FILE and SSL_CERT_DIR, mentioned
52616 in the x509 package (https://godoc.org/crypto/x509), provide an addi‐
52617 tional way to provide the SSL root certificates.
52618 Note that you may need to add the --insecure option to the curl command
52620 line if it doesn't work without.
52621 curl --insecure -o /etc/ssl/certs/ca-certificates.crt https://raw.githubusercontent.com/bagder/ca-bundle/master/ca-bundle.crt
52623 Rclone gives Failed to load config file: function not implemented error
52625 Likely this means that you are running rclone on Linux version not sup‐
52626 ported by the go runtime, ie earlier than version 2.6.23.
52627 See the system requirements section in the go install docs
52629 (https://golang.org/doc/install) for full details.
52630 All my uploaded docx/xlsx/pptx files appear as archive/zip
52632 This is caused by uploading these files from a Windows computer which
52633 hasn't got the Microsoft Office suite installed. The easiest way to
52634 fix is to install the Word viewer and the Microsoft Office Compatibili‐
52635 ty Pack for Word, Excel, and PowerPoint 2007 and later versions' file
52636 formats
52637 tcp lookup some.domain.com no such host
52639 This happens when rclone cannot resolve a domain. Please check that
52640 your DNS setup is generally working, e.g.
52641 # both should print a long list of possible IP addresses
52643 dig www.googleapis.com # resolve using your default DNS
52644 dig www.googleapis.com @8.8.8.8 # resolve with Google's DNS server
52645 If you are using systemd-resolved (default on Arch Linux), ensure it is
52647 at version 233 or higher. Previous releases contain a bug which causes
52648 not all domains to be resolved properly.
52649 Additionally with the GODEBUG=netdns= environment variable the Go re‐
52651 solver decision can be influenced. This also allows to resolve certain
52652 issues with DNS resolution. See the name resolution section in the go
52653 docs (https://golang.org/pkg/net/#hdr-Name_Resolution).
52654 The total size reported in the stats for a sync is wrong and keeps changing
52656 It is likely you have more than 10,000 files that need to be synced.
52657 By default, rclone only gets 10,000 files ahead in a sync so as not to
52658 use up too much memory. You can change this default with the --max-
52659 backlog (https://rclone.org/docs/#max-backlog-n) flag.
52660 Rclone is using too much memory or appears to have a memory leak
52662 Rclone is written in Go which uses a garbage collector. The default
52663 settings for the garbage collector mean that it runs when the heap size
52664 has doubled.
52665 However it is possible to tune the garbage collector to use less memory
52667 by setting GOGC (https://dave.cheney.net/tag/gogc) to a lower value,
52668 say export GOGC=20. This will make the garbage collector work harder,
52669 reducing memory size at the expense of CPU usage.
52670 The most common cause of rclone using lots of memory is a single direc‐
52672 tory with thousands or millions of files in. Rclone has to load this
52673 entirely into memory as rclone objects. Each rclone object takes
52674 0.5k-1k of memory.
52675 Rclone changes fullwidth Unicode punctuation marks in file names
52677 For example: On a Windows system, you have a file with name
52678 Test：1.jpg, where ： is the Unicode fullwidth colon symbol. When us‐
52679 ing rclone to copy this to your Google Drive, you will notice that the
52680 file gets renamed to Test:1.jpg, where : is the regular (halfwidth)
52681 colon.
52682 The reason for such renames is the way rclone handles different re‐
52684 stricted filenames (https://rclone.org/overview/#restricted-filenames)
52685 on different cloud storage systems. It tries to avoid ambiguous file
52686 names as much and allow moving files between many cloud storage systems
52687 transparently, by replacing invalid characters with similar looking
52688 Unicode characters when transferring to one storage system, and replac‐
52689 ing back again when transferring to a different storage system where
52690 the original characters are supported. When the same Unicode charac‐
52691 ters are intentionally used in file names, this replacement strategy
52692 leads to unwanted renames. Read more here (https://rclone.org/over‐
52693 view/#restricted-filenames-caveats).
52694
52696 This is free software under the terms of the MIT license (check the
52697 COPYING file included with the source code).
52698 Copyright (C) 2019 by Nick Craig-Wood https://www.craig-wood.com/nick/
52700 Permission is hereby granted, free of charge, to any person obtaining a copy
52702 of this software and associated documentation files (the "Software"), to deal
52703 in the Software without restriction, including without limitation the rights
52704 to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
52705 copies of the Software, and to permit persons to whom the Software is
52706 furnished to do so, subject to the following conditions:
52707 The above copyright notice and this permission notice shall be included in
52709 all copies or substantial portions of the Software.
52710 THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
52712 IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
52713 FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
52714 AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
52715 LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
52716 OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
52717 THE SOFTWARE.
52718
52720 Authors
52721 • Nick Craig-Wood <nick@craig-wood.com>
52722 Contributors
52724 {{< rem email addresses removed from here need to be addeed to bin/.ig‐
52725 nore-emails to make sure update-authors.py doesn't immediately put them
52726 back in again. >}}
52727 • Alex Couper <amcouper@gmail.com>
52729 • Leonid Shalupov <leonid@shalupov.com> <shalupov@diverse.org.ru>
52731 • Shimon Doodkin <helpmepro1@gmail.com>
52733 • Colin Nicholson <colin@colinn.com>
52735 • Klaus Post <klauspost@gmail.com>
52737 • Sergey Tolmachev <tolsi.ru@gmail.com>
52739 • Adriano Aurélio Meirelles <adriano@atinge.com>
52741 • C. Bess <cbess@users.noreply.github.com>
52743 • Dmitry Burdeev <dibu28@gmail.com>
52745 • Joseph Spurrier <github@josephspurrier.com>
52747 • Björn Harrtell <bjorn@wololo.org>
52749 • Xavier Lucas <xavier.lucas@corp.ovh.com>
52751 • Werner Beroux <werner@beroux.com>
52753 • Brian Stengaard <brian@stengaard.eu>
52755 • Jakub Gedeon <jgedeon@sofi.com>
52757 • Jim Tittsler <jwt@onjapan.net>
52759 • Michal Witkowski <michal@improbable.io>
52761 • Fabian Ruff <fabian.ruff@sap.com>
52763 • Leigh Klotz <klotz@quixey.com>
52765 • Romain Lapray <lapray.romain@gmail.com>
52767 • Justin R. Wilson <jrw972@gmail.com>
52769 • Antonio Messina <antonio.s.messina@gmail.com>
52771 • Stefan G. Weichinger <office@oops.co.at>
52773 • Per Cederberg <cederberg@gmail.com>
52775 • Radek Šenfeld <rush@logic.cz>
52777 • Fredrik Fornwall <fredrik@fornwall.net>
52779 • Asko Tamm <asko@deekit.net>
52781 • xor-zz <xor@gstocco.com>
52783 • Tomasz Mazur <tmazur90@gmail.com>
52785 • Marco Paganini <paganini@paganini.net>
52787 • Felix Bünemann <buenemann@louis.info>
52789 • Durval Menezes <jmrclone@durval.com>
52791 • Luiz Carlos Rumbelsperger Viana <maxd13_luiz_carlos@hotmail.com>
52793 • Stefan Breunig <stefan-github@yrden.de>
52795 • Alishan Ladhani <ali-l@users.noreply.github.com>
52797 • 0xJAKE <0xJAKE@users.noreply.github.com>
52799 • Thibault Molleman <thibaultmol@users.noreply.github.com>
52801 • Scott McGillivray <scott.mcgillivray@gmail.com>
52803 • Bjørn Erik Pedersen <bjorn.erik.pedersen@gmail.com>
52805 • Lukas Loesche <lukas@mesosphere.io>
52807 • emyarod <allllaboutyou@gmail.com>
52809 • T.C. Ferguson <tcf909@gmail.com>
52811 • Brandur <brandur@mutelight.org>
52813 • Dario Giovannetti <dev@dariogiovannetti.net>
52815 • Károly Oláh <okaresz@aol.com>
52817 • Jon Yergatian <jon@macfanatic.ca>
52819 • Jack Schmidt <github@mowsey.org>
52821 • Dedsec1 <Dedsec1@users.noreply.github.com>
52823 • Hisham Zarka <hzarka@gmail.com>
52825 • Jérôme Vizcaino <jerome.vizcaino@gmail.com>
52827 • Mike Tesch <mjt6129@rit.edu>
52829 • Marvin Watson <marvwatson@users.noreply.github.com>
52831 • Danny Tsai <danny8376@gmail.com>
52833 • Yoni Jah <yonjah+git@gmail.com> <yonjah+github@gmail.com>
52835 • Stephen Harris <github@spuddy.org> <sweharris@users.nore‐
52837 ply.github.com>
52838 • Ihor Dvoretskyi <ihor.dvoretskyi@gmail.com>
52840 • Jon Craton <jncraton@gmail.com>
52842 • Hraban Luyat <hraban@0brg.net>
52844 • Michael Ledin <mledin89@gmail.com>
52846 • Martin Kristensen <me@azgul.com>
52848 • Too Much IO <toomuchio@users.noreply.github.com>
52850 • Anisse Astier <anisse@astier.eu>
52852 • Zahiar Ahmed <zahiar@live.com>
52854 • Igor Kharin <igorkharin@gmail.com>
52856 • Bill Zissimopoulos <billziss@navimatics.com>
52858 • Bob Potter <bobby.potter@gmail.com>
52860 • Steven Lu <tacticalazn@gmail.com>
52862 • Sjur Fredriksen <sjurtf@ifi.uio.no>
52864 • Ruwbin <hubus12345@gmail.com>
52866 • Fabian Möller <fabianm88@gmail.com> <f.moeller@nynex.de>
52868 • Edward Q. Bridges <github@eqbridges.com>
52870 • Vasiliy Tolstov <v.tolstov@selfip.ru>
52872 • Harshavardhana <harsha@minio.io>
52874 • sainaen <sainaen@gmail.com>
52876 • gdm85 <gdm85@users.noreply.github.com>
52878 • Yaroslav Halchenko <debian@onerussian.com>
52880 • John Papandriopoulos <jpap@users.noreply.github.com>
52882 • Zhiming Wang <zmwangx@gmail.com>
52884 • Andy Pilate <cubox@cubox.me>
52886 • Oliver Heyme <olihey@googlemail.com> <olihey@users.nore‐
52888 ply.github.com> <de8olihe@lego.com>
52889 • wuyu <wuyu@yunify.com>
52891 • Andrei Dragomir <adragomi@adobe.com>
52893 • Christian Brüggemann <mail@cbruegg.com>
52895 • Alex McGrath Kraak <amkdude@gmail.com>
52897 • bpicode <bjoern.pirnay@googlemail.com>
52899 • Daniel Jagszent <daniel@jagszent.de>
52901 • Josiah White <thegenius2009@gmail.com>
52903 • Ishuah Kariuki <kariuki@ishuah.com> <ishuah91@gmail.com>
52905 • Jan Varho <jan@varho.org>
52907 • Girish Ramakrishnan <girish@cloudron.io>
52909 • LingMan <LingMan@users.noreply.github.com>
52911 • Jacob McNamee <jacobmcnamee@gmail.com>
52913 • jersou <jertux@gmail.com>
52915 • thierry <thierry@substantiel.fr>
52917 • Simon Leinen <simon.leinen@gmail.com> <ubuntu@s3-test.novalocal>
52919 • Dan Dascalescu <ddascalescu+github@gmail.com>
52921 • Jason Rose <jason@jro.io>
52923 • Andrew Starr-Bochicchio <a.starr.b@gmail.com>
52925 • John Leach <john@johnleach.co.uk>
52927 • Corban Raun <craun@instructure.com>
52929 • Pierre Carlson <mpcarl@us.ibm.com>
52931 • Ernest Borowski <er.borowski@gmail.com>
52933 • Remus Bunduc <remus.bunduc@gmail.com>
52935 • Iakov Davydov <iakov.davydov@unil.ch> <dav05.gith@myths.ru>
52937 • Jakub Tasiemski <tasiemski@gmail.com>
52939 • David Minor <dminor@saymedia.com>
52941 • Tim Cooijmans <cooijmans.tim@gmail.com>
52943 • Laurence <liuxy6@gmail.com>
52945 • Giovanni Pizzi <gio.piz@gmail.com>
52947 • Filip Bartodziej <filipbartodziej@gmail.com>
52949 • Jon Fautley <jon@dead.li>
52951 • lewapm <32110057+lewapm@users.noreply.github.com>
52953 • Yassine Imounachen <yassine256@gmail.com>
52955 • Chris Redekop <chris-redekop@users.noreply.github.com> <chris.re‐
52957 dekop@gmail.com>
52958 • Jon Fautley <jon@adenoid.appstal.co.uk>
52960 • Will Gunn <WillGunn@users.noreply.github.com>
52962 • Lucas Bremgartner <lucas@bremis.ch>
52964 • Jody Frankowski <jody.frankowski@gmail.com>
52966 • Andreas Roussos <arouss1980@gmail.com>
52968 • nbuchanan <nbuchanan@utah.gov>
52970 • Durval Menezes <rclone@durval.com>
52972 • Victor <vb-github@viblo.se>
52974 • Mateusz <pabian.mateusz@gmail.com>
52976 • Daniel Loader <spicypixel@gmail.com>
52978 • David0rk <davidork@gmail.com>
52980 • Alexander Neumann <alexander@bumpern.de>
52982 • Giri Badanahatti <gbadanahatti@us.ibm.com@Giris-MacBook-Pro.local>
52984 • Leo R. Lundgren <leo@finalresort.org>
52986 • wolfv <wolfv6@users.noreply.github.com>
52988 • Dave Pedu <dave@davepedu.com>
52990 • Stefan Lindblom <lindblom@spotify.com>
52992 • seuffert <oliver@seuffert.biz>
52994 • gbadanahatti <37121690+gbadanahatti@users.noreply.github.com>
52996 • Keith Goldfarb <barkofdelight@gmail.com>
52998 • Steve Kriss <steve@heptio.com>
53000 • Chih-Hsuan Yen <yan12125@gmail.com>
53002 • Alexander Neumann <fd0@users.noreply.github.com>
53004 • Matt Holt <mholt@users.noreply.github.com>
53006 • Eri Bastos <bastos.eri@gmail.com>
53008 • Michael P. Dubner <pywebmail@list.ru>
53010 • Antoine GIRARD <sapk@users.noreply.github.com>
53012 • Mateusz Piotrowski <mpp302@gmail.com>
53014 • Animosity022 <animosity22@users.noreply.github.com> <earl.tex‐
53016 ter@gmail.com>
53017 • Peter Baumgartner <pete@lincolnloop.com>
53019 • Craig Rachel <craig@craigrachel.com>
53021 • Michael G. Noll <miguno@users.noreply.github.com>
53023 • hensur <me@hensur.de>
53025 • Oliver Heyme <de8olihe@lego.com>
53027 • Richard Yang <richard@yenforyang.com>
53029 • Piotr Oleszczyk <piotr.oleszczyk@gmail.com>
53031 • Rodrigo <rodarima@gmail.com>
53033 • NoLooseEnds <NoLooseEnds@users.noreply.github.com>
53035 • Jakub Karlicek <jakub@karlicek.me>
53037 • John Clayton <john@codemonkeylabs.com>
53039 • Kasper Byrdal Nielsen <byrdal76@gmail.com>
53041 • Benjamin Joseph Dag <bjdag1234@users.noreply.github.com>
53043 • themylogin <themylogin@gmail.com>
53045 • Onno Zweers <onno.zweers@surfsara.nl>
53047 • Jasper Lievisse Adriaanse <jasper@humppa.nl>
53049 • sandeepkru <sandeep.ummadi@gmail.com> <sandeepkru@users.nore‐
53051 ply.github.com>
53052 • HerrH <atomtigerzoo@users.noreply.github.com>
53054 • Andrew <4030760+sparkyman215@users.noreply.github.com>
53056 • dan smith <XX1011@gmail.com>
53058 • Oleg Kovalov <iamolegkovalov@gmail.com>
53060 • Ruben Vandamme <github-com-00ff86@vandamme.email>
53062 • Cnly <minecnly@gmail.com>
53064 • Andres Alvarez <1671935+kir4h@users.noreply.github.com>
53066 • reddi1 <xreddi@gmail.com>
53068 • Matt Tucker <matthewtckr@gmail.com>
53070 • Sebastian Bünger <buengese@gmail.com> <buengese@protonmail.com>
53072 • Martin Polden <mpolden@mpolden.no>
53074 • Alex Chen <Cnly@users.noreply.github.com>
53076 • Denis <deniskovpen@gmail.com>
53078 • bsteiss <35940619+bsteiss@users.noreply.github.com>
53080 • Cédric Connes <cedric.connes@gmail.com>
53082 • Dr. Tobias Quathamer <toddy15@users.noreply.github.com>
53084 • dcpu <42736967+dcpu@users.noreply.github.com>
53086 • Sheldon Rupp <me@shel.io>
53088 • albertony <12441419+albertony@users.noreply.github.com>
53090 • cron410 <cron410@gmail.com>
53092 • Anagh Kumar Baranwal <6824881+darthShadow@users.noreply.github.com>
53094 • Felix Brucker <felix@felixbrucker.com>
53096 • Santiago Rodríguez <scollazo@users.noreply.github.com>
53098 • Craig Miskell <craig.miskell@fluxfederation.com>
53100 • Antoine GIRARD <sapk@sapk.fr>
53102 • Joanna Marek <joanna.marek@u2i.com>
53104 • frenos <frenos@users.noreply.github.com>
53106 • ssaqua <ssaqua@users.noreply.github.com>
53108 • xnaas <me@xnaas.info>
53110 • Frantisek Fuka <fuka@fuxoft.cz>
53112 • Paul Kohout <pauljkohout@yahoo.com>
53114 • dcpu <43330287+dcpu@users.noreply.github.com>
53116 • jackyzy823 <jackyzy823@gmail.com>
53118 • David Haguenauer <ml@kurokatta.org>
53120 • teresy <hi.teresy@gmail.com>
53122 • buergi <patbuergi@gmx.de>
53124 • Florian Gamboeck <mail@floga.de>
53126 • Ralf Hemberger <10364191+rhemberger@users.noreply.github.com>
53128 • Scott Edlund <sedlund@users.noreply.github.com>
53130 • Erik Swanson <erik@retailnext.net>
53132 • Jake Coggiano <jake@stripe.com>
53134 • brused27 <brused27@noemailaddress>
53136 • Peter Kaminski <kaminski@istori.com>
53138 • Henry Ptasinski <henry@logout.com>
53140 • Alexander <kharkovalexander@gmail.com>
53142 • Garry McNulty <garrmcnu@gmail.com>
53144 • Mathieu Carbou <mathieu.carbou@gmail.com>
53146 • Mark Otway <mark@otway.com>
53148 • William Cocker <37018962+WilliamCocker@users.noreply.github.com>
53150 • François Leurent <131.js@cloudyks.org>
53152 • Arkadius Stefanski <arkste@gmail.com>
53154 • Jay <dev@jaygoel.com>
53156 • andrea rota <a@xelera.eu>
53158 • nicolov <nicolov@users.noreply.github.com>
53160 • Dario Guzik <dario@guzik.com.ar>
53162 • qip <qip@users.noreply.github.com>
53164 • yair@unicorn <yair@unicorn>
53166 • Matt Robinson <brimstone@the.narro.ws>
53168 • kayrus <kay.diam@gmail.com>
53170 • Rémy Léone <remy.leone@gmail.com>
53172 • Wojciech Smigielski <wojciech.hieronim.smigielski@gmail.com>
53174 • weetmuts <oehrstroem@gmail.com>
53176 • Jonathan <vanillajonathan@users.noreply.github.com>
53178 • James Carpenter <orbsmiv@users.noreply.github.com>
53180 • Vince <vince0villamora@gmail.com>
53182 • Nestar47 <47841759+Nestar47@users.noreply.github.com>
53184 • Six <brbsix@gmail.com>
53186 • Alexandru Bumbacea <alexandru.bumbacea@booking.com>
53188 • calisro <robert.calistri@gmail.com>
53190 • Dr.Rx <david.rey@nventive.com>
53192 • marcintustin <marcintustin@users.noreply.github.com>
53194 • jaKa Močnik <jaka@koofr.net>
53196 • Fionera <fionera@fionera.de>
53198 • Dan Walters <dan@walters.io>
53200 • Danil Semelenov <sgtpep@users.noreply.github.com>
53202 • xopez <28950736+xopez@users.noreply.github.com>
53204 • Ben Boeckel <mathstuf@gmail.com>
53206 • Manu <manu@snapdragon.cc>
53208 • Kyle E. Mitchell <kyle@kemitchell.com>
53210 • Gary Kim <gary@garykim.dev>
53212 • Jon <jonathn@github.com>
53214 • Jeff Quinn <jeffrey.quinn@bluevoyant.com>
53216 • Peter Berbec <peter@berbec.com>
53218 • didil <1284255+didil@users.noreply.github.com>
53220 • id01 <gaviniboom@gmail.com>
53222 • Robert Marko <robimarko@gmail.com>
53224 • Philip Harvey <32467456+pharveybattelle@users.noreply.github.com>
53226 • JorisE <JorisE@users.noreply.github.com>
53228 • garry415 <garry.415@gmail.com>
53230 • forgems <forgems@gmail.com>
53232 • Florian Apolloner <florian@apolloner.eu>
53234 • Aleksandar Janković <office@ajankovic.com> <ajankovic@users.nore‐
53236 ply.github.com>
53237 • Maran <maran@protonmail.com>
53239 • nguyenhuuluan434 <nguyenhuuluan434@gmail.com>
53241 • Laura Hausmann <zotan@zotan.pw> <laura@hausmann.dev>
53243 • yparitcher <y@paritcher.com>
53245 • AbelThar <abela.tharen@gmail.com>
53247 • Matti Niemenmaa <matti.niemenmaa+git@iki.fi>
53249 • Russell Davis <russelldavis@users.noreply.github.com>
53251 • Yi FU <yi.fu@tink.se>
53253 • Paul Millar <paul.millar@desy.de>
53255 • justinalin <justinalin@qnap.com>
53257 • EliEron <subanimehd@gmail.com>
53259 • justina777 <chiahuei.lin@gmail.com>
53261 • Chaitanya Bankanhal <bchaitanya15@gmail.com>
53263 • Michał Matczuk <michal@scylladb.com>
53265 • Macavirus <macavirus@zoho.com>
53267 • Abhinav Sharma <abhi18av@outlook.com>
53269 • ginvine <34869051+ginvine@users.noreply.github.com>
53271 • Patrick Wang <mail6543210@yahoo.com.tw>
53273 • Cenk Alti <cenkalti@gmail.com>
53275 • Andreas Chlupka <andy@chlupka.com>
53277 • Alfonso Montero <amontero@tinet.org>
53279 • Ivan Andreev <ivandeex@gmail.com>
53281 • David Baumgold <david@davidbaumgold.com>
53283 • Lars Lehtonen <lars.lehtonen@gmail.com>
53285 • Matei David <matei.david@gmail.com>
53287 • David <david.bramwell@endemolshine.com>
53289 • Anthony Rusdi <33247310+antrusd@users.noreply.github.com>
53291 • Richard Patel <me@terorie.dev>
53293 • 庄天翼 <zty0826@gmail.com>
53295 • SwitchJS <dev@switchjs.com>
53297 • Raphael <PowershellNinja@users.noreply.github.com>
53299 • Sezal Agrawal <sezalagrawal@gmail.com>
53301 • Tyler <TylerNakamura@users.noreply.github.com>
53303 • Brett Dutro <brett.dutro@gmail.com>
53305 • Vighnesh SK <booterror99@gmail.com>
53307 • Arijit Biswas <dibbyo456@gmail.com>
53309 • Michele Caci <michele.caci@gmail.com>
53311 • AlexandrBoltris <ua2fgb@gmail.com>
53313 • Bryce Larson <blarson@saltstack.com>
53315 • Carlos Ferreyra <crypticmind@gmail.com>
53317 • Saksham Khanna <sakshamkhanna@outlook.com>
53319 • dausruddin <5763466+dausruddin@users.noreply.github.com>
53321 • zero-24 <zero-24@users.noreply.github.com>
53323 • Xiaoxing Ye <ye@xiaoxing.us>
53325 • Barry Muldrey <barry@muldrey.net>
53327 • Sebastian Brandt <sebastian.brandt@friday.de>
53329 • Marco Molteni <marco.molteni@mailbox.org>
53331 • Ankur Gupta <7876747+ankur0493@users.noreply.github.com>
53333 • Maciej Zimnoch <maciej@scylladb.com>
53335 • anuar45 <serdaliyev.anuar@gmail.com>
53337 • Fernando <ferferga@users.noreply.github.com>
53339 • David Cole <david.cole@sohonet.com>
53341 • Wei He <git@weispot.com>
53343 • Outvi V <19144373+outloudvi@users.noreply.github.com>
53345 • Thomas Kriechbaumer <thomas@kriechbaumer.name>
53347 • Tennix <tennix@users.noreply.github.com>
53349 • Ole Schütt <ole@schuett.name>
53351 • Kuang-che Wu <kcwu@csie.org>
53353 • Thomas Eales <wingsuit@users.noreply.github.com>
53355 • Paul Tinsley <paul.tinsley@vitalsource.com>
53357 • Felix Hungenberg <git@shiftgeist.com>
53359 • Benjamin Richter <github@dev.telepath.de>
53361 • landall <cst_zf@qq.com>
53363 • thestigma <thestigma@gmail.com>
53365 • jtagcat <38327267+jtagcat@users.noreply.github.com>
53367 • Damon Permezel <permezel@me.com>
53369 • boosh <boosh@users.noreply.github.com>
53371 • unbelauscht <58393353+unbelauscht@users.noreply.github.com>
53373 • Motonori IWAMURO <vmi@nifty.com>
53375 • Benjapol Worakan <benwrk@live.com>
53377 • Dave Koston <dave.koston@stackpath.com>
53379 • Durval Menezes <DurvalMenezes@users.noreply.github.com>
53381 • Tim Gallant <me@timgallant.us>
53383 • Frederick Zhang <frederick888@tsundere.moe>
53385 • valery1707 <valery1707@gmail.com>
53387 • Yves G <theYinYeti@yalis.fr>
53389 • Shing Kit Chan <chanshingkit@gmail.com>
53391 • Franklyn Tackitt <franklyn@tackitt.net>
53393 • Robert-André Mauchin <zebob.m@gmail.com>
53395 • evileye <48332831+ibiruai@users.noreply.github.com>
53397 • Joachim Brandon LeBlanc <brandon@leblanc.codes>
53399 • Patryk Jakuszew <patryk.jakuszew@gmail.com>
53401 • fishbullet <shindu666@gmail.com>
53403 • greatroar <@>
53405 • Bernd Schoolmann <mail@quexten.com>
53407 • Elan Ruusamäe <glen@pld-linux.org>
53409 • Max Sum <max@lolyculture.com>
53411 • Mark Spieth <mspieth@users.noreply.github.com>
53413 • harry <me@harry.plus>
53415 • Samantha McVey <samantham@posteo.net>
53417 • Jack Anderson <jack.anderson@metaswitch.com>
53419 • Michael G <draget@speciesm.net>
53421 • Brandon Philips <brandon@ifup.org>
53423 • Daven <dooven@users.noreply.github.com>
53425 • Martin Stone <martin@d7415.co.uk>
53427 • David Bramwell <13053834+dbramwell@users.noreply.github.com>
53429 • Sunil Patra <snl_su@live.com>
53431 • Adam Stroud <adam.stroud@gmail.com>
53433 • Kush <kushsharma@users.noreply.github.com>
53435 • Matan Rosenberg <matan129@gmail.com>
53437 • gitch1 <63495046+gitch1@users.noreply.github.com>
53439 • ElonH <elonhhuang@gmail.com>
53441 • Fred <fred@creativeprojects.tech>
53443 • Sébastien Gross <renard@users.noreply.github.com>
53445 • Maxime Suret <11944422+msuret@users.noreply.github.com>
53447 • Caleb Case <caleb@storj.io> <calebcase@gmail.com>
53449 • Ben Zenker <imbenzenker@gmail.com>
53451 • Martin Michlmayr <tbm@cyrius.com>
53453 • Brandon McNama <bmcnama@pagerduty.com>
53455 • Daniel Slyman <github@skylayer.eu>
53457 • Alex Guerrero <guerrero@users.noreply.github.com>
53459 • Matteo Pietro Dazzi <matteopietro.dazzi@gft.com>
53461 • edwardxml <56691903+edwardxml@users.noreply.github.com>
53463 • Roman Kredentser <shareed2k@gmail.com>
53465 • Kamil Trzciński <ayufan@ayufan.eu>
53467 • Zac Rubin <z-0@users.noreply.github.com>
53469 • Vincent Feltz <psycho@feltzv.fr>
53471 • Heiko Bornholdt <bornholdt@informatik.uni-hamburg.de>
53473 • Matteo Pietro Dazzi <matteopietro.dazzi@gmail.com>
53475 • jtagcat <gitlab@c7.ee>
53477 • Petri Salminen <petri@salminen.dev>
53479 • Tim Burke <tim.burke@gmail.com>
53481 • Kai Lüke <kai@kinvolk.io>
53483 • Garrett Squire <github@garrettsquire.com>
53485 • Evan Harris <eharris@puremagic.com>
53487 • Kevin <keyam@microsoft.com>
53489 • Morten Linderud <morten@linderud.pw>
53491 • Dmitry Ustalov <dmitry.ustalov@gmail.com>
53493 • Jack <196648+jdeng@users.noreply.github.com>
53495 • kcris <cristian.tarsoaga@gmail.com>
53497 • tyhuber1 <68970760+tyhuber1@users.noreply.github.com>
53499 • David Ibarra <david.ibarra@realty.com>
53501 • Tim Gallant <tim@lilt.com>
53503 • Kaloyan Raev <kaloyan@storj.io>
53505 • Jay McEntire <jay.mcentire@gmail.com>
53507 • Leo Luan <leoluan@us.ibm.com>
53509 • aus <549081+aus@users.noreply.github.com>
53511 • Aaron Gokaslan <agokaslan@fb.com>
53513 • Egor Margineanu <egmar@users.noreply.github.com>
53515 • Lucas Kanashiro <lucas.kanashiro@canonical.com>
53517 • WarpedPixel <WarpedPixel@users.noreply.github.com>
53519 • Sam Edwards <sam@samedwards.ca>
53521 • wjielai <gouki0123@gmail.com>
53523 • Muffin King <jinxz_k@live.com>
53525 • Christopher Stewart <6573710+1f47a@users.noreply.github.com>
53527 • Russell Cattelan <cattelan@digitalelves.com>
53529 • gyutw <30371241+gyutw@users.noreply.github.com>
53531 • Hekmon <edouardhur@gmail.com>
53533 • LaSombra <lasombra@users.noreply.github.com>
53535 • Dov Murik <dov.murik@gmail.com>
53537 • Ameer Dawood <ameer1234567890@gmail.com>
53539 • Dan Hipschman <dan.hipschman@opendoor.com>
53541 • Josh Soref <jsoref@users.noreply.github.com>
53543 • David <david@staron.nl>
53545 • Ingo <ingo@hoffmann.cx>
53547 • Adam Plánský <adamplansky@users.noreply.github.com> <adamplan‐
53549 sky@gmail.com>
53550 • Manish Gupta <manishgupta.ait@gmail.com>
53552 • Deepak Sah <sah.sslpu@gmail.com>
53554 • Marcin Zelent <marcin@zelent.net>
53556 • zhucan <zhucan.k8s@gmail.com>
53558 • James Lim <james.lim@samsara.com>
53560 • Laurens Janssen <BD69BM@insim.biz>
53562 • Bob Bagwill <bobbagwill@gmail.com>
53564 • Nathan Collins <colli372@msu.edu>
53566 • lostheli
53568 • kelv <kelvin@acks.org>
53570 • Milly <milly.ca@gmail.com>
53572 • gtorelly <gtorelly@gmail.com>
53574 • Brad Ackerman <brad@facefault.org>
53576 • Mitsuo Heijo <mitsuo.heijo@gmail.com>
53578 • Claudio Bantaloukas <rockdreamer@gmail.com>
53580 • Benjamin Gustin <gustin.ben@gmail.com>
53582 • Ingo Weiss <ingo@redhat.com>
53584 • Kerry Su <me@sshockwave.net>
53586 • Ilyess Bachiri <ilyess.bachiri@sonder.com>
53588 • Yury Stankevich <urykhy@gmail.com>
53590 • kice <wslikerqs@gmail.com>
53592 • Denis Neuling <denisneuling@gmail.com>
53594 • Janne Johansson <icepic.dz@gmail.com>
53596 • Patrik Nordlén <patriki@gmail.com>
53598 • CokeMine <aptx4561@gmail.com>
53600 • Sơn Trần-Nguyễn <github@sntran.com>
53602 • lluuaapp <266615+lluuaapp@users.noreply.github.com>
53604 • Zach Kipp <kipp.zach@gmail.com>
53606 • Riccardo Iaconelli <riccardo@kde.org>
53608 • Sakuragawa Misty <gyc990326@gmail.com>
53610 • Nicolas Rueff <nicolas@rueff.fr>
53612 • Pau Rodriguez-Estivill <prodrigestivill@gmail.com>
53614 • Bob Pusateri <BobPusateri@users.noreply.github.com>
53616 • Alex JOST <25005220+dimejo@users.noreply.github.com>
53618 • Alexey Tabakman <samosad.ru@gmail.com>
53620 • David Sze <sze.david@gmail.com>
53622 • cynthia kwok <cynthia.m.kwok@gmail.com>
53624 • Miron Veryanskiy <MironVeryanskiy@gmail.com>
53626 • K265 <k.265@qq.com>
53628 • Vesnyx <Vesnyx@users.noreply.github.com>
53630 • Dmitry Chepurovskiy <me@dm3ch.net>
53632 • Rauno Ots <rauno.ots@cgi.com>
53634 • Georg Neugschwandtner <georg.neugschwandtner@gmx.net>
53636 • pvalls <polvallsrue@gmail.com>
53638 • Robert Thomas <31854736+wolveix@users.noreply.github.com>
53640 • Romeo Kienzler <romeo.kienzler@gmail.com>
53642 • tYYGH <tYYGH@users.noreply.github.com>
53644 • georne <77802995+georne@users.noreply.github.com>
53646 • Maxwell Calman <mcalman@MacBook-Pro.local>
53648 • Naveen Honest Raj <naveendurai19@gmail.com>
53650 • Lucas Messenger <lmesseng@cisco.com>
53652 • Manish Kumar <krmanish260@gmail.com>
53654 • x0b <x0bdev@gmail.com>
53656 • CERN through the CS3MESH4EOSC Project
53658 • Nick Gaya <nicholasgaya+github@gmail.com>
53660 • Ashok Gelal <401055+ashokgelal@users.noreply.github.com>
53662 • Dominik Mydlil <dominik.mydlil@outlook.com>
53664 • Nazar Mishturak <nazarmx@gmail.com>
53666 • Ansh Mittal <iamAnshMittal@gmail.com>
53668 • noabody <noabody@yahoo.com>
53670 • OleFrost <82263101+olefrost@users.noreply.github.com>
53672 • Kenny Parsons <kennyparsons93@gmail.com>
53674 • Jeffrey Tolar <tolar.jeffrey@gmail.com>
53676 • jtagcat <git-514635f7@jtag.cat>
53678 • Tatsuya Noyori <63089076+public-tatsuya-noyori@users.nore‐
53680 ply.github.com>
53681 • lewisxy <lewisxy@users.noreply.github.com>
53683 • Nolan Woods <nolan_w@sfu.ca>
53685 • Gautam Kumar <25435568+gautamajay52@users.noreply.github.com>
53687 • Chris Macklin <chris.macklin@10xgenomics.com>
53689 • Antoon Prins <antoon.prins@surfsara.nl>
53691 • Alexey Ivanov <rbtz@dropbox.com>
53693 • Serge Pouliquen <sp31415@free.fr>
53695 • acsfer <carlos@reendex.com>
53697 • Tom <tom@tom-fitzhenry.me.uk>
53699 • Tyson Moore <tyson@tyson.me>
53701 • database64128 <free122448@hotmail.com>
53703 • Chris Lu <chrislusf@users.noreply.github.com>
53705 • Reid Buzby <reid@rethink.software>
53707 • darrenrhs <darrenrhs@gmail.com>
53709 • Florian Penzkofer <fp@nullptr.de>
53711 • Xuanchen Wu <117010292@link.cuhk.edu.cn>
53713 • partev <petrosyan@gmail.com>
53715 • Dmitry Sitnikov <fo2@inbox.ru>
53717 • Haochen Tong <i@hexchain.org>
53719 • Michael Hanselmann <public@hansmi.ch>
53721 • Chuan Zh <zhchuan7@gmail.com>
53723 • Antoine GIRARD <antoine.girard@sapk.fr>
53725 • Justin Winokur (Jwink3101) <Jwink3101@users.noreply.github.com>
53727 • Mariano Absatz (git) <scm@baby.com.ar>
53729 • Greg Sadetsky <lepetitg@gmail.com>
53731 • yedamo <logindaveye@gmail.com>
53733 • hota <lindwurm.q@gmail.com>
53735 • vinibali <vinibali1@gmail.com>
53737 • Ken Enrique Morel <ken.morel.santana@gmail.com>
53739 • Justin Hellings <justin.hellings@gmail.com>
53741 • Parth Shukla <pparth@pparth.net>
53743 • wzl <wangzl31@outlook.com>
53745 • HNGamingUK <connor@earnshawhome.co.uk>
53747 • Jonta <359397+Jonta@users.noreply.github.com>
53749 • YenForYang <YenForYang@users.noreply.github.com>
53751 • Joda Stößer <stoesser@yay-digital.de> <services+github@simjo.st>
53753 • Logeshwaran <waranlogesh@gmail.com>
53755 • Rajat Goel <rajat@dropbox.com>
53757 • r0kk3rz <r0kk3rz@gmail.com>
53759 • Matthew Sevey <mjsevey@gmail.com>
53761 • Filip Rysavy <fil@siasky.net>
53763 • Ian Levesque <ian@ianlevesque.org>
53765 • Thomas Stachl <thomas@stachl.me>
53767 • Dmitry Bogatov <git#v1@kaction.cc>
53769 • thomae <4493560+thomae@users.noreply.github.com>
53771 • trevyn <trevyn-git@protonmail.com>
53773 • David Liu <david.yx.liu@oracle.com>
53775 • Chris Nelson <stuff@cjnaz.com>
53777 • Felix Bünemann <felix.buenemann@gmail.com>
53779 • Atílio Antônio <atiliodadalto@hotmail.com>
53781 • Roberto Ricci <ricci@disroot.org>
53783 • Carlo Mion <mion00@gmail.com>
53785 • Chris Lu <chris.lu@gmail.com>
53787 • Vitor Arruda <vitor.pimenta.arruda@gmail.com>
53789 • bbabich <bbabich@datamossa.com>
53791 • David <dp.davide.palma@gmail.com>
53793 • Borna Butkovic <borna@favicode.net>
53795 • Fredric Arklid <fredric.arklid@consid.se>
53797 • Andy Jackson <Andrew.Jackson@bl.uk>
53799 • Sinan Tan <i@tinytangent.com>
53801 • deinferno <14363193+deinferno@users.noreply.github.com>
53803 • rsapkf <rsapkfff@pm.me>
53805 • Will Holtz <wholtz@gmail.com>
53807 • GGG KILLER <gggkiller2@gmail.com>
53809 • Logeshwaran Murugesan <logeshwaran@testpress.in>
53811 • Lu Wang <coolwanglu@gmail.com>
53813 • Bumsu Hyeon <ksitht@gmail.com>
53815 • Shmz Ozggrn <98463324+ShmzOzggrn@users.noreply.github.com>
53817 • Kim <kim@jotta.no>
53819 • Niels van de Weem <n.van.de.weem@smile.nl>
53821 • Koopa <codingkoopa@gmail.com>
53823 • Yunhai Luo <yunhai-luo@hotmail.com>
53825 • Charlie Jiang <w@chariri.moe>
53827 • Alain Nussbaumer <alain.nussbaumer@alleluia.ch>
53829 • Vanessasaurus <814322+vsoch@users.noreply.github.com>
53831 • Isaac Levy <isaac.r.levy@gmail.com>
53833 • Gourav T <workflowautomation@protonmail.com>
53835 • Paulo Martins <paulo.pontes.m@gmail.com>
53837 • viveknathani <viveknathani2402@gmail.com>
53839 • Eng Zer Jun <engzerjun@gmail.com>
53841 • Abhiraj <abhiraj.official15@gmail.com>
53843 • Márton Elek <elek@apache.org> <elek@users.noreply.github.com>
53845 • Vincent Murphy <vdm@vdm.ie>
53847 • ctrl-q <34975747+ctrl-q@users.noreply.github.com>
53849 • Nil Alexandrov <nalexand@akamai.com>
53851 • GuoXingbin <101376330+guoxingbin@users.noreply.github.com>
53853 • Berkan Teber <berkan@berkanteber.com>
53855 • Tobias Klauser <tklauser@distanz.ch>
53857 • KARBOWSKI Piotr <piotr.karbowski@gmail.com>
53859 • GH <geeklihui@foxmail.com>
53861 • rafma0 <int.main@gmail.com>
53863 • Adrien Rey-Jarthon <jobs@adrienjarthon.com>
53865 • Nick Gooding <73336146+nickgooding@users.noreply.github.com>
53867 • Leroy van Logchem <lr.vanlogchem@gmail.com>
53869 • Zsolt Ero <zsolt.ero@gmail.com>
53871 • Lesmiscore <nao20010128@gmail.com>
53873 • ehsantdy <ehsan.tadayon@arvancloud.com>
53875 • SwazRGB <65694696+swazrgb@users.noreply.github.com>
53877 • Mateusz Puczyński <mati6095@gmail.com>
53879 • Michael C Tiernan - MIT-Research Computing Project <mtiernan@mit.edu>
53881 • Kaspian <34658474+KaspianDev@users.noreply.github.com>
53883 • Werner <EvilOlaf@users.noreply.github.com>
53885 • Hugal31 <hugo.laloge@gmail.com>
53887 • Christian Galo <36752715+cgalo5758@users.noreply.github.com>
53889 • Erik van Velzen <erik@evanv.nl>
53891 • Derek Battams <derek@battams.ca>
53893 • SimonLiu <simonliu009@users.noreply.github.com>
53895 • Hugo Laloge <hla@lescompanions.com>
53897 • Mr-Kanister <68117355+Mr-Kanister@users.noreply.github.com>
53899 • Rob Pickerill <r.pickerill@gmail.com>
53901 • Andrey <to.merge@gmail.com>
53903 • Eric Wolf <19wolf@gmail.com>
53905 • Nick <nick.naumann@mailbox.tu-dresden.de>
53907 • Jason Zheng <jszheng17@gmail.com>
53909 • Matthew Vernon <mvernon@wikimedia.org>
53911 • Noah Hsu <i@nn.ci>
53913 • m00594701 <mengpengbo@huawei.com>
53915 • Art M. Gallagher <artmg50@gmail.com>
53917 • Sven Gerber <49589423+svengerber@users.noreply.github.com>
53919 • CrossR <r.cross@lancaster.ac.uk>
53921 • Maciej Radzikowski <maciej@radzikowski.com.pl>
53923 • Scott Grimes <scott.grimes@spaciq.com>
53925 • Phil Shackleton <71221528+philshacks@users.noreply.github.com>
53927 • eNV25 <env252525@gmail.com>
53929 • Caleb <inventor96@users.noreply.github.com>
53931 • J-P Treen <jp@wraptious.com>
53933 • Martin Czygan <53705+miku@users.noreply.github.com>
53935 • buda <sandrojijavadze@protonmail.com>
53937 • mirekphd <36706320+mirekphd@users.noreply.github.com>
53939 • vyloy <vyloy@qq.com>
53941 • Anthrazz <25553648+Anthrazz@users.noreply.github.com>
53943 • zzr93 <34027824+zzr93@users.noreply.github.com>
53945 • Paul Norman <penorman@mac.com>
53947 • Lorenzo Maiorfi <maiorfi@gmail.com>
53949 • Claudio Maradonna <penguyman@stronzi.org>
53951 • Ovidiu Victor Tatar <ovi.tatar@googlemail.com>
53953 • Evan Spensley <epspensley@gmail.com>
53955 • Yen Hu <61753151+0x59656e@users.noreply.github.com>
53957 • Steve Kowalik <steven@wedontsleep.org>
53959 • Jordi Gonzalez Muñoz <jordigonzm@gmail.com>
53961 • Joram Schrijver <i@joram.io>
53963 • Mark Trolley <marktrolley@gmail.com>
53965 • João Henrique Franco <joaohenrique.franco@gmail.com>
53967 • anonion <aman207@users.noreply.github.com>
53969 • Ryan Morey <4590343+rmorey@users.noreply.github.com>
53971 • Simon Bos <simonbos9@gmail.com>
53973 • YFdyh000 <yfdyh000@gmail.com> * Josh Soref
53975 <2119212+jsoref@users.noreply.github.com>
53976 • Øyvind Heddeland Instefjord <instefjord@outlook.com>
53978 • Dmitry Deniskin <110819396+ddeniskin@users.noreply.github.com>
53980 • Alexander Knorr <106825+opexxx@users.noreply.github.com>
53982 • Richard Bateman <richard@batemansr.us>
53984 • Dimitri Papadopoulos Orfanos <3234522+DimitriPapadopoulos@users.nore‐
53986 ply.github.com>
53987 • Lorenzo Milesi <lorenzo.milesi@yetopen.com>
53989 • Isaac Aymerich <isaac.aymerich@gmail.com>
53991 • YanceyChiew <35898533+YanceyChiew@users.noreply.github.com>
53993 • Manoj Ghosh <msays2000@gmail.com>
53995 • Bachue Zhou <bachue.shu@gmail.com>
53997 • Manoj Ghosh <manoj.ghosh@oracle.com>
53999 • Tom Mombourquette <tom@devnode.com>
54001 • Robert Newson <rnewson@apache.org>
54003
54005 Forum
54006 Forum for questions and general discussion:
54007 • https://forum.rclone.org
54009 GitHub repository
54011 The project's repository is located at:
54012 • https://github.com/rclone/rclone
54014 There you can file bug reports or contribute with pull requests.
54016 Twitter
54018 You can also follow me on twitter for rclone announcements:
54019 • [@njcw](https://twitter.com/njcw)
54021 Email
54023 Or if all else fails or you want to ask something private or confiden‐
54024 tial email Nick Craig-Wood (mailto:nick@craig-wood.com). Please don't
54025 email me requests for help - those are better directed to the forum.
54026 Thanks!
54027
54029 Nick Craig-Wood.
54030User Manual Nov 17, 2022 rclone(1)