1rclone(1) rclone(1)
2
6 Rclone is a command line program to sync files and directories to and
7 from:
8 · 1Fichier
10 · Alibaba Cloud (Aliyun) Object Storage System (OSS)
12 · Amazon Drive (See note (/amazonclouddrive/#status))
14 · Amazon S3
16 · Backblaze B2
18 · Box
20 · Ceph
22 · Citrix ShareFile
24 · C14
26 · DigitalOcean Spaces
28 · Dreamhost
30 · Dropbox
32 · FTP
34 · Google Cloud Storage
36 · Google Drive
38 · Google Photos
40 · HTTP
42 · Hubic
44 · Jottacloud
46 · IBM COS S3
48 · Koofr
50 · Mail.ru Cloud
52 · Memset Memstore
54 · Mega
56 · Memory
58 · Microsoft Azure Blob Storage
60 · Microsoft OneDrive
62 · Minio
64 · Nextcloud
66 · OVH
68 · OpenDrive
70 · Openstack Swift
72 · Oracle Cloud Storage
74 · ownCloud
76 · pCloud
78 · premiumize.me
80 · put.io
82 · QingStor
84 · Rackspace Cloud Files
86 · rsync.net
88 · Scaleway
90 · SFTP
92 · SugarSync
94 · Wasabi
96 · WebDAV
98 · Yandex Disk
100 · The local filesystem
102 Features
104 · MD5/SHA1 hashes checked at all times for file integrity
106 · Timestamps preserved on files
108 · Partial syncs supported on a whole file basis
110 · Copy (https://rclone.org/commands/rclone_copy/) mode to just copy
112 new/changed files
113 · Sync (https://rclone.org/commands/rclone_sync/) (one way) mode to
115 make a directory identical
116 · Check (https://rclone.org/commands/rclone_check/) mode to check for
118 file hash equality
119 · Can sync to and from network, eg two different cloud accounts
121 · Encryption (https://rclone.org/crypt/) backend
123 · Cache (https://rclone.org/cache/) backend
125 · Chunking (https://rclone.org/chunker/) backend
127 · Union (https://rclone.org/union/) backend
129 · Optional FUSE mount (rclone mount (https://rclone.org/com‐
131 mands/rclone_mount/))
132 · Multi-threaded downloads to local disk
134 · Can serve (https://rclone.org/commands/rclone_serve/) local or remote
136 files over HTTP (https://rclone.org/commands/rclone_serve_http/)/Web‐
137 Dav (https://rclone.org/commands/rclone_serve_webdav/)/FTP
138 (https://rclone.org/commands/rclone_serve_ftp/)/SFTP
139 (https://rclone.org/commands/rclone_serve_sftp/)/dlna
140 (https://rclone.org/commands/rclone_serve_dlna/)
141 · Experimental Web based GUI (https://rclone.org/gui/)
143 Links
145 · Home page (https://rclone.org/)
147 · GitHub project page for source and bug tracker
149 (https://github.com/rclone/rclone)
150 · Rclone Forum (https://forum.rclone.org)
152 · Downloads (https://rclone.org/downloads/)
154
156 Rclone is a Go program and comes as a single binary file.
157 Quickstart
159 · Download (https://rclone.org/downloads/) the relevant binary.
160 · Extract the rclone or rclone.exe binary from the archive
162 · Run rclone config to setup. See rclone config docs
164 (https://rclone.org/docs/) for more details.
165 See below for some expanded Linux / macOS instructions.
167 See the Usage section (https://rclone.org/docs/) of the docs for how to
169 use rclone, or run rclone -h.
170 Script installation
172 To install rclone on Linux/macOS/BSD systems, run:
173 curl https://rclone.org/install.sh | sudo bash
175 For beta installation, run:
177 curl https://rclone.org/install.sh | sudo bash -s beta
179 Note that this script checks the version of rclone installed first and
181 won’t re-download if not needed.
182 Linux installation from precompiled binary
184 Fetch and unpack
185 curl -O https://downloads.rclone.org/rclone-current-linux-amd64.zip
187 unzip rclone-current-linux-amd64.zip
188 cd rclone-*-linux-amd64
189 Copy binary file
191 sudo cp rclone /usr/bin/
193 sudo chown root:root /usr/bin/rclone
194 sudo chmod 755 /usr/bin/rclone
195 Install manpage
197 sudo mkdir -p /usr/local/share/man/man1
199 sudo cp rclone.1 /usr/local/share/man/man1/
200 sudo mandb
201 Run rclone config to setup. See rclone config docs
203 (https://rclone.org/docs/) for more details.
204 rclone config
206 macOS installation with brew
208 brew install rclone
209 macOS installation from precompiled binary, using curl
211 To avoid problems with macOS gatekeeper enforcing the binary to be
212 signed and notarized it is enough to download with curl.
213 Download the latest version of rclone.
215 cd && curl -O https://downloads.rclone.org/rclone-current-osx-amd64.zip
217 Unzip the download and cd to the extracted folder.
219 unzip -a rclone-current-osx-amd64.zip && cd rclone-*-osx-amd64
221 Move rclone to your $PATH. You will be prompted for your password.
223 sudo mkdir -p /usr/local/bin
225 sudo mv rclone /usr/local/bin/
226 (the mkdir command is safe to run, even if the directory already ex‐
228 ists).
229 Remove the leftover files.
231 cd .. && rm -rf rclone-*-osx-amd64 rclone-current-osx-amd64.zip
233 Run rclone config to setup. See rclone config docs
235 (https://rclone.org/docs/) for more details.
236 rclone config
238 macOS installation from precompiled binary, using a web browser
240 When downloading a binary with a web browser, the browser will set the
241 macOS gatekeeper quarantine attribute. Starting from Catalina, when
242 attempting to run rclone, a pop-up will appear saying:
243 “rclone” cannot be opened because the developer cannot be verified.
245 macOS cannot verify that this app is free from malware.
246 The simplest fix is to run
248 xattr -d com.apple.quarantine rclone
250 Install with docker
252 The rclone maintains a docker image for rclone (https://hub.dock‐
253 er.com/r/rclone/rclone). These images are autobuilt by docker hub from
254 the rclone source based on a minimal Alpine linux image.
255 The :latest tag will always point to the latest stable release. You
257 can use the :beta tag to get the latest build from master. You can al‐
258 so use version tags, eg :1.49.1, :1.49 or :1.
259 $ docker pull rclone/rclone:latest
261 latest: Pulling from rclone/rclone
262 Digest: sha256:0e0ced72671989bb837fea8e88578b3fc48371aa45d209663683e24cfdaa0e11
263 ...
264 $ docker run --rm rclone/rclone:latest version
265 rclone v1.49.1
266 - os/arch: linux/amd64
267 - go version: go1.12.9
268 There are a few command line options to consider when starting an
270 rclone Docker container from the rclone image.
271 · You need to mount the host rclone config dir at /config/rclone into
273 the Docker container. Due to the fact that rclone updates tokens in‐
274 side its config file, and that the update process involves a file re‐
275 name, you need to mount the whole host rclone config dir, not just
276 the single host rclone config file.
277 · You need to mount a host data dir at /data into the Docker container.
279 · By default, the rclone binary inside a Docker container runs with
281 UID=0 (root). As a result, all files created in a run will have
282 UID=0. If your config and data files reside on the host with a
283 non-root UID:GID, you need to pass these on the container start com‐
284 mand line.
285 · It is possible to use rclone mount inside a userspace Docker contain‐
287 er, and expose the resulting fuse mount to the host. The exact dock‐
288 er run options to do that might vary slightly between hosts. See,
289 e.g. the discussion in this thread (https://github.com/moby/moby/is‐
290 sues/9448).
291 You also need to mount the host /etc/passwd and /etc/group for fuse
293 to work inside the container.
294 Here are some commands tested on an Ubuntu 18.04.3 host:
296 # config on host at ~/.config/rclone/rclone.conf
298 # data on host at ~/data
299 # make sure the config is ok by listing the remotes
301 docker run --rm \
302 --volume ~/.config/rclone:/config/rclone \
303 --volume ~/data:/data:shared \
304 --user $(id -u):$(id -g) \
305 rclone/rclone \
306 listremotes
307 # perform mount inside Docker container, expose result to host
309 mkdir -p ~/data/mount
310 docker run --rm \
311 --volume ~/.config/rclone:/config/rclone \
312 --volume ~/data:/data:shared \
313 --user $(id -u):$(id -g) \
314 --volume /etc/passwd:/etc/passwd:ro --volume /etc/group:/etc/group:ro \
315 --device /dev/fuse --cap-add SYS_ADMIN --security-opt apparmor:unconfined \
316 rclone/rclone \
317 mount dropbox:Photos /data/mount &
318 ls ~/data/mount
319 kill %1
320 Install from source
322 Make sure you have at least Go (https://golang.org/) 1.7 installed.
323 Download go (https://golang.org/dl/) if necessary. The latest release
324 is recommended. Then
325 git clone https://github.com/rclone/rclone.git
327 cd rclone
328 go build
329 ./rclone version
330 You can also build and install rclone in the GOPATH
332 (https://github.com/golang/go/wiki/GOPATH) (which defaults to ~/go)
333 with:
334 go get -u -v github.com/rclone/rclone
336 and this will build the binary in $GOPATH/bin (~/go/bin/rclone by de‐
338 fault) after downloading the source to
339 $GOPATH/src/github.com/rclone/rclone (~/go/src/github.com/rclone/rclone
340 by default).
341 Installation with Ansible
343 This can be done with Stefan Weichinger’s ansible role
344 (https://github.com/stefangweichinger/ansible-rclone).
345 Instructions
347 1. git clone https://github.com/stefangweichinger/ansible-rclone.git
349 into your local roles-directory
350 2. add the role to the hosts you want rclone installed to:
352 - hosts: rclone-hosts
354 roles:
355 - rclone
356 Configure
358 First, you’ll need to configure rclone. As the object storage systems
359 have quite complicated authentication these are kept in a config file.
360 (See the --config entry for how to find the config file and choose its
361 location.)
362 The easiest way to make the config is to run rclone with the config op‐
364 tion:
365 rclone config
367 See the following for detailed instructions for
369 · 1Fichier (https://rclone.org/fichier/)
371 · Alias (https://rclone.org/alias/)
373 · Amazon Drive (https://rclone.org/amazonclouddrive/)
375 · Amazon S3 (https://rclone.org/s3/)
377 · Backblaze B2 (https://rclone.org/b2/)
379 · Box (https://rclone.org/box/)
381 · Cache (https://rclone.org/cache/)
383 · Chunker (https://rclone.org/chunker/) - transparently splits large
385 files for other remotes
386 · Citrix ShareFile (https://rclone.org/sharefile/)
388 · Crypt (https://rclone.org/crypt/) - to encrypt other remotes
390 · DigitalOcean Spaces (/s3/#digitalocean-spaces)
392 · Dropbox (https://rclone.org/dropbox/)
394 · FTP (https://rclone.org/ftp/)
396 · Google Cloud Storage (https://rclone.org/googlecloudstorage/)
398 · Google Drive (https://rclone.org/drive/)
400 · Google Photos (https://rclone.org/googlephotos/)
402 · HTTP (https://rclone.org/http/)
404 · Hubic (https://rclone.org/hubic/)
406 · Jottacloud / GetSky.no (https://rclone.org/jottacloud/)
408 · Koofr (https://rclone.org/koofr/)
410 · Mail.ru Cloud (https://rclone.org/mailru/)
412 · Mega (https://rclone.org/mega/)
414 · Memory (https://rclone.org/memory/)
416 · Microsoft Azure Blob Storage (https://rclone.org/azureblob/)
418 · Microsoft OneDrive (https://rclone.org/onedrive/)
420 · Openstack Swift / Rackspace Cloudfiles / Memset Memstore
422 (https://rclone.org/swift/)
423 · OpenDrive (https://rclone.org/opendrive/)
425 · Pcloud (https://rclone.org/pcloud/)
427 · premiumize.me (https://rclone.org/premiumizeme/)
429 · put.io (https://rclone.org/putio/)
431 · QingStor (https://rclone.org/qingstor/)
433 · SFTP (https://rclone.org/sftp/)
435 · SugarSync (https://rclone.org/sugarsync/)
437 · Union (https://rclone.org/union/)
439 · WebDAV (https://rclone.org/webdav/)
441 · Yandex Disk (https://rclone.org/yandex/)
443 · The local filesystem (https://rclone.org/local/)
445 Usage
447 Rclone syncs a directory tree from one storage system to another.
448 Its syntax is like this
450 Syntax: [options] subcommand <parameters> <parameters...>
452 Source and destination paths are specified by the name you gave the
454 storage system in the config file then the sub path, eg “drive:myfold‐
455 er” to look at “myfolder” in Google drive.
456 You can define as many storage paths as you like in the config file.
458 Subcommands
460 rclone uses a system of subcommands. For example
461 rclone ls remote:path # lists a remote
463 rclone copy /local/path remote:path # copies /local/path to the remote
464 rclone sync /local/path remote:path # syncs /local/path to the remote
465 rclone config
467 Enter an interactive configuration session.
468 Synopsis
470 Enter an interactive configuration session where you can setup new re‐
471 motes and manage existing ones. You may also set or remove a password
472 to protect your configuration.
473 rclone config [flags]
475 Options
477 -h, --help help for config
478 See the global flags page (https://rclone.org/flags/) for global op‐
480 tions not listed here.
481 SEE ALSO
483 · rclone (https://rclone.org/commands/rclone/) - Show help for rclone
484 commands, flags and backends.
485 · rclone config create (https://rclone.org/commands/rclone_config_cre‐
487 ate/) - Create a new remote with name, type and options.
488 · rclone config delete (https://rclone.org/commands/rclone_con‐
490 fig_delete/) - Delete an existing remote .
491 · rclone config disconnect (https://rclone.org/commands/rclone_con‐
493 fig_disconnect/) - Disconnects user from remote
494 · rclone config dump (https://rclone.org/commands/rclone_config_dump/)
496 - Dump the config file as JSON.
497 · rclone config edit (https://rclone.org/commands/rclone_config_edit/)
499 - Enter an interactive configuration session.
500 · rclone config file (https://rclone.org/commands/rclone_config_file/)
502 - Show path of configuration file in use.
503 · rclone config password (https://rclone.org/commands/rclone_con‐
505 fig_password/) - Update password in an existing remote.
506 · rclone config providers (https://rclone.org/commands/rclone_con‐
508 fig_providers/) - List in JSON format all the providers and options.
509 · rclone config reconnect (https://rclone.org/commands/rclone_con‐
511 fig_reconnect/) - Re-authenticates user with remote.
512 · rclone config show (https://rclone.org/commands/rclone_config_show/)
514 - Print (decrypted) config file, or the config for a single remote.
515 · rclone config update (https://rclone.org/commands/rclone_config_up‐
517 date/) - Update options in an existing remote.
518 · rclone config userinfo (https://rclone.org/commands/rclone_con‐
520 fig_userinfo/) - Prints info about logged in user of remote.
521 rclone copy
523 Copy files from source to dest, skipping already copied
524 Synopsis
526 Copy the source to the destination. Doesn’t transfer unchanged files,
527 testing by size and modification time or MD5SUM. Doesn’t delete files
528 from the destination.
529 Note that it is always the contents of the directory that is synced,
531 not the directory so when source:path is a directory, it’s the contents
532 of source:path that are copied, not the directory name and contents.
533 If dest:path doesn’t exist, it is created and the source:path contents
535 go there.
536 For example
538 rclone copy source:sourcepath dest:destpath
540 Let’s say there are two files in sourcepath
542 sourcepath/one.txt
544 sourcepath/two.txt
545 This copies them to
547 destpath/one.txt
549 destpath/two.txt
550 Not to
552 destpath/sourcepath/one.txt
554 destpath/sourcepath/two.txt
555 If you are familiar with rsync, rclone always works as if you had writ‐
557 ten a trailing / - meaning “copy the contents of this directory”. This
558 applies to all commands and whether you are talking about the source or
559 destination.
560 See the –no-traverse (/docs/#no-traverse) option for controlling
562 whether rclone lists the destination directory or not. Supplying this
563 option when copying a small number of files into a large destination
564 can speed transfers up greatly.
565 For example, if you have many files in /path/to/src but only a few of
567 them change every day, you can to copy all the files which have changed
568 recently very efficiently like this:
569 rclone copy --max-age 24h --no-traverse /path/to/src remote:
571 Note: Use the -P/--progress flag to view real-time transfer statistics
573 rclone copy source:path dest:path [flags]
575 Options
577 --create-empty-src-dirs Create empty source dirs on destination after copy
578 -h, --help help for copy
579 See the global flags page (https://rclone.org/flags/) for global op‐
581 tions not listed here.
582 SEE ALSO
584 · rclone (https://rclone.org/commands/rclone/) - Show help for rclone
585 commands, flags and backends.
586 rclone sync
588 Make source and dest identical, modifying destination only.
589 Synopsis
591 Sync the source to the destination, changing the destination only.
592 Doesn’t transfer unchanged files, testing by size and modification time
593 or MD5SUM. Destination is updated to match source, including deleting
594 files if necessary.
595 Important: Since this can cause data loss, test first with the
597 --dry-run flag to see exactly what would be copied and deleted.
598 Note that files in the destination won’t be deleted if there were any
600 errors at any point.
601 It is always the contents of the directory that is synced, not the di‐
603 rectory so when source:path is a directory, it’s the contents of
604 source:path that are copied, not the directory name and contents. See
605 extended explanation in the copy command above if unsure.
606 If dest:path doesn’t exist, it is created and the source:path contents
608 go there.
609 Note: Use the -P/--progress flag to view real-time transfer statistics
611 rclone sync source:path dest:path [flags]
613 Options
615 --create-empty-src-dirs Create empty source dirs on destination after sync
616 -h, --help help for sync
617 See the global flags page (https://rclone.org/flags/) for global op‐
619 tions not listed here.
620 SEE ALSO
622 · rclone (https://rclone.org/commands/rclone/) - Show help for rclone
623 commands, flags and backends.
624 rclone move
626 Move files from source to dest.
627 Synopsis
629 Moves the contents of the source directory to the destination directo‐
630 ry. Rclone will error if the source and destination overlap and the
631 remote does not support a server side directory move operation.
632 If no filters are in use and if possible this will server side move
634 source:path into dest:path. After this source:path will no longer
635 longer exist.
636 Otherwise for each file in source:path selected by the filters (if any)
638 this will move it into dest:path. If possible a server side move will
639 be used, otherwise it will copy it (server side if possible) into
640 dest:path then delete the original (if no errors on copy) in
641 source:path.
642 If you want to delete empty source directories after move, use the
644 –delete-empty-src-dirs flag.
645 See the –no-traverse (/docs/#no-traverse) option for controlling
647 whether rclone lists the destination directory or not. Supplying this
648 option when moving a small number of files into a large destination can
649 speed transfers up greatly.
650 Important: Since this can cause data loss, test first with the –dry-run
652 flag.
653 Note: Use the -P/--progress flag to view real-time transfer statistics.
655 rclone move source:path dest:path [flags]
657 Options
659 --create-empty-src-dirs Create empty source dirs on destination after move
660 --delete-empty-src-dirs Delete empty source dirs after move
661 -h, --help help for move
662 See the global flags page (https://rclone.org/flags/) for global op‐
664 tions not listed here.
665 SEE ALSO
667 · rclone (https://rclone.org/commands/rclone/) - Show help for rclone
668 commands, flags and backends.
669 rclone delete
671 Remove the contents of path.
672 Synopsis
674 Remove the files in path. Unlike purge it obeys include/exclude fil‐
675 ters so can be used to selectively delete files.
676 rclone delete only deletes objects but leaves the directory structure
678 alone. If you want to delete a directory and all of its contents use
679 rclone purge
680 Eg delete all files bigger than 100MBytes
682 Check what would be deleted first (use either)
684 rclone --min-size 100M lsl remote:path
686 rclone --dry-run --min-size 100M delete remote:path
687 Then delete
689 rclone --min-size 100M delete remote:path
691 That reads “delete everything with a minimum size of 100 MB”, hence
693 delete all files bigger than 100MBytes.
694 rclone delete remote:path [flags]
696 Options
698 -h, --help help for delete
699 See the global flags page (https://rclone.org/flags/) for global op‐
701 tions not listed here.
702 SEE ALSO
704 · rclone (https://rclone.org/commands/rclone/) - Show help for rclone
705 commands, flags and backends.
706 rclone purge
708 Remove the path and all of its contents.
709 Synopsis
711 Remove the path and all of its contents. Note that this does not obey
712 include/exclude filters - everything will be removed. Use delete if
713 you want to selectively delete files.
714 rclone purge remote:path [flags]
716 Options
718 -h, --help help for purge
719 See the global flags page (https://rclone.org/flags/) for global op‐
721 tions not listed here.
722 SEE ALSO
724 · rclone (https://rclone.org/commands/rclone/) - Show help for rclone
725 commands, flags and backends.
726 rclone mkdir
728 Make the path if it doesn’t already exist.
729 Synopsis
731 Make the path if it doesn’t already exist.
732 rclone mkdir remote:path [flags]
734 Options
736 -h, --help help for mkdir
737 See the global flags page (https://rclone.org/flags/) for global op‐
739 tions not listed here.
740 SEE ALSO
742 · rclone (https://rclone.org/commands/rclone/) - Show help for rclone
743 commands, flags and backends.
744 rclone rmdir
746 Remove the path if empty.
747 Synopsis
749 Remove the path. Note that you can’t remove a path with objects in it,
750 use purge for that.
751 rclone rmdir remote:path [flags]
753 Options
755 -h, --help help for rmdir
756 See the global flags page (https://rclone.org/flags/) for global op‐
758 tions not listed here.
759 SEE ALSO
761 · rclone (https://rclone.org/commands/rclone/) - Show help for rclone
762 commands, flags and backends.
763 rclone check
765 Checks the files in the source and destination match.
766 Synopsis
768 Checks the files in the source and destination match. It compares
769 sizes and hashes (MD5 or SHA1) and logs a report of files which don’t
770 match. It doesn’t alter the source or destination.
771 If you supply the –size-only flag, it will only compare the sizes not
773 the hashes as well. Use this for a quick check.
774 If you supply the –download flag, it will download the data from both
776 remotes and check them against each other on the fly. This can be use‐
777 ful for remotes that don’t support hashes or if you really want to
778 check all the data.
779 If you supply the –one-way flag, it will only check that files in
781 source match the files in destination, not the other way around. Mean‐
782 ing extra files in destination that are not in the source will not
783 trigger an error.
784 rclone check source:path dest:path [flags]
786 Options
788 --download Check by downloading rather than with hash.
789 -h, --help help for check
790 --one-way Check one way only, source files must exist on remote
791 See the global flags page (https://rclone.org/flags/) for global op‐
793 tions not listed here.
794 SEE ALSO
796 · rclone (https://rclone.org/commands/rclone/) - Show help for rclone
797 commands, flags and backends.
798 rclone ls
800 List the objects in the path with size and path.
801 Synopsis
803 Lists the objects in the source path to standard output in a human
804 readable format with size and path. Recurses by default.
805 Eg
807 $ rclone ls swift:bucket
809 60295 bevajer5jef
810 90613 canole
811 94467 diwogej7
812 37600 fubuwic
813 Any of the filtering options can be applied to this command.
815 There are several related list commands
817 · ls to list size and path of objects only
819 · lsl to list modification time, size and path of objects only
821 · lsd to list directories only
823 · lsf to list objects and directories in easy to parse format
825 · lsjson to list objects and directories in JSON format
827 ls,lsl,lsd are designed to be human readable. lsf is designed to be
829 human and machine readable. lsjson is designed to be machine readable.
830 Note that ls and lsl recurse by default - use “–max-depth 1” to stop
832 the recursion.
833 The other list commands lsd,lsf,lsjson do not recurse by default - use
835 “-R” to make them recurse.
836 Listing a non existent directory will produce an error except for re‐
838 motes which can’t have empty directories (eg s3, swift, gcs, etc - the
839 bucket based remotes).
840 rclone ls remote:path [flags]
842 Options
844 -h, --help help for ls
845 See the global flags page (https://rclone.org/flags/) for global op‐
847 tions not listed here.
848 SEE ALSO
850 · rclone (https://rclone.org/commands/rclone/) - Show help for rclone
851 commands, flags and backends.
852 rclone lsd
854 List all directories/containers/buckets in the path.
855 Synopsis
857 Lists the directories in the source path to standard output. Does not
858 recurse by default. Use the -R flag to recurse.
859 This command lists the total size of the directory (if known, -1 if
861 not), the modification time (if known, the current time if not), the
862 number of objects in the directory (if known, -1 if not) and the name
863 of the directory, Eg
864 $ rclone lsd swift:
866 494000 2018-04-26 08:43:20 10000 10000files
867 65 2018-04-26 08:43:20 1 1File
868 Or
870 $ rclone lsd drive:test
872 -1 2016-10-17 17:41:53 -1 1000files
873 -1 2017-01-03 14:40:54 -1 2500files
874 -1 2017-07-08 14:39:28 -1 4000files
875 If you just want the directory names use “rclone lsf –dirs-only”.
877 Any of the filtering options can be applied to this command.
879 There are several related list commands
881 · ls to list size and path of objects only
883 · lsl to list modification time, size and path of objects only
885 · lsd to list directories only
887 · lsf to list objects and directories in easy to parse format
889 · lsjson to list objects and directories in JSON format
891 ls,lsl,lsd are designed to be human readable. lsf is designed to be
893 human and machine readable. lsjson is designed to be machine readable.
894 Note that ls and lsl recurse by default - use “–max-depth 1” to stop
896 the recursion.
897 The other list commands lsd,lsf,lsjson do not recurse by default - use
899 “-R” to make them recurse.
900 Listing a non existent directory will produce an error except for re‐
902 motes which can’t have empty directories (eg s3, swift, gcs, etc - the
903 bucket based remotes).
904 rclone lsd remote:path [flags]
906 Options
908 -h, --help help for lsd
909 -R, --recursive Recurse into the listing.
910 See the global flags page (https://rclone.org/flags/) for global op‐
912 tions not listed here.
913 SEE ALSO
915 · rclone (https://rclone.org/commands/rclone/) - Show help for rclone
916 commands, flags and backends.
917 rclone lsl
919 List the objects in path with modification time, size and path.
920 Synopsis
922 Lists the objects in the source path to standard output in a human
923 readable format with modification time, size and path. Recurses by de‐
924 fault.
925 Eg
927 $ rclone lsl swift:bucket
929 60295 2016-06-25 18:55:41.062626927 bevajer5jef
930 90613 2016-06-25 18:55:43.302607074 canole
931 94467 2016-06-25 18:55:43.046609333 diwogej7
932 37600 2016-06-25 18:55:40.814629136 fubuwic
933 Any of the filtering options can be applied to this command.
935 There are several related list commands
937 · ls to list size and path of objects only
939 · lsl to list modification time, size and path of objects only
941 · lsd to list directories only
943 · lsf to list objects and directories in easy to parse format
945 · lsjson to list objects and directories in JSON format
947 ls,lsl,lsd are designed to be human readable. lsf is designed to be
949 human and machine readable. lsjson is designed to be machine readable.
950 Note that ls and lsl recurse by default - use “–max-depth 1” to stop
952 the recursion.
953 The other list commands lsd,lsf,lsjson do not recurse by default - use
955 “-R” to make them recurse.
956 Listing a non existent directory will produce an error except for re‐
958 motes which can’t have empty directories (eg s3, swift, gcs, etc - the
959 bucket based remotes).
960 rclone lsl remote:path [flags]
962 Options
964 -h, --help help for lsl
965 See the global flags page (https://rclone.org/flags/) for global op‐
967 tions not listed here.
968 SEE ALSO
970 · rclone (https://rclone.org/commands/rclone/) - Show help for rclone
971 commands, flags and backends.
972 rclone md5sum
974 Produces an md5sum file for all the objects in the path.
975 Synopsis
977 Produces an md5sum file for all the objects in the path. This is in
978 the same format as the standard md5sum tool produces.
979 rclone md5sum remote:path [flags]
981 Options
983 -h, --help help for md5sum
984 See the global flags page (https://rclone.org/flags/) for global op‐
986 tions not listed here.
987 SEE ALSO
989 · rclone (https://rclone.org/commands/rclone/) - Show help for rclone
990 commands, flags and backends.
991 rclone sha1sum
993 Produces an sha1sum file for all the objects in the path.
994 Synopsis
996 Produces an sha1sum file for all the objects in the path. This is in
997 the same format as the standard sha1sum tool produces.
998 rclone sha1sum remote:path [flags]
1000 Options
1002 -h, --help help for sha1sum
1003 See the global flags page (https://rclone.org/flags/) for global op‐
1005 tions not listed here.
1006 SEE ALSO
1008 · rclone (https://rclone.org/commands/rclone/) - Show help for rclone
1009 commands, flags and backends.
1010 rclone size
1012 Prints the total size and number of objects in remote:path.
1013 Synopsis
1015 Prints the total size and number of objects in remote:path.
1016 rclone size remote:path [flags]
1018 Options
1020 -h, --help help for size
1021 --json format output as JSON
1022 See the global flags page (https://rclone.org/flags/) for global op‐
1024 tions not listed here.
1025 SEE ALSO
1027 · rclone (https://rclone.org/commands/rclone/) - Show help for rclone
1028 commands, flags and backends.
1029 rclone version
1031 Show the version number.
1032 Synopsis
1034 Show the version number, the go version and the architecture.
1035 Eg
1037 $ rclone version
1039 rclone v1.41
1040 - os/arch: linux/amd64
1041 - go version: go1.10
1042 If you supply the –check flag, then it will do an online check to com‐
1044 pare your version with the latest release and the latest beta.
1045 $ rclone version --check
1047 yours: 1.42.0.6
1048 latest: 1.42 (released 2018-06-16)
1049 beta: 1.42.0.5 (released 2018-06-17)
1050 Or
1052 $ rclone version --check
1054 yours: 1.41
1055 latest: 1.42 (released 2018-06-16)
1056 upgrade: https://downloads.rclone.org/v1.42
1057 beta: 1.42.0.5 (released 2018-06-17)
1058 upgrade: https://beta.rclone.org/v1.42-005-g56e1e820
1059 rclone version [flags]
1061 Options
1063 --check Check for new version.
1064 -h, --help help for version
1065 See the global flags page (https://rclone.org/flags/) for global op‐
1067 tions not listed here.
1068 SEE ALSO
1070 · rclone (https://rclone.org/commands/rclone/) - Show help for rclone
1071 commands, flags and backends.
1072 rclone cleanup
1074 Clean up the remote if possible
1075 Synopsis
1077 Clean up the remote if possible. Empty the trash or delete old file
1078 versions. Not supported by all remotes.
1079 rclone cleanup remote:path [flags]
1081 Options
1083 -h, --help help for cleanup
1084 See the global flags page (https://rclone.org/flags/) for global op‐
1086 tions not listed here.
1087 SEE ALSO
1089 · rclone (https://rclone.org/commands/rclone/) - Show help for rclone
1090 commands, flags and backends.
1091 rclone dedupe
1093 Interactively find duplicate files and delete/rename them.
1094 Synopsis
1096 By default dedupe interactively finds duplicate files and offers to
1097 delete all but one or rename them to be different. Only useful with
1098 Google Drive which can have duplicate file names.
1099 In the first pass it will merge directories with the same name. It
1101 will do this iteratively until all the identical directories have been
1102 merged.
1103 The dedupe command will delete all but one of any identical (same
1105 md5sum) files it finds without confirmation. This means that for most
1106 duplicated files the dedupe command will not be interactive. You can
1107 use --dry-run to see what would happen without doing anything.
1108 Here is an example run.
1110 Before - with duplicates
1112 $ rclone lsl drive:dupes
1114 6048320 2016-03-05 16:23:16.798000000 one.txt
1115 6048320 2016-03-05 16:23:11.775000000 one.txt
1116 564374 2016-03-05 16:23:06.731000000 one.txt
1117 6048320 2016-03-05 16:18:26.092000000 one.txt
1118 6048320 2016-03-05 16:22:46.185000000 two.txt
1119 1744073 2016-03-05 16:22:38.104000000 two.txt
1120 564374 2016-03-05 16:22:52.118000000 two.txt
1121 Now the dedupe session
1123 $ rclone dedupe drive:dupes
1125 2016/03/05 16:24:37 Google drive root 'dupes': Looking for duplicates using interactive mode.
1126 one.txt: Found 4 duplicates - deleting identical copies
1127 one.txt: Deleting 2/3 identical duplicates (md5sum "1eedaa9fe86fd4b8632e2ac549403b36")
1128 one.txt: 2 duplicates remain
1129 1: 6048320 bytes, 2016-03-05 16:23:16.798000000, md5sum 1eedaa9fe86fd4b8632e2ac549403b36
1130 2: 564374 bytes, 2016-03-05 16:23:06.731000000, md5sum 7594e7dc9fc28f727c42ee3e0749de81
1131 s) Skip and do nothing
1132 k) Keep just one (choose which in next step)
1133 r) Rename all to be different (by changing file.jpg to file-1.jpg)
1134 s/k/r> k
1135 Enter the number of the file to keep> 1
1136 one.txt: Deleted 1 extra copies
1137 two.txt: Found 3 duplicates - deleting identical copies
1138 two.txt: 3 duplicates remain
1139 1: 564374 bytes, 2016-03-05 16:22:52.118000000, md5sum 7594e7dc9fc28f727c42ee3e0749de81
1140 2: 6048320 bytes, 2016-03-05 16:22:46.185000000, md5sum 1eedaa9fe86fd4b8632e2ac549403b36
1141 3: 1744073 bytes, 2016-03-05 16:22:38.104000000, md5sum 851957f7fb6f0bc4ce76be966d336802
1142 s) Skip and do nothing
1143 k) Keep just one (choose which in next step)
1144 r) Rename all to be different (by changing file.jpg to file-1.jpg)
1145 s/k/r> r
1146 two-1.txt: renamed from: two.txt
1147 two-2.txt: renamed from: two.txt
1148 two-3.txt: renamed from: two.txt
1149 The result being
1151 $ rclone lsl drive:dupes
1153 6048320 2016-03-05 16:23:16.798000000 one.txt
1154 564374 2016-03-05 16:22:52.118000000 two-1.txt
1155 6048320 2016-03-05 16:22:46.185000000 two-2.txt
1156 1744073 2016-03-05 16:22:38.104000000 two-3.txt
1157 Dedupe can be run non interactively using the --dedupe-mode flag or by
1159 using an extra parameter with the same value
1160 · --dedupe-mode interactive - interactive as above.
1162 · --dedupe-mode skip - removes identical files then skips anything
1164 left.
1165 · --dedupe-mode first - removes identical files then keeps the first
1167 one.
1168 · --dedupe-mode newest - removes identical files then keeps the newest
1170 one.
1171 · --dedupe-mode oldest - removes identical files then keeps the oldest
1173 one.
1174 · --dedupe-mode largest - removes identical files then keeps the
1176 largest one.
1177 · --dedupe-mode smallest - removes identical files then keeps the
1179 smallest one.
1180 · --dedupe-mode rename - removes identical files then renames the rest
1182 to be different.
1183 For example to rename all the identically named photos in your Google
1185 Photos directory, do
1186 rclone dedupe --dedupe-mode rename "drive:Google Photos"
1188 Or
1190 rclone dedupe rename "drive:Google Photos"
1192 rclone dedupe [mode] remote:path [flags]
1194 Options
1196 --dedupe-mode string Dedupe mode interactive|skip|first|newest|oldest|largest|smallest|rename. (default "interactive")
1197 -h, --help help for dedupe
1198 See the global flags page (https://rclone.org/flags/) for global op‐
1200 tions not listed here.
1201 SEE ALSO
1203 · rclone (https://rclone.org/commands/rclone/) - Show help for rclone
1204 commands, flags and backends.
1205 rclone about
1207 Get quota information from the remote.
1208 Synopsis
1210 Get quota information from the remote, like bytes used/free/quota and
1211 bytes used in the trash. Not supported by all remotes.
1212 This will print to stdout something like this:
1214 Total: 17G
1216 Used: 7.444G
1217 Free: 1.315G
1218 Trashed: 100.000M
1219 Other: 8.241G
1220 Where the fields are:
1222 · Total: total size available.
1224 · Used: total size used
1226 · Free: total amount this user could upload.
1228 · Trashed: total amount in the trash
1230 · Other: total amount in other storage (eg Gmail, Google Photos)
1232 · Objects: total number of objects in the storage
1234 Note that not all the backends provide all the fields - they will be
1236 missing if they are not known for that backend. Where it is known that
1237 the value is unlimited the value will also be omitted.
1238 Use the –full flag to see the numbers written out in full, eg
1240 Total: 18253611008
1242 Used: 7993453766
1243 Free: 1411001220
1244 Trashed: 104857602
1245 Other: 8849156022
1246 Use the –json flag for a computer readable output, eg
1248 {
1250 "total": 18253611008,
1251 "used": 7993453766,
1252 "trashed": 104857602,
1253 "other": 8849156022,
1254 "free": 1411001220
1255 }
1256 rclone about remote: [flags]
1258 Options
1260 --full Full numbers instead of SI units
1261 -h, --help help for about
1262 --json Format output as JSON
1263 See the global flags page (https://rclone.org/flags/) for global op‐
1265 tions not listed here.
1266 SEE ALSO
1268 · rclone (https://rclone.org/commands/rclone/) - Show help for rclone
1269 commands, flags and backends.
1270 rclone authorize
1272 Remote authorization.
1273 Synopsis
1275 Remote authorization. Used to authorize a remote or headless rclone
1276 from a machine with a browser - use as instructed by rclone config.
1277 Use the –auth-no-open-browser to prevent rclone to open auth link in
1279 default browser automatically.
1280 rclone authorize [flags]
1282 Options
1284 --auth-no-open-browser Do not automatically open auth link in default browser
1285 -h, --help help for authorize
1286 See the global flags page (https://rclone.org/flags/) for global op‐
1288 tions not listed here.
1289 SEE ALSO
1291 · rclone (https://rclone.org/commands/rclone/) - Show help for rclone
1292 commands, flags and backends.
1293 rclone cachestats
1295 Print cache stats for a remote
1296 Synopsis
1298 Print cache stats for a remote in JSON format
1299 rclone cachestats source: [flags]
1301 Options
1303 -h, --help help for cachestats
1304 See the global flags page (https://rclone.org/flags/) for global op‐
1306 tions not listed here.
1307 SEE ALSO
1309 · rclone (https://rclone.org/commands/rclone/) - Show help for rclone
1310 commands, flags and backends.
1311 rclone cat
1313 Concatenates any files and sends them to stdout.
1314 Synopsis
1316 rclone cat sends any files to standard output.
1317 You can use it like this to output a single file
1319 rclone cat remote:path/to/file
1321 Or like this to output any file in dir or subdirectories.
1323 rclone cat remote:path/to/dir
1325 Or like this to output any .txt files in dir or subdirectories.
1327 rclone --include "*.txt" cat remote:path/to/dir
1329 Use the –head flag to print characters only at the start, –tail for the
1331 end and –offset and –count to print a section in the middle. Note that
1332 if offset is negative it will count from the end, so –offset -1 –count
1333 1 is equivalent to –tail 1.
1334 rclone cat remote:path [flags]
1336 Options
1338 --count int Only print N characters. (default -1)
1339 --discard Discard the output instead of printing.
1340 --head int Only print the first N characters.
1341 -h, --help help for cat
1342 --offset int Start printing at offset N (or from end if -ve).
1343 --tail int Only print the last N characters.
1344 See the global flags page (https://rclone.org/flags/) for global op‐
1346 tions not listed here.
1347 SEE ALSO
1349 · rclone (https://rclone.org/commands/rclone/) - Show help for rclone
1350 commands, flags and backends.
1351 rclone config create
1353 Create a new remote with name, type and options.
1354 Synopsis
1356 Create a new remote of with and options. The options should be passed
1357 in in pairs of .
1358 For example to make a swift remote of name myremote using auto config
1360 you would do:
1361 rclone config create myremote swift env_auth true
1363 Note that if the config process would normally ask a question the de‐
1365 fault is taken. Each time that happens rclone will print a message
1366 saying how to affect the value taken.
1367 If any of the parameters passed is a password field, then rclone will
1369 automatically obscure them before putting them in the config file.
1370 So for example if you wanted to configure a Google Drive remote but us‐
1372 ing remote authorization you would do this:
1373 rclone config create mydrive drive config_is_local false
1375 rclone config create <name> <type> [<key> <value>]* [flags]
1377 Options
1379 -h, --help help for create
1380 See the global flags page (https://rclone.org/flags/) for global op‐
1382 tions not listed here.
1383 SEE ALSO
1385 · rclone config (https://rclone.org/commands/rclone_config/) - Enter an
1386 interactive configuration session.
1387 rclone config delete
1389 Delete an existing remote .
1390 Synopsis
1392 Delete an existing remote .
1393 rclone config delete <name> [flags]
1395 Options
1397 -h, --help help for delete
1398 See the global flags page (https://rclone.org/flags/) for global op‐
1400 tions not listed here.
1401 SEE ALSO
1403 · rclone config (https://rclone.org/commands/rclone_config/) - Enter an
1404 interactive configuration session.
1405 rclone config disconnect
1407 Disconnects user from remote
1408 Synopsis
1410 This disconnects the remote: passed in to the cloud storage system.
1411 This normally means revoking the oauth token.
1413 To reconnect use “rclone config reconnect”.
1415 rclone config disconnect remote: [flags]
1417 Options
1419 -h, --help help for disconnect
1420 See the global flags page (https://rclone.org/flags/) for global op‐
1422 tions not listed here.
1423 SEE ALSO
1425 · rclone config (https://rclone.org/commands/rclone_config/) - Enter an
1426 interactive configuration session.
1427 rclone config dump
1429 Dump the config file as JSON.
1430 Synopsis
1432 Dump the config file as JSON.
1433 rclone config dump [flags]
1435 Options
1437 -h, --help help for dump
1438 See the global flags page (https://rclone.org/flags/) for global op‐
1440 tions not listed here.
1441 SEE ALSO
1443 · rclone config (https://rclone.org/commands/rclone_config/) - Enter an
1444 interactive configuration session.
1445 rclone config edit
1447 Enter an interactive configuration session.
1448 Synopsis
1450 Enter an interactive configuration session where you can setup new re‐
1451 motes and manage existing ones. You may also set or remove a password
1452 to protect your configuration.
1453 rclone config edit [flags]
1455 Options
1457 -h, --help help for edit
1458 See the global flags page (https://rclone.org/flags/) for global op‐
1460 tions not listed here.
1461 SEE ALSO
1463 · rclone config (https://rclone.org/commands/rclone_config/) - Enter an
1464 interactive configuration session.
1465 rclone config file
1467 Show path of configuration file in use.
1468 Synopsis
1470 Show path of configuration file in use.
1471 rclone config file [flags]
1473 Options
1475 -h, --help help for file
1476 See the global flags page (https://rclone.org/flags/) for global op‐
1478 tions not listed here.
1479 SEE ALSO
1481 · rclone config (https://rclone.org/commands/rclone_config/) - Enter an
1482 interactive configuration session.
1483 rclone config password
1485 Update password in an existing remote.
1486 Synopsis
1488 Update an existing remote’s password. The password should be passed in
1489 in pairs of .
1490 For example to set password of a remote of name myremote you would do:
1492 rclone config password myremote fieldname mypassword
1494 This command is obsolete now that “config update” and “config create”
1496 both support obscuring passwords directly.
1497 rclone config password <name> [<key> <value>]+ [flags]
1499 Options
1501 -h, --help help for password
1502 See the global flags page (https://rclone.org/flags/) for global op‐
1504 tions not listed here.
1505 SEE ALSO
1507 · rclone config (https://rclone.org/commands/rclone_config/) - Enter an
1508 interactive configuration session.
1509 rclone config providers
1511 List in JSON format all the providers and options.
1512 Synopsis
1514 List in JSON format all the providers and options.
1515 rclone config providers [flags]
1517 Options
1519 -h, --help help for providers
1520 See the global flags page (https://rclone.org/flags/) for global op‐
1522 tions not listed here.
1523 SEE ALSO
1525 · rclone config (https://rclone.org/commands/rclone_config/) - Enter an
1526 interactive configuration session.
1527 rclone config reconnect
1529 Re-authenticates user with remote.
1530 Synopsis
1532 This reconnects remote: passed in to the cloud storage system.
1533 To disconnect the remote use “rclone config disconnect”.
1535 This normally means going through the interactive oauth flow again.
1537 rclone config reconnect remote: [flags]
1539 Options
1541 -h, --help help for reconnect
1542 See the global flags page (https://rclone.org/flags/) for global op‐
1544 tions not listed here.
1545 SEE ALSO
1547 · rclone config (https://rclone.org/commands/rclone_config/) - Enter an
1548 interactive configuration session.
1549 rclone config show
1551 Print (decrypted) config file, or the config for a single remote.
1552 Synopsis
1554 Print (decrypted) config file, or the config for a single remote.
1555 rclone config show [<remote>] [flags]
1557 Options
1559 -h, --help help for show
1560 See the global flags page (https://rclone.org/flags/) for global op‐
1562 tions not listed here.
1563 SEE ALSO
1565 · rclone config (https://rclone.org/commands/rclone_config/) - Enter an
1566 interactive configuration session.
1567 rclone config update
1569 Update options in an existing remote.
1570 Synopsis
1572 Update an existing remote’s options. The options should be passed in
1573 in pairs of .
1574 For example to update the env_auth field of a remote of name myremote
1576 you would do:
1577 rclone config update myremote swift env_auth true
1579 If any of the parameters passed is a password field, then rclone will
1581 automatically obscure them before putting them in the config file.
1582 If the remote uses oauth the token will be updated, if you don’t re‐
1584 quire this add an extra parameter thus:
1585 rclone config update myremote swift env_auth true config_refresh_token false
1587 rclone config update <name> [<key> <value>]+ [flags]
1589 Options
1591 -h, --help help for update
1592 See the global flags page (https://rclone.org/flags/) for global op‐
1594 tions not listed here.
1595 SEE ALSO
1597 · rclone config (https://rclone.org/commands/rclone_config/) - Enter an
1598 interactive configuration session.
1599 rclone config userinfo
1601 Prints info about logged in user of remote.
1602 Synopsis
1604 This prints the details of the person logged in to the cloud storage
1605 system.
1606 rclone config userinfo remote: [flags]
1608 Options
1610 -h, --help help for userinfo
1611 --json Format output as JSON
1612 See the global flags page (https://rclone.org/flags/) for global op‐
1614 tions not listed here.
1615 SEE ALSO
1617 · rclone config (https://rclone.org/commands/rclone_config/) - Enter an
1618 interactive configuration session.
1619 rclone copyto
1621 Copy files from source to dest, skipping already copied
1622 Synopsis
1624 If source:path is a file or directory then it copies it to a file or
1625 directory named dest:path.
1626 This can be used to upload single files to other than their current
1628 name. If the source is a directory then it acts exactly like the copy
1629 command.
1630 So
1632 rclone copyto src dst
1634 where src and dst are rclone paths, either remote:path or /path/to/lo‐
1636 cal or C:.
1637 This will:
1639 if src is file
1641 copy it to dst, overwriting an existing file if it exists
1642 if src is directory
1643 copy it to dst, overwriting existing files if they exist
1644 see copy command for full details
1645 This doesn’t transfer unchanged files, testing by size and modification
1647 time or MD5SUM. It doesn’t delete files from the destination.
1648 Note: Use the -P/--progress flag to view real-time transfer statistics
1650 rclone copyto source:path dest:path [flags]
1652 Options
1654 -h, --help help for copyto
1655 See the global flags page (https://rclone.org/flags/) for global op‐
1657 tions not listed here.
1658 SEE ALSO
1660 · rclone (https://rclone.org/commands/rclone/) - Show help for rclone
1661 commands, flags and backends.
1662 rclone copyurl
1664 Copy url content to dest.
1665 Synopsis
1667 Download a URL’s content and copy it to the destination without saving
1668 it in temporary storage.
1669 Setting –auto-filename will cause the file name to be retreived from
1671 the from URL (after any redirections) and used in the destination path.
1672 Setting –stdout or making the output file name “-” will cause the out‐
1674 put to be written to standard output.
1675 rclone copyurl https://example.com dest:path [flags]
1677 Options
1679 -a, --auto-filename Get the file name from the URL and use it for destination file path
1680 -h, --help help for copyurl
1681 --stdout Write the output to stdout rather than a file
1682 See the global flags page (https://rclone.org/flags/) for global op‐
1684 tions not listed here.
1685 SEE ALSO
1687 · rclone (https://rclone.org/commands/rclone/) - Show help for rclone
1688 commands, flags and backends.
1689 rclone cryptcheck
1691 Cryptcheck checks the integrity of a crypted remote.
1692 Synopsis
1694 rclone cryptcheck checks a remote against a crypted remote. This is
1695 the equivalent of running rclone check, but able to check the checksums
1696 of the crypted remote.
1697 For it to work the underlying remote of the cryptedremote must support
1699 some kind of checksum.
1700 It works by reading the nonce from each file on the cryptedremote: and
1702 using that to encrypt each file on the remote:. It then checks the
1703 checksum of the underlying file on the cryptedremote: against the
1704 checksum of the file it has just encrypted.
1705 Use it like this
1707 rclone cryptcheck /path/to/files encryptedremote:path
1709 You can use it like this also, but that will involve downloading all
1711 the files in remote:path.
1712 rclone cryptcheck remote:path encryptedremote:path
1714 After it has run it will log the status of the encryptedremote:.
1716 If you supply the –one-way flag, it will only check that files in
1718 source match the files in destination, not the other way around. Mean‐
1719 ing extra files in destination that are not in the source will not
1720 trigger an error.
1721 rclone cryptcheck remote:path cryptedremote:path [flags]
1723 Options
1725 -h, --help help for cryptcheck
1726 --one-way Check one way only, source files must exist on destination
1727 See the global flags page (https://rclone.org/flags/) for global op‐
1729 tions not listed here.
1730 SEE ALSO
1732 · rclone (https://rclone.org/commands/rclone/) - Show help for rclone
1733 commands, flags and backends.
1734 rclone cryptdecode
1736 Cryptdecode returns unencrypted file names.
1737 Synopsis
1739 rclone cryptdecode returns unencrypted file names when provided with a
1740 list of encrypted file names. List limit is 10 items.
1741 If you supply the –reverse flag, it will return encrypted file names.
1743 use it like this
1745 rclone cryptdecode encryptedremote: encryptedfilename1 encryptedfilename2
1747 rclone cryptdecode --reverse encryptedremote: filename1 filename2
1749 rclone cryptdecode encryptedremote: encryptedfilename [flags]
1751 Options
1753 -h, --help help for cryptdecode
1754 --reverse Reverse cryptdecode, encrypts filenames
1755 See the global flags page (https://rclone.org/flags/) for global op‐
1757 tions not listed here.
1758 SEE ALSO
1760 · rclone (https://rclone.org/commands/rclone/) - Show help for rclone
1761 commands, flags and backends.
1762 rclone dbhashsum
1764 Produces a Dropbox hash file for all the objects in the path.
1765 Synopsis
1767 Produces a Dropbox hash file for all the objects in the path. The
1768 hashes are calculated according to Dropbox content hash rules
1769 (https://www.dropbox.com/developers/reference/content-hash). The out‐
1770 put is in the same format as md5sum and sha1sum.
1771 rclone dbhashsum remote:path [flags]
1773 Options
1775 -h, --help help for dbhashsum
1776 See the global flags page (https://rclone.org/flags/) for global op‐
1778 tions not listed here.
1779 SEE ALSO
1781 · rclone (https://rclone.org/commands/rclone/) - Show help for rclone
1782 commands, flags and backends.
1783 rclone deletefile
1785 Remove a single file from remote.
1786 Synopsis
1788 Remove a single file from remote. Unlike delete it cannot be used to
1789 remove a directory and it doesn’t obey include/exclude filters - if the
1790 specified file exists, it will always be removed.
1791 rclone deletefile remote:path [flags]
1793 Options
1795 -h, --help help for deletefile
1796 See the global flags page (https://rclone.org/flags/) for global op‐
1798 tions not listed here.
1799 SEE ALSO
1801 · rclone (https://rclone.org/commands/rclone/) - Show help for rclone
1802 commands, flags and backends.
1803 rclone genautocomplete
1805 Output completion script for a given shell.
1806 Synopsis
1808 Generates a shell completion script for rclone. Run with –help to list
1809 the supported shells.
1810 Options
1812 -h, --help help for genautocomplete
1813 See the global flags page (https://rclone.org/flags/) for global op‐
1815 tions not listed here.
1816 SEE ALSO
1818 · rclone (https://rclone.org/commands/rclone/) - Show help for rclone
1819 commands, flags and backends.
1820 · rclone genautocomplete bash (https://rclone.org/commands/rclone_ge‐
1822 nautocomplete_bash/) - Output bash completion script for rclone.
1823 · rclone genautocomplete zsh (https://rclone.org/commands/rclone_genau‐
1825 tocomplete_zsh/) - Output zsh completion script for rclone.
1826 rclone genautocomplete bash
1828 Output bash completion script for rclone.
1829 Synopsis
1831 Generates a bash shell autocompletion script for rclone.
1832 This writes to /etc/bash_completion.d/rclone by default so will proba‐
1834 bly need to be run with sudo or as root, eg
1835 sudo rclone genautocomplete bash
1837 Logout and login again to use the autocompletion scripts, or source
1839 them directly
1840 . /etc/bash_completion
1842 If you supply a command line argument the script will be written there.
1844 rclone genautocomplete bash [output_file] [flags]
1846 Options
1848 -h, --help help for bash
1849 See the global flags page (https://rclone.org/flags/) for global op‐
1851 tions not listed here.
1852 SEE ALSO
1854 · rclone genautocomplete (https://rclone.org/commands/rclone_genauto‐
1855 complete/) - Output completion script for a given shell.
1856 rclone genautocomplete zsh
1858 Output zsh completion script for rclone.
1859 Synopsis
1861 Generates a zsh autocompletion script for rclone.
1862 This writes to /usr/share/zsh/vendor-completions/_rclone by default so
1864 will probably need to be run with sudo or as root, eg
1865 sudo rclone genautocomplete zsh
1867 Logout and login again to use the autocompletion scripts, or source
1869 them directly
1870 autoload -U compinit && compinit
1872 If you supply a command line argument the script will be written there.
1874 rclone genautocomplete zsh [output_file] [flags]
1876 Options
1878 -h, --help help for zsh
1879 See the global flags page (https://rclone.org/flags/) for global op‐
1881 tions not listed here.
1882 SEE ALSO
1884 · rclone genautocomplete (https://rclone.org/commands/rclone_genauto‐
1885 complete/) - Output completion script for a given shell.
1886 rclone gendocs
1888 Output markdown docs for rclone to the directory supplied.
1889 Synopsis
1891 This produces markdown docs for the rclone commands to the directory
1892 supplied. These are in a format suitable for hugo to render into the
1893 rclone.org website.
1894 rclone gendocs output_directory [flags]
1896 Options
1898 -h, --help help for gendocs
1899 See the global flags page (https://rclone.org/flags/) for global op‐
1901 tions not listed here.
1902 SEE ALSO
1904 · rclone (https://rclone.org/commands/rclone/) - Show help for rclone
1905 commands, flags and backends.
1906 rclone hashsum
1908 Produces an hashsum file for all the objects in the path.
1909 Synopsis
1911 Produces a hash file for all the objects in the path using the hash
1912 named. The output is in the same format as the standard md5sum/sha1sum
1913 tool.
1914 Run without a hash to see the list of supported hashes, eg
1916 $ rclone hashsum
1918 Supported hashes are:
1919 * MD5
1920 * SHA-1
1921 * DropboxHash
1922 * QuickXorHash
1923 Then
1925 $ rclone hashsum MD5 remote:path
1927 rclone hashsum <hash> remote:path [flags]
1929 Options
1931 --base64 Output base64 encoded hashsum
1932 -h, --help help for hashsum
1933 See the global flags page (https://rclone.org/flags/) for global op‐
1935 tions not listed here.
1936 SEE ALSO
1938 · rclone (https://rclone.org/commands/rclone/) - Show help for rclone
1939 commands, flags and backends.
1940 rclone link
1942 Generate public link to file/folder.
1943 Synopsis
1945 rclone link will create or retrieve a public link to the given file or
1946 folder.
1947 rclone link remote:path/to/file
1949 rclone link remote:path/to/folder/
1950 If successful, the last line of the output will contain the link. Ex‐
1952 act capabilities depend on the remote, but the link will always be cre‐
1953 ated with the least constraints – e.g. no expiry, no password protec‐
1954 tion, accessible without account.
1955 rclone link remote:path [flags]
1957 Options
1959 -h, --help help for link
1960 See the global flags page (https://rclone.org/flags/) for global op‐
1962 tions not listed here.
1963 SEE ALSO
1965 · rclone (https://rclone.org/commands/rclone/) - Show help for rclone
1966 commands, flags and backends.
1967 rclone listremotes
1969 List all the remotes in the config file.
1970 Synopsis
1972 rclone listremotes lists all the available remotes from the config
1973 file.
1974 When uses with the -l flag it lists the types too.
1976 rclone listremotes [flags]
1978 Options
1980 -h, --help help for listremotes
1981 --long Show the type as well as names.
1982 See the global flags page (https://rclone.org/flags/) for global op‐
1984 tions not listed here.
1985 SEE ALSO
1987 · rclone (https://rclone.org/commands/rclone/) - Show help for rclone
1988 commands, flags and backends.
1989 rclone lsf
1991 List directories and objects in remote:path formatted for parsing
1992 Synopsis
1994 List the contents of the source path (directories and objects) to stan‐
1995 dard output in a form which is easy to parse by scripts. By default
1996 this will just be the names of the objects and directories, one per
1997 line. The directories will have a / suffix.
1998 Eg
2000 $ rclone lsf swift:bucket
2002 bevajer5jef
2003 canole
2004 diwogej7
2005 ferejej3gux/
2006 fubuwic
2007 Use the –format option to control what gets listed. By default this is
2009 just the path, but you can use these parameters to control the output:
2010 p - path
2012 s - size
2013 t - modification time
2014 h - hash
2015 i - ID of object
2016 o - Original ID of underlying object
2017 m - MimeType of object if known
2018 e - encrypted name
2019 T - tier of storage if known, eg "Hot" or "Cool"
2020 So if you wanted the path, size and modification time, you would use
2022 –format “pst”, or maybe –format “tsp” to put the path last.
2023 Eg
2025 $ rclone lsf --format "tsp" swift:bucket
2027 2016-06-25 18:55:41;60295;bevajer5jef
2028 2016-06-25 18:55:43;90613;canole
2029 2016-06-25 18:55:43;94467;diwogej7
2030 2018-04-26 08:50:45;0;ferejej3gux/
2031 2016-06-25 18:55:40;37600;fubuwic
2032 If you specify “h” in the format you will get the MD5 hash by default,
2034 use the “–hash” flag to change which hash you want. Note that this can
2035 be returned as an empty string if it isn’t available on the object (and
2036 for directories), “ERROR” if there was an error reading it from the ob‐
2037 ject and “UNSUPPORTED” if that object does not support that hash type.
2038 For example to emulate the md5sum command you can use
2040 rclone lsf -R --hash MD5 --format hp --separator " " --files-only .
2042 Eg
2044 $ rclone lsf -R --hash MD5 --format hp --separator " " --files-only swift:bucket
2046 7908e352297f0f530b84a756f188baa3 bevajer5jef
2047 cd65ac234e6fea5925974a51cdd865cc canole
2048 03b5341b4f234b9d984d03ad076bae91 diwogej7
2049 8fd37c3810dd660778137ac3a66cc06d fubuwic
2050 99713e14a4c4ff553acaf1930fad985b gixacuh7ku
2051 (Though “rclone md5sum .” is an easier way of typing this.)
2053 By default the separator is “;” this can be changed with the –separator
2055 flag. Note that separators aren’t escaped in the path so putting it
2056 last is a good strategy.
2057 Eg
2059 $ rclone lsf --separator "," --format "tshp" swift:bucket
2061 2016-06-25 18:55:41,60295,7908e352297f0f530b84a756f188baa3,bevajer5jef
2062 2016-06-25 18:55:43,90613,cd65ac234e6fea5925974a51cdd865cc,canole
2063 2016-06-25 18:55:43,94467,03b5341b4f234b9d984d03ad076bae91,diwogej7
2064 2018-04-26 08:52:53,0,,ferejej3gux/
2065 2016-06-25 18:55:40,37600,8fd37c3810dd660778137ac3a66cc06d,fubuwic
2066 You can output in CSV standard format. This will escape things in " if
2068 they contain ,
2069 Eg
2071 $ rclone lsf --csv --files-only --format ps remote:path
2073 test.log,22355
2074 test.sh,449
2075 "this file contains a comma, in the file name.txt",6
2076 Note that the –absolute parameter is useful for making lists of files
2078 to pass to an rclone copy with the –files-from flag.
2079 For example to find all the files modified within one day and copy
2081 those only (without traversing the whole directory structure):
2082 rclone lsf --absolute --files-only --max-age 1d /path/to/local > new_files
2084 rclone copy --files-from new_files /path/to/local remote:path
2085 Any of the filtering options can be applied to this command.
2087 There are several related list commands
2089 · ls to list size and path of objects only
2091 · lsl to list modification time, size and path of objects only
2093 · lsd to list directories only
2095 · lsf to list objects and directories in easy to parse format
2097 · lsjson to list objects and directories in JSON format
2099 ls,lsl,lsd are designed to be human readable. lsf is designed to be
2101 human and machine readable. lsjson is designed to be machine readable.
2102 Note that ls and lsl recurse by default - use “–max-depth 1” to stop
2104 the recursion.
2105 The other list commands lsd,lsf,lsjson do not recurse by default - use
2107 “-R” to make them recurse.
2108 Listing a non existent directory will produce an error except for re‐
2110 motes which can’t have empty directories (eg s3, swift, gcs, etc - the
2111 bucket based remotes).
2112 rclone lsf remote:path [flags]
2114 Options
2116 --absolute Put a leading / in front of path names.
2117 --csv Output in CSV format.
2118 -d, --dir-slash Append a slash to directory names. (default true)
2119 --dirs-only Only list directories.
2120 --files-only Only list files.
2121 -F, --format string Output format - see help for details (default "p")
2122 --hash h Use this hash when h is used in the format MD5|SHA-1|DropboxHash (default "MD5")
2123 -h, --help help for lsf
2124 -R, --recursive Recurse into the listing.
2125 -s, --separator string Separator for the items in the format. (default ";")
2126 See the global flags page (https://rclone.org/flags/) for global op‐
2128 tions not listed here.
2129 SEE ALSO
2131 · rclone (https://rclone.org/commands/rclone/) - Show help for rclone
2132 commands, flags and backends.
2133 rclone lsjson
2135 List directories and objects in the path in JSON format.
2136 Synopsis
2138 List directories and objects in the path in JSON format.
2139 The output is an array of Items, where each Item looks like this
2141 { “Hashes” : { “SHA-1” : “f572d396fae9206628714fb2ce00f72e94f2258f”,
2143 “MD5” : “b1946ac92492d2347c6235b4d2611184”, “DropboxHash” :
2144 “ecb65bb98f9d905b70458986c39fcbad7715e5f2fcc3b1f07767d7c83e2438cc” },
2145 “ID”: “y2djkhiujf83u33”, “OrigID”: “UYOJVTUW00Q1RzTDA”, “IsBucket” :
2146 false, “IsDir” : false, “MimeType” : “application/octet-stream”, “Mod‐
2147 Time” : “2017-05-31T16:15:57.034468261+01:00”, “Name” : “file.txt”,
2148 “Encrypted” : “v0qpsdq8anpci8n929v3uu9338”, “EncryptedPath” :
2149 “kja9098349023498/v0qpsdq8anpci8n929v3uu9338”, “Path” :
2150 “full/path/goes/here/file.txt”, “Size” : 6, “Tier” : “hot”, }
2151 If –hash is not specified the Hashes property won’t be emitted.
2153 If –no-modtime is specified then ModTime will be blank. This can speed
2155 things up on remotes where reading the ModTime takes an extra request
2156 (eg s3, swift).
2157 If –no-mimetype is specified then MimeType will be blank. This can
2159 speed things up on remotes where reading the MimeType takes an extra
2160 request (eg s3, swift).
2161 If –encrypted is not specified the Encrypted won’t be emitted.
2163 If –dirs-only is not specified files in addition to directories are re‐
2165 turned
2166 If –files-only is not specified directories in addition to the files
2168 will be returned.
2169 The Path field will only show folders below the remote path being list‐
2171 ed. If “remote:path” contains the file “subfolder/file.txt”, the Path
2172 for “file.txt” will be “subfolder/file.txt”, not “remote:path/subfold‐
2173 er/file.txt”. When used without –recursive the Path will always be the
2174 same as Name.
2175 If the directory is a bucket in a bucket based backend, then “IsBucket”
2177 will be set to true. This key won’t be present unless it is “true”.
2178 The time is in RFC3339 format with up to nanosecond precision. The
2180 number of decimal digits in the seconds will depend on the precision
2181 that the remote can hold the times, so if times are accurate to the
2182 nearest millisecond (eg Google Drive) then 3 digits will always be
2183 shown (“2017-05-31T16:15:57.034+01:00”) whereas if the times are accu‐
2184 rate to the nearest second (Dropbox, Box, WebDav etc) no digits will be
2185 shown (“2017-05-31T16:15:57+01:00”).
2186 The whole output can be processed as a JSON blob, or alternatively it
2188 can be processed line by line as each item is written one to a line.
2189 Any of the filtering options can be applied to this command.
2191 There are several related list commands
2193 · ls to list size and path of objects only
2195 · lsl to list modification time, size and path of objects only
2197 · lsd to list directories only
2199 · lsf to list objects and directories in easy to parse format
2201 · lsjson to list objects and directories in JSON format
2203 ls,lsl,lsd are designed to be human readable. lsf is designed to be
2205 human and machine readable. lsjson is designed to be machine readable.
2206 Note that ls and lsl recurse by default - use “–max-depth 1” to stop
2208 the recursion.
2209 The other list commands lsd,lsf,lsjson do not recurse by default - use
2211 “-R” to make them recurse.
2212 Listing a non existent directory will produce an error except for re‐
2214 motes which can’t have empty directories (eg s3, swift, gcs, etc - the
2215 bucket based remotes).
2216 rclone lsjson remote:path [flags]
2218 Options
2220 --dirs-only Show only directories in the listing.
2221 -M, --encrypted Show the encrypted names.
2222 --files-only Show only files in the listing.
2223 --hash Include hashes in the output (may take longer).
2224 -h, --help help for lsjson
2225 --no-mimetype Don't read the mime type (can speed things up).
2226 --no-modtime Don't read the modification time (can speed things up).
2227 --original Show the ID of the underlying Object.
2228 -R, --recursive Recurse into the listing.
2229 See the global flags page (https://rclone.org/flags/) for global op‐
2231 tions not listed here.
2232 SEE ALSO
2234 · rclone (https://rclone.org/commands/rclone/) - Show help for rclone
2235 commands, flags and backends.
2236 rclone mount
2238 Mount the remote as file system on a mountpoint.
2239 Synopsis
2241 rclone mount allows Linux, FreeBSD, macOS and Windows to mount any of
2242 Rclone’s cloud storage systems as a file system with FUSE.
2243 First set up your remote using rclone config. Check it works with
2245 rclone ls etc.
2246 Start the mount like this
2248 rclone mount remote:path/to/files /path/to/local/mount
2250 Or on Windows like this where X: is an unused drive letter
2252 rclone mount remote:path/to/files X:
2254 When the program ends, either via Ctrl+C or receiving a SIGINT or
2256 SIGTERM signal, the mount is automatically stopped.
2257 The umount operation can fail, for example when the mountpoint is busy.
2259 When that happens, it is the user’s responsibility to stop the mount
2260 manually with
2261 # Linux
2263 fusermount -u /path/to/local/mount
2264 # OS X
2265 umount /path/to/local/mount
2266 Installing on Windows
2268 To run rclone mount on Windows, you will need to download and install
2269 WinFsp (http://www.secfs.net/winfsp/).
2270 WinFsp is an open source (https://github.com/billziss-gh/winfsp) Win‐
2272 dows File System Proxy which makes it easy to write user space file
2273 systems for Windows. It provides a FUSE emulation layer which rclone
2274 uses combination with cgofuse (https://github.com/billziss-gh/cgofuse).
2275 Both of these packages are by Bill Zissimopoulos who was very helpful
2276 during the implementation of rclone mount for Windows.
2277 Windows caveats
2279 Note that drives created as Administrator are not visible by other ac‐
2280 counts (including the account that was elevated as Administrator). So
2281 if you start a Windows drive from an Administrative Command Prompt and
2282 then try to access the same drive from Explorer (which does not run as
2283 Administrator), you will not be able to see the new drive.
2284 The easiest way around this is to start the drive from a normal command
2286 prompt. It is also possible to start a drive from the SYSTEM account
2287 (using the WinFsp.Launcher infrastructure (https://github.com/billziss-
2288 gh/winfsp/wiki/WinFsp-Service-Architecture)) which creates drives ac‐
2289 cessible for everyone on the system or alternatively using the nssm
2290 service manager (https://nssm.cc/usage).
2291 Limitations
2293 Without the use of “–vfs-cache-mode” this can only write files sequen‐
2294 tially, it can only seek when reading. This means that many applica‐
2295 tions won’t work with their files on an rclone mount without
2296 “–vfs-cache-mode writes” or “–vfs-cache-mode full”. See the File
2297 Caching section for more info.
2298 The bucket based remotes (eg Swift, S3, Google Compute Storage, B2, Hu‐
2300 bic) do not support the concept of empty directories, so empty directo‐
2301 ries will have a tendency to disappear once they fall out of the direc‐
2302 tory cache.
2303 Only supported on Linux, FreeBSD, OS X and Windows at the moment.
2305 rclone mount vs rclone sync/copy
2307 File systems expect things to be 100% reliable, whereas cloud storage
2308 systems are a long way from 100% reliable. The rclone sync/copy com‐
2309 mands cope with this with lots of retries. However rclone mount can’t
2310 use retries in the same way without making local copies of the uploads.
2311 Look at the file caching for solutions to make mount more reliable.
2312 Attribute caching
2314 You can use the flag –attr-timeout to set the time the kernel caches
2315 the attributes (size, modification time etc) for directory entries.
2316 The default is “1s” which caches files just long enough to avoid too
2318 many callbacks to rclone from the kernel.
2319 In theory 0s should be the correct value for filesystems which can
2321 change outside the control of the kernel. However this causes quite a
2322 few problems such as rclone using too much memory
2323 (https://github.com/rclone/rclone/issues/2157), rclone not serving
2324 files to samba (https://forum.rclone.org/t/rclone-1-39-vs-1-40-mount-
2325 issue/5112) and excessive time listing directories
2326 (https://github.com/rclone/rclone/issues/2095#issuecomment-371141147).
2327 The kernel can cache the info about a file for the time given by “–at‐
2329 tr-timeout”. You may see corruption if the remote file changes length
2330 during this window. It will show up as either a truncated file or a
2331 file with garbage on the end. With “–attr-timeout 1s” this is very un‐
2332 likely but not impossible. The higher you set “–attr-timeout” the more
2333 likely it is. The default setting of “1s” is the lowest setting which
2334 mitigates the problems above.
2335 If you set it higher (`10s' or `1m' say) then the kernel will call back
2337 to rclone less often making it more efficient, however there is more
2338 chance of the corruption issue above.
2339 If files don’t change on the remote outside of the control of rclone
2341 then there is no chance of corruption.
2342 This is the same as setting the attr_timeout option in mount.fuse.
2344 Filters
2346 Note that all the rclone filters can be used to select a subset of the
2347 files to be visible in the mount.
2348 systemd
2350 When running rclone mount as a systemd service, it is possible to use
2351 Type=notify. In this case the service will enter the started state af‐
2352 ter the mountpoint has been successfully set up. Units having the
2353 rclone mount service specified as a requirement will see all files and
2354 folders immediately in this mode.
2355 chunked reading
2357 –vfs-read-chunk-size will enable reading the source objects in parts.
2358 This can reduce the used download quota for some remotes by requesting
2359 only chunks from the remote that are actually read at the cost of an
2360 increased number of requests.
2361 When –vfs-read-chunk-size-limit is also specified and greater than
2363 –vfs-read-chunk-size, the chunk size for each open file will get dou‐
2364 bled for each chunk read, until the specified value is reached. A val‐
2365 ue of -1 will disable the limit and the chunk size will grow indefi‐
2366 nitely.
2367 With –vfs-read-chunk-size 100M and –vfs-read-chunk-size-limit 0 the
2369 following parts will be downloaded: 0-100M, 100M-200M, 200M-300M,
2370 300M-400M and so on. When –vfs-read-chunk-size-limit 500M is speci‐
2371 fied, the result would be 0-100M, 100M-300M, 300M-700M, 700M-1200M,
2372 1200M-1700M and so on.
2373 Chunked reading will only work with –vfs-cache-mode < full, as the file
2375 will always be copied to the vfs cache before opening with
2376 –vfs-cache-mode full.
2377 Directory Cache
2379 Using the --dir-cache-time flag, you can set how long a directory
2380 should be considered up to date and not refreshed from the backend.
2381 Changes made locally in the mount may appear immediately or invalidate
2382 the cache. However, changes done on the remote will only be picked up
2383 once the cache expires.
2384 Alternatively, you can send a SIGHUP signal to rclone for it to flush
2386 all directory caches, regardless of how old they are. Assuming only
2387 one rclone instance is running, you can reset the cache like this:
2388 kill -SIGHUP $(pidof rclone)
2390 If you configure rclone with a remote control (/rc) then you can use
2392 rclone rc to flush the whole directory cache:
2393 rclone rc vfs/forget
2395 Or individual files or directories:
2397 rclone rc vfs/forget file=path/to/file dir=path/to/dir
2399 File Buffering
2401 The --buffer-size flag determines the amount of memory, that will be
2402 used to buffer data in advance.
2403 Each open file descriptor will try to keep the specified amount of data
2405 in memory at all times. The buffered data is bound to one file de‐
2406 scriptor and won’t be shared between multiple open file descriptors of
2407 the same file.
2408 This flag is a upper limit for the used memory per file descriptor.
2410 The buffer will only use memory for data that is downloaded but not not
2411 yet read. If the buffer is empty, only a small amount of memory will
2412 be used. The maximum memory used by rclone for buffering can be up to
2413 --buffer-size * open files.
2414 File Caching
2416 These flags control the VFS file caching options. The VFS layer is
2417 used by rclone mount to make a cloud storage system work more like a
2418 normal file system.
2419 You’ll need to enable VFS caching if you want, for example, to read and
2421 write simultaneously to a file. See below for more details.
2422 Note that the VFS cache works in addition to the cache backend and you
2424 may find that you need one or the other or both.
2425 --cache-dir string Directory rclone will use for caching.
2427 --vfs-cache-max-age duration Max age of objects in the cache. (default 1h0m0s)
2428 --vfs-cache-mode string Cache mode off|minimal|writes|full (default "off")
2429 --vfs-cache-poll-interval duration Interval to poll the cache for stale objects. (default 1m0s)
2430 --vfs-cache-max-size int Max total size of objects in the cache. (default off)
2431 If run with -vv rclone will print the location of the file cache. The
2433 files are stored in the user cache file area which is OS dependent but
2434 can be controlled with --cache-dir or setting the appropriate environ‐
2435 ment variable.
2436 The cache has 4 different modes selected by --vfs-cache-mode. The
2438 higher the cache mode the more compatible rclone becomes at the cost of
2439 using disk space.
2440 Note that files are written back to the remote only when they are
2442 closed so if rclone is quit or dies with open files then these won’t
2443 get written back to the remote. However they will still be in the on
2444 disk cache.
2445 If using –vfs-cache-max-size note that the cache may exceed this size
2447 for two reasons. Firstly because it is only checked every
2448 –vfs-cache-poll-interval. Secondly because open files cannot be evict‐
2449 ed from the cache.
2450 –vfs-cache-mode off
2452 In this mode the cache will read directly from the remote and write di‐
2453 rectly to the remote without caching anything on disk.
2454 This will mean some operations are not possible
2456 · Files can’t be opened for both read AND write
2458 · Files opened for write can’t be seeked
2460 · Existing files opened for write must have O_TRUNC set
2462 · Files open for read with O_TRUNC will be opened write only
2464 · Files open for write only will behave as if O_TRUNC was supplied
2466 · Open modes O_APPEND, O_TRUNC are ignored
2468 · If an upload fails it can’t be retried
2470 –vfs-cache-mode minimal
2472 This is very similar to “off” except that files opened for read AND
2473 write will be buffered to disks. This means that files opened for
2474 write will be a lot more compatible, but uses the minimal disk space.
2475 These operations are not possible
2477 · Files opened for write only can’t be seeked
2479 · Existing files opened for write must have O_TRUNC set
2481 · Files opened for write only will ignore O_APPEND, O_TRUNC
2483 · If an upload fails it can’t be retried
2485 –vfs-cache-mode writes
2487 In this mode files opened for read only are still read directly from
2488 the remote, write only and read/write files are buffered to disk first.
2489 This mode should support all normal file system operations.
2491 If an upload fails it will be retried up to –low-level-retries times.
2493 –vfs-cache-mode full
2495 In this mode all reads and writes are buffered to and from disk. When
2496 a file is opened for read it will be downloaded in its entirety first.
2497 This may be appropriate for your needs, or you may prefer to look at
2499 the cache backend which does a much more sophisticated job of caching,
2500 including caching directory hierarchies and chunks of files.
2501 In this mode, unlike the others, when a file is written to the disk, it
2503 will be kept on the disk after it is written to the remote. It will be
2504 purged on a schedule according to --vfs-cache-max-age.
2505 This mode should support all normal file system operations.
2507 If an upload or download fails it will be retried up to –low-level-re‐
2509 tries times.
2510 rclone mount remote:path /path/to/mountpoint [flags]
2512 Options
2514 --allow-non-empty Allow mounting over a non-empty directory.
2515 --allow-other Allow access to other users.
2516 --allow-root Allow access to root user.
2517 --attr-timeout duration Time for which file/directory attributes are cached. (default 1s)
2518 --daemon Run mount as a daemon (background mode).
2519 --daemon-timeout duration Time limit for rclone to respond to kernel (not supported by all OSes).
2520 --debug-fuse Debug the FUSE internals - needs -v.
2521 --default-permissions Makes kernel enforce access control based on the file mode.
2522 --dir-cache-time duration Time to cache directory entries for. (default 5m0s)
2523 --dir-perms FileMode Directory permissions (default 0777)
2524 --file-perms FileMode File permissions (default 0666)
2525 --fuse-flag stringArray Flags or arguments to be passed direct to libfuse/WinFsp. Repeat if required.
2526 --gid uint32 Override the gid field set by the filesystem. (default 1000)
2527 -h, --help help for mount
2528 --max-read-ahead SizeSuffix The number of bytes that can be prefetched for sequential reads. (default 128k)
2529 --no-checksum Don't compare checksums on up/download.
2530 --no-modtime Don't read/write the modification time (can speed things up).
2531 --no-seek Don't allow seeking in files.
2532 -o, --option stringArray Option for libfuse/WinFsp. Repeat if required.
2533 --poll-interval duration Time to wait between polling for changes. Must be smaller than dir-cache-time. Only on supported remotes. Set to 0 to disable. (default 1m0s)
2534 --read-only Mount read-only.
2535 --uid uint32 Override the uid field set by the filesystem. (default 1000)
2536 --umask int Override the permission bits set by the filesystem.
2537 --vfs-cache-max-age duration Max age of objects in the cache. (default 1h0m0s)
2538 --vfs-cache-max-size SizeSuffix Max total size of objects in the cache. (default off)
2539 --vfs-cache-mode CacheMode Cache mode off|minimal|writes|full (default off)
2540 --vfs-cache-poll-interval duration Interval to poll the cache for stale objects. (default 1m0s)
2541 --vfs-case-insensitive If a file name not found, find a case insensitive match.
2542 --vfs-read-chunk-size SizeSuffix Read the source objects in chunks. (default 128M)
2543 --vfs-read-chunk-size-limit SizeSuffix If greater than --vfs-read-chunk-size, double the chunk size after each chunk read, until the limit is reached. 'off' is unlimited. (default off)
2544 --volname string Set the volume name (not supported by all OSes).
2545 --write-back-cache Makes kernel buffer writes before sending them to rclone. Without this, writethrough caching is used.
2546 See the global flags page (https://rclone.org/flags/) for global op‐
2548 tions not listed here.
2549 SEE ALSO
2551 · rclone (https://rclone.org/commands/rclone/) - Show help for rclone
2552 commands, flags and backends.
2553 rclone moveto
2555 Move file or directory from source to dest.
2556 Synopsis
2558 If source:path is a file or directory then it moves it to a file or di‐
2559 rectory named dest:path.
2560 This can be used to rename files or upload single files to other than
2562 their existing name. If the source is a directory then it acts exactly
2563 like the move command.
2564 So
2566 rclone moveto src dst
2568 where src and dst are rclone paths, either remote:path or /path/to/lo‐
2570 cal or C:.
2571 This will:
2573 if src is file
2575 move it to dst, overwriting an existing file if it exists
2576 if src is directory
2577 move it to dst, overwriting existing files if they exist
2578 see move command for full details
2579 This doesn’t transfer unchanged files, testing by size and modification
2581 time or MD5SUM. src will be deleted on successful transfer.
2582 Important: Since this can cause data loss, test first with the –dry-run
2584 flag.
2585 Note: Use the -P/--progress flag to view real-time transfer statistics.
2587 rclone moveto source:path dest:path [flags]
2589 Options
2591 -h, --help help for moveto
2592 See the global flags page (https://rclone.org/flags/) for global op‐
2594 tions not listed here.
2595 SEE ALSO
2597 · rclone (https://rclone.org/commands/rclone/) - Show help for rclone
2598 commands, flags and backends.
2599 rclone ncdu
2601 Explore a remote with a text based user interface.
2602 Synopsis
2604 This displays a text based user interface allowing the navigation of a
2605 remote. It is most useful for answering the question - “What is using
2606 all my disk space?”.
2607 To make the user interface it first scans the entire remote given and
2609 builds an in memory representation. rclone ncdu can be used during
2610 this scanning phase and you will see it building up the directory
2611 structure as it goes along.
2612 Here are the keys - press `?' to toggle the help on and off
2614 ↑,↓ or k,j to Move
2616 →,l to enter
2617 ←,h to return
2618 c toggle counts
2619 g toggle graph
2620 n,s,C sort by name,size,count
2621 d delete file/directory
2622 y copy current path to clipbard
2623 Y display current path
2624 ^L refresh screen
2625 ? to toggle help on and off
2626 q/ESC/c-C to quit
2627 This an homage to the ncdu tool (https://dev.yorhel.nl/ncdu) but for
2629 rclone remotes. It is missing lots of features at the moment but is
2630 useful as it stands.
2631 Note that it might take some time to delete big files/folders. The UI
2633 won’t respond in the meantime since the deletion is done synchronously.
2634 rclone ncdu remote:path [flags]
2636 Options
2638 -h, --help help for ncdu
2639 See the global flags page (https://rclone.org/flags/) for global op‐
2641 tions not listed here.
2642 SEE ALSO
2644 · rclone (https://rclone.org/commands/rclone/) - Show help for rclone
2645 commands, flags and backends.
2646 rclone obscure
2648 Obscure password for use in the rclone.conf
2649 Synopsis
2651 Obscure password for use in the rclone.conf
2652 rclone obscure password [flags]
2654 Options
2656 -h, --help help for obscure
2657 See the global flags page (https://rclone.org/flags/) for global op‐
2659 tions not listed here.
2660 SEE ALSO
2662 · rclone (https://rclone.org/commands/rclone/) - Show help for rclone
2663 commands, flags and backends.
2664 rclone rc
2666 Run a command against a running rclone.
2667 Synopsis
2669 This runs a command against a running rclone. Use the –url flag to
2670 specify an non default URL to connect on. This can be either a “:port”
2671 which is taken to mean “http://localhost:port” or a “host:port” which
2672 is taken to mean “http://host:port”
2673 A username and password can be passed in with –user and –pass.
2675 Note that –rc-addr, –rc-user, –rc-pass will be read also for –url, –us‐
2677 er, –pass.
2678 Arguments should be passed in as parameter=value.
2680 The result will be returned as a JSON object by default.
2682 The –json parameter can be used to pass in a JSON blob as an input in‐
2684 stead of key=value arguments. This is the only way of passing in more
2685 complicated values.
2686 Use –loopback to connect to the rclone instance running “rclone rc”.
2688 This is very useful for testing commands without having to run an
2689 rclone rc server, eg:
2690 rclone rc --loopback operations/about fs=/
2692 Use “rclone rc” to see a list of all possible commands.
2694 rclone rc commands parameter [flags]
2696 Options
2698 -h, --help help for rc
2699 --json string Input JSON - use instead of key=value args.
2700 --loopback If set connect to this rclone instance not via HTTP.
2701 --no-output If set don't output the JSON result.
2702 --pass string Password to use to connect to rclone remote control.
2703 --url string URL to connect to rclone remote control. (default "http://localhost:5572/")
2704 --user string Username to use to rclone remote control.
2705 See the global flags page (https://rclone.org/flags/) for global op‐
2707 tions not listed here.
2708 SEE ALSO
2710 · rclone (https://rclone.org/commands/rclone/) - Show help for rclone
2711 commands, flags and backends.
2712 rclone rcat
2714 Copies standard input to file on remote.
2715 Synopsis
2717 rclone rcat reads from standard input (stdin) and copies it to a single
2718 remote file.
2719 echo "hello world" | rclone rcat remote:path/to/file
2721 ffmpeg - | rclone rcat remote:path/to/file
2722 If the remote file already exists, it will be overwritten.
2724 rcat will try to upload small files in a single request, which is usu‐
2726 ally more efficient than the streaming/chunked upload endpoints, which
2727 use multiple requests. Exact behaviour depends on the remote. What is
2728 considered a small file may be set through --streaming-upload-cutoff.
2729 Uploading only starts after the cutoff is reached or if the file ends
2730 before that. The data must fit into RAM. The cutoff needs to be small
2731 enough to adhere the limits of your remote, please see there. General‐
2732 ly speaking, setting this cutoff too high will decrease your perfor‐
2733 mance.
2734 Note that the upload can also not be retried because the data is not
2736 kept around until the upload succeeds. If you need to transfer a lot
2737 of data, you’re better off caching locally and then rclone move it to
2738 the destination.
2739 rclone rcat remote:path [flags]
2741 Options
2743 -h, --help help for rcat
2744 See the global flags page (https://rclone.org/flags/) for global op‐
2746 tions not listed here.
2747 SEE ALSO
2749 · rclone (https://rclone.org/commands/rclone/) - Show help for rclone
2750 commands, flags and backends.
2751 rclone rcd
2753 Run rclone listening to remote control commands only.
2754 Synopsis
2756 This runs rclone so that it only listens to remote control commands.
2757 This is useful if you are controlling rclone via the rc API.
2759 If you pass in a path to a directory, rclone will serve that directory
2761 for GET requests on the URL passed in. It will also open the URL in
2762 the browser when rclone is run.
2763 See the rc documentation (https://rclone.org/rc/) for more info on the
2765 rc flags.
2766 rclone rcd <path to files to serve>* [flags]
2768 Options
2770 -h, --help help for rcd
2771 See the global flags page (https://rclone.org/flags/) for global op‐
2773 tions not listed here.
2774 SEE ALSO
2776 · rclone (https://rclone.org/commands/rclone/) - Show help for rclone
2777 commands, flags and backends.
2778 rclone rmdirs
2780 Remove empty directories under the path.
2781 Synopsis
2783 This removes any empty directories (or directories that only contain
2784 empty directories) under the path that it finds, including the path if
2785 it has nothing in.
2786 If you supply the –leave-root flag, it will not remove the root direc‐
2788 tory.
2789 This is useful for tidying up remotes that rclone has left a lot of
2791 empty directories in.
2792 rclone rmdirs remote:path [flags]
2794 Options
2796 -h, --help help for rmdirs
2797 --leave-root Do not remove root directory if empty
2798 See the global flags page (https://rclone.org/flags/) for global op‐
2800 tions not listed here.
2801 SEE ALSO
2803 · rclone (https://rclone.org/commands/rclone/) - Show help for rclone
2804 commands, flags and backends.
2805 rclone serve
2807 Serve a remote over a protocol.
2808 Synopsis
2810 rclone serve is used to serve a remote over a given protocol. This
2811 command requires the use of a subcommand to specify the protocol, eg
2812 rclone serve http remote:
2814 Each subcommand has its own options which you can see in their help.
2816 rclone serve <protocol> [opts] <remote> [flags]
2818 Options
2820 -h, --help help for serve
2821 See the global flags page (https://rclone.org/flags/) for global op‐
2823 tions not listed here.
2824 SEE ALSO
2826 · rclone (https://rclone.org/commands/rclone/) - Show help for rclone
2827 commands, flags and backends.
2828 · rclone serve dlna (https://rclone.org/commands/rclone_serve_dlna/) -
2830 Serve remote:path over DLNA
2831 · rclone serve ftp (https://rclone.org/commands/rclone_serve_ftp/) -
2833 Serve remote:path over FTP.
2834 · rclone serve http (https://rclone.org/commands/rclone_serve_http/) -
2836 Serve the remote over HTTP.
2837 · rclone serve restic (https://rclone.org/com‐
2839 mands/rclone_serve_restic/) - Serve the remote for restic’s REST API.
2840 · rclone serve sftp (https://rclone.org/commands/rclone_serve_sftp/) -
2842 Serve the remote over SFTP.
2843 · rclone serve webdav (https://rclone.org/commands/rclone_serve_web‐
2845 dav/) - Serve remote:path over webdav.
2846 rclone serve dlna
2848 Serve remote:path over DLNA
2849 Synopsis
2851 rclone serve dlna is a DLNA media server for media stored in a rclone
2852 remote. Many devices, such as the Xbox and PlayStation, can automati‐
2853 cally discover this server in the LAN and play audio/video from it.
2854 VLC is also supported. Service discovery uses UDP multicast packets
2855 (SSDP) and will thus only work on LANs.
2856 Rclone will list all files present in the remote, without filtering
2858 based on media formats or file extensions. Additionally, there is no
2859 media transcoding support. This means that some players might show
2860 files that they are not able to play back correctly.
2861 Server options
2863 Use –addr to specify which IP address and port the server should listen
2864 on, eg –addr 1.2.3.4:8000 or –addr :8080 to listen to all IPs.
2865 Use –name to choose the friendly server name, which is by default
2867 “rclone (hostname)”.
2868 Use –log-trace in conjunction with -vv to enable additional debug log‐
2870 ging of all UPNP traffic.
2871 Directory Cache
2873 Using the --dir-cache-time flag, you can set how long a directory
2874 should be considered up to date and not refreshed from the backend.
2875 Changes made locally in the mount may appear immediately or invalidate
2876 the cache. However, changes done on the remote will only be picked up
2877 once the cache expires.
2878 Alternatively, you can send a SIGHUP signal to rclone for it to flush
2880 all directory caches, regardless of how old they are. Assuming only
2881 one rclone instance is running, you can reset the cache like this:
2882 kill -SIGHUP $(pidof rclone)
2884 If you configure rclone with a remote control (/rc) then you can use
2886 rclone rc to flush the whole directory cache:
2887 rclone rc vfs/forget
2889 Or individual files or directories:
2891 rclone rc vfs/forget file=path/to/file dir=path/to/dir
2893 File Buffering
2895 The --buffer-size flag determines the amount of memory, that will be
2896 used to buffer data in advance.
2897 Each open file descriptor will try to keep the specified amount of data
2899 in memory at all times. The buffered data is bound to one file de‐
2900 scriptor and won’t be shared between multiple open file descriptors of
2901 the same file.
2902 This flag is a upper limit for the used memory per file descriptor.
2904 The buffer will only use memory for data that is downloaded but not not
2905 yet read. If the buffer is empty, only a small amount of memory will
2906 be used. The maximum memory used by rclone for buffering can be up to
2907 --buffer-size * open files.
2908 File Caching
2910 These flags control the VFS file caching options. The VFS layer is
2911 used by rclone mount to make a cloud storage system work more like a
2912 normal file system.
2913 You’ll need to enable VFS caching if you want, for example, to read and
2915 write simultaneously to a file. See below for more details.
2916 Note that the VFS cache works in addition to the cache backend and you
2918 may find that you need one or the other or both.
2919 --cache-dir string Directory rclone will use for caching.
2921 --vfs-cache-max-age duration Max age of objects in the cache. (default 1h0m0s)
2922 --vfs-cache-mode string Cache mode off|minimal|writes|full (default "off")
2923 --vfs-cache-poll-interval duration Interval to poll the cache for stale objects. (default 1m0s)
2924 --vfs-cache-max-size int Max total size of objects in the cache. (default off)
2925 If run with -vv rclone will print the location of the file cache. The
2927 files are stored in the user cache file area which is OS dependent but
2928 can be controlled with --cache-dir or setting the appropriate environ‐
2929 ment variable.
2930 The cache has 4 different modes selected by --vfs-cache-mode. The
2932 higher the cache mode the more compatible rclone becomes at the cost of
2933 using disk space.
2934 Note that files are written back to the remote only when they are
2936 closed so if rclone is quit or dies with open files then these won’t
2937 get written back to the remote. However they will still be in the on
2938 disk cache.
2939 If using –vfs-cache-max-size note that the cache may exceed this size
2941 for two reasons. Firstly because it is only checked every
2942 –vfs-cache-poll-interval. Secondly because open files cannot be evict‐
2943 ed from the cache.
2944 –vfs-cache-mode off
2946 In this mode the cache will read directly from the remote and write di‐
2947 rectly to the remote without caching anything on disk.
2948 This will mean some operations are not possible
2950 · Files can’t be opened for both read AND write
2952 · Files opened for write can’t be seeked
2954 · Existing files opened for write must have O_TRUNC set
2956 · Files open for read with O_TRUNC will be opened write only
2958 · Files open for write only will behave as if O_TRUNC was supplied
2960 · Open modes O_APPEND, O_TRUNC are ignored
2962 · If an upload fails it can’t be retried
2964 –vfs-cache-mode minimal
2966 This is very similar to “off” except that files opened for read AND
2967 write will be buffered to disks. This means that files opened for
2968 write will be a lot more compatible, but uses the minimal disk space.
2969 These operations are not possible
2971 · Files opened for write only can’t be seeked
2973 · Existing files opened for write must have O_TRUNC set
2975 · Files opened for write only will ignore O_APPEND, O_TRUNC
2977 · If an upload fails it can’t be retried
2979 –vfs-cache-mode writes
2981 In this mode files opened for read only are still read directly from
2982 the remote, write only and read/write files are buffered to disk first.
2983 This mode should support all normal file system operations.
2985 If an upload fails it will be retried up to –low-level-retries times.
2987 –vfs-cache-mode full
2989 In this mode all reads and writes are buffered to and from disk. When
2990 a file is opened for read it will be downloaded in its entirety first.
2991 This may be appropriate for your needs, or you may prefer to look at
2993 the cache backend which does a much more sophisticated job of caching,
2994 including caching directory hierarchies and chunks of files.
2995 In this mode, unlike the others, when a file is written to the disk, it
2997 will be kept on the disk after it is written to the remote. It will be
2998 purged on a schedule according to --vfs-cache-max-age.
2999 This mode should support all normal file system operations.
3001 If an upload or download fails it will be retried up to –low-level-re‐
3003 tries times.
3004 rclone serve dlna remote:path [flags]
3006 Options
3008 --addr string ip:port or :port to bind the DLNA http server to. (default ":7879")
3009 --dir-cache-time duration Time to cache directory entries for. (default 5m0s)
3010 --dir-perms FileMode Directory permissions (default 0777)
3011 --file-perms FileMode File permissions (default 0666)
3012 --gid uint32 Override the gid field set by the filesystem. (default 1000)
3013 -h, --help help for dlna
3014 --log-trace enable trace logging of SOAP traffic
3015 --name string name of DLNA server
3016 --no-checksum Don't compare checksums on up/download.
3017 --no-modtime Don't read/write the modification time (can speed things up).
3018 --no-seek Don't allow seeking in files.
3019 --poll-interval duration Time to wait between polling for changes. Must be smaller than dir-cache-time. Only on supported remotes. Set to 0 to disable. (default 1m0s)
3020 --read-only Mount read-only.
3021 --uid uint32 Override the uid field set by the filesystem. (default 1000)
3022 --umask int Override the permission bits set by the filesystem. (default 2)
3023 --vfs-cache-max-age duration Max age of objects in the cache. (default 1h0m0s)
3024 --vfs-cache-max-size SizeSuffix Max total size of objects in the cache. (default off)
3025 --vfs-cache-mode CacheMode Cache mode off|minimal|writes|full (default off)
3026 --vfs-cache-poll-interval duration Interval to poll the cache for stale objects. (default 1m0s)
3027 --vfs-case-insensitive If a file name not found, find a case insensitive match.
3028 --vfs-read-chunk-size SizeSuffix Read the source objects in chunks. (default 128M)
3029 --vfs-read-chunk-size-limit SizeSuffix If greater than --vfs-read-chunk-size, double the chunk size after each chunk read, until the limit is reached. 'off' is unlimited. (default off)
3030 See the global flags page (https://rclone.org/flags/) for global op‐
3032 tions not listed here.
3033 SEE ALSO
3035 · rclone serve (https://rclone.org/commands/rclone_serve/) - Serve a
3036 remote over a protocol.
3037 rclone serve ftp
3039 Serve remote:path over FTP.
3040 Synopsis
3042 rclone serve ftp implements a basic ftp server to serve the remote over
3043 FTP protocol. This can be viewed with a ftp client or you can make a
3044 remote of type ftp to read and write it.
3045 Server options
3047 Use –addr to specify which IP address and port the server should listen
3048 on, eg –addr 1.2.3.4:8000 or –addr :8080 to listen to all IPs. By de‐
3049 fault it only listens on localhost. You can use port :0 to let the OS
3050 choose an available port.
3051 If you set –addr to listen on a public or LAN accessible IP address
3053 then using Authentication is advised - see the next section for info.
3054 Authentication
3056 By default this will serve files without needing a login.
3057 You can set a single username and password with the –user and –pass
3059 flags.
3060 Directory Cache
3062 Using the --dir-cache-time flag, you can set how long a directory
3063 should be considered up to date and not refreshed from the backend.
3064 Changes made locally in the mount may appear immediately or invalidate
3065 the cache. However, changes done on the remote will only be picked up
3066 once the cache expires.
3067 Alternatively, you can send a SIGHUP signal to rclone for it to flush
3069 all directory caches, regardless of how old they are. Assuming only
3070 one rclone instance is running, you can reset the cache like this:
3071 kill -SIGHUP $(pidof rclone)
3073 If you configure rclone with a remote control (/rc) then you can use
3075 rclone rc to flush the whole directory cache:
3076 rclone rc vfs/forget
3078 Or individual files or directories:
3080 rclone rc vfs/forget file=path/to/file dir=path/to/dir
3082 File Buffering
3084 The --buffer-size flag determines the amount of memory, that will be
3085 used to buffer data in advance.
3086 Each open file descriptor will try to keep the specified amount of data
3088 in memory at all times. The buffered data is bound to one file de‐
3089 scriptor and won’t be shared between multiple open file descriptors of
3090 the same file.
3091 This flag is a upper limit for the used memory per file descriptor.
3093 The buffer will only use memory for data that is downloaded but not not
3094 yet read. If the buffer is empty, only a small amount of memory will
3095 be used. The maximum memory used by rclone for buffering can be up to
3096 --buffer-size * open files.
3097 File Caching
3099 These flags control the VFS file caching options. The VFS layer is
3100 used by rclone mount to make a cloud storage system work more like a
3101 normal file system.
3102 You’ll need to enable VFS caching if you want, for example, to read and
3104 write simultaneously to a file. See below for more details.
3105 Note that the VFS cache works in addition to the cache backend and you
3107 may find that you need one or the other or both.
3108 --cache-dir string Directory rclone will use for caching.
3110 --vfs-cache-max-age duration Max age of objects in the cache. (default 1h0m0s)
3111 --vfs-cache-mode string Cache mode off|minimal|writes|full (default "off")
3112 --vfs-cache-poll-interval duration Interval to poll the cache for stale objects. (default 1m0s)
3113 --vfs-cache-max-size int Max total size of objects in the cache. (default off)
3114 If run with -vv rclone will print the location of the file cache. The
3116 files are stored in the user cache file area which is OS dependent but
3117 can be controlled with --cache-dir or setting the appropriate environ‐
3118 ment variable.
3119 The cache has 4 different modes selected by --vfs-cache-mode. The
3121 higher the cache mode the more compatible rclone becomes at the cost of
3122 using disk space.
3123 Note that files are written back to the remote only when they are
3125 closed so if rclone is quit or dies with open files then these won’t
3126 get written back to the remote. However they will still be in the on
3127 disk cache.
3128 If using –vfs-cache-max-size note that the cache may exceed this size
3130 for two reasons. Firstly because it is only checked every
3131 –vfs-cache-poll-interval. Secondly because open files cannot be evict‐
3132 ed from the cache.
3133 –vfs-cache-mode off
3135 In this mode the cache will read directly from the remote and write di‐
3136 rectly to the remote without caching anything on disk.
3137 This will mean some operations are not possible
3139 · Files can’t be opened for both read AND write
3141 · Files opened for write can’t be seeked
3143 · Existing files opened for write must have O_TRUNC set
3145 · Files open for read with O_TRUNC will be opened write only
3147 · Files open for write only will behave as if O_TRUNC was supplied
3149 · Open modes O_APPEND, O_TRUNC are ignored
3151 · If an upload fails it can’t be retried
3153 –vfs-cache-mode minimal
3155 This is very similar to “off” except that files opened for read AND
3156 write will be buffered to disks. This means that files opened for
3157 write will be a lot more compatible, but uses the minimal disk space.
3158 These operations are not possible
3160 · Files opened for write only can’t be seeked
3162 · Existing files opened for write must have O_TRUNC set
3164 · Files opened for write only will ignore O_APPEND, O_TRUNC
3166 · If an upload fails it can’t be retried
3168 –vfs-cache-mode writes
3170 In this mode files opened for read only are still read directly from
3171 the remote, write only and read/write files are buffered to disk first.
3172 This mode should support all normal file system operations.
3174 If an upload fails it will be retried up to –low-level-retries times.
3176 –vfs-cache-mode full
3178 In this mode all reads and writes are buffered to and from disk. When
3179 a file is opened for read it will be downloaded in its entirety first.
3180 This may be appropriate for your needs, or you may prefer to look at
3182 the cache backend which does a much more sophisticated job of caching,
3183 including caching directory hierarchies and chunks of files.
3184 In this mode, unlike the others, when a file is written to the disk, it
3186 will be kept on the disk after it is written to the remote. It will be
3187 purged on a schedule according to --vfs-cache-max-age.
3188 This mode should support all normal file system operations.
3190 If an upload or download fails it will be retried up to –low-level-re‐
3192 tries times.
3193 Auth Proxy
3195 If you supply the parameter --auth-proxy /path/to/program then rclone
3196 will use that program to generate backends on the fly which then are
3197 used to authenticate incoming requests. This uses a simple JSON based
3198 protocl with input on STDIN and output on STDOUT.
3199 There is an example program bin/test_proxy.py
3201 (https://github.com/rclone/rclone/blob/master/test_proxy.py) in the
3202 rclone source code.
3203 The program’s job is to take a user and pass on the input and turn
3205 those into the config for a backend on STDOUT in JSON format. This
3206 config will have any default parameters for the backend added, but it
3207 won’t use configuration from environment variables or command line op‐
3208 tions - it is the job of the proxy program to make a complete config.
3209 This config generated must have this extra parameter - _root - root to
3211 use for the backend
3212 And it may have this parameter - _obscure - comma separated strings for
3214 parameters to obscure
3215 For example the program might take this on STDIN
3217 {
3219 "user": "me",
3220 "pass": "mypassword"
3221 }
3222 And return this on STDOUT
3224 {
3226 "type": "sftp",
3227 "_root": "",
3228 "_obscure": "pass",
3229 "user": "me",
3230 "pass": "mypassword",
3231 "host": "sftp.example.com"
3232 }
3233 This would mean that an SFTP backend would be created on the fly for
3235 the user and pass returned in the output to the host given. Note that
3236 since _obscure is set to pass, rclone will obscure the pass parameter
3237 before creating the backend (which is required for sftp backends).
3238 The progam can manipulate the supplied user in any way, for example to
3240 make proxy to many different sftp backends, you could make the user be
3241 user@example.com and then set the host to example.com in the output and
3242 the user to user. For security you’d probably want to restrict the
3243 host to a limited list.
3244 Note that an internal cache is keyed on user so only use that for con‐
3246 figuration, don’t use pass. This also means that if a user’s password
3247 is changed the cache will need to expire (which takes 5 mins) before it
3248 takes effect.
3249 This can be used to build general purpose proxies to any kind of back‐
3251 end that rclone supports.
3252 rclone serve ftp remote:path [flags]
3254 Options
3256 --addr string IPaddress:Port or :Port to bind server to. (default "localhost:2121")
3257 --auth-proxy string A program to use to create the backend from the auth.
3258 --dir-cache-time duration Time to cache directory entries for. (default 5m0s)
3259 --dir-perms FileMode Directory permissions (default 0777)
3260 --file-perms FileMode File permissions (default 0666)
3261 --gid uint32 Override the gid field set by the filesystem. (default 1000)
3262 -h, --help help for ftp
3263 --no-checksum Don't compare checksums on up/download.
3264 --no-modtime Don't read/write the modification time (can speed things up).
3265 --no-seek Don't allow seeking in files.
3266 --pass string Password for authentication. (empty value allow every password)
3267 --passive-port string Passive port range to use. (default "30000-32000")
3268 --poll-interval duration Time to wait between polling for changes. Must be smaller than dir-cache-time. Only on supported remotes. Set to 0 to disable. (default 1m0s)
3269 --public-ip string Public IP address to advertise for passive connections.
3270 --read-only Mount read-only.
3271 --uid uint32 Override the uid field set by the filesystem. (default 1000)
3272 --umask int Override the permission bits set by the filesystem. (default 2)
3273 --user string User name for authentication. (default "anonymous")
3274 --vfs-cache-max-age duration Max age of objects in the cache. (default 1h0m0s)
3275 --vfs-cache-max-size SizeSuffix Max total size of objects in the cache. (default off)
3276 --vfs-cache-mode CacheMode Cache mode off|minimal|writes|full (default off)
3277 --vfs-cache-poll-interval duration Interval to poll the cache for stale objects. (default 1m0s)
3278 --vfs-case-insensitive If a file name not found, find a case insensitive match.
3279 --vfs-read-chunk-size SizeSuffix Read the source objects in chunks. (default 128M)
3280 --vfs-read-chunk-size-limit SizeSuffix If greater than --vfs-read-chunk-size, double the chunk size after each chunk read, until the limit is reached. 'off' is unlimited. (default off)
3281 See the global flags page (https://rclone.org/flags/) for global op‐
3283 tions not listed here.
3284 SEE ALSO
3286 · rclone serve (https://rclone.org/commands/rclone_serve/) - Serve a
3287 remote over a protocol.
3288 rclone serve http
3290 Serve the remote over HTTP.
3291 Synopsis
3293 rclone serve http implements a basic web server to serve the remote
3294 over HTTP. This can be viewed in a web browser or you can make a re‐
3295 mote of type http read from it.
3296 You can use the filter flags (eg –include, –exclude) to control what is
3298 served.
3299 The server will log errors. Use -v to see access logs.
3301 –bwlimit will be respected for file transfers. Use –stats to control
3303 the stats printing.
3304 Server options
3306 Use –addr to specify which IP address and port the server should listen
3307 on, eg –addr 1.2.3.4:8000 or –addr :8080 to listen to all IPs. By de‐
3308 fault it only listens on localhost. You can use port :0 to let the OS
3309 choose an available port.
3310 If you set –addr to listen on a public or LAN accessible IP address
3312 then using Authentication is advised - see the next section for info.
3313 –server-read-timeout and –server-write-timeout can be used to control
3315 the timeouts on the server. Note that this is the total time for a
3316 transfer.
3317 –max-header-bytes controls the maximum number of bytes the server will
3319 accept in the HTTP header.
3320 –baseurl controls the URL prefix that rclone serves from. By default
3322 rclone will serve from the root. If you used –baseurl “/rclone” then
3323 rclone would serve from a URL starting with “/rclone/”. This is useful
3324 if you wish to proxy rclone serve. Rclone automatically inserts lead‐
3325 ing and trailing “/” on –baseurl, so –baseurl “rclone”, –baseurl
3326 “/rclone” and –baseurl “/rclone/” are all treated identically.
3327 Authentication
3329 By default this will serve files without needing a login.
3330 You can either use an htpasswd file which can take lots of users, or
3332 set a single username and password with the –user and –pass flags.
3333 Use –htpasswd /path/to/htpasswd to provide an htpasswd file. This is
3335 in standard apache format and supports MD5, SHA1 and BCrypt for basic
3336 authentication. Bcrypt is recommended.
3337 To create an htpasswd file:
3339 touch htpasswd
3341 htpasswd -B htpasswd user
3342 htpasswd -B htpasswd anotherUser
3343 The password file can be updated while rclone is running.
3345 Use –realm to set the authentication realm.
3347 SSL/TLS
3349 By default this will serve over http. If you want you can serve over
3350 https. You will need to supply the –cert and –key flags. If you wish
3351 to do client side certificate validation then you will need to supply
3352 –client-ca also.
3353 –cert should be a either a PEM encoded certificate or a concatenation
3355 of that with the CA certificate. –key should be the PEM encoded pri‐
3356 vate key and –client-ca should be the PEM encoded client certificate
3357 authority certificate.
3358 Directory Cache
3360 Using the --dir-cache-time flag, you can set how long a directory
3361 should be considered up to date and not refreshed from the backend.
3362 Changes made locally in the mount may appear immediately or invalidate
3363 the cache. However, changes done on the remote will only be picked up
3364 once the cache expires.
3365 Alternatively, you can send a SIGHUP signal to rclone for it to flush
3367 all directory caches, regardless of how old they are. Assuming only
3368 one rclone instance is running, you can reset the cache like this:
3369 kill -SIGHUP $(pidof rclone)
3371 If you configure rclone with a remote control (/rc) then you can use
3373 rclone rc to flush the whole directory cache:
3374 rclone rc vfs/forget
3376 Or individual files or directories:
3378 rclone rc vfs/forget file=path/to/file dir=path/to/dir
3380 File Buffering
3382 The --buffer-size flag determines the amount of memory, that will be
3383 used to buffer data in advance.
3384 Each open file descriptor will try to keep the specified amount of data
3386 in memory at all times. The buffered data is bound to one file de‐
3387 scriptor and won’t be shared between multiple open file descriptors of
3388 the same file.
3389 This flag is a upper limit for the used memory per file descriptor.
3391 The buffer will only use memory for data that is downloaded but not not
3392 yet read. If the buffer is empty, only a small amount of memory will
3393 be used. The maximum memory used by rclone for buffering can be up to
3394 --buffer-size * open files.
3395 File Caching
3397 These flags control the VFS file caching options. The VFS layer is
3398 used by rclone mount to make a cloud storage system work more like a
3399 normal file system.
3400 You’ll need to enable VFS caching if you want, for example, to read and
3402 write simultaneously to a file. See below for more details.
3403 Note that the VFS cache works in addition to the cache backend and you
3405 may find that you need one or the other or both.
3406 --cache-dir string Directory rclone will use for caching.
3408 --vfs-cache-max-age duration Max age of objects in the cache. (default 1h0m0s)
3409 --vfs-cache-mode string Cache mode off|minimal|writes|full (default "off")
3410 --vfs-cache-poll-interval duration Interval to poll the cache for stale objects. (default 1m0s)
3411 --vfs-cache-max-size int Max total size of objects in the cache. (default off)
3412 If run with -vv rclone will print the location of the file cache. The
3414 files are stored in the user cache file area which is OS dependent but
3415 can be controlled with --cache-dir or setting the appropriate environ‐
3416 ment variable.
3417 The cache has 4 different modes selected by --vfs-cache-mode. The
3419 higher the cache mode the more compatible rclone becomes at the cost of
3420 using disk space.
3421 Note that files are written back to the remote only when they are
3423 closed so if rclone is quit or dies with open files then these won’t
3424 get written back to the remote. However they will still be in the on
3425 disk cache.
3426 If using –vfs-cache-max-size note that the cache may exceed this size
3428 for two reasons. Firstly because it is only checked every
3429 –vfs-cache-poll-interval. Secondly because open files cannot be evict‐
3430 ed from the cache.
3431 –vfs-cache-mode off
3433 In this mode the cache will read directly from the remote and write di‐
3434 rectly to the remote without caching anything on disk.
3435 This will mean some operations are not possible
3437 · Files can’t be opened for both read AND write
3439 · Files opened for write can’t be seeked
3441 · Existing files opened for write must have O_TRUNC set
3443 · Files open for read with O_TRUNC will be opened write only
3445 · Files open for write only will behave as if O_TRUNC was supplied
3447 · Open modes O_APPEND, O_TRUNC are ignored
3449 · If an upload fails it can’t be retried
3451 –vfs-cache-mode minimal
3453 This is very similar to “off” except that files opened for read AND
3454 write will be buffered to disks. This means that files opened for
3455 write will be a lot more compatible, but uses the minimal disk space.
3456 These operations are not possible
3458 · Files opened for write only can’t be seeked
3460 · Existing files opened for write must have O_TRUNC set
3462 · Files opened for write only will ignore O_APPEND, O_TRUNC
3464 · If an upload fails it can’t be retried
3466 –vfs-cache-mode writes
3468 In this mode files opened for read only are still read directly from
3469 the remote, write only and read/write files are buffered to disk first.
3470 This mode should support all normal file system operations.
3472 If an upload fails it will be retried up to –low-level-retries times.
3474 –vfs-cache-mode full
3476 In this mode all reads and writes are buffered to and from disk. When
3477 a file is opened for read it will be downloaded in its entirety first.
3478 This may be appropriate for your needs, or you may prefer to look at
3480 the cache backend which does a much more sophisticated job of caching,
3481 including caching directory hierarchies and chunks of files.
3482 In this mode, unlike the others, when a file is written to the disk, it
3484 will be kept on the disk after it is written to the remote. It will be
3485 purged on a schedule according to --vfs-cache-max-age.
3486 This mode should support all normal file system operations.
3488 If an upload or download fails it will be retried up to –low-level-re‐
3490 tries times.
3491 rclone serve http remote:path [flags]
3493 Options
3495 --addr string IPaddress:Port or :Port to bind server to. (default "localhost:8080")
3496 --baseurl string Prefix for URLs - leave blank for root.
3497 --cert string SSL PEM key (concatenation of certificate and CA certificate)
3498 --client-ca string Client certificate authority to verify clients with
3499 --dir-cache-time duration Time to cache directory entries for. (default 5m0s)
3500 --dir-perms FileMode Directory permissions (default 0777)
3501 --file-perms FileMode File permissions (default 0666)
3502 --gid uint32 Override the gid field set by the filesystem. (default 1000)
3503 -h, --help help for http
3504 --htpasswd string htpasswd file - if not provided no authentication is done
3505 --key string SSL PEM Private key
3506 --max-header-bytes int Maximum size of request header (default 4096)
3507 --no-checksum Don't compare checksums on up/download.
3508 --no-modtime Don't read/write the modification time (can speed things up).
3509 --no-seek Don't allow seeking in files.
3510 --pass string Password for authentication.
3511 --poll-interval duration Time to wait between polling for changes. Must be smaller than dir-cache-time. Only on supported remotes. Set to 0 to disable. (default 1m0s)
3512 --read-only Mount read-only.
3513 --realm string realm for authentication (default "rclone")
3514 --server-read-timeout duration Timeout for server reading data (default 1h0m0s)
3515 --server-write-timeout duration Timeout for server writing data (default 1h0m0s)
3516 --uid uint32 Override the uid field set by the filesystem. (default 1000)
3517 --umask int Override the permission bits set by the filesystem. (default 2)
3518 --user string User name for authentication.
3519 --vfs-cache-max-age duration Max age of objects in the cache. (default 1h0m0s)
3520 --vfs-cache-max-size SizeSuffix Max total size of objects in the cache. (default off)
3521 --vfs-cache-mode CacheMode Cache mode off|minimal|writes|full (default off)
3522 --vfs-cache-poll-interval duration Interval to poll the cache for stale objects. (default 1m0s)
3523 --vfs-case-insensitive If a file name not found, find a case insensitive match.
3524 --vfs-read-chunk-size SizeSuffix Read the source objects in chunks. (default 128M)
3525 --vfs-read-chunk-size-limit SizeSuffix If greater than --vfs-read-chunk-size, double the chunk size after each chunk read, until the limit is reached. 'off' is unlimited. (default off)
3526 See the global flags page (https://rclone.org/flags/) for global op‐
3528 tions not listed here.
3529 SEE ALSO
3531 · rclone serve (https://rclone.org/commands/rclone_serve/) - Serve a
3532 remote over a protocol.
3533 rclone serve restic
3535 Serve the remote for restic’s REST API.
3536 Synopsis
3538 rclone serve restic implements restic’s REST backend API over HTTP.
3539 This allows restic to use rclone as a data storage mechanism for cloud
3540 providers that restic does not support directly.
3541 Restic (https://restic.net/) is a command line program for doing back‐
3543 ups.
3544 The server will log errors. Use -v to see access logs.
3546 –bwlimit will be respected for file transfers. Use –stats to control
3548 the stats printing.
3549 Setting up rclone for use by restic
3551 First set up a remote for your chosen cloud provider (/docs/#config‐
3552 ure).
3553 Once you have set up the remote, check it is working with, for example
3555 “rclone lsd remote:”. You may have called the remote something other
3556 than “remote:” - just substitute whatever you called it in the follow‐
3557 ing instructions.
3558 Now start the rclone restic server
3560 rclone serve restic -v remote:backup
3562 Where you can replace “backup” in the above by whatever path in the re‐
3564 mote you wish to use.
3565 By default this will serve on “localhost:8080” you can change this with
3567 use of the “–addr” flag.
3568 You might wish to start this server on boot.
3570 Setting up restic to use rclone
3572 Now you can follow the restic instructions (http://restic.readthedo‐
3573 cs.io/en/latest/030_preparing_a_new_repo.html#rest-server) on setting
3574 up restic.
3575 Note that you will need restic 0.8.2 or later to interoperate with
3577 rclone.
3578 For the example above you will want to use “http://localhost:8080/” as
3580 the URL for the REST server.
3581 For example:
3583 $ export RESTIC_REPOSITORY=rest:http://localhost:8080/
3585 $ export RESTIC_PASSWORD=yourpassword
3586 $ restic init
3587 created restic backend 8b1a4b56ae at rest:http://localhost:8080/
3588 Please note that knowledge of your password is required to access
3590 the repository. Losing your password means that your data is
3591 irrecoverably lost.
3592 $ restic backup /path/to/files/to/backup
3593 scan [/path/to/files/to/backup]
3594 scanned 189 directories, 312 files in 0:00
3595 [0:00] 100.00% 38.128 MiB / 38.128 MiB 501 / 501 items 0 errors ETA 0:00
3596 duration: 0:00
3597 snapshot 45c8fdd8 saved
3598 Multiple repositories
3600 Note that you can use the endpoint to host multiple repositories. Do
3601 this by adding a directory name or path after the URL. Note that these
3602 must end with /. Eg
3603 $ export RESTIC_REPOSITORY=rest:http://localhost:8080/user1repo/
3605 # backup user1 stuff
3606 $ export RESTIC_REPOSITORY=rest:http://localhost:8080/user2repo/
3607 # backup user2 stuff
3608 Private repositories
3610 The “–private-repos” flag can be used to limit users to repositories
3611 starting with a path of “//”.
3612 Server options
3614 Use –addr to specify which IP address and port the server should listen
3615 on, eg –addr 1.2.3.4:8000 or –addr :8080 to listen to all IPs. By de‐
3616 fault it only listens on localhost. You can use port :0 to let the OS
3617 choose an available port.
3618 If you set –addr to listen on a public or LAN accessible IP address
3620 then using Authentication is advised - see the next section for info.
3621 –server-read-timeout and –server-write-timeout can be used to control
3623 the timeouts on the server. Note that this is the total time for a
3624 transfer.
3625 –max-header-bytes controls the maximum number of bytes the server will
3627 accept in the HTTP header.
3628 –baseurl controls the URL prefix that rclone serves from. By default
3630 rclone will serve from the root. If you used –baseurl “/rclone” then
3631 rclone would serve from a URL starting with “/rclone/”. This is useful
3632 if you wish to proxy rclone serve. Rclone automatically inserts lead‐
3633 ing and trailing “/” on –baseurl, so –baseurl “rclone”, –baseurl
3634 “/rclone” and –baseurl “/rclone/” are all treated identically.
3635 Authentication
3637 By default this will serve files without needing a login.
3638 You can either use an htpasswd file which can take lots of users, or
3640 set a single username and password with the –user and –pass flags.
3641 Use –htpasswd /path/to/htpasswd to provide an htpasswd file. This is
3643 in standard apache format and supports MD5, SHA1 and BCrypt for basic
3644 authentication. Bcrypt is recommended.
3645 To create an htpasswd file:
3647 touch htpasswd
3649 htpasswd -B htpasswd user
3650 htpasswd -B htpasswd anotherUser
3651 The password file can be updated while rclone is running.
3653 Use –realm to set the authentication realm.
3655 SSL/TLS
3657 By default this will serve over http. If you want you can serve over
3658 https. You will need to supply the –cert and –key flags. If you wish
3659 to do client side certificate validation then you will need to supply
3660 –client-ca also.
3661 –cert should be a either a PEM encoded certificate or a concatenation
3663 of that with the CA certificate. –key should be the PEM encoded pri‐
3664 vate key and –client-ca should be the PEM encoded client certificate
3665 authority certificate.
3666 rclone serve restic remote:path [flags]
3668 Options
3670 --addr string IPaddress:Port or :Port to bind server to. (default "localhost:8080")
3671 --append-only disallow deletion of repository data
3672 --baseurl string Prefix for URLs - leave blank for root.
3673 --cert string SSL PEM key (concatenation of certificate and CA certificate)
3674 --client-ca string Client certificate authority to verify clients with
3675 -h, --help help for restic
3676 --htpasswd string htpasswd file - if not provided no authentication is done
3677 --key string SSL PEM Private key
3678 --max-header-bytes int Maximum size of request header (default 4096)
3679 --pass string Password for authentication.
3680 --private-repos users can only access their private repo
3681 --realm string realm for authentication (default "rclone")
3682 --server-read-timeout duration Timeout for server reading data (default 1h0m0s)
3683 --server-write-timeout duration Timeout for server writing data (default 1h0m0s)
3684 --stdio run an HTTP2 server on stdin/stdout
3685 --user string User name for authentication.
3686 See the global flags page (https://rclone.org/flags/) for global op‐
3688 tions not listed here.
3689 SEE ALSO
3691 · rclone serve (https://rclone.org/commands/rclone_serve/) - Serve a
3692 remote over a protocol.
3693 rclone serve sftp
3695 Serve the remote over SFTP.
3696 Synopsis
3698 rclone serve sftp implements an SFTP server to serve the remote over
3699 SFTP. This can be used with an SFTP client or you can make a remote of
3700 type sftp to use with it.
3701 You can use the filter flags (eg –include, –exclude) to control what is
3703 served.
3704 The server will log errors. Use -v to see access logs.
3706 –bwlimit will be respected for file transfers. Use –stats to control
3708 the stats printing.
3709 You must provide some means of authentication, either with –user/–pass,
3711 an authorized keys file (specify location with –authorized-keys - the
3712 default is the same as ssh) or set the –no-auth flag for no authentica‐
3713 tion when logging in.
3714 Note that this also implements a small number of shell commands so that
3716 it can provide md5sum/sha1sum/df information for the rclone sftp back‐
3717 end. This means that is can support SHA1SUMs, MD5SUMs and the about
3718 command when paired with the rclone sftp backend.
3719 If you don’t supply a –key then rclone will generate one and cache it
3721 for later use.
3722 By default the server binds to localhost:2022 - if you want it to be
3724 reachable externally then supply “–addr :2022” for example.
3725 Note that the default of “–vfs-cache-mode off” is fine for the rclone
3727 sftp backend, but it may not be with other SFTP clients.
3728 Directory Cache
3730 Using the --dir-cache-time flag, you can set how long a directory
3731 should be considered up to date and not refreshed from the backend.
3732 Changes made locally in the mount may appear immediately or invalidate
3733 the cache. However, changes done on the remote will only be picked up
3734 once the cache expires.
3735 Alternatively, you can send a SIGHUP signal to rclone for it to flush
3737 all directory caches, regardless of how old they are. Assuming only
3738 one rclone instance is running, you can reset the cache like this:
3739 kill -SIGHUP $(pidof rclone)
3741 If you configure rclone with a remote control (/rc) then you can use
3743 rclone rc to flush the whole directory cache:
3744 rclone rc vfs/forget
3746 Or individual files or directories:
3748 rclone rc vfs/forget file=path/to/file dir=path/to/dir
3750 File Buffering
3752 The --buffer-size flag determines the amount of memory, that will be
3753 used to buffer data in advance.
3754 Each open file descriptor will try to keep the specified amount of data
3756 in memory at all times. The buffered data is bound to one file de‐
3757 scriptor and won’t be shared between multiple open file descriptors of
3758 the same file.
3759 This flag is a upper limit for the used memory per file descriptor.
3761 The buffer will only use memory for data that is downloaded but not not
3762 yet read. If the buffer is empty, only a small amount of memory will
3763 be used. The maximum memory used by rclone for buffering can be up to
3764 --buffer-size * open files.
3765 File Caching
3767 These flags control the VFS file caching options. The VFS layer is
3768 used by rclone mount to make a cloud storage system work more like a
3769 normal file system.
3770 You’ll need to enable VFS caching if you want, for example, to read and
3772 write simultaneously to a file. See below for more details.
3773 Note that the VFS cache works in addition to the cache backend and you
3775 may find that you need one or the other or both.
3776 --cache-dir string Directory rclone will use for caching.
3778 --vfs-cache-max-age duration Max age of objects in the cache. (default 1h0m0s)
3779 --vfs-cache-mode string Cache mode off|minimal|writes|full (default "off")
3780 --vfs-cache-poll-interval duration Interval to poll the cache for stale objects. (default 1m0s)
3781 --vfs-cache-max-size int Max total size of objects in the cache. (default off)
3782 If run with -vv rclone will print the location of the file cache. The
3784 files are stored in the user cache file area which is OS dependent but
3785 can be controlled with --cache-dir or setting the appropriate environ‐
3786 ment variable.
3787 The cache has 4 different modes selected by --vfs-cache-mode. The
3789 higher the cache mode the more compatible rclone becomes at the cost of
3790 using disk space.
3791 Note that files are written back to the remote only when they are
3793 closed so if rclone is quit or dies with open files then these won’t
3794 get written back to the remote. However they will still be in the on
3795 disk cache.
3796 If using –vfs-cache-max-size note that the cache may exceed this size
3798 for two reasons. Firstly because it is only checked every
3799 –vfs-cache-poll-interval. Secondly because open files cannot be evict‐
3800 ed from the cache.
3801 –vfs-cache-mode off
3803 In this mode the cache will read directly from the remote and write di‐
3804 rectly to the remote without caching anything on disk.
3805 This will mean some operations are not possible
3807 · Files can’t be opened for both read AND write
3809 · Files opened for write can’t be seeked
3811 · Existing files opened for write must have O_TRUNC set
3813 · Files open for read with O_TRUNC will be opened write only
3815 · Files open for write only will behave as if O_TRUNC was supplied
3817 · Open modes O_APPEND, O_TRUNC are ignored
3819 · If an upload fails it can’t be retried
3821 –vfs-cache-mode minimal
3823 This is very similar to “off” except that files opened for read AND
3824 write will be buffered to disks. This means that files opened for
3825 write will be a lot more compatible, but uses the minimal disk space.
3826 These operations are not possible
3828 · Files opened for write only can’t be seeked
3830 · Existing files opened for write must have O_TRUNC set
3832 · Files opened for write only will ignore O_APPEND, O_TRUNC
3834 · If an upload fails it can’t be retried
3836 –vfs-cache-mode writes
3838 In this mode files opened for read only are still read directly from
3839 the remote, write only and read/write files are buffered to disk first.
3840 This mode should support all normal file system operations.
3842 If an upload fails it will be retried up to –low-level-retries times.
3844 –vfs-cache-mode full
3846 In this mode all reads and writes are buffered to and from disk. When
3847 a file is opened for read it will be downloaded in its entirety first.
3848 This may be appropriate for your needs, or you may prefer to look at
3850 the cache backend which does a much more sophisticated job of caching,
3851 including caching directory hierarchies and chunks of files.
3852 In this mode, unlike the others, when a file is written to the disk, it
3854 will be kept on the disk after it is written to the remote. It will be
3855 purged on a schedule according to --vfs-cache-max-age.
3856 This mode should support all normal file system operations.
3858 If an upload or download fails it will be retried up to –low-level-re‐
3860 tries times.
3861 Auth Proxy
3863 If you supply the parameter --auth-proxy /path/to/program then rclone
3864 will use that program to generate backends on the fly which then are
3865 used to authenticate incoming requests. This uses a simple JSON based
3866 protocl with input on STDIN and output on STDOUT.
3867 There is an example program bin/test_proxy.py
3869 (https://github.com/rclone/rclone/blob/master/test_proxy.py) in the
3870 rclone source code.
3871 The program’s job is to take a user and pass on the input and turn
3873 those into the config for a backend on STDOUT in JSON format. This
3874 config will have any default parameters for the backend added, but it
3875 won’t use configuration from environment variables or command line op‐
3876 tions - it is the job of the proxy program to make a complete config.
3877 This config generated must have this extra parameter - _root - root to
3879 use for the backend
3880 And it may have this parameter - _obscure - comma separated strings for
3882 parameters to obscure
3883 For example the program might take this on STDIN
3885 {
3887 "user": "me",
3888 "pass": "mypassword"
3889 }
3890 And return this on STDOUT
3892 {
3894 "type": "sftp",
3895 "_root": "",
3896 "_obscure": "pass",
3897 "user": "me",
3898 "pass": "mypassword",
3899 "host": "sftp.example.com"
3900 }
3901 This would mean that an SFTP backend would be created on the fly for
3903 the user and pass returned in the output to the host given. Note that
3904 since _obscure is set to pass, rclone will obscure the pass parameter
3905 before creating the backend (which is required for sftp backends).
3906 The progam can manipulate the supplied user in any way, for example to
3908 make proxy to many different sftp backends, you could make the user be
3909 user@example.com and then set the host to example.com in the output and
3910 the user to user. For security you’d probably want to restrict the
3911 host to a limited list.
3912 Note that an internal cache is keyed on user so only use that for con‐
3914 figuration, don’t use pass. This also means that if a user’s password
3915 is changed the cache will need to expire (which takes 5 mins) before it
3916 takes effect.
3917 This can be used to build general purpose proxies to any kind of back‐
3919 end that rclone supports.
3920 rclone serve sftp remote:path [flags]
3922 Options
3924 --addr string IPaddress:Port or :Port to bind server to. (default "localhost:2022")
3925 --auth-proxy string A program to use to create the backend from the auth.
3926 --authorized-keys string Authorized keys file (default "~/.ssh/authorized_keys")
3927 --dir-cache-time duration Time to cache directory entries for. (default 5m0s)
3928 --dir-perms FileMode Directory permissions (default 0777)
3929 --file-perms FileMode File permissions (default 0666)
3930 --gid uint32 Override the gid field set by the filesystem. (default 1000)
3931 -h, --help help for sftp
3932 --key string SSH private key file (leave blank to auto generate)
3933 --no-auth Allow connections with no authentication if set.
3934 --no-checksum Don't compare checksums on up/download.
3935 --no-modtime Don't read/write the modification time (can speed things up).
3936 --no-seek Don't allow seeking in files.
3937 --pass string Password for authentication.
3938 --poll-interval duration Time to wait between polling for changes. Must be smaller than dir-cache-time. Only on supported remotes. Set to 0 to disable. (default 1m0s)
3939 --read-only Mount read-only.
3940 --uid uint32 Override the uid field set by the filesystem. (default 1000)
3941 --umask int Override the permission bits set by the filesystem. (default 2)
3942 --user string User name for authentication.
3943 --vfs-cache-max-age duration Max age of objects in the cache. (default 1h0m0s)
3944 --vfs-cache-max-size SizeSuffix Max total size of objects in the cache. (default off)
3945 --vfs-cache-mode CacheMode Cache mode off|minimal|writes|full (default off)
3946 --vfs-cache-poll-interval duration Interval to poll the cache for stale objects. (default 1m0s)
3947 --vfs-case-insensitive If a file name not found, find a case insensitive match.
3948 --vfs-read-chunk-size SizeSuffix Read the source objects in chunks. (default 128M)
3949 --vfs-read-chunk-size-limit SizeSuffix If greater than --vfs-read-chunk-size, double the chunk size after each chunk read, until the limit is reached. 'off' is unlimited. (default off)
3950 See the global flags page (https://rclone.org/flags/) for global op‐
3952 tions not listed here.
3953 SEE ALSO
3955 · rclone serve (https://rclone.org/commands/rclone_serve/) - Serve a
3956 remote over a protocol.
3957 rclone serve webdav
3959 Serve remote:path over webdav.
3960 Synopsis
3962 rclone serve webdav implements a basic webdav server to serve the re‐
3963 mote over HTTP via the webdav protocol. This can be viewed with a web‐
3964 dav client, through a web browser, or you can make a remote of type
3965 webdav to read and write it.
3966 Webdav options
3968 –etag-hash
3969 This controls the ETag header. Without this flag the ETag will be
3970 based on the ModTime and Size of the object.
3971 If this flag is set to “auto” then rclone will choose the first sup‐
3973 ported hash on the backend or you can use a named hash such as “MD5” or
3974 “SHA-1”.
3975 Use “rclone hashsum” to see the full list.
3977 Server options
3979 Use –addr to specify which IP address and port the server should listen
3980 on, eg –addr 1.2.3.4:8000 or –addr :8080 to listen to all IPs. By de‐
3981 fault it only listens on localhost. You can use port :0 to let the OS
3982 choose an available port.
3983 If you set –addr to listen on a public or LAN accessible IP address
3985 then using Authentication is advised - see the next section for info.
3986 –server-read-timeout and –server-write-timeout can be used to control
3988 the timeouts on the server. Note that this is the total time for a
3989 transfer.
3990 –max-header-bytes controls the maximum number of bytes the server will
3992 accept in the HTTP header.
3993 –baseurl controls the URL prefix that rclone serves from. By default
3995 rclone will serve from the root. If you used –baseurl “/rclone” then
3996 rclone would serve from a URL starting with “/rclone/”. This is useful
3997 if you wish to proxy rclone serve. Rclone automatically inserts lead‐
3998 ing and trailing “/” on –baseurl, so –baseurl “rclone”, –baseurl
3999 “/rclone” and –baseurl “/rclone/” are all treated identically.
4000 Authentication
4002 By default this will serve files without needing a login.
4003 You can either use an htpasswd file which can take lots of users, or
4005 set a single username and password with the –user and –pass flags.
4006 Use –htpasswd /path/to/htpasswd to provide an htpasswd file. This is
4008 in standard apache format and supports MD5, SHA1 and BCrypt for basic
4009 authentication. Bcrypt is recommended.
4010 To create an htpasswd file:
4012 touch htpasswd
4014 htpasswd -B htpasswd user
4015 htpasswd -B htpasswd anotherUser
4016 The password file can be updated while rclone is running.
4018 Use –realm to set the authentication realm.
4020 SSL/TLS
4022 By default this will serve over http. If you want you can serve over
4023 https. You will need to supply the –cert and –key flags. If you wish
4024 to do client side certificate validation then you will need to supply
4025 –client-ca also.
4026 –cert should be a either a PEM encoded certificate or a concatenation
4028 of that with the CA certificate. –key should be the PEM encoded pri‐
4029 vate key and –client-ca should be the PEM encoded client certificate
4030 authority certificate.
4031 Directory Cache
4033 Using the --dir-cache-time flag, you can set how long a directory
4034 should be considered up to date and not refreshed from the backend.
4035 Changes made locally in the mount may appear immediately or invalidate
4036 the cache. However, changes done on the remote will only be picked up
4037 once the cache expires.
4038 Alternatively, you can send a SIGHUP signal to rclone for it to flush
4040 all directory caches, regardless of how old they are. Assuming only
4041 one rclone instance is running, you can reset the cache like this:
4042 kill -SIGHUP $(pidof rclone)
4044 If you configure rclone with a remote control (/rc) then you can use
4046 rclone rc to flush the whole directory cache:
4047 rclone rc vfs/forget
4049 Or individual files or directories:
4051 rclone rc vfs/forget file=path/to/file dir=path/to/dir
4053 File Buffering
4055 The --buffer-size flag determines the amount of memory, that will be
4056 used to buffer data in advance.
4057 Each open file descriptor will try to keep the specified amount of data
4059 in memory at all times. The buffered data is bound to one file de‐
4060 scriptor and won’t be shared between multiple open file descriptors of
4061 the same file.
4062 This flag is a upper limit for the used memory per file descriptor.
4064 The buffer will only use memory for data that is downloaded but not not
4065 yet read. If the buffer is empty, only a small amount of memory will
4066 be used. The maximum memory used by rclone for buffering can be up to
4067 --buffer-size * open files.
4068 File Caching
4070 These flags control the VFS file caching options. The VFS layer is
4071 used by rclone mount to make a cloud storage system work more like a
4072 normal file system.
4073 You’ll need to enable VFS caching if you want, for example, to read and
4075 write simultaneously to a file. See below for more details.
4076 Note that the VFS cache works in addition to the cache backend and you
4078 may find that you need one or the other or both.
4079 --cache-dir string Directory rclone will use for caching.
4081 --vfs-cache-max-age duration Max age of objects in the cache. (default 1h0m0s)
4082 --vfs-cache-mode string Cache mode off|minimal|writes|full (default "off")
4083 --vfs-cache-poll-interval duration Interval to poll the cache for stale objects. (default 1m0s)
4084 --vfs-cache-max-size int Max total size of objects in the cache. (default off)
4085 If run with -vv rclone will print the location of the file cache. The
4087 files are stored in the user cache file area which is OS dependent but
4088 can be controlled with --cache-dir or setting the appropriate environ‐
4089 ment variable.
4090 The cache has 4 different modes selected by --vfs-cache-mode. The
4092 higher the cache mode the more compatible rclone becomes at the cost of
4093 using disk space.
4094 Note that files are written back to the remote only when they are
4096 closed so if rclone is quit or dies with open files then these won’t
4097 get written back to the remote. However they will still be in the on
4098 disk cache.
4099 If using –vfs-cache-max-size note that the cache may exceed this size
4101 for two reasons. Firstly because it is only checked every
4102 –vfs-cache-poll-interval. Secondly because open files cannot be evict‐
4103 ed from the cache.
4104 –vfs-cache-mode off
4106 In this mode the cache will read directly from the remote and write di‐
4107 rectly to the remote without caching anything on disk.
4108 This will mean some operations are not possible
4110 · Files can’t be opened for both read AND write
4112 · Files opened for write can’t be seeked
4114 · Existing files opened for write must have O_TRUNC set
4116 · Files open for read with O_TRUNC will be opened write only
4118 · Files open for write only will behave as if O_TRUNC was supplied
4120 · Open modes O_APPEND, O_TRUNC are ignored
4122 · If an upload fails it can’t be retried
4124 –vfs-cache-mode minimal
4126 This is very similar to “off” except that files opened for read AND
4127 write will be buffered to disks. This means that files opened for
4128 write will be a lot more compatible, but uses the minimal disk space.
4129 These operations are not possible
4131 · Files opened for write only can’t be seeked
4133 · Existing files opened for write must have O_TRUNC set
4135 · Files opened for write only will ignore O_APPEND, O_TRUNC
4137 · If an upload fails it can’t be retried
4139 –vfs-cache-mode writes
4141 In this mode files opened for read only are still read directly from
4142 the remote, write only and read/write files are buffered to disk first.
4143 This mode should support all normal file system operations.
4145 If an upload fails it will be retried up to –low-level-retries times.
4147 –vfs-cache-mode full
4149 In this mode all reads and writes are buffered to and from disk. When
4150 a file is opened for read it will be downloaded in its entirety first.
4151 This may be appropriate for your needs, or you may prefer to look at
4153 the cache backend which does a much more sophisticated job of caching,
4154 including caching directory hierarchies and chunks of files.
4155 In this mode, unlike the others, when a file is written to the disk, it
4157 will be kept on the disk after it is written to the remote. It will be
4158 purged on a schedule according to --vfs-cache-max-age.
4159 This mode should support all normal file system operations.
4161 If an upload or download fails it will be retried up to –low-level-re‐
4163 tries times.
4164 Auth Proxy
4166 If you supply the parameter --auth-proxy /path/to/program then rclone
4167 will use that program to generate backends on the fly which then are
4168 used to authenticate incoming requests. This uses a simple JSON based
4169 protocl with input on STDIN and output on STDOUT.
4170 There is an example program bin/test_proxy.py
4172 (https://github.com/rclone/rclone/blob/master/test_proxy.py) in the
4173 rclone source code.
4174 The program’s job is to take a user and pass on the input and turn
4176 those into the config for a backend on STDOUT in JSON format. This
4177 config will have any default parameters for the backend added, but it
4178 won’t use configuration from environment variables or command line op‐
4179 tions - it is the job of the proxy program to make a complete config.
4180 This config generated must have this extra parameter - _root - root to
4182 use for the backend
4183 And it may have this parameter - _obscure - comma separated strings for
4185 parameters to obscure
4186 For example the program might take this on STDIN
4188 {
4190 "user": "me",
4191 "pass": "mypassword"
4192 }
4193 And return this on STDOUT
4195 {
4197 "type": "sftp",
4198 "_root": "",
4199 "_obscure": "pass",
4200 "user": "me",
4201 "pass": "mypassword",
4202 "host": "sftp.example.com"
4203 }
4204 This would mean that an SFTP backend would be created on the fly for
4206 the user and pass returned in the output to the host given. Note that
4207 since _obscure is set to pass, rclone will obscure the pass parameter
4208 before creating the backend (which is required for sftp backends).
4209 The progam can manipulate the supplied user in any way, for example to
4211 make proxy to many different sftp backends, you could make the user be
4212 user@example.com and then set the host to example.com in the output and
4213 the user to user. For security you’d probably want to restrict the
4214 host to a limited list.
4215 Note that an internal cache is keyed on user so only use that for con‐
4217 figuration, don’t use pass. This also means that if a user’s password
4218 is changed the cache will need to expire (which takes 5 mins) before it
4219 takes effect.
4220 This can be used to build general purpose proxies to any kind of back‐
4222 end that rclone supports.
4223 rclone serve webdav remote:path [flags]
4225 Options
4227 --addr string IPaddress:Port or :Port to bind server to. (default "localhost:8080")
4228 --auth-proxy string A program to use to create the backend from the auth.
4229 --baseurl string Prefix for URLs - leave blank for root.
4230 --cert string SSL PEM key (concatenation of certificate and CA certificate)
4231 --client-ca string Client certificate authority to verify clients with
4232 --dir-cache-time duration Time to cache directory entries for. (default 5m0s)
4233 --dir-perms FileMode Directory permissions (default 0777)
4234 --disable-dir-list Disable HTML directory list on GET request for a directory
4235 --etag-hash string Which hash to use for the ETag, or auto or blank for off
4236 --file-perms FileMode File permissions (default 0666)
4237 --gid uint32 Override the gid field set by the filesystem. (default 1000)
4238 -h, --help help for webdav
4239 --htpasswd string htpasswd file - if not provided no authentication is done
4240 --key string SSL PEM Private key
4241 --max-header-bytes int Maximum size of request header (default 4096)
4242 --no-checksum Don't compare checksums on up/download.
4243 --no-modtime Don't read/write the modification time (can speed things up).
4244 --no-seek Don't allow seeking in files.
4245 --pass string Password for authentication.
4246 --poll-interval duration Time to wait between polling for changes. Must be smaller than dir-cache-time. Only on supported remotes. Set to 0 to disable. (default 1m0s)
4247 --read-only Mount read-only.
4248 --realm string realm for authentication (default "rclone")
4249 --server-read-timeout duration Timeout for server reading data (default 1h0m0s)
4250 --server-write-timeout duration Timeout for server writing data (default 1h0m0s)
4251 --uid uint32 Override the uid field set by the filesystem. (default 1000)
4252 --umask int Override the permission bits set by the filesystem. (default 2)
4253 --user string User name for authentication.
4254 --vfs-cache-max-age duration Max age of objects in the cache. (default 1h0m0s)
4255 --vfs-cache-max-size SizeSuffix Max total size of objects in the cache. (default off)
4256 --vfs-cache-mode CacheMode Cache mode off|minimal|writes|full (default off)
4257 --vfs-cache-poll-interval duration Interval to poll the cache for stale objects. (default 1m0s)
4258 --vfs-case-insensitive If a file name not found, find a case insensitive match.
4259 --vfs-read-chunk-size SizeSuffix Read the source objects in chunks. (default 128M)
4260 --vfs-read-chunk-size-limit SizeSuffix If greater than --vfs-read-chunk-size, double the chunk size after each chunk read, until the limit is reached. 'off' is unlimited. (default off)
4261 See the global flags page (https://rclone.org/flags/) for global op‐
4263 tions not listed here.
4264 SEE ALSO
4266 · rclone serve (https://rclone.org/commands/rclone_serve/) - Serve a
4267 remote over a protocol.
4268 rclone settier
4270 Changes storage class/tier of objects in remote.
4271 Synopsis
4273 rclone settier changes storage tier or class at remote if supported.
4274 Few cloud storage services provides different storage classes on ob‐
4275 jects, for example AWS S3 and Glacier, Azure Blob storage - Hot, Cool
4276 and Archive, Google Cloud Storage, Regional Storage, Nearline, Coldline
4277 etc.
4278 Note that, certain tier changes make objects not available to access
4280 immediately. For example tiering to archive in azure blob storage
4281 makes objects in frozen state, user can restore by setting tier to
4282 Hot/Cool, similarly S3 to Glacier makes object inaccessible.true
4283 You can use it to tier single object
4285 rclone settier Cool remote:path/file
4287 Or use rclone filters to set tier on only specific files
4289 rclone --include "*.txt" settier Hot remote:path/dir
4291 Or just provide remote directory and all files in directory will be
4293 tiered
4294 rclone settier tier remote:path/dir
4296 rclone settier tier remote:path [flags]
4298 Options
4300 -h, --help help for settier
4301 See the global flags page (https://rclone.org/flags/) for global op‐
4303 tions not listed here.
4304 SEE ALSO
4306 · rclone (https://rclone.org/commands/rclone/) - Show help for rclone
4307 commands, flags and backends.
4308 rclone touch
4310 Create new file or change file modification time.
4311 Synopsis
4313 Create new file or change file modification time.
4314 rclone touch remote:path [flags]
4316 Options
4318 -h, --help help for touch
4319 -C, --no-create Do not create the file if it does not exist.
4320 -t, --timestamp string Change the modification times to the specified time instead of the current time of day. The argument is of the form 'YYMMDD' (ex. 17.10.30) or 'YYYY-MM-DDTHH:MM:SS' (ex. 2006-01-02T15:04:05)
4321 See the global flags page (https://rclone.org/flags/) for global op‐
4323 tions not listed here.
4324 SEE ALSO
4326 · rclone (https://rclone.org/commands/rclone/) - Show help for rclone
4327 commands, flags and backends.
4328 rclone tree
4330 List the contents of the remote in a tree like fashion.
4331 Synopsis
4333 rclone tree lists the contents of a remote in a similar way to the unix
4334 tree command.
4335 For example
4337 $ rclone tree remote:path
4339 /
4340 ├── file1
4341 ├── file2
4342 ├── file3
4343 └── subdir
4344 ├── file4
4345 └── file5
4346 1 directories, 5 files
4348 You can use any of the filtering options with the tree command (eg –in‐
4350 clude and –exclude). You can also use –fast-list.
4351 The tree command has many options for controlling the listing which are
4353 compatible with the tree command. Note that not all of them have short
4354 options as they conflict with rclone’s short options.
4355 rclone tree remote:path [flags]
4357 Options
4359 -a, --all All files are listed (list . files too).
4360 -C, --color Turn colorization on always.
4361 -d, --dirs-only List directories only.
4362 --dirsfirst List directories before files (-U disables).
4363 --full-path Print the full path prefix for each file.
4364 -h, --help help for tree
4365 --human Print the size in a more human readable way.
4366 --level int Descend only level directories deep.
4367 -D, --modtime Print the date of last modification.
4368 -i, --noindent Don't print indentation lines.
4369 --noreport Turn off file/directory count at end of tree listing.
4370 -o, --output string Output to file instead of stdout.
4371 -p, --protections Print the protections for each file.
4372 -Q, --quote Quote filenames with double quotes.
4373 -s, --size Print the size in bytes of each file.
4374 --sort string Select sort: name,version,size,mtime,ctime.
4375 --sort-ctime Sort files by last status change time.
4376 -t, --sort-modtime Sort files by last modification time.
4377 -r, --sort-reverse Reverse the order of the sort.
4378 -U, --unsorted Leave files unsorted.
4379 --version Sort files alphanumerically by version.
4380 See the global flags page (https://rclone.org/flags/) for global op‐
4382 tions not listed here.
4383 SEE ALSO
4385 · rclone (https://rclone.org/commands/rclone/) - Show help for rclone
4386 commands, flags and backends.
4387 Copying single files
4389 rclone normally syncs or copies directories. However, if the source
4390 remote points to a file, rclone will just copy that file. The destina‐
4391 tion remote must point to a directory - rclone will give the error
4392 Failed to create file system for "remote:file": is a file not a direc‐
4393 tory if it isn’t.
4394 For example, suppose you have a remote with a file in called test.jpg,
4396 then you could copy just that file like this
4397 rclone copy remote:test.jpg /tmp/download
4399 The file test.jpg will be placed inside /tmp/download.
4401 This is equivalent to specifying
4403 rclone copy --files-from /tmp/files remote: /tmp/download
4405 Where /tmp/files contains the single line
4407 test.jpg
4409 It is recommended to use copy when copying individual files, not sync.
4411 They have pretty much the same effect but copy will use a lot less mem‐
4412 ory.
4413 Syntax of remote paths
4415 The syntax of the paths passed to the rclone command are as follows.
4416 /path/to/dir
4418 This refers to the local file system.
4419 On Windows only \ may be used instead of / in local paths only, non lo‐
4421 cal paths must use /.
4422 These paths needn’t start with a leading / - if they don’t then they
4424 will be relative to the current directory.
4425 remote:path/to/dir
4427 This refers to a directory path/to/dir on remote: as defined in the
4428 config file (configured with rclone config).
4429 remote:/path/to/dir
4431 On most backends this is refers to the same directory as re‐
4432 mote:path/to/dir and that format should be preferred. On a very small
4433 number of remotes (FTP, SFTP, Dropbox for business) this will refer to
4434 a different directory. On these, paths without a leading / will refer
4435 to your “home” directory and paths with a leading / will refer to the
4436 root.
4437 :backend:path/to/dir
4439 This is an advanced form for creating remotes on the fly. backend
4440 should be the name or prefix of a backend (the type in the config file)
4441 and all the configuration for the backend should be provided on the
4442 command line (or in environment variables).
4443 Here are some examples:
4445 rclone lsd --http-url https://pub.rclone.org :http:
4447 To list all the directories in the root of https://pub.rclone.org/.
4449 rclone lsf --http-url https://example.com :http:path/to/dir
4451 To list files and directories in https://example.com/path/to/dir/
4453 rclone copy --http-url https://example.com :http:path/to/dir /tmp/dir
4455 To copy files and directories in https://example.com/path/to/dir to
4457 /tmp/dir.
4458 rclone copy --sftp-host example.com :sftp:path/to/dir /tmp/dir
4460 To copy files and directories from example.com in the relative directo‐
4462 ry path/to/dir to /tmp/dir using sftp.
4463 Quoting and the shell
4465 When you are typing commands to your computer you are using something
4466 called the command line shell. This interprets various characters in
4467 an OS specific way.
4468 Here are some gotchas which may help users unfamiliar with the shell
4470 rules
4471 Linux / OSX
4473 If your names have spaces or shell metacharacters (eg *, ?, $, ', "
4474 etc) then you must quote them. Use single quotes ' by default.
4475 rclone copy 'Important files?' remote:backup
4477 If you want to send a ' you will need to use ", eg
4479 rclone copy "O'Reilly Reviews" remote:backup
4481 The rules for quoting metacharacters are complicated and if you want
4483 the full details you’ll have to consult the manual page for your shell.
4484 Windows
4486 If your names have spaces in you need to put them in ", eg
4487 rclone copy "E:\folder name\folder name\folder name" remote:backup
4489 If you are using the root directory on its own then don’t quote it (see
4491 #464 (https://github.com/rclone/rclone/issues/464) for why), eg
4492 rclone copy E:\ remote:backup
4494 Copying files or directories with : in the names
4496 rclone uses : to mark a remote name. This is, however, a valid file‐
4497 name component in non-Windows OSes. The remote name parser will only
4498 search for a : up to the first / so if you need to act on a file or di‐
4499 rectory like this then use the full path starting with a /, or use ./
4500 as a current directory prefix.
4501 So to sync a directory called sync:me to a remote called remote: use
4503 rclone sync ./sync:me remote:path
4505 or
4507 rclone sync /full/path/to/sync:me remote:path
4509 Server Side Copy
4511 Most remotes (but not all - see the overview (/overview/#optional-fea‐
4512 tures)) support server side copy.
4513 This means if you want to copy one folder to another then rclone won’t
4515 download all the files and re-upload them; it will instruct the server
4516 to copy them in place.
4517 Eg
4519 rclone copy s3:oldbucket s3:newbucket
4521 Will copy the contents of oldbucket to newbucket without downloading
4523 and re-uploading.
4524 Remotes which don’t support server side copy will download and re-up‐
4526 load in this case.
4527 Server side copies are used with sync and copy and will be identified
4529 in the log when using the -v flag. The move command may also use them
4530 if remote doesn’t support server side move directly. This is done by
4531 issuing a server side copy then a delete which is much quicker than a
4532 download and re-upload.
4533 Server side copies will only be attempted if the remote names are the
4535 same.
4536 This can be used when scripting to make aged backups efficiently, eg
4538 rclone sync remote:current-backup remote:previous-backup
4540 rclone sync /path/to/files remote:current-backup
4541 Options
4543 Rclone has a number of options to control its behaviour.
4544 Options that take parameters can have the values passed in two ways,
4546 --option=value or --option value. However boolean (true/false) options
4547 behave slightly differently to the other options in that --boolean sets
4548 the option to true and the absence of the flag sets it to false. It is
4549 also possible to specify --boolean=false or --boolean=true. Note that
4550 --boolean false is not valid - this is parsed as --boolean and the
4551 false is parsed as an extra command line argument for rclone.
4552 Options which use TIME use the go time parser. A duration string is a
4554 possibly signed sequence of decimal numbers, each with optional frac‐
4555 tion and a unit suffix, such as “300ms”, “-1.5h” or “2h45m”. Valid
4556 time units are “ns”, “us” (or “µs”), “ms”, “s”, “m”, “h”.
4557 Options which use SIZE use kByte by default. However, a suffix of b
4559 for bytes, k for kBytes, M for MBytes, G for GBytes, T for TBytes and P
4560 for PBytes may be used. These are the binary units, eg 1, 2**10,
4561 2**20, 2**30 respectively.
4562 –backup-dir=DIR
4564 When using sync, copy or move any files which would have been overwrit‐
4565 ten or deleted are moved in their original hierarchy into this directo‐
4566 ry.
4567 If --suffix is set, then the moved files will have the suffix added to
4569 them. If there is a file with the same path (after the suffix has been
4570 added) in DIR, then it will be overwritten.
4571 The remote in use must support server side move or copy and you must
4573 use the same remote as the destination of the sync. The backup direc‐
4574 tory must not overlap the destination directory.
4575 For example
4577 rclone sync /path/to/local remote:current --backup-dir remote:old
4579 will sync /path/to/local to remote:current, but for any files which
4581 would have been updated or deleted will be stored in remote:old.
4582 If running rclone from a script you might want to use today’s date as
4584 the directory name passed to --backup-dir to store the old files, or
4585 you might want to pass --suffix with today’s date.
4586 See --compare-dest and --copy-dest.
4588 –bind string
4590 Local address to bind to for outgoing connections. This can be an IPv4
4591 address (1.2.3.4), an IPv6 address (1234::789A) or host name. If the
4592 host name doesn’t resolve or resolves to more than one IP address it
4593 will give an error.
4594 –bwlimit=BANDWIDTH_SPEC
4596 This option controls the bandwidth limit. Limits can be specified in
4597 two ways: As a single limit, or as a timetable.
4598 Single limits last for the duration of the session. To use a single
4600 limit, specify the desired bandwidth in kBytes/s, or use a suffix
4601 b|k|M|G. The default is 0 which means to not limit bandwidth.
4602 For example, to limit bandwidth usage to 10 MBytes/s use --bwlimit 10M
4604 It is also possible to specify a “timetable” of limits, which will
4606 cause certain limits to be applied at certain times. To specify a
4607 timetable, format your entries as “WEEKDAY-HH:MM,BANDWIDTH WEEK‐
4608 DAY-HH:MM,BANDWIDTH...” where: WEEKDAY is optional element. It could
4609 be written as whole world or only using 3 first characters. HH:MM is
4610 an hour from 00:00 to 23:59.
4611 An example of a typical timetable to avoid link saturation during day‐
4613 time working hours could be:
4614 --bwlimit "08:00,512 12:00,10M 13:00,512 18:00,30M 23:00,off"
4616 In this example, the transfer bandwidth will be every day set to
4618 512kBytes/sec at 8am. At noon, it will raise to 10Mbytes/s, and drop
4619 back to 512kBytes/sec at 1pm. At 6pm, the bandwidth limit will be set
4620 to 30MBytes/s, and at 11pm it will be completely disabled (full speed).
4621 Anything between 11pm and 8am will remain unlimited.
4622 An example of timetable with WEEKDAY could be:
4624 --bwlimit "Mon-00:00,512 Fri-23:59,10M Sat-10:00,1M Sun-20:00,off"
4626 It mean that, the transfer bandwidth will be set to 512kBytes/sec on
4628 Monday. It will raise to 10Mbytes/s before the end of Friday. At
4629 10:00 on Sunday it will be set to 1Mbyte/s. From 20:00 at Sunday will
4630 be unlimited.
4631 Timeslots without weekday are extended to whole week. So this one ex‐
4633 ample:
4634 --bwlimit "Mon-00:00,512 12:00,1M Sun-20:00,off"
4636 Is equal to this:
4638 --bwlimit "Mon-00:00,512Mon-12:00,1M Tue-12:00,1M Wed-12:00,1M
4640 Thu-12:00,1M Fri-12:00,1M Sat-12:00,1M Sun-12:00,1M Sun-20:00,off"
4641 Bandwidth limits only apply to the data transfer. They don’t apply to
4643 the bandwidth of the directory listings etc.
4644 Note that the units are Bytes/s, not Bits/s. Typically connections are
4646 measured in Bits/s - to convert divide by 8. For example, let’s say
4647 you have a 10 Mbit/s connection and you wish rclone to use half of it -
4648 5 Mbit/s. This is 5/8 = 0.625MByte/s so you would use a --bwlimit
4649 0.625M parameter for rclone.
4650 On Unix systems (Linux, MacOS, ...) the bandwidth limiter can be tog‐
4652 gled by sending a SIGUSR2 signal to rclone. This allows to remove the
4653 limitations of a long running rclone transfer and to restore it back to
4654 the value specified with --bwlimit quickly when needed. Assuming there
4655 is only one rclone instance running, you can toggle the limiter like
4656 this:
4657 kill -SIGUSR2 $(pidof rclone)
4659 If you configure rclone with a remote control (/rc) then you can use
4661 change the bwlimit dynamically:
4662 rclone rc core/bwlimit rate=1M
4664 –buffer-size=SIZE
4666 Use this sized buffer to speed up file transfers. Each --transfer will
4667 use this much memory for buffering.
4668 When using mount or cmount each open file descriptor will use this much
4670 memory for buffering. See the mount (/commands/rclone_mount/#file-
4671 buffering) documentation for more details.
4672 Set to 0 to disable the buffering for the minimum memory usage.
4674 Note that the memory allocation of the buffers is influenced by the
4676 –use-mmap flag.
4677 –checkers=N
4679 The number of checkers to run in parallel. Checkers do the equality
4680 checking of files during a sync. For some storage systems (eg S3,
4681 Swift, Dropbox) this can take a significant amount of time so they are
4682 run in parallel.
4683 The default is to run 8 checkers in parallel.
4685 -c, –checksum
4687 Normally rclone will look at modification time and size of files to see
4688 if they are equal. If you set this flag then rclone will check the
4689 file hash and size to determine if files are equal.
4690 This is useful when the remote doesn’t support setting modified time
4692 and a more accurate sync is desired than just checking the file size.
4693 This is very useful when transferring between remotes which store the
4695 same hash type on the object, eg Drive and Swift. For details of which
4696 remotes support which hash type see the table in the overview section
4697 (https://rclone.org/overview/).
4698 Eg rclone --checksum sync s3:/bucket swift:/bucket would run much
4700 quicker than without the --checksum flag.
4701 When using this flag, rclone won’t update mtimes of remote files if
4703 they are incorrect as it would normally.
4704 –compare-dest=DIR
4706 When using sync, copy or move DIR is checked in addition to the desti‐
4707 nation for files. If a file identical to the source is found that file
4708 is NOT copied from source. This is useful to copy just files that have
4709 changed since the last backup.
4710 You must use the same remote as the destination of the sync. The com‐
4712 pare directory must not overlap the destination directory.
4713 See --copy-dest and --backup-dir.
4715 –config=CONFIG_FILE
4717 Specify the location of the rclone config file.
4718 Normally the config file is in your home directory as a file called
4720 .config/rclone/rclone.conf (or .rclone.conf if created with an older
4721 version). If $XDG_CONFIG_HOME is set it will be at $XDG_CON‐
4722 FIG_HOME/rclone/rclone.conf.
4723 If there is a file rclone.conf in the same directory as the rclone exe‐
4725 cutable it will be preferred. This file must be created manually for
4726 Rclone to use it, it will never be created automatically.
4727 If you run rclone config file you will see where the default location
4729 is for you.
4730 Use this flag to override the config location, eg rclone --config=".my‐
4732 config" .config.
4733 –contimeout=TIME
4735 Set the connection timeout. This should be in go time format which
4736 looks like 5s for 5 seconds, 10m for 10 minutes, or 3h30m.
4737 The connection timeout is the amount of time rclone will wait for a
4739 connection to go through to a remote object storage system. It is 1m
4740 by default.
4741 –copy-dest=DIR
4743 When using sync, copy or move DIR is checked in addition to the desti‐
4744 nation for files. If a file identical to the source is found that file
4745 is server side copied from DIR to the destination. This is useful for
4746 incremental backup.
4747 The remote in use must support server side copy and you must use the
4749 same remote as the destination of the sync. The compare directory must
4750 not overlap the destination directory.
4751 See --compare-dest and --backup-dir.
4753 –dedupe-mode MODE
4755 Mode to run dedupe command in. One of interactive, skip, first, new‐
4756 est, oldest, rename. The default is interactive. See the dedupe com‐
4757 mand for more information as to what these options mean.
4758 –disable FEATURE,FEATURE,...
4760 This disables a comma separated list of optional features. For example
4761 to disable server side move and server side copy use:
4762 --disable move,copy
4764 The features can be put in in any case.
4766 To see a list of which features can be disabled use:
4768 --disable help
4770 See the overview features (/overview/#features) and optional features
4772 (/overview/#optional-features) to get an idea of which feature does
4773 what.
4774 This flag can be useful for debugging and in exceptional circumstances
4776 (eg Google Drive limiting the total volume of Server Side Copies to
4777 100GB/day).
4778 -n, –dry-run
4780 Do a trial run with no permanent changes. Use this to see what rclone
4781 would do without actually doing it. Useful when setting up the sync
4782 command which deletes files in the destination.
4783 –expect-continue-timeout=TIME
4785 This specifies the amount of time to wait for a server’s first response
4786 headers after fully writing the request headers if the request has an
4787 “Expect: 100-continue” header. Not all backends support using this.
4788 Zero means no timeout and causes the body to be sent immediately, with‐
4790 out waiting for the server to approve. This time does not include the
4791 time to send the request header.
4792 The default is 1s. Set to 0 to disable.
4794 –ignore-case-sync
4796 Using this option will cause rclone to ignore the case of the files
4797 when synchronizing so files will not be copied/synced when the existing
4798 filenames are the same, even if the casing is different.
4799 –ignore-checksum
4801 Normally rclone will check that the checksums of transferred files
4802 match, and give an error “corrupted on transfer” if they don’t.
4803 You can use this option to skip that check. You should only use it if
4805 you have had the “corrupted on transfer” error message and you are sure
4806 you might want to transfer potentially corrupted data.
4807 –ignore-existing
4809 Using this option will make rclone unconditionally skip all files that
4810 exist on the destination, no matter the content of these files.
4811 While this isn’t a generally recommended option, it can be useful in
4813 cases where your files change due to encryption. However, it cannot
4814 correct partial transfers in case a transfer was interrupted.
4815 –ignore-size
4817 Normally rclone will look at modification time and size of files to see
4818 if they are equal. If you set this flag then rclone will check only
4819 the modification time. If --checksum is set then it only checks the
4820 checksum.
4821 It will also cause rclone to skip verifying the sizes are the same af‐
4823 ter transfer.
4824 This can be useful for transferring files to and from OneDrive which
4826 occasionally misreports the size of image files (see #399
4827 (https://github.com/rclone/rclone/issues/399) for more info).
4828 -I, –ignore-times
4830 Using this option will cause rclone to unconditionally upload all files
4831 regardless of the state of files on the destination.
4832 Normally rclone would skip any files that have the same modification
4834 time and are the same size (or have the same checksum if using --check‐
4835 sum).
4836 –immutable
4838 Treat source and destination files as immutable and disallow modifica‐
4839 tion.
4840 With this option set, files will be created and deleted as requested,
4842 but existing files will never be updated. If an existing file does not
4843 match between the source and destination, rclone will give the error
4844 Source and destination exist but do not match: immutable file modified.
4845 Note that only commands which transfer files (e.g. sync, copy, move)
4847 are affected by this behavior, and only modification is disallowed.
4848 Files may still be deleted explicitly (e.g. delete, purge) or implicit‐
4849 ly (e.g. sync, move). Use copy --immutable if it is desired to avoid
4850 deletion as well as modification.
4851 This can be useful as an additional layer of protection for immutable
4853 or append-only data sets (notably backup archives), where modification
4854 implies corruption and should not be propagated.
4855 –leave-root
4857 During rmdirs it will not remove root directory, even if it’s empty.
4858 –log-file=FILE
4860 Log all of rclone’s output to FILE. This is not active by default.
4861 This can be useful for tracking down problems with syncs in combination
4862 with the -v flag. See the Logging section for more info.
4863 Note that if you are using the logrotate program to manage rclone’s
4865 logs, then you should use the copytruncate option as rclone doesn’t
4866 have a signal to rotate logs.
4867 –log-format LIST
4869 Comma separated list of log format options. date, time, microseconds,
4870 longfile, shortfile, UTC. The default is “date,time”.
4871 –log-level LEVEL
4873 This sets the log level for rclone. The default log level is NOTICE.
4874 DEBUG is equivalent to -vv. It outputs lots of debug info - useful for
4876 bug reports and really finding out what rclone is doing.
4877 INFO is equivalent to -v. It outputs information about each transfer
4879 and prints stats once a minute by default.
4880 NOTICE is the default log level if no logging flags are supplied. It
4882 outputs very little when things are working normally. It outputs warn‐
4883 ings and significant events.
4884 ERROR is equivalent to -q. It only outputs error messages.
4886 –use-json-log
4888 This switches the log format to JSON for rclone. The fields of json
4889 log are level, msg, source, time.
4890 –low-level-retries NUMBER
4892 This controls the number of low level retries rclone does.
4893 A low level retry is used to retry a failing operation - typically one
4895 HTTP request. This might be uploading a chunk of a big file for exam‐
4896 ple. You will see low level retries in the log with the -v flag.
4897 This shouldn’t need to be changed from the default in normal opera‐
4899 tions. However, if you get a lot of low level retries you may wish to
4900 reduce the value so rclone moves on to a high level retry (see the
4901 --retries flag) quicker.
4902 Disable low level retries with --low-level-retries 1.
4904 –max-backlog=N
4906 This is the maximum allowable backlog of files in a sync/copy/move
4907 queued for being checked or transferred.
4908 This can be set arbitrarily large. It will only use memory when the
4910 queue is in use. Note that it will use in the order of N kB of memory
4911 when the backlog is in use.
4912 Setting this large allows rclone to calculate how many files are pend‐
4914 ing more accurately, give a more accurate estimated finish time and
4915 make --order-by work more accurately.
4916 Setting this small will make rclone more synchronous to the listings of
4918 the remote which may be desirable.
4919 –max-delete=N
4921 This tells rclone not to delete more than N files. If that limit is
4922 exceeded then a fatal error will be generated and rclone will stop the
4923 operation in progress.
4924 –max-depth=N
4926 This modifies the recursion depth for all the commands except purge.
4927 So if you do rclone --max-depth 1 ls remote:path you will see only the
4929 files in the top level directory. Using --max-depth 2 means you will
4930 see all the files in first two directory levels and so on.
4931 For historical reasons the lsd command defaults to using a --max-depth
4933 of 1 - you can override this with the command line flag.
4934 You can use this command to disable recursion (with --max-depth 1).
4936 Note that if you use this with sync and --delete-excluded the files not
4938 recursed through are considered excluded and will be deleted on the
4939 destination. Test first with --dry-run if you are not sure what will
4940 happen.
4941 –max-duration=TIME
4943 Rclone will stop scheduling new transfers when it has run for the dura‐
4944 tion specified.
4945 Defaults to off.
4947 When the limit is reached any existing transfers will complete.
4949 Rclone won’t exit with an error if the transfer limit is reached.
4951 –max-transfer=SIZE
4953 Rclone will stop transferring when it has reached the size specified.
4954 Defaults to off.
4955 When the limit is reached all transfers will stop immediately.
4957 Rclone will exit with exit code 8 if the transfer limit is reached.
4959 –modify-window=TIME
4961 When checking whether a file has been modified, this is the maximum al‐
4962 lowed time difference that a file can have and still be considered
4963 equivalent.
4964 The default is 1ns unless this is overridden by a remote. For example
4966 OS X only stores modification times to the nearest second so if you are
4967 reading and writing to an OS X filing system this will be 1s by de‐
4968 fault.
4969 This command line flag allows you to override that computed default.
4971 –multi-thread-cutoff=SIZE
4973 When downloading files to the local backend above this size, rclone
4974 will use multiple threads to download the file. (default 250M)
4975 Rclone preallocates the file (using fallocate(FALLOC_FL_KEEP_SIZE) on
4977 unix or NTSetInformationFile on Windows both of which takes no time)
4978 then each thread writes directly into the file at the correct place.
4979 This means that rclone won’t create fragmented or sparse files and
4980 there won’t be any assembly time at the end of the transfer.
4981 The number of threads used to dowload is controlled by --mul‐
4983 ti-thread-streams.
4984 Use -vv if you wish to see info about the threads.
4986 This will work with the sync/copy/move commands and friends copy‐
4988 to/moveto. Multi thread downloads will be used with rclone mount and
4989 rclone serve if --vfs-cache-mode is set to writes or above.
4990 NB that this only works for a local destination but will work with any
4992 source.
4993 NB that multi thread copies are disabled for local to local copies as
4995 they are faster without unless --multi-thread-streams is set explicit‐
4996 ly.
4997 –multi-thread-streams=N
4999 When using multi thread downloads (see above --multi-thread-cutoff)
5000 this sets the maximum number of streams to use. Set to 0 to disable
5001 multi thread downloads. (Default 4)
5002 Exactly how many streams rclone uses for the download depends on the
5004 size of the file. To calculate the number of download streams Rclone
5005 divides the size of the file by the --multi-thread-cutoff and rounds
5006 up, up to the maximum set with --multi-thread-streams.
5007 So if --multi-thread-cutoff 250MB and --multi-thread-streams 4 are in
5009 effect (the defaults):
5010 · 0MB.250MB files will be downloaded with 1 stream
5012 · 250MB..500MB files will be downloaded with 2 streams
5014 · 500MB..750MB files will be downloaded with 3 streams
5016 · 750MB+ files will be downloaded with 4 streams
5018 –no-check-dest
5020 The --no-check-dest can be used with move or copy and it causes rclone
5021 not to check the destination at all when copying files.
5022 This means that:
5024 · the destination is not listed minimising the API calls
5026 · files are always transferred
5028 · this can cause duplicates on remotes which allow it (eg Google Drive)
5030 · --retries 1 is recommended otherwise you’ll transfer everything again
5032 on a retry
5033 This flag is useful to minimise the transactions if you know that none
5035 of the files are on the destination.
5036 This is a specialized flag which should be ignored by most users!
5038 –no-gzip-encoding
5040 Don’t set Accept-Encoding: gzip. This means that rclone won’t ask the
5041 server for compressed files automatically. Useful if you’ve set the
5042 server to return files with Content-Encoding: gzip but you uploaded
5043 compressed files.
5044 There is no need to set this in normal operation, and doing so will de‐
5046 crease the network transfer efficiency of rclone.
5047 –no-traverse
5049 The --no-traverse flag controls whether the destination file system is
5050 traversed when using the copy or move commands. --no-traverse is not
5051 compatible with sync and will be ignored if you supply it with sync.
5052 If you are only copying a small number of files (or are filtering most
5054 of the files) and/or have a large number of files on the destination
5055 then --no-traverse will stop rclone listing the destination and save
5056 time.
5057 However, if you are copying a large number of files, especially if you
5059 are doing a copy where lots of the files under consideration haven’t
5060 changed and won’t need copying then you shouldn’t use --no-traverse.
5061 See rclone copy (https://rclone.org/commands/rclone_copy/) for an exam‐
5063 ple of how to use it.
5064 –no-update-modtime
5066 When using this flag, rclone won’t update modification times of remote
5067 files if they are incorrect as it would normally.
5068 This can be used if the remote is being synced with another tool also
5070 (eg the Google Drive client).
5071 –order-by string
5073 The --order-by flag controls the order in which files in the backlog
5074 are processed in rclone sync, rclone copy and rclone move.
5075 The order by string is constructed like this. The first part describes
5077 what aspect is being measured:
5078 · size - order by the size of the files
5080 · name - order by the full path of the files
5082 · modtime - order by the modification date of the files
5084 This can have a modifier appended with a comma:
5086 · ascending or asc - order so that the smallest (or oldest) is pro‐
5088 cessed first
5089 · descending or desc - order so that the largest (or newest) is pro‐
5091 cessed first
5092 If no modifier is supplied then the order is ascending.
5094 For example
5096 · --order-by size,desc - send the largest files first
5098 · --order-by modtime,ascending - send the oldest files first
5100 · --order-by name - send the files with alphabetically by path first
5102 If the --order-by flag is not supplied or it is supplied with an empty
5104 string then the default ordering will be used which is as scanned.
5105 With --checkers 1 this is mostly alphabetical, however with the default
5106 --checkers 8 it is somewhat random.
5107 Limitations
5109 The --order-by flag does not do a separate pass over the data. This
5110 means that it may transfer some files out of the order specified if
5111 · there are no files in the backlog or the source has not been fully
5113 scanned yet
5114 · there are more than –max-backlog files in the backlog
5116 Rclone will do its best to transfer the best file it has so in practice
5118 this should not cause a problem. Think of --order-by as being more of
5119 a best efforts flag rather than a perfect ordering.
5120 –password-command SpaceSepList
5122 This flag supplies a program which should supply the config password
5123 when run. This is an alternative to rclone prompting for the password
5124 or setting the RCLONE_CONFIG_PASS variable.
5125 The argument to this should be a command with a space separated list of
5127 arguments. If one of the arguments has a space in then enclose it in
5128 ", if you want a literal " in an argument then enclose the argument in
5129 " and double the ". See CSV encoding (https://godoc.org/encoding/csv)
5130 for more info.
5131 Eg
5133 --password-command echo hello
5135 --password-command echo "hello with space"
5136 --password-command echo "hello with ""quotes"" and space"
5137 See the Configuration Encryption for more info.
5139 -P, –progress
5141 This flag makes rclone update the stats in a static block in the termi‐
5142 nal providing a realtime overview of the transfer.
5143 Any log messages will scroll above the static block. Log messages will
5145 push the static block down to the bottom of the terminal where it will
5146 stay.
5147 Normally this is updated every 500mS but this period can be overridden
5149 with the --stats flag.
5150 This can be used with the --stats-one-line flag for a simpler display.
5152 Note: On Windows until this bug (https://github.com/Azure/go-an‐
5154 siterm/issues/26) is fixed all non-ASCII characters will be replaced
5155 with . when --progress is in use.
5156 -q, –quiet
5158 Normally rclone outputs stats and a completion message. If you set
5159 this flag it will make as little output as possible.
5160 –retries int
5162 Retry the entire sync if it fails this many times it fails (default 3).
5163 Some remotes can be unreliable and a few retries help pick up the files
5165 which didn’t get transferred because of errors.
5166 Disable retries with --retries 1.
5168 –retries-sleep=TIME
5170 This sets the interval between each retry specified by --retries
5171 The default is 0. Use 0 to disable.
5173 –size-only
5175 Normally rclone will look at modification time and size of files to see
5176 if they are equal. If you set this flag then rclone will check only
5177 the size.
5178 This can be useful transferring files from Dropbox which have been mod‐
5180 ified by the desktop sync client which doesn’t set checksums of modifi‐
5181 cation times in the same way as rclone.
5182 –stats=TIME
5184 Commands which transfer data (sync, copy, copyto, move, moveto) will
5185 print data transfer stats at regular intervals to show their progress.
5186 This sets the interval.
5188 The default is 1m. Use 0 to disable.
5190 If you set the stats interval then all commands can show stats. This
5192 can be useful when running other commands, check or mount for example.
5193 Stats are logged at INFO level by default which means they won’t show
5195 at default log level NOTICE. Use --stats-log-level NOTICE or -v to
5196 make them show. See the Logging section for more info on log levels.
5197 Note that on macOS you can send a SIGINFO (which is normally ctrl-T in
5199 the terminal) to make the stats print immediately.
5200 –stats-file-name-length integer
5202 By default, the --stats output will truncate file names and paths
5203 longer than 40 characters. This is equivalent to providing
5204 --stats-file-name-length 40. Use --stats-file-name-length 0 to disable
5205 any truncation of file names printed by stats.
5206 –stats-log-level string
5208 Log level to show --stats output at. This can be DEBUG, INFO, NOTICE,
5209 or ERROR. The default is INFO. This means at the default level of
5210 logging which is NOTICE the stats won’t show - if you want them to then
5211 use --stats-log-level NOTICE. See the Logging section for more info on
5212 log levels.
5213 –stats-one-line
5215 When this is specified, rclone condenses the stats into a single line
5216 showing the most important stats only.
5217 –stats-one-line-date
5219 When this is specified, rclone enables the single-line stats and
5220 prepends the display with a date string. The default is 2006/01/02
5221 15:04:05 -
5222 –stats-one-line-date-format
5224 When this is specified, rclone enables the single-line stats and
5225 prepends the display with a user-supplied date string. The date string
5226 MUST be enclosed in quotes. Follow golang specs
5227 (https://golang.org/pkg/time/#Time.Format) for date formatting syntax.
5228 –stats-unit=bits|bytes
5230 By default, data transfer rates will be printed in bytes/second.
5231 This option allows the data rate to be printed in bits/second.
5233 Data transfer volume will still be reported in bytes.
5235 The rate is reported as a binary unit, not SI unit. So 1 Mbit/s equals
5237 1,048,576 bits/s and not 1,000,000 bits/s.
5238 The default is bytes.
5240 –suffix=SUFFIX
5242 When using sync, copy or move any files which would have been overwrit‐
5243 ten or deleted will have the suffix added to them. If there is a file
5244 with the same path (after the suffix has been added), then it will be
5245 overwritten.
5246 The remote in use must support server side move or copy and you must
5248 use the same remote as the destination of the sync.
5249 This is for use with files to add the suffix in the current directory
5251 or with --backup-dir. See --backup-dir for more info.
5252 For example
5254 rclone sync /path/to/local/file remote:current --suffix .bak
5256 will sync /path/to/local to remote:current, but for any files which
5258 would have been updated or deleted have .bak added.
5259 –suffix-keep-extension
5261 When using --suffix, setting this causes rclone put the SUFFIX before
5262 the extension of the files that it backs up rather than after.
5263 So let’s say we had --suffix -2019-01-01, without the flag file.txt
5265 would be backed up to file.txt-2019-01-01 and with the flag it would be
5266 backed up to file-2019-01-01.txt. This can be helpful to make sure the
5267 suffixed files can still be opened.
5268 –syslog
5270 On capable OSes (not Windows or Plan9) send all log output to syslog.
5271 This can be useful for running rclone in a script or rclone mount.
5273 –syslog-facility string
5275 If using --syslog this sets the syslog facility (eg KERN, USER). See
5276 man syslog for a list of possible facilities. The default facility is
5277 DAEMON.
5278 –tpslimit float
5280 Limit HTTP transactions per second to this. Default is 0 which is used
5281 to mean unlimited transactions per second.
5282 For example to limit rclone to 10 HTTP transactions per second use
5284 --tpslimit 10, or to 1 transaction every 2 seconds use --tpslimit 0.5.
5285 Use this when the number of transactions per second from rclone is
5287 causing a problem with the cloud storage provider (eg getting you
5288 banned or rate limited).
5289 This can be very useful for rclone mount to control the behaviour of
5291 applications using it.
5292 See also --tpslimit-burst.
5294 –tpslimit-burst int
5296 Max burst of transactions for --tpslimit. (default 1)
5297 Normally --tpslimit will do exactly the number of transaction per sec‐
5299 ond specified. However if you supply --tps-burst then rclone can save
5300 up some transactions from when it was idle giving a burst of up to the
5301 parameter supplied.
5302 For example if you provide --tpslimit-burst 10 then if rclone has been
5304 idle for more than 10*--tpslimit then it can do 10 transactions very
5305 quickly before they are limited again.
5306 This may be used to increase performance of --tpslimit without changing
5308 the long term average number of transactions per second.
5309 –track-renames
5311 By default, rclone doesn’t keep track of renamed files, so if you re‐
5312 name a file locally then sync it to a remote, rclone will delete the
5313 old file on the remote and upload a new copy.
5314 If you use this flag, and the remote supports server side copy or serv‐
5316 er side move, and the source and destination have a compatible hash,
5317 then this will track renames during sync operations and perform renam‐
5318 ing server-side.
5319 Files will be matched by size and hash - if both match then a rename
5321 will be considered.
5322 If the destination does not support server-side copy or move, rclone
5324 will fall back to the default behaviour and log an error level message
5325 to the console. Note: Encrypted destinations are not supported by
5326 --track-renames.
5327 Note that --track-renames is incompatible with --no-traverse and that
5329 it uses extra memory to keep track of all the rename candidates.
5330 Note also that --track-renames is incompatible with --delete-before and
5332 will select --delete-after instead of --delete-during.
5333 –delete-(before,during,after)
5335 This option allows you to specify when files on your destination are
5336 deleted when you sync folders.
5337 Specifying the value --delete-before will delete all files present on
5339 the destination, but not on the source before starting the transfer of
5340 any new or updated files. This uses two passes through the file sys‐
5341 tems, one for the deletions and one for the copies.
5342 Specifying --delete-during will delete files while checking and upload‐
5344 ing files. This is the fastest option and uses the least memory.
5345 Specifying --delete-after (the default value) will delay deletion of
5347 files until all new/updated files have been successfully transferred.
5348 The files to be deleted are collected in the copy pass then deleted af‐
5349 ter the copy pass has completed successfully. The files to be deleted
5350 are held in memory so this mode may use more memory. This is the
5351 safest mode as it will only delete files if there have been no errors
5352 subsequent to that. If there have been errors before the deletions
5353 start then you will get the message not deleting files as there were IO
5354 errors.
5355 –fast-list
5357 When doing anything which involves a directory listing (eg sync, copy,
5358 ls - in fact nearly every command), rclone normally lists a directory
5359 and processes it before using more directory lists to process any sub‐
5360 directories. This can be parallelised and works very quickly using the
5361 least amount of memory.
5362 However, some remotes have a way of listing all files beneath a direc‐
5364 tory in one (or a small number) of transactions. These tend to be the
5365 bucket based remotes (eg S3, B2, GCS, Swift, Hubic).
5366 If you use the --fast-list flag then rclone will use this method for
5368 listing directories. This will have the following consequences for the
5369 listing:
5370 · It will use fewer transactions (important if you pay for them)
5372 · It will use more memory. Rclone has to load the whole listing into
5374 memory.
5375 · It may be faster because it uses fewer transactions
5377 · It may be slower because it can’t be parallelized
5379 rclone should always give identical results with and without
5381 --fast-list.
5382 If you pay for transactions and can fit your entire sync listing into
5384 memory then --fast-list is recommended. If you have a very big sync to
5385 do then don’t use --fast-list otherwise you will run out of memory.
5386 If you use --fast-list on a remote which doesn’t support it, then
5388 rclone will just ignore it.
5389 –timeout=TIME
5391 This sets the IO idle timeout. If a transfer has started but then be‐
5392 comes idle for this long it is considered broken and disconnected.
5393 The default is 5m. Set to 0 to disable.
5395 –transfers=N
5397 The number of file transfers to run in parallel. It can sometimes be
5398 useful to set this to a smaller number if the remote is giving a lot of
5399 timeouts or bigger if you have lots of bandwidth and a fast remote.
5400 The default is to run 4 file transfers in parallel.
5402 -u, –update
5404 This forces rclone to skip any files which exist on the destination and
5405 have a modified time that is newer than the source file.
5406 This can be useful when transferring to a remote which doesn’t support
5408 mod times directly (or when using --use-server-modtime to avoid extra
5409 API calls) as it is more accurate than a --size-only check and faster
5410 than using --checksum.
5411 If an existing destination file has a modification time equal (within
5413 the computed modify window precision) to the source file’s, it will be
5414 updated if the sizes are different. If --checksum is set then rclone
5415 will update the destination if the checksums differ too.
5416 If an existing destination file is older than the source file then it
5418 will be updated if the size or checksum differs from the source file.
5419 On remotes which don’t support mod time directly (or when using
5421 --use-server-modtime) the time checked will be the uploaded time. This
5422 means that if uploading to one of these remotes, rclone will skip any
5423 files which exist on the destination and have an uploaded time that is
5424 newer than the modification time of the source file.
5425 –use-mmap
5427 If this flag is set then rclone will use anonymous memory allocated by
5428 mmap on Unix based platforms and VirtualAlloc on Windows for its trans‐
5429 fer buffers (size controlled by --buffer-size). Memory allocated like
5430 this does not go on the Go heap and can be returned to the OS immedi‐
5431 ately when it is finished with.
5432 If this flag is not set then rclone will allocate and free the buffers
5434 using the Go memory allocator which may use more memory as memory pages
5435 are returned less aggressively to the OS.
5436 It is possible this does not work well on all platforms so it is dis‐
5438 abled by default; in the future it may be enabled by default.
5439 –use-server-modtime
5441 Some object-store backends (e.g, Swift, S3) do not preserve file modi‐
5442 fication times (modtime). On these backends, rclone stores the origi‐
5443 nal modtime as additional metadata on the object. By default it will
5444 make an API call to retrieve the metadata when the modtime is needed by
5445 an operation.
5446 Use this flag to disable the extra API call and rely instead on the
5448 server’s modified time. In cases such as a local to remote sync using
5449 --update, knowing the local file is newer than the time it was last up‐
5450 loaded to the remote is sufficient. In those cases, this flag can
5451 speed up the process and reduce the number of API calls necessary.
5452 Using this flag on a sync operation without also using --update would
5454 cause all files modified at any time other than the last upload time to
5455 be uploaded again, which is probably not what you want.
5456 -v, -vv, –verbose
5458 With -v rclone will tell you about each file that is transferred and a
5459 small number of significant events.
5460 With -vv rclone will become very verbose telling you about every file
5462 it considers and transfers. Please send bug reports with a log with
5463 this setting.
5464 -V, –version
5466 Prints the version number
5467 SSL/TLS options
5469 The outoing SSL/TLS connections rclone makes can be controlled with
5470 these options. For example this can be very useful with the HTTP or
5471 WebDAV backends. Rclone HTTP servers have their own set of configura‐
5472 tion for SSL/TLS which you can find in their documentation.
5473 –ca-cert string
5475 This loads the PEM encoded certificate authority certificate and uses
5476 it to verify the certificates of the servers rclone connects to.
5477 If you have generated certificates signed with a local CA then you will
5479 need this flag to connect to servers using those certificates.
5480 –client-cert string
5482 This loads the PEM encoded client side certificate.
5483 This is used for mutual TLS authentication
5485 (https://en.wikipedia.org/wiki/Mutual_authentication).
5486 The --client-key flag is required too when using this.
5488 –client-key string
5490 This loads the PEM encoded client side private key used for mutual TLS
5491 authentication. Used in conjunction with --client-cert.
5492 –no-check-certificate=true/false
5494 --no-check-certificate controls whether a client verifies the server’s
5495 certificate chain and host name. If --no-check-certificate is true,
5496 TLS accepts any certificate presented by the server and any host name
5497 in that certificate. In this mode, TLS is susceptible to
5498 man-in-the-middle attacks.
5499 This option defaults to false.
5501 This should be used only for testing.
5503 Configuration Encryption
5505 Your configuration file contains information for logging in to your
5506 cloud services. This means that you should keep your .rclone.conf file
5507 in a secure location.
5508 If you are in an environment where that isn’t possible, you can add a
5510 password to your configuration. This means that you will have to sup‐
5511 ply the password every time you start rclone.
5512 To add a password to your rclone configuration, execute rclone config.
5514 >rclone config
5516 Current remotes:
5517 e) Edit existing remote
5519 n) New remote
5520 d) Delete remote
5521 s) Set configuration password
5522 q) Quit config
5523 e/n/d/s/q>
5524 Go into s, Set configuration password:
5526 e/n/d/s/q> s
5528 Your configuration is not encrypted.
5529 If you add a password, you will protect your login information to cloud services.
5530 a) Add Password
5531 q) Quit to main menu
5532 a/q> a
5533 Enter NEW configuration password:
5534 password:
5535 Confirm NEW password:
5536 password:
5537 Password set
5538 Your configuration is encrypted.
5539 c) Change Password
5540 u) Unencrypt configuration
5541 q) Quit to main menu
5542 c/u/q>
5543 Your configuration is now encrypted, and every time you start rclone
5545 you will have to supply the password. See below for details. In the
5546 same menu, you can change the password or completely remove encryption
5547 from your configuration.
5548 There is no way to recover the configuration if you lose your password.
5550 rclone uses nacl secretbox (https://godoc.org/golang.org/x/crypto/na‐
5552 cl/secretbox) which in turn uses XSalsa20 and Poly1305 to encrypt and
5553 authenticate your configuration with secret-key cryptography. The
5554 password is SHA-256 hashed, which produces the key for secretbox. The
5555 hashed password is not stored.
5556 While this provides very good security, we do not recommend storing
5558 your encrypted rclone configuration in public if it contains sensitive
5559 information, maybe except if you use a very strong password.
5560 If it is safe in your environment, you can set the RCLONE_CONFIG_PASS
5562 environment variable to contain your password, in which case it will be
5563 used for decrypting the configuration.
5564 You can set this for a session from a script. For unix like systems
5566 save this to a file called set-rclone-password:
5567 #!/bin/echo Source this file don't run it
5569 read -s RCLONE_CONFIG_PASS
5571 export RCLONE_CONFIG_PASS
5572 Then source the file when you want to use it. From the shell you would
5574 do source set-rclone-password. It will then ask you for the password
5575 and set it in the environment variable.
5576 An alternate means of supplying the password is to provide a script
5578 which will retrieve the password and print on standard output. This
5579 script should have a fully specified path name and not rely on any en‐
5580 vironment variables. The script is supplied either via --password-com‐
5581 mand="..." command line argument or via the RCLONE_PASSWORD_COMMAND en‐
5582 vironment variable.
5583 One useful example of this is using the passwordstore application to
5585 retrieve the password:
5586 export RCLONE_PASSWORD_COMMAND="pass rclone/config"
5588 If the passwordstore password manager holds the password for the rclone
5590 configuration, using the script method means the password is primarily
5591 protected by the passwordstore system, and is never embedded in the
5592 clear in scripts, nor available for examination using the standard com‐
5593 mands available. It is quite possible with long running rclone ses‐
5594 sions for copies of passwords to be innocently captured in log files or
5595 terminal scroll buffers, etc. Using the script method of supplying the
5596 password enhances the security of the config password considerably.
5597 If you are running rclone inside a script, unless you are using the
5599 --password-command method, you might want to disable password prompts.
5600 To do that, pass the parameter --ask-password=false to rclone. This
5601 will make rclone fail instead of asking for a password if RCLONE_CON‐
5602 FIG_PASS doesn’t contain a valid password, and --password-command has
5603 not been supplied.
5604 Developer options
5606 These options are useful when developing or debugging rclone. There
5607 are also some more remote specific options which aren’t documented here
5608 which are used for testing. These start with remote name eg
5609 --drive-test-option - see the docs for the remote in question.
5610 –cpuprofile=FILE
5612 Write CPU profile to file. This can be analysed with go tool pprof.
5613 –dump flag,flag,flag
5615 The --dump flag takes a comma separated list of flags to dump info
5616 about.
5617 Note that some headers including Accept-Encoding as shown may not be
5619 correct in the request and the response may not show Content-Encoding
5620 if the go standard libraries auto gzip encoding was in effect. In this
5621 case the body of the request will be gunzipped before showing it.
5622 The available flags are:
5624 –dump headers
5626 Dump HTTP headers with Authorization: lines removed. May still contain
5627 sensitive info. Can be very verbose. Useful for debugging only.
5628 Use --dump auth if you do want the Authorization: headers.
5630 –dump bodies
5632 Dump HTTP headers and bodies - may contain sensitive info. Can be very
5633 verbose. Useful for debugging only.
5634 Note that the bodies are buffered in memory so don’t use this for enor‐
5636 mous files.
5637 –dump requests
5639 Like --dump bodies but dumps the request bodies and the response head‐
5640 ers. Useful for debugging download problems.
5641 –dump responses
5643 Like --dump bodies but dumps the response bodies and the request head‐
5644 ers. Useful for debugging upload problems.
5645 –dump auth
5647 Dump HTTP headers - will contain sensitive info such as Authorization:
5648 headers - use --dump headers to dump without Authorization: headers.
5649 Can be very verbose. Useful for debugging only.
5650 –dump filters
5652 Dump the filters to the output. Useful to see exactly what include and
5653 exclude options are filtering on.
5654 –dump goroutines
5656 This dumps a list of the running go-routines at the end of the command
5657 to standard output.
5658 –dump openfiles
5660 This dumps a list of the open files at the end of the command. It uses
5661 the lsof command to do that so you’ll need that installed to use it.
5662 –memprofile=FILE
5664 Write memory profile to file. This can be analysed with go tool pprof.
5665 Filtering
5667 For the filtering options
5668 · --delete-excluded
5670 · --filter
5672 · --filter-from
5674 · --exclude
5676 · --exclude-from
5678 · --include
5680 · --include-from
5682 · --files-from
5684 · --min-size
5686 · --max-size
5688 · --min-age
5690 · --max-age
5692 · --dump filters
5694 See the filtering section (https://rclone.org/filtering/).
5696 Remote control
5698 For the remote control options and for instructions on how to remote
5699 control rclone
5700 · --rc
5702 · and anything starting with --rc-
5704 See the remote control section (https://rclone.org/rc/).
5706 Logging
5708 rclone has 4 levels of logging, ERROR, NOTICE, INFO and DEBUG.
5709 By default, rclone logs to standard error. This means you can redirect
5711 standard error and still see the normal output of rclone commands (eg
5712 rclone ls).
5713 By default, rclone will produce Error and Notice level messages.
5715 If you use the -q flag, rclone will only produce Error messages.
5717 If you use the -v flag, rclone will produce Error, Notice and Info mes‐
5719 sages.
5720 If you use the -vv flag, rclone will produce Error, Notice, Info and
5722 Debug messages.
5723 You can also control the log levels with the --log-level flag.
5725 If you use the --log-file=FILE option, rclone will redirect Error, Info
5727 and Debug messages along with standard error to FILE.
5728 If you use the --syslog flag then rclone will log to syslog and the
5730 --syslog-facility control which facility it uses.
5731 Rclone prefixes all log messages with their level in capitals, eg INFO
5733 which makes it easy to grep the log file for different kinds of infor‐
5734 mation.
5735 Exit Code
5737 If any errors occur during the command execution, rclone will exit with
5738 a non-zero exit code. This allows scripts to detect when rclone opera‐
5739 tions have failed.
5740 During the startup phase, rclone will exit immediately if an error is
5742 detected in the configuration. There will always be a log message im‐
5743 mediately before exiting.
5744 When rclone is running it will accumulate errors as it goes along, and
5746 only exit with a non-zero exit code if (after retries) there were still
5747 failed transfers. For every error counted there will be a high priori‐
5748 ty log message (visible with -q) showing the message and which file
5749 caused the problem. A high priority message is also shown when start‐
5750 ing a retry so the user can see that any previous error messages may
5751 not be valid after the retry. If rclone has done a retry it will log a
5752 high priority message if the retry was successful.
5753 List of exit codes
5755 · 0 - success
5756 · 1 - Syntax or usage error
5758 · 2 - Error not otherwise categorised
5760 · 3 - Directory not found
5762 · 4 - File not found
5764 · 5 - Temporary error (one that more retries might fix) (Retry errors)
5766 · 6 - Less serious errors (like 461 errors from dropbox) (NoRetry er‐
5768 rors)
5769 · 7 - Fatal error (one that more retries won’t fix, like account sus‐
5771 pended) (Fatal errors)
5772 · 8 - Transfer exceeded - limit set by –max-transfer reached
5774 Environment Variables
5776 Rclone can be configured entirely using environment variables. These
5777 can be used to set defaults for options or config file entries.
5778 Options
5780 Every option in rclone can have its default set by environment vari‐
5781 able.
5782 To find the name of the environment variable, first, take the long op‐
5784 tion name, strip the leading --, change - to _, make upper case and
5785 prepend RCLONE_.
5786 For example, to always set --stats 5s, set the environment variable
5788 RCLONE_STATS=5s. If you set stats on the command line this will over‐
5789 ride the environment variable setting.
5790 Or to always use the trash in drive --drive-use-trash, set
5792 RCLONE_DRIVE_USE_TRASH=true.
5793 The same parser is used for the options and the environment variables
5795 so they take exactly the same form.
5796 Config file
5798 You can set defaults for values in the config file on an individual re‐
5799 mote basis. If you want to use this feature, you will need to discover
5800 the name of the config items that you want. The easiest way is to run
5801 through rclone config by hand, then look in the config file to see what
5802 the values are (the config file can be found by looking at the help for
5803 --config in rclone help).
5804 To find the name of the environment variable, you need to set, take
5806 RCLONE_CONFIG_ + name of remote + _ + name of config file option and
5807 make it all uppercase.
5808 For example, to configure an S3 remote named mys3: without a config
5810 file (using unix ways of setting environment variables):
5811 $ export RCLONE_CONFIG_MYS3_TYPE=s3
5813 $ export RCLONE_CONFIG_MYS3_ACCESS_KEY_ID=XXX
5814 $ export RCLONE_CONFIG_MYS3_SECRET_ACCESS_KEY=XXX
5815 $ rclone lsd MYS3:
5816 -1 2016-09-21 12:54:21 -1 my-bucket
5817 $ rclone listremotes | grep mys3
5818 mys3:
5819 Note that if you want to create a remote using environment variables
5821 you must create the ..._TYPE variable as above.
5822 Other environment variables
5824 · RCLONE_CONFIG_PASS` set to contain your config file password (see
5825 Configuration Encryption section)
5826 · HTTP_PROXY, HTTPS_PROXY and NO_PROXY (or the lowercase versions
5828 thereof).
5829 · HTTPS_PROXY takes precedence over HTTP_PROXY for https requests.
5831 · The environment values may be either a complete URL or a
5833 “host[:port]” for, in which case the “http” scheme is assumed.
5834
5836 Some of the configurations (those involving oauth2) require an Internet
5837 connected web browser.
5838 If you are trying to set rclone up on a remote or headless box with no
5840 browser available on it (eg a NAS or a server in a datacenter) then you
5841 will need to use an alternative means of configuration. There are two
5842 ways of doing it, described below.
5843 Configuring using rclone authorize
5845 On the headless box
5846 ...
5848 Remote config
5849 Use auto config?
5850 * Say Y if not sure
5851 * Say N if you are working on a remote or headless machine
5852 y) Yes
5853 n) No
5854 y/n> n
5855 For this to work, you will need rclone available on a machine that has a web browser available.
5856 Execute the following on your machine:
5857 rclone authorize "amazon cloud drive"
5858 Then paste the result below:
5859 result>
5860 Then on your main desktop machine
5862 rclone authorize "amazon cloud drive"
5864 If your browser doesn't open automatically go to the following link: http://127.0.0.1:53682/auth
5865 Log in and authorize rclone for access
5866 Waiting for code...
5867 Got code
5868 Paste the following into your remote machine --->
5869 SECRET_TOKEN
5870 <---End paste
5871 Then back to the headless box, paste in the code
5873 result> SECRET_TOKEN
5875 --------------------
5876 [acd12]
5877 client_id =
5878 client_secret =
5879 token = SECRET_TOKEN
5880 --------------------
5881 y) Yes this is OK
5882 e) Edit this remote
5883 d) Delete this remote
5884 y/e/d>
5885 Configuring by copying the config file
5887 Rclone stores all of its config in a single configuration file. This
5888 can easily be copied to configure a remote rclone.
5889 So first configure rclone on your desktop machine
5891 rclone config
5893 to set up the config file.
5895 Find the config file by running rclone config file, for example
5897 $ rclone config file
5899 Configuration file is stored at:
5900 /home/user/.rclone.conf
5901 Now transfer it to the remote box (scp, cut paste, ftp, sftp etc) and
5903 place it in the correct place (use rclone config file on the remote box
5904 to find out where).
5905
5907 Rclone has a sophisticated set of include and exclude rules. Some of
5908 these are based on patterns and some on other things like file size.
5909 The filters are applied for the copy, sync, move, ls, lsl, md5sum,
5911 sha1sum, size, delete and check operations. Note that purge does not
5912 obey the filters.
5913 Each path as it passes through rclone is matched against the include
5915 and exclude rules like --include, --exclude, --include-from, --ex‐
5916 clude-from, --filter, or --filter-from. The simplest way to try them
5917 out is using the ls command, or --dry-run together with -v.
5918 Patterns
5920 The patterns used to match files for inclusion or exclusion are based
5921 on “file globs” as used by the unix shell.
5922 If the pattern starts with a / then it only matches at the top level of
5924 the directory tree, relative to the root of the remote (not necessarily
5925 the root of the local drive). If it doesn’t start with / then it is
5926 matched starting at the end of the path, but it will only match a com‐
5927 plete path element:
5928 file.jpg - matches "file.jpg"
5930 - matches "directory/file.jpg"
5931 - doesn't match "afile.jpg"
5932 - doesn't match "directory/afile.jpg"
5933 /file.jpg - matches "file.jpg" in the root directory of the remote
5934 - doesn't match "afile.jpg"
5935 - doesn't match "directory/file.jpg"
5936 Important Note that you must use / in patterns and not \ even if run‐
5938 ning on Windows.
5939 A * matches anything but not a /.
5941 *.jpg - matches "file.jpg"
5943 - matches "directory/file.jpg"
5944 - doesn't match "file.jpg/something"
5945 Use ** to match anything, including slashes (/).
5947 dir/** - matches "dir/file.jpg"
5949 - matches "dir/dir1/dir2/file.jpg"
5950 - doesn't match "directory/file.jpg"
5951 - doesn't match "adir/file.jpg"
5952 A ? matches any character except a slash /.
5954 l?ss - matches "less"
5956 - matches "lass"
5957 - doesn't match "floss"
5958 A [ and ] together make a character class, such as [a-z] or [aeiou] or
5960 [[:alpha:]]. See the go regexp docs (https://golang.org/pkg/reg‐
5961 exp/syntax/) for more info on these.
5962 h[ae]llo - matches "hello"
5964 - matches "hallo"
5965 - doesn't match "hullo"
5966 A { and } define a choice between elements. It should contain a comma
5968 separated list of patterns, any of which might match. These patterns
5969 can contain wildcards.
5970 {one,two}_potato - matches "one_potato"
5972 - matches "two_potato"
5973 - doesn't match "three_potato"
5974 - doesn't match "_potato"
5975 Special characters can be escaped with a \ before them.
5977 \*.jpg - matches "*.jpg"
5979 \\.jpg - matches "\.jpg"
5980 \[one\].jpg - matches "[one].jpg"
5981 Patterns are case sensitive unless the --ignore-case flag is used.
5983 Without --ignore-case (default)
5985 potato - matches "potato"
5987 - doesn't match "POTATO"
5988 With --ignore-case
5990 potato - matches "potato"
5992 - matches "POTATO"
5993 Note also that rclone filter globs can only be used in one of the fil‐
5995 ter command line flags, not in the specification of the remote, so
5996 rclone copy "remote:dir*.jpg" /path/to/dir won’t work - what is re‐
5997 quired is rclone --include "*.jpg" copy remote:dir /path/to/dir
5998 Directories
6000 Rclone keeps track of directories that could match any file patterns.
6001 Eg if you add the include rule
6003 /a/*.jpg
6005 Rclone will synthesize the directory include rule
6007 /a/
6009 If you put any rules which end in / then it will only match directo‐
6011 ries.
6012 Directory matches are only used to optimise directory access patterns -
6014 you must still match the files that you want to match. Directory
6015 matches won’t optimise anything on bucket based remotes (eg s3, swift,
6016 google compute storage, b2) which don’t have a concept of directory.
6017 Differences between rsync and rclone patterns
6019 Rclone implements bash style {a,b,c} glob matching which rsync doesn’t.
6020 Rclone always does a wildcard match so \ must always escape a \.
6022 How the rules are used
6024 Rclone maintains a combined list of include rules and exclude rules.
6025 Each file is matched in order, starting from the top, against the rule
6027 in the list until it finds a match. The file is then included or ex‐
6028 cluded according to the rule type.
6029 If the matcher fails to find a match after testing against all the en‐
6031 tries in the list then the path is included.
6032 For example given the following rules, + being include, - being ex‐
6034 clude,
6035 - secret*.jpg
6037 + *.jpg
6038 + *.png
6039 + file2.avi
6040 - *
6041 This would include
6043 · file1.jpg
6045 · file3.png
6047 · file2.avi
6049 This would exclude
6051 · secret17.jpg
6053 · non *.jpg and *.png
6055 A similar process is done on directory entries before recursing into
6057 them. This only works on remotes which have a concept of directory (Eg
6058 local, google drive, onedrive, amazon drive) and not on bucket based
6059 remotes (eg s3, swift, google compute storage, b2).
6060 Adding filtering rules
6062 Filtering rules are added with the following command line flags.
6063 Repeating options
6065 You can repeat the following options to add more than one rule of that
6066 type.
6067 · --include
6069 · --include-from
6071 · --exclude
6073 · --exclude-from
6075 · --filter
6077 · --filter-from
6079 Important You should not use --include* together with --exclude*. It
6081 may produce different results than you expected. In that case try to
6082 use: --filter*.
6083 Note that all the options of the same type are processed together in
6085 the order above, regardless of what order they were placed on the com‐
6086 mand line.
6087 So all --include options are processed first in the order they appeared
6089 on the command line, then all --include-from options etc.
6090 To mix up the order includes and excludes, the --filter flag can be
6092 used.
6093 --exclude - Exclude files matching pattern
6095 Add a single exclude rule with --exclude.
6096 This flag can be repeated. See above for the order the flags are pro‐
6098 cessed in.
6099 Eg --exclude *.bak to exclude all bak files from the sync.
6101 --exclude-from - Read exclude patterns from file
6103 Add exclude rules from a file.
6104 This flag can be repeated. See above for the order the flags are pro‐
6106 cessed in.
6107 Prepare a file like this exclude-file.txt
6109 # a sample exclude rule file
6111 *.bak
6112 file2.jpg
6113 Then use as --exclude-from exclude-file.txt. This will sync all files
6115 except those ending in bak and file2.jpg.
6116 This is useful if you have a lot of rules.
6118 --include - Include files matching pattern
6120 Add a single include rule with --include.
6121 This flag can be repeated. See above for the order the flags are pro‐
6123 cessed in.
6124 Eg --include *.{png,jpg} to include all png and jpg files in the backup
6126 and no others.
6127 This adds an implicit --exclude * at the very end of the filter list.
6129 This means you can mix --include and --include-from with the other fil‐
6130 ters (eg --exclude) but you must include all the files you want in the
6131 include statement. If this doesn’t provide enough flexibility then you
6132 must use --filter-from.
6133 --include-from - Read include patterns from file
6135 Add include rules from a file.
6136 This flag can be repeated. See above for the order the flags are pro‐
6138 cessed in.
6139 Prepare a file like this include-file.txt
6141 # a sample include rule file
6143 *.jpg
6144 *.png
6145 file2.avi
6146 Then use as --include-from include-file.txt. This will sync all jpg,
6148 png files and file2.avi.
6149 This is useful if you have a lot of rules.
6151 This adds an implicit --exclude * at the very end of the filter list.
6153 This means you can mix --include and --include-from with the other fil‐
6154 ters (eg --exclude) but you must include all the files you want in the
6155 include statement. If this doesn’t provide enough flexibility then you
6156 must use --filter-from.
6157 --filter - Add a file-filtering rule
6159 This can be used to add a single include or exclude rule. Include
6160 rules start with + and exclude rules start with -. A special rule
6161 called ! can be used to clear the existing rules.
6162 This flag can be repeated. See above for the order the flags are pro‐
6164 cessed in.
6165 Eg --filter "- *.bak" to exclude all bak files from the sync.
6167 --filter-from - Read filtering patterns from a file
6169 Add include/exclude rules from a file.
6170 This flag can be repeated. See above for the order the flags are pro‐
6172 cessed in.
6173 Prepare a file like this filter-file.txt
6175 # a sample filter rule file
6177 - secret*.jpg
6178 + *.jpg
6179 + *.png
6180 + file2.avi
6181 - /dir/Trash/**
6182 + /dir/**
6183 # exclude everything else
6184 - *
6185 Then use as --filter-from filter-file.txt. The rules are processed in
6187 the order that they are defined.
6188 This example will include all jpg and png files, exclude any files
6190 matching secret*.jpg and include file2.avi. It will also include ev‐
6191 erything in the directory dir at the root of the sync, except dir/Trash
6192 which it will exclude. Everything else will be excluded from the sync.
6193 --files-from - Read list of source-file names
6195 This reads a list of file names from the file passed in and only these
6196 files are transferred. The filtering rules are ignored completely if
6197 you use this option.
6198 --files-from expects a list of files as it’s input. rclone lsf
6200 (https://rclone.org/commands/rclone_lsf/) has a compatible format that
6201 can be used to export file lists from remotes.
6202 Rclone will traverse the file system if you use --files-from, effec‐
6204 tively using the files in --files-from as a set of filters. Rclone
6205 will not error if any of the files are missing.
6206 If you use --no-traverse as well as --files-from then rclone will not
6208 traverse the destination file system, it will find each file individu‐
6209 ally using approximately 1 API call. This can be more efficient for
6210 small lists of files.
6211 This option can be repeated to read from more than one file. These are
6213 read in the order that they are placed on the command line.
6214 Paths within the --files-from file will be interpreted as starting with
6216 the root specified in the command. Leading / characters are ignored.
6217 For example, suppose you had files-from.txt with this content:
6219 # comment
6221 file1.jpg
6222 subdir/file2.jpg
6223 You could then use it like this:
6225 rclone copy --files-from files-from.txt /home/me/pics remote:pics
6227 This will transfer these files only (if they exist)
6229 /home/me/pics/file1.jpg → remote:pics/file1.jpg
6231 /home/me/pics/subdir/file2.jpg → remote:pics/subdir/file2.jpg
6232 To take a more complicated example, let’s say you had a few files you
6234 want to back up regularly with these absolute paths:
6235 /home/user1/important
6237 /home/user1/dir/file
6238 /home/user2/stuff
6239 To copy these you’d find a common subdirectory - in this case /home and
6241 put the remaining files in files-from.txt with or without leading /, eg
6242 user1/important
6244 user1/dir/file
6245 user2/stuff
6246 You could then copy these to a remote like this
6248 rclone copy --files-from files-from.txt /home remote:backup
6250 The 3 files will arrive in remote:backup with the paths as in the
6252 files-from.txt like this:
6253 /home/user1/important → remote:backup/user1/important
6255 /home/user1/dir/file → remote:backup/user1/dir/file
6256 /home/user2/stuff → remote:backup/user2/stuff
6257 You could of course choose / as the root too in which case your
6259 files-from.txt might look like this.
6260 /home/user1/important
6262 /home/user1/dir/file
6263 /home/user2/stuff
6264 And you would transfer it like this
6266 rclone copy --files-from files-from.txt / remote:backup
6268 In this case there will be an extra home directory on the remote:
6270 /home/user1/important → remote:backup/home/user1/important
6272 /home/user1/dir/file → remote:backup/home/user1/dir/file
6273 /home/user2/stuff → remote:backup/home/user2/stuff
6274 --min-size - Don’t transfer any file smaller than this
6276 This option controls the minimum size file which will be transferred.
6277 This defaults to kBytes but a suffix of k, M, or G can be used.
6278 For example --min-size 50k means no files smaller than 50kByte will be
6280 transferred.
6281 --max-size - Don’t transfer any file larger than this
6283 This option controls the maximum size file which will be transferred.
6284 This defaults to kBytes but a suffix of k, M, or G can be used.
6285 For example --max-size 1G means no files larger than 1GByte will be
6287 transferred.
6288 --max-age - Don’t transfer any file older than this
6290 This option controls the maximum age of files to transfer. Give in
6291 seconds or with a suffix of:
6292 · ms - Milliseconds
6294 · s - Seconds
6296 · m - Minutes
6298 · h - Hours
6300 · d - Days
6302 · w - Weeks
6304 · M - Months
6306 · y - Years
6308 For example --max-age 2d means no files older than 2 days will be
6310 transferred.
6311 --min-age - Don’t transfer any file younger than this
6313 This option controls the minimum age of files to transfer. Give in
6314 seconds or with a suffix (see --max-age for list of suffixes)
6315 For example --min-age 2d means no files younger than 2 days will be
6317 transferred.
6318 --delete-excluded - Delete files on dest excluded from sync
6320 Important this flag is dangerous - use with --dry-run and -v first.
6321 When doing rclone sync this will delete any files which are excluded
6323 from the sync on the destination.
6324 If for example you did a sync from A to B without the --min-size 50k
6326 flag
6327 rclone sync A: B:
6329 Then you repeated it like this with the --delete-excluded
6331 rclone --min-size 50k --delete-excluded sync A: B:
6333 This would delete all files on B which are less than 50 kBytes as these
6335 are now excluded from the sync.
6336 Always test first with --dry-run and -v before using this flag.
6338 --dump filters - dump the filters to the output
6340 This dumps the defined filters to the output as regular expressions.
6341 Useful for debugging.
6343 --ignore-case - make searches case insensitive
6345 Normally filter patterns are case sensitive. If this flag is supplied
6346 then filter patterns become case insensitive.
6347 Normally a --include "file.txt" will not match a file called FILE.txt.
6349 However if you use the --ignore-case flag then --include "file.txt"
6350 this will match a file called FILE.txt.
6351 Quoting shell metacharacters
6353 The examples above may not work verbatim in your shell as they have
6354 shell metacharacters in them (eg *), and may require quoting.
6355 Eg linux, OSX
6357 · --include \*.jpg
6359 · --include '*.jpg'
6361 · --include='*.jpg'
6363 In Windows the expansion is done by the command not the shell so this
6365 should work fine
6366 · --include *.jpg
6368 Exclude directory based on a file
6370 It is possible to exclude a directory based on a file, which is present
6371 in this directory. Filename should be specified using the --ex‐
6372 clude-if-present flag. This flag has a priority over the other filter‐
6373 ing flags.
6374 Imagine, you have the following directory structure:
6376 dir1/file1
6378 dir1/dir2/file2
6379 dir1/dir2/dir3/file3
6380 dir1/dir2/dir3/.ignore
6381 You can exclude dir3 from sync by running the following command:
6383 rclone sync --exclude-if-present .ignore dir1 remote:backup
6385 Currently only one filename is supported, i.e. --exclude-if-present
6387 should not be used multiple times.
6388
6390 Rclone can serve a web based GUI (graphical user interface). This is
6391 somewhat experimental at the moment so things may be subject to change.
6392 Run this command in a terminal and rclone will download and then dis‐
6394 play the GUI in a web browser.
6395 rclone rcd --rc-web-gui
6397 This will produce logs like this and rclone needs to continue to run to
6399 serve the GUI:
6400 2019/08/25 11:40:14 NOTICE: A new release for gui is present at https://github.com/rclone/rclone-webui-react/releases/download/v0.0.6/currentbuild.zip
6402 2019/08/25 11:40:14 NOTICE: Downloading webgui binary. Please wait. [Size: 3813937, Path : /home/USER/.cache/rclone/webgui/v0.0.6.zip]
6403 2019/08/25 11:40:16 NOTICE: Unzipping
6404 2019/08/25 11:40:16 NOTICE: Serving remote control on http://127.0.0.1:5572/
6405 This assumes you are running rclone locally on your machine. It is
6407 possible to separate the rclone and the GUI - see below for details.
6408 If you wish to check for updates then you can add --rc-web-gui-update
6410 to the command line.
6411 If you find your GUI broken, you may force it to update by add
6413 --rc-web-gui-force-update.
6414 By default, rclone will open your browser. Add
6416 --rc-web-gui-no-open-browser to disable this feature.
6417 Using the GUI
6419 Once the GUI opens, you will be looking at the dashboard which has an
6420 overall overview.
6421 On the left hand side you will see a series of view buttons you can
6423 click on:
6424 · Dashboard - main overview
6426 · Configs - examine and create new configurations
6428 · Explorer - view, download and upload files to the cloud storage sys‐
6430 tems
6431 · Backend - view or alter the backend config
6433 · Log out
6435 (More docs and walkthrough video to come!)
6437 How it works
6439 When you run the rclone rcd --rc-web-gui this is what happens
6440 · Rclone starts but only runs the remote control API (“rc”).
6442 · The API is bound to localhost with an auto generated username and
6444 password.
6445 · If the API bundle is missing then rclone will download it.
6447 · rclone will start serving the files from the API bundle over the same
6449 port as the API
6450 · rclone will open the browser with a login_token so it can log
6452 straight in.
6453 Advanced use
6455 The rclone rcd may use any of the flags documented on the rc page
6456 (https://rclone.org/rc/#supported-parameters).
6457 The flag --rc-web-gui is shorthand for
6459 · Download the web GUI if necessary
6461 · Check we are using some authentication
6463 · --rc-user gui
6465 · --rc-pass <random password>
6467 · --rc-serve
6469 These flags can be overidden as desired.
6471 See also the rclone rcd documentation (https://rclone.org/com‐
6473 mands/rclone_rcd/).
6474 Example: Running a public GUI
6476 For example the GUI could be served on a public port over SSL using an
6477 htpasswd file using the following flags:
6478 · --rc-web-gui
6480 · --rc-addr :443
6482 · --rc-htpasswd /path/to/htpasswd
6484 · --rc-cert /path/to/ssl.crt
6486 · --rc-key /path/to/ssl.key
6488 Example: Running a GUI behind a proxy
6490 If you want to run the GUI behind a proxy at /rclone you could use
6491 these flags:
6492 · --rc-web-gui
6494 · --rc-baseurl rclone
6496 · --rc-htpasswd /path/to/htpasswd
6498 Or instead of htpassword if you just want a single user and password:
6500 · --rc-user me
6502 · --rc-pass mypassword
6504 Project
6506 The GUI is being developed in the: rclone/rclone-webui-react resposito‐
6507 ry (https://github.com/rclone/rclone-webui-react).
6508 Bug reports and contributions are very welcome :-)
6510 If you have questions then please ask them on the rclone forum
6512 (https://forum.rclone.org/).
6513
6515 If rclone is run with the --rc flag then it starts an http server which
6516 can be used to remote control rclone.
6517 If you just want to run a remote control then see the rcd command
6519 (https://rclone.org/commands/rclone_rcd/).
6520 NB this is experimental and everything here is subject to change!
6522 Supported parameters
6524 –rc
6525 Flag to start the http server listen on remote requests
6526 –rc-addr=IP
6528 IPaddress:Port or :Port to bind server to. (default “localhost:5572”)
6529 –rc-cert=KEY
6531 SSL PEM key (concatenation of certificate and CA certificate)
6532 –rc-client-ca=PATH
6534 Client certificate authority to verify clients with
6535 –rc-htpasswd=PATH
6537 htpasswd file - if not provided no authentication is done
6538 –rc-key=PATH
6540 SSL PEM Private key
6541 –rc-max-header-bytes=VALUE
6543 Maximum size of request header (default 4096)
6544 –rc-user=VALUE
6546 User name for authentication.
6547 –rc-pass=VALUE
6549 Password for authentication.
6550 –rc-realm=VALUE
6552 Realm for authentication (default “rclone”)
6553 –rc-server-read-timeout=DURATION
6555 Timeout for server reading data (default 1h0m0s)
6556 –rc-server-write-timeout=DURATION
6558 Timeout for server writing data (default 1h0m0s)
6559 –rc-serve
6561 Enable the serving of remote objects via the HTTP interface. This
6562 means objects will be accessible at http://127.0.0.1:5572/ by default,
6563 so you can browse to http://127.0.0.1:5572/ or http://127.0.0.1:5572/*
6564 to see a listing of the remotes. Objects may be requested from remotes
6565 using this syntax http://127.0.0.1:5572/[remote:path]/path/to/object
6566 Default Off.
6568 –rc-files /path/to/directory
6570 Path to local files to serve on the HTTP server.
6571 If this is set then rclone will serve the files in that directory. It
6573 will also open the root in the web browser if specified. This is for
6574 implementing browser based GUIs for rclone functions.
6575 If --rc-user or --rc-pass is set then the URL that is opened will have
6577 the authorization in the URL in the http://user:pass@localhost/ style.
6578 Default Off.
6580 –rc-web-gui
6582 Set this flag to serve the default web gui on the same port as rclone.
6583 Default Off.
6585 –rc-allow-origin
6587 Set the allowed Access-Control-Allow-Origin for rc requests.
6588 Can be used with –rc-web-gui if the rclone is running on different IP
6590 than the web-gui.
6591 Default is IP address on which rc is running.
6593 –rc-web-fetch-url
6595 Set the URL to fetch the rclone-web-gui files from.
6596 Default https://api.github.com/repos/rclone/rclone-webui-react/releas‐
6598 es/latest.
6599 –rc-web-gui-update
6601 Set this flag to check and update rclone-webui-react from the
6602 rc-web-fetch-url.
6603 Default Off.
6605 –rc-web-gui-force-update
6607 Set this flag to force update rclone-webui-react from the
6608 rc-web-fetch-url.
6609 Default Off.
6611 –rc-web-gui-no-open-browser
6613 Set this flag to disable opening browser automatically when using
6614 web-gui.
6615 Default Off.
6617 –rc-job-expire-duration=DURATION
6619 Expire finished async jobs older than DURATION (default 60s).
6620 –rc-job-expire-interval=DURATION
6622 Interval duration to check for expired async jobs (default 10s).
6623 –rc-no-auth
6625 By default rclone will require authorisation to have been set up on the
6626 rc interface in order to use any methods which access any rclone re‐
6627 motes. Eg operations/list is denied as it involved creating a remote
6628 as is sync/copy.
6629 If this is set then no authorisation will be required on the server to
6631 use these methods. The alternative is to use --rc-user and --rc-pass
6632 and use these credentials in the request.
6633 Default Off.
6635 Accessing the remote control via the rclone rc command
6637 Rclone itself implements the remote control protocol in its rclone rc
6638 command.
6639 You can use it like this
6641 $ rclone rc rc/noop param1=one param2=two
6643 {
6644 "param1": "one",
6645 "param2": "two"
6646 }
6647 Run rclone rc on its own to see the help for the installed remote con‐
6649 trol commands.
6650 rclone rc also supports a --json flag which can be used to send more
6652 complicated input parameters.
6653 $ rclone rc --json '{ "p1": [1,"2",null,4], "p2": { "a":1, "b":2 } }' rc/noop
6655 {
6656 "p1": [
6657 1,
6658 "2",
6659 null,
6660 4
6661 ],
6662 "p2": {
6663 "a": 1,
6664 "b": 2
6665 }
6666 }
6667 Special parameters
6669 The rc interface supports some special parameters which apply to all
6670 commands. These start with _ to show they are different.
6671 Running asynchronous jobs with _async = true
6673 Each rc call is classified as a job and it is assigned its own id. By
6674 default jobs are executed immediately as they are created or syn‐
6675 chronously.
6676 If _async has a true value when supplied to an rc call then it will re‐
6678 turn immediately with a job id and the task will be run in the back‐
6679 ground. The job/status call can be used to get information of the
6680 background job. The job can be queried for up to 1 minute after it has
6681 finished.
6682 It is recommended that potentially long running jobs, eg sync/sync,
6684 sync/copy, sync/move, operations/purge are run with the _async flag to
6685 avoid any potential problems with the HTTP request and response timing
6686 out.
6687 Starting a job with the _async flag:
6689 $ rclone rc --json '{ "p1": [1,"2",null,4], "p2": { "a":1, "b":2 }, "_async": true }' rc/noop
6691 {
6692 "jobid": 2
6693 }
6694 Query the status to see if the job has finished. For more information
6696 on the meaning of these return parameters see the job/status call.
6697 $ rclone rc --json '{ "jobid":2 }' job/status
6699 {
6700 "duration": 0.000124163,
6701 "endTime": "2018-10-27T11:38:07.911245881+01:00",
6702 "error": "",
6703 "finished": true,
6704 "id": 2,
6705 "output": {
6706 "_async": true,
6707 "p1": [
6708 1,
6709 "2",
6710 null,
6711 4
6712 ],
6713 "p2": {
6714 "a": 1,
6715 "b": 2
6716 }
6717 },
6718 "startTime": "2018-10-27T11:38:07.911121728+01:00",
6719 "success": true
6720 }
6721 job/list can be used to show the running or recently completed jobs
6723 $ rclone rc job/list
6725 {
6726 "jobids": [
6727 2
6728 ]
6729 }
6730 Assigning operations to groups with _group =
6732 Each rc call has it’s own stats group for tracking it’s metrics. By
6733 default grouping is done by the composite group name from prefix job/
6734 and id of the job like so job/1.
6735 If _group has a value then stats for that request will be grouped under
6737 that value. This allows caller to group stats under their own name.
6738 Stats for specific group can be accessed by passing group to
6740 core/stats:
6741 $ rclone rc --json '{ "group": "job/1" }' core/stats
6743 {
6744 "speed": 12345
6745 ...
6746 }
6747 Supported commands
6749 cache/expire: Purge a remote from cache {#cache/expire}
6750 Purge a remote from the cache backend. Supports either a directory or
6751 a file. Params: - remote = path to remote (required) - withData =
6752 true/false to delete cached data (chunks) as well (optional)
6753 Eg
6755 rclone rc cache/expire remote=path/to/sub/folder/
6757 rclone rc cache/expire remote=/ withData=true
6758 cache/fetch: Fetch file chunks {#cache/fetch}
6760 Ensure the specified file chunks are cached on disk.
6761 The chunks= parameter specifies the file chunks to check. It takes a
6763 comma separated list of array slice indices. The slice indices are
6764 similar to Python slices: start[:end]
6765 start is the 0 based chunk number from the beginning of the file to
6767 fetch inclusive. end is 0 based chunk number from the beginning of the
6768 file to fetch exclusive. Both values can be negative, in which case
6769 they count from the back of the file. The value “-5:” represents the
6770 last 5 chunks of a file.
6771 Some valid examples are: “:5,-5:” -> the first and last five chunks
6773 “0,-2” -> the first and the second last chunk “0:10” -> the first ten
6774 chunks
6775 Any parameter with a key that starts with “file” can be used to specify
6777 files to fetch, eg
6778 rclone rc cache/fetch chunks=0 file=hello file2=home/goodbye
6780 File names will automatically be encrypted when the a crypt remote is
6782 used on top of the cache.
6783 cache/stats: Get cache stats {#cache/stats}
6785 Show statistics for the cache remote.
6786 config/create: create the config for a remote. {#config/create}
6788 This takes the following parameters
6789 · name - name of remote
6791 · parameters - a map of { “key”: “value” } pairs
6793 · type - type of the new remote
6795 See the config create command (https://rclone.org/commands/rclone_con‐
6797 fig_create/) command for more information on the above.
6798 Authentication is required for this call.
6800 config/delete: Delete a remote in the config file. {#config/delete}
6802 Parameters:
6803 · name - name of remote to delete
6805 See the config delete command (https://rclone.org/commands/rclone_con‐
6807 fig_delete/) command for more information on the above.
6808 Authentication is required for this call.
6810 config/dump: Dumps the config file. {#config/dump}
6812 Returns a JSON object: - key: value
6813 Where keys are remote names and values are the config parameters.
6815 See the config dump command (https://rclone.org/commands/rclone_con‐
6817 fig_dump/) command for more information on the above.
6818 Authentication is required for this call.
6820 config/get: Get a remote in the config file. {#config/get}
6822 Parameters:
6823 · name - name of remote to get
6825 See the config dump command (https://rclone.org/commands/rclone_con‐
6827 fig_dump/) command for more information on the above.
6828 Authentication is required for this call.
6830 config/listremotes: Lists the remotes in the config file. {#con‐
6832 fig/listremotes}
6833 Returns - remotes - array of remote names
6834 See the listremotes command (https://rclone.org/com‐
6836 mands/rclone_listremotes/) command for more information on the above.
6837 Authentication is required for this call.
6839 config/password: password the config for a remote. {#config/password}
6841 This takes the following parameters
6842 · name - name of remote
6844 · parameters - a map of { “key”: “value” } pairs
6846 See the config password command (https://rclone.org/com‐
6848 mands/rclone_config_password/) command for more information on the
6849 above.
6850 Authentication is required for this call.
6852 config/providers: Shows how providers are configured in the config file.
6854 {#config/providers}
6855 Returns a JSON object: - providers - array of objects
6856 See the config providers command (https://rclone.org/com‐
6858 mands/rclone_config_providers/) command for more information on the
6859 above.
6860 Authentication is required for this call.
6862 config/update: update the config for a remote. {#config/update}
6864 This takes the following parameters
6865 · name - name of remote
6867 · parameters - a map of { “key”: “value” } pairs
6869 See the config update command (https://rclone.org/commands/rclone_con‐
6871 fig_update/) command for more information on the above.
6872 Authentication is required for this call.
6874 core/bwlimit: Set the bandwidth limit. {#core/bwlimit}
6876 This sets the bandwidth limit to that passed in.
6877 Eg
6879 rclone rc core/bwlimit rate=off
6881 {
6882 "bytesPerSecond": -1,
6883 "rate": "off"
6884 }
6885 rclone rc core/bwlimit rate=1M
6886 {
6887 "bytesPerSecond": 1048576,
6888 "rate": "1M"
6889 }
6890 If the rate parameter is not suppied then the bandwidth is queried
6892 rclone rc core/bwlimit
6894 {
6895 "bytesPerSecond": 1048576,
6896 "rate": "1M"
6897 }
6898 The format of the parameter is exactly the same as passed to –bwlimit
6900 except only one bandwidth may be specified.
6901 In either case “rate” is returned as a human readable string, and
6903 “bytesPerSecond” is returned as a number.
6904 core/gc: Runs a garbage collection. {#core/gc}
6906 This tells the go runtime to do a garbage collection run. It isn’t
6907 necessary to call this normally, but it can be useful for debugging
6908 memory problems.
6909 core/group-list: Returns list of stats. {#core/group-list}
6911 This returns list of stats groups currently in memory.
6912 Returns the following values:
6914 {
6916 "groups": an array of group names:
6917 [
6918 "group1",
6919 "group2",
6920 ...
6921 ]
6922 }
6923 core/memstats: Returns the memory statistics {#core/memstats}
6925 This returns the memory statistics of the running program. What the
6926 values mean are explained in the go docs: https://golang.org/pkg/run‐
6927 time/#MemStats
6928 The most interesting values for most people are:
6930 · HeapAlloc: This is the amount of memory rclone is actually using
6932 · HeapSys: This is the amount of memory rclone has obtained from the OS
6934 · Sys: this is the total amount of memory requested from the OS
6936 · It is virtual memory so may include unused memory
6938 core/obscure: Obscures a string passed in. {#core/obscure}
6940 Pass a clear string and rclone will obscure it for the config file: -
6941 clear - string
6942 Returns - obscured - string
6944 core/pid: Return PID of current process {#core/pid}
6946 This returns PID of current process. Useful for stopping rclone
6947 process.
6948 core/quit: Terminates the app. {#core/quit}
6950 (optional) Pass an exit code to be used for terminating the app: - ex‐
6951 itCode - int
6952 core/stats: Returns stats about current transfers. {#core/stats}
6954 This returns all available stats:
6955 rclone rc core/stats
6957 If group is not provided then summed up stats for all groups will be
6959 returned.
6960 Parameters
6962 · group - name of the stats group (string)
6964 Returns the following values:
6966 {
6968 "speed": average speed in bytes/sec since start of the process,
6969 "bytes": total transferred bytes since the start of the process,
6970 "errors": number of errors,
6971 "fatalError": whether there has been at least one FatalError,
6972 "retryError": whether there has been at least one non-NoRetryError,
6973 "checks": number of checked files,
6974 "transfers": number of transferred files,
6975 "deletes" : number of deleted files,
6976 "elapsedTime": time in seconds since the start of the process,
6977 "lastError": last occurred error,
6978 "transferring": an array of currently active file transfers:
6979 [
6980 {
6981 "bytes": total transferred bytes for this file,
6982 "eta": estimated time in seconds until file transfer completion
6983 "name": name of the file,
6984 "percentage": progress of the file transfer in percent,
6985 "speed": speed in bytes/sec,
6986 "speedAvg": speed in bytes/sec as an exponentially weighted moving average,
6987 "size": size of the file in bytes
6988 }
6989 ],
6990 "checking": an array of names of currently active file checks
6991 []
6992 }
6993 Values for “transferring”, “checking” and “lastError” are only assigned
6995 if data is available. The value for “eta” is null if an eta cannot be
6996 determined.
6997 core/stats-delete: Delete stats group. {#core/stats-delete}
6999 This deletes entire stats group
7000 Parameters
7002 · group - name of the stats group (string)
7004 core/stats-reset: Reset stats. {#core/stats-reset}
7006 This clears counters, errors and finished transfers for all stats or
7007 specific stats group if group is provided.
7008 Parameters
7010 · group - name of the stats group (string)
7012 core/transferred: Returns stats about completed transfers. {#core/trans‐
7014 ferred}
7015 This returns stats about completed transfers:
7016 rclone rc core/transferred
7018 If group is not provided then completed transfers for all groups will
7020 be returned.
7021 Note only the last 100 completed transfers are returned.
7023 Parameters
7025 · group - name of the stats group (string)
7027 Returns the following values:
7029 {
7031 "transferred": an array of completed transfers (including failed ones):
7032 [
7033 {
7034 "name": name of the file,
7035 "size": size of the file in bytes,
7036 "bytes": total transferred bytes for this file,
7037 "checked": if the transfer is only checked (skipped, deleted),
7038 "timestamp": integer representing millisecond unix epoch,
7039 "error": string description of the error (empty if successfull),
7040 "jobid": id of the job that this transfer belongs to
7041 }
7042 ]
7043 }
7044 core/version: Shows the current version of rclone and the go runtime.
7046 {#core/version}
7047 This shows the current version of go and the go runtime
7048 · version - rclone version, eg “v1.44”
7050 · decomposed - version number as [major, minor, patch, subpatch]
7052 · note patch and subpatch will be 999 for a git compiled version
7054 · isGit - boolean - true if this was compiled from the git version
7056 · os - OS in use as according to Go
7058 · arch - cpu architecture in use according to Go
7060 · goVersion - version of Go runtime in use
7062 debug/set-block-profile-rate: Set runtime.SetBlockProfileRate for blocking
7064 profiling. {#debug/set-block-profile-rate}
7065 SetBlockProfileRate controls the fraction of goroutine blocking events
7066 that are reported in the blocking profile. The profiler aims to sample
7067 an average of one blocking event per rate nanoseconds spent blocked.
7068 To include every blocking event in the profile, pass rate = 1. To turn
7070 off profiling entirely, pass rate <= 0.
7071 After calling this you can use this to see the blocking profile:
7073 go tool pprof http://localhost:5572/debug/pprof/block
7075 Parameters
7077 · rate - int
7079 debug/set-mutex-profile-fraction: Set runtime.SetMutexProfileFraction for
7081 mutex profiling. {#debug/set-mutex-profile-fraction}
7082 SetMutexProfileFraction controls the fraction of mutex contention
7083 events that are reported in the mutex profile. On average 1/rate
7084 events are reported. The previous rate is returned.
7085 To turn off profiling entirely, pass rate 0. To just read the current
7087 rate, pass rate < 0. (For n>1 the details of sampling may change.)
7088 Once this is set you can look use this to profile the mutex contention:
7090 go tool pprof http://localhost:5572/debug/pprof/mutex
7092 Parameters
7094 · rate - int
7096 Results
7098 · previousRate - int
7100 job/list: Lists the IDs of the running jobs {#job/list}
7102 Parameters - None
7103 Results
7105 · jobids - array of integer job ids
7107 job/status: Reads the status of the job ID {#job/status}
7109 Parameters
7110 · jobid - id of the job (integer)
7112 Results
7114 · finished - boolean
7116 · duration - time in seconds that the job ran for
7118 · endTime - time the job finished (eg
7120 “2018-10-26T18:50:20.528746884+01:00”)
7121 · error - error from the job or empty string for no error
7123 · finished - boolean whether the job has finished or not
7125 · id - as passed in above
7127 · startTime - time the job started (eg
7129 “2018-10-26T18:50:20.528336039+01:00”)
7130 · success - boolean - true for success false otherwise
7132 · output - output of the job as would have been returned if called syn‐
7134 chronously
7135 · progress - output of the progress related to the underlying job
7137 job/stop: Stop the running job {#job/stop}
7139 Parameters
7140 · jobid - id of the job (integer)
7142 operations/about: Return the space used on the remote {#operations/about}
7144 This takes the following parameters
7145 · fs - a remote name string eg “drive:”
7147 The result is as returned from rclone about –json
7149 See the about command (https://rclone.org/commands/rclone_size/) com‐
7151 mand for more information on the above.
7152 Authentication is required for this call.
7154 operations/cleanup: Remove trashed files in the remote or path {#opera‐
7156 tions/cleanup}
7157 This takes the following parameters
7158 · fs - a remote name string eg “drive:”
7160 See the cleanup command (https://rclone.org/commands/rclone_cleanup/)
7162 command for more information on the above.
7163 Authentication is required for this call.
7165 operations/copyfile: Copy a file from source remote to destination remote
7167 {#operations/copyfile}
7168 This takes the following parameters
7169 · srcFs - a remote name string eg “drive:” for the source
7171 · srcRemote - a path within that remote eg “file.txt” for the source
7173 · dstFs - a remote name string eg “drive2:” for the destination
7175 · dstRemote - a path within that remote eg “file2.txt” for the destina‐
7177 tion
7178 Authentication is required for this call.
7180 operations/copyurl: Copy the URL to the object {#operations/copyurl}
7182 This takes the following parameters
7183 · fs - a remote name string eg “drive:”
7185 · remote - a path within that remote eg “dir”
7187 · url - string, URL to read from
7189 · autoFilename - boolean, set to true to retrieve destination file name
7191 from url See the copyurl command (https://rclone.org/com‐
7192 mands/rclone_copyurl/) command for more information on the above.
7193 Authentication is required for this call.
7195 operations/delete: Remove files in the path {#operations/delete}
7197 This takes the following parameters
7198 · fs - a remote name string eg “drive:”
7200 See the delete command (https://rclone.org/commands/rclone_delete/)
7202 command for more information on the above.
7203 Authentication is required for this call.
7205 operations/deletefile: Remove the single file pointed to {#opera‐
7207 tions/deletefile}
7208 This takes the following parameters
7209 · fs - a remote name string eg “drive:”
7211 · remote - a path within that remote eg “dir”
7213 See the deletefile command (https://rclone.org/commands/rclone_delete‐
7215 file/) command for more information on the above.
7216 Authentication is required for this call.
7218 operations/fsinfo: Return information about the remote {#operations/fsinfo}
7220 This takes the following parameters
7221 · fs - a remote name string eg “drive:”
7223 This returns info about the remote passed in;
7225 {
7227 // optional features and whether they are available or not
7228 "Features": {
7229 "About": true,
7230 "BucketBased": false,
7231 "CanHaveEmptyDirectories": true,
7232 "CaseInsensitive": false,
7233 "ChangeNotify": false,
7234 "CleanUp": false,
7235 "Copy": false,
7236 "DirCacheFlush": false,
7237 "DirMove": true,
7238 "DuplicateFiles": false,
7239 "GetTier": false,
7240 "ListR": false,
7241 "MergeDirs": false,
7242 "Move": true,
7243 "OpenWriterAt": true,
7244 "PublicLink": false,
7245 "Purge": true,
7246 "PutStream": true,
7247 "PutUnchecked": false,
7248 "ReadMimeType": false,
7249 "ServerSideAcrossConfigs": false,
7250 "SetTier": false,
7251 "SetWrapper": false,
7252 "UnWrap": false,
7253 "WrapFs": false,
7254 "WriteMimeType": false
7255 },
7256 // Names of hashes available
7257 "Hashes": [
7258 "MD5",
7259 "SHA-1",
7260 "DropboxHash",
7261 "QuickXorHash"
7262 ],
7263 "Name": "local", // Name as created
7264 "Precision": 1, // Precision of timestamps in ns
7265 "Root": "/", // Path as created
7266 "String": "Local file system at /" // how the remote will appear in logs
7267 }
7268 This command does not have a command line equivalent so use this in‐
7270 stead:
7271 rclone rc --loopback operations/fsinfo fs=remote:
7273 operations/list: List the given remote and path in JSON format {#opera‐
7275 tions/list}
7276 This takes the following parameters
7277 · fs - a remote name string eg “drive:”
7279 · remote - a path within that remote eg “dir”
7281 · opt - a dictionary of options to control the listing (optional)
7283 · recurse - If set recurse directories
7285 · noModTime - If set return modification time
7287 · showEncrypted - If set show decrypted names
7289 · showOrigIDs - If set show the IDs for each item if known
7291 · showHash - If set return a dictionary of hashes
7293 The result is
7295 · list
7297 · This is an array of objects as described in the lsjson command
7299 See the lsjson command (https://rclone.org/commands/rclone_lsjson/) for
7301 more information on the above and examples.
7302 Authentication is required for this call.
7304 operations/mkdir: Make a destination directory or container {#opera‐
7306 tions/mkdir}
7307 This takes the following parameters
7308 · fs - a remote name string eg “drive:”
7310 · remote - a path within that remote eg “dir”
7312 See the mkdir command (https://rclone.org/commands/rclone_mkdir/) com‐
7314 mand for more information on the above.
7315 Authentication is required for this call.
7317 operations/movefile: Move a file from source remote to destination remote
7319 {#operations/movefile}
7320 This takes the following parameters
7321 · srcFs - a remote name string eg “drive:” for the source
7323 · srcRemote - a path within that remote eg “file.txt” for the source
7325 · dstFs - a remote name string eg “drive2:” for the destination
7327 · dstRemote - a path within that remote eg “file2.txt” for the destina‐
7329 tion
7330 Authentication is required for this call.
7332 operations/publiclink: Create or retrieve a public link to the given file
7334 or folder. {#operations/publiclink}
7335 This takes the following parameters
7336 · fs - a remote name string eg “drive:”
7338 · remote - a path within that remote eg “dir”
7340 Returns
7342 · url - URL of the resource
7344 See the link command (https://rclone.org/commands/rclone_link/) command
7346 for more information on the above.
7347 Authentication is required for this call.
7349 operations/purge: Remove a directory or container and all of its contents
7351 {#operations/purge}
7352 This takes the following parameters
7353 · fs - a remote name string eg “drive:”
7355 · remote - a path within that remote eg “dir”
7357 See the purge command (https://rclone.org/commands/rclone_purge/) com‐
7359 mand for more information on the above.
7360 Authentication is required for this call.
7362 operations/rmdir: Remove an empty directory or container {#opera‐
7364 tions/rmdir}
7365 This takes the following parameters
7366 · fs - a remote name string eg “drive:”
7368 · remote - a path within that remote eg “dir”
7370 See the rmdir command (https://rclone.org/commands/rclone_rmdir/) com‐
7372 mand for more information on the above.
7373 Authentication is required for this call.
7375 operations/rmdirs: Remove all the empty directories in the path {#opera‐
7377 tions/rmdirs}
7378 This takes the following parameters
7379 · fs - a remote name string eg “drive:”
7381 · remote - a path within that remote eg “dir”
7383 · leaveRoot - boolean, set to true not to delete the root
7385 See the rmdirs command (https://rclone.org/commands/rclone_rmdirs/)
7387 command for more information on the above.
7388 Authentication is required for this call.
7390 operations/size: Count the number of bytes and files in remote {#opera‐
7392 tions/size}
7393 This takes the following parameters
7394 · fs - a remote name string eg “drive:path/to/dir”
7396 Returns
7398 · count - number of files
7400 · bytes - number of bytes in those files
7402 See the size command (https://rclone.org/commands/rclone_size/) command
7404 for more information on the above.
7405 Authentication is required for this call.
7407 options/blocks: List all the option blocks {#options/blocks}
7409 Returns - options - a list of the options block names
7410 options/get: Get all the options {#options/get}
7412 Returns an object where keys are option block names and values are an
7413 object with the current option values in.
7414 This shows the internal names of the option within rclone which should
7416 map to the external options very easily with a few exceptions.
7417 options/set: Set an option {#options/set}
7419 Parameters
7420 · option block name containing an object with
7422 · key: value
7424 Repeated as often as required.
7426 Only supply the options you wish to change. If an option is unknown it
7428 will be silently ignored. Not all options will have an effect when
7429 changed like this.
7430 For example:
7432 This sets DEBUG level logs (-vv)
7434 rclone rc options/set --json '{"main": {"LogLevel": 8}}'
7436 And this sets INFO level logs (-v)
7438 rclone rc options/set --json '{"main": {"LogLevel": 7}}'
7440 And this sets NOTICE level logs (normal without -v)
7442 rclone rc options/set --json '{"main": {"LogLevel": 6}}'
7444 rc/error: This returns an error {#rc/error}
7446 This returns an error with the input as part of its error string. Use‐
7447 ful for testing error handling.
7448 rc/list: List all the registered remote control commands {#rc/list}
7450 This lists all the registered remote control commands as a JSON map in
7451 the commands response.
7452 rc/noop: Echo the input to the output parameters {#rc/noop}
7454 This echoes the input parameters to the output parameters for testing
7455 purposes. It can be used to check that rclone is still alive and to
7456 check that parameter passing is working properly.
7457 rc/noopauth: Echo the input to the output parameters requiring auth
7459 {#rc/noopauth}
7460 This echoes the input parameters to the output parameters for testing
7461 purposes. It can be used to check that rclone is still alive and to
7462 check that parameter passing is working properly.
7463 Authentication is required for this call.
7465 sync/copy: copy a directory from source remote to destination remote
7467 {#sync/copy}
7468 This takes the following parameters
7469 · srcFs - a remote name string eg “drive:src” for the source
7471 · dstFs - a remote name string eg “drive:dst” for the destination
7473 See the copy command (https://rclone.org/commands/rclone_copy/) command
7475 for more information on the above.
7476 Authentication is required for this call.
7478 sync/move: move a directory from source remote to destination remote
7480 {#sync/move}
7481 This takes the following parameters
7482 · srcFs - a remote name string eg “drive:src” for the source
7484 · dstFs - a remote name string eg “drive:dst” for the destination
7486 · deleteEmptySrcDirs - delete empty src directories if set
7488 See the move command (https://rclone.org/commands/rclone_move/) command
7490 for more information on the above.
7491 Authentication is required for this call.
7493 sync/sync: sync a directory from source remote to destination remote
7495 {#sync/sync}
7496 This takes the following parameters
7497 · srcFs - a remote name string eg “drive:src” for the source
7499 · dstFs - a remote name string eg “drive:dst” for the destination
7501 See the sync command (https://rclone.org/commands/rclone_sync/) command
7503 for more information on the above.
7504 Authentication is required for this call.
7506 vfs/forget: Forget files or directories in the directory cache. {#vfs/for‐
7508 get}
7509 This forgets the paths in the directory cache causing them to be
7510 re-read from the remote when needed.
7511 If no paths are passed in then it will forget all the paths in the di‐
7513 rectory cache.
7514 rclone rc vfs/forget
7516 Otherwise pass files or dirs in as file=path or dir=path. Any parame‐
7518 ter key starting with file will forget that file and any starting with
7519 dir will forget that dir, eg
7520 rclone rc vfs/forget file=hello file2=goodbye dir=home/junk
7522 vfs/poll-interval: Get the status or update the value of the poll-interval
7524 option. {#vfs/poll-interval}
7525 Without any parameter given this returns the current status of the
7526 poll-interval setting.
7527 When the interval=duration parameter is set, the poll-interval value is
7529 updated and the polling function is notified. Setting interval=0 dis‐
7530 ables poll-interval.
7531 rclone rc vfs/poll-interval interval=5m
7533 The timeout=duration parameter can be used to specify a time to wait
7535 for the current poll function to apply the new value. If timeout is
7536 less or equal 0, which is the default, wait indefinitely.
7537 The new poll-interval value will only be active when the timeout is not
7539 reached.
7540 If poll-interval is updated or disabled temporarily, some changes might
7542 not get picked up by the polling function, depending on the used re‐
7543 mote.
7544 vfs/refresh: Refresh the directory cache. {#vfs/refresh}
7546 This reads the directories for the specified paths and freshens the di‐
7547 rectory cache.
7548 If no paths are passed in then it will refresh the root directory.
7550 rclone rc vfs/refresh
7552 Otherwise pass directories in as dir=path. Any parameter key starting
7554 with dir will refresh that directory, eg
7555 rclone rc vfs/refresh dir=home/junk dir2=data/misc
7557 If the parameter recursive=true is given the whole directory tree will
7559 get refreshed. This refresh will use –fast-list if enabled.
7560 Accessing the remote control via HTTP
7562 Rclone implements a simple HTTP based protocol.
7563 Each endpoint takes an JSON object and returns a JSON object or an er‐
7565 ror. The JSON objects are essentially a map of string names to values.
7566 All calls must made using POST.
7568 The input objects can be supplied using URL parameters, POST parameters
7570 or by supplying “Content-Type: application/json” and a JSON blob in the
7571 body. There are examples of these below using curl.
7572 The response will be a JSON blob in the body of the response. This is
7574 formatted to be reasonably human readable.
7575 Error returns
7577 If an error occurs then there will be an HTTP error status (eg 500) and
7578 the body of the response will contain a JSON encoded error object, eg
7579 {
7581 "error": "Expecting string value for key \"remote\" (was float64)",
7582 "input": {
7583 "fs": "/tmp",
7584 "remote": 3
7585 },
7586 "status": 400
7587 "path": "operations/rmdir",
7588 }
7589 The keys in the error response are - error - error string - input - the
7591 input parameters to the call - status - the HTTP status code - path -
7592 the path of the call
7593 CORS
7595 The sever implements basic CORS support and allows all origins for
7596 that. The response to a preflight OPTIONS request will echo the re‐
7597 quested “Access-Control-Request-Headers” back.
7598 Using POST with URL parameters only
7600 curl -X POST 'http://localhost:5572/rc/noop?potato=1&sausage=2'
7601 Response
7603 {
7605 "potato": "1",
7606 "sausage": "2"
7607 }
7608 Here is what an error response looks like:
7610 curl -X POST 'http://localhost:5572/rc/error?potato=1&sausage=2'
7612 {
7614 "error": "arbitrary error on input map[potato:1 sausage:2]",
7615 "input": {
7616 "potato": "1",
7617 "sausage": "2"
7618 }
7619 }
7620 Note that curl doesn’t return errors to the shell unless you use the -f
7622 option
7623 $ curl -f -X POST 'http://localhost:5572/rc/error?potato=1&sausage=2'
7625 curl: (22) The requested URL returned error: 400 Bad Request
7626 $ echo $?
7627 22
7628 Using POST with a form
7630 curl --data "potato=1" --data "sausage=2" http://localhost:5572/rc/noop
7631 Response
7633 {
7635 "potato": "1",
7636 "sausage": "2"
7637 }
7638 Note that you can combine these with URL parameters too with the POST
7640 parameters taking precedence.
7641 curl --data "potato=1" --data "sausage=2" "http://localhost:5572/rc/noop?rutabaga=3&sausage=4"
7643 Response
7645 {
7647 "potato": "1",
7648 "rutabaga": "3",
7649 "sausage": "4"
7650 }
7651 Using POST with a JSON blob
7653 curl -H "Content-Type: application/json" -X POST -d '{"potato":2,"sausage":1}' http://localhost:5572/rc/noop
7654 response
7656 {
7658 "password": "xyz",
7659 "username": "xyz"
7660 }
7661 This can be combined with URL parameters too if required. The JSON
7663 blob takes precedence.
7664 curl -H "Content-Type: application/json" -X POST -d '{"potato":2,"sausage":1}' 'http://localhost:5572/rc/noop?rutabaga=3&potato=4'
7666 {
7668 "potato": 2,
7669 "rutabaga": "3",
7670 "sausage": 1
7671 }
7672 Debugging rclone with pprof
7674 If you use the --rc flag this will also enable the use of the go pro‐
7675 filing tools on the same port.
7676 To use these, first install go (https://golang.org/doc/install).
7678 Debugging memory use
7680 To profile rclone’s memory use you can run:
7681 go tool pprof -web http://localhost:5572/debug/pprof/heap
7683 This should open a page in your browser showing what is using what mem‐
7685 ory.
7686 You can also use the -text flag to produce a textual summary
7688 $ go tool pprof -text http://localhost:5572/debug/pprof/heap
7690 Showing nodes accounting for 1537.03kB, 100% of 1537.03kB total
7691 flat flat% sum% cum cum%
7692 1024.03kB 66.62% 66.62% 1024.03kB 66.62% github.com/rclone/rclone/vendor/golang.org/x/net/http2/hpack.addDecoderNode
7693 513kB 33.38% 100% 513kB 33.38% net/http.newBufioWriterSize
7694 0 0% 100% 1024.03kB 66.62% github.com/rclone/rclone/cmd/all.init
7695 0 0% 100% 1024.03kB 66.62% github.com/rclone/rclone/cmd/serve.init
7696 0 0% 100% 1024.03kB 66.62% github.com/rclone/rclone/cmd/serve/restic.init
7697 0 0% 100% 1024.03kB 66.62% github.com/rclone/rclone/vendor/golang.org/x/net/http2.init
7698 0 0% 100% 1024.03kB 66.62% github.com/rclone/rclone/vendor/golang.org/x/net/http2/hpack.init
7699 0 0% 100% 1024.03kB 66.62% github.com/rclone/rclone/vendor/golang.org/x/net/http2/hpack.init.0
7700 0 0% 100% 1024.03kB 66.62% main.init
7701 0 0% 100% 513kB 33.38% net/http.(*conn).readRequest
7702 0 0% 100% 513kB 33.38% net/http.(*conn).serve
7703 0 0% 100% 1024.03kB 66.62% runtime.main
7704 Debugging go routine leaks
7706 Memory leaks are most often caused by go routine leaks keeping memory
7707 alive which should have been garbage collected.
7708 See all active go routines using
7710 curl http://localhost:5572/debug/pprof/goroutine?debug=1
7712 Or go to http://localhost:5572/debug/pprof/goroutine?debug=1 in your
7714 browser.
7715 Other profiles to look at
7717 You can see a summary of profiles available at http://local‐
7718 host:5572/debug/pprof/
7719 Here is how to use some of them:
7721 · Memory: go tool pprof http://localhost:5572/debug/pprof/heap
7723 · Go routines: curl http://localhost:5572/debug/pprof/goroutine?debug=1
7725 · 30-second CPU profile: go tool pprof http://localhost:5572/de‐
7727 bug/pprof/profile
7728 · 5-second execution trace: wget http://localhost:5572/de‐
7730 bug/pprof/trace?seconds=5
7731 · Goroutine blocking profile
7733 · Enable first with: rclone rc debug/set-block-profile-rate rate=1
7735 (docs)
7736 · go tool pprof http://localhost:5572/debug/pprof/block
7738 · Contended mutexes:
7740 · Enable first with: rclone rc debug/set-mutex-profile-fraction
7742 rate=1 (docs)
7743 · go tool pprof http://localhost:5572/debug/pprof/mutex
7745 See the net/http/pprof docs (https://golang.org/pkg/net/http/pprof/)
7747 for more info on how to use the profiling and for a general overview
7748 see the Go team’s blog post on profiling go programs
7749 (https://blog.golang.org/profiling-go-programs).
7750 The profiling hook is zero overhead unless it is used (https://stack‐
7752 overflow.com/q/26545159/164234).
7753
7755 Each cloud storage system is slightly different. Rclone attempts to
7756 provide a unified interface to them, but some underlying differences
7757 show through.
7758 Features
7760 Here is an overview of the major features of each cloud storage system.
7761 Name Hash ModTime Case Insen‐ Duplicate MIME Type
7763 sitive Files
7764 ──────────────────────────────────────────────────────────────────────────
7765 1Fichier Whirlpool No No Yes R
7766 Amazon MD5 No Yes No R
7767 Drive
7768 Amazon S3 MD5 Yes No No R/W
7769 Backblaze SHA1 Yes No No R/W
7770 B2
7771 Box SHA1 Yes Yes No -
7772 Citrix MD5 Yes Yes No -
7773 ShareFile
7774 Dropbox DBHASH † Yes Yes No -
7775 FTP - No No No -
7777 Google MD5 Yes No No R/W
7778 Cloud Stor‐
7779 age
7780 Google MD5 Yes No Yes R/W
7781 Drive
7782 Google Pho‐ - No No Yes R
7783 tos
7784 HTTP - No No No R
7785 Hubic MD5 Yes No No R/W
7786 Jottacloud MD5 Yes Yes No R/W
7787 Koofr MD5 No Yes No -
7788 Mail.ru Mailru ‡‡‡ Yes Yes No -
7789 Cloud
7790 Mega - No No Yes -
7791 Memory MD5 Yes No No -
7792 Microsoft MD5 Yes No No R/W
7793 Azure Blob
7794 Storage
7795 Microsoft SHA1 ‡‡ Yes Yes No R
7796 OneDrive
7797 OpenDrive MD5 Yes Yes No -
7798 Openstack MD5 Yes No No R/W
7799 Swift
7800 pCloud MD5, SHA1 Yes No No W
7801 premiu‐ - No Yes No R
7802 mize.me
7803 put.io CRC-32 Yes No Yes R
7804 QingStor MD5 No No No R/W
7805 SFTP MD5, SHA1 ‡ Yes Depends No -
7806 SugarSync - No No No -
7807 WebDAV MD5, SHA1 Yes ††† Depends No -
7808 ††
7809 Yandex Disk MD5 Yes No No R/W
7810 The local All Yes Depends No -
7811 filesystem
7812 Hash
7814 The cloud storage system supports various hash types of the objects.
7815 The hashes are used when transferring data as an integrity check and
7816 can be specifically used with the --checksum flag in syncs and in the
7817 check command.
7818 To use the verify checksums when transferring between cloud storage
7820 systems they must support a common hash type.
7821 † Note that Dropbox supports its own custom hash (https://www.drop‐
7823 box.com/developers/reference/content-hash). This is an SHA256 sum of
7824 all the 4MB block SHA256s.
7825 ‡ SFTP supports checksums if the same login has shell access and md5sum
7827 or sha1sum as well as echo are in the remote’s PATH.
7828 †† WebDAV supports hashes when used with Owncloud and Nextcloud only.
7830 ††† WebDAV supports modtimes when used with Owncloud and Nextcloud on‐
7832 ly.
7833 ‡‡ Microsoft OneDrive Personal supports SHA1 hashes, whereas OneDrive
7835 for business and SharePoint server support Microsoft’s own QuickXorHash
7836 (https://docs.microsoft.com/en-us/onedrive/developer/code-snip‐
7837 pets/quickxorhash).
7838 ‡‡‡ Mail.ru uses its own modified SHA1 hash
7840 ModTime
7842 The cloud storage system supports setting modification times on ob‐
7843 jects. If it does then this enables a using the modification times as
7844 part of the sync. If not then only the size will be checked by de‐
7845 fault, though the MD5SUM can be checked with the --checksum flag.
7846 All cloud storage systems support some kind of date on the object and
7848 these will be set when transferring from the cloud storage system.
7849 Case Insensitive
7851 If a cloud storage systems is case sensitive then it is possible to
7852 have two files which differ only in case, eg file.txt and FILE.txt. If
7853 a cloud storage system is case insensitive then that isn’t possible.
7854 This can cause problems when syncing between a case insensitive system
7856 and a case sensitive system. The symptom of this is that no matter how
7857 many times you run the sync it never completes fully.
7858 The local filesystem and SFTP may or may not be case sensitive depend‐
7860 ing on OS.
7861 · Windows - usually case insensitive, though case is preserved
7863 · OSX - usually case insensitive, though it is possible to format case
7865 sensitive
7866 · Linux - usually case sensitive, but there are case insensitive file
7868 systems (eg FAT formatted USB keys)
7869 Most of the time this doesn’t cause any problems as people tend to
7871 avoid files whose name differs only by case even on case sensitive sys‐
7872 tems.
7873 Duplicate files
7875 If a cloud storage system allows duplicate files then it can have two
7876 objects with the same name.
7877 This confuses rclone greatly when syncing - use the rclone dedupe com‐
7879 mand to rename or remove duplicates.
7880 Restricted filenames
7882 Some cloud storage systems might have restrictions on the characters
7883 that are usable in file or directory names. When rclone detects such a
7884 name during a file upload, it will transparently replace the restricted
7885 characters with similar looking Unicode characters.
7886 This process is designed to avoid ambiguous file names as much as pos‐
7888 sible and allow to move files between many cloud storage systems trans‐
7889 parently.
7890 The name shown by rclone to the user or during log output will only
7892 contain a minimal set of replaced characters to ensure correct format‐
7893 ting and not necessarily the actual name used on the cloud storage.
7894 This transformation is reversed when downloading a file or parsing
7896 rclone arguments. For example, when uploading a file named my
7897 file?.txt to Onedrive will be displayed as my file?.txt on the console,
7898 but stored as my file？.txt (the ? gets replaced by the similar looking
7899 ？ character) to Onedrive. The reverse transformation allows to read a
7900 fileunusual/name.txt from Google Drive, by passing the name unusu‐
7901 al／name.txt (the / needs to be replaced by the similar looking ／
7902 character) on the command line.
7903 Default restricted characters
7905 The table below shows the characters that are replaced by default.
7906 When a replacement character is found in a filename, this character
7908 will be escaped with the ‛ character to avoid ambiguous file names.
7909 (e.g. a file named ␀.txt would shown as ‛␀.txt)
7910 Each cloud storage backend can use a different set of characters, which
7912 will be specified in the documentation for each backend.
7913 Character Value Replacement
7915 ────────────────────────────────
7916 NUL 0x00 ␀
7917 SOH 0x01 ␁
7918 STX 0x02 ␂
7919 ETX 0x03 ␃
7920 EOT 0x04 ␄
7921 ENQ 0x05 ␅
7922 ACK 0x06 ␆
7923 BEL 0x07 ␇
7924 BS 0x08 ␈
7925 HT 0x09 ␉
7926 LF 0x0A ␊
7927 VT 0x0B ␋
7928 FF 0x0C ␌
7929 CR 0x0D ␍
7930 SO 0x0E ␎
7931 SI 0x0F ␏
7932 DLE 0x10 ␐
7933 DC1 0x11 ␑
7934 DC2 0x12 ␒
7935 DC3 0x13 ␓
7936 DC4 0x14 ␔
7937 NAK 0x15 ␕
7938 SYN 0x16 ␖
7939 ETB 0x17 ␗
7940 CAN 0x18 ␘
7941 EM 0x19 ␙
7942 SUB 0x1A ␚
7943 ESC 0x1B ␛
7944 FS 0x1C ␜
7945 GS 0x1D ␝
7946 RS 0x1E ␞
7947 US 0x1F ␟
7948 / 0x2F ／
7949 DEL 0x7F ␡
7950 The default encoding will also encode these file names as they are
7952 problematic with many cloud storage systems.
7953 File name Replacement
7955 ────────────────────────
7956 . ．
7957 .. ．．
7958 Invalid UTF-8 bytes
7960 Some backends only support a sequence of well formed UTF-8 bytes as
7961 file or directory names.
7962 In this case all invalid UTF-8 bytes will be replaced with a quoted
7964 representation of the byte value to allow uploading a file to such a
7965 backend. For example, the invalid byte 0xFE will be encoded as ‛FE.
7966 A common source of invalid UTF-8 bytes are local filesystems, that
7968 store names in a different encoding than UTF-8 or UTF-16, like latin1.
7969 See the local filenames (/local/#filenames) section for details.
7970 Encoding option
7972 Most backends have an encoding options, specified as a flag --back‐
7973 end-encoding where backend is the name of the backend, or as a config
7974 parameter encoding (you’ll need to select the Advanced config in rclone
7975 config to see it).
7976 This will have default value which encodes and decodes characters in
7978 such a way as to preserve the maximum number of characters (see above).
7979 However this can be incorrect in some scenarios, for example if you
7981 have a Windows file system with characters such as ＊ and ？ that you
7982 want to remain as those characters on the remote rather than being
7983 translated to * and ?.
7984 The --backend-encoding flags allow you to change that. You can disable
7986 the encoding completely with --backend-encoding None or set encoding =
7987 None in the config file.
7988 Encoding takes a comma separated list of encodings. You can see the
7990 list of all available characters by passing an invalid value to this
7991 flag, eg --local-encoding "help" and rclone help flags encoding will
7992 show you the defaults for the backends.
7993 Encoding Characters
7995 ───────────────────────────────────────────
7996 Asterisk *
7997 BackQuote `
7998 BackSlash \
7999 Colon :
8000 CrLf CR 0x0D, LF 0x0A
8001 Ctl All control characters
8002 0x00-0x1F
8003 Del DEL 0x7F
8004 Dollar $
8005 Dot .
8006 DoubleQuote "
8007 Hash #
8008 InvalidUtf8 An invalid UTF-8 character
8009 (eg latin1)
8010 LeftCrLfHtVt CR 0x0D, LF 0x0A,HT 0x09,
8011 VT 0x0B on the left of a
8012 string
8013 LeftPeriod . on the left of a string
8014 LeftSpace SPACE on the left of a
8015 string
8016 LeftTilde ~ on the left of a string
8017 LtGt <, >
8018 None No characters are encoded
8019 Percent %
8020 Pipe |
8021 Question ?
8022 RightCrLfHtVt CR 0x0D, LF 0x0A, HT 0x09,
8023 VT 0x0B on the right of a
8024 string
8025 RightPeriod . on the right of a string
8026 RightSpace SPACE on the right of a
8027 string
8028 SingleQuote '
8029 Slash /
8030 To take a specific example, the FTP backend’s default encoding is
8032 --ftp-encoding "Slash,Del,Ctl,RightSpace,Dot"
8034 However, let’s say the FTP server is running on Windows and can’t have
8036 any of the invalid Windows characters in file names. You are backing
8037 up Linux servers to this FTP server which do have those characters in
8038 file names. So you would add the Windows set which are
8039 Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,BackSlash,Ctl,RightSpace,RightPeriod,InvalidUtf8,Dot
8041 to the existing ones, giving:
8043 Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,BackSlash,Ctl,RightSpace,RightPeriod,InvalidUtf8,Dot,Del,RightSpace
8045 This can be specified using the --ftp-encoding flag or using an encod‐
8047 ing parameter in the config file.
8048 Or let’s say you have a Windows server but you want to preserve ＊ and
8050 ？, you would then have this as the encoding (the Windows encoding mi‐
8051 nus Asterisk and Question).
8052 Slash,LtGt,DoubleQuote,Colon,Pipe,BackSlash,Ctl,RightSpace,RightPeriod,InvalidUtf8,Dot
8054 This can be specified using the --local-encoding flag or using an en‐
8056 coding parameter in the config file.
8057 MIME Type
8059 MIME types (also known as media types) classify types of documents us‐
8060 ing a simple text classification, eg text/html or application/pdf.
8061 Some cloud storage systems support reading (R) the MIME type of objects
8063 and some support writing (W) the MIME type of objects.
8064 The MIME type can be important if you are serving files directly to
8066 HTTP from the storage system.
8067 If you are copying from a remote which supports reading (R) to a remote
8069 which supports writing (W) then rclone will preserve the MIME types.
8070 Otherwise they will be guessed from the extension, or the remote itself
8071 may assign the MIME type.
8072 Optional Features
8074 All the remotes support a basic set of features, but there are some op‐
8075 tional features supported by some remotes used to make some operations
8076 more efficient.
8077 Name Purge Copy Move DirMove CleanUp ListR StreamU‐ LinkShar‐ About Empty‐
8079 pload ing Dir
8080 ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
8081 1Fichi‐ No No No No No No No No No Yes
8082 er
8083 Amazon Yes No Yes Yes No #575 No No No #2178 No Yes
8084 Drive (https://github.com/rclone/rclone/is‐ (https://github.com/rclone/rclone/is‐
8085 sues/575) sues/2178)
8086 Amazon No Yes No No No Yes Yes No #2178 No No
8087 S3 (https://github.com/rclone/rclone/is‐
8088 sues/2178)
8089 Back‐ No Yes No No Yes Yes Yes Yes No No
8090 blaze
8091 B2
8092 Box Yes Yes Yes Yes No #575 No Yes Yes No Yes
8093 (https://github.com/rclone/rclone/is‐
8094 sues/575)
8095 Citrix Yes Yes Yes Yes No No Yes No No Yes
8096 Share‐
8097 File
8098 Dropbox Yes Yes Yes Yes No #575 No Yes Yes Yes Yes
8099 (https://github.com/rclone/rclone/is‐
8100 sues/575)
8101 FTP No No Yes Yes No No Yes No #2178 No Yes
8102 (https://github.com/rclone/rclone/is‐
8103 sues/2178)
8104 Google Yes Yes No No No Yes Yes No #2178 No No
8105 Cloud (https://github.com/rclone/rclone/is‐
8106 Storage sues/2178)
8107 Google Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes
8108 Drive
8109 Google No No No No No No No No No No
8110 Photos
8111 HTTP No No No No No No No No #2178 No Yes
8112 (https://github.com/rclone/rclone/is‐
8113 sues/2178)
8114 Hubic Yes † Yes No No No Yes Yes No #2178 Yes No
8115 (https://github.com/rclone/rclone/is‐
8116 sues/2178)
8117 Jotta‐ Yes Yes Yes Yes No Yes No Yes Yes Yes
8118 cloud
8119 Mail.ru Yes Yes Yes Yes Yes No No Yes Yes Yes
8120 Cloud
8121 Mega Yes No Yes Yes Yes No No No #2178 Yes Yes
8122 (https://github.com/rclone/rclone/is‐
8123 sues/2178)
8124 Memory No Yes No No No Yes Yes No No No
8125 Micro‐ Yes Yes No No No Yes No No #2178 No No
8126 soft (https://github.com/rclone/rclone/is‐
8127 Azure sues/2178)
8128 Blob
8129 Storage
8130 Micro‐ Yes Yes Yes Yes No #575 No No Yes Yes Yes
8131 soft (https://github.com/rclone/rclone/is‐
8132 OneDrive sues/575)
8133 Open‐ Yes Yes Yes Yes No No No No No Yes
8134 Drive
8135 Open‐ Yes † Yes No No No Yes Yes No #2178 Yes No
8136 stack (https://github.com/rclone/rclone/is‐
8137 Swift sues/2178)
8138 pCloud Yes Yes Yes Yes Yes No No No #2178 Yes Yes
8139 (https://github.com/rclone/rclone/is‐
8140 sues/2178)
8141 premiu‐ Yes No Yes Yes No No No Yes Yes Yes
8142 mize.me
8143 put.io Yes No Yes Yes Yes No Yes No #2178 Yes Yes
8144 (https://github.com/rclone/rclone/is‐
8145 sues/2178)
8146 QingStor No Yes No No No Yes No No #2178 No No
8147 (https://github.com/rclone/rclone/is‐
8148 sues/2178)
8149 SFTP No No Yes Yes No No Yes No #2178 Yes Yes
8150 (https://github.com/rclone/rclone/is‐
8151 sues/2178)
8152 Sug‐ Yes Yes Yes Yes No No Yes Yes No Yes
8153 arSync
8154 WebDAV Yes Yes Yes Yes No No Yes ‡ No #2178 Yes Yes
8155 (https://github.com/rclone/rclone/is‐
8156 sues/2178)
8157 Yandex Yes Yes Yes Yes Yes No Yes Yes Yes Yes
8158 Disk
8159 The lo‐ Yes No Yes Yes No No Yes No Yes Yes
8160 cal
8161 filesys‐
8162 tem
8163 Purge
8165 This deletes a directory quicker than just deleting all the files in
8166 the directory.
8167 † Note Swift and Hubic implement this in order to delete directory
8169 markers but they don’t actually have a quicker way of deleting files
8170 other than deleting them individually.
8171 ‡ StreamUpload is not supported with Nextcloud
8173 Copy
8175 Used when copying an object to and from the same remote. This known as
8176 a server side copy so you can copy a file without downloading it and
8177 uploading it again. It is used if you use rclone copy or rclone move
8178 if the remote doesn’t support Move directly.
8179 If the server doesn’t support Copy directly then for copy operations
8181 the file is downloaded then re-uploaded.
8182 Move
8184 Used when moving/renaming an object on the same remote. This is known
8185 as a server side move of a file. This is used in rclone move if the
8186 server doesn’t support DirMove.
8187 If the server isn’t capable of Move then rclone simulates it with Copy
8189 then delete. If the server doesn’t support Copy then rclone will down‐
8190 load the file and re-upload it.
8191 DirMove
8193 This is used to implement rclone move to move a directory if possible.
8194 If it isn’t then it will use Move on each file (which falls back to
8195 Copy then download and upload - see Move section).
8196 CleanUp
8198 This is used for emptying the trash for a remote by rclone cleanup.
8199 If the server can’t do CleanUp then rclone cleanup will return an er‐
8201 ror.
8202 ListR
8204 The remote supports a recursive list to list all the contents beneath a
8205 directory quickly. This enables the --fast-list flag to work. See the
8206 rclone docs (/docs/#fast-list) for more details.
8207 StreamUpload
8209 Some remotes allow files to be uploaded without knowing the file size
8210 in advance. This allows certain operations to work without spooling
8211 the file to local disk first, e.g. rclone rcat.
8212 LinkSharing
8214 Sets the necessary permissions on a file or folder and prints a link
8215 that allows others to access them, even if they don’t have an account
8216 on the particular cloud provider.
8217 About
8219 This is used to fetch quota information from the remote, like bytes
8220 used/free/quota and bytes used in the trash.
8221 This is also used to return the space used, available for rclone mount.
8223 If the server can’t do About then rclone about will return an error.
8225 EmptyDir
8227 The remote supports empty directories. See Limitations (/bugs/#limita‐
8228 tions) for details. Most Object/Bucket based remotes do not support
8229 this.
8230
8232 This describes the global flags available to every rclone command split
8233 into two groups, non backend and backend flags.
8234 Non Backend Flags
8236 These flags are available for every command.
8237 --ask-password Allow prompt for password for encrypted configuration. (default true)
8239 --auto-confirm If enabled, do not request console confirmation.
8240 --backup-dir string Make backups into hierarchy based in DIR.
8241 --bind string Local address to bind to for outgoing connections, IPv4, IPv6 or name.
8242 --buffer-size SizeSuffix In memory buffer size when reading files for each --transfer. (default 16M)
8243 --bwlimit BwTimetable Bandwidth limit in kBytes/s, or use suffix b|k|M|G or a full timetable.
8244 --ca-cert string CA certificate used to verify servers
8245 --cache-dir string Directory rclone will use for caching. (default "$HOME/.cache/rclone")
8246 --checkers int Number of checkers to run in parallel. (default 8)
8247 -c, --checksum Skip based on checksum (if available) & size, not mod-time & size
8248 --client-cert string Client SSL certificate (PEM) for mutual TLS auth
8249 --client-key string Client SSL private key (PEM) for mutual TLS auth
8250 --compare-dest string Include additional server-side path during comparison.
8251 --config string Config file. (default "$HOME/.config/rclone/rclone.conf")
8252 --contimeout duration Connect timeout (default 1m0s)
8253 --copy-dest string Implies --compare-dest but also copies files from path into destination.
8254 --cpuprofile string Write cpu profile to file
8255 --delete-after When synchronizing, delete files on destination after transferring (default)
8256 --delete-before When synchronizing, delete files on destination before transferring
8257 --delete-during When synchronizing, delete files during transfer
8258 --delete-excluded Delete files on dest excluded from sync
8259 --disable string Disable a comma separated list of features. Use help to see a list.
8260 -n, --dry-run Do a trial run with no permanent changes
8261 --dump DumpFlags List of items to dump from: headers,bodies,requests,responses,auth,filters,goroutines,openfiles
8262 --dump-bodies Dump HTTP headers and bodies - may contain sensitive info
8263 --dump-headers Dump HTTP headers - may contain sensitive info
8264 --exclude stringArray Exclude files matching pattern
8265 --exclude-from stringArray Read exclude patterns from file
8266 --exclude-if-present string Exclude directories if filename is present
8267 --expect-continue-timeout duration Timeout when using expect / 100-continue in HTTP (default 1s)
8268 --fast-list Use recursive list if available. Uses more memory but fewer transactions.
8269 --files-from stringArray Read list of source-file names from file
8270 -f, --filter stringArray Add a file-filtering rule
8271 --filter-from stringArray Read filtering patterns from a file
8272 --ignore-case Ignore case in filters (case insensitive)
8273 --ignore-case-sync Ignore case when synchronizing
8274 --ignore-checksum Skip post copy check of checksums.
8275 --ignore-errors delete even if there are I/O errors
8276 --ignore-existing Skip all files that exist on destination
8277 --ignore-size Ignore size when skipping use mod-time or checksum.
8278 -I, --ignore-times Don't skip files that match size and time - transfer all files
8279 --immutable Do not modify files. Fail if existing files have been modified.
8280 --include stringArray Include files matching pattern
8281 --include-from stringArray Read include patterns from file
8282 --log-file string Log everything to this file
8283 --log-format string Comma separated list of log format options (default "date,time")
8284 --log-level string Log level DEBUG|INFO|NOTICE|ERROR (default "NOTICE")
8285 --low-level-retries int Number of low level retries to do. (default 10)
8286 --max-age Duration Only transfer files younger than this in s or suffix ms|s|m|h|d|w|M|y (default off)
8287 --max-backlog int Maximum number of objects in sync or check backlog. (default 10000)
8288 --max-delete int When synchronizing, limit the number of deletes (default -1)
8289 --max-depth int If set limits the recursion depth to this. (default -1)
8290 --max-duration duration Maximum duration rclone will transfer data for.
8291 --max-size SizeSuffix Only transfer files smaller than this in k or suffix b|k|M|G (default off)
8292 --max-stats-groups int Maximum number of stats groups to keep in memory. On max oldest is discarded. (default 1000)
8293 --max-transfer SizeSuffix Maximum size of data to transfer. (default off)
8294 --memprofile string Write memory profile to file
8295 --min-age Duration Only transfer files older than this in s or suffix ms|s|m|h|d|w|M|y (default off)
8296 --min-size SizeSuffix Only transfer files bigger than this in k or suffix b|k|M|G (default off)
8297 --modify-window duration Max time diff to be considered the same (default 1ns)
8298 --multi-thread-cutoff SizeSuffix Use multi-thread downloads for files above this size. (default 250M)
8299 --multi-thread-streams int Max number of streams to use for multi-thread downloads. (default 4)
8300 --no-check-certificate Do not verify the server SSL certificate. Insecure.
8301 --no-check-dest Don't check the destination, copy regardless.
8302 --no-gzip-encoding Don't set Accept-Encoding: gzip.
8303 --no-traverse Don't traverse destination file system on copy.
8304 --no-update-modtime Don't update destination mod-time if files identical.
8305 --order-by string Instructions on how to order the transfers, eg 'size,descending'
8306 --password-command SpaceSepList Command for supplying password for encrypted configuration.
8307 -P, --progress Show progress during transfer.
8308 -q, --quiet Print as little stuff as possible
8309 --rc Enable the remote control server.
8310 --rc-addr string IPaddress:Port or :Port to bind server to. (default "localhost:5572")
8311 --rc-allow-origin string Set the allowed origin for CORS.
8312 --rc-baseurl string Prefix for URLs - leave blank for root.
8313 --rc-cert string SSL PEM key (concatenation of certificate and CA certificate)
8314 --rc-client-ca string Client certificate authority to verify clients with
8315 --rc-files string Path to local files to serve on the HTTP server.
8316 --rc-htpasswd string htpasswd file - if not provided no authentication is done
8317 --rc-job-expire-duration duration expire finished async jobs older than this value (default 1m0s)
8318 --rc-job-expire-interval duration interval to check for expired async jobs (default 10s)
8319 --rc-key string SSL PEM Private key
8320 --rc-max-header-bytes int Maximum size of request header (default 4096)
8321 --rc-no-auth Don't require auth for certain methods.
8322 --rc-pass string Password for authentication.
8323 --rc-realm string realm for authentication (default "rclone")
8324 --rc-serve Enable the serving of remote objects.
8325 --rc-server-read-timeout duration Timeout for server reading data (default 1h0m0s)
8326 --rc-server-write-timeout duration Timeout for server writing data (default 1h0m0s)
8327 --rc-user string User name for authentication.
8328 --rc-web-fetch-url string URL to fetch the releases for webgui. (default "https://api.github.com/repos/rclone/rclone-webui-react/releases/latest")
8329 --rc-web-gui Launch WebGUI on localhost
8330 --rc-web-gui-force-update Force update to latest version of web gui
8331 --rc-web-gui-no-open-browser Don't open the browser automatically
8332 --rc-web-gui-update Check and update to latest version of web gui
8333 --retries int Retry operations this many times if they fail (default 3)
8334 --retries-sleep duration Interval between retrying operations if they fail, e.g 500ms, 60s, 5m. (0 to disable)
8335 --size-only Skip based on size only, not mod-time or checksum
8336 --stats duration Interval between printing stats, e.g 500ms, 60s, 5m. (0 to disable) (default 1m0s)
8337 --stats-file-name-length int Max file name length in stats. 0 for no limit (default 45)
8338 --stats-log-level string Log level to show --stats output DEBUG|INFO|NOTICE|ERROR (default "INFO")
8339 --stats-one-line Make the stats fit on one line.
8340 --stats-one-line-date Enables --stats-one-line and add current date/time prefix.
8341 --stats-one-line-date-format string Enables --stats-one-line-date and uses custom formatted date. Enclose date string in double quotes ("). See https://golang.org/pkg/time/#Time.Format
8342 --stats-unit string Show data rate in stats as either 'bits' or 'bytes'/s (default "bytes")
8343 --streaming-upload-cutoff SizeSuffix Cutoff for switching to chunked upload if file size is unknown. Upload starts after reaching cutoff or when file ends. (default 100k)
8344 --suffix string Suffix to add to changed files.
8345 --suffix-keep-extension Preserve the extension when using --suffix.
8346 --syslog Use Syslog for logging
8347 --syslog-facility string Facility for syslog, eg KERN,USER,... (default "DAEMON")
8348 --timeout duration IO idle timeout (default 5m0s)
8349 --tpslimit float Limit HTTP transactions per second to this.
8350 --tpslimit-burst int Max burst of transactions for --tpslimit. (default 1)
8351 --track-renames When synchronizing, track file renames and do a server side move if possible
8352 --transfers int Number of file transfers to run in parallel. (default 4)
8353 -u, --update Skip files that are newer on the destination.
8354 --use-cookies Enable session cookiejar.
8355 --use-json-log Use json log format.
8356 --use-mmap Use mmap allocator (see docs).
8357 --use-server-modtime Use server modified time instead of object metadata
8358 --user-agent string Set the user-agent to a specified string. The default is rclone/ version (default "rclone/v1.51.0")
8359 -v, --verbose count Print lots more stuff (repeat for more)
8360 Backend Flags
8362 These flags are available for every command. They control the backends
8363 and may be set in the config file.
8364 --acd-auth-url string Auth server URL.
8366 --acd-client-id string Amazon Application Client ID.
8367 --acd-client-secret string Amazon Application Client Secret.
8368 --acd-encoding MultiEncoder This sets the encoding for the backend. (default Slash,InvalidUtf8,Dot)
8369 --acd-templink-threshold SizeSuffix Files >= this size will be downloaded via their tempLink. (default 9G)
8370 --acd-token-url string Token server url.
8371 --acd-upload-wait-per-gb Duration Additional time per GB to wait after a failed complete upload to see if it appears. (default 3m0s)
8372 --alias-remote string Remote or path to alias.
8373 --azureblob-access-tier string Access tier of blob: hot, cool or archive.
8374 --azureblob-account string Storage Account Name (leave blank to use SAS URL or Emulator)
8375 --azureblob-chunk-size SizeSuffix Upload chunk size (<= 100MB). (default 4M)
8376 --azureblob-encoding MultiEncoder This sets the encoding for the backend. (default Slash,BackSlash,Del,Ctl,RightPeriod,InvalidUtf8)
8377 --azureblob-endpoint string Endpoint for the service
8378 --azureblob-key string Storage Account Key (leave blank to use SAS URL or Emulator)
8379 --azureblob-list-chunk int Size of blob list. (default 5000)
8380 --azureblob-sas-url string SAS URL for container level access only
8381 --azureblob-upload-cutoff SizeSuffix Cutoff for switching to chunked upload (<= 256MB). (default 256M)
8382 --azureblob-use-emulator Uses local storage emulator if provided as 'true' (leave blank if using real azure storage endpoint)
8383 --b2-account string Account ID or Application Key ID
8384 --b2-chunk-size SizeSuffix Upload chunk size. Must fit in memory. (default 96M)
8385 --b2-disable-checksum Disable checksums for large (> upload cutoff) files
8386 --b2-download-auth-duration Duration Time before the authorization token will expire in s or suffix ms|s|m|h|d. (default 1w)
8387 --b2-download-url string Custom endpoint for downloads.
8388 --b2-encoding MultiEncoder This sets the encoding for the backend. (default Slash,BackSlash,Del,Ctl,InvalidUtf8,Dot)
8389 --b2-endpoint string Endpoint for the service.
8390 --b2-hard-delete Permanently delete files on remote removal, otherwise hide files.
8391 --b2-key string Application Key
8392 --b2-test-mode string A flag string for X-Bz-Test-Mode header for debugging.
8393 --b2-upload-cutoff SizeSuffix Cutoff for switching to chunked upload. (default 200M)
8394 --b2-versions Include old versions in directory listings.
8395 --box-box-config-file string Box App config.json location
8396 --box-box-sub-type string (default "user")
8397 --box-client-id string Box App Client Id.
8398 --box-client-secret string Box App Client Secret
8399 --box-commit-retries int Max number of times to try committing a multipart file. (default 100)
8400 --box-encoding MultiEncoder This sets the encoding for the backend. (default Slash,BackSlash,Del,Ctl,RightSpace,InvalidUtf8,Dot)
8401 --box-upload-cutoff SizeSuffix Cutoff for switching to multipart upload (>= 50MB). (default 50M)
8402 --cache-chunk-clean-interval Duration How often should the cache perform cleanups of the chunk storage. (default 1m0s)
8403 --cache-chunk-no-memory Disable the in-memory cache for storing chunks during streaming.
8404 --cache-chunk-path string Directory to cache chunk files. (default "$HOME/.cache/rclone/cache-backend")
8405 --cache-chunk-size SizeSuffix The size of a chunk (partial file data). (default 5M)
8406 --cache-chunk-total-size SizeSuffix The total size that the chunks can take up on the local disk. (default 10G)
8407 --cache-db-path string Directory to store file structure metadata DB. (default "$HOME/.cache/rclone/cache-backend")
8408 --cache-db-purge Clear all the cached data for this remote on start.
8409 --cache-db-wait-time Duration How long to wait for the DB to be available - 0 is unlimited (default 1s)
8410 --cache-info-age Duration How long to cache file structure information (directory listings, file size, times etc). (default 6h0m0s)
8411 --cache-plex-insecure string Skip all certificate verifications when connecting to the Plex server
8412 --cache-plex-password string The password of the Plex user
8413 --cache-plex-url string The URL of the Plex server
8414 --cache-plex-username string The username of the Plex user
8415 --cache-read-retries int How many times to retry a read from a cache storage. (default 10)
8416 --cache-remote string Remote to cache.
8417 --cache-rps int Limits the number of requests per second to the source FS (-1 to disable) (default -1)
8418 --cache-tmp-upload-path string Directory to keep temporary files until they are uploaded.
8419 --cache-tmp-wait-time Duration How long should files be stored in local cache before being uploaded (default 15s)
8420 --cache-workers int How many workers should run in parallel to download chunks. (default 4)
8421 --cache-writes Cache file data on writes through the FS
8422 --chunker-chunk-size SizeSuffix Files larger than chunk size will be split in chunks. (default 2G)
8423 --chunker-fail-hard Choose how chunker should handle files with missing or invalid chunks.
8424 --chunker-hash-type string Choose how chunker handles hash sums. All modes but "none" require metadata. (default "md5")
8425 --chunker-meta-format string Format of the metadata object or "none". By default "simplejson". (default "simplejson")
8426 --chunker-name-format string String format of chunk file names. (default "*.rclone_chunk.###")
8427 --chunker-remote string Remote to chunk/unchunk.
8428 --chunker-start-from int Minimum valid chunk number. Usually 0 or 1. (default 1)
8429 -L, --copy-links Follow symlinks and copy the pointed to item.
8430 --crypt-directory-name-encryption Option to either encrypt directory names or leave them intact. (default true)
8431 --crypt-filename-encryption string How to encrypt the filenames. (default "standard")
8432 --crypt-password string Password or pass phrase for encryption.
8433 --crypt-password2 string Password or pass phrase for salt. Optional but recommended.
8434 --crypt-remote string Remote to encrypt/decrypt.
8435 --crypt-show-mapping For all files listed show how the names encrypt.
8436 --drive-acknowledge-abuse Set to allow files which return cannotDownloadAbusiveFile to be downloaded.
8437 --drive-allow-import-name-change Allow the filetype to change when uploading Google docs (e.g. file.doc to file.docx). This will confuse sync and reupload every time.
8438 --drive-alternate-export Use alternate export URLs for google documents export.,
8439 --drive-auth-owner-only Only consider files owned by the authenticated user.
8440 --drive-chunk-size SizeSuffix Upload chunk size. Must a power of 2 >= 256k. (default 8M)
8441 --drive-client-id string Google Application Client Id
8442 --drive-client-secret string Google Application Client Secret
8443 --drive-disable-http2 Disable drive using http2 (default true)
8444 --drive-encoding MultiEncoder This sets the encoding for the backend. (default InvalidUtf8)
8445 --drive-export-formats string Comma separated list of preferred formats for downloading Google docs. (default "docx,xlsx,pptx,svg")
8446 --drive-formats string Deprecated: see export_formats
8447 --drive-impersonate string Impersonate this user when using a service account.
8448 --drive-import-formats string Comma separated list of preferred formats for uploading Google docs.
8449 --drive-keep-revision-forever Keep new head revision of each file forever.
8450 --drive-list-chunk int Size of listing chunk 100-1000. 0 to disable. (default 1000)
8451 --drive-pacer-burst int Number of API calls to allow without sleeping. (default 100)
8452 --drive-pacer-min-sleep Duration Minimum time to sleep between API calls. (default 100ms)
8453 --drive-root-folder-id string ID of the root folder
8454 --drive-scope string Scope that rclone should use when requesting access from drive.
8455 --drive-server-side-across-configs Allow server side operations (eg copy) to work across different drive configs.
8456 --drive-service-account-credentials string Service Account Credentials JSON blob
8457 --drive-service-account-file string Service Account Credentials JSON file path
8458 --drive-shared-with-me Only show files that are shared with me.
8459 --drive-size-as-quota Show sizes as storage quota usage, not actual size.
8460 --drive-skip-checksum-gphotos Skip MD5 checksum on Google photos and videos only.
8461 --drive-skip-gdocs Skip google documents in all listings.
8462 --drive-stop-on-upload-limit Make upload limit errors be fatal
8463 --drive-team-drive string ID of the Team Drive
8464 --drive-trashed-only Only show files that are in the trash.
8465 --drive-upload-cutoff SizeSuffix Cutoff for switching to chunked upload (default 8M)
8466 --drive-use-created-date Use file created date instead of modified date.,
8467 --drive-use-shared-date Use date file was shared instead of modified date.
8468 --drive-use-trash Send files to the trash instead of deleting permanently. (default true)
8469 --drive-v2-download-min-size SizeSuffix If Object's are greater, use drive v2 API to download. (default off)
8470 --dropbox-chunk-size SizeSuffix Upload chunk size. (< 150M). (default 48M)
8471 --dropbox-client-id string Dropbox App Client Id
8472 --dropbox-client-secret string Dropbox App Client Secret
8473 --dropbox-encoding MultiEncoder This sets the encoding for the backend. (default Slash,BackSlash,Del,RightSpace,InvalidUtf8,Dot)
8474 --dropbox-impersonate string Impersonate this user when using a business account.
8475 --fichier-api-key string Your API Key, get it from https://1fichier.com/console/params.pl
8476 --fichier-encoding MultiEncoder This sets the encoding for the backend. (default Slash,LtGt,DoubleQuote,SingleQuote,BackQuote,Dollar,BackSlash,Del,Ctl,LeftSpace,RightSpace,InvalidUtf8,Dot)
8477 --fichier-shared-folder string If you want to download a shared folder, add this parameter
8478 --ftp-concurrency int Maximum number of FTP simultaneous connections, 0 for unlimited
8479 --ftp-disable-epsv Disable using EPSV even if server advertises support
8480 --ftp-encoding MultiEncoder This sets the encoding for the backend. (default Slash,Del,Ctl,RightSpace,Dot)
8481 --ftp-host string FTP host to connect to
8482 --ftp-no-check-certificate Do not verify the TLS certificate of the server
8483 --ftp-pass string FTP password
8484 --ftp-port string FTP port, leave blank to use default (21)
8485 --ftp-tls Use FTP over TLS (Implicit)
8486 --ftp-user string FTP username, leave blank for current username, $USER
8487 --gcs-bucket-acl string Access Control List for new buckets.
8488 --gcs-bucket-policy-only Access checks should use bucket-level IAM policies.
8489 --gcs-client-id string Google Application Client Id
8490 --gcs-client-secret string Google Application Client Secret
8491 --gcs-encoding MultiEncoder This sets the encoding for the backend. (default Slash,CrLf,InvalidUtf8,Dot)
8492 --gcs-location string Location for the newly created buckets.
8493 --gcs-object-acl string Access Control List for new objects.
8494 --gcs-project-number string Project number.
8495 --gcs-service-account-file string Service Account Credentials JSON file path
8496 --gcs-storage-class string The storage class to use when storing objects in Google Cloud Storage.
8497 --gphotos-client-id string Google Application Client Id
8498 --gphotos-client-secret string Google Application Client Secret
8499 --gphotos-read-only Set to make the Google Photos backend read only.
8500 --gphotos-read-size Set to read the size of media items.
8501 --http-headers CommaSepList Set HTTP headers for all transactions
8502 --http-no-head Don't use HEAD requests to find file sizes in dir listing
8503 --http-no-slash Set this if the site doesn't end directories with /
8504 --http-url string URL of http host to connect to
8505 --hubic-chunk-size SizeSuffix Above this size files will be chunked into a _segments container. (default 5G)
8506 --hubic-client-id string Hubic Client Id
8507 --hubic-client-secret string Hubic Client Secret
8508 --hubic-encoding MultiEncoder This sets the encoding for the backend. (default Slash,InvalidUtf8)
8509 --hubic-no-chunk Don't chunk files during streaming upload.
8510 --jottacloud-encoding MultiEncoder This sets the encoding for the backend. (default Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,Del,Ctl,InvalidUtf8,Dot)
8511 --jottacloud-hard-delete Delete files permanently rather than putting them into the trash.
8512 --jottacloud-md5-memory-limit SizeSuffix Files bigger than this will be cached on disk to calculate the MD5 if required. (default 10M)
8513 --jottacloud-unlink Remove existing public link to file/folder with link command rather than creating.
8514 --jottacloud-upload-resume-limit SizeSuffix Files bigger than this can be resumed if the upload fail's. (default 10M)
8515 --koofr-encoding MultiEncoder This sets the encoding for the backend. (default Slash,BackSlash,Del,Ctl,InvalidUtf8,Dot)
8516 --koofr-endpoint string The Koofr API endpoint to use (default "https://app.koofr.net")
8517 --koofr-mountid string Mount ID of the mount to use. If omitted, the primary mount is used.
8518 --koofr-password string Your Koofr password for rclone (generate one at https://app.koofr.net/app/admin/preferences/password)
8519 --koofr-setmtime Does the backend support setting modification time. Set this to false if you use a mount ID that points to a Dropbox or Amazon Drive backend. (default true)
8520 --koofr-user string Your Koofr user name
8521 -l, --links Translate symlinks to/from regular files with a '.rclonelink' extension
8522 --local-case-insensitive Force the filesystem to report itself as case insensitive
8523 --local-case-sensitive Force the filesystem to report itself as case sensitive.
8524 --local-encoding MultiEncoder This sets the encoding for the backend. (default Slash,Dot)
8525 --local-no-check-updated Don't check to see if the files change during upload
8526 --local-no-unicode-normalization Don't apply unicode normalization to paths and filenames (Deprecated)
8527 --local-nounc string Disable UNC (long path names) conversion on Windows
8528 --mailru-check-hash What should copy do if file checksum is mismatched or invalid (default true)
8529 --mailru-encoding MultiEncoder This sets the encoding for the backend. (default Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,BackSlash,Del,Ctl,InvalidUtf8,Dot)
8530 --mailru-pass string Password
8531 --mailru-speedup-enable Skip full upload if there is another file with same data hash. (default true)
8532 --mailru-speedup-file-patterns string Comma separated list of file name patterns eligible for speedup (put by hash). (default "*.mkv,*.avi,*.mp4,*.mp3,*.zip,*.gz,*.rar,*.pdf")
8533 --mailru-speedup-max-disk SizeSuffix This option allows you to disable speedup (put by hash) for large files (default 3G)
8534 --mailru-speedup-max-memory SizeSuffix Files larger than the size given below will always be hashed on disk. (default 32M)
8535 --mailru-user string User name (usually email)
8536 --mega-debug Output more debug from Mega.
8537 --mega-encoding MultiEncoder This sets the encoding for the backend. (default Slash,InvalidUtf8,Dot)
8538 --mega-hard-delete Delete files permanently rather than putting them into the trash.
8539 --mega-pass string Password.
8540 --mega-user string User name
8541 -x, --one-file-system Don't cross filesystem boundaries (unix/macOS only).
8542 --onedrive-chunk-size SizeSuffix Chunk size to upload files with - must be multiple of 320k (327,680 bytes). (default 10M)
8543 --onedrive-client-id string Microsoft App Client Id
8544 --onedrive-client-secret string Microsoft App Client Secret
8545 --onedrive-drive-id string The ID of the drive to use
8546 --onedrive-drive-type string The type of the drive ( personal | business | documentLibrary )
8547 --onedrive-encoding MultiEncoder This sets the encoding for the backend. (default Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,Hash,Percent,BackSlash,Del,Ctl,LeftSpace,LeftTilde,RightSpace,RightPeriod,InvalidUtf8,Dot)
8548 --onedrive-expose-onenote-files Set to make OneNote files show up in directory listings.
8549 --opendrive-chunk-size SizeSuffix Files will be uploaded in chunks this size. (default 10M)
8550 --opendrive-encoding MultiEncoder This sets the encoding for the backend. (default Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,BackSlash,LeftSpace,LeftCrLfHtVt,RightSpace,RightCrLfHtVt,InvalidUtf8,Dot)
8551 --opendrive-password string Password.
8552 --opendrive-username string Username
8553 --pcloud-client-id string Pcloud App Client Id
8554 --pcloud-client-secret string Pcloud App Client Secret
8555 --pcloud-encoding MultiEncoder This sets the encoding for the backend. (default Slash,BackSlash,Del,Ctl,InvalidUtf8,Dot)
8556 --premiumizeme-encoding MultiEncoder This sets the encoding for the backend. (default Slash,DoubleQuote,BackSlash,Del,Ctl,InvalidUtf8,Dot)
8557 --putio-encoding MultiEncoder This sets the encoding for the backend. (default Slash,BackSlash,Del,Ctl,InvalidUtf8,Dot)
8558 --qingstor-access-key-id string QingStor Access Key ID
8559 --qingstor-chunk-size SizeSuffix Chunk size to use for uploading. (default 4M)
8560 --qingstor-connection-retries int Number of connection retries. (default 3)
8561 --qingstor-encoding MultiEncoder This sets the encoding for the backend. (default Slash,Ctl,InvalidUtf8)
8562 --qingstor-endpoint string Enter a endpoint URL to connection QingStor API.
8563 --qingstor-env-auth Get QingStor credentials from runtime. Only applies if access_key_id and secret_access_key is blank.
8564 --qingstor-secret-access-key string QingStor Secret Access Key (password)
8565 --qingstor-upload-concurrency int Concurrency for multipart uploads. (default 1)
8566 --qingstor-upload-cutoff SizeSuffix Cutoff for switching to chunked upload (default 200M)
8567 --qingstor-zone string Zone to connect to.
8568 --s3-access-key-id string AWS Access Key ID.
8569 --s3-acl string Canned ACL used when creating buckets and storing or copying objects.
8570 --s3-bucket-acl string Canned ACL used when creating buckets.
8571 --s3-chunk-size SizeSuffix Chunk size to use for uploading. (default 5M)
8572 --s3-copy-cutoff SizeSuffix Cutoff for switching to multipart copy (default 5G)
8573 --s3-disable-checksum Don't store MD5 checksum with object metadata
8574 --s3-encoding MultiEncoder This sets the encoding for the backend. (default Slash,InvalidUtf8,Dot)
8575 --s3-endpoint string Endpoint for S3 API.
8576 --s3-env-auth Get AWS credentials from runtime (environment variables or EC2/ECS meta data if no env vars).
8577 --s3-force-path-style If true use path style access if false use virtual hosted style. (default true)
8578 --s3-leave-parts-on-error If true avoid calling abort upload on a failure, leaving all successfully uploaded parts on S3 for manual recovery.
8579 --s3-list-chunk int Size of listing chunk (response list for each ListObject S3 request). (default 1000)
8580 --s3-location-constraint string Location constraint - must be set to match the Region.
8581 --s3-provider string Choose your S3 provider.
8582 --s3-region string Region to connect to.
8583 --s3-secret-access-key string AWS Secret Access Key (password)
8584 --s3-server-side-encryption string The server-side encryption algorithm used when storing this object in S3.
8585 --s3-session-token string An AWS session token
8586 --s3-sse-kms-key-id string If using KMS ID you must provide the ARN of Key.
8587 --s3-storage-class string The storage class to use when storing new objects in S3.
8588 --s3-upload-concurrency int Concurrency for multipart uploads. (default 4)
8589 --s3-upload-cutoff SizeSuffix Cutoff for switching to chunked upload (default 200M)
8590 --s3-use-accelerate-endpoint If true use the AWS S3 accelerated endpoint.
8591 --s3-v2-auth If true use v2 authentication.
8592 --sftp-ask-password Allow asking for SFTP password when needed.
8593 --sftp-disable-hashcheck Disable the execution of SSH commands to determine if remote file hashing is available.
8594 --sftp-host string SSH host to connect to
8595 --sftp-key-file string Path to PEM-encoded private key file, leave blank or set key-use-agent to use ssh-agent.
8596 --sftp-key-file-pass string The passphrase to decrypt the PEM-encoded private key file.
8597 --sftp-key-use-agent When set forces the usage of the ssh-agent.
8598 --sftp-md5sum-command string The command used to read md5 hashes. Leave blank for autodetect.
8599 --sftp-pass string SSH password, leave blank to use ssh-agent.
8600 --sftp-path-override string Override path used by SSH connection.
8601 --sftp-port string SSH port, leave blank to use default (22)
8602 --sftp-set-modtime Set the modified time on the remote if set. (default true)
8603 --sftp-sha1sum-command string The command used to read sha1 hashes. Leave blank for autodetect.
8604 --sftp-skip-links Set to skip any symlinks and any other non regular files.
8605 --sftp-use-insecure-cipher Enable the use of insecure ciphers and key exchange methods.
8606 --sftp-user string SSH username, leave blank for current username, ncw
8607 --sharefile-chunk-size SizeSuffix Upload chunk size. Must a power of 2 >= 256k. (default 64M)
8608 --sharefile-encoding MultiEncoder This sets the encoding for the backend. (default Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,BackSlash,Ctl,LeftSpace,LeftPeriod,RightSpace,RightPeriod,InvalidUtf8,Dot)
8609 --sharefile-endpoint string Endpoint for API calls.
8610 --sharefile-root-folder-id string ID of the root folder
8611 --sharefile-upload-cutoff SizeSuffix Cutoff for switching to multipart upload. (default 128M)
8612 --skip-links Don't warn about skipped symlinks.
8613 --sugarsync-access-key-id string Sugarsync Access Key ID.
8614 --sugarsync-app-id string Sugarsync App ID.
8615 --sugarsync-authorization string Sugarsync authorization
8616 --sugarsync-authorization-expiry string Sugarsync authorization expiry
8617 --sugarsync-deleted-id string Sugarsync deleted folder id
8618 --sugarsync-encoding MultiEncoder This sets the encoding for the backend. (default Slash,Ctl,InvalidUtf8,Dot)
8619 --sugarsync-hard-delete Permanently delete files if true
8620 --sugarsync-private-access-key string Sugarsync Private Access Key
8621 --sugarsync-refresh-token string Sugarsync refresh token
8622 --sugarsync-root-id string Sugarsync root id
8623 --sugarsync-user string Sugarsync user
8624 --swift-application-credential-id string Application Credential ID (OS_APPLICATION_CREDENTIAL_ID)
8625 --swift-application-credential-name string Application Credential Name (OS_APPLICATION_CREDENTIAL_NAME)
8626 --swift-application-credential-secret string Application Credential Secret (OS_APPLICATION_CREDENTIAL_SECRET)
8627 --swift-auth string Authentication URL for server (OS_AUTH_URL).
8628 --swift-auth-token string Auth Token from alternate authentication - optional (OS_AUTH_TOKEN)
8629 --swift-auth-version int AuthVersion - optional - set to (1,2,3) if your auth URL has no version (ST_AUTH_VERSION)
8630 --swift-chunk-size SizeSuffix Above this size files will be chunked into a _segments container. (default 5G)
8631 --swift-domain string User domain - optional (v3 auth) (OS_USER_DOMAIN_NAME)
8632 --swift-encoding MultiEncoder This sets the encoding for the backend. (default Slash,InvalidUtf8)
8633 --swift-endpoint-type string Endpoint type to choose from the service catalogue (OS_ENDPOINT_TYPE) (default "public")
8634 --swift-env-auth Get swift credentials from environment variables in standard OpenStack form.
8635 --swift-key string API key or password (OS_PASSWORD).
8636 --swift-no-chunk Don't chunk files during streaming upload.
8637 --swift-region string Region name - optional (OS_REGION_NAME)
8638 --swift-storage-policy string The storage policy to use when creating a new container
8639 --swift-storage-url string Storage URL - optional (OS_STORAGE_URL)
8640 --swift-tenant string Tenant name - optional for v1 auth, this or tenant_id required otherwise (OS_TENANT_NAME or OS_PROJECT_NAME)
8641 --swift-tenant-domain string Tenant domain - optional (v3 auth) (OS_PROJECT_DOMAIN_NAME)
8642 --swift-tenant-id string Tenant ID - optional for v1 auth, this or tenant required otherwise (OS_TENANT_ID)
8643 --swift-user string User name to log in (OS_USERNAME).
8644 --swift-user-id string User ID to log in - optional - most swift systems use user and leave this blank (v3 auth) (OS_USER_ID).
8645 --union-remotes string List of space separated remotes.
8646 --webdav-bearer-token string Bearer token instead of user/pass (eg a Macaroon)
8647 --webdav-bearer-token-command string Command to run to get a bearer token
8648 --webdav-pass string Password.
8649 --webdav-url string URL of http host to connect to
8650 --webdav-user string User name
8651 --webdav-vendor string Name of the Webdav site/service/software you are using
8652 --yandex-client-id string Yandex Client Id
8653 --yandex-client-secret string Yandex Client Secret
8654 --yandex-encoding MultiEncoder This sets the encoding for the backend. (default Slash,Del,Ctl,InvalidUtf8,Dot)
8655 --yandex-unlink Remove existing public link to file/folder with link command rather than creating.
8656 1Fichier
8658 This is a backend for the 1ficher (https://1fichier.com) cloud storage
8659 service. Note that a Premium subscription is required to use the API.
8660 Paths are specified as remote:path
8662 Paths may be as deep as required, eg remote:directory/subdirectory.
8664 The initial setup for 1Fichier involves getting the API key from the
8666 website which you need to do in your browser.
8667 Here is an example of how to make a remote called remote. First run:
8669 rclone config
8671 This will guide you through an interactive setup process:
8673 No remotes found - make a new one
8675 n) New remote
8676 s) Set configuration password
8677 q) Quit config
8678 n/s/q> n
8679 name> remote
8680 Type of storage to configure.
8681 Enter a string value. Press Enter for the default ("").
8682 Choose a number from below, or type in your own value
8683 [snip]
8684 XX / 1Fichier
8685 \ "fichier"
8686 [snip]
8687 Storage> fichier
8688 ** See help for fichier backend at: https://rclone.org/fichier/ **
8689 Your API Key, get it from https://1fichier.com/console/params.pl
8691 Enter a string value. Press Enter for the default ("").
8692 api_key> example_key
8693 Edit advanced config? (y/n)
8695 y) Yes
8696 n) No
8697 y/n>
8698 Remote config
8699 --------------------
8700 [remote]
8701 type = fichier
8702 api_key = example_key
8703 --------------------
8704 y) Yes this is OK
8705 e) Edit this remote
8706 d) Delete this remote
8707 y/e/d> y
8708 Once configured you can then use rclone like this,
8710 List directories in top level of your 1Fichier account
8712 rclone lsd remote:
8714 List all the files in your 1Fichier account
8716 rclone ls remote:
8718 To copy a local directory to a 1Fichier directory called backup
8720 rclone copy /home/source remote:backup
8722 Modified time and hashes
8724 1Fichier does not support modification times. It supports the
8725 Whirlpool hash algorithm.
8726 Duplicated files
8728 1Fichier can have two files with exactly the same name and path (unlike
8729 a normal file system).
8730 Duplicated files cause problems with the syncing and you will see mes‐
8732 sages in the log about duplicates.
8733 Restricted filename characters
8735 In addition to the default restricted characters set (/overview/#re‐
8736 stricted-characters) the following characters are also replaced:
8737 Character Value Replacement
8739 ────────────────────────────────
8740 \ 0x5C ＼
8741 < 0x3C ＜
8742 > 0x3E ＞
8743 " 0x22 ＂
8744 $ 0x24 ＄
8745 ` 0x60 ｀
8746 ’ 0x27 ＇
8747 File names can also not start or end with the following characters.
8749 These only get replaced if they are first or last character in the
8750 name:
8751 Character Value Replacement
8753 ────────────────────────────────
8754 SP 0x20 ␠
8755 Invalid UTF-8 bytes will also be replaced (/overview/#invalid-utf8), as
8757 they can’t be used in JSON strings.
8758 Standard Options
8760 Here are the standard options specific to fichier (1Fichier).
8761 –fichier-api-key
8763 Your API Key, get it from https://1fichier.com/console/params.pl
8764 · Config: api_key
8766 · Env Var: RCLONE_FICHIER_API_KEY
8768 · Type: string
8770 · Default: ""
8772 Advanced Options
8774 Here are the advanced options specific to fichier (1Fichier).
8775 –fichier-shared-folder
8777 If you want to download a shared folder, add this parameter
8778 · Config: shared_folder
8780 · Env Var: RCLONE_FICHIER_SHARED_FOLDER
8782 · Type: string
8784 · Default: ""
8786 –fichier-encoding
8788 This sets the encoding for the backend.
8789 See: the encoding section in the overview (/overview/#encoding) for
8791 more info.
8792 · Config: encoding
8794 · Env Var: RCLONE_FICHIER_ENCODING
8796 · Type: MultiEncoder
8798 · Default: Slash,LtGt,DoubleQuote,SingleQuote,BackQuote,Dollar,Back‐
8800 Slash,Del,Ctl,LeftSpace,RightSpace,InvalidUtf8,Dot
8801 Alias
8803 The alias remote provides a new name for another remote.
8804 Paths may be as deep as required or a local path, eg remote:directo‐
8806 ry/subdirectory or /directory/subdirectory.
8807 During the initial setup with rclone config you will specify the target
8809 remote. The target remote can either be a local path or another re‐
8810 mote.
8811 Subfolders can be used in target remote. Assume a alias remote named
8813 backup with the target mydrive:private/backup. Invoking rclone mkdir
8814 backup:desktop is exactly the same as invoking rclone mkdir my‐
8815 drive:private/backup/desktop.
8816 There will be no special handling of paths containing .. segments.
8818 Invoking rclone mkdir backup:../desktop is exactly the same as invoking
8819 rclone mkdir mydrive:private/backup/../desktop. The empty path is not
8820 allowed as a remote. To alias the current directory use . instead.
8821 Here is an example of how to make a alias called remote for local fold‐
8823 er. First run:
8824 rclone config
8826 This will guide you through an interactive setup process:
8828 No remotes found - make a new one
8830 n) New remote
8831 s) Set configuration password
8832 q) Quit config
8833 n/s/q> n
8834 name> remote
8835 Type of storage to configure.
8836 Choose a number from below, or type in your own value
8837 [snip]
8838 XX / Alias for an existing remote
8839 \ "alias"
8840 [snip]
8841 Storage> alias
8842 Remote or path to alias.
8843 Can be "myremote:path/to/dir", "myremote:bucket", "myremote:" or "/local/path".
8844 remote> /mnt/storage/backup
8845 Remote config
8846 --------------------
8847 [remote]
8848 remote = /mnt/storage/backup
8849 --------------------
8850 y) Yes this is OK
8851 e) Edit this remote
8852 d) Delete this remote
8853 y/e/d> y
8854 Current remotes:
8855 Name Type
8857 ==== ====
8858 remote alias
8859 e) Edit existing remote
8861 n) New remote
8862 d) Delete remote
8863 r) Rename remote
8864 c) Copy remote
8865 s) Set configuration password
8866 q) Quit config
8867 e/n/d/r/c/s/q> q
8868 Once configured you can then use rclone like this,
8870 List directories in top level in /mnt/storage/backup
8872 rclone lsd remote:
8874 List all the files in /mnt/storage/backup
8876 rclone ls remote:
8878 Copy another local directory to the alias directory called source
8880 rclone copy /home/source remote:source
8882 Standard Options
8884 Here are the standard options specific to alias (Alias for an existing
8885 remote).
8886 –alias-remote
8888 Remote or path to alias. Can be “myremote:path/to/dir”, “myre‐
8889 mote:bucket”, “myremote:” or “/local/path”.
8890 · Config: remote
8892 · Env Var: RCLONE_ALIAS_REMOTE
8894 · Type: string
8896 · Default: ""
8898 Amazon Drive
8900 Amazon Drive, formerly known as Amazon Cloud Drive, is a cloud storage
8901 service run by Amazon for consumers.
8902 Status
8904 Important: rclone supports Amazon Drive only if you have your own set
8905 of API keys. Unfortunately the Amazon Drive developer program
8906 (https://developer.amazon.com/amazon-drive) is now closed to new en‐
8907 tries so if you don’t already have your own set of keys you will not be
8908 able to use rclone with Amazon Drive.
8909 For the history on why rclone no longer has a set of Amazon Drive API
8911 keys see the forum (https://forum.rclone.org/t/rclone-has-been-banned-
8912 from-amazon-drive/2314).
8913 If you happen to know anyone who works at Amazon then please ask them
8915 to re-instate rclone into the Amazon Drive developer program - thanks!
8916 Setup
8918 The initial setup for Amazon Drive involves getting a token from Amazon
8919 which you need to do in your browser. rclone config walks you through
8920 it.
8921 The configuration process for Amazon Drive may involve using an oauth
8923 proxy (https://github.com/ncw/oauthproxy). This is used to keep the
8924 Amazon credentials out of the source code. The proxy runs in Google’s
8925 very secure App Engine environment and doesn’t store any credentials
8926 which pass through it.
8927 Since rclone doesn’t currently have its own Amazon Drive credentials so
8929 you will either need to have your own client_id and client_secret with
8930 Amazon Drive, or use a a third party ouath proxy in which case you will
8931 need to enter client_id, client_secret, auth_url and token_url.
8932 Note also if you are not using Amazon’s auth_url and token_url, (ie you
8934 filled in something for those) then if setting up on a remote machine
8935 you can only use the copying the config method of configuration
8936 (https://rclone.org/remote_setup/#configuring-by-copying-the-config-
8937 file) - rclone authorize will not work.
8938 Here is an example of how to make a remote called remote. First run:
8940 rclone config
8942 This will guide you through an interactive setup process:
8944 No remotes found - make a new one
8946 n) New remote
8947 r) Rename remote
8948 c) Copy remote
8949 s) Set configuration password
8950 q) Quit config
8951 n/r/c/s/q> n
8952 name> remote
8953 Type of storage to configure.
8954 Choose a number from below, or type in your own value
8955 [snip]
8956 XX / Amazon Drive
8957 \ "amazon cloud drive"
8958 [snip]
8959 Storage> amazon cloud drive
8960 Amazon Application Client Id - required.
8961 client_id> your client ID goes here
8962 Amazon Application Client Secret - required.
8963 client_secret> your client secret goes here
8964 Auth server URL - leave blank to use Amazon's.
8965 auth_url> Optional auth URL
8966 Token server url - leave blank to use Amazon's.
8967 token_url> Optional token URL
8968 Remote config
8969 Make sure your Redirect URL is set to "http://127.0.0.1:53682/" in your custom config.
8970 Use auto config?
8971 * Say Y if not sure
8972 * Say N if you are working on a remote or headless machine
8973 y) Yes
8974 n) No
8975 y/n> y
8976 If your browser doesn't open automatically go to the following link: http://127.0.0.1:53682/auth
8977 Log in and authorize rclone for access
8978 Waiting for code...
8979 Got code
8980 --------------------
8981 [remote]
8982 client_id = your client ID goes here
8983 client_secret = your client secret goes here
8984 auth_url = Optional auth URL
8985 token_url = Optional token URL
8986 token = {"access_token":"xxxxxxxxxxxxxxxxxxxxxxx","token_type":"bearer","refresh_token":"xxxxxxxxxxxxxxxxxx","expiry":"2015-09-06T16:07:39.658438471+01:00"}
8987 --------------------
8988 y) Yes this is OK
8989 e) Edit this remote
8990 d) Delete this remote
8991 y/e/d> y
8992 See the remote setup docs (https://rclone.org/remote_setup/) for how to
8994 set it up on a machine with no Internet browser available.
8995 Note that rclone runs a webserver on your local machine to collect the
8997 token as returned from Amazon. This only runs from the moment it opens
8998 your browser to the moment you get back the verification code. This is
8999 on http://127.0.0.1:53682/ and this it may require you to unblock it
9000 temporarily if you are running a host firewall.
9001 Once configured you can then use rclone like this,
9003 List directories in top level of your Amazon Drive
9005 rclone lsd remote:
9007 List all the files in your Amazon Drive
9009 rclone ls remote:
9011 To copy a local directory to an Amazon Drive directory called backup
9013 rclone copy /home/source remote:backup
9015 Modified time and MD5SUMs
9017 Amazon Drive doesn’t allow modification times to be changed via the API
9018 so these won’t be accurate or used for syncing.
9019 It does store MD5SUMs so for a more accurate sync, you can use the
9021 --checksum flag.
9022 Restricted filename characters
9024 Character Value Replacement
9025 ────────────────────────────────
9026 NUL 0x00 ␀
9027 / 0x2F ／
9028 Invalid UTF-8 bytes will also be replaced (/overview/#invalid-utf8), as
9030 they can’t be used in JSON strings.
9031 Deleting files
9033 Any files you delete with rclone will end up in the trash. Amazon
9034 don’t provide an API to permanently delete files, nor to empty the
9035 trash, so you will have to do that with one of Amazon’s apps or via the
9036 Amazon Drive website. As of November 17, 2016, files are automatically
9037 deleted by Amazon from the trash after 30 days.
9038 Using with non .com Amazon accounts
9040 Let’s say you usually use amazon.co.uk. When you authenticate with
9041 rclone it will take you to an amazon.com page to log in. Your ama‐
9042 zon.co.uk email and password should work here just fine.
9043 Standard Options
9045 Here are the standard options specific to amazon cloud drive (Amazon
9046 Drive).
9047 –acd-client-id
9049 Amazon Application Client ID.
9050 · Config: client_id
9052 · Env Var: RCLONE_ACD_CLIENT_ID
9054 · Type: string
9056 · Default: ""
9058 –acd-client-secret
9060 Amazon Application Client Secret.
9061 · Config: client_secret
9063 · Env Var: RCLONE_ACD_CLIENT_SECRET
9065 · Type: string
9067 · Default: ""
9069 Advanced Options
9071 Here are the advanced options specific to amazon cloud drive (Amazon
9072 Drive).
9073 –acd-auth-url
9075 Auth server URL. Leave blank to use Amazon’s.
9076 · Config: auth_url
9078 · Env Var: RCLONE_ACD_AUTH_URL
9080 · Type: string
9082 · Default: ""
9084 –acd-token-url
9086 Token server url. leave blank to use Amazon’s.
9087 · Config: token_url
9089 · Env Var: RCLONE_ACD_TOKEN_URL
9091 · Type: string
9093 · Default: ""
9095 –acd-checkpoint
9097 Checkpoint for internal polling (debug).
9098 · Config: checkpoint
9100 · Env Var: RCLONE_ACD_CHECKPOINT
9102 · Type: string
9104 · Default: ""
9106 –acd-upload-wait-per-gb
9108 Additional time per GB to wait after a failed complete upload to see if
9109 it appears.
9110 Sometimes Amazon Drive gives an error when a file has been fully up‐
9112 loaded but the file appears anyway after a little while. This happens
9113 sometimes for files over 1GB in size and nearly every time for files
9114 bigger than 10GB. This parameter controls the time rclone waits for
9115 the file to appear.
9116 The default value for this parameter is 3 minutes per GB, so by default
9118 it will wait 3 minutes for every GB uploaded to see if the file ap‐
9119 pears.
9120 You can disable this feature by setting it to 0. This may cause con‐
9122 flict errors as rclone retries the failed upload but the file will most
9123 likely appear correctly eventually.
9124 These values were determined empirically by observing lots of uploads
9126 of big files for a range of file sizes.
9127 Upload with the “-v” flag to see more info about what rclone is doing
9129 in this situation.
9130 · Config: upload_wait_per_gb
9132 · Env Var: RCLONE_ACD_UPLOAD_WAIT_PER_GB
9134 · Type: Duration
9136 · Default: 3m0s
9138 –acd-templink-threshold
9140 Files >= this size will be downloaded via their tempLink.
9141 Files this size or more will be downloaded via their “tempLink”. This
9143 is to work around a problem with Amazon Drive which blocks downloads of
9144 files bigger than about 10GB. The default for this is 9GB which
9145 shouldn’t need to be changed.
9146 To download files above this threshold, rclone requests a “tempLink”
9148 which downloads the file through a temporary URL directly from the un‐
9149 derlying S3 storage.
9150 · Config: templink_threshold
9152 · Env Var: RCLONE_ACD_TEMPLINK_THRESHOLD
9154 · Type: SizeSuffix
9156 · Default: 9G
9158 –acd-encoding
9160 This sets the encoding for the backend.
9161 See: the encoding section in the overview (/overview/#encoding) for
9163 more info.
9164 · Config: encoding
9166 · Env Var: RCLONE_ACD_ENCODING
9168 · Type: MultiEncoder
9170 · Default: Slash,InvalidUtf8,Dot
9172 Limitations
9174 Note that Amazon Drive is case insensitive so you can’t have a file
9175 called “Hello.doc” and one called “hello.doc”.
9176 Amazon Drive has rate limiting so you may notice errors in the sync
9178 (429 errors). rclone will automatically retry the sync up to 3 times
9179 by default (see --retries flag) which should hopefully work around this
9180 problem.
9181 Amazon Drive has an internal limit of file sizes that can be uploaded
9183 to the service. This limit is not officially published, but all files
9184 larger than this will fail.
9185 At the time of writing (Jan 2016) is in the area of 50GB per file.
9187 This means that larger files are likely to fail.
9188 Unfortunately there is no way for rclone to see that this failure is
9190 because of file size, so it will retry the operation, as any other
9191 failure. To avoid this problem, use --max-size 50000M option to limit
9192 the maximum size of uploaded files. Note that --max-size does not
9193 split files into segments, it only ignores files over this size.
9194 Amazon S3 Storage Providers
9196 The S3 backend can be used with a number of different providers:
9197 · AWS S3
9199 · Alibaba Cloud (Aliyun) Object Storage System (OSS)
9201 · Ceph
9203 · DigitalOcean Spaces
9205 · Dreamhost
9207 · IBM COS S3
9209 · Minio
9211 · Scaleway
9213 · Wasabi
9215 Paths are specified as remote:bucket (or remote: for the lsd command.)
9217 You may put subdirectories in too, eg remote:bucket/path/to/dir.
9218 Once you have made a remote (see the provider specific section above)
9220 you can use it like this:
9221 See all buckets
9223 rclone lsd remote:
9225 Make a new bucket
9227 rclone mkdir remote:bucket
9229 List the contents of a bucket
9231 rclone ls remote:bucket
9233 Sync /home/local/directory to the remote bucket, deleting any excess
9235 files in the bucket.
9236 rclone sync /home/local/directory remote:bucket
9238 AWS S3
9240 Here is an example of making an s3 configuration. First run
9241 rclone config
9243 This will guide you through an interactive setup process.
9245 No remotes found - make a new one
9247 n) New remote
9248 s) Set configuration password
9249 q) Quit config
9250 n/s/q> n
9251 name> remote
9252 Type of storage to configure.
9253 Choose a number from below, or type in your own value
9254 [snip]
9255 XX / Amazon S3 Compliant Storage Providers (AWS, Ceph, Dreamhost, IBM COS, Minio)
9256 \ "s3"
9257 [snip]
9258 Storage> s3
9259 Choose your S3 provider.
9260 Choose a number from below, or type in your own value
9261 1 / Amazon Web Services (AWS) S3
9262 \ "AWS"
9263 2 / Ceph Object Storage
9264 \ "Ceph"
9265 3 / Digital Ocean Spaces
9266 \ "DigitalOcean"
9267 4 / Dreamhost DreamObjects
9268 \ "Dreamhost"
9269 5 / IBM COS S3
9270 \ "IBMCOS"
9271 6 / Minio Object Storage
9272 \ "Minio"
9273 7 / Wasabi Object Storage
9274 \ "Wasabi"
9275 8 / Any other S3 compatible provider
9276 \ "Other"
9277 provider> 1
9278 Get AWS credentials from runtime (environment variables or EC2/ECS meta data if no env vars). Only applies if access_key_id and secret_access_key is blank.
9279 Choose a number from below, or type in your own value
9280 1 / Enter AWS credentials in the next step
9281 \ "false"
9282 2 / Get AWS credentials from the environment (env vars or IAM)
9283 \ "true"
9284 env_auth> 1
9285 AWS Access Key ID - leave blank for anonymous access or runtime credentials.
9286 access_key_id> XXX
9287 AWS Secret Access Key (password) - leave blank for anonymous access or runtime credentials.
9288 secret_access_key> YYY
9289 Region to connect to.
9290 Choose a number from below, or type in your own value
9291 / The default endpoint - a good choice if you are unsure.
9292 1 | US Region, Northern Virginia or Pacific Northwest.
9293 | Leave location constraint empty.
9294 \ "us-east-1"
9295 / US East (Ohio) Region
9296 2 | Needs location constraint us-east-2.
9297 \ "us-east-2"
9298 / US West (Oregon) Region
9299 3 | Needs location constraint us-west-2.
9300 \ "us-west-2"
9301 / US West (Northern California) Region
9302 4 | Needs location constraint us-west-1.
9303 \ "us-west-1"
9304 / Canada (Central) Region
9305 5 | Needs location constraint ca-central-1.
9306 \ "ca-central-1"
9307 / EU (Ireland) Region
9308 6 | Needs location constraint EU or eu-west-1.
9309 \ "eu-west-1"
9310 / EU (London) Region
9311 7 | Needs location constraint eu-west-2.
9312 \ "eu-west-2"
9313 / EU (Frankfurt) Region
9314 8 | Needs location constraint eu-central-1.
9315 \ "eu-central-1"
9316 / Asia Pacific (Singapore) Region
9317 9 | Needs location constraint ap-southeast-1.
9318 \ "ap-southeast-1"
9319 / Asia Pacific (Sydney) Region
9320 10 | Needs location constraint ap-southeast-2.
9321 \ "ap-southeast-2"
9322 / Asia Pacific (Tokyo) Region
9323 11 | Needs location constraint ap-northeast-1.
9324 \ "ap-northeast-1"
9325 / Asia Pacific (Seoul)
9326 12 | Needs location constraint ap-northeast-2.
9327 \ "ap-northeast-2"
9328 / Asia Pacific (Mumbai)
9329 13 | Needs location constraint ap-south-1.
9330 \ "ap-south-1"
9331 / Asia Patific (Hong Kong) Region
9332 14 | Needs location constraint ap-east-1.
9333 \ "ap-east-1"
9334 / South America (Sao Paulo) Region
9335 15 | Needs location constraint sa-east-1.
9336 \ "sa-east-1"
9337 region> 1
9338 Endpoint for S3 API.
9339 Leave blank if using AWS to use the default endpoint for the region.
9340 endpoint>
9341 Location constraint - must be set to match the Region. Used when creating buckets only.
9342 Choose a number from below, or type in your own value
9343 1 / Empty for US Region, Northern Virginia or Pacific Northwest.
9344 \ ""
9345 2 / US East (Ohio) Region.
9346 \ "us-east-2"
9347 3 / US West (Oregon) Region.
9348 \ "us-west-2"
9349 4 / US West (Northern California) Region.
9350 \ "us-west-1"
9351 5 / Canada (Central) Region.
9352 \ "ca-central-1"
9353 6 / EU (Ireland) Region.
9354 \ "eu-west-1"
9355 7 / EU (London) Region.
9356 \ "eu-west-2"
9357 8 / EU Region.
9358 \ "EU"
9359 9 / Asia Pacific (Singapore) Region.
9360 \ "ap-southeast-1"
9361 10 / Asia Pacific (Sydney) Region.
9362 \ "ap-southeast-2"
9363 11 / Asia Pacific (Tokyo) Region.
9364 \ "ap-northeast-1"
9365 12 / Asia Pacific (Seoul)
9366 \ "ap-northeast-2"
9367 13 / Asia Pacific (Mumbai)
9368 \ "ap-south-1"
9369 14 / Asia Pacific (Hong Kong)
9370 \ "ap-east-1"
9371 15 / South America (Sao Paulo) Region.
9372 \ "sa-east-1"
9373 location_constraint> 1
9374 Canned ACL used when creating buckets and/or storing objects in S3.
9375 For more info visit https://docs.aws.amazon.com/AmazonS3/latest/dev/acl-overview.html#canned-acl
9376 Choose a number from below, or type in your own value
9377 1 / Owner gets FULL_CONTROL. No one else has access rights (default).
9378 \ "private"
9379 2 / Owner gets FULL_CONTROL. The AllUsers group gets READ access.
9380 \ "public-read"
9381 / Owner gets FULL_CONTROL. The AllUsers group gets READ and WRITE access.
9382 3 | Granting this on a bucket is generally not recommended.
9383 \ "public-read-write"
9384 4 / Owner gets FULL_CONTROL. The AuthenticatedUsers group gets READ access.
9385 \ "authenticated-read"
9386 / Object owner gets FULL_CONTROL. Bucket owner gets READ access.
9387 5 | If you specify this canned ACL when creating a bucket, Amazon S3 ignores it.
9388 \ "bucket-owner-read"
9389 / Both the object owner and the bucket owner get FULL_CONTROL over the object.
9390 6 | If you specify this canned ACL when creating a bucket, Amazon S3 ignores it.
9391 \ "bucket-owner-full-control"
9392 acl> 1
9393 The server-side encryption algorithm used when storing this object in S3.
9394 Choose a number from below, or type in your own value
9395 1 / None
9396 \ ""
9397 2 / AES256
9398 \ "AES256"
9399 server_side_encryption> 1
9400 The storage class to use when storing objects in S3.
9401 Choose a number from below, or type in your own value
9402 1 / Default
9403 \ ""
9404 2 / Standard storage class
9405 \ "STANDARD"
9406 3 / Reduced redundancy storage class
9407 \ "REDUCED_REDUNDANCY"
9408 4 / Standard Infrequent Access storage class
9409 \ "STANDARD_IA"
9410 5 / One Zone Infrequent Access storage class
9411 \ "ONEZONE_IA"
9412 6 / Glacier storage class
9413 \ "GLACIER"
9414 7 / Glacier Deep Archive storage class
9415 \ "DEEP_ARCHIVE"
9416 8 / Intelligent-Tiering storage class
9417 \ "INTELLIGENT_TIERING"
9418 storage_class> 1
9419 Remote config
9420 --------------------
9421 [remote]
9422 type = s3
9423 provider = AWS
9424 env_auth = false
9425 access_key_id = XXX
9426 secret_access_key = YYY
9427 region = us-east-1
9428 endpoint =
9429 location_constraint =
9430 acl = private
9431 server_side_encryption =
9432 storage_class =
9433 --------------------
9434 y) Yes this is OK
9435 e) Edit this remote
9436 d) Delete this remote
9437 y/e/d>
9438 –fast-list
9440 This remote supports --fast-list which allows you to use fewer transac‐
9441 tions in exchange for more memory. See the rclone docs (/docs/#fast-
9442 list) for more details.
9443 –update and –use-server-modtime
9445 As noted below, the modified time is stored on metadata on the object.
9446 It is used by default for all operations that require checking the time
9447 a file was last updated. It allows rclone to treat the remote more
9448 like a true filesystem, but it is inefficient because it requires an
9449 extra API call to retrieve the metadata.
9450 For many operations, the time the object was last uploaded to the re‐
9452 mote is sufficient to determine if it is “dirty”. By using --update
9453 along with --use-server-modtime, you can avoid the extra API call and
9454 simply upload files whose local modtime is newer than the time it was
9455 last uploaded.
9456 Modified time
9458 The modified time is stored as metadata on the object as
9459 X-Amz-Meta-Mtime as floating point since the epoch accurate to 1 ns.
9460 If the modification time needs to be updated rclone will attempt to
9462 perform a server side copy to update the modification if the object can
9463 be copied in a single part. In the case the object is larger than 5Gb
9464 or is in Glacier or Glacier Deep Archive storage the object will be up‐
9465 loaded rather than copied.
9466 Restricted filename characters
9468 S3 allows any valid UTF-8 string as a key.
9469 Invalid UTF-8 bytes will be replaced (/overview/#invalid-utf8), as they
9471 can’t be used in XML.
9472 The following characters are replaced since these are problematic when
9474 dealing with the REST API:
9475 Character Value Replacement
9477 ────────────────────────────────
9478 NUL 0x00 ␀
9479 / 0x2F ／
9480 The encoding will also encode these file names as they don’t seem to
9482 work with the SDK properly:
9483 File name Replacement
9485 ────────────────────────
9486 . ．
9487 .. ．．
9488 Multipart uploads
9490 rclone supports multipart uploads with S3 which means that it can up‐
9491 load files bigger than 5GB.
9492 Note that files uploaded both with multipart upload and through crypt
9494 remotes do not have MD5 sums.
9495 rclone switches from single part uploads to multipart uploads at the
9497 point specified by --s3-upload-cutoff. This can be a maximum of 5GB
9498 and a minimum of 0 (ie always upload multipart files).
9499 The chunk sizes used in the multipart upload are specified by
9501 --s3-chunk-size and the number of chunks uploaded concurrently is spec‐
9502 ified by --s3-upload-concurrency.
9503 Multipart uploads will use --transfers * --s3-upload-concurrency *
9505 --s3-chunk-size extra memory. Single part uploads to not use extra
9506 memory.
9507 Single part transfers can be faster than multipart transfers or slower
9509 depending on your latency from S3 - the more latency, the more likely
9510 single part transfers will be faster.
9511 Increasing --s3-upload-concurrency will increase throughput (8 would be
9513 a sensible value) and increasing --s3-chunk-size also increases
9514 throughput (16M would be sensible). Increasing either of these will
9515 use more memory. The default values are high enough to gain most of
9516 the possible performance without using too much memory.
9517 Buckets and Regions
9519 With Amazon S3 you can list buckets (rclone lsd) using any region, but
9520 you can only access the content of a bucket from the region it was cre‐
9521 ated in. If you attempt to access a bucket from the wrong region, you
9522 will get an error, incorrect region, the bucket is not in 'XXX' region.
9523 Authentication
9525 There are a number of ways to supply rclone with a set of AWS creden‐
9526 tials, with and without using the environment.
9527 The different authentication methods are tried in this order:
9529 · Directly in the rclone configuration file (env_auth = false in the
9531 config file):
9532 · access_key_id and secret_access_key are required.
9534 · session_token can be optionally set when using AWS STS.
9536 · Runtime configuration (env_auth = true in the config file):
9538 · Export the following environment variables before running rclone:
9540 · Access Key ID: AWS_ACCESS_KEY_ID or AWS_ACCESS_KEY
9542 · Secret Access Key: AWS_SECRET_ACCESS_KEY or AWS_SECRET_KEY
9544 · Session Token: AWS_SESSION_TOKEN (optional)
9546 · Or, use a named profile (https://docs.aws.amazon.com/cli/lat‐
9548 est/userguide/cli-multiple-profiles.html):
9549 · Profile files are standard files used by AWS CLI tools
9551 · By default it will use the profile in your home directory (eg
9553 ~/.aws/credentials on unix based systems) file and the “default”
9554 profile, to change set these environment variables:
9555 · AWS_SHARED_CREDENTIALS_FILE to control which file.
9557 · AWS_PROFILE to control which profile to use.
9559 · Or, run rclone in an ECS task with an IAM role (AWS only).
9561 · Or, run rclone on an EC2 instance with an IAM role (AWS only).
9563 · Or, run rclone in an EKS pod with an IAM role that is associated
9565 with a service account (AWS only).
9566 If none of these option actually end up providing rclone with AWS cre‐
9568 dentials then S3 interaction will be non-authenticated (see below).
9569 S3 Permissions
9571 When using the sync subcommand of rclone the following minimum permis‐
9572 sions are required to be available on the bucket being written to:
9573 · ListBucket
9575 · DeleteObject
9577 · GetObject
9579 · PutObject
9581 · PutObjectACL
9583 When using the lsd subcommand, the ListAllMyBuckets permission is re‐
9585 quired.
9586 Example policy:
9588 {
9590 "Version": "2012-10-17",
9591 "Statement": [
9592 {
9593 "Effect": "Allow",
9594 "Principal": {
9595 "AWS": "arn:aws:iam::USER_SID:user/USER_NAME"
9596 },
9597 "Action": [
9598 "s3:ListBucket",
9599 "s3:DeleteObject",
9600 "s3:GetObject",
9601 "s3:PutObject",
9602 "s3:PutObjectAcl"
9603 ],
9604 "Resource": [
9605 "arn:aws:s3:::BUCKET_NAME/*",
9606 "arn:aws:s3:::BUCKET_NAME"
9607 ]
9608 },
9609 {
9610 "Effect": "Allow",
9611 "Action": "s3:ListAllMyBuckets",
9612 "Resource": "arn:aws:s3:::*"
9613 }
9614 ]
9615 }
9616 Notes on above:
9618 1. This is a policy that can be used when creating bucket. It assumes
9620 that USER_NAME has been created.
9621 2. The Resource entry must include both resource ARNs, as one implies
9623 the bucket and the other implies the bucket’s objects.
9624 For reference, here’s an Ansible script
9626 (https://gist.github.com/ebridges/ebfc9042dd7c756cd101cfa807b7ae2b)
9627 that will generate one or more buckets that will work with rclone sync.
9628 Key Management System (KMS)
9630 If you are using server side encryption with KMS then you will find you
9631 can’t transfer small objects. As a work-around you can use the --ig‐
9632 nore-checksum flag.
9633 A proper fix is being worked on in issue #1824
9635 (https://github.com/rclone/rclone/issues/1824).
9636 Glacier and Glacier Deep Archive
9638 You can upload objects using the glacier storage class or transition
9639 them to glacier using a lifecycle policy (http://docs.aws.ama‐
9640 zon.com/AmazonS3/latest/user-guide/create-lifecycle.html). The bucket
9641 can still be synced or copied into normally, but if rclone tries to ac‐
9642 cess data from the glacier storage class you will see an error like be‐
9643 low.
9644 2017/09/11 19:07:43 Failed to sync: failed to open source object: Object in GLACIER, restore first: path/to/file
9646 In this case you need to restore (http://docs.aws.amazon.com/Ama‐
9648 zonS3/latest/user-guide/restore-archived-objects.html) the object(s) in
9649 question before using rclone.
9650 Note that rclone only speaks the S3 API it does not speak the Glacier
9652 Vault API, so rclone cannot directly access Glacier Vaults.
9653 Standard Options
9655 Here are the standard options specific to s3 (Amazon S3 Compliant Stor‐
9656 age Provider (AWS, Alibaba, Ceph, Digital Ocean, Dreamhost, IBM COS,
9657 Minio, etc)).
9658 –s3-provider
9660 Choose your S3 provider.
9661 · Config: provider
9663 · Env Var: RCLONE_S3_PROVIDER
9665 · Type: string
9667 · Default: ""
9669 · Examples:
9671 · “AWS”
9673 · Amazon Web Services (AWS) S3
9675 · “Alibaba”
9677 · Alibaba Cloud Object Storage System (OSS) formerly Aliyun
9679 · “Ceph”
9681 · Ceph Object Storage
9683 · “DigitalOcean”
9685 · Digital Ocean Spaces
9687 · “Dreamhost”
9689 · Dreamhost DreamObjects
9691 · “IBMCOS”
9693 · IBM COS S3
9695 · “Minio”
9697 · Minio Object Storage
9699 · “Netease”
9701 · Netease Object Storage (NOS)
9703 · “Wasabi”
9705 · Wasabi Object Storage
9707 · “Other”
9709 · Any other S3 compatible provider
9711 –s3-env-auth
9713 Get AWS credentials from runtime (environment variables or EC2/ECS meta
9714 data if no env vars). Only applies if access_key_id and secret_ac‐
9715 cess_key is blank.
9716 · Config: env_auth
9718 · Env Var: RCLONE_S3_ENV_AUTH
9720 · Type: bool
9722 · Default: false
9724 · Examples:
9726 · “false”
9728 · Enter AWS credentials in the next step
9730 · “true”
9732 · Get AWS credentials from the environment (env vars or IAM)
9734 –s3-access-key-id
9736 AWS Access Key ID. Leave blank for anonymous access or runtime creden‐
9737 tials.
9738 · Config: access_key_id
9740 · Env Var: RCLONE_S3_ACCESS_KEY_ID
9742 · Type: string
9744 · Default: ""
9746 –s3-secret-access-key
9748 AWS Secret Access Key (password) Leave blank for anonymous access or
9749 runtime credentials.
9750 · Config: secret_access_key
9752 · Env Var: RCLONE_S3_SECRET_ACCESS_KEY
9754 · Type: string
9756 · Default: ""
9758 –s3-region
9760 Region to connect to.
9761 · Config: region
9763 · Env Var: RCLONE_S3_REGION
9765 · Type: string
9767 · Default: ""
9769 · Examples:
9771 · “us-east-1”
9773 · The default endpoint - a good choice if you are unsure.
9775 · US Region, Northern Virginia or Pacific Northwest.
9777 · Leave location constraint empty.
9779 · “us-east-2”
9781 · US East (Ohio) Region
9783 · Needs location constraint us-east-2.
9785 · “us-west-2”
9787 · US West (Oregon) Region
9789 · Needs location constraint us-west-2.
9791 · “us-west-1”
9793 · US West (Northern California) Region
9795 · Needs location constraint us-west-1.
9797 · “ca-central-1”
9799 · Canada (Central) Region
9801 · Needs location constraint ca-central-1.
9803 · “eu-west-1”
9805 · EU (Ireland) Region
9807 · Needs location constraint EU or eu-west-1.
9809 · “eu-west-2”
9811 · EU (London) Region
9813 · Needs location constraint eu-west-2.
9815 · “eu-north-1”
9817 · EU (Stockholm) Region
9819 · Needs location constraint eu-north-1.
9821 · “eu-central-1”
9823 · EU (Frankfurt) Region
9825 · Needs location constraint eu-central-1.
9827 · “ap-southeast-1”
9829 · Asia Pacific (Singapore) Region
9831 · Needs location constraint ap-southeast-1.
9833 · “ap-southeast-2”
9835 · Asia Pacific (Sydney) Region
9837 · Needs location constraint ap-southeast-2.
9839 · “ap-northeast-1”
9841 · Asia Pacific (Tokyo) Region
9843 · Needs location constraint ap-northeast-1.
9845 · “ap-northeast-2”
9847 · Asia Pacific (Seoul)
9849 · Needs location constraint ap-northeast-2.
9851 · “ap-south-1”
9853 · Asia Pacific (Mumbai)
9855 · Needs location constraint ap-south-1.
9857 · “ap-east-1”
9859 · Asia Patific (Hong Kong) Region
9861 · Needs location constraint ap-east-1.
9863 · “sa-east-1”
9865 · South America (Sao Paulo) Region
9867 · Needs location constraint sa-east-1.
9869 –s3-region
9871 Region to connect to. Leave blank if you are using an S3 clone and you
9872 don’t have a region.
9873 · Config: region
9875 · Env Var: RCLONE_S3_REGION
9877 · Type: string
9879 · Default: ""
9881 · Examples:
9883 · ""
9885 · Use this if unsure. Will use v4 signatures and an empty region.
9887 · “other-v2-signature”
9889 · Use this only if v4 signatures don’t work, eg pre Jewel/v10 CEPH.
9891 –s3-endpoint
9893 Endpoint for S3 API. Leave blank if using AWS to use the default end‐
9894 point for the region.
9895 · Config: endpoint
9897 · Env Var: RCLONE_S3_ENDPOINT
9899 · Type: string
9901 · Default: ""
9903 –s3-endpoint
9905 Endpoint for IBM COS S3 API. Specify if using an IBM COS On Premise.
9906 · Config: endpoint
9908 · Env Var: RCLONE_S3_ENDPOINT
9910 · Type: string
9912 · Default: ""
9914 · Examples:
9916 · “s3-api.us-geo.objectstorage.softlayer.net”
9918 · US Cross Region Endpoint
9920 · “s3-api.dal.us-geo.objectstorage.softlayer.net”
9922 · US Cross Region Dallas Endpoint
9924 · “s3-api.wdc-us-geo.objectstorage.softlayer.net”
9926 · US Cross Region Washington DC Endpoint
9928 · “s3-api.sjc-us-geo.objectstorage.softlayer.net”
9930 · US Cross Region San Jose Endpoint
9932 · “s3-api.us-geo.objectstorage.service.networklayer.com”
9934 · US Cross Region Private Endpoint
9936 · “s3-api.dal-us-geo.objectstorage.service.networklayer.com”
9938 · US Cross Region Dallas Private Endpoint
9940 · “s3-api.wdc-us-geo.objectstorage.service.networklayer.com”
9942 · US Cross Region Washington DC Private Endpoint
9944 · “s3-api.sjc-us-geo.objectstorage.service.networklayer.com”
9946 · US Cross Region San Jose Private Endpoint
9948 · “s3.us-east.objectstorage.softlayer.net”
9950 · US Region East Endpoint
9952 · “s3.us-east.objectstorage.service.networklayer.com”
9954 · US Region East Private Endpoint
9956 · “s3.us-south.objectstorage.softlayer.net”
9958 · US Region South Endpoint
9960 · “s3.us-south.objectstorage.service.networklayer.com”
9962 · US Region South Private Endpoint
9964 · “s3.eu-geo.objectstorage.softlayer.net”
9966 · EU Cross Region Endpoint
9968 · “s3.fra-eu-geo.objectstorage.softlayer.net”
9970 · EU Cross Region Frankfurt Endpoint
9972 · “s3.mil-eu-geo.objectstorage.softlayer.net”
9974 · EU Cross Region Milan Endpoint
9976 · “s3.ams-eu-geo.objectstorage.softlayer.net”
9978 · EU Cross Region Amsterdam Endpoint
9980 · “s3.eu-geo.objectstorage.service.networklayer.com”
9982 · EU Cross Region Private Endpoint
9984 · “s3.fra-eu-geo.objectstorage.service.networklayer.com”
9986 · EU Cross Region Frankfurt Private Endpoint
9988 · “s3.mil-eu-geo.objectstorage.service.networklayer.com”
9990 · EU Cross Region Milan Private Endpoint
9992 · “s3.ams-eu-geo.objectstorage.service.networklayer.com”
9994 · EU Cross Region Amsterdam Private Endpoint
9996 · “s3.eu-gb.objectstorage.softlayer.net”
9998 · Great Britain Endpoint
10000 · “s3.eu-gb.objectstorage.service.networklayer.com”
10002 · Great Britain Private Endpoint
10004 · “s3.ap-geo.objectstorage.softlayer.net”
10006 · APAC Cross Regional Endpoint
10008 · “s3.tok-ap-geo.objectstorage.softlayer.net”
10010 · APAC Cross Regional Tokyo Endpoint
10012 · “s3.hkg-ap-geo.objectstorage.softlayer.net”
10014 · APAC Cross Regional HongKong Endpoint
10016 · “s3.seo-ap-geo.objectstorage.softlayer.net”
10018 · APAC Cross Regional Seoul Endpoint
10020 · “s3.ap-geo.objectstorage.service.networklayer.com”
10022 · APAC Cross Regional Private Endpoint
10024 · “s3.tok-ap-geo.objectstorage.service.networklayer.com”
10026 · APAC Cross Regional Tokyo Private Endpoint
10028 · “s3.hkg-ap-geo.objectstorage.service.networklayer.com”
10030 · APAC Cross Regional HongKong Private Endpoint
10032 · “s3.seo-ap-geo.objectstorage.service.networklayer.com”
10034 · APAC Cross Regional Seoul Private Endpoint
10036 · “s3.mel01.objectstorage.softlayer.net”
10038 · Melbourne Single Site Endpoint
10040 · “s3.mel01.objectstorage.service.networklayer.com”
10042 · Melbourne Single Site Private Endpoint
10044 · “s3.tor01.objectstorage.softlayer.net”
10046 · Toronto Single Site Endpoint
10048 · “s3.tor01.objectstorage.service.networklayer.com”
10050 · Toronto Single Site Private Endpoint
10052 –s3-endpoint
10054 Endpoint for OSS API.
10055 · Config: endpoint
10057 · Env Var: RCLONE_S3_ENDPOINT
10059 · Type: string
10061 · Default: ""
10063 · Examples:
10065 · “oss-cn-hangzhou.aliyuncs.com”
10067 · East China 1 (Hangzhou)
10069 · “oss-cn-shanghai.aliyuncs.com”
10071 · East China 2 (Shanghai)
10073 · “oss-cn-qingdao.aliyuncs.com”
10075 · North China 1 (Qingdao)
10077 · “oss-cn-beijing.aliyuncs.com”
10079 · North China 2 (Beijing)
10081 · “oss-cn-zhangjiakou.aliyuncs.com”
10083 · North China 3 (Zhangjiakou)
10085 · “oss-cn-huhehaote.aliyuncs.com”
10087 · North China 5 (Huhehaote)
10089 · “oss-cn-shenzhen.aliyuncs.com”
10091 · South China 1 (Shenzhen)
10093 · “oss-cn-hongkong.aliyuncs.com”
10095 · Hong Kong (Hong Kong)
10097 · “oss-us-west-1.aliyuncs.com”
10099 · US West 1 (Silicon Valley)
10101 · “oss-us-east-1.aliyuncs.com”
10103 · US East 1 (Virginia)
10105 · “oss-ap-southeast-1.aliyuncs.com”
10107 · Southeast Asia Southeast 1 (Singapore)
10109 · “oss-ap-southeast-2.aliyuncs.com”
10111 · Asia Pacific Southeast 2 (Sydney)
10113 · “oss-ap-southeast-3.aliyuncs.com”
10115 · Southeast Asia Southeast 3 (Kuala Lumpur)
10117 · “oss-ap-southeast-5.aliyuncs.com”
10119 · Asia Pacific Southeast 5 (Jakarta)
10121 · “oss-ap-northeast-1.aliyuncs.com”
10123 · Asia Pacific Northeast 1 (Japan)
10125 · “oss-ap-south-1.aliyuncs.com”
10127 · Asia Pacific South 1 (Mumbai)
10129 · “oss-eu-central-1.aliyuncs.com”
10131 · Central Europe 1 (Frankfurt)
10133 · “oss-eu-west-1.aliyuncs.com”
10135 · West Europe (London)
10137 · “oss-me-east-1.aliyuncs.com”
10139 · Middle East 1 (Dubai)
10141 –s3-endpoint
10143 Endpoint for S3 API. Required when using an S3 clone.
10144 · Config: endpoint
10146 · Env Var: RCLONE_S3_ENDPOINT
10148 · Type: string
10150 · Default: ""
10152 · Examples:
10154 · “objects-us-east-1.dream.io”
10156 · Dream Objects endpoint
10158 · “nyc3.digitaloceanspaces.com”
10160 · Digital Ocean Spaces New York 3
10162 · “ams3.digitaloceanspaces.com”
10164 · Digital Ocean Spaces Amsterdam 3
10166 · “sgp1.digitaloceanspaces.com”
10168 · Digital Ocean Spaces Singapore 1
10170 · “s3.wasabisys.com”
10172 · Wasabi US East endpoint
10174 · “s3.us-west-1.wasabisys.com”
10176 · Wasabi US West endpoint
10178 · “s3.eu-central-1.wasabisys.com”
10180 · Wasabi EU Central endpoint
10182 –s3-location-constraint
10184 Location constraint - must be set to match the Region. Used when cre‐
10185 ating buckets only.
10186 · Config: location_constraint
10188 · Env Var: RCLONE_S3_LOCATION_CONSTRAINT
10190 · Type: string
10192 · Default: ""
10194 · Examples:
10196 · ""
10198 · Empty for US Region, Northern Virginia or Pacific Northwest.
10200 · “us-east-2”
10202 · US East (Ohio) Region.
10204 · “us-west-2”
10206 · US West (Oregon) Region.
10208 · “us-west-1”
10210 · US West (Northern California) Region.
10212 · “ca-central-1”
10214 · Canada (Central) Region.
10216 · “eu-west-1”
10218 · EU (Ireland) Region.
10220 · “eu-west-2”
10222 · EU (London) Region.
10224 · “eu-north-1”
10226 · EU (Stockholm) Region.
10228 · “EU”
10230 · EU Region.
10232 · “ap-southeast-1”
10234 · Asia Pacific (Singapore) Region.
10236 · “ap-southeast-2”
10238 · Asia Pacific (Sydney) Region.
10240 · “ap-northeast-1”
10242 · Asia Pacific (Tokyo) Region.
10244 · “ap-northeast-2”
10246 · Asia Pacific (Seoul)
10248 · “ap-south-1”
10250 · Asia Pacific (Mumbai)
10252 · “ap-east-1”
10254 · Asia Pacific (Hong Kong)
10256 · “sa-east-1”
10258 · South America (Sao Paulo) Region.
10260 –s3-location-constraint
10262 Location constraint - must match endpoint when using IBM Cloud Public.
10263 For on-prem COS, do not make a selection from this list, hit enter
10264 · Config: location_constraint
10266 · Env Var: RCLONE_S3_LOCATION_CONSTRAINT
10268 · Type: string
10270 · Default: ""
10272 · Examples:
10274 · “us-standard”
10276 · US Cross Region Standard
10278 · “us-vault”
10280 · US Cross Region Vault
10282 · “us-cold”
10284 · US Cross Region Cold
10286 · “us-flex”
10288 · US Cross Region Flex
10290 · “us-east-standard”
10292 · US East Region Standard
10294 · “us-east-vault”
10296 · US East Region Vault
10298 · “us-east-cold”
10300 · US East Region Cold
10302 · “us-east-flex”
10304 · US East Region Flex
10306 · “us-south-standard”
10308 · US South Region Standard
10310 · “us-south-vault”
10312 · US South Region Vault
10314 · “us-south-cold”
10316 · US South Region Cold
10318 · “us-south-flex”
10320 · US South Region Flex
10322 · “eu-standard”
10324 · EU Cross Region Standard
10326 · “eu-vault”
10328 · EU Cross Region Vault
10330 · “eu-cold”
10332 · EU Cross Region Cold
10334 · “eu-flex”
10336 · EU Cross Region Flex
10338 · “eu-gb-standard”
10340 · Great Britain Standard
10342 · “eu-gb-vault”
10344 · Great Britain Vault
10346 · “eu-gb-cold”
10348 · Great Britain Cold
10350 · “eu-gb-flex”
10352 · Great Britain Flex
10354 · “ap-standard”
10356 · APAC Standard
10358 · “ap-vault”
10360 · APAC Vault
10362 · “ap-cold”
10364 · APAC Cold
10366 · “ap-flex”
10368 · APAC Flex
10370 · “mel01-standard”
10372 · Melbourne Standard
10374 · “mel01-vault”
10376 · Melbourne Vault
10378 · “mel01-cold”
10380 · Melbourne Cold
10382 · “mel01-flex”
10384 · Melbourne Flex
10386 · “tor01-standard”
10388 · Toronto Standard
10390 · “tor01-vault”
10392 · Toronto Vault
10394 · “tor01-cold”
10396 · Toronto Cold
10398 · “tor01-flex”
10400 · Toronto Flex
10402 –s3-location-constraint
10404 Location constraint - must be set to match the Region. Leave blank if
10405 not sure. Used when creating buckets only.
10406 · Config: location_constraint
10408 · Env Var: RCLONE_S3_LOCATION_CONSTRAINT
10410 · Type: string
10412 · Default: ""
10414 –s3-acl
10416 Canned ACL used when creating buckets and storing or copying objects.
10417 This ACL is used for creating objects and if bucket_acl isn’t set, for
10419 creating buckets too.
10420 For more info visit https://docs.aws.amazon.com/AmazonS3/lat‐
10422 est/dev/acl-overview.html#canned-acl
10423 Note that this ACL is applied when server side copying objects as S3
10425 doesn’t copy the ACL from the source but rather writes a fresh one.
10426 · Config: acl
10428 · Env Var: RCLONE_S3_ACL
10430 · Type: string
10432 · Default: ""
10434 · Examples:
10436 · “private”
10438 · Owner gets FULL_CONTROL. No one else has access rights (de‐
10440 fault).
10441 · “public-read”
10443 · Owner gets FULL_CONTROL. The AllUsers group gets READ access.
10445 · “public-read-write”
10447 · Owner gets FULL_CONTROL. The AllUsers group gets READ and WRITE
10449 access.
10450 · Granting this on a bucket is generally not recommended.
10452 · “authenticated-read”
10454 · Owner gets FULL_CONTROL. The AuthenticatedUsers group gets READ
10456 access.
10457 · “bucket-owner-read”
10459 · Object owner gets FULL_CONTROL. Bucket owner gets READ access.
10461 · If you specify this canned ACL when creating a bucket, Amazon S3
10463 ignores it.
10464 · “bucket-owner-full-control”
10466 · Both the object owner and the bucket owner get FULL_CONTROL over
10468 the object.
10469 · If you specify this canned ACL when creating a bucket, Amazon S3
10471 ignores it.
10472 · “private”
10474 · Owner gets FULL_CONTROL. No one else has access rights (de‐
10476 fault). This acl is available on IBM Cloud (Infra), IBM Cloud
10477 (Storage), On-Premise COS
10478 · “public-read”
10480 · Owner gets FULL_CONTROL. The AllUsers group gets READ access.
10482 This acl is available on IBM Cloud (Infra), IBM Cloud (Storage),
10483 On-Premise IBM COS
10484 · “public-read-write”
10486 · Owner gets FULL_CONTROL. The AllUsers group gets READ and WRITE
10488 access. This acl is available on IBM Cloud (Infra), On-Premise
10489 IBM COS
10490 · “authenticated-read”
10492 · Owner gets FULL_CONTROL. The AuthenticatedUsers group gets READ
10494 access. Not supported on Buckets. This acl is available on IBM
10495 Cloud (Infra) and On-Premise IBM COS
10496 –s3-server-side-encryption
10498 The server-side encryption algorithm used when storing this object in
10499 S3.
10500 · Config: server_side_encryption
10502 · Env Var: RCLONE_S3_SERVER_SIDE_ENCRYPTION
10504 · Type: string
10506 · Default: ""
10508 · Examples:
10510 · ""
10512 · None
10514 · “AES256”
10516 · AES256
10518 · “aws:kms”
10520 · aws:kms
10522 –s3-sse-kms-key-id
10524 If using KMS ID you must provide the ARN of Key.
10525 · Config: sse_kms_key_id
10527 · Env Var: RCLONE_S3_SSE_KMS_KEY_ID
10529 · Type: string
10531 · Default: ""
10533 · Examples:
10535 · ""
10537 · None
10539 · "arn:aws:kms:us-east-1:*"
10541 · arn:aws:kms:*
10543 –s3-storage-class
10545 The storage class to use when storing new objects in S3.
10546 · Config: storage_class
10548 · Env Var: RCLONE_S3_STORAGE_CLASS
10550 · Type: string
10552 · Default: ""
10554 · Examples:
10556 · ""
10558 · Default
10560 · “STANDARD”
10562 · Standard storage class
10564 · “REDUCED_REDUNDANCY”
10566 · Reduced redundancy storage class
10568 · “STANDARD_IA”
10570 · Standard Infrequent Access storage class
10572 · “ONEZONE_IA”
10574 · One Zone Infrequent Access storage class
10576 · “GLACIER”
10578 · Glacier storage class
10580 · “DEEP_ARCHIVE”
10582 · Glacier Deep Archive storage class
10584 · “INTELLIGENT_TIERING”
10586 · Intelligent-Tiering storage class
10588 –s3-storage-class
10590 The storage class to use when storing new objects in OSS.
10591 · Config: storage_class
10593 · Env Var: RCLONE_S3_STORAGE_CLASS
10595 · Type: string
10597 · Default: ""
10599 · Examples:
10601 · ""
10603 · Default
10605 · “STANDARD”
10607 · Standard storage class
10609 · “GLACIER”
10611 · Archive storage mode.
10613 · “STANDARD_IA”
10615 · Infrequent access storage mode.
10617 Advanced Options
10619 Here are the advanced options specific to s3 (Amazon S3 Compliant Stor‐
10620 age Provider (AWS, Alibaba, Ceph, Digital Ocean, Dreamhost, IBM COS,
10621 Minio, etc)).
10622 –s3-bucket-acl
10624 Canned ACL used when creating buckets.
10625 For more info visit https://docs.aws.amazon.com/AmazonS3/lat‐
10627 est/dev/acl-overview.html#canned-acl
10628 Note that this ACL is applied when only when creating buckets. If it
10630 isn’t set then “acl” is used instead.
10631 · Config: bucket_acl
10633 · Env Var: RCLONE_S3_BUCKET_ACL
10635 · Type: string
10637 · Default: ""
10639 · Examples:
10641 · “private”
10643 · Owner gets FULL_CONTROL. No one else has access rights (de‐
10645 fault).
10646 · “public-read”
10648 · Owner gets FULL_CONTROL. The AllUsers group gets READ access.
10650 · “public-read-write”
10652 · Owner gets FULL_CONTROL. The AllUsers group gets READ and WRITE
10654 access.
10655 · Granting this on a bucket is generally not recommended.
10657 · “authenticated-read”
10659 · Owner gets FULL_CONTROL. The AuthenticatedUsers group gets READ
10661 access.
10662 –s3-upload-cutoff
10664 Cutoff for switching to chunked upload
10665 Any files larger than this will be uploaded in chunks of chunk_size.
10667 The minimum is 0 and the maximum is 5GB.
10668 · Config: upload_cutoff
10670 · Env Var: RCLONE_S3_UPLOAD_CUTOFF
10672 · Type: SizeSuffix
10674 · Default: 200M
10676 –s3-chunk-size
10678 Chunk size to use for uploading.
10679 When uploading files larger than upload_cutoff or files with unknown
10681 size (eg from “rclone rcat” or uploaded with “rclone mount” or google
10682 photos or google docs) they will be uploaded as multipart uploads using
10683 this chunk size.
10684 Note that “–s3-upload-concurrency” chunks of this size are buffered in
10686 memory per transfer.
10687 If you are transferring large files over high speed links and you have
10689 enough memory, then increasing this will speed up the transfers.
10690 Rclone will automatically increase the chunk size when uploading a
10692 large file of known size to stay below the 10,000 chunks limit.
10693 Files of unknown size are uploaded with the configured chunk_size.
10695 Since the default chunk size is 5MB and there can be at most 10,000
10696 chunks, this means that by default the maximum size of file you can
10697 stream upload is 48GB. If you wish to stream upload larger files then
10698 you will need to increase chunk_size.
10699 · Config: chunk_size
10701 · Env Var: RCLONE_S3_CHUNK_SIZE
10703 · Type: SizeSuffix
10705 · Default: 5M
10707 –s3-copy-cutoff
10709 Cutoff for switching to multipart copy
10710 Any files larger than this that need to be server side copied will be
10712 copied in chunks of this size.
10713 The minimum is 0 and the maximum is 5GB.
10715 · Config: copy_cutoff
10717 · Env Var: RCLONE_S3_COPY_CUTOFF
10719 · Type: SizeSuffix
10721 · Default: 5G
10723 –s3-disable-checksum
10725 Don’t store MD5 checksum with object metadata
10726 · Config: disable_checksum
10728 · Env Var: RCLONE_S3_DISABLE_CHECKSUM
10730 · Type: bool
10732 · Default: false
10734 –s3-session-token
10736 An AWS session token
10737 · Config: session_token
10739 · Env Var: RCLONE_S3_SESSION_TOKEN
10741 · Type: string
10743 · Default: ""
10745 –s3-upload-concurrency
10747 Concurrency for multipart uploads.
10748 This is the number of chunks of the same file that are uploaded concur‐
10750 rently.
10751 If you are uploading small numbers of large file over high speed link
10753 and these uploads do not fully utilize your bandwidth, then increasing
10754 this may help to speed up the transfers.
10755 · Config: upload_concurrency
10757 · Env Var: RCLONE_S3_UPLOAD_CONCURRENCY
10759 · Type: int
10761 · Default: 4
10763 –s3-force-path-style
10765 If true use path style access if false use virtual hosted style.
10766 If this is true (the default) then rclone will use path style access,
10768 if false then rclone will use virtual path style. See the AWS S3 docs
10769 (https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingBucket.html#ac‐
10770 cess-bucket-intro) for more info.
10771 Some providers (eg AWS, Aliyun OSS or Netease COS) require this set to
10773 false - rclone will do this automatically based on the provider set‐
10774 ting.
10775 · Config: force_path_style
10777 · Env Var: RCLONE_S3_FORCE_PATH_STYLE
10779 · Type: bool
10781 · Default: true
10783 –s3-v2-auth
10785 If true use v2 authentication.
10786 If this is false (the default) then rclone will use v4 authentication.
10788 If it is set then rclone will use v2 authentication.
10789 Use this only if v4 signatures don’t work, eg pre Jewel/v10 CEPH.
10791 · Config: v2_auth
10793 · Env Var: RCLONE_S3_V2_AUTH
10795 · Type: bool
10797 · Default: false
10799 –s3-use-accelerate-endpoint
10801 If true use the AWS S3 accelerated endpoint.
10802 See: AWS S3 Transfer acceleration (https://docs.aws.amazon.com/Ama‐
10804 zonS3/latest/dev/transfer-acceleration-examples.html)
10805 · Config: use_accelerate_endpoint
10807 · Env Var: RCLONE_S3_USE_ACCELERATE_ENDPOINT
10809 · Type: bool
10811 · Default: false
10813 –s3-leave-parts-on-error
10815 If true avoid calling abort upload on a failure, leaving all success‐
10816 fully uploaded parts on S3 for manual recovery.
10817 It should be set to true for resuming uploads across different ses‐
10819 sions.
10820 WARNING: Storing parts of an incomplete multipart upload counts towards
10822 space usage on S3 and will add additional costs if not cleaned up.
10823 · Config: leave_parts_on_error
10825 · Env Var: RCLONE_S3_LEAVE_PARTS_ON_ERROR
10827 · Type: bool
10829 · Default: false
10831 –s3-list-chunk
10833 Size of listing chunk (response list for each ListObject S3 request).
10834 This option is also known as “MaxKeys”, “max-items”, or “page-size”
10836 from the AWS S3 specification. Most services truncate the response
10837 list to 1000 objects even if requested more than that. In AWS S3 this
10838 is a global maximum and cannot be changed, see AWS S3
10839 (https://docs.aws.amazon.com/cli/latest/reference/s3/ls.html). In
10840 Ceph, this can be increased with the “rgw list buckets max chunk” op‐
10841 tion.
10842 · Config: list_chunk
10844 · Env Var: RCLONE_S3_LIST_CHUNK
10846 · Type: int
10848 · Default: 1000
10850 –s3-encoding
10852 This sets the encoding for the backend.
10853 See: the encoding section in the overview (/overview/#encoding) for
10855 more info.
10856 · Config: encoding
10858 · Env Var: RCLONE_S3_ENCODING
10860 · Type: MultiEncoder
10862 · Default: Slash,InvalidUtf8,Dot
10864 Anonymous access to public buckets
10866 If you want to use rclone to access a public bucket, configure with a
10867 blank access_key_id and secret_access_key. Your config should end up
10868 looking like this:
10869 [anons3]
10871 type = s3
10872 provider = AWS
10873 env_auth = false
10874 access_key_id =
10875 secret_access_key =
10876 region = us-east-1
10877 endpoint =
10878 location_constraint =
10879 acl = private
10880 server_side_encryption =
10881 storage_class =
10882 Then use it as normal with the name of the public bucket, eg
10884 rclone lsd anons3:1000genomes
10886 You will be able to list and copy data but not upload it.
10888 Ceph
10890 Ceph (https://ceph.com/) is an open source unified, distributed storage
10891 system designed for excellent performance, reliability and scalability.
10892 It has an S3 compatible object storage interface.
10893 To use rclone with Ceph, configure as above but leave the region blank
10895 and set the endpoint. You should end up with something like this in
10896 your config:
10897 [ceph]
10899 type = s3
10900 provider = Ceph
10901 env_auth = false
10902 access_key_id = XXX
10903 secret_access_key = YYY
10904 region =
10905 endpoint = https://ceph.endpoint.example.com
10906 location_constraint =
10907 acl =
10908 server_side_encryption =
10909 storage_class =
10910 If you are using an older version of CEPH, eg 10.2.x Jewel, then you
10912 may need to supply the parameter --s3-upload-cutoff 0 or put this in
10913 the config file as upload_cutoff 0 to work around a bug which causes
10914 uploading of small files to fail.
10915 Note also that Ceph sometimes puts / in the passwords it gives users.
10917 If you read the secret access key using the command line tools you will
10918 get a JSON blob with the / escaped as \/. Make sure you only write /
10919 in the secret access key.
10920 Eg the dump from Ceph looks something like this (irrelevant keys re‐
10922 moved).
10923 {
10925 "user_id": "xxx",
10926 "display_name": "xxxx",
10927 "keys": [
10928 {
10929 "user": "xxx",
10930 "access_key": "xxxxxx",
10931 "secret_key": "xxxxxx\/xxxx"
10932 }
10933 ],
10934 }
10935 Because this is a json dump, it is encoding the / as \/, so if you use
10937 the secret key as xxxxxx/xxxx it will work fine.
10938 Dreamhost
10940 Dreamhost DreamObjects (https://www.dreamhost.com/cloud/storage/) is an
10941 object storage system based on CEPH.
10942 To use rclone with Dreamhost, configure as above but leave the region
10944 blank and set the endpoint. You should end up with something like this
10945 in your config:
10946 [dreamobjects]
10948 type = s3
10949 provider = DreamHost
10950 env_auth = false
10951 access_key_id = your_access_key
10952 secret_access_key = your_secret_key
10953 region =
10954 endpoint = objects-us-west-1.dream.io
10955 location_constraint =
10956 acl = private
10957 server_side_encryption =
10958 storage_class =
10959 DigitalOcean Spaces
10961 Spaces (https://www.digitalocean.com/products/object-storage/) is an
10962 S3-interoperable (https://developers.digitalocean.com/documenta‐
10963 tion/spaces/) object storage service from cloud provider DigitalOcean.
10964 To connect to DigitalOcean Spaces you will need an access key and se‐
10966 cret key. These can be retrieved on the “Applications & API
10967 (https://cloud.digitalocean.com/settings/api/tokens)” page of the Digi‐
10968 talOcean control panel. They will be needed when promted by rclone
10969 config for your access_key_id and secret_access_key.
10970 When prompted for a region or location_constraint, press enter to use
10972 the default value. The region must be included in the endpoint setting
10973 (e.g. nyc3.digitaloceanspaces.com). The default values can be used for
10974 other settings.
10975 Going through the whole process of creating a new remote by running
10977 rclone config, each prompt should be answered as shown below:
10978 Storage> s3
10980 env_auth> 1
10981 access_key_id> YOUR_ACCESS_KEY
10982 secret_access_key> YOUR_SECRET_KEY
10983 region>
10984 endpoint> nyc3.digitaloceanspaces.com
10985 location_constraint>
10986 acl>
10987 storage_class>
10988 The resulting configuration file should look like:
10990 [spaces]
10992 type = s3
10993 provider = DigitalOcean
10994 env_auth = false
10995 access_key_id = YOUR_ACCESS_KEY
10996 secret_access_key = YOUR_SECRET_KEY
10997 region =
10998 endpoint = nyc3.digitaloceanspaces.com
10999 location_constraint =
11000 acl =
11001 server_side_encryption =
11002 storage_class =
11003 Once configured, you can create a new Space and begin copying files.
11005 For example:
11006 rclone mkdir spaces:my-new-space
11008 rclone copy /path/to/files spaces:my-new-space
11009 IBM COS (S3)
11011 Information stored with IBM Cloud Object Storage is encrypted and dis‐
11012 persed across multiple geographic locations, and accessed through an
11013 implementation of the S3 API. This service makes use of the distrib‐
11014 uted storage technologies provided by IBM’s Cloud Object Storage System
11015 (formerly Cleversafe). For more information visit:
11016 (http://www.ibm.com/cloud/object-storage)
11017 To configure access to IBM COS S3, follow the steps below:
11019 1. Run rclone config and select n for a new remote.
11021 2018/02/14 14:13:11 NOTICE: Config file "C:\\Users\\a\\.config\\rclone\\rclone.conf" not found - using defaults
11023 No remotes found - make a new one
11024 n) New remote
11025 s) Set configuration password
11026 q) Quit config
11027 n/s/q> n
11028 2. Enter the name for the configuration
11030 name> <YOUR NAME>
11032 3. Select “s3” storage.
11034 Choose a number from below, or type in your own value
11036 1 / Alias for an existing remote
11037 \ "alias"
11038 2 / Amazon Drive
11039 \ "amazon cloud drive"
11040 3 / Amazon S3 Complaint Storage Providers (Dreamhost, Ceph, Minio, IBM COS)
11041 \ "s3"
11042 4 / Backblaze B2
11043 \ "b2"
11044 [snip]
11045 23 / http Connection
11046 \ "http"
11047 Storage> 3
11048 4. Select IBM COS as the S3 Storage Provider.
11050 Choose the S3 provider.
11052 Choose a number from below, or type in your own value
11053 1 / Choose this option to configure Storage to AWS S3
11054 \ "AWS"
11055 2 / Choose this option to configure Storage to Ceph Systems
11056 \ "Ceph"
11057 3 / Choose this option to configure Storage to Dreamhost
11058 \ "Dreamhost"
11059 4 / Choose this option to the configure Storage to IBM COS S3
11060 \ "IBMCOS"
11061 5 / Choose this option to the configure Storage to Minio
11062 \ "Minio"
11063 Provider>4
11064 5. Enter the Access Key and Secret.
11066 AWS Access Key ID - leave blank for anonymous access or runtime credentials.
11068 access_key_id> <>
11069 AWS Secret Access Key (password) - leave blank for anonymous access or runtime credentials.
11070 secret_access_key> <>
11071 6. Specify the endpoint for IBM COS. For Public IBM COS, choose from
11073 the option below. For On Premise IBM COS, enter an enpoint address.
11074 Endpoint for IBM COS S3 API.
11076 Specify if using an IBM COS On Premise.
11077 Choose a number from below, or type in your own value
11078 1 / US Cross Region Endpoint
11079 \ "s3-api.us-geo.objectstorage.softlayer.net"
11080 2 / US Cross Region Dallas Endpoint
11081 \ "s3-api.dal.us-geo.objectstorage.softlayer.net"
11082 3 / US Cross Region Washington DC Endpoint
11083 \ "s3-api.wdc-us-geo.objectstorage.softlayer.net"
11084 4 / US Cross Region San Jose Endpoint
11085 \ "s3-api.sjc-us-geo.objectstorage.softlayer.net"
11086 5 / US Cross Region Private Endpoint
11087 \ "s3-api.us-geo.objectstorage.service.networklayer.com"
11088 6 / US Cross Region Dallas Private Endpoint
11089 \ "s3-api.dal-us-geo.objectstorage.service.networklayer.com"
11090 7 / US Cross Region Washington DC Private Endpoint
11091 \ "s3-api.wdc-us-geo.objectstorage.service.networklayer.com"
11092 8 / US Cross Region San Jose Private Endpoint
11093 \ "s3-api.sjc-us-geo.objectstorage.service.networklayer.com"
11094 9 / US Region East Endpoint
11095 \ "s3.us-east.objectstorage.softlayer.net"
11096 10 / US Region East Private Endpoint
11097 \ "s3.us-east.objectstorage.service.networklayer.com"
11098 11 / US Region South Endpoint
11099 [snip]
11100 34 / Toronto Single Site Private Endpoint
11101 \ "s3.tor01.objectstorage.service.networklayer.com"
11102 endpoint>1
11103 7. Specify a IBM COS Location Constraint. The location constraint must
11105 match endpoint when using IBM Cloud Public. For on-prem COS, do not
11106 make a selection from this list, hit enter
11107 1 / US Cross Region Standard
11109 \ "us-standard"
11110 2 / US Cross Region Vault
11111 \ "us-vault"
11112 3 / US Cross Region Cold
11113 \ "us-cold"
11114 4 / US Cross Region Flex
11115 \ "us-flex"
11116 5 / US East Region Standard
11117 \ "us-east-standard"
11118 6 / US East Region Vault
11119 \ "us-east-vault"
11120 7 / US East Region Cold
11121 \ "us-east-cold"
11122 8 / US East Region Flex
11123 \ "us-east-flex"
11124 9 / US South Region Standard
11125 \ "us-south-standard"
11126 10 / US South Region Vault
11127 \ "us-south-vault"
11128 [snip]
11129 32 / Toronto Flex
11130 \ "tor01-flex"
11131 location_constraint>1
11132 9. Specify a canned ACL. IBM Cloud (Strorage) supports “public-read”
11134 and “private”. IBM Cloud(Infra) supports all the canned ACLs.
11135 On-Premise COS supports all the canned ACLs.
11136 Canned ACL used when creating buckets and/or storing objects in S3.
11138 For more info visit https://docs.aws.amazon.com/AmazonS3/latest/dev/acl-overview.html#canned-acl
11139 Choose a number from below, or type in your own value
11140 1 / Owner gets FULL_CONTROL. No one else has access rights (default). This acl is available on IBM Cloud (Infra), IBM Cloud (Storage), On-Premise COS
11141 \ "private"
11142 2 / Owner gets FULL_CONTROL. The AllUsers group gets READ access. This acl is available on IBM Cloud (Infra), IBM Cloud (Storage), On-Premise IBM COS
11143 \ "public-read"
11144 3 / Owner gets FULL_CONTROL. The AllUsers group gets READ and WRITE access. This acl is available on IBM Cloud (Infra), On-Premise IBM COS
11145 \ "public-read-write"
11146 4 / Owner gets FULL_CONTROL. The AuthenticatedUsers group gets READ access. Not supported on Buckets. This acl is available on IBM Cloud (Infra) and On-Premise IBM COS
11147 \ "authenticated-read"
11148 acl> 1
11149 12. Review the displayed configuration and accept to save the “remote”
11151 then quit. The config file should look like this
11152 [xxx]
11154 type = s3
11155 Provider = IBMCOS
11156 access_key_id = xxx
11157 secret_access_key = yyy
11158 endpoint = s3-api.us-geo.objectstorage.softlayer.net
11159 location_constraint = us-standard
11160 acl = private
11161 13. Execute rclone commands
11163 1) Create a bucket.
11165 rclone mkdir IBM-COS-XREGION:newbucket
11166 2) List available buckets.
11167 rclone lsd IBM-COS-XREGION:
11168 -1 2017-11-08 21:16:22 -1 test
11169 -1 2018-02-14 20:16:39 -1 newbucket
11170 3) List contents of a bucket.
11171 rclone ls IBM-COS-XREGION:newbucket
11172 18685952 test.exe
11173 4) Copy a file from local to remote.
11174 rclone copy /Users/file.txt IBM-COS-XREGION:newbucket
11175 5) Copy a file from remote to local.
11176 rclone copy IBM-COS-XREGION:newbucket/file.txt .
11177 6) Delete a file on remote.
11178 rclone delete IBM-COS-XREGION:newbucket/file.txt
11179 Minio
11181 Minio (https://minio.io/) is an object storage server built for cloud
11182 application developers and devops.
11183 It is very easy to install and provides an S3 compatible server which
11185 can be used by rclone.
11186 To use it, install Minio following the instructions here
11188 (https://docs.minio.io/docs/minio-quickstart-guide).
11189 When it configures itself Minio will print something like this
11191 Endpoint: http://192.168.1.106:9000 http://172.23.0.1:9000
11193 AccessKey: USWUXHGYZQYFYFFIT3RE
11194 SecretKey: MOJRH0mkL1IPauahWITSVvyDrQbEEIwljvmxdq03
11195 Region: us-east-1
11196 SQS ARNs: arn:minio:sqs:us-east-1:1:redis arn:minio:sqs:us-east-1:2:redis
11197 Browser Access:
11199 http://192.168.1.106:9000 http://172.23.0.1:9000
11200 Command-line Access: https://docs.minio.io/docs/minio-client-quickstart-guide
11202 $ mc config host add myminio http://192.168.1.106:9000 USWUXHGYZQYFYFFIT3RE MOJRH0mkL1IPauahWITSVvyDrQbEEIwljvmxdq03
11203 Object API (Amazon S3 compatible):
11205 Go: https://docs.minio.io/docs/golang-client-quickstart-guide
11206 Java: https://docs.minio.io/docs/java-client-quickstart-guide
11207 Python: https://docs.minio.io/docs/python-client-quickstart-guide
11208 JavaScript: https://docs.minio.io/docs/javascript-client-quickstart-guide
11209 .NET: https://docs.minio.io/docs/dotnet-client-quickstart-guide
11210 Drive Capacity: 26 GiB Free, 165 GiB Total
11212 These details need to go into rclone config like this. Note that it is
11214 important to put the region in as stated above.
11215 env_auth> 1
11217 access_key_id> USWUXHGYZQYFYFFIT3RE
11218 secret_access_key> MOJRH0mkL1IPauahWITSVvyDrQbEEIwljvmxdq03
11219 region> us-east-1
11220 endpoint> http://192.168.1.106:9000
11221 location_constraint>
11222 server_side_encryption>
11223 Which makes the config file look like this
11225 [minio]
11227 type = s3
11228 provider = Minio
11229 env_auth = false
11230 access_key_id = USWUXHGYZQYFYFFIT3RE
11231 secret_access_key = MOJRH0mkL1IPauahWITSVvyDrQbEEIwljvmxdq03
11232 region = us-east-1
11233 endpoint = http://192.168.1.106:9000
11234 location_constraint =
11235 server_side_encryption =
11236 So once set up, for example to copy files into a bucket
11238 rclone copy /path/to/files minio:bucket
11240 Scaleway
11242 Scaleway (https://www.scaleway.com/object-storage/) The Object Storage
11243 platform allows you to store anything from backups, logs and web assets
11244 to documents and photos. Files can be dropped from the Scaleway con‐
11245 sole or transferred through our API and CLI or using any S3-compatible
11246 tool.
11247 Scaleway provides an S3 interface which can be configured for use with
11249 rclone like this:
11250 [scaleway]
11252 type = s3
11253 env_auth = false
11254 endpoint = s3.nl-ams.scw.cloud
11255 access_key_id = SCWXXXXXXXXXXXXXX
11256 secret_access_key = 1111111-2222-3333-44444-55555555555555
11257 region = nl-ams
11258 location_constraint =
11259 acl = private
11260 force_path_style = false
11261 server_side_encryption =
11262 storage_class =
11263 Wasabi
11265 Wasabi (https://wasabi.com) is a cloud-based object storage service for
11266 a broad range of applications and use cases. Wasabi is designed for
11267 individuals and organizations that require a high-performance, reli‐
11268 able, and secure data storage infrastructure at minimal cost.
11269 Wasabi provides an S3 interface which can be configured for use with
11271 rclone like this.
11272 No remotes found - make a new one
11274 n) New remote
11275 s) Set configuration password
11276 n/s> n
11277 name> wasabi
11278 Type of storage to configure.
11279 Choose a number from below, or type in your own value
11280 [snip]
11281 XX / Amazon S3 (also Dreamhost, Ceph, Minio)
11282 \ "s3"
11283 [snip]
11284 Storage> s3
11285 Get AWS credentials from runtime (environment variables or EC2/ECS meta data if no env vars). Only applies if access_key_id and secret_access_key is blank.
11286 Choose a number from below, or type in your own value
11287 1 / Enter AWS credentials in the next step
11288 \ "false"
11289 2 / Get AWS credentials from the environment (env vars or IAM)
11290 \ "true"
11291 env_auth> 1
11292 AWS Access Key ID - leave blank for anonymous access or runtime credentials.
11293 access_key_id> YOURACCESSKEY
11294 AWS Secret Access Key (password) - leave blank for anonymous access or runtime credentials.
11295 secret_access_key> YOURSECRETACCESSKEY
11296 Region to connect to.
11297 Choose a number from below, or type in your own value
11298 / The default endpoint - a good choice if you are unsure.
11299 1 | US Region, Northern Virginia or Pacific Northwest.
11300 | Leave location constraint empty.
11301 \ "us-east-1"
11302 [snip]
11303 region> us-east-1
11304 Endpoint for S3 API.
11305 Leave blank if using AWS to use the default endpoint for the region.
11306 Specify if using an S3 clone such as Ceph.
11307 endpoint> s3.wasabisys.com
11308 Location constraint - must be set to match the Region. Used when creating buckets only.
11309 Choose a number from below, or type in your own value
11310 1 / Empty for US Region, Northern Virginia or Pacific Northwest.
11311 \ ""
11312 [snip]
11313 location_constraint>
11314 Canned ACL used when creating buckets and/or storing objects in S3.
11315 For more info visit https://docs.aws.amazon.com/AmazonS3/latest/dev/acl-overview.html#canned-acl
11316 Choose a number from below, or type in your own value
11317 1 / Owner gets FULL_CONTROL. No one else has access rights (default).
11318 \ "private"
11319 [snip]
11320 acl>
11321 The server-side encryption algorithm used when storing this object in S3.
11322 Choose a number from below, or type in your own value
11323 1 / None
11324 \ ""
11325 2 / AES256
11326 \ "AES256"
11327 server_side_encryption>
11328 The storage class to use when storing objects in S3.
11329 Choose a number from below, or type in your own value
11330 1 / Default
11331 \ ""
11332 2 / Standard storage class
11333 \ "STANDARD"
11334 3 / Reduced redundancy storage class
11335 \ "REDUCED_REDUNDANCY"
11336 4 / Standard Infrequent Access storage class
11337 \ "STANDARD_IA"
11338 storage_class>
11339 Remote config
11340 --------------------
11341 [wasabi]
11342 env_auth = false
11343 access_key_id = YOURACCESSKEY
11344 secret_access_key = YOURSECRETACCESSKEY
11345 region = us-east-1
11346 endpoint = s3.wasabisys.com
11347 location_constraint =
11348 acl =
11349 server_side_encryption =
11350 storage_class =
11351 --------------------
11352 y) Yes this is OK
11353 e) Edit this remote
11354 d) Delete this remote
11355 y/e/d> y
11356 This will leave the config file looking like this.
11358 [wasabi]
11360 type = s3
11361 provider = Wasabi
11362 env_auth = false
11363 access_key_id = YOURACCESSKEY
11364 secret_access_key = YOURSECRETACCESSKEY
11365 region =
11366 endpoint = s3.wasabisys.com
11367 location_constraint =
11368 acl =
11369 server_side_encryption =
11370 storage_class =
11371 Alibaba OSS
11373 Here is an example of making an Alibaba Cloud (Aliyun) OSS
11374 (https://www.alibabacloud.com/product/oss/) configuration. First run:
11375 rclone config
11377 This will guide you through an interactive setup process.
11379 No remotes found - make a new one
11381 n) New remote
11382 s) Set configuration password
11383 q) Quit config
11384 n/s/q> n
11385 name> oss
11386 Type of storage to configure.
11387 Enter a string value. Press Enter for the default ("").
11388 Choose a number from below, or type in your own value
11389 [snip]
11390 4 / Amazon S3 Compliant Storage Provider (AWS, Alibaba, Ceph, Digital Ocean, Dreamhost, IBM COS, Minio, etc)
11391 \ "s3"
11392 [snip]
11393 Storage> s3
11394 Choose your S3 provider.
11395 Enter a string value. Press Enter for the default ("").
11396 Choose a number from below, or type in your own value
11397 1 / Amazon Web Services (AWS) S3
11398 \ "AWS"
11399 2 / Alibaba Cloud Object Storage System (OSS) formerly Aliyun
11400 \ "Alibaba"
11401 3 / Ceph Object Storage
11402 \ "Ceph"
11403 [snip]
11404 provider> Alibaba
11405 Get AWS credentials from runtime (environment variables or EC2/ECS meta data if no env vars).
11406 Only applies if access_key_id and secret_access_key is blank.
11407 Enter a boolean value (true or false). Press Enter for the default ("false").
11408 Choose a number from below, or type in your own value
11409 1 / Enter AWS credentials in the next step
11410 \ "false"
11411 2 / Get AWS credentials from the environment (env vars or IAM)
11412 \ "true"
11413 env_auth> 1
11414 AWS Access Key ID.
11415 Leave blank for anonymous access or runtime credentials.
11416 Enter a string value. Press Enter for the default ("").
11417 access_key_id> accesskeyid
11418 AWS Secret Access Key (password)
11419 Leave blank for anonymous access or runtime credentials.
11420 Enter a string value. Press Enter for the default ("").
11421 secret_access_key> secretaccesskey
11422 Endpoint for OSS API.
11423 Enter a string value. Press Enter for the default ("").
11424 Choose a number from below, or type in your own value
11425 1 / East China 1 (Hangzhou)
11426 \ "oss-cn-hangzhou.aliyuncs.com"
11427 2 / East China 2 (Shanghai)
11428 \ "oss-cn-shanghai.aliyuncs.com"
11429 3 / North China 1 (Qingdao)
11430 \ "oss-cn-qingdao.aliyuncs.com"
11431 [snip]
11432 endpoint> 1
11433 Canned ACL used when creating buckets and storing or copying objects.
11434 Note that this ACL is applied when server side copying objects as S3
11436 doesn't copy the ACL from the source but rather writes a fresh one.
11437 Enter a string value. Press Enter for the default ("").
11438 Choose a number from below, or type in your own value
11439 1 / Owner gets FULL_CONTROL. No one else has access rights (default).
11440 \ "private"
11441 2 / Owner gets FULL_CONTROL. The AllUsers group gets READ access.
11442 \ "public-read"
11443 / Owner gets FULL_CONTROL. The AllUsers group gets READ and WRITE access.
11444 [snip]
11445 acl> 1
11446 The storage class to use when storing new objects in OSS.
11447 Enter a string value. Press Enter for the default ("").
11448 Choose a number from below, or type in your own value
11449 1 / Default
11450 \ ""
11451 2 / Standard storage class
11452 \ "STANDARD"
11453 3 / Archive storage mode.
11454 \ "GLACIER"
11455 4 / Infrequent access storage mode.
11456 \ "STANDARD_IA"
11457 storage_class> 1
11458 Edit advanced config? (y/n)
11459 y) Yes
11460 n) No
11461 y/n> n
11462 Remote config
11463 --------------------
11464 [oss]
11465 type = s3
11466 provider = Alibaba
11467 env_auth = false
11468 access_key_id = accesskeyid
11469 secret_access_key = secretaccesskey
11470 endpoint = oss-cn-hangzhou.aliyuncs.com
11471 acl = private
11472 storage_class = Standard
11473 --------------------
11474 y) Yes this is OK
11475 e) Edit this remote
11476 d) Delete this remote
11477 y/e/d> y
11478 Netease NOS
11480 For Netease NOS configure as per the configurator rclone config setting
11481 the provider Netease. This will automatically set force_path_style =
11482 false which is necessary for it to run properly.
11483 Backblaze B2
11485 B2 is Backblaze’s cloud storage system (https://www.backblaze.com/b2/).
11486 Paths are specified as remote:bucket (or remote: for the lsd command.)
11488 You may put subdirectories in too, eg remote:bucket/path/to/dir.
11489 Here is an example of making a b2 configuration. First run
11491 rclone config
11493 This will guide you through an interactive setup process. To authenti‐
11495 cate you will either need your Account ID (a short hex number) and Mas‐
11496 ter Application Key (a long hex number) OR an Application Key, which is
11497 the recommended method. See below for further details on generating
11498 and using an Application Key.
11499 No remotes found - make a new one
11501 n) New remote
11502 q) Quit config
11503 n/q> n
11504 name> remote
11505 Type of storage to configure.
11506 Choose a number from below, or type in your own value
11507 [snip]
11508 XX / Backblaze B2
11509 \ "b2"
11510 [snip]
11511 Storage> b2
11512 Account ID or Application Key ID
11513 account> 123456789abc
11514 Application Key
11515 key> 0123456789abcdef0123456789abcdef0123456789
11516 Endpoint for the service - leave blank normally.
11517 endpoint>
11518 Remote config
11519 --------------------
11520 [remote]
11521 account = 123456789abc
11522 key = 0123456789abcdef0123456789abcdef0123456789
11523 endpoint =
11524 --------------------
11525 y) Yes this is OK
11526 e) Edit this remote
11527 d) Delete this remote
11528 y/e/d> y
11529 This remote is called remote and can now be used like this
11531 See all buckets
11533 rclone lsd remote:
11535 Create a new bucket
11537 rclone mkdir remote:bucket
11539 List the contents of a bucket
11541 rclone ls remote:bucket
11543 Sync /home/local/directory to the remote bucket, deleting any excess
11545 files in the bucket.
11546 rclone sync /home/local/directory remote:bucket
11548 Application Keys
11550 B2 supports multiple Application Keys for different access permission
11551 to B2 Buckets (https://www.backblaze.com/b2/docs/applica‐
11552 tion_keys.html).
11553 You can use these with rclone too; you will need to use rclone version
11555 1.43 or later.
11556 Follow Backblaze’s docs to create an Application Key with the required
11558 permission and add the applicationKeyId as the account and the Applica‐
11559 tion Key itself as the key.
11560 Note that you must put the applicationKeyId as the account – you can’t
11562 use the master Account ID. If you try then B2 will return 401 errors.
11563 –fast-list
11565 This remote supports --fast-list which allows you to use fewer transac‐
11566 tions in exchange for more memory. See the rclone docs (/docs/#fast-
11567 list) for more details.
11568 Modified time
11570 The modified time is stored as metadata on the object as X-Bz-In‐
11571 fo-src_last_modified_millis as milliseconds since 1970-01-01 in the
11572 Backblaze standard. Other tools should be able to use this as a modi‐
11573 fied time.
11574 Modified times are used in syncing and are fully supported. Note that
11576 if a modification time needs to be updated on an object then it will
11577 create a new version of the object.
11578 Restricted filename characters
11580 In addition to the default restricted characters set (/overview/#re‐
11581 stricted-characters) the following characters are also replaced:
11582 Character Value Replacement
11584 ────────────────────────────────
11585 \ 0x5C ＼
11586 Invalid UTF-8 bytes will also be replaced (/overview/#invalid-utf8), as
11588 they can’t be used in JSON strings.
11589 SHA1 checksums
11591 The SHA1 checksums of the files are checked on upload and download and
11592 will be used in the syncing process.
11593 Large files (bigger than the limit in --b2-upload-cutoff) which are up‐
11595 loaded in chunks will store their SHA1 on the object as X-Bz-In‐
11596 fo-large_file_sha1 as recommended by Backblaze.
11597 For a large file to be uploaded with an SHA1 checksum, the source needs
11599 to support SHA1 checksums. The local disk supports SHA1 checksums so
11600 large file transfers from local disk will have an SHA1. See the over‐
11601 view (/overview/#features) for exactly which remotes support SHA1.
11602 Sources which don’t support SHA1, in particular crypt will upload large
11604 files without SHA1 checksums. This may be fixed in the future (see
11605 #1767 (https://github.com/rclone/rclone/issues/1767)).
11606 Files sizes below --b2-upload-cutoff will always have an SHA1 regard‐
11608 less of the source.
11609 Transfers
11611 Backblaze recommends that you do lots of transfers simultaneously for
11612 maximum speed. In tests from my SSD equipped laptop the optimum set‐
11613 ting is about --transfers 32 though higher numbers may be used for a
11614 slight speed improvement. The optimum number for you may vary depend‐
11615 ing on your hardware, how big the files are, how much you want to load
11616 your computer, etc. The default of --transfers 4 is definitely too low
11617 for Backblaze B2 though.
11618 Note that uploading big files (bigger than 200 MB by default) will use
11620 a 96 MB RAM buffer by default. There can be at most --transfers of
11621 these in use at any moment, so this sets the upper limit on the memory
11622 used.
11623 Versions
11625 When rclone uploads a new version of a file it creates a new version of
11626 it (https://www.backblaze.com/b2/docs/file_versions.html). Likewise
11627 when you delete a file, the old version will be marked hidden and still
11628 be available. Conversely, you may opt in to a “hard delete” of files
11629 with the --b2-hard-delete flag which would permanently remove the file
11630 instead of hiding it.
11631 Old versions of files, where available, are visible using the --b2-ver‐
11633 sions flag.
11634 NB Note that --b2-versions does not work with crypt at the moment #1627
11636 (https://github.com/rclone/rclone/issues/1627). Using –backup-dir
11637 (/docs/#backup-dir-dir) with rclone is the recommended way of working
11638 around this.
11639 If you wish to remove all the old versions then you can use the rclone
11641 cleanup remote:bucket command which will delete all the old versions of
11642 files, leaving the current ones intact. You can also supply a path and
11643 only old versions under that path will be deleted, eg rclone cleanup
11644 remote:bucket/path/to/stuff.
11645 Note that cleanup will remove partially uploaded files from the bucket
11647 if they are more than a day old.
11648 When you purge a bucket, the current and the old versions will be
11650 deleted then the bucket will be deleted.
11651 However delete will cause the current versions of the files to become
11653 hidden old versions.
11654 Here is a session showing the listing and retrieval of an old version
11656 followed by a cleanup of the old versions.
11657 Show current version and all the versions with --b2-versions flag.
11659 $ rclone -q ls b2:cleanup-test
11661 9 one.txt
11662 $ rclone -q --b2-versions ls b2:cleanup-test
11664 9 one.txt
11665 8 one-v2016-07-04-141032-000.txt
11666 16 one-v2016-07-04-141003-000.txt
11667 15 one-v2016-07-02-155621-000.txt
11668 Retrieve an old version
11670 $ rclone -q --b2-versions copy b2:cleanup-test/one-v2016-07-04-141003-000.txt /tmp
11672 $ ls -l /tmp/one-v2016-07-04-141003-000.txt
11674 -rw-rw-r-- 1 ncw ncw 16 Jul 2 17:46 /tmp/one-v2016-07-04-141003-000.txt
11675 Clean up all the old versions and show that they’ve gone.
11677 $ rclone -q cleanup b2:cleanup-test
11679 $ rclone -q ls b2:cleanup-test
11681 9 one.txt
11682 $ rclone -q --b2-versions ls b2:cleanup-test
11684 9 one.txt
11685 Data usage
11687 It is useful to know how many requests are sent to the server in dif‐
11688 ferent scenarios.
11689 All copy commands send the following 4 requests:
11691 /b2api/v1/b2_authorize_account
11693 /b2api/v1/b2_create_bucket
11694 /b2api/v1/b2_list_buckets
11695 /b2api/v1/b2_list_file_names
11696 The b2_list_file_names request will be sent once for every 1k files in
11698 the remote path, providing the checksum and modification time of the
11699 listed files. As of version 1.33 issue #818
11700 (https://github.com/rclone/rclone/issues/818) causes extra requests to
11701 be sent when using B2 with Crypt. When a copy operation does not re‐
11702 quire any files to be uploaded, no more requests will be sent.
11703 Uploading files that do not require chunking, will send 2 requests per
11705 file upload:
11706 /b2api/v1/b2_get_upload_url
11708 /b2api/v1/b2_upload_file/
11709 Uploading files requiring chunking, will send 2 requests (one each to
11711 start and finish the upload) and another 2 requests for each chunk:
11712 /b2api/v1/b2_start_large_file
11714 /b2api/v1/b2_get_upload_part_url
11715 /b2api/v1/b2_upload_part/
11716 /b2api/v1/b2_finish_large_file
11717 Versions
11719 Versions can be viewed with the --b2-versions flag. When it is set
11720 rclone will show and act on older versions of files. For example
11721 Listing without --b2-versions
11723 $ rclone -q ls b2:cleanup-test
11725 9 one.txt
11726 And with
11728 $ rclone -q --b2-versions ls b2:cleanup-test
11730 9 one.txt
11731 8 one-v2016-07-04-141032-000.txt
11732 16 one-v2016-07-04-141003-000.txt
11733 15 one-v2016-07-02-155621-000.txt
11734 Showing that the current version is unchanged but older versions can be
11736 seen. These have the UTC date that they were uploaded to the server to
11737 the nearest millisecond appended to them.
11738 Note that when using --b2-versions no file write operations are permit‐
11740 ted, so you can’t upload files or delete them.
11741 B2 and rclone link
11743 Rclone supports generating file share links for private B2 buckets.
11744 They can either be for a file for example:
11745 ./rclone link B2:bucket/path/to/file.txt
11747 https://f002.backblazeb2.com/file/bucket/path/to/file.txt?Authorization=xxxxxxxx
11748 or if run on a directory you will get:
11750 ./rclone link B2:bucket/path
11752 https://f002.backblazeb2.com/file/bucket/path?Authorization=xxxxxxxx
11753 you can then use the authorization token (the part of the url from the
11755 ?Authorization= on) on any file path under that directory. For exam‐
11756 ple:
11757 https://f002.backblazeb2.com/file/bucket/path/to/file1?Authorization=xxxxxxxx
11759 https://f002.backblazeb2.com/file/bucket/path/file2?Authorization=xxxxxxxx
11760 https://f002.backblazeb2.com/file/bucket/path/folder/file3?Authorization=xxxxxxxx
11761 Standard Options
11763 Here are the standard options specific to b2 (Backblaze B2).
11764 –b2-account
11766 Account ID or Application Key ID
11767 · Config: account
11769 · Env Var: RCLONE_B2_ACCOUNT
11771 · Type: string
11773 · Default: ""
11775 –b2-key
11777 Application Key
11778 · Config: key
11780 · Env Var: RCLONE_B2_KEY
11782 · Type: string
11784 · Default: ""
11786 –b2-hard-delete
11788 Permanently delete files on remote removal, otherwise hide files.
11789 · Config: hard_delete
11791 · Env Var: RCLONE_B2_HARD_DELETE
11793 · Type: bool
11795 · Default: false
11797 Advanced Options
11799 Here are the advanced options specific to b2 (Backblaze B2).
11800 –b2-endpoint
11802 Endpoint for the service. Leave blank normally.
11803 · Config: endpoint
11805 · Env Var: RCLONE_B2_ENDPOINT
11807 · Type: string
11809 · Default: ""
11811 –b2-test-mode
11813 A flag string for X-Bz-Test-Mode header for debugging.
11814 This is for debugging purposes only. Setting it to one of the strings
11816 below will cause b2 to return specific errors:
11817 · “fail_some_uploads”
11819 · “expire_some_account_authorization_tokens”
11821 · “force_cap_exceeded”
11823 These will be set in the “X-Bz-Test-Mode” header which is documented in
11825 the b2 integrations checklist (https://www.backblaze.com/b2/docs/inte‐
11826 gration_checklist.html).
11827 · Config: test_mode
11829 · Env Var: RCLONE_B2_TEST_MODE
11831 · Type: string
11833 · Default: ""
11835 –b2-versions
11837 Include old versions in directory listings. Note that when using this
11838 no file write operations are permitted, so you can’t upload files or
11839 delete them.
11840 · Config: versions
11842 · Env Var: RCLONE_B2_VERSIONS
11844 · Type: bool
11846 · Default: false
11848 –b2-upload-cutoff
11850 Cutoff for switching to chunked upload.
11851 Files above this size will be uploaded in chunks of “–b2-chunk-size”.
11853 This value should be set no larger than 4.657GiB (== 5GB).
11855 · Config: upload_cutoff
11857 · Env Var: RCLONE_B2_UPLOAD_CUTOFF
11859 · Type: SizeSuffix
11861 · Default: 200M
11863 –b2-chunk-size
11865 Upload chunk size. Must fit in memory.
11866 When uploading large files, chunk the file into this size. Note that
11868 these chunks are buffered in memory and there might a maximum of
11869 “–transfers” chunks in progress at once. 5,000,000 Bytes is the mini‐
11870 mum size.
11871 · Config: chunk_size
11873 · Env Var: RCLONE_B2_CHUNK_SIZE
11875 · Type: SizeSuffix
11877 · Default: 96M
11879 –b2-disable-checksum
11881 Disable checksums for large (> upload cutoff) files
11882 · Config: disable_checksum
11884 · Env Var: RCLONE_B2_DISABLE_CHECKSUM
11886 · Type: bool
11888 · Default: false
11890 –b2-download-url
11892 Custom endpoint for downloads.
11893 This is usually set to a Cloudflare CDN URL as Backblaze offers free
11895 egress for data downloaded through the Cloudflare network. This is
11896 probably only useful for a public bucket. Leave blank if you want to
11897 use the endpoint provided by Backblaze.
11898 · Config: download_url
11900 · Env Var: RCLONE_B2_DOWNLOAD_URL
11902 · Type: string
11904 · Default: ""
11906 –b2-download-auth-duration
11908 Time before the authorization token will expire in s or suffix
11909 ms|s|m|h|d.
11910 The duration before the download authorization token will expire. The
11912 minimum value is 1 second. The maximum value is one week.
11913 · Config: download_auth_duration
11915 · Env Var: RCLONE_B2_DOWNLOAD_AUTH_DURATION
11917 · Type: Duration
11919 · Default: 1w
11921 –b2-encoding
11923 This sets the encoding for the backend.
11924 See: the encoding section in the overview (/overview/#encoding) for
11926 more info.
11927 · Config: encoding
11929 · Env Var: RCLONE_B2_ENCODING
11931 · Type: MultiEncoder
11933 · Default: Slash,BackSlash,Del,Ctl,InvalidUtf8,Dot
11935 Box
11937 Paths are specified as remote:path
11938 Paths may be as deep as required, eg remote:directory/subdirectory.
11940 The initial setup for Box involves getting a token from Box which you
11942 can do either in your browser, or with a config.json downloaded from
11943 Box to use JWT authentication. rclone config walks you through it.
11944 Here is an example of how to make a remote called remote. First run:
11946 rclone config
11948 This will guide you through an interactive setup process:
11950 No remotes found - make a new one
11952 n) New remote
11953 s) Set configuration password
11954 q) Quit config
11955 n/s/q> n
11956 name> remote
11957 Type of storage to configure.
11958 Choose a number from below, or type in your own value
11959 [snip]
11960 XX / Box
11961 \ "box"
11962 [snip]
11963 Storage> box
11964 Box App Client Id - leave blank normally.
11965 client_id>
11966 Box App Client Secret - leave blank normally.
11967 client_secret>
11968 Box App config.json location
11969 Leave blank normally.
11970 Enter a string value. Press Enter for the default ("").
11971 config_json>
11972 'enterprise' or 'user' depending on the type of token being requested.
11973 Enter a string value. Press Enter for the default ("user").
11974 box_sub_type>
11975 Remote config
11976 Use auto config?
11977 * Say Y if not sure
11978 * Say N if you are working on a remote or headless machine
11979 y) Yes
11980 n) No
11981 y/n> y
11982 If your browser doesn't open automatically go to the following link: http://127.0.0.1:53682/auth
11983 Log in and authorize rclone for access
11984 Waiting for code...
11985 Got code
11986 --------------------
11987 [remote]
11988 client_id =
11989 client_secret =
11990 token = {"access_token":"XXX","token_type":"bearer","refresh_token":"XXX","expiry":"XXX"}
11991 --------------------
11992 y) Yes this is OK
11993 e) Edit this remote
11994 d) Delete this remote
11995 y/e/d> y
11996 See the remote setup docs (https://rclone.org/remote_setup/) for how to
11998 set it up on a machine with no Internet browser available.
11999 Note that rclone runs a webserver on your local machine to collect the
12001 token as returned from Box. This only runs from the moment it opens
12002 your browser to the moment you get back the verification code. This is
12003 on http://127.0.0.1:53682/ and this it may require you to unblock it
12004 temporarily if you are running a host firewall.
12005 Once configured you can then use rclone like this,
12007 List directories in top level of your Box
12009 rclone lsd remote:
12011 List all the files in your Box
12013 rclone ls remote:
12015 To copy a local directory to an Box directory called backup
12017 rclone copy /home/source remote:backup
12019 Using rclone with an Enterprise account with SSO
12021 If you have an “Enterprise” account type with Box with single sign on
12022 (SSO), you need to create a password to use Box with rclone. This can
12023 be done at your Enterprise Box account by going to Settings, “Account”
12024 Tab, and then set the password in the “Authentication” field.
12025 Once you have done this, you can setup your Enterprise Box account us‐
12027 ing the same procedure detailed above in the, using the password you
12028 have just set.
12029 Invalid refresh token
12031 According to the box docs (https://develop‐
12032 er.box.com/v2.0/docs/oauth-20#section-6-using-the-access-and-refresh-
12033 tokens):
12034 Each refresh_token is valid for one use in 60 days.
12036 This means that if you
12038 · Don’t use the box remote for 60 days
12040 · Copy the config file with a box refresh token in and use it in two
12042 places
12043 · Get an error on a token refresh
12045 then rclone will return an error which includes the text Invalid re‐
12047 fresh token.
12048 To fix this you will need to use oauth2 again to update the refresh to‐
12050 ken. You can use the methods in the remote setup docs
12051 (https://rclone.org/remote_setup/), bearing in mind that if you use the
12052 copy the config file method, you should not use that remote on the com‐
12053 puter you did the authentication on.
12054 Here is how to do it.
12056 $ rclone config
12058 Current remotes:
12059 Name Type
12061 ==== ====
12062 remote box
12063 e) Edit existing remote
12065 n) New remote
12066 d) Delete remote
12067 r) Rename remote
12068 c) Copy remote
12069 s) Set configuration password
12070 q) Quit config
12071 e/n/d/r/c/s/q> e
12072 Choose a number from below, or type in an existing value
12073 1 > remote
12074 remote> remote
12075 --------------------
12076 [remote]
12077 type = box
12078 token = {"access_token":"XXX","token_type":"bearer","refresh_token":"XXX","expiry":"2017-07-08T23:40:08.059167677+01:00"}
12079 --------------------
12080 Edit remote
12081 Value "client_id" = ""
12082 Edit? (y/n)>
12083 y) Yes
12084 n) No
12085 y/n> n
12086 Value "client_secret" = ""
12087 Edit? (y/n)>
12088 y) Yes
12089 n) No
12090 y/n> n
12091 Remote config
12092 Already have a token - refresh?
12093 y) Yes
12094 n) No
12095 y/n> y
12096 Use auto config?
12097 * Say Y if not sure
12098 * Say N if you are working on a remote or headless machine
12099 y) Yes
12100 n) No
12101 y/n> y
12102 If your browser doesn't open automatically go to the following link: http://127.0.0.1:53682/auth
12103 Log in and authorize rclone for access
12104 Waiting for code...
12105 Got code
12106 --------------------
12107 [remote]
12108 type = box
12109 token = {"access_token":"YYY","token_type":"bearer","refresh_token":"YYY","expiry":"2017-07-23T12:22:29.259137901+01:00"}
12110 --------------------
12111 y) Yes this is OK
12112 e) Edit this remote
12113 d) Delete this remote
12114 y/e/d> y
12115 Modified time and hashes
12117 Box allows modification times to be set on objects accurate to 1 sec‐
12118 ond. These will be used to detect whether objects need syncing or not.
12119 Box supports SHA1 type hashes, so you can use the --checksum flag.
12121 Restricted filename characters
12123 In addition to the default restricted characters set (/overview/#re‐
12124 stricted-characters) the following characters are also replaced:
12125 Character Value Replacement
12127 ────────────────────────────────
12128 \ 0x5C ＼
12129 File names can also not end with the following characters. These only
12131 get replaced if they are last character in the name:
12132 Character Value Replacement
12134 ────────────────────────────────
12135 SP 0x20 ␠
12136 Invalid UTF-8 bytes will also be replaced (/overview/#invalid-utf8), as
12138 they can’t be used in JSON strings.
12139 Transfers
12141 For files above 50MB rclone will use a chunked transfer. Rclone will
12142 upload up to --transfers chunks at the same time (shared among all the
12143 multipart uploads). Chunks are buffered in memory and are normally 8MB
12144 so increasing --transfers will increase memory use.
12145 Deleting files
12147 Depending on the enterprise settings for your user, the item will ei‐
12148 ther be actually deleted from Box or moved to the trash.
12149 Standard Options
12151 Here are the standard options specific to box (Box).
12152 –box-client-id
12154 Box App Client Id. Leave blank normally.
12155 · Config: client_id
12157 · Env Var: RCLONE_BOX_CLIENT_ID
12159 · Type: string
12161 · Default: ""
12163 –box-client-secret
12165 Box App Client Secret Leave blank normally.
12166 · Config: client_secret
12168 · Env Var: RCLONE_BOX_CLIENT_SECRET
12170 · Type: string
12172 · Default: ""
12174 –box-box-config-file
12176 Box App config.json location Leave blank normally.
12177 · Config: box_config_file
12179 · Env Var: RCLONE_BOX_BOX_CONFIG_FILE
12181 · Type: string
12183 · Default: ""
12185 –box-box-sub-type
12187 · Config: box_sub_type
12188 · Env Var: RCLONE_BOX_BOX_SUB_TYPE
12190 · Type: string
12192 · Default: “user”
12194 · Examples:
12196 · “user”
12198 · Rclone should act on behalf of a user
12200 · “enterprise”
12202 · Rclone should act on behalf of a service account
12204 Advanced Options
12206 Here are the advanced options specific to box (Box).
12207 –box-upload-cutoff
12209 Cutoff for switching to multipart upload (>= 50MB).
12210 · Config: upload_cutoff
12212 · Env Var: RCLONE_BOX_UPLOAD_CUTOFF
12214 · Type: SizeSuffix
12216 · Default: 50M
12218 –box-commit-retries
12220 Max number of times to try committing a multipart file.
12221 · Config: commit_retries
12223 · Env Var: RCLONE_BOX_COMMIT_RETRIES
12225 · Type: int
12227 · Default: 100
12229 –box-encoding
12231 This sets the encoding for the backend.
12232 See: the encoding section in the overview (/overview/#encoding) for
12234 more info.
12235 · Config: encoding
12237 · Env Var: RCLONE_BOX_ENCODING
12239 · Type: MultiEncoder
12241 · Default: Slash,BackSlash,Del,Ctl,RightSpace,InvalidUtf8,Dot
12243 Limitations
12245 Note that Box is case insensitive so you can’t have a file called “Hel‐
12246 lo.doc” and one called “hello.doc”.
12247 Box file names can’t have the \ character in. rclone maps this to and
12249 from an identical looking unicode equivalent ＼.
12250 Box only supports filenames up to 255 characters in length.
12252 Cache (BETA)
12254 The cache remote wraps another existing remote and stores file struc‐
12255 ture and its data for long running tasks like rclone mount.
12256 To get started you just need to have an existing remote which can be
12258 configured with cache.
12259 Here is an example of how to make a remote called test-cache. First
12261 run:
12262 rclone config
12264 This will guide you through an interactive setup process:
12266 No remotes found - make a new one
12268 n) New remote
12269 r) Rename remote
12270 c) Copy remote
12271 s) Set configuration password
12272 q) Quit config
12273 n/r/c/s/q> n
12274 name> test-cache
12275 Type of storage to configure.
12276 Choose a number from below, or type in your own value
12277 [snip]
12278 XX / Cache a remote
12279 \ "cache"
12280 [snip]
12281 Storage> cache
12282 Remote to cache.
12283 Normally should contain a ':' and a path, eg "myremote:path/to/dir",
12284 "myremote:bucket" or maybe "myremote:" (not recommended).
12285 remote> local:/test
12286 Optional: The URL of the Plex server
12287 plex_url> http://127.0.0.1:32400
12288 Optional: The username of the Plex user
12289 plex_username> dummyusername
12290 Optional: The password of the Plex user
12291 y) Yes type in my own password
12292 g) Generate random password
12293 n) No leave this optional password blank
12294 y/g/n> y
12295 Enter the password:
12296 password:
12297 Confirm the password:
12298 password:
12299 The size of a chunk. Lower value good for slow connections but can affect seamless reading.
12300 Default: 5M
12301 Choose a number from below, or type in your own value
12302 1 / 1MB
12303 \ "1m"
12304 2 / 5 MB
12305 \ "5M"
12306 3 / 10 MB
12307 \ "10M"
12308 chunk_size> 2
12309 How much time should object info (file size, file hashes etc) be stored in cache. Use a very high value if you don't plan on changing the source FS from outside the cache.
12310 Accepted units are: "s", "m", "h".
12311 Default: 5m
12312 Choose a number from below, or type in your own value
12313 1 / 1 hour
12314 \ "1h"
12315 2 / 24 hours
12316 \ "24h"
12317 3 / 24 hours
12318 \ "48h"
12319 info_age> 2
12320 The maximum size of stored chunks. When the storage grows beyond this size, the oldest chunks will be deleted.
12321 Default: 10G
12322 Choose a number from below, or type in your own value
12323 1 / 500 MB
12324 \ "500M"
12325 2 / 1 GB
12326 \ "1G"
12327 3 / 10 GB
12328 \ "10G"
12329 chunk_total_size> 3
12330 Remote config
12331 --------------------
12332 [test-cache]
12333 remote = local:/test
12334 plex_url = http://127.0.0.1:32400
12335 plex_username = dummyusername
12336 plex_password = *** ENCRYPTED ***
12337 chunk_size = 5M
12338 info_age = 48h
12339 chunk_total_size = 10G
12340 You can then use it like this,
12342 List directories in top level of your drive
12344 rclone lsd test-cache:
12346 List all the files in your drive
12348 rclone ls test-cache:
12350 To start a cached mount
12352 rclone mount --allow-other test-cache: /var/tmp/test-cache
12354 Write Features
12356 Offline uploading
12357 In an effort to make writing through cache more reliable, the backend
12358 now supports this feature which can be activated by specifying a
12359 cache-tmp-upload-path.
12360 A files goes through these states when using this feature:
12362 1. An upload is started (usually by copying a file on the cache remote)
12364 2. When the copy to the temporary location is complete the file is part
12366 of the cached remote and looks and behaves like any other file
12367 (reading included)
12368 3. After cache-tmp-wait-time passes and the file is next in line,
12370 rclone move is used to move the file to the cloud provider
12371 4. Reading the file still works during the upload but most modifica‐
12373 tions on it will be prohibited
12374 5. Once the move is complete the file is unlocked for modifications as
12376 it becomes as any other regular file
12377 6. If the file is being read through cache when it’s actually deleted
12379 from the temporary path then cache will simply swap the source to
12380 the cloud provider without interrupting the reading (small blip can
12381 happen though)
12382 Files are uploaded in sequence and only one file is uploaded at a time.
12384 Uploads will be stored in a queue and be processed based on the order
12385 they were added. The queue and the temporary storage is persistent
12386 across restarts but can be cleared on startup with the --cache-db-purge
12387 flag.
12388 Write Support
12390 Writes are supported through cache. One caveat is that a mounted cache
12391 remote does not add any retry or fallback mechanism to the upload oper‐
12392 ation. This will depend on the implementation of the wrapped remote.
12393 Consider using Offline uploading for reliable writes.
12394 One special case is covered with cache-writes which will cache the file
12396 data at the same time as the upload when it is enabled making it avail‐
12397 able from the cache store immediately once the upload is finished.
12398 Read Features
12400 Multiple connections
12401 To counter the high latency between a local PC where rclone is running
12402 and cloud providers, the cache remote can split multiple requests to
12403 the cloud provider for smaller file chunks and combines them together
12404 locally where they can be available almost immediately before the read‐
12405 er usually needs them.
12406 This is similar to buffering when media files are played online.
12408 Rclone will stay around the current marker but always try its best to
12409 stay ahead and prepare the data before.
12410 Plex Integration
12412 There is a direct integration with Plex which allows cache to detect
12413 during reading if the file is in playback or not. This helps cache to
12414 adapt how it queries the cloud provider depending on what is needed
12415 for.
12416 Scans will have a minimum amount of workers (1) while in a confirmed
12418 playback cache will deploy the configured number of workers.
12419 This integration opens the doorway to additional performance improve‐
12421 ments which will be explored in the near future.
12422 Note: If Plex options are not configured, cache will function with its
12424 configured options without adapting any of its settings.
12425 How to enable? Run rclone config and add all the Plex options (end‐
12427 point, username and password) in your remote and it will be automati‐
12428 cally enabled.
12429 Affected settings: - cache-workers: Configured value during confirmed
12431 playback or 1 all the other times
12432 Certificate Validation
12434 When the Plex server is configured to only accept secure connections,
12435 it is possible to use .plex.direct URL’s to ensure certificate valida‐
12436 tion succeeds. These URL’s are used by Plex internally to connect to
12437 the Plex server securely.
12438 The format for this URL’s is the following:
12440 https://ip-with-dots-replaced.server-hash.plex.direct:32400/
12442 The ip-with-dots-replaced part can be any IPv4 address, where the dots
12444 have been replaced with dashes, e.g. 127.0.0.1 becomes 127-0-0-1.
12445 To get the server-hash part, the easiest way is to visit
12447 https://plex.tv/api/resources?includeHttps=1&X-Plex-Token=your-plex-to‐
12449 ken
12450 This page will list all the available Plex servers for your account
12452 with at least one .plex.direct link for each. Copy one URL and replace
12453 the IP address with the desired address. This can be used as the
12454 plex_url value.
12455 Known issues
12457 Mount and –dir-cache-time
12458 –dir-cache-time controls the first layer of directory caching which
12459 works at the mount layer. Being an independent caching mechanism from
12460 the cache backend, it will manage its own entries based on the config‐
12461 ured time.
12462 To avoid getting in a scenario where dir cache has obsolete data and
12464 cache would have the correct one, try to set --dir-cache-time to a low‐
12465 er time than --cache-info-age. Default values are already configured
12466 in this way.
12467 Windows support - Experimental
12469 There are a couple of issues with Windows mount functionality that
12470 still require some investigations. It should be considered as experi‐
12471 mental thus far as fixes come in for this OS.
12472 Most of the issues seem to be related to the difference between
12474 filesystems on Linux flavors and Windows as cache is heavily dependant
12475 on them.
12476 Any reports or feedback on how cache behaves on this OS is greatly ap‐
12478 preciated.
12479 · https://github.com/rclone/rclone/issues/1935
12481 · https://github.com/rclone/rclone/issues/1907
12483 · https://github.com/rclone/rclone/issues/1834
12485 Risk of throttling
12487 Future iterations of the cache backend will make use of the pooling
12488 functionality of the cloud provider to synchronize and at the same time
12489 make writing through it more tolerant to failures.
12490 There are a couple of enhancements in track to add these but in the
12492 meantime there is a valid concern that the expiring cache listings can
12493 lead to cloud provider throttles or bans due to repeated queries on it
12494 for very large mounts.
12495 Some recommendations: - don’t use a very small interval for entry in‐
12497 formations (--cache-info-age) - while writes aren’t yet optimised, you
12498 can still write through cache which gives you the advantage of adding
12499 the file in the cache at the same time if configured to do so.
12500 Future enhancements:
12502 · https://github.com/rclone/rclone/issues/1937
12504 · https://github.com/rclone/rclone/issues/1936
12506 cache and crypt
12508 One common scenario is to keep your data encrypted in the cloud
12509 provider using the crypt remote. crypt uses a similar technique to
12510 wrap around an existing remote and handles this translation in a seam‐
12511 less way.
12512 There is an issue with wrapping the remotes in this order: cloud remote
12514 -> crypt -> cache
12515 During testing, I experienced a lot of bans with the remotes in this
12517 order. I suspect it might be related to how crypt opens files on the
12518 cloud provider which makes it think we’re downloading the full file in‐
12519 stead of small chunks. Organizing the remotes in this order yields
12520 better results: cloud remote -> cache -> crypt
12521 absolute remote paths
12523 cache can not differentiate between relative and absolute paths for the
12524 wrapped remote. Any path given in the remote config setting and on the
12525 command line will be passed to the wrapped remote as is, but for stor‐
12526 ing the chunks on disk the path will be made relative by removing any
12527 leading / character.
12528 This behavior is irrelevant for most backend types, but there are back‐
12530 ends where a leading / changes the effective directory, e.g. in the
12531 sftp backend paths starting with a / are relative to the root of the
12532 SSH server and paths without are relative to the user home directory.
12533 As a result sftp:bin and sftp:/bin will share the same cache folder,
12534 even if they represent a different directory on the SSH server.
12535 Cache and Remote Control (–rc)
12537 Cache supports the new --rc mode in rclone and can be remote controlled
12538 through the following end points: By default, the listener is disabled
12539 if you do not add the flag.
12540 rc cache/expire
12542 Purge a remote from the cache backend. Supports either a directory or
12543 a file. It supports both encrypted and unencrypted file names if cache
12544 is wrapped by crypt.
12545 Params: - remote = path to remote (required) - withData = true/false to
12547 delete cached data (chunks) as well (optional, false by default)
12548 Standard Options
12550 Here are the standard options specific to cache (Cache a remote).
12551 –cache-remote
12553 Remote to cache. Normally should contain a `:' and a path, eg “myre‐
12554 mote:path/to/dir”, “myremote:bucket” or maybe “myremote:” (not recom‐
12555 mended).
12556 · Config: remote
12558 · Env Var: RCLONE_CACHE_REMOTE
12560 · Type: string
12562 · Default: ""
12564 –cache-plex-url
12566 The URL of the Plex server
12567 · Config: plex_url
12569 · Env Var: RCLONE_CACHE_PLEX_URL
12571 · Type: string
12573 · Default: ""
12575 –cache-plex-username
12577 The username of the Plex user
12578 · Config: plex_username
12580 · Env Var: RCLONE_CACHE_PLEX_USERNAME
12582 · Type: string
12584 · Default: ""
12586 –cache-plex-password
12588 The password of the Plex user
12589 · Config: plex_password
12591 · Env Var: RCLONE_CACHE_PLEX_PASSWORD
12593 · Type: string
12595 · Default: ""
12597 –cache-chunk-size
12599 The size of a chunk (partial file data).
12600 Use lower numbers for slower connections. If the chunk size is
12602 changed, any downloaded chunks will be invalid and cache-chunk-path
12603 will need to be cleared or unexpected EOF errors will occur.
12604 · Config: chunk_size
12606 · Env Var: RCLONE_CACHE_CHUNK_SIZE
12608 · Type: SizeSuffix
12610 · Default: 5M
12612 · Examples:
12614 · “1m”
12616 · 1MB
12618 · “5M”
12620 · 5 MB
12622 · “10M”
12624 · 10 MB
12626 –cache-info-age
12628 How long to cache file structure information (directory listings, file
12629 size, times etc). If all write operations are done through the cache
12630 then you can safely make this value very large as the cache store will
12631 also be updated in real time.
12632 · Config: info_age
12634 · Env Var: RCLONE_CACHE_INFO_AGE
12636 · Type: Duration
12638 · Default: 6h0m0s
12640 · Examples:
12642 · “1h”
12644 · 1 hour
12646 · “24h”
12648 · 24 hours
12650 · “48h”
12652 · 48 hours
12654 –cache-chunk-total-size
12656 The total size that the chunks can take up on the local disk.
12657 If the cache exceeds this value then it will start to delete the oldest
12659 chunks until it goes under this value.
12660 · Config: chunk_total_size
12662 · Env Var: RCLONE_CACHE_CHUNK_TOTAL_SIZE
12664 · Type: SizeSuffix
12666 · Default: 10G
12668 · Examples:
12670 · “500M”
12672 · 500 MB
12674 · “1G”
12676 · 1 GB
12678 · “10G”
12680 · 10 GB
12682 Advanced Options
12684 Here are the advanced options specific to cache (Cache a remote).
12685 –cache-plex-token
12687 The plex token for authentication - auto set normally
12688 · Config: plex_token
12690 · Env Var: RCLONE_CACHE_PLEX_TOKEN
12692 · Type: string
12694 · Default: ""
12696 –cache-plex-insecure
12698 Skip all certificate verifications when connecting to the Plex server
12699 · Config: plex_insecure
12701 · Env Var: RCLONE_CACHE_PLEX_INSECURE
12703 · Type: string
12705 · Default: ""
12707 –cache-db-path
12709 Directory to store file structure metadata DB. The remote name is used
12710 as the DB file name.
12711 · Config: db_path
12713 · Env Var: RCLONE_CACHE_DB_PATH
12715 · Type: string
12717 · Default: “$HOME/.cache/rclone/cache-backend”
12719 –cache-chunk-path
12721 Directory to cache chunk files.
12722 Path to where partial file data (chunks) are stored locally. The re‐
12724 mote name is appended to the final path.
12725 This config follows the “–cache-db-path”. If you specify a custom lo‐
12727 cation for “–cache-db-path” and don’t specify one for
12728 “–cache-chunk-path” then “–cache-chunk-path” will use the same path as
12729 “–cache-db-path”.
12730 · Config: chunk_path
12732 · Env Var: RCLONE_CACHE_CHUNK_PATH
12734 · Type: string
12736 · Default: “$HOME/.cache/rclone/cache-backend”
12738 –cache-db-purge
12740 Clear all the cached data for this remote on start.
12741 · Config: db_purge
12743 · Env Var: RCLONE_CACHE_DB_PURGE
12745 · Type: bool
12747 · Default: false
12749 –cache-chunk-clean-interval
12751 How often should the cache perform cleanups of the chunk storage. The
12752 default value should be ok for most people. If you find that the cache
12753 goes over “cache-chunk-total-size” too often then try to lower this
12754 value to force it to perform cleanups more often.
12755 · Config: chunk_clean_interval
12757 · Env Var: RCLONE_CACHE_CHUNK_CLEAN_INTERVAL
12759 · Type: Duration
12761 · Default: 1m0s
12763 –cache-read-retries
12765 How many times to retry a read from a cache storage.
12766 Since reading from a cache stream is independent from downloading file
12768 data, readers can get to a point where there’s no more data in the
12769 cache. Most of the times this can indicate a connectivity issue if
12770 cache isn’t able to provide file data anymore.
12771 For really slow connections, increase this to a point where the stream
12773 is able to provide data but your experience will be very stuttering.
12774 · Config: read_retries
12776 · Env Var: RCLONE_CACHE_READ_RETRIES
12778 · Type: int
12780 · Default: 10
12782 –cache-workers
12784 How many workers should run in parallel to download chunks.
12785 Higher values will mean more parallel processing (better CPU needed)
12787 and more concurrent requests on the cloud provider. This impacts sev‐
12788 eral aspects like the cloud provider API limits, more stress on the
12789 hardware that rclone runs on but it also means that streams will be
12790 more fluid and data will be available much more faster to readers.
12791 Note: If the optional Plex integration is enabled then this setting
12793 will adapt to the type of reading performed and the value specified
12794 here will be used as a maximum number of workers to use.
12795 · Config: workers
12797 · Env Var: RCLONE_CACHE_WORKERS
12799 · Type: int
12801 · Default: 4
12803 –cache-chunk-no-memory
12805 Disable the in-memory cache for storing chunks during streaming.
12806 By default, cache will keep file data during streaming in RAM as well
12808 to provide it to readers as fast as possible.
12809 This transient data is evicted as soon as it is read and the number of
12811 chunks stored doesn’t exceed the number of workers. However, depending
12812 on other settings like “cache-chunk-size” and “cache-workers” this
12813 footprint can increase if there are parallel streams too (multiple
12814 files being read at the same time).
12815 If the hardware permits it, use this feature to provide an overall bet‐
12817 ter performance during streaming but it can also be disabled if RAM is
12818 not available on the local machine.
12819 · Config: chunk_no_memory
12821 · Env Var: RCLONE_CACHE_CHUNK_NO_MEMORY
12823 · Type: bool
12825 · Default: false
12827 –cache-rps
12829 Limits the number of requests per second to the source FS (-1 to dis‐
12830 able)
12831 This setting places a hard limit on the number of requests per second
12833 that cache will be doing to the cloud provider remote and try to re‐
12834 spect that value by setting waits between reads.
12835 If you find that you’re getting banned or limited on the cloud provider
12837 through cache and know that a smaller number of requests per second
12838 will allow you to work with it then you can use this setting for that.
12839 A good balance of all the other settings should make this setting use‐
12841 less but it is available to set for more special cases.
12842 NOTE: This will limit the number of requests during streams but other
12844 API calls to the cloud provider like directory listings will still
12845 pass.
12846 · Config: rps
12848 · Env Var: RCLONE_CACHE_RPS
12850 · Type: int
12852 · Default: -1
12854 –cache-writes
12856 Cache file data on writes through the FS
12857 If you need to read files immediately after you upload them through
12859 cache you can enable this flag to have their data stored in the cache
12860 store at the same time during upload.
12861 · Config: writes
12863 · Env Var: RCLONE_CACHE_WRITES
12865 · Type: bool
12867 · Default: false
12869 –cache-tmp-upload-path
12871 Directory to keep temporary files until they are uploaded.
12872 This is the path where cache will use as a temporary storage for new
12874 files that need to be uploaded to the cloud provider.
12875 Specifying a value will enable this feature. Without it, it is com‐
12877 pletely disabled and files will be uploaded directly to the cloud
12878 provider
12879 · Config: tmp_upload_path
12881 · Env Var: RCLONE_CACHE_TMP_UPLOAD_PATH
12883 · Type: string
12885 · Default: ""
12887 –cache-tmp-wait-time
12889 How long should files be stored in local cache before being uploaded
12890 This is the duration that a file must wait in the temporary location
12892 cache-tmp-upload-path before it is selected for upload.
12893 Note that only one file is uploaded at a time and it can take longer to
12895 start the upload if a queue formed for this purpose.
12896 · Config: tmp_wait_time
12898 · Env Var: RCLONE_CACHE_TMP_WAIT_TIME
12900 · Type: Duration
12902 · Default: 15s
12904 –cache-db-wait-time
12906 How long to wait for the DB to be available - 0 is unlimited
12907 Only one process can have the DB open at any one time, so rclone waits
12909 for this duration for the DB to become available before it gives an er‐
12910 ror.
12911 If you set it to 0 then it will wait forever.
12913 · Config: db_wait_time
12915 · Env Var: RCLONE_CACHE_DB_WAIT_TIME
12917 · Type: Duration
12919 · Default: 1s
12921 Chunker (BETA)
12923 The chunker overlay transparently splits large files into smaller
12924 chunks during upload to wrapped remote and transparently assembles them
12925 back when the file is downloaded. This allows to effectively overcome
12926 size limits imposed by storage providers.
12927 To use it, first set up the underlying remote following the configura‐
12929 tion instructions for that remote. You can also use a local pathname
12930 instead of a remote.
12931 First check your chosen remote is working - we’ll call it remote:path
12933 here. Note that anything inside remote:path will be chunked and any‐
12934 thing outside won’t. This means that if you are using a bucket based
12935 remote (eg S3, B2, swift) then you should probably put the bucket in
12936 the remote s3:bucket.
12937 Now configure chunker using rclone config. We will call this one over‐
12939 lay to separate it from the remote itself.
12940 No remotes found - make a new one
12942 n) New remote
12943 s) Set configuration password
12944 q) Quit config
12945 n/s/q> n
12946 name> overlay
12947 Type of storage to configure.
12948 Choose a number from below, or type in your own value
12949 [snip]
12950 XX / Transparently chunk/split large files
12951 \ "chunker"
12952 [snip]
12953 Storage> chunker
12954 Remote to chunk/unchunk.
12955 Normally should contain a ':' and a path, eg "myremote:path/to/dir",
12956 "myremote:bucket" or maybe "myremote:" (not recommended).
12957 Enter a string value. Press Enter for the default ("").
12958 remote> remote:path
12959 Files larger than chunk size will be split in chunks.
12960 Enter a size with suffix k,M,G,T. Press Enter for the default ("2G").
12961 chunk_size> 100M
12962 Choose how chunker handles hash sums. All modes but "none" require metadata.
12963 Enter a string value. Press Enter for the default ("md5").
12964 Choose a number from below, or type in your own value
12965 1 / Pass any hash supported by wrapped remote for non-chunked files, return nothing otherwise
12966 \ "none"
12967 2 / MD5 for composite files
12968 \ "md5"
12969 3 / SHA1 for composite files
12970 \ "sha1"
12971 4 / MD5 for all files
12972 \ "md5all"
12973 5 / SHA1 for all files
12974 \ "sha1all"
12975 6 / Copying a file to chunker will request MD5 from the source falling back to SHA1 if unsupported
12976 \ "md5quick"
12977 7 / Similar to "md5quick" but prefers SHA1 over MD5
12978 \ "sha1quick"
12979 hash_type> md5
12980 Edit advanced config? (y/n)
12981 y) Yes
12982 n) No
12983 y/n> n
12984 Remote config
12985 --------------------
12986 [overlay]
12987 type = chunker
12988 remote = remote:bucket
12989 chunk_size = 100M
12990 hash_type = md5
12991 --------------------
12992 y) Yes this is OK
12993 e) Edit this remote
12994 d) Delete this remote
12995 y/e/d> y
12996 Specifying the remote
12998 In normal use, make sure the remote has a : in. If you specify the re‐
12999 mote without a : then rclone will use a local directory of that name.
13000 So if you use a remote of /path/to/secret/files then rclone will chunk
13001 stuff in that directory. If you use a remote of name then rclone will
13002 put files in a directory called name in the current directory.
13003 Chunking
13005 When rclone starts a file upload, chunker checks the file size. If it
13006 doesn’t exceed the configured chunk size, chunker will just pass the
13007 file to the wrapped remote. If a file is large, chunker will transpar‐
13008 ently cut data in pieces with temporary names and stream them one by
13009 one, on the fly. Each data chunk will contain the specified number of
13010 bytes, except for the last one which may have less data. If file size
13011 is unknown in advance (this is called a streaming upload), chunker will
13012 internally create a temporary copy, record its size and repeat the
13013 above process.
13014 When upload completes, temporary chunk files are finally renamed. This
13016 scheme guarantees that operations can be run in parallel and look from
13017 outside as atomic. A similar method with hidden temporary chunks is
13018 used for other operations (copy/move/rename etc). If an operation
13019 fails, hidden chunks are normally destroyed, and the target composite
13020 file stays intact.
13021 When a composite file download is requested, chunker transparently as‐
13023 sembles it by concatenating data chunks in order. As the split is
13024 trivial one could even manually concatenate data chunks together to ob‐
13025 tain the original content.
13026 When the list rclone command scans a directory on wrapped remote, the
13028 potential chunk files are accounted for, grouped and assembled into
13029 composite directory entries. Any temporary chunks are hidden.
13030 List and other commands can sometimes come across composite files with
13032 missing or invalid chunks, eg. shadowed by like-named directory or an‐
13033 other file. This usually means that wrapped file system has been di‐
13034 rectly tampered with or damaged. If chunker detects a missing chunk it
13035 will by default print warning, skip the whole incomplete group of
13036 chunks but proceed with current command. You can set the --chun‐
13037 ker-fail-hard flag to have commands abort with error message in such
13038 cases.
13039 Chunk names
13041 The default chunk name format is *.rclone_chunk.###, hence by default
13042 chunk names are BIG_FILE_NAME.rclone_chunk.001,
13043 BIG_FILE_NAME.rclone_chunk.002 etc. You can configure another name
13044 format using the name_format configuration file option. The format us‐
13045 es asterisk * as a placeholder for the base file name and one or more
13046 consecutive hash characters # as a placeholder for sequential chunk
13047 number. There must be one and only one asterisk. The number of con‐
13048 secutive hash characters defines the minimum length of a string repre‐
13049 senting a chunk number. If decimal chunk number has less digits than
13050 the number of hashes, it is left-padded by zeros. If the decimal
13051 string is longer, it is left intact. By default numbering starts from
13052 1 but there is another option that allows user to start from 0, eg.
13053 for compatibility with legacy software.
13054 For example, if name format is big_*-##.part and original file name is
13056 data.txt and numbering starts from 0, then the first chunk will be
13057 named big_data.txt-00.part, the 99th chunk will be big_data.txt-98.part
13058 and the 302nd chunk will become big_data.txt-301.part.
13059 Note that list assembles composite directory entries only when chunk
13061 names match the configured format and treats non-conforming file names
13062 as normal non-chunked files.
13063 Metadata
13065 Besides data chunks chunker will by default create metadata object for
13066 a composite file. The object is named after the original file. Chun‐
13067 ker allows user to disable metadata completely (the none format). Note
13068 that metadata is normally not created for files smaller than the con‐
13069 figured chunk size. This may change in future rclone releases.
13070 Simple JSON metadata format
13072 This is the default format. It supports hash sums and chunk validation
13073 for composite files. Meta objects carry the following fields:
13074 · ver - version of format, currently 1
13076 · size - total size of composite file
13078 · nchunks - number of data chunks in file
13080 · md5 - MD5 hashsum of composite file (if present)
13082 · sha1 - SHA1 hashsum (if present)
13084 There is no field for composite file name as it’s simply equal to the
13086 name of meta object on the wrapped remote. Please refer to respective
13087 sections for details on hashsums and modified time handling.
13088 No metadata
13090 You can disable meta objects by setting the meta format option to none.
13091 In this mode chunker will scan directory for all files that follow con‐
13092 figured chunk name format, group them by detecting chunks with the same
13093 base name and show group names as virtual composite files. This method
13094 is more prone to missing chunk errors (especially missing last chunk)
13095 than format with metadata enabled.
13096 Hashsums
13098 Chunker supports hashsums only when a compatible metadata is present.
13099 Hence, if you choose metadata format of none, chunker will report hash‐
13100 sum as UNSUPPORTED.
13101 Please note that by default metadata is stored only for composite
13103 files. If a file is smaller than configured chunk size, chunker will
13104 transparently redirect hash requests to wrapped remote, so support de‐
13105 pends on that. You will see the empty string as a hashsum of requested
13106 type for small files if the wrapped remote doesn’t support it.
13107 Many storage backends support MD5 and SHA1 hash types, so does chunker.
13109 With chunker you can choose one or another but not both. MD5 is set by
13110 default as the most supported type. Since chunker keeps hashes for
13111 composite files and falls back to the wrapped remote hash for non-chun‐
13112 ked ones, we advise you to choose the same hash type as supported by
13113 wrapped remote so that your file listings look coherent.
13114 If your storage backend does not support MD5 or SHA1 but you need con‐
13116 sistent file hashing, configure chunker with md5all or sha1all. These
13117 two modes guarantee given hash for all files. If wrapped remote
13118 doesn’t support it, chunker will then add metadata to all files, even
13119 small. However, this can double the amount of small files in storage
13120 and incur additional service charges. You can even use chunker to
13121 force md5/sha1 support in any other remote at expence of sidecar meta
13122 objects by setting eg. chunk_type=sha1all to force hashsums and
13123 chunk_size=1P to effectively disable chunking.
13124 Normally, when a file is copied to chunker controlled remote, chunker
13126 will ask the file source for compatible file hash and revert to
13127 on-the-fly calculation if none is found. This involves some CPU over‐
13128 head but provides a guarantee that given hashsum is available. Also,
13129 chunker will reject a server-side copy or move operation if source and
13130 destination hashsum types are different resulting in the extra network
13131 bandwidth, too. In some rare cases this may be undesired, so chunker
13132 provides two optional choices: sha1quick and md5quick. If the source
13133 does not support primary hash type and the quick mode is enabled, chun‐
13134 ker will try to fall back to the secondary type. This will save CPU
13135 and bandwidth but can result in empty hashsums at destination. Beware
13136 of consequences: the sync command will revert (sometimes silently) to
13137 time/size comparison if compatible hashsums between source and target
13138 are not found.
13139 Modified time
13141 Chunker stores modification times using the wrapped remote so support
13142 depends on that. For a small non-chunked file the chunker overlay sim‐
13143 ply manipulates modification time of the wrapped remote file. For a
13144 composite file with metadata chunker will get and set modification time
13145 of the metadata object on the wrapped remote. If file is chunked but
13146 metadata format is none then chunker will use modification time of the
13147 first data chunk.
13148 Migrations
13150 The idiomatic way to migrate to a different chunk size, hash type or
13151 chunk naming scheme is to:
13152 · Collect all your chunked files under a directory and have your chun‐
13154 ker remote point to it.
13155 · Create another directory (most probably on the same cloud storage)
13157 and configure a new remote with desired metadata format, hash type,
13158 chunk naming etc.
13159 · Now run rclone sync oldchunks: newchunks: and all your data will be
13161 transparently converted in transfer. This may take some time, yet
13162 chunker will try server-side copy if possible.
13163 · After checking data integrity you may remove configuration section of
13165 the old remote.
13166 If rclone gets killed during a long operation on a big composite file,
13168 hidden temporary chunks may stay in the directory. They will not be
13169 shown by the list command but will eat up your account quota. Please
13170 note that the deletefile command deletes only active chunks of a file.
13171 As a workaround, you can use remote of the wrapped file system to see
13172 them. An easy way to get rid of hidden garbage is to copy littered di‐
13173 rectory somewhere using the chunker remote and purge the original di‐
13174 rectory. The copy command will copy only active chunks while the purge
13175 will remove everything including garbage.
13176 Caveats and Limitations
13178 Chunker requires wrapped remote to support server side move (or copy +
13179 delete) operations, otherwise it will explicitly refuse to start. This
13180 is because it internally renames temporary chunk files to their final
13181 names when an operation completes successfully.
13182 Chunker encodes chunk number in file name, so with default name_format
13184 setting it adds 17 characters. Also chunker adds 7 characters of tem‐
13185 porary suffix during operations. Many file systems limit base file
13186 name without path by 255 characters. Using rclone’s crypt remote as a
13187 base file system limits file name by 143 characters. Thus, maximum
13188 name length is 231 for most files and 119 for chunker-over-crypt. A
13189 user in need can change name format to eg. *.rcc## and save 10 charac‐
13190 ters (provided at most 99 chunks per file).
13191 Note that a move implemented using the copy-and-delete method may incur
13193 double charging with some cloud storage providers.
13194 Chunker will not automatically rename existing chunks when you run
13196 rclone config on a live remote and change the chunk name format. Be‐
13197 ware that in result of this some files which have been treated as
13198 chunks before the change can pop up in directory listings as normal
13199 files and vice versa. The same warning holds for the chunk size. If
13200 you desperately need to change critical chunking setings, you should
13201 run data migration as described above.
13202 If wrapped remote is case insensitive, the chunker overlay will inherit
13204 that property (so you can’t have a file called “Hello.doc” and “hel‐
13205 lo.doc” in the same directory).
13206 Standard Options
13208 Here are the standard options specific to chunker (Transparently
13209 chunk/split large files).
13210 –chunker-remote
13212 Remote to chunk/unchunk. Normally should contain a `:' and a path, eg
13213 “myremote:path/to/dir”, “myremote:bucket” or maybe “myremote:” (not
13214 recommended).
13215 · Config: remote
13217 · Env Var: RCLONE_CHUNKER_REMOTE
13219 · Type: string
13221 · Default: ""
13223 –chunker-chunk-size
13225 Files larger than chunk size will be split in chunks.
13226 · Config: chunk_size
13228 · Env Var: RCLONE_CHUNKER_CHUNK_SIZE
13230 · Type: SizeSuffix
13232 · Default: 2G
13234 –chunker-hash-type
13236 Choose how chunker handles hash sums. All modes but “none” require
13237 metadata.
13238 · Config: hash_type
13240 · Env Var: RCLONE_CHUNKER_HASH_TYPE
13242 · Type: string
13244 · Default: “md5”
13246 · Examples:
13248 · “none”
13250 · Pass any hash supported by wrapped remote for non-chunked files,
13252 return nothing otherwise
13253 · “md5”
13255 · MD5 for composite files
13257 · “sha1”
13259 · SHA1 for composite files
13261 · “md5all”
13263 · MD5 for all files
13265 · “sha1all”
13267 · SHA1 for all files
13269 · “md5quick”
13271 · Copying a file to chunker will request MD5 from the source fall‐
13273 ing back to SHA1 if unsupported
13274 · “sha1quick”
13276 · Similar to “md5quick” but prefers SHA1 over MD5
13278 Advanced Options
13280 Here are the advanced options specific to chunker (Transparently
13281 chunk/split large files).
13282 –chunker-name-format
13284 String format of chunk file names. The two placeholders are: base file
13285 name (*) and chunk number (#...). There must be one and only one as‐
13286 terisk and one or more consecutive hash characters. If chunk number
13287 has less digits than the number of hashes, it is left-padded by zeros.
13288 If there are more digits in the number, they are left as is. Possible
13289 chunk files are ignored if their name does not match given format.
13290 · Config: name_format
13292 · Env Var: RCLONE_CHUNKER_NAME_FORMAT
13294 · Type: string
13296 · Default: "*.rclone_chunk.###"
13298 –chunker-start-from
13300 Minimum valid chunk number. Usually 0 or 1. By default chunk numbers
13301 start from 1.
13302 · Config: start_from
13304 · Env Var: RCLONE_CHUNKER_START_FROM
13306 · Type: int
13308 · Default: 1
13310 –chunker-meta-format
13312 Format of the metadata object or “none”. By default “simplejson”.
13313 Metadata is a small JSON file named after the composite file.
13314 · Config: meta_format
13316 · Env Var: RCLONE_CHUNKER_META_FORMAT
13318 · Type: string
13320 · Default: “simplejson”
13322 · Examples:
13324 · “none”
13326 · Do not use metadata files at all. Requires hash type “none”.
13328 · “simplejson”
13330 · Simple JSON supports hash sums and chunk validation.
13332 · It has the following fields: ver, size, nchunks, md5, sha1.
13334 –chunker-fail-hard
13336 Choose how chunker should handle files with missing or invalid chunks.
13337 · Config: fail_hard
13339 · Env Var: RCLONE_CHUNKER_FAIL_HARD
13341 · Type: bool
13343 · Default: false
13345 · Examples:
13347 · “true”
13349 · Report errors and abort current command.
13351 · “false”
13353 · Warn user, skip incomplete file and proceed.
13355 Citrix ShareFile
13357 Citrix ShareFile (https://sharefile.com) is a secure file sharing and
13358 transfer service aimed as business.
13359 The initial setup for Citrix ShareFile involves getting a token from
13361 Citrix ShareFile which you can in your browser. rclone config walks
13362 you through it.
13363 Here is an example of how to make a remote called remote. First run:
13365 rclone config
13367 This will guide you through an interactive setup process:
13369 No remotes found - make a new one
13371 n) New remote
13372 s) Set configuration password
13373 q) Quit config
13374 n/s/q> n
13375 name> remote
13376 Type of storage to configure.
13377 Enter a string value. Press Enter for the default ("").
13378 Choose a number from below, or type in your own value
13379 XX / Citrix Sharefile
13380 \ "sharefile"
13381 Storage> sharefile
13382 ** See help for sharefile backend at: https://rclone.org/sharefile/ **
13383 ID of the root folder
13385 Leave blank to access "Personal Folders". You can use one of the
13387 standard values here or any folder ID (long hex number ID).
13388 Enter a string value. Press Enter for the default ("").
13389 Choose a number from below, or type in your own value
13390 1 / Access the Personal Folders. (Default)
13391 \ ""
13392 2 / Access the Favorites folder.
13393 \ "favorites"
13394 3 / Access all the shared folders.
13395 \ "allshared"
13396 4 / Access all the individual connectors.
13397 \ "connectors"
13398 5 / Access the home, favorites, and shared folders as well as the connectors.
13399 \ "top"
13400 root_folder_id>
13401 Edit advanced config? (y/n)
13402 y) Yes
13403 n) No
13404 y/n> n
13405 Remote config
13406 Use auto config?
13407 * Say Y if not sure
13408 * Say N if you are working on a remote or headless machine
13409 y) Yes
13410 n) No
13411 y/n> y
13412 If your browser doesn't open automatically go to the following link: http://127.0.0.1:53682/auth?state=XXX
13413 Log in and authorize rclone for access
13414 Waiting for code...
13415 Got code
13416 --------------------
13417 [remote]
13418 type = sharefile
13419 endpoint = https://XXX.sharefile.com
13420 token = {"access_token":"XXX","token_type":"bearer","refresh_token":"XXX","expiry":"2019-09-30T19:41:45.878561877+01:00"}
13421 --------------------
13422 y) Yes this is OK
13423 e) Edit this remote
13424 d) Delete this remote
13425 y/e/d> y
13426 See the remote setup docs (https://rclone.org/remote_setup/) for how to
13428 set it up on a machine with no Internet browser available.
13429 Note that rclone runs a webserver on your local machine to collect the
13431 token as returned from Citrix ShareFile. This only runs from the mo‐
13432 ment it opens your browser to the moment you get back the verification
13433 code. This is on http://127.0.0.1:53682/ and this it may require you
13434 to unblock it temporarily if you are running a host firewall.
13435 Once configured you can then use rclone like this,
13437 List directories in top level of your ShareFile
13439 rclone lsd remote:
13441 List all the files in your ShareFile
13443 rclone ls remote:
13445 To copy a local directory to an ShareFile directory called backup
13447 rclone copy /home/source remote:backup
13449 Paths may be as deep as required, eg remote:directory/subdirectory.
13451 Modified time and hashes
13453 ShareFile allows modification times to be set on objects accurate to 1
13454 second. These will be used to detect whether objects need syncing or
13455 not.
13456 ShareFile supports MD5 type hashes, so you can use the --checksum flag.
13458 Transfers
13460 For files above 128MB rclone will use a chunked transfer. Rclone will
13461 upload up to --transfers chunks at the same time (shared among all the
13462 multipart uploads). Chunks are buffered in memory and are normally
13463 64MB so increasing --transfers will increase memory use.
13464 Limitations
13466 Note that ShareFile is case insensitive so you can’t have a file called
13467 “Hello.doc” and one called “hello.doc”.
13468 ShareFile only supports filenames up to 256 characters in length.
13470 Restricted filename characters
13472 In addition to the default restricted characters set (/overview/#re‐
13473 stricted-characters) the following characters are also replaced:
13474 Character Value Replacement
13476 ────────────────────────────────
13477 \ 0x5C ＼
13478 * 0x2A ＊
13479 < 0x3C ＜
13480 > 0x3E ＞
13481 ? 0x3F ？
13482 : 0x3A ：
13483 | 0x7C ｜
13484 " 0x22 ＂
13485 File names can also not start or end with the following characters.
13487 These only get replaced if they are first or last character in the
13488 name:
13489 Character Value Replacement
13491 ────────────────────────────────
13492 SP 0x20 ␠
13493 . 0x2E ．
13494 Invalid UTF-8 bytes will also be replaced (/overview/#invalid-utf8), as
13496 they can’t be used in JSON strings.
13497 Standard Options
13499 Here are the standard options specific to sharefile (Citrix Sharefile).
13500 –sharefile-root-folder-id
13502 ID of the root folder
13503 Leave blank to access “Personal Folders”. You can use one of the stan‐
13505 dard values here or any folder ID (long hex number ID).
13506 · Config: root_folder_id
13508 · Env Var: RCLONE_SHAREFILE_ROOT_FOLDER_ID
13510 · Type: string
13512 · Default: ""
13514 · Examples:
13516 · ""
13518 · Access the Personal Folders. (Default)
13520 · “favorites”
13522 · Access the Favorites folder.
13524 · “allshared”
13526 · Access all the shared folders.
13528 · “connectors”
13530 · Access all the individual connectors.
13532 · “top”
13534 · Access the home, favorites, and shared folders as well as the
13536 connectors.
13537 Advanced Options
13539 Here are the advanced options specific to sharefile (Citrix Sharefile).
13540 –sharefile-upload-cutoff
13542 Cutoff for switching to multipart upload.
13543 · Config: upload_cutoff
13545 · Env Var: RCLONE_SHAREFILE_UPLOAD_CUTOFF
13547 · Type: SizeSuffix
13549 · Default: 128M
13551 –sharefile-chunk-size
13553 Upload chunk size. Must a power of 2 >= 256k.
13554 Making this larger will improve performance, but note that each chunk
13556 is buffered in memory one per transfer.
13557 Reducing this will reduce memory usage but decrease performance.
13559 · Config: chunk_size
13561 · Env Var: RCLONE_SHAREFILE_CHUNK_SIZE
13563 · Type: SizeSuffix
13565 · Default: 64M
13567 –sharefile-endpoint
13569 Endpoint for API calls.
13570 This is usually auto discovered as part of the oauth process, but can
13572 be set manually to something like: https://XXX.sharefile.com
13573 · Config: endpoint
13575 · Env Var: RCLONE_SHAREFILE_ENDPOINT
13577 · Type: string
13579 · Default: ""
13581 –sharefile-encoding
13583 This sets the encoding for the backend.
13584 See: the encoding section in the overview (/overview/#encoding) for
13586 more info.
13587 · Config: encoding
13589 · Env Var: RCLONE_SHAREFILE_ENCODING
13591 · Type: MultiEncoder
13593 · Default: Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,Back‐
13595 Slash,Ctl,LeftSpace,LeftPeriod,RightSpace,RightPeriod,InvalidUtf8,Dot
13596 Crypt
13598 The crypt remote encrypts and decrypts another remote.
13599 To use it first set up the underlying remote following the config in‐
13601 structions for that remote. You can also use a local pathname instead
13602 of a remote which will encrypt and decrypt from that directory which
13603 might be useful for encrypting onto a USB stick for example.
13604 First check your chosen remote is working - we’ll call it remote:path
13606 in these docs. Note that anything inside remote:path will be encrypted
13607 and anything outside won’t. This means that if you are using a bucket
13608 based remote (eg S3, B2, swift) then you should probably put the bucket
13609 in the remote s3:bucket. If you just use s3: then rclone will make en‐
13610 crypted bucket names too (if using file name encryption) which may or
13611 may not be what you want.
13612 Now configure crypt using rclone config. We will call this one secret
13614 to differentiate it from the remote.
13615 No remotes found - make a new one
13617 n) New remote
13618 s) Set configuration password
13619 q) Quit config
13620 n/s/q> n
13621 name> secret
13622 Type of storage to configure.
13623 Choose a number from below, or type in your own value
13624 [snip]
13625 XX / Encrypt/Decrypt a remote
13626 \ "crypt"
13627 [snip]
13628 Storage> crypt
13629 Remote to encrypt/decrypt.
13630 Normally should contain a ':' and a path, eg "myremote:path/to/dir",
13631 "myremote:bucket" or maybe "myremote:" (not recommended).
13632 remote> remote:path
13633 How to encrypt the filenames.
13634 Choose a number from below, or type in your own value
13635 1 / Don't encrypt the file names. Adds a ".bin" extension only.
13636 \ "off"
13637 2 / Encrypt the filenames see the docs for the details.
13638 \ "standard"
13639 3 / Very simple filename obfuscation.
13640 \ "obfuscate"
13641 filename_encryption> 2
13642 Option to either encrypt directory names or leave them intact.
13643 Choose a number from below, or type in your own value
13644 1 / Encrypt directory names.
13645 \ "true"
13646 2 / Don't encrypt directory names, leave them intact.
13647 \ "false"
13648 filename_encryption> 1
13649 Password or pass phrase for encryption.
13650 y) Yes type in my own password
13651 g) Generate random password
13652 y/g> y
13653 Enter the password:
13654 password:
13655 Confirm the password:
13656 password:
13657 Password or pass phrase for salt. Optional but recommended.
13658 Should be different to the previous password.
13659 y) Yes type in my own password
13660 g) Generate random password
13661 n) No leave this optional password blank
13662 y/g/n> g
13663 Password strength in bits.
13664 64 is just about memorable
13665 128 is secure
13666 1024 is the maximum
13667 Bits> 128
13668 Your password is: JAsJvRcgR-_veXNfy_sGmQ
13669 Use this password?
13670 y) Yes
13671 n) No
13672 y/n> y
13673 Remote config
13674 --------------------
13675 [secret]
13676 remote = remote:path
13677 filename_encryption = standard
13678 password = *** ENCRYPTED ***
13679 password2 = *** ENCRYPTED ***
13680 --------------------
13681 y) Yes this is OK
13682 e) Edit this remote
13683 d) Delete this remote
13684 y/e/d> y
13685 Important The password is stored in the config file is lightly obscured
13687 so it isn’t immediately obvious what it is. It is in no way secure un‐
13688 less you use config file encryption.
13689 A long passphrase is recommended, or you can use a random one. Note
13691 that if you reconfigure rclone with the same passwords/passphrases
13692 elsewhere it will be compatible - all the secrets used are derived from
13693 those two passwords/passphrases.
13694 Note that rclone does not encrypt
13696 · file length - this can be calcuated within 16 bytes
13698 · modification time - used for syncing
13700 Specifying the remote
13702 In normal use, make sure the remote has a : in. If you specify the re‐
13703 mote without a : then rclone will use a local directory of that name.
13704 So if you use a remote of /path/to/secret/files then rclone will en‐
13705 crypt stuff to that directory. If you use a remote of name then rclone
13706 will put files in a directory called name in the current directory.
13707 If you specify the remote as remote:path/to/dir then rclone will store
13709 encrypted files in path/to/dir on the remote. If you are using file
13710 name encryption, then when you save files to secret:subdir/subfile this
13711 will store them in the unencrypted path path/to/dir but the subdir/sub‐
13712 path bit will be encrypted.
13713 Note that unless you want encrypted bucket names (which are difficult
13715 to manage because you won’t know what directory they represent in web
13716 interfaces etc), you should probably specify a bucket, eg remote:se‐
13717 cretbucket when using bucket based remotes such as S3, Swift, Hubic,
13718 B2, GCS.
13719 Example
13721 To test I made a little directory of files using “standard” file name
13722 encryption.
13723 plaintext/
13725 ├── file0.txt
13726 ├── file1.txt
13727 └── subdir
13728 ├── file2.txt
13729 ├── file3.txt
13730 └── subsubdir
13731 └── file4.txt
13732 Copy these to the remote and list them back
13734 $ rclone -q copy plaintext secret:
13736 $ rclone -q ls secret:
13737 7 file1.txt
13738 6 file0.txt
13739 8 subdir/file2.txt
13740 10 subdir/subsubdir/file4.txt
13741 9 subdir/file3.txt
13742 Now see what that looked like when encrypted
13744 $ rclone -q ls remote:path
13746 55 hagjclgavj2mbiqm6u6cnjjqcg
13747 54 v05749mltvv1tf4onltun46gls
13748 57 86vhrsv86mpbtd3a0akjuqslj8/dlj7fkq4kdq72emafg7a7s41uo
13749 58 86vhrsv86mpbtd3a0akjuqslj8/7uu829995du6o42n32otfhjqp4/b9pausrfansjth5ob3jkdqd4lc
13750 56 86vhrsv86mpbtd3a0akjuqslj8/8njh1sk437gttmep3p70g81aps
13751 Note that this retains the directory structure which means you can do
13753 this
13754 $ rclone -q ls secret:subdir
13756 8 file2.txt
13757 9 file3.txt
13758 10 subsubdir/file4.txt
13759 If don’t use file name encryption then the remote will look like this -
13761 note the .bin extensions added to prevent the cloud provider attempting
13762 to interpret the data.
13763 $ rclone -q ls remote:path
13765 54 file0.txt.bin
13766 57 subdir/file3.txt.bin
13767 56 subdir/file2.txt.bin
13768 58 subdir/subsubdir/file4.txt.bin
13769 55 file1.txt.bin
13770 File name encryption modes
13772 Here are some of the features of the file name encryption modes
13773 Off
13775 · doesn’t hide file names or directory structure
13777 · allows for longer file names (~246 characters)
13779 · can use sub paths and copy single files
13781 Standard
13783 · file names encrypted
13785 · file names can’t be as long (~143 characters)
13787 · can use sub paths and copy single files
13789 · directory structure visible
13791 · identical files names will have identical uploaded names
13793 · can use shortcuts to shorten the directory recursion
13795 Obfuscation
13797 This is a simple “rotate” of the filename, with each file having a rot
13799 distance based on the filename. We store the distance at the beginning
13800 of the filename. So a file called “hello” may become “53.jgnnq”
13801 This is not a strong encryption of filenames, but it may stop automated
13803 scanning tools from picking up on filename patterns. As such it’s an
13804 intermediate between “off” and “standard”. The advantage is that it
13805 allows for longer path segment names.
13806 There is a possibility with some unicode based filenames that the ob‐
13808 fuscation is weak and may map lower case characters to upper case
13809 equivalents. You can not rely on this for strong protection.
13810 · file names very lightly obfuscated
13812 · file names can be longer than standard encryption
13814 · can use sub paths and copy single files
13816 · directory structure visible
13818 · identical files names will have identical uploaded names
13820 Cloud storage systems have various limits on file name length and total
13822 path length which you are more likely to hit using “Standard” file name
13823 encryption. If you keep your file names to below 156 characters in
13824 length then you should be OK on all providers.
13825 There may be an even more secure file name encryption mode in the fu‐
13827 ture which will address the long file name problem.
13828 Directory name encryption
13830 Crypt offers the option of encrypting dir names or leaving them intact.
13831 There are two options:
13832 True
13834 Encrypts the whole file path including directory names Example:
13836 1/12/123.txt is encrypted to
13837 p0e52nreeaj0a5ea7s64m4j72s/l42g6771hnv3an9cgc8cr2n1ng/qgm4avr35m5loi1th53ato71v0
13838 False
13840 Only encrypts file names, skips directory names Example: 1/12/123.txt
13842 is encrypted to 1/12/qgm4avr35m5loi1th53ato71v0
13843 Modified time and hashes
13845 Crypt stores modification times using the underlying remote so support
13846 depends on that.
13847 Hashes are not stored for crypt. However the data integrity is pro‐
13849 tected by an extremely strong crypto authenticator.
13850 Note that you should use the rclone cryptcheck command to check the in‐
13852 tegrity of a crypted remote instead of rclone check which can’t check
13853 the checksums properly.
13854 Standard Options
13856 Here are the standard options specific to crypt (Encrypt/Decrypt a re‐
13857 mote).
13858 –crypt-remote
13860 Remote to encrypt/decrypt. Normally should contain a `:' and a path,
13861 eg “myremote:path/to/dir”, “myremote:bucket” or maybe “myremote:” (not
13862 recommended).
13863 · Config: remote
13865 · Env Var: RCLONE_CRYPT_REMOTE
13867 · Type: string
13869 · Default: ""
13871 –crypt-filename-encryption
13873 How to encrypt the filenames.
13874 · Config: filename_encryption
13876 · Env Var: RCLONE_CRYPT_FILENAME_ENCRYPTION
13878 · Type: string
13880 · Default: “standard”
13882 · Examples:
13884 · “standard”
13886 · Encrypt the filenames see the docs for the details.
13888 · “obfuscate”
13890 · Very simple filename obfuscation.
13892 · “off”
13894 · Don’t encrypt the file names. Adds a “.bin” extension only.
13896 –crypt-directory-name-encryption
13898 Option to either encrypt directory names or leave them intact.
13899 · Config: directory_name_encryption
13901 · Env Var: RCLONE_CRYPT_DIRECTORY_NAME_ENCRYPTION
13903 · Type: bool
13905 · Default: true
13907 · Examples:
13909 · “true”
13911 · Encrypt directory names.
13913 · “false”
13915 · Don’t encrypt directory names, leave them intact.
13917 –crypt-password
13919 Password or pass phrase for encryption.
13920 · Config: password
13922 · Env Var: RCLONE_CRYPT_PASSWORD
13924 · Type: string
13926 · Default: ""
13928 –crypt-password2
13930 Password or pass phrase for salt. Optional but recommended. Should be
13931 different to the previous password.
13932 · Config: password2
13934 · Env Var: RCLONE_CRYPT_PASSWORD2
13936 · Type: string
13938 · Default: ""
13940 Advanced Options
13942 Here are the advanced options specific to crypt (Encrypt/Decrypt a re‐
13943 mote).
13944 –crypt-show-mapping
13946 For all files listed show how the names encrypt.
13947 If this flag is set then for each file that the remote is asked to
13949 list, it will log (at level INFO) a line stating the decrypted file
13950 name and the encrypted file name.
13951 This is so you can work out which encrypted names are which decrypted
13953 names just in case you need to do something with the encrypted file
13954 names, or for debugging purposes.
13955 · Config: show_mapping
13957 · Env Var: RCLONE_CRYPT_SHOW_MAPPING
13959 · Type: bool
13961 · Default: false
13963 Backing up a crypted remote
13965 If you wish to backup a crypted remote, it it recommended that you use
13966 rclone sync on the encrypted files, and make sure the passwords are the
13967 same in the new encrypted remote.
13968 This will have the following advantages
13970 · rclone sync will check the checksums while copying
13972 · you can use rclone check between the encrypted remotes
13974 · you don’t decrypt and encrypt unnecessarily
13976 For example, let’s say you have your original remote at remote: with
13978 the encrypted version at eremote: with path remote:crypt. You would
13979 then set up the new remote remote2: and then the encrypted version ere‐
13980 mote2: with path remote2:crypt using the same passwords as eremote:.
13981 To sync the two remotes you would do
13983 rclone sync remote:crypt remote2:crypt
13985 And to check the integrity you would do
13987 rclone check remote:crypt remote2:crypt
13989 File formats
13991 File encryption
13992 Files are encrypted 1:1 source file to destination object. The file
13993 has a header and is divided into chunks.
13994 Header
13996 · 8 bytes magic string RCLONE\x00\x00
13997 · 24 bytes Nonce (IV)
13999 The initial nonce is generated from the operating systems crypto strong
14001 random number generator. The nonce is incremented for each chunk read
14002 making sure each nonce is unique for each block written. The chance of
14003 a nonce being re-used is minuscule. If you wrote an exabyte of data
14004 (10¹⁸ bytes) you would have a probability of approximately 2×10⁻³² of
14005 re-using a nonce.
14006 Chunk
14008 Each chunk will contain 64kB of data, except for the last one which may
14009 have less data. The data chunk is in standard NACL secretbox format.
14010 Secretbox uses XSalsa20 and Poly1305 to encrypt and authenticate mes‐
14011 sages.
14012 Each chunk contains:
14014 · 16 Bytes of Poly1305 authenticator
14016 · 1 - 65536 bytes XSalsa20 encrypted data
14018 64k chunk size was chosen as the best performing chunk size (the au‐
14020 thenticator takes too much time below this and the performance drops
14021 off due to cache effects above this). Note that these chunks are
14022 buffered in memory so they can’t be too big.
14023 This uses a 32 byte (256 bit key) key derived from the user password.
14025 Examples
14027 1 byte file will encrypt to
14028 · 32 bytes header
14030 · 17 bytes data chunk
14032 49 bytes total
14034 1MB (1048576 bytes) file will encrypt to
14036 · 32 bytes header
14038 · 16 chunks of 65568 bytes
14040 1049120 bytes total (a 0.05% overhead). This is the overhead for big
14042 files.
14043 Name encryption
14045 File names are encrypted segment by segment - the path is broken up in‐
14046 to / separated strings and these are encrypted individually.
14047 File segments are padded using using PKCS#7 to a multiple of 16 bytes
14049 before encryption.
14050 They are then encrypted with EME using AES with 256 bit key. EME
14052 (ECB-Mix-ECB) is a wide-block encryption mode presented in the 2003 pa‐
14053 per “A Parallelizable Enciphering Mode” by Halevi and Rogaway.
14054 This makes for deterministic encryption which is what we want - the
14056 same filename must encrypt to the same thing otherwise we can’t find it
14057 on the cloud storage system.
14058 This means that
14060 · filenames with the same name will encrypt the same
14062 · filenames which start the same won’t have a common prefix
14064 This uses a 32 byte key (256 bits) and a 16 byte (128 bits) IV both of
14066 which are derived from the user password.
14067 After encryption they are written out using a modified version of stan‐
14069 dard base32 encoding as described in RFC4648. The standard encoding is
14070 modified in two ways:
14071 · it becomes lower case (no-one likes upper case filenames!)
14073 · we strip the padding character =
14075 base32 is used rather than the more efficient base64 so rclone can be
14077 used on case insensitive remotes (eg Windows, Amazon Drive).
14078 Key derivation
14080 Rclone uses scrypt with parameters N=16384, r=8, p=1 with an optional
14081 user supplied salt (password2) to derive the 32+32+16 = 80 bytes of key
14082 material required. If the user doesn’t supply a salt then rclone uses
14083 an internal one.
14084 scrypt makes it impractical to mount a dictionary attack on rclone en‐
14086 crypted data. For full protection against this you should always use a
14087 salt.
14088 Dropbox
14090 Paths are specified as remote:path
14091 Dropbox paths may be as deep as required, eg remote:directory/subdirec‐
14093 tory.
14094 The initial setup for dropbox involves getting a token from Dropbox
14096 which you need to do in your browser. rclone config walks you through
14097 it.
14098 Here is an example of how to make a remote called remote. First run:
14100 rclone config
14102 This will guide you through an interactive setup process:
14104 n) New remote
14106 d) Delete remote
14107 q) Quit config
14108 e/n/d/q> n
14109 name> remote
14110 Type of storage to configure.
14111 Choose a number from below, or type in your own value
14112 [snip]
14113 XX / Dropbox
14114 \ "dropbox"
14115 [snip]
14116 Storage> dropbox
14117 Dropbox App Key - leave blank normally.
14118 app_key>
14119 Dropbox App Secret - leave blank normally.
14120 app_secret>
14121 Remote config
14122 Please visit:
14123 https://www.dropbox.com/1/oauth2/authorize?client_id=XXXXXXXXXXXXXXX&response_type=code
14124 Enter the code: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX_XXXXXXXXXX
14125 --------------------
14126 [remote]
14127 app_key =
14128 app_secret =
14129 token = XXXXXXXXXXXXXXXXXXXXXXXXXXXXX_XXXX_XXXXXXXXXXXXXXXXXXXXXXXXXXXXX
14130 --------------------
14131 y) Yes this is OK
14132 e) Edit this remote
14133 d) Delete this remote
14134 y/e/d> y
14135 You can then use it like this,
14137 List directories in top level of your dropbox
14139 rclone lsd remote:
14141 List all the files in your dropbox
14143 rclone ls remote:
14145 To copy a local directory to a dropbox directory called backup
14147 rclone copy /home/source remote:backup
14149 Dropbox for business
14151 Rclone supports Dropbox for business and Team Folders.
14152 When using Dropbox for business remote: and remote:path/to/file will
14154 refer to your personal folder.
14155 If you wish to see Team Folders you must use a leading / in the path,
14157 so rclone lsd remote:/ will refer to the root and show you all Team
14158 Folders and your User Folder.
14159 You can then use team folders like this remote:/TeamFolder and re‐
14161 mote:/TeamFolder/path/to/file.
14162 A leading / for a Dropbox personal account will do nothing, but it will
14164 take an extra HTTP transaction so it should be avoided.
14165 Modified time and Hashes
14167 Dropbox supports modified times, but the only way to set a modification
14168 time is to re-upload the file.
14169 This means that if you uploaded your data with an older version of
14171 rclone which didn’t support the v2 API and modified times, rclone will
14172 decide to upload all your old data to fix the modification times. If
14173 you don’t want this to happen use --size-only or --checksum flag to
14174 stop it.
14175 Dropbox supports its own hash type (https://www.dropbox.com/develop‐
14177 ers/reference/content-hash) which is checked for all transfers.
14178 Restricted filename characters
14180 Character Value Replacement
14181 ────────────────────────────────
14182 NUL 0x00 ␀
14183 / 0x2F ／
14184 DEL 0x7F ␡
14185 \ 0x5C ＼
14186 File names can also not end with the following characters. These only
14188 get replaced if they are last character in the name:
14189 Character Value Replacement
14191 ────────────────────────────────
14192 SP 0x20 ␠
14193 Invalid UTF-8 bytes will also be replaced (/overview/#invalid-utf8), as
14195 they can’t be used in JSON strings.
14196 Standard Options
14198 Here are the standard options specific to dropbox (Dropbox).
14199 –dropbox-client-id
14201 Dropbox App Client Id Leave blank normally.
14202 · Config: client_id
14204 · Env Var: RCLONE_DROPBOX_CLIENT_ID
14206 · Type: string
14208 · Default: ""
14210 –dropbox-client-secret
14212 Dropbox App Client Secret Leave blank normally.
14213 · Config: client_secret
14215 · Env Var: RCLONE_DROPBOX_CLIENT_SECRET
14217 · Type: string
14219 · Default: ""
14221 Advanced Options
14223 Here are the advanced options specific to dropbox (Dropbox).
14224 –dropbox-chunk-size
14226 Upload chunk size. (< 150M).
14227 Any files larger than this will be uploaded in chunks of this size.
14229 Note that chunks are buffered in memory (one at a time) so rclone can
14231 deal with retries. Setting this larger will increase the speed slight‐
14232 ly (at most 10% for 128MB in tests) at the cost of using more memory.
14233 It can be set smaller if you are tight on memory.
14234 · Config: chunk_size
14236 · Env Var: RCLONE_DROPBOX_CHUNK_SIZE
14238 · Type: SizeSuffix
14240 · Default: 48M
14242 –dropbox-impersonate
14244 Impersonate this user when using a business account.
14245 · Config: impersonate
14247 · Env Var: RCLONE_DROPBOX_IMPERSONATE
14249 · Type: string
14251 · Default: ""
14253 –dropbox-encoding
14255 This sets the encoding for the backend.
14256 See: the encoding section in the overview (/overview/#encoding) for
14258 more info.
14259 · Config: encoding
14261 · Env Var: RCLONE_DROPBOX_ENCODING
14263 · Type: MultiEncoder
14265 · Default: Slash,BackSlash,Del,RightSpace,InvalidUtf8,Dot
14267 Limitations
14269 Note that Dropbox is case insensitive so you can’t have a file called
14270 “Hello.doc” and one called “hello.doc”.
14271 There are some file names such as thumbs.db which Dropbox can’t store.
14273 There is a full list of them in the “Ignored Files” section of this
14274 document (https://www.dropbox.com/en/help/145). Rclone will issue an
14275 error message File name disallowed - not uploading if it attempts to
14276 upload one of those file names, but the sync won’t fail.
14277 If you have more than 10,000 files in a directory then rclone purge
14279 dropbox:dir will return the error Failed to purge: There are too many
14280 files involved in this operation. As a work-around do an rclone delete
14281 dropbox:dir followed by an rclone rmdir dropbox:dir.
14282 Get your own Dropbox App ID
14284 When you use rclone with Dropbox in its default configuration you are
14285 using rclone’s App ID. This is shared between all the rclone users.
14286 Here is how to create your own Dropbox App ID for rclone:
14288 1. Log into the Dropbox App console (https://www.dropbox.com/develop‐
14290 ers/apps/create) with your Dropbox Account (It need not to be the
14291 same account as the Dropbox you want to access)
14292 2. Choose an API => Usually this should be Dropbox API
14294 3. Choose the type of access you want to use => Full Dropbox or App
14296 Folder
14297 4. Name your App
14299 5. Click the button Create App
14301 6. Find the App key and App secret Use these values in rclone config to
14303 add a new remote or edit an existing remote.
14304 FTP
14306 FTP is the File Transfer Protocol. FTP support is provided using the
14307 github.com/jlaffaye/ftp (https://godoc.org/github.com/jlaffaye/ftp)
14308 package.
14309 Here is an example of making an FTP configuration. First run
14311 rclone config
14313 This will guide you through an interactive setup process. An FTP re‐
14315 mote only needs a host together with and a username and a password.
14316 With anonymous FTP server, you will need to use anonymous as username
14317 and your email address as the password.
14318 No remotes found - make a new one
14320 n) New remote
14321 r) Rename remote
14322 c) Copy remote
14323 s) Set configuration password
14324 q) Quit config
14325 n/r/c/s/q> n
14326 name> remote
14327 Type of storage to configure.
14328 Enter a string value. Press Enter for the default ("").
14329 Choose a number from below, or type in your own value
14330 [snip]
14331 XX / FTP Connection
14332 \ "ftp"
14333 [snip]
14334 Storage> ftp
14335 ** See help for ftp backend at: https://rclone.org/ftp/ **
14336 FTP host to connect to
14338 Enter a string value. Press Enter for the default ("").
14339 Choose a number from below, or type in your own value
14340 1 / Connect to ftp.example.com
14341 \ "ftp.example.com"
14342 host> ftp.example.com
14343 FTP username, leave blank for current username, ncw
14344 Enter a string value. Press Enter for the default ("").
14345 user>
14346 FTP port, leave blank to use default (21)
14347 Enter a string value. Press Enter for the default ("").
14348 port>
14349 FTP password
14350 y) Yes type in my own password
14351 g) Generate random password
14352 y/g> y
14353 Enter the password:
14354 password:
14355 Confirm the password:
14356 password:
14357 Use FTP over TLS (Implicit)
14358 Enter a boolean value (true or false). Press Enter for the default ("false").
14359 tls>
14360 Remote config
14361 --------------------
14362 [remote]
14363 type = ftp
14364 host = ftp.example.com
14365 pass = *** ENCRYPTED ***
14366 --------------------
14367 y) Yes this is OK
14368 e) Edit this remote
14369 d) Delete this remote
14370 y/e/d> y
14371 This remote is called remote and can now be used like this
14373 See all directories in the home directory
14375 rclone lsd remote:
14377 Make a new directory
14379 rclone mkdir remote:path/to/directory
14381 List the contents of a directory
14383 rclone ls remote:path/to/directory
14385 Sync /home/local/directory to the remote directory, deleting any excess
14387 files in the directory.
14388 rclone sync /home/local/directory remote:directory
14390 Modified time
14392 FTP does not support modified times. Any times you see on the server
14393 will be time of upload.
14394 Checksums
14396 FTP does not support any checksums.
14397 Restricted filename characters
14399 In addition to the default restricted characters set (/overview/#re‐
14400 stricted-characters) the following characters are also replaced:
14401 File names can also not end with the following characters. These only
14403 get replaced if they are last character in the name:
14404 Character Value Replacement
14406 ────────────────────────────────
14407 SP 0x20 ␠
14408 Note that not all FTP servers can have all characters in file names,
14410 for example:
14411 FTP Server Forbidden characters
14413 ──────────────────────────────────
14414 proftpd *
14415 pureftpd \ [ ]
14416 Implicit TLS
14418 FTP supports implicit FTP over TLS servers (FTPS). This has to be en‐
14419 abled in the config for the remote. The default FTPS port is 990 so
14420 the port will likely have to be explictly set in the config for the re‐
14421 mote.
14422 Standard Options
14424 Here are the standard options specific to ftp (FTP Connection).
14425 –ftp-host
14427 FTP host to connect to
14428 · Config: host
14430 · Env Var: RCLONE_FTP_HOST
14432 · Type: string
14434 · Default: ""
14436 · Examples:
14438 · “ftp.example.com”
14440 · Connect to ftp.example.com
14442 –ftp-user
14444 FTP username, leave blank for current username, $USER
14445 · Config: user
14447 · Env Var: RCLONE_FTP_USER
14449 · Type: string
14451 · Default: ""
14453 –ftp-port
14455 FTP port, leave blank to use default (21)
14456 · Config: port
14458 · Env Var: RCLONE_FTP_PORT
14460 · Type: string
14462 · Default: ""
14464 –ftp-pass
14466 FTP password
14467 · Config: pass
14469 · Env Var: RCLONE_FTP_PASS
14471 · Type: string
14473 · Default: ""
14475 –ftp-tls
14477 Use FTP over TLS (Implicit)
14478 · Config: tls
14480 · Env Var: RCLONE_FTP_TLS
14482 · Type: bool
14484 · Default: false
14486 Advanced Options
14488 Here are the advanced options specific to ftp (FTP Connection).
14489 –ftp-concurrency
14491 Maximum number of FTP simultaneous connections, 0 for unlimited
14492 · Config: concurrency
14494 · Env Var: RCLONE_FTP_CONCURRENCY
14496 · Type: int
14498 · Default: 0
14500 –ftp-no-check-certificate
14502 Do not verify the TLS certificate of the server
14503 · Config: no_check_certificate
14505 · Env Var: RCLONE_FTP_NO_CHECK_CERTIFICATE
14507 · Type: bool
14509 · Default: false
14511 –ftp-disable-epsv
14513 Disable using EPSV even if server advertises support
14514 · Config: disable_epsv
14516 · Env Var: RCLONE_FTP_DISABLE_EPSV
14518 · Type: bool
14520 · Default: false
14522 –ftp-encoding
14524 This sets the encoding for the backend.
14525 See: the encoding section in the overview (/overview/#encoding) for
14527 more info.
14528 · Config: encoding
14530 · Env Var: RCLONE_FTP_ENCODING
14532 · Type: MultiEncoder
14534 · Default: Slash,Del,Ctl,RightSpace,Dot
14536 Limitations
14538 Note that since FTP isn’t HTTP based the following flags don’t work
14539 with it: --dump-headers, --dump-bodies, --dump-auth
14540 Note that --timeout isn’t supported (but --contimeout is).
14542 Note that --bind isn’t supported.
14544 FTP could support server side move but doesn’t yet.
14546 Note that the ftp backend does not support the ftp_proxy environment
14548 variable yet.
14549 Note that while implicit FTP over TLS is supported, explicit FTP over
14551 TLS is not.
14552 Google Cloud Storage
14554 Paths are specified as remote:bucket (or remote: for the lsd command.)
14555 You may put subdirectories in too, eg remote:bucket/path/to/dir.
14556 The initial setup for google cloud storage involves getting a token
14558 from Google Cloud Storage which you need to do in your browser. rclone
14559 config walks you through it.
14560 Here is an example of how to make a remote called remote. First run:
14562 rclone config
14564 This will guide you through an interactive setup process:
14566 n) New remote
14568 d) Delete remote
14569 q) Quit config
14570 e/n/d/q> n
14571 name> remote
14572 Type of storage to configure.
14573 Choose a number from below, or type in your own value
14574 [snip]
14575 XX / Google Cloud Storage (this is not Google Drive)
14576 \ "google cloud storage"
14577 [snip]
14578 Storage> google cloud storage
14579 Google Application Client Id - leave blank normally.
14580 client_id>
14581 Google Application Client Secret - leave blank normally.
14582 client_secret>
14583 Project number optional - needed only for list/create/delete buckets - see your developer console.
14584 project_number> 12345678
14585 Service Account Credentials JSON file path - needed only if you want use SA instead of interactive login.
14586 service_account_file>
14587 Access Control List for new objects.
14588 Choose a number from below, or type in your own value
14589 1 / Object owner gets OWNER access, and all Authenticated Users get READER access.
14590 \ "authenticatedRead"
14591 2 / Object owner gets OWNER access, and project team owners get OWNER access.
14592 \ "bucketOwnerFullControl"
14593 3 / Object owner gets OWNER access, and project team owners get READER access.
14594 \ "bucketOwnerRead"
14595 4 / Object owner gets OWNER access [default if left blank].
14596 \ "private"
14597 5 / Object owner gets OWNER access, and project team members get access according to their roles.
14598 \ "projectPrivate"
14599 6 / Object owner gets OWNER access, and all Users get READER access.
14600 \ "publicRead"
14601 object_acl> 4
14602 Access Control List for new buckets.
14603 Choose a number from below, or type in your own value
14604 1 / Project team owners get OWNER access, and all Authenticated Users get READER access.
14605 \ "authenticatedRead"
14606 2 / Project team owners get OWNER access [default if left blank].
14607 \ "private"
14608 3 / Project team members get access according to their roles.
14609 \ "projectPrivate"
14610 4 / Project team owners get OWNER access, and all Users get READER access.
14611 \ "publicRead"
14612 5 / Project team owners get OWNER access, and all Users get WRITER access.
14613 \ "publicReadWrite"
14614 bucket_acl> 2
14615 Location for the newly created buckets.
14616 Choose a number from below, or type in your own value
14617 1 / Empty for default location (US).
14618 \ ""
14619 2 / Multi-regional location for Asia.
14620 \ "asia"
14621 3 / Multi-regional location for Europe.
14622 \ "eu"
14623 4 / Multi-regional location for United States.
14624 \ "us"
14625 5 / Taiwan.
14626 \ "asia-east1"
14627 6 / Tokyo.
14628 \ "asia-northeast1"
14629 7 / Singapore.
14630 \ "asia-southeast1"
14631 8 / Sydney.
14632 \ "australia-southeast1"
14633 9 / Belgium.
14634 \ "europe-west1"
14635 10 / London.
14636 \ "europe-west2"
14637 11 / Iowa.
14638 \ "us-central1"
14639 12 / South Carolina.
14640 \ "us-east1"
14641 13 / Northern Virginia.
14642 \ "us-east4"
14643 14 / Oregon.
14644 \ "us-west1"
14645 location> 12
14646 The storage class to use when storing objects in Google Cloud Storage.
14647 Choose a number from below, or type in your own value
14648 1 / Default
14649 \ ""
14650 2 / Multi-regional storage class
14651 \ "MULTI_REGIONAL"
14652 3 / Regional storage class
14653 \ "REGIONAL"
14654 4 / Nearline storage class
14655 \ "NEARLINE"
14656 5 / Coldline storage class
14657 \ "COLDLINE"
14658 6 / Durable reduced availability storage class
14659 \ "DURABLE_REDUCED_AVAILABILITY"
14660 storage_class> 5
14661 Remote config
14662 Use auto config?
14663 * Say Y if not sure
14664 * Say N if you are working on a remote or headless machine or Y didn't work
14665 y) Yes
14666 n) No
14667 y/n> y
14668 If your browser doesn't open automatically go to the following link: http://127.0.0.1:53682/auth
14669 Log in and authorize rclone for access
14670 Waiting for code...
14671 Got code
14672 --------------------
14673 [remote]
14674 type = google cloud storage
14675 client_id =
14676 client_secret =
14677 token = {"AccessToken":"xxxx.xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx","RefreshToken":"x/xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx_xxxxxxxxx","Expiry":"2014-07-17T20:49:14.929208288+01:00","Extra":null}
14678 project_number = 12345678
14679 object_acl = private
14680 bucket_acl = private
14681 --------------------
14682 y) Yes this is OK
14683 e) Edit this remote
14684 d) Delete this remote
14685 y/e/d> y
14686 Note that rclone runs a webserver on your local machine to collect the
14688 token as returned from Google if you use auto config mode. This only
14689 runs from the moment it opens your browser to the moment you get back
14690 the verification code. This is on http://127.0.0.1:53682/ and this it
14691 may require you to unblock it temporarily if you are running a host
14692 firewall, or use manual mode.
14693 This remote is called remote and can now be used like this
14695 See all the buckets in your project
14697 rclone lsd remote:
14699 Make a new bucket
14701 rclone mkdir remote:bucket
14703 List the contents of a bucket
14705 rclone ls remote:bucket
14707 Sync /home/local/directory to the remote bucket, deleting any excess
14709 files in the bucket.
14710 rclone sync /home/local/directory remote:bucket
14712 Service Account support
14714 You can set up rclone with Google Cloud Storage in an unattended mode,
14715 i.e. not tied to a specific end-user Google account. This is useful
14716 when you want to synchronise files onto machines that don’t have ac‐
14717 tively logged-in users, for example build machines.
14718 To get credentials for Google Cloud Platform IAM Service Accounts
14720 (https://cloud.google.com/iam/docs/service-accounts), please head to
14721 the Service Account (https://console.cloud.google.com/permissions/ser‐
14722 viceaccounts) section of the Google Developer Console. Service Ac‐
14723 counts behave just like normal User permissions in Google Cloud Storage
14724 ACLs (https://cloud.google.com/storage/docs/access-control), so you can
14725 limit their access (e.g. make them read only). After creating an ac‐
14726 count, a JSON file containing the Service Account’s credentials will be
14727 downloaded onto your machines. These credentials are what rclone will
14728 use for authentication.
14729 To use a Service Account instead of OAuth2 token flow, enter the path
14731 to your Service Account credentials at the service_account_file prompt
14732 and rclone won’t use the browser based authentication flow. If you’d
14733 rather stuff the contents of the credentials file into the rclone con‐
14734 fig file, you can set service_account_credentials with the actual con‐
14735 tents of the file instead, or set the equivalent environment variable.
14736 Application Default Credentials
14738 If no other source of credentials is provided, rclone will fall back to
14739 Application Default Credentials (https://cloud.google.com/video-intel‐
14740 ligence/docs/common/auth#authenticating_with_application_default_cre‐
14741 dentials) this is useful both when you already have configured authen‐
14742 tication for your developer account, or in production when running on a
14743 google compute host. Note that if running in docker, you may need to
14744 run additional commands on your google compute machine - see this page
14745 (https://cloud.google.com/container-registry/docs/advanced-authentica‐
14746 tion#gcloud_as_a_docker_credential_helper).
14747 Note that in the case application default credentials are used, there
14749 is no need to explicitly configure a project number.
14750 –fast-list
14752 This remote supports --fast-list which allows you to use fewer transac‐
14753 tions in exchange for more memory. See the rclone docs (/docs/#fast-
14754 list) for more details.
14755 Modified time
14757 Google google cloud storage stores md5sums natively and rclone stores
14758 modification times as metadata on the object, under the “mtime” key in
14759 RFC3339 format accurate to 1ns.
14760 Restricted filename characters
14762 Character Value Replacement
14763 ────────────────────────────────
14764 NUL 0x00 ␀
14765 LF 0x0A ␊
14766 CR 0x0D ␍
14767 / 0x2F ／
14768 Invalid UTF-8 bytes will also be replaced (/overview/#invalid-utf8), as
14770 they can’t be used in JSON strings.
14771 Standard Options
14773 Here are the standard options specific to google cloud storage (Google
14774 Cloud Storage (this is not Google Drive)).
14775 –gcs-client-id
14777 Google Application Client Id Leave blank normally.
14778 · Config: client_id
14780 · Env Var: RCLONE_GCS_CLIENT_ID
14782 · Type: string
14784 · Default: ""
14786 –gcs-client-secret
14788 Google Application Client Secret Leave blank normally.
14789 · Config: client_secret
14791 · Env Var: RCLONE_GCS_CLIENT_SECRET
14793 · Type: string
14795 · Default: ""
14797 –gcs-project-number
14799 Project number. Optional - needed only for list/create/delete buckets
14800 - see your developer console.
14801 · Config: project_number
14803 · Env Var: RCLONE_GCS_PROJECT_NUMBER
14805 · Type: string
14807 · Default: ""
14809 –gcs-service-account-file
14811 Service Account Credentials JSON file path Leave blank normally. Need‐
14812 ed only if you want use SA instead of interactive login.
14813 · Config: service_account_file
14815 · Env Var: RCLONE_GCS_SERVICE_ACCOUNT_FILE
14817 · Type: string
14819 · Default: ""
14821 –gcs-service-account-credentials
14823 Service Account Credentials JSON blob Leave blank normally. Needed on‐
14824 ly if you want use SA instead of interactive login.
14825 · Config: service_account_credentials
14827 · Env Var: RCLONE_GCS_SERVICE_ACCOUNT_CREDENTIALS
14829 · Type: string
14831 · Default: ""
14833 –gcs-object-acl
14835 Access Control List for new objects.
14836 · Config: object_acl
14838 · Env Var: RCLONE_GCS_OBJECT_ACL
14840 · Type: string
14842 · Default: ""
14844 · Examples:
14846 · “authenticatedRead”
14848 · Object owner gets OWNER access, and all Authenticated Users get
14850 READER access.
14851 · “bucketOwnerFullControl”
14853 · Object owner gets OWNER access, and project team owners get OWNER
14855 access.
14856 · “bucketOwnerRead”
14858 · Object owner gets OWNER access, and project team owners get READ‐
14860 ER access.
14861 · “private”
14863 · Object owner gets OWNER access [default if left blank].
14865 · “projectPrivate”
14867 · Object owner gets OWNER access, and project team members get ac‐
14869 cess according to their roles.
14870 · “publicRead”
14872 · Object owner gets OWNER access, and all Users get READER access.
14874 –gcs-bucket-acl
14876 Access Control List for new buckets.
14877 · Config: bucket_acl
14879 · Env Var: RCLONE_GCS_BUCKET_ACL
14881 · Type: string
14883 · Default: ""
14885 · Examples:
14887 · “authenticatedRead”
14889 · Project team owners get OWNER access, and all Authenticated Users
14891 get READER access.
14892 · “private”
14894 · Project team owners get OWNER access [default if left blank].
14896 · “projectPrivate”
14898 · Project team members get access according to their roles.
14900 · “publicRead”
14902 · Project team owners get OWNER access, and all Users get READER
14904 access.
14905 · “publicReadWrite”
14907 · Project team owners get OWNER access, and all Users get WRITER
14909 access.
14910 –gcs-bucket-policy-only
14912 Access checks should use bucket-level IAM policies.
14913 If you want to upload objects to a bucket with Bucket Policy Only set
14915 then you will need to set this.
14916 When it is set, rclone:
14918 · ignores ACLs set on buckets
14920 · ignores ACLs set on objects
14922 · creates buckets with Bucket Policy Only set
14924 Docs: https://cloud.google.com/storage/docs/bucket-policy-only
14926 · Config: bucket_policy_only
14928 · Env Var: RCLONE_GCS_BUCKET_POLICY_ONLY
14930 · Type: bool
14932 · Default: false
14934 –gcs-location
14936 Location for the newly created buckets.
14937 · Config: location
14939 · Env Var: RCLONE_GCS_LOCATION
14941 · Type: string
14943 · Default: ""
14945 · Examples:
14947 · ""
14949 · Empty for default location (US).
14951 · “asia”
14953 · Multi-regional location for Asia.
14955 · “eu”
14957 · Multi-regional location for Europe.
14959 · “us”
14961 · Multi-regional location for United States.
14963 · “asia-east1”
14965 · Taiwan.
14967 · “asia-east2”
14969 · Hong Kong.
14971 · “asia-northeast1”
14973 · Tokyo.
14975 · “asia-south1”
14977 · Mumbai.
14979 · “asia-southeast1”
14981 · Singapore.
14983 · “australia-southeast1”
14985 · Sydney.
14987 · “europe-north1”
14989 · Finland.
14991 · “europe-west1”
14993 · Belgium.
14995 · “europe-west2”
14997 · London.
14999 · “europe-west3”
15001 · Frankfurt.
15003 · “europe-west4”
15005 · Netherlands.
15007 · “us-central1”
15009 · Iowa.
15011 · “us-east1”
15013 · South Carolina.
15015 · “us-east4”
15017 · Northern Virginia.
15019 · “us-west1”
15021 · Oregon.
15023 · “us-west2”
15025 · California.
15027 –gcs-storage-class
15029 The storage class to use when storing objects in Google Cloud Storage.
15030 · Config: storage_class
15032 · Env Var: RCLONE_GCS_STORAGE_CLASS
15034 · Type: string
15036 · Default: ""
15038 · Examples:
15040 · ""
15042 · Default
15044 · “MULTI_REGIONAL”
15046 · Multi-regional storage class
15048 · “REGIONAL”
15050 · Regional storage class
15052 · “NEARLINE”
15054 · Nearline storage class
15056 · “COLDLINE”
15058 · Coldline storage class
15060 · “DURABLE_REDUCED_AVAILABILITY”
15062 · Durable reduced availability storage class
15064 Advanced Options
15066 Here are the advanced options specific to google cloud storage (Google
15067 Cloud Storage (this is not Google Drive)).
15068 –gcs-encoding
15070 This sets the encoding for the backend.
15071 See: the encoding section in the overview (/overview/#encoding) for
15073 more info.
15074 · Config: encoding
15076 · Env Var: RCLONE_GCS_ENCODING
15078 · Type: MultiEncoder
15080 · Default: Slash,CrLf,InvalidUtf8,Dot
15082 Google Drive
15084 Paths are specified as drive:path
15085 Drive paths may be as deep as required, eg drive:directory/subdirecto‐
15087 ry.
15088 The initial setup for drive involves getting a token from Google drive
15090 which you need to do in your browser. rclone config walks you through
15091 it.
15092 Here is an example of how to make a remote called remote. First run:
15094 rclone config
15096 This will guide you through an interactive setup process:
15098 No remotes found - make a new one
15100 n) New remote
15101 r) Rename remote
15102 c) Copy remote
15103 s) Set configuration password
15104 q) Quit config
15105 n/r/c/s/q> n
15106 name> remote
15107 Type of storage to configure.
15108 Choose a number from below, or type in your own value
15109 [snip]
15110 XX / Google Drive
15111 \ "drive"
15112 [snip]
15113 Storage> drive
15114 Google Application Client Id - leave blank normally.
15115 client_id>
15116 Google Application Client Secret - leave blank normally.
15117 client_secret>
15118 Scope that rclone should use when requesting access from drive.
15119 Choose a number from below, or type in your own value
15120 1 / Full access all files, excluding Application Data Folder.
15121 \ "drive"
15122 2 / Read-only access to file metadata and file contents.
15123 \ "drive.readonly"
15124 / Access to files created by rclone only.
15125 3 | These are visible in the drive website.
15126 | File authorization is revoked when the user deauthorizes the app.
15127 \ "drive.file"
15128 / Allows read and write access to the Application Data folder.
15129 4 | This is not visible in the drive website.
15130 \ "drive.appfolder"
15131 / Allows read-only access to file metadata but
15132 5 | does not allow any access to read or download file content.
15133 \ "drive.metadata.readonly"
15134 scope> 1
15135 ID of the root folder - leave blank normally. Fill in to access "Computers" folders. (see docs).
15136 root_folder_id>
15137 Service Account Credentials JSON file path - needed only if you want use SA instead of interactive login.
15138 service_account_file>
15139 Remote config
15140 Use auto config?
15141 * Say Y if not sure
15142 * Say N if you are working on a remote or headless machine or Y didn't work
15143 y) Yes
15144 n) No
15145 y/n> y
15146 If your browser doesn't open automatically go to the following link: http://127.0.0.1:53682/auth
15147 Log in and authorize rclone for access
15148 Waiting for code...
15149 Got code
15150 Configure this as a team drive?
15151 y) Yes
15152 n) No
15153 y/n> n
15154 --------------------
15155 [remote]
15156 client_id =
15157 client_secret =
15158 scope = drive
15159 root_folder_id =
15160 service_account_file =
15161 token = {"access_token":"XXX","token_type":"Bearer","refresh_token":"XXX","expiry":"2014-03-16T13:57:58.955387075Z"}
15162 --------------------
15163 y) Yes this is OK
15164 e) Edit this remote
15165 d) Delete this remote
15166 y/e/d> y
15167 Note that rclone runs a webserver on your local machine to collect the
15169 token as returned from Google if you use auto config mode. This only
15170 runs from the moment it opens your browser to the moment you get back
15171 the verification code. This is on http://127.0.0.1:53682/ and this it
15172 may require you to unblock it temporarily if you are running a host
15173 firewall, or use manual mode.
15174 You can then use it like this,
15176 List directories in top level of your drive
15178 rclone lsd remote:
15180 List all the files in your drive
15182 rclone ls remote:
15184 To copy a local directory to a drive directory called backup
15186 rclone copy /home/source remote:backup
15188 Scopes
15190 Rclone allows you to select which scope you would like for rclone to
15191 use. This changes what type of token is granted to rclone. The scopes
15192 are defined here. (https://developers.google.com/drive/v3/web/about-
15193 auth).
15194 The scope are
15196 drive
15198 This is the default scope and allows full access to all files, except
15199 for the Application Data Folder (see below).
15200 Choose this one if you aren’t sure.
15202 drive.readonly
15204 This allows read only access to all files. Files may be listed and
15205 downloaded but not uploaded, renamed or deleted.
15206 drive.file
15208 With this scope rclone can read/view/modify only those files and fold‐
15209 ers it creates.
15210 So if you uploaded files to drive via the web interface (or any other
15212 means) they will not be visible to rclone.
15213 This can be useful if you are using rclone to backup data and you want
15215 to be sure confidential data on your drive is not visible to rclone.
15216 Files created with this scope are visible in the web interface.
15218 drive.appfolder
15220 This gives rclone its own private area to store files. Rclone will not
15221 be able to see any other files on your drive and you won’t be able to
15222 see rclone’s files from the web interface either.
15223 drive.metadata.readonly
15225 This allows read only access to file names only. It does not allow
15226 rclone to download or upload data, or rename or delete files or direc‐
15227 tories.
15228 Root folder ID
15230 You can set the root_folder_id for rclone. This is the directory
15231 (identified by its Folder ID) that rclone considers to be the root of
15232 your drive.
15233 Normally you will leave this blank and rclone will determine the cor‐
15235 rect root to use itself.
15236 However you can set this to restrict rclone to a specific folder hier‐
15238 archy or to access data within the “Computers” tab on the drive web in‐
15239 terface (where files from Google’s Backup and Sync desktop program go).
15240 In order to do this you will have to find the Folder ID of the directo‐
15242 ry you wish rclone to display. This will be the last segment of the
15243 URL when you open the relevant folder in the drive web interface.
15244 So if the folder you want rclone to use has a URL which looks like
15246 https://drive.google.com/drive/fold‐
15247 ers/1XyfxxxxxxxxxxxxxxxxxxxxxxxxxKHCh in the browser, then you use
15248 1XyfxxxxxxxxxxxxxxxxxxxxxxxxxKHCh as the root_folder_id in the config.
15249 NB folders under the “Computers” tab seem to be read only (drive gives
15251 a 500 error) when using rclone.
15252 There doesn’t appear to be an API to discover the folder IDs of the
15254 “Computers” tab - please contact us if you know otherwise!
15255 Note also that rclone can’t access any data under the “Backups” tab on
15257 the google drive web interface yet.
15258 Service Account support
15260 You can set up rclone with Google Drive in an unattended mode, i.e. not
15261 tied to a specific end-user Google account. This is useful when you
15262 want to synchronise files onto machines that don’t have actively
15263 logged-in users, for example build machines.
15264 To use a Service Account instead of OAuth2 token flow, enter the path
15266 to your Service Account credentials at the service_account_file prompt
15267 during rclone config and rclone won’t use the browser based authentica‐
15268 tion flow. If you’d rather stuff the contents of the credentials file
15269 into the rclone config file, you can set service_account_credentials
15270 with the actual contents of the file instead, or set the equivalent en‐
15271 vironment variable.
15272 Use case - Google Apps/G-suite account and individual Drive
15274 Let’s say that you are the administrator of a Google Apps (old) or
15275 G-suite account. The goal is to store data on an individual’s Drive
15276 account, who IS a member of the domain. We’ll call the domain exam‐
15277 ple.com, and the user foo@example.com.
15278 There’s a few steps we need to go through to accomplish this:
15280 1. Create a service account for example.com
15282 · To create a service account and obtain its credentials, go to the
15283 Google Developer Console (https://console.developers.google.com).
15284 · You must have a project - create one if you don’t.
15286 · Then go to “IAM & admin” -> “Service Accounts”.
15288 · Use the “Create Credentials” button. Fill in “Service account name”
15290 with something that identifies your client. “Role” can be empty.
15291 · Tick “Furnish a new private key” - select “Key type JSON”.
15293 · Tick “Enable G Suite Domain-wide Delegation”. This option makes “im‐
15295 personation” possible, as documented here: Delegating domain-wide au‐
15296 thority to the service account (https://developers.google.com/identi‐
15297 ty/protocols/OAuth2ServiceAccount#delegatingauthority)
15298 · These credentials are what rclone will use for authentication. If
15300 you ever need to remove access, press the “Delete service account
15301 key” button.
15302 2. Allowing API access to example.com Google Drive
15304 · Go to example.com’s admin console
15305 · Go into “Security” (or use the search bar)
15307 · Select “Show more” and then “Advanced settings”
15309 · Select “Manage API client access” in the “Authentication” section
15311 · In the “Client Name” field enter the service account’s “Client ID” -
15313 this can be found in the Developer Console under “IAM & Admin” ->
15314 “Service Accounts”, then “View Client ID” for the newly created ser‐
15315 vice account. It is a ~21 character numerical string.
15316 · In the next field, “One or More API Scopes”, enter
15318 https://www.googleapis.com/auth/drive to grant access to Google Drive
15319 specifically.
15320 3. Configure rclone, assuming a new install
15322 rclone config
15323 n/s/q> n # New
15325 name>gdrive # Gdrive is an example name
15326 Storage> # Select the number shown for Google Drive
15327 client_id> # Can be left blank
15328 client_secret> # Can be left blank
15329 scope> # Select your scope, 1 for example
15330 root_folder_id> # Can be left blank
15331 service_account_file> /home/foo/myJSONfile.json # This is where the JSON file goes!
15332 y/n> # Auto config, y
15333 4. Verify that it’s working
15335 · rclone -v --drive-impersonate foo@example.com lsf gdrive:backup
15336 · The arguments do:
15338 · -v - verbose logging
15340 · --drive-impersonate foo@example.com - this is what does the magic,
15342 pretending to be user foo.
15343 · lsf - list files in a parsing friendly way
15345 · gdrive:backup - use the remote called gdrive, work in the folder
15347 named backup.
15348 Team drives
15350 If you want to configure the remote to point to a Google Team Drive
15351 then answer y to the question Configure this as a team drive?.
15352 This will fetch the list of Team Drives from google and allow you to
15354 configure which one you want to use. You can also type in a team drive
15355 ID if you prefer.
15356 For example:
15358 Configure this as a team drive?
15360 y) Yes
15361 n) No
15362 y/n> y
15363 Fetching team drive list...
15364 Choose a number from below, or type in your own value
15365 1 / Rclone Test
15366 \ "xxxxxxxxxxxxxxxxxxxx"
15367 2 / Rclone Test 2
15368 \ "yyyyyyyyyyyyyyyyyyyy"
15369 3 / Rclone Test 3
15370 \ "zzzzzzzzzzzzzzzzzzzz"
15371 Enter a Team Drive ID> 1
15372 --------------------
15373 [remote]
15374 client_id =
15375 client_secret =
15376 token = {"AccessToken":"xxxx.x.xxxxx_xxxxxxxxxxx_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx","RefreshToken":"1/xxxxxxxxxxxxxxxx_xxxxxxxxxxxxxxxxxxxxxxxxxx","Expiry":"2014-03-16T13:57:58.955387075Z","Extra":null}
15377 team_drive = xxxxxxxxxxxxxxxxxxxx
15378 --------------------
15379 y) Yes this is OK
15380 e) Edit this remote
15381 d) Delete this remote
15382 y/e/d> y
15383 –fast-list
15385 This remote supports --fast-list which allows you to use fewer transac‐
15386 tions in exchange for more memory. See the rclone docs (/docs/#fast-
15387 list) for more details.
15388 It does this by combining multiple list calls into a single API re‐
15390 quest.
15391 This works by combining many '%s' in parents filters into one expres‐
15393 sion. To list the contents of directories a, b and c, the following
15394 requests will be send by the regular List function:
15395 trashed=false and 'a' in parents
15397 trashed=false and 'b' in parents
15398 trashed=false and 'c' in parents
15399 These can now be combined into a single request:
15401 trashed=false and ('a' in parents or 'b' in parents or 'c' in parents)
15403 The implementation of ListR will put up to 50 parents filters into one
15405 request. It will use the --checkers value to specify the number of re‐
15406 quests to run in parallel.
15407 In tests, these batch requests were up to 20x faster than the regular
15409 method. Running the following command against different sized folders
15410 gives:
15411 rclone lsjson -vv -R --checkers=6 gdrive:folder
15413 small folder (220 directories, 700 files):
15415 · without --fast-list: 38s
15417 · with --fast-list: 10s
15419 large folder (10600 directories, 39000 files):
15421 · without --fast-list: 22:05 min
15423 · with --fast-list: 58s
15425 Modified time
15427 Google drive stores modification times accurate to 1 ms.
15428 Restricted filename characters
15430 Only Invalid UTF-8 bytes will be replaced (/overview/#invalid-utf8), as
15431 they can’t be used in JSON strings.
15432 In contrast to other backends, / can also be used in names and . or ..
15434 are valid names.
15435 Revisions
15437 Google drive stores revisions of files. When you upload a change to an
15438 existing file to google drive using rclone it will create a new revi‐
15439 sion of that file.
15440 Revisions follow the standard google policy which at time of writing
15442 was
15443 · They are deleted after 30 days or 100 revisions (whatever comes
15445 first).
15446 · They do not count towards a user storage quota.
15448 Deleting files
15450 By default rclone will send all files to the trash when deleting files.
15451 If deleting them permanently is required then use the
15452 --drive-use-trash=false flag, or set the equivalent environment vari‐
15453 able.
15454 Emptying trash
15456 If you wish to empty your trash you can use the rclone cleanup remote:
15457 command which will permanently delete all your trashed files. This
15458 command does not take any path arguments.
15459 Note that Google Drive takes some time (minutes to days) to empty the
15461 trash even though the command returns within a few seconds. No output
15462 is echoed, so there will be no confirmation even using -v or -vv.
15463 Quota information
15465 To view your current quota you can use the rclone about remote: command
15466 which will display your usage limit (quota), the usage in Google Drive,
15467 the size of all files in the Trash and the space used by other Google
15468 services such as Gmail. This command does not take any path arguments.
15469 Import/Export of google documents
15471 Google documents can be exported from and uploaded to Google Drive.
15472 When rclone downloads a Google doc it chooses a format to download de‐
15474 pending upon the --drive-export-formats setting. By default the export
15475 formats are docx,xlsx,pptx,svg which are a sensible default for an ed‐
15476 itable document.
15477 When choosing a format, rclone runs down the list provided in order and
15479 chooses the first file format the doc can be exported as from the list.
15480 If the file can’t be exported to a format on the formats list, then
15481 rclone will choose a format from the default list.
15482 If you prefer an archive copy then you might use --drive-export-formats
15484 pdf, or if you prefer openoffice/libreoffice formats you might use
15485 --drive-export-formats ods,odt,odp.
15486 Note that rclone adds the extension to the google doc, so if it is
15488 called My Spreadsheet on google docs, it will be exported as My Spread‐
15489 sheet.xlsx or My Spreadsheet.pdf etc.
15490 When importing files into Google Drive, rclone will convert all files
15492 with an extension in --drive-import-formats to their associated docu‐
15493 ment type. rclone will not convert any files by default, since the
15494 conversion is lossy process.
15495 The conversion must result in a file with the same extension when the
15497 --drive-export-formats rules are applied to the uploaded document.
15498 Here are some examples for allowed and prohibited conversions.
15500 export-for‐ import-for‐ Upload Ext Document Ext Allowed
15502 mats mats
15503 ────────────────────────────────────────────────────────────────
15504 odt odt odt odt Yes
15505 odt docx,odt odt odt Yes
15506 docx docx docx Yes
15507 odt odt docx No
15508 odt,docx docx,odt docx odt No
15509 docx,odt docx,odt docx docx Yes
15510 docx,odt docx,odt odt docx No
15511 This limitation can be disabled by specifying --drive-allow-im‐
15513 port-name-change. When using this flag, rclone can convert multiple
15514 files types resulting in the same document type at once, eg with
15515 --drive-import-formats docx,odt,txt, all files having these extension
15516 would result in a document represented as a docx file. This brings the
15517 additional risk of overwriting a document, if multiple files have the
15518 same stem. Many rclone operations will not handle this name change in
15519 any way. They assume an equal name when copying files and might copy
15520 the file again or delete them when the name changes.
15521 Here are the possible export extensions with their corresponding mime
15523 types. Most of these can also be used for importing, but there more
15524 that are not listed here. Some of these additional ones might only be
15525 available when the operating system provides the correct MIME type en‐
15526 tries.
15527 This list can be changed by Google Drive at any time and might not rep‐
15529 resent the currently available conversions.
15530 Extension Mime Type Description
15532 ─────────────────────────────────────────────────────────────────────────────────────
15533 csv text/csv Standard CSV format for
15534 Spreadsheets
15535 docx application/vnd.openxml‐ Microsoft Office Document
15536 formats-officedocu‐
15537 ment.wordprocess‐
15538 ingml.document
15539 epub application/epub+zip E-book format
15540 html text/html An HTML Document
15541 jpg image/jpeg A JPEG Image File
15542 json applica‐ JSON Text Format
15543 tion/vnd.google-apps.script+json
15544 odp application/vnd.oasis.opendocu‐ Openoffice Presentation
15545 ment.presentation
15546 ods application/vnd.oasis.opendocu‐ Openoffice Spreadsheet
15547 ment.spreadsheet
15548 ods application/x-vnd.oasis.opendoc‐ Openoffice Spreadsheet
15549 ument.spreadsheet
15550 odt application/vnd.oasis.opendocu‐ Openoffice Document
15551 ment.text
15552 pdf application/pdf Adobe PDF Format
15553 png image/png PNG Image Format
15554 pptx application/vnd.openxmlfor‐ Microsoft Office Power‐
15555 mats-officedocument.presenta‐ point
15556 tionml.presentation
15557 rtf application/rtf Rich Text Format
15558 svg image/svg+xml Scalable Vector Graphics
15559 Format
15560 tsv text/tab-separated-values Standard TSV format for
15561 spreadsheets
15562 txt text/plain Plain Text
15563 xlsx application/vnd.openxmlfor‐ Microsoft Office Spread‐
15564 mats-officedocument.spread‐ sheet
15565 sheetml.sheet
15566 zip application/zip A ZIP file of HTML, Images
15567 CSS
15568 Google documents can also be exported as link files. These files will
15570 open a browser window for the Google Docs website of that document when
15571 opened. The link file extension has to be specified as a --drive-ex‐
15572 port-formats parameter. They will match all available Google Docu‐
15573 ments.
15574 Extension Description OS Support
15576 ─────────────────────────────────────────────────
15577 desktop freedesktop.org Linux
15578 specified desktop
15579 entry
15580 link.html An HTML Document All
15581 with a redirect
15582 url INI style link file macOS, Windows
15583 webloc macOS specific XML macOS
15584 format
15585 Standard Options
15587 Here are the standard options specific to drive (Google Drive).
15588 –drive-client-id
15590 Google Application Client Id Setting your own is recommended. See
15591 https://rclone.org/drive/#making-your-own-client-id for how to create
15592 your own. If you leave this blank, it will use an internal key which
15593 is low performance.
15594 · Config: client_id
15596 · Env Var: RCLONE_DRIVE_CLIENT_ID
15598 · Type: string
15600 · Default: ""
15602 –drive-client-secret
15604 Google Application Client Secret Setting your own is recommended.
15605 · Config: client_secret
15607 · Env Var: RCLONE_DRIVE_CLIENT_SECRET
15609 · Type: string
15611 · Default: ""
15613 –drive-scope
15615 Scope that rclone should use when requesting access from drive.
15616 · Config: scope
15618 · Env Var: RCLONE_DRIVE_SCOPE
15620 · Type: string
15622 · Default: ""
15624 · Examples:
15626 · “drive”
15628 · Full access all files, excluding Application Data Folder.
15630 · “drive.readonly”
15632 · Read-only access to file metadata and file contents.
15634 · “drive.file”
15636 · Access to files created by rclone only.
15638 · These are visible in the drive website.
15640 · File authorization is revoked when the user deauthorizes the app.
15642 · “drive.appfolder”
15644 · Allows read and write access to the Application Data folder.
15646 · This is not visible in the drive website.
15648 · “drive.metadata.readonly”
15650 · Allows read-only access to file metadata but
15652 · does not allow any access to read or download file content.
15654 –drive-root-folder-id
15656 ID of the root folder Leave blank normally.
15657 Fill in to access “Computers” folders (see docs), or for rclone to use
15659 a non root folder as its starting point.
15660 Note that if this is blank, the first time rclone runs it will fill it
15662 in with the ID of the root folder.
15663 · Config: root_folder_id
15665 · Env Var: RCLONE_DRIVE_ROOT_FOLDER_ID
15667 · Type: string
15669 · Default: ""
15671 –drive-service-account-file
15673 Service Account Credentials JSON file path Leave blank normally. Need‐
15674 ed only if you want use SA instead of interactive login.
15675 · Config: service_account_file
15677 · Env Var: RCLONE_DRIVE_SERVICE_ACCOUNT_FILE
15679 · Type: string
15681 · Default: ""
15683 Advanced Options
15685 Here are the advanced options specific to drive (Google Drive).
15686 –drive-service-account-credentials
15688 Service Account Credentials JSON blob Leave blank normally. Needed on‐
15689 ly if you want use SA instead of interactive login.
15690 · Config: service_account_credentials
15692 · Env Var: RCLONE_DRIVE_SERVICE_ACCOUNT_CREDENTIALS
15694 · Type: string
15696 · Default: ""
15698 –drive-team-drive
15700 ID of the Team Drive
15701 · Config: team_drive
15703 · Env Var: RCLONE_DRIVE_TEAM_DRIVE
15705 · Type: string
15707 · Default: ""
15709 –drive-auth-owner-only
15711 Only consider files owned by the authenticated user.
15712 · Config: auth_owner_only
15714 · Env Var: RCLONE_DRIVE_AUTH_OWNER_ONLY
15716 · Type: bool
15718 · Default: false
15720 –drive-use-trash
15722 Send files to the trash instead of deleting permanently. Defaults to
15723 true, namely sending files to the trash. Use --drive-use-trash=false
15724 to delete files permanently instead.
15725 · Config: use_trash
15727 · Env Var: RCLONE_DRIVE_USE_TRASH
15729 · Type: bool
15731 · Default: true
15733 –drive-skip-gdocs
15735 Skip google documents in all listings. If given, gdocs practically be‐
15736 come invisible to rclone.
15737 · Config: skip_gdocs
15739 · Env Var: RCLONE_DRIVE_SKIP_GDOCS
15741 · Type: bool
15743 · Default: false
15745 –drive-skip-checksum-gphotos
15747 Skip MD5 checksum on Google photos and videos only.
15748 Use this if you get checksum errors when transferring Google photos or
15750 videos.
15751 Setting this flag will cause Google photos and videos to return a blank
15753 MD5 checksum.
15754 Google photos are identifed by being in the “photos” space.
15756 Corrupted checksums are caused by Google modifying the image/video but
15758 not updating the checksum.
15759 · Config: skip_checksum_gphotos
15761 · Env Var: RCLONE_DRIVE_SKIP_CHECKSUM_GPHOTOS
15763 · Type: bool
15765 · Default: false
15767 –drive-shared-with-me
15769 Only show files that are shared with me.
15770 Instructs rclone to operate on your “Shared with me” folder (where
15772 Google Drive lets you access the files and folders others have shared
15773 with you).
15774 This works both with the “list” (lsd, lsl, etc) and the “copy” commands
15776 (copy, sync, etc), and with all other commands too.
15777 · Config: shared_with_me
15779 · Env Var: RCLONE_DRIVE_SHARED_WITH_ME
15781 · Type: bool
15783 · Default: false
15785 –drive-trashed-only
15787 Only show files that are in the trash. This will show trashed files in
15788 their original directory structure.
15789 · Config: trashed_only
15791 · Env Var: RCLONE_DRIVE_TRASHED_ONLY
15793 · Type: bool
15795 · Default: false
15797 –drive-formats
15799 Deprecated: see export_formats
15800 · Config: formats
15802 · Env Var: RCLONE_DRIVE_FORMATS
15804 · Type: string
15806 · Default: ""
15808 –drive-export-formats
15810 Comma separated list of preferred formats for downloading Google docs.
15811 · Config: export_formats
15813 · Env Var: RCLONE_DRIVE_EXPORT_FORMATS
15815 · Type: string
15817 · Default: “docx,xlsx,pptx,svg”
15819 –drive-import-formats
15821 Comma separated list of preferred formats for uploading Google docs.
15822 · Config: import_formats
15824 · Env Var: RCLONE_DRIVE_IMPORT_FORMATS
15826 · Type: string
15828 · Default: ""
15830 –drive-allow-import-name-change
15832 Allow the filetype to change when uploading Google docs (e.g. file.doc
15833 to file.docx). This will confuse sync and reupload every time.
15834 · Config: allow_import_name_change
15836 · Env Var: RCLONE_DRIVE_ALLOW_IMPORT_NAME_CHANGE
15838 · Type: bool
15840 · Default: false
15842 –drive-use-created-date
15844 Use file created date instead of modified date.,
15845 Useful when downloading data and you want the creation date used in
15847 place of the last modified date.
15848 WARNING: This flag may have some unexpected consequences.
15850 When uploading to your drive all files will be overwritten unless they
15852 haven’t been modified since their creation. And the inverse will occur
15853 while downloading. This side effect can be avoided by using the
15854 “–checksum” flag.
15855 This feature was implemented to retain photos capture date as recorded
15857 by google photos. You will first need to check the “Create a Google
15858 Photos folder” option in your google drive settings. You can then copy
15859 or move the photos locally and use the date the image was taken (creat‐
15860 ed) set as the modification date.
15861 · Config: use_created_date
15863 · Env Var: RCLONE_DRIVE_USE_CREATED_DATE
15865 · Type: bool
15867 · Default: false
15869 –drive-use-shared-date
15871 Use date file was shared instead of modified date.
15872 Note that, as with “–drive-use-created-date”, this flag may have unex‐
15874 pected consequences when uploading/downloading files.
15875 If both this flag and “–drive-use-created-date” are set, the created
15877 date is used.
15878 · Config: use_shared_date
15880 · Env Var: RCLONE_DRIVE_USE_SHARED_DATE
15882 · Type: bool
15884 · Default: false
15886 –drive-list-chunk
15888 Size of listing chunk 100-1000. 0 to disable.
15889 · Config: list_chunk
15891 · Env Var: RCLONE_DRIVE_LIST_CHUNK
15893 · Type: int
15895 · Default: 1000
15897 –drive-impersonate
15899 Impersonate this user when using a service account.
15900 · Config: impersonate
15902 · Env Var: RCLONE_DRIVE_IMPERSONATE
15904 · Type: string
15906 · Default: ""
15908 –drive-alternate-export
15910 Use alternate export URLs for google documents export.,
15911 If this option is set this instructs rclone to use an alternate set of
15913 export URLs for drive documents. Users have reported that the official
15914 export URLs can’t export large documents, whereas these unofficial ones
15915 can.
15916 See rclone issue #2243 (https://github.com/rclone/rclone/issues/2243)
15918 for background, this google drive issue (https://issuetrack‐
15919 er.google.com/issues/36761333) and this helpful post (https://www.lab‐
15920 nol.org/internet/direct-links-for-google-drive/28356/).
15921 · Config: alternate_export
15923 · Env Var: RCLONE_DRIVE_ALTERNATE_EXPORT
15925 · Type: bool
15927 · Default: false
15929 –drive-upload-cutoff
15931 Cutoff for switching to chunked upload
15932 · Config: upload_cutoff
15934 · Env Var: RCLONE_DRIVE_UPLOAD_CUTOFF
15936 · Type: SizeSuffix
15938 · Default: 8M
15940 –drive-chunk-size
15942 Upload chunk size. Must a power of 2 >= 256k.
15943 Making this larger will improve performance, but note that each chunk
15945 is buffered in memory one per transfer.
15946 Reducing this will reduce memory usage but decrease performance.
15948 · Config: chunk_size
15950 · Env Var: RCLONE_DRIVE_CHUNK_SIZE
15952 · Type: SizeSuffix
15954 · Default: 8M
15956 –drive-acknowledge-abuse
15958 Set to allow files which return cannotDownloadAbusiveFile to be down‐
15959 loaded.
15960 If downloading a file returns the error “This file has been identified
15962 as malware or spam and cannot be downloaded” with the error code “can‐
15963 notDownloadAbusiveFile” then supply this flag to rclone to indicate you
15964 acknowledge the risks of downloading the file and rclone will download
15965 it anyway.
15966 · Config: acknowledge_abuse
15968 · Env Var: RCLONE_DRIVE_ACKNOWLEDGE_ABUSE
15970 · Type: bool
15972 · Default: false
15974 –drive-keep-revision-forever
15976 Keep new head revision of each file forever.
15977 · Config: keep_revision_forever
15979 · Env Var: RCLONE_DRIVE_KEEP_REVISION_FOREVER
15981 · Type: bool
15983 · Default: false
15985 –drive-size-as-quota
15987 Show sizes as storage quota usage, not actual size.
15988 Show the size of a file as the the storage quota used. This is the
15990 current version plus any older versions that have been set to keep for‐
15991 ever.
15992 WARNING: This flag may have some unexpected consequences.
15994 It is not recommended to set this flag in your config - the recommended
15996 usage is using the flag form –drive-size-as-quota when doing rclone
15997 ls/lsl/lsf/lsjson/etc only.
15998 If you do use this flag for syncing (not recommended) then you will
16000 need to use –ignore size also.
16001 · Config: size_as_quota
16003 · Env Var: RCLONE_DRIVE_SIZE_AS_QUOTA
16005 · Type: bool
16007 · Default: false
16009 –drive-v2-download-min-size
16011 If Object’s are greater, use drive v2 API to download.
16012 · Config: v2_download_min_size
16014 · Env Var: RCLONE_DRIVE_V2_DOWNLOAD_MIN_SIZE
16016 · Type: SizeSuffix
16018 · Default: off
16020 –drive-pacer-min-sleep
16022 Minimum time to sleep between API calls.
16023 · Config: pacer_min_sleep
16025 · Env Var: RCLONE_DRIVE_PACER_MIN_SLEEP
16027 · Type: Duration
16029 · Default: 100ms
16031 –drive-pacer-burst
16033 Number of API calls to allow without sleeping.
16034 · Config: pacer_burst
16036 · Env Var: RCLONE_DRIVE_PACER_BURST
16038 · Type: int
16040 · Default: 100
16042 –drive-server-side-across-configs
16044 Allow server side operations (eg copy) to work across different drive
16045 configs.
16046 This can be useful if you wish to do a server side copy between two
16048 different Google drives. Note that this isn’t enabled by default be‐
16049 cause it isn’t easy to tell if it will work between any two configura‐
16050 tions.
16051 · Config: server_side_across_configs
16053 · Env Var: RCLONE_DRIVE_SERVER_SIDE_ACROSS_CONFIGS
16055 · Type: bool
16057 · Default: false
16059 –drive-disable-http2
16061 Disable drive using http2
16062 There is currently an unsolved issue with the google drive backend and
16064 HTTP/2. HTTP/2 is therefore disabled by default for the drive backend
16065 but can be re-enabled here. When the issue is solved this flag will be
16066 removed.
16067 See: https://github.com/rclone/rclone/issues/3631
16069 · Config: disable_http2
16071 · Env Var: RCLONE_DRIVE_DISABLE_HTTP2
16073 · Type: bool
16075 · Default: true
16077 –drive-stop-on-upload-limit
16079 Make upload limit errors be fatal
16080 At the time of writing it is only possible to upload 750GB of data to
16082 Google Drive a day (this is an undocumented limit). When this limit is
16083 reached Google Drive produces a slightly different error message. When
16084 this flag is set it causes these errors to be fatal. These will stop
16085 the in-progress sync.
16086 Note that this detection is relying on error message strings which
16088 Google don’t document so it may break in the future.
16089 See: https://github.com/rclone/rclone/issues/3857
16091 · Config: stop_on_upload_limit
16093 · Env Var: RCLONE_DRIVE_STOP_ON_UPLOAD_LIMIT
16095 · Type: bool
16097 · Default: false
16099 –drive-encoding
16101 This sets the encoding for the backend.
16102 See: the encoding section in the overview (/overview/#encoding) for
16104 more info.
16105 · Config: encoding
16107 · Env Var: RCLONE_DRIVE_ENCODING
16109 · Type: MultiEncoder
16111 · Default: InvalidUtf8
16113 Limitations
16115 Drive has quite a lot of rate limiting. This causes rclone to be lim‐
16116 ited to transferring about 2 files per second only. Individual files
16117 may be transferred much faster at 100s of MBytes/s but lots of small
16118 files can take a long time.
16119 Server side copies are also subject to a separate rate limit. If you
16121 see User rate limit exceeded errors, wait at least 24 hours and retry.
16122 You can disable server side copies with --disable copy to download and
16123 upload the files if you prefer.
16124 Limitations of Google Docs
16126 Google docs will appear as size -1 in rclone ls and as size 0 in any‐
16127 thing which uses the VFS layer, eg rclone mount, rclone serve.
16128 This is because rclone can’t find out the size of the Google docs with‐
16130 out downloading them.
16131 Google docs will transfer correctly with rclone sync, rclone copy etc
16133 as rclone knows to ignore the size when doing the transfer.
16134 However an unfortunate consequence of this is that you may not be able
16136 to download Google docs using rclone mount. If it doesn’t work you
16137 will get a 0 sized file. If you try again the doc may gain its correct
16138 size and be downloadable. Whther it will work on not depends on the
16139 application accessing the mount and the OS you are running - experiment
16140 to find out if it does work for you!
16141 Duplicated files
16143 Sometimes, for no reason I’ve been able to track down, drive will du‐
16144 plicate a file that rclone uploads. Drive unlike all the other remotes
16145 can have duplicated files.
16146 Duplicated files cause problems with the syncing and you will see mes‐
16148 sages in the log about duplicates.
16149 Use rclone dedupe to fix duplicated files.
16151 Note that this isn’t just a problem with rclone, even Google Photos on
16153 Android duplicates files on drive sometimes.
16154 Rclone appears to be re-copying files it shouldn’t
16156 The most likely cause of this is the duplicated file issue above - run
16157 rclone dedupe and check your logs for duplicate object or directory
16158 messages.
16159 This can also be caused by a delay/caching on google drive’s end when
16161 comparing directory listings. Specifically with team drives used in
16162 combination with –fast-list. Files that were uploaded recently may not
16163 appear on the directory list sent to rclone when using –fast-list.
16164 Waiting a moderate period of time between attempts (estimated to be ap‐
16166 proximately 1 hour) and/or not using –fast-list both seem to be effec‐
16167 tive in preventing the problem.
16168 Making your own client_id
16170 When you use rclone with Google drive in its default configuration you
16171 are using rclone’s client_id. This is shared between all the rclone
16172 users. There is a global rate limit on the number of queries per sec‐
16173 ond that each client_id can do set by Google. rclone already has a
16174 high quota and I will continue to make sure it is high enough by con‐
16175 tacting Google.
16176 It is strongly recommended to use your own client ID as the default
16178 rclone ID is heavily used. If you have multiple services running, it
16179 is recommended to use an API key for each service. The default Google
16180 quota is 10 transactions per second so it is recommended to stay under
16181 that number as if you use more than that, it will cause rclone to rate
16182 limit and make things slower.
16183 Here is how to create your own Google Drive client ID for rclone:
16185 1. Log into the Google API Console (https://console.develop‐
16187 ers.google.com/) with your Google account. It doesn’t matter what
16188 Google account you use. (It need not be the same account as the
16189 Google Drive you want to access)
16190 2. Select a project or create a new project.
16192 3. Under “ENABLE APIS AND SERVICES” search for “Drive”, and enable the
16194 “Google Drive API”.
16195 4. Click “Credentials” in the left-side panel (not “Create creden‐
16197 tials”, which opens the wizard), then “Create credentials”, then
16198 “OAuth client ID”. It will prompt you to set the OAuth consent
16199 screen product name, if you haven’t set one already.
16200 5. Choose an application type of “other”, and click “Create”. (the de‐
16202 fault name is fine)
16203 6. It will show you a client ID and client secret. Use these values in
16205 rclone config to add a new remote or edit an existing remote.
16206 (Thanks to @balazer on github for these instructions.)
16208 Google Photos
16210 The rclone backend for Google Photos (https://www.google.com/pho‐
16211 tos/about/) is a specialized backend for transferring photos and videos
16212 to and from Google Photos.
16213 NB The Google Photos API which rclone uses has quite a few limitations,
16215 so please read the limitations section carefully to make sure it is
16216 suitable for your use.
16217 Configuring Google Photos
16219 The initial setup for google cloud storage involves getting a token
16220 from Google Photos which you need to do in your browser. rclone config
16221 walks you through it.
16222 Here is an example of how to make a remote called remote. First run:
16224 rclone config
16226 This will guide you through an interactive setup process:
16228 No remotes found - make a new one
16230 n) New remote
16231 s) Set configuration password
16232 q) Quit config
16233 n/s/q> n
16234 name> remote
16235 Type of storage to configure.
16236 Enter a string value. Press Enter for the default ("").
16237 Choose a number from below, or type in your own value
16238 [snip]
16239 XX / Google Photos
16240 \ "google photos"
16241 [snip]
16242 Storage> google photos
16243 ** See help for google photos backend at: https://rclone.org/googlephotos/ **
16244 Google Application Client Id
16246 Leave blank normally.
16247 Enter a string value. Press Enter for the default ("").
16248 client_id>
16249 Google Application Client Secret
16250 Leave blank normally.
16251 Enter a string value. Press Enter for the default ("").
16252 client_secret>
16253 Set to make the Google Photos backend read only.
16254 If you choose read only then rclone will only request read only access
16256 to your photos, otherwise rclone will request full access.
16257 Enter a boolean value (true or false). Press Enter for the default ("false").
16258 read_only>
16259 Edit advanced config? (y/n)
16260 y) Yes
16261 n) No
16262 y/n> n
16263 Remote config
16264 Use auto config?
16265 * Say Y if not sure
16266 * Say N if you are working on a remote or headless machine
16267 y) Yes
16268 n) No
16269 y/n> y
16270 If your browser doesn't open automatically go to the following link: http://127.0.0.1:53682/auth
16271 Log in and authorize rclone for access
16272 Waiting for code...
16273 Got code
16274 *** IMPORTANT: All media items uploaded to Google Photos with rclone
16276 *** are stored in full resolution at original quality. These uploads
16277 *** will count towards storage in your Google Account.
16278 --------------------
16280 [remote]
16281 type = google photos
16282 token = {"access_token":"XXX","token_type":"Bearer","refresh_token":"XXX","expiry":"2019-06-28T17:38:04.644930156+01:00"}
16283 --------------------
16284 y) Yes this is OK
16285 e) Edit this remote
16286 d) Delete this remote
16287 y/e/d> y
16288 Note that rclone runs a webserver on your local machine to collect the
16290 token as returned from Google if you use auto config mode. This only
16291 runs from the moment it opens your browser to the moment you get back
16292 the verification code. This is on http://127.0.0.1:53682/ and this may
16293 require you to unblock it temporarily if you are running a host fire‐
16294 wall, or use manual mode.
16295 This remote is called remote and can now be used like this
16297 See all the albums in your photos
16299 rclone lsd remote:album
16301 Make a new album
16303 rclone mkdir remote:album/newAlbum
16305 List the contents of an album
16307 rclone ls remote:album/newAlbum
16309 Sync /home/local/images to the Google Photos, removing any excess files
16311 in the album.
16312 rclone sync /home/local/image remote:album/newAlbum
16314 Layout
16316 As Google Photos is not a general purpose cloud storage system the
16317 backend is laid out to help you navigate it.
16318 The directories under media show different ways of categorizing the me‐
16320 dia. Each file will appear multiple times. So if you want to make a
16321 backup of your google photos you might choose to backup remote:me‐
16322 dia/by-month. (NB remote:media/by-day is rather slow at the moment so
16323 avoid for syncing.)
16324 Note that all your photos and videos will appear somewhere under media,
16326 but they may not appear under album unless you’ve put them into albums.
16327 /
16329 - upload
16330 - file1.jpg
16331 - file2.jpg
16332 - ...
16333 - media
16334 - all
16335 - file1.jpg
16336 - file2.jpg
16337 - ...
16338 - by-year
16339 - 2000
16340 - file1.jpg
16341 - ...
16342 - 2001
16343 - file2.jpg
16344 - ...
16345 - ...
16346 - by-month
16347 - 2000
16348 - 2000-01
16349 - file1.jpg
16350 - ...
16351 - 2000-02
16352 - file2.jpg
16353 - ...
16354 - ...
16355 - by-day
16356 - 2000
16357 - 2000-01-01
16358 - file1.jpg
16359 - ...
16360 - 2000-01-02
16361 - file2.jpg
16362 - ...
16363 - ...
16364 - album
16365 - album name
16366 - album name/sub
16367 - shared-album
16368 - album name
16369 - album name/sub
16370 There are two writable parts of the tree, the upload directory and sub
16372 directories of the the album directory.
16373 The upload directory is for uploading files you don’t want to put into
16375 albums. This will be empty to start with and will contain the files
16376 you’ve uploaded for one rclone session only, becoming empty again when
16377 you restart rclone. The use case for this would be if you have a load
16378 of files you just want to once off dump into Google Photos. For re‐
16379 peated syncing, uploading to album will work better.
16380 Directories within the album directory are also writeable and you may
16382 create new directories (albums) under album. If you copy files with a
16383 directory hierarchy in there then rclone will create albums with the /
16384 character in them. For example if you do
16385 rclone copy /path/to/images remote:album/images
16387 and the images directory contains
16389 images
16391 - file1.jpg
16392 dir
16393 file2.jpg
16394 dir2
16395 dir3
16396 file3.jpg
16397 Then rclone will create the following albums with the following files
16399 in
16400 · images
16402 · file1.jpg
16404 · images/dir
16406 · file2.jpg
16408 · images/dir2/dir3
16410 · file3.jpg
16412 This means that you can use the album path pretty much like a normal
16414 filesystem and it is a good target for repeated syncing.
16415 The shared-album directory shows albums shared with you or by you.
16417 This is similar to the Sharing tab in the Google Photos web interface.
16418 Limitations
16420 Only images and videos can be uploaded. If you attempt to upload non
16421 videos or images or formats that Google Photos doesn’t understand,
16422 rclone will upload the file, then Google Photos will give an error when
16423 it is put turned into a media item.
16424 Note that all media items uploaded to Google Photos through the API are
16426 stored in full resolution at “original quality” and will count towards
16427 your storage quota in your Google Account. The API does not offer a
16428 way to upload in “high quality” mode..
16429 Downloading Images
16431 When Images are downloaded this strips EXIF location (according to the
16432 docs and my tests). This is a limitation of the Google Photos API and
16433 is covered by bug #112096115 (https://issuetracker.google.com/is‐
16434 sues/112096115).
16435 The current google API does not allow photos to be downloaded at origi‐
16437 nal resolution. This is very important if you are, for example, relying
16438 on “Google Photos” as a backup of your photos. You will not be able to
16439 use rclone to redownload original images. You could use `google take‐
16440 out' to recover the original photos as a last resort
16441 Downloading Videos
16443 When videos are downloaded they are downloaded in a really compressed
16444 version of the video compared to downloading it via the Google Photos
16445 web interface. This is covered by bug #113672044 (https://issuetrack‐
16446 er.google.com/issues/113672044).
16447 Duplicates
16449 If a file name is duplicated in a directory then rclone will add the
16450 file ID into its name. So two files called file.jpg would then appear
16451 as file {123456}.jpg and file {ABCDEF}.jpg (the actual IDs are a lot
16452 longer alas!).
16453 If you upload the same image (with the same binary data) twice then
16455 Google Photos will deduplicate it. However it will retain the filename
16456 from the first upload which may confuse rclone. For example if you up‐
16457 loaded an image to upload then uploaded the same image to album/my_al‐
16458 bum the filename of the image in album/my_album will be what it was up‐
16459 loaded with initially, not what you uploaded it with to album. In
16460 practise this shouldn’t cause too many problems.
16461 Modified time
16463 The date shown of media in Google Photos is the creation date as deter‐
16464 mined by the EXIF information, or the upload date if that is not known.
16465 This is not changeable by rclone and is not the modification date of
16467 the media on local disk. This means that rclone cannot use the dates
16468 from Google Photos for syncing purposes.
16469 Size
16471 The Google Photos API does not return the size of media. This means
16472 that when syncing to Google Photos, rclone can only do a file existence
16473 check.
16474 It is possible to read the size of the media, but this needs an extra
16476 HTTP HEAD request per media item so is very slow and uses up a lot of
16477 transactions. This can be enabled with the --gphotos-read-size option
16478 or the read_size = true config parameter.
16479 If you want to use the backend with rclone mount you may need to enable
16481 this flag (depending on your OS and application using the photos) oth‐
16482 erwise you may not be able to read media off the mount. You’ll need to
16483 experiment to see if it works for you without the flag.
16484 Albums
16486 Rclone can only upload files to albums it created. This is a limita‐
16487 tion of the Google Photos API (https://developers.google.com/photos/li‐
16488 brary/guides/manage-albums).
16489 Rclone can remove files it uploaded from albums it created only.
16491 Deleting files
16493 Rclone can remove files from albums it created, but note that the
16494 Google Photos API does not allow media to be deleted permanently so
16495 this media will still remain. See bug #109759781 (https://issuetrack‐
16496 er.google.com/issues/109759781).
16497 Rclone cannot delete files anywhere except under album.
16499 Deleting albums
16501 The Google Photos API does not support deleting albums - see bug
16502 #135714733 (https://issuetracker.google.com/issues/135714733).
16503 Standard Options
16505 Here are the standard options specific to google photos (Google Pho‐
16506 tos).
16507 –gphotos-client-id
16509 Google Application Client Id Leave blank normally.
16510 · Config: client_id
16512 · Env Var: RCLONE_GPHOTOS_CLIENT_ID
16514 · Type: string
16516 · Default: ""
16518 –gphotos-client-secret
16520 Google Application Client Secret Leave blank normally.
16521 · Config: client_secret
16523 · Env Var: RCLONE_GPHOTOS_CLIENT_SECRET
16525 · Type: string
16527 · Default: ""
16529 –gphotos-read-only
16531 Set to make the Google Photos backend read only.
16532 If you choose read only then rclone will only request read only access
16534 to your photos, otherwise rclone will request full access.
16535 · Config: read_only
16537 · Env Var: RCLONE_GPHOTOS_READ_ONLY
16539 · Type: bool
16541 · Default: false
16543 Advanced Options
16545 Here are the advanced options specific to google photos (Google Pho‐
16546 tos).
16547 –gphotos-read-size
16549 Set to read the size of media items.
16550 Normally rclone does not read the size of media items since this takes
16552 another transaction. This isn’t necessary for syncing. However rclone
16553 mount needs to know the size of files in advance of reading them, so
16554 setting this flag when using rclone mount is recommended if you want to
16555 read the media.
16556 · Config: read_size
16558 · Env Var: RCLONE_GPHOTOS_READ_SIZE
16560 · Type: bool
16562 · Default: false
16564 HTTP
16566 The HTTP remote is a read only remote for reading files of a webserver.
16567 The webserver should provide file listings which rclone will read and
16568 turn into a remote. This has been tested with common webservers such
16569 as Apache/Nginx/Caddy and will likely work with file listings from most
16570 web servers. (If it doesn’t then please file an issue, or send a pull
16571 request!)
16572 Paths are specified as remote: or remote:path/to/dir.
16574 Here is an example of how to make a remote called remote. First run:
16576 rclone config
16578 This will guide you through an interactive setup process:
16580 No remotes found - make a new one
16582 n) New remote
16583 s) Set configuration password
16584 q) Quit config
16585 n/s/q> n
16586 name> remote
16587 Type of storage to configure.
16588 Choose a number from below, or type in your own value
16589 [snip]
16590 XX / http Connection
16591 \ "http"
16592 [snip]
16593 Storage> http
16594 URL of http host to connect to
16595 Choose a number from below, or type in your own value
16596 1 / Connect to example.com
16597 \ "https://example.com"
16598 url> https://beta.rclone.org
16599 Remote config
16600 --------------------
16601 [remote]
16602 url = https://beta.rclone.org
16603 --------------------
16604 y) Yes this is OK
16605 e) Edit this remote
16606 d) Delete this remote
16607 y/e/d> y
16608 Current remotes:
16609 Name Type
16611 ==== ====
16612 remote http
16613 e) Edit existing remote
16615 n) New remote
16616 d) Delete remote
16617 r) Rename remote
16618 c) Copy remote
16619 s) Set configuration password
16620 q) Quit config
16621 e/n/d/r/c/s/q> q
16622 This remote is called remote and can now be used like this
16624 See all the top level directories
16626 rclone lsd remote:
16628 List the contents of a directory
16630 rclone ls remote:directory
16632 Sync the remote directory to /home/local/directory, deleting any excess
16634 files.
16635 rclone sync remote:directory /home/local/directory
16637 Read only
16639 This remote is read only - you can’t upload files to an HTTP server.
16640 Modified time
16642 Most HTTP servers store time accurate to 1 second.
16643 Checksum
16645 No checksums are stored.
16646 Usage without a config file
16648 Since the http remote only has one config parameter it is easy to use
16649 without a config file:
16650 rclone lsd --http-url https://beta.rclone.org :http:
16652 Standard Options
16654 Here are the standard options specific to http (http Connection).
16655 –http-url
16657 URL of http host to connect to
16658 · Config: url
16660 · Env Var: RCLONE_HTTP_URL
16662 · Type: string
16664 · Default: ""
16666 · Examples:
16668 · “https://example.com”
16670 · Connect to example.com
16672 · “https://user:pass@example.com”
16674 · Connect to example.com using a username and password
16676 Advanced Options
16678 Here are the advanced options specific to http (http Connection).
16679 –http-headers
16681 Set HTTP headers for all transactions
16682 Use this to set additional HTTP headers for all transactions
16684 The input format is comma separated list of key,value pairs. Standard
16686 CSV encoding (https://godoc.org/encoding/csv) may be used.
16687 For example to set a Cookie use `Cookie,name=value', or `“Cook‐
16689 ie”,“name=value”'.
16690 You can set multiple headers, eg `“Cookie”,“name=value”,“Authoriza‐
16692 tion”,“xxx”'.
16693 · Config: headers
16695 · Env Var: RCLONE_HTTP_HEADERS
16697 · Type: CommaSepList
16699 · Default:
16701 –http-no-slash
16703 Set this if the site doesn’t end directories with /
16704 Use this if your target website does not use / on the end of directo‐
16706 ries.
16707 A / on the end of a path is how rclone normally tells the difference
16709 between files and directories. If this flag is set, then rclone will
16710 treat all files with Content-Type: text/html as directories and read
16711 URLs from them rather than downloading them.
16712 Note that this may cause rclone to confuse genuine HTML files with di‐
16714 rectories.
16715 · Config: no_slash
16717 · Env Var: RCLONE_HTTP_NO_SLASH
16719 · Type: bool
16721 · Default: false
16723 –http-no-head
16725 Don’t use HEAD requests to find file sizes in dir listing
16726 If your site is being very slow to load then you can try this option.
16728 Normally rclone does a HEAD request for each potential file in a direc‐
16729 tory listing to:
16730 · find its size
16732 · check it really exists
16734 · check to see if it is a directory
16736 If you set this option, rclone will not do the HEAD request. This will
16738 mean
16739 · directory listings are much quicker
16741 · rclone won’t have the times or sizes of any files
16743 · some files that don’t exist may be in the listing
16745 · Config: no_head
16747 · Env Var: RCLONE_HTTP_NO_HEAD
16749 · Type: bool
16751 · Default: false
16753 Hubic
16755 Paths are specified as remote:path
16756 Paths are specified as remote:container (or remote: for the lsd com‐
16758 mand.) You may put subdirectories in too, eg remote:contain‐
16759 er/path/to/dir.
16760 The initial setup for Hubic involves getting a token from Hubic which
16762 you need to do in your browser. rclone config walks you through it.
16763 Here is an example of how to make a remote called remote. First run:
16765 rclone config
16767 This will guide you through an interactive setup process:
16769 n) New remote
16771 s) Set configuration password
16772 n/s> n
16773 name> remote
16774 Type of storage to configure.
16775 Choose a number from below, or type in your own value
16776 [snip]
16777 XX / Hubic
16778 \ "hubic"
16779 [snip]
16780 Storage> hubic
16781 Hubic Client Id - leave blank normally.
16782 client_id>
16783 Hubic Client Secret - leave blank normally.
16784 client_secret>
16785 Remote config
16786 Use auto config?
16787 * Say Y if not sure
16788 * Say N if you are working on a remote or headless machine
16789 y) Yes
16790 n) No
16791 y/n> y
16792 If your browser doesn't open automatically go to the following link: http://127.0.0.1:53682/auth
16793 Log in and authorize rclone for access
16794 Waiting for code...
16795 Got code
16796 --------------------
16797 [remote]
16798 client_id =
16799 client_secret =
16800 token = {"access_token":"XXXXXX"}
16801 --------------------
16802 y) Yes this is OK
16803 e) Edit this remote
16804 d) Delete this remote
16805 y/e/d> y
16806 See the remote setup docs (https://rclone.org/remote_setup/) for how to
16808 set it up on a machine with no Internet browser available.
16809 Note that rclone runs a webserver on your local machine to collect the
16811 token as returned from Hubic. This only runs from the moment it opens
16812 your browser to the moment you get back the verification code. This is
16813 on http://127.0.0.1:53682/ and this it may require you to unblock it
16814 temporarily if you are running a host firewall.
16815 Once configured you can then use rclone like this,
16817 List containers in the top level of your Hubic
16819 rclone lsd remote:
16821 List all the files in your Hubic
16823 rclone ls remote:
16825 To copy a local directory to an Hubic directory called backup
16827 rclone copy /home/source remote:backup
16829 If you want the directory to be visible in the official Hubic browser,
16831 you need to copy your files to the default directory
16832 rclone copy /home/source remote:default/backup
16834 –fast-list
16836 This remote supports --fast-list which allows you to use fewer transac‐
16837 tions in exchange for more memory. See the rclone docs (/docs/#fast-
16838 list) for more details.
16839 Modified time
16841 The modified time is stored as metadata on the object as X-Ob‐
16842 ject-Meta-Mtime as floating point since the epoch accurate to 1 ns.
16843 This is a de facto standard (used in the official python-swiftclient
16845 amongst others) for storing the modification time for an object.
16846 Note that Hubic wraps the Swift backend, so most of the properties of
16848 are the same.
16849 Standard Options
16851 Here are the standard options specific to hubic (Hubic).
16852 –hubic-client-id
16854 Hubic Client Id Leave blank normally.
16855 · Config: client_id
16857 · Env Var: RCLONE_HUBIC_CLIENT_ID
16859 · Type: string
16861 · Default: ""
16863 –hubic-client-secret
16865 Hubic Client Secret Leave blank normally.
16866 · Config: client_secret
16868 · Env Var: RCLONE_HUBIC_CLIENT_SECRET
16870 · Type: string
16872 · Default: ""
16874 Advanced Options
16876 Here are the advanced options specific to hubic (Hubic).
16877 –hubic-chunk-size
16879 Above this size files will be chunked into a _segments container.
16880 Above this size files will be chunked into a _segments container. The
16882 default for this is 5GB which is its maximum value.
16883 · Config: chunk_size
16885 · Env Var: RCLONE_HUBIC_CHUNK_SIZE
16887 · Type: SizeSuffix
16889 · Default: 5G
16891 –hubic-no-chunk
16893 Don’t chunk files during streaming upload.
16894 When doing streaming uploads (eg using rcat or mount) setting this flag
16896 will cause the swift backend to not upload chunked files.
16897 This will limit the maximum upload size to 5GB. However non chunked
16899 files are easier to deal with and have an MD5SUM.
16900 Rclone will still chunk files bigger than chunk_size when doing normal
16902 copy operations.
16903 · Config: no_chunk
16905 · Env Var: RCLONE_HUBIC_NO_CHUNK
16907 · Type: bool
16909 · Default: false
16911 –hubic-encoding
16913 This sets the encoding for the backend.
16914 See: the encoding section in the overview (/overview/#encoding) for
16916 more info.
16917 · Config: encoding
16919 · Env Var: RCLONE_HUBIC_ENCODING
16921 · Type: MultiEncoder
16923 · Default: Slash,InvalidUtf8
16925 Limitations
16927 This uses the normal OpenStack Swift mechanism to refresh the Swift API
16928 credentials and ignores the expires field returned by the Hubic API.
16929 The Swift API doesn’t return a correct MD5SUM for segmented files (Dy‐
16931 namic or Static Large Objects) so rclone won’t check or use the MD5SUM
16932 for these.
16933 Jottacloud
16935 Paths are specified as remote:path
16936 Paths may be as deep as required, eg remote:directory/subdirectory.
16938 To configure Jottacloud you will need to generate a personal security
16940 token in the Jottacloud web inteface. You will the option to do in
16941 your account security settings (https://www.jottacloud.com/web/secure).
16942 Note that the web inteface may refer to this token as a JottaCli token.
16943 Here is an example of how to make a remote called remote. First run:
16945 rclone config
16947 This will guide you through an interactive setup process:
16949 No remotes found - make a new one
16951 n) New remote
16952 s) Set configuration password
16953 q) Quit config
16954 n/s/q> n
16955 name> jotta
16956 Type of storage to configure.
16957 Enter a string value. Press Enter for the default ("").
16958 Choose a number from below, or type in your own value
16959 [snip]
16960 XX / JottaCloud
16961 \ "jottacloud"
16962 [snip]
16963 Storage> jottacloud
16964 ** See help for jottacloud backend at: https://rclone.org/jottacloud/ **
16965 Edit advanced config? (y/n)
16967 y) Yes
16968 n) No
16969 y/n> n
16970 Remote config
16971 Generate a personal login token here: https://www.jottacloud.com/web/secure
16973 Login Token> <your token here>
16974 Do you want to use a non standard device/mountpoint e.g. for accessing files uploaded using the official Jottacloud client?
16976 y) Yes
16978 n) No
16979 y/n> y
16980 Please select the device to use. Normally this will be Jotta
16981 Choose a number from below, or type in an existing value
16982 1 > DESKTOP-3H31129
16983 2 > fla1
16984 3 > Jotta
16985 Devices> 3
16986 Please select the mountpoint to user. Normally this will be Archive
16987 Choose a number from below, or type in an existing value
16988 1 > Archive
16989 2 > Shared
16990 3 > Sync
16991 Mountpoints> 1
16992 --------------------
16993 [jotta]
16994 type = jottacloud
16995 user = 0xC4KE@gmail.com
16996 token = {........}
16997 device = Jotta
16998 mountpoint = Archive
16999 configVersion = 1
17000 --------------------
17001 y) Yes this is OK
17002 e) Edit this remote
17003 d) Delete this remote
17004 y/e/d> y
17005 Once configured you can then use rclone like this,
17007 List directories in top level of your Jottacloud
17009 rclone lsd remote:
17011 List all the files in your Jottacloud
17013 rclone ls remote:
17015 To copy a local directory to an Jottacloud directory called backup
17017 rclone copy /home/source remote:backup
17019 Devices and Mountpoints
17021 The official Jottacloud client registers a device for each computer you
17022 install it on and then creates a mountpoint for each folder you select
17023 for Backup. The web interface uses a special device called Jotta for
17024 the Archive, Sync and Shared mountpoints. In most cases you’ll want to
17025 use the Jotta/Archive device/mounpoint however if you want to access
17026 files uploaded by any of the official clients rclone provides the op‐
17027 tion to select other devices and mountpoints during config.
17028 –fast-list
17030 This remote supports --fast-list which allows you to use fewer transac‐
17031 tions in exchange for more memory. See the rclone docs (/docs/#fast-
17032 list) for more details.
17033 Note that the implementation in Jottacloud always uses only a single
17035 API request to get the entire list, so for large folders this could
17036 lead to long wait time before the first results are shown.
17037 Modified time and hashes
17039 Jottacloud allows modification times to be set on objects accurate to 1
17040 second. These will be used to detect whether objects need syncing or
17041 not.
17042 Jottacloud supports MD5 type hashes, so you can use the --checksum
17044 flag.
17045 Note that Jottacloud requires the MD5 hash before upload so if the
17047 source does not have an MD5 checksum then the file will be cached tem‐
17048 porarily on disk (wherever the TMPDIR environment variable points to)
17049 before it is uploaded. Small files will be cached in memory - see the
17050 --jottacloud-md5-memory-limit flag.
17051 Restricted filename characters
17053 In addition to the default restricted characters set (/overview/#re‐
17054 stricted-characters) the following characters are also replaced:
17055 Character Value Replacement
17057 ────────────────────────────────
17058 " 0x22 ＂
17059 * 0x2A ＊
17060 : 0x3A ：
17061 < 0x3C ＜
17062 > 0x3E ＞
17064 ? 0x3F ？
17065 | 0x7C ｜
17066 Invalid UTF-8 bytes will also be replaced (/overview/#invalid-utf8), as
17068 they can’t be used in XML strings.
17069 Deleting files
17071 By default rclone will send all files to the trash when deleting files.
17072 Due to a lack of API documentation emptying the trash is currently only
17073 possible via the Jottacloud website. If deleting permanently is re‐
17074 quired then use the --jottacloud-hard-delete flag, or set the equiva‐
17075 lent environment variable.
17076 Versions
17078 Jottacloud supports file versioning. When rclone uploads a new version
17079 of a file it creates a new version of it. Currently rclone only sup‐
17080 ports retrieving the current version but older versions can be accessed
17081 via the Jottacloud Website.
17082 Quota information
17084 To view your current quota you can use the rclone about remote: command
17085 which will display your usage limit (unless it is unlimited) and the
17086 current usage.
17087 Device IDs
17089 Jottacloud requires each `device' to be registered. Rclone brings such
17090 a registration to easily access your account but if you want to use
17091 Jottacloud together with rclone on multiple machines you NEED to create
17092 a seperate deviceID/deviceSecrect on each machine. You will asked dur‐
17093 ing setting up the remote. Please be aware that this also means that
17094 copying the rclone config from one machine to another does NOT work
17095 with Jottacloud accounts. You have to create it on each machine.
17096 Advanced Options
17098 Here are the advanced options specific to jottacloud (JottaCloud).
17099 –jottacloud-md5-memory-limit
17101 Files bigger than this will be cached on disk to calculate the MD5 if
17102 required.
17103 · Config: md5_memory_limit
17105 · Env Var: RCLONE_JOTTACLOUD_MD5_MEMORY_LIMIT
17107 · Type: SizeSuffix
17109 · Default: 10M
17111 –jottacloud-hard-delete
17113 Delete files permanently rather than putting them into the trash.
17114 · Config: hard_delete
17116 · Env Var: RCLONE_JOTTACLOUD_HARD_DELETE
17118 · Type: bool
17120 · Default: false
17122 –jottacloud-unlink
17124 Remove existing public link to file/folder with link command rather
17125 than creating. Default is false, meaning link command will create or
17126 retrieve public link.
17127 · Config: unlink
17129 · Env Var: RCLONE_JOTTACLOUD_UNLINK
17131 · Type: bool
17133 · Default: false
17135 –jottacloud-upload-resume-limit
17137 Files bigger than this can be resumed if the upload fail’s.
17138 · Config: upload_resume_limit
17140 · Env Var: RCLONE_JOTTACLOUD_UPLOAD_RESUME_LIMIT
17142 · Type: SizeSuffix
17144 · Default: 10M
17146 –jottacloud-encoding
17148 This sets the encoding for the backend.
17149 See: the encoding section in the overview (/overview/#encoding) for
17151 more info.
17152 · Config: encoding
17154 · Env Var: RCLONE_JOTTACLOUD_ENCODING
17156 · Type: MultiEncoder
17158 · Default: Slash,LtGt,DoubleQuote,Colon,Question,Aster‐
17160 isk,Pipe,Del,Ctl,InvalidUtf8,Dot
17161 Limitations
17163 Note that Jottacloud is case insensitive so you can’t have a file
17164 called “Hello.doc” and one called “hello.doc”.
17165 There are quite a few characters that can’t be in Jottacloud file
17167 names. Rclone will map these names to and from an identical looking
17168 unicode equivalent. For example if a file has a ? in it will be
17169 mapped to ？ instead.
17170 Jottacloud only supports filenames up to 255 characters in length.
17172 Troubleshooting
17174 Jottacloud exhibits some inconsistent behaviours regarding deleted
17175 files and folders which may cause Copy, Move and DirMove operations to
17176 previously deleted paths to fail. Emptying the trash should help in
17177 such cases.
17178 Koofr
17180 Paths are specified as remote:path
17181 Paths may be as deep as required, eg remote:directory/subdirectory.
17183 The initial setup for Koofr involves creating an application password
17185 for rclone. You can do that by opening the Koofr web application
17186 (https://app.koofr.net/app/admin/preferences/password), giving the
17187 password a nice name like rclone and clicking on generate.
17188 Here is an example of how to make a remote called koofr. First run:
17190 rclone config
17192 This will guide you through an interactive setup process:
17194 No remotes found - make a new one
17196 n) New remote
17197 s) Set configuration password
17198 q) Quit config
17199 n/s/q> n
17200 name> koofr
17201 Type of storage to configure.
17202 Enter a string value. Press Enter for the default ("").
17203 Choose a number from below, or type in your own value
17204 [snip]
17205 XX / Koofr
17206 \ "koofr"
17207 [snip]
17208 Storage> koofr
17209 ** See help for koofr backend at: https://rclone.org/koofr/ **
17210 Your Koofr user name
17212 Enter a string value. Press Enter for the default ("").
17213 user> USER@NAME
17214 Your Koofr password for rclone (generate one at https://app.koofr.net/app/admin/preferences/password)
17215 y) Yes type in my own password
17216 g) Generate random password
17217 y/g> y
17218 Enter the password:
17219 password:
17220 Confirm the password:
17221 password:
17222 Edit advanced config? (y/n)
17223 y) Yes
17224 n) No
17225 y/n> n
17226 Remote config
17227 --------------------
17228 [koofr]
17229 type = koofr
17230 baseurl = https://app.koofr.net
17231 user = USER@NAME
17232 password = *** ENCRYPTED ***
17233 --------------------
17234 y) Yes this is OK
17235 e) Edit this remote
17236 d) Delete this remote
17237 y/e/d> y
17238 You can choose to edit advanced config in order to enter your own ser‐
17240 vice URL if you use an on-premise or white label Koofr instance, or
17241 choose an alternative mount instead of your primary storage.
17242 Once configured you can then use rclone like this,
17244 List directories in top level of your Koofr
17246 rclone lsd koofr:
17248 List all the files in your Koofr
17250 rclone ls koofr:
17252 To copy a local directory to an Koofr directory called backup
17254 rclone copy /home/source remote:backup
17256 Restricted filename characters
17258 In addition to the default restricted characters set (/overview/#re‐
17259 stricted-characters) the following characters are also replaced:
17260 Character Value Replacement
17262 ────────────────────────────────
17263 \ 0x5C ＼
17264 Invalid UTF-8 bytes will also be replaced (/overview/#invalid-utf8), as
17266 they can’t be used in XML strings.
17267 Standard Options
17269 Here are the standard options specific to koofr (Koofr).
17270 –koofr-user
17272 Your Koofr user name
17273 · Config: user
17275 · Env Var: RCLONE_KOOFR_USER
17277 · Type: string
17279 · Default: ""
17281 –koofr-password
17283 Your Koofr password for rclone (generate one at
17284 https://app.koofr.net/app/admin/preferences/password)
17285 · Config: password
17287 · Env Var: RCLONE_KOOFR_PASSWORD
17289 · Type: string
17291 · Default: ""
17293 Advanced Options
17295 Here are the advanced options specific to koofr (Koofr).
17296 –koofr-endpoint
17298 The Koofr API endpoint to use
17299 · Config: endpoint
17301 · Env Var: RCLONE_KOOFR_ENDPOINT
17303 · Type: string
17305 · Default: “https://app.koofr.net”
17307 –koofr-mountid
17309 Mount ID of the mount to use. If omitted, the primary mount is used.
17310 · Config: mountid
17312 · Env Var: RCLONE_KOOFR_MOUNTID
17314 · Type: string
17316 · Default: ""
17318 –koofr-setmtime
17320 Does the backend support setting modification time. Set this to false
17321 if you use a mount ID that points to a Dropbox or Amazon Drive backend.
17322 · Config: setmtime
17324 · Env Var: RCLONE_KOOFR_SETMTIME
17326 · Type: bool
17328 · Default: true
17330 –koofr-encoding
17332 This sets the encoding for the backend.
17333 See: the encoding section in the overview (/overview/#encoding) for
17335 more info.
17336 · Config: encoding
17338 · Env Var: RCLONE_KOOFR_ENCODING
17340 · Type: MultiEncoder
17342 · Default: Slash,BackSlash,Del,Ctl,InvalidUtf8,Dot
17344 Limitations
17346 Note that Koofr is case insensitive so you can’t have a file called
17347 “Hello.doc” and one called “hello.doc”.
17348 Mail.ru Cloud
17350 Mail.ru Cloud (https://cloud.mail.ru/) is a cloud storage provided by a
17351 Russian internet company Mail.Ru Group (https://mail.ru). The official
17352 desktop client is Disk-O: (https://disk-o.cloud/), available only on
17353 Windows. (Please note that official sites are in Russian)
17354 Features highlights
17356 · Paths may be as deep as required, eg remote:directory/subdirectory
17357 · Files have a last modified time property, directories don’t
17359 · Deleted files are by default moved to the trash
17361 · Files and directories can be shared via public links
17363 · Partial uploads or streaming are not supported, file size must be
17365 known before upload
17366 · Maximum file size is limited to 2G for a free acount, unlimited for
17368 paid accounts
17369 · Storage keeps hash for all files and performs transparent deduplica‐
17371 tion, the hash algorithm is a modified SHA1
17372 · If a particular file is already present in storage, one can quickly
17374 submit file hash instead of long file upload (this optimization is
17375 supported by rclone)
17376 Configuration
17378 Here is an example of making a mailru configuration. First create a
17379 Mail.ru Cloud account and choose a tariff, then run
17380 rclone config
17382 This will guide you through an interactive setup process:
17384 No remotes found - make a new one
17386 n) New remote
17387 s) Set configuration password
17388 q) Quit config
17389 n/s/q> n
17390 name> remote
17391 Type of storage to configure.
17392 Type of storage to configure.
17393 Enter a string value. Press Enter for the default ("").
17394 Choose a number from below, or type in your own value
17395 [snip]
17396 XX / Mail.ru Cloud
17397 \ "mailru"
17398 [snip]
17399 Storage> mailru
17400 User name (usually email)
17401 Enter a string value. Press Enter for the default ("").
17402 user> username@mail.ru
17403 Password
17404 y) Yes type in my own password
17405 g) Generate random password
17406 y/g> y
17407 Enter the password:
17408 password:
17409 Confirm the password:
17410 password:
17411 Skip full upload if there is another file with same data hash.
17412 This feature is called "speedup" or "put by hash". It is especially efficient
17413 in case of generally available files like popular books, video or audio clips
17414 [snip]
17415 Enter a boolean value (true or false). Press Enter for the default ("true").
17416 Choose a number from below, or type in your own value
17417 1 / Enable
17418 \ "true"
17419 2 / Disable
17420 \ "false"
17421 speedup_enable> 1
17422 Edit advanced config? (y/n)
17423 y) Yes
17424 n) No
17425 y/n> n
17426 Remote config
17427 --------------------
17428 [remote]
17429 type = mailru
17430 user = username@mail.ru
17431 pass = *** ENCRYPTED ***
17432 speedup_enable = true
17433 --------------------
17434 y) Yes this is OK
17435 e) Edit this remote
17436 d) Delete this remote
17437 y/e/d> y
17438 Configuration of this backend does not require a local web browser.
17440 You can use the configured backend as shown below:
17441 See top level directories
17443 rclone lsd remote:
17445 Make a new directory
17447 rclone mkdir remote:directory
17449 List the contents of a directory
17451 rclone ls remote:directory
17453 Sync /home/local/directory to the remote path, deleting any excess
17455 files in the path.
17456 rclone sync /home/local/directory remote:directory
17458 Modified time
17460 Files support a modification time attribute with up to 1 second preci‐
17461 sion. Directories do not have a modification time, which is shown as
17462 “Jan 1 1970”.
17463 Hash checksums
17465 Hash sums use a custom Mail.ru algorithm based on SHA1. If file size
17466 is less than or equal to the SHA1 block size (20 bytes), its hash is
17467 simply its data right-padded with zero bytes. Hash sum of a larger
17468 file is computed as a SHA1 sum of the file data bytes concatenated with
17469 a decimal representation of the data length.
17470 Emptying Trash
17472 Removing a file or directory actually moves it to the trash, which is
17473 not visible to rclone but can be seen in a web browser. The trashed
17474 file still occupies part of total quota. If you wish to empty your
17475 trash and free some quota, you can use the rclone cleanup remote: com‐
17476 mand, which will permanently delete all your trashed files. This com‐
17477 mand does not take any path arguments.
17478 Quota information
17480 To view your current quota you can use the rclone about remote: command
17481 which will display your usage limit (quota) and the current usage.
17482 Restricted filename characters
17484 In addition to the default restricted characters set (/overview/#re‐
17485 stricted-characters) the following characters are also replaced:
17486 Character Value Replacement
17488 ────────────────────────────────
17489 " 0x22 ＂
17490 * 0x2A ＊
17491 : 0x3A ：
17492 < 0x3C ＜
17493 > 0x3E ＞
17494 ? 0x3F ？
17495 \ 0x5C ＼
17496 | 0x7C ｜
17497 Invalid UTF-8 bytes will also be replaced (/overview/#invalid-utf8), as
17499 they can’t be used in JSON strings.
17500 Limitations
17502 File size limits depend on your account. A single file size is limited
17503 by 2G for a free account and unlimited for paid tariffs. Please refer
17504 to the Mail.ru site for the total uploaded size limits.
17505 Note that Mailru is case insensitive so you can’t have a file called
17507 “Hello.doc” and one called “hello.doc”.
17508 Standard Options
17510 Here are the standard options specific to mailru (Mail.ru Cloud).
17511 –mailru-user
17513 User name (usually email)
17514 · Config: user
17516 · Env Var: RCLONE_MAILRU_USER
17518 · Type: string
17520 · Default: ""
17522 –mailru-pass
17524 Password
17525 · Config: pass
17527 · Env Var: RCLONE_MAILRU_PASS
17529 · Type: string
17531 · Default: ""
17533 –mailru-speedup-enable
17535 Skip full upload if there is another file with same data hash. This
17536 feature is called “speedup” or “put by hash”. It is especially effi‐
17537 cient in case of generally available files like popular books, video or
17538 audio clips, because files are searched by hash in all accounts of all
17539 mailru users. Please note that rclone may need local memory and disk
17540 space to calculate content hash in advance and decide whether full up‐
17541 load is required. Also, if rclone does not know file size in advance
17542 (e.g. in case of streaming or partial uploads), it will not even try
17543 this optimization.
17544 · Config: speedup_enable
17546 · Env Var: RCLONE_MAILRU_SPEEDUP_ENABLE
17548 · Type: bool
17550 · Default: true
17552 · Examples:
17554 · “true”
17556 · Enable
17558 · “false”
17560 · Disable
17562 Advanced Options
17564 Here are the advanced options specific to mailru (Mail.ru Cloud).
17565 –mailru-speedup-file-patterns
17567 Comma separated list of file name patterns eligible for speedup (put by
17568 hash). Patterns are case insensitive and can contain ’*’ or `?' meta
17569 characters.
17570 · Config: speedup_file_patterns
17572 · Env Var: RCLONE_MAILRU_SPEEDUP_FILE_PATTERNS
17574 · Type: string
17576 · Default: “.mkv,.avi,.mp4,.mp3,.zip,.gz,.rar,.pdf”
17578 · Examples:
17580 · ""
17582 · Empty list completely disables speedup (put by hash).
17584 · "*"
17586 · All files will be attempted for speedup.
17588 · “.mkv,.avi,.mp4,.mp3”
17590 · Only common audio/video files will be tried for put by hash.
17592 · “.zip,.gz,.rar,.pdf”
17594 · Only common archives or PDF books will be tried for speedup.
17596 –mailru-speedup-max-disk
17598 This option allows you to disable speedup (put by hash) for large files
17599 (because preliminary hashing can exhaust you RAM or disk space)
17600 · Config: speedup_max_disk
17602 · Env Var: RCLONE_MAILRU_SPEEDUP_MAX_DISK
17604 · Type: SizeSuffix
17606 · Default: 3G
17608 · Examples:
17610 · “0”
17612 · Completely disable speedup (put by hash).
17614 · “1G”
17616 · Files larger than 1Gb will be uploaded directly.
17618 · “3G”
17620 · Choose this option if you have less than 3Gb free on local disk.
17622 –mailru-speedup-max-memory
17624 Files larger than the size given below will always be hashed on disk.
17625 · Config: speedup_max_memory
17627 · Env Var: RCLONE_MAILRU_SPEEDUP_MAX_MEMORY
17629 · Type: SizeSuffix
17631 · Default: 32M
17633 · Examples:
17635 · “0”
17637 · Preliminary hashing will always be done in a temporary disk loca‐
17639 tion.
17640 · “32M”
17642 · Do not dedicate more than 32Mb RAM for preliminary hashing.
17644 · “256M”
17646 · You have at most 256Mb RAM free for hash calculations.
17648 –mailru-check-hash
17650 What should copy do if file checksum is mismatched or invalid
17651 · Config: check_hash
17653 · Env Var: RCLONE_MAILRU_CHECK_HASH
17655 · Type: bool
17657 · Default: true
17659 · Examples:
17661 · “true”
17663 · Fail with error.
17665 · “false”
17667 · Ignore and continue.
17669 –mailru-user-agent
17671 HTTP user agent used internally by client. Defaults to “rclone/VER‐
17672 SION” or “–user-agent” provided on command line.
17673 · Config: user_agent
17675 · Env Var: RCLONE_MAILRU_USER_AGENT
17677 · Type: string
17679 · Default: ""
17681 –mailru-quirks
17683 Comma separated list of internal maintenance flags. This option must
17684 not be used by an ordinary user. It is intended only to facilitate re‐
17685 mote troubleshooting of backend issues. Strict meaning of flags is not
17686 documented and not guaranteed to persist between releases. Quirks will
17687 be removed when the backend grows stable. Supported quirks: atom‐
17688 icmkdir binlist gzip insecure retry400
17689 · Config: quirks
17691 · Env Var: RCLONE_MAILRU_QUIRKS
17693 · Type: string
17695 · Default: ""
17697 –mailru-encoding
17699 This sets the encoding for the backend.
17700 See: the encoding section in the overview (/overview/#encoding) for
17702 more info.
17703 · Config: encoding
17705 · Env Var: RCLONE_MAILRU_ENCODING
17707 · Type: MultiEncoder
17709 · Default: Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,Back‐
17711 Slash,Del,Ctl,InvalidUtf8,Dot
17712 Mega
17714 Mega (https://mega.nz/) is a cloud storage and file hosting service
17715 known for its security feature where all files are encrypted locally
17716 before they are uploaded. This prevents anyone (including employees of
17717 Mega) from accessing the files without knowledge of the key used for
17718 encryption.
17719 This is an rclone backend for Mega which supports the file transfer
17721 features of Mega using the same client side encryption.
17722 Paths are specified as remote:path
17724 Paths may be as deep as required, eg remote:directory/subdirectory.
17726 Here is an example of how to make a remote called remote. First run:
17728 rclone config
17730 This will guide you through an interactive setup process:
17732 No remotes found - make a new one
17734 n) New remote
17735 s) Set configuration password
17736 q) Quit config
17737 n/s/q> n
17738 name> remote
17739 Type of storage to configure.
17740 Choose a number from below, or type in your own value
17741 [snip]
17742 XX / Mega
17743 \ "mega"
17744 [snip]
17745 Storage> mega
17746 User name
17747 user> you@example.com
17748 Password.
17749 y) Yes type in my own password
17750 g) Generate random password
17751 n) No leave this optional password blank
17752 y/g/n> y
17753 Enter the password:
17754 password:
17755 Confirm the password:
17756 password:
17757 Remote config
17758 --------------------
17759 [remote]
17760 type = mega
17761 user = you@example.com
17762 pass = *** ENCRYPTED ***
17763 --------------------
17764 y) Yes this is OK
17765 e) Edit this remote
17766 d) Delete this remote
17767 y/e/d> y
17768 NOTE: The encryption keys need to have been already generated after a
17770 regular login via the browser, otherwise attempting to use the creden‐
17771 tials in rclone will fail.
17772 Once configured you can then use rclone like this,
17774 List directories in top level of your Mega
17776 rclone lsd remote:
17778 List all the files in your Mega
17780 rclone ls remote:
17782 To copy a local directory to an Mega directory called backup
17784 rclone copy /home/source remote:backup
17786 Modified time and hashes
17788 Mega does not support modification times or hashes yet.
17789 Restricted filename characters
17791 Character Value Replacement
17792 ────────────────────────────────
17793 NUL 0x00 ␀
17794 / 0x2F ／
17795 Invalid UTF-8 bytes will also be replaced (/overview/#invalid-utf8), as
17797 they can’t be used in JSON strings.
17798 Duplicated files
17800 Mega can have two files with exactly the same name and path (unlike a
17801 normal file system).
17802 Duplicated files cause problems with the syncing and you will see mes‐
17804 sages in the log about duplicates.
17805 Use rclone dedupe to fix duplicated files.
17807 Failure to log-in
17809 Mega remotes seem to get blocked (reject logins) under “heavy use”. We
17810 haven’t worked out the exact blocking rules but it seems to be related
17811 to fast paced, sucessive rclone commands.
17812 For example, executing this command 90 times in a row rclone link re‐
17814 mote:file will cause the remote to become “blocked”. This is not an
17815 abnormal situation, for example if you wish to get the public links of
17816 a directory with hundred of files... After more or less a week, the re‐
17817 mote will remote accept rclone logins normally again.
17818 You can mitigate this issue by mounting the remote it with rclone
17820 mount. This will log-in when mounting and a log-out when unmounting
17821 only. You can also run rclone rcd and then use rclone rc to run the
17822 commands over the API to avoid logging in each time.
17823 Rclone does not currently close mega sessions (you can see them in the
17825 web interface), however closing the sessions does not solve the issue.
17826 If you space rclone commands by 3 seconds it will avoid blocking the
17828 remote. We haven’t identified the exact blocking rules, so perhaps one
17829 could execute the command 80 times without waiting and avoid blocking
17830 by waiting 3 seconds, then continuing...
17831 Note that this has been observed by trial and error and might not be
17833 set in stone.
17834 Other tools seem not to produce this blocking effect, as they use a
17836 different working approach (state-based, using sessionIDs instead of
17837 log-in) which isn’t compatible with the current stateless rclone ap‐
17838 proach.
17839 Note that once blocked, the use of other tools (such as megacmd) is not
17841 a sure workaround: following megacmd login times have been observed in
17842 sucession for blocked remote: 7 minutes, 20 min, 30min, 30 min, 30min.
17843 Web access looks unaffected though.
17844 Investigation is continuing in relation to workarounds based on time‐
17846 outs, pacers, retrials and tpslimits - if you discover something rele‐
17847 vant, please post on the forum.
17848 So, if rclone was working nicely and suddenly you are unable to log-in
17850 and you are sure the user and the password are correct, likely you have
17851 got the remote blocked for a while.
17852 Standard Options
17854 Here are the standard options specific to mega (Mega).
17855 –mega-user
17857 User name
17858 · Config: user
17860 · Env Var: RCLONE_MEGA_USER
17862 · Type: string
17864 · Default: ""
17866 –mega-pass
17868 Password.
17869 · Config: pass
17871 · Env Var: RCLONE_MEGA_PASS
17873 · Type: string
17875 · Default: ""
17877 Advanced Options
17879 Here are the advanced options specific to mega (Mega).
17880 –mega-debug
17882 Output more debug from Mega.
17883 If this flag is set (along with -vv) it will print further debugging
17885 information from the mega backend.
17886 · Config: debug
17888 · Env Var: RCLONE_MEGA_DEBUG
17890 · Type: bool
17892 · Default: false
17894 –mega-hard-delete
17896 Delete files permanently rather than putting them into the trash.
17897 Normally the mega backend will put all deletions into the trash rather
17899 than permanently deleting them. If you specify this then rclone will
17900 permanently delete objects instead.
17901 · Config: hard_delete
17903 · Env Var: RCLONE_MEGA_HARD_DELETE
17905 · Type: bool
17907 · Default: false
17909 –mega-encoding
17911 This sets the encoding for the backend.
17912 See: the encoding section in the overview (/overview/#encoding) for
17914 more info.
17915 · Config: encoding
17917 · Env Var: RCLONE_MEGA_ENCODING
17919 · Type: MultiEncoder
17921 · Default: Slash,InvalidUtf8,Dot
17923 Limitations
17925 This backend uses the go-mega go library
17926 (https://github.com/t3rm1n4l/go-mega) which is an opensource go library
17927 implementing the Mega API. There doesn’t appear to be any documenta‐
17928 tion for the mega protocol beyond the mega C++ SDK
17929 (https://github.com/meganz/sdk) source code so there are likely quite a
17930 few errors still remaining in this library.
17931 Mega allows duplicate files which may confuse rclone.
17933 Memory
17935 The memory backend is an in RAM backend. It does not persist its data
17936 - use the local backend for that.
17937 The memory backend behaves like a bucket based remote (eg like s3).
17939 Because it has no parameters you can just use it with the :memory: re‐
17940 mote name.
17941 You can configure it as a remote like this with rclone config too if
17943 you want to:
17944 No remotes found - make a new one
17946 n) New remote
17947 s) Set configuration password
17948 q) Quit config
17949 n/s/q> n
17950 name> remote
17951 Type of storage to configure.
17952 Enter a string value. Press Enter for the default ("").
17953 Choose a number from below, or type in your own value
17954 [snip]
17955 XX / Memory
17956 \ "memory"
17957 [snip]
17958 Storage> memory
17959 ** See help for memory backend at: https://rclone.org/memory/ **
17960 Remote config
17962 --------------------
17964 [remote]
17965 type = memory
17966 --------------------
17967 y) Yes this is OK (default)
17968 e) Edit this remote
17969 d) Delete this remote
17970 y/e/d> y
17971 Because the memory backend isn’t persistent it is most useful for test‐
17973 ing or with an rclone server or rclone mount, eg
17974 rclone mount :memory: /mnt/tmp
17976 rclone serve webdav :memory:
17977 rclone serve sftp :memory:
17978 Modified time and hashes
17980 The memory backend supports MD5 hashes and modification times accurate
17981 to 1 nS.
17982 Restricted filename characters
17984 The memory backend replaces the default restricted characters set
17985 (/overview/#restricted-characters).
17986 Microsoft Azure Blob Storage
17988 Paths are specified as remote:container (or remote: for the lsd com‐
17989 mand.) You may put subdirectories in too, eg remote:contain‐
17990 er/path/to/dir.
17991 Here is an example of making a Microsoft Azure Blob Storage configura‐
17993 tion. For a remote called remote. First run:
17994 rclone config
17996 This will guide you through an interactive setup process:
17998 No remotes found - make a new one
18000 n) New remote
18001 s) Set configuration password
18002 q) Quit config
18003 n/s/q> n
18004 name> remote
18005 Type of storage to configure.
18006 Choose a number from below, or type in your own value
18007 [snip]
18008 XX / Microsoft Azure Blob Storage
18009 \ "azureblob"
18010 [snip]
18011 Storage> azureblob
18012 Storage Account Name
18013 account> account_name
18014 Storage Account Key
18015 key> base64encodedkey==
18016 Endpoint for the service - leave blank normally.
18017 endpoint>
18018 Remote config
18019 --------------------
18020 [remote]
18021 account = account_name
18022 key = base64encodedkey==
18023 endpoint =
18024 --------------------
18025 y) Yes this is OK
18026 e) Edit this remote
18027 d) Delete this remote
18028 y/e/d> y
18029 See all containers
18031 rclone lsd remote:
18033 Make a new container
18035 rclone mkdir remote:container
18037 List the contents of a container
18039 rclone ls remote:container
18041 Sync /home/local/directory to the remote container, deleting any excess
18043 files in the container.
18044 rclone sync /home/local/directory remote:container
18046 –fast-list
18048 This remote supports --fast-list which allows you to use fewer transac‐
18049 tions in exchange for more memory. See the rclone docs (/docs/#fast-
18050 list) for more details.
18051 Modified time
18053 The modified time is stored as metadata on the object with the mtime
18054 key. It is stored using RFC3339 Format time with nanosecond precision.
18055 The metadata is supplied during directory listings so there is no over‐
18056 head to using it.
18057 Restricted filename characters
18059 In addition to the default restricted characters set (/overview/#re‐
18060 stricted-characters) the following characters are also replaced:
18061 Character Value Replacement
18063 ────────────────────────────────
18064 / 0x2F ／
18065 \ 0x5C ＼
18066 File names can also not end with the following characters. These only
18068 get replaced if they are last character in the name:
18069 Character Value Replacement
18071 ────────────────────────────────
18072 . 0x2E ．
18073 Invalid UTF-8 bytes will also be replaced (/overview/#invalid-utf8), as
18075 they can’t be used in JSON strings.
18076 Hashes
18078 MD5 hashes are stored with blobs. However blobs that were uploaded in
18079 chunks only have an MD5 if the source remote was capable of MD5 hashes,
18080 eg the local disk.
18081 Authenticating with Azure Blob Storage
18083 Rclone has 3 ways of authenticating with Azure Blob Storage:
18084 Account and Key
18086 This is the most straight forward and least flexible way. Just fill in
18087 the account and key lines and leave the rest blank.
18088 SAS URL
18090 This can be an account level SAS URL or container level SAS URL
18091 To use it leave account, key blank and fill in sas_url.
18093 Account level SAS URL or container level SAS URL can be obtained from
18095 Azure portal or Azure Storage Explorer. To get a container level SAS
18096 URL right click on a container in the Azure Blob explorer in the Azure
18097 portal.
18098 If You use container level SAS URL, rclone operations are permitted on‐
18100 ly on particular container, eg
18101 rclone ls azureblob:container or rclone ls azureblob:
18103 Since container name already exists in SAS URL, you can leave it empty
18105 as well.
18106 However these will not work
18108 rclone lsd azureblob:
18110 rclone ls azureblob:othercontainer
18111 This would be useful for temporarily allowing third parties access to a
18113 single container or putting credentials into an untrusted environment.
18114 Multipart uploads
18116 Rclone supports multipart uploads with Azure Blob storage. Files big‐
18117 ger than 256MB will be uploaded using chunked upload by default.
18118 The files will be uploaded in parallel in 4MB chunks (by default).
18120 Note that these chunks are buffered in memory and there may be up to
18121 --transfers of them being uploaded at once.
18122 Files can’t be split into more than 50,000 chunks so by default, so the
18124 largest file that can be uploaded with 4MB chunk size is 195GB. Above
18125 this rclone will double the chunk size until it creates less than
18126 50,000 chunks. By default this will mean a maximum file size of 3.2TB
18127 can be uploaded. This can be raised to 5TB using --azure‐
18128 blob-chunk-size 100M.
18129 Note that rclone doesn’t commit the block list until the end of the up‐
18131 load which means that there is a limit of 9.5TB of multipart uploads in
18132 progress as Azure won’t allow more than that amount of uncommitted
18133 blocks.
18134 Standard Options
18136 Here are the standard options specific to azureblob (Microsoft Azure
18137 Blob Storage).
18138 –azureblob-account
18140 Storage Account Name (leave blank to use SAS URL or Emulator)
18141 · Config: account
18143 · Env Var: RCLONE_AZUREBLOB_ACCOUNT
18145 · Type: string
18147 · Default: ""
18149 –azureblob-key
18151 Storage Account Key (leave blank to use SAS URL or Emulator)
18152 · Config: key
18154 · Env Var: RCLONE_AZUREBLOB_KEY
18156 · Type: string
18158 · Default: ""
18160 –azureblob-sas-url
18162 SAS URL for container level access only (leave blank if using ac‐
18163 count/key or Emulator)
18164 · Config: sas_url
18166 · Env Var: RCLONE_AZUREBLOB_SAS_URL
18168 · Type: string
18170 · Default: ""
18172 –azureblob-use-emulator
18174 Uses local storage emulator if provided as `true' (leave blank if using
18175 real azure storage endpoint)
18176 · Config: use_emulator
18178 · Env Var: RCLONE_AZUREBLOB_USE_EMULATOR
18180 · Type: bool
18182 · Default: false
18184 Advanced Options
18186 Here are the advanced options specific to azureblob (Microsoft Azure
18187 Blob Storage).
18188 –azureblob-endpoint
18190 Endpoint for the service Leave blank normally.
18191 · Config: endpoint
18193 · Env Var: RCLONE_AZUREBLOB_ENDPOINT
18195 · Type: string
18197 · Default: ""
18199 –azureblob-upload-cutoff
18201 Cutoff for switching to chunked upload (<= 256MB).
18202 · Config: upload_cutoff
18204 · Env Var: RCLONE_AZUREBLOB_UPLOAD_CUTOFF
18206 · Type: SizeSuffix
18208 · Default: 256M
18210 –azureblob-chunk-size
18212 Upload chunk size (<= 100MB).
18213 Note that this is stored in memory and there may be up to “–transfers”
18215 chunks stored at once in memory.
18216 · Config: chunk_size
18218 · Env Var: RCLONE_AZUREBLOB_CHUNK_SIZE
18220 · Type: SizeSuffix
18222 · Default: 4M
18224 –azureblob-list-chunk
18226 Size of blob list.
18227 This sets the number of blobs requested in each listing chunk. Default
18229 is the maximum, 5000. “List blobs” requests are permitted 2 minutes
18230 per megabyte to complete. If an operation is taking longer than 2 min‐
18231 utes per megabyte on average, it will time out ( source
18232 (https://docs.microsoft.com/en-us/rest/api/storageservices/setting-
18233 timeouts-for-blob-service-operations#exceptions-to-default-timeout-in‐
18234 terval) ). This can be used to limit the number of blobs items to re‐
18235 turn, to avoid the time out.
18236 · Config: list_chunk
18238 · Env Var: RCLONE_AZUREBLOB_LIST_CHUNK
18240 · Type: int
18242 · Default: 5000
18244 –azureblob-access-tier
18246 Access tier of blob: hot, cool or archive.
18247 Archived blobs can be restored by setting access tier to hot or cool.
18249 Leave blank if you intend to use default access tier, which is set at
18250 account level
18251 If there is no “access tier” specified, rclone doesn’t apply any tier.
18253 rclone performs “Set Tier” operation on blobs while uploading, if ob‐
18254 jects are not modified, specifying “access tier” to new one will have
18255 no effect. If blobs are in “archive tier” at remote, trying to perform
18256 data transfer operations from remote will not be allowed. User should
18257 first restore by tiering blob to “Hot” or “Cool”.
18258 · Config: access_tier
18260 · Env Var: RCLONE_AZUREBLOB_ACCESS_TIER
18262 · Type: string
18264 · Default: ""
18266 –azureblob-encoding
18268 This sets the encoding for the backend.
18269 See: the encoding section in the overview (/overview/#encoding) for
18271 more info.
18272 · Config: encoding
18274 · Env Var: RCLONE_AZUREBLOB_ENCODING
18276 · Type: MultiEncoder
18278 · Default: Slash,BackSlash,Del,Ctl,RightPeriod,InvalidUtf8
18280 Limitations
18282 MD5 sums are only uploaded with chunked files if the source has an MD5
18283 sum. This will always be the case for a local to azure copy.
18284 Azure Storage Emulator Support
18286 You can test rlcone with storage emulator locally, to do this make sure
18287 azure storage emulator installed locally and set up a new remote with
18288 rclone config follow instructions described in introduction, set
18289 use_emulator config as true, you do not need to provide default account
18290 name or key if using emulator.
18291 Microsoft OneDrive
18293 Paths are specified as remote:path
18294 Paths may be as deep as required, eg remote:directory/subdirectory.
18296 The initial setup for OneDrive involves getting a token from Microsoft
18298 which you need to do in your browser. rclone config walks you through
18299 it.
18300 Here is an example of how to make a remote called remote. First run:
18302 rclone config
18304 This will guide you through an interactive setup process:
18306 e) Edit existing remote
18308 n) New remote
18309 d) Delete remote
18310 r) Rename remote
18311 c) Copy remote
18312 s) Set configuration password
18313 q) Quit config
18314 e/n/d/r/c/s/q> n
18315 name> remote
18316 Type of storage to configure.
18317 Enter a string value. Press Enter for the default ("").
18318 Choose a number from below, or type in your own value
18319 [snip]
18320 XX / Microsoft OneDrive
18321 \ "onedrive"
18322 [snip]
18323 Storage> onedrive
18324 Microsoft App Client Id
18325 Leave blank normally.
18326 Enter a string value. Press Enter for the default ("").
18327 client_id>
18328 Microsoft App Client Secret
18329 Leave blank normally.
18330 Enter a string value. Press Enter for the default ("").
18331 client_secret>
18332 Edit advanced config? (y/n)
18333 y) Yes
18334 n) No
18335 y/n> n
18336 Remote config
18337 Use auto config?
18338 * Say Y if not sure
18339 * Say N if you are working on a remote or headless machine
18340 y) Yes
18341 n) No
18342 y/n> y
18343 If your browser doesn't open automatically go to the following link: http://127.0.0.1:53682/auth
18344 Log in and authorize rclone for access
18345 Waiting for code...
18346 Got code
18347 Choose a number from below, or type in an existing value
18348 1 / OneDrive Personal or Business
18349 \ "onedrive"
18350 2 / Sharepoint site
18351 \ "sharepoint"
18352 3 / Type in driveID
18353 \ "driveid"
18354 4 / Type in SiteID
18355 \ "siteid"
18356 5 / Search a Sharepoint site
18357 \ "search"
18358 Your choice> 1
18359 Found 1 drives, please select the one you want to use:
18360 0: OneDrive (business) id=b!Eqwertyuiopasdfghjklzxcvbnm-7mnbvcxzlkjhgfdsapoiuytrewqk
18361 Chose drive to use:> 0
18362 Found drive 'root' of type 'business', URL: https://org-my.sharepoint.com/personal/you/Documents
18363 Is that okay?
18364 y) Yes
18365 n) No
18366 y/n> y
18367 --------------------
18368 [remote]
18369 type = onedrive
18370 token = {"access_token":"youraccesstoken","token_type":"Bearer","refresh_token":"yourrefreshtoken","expiry":"2018-08-26T22:39:52.486512262+08:00"}
18371 drive_id = b!Eqwertyuiopasdfghjklzxcvbnm-7mnbvcxzlkjhgfdsapoiuytrewqk
18372 drive_type = business
18373 --------------------
18374 y) Yes this is OK
18375 e) Edit this remote
18376 d) Delete this remote
18377 y/e/d> y
18378 See the remote setup docs (https://rclone.org/remote_setup/) for how to
18380 set it up on a machine with no Internet browser available.
18381 Note that rclone runs a webserver on your local machine to collect the
18383 token as returned from Microsoft. This only runs from the moment it
18384 opens your browser to the moment you get back the verification code.
18385 This is on http://127.0.0.1:53682/ and this it may require you to un‐
18386 block it temporarily if you are running a host firewall.
18387 Once configured you can then use rclone like this,
18389 List directories in top level of your OneDrive
18391 rclone lsd remote:
18393 List all the files in your OneDrive
18395 rclone ls remote:
18397 To copy a local directory to an OneDrive directory called backup
18399 rclone copy /home/source remote:backup
18401 Getting your own Client ID and Key
18403 You can use your own Client ID if the default (client_id left blank)
18404 one doesn’t work for you or you see lots of throttling. The default
18405 Client ID and Key is shared by all rclone users when performing re‐
18406 quests.
18407 If you are having problems with them (E.g., seeing a lot of throt‐
18409 tling), you can get your own Client ID and Key by following the steps
18410 below:
18411 1. Open https://portal.azure.com/#blade/Microsoft_AAD_Regis‐
18413 teredApps/ApplicationsListBlade, then click New registration.
18414 2. Enter a name for your app, choose account type Any Azure AD directo‐
18416 ry - Multitenant, select Web in Redirect URI Enter http://local‐
18417 host:53682/ and click Register. Copy and keep the Application
18418 (client) ID under the app name for later use.
18419 3. Under manage select Certificates & secrets, click New client secret.
18421 Copy and keep that secret for later use.
18422 4. Under manage select API permissions, click Add a permission and se‐
18424 lect Microsoft Graph then select delegated permissions.
18425 5. Search and select the follwing permssions: Files.Read, Files.Read‐
18427 Write, Files.Read.All, Files.ReadWrite.All, offline_access, Us‐
18428 er.Read. Once selected click Add permissions at the bottom.
18429 Now the application is complete. Run rclone config to create or edit a
18431 OneDrive remote. Supply the app ID and password as Client ID and Se‐
18432 cret, respectively. rclone will walk you through the remaining steps.
18433 Modification time and hashes
18435 OneDrive allows modification times to be set on objects accurate to 1
18436 second. These will be used to detect whether objects need syncing or
18437 not.
18438 OneDrive personal supports SHA1 type hashes. OneDrive for business and
18440 Sharepoint Server support QuickXorHash (https://docs.microsoft.com/en-
18441 us/onedrive/developer/code-snippets/quickxorhash).
18442 For all types of OneDrive you can use the --checksum flag.
18444 Restricted filename characters
18446 In addition to the default restricted characters set (/overview/#re‐
18447 stricted-characters) the following characters are also replaced:
18448 Character Value Replacement
18450 ────────────────────────────────
18451 " 0x22 ＂
18452 * 0x2A ＊
18453 : 0x3A ：
18454 < 0x3C ＜
18455 > 0x3E ＞
18456 ? 0x3F ？
18457 \ 0x5C ＼
18458 | 0x7C ｜
18459 # 0x23 ＃
18460 % 0x25 ％
18461 File names can also not end with the following characters. These only
18463 get replaced if they are last character in the name:
18464 Character Value Replacement
18466 ────────────────────────────────
18467 SP 0x20 ␠
18468 . 0x2E ．
18469 File names can also not begin with the following characters. These on‐
18471 ly get replaced if they are first character in the name:
18472 Character Value Replacement
18474 ────────────────────────────────
18475 SP 0x20 ␠
18476 ~ 0x7E ～
18477 Invalid UTF-8 bytes will also be replaced (/overview/#invalid-utf8), as
18479 they can’t be used in JSON strings.
18480 Deleting files
18482 Any files you delete with rclone will end up in the trash. Microsoft
18483 doesn’t provide an API to permanently delete files, nor to empty the
18484 trash, so you will have to do that with one of Microsoft’s apps or via
18485 the OneDrive website.
18486 Standard Options
18488 Here are the standard options specific to onedrive (Microsoft
18489 OneDrive).
18490 –onedrive-client-id
18492 Microsoft App Client Id Leave blank normally.
18493 · Config: client_id
18495 · Env Var: RCLONE_ONEDRIVE_CLIENT_ID
18497 · Type: string
18499 · Default: ""
18501 –onedrive-client-secret
18503 Microsoft App Client Secret Leave blank normally.
18504 · Config: client_secret
18506 · Env Var: RCLONE_ONEDRIVE_CLIENT_SECRET
18508 · Type: string
18510 · Default: ""
18512 Advanced Options
18514 Here are the advanced options specific to onedrive (Microsoft
18515 OneDrive).
18516 –onedrive-chunk-size
18518 Chunk size to upload files with - must be multiple of 320k (327,680
18519 bytes).
18520 Above this size files will be chunked - must be multiple of 320k
18522 (327,680 bytes). Note that the chunks will be buffered into memory.
18523 · Config: chunk_size
18525 · Env Var: RCLONE_ONEDRIVE_CHUNK_SIZE
18527 · Type: SizeSuffix
18529 · Default: 10M
18531 –onedrive-drive-id
18533 The ID of the drive to use
18534 · Config: drive_id
18536 · Env Var: RCLONE_ONEDRIVE_DRIVE_ID
18538 · Type: string
18540 · Default: ""
18542 –onedrive-drive-type
18544 The type of the drive ( personal | business | documentLibrary )
18545 · Config: drive_type
18547 · Env Var: RCLONE_ONEDRIVE_DRIVE_TYPE
18549 · Type: string
18551 · Default: ""
18553 –onedrive-expose-onenote-files
18555 Set to make OneNote files show up in directory listings.
18556 By default rclone will hide OneNote files in directory listings because
18558 operations like “Open” and “Update” won’t work on them. But this be‐
18559 haviour may also prevent you from deleting them. If you want to delete
18560 OneNote files or otherwise want them to show up in directory listing,
18561 set this option.
18562 · Config: expose_onenote_files
18564 · Env Var: RCLONE_ONEDRIVE_EXPOSE_ONENOTE_FILES
18566 · Type: bool
18568 · Default: false
18570 –onedrive-encoding
18572 This sets the encoding for the backend.
18573 See: the encoding section in the overview (/overview/#encoding) for
18575 more info.
18576 · Config: encoding
18578 · Env Var: RCLONE_ONEDRIVE_ENCODING
18580 · Type: MultiEncoder
18582 · Default: Slash,LtGt,DoubleQuote,Colon,Question,Aster‐
18584 isk,Pipe,Hash,Percent,BackSlash,Del,Ctl,LeftSpace,Left‐
18585 Tilde,RightSpace,RightPeriod,InvalidUtf8,Dot
18586 Limitations
18588 Naming
18589 Note that OneDrive is case insensitive so you can’t have a file called
18590 “Hello.doc” and one called “hello.doc”.
18591 There are quite a few characters that can’t be in OneDrive file names.
18593 These can’t occur on Windows platforms, but on non-Windows platforms
18594 they are common. Rclone will map these names to and from an identical
18595 looking unicode equivalent. For example if a file has a ? in it will
18596 be mapped to ？ instead.
18597 File sizes
18599 The largest allowed file sizes are 15GB for OneDrive for Business and
18600 35GB for OneDrive Personal (Updated 4 Jan 2019).
18601 Path length
18603 The entire path, including the file name, must contain fewer than 400
18604 characters for OneDrive, OneDrive for Business and SharePoint Online.
18605 If you are encrypting file and folder names with rclone, you may want
18606 to pay attention to this limitation because the encrypted names are
18607 typically longer than the original ones.
18608 Number of files
18610 OneDrive seems to be OK with at least 50,000 files in a folder, but at
18611 100,000 rclone will get errors listing the directory like couldn’t list
18612 files: UnknownError:. See #2707 (https://github.com/rclone/rclone/is‐
18613 sues/2707) for more info.
18614 An official document about the limitations for different types of
18616 OneDrive can be found here (https://support.office.com/en-us/arti‐
18617 cle/invalid-file-names-and-file-types-in-onedrive-onedrive-for-busi‐
18618 ness-and-sharepoint-64883a5d-228e-48f5-b3d2-eb39e07630fa).
18619 Versioning issue
18621 Every change in OneDrive causes the service to create a new version.
18622 This counts against a users quota. For example changing the modifica‐
18623 tion time of a file creates a second version, so the file is using
18624 twice the space.
18625 The copy is the only rclone command affected by this as we copy the
18627 file and then afterwards set the modification time to match the source
18628 file.
18629 Note: Starting October 2018, users will no longer be able to disable
18631 versioning by default. This is because Microsoft has brought an update
18632 (https://techcommunity.microsoft.com/t5/Microsoft-OneDrive-Blog/New-Up‐
18633 dates-to-OneDrive-and-SharePoint-Team-Site-Versioning/ba-p/204390) to
18634 the mechanism. To change this new default setting, a PowerShell com‐
18635 mand is required to be run by a SharePoint admin. If you are an admin,
18636 you can run these commands in PowerShell to change that setting:
18637 1. Install-Module -Name Microsoft.Online.SharePoint.PowerShell (in case
18639 you haven’t installed this already)
18640 2. Import-Module Microsoft.Online.SharePoint.PowerShell -Disable‐
18642 NameChecking
18643 3. Connect-SPOService -Url https://YOURSITE-admin.sharepoint.com -Cre‐
18645 dential YOU@YOURSITE.COM (replacing YOURSITE, YOU, YOURSITE.COM with
18646 the actual values; this will prompt for your credentials)
18647 4. Set-SPOTenant -EnableMinimumVersionRequirement $False
18649 5. Disconnect-SPOService (to disconnect from the server)
18651 Below are the steps for normal users to disable versioning. If you
18653 don’t see the “No Versioning” option, make sure the above requirements
18654 are met.
18655 User Weropol (https://github.com/Weropol) has found a method to disable
18657 versioning on OneDrive
18658 1. Open the settings menu by clicking on the gear symbol at the top of
18660 the OneDrive Business page.
18661 2. Click Site settings.
18663 3. Once on the Site settings page, navigate to Site Administration >
18665 Site libraries and lists.
18666 4. Click Customize “Documents”.
18668 5. Click General Settings > Versioning Settings.
18670 6. Under Document Version History select the option No versioning.
18672 Note: This will disable the creation of new file versions, but will
18673 not remove any previous versions. Your documents are safe.
18674 7. Apply the changes by clicking OK.
18676 8. Use rclone to upload or modify files. (I also use the –no-up‐
18678 date-modtime flag)
18679 9. Restore the versioning settings after using rclone. (Optional)
18681 Troubleshooting
18683 Unexpected file size/hash differences on Sharepoint
18684 It is a known (https://github.com/OneDrive/onedrive-api-docs/is‐
18685 sues/935#issuecomment-441741631) issue that Sharepoint (not OneDrive or
18686 OneDrive for Business) silently modifies uploaded files, mainly Office
18687 files (.docx, .xlsx, etc.), causing file size and hash checks to fail.
18688 To use rclone with such affected files on Sharepoint, you may disable
18689 these checks with the following command line arguments:
18690 --ignore-checksum --ignore-size
18692 Replacing/deleting existing files on Sharepoint gets “item not found”
18694 It is a known (https://github.com/OneDrive/onedrive-api-docs/is‐
18695 sues/1068) issue that Sharepoint (not OneDrive or OneDrive for Busi‐
18696 ness) may return “item not found” errors when users try to replace or
18697 delete uploaded files; this seems to mainly affect Office files (.docx,
18698 .xlsx, etc.). As a workaround, you may use the --backup-dir <BACK‐
18699 UP_DIR> command line argument so rclone moves the files to be re‐
18700 placed/deleted into a given backup directory (instead of directly re‐
18701 placing/deleting them). For example, to instruct rclone to move the
18702 files into the directory rclone-backup-dir on backend mysharepoint, you
18703 may use:
18704 --backup-dir mysharepoint:rclone-backup-dir
18706 access_denied (AADSTS65005)
18708 Error: access_denied
18709 Code: AADSTS65005
18710 Description: Using application 'rclone' is currently not supported for your organization [YOUR_ORGANIZATION] because it is in an unmanaged state. An administrator needs to claim ownership of the company by DNS validation of [YOUR_ORGANIZATION] before the application rclone can be provisioned.
18711 This means that rclone can’t use the OneDrive for Business API with
18713 your account. You can’t do much about it, maybe write an email to your
18714 admins.
18715 However, there are other ways to interact with your OneDrive account.
18717 Have a look at the webdav backend: https://rclone.org/webdav/#share‐
18718 point
18719 invalid_grant (AADSTS50076)
18721 Error: invalid_grant
18722 Code: AADSTS50076
18723 Description: Due to a configuration change made by your administrator, or because you moved to a new location, you must use multi-factor authentication to access '...'.
18724 If you see the error above after enabling multi-factor authentication
18726 for your account, you can fix it by refreshing your OAuth refresh to‐
18727 ken. To do that, run rclone config, and choose to edit your OneDrive
18728 backend. Then, you don’t need to actually make any changes until you
18729 reach this question: Already have a token - refresh?. For this ques‐
18730 tion, answer y and go through the process to refresh your token, just
18731 like the first time the backend is configured. After this, rclone
18732 should work again for this backend.
18733 OpenDrive
18735 Paths are specified as remote:path
18736 Paths may be as deep as required, eg remote:directory/subdirectory.
18738 Here is an example of how to make a remote called remote. First run:
18740 rclone config
18742 This will guide you through an interactive setup process:
18744 n) New remote
18746 d) Delete remote
18747 q) Quit config
18748 e/n/d/q> n
18749 name> remote
18750 Type of storage to configure.
18751 Choose a number from below, or type in your own value
18752 [snip]
18753 XX / OpenDrive
18754 \ "opendrive"
18755 [snip]
18756 Storage> opendrive
18757 Username
18758 username>
18759 Password
18760 y) Yes type in my own password
18761 g) Generate random password
18762 y/g> y
18763 Enter the password:
18764 password:
18765 Confirm the password:
18766 password:
18767 --------------------
18768 [remote]
18769 username =
18770 password = *** ENCRYPTED ***
18771 --------------------
18772 y) Yes this is OK
18773 e) Edit this remote
18774 d) Delete this remote
18775 y/e/d> y
18776 List directories in top level of your OpenDrive
18778 rclone lsd remote:
18780 List all the files in your OpenDrive
18782 rclone ls remote:
18784 To copy a local directory to an OpenDrive directory called backup
18786 rclone copy /home/source remote:backup
18788 Modified time and MD5SUMs
18790 OpenDrive allows modification times to be set on objects accurate to 1
18791 second. These will be used to detect whether objects need syncing or
18792 not.
18793 Restricted filename characters
18795 Character Value Replacement
18796 ────────────────────────────────
18797 NUL 0x00 ␀
18798 / 0x2F ／
18799 " 0x22 ＂
18800 * 0x2A ＊
18801 : 0x3A ：
18802 < 0x3C ＜
18803 > 0x3E ＞
18804 ? 0x3F ？
18805 \ 0x5C ＼
18806 | 0x7C ｜
18807 File names can also not begin or end with the following characters.
18809 These only get replaced if they are the first or last character in the
18810 name:
18811 Character Value Replacement
18813 ────────────────────────────────
18814 SP 0x20 ␠
18815 HT 0x09 ␉
18816 LF 0x0A ␊
18817 VT 0x0B ␋
18818 CR 0x0D ␍
18819 Invalid UTF-8 bytes will also be replaced (/overview/#invalid-utf8), as
18821 they can’t be used in JSON strings.
18822 Standard Options
18824 Here are the standard options specific to opendrive (OpenDrive).
18825 –opendrive-username
18827 Username
18828 · Config: username
18830 · Env Var: RCLONE_OPENDRIVE_USERNAME
18832 · Type: string
18834 · Default: ""
18836 –opendrive-password
18838 Password.
18839 · Config: password
18841 · Env Var: RCLONE_OPENDRIVE_PASSWORD
18843 · Type: string
18845 · Default: ""
18847 Advanced Options
18849 Here are the advanced options specific to opendrive (OpenDrive).
18850 –opendrive-encoding
18852 This sets the encoding for the backend.
18853 See: the encoding section in the overview (/overview/#encoding) for
18855 more info.
18856 · Config: encoding
18858 · Env Var: RCLONE_OPENDRIVE_ENCODING
18860 · Type: MultiEncoder
18862 · Default: Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,Back‐
18864 Slash,LeftSpace,LeftCrLfHtVt,RightSpace,RightCrLfHtVt,InvalidUtf8,Dot
18865 –opendrive-chunk-size
18867 Files will be uploaded in chunks this size.
18868 Note that these chunks are buffered in memory so increasing them will
18870 increase memory use.
18871 · Config: chunk_size
18873 · Env Var: RCLONE_OPENDRIVE_CHUNK_SIZE
18875 · Type: SizeSuffix
18877 · Default: 10M
18879 Limitations
18881 Note that OpenDrive is case insensitive so you can’t have a file called
18882 “Hello.doc” and one called “hello.doc”.
18883 There are quite a few characters that can’t be in OpenDrive file names.
18885 These can’t occur on Windows platforms, but on non-Windows platforms
18886 they are common. Rclone will map these names to and from an identical
18887 looking unicode equivalent. For example if a file has a ? in it will
18888 be mapped to ？ instead.
18889 QingStor
18891 Paths are specified as remote:bucket (or remote: for the lsd command.)
18892 You may put subdirectories in too, eg remote:bucket/path/to/dir.
18893 Here is an example of making an QingStor configuration. First run
18895 rclone config
18897 This will guide you through an interactive setup process.
18899 No remotes found - make a new one
18901 n) New remote
18902 r) Rename remote
18903 c) Copy remote
18904 s) Set configuration password
18905 q) Quit config
18906 n/r/c/s/q> n
18907 name> remote
18908 Type of storage to configure.
18909 Choose a number from below, or type in your own value
18910 [snip]
18911 XX / QingStor Object Storage
18912 \ "qingstor"
18913 [snip]
18914 Storage> qingstor
18915 Get QingStor credentials from runtime. Only applies if access_key_id and secret_access_key is blank.
18916 Choose a number from below, or type in your own value
18917 1 / Enter QingStor credentials in the next step
18918 \ "false"
18919 2 / Get QingStor credentials from the environment (env vars or IAM)
18920 \ "true"
18921 env_auth> 1
18922 QingStor Access Key ID - leave blank for anonymous access or runtime credentials.
18923 access_key_id> access_key
18924 QingStor Secret Access Key (password) - leave blank for anonymous access or runtime credentials.
18925 secret_access_key> secret_key
18926 Enter a endpoint URL to connection QingStor API.
18927 Leave blank will use the default value "https://qingstor.com:443"
18928 endpoint>
18929 Zone connect to. Default is "pek3a".
18930 Choose a number from below, or type in your own value
18931 / The Beijing (China) Three Zone
18932 1 | Needs location constraint pek3a.
18933 \ "pek3a"
18934 / The Shanghai (China) First Zone
18935 2 | Needs location constraint sh1a.
18936 \ "sh1a"
18937 zone> 1
18938 Number of connnection retry.
18939 Leave blank will use the default value "3".
18940 connection_retries>
18941 Remote config
18942 --------------------
18943 [remote]
18944 env_auth = false
18945 access_key_id = access_key
18946 secret_access_key = secret_key
18947 endpoint =
18948 zone = pek3a
18949 connection_retries =
18950 --------------------
18951 y) Yes this is OK
18952 e) Edit this remote
18953 d) Delete this remote
18954 y/e/d> y
18955 This remote is called remote and can now be used like this
18957 See all buckets
18959 rclone lsd remote:
18961 Make a new bucket
18963 rclone mkdir remote:bucket
18965 List the contents of a bucket
18967 rclone ls remote:bucket
18969 Sync /home/local/directory to the remote bucket, deleting any excess
18971 files in the bucket.
18972 rclone sync /home/local/directory remote:bucket
18974 –fast-list
18976 This remote supports --fast-list which allows you to use fewer transac‐
18977 tions in exchange for more memory. See the rclone docs (/docs/#fast-
18978 list) for more details.
18979 Multipart uploads
18981 rclone supports multipart uploads with QingStor which means that it can
18982 upload files bigger than 5GB. Note that files uploaded with multipart
18983 upload don’t have an MD5SUM.
18984 Buckets and Zone
18986 With QingStor you can list buckets (rclone lsd) using any zone, but you
18987 can only access the content of a bucket from the zone it was created
18988 in. If you attempt to access a bucket from the wrong zone, you will
18989 get an error, incorrect zone, the bucket is not in 'XXX' zone.
18990 Authentication
18992 There are two ways to supply rclone with a set of QingStor credentials.
18993 In order of precedence:
18994 · Directly in the rclone configuration file (as configured by rclone
18996 config)
18997 · set access_key_id and secret_access_key
18999 · Runtime configuration:
19001 · set env_auth to true in the config file
19003 · Exporting the following environment variables before running rclone
19005 · Access Key ID: QS_ACCESS_KEY_ID or QS_ACCESS_KEY
19007 · Secret Access Key: QS_SECRET_ACCESS_KEY or QS_SECRET_KEY
19009 Restricted filename characters
19011 The control characters 0x00-0x1F and / are replaced as in the default
19012 restricted characters set (/overview/#restricted-characters). Note
19013 that 0x7F is not replaced.
19014 Invalid UTF-8 bytes will also be replaced (/overview/#invalid-utf8), as
19016 they can’t be used in JSON strings.
19017 Standard Options
19019 Here are the standard options specific to qingstor (QingCloud Object
19020 Storage).
19021 –qingstor-env-auth
19023 Get QingStor credentials from runtime. Only applies if access_key_id
19024 and secret_access_key is blank.
19025 · Config: env_auth
19027 · Env Var: RCLONE_QINGSTOR_ENV_AUTH
19029 · Type: bool
19031 · Default: false
19033 · Examples:
19035 · “false”
19037 · Enter QingStor credentials in the next step
19039 · “true”
19041 · Get QingStor credentials from the environment (env vars or IAM)
19043 –qingstor-access-key-id
19045 QingStor Access Key ID Leave blank for anonymous access or runtime cre‐
19046 dentials.
19047 · Config: access_key_id
19049 · Env Var: RCLONE_QINGSTOR_ACCESS_KEY_ID
19051 · Type: string
19053 · Default: ""
19055 –qingstor-secret-access-key
19057 QingStor Secret Access Key (password) Leave blank for anonymous access
19058 or runtime credentials.
19059 · Config: secret_access_key
19061 · Env Var: RCLONE_QINGSTOR_SECRET_ACCESS_KEY
19063 · Type: string
19065 · Default: ""
19067 –qingstor-endpoint
19069 Enter a endpoint URL to connection QingStor API. Leave blank will use
19070 the default value “https://qingstor.com:443”
19071 · Config: endpoint
19073 · Env Var: RCLONE_QINGSTOR_ENDPOINT
19075 · Type: string
19077 · Default: ""
19079 –qingstor-zone
19081 Zone to connect to. Default is “pek3a”.
19082 · Config: zone
19084 · Env Var: RCLONE_QINGSTOR_ZONE
19086 · Type: string
19088 · Default: ""
19090 · Examples:
19092 · “pek3a”
19094 · The Beijing (China) Three Zone
19096 · Needs location constraint pek3a.
19098 · “sh1a”
19100 · The Shanghai (China) First Zone
19102 · Needs location constraint sh1a.
19104 · “gd2a”
19106 · The Guangdong (China) Second Zone
19108 · Needs location constraint gd2a.
19110 Advanced Options
19112 Here are the advanced options specific to qingstor (QingCloud Object
19113 Storage).
19114 –qingstor-connection-retries
19116 Number of connection retries.
19117 · Config: connection_retries
19119 · Env Var: RCLONE_QINGSTOR_CONNECTION_RETRIES
19121 · Type: int
19123 · Default: 3
19125 –qingstor-upload-cutoff
19127 Cutoff for switching to chunked upload
19128 Any files larger than this will be uploaded in chunks of chunk_size.
19130 The minimum is 0 and the maximum is 5GB.
19131 · Config: upload_cutoff
19133 · Env Var: RCLONE_QINGSTOR_UPLOAD_CUTOFF
19135 · Type: SizeSuffix
19137 · Default: 200M
19139 –qingstor-chunk-size
19141 Chunk size to use for uploading.
19142 When uploading files larger than upload_cutoff they will be uploaded as
19144 multipart uploads using this chunk size.
19145 Note that “–qingstor-upload-concurrency” chunks of this size are
19147 buffered in memory per transfer.
19148 If you are transferring large files over high speed links and you have
19150 enough memory, then increasing this will speed up the transfers.
19151 · Config: chunk_size
19153 · Env Var: RCLONE_QINGSTOR_CHUNK_SIZE
19155 · Type: SizeSuffix
19157 · Default: 4M
19159 –qingstor-upload-concurrency
19161 Concurrency for multipart uploads.
19162 This is the number of chunks of the same file that are uploaded concur‐
19164 rently.
19165 NB if you set this to > 1 then the checksums of multpart uploads become
19167 corrupted (the uploads themselves are not corrupted though).
19168 If you are uploading small numbers of large file over high speed link
19170 and these uploads do not fully utilize your bandwidth, then increasing
19171 this may help to speed up the transfers.
19172 · Config: upload_concurrency
19174 · Env Var: RCLONE_QINGSTOR_UPLOAD_CONCURRENCY
19176 · Type: int
19178 · Default: 1
19180 –qingstor-encoding
19182 This sets the encoding for the backend.
19183 See: the encoding section in the overview (/overview/#encoding) for
19185 more info.
19186 · Config: encoding
19188 · Env Var: RCLONE_QINGSTOR_ENCODING
19190 · Type: MultiEncoder
19192 · Default: Slash,Ctl,InvalidUtf8
19194 Swift
19196 Swift refers to Openstack Object Storage (https://docs.open‐
19197 stack.org/swift/latest/). Commercial implementations of that being:
19198 · Rackspace Cloud Files (https://www.rackspace.com/cloud/files/)
19200 · Memset Memstore (https://www.memset.com/cloud/storage/)
19202 · OVH Object Storage (https://www.ovh.co.uk/public-cloud/storage/ob‐
19204 ject-storage/)
19205 · Oracle Cloud Storage (https://cloud.oracle.com/storage-opc)
19207 · IBM Bluemix Cloud ObjectStorage Swift (https://con‐
19209 sole.bluemix.net/docs/infrastructure/objectstorage-swift/index.html)
19210 Paths are specified as remote:container (or remote: for the lsd com‐
19212 mand.) You may put subdirectories in too, eg remote:contain‐
19213 er/path/to/dir.
19214 Here is an example of making a swift configuration. First run
19216 rclone config
19218 This will guide you through an interactive setup process.
19220 No remotes found - make a new one
19222 n) New remote
19223 s) Set configuration password
19224 q) Quit config
19225 n/s/q> n
19226 name> remote
19227 Type of storage to configure.
19228 Choose a number from below, or type in your own value
19229 [snip]
19230 XX / Openstack Swift (Rackspace Cloud Files, Memset Memstore, OVH)
19231 \ "swift"
19232 [snip]
19233 Storage> swift
19234 Get swift credentials from environment variables in standard OpenStack form.
19235 Choose a number from below, or type in your own value
19236 1 / Enter swift credentials in the next step
19237 \ "false"
19238 2 / Get swift credentials from environment vars. Leave other fields blank if using this.
19239 \ "true"
19240 env_auth> true
19241 User name to log in (OS_USERNAME).
19242 user>
19243 API key or password (OS_PASSWORD).
19244 key>
19245 Authentication URL for server (OS_AUTH_URL).
19246 Choose a number from below, or type in your own value
19247 1 / Rackspace US
19248 \ "https://auth.api.rackspacecloud.com/v1.0"
19249 2 / Rackspace UK
19250 \ "https://lon.auth.api.rackspacecloud.com/v1.0"
19251 3 / Rackspace v2
19252 \ "https://identity.api.rackspacecloud.com/v2.0"
19253 4 / Memset Memstore UK
19254 \ "https://auth.storage.memset.com/v1.0"
19255 5 / Memset Memstore UK v2
19256 \ "https://auth.storage.memset.com/v2.0"
19257 6 / OVH
19258 \ "https://auth.cloud.ovh.net/v3"
19259 auth>
19260 User ID to log in - optional - most swift systems use user and leave this blank (v3 auth) (OS_USER_ID).
19261 user_id>
19262 User domain - optional (v3 auth) (OS_USER_DOMAIN_NAME)
19263 domain>
19264 Tenant name - optional for v1 auth, this or tenant_id required otherwise (OS_TENANT_NAME or OS_PROJECT_NAME)
19265 tenant>
19266 Tenant ID - optional for v1 auth, this or tenant required otherwise (OS_TENANT_ID)
19267 tenant_id>
19268 Tenant domain - optional (v3 auth) (OS_PROJECT_DOMAIN_NAME)
19269 tenant_domain>
19270 Region name - optional (OS_REGION_NAME)
19271 region>
19272 Storage URL - optional (OS_STORAGE_URL)
19273 storage_url>
19274 Auth Token from alternate authentication - optional (OS_AUTH_TOKEN)
19275 auth_token>
19276 AuthVersion - optional - set to (1,2,3) if your auth URL has no version (ST_AUTH_VERSION)
19277 auth_version>
19278 Endpoint type to choose from the service catalogue (OS_ENDPOINT_TYPE)
19279 Choose a number from below, or type in your own value
19280 1 / Public (default, choose this if not sure)
19281 \ "public"
19282 2 / Internal (use internal service net)
19283 \ "internal"
19284 3 / Admin
19285 \ "admin"
19286 endpoint_type>
19287 Remote config
19288 --------------------
19289 [test]
19290 env_auth = true
19291 user =
19292 key =
19293 auth =
19294 user_id =
19295 domain =
19296 tenant =
19297 tenant_id =
19298 tenant_domain =
19299 region =
19300 storage_url =
19301 auth_token =
19302 auth_version =
19303 endpoint_type =
19304 --------------------
19305 y) Yes this is OK
19306 e) Edit this remote
19307 d) Delete this remote
19308 y/e/d> y
19309 This remote is called remote and can now be used like this
19311 See all containers
19313 rclone lsd remote:
19315 Make a new container
19317 rclone mkdir remote:container
19319 List the contents of a container
19321 rclone ls remote:container
19323 Sync /home/local/directory to the remote container, deleting any excess
19325 files in the container.
19326 rclone sync /home/local/directory remote:container
19328 Configuration from an OpenStack credentials file
19330 An OpenStack credentials file typically looks something something like
19331 this (without the comments)
19332 export OS_AUTH_URL=https://a.provider.net/v2.0
19334 export OS_TENANT_ID=ffffffffffffffffffffffffffffffff
19335 export OS_TENANT_NAME="1234567890123456"
19336 export OS_USERNAME="123abc567xy"
19337 echo "Please enter your OpenStack Password: "
19338 read -sr OS_PASSWORD_INPUT
19339 export OS_PASSWORD=$OS_PASSWORD_INPUT
19340 export OS_REGION_NAME="SBG1"
19341 if [ -z "$OS_REGION_NAME" ]; then unset OS_REGION_NAME; fi
19342 The config file needs to look something like this where $OS_USERNAME
19344 represents the value of the OS_USERNAME variable - 123abc567xy in the
19345 example above.
19346 [remote]
19348 type = swift
19349 user = $OS_USERNAME
19350 key = $OS_PASSWORD
19351 auth = $OS_AUTH_URL
19352 tenant = $OS_TENANT_NAME
19353 Note that you may (or may not) need to set region too - try without
19355 first.
19356 Configuration from the environment
19358 If you prefer you can configure rclone to use swift using a standard
19359 set of OpenStack environment variables.
19360 When you run through the config, make sure you choose true for env_auth
19362 and leave everything else blank.
19363 rclone will then set any empty config parameters from the environment
19365 using standard OpenStack environment variables. There is a list of the
19366 variables (https://godoc.org/github.com/ncw/swift#Connection.ApplyEnvi‐
19367 ronment) in the docs for the swift library.
19368 Using an alternate authentication method
19370 If your OpenStack installation uses a non-standard authentication
19371 method that might not be yet supported by rclone or the underlying
19372 swift library, you can authenticate externally (e.g. calling manually
19373 the openstack commands to get a token). Then, you just need to pass
19374 the two configuration variables auth_token and storage_url. If they
19375 are both provided, the other variables are ignored. rclone will not
19376 try to authenticate but instead assume it is already authenticated and
19377 use these two variables to access the OpenStack installation.
19378 Using rclone without a config file
19380 You can use rclone with swift without a config file, if desired, like
19381 this:
19382 source openstack-credentials-file
19384 export RCLONE_CONFIG_MYREMOTE_TYPE=swift
19385 export RCLONE_CONFIG_MYREMOTE_ENV_AUTH=true
19386 rclone lsd myremote:
19387 –fast-list
19389 This remote supports --fast-list which allows you to use fewer transac‐
19390 tions in exchange for more memory. See the rclone docs (/docs/#fast-
19391 list) for more details.
19392 –update and –use-server-modtime
19394 As noted below, the modified time is stored on metadata on the object.
19395 It is used by default for all operations that require checking the time
19396 a file was last updated. It allows rclone to treat the remote more
19397 like a true filesystem, but it is inefficient because it requires an
19398 extra API call to retrieve the metadata.
19399 For many operations, the time the object was last uploaded to the re‐
19401 mote is sufficient to determine if it is “dirty”. By using --update
19402 along with --use-server-modtime, you can avoid the extra API call and
19403 simply upload files whose local modtime is newer than the time it was
19404 last uploaded.
19405 Standard Options
19407 Here are the standard options specific to swift (Openstack Swift
19408 (Rackspace Cloud Files, Memset Memstore, OVH)).
19409 –swift-env-auth
19411 Get swift credentials from environment variables in standard OpenStack
19412 form.
19413 · Config: env_auth
19415 · Env Var: RCLONE_SWIFT_ENV_AUTH
19417 · Type: bool
19419 · Default: false
19421 · Examples:
19423 · “false”
19425 · Enter swift credentials in the next step
19427 · “true”
19429 · Get swift credentials from environment vars. Leave other fields
19431 blank if using this.
19432 –swift-user
19434 User name to log in (OS_USERNAME).
19435 · Config: user
19437 · Env Var: RCLONE_SWIFT_USER
19439 · Type: string
19441 · Default: ""
19443 –swift-key
19445 API key or password (OS_PASSWORD).
19446 · Config: key
19448 · Env Var: RCLONE_SWIFT_KEY
19450 · Type: string
19452 · Default: ""
19454 –swift-auth
19456 Authentication URL for server (OS_AUTH_URL).
19457 · Config: auth
19459 · Env Var: RCLONE_SWIFT_AUTH
19461 · Type: string
19463 · Default: ""
19465 · Examples:
19467 · “https://auth.api.rackspacecloud.com/v1.0”
19469 · Rackspace US
19471 · “https://lon.auth.api.rackspacecloud.com/v1.0”
19473 · Rackspace UK
19475 · “https://identity.api.rackspacecloud.com/v2.0”
19477 · Rackspace v2
19479 · “https://auth.storage.memset.com/v1.0”
19481 · Memset Memstore UK
19483 · “https://auth.storage.memset.com/v2.0”
19485 · Memset Memstore UK v2
19487 · “https://auth.cloud.ovh.net/v3”
19489 · OVH
19491 –swift-user-id
19493 User ID to log in - optional - most swift systems use user and leave
19494 this blank (v3 auth) (OS_USER_ID).
19495 · Config: user_id
19497 · Env Var: RCLONE_SWIFT_USER_ID
19499 · Type: string
19501 · Default: ""
19503 –swift-domain
19505 User domain - optional (v3 auth) (OS_USER_DOMAIN_NAME)
19506 · Config: domain
19508 · Env Var: RCLONE_SWIFT_DOMAIN
19510 · Type: string
19512 · Default: ""
19514 –swift-tenant
19516 Tenant name - optional for v1 auth, this or tenant_id required other‐
19517 wise (OS_TENANT_NAME or OS_PROJECT_NAME)
19518 · Config: tenant
19520 · Env Var: RCLONE_SWIFT_TENANT
19522 · Type: string
19524 · Default: ""
19526 –swift-tenant-id
19528 Tenant ID - optional for v1 auth, this or tenant required otherwise
19529 (OS_TENANT_ID)
19530 · Config: tenant_id
19532 · Env Var: RCLONE_SWIFT_TENANT_ID
19534 · Type: string
19536 · Default: ""
19538 –swift-tenant-domain
19540 Tenant domain - optional (v3 auth) (OS_PROJECT_DOMAIN_NAME)
19541 · Config: tenant_domain
19543 · Env Var: RCLONE_SWIFT_TENANT_DOMAIN
19545 · Type: string
19547 · Default: ""
19549 –swift-region
19551 Region name - optional (OS_REGION_NAME)
19552 · Config: region
19554 · Env Var: RCLONE_SWIFT_REGION
19556 · Type: string
19558 · Default: ""
19560 –swift-storage-url
19562 Storage URL - optional (OS_STORAGE_URL)
19563 · Config: storage_url
19565 · Env Var: RCLONE_SWIFT_STORAGE_URL
19567 · Type: string
19569 · Default: ""
19571 –swift-auth-token
19573 Auth Token from alternate authentication - optional (OS_AUTH_TOKEN)
19574 · Config: auth_token
19576 · Env Var: RCLONE_SWIFT_AUTH_TOKEN
19578 · Type: string
19580 · Default: ""
19582 –swift-application-credential-id
19584 Application Credential ID (OS_APPLICATION_CREDENTIAL_ID)
19585 · Config: application_credential_id
19587 · Env Var: RCLONE_SWIFT_APPLICATION_CREDENTIAL_ID
19589 · Type: string
19591 · Default: ""
19593 –swift-application-credential-name
19595 Application Credential Name (OS_APPLICATION_CREDENTIAL_NAME)
19596 · Config: application_credential_name
19598 · Env Var: RCLONE_SWIFT_APPLICATION_CREDENTIAL_NAME
19600 · Type: string
19602 · Default: ""
19604 –swift-application-credential-secret
19606 Application Credential Secret (OS_APPLICATION_CREDENTIAL_SECRET)
19607 · Config: application_credential_secret
19609 · Env Var: RCLONE_SWIFT_APPLICATION_CREDENTIAL_SECRET
19611 · Type: string
19613 · Default: ""
19615 –swift-auth-version
19617 AuthVersion - optional - set to (1,2,3) if your auth URL has no version
19618 (ST_AUTH_VERSION)
19619 · Config: auth_version
19621 · Env Var: RCLONE_SWIFT_AUTH_VERSION
19623 · Type: int
19625 · Default: 0
19627 –swift-endpoint-type
19629 Endpoint type to choose from the service catalogue (OS_ENDPOINT_TYPE)
19630 · Config: endpoint_type
19632 · Env Var: RCLONE_SWIFT_ENDPOINT_TYPE
19634 · Type: string
19636 · Default: “public”
19638 · Examples:
19640 · “public”
19642 · Public (default, choose this if not sure)
19644 · “internal”
19646 · Internal (use internal service net)
19648 · “admin”
19650 · Admin
19652 –swift-storage-policy
19654 The storage policy to use when creating a new container
19655 This applies the specified storage policy when creating a new contain‐
19657 er. The policy cannot be changed afterwards. The allowed configura‐
19658 tion values and their meaning depend on your Swift storage provider.
19659 · Config: storage_policy
19661 · Env Var: RCLONE_SWIFT_STORAGE_POLICY
19663 · Type: string
19665 · Default: ""
19667 · Examples:
19669 · ""
19671 · Default
19673 · “pcs”
19675 · OVH Public Cloud Storage
19677 · “pca”
19679 · OVH Public Cloud Archive
19681 Advanced Options
19683 Here are the advanced options specific to swift (Openstack Swift
19684 (Rackspace Cloud Files, Memset Memstore, OVH)).
19685 –swift-chunk-size
19687 Above this size files will be chunked into a _segments container.
19688 Above this size files will be chunked into a _segments container. The
19690 default for this is 5GB which is its maximum value.
19691 · Config: chunk_size
19693 · Env Var: RCLONE_SWIFT_CHUNK_SIZE
19695 · Type: SizeSuffix
19697 · Default: 5G
19699 –swift-no-chunk
19701 Don’t chunk files during streaming upload.
19702 When doing streaming uploads (eg using rcat or mount) setting this flag
19704 will cause the swift backend to not upload chunked files.
19705 This will limit the maximum upload size to 5GB. However non chunked
19707 files are easier to deal with and have an MD5SUM.
19708 Rclone will still chunk files bigger than chunk_size when doing normal
19710 copy operations.
19711 · Config: no_chunk
19713 · Env Var: RCLONE_SWIFT_NO_CHUNK
19715 · Type: bool
19717 · Default: false
19719 –swift-encoding
19721 This sets the encoding for the backend.
19722 See: the encoding section in the overview (/overview/#encoding) for
19724 more info.
19725 · Config: encoding
19727 · Env Var: RCLONE_SWIFT_ENCODING
19729 · Type: MultiEncoder
19731 · Default: Slash,InvalidUtf8
19733 Modified time
19735 The modified time is stored as metadata on the object as X-Ob‐
19736 ject-Meta-Mtime as floating point since the epoch accurate to 1 ns.
19737 This is a defacto standard (used in the official python-swiftclient
19739 amongst others) for storing the modification time for an object.
19740 Restricted filename characters
19742 Character Value Replacement
19743 ────────────────────────────────
19744 NUL 0x00 ␀
19745 / 0x2F ／
19746 Invalid UTF-8 bytes will also be replaced (/overview/#invalid-utf8), as
19748 they can’t be used in JSON strings.
19749 Limitations
19751 The Swift API doesn’t return a correct MD5SUM for segmented files (Dy‐
19752 namic or Static Large Objects) so rclone won’t check or use the MD5SUM
19753 for these.
19754 Troubleshooting
19756 Rclone gives Failed to create file system for “remote:”: Bad Request
19757 Due to an oddity of the underlying swift library, it gives a “Bad Re‐
19758 quest” error rather than a more sensible error when the authentication
19759 fails for Swift.
19760 So this most likely means your username / password is wrong. You can
19762 investigate further with the --dump-bodies flag.
19763 This may also be caused by specifying the region when you shouldn’t
19765 have (eg OVH).
19766 Rclone gives Failed to create file system: Response didn’t have storage
19768 storage url and auth token
19769 This is most likely caused by forgetting to specify your tenant when
19770 setting up a swift remote.
19771 pCloud
19773 Paths are specified as remote:path
19774 Paths may be as deep as required, eg remote:directory/subdirectory.
19776 The initial setup for pCloud involves getting a token from pCloud which
19778 you need to do in your browser. rclone config walks you through it.
19779 Here is an example of how to make a remote called remote. First run:
19781 rclone config
19783 This will guide you through an interactive setup process:
19785 No remotes found - make a new one
19787 n) New remote
19788 s) Set configuration password
19789 q) Quit config
19790 n/s/q> n
19791 name> remote
19792 Type of storage to configure.
19793 Choose a number from below, or type in your own value
19794 [snip]
19795 XX / Pcloud
19796 \ "pcloud"
19797 [snip]
19798 Storage> pcloud
19799 Pcloud App Client Id - leave blank normally.
19800 client_id>
19801 Pcloud App Client Secret - leave blank normally.
19802 client_secret>
19803 Remote config
19804 Use auto config?
19805 * Say Y if not sure
19806 * Say N if you are working on a remote or headless machine
19807 y) Yes
19808 n) No
19809 y/n> y
19810 If your browser doesn't open automatically go to the following link: http://127.0.0.1:53682/auth
19811 Log in and authorize rclone for access
19812 Waiting for code...
19813 Got code
19814 --------------------
19815 [remote]
19816 client_id =
19817 client_secret =
19818 token = {"access_token":"XXX","token_type":"bearer","expiry":"0001-01-01T00:00:00Z"}
19819 --------------------
19820 y) Yes this is OK
19821 e) Edit this remote
19822 d) Delete this remote
19823 y/e/d> y
19824 See the remote setup docs (https://rclone.org/remote_setup/) for how to
19826 set it up on a machine with no Internet browser available.
19827 Note that rclone runs a webserver on your local machine to collect the
19829 token as returned from pCloud. This only runs from the moment it opens
19830 your browser to the moment you get back the verification code. This is
19831 on http://127.0.0.1:53682/ and this it may require you to unblock it
19832 temporarily if you are running a host firewall.
19833 Once configured you can then use rclone like this,
19835 List directories in top level of your pCloud
19837 rclone lsd remote:
19839 List all the files in your pCloud
19841 rclone ls remote:
19843 To copy a local directory to an pCloud directory called backup
19845 rclone copy /home/source remote:backup
19847 Modified time and hashes
19849 pCloud allows modification times to be set on objects accurate to 1
19850 second. These will be used to detect whether objects need syncing or
19851 not. In order to set a Modification time pCloud requires the object be
19852 re-uploaded.
19853 pCloud supports MD5 and SHA1 type hashes, so you can use the --checksum
19855 flag.
19856 Restricted filename characters
19858 In addition to the default restricted characters set (/overview/#re‐
19859 stricted-characters) the following characters are also replaced:
19860 Character Value Replacement
19862 ────────────────────────────────
19863 \ 0x5C ＼
19864 Invalid UTF-8 bytes will also be replaced (/overview/#invalid-utf8), as
19866 they can’t be used in JSON strings.
19867 Deleting files
19869 Deleted files will be moved to the trash. Your subscription level will
19870 determine how long items stay in the trash. rclone cleanup can be used
19871 to empty the trash.
19872 Standard Options
19874 Here are the standard options specific to pcloud (Pcloud).
19875 –pcloud-client-id
19877 Pcloud App Client Id Leave blank normally.
19878 · Config: client_id
19880 · Env Var: RCLONE_PCLOUD_CLIENT_ID
19882 · Type: string
19884 · Default: ""
19886 –pcloud-client-secret
19888 Pcloud App Client Secret Leave blank normally.
19889 · Config: client_secret
19891 · Env Var: RCLONE_PCLOUD_CLIENT_SECRET
19893 · Type: string
19895 · Default: ""
19897 Advanced Options
19899 Here are the advanced options specific to pcloud (Pcloud).
19900 –pcloud-encoding
19902 This sets the encoding for the backend.
19903 See: the encoding section in the overview (/overview/#encoding) for
19905 more info.
19906 · Config: encoding
19908 · Env Var: RCLONE_PCLOUD_ENCODING
19910 · Type: MultiEncoder
19912 · Default: Slash,BackSlash,Del,Ctl,InvalidUtf8,Dot
19914 premiumize.me
19916 Paths are specified as remote:path
19917 Paths may be as deep as required, eg remote:directory/subdirectory.
19919 The initial setup for premiumize.me (https://premiumize.me/) involves
19921 getting a token from premiumize.me which you need to do in your brows‐
19922 er. rclone config walks you through it.
19923 Here is an example of how to make a remote called remote. First run:
19925 rclone config
19927 This will guide you through an interactive setup process:
19929 No remotes found - make a new one
19931 n) New remote
19932 s) Set configuration password
19933 q) Quit config
19934 n/s/q> n
19935 name> remote
19936 Type of storage to configure.
19937 Enter a string value. Press Enter for the default ("").
19938 Choose a number from below, or type in your own value
19939 [snip]
19940 XX / premiumize.me
19941 \ "premiumizeme"
19942 [snip]
19943 Storage> premiumizeme
19944 ** See help for premiumizeme backend at: https://rclone.org/premiumizeme/ **
19945 Remote config
19947 Use auto config?
19948 * Say Y if not sure
19949 * Say N if you are working on a remote or headless machine
19950 y) Yes
19951 n) No
19952 y/n> y
19953 If your browser doesn't open automatically go to the following link: http://127.0.0.1:53682/auth
19954 Log in and authorize rclone for access
19955 Waiting for code...
19956 Got code
19957 --------------------
19958 [remote]
19959 type = premiumizeme
19960 token = {"access_token":"XXX","token_type":"Bearer","refresh_token":"XXX","expiry":"2029-08-07T18:44:15.548915378+01:00"}
19961 --------------------
19962 y) Yes this is OK
19963 e) Edit this remote
19964 d) Delete this remote
19965 y/e/d>
19966 See the remote setup docs (https://rclone.org/remote_setup/) for how to
19968 set it up on a machine with no Internet browser available.
19969 Note that rclone runs a webserver on your local machine to collect the
19971 token as returned from premiumize.me. This only runs from the moment
19972 it opens your browser to the moment you get back the verification code.
19973 This is on http://127.0.0.1:53682/ and this it may require you to un‐
19974 block it temporarily if you are running a host firewall.
19975 Once configured you can then use rclone like this,
19977 List directories in top level of your premiumize.me
19979 rclone lsd remote:
19981 List all the files in your premiumize.me
19983 rclone ls remote:
19985 To copy a local directory to an premiumize.me directory called backup
19987 rclone copy /home/source remote:backup
19989 Modified time and hashes
19991 premiumize.me does not support modification times or hashes, therefore
19992 syncing will default to --size-only checking. Note that using --update
19993 will work.
19994 Restricted filename characters
19996 In addition to the default restricted characters set (/overview/#re‐
19997 stricted-characters) the following characters are also replaced:
19998 Character Value Replacement
20000 ────────────────────────────────
20001 \ 0x5C ＼
20002 " 0x22 ＂
20003 Invalid UTF-8 bytes will also be replaced (/overview/#invalid-utf8), as
20005 they can’t be used in JSON strings.
20006 Standard Options
20008 Here are the standard options specific to premiumizeme (premiumize.me).
20009 –premiumizeme-api-key
20011 API Key.
20012 This is not normally used - use oauth instead.
20014 · Config: api_key
20016 · Env Var: RCLONE_PREMIUMIZEME_API_KEY
20018 · Type: string
20020 · Default: ""
20022 Advanced Options
20024 Here are the advanced options specific to premiumizeme (premiumize.me).
20025 –premiumizeme-encoding
20027 This sets the encoding for the backend.
20028 See: the encoding section in the overview (/overview/#encoding) for
20030 more info.
20031 · Config: encoding
20033 · Env Var: RCLONE_PREMIUMIZEME_ENCODING
20035 · Type: MultiEncoder
20037 · Default: Slash,DoubleQuote,BackSlash,Del,Ctl,InvalidUtf8,Dot
20039 Limitations
20041 Note that premiumize.me is case insensitive so you can’t have a file
20042 called “Hello.doc” and one called “hello.doc”.
20043 premiumize.me file names can’t have the \ or " characters in. rclone
20045 maps these to and from an identical looking unicode equivalents ＼ and
20046 ＂
20047 premiumize.me only supports filenames up to 255 characters in length.
20049 put.io
20051 Paths are specified as remote:path
20052 put.io paths may be as deep as required, eg remote:directory/subdirec‐
20054 tory.
20055 The initial setup for put.io involves getting a token from put.io which
20057 you need to do in your browser. rclone config walks you through it.
20058 Here is an example of how to make a remote called remote. First run:
20060 rclone config
20062 This will guide you through an interactive setup process:
20064 No remotes found - make a new one
20066 n) New remote
20067 s) Set configuration password
20068 q) Quit config
20069 n/s/q> n
20070 name> putio
20071 Type of storage to configure.
20072 Enter a string value. Press Enter for the default ("").
20073 Choose a number from below, or type in your own value
20074 [snip]
20075 XX / Put.io
20076 \ "putio"
20077 [snip]
20078 Storage> putio
20079 ** See help for putio backend at: https://rclone.org/putio/ **
20080 Remote config
20082 Use auto config?
20083 * Say Y if not sure
20084 * Say N if you are working on a remote or headless machine
20085 y) Yes
20086 n) No
20087 y/n> y
20088 If your browser doesn't open automatically go to the following link: http://127.0.0.1:53682/auth
20089 Log in and authorize rclone for access
20090 Waiting for code...
20091 Got code
20092 --------------------
20093 [putio]
20094 type = putio
20095 token = {"access_token":"XXXXXXXX","expiry":"0001-01-01T00:00:00Z"}
20096 --------------------
20097 y) Yes this is OK
20098 e) Edit this remote
20099 d) Delete this remote
20100 y/e/d> y
20101 Current remotes:
20102 Name Type
20104 ==== ====
20105 putio putio
20106 e) Edit existing remote
20108 n) New remote
20109 d) Delete remote
20110 r) Rename remote
20111 c) Copy remote
20112 s) Set configuration password
20113 q) Quit config
20114 e/n/d/r/c/s/q> q
20115 Note that rclone runs a webserver on your local machine to collect the
20117 token as returned from Google if you use auto config mode. This only
20118 runs from the moment it opens your browser to the moment you get back
20119 the verification code. This is on http://127.0.0.1:53682/ and this it
20120 may require you to unblock it temporarily if you are running a host
20121 firewall, or use manual mode.
20122 You can then use it like this,
20124 List directories in top level of your put.io
20126 rclone lsd remote:
20128 List all the files in your put.io
20130 rclone ls remote:
20132 To copy a local directory to a put.io directory called backup
20134 rclone copy /home/source remote:backup
20136 Restricted filename characters
20138 In addition to the default restricted characters set (/overview/#re‐
20139 stricted-characters) the following characters are also replaced:
20140 Character Value Replacement
20142 ────────────────────────────────
20143 \ 0x5C ＼
20144 Invalid UTF-8 bytes will also be replaced (/overview/#invalid-utf8), as
20146 they can’t be used in JSON strings.
20147 Advanced Options
20149 Here are the advanced options specific to putio (Put.io).
20150 –putio-encoding
20152 This sets the encoding for the backend.
20153 See: the encoding section in the overview (/overview/#encoding) for
20155 more info.
20156 · Config: encoding
20158 · Env Var: RCLONE_PUTIO_ENCODING
20160 · Type: MultiEncoder
20162 · Default: Slash,BackSlash,Del,Ctl,InvalidUtf8,Dot
20164 SFTP
20166 SFTP is the Secure (or SSH) File Transfer Protocol
20167 (https://en.wikipedia.org/wiki/SSH_File_Transfer_Protocol).
20168 The SFTP backend can be used with a number of different providers:
20170 · C14
20172 · rsync.net
20174 SFTP runs over SSH v2 and is installed as standard with most modern SSH
20176 installations.
20177 Paths are specified as remote:path. If the path does not begin with a
20179 / it is relative to the home directory of the user. An empty path re‐
20180 mote: refers to the user’s home directory.
20181 "Note that some SFTP servers will need the leading / - Synology is a
20183 good example of this. rsync.net, on the other hand, requires users to
20184 OMIT the leading /.
20185 Here is an example of making an SFTP configuration. First run
20187 rclone config
20189 This will guide you through an interactive setup process.
20191 No remotes found - make a new one
20193 n) New remote
20194 s) Set configuration password
20195 q) Quit config
20196 n/s/q> n
20197 name> remote
20198 Type of storage to configure.
20199 Choose a number from below, or type in your own value
20200 [snip]
20201 XX / SSH/SFTP Connection
20202 \ "sftp"
20203 [snip]
20204 Storage> sftp
20205 SSH host to connect to
20206 Choose a number from below, or type in your own value
20207 1 / Connect to example.com
20208 \ "example.com"
20209 host> example.com
20210 SSH username, leave blank for current username, ncw
20211 user> sftpuser
20212 SSH port, leave blank to use default (22)
20213 port>
20214 SSH password, leave blank to use ssh-agent.
20215 y) Yes type in my own password
20216 g) Generate random password
20217 n) No leave this optional password blank
20218 y/g/n> n
20219 Path to unencrypted PEM-encoded private key file, leave blank to use ssh-agent.
20220 key_file>
20221 Remote config
20222 --------------------
20223 [remote]
20224 host = example.com
20225 user = sftpuser
20226 port =
20227 pass =
20228 key_file =
20229 --------------------
20230 y) Yes this is OK
20231 e) Edit this remote
20232 d) Delete this remote
20233 y/e/d> y
20234 This remote is called remote and can now be used like this:
20236 See all directories in the home directory
20238 rclone lsd remote:
20240 Make a new directory
20242 rclone mkdir remote:path/to/directory
20244 List the contents of a directory
20246 rclone ls remote:path/to/directory
20248 Sync /home/local/directory to the remote directory, deleting any excess
20250 files in the directory.
20251 rclone sync /home/local/directory remote:directory
20253 SSH Authentication
20255 The SFTP remote supports three authentication methods:
20256 · Password
20258 · Key file
20260 · ssh-agent
20262 Key files should be PEM-encoded private key files. For instance
20264 /home/$USER/.ssh/id_rsa. Only unencrypted OpenSSH or PEM encrypted
20265 files are supported.
20266 If you don’t specify pass or key_file then rclone will attempt to con‐
20268 tact an ssh-agent.
20269 You can also specify key_use_agent to force the usage of an ssh-agent.
20271 In this case key_file can also be specified to force the usage of a
20272 specific key in the ssh-agent.
20273 Using an ssh-agent is the only way to load encrypted OpenSSH keys at
20275 the moment.
20276 If you set the --sftp-ask-password option, rclone will prompt for a
20278 password when needed and no password has been configured.
20279 ssh-agent on macOS
20281 Note that there seem to be various problems with using an ssh-agent on
20282 macOS due to recent changes in the OS. The most effective work-around
20283 seems to be to start an ssh-agent in each session, eg
20284 eval `ssh-agent -s` && ssh-add -A
20286 And then at the end of the session
20288 eval `ssh-agent -k`
20290 These commands can be used in scripts of course.
20292 Modified time
20294 Modified times are stored on the server to 1 second precision.
20295 Modified times are used in syncing and are fully supported.
20297 Some SFTP servers disable setting/modifying the file modification time
20299 after upload (for example, certain configurations of ProFTPd with
20300 mod_sftp). If you are using one of these servers, you can set the op‐
20301 tion set_modtime = false in your RClone backend configuration to dis‐
20302 able this behaviour.
20303 Standard Options
20305 Here are the standard options specific to sftp (SSH/SFTP Connection).
20306 –sftp-host
20308 SSH host to connect to
20309 · Config: host
20311 · Env Var: RCLONE_SFTP_HOST
20313 · Type: string
20315 · Default: ""
20317 · Examples:
20319 · “example.com”
20321 · Connect to example.com
20323 –sftp-user
20325 SSH username, leave blank for current username, ncw
20326 · Config: user
20328 · Env Var: RCLONE_SFTP_USER
20330 · Type: string
20332 · Default: ""
20334 –sftp-port
20336 SSH port, leave blank to use default (22)
20337 · Config: port
20339 · Env Var: RCLONE_SFTP_PORT
20341 · Type: string
20343 · Default: ""
20345 –sftp-pass
20347 SSH password, leave blank to use ssh-agent.
20348 · Config: pass
20350 · Env Var: RCLONE_SFTP_PASS
20352 · Type: string
20354 · Default: ""
20356 –sftp-key-file
20358 Path to PEM-encoded private key file, leave blank or set key-use-agent
20359 to use ssh-agent.
20360 · Config: key_file
20362 · Env Var: RCLONE_SFTP_KEY_FILE
20364 · Type: string
20366 · Default: ""
20368 –sftp-key-file-pass
20370 The passphrase to decrypt the PEM-encoded private key file.
20371 Only PEM encrypted key files (old OpenSSH format) are supported. En‐
20373 crypted keys in the new OpenSSH format can’t be used.
20374 · Config: key_file_pass
20376 · Env Var: RCLONE_SFTP_KEY_FILE_PASS
20378 · Type: string
20380 · Default: ""
20382 –sftp-key-use-agent
20384 When set forces the usage of the ssh-agent.
20385 When key-file is also set, the “.pub” file of the specified key-file is
20387 read and only the associated key is requested from the ssh-agent. This
20388 allows to avoid Too many authentication failures for *username* errors
20389 when the ssh-agent contains many keys.
20390 · Config: key_use_agent
20392 · Env Var: RCLONE_SFTP_KEY_USE_AGENT
20394 · Type: bool
20396 · Default: false
20398 –sftp-use-insecure-cipher
20400 Enable the use of insecure ciphers and key exchange methods.
20401 This enables the use of the the following insecure ciphers and key ex‐
20403 change methods:
20404 · aes128-cbc
20406 · aes192-cbc
20408 · aes256-cbc
20410 · 3des-cbc
20412 · diffie-hellman-group-exchange-sha256
20414 · diffie-hellman-group-exchange-sha1
20416 Those algorithms are insecure and may allow plaintext data to be recov‐
20418 ered by an attacker.
20419 · Config: use_insecure_cipher
20421 · Env Var: RCLONE_SFTP_USE_INSECURE_CIPHER
20423 · Type: bool
20425 · Default: false
20427 · Examples:
20429 · “false”
20431 · Use default Cipher list.
20433 · “true”
20435 · Enables the use of the aes128-cbc cipher and diffie-hell‐
20437 man-group-exchange-sha256, diffie-hellman-group-exchange-sha1 key
20438 exchange.
20439 –sftp-disable-hashcheck
20441 Disable the execution of SSH commands to determine if remote file hash‐
20442 ing is available. Leave blank or set to false to enable hashing (rec‐
20443 ommended), set to true to disable hashing.
20444 · Config: disable_hashcheck
20446 · Env Var: RCLONE_SFTP_DISABLE_HASHCHECK
20448 · Type: bool
20450 · Default: false
20452 Advanced Options
20454 Here are the advanced options specific to sftp (SSH/SFTP Connection).
20455 –sftp-ask-password
20457 Allow asking for SFTP password when needed.
20458 If this is set and no password is supplied then rclone will: - ask for
20460 a password - not contact the ssh agent
20461 · Config: ask_password
20463 · Env Var: RCLONE_SFTP_ASK_PASSWORD
20465 · Type: bool
20467 · Default: false
20469 –sftp-path-override
20471 Override path used by SSH connection.
20472 This allows checksum calculation when SFTP and SSH paths are different.
20474 This issue affects among others Synology NAS boxes.
20475 Shared folders can be found in directories representing volumes
20477 rclone sync /home/local/directory remote:/directory --ssh-path-override /volume2/directory
20479 Home directory can be found in a shared folder called “home”
20481 rclone sync /home/local/directory remote:/home/directory --ssh-path-override /volume1/homes/USER/directory
20483 · Config: path_override
20485 · Env Var: RCLONE_SFTP_PATH_OVERRIDE
20487 · Type: string
20489 · Default: ""
20491 –sftp-set-modtime
20493 Set the modified time on the remote if set.
20494 · Config: set_modtime
20496 · Env Var: RCLONE_SFTP_SET_MODTIME
20498 · Type: bool
20500 · Default: true
20502 –sftp-md5sum-command
20504 The command used to read md5 hashes. Leave blank for autodetect.
20505 · Config: md5sum_command
20507 · Env Var: RCLONE_SFTP_MD5SUM_COMMAND
20509 · Type: string
20511 · Default: ""
20513 –sftp-sha1sum-command
20515 The command used to read sha1 hashes. Leave blank for autodetect.
20516 · Config: sha1sum_command
20518 · Env Var: RCLONE_SFTP_SHA1SUM_COMMAND
20520 · Type: string
20522 · Default: ""
20524 –sftp-skip-links
20526 Set to skip any symlinks and any other non regular files.
20527 · Config: skip_links
20529 · Env Var: RCLONE_SFTP_SKIP_LINKS
20531 · Type: bool
20533 · Default: false
20535 Limitations
20537 SFTP supports checksums if the same login has shell access and md5sum
20538 or sha1sum as well as echo are in the remote’s PATH. This remote
20539 checksumming (file hashing) is recommended and enabled by default.
20540 Disabling the checksumming may be required if you are connecting to
20541 SFTP servers which are not under your control, and to which the execu‐
20542 tion of remote commands is prohibited. Set the configuration option
20543 disable_hashcheck to true to disable checksumming.
20544 SFTP also supports about if the same login has shell access and df are
20546 in the remote’s PATH. about will return the total space, free space,
20547 and used space on the remote for the disk of the specified path on the
20548 remote or, if not set, the disk of the root on the remote. about will
20549 fail if it does not have shell access or if df is not in the remote’s
20550 PATH.
20551 Note that some SFTP servers (eg Synology) the paths are different for
20553 SSH and SFTP so the hashes can’t be calculated properly. For them us‐
20554 ing disable_hashcheck is a good idea.
20555 The only ssh agent supported under Windows is Putty’s pageant.
20557 The Go SSH library disables the use of the aes128-cbc cipher by de‐
20559 fault, due to security concerns. This can be re-enabled on a per-con‐
20560 nection basis by setting the use_insecure_cipher setting in the config‐
20561 uration file to true. Further details on the insecurity of this cipher
20562 can be found [in this paper] (http://www.isg.rhul.ac.uk/~kp/SandPfi‐
20563 nal.pdf).
20564 SFTP isn’t supported under plan9 until this issue
20566 (https://github.com/pkg/sftp/issues/156) is fixed.
20567 Note that since SFTP isn’t HTTP based the following flags don’t work
20569 with it: --dump-headers, --dump-bodies, --dump-auth
20570 Note that --timeout isn’t supported (but --contimeout is).
20572 C14
20574 C14 is supported through the SFTP backend.
20575 See C14’s documentation (https://www.online.net/en/storage/c14-cold-
20577 storage)
20578 rsync.net
20580 rsync.net is supported through the SFTP backend.
20581 See rsync.net’s documentation of rclone examples
20583 (https://www.rsync.net/products/rclone.html).
20584 SugarSync
20586 SugarSync (https://sugarsync.com) is a cloud service that enables ac‐
20587 tive synchronization of files across computers and other devices for
20588 file backup, access, syncing, and sharing.
20589 The initial setup for SugarSync involves getting a token from SugarSync
20591 which you can do with rclone. rclone config walks you through it.
20592 Here is an example of how to make a remote called remote. First run:
20594 rclone config
20596 This will guide you through an interactive setup process:
20598 No remotes found - make a new one
20600 n) New remote
20601 s) Set configuration password
20602 q) Quit config
20603 n/s/q> n
20604 name> remote
20605 Type of storage to configure.
20606 Enter a string value. Press Enter for the default ("").
20607 Choose a number from below, or type in your own value
20608 [snip]
20609 XX / Sugarsync
20610 \ "sugarsync"
20611 [snip]
20612 Storage> sugarsync
20613 ** See help for sugarsync backend at: https://rclone.org/sugarsync/ **
20614 Sugarsync App ID.
20616 Leave blank to use rclone's.
20617 Enter a string value. Press Enter for the default ("").
20618 app_id>
20619 Sugarsync Access Key ID.
20620 Leave blank to use rclone's.
20621 Enter a string value. Press Enter for the default ("").
20622 access_key_id>
20623 Sugarsync Private Access Key
20624 Leave blank to use rclone's.
20625 Enter a string value. Press Enter for the default ("").
20626 private_access_key>
20627 Permanently delete files if true
20628 otherwise put them in the deleted files.
20629 Enter a boolean value (true or false). Press Enter for the default ("false").
20630 hard_delete>
20631 Edit advanced config? (y/n)
20632 y) Yes
20633 n) No (default)
20634 y/n> n
20635 Remote config
20636 Username (email address)> nick@craig-wood.com
20637 Your Sugarsync password is only required during setup and will not be stored.
20638 password:
20639 --------------------
20640 [remote]
20641 type = sugarsync
20642 refresh_token = https://api.sugarsync.com/app-authorization/XXXXXXXXXXXXXXXXXX
20643 --------------------
20644 y) Yes this is OK (default)
20645 e) Edit this remote
20646 d) Delete this remote
20647 y/e/d> y
20648 Note that the config asks for your email and password but doesn’t store
20650 them, it only uses them to get the initial token.
20651 Once configured you can then use rclone like this,
20653 List directories (sync folders) in top level of your SugarSync
20655 rclone lsd remote:
20657 List all the files in your SugarSync folder “Test”
20659 rclone ls remote:Test
20661 To copy a local directory to an SugarSync folder called backup
20663 rclone copy /home/source remote:backup
20665 Paths are specified as remote:path
20667 Paths may be as deep as required, eg remote:directory/subdirectory.
20669 NB you can’t create files in the top level folder you have to create a
20671 folder, which rclone will create as a “Sync Folder” with SugarSync.
20672 Modified time and hashes
20674 SugarSync does not support modification times or hashes, therefore
20675 syncing will default to --size-only checking. Note that using --update
20676 will work as rclone can read the time files were uploaded.
20677 Restricted filename characters
20679 SugarSync replaces the default restricted characters set (/over‐
20680 view/#restricted-characters) except for DEL.
20681 Invalid UTF-8 bytes will also be replaced (/overview/#invalid-utf8), as
20683 they can’t be used in XML strings.
20684 Deleting files
20686 Deleted files will be moved to the “Deleted items” folder by default.
20687 However you can supply the flag --sugarsync-hard-delete or set the con‐
20689 fig parameter hard_delete = true if you would like files to be deleted
20690 straight away.
20691 Standard Options
20693 Here are the standard options specific to sugarsync (Sugarsync).
20694 –sugarsync-app-id
20696 Sugarsync App ID.
20697 Leave blank to use rclone’s.
20699 · Config: app_id
20701 · Env Var: RCLONE_SUGARSYNC_APP_ID
20703 · Type: string
20705 · Default: ""
20707 –sugarsync-access-key-id
20709 Sugarsync Access Key ID.
20710 Leave blank to use rclone’s.
20712 · Config: access_key_id
20714 · Env Var: RCLONE_SUGARSYNC_ACCESS_KEY_ID
20716 · Type: string
20718 · Default: ""
20720 –sugarsync-private-access-key
20722 Sugarsync Private Access Key
20723 Leave blank to use rclone’s.
20725 · Config: private_access_key
20727 · Env Var: RCLONE_SUGARSYNC_PRIVATE_ACCESS_KEY
20729 · Type: string
20731 · Default: ""
20733 –sugarsync-hard-delete
20735 Permanently delete files if true otherwise put them in the deleted
20736 files.
20737 · Config: hard_delete
20739 · Env Var: RCLONE_SUGARSYNC_HARD_DELETE
20741 · Type: bool
20743 · Default: false
20745 Advanced Options
20747 Here are the advanced options specific to sugarsync (Sugarsync).
20748 –sugarsync-refresh-token
20750 Sugarsync refresh token
20751 Leave blank normally, will be auto configured by rclone.
20753 · Config: refresh_token
20755 · Env Var: RCLONE_SUGARSYNC_REFRESH_TOKEN
20757 · Type: string
20759 · Default: ""
20761 –sugarsync-authorization
20763 Sugarsync authorization
20764 Leave blank normally, will be auto configured by rclone.
20766 · Config: authorization
20768 · Env Var: RCLONE_SUGARSYNC_AUTHORIZATION
20770 · Type: string
20772 · Default: ""
20774 –sugarsync-authorization-expiry
20776 Sugarsync authorization expiry
20777 Leave blank normally, will be auto configured by rclone.
20779 · Config: authorization_expiry
20781 · Env Var: RCLONE_SUGARSYNC_AUTHORIZATION_EXPIRY
20783 · Type: string
20785 · Default: ""
20787 –sugarsync-user
20789 Sugarsync user
20790 Leave blank normally, will be auto configured by rclone.
20792 · Config: user
20794 · Env Var: RCLONE_SUGARSYNC_USER
20796 · Type: string
20798 · Default: ""
20800 –sugarsync-root-id
20802 Sugarsync root id
20803 Leave blank normally, will be auto configured by rclone.
20805 · Config: root_id
20807 · Env Var: RCLONE_SUGARSYNC_ROOT_ID
20809 · Type: string
20811 · Default: ""
20813 –sugarsync-deleted-id
20815 Sugarsync deleted folder id
20816 Leave blank normally, will be auto configured by rclone.
20818 · Config: deleted_id
20820 · Env Var: RCLONE_SUGARSYNC_DELETED_ID
20822 · Type: string
20824 · Default: ""
20826 –sugarsync-encoding
20828 This sets the encoding for the backend.
20829 See: the encoding section in the overview (/overview/#encoding) for
20831 more info.
20832 · Config: encoding
20834 · Env Var: RCLONE_SUGARSYNC_ENCODING
20836 · Type: MultiEncoder
20838 · Default: Slash,Ctl,InvalidUtf8,Dot
20840 Union
20842 The union remote provides a unification similar to UnionFS using other
20843 remotes.
20844 Paths may be as deep as required or a local path, eg remote:directo‐
20846 ry/subdirectory or /directory/subdirectory.
20847 During the initial setup with rclone config you will specify the target
20849 remotes as a space separated list. The target remotes can either be a
20850 local paths or other remotes.
20851 The order of the remotes is important as it defines which remotes take
20853 precedence over others if there are files with the same name in the
20854 same logical path. The last remote is the topmost remote and replaces
20855 files with the same name from previous remotes.
20856 Only the last remote is used to write to and delete from, all other re‐
20858 motes are read-only.
20859 Subfolders can be used in target remote. Assume a union remote named
20861 backup with the remotes mydrive:private/backup mydrive2:/backup. In‐
20862 voking rclone mkdir backup:desktop is exactly the same as invoking
20863 rclone mkdir mydrive2:/backup/desktop.
20864 There will be no special handling of paths containing .. segments.
20866 Invoking rclone mkdir backup:../desktop is exactly the same as invoking
20867 rclone mkdir mydrive2:/backup/../desktop.
20868 Here is an example of how to make a union called remote for local fold‐
20870 ers. First run:
20871 rclone config
20873 This will guide you through an interactive setup process:
20875 No remotes found - make a new one
20877 n) New remote
20878 s) Set configuration password
20879 q) Quit config
20880 n/s/q> n
20881 name> remote
20882 Type of storage to configure.
20883 Choose a number from below, or type in your own value
20884 [snip]
20885 XX / Union merges the contents of several remotes
20886 \ "union"
20887 [snip]
20888 Storage> union
20889 List of space separated remotes.
20890 Can be 'remotea:test/dir remoteb:', '"remotea:test/space dir" remoteb:', etc.
20891 The last remote is used to write to.
20892 Enter a string value. Press Enter for the default ("").
20893 remotes>
20894 Remote config
20895 --------------------
20896 [remote]
20897 type = union
20898 remotes = C:\dir1 C:\dir2 C:\dir3
20899 --------------------
20900 y) Yes this is OK
20901 e) Edit this remote
20902 d) Delete this remote
20903 y/e/d> y
20904 Current remotes:
20905 Name Type
20907 ==== ====
20908 remote union
20909 e) Edit existing remote
20911 n) New remote
20912 d) Delete remote
20913 r) Rename remote
20914 c) Copy remote
20915 s) Set configuration password
20916 q) Quit config
20917 e/n/d/r/c/s/q> q
20918 Once configured you can then use rclone like this,
20920 List directories in top level in C:\dir1, C:\dir2 and C:\dir3
20922 rclone lsd remote:
20924 List all the files in C:\dir1, C:\dir2 and C:\dir3
20926 rclone ls remote:
20928 Copy another local directory to the union directory called source,
20930 which will be placed into C:\dir3
20931 rclone copy C:\source remote:source
20933 Standard Options
20935 Here are the standard options specific to union (Union merges the con‐
20936 tents of several remotes).
20937 –union-remotes
20939 List of space separated remotes. Can be `remotea:test/dir remoteb:',
20940 `“remotea:test/space dir” remoteb:', etc. The last remote is used to
20941 write to.
20942 · Config: remotes
20944 · Env Var: RCLONE_UNION_REMOTES
20946 · Type: string
20948 · Default: ""
20950 WebDAV
20952 Paths are specified as remote:path
20953 Paths may be as deep as required, eg remote:directory/subdirectory.
20955 To configure the WebDAV remote you will need to have a URL for it, and
20957 a username and password. If you know what kind of system you are con‐
20958 necting to then rclone can enable extra features.
20959 Here is an example of how to make a remote called remote. First run:
20961 rclone config
20963 This will guide you through an interactive setup process:
20965 No remotes found - make a new one
20967 n) New remote
20968 s) Set configuration password
20969 q) Quit config
20970 n/s/q> n
20971 name> remote
20972 Type of storage to configure.
20973 Choose a number from below, or type in your own value
20974 [snip]
20975 XX / Webdav
20976 \ "webdav"
20977 [snip]
20978 Storage> webdav
20979 URL of http host to connect to
20980 Choose a number from below, or type in your own value
20981 1 / Connect to example.com
20982 \ "https://example.com"
20983 url> https://example.com/remote.php/webdav/
20984 Name of the Webdav site/service/software you are using
20985 Choose a number from below, or type in your own value
20986 1 / Nextcloud
20987 \ "nextcloud"
20988 2 / Owncloud
20989 \ "owncloud"
20990 3 / Sharepoint
20991 \ "sharepoint"
20992 4 / Other site/service or software
20993 \ "other"
20994 vendor> 1
20995 User name
20996 user> user
20997 Password.
20998 y) Yes type in my own password
20999 g) Generate random password
21000 n) No leave this optional password blank
21001 y/g/n> y
21002 Enter the password:
21003 password:
21004 Confirm the password:
21005 password:
21006 Bearer token instead of user/pass (eg a Macaroon)
21007 bearer_token>
21008 Remote config
21009 --------------------
21010 [remote]
21011 type = webdav
21012 url = https://example.com/remote.php/webdav/
21013 vendor = nextcloud
21014 user = user
21015 pass = *** ENCRYPTED ***
21016 bearer_token =
21017 --------------------
21018 y) Yes this is OK
21019 e) Edit this remote
21020 d) Delete this remote
21021 y/e/d> y
21022 Once configured you can then use rclone like this,
21024 List directories in top level of your WebDAV
21026 rclone lsd remote:
21028 List all the files in your WebDAV
21030 rclone ls remote:
21032 To copy a local directory to an WebDAV directory called backup
21034 rclone copy /home/source remote:backup
21036 Modified time and hashes
21038 Plain WebDAV does not support modified times. However when used with
21039 Owncloud or Nextcloud rclone will support modified times.
21040 Likewise plain WebDAV does not support hashes, however when used with
21042 Owncloud or Nextcloud rclone will support SHA1 and MD5 hashes. Depend‐
21043 ing on the exact version of Owncloud or Nextcloud hashes may appear on
21044 all objects, or only on objects which had a hash uploaded with them.
21045 Standard Options
21047 Here are the standard options specific to webdav (Webdav).
21048 –webdav-url
21050 URL of http host to connect to
21051 · Config: url
21053 · Env Var: RCLONE_WEBDAV_URL
21055 · Type: string
21057 · Default: ""
21059 · Examples:
21061 · “https://example.com”
21063 · Connect to example.com
21065 –webdav-vendor
21067 Name of the Webdav site/service/software you are using
21068 · Config: vendor
21070 · Env Var: RCLONE_WEBDAV_VENDOR
21072 · Type: string
21074 · Default: ""
21076 · Examples:
21078 · “nextcloud”
21080 · Nextcloud
21082 · “owncloud”
21084 · Owncloud
21086 · “sharepoint”
21088 · Sharepoint
21090 · “other”
21092 · Other site/service or software
21094 –webdav-user
21096 User name
21097 · Config: user
21099 · Env Var: RCLONE_WEBDAV_USER
21101 · Type: string
21103 · Default: ""
21105 –webdav-pass
21107 Password.
21108 · Config: pass
21110 · Env Var: RCLONE_WEBDAV_PASS
21112 · Type: string
21114 · Default: ""
21116 –webdav-bearer-token
21118 Bearer token instead of user/pass (eg a Macaroon)
21119 · Config: bearer_token
21121 · Env Var: RCLONE_WEBDAV_BEARER_TOKEN
21123 · Type: string
21125 · Default: ""
21127 Advanced Options
21129 Here are the advanced options specific to webdav (Webdav).
21130 –webdav-bearer-token-command
21132 Command to run to get a bearer token
21133 · Config: bearer_token_command
21135 · Env Var: RCLONE_WEBDAV_BEARER_TOKEN_COMMAND
21137 · Type: string
21139 · Default: ""
21141 Provider notes
21143 See below for notes on specific providers.
21144 Owncloud
21146 Click on the settings cog in the bottom right of the page and this will
21147 show the WebDAV URL that rclone needs in the config step. It will look
21148 something like https://example.com/remote.php/webdav/.
21149 Owncloud supports modified times using the X-OC-Mtime header.
21151 Nextcloud
21153 This is configured in an identical way to Owncloud. Note that
21154 Nextcloud does not support streaming of files (rcat) whereas Owncloud
21155 does. This may be fixed (https://github.com/nextcloud/nextcloud-
21156 snap/issues/365) in the future.
21157 Sharepoint
21159 Rclone can be used with Sharepoint provided by OneDrive for Business or
21160 Office365 Education Accounts. This feature is only needed for a few of
21161 these Accounts, mostly Office365 Education ones. These accounts are
21162 sometimes not verified by the domain owner github#1975
21163 (https://github.com/rclone/rclone/issues/1975)
21164 This means that these accounts can’t be added using the official API
21166 (other Accounts should work with the “onedrive” option). However, it
21167 is possible to access them using webdav.
21168 To use a sharepoint remote with rclone, add it like this: First, you
21170 need to get your remote’s URL:
21171 · Go here (https://onedrive.live.com/about/en-us/signin/) to open your
21173 OneDrive or to sign in
21174 · Now take a look at your address bar, the URL should look like this:
21176 https://[YOUR-DOMAIN]-my.sharepoint.com/personal/[YOUR-EMAIL]/_lay‐
21177 outs/15/onedrive.aspx
21178 You’ll only need this URL upto the email address. After that, you’ll
21180 most likely want to add “/Documents”. That subdirectory contains the
21181 actual data stored on your OneDrive.
21182 Add the remote to rclone like this: Configure the url as
21184 https://[YOUR-DOMAIN]-my.sharepoint.com/personal/[YOUR-EMAIL]/Documents
21185 and use your normal account email and password for user and pass. If
21186 you have 2FA enabled, you have to generate an app password. Set the
21187 vendor to sharepoint.
21188 Your config file should look like this:
21190 [sharepoint]
21192 type = webdav
21193 url = https://[YOUR-DOMAIN]-my.sharepoint.com/personal/[YOUR-EMAIL]/Documents
21194 vendor = other
21195 user = YourEmailAddress
21196 pass = encryptedpassword
21197 Required Flags for SharePoint
21199 As SharePoint does some special things with uploaded documents, you
21200 won’t be able to use the documents size or the documents hash to com‐
21201 pare if a file has been changed since the upload / which file is newer.
21202 For Rclone calls copying files (especially Office files such as .docx,
21204 .xlsx, etc.) from/to SharePoint (like copy, sync, etc.), you should ap‐
21205 pend these flags to ensure Rclone uses the “Last Modified” datetime
21206 property to compare your documents:
21207 --ignore-size --ignore-checksum --update
21209 dCache
21211 dCache is a storage system that supports many protocols and authentica‐
21212 tion/authorisation schemes. For WebDAV clients, it allows users to au‐
21213 thenticate with username and password (BASIC), X.509, Kerberos, and
21214 various bearer tokens, including Macaroons (https://www.dcache.org/man‐
21215 uals/workshop-2017-05-29-Umea/000-Final/anupam_macaroons_v02.pdf) and
21216 OpenID-Connect (https://en.wikipedia.org/wiki/OpenID_Connect) access
21217 tokens.
21218 Configure as normal using the other type. Don’t enter a username or
21220 password, instead enter your Macaroon as the bearer_token.
21221 The config will end up looking something like this.
21223 [dcache]
21225 type = webdav
21226 url = https://dcache...
21227 vendor = other
21228 user =
21229 pass =
21230 bearer_token = your-macaroon
21231 There is a script (https://github.com/sara-nl/GridScripts/blob/mas‐
21233 ter/get-macaroon) that obtains a Macaroon from a dCache WebDAV end‐
21234 point, and creates an rclone config file.
21235 Macaroons may also be obtained from the dCacheView web-brows‐
21237 er/JavaScript client that comes with dCache.
21238 OpenID-Connect
21240 dCache also supports authenticating with OpenID-Connect access tokens.
21241 OpenID-Connect is a protocol (based on OAuth 2.0) that allows services
21242 to identify users who have authenticated with some central service.
21243 Support for OpenID-Connect in rclone is currently achieved using anoth‐
21245 er software package called oidc-agent (https://github.com/indigo-
21246 dc/oidc-agent). This is a command-line tool that facilitates obtaining
21247 an access token. Once installed and configured, an access token is ob‐
21248 tained by running the oidc-token command. The following example shows
21249 a (shortened) access token obtained from the XDC OIDC Provider.
21250 paul@celebrimbor:~$ oidc-token XDC
21252 eyJraWQ[...]QFXDt0
21253 paul@celebrimbor:~$
21254 Note Before the oidc-token command will work, the refresh token must be
21256 loaded into the oidc agent. This is done with the oidc-add command
21257 (e.g., oidc-add XDC). This is typically done once per login session.
21258 Full details on this and how to register oidc-agent with your OIDC
21259 Provider are provided in the oidc-agent documentation (https://indigo-
21260 dc.gitbooks.io/oidc-agent/).
21261 The rclone bearer_token_command configuration option is used to fetch
21263 the access token from oidc-agent.
21264 Configure as a normal WebDAV endpoint, using the `other' vendor, leav‐
21266 ing the username and password empty. When prompted, choose to edit the
21267 advanced config and enter the command to get a bearer token (e.g.,
21268 oidc-agent XDC).
21269 The following example config shows a WebDAV endpoint that uses
21271 oidc-agent to supply an access token from the XDC OIDC Provider.
21272 [dcache]
21274 type = webdav
21275 url = https://dcache.example.org/
21276 vendor = other
21277 bearer_token_command = oidc-token XDC
21278 Yandex Disk
21280 Yandex Disk (https://disk.yandex.com) is a cloud storage solution cre‐
21281 ated by Yandex (https://yandex.com).
21282 Here is an example of making a yandex configuration. First run
21284 rclone config
21286 This will guide you through an interactive setup process:
21288 No remotes found - make a new one
21290 n) New remote
21291 s) Set configuration password
21292 n/s> n
21293 name> remote
21294 Type of storage to configure.
21295 Choose a number from below, or type in your own value
21296 [snip]
21297 XX / Yandex Disk
21298 \ "yandex"
21299 [snip]
21300 Storage> yandex
21301 Yandex Client Id - leave blank normally.
21302 client_id>
21303 Yandex Client Secret - leave blank normally.
21304 client_secret>
21305 Remote config
21306 Use auto config?
21307 * Say Y if not sure
21308 * Say N if you are working on a remote or headless machine
21309 y) Yes
21310 n) No
21311 y/n> y
21312 If your browser doesn't open automatically go to the following link: http://127.0.0.1:53682/auth
21313 Log in and authorize rclone for access
21314 Waiting for code...
21315 Got code
21316 --------------------
21317 [remote]
21318 client_id =
21319 client_secret =
21320 token = {"access_token":"xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx","token_type":"bearer","expiry":"2016-12-29T12:27:11.362788025Z"}
21321 --------------------
21322 y) Yes this is OK
21323 e) Edit this remote
21324 d) Delete this remote
21325 y/e/d> y
21326 See the remote setup docs (https://rclone.org/remote_setup/) for how to
21328 set it up on a machine with no Internet browser available.
21329 Note that rclone runs a webserver on your local machine to collect the
21331 token as returned from Yandex Disk. This only runs from the moment it
21332 opens your browser to the moment you get back the verification code.
21333 This is on http://127.0.0.1:53682/ and this it may require you to un‐
21334 block it temporarily if you are running a host firewall.
21335 Once configured you can then use rclone like this,
21337 See top level directories
21339 rclone lsd remote:
21341 Make a new directory
21343 rclone mkdir remote:directory
21345 List the contents of a directory
21347 rclone ls remote:directory
21349 Sync /home/local/directory to the remote path, deleting any excess
21351 files in the path.
21352 rclone sync /home/local/directory remote:directory
21354 Yandex paths may be as deep as required, eg remote:directory/subdirec‐
21356 tory.
21357 Modified time
21359 Modified times are supported and are stored accurate to 1 ns in custom
21360 metadata called rclone_modified in RFC3339 with nanoseconds format.
21361 MD5 checksums
21363 MD5 checksums are natively supported by Yandex Disk.
21364 Emptying Trash
21366 If you wish to empty your trash you can use the rclone cleanup remote:
21367 command which will permanently delete all your trashed files. This
21368 command does not take any path arguments.
21369 Quota information
21371 To view your current quota you can use the rclone about remote: command
21372 which will display your usage limit (quota) and the current usage.
21373 Restricted filename characters
21375 The default restricted characters set (/overview/#restricted-charac‐
21376 ters) are replaced.
21377 Invalid UTF-8 bytes will also be replaced (/overview/#invalid-utf8), as
21379 they can’t be used in JSON strings.
21380 Limitations
21382 When uploading very large files (bigger than about 5GB) you will need
21383 to increase the --timeout parameter. This is because Yandex pauses
21384 (perhaps to calculate the MD5SUM for the entire file) before returning
21385 confirmation that the file has been uploaded. The default handling of
21386 timeouts in rclone is to assume a 5 minute pause is an error and close
21387 the connection - you’ll see net/http: timeout awaiting response headers
21388 errors in the logs if this is happening. Setting the timeout to twice
21389 the max size of file in GB should be enough, so if you want to upload a
21390 30GB file set a timeout of 2 * 30 = 60m, that is --timeout 60m.
21391 Standard Options
21393 Here are the standard options specific to yandex (Yandex Disk).
21394 –yandex-client-id
21396 Yandex Client Id Leave blank normally.
21397 · Config: client_id
21399 · Env Var: RCLONE_YANDEX_CLIENT_ID
21401 · Type: string
21403 · Default: ""
21405 –yandex-client-secret
21407 Yandex Client Secret Leave blank normally.
21408 · Config: client_secret
21410 · Env Var: RCLONE_YANDEX_CLIENT_SECRET
21412 · Type: string
21414 · Default: ""
21416 Advanced Options
21418 Here are the advanced options specific to yandex (Yandex Disk).
21419 –yandex-unlink
21421 Remove existing public link to file/folder with link command rather
21422 than creating. Default is false, meaning link command will create or
21423 retrieve public link.
21424 · Config: unlink
21426 · Env Var: RCLONE_YANDEX_UNLINK
21428 · Type: bool
21430 · Default: false
21432 –yandex-encoding
21434 This sets the encoding for the backend.
21435 See: the encoding section in the overview (/overview/#encoding) for
21437 more info.
21438 · Config: encoding
21440 · Env Var: RCLONE_YANDEX_ENCODING
21442 · Type: MultiEncoder
21444 · Default: Slash,Del,Ctl,InvalidUtf8,Dot
21446 Local Filesystem
21448 Local paths are specified as normal filesystem paths, eg /path/to/wher‐
21449 ever, so
21450 rclone sync /home/source /tmp/destination
21452 Will sync /home/source to /tmp/destination
21454 These can be configured into the config file for consistencies sake,
21456 but it is probably easier not to.
21457 Modified time
21459 Rclone reads and writes the modified time using an accuracy determined
21460 by the OS. Typically this is 1ns on Linux, 10 ns on Windows and 1 Sec‐
21461 ond on OS X.
21462 Filenames
21464 Filenames should be encoded in UTF-8 on disk. This is the normal case
21465 for Windows and OS X.
21466 There is a bit more uncertainty in the Linux world, but new distribu‐
21468 tions will have UTF-8 encoded files names. If you are using an old
21469 Linux filesystem with non UTF-8 file names (eg latin1) then you can use
21470 the convmv tool to convert the filesystem to UTF-8. This tool is
21471 available in most distributions’ package managers.
21472 If an invalid (non-UTF8) filename is read, the invalid characters will
21474 be replaced with a quoted representation of the invalid bytes. The
21475 name gro\xdf will be transferred as gro‛DF. rclone will emit a debug
21476 message in this case (use -v to see), eg
21477 Local file system at .: Replacing invalid UTF-8 characters in "gro\xdf"
21479 Restricted characters
21481 On non Windows platforms the following characters are replaced when
21482 handling file names.
21483 Character Value Replacement
21485 ────────────────────────────────
21486 NUL 0x00 ␀
21487 / 0x2F ／
21488 When running on Windows the following characters are replaced. This
21490 list is based on the Windows file naming conventions (https://docs.mi‐
21491 crosoft.com/de-de/windows/desktop/FileIO/naming-a-file#naming-conven‐
21492 tions).
21493 Character Value Replacement
21495 ────────────────────────────────
21496 NUL 0x00 ␀
21497 SOH 0x01 ␁
21498 STX 0x02 ␂
21499 ETX 0x03 ␃
21500 EOT 0x04 ␄
21501 ENQ 0x05 ␅
21502 ACK 0x06 ␆
21503 BEL 0x07 ␇
21504 BS 0x08 ␈
21505 HT 0x09 ␉
21506 LF 0x0A ␊
21507 VT 0x0B ␋
21508 FF 0x0C ␌
21509 CR 0x0D ␍
21510 SO 0x0E ␎
21511 SI 0x0F ␏
21512 DLE 0x10 ␐
21513 DC1 0x11 ␑
21514 DC2 0x12 ␒
21515 DC3 0x13 ␓
21516 DC4 0x14 ␔
21517 NAK 0x15 ␕
21518 SYN 0x16 ␖
21519 ETB 0x17 ␗
21520 CAN 0x18 ␘
21521 EM 0x19 ␙
21522 SUB 0x1A ␚
21523 ESC 0x1B ␛
21524 FS 0x1C ␜
21525 GS 0x1D ␝
21526 RS 0x1E ␞
21528 US 0x1F ␟
21529 / 0x2F ／
21530 " 0x22 ＂
21531 * 0x2A ＊
21532 : 0x3A ：
21533 < 0x3C ＜
21534 > 0x3E ＞
21535 ? 0x3F ？
21536 \ 0x5C ＼
21537 | 0x7C ｜
21538 File names on Windows can also not end with the following characters.
21540 These only get replaced if they are last character in the name:
21541 Character Value Replacement
21543 ────────────────────────────────
21544 SP 0x20 ␠
21545 . 0x2E ．
21546 Invalid UTF-8 bytes will also be replaced (/overview/#invalid-utf8), as
21548 they can’t be converted to UTF-16.
21549 Long paths on Windows
21551 Rclone handles long paths automatically, by converting all paths to
21552 long UNC paths (https://msdn.microsoft.com/en-us/library/windows/desk‐
21553 top/aa365247(v=vs.85).aspx#maxpath) which allows paths up to 32,767
21554 characters.
21555 This is why you will see that your paths, for instance c:\files is con‐
21557 verted to the UNC path \\?\c:\files in the output, and \\server\share
21558 is converted to \\?\UNC\server\share.
21559 However, in rare cases this may cause problems with buggy file system
21561 drivers like EncFS (https://github.com/rclone/rclone/issues/261). To
21562 disable UNC conversion globally, add this to your .rclone.conf file:
21563 [local]
21565 nounc = true
21566 If you want to selectively disable UNC, you can add it to a separate
21568 entry like this:
21569 [nounc]
21571 type = local
21572 nounc = true
21573 And use rclone like this:
21575 rclone copy c:\src nounc:z:\dst
21577 This will use UNC paths on c:\src but not on z:\dst. Of course this
21579 will cause problems if the absolute path length of a file exceeds 258
21580 characters on z, so only use this option if you have to.
21581 Symlinks / Junction points
21583 Normally rclone will ignore symlinks or junction points (which behave
21584 like symlinks under Windows).
21585 If you supply --copy-links or -L then rclone will follow the symlink
21587 and copy the pointed to file or directory. Note that this flag is in‐
21588 compatible with -links / -l.
21589 This flag applies to all commands.
21591 For example, supposing you have a directory structure like this
21593 $ tree /tmp/a
21595 /tmp/a
21596 ├── b -> ../b
21597 ├── expected -> ../expected
21598 ├── one
21599 └── two
21600 └── three
21601 Then you can see the difference with and without the flag like this
21603 $ rclone ls /tmp/a
21605 6 one
21606 6 two/three
21607 and
21609 $ rclone -L ls /tmp/a
21611 4174 expected
21612 6 one
21613 6 two/three
21614 6 b/two
21615 6 b/one
21616 –links, -l
21618 Normally rclone will ignore symlinks or junction points (which behave
21619 like symlinks under Windows).
21620 If you supply this flag then rclone will copy symbolic links from the
21622 local storage, and store them as text files, with a `.rclonelink' suf‐
21623 fix in the remote storage.
21624 The text file will contain the target of the symbolic link (see exam‐
21626 ple).
21627 This flag applies to all commands.
21629 For example, supposing you have a directory structure like this
21631 $ tree /tmp/a
21633 /tmp/a
21634 ├── file1 -> ./file4
21635 └── file2 -> /home/user/file3
21636 Copying the entire directory with `-l'
21638 $ rclone copyto -l /tmp/a/file1 remote:/tmp/a/
21640 The remote files are created with a `.rclonelink' suffix
21642 $ rclone ls remote:/tmp/a
21644 5 file1.rclonelink
21645 14 file2.rclonelink
21646 The remote files will contain the target of the symbolic links
21648 $ rclone cat remote:/tmp/a/file1.rclonelink
21650 ./file4
21651 $ rclone cat remote:/tmp/a/file2.rclonelink
21653 /home/user/file3
21654 Copying them back with `-l'
21656 $ rclone copyto -l remote:/tmp/a/ /tmp/b/
21658 $ tree /tmp/b
21660 /tmp/b
21661 ├── file1 -> ./file4
21662 └── file2 -> /home/user/file3
21663 However, if copied back without `-l'
21665 $ rclone copyto remote:/tmp/a/ /tmp/b/
21667 $ tree /tmp/b
21669 /tmp/b
21670 ├── file1.rclonelink
21671 └── file2.rclonelink
21672 Note that this flag is incompatible with -copy-links / -L.
21674 Restricting filesystems with –one-file-system
21676 Normally rclone will recurse through filesystems as mounted.
21677 However if you set --one-file-system or -x this tells rclone to stay in
21679 the filesystem specified by the root and not to recurse into different
21680 file systems.
21681 For example if you have a directory hierarchy like this
21683 root
21685 ├── disk1 - disk1 mounted on the root
21686 │ └── file3 - stored on disk1
21687 ├── disk2 - disk2 mounted on the root
21688 │ └── file4 - stored on disk12
21689 ├── file1 - stored on the root disk
21690 └── file2 - stored on the root disk
21691 Using rclone --one-file-system copy root remote: will only copy file1
21693 and file2. Eg
21694 $ rclone -q --one-file-system ls root
21696 0 file1
21697 0 file2
21698 $ rclone -q ls root
21700 0 disk1/file3
21701 0 disk2/file4
21702 0 file1
21703 0 file2
21704 NB Rclone (like most unix tools such as du, rsync and tar) treats a
21706 bind mount to the same device as being on the same filesystem.
21707 NB This flag is only available on Unix based systems. On systems where
21709 it isn’t supported (eg Windows) it will be ignored.
21710 Standard Options
21712 Here are the standard options specific to local (Local Disk).
21713 –local-nounc
21715 Disable UNC (long path names) conversion on Windows
21716 · Config: nounc
21718 · Env Var: RCLONE_LOCAL_NOUNC
21720 · Type: string
21722 · Default: ""
21724 · Examples:
21726 · “true”
21728 · Disables long file names
21730 Advanced Options
21732 Here are the advanced options specific to local (Local Disk).
21733 –copy-links / -L
21735 Follow symlinks and copy the pointed to item.
21736 · Config: copy_links
21738 · Env Var: RCLONE_LOCAL_COPY_LINKS
21740 · Type: bool
21742 · Default: false
21744 –links / -l
21746 Translate symlinks to/from regular files with a `.rclonelink' extension
21747 · Config: links
21749 · Env Var: RCLONE_LOCAL_LINKS
21751 · Type: bool
21753 · Default: false
21755 –skip-links
21757 Don’t warn about skipped symlinks. This flag disables warning messages
21758 on skipped symlinks or junction points, as you explicitly acknowledge
21759 that they should be skipped.
21760 · Config: skip_links
21762 · Env Var: RCLONE_LOCAL_SKIP_LINKS
21764 · Type: bool
21766 · Default: false
21768 –local-no-unicode-normalization
21770 Don’t apply unicode normalization to paths and filenames (Deprecated)
21771 This flag is deprecated now. Rclone no longer normalizes unicode file
21773 names, but it compares them with unicode normalization in the sync rou‐
21774 tine instead.
21775 · Config: no_unicode_normalization
21777 · Env Var: RCLONE_LOCAL_NO_UNICODE_NORMALIZATION
21779 · Type: bool
21781 · Default: false
21783 –local-no-check-updated
21785 Don’t check to see if the files change during upload
21786 Normally rclone checks the size and modification time of files as they
21788 are being uploaded and aborts with a message which starts “can’t copy -
21789 source file is being updated” if the file changes during upload.
21790 However on some file systems this modification time check may fail (eg
21792 Glusterfs #2206 (https://github.com/rclone/rclone/issues/2206)) so this
21793 check can be disabled with this flag.
21794 · Config: no_check_updated
21796 · Env Var: RCLONE_LOCAL_NO_CHECK_UPDATED
21798 · Type: bool
21800 · Default: false
21802 –one-file-system / -x
21804 Don’t cross filesystem boundaries (unix/macOS only).
21805 · Config: one_file_system
21807 · Env Var: RCLONE_LOCAL_ONE_FILE_SYSTEM
21809 · Type: bool
21811 · Default: false
21813 –local-case-sensitive
21815 Force the filesystem to report itself as case sensitive.
21816 Normally the local backend declares itself as case insensitive on Win‐
21818 dows/macOS and case sensitive for everything else. Use this flag to
21819 override the default choice.
21820 · Config: case_sensitive
21822 · Env Var: RCLONE_LOCAL_CASE_SENSITIVE
21824 · Type: bool
21826 · Default: false
21828 –local-case-insensitive
21830 Force the filesystem to report itself as case insensitive
21831 Normally the local backend declares itself as case insensitive on Win‐
21833 dows/macOS and case sensitive for everything else. Use this flag to
21834 override the default choice.
21835 · Config: case_insensitive
21837 · Env Var: RCLONE_LOCAL_CASE_INSENSITIVE
21839 · Type: bool
21841 · Default: false
21843 –local-encoding
21845 This sets the encoding for the backend.
21846 See: the encoding section in the overview (/overview/#encoding) for
21848 more info.
21849 · Config: encoding
21851 · Env Var: RCLONE_LOCAL_ENCODING
21853 · Type: MultiEncoder
21855 · Default: Slash,Dot
21857
21859 v1.51.0 - 2020-02-01
21860 · New backends
21861 · Memory (/memory) (Nick Craig-Wood)
21863 · Sugarsync (/sugarsync) (Nick Craig-Wood)
21865 · New Features
21867 · Adjust all backends to have --backend-encoding parameter (Nick
21869 Craig-Wood)
21870 · this enables the encoding for special characters to be adjusted
21872 or disabled
21873 · Add --max-duration flag to control the maximum duration of a trans‐
21875 fer session (boosh)
21876 · Add --expect-continue-timeout flag, default 1s (Nick Craig-Wood)
21878 · Add --no-check-dest flag for copying without testing the destina‐
21880 tion (Nick Craig-Wood)
21881 · Implement --order-by flag to order transfers (Nick Craig-Wood)
21883 · accounting
21885 · Don’t show entries in both transferring and checking (Nick
21887 Craig-Wood)
21888 · Add option to delete stats (Aleksandar Jankovic)
21890 · build
21892 · Compress the test builds with gzip (Nick Craig-Wood)
21894 · Implement a framework for starting test servers during tests
21896 (Nick Craig-Wood)
21897 · cmd: Always print elapsed time to tenth place seconds in progress
21899 (Gary Kim)
21900 · config
21902 · Add --password-command to allow dynamic config password (Damon
21904 Permezel)
21905 · Give config questions default values (Nick Craig-Wood)
21907 · Check a remote exists when creating a new one (Nick Craig-Wood)
21909 · copyurl: Add --stdout flag to write to stdout (Nick Craig-Wood)
21911 · dedupe: Implement keep smallest too (Nick Craig-Wood)
21913 · hashsum: Add flag --base64 flag (landall)
21915 · lsf: Speed up on s3/swift/etc by not reading mimetype by default
21917 (Nick Craig-Wood)
21918 · lsjson: Add --no-mimetype flag (Nick Craig-Wood)
21920 · rc: Add methods to turn on blocking and mutex profiling (Nick
21922 Craig-Wood)
21923 · rcd
21925 · Adding group parameter to stats (Chaitanya)
21927 · Move webgui apart; option to disable browser (Xiaoxing Ye)
21929 · serve sftp: Add support for public key with auth proxy (Paul Tins‐
21931 ley)
21932 · stats: Show deletes in stats and hide zero stats (anuar45)
21934 · Bug Fixes
21936 · accounting
21938 · Fix error counter counting multiple times (Ankur Gupta)
21940 · Fix error count shown as checks (Cnly)
21942 · Clear finished transfer in stats-reset (Maciej Zimnoch)
21944 · Added StatsInfo locking in statsGroups sum function (Michał
21946 Matczuk)
21947 · asyncreader: Fix EOF error (buengese)
21949 · check: Fix --one-way recursing more directories than it needs to
21951 (Nick Craig-Wood)
21952 · chunkedreader: Disable hash calculation for first segment (Nick
21954 Craig-Wood)
21955 · config
21957 · Do not open browser on headless on drive/gcs/google photos (Xi‐
21959 aoxing Ye)
21960 · SetValueAndSave ignore error if config section does not exist yet
21962 (buengese)
21963 · cmd: Fix completion with an encrypted config (Danil Semelenov)
21965 · dbhashsum: Stop it returning UNSUPPORTED on dropbox (Nick
21967 Craig-Wood)
21968 · dedupe: Add missing modes to help string (Nick Craig-Wood)
21970 · operations
21972 · Fix dedupe continuing on errors like insufficientFilePermisson
21974 (SezalAgrawal)
21975 · Clear accounting before low level retry (Maciej Zimnoch)
21977 · Write debug message when hashes could not be checked (Ole Schütt)
21979 · Move interface assertion to tests to remove pflag dependency
21981 (Nick Craig-Wood)
21982 · Make NewOverrideObjectInfo public and factor uses (Nick
21984 Craig-Wood)
21985 · proxy: Replace use of bcrypt with sha256 (Nick Craig-Wood)
21987 · vendor
21989 · Update bazil.org/fuse to fix FreeBSD 12.1 (Nick Craig-Wood)
21991 · Update github.com/t3rm1n4l/go-mega to fix mega “illegal base64
21993 data at input byte 22” (Nick Craig-Wood)
21994 · Update termbox-go to fix ncdu command on FreeBSD (Kuang-che Wu)
21996 · Update t3rm1n4l/go-mega - fixes mega: couldn’t login: crypto/aes:
21998 invalid key size 0 (Nick Craig-Wood)
21999 · Mount
22001 · Enable async reads for a 20% speedup (Nick Craig-Wood)
22003 · Replace use of WriteAt with Write for cache mode >= writes and
22005 O_APPEND (Brett Dutro)
22006 · Make sure we call unmount when exiting (Nick Craig-Wood)
22008 · Don’t build on go1.10 as bazil/fuse no longer supports it (Nick
22010 Craig-Wood)
22011 · When setting dates discard out of range dates (Nick Craig-Wood)
22013 · VFS
22015 · Add a newly created file straight into the directory (Nick
22017 Craig-Wood)
22018 · Only calculate one hash for reads for a speedup (Nick Craig-Wood)
22020 · Make ReadAt for non cached files work better with non-sequential
22022 reads (Nick Craig-Wood)
22023 · Fix edge cases when reading ModTime from file (Nick Craig-Wood)
22025 · Make sure existing files opened for write show correct size (Nick
22027 Craig-Wood)
22028 · Don’t cache the path in RW file objects to fix renaming (Nick
22030 Craig-Wood)
22031 · Fix rename of open files when using the VFS cache (Nick Craig-Wood)
22033 · When renaming files in the cache, rename the cache item in memory
22035 too (Nick Craig-Wood)
22036 · Fix open file renaming on drive when using --vfs-cache-mode writes
22038 (Nick Craig-Wood)
22039 · Fix incorrect modtime for mv into mount with --vfs-cache-modes
22041 writes (Nick Craig-Wood)
22042 · On rename, rename in cache too if the file exists (Anagh Kumar
22044 Baranwal)
22045 · Local
22047 · Make source file being updated errors be NoLowLevelRetry errors
22049 (Nick Craig-Wood)
22050 · Fix update of hidden files on Windows (Nick Craig-Wood)
22052 · Cache
22054 · Follow move of upstream library github.com/coreos/bbolt
22056 github.com/etcd-io/bbolt (Nick Craig-Wood)
22057 · Fix fatal error: concurrent map writes (Nick Craig-Wood)
22059 · Crypt
22061 · Reorder the filename encryption options (Thomas Eales)
22063 · Correctly handle trailing dot (buengese)
22065 · Chunker
22067 · Reduce length of temporary suffix (Ivan Andreev)
22069 · Drive
22071 · Add --drive-stop-on-upload-limit flag to stop syncs when upload
22073 limit reached (Nick Craig-Wood)
22074 · Add --drive-use-shared-date to use date file was shared instead of
22076 modified date (Garry McNulty)
22077 · Make sure invalid auth for teamdrives always reports an error (Nick
22079 Craig-Wood)
22080 · Fix --fast-list when using appDataFolder (Nick Craig-Wood)
22082 · Use multipart resumable uploads for streaming and uploads in mount
22084 (Nick Craig-Wood)
22085 · Log an ERROR if an incomplete search is returned (Nick Craig-Wood)
22087 · Hide dangerous config from the configurator (Nick Craig-Wood)
22089 · Dropbox
22091 · Treat insufficient_space errors as non retriable errors (Nick
22093 Craig-Wood)
22094 · Jottacloud
22096 · Use new auth method used by official client (buengese)
22098 · Add URL to generate Login Token to config wizard (Nick Craig-Wood)
22100 · Add support whitelabel versions (buengese)
22102 · Koofr
22104 · Use rclone HTTP client. (jaKa)
22106 · Onedrive
22108 · Add Sites.Read.All permission (Benjamin Richter)
22110 · Add support “Retry-After” header (Motonori IWAMURO)
22112 · Opendrive
22114 · Implement --opendrive-chunk-size (Nick Craig-Wood)
22116 · S3
22118 · Re-implement multipart upload to fix memory issues (Nick
22120 Craig-Wood)
22121 · Add --s3-copy-cutoff for size to switch to multipart copy (Nick
22123 Craig-Wood)
22124 · Add new region Asia Patific (Hong Kong) (Outvi V)
22126 · Reduce memory usage streaming files by reducing max stream upload
22128 size (Nick Craig-Wood)
22129 · Add --s3-list-chunk option for bucket listing (Thomas Kriechbaumer)
22131 · Force path style bucket access to off for AWS deprecation (Nick
22133 Craig-Wood)
22134 · Use AWS web identity role provider if available (Tennix)
22136 · Fix ExpiryWindow value (Aleksandar Jankovic)
22138 · Fix DisableChecksum condition (Aleksandar Janković)
22140 · Fix URL decoding of NextMarker (Nick Craig-Wood)
22142 · SFTP
22144 · Add --sftp-skip-links to skip symlinks and non regular files (Nick
22146 Craig-Wood)
22147 · Retry Creation of Connection (Sebastian Brandt)
22149 · Fix “failed to parse private key file: ssh: not an encrypted key”
22151 error (Nick Craig-Wood)
22152 · Open files for update write only to fix AWS SFTP interop (Nick
22154 Craig-Wood)
22155 · Swift
22157 · Reserve segments of dynamic large object when delete objects in
22159 container what was enabled versioning. (Nguyễn Hữu Luân)
22160 · Fix parsing of X-Object-Manifest (Nick Craig-Wood)
22162 · Update OVH API endpoint (unbelauscht)
22164 · WebDAV
22166 · Make nextcloud only upload SHA1 checksums (Nick Craig-Wood)
22168 · Fix case of “Bearer” in Authorization: header to agree with RFC
22170 (Nick Craig-Wood)
22171 · Add Referer header to fix problems with WAFs (Nick Craig-Wood)
22173 v1.50.2 - 2019-11-19
22175 · Bug Fixes
22176 · accounting: Fix memory leak on retries operations (Nick Craig-Wood)
22178 · Drive
22180 · Fix listing of the root directory with drive.files scope (Nick
22182 Craig-Wood)
22183 · Fix –drive-root-folder-id with team/shared drives (Nick Craig-Wood)
22185 v1.50.1 - 2019-11-02
22187 · Bug Fixes
22188 · hash: Fix accidentally changed hash names for DropboxHash and
22190 CRC-32 (Nick Craig-Wood)
22191 · fshttp: Fix error reporting on tpslimit token bucket errors (Nick
22193 Craig-Wood)
22194 · fshttp: Don’t print token bucket errors on context cancelled (Nick
22196 Craig-Wood)
22197 · Local
22199 · Fix listings of . on Windows (Nick Craig-Wood)
22201 · Onedrive
22203 · Fix DirMove/Move after Onedrive change (Xiaoxing Ye)
22205 v1.50.0 - 2019-10-26
22207 · New backends
22208 · Citrix Sharefile (/sharefile) (Nick Craig-Wood)
22210 · Chunker (/chunker) - an overlay backend to split files into smaller
22212 parts (Ivan Andreev)
22213 · Mail.ru Cloud (/mailru) (Ivan Andreev)
22215 · New Features
22217 · encodings (Fabian Möller & Nick Craig-Wood)
22219 · All backends now use file name encoding to ensure any file name
22221 can be written to any backend.
22222 · See the restricted file name docs (/overview/#restricted-file‐
22224 names) for more info and the local backend docs (/local/#file‐
22225 names).
22226 · Some file names may look different in rclone if you are using any
22228 control characters in names or unicode FULLWIDTH symbols
22229 (https://en.wikipedia.org/wiki/Halfwidth_and_Full‐
22230 width_Forms_(Unicode_block)).
22231 · build
22233 · Update to use go1.13 for the build (Nick Craig-Wood)
22235 · Drop support for go1.9 (Nick Craig-Wood)
22237 · Build rclone with GitHub actions (Nick Craig-Wood)
22239 · Convert python scripts to python3 (Nick Craig-Wood)
22241 · Swap Azure/go-ansiterm for mattn/go-colorable (Nick Craig-Wood)
22243 · Dockerfile fixes (Matei David)
22245 · Add plugin support (https://github.com/rclone/rclone/blob/mas‐
22247 ter/CONTRIBUTING.md#writing-a-plugin) for backends and commands
22248 (Richard Patel)
22249 · config
22251 · Use alternating Red/Green in config to make more obvious (Nick
22253 Craig-Wood)
22254 · contrib
22256 · Add sample DLNA server Docker Compose manifest. (pataquets)
22258 · Add sample WebDAV server Docker Compose manifest. (pataquets)
22260 · copyurl
22262 · Add --auto-filename flag for using file name from URL in destina‐
22264 tion path (Denis)
22265 · serve dlna:
22267 · Many compatability improvements (Dan Walters)
22269 · Support for external srt subtitles (Dan Walters)
22271 · rc
22273 · Added command core/quit (Saksham Khanna)
22275 · Bug Fixes
22277 · sync
22279 · Make --update/-u not transfer files that haven’t changed (Nick
22281 Craig-Wood)
22282 · Free objects after they come out of the transfer pipe to save
22284 memory (Nick Craig-Wood)
22285 · Fix --files-from without --no-traverse doing a recursive scan
22287 (Nick Craig-Wood)
22288 · operations
22290 · Fix accounting for server side copies (Nick Craig-Wood)
22292 · Display `All duplicates removed' only if dedupe successful (Sezal
22294 Agrawal)
22295 · Display `Deleted X extra copies' only if dedupe successful (Sezal
22297 Agrawal)
22298 · accounting
22300 · Only allow up to 100 completed transfers in the accounting list
22302 to save memory (Nick Craig-Wood)
22303 · Cull the old time ranges when possible to save memory (Nick
22305 Craig-Wood)
22306 · Fix panic due to server-side copy fallback (Ivan Andreev)
22308 · Fix memory leak noticeable for transfers of large numbers of ob‐
22310 jects (Nick Craig-Wood)
22311 · Fix total duration calculation (Nick Craig-Wood)
22313 · cmd
22315 · Fix environment variables not setting command line flags (Nick
22317 Craig-Wood)
22318 · Make autocomplete compatible with bash’s posix mode for macOS
22320 (Danil Semelenov)
22321 · Make --progress work in git bash on Windows (Nick Craig-Wood)
22323 · Fix `compopt: command not found' on autocomplete on macOS (Danil
22325 Semelenov)
22326 · config
22328 · Fix setting of non top level flags from environment variables
22330 (Nick Craig-Wood)
22331 · Check config names more carefully and report errors (Nick
22333 Craig-Wood)
22334 · Remove error: can’t use --size-only and --ignore-size together.
22336 (Nick Craig-Wood)
22337 · filter: Prevent mixing options when --files-from is in use (Michele
22339 Caci)
22340 · serve sftp: Fix crash on unsupported operations (eg Readlink) (Nick
22342 Craig-Wood)
22343 · Mount
22345 · Allow files of unkown size to be read properly (Nick Craig-Wood)
22347 · Skip tests on <= 2 CPUs to avoid lockup (Nick Craig-Wood)
22349 · Fix panic on File.Open (Nick Craig-Wood)
22351 · Fix “mount_fusefs: -o timeout=: option not supported” on FreeBSD
22353 (Nick Craig-Wood)
22354 · Don’t pass huge filenames (>4k) to FUSE as it can’t cope (Nick
22356 Craig-Wood)
22357 · VFS
22359 · Add flag --vfs-case-insensitive for windows/macOS mounts (Ivan An‐
22361 dreev)
22362 · Make objects of unknown size readable through the VFS (Nick
22364 Craig-Wood)
22365 · Move writeback of dirty data out of close() method into its own
22367 method (FlushWrites) and remove close() call from Flush() (Brett
22368 Dutro)
22369 · Stop empty dirs disappearing when renamed on bucket based remotes
22371 (Nick Craig-Wood)
22372 · Stop change notify polling clearing so much of the directory cache
22374 (Nick Craig-Wood)
22375 · Azure Blob
22377 · Disable logging to the Windows event log (Nick Craig-Wood)
22379 · B2
22381 · Remove unverified: prefix on sha1 to improve interop (eg with Cy‐
22383 berDuck) (Nick Craig-Wood)
22384 · Box
22386 · Add options to get access token via JWT auth (David)
22388 · Drive
22390 · Disable HTTP/2 by default to work around INTERNAL_ERROR problems
22392 (Nick Craig-Wood)
22393 · Make sure that drive root ID is always canonical (Nick Craig-Wood)
22395 · Fix --drive-shared-with-me from the root with lsand --fast-list
22397 (Nick Craig-Wood)
22398 · Fix ChangeNotify polling for shared drives (Nick Craig-Wood)
22400 · Fix change notify polling when using appDataFolder (Nick
22402 Craig-Wood)
22403 · Dropbox
22405 · Make disallowed filenames errors not retry (Nick Craig-Wood)
22407 · Fix nil pointer exception on restricted files (Nick Craig-Wood)
22409 · Fichier
22411 · Fix accessing files > 2GB on 32 bit systems (Nick Craig-Wood)
22413 · FTP
22415 · Allow disabling EPSV mode (Jon Fautley)
22417 · HTTP
22419 · HEAD directory entries in parallel to speedup (Nick Craig-Wood)
22421 · Add --http-no-head to stop rclone doing HEAD in listings (Nick
22423 Craig-Wood)
22424 · Putio
22426 · Add ability to resume uploads (Cenk Alti)
22428 · S3
22430 · Fix signature v2_auth headers (Anthony Rusdi)
22432 · Fix encoding for control characters (Nick Craig-Wood)
22434 · Only ask for URL encoded directory listings if we need them on Ceph
22436 (Nick Craig-Wood)
22437 · Add option for multipart failiure behaviour (Aleksandar Jankovic)
22439 · Support for multipart copy (庄天翼)
22441 · Fix nil pointer reference if no metadata returned for object (Nick
22443 Craig-Wood)
22444 · SFTP
22446 · Fix --sftp-ask-password trying to contact the ssh agent (Nick
22448 Craig-Wood)
22449 · Fix hashes of files with backslashes (Nick Craig-Wood)
22451 · Include more ciphers with --sftp-use-insecure-cipher (Carlos Fer‐
22453 reyra)
22454 · WebDAV
22456 · Parse and return Sharepoint error response (Henning Surmeier)
22458 v1.49.5 - 2019-10-05
22460 · Bug Fixes
22461 · Revert back to go1.12.x for the v1.49.x builds as go1.13.x was
22463 causing issues (Nick Craig-Wood)
22464 · Fix rpm packages by using master builds of nfpm (Nick Craig-Wood)
22466 · Fix macOS build after brew changes (Nick Craig-Wood)
22468 v1.49.4 - 2019-09-29
22470 · Bug Fixes
22471 · cmd/rcd: Address ZipSlip vulnerability (Richard Patel)
22473 · accounting: Fix file handle leak on errors (Nick Craig-Wood)
22475 · oauthutil: Fix security problem when running with two users on the
22477 same machine (Nick Craig-Wood)
22478 · FTP
22480 · Fix listing of an empty root returning: error dir not found (Nick
22482 Craig-Wood)
22483 · S3
22485 · Fix SetModTime on GLACIER/ARCHIVE objects and implement set/get
22487 tier (Nick Craig-Wood)
22488 v1.49.3 - 2019-09-15
22490 · Bug Fixes
22491 · accounting
22493 · Fix total duration calculation (Aleksandar Jankovic)
22495 · Fix “file already closed” on transfer retries (Nick Craig-Wood)
22497 v1.49.2 - 2019-09-08
22499 · New Features
22500 · build: Add Docker workflow support (Alfonso Montero)
22502 · Bug Fixes
22504 · accounting: Fix locking in Transfer to avoid deadlock with
22506 --progress (Nick Craig-Wood)
22507 · docs: Fix template argument for mktemp in install.sh (Cnly)
22509 · operations: Fix -u/--update with google photos / files of unknown
22511 size (Nick Craig-Wood)
22512 · rc: Fix docs for config/create /update /password (Nick Craig-Wood)
22514 · Google Cloud Storage
22516 · Fix need for elevated permissions on SetModTime (Nick Craig-Wood)
22518 v1.49.1 - 2019-08-28
22520 · Bug Fixes
22521 · config: Fix generated passwords being stored as empty password
22523 (Nick Craig-Wood)
22524 · rcd: Added missing parameter for web-gui info logs. (Chaitanya)
22526 · Googlephotos
22528 · Fix crash on error response (Nick Craig-Wood)
22530 · Onedrive
22532 · Fix crash on error response (Nick Craig-Wood)
22534 v1.49.0 - 2019-08-26
22536 · New backends
22537 · 1fichier (https://rclone.org/fichier/) (Laura Hausmann)
22539 · Google Photos (/googlephotos) (Nick Craig-Wood)
22541 · Putio (https://rclone.org/putio/) (Cenk Alti)
22543 · premiumize.me (https://rclone.org/premiumizeme/) (Nick Craig-Wood)
22545 · New Features
22547 · Experimental web GUI (https://rclone.org/gui/) (Chaitanya Bankan‐
22549 hal)
22550 · Implement --compare-dest & --copy-dest (yparitcher)
22552 · Implement --suffix without --backup-dir for backup to current dir
22554 (yparitcher)
22555 · config reconnect to re-login (re-run the oauth login) for the back‐
22557 end. (Nick Craig-Wood)
22558 · config userinfo to discover which user you are logged in as. (Nick
22560 Craig-Wood)
22561 · config disconnect to disconnect you (log out) from the backend.
22563 (Nick Craig-Wood)
22564 · Add --use-json-log for JSON logging (justinalin)
22566 · Add context propagation to rclone (Aleksandar Jankovic)
22568 · Reworking internal statistics interfaces so they work with rc jobs
22570 (Aleksandar Jankovic)
22571 · Add Higher units for ETA (AbelThar)
22573 · Update rclone logos to new design (Andreas Chlupka)
22575 · hash: Add CRC-32 support (Cenk Alti)
22577 · help showbackend: Fixed advanced option category when there are no
22579 standard options (buengese)
22580 · ncdu: Display/Copy to Clipboard Current Path (Gary Kim)
22582 · operations:
22584 · Run hashing operations in parallel (Nick Craig-Wood)
22586 · Don’t calculate checksums when using --ignore-checksum (Nick
22588 Craig-Wood)
22589 · Check transfer hashes when using --size-only mode (Nick
22591 Craig-Wood)
22592 · Disable multi thread copy for local to local copies (Nick
22594 Craig-Wood)
22595 · Debug successful hashes as well as failures (Nick Craig-Wood)
22597 · rc
22599 · Add ability to stop async jobs (Aleksandar Jankovic)
22601 · Return current settings if core/bwlimit called without parameters
22603 (Nick Craig-Wood)
22604 · Rclone-WebUI integration with rclone (Chaitanya Bankanhal)
22606 · Added command line parameter to control the cross origin resource
22608 sharing (CORS) in the rcd. (Security Improvement) (Chaitanya
22609 Bankanhal)
22610 · Add anchor tags to the docs so links are consistent (Nick
22612 Craig-Wood)
22613 · Remove _async key from input parameters after parsing so later
22615 operations won’t get confused (buengese)
22616 · Add call to clear stats (Aleksandar Jankovic)
22618 · rcd
22620 · Auto-login for web-gui (Chaitanya Bankanhal)
22622 · Implement --baseurl for rcd and web-gui (Chaitanya Bankanhal)
22624 · serve dlna
22626 · Only select interfaces which can multicast for SSDP (Nick
22628 Craig-Wood)
22629 · Add more builtin mime types to cover standard audio/video (Nick
22631 Craig-Wood)
22632 · Fix missing mime types on Android causing missing videos (Nick
22634 Craig-Wood)
22635 · serve ftp
22637 · Refactor to bring into line with other serve commands (Nick
22639 Craig-Wood)
22640 · Implement --auth-proxy (Nick Craig-Wood)
22642 · serve http: Implement --baseurl (Nick Craig-Wood)
22644 · serve restic: Implement --baseurl (Nick Craig-Wood)
22646 · serve sftp
22648 · Implement auth proxy (Nick Craig-Wood)
22650 · Fix detection of whether server is authorized (Nick Craig-Wood)
22652 · serve webdav
22654 · Implement --baseurl (Nick Craig-Wood)
22656 · Support --auth-proxy (Nick Craig-Wood)
22658 · Bug Fixes
22660 · Make “bad record MAC” a retriable error (Nick Craig-Wood)
22662 · copyurl: Fix copying files that return HTTP errors (Nick
22664 Craig-Wood)
22665 · march: Fix checking sub-directories when using --no-traverse
22667 (buengese)
22668 · rc
22670 · Fix unmarshalable http.AuthFn in options and put in test for mar‐
22672 shalability (Nick Craig-Wood)
22673 · Move job expire flags to rc to fix initalization problem (Nick
22675 Craig-Wood)
22676 · Fix --loopback with rc/list and others (Nick Craig-Wood)
22678 · rcat: Fix slowdown on systems with multiple hashes (Nick
22680 Craig-Wood)
22681 · rcd: Fix permissions problems on cache directory with web gui down‐
22683 load (Nick Craig-Wood)
22684 · Mount
22686 · Default --deamon-timout to 15 minutes on macOS and FreeBSD (Nick
22688 Craig-Wood)
22689 · Update docs to show mounting from root OK for bucket based (Nick
22691 Craig-Wood)
22692 · Remove nonseekable flag from write files (Nick Craig-Wood)
22694 · VFS
22696 · Make write without cache more efficient (Nick Craig-Wood)
22698 · Fix --vfs-cache-mode minimal and writes ignoring cached files (Nick
22700 Craig-Wood)
22701 · Local
22703 · Add --local-case-sensitive and --local-case-insensitive (Nick
22705 Craig-Wood)
22706 · Avoid polluting page cache when uploading local files to remote
22708 backends (Michał Matczuk)
22709 · Don’t calculate any hashes by default (Nick Craig-Wood)
22711 · Fadvise run syscall on a dedicated go routine (Michał Matczuk)
22713 · Azure Blob
22715 · Azure Storage Emulator support (Sandeep)
22717 · Updated config help details to remove connection string references
22719 (Sandeep)
22720 · Make all operations work from the root (Nick Craig-Wood)
22722 · B2
22724 · Implement link sharing (yparitcher)
22726 · Enable server side copy to copy between buckets (Nick Craig-Wood)
22728 · Make all operations work from the root (Nick Craig-Wood)
22730 · Drive
22732 · Fix server side copy of big files (Nick Craig-Wood)
22734 · Update API for teamdrive use (Nick Craig-Wood)
22736 · Add error for purge with --drive-trashed-only (ginvine)
22738 · Fichier
22740 · Make FolderID int and adjust related code (buengese)
22742 · Google Cloud Storage
22744 · Reduce oauth scope requested as suggested by Google (Nick
22746 Craig-Wood)
22747 · Make all operations work from the root (Nick Craig-Wood)
22749 · HTTP
22751 · Add --http-headers flag for setting arbitrary headers (Nick
22753 Craig-Wood)
22754 · Jottacloud
22756 · Use new api for retrieving internal username (buengese)
22758 · Refactor configuration and minor cleanup (buengese)
22760 · Koofr
22762 · Support setting modification times on Koofr backend. (jaKa)
22764 · Opendrive
22766 · Refactor to use existing lib/rest facilities for uploads (Nick
22768 Craig-Wood)
22769 · Qingstor
22771 · Upgrade to v3 SDK and fix listing loop (Nick Craig-Wood)
22773 · Make all operations work from the root (Nick Craig-Wood)
22775 · S3
22777 · Add INTELLIGENT_TIERING storage class (Matti Niemenmaa)
22779 · Make all operations work from the root (Nick Craig-Wood)
22781 · SFTP
22783 · Add missing interface check and fix About (Nick Craig-Wood)
22785 · Completely ignore all modtime checks if SetModTime=false (Jon Faut‐
22787 ley)
22788 · Support md5/sha1 with rsync.net (Nick Craig-Wood)
22790 · Save the md5/sha1 command in use to the config file for efficiency
22792 (Nick Craig-Wood)
22793 · Opt-in support for diffie-hellman-group-exchange-sha256
22795 diffie-hellman-group-exchange-sha1 (Yi FU)
22796 · Swift
22798 · Use FixRangeOption to fix 0 length files via the VFS (Nick
22800 Craig-Wood)
22801 · Fix upload when using no_chunk to return the correct size (Nick
22803 Craig-Wood)
22804 · Make all operations work from the root (Nick Craig-Wood)
22806 · Fix segments leak during failed large file uploads. (nguyenhuulu‐
22808 an434)
22809 · WebDAV
22811 · Add --webdav-bearer-token-command (Nick Craig-Wood)
22813 · Refresh token when it expires with --webdav-bearer-token-command
22815 (Nick Craig-Wood)
22816 · Add docs for using bearer_token_command with oidc-agent (Paul Mil‐
22818 lar)
22819 v1.48.0 - 2019-06-15
22821 · New commands
22822 · serve sftp: Serve an rclone remote over SFTP (Nick Craig-Wood)
22824 · New Features
22826 · Multi threaded downloads to local storage (Nick Craig-Wood)
22828 · controlled with --multi-thread-cutoff and --multi-thread-streams
22830 · Use rclone.conf from rclone executable directory to enable portable
22832 use (albertony)
22833 · Allow sync of a file and a directory with the same name (forgems)
22835 · this is common on bucket based remotes, eg s3, gcs
22837 · Add --ignore-case-sync for forced case insensitivity (garry415)
22839 · Implement --stats-one-line-date and --stats-one-line-date-format
22841 (Peter Berbec)
22842 · Log an ERROR for all commands which exit with non-zero status (Nick
22844 Craig-Wood)
22845 · Use go-homedir to read the home directory more reliably (Nick
22847 Craig-Wood)
22848 · Enable creating encrypted config through external script invocation
22850 (Wojciech Smigielski)
22851 · build: Drop support for go1.8 (Nick Craig-Wood)
22853 · config: Make config create/update encrypt passwords where necessary
22855 (Nick Craig-Wood)
22856 · copyurl: Honor --no-check-certificate (Stefan Breunig)
22858 · install: Linux skip man pages if no mandb (didil)
22860 · lsf: Support showing the Tier of the object (Nick Craig-Wood)
22862 · lsjson
22864 · Added EncryptedPath to output (calisro)
22866 · Support showing the Tier of the object (Nick Craig-Wood)
22868 · Add IsBucket field for bucket based remote listing of the root
22870 (Nick Craig-Wood)
22871 · rc
22873 · Add --loopback flag to run commands directly without a server
22875 (Nick Craig-Wood)
22876 · Add operations/fsinfo: Return information about the remote (Nick
22878 Craig-Wood)
22879 · Skip auth for OPTIONS request (Nick Craig-Wood)
22881 · cmd/providers: Add DefaultStr, ValueStr and Type fields (Nick
22883 Craig-Wood)
22884 · jobs: Make job expiry timeouts configurable (Aleksandar Jankovic)
22886 · serve dlna reworked and improved (Dan Walters)
22888 · serve ftp: add --ftp-public-ip flag to specify public IP (calistri)
22890 · serve restic: Add support for --private-repos in serve restic (Flo‐
22892 rian Apolloner)
22893 · serve webdav: Combine serve webdav and serve http (Gary Kim)
22895 · size: Ignore negative sizes when calculating total (Garry McNulty)
22897 · Bug Fixes
22899 · Make move and copy individual files obey --backup-dir (Nick
22901 Craig-Wood)
22902 · If --ignore-checksum is in effect, don’t calculate checksum (Nick
22904 Craig-Wood)
22905 · moveto: Fix case-insensitive same remote move (Gary Kim)
22907 · rc: Fix serving bucket based objects with --rc-serve (Nick
22909 Craig-Wood)
22910 · serve webdav: Fix serveDir not being updated with changes from web‐
22912 dav (Gary Kim)
22913 · Mount
22915 · Fix poll interval documentation (Animosity022)
22917 · VFS
22919 · Make WriteAt for non cached files work with non-sequential writes
22921 (Nick Craig-Wood)
22922 · Local
22924 · Only calculate the required hashes for big speedup (Nick
22926 Craig-Wood)
22927 · Log errors when listing instead of returning an error (Nick
22929 Craig-Wood)
22930 · Fix preallocate warning on Linux with ZFS (Nick Craig-Wood)
22932 · Crypt
22934 · Make rclone dedupe work through crypt (Nick Craig-Wood)
22936 · Fix wrapping of ChangeNotify to decrypt directories properly (Nick
22938 Craig-Wood)
22939 · Support PublicLink (rclone link) of underlying backend (Nick
22941 Craig-Wood)
22942 · Implement Optional methods SetTier, GetTier (Nick Craig-Wood)
22944 · B2
22946 · Implement server side copy (Nick Craig-Wood)
22948 · Implement SetModTime (Nick Craig-Wood)
22950 · Drive
22952 · Fix move and copy from TeamDrive to GDrive (Fionera)
22954 · Add notes that cleanup works in the background on drive (Nick
22956 Craig-Wood)
22957 · Add --drive-server-side-across-configs to default back to old serv‐
22959 er side copy semantics by default (Nick Craig-Wood)
22960 · Add --drive-size-as-quota to show storage quota usage for file size
22962 (Garry McNulty)
22963 · FTP
22965 · Add FTP List timeout (Jeff Quinn)
22967 · Add FTP over TLS support (Gary Kim)
22969 · Add --ftp-no-check-certificate option for FTPS (Gary Kim)
22971 · Google Cloud Storage
22973 · Fix upload errors when uploading pre 1970 files (Nick Craig-Wood)
22975 · Jottacloud
22977 · Add support for selecting device and mountpoint. (buengese)
22979 · Mega
22981 · Add cleanup support (Gary Kim)
22983 · Onedrive
22985 · More accurately check if root is found (Cnly)
22987 · S3
22989 · Suppport S3 Accelerated endpoints with --s3-use-accelerate-endpoint
22991 (Nick Craig-Wood)
22992 · Add config info for Wasabi’s EU Central endpoint (Robert Marko)
22994 · Make SetModTime work for GLACIER while syncing (Philip Harvey)
22996 · SFTP
22998 · Add About support (Gary Kim)
23000 · Fix about parsing of df results so it can cope with -ve results
23002 (Nick Craig-Wood)
23003 · Send custom client version and debug server version (Nick
23005 Craig-Wood)
23006 · WebDAV
23008 · Retry on 423 Locked errors (Nick Craig-Wood)
23010 v1.47.0 - 2019-04-13
23012 · New backends
23013 · Backend for Koofr cloud storage service. (jaKa)
23015 · New Features
23017 · Resume downloads if the reader fails in copy (Nick Craig-Wood)
23019 · this means rclone will restart transfers if the source has an er‐
23021 ror
23022 · this is most useful for downloads or cloud to cloud copies
23024 · Use --fast-list for listing operations where it won’t use more mem‐
23026 ory (Nick Craig-Wood)
23027 · this should speed up the following operations on remotes which
23029 support ListR
23030 · dedupe, serve restic lsf, ls, lsl, lsjson, lsd, md5sum, sha1sum,
23032 hashsum, size, delete, cat, settier
23033 · use --disable ListR to get old behaviour if required
23035 · Make --files-from traverse the destination unless --no-traverse is
23037 set (Nick Craig-Wood)
23038 · this fixes --files-from with Google drive and excessive API use
23040 in general.
23041 · Make server side copy account bytes and obey --max-transfer (Nick
23043 Craig-Wood)
23044 · Add --create-empty-src-dirs flag and default to not creating empty
23046 dirs (ishuah)
23047 · Add client side TLS/SSL flags --ca-cert/--client-cert/--client-key
23049 (Nick Craig-Wood)
23050 · Implement --suffix-keep-extension for use with --suffix (Nick
23052 Craig-Wood)
23053 · build:
23055 · Switch to semvar compliant version tags to be go modules compli‐
23057 ant (Nick Craig-Wood)
23058 · Update to use go1.12.x for the build (Nick Craig-Wood)
23060 · serve dlna: Add connection manager service description to improve
23062 compatibility (Dan Walters)
23063 · lsf: Add `e' format to show encrypted names and `o' for original
23065 IDs (Nick Craig-Wood)
23066 · lsjson: Added --files-only and --dirs-only flags (calistri)
23068 · rc: Implement operations/publiclink the equivalent of rclone link
23070 (Nick Craig-Wood)
23071 · Bug Fixes
23073 · accounting: Fix total ETA when --stats-unit bits is in effect (Nick
23075 Craig-Wood)
23076 · Bash TAB completion
23078 · Use private custom func to fix clash between rclone and kubectl
23080 (Nick Craig-Wood)
23081 · Fix for remotes with underscores in their names (Six)
23083 · Fix completion of remotes (Florian Gamböck)
23085 · Fix autocompletion of remote paths with spaces (Danil Semelenov)
23087 · serve dlna: Fix root XML service descriptor (Dan Walters)
23089 · ncdu: Fix display corruption with Chinese characters (Nick
23091 Craig-Wood)
23092 · Add SIGTERM to signals which run the exit handlers on unix (Nick
23094 Craig-Wood)
23095 · rc: Reload filter when the options are set via the rc (Nick
23097 Craig-Wood)
23098 · VFS / Mount
23100 · Fix FreeBSD: Ignore Truncate if called with no readers and already
23102 the correct size (Nick Craig-Wood)
23103 · Read directory and check for a file before mkdir (Nick Craig-Wood)
23105 · Shorten the locking window for vfs/refresh (Nick Craig-Wood)
23107 · Azure Blob
23109 · Enable MD5 checksums when uploading files bigger than the “Cutoff”
23111 (Dr.Rx)
23112 · Fix SAS URL support (Nick Craig-Wood)
23114 · B2
23116 · Allow manual configuration of backblaze downloadUrl (Vince)
23118 · Ignore already_hidden error on remove (Nick Craig-Wood)
23120 · Ignore malformed src_last_modified_millis (Nick Craig-Wood)
23122 · Drive
23124 · Add --skip-checksum-gphotos to ignore incorrect checksums on Google
23126 Photos (Nick Craig-Wood)
23127 · Allow server side move/copy between different remotes. (Fionera)
23129 · Add docs on team drives and --fast-list eventual consistency (Nes‐
23131 tar47)
23132 · Fix imports of text files (Nick Craig-Wood)
23134 · Fix range requests on 0 length files (Nick Craig-Wood)
23136 · Fix creation of duplicates with server side copy (Nick Craig-Wood)
23138 · Dropbox
23140 · Retry blank errors to fix long listings (Nick Craig-Wood)
23142 · FTP
23144 · Add --ftp-concurrency to limit maximum number of connections (Nick
23146 Craig-Wood)
23147 · Google Cloud Storage
23149 · Fall back to default application credentials (marcintustin)
23151 · Allow bucket policy only buckets (Nick Craig-Wood)
23153 · HTTP
23155 · Add --http-no-slash for websites with directories with no slashes
23157 (Nick Craig-Wood)
23158 · Remove duplicates from listings (Nick Craig-Wood)
23160 · Fix socket leak on 404 errors (Nick Craig-Wood)
23162 · Jottacloud
23164 · Fix token refresh (Sebastian Bünger)
23166 · Add device registration (Oliver Heyme)
23168 · Onedrive
23170 · Implement graceful cancel of multipart uploads if rclone is inter‐
23172 rupted (Cnly)
23173 · Always add trailing colon to path when addressing items, (Cnly)
23175 · Return errors instead of panic for invalid uploads (Fabian Möller)
23177 · S3
23179 · Add support for “Glacier Deep Archive” storage class (Manu)
23181 · Update Dreamhost endpoint (Nick Craig-Wood)
23183 · Note incompatibility with CEPH Jewel (Nick Craig-Wood)
23185 · SFTP
23187 · Allow custom ssh client config (Alexandru Bumbacea)
23189 · Swift
23191 · Obey Retry-After to enable OVH restore from cold storage (Nick
23193 Craig-Wood)
23194 · Work around token expiry on CEPH (Nick Craig-Wood)
23196 · WebDAV
23198 · Allow IsCollection property to be integer or boolean (Nick
23200 Craig-Wood)
23201 · Fix race when creating directories (Nick Craig-Wood)
23203 · Fix About/df when reading the available/total returns 0 (Nick
23205 Craig-Wood)
23206 v1.46 - 2019-02-09
23208 · New backends
23209 · Support Alibaba Cloud (Aliyun) OSS via the s3 backend (Nick
23211 Craig-Wood)
23212 · New commands
23214 · serve dlna: serves a remove via DLNA for the local network (ni‐
23216 colov)
23217 · New Features
23219 · copy, move: Restore deprecated --no-traverse flag (Nick Craig-Wood)
23221 · This is useful for when transferring a small number of files into
23223 a large destination
23224 · genautocomplete: Add remote path completion for bash completion
23226 (Christopher Peterson & Danil Semelenov)
23227 · Buffer memory handling reworked to return memory to the OS better
23229 (Nick Craig-Wood)
23230 · Buffer recycling library to replace sync.Pool
23232 · Optionally use memory mapped memory for better memory shrinking
23234 · Enable with --use-mmap if having memory problems - not default
23236 yet
23237 · Parallelise reading of files specified by --files-from (Nick
23239 Craig-Wood)
23240 · check: Add stats showing total files matched. (Dario Guzik)
23242 · Allow rename/delete open files under Windows (Nick Craig-Wood)
23244 · lsjson: Use exactly the correct number of decimal places in the
23246 seconds (Nick Craig-Wood)
23247 · Add cookie support with cmdline switch --use-cookies for all HTTP
23249 based remotes (qip)
23250 · Warn if --checksum is set but there are no hashes available (Nick
23252 Craig-Wood)
23253 · Rework rate limiting (pacer) to be more accurate and allow bursting
23255 (Nick Craig-Wood)
23256 · Improve error reporting for too many/few arguments in commands
23258 (Nick Craig-Wood)
23259 · listremotes: Remove -l short flag as it conflicts with the new
23261 global flag (weetmuts)
23262 · Make http serving with auth generate INFO messages on auth fail
23264 (Nick Craig-Wood)
23265 · Bug Fixes
23267 · Fix layout of stats (Nick Craig-Wood)
23269 · Fix --progress crash under Windows Jenkins (Nick Craig-Wood)
23271 · Fix transfer of google/onedrive docs by calling Rcat in Copy when
23273 size is -1 (Cnly)
23274 · copyurl: Fix checking of --dry-run (Denis Skovpen)
23276 · Mount
23278 · Check that mountpoint and local directory to mount don’t overlap
23280 (Nick Craig-Wood)
23281 · Fix mount size under 32 bit Windows (Nick Craig-Wood)
23283 · VFS
23285 · Implement renaming of directories for backends without DirMove
23287 (Nick Craig-Wood)
23288 · now all backends except b2 support renaming directories
23290 · Implement --vfs-cache-max-size to limit the total size of the cache
23292 (Nick Craig-Wood)
23293 · Add --dir-perms and --file-perms flags to set default permissions
23295 (Nick Craig-Wood)
23296 · Fix deadlock on concurrent operations on a directory (Nick
23298 Craig-Wood)
23299 · Fix deadlock between RWFileHandle.close and File.Remove (Nick
23301 Craig-Wood)
23302 · Fix renaming/deleting open files with cache mode “writes” under
23304 Windows (Nick Craig-Wood)
23305 · Fix panic on rename with --dry-run set (Nick Craig-Wood)
23307 · Fix vfs/refresh with recurse=true needing the --fast-list flag
23309 · Local
23311 · Add support for -l/--links (symbolic link translation) (yair@uni‐
23313 corn)
23314 · this works by showing links as link.rclonelink - see local back‐
23316 end docs for more info
23317 · this errors if used with -L/--copy-links
23319 · Fix renaming/deleting open files on Windows (Nick Craig-Wood)
23321 · Crypt
23323 · Check for maximum length before decrypting filename to fix panic
23325 (Garry McNulty)
23326 · Azure Blob
23328 · Allow building azureblob backend on *BSD (themylogin)
23330 · Use the rclone HTTP client to support --dump headers, --tpslimit
23332 etc (Nick Craig-Wood)
23333 · Use the s3 pacer for 0 delay in non error conditions (Nick
23335 Craig-Wood)
23336 · Ignore directory markers (Nick Craig-Wood)
23338 · Stop Mkdir attempting to create existing containers (Nick
23340 Craig-Wood)
23341 · B2
23343 · cleanup: will remove unfinished large files >24hrs old (Garry Mc‐
23345 Nulty)
23346 · For a bucket limited application key check the bucket name (Nick
23348 Craig-Wood)
23349 · before this, rclone would use the authorised bucket regardless of
23351 what you put on the command line
23352 · Added --b2-disable-checksum flag (Wojciech Smigielski)
23354 · this enables large files to be uploaded without a SHA-1 hash for
23356 speed reasons
23357 · Drive
23359 · Set default pacer to 100ms for 10 tps (Nick Craig-Wood)
23361 · This fits the Google defaults much better and reduces the 403 er‐
23363 rors massively
23364 · Add --drive-pacer-min-sleep and --drive-pacer-burst to control
23366 the pacer
23367 · Improve ChangeNotify support for items with multiple parents (Fabi‐
23369 an Möller)
23370 · Fix ListR for items with multiple parents - this fixes oddities
23372 with vfs/refresh (Fabian Möller)
23373 · Fix using --drive-impersonate and appfolders (Nick Craig-Wood)
23375 · Fix google docs in rclone mount for some (not all) applications
23377 (Nick Craig-Wood)
23378 · Dropbox
23380 · Retry-After support for Dropbox backend (Mathieu Carbou)
23382 · FTP
23384 · Wait for 60 seconds for a connection to Close then declare it dead
23386 (Nick Craig-Wood)
23387 · helps with indefinite hangs on some FTP servers
23389 · Google Cloud Storage
23391 · Update google cloud storage endpoints (weetmuts)
23393 · HTTP
23395 · Add an example with username and password which is supported but
23397 wasn’t documented (Nick Craig-Wood)
23398 · Fix backend with --files-from and non-existent files (Nick
23400 Craig-Wood)
23401 · Hubic
23403 · Make error message more informative if authentication fails (Nick
23405 Craig-Wood)
23406 · Jottacloud
23408 · Resume and deduplication support (Oliver Heyme)
23410 · Use token auth for all API requests Don’t store password anymore
23412 (Sebastian Bünger)
23413 · Add support for 2-factor authentification (Sebastian Bünger)
23415 · Mega
23417 · Implement v2 account login which fixes logins for newer Mega ac‐
23419 counts (Nick Craig-Wood)
23420 · Return error if an unknown length file is attempted to be uploaded
23422 (Nick Craig-Wood)
23423 · Add new error codes for better error reporting (Nick Craig-Wood)
23425 · Onedrive
23427 · Fix broken support for “shared with me” folders (Alex Chen)
23429 · Fix root ID not normalised (Cnly)
23431 · Return err instead of panic on unknown-sized uploads (Cnly)
23433 · Qingstor
23435 · Fix go routine leak on multipart upload errors (Nick Craig-Wood)
23437 · Add upload chunk size/concurrency/cutoff control (Nick Craig-Wood)
23439 · Default --qingstor-upload-concurrency to 1 to work around bug (Nick
23441 Craig-Wood)
23442 · S3
23444 · Implement --s3-upload-cutoff for single part uploads below this
23446 (Nick Craig-Wood)
23447 · Change --s3-upload-concurrency default to 4 to increase perfomance
23449 (Nick Craig-Wood)
23450 · Add --s3-bucket-acl to control bucket ACL (Nick Craig-Wood)
23452 · Auto detect region for buckets on operation failure (Nick
23454 Craig-Wood)
23455 · Add GLACIER storage class (William Cocker)
23457 · Add Scaleway to s3 documentation (Rémy Léone)
23459 · Add AWS endpoint eu-north-1 (weetmuts)
23461 · SFTP
23463 · Add support for PEM encrypted private keys (Fabian Möller)
23465 · Add option to force the usage of an ssh-agent (Fabian Möller)
23467 · Perform environment variable expansion on key-file (Fabian Möller)
23469 · Fix rmdir on Windows based servers (eg CrushFTP) (Nick Craig-Wood)
23471 · Fix rmdir deleting directory contents on some SFTP servers (Nick
23473 Craig-Wood)
23474 · Fix error on dangling symlinks (Nick Craig-Wood)
23476 · Swift
23478 · Add --swift-no-chunk to disable segmented uploads in rcat/mount
23480 (Nick Craig-Wood)
23481 · Introduce application credential auth support (kayrus)
23483 · Fix memory usage by slimming Object (Nick Craig-Wood)
23485 · Fix extra requests on upload (Nick Craig-Wood)
23487 · Fix reauth on big files (Nick Craig-Wood)
23489 · Union
23491 · Fix poll-interval not working (Nick Craig-Wood)
23493 · WebDAV
23495 · Support About which means rclone mount will show the correct disk
23497 size (Nick Craig-Wood)
23498 · Support MD5 and SHA1 hashes with Owncloud and Nextcloud (Nick
23500 Craig-Wood)
23501 · Fail soft on time parsing errors (Nick Craig-Wood)
23503 · Fix infinite loop on failed directory creation (Nick Craig-Wood)
23505 · Fix identification of directories for Bitrix Site Manager (Nick
23507 Craig-Wood)
23508 · Fix upload of 0 length files on some servers (Nick Craig-Wood)
23510 · Fix if MKCOL fails with 423 Locked assume the directory exists
23512 (Nick Craig-Wood)
23513 v1.45 - 2018-11-24
23515 · New backends
23516 · The Yandex backend was re-written - see below for details (Sebas‐
23518 tian Bünger)
23519 · New commands
23521 · rcd: New command just to serve the remote control API (Nick
23523 Craig-Wood)
23524 · New Features
23526 · The remote control API (rc) was greatly expanded to allow full con‐
23528 trol over rclone (Nick Craig-Wood)
23529 · sensitive operations require authorization or the --rc-no-auth
23531 flag
23532 · config/* operations to configure rclone
23534 · options/* for reading/setting command line flags
23536 · operations/* for all low level operations, eg copy file, list di‐
23538 rectory
23539 · sync/* for sync, copy and move
23541 · --rc-files flag to serve files on the rc http server
23543 · this is for building web native GUIs for rclone
23545 · Optionally serving objects on the rc http server
23547 · Ensure rclone fails to start up if the --rc port is in use al‐
23549 ready
23550 · See the rc docs (https://rclone.org/rc/) for more info
23552 · sync/copy/move
23554 · Make --files-from only read the objects specified and don’t scan
23556 directories (Nick Craig-Wood)
23557 · This is a huge speed improvement for destinations with lots of
23559 files
23560 · filter: Add --ignore-case flag (Nick Craig-Wood)
23562 · ncdu: Add remove function (`d' key) (Henning Surmeier)
23564 · rc command
23566 · Add --json flag for structured JSON input (Nick Craig-Wood)
23568 · Add --user and --pass flags and interpret --rc-user, --rc-pass,
23570 --rc-addr (Nick Craig-Wood)
23571 · build
23573 · Require go1.8 or later for compilation (Nick Craig-Wood)
23575 · Enable softfloat on MIPS arch (Scott Edlund)
23577 · Integration test framework revamped with a better report and bet‐
23579 ter retries (Nick Craig-Wood)
23580 · Bug Fixes
23582 · cmd: Make --progress update the stats correctly at the end (Nick
23584 Craig-Wood)
23585 · config: Create config directory on save if it is missing (Nick
23587 Craig-Wood)
23588 · dedupe: Check for existing filename before renaming a dupe file
23590 (ssaqua)
23591 · move: Don’t create directories with --dry-run (Nick Craig-Wood)
23593 · operations: Fix Purge and Rmdirs when dir is not the root (Nick
23595 Craig-Wood)
23596 · serve http/webdav/restic: Ensure rclone exits if the port is in use
23598 (Nick Craig-Wood)
23599 · Mount
23601 · Make --volname work for Windows and macOS (Nick Craig-Wood)
23603 · Azure Blob
23605 · Avoid context deadline exceeded error by setting a large TryTimeout
23607 value (brused27)
23608 · Fix erroneous Rmdir error “directory not empty” (Nick Craig-Wood)
23610 · Wait for up to 60s to create a just deleted container (Nick
23612 Craig-Wood)
23613 · Dropbox
23615 · Add dropbox impersonate support (Jake Coggiano)
23617 · Jottacloud
23619 · Fix bug in --fast-list handing of empty folders (albertony)
23621 · Opendrive
23623 · Fix transfer of files with + and & in (Nick Craig-Wood)
23625 · Fix retries of upload chunks (Nick Craig-Wood)
23627 · S3
23629 · Set ACL for server side copies to that provided by the user (Nick
23631 Craig-Wood)
23632 · Fix role_arn, credential_source, ... (Erik Swanson)
23634 · Add config info for Wasabi’s US-West endpoint (Henry Ptasinski)
23636 · SFTP
23638 · Ensure file hash checking is really disabled (Jon Fautley)
23640 · Swift
23642 · Add pacer for retries to make swift more reliable (Nick Craig-Wood)
23644 · WebDAV
23646 · Add Content-Type to PUT requests (Nick Craig-Wood)
23648 · Fix config parsing so --webdav-user and --webdav-pass flags work
23650 (Nick Craig-Wood)
23651 · Add RFC3339 date format (Ralf Hemberger)
23653 · Yandex
23655 · The yandex backend was re-written (Sebastian Bünger)
23657 · This implements low level retries (Sebastian Bünger)
23659 · Copy, Move, DirMove, PublicLink and About optional interfaces
23661 (Sebastian Bünger)
23662 · Improved general error handling (Sebastian Bünger)
23664 · Removed ListR for now due to inconsistent behaviour (Sebastian
23666 Bünger)
23667 v1.44 - 2018-10-15
23669 · New commands
23670 · serve ftp: Add ftp server (Antoine GIRARD)
23672 · settier: perform storage tier changes on supported remotes
23674 (sandeepkru)
23675 · New Features
23677 · Reworked command line help
23679 · Make default help less verbose (Nick Craig-Wood)
23681 · Split flags up into global and backend flags (Nick Craig-Wood)
23683 · Implement specialised help for flags and backends (Nick
23685 Craig-Wood)
23686 · Show URL of backend help page when starting config (Nick
23688 Craig-Wood)
23689 · stats: Long names now split in center (Joanna Marek)
23691 · Add --log-format flag for more control over log output (dcpu)
23693 · rc: Add support for OPTIONS and basic CORS (frenos)
23695 · stats: show FatalErrors and NoRetryErrors in stats (Cédric Connes)
23697 · Bug Fixes
23699 · Fix -P not ending with a new line (Nick Craig-Wood)
23701 · config: don’t create default config dir when user supplies --config
23703 (albertony)
23704 · Don’t print non-ASCII characters with --progress on windows (Nick
23706 Craig-Wood)
23707 · Correct logs for excluded items (ssaqua)
23709 · Mount
23711 · Remove EXPERIMENTAL tags (Nick Craig-Wood)
23713 · VFS
23715 · Fix race condition detected by serve ftp tests (Nick Craig-Wood)
23717 · Add vfs/poll-interval rc command (Fabian Möller)
23719 · Enable rename for nearly all remotes using server side Move or Copy
23721 (Nick Craig-Wood)
23722 · Reduce directory cache cleared by poll-interval (Fabian Möller)
23724 · Remove EXPERIMENTAL tags (Nick Craig-Wood)
23726 · Local
23728 · Skip bad symlinks in dir listing with -L enabled (Cédric Connes)
23730 · Preallocate files on Windows to reduce fragmentation (Nick
23732 Craig-Wood)
23733 · Preallocate files on linux with fallocate(2) (Nick Craig-Wood)
23735 · Cache
23737 · Add cache/fetch rc function (Fabian Möller)
23739 · Fix worker scale down (Fabian Möller)
23741 · Improve performance by not sending info requests for cached chunks
23743 (dcpu)
23744 · Fix error return value of cache/fetch rc method (Fabian Möller)
23746 · Documentation fix for cache-chunk-total-size (Anagh Kumar Baranwal)
23748 · Preserve leading / in wrapped remote path (Fabian Möller)
23750 · Add plex_insecure option to skip certificate validation (Fabian
23752 Möller)
23753 · Remove entries that no longer exist in the source (dcpu)
23755 · Crypt
23757 · Preserve leading / in wrapped remote path (Fabian Möller)
23759 · Alias
23761 · Fix handling of Windows network paths (Nick Craig-Wood)
23763 · Azure Blob
23765 · Add --azureblob-list-chunk parameter (Santiago Rodríguez)
23767 · Implemented settier command support on azureblob remote. (sandeep‐
23769 kru)
23770 · Work around SDK bug which causes errors for chunk-sized files (Nick
23772 Craig-Wood)
23773 · Box
23775 · Implement link sharing. (Sebastian Bünger)
23777 · Drive
23779 · Add --drive-import-formats - google docs can now be imported (Fabi‐
23781 an Möller)
23782 · Rewrite mime type and extension handling (Fabian Möller)
23784 · Add document links (Fabian Möller)
23786 · Add support for multipart document extensions (Fabian Möller)
23788 · Add support for apps-script to json export (Fabian Möller)
23790 · Fix escaped chars in documents during list (Fabian Möller)
23792 · Add --drive-v2-download-min-size a workaround for slow downloads
23794 (Fabian Möller)
23795 · Improve directory notifications in ChangeNotify (Fabian Möller)
23797 · When listing team drives in config, continue on failure (Nick
23799 Craig-Wood)
23800 · FTP
23802 · Add a small pause after failed upload before deleting file (Nick
23804 Craig-Wood)
23805 · Google Cloud Storage
23807 · Fix service_account_file being ignored (Fabian Möller)
23809 · Jottacloud
23811 · Minor improvement in quota info (omit if unlimited) (albertony)
23813 · Add --fast-list support (albertony)
23815 · Add permanent delete support: --jottacloud-hard-delete (albertony)
23817 · Add link sharing support (albertony)
23819 · Fix handling of reserved characters. (Sebastian Bünger)
23821 · Fix socket leak on Object.Remove (Nick Craig-Wood)
23823 · Onedrive
23825 · Rework to support Microsoft Graph (Cnly)
23827 · NB this will require re-authenticating the remote
23829 · Removed upload cutoff and always do session uploads (Oliver Heyme)
23831 · Use single-part upload for empty files (Cnly)
23833 · Fix new fields not saved when editing old config (Alex Chen)
23835 · Fix sometimes special chars in filenames not replaced (Alex Chen)
23837 · Ignore OneNote files by default (Alex Chen)
23839 · Add link sharing support (jackyzy823)
23841 · S3
23843 · Use custom pacer, to retry operations when reasonable (Craig
23845 Miskell)
23846 · Use configured server-side-encryption and storace class options
23848 when calling CopyObject() (Paul Kohout)
23849 · Make --s3-v2-auth flag (Nick Craig-Wood)
23851 · Fix v2 auth on files with spaces (Nick Craig-Wood)
23853 · Union
23855 · Implement union backend which reads from multiple backends (Felix
23857 Brucker)
23858 · Implement optional interfaces (Move, DirMove, Copy etc) (Nick
23860 Craig-Wood)
23861 · Fix ChangeNotify to support multiple remotes (Fabian Möller)
23863 · Fix --backup-dir on union backend (Nick Craig-Wood)
23865 · WebDAV
23867 · Add another time format (Nick Craig-Wood)
23869 · Add a small pause after failed upload before deleting file (Nick
23871 Craig-Wood)
23872 · Add workaround for missing mtime (buergi)
23874 · Sharepoint: Renew cookies after 12hrs (Henning Surmeier)
23876 · Yandex
23878 · Remove redundant nil checks (teresy)
23880 v1.43.1 - 2018-09-07
23882 Point release to fix hubic and azureblob backends.
23883 · Bug Fixes
23885 · ncdu: Return error instead of log.Fatal in Show (Fabian Möller)
23887 · cmd: Fix crash with --progress and --stats 0 (Nick Craig-Wood)
23889 · docs: Tidy website display (Anagh Kumar Baranwal)
23891 · Azure Blob:
23893 · Fix multi-part uploads. (sandeepkru)
23895 · Hubic
23897 · Fix uploads (Nick Craig-Wood)
23899 · Retry auth fetching if it fails to make hubic more reliable (Nick
23901 Craig-Wood)
23902 v1.43 - 2018-09-01
23904 · New backends
23905 · Jottacloud (Sebastian Bünger)
23907 · New commands
23909 · copyurl: copies a URL to a remote (Denis)
23911 · New Features
23913 · Reworked config for backends (Nick Craig-Wood)
23915 · All backend config can now be supplied by command line, env var
23917 or config file
23918 · Advanced section in the config wizard for the optional items
23920 · A large step towards rclone backends being usable in other go
23922 software
23923 · Allow on the fly remotes with :backend: syntax
23925 · Stats revamp
23927 · Add --progress/-P flag to show interactive progress (Nick
23929 Craig-Wood)
23930 · Show the total progress of the sync in the stats (Nick
23932 Craig-Wood)
23933 · Add --stats-one-line flag for single line stats (Nick Craig-Wood)
23935 · Added weekday schedule into --bwlimit (Mateusz)
23937 · lsjson: Add option to show the original object IDs (Fabian Möller)
23939 · serve webdav: Make Content-Type without reading the file and add
23941 --etag-hash (Nick Craig-Wood)
23942 · build
23944 · Build macOS with native compiler (Nick Craig-Wood)
23946 · Update to use go1.11 for the build (Nick Craig-Wood)
23948 · rc
23950 · Added core/stats to return the stats (reddi1)
23952 · version --check: Prints the current release and beta versions (Nick
23954 Craig-Wood)
23955 · Bug Fixes
23957 · accounting
23959 · Fix time to completion estimates (Nick Craig-Wood)
23961 · Fix moving average speed for file stats (Nick Craig-Wood)
23963 · config: Fix error reading password from piped input (Nick
23965 Craig-Wood)
23966 · move: Fix --delete-empty-src-dirs flag to delete all empty dirs on
23968 move (ishuah)
23969 · Mount
23971 · Implement --daemon-timeout flag for OSXFUSE (Nick Craig-Wood)
23973 · Fix mount --daemon not working with encrypted config (Alex Chen)
23975 · Clip the number of blocks to 2^32-1 on macOS - fixes borg backup
23977 (Nick Craig-Wood)
23978 · VFS
23980 · Enable vfs-read-chunk-size by default (Fabian Möller)
23982 · Add the vfs/refresh rc command (Fabian Möller)
23984 · Add non recursive mode to vfs/refresh rc command (Fabian Möller)
23986 · Try to seek buffer on read only files (Fabian Möller)
23988 · Local
23990 · Fix crash when deprecated --local-no-unicode-normalization is sup‐
23992 plied (Nick Craig-Wood)
23993 · Fix mkdir error when trying to copy files to the root of a drive on
23995 windows (Nick Craig-Wood)
23996 · Cache
23998 · Fix nil pointer deref when using lsjson on cached directory (Nick
24000 Craig-Wood)
24001 · Fix nil pointer deref for occasional crash on playback (Nick
24003 Craig-Wood)
24004 · Crypt
24006 · Fix accounting when checking hashes on upload (Nick Craig-Wood)
24008 · Amazon Cloud Drive
24010 · Make very clear in the docs that rclone has no ACD keys (Nick
24012 Craig-Wood)
24013 · Azure Blob
24015 · Add connection string and SAS URL auth (Nick Craig-Wood)
24017 · List the container to see if it exists (Nick Craig-Wood)
24019 · Port new Azure Blob Storage SDK (sandeepkru)
24021 · Added blob tier, tier between Hot, Cool and Archive. (sandeepkru)
24023 · Remove leading / from paths (Nick Craig-Wood)
24025 · B2
24027 · Support Application Keys (Nick Craig-Wood)
24029 · Remove leading / from paths (Nick Craig-Wood)
24031 · Box
24033 · Fix upload of > 2GB files on 32 bit platforms (Nick Craig-Wood)
24035 · Make --box-commit-retries flag defaulting to 100 to fix large up‐
24037 loads (Nick Craig-Wood)
24038 · Drive
24040 · Add --drive-keep-revision-forever flag (lewapm)
24042 · Handle gdocs when filtering file names in list (Fabian Möller)
24044 · Support using --fast-list for large speedups (Fabian Möller)
24046 · FTP
24048 · Fix Put mkParentDir failed: 521 for BunnyCDN (Nick Craig-Wood)
24050 · Google Cloud Storage
24052 · Fix index out of range error with --fast-list (Nick Craig-Wood)
24054 · Jottacloud
24056 · Fix MD5 error check (Oliver Heyme)
24058 · Handle empty time values (Martin Polden)
24060 · Calculate missing MD5s (Oliver Heyme)
24062 · Docs, fixes and tests for MD5 calculation (Nick Craig-Wood)
24064 · Add optional MimeTyper interface. (Sebastian Bünger)
24066 · Implement optional About interface (for df support). (Sebastian
24068 Bünger)
24069 · Mega
24071 · Wait for events instead of arbitrary sleeping (Nick Craig-Wood)
24073 · Add --mega-hard-delete flag (Nick Craig-Wood)
24075 · Fix failed logins with upper case chars in email (Nick Craig-Wood)
24077 · Onedrive
24079 · Shared folder support (Yoni Jah)
24081 · Implement DirMove (Cnly)
24083 · Fix rmdir sometimes deleting directories with contents (Nick
24085 Craig-Wood)
24086 · Pcloud
24088 · Delete half uploaded files on upload error (Nick Craig-Wood)
24090 · Qingstor
24092 · Remove leading / from paths (Nick Craig-Wood)
24094 · S3
24096 · Fix index out of range error with --fast-list (Nick Craig-Wood)
24098 · Add --s3-force-path-style (Nick Craig-Wood)
24100 · Add support for KMS Key ID (bsteiss)
24102 · Remove leading / from paths (Nick Craig-Wood)
24104 · Swift
24106 · Add storage_policy (Ruben Vandamme)
24108 · Make it so just storage_url or auth_token can be overidden (Nick
24110 Craig-Wood)
24111 · Fix server side copy bug for unusal file names (Nick Craig-Wood)
24113 · Remove leading / from paths (Nick Craig-Wood)
24115 · WebDAV
24117 · Ensure we call MKCOL with a URL with a trailing / for QNAP interop
24119 (Nick Craig-Wood)
24120 · If root ends with / then don’t check if it is a file (Nick
24122 Craig-Wood)
24123 · Don’t accept redirects when reading metadata (Nick Craig-Wood)
24125 · Add bearer token (Macaroon) support for dCache (Nick Craig-Wood)
24127 · Document dCache and Macaroons (Onno Zweers)
24129 · Sharepoint recursion with different depth (Henning)
24131 · Attempt to remove failed uploads (Nick Craig-Wood)
24133 · Yandex
24135 · Fix listing/deleting files in the root (Nick Craig-Wood)
24137 v1.42 - 2018-06-16
24139 · New backends
24140 · OpenDrive (Oliver Heyme, Jakub Karlicek, ncw)
24142 · New commands
24144 · deletefile command (Filip Bartodziej)
24146 · New Features
24148 · copy, move: Copy single files directly, don’t use --files-from
24150 work-around
24151 · this makes them much more efficient
24153 · Implement --max-transfer flag to quit transferring at a limit
24155 · make exit code 8 for --max-transfer exceeded
24157 · copy: copy empty source directories to destination (Ishuah Kariuki)
24159 · check: Add --one-way flag (Kasper Byrdal Nielsen)
24161 · Add siginfo handler for macOS for ctrl-T stats (kubatasiemski)
24163 · rc
24165 · add core/gc to run a garbage collection on demand
24167 · enable go profiling by default on the --rc port
24169 · return error from remote on failure
24171 · lsf
24173 · Add --absolute flag to add a leading / onto path names
24175 · Add --csv flag for compliant CSV output
24177 · Add `m' format specifier to show the MimeType
24179 · Implement `i' format for showing object ID
24181 · lsjson
24183 · Add MimeType to the output
24185 · Add ID field to output to show Object ID
24187 · Add --retries-sleep flag (Benjamin Joseph Dag)
24189 · Oauth tidy up web page and error handling (Henning Surmeier)
24191 · Bug Fixes
24193 · Password prompt output with --log-file fixed for unix (Filip Bar‐
24195 todziej)
24196 · Calculate ModifyWindow each time on the fly to fix various problems
24198 (Stefan Breunig)
24199 · Mount
24201 · Only print “File.rename error” if there actually is an error (Ste‐
24203 fan Breunig)
24204 · Delay rename if file has open writers instead of failing outright
24206 (Stefan Breunig)
24207 · Ensure atexit gets run on interrupt
24209 · macOS enhancements
24211 · Make --noappledouble --noapplexattr
24213 · Add --volname flag and remove special chars from it
24215 · Make Get/List/Set/Remove xattr return ENOSYS for efficiency
24217 · Make --daemon work for macOS without CGO
24219 · VFS
24221 · Add --vfs-read-chunk-size and --vfs-read-chunk-size-limit (Fabian
24223 Möller)
24224 · Fix ChangeNotify for new or changed folders (Fabian Möller)
24226 · Local
24228 · Fix symlink/junction point directory handling under Windows
24230 · NB you will need to add -L to your command line to copy files
24232 with reparse points
24233 · Cache
24235 · Add non cached dirs on notifications (Remus Bunduc)
24237 · Allow root to be expired from rc (Remus Bunduc)
24239 · Clean remaining empty folders from temp upload path (Remus Bunduc)
24241 · Cache lists using batch writes (Remus Bunduc)
24243 · Use secure websockets for HTTPS Plex addresses (John Clayton)
24245 · Reconnect plex websocket on failures (Remus Bunduc)
24247 · Fix panic when running without plex configs (Remus Bunduc)
24249 · Fix root folder caching (Remus Bunduc)
24251 · Crypt
24253 · Check the crypted hash of files when uploading for extra data secu‐
24255 rity
24256 · Dropbox
24258 · Make Dropbox for business folders accessible using an initial / in
24260 the path
24261 · Google Cloud Storage
24263 · Low level retry all operations if necessary
24265 · Google Drive
24267 · Add --drive-acknowledge-abuse to download flagged files
24269 · Add --drive-alternate-export to fix large doc export
24271 · Don’t attempt to choose Team Drives when using rclone config create
24273 · Fix change list polling with team drives
24275 · Fix ChangeNotify for folders (Fabian Möller)
24277 · Fix about (and df on a mount) for team drives
24279 · Onedrive
24281 · Errorhandler for onedrive for business requests (Henning Surmeier)
24283 · S3
24285 · Adjust upload concurrency with --s3-upload-concurrency (themylogin)
24287 · Fix --s3-chunk-size which was always using the minimum
24289 · SFTP
24291 · Add --ssh-path-override flag (Piotr Oleszczyk)
24293 · Fix slow downloads for long latency connections
24295 · Webdav
24297 · Add workarounds for biz.mail.ru
24299 · Ignore Reason-Phrase in status line to fix 4shared (Rodrigo)
24301 · Better error message generation
24303 v1.41 - 2018-04-28
24305 · New backends
24306 · Mega support added
24308 · Webdav now supports SharePoint cookie authentication (hensur)
24310 · New commands
24312 · link: create public link to files and folders (Stefan Breunig)
24314 · about: gets quota info from a remote (a-roussos, ncw)
24316 · hashsum: a generic tool for any hash to produce md5sum like output
24318 · New Features
24320 · lsd: Add -R flag and fix and update docs for all ls commands
24322 · ncdu: added a “refresh” key - CTRL-L (Keith Goldfarb)
24324 · serve restic: Add append-only mode (Steve Kriss)
24326 · serve restic: Disallow overwriting files in append-only mode
24328 (Alexander Neumann)
24329 · serve restic: Print actual listener address (Matt Holt)
24331 · size: Add –json flag (Matthew Holt)
24333 · sync: implement –ignore-errors (Mateusz Pabian)
24335 · dedupe: Add dedupe largest functionality (Richard Yang)
24337 · fs: Extend SizeSuffix to include TB and PB for rclone about
24339 · fs: add –dump goroutines and –dump openfiles for debugging
24341 · rc: implement core/memstats to print internal memory usage info
24343 · rc: new call rc/pid (Michael P. Dubner)
24345 · Compile
24347 · Drop support for go1.6
24349 · Release
24351 · Fix make tarball (Chih-Hsuan Yen)
24353 · Bug Fixes
24355 · filter: fix –min-age and –max-age together check
24357 · fs: limit MaxIdleConns and MaxIdleConnsPerHost in transport
24359 · lsd,lsf: make sure all times we output are in local time
24361 · rc: fix setting bwlimit to unlimited
24363 · rc: take note of the –rc-addr flag too as per the docs
24365 · Mount
24367 · Use About to return the correct disk total/used/free (eg in df)
24369 · Set --attr-timeout default to 1s - fixes:
24371 · rclone using too much memory
24373 · rclone not serving files to samba
24375 · excessive time listing directories
24377 · Fix df -i (upstream fix)
24379 · VFS
24381 · Filter files . and .. from directory listing
24383 · Only make the VFS cache if –vfs-cache-mode > Off
24385 · Local
24387 · Add –local-no-check-updated to disable updated file checks
24389 · Retry remove on Windows sharing violation error
24391 · Cache
24393 · Flush the memory cache after close
24395 · Purge file data on notification
24397 · Always forget parent dir for notifications
24399 · Integrate with Plex websocket
24401 · Add rc cache/stats (seuffert)
24403 · Add info log on notification
24405 · Box
24407 · Fix failure reading large directories - parse file/directory size
24409 as float
24410 · Dropbox
24412 · Fix crypt+obfuscate on dropbox
24414 · Fix repeatedly uploading the same files
24416 · FTP
24418 · Work around strange response from box FTP server
24420 · More workarounds for FTP servers to fix mkParentDir error
24422 · Fix no error on listing non-existent directory
24424 · Google Cloud Storage
24426 · Add service_account_credentials (Matt Holt)
24428 · Detect bucket presence by listing it - minimises permissions needed
24430 · Ignore zero length directory markers
24432 · Google Drive
24434 · Add service_account_credentials (Matt Holt)
24436 · Fix directory move leaving a hardlinked directory behind
24438 · Return proper google errors when Opening files
24440 · When initialized with a filepath, optional features used incorrect
24442 root path (Stefan Breunig)
24443 · HTTP
24445 · Fix sync for servers which don’t return Content-Length in HEAD
24447 · Onedrive
24449 · Add QuickXorHash support for OneDrive for business
24451 · Fix socket leak in multipart session upload
24453 · S3
24455 · Look in S3 named profile files for credentials
24457 · Add --s3-disable-checksum to disable checksum uploading (Chris Re‐
24459 dekop)
24460 · Hierarchical configuration support (Giri Badanahatti)
24462 · Add in config for all the supported S3 providers
24464 · Add One Zone Infrequent Access storage class (Craig Rachel)
24466 · Add –use-server-modtime support (Peter Baumgartner)
24468 · Add –s3-chunk-size option to control multipart uploads
24470 · Ignore zero length directory markers
24472 · SFTP
24474 · Update docs to match code, fix typos and clarify disable_hashcheck
24476 prompt (Michael G. Noll)
24477 · Update docs with Synology quirks
24479 · Fail soft with a debug on hash failure
24481 · Swift
24483 · Add –use-server-modtime support (Peter Baumgartner)
24485 · Webdav
24487 · Support SharePoint cookie authentication (hensur)
24489 · Strip leading and trailing / off root
24491 v1.40 - 2018-03-19
24493 · New backends
24494 · Alias backend to create aliases for existing remote names (Fabian
24496 Möller)
24497 · New commands
24499 · lsf: list for parsing purposes (Jakub Tasiemski)
24501 · by default this is a simple non recursive list of files and di‐
24503 rectories
24504 · it can be configured to add more info in an easy to parse way
24506 · serve restic: for serving a remote as a Restic REST endpoint
24508 · This enables restic to use any backends that rclone can access
24510 · Thanks Alexander Neumann for help, patches and review
24512 · rc: enable the remote control of a running rclone
24514 · The running rclone must be started with –rc and related flags.
24516 · Currently there is support for bwlimit, and flushing for mount
24518 and cache.
24519 · New Features
24521 · --max-delete flag to add a delete threshold (Bjørn Erik Pedersen)
24523 · All backends now support RangeOption for ranged Open
24525 · cat: Use RangeOption for limited fetches to make more efficient
24527 · cryptcheck: make reading of nonce more efficient with RangeOption
24529 · serve http/webdav/restic
24531 · support SSL/TLS
24533 · add --user --pass and --htpasswd for authentication
24535 · copy/move: detect file size change during copy/move and abort
24537 transfer (ishuah)
24538 · cryptdecode: added option to return encrypted file names. (ishuah)
24540 · lsjson: add --encrypted to show encrypted name (Jakub Tasiemski)
24542 · Add --stats-file-name-length to specify the printed file name
24544 length for stats (Will Gunn)
24545 · Compile
24547 · Code base was shuffled and factored
24549 · backends moved into a backend directory
24551 · large packages split up
24553 · See the CONTRIBUTING.md doc for info as to what lives where now
24555 · Update to using go1.10 as the default go version
24557 · Implement daily full integration tests (https://pub.rclone.org/in‐
24559 tegration-tests/)
24560 · Release
24562 · Include a source tarball and sign it and the binaries
24564 · Sign the git tags as part of the release process
24566 · Add .deb and .rpm packages as part of the build
24568 · Make a beta release for all branches on the main repo (but not pull
24570 requests)
24571 · Bug Fixes
24573 · config: fixes errors on non existing config by loading config file
24575 only on first access
24576 · config: retry saving the config after failure (Mateusz)
24578 · sync: when using --backup-dir don’t delete files if we can’t set
24580 their modtime
24581 · this fixes odd behaviour with Dropbox and --backup-dir
24583 · fshttp: fix idle timeouts for HTTP connections
24585 · serve http: fix serving files with : in - fixes
24587 · Fix --exclude-if-present to ignore directories which it doesn’t
24589 have permission for (Iakov Davydov)
24590 · Make accounting work properly with crypt and b2
24592 · remove --no-traverse flag because it is obsolete
24594 · Mount
24596 · Add --attr-timeout flag to control attribute caching in kernel
24598 · this now defaults to 0 which is correct but less efficient
24600 · see the mount docs (/commands/rclone_mount/#attribute-caching)
24602 for more info
24603 · Add --daemon flag to allow mount to run in the background (ishuah)
24605 · Fix: Return ENOSYS rather than EIO on attempted link
24607 · This fixes FileZilla accessing an rclone mount served over sftp.
24609 · Fix setting modtime twice
24611 · Mount tests now run on CI for Linux (mount & cmount)/Mac/Windows
24613 · Many bugs fixed in the VFS layer - see below
24615 · VFS
24617 · Many fixes for --vfs-cache-mode writes and above
24619 · Update cached copy if we know it has changed (fixes stale data)
24621 · Clean path names before using them in the cache
24623 · Disable cache cleaner if --vfs-cache-poll-interval=0
24625 · Fill and clean the cache immediately on startup
24627 · Fix Windows opening every file when it stats the file
24629 · Fix applying modtime for an open Write Handle
24631 · Fix creation of files when truncating
24633 · Write 0 bytes when flushing unwritten handles to avoid race condi‐
24635 tions in FUSE
24636 · Downgrade “poll-interval is not supported” message to Info
24638 · Make OpenFile and friends return EINVAL if O_RDONLY and O_TRUNC
24640 · Local
24642 · Downgrade “invalid cross-device link: trying copy” to debug
24644 · Make DirMove return fs.ErrorCantDirMove to allow fallback to Copy
24646 for cross device
24647 · Fix race conditions updating the hashes
24649 · Cache
24651 · Add support for polling - cache will update when remote changes on
24653 supported backends
24654 · Reduce log level for Plex api
24656 · Fix dir cache issue
24658 · Implement --cache-db-wait-time flag
24660 · Improve efficiency with RangeOption and RangeSeek
24662 · Fix dirmove with temp fs enabled
24664 · Notify vfs when using temp fs
24666 · Offline uploading
24668 · Remote control support for path flushing
24670 · Amazon cloud drive
24672 · Rclone no longer has any working keys - disable integration tests
24674 · Implement DirChangeNotify to notify cache/vfs/mount of changes
24676 · Azureblob
24678 · Don’t check for bucket/container presense if listing was OK
24680 · this makes rclone do one less request per invocation
24682 · Improve accounting for chunked uploads
24684 · Backblaze B2
24686 · Don’t check for bucket/container presense if listing was OK
24688 · this makes rclone do one less request per invocation
24690 · Box
24692 · Improve accounting for chunked uploads
24694 · Dropbox
24696 · Fix custom oauth client parameters
24698 · Google Cloud Storage
24700 · Don’t check for bucket/container presense if listing was OK
24702 · this makes rclone do one less request per invocation
24704 · Google Drive
24706 · Migrate to api v3 (Fabian Möller)
24708 · Add scope configuration and root folder selection
24710 · Add --drive-impersonate for service accounts
24712 · thanks to everyone who tested, explored and contributed docs
24714 · Add --drive-use-created-date to use created date as modified date
24716 (nbuchanan)
24717 · Request the export formats only when required
24719 · This makes rclone quicker when there are no google docs
24721 · Fix finding paths with latin1 chars (a workaround for a drive bug)
24723 · Fix copying of a single Google doc file
24725 · Fix --drive-auth-owner-only to look in all directories
24727 · HTTP
24729 · Fix handling of directories with & in
24731 · Onedrive
24733 · Removed upload cutoff and always do session uploads
24735 · this stops the creation of multiple versions on business onedrive
24737 · Overwrite object size value with real size when reading file.
24739 (Victor)
24740 · this fixes oddities when onedrive misreports the size of images
24742 · Pcloud
24744 · Remove unused chunked upload flag and code
24746 · Qingstor
24748 · Don’t check for bucket/container presense if listing was OK
24750 · this makes rclone do one less request per invocation
24752 · S3
24754 · Support hashes for multipart files (Chris Redekop)
24756 · Initial support for IBM COS (S3) (Giri Badanahatti)
24758 · Update docs to discourage use of v2 auth with CEPH and others
24760 · Don’t check for bucket/container presense if listing was OK
24762 · this makes rclone do one less request per invocation
24764 · Fix server side copy and set modtime on files with + in
24766 · SFTP
24768 · Add option to disable remote hash check command execution (Jon
24770 Fautley)
24771 · Add --sftp-ask-password flag to prompt for password when needed
24773 (Leo R. Lundgren)
24774 · Add set_modtime configuration option
24776 · Fix following of symlinks
24778 · Fix reading config file outside of Fs setup
24780 · Fix reading $USER in username fallback not $HOME
24782 · Fix running under crontab - Use correct OS way of reading username
24784 · Swift
24786 · Fix refresh of authentication token
24788 · in v1.39 a bug was introduced which ignored new tokens - this
24790 fixes it
24791 · Fix extra HEAD transaction when uploading a new file
24793 · Don’t check for bucket/container presense if listing was OK
24795 · this makes rclone do one less request per invocation
24797 · Webdav
24799 · Add new time formats to support mydrive.ch and others
24801 v1.39 - 2017-12-23
24803 · New backends
24804 · WebDAV
24806 · tested with nextcloud, owncloud, put.io and others!
24808 · Pcloud
24810 · cache - wraps a cache around other backends (Remus Bunduc)
24812 · useful in combination with mount
24814 · NB this feature is in beta so use with care
24816 · New commands
24818 · serve command with subcommands:
24820 · serve webdav: this implements a webdav server for any rclone re‐
24822 mote.
24823 · serve http: command to serve a remote over HTTP
24825 · config: add sub commands for full config file management
24827 · create/delete/dump/edit/file/password/providers/show/update
24829 · touch: to create or update the timestamp of a file (Jakub Tasiems‐
24831 ki)
24832 · New Features
24834 · curl install for rclone (Filip Bartodziej)
24836 · –stats now shows percentage, size, rate and ETA in condensed form
24838 (Ishuah Kariuki)
24839 · –exclude-if-present to exclude a directory if a file is present
24841 (Iakov Davydov)
24842 · rmdirs: add –leave-root flag (lewpam)
24844 · move: add –delete-empty-src-dirs flag to remove dirs after move
24846 (Ishuah Kariuki)
24847 · Add –dump flag, introduce –dump requests, responses and remove
24849 –dump-auth, –dump-filters
24850 · Obscure X-Auth-Token: from headers when dumping too
24852 · Document and implement exit codes for different failure modes
24854 (Ishuah Kariuki)
24855 · Compile
24857 · Bug Fixes
24859 · Retry lots more different types of errors to make multipart trans‐
24861 fers more reliable
24862 · Save the config before asking for a token, fixes disappearing oauth
24864 config
24865 · Warn the user if –include and –exclude are used together (Ernest
24867 Borowski)
24868 · Fix duplicate files (eg on Google drive) causing spurious copies
24870 · Allow trailing and leading whitespace for passwords (Jason Rose)
24872 · ncdu: fix crashes on empty directories
24874 · rcat: fix goroutine leak
24876 · moveto/copyto: Fix to allow copying to the same name
24878 · Mount
24880 · –vfs-cache mode to make writes into mounts more reliable.
24882 · this requires caching files on the disk (see –cache-dir)
24884 · As this is a new feature, use with care
24886 · Use sdnotify to signal systemd the mount is ready (Fabian Möller)
24888 · Check if directory is not empty before mounting (Ernest Borowski)
24890 · Local
24892 · Add error message for cross file system moves
24894 · Fix equality check for times
24896 · Dropbox
24898 · Rework multipart upload
24900 · buffer the chunks when uploading large files so they can be re‐
24902 tried
24903 · change default chunk size to 48MB now we are buffering them in
24905 memory
24906 · retry every error after the first chunk is done successfully
24908 · Fix error when renaming directories
24910 · Swift
24912 · Fix crash on bad authentication
24914 · Google Drive
24916 · Add service account support (Tim Cooijmans)
24918 · S3
24920 · Make it work properly with Digital Ocean Spaces (Andrew
24922 Starr-Bochicchio)
24923 · Fix crash if a bad listing is received
24925 · Add support for ECS task IAM roles (David Minor)
24927 · Backblaze B2
24929 · Fix multipart upload retries
24931 · Fix –hard-delete to make it work 100% of the time
24933 · Swift
24935 · Allow authentication with storage URL and auth key (Giovanni Pizzi)
24937 · Add new fields for swift configuration to support IBM Bluemix Swift
24939 (Pierre Carlson)
24940 · Add OS_TENANT_ID and OS_USER_ID to config
24942 · Allow configs with user id instead of user name
24944 · Check if swift segments container exists before creating (John
24946 Leach)
24947 · Fix memory leak in swift transfers (upstream fix)
24949 · SFTP
24951 · Add option to enable the use of aes128-cbc cipher (Jon Fautley)
24953 · Amazon cloud drive
24955 · Fix download of large files failing with “Only one auth mechanism
24957 allowed”
24958 · crypt
24960 · Option to encrypt directory names or leave them intact
24962 · Implement DirChangeNotify (Fabian Möller)
24964 · onedrive
24966 · Add option to choose resourceURL during setup of OneDrive Business
24968 account if more than one is available for user
24969 v1.38 - 2017-09-30
24971 · New backends
24972 · Azure Blob Storage (thanks Andrei Dragomir)
24974 · Box
24976 · Onedrive for Business (thanks Oliver Heyme)
24978 · QingStor from QingCloud (thanks wuyu)
24980 · New commands
24982 · rcat - read from standard input and stream upload
24984 · tree - shows a nicely formatted recursive listing
24986 · cryptdecode - decode crypted file names (thanks ishuah)
24988 · config show - print the config file
24990 · config file - print the config file location
24992 · New Features
24994 · Empty directories are deleted on sync
24996 · dedupe - implement merging of duplicate directories
24998 · check and cryptcheck made more consistent and use less memory
25000 · cleanup for remaining remotes (thanks ishuah)
25002 · --immutable for ensuring that files don’t change (thanks Jacob Mc‐
25004 Namee)
25005 · --user-agent option (thanks Alex McGrath Kraak)
25007 · --disable flag to disable optional features
25009 · --bind flag for choosing the local addr on outgoing connections
25011 · Support for zsh auto-completion (thanks bpicode)
25013 · Stop normalizing file names but do a normalized compare in sync
25015 · Compile
25017 · Update to using go1.9 as the default go version
25019 · Remove snapd build due to maintenance problems
25021 · Bug Fixes
25023 · Improve retriable error detection which makes multipart uploads
25025 better
25026 · Make check obey --ignore-size
25028 · Fix bwlimit toggle in conjunction with schedules (thanks cbruegg)
25030 · config ensures newly written config is on the same mount
25032 · Local
25034 · Revert to copy when moving file across file system boundaries
25036 · --skip-links to suppress symlink warnings (thanks Zhiming Wang)
25038 · Mount
25040 · Re-use rcat internals to support uploads from all remotes
25042 · Dropbox
25044 · Fix “entry doesn’t belong in directory” error
25046 · Stop using deprecated API methods
25048 · Swift
25050 · Fix server side copy to empty container with --fast-list
25052 · Google Drive
25054 · Change the default for --drive-use-trash to true
25056 · S3
25058 · Set session token when using STS (thanks Girish Ramakrishnan)
25060 · Glacier docs and error messages (thanks Jan Varho)
25062 · Read 1000 (not 1024) items in dir listings to fix Wasabi
25064 · Backblaze B2
25066 · Fix SHA1 mismatch when downloading files with no SHA1
25068 · Calculate missing hashes on the fly instead of spooling
25070 · --b2-hard-delete to permanently delete (not hide) files (thanks
25072 John Papandriopoulos)
25073 · Hubic
25075 · Fix creating containers - no longer have to use the default con‐
25077 tainer
25078 · Swift
25080 · Optionally configure from a standard set of OpenStack environment
25082 vars
25083 · Add endpoint_type config
25085 · Google Cloud Storage
25087 · Fix bucket creation to work with limited permission users
25089 · SFTP
25091 · Implement connection pooling for multiple ssh connections
25093 · Limit new connections per second
25095 · Add support for MD5 and SHA1 hashes where available (thanks Chris‐
25097 tian Brüggemann)
25098 · HTTP
25100 · Fix URL encoding issues
25102 · Fix directories with : in
25104 · Fix panic with URL encoded content
25106 v1.37 - 2017-07-22
25108 · New backends
25109 · FTP - thanks to Antonio Messina
25111 · HTTP - thanks to Vasiliy Tolstov
25113 · New commands
25115 · rclone ncdu - for exploring a remote with a text based user inter‐
25117 face.
25118 · rclone lsjson - for listing with a machine readable output
25120 · rclone dbhashsum - to show Dropbox style hashes of files (local or
25122 Dropbox)
25123 · New Features
25125 · Implement –fast-list flag
25127 · This allows remotes to list recursively if they can
25129 · This uses less transactions (important if you pay for them)
25131 · This may or may not be quicker
25133 · This will use more memory as it has to hold the listing in memory
25135 · –old-sync-method deprecated - the remaining uses are covered by
25137 –fast-list
25138 · This involved a major re-write of all the listing code
25140 · Add –tpslimit and –tpslimit-burst to limit transactions per second
25142 · this is useful in conjuction with rclone mount to limit external
25144 apps
25145 · Add –stats-log-level so can see –stats without -v
25147 · Print password prompts to stderr - Hraban Luyat
25149 · Warn about duplicate files when syncing
25151 · Oauth improvements
25153 · allow auth_url and token_url to be set in the config file
25155 · Print redirection URI if using own credentials.
25157 · Don’t Mkdir at the start of sync to save transactions
25159 · Compile
25161 · Update build to go1.8.3
25163 · Require go1.6 for building rclone
25165 · Compile 386 builds with “GO386=387” for maximum compatibility
25167 · Bug Fixes
25169 · Fix menu selection when no remotes
25171 · Config saving reworked to not kill the file if disk gets full
25173 · Don’t delete remote if name does not change while renaming
25175 · moveto, copyto: report transfers and checks as per move and copy
25177 · Local
25179 · Add –local-no-unicode-normalization flag - Bob Potter
25181 · Mount
25183 · Now supported on Windows using cgofuse and WinFsp - thanks to Bill
25185 Zissimopoulos for much help
25186 · Compare checksums on upload/download via FUSE
25188 · Unmount when program ends with SIGINT (Ctrl+C) or SIGTERM - Jérôme
25190 Vizcaino
25191 · On read only open of file, make open pending until first read
25193 · Make –read-only reject modify operations
25195 · Implement ModTime via FUSE for remotes that support it
25197 · Allow modTime to be changed even before all writers are closed
25199 · Fix panic on renames
25201 · Fix hang on errored upload
25203 · Crypt
25205 · Report the name:root as specified by the user
25207 · Add an “obfuscate” option for filename encryption - Stephen Harris
25209 · Amazon Drive
25211 · Fix initialization order for token renewer
25213 · Remove revoked credentials, allow oauth proxy config and update
25215 docs
25216 · B2
25218 · Reduce minimum chunk size to 5MB
25220 · Drive
25222 · Add team drive support
25224 · Reduce bandwidth by adding fields for partial responses - Martin
25226 Kristensen
25227 · Implement –drive-shared-with-me flag to view shared with me files -
25229 Danny Tsai
25230 · Add –drive-trashed-only to read only the files in the trash
25232 · Remove obsolete –drive-full-list
25234 · Add missing seek to start on retries of chunked uploads
25236 · Fix stats accounting for upload
25238 · Convert / in names to a unicode equivalent (／)
25240 · Poll for Google Drive changes when mounted
25242 · OneDrive
25244 · Fix the uploading of files with spaces
25246 · Fix initialization order for token renewer
25248 · Display speeds accurately when uploading - Yoni Jah
25250 · Swap to using http://localhost:53682/ as redirect URL - Michael
25252 Ledin
25253 · Retry on token expired error, reset upload body on retry - Yoni Jah
25255 · Google Cloud Storage
25257 · Add ability to specify location and storage class via config and
25259 command line - thanks gdm85
25260 · Create container if necessary on server side copy
25262 · Increase directory listing chunk to 1000 to increase performance
25264 · Obtain a refresh token for GCS - Steven Lu
25266 · Yandex
25268 · Fix the name reported in log messages (was empty)
25270 · Correct error return for listing empty directory
25272 · Dropbox
25274 · Rewritten to use the v2 API
25276 · Now supports ModTime
25278 · Can only set by uploading the file again
25280 · If you uploaded with an old rclone, rclone may upload every‐
25282 thing again
25283 · Use --size-only or --checksum to avoid this
25285 · Now supports the Dropbox content hashing scheme
25287 · Now supports low level retries
25289 · S3
25291 · Work around eventual consistency in bucket creation
25293 · Create container if necessary on server side copy
25295 · Add us-east-2 (Ohio) and eu-west-2 (London) S3 regions - Zahiar
25297 Ahmed
25298 · Swift, Hubic
25300 · Fix zero length directory markers showing in the subdirectory list‐
25302 ing
25303 · this caused lots of duplicate transfers
25305 · Fix paged directory listings
25307 · this caused duplicate directory errors
25309 · Create container if necessary on server side copy
25311 · Increase directory listing chunk to 1000 to increase performance
25313 · Make sensible error if the user forgets the container
25315 · SFTP
25317 · Add support for using ssh key files
25319 · Fix under Windows
25321 · Fix ssh agent on Windows
25323 · Adapt to latest version of library - Igor Kharin
25325 v1.36 - 2017-03-18
25327 · New Features
25328 · SFTP remote (Jack Schmidt)
25330 · Re-implement sync routine to work a directory at a time reducing
25332 memory usage
25333 · Logging revamped to be more inline with rsync - now much quieter *
25335 -v only shows transfers * -vv is for full debug * –syslog to log to
25336 syslog on capable platforms
25337 · Implement –backup-dir and –suffix
25339 · Implement –track-renames (initial implementation by Bjørn Erik Ped‐
25341 ersen)
25342 · Add time-based bandwidth limits (Lukas Loesche)
25344 · rclone cryptcheck: checks integrity of crypt remotes
25346 · Allow all config file variables and options to be set from environ‐
25348 ment variables
25349 · Add –buffer-size parameter to control buffer size for copy
25351 · Make –delete-after the default
25353 · Add –ignore-checksum flag (fixed by Hisham Zarka)
25355 · rclone check: Add –download flag to check all the data, not just
25357 hashes
25358 · rclone cat: add –head, –tail, –offset, –count and –discard
25360 · rclone config: when choosing from a list, allow the value to be en‐
25362 tered too
25363 · rclone config: allow rename and copy of remotes
25365 · rclone obscure: for generating encrypted passwords for rclone’s
25367 config (T.C. Ferguson)
25368 · Comply with XDG Base Directory specification (Dario Giovannetti)
25370 · this moves the default location of the config file in a backwards
25372 compatible way
25373 · Release changes
25375 · Ubuntu snap support (Dedsec1)
25377 · Compile with go 1.8
25379 · MIPS/Linux big and little endian support
25381 · Bug Fixes
25383 · Fix copyto copying things to the wrong place if the destination dir
25385 didn’t exist
25386 · Fix parsing of remotes in moveto and copyto
25388 · Fix –delete-before deleting files on copy
25390 · Fix –files-from with an empty file copying everything
25392 · Fix sync: don’t update mod times if –dry-run set
25394 · Fix MimeType propagation
25396 · Fix filters to add ** rules to directory rules
25398 · Local
25400 · Implement -L, –copy-links flag to allow rclone to follow symlinks
25402 · Open files in write only mode so rclone can write to an rclone
25404 mount
25405 · Fix unnormalised unicode causing problems reading directories
25407 · Fix interaction between -x flag and –max-depth
25409 · Mount
25411 · Implement proper directory handling (mkdir, rmdir, renaming)
25413 · Make include and exclude filters apply to mount
25415 · Implement read and write async buffers - control with –buffer-size
25417 · Fix fsync on for directories
25419 · Fix retry on network failure when reading off crypt
25421 · Crypt
25423 · Add –crypt-show-mapping to show encrypted file mapping
25425 · Fix crypt writer getting stuck in a loop
25427 · IMPORTANT this bug had the potential to cause data corruption
25429 when
25430 · reading data from a network based remote and
25432 · writing to a crypt on Google Drive
25434 · Use the cryptcheck command to validate your data if you are con‐
25436 cerned
25437 · If syncing two crypt remotes, sync the unencrypted remote
25439 · Amazon Drive
25441 · Fix panics on Move (rename)
25443 · Fix panic on token expiry
25445 · B2
25447 · Fix inconsistent listings and rclone check
25449 · Fix uploading empty files with go1.8
25451 · Constrain memory usage when doing multipart uploads
25453 · Fix upload url not being refreshed properly
25455 · Drive
25457 · Fix Rmdir on directories with trashed files
25459 · Fix “Ignoring unknown object” when downloading
25461 · Add –drive-list-chunk
25463 · Add –drive-skip-gdocs (Károly Oláh)
25465 · OneDrive
25467 · Implement Move
25469 · Fix Copy
25471 · Fix overwrite detection in Copy
25473 · Fix waitForJob to parse errors correctly
25475 · Use token renewer to stop auth errors on long uploads
25477 · Fix uploading empty files with go1.8
25479 · Google Cloud Storage
25481 · Fix depth 1 directory listings
25483 · Yandex
25485 · Fix single level directory listing
25487 · Dropbox
25489 · Normalise the case for single level directory listings
25491 · Fix depth 1 listing
25493 · S3
25495 · Added ca-central-1 region (Jon Yergatian)
25497 v1.35 - 2017-01-02
25499 · New Features
25500 · moveto and copyto commands for choosing a destination name on
25502 copy/move
25503 · rmdirs command to recursively delete empty directories
25505 · Allow repeated –include/–exclude/–filter options
25507 · Only show transfer stats on commands which transfer stuff
25509 · show stats on any command using the --stats flag
25511 · Allow overlapping directories in move when server side dir move is
25513 supported
25514 · Add –stats-unit option - thanks Scott McGillivray
25516 · Bug Fixes
25518 · Fix the config file being overwritten when two rclones are running
25520 · Make rclone lsd obey the filters properly
25522 · Fix compilation on mips
25524 · Fix not transferring files that don’t differ in size
25526 · Fix panic on nil retry/fatal error
25528 · Mount
25530 · Retry reads on error - should help with reliability a lot
25532 · Report the modification times for directories from the remote
25534 · Add bandwidth accounting and limiting (fixes –bwlimit)
25536 · If –stats provided will show stats and which files are transferring
25538 · Support R/W files if truncate is set.
25540 · Implement statfs interface so df works
25542 · Note that write is now supported on Amazon Drive
25544 · Report number of blocks in a file - thanks Stefan Breunig
25546 · Crypt
25548 · Prevent the user pointing crypt at itself
25550 · Fix failed to authenticate decrypted block errors
25552 · these will now return the underlying unexpected EOF instead
25554 · Amazon Drive
25556 · Add support for server side move and directory move - thanks Stefan
25558 Breunig
25559 · Fix nil pointer deref on size attribute
25561 · B2
25563 · Use new prefix and delimiter parameters in directory listings
25565 · This makes –max-depth 1 dir listings as used in mount much faster
25567 · Reauth the account while doing uploads too - should help with token
25569 expiry
25570 · Drive
25572 · Make DirMove more efficient and complain about moving the root
25574 · Create destination directory on Move()
25576 v1.34 - 2016-11-06
25578 · New Features
25579 · Stop single file and --files-from operations iterating through the
25581 source bucket.
25582 · Stop removing failed upload to cloud storage remotes
25584 · Make ContentType be preserved for cloud to cloud copies
25586 · Add support to toggle bandwidth limits via SIGUSR2 - thanks Marco
25588 Paganini
25589 · rclone check shows count of hashes that couldn’t be checked
25591 · rclone listremotes command
25593 · Support linux/arm64 build - thanks Fredrik Fornwall
25595 · Remove Authorization: lines from --dump-headers output
25597 · Bug Fixes
25599 · Ignore files with control characters in the names
25601 · Fix rclone move command
25603 · Delete src files which already existed in dst
25605 · Fix deletion of src file when dst file older
25607 · Fix rclone check on crypted file systems
25609 · Make failed uploads not count as “Transferred”
25611 · Make sure high level retries show with -q
25613 · Use a vendor directory with godep for repeatable builds
25615 · rclone mount - FUSE
25617 · Implement FUSE mount options
25619 · --no-modtime, --debug-fuse, --read-only, --allow-non-empty, --al‐
25621 low-root, --allow-other
25622 · --default-permissions, --write-back-cache, --max-read-ahead,
25624 --umask, --uid, --gid
25625 · Add --dir-cache-time to control caching of directory entries
25627 · Implement seek for files opened for read (useful for video players)
25629 · with -no-seek flag to disable
25631 · Fix crash on 32 bit ARM (alignment of 64 bit counter)
25633 · ...and many more internal fixes and improvements!
25635 · Crypt
25637 · Don’t show encrypted password in configurator to stop confusion
25639 · Amazon Drive
25641 · New wait for upload option --acd-upload-wait-per-gb
25643 · upload timeouts scale by file size and can be disabled
25645 · Add 502 Bad Gateway to list of errors we retry
25647 · Fix overwriting a file with a zero length file
25649 · Fix ACD file size warning limit - thanks Felix Bünemann
25651 · Local
25653 · Unix: implement -x/--one-file-system to stay on a single file sys‐
25655 tem
25656 · thanks Durval Menezes and Luiz Carlos Rumbelsperger Viana
25658 · Windows: ignore the symlink bit on files
25660 · Windows: Ignore directory based junction points
25662 · B2
25664 · Make sure each upload has at least one upload slot - fixes strange
25666 upload stats
25667 · Fix uploads when using crypt
25669 · Fix download of large files (sha1 mismatch)
25671 · Return error when we try to create a bucket which someone else owns
25673 · Update B2 docs with Data usage, and Crypt section - thanks Tomasz
25675 Mazur
25676 · S3
25678 · Command line and config file support for
25680 · Setting/overriding ACL - thanks Radek Senfeld
25682 · Setting storage class - thanks Asko Tamm
25684 · Drive
25686 · Make exponential backoff work exactly as per Google specification
25688 · add .epub, .odp and .tsv as export formats.
25690 · Swift
25692 · Don’t read metadata for directory marker objects
25694 v1.33 - 2016-08-24
25696 · New Features
25697 · Implement encryption
25699 · data encrypted in NACL secretbox format
25701 · with optional file name encryption
25703 · New commands
25705 · rclone mount - implements FUSE mounting of remotes (EXPERIMENTAL)
25707 · works on Linux, FreeBSD and OS X (need testers for the last 2!)
25709 · rclone cat - outputs remote file or files to the terminal
25711 · rclone genautocomplete - command to make a bash completion script
25713 for rclone
25714 · Editing a remote using rclone config now goes through the wizard
25716 · Compile with go 1.7 - this fixes rclone on macOS Sierra and on 386
25718 processors
25719 · Use cobra for sub commands and docs generation
25721 · drive
25723 · Document how to make your own client_id
25725 · s3
25727 · User-configurable Amazon S3 ACL (thanks Radek Šenfeld)
25729 · b2
25731 · Fix stats accounting for upload - no more jumping to 100% done
25733 · On cleanup delete hide marker if it is the current file
25735 · New B2 API endpoint (thanks Per Cederberg)
25737 · Set maximum backoff to 5 Minutes
25739 · onedrive
25741 · Fix URL escaping in file names - eg uploading files with + in them.
25743 · amazon cloud drive
25745 · Fix token expiry during large uploads
25747 · Work around 408 REQUEST_TIMEOUT and 504 GATEWAY_TIMEOUT errors
25749 · local
25751 · Fix filenames with invalid UTF-8 not being uploaded
25753 · Fix problem with some UTF-8 characters on OS X
25755 v1.32 - 2016-07-13
25757 · Backblaze B2
25758 · Fix upload of files large files not in root
25760 v1.31 - 2016-07-13
25762 · New Features
25763 · Reduce memory on sync by about 50%
25765 · Implement –no-traverse flag to stop copy traversing the destination
25767 remote.
25768 · This can be used to reduce memory usage down to the smallest pos‐
25770 sible.
25771 · Useful to copy a small number of files into a large destination
25773 folder.
25774 · Implement cleanup command for emptying trash / removing old ver‐
25776 sions of files
25777 · Currently B2 only
25779 · Single file handling improved
25781 · Now copied with –files-from
25783 · Automatically sets –no-traverse when copying a single file
25785 · Info on using installing with ansible - thanks Stefan Weichinger
25787 · Implement –no-update-modtime flag to stop rclone fixing the remote
25789 modified times.
25790 · Bug Fixes
25792 · Fix move command - stop it running for overlapping Fses - this was
25794 causing data loss.
25795 · Local
25797 · Fix incomplete hashes - this was causing problems for B2.
25799 · Amazon Drive
25801 · Rename Amazon Cloud Drive to Amazon Drive - no changes to config
25803 file needed.
25804 · Swift
25806 · Add support for non-default project domain - thanks Antonio Messi‐
25808 na.
25809 · S3
25811 · Add instructions on how to use rclone with minio.
25813 · Add ap-northeast-2 (Seoul) and ap-south-1 (Mumbai) regions.
25815 · Skip setting the modified time for objects > 5GB as it isn’t possi‐
25817 ble.
25818 · Backblaze B2
25820 · Add –b2-versions flag so old versions can be listed and retreived.
25822 · Treat 403 errors (eg cap exceeded) as fatal.
25824 · Implement cleanup command for deleting old file versions.
25826 · Make error handling compliant with B2 integrations notes.
25828 · Fix handling of token expiry.
25830 · Implement –b2-test-mode to set X-Bz-Test-Mode header.
25832 · Set cutoff for chunked upload to 200MB as per B2 guidelines.
25834 · Make upload multi-threaded.
25836 · Dropbox
25838 · Don’t retry 461 errors.
25840 v1.30 - 2016-06-18
25842 · New Features
25843 · Directory listing code reworked for more features and better error
25845 reporting (thanks to Klaus Post for help). This enables
25846 · Directory include filtering for efficiency
25848 · –max-depth parameter
25850 · Better error reporting
25852 · More to come
25854 · Retry more errors
25856 · Add –ignore-size flag - for uploading images to onedrive
25858 · Log -v output to stdout by default
25860 · Display the transfer stats in more human readable form
25862 · Make 0 size files specifiable with --max-size 0b
25864 · Add b suffix so we can specify bytes in –bwlimit, –min-size etc
25866 · Use “password:” instead of “password>” prompt - thanks Klaus Post
25868 and Leigh Klotz
25869 · Bug Fixes
25871 · Fix retry doing one too many retries
25873 · Local
25875 · Fix problems with OS X and UTF-8 characters
25877 · Amazon Drive
25879 · Check a file exists before uploading to help with 408 Conflict er‐
25881 rors
25882 · Reauth on 401 errors - this has been causing a lot of problems
25884 · Work around spurious 403 errors
25886 · Restart directory listings on error
25888 · Google Drive
25890 · Check a file exists before uploading to help with duplicates
25892 · Fix retry of multipart uploads
25894 · Backblaze B2
25896 · Implement large file uploading
25898 · S3
25900 · Add AES256 server-side encryption for - thanks Justin R. Wilson
25902 · Google Cloud Storage
25904 · Make sure we don’t use conflicting content types on upload
25906 · Add service account support - thanks Michal Witkowski
25908 · Swift
25910 · Add auth version parameter
25912 · Add domain option for openstack (v3 auth) - thanks Fabian Ruff
25914 v1.29 - 2016-04-18
25916 · New Features
25917 · Implement -I, --ignore-times for unconditional upload
25919 · Improve dedupecommand
25921 · Now removes identical copies without asking
25923 · Now obeys --dry-run
25925 · Implement --dedupe-mode for non interactive running
25927 · --dedupe-mode interactive - interactive the default.
25929 · --dedupe-mode skip - removes identical files then skips any‐
25931 thing left.
25932 · --dedupe-mode first - removes identical files then keeps the
25934 first one.
25935 · --dedupe-mode newest - removes identical files then keeps the
25937 newest one.
25938 · --dedupe-mode oldest - removes identical files then keeps the
25940 oldest one.
25941 · --dedupe-mode rename - removes identical files then renames the
25943 rest to be different.
25944 · Bug fixes
25946 · Make rclone check obey the --size-only flag.
25948 · Use “application/octet-stream” if discovered mime type is invalid.
25950 · Fix missing “quit” option when there are no remotes.
25952 · Google Drive
25954 · Increase default chunk size to 8 MB - increases upload speed of big
25956 files
25957 · Speed up directory listings and make more reliable
25959 · Add missing retries for Move and DirMove - increases reliability
25961 · Preserve mime type on file update
25963 · Backblaze B2
25965 · Enable mod time syncing
25967 · This means that B2 will now check modification times
25969 · It will upload new files to update the modification times
25971 · (there isn’t an API to just set the mod time.)
25973 · If you want the old behaviour use --size-only.
25975 · Update API to new version
25977 · Fix parsing of mod time when not in metadata
25979 · Swift/Hubic
25981 · Don’t return an MD5SUM for static large objects
25983 · S3
25985 · Fix uploading files bigger than 50GB
25987 v1.28 - 2016-03-01
25989 · New Features
25990 · Configuration file encryption - thanks Klaus Post
25992 · Improve rclone config adding more help and making it easier to un‐
25994 derstand
25995 · Implement -u/--update so creation times can be used on all remotes
25997 · Implement --low-level-retries flag
25999 · Optionally disable gzip compression on downloads with --no-gzip-en‐
26001 coding
26002 · Bug fixes
26004 · Don’t make directories if --dry-run set
26006 · Fix and document the move command
26008 · Fix redirecting stderr on unix-like OSes when using --log-file
26010 · Fix delete command to wait until all finished - fixes missing
26012 deletes.
26013 · Backblaze B2
26015 · Use one upload URL per go routine fixes more than one upload using
26017 auth token
26018 · Add pacing, retries and reauthentication - fixes token expiry prob‐
26020 lems
26021 · Upload without using a temporary file from local (and remotes which
26023 support SHA1)
26024 · Fix reading metadata for all files when it shouldn’t have been
26026 · Drive
26028 · Fix listing drive documents at root
26030 · Disable copy and move for Google docs
26032 · Swift
26034 · Fix uploading of chunked files with non ASCII characters
26036 · Allow setting of storage_url in the config - thanks Xavier Lucas
26038 · S3
26040 · Allow IAM role and credentials from environment variables - thanks
26042 Brian Stengaard
26043 · Allow low privilege users to use S3 (check if directory exists dur‐
26045 ing Mkdir) - thanks Jakub Gedeon
26046 · Amazon Drive
26048 · Retry on more things to make directory listings more reliable
26050 v1.27 - 2016-01-31
26052 · New Features
26053 · Easier headless configuration with rclone authorize
26055 · Add support for multiple hash types - we now check SHA1 as well as
26057 MD5 hashes.
26058 · delete command which does obey the filters (unlike purge)
26060 · dedupe command to deduplicate a remote. Useful with Google Drive.
26062 · Add --ignore-existing flag to skip all files that exist on destina‐
26064 tion.
26065 · Add --delete-before, --delete-during, --delete-after flags.
26067 · Add --memprofile flag to debug memory use.
26069 · Warn the user about files with same name but different case
26071 · Make --include rules add their implict exclude * at the end of the
26073 filter list
26074 · Deprecate compiling with go1.3
26076 · Amazon Drive
26078 · Fix download of files > 10 GB
26080 · Fix directory traversal (“Next token is expired”) for large direc‐
26082 tory listings
26083 · Remove 409 conflict from error codes we will retry - stops very
26085 long pauses
26086 · Backblaze B2
26088 · SHA1 hashes now checked by rclone core
26090 · Drive
26092 · Add --drive-auth-owner-only to only consider files owned by the us‐
26094 er - thanks Björn Harrtell
26095 · Export Google documents
26097 · Dropbox
26099 · Make file exclusion error controllable with -q
26101 · Swift
26103 · Fix upload from unprivileged user.
26105 · S3
26107 · Fix updating of mod times of files with + in.
26109 · Local
26111 · Add local file system option to disable UNC on Windows.
26113 v1.26 - 2016-01-02
26115 · New Features
26116 · Yandex storage backend - thank you Dmitry Burdeev (“dibu”)
26118 · Implement Backblaze B2 storage backend
26120 · Add –min-age and –max-age flags - thank you Adriano Aurélio
26122 Meirelles
26123 · Make ls/lsl/md5sum/size/check obey includes and excludes
26125 · Fixes
26127 · Fix crash in http logging
26129 · Upload releases to github too
26131 · Swift
26133 · Fix sync for chunked files
26135 · OneDrive
26137 · Re-enable server side copy
26139 · Don’t mask HTTP error codes with JSON decode error
26141 · S3
26143 · Fix corrupting Content-Type on mod time update (thanks Joseph
26145 Spurrier)
26146 v1.25 - 2015-11-14
26148 · New features
26149 · Implement Hubic storage system
26151 · Fixes
26153 · Fix deletion of some excluded files without –delete-excluded
26155 · This could have deleted files unexpectedly on sync
26157 · Always check first with --dry-run!
26159 · Swift
26161 · Stop SetModTime losing metadata (eg X-Object-Manifest)
26163 · This could have caused data loss for files > 5GB in size
26165 · Use ContentType from Object to avoid lookups in listings
26167 · OneDrive
26169 · disable server side copy as it seems to be broken at Microsoft
26171 v1.24 - 2015-11-07
26173 · New features
26174 · Add support for Microsoft OneDrive
26176 · Add --no-check-certificate option to disable server certificate
26178 verification
26179 · Add async readahead buffer for faster transfer of big files
26181 · Fixes
26183 · Allow spaces in remotes and check remote names for validity at cre‐
26185 ation time
26186 · Allow `&' and disallow `:' in Windows filenames.
26188 · Swift
26190 · Ignore directory marker objects where appropriate - allows working
26192 with Hubic
26193 · Don’t delete the container if fs wasn’t at root
26195 · S3
26197 · Don’t delete the bucket if fs wasn’t at root
26199 · Google Cloud Storage
26201 · Don’t delete the bucket if fs wasn’t at root
26203 v1.23 - 2015-10-03
26205 · New features
26206 · Implement rclone size for measuring remotes
26208 · Fixes
26210 · Fix headless config for drive and gcs
26212 · Tell the user they should try again if the webserver method failed
26214 · Improve output of --dump-headers
26216 · S3
26218 · Allow anonymous access to public buckets
26220 · Swift
26222 · Stop chunked operations logging “Failed to read info: Object Not
26224 Found”
26225 · Use Content-Length on uploads for extra reliability
26227 v1.22 - 2015-09-28
26229 · Implement rsync like include and exclude flags
26230 · swift
26232 · Support files > 5GB - thanks Sergey Tolmachev
26234 v1.21 - 2015-09-22
26236 · New features
26237 · Display individual transfer progress
26239 · Make lsl output times in localtime
26241 · Fixes
26243 · Fix allowing user to override credentials again in Drive, GCS and
26245 ACD
26246 · Amazon Drive
26248 · Implement compliant pacing scheme
26250 · Google Drive
26252 · Make directory reads concurrent for increased speed.
26254 v1.20 - 2015-09-15
26256 · New features
26257 · Amazon Drive support
26259 · Oauth support redone - fix many bugs and improve usability
26261 · Use “golang.org/x/oauth2” as oauth libary of choice
26263 · Improve oauth usability for smoother initial signup
26265 · drive, googlecloudstorage: optionally use auto config for the
26267 oauth token
26268 · Implement –dump-headers and –dump-bodies debug flags
26270 · Show multiple matched commands if abbreviation too short
26272 · Implement server side move where possible
26274 · local
26276 · Always use UNC paths internally on Windows - fixes a lot of bugs
26278 · dropbox
26280 · force use of our custom transport which makes timeouts work
26282 · Thanks to Klaus Post for lots of help with this release
26284 v1.19 - 2015-08-28
26286 · New features
26287 · Server side copies for s3/swift/drive/dropbox/gcs
26289 · Move command - uses server side copies if it can
26291 · Implement –retries flag - tries 3 times by default
26293 · Build for plan9/amd64 and solaris/amd64 too
26295 · Fixes
26297 · Make a current version download with a fixed URL for scripting
26299 · Ignore rmdir in limited fs rather than throwing error
26301 · dropbox
26303 · Increase chunk size to improve upload speeds massively
26305 · Issue an error message when trying to upload bad file name
26307 v1.18 - 2015-08-17
26309 · drive
26310 · Add --drive-use-trash flag so rclone trashes instead of deletes
26312 · Add “Forbidden to download” message for files with no downloadURL
26314 · dropbox
26316 · Remove datastore
26318 · This was deprecated and it caused a lot of problems
26320 · Modification times and MD5SUMs no longer stored
26322 · Fix uploading files > 2GB
26324 · s3
26326 · use official AWS SDK from github.com/aws/aws-sdk-go
26328 · NB will most likely require you to delete and recreate remote
26330 · enable multipart upload which enables files > 5GB
26332 · tested with Ceph / RadosGW / S3 emulation
26334 · many thanks to Sam Liston and Brian Haymore at the Utah Center for
26336 High Performance Computing (https://www.chpc.utah.edu/) for a Ceph
26337 test account
26338 · misc
26340 · Show errors when reading the config file
26342 · Do not print stats in quiet mode - thanks Leonid Shalupov
26344 · Add FAQ
26346 · Fix created directories not obeying umask
26348 · Linux installation instructions - thanks Shimon Doodkin
26350 v1.17 - 2015-06-14
26352 · dropbox: fix case insensitivity issues - thanks Leonid Shalupov
26353 v1.16 - 2015-06-09
26355 · Fix uploading big files which was causing timeouts or panics
26356 · Don’t check md5sum after download with –size-only
26358 v1.15 - 2015-06-06
26360 · Add –checksum flag to only discard transfers by MD5SUM - thanks Alex
26361 Couper
26362 · Implement –size-only flag to sync on size not checksum & modtime
26364 · Expand docs and remove duplicated information
26366 · Document rclone’s limitations with directories
26368 · dropbox: update docs about case insensitivity
26370 v1.14 - 2015-05-21
26372 · local: fix encoding of non utf-8 file names - fixes a duplicate file
26373 problem
26374 · drive: docs about rate limiting
26376 · google cloud storage: Fix compile after API change in
26378 “google.golang.org/api/storage/v1”
26379 v1.13 - 2015-05-10
26381 · Revise documentation (especially sync)
26382 · Implement –timeout and –conntimeout
26384 · s3: ignore etags from multipart uploads which aren’t md5sums
26386 v1.12 - 2015-03-15
26388 · drive: Use chunked upload for files above a certain size
26389 · drive: add –drive-chunk-size and –drive-upload-cutoff parameters
26391 · drive: switch to insert from update when a failed copy deletes the
26393 upload
26394 · core: Log duplicate files if they are detected
26396 v1.11 - 2015-03-04
26398 · swift: add region parameter
26399 · drive: fix crash on failed to update remote mtime
26401 · In remote paths, change native directory separators to /
26403 · Add synchronization to ls/lsl/lsd output to stop corruptions
26405 · Ensure all stats/log messages to go stderr
26407 · Add –log-file flag to log everything (including panics) to file
26409 · Make it possible to disable stats printing with –stats=0
26411 · Implement –bwlimit to limit data transfer bandwidth
26413 v1.10 - 2015-02-12
26415 · s3: list an unlimited number of items
26416 · Fix getting stuck in the configurator
26418 v1.09 - 2015-02-07
26420 · windows: Stop drive letters (eg C:) getting mixed up with remotes (eg
26421 drive:)
26422 · local: Fix directory separators on Windows
26424 · drive: fix rate limit exceeded errors
26426 v1.08 - 2015-02-04
26428 · drive: fix subdirectory listing to not list entire drive
26429 · drive: Fix SetModTime
26431 · dropbox: adapt code to recent library changes
26433 v1.07 - 2014-12-23
26435 · google cloud storage: fix memory leak
26436 v1.06 - 2014-12-12
26438 · Fix “Couldn’t find home directory” on OSX
26439 · swift: Add tenant parameter
26441 · Use new location of Google API packages
26443 v1.05 - 2014-08-09
26445 · Improved tests and consequently lots of minor fixes
26446 · core: Fix race detected by go race detector
26448 · core: Fixes after running errcheck
26450 · drive: reset root directory on Rmdir and Purge
26452 · fs: Document that Purger returns error on empty directory, test and
26454 fix
26455 · google cloud storage: fix ListDir on subdirectory
26457 · google cloud storage: re-read metadata in SetModTime
26459 · s3: make reading metadata more reliable to work around eventual con‐
26461 sistency problems
26462 · s3: strip trailing / from ListDir()
26464 · swift: return directories without / in ListDir
26466 v1.04 - 2014-07-21
26468 · google cloud storage: Fix crash on Update
26469 v1.03 - 2014-07-20
26471 · swift, s3, dropbox: fix updated files being marked as corrupted
26472 · Make compile with go 1.1 again
26474 v1.02 - 2014-07-19
26476 · Implement Dropbox remote
26477 · Implement Google Cloud Storage remote
26479 · Verify Md5sums and Sizes after copies
26481 · Remove times from “ls” command - lists sizes only
26483 · Add add “lsl” - lists times and sizes
26485 · Add “md5sum” command
26487 v1.01 - 2014-07-04
26489 · drive: fix transfer of big files using up lots of memory
26490 v1.00 - 2014-07-03
26492 · drive: fix whole second dates
26493 v0.99 - 2014-06-26
26495 · Fix –dry-run not working
26496 · Make compatible with go 1.1
26498 v0.98 - 2014-05-30
26500 · s3: Treat missing Content-Length as 0 for some ceph installations
26501 · rclonetest: add file with a space in
26503 v0.97 - 2014-05-05
26505 · Implement copying of single files
26506 · s3 & swift: support paths inside containers/buckets
26508 v0.96 - 2014-04-24
26510 · drive: Fix multiple files of same name being created
26511 · drive: Use o.Update and fs.Put to optimise transfers
26513 · Add version number, -V and –version
26515 v0.95 - 2014-03-28
26517 · rclone.org: website, docs and graphics
26518 · drive: fix path parsing
26520 v0.94 - 2014-03-27
26522 · Change remote format one last time
26523 · GNU style flags
26525 v0.93 - 2014-03-16
26527 · drive: store token in config file
26528 · cross compile other versions
26530 · set strict permissions on config file
26532 v0.92 - 2014-03-15
26534 · Config fixes and –config option
26535 v0.91 - 2014-03-15
26537 · Make config file
26538 v0.90 - 2013-06-27
26540 · Project named rclone
26541 v0.00 - 2012-11-18
26543 · Project started
26544
26546 Limitations
26547 Directory timestamps aren’t preserved
26548 Rclone doesn’t currently preserve the timestamps of directories. This
26549 is because rclone only really considers objects when syncing.
26550 Rclone struggles with millions of files in a directory
26552 Currently rclone loads each directory entirely into memory before using
26553 it. Since each Rclone object takes 0.5k-1k of memory this can take a
26554 very long time and use an extremely large amount of memory.
26555 Millions of files in a directory tend caused by software writing cloud
26557 storage (eg S3 buckets).
26558 Bucket based remotes and folders
26560 Bucket based remotes (eg S3/GCS/Swift/B2) do not have a concept of di‐
26561 rectories. Rclone therefore cannot create directories in them which
26562 means that empty directories on a bucket based remote will tend to dis‐
26563 appear.
26564 Some software creates empty keys ending in / as directory markers.
26566 Rclone doesn’t do this as it potentially creates more objects and costs
26567 more. It may do in future (probably with a flag).
26568 Bugs
26570 Bugs are stored in rclone’s GitHub project:
26571 · Reported bugs (https://github.com/rclone/rclone/is‐
26573 sues?q=is%3Aopen+is%3Aissue+label%3Abug)
26574 · Known issues (https://github.com/rclone/rclone/is‐
26576 sues?q=is%3Aopen+is%3Aissue+milestone%3A%22Known+Problem%22)
26577 Frequently Asked Questions
26579 Do all cloud storage systems support all rclone commands
26580 Yes they do. All the rclone commands (eg sync, copy etc) will work on
26581 all the remote storage systems.
26582 Can I copy the config from one machine to another
26584 Sure! Rclone stores all of its config in a single file. If you want to
26585 find this file, run rclone config file which will tell you where it is.
26586 See the remote setup docs (https://rclone.org/remote_setup/) for more
26588 info.
26589 How do I configure rclone on a remote / headless box with no browser?
26591 This has now been documented in its own remote setup page
26592 (https://rclone.org/remote_setup/).
26593 Can rclone sync directly from drive to s3
26595 Rclone can sync between two remote cloud storage systems just fine.
26596 Note that it effectively downloads the file and uploads it again, so
26598 the node running rclone would need to have lots of bandwidth.
26599 The syncs would be incremental (on a file by file basis).
26601 Eg
26603 rclone sync drive:Folder s3:bucket
26605 Using rclone from multiple locations at the same time
26607 You can use rclone from multiple places at the same time if you choose
26608 different subdirectory for the output, eg
26609 Server A> rclone sync /tmp/whatever remote:ServerA
26611 Server B> rclone sync /tmp/whatever remote:ServerB
26612 If you sync to the same directory then you should use rclone copy oth‐
26614 erwise the two rclones may delete each others files, eg
26615 Server A> rclone copy /tmp/whatever remote:Backup
26617 Server B> rclone copy /tmp/whatever remote:Backup
26618 The file names you upload from Server A and Server B should be differ‐
26620 ent in this case, otherwise some file systems (eg Drive) may make du‐
26621 plicates.
26622 Why doesn’t rclone support partial transfers / binary diffs like rsync?
26624 Rclone stores each file you transfer as a native object on the remote
26625 cloud storage system. This means that you can see the files you upload
26626 as expected using alternative access methods (eg using the Google Drive
26627 web interface). There is a 1:1 mapping between files on your hard disk
26628 and objects created in the cloud storage system.
26629 Cloud storage systems (at least none I’ve come across yet) don’t sup‐
26631 port partially uploading an object. You can’t take an existing object,
26632 and change some bytes in the middle of it.
26633 It would be possible to make a sync system which stored binary diffs
26635 instead of whole objects like rclone does, but that would break the 1:1
26636 mapping of files on your hard disk to objects in the remote cloud stor‐
26637 age system.
26638 All the cloud storage systems support partial downloads of content, so
26640 it would be possible to make partial downloads work. However to make
26641 this work efficiently this would require storing a significant amount
26642 of metadata, which breaks the desired 1:1 mapping of files to objects.
26643 Can rclone do bi-directional sync?
26645 No, not at present. rclone only does uni-directional sync from A -> B.
26646 It may do in the future though since it has all the primitives - it
26647 just requires writing the algorithm to do it.
26648 Can I use rclone with an HTTP proxy?
26650 Yes. rclone will follow the standard environment variables for prox‐
26651 ies, similar to cURL and other programs.
26652 In general the variables are called http_proxy (for services reached
26654 over http) and https_proxy (for services reached over https). Most
26655 public services will be using https, but you may wish to set both.
26656 The content of the variable is protocol://server:port. The protocol
26658 value is the one used to talk to the proxy server, itself, and is com‐
26659 monly either http or socks5.
26660 Slightly annoyingly, there is no standard for the name; some applica‐
26662 tions may use http_proxy but another one HTTP_PROXY. The Go libraries
26663 used by rclone will try both variations, but you may wish to set all
26664 possibilities. So, on Linux, you may end up with code similar to
26665 export http_proxy=http://proxyserver:12345
26667 export https_proxy=$http_proxy
26668 export HTTP_PROXY=$http_proxy
26669 export HTTPS_PROXY=$http_proxy
26670 The NO_PROXY allows you to disable the proxy for specific hosts. Hosts
26672 must be comma separated, and can contain domains or parts. For in‐
26673 stance “foo.com” also matches “bar.foo.com”.
26674 e.g.
26676 export no_proxy=localhost,127.0.0.0/8,my.host.name
26678 export NO_PROXY=$no_proxy
26679 Note that the ftp backend does not support ftp_proxy yet.
26681 Rclone gives x509: failed to load system roots and no roots provided error
26683 This means that rclone can’t file the SSL root certificates. Likely
26684 you are running rclone on a NAS with a cut-down Linux OS, or possibly
26685 on Solaris.
26686 Rclone (via the Go runtime) tries to load the root certificates from
26688 these places on Linux.
26689 "/etc/ssl/certs/ca-certificates.crt", // Debian/Ubuntu/Gentoo etc.
26691 "/etc/pki/tls/certs/ca-bundle.crt", // Fedora/RHEL
26692 "/etc/ssl/ca-bundle.pem", // OpenSUSE
26693 "/etc/pki/tls/cacert.pem", // OpenELEC
26694 So doing something like this should fix the problem. It also sets the
26696 time which is important for SSL to work properly.
26697 mkdir -p /etc/ssl/certs/
26699 curl -o /etc/ssl/certs/ca-certificates.crt https://raw.githubusercontent.com/bagder/ca-bundle/master/ca-bundle.crt
26700 ntpclient -s -h pool.ntp.org
26701 The two environment variables SSL_CERT_FILE and SSL_CERT_DIR, mentioned
26703 in the x509 package (https://godoc.org/crypto/x509), provide an addi‐
26704 tional way to provide the SSL root certificates.
26705 Note that you may need to add the --insecure option to the curl command
26707 line if it doesn’t work without.
26708 curl --insecure -o /etc/ssl/certs/ca-certificates.crt https://raw.githubusercontent.com/bagder/ca-bundle/master/ca-bundle.crt
26710 Rclone gives Failed to load config file: function not implemented error
26712 Likely this means that you are running rclone on Linux version not sup‐
26713 ported by the go runtime, ie earlier than version 2.6.23.
26714 See the system requirements section in the go install docs
26716 (https://golang.org/doc/install) for full details.
26717 All my uploaded docx/xlsx/pptx files appear as archive/zip
26719 This is caused by uploading these files from a Windows computer which
26720 hasn’t got the Microsoft Office suite installed. The easiest way to
26721 fix is to install the Word viewer and the Microsoft Office Compatibili‐
26722 ty Pack for Word, Excel, and PowerPoint 2007 and later versions’ file
26723 formats
26724 tcp lookup some.domain.com no such host
26726 This happens when rclone cannot resolve a domain. Please check that
26727 your DNS setup is generally working, e.g.
26728 # both should print a long list of possible IP addresses
26730 dig www.googleapis.com # resolve using your default DNS
26731 dig www.googleapis.com @8.8.8.8 # resolve with Google's DNS server
26732 If you are using systemd-resolved (default on Arch Linux), ensure it is
26734 at version 233 or higher. Previous releases contain a bug which causes
26735 not all domains to be resolved properly.
26736 Additionally with the GODEBUG=netdns= environment variable the Go re‐
26738 solver decision can be influenced. This also allows to resolve certain
26739 issues with DNS resolution. See the name resolution section in the go
26740 docs (https://golang.org/pkg/net/#hdr-Name_Resolution).
26741 The total size reported in the stats for a sync is wrong and keeps changing
26743 It is likely you have more than 10,000 files that need to be synced.
26744 By default rclone only gets 10,000 files ahead in a sync so as not to
26745 use up too much memory. You can change this default with the
26746 –max-backlog (/docs/#max-backlog-n) flag.
26747 Rclone is using too much memory or appears to have a memory leak
26749 Rclone is written in Go which uses a garbage collector. The default
26750 settings for the garbage collector mean that it runs when the heap size
26751 has doubled.
26752 However it is possible to tune the garbage collector to use less memory
26754 by setting GOGC (https://dave.cheney.net/tag/gogc) to a lower value,
26755 say export GOGC=20. This will make the garbage collector work harder,
26756 reducing memory size at the expense of CPU usage.
26757 The most common cause of rclone using lots of memory is a single direc‐
26759 tory with thousands or millions of files in. Rclone has to load this
26760 entirely into memory as rclone objects. Each rclone object takes
26761 0.5k-1k of memory.
26762 License
26764 This is free software under the terms of MIT the license (check the
26765 COPYING file included with the source code).
26766 Copyright (C) 2019 by Nick Craig-Wood https://www.craig-wood.com/nick/
26768 Permission is hereby granted, free of charge, to any person obtaining a copy
26770 of this software and associated documentation files (the "Software"), to deal
26771 in the Software without restriction, including without limitation the rights
26772 to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
26773 copies of the Software, and to permit persons to whom the Software is
26774 furnished to do so, subject to the following conditions:
26775 The above copyright notice and this permission notice shall be included in
26777 all copies or substantial portions of the Software.
26778 THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
26780 IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
26781 FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
26782 AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
26783 LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
26784 OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
26785 THE SOFTWARE.
26786 Authors
26788 · Nick Craig-Wood <nick@craig-wood.com>
26789 Contributors
26791 · Alex Couper <amcouper@gmail.com>
26792 · Leonid Shalupov <leonid@shalupov.com> <shalupov@diverse.org.ru>
26794 · Shimon Doodkin <helpmepro1@gmail.com>
26796 · Colin Nicholson <colin@colinn.com>
26798 · Klaus Post <klauspost@gmail.com>
26800 · Sergey Tolmachev <tolsi.ru@gmail.com>
26802 · Adriano Aurélio Meirelles <adriano@atinge.com>
26804 · C. Bess <cbess@users.noreply.github.com>
26806 · Dmitry Burdeev <dibu28@gmail.com>
26808 · Joseph Spurrier <github@josephspurrier.com>
26810 · Björn Harrtell <bjorn@wololo.org>
26812 · Xavier Lucas <xavier.lucas@corp.ovh.com>
26814 · Werner Beroux <werner@beroux.com>
26816 · Brian Stengaard <brian@stengaard.eu>
26818 · Jakub Gedeon <jgedeon@sofi.com>
26820 · Jim Tittsler <jwt@onjapan.net>
26822 · Michal Witkowski <michal@improbable.io>
26824 · Fabian Ruff <fabian.ruff@sap.com>
26826 · Leigh Klotz <klotz@quixey.com>
26828 · Romain Lapray <lapray.romain@gmail.com>
26830 · Justin R. Wilson <jrw972@gmail.com>
26832 · Antonio Messina <antonio.s.messina@gmail.com>
26834 · Stefan G. Weichinger <office@oops.co.at>
26836 · Per Cederberg <cederberg@gmail.com>
26838 · Radek Šenfeld <rush@logic.cz>
26840 · Fredrik Fornwall <fredrik@fornwall.net>
26842 · Asko Tamm <asko@deekit.net>
26844 · xor-zz <xor@gstocco.com>
26846 · Tomasz Mazur <tmazur90@gmail.com>
26848 · Marco Paganini <paganini@paganini.net>
26850 · Felix Bünemann <buenemann@louis.info>
26852 · Durval Menezes <jmrclone@durval.com>
26854 · Luiz Carlos Rumbelsperger Viana <maxd13_luiz_carlos@hotmail.com>
26856 · Stefan Breunig <stefan-github@yrden.de>
26858 · Alishan Ladhani <ali-l@users.noreply.github.com>
26860 · 0xJAKE <0xJAKE@users.noreply.github.com>
26862 · Thibault Molleman <thibaultmol@users.noreply.github.com>
26864 · Scott McGillivray <scott.mcgillivray@gmail.com>
26866 · Bjørn Erik Pedersen <bjorn.erik.pedersen@gmail.com>
26868 · Lukas Loesche <lukas@mesosphere.io>
26870 · emyarod <allllaboutyou@gmail.com>
26872 · T.C. Ferguson <tcf909@gmail.com>
26874 · Brandur <brandur@mutelight.org>
26876 · Dario Giovannetti <dev@dariogiovannetti.net>
26878 · Károly Oláh <okaresz@aol.com>
26880 · Jon Yergatian <jon@macfanatic.ca>
26882 · Jack Schmidt <github@mowsey.org>
26884 · Dedsec1 <Dedsec1@users.noreply.github.com>
26886 · Hisham Zarka <hzarka@gmail.com>
26888 · Jérôme Vizcaino <jerome.vizcaino@gmail.com>
26890 · Mike Tesch <mjt6129@rit.edu>
26892 · Marvin Watson <marvwatson@users.noreply.github.com>
26894 · Danny Tsai <danny8376@gmail.com>
26896 · Yoni Jah <yonjah+git@gmail.com> <yonjah+github@gmail.com>
26898 · Stephen Harris <github@spuddy.org> <sweharris@users.nore‐
26900 ply.github.com>
26901 · Ihor Dvoretskyi <ihor.dvoretskyi@gmail.com>
26903 · Jon Craton <jncraton@gmail.com>
26905 · Hraban Luyat <hraban@0brg.net>
26907 · Michael Ledin <mledin89@gmail.com>
26909 · Martin Kristensen <me@azgul.com>
26911 · Too Much IO <toomuchio@users.noreply.github.com>
26913 · Anisse Astier <anisse@astier.eu>
26915 · Zahiar Ahmed <zahiar@live.com>
26917 · Igor Kharin <igorkharin@gmail.com>
26919 · Bill Zissimopoulos <billziss@navimatics.com>
26921 · Bob Potter <bobby.potter@gmail.com>
26923 · Steven Lu <tacticalazn@gmail.com>
26925 · Sjur Fredriksen <sjurtf@ifi.uio.no>
26927 · Ruwbin <hubus12345@gmail.com>
26929 · Fabian Möller <fabianm88@gmail.com> <f.moeller@nynex.de>
26931 · Edward Q. Bridges <github@eqbridges.com>
26933 · Vasiliy Tolstov <v.tolstov@selfip.ru>
26935 · Harshavardhana <harsha@minio.io>
26937 · sainaen <sainaen@gmail.com>
26939 · gdm85 <gdm85@users.noreply.github.com>
26941 · Yaroslav Halchenko <debian@onerussian.com>
26943 · John Papandriopoulos <jpap@users.noreply.github.com>
26945 · Zhiming Wang <zmwangx@gmail.com>
26947 · Andy Pilate <cubox@cubox.me>
26949 · Oliver Heyme <olihey@googlemail.com> <olihey@users.nore‐
26951 ply.github.com> <de8olihe@lego.com>
26952 · wuyu <wuyu@yunify.com>
26954 · Andrei Dragomir <adragomi@adobe.com>
26956 · Christian Brüggemann <mail@cbruegg.com>
26958 · Alex McGrath Kraak <amkdude@gmail.com>
26960 · bpicode <bjoern.pirnay@googlemail.com>
26962 · Daniel Jagszent <daniel@jagszent.de>
26964 · Josiah White <thegenius2009@gmail.com>
26966 · Ishuah Kariuki <kariuki@ishuah.com> <ishuah91@gmail.com>
26968 · Jan Varho <jan@varho.org>
26970 · Girish Ramakrishnan <girish@cloudron.io>
26972 · LingMan <LingMan@users.noreply.github.com>
26974 · Jacob McNamee <jacobmcnamee@gmail.com>
26976 · jersou <jertux@gmail.com>
26978 · thierry <thierry@substantiel.fr>
26980 · Simon Leinen <simon.leinen@gmail.com> <ubuntu@s3-test.novalocal>
26982 · Dan Dascalescu <ddascalescu+github@gmail.com>
26984 · Jason Rose <jason@jro.io>
26986 · Andrew Starr-Bochicchio <a.starr.b@gmail.com>
26988 · John Leach <john@johnleach.co.uk>
26990 · Corban Raun <craun@instructure.com>
26992 · Pierre Carlson <mpcarl@us.ibm.com>
26994 · Ernest Borowski <er.borowski@gmail.com>
26996 · Remus Bunduc <remus.bunduc@gmail.com>
26998 · Iakov Davydov <iakov.davydov@unil.ch> <dav05.gith@myths.ru>
27000 · Jakub Tasiemski <tasiemski@gmail.com>
27002 · David Minor <dminor@saymedia.com>
27004 · Tim Cooijmans <cooijmans.tim@gmail.com>
27006 · Laurence <liuxy6@gmail.com>
27008 · Giovanni Pizzi <gio.piz@gmail.com>
27010 · Filip Bartodziej <filipbartodziej@gmail.com>
27012 · Jon Fautley <jon@dead.li>
27014 · lewapm <32110057+lewapm@users.noreply.github.com>
27016 · Yassine Imounachen <yassine256@gmail.com>
27018 · Chris Redekop <chris-redekop@users.noreply.github.com> <chris.re‐
27020 dekop@gmail.com>
27021 · Jon Fautley <jon@adenoid.appstal.co.uk>
27023 · Will Gunn <WillGunn@users.noreply.github.com>
27025 · Lucas Bremgartner <lucas@bremis.ch>
27027 · Jody Frankowski <jody.frankowski@gmail.com>
27029 · Andreas Roussos <arouss1980@gmail.com>
27031 · nbuchanan <nbuchanan@utah.gov>
27033 · Durval Menezes <rclone@durval.com>
27035 · Victor <vb-github@viblo.se>
27037 · Mateusz <pabian.mateusz@gmail.com>
27039 · Daniel Loader <spicypixel@gmail.com>
27041 · David0rk <davidork@gmail.com>
27043 · Alexander Neumann <alexander@bumpern.de>
27045 · Giri Badanahatti <gbadanahatti@us.ibm.com@Giris-MacBook-Pro.local>
27047 · Leo R. Lundgren <leo@finalresort.org>
27049 · wolfv <wolfv6@users.noreply.github.com>
27051 · Dave Pedu <dave@davepedu.com>
27053 · Stefan Lindblom <lindblom@spotify.com>
27055 · seuffert <oliver@seuffert.biz>
27057 · gbadanahatti <37121690+gbadanahatti@users.noreply.github.com>
27059 · Keith Goldfarb <barkofdelight@gmail.com>
27061 · Steve Kriss <steve@heptio.com>
27063 · Chih-Hsuan Yen <yan12125@gmail.com>
27065 · Alexander Neumann <fd0@users.noreply.github.com>
27067 · Matt Holt <mholt@users.noreply.github.com>
27069 · Eri Bastos <bastos.eri@gmail.com>
27071 · Michael P. Dubner <pywebmail@list.ru>
27073 · Antoine GIRARD <sapk@users.noreply.github.com>
27075 · Mateusz Piotrowski <mpp302@gmail.com>
27077 · Animosity022 <animosity22@users.noreply.github.com> <earl.tex‐
27079 ter@gmail.com>
27080 · Peter Baumgartner <pete@lincolnloop.com>
27082 · Craig Rachel <craig@craigrachel.com>
27084 · Michael G. Noll <miguno@users.noreply.github.com>
27086 · hensur <me@hensur.de>
27088 · Oliver Heyme <de8olihe@lego.com>
27090 · Richard Yang <richard@yenforyang.com>
27092 · Piotr Oleszczyk <piotr.oleszczyk@gmail.com>
27094 · Rodrigo <rodarima@gmail.com>
27096 · NoLooseEnds <NoLooseEnds@users.noreply.github.com>
27098 · Jakub Karlicek <jakub@karlicek.me>
27100 · John Clayton <john@codemonkeylabs.com>
27102 · Kasper Byrdal Nielsen <byrdal76@gmail.com>
27104 · Benjamin Joseph Dag <bjdag1234@users.noreply.github.com>
27106 · themylogin <themylogin@gmail.com>
27108 · Onno Zweers <onno.zweers@surfsara.nl>
27110 · Jasper Lievisse Adriaanse <jasper@humppa.nl>
27112 · sandeepkru <sandeep.ummadi@gmail.com> <sandeepkru@users.nore‐
27114 ply.github.com>
27115 · HerrH <atomtigerzoo@users.noreply.github.com>
27117 · Andrew <4030760+sparkyman215@users.noreply.github.com>
27119 · dan smith <XX1011@gmail.com>
27121 · Oleg Kovalov <iamolegkovalov@gmail.com>
27123 · Ruben Vandamme <github-com-00ff86@vandamme.email>
27125 · Cnly <minecnly@gmail.com>
27127 · Andres Alvarez <1671935+kir4h@users.noreply.github.com>
27129 · reddi1 <xreddi@gmail.com>
27131 · Matt Tucker <matthewtckr@gmail.com>
27133 · Sebastian Bünger <buengese@gmail.com>
27135 · Martin Polden <mpolden@mpolden.no>
27137 · Alex Chen <Cnly@users.noreply.github.com>
27139 · Denis <deniskovpen@gmail.com>
27141 · bsteiss <35940619+bsteiss@users.noreply.github.com>
27143 · Cédric Connes <cedric.connes@gmail.com>
27145 · Dr. Tobias Quathamer <toddy15@users.noreply.github.com>
27147 · dcpu <42736967+dcpu@users.noreply.github.com>
27149 · Sheldon Rupp <me@shel.io>
27151 · albertony <12441419+albertony@users.noreply.github.com>
27153 · cron410 <cron410@gmail.com>
27155 · Anagh Kumar Baranwal <anaghk.dos@gmail.com> <6824881+darthShad‐
27157 ow@users.noreply.github.com>
27158 · Felix Brucker <felix@felixbrucker.com>
27160 · Santiago Rodríguez <scollazo@users.noreply.github.com>
27162 · Craig Miskell <craig.miskell@fluxfederation.com>
27164 · Antoine GIRARD <sapk@sapk.fr>
27166 · Joanna Marek <joanna.marek@u2i.com>
27168 · frenos <frenos@users.noreply.github.com>
27170 · ssaqua <ssaqua@users.noreply.github.com>
27172 · xnaas <me@xnaas.info>
27174 · Frantisek Fuka <fuka@fuxoft.cz>
27176 · Paul Kohout <pauljkohout@yahoo.com>
27178 · dcpu <43330287+dcpu@users.noreply.github.com>
27180 · jackyzy823 <jackyzy823@gmail.com>
27182 · David Haguenauer <ml@kurokatta.org>
27184 · teresy <hi.teresy@gmail.com>
27186 · buergi <patbuergi@gmx.de>
27188 · Florian Gamboeck <mail@floga.de>
27190 · Ralf Hemberger <10364191+rhemberger@users.noreply.github.com>
27192 · Scott Edlund <sedlund@users.noreply.github.com>
27194 · Erik Swanson <erik@retailnext.net>
27196 · Jake Coggiano <jake@stripe.com>
27198 · brused27 <brused27@noemailaddress>
27200 · Peter Kaminski <kaminski@istori.com>
27202 · Henry Ptasinski <henry@logout.com>
27204 · Alexander <kharkovalexander@gmail.com>
27206 · Garry McNulty <garrmcnu@gmail.com>
27208 · Mathieu Carbou <mathieu.carbou@gmail.com>
27210 · Mark Otway <mark@otway.com>
27212 · William Cocker <37018962+WilliamCocker@users.noreply.github.com>
27214 · François Leurent <131.js@cloudyks.org>
27216 · Arkadius Stefanski <arkste@gmail.com>
27218 · Jay <dev@jaygoel.com>
27220 · andrea rota <a@xelera.eu>
27222 · nicolov <nicolov@users.noreply.github.com>
27224 · Dario Guzik <dario@guzik.com.ar>
27226 · qip <qip@users.noreply.github.com>
27228 · yair@unicorn <yair@unicorn>
27230 · Matt Robinson <brimstone@the.narro.ws>
27232 · kayrus <kay.diam@gmail.com>
27234 · Rémy Léone <remy.leone@gmail.com>
27236 · Wojciech Smigielski <wojciech.hieronim.smigielski@gmail.com>
27238 · weetmuts <oehrstroem@gmail.com>
27240 · Jonathan <vanillajonathan@users.noreply.github.com>
27242 · James Carpenter <orbsmiv@users.noreply.github.com>
27244 · Vince <vince0villamora@gmail.com>
27246 · Nestar47 <47841759+Nestar47@users.noreply.github.com>
27248 · Six <brbsix@gmail.com>
27250 · Alexandru Bumbacea <alexandru.bumbacea@booking.com>
27252 · calisro <robert.calistri@gmail.com>
27254 · Dr.Rx <david.rey@nventive.com>
27256 · marcintustin <marcintustin@users.noreply.github.com>
27258 · jaKa Močnik <jaka@koofr.net>
27260 · Fionera <fionera@fionera.de>
27262 · Dan Walters <dan@walters.io>
27264 · Danil Semelenov <sgtpep@users.noreply.github.com>
27266 · xopez <28950736+xopez@users.noreply.github.com>
27268 · Ben Boeckel <mathstuf@gmail.com>
27270 · Manu <manu@snapdragon.cc>
27272 · Kyle E. Mitchell <kyle@kemitchell.com>
27274 · Gary Kim <gary@garykim.dev>
27276 · Jon <jonathn@github.com>
27278 · Jeff Quinn <jeffrey.quinn@bluevoyant.com>
27280 · Peter Berbec <peter@berbec.com>
27282 · didil <1284255+didil@users.noreply.github.com>
27284 · id01 <gaviniboom@gmail.com>
27286 · Robert Marko <robimarko@gmail.com>
27288 · Philip Harvey <32467456+pharveybattelle@users.noreply.github.com>
27290 · JorisE <JorisE@users.noreply.github.com>
27292 · garry415 <garry.415@gmail.com>
27294 · forgems <forgems@gmail.com>
27296 · Florian Apolloner <florian@apolloner.eu>
27298 · Aleksandar Janković <office@ajankovic.com> <ajankovic@users.nore‐
27300 ply.github.com>
27301 · Maran <maran@protonmail.com>
27303 · nguyenhuuluan434 <nguyenhuuluan434@gmail.com>
27305 · Laura Hausmann <zotan@zotan.pw> <laura@hausmann.dev>
27307 · yparitcher <y@paritcher.com>
27309 · AbelThar <abela.tharen@gmail.com>
27311 · Matti Niemenmaa <matti.niemenmaa+git@iki.fi>
27313 · Russell Davis <russelldavis@users.noreply.github.com>
27315 · Yi FU <yi.fu@tink.se>
27317 · Paul Millar <paul.millar@desy.de>
27319 · justinalin <justinalin@qnap.com>
27321 · EliEron <subanimehd@gmail.com>
27323 · justina777 <chiahuei.lin@gmail.com>
27325 · Chaitanya Bankanhal <bchaitanya15@gmail.com>
27327 · Michał Matczuk <michal@scylladb.com>
27329 · Macavirus <macavirus@zoho.com>
27331 · Abhinav Sharma <abhi18av@users.noreply.github.com>
27333 · ginvine <34869051+ginvine@users.noreply.github.com>
27335 · Patrick Wang <mail6543210@yahoo.com.tw>
27337 · Cenk Alti <cenkalti@gmail.com>
27339 · Andreas Chlupka <andy@chlupka.com>
27341 · Alfonso Montero <amontero@tinet.org>
27343 · Ivan Andreev <ivandeex@gmail.com>
27345 · David Baumgold <david@davidbaumgold.com>
27347 · Lars Lehtonen <lars.lehtonen@gmail.com>
27349 · Matei David <matei.david@gmail.com>
27351 · David <david.bramwell@endemolshine.com>
27353 · Anthony Rusdi <33247310+antrusd@users.noreply.github.com>
27355 · Richard Patel <me@terorie.dev>
27357 · 庄天翼 <zty0826@gmail.com>
27359 · SwitchJS <dev@switchjs.com>
27361 · Raphael <PowershellNinja@users.noreply.github.com>
27363 · Sezal Agrawal <sezalagrawal@gmail.com>
27365 · Tyler <TylerNakamura@users.noreply.github.com>
27367 · Brett Dutro <brett.dutro@gmail.com>
27369 · Vighnesh SK <booterror99@gmail.com>
27371 · Arijit Biswas <dibbyo456@gmail.com>
27373 · Michele Caci <michele.caci@gmail.com>
27375 · AlexandrBoltris <ua2fgb@gmail.com>
27377 · Bryce Larson <blarson@saltstack.com>
27379 · Carlos Ferreyra <crypticmind@gmail.com>
27381 · Saksham Khanna <sakshamkhanna@outlook.com>
27383 · dausruddin <5763466+dausruddin@users.noreply.github.com>
27385 · zero-24 <zero-24@users.noreply.github.com>
27387 · Xiaoxing Ye <ye@xiaoxing.us>
27389 · Barry Muldrey <barry@muldrey.net>
27391 · Sebastian Brandt <sebastian.brandt@friday.de>
27393 · Marco Molteni <marco.molteni@mailbox.org>
27395 · Ankur Gupta <ankur0493@gmail.com>
27397 · Maciej Zimnoch <maciej@scylladb.com>
27399 · anuar45 <serdaliyev.anuar@gmail.com>
27401 · Fernando <ferferga@users.noreply.github.com>
27403 · David Cole <david.cole@sohonet.com>
27405 · Wei He <git@weispot.com>
27407 · Outvi V <19144373+outloudvi@users.noreply.github.com>
27409 · Thomas Kriechbaumer <thomas@kriechbaumer.name>
27411 · Tennix <tennix@users.noreply.github.com>
27413 · Ole Schütt <ole@schuett.name>
27415 · Kuang-che Wu <kcwu@csie.org>
27417 · Thomas Eales <wingsuit@users.noreply.github.com>
27419 · Paul Tinsley <paul.tinsley@vitalsource.com>
27421 · Felix Hungenberg <git@shiftgeist.com>
27423 · Benjamin Richter <github@dev.telepath.de>
27425 · landall <cst_zf@qq.com>
27427 · thestigma <thestigma@gmail.com>
27429 · jtagcat <38327267+jtagcat@users.noreply.github.com>
27431 · Damon Permezel <permezel@me.com>
27433 · boosh <boosh@users.noreply.github.com>
27435 · unbelauscht <58393353+unbelauscht@users.noreply.github.com>
27437 · Motonori IWAMURO <vmi@nifty.com>
27439 · Benjapol Worakan <benwrk@live.com>
27441
27443 Forum
27444 Forum for questions and general discussion:
27445 · https://forum.rclone.org
27447 Gitub project
27449 The project website is at:
27450 · https://github.com/rclone/rclone
27452 There you can file bug reports or contribute pull requests.
27454 Twitter
27456 You can also follow me on twitter for rclone announcements:
27457 · [@njcw](https://twitter.com/njcw)
27459 Email
27461 Or if all else fails or you want to ask something private or confiden‐
27462 tial email Nick Craig-Wood (mailto:nick@craig-wood.com). Please don’t
27463 email me requests for help - those are better directed to the forum -
27464 thanks!
27465
27467 Nick Craig-Wood.
27468User Manual Feb 01, 2020 rclone(1)