1SYNCTHING-FAQ(7) Syncthing SYNCTHING-FAQ(7)
2
6 syncthing-faq - Frequently Asked Questions
7
9 Syncthing is an application that lets you synchronize your files across
10 multiple devices. This means the creation, modification or deletion of
11 files on one machine will automatically be replicated to your other
12 devices. We believe your data is your data alone and you deserve to
13 choose where it is stored. Therefore Syncthing does not upload your
14 data to the cloud but exchanges your data across your machines as soon
15 as they are online at the same time.
16
18 It’s Syncthing, although the command and source repository is spelled
19 syncthing so it may be referred to in that way as well. It’s definitely
20 not SyncThing, even though the abbreviation st is used in some circum‐
21 stances and file names.
22
24 The two are different and not related. Syncthing and BitTorrent/Resilio
25 Sync accomplish some of the same things, namely syncing files between
26 two or more computers.
27 BitTorrent Sync, now called Resilio Sync, is a proprietary peer-to-peer
29 file synchronization tool available for Windows, Mac, Linux, Android,
30 iOS, Windows Phone, Amazon Kindle Fire and BSD. [1] Syncthing is an
31 open source file synchronization tool.
32 Syncthing uses an open and documented protocol, and likewise the secu‐
34 rity mechanisms in use are well defined and visible in the source code.
35 Resilio Sync uses an undocumented, closed protocol with unknown secu‐
36 rity properties.
37 [1] https://en.wikipedia.org/wiki/Resilio_Sync
39
41 The following things are always synchronized:
42 · File Contents
44 · File Modification Times
46 The following may be synchronized or not, depending:
48 · File Permissions (When supported by file system. On Windows, only the
50 read only bit is synchronized.)
51 · Symbolic Links (synced, except on Windows, but never followed.)
53 The following are not synchronized;
55 · File or Directory Owners and Groups (not preserved)
57 · Directory Modification Times (not preserved)
59 · Hard Links (followed, not preserved)
61 · Extended Attributes, Resource Forks (not preserved)
63 · Windows, POSIX or NFS ACLs (not preserved)
65 · Devices, FIFOs, and Other Specials (ignored)
67 · Sparse file sparseness (will become sparse, when supported by the OS
69 & filesystem)
70
72 Syncthing segments files into pieces, called blocks, to transfer data
73 from one device to another. Therefore, multiple devices can share the
74 synchronization load, in a similar way to the torrent protocol. The
75 more devices you have online, the faster an additional device will
76 receive the data because small blocks will be fetched from all devices
77 in parallel.
78 Syncthing handles renaming files and updating their metadata in an
80 efficient manner. This means that renaming a large file will not cause
81 a retransmission of that file. Additionally, appending data to existing
82 large files should be handled efficiently as well.
83 Temporary files are used to store partial data downloaded from other
85 devices. They are automatically removed whenever a file transfer has
86 been completed or after the configured amount of time which is set in
87 the configuration file (24 hours by default).
88
90 When troubleshooting a slow sync, there are a number of things to
91 check.
92 First of all, verify that you are not connected via a relay. In the
94 “Remote Devices” list on the right side of the GUI, double check that
95 you see “Address: <some address>” and not “Relay: <some address>”.
96 [image]
97 If you are connected via a relay, this is because a direct connection
99 could not be established. Double check and follow the suggestions in
100 firewall-setup to enable direct connections.
101 Second, if one of the devices is a very low powered machine (a Rasp‐
103 berry Pi, or a phone, or a NAS, or similar) you are likely constrained
104 by the CPU on that device. See the next question for reasons Syncthing
105 likes a faster CPU. You can verify this by looking at the CPU utiliza‐
106 tion in the GUI. If it is constantly at or close to 100%, you are lim‐
107 ited by the CPU speed. In some cases a lower CPU usage number can also
108 indicate being limited by the CPU - for example constant 25% usage on a
109 four core CPU likely means that Syncthing is doing something that is
110 not parallellizable and thus limited to a single CPU core.
111 Third, verify that the network connection is OK. Tools such as iperf or
113 just an Internet speed test can be used to verify the performance here.
114
116 1. When new or changed files are detected, or Syncthing starts for the
117 first time, your files are hashed using SHA-256.
118 2. Data that is sent over the network is compressed (optionally) and
120 encrypted (always). When receiving data it must be decrypted and
121 then (if compressed) decompressed.
122 3. There is a certain amount of housekeeping that must be done to track
124 the current and available versions of each file in the index data‐
125 base.
126 4. By default Syncthing uses periodic scanning every hour when watching
128 for changes or every minute if that’s disabled to detect file
129 changes. This means checking every file’s modification time and com‐
130 paring it to the database. This can cause spikes of CPU usage for
131 large folders.
132 Hashing, compression and encryption cost CPU time. Also, using the GUI
134 causes a certain amount of extra CPU usage to calculate the summary
135 data it presents. Note however that once things are in sync CPU usage
136 should be negligible.
137 To minimize the impact of this, Syncthing attempts to lower the process
139 priority when starting up.
140 To further limit the amount of CPU used when syncing and scanning, set
142 the environment variable GOMAXPROCS to the maximum number of CPU cores
143 Syncthing should use at any given moment. For example, GOMAXPROCS=2 on
144 a machine with four cores will limit Syncthing to no more than half the
145 system’s CPU power.
146
148 No. The IDs are not sensitive. Given a device ID it’s possible to find
149 the IP address for that device, if global discovery is enabled on it.
150 Knowing the device ID doesn’t help you actually establish a connection
151 to that device or get a list of files, etc.
152 For a connection to be established, both devices need to know about the
154 other’s device ID. It’s not possible (in practice) to forge a device
155 ID. (To forge a device ID you need to create a TLS certificate with
156 that specific SHA-256 hash. If you can do that, you can spoof any TLS
157 certificate. The world is your oyster!)
158 SEE ALSO:
160 device-ids
161
163 Syncthing does recognize conflicts. When a file has been modified on
164 two devices simultaneously and the content actually differs, one of the
165 files will be renamed to <filename>.sync-conflict-<date>-<time>-<modi‐
166 fiedBy>.<ext>. The file with the older modification time will be marked
167 as the conflicting file and thus be renamed. If the modification times
168 are equal, the file originating from the device which has the larger
169 value of the first 63 bits for his device ID will be marked as the con‐
170 flicting file. If the conflict is between a modification and a dele‐
171 tion of the file, the modified file always wins and is resurrected
172 without renaming on the device where it was deleted.
173 Beware that the <filename>.sync-conflict-<date>-<time>-<modi‐
175 fiedBy>.<ext> files are treated as normal files after they are created,
176 so they are propagated between devices. We do this because the conflict
177 is detected and resolved on one device, creating the sync-conflict
178 file, but it’s just as much of a conflict everywhere else and we don’t
179 know which of the conflicting files is the “best” from the user point
180 of view.
181
183 Syncthing requires a “folder marker” to indicate that the folder is
184 present and healthy. By default this is a directory called .stfolder
185 that is created by Syncthing when the folder is added. If this folder
186 can’t be created (you are serving files from a CD or something) you can
187 instead set the advanced config Marker Name to the name of some file or
188 folder that you know will always exist in the folder.
189
191 See the previous question.
192
194 Sharing a folder that is within an already shared folder is possible,
195 but it has its caveats. What you must absolutely avoid are circular
196 shares. This is just one example, there may be other undesired effects.
197 Nesting shared folders is not supported, recommended or coded for, but
198 it can be done successfully when you know what you’re doing - you have
199 been warned.
200
202 Syncthing doesn’t have a direct way to do this, as it’s potentially
203 dangerous to do so if you’re not careful - it may result in data loss
204 if something goes wrong during the move and is synchronized to your
205 other devices.
206 The easy way to rename or move a synced folder on the local system is
208 to remove the folder in the Syncthing UI, move it on disk, then re-add
209 it using the new path.
210 It’s best to do this when the folder is already in sync between your
212 devices, as it is otherwise unpredictable which changes will “win”
213 after the move. Changes made on other devices may be overwritten, or
214 changes made locally may be overwritten by those on other devices.
215 An alternative way is to shut down Syncthing, move the folder on disk
217 (including the .stfolder marker), edit the path directly in config.xml
218 in the configuration folder (see config) and then start Syncthing
219 again.
220
222 Each user should run their own Syncthing instance. Be aware that you
223 might need to configure listening ports such that they do not overlap
224 (see config).
225
227 No. Syncthing is not designed to sync locally and the overhead involved
228 in doing so using Syncthing’s method would be wasteful. There are bet‐
229 ter programs to achieve this such as rsync or Unison.
230
232 SYNCTHING HANDLE MOVING FILES BETWEEN THEM?
233 Syncthing does not specially handle this case, and most files most
234 likely get re-downloaded.
235 In detail, the behavior depends on the scan order. If you have folder A
237 and B, and move files from A to B, if A gets scanned first, it will
238 announce removal of the files to others who will remove the files. As
239 you rescan B, B will announce addition of new files, and other peers
240 will have nowhere to get them from apart from re-downloading them.
241 If B gets rescanned first, B will announce additions first, remote
243 peers will reconstruct the files (not rename, more like copy block by
244 block) from A, and then as A gets rescanned remove the files from A.
245 A workaround would be to copy first from A to B, rescan B, wait for B
247 to rebuild on remote ends, and then delete from A.
248
250 No. Syncthing is not a great backup application because all changes to
251 your files (modifications, deletions, etc.) will be propagated to all
252 your devices. You can enable versioning, but we encourage the use of
253 other tools to keep your data safe from your (or our) mistakes.
254
256 There is an alternative implementation of Syncthing (using the same
257 network protocol) called fsync(). There are no plans by the current
258 Syncthing team to support iOS in the foreseeable future, as the code
259 required to do so would be quite different from what Syncthing is
260 today.
261
263 The patterns in .stignore are glob patterns, where brackets are used to
264 denote character ranges. That is, the pattern q[abc]x will match the
265 files qax, qbx and qcx.
266 To match an actual file called q[abc]x the pattern needs to “escape”
268 the brackets, like so: q\[abc\]x.
269 On Windows, escaping special characters is not supported as the \ char‐
271 acter is used as a path separator. On the other hand, special charac‐
272 ters such as [ and ? are not allowed in file names on Windows.
273
275 Security over convenience. In Syncthing you have to setup both sides to
276 connect two devices. An attacker can’t do much with a stolen device ID,
277 because you have to add the device on the other side too. You have bet‐
278 ter control where your files are transferred.
279 This is an area that we are working to improve in the long term.
281
283 The default listening address is 127.0.0.1:8384, so you can only access
284 the GUI from the same machine. This is for security reasons. Change the
285 GUI listen address through the web UI from 127.0.0.1:8384 to
286 0.0.0.0:8384 or change the config.xml:
287 <gui enabled="true" tls="false">
289 <address>127.0.0.1:8384</address>
290 to
292 <gui enabled="true" tls="false">
294 <address>0.0.0.0:8384</address>
295 Then the GUI is accessible from everywhere. You should set a password
297 and enable HTTPS with this configuration. You can do this from inside
298 the GUI.
299 If both your computers are Unix-like (Linux, Mac, etc.) you can also
301 leave the GUI settings at default and use an ssh port forward to access
302 it. For example,
303 $ ssh -L 9090:127.0.0.1:8384 user@othercomputer.example.com
305 will log you into othercomputer.example.com, and present the remote
307 Syncthing GUI on http://localhost:9090 on your local computer.
308 If you only want to access the remote gui and don’t want the terminal
310 session, use this example,
311 $ ssh -N -L 9090:127.0.0.1:8384 user@othercomputer.example.com
313 If only your remote computer is Unix-like, you can still access it with
315 ssh from Windows.
316 Under Windows 10 (64 bit) you can use the same ssh command if you
318 install the Windows Subsystem for Linux.
319 https://msdn.microsoft.com/en-gb/commandline/wsl/install_guide
320 Another Windows way to run ssh is to install gow. (Gnu On Windows)
322 https://github.com/bmatzelle/gow
323 The easiest way to install gow is with chocolatey.
325 https://chocolatey.org/
326
328 Since version 0.14.6 Syncthing does an extra security check when the
329 GUI/API is bound to localhost - namely that the browser is talking to
330 localhost. This protects against most forms of DNS rebinding attack
331 <https://en.wikipedia.org/wiki/DNS_rebinding> against the GUI.
332 To pass this test, ensure that you are accessing the GUI using an URL
334 that begins with http://localhost, http://127.0.0.1 or http://[::1].
335 HTTPS is fine too, of course.
336 If you are using a proxy in front of Syncthing you may need to disable
338 this check, after ensuring that the proxy provides sufficient authenti‐
339 cation to protect against unauthorized access. Either:
340 · Make sure the proxy sets a Host header containing localhost, or
342 · Set insecureSkipHostcheck in the advanced settings, or
344 · Bind the GUI/API to a non-localhost listen port.
346 In all cases, username/password authentication and HTTPS should be
348 used.
349
351 This is almost always a result of bad RAM, storage device or other
352 hardware. When the index database is found to be corrupt Syncthing can‐
353 not operate and will note this in the logs and exit. To overcome this
354 delete the database folder <https://docs.syncthing.net/users/con‐
355 fig.html#description> inside Syncthing’s home directory and re-start
356 Syncthing. It will then need to perform a full re-hashing of all shared
357 folders. You should check your system in case the underlying cause is
358 indeed faulty hardware which may put the system at risk of further data
359 loss.
360
362 You can change the theme in the settings. Syncthing ships with other
363 themes than the default.
364 If you want a custom theme or a completely different GUI, you can add
366 your own. By default, Syncthing will look for a directory gui inside
367 the Syncthing home folder. To change the directory to look for themes,
368 you need to set the STGUIASSETS environment variable. To get the con‐
369 crete directory, run syncthing with the -paths parameter. It will print
370 all the relevant paths, including the “GUI override directory”.
371 To add e.g. a red theme, you can create the file
373 red/assets/css/theme.css inside the GUI override directory to override
374 the default CSS styles.
375 To create a whole new GUI, you should checkout the files at
377 https://github.com/syncthing/syncthing/tree/master/gui/default to get
378 an idea how to do that.
379
381 One process manages the other, to capture logs and manage restarts.
382 This makes it easier to handle upgrades from within Syncthing itself,
383 and also ensures that we get a nice log file to help us narrow down the
384 cause for crashes and other bugs.
385
387 Syncthing logs to stdout by default. On Windows Syncthing by default
388 also creates syncthing.log in Syncthing’s home directory (run syncthing
389 -paths to see where that is). Command line option -logfile can be used
390 to specify a user-defined logfile.
391
393 The web GUI contains a Global Changes button under the device list
394 which displays changes since the last (re)start of Syncthing. With the
395 -audit option you can enable a persistent, detailed log of changes and
396 most activities, which contains a JSON formatted sequence of events in
397 the ~/.config/syncthing/audit-_date_-_time_.log file.
398
400 The audit log (and the Global Changes window) sees the changes that
401 your Syncthing sees. When Syncthing is continuously connected it usu‐
402 ally sees every change happening immediately and thus knows which node
403 initiated the change. When topology gets complex or when your node
404 reconnects after some time offline, Syncthing synchronises with its
405 neighbours: It gets the latest synchronised state from the neighbour,
406 which is the result of all the changes between the last known state
407 (before disconnect or network delay) and the current state at the
408 neighbour, and if there were updates, deletes, creates, conflicts,
409 which were overlapping we only see the latest change for a given file
410 or directory (and the node where that latest change occurred). When we
411 connect to multiple neighbours Syncthing decides which neighbor has the
412 latest state, or if the states conflict it initiates the conflict reso‐
413 lution procedure, which in the end results in a consistent up-to-date
414 state with all the neighbours.
415
417 If you use a package manager such as Debian’s apt-get, you should
418 upgrade using the package manager. If you use the binary packages
419 linked from Syncthing.net, you can use Syncthing built in automatic
420 upgrades.
421 · If automatic upgrades is enabled (which is the default), Syncthing
423 will upgrade itself automatically within 24 hours of a new release.
424 · The upgrade button appears in the web GUI when a new version has been
426 released. Pressing it will perform an upgrade.
427 · To force an upgrade from the command line, run syncthing -upgrade.
429 Note that your system should have CA certificates installed which allow
431 a secure connection to GitHub (e.g. FreeBSD requires sudo pkg install
432 ca_root_nss). If curl or wget works with normal HTTPS sites, then so
433 should Syncthing.
434
436 We release new versions through GitHub. The latest release is always
437 found on the release page <https://github.com/syncthing/sync‐
438 thing/releases/latest>. Unfortunately GitHub does not provide a single
439 URL to automatically download the latest version. We suggest to use the
440 GitHub API at
441 https://api.github.com/repos/syncthing/syncthing/releases/latest and
442 parsing the JSON response.
443
445 If you’re using systemd, runit, or upstart, we already ship examples,
446 check https://github.com/syncthing/syncthing/tree/master/etc for exam‐
447 ple configurations.
448 If however you’re not using one of these tools, you have a couple of
450 options. If your system has a tool called start-stop-daemon installed
451 (that’s the name of the command, not the package), look into the local
452 documentation for that, it will almost certainly cover 100% of what you
453 want to do. If you don’t have start-stop-daemon, there are a bunch of
454 other software packages you could use to do this. The most well known
455 is called daemontools, and can be found in the standard package reposi‐
456 tories for almost every modern Linux distribution. Other popular
457 tools with similar functionality include S6 and the aforementioned
458 runit.
459
461 You are probably reading this because you encountered the following
462 error with the filesystem watcher on linux:
463 Failed to start filesystem watcher for folder yourLabel (yourID):
464 failed to setup inotify handler. Please increase inotify limits, see
465 https://docs.syncthing.net/users/faq.html#inotify-limits
466 Linux typically restricts the amount of watches per user (usually
468 8192). When you have more directories you need to adjust that number.
469 On many Linux distributions you can run the following to fix it:
471 echo "fs.inotify.max_user_watches=204800" | sudo tee -a /etc/sysctl.conf
473 On Arch Linux and potentially others it is preferred to write this line
475 into a separate file, i.e. you should run:
476 echo "fs.inotify.max_user_watches=204800" | sudo tee -a /etc/sysctl.d/90-override.conf
478 This only takes effect after a reboot. To adjust the limit immediately,
480 run:
481 sudo sh -c 'echo 204800 > /proc/sys/fs/inotify/max_user_watches'
483
485 The Syncthing Authors
486
488 2014-2019, The Syncthing Authors
489v1 Apr 13, 2019 SYNCTHING-FAQ(7)