1SYNCTHING-BEP(7)                   Syncthing                  SYNCTHING-BEP(7)
2
3
4

NAME

6       syncthing-bep - Block Exchange Protocol v1
7

INTRODUCTION AND DEFINITIONS

9       The  Block  Exchange Protocol (BEP) is used between two or more devices
10       thus forming a cluster. Each device has one or more  folders  of  files
11       described by the local model, containing metadata and block hashes. The
12       local model is sent to the other devices in the cluster. The  union  of
13       all  files  in the local models, with files selected for highest change
14       version, forms the global model. Each device strives to get its folders
15       in  sync with the global model by requesting missing or outdated blocks
16       from the other devices in the cluster.
17
18       File data is described and transferred in units of blocks,  each  being
19       from  128  KiB  (131072 bytes) to 16 MiB in size, in steps of powers of
20       two. The block size may vary between files but is constant in any given
21       file, except for the last block which may be smaller.
22
23       The  key  words  “MUST”,  “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”,
24       “SHOULD”, “SHOULD NOT”, “RECOMMENDED”, “MAY”, and  “OPTIONAL”  in  this
25       document  are to be interpreted as described in RFC 2119 <https://data
26       tracker.ietf.org/doc/html/rfc2119.html>.
27

TRANSPORT AND AUTHENTICATION

29       BEP is deployed as the highest level in  a  protocol  stack,  with  the
30       lower level protocols providing encryption and authentication.
31
32          +-----------------------------+
33          |   Block Exchange Protocol   |
34          |-----------------------------|
35          | Encryption & Auth (TLS 1.2) |
36          |-----------------------------|
37          |      Reliable Transport     |
38          |-----------------------------|
39          v             ...             v
40
41       The  encryption  and authentication layer SHALL use TLS 1.2 or a higher
42       revision. A strong cipher suite SHALL  be  used,  with  “strong  cipher
43       suite”  being  defined  as being without known weaknesses and providing
44       Perfect Forward Secrecy (PFS). Examples of  strong  cipher  suites  are
45       given  at  the  end of this document. This is not to be taken as an ex‐
46       haustive list of allowed cipher suites but represents best practices at
47       the time of writing.
48
49       The  exact  nature of the authentication is up to the application, how‐
50       ever it SHALL be based on the TLS certificate presented at the start of
51       the  connection.  Possibilities include certificates signed by a common
52       trusted CA, preshared certificates, preshared certificate  fingerprints
53       or  certificate  pinning combined with some out of band first verifica‐
54       tion. The reference implementation uses preshared  certificate  finger‐
55       prints (SHA-256) referred to as “Device IDs”.
56
57       There is no required order or synchronization among BEP messages except
58       as noted per message type - any message type may be sent  at  any  time
59       and  the sender need not await a response to one message before sending
60       another.
61
62       The underlying transport protocol MUST guarantee reliable packet deliv‐
63       ery.
64
65       In this document, in diagrams and text, “bit 0” refers to the most sig‐
66       nificant bit of a word; “bit 15” is thus the least significant bit of a
67       16  bit  word (int16) and “bit 31” is the least significant bit of a 32
68       bit word (int32). Non protocol buffer integers are  always  represented
69       in  network  byte order (i.e., big endian) and are signed unless stated
70       otherwise, but when describing message lengths negative values  do  not
71       make sense and the most significant bit MUST be zero.
72
73       The protocol buffer schemas in this document are in proto3 syntax. This
74       means, among other things, that all fields are optional and will assume
75       their default value when missing. This does not necessarily mean that a
76       message is valid with all fields empty - for example,  an  index  entry
77       for  a file that does not have a name is not useful and MAY be rejected
78       by the implementation. However the folder label is for  human  consump‐
79       tion  only  so  an  empty label should be accepted - the implementation
80       will have to choose some way to represent the folder, perhaps by  using
81       the ID in it’s place or automatically generating a label.
82

PRE-AUTHENTICATION MESSAGES

84       AFTER  establishing a connection, but BEFORE performing any authentica‐
85       tion, devices MUST exchange Hello messages.
86
87       Hello messages are used to carry additional information about the peer,
88       which  might be of interest to the user even if the peer is not permit‐
89       ted to communicate due to failing authentication. Note  that  the  cer‐
90       tificate  based  authentication may be considered part of the TLS hand‐
91       shake that precedes the Hello message exchange, but even  in  the  case
92       that  a  connection is rejected a Hello message must be sent before the
93       connection is terminated.
94
95       Hello messages MUST be prefixed with an int32 containing the magic num‐
96       ber  0x2EA7D90B, followed by an int16 representing the size of the mes‐
97       sage, followed by the contents of the Hello message itself.
98
99           0                   1
100           0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5
101          +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
102          |             Magic             |
103          |           (32 bits)           |
104          +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
105          |             Length            |
106          +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
107          /                               /
108          \             Hello             \
109          /                               /
110          +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
111
112       The Hello message itself is in protocol buffer format with the  follow‐
113       ing schema:
114
115          message Hello {
116              string device_name    = 1;
117              string client_name    = 2;
118              string client_version = 3;
119          }
120
121   Fields (Hello message)
122       The  device_name  is a human readable (configured or auto detected) de‐
123       vice name or host name, for the remote device.
124
125       The client_name and client_version identifies the  implementation.  The
126       values  SHOULD   be simple strings identifying the implementation name,
127       as a user would expect to see it, and the version string  in  the  same
128       manner.  An  example  client  name is “syncthing” and an example client
129       version is “v0.7.2”.  The client version field SHOULD follow  the  pat‐
130       terns  laid  out in the Semantic Versioning <https://semver.org/> stan‐
131       dard.
132
133       Immediately after exchanging Hello messages,  the  connection  MUST  be
134       dropped if the remote device does not pass authentication.
135

POST-AUTHENTICATION MESSAGES

137       Every message post authentication is made up of several parts:
138
139       • A header length word
140
141       • A Header
142
143       • A message length word
144
145       • A Message
146
147           0                   1
148           0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5
149          +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
150          |         Header Length         |
151          +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
152          /                               /
153          \            Header             \
154          /                               /
155          +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
156          |         Message Length        |
157          |           (32 bits)           |
158          +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
159          /                               /
160          \            Message            \
161          /                               /
162          +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
163
164       The  header length word is 16 bits. It indicates the length of the fol‐
165       lowing Header message. The Header is in  protocol  buffer  format.  The
166       Header  describes the type and compression status of the following mes‐
167       sage.
168
169       The message is preceded by the 32 bit message length word and is one of
170       the concrete BEP messages described below, identified by the type field
171       of the Header.
172
173       As always, the length words are in network byte order (big endian).
174
175          message Header {
176              MessageType        type        = 1;
177              MessageCompression compression = 2;
178          }
179
180          enum MessageType {
181              CLUSTER_CONFIG    = 0;
182              INDEX             = 1;
183              INDEX_UPDATE      = 2;
184              REQUEST           = 3;
185              RESPONSE          = 4;
186              DOWNLOAD_PROGRESS = 5;
187              PING              = 6;
188              CLOSE             = 7;
189          }
190
191          enum MessageCompression {
192              NONE = 0;
193              LZ4  = 1;
194          }
195
196       When the compression field is NONE, the message is directly in protocol
197       buffer format.
198
199       When the compression field is LZ4, the message consists of a 32 bit in‐
200       teger describing the uncompressed message length followed by  a  single
201       LZ4  block.  After decompressing the LZ4 block it should be interpreted
202       as a protocol buffer message just as in the uncompressed case.
203

MESSAGE SUBTYPES

205   Cluster Config
206       This informational message provides information about the cluster  con‐
207       figuration  as  it pertains to the current connection. A Cluster Config
208       message MUST be the first post authentication message  sent  on  a  BEP
209       connection.  Additional  Cluster Config messages MUST NOT be sent after
210       the initial exchange.
211
212   Protocol Buffer Schema
213          message ClusterConfig {
214              repeated Folder folders = 1;
215          }
216
217          message Folder {
218              string id                   = 1;
219              string label                = 2;
220              bool   read_only            = 3;
221              bool   ignore_permissions   = 4;
222              bool   ignore_delete        = 5;
223              bool   disable_temp_indexes = 6;
224              bool   paused               = 7;
225
226              repeated Device devices = 16;
227          }
228
229          message Device {
230              bytes           id                         = 1;
231              string          name                       = 2;
232              repeated string addresses                  = 3;
233              Compression     compression                = 4;
234              string          cert_name                  = 5;
235              int64           max_sequence               = 6;
236              bool            introducer                 = 7;
237              uint64          index_id                   = 8;
238              bool            skip_introduction_removals = 9;
239              bytes           encryption_password_token  = 10;
240          }
241
242          enum Compression {
243              METADATA = 0;
244              NEVER    = 1;
245              ALWAYS   = 2;
246          }
247
248   Fields (Cluster Config Message)
249       The folders field contains the list of folders that  will  be  synchro‐
250       nized over the current connection.
251
252   Fields (Folder Message)
253       The  id field contains the folder ID, which is the unique identifier of
254       the folder.
255
256       The label field contains the folder label, the human readable  name  of
257       the folder.
258
259       The  read  only field is set for folders that the device will accept no
260       updates from the network for.
261
262       The ignore permissions field is set for folders that  the  device  will
263       not accept or announce file permissions for.
264
265       The  ignore delete field is set for folders that the device will ignore
266       deletes for.
267
268       The disable temp indexes field is set for folders that  will  not  dis‐
269       patch and do not wish to receive progress updates about partially down‐
270       loaded files via Download Progress messages.
271
272       The paused field is set for folders that are currently paused.
273
274       The devices field is a list of devices participating  in  sharing  this
275       folder.
276
277   Fields (Device Message)
278       The  device  id  field is a 32 byte number that uniquely identifies the
279       device. For instance, the reference implementation uses the SHA-256  of
280       the device X.509 certificate.
281
282       The  name  field is a human readable name assigned to the described de‐
283       vice by the sending device. It MAY be empty and it need not be unique.
284
285       The list of addresses is that used by the sending device to connect  to
286       the described device.
287
288       The  compression  field  indicates the compression mode in use for this
289       device and folder. The following values are valid:
290
291       0      Compress metadata. This enables compression of metadata messages
292              such as Index.
293
294       1      Compression disabled. No compression is used on any message.
295
296       2      Compress  always. Metadata messages as well as Response messages
297              are compressed.
298
299       The cert name field indicates the expected certificate  name  for  this
300       device.  It is commonly blank, indicating to use the implementation de‐
301       fault.
302
303       The max sequence field contains the  highest  sequence  number  of  the
304       files  in  the  index.  See  Delta Index Exchange for the usage of this
305       field.
306
307       The introducer field is set for devices that are trusted as cluster in‐
308       troducers.
309
310       The  index  id field contains the unique identifier for the current set
311       of index data. See Delta Index Exchange for the usage of this field.
312
313       The skip introduction removals field signifies if the remote device has
314       opted  to  ignore introduction removals for the given device. This set‐
315       ting is copied across as we are being introduced to a new device.
316
317       The enc pw token field contains a token derived from the password, that
318       is  used to encrypt data sent to this device. If the device is the same
319       as the device sending the message, it signifies that the device  itself
320       has encrypted data that was encrypted with the given token. It is empty
321       or missing if there is no encryption. See untrusted for details on  the
322       encryption scheme.
323
324   Index and Index Update
325       The  Index and Index Update messages define the contents of the senders
326       folder. An Index message represents the full contents of the folder and
327       thus  supersedes any previous index. An Index Update amends an existing
328       index with new information, not affecting any entries not  included  in
329       the  message. An Index Update MAY NOT be sent unless preceded by an In‐
330       dex, unless a non-zero Max Sequence has been announced  for  the  given
331       folder by the peer device.
332
333       The  Index and Index Update messages are currently identical in format,
334       although this is not guaranteed to be the case in the future.
335
336   Protocol Buffer Schema
337          message Index {
338              string            folder = 1;
339              repeated FileInfo files  = 2;
340          }
341
342          message IndexUpdate {
343              string            folder = 1;
344              repeated FileInfo files  = 2;
345          }
346
347          message FileInfo {
348              string       name           = 1;
349              FileInfoType type           = 2;
350              int64        size           = 3;
351              uint32       permissions    = 4;
352              int64        modified_s     = 5;
353              int32        modified_ns    = 11;
354              uint64       modified_by    = 12;
355              bool         deleted        = 6;
356              bool         invalid        = 7;
357              bool         no_permissions = 8;
358              Vector       version        = 9;
359              int64        sequence       = 10;
360              int32        block_size     = 13;
361
362              repeated BlockInfo Blocks         = 16;
363              string             symlink_target = 17;
364          }
365
366          enum FileInfoType {
367              FILE              = 0;
368              DIRECTORY         = 1;
369              SYMLINK_FILE      = 2 [deprecated = true];
370              SYMLINK_DIRECTORY = 3 [deprecated = true];
371              SYMLINK           = 4;
372          }
373
374          message BlockInfo {
375              int64 offset     = 1;
376              int32 size       = 2;
377              bytes hash       = 3;
378              uint32 weak_hash = 4;
379          }
380
381          message Vector {
382              repeated Counter counters = 1;
383          }
384
385          message Counter {
386              uint64 id    = 1;
387              uint64 value = 2;
388          }
389
390   Fields (Index Message)
391       The folder field identifies the folder that the index message  pertains
392       to.
393
394       The files field is a list of files making up the index information.
395
396   Fields (FileInfo Message)
397       The  name  is  the file name path relative to the folder root. Like all
398       strings in BEP, the Name is always in UTF-8 NFC regardless of operating
399       system  or  file  system  specific conventions. The name field uses the
400       slash character (“/”) as path separator, regardless of the  implementa‐
401       tion’s operating system conventions. The combination of folder and name
402       uniquely identifies each file in a cluster.
403
404       The type field contains the type of the described item. The type is one
405       of file (0), directory (1), or symlink (4).
406
407       The size field contains the size of the file, in bytes. For directories
408       and symlinks the size is zero.
409
410       The permissions field holds the common Unix permission bits. An  imple‐
411       mentation  MAY ignore or interpret these as is suitable on the host op‐
412       erating system.
413
414       The modified_s time is expressed as the number  of  seconds  since  the
415       Unix  Epoch  (1970-01-01 00:00:00 UTC). The modified_ns field holds the
416       nanosecond part of the modification time.
417
418       The modified_by field holds the short id of the client that  last  made
419       any  modification to the file whether add, change or delete.  This will
420       be overwritten every time a change is made to  the  file  by  the  last
421       client to do so and so does not hold history.
422
423       The deleted field is set when the file has been deleted. The block list
424       SHALL be of length zero and the modification time indicates the time of
425       deletion  or, if the time of deletion is not reliably determinable, the
426       last known modification time.
427
428       The invalid field is set when the file is invalid and  unavailable  for
429       synchronization. A peer MAY set this bit to indicate that it can tempo‐
430       rarily not serve data for the file.
431
432       The no permissions field is set when there is no permission information
433       for  the  file.  This  is  the case when it originates on a file system
434       which does not support permissions. Changes  to  only  permission  bits
435       SHOULD  be disregarded on files with this bit set. The permissions bits
436       MUST be set to the octal value 0666.
437
438       The version field is a version vector describing the updates  performed
439       to  a  file  by all members in the cluster. Each counter in the version
440       vector is an ID-Value tuple. The ID is the first 64 bits of the  device
441       ID.  The  Value is a simple incrementing counter, starting at zero. The
442       combination of Folder, Name and Version uniquely  identifies  the  con‐
443       tents of a file at a given point in time.
444
445       The  sequence  field  is the value of a device local monotonic clock at
446       the time of last local database update to a file. The  clock  ticks  on
447       every  local database update, thus forming a sequence number over data‐
448       base updates.
449
450       The block_size field is the size, in bytes, of each individual block in
451       the  block  list  (except,  possibly, the last block). If this field is
452       missing or zero, the block size  is  assumed  to  be  128  KiB  (131072
453       bytes).  Valid  values of this field are the powers of two from 128 KiB
454       through 16 MiB. See also Selection of Block Size.
455
456       The blocks list contains the size and hash for each block in the  file.
457       Each  block represents a block_size-sized slice of the file, except for
458       the last block which may represent a smaller amount of data. The  block
459       list is empty for directories and symlinks.
460
461       The  symlink_target  field  contains the symlink target, for entries of
462       symlink type. It is empty for all other entry types.
463
464   Request
465       The Request message expresses the desire to receive a data block corre‐
466       sponding to a part of a certain file in the peer’s folder.
467
468   Protocol Buffer Schema
469          message Request {
470              int32  id             = 1;
471              string folder         = 2;
472              string name           = 3;
473              int64  offset         = 4;
474              int32  size           = 5;
475              bytes  hash           = 6;
476              bool   from_temporary = 7;
477          }
478
479   Fields
480       The id is the request identifier. It will be matched in the correspond‐
481       ing Response message. Each outstanding request must have a unique ID.
482
483       The folder and name fields are as documented  for  the  Index  message.
484       The  offset and size fields specify the region of the file to be trans‐
485       ferred. This SHOULD equate to exactly one block as  seen  in  an  Index
486       message.
487
488       The  hash  field MAY be set to the expected hash value of the block. If
489       set, the other device SHOULD ensure that the transmitted block  matches
490       the requested hash. The other device MAY reuse a block from a different
491       file and offset having the same size and hash, if one exists.
492
493       The from temporary field is set to indicate that  the  read  should  be
494       performed  from  the  temporary file (converting name to it’s temporary
495       form) and falling back to the non temporary file if any  error  occurs.
496       Knowledge  of  contents  of temporary files comes from DownloadProgress
497       messages.
498
499   Response
500       The Response message is sent in response to a Request message.
501
502   Protocol Buffer Schema
503          message Response {
504              int32     id   = 1;
505              bytes     data = 2;
506              ErrorCode code = 3;
507          }
508
509          enum ErrorCode {
510              NO_ERROR     = 0;
511              GENERIC      = 1;
512              NO_SUCH_FILE = 2;
513              INVALID_FILE = 3;
514          }
515
516   Fields
517       The id field is the request identifier. It must match the ID of the Re‐
518       quest that is being responded to.
519
520       The  data field contains either the requested data block or is empty if
521       the requested block is not available.
522
523       The code field contains an error code describing the reason  a  Request
524       could  not  be  fulfilled,  in  the case where zero length data was re‐
525       turned. The following values are defined:
526
527       0      No Error (data should be present)
528
529       1      Generic Error
530
531       2      No Such File (the requested file does not exist, or  the  offset
532              is outside the acceptable range for the file)
533
534       3      Invalid (file exists but has invalid bit set or is otherwise un‐
535              available)
536
537   DownloadProgress
538       The DownloadProgress message is used to  notify  remote  devices  about
539       partial  availability of files. By default, these messages are sent ev‐
540       ery 5 seconds, and only in the cases where progress  or  state  changes
541       have  been  detected.   Each DownloadProgress message is addressed to a
542       specific folder and MUST contain zero or  more  FileDownloadProgressUp‐
543       date messages.
544
545   Protocol Buffer Schema
546          message DownloadProgress {
547              string                              folder  = 1;
548              repeated FileDownloadProgressUpdate updates = 2;
549          }
550
551          message FileDownloadProgressUpdate {
552              FileDownloadProgressUpdateType update_type   = 1;
553              string                         name          = 2;
554              Vector                         version       = 3;
555              repeated int32                 block_indexes = 4;
556          }
557
558          enum FileDownloadProgressUpdateType {
559              APPEND = 0;
560              FORGET = 1;
561          }
562
563   Fields (DownloadProgress Message)
564       The  folder  field represents the ID of the folder for which the update
565       is being provided.
566
567       The updates field is a list of progress update messages.
568
569   Fields (FileDownloadProgressUpdate Message)
570       The update type indicates whether the update is of type append (0) (new
571       blocks are available) or forget (1) (the file transfer has completed or
572       failed).
573
574       The name field defines the file name from the global  index  for  which
575       this update is being sent.
576
577       The  version message defines the version of the file for which this up‐
578       date is being sent.
579
580       The block indexes field is a list of positive integers, where each  in‐
581       teger  represents the index of the block in the FileInfo message Blocks
582       array that has become available for download.
583
584       For example an integer with value 3 represents that the data defined in
585       the  fourth  BlockInfo  message of the FileInfo message of that file is
586       now available. Please note that matching should be  done  on  name  AND
587       version.  Furthermore, each update received is incremental, for example
588       the initial update message might contain indexes 0, 1, 2, an  update  5
589       seconds later might contain indexes 3, 4, 5 which should be appended to
590       the original list, which implies that blocks 0-5 are  currently  avail‐
591       able.
592
593       Block indexes MAY be added in any order. An implementation MUST NOT as‐
594       sume that block indexes are added in any specific order.
595
596       The forget field being set implies that previously advertised  file  is
597       no  longer  available,  therefore  the  list of block indexes should be
598       truncated.
599
600       Messages with the forget field set MUST NOT have any block indexes.
601
602       Any update message which is being sent for a different version  of  the
603       same file name must be preceded with an update message for the old ver‐
604       sion of that file with the forget field set.
605
606       As a safeguard on the receiving side, the value of version changing be‐
607       tween  update  messages  implies that the file has changed and that any
608       indexes previously advertised are no  longer  available.  The  list  of
609       available  block  indexes  MUST be replaced (rather than appended) with
610       the indexes specified in this message.
611
612   Ping
613       The Ping message is used to determine that a connection is  alive,  and
614       to  keep connections alive through state tracking network elements such
615       as firewalls and NAT gateways. A Ping message is sent every 90 seconds,
616       if no other message has been sent in the preceding 90 seconds.
617
618   Protocol Buffer Schema
619          message Ping {
620          }
621
622   Close
623       The  Close  message MAY be sent to indicate that the connection will be
624       torn down due to an error condition. A Close message MUST NOT  be  fol‐
625       lowed by further messages.
626
627   Protocol Buffer Schema
628          message Close {
629              string reason = 1;
630          }
631
632   Fields
633       The  reason  field  contains  a human readable description of the error
634       condition.
635

SHARING MODES

637   Trusted
638       Trusted mode is the default sharing mode. Updates are exchanged in both
639       directions.
640
641          +------------+     Updates      /---------\
642          |            |  ----------->   /           \
643          |   Device   |                 |  Cluster  |
644          |            |  <-----------   \           /
645          +------------+     Updates      \---------/
646
647   Send Only
648       In  send  only mode, a device does not apply any updates from the clus‐
649       ter, but publishes changes of its local folder to the cluster as usual.
650
651          +------------+     Updates      /---------\
652          |            |  ----------->   /           \
653          |   Device   |                 |  Cluster  |
654          |            |                 \           /
655          +------------+                  \---------/
656
657   Receive Only
658       In receive only mode, a device does not send any updates to  the  clus‐
659       ter, but accepts changes to its local folder from the cluster as usual.
660
661          +------------+     Updates      /---------\
662          |            |  <-----------   /           \
663          |   Device   |                 |  Cluster  |
664          |            |                 \           /
665          +------------+                  \---------/
666

DELTA INDEX EXCHANGE

668       Index  data  must be exchanged whenever two devices connect so that one
669       knows the files available on the other. In the  most  basic  case  this
670       happens  by way of sending an Index message followed by one or more In‐
671       dex Update messages. Any previous index data known for a remote  device
672       is  removed  and  replaced with the new index data received in an Index
673       message, while the contents of an Index Update message is simply  added
674       to the existing index data.
675
676       For  situations  with  large indexes or frequent reconnects this can be
677       quite inefficient. A mechanism can then be used to  retain  index  data
678       between  connections  and  only transmit any changes since that data on
679       connection start. This is called “delta indexes”. To enable this mecha‐
680       nism the sequence and index ID fields are used.
681
682       Sequence:
683              Each  index  item  (i.e.,  file, directory or symlink) has a se‐
684              quence number field. It contains the value of a counter  at  the
685              time  the  index item was updated. The counter increments by one
686              for each change. That is, as files are scanned and added to  the
687              index  they get assigned sequence numbers 1, 2, 3 and so on. The
688              next file to be changed or detected gets sequence number 4,  and
689              future updates continue in the same fashion.
690
691       Index ID:
692              Each  folder has an Index ID. This is a 64 bit random identifier
693              set at index creation time.
694
695       Given the above, we know that the tuple  {index  ID,  maximum  sequence
696       number}  uniquely identifies a point in time of a given index. Any fur‐
697       ther changes will increase the sequence number of some item,  and  thus
698       the  maximum  sequence number for the index itself. Should the index be
699       reset or removed (i.e., the sequence number reset to zero), a new index
700       ID must be generated.
701
702       By  letting  a  device  know the {index ID, maximum sequence number} we
703       have for their index data, that device can arrange to only transmit In‐
704       dex  Update  messages  for items with a higher sequence number. This is
705       the delta index mechanism.
706
707       The index ID and maximum sequence  number  known  for  each  device  is
708       transmitted in the Cluster Config message at connection start.
709
710       For  this  mechanism to be reliable it is essential that outgoing index
711       information is ordered by increasing sequence number. Devices  announc‐
712       ing a non-zero index ID in the Cluster Config message MUST send all in‐
713       dex data ordered by increasing sequence number. Devices  not  intending
714       to  participate  in  delta index exchange MUST send a zero index ID or,
715       equivalently, not send the index_id attribute at all.
716

MESSAGE LIMITS

718       An implementation MAY impose reasonable limits on the  length  of  mes‐
719       sages and message fields to aid robustness in the face of corruption or
720       broken implementations. An implementation should strive  to  keep  mes‐
721       sages  short and to the point, favouring more and smaller messages over
722       fewer and larger.  For example, favour a smaller Index message followed
723       by  one  or more Index Update messages rather than sending a very large
724       Index message.
725
726       The Syncthing implementation imposes a hard limit of 500,000,000  bytes
727       on  all  messages.  Attempting to send or receive a larger message will
728       result in a connection close. This size was chosen to accommodate Index
729       messages  containing  a  large block list. It’s intended that the limit
730       may be further reduced in a future protocol update supporting  variable
731       block sizes (and thus shorter block lists for large files).
732

SELECTION OF BLOCK SIZE

734       The  desired  block  size for any given file is the smallest block size
735       that results in fewer than 2000 blocks, or the maximum block  size  for
736       larger  files.  This rule results in the following table of block sizes
737       per file size:
738
739                          ┌──────────────────┬────────────┐
740                          │File Size         │ Block Size │
741                          └──────────────────┴────────────┘
742
743                          │0 - 250 MiB       │ 128 KiB    │
744                          ├──────────────────┼────────────┤
745                          │250 MiB - 500 MiB │ 256 KiB    │
746                          ├──────────────────┼────────────┤
747                          │500 MiB - 1 GiB   │ 512 KiB    │
748                          ├──────────────────┼────────────┤
749                          │1 GiB - 2 GiB     │ 1 MiB      │
750                          ├──────────────────┼────────────┤
751                          │2 GiB - 4 GiB     │ 2 MiB      │
752                          ├──────────────────┼────────────┤
753                          │4 GiB - 8 GiB     │ 4 MiB      │
754                          ├──────────────────┼────────────┤
755                          │8 GiB - 16 GiB    │ 8 MiB      │
756                          ├──────────────────┼────────────┤
757                          │16 GiB - up       │ 16 MiB     │
758                          └──────────────────┴────────────┘
759
760       An implementation MAY deviate from the block size rule  when  there  is
761       good reason to do so. For example, if a file has been indexed at a cer‐
762       tain block size and grows beyond 2000 blocks it may be retained at  the
763       current  block  size for practical reasons. When there is no overriding
764       reason to the contrary, such as when indexing a new file for the  first
765       time, the block size rule above SHOULD be followed.
766
767       An implementation MUST therefore accept files with a block size differ‐
768       ing from the above rule. This does not mean that arbitrary block  sizes
769       are   allowed.  The  block  size  used  MUST  be  exactly  one  of  the
770       power-of-two block sizes listed in the table above.
771

EXAMPLE EXCHANGE

773                   ┌───┬───────────────────┬─────────────────────┐
774                   │#  │ A                 │ B                   │
775                   ├───┼───────────────────┼─────────────────────┤
776                   │1  │ ClusterConfigura‐ │ <-ClusterConfigura‐ │
777                   │   │ tion->            │ tion                │
778                   ├───┼───────────────────┼─────────────────────┤
779                   │2  │ Index->           │ <-Index             │
780                   ├───┼───────────────────┼─────────────────────┤
781                   │3  │ IndexUpdate->     │ <-IndexUpdate       │
782                   ├───┼───────────────────┼─────────────────────┤
783                   │4  │ IndexUpdate->     │                     │
784                   ├───┼───────────────────┼─────────────────────┤
785                   │5  │ Request->         │                     │
786                   ├───┼───────────────────┼─────────────────────┤
787                   │6  │ Request->         │                     │
788                   ├───┼───────────────────┼─────────────────────┤
789                   │7  │ Request->         │                     │
790                   ├───┼───────────────────┼─────────────────────┤
791                   │8  │ Request->         │                     │
792                   ├───┼───────────────────┼─────────────────────┤
793                   │9  │                   │ <-Response          │
794                   ├───┼───────────────────┼─────────────────────┤
795                   │10 │                   │ <-Response          │
796                   ├───┼───────────────────┼─────────────────────┤
797                   │11 │                   │ <-Response          │
798                   ├───┼───────────────────┼─────────────────────┤
799                   │12 │                   │ <-Response          │
800                   ├───┼───────────────────┼─────────────────────┤
801                   │13 │ Index Update->    │                     │
802                   ├───┼───────────────────┼─────────────────────┤
803                   │…  │                   │                     │
804                   ├───┼───────────────────┼─────────────────────┤
805                   │14 │                   │ <-Ping              │
806                   ├───┼───────────────────┼─────────────────────┤
807                   │15 │ Ping->            │                     │
808                   └───┴───────────────────┴─────────────────────┘
809
810       The connection is established and at 1. both peers send ClusterConfigu‐
811       ration  messages and then Index records. The Index records are received
812       and both peers recompute their knowledge of the data in the cluster. In
813       this  example, peer A has four missing or outdated blocks. At 5 through
814       8 peer A sends requests for these blocks. The requests are received  by
815       peer  B,  who retrieves the data from the folder and transmits Response
816       records (9 through 12). Device A  updates  their  folder  contents  and
817       transmits an Index Update message (13). Both peers enter idle state af‐
818       ter 13. At some later time 14, the ping timer on device B expires and a
819       Ping message is sent. The same process occurs for device A at 15.
820

EXAMPLES OF STRONG CIPHER SUITES

822            ┌───────┬─────────────────────────────┬─────────────────────┐
823            │ID     │ Name                        │ Description         │
824            ├───────┼─────────────────────────────┼─────────────────────┤
825            │0x009F │ DHE-RSA-AES256-GCM-SHA384   │ TLSv1.2 DH RSA AES‐ │
826            │       │                             │ GCM(256) AEAD       │
827            ├───────┼─────────────────────────────┼─────────────────────┤
828            │0x006B │ DHE-RSA-AES256-SHA256       │ TLSv1.2    DH   RSA │
829            │       │                             │ AES(256) SHA256     │
830            ├───────┼─────────────────────────────┼─────────────────────┤
831            │0xC030 │ ECDHE-RSA-AES256-GCM-SHA384 │ TLSv1.2   ECDH  RSA │
832            │       │                             │ AESGCM(256) AEAD    │
833            ├───────┼─────────────────────────────┼─────────────────────┤
834            │0xC028 │ ECDHE-RSA-AES256-SHA384     │ TLSv1.2  ECDH   RSA │
835            │       │                             │ AES(256) SHA384     │
836            ├───────┼─────────────────────────────┼─────────────────────┤
837            │0x009E │ DHE-RSA-AES128-GCM-SHA256   │ TLSv1.2 DH RSA AES‐ │
838            │       │                             │ GCM(128) AEAD       │
839            ├───────┼─────────────────────────────┼─────────────────────┤
840            │0x0067 │ DHE-RSA-AES128-SHA256       │ TLSv1.2   DH    RSA │
841            │       │                             │ AES(128) SHA256     │
842            ├───────┼─────────────────────────────┼─────────────────────┤
843            │0xC02F │ ECDHE-RSA-AES128-GCM-SHA256 │ TLSv1.2   ECDH  RSA │
844            │       │                             │ AESGCM(128) AEAD    │
845            ├───────┼─────────────────────────────┼─────────────────────┤
846            │0xC027 │ ECDHE-RSA-AES128-SHA256     │ TLSv1.2  ECDH   RSA │
847            │       │                             │ AES(128) SHA256     │
848            └───────┴─────────────────────────────┴─────────────────────┘
849

AUTHOR

851       The Syncthing Authors
852
854       2014-2019, The Syncthing Authors
855
856
857
858
859v1.22.2                          Dec 29, 2022                 SYNCTHING-BEP(7)
Impressum