1SYNCTHING-BEP(7)                   Syncthing                  SYNCTHING-BEP(7)
2
3
4

NAME

6       syncthing-bep - Block Exchange Protocol v1
7

INTRODUCTION AND DEFINITIONS

9       The  Block  Exchange Protocol (BEP) is used between two or more devices
10       thus forming a cluster. Each device has one or more  folders  of  files
11       described by the local model, containing metadata and block hashes. The
12       local model is sent to the other devices in the cluster. The  union  of
13       all  files  in the local models, with files selected for highest change
14       version, forms the global model. Each device strives to get its folders
15       in  sync with the global model by requesting missing or outdated blocks
16       from the other devices in the cluster.
17
18       File data is described and transferred in units of blocks,  each  being
19       from  128  KiB  (131072 bytes) to 16 MiB in size, in steps of powers of
20       two. The block size may vary between files but is constant in any given
21       file, except for the last block which may be smaller.
22
23       The  key  words  “MUST”,  “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”,
24       “SHOULD”, “SHOULD NOT”, “RECOMMENDED”, “MAY”, and  “OPTIONAL”  in  this
25       document   are   to   be  interpreted  as  described  in  [RFC  2119](‐
26       https://tools.ietf.org/html/rfc2119).
27

TRANSPORT AND AUTHENTICATION

29       BEP is deployed as the highest level in  a  protocol  stack,  with  the
30       lower level protocols providing encryption and authentication.
31
32          +-----------------------------+
33          |   Block Exchange Protocol   |
34          |-----------------------------|
35          | Encryption & Auth (TLS 1.2) |
36          |-----------------------------|
37          |      Reliable Transport     |
38          |-----------------------------|
39          v             ...             v
40
41       The  encryption  and authentication layer SHALL use TLS 1.2 or a higher
42       revision. A strong cipher suite SHALL  be  used,  with  “strong  cipher
43       suite”  being  defined  as being without known weaknesses and providing
44       Perfect Forward Secrecy (PFS). Examples of  strong  cipher  suites  are
45       given  at  the  end  of  this  document.  This is not to be taken as an
46       exhaustive list of allowed cipher suites but represents best  practices
47       at the time of writing.
48
49       The  exact  nature of the authentication is up to the application, how‐
50       ever it SHALL be based on the TLS certificate presented at the start of
51       the  connection.  Possibilities include certificates signed by a common
52       trusted CA, preshared certificates, preshared certificate  fingerprints
53       or  certificate  pinning combined with some out of band first verifica‐
54       tion. The reference implementation uses preshared  certificate  finger‐
55       prints (SHA-256) referred to as “Device IDs”.
56
57       There is no required order or synchronization among BEP messages except
58       as noted per message type - any message type may be sent  at  any  time
59       and  the sender need not await a response to one message before sending
60       another.
61
62       The underlying transport protocol MUST guarantee reliable packet deliv‐
63       ery.
64
65       In this document, in diagrams and text, “bit 0” refers to the most sig‐
66       nificant bit of a word; “bit 15” is thus the least significant bit of a
67       16  bit  word (int16) and “bit 31” is the least significant bit of a 32
68       bit word (int32). Non protocol buffer integers are  always  represented
69       in  network  byte order (i.e., big endian) and are signed unless stated
70       otherwise, but when describing message lengths negative values  do  not
71       make sense and the most significant bit MUST be zero.
72
73       The protocol buffer schemas in this document are in proto3 syntax. This
74       means, among other things, that all fields are optional and will assume
75       their default value when missing. This does not necessarily mean that a
76       message is valid with all fields empty - for example,  an  index  entry
77       for  a file that does not have a name is not useful and MAY be rejected
78       by the implementation. However the folder label is for  human  consump‐
79       tion  only  so  an  empty label should be accepted - the implementation
80       will have to choose some way to represent the folder, perhaps by  using
81       the ID in it’s place or automatically generating a label.
82

PRE-AUTHENTICATION MESSAGES

84       AFTER  establishing a connection, but BEFORE performing any authentica‐
85       tion, devices MUST exchange Hello messages.
86
87       Hello messages are used to carry additional information about the peer,
88       which  might be of interest to the user even if the peer is not permit‐
89       ted to communicate due to failing authentication. Note  that  the  cer‐
90       tificate  based  authentication may be considered part of the TLS hand‐
91       shake that precedes the Hello message exchange, but even  in  the  case
92       that  a  connection is rejected a Hello message must be sent before the
93       connection is terminated.
94
95       Hello messages MUST be prefixed with an int32 containing the magic num‐
96       ber  0x2EA7D90B, followed by an int16 representing the size of the mes‐
97       sage, followed by the contents of the Hello message itself.
98
99           0                   1
100           0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5
101          +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
102          |             Magic             |
103          |           (32 bits)           |
104          +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
105          |             Length            |
106          +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
107          /                               /
108          \             Hello             \
109          /                               /
110          +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
111
112       The Hello message itself is in protocol buffer format with the  follow‐
113       ing schema:
114
115          message Hello {
116              string device_name    = 1;
117              string client_name    = 2;
118              string client_version = 3;
119          }
120
121   Fields (Hello message)
122       The  device_name  is  a  human  readable  (configured or auto detected)
123       device name or host name, for the remote device.
124
125       The client_name and client_version identifies the  implementation.  The
126       values  SHOULD   be simple strings identifying the implementation name,
127       as a user would expect to see it, and the version string  in  the  same
128       manner.  An  example  client  name is “syncthing” and an example client
129       version is “v0.7.2”.  The client version field SHOULD follow  the  pat‐
130       terns  laid  out  in the Semantic Versioning <http://semver.org/> stan‐
131       dard.
132
133       Immediately after exchanging Hello messages,  the  connection  MUST  be
134       dropped if the remote device does not pass authentication.
135

POST-AUTHENTICATION MESSAGES

137       Every message post authentication is made up of several parts:
138
139       · A header length word
140
141       · A Header
142
143       · A message length word
144
145       · A Message
146
147           0                   1
148           0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5
149          +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
150          |         Header Length         |
151          +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
152          /                               /
153          \            Header             \
154          /                               /
155          +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
156          |         Message Length        |
157          |           (32 bits)           |
158          +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
159          /                               /
160          \            Message            \
161          /                               /
162          +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
163
164       The  header length word is 16 bits. It indicates the length of the fol‐
165       lowing Header message. The Header is in  protocol  buffer  format.  The
166       Header  describes the type and compression status of the following mes‐
167       sage.
168
169       The message is preceded by the 32 bit message length word and is one of
170       the concrete BEP messages described below, identified by the type field
171       of the Header.
172
173       As always, the length words are in network byte order (big endian).
174
175          message Header {
176              MessageType        type        = 1;
177              MessageCompression compression = 2;
178          }
179
180          enum MessageType {
181              CLUSTER_CONFIG    = 0;
182              INDEX             = 1;
183              INDEX_UPDATE      = 2;
184              REQUEST           = 3;
185              RESPONSE          = 4;
186              DOWNLOAD_PROGRESS = 5;
187              PING              = 6;
188              CLOSE             = 7;
189          }
190
191          enum MessageCompression {
192              NONE = 0;
193              LZ4  = 1;
194          }
195
196       When the compression field is NONE, the message is directly in protocol
197       buffer format.
198
199       When  the  compression  field  is LZ4, the message consists of a 32 bit
200       integer describing the uncompressed message length followed by a single
201       LZ4  block.  After decompressing the LZ4 block it should be interpreted
202       as a protocol buffer message just as in the uncompressed case.
203

MESSAGE SUBTYPES

205   Cluster Config
206       This informational message provides information about the cluster  con‐
207       figuration  as  it pertains to the current connection. A Cluster Config
208       message MUST be the first post authentication message  sent  on  a  BEP
209       connection.  Additional  Cluster Config messages MUST NOT be sent after
210       the initial exchange.
211
212   Protocol Buffer Schema
213          message ClusterConfig {
214              repeated Folder folders = 1;
215          }
216
217          message Folder {
218              string id                   = 1;
219              string label                = 2;
220              bool   read_only            = 3;
221              bool   ignore_permissions   = 4;
222              bool   ignore_delete        = 5;
223              bool   disable_temp_indexes = 6;
224              bool   paused               = 7;
225
226              repeated Device devices = 16;
227          }
228
229          message Device {
230              bytes           id                         = 1;
231              string          name                       = 2;
232              repeated string addresses                  = 3;
233              Compression     compression                = 4;
234              string          cert_name                  = 5;
235              int64           max_sequence               = 6;
236              bool            introducer                 = 7;
237              uint64          index_id                   = 8;
238              bool            skip_introduction_removals = 9;
239          }
240
241          enum Compression {
242              METADATA = 0;
243              NEVER    = 1;
244              ALWAYS   = 2;
245          }
246
247   Fields (Cluster Config Message)
248       The folders field contains the list of folders that  will  be  synchro‐
249       nized over the current connection.
250
251   Fields (Folder Message)
252       The  id field contains the folder ID, which is the unique identifier of
253       the folder.
254
255       The label field contains the folder label, the human readable  name  of
256       the folder.
257
258       The  read  only field is set for folders that the device will accept no
259       updates from the network for.
260
261       The ignore permissions field is set for folders that  the  device  will
262       not accept or announce file permissions for.
263
264       The  ignore delete field is set for folders that the device will ignore
265       deletes for.
266
267       The disable temp indexes field is set for folders that  will  not  dis‐
268       patch and do not wish to receive progress updates about partially down‐
269       loaded files via Download Progress messages.
270
271       The paused field is set for folders that are currently paused.
272
273       The devices field is a list of devices participating  in  sharing  this
274       folder.
275
276   Fields (Device Message)
277       The  device  id  field is a 32 byte number that uniquely identifies the
278       device. For instance, the reference implementation uses the SHA-256  of
279       the device X.509 certificate.
280
281       The  name  field  is  a  human  readable name assigned to the described
282       device by the sending device. It MAY  be  empty  and  it  need  not  be
283       unique.
284
285       The  list of addresses is that used by the sending device to connect to
286       the described device.
287
288       The compression field indicates the compression mode in  use  for  this
289       device and folder. The following values are valid:
290
291       0      Compress metadata. This enables compression of metadata messages
292              such as Index.
293
294       1      Compression disabled. No compression is used on any message.
295
296       2      Compress always. Metadata messages as well as Response  messages
297              are compressed.
298
299       The  cert  name  field indicates the expected certificate name for this
300       device. It is commonly blank,  indicating  to  use  the  implementation
301       default.
302
303       The  max  sequence  field  contains  the highest sequence number of the
304       files in the index. See Delta Index Exchange  for  the  usage  of  this
305       field.
306
307       The  introducer  field  is  set for devices that are trusted as cluster
308       introducers.
309
310       The index id field contains the unique identifier for the  current  set
311       of index data. See Delta Index Exchange for the usage of this field.
312
313       The skip introduction removals field signifies if the remote device has
314       opted to ignore introduction removals for the given device.  This  set‐
315       ting is copied across as we are being introduced to a new device.
316
317   Index and Index Update
318       The  Index and Index Update messages define the contents of the senders
319       folder. An Index message represents the full contents of the folder and
320       thus  supersedes any previous index. An Index Update amends an existing
321       index with new information, not affecting any entries not  included  in
322       the  message.  An  Index  Update  MAY NOT be sent unless preceded by an
323       Index, unless a non-zero Max Sequence has been announced for the  given
324       folder by the peer device.
325
326       The  Index and Index Update messages are currently identical in format,
327       although this is not guaranteed to be the case in the future.
328
329   Protocol Buffer Schema
330          message Index {
331              string            folder = 1;
332              repeated FileInfo files  = 2;
333          }
334
335          message IndexUpdate {
336              string            folder = 1;
337              repeated FileInfo files  = 2;
338          }
339
340          message FileInfo {
341              string       name           = 1;
342              FileInfoType type           = 2;
343              int64        size           = 3;
344              uint32       permissions    = 4;
345              int64        modified_s     = 5;
346              int32        modified_ns    = 11;
347              uint64       modified_by    = 12;
348              bool         deleted        = 6;
349              bool         invalid        = 7;
350              bool         no_permissions = 8;
351              Vector       version        = 9;
352              int64        sequence       = 10;
353              int32        block_size     = 13;
354
355              repeated BlockInfo Blocks         = 16;
356              string             symlink_target = 17;
357          }
358
359          enum FileInfoType {
360              FILE              = 0;
361              DIRECTORY         = 1;
362              SYMLINK_FILE      = 2 [deprecated = true];
363              SYMLINK_DIRECTORY = 3 [deprecated = true];
364              SYMLINK           = 4;
365          }
366
367          message BlockInfo {
368              int64 offset     = 1;
369              int32 size       = 2;
370              bytes hash       = 3;
371              uint32 weak_hash = 4;
372          }
373
374          message Vector {
375              repeated Counter counters = 1;
376          }
377
378          message Counter {
379              uint64 id    = 1;
380              uint64 value = 2;
381          }
382
383   Fields (Index Message)
384       The folder field identifies the folder that the index message  pertains
385       to.
386
387       The files field is a list of files making up the index information.
388
389   Fields (FileInfo Message)
390       The  name  is  the file name path relative to the folder root. Like all
391       strings in BEP, the Name is always in UTF-8 NFC regardless of operating
392       system  or  file  system  specific conventions. The name field uses the
393       slash character (“/”) as path separator, regardless of the  implementa‐
394       tion’s operating system conventions. The combination of folder and name
395       uniquely identifies each file in a cluster.
396
397       The type field contains the type of the described item. The type is one
398       of file (0), directory (1), or symlink (4).
399
400       The size field contains the size of the file, in bytes. For directories
401       and symlinks the size is zero.
402
403       The permissions field holds the common Unix permission bits. An  imple‐
404       mentation  MAY  ignore  or  interpret  these as is suitable on the host
405       operating system.
406
407       The modified_s time is expressed as the number  of  seconds  since  the
408       Unix  Epoch  (1970-01-01 00:00:00 UTC). The modified_ns field holds the
409       nanosecond part of the modification time.
410
411       The modified_by field holds the short id of the client that  last  made
412       any  modification to the file whether add, change or delete.  This will
413       be overwritten every time a change is made to  the  file  by  the  last
414       client to do so and so does not hold history.
415
416       The deleted field is set when the file has been deleted. The block list
417       SHALL be of length zero and the modification time indicates the time of
418       deletion  or, if the time of deletion is not reliably determinable, the
419       last known modification time.
420
421       The invalid field is set when the file is invalid and  unavailable  for
422       synchronization. A peer MAY set this bit to indicate that it can tempo‐
423       rarily not serve data for the file.
424
425       The no permissions field is set when there is no permission information
426       for  the  file.  This  is  the case when it originates on a file system
427       which does not support permissions. Changes  to  only  permission  bits
428       SHOULD  be disregarded on files with this bit set. The permissions bits
429       MUST be set to the octal value 0666.
430
431       The version field is a version vector describing the updates  performed
432       to  a  file  by all members in the cluster. Each counter in the version
433       vector is an ID-Value tuple. The ID is the first 64 bits of the  device
434       ID.  The  Value is a simple incrementing counter, starting at zero. The
435       combination of Folder, Name and Version uniquely  identifies  the  con‐
436       tents of a file at a given point in time.
437
438       The  sequence  field  is the value of a device local monotonic clock at
439       the time of last local database update to a file. The  clock  ticks  on
440       every  local database update, thus forming a sequence number over data‐
441       base updates.
442
443       The block_size field is the size, in bytes, of each individual block in
444       the  block  list  (except,  possibly, the last block). If this field is
445       missing or zero, the block size  is  assumed  to  be  128  KiB  (131072
446       bytes).  Valid  values of this field are the powers of two from 128 KiB
447       through 16 MiB. See also Selection of Block Size.
448
449       The blocks list contains the size and hash for each block in the  file.
450       Each  block represents a block_size-sized slice of the file, except for
451       the last block which may represent a smaller amount of data. The  block
452       list is empty for directories and symlinks.
453
454       The  symlink_target  field  contains the symlink target, for entries of
455       symlink type. It is empty for all other entry types.
456
457   Request
458       The Request message expresses the desire to receive a data block corre‐
459       sponding to a part of a certain file in the peer’s folder.
460
461   Protocol Buffer Schema
462          message Request {
463              int32  id             = 1;
464              string folder         = 2;
465              string name           = 3;
466              int64  offset         = 4;
467              int32  size           = 5;
468              bytes  hash           = 6;
469              bool   from_temporary = 7;
470          }
471
472   Fields
473       The id is the request identifier. It will be matched in the correspond‐
474       ing Response message. Each outstanding request must have a unique ID.
475
476       The folder and name fields are as documented  for  the  Index  message.
477       The  offset and size fields specify the region of the file to be trans‐
478       ferred. This SHOULD equate to exactly one block as  seen  in  an  Index
479       message.
480
481       The  hash  field MAY be set to the expected hash value of the block. If
482       set, the other device SHOULD ensure that the transmitted block  matches
483       the requested hash. The other device MAY reuse a block from a different
484       file and offset having the same size and hash, if one exists.
485
486       The from temporary field is set to indicate that  the  read  should  be
487       performed  from  the  temporary file (converting name to it’s temporary
488       form) and falling back to the non temporary file if any  error  occurs.
489       Knowledge  of  contents  of temporary files comes from DownloadProgress
490       messages.
491
492   Response
493       The Response message is sent in response to a Request message.
494
495   Protocol Buffer Schema
496          message Response {
497              int32     id   = 1;
498              bytes     data = 2;
499              ErrorCode code = 3;
500          }
501
502          enum ErrorCode {
503              NO_ERROR     = 0;
504              GENERIC      = 1;
505              NO_SUCH_FILE = 2;
506              INVALID_FILE = 3;
507          }
508
509   Fields
510       The id field is the request identifier. It must match  the  ID  of  the
511       Request that is being responded to.
512
513       The  data field contains either the requested data block or is empty if
514       the requested block is not available.
515
516       The code field contains an error code describing the reason  a  Request
517       could  not  be  fulfilled,  in  the  case  where  zero  length data was
518       returned. The following values are defined:
519
520       0      No Error (data should be present)
521
522       1      Generic Error
523
524       2      No Such File (the requested file does not exist, or  the  offset
525              is outside the acceptable range for the file)
526
527       3      Invalid  (file  exists  but  has invalid bit set or is otherwise
528              unavailable)
529
530   DownloadProgress
531       The DownloadProgress message is used to  notify  remote  devices  about
532       partial  availability  of  files.  By  default, these messages are sent
533       every 5 seconds, and only in the cases where progress or state  changes
534       have  been  detected.   Each DownloadProgress message is addressed to a
535       specific folder and MUST contain zero or  more  FileDownloadProgressUp‐
536       date messages.
537
538   Protocol Buffer Schema
539          message DownloadProgress {
540              string                              folder  = 1;
541              repeated FileDownloadProgressUpdate updates = 2;
542          }
543
544          message FileDownloadProgressUpdate {
545              FileDownloadProgressUpdateType update_type   = 1;
546              string                         name          = 2;
547              Vector                         version       = 3;
548              repeated int32                 block_indexes = 4;
549          }
550
551          enum FileDownloadProgressUpdateType {
552              APPEND = 0;
553              FORGET = 1;
554          }
555
556   Fields (DownloadProgress Message)
557       The  folder  field represents the ID of the folder for which the update
558       is being provided.
559
560       The updates field is a list of progress update messages.
561
562   Fields (FileDownloadProgressUpdate Message)
563       The update type indicates whether the update is of type append (0) (new
564       blocks are available) or forget (1) (the file transfer has completed or
565       failed).
566
567       The name field defines the file name from the global  index  for  which
568       this update is being sent.
569
570       The  version  message  defines  the  version of the file for which this
571       update is being sent.
572
573       The block indexes field is a list  of  positive  integers,  where  each
574       integer  represents  the  index  of  the  block in the FileInfo message
575       Blocks array that has become available for download.
576
577       For example an integer with value 3 represents that the data defined in
578       the  fourth  BlockInfo  message of the FileInfo message of that file is
579       now available. Please note that matching should be  done  on  name  AND
580       version.  Furthermore, each update received is incremental, for example
581       the initial update message might contain indexes 0, 1, 2, an  update  5
582       seconds later might contain indexes 3, 4, 5 which should be appended to
583       the original list, which implies that blocks 0-5 are  currently  avail‐
584       able.
585
586       Block  indexes  MAY  be  added in any order. An implementation MUST NOT
587       assume that block indexes are added in any specific order.
588
589       The forget field being set implies that previously advertised  file  is
590       no  longer  available,  therefore  the  list of block indexes should be
591       truncated.
592
593       Messages with the forget field set MUST NOT have any block indexes.
594
595       Any update message which is being sent for a different version  of  the
596       same file name must be preceded with an update message for the old ver‐
597       sion of that file with the forget field set.
598
599       As a safeguard on the receiving side, the  value  of  version  changing
600       between  update messages implies that the file has changed and that any
601       indexes previously advertised are no  longer  available.  The  list  of
602       available  block  indexes  MUST be replaced (rather than appended) with
603       the indexes specified in this message.
604
605   Ping
606       The Ping message is used to determine that a connection is  alive,  and
607       to  keep connections alive through state tracking network elements such
608       as firewalls and NAT gateways. A Ping message is sent every 90 seconds,
609       if no other message has been sent in the preceding 90 seconds.
610
611   Protocol Buffer Schema
612          message Ping {
613          }
614
615   Close
616       The  Close  message MAY be sent to indicate that the connection will be
617       torn down due to an error condition. A Close message MUST NOT  be  fol‐
618       lowed by further messages.
619
620   Protocol Buffer Schema
621          message Close {
622              string reason = 1;
623          }
624
625   Fields
626       The  reason  field  contains  a human readable description of the error
627       condition.
628

SHARING MODES

630   Trusted
631       Trusted mode is the default sharing mode. Updates are exchanged in both
632       directions.
633
634          +------------+     Updates      /---------\
635          |            |  ----------->   /           \
636          |   Device   |                 |  Cluster  |
637          |            |  <-----------   \           /
638          +------------+     Updates      \---------/
639
640   Send Only
641       In  send  only mode, a device does not apply any updates from the clus‐
642       ter, but publishes changes of its local folder to the cluster as usual.
643
644          +------------+     Updates      /---------\
645          |            |  ----------->   /           \
646          |   Device   |                 |  Cluster  |
647          |            |                 \           /
648          +------------+                  \---------/
649
650   Receive Only
651       In receive only mode, a device does not send any updates to  the  clus‐
652       ter, but accepts changes to its local folder from the cluster as usual.
653
654          +------------+     Updates      /---------\
655          |            |  <-----------   /           \
656          |   Device   |                 |  Cluster  |
657          |            |                 \           /
658          +------------+                  \---------/
659

DELTA INDEX EXCHANGE

661       Index  data  must be exchanged whenever two devices connect so that one
662       knows the files available on the other. In the  most  basic  case  this
663       happens  by  way  of  sending  an Index message followed by one or more
664       Index Update messages. Any previous  index  data  known  for  a  remote
665       device  is  removed and replaced with the new index data received in an
666       Index message, while the contents of an Index Update message is  simply
667       added to the existing index data.
668
669       For  situations  with  large indexes or frequent reconnects this can be
670       quite inefficient. A mechanism can then be used to  retain  index  data
671       between  connections  and  only transmit any changes since that data on
672       connection start. This is called “delta indexes”. To enable this mecha‐
673       nism the sequence and index ID fields are used.
674
675       Sequence:
676              Each  index  item  (i.e.,  file,  directory  or  symlink)  has a
677              sequence number field. It contains the value of a counter at the
678              time  the  index item was updated. The counter increments by one
679              for each change. That is, as files are scanned and added to  the
680              index  they get assigned sequence numbers 1, 2, 3 and so on. The
681              next file to be changed or detected gets sequence number 4,  and
682              future updates continue in the same fashion.
683
684       Index ID:
685              Each  folder has an Index ID. This is a 64 bit random identifier
686              set at index creation time.
687
688       Given the above, we know that the tuple  {index  ID,  maximum  sequence
689       number}  uniquely identifies a point in time of a given index. Any fur‐
690       ther changes will increase the sequence number of some item,  and  thus
691       the  maximum  sequence number for the index itself. Should the index be
692       reset or removed (i.e., the sequence number reset to zero), a new index
693       ID must be generated.
694
695       By  letting  a  device  know the {index ID, maximum sequence number} we
696       have for their index data, that device can  arrange  to  only  transmit
697       Index  Update messages for items with a higher sequence number. This is
698       the delta index mechanism.
699
700       The index ID and maximum sequence  number  known  for  each  device  is
701       transmitted in the Cluster Config message at connection start.
702
703       For  this  mechanism to be reliable it is essential that outgoing index
704       information is ordered by increasing sequence number. Devices  announc‐
705       ing  a  non-zero  index  ID in the Cluster Config message MUST send all
706       index data ordered by increasing sequence number. Devices not intending
707       to  participate  in  delta index exchange MUST send a zero index ID or,
708       equivalently, not send the index_id attribute at all.
709

MESSAGE LIMITS

711       An implementation MAY impose reasonable limits on the  length  of  mes‐
712       sages and message fields to aid robustness in the face of corruption or
713       broken implementations. An implementation should strive  to  keep  mes‐
714       sages  short and to the point, favouring more and smaller messages over
715       fewer and larger.  For example, favour a smaller Index message followed
716       by  one  or more Index Update messages rather than sending a very large
717       Index message.
718
719       The Syncthing implementation imposes a hard limit of 500,000,000  bytes
720       on  all  messages.  Attempting to send or receive a larger message will
721       result in a connection close. This size was chosen to accommodate Index
722       messages  containing  a  large block list. It’s intended that the limit
723       may be further reduced in a future protocol update supporting  variable
724       block sizes (and thus shorter block lists for large files).
725

SELECTION OF BLOCK SIZE

727       The  desired  block  size for any given file is the smallest block size
728       that results in fewer than 2000 blocks, or the maximum block  size  for
729       larger  files.  This rule results in the following table of block sizes
730       per file size:
731
732
733
734                          ┌──────────────────┬────────────┐
735                          │File Size         │ Block Size │
736                          ├──────────────────┼────────────┤
737                          │0 - 250 MiB       │ 128 KiB    │
738                          ├──────────────────┼────────────┤
739                          │250 MiB - 500 MiB │ 256 KiB    │
740                          ├──────────────────┼────────────┤
741                          │500 MiB - 1 GiB   │ 512 KiB    │
742                          ├──────────────────┼────────────┤
743                          │1 GiB - 2 GiB     │ 1 MiB      │
744                          ├──────────────────┼────────────┤
745                          │2 GiB - 4 GiB     │ 2 MiB      │
746                          ├──────────────────┼────────────┤
747                          │4 GiB - 8 GiB     │ 4 MiB      │
748                          ├──────────────────┼────────────┤
749                          │8 GiB - 16 GiB    │ 8 MiB      │
750                          ├──────────────────┼────────────┤
751                          │16 GiB - up       │ 16 MiB     │
752                          └──────────────────┴────────────┘
753
754       An implementation MAY deviate from the block size rule  when  there  is
755       good reason to do so. For example, if a file has been indexed at a cer‐
756       tain block size and grows beyond 2000 blocks it may be retained at  the
757       current  block  size for practical reasons. When there is no overriding
758       reason to the contrary, such as when indexing a new file for the  first
759       time, the block size rule above SHOULD be followed.
760
761       An implementation MUST therefore accept files with a block size differ‐
762       ing from the above rule. This does not mean that arbitrary block  sizes
763       are   allowed.  The  block  size  used  MUST  be  exactly  one  of  the
764       power-of-two block sizes listed in the table above.
765

EXAMPLE EXCHANGE

767                   ┌───┬───────────────────┬─────────────────────┐
768                   │#  │ A                 │ B                   │
769                   ├───┼───────────────────┼─────────────────────┤
770                   │1  │ ClusterConfigura‐ │ <-ClusterConfigura‐ │
771                   │   │ tion->            │ tion                │
772                   ├───┼───────────────────┼─────────────────────┤
773                   │2  │ Index->           │ <-Index             │
774                   ├───┼───────────────────┼─────────────────────┤
775                   │3  │ IndexUpdate->     │ <-IndexUpdate       │
776                   ├───┼───────────────────┼─────────────────────┤
777                   │4  │ IndexUpdate->     │                     │
778                   ├───┼───────────────────┼─────────────────────┤
779                   │5  │ Request->         │                     │
780                   ├───┼───────────────────┼─────────────────────┤
781                   │6  │ Request->         │                     │
782                   ├───┼───────────────────┼─────────────────────┤
783                   │7  │ Request->         │                     │
784                   ├───┼───────────────────┼─────────────────────┤
785                   │8  │ Request->         │                     │
786                   ├───┼───────────────────┼─────────────────────┤
787                   │9  │                   │ <-Response          │
788                   ├───┼───────────────────┼─────────────────────┤
789                   │10 │                   │ <-Response          │
790                   ├───┼───────────────────┼─────────────────────┤
791                   │11 │                   │ <-Response          │
792                   ├───┼───────────────────┼─────────────────────┤
793                   │12 │                   │ <-Response          │
794                   ├───┼───────────────────┼─────────────────────┤
795                   │13 │ Index Update->    │                     │
796                   ├───┼───────────────────┼─────────────────────┤
797                   │…  │                   │                     │
798                   └───┴───────────────────┴─────────────────────┘
799
800
801                   │14 │                   │ <-Ping              │
802                   ├───┼───────────────────┼─────────────────────┤
803                   │15 │ Ping->            │                     │
804                   └───┴───────────────────┴─────────────────────┘
805
806       The connection is established and at 1. both peers send ClusterConfigu‐
807       ration  messages and then Index records. The Index records are received
808       and both peers recompute their knowledge of the data in the cluster. In
809       this  example, peer A has four missing or outdated blocks. At 5 through
810       8 peer A sends requests for these blocks. The requests are received  by
811       peer  B,  who retrieves the data from the folder and transmits Response
812       records (9 through 12). Device A  updates  their  folder  contents  and
813       transmits  an  Index  Update  message (13). Both peers enter idle state
814       after 13. At some later time 14, the ping timer on device B expires and
815       a Ping message is sent. The same process occurs for device A at 15.
816

EXAMPLES OF STRONG CIPHER SUITES

818            ┌───────┬─────────────────────────────┬─────────────────────┐
819            │ID     │ Name                        │ Description         │
820            ├───────┼─────────────────────────────┼─────────────────────┤
821            │0x009F │ DHE-RSA-AES256-GCM-SHA384   │ TLSv1.2 DH RSA AES‐ │
822            │       │                             │ GCM(256) AEAD       │
823            ├───────┼─────────────────────────────┼─────────────────────┤
824            │0x006B │ DHE-RSA-AES256-SHA256       │ TLSv1.2    DH   RSA │
825            │       │                             │ AES(256) SHA256     │
826            ├───────┼─────────────────────────────┼─────────────────────┤
827            │0xC030 │ ECDHE-RSA-AES256-GCM-SHA384 │ TLSv1.2   ECDH  RSA │
828            │       │                             │ AESGCM(256) AEAD    │
829            ├───────┼─────────────────────────────┼─────────────────────┤
830            │0xC028 │ ECDHE-RSA-AES256-SHA384     │ TLSv1.2  ECDH   RSA │
831            │       │                             │ AES(256) SHA384     │
832            ├───────┼─────────────────────────────┼─────────────────────┤
833            │0x009E │ DHE-RSA-AES128-GCM-SHA256   │ TLSv1.2 DH RSA AES‐ │
834            │       │                             │ GCM(128) AEAD       │
835            ├───────┼─────────────────────────────┼─────────────────────┤
836            │0x0067 │ DHE-RSA-AES128-SHA256       │ TLSv1.2   DH    RSA │
837            │       │                             │ AES(128) SHA256     │
838            ├───────┼─────────────────────────────┼─────────────────────┤
839            │0xC02F │ ECDHE-RSA-AES128-GCM-SHA256 │ TLSv1.2   ECDH  RSA │
840            │       │                             │ AESGCM(128) AEAD    │
841            ├───────┼─────────────────────────────┼─────────────────────┤
842            │0xC027 │ ECDHE-RSA-AES128-SHA256     │ TLSv1.2  ECDH   RSA │
843            │       │                             │ AES(128) SHA256     │
844            └───────┴─────────────────────────────┴─────────────────────┘
845

AUTHOR

847       The Syncthing Authors
848
850       2014-2019, The Syncthing Authors
851
852
853
854
855v1                               Mar 17, 2020                 SYNCTHING-BEP(7)
Impressum