1SYNCTHING-BEP(7) Syncthing SYNCTHING-BEP(7)
2
3
4
6 syncthing-bep - Block Exchange Protocol v1
7
9 The Block Exchange Protocol (BEP) is used between two or more devices
10 thus forming a cluster. Each device has one or more folders of files
11 described by the local model, containing metadata and block hashes. The
12 local model is sent to the other devices in the cluster. The union of
13 all files in the local models, with files selected for highest change
14 version, forms the global model. Each device strives to get its folders
15 in sync with the global model by requesting missing or outdated blocks
16 from the other devices in the cluster.
17
18 File data is described and transferred in units of blocks, each being
19 from 128 KiB (131072 bytes) to 16 MiB in size, in steps of powers of
20 two. The block size may vary between files but is constant in any given
21 file, except for the last block which may be smaller.
22
23 The key words “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”,
24 “SHOULD”, “SHOULD NOT”, “RECOMMENDED”, “MAY”, and “OPTIONAL” in this
25 document are to be interpreted as described in RFC 2119 <https://data‐
26 tracker.ietf.org/doc/html/rfc2119.html>.
27
29 BEP is deployed as the highest level in a protocol stack, with the
30 lower level protocols providing encryption and authentication.
31
32 +-----------------------------+
33 | Block Exchange Protocol |
34 |-----------------------------|
35 | Encryption & Auth (TLS 1.2) |
36 |-----------------------------|
37 | Reliable Transport |
38 |-----------------------------|
39 v ... v
40
41 The encryption and authentication layer SHALL use TLS 1.2 or a higher
42 revision. A strong cipher suite SHALL be used, with “strong cipher
43 suite” being defined as being without known weaknesses and providing
44 Perfect Forward Secrecy (PFS). Examples of strong cipher suites are
45 given at the end of this document. This is not to be taken as an ex‐
46 haustive list of allowed cipher suites but represents best practices at
47 the time of writing.
48
49 The exact nature of the authentication is up to the application, how‐
50 ever it SHALL be based on the TLS certificate presented at the start of
51 the connection. Possibilities include certificates signed by a common
52 trusted CA, preshared certificates, preshared certificate fingerprints
53 or certificate pinning combined with some out of band first verifica‐
54 tion. The reference implementation uses preshared certificate finger‐
55 prints (SHA-256) referred to as “Device IDs”.
56
57 There is no required order or synchronization among BEP messages except
58 as noted per message type - any message type may be sent at any time
59 and the sender need not await a response to one message before sending
60 another.
61
62 The underlying transport protocol MUST guarantee reliable packet deliv‐
63 ery.
64
65 In this document, in diagrams and text, “bit 0” refers to the most sig‐
66 nificant bit of a word; “bit 15” is thus the least significant bit of a
67 16 bit word (int16) and “bit 31” is the least significant bit of a 32
68 bit word (int32). Non protocol buffer integers are always represented
69 in network byte order (i.e., big endian) and are signed unless stated
70 otherwise, but when describing message lengths negative values do not
71 make sense and the most significant bit MUST be zero.
72
73 The protocol buffer schemas in this document are in proto3 syntax. This
74 means, among other things, that all fields are optional and will assume
75 their default value when missing. This does not necessarily mean that a
76 message is valid with all fields empty - for example, an index entry
77 for a file that does not have a name is not useful and MAY be rejected
78 by the implementation. However the folder label is for human consump‐
79 tion only so an empty label should be accepted - the implementation
80 will have to choose some way to represent the folder, perhaps by using
81 the ID in it’s place or automatically generating a label.
82
84 AFTER establishing a connection, but BEFORE performing any authentica‐
85 tion, devices MUST exchange Hello messages.
86
87 Hello messages are used to carry additional information about the peer,
88 which might be of interest to the user even if the peer is not permit‐
89 ted to communicate due to failing authentication. Note that the cer‐
90 tificate based authentication may be considered part of the TLS hand‐
91 shake that precedes the Hello message exchange, but even in the case
92 that a connection is rejected a Hello message must be sent before the
93 connection is terminated.
94
95 Hello messages MUST be prefixed with an int32 containing the magic num‐
96 ber 0x2EA7D90B, followed by an int16 representing the size of the mes‐
97 sage, followed by the contents of the Hello message itself.
98
99 0 1
100 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5
101 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
102 | Magic |
103 | (32 bits) |
104 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
105 | Length |
106 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
107 / /
108 \ Hello \
109 / /
110 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
111
112 The Hello message itself is in protocol buffer format with the follow‐
113 ing schema:
114
115 message Hello {
116 string device_name = 1;
117 string client_name = 2;
118 string client_version = 3;
119 }
120
121 Fields (Hello message)
122 The device_name is a human readable (configured or auto detected) de‐
123 vice name or host name, for the remote device.
124
125 The client_name and client_version identifies the implementation. The
126 values SHOULD be simple strings identifying the implementation name,
127 as a user would expect to see it, and the version string in the same
128 manner. An example client name is “syncthing” and an example client
129 version is “v0.7.2”. The client version field SHOULD follow the pat‐
130 terns laid out in the Semantic Versioning <https://semver.org/> stan‐
131 dard.
132
133 Immediately after exchanging Hello messages, the connection MUST be
134 dropped if the remote device does not pass authentication.
135
137 Every message post authentication is made up of several parts:
138
139 • A header length word
140
141 • A Header
142
143 • A message length word
144
145 • A Message
146
147 0 1
148 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5
149 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
150 | Header Length |
151 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
152 / /
153 \ Header \
154 / /
155 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
156 | Message Length |
157 | (32 bits) |
158 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
159 / /
160 \ Message \
161 / /
162 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
163
164 The header length word is 16 bits. It indicates the length of the fol‐
165 lowing Header message. The Header is in protocol buffer format. The
166 Header describes the type and compression status of the following mes‐
167 sage.
168
169 The message is preceded by the 32 bit message length word and is one of
170 the concrete BEP messages described below, identified by the type field
171 of the Header.
172
173 As always, the length words are in network byte order (big endian).
174
175 message Header {
176 MessageType type = 1;
177 MessageCompression compression = 2;
178 }
179
180 enum MessageType {
181 CLUSTER_CONFIG = 0;
182 INDEX = 1;
183 INDEX_UPDATE = 2;
184 REQUEST = 3;
185 RESPONSE = 4;
186 DOWNLOAD_PROGRESS = 5;
187 PING = 6;
188 CLOSE = 7;
189 }
190
191 enum MessageCompression {
192 NONE = 0;
193 LZ4 = 1;
194 }
195
196 When the compression field is NONE, the message is directly in protocol
197 buffer format.
198
199 When the compression field is LZ4, the message consists of a 32 bit in‐
200 teger describing the uncompressed message length followed by a single
201 LZ4 block. After decompressing the LZ4 block it should be interpreted
202 as a protocol buffer message just as in the uncompressed case.
203
205 Cluster Config
206 This informational message provides information about the cluster con‐
207 figuration as it pertains to the current connection. A Cluster Config
208 message MUST be the first post authentication message sent on a BEP
209 connection. Additional Cluster Config messages MUST NOT be sent after
210 the initial exchange.
211
212 Protocol Buffer Schema
213 message ClusterConfig {
214 repeated Folder folders = 1;
215 }
216
217 message Folder {
218 string id = 1;
219 string label = 2;
220 bool read_only = 3;
221 bool ignore_permissions = 4;
222 bool ignore_delete = 5;
223 bool disable_temp_indexes = 6;
224 bool paused = 7;
225
226 repeated Device devices = 16;
227 }
228
229 message Device {
230 bytes id = 1;
231 string name = 2;
232 repeated string addresses = 3;
233 Compression compression = 4;
234 string cert_name = 5;
235 int64 max_sequence = 6;
236 bool introducer = 7;
237 uint64 index_id = 8;
238 bool skip_introduction_removals = 9;
239 bytes encryption_password_token = 10;
240 }
241
242 enum Compression {
243 METADATA = 0;
244 NEVER = 1;
245 ALWAYS = 2;
246 }
247
248 Fields (Cluster Config Message)
249 The folders field contains the list of folders that will be synchro‐
250 nized over the current connection.
251
252 Fields (Folder Message)
253 The id field contains the folder ID, which is the unique identifier of
254 the folder.
255
256 The label field contains the folder label, the human readable name of
257 the folder.
258
259 The read_only field is set for folders that the device will accept no
260 updates from the network for.
261
262 The ignore_permissions field is set for folders that the device will
263 not accept or announce file permissions for.
264
265 The ignore_delete field is set for folders that the device will ignore
266 deletes for.
267
268 The disable_temp_indexes field is set for folders that will not dis‐
269 patch and do not wish to receive progress updates about partially down‐
270 loaded files via Download Progress messages.
271
272 The paused field is set for folders that are currently paused.
273
274 The devices field is a list of devices participating in sharing this
275 folder.
276
277 Fields (Device Message)
278 The device id field is a 32 byte number that uniquely identifies the
279 device. For instance, the reference implementation uses the SHA-256 of
280 the device X.509 certificate.
281
282 The name field is a human readable name assigned to the described de‐
283 vice by the sending device. It MAY be empty and it need not be unique.
284
285 The list of addresses is that used by the sending device to connect to
286 the described device.
287
288 The compression field indicates the compression mode in use for this
289 device and folder. The following values are valid:
290
291 0 Compress metadata. This enables compression of metadata messages
292 such as Index.
293
294 1 Compression disabled. No compression is used on any message.
295
296 2 Compress always. Metadata messages as well as Response messages
297 are compressed.
298
299 The cert_name field indicates the expected certificate name for this
300 device. It is commonly blank, indicating to use the implementation de‐
301 fault.
302
303 The max_sequence field contains the highest sequence number of the
304 files in the index. See Delta Index Exchange for the usage of this
305 field.
306
307 The introducer field is set for devices that are trusted as cluster in‐
308 troducers.
309
310 The index_id field contains the unique identifier for the current set
311 of index data. See Delta Index Exchange for the usage of this field.
312
313 The skip_introduction_removals field signifies if the remote device has
314 opted to ignore introduction removals for the given device. This set‐
315 ting is copied across as we are being introduced to a new device.
316
317 The encryption_password_token field contains a token derived from the
318 password, that is used to encrypt data sent to this device. If the de‐
319 vice is the same as the device sending the message, it signifies that
320 the device itself has encrypted data that was encrypted with the given
321 token. It is empty or missing if there is no encryption. See Untrusted
322 Device Encryption for details on the encryption scheme.
323
324 Index and Index Update
325 The Index and Index Update messages define the contents of the senders
326 folder. An Index message represents the full contents of the folder and
327 thus supersedes any previous index. An Index Update amends an existing
328 index with new information, not affecting any entries not included in
329 the message. An Index Update MAY NOT be sent unless preceded by an In‐
330 dex, unless a non-zero Max Sequence has been announced for the given
331 folder by the peer device.
332
333 The Index and Index Update messages are currently identical in format,
334 although this is not guaranteed to be the case in the future.
335
336 Protocol Buffer Schema
337 message Index {
338 string folder = 1;
339 repeated FileInfo files = 2;
340 }
341
342 message IndexUpdate {
343 string folder = 1;
344 repeated FileInfo files = 2;
345 }
346
347 message FileInfo {
348 string name = 1;
349 FileInfoType type = 2;
350 int64 size = 3;
351 uint32 permissions = 4;
352 int64 modified_s = 5;
353 int32 modified_ns = 11;
354 uint64 modified_by = 12;
355 bool deleted = 6;
356 bool invalid = 7;
357 bool no_permissions = 8;
358 Vector version = 9;
359 int64 sequence = 10;
360 int32 block_size = 13;
361
362 repeated BlockInfo Blocks = 16;
363 string symlink_target = 17;
364 }
365
366 enum FileInfoType {
367 FILE = 0;
368 DIRECTORY = 1;
369 SYMLINK_FILE = 2 [deprecated = true];
370 SYMLINK_DIRECTORY = 3 [deprecated = true];
371 SYMLINK = 4;
372 }
373
374 message BlockInfo {
375 int64 offset = 1;
376 int32 size = 2;
377 bytes hash = 3;
378 uint32 weak_hash = 4;
379 }
380
381 message Vector {
382 repeated Counter counters = 1;
383 }
384
385 message Counter {
386 uint64 id = 1;
387 uint64 value = 2;
388 }
389
390 Fields (Index Message)
391 The folder field identifies the folder that the index message pertains
392 to.
393
394 The files field is a list of files making up the index information.
395
396 Fields (FileInfo Message)
397 The name is the file name path relative to the folder root. Like all
398 strings in BEP, the Name is always in UTF-8 NFC regardless of operating
399 system or file system specific conventions. The name field uses the
400 slash character (“/”) as path separator, regardless of the implementa‐
401 tion’s operating system conventions. The combination of folder and name
402 uniquely identifies each file in a cluster.
403
404 The type field contains the type of the described item. The type is one
405 of file (0), directory (1), or symlink (4).
406
407 The size field contains the size of the file, in bytes. For directories
408 and symlinks the size is zero.
409
410 The permissions field holds the common Unix permission bits. An imple‐
411 mentation MAY ignore or interpret these as is suitable on the host op‐
412 erating system.
413
414 The modified_s time is expressed as the number of seconds since the
415 Unix Epoch (1970-01-01 00:00:00 UTC). The modified_ns field holds the
416 nanosecond part of the modification time.
417
418 The modified_by field holds the short id of the client that last made
419 any modification to the file whether add, change or delete. This will
420 be overwritten every time a change is made to the file by the last
421 client to do so and so does not hold history.
422
423 The deleted field is set when the file has been deleted. The block list
424 SHALL be of length zero and the modification time indicates the time of
425 deletion or, if the time of deletion is not reliably determinable, the
426 last known modification time.
427
428 The invalid field is set when the file is invalid and unavailable for
429 synchronization. A peer MAY set this bit to indicate that it can tempo‐
430 rarily not serve data for the file.
431
432 The no_permissions field is set when there is no permission information
433 for the file. This is the case when it originates on a file system
434 which does not support permissions. Changes to only permission bits
435 SHOULD be disregarded on files with this bit set. The permissions bits
436 MUST be set to the octal value 0666.
437
438 The version field is a version vector describing the updates performed
439 to a file by all members in the cluster. Each counter in the version
440 vector is an ID-Value tuple. The ID is the first 64 bits of the device
441 ID. The Value is a simple incrementing counter, starting at zero. The
442 combination of Folder, Name and Version uniquely identifies the con‐
443 tents of a file at a given point in time.
444
445 The sequence field is the value of a device local monotonic clock at
446 the time of last local database update to a file. The clock ticks on
447 every local database update, thus forming a sequence number over data‐
448 base updates.
449
450 The block_size field is the size, in bytes, of each individual block in
451 the block list (except, possibly, the last block). If this field is
452 missing or zero, the block size is assumed to be 128 KiB (131072
453 bytes). Valid values of this field are the powers of two from 128 KiB
454 through 16 MiB. See also Selection of Block Size.
455
456 The blocks list contains the size and hash for each block in the file.
457 Each block represents a block_size-sized slice of the file, except for
458 the last block which may represent a smaller amount of data. The block
459 list is empty for directories and symlinks.
460
461 The symlink_target field contains the symlink target, for entries of
462 symlink type. It is empty for all other entry types.
463
464 Request
465 The Request message expresses the desire to receive a data block corre‐
466 sponding to a part of a certain file in the peer’s folder.
467
468 Protocol Buffer Schema
469 message Request {
470 int32 id = 1;
471 string folder = 2;
472 string name = 3;
473 int64 offset = 4;
474 int32 size = 5;
475 bytes hash = 6;
476 bool from_temporary = 7;
477 }
478
479 Fields
480 The id is the request identifier. It will be matched in the correspond‐
481 ing Response message. Each outstanding request must have a unique ID.
482
483 The folder and name fields are as documented for the Index message.
484 The offset and size fields specify the region of the file to be trans‐
485 ferred. This SHOULD equate to exactly one block as seen in an Index
486 message.
487
488 The hash field MAY be set to the expected hash value of the block. If
489 set, the other device SHOULD ensure that the transmitted block matches
490 the requested hash. The other device MAY reuse a block from a different
491 file and offset having the same size and hash, if one exists.
492
493 The from_temporary field is set to indicate that the read should be
494 performed from the temporary file (converting name to it’s temporary
495 form) and falling back to the non temporary file if any error occurs.
496 Knowledge of contents of temporary files comes from DownloadProgress
497 messages.
498
499 Response
500 The Response message is sent in response to a Request message.
501
502 Protocol Buffer Schema
503 message Response {
504 int32 id = 1;
505 bytes data = 2;
506 ErrorCode code = 3;
507 }
508
509 enum ErrorCode {
510 NO_ERROR = 0;
511 GENERIC = 1;
512 NO_SUCH_FILE = 2;
513 INVALID_FILE = 3;
514 }
515
516 Fields
517 The id field is the request identifier. It must match the ID of the Re‐
518 quest that is being responded to.
519
520 The data field contains either the requested data block or is empty if
521 the requested block is not available.
522
523 The code field contains an error code describing the reason a Request
524 could not be fulfilled, in the case where zero length data was re‐
525 turned. The following values are defined:
526
527 0 No Error (data should be present)
528
529 1 Generic Error
530
531 2 No Such File (the requested file does not exist, or the offset
532 is outside the acceptable range for the file)
533
534 3 Invalid (file exists but has invalid bit set or is otherwise un‐
535 available)
536
537 DownloadProgress
538 The DownloadProgress message is used to notify remote devices about
539 partial availability of files. By default, these messages are sent ev‐
540 ery 5 seconds, and only in the cases where progress or state changes
541 have been detected. Each DownloadProgress message is addressed to a
542 specific folder and MUST contain zero or more FileDownloadProgressUp‐
543 date messages.
544
545 Protocol Buffer Schema
546 message DownloadProgress {
547 string folder = 1;
548 repeated FileDownloadProgressUpdate updates = 2;
549 }
550
551 message FileDownloadProgressUpdate {
552 FileDownloadProgressUpdateType update_type = 1;
553 string name = 2;
554 Vector version = 3;
555 repeated int32 block_indexes = 4;
556 }
557
558 enum FileDownloadProgressUpdateType {
559 APPEND = 0;
560 FORGET = 1;
561 }
562
563 Fields (DownloadProgress Message)
564 The folder field represents the ID of the folder for which the update
565 is being provided.
566
567 The updates field is a list of progress update messages.
568
569 Fields (FileDownloadProgressUpdate Message)
570 The update_type indicates whether the update is of type append (0) (new
571 blocks are available) or forget (1) (the file transfer has completed or
572 failed).
573
574 The name field defines the file name from the global index for which
575 this update is being sent.
576
577 The version message defines the version of the file for which this up‐
578 date is being sent.
579
580 The block_indexes field is a list of positive integers, where each in‐
581 teger represents the index of the block in the FileInfo message Blocks
582 array that has become available for download.
583
584 For example an integer with value 3 represents that the data defined in
585 the fourth BlockInfo message of the FileInfo message of that file is
586 now available. Please note that matching should be done on name AND
587 version. Furthermore, each update received is incremental, for example
588 the initial update message might contain indexes 0, 1, 2, an update 5
589 seconds later might contain indexes 3, 4, 5 which should be appended to
590 the original list, which implies that blocks 0-5 are currently avail‐
591 able.
592
593 Block indexes MAY be added in any order. An implementation MUST NOT as‐
594 sume that block indexes are added in any specific order.
595
596 The forget field being set implies that previously advertised file is
597 no longer available, therefore the list of block indexes should be
598 truncated.
599
600 Messages with the forget field set MUST NOT have any block indexes.
601
602 Any update message which is being sent for a different version of the
603 same file name must be preceded with an update message for the old ver‐
604 sion of that file with the forget field set.
605
606 As a safeguard on the receiving side, the value of version changing be‐
607 tween update messages implies that the file has changed and that any
608 indexes previously advertised are no longer available. The list of
609 available block indexes MUST be replaced (rather than appended) with
610 the indexes specified in this message.
611
612 Ping
613 The Ping message is used to determine that a connection is alive, and
614 to keep connections alive through state tracking network elements such
615 as firewalls and NAT gateways. A Ping message is sent every 90 seconds,
616 if no other message has been sent in the preceding 90 seconds.
617
618 Protocol Buffer Schema
619 message Ping {
620 }
621
622 Close
623 The Close message MAY be sent to indicate that the connection will be
624 torn down due to an error condition. A Close message MUST NOT be fol‐
625 lowed by further messages.
626
627 Protocol Buffer Schema
628 message Close {
629 string reason = 1;
630 }
631
632 Fields
633 The reason field contains a human readable description of the error
634 condition.
635
637 Trusted
638 Trusted mode is the default sharing mode. Updates are exchanged in both
639 directions.
640
641 +------------+ Updates /---------\
642 | | -----------> / \
643 | Device | | Cluster |
644 | | <----------- \ /
645 +------------+ Updates \---------/
646
647 Send Only
648 In send only mode, a device does not apply any updates from the clus‐
649 ter, but publishes changes of its local folder to the cluster as usual.
650
651 +------------+ Updates /---------\
652 | | -----------> / \
653 | Device | | Cluster |
654 | | \ /
655 +------------+ \---------/
656
657 Receive Only
658 In receive only mode, a device does not send any updates to the clus‐
659 ter, but accepts changes to its local folder from the cluster as usual.
660
661 +------------+ Updates /---------\
662 | | <----------- / \
663 | Device | | Cluster |
664 | | \ /
665 +------------+ \---------/
666
668 Index data must be exchanged whenever two devices connect so that one
669 knows the files available on the other. In the most basic case this
670 happens by way of sending an Index message followed by one or more In‐
671 dex Update messages. Any previous index data known for a remote device
672 is removed and replaced with the new index data received in an Index
673 message, while the contents of an Index Update message is simply added
674 to the existing index data.
675
676 For situations with large indexes or frequent reconnects this can be
677 quite inefficient. A mechanism can then be used to retain index data
678 between connections and only transmit any changes since that data on
679 connection start. This is called “delta indexes”. To enable this mecha‐
680 nism the sequence and index ID fields are used.
681
682 Sequence:
683 Each index item (i.e., file, directory or symlink) has a se‐
684 quence number field. It contains the value of a counter at the
685 time the index item was updated. The counter increments by one
686 for each change. That is, as files are scanned and added to the
687 index they get assigned sequence numbers 1, 2, 3 and so on. The
688 next file to be changed or detected gets sequence number 4, and
689 future updates continue in the same fashion.
690
691 Index ID:
692 Each folder has an Index ID. This is a 64 bit random identifier
693 set at index creation time.
694
695 Given the above, we know that the tuple {index ID, maximum sequence
696 number} uniquely identifies a point in time of a given index. Any fur‐
697 ther changes will increase the sequence number of some item, and thus
698 the maximum sequence number for the index itself. Should the index be
699 reset or removed (i.e., the sequence number reset to zero), a new index
700 ID must be generated.
701
702 By letting a device know the {index ID, maximum sequence number} we
703 have for their index data, that device can arrange to only transmit In‐
704 dex Update messages for items with a higher sequence number. This is
705 the delta index mechanism.
706
707 The index ID and maximum sequence number known for each device is
708 transmitted in the Cluster Config message at connection start.
709
710 For this mechanism to be reliable it is essential that outgoing index
711 information is ordered by increasing sequence number. Devices announc‐
712 ing a non-zero index ID in the Cluster Config message MUST send all in‐
713 dex data ordered by increasing sequence number. Devices not intending
714 to participate in delta index exchange MUST send a zero index ID or,
715 equivalently, not send the index_id attribute at all.
716
718 An implementation MAY impose reasonable limits on the length of mes‐
719 sages and message fields to aid robustness in the face of corruption or
720 broken implementations. An implementation should strive to keep mes‐
721 sages short and to the point, favouring more and smaller messages over
722 fewer and larger. For example, favour a smaller Index message followed
723 by one or more Index Update messages rather than sending a very large
724 Index message.
725
726 The Syncthing implementation imposes a hard limit of 500,000,000 bytes
727 on all messages. Attempting to send or receive a larger message will
728 result in a connection close. This size was chosen to accommodate Index
729 messages containing a large block list. It’s intended that the limit
730 may be further reduced in a future protocol update supporting variable
731 block sizes (and thus shorter block lists for large files).
732
734 The desired block size for any given file is the smallest block size
735 that results in fewer than 2000 blocks, or the maximum block size for
736 larger files. This rule results in the following table of block sizes
737 per file size:
738
739 ┌──────────────────┬────────────┐
740 │File Size │ Block Size │
741 └──────────────────┴────────────┘
742
743 │0 - 250 MiB │ 128 KiB │
744 ├──────────────────┼────────────┤
745 │250 MiB - 500 MiB │ 256 KiB │
746 ├──────────────────┼────────────┤
747 │500 MiB - 1 GiB │ 512 KiB │
748 ├──────────────────┼────────────┤
749 │1 GiB - 2 GiB │ 1 MiB │
750 ├──────────────────┼────────────┤
751 │2 GiB - 4 GiB │ 2 MiB │
752 ├──────────────────┼────────────┤
753 │4 GiB - 8 GiB │ 4 MiB │
754 ├──────────────────┼────────────┤
755 │8 GiB - 16 GiB │ 8 MiB │
756 ├──────────────────┼────────────┤
757 │16 GiB - up │ 16 MiB │
758 └──────────────────┴────────────┘
759
760 An implementation MAY deviate from the block size rule when there is
761 good reason to do so. For example, if a file has been indexed at a cer‐
762 tain block size and grows beyond 2000 blocks it may be retained at the
763 current block size for practical reasons. When there is no overriding
764 reason to the contrary, such as when indexing a new file for the first
765 time, the block size rule above SHOULD be followed.
766
767 An implementation MUST therefore accept files with a block size differ‐
768 ing from the above rule. This does not mean that arbitrary block sizes
769 are allowed. The block size used MUST be exactly one of the
770 power-of-two block sizes listed in the table above.
771
773 ┌───┬───────────────────┬─────────────────────┐
774 │# │ A │ B │
775 ├───┼───────────────────┼─────────────────────┤
776 │1 │ ClusterConfigura‐ │ <-ClusterConfigura‐ │
777 │ │ tion-> │ tion │
778 ├───┼───────────────────┼─────────────────────┤
779 │2 │ Index-> │ <-Index │
780 ├───┼───────────────────┼─────────────────────┤
781 │3 │ IndexUpdate-> │ <-IndexUpdate │
782 ├───┼───────────────────┼─────────────────────┤
783 │4 │ IndexUpdate-> │ │
784 ├───┼───────────────────┼─────────────────────┤
785 │5 │ Request-> │ │
786 ├───┼───────────────────┼─────────────────────┤
787 │6 │ Request-> │ │
788 ├───┼───────────────────┼─────────────────────┤
789 │7 │ Request-> │ │
790 ├───┼───────────────────┼─────────────────────┤
791 │8 │ Request-> │ │
792 ├───┼───────────────────┼─────────────────────┤
793 │9 │ │ <-Response │
794 ├───┼───────────────────┼─────────────────────┤
795 │10 │ │ <-Response │
796 ├───┼───────────────────┼─────────────────────┤
797 │11 │ │ <-Response │
798 ├───┼───────────────────┼─────────────────────┤
799 │12 │ │ <-Response │
800 ├───┼───────────────────┼─────────────────────┤
801 │13 │ Index Update-> │ │
802 ├───┼───────────────────┼─────────────────────┤
803 │… │ │ │
804 ├───┼───────────────────┼─────────────────────┤
805 │14 │ │ <-Ping │
806 ├───┼───────────────────┼─────────────────────┤
807 │15 │ Ping-> │ │
808 └───┴───────────────────┴─────────────────────┘
809
810 The connection is established and at 1. both peers send ClusterConfigu‐
811 ration messages and then Index records. The Index records are received
812 and both peers recompute their knowledge of the data in the cluster. In
813 this example, peer A has four missing or outdated blocks. At 5 through
814 8 peer A sends requests for these blocks. The requests are received by
815 peer B, who retrieves the data from the folder and transmits Response
816 records (9 through 12). Device A updates their folder contents and
817 transmits an Index Update message (13). Both peers enter idle state af‐
818 ter 13. At some later time 14, the ping timer on device B expires and a
819 Ping message is sent. The same process occurs for device A at 15.
820
822 ┌───────┬─────────────────────────────┬─────────────────────┐
823 │ID │ Name │ Description │
824 ├───────┼─────────────────────────────┼─────────────────────┤
825 │0x009F │ DHE-RSA-AES256-GCM-SHA384 │ TLSv1.2 DH RSA AES‐ │
826 │ │ │ GCM(256) AEAD │
827 ├───────┼─────────────────────────────┼─────────────────────┤
828 │0x006B │ DHE-RSA-AES256-SHA256 │ TLSv1.2 DH RSA │
829 │ │ │ AES(256) SHA256 │
830 ├───────┼─────────────────────────────┼─────────────────────┤
831 │0xC030 │ ECDHE-RSA-AES256-GCM-SHA384 │ TLSv1.2 ECDH RSA │
832 │ │ │ AESGCM(256) AEAD │
833 ├───────┼─────────────────────────────┼─────────────────────┤
834 │0xC028 │ ECDHE-RSA-AES256-SHA384 │ TLSv1.2 ECDH RSA │
835 │ │ │ AES(256) SHA384 │
836 ├───────┼─────────────────────────────┼─────────────────────┤
837 │0x009E │ DHE-RSA-AES128-GCM-SHA256 │ TLSv1.2 DH RSA AES‐ │
838 │ │ │ GCM(128) AEAD │
839 ├───────┼─────────────────────────────┼─────────────────────┤
840 │0x0067 │ DHE-RSA-AES128-SHA256 │ TLSv1.2 DH RSA │
841 │ │ │ AES(128) SHA256 │
842 ├───────┼─────────────────────────────┼─────────────────────┤
843 │0xC02F │ ECDHE-RSA-AES128-GCM-SHA256 │ TLSv1.2 ECDH RSA │
844 │ │ │ AESGCM(128) AEAD │
845 ├───────┼─────────────────────────────┼─────────────────────┤
846 │0xC027 │ ECDHE-RSA-AES128-SHA256 │ TLSv1.2 ECDH RSA │
847 │ │ │ AES(128) SHA256 │
848 └───────┴─────────────────────────────┴─────────────────────┘
849
851 The Syncthing Authors
852
854 2014-2019, The Syncthing Authors
855
856
857
858
859v1.25.0 Oct 05, 2023 SYNCTHING-BEP(7)