1ddr_crypt(1) En/Decryption plugin for dd_rescue ddr_crypt(1)
2
6 ddr_crypt - Data de/encryption plugin for dd_rescue
7
9 -L crypt[=option[:option[:...]]]
10 or
11 -L /path/to/libddr_crypt.so[=option[:option[:...]]]
12
14 About
15 This plugin allows to en/decrypt data on the fly as it is written out
16 by dd_rescue. It supports a variance of AES ciphers and uses the
17 hardware acceleration on x86 (AESNI) and ARMv8 -- if available -- to
18 provide good performance.
19 This plugin has been written for dd_rescue and uses the plugin
21 interface from it. See the dd_rescue(1) man page for more information
22 on dd_rescue.
23
25 Options are passed using dd_rescue option passing syntax: The name of
26 the plugin (crypt) is followed by an equal sign (=) and options are
27 separated by a colon (:). the crypt plugin also allows for some
28 options to be abbreviated. At least encryption or decryption and a way
29 to determine the key (and IV) needs to be passed.
30 Help
32 Pass help to have ddr_crypt output a short list of options.
33 Encryption or Decryption
35 The crypt dd_rescue plugin (subsequently referred to as just ddr_crypt
36 which reflects the variable parts of the file name libddr_crypt.so)
37 need to be told whether to encrypt or decrypt. This is done by
38 specifying enc[rypt] or dec[rypt] parameters on the command line.
39 De/encryption algorithm
41 The crypt plugin supports a number of the de/encryption variants of
42 AES. You can specify which one you want to use by passing
43 algo=AESxxx-yyy,
45 where xxx can be 128, 192, 256, 128+, 192+, 256+, 128x2, 192x2, or
46 256x2, and yyy can be ECB, CBC, or CTR. Pass algo=help to get a
47 list of available algorithms. See section ALGORITHMS AND STREAM
48 MODES for a discussion of the options. Note that decrypting a file
49 can only be successful if the exact same algorithms is chosen as
50 was used for encryption. There is no way to tell from an encrypted
51 file which algorithm has been chosen.
52 The default (AES192-CTR) is a good choice if you can ensure that
53 the key/IV combination is NEVER reused.
54 Crypto engine
56 There are several implementations of the AES algorithms that this
57 plugin can currently use:
58 engine=aesni
60 This will use an own AES implementation using the AESNI
61 instructions of the intel x86 core CPUs since Westmere. If
62 possible, this should be used, as it provides both superior
63 performance and avoids some of the cache timing attacks that the
64 lookup tables used in discrete implementations may be prone to.
65 This engine supports all algorithms.
66 engine=aesarm64
68 This will use an own AES implementation using the AESv8 crypto
69 extensions of the ARMv8 (AArch64 = ARM 64bit) architecture
70 (available on CPUs/SOCs based on ARM's Cortex A53,A57,A72 or
71 Qualcomm's Kryo or Apple's A7x or newer designs). If possible,
72 this should be used, as it provides both superior performance and
73 avoids some of the cache timing attacks that the lookup tables used
74 in discrete implementations may be prone to. This engine supports
75 all algorithms.
76 engine=aes_c
78 This uses an AES implementation in C which is based on the rijndael
79 reference code. It is available on all CPUs and supports all
80 algorithms. Some prefetching is done on the tables to make cache
81 timing attacks a bit harder, but this is most likely insufficient
82 to thwart sophisticated attackers.
83 engine=openssl
85 This uses the openssl library to implement the AES routines, which
86 should take advantage of hardware acceleration where available and
87 have received some hardening against attacks. Note that the openssl
88 implementation does not support the + variants (increased number of
89 rounds) that the other engines provide.
90 The engines are used in this order if available, which means that aesni
92 will be used by default on x86 if supported, aesarm64 will be used on
93 ARMv8 and aes_c by default on all other platforms.
94 Padding
96 As AES is a block cipher (with a block size of 16 bytes), files which
97 are not a multiple of the block size need padding at the end to have a
98 full block.
99 pad=zero
101 The last block is filled up with zeroes and then encrypted. Note
102 that on decryption, these zeroes can't be automatically stripped,
103 as the decryptor has no way to tell whether there are true trailing
104 zeros or whether those had been added by padding. This option is
105 thus not recommended for copying files of arbitrary length.
106 pad=always
108 This uses the PKCS7 scheme used by openssl, appending one to
109 sixteen bytes with the number of bytes to discard when decrypting.
110 This is always safe, but always makes the output file larger than
111 the input file. Use this when copying files. This is the default.
112 pad=asneeded
114 This is a hybrid between zero and always. This does PKCS7 padding
115 when the file size does not fill a complete block, but does no
116 padding when it does. This mechanism has in the worst case (no pad
117 bytes) a chance of 1/255 to produce an incorrect decryption result
118 by wrong unpadding (which means that it has an overall chance of
119 misrepresenting the final file size of 1/4080 for arbitrary file
120 sizes). Don't use this unless you can deal with such errors (or
121 exclude them)!
122 Note that the CTR stream mode does NOT require padding and indeed
124 ddr_crypt does not apply any padding regardless of the pad option when
125 it is used.
126 Debugging and Benchmarking
128 The debug parameter makes the ddr_crypt module output some debugging
129 information. With outkeyiv the key and IV will be output. The
130 benchmark parameter will result in reporting the en/decryption speed.
131
133 There are many ways to set the key and IV (for -CBC and -CTR stream
134 modes).
135 Setting directly
137 They can be set directly using keyhex= and ivhex= on the command line.
138 The argument is interpreted as a hex ASCII representation of the key/IV
139 (without leading 0x). This way to specify keys/IVs should only be used
140 for testing - the key will be visible to all local users by looking at
141 the command line (unless specific measures are taken to lock down
142 access to /proc). Not normally a good idea on a multi-user system.
143 ddr_crypt does overwrite the sensitive data with X to make the attack a
144 bit harder, but this still leaks the length and -- more importantly --
145 there is still a time window where the sensitive data is visible.
146 Reading from file of file descriptor
148 It's better to pass in the key and IV via a file descriptor via keyfd=
149 and ivfd= . If the integer is preceded by an x, the key/IV will be
150 read as hex string, otherwise binary data will be used. Optionally,
151 @OFF@LEN can be appended, designating the offset and length (in bytes)
152 to be read in the file passed via the file descriptor. Note that a file
153 descriptor of 0 without shell redirection will result in an interactive
154 prompt to the user and the answer won't be echoed to the screen of
155 course.
156 This is useful mainly when dd_rescue is called from another program.
157 Alternatively, with keyfile= and ivfile= a file name to be opened and
159 read from can be specified. The syntax does support the same optional
160 @OFF@LEN designation, but the key and IV will always be read in binary
161 form. (See below, Index files for a way to read in hex form.)
162 Currently (unlike with salt) there is no way to use ddr_crypt to write
163 out binary key and IV data with these options.
164 Generating random key and IV
166 The Operating System's random number generator can be used to generate
167 key and IV on the fly; if your system offers good random numbers, this
168 is the most secure way to specify and encryption key. The options to
169 specify are keygen and ivgen . You need to save the key/IV somehow,
170 otherwise you can not decrypt again later. (The program will warn you!)
171 Best way is to use the next options.
172 Index files
174 Keys and IVs can be stored as hex strings in index files; the file
175 format is the same as the one used in MD5SUMS: The hex representation
176 of the key/IV is followed by the file name. Obviously, appropriate
177 care needs to be taken to keep those files confidential.
178 If the ddr_crypt plugin gets the option keysfile and ivsfile it will
180 store already created keys/IVs (from the other options) to files names
181 KEYS.algname and IVS.algname in the MD5SUMS format. (The files will be
182 created or updated accordingly.) If key/IV have not been created yet,
183 ddr_crypt will try to retrieve the key/IV from those files and error
184 out upon failure.
185 These options combine well with keygen and ivgen on encryption (and
186 should be used alone on decryption).
187 Extended attributes
189 Similar to index files, keys and IVs can also be stored in and
190 retrieved from the encrypted file's extended attributes. This can be
191 achieved using the options keyxattr and ivxattr . Please review the
192 comments in the main dd_rescue (1) man page for general considerations
193 about using extended attributes (xattrs).
194 Note that storing the key in the xattr is normally not a good idea. A
195 user who can access the encrypted file can (locally) also read the
196 xattrs -- so the secrecy normally achieved by encryption is defeated
197 this way. (There may be valid scenarios, though, e.g. when the file
198 tself is only accessible via a remote protocol that does not expose the
199 xattrs, such as http or DAV or NFS.)
200 You can specify kxfallb[ack] and ixfallb[ack] in addition if you want
201 ddr_crypt to try using xattrs and falling back to keysfile and ivsfile
202 in case the file system does not support the extended attributes.
203 Password based key and IV generation
205 Using the same key/IV for many files harms security severely (see below
206 in ALGORITHMS). So using a directly specified (non-generated) key is
207 not a good idea. However, if you prefer to have something memorable
208 rather than stored, you can use a password and salt to generate many
209 keys from one password.
210 The key and IV are derived from an expensive to compute (and even more
212 expensive to revert) function of password and salt. By default,
213 ddr_crypt uses 17000 rounds of pbkdf2() for the key (and a third for
214 the IV), although a more compute intense function (like scrypt) is
215 planned for the future. The expensiveness of this function is a
216 protection against brute forcing passwords. To use pbkdf2, you need to
217 specify pbkdf2 or pbkdf2=rounds . The latter format allows overriding
218 the number of iterations for key generation. (IV generation will be
219 done with a third again.)
220 For compatibility with openssl, key and IV can also be derived using an
222 openssl compatible key derivation function with opbkdf . Note that
223 this is not recommended; only one round of md5 hashing is used which
224 makes brute-forcing very effective. Using this option also has the
225 side-effect of writing (encryption) or parsing (decryption) an openSSL
226 style Salted__ header. Note the openssl version 1.1 started to default
227 to one round of sha256 hashing instead which can be forced on older
228 openssl versions with -md sha256 and overriden by specifying -md md5 on
229 the openssl command line. You can instruct dd_rescue to use an openssl
230 compatible KDF with sha256 by specifying opbkdf11 . One round of
231 sha256 can of course still be very efficiently brute-forced, so use
232 high-entropy passwords if you really need to use this. nosalthdr .
233 Openssl starting with version 3.0 does no longer write a Salted__
234 header if the salt is passed explicitly. For compatibility with 3.0+,
235 this option allows to no longer write this header (on encryption) nor
236 to look for it (on decryption).
237 The salt can be derived automatically from the name (and length) of the
239 encrypted file; this allows to work with just one password to be
240 memorized. However, be aware that file size or name changes will result
241 in a different salt and thus different key/IV which render your
242 encrypted file undecryptable. If there is a risk of this to happen,
243 rather memorize one salt per file (or better save key and IV using
244 keysfile and ivsfile options or save the salt using saltsfile or
245 saltxattr, see below). Remember that file names are case sensitive (as
246 always with Un*x). Of course the keysfile needs to be well protected
247 from being read by unauthorized persons.
248 Password and salt can be specified with a string pass= and salt= or
250 using the passfd= passfile= and salthex= saltfd= saltfile= options with
251 the same possible parameters as above for direct specification of key
252 and IV. (Note that the salt is hashed, like when derived from file name
253 and length.) The password/passphrase is treated as a string, null-
254 terminated and with a trailing CRLF stripped off.
255 The warnings about passing confidential data (here: pass, salt,
256 salthex) on the command line apply -- only do it for testing or in a
257 single-user environment.
258 If the file name based automatic salt derivation is used, the assumed
260 file length for salt generation can be overridden by saltlen= .
261 Alternatively, the salts can also be stored and retrieved from an
263 MD5SUMS style index file (like with keysfile and ivsfile) by specifying
264 the option saltsfile . When saltsfile is used to store salts, using
265 random salts on encryption becomes a good idea. This can be achieved by
266 specifying the saltgen option.
267 Instead of a salt index file (saltsfile), the salt can also be stored
269 in (and retrieved from) an extended attribute. This can be done using
270 the saltxattr[=xattr_name] option. The attribute name is optional and
271 defaults to user.salt.ALGNAME (with ALGNAME replaced by the algorithm).
272 Since ddr_crypt 1.99, the password-based key derivation function (and
273 the number of iterations) is also stored and retrieved in the xattr
274 user.pbkdf with this option.
275 It's also possible to try using the xattr feature and fall back to
276 using the index file (saltsfile) if your file system does not support
277 extended attributes. Use the sxfallback option to tell ddr_crypt to do
278 this. Note that the pbkdf can not be stored (or retrieved) if the
279 fallback actually takes place.
280 See the main dd_rescue (1) man page for a discussion of advantages and
281 disadvantages of using extended attributes.
282
284 The AES (Rijndael) family of algorithms is considered cryptographically
285 safe at the time of writing, as no practicable attacks have been
286 published against it. It is up to the reader to judge whether (s)he
287 believes that the worst criminals or intelligence agencies are
288 significantly ahead of common (published) knowledge. In reality, it is
289 typically easier to use social engineering or flaws in key handling and
290 random number generation to carry out attacks.
291 Plus modes
293 Given that the best known attacks are against AES versions with a
294 reduced number of rounds with only small round number reductions, it
295 appears that increasing the number of rounds would seem a reasonable
296 countermeasure against cryptographic attacks. (This has been inspired
297 by a comment from Bruce Schneier who the author of this document has
298 very high respect for.)
299 The C and AESNI implementations support AES128,192,256 modes with 2,3,4
301 additional rounds respectively, resulting in 12, 15, 18 rounds. These
302 modes are named AES128+, AES192+, and AES256+ (plus modes)
303 respectively. They do offer a computationally relatively cheap way to
304 enhance security. The author of this document e.g. would chose AES192+
305 over AES256. While the author of this document would never judge
306 himself as a cryptography expert strong enough to create new algorithms
307 or even devise significant changes to existing ones, he considers this
308 variation a choice that is more secure than the original. Please note
309 however, that these custom algorithms result in files that can not be
310 decrypted using any other tools. Also, the openSSL engine does not
311 support the plus modes.
312 Double modes
314 A computationally more expensive method to enhance security is doubling
315 the number of rounds. This is equivalent to encrypting twice (where the
316 second key is a simple derivation of the first). These methods are
317 supported by all engines and are named AES128x2, AES192x2, and
318 AES256x2.
319 Stream modes
321 The AES algorithm is a block cipher -- it transforms 16 byte blocks.
322 The trivial application to a file of arbitrary size is to apply this to
323 every block in the file. This is called ECB (electronic codebook) mode.
324 This is very insecure ... the same input will always result in the same
325 output. Patterns can be easily recognized and known plain text attacks
326 are trivial.
327 It's better to make the transformation dependent on the previous
329 content of the file or the position within it. This is what the CBC
330 (chained block cipher) and CTR modes do.
331 The CBC mode has several disadvantages: It can't be parallelized (every
333 block depends on all previous blocks for encryption; things are better
334 for decryption) and random access is impossible.
335 The CTR mode has many desirable properties. It is basically a stream of
337 (reproducible) pseudo random numbers that are XORed with the input for
338 encryption. Decryption is just another XOR of course. It's a one time
339 pad -- which has been proven to be secure, if the pad is unknown to an
340 attacker and only used once.
341 The latter can't be stressed enough: Don't ever use the same key/IV
342 combination for two files. Mathematically spoken: c1 = r1 XOR p1 and c2
343 = r2 XOR p2 (c = ciphertext, r = AES random numbers, p = plaintext).
344 With r == r1 == r2, it can be trivially seen that the attacker can
345 calculate c1 XOR c2 = r XOR p1 XOR r XOR p2 = p1 XOR p2. If the
346 plaintext of one of the files is partially known, so is the other.
347 The CTR mode has more nice properties: It allows random access as the
349 AES random numbers (belonging to a key/IV combination) with a certain
350 offset can be directly calculated and the last block does not require
351 padding, as partial blocks can be processed.
352 The author of this documents prefers CTR stream mode and ensures that
354 keys/IVs are not reused.
355
357 With CTR mode, you can do partial writes to encrypted files and the
358 result will still be a consistent file (of course assuming that the
359 used key and IV are the same). Same with appending (-x) or reverse
360 direction copies.
361 With ECB mode, this will only work, if file size and offsets are all
362 block (16byte) aligned. With CBC, none of this is possible.
363 The ddr_crypt plugin has no specific support for encoding holes; if
365 however previous correctly encrypted content is present in a hole, the
366 support for partial writes in CTR and ECB mode will result in a
367 meaningful output. If no previous content is in holes, then the result
368 of decrypting zeros will result upon decryption.
369 You can pass the option skiphole to make ddr_crypt leave 512byte blocks
370 of zeros untouched. This will reveal blocks of zeros and may thus
371 disclose valuable information to an attacker, so use with care. Also
372 note that you need to use this with en- and decryption and with the
373 same alignment (mod 512) for encryption to be reversible. You have been
374 warned. (You don't need to be worried about misdetecting zeros on
375 decrypting -- the chances of non-zero plaintext resulting in an aligned
376 512byte block of zeros is smaller than 2^-4096. So this option is safe
377 on decrypting -- if some of the ciphertext has been overwritten with
378 blocks of zeros, you might even prefer to have zeros in the decrypted
379 file rather than random gibberish.)
380 Note that you can compress and encode holes with ddr_lzo and then pass
382 to ddr_crypt to encrypt and pass through ddr_crypt to decrypt and
383 ddr_lzo to uncompress and extract holes again. This only works with CTR
384 mode.
385 The option weakrnd is provided for testing in environments, where
387 strong random numbers are not available. It will cause weaker random
388 numbers to be used for key generation. Don't use it if you want
389 security.
390
392 Files that are encrypted with openssl enc where you specify the key
393 (with -K) and the IV (with -iv) result in the same output that
394 ddr_crypt generates for -ECB and -CBC modes. ddr_crypt uses a 64bit
395 counter in -CTR modes.
396 With the option opbkdf ddr_crypt also reads/writes the openSSL Salted__
398 header to be compatible with openssl. This function needs more testing
399 and better error handling though.
400
402 Maturity
403 The plugin is new as of dd_rescue 1.98. Do not yet rely on data saved
404 with ddr_crypt as the only backup for valuable data. Also expect some
405 changes to ddr_crypt in the not too distant future.
406 Due to an issue in ddr_crypt's initialization of the IV for CTR mode,
408 the last 32bits would always be zeroed out prior to adding the counter.
409 This has been fixed in 1.99. It order to be compatible with 1.98, the
410 option ctrbug198 can be specified on the command line.
411 Security
413 While care has been applied to check the result of memory allocations
414 ..., the code has not been audited and only limited fuzzing has been
415 applied to ensure it's not vulnerable to malicious data -- be careful
416 when you process data from untrusted sources.
417 Key handling is a tricky business -- the author may have screwed up
418 resulting in some ways to use this program to encrypt data may not
419 result in the level of secrecy that is desired.
420 Testing
422 The crypt plugin does not yet have the same test coverage as the other
423 plugins, which means it has not been tested as intensively as the
424 others.
425 Future work
427 Except for more testing and auditing a few more features are envisioned
428 for this plugin:
429 Support for other (non-AES) algorithms such as twofish (and possibly
430 also threefish).
431 Stronger function to derive keys/IVs from passwords than pbkdf2.
432 Support for other streaming modes (XTS, GCM, ...)
433
435 dd_rescue -ptAL crypt=algo=AES256-CTR:enc:keygen:ivgen:keysfile:ivsfile infile outfile
436 encrypts data from infile with AES256 in CTR mode using a
437 generated (random) key and IV and writes the result to outfile
438 It adds a line to KEYS.AES256-CTR and to IVS.AES256-CTR where
439 the used key and IV are written to respectively. (Please ensure
440 that this file is not accessible by any unauthorized person!)
441 Decryption can be performed by
442 dd_rescue -ptAL crypt=algo=AES256-CTR:dec:keysfile:ivsfile outfile infile
444 dd_rescue -AL crypt=AES192+-CTR:enc:saltgen:saltxattr:sxfallback:passfd=0:pbkdf2 infile outfile
446 will ask for a password, generate a random salt and store it in
447 the extended attribute of outfile (and fallback to
448 SALTS.AES192+-CTR index file) and uses pbkdf2 function to
449 produce a key and IV for encrypting the data. For decrypting,
450 just omit the saltgen parameter.
451 dd_rescue -ptL lzo=compr,crypt=AES256-CTR:enc:keygen:ivgen:keysfile:ivsfile infile outfile
453 will compress the data (using lzo) and then encrypt. Use the
454 reverse order and omit keygen and ivgen to decrypt and
455 uncompress. Compression has the nice side effect of dealing with
456 holes, which otherwise get compressed to non-zero values (unless
457 you specify skiphole). Feel free to add the hash plugin at the
458 beginning and/or the end to produce cryptographic checksums for
459 both the original file and the end result.
460
462 dd_rescue(1)
463
465 Kurt Garloff <kurt@garloff.de>
466
468 The x86 AESNI optimized AES implementation has been inspired by an
469 intel whitepaper from 2009:
470 https://software.intel.com/sites/default/files/article/165683/aes-
471 wp-2012-09-22-v01.pdf
472 The ARMv8 AES support has been inspired by studying openSSL assembly as
473 well as Linaro's in-kernel implementation.
474
476 This plugin is under the same license as dd_rescue: The GNU General
477 Public License (GPL) v2 or v3 - at your option.
478
480 ddr_crypt plugin was first introduced with dd_rescue 1.98 (May 2015).
481 Version 1.99 brought support for ARMv8 crypto acceleration and support
482 for openssl style key derivation and Salted__ headers. It also added
483 storing pbkdf related infos in xattrs and added support for storing and
484 retrieving keys (not recommended!) and IVs in/from xattrs. A bug with
485 CTR initialization was resolved (see ctrbug198 option).
486 Some additional information can be found on
488 http://garloff.de/kurt/linux/ddrescue/
489Kurt Garloff 2015-04-15 ddr_crypt(1)