1ddr_crypt(1) En/Decryption plugin for dd_rescue ddr_crypt(1)
2
6 ddr_crypt - Data de/encryption plugin for dd_rescue
7
9 -L crypt[=option[:option[:...]]]
10 or
11 -L /path/to/libddr_crypt.so[=option[:option[:...]]]
12
14 About
15 This plugin allows to en/decrypt data on the fly as it is written out
16 by dd_rescue. It supports a variance of AES ciphers and uses the
17 hardware acceleration on x86 (AESNI) and ARMv8 -- if available -- to
18 provide good performance.
19 This plugin has been written for dd_rescue and uses the plugin
21 interface from it. See the dd_rescue(1) man page for more information
22 on dd_rescue.
23
25 Options are passed using dd_rescue option passing syntax: The name of
26 the plugin (crypt) is followed by an equal sign (=) and options are
27 separated by a colon (:). the crypt plugin also allows for some
28 options to be abbreviated. At least encryption or decryption and a way
29 to determine the key (and IV) needs to be passed.
30 Help
32 Pass help to have ddr_crypt output a short list of options.
33 Encryption or Decryption
35 The crypt dd_rescue plugin (subsequently referred to as just ddr_crypt
36 which reflects the variable parts of the file name libddr_crypt.so)
37 need to be told whether to encrypt or decrypt. This is done by
38 specifying enc[rypt] or dec[rypt] parameters on the command line.
39 De/encryption algorithm
41 The crypt plugin supports a number of the de/encryption variants of
42 AES. You can specify which one you want to use by passing
43 algo=AESxxx-yyy,
45 where xxx can be 128, 192, 256, 128+, 192+, 256+, 128x2, 192x2, or
46 256x2, and yyy can be ECB, CBC, or CTR. Pass algo=help to get a
47 list of available algorithms. See section ALGORITHMS AND STREAM
48 MODES for a discussion of the options. Note that decrypting a file
49 can only be successful if the exact same algorithms is chosen as
50 was used for encryption. There is no way to tell from an encrypted
51 file which algorithm has been chosen.
52 The default (AES192-CTR) is a good choice if you can ensure that
53 the key/IV combination is NEVER reused.
54 Crypto engine
56 There are several implementations of the AES algorithms that this
57 plugin can currently use:
58 engine=aesni
60 This will use an own AES implementation using the AESNI
61 instructions of the intel x86 core CPUs since Westmere. If
62 possible, this should be used, as it provides both superior
63 performance and avoids some of the cache timing attacks that the
64 lookup tables used in discrete implementations may be prone to.
65 This engine supports all algorithms.
66 engine=aesarm64
68 This will use an own AES implementation using the AESv8 crypto
69 extensions of the ARMv8 (AArch64 = ARM 64bit) architecture
70 (available on CPUs/SOCs based on ARM's Cortex A53,A57,A72 or
71 Qualcomm's Kryo or Apple's A7x or newer designs). If possible,
72 this should be used, as it provides both superior performance and
73 avoids some of the cache timing attacks that the lookup tables used
74 in discrete implementations may be prone to. This engine supports
75 all algorithms.
76 engine=aes_c
78 This uses an AES implementation in C which is based on the rijndael
79 reference code. It is available on all CPUs and supports all
80 algorithms. Some prefetching is done on the tables to make cache
81 timing attacks a bit harder, but this is most likely insufficient
82 to thwart sophisticated attackers.
83 engine=openssl
85 This uses the openssl library to implement the AES routines, which
86 should take advantage of hardware acceleration where available and
87 have received some hardening against attacks. Note that the openssl
88 implementation does not support the + variants (increased number of
89 rounds) that the other engines provide.
90 The engines are used in this order if available, which means that aesni
92 will be used by default on x86 if supported, aesarm64 will be used on
93 ARMv8 and aes_c by default on all other platforms.
94 Padding
96 As AES is a block cipher (with a block size of 16 bytes), files which
97 are not a multiple of the block size need padding at the end to have a
98 full block.
99 pad=zero
101 The last block is filled up with zeroes and then encrypted. Note
102 that on decryption, these zeroes can't be automatically stripped,
103 as the decryptor has no way to tell whether there are true trailing
104 zeros or whether those had been added by padding. This option is
105 thus not recommended for copying files of arbitrary length.
106 pad=always
108 This uses the PKCS7 scheme used by openssl, appending one to
109 sixteen bytes with the number of bytes to discard when decrypting.
110 This is always safe, but always makes the output file larger than
111 the input file. Use this when copying files. This is the default.
112 pad=asneeded
114 This is a hybrid between zero and always. This does PKCS7 padding
115 when the file size does not fill a complete block, but does no
116 padding when it does. This mechanism has in the worst case (no pad
117 bytes) a chance of 1/255 to produce an incorrect decryption result
118 by wrong unpadding (which means that it has an overall chance of
119 misrepresenting the final file size of 1/4080 for arbitrary file
120 sizes). Don't use this unless you can deal with such errors (or
121 exclude them)!
122 Note that the CTR stream mode does NOT require padding and indeed
124 ddr_crypt does not apply any padding regardless of the pad option when
125 it is used.
126 Debugging and Benchmarking
128 The debug parameter makes the ddr_crypt module output some debugging
129 information. With outkeyiv the key and IV will be output. The
130 benchmark parameter will result in reporting the en/decryption speed.
131
133 There are many ways to set the key and IV (for -CBC and -CTR stream
134 modes).
135 Setting directly
137 They can be set directly using keyhex= and ivhex= on the command line.
138 The argument is interpreted as a hex ASCII representation of the key/IV
139 (without leading 0x). This way to specify keys/IVs should only be used
140 for testing - the key will be visible to all local users by looking at
141 the command line (unless specific measures are taken to lock down
142 access to /proc). Not normally a good idea on a multi-user system.
143 ddr_crypt does overwrite the sensitive data with X to make the attack a
144 bit harder, but this still leaks the length and -- more importantly --
145 there is still a time window where the sensitive data is visible.
146 Reading from file of file descriptor
148 It's better to pass in the key and IV via a file descriptor via keyfd=
149 and ivfd= . If the integer is preceded by an x, the key/IV will be
150 read as hex string, otherwise binary data will be used. Optionally,
151 @OFF@LEN can be appended, designating the offset and length (in bytes)
152 to be read in the file passed via the file descriptor. Note that a file
153 descriptor of 0 without shell redirection will result in an interactive
154 prompt to the user and the answer won't be echoed to the screen of
155 course.
156 This is useful mainly when dd_rescue is called from another program.
157 Alternatively, with keyfile= and ivfile= a file name to be opened and
159 read from can be specified. The syntax does support the same optional
160 @OFF@LEN designation, but the key and IV will always be read in binary
161 form. (See below, Index files for a way to read in hex form.)
162 Currently (unlike with salt) there is no way to use ddr_crypt to write
163 out binary key and IV data with these options.
164 Generating random key and IV
166 The Operating System's random number generator can be used to generate
167 key and IV on the fly; if your system offers good random numbers, this
168 is the most secure way to specify and encryption key. The options to
169 specify are keygen and ivgen . You need to save the key/IV somehow,
170 otherwise you can not decrypt again later. (The program will warn you!)
171 Best way is to use the next options.
172 Index files
174 Keys and IVs can be stored as hex strings in index files; the file
175 format is the same as the one used in MD5SUMS: The hex representation
176 of the key/IV is followed by the file name. Obviously, appropriate
177 care needs to be taken to keep those files confidential.
178 If the ddr_crypt plugin gets the option keysfile and ivsfile it will
180 store already created keys/IVs (from the other options) to files names
181 KEYS.algname and IVS.algname in the MD5SUMS format. (The files will be
182 created or updated accordingly.) If key/IV have not been created yet,
183 ddr_crypt will try to retrieve the key/IV from those files and error
184 out upon failure.
185 These options combine well with keygen and ivgen on encryption (and
186 should be used alone on decryption).
187 Extended attributes
189 Similar to index files, keys and IVs can also be stored in and
190 retrieved from the encrypted file's extended attributes. This can be
191 achieved using the options keyxattr and ivxattr . Please review the
192 comments in the main dd_rescue (1) man page for general considerations
193 about using extended attributes (xattrs).
194 Note that storing the key in the xattr is normally not a good idea. A
195 user who can access the encrypted file can (locally) also read the
196 xattrs -- so the secrecy normally achieved by encryption is defeated
197 this way. (There may be valid scenarios, though, e.g. when the file
198 tself is only accessible via a remote protocol that does not expose the
199 xattrs, such as http or DAV or NFS.)
200 You can specify kxfallb[ack] and ixfallb[ack] in addition if you want
201 ddr_crypt to try using xattrs and falling back to keysfile and ivsfile
202 in case the file system does not support the extended attributes.
203 Password based key and IV generation
205 Using the same key/IV for many files harms security severely (see below
206 in ALGORITHMS). So using a directly specified (non-generated) key is
207 not a good idea. However, if you prefer to have something memorable
208 rather than stored, you can use a password and salt to generate many
209 keys from one password.
210 The key and IV are derived from an expensive to compute (and even more
212 expensive to revert) function of password and salt. By default,
213 ddr_crypt uses 17000 rounds of pbkdf2() for the key (and a third for
214 the IV), although a more compute intense function (like scrypt) is
215 planned for the future. The expensiveness of this function is a
216 protection against brute forcing passwords. To use pbkdf2, you need to
217 specify pbkdf2 or pbkdf2=rounds . The latter format allows overriding
218 the number of iterations for key generation. (IV generation will be
219 done with a third again.)
220 For compatibility with openssl, key and IV can also be derived using an
222 openssl compatible key derivation function with opbkdf . Note that
223 this is not recommended; only one round of md5 hashing is used which
224 makes brute-forcing very effective. Using this option also has the
225 side-effect of writing (encryption) or parsing (decryption) an openSSL
226 style Salted__ header. Note the openssl version 1.1 started to default
227 to one round of sha256 hashing instead which can be forced on older
228 openssl versions with -md sha256 and overriden by specifying -md md5 on
229 the openssl command line. You can instruct dd_rescue to use an openssl
230 compatible KDF with sha256 by specifying opbkdf11 . One round of
231 sha256 can of course still be very efficiently brute-forced, so use
232 high-entropy passwords if you really need to use this.
233 The salt can be derived automatically from the name (and length) of the
235 encrypted file; this allows to work with just one password to be
236 memorized. However, be aware that file size or name changes will result
237 in a different salt and thus different key/IV which render your
238 encrypted file undecryptable. If there is a risk of this to happen,
239 rather memorize one salt per file (or better save key and IV using
240 keysfile and ivsfile options or save the salt using saltsfile or
241 saltxattr, see below). Remember that file names are case sensitive (as
242 always with Un*x). Of course the keysfile needs to be well protected
243 from being read by unauthorized persons.
244 Password and salt can be specified with a string pass= and salt= or
246 using the passfd= passfile= and salthex= saltfd= saltfile= options with
247 the same possible parameters as above for direct specification of key
248 and IV. (Note that the salt is hashed, like when derived from file name
249 and length.) The password/passphrase is treated as a string, null-
250 terminated and with a trailing CRLF stripped off.
251 The warnings about passing confidential data (here: pass, salt,
252 salthex) on the command line apply -- only do it for testing or in a
253 single-user environment.
254 If the file name based automatic salt derivation is used, the assumed
256 file length for salt generation can be overridden by saltlen= .
257 Alternatively, the salts can also be stored and retrieved from an
259 MD5SUMS style index file (like with keysfile and ivsfile) by specifying
260 the option saltsfile . When saltsfile is used to store salts, using
261 random salts on encryption becomes a good idea. This can be achieved by
262 specifying the saltgen option.
263 Instead of a salt index file (saltsfile), the salt can also be stored
265 in (and retrieved from) an extended attribute. This can be done using
266 the saltxattr[=xattr_name] option. The attribute name is optional and
267 defaults to user.salt.ALGNAME (with ALGNAME replaced by the algorithm).
268 Since ddr_crypt 1.99, the password-based key derivation function (and
269 the number of iterations) is also stored and retrieved in the xattr
270 user.pbkdf with this option.
271 It's also possible to try using the xattr feature and fall back to
272 using the index file (saltsfile) if your file system does not support
273 extended attributes. Use the sxfallback option to tell ddr_crypt to do
274 this. Note that the pbkdf can not be stored (or retrieved) if the
275 fallback actually takes place.
276 See the main dd_rescue (1) man page for a discussion of advantages and
277 disadvantages of using extended attributes.
278
280 The AES (Rijndael) family of algorithms is considered cryptographically
281 safe at the time of writing, as no practicable attacks have been
282 published against it. It is up to the reader to judge whether (s)he
283 believes that the worst criminals or intelligence agencies are
284 significantly ahead of common (published) knowledge. In reality, it is
285 typically easier to use social engineering or flaws in key handling and
286 random number generation to carry out attacks.
287 Plus modes
289 Given that the best known attacks are against AES versions with a
290 reduced number of rounds with only small round number reductions, it
291 appears that increasing the number of rounds would seem a reasonable
292 countermeasure against cryptographic attacks. (This has been inspired
293 by a comment from Bruce Schneier who the author of this document has
294 very high respect for.)
295 The C and AESNI implementations support AES128,192,256 modes with 2,3,4
297 additional rounds respectively, resulting in 12, 15, 18 rounds. These
298 modes are named AES128+, AES192+, and AES256+ (plus modes)
299 respectively. They do offer a computationally relatively cheap way to
300 enhance security. The author of this document e.g. would chose AES192+
301 over AES256. While the author of this document would never judge
302 himself as a cryptography expert strong enough to create new algorithms
303 or even devise significant changes to existing ones, he considers this
304 variation a choice that is more secure than the original. Please note
305 however, that these custom algorithms result in files that can not be
306 decrypted using any other tools. Also, the openSSL engine does not
307 support the plus modes.
308 Double modes
310 A computationally more expensive method to enhance security is doubling
311 the number of rounds. This is equivalent to encrypting twice (where the
312 second key is a simple derivation of the first). These methods are
313 supported by all engines and are named AES128x2, AES192x2, and
314 AES256x2.
315 Stream modes
317 The AES algorithm is a block cipher -- it transforms 16 byte blocks.
318 The trivial application to a file of arbitrary size is to apply this to
319 every block in the file. This is called ECB (electronic codebook) mode.
320 This is very insecure ... the same input will always result in the same
321 output. Patterns can be easily recognized and known plain text attacks
322 are trivial.
323 It's better to make the transformation dependent on the previous
325 content of the file or the position within it. This is what the CBC
326 (chained block cipher) and CTR modes do.
327 The CBC mode has several disadvantages: It can't be parallelized (every
329 block depends on all previous blocks for encryption; things are better
330 for decryption) and random access is impossible.
331 The CTR mode has many desirable properties. It is basically a stream of
333 (reproducible) pseudo random numbers that are XORed with the input for
334 encryption. Decryption is just another XOR of course. It's a one time
335 pad -- which has been proven to be secure, if the pad is unknown to an
336 attacker and only used once.
337 The latter can't be stressed enough: Don't ever use the same key/IV
338 combination for two files. Mathematically spoken: c1 = r1 XOR p1 and c2
339 = r2 XOR p2 (c = ciphertext, r = AES random numbers, p = plaintext).
340 With r == r1 == r2, it can be trivially seen that the attacker can
341 calculate c1 XOR c2 = r XOR p1 XOR r XOR p2 = p1 XOR p2. If the
342 plaintext of one of the files is partially known, so is the other.
343 The CTR mode has more nice properties: It allows random access as the
345 AES random numbers (belonging to a key/IV combination) with a certain
346 offset can be directly calculated and the last block does not require
347 padding, as partial blocks can be processed.
348 The author of this documents prefers CTR stream mode and ensures that
350 keys/IVs are not reused.
351
353 With CTR mode, you can do partial writes to encrypted files and the
354 result will still be a consistent file (of course assuming that the
355 used key and IV are the same). Same with appending (-x) or reverse
356 direction copies.
357 With ECB mode, this will only work, if file size and offsets are all
358 block (16byte) aligned. With CBC, none of this is possible.
359 The ddr_crypt plugin has no specific support for encoding holes; if
361 however previous correctly encrypted content is present in a hole, the
362 support for partial writes in CTR and ECB mode will result in a
363 meaningful output. If no previous content is in holes, then the result
364 of decrypting zeros will result upon decryption.
365 You can pass the option skiphole to make ddr_crypt leave 512byte blocks
366 of zeros untouched. This will reveal blocks of zeros and may thus
367 disclose valuable information to an attacker, so use with care. Also
368 note that you need to use this with en- and decryption and with the
369 same alignment (mod 512) for encryption to be reversible. You have been
370 warned. (You don't need to be worried about misdetecting zeros on
371 decrypting -- the chances of non-zero plaintext resulting in an aligned
372 512byte block of zeros is smaller than 2^-4096. So this option is safe
373 on decrypting -- if some of the ciphertext has been overwritten with
374 blocks of zeros, you might even prefer to have zeros in the decrypted
375 file rather than random gibberish.)
376 Note that you can compress and encode holes with ddr_lzo and then pass
378 to ddr_crypt to encrypt and pass through ddr_crypt to decrypt and
379 ddr_lzo to uncompress and extract holes again. This only works with CTR
380 mode.
381 The option weakrnd is provided for testing in environments, where
383 strong random numbers are not available. It will cause weaker random
384 numbers to be used for key generation. Don't use it if you want
385 security.
386
388 Files that are encrypted with openssl enc where you specify the key
389 (with -K) and the IV (with -iv) result in the same output that
390 ddr_crypt generates for -ECB and -CBC modes. ddr_crypt uses a 64bit
391 counter in -CTR modes.
392 With the option opbkdf ddr_crypt also reads/writes the openSSL Salted__
394 header to be compatible with openssl. This function needs more testing
395 and better error handling though.
396
398 Maturity
399 The plugin is new as of dd_rescue 1.98. Do not yet rely on data saved
400 with ddr_crypt as the only backup for valuable data. Also expect some
401 changes to ddr_crypt in the not too distant future.
402 Due to an issue in ddr_crypt's initialization of the IV for CTR mode,
404 the last 32bits would always be zeroed out prior to adding the counter.
405 This has been fixed in 1.99. It order to be compatible with 1.98, the
406 option ctrbug198 can be specified on the command line.
407 Security
409 While care has been applied to check the result of memory allocations
410 ..., the code has not been audited and only limited fuzzing has been
411 applied to ensure it's not vulnerable to malicious data -- be careful
412 when you process data from untrusted sources.
413 Key handling is a tricky business -- the author may have screwed up
414 resulting in some ways to use this program to encrypt data may not
415 result in the level of secrecy that is desired.
416 Testing
418 The crypt plugin does not yet have the same test coverage as the other
419 plugins, which means it has not been tested as intensively as the
420 others.
421 Future work
423 Except for more testing and auditing a few more features are envisioned
424 for this plugin:
425 Support for other (non-AES) algorithms such as twofish (and possibly
426 also threefish).
427 Stronger function to derive keys/IVs from passwords than pbkdf2.
428 Support for other streaming modes (XTS, GCM, ...)
429
431 dd_rescue -ptAL crypt=algo=AES256-CTR:enc:keygen:ivgen:keysfile:ivsfile infile outfile
432 encrypts data from infile with AES256 in CTR mode using a
433 generated (random) key and IV and writes the result to outfile
434 It adds a line to KEYS.AES256-CTR and to IVS.AES256-CTR where
435 the used key and IV are written to respectively. (Please ensure
436 that this file is not accessible by any unauthorized person!)
437 Decryption can be performed by
438 dd_rescue -ptAL crypt=algo=AES256-CTR:dec:keysfile:ivsfile outfile infile
440 dd_rescue -AL crypt=AES192+-CTR:enc:saltgen:saltxattr:sxfallback:passfd=0:pbkdf2 infile outfile
442 will ask for a password, generate a random salt and store it in
443 the extended attribute of outfile (and fallback to
444 SALTS.AES192+-CTR index file) and uses pbkdf2 function to
445 produce a key and IV for encrypting the data. For decrypting,
446 just omit the saltgen parameter.
447 dd_rescue -ptL lzo=compr,crypt=AES256-CTR:enc:keygen:ivgen:keysfile:ivsfile infile outfile
449 will compress the data (using lzo) and then encrypt. Use the
450 reverse order and omit keygen and ivgen to decrypt and
451 uncompress. Compression has the nice side effect of dealing with
452 holes, which otherwise get compressed to non-zero values (unless
453 you specify skiphole). Feel free to add the hash plugin at the
454 beginning and/or the end to produce cryptographic checksums for
455 both the original file and the end result.
456
458 dd_rescue(1)
459
461 Kurt Garloff <kurt@garloff.de>
462
464 The x86 AESNI optimized AES implementation has been inspired by an
465 intel whitepaper from 2009:
466 https://software.intel.com/sites/default/files/article/165683/aes-
467 wp-2012-09-22-v01.pdf
468 The ARMv8 AES support has been inspired by studying openSSL assembly as
469 well as Linaro's in-kernel implementation.
470
472 This plugin is under the same license as dd_rescue: The GNU General
473 Public License (GPL) v2 or v3 - at your option.
474
476 ddr_crypt plugin was first introduced with dd_rescue 1.98 (May 2015).
477 Version 1.99 brought support for ARMv8 crypto acceleration and support
478 for openssl style key derivation and Salted__ headers. It also added
479 storing pbkdf related infos in xattrs and added support for storing and
480 retrieving keys (not recommended!) and IVs in/from xattrs. A bug with
481 CTR initialization was resolved (see ctrbug198 option).
482 Some additional information can be found on
484 http://garloff.de/kurt/linux/ddrescue/
485Kurt Garloff 2015-04-15 ddr_crypt(1)