1MD5DEEP(1) United States Air Force MD5DEEP(1)
2
6 md5deep - Compute and compare MD5 message digests
7 sha1deep - Compute and compare SHA-1 message digests
8 sha256deep - Compute and compare SHA-256 message digests
9 sha3deep - Compute and compare SHA-3-256 message digests
10 tigerdeep - Compute and compare Tiger message digests
11 whirlpooldeep - Compute and compare Whirlpool message digests
12
15 md5deep -v | -V | -h
16 md5deep [-m|-M|-x|-X <file>] [-a|-A <hash>] [-f <file>] [-p <size>]
17 [-i <size>] [-tnwzresS0lbkqZud] [-F <bum>] [-o <fbcplsde>] [-j <num>]
18 [[FILES]
19
22 Computes the hashes, or message digest, for any number of files while
23 optionally recursively digging through the directory structure. Can
24 also take a list of known hashes and display the filenames of input
25 files whose hashes either do or do not match any of the known hashes.
26 Errors are reported to standard error. If no FILES are specified, reads
27 from standard input.
28 -p <size>
31 Piecewise mode. Breaks files into chunks before hashing. Chunks
32 may be specified using IEC multipliers b, k, m, g, t, p, or e.
33 (Never let it be said that the author didn't plan ahead!) This
34 mode cannot be used with the -z mode.
35 -i|-I <size>
38 Size threshold mode. Only hash files smaller than the given the
39 threshold. In -i mode, simply omits those files larger than the
40 threshold. In -I mode, displays all files, but uses asterisks
41 for the hashes of files larger than the threshold. Sizes may be
42 specified using IEC multipliers b, k, m, g, t, p, or e.
43 -r Enables recursive mode. All subdirectories are traversed. Please
46 note that recursive mode cannot be used to examine all files of
47 a given file extension. For example, calling md5deep -r *.txt
48 will examine all files in directories that end in .txt.
49 -e Displays a progress indicator and estimate of time remaining for
52 each file being processed. Time estimates for files larger than
53 4GB are not available on Windows. This mode may not be used with
54 th -p mode.
55 -m <file>
58 Enables matching mode. The file given should be a list of known
59 hashes. The input files are examined one at a time, and only
60 those files that match the list of known hashes are output. This
61 flag may be used more than once to add multiple sets of known
62 hashes. Acceptable formats for lists of known hashes are plain
63 (such as those generated by md5deep or md5sum), Hashkeeper
64 files, iLook, and the National Software Reference Library (NSRL)
65 as produced by the National Institute for Standards in Technol‐
66 ogy.
67 If standard input is used with the -m flag, displays "stdin" if
68 the input matches one of the hashes in the list of known hashes.
69 If the hash does not match, the program displays no output.
70 This flag may not be used in conjunction with the -x, -X, or -A
71 flags. See the section "UNICODE SUPPORT" below.
72 -x <file>
75 Same as the -m flag above, but does negative matching. That is,
76 only those files NOT in the list of known hashes are displayed.
77 This flag may not be used in conjunction with the -m, -M, or -a
78 flags. See the section "UNICODE SUPPORT" below.
79 -M and -X <file>
81 Same as -m and -x above, but displays the hash for each file
82 that does (or does not) match the list of known hashes.
83 -a <hash>
86 Adds a single hash to the list of known hashes used for matching
87 mode, and if not already enabled, enables matching mode. Adding
88 single hashes cannot, by itself, be used to print the hashes of
89 matching files like the -M flag does. When used in conjunction
90 with the -w flag, the filename displayed is just the hash sub‐
91 mitted on the command line.
92 This flag may not be used in conjunction with the -x, -X, or -A
93 flags.
94 -A <hash>
97 Same as -a above, but does negative matching. This flag may not
98 be used in conjunction with the -m, -M, or -A flags.
99 -f <file>
102 Takes a list of files to be hashed from the specified file. Each
103 line is assumed to be a filename. This flag can only be used
104 once per invocation. If it's used a second time, the second
105 instance will clobber the first.
106 Note that you can still use other flags, such as the -m or -x
107 modes, and submit additional FILES on the command line.
108 -w During any of the matching modes (-m,-M,-x,or -X), displays the
111 filename of the known hash that matched the input file. See the
112 section "UNICODE SUPPORT" below.
113 -t Display a timestamp in GMT with each result. On Windows this
116 timestamp will be the file's creation time. On all other systems
117 it should be the file's change time.
118 -n During any of the matching modes (-m,-M,-x,or -X), displays only
121 the filenames of any known hashes that were not matched by any
122 of the input files.
123 -s Enables silent mode. All error messages are supressed.
126 -S Like silent mode, but still displays warnings on improperly for‐
129 matted hashes in the list of known hashes.
130 -z Enables file size mode. Prepends the hash with a ten digit rep‐
133 resentation of the size of each file processed. If the file size
134 is greater than 9999999999 bytes (about 9.3GB) the program dis‐
135 plays 9999999999 for the size.
136 -q Quiet mode. File names are omitted from the output. Each hash is
139 still followed by two spaces before the newline.
140 -Z Produces output in Triage format. Each line contans the file's
143 size, a tab, a hash of the first 512 bytes, a tab, the hash of
144 the complete file, a tab, and the file name. These values are
145 intended in increasing order of specificity. That is, two files
146 with different sizes cannot possibly match. This is a fast com‐
147 parison and should be done first. Next, two files with different
148 partial hashes cannot possibly match. This is often faster than
149 hashing the whole file. Finally, if those two pieces align, then
150 it's worth reading and hashing the entire file.
151 -0 Uses a NULL character (/0) to terminate each line instead of a
154 newline. Useful for processing filenames with strange charac‐
155 ters.
156 -l Enables relative file paths. Instead of printing the absolute
159 path for each file, displays the relative file path as indicated
160 on the command line. This flag may not be used in conjunction
161 with the -b flag.
162 -b Enables bare mode. Strips any leading directory information from
165 displayed filenames. This flag may not be used in conjunction
166 with the -l flag.
167 -k Enables asterisk mode. An asterisk is inserted in lieu of a sec‐
170 ond space between the filename and the hash, just like md5sum in
171 its binary (-b) mode.
172 -c Enables comma separated values output, or CSV mode. This mode
175 has the side effect of removing the 10 digit size limitation
176 from -z mode. Also note that asterisks from -k mode are not
177 displayed when in CSV mode.
178 -o <bcpflsd>
181 Enables expert mode. Allows the user specify which (and only
182 which) types of files are processed. Directory processing is
183 still controlled with the -r flag. The expert mode options
184 allowed are:
185 f - Regular files
186 b - Block Devices
187 c - Character Devices
188 p - Named Pipes
189 l - Symbolic Links
190 s - Sockets
191 d - Solaris Doors
192 e - Windows PE executables
193 -jnn Controls multi-threading. By default the program will create one
196 producer thread to scan the file system and one hashing thread
197 per CPU core. Multi-threading causes output filenames to be in
198 non-deterministic order, as files that take longer to hash will
199 be delayed while they are hashed. If a deterministic order is
200 required, specify -j0 to disable multi-threading
201 -d Output in Digital Forensics XML (DFXML) format.
204 -u Quote Unicode output. For example, the snowman is shown as
207 U+C426.
208 -F<bum>
211 Specifies the input mode that is used to read files. The default
212 is -Fb (buffered I/O) which reads files with fopen(). Specifying
213 -Fu will use unbuffered I/O and read the file with open(). Spec‐
214 ifying -Fm will use memory-mapped I/O which will be faster on
215 some platforms, but which (currently) will not work with files
216 that produce I/O errors.
217 -h Show a help screen and exit.
220 -v Show the version number and exit.
223 -V Show copyright information and exit.
226
229 As of version 3.0 the program supports Unicode characters in filenames
230 on Microsoft Windows systems for filenames specified on the command
231 line with globbing (e.g. *), for files specified with the -f of files
232 to hash, and for files read from directories using the -r option.
233 By default all program input and output should be in UTF-8. The pro‐
235 gram automatically converts this to UTF-16 for opening files).
236 On Unix/Linux/MacOS, you should use a terminal emulator that supports
238 UTF-8 and UTF-8 characters in filenames will be properly displayed.
239 On Windows, the programs do not display Unicode characters on the con‐
241 sole. You must either redirect output to a file and open the file with
242 Wordpad (which can display Unicode), or you must specify the -u option
243 to quote Unicode using standard U+XXXX notation.
244 Currently the file name of a file containing known hashes may not be
246 specified as a unicode filename, but you can specify the name using tab
247 completition or an asterisk (e.g. md5deep -m *.txt where there is only
248 one file with a .txt extension).
249
252 Returns a bit-wise value based on the success of the operation and the
253 status of any matching operations.
254 0 Success. Note that the program considers itself successful even
256 when it encounters read errors, permission denied errors, or
257 finds directories when not in recursive mode.
258 1 Unused hashes. Under any of the matching modes, returns this
260 value if one or more of the known hashes was not matched by any
261 of the input files.
262 2 Unmatched inputs. Under any of the matching modes, returns this
264 value if one or more of the input values did not match any of
265 the known hashes.
266 64 User error, such as trying to do both positive and negative
268 matching at the same time.
269 128 Internal error, such as memory corruption or uncaught cycle.
271 All internal errors should be reported to the developer! See the
272 section "Reporting Bugs" below.
273
277 md5deep was written by Jesse Kornblum, research@jessekornblum.com and
278 Simson Garfinkel.
279
282 Using the -r flag cannot be used to recursively process all files of a
283 given extension in a directory. This is a feature, not a bug. If you
284 need to do this, use the find(1) command.
285
288 We take all bug reports very seriously. Any bug that jeopardizes the
289 forensic integrity of this program could have serious consequences on
290 people's lives. When submitting a bug report, please include a descrip‐
291 tion of the problem, how you found it, and your contact information.
292 Send bug reports to the author at the address above.
294
297 This program is a work of the US Government. In accordance with 17 USC
298 105, copyright protection is not available for any work of the US Gov‐
299 ernment. This program is PUBLIC DOMAIN. Portions of this program con‐
300 tain code that is licensed under the terms of the General Public
301 License (GPL). Those portions retain their original copyright and
302 license. See the file COPYING for more details.
303 There is NO warranty for this program; not even for MERCHANTABILITY or
305 FITNESS FOR A PARTICULAR PURPOSE.
306
309 More information and installation instructions can be found in the
310 README file. Current versions of both documents can be found on the
311 project homepage: http://md5deep.sourceforge.net/
312 The MD5 specification, RFC 1321, is available at
314 http://www.ietf.org/rfc/rfc1321.txt
315 The SHA-1 specification, RFC 3174, is available at
317 http://www.faqs.org/rfcs/rfc3174.html
318 The SHA-256 specification, FIPS 180-2, is available at
320 http://csrc.nist.gov/publications/fips/fips180-2/fips180-2.pdf
321 The SHA-3-256 specification is available at
323 http://keccak.noekeon.org/
324 The Tiger specification is available at
326 http://www.cs.technion.ac.il/~biham/Reports/Tiger/
327 The Whirlpool specification is available at
329 http://planeta.terra.com.br/informatica/paulobarreto/WhirlpoolPage.html
330AFOSI v4.4 - 29 Jan 2014 MD5DEEP(1)