1HASHDEEP(1) United States Air Force HASHDEEP(1)
2
6 hashdeep - Compute, compare, or audit multiple message digests
7
10 hashdeep -V | -h
11 hashdeep [-c <alg1>[,<alg2>]] [-k <file>] [-i <size>] [-f <file>] [-o
12 <fbcplsde>] [-amxwMXreEspblvv] [-F<bum>] [-j <num>] [FILES]
13
17 Computes multiple hashes, or message digests, for any number of files
18 while optionally recursively digging through the directory structure.
19 By default the program computes MD5 and SHA-256 hashes, equivalent to
20 -c md5,sha256. Can also take a list of known hashes and display the
21 filenames of input files whose hashes either do or do not match any of
22 the known hashes. Can also use a list of known hashes to audit a set
23 of FILES. Errors are reported to standard error. If no FILES are spec‐
24 ified, reads from standard input.
25 -c <alg1>[,<alg2>...]
30 Computation mode. Compute hashes of FILES using the algorithms
31 specified. Legal values are md5, sha1, sha256, tiger, and
32 whirlpool.
33 -k Load a file of known hashes. This flag is required when using
37 any of the matching or audit modes (i.e. -m, -x, -M, -X, or -a)
38 This flag may be used more than once to add multiple sets of
39 known hashes.
40 Loading sets with different hash algorithms can sometimes gener‐
42 ate spurrious hash collisions. For example, let's say we have
43 two hash sets, A and B, which have some overlapping files. For
44 example, the file /usr/bin/bad is in both sets. In A we've
45 recorded the MD5 and SHA-256. In B we've recorded the MD5,
46 SHA-1, and SHA-256. Because these two records are different,
47 they will both be loaded. When the program computes all three
48 hashes and compares them to the set of knowns, we will get an
49 exact match from the record in B and a collision from the record
50 in A.
51 -a Audit mode. Each input file is compared against the set of
55 knowns. An audit is said to pass if each input file is matched
56 against exactly one file in set of knowns. Any collisions, new
57 files, or missing files will make the audit fail. Using this
58 flag alone produces a message, either "Audit passed" or "Audit
59 Failed". Use the verbose modes, -v, for more details. Using -v
60 prints the number of files in each category. Using -v a second
61 time prints any discrepancies. Using -v a third time prints the
62 results for every file examined and every known file.
63 Due to limitations in the program, any filenames with Unicode
64 characters will appear to have moved during an audit. See the
65 section "UNICODE SUPPORT" below.
66 -m Positive matching, requires at least one use of the -k flag.
69 The input files are examined one at a time, and only those files
70 that match the list of known hashes are output. The only accept‐
71 able format for known hashes is the output of previous hashdeep
72 runs.
73 If standard input is used with the -m flag, displays "stdin" if
74 the input matches one of the hashes in the list of known hashes.
75 If the hash does not match, the program displays no output.
76 This flag may not be used in conjunction with the -x, -X, or -a
77 flags. See the section "UNICODE SUPPORT" below.
78 -x Negative matching. Same as the -m flag above, but does negative
81 matching. That is, only those files NOT in the list of known
82 hashes are displayed.
83 This flag may not be used in conjunction with the -m, -M, or -a
84 flags. See the section "UNICODE SUPPORT" below.
85 -f <file>
88 Takes a list of files to be hashed from the specified file. Each
89 line is assumed to be a filename. This flag can only be used
90 once per invocation. If it's used a second time, the second
91 instance will clobber the first.
92 Note that you can still use other flags, such as the -m or -x
93 modes, and submit additional FILES on the command line.
94 -w When used with positive matching modes (-m,-M) displays the
97 filename of the known hash that matched the input file. See the
98 section "UNICODE SUPPORT" below.
99 -M and -X
102 Same as -m and -x above, but displays the hash for each file
103 that does (or does not) match the list of known hashes.
104 -r Enables recursive mode. All subdirectories are traversed. Please
108 note that recursive mode cannot be used to examine all files of
109 a given file extension. For example, calling hashdeep -r *.txt
110 will examine all files in directories that end in .txt.
111 -e Displays a progress indicator and estimate of time remaining for
115 each file being processed. Time estimates for files larger than
116 4GB are not available on Windows. This mode may not be used with
117 th -p mode.
118 -E When in audit mode, performs case insensitive matching of file‐
121 names. For example, \foo\bar will match to \Foo\BAR. This can
122 be important on Windows systems, where filenames are case insen‐
123 sitive.
124 -i <size>
127 Size threshold mode. Only hash files smaller than the given the
128 threshold. Sizes may be specified using IEC multipliers
129 b,k,m,g,t,p, and e.
130 -o <bcpflsd>
134 Enables expert mode. Allows the user specify which (and only
135 which) types of files are processed. Directory processing is
136 still controlled with the -r flag. The expert mode options
137 allowed are:
138 f - Regular files
139 b - Block Devices
140 c - Character Devices
141 p - Named Pipes
142 l - Symbolic Links
143 s - Sockets
144 d - Solaris Doors
145 e - Windows PE executables
146 -s Enables silent mode. All error messages are supressed.
150 -p Piecewise mode. Breaks files into chunks before hashing. Chunks
154 may be specified using IEC multipliers b,k,m,g,t,p, and e.
155 (Never let it be said that the author didn’t plan ahead.)
156 -b Enables bare mode. Strips any leading directory information from
160 displayed filenames. This flag may not be used in conjunction
161 with the -l flag.
162 -l Enables relative file paths. Instead of printing the absolute
165 path for each file, displays the relative file path as indicated
166 on the command line. This flag may not be used in conjunction
167 with the -b flag.
168 -v Enables verbose mode. Use again to make the program more ver‐
171 bose. This mostly changes the behvaior of the audit mode, -a.
172 -jnn Controls multi-threading. By default the program will create one
175 producer thread to scan the file system and one hashing thread
176 per CPU core. Multi-threading causes output filenames to be in
177 non-deterministic order, as files that take longer to hash will
178 be delayed while they are hashed. If a deterministic order is
179 required, specify -j0 to disable multi-threading
180 -d Output in Digital Forensics XML (DFXML) format.
183 -u Quote Unicode output. For example, the snowman is shown as
186 U+C426.
187 -F<bum>
190 Specifies the input mode that is used to read files. The default
191 is -Fb (buffered I/O) which reads files with fopen(). Specifying
192 -Fu will use unbuffered I/O and read the file with open(). Spec‐
193 ifying -Fm will use memory-mapped I/O which will be faster on
194 some platforms, but which (currently) will not work with files
195 that produce I/O errors.
196 -h Show a help screen and exit.
201 -V Show the version number and exit.
204
209 As of version 3.0 the program supports Unicode characters in filenames
210 on Microsoft Windows systems for filenames specified on the command
211 line with globbing (e.g. *), for files specified with the -f of files
212 to hash, and for files read from directories using the -r option.
213 By default all program input and output should be in UTF-8. The pro‐
215 gram automatically converts this to UTF-16 for opening files).
216 On Unix/Linux/MacOS, you should use a terminal emulator that supports
218 UTF-8 and UTF-8 characters in filenames will be properly displayed.
219 On Windows, the programs do not display Unicode characters on the con‐
221 sole. You must either redirect output to a file and open the file with
222 Wordpad (which can display Unicode), or you must specify the -u option
223 to quote Unicode using standard U+XXXX notation.
224 Currently the file name of a file containing known hashes may not be
226 specified as a unicode filename, but you can specify the name using tab
227 completition or an asterisk (e.g. md5deep -m *.txt where there is only
228 one file with a .txt extension).
229
232 Returns a bit-wise value based on the success of the operation and the
233 status of any matching operations.
234 0 Success. Note that the program considers itself successful even
236 when it encounters read errors, permission denied errors, or
237 finds directories when not in recursive mode.
238 1 Unused hashes. Under any of the matching modes, returns this
240 value if one or more of the known hashes was not matched by any
241 of the input files.
242 2 Unmatched inputs. Under any of the matching modes, returns this
244 value if one or more of the input values did not match any of
245 the known hashes.
246 64 User error, such as trying to do both positive and negative
248 matching at the same time.
249 128 Internal error, such as memory corruption or uncaught cycle.
251 All internal errors should be reported to the developer! See the
252 section "Reporting Bugs" below.
253
257 hashdeep was written by Jesse Kornblum, research@jessekornblum.com, and
258 Simson Garfinkel.
259
262 Using the -r flag cannot be used to recursively process all files of a
263 given extension in a directory. This is a feature, not a bug. If you
264 need to do this, use the find(1) command.
265 The program will fail if you attempt to compare 2^64 or more input
267 files against a set of known files.
268
272 We take all bug reports very seriously. Any bug that jeopardizes the
273 forensic integrity of this program could have serious consequences on
274 people's lives. When submitting a bug report, please include a descrip‐
275 tion of the problem, how you found it, and your contact information.
276 Send bug reports to the author at the address above.
278
281 This program is a work of the US Government. In accordance with 17 USC
282 105, copyright protection is not available for any work of the US Gov‐
283 ernment. This program is PUBLIC DOMAIN. Portions of this program con‐
284 tain code that is licensed under the terms of the General Public
285 License (GPL). Those portions retain their original copyright and
286 license. See the file COPYING for more details.
287 There is NO warranty for this program; not even for MERCHANTABILITY or
289 FITNESS FOR A PARTICULAR PURPOSE.
290
293 More information and installation instructions can be found in the
294 README file. Current versions of both documents can be found on the
295 project homepage: http://md5deep.sourceforge.net/
296 The MD5 specification, RFC 1321, is available at
298 http://www.ietf.org/rfc/rfc1321.txt
299 The SHA-1 specification, RFC 3174, is available at
301 http://www.faqs.org/rfcs/rfc3174.html
302 The SHA-256 specification, FIPS 180-2, is available at
304 http://csrc.nist.gov/publications/fips/fips180-2/fips180-2.pdf
305 The Tiger specification is available at
307 http://www.cs.technion.ac.il/~biham/Reports/Tiger/
308 The Whirlpool specification is available at
310 http://planeta.terra.com.br/informatica/paulobarreto/WhirlpoolPage.html
311AFOSI v4.4 - 29 Jan 2014 HASHDEEP(1)