1HASHDEEP(1)                 United States Air Force                HASHDEEP(1)
2
3
4

NAME

6       hashdeep - Compute, compare, or audit multiple message digests
7
8

SYNOPSIS

10       hashdeep -V | -h
11       hashdeep  [-c  <alg1>[,<alg2>]] [-k <file>] [-i <size>] [-f <file>] [-o
12       <fbcplsde>] [-amxwMXreEspblvv] [-F<bum>] [-j <num>] [FILES]
13
14
15

DESCRIPTION

17       Computes multiple hashes, or message digests, for any number  of  files
18       while  optionally  recursively digging through the directory structure.
19       By default the program computes MD5 and SHA-256 hashes,  equivalent  to
20       -c  md5,sha256.   Can  also take a list of known hashes and display the
21       filenames of input files whose hashes either do or do not match any  of
22       the  known  hashes.  Can also use a list of known hashes to audit a set
23       of FILES.  Errors are reported to standard error. If no FILES are spec‐
24       ified, reads from standard input.
25
26
27
28
29       -c <alg1>[,<alg2>...]
30              Computation  mode.  Compute hashes of FILES using the algorithms
31              specified. Legal  values  are  md5,  sha1,  sha256,  tiger,  and
32              whirlpool.
33
34
35
36       -k     Load  a  file of known hashes.  This flag is required when using
37              any of the matching or audit modes (i.e. -m, -x, -M, -X, or  -a)
38              This  flag  may  be  used more than once to add multiple sets of
39              known hashes.
40
41              Loading sets with different hash algorithms can sometimes gener‐
42              ate  spurrious  hash  collisions. For example, let's say we have
43              two hash sets, A and B, which have some overlapping  files.  For
44              example,  the  file  /usr/bin/bad  is  in  both sets. In A we've
45              recorded the MD5 and SHA-256.  In  B  we've  recorded  the  MD5,
46              SHA-1,  and  SHA-256.  Because  these two records are different,
47              they will both be loaded. When the program  computes  all  three
48              hashes  and  compares  them to the set of knowns, we will get an
49              exact match from the record in B and a collision from the record
50              in A.
51
52
53
54       -a     Audit  mode.  Each  input  file  is  compared against the set of
55              knowns.  An audit is said to pass if each input file is  matched
56              against  exactly  one file in set of knowns. Any collisions, new
57              files, or missing files will make the  audit  fail.  Using  this
58              flag  alone  produces a message, either "Audit passed" or "Audit
59              Failed". Use the verbose modes, -v, for more details.  Using  -v
60              prints  the  number of files in each category. Using -v a second
61              time prints any discrepancies. Using -v a third time prints  the
62              results for every file examined and every known file.
63              Due  to  limitations  in the program, any filenames with Unicode
64              characters will appear to have moved during an  audit.  See  the
65              section "UNICODE SUPPORT" below.
66
67
68       -m     Positive  matching,  requires  at  least one use of the -k flag.
69              The input files are examined one at a time, and only those files
70              that match the list of known hashes are output. The only accept‐
71              able format for known hashes is the output of previous  hashdeep
72              runs.
73               If standard input is used with the -m flag, displays "stdin" if
74              the input matches one of the hashes in the list of known hashes.
75              If the hash does not match, the program displays no output.
76               This flag may not be used in conjunction with the -x, -X, or -a
77              flags.  See the section "UNICODE SUPPORT" below.
78
79
80       -x     Negative matching.  Same as the -m flag above, but does negative
81              matching.  That  is,  only  those files NOT in the list of known
82              hashes are displayed.
83               This flag may not be used in conjunction with the -m, -M, or -a
84              flags.  See the section "UNICODE SUPPORT" below.
85
86
87       -f <file>
88              Takes a list of files to be hashed from the specified file. Each
89              line is assumed to be a filename. This flag  can  only  be  used
90              once  per  invocation.  If  it's  used a second time, the second
91              instance will clobber the first.
92              Note that you can still use other flags, such as the  -m  or  -x
93              modes, and submit additional FILES on the command line.
94
95
96       -w     When  used  with  positive  matching  modes (-m,-M) displays the
97              filename of the known hash that matched the input file.  See the
98              section "UNICODE SUPPORT" below.
99
100
101       -M and -X
102              Same  as  -m  and  -x above, but displays the hash for each file
103              that does (or does not) match the list of known hashes.
104
105
106
107       -r     Enables recursive mode. All subdirectories are traversed. Please
108              note  that recursive mode cannot be used to examine all files of
109              a given file extension. For example, calling hashdeep  -r  *.txt
110              will examine all files in directories that end in .txt.
111
112
113
114       -e     Displays a progress indicator and estimate of time remaining for
115              each file being processed. Time estimates for files larger  than
116              4GB are not available on Windows. This mode may not be used with
117              th -p mode.
118
119
120       -E     When in audit mode, performs case insensitive matching of  file‐
121              names.   For  example, \foo\bar will match to \Foo\BAR. This can
122              be important on Windows systems, where filenames are case insen‐
123              sitive.
124
125
126       -i <size>
127              Size  threshold mode. Only hash files smaller than the given the
128              threshold.  Sizes  may  be  specified  using   IEC   multipliers
129              b,k,m,g,t,p, and e.
130
131
132
133       -o <bcpflsd>
134              Enables  expert  mode.  Allows  the user specify which (and only
135              which) types of files are  processed.  Directory  processing  is
136              still  controlled  with  the  -r  flag.  The expert mode options
137              allowed are:
138              f - Regular files
139              b - Block Devices
140              c - Character Devices
141              p - Named Pipes
142              l - Symbolic Links
143              s - Sockets
144              d - Solaris Doors
145              e - Windows PE executables
146
147
148
149       -s     Enables silent mode. All error messages are supressed.
150
151
152
153       -p     Piecewise mode. Breaks files into chunks before hashing.  Chunks
154              may  be  specified  using  IEC  multipliers  b,k,m,g,t,p, and e.
155              (Never let it be said that the author didn’t plan ahead.)
156
157
158
159       -b     Enables bare mode. Strips any leading directory information from
160              displayed  filenames.   This flag may not be used in conjunction
161              with the -l flag.
162
163
164       -l     Enables relative file paths. Instead of  printing  the  absolute
165              path for each file, displays the relative file path as indicated
166              on the command line. This flag may not be  used  in  conjunction
167              with the -b flag.
168
169
170       -v     Enables  verbose  mode.  Use again to make the program more ver‐
171              bose.  This mostly changes the behvaior of the audit mode, -a.
172
173
174       -jnn   Controls multi-threading. By default the program will create one
175              producer  thread  to scan the file system and one hashing thread
176              per CPU core. Multi-threading causes output filenames to  be  in
177              non-deterministic  order, as files that take longer to hash will
178              be delayed while they are hashed. If a  deterministic  order  is
179              required, specify -j0 to disable multi-threading
180
181
182       -d     Output in Digital Forensics XML (DFXML) format.
183
184
185       -u     Quote  Unicode  output.  For  example,  the  snowman is shown as
186              U+C426.
187
188
189       -F<bum>
190              Specifies the input mode that is used to read files. The default
191              is -Fb (buffered I/O) which reads files with fopen(). Specifying
192              -Fu will use unbuffered I/O and read the file with open(). Spec‐
193              ifying  -Fm  will  use memory-mapped I/O which will be faster on
194              some platforms, but which (currently) will not work  with  files
195              that produce I/O errors.
196
197
198
199
200       -h     Show a help screen and exit.
201
202
203       -V     Show the version number and exit.
204
205
206
207

UNICODE SUPPORT

209       As  of version 3.0 the program supports Unicode characters in filenames
210       on Microsoft Windows systems for filenames  specified  on  the  command
211       line  with  globbing (e.g. *), for files specified with the -f of files
212       to hash, and for files read from directories using the -r option.
213
214       By default all program input and output should be in UTF-8.   The  pro‐
215       gram automatically converts this to UTF-16 for opening files).
216
217       On  Unix/Linux/MacOS,  you should use a terminal emulator that supports
218       UTF-8 and UTF-8 characters in filenames will be properly displayed.
219
220       On Windows, the programs do not display Unicode characters on the  con‐
221       sole.  You must either redirect output to a file and open the file with
222       Wordpad (which can display Unicode), or you must specify the -u  option
223       to quote Unicode using standard U+XXXX notation.
224
225       Currently  the  file  name of a file containing known hashes may not be
226       specified as a unicode filename, but you can specify the name using tab
227       completition  or an asterisk (e.g. md5deep -m *.txt where there is only
228       one file with a .txt extension).
229
230

RETURN VALUE

232       Returns a bit-wise value based on the success of the operation and  the
233       status of any matching operations.
234
235       0      Success.  Note that the program considers itself successful even
236              when it encounters read errors,  permission  denied  errors,  or
237              finds directories when not in recursive mode.
238
239       1      Unused  hashes.  Under  any  of the matching modes, returns this
240              value if one or more of the known hashes was not matched by  any
241              of the input files.
242
243       2      Unmatched  inputs. Under any of the matching modes, returns this
244              value if one or more of the input values did not  match  any  of
245              the known hashes.
246
247       64     User  error,  such  as  trying  to do both positive and negative
248              matching at the same time.
249
250       128    Internal error, such as memory  corruption  or  uncaught  cycle.
251              All internal errors should be reported to the developer! See the
252              section "Reporting Bugs" below.
253
254
255

AUTHOR

257       hashdeep was written by Jesse Kornblum, research@jessekornblum.com, and
258       Simson Garfinkel.
259
260

KNOWN ISSUES

262       Using  the -r flag cannot be used to recursively process all files of a
263       given extension in a directory. This is a feature, not a bug.   If  you
264       need to do this, use the find(1) command.
265
266       The  program  will  fail  if  you attempt to compare 2^64 or more input
267       files against a set of known files.
268
269
270

REPORTING BUGS

272       We take all bug reports very seriously. Any bug  that  jeopardizes  the
273       forensic  integrity  of this program could have serious consequences on
274       people's lives. When submitting a bug report, please include a descrip‐
275       tion of the problem, how you found it, and your contact information.
276
277       Send bug reports to the author at the address above.
278
279
281       This  program is a work of the US Government. In accordance with 17 USC
282       105, copyright protection is not available for any work of the US  Gov‐
283       ernment.   This program is PUBLIC DOMAIN. Portions of this program con‐
284       tain code that is licensed  under  the  terms  of  the  General  Public
285       License  (GPL).   Those  portions  retain  their original copyright and
286       license. See the file COPYING for more details.
287
288       There is NO warranty for this program; not even for MERCHANTABILITY  or
289       FITNESS FOR A PARTICULAR PURPOSE.
290
291

SEE ALSO

293       More  information  and  installation  instructions  can be found in the
294       README file. Current versions of both documents can  be  found  on  the
295       project homepage: http://md5deep.sourceforge.net/
296
297       The MD5 specification, RFC 1321, is available at
298       http://www.ietf.org/rfc/rfc1321.txt
299
300       The SHA-1 specification, RFC 3174, is available at
301       http://www.faqs.org/rfcs/rfc3174.html
302
303       The SHA-256 specification, FIPS 180-2, is available at
304       http://csrc.nist.gov/publications/fips/fips180-2/fips180-2.pdf
305
306       The Tiger specification is available at
307       http://www.cs.technion.ac.il/~biham/Reports/Tiger/
308
309       The Whirlpool specification is available at
310       http://planeta.terra.com.br/informatica/paulobarreto/WhirlpoolPage.html
311
312
313
314AFOSI                         v4.4 - 29 Jan 2014                   HASHDEEP(1)
Impressum