1MD5DEEP(1)                  United States Air Force                 MD5DEEP(1)
2
3
4

NAME

6       md5deep - Compute and compare MD5 message digests
7       sha1deep - Compute and compare SHA-1 message digests
8       sha256deep - Compute and compare SHA-256 message digests
9       sha3deep - Compute and compare SHA-3-256 message digests
10       tigerdeep - Compute and compare Tiger message digests
11       whirlpooldeep - Compute and compare Whirlpool message digests
12
13

SYNOPSIS

15       md5deep -v | -V | -h
16       md5deep  [-m|-M|-x|-X  <file>]   [-a|-A <hash>] [-f <file>] [-p <size>]
17       [-i <size>] [-tnwzresS0lbkqZud] [-F <bum>] [-o <fbcplsde>]  [-j  <num>]
18       [[FILES]
19
20

DESCRIPTION

22       Computes  the  hashes, or message digest, for any number of files while
23       optionally recursively digging through the  directory  structure.   Can
24       also  take  a  list  of known hashes and display the filenames of input
25       files whose hashes either do or do not match any of the  known  hashes.
26       Errors are reported to standard error. If no FILES are specified, reads
27       from standard input.
28
29
30       -p <size>
31              Piecewise mode. Breaks files into chunks before hashing.  Chunks
32              may  be  specified using IEC multipliers b, k, m, g, t, p, or e.
33              (Never let it be said that the author didn't plan ahead!)   This
34              mode cannot be used with the -z mode.
35
36
37       -i|-I <size>
38              Size  threshold mode. Only hash files smaller than the given the
39              threshold. In -i mode, simply omits those files larger than  the
40              threshold.  In  -I  mode, displays all files, but uses asterisks
41              for the hashes of files larger than the threshold.  Sizes may be
42              specified using IEC multipliers b, k, m, g, t, p, or e.
43
44
45       -r     Enables recursive mode. All subdirectories are traversed. Please
46              note that recursive mode cannot be used to examine all files  of
47              a  given  file  extension. For example, calling md5deep -r *.txt
48              will examine all files in directories that end in .txt.
49
50
51       -e     Displays a progress indicator and estimate of time remaining for
52              each  file being processed. Time estimates for files larger than
53              4GB are not available on Windows. This mode may not be used with
54              th -p mode.
55
56
57       -m <file>
58              Enables  matching mode. The file given should be a list of known
59              hashes.  The input files are examined one at a  time,  and  only
60              those files that match the list of known hashes are output. This
61              flag may be used more than once to add multiple  sets  of  known
62              hashes.  Acceptable  formats for lists of known hashes are plain
63              (such as those  generated  by  md5deep  or  md5sum),  Hashkeeper
64              files, iLook, and the National Software Reference Library (NSRL)
65              as produced by the National Institute for Standards in  Technol‐
66              ogy.
67               If standard input is used with the -m flag, displays "stdin" if
68              the input matches one of the hashes in the list of known hashes.
69              If the hash does not match, the program displays no output.
70               This flag may not be used in conjunction with the -x, -X, or -A
71              flags.  See the section "UNICODE SUPPORT" below.
72
73
74       -x <file>
75              Same as the -m flag above, but does negative matching. That  is,
76              only those files NOT in the list of known hashes are displayed.
77               This flag may not be used in conjunction with the -m, -M, or -a
78              flags.  See the section "UNICODE SUPPORT" below.
79
80       -M and -X <file>
81              Same as -m and -x above, but displays the  hash  for  each  file
82              that does (or does not) match the list of known hashes.
83
84
85       -a <hash>
86              Adds a single hash to the list of known hashes used for matching
87              mode, and if not already enabled, enables matching mode.  Adding
88              single  hashes cannot, by itself, be used to print the hashes of
89              matching files like the -M flag does. When used  in  conjunction
90              with  the  -w flag, the filename displayed is just the hash sub‐
91              mitted on the command line.
92               This flag may not be used in conjunction with the -x, -X, or -A
93              flags.
94
95
96       -A <hash>
97              Same as -a above, but does negative matching.  This flag may not
98              be used in conjunction with the -m, -M, or -A flags.
99
100
101       -f <file>
102              Takes a list of files to be hashed from the specified file. Each
103              line  is  assumed  to  be a filename. This flag can only be used
104              once per invocation. If it's used  a  second  time,  the  second
105              instance will clobber the first.
106              Note  that  you  can still use other flags, such as the -m or -x
107              modes, and submit additional FILES on the command line.
108
109
110       -w     During any of the matching modes (-m,-M,-x,or -X), displays  the
111              filename of the known hash that matched the input file.  See the
112              section "UNICODE SUPPORT" below.
113
114
115       -t     Display a timestamp in GMT with each  result.  On  Windows  this
116              timestamp will be the file's creation time. On all other systems
117              it should be the file's change time.
118
119
120       -n     During any of the matching modes (-m,-M,-x,or -X), displays only
121              the  filenames  of any known hashes that were not matched by any
122              of the input files.
123
124
125       -s     Enables silent mode. All error messages are supressed.
126
127
128       -S     Like silent mode, but still displays warnings on improperly for‐
129              matted hashes in the list of known hashes.
130
131
132       -z     Enables  file size mode. Prepends the hash with a ten digit rep‐
133              resentation of the size of each file processed. If the file size
134              is  greater than 9999999999 bytes (about 9.3GB) the program dis‐
135              plays 9999999999 for the size.
136
137
138       -q     Quiet mode. File names are omitted from the output. Each hash is
139              still followed by two spaces before the newline.
140
141
142       -Z     Produces  output  in Triage format. Each line contans the file's
143              size, a tab, a hash of the first 512 bytes, a tab, the  hash  of
144              the  complete  file, a tab, and the file name.  These values are
145              intended in increasing order of specificity. That is, two  files
146              with  different sizes cannot possibly match. This is a fast com‐
147              parison and should be done first. Next, two files with different
148              partial  hashes cannot possibly match. This is often faster than
149              hashing the whole file. Finally, if those two pieces align, then
150              it's worth reading and hashing the entire file.
151
152
153       -0     Uses  a  NULL character (/0) to terminate each line instead of a
154              newline.  Useful for processing filenames with  strange  charac‐
155              ters.
156
157
158       -l     Enables  relative  file  paths. Instead of printing the absolute
159              path for each file, displays the relative file path as indicated
160              on  the  command  line. This flag may not be used in conjunction
161              with the -b flag.
162
163
164       -b     Enables bare mode. Strips any leading directory information from
165              displayed  filenames.   This flag may not be used in conjunction
166              with the -l flag.
167
168
169       -k     Enables asterisk mode. An asterisk is inserted in lieu of a sec‐
170              ond space between the filename and the hash, just like md5sum in
171              its binary (-b) mode.
172
173
174       -c     Enables comma separated values output, or CSV  mode.  This  mode
175              has  the  side  effect  of removing the 10 digit size limitation
176              from -z mode.  Also note that asterisks from  -k  mode  are  not
177              displayed when in CSV mode.
178
179
180       -o <bcpflsd>
181              Enables  expert  mode.  Allows  the user specify which (and only
182              which) types of files are  processed.  Directory  processing  is
183              still  controlled  with  the  -r  flag.  The expert mode options
184              allowed are:
185              f - Regular files
186              b - Block Devices
187              c - Character Devices
188              p - Named Pipes
189              l - Symbolic Links
190              s - Sockets
191              d - Solaris Doors
192              e - Windows PE executables
193
194
195       -jnn   Controls multi-threading. By default the program will create one
196              producer  thread  to scan the file system and one hashing thread
197              per CPU core. Multi-threading causes output filenames to  be  in
198              non-deterministic  order, as files that take longer to hash will
199              be delayed while they are hashed. If a  deterministic  order  is
200              required, specify -j0 to disable multi-threading
201
202
203       -d     Output in Digital Forensics XML (DFXML) format.
204
205
206       -u     Quote  Unicode  output.  For  example,  the  snowman is shown as
207              U+C426.
208
209
210       -F<bum>
211              Specifies the input mode that is used to read files. The default
212              is -Fb (buffered I/O) which reads files with fopen(). Specifying
213              -Fu will use unbuffered I/O and read the file with open(). Spec‐
214              ifying  -Fm  will  use memory-mapped I/O which will be faster on
215              some platforms, but which (currently) will not work  with  files
216              that produce I/O errors.
217
218
219       -h     Show a help screen and exit.
220
221
222       -v     Show the version number and exit.
223
224
225       -V     Show copyright information and exit.
226
227

UNICODE SUPPORT

229       As  of version 3.0 the program supports Unicode characters in filenames
230       on Microsoft Windows systems for filenames  specified  on  the  command
231       line  with  globbing (e.g. *), for files specified with the -f of files
232       to hash, and for files read from directories using the -r option.
233
234       By default all program input and output should be in UTF-8.   The  pro‐
235       gram automatically converts this to UTF-16 for opening files).
236
237       On  Unix/Linux/MacOS,  you should use a terminal emulator that supports
238       UTF-8 and UTF-8 characters in filenames will be properly displayed.
239
240       On Windows, the programs do not display Unicode characters on the  con‐
241       sole.  You must either redirect output to a file and open the file with
242       Wordpad (which can display Unicode), or you must specify the -u  option
243       to quote Unicode using standard U+XXXX notation.
244
245       Currently  the  file  name of a file containing known hashes may not be
246       specified as a unicode filename, but you can specify the name using tab
247       completition  or an asterisk (e.g. md5deep -m *.txt where there is only
248       one file with a .txt extension).
249
250

RETURN VALUE

252       Returns a bit-wise value based on the success of the operation and  the
253       status of any matching operations.
254
255       0      Success.  Note that the program considers itself successful even
256              when it encounters read errors,  permission  denied  errors,  or
257              finds directories when not in recursive mode.
258
259       1      Unused  hashes.  Under  any  of the matching modes, returns this
260              value if one or more of the known hashes was not matched by  any
261              of the input files.
262
263       2      Unmatched  inputs. Under any of the matching modes, returns this
264              value if one or more of the input values did not  match  any  of
265              the known hashes.
266
267       64     User  error,  such  as  trying  to do both positive and negative
268              matching at the same time.
269
270       128    Internal error, such as memory  corruption  or  uncaught  cycle.
271              All internal errors should be reported to the developer! See the
272              section "Reporting Bugs" below.
273
274
275

AUTHOR

277       md5deep was written by Jesse Kornblum,  research@jessekornblum.com  and
278       Simson Garfinkel.
279
280

KNOWN ISSUES

282       Using  the -r flag cannot be used to recursively process all files of a
283       given extension in a directory. This is a feature, not a bug.   If  you
284       need to do this, use the find(1) command.
285
286

REPORTING BUGS

288       We  take  all  bug reports very seriously. Any bug that jeopardizes the
289       forensic integrity of this program could have serious  consequences  on
290       people's lives. When submitting a bug report, please include a descrip‐
291       tion of the problem, how you found it, and your contact information.
292
293       Send bug reports to the author at the address above.
294
295
297       This program is a work of the US Government. In accordance with 17  USC
298       105,  copyright protection is not available for any work of the US Gov‐
299       ernment.  This program is PUBLIC DOMAIN. Portions of this program  con‐
300       tain  code  that  is  licensed  under  the  terms of the General Public
301       License (GPL).  Those portions  retain  their  original  copyright  and
302       license. See the file COPYING for more details.
303
304       There  is NO warranty for this program; not even for MERCHANTABILITY or
305       FITNESS FOR A PARTICULAR PURPOSE.
306
307

SEE ALSO

309       More information and installation instructions  can  be  found  in  the
310       README  file.  Current  versions  of both documents can be found on the
311       project homepage: http://md5deep.sourceforge.net/
312
313       The MD5 specification, RFC 1321, is available at
314       http://www.ietf.org/rfc/rfc1321.txt
315
316       The SHA-1 specification, RFC 3174, is available at
317       http://www.faqs.org/rfcs/rfc3174.html
318
319       The SHA-256 specification, FIPS 180-2, is available at
320       http://csrc.nist.gov/publications/fips/fips180-2/fips180-2.pdf
321
322       The SHA-3-256 specification is available at
323       http://keccak.noekeon.org/
324
325       The Tiger specification is available at
326       http://www.cs.technion.ac.il/~biham/Reports/Tiger/
327
328       The Whirlpool specification is available at
329       http://planeta.terra.com.br/informatica/paulobarreto/WhirlpoolPage.html
330
331
332
333AFOSI                         v4.4 - 29 Jan 2014                    MD5DEEP(1)
Impressum