1MU-INDEX(1) General Commands Manual MU-INDEX(1)
2
6 mu index - index e-mail messages stored in Maildirs
7
10 mu index [options]
11
14 mu index is the mu command for scanning the contents of Maildir direc‐
15 tories and storing the results in a Xapian database. The data can then
16 be queried using mu-find(1).
17 Note that before the first time you run mu index, you must run mu init
19 to initialize the database.
20 index understands Maildirs as defined by Daniel Bernstein for qmail(7).
22 In addition, it understands recursive Maildirs (Maildirs within
23 Maildirs), Maildir++. It can also deal with VFAT-based Maildirs which
24 use '!' as the separators instead of ':'.
25 E-mail messages which are not stored in something resembling a maildir
27 leaf-directory (cur and new) are ignored, as are the cache directories
28 for notmuch and gnus, and any dot-directory.
29 The maildir must be on a single file-system; symlinks are not followed.
31 If there is a file called .noindex in a directory, the contents of that
33 directory and all of its subdirectories will be ignored. This can be
34 useful to exclude certain directories from the indexing process, for
35 example directories with spam-messages.
36 If there is a file called .noupdate in a directory, the contents of
38 that directory and all of its subdirectories will be ignored, unless we
39 do a full rebuild (with mu init). This can be useful to speed up things
40 you have some maildirs that never change. Note that you can still
41 search for these messages, this only affects updating the database.
42 There also the --lazy-check which can greatly speed up indexing; see
44 below for details.
45 The first run of mu index may take a few minutes if you have a lot of
47 mail (tens of thousands of messages). Fortunately, such a full scan
48 needs to be done only once; after that it suffices to index the
49 changes, which goes much faster. See the 'Note on performance
50 (i,ii,iii)' below for more information.
51 The optional 'phase two' of the indexing-process is the removal of mes‐
53 sages from the database for which there is no longer a corresponding
54 file in the Maildir. If you do not want this, you can use -n,
55 --nocleanup.
56 When mu index catches one of the signals SIGINT, SIGHUP or SIGTERM
58 (e.g., when you press Ctrl-C during the indexing process), it tries to
59 shutdown gracefully; it tries to save and commit data, and close the
60 database etc. If it receives another signal (e.g., when pressing Ctrl-C
61 once more), mu index will terminate immediately.
62
65 Note, some of the general options are described in the mu(1) man-page
66 and not here, as they apply to multiple mu commands.
67 --lazy-check
70 in lazy-check mode, mu does not consider messages for which the
71 time-stamp (ctime) of the directory they reside in has not
72 changed since the previous indexing run. This is much faster
73 than the non-lazy check, but won't update messages that have
74 change (rather than having been added or removed), since merely
75 editing a message does not update the directory time-stamp. Of
76 course, you can run mu-index occasionally without --lazy-check,
77 to pick up such messages.
78 --nocleanup
81 disables the database cleanup that mu does by default after
82 indexing.
83 A note on performance (i)
86 As a non-scientific benchmark, a simple test on the author's machine (a
87 Thinkpad X61s laptop using Linux 2.6.35 and an ext3 file system) with
88 no existing database, and a maildir with 27273 messages:
89 $ sudo sh -c 'sync && echo 3 > /proc/sys/vm/drop_caches'
91 $ time mu index --quiet
92 66,65s user 6,05s system 27% cpu 4:24,20 total
93 (about 103 messages per second)
94 A second run, which is the more typical use case when there is a data‐
96 base already, goes much faster:
97 $ sudo sh -c 'sync && echo 3 > /proc/sys/vm/drop_caches'
99 $ time mu index --quiet
100 0,48s user 0,76s system 10% cpu 11,796 total
101 (more than 56818 messages per second)
102 Note that each test flushes the caches first; a more common use case
104 might be to run mu index when new mail has arrived; the cache may stay
105 quite 'warm' in that case:
106 $ time mu index --quiet
108 0,33s user 0,40s system 80% cpu 0,905 total
109 which is more than 30000 messages per second.
110 A note on performance (ii)
114 As per June 2012, we did the same non-scientific benchmark, this time
115 with an Intel i5-2500 CPU @ 3.30GHz, an ext4 file system and a maildir
116 with 22589 messages. We start without an existing database.
117 $ sudo sh -c 'sync && echo 3 > /proc/sys/vm/drop_caches'
119 $ time mu index --quiet
120 27,79s user 2,17s system 48% cpu 1:01,47 total
121 (about 813 messages per second)
122 A second run, which is the more typical use case when there is a data‐
124 base already, goes much faster:
125 $ sudo sh -c 'sync && echo 3 > /proc/sys/vm/drop_caches'
127 $ time mu index --quiet
128 0,13s user 0,30s system 19% cpu 2,162 total
129 (more than 173000 messages per second)
130 A note on performance (iii)
134 As per July 2016, we did the same non-scientific benchmark, again with
135 the Intel i5-2500 CPU @ 3.30GHz, an ext4 file system. This time, the
136 maildir contains 72525 messages.
137 $ sudo sh -c 'sync && echo 3 > /proc/sys/vm/drop_caches'
139 $ time mu index --quiet
140 40,34s user 2,56s system 64% cpu 1:06,17 total
141 (about 1099 messages per second).
142 As shown, mu has been getting faster with each release, even with rela‐
144 tively expensive new features such as text-normalization (for case-
145 insensitve/accent-insensitive matching). The profiles are dominated by
146 operations in the Xapian database now.
147
150 mu stores logs of its operations and queries in <muhome>/mu.log (by
151 default, this is ~/.cache/mu/mu.log). Upon startup, mu checks the size
152 of this log file. If it exceeds 1 MB, it will be moved to
153 ~/.cache/mu/mu.log.old, overwriting any existing file of that name, and
154 start with an empty log file. This scheme allows for continued use of
155 mu without the need for any manual maintenance of log files.
156
159 mu index uses MAILDIR to find the user's Maildir if it has not been
160 specified explicitly with --maildir=<maildir>. If MAILDIR is not set,
161 mu index will try ~/Maildir.
162
165 mu index return 0 upon successful completion, and any other number
166 greater than 0 signals an error.
167
170 Please report bugs if you find them: https://github.com/djcb/mu/issues
171
174 Dirk-Jan C. Binnema <djcb@djcbsoftware.nl>
175
178 maildir(5), mu(1), mu-init(1), mu-find(1), mu-cfind(1)
179User Manuals February 2020 MU-INDEX(1)