1MAILDIR(5) Double Precision, Inc. MAILDIR(5)
2
6 maildir - E-mail directory
7
9 $HOME/Maildir
10
13 A “Maildir” is a structured directory that holds E-mail messages.
14 Maildirs were first implemented by the Qmail mail server. Qmail's
15 maildirs were a simple data structure, nothing more than a single
16 collection of E-mail messages. Courier builds upon Qmail's maildirs to
17 provide extended functionality, such as folders and quotas. This
18 document describes Courier's extended maildirs, without explicitly
19 identifying Courier-specific extensions. See maildir(5) in Qmail's
20 documentation for the original definition of maildirs.
21 Traditionally, E-mail folders were saved as plain text files, called
23 “mboxes”. Mboxes have known limitations. Only one application can use
24 an mbox at the same time. Locking is required in order to allow
25 simultaneous concurrent access by different applications. Locking is
26 often problematic, and not very reliable in network-based filesystem
27 requirements. Some network-based filesystems don't offer any reliable
28 locking mechanism at all. Furthermore, even bulletproof locking won't
29 prevent occasional mbox corruption. A process can be killed or
30 terminated in the middle of updating an mbox. This will likely result
31 in corruption, and a loss of most messages in the mbox.
32 Maildirs allow multiple concurrent access by different applications.
34 Maildirs do not require locking. Multiple applications can update a
35 maildir at the same time, without stepping on each other's feet.
36 Maildir contents
38 A “maildir” is a directory that's created by maildirmake(1)[1].
39 Naturally, maildirs should not have any group or world permissions,
40 unless you want other people to read your mail. A maildir contains
41 three subdirectories: tmp, new, and cur. These three subdirectories
42 comprise the primary folder, where new mail is delivered by the system.
43 Folders are additional subdirectories in the maildir whose names begin
45 with a period: such as .Drafts or .Sent. Each folder itself contains
46 the same three subdirectories, tmp, new, and cur, and an additional
47 zero-length file named maildirfolder, whose purpose is to inform any
48 mail delivery agent that it's really delivering to a folder, and that
49 the mail delivery agent should look in the parent directory for any
50 maildir-related information.
51 Folders are not physically nested. A folder subdirectory, such as .Sent
53 does not itself contain any subfolders. The main maildir contains a
54 single, flat list of subfolders. These folders are logically nested,
55 and periods serve to separate folder hierarchies. For example,
56 .Sent.2002 is considered to be a subfolder called “2002” which is a
57 subfolder of “Sent”.
58 Folder name encoding
60 Folder names can contain any Unicode character, except for
61 control characters. US-ASCII characters, U+0x0020 - U+0x007F,
62 except for the period, forward-slash, and ampersand characters
63 (U+0x002E, U+0x002F, and U+0x0026) represent themselves. The
64 ampersand is represent by the two character sequence “&-”. The
65 period, forward slash, and non US-ASCII Unicode characters are
66 represented using the UTF-7 character set, and encoded with a
67 modified form of base64-encoding.
68 The “&” character starts the modified base64-encoded sequence;
70 the sequence is terminated by the “-” character. The sequence of
71 16-bit Unicode characters is written in big-endian order, and
72 encoded using the base64-encoding method described in section
73 5.2 of RFC 1521[2], with the following modifications:
74 · The “=” padding character is omitted. When decoding, an
75 incomplete 16-bit character is discarded.
76 · The comma character, “,” is used in place of the “/”
77 character in the base64 alphabet.
78 For example, the word “Resume” with both "e"s being the e-acute
80 character, U+0x00e9, is encoded as “R&AOk-sum&AOk-” (so a folder
81 of that name would be a maildir subdirectory called
82 “.R&AOk-sum&AOk-”).
83 Other maildir contents
85 Software that uses maildirs may also create additional files
86 besides the tmp, new, and cur subdirectories -- in the main
87 maildir or a subfolder -- for its own purposes.
88 Messages
90 E-mail messages are stored in separate, individual files, one E-mail
91 message per file. The tmp subdirectory temporarily stores E-mail
92 messages that are in the process of being delivered to this maildir.
93 tmp may also store other kinds of temporary files, as long as they are
94 created in the same way that message files are created in tmp. The new
95 subdirectory stores messages that have been delivered to this maildir,
96 but have not yet been seen by any mail application. The cur
97 subdirectory stores messages that have already been seen by mail
98 applications.
99 Adding new mail to maildirs
101 The following process delivers a new message to the maildir:
102 A new unique filename is created using one of two possible forms:
104 “time.MusecPpid.host”, or “time.MusecPpid_unique.host”. “time” and
105 “usec” is the current system time, obtained from gettimeofday(2).
106 “pid” is the process number of the process that is delivering this
107 message to the maildir. “host” is the name of the machine where the
108 mail is being delivered. In the event that the same process creates
109 multiple messages, a suffix unique to each message is appended to the
110 process id; preferrably an underscore, followed by an increasing
111 counter. This applies whether messages created by a process are all
112 added to the same, or different, maildirs. This protocol allows
113 multiple processes running on multiple machines on the same network to
114 simultaneously create new messages without stomping on each other.
115 The filename created in the previous step is checked for existence by
117 executing the stat(2) system call. If stat(2) results in ANYTHING OTHER
118 than the system error ENOENT, the process must sleep for two seconds,
119 then go back and create another unique filename. This is an extra step
120 to insure that each new message has a completely unique filename.
121 Other applications that wish to use tmp for temporary storage should
123 observe the same protocol (but see READING MAIL FROM MAILDIRS below,
124 because old files in tmp will be eventually deleted).
125 If the stat(2) system call returned ENOENT, the process may proceed to
127 create the file in the tmp subdirectory, and save the entire message in
128 the new file. The message saved MUST NOT have the “From_” header that
129 is used to mboxes. The message also MUST NOT have any “From_” lines in
130 the contents of the message prefixed by the “>” character.
131 When saving the message, the number of bytes returned by the write(2)
133 system call must be checked, in order to make sure that the complete
134 message has been written out.
135 After the message is saved, the file descriptor is fstat(2)-ed. The
137 file's device number, inode number, and the its byte size, are saved.
138 The file is closed and is then immediately moved/renamed into the new
139 subdirectory. The name of the file in new should be
140 “time.MusecPpidVdevIino.host,S=cnt”, or
141 “time.MusecPpidVdevIino_unique.host,S=cnt”. “dev” is the message's
142 device number, “ino” is the message's inode number (from the previous
143 fstat(2) call); and “cnt” is the message's size, in bytes.
144 The “,S=cnt” part optimizes Courier[3]'s maildir quota enhancement; it
146 allows the size of all the mail stored in the maildir to be added up
147 without issuing the stat(2) system call for each individual message
148 (this can be quite a performance drain with certain network
149 filesystems).
150 READING MAIL FROM MAILDIRS
152 Applications that read mail from maildirs should do it in the following
153 order:
154 When opening a maildir or a maildir folder, read the tmp subdirectory
156 and delete any files in there that are at least 36 hours old.
157 Look for new messages in the new subdirectory. Rename new/filename, as
159 cur/filename:2,info. Here, info represents the state of the message,
160 and it consists of zero or more boolean flags chosen from the
161 following: “D” - this is a 'draft' message, “R” - this message has been
162 replied to, “S” - this message has been viewed (seen), “T” - this
163 message has been marked to be deleted (trashed), but is not yet removed
164 (messages are removed from maildirs simply by deleting their file), “F”
165 - this message has been marked by the user, for some purpose. These
166 flags must be stored in alphabetical order. New messages contain only
167 the :2, suffix, with no flags, indicating that the messages were not
168 seen, replied, marked, or deleted.
169 Maildirs may have maximum size quotas defined, but these quotas are
171 purely voluntary. If you need to implement mandatory quotas, you should
172 use any quota facilities provided by the underlying filesystem that is
173 used to store the maildirs. The maildir quota enhancement is designed
174 to be used in certain situations where filesystem-based quotas cannot
175 be used for some reason. The implementation is designed to avoid the
176 use of any locking. As such, at certain times the calculated quota may
177 be imprecise, and certain anomalous situations may result in the
178 maildir actually going over the stated quota. One such situation would
179 be when applications create messages without updating the quota
180 estimate for the maildir. Eventually it will be precisely recalculated,
181 but wherever possible new messages should be created in compliance with
182 the voluntary quota protocol.
183 The voluntary quota protocol involves some additional procedures that
185 must be followed when creating or deleting messages within a given
186 maildir or its subfolders. The deliverquota(8)[4] command is a tiny
187 application that delivers a single message to a maildir using the
188 voluntary quota protocol, and hopefully it can be used as a measure of
189 last resort. Alternatively, applications can use the libmaildir.a
190 library to handle all the low-level dirty details for them. The
191 voluntary quota enhancement is described in the maildirquota(7)[5] man
192 page.
193 Maildir Quotas
195 This is a voluntary mechanism for enforcing "loose" quotas on the
196 maximum sizes of maildirs. This mechanism is enforced in software, and
197 not by the operating system. Therefore it is only effective as long as
198 the maildirs themselves are not directly accessible by their users,
199 since this mechanism is trivially disabled.
200 If possible, operating system-enforced quotas are preferrable. Where
202 operating system quota enforcement is not available, or not possible,
203 this voluntary quota enforcement mechanism might be an acceptable
204 compromise. Since it's enforced in software, all software that modifies
205 or accesses the maildirs is required to voluntary obey and enforce a
206 quota. The voluntary quota implementation is flexible enough to allow
207 non quota-aware applications to also access the maildirs, without any
208 drastic consequences. There will be some non-drastic consequences,
209 though. Of course, non quota-aware applications will not enforce any
210 defined quotas. Furthermore, this voluntary maildir quota mechanism
211 works by estimating the current size of the maildir, with periodic
212 exact recalculation. Obviously non quota-aware maildir applications
213 will not update the maildir size estimation, so the estimate will be
214 thrown off for some period of time, until the next recalculation.
215 This voluntary quota mechanism is designed to be a reasonable
217 compromise between effectiveness, and performance. The entire purpose
218 of using maildir-based mail storage is to avoid any kind of locking,
219 and to permit parallel access to mail by multiple applications. In
220 order to compute the exact size of a maildir, the maildir must be
221 locked somehow to prevent any modifications while its contents are
222 added up. Obviously something like that defeats the original purpose of
223 using maildirs, therefore the voluntary quota mechanism does not use
224 locking, and that's why the current recorded maildir size is always
225 considered to be an estimate. Regular size recalculations will
226 compensate for any occasional race conditions that result in the
227 estimate to be thrown off.
228 A quota for an existing maildir is installed by running maildirmake
230 with the -q option, and naming an existing maildir. The -q option takes
231 a parameter, quota, which is a comma-separated list of quota
232 specifications. A quota specification consists of a number followed by
233 either 'S', indicating the maximum message size in bytes, or 'C',
234 maximum number of messages. For example:
235 This sets the quota to 5,000,000 bytes or 1000 messages, whichever
237 comes first.
238 This sets the quota to 1,000,000 bytes, without limiting the number of
240 messages.
241 A quota of an existing maildir can be changed by rerunning the
243 maildirmake command with a new -q option. To delete a quota entirely,
244 delete the Maildir/maildirsize file.
245
247 maildirmake(1)[1].
248
250 1. maildirmake(1)
251 maildirmake.html
252 2. RFC 1521
254 http://www.rfc-editor.org/rfc/rfc1521.txt
255 3. Courier
257 http://www.courier-mta.org
258 4. deliverquota(8)
260 deliverquota.html
261 5. maildirquota(7)
263 maildirquota.html
264Double Precision, Inc. 04/22/2007 MAILDIR(5)