1MAILDIR(5)                  Double Precision, Inc.                  MAILDIR(5)
2
3
4

NAME

6       maildir - E-mail directory
7

SYNOPSIS

9       $HOME/Maildir
10

DESCRIPTION

12       A “Maildir” is a structured directory that holds E-mail messages.
13       Maildirs were first implemented by the Qmail mail server. Qmail´s
14       maildirs were a simple data structure, nothing more than a single
15       collection of E-mail messages. The Courier mail server builds upon
16       Qmail´s maildirs to provide extended functionality, such as folders and
17       quotas. This document describes the Courier mail server´s extended
18       maildirs, without explicitly identifying The Courier mail
19       server-specific extensions. See maildir(5) in Qmail´s documentation for
20       the original definition of maildirs.
21
22       Traditionally, E-mail folders were saved as plain text files, called
23       “mboxes”. Mboxes have known limitations. Only one application can use
24       an mbox at the same time. Locking is required in order to allow
25       simultaneous concurrent access by different applications. Locking is
26       often problematic, and not very reliable in network-based filesystem
27       requirements. Some network-based filesystems don´t offer any reliable
28       locking mechanism at all. Furthermore, even bulletproof locking won´t
29       prevent occasional mbox corruption. A process can be killed or
30       terminated in the middle of updating an mbox. This will likely result
31       in corruption, and a loss of most messages in the mbox.
32
33       Maildirs allow multiple concurrent access by different applications.
34       Maildirs do not require locking. Multiple applications can update a
35       maildir at the same time, without stepping on each other´s feet.
36
37   Maildir contents
38       A “maildir” is a directory that´s created by maildirmake(1)[1].
39       Naturally, maildirs should not have any group or world permissions,
40       unless you want other people to read your mail. A maildir contains
41       three subdirectories: tmp, new, and cur. These three subdirectories
42       comprise the primary folder, where new mail is delivered by the system.
43
44       Folders are additional subdirectories in the maildir whose names begin
45       with a period: such as .Drafts or .Sent. Each folder itself contains
46       the same three subdirectories, tmp, new, and cur, and an additional
47       zero-length file named maildirfolder, whose purpose is to inform any
48       mail delivery agent that it´s really delivering to a folder, and that
49       the mail delivery agent should look in the parent directory for any
50       maildir-related information.
51
52       Folders are not physically nested. A folder subdirectory, such as .Sent
53       does not itself contain any subfolders. The main maildir contains a
54       single, flat list of subfolders. These folders are logically nested,
55       and periods serve to separate folder hierarchies. For example,
56       .Sent.2002 is considered to be a subfolder called “2002” which is a
57       subfolder of “Sent”.
58
59       Folder name encoding
60           Folder names can contain any Unicode character, except for control
61           characters. US-ASCII characters, U+0x0020 - U+0x007F, except for
62           the period, forward-slash, and ampersand characters (U+0x002E,
63           U+0x002F, and U+0x0026) represent themselves. The ampersand is
64           represent by the two character sequence “&-”. The period, forward
65           slash, and non US-ASCII Unicode characters are represented using
66           the UTF-7 character set, and encoded with a modified form of
67           base64-encoding.
68
69           The “&” character starts the modified base64-encoded sequence; the
70           sequence is terminated by the “-” character. The sequence of 16-bit
71           Unicode characters is written in big-endian order, and encoded
72           using the base64-encoding method described in section 5.2 of RFC
73           1521[2], with the following modifications:
74
75           ·   The “=” padding character is omitted. When decoding, an
76               incomplete 16-bit character is discarded.
77
78           ·   The comma character, “,” is used in place of the “/” character
79               in the base64 alphabet.
80
81           For example, the word “Resume” with both "e"s being the e-acute
82           character, U+0x00e9, is encoded as “R&AOk-sum&AOk-” (so a folder of
83           that name would be a maildir subdirectory called
84           “.R&AOk-sum&AOk-”).
85
86       Other maildir contents
87           Software that uses maildirs may also create additional files
88           besides the tmp, new, and cur subdirectories -- in the main maildir
89           or a subfolder -- for its own purposes.
90
91   Messages
92       E-mail messages are stored in separate, individual files, one E-mail
93       message per file. The tmp subdirectory temporarily stores E-mail
94       messages that are in the process of being delivered to this maildir.
95       tmp may also store other kinds of temporary files, as long as they are
96       created in the same way that message files are created in tmp. The new
97       subdirectory stores messages that have been delivered to this maildir,
98       but have not yet been seen by any mail application. The cur
99       subdirectory stores messages that have already been seen by mail
100       applications.
101
102   Adding new mail to maildirs
103       The following process delivers a new message to the maildir:
104
105       A new unique filename is created using one of two possible forms:
106       “time.MusecPpid.host”, or “time.MusecPpid_unique.host”.  “time” and
107       “usec” is the current system time, obtained from gettimeofday(2).
108       “pid” is the process number of the process that is delivering this
109       message to the maildir.  “host” is the name of the machine where the
110       mail is being delivered. In the event that the same process creates
111       multiple messages, a suffix unique to each message is appended to the
112       process id; preferrably an underscore, followed by an increasing
113       counter. This applies whether messages created by a process are all
114       added to the same, or different, maildirs. This protocol allows
115       multiple processes running on multiple machines on the same network to
116       simultaneously create new messages without stomping on each other.
117
118       The filename created in the previous step is checked for existence by
119       executing the stat(2) system call. If stat(2) results in ANYTHING OTHER
120       than the system error ENOENT, the process must sleep for two seconds,
121       then go back and create another unique filename. This is an extra step
122       to insure that each new message has a completely unique filename.
123
124       Other applications that wish to use tmp for temporary storage should
125       observe the same protocol (but see READING MAIL FROM MAILDIRS below,
126       because old files in tmp will be eventually deleted).
127
128       If the stat(2) system call returned ENOENT, the process may proceed to
129       create the file in the tmp subdirectory, and save the entire message in
130       the new file. The message saved MUST NOT have the “From_” header that
131       is used to mboxes. The message also MUST NOT have any “From_” lines in
132       the contents of the message prefixed by the “>” character.
133
134       When saving the message, the number of bytes returned by the write(2)
135       system call must be checked, in order to make sure that the complete
136       message has been written out.
137
138       After the message is saved, the file descriptor is fstat(2)-ed. The
139       file´s device number, inode number, and the its byte size, are saved.
140       The file is closed and is then immediately moved/renamed into the new
141       subdirectory. The name of the file in new should be
142       “time.MusecPpidVdevIino.host,S=cnt”, or
143       “time.MusecPpidVdevIino_unique.host,S=cnt”.  “dev” is the message´s
144       device number, “ino” is the message´s inode number (from the previous
145       fstat(2) call); and “cnt” is the message´s size, in bytes.
146
147       The “,S=cnt” part optimizes the Courier[3] mail server´s maildir quota
148       enhancement; it allows the size of all the mail stored in the maildir
149       to be added up without issuing the stat(2) system call for each
150       individual message (this can be quite a performance drain with certain
151       network filesystems).
152
153   READING MAIL FROM MAILDIRS
154       Applications that read mail from maildirs should do it in the following
155       order:
156
157       When opening a maildir or a maildir folder, read the tmp subdirectory
158       and delete any files in there that are at least 36 hours old.
159
160       Look for new messages in the new subdirectory. Rename new/filename, as
161       cur/filename:2,info. Here, info represents the state of the message,
162       and it consists of zero or more boolean flags chosen from the
163       following: “D” - this is a ´draft´ message, “R” - this message has been
164       replied to, “S” - this message has been viewed (seen), “T” - this
165       message has been marked to be deleted (trashed), but is not yet removed
166       (messages are removed from maildirs simply by deleting their file), “F”
167       - this message has been marked by the user, for some purpose. These
168       flags must be stored in alphabetical order. New messages contain only
169       the :2, suffix, with no flags, indicating that the messages were not
170       seen, replied, marked, or deleted.
171
172       Maildirs may have maximum size quotas defined, but these quotas are
173       purely voluntary. If you need to implement mandatory quotas, you should
174       use any quota facilities provided by the underlying filesystem that is
175       used to store the maildirs. The maildir quota enhancement is designed
176       to be used in certain situations where filesystem-based quotas cannot
177       be used for some reason. The implementation is designed to avoid the
178       use of any locking. As such, at certain times the calculated quota may
179       be imprecise, and certain anomalous situations may result in the
180       maildir actually going over the stated quota. One such situation would
181       be when applications create messages without updating the quota
182       estimate for the maildir. Eventually it will be precisely recalculated,
183       but wherever possible new messages should be created in compliance with
184       the voluntary quota protocol.
185
186       The voluntary quota protocol involves some additional procedures that
187       must be followed when creating or deleting messages within a given
188       maildir or its subfolders. The deliverquota(8)[4] command is a tiny
189       application that delivers a single message to a maildir using the
190       voluntary quota protocol, and hopefully it can be used as a measure of
191       last resort. Alternatively, applications can use the libmaildir.a
192       library to handle all the low-level dirty details for them. The
193       voluntary quota enhancement is described in the maildirquota(7)[5] man
194       page.
195
196   Maildir Quotas
197       This is a voluntary mechanism for enforcing "loose" quotas on the
198       maximum sizes of maildirs. This mechanism is enforced in software, and
199       not by the operating system. Therefore it is only effective as long as
200       the maildirs themselves are not directly accessible by their users,
201       since this mechanism is trivially disabled.
202
203       If possible, operating system-enforced quotas are preferrable. Where
204       operating system quota enforcement is not available, or not possible,
205       this voluntary quota enforcement mechanism might be an acceptable
206       compromise. Since it´s enforced in software, all software that modifies
207       or accesses the maildirs is required to voluntary obey and enforce a
208       quota. The voluntary quota implementation is flexible enough to allow
209       non quota-aware applications to also access the maildirs, without any
210       drastic consequences. There will be some non-drastic consequences,
211       though. Of course, non quota-aware applications will not enforce any
212       defined quotas. Furthermore, this voluntary maildir quota mechanism
213       works by estimating the current size of the maildir, with periodic
214       exact recalculation. Obviously non quota-aware maildir applications
215       will not update the maildir size estimation, so the estimate will be
216       thrown off for some period of time, until the next recalculation.
217
218       This voluntary quota mechanism is designed to be a reasonable
219       compromise between effectiveness, and performance. The entire purpose
220       of using maildir-based mail storage is to avoid any kind of locking,
221       and to permit parallel access to mail by multiple applications. In
222       order to compute the exact size of a maildir, the maildir must be
223       locked somehow to prevent any modifications while its contents are
224       added up. Obviously something like that defeats the original purpose of
225       using maildirs, therefore the voluntary quota mechanism does not use
226       locking, and that´s why the current recorded maildir size is always
227       considered to be an estimate. Regular size recalculations will
228       compensate for any occasional race conditions that result in the
229       estimate to be thrown off.
230
231       A quota for an existing maildir is installed by running maildirmake
232       with the -q option, and naming an existing maildir. The -q option takes
233       a parameter, quota, which is a comma-separated list of quota
234       specifications. A quota specification consists of a number followed by
235       either ´S´, indicating the maximum message size in bytes, or ´C´,
236       maximum number of messages. For example:
237
238       This sets the quota to 5,000,000 bytes or 1000 messages, whichever
239       comes first.
240
241       This sets the quota to 1,000,000 bytes, without limiting the number of
242       messages.
243
244       A quota of an existing maildir can be changed by rerunning the
245       maildirmake command with a new -q option. To delete a quota entirely,
246       delete the Maildir/maildirsize file.
247

SEE ALSO

249       maildirmake(1)[1].
250

NOTES

252        1. maildirmake(1)
253           [set $man.base.url.for.relative.links]/maildirmake.html
254
255        2. RFC 1521
256           http://www.rfc-editor.org/rfc/rfc1521.txt
257
258        3. Courier
259           http://www.courier-mta.org
260
261        4. deliverquota(8)
262           [set $man.base.url.for.relative.links]/deliverquota.html
263
264        5. maildirquota(7)
265           [set $man.base.url.for.relative.links]/maildirquota.html
266
267
268
269Double Precision, Inc.            02/19/2010                        MAILDIR(5)
Impressum