changeset 429:58899a413569 HEAD

Documentation updates.
author Timo Sirainen <tss@iki.fi>
date Tue, 15 Oct 2002 03:45:46 +0300
parents 4c7d08ae3f6e
children b3073f5b1e37
files .cvsignore INSTALL README doc/auth.txt doc/configuration.txt doc/mail-storages.txt
diffstat 6 files changed, 252 insertions(+), 71 deletions(-) [+]
line wrap: on
line diff
--- a/.cvsignore	Tue Oct 15 00:21:48 2002 +0300
+++ b/.cvsignore	Tue Oct 15 03:45:46 2002 +0300
@@ -22,7 +22,6 @@
 install-sh
 missing
 mkinstalldirs
-INSTALL
 intl
 ABOUT-NLS
 COPYING
--- a/INSTALL	Tue Oct 15 00:21:48 2002 +0300
+++ b/INSTALL	Tue Oct 15 03:45:46 2002 +0300
@@ -13,6 +13,8 @@
   mv /usr/local/etc/dovecot-example.conf /usr/local/etc/dovecot.conf
 
 Read through it, make needed modifications and start imap-master.
+doc/configuration.txt tells you more about configuring Dovecot.
+
 
 SSL/TLS
 -------
@@ -20,6 +22,7 @@
 GNUTLS v0.5.5 or later is required to enable SSL and TLS support. OpenSSL
 isn't supported. You can get GNUTLS from http://www.gnutls.org/
 
+
 Optional configure options
 --------------------------
 
--- a/README	Tue Oct 15 00:21:48 2002 +0300
+++ b/README	Tue Oct 15 03:45:46 2002 +0300
@@ -1,56 +1,17 @@
-
- * Alpha release
-
-This release has not been extensively tested, I don't recommend putting it
-into production usage yet. All the IMAP4rev1 features are fully supported,
-except custom message flags still need to be implemented. There's several
-minor problems left, see TODO file.
-
- * Mail storages
+Installation
+------------
 
-Maildir is the preferred storage with most robust implementation. mbox
-support is available but currently it relies a little bit on good luck, I
-made it mostly for personal use and I strongly recommend using maildir if
-at all possible.
+See INSTALL file.
 
-Maildir folders are done like with courier. INBOX is the ~/Maildir
-directory, all the other folders named as ".<folder>.<subfolder>.<etc>"
-under it, any directories not starting with a dot are simply ignored.
-Deleting folders is made atomic by renaming them as "..<folder name>" and
-deleting the directory after that. So, all directories beginning with ".."
-are deleted whenever they're noticed. Indexes are stored into each folder's
-root directory.
 
-mbox storage currently requires that all mail is in user-writable
-directory, eg. ~/mail. Especially using /var/mail/user is NOT supported.
-You may however create a ~/mail/inbox hardlink (or symlink if not chrooted)
-to pointing to /var/mail/user. All files in the mail directory are
-considered as mailboxes. Directories specify subfolders in IMAP. "inbox"
-file specifies the INBOX folder and can not be named otherwise. Indexes are
-stored into ".imap/<mailbox name>/" directories.
-
-imap process detects the storage from MAIL-environment which is preferred
-to be in format "<storage>:<data>", for example "maildir:~/Maildir". It's
-anyway allowed to be in pretty much any format as long as some of the
-storages recognizes it as a valid data, so for example "MAIL=~/mail" is
-first checked by maildir storage to see if it's valid maildir and
-mbox storage after that.
+Configuration
+-------------
 
-If the MAIL environment isn't given at all, all the storages are gone
-through which try to find a valid directory for themselves to use. Also as
-a special case, if MAILDIR environment exists, maildir storage is used with
-the directory specified in it.
+See doc/configuration.txt file.
+
 
-Maildir storage is autodetected by checking if <directory>/cur/ exists and
-we have rwx access to it. If directory isn't known, / and ~/Maildir are
-checked. / is checked because we could be chrooted.
-
-mbox storage is autodetected by checking if .imap/ (+rwx), inbox (+rw) or
-mbox (+rw) exists in directory. If directory isn't known, / is tried first.
-After that, ~/mail and ~/Mail directories are used if they're found without
-checking if they even contain any files.
-
- * Code
+Code
+----
 
 The code is split into a several libraries and binaries:
 
@@ -67,20 +28,22 @@
                    imap binary uses only the interface so it's possible to
 		   add support for any kind of mail storage (eg. SQL).
 
- * RFCs conformed
+
+RFCs conformed
+--------------
 
-822  - Standard for ARPA Internet Text Messages
-2822 - Internet Message Format (updated rfc822)
-2045..2049 - Multipurpose Internet Mail Extensions (MIME)
+ 822        - Standard for ARPA Internet Text Messages
+ 2822       - Internet Message Format (updated rfc822)
+ 2045..2049 - Multipurpose Internet Mail Extensions (MIME)
 
-2060 - IMAP4rev1
-2180 - IMAP4 Multi-Accessed Mailbox Practice
+ 2060       - IMAP4rev1
+ 2180       - IMAP4 Multi-Accessed Mailbox Practice
 
-2595 - Using TLS with IMAP, POP3 and ACAP
-2831 - Using Digest Authentication as a SASL Mechanism (DIGEST-MD5)
+ 2595       - Using TLS with IMAP, POP3 and ACAP
+ 2831       - Using Digest Authentication as a SASL Mechanism (DIGEST-MD5)
 
-rfc2831
 
- * Contact info
+Contact info
+------------
 
 Timo Sirainen <tss@iki.fi>, http://dovecot.procontrol.fi/
--- a/doc/auth.txt	Tue Oct 15 00:21:48 2002 +0300
+++ b/doc/auth.txt	Tue Oct 15 03:45:46 2002 +0300
@@ -1,12 +1,11 @@
-Authentication is split into two separate parts: the authentication method,
-and the password/user information backend (FIXME: is there a good name for
-that? I can't think of any).
+Authentication is split into two separate parts: the authentication
+mechanism, and the authentication backend.
 
-Currently supported methods:
+Currently supported mechanisms:
 
  - plaintext: By itself it's very insecure, but through secured SSL/TLS
    connection it should be fine.
- - Digest-MD5: Should be quite secure by itself, and it also supports
+ - Digest-MD5: Should be quite secure by itself. It also supports
    integrity protection and crypting the rest of the communication, but
    we don't support those yet.
 
@@ -16,11 +15,10 @@
  - shadow: /etc/shadow or similiar, using getspnam()
  - pam: PAM authentication
  - passwd-file: /etc/passwd-like file
+ - vpopmail: External software used to handle virtual domains
 
-passwd, shadow and pam backends work only with plaintext authentication.
-passwd-file can be used with both plaintext and Digest-MD5. More backends
-can be easily added later.
-
+Digest-MD5 works only with passwd-file. More mechanisms and backends can be
+easily added later.
 
 passwd-file
 -----------
@@ -37,10 +35,10 @@
 
 Either home or mail is required to exist. Home specifies the home directory
 of user under which mail is located. The actual mail format and location is
-automatically detected, just as if you run the imap-binary directly. Other
+automatically detected, just as if you run the imap binary directly. Other
 way is to specify the mail storage parameters directly using the
-mail-field, see README file for more information about it (the MAIL
-environment).
+mail-field, see "Detecting what to use" chapter in mail-storage.txt file
+for more information about it (the MAIL environment).
 
 Flags is a comma-separated list of flags, currently only recognized value
 is "chroot", which makes the imap process chroot into home directory, if
--- /dev/null	Thu Jan 01 00:00:00 1970 +0000
+++ b/doc/configuration.txt	Tue Oct 15 03:45:46 2002 +0300
@@ -0,0 +1,133 @@
+Authentication
+--------------
+
+See auth.txt.
+
+
+Maildir or mbox?
+----------------
+
+Maildir stores each message into a separate file, message flags are stored
+into file name. These make maildir very unlikely to get corrupted in any
+way.
+
+Reading lots of mails from maildir is somewhat slower than from mbox, since
+each mail file needs to be separately opened. Updating the mailbox however
+is much faster than with mbox.
+
+With larger mailboxes it's a good idea to use a filesystem which uses
+b-tree or hash indexes for directories, for example ReiserFS, XFS or JFS.
+ext2 and ext3 have some patches to implement this but they're not in Linux
+2.4.19 yet. I'm not sure about *BSD's filesystems, FreeBSD's ufs had some
+support for hashes.
+
+mbox is just a single file where new mails are appeneded, flags are stored
+in each message's headers. Deleting mails is slow as the rest of the file
+needs to be moved over the deleted mail. Changing message flags is usually
+quite fast since we use some tricks to avoid copying too much data, but it
+may result as well in large data copying.
+
+Besides the copying being slow, it's also a bit dangerous. If the copying
+is aborted (crashed, killed, power lost) the mail file may be left in
+somewhat corrupted stated.
+
+Bottom line: mbox is good for read-only mailboxes, maildir for everything
+else.
+
+
+Creating new users
+------------------
+
+Dovecot is interested in only one thing - being able to find the user's
+mail directory. With maildir you need to do mkdir ~user/Maildir, with mbox
+mkdir ~user/mail.
+
+
+System with local users
+-----------------------
+
+First of all, don't make imap process chroot to users' mail directory,
+UNLESS that filesystem is mounted with nosuid-flag. Otherwise there's a
+small possibility for the user to gain root privileges.
+
+Then you need to decide if you want users to have the same IMAP password
+than the system password. If yes, disable_plaintext_auth = yes is a very
+good idea.
+
+
+System without local users
+--------------------------
+
+First you'll need to decide if you want to use one or multiple system uids.
+For example one for everything, one per each virtual domain or one per each
+user. In any case the uids should be different than the uids used for other
+parts of Dovecot (login or auth processes).
+
+Having one uid per user would mean that in case of a security hole in
+Dovecot, the user still couldn't read other peoples mails. Use this if
+possible.
+
+chrooting imap processes would be good idea, but you should still think
+about having the filesystem nosuid-mounted.
+
+
+Performance
+-----------
+
+Usually the bottleneck with IMAP server is disk I/O, so get fast disks and
+lots of memory to act as operating system's file cache.
+
+One performance tweak is to save mails with CR+LFs instead of just LFs.
+This can result in faster indexing of mails and smaller CPU usage when
+sending mails. With Linux and FreeBSD Dovecot can use sendfile() syscall to
+send such mails. However extra CRs do increase the mail size, meaning more
+I/O and potentially losing the gained performance. You can enable this for
+mails saved by Dovecot by setting mail_save_crs = yes. For mails saved by
+your mailer you'll need to do something else, not yet covered by this
+documentation.
+
+COPY command can be made much faster with maildir by setting
+maildir_copy_with_hardlinks = yes. This is problematic only if something
+modifies the mail in one folder but doesn't want it modified in the others.
+I don't know any MUA which would modify any mail files directly.
+
+Logins can be handled either fast, or securely. Doing it securely means
+creating a new login process for each connection instead of having only
+few processes handling multiple connections. The problem with sharing
+connections is that if a security hole is found, the attacker could hijack
+other peoples connections or steal their passwords if plaintext
+authentication was used (even with SSL/TLS).
+
+Dovecot's memory usage is very small. Almost all memory usage you see with
+ps/top is from mmap()ed files, meaning that operating system can drop any
+of those memory pages at any time without needing to swap them, unless
+they're just being modified. With Linux/x86 Dovecot usually takes about
+48kB of non-mmaped memory plus ~4kB per cached mail (max. 16 by default).
+
+
+Rootless Dovecot
+----------------
+
+It's possible to make Dovecot run under one uid, not requiring root
+privileges at any point. This shouldn't be thought of as any security
+feature, but instead just as a way for non-admins to run imap server from
+their favourite mail computer.
+
+If you do think of this as a good way to achieve security, ask yourself
+which is worse:
+
+a) near-zero possibility to get root privileges, small possibility to get
+into system as imapd user chrooted into empty directory without logging in,
+small possibility to get logged user's privileges but no possiblity to read
+others mails since they're saved with different uid (plus you might be
+chrooted to your own mailbox).
+
+b) zero possibility to get root privileges through Dovecot, small
+possibility to get into system as mail user, possibly even without logging
+in, and being able to read everyone's mail (and finally getting roots by
+exploiting some local just released vulnerability, unless you bothered to
+set up special chroot environment).
+
+Anyway, doing it is easy. configure --prefix=$HOME, make install, change
+login_user and auth_user in dovecot.conf to your user id and use
+passwd-file authentication.
--- /dev/null	Thu Jan 01 00:00:00 1970 +0000
+++ b/doc/mail-storages.txt	Tue Oct 15 03:45:46 2002 +0300
@@ -0,0 +1,85 @@
+ * Maildir
+
+We support Courier-compatible Maildir++. INBOX is the ~/Maildir directory,
+all the other folders named as ".<folder>.<subfolder>.<etc>" under it, any
+directories not starting with a dot are simply ignored. Deleting folders is
+made atomic by renaming them as "..<folder name>" and deleting the
+directory after that. So, all directories beginning with ".." are deleted
+whenever they're noticed. Indexes are stored into each folder's root
+directory.
+
+Custom flags saved using a..z letters as Maildir file flags. .customflags
+file is then used to map the letters to names (a=0, b=1, etc).
+
+ * mbox
+
+mbox storage currently requires that all mail is in user-writable
+directory, eg. ~/mail. Especially using /var/mail/user is NOT supported.
+You may however create a ~/mail/inbox hardlink (or symlink if not chrooted)
+to pointing to /var/mail/user. All files in the mail directory are
+considered as mailboxes. Directories specify subfolders in IMAP. "inbox"
+file specifies the INBOX folder and can not be named otherwise. Indexes are
+stored into ".imap/<mailbox name>/" directories.
+
+ * Detecting what to use
+
+imap process detects the storage from MAIL-environment. It's preferred
+format is "<storage>:<data>", for example "maildir:~/Maildir". It's anyway
+allowed to be in pretty much any format as long as some of the storages
+recognizes it as a valid data, so for example "MAIL=~/mail" is first
+checked by maildir storage to see if it's valid maildir and mbox storage
+after that.
+
+If the MAIL environment isn't given at all, all storages are gone through
+and asked if they can find a usable mail folder. Also as a special case, if
+MAILDIR environment exists, maildir storage is used with the directory
+specified in it.
+
+Maildir storage is autodetected by checking if <directory>/cur/ exists and
+we have rwx access to it. If directory isn't known, / and ~/Maildir are
+checked. / is checked because we could be chrooted.
+
+mbox storage is autodetected by checking if .imap/ (+rwx), inbox (+rw) or
+mbox (+rw) exists in directory. If directory isn't known, / is tried first.
+After that, ~/mail and ~/Mail directories are used.
+
+ * Files created by Dovecot
+
+.subscriptions: One folder per line listing subscribed IMAP folders.
+There's only one of these files in the root mail directory (~/Maildir/ or
+~/mail/.imap/).
+
+Then there's the per-folder files:
+
+.imap.index: Fixed width mail index records.
+
+.imap.index.data: Variable width data, for example mail file name and
+cached body, envelope, etc.
+
+.imap.index.log: Log file describing made changes. This file is used to
+mark external changes (expunges and flag changes) temporarily until they're
+synced into IMAP clients. If multiple Dovecots have the same folder open,
+this can be a big speedup.
+
+.imap.index.tree: Binary tree used for looking up messages from .imap.index
+file. This is used for both UID lookups and message sequence lookups.
+Sequences can be looked up directly from the index as long as no messages
+have been expunged, but after that it gets slow unless it's compressed.
+Using this file however can keep the sequence lookups and expunges fast in
+all situations without requiring compression.
+
+.customflags: Lists custom message flags. First line is header which
+Dovecots check to see if the file has changed. Other lines list the flags
+in format "<flag number> <flag name>". Currently there can be 26 different
+flags (0..25).
+
+.imap.index.log and .imap.index.tree files can be deleted without any data
+loss, they're simply recreated/rebuilt when they're noticed missing.
+
+Deleting .imap.index and .imap.index.data files results in loss of message
+UIDs. That may be a bad thing with some IMAP clients which store some data
+about message locally and identifies the message by it's UID.
+
+Deleting .customflags results in loss of custom flag names for Maildir.
+With mbox the .customflags is simply recreated based on the flags listed
+in the mbox file itself (ie. no data loss).