# HG changeset patch # User Timo Sirainen # Date 1171301245 -7200 # Node ID c2af4ee1f25e2e2f16da461bc7ec904316de9176 # Parent 02f7ce1e75f6b14e2f3a10eee75a4653e68b69e8 Removed the old and wrong .txt files. diff -r 02f7ce1e75f6 -r c2af4ee1f25e doc/Makefile.am --- a/doc/Makefile.am Wed Feb 07 03:45:33 2007 +0200 +++ b/doc/Makefile.am Mon Feb 12 19:27:25 2007 +0200 @@ -2,14 +2,8 @@ doc_DATA = \ USE-WIKI-INSTEAD \ - auth.txt \ auth-protocol.txt \ configuration.txt \ - design.txt \ - index.txt \ - mail-storages.txt \ - multiaccess.txt \ - nfs.txt \ securecoding.txt \ variables.txt diff -r 02f7ce1e75f6 -r c2af4ee1f25e doc/USE-WIKI-INSTEAD --- a/doc/USE-WIKI-INSTEAD Wed Feb 07 03:45:33 2007 +0200 +++ b/doc/USE-WIKI-INSTEAD Mon Feb 12 19:27:25 2007 +0200 @@ -1,8 +1,6 @@ -Some of these documentation files are out of date and generally not that -useful. See wiki instead: +All the useful documentation is currently only in Dovecot's wiki: http://wiki.dovecot.org/ -Some of these contents in these files aren't yet in Wiki, so they haven't -yet been deleted. At some point the Wiki files will be included in this -directory as plaintext. +Perhaps some day they'll also be included in this directory as .txt files. +You can speed that up by creating a script which does that. diff -r 02f7ce1e75f6 -r c2af4ee1f25e doc/auth.txt --- a/doc/auth.txt Wed Feb 07 03:45:33 2007 +0200 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,172 +0,0 @@ -Authentication is split into three parts: authentication mechanism, -password database and user database. - -Currently supported authentication mechanisms: - - - PLAIN: By itself it's very insecure, but through secured SSL/TLS - connection it should be fine. - - DIGEST-MD5: Should be quite secure by itself. It also supports - integrity protecting and crypting the rest of the communication, but - we don't support those yet. - - CRAM-MD5: Protects the secret in transit from eavesdroppers. Doesn't - provide any integrity guarantees. - - ANONYMOUS: No authentication required. User will be logged in as the user - specified by auth_anonymous_username setting (default "anonymous"). There's - no special restrictions given for anonymous users so you have to make sure - it doesn't have access to unwanted locations. - -Currently supported password databases: - - - passwd: /etc/passwd or similiar, using getpwnam() - - shadow: /etc/shadow or similiar, using getspnam() - - pam: Pluggable Authentication Modules - - passwd-file: /etc/passwd-like file in specified location - - ldap: Lightweight Directory Access Protocol - - vpopmail: External software used to handle virtual domains - - pgsql: A PostgreSQL database. - -Currently supported user databases: - - - passwd: /etc/passwd or similiar, using getpwnam() - - passwd-file: /etc/passwd-like file in specified location - - ldap: Lightweight Directory Access Protocol - - vpopmail: External software used to handle virtual domains - - static: Static UID and GID, home directory from given template - - pgsql: A PostgreSQL database. - -Most password databases support only plaintext authentication. passwd-file -and LDAP exceptions since they support multiple password schemes. - -Password schemes supporting only plaintext authentication: - - - CRYPT: Use crypt(). Usually DES, but some systems support others too - (eg. MD5 and SHA1) - - MD5: MD5crypt algorithm, sometimes used in /etc/passwd and likes - - PLAIN-MD5: Simple MD5 sum of password. Used by libpam-pwdfile - -Password schemes supporting plaintext authentication and more: - - - PLAIN: Although not that good idea, it enables support for all current - and future authentication mechanisms. - - HMAC-MD5: HMAC-MD5 context of password, for the CRAM-MD5 mechanism. - - DIGEST-MD5: MD5 sum of "user:realm:password", as required by DIGEST-MD5 - mechanism. - -Realms (or virtual domains) are supported by appending the "@realm" after -the user name. This behaviour works with all authentication mechanisms and -databases. - -Home directory can be prefixed with "/./" in which case -directory will be chrooted into. The actual home directory follows the -"/./". For example "/chroot/./home/user". - - -passwd ------- - -Most commonly used as user database. Many systems use shadow passwords -nowadays so it doesn't usually work as password database. BSDs are an -exception to this, they still set the password field even with shadow -passwords. - - -shadow ------- - -Works at least with Linux and Solaris. - - -PAM ---- - -We should work with Linux PAM, Solaris PAM, OpenPAM (FreeBSD) and -ApplePAM (OSX). PAM doesn't provide user database, so you have to use -something else for that - passwd usually. - -By default Dovecot uses "dovecot" service, ie. the PAM configuration is in -/etc/pam.d/dovecot file. You can override this by giving the wanted service -name as parameter for pam. For example "auth_passdb = pam dovecot2". If you -give "*" as service name, Dovecot uses "imap" service for IMAP connections -and "pop3" service for POP3 connections. - -Here's an example /etc/pam.d/dovecot configuration file which uses standard -UNIX authentication: - -auth required pam_unix.so nullok -account required pam_unix.so - - -passwd-file ------------ - -This is compatible with regular /etc/passwd, and a password file used by -libpam-pwdfile. It's in the following format: - -user:password:uid:gid:(gecos):home:(shell):flags:mail - -For password database, it's enough to have only user and password fields. -For user database, you need to set also uid, gid and either home or mail. - -Flags is a comma-separated list of flags, currently only recognized value -is "chroot", which makes the imap process chroot into home directory, if -allowed by master process. - -The password field can be in three formats: - - - password: Assume CRYPT scheme - - password[type]: libpam-passwd file compatible format. Type is one of: - 13: CRYPT scheme - 34: MD5 scheme - 56: DIGEST-MD5 scheme (Dovecot extension, deprecated) - - {SCHEME}password - - -LDAP ----- - -See dovecot-ldap.conf for more information. Password and user databases may -use different configuration files to keep the information in separate -locations. If both refer to same file, they share the same LDAP connection. - - -vpopmail --------- - -This is an external software intended to make handling virtual domains -easier. Supports Qmail and Postfix. See http://inter7.com/vpopmail.html - - -static ------- - -static uid= gid= home= - -All users share the same UID and GID. Home directory template can use %u, -%n and %d variables, see default_mail_env description in dovecot-example.conf. - - -PostgreSQL ----------- - -See dovecot-pgsql.conf for more information. Password and user databases may -use different configuration files to keep the information in separate -locations. If both refer to same file, they share the same PostgreSQL -connection. - - -Generating passwords --------------------- - -DES: - mkpasswd - perl -e 'printf "%s\n", crypt("pass", "two-letter-salt")' - -MD5: - mkpasswd --hash=md5 - perl -e 'printf "%s\n", crypt("pass", "\$1\$6-8-letter-salt\$")' - -PLAIN-MD5: - perl -MDigest::MD5 -e 'printf "{PLAIN-MD5}%s\n", Digest::MD5::md5_hex("pass")' - -DIGEST-MD5: - perl -MDigest::MD5 -e 'printf "{DIGEST-MD5}%s\n", Digest::MD5::md5_hex("user:realm:pass")' diff -r 02f7ce1e75f6 -r c2af4ee1f25e doc/configuration.txt --- a/doc/configuration.txt Wed Feb 07 03:45:33 2007 +0200 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,164 +0,0 @@ -Quick setup ------------ - -If you use mbox, make sure that mbox_locks is set up the same way as rest -of your system. - -Check client_workarounds and enable those you think you need. - -If you need to create new SSL certificate, edit dovecot-openssl.cnf and -run mkcert.sh. - -Going through settings in dovecot-example.conf is a good idea, they should -be well commented. - - -Authentication --------------- - -See auth.txt. - - -Maildir or mbox? ----------------- - -Maildir stores each message into a separate file, message flags are stored -into file name. These make maildir very unlikely to get corrupted in any -way. - -Reading lots of mails from maildir is somewhat slower than from mbox, since -each mail file needs to be separately opened. Updating the mailbox however -is much faster than with mbox. - -With larger mailboxes it's a good idea to use a filesystem which uses -b-tree or hash indexes for directories, for example ReiserFS, XFS or JFS. -ext2 and ext3 have some patches to implement this but they're not in Linux -2.4.20 yet. I'm not sure about *BSD's filesystems, FreeBSD's ufs had some -support for hashes. - -mbox is just a single file where new mails are appeneded, flags are stored -in each message's headers. Deleting mails is slow as the rest of the file -needs to be moved over the deleted mail. Changing message flags is usually -quite fast since we use some tricks to avoid copying too much data, but it -may result as well in large data copying. - -Besides the copying being slow, it's also a bit dangerous. If the copying -is aborted (crashed, killed, power lost) the mail file may be left in -somewhat corrupted stated. - -Bottom line: mbox is good for read-only mailboxes, maildir for everything -else. - - -Creating new users ------------------- - -Dovecot is interested in only one thing - being able to find the user's -mail directory. With maildir you need to do mkdir ~user/Maildir, with mbox -mkdir ~user/mail. - - -Chrooting ---------- - -Chrooting can be used for extra security hardening to prevent users from -having full access to the system in case some security hole was found. If -used incorrectly, it can also allow local users to gain root privileges. -This is possible by hardlinking setuid binaries inside the chroot jail and -tricking them. There's at least two possibilities: create your own -chroot/etc/passwd and run /bin/su, or create your own chroot/lib/libc.so -and run any setuid binary. - -If you want chrooting, make sure that no local users can hardlink setuid -binaries inside the jail. The safest way to do this would be to mount those -filesystems with nosuid flag. - - -System with local users ------------------------ - -It's possible to use either the default system passwords or create separate -IMAP passwords using eg. passwd-file authentication. If you use system -passwords, disable_plaintext_auth = yes is a very good idea. - - -System without local users --------------------------- - -First you'll need to decide if you want to use one or multiple system uids. -For example one for everything, one per each virtual domain or one per each -user. In any case the uids should be different than the uids used for other -parts of Dovecot (login or auth processes). - -Having one uid per user would mean that in case of a security hole in -Dovecot, the user still couldn't read other peoples mails. Use this if -possible. - -chrooting imap processes would be good idea, but you should still think -about having the filesystem nosuid-mounted. - - -Performance ------------ - -Usually the bottleneck with IMAP server is disk I/O, so get fast disks and -lots of memory to act as operating system's file cache. - -One performance tweak is to save mails with CR+LFs instead of just LFs. -This can result in faster indexing of mails and smaller CPU usage when -sending mails. With Linux, FreeBSD and Solaris 9 Dovecot can use sendfile() -syscall to send such mails. However extra CRs do increase the mail size, -meaning more I/O and potentially losing the gained performance. You can -enable this for mails saved by Dovecot by setting mail_save_crlf = yes. For -mails saved by your mailer you'll need to do something else, not yet -covered by this documentation. - -COPY command can be made much faster with maildir by setting -maildir_copy_with_hardlinks = yes. This is problematic only if something -modifies the mail in one folder but doesn't want it modified in the others. -I don't know any MUA which would modify mail files directly. IMAP protocol -also requires that the mails don't change, so it would be problematic in -any case. - -Logins can be handled either fast or securely. Doing it securely means -creating a new login process for each connection instead of having only -few processes handling multiple connections. The problem with sharing -connections is that if a security hole is found, the attacker could hijack -other peoples connections or steal their passwords if plaintext -authentication was used (even with SSL/TLS). If you want to be fast, -set login_process_per_connection = no. - -Dovecot's memory usage is very small. Almost all memory usage you see with -ps/top is from mmap()ed files, meaning that operating system can drop any -of those memory pages at any time without needing to swap them. With -Linux/x86 Dovecot usually takes about 70-100kB of non-mmaped memory. Some -commands such as SORT and THREAD will use more memory though (around 700kB -for threading 4600 mails). - - -Rootless Dovecot ----------------- - -It's possible to make Dovecot run under one uid, not requiring root -privileges at any point. This shouldn't be thought of as any security -feature, but instead just as a way for non-admins to run imap server in -their favourite mail server. - -If you do think of this as a good way to achieve security, ask yourself -which is worse: - -a) near-zero possibility to get root privileges, small possibility to get -into system as imapd user chrooted into empty directory without logging in, -small possibility to get logged user's privileges but no possiblity to read -others mails since they're saved with different uid (plus you might be -chrooted to your own mailbox). - -b) zero possibility to get root privileges through Dovecot, small -possibility to get into system as mail user, possibly even without logging -in, and being able to read everyone's mail (and finally getting roots by -exploiting some local just discovered vulnerability, unless you bothered to -set up special chroot environment). - -Anyway, doing it is easy. configure --prefix=$HOME, make install, change -login_user and auth_user in dovecot.conf to your user id, disable all -chrooting and use passwd-file authentication. diff -r 02f7ce1e75f6 -r c2af4ee1f25e doc/design.txt --- a/doc/design.txt Wed Feb 07 03:45:33 2007 +0200 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,130 +0,0 @@ -Design ------- - -Security is the major goal of this project, with reliability coming second. -I also try to keep things fast, extensible and portable. - -Things are broken into multiple processes running with minimal required -privileges. Communication between processes is trusted as little as -possible. Processes running as root are kept as simple as possible even if -it means minor performance hits. - - -imap-master ------------ - -Runs as root. Executes new processes, some by itself and some by request of -another process. The requested processes can never be started as root, and -the allowed UID range can also be limited. - -It's also possible to configure everything to be run under a single UID. -This is useful only if you wish to use imap somewhere you don't have root -access. - - -imap-login ----------- - -Runs as non-privileged user (imapd). Handles accepting new client -connections and parsing commands until the client is authenticated. There -can be either a few of them which handle multiple connections, or one -process per connection. One per connection is much more secure in case it's -ever found to be exploitable, but it's also less efficient. - -SSL and TLS connections are also fully handled by the login process. -Instead of passing the connection's fd to imap process, login creates a new -anonymous UNIX socket and uses it to translate communication between imap -process and the client. If you're using one login process per connection, -this also means that you have actually two processes all the time for an -active SSL IMAP connection. - -Since SSL protocol is quite complex and I'm using gnutls which is still in -beta, it shouldn't be trusted to be fully secure. Using one login process -per connection should however make it quite safe to use, as the login is -running in a chrooted environment without any privileges. However, the -attacker could get your private SSL key.. - -Note that if you let a single login process handle multiple connections, a -security flaw would allow the attacker to see all the other user -connections connecting to it, possibly hijacking them or stealing their -passwords if plaintext authentication was used. - - -imap-auth ---------- - -Runs under minimal required privileges to be able to authenticate users. -In case of shadow passwords or PAM, that's root. Communicates with -imap-login and imap-master to authenticate users. - - * imap-login - - Receives LOGIN or AUTHENTICATE command - - Begins authentication with imap-auth process, with AUTHENTICATE - continuing passing data between client and imap-auth until done - - If successful, we've received a cookie which we send to imap-master - * imap-master - - Requests data from imap-auth for the cookie. Data includes - UID, GID and mail format and mail format specific data (eg. - mail directory). Optionally also receives chroot directory. - - Checks that the UID is in valid range, and that it's allowed - to be chrooted under given directory - - If everything is valid, pass the connection to imap process - - Replies to imap-login whether everything was valid - - If successful, stop processing the connection, imap process takes - care of the rest - - * imap-auth - a) Receives authentication request with given protocol - - Initialize the request in protocol-specific manner - - If failed, send a failure-reply to client - - Otherwise send a cookie to client - b) Receives continued authentication request for given cookie - - Verifies that the cookie is valid, replying if not - - Handle the data in protocol-specific manner - - Reply to client with information whether the authentication - is finished - - Reset cookie expiration time - c) Receives a request to receive data associated to cookie - - Verifies that the cookie is valid, replying if not - - Reply with the data - -Cookies are associated to a specific imap-login process, so one process -cannot steal another one's authentication request by pretending to be it. - - -imap ----- - -Runs non-privileged and optionally chrooted (when it's safe). Since this is -the largest part of the imapd, this is where most of the potential security -flaws are. - -Maildir and mbox formats use a few index files to look up data fast, the -actual mail files aren't opened unless they're really needed. The index -files are accessed using shared memory maps and locked using fcntl(). - -Using memory maps creates a security problem if the file isn't trusted. It -might well be possible to create a buffer overflow in some function by -modifying the file as while it's being used. Currently this should not be a -problem as we require write access to the files ourself, so attacker -shouldn't be able to get any extra privileges by exploiting the imap -process. - -Other than the memory mapping problem, index files are not trusted to -contain valid data. Everything in them is validated before being used. - -Supporting shared mailboxes will be a small problem. We probably shouldn't -even try to support using non-trusted index files but rather create trusted -indexes separately for each user. If however the users don't have direct -access to the indexes files, they could optionally be shared. - - -indexer -------- - -Indexer may be started by master process when it thinks there's some extra -time to be used. It goes through users' mailboxes and compresses or -rebuilds indexes when it sees a need for it. The actual indexing is done by -dropping root privileges just as with imap process. - -Well, this is the plan anyway. The indexer doesn't exist yet. diff -r 02f7ce1e75f6 -r c2af4ee1f25e doc/index.txt --- a/doc/index.txt Wed Feb 07 03:45:33 2007 +0200 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,142 +0,0 @@ -IMAP Index Files ----------------- - -Should support pretty much any mail format, at least maildir and mbox can -be implemented with it. - -The data in the files are stored using native byte order and native type -sizes. That information is saved into index file's "compatibility" fields. -No attempt is made to deal with incompatible index files, they're just -overwritten. - -Indexes are bound to each others by "indexid" field in headers. If they -don't match, the file is assumed to be corrupted and will be rebuilt. - - -Main File ---------- - -.imap.index: Fixed size data - -This is the main index file. It's kept as small as possible so scanning -through it is quite fast operation. - -Index is usually accessed by getting pointer to first record in wanted -range, after that it's just accessed by jumping forward. There may be -deleted records (UID field 0) which are skipped. - -first_hole_position and first_hole_size fields in header specify the first -deleted block in the index file. If we wish to access a message with a -sequence number before the deleted block, we can do it with a simple array -lookup. - -The deleted blocks are compressed when INDEX_COMPRESS_PERCENTAGE of the -file consists of deleted space, 50% by default. Also the indexer process -should try to compress the files when there's extra time, to keep the -sequence lookups faster. - -cache_fields field in header contains the bitmask of fields that should be -saved into index when indexing new messages. It may change at any time, and -the old messages won't (immediately) be updated to reflect the change. - - -Data File ---------- - -.imap.index.data: Variable length data - -Contains the non-critical fields for messages. Each message has a fixed -size "data header" and zero or more variable width fields. It's possible to -add, remove and modify fields, but if doing so would exceed the allocated -space, the whole data block is copied to the end of file. - -Only the total amount of deleted space is remembered, ie. empty blocks in -the middle of file aren't reused. The deleted blocks are compressed when -COMPRESS_PERCENTAGE of the file consists of deleted space, 20% by default. - -This file is actually a very dummy database, and will likely later be -replaced with something smarter (Berkeley DB?). Currently it should be good -enough since there's not much need to insert or update fields, but ANNOTATE -extensions will something better. Although ANNOTATE would require permanent -storage, which index really isn't.. - -One nice TODO idea would be simple compression. Mailboxes contain a lot of -identical mail addresses and subjects, we could simply save one instance of -them. - - -Tree File ---------- - -.imap.index.tree: UID and sequence lookups - -This is a red-black binary in tree file, used to find message record -positions in main index file. TODO: I should rather have used B+tree or -similiar.. - - -Modify Log File ---------------- - -.imap.index.log: External change log - -Communication between two IMAP servers accessing the same mailbox is -usually non-existent. If a change occurs, they both have to go through the -mailbox to see what changed and notify client about it. - -Dovecot uses modify log file to log changes made to index files; currently -message flag changes and expunges. This way only one of the Dovecot -processes has to scan the mailbox, other processes simply check from the -log file what changes occurred. - -All external changes that Dovecot notices (eg. another MUA expunging mails) -are also saved into log file. They can quickly be found from there when -client is ready to be notified of them. - -Message sequence handling is also a bit annoying with IMAP. Since clients -cannot be immediately informed about expunged messages, each client can -have slightly different sequences. The easiest way to deal with this is to -simply allocate an seq_array[] for each client which contains pointers to -messages or message UIDs. This is probably how all other IMAP servers -implement it. - -Dovecot doesn't allocate such array, it simply looks up from the log file -what the differences there are between index sequences and client -sequences. It's almost always none. This was a bit tricky to implement, but -it seems to be working fine now. - -If there's only one Dovecot accessing the mailbox, there's no need to write -to log file. To find out if this is the case, we use fcntl() locking -tricks. Each Dovecot process locks the modify log in shared mode, checking -if we're the only one is then as simple as trying to lock it exclusively. -It's safe because only the main index file is used for real locking. - - -External changes ----------------- - -(Maildir-specific) - -External changes are checked when index file's timestamp is different than -the maildir's timestamp. When modifying the index file, it's timestamp -should be set to maildir directory's timestamp when it was last in a known -synced state. - -There's still the possibility that new mail comes just after we've synced -the dir (or in the middle of it), but that's a bit difficult to fix without -scanning the directory through all the time which gets slow. - -Luckily however this shouldn't be much of a problem, as new mail comes to -new/ directory where it's always noticed. It's only the cur/ directory that -may not always be exactly synced if someone else has been messing up with -it. And if someone else has done that, she most likely has also seen the -mail using that other mail client. - - -Locking -------- - -File locking is done using fcntl(), so currently there's no support for NFS -servers that don't support it. There probably isn't even need to, indexes -could just be saved into local disk even if mailboxes are accessed through -NFS. diff -r 02f7ce1e75f6 -r c2af4ee1f25e doc/mail-storages.txt --- a/doc/mail-storages.txt Wed Feb 07 03:45:33 2007 +0200 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,119 +0,0 @@ -Maildir -------- - -We support Courier-compatible Maildir++. INBOX is the ~/Maildir directory, -all the other folders named as "..." under it, any -directories not starting with a dot are simply ignored. Deleting folders is -made atomic by renaming them as ".." and deleting the -directory after that. So, all directories beginning with ".." are deleted -whenever they're noticed. Indexes are stored into each folder's root -directory. - -Custom flags saved using a..z letters as Maildir file flags. .customflags -file is then used to map the letters to names (a=0, b=1, etc). - - -mbox ----- - -mbox storage requires an IMAP root directory where to store IMAP folders -and some other information. mbox file for INBOX is a special case which may -exist elsewhere in filesystem. - -All files in the IMAP root directory not beginning with a "." are treated -as IMAP folders. Directories are treated as being IMAP subfolders. - -Dovecot's indexes are stored in ".imap//" directories under -the folder directories. - -MAIL environment for mbox is: |[:INBOX=]. -For example ~/Mail:INBOX=/var/mail/username. - -indexes -------- - -It's possible to specify different location for index files. This could be -wanted if mailboxes are accessed via remote filesystem (eg. NFS). Indexes -are mmap()ed and fcntl() locked, so they may not work too well with NFS, or -at least they'll work a bit slowly. - -Indexes don't contain anything really important in them, so it doesn't -matter if they're lost or are out of sync. Dovecot just rebuilds or -refreshes them. This would allow creating a cluster where each computer -keeps the indexes locally while accessing the actual mailboxes via NFS. To -avoid unnecessarily rebuilding indexes all the time, the users should be -redirected to their primary server whenever possible. - -Adding :INDEX= to MAIL environment overrides the default location. The -given directory must exist. - -It's also possible to disable index files by building them into memory. -They're also automatically built into memory if the index files can't be -opened or created for some reason (eg. out of disk space). To disable -indexes always, add :INDEX=MEMORY to MAIL environment. - - -Detecting what to use ---------------------- - -imap process detects the storage from MAIL-environment. It's preferred -format is ":", for example "maildir:~/Maildir". It's anyway -allowed to be in pretty much any format as long as some of the storages -recognizes it as a valid data, so for example "MAIL=~/mail" is first -checked by maildir storage to see if it's valid maildir and mbox storage -after that. - -If the MAIL environment isn't given at all, all storages are gone through -and asked if they can find a usable mail folder. Also as a special case, if -MAILDIR environment exists, maildir storage is used with the directory -specified in it. - -Maildir storage is autodetected by checking if /cur/ exists and -we have rwx access to it. If directory isn't known, / and ~/Maildir are -checked. / is checked because we could be chrooted. - -mbox storage requires detecting INBOX location and IMAP root directory. If -MAIL environment points to file (+rw) it's treated as the INBOX file. If it -points to directory (+rwx) which contains .imap/ (+rwx), inbox (+rw) or -mbox (+rw) it's treated as IMAP root directory. If INBOX isn't known, -imap_root/inbox file is used by default. If root directory isn't known, we -try to find it from /, ~/mail or ~/Mail. - - -Files created by Dovecot ------------------------- - -.subscriptions: One folder per line listing subscribed IMAP folders. -There's only one of these files in the root mail directory (~/Maildir/ or -~/mail). - -Then there's the per-folder files: - -.imap.index: Fixed width mail index records. - -.imap.index.data: Variable width data, for example mail file name and -cached body, envelope, etc. - -.imap.index.log: Log file describing made changes. This file is used to -mark external changes (expunges and flag changes) temporarily until they're -synced into IMAP clients. If multiple Dovecots have the same folder open, -this can be a big speedup. - -.imap.index.tree: Binary tree used for looking up messages from .imap.index -file. This is used for both UID lookups and message sequence lookups. -Sequences can be looked up directly from the index as long as no messages -have been expunged, but after that it gets slow unless it's compressed. -Using this file however can keep the sequence lookups and expunges fast in -all situations without requiring compression. - -.customflags: Lists custom message flags. First line is header which -Dovecots check to see if the file has changed. Other lines list the flags -in format " ". Currently there can be 26 different -flags (0..25). - -.imap.index* files can be deleted without any data loss, they're simply -recreated/rebuilt when they're noticed missing. - -Deleting .customflags results in loss of custom flag names for Maildir. -With mbox the .customflags is simply recreated based on the flags listed -in the mbox file itself (ie. no data loss). diff -r 02f7ce1e75f6 -r c2af4ee1f25e doc/multiaccess.txt --- a/doc/multiaccess.txt Wed Feb 07 03:45:33 2007 +0200 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,16 +0,0 @@ -RFC2180 describes several ways to handle multiple client access in same -mailbox. Here's how we've implemented them. - -DELETE and RENAME commands work always, other clients that had the mailbox -selected are disconnected next time they try to use a selected-state -command. - -FETCH command sends replies to all non-expunged messages and if any -expunged messages were referenced, replies with a tagged NO. - -STORE with .SILENT silently ignores expunged messages. Without .SILENT it's -handled the same way as FETCH. - -SEARCH ignores expunged messages. - -COPY fails if any of the given messages were expunged. diff -r 02f7ce1e75f6 -r c2af4ee1f25e doc/nfs.txt --- a/doc/nfs.txt Wed Feb 07 03:45:33 2007 +0200 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,68 +0,0 @@ -Index files can be stored over NFS by setting index_mmap_invalidate = yes. -This may not be such a good idea however since they are not cached locally -at all. Rereading them constantly over network can be more inefficient than -simply accessing the mail files directly. - -Better way is to store indexes locally while still keeping the mailboxes -stored over NFS. The local index doesn't have to be fully synced all the -time so it doesn't matter if you use a couple of machines to access the -mailbox. In clustered environment it'd be better to try to keep the users -directed into same machine whenever possible. - -So append either :INDEX=MEMORY or :INDEX=/local/var/indexes/%u to -default_mail_env. - - - .customflags files requires fcntl() locking currently. Many NFS servers - don't support it, at least without a separate lockd daemon. This will be - fixed later to be NFS-safe by default. .customflags belongs to indexes - with mbox, so this is a real problem only with Maildir. - - - gethostname() must return different name for each IMAP server accessing - a user's mailboxes - - - Clocks should be synchronized or things can start to fail. - - -If you _really_ wish to try using indexes via NFS: - - - Set index_mmap_invalidate = yes - - - Indexes require fcntl() locking. .lock files would be pretty slow, but - possible except for modify log. - - -Ideas how to make indexes work pretty well with NFS: - -Reading shouldn't require locks, so modifying should be done using only -atomic operations. These include: - - - Replacing the file completely using rename() - - We probably can't assume that writing more than one byte at a time is - atomic. If we have to modify a larger dataset, we could do: - - struct { bit_t use_first; dataset_t first; dataset_t second; } - - when reading, we use first if use_first is set, second if it's unset - - when writing, we first write to the non-used variable, only then we - update the flag. - - This of course requires twice the amount of space for dataset plus - one extra bit, so it shouldn't be used too much - - If data can be only set, but never changed, we need only one extra bit - to specify if the data is set. - - Appending new data to end of file. We'd have to have used_file_size - variable in header, done like described above. - - Each cached message record would have a pointer to next part, so more - cached data could be appended to file. - -Message flags are the most commonly modified data. If we just modify them -directly, a simultaneous read might catch the change only partially. But -luckily for us, this is accepted behaviour so we can do it. - -Another commonly modified data is maildir filenames. We probably want to -store only the base name in index and keep the full name synchronized only -locally. - -Compressing unused data from index files would have to be done by rewriting -the index into index.lock file and renaming it over the index file. - -All file operations should probably be done with lseek(), read() and -write() to avoid extra network traffic. There should be some clever -read-ahead caching however.