Mercurial > dovecot > original-hg > dovecot-1.2
view TODO @ 2375:faeb3d17579a HEAD
updated
author | Timo Sirainen <tss@iki.fi> |
---|---|
date | Tue, 27 Jul 2004 09:39:42 +0300 |
parents | 11c1f1203b65 |
children | 689e4235681f |
line wrap: on
line source
- mbox - complain if mbox isn't in mbox format.. and if we're relying on from_offsets and they point to invalid location, complain about that too - does it work: two adjacent From-lines breaks us. - Handle UW imapd's "DON'T DELETE THIS MESSAGE" message. - keep mbox lock for two extra seconds after sync - don't force dotlock to be first in mbox locking.. - istream-mbox breaks with 0 sized body - move /var/mail/user to ~/mbox if ~/mbox exists.. supposedly this could be useful if /var/mail doesn't have quota, but ~/mail does. now, what do we then do if we can move only some of the mails?.. - if we can't create dotlock file for mbox, make sure it still can be selected in read-only state - pop3: lock the mbox for the whole duration we exist - recent counters - mbox: not implemented - maildir: new mails in cur/ aren't \recent! - recent counter may be messed up as it shows recent for messages that have already been expunged, or messages that we don't see yet.. should it be view-specific in some way? .. - index - save extra data mappings in index file and use them rather than just relying that they're registered in right order - optimize initial left_idx in mail_index_lookup_uid_range() - when referencing log files, open them immediately? - if log file is lost, generate it from old and new index - make sure when changing flags multiple times in transaction it goes ok. especially replace + dirty flag - post-login-sql-command - CAPABILITY/CAPA should wait until dovecot-auth is connected (does it?) - add setting: ssl_options = bitmask. by default we enable all openssl workarounds, this could be used to disable some of them - we probably shouldn't use alarm() constantly as it can cause random problems.. it's mostly useful for timeouting while writing data to clients. maybe rather just use poll() even though it causes more syscalls. - undirty dirty flags when syncing - external mail move from INBOX -> box2 -> INBOX. if filename stays same, we'll get "inserting mail with UID xx in the middle of the mailbox".. Does this really happen? - open("/var/run/dovecot//ssl-parameters.dat", O_RDONLY|O_LARGEFILE) =-1 ENOENT ^ loops forever - If postgresql starts after dovecot, dovecot doesn't reconnect to it.. same for LDAP? - don't return \* in PERMANENTFLAGS when we're full - mail_cache_delete() is never called - support read-only logins. user could with alternative password get only read-access to mails so mails could be read relatively safely with untrusted computers. Maybe always send [ALERT] about the previous read-only login time with IP? - dovecot --exec-mail imap - ~/.dovecotrc to override system wide settings. namespace settings should override all the previous namespace settings instead of adding new. - initial caching decisions: - we need to know if we want to cache stuff when it's accessed the first time (with only one client with local cache it's not needed) - get them from INBOX - master process could remember the globally most common decisions - SIGHUPing master crashes - lines longer than input buffer size causes breakages, at least in header parser. - we could process multiple commands (especially command + implicit sync after it) without dropping (index) lock in the middle. that'd mean all commands should be able to begin in any lock state and they shouldn't drop the lock. there'd be some new drop_lock() function which finally does that. - fix cygwin compile? _close and _read in *stream*.c fails. - mailbox_check_interval: we shouldn't send anything if write() returns EAGAIN. not all clients are reading input all the time and there's no point in sending updates when no-one is reading them. - don't break if we rename selected mailbox - handle out of quota conditions: - if dovecot-uidlist can't be written, assume the new mails have UIDs beginning from uidlist.next_uid. Whenever mails are expunged, overwrite the next_uid field with the current highest next_uid. Whenever we have assumed UIDs and uidlist gets updated, throw the client out with "inconsist mailbox". - make sure all syscalls check for ENOSPACE (and ENOACCESS while at it) - NFS safety: - .customflags: use rename(), but there's a problem when we have to remove unused flags to make room for new ones. to fix that add new field in the file, it would be set for flags which are currently unused. if that field is set, .customflags must be locked before the flag is set to any messages. but make sure there's no race conditions, we probably have to wait a few seconds just to make sure no-one set a flag we want to remove - ESTALE - send client IP immediately after accept() to master process. make sure master shows the IP if login dies unexpectedly. master should probably also kill the login process if it doesn't kill itself soon enough.. or maybe just log the IP immediately. - ioloop-kqueue.c patch - does dovecot-auth really break when it runs out of fds? - safe_mkdir_path() and create mbox / maildir paths - create auth_chroot dir at startup - maildir: create foo/bar -> OK, delete foo/bar -> error - full_filesystem_access=yes + Maildir + SELECT "INBOX/" works? .. - should we support some non-tokenizing way to parse mail addresses? .. eg. "foo ? bar <x@y>" would now show up as "foo? bar <x@y>" - DELETE/RENAME: when someone else had the mailbox open, we should disconnect it (when stat() fails with ENOENT while syncing). - bugs - SIGHUP didn't update imap_listen. this is a bit annoying to fix though, since new listen() may fail for a few times because login processes may not die immediately.. - SIGHUP doesn't update log file location. - CREATE a, CREATE b, save mails into them, DELETE a, RENAME b a. -> breaks if a+b have same UIDVALIDITY. We could update the UIDVALIDITY for the renamed mailbox and all mailboxes under it. Then return with tagged "OK [NEW-UIDVALIDITY 1234] Renamed". Assuming other IMAP people agree to that. - reliability fixes: - we don't handle out of memory conditions too well, malloc failing kills the process which is good enough (and likely never happens), but mmap() failures aren't handled too well. Rather should be handled in similiar way to locking failures, so that at least we don't try to rebuild the index because of it. - limit folder hierarchy levels? user can now create eg. a/a/a/a/... and then start renaming them from end to beginning, which probably will at some point start causing syscall failures which will fill up logs. - dotlock overriding is racy, but it's pretty difficult to fix it. Also overriding someone else's dotlock in shared folder isn't possible. These could be fixed by having separate lock process running as root, which would chown() the file for another uid and then unlink() it as that user. One problem with that is that if malicious user sets setuid+execute bits on for the file, he could run the file and get changed to the new uid. That hopefully shouldn't matter much since the new uid should be user with minimum possible privileges. Anyway, optional.. - checks: - make sure SELECT rebuilds index properly when next_uid is near 32bit value - make sure connection limits work - check if some asserts should be changed into if/i_panic code, so that disabling asserts would be possible - enhancements: - option to disable SORT, SEARCH and other memory/cpu-intensive features. defaults and per-user by dovecot-auth. - maildir could support also the dirty-flag in messages. files would be renamed "whenever there's time" (that'd require the indexer program, or forking and doing it in background) - OpenSSL: support generated DH parameters - SSL: Support password protected key files. Support reading the password from user at runtime (dovecot startssl or something). currently it just hangs. - check with strace what dovecot does when evolution checks new mail, it's quite a lot. some things probably wouldn't need to be done (mkdirs inbox) and other things could be cached in memory. - sort: we could create alternative indexes for different sort conditions. sort code itself already supports this optimization. - use vsftpd-like safebufs, ie. keep non-rwx page before and after the memory we want to use. - mmap_anon() - mmap()ing files would probably need to first go through anon_mmap() and then use MAP_FIXED. annoying that it slows the mmaping.. - data stack should use mmap_anon() - support zlib compressed mbox/maildir? mbox maybe just read-only. do it through istream-zlib wrapper - THREAD=ORDEREDSUBJECT - although pretty useless I'd think. - POP3: add support for LAST command, but make it optional since it has to play with \Seen flags (RSET would unset them all with it enabled, not very nice). - logging - Login: username 1.2.3.4:1025 5.6.7.8:993 imaps,compressed - Logout: username 1.2.3.4:1025 5.6.7.8:993 imaps,compressed in:1000 out:1000000 - n failed login attepts (before failure or success, once in n seconds) - lib-charset: - utf8_toupper() is a must. and a bit difficult if we want to do it right. - add support for other things than iconv() as well? we could reuse the code from cyrus or courier - cache iconvs? they'd probably be faster if we just reset the conversion instead of opening new one every time. and there will likely be only one or two charsets which are used for nearly all conversions. - should we allow following symlinks in mbox/maildirs? they are now. - if we implement shared mailboxes with shared indexes, never do that or others could symlink your personal mailboxes and see the indexes created for it which may contain envelope etc. data - this allows circular mailbox hierarchies which should be prevented by eg. allowing max. 20 hierarchies. - index: - we could try compressing same from/to/subject fields into a single location in data file. requires larger changes.. - Most messages are text/plain/7bit/us-ascii/no-other-content-type-params. Instead of saving tons of nearly identical BODY/BODYSTRUCTURE fields into cache, we could just set INDEX_MAIL_FLAG_TEXT_PLAIN_7BIT_ASCII bit on and generate the bodystructure for such messages on the fly. - read-only support for mailboxes where we don't have write-access - if .customflags is removed and Maildir files have custom flags, add "unknown1" "unknown2" etc. flags to .customflags file for each found flag - support storing message headers into indexes. this could be useful when indexes are in local disk but actual mails are accessed through NFS. - cost-based caching to indexes? when accessing the indexes, save how much extra data we had to read because we did/didn't cache some field, and also how much extra data we avoided reading because we had it cached. but is it fast enough to find out the lengths without uselessly slowing down? - if we wanted to support huge mailboxes with small memory usage, it'd now be possible if we just instead of mmap()ing the whole index files would have maybe 3-4 256k mmap()ed areas which we move based on the need. - should work fine with imap.index and imap.index.cache - log files aren't affected by mailbox size - but, is it worth the trouble really? the OS can do all this itself, only thing we're doing is keeping the processes virtual memory usage small. - SEARCH: - message_body_search() could accept multiple search keywords so we wouldn't need to call it separately for each one (so we wouldn't need to parse the message multiple times). - message_body_search() could support NULL MessagePart and the searching could be done while parsing the message. this would need changes to message_parse() as well. - could optionally support scanning inside file attachments and use plugins to extract text out of them (word, excel, pdf, etc. etc.) - use a trie index for fast text searching, like cyrus squat? - Create our own extension: When searching with TEXT/BODY, return the message text surrounding the keywords just like web search engines do. like: SEARCH X-PRINT-MATCHES TEXT "hello" -> * SEARCH 1 "He said: Hello world!" 2 "Hello, I'm ...". This would be especially useful with the above attachment scanning. - general: - sieve (rfc3028), we can use Cyrus Sieve - rfc2231 continuation support - rfc2557 support for BODYSTRUCTURE, as specified by RFC3501 - lmtp server - is it needed? dovecot-deliver binary at least would be useful - create indexer binary - support Maildir++ quota - some kind of IMAP proxy for load distributing - maybe give more untagged NO/ALERT replies? like when mailbox is in inconsistent state. and when UIDs are reordered because they're too large. - imap/ and lib-imap/ should allow infinite number of custom flags, it's storage's problem if it can't handle too many of them. - cleanups: - check if t_push()/t_pop() should be added somewhere - try to fix @UNSAFE code to use buffer API instead - subscription-file.c, custom_flags - [io]stream-file.c? - grep for FIXME - auth / login: - kchuid, SRP - Digest-MD5: support integrity protection, and maybe crypting. Do it through login process like SSL is done? - dovecot-auth should limit how fast authentication requests are allowed from login processes. especially if there's one login/connection the speed should be something like once/sec. also limit how fast to accept new connections. capabilities: - preferrably all should be possible to #ifdef away by a configure option (--without-capabilities=acl,namespace,...) - possibility to disable them from config file - acl (rfc2086, draft-ietf-imapext-acl), namespace (rfc2342) - probably do it like cyrus. "user.<username>" to access other users, with "" defaulting to "user.<myself>". these should be configurable however. - shared namespaces? maybe configurable in config file - easiest way to do ACL would be to use unix modes, but is that useful at all? Well, ACL2 has a bit better support for that, so maybe we could support it. - otherwise gets a bit trickly, we could keep all mail in "imapmail" group and 0600/0700 mode by default, but when mail is shared to others, the group read/write access bits would be set. or alternatively we could launch another imap process to handle it, which we should support anyway. ACLs could be stored into ".acl" ascii file in each folder. - support for private and shared flags, configurable by mailbox admin. this isn't in any draft yet, but ACL2 author was going to create one. [SHAREDFLAGS (...)] would specify which ones are shared, don't know yet how they would be configured. - quota (rfc2087, draft-cridland-imap-quota) - give filesystem values only to admins - support for Maildir++, probably no need to support more. quota capability supports complex quota configuration, but if no mailer supports them we probably shouldn't bother either - id (rfc2971) - must be configurable what gets sent, default to only name=Dovecot - separate pre/post-login settings - optionally log configured parts of the client information, but only once, probably at the same time as logging "Logged in", "Disconnected", etc. - remember to force truncating values longer than 30 chars, especially before logging - mailbox-referrals (rfc2193) - this is useful whenever we would otherwise need to make the connection ourself. for example load balancing and shared mailboxes requiring another UID to run. - this rfc defines no exact way for server to detect if client supports referrals or not. I don't think there's much point in supporting only referrals, as most clients don't support them. Instead we should return referrals when we know that client supports them, otherwise do the connecting ourself. If client issues RLIST or RLSUB command, it's safe to assume it supports referrals. - for load balancing this works just fine, but what about shared mailboxes which require different UID? If we login with our own username, we end up with our own UID instead of what we wanted. IMAP URLs don't support separated authorization id which would have made this very easy.. We could give the "userid@group" as userid, but clients probably treat it as different userid and ask the password again. - problems, problems, .. maybe not worth the trouble. - uidplus (rfc2359) - uid expunge: no problem - append, copy: oh no. these would slow down things and make handling them much more difficult. currently we just store the mails to destination mailbox without touching the indexes. since we'd need to know their final UID, we'd have to lock the indexes and mbox) fsck() first and append() next to find out the uid, maildir) move the mail directly into cur/ and index it. - except, the index updating is faster than requiring to sync the changed mailbox. we'll probably create such deliver-binary anyway, so it can't be that difficult to code. - drafts: - http://www.imc.org/ids.html - annotate (draft-ietf-imapext-annotate) - per-message annotations. this will be major change. especially because currently there's no suitable storage for them, and they'll probably change all the time.. maybe if we moved into berkeley db to store the .data file and these annotations. - this is separate problem from index files. indexes are treated as temporary files, annotations are permanent data. we'd have to support non-db way to do this too, which would probably be just a simple (slow) text file. - annotatemore (draft-daboo-imap-annotatemore) - server and per-mailbox annotations. much easier than per-message annotations, but they'd be easier to place into db as well. - binary (draft-nerenberg-imap-binary) - perhaps not too useful. I'd like to make Dovecot fully binary-safe though. - view (draft-ietf-imapext-view) - slow, complex, luckily draft expired almost two years ago. i hope i don't have to implement this :) - can be done client-side just fine (evolution's virtual folders)