view TODO @ 29:e9375147c0cb HEAD

Added write_full() which is a simple wrapper around write() meant for writing into files. When there's too much deleted data in index files, they're now compressed when the index is being opened.
author Timo Sirainen <tss@iki.fi>
date Mon, 26 Aug 2002 02:46:59 +0300
parents 6491ac40cdf5
children 2ffff61ee5e1
line wrap: on
line source

test:
 - make sure mmap()s work properly with NFS
 - make sure first_hole_records is updated properly
 - make sure locking is done properly when opening/switching modifylog 
 - make sure index->header->flags are updated at correct times
 - make sure SELECT rebuilds index properly when next_uid is near 32bit value
 - make sure io_buffer_send() handles huge inputs properly
 - make sure rfc822_parse_date() works properly
 - make sure imap_match functions work properly
 - make sure connection limits work

index:
 - optimization:
     - optimize so that when all mail is deleted, the indexes will be
       truncated
     - could hash function be better..? like uid*uid? what about changing
       probe strategy from linear to something else?
     - support shrinking hash file when it becomes 99% empty or so
     - save part sizes + positions so MessagePart could be regenerated from
       index..? probably only needed if BODY/BODYSTRUCTURE is saved, as it's
       only useful with FETCH BODY[mime.sections] and they wouldn't be known
       without bodystructure..
     - index->lookup_uid_range(): first_uid could quite often be either the
       first UID or some UIDs below the first. optimize these by remembering
       the first UID in index.
 - mbox:
     - BUG: adding new mail after indexes are created doesn't work
     - update Status and X-Status headers when flags are changed.
       how? probably by just writing a new mbox file at close time.
     - EXPUNGE doesn't delete the mail from the mbox file
 - there's some race condition issues when opening mailboxes..
 - set_lock() is ugly and horrible and should really be done something.
   does the syncing really need to be there? maybe put it into separate
   function which can be called after set_lock() by functions which actually
   care about the sync state (fetch, search, store, etc).
 - read-only support for mailboxes where we don't have write-access? Could be
   a bigger job. At least the mmap()ed file contents (== pretty much
   everything) can't be trusted since they might change at any time.
 - if index was just rebuilt, modify log complains about indexid mismatch
   at first open
 - does append work?

lib-storage:
 - support multiple mailbox formats and locations for one user. that would
   require support for multiple MailStorages, and since we're chroot()ed,
   usually the only way to communicate with others would be to create
   RemoteMailStorage which would use TCP/UNIX sockets to connect to another 
   imap session.
 - DELETE/RENAME: when someone else had the mailbox open, we should
   disconnect it (when stat() fails with ENOENT while syncing)
 - optimize SEARCH [UN]SEEN, [UN]DELETED and [UN]RECENT. They're able to
   skip lots of messages based on the index header data.
 - use a trie index for fast text searching, like cyrus squat?
 - hardlink-COPY doesn't copy flags
 - maildir: atomic COPY could be done by setting a "temporary" flag into the
   file's name. once copying is done, set an ignore-temporary field into
   index's header. at next sync the temporary flag will be removed.
 - mbox: should we bother checking if mbox file doesn't end with \n when 
   appending?
 - select "" shouldn't work.

general:
 - capabilities:
     - acl (rfc2086)
     - quota (rfc2087)
     - namespace (rfc2342), id (rfc2971), mailbox-referrals (rfc2193),
       literal+ (rfc2088), idle (rfc2177), uidplus (rfc2359)
     - drafts: listext, children, unselect, multiappend, annotatemore
         - sort, thread: are these really useful for clients? do any actually
	   use them? i'd think most clients want to know all the messages
	   anyway and can do the sorting/threading themselves.
         - http://www.imc.org/ids.html
 - rfc-2231 continuation support
 - "UID FETCH|SEARCH|STORE *" doesn't work if latest message was deleted.
   should we bother to fix this? I doubt there's a client that would use this.
 - RENAME INBOX isn't atomic with Maildir. And in general, RENAME can't
   be moved to another storage. Maybe support doing also using COPY + delete
   once COPY is atomic?

 - go through .temp files and delete them
 - if auth process died and login couldn't immediately reconnect to it, it's
   left until next user connects. however the connection needs to read the
   init data before it can be used, so the user gets "NO Unknown
   authentication method" error the first time
 - ulimit / setrlimit() should be set somewhere
 - create indexer binary
 - SEARCH CHARSET support, iconv()?
 - Fix the blocking SSL handshake (req. gnutls 0.5.2)
 - SRP authentication support?
 - Digest-MD5: support integrity protection, and maybe crypting. Do it
   through imap-login like SSL is done?
 - imap-auth should limit how fast authentication requests are allowed from
   login processes. especially if there's one login/connection the speed
   should be something like once/sec.
 - support executing each login in it's own process, so if an exploit is ever
   found from it, the attacker can't see other users' passwords
 - put IMAP_LOGFILE into config file. and the timestamp format.
 - SIGHUPing master should reload the configuration
 - Something's wrong with expunging mails from maildir ..
 - PAM: support some options so /etc/passwd-lookup isn't needed. uid=x, gid=y,
   mailroot=/var/mail. maildirs should be then created when needed
 - index_rec->full_virtual_size could be 0 to indicate it's not calculated
   yet. this way we don't need to read the messages fully just to find out
   that.
 - ability to automatically try again if some command fails because error
   occured in the middle of it, but was most likely also fixed. for example
   if mbox is compressed in the middle of FETCH operation or some index
   corruption was noticed

cleanups / checks:
 - grep for FIXME
 - check if t_push()/t_pop() should be added somewhere
 - check that (off_t) castings are safe
 - IOBuffer is a bit confusing and weird. especially the offset-parameter
   works strangely.. And maybe split it into IBuffer and OBuffer?
 - io_buffer_fd_ref() .. unref() and destroy() would close if refcount = 0?
   annoying those close(inbuf->fd)s with open_mail()..
 - allocating readwrite pools now just uses system_pool .. so pool_unref()
   can't free memory used by it .. what to do about it? at least count the
   malloc/free calls and make complain if at the exit they don't match
 - Make sure messages of size INT_MAX..UINT_MAX (and more) work correctly.
   virtual_size can also overflow making it less than physical_size

optional optimizations:
 - provide some helper binary to save new mail into mailboxes with CR+LF
   line breaks?
 - disk I/O is the biggest problem, so split the mail into multiple computers
   based on user and have a proxy in the front redirecting the connection.
   cyrus had something like this except a lot more complicated - it tried
   to fix the problem of having shared mailboxes. we have the same problem
   with local shared mailboxes as we chroot(), so locally we could communicate
   with UNIX sockets, remotely that could be done with TCP sockets.