view TODO @ 232:9277e893304e HEAD

s/io_buffer_read_data/io_buffer_read_data_blocking/ and fixed the various kludges elsewhere to use it more easily
author Timo Sirainen <tss@iki.fi>
date Sun, 15 Sep 2002 14:09:08 +0300
parents fb7500933a11
children 1fe8eae6fd89
line wrap: on
line source


test:
 - make sure mmap()s work properly with NFS
 - make sure locking is done properly when opening/switching modifylog 
 - make sure index->header->flags are updated at correct times
 - make sure SELECT rebuilds index properly when next_uid is near 32bit value
 - make sure rfc822_parse_date() works properly
 - make sure imap_match functions work properly
 - make sure connection limits work

index:
 - optimization:
     - could hash function be better..? like uid*uid? what about changing
       probe strategy from linear to something else?
     - support shrinking hash file when it becomes 99% empty or so
     - save part sizes + positions so MessagePart could be regenerated from
       index..? probably only needed if BODY/BODYSTRUCTURE is saved, as it's
       only useful with FETCH BODY[mime.sections] and they wouldn't be known
       without bodystructure..
     - index->lookup_uid_range(): first_uid could quite often be either the
       first UID or some UIDs below the first. optimize these by remembering
       the first UID in index.
     - index_rec->full_virtual_size could be 0 to indicate it's not calculated
       yet. this way we don't need to read the messages fully just to find out
       that.
 - mbox:
     - update Status and X-Status headers when flags are changed.
       how? probably by just writing a new mbox file at close time.
     - EXPUNGE doesn't delete the mail from the mbox file
     - if a file isn't valid mbox and it's tried to be opened, say it in one
       line in error log, not 6..
     - if we read-locked mbox file when we're accessing it, we could get it
       pretty reliable.. do other MUAs do that? if yes, drop the dotlock
       and only support flock() or fcntl() locking.
 - there's some race condition issues when opening mailboxes..
 - set_lock() is ugly and horrible and should really be done something.
   does the syncing really need to be there? maybe put it into separate
   function which can be called after set_lock() by functions which actually
   care about the sync state (fetch, search, store, etc).
 - read-only support for mailboxes where we don't have write-access? Maybe,
   but don't try to use their indexes since that's way too problematic, and
   probably even impossible since we can't lock it.
 - if index was just rebuilt, modify log complains about indexid mismatch
   at first open
 - does append work?
 - "UID FETCH|SEARCH|STORE *" doesn't work if latest message was deleted.
   should we bother to fix this? I doubt there's a client that would use this.

lib-storage:
 - support multiple mailbox formats and locations for one user. that would
   require support for multiple MailStorages, and since we're chroot()ed,
   usually the only way to communicate with others would be to create
   RemoteMailStorage which would use TCP/UNIX sockets to connect to another 
   imap session.
 - DELETE/RENAME: when someone else had the mailbox open, we should
   disconnect it (when stat() fails with ENOENT while syncing)
 - optimize SEARCH [UN]SEEN, [UN]DELETED and [UN]RECENT. They're able to
   skip lots of messages based on the index header data.
 - use a trie index for fast text searching, like cyrus squat?
 - BUG: hardlink-COPY doesn't work right:
     - it should generate new filename for destination folder, so copying
       same message twice won't break it
     - custom flags aren't copied
 - maildir: atomic COPY could be done by setting a "temporary" flag into the
   file's name. once copying is done, set an ignore-temporary field into
   index's header. at next sync the temporary flag will be removed.
 - we should probably do some light checking that appended mails actually
   look like valid rfc822 mails..
 - SEARCH CHARSET support, iconv()?
 - SEARCH could optionally support scanning inside file attachments and use
   plugins to extract text out of them (word, excel, pdf, etc. etc.)
 - RENAME INBOX isn't atomic with Maildir. And in general, RENAME can't
   be moved to another storage. Maybe support doing also using COPY + delete
   once COPY is atomic?

general:
 - capabilities:
     - acl (rfc2086)
     - quota (rfc2087)
     - namespace (rfc2342), id (rfc2971), mailbox-referrals (rfc2193),
       literal+ (rfc2088), idle (rfc2177), uidplus (rfc2359)
     - drafts: listext, children, unselect, multiappend, annotatemore
         - sort, thread: are these really useful for clients? do any actually
	   use them? i'd think most clients want to know all the messages
	   anyway and can do the sorting/threading themselves.
         - http://www.imc.org/ids.html
 - rfc-2231 continuation support

 - go through .temp files and delete them
 - Content-Language isn't parsed correctly
 - ulimit / setrlimit() should be set somewhere
 - create indexer binary
 - SIGHUPing master should reload the configuration
 - users should always be able to delete mail from mailbox, even if their
   quota is completely full. this would require us to create the indexes
   elsewhere .. in-memory should work fine?
 - if index was rebuilt (because corruption was noticed), the user should be
   disconnected because everything might have changed

auth / login:
 - SRP authentication support?
 - PAM: support some options so /etc/passwd-lookup isn't needed. uid=x, gid=y,
   mailroot=/var/mail. maildirs should be then created when needed
 - vpopmail support
 - Digest-MD5: support integrity protection, and maybe crypting. Do it
   through imap-login like SSL is done?
 - imap-auth should limit how fast authentication requests are allowed from
   login processes. especially if there's one login/connection the speed
   should be something like once/sec.
 - HIGH: support executing each login in it's own process, so if an exploit
   is ever found from it, the attacker can't see other users' passwords.
    - master should limit number of login processes to max_logging_users,
      killing old processes when limit is reached
    - master should try to keep login_processes_count extra processes all
      the time
    - login should notify master after it accept()s, and it must close the
      listening socket immediately

cleanups / checks:
 - grep for FIXME
 - check if t_push()/t_pop() should be added somewhere
 - IOBuffer should probably be split into IBuffer and OBuffer, and maybe
   making it's internals hidden .. or at least only partly visible.
 - io_buffer_fd_ref() .. unref() and destroy() would close if refcount = 0?
   annoying those close(inbuf->fd)s with open_mail()..
 - allocating readwrite pools now just uses system_pool .. so pool_unref()
   can't free memory used by it .. what to do about it? at least count the
   malloc/free calls and make complain if at the exit they don't match
 - ..wonder what it would look like if I did s/FooBarBaz/struct foo_bar_baz/..
 - HIGH: Make sure messages of size INT_MAX..UINT_MAX (and more) work
   correctly. virtual_size can also overflow making it less than physical_size
 - verify memory alignment is valid when reading from index files
 - create env_put() and env_clean()

optional optimizations:
 - provide some helper binary to save new mail into mailboxes with CR+LF
   line breaks?
 - disk I/O is the biggest problem, so split the mail into multiple computers
   based on user and have a proxy in the front redirecting the connection.
   cyrus had something like this except a lot more complicated - it tried
   to fix the problem of having shared mailboxes. we have the same problem
   with local shared mailboxes as we chroot(), so locally we could communicate
   with UNIX sockets, remotely that could be done with TCP sockets.