--- a/TODO	Mon Sep 16 06:11:40 2002 +0300
+++ b/TODO	Mon Sep 16 06:13:10 2002 +0300
@@ -1,4 +1,3 @@
-
 test:
  - make sure mmap()s work properly with NFS
  - make sure locking is done properly when opening/switching modifylog 
@@ -7,44 +6,61 @@
  - make sure rfc822_parse_date() works properly
  - make sure imap_match functions work properly
  - make sure connection limits work
+ - make sure it's noticed by other processes if a) data file is compressed,
+   b) hash is rebuilt
+ - make sure the index's ftruncate stuff works
+ - make sure modify log works properly, especially switching the files
 
 index:
  - optimization:
      - could hash function be better..? like uid*uid? what about changing
        probe strategy from linear to something else?
      - support shrinking hash file when it becomes 99% empty or so
-     - save part sizes + positions so MessagePart could be regenerated from
-       index..? probably only needed if BODY/BODYSTRUCTURE is saved, as it's
-       only useful with FETCH BODY[mime.sections] and they wouldn't be known
-       without bodystructure..
-     - index->lookup_uid_range(): first_uid could quite often be either the
-       first UID or some UIDs below the first. optimize these by remembering
-       the first UID in index.
-     - index_rec->full_virtual_size could be 0 to indicate it's not calculated
-       yet. this way we don't need to read the messages fully just to find out
-       that.
+     - if first_hole_records == MAIL_INDEX_RECORD_COUNT() -
+       header->messages_count, we know we can just skip over the hole and do
+       another direct lookup there
+     - we could use tree structure to keep track of seqnumbers.. each node
+       would store how many subnodes it has. deleting nodes (mails) would just
+       update those counts. this increases the cost of lookups/inserts/deletions
+       but is faster when more than one hole appears in file.. is it worth it?
+       maybe #ifdefed away. except we could get rid of the hash file with this
+       as well, since it could be used to look for both sequences and uids. it
+       also speeds up UID range lookups when the first UIDs don't exist. use
+       right-threaded redblack/avl trees (we need to know all child node counts,
+       does that affect redblack's performance?)
  - mbox:
-     - update Status and X-Status headers when flags are changed.
-       how? probably by just writing a new mbox file at close time.
-     - EXPUNGE doesn't delete the mail from the mbox file
      - if a file isn't valid mbox and it's tried to be opened, say it in one
        line in error log, not 6..
      - if we read-locked mbox file when we're accessing it, we could get it
        pretty reliable.. do other MUAs do that? if yes, drop the dotlock
-       and only support flock() or fcntl() locking.
- - there's some race condition issues when opening mailboxes..
- - set_lock() is ugly and horrible and should really be done something.
-   does the syncing really need to be there? maybe put it into separate
-   function which can be called after set_lock() by functions which actually
-   care about the sync state (fetch, search, store, etc).
+       and only support flock() or fcntl() locking
+     - don't open() + close() the mbox file all the time, at least while
+       it's being locked..
+     - maybe support Content-Length for figuring out size of text? at least
+       mutt doesn't prefix "From " in outbox.. If we verify that both
+       Content-Length and Lines match correctly, there's quite a little chance
+       that it could be broken by sending them invalid (doesn't local MTA
+       update them anyway?).
+     - rewriting could try to preserve the locations of fields it changes
+       instead of writing them all to end..
+     - mbox-rewrite rename()s the file, which breaks if the original was a
+       symlink. but how do we fix this? we may not have write-access to the
+       directory where it points to, so we'd need to manually copy it..
+ - mbox-append.c changes iobuffer->size .. it's not very good behaviour, but
+   since it calls io_buffer_seek() after it works. but maybe a separate
+   function to do that? io_buffer_limit_mmap_size(IOBuffer *, uoff_t size)
  - read-only support for mailboxes where we don't have write-access? Maybe,
    but don't try to use their indexes since that's way too problematic, and
    probably even impossible since we can't lock it.
- - if index was just rebuilt, modify log complains about indexid mismatch
-   at first open
- - does append work?
- - "UID FETCH|SEARCH|STORE *" doesn't work if latest message was deleted.
-   should we bother to fix this? I doubt there's a client that would use this.
+ - we should try to avoid completely rebuilding indexes unless they're
+   corrupted. especially if we later want to support some read-only boxes
+   and keep the mail flags only in index file. fsck() could verify that
+   records are ok, and that if data file isn't ok the record is deleted.
+ - some fast-open flag for index, which wouldn't trigger cache_update on
+   open (for STATUS command).
+ - if .customflags is removed and Maildir files have custom flags, add
+   "unknown1" "unknown2" etc. flags to .customflags file for each found flag
+ - debug: index could be read-only mmaped when it's not locked. 
 
 lib-storage:
  - support multiple mailbox formats and locations for one user. that would
@@ -66,12 +82,20 @@
    index's header. at next sync the temporary flag will be removed.
  - we should probably do some light checking that appended mails actually
    look like valid rfc822 mails..
- - SEARCH CHARSET support, iconv()?
+ - SEARCH CHARSET support, iconv()? also means we need to parse the charset
+   stuff in headers.
  - SEARCH could optionally support scanning inside file attachments and use
    plugins to extract text out of them (word, excel, pdf, etc. etc.)
  - RENAME INBOX isn't atomic with Maildir. And in general, RENAME can't
-   be moved to another storage. Maybe support doing also using COPY + delete
-   once COPY is atomic?
+   move mails between different storages. Maybe support doing also using
+   COPY + delete once COPY is atomic?
+ - "UID FETCH|SEARCH|STORE *" doesn't work if latest message was deleted.
+ - maybe limit the length of custom flags? we don't really have a problem
+   with them, but with mbox a long X-IMAPbase could break something.. Maybe
+   configurable, default to 50 chars?
+ - "APPEND invalid data {5}" - says "+ OK" and after that says it's invalid.
+   that "+ OK" shouldn't be sent by imap-parser if LITERAL_SIZE is used..
+ - SEARCH should use imap-msgcache, especially for size checking
 
 general:
  - capabilities:
@@ -84,18 +108,36 @@
 	   use them? i'd think most clients want to know all the messages
 	   anyway and can do the sorting/threading themselves.
          - http://www.imc.org/ids.html
+ - sieve? (rfc-3028)
  - rfc-2231 continuation support
 
  - go through .temp files and delete them
  - Content-Language isn't parsed correctly
- - ulimit / setrlimit() should be set somewhere
+ - ulimit / setrlimit() should be set somewhere for imap process
  - create indexer binary
- - SIGHUPing master should reload the configuration
+ - SIGHUPing master should reload the configuration .. killing imap-auth and
+   imap-login processes? or just signal imap-login to stop accepting new
+   connections and let it kill itself
  - users should always be able to delete mail from mailbox, even if their
    quota is completely full. this would require us to create the indexes
    elsewhere .. in-memory should work fine?
  - if index was rebuilt (because corruption was noticed), the user should be
-   disconnected because everything might have changed
+   disconnected because everything might have changed (unless it's noticed
+   while just opening the indexes).
+ - settings for specifying what sort of data to cache by default
+   (index->cache_fields)
+ - setting for choosing mbox locking method
+ - maybe a bit more verbose warnings for some errors, like "invalid date:
+   <date that was tried>". easier than sniffing the traffic.
+ - imap-login writes UTC timestamps to log file .. why is that?
+ - imap-login leaks I/O descriptors when killed (ssl_input + plain_input)
+ - logins are always sent now using syslog(), we'd need to have i_info()
+   or something so they could also be written to log files.. also make it
+   possible to log into different log than errors.
+ - should we bother checking if there's invalid 8bit headers in
+   BODY/BODYSTRUCTURE output and converting them to quoted printable?
+ - virtual mail which shows up every time we're out of disk space. but how?..
+ - update docs/index.txt
 
 auth / login:
  - SRP authentication support?
@@ -106,7 +148,8 @@
    through imap-login like SSL is done?
  - imap-auth should limit how fast authentication requests are allowed from
    login processes. especially if there's one login/connection the speed
-   should be something like once/sec.
+   should be something like once/sec. also limit how fast to accept new
+   connections.
  - HIGH: support executing each login in it's own process, so if an exploit
    is ever found from it, the attacker can't see other users' passwords.
     - master should limit number of login processes to max_logging_users,
@@ -125,12 +168,14 @@
    annoying those close(inbuf->fd)s with open_mail()..
  - allocating readwrite pools now just uses system_pool .. so pool_unref()
    can't free memory used by it .. what to do about it? at least count the
-   malloc/free calls and make complain if at the exit they don't match
+   malloc/free calls and complain if at the exit they don't match
  - ..wonder what it would look like if I did s/FooBarBaz/struct foo_bar_baz/..
  - HIGH: Make sure messages of size INT_MAX..UINT_MAX (and more) work
    correctly. virtual_size can also overflow making it less than physical_size
  - verify memory alignment is valid when reading from index files
  - create env_put() and env_clean()
+ - nearest_power() could be problematic with things that want it for ints,
+   not size_t..
 
 optional optimizations:
  - provide some helper binary to save new mail into mailboxes with CR+LF