Mercurial > dovecot > core-2.2
annotate TODO @ 21566:7dfa73aba5f6
mail-log: Add mail_log_cached_only setting.
If enabled, everything except "save" event will log only the fields that can
be looked up from cache. This improves performance if some of the fields
aren't cached and it's not a strict requirement to log them.
author | Timo Sirainen <timo.sirainen@dovecot.fi> |
---|---|
date | Wed, 15 Feb 2017 23:32:52 +0200 |
parents | 673a12afb3c5 |
children |
rev | line source |
---|---|
19501 | 1 - remove mail_deliver_session after all, do all the stuff transparently |
2 by hooking into mailbox_copy(). | |
3 - use this hook also to do the mail deduplication: 1) sort all destination | |
4 users, 2) create mail_user only once for each user, 3) remember in | |
5 src_mail the previously copied mail, 4) use that for mailbox_copy()ing | |
6 to following recipients | |
7 - make sure this removes duplicate dbox mails when sieve saves mail to | |
8 multiple mailboxes | |
9 - auth: user iterations shouldn't be able to use up all the workers | |
10 - indexer: if workers are stuck, we keep adding more and more stuff to them | |
11 which causes the ostream size to become huge. | |
12 - quota: maybe check quota once more at commit time to make sure the whole | |
13 transaction fits. avoids multiple parallel slow COPY commands from being | |
14 able to go over quota | |
15 - lmtp: Calculate incoming mail's hash, forward it via proxying, have the | |
16 final delivery code verify that it's correct | |
17 - METADATA: quota, NOTIFY interaction, METADATA-SERVER capability | |
18 - fts: if SEARCH X-MAILBOX is used on virtual/all folder, it doesn't update | |
19 any indexes. (and it should skip those physical mailboxes that don't | |
20 match the X-MAILBOX) | |
21 - fts: if indexer has request queued, SEARCH won't return anything until | |
22 it's done. | |
23 - maybe abort entirely after X time and return NO | |
24 - prioritize small quick indexing before slow large indexing? | |
25 - in virtual mailbox searches don't wait for indexing to finish to | |
26 large unindexed mailboxes, just show what you got | |
27 - figure out some way to avoid a million error messages getting logged | |
28 when service imap/pop3 reaches process_limit (some kind of notification | |
29 to login process that the post-login process is full?) | |
30 - lda: mail sending (bounce? forward?) is sending mixed CRLF+LFs | |
31 - auth: remove protocol !flop {} requirement. try again remote {} and local {} | |
32 support for auth. where do we go stuck? at least need to be able to share | |
33 identical passdb/userdbs | |
34 - doveadm sync -l: lock also when syncing public mailboxes? per-mailbox locks? | |
35 - dsync: dsync_mailbox_export_init() can be very slow and not send anything | |
36 to remote dsync for a long time, which thinks the other side is dead and | |
37 kills it. need to send some kind of keepalive-notifications. | |
38 - dsync: rename + re-subscribe isn't handled right in first sync, because | |
39 dsync moves the subscribed-flag when it renames the node | |
40 - "/asdf" in subscriptions -> LSUB lists -> dsync assert-crashes | |
41 - replicator: automatically remove users who don't exist | |
42 - imapc: sync_uid_next handling doesn't seem to be correct, especially with | |
43 Courier that doesn't send UIDNEXT on SELECT | |
44 - sdbox: dbox_file_fix() should assume there is only one message.. | |
45 - pop3: if we can't fetch "order" field for UIDL (but could fetch it | |
46 initially), the order will be wrong and error is logged. probably just | |
47 need to read all the UIDLs into memory at startup?.. | |
48 | |
49 - fs_list_get_mailbox_flags() is unnecessarily stat()ing files/dirs | |
50 - doveadm-server: dsync doesn't work through proxying, because the data isn't | |
51 actually being proxied but handled via doveadm_print() | |
52 - CATENATE: Allow ~{binary} data but fail if there are any c-t-e: binary parts? | |
53 or simply silently save it? | |
54 | |
55 - master-settings.c warnings aren't logged to log file at startup | |
16199 | 56 - dsync: delete foo, rename bar foo -> foo, foo-temp-1 |
57 - dsync+imapc: | |
58 - mailbox list could be synced pretty optimally by ignoring | |
59 (name, uidvalidity) matches. for the left if uidvalidities are unique | |
60 and can be matched -> rename mailbox. | |
61 - GUID-less sync could optionally use just rfc822.size [and internaldate] | |
62 to match messages. | |
63 | |
16523 | 64 - virtual plugin doesn't verify the index file's data, crashes if broken. |
19501 | 65 - libsasl: use it in pop3c, managesieve-login, doveadm auth |
16523 | 66 - per-msg checksums? per-cache-msg checksums? per-log record checksums? |
15875 | 67 - if transaction log file corruption is noticed, make sure new dovecot.index |
68 snapshot gets written and don't mark the whole file corrupted.. rather maybe | |
69 just rotate and truncate it | |
16199 | 70 - mdbox: purging in alt storage could create files back to alt storage |
15278 | 71 - LAYOUT=index: |
72 - after doing a lot of changes the list's memory pool keeps growing. | |
73 do an occasional re-parsing to clear the pool | |
74 - quota recalc + dict-file [+acl?] assert-crashes in !indexing->syncing | |
15875 | 75 - imaptest: add condstore, qresync tests |
76 | |
77 - Track highestmodseq always, just don't keep per-message modseqs unless | |
78 they're enabled. Then don't return [NOMODSEQ] on select. | |
15278 | 79 - URLAUTH: if client tries to access nonexistent user, do a delay in |
80 imap-urlauth-client.c (AFTER destroying the worker) | |
81 - special response in the control connection to make the imap-urlauth | |
82 master wait before starting a new worker | |
15357 | 83 - shared user should get settings from userdb extra fields, especially |
84 plugin/quota_rule to get different quota limits for shared mailboxes. | |
85 the problem is that user doesn't currently have set_parser available, | |
86 and adding it would probably waste memory.. | |
87 - auth_debug[_passwords]=yes ability for specific users via doveadm. for | |
88 both login-common and auth | |
15278 | 89 - settings parsing is horribly bloaty |
14926
9c69df65af7b
Simplified hash table union now that it again works only with pointers.
Timo Sirainen <tss@iki.fi>
parents:
14865
diff
changeset
|
90 - doveadm: if running via doveadm-server and it fails, say something about |
9c69df65af7b
Simplified hash table union now that it again works only with pointers.
Timo Sirainen <tss@iki.fi>
parents:
14865
diff
changeset
|
91 error being in the log |
14753 | 92 - indexer-worker and maybe others (doveadm?) could support dropping privileges |
93 permanently when service_count=1. Note that LMTP can't with multiple RCPT | |
94 TOs. | |
95 - after reading whole message text, update has_nul-state to cache | |
14585 | 96 - FIFOs maybe should be counted as connections, but unlisten should |
97 unlink+reopen it in master? | |
14140 | 98 - lmtp client/proxy: Handle multiline replies better |
15875 | 99 - lmtp: support DSN extension (especially ORCPT) |
14140 | 100 - recreate mailbox -> existing sessions log "indexid changed" error |
13907 | 101 - add message/mime limits |
13677 | 102 - imapc: |
103 - prefetching to THREAD and SORT | |
104 - check all imap extensions and see if some don't work (condstore) | |
13900 | 105 - per-namespace imapc_* settings? create a way to "copy" a settings struct, |
106 so mail_storage_settings are copied to mail_namespace_settings. use the | |
107 change tracking to figure out what settings are namespace-specific. | |
13677 | 108 |
15940 | 109 - doveadm import: add -d parameter to deduplicate mails based on their GUID |
110 (or perhaps do it by default?) | |
111 - sdbox: altmoving is done with mailbox locked. that's not necessary, it could | |
112 do the copying while unlocked and delete the primary files while locked | |
13900 | 113 - passdb, userdb { username_format } that doesn't permanently change |
114 the username | |
13470 | 115 - mdbox/sdbox index rebuild -> quota rebuild? |
13360 | 116 - solr separate attachments (patch) |
117 - sql connection pooling: Count lookup latencies, avoid servers with | |
13777 | 118 significantly higher latencies. optionally use the secondary server only |
119 as fallback | |
13360 | 120 - maildir_storage_sync_force() shouldn't do anything except find the new |
121 file, don't go expunging any more stuff or we could get recursively back to | |
122 where we started, and stuff would break | |
123 - fuzzy: be fuzzy about date/size | |
12756 | 124 - mailbox list index: |
125 - with in-memory indexes be sure to refresh it more often | |
126 - refreshing could refresh only the parts that are actually requested, | |
127 e.g. % | |
12551 | 128 - notify_sync() could have "what changed" struct with old/new flags |
12546 | 129 - maildir: copy dovecot-shared file from parent mailbox, not root. |
130 | |
12476 | 131 - master passdb preserves userdb_* extra fields. should it preserve |
132 non-userdb_* extra fields too? | |
12400 | 133 - imap, pop3: if client init fails, wait a second or two before disconnecting |
134 client. | |
135 - doveadm search savedbefore 7d could be optimized in large mailboxes.. | |
12336 | 136 - mdbox: storage rebuilding could log about changes it does |
12400 | 137 - mdbox: broken extrefs header keeps causing index rebuilds |
12231 | 138 - sent, drafts: .Sent/dovecot.index: modseq_hdr.log_offset too large |
12181 | 139 - mail_max_lock_timeout error could be reported more nicely, also ones coming |
140 from lib-index | |
12095 | 141 - sql pool: if async query is pending and sync query is sent and there |
142 are no more empty connections, it should flush the async query first | |
11683 | 143 - NTLMv1 and LM should be disabled if disable_plaintext_auth=yes |
11387 | 144 - SEARCH SENT*/HEADER/etc. doesn't seem optimized when using with TEXT/BODY |
12476 | 145 - dict sql: support ignoring some search key hierarchies (e.g. acl "anyone") |
15940 | 146 - dsync: avoid sending email when it could be copied from another mailbox. |
147 probably requires storage to have guid => { instances } map? that's | |
148 rather annoying to add. | |
11273 | 149 |
10479 | 150 - mdbox |
11485 | 151 - dotlocking: cleanup should delete stale *.lock files |
10968 | 152 - purging seems to be inefficient. run imaptest for a while, get >500 |
153 files, start purging, it's slow until there are about 100 files left, | |
154 then the rest is suddenly fast. | |
155 - make sure that when reading mdbox mails sequentially the data is being | |
156 read from disk in n kB blocks and reads cross mail boundaries and when | |
157 reading the next mail it uses the previously read data in buffer | |
10479 | 158 - Add some kind of checksum about data+metadata and use it when checking |
159 consistency | |
10798 | 160 - figure out a way to efficiently trigger purging when user has too much |
161 mail expunged (e.g. keep track of total storage size, trigger purging | |
162 when it's 2*quota limit) | |
163 - keep track of total bytes in dbox storage in map header. also if | |
164 possible keep track of refcount=0 bytes. use these to optimize checks. | |
165 - save some stuff to map index header so we don't need to keep retrying | |
166 it. like when saving the lowest file_id which to bother checking. | |
167 - test crash-fixing | |
10968 | 168 - optimize away reading file header? |
10479 | 169 - maildir: out-of-disk-space failures apparently cause all kinds of |
170 problems, e.g. "Expunged message reappeared", "Duplicate file entry"? | |
10163 | 171 - deliver -r <address> used as autoreplies' From-address? |
10546 | 172 - istream-seekable is inefficient. it shouldn't be reading the temp file |
173 immediately after writing to it | |
10968 | 174 - config process is handling requests too slowly. maybe add some caching. |
175 - maybe config should return all of the protocol/local/remote overrides | |
176 when requested? then the caller could do a single lookup at start and | |
177 merge them later internally. this would really help login processes. | |
178 - ipv6: auth penalty should begin from /64 and gradually grow to /48 if | |
179 necessary. and the same could be done for ipv4 as well.. | |
10052 | 180 |
10163 | 181 - ldap: fix multiple-gid support somehow |
9459 | 182 - search: use mail_get_parts() only when it's already cached. if it's not, |
183 add it to cache afterwards. | |
9015 | 184 |
9000 | 185 /* currently non-external transactions can be applied multiple times, |
186 causing multiple increments. */ | |
187 //FIXME:i_assert((t->flags & MAIL_INDEX_TRANSACTION_FLAG_EXTERNAL) != 0); | |
188 ^ appears to work now though, probably because of the added syncing stuff.. | |
189 | |
190 - use backup index in mail_index_fsck() | |
8564
3c4934783aff
Mention SSL/TLS in "plaintext auth disallowed" error.
Timo Sirainen <tss@iki.fi>
parents:
8555
diff
changeset
|
191 - proxying: support fallbacking to local (or other?) server if the first |
3c4934783aff
Mention SSL/TLS in "plaintext auth disallowed" error.
Timo Sirainen <tss@iki.fi>
parents:
8555
diff
changeset
|
192 one is down |
11765 | 193 - virtual: If last message matching INTHREAD rule gets expunged, the rest of |
194 the thread doesn't go away | |
8488 | 195 - how do shared mailboxes work with plugins? |
196 - lazy-expunge, fts, etc.? | |
11974 | 197 - listescape+acl can't handle shared mailboxes with escape chars |
8488 | 198 - dovecot-acl-list: |
199 - how does it work with global acls? | |
200 - update immediately after SETACL: add/remove entries, update timestamps | |
201 - read the entire file to memory only once and keep it there, stat() later | |
202 to see if it has changed. if not, perhaps don't even bother stat()ing | |
203 dovecot-acl files? at least not that often.. | |
204 - fs quota: getquotaroot inbox vs. other-box should return different quotas | |
205 if two quotas are defined | |
8158 | 206 - auth_log_prefix setting similar to mail_log_prefix |
207 | |
208 - thread indexes: if we expunge a duplicate message-id: and we have a sibling | |
209 with identical message-id:, we can probably just move the children? | |
210 (unless there are non-sibling duplicates) | |
211 - SEARCH INTHREAD requires no thread sorting by date - don't do it | |
212 - CONDSTORE: use per-flag/per-keyword conflict checking | |
213 - QRESYNC: Drop expunges from the middle of given seq sets if possible | |
214 - use universal hash functions? | |
215 | |
216 - UIDVALIDITY changed while saving -> sync errors | |
217 - mbox: copy to Trash, manually delete copied msg, change uidvalidity, | |
218 set nextuid=1, copy again -> error | |
219 - recent_uids assert at least with mbox | |
220 - quota fs: Should values returned by quota be divided by the actual | |
221 filesystem block size instead of hardcoded DEV_BSIZE? not with AIX.. | |
7259 | 222 - squat: |
7380 | 223 - wrong indexid |
224 - fts_build_init() assertion failed: (last_uid < last_uid_locked) | |
7259 | 225 - is locking done right? it reads header without file being locked? |
226 - split after ~8 bytes? | |
7391 | 227 - expunges are delayed until more mails are added |
7259 | 228 - test replacement chars (SEARCH / SORT / Squat) |
6987 | 229 |
7449 | 230 - DEBUG: buffer overflow checking code probably doesn't handle a successful |
231 t_try_realloc() or pool_alloconly_realloc() properly | |
8738 | 232 - ldap: |
14140 | 233 - multiple ldap values could be joined into one field with specified |
234 separator (e.g. mail_access_groups=%{ldap:gidNumber:,}) | |
5621 | 235 |
9159
6324a79d3ee1
Initial commit for v2.0 master rewrite. Several features are still missing.
Timo Sirainen <tss@iki.fi>
parents:
9015
diff
changeset
|
236 - maildir+pop3 fast updates: |
6105 | 237 - don't update dovecot-uidlist if dovecot.index.cache doesn't exist / |
238 there's nothing to cache | |
239 - if all messages are expunged and there are no unknown extensions in index, | |
240 unlink dovecot.index and rotate log and add some initial useful info to | |
241 the log (uidvalidity, nextuid) | |
5621 | 242 |
6105 | 243 - maildir |
6987 | 244 - don't allow more than 26 keywords |
5621 | 245 |
5557 | 246 - file_cache: we're growing the mmap in page size blocks, which is horribly |
247 slow if mremap() doesn't exist. | |
248 | |
3087 | 249 - keywords: |
250 - add some limits to how many there can be | |
251 - don't return \* in PERMANENTFLAGS when we're full | |
252 - remove unused keywords? | |
2774 | 253 |
11273 | 254 - mail caching |
3904 | 255 - force bits should be used only for nonregistered fields |
256 - change envelope parsing not to use get_headers() so imap.envelope can | |
257 actually be cached without all the headers.. | |
6105 | 258 - if there's no other pressure for compression, we should do it when |
259 enough temp fields are ready to be dropped | |
3904 | 260 - we could try compressing same field values into a single |
261 location in cache file. | |
7259 | 262 - place some maximum limit of fields to cache file? maybe some soft and |
263 hard limits, so when soft limit is reached drop fields that have | |
264 been used only once. when hard limit is reached drop any fields to get | |
265 more space. all this to avoid cache file growing infinitely. | |
2774 | 266 |
3087 | 267 - mbox |
11273 | 268 - UID renumbering doesn't really work after all? |
12476 | 269 - still problems with CRLF mboxes.. especially with broken Content-Length |
270 headers (pointing between CR-LF?) | |
3904 | 271 - syncing existing indexes takes 4x longer than creating new one, why? |
272 - how well does dirty sync + status work? it reads the last mail every | |
273 time? not very good.. | |
274 - always add empty line. make the parser require it too? syncing should | |
275 make sure there always exists two LFs at end of file. raw-mbox-stream | |
276 should make sure the last message ends with LF even if it doesn't exist | |
277 in the file | |
278 - Quote "From ", unquote ">From " | |
4449 | 279 - COPY doesn't work to itself (lock assert crash, for now just disallowed) |
2143 | 280 |
281 - index | |
6105 | 282 - index file format changes: |
10968 | 283 - split to "old" and "new" indexes and try to avoid loading "old" into |
284 memory until needed | |
6105 | 285 - pack UIDs to beginning of file with UID ranges |
286 - use squat-like compressed uid ranges everywhere | |
6782 | 287 - write first extension intros in dovecot.index.log always with names |
288 - or better yet, drop the intro concept completely as it is now | |
2143 | 289 |
3087 | 290 - login |
291 - Digest-MD5: support integrity protection, and maybe crypting. Do it | |
292 through login process like SSL is done? | |
474 | 293 |
3087 | 294 - auth |
4449 | 295 - with blocking passdb we're not caching lookups if the password was wrong |
5557 | 296 - non-plaintext authentication doesn't support all features: |
297 - multiple passdbs don't work, only the first one is used | |
298 - auth cache's last_success password change check doesn't exist | |
9459 | 299 - auth_cache_negative_ttl doesn't check password mismatches |
4449 | 300 - dovecot-auth should limit how fast authentication requests are allowed |
301 from login processes. especially if there's one login/connection the speed | |
3087 | 302 should be something like once/sec. also limit how fast to accept new |
303 connections. | |
304 - support read-only logins. user could with alternative password get only | |
305 read-access to mails so mails could be read relatively safely with | |
306 untrusted computers. Maybe always send [ALERT] about the previous | |
307 read-only login time with IP? | |
308 | |
309 - ssl | |
310 - add setting: ssl_options = bitmask. by default we enable all openssl | |
311 workarounds, this could be used to disable some of them | |
312 - gnutls support isn't working | |
965 | 313 |
3087 | 314 - search |
6232 | 315 - message header search: we should ignore LWSP between two MIME blocks(?) |
316 - message_search_init() could accept multiple search keywords so we | |
1250 | 317 wouldn't need to call it separately for each one (so we wouldn't need |
318 to parse the message multiple times). | |
319 - Create our own extension: When searching with TEXT/BODY, return | |
320 the message text surrounding the keywords just like web search engines | |
321 do. like: SEARCH X-PRINT-MATCHES TEXT "hello" -> * SEARCH 1 "He said: | |
322 Hello world!" 2 "Hello, I'm ...". This would be especially useful with | |
323 the above attachment scanning. | |
0 | 324 |
3087 | 325 - general |
326 - things break if next_uid gets to 2^32 | |
17629
edcbd75b76ba
lib-http: Added initial support for server-side HTTP API.
Stephan Bosch <stephan@rename-it.nl>
parents:
16958
diff
changeset
|
327 |
edcbd75b76ba
lib-http: Added initial support for server-side HTTP API.
Stephan Bosch <stephan@rename-it.nl>
parents:
16958
diff
changeset
|
328 - lib-http: |
edcbd75b76ba
lib-http: Added initial support for server-side HTTP API.
Stephan Bosch <stephan@rename-it.nl>
parents:
16958
diff
changeset
|
329 - Client: |
edcbd75b76ba
lib-http: Added initial support for server-side HTTP API.
Stephan Bosch <stephan@rename-it.nl>
parents:
16958
diff
changeset
|
330 - Handle HTTP/1.0 servers properly: |
edcbd75b76ba
lib-http: Added initial support for server-side HTTP API.
Stephan Bosch <stephan@rename-it.nl>
parents:
16958
diff
changeset
|
331 -> Transfer-Encoding is not allowed |
edcbd75b76ba
lib-http: Added initial support for server-side HTTP API.
Stephan Bosch <stephan@rename-it.nl>
parents:
16958
diff
changeset
|
332 - Implement support for priority/deadline-based scheduling. |
edcbd75b76ba
lib-http: Added initial support for server-side HTTP API.
Stephan Bosch <stephan@rename-it.nl>
parents:
16958
diff
changeset
|
333 Much like: https://httpd.apache.org/docs/2.2/mod/mod_proxy_balancer.html |
edcbd75b76ba
lib-http: Added initial support for server-side HTTP API.
Stephan Bosch <stephan@rename-it.nl>
parents:
16958
diff
changeset
|
334 - Allow handling non-idempotent requests specially |
edcbd75b76ba
lib-http: Added initial support for server-side HTTP API.
Stephan Bosch <stephan@rename-it.nl>
parents:
16958
diff
changeset
|
335 (no automatic retry, block pipeline) |
edcbd75b76ba
lib-http: Added initial support for server-side HTTP API.
Stephan Bosch <stephan@rename-it.nl>
parents:
16958
diff
changeset
|
336 - Implement support for `Range:' requests. |
edcbd75b76ba
lib-http: Added initial support for server-side HTTP API.
Stephan Bosch <stephan@rename-it.nl>
parents:
16958
diff
changeset
|
337 - Implement optional round-robin request scheduling for when |
edcbd75b76ba
lib-http: Added initial support for server-side HTTP API.
Stephan Bosch <stephan@rename-it.nl>
parents:
16958
diff
changeset
|
338 host has multiple IPs. |
edcbd75b76ba
lib-http: Added initial support for server-side HTTP API.
Stephan Bosch <stephan@rename-it.nl>
parents:
16958
diff
changeset
|
339 - Server: |
edcbd75b76ba
lib-http: Added initial support for server-side HTTP API.
Stephan Bosch <stephan@rename-it.nl>
parents:
16958
diff
changeset
|
340 - Implement API structure for virtual hosts and resources. This way, |
edcbd75b76ba
lib-http: Added initial support for server-side HTTP API.
Stephan Bosch <stephan@rename-it.nl>
parents:
16958
diff
changeset
|
341 multiple services can coexist independently on the same HTTP server. |
edcbd75b76ba
lib-http: Added initial support for server-side HTTP API.
Stephan Bosch <stephan@rename-it.nl>
parents:
16958
diff
changeset
|
342 - Implement support for `Range:' requests. |
edcbd75b76ba
lib-http: Added initial support for server-side HTTP API.
Stephan Bosch <stephan@rename-it.nl>
parents:
16958
diff
changeset
|
343 - Review compliance with RFC 7230 and RFC 7231 |