dovecot/original-hg/dovecot-1.2: src/lib-storage/index/maildir/maildir-uidlist.c annotate

annotate src/lib-storage/index/maildir/maildir-uidlist.c @ 6857:41911abe6fa7 HEAD

NFS cache flushing updates.

author	Timo Sirainen <tss@iki.fi>
date	Sun, 25 Nov 2007 15:47:36 +0200
parents	dbab5e592577
children	64e0f45e8dc3

rev	line source
6429 65c69a53a7be Replaced my Copyright notices. The year range always ends with 2007 now. Timo Sirainen <tss@iki.fi> parents: 6428 diff changeset	1 /* Copyright (c) 2003-2007 Dovecot authors, see the included COPYING file */
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	2
5944 760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	3 /*
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	4 Version 1 format has been used for most versions of Dovecot up to v1.0.x.
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	5 It's also compatible with Courier IMAP's courierimapuiddb file.
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	6 The format is:
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	7
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	8 header: 1 <uid validity> <next uid>
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	9 entry: <uid> <filename>
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	10
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	11 --
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	12
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	13 Version 2 format was written by a few development Dovecot versions, but
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	14 v1.0.x still parses the format. The format has <flags> field after <uid>.
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	15
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	16 --
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	17
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	18 Version 3 format is an extensible format used by Dovecot v1.1 and later.
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	19 It's also parsed by v1.0.2 (and later). The format is:
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	20
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	21 header: 3 [<key><value> ...]
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	22 entry: <uid> [<key><value> ...] :<filename>
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	23
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	24 See enum maildir_uidlist_*_ext_key for used keys.
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	25 */
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	26
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	27 #include "lib.h"
5890 989abd5017a4 Changed record buffers to arrays. Timo Sirainen <tss@iki.fi> parents: 5740 diff changeset	28 #include "array.h"
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	29 #include "hash.h"
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	30 #include "istream.h"
5917 525b449313a7 Don't use uidlist lock to update uidlist. Have a separate real dotlock for Timo Sirainen <tss@iki.fi> parents: 5915 diff changeset	31 #include "ostream.h"
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	32 #include "str.h"
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	33 #include "file-dotlock.h"
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	34 #include "close-keep-errno.h"
4071 cd3d26cf124a Renamed safe-open.* to nfs-workarounds., safe_open() to nfs_safe_open() and Timo Sirainen <tss@iki.fi>* parents: 4070 diff changeset	35 #include "nfs-workarounds.h"
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	36 #include "maildir-storage.h"
5365 35f18edd5f17 If uidlist file doesn't exist but index files do, resync the maildir to Timo Sirainen <tss@iki.fi> parents: 5176 diff changeset	37 #include "maildir-sync.h"
5899 f29b93c0519c Moved maildir filename related functions to maildir-filename.c Timo Sirainen <tss@iki.fi> parents: 5892 diff changeset	38 #include "maildir-filename.h"
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	39 #include "maildir-uidlist.h"
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	40
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	41 #include <stdio.h>
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	42 #include <stdlib.h>
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	43 #include <sys/stat.h>
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	44
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	45 /* NFS: How many times to retry reading dovecot-uidlist file if ESTALE
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	46 error occurs in the middle of reading it */
4071 cd3d26cf124a Renamed safe-open.* to nfs-workarounds., safe_open() to nfs_safe_open() and Timo Sirainen <tss@iki.fi>* parents: 4070 diff changeset	47 #define UIDLIST_ESTALE_RETRY_COUNT NFS_ESTALE_RETRY_COUNT
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	48
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	49 /* how many seconds to wait before overriding uidlist.lock */
3981 0d64f8888dcd Removed immediate_stale_timeout and changed the stale_timeout behavior to Timo Sirainen <tss@iki.fi> parents: 3879 diff changeset	50 #define UIDLIST_LOCK_STALE_TIMEOUT (60*2)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	51
5933 fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	52 #define UIDLIST_COMPRESS_PERCENTAGE 75
fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	53
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	54 #define UIDLIST_IS_LOCKED(uidlist) \
3447 96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3280 diff changeset	55 ((uidlist)->lock_count > 0)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	56
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	57 struct maildir_uidlist_rec {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	58 uint32_t uid;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	59 uint32_t flags;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	60 char *filename;
5944 760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	61 char extensions; / <data>\0[<data>\0 ...]\0 */
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	62 };
5890 989abd5017a4 Changed record buffers to arrays. Timo Sirainen <tss@iki.fi> parents: 5740 diff changeset	63 ARRAY_DEFINE_TYPE(maildir_uidlist_rec_p, struct maildir_uidlist_rec *);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	64
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	65 struct maildir_uidlist {
3279 b698ae839a18 Moved mbox/maildir-specific variables from struct index_mailbox to Timo Sirainen <tss@iki.fi> parents: 3133 diff changeset	66 struct maildir_mailbox *mbox;
6599 a6f51026b969 Added maildir_uidlist_init_readonly() and maildir_uidlist_get_uid(). Timo Sirainen <tss@iki.fi> parents: 6598 diff changeset	67 struct index_mailbox *ibox;
5917 525b449313a7 Don't use uidlist lock to update uidlist. Have a separate real dotlock for Timo Sirainen <tss@iki.fi> parents: 5915 diff changeset	68 char *path;
3447 96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3280 diff changeset	69
5891 db959f0d94e3 Leave uidlist fd open so we can check if uidlist has been rewritten from Timo Sirainen <tss@iki.fi> parents: 5890 diff changeset	70 int fd;
db959f0d94e3 Leave uidlist fd open so we can check if uidlist has been rewritten from Timo Sirainen <tss@iki.fi> parents: 5890 diff changeset	71 dev_t fd_dev;
db959f0d94e3 Leave uidlist fd open so we can check if uidlist has been rewritten from Timo Sirainen <tss@iki.fi> parents: 5890 diff changeset	72 ino_t fd_ino;
5933 fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	73 off_t fd_size;
5891 db959f0d94e3 Leave uidlist fd open so we can check if uidlist has been rewritten from Timo Sirainen <tss@iki.fi> parents: 5890 diff changeset	74
3447 96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3280 diff changeset	75 unsigned int lock_count;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	76
5917 525b449313a7 Don't use uidlist lock to update uidlist. Have a separate real dotlock for Timo Sirainen <tss@iki.fi> parents: 5915 diff changeset	77 struct dotlock_settings dotlock_settings;
525b449313a7 Don't use uidlist lock to update uidlist. Have a separate real dotlock for Timo Sirainen <tss@iki.fi> parents: 5915 diff changeset	78 struct dotlock *dotlock;
525b449313a7 Don't use uidlist lock to update uidlist. Have a separate real dotlock for Timo Sirainen <tss@iki.fi> parents: 5915 diff changeset	79
1962 4775ffa73874 Syncing should be fixed now. Timo Sirainen <tss@iki.fi> parents: 1960 diff changeset	80 pool_t record_pool;
5890 989abd5017a4 Changed record buffers to arrays. Timo Sirainen <tss@iki.fi> parents: 5740 diff changeset	81 ARRAY_TYPE(maildir_uidlist_rec_p) records;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	82 struct hash_table *files;
5913 699356ffe549 If uidlist changes while it's being iterated, don't crash. Timo Sirainen <tss@iki.fi> parents: 5903 diff changeset	83 unsigned int change_counter;
699356ffe549 If uidlist changes while it's being iterated, don't crash. Timo Sirainen <tss@iki.fi> parents: 5903 diff changeset	84
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	85 unsigned int version;
2565 e14c2a36a0a2 "UIDs not ordered in file" errors could have occured sometimes wrongly. Timo Sirainen <tss@iki.fi> parents: 2531 diff changeset	86 unsigned int uid_validity, next_uid, prev_read_uid, last_seen_uid;
5933 fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	87 unsigned int read_records_count;
fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	88 uoff_t last_read_offset;
5944 760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	89 string_t *hdr_extensions;
1962 4775ffa73874 Syncing should be fixed now. Timo Sirainen <tss@iki.fi> parents: 1960 diff changeset	90
5933 fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	91 unsigned int recreate:1;
1962 4775ffa73874 Syncing should be fixed now. Timo Sirainen <tss@iki.fi> parents: 1960 diff changeset	92 unsigned int initial_read:1;
2038 df504dad3aec Recent flag fixes. Should work perfectly now with maildir. Timo Sirainen <tss@iki.fi> parents: 2005 diff changeset	93 unsigned int initial_sync:1;
6800 ca849e28a062 NFS: Delay flushing dovecot-uidlist's attribute cache until we find an Timo Sirainen <tss@iki.fi> parents: 6798 diff changeset	94 unsigned int nfs_dirty_refresh:1;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	95 };
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	96
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	97 struct maildir_uidlist_sync_ctx {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	98 struct maildir_uidlist *uidlist;
5393 434047e53586 When saving/copying mails, we must not do any sorting based on the filename Timo Sirainen <tss@iki.fi> parents: 5388 diff changeset	99 enum maildir_uidlist_sync_flags sync_flags;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	100
1962 4775ffa73874 Syncing should be fixed now. Timo Sirainen <tss@iki.fi> parents: 1960 diff changeset	101 pool_t record_pool;
5890 989abd5017a4 Changed record buffers to arrays. Timo Sirainen <tss@iki.fi> parents: 5740 diff changeset	102 ARRAY_TYPE(maildir_uidlist_rec_p) records;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	103 struct hash_table *files;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	104
6034 dcaa8d5296a9 Fixes to handling appends and unexpunges. Timo Sirainen <tss@iki.fi> parents: 6029 diff changeset	105 unsigned int first_unwritten_pos, first_nouid_pos;
1973 f6d2dba3b78b and yet more syncing fixes.. Timo Sirainen <tss@iki.fi> parents: 1971 diff changeset	106 unsigned int new_files_count;
1954 2f6e137cdc44 Syncing optimizations. Timo Sirainen <tss@iki.fi> parents: 1947 diff changeset	107
2f6e137cdc44 Syncing optimizations. Timo Sirainen <tss@iki.fi> parents: 1947 diff changeset	108 unsigned int partial:1;
1984 9c159272f721 syncing fixes Timo Sirainen <tss@iki.fi> parents: 1978 diff changeset	109 unsigned int finished:1;
5923 239078f515a8 Added maildir_uidlist_sync_remove() Timo Sirainen <tss@iki.fi> parents: 5922 diff changeset	110 unsigned int changed:1;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	111 unsigned int failed:1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	112 };
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	113
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	114 struct maildir_uidlist_iter_ctx {
5913 699356ffe549 If uidlist changes while it's being iterated, don't crash. Timo Sirainen <tss@iki.fi> parents: 5903 diff changeset	115 struct maildir_uidlist *uidlist;
5890 989abd5017a4 Changed record buffers to arrays. Timo Sirainen <tss@iki.fi> parents: 5740 diff changeset	116 struct maildir_uidlist_rec const next, const end;
5913 699356ffe549 If uidlist changes while it's being iterated, don't crash. Timo Sirainen <tss@iki.fi> parents: 5903 diff changeset	117
699356ffe549 If uidlist changes while it's being iterated, don't crash. Timo Sirainen <tss@iki.fi> parents: 5903 diff changeset	118 unsigned int change_counter;
699356ffe549 If uidlist changes while it's being iterated, don't crash. Timo Sirainen <tss@iki.fi> parents: 5903 diff changeset	119 uint32_t prev_uid;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	120 };
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	121
5944 760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	122 static bool maildir_uidlist_iter_next_rec(struct maildir_uidlist_iter_ctx *ctx,
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	123 struct maildir_uidlist_rec **rec_r);
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	124
2271 15d959d262a4 UIDs for appended messages can now be assigned all at once. Maildir now Timo Sirainen <tss@iki.fi> parents: 2123 diff changeset	125 static int maildir_uidlist_lock_timeout(struct maildir_uidlist *uidlist,
3863 55df57c028d4 Added "bool" type and changed all ints that were used as booleans to bool. Timo Sirainen <tss@iki.fi> parents: 3534 diff changeset	126 bool nonblock)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	127 {
6599 a6f51026b969 Added maildir_uidlist_init_readonly() and maildir_uidlist_get_uid(). Timo Sirainen <tss@iki.fi> parents: 6598 diff changeset	128 struct mailbox *box = &uidlist->ibox->box;
a6f51026b969 Added maildir_uidlist_init_readonly() and maildir_uidlist_get_uid(). Timo Sirainen <tss@iki.fi> parents: 6598 diff changeset	129 const char control_dir, path;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	130 mode_t old_mask;
6777 186b164a9579 Delay creating directories until we really need them. Timo Sirainen <tss@iki.fi> parents: 6738 diff changeset	131 const enum dotlock_create_flags dotlock_flags =
186b164a9579 Delay creating directories until we really need them. Timo Sirainen <tss@iki.fi> parents: 6738 diff changeset	132 nonblock ? DOTLOCK_CREATE_FLAG_NONBLOCK : 0;
186b164a9579 Delay creating directories until we really need them. Timo Sirainen <tss@iki.fi> parents: 6738 diff changeset	133 int i, ret;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	134
3447 96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3280 diff changeset	135 if (uidlist->lock_count > 0) {
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3280 diff changeset	136 uidlist->lock_count++;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	137 return 1;
3447 96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3280 diff changeset	138 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	139
6599 a6f51026b969 Added maildir_uidlist_init_readonly() and maildir_uidlist_get_uid(). Timo Sirainen <tss@iki.fi> parents: 6598 diff changeset	140 control_dir = mailbox_list_get_path(box->storage->list, box->name,
a6f51026b969 Added maildir_uidlist_init_readonly() and maildir_uidlist_get_uid(). Timo Sirainen <tss@iki.fi> parents: 6598 diff changeset	141 MAILBOX_LIST_PATH_TYPE_CONTROL);
6777 186b164a9579 Delay creating directories until we really need them. Timo Sirainen <tss@iki.fi> parents: 6738 diff changeset	142 path = t_strconcat(control_dir, "/" MAILDIR_UIDLIST_NAME, NULL);
6599 a6f51026b969 Added maildir_uidlist_init_readonly() and maildir_uidlist_get_uid(). Timo Sirainen <tss@iki.fi> parents: 6598 diff changeset	143
6777 186b164a9579 Delay creating directories until we really need them. Timo Sirainen <tss@iki.fi> parents: 6738 diff changeset	144 for (i = 0;; i++) {
186b164a9579 Delay creating directories until we really need them. Timo Sirainen <tss@iki.fi> parents: 6738 diff changeset	145 old_mask = umask(0777 & ~box->file_create_mode);
186b164a9579 Delay creating directories until we really need them. Timo Sirainen <tss@iki.fi> parents: 6738 diff changeset	146 ret = file_dotlock_create(&uidlist->dotlock_settings, path,
186b164a9579 Delay creating directories until we really need them. Timo Sirainen <tss@iki.fi> parents: 6738 diff changeset	147 dotlock_flags, &uidlist->dotlock);
186b164a9579 Delay creating directories until we really need them. Timo Sirainen <tss@iki.fi> parents: 6738 diff changeset	148 umask(old_mask);
186b164a9579 Delay creating directories until we really need them. Timo Sirainen <tss@iki.fi> parents: 6738 diff changeset	149 if (ret > 0)
186b164a9579 Delay creating directories until we really need them. Timo Sirainen <tss@iki.fi> parents: 6738 diff changeset	150 break;
186b164a9579 Delay creating directories until we really need them. Timo Sirainen <tss@iki.fi> parents: 6738 diff changeset	151
186b164a9579 Delay creating directories until we really need them. Timo Sirainen <tss@iki.fi> parents: 6738 diff changeset	152 /* failure */
5917 525b449313a7 Don't use uidlist lock to update uidlist. Have a separate real dotlock for Timo Sirainen <tss@iki.fi> parents: 5915 diff changeset	153 if (ret == 0) {
6599 a6f51026b969 Added maildir_uidlist_init_readonly() and maildir_uidlist_get_uid(). Timo Sirainen <tss@iki.fi> parents: 6598 diff changeset	154 mail_storage_set_error(box->storage,
5613 f717fb4b31c0 Error handling rewrite. Timo Sirainen <tss@iki.fi> parents: 5584 diff changeset	155 MAIL_ERROR_TEMP, MAIL_ERRSTR_LOCK_TIMEOUT);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	156 return 0;
4152 e2edd333c473 Added MAILBOX_OPEN_KEEP_LOCKED flag to mailbox opening and implemented it Timo Sirainen <tss@iki.fi> parents: 4072 diff changeset	157 }
6777 186b164a9579 Delay creating directories until we really need them. Timo Sirainen <tss@iki.fi> parents: 6738 diff changeset	158 if (errno != ENOENT \|\| i == MAILDIR_DELETE_RETRY_COUNT \|\|
186b164a9579 Delay creating directories until we really need them. Timo Sirainen <tss@iki.fi> parents: 6738 diff changeset	159 uidlist->mbox == NULL) {
186b164a9579 Delay creating directories until we really need them. Timo Sirainen <tss@iki.fi> parents: 6738 diff changeset	160 mail_storage_set_critical(box->storage,
186b164a9579 Delay creating directories until we really need them. Timo Sirainen <tss@iki.fi> parents: 6738 diff changeset	161 "file_dotlock_create(%s) failed: %m", path);
186b164a9579 Delay creating directories until we really need them. Timo Sirainen <tss@iki.fi> parents: 6738 diff changeset	162 return -1;
186b164a9579 Delay creating directories until we really need them. Timo Sirainen <tss@iki.fi> parents: 6738 diff changeset	163 }
186b164a9579 Delay creating directories until we really need them. Timo Sirainen <tss@iki.fi> parents: 6738 diff changeset	164 /* the control dir doesn't exist. create it unless the whole
186b164a9579 Delay creating directories until we really need them. Timo Sirainen <tss@iki.fi> parents: 6738 diff changeset	165 mailbox was just deleted. */
186b164a9579 Delay creating directories until we really need them. Timo Sirainen <tss@iki.fi> parents: 6738 diff changeset	166 if (maildir_set_deleted(uidlist->mbox))
186b164a9579 Delay creating directories until we really need them. Timo Sirainen <tss@iki.fi> parents: 6738 diff changeset	167 return -1;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	168 }
2121 0840edf34f37 Locking fixes. use less memory Timo Sirainen <tss@iki.fi> parents: 2094 diff changeset	169
3447 96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3280 diff changeset	170 uidlist->lock_count++;
5917 525b449313a7 Don't use uidlist lock to update uidlist. Have a separate real dotlock for Timo Sirainen <tss@iki.fi> parents: 5915 diff changeset	171
525b449313a7 Don't use uidlist lock to update uidlist. Have a separate real dotlock for Timo Sirainen <tss@iki.fi> parents: 5915 diff changeset	172 /* make sure we have the latest changes before changing anything */
6800 ca849e28a062 NFS: Delay flushing dovecot-uidlist's attribute cache until we find an Timo Sirainen <tss@iki.fi> parents: 6798 diff changeset	173 if (maildir_uidlist_refresh(uidlist, FALSE) < 0) {
5917 525b449313a7 Don't use uidlist lock to update uidlist. Have a separate real dotlock for Timo Sirainen <tss@iki.fi> parents: 5915 diff changeset	174 maildir_uidlist_unlock(uidlist);
525b449313a7 Don't use uidlist lock to update uidlist. Have a separate real dotlock for Timo Sirainen <tss@iki.fi> parents: 5915 diff changeset	175 return -1;
525b449313a7 Don't use uidlist lock to update uidlist. Have a separate real dotlock for Timo Sirainen <tss@iki.fi> parents: 5915 diff changeset	176 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	177 return 1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	178 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	179
2271 15d959d262a4 UIDs for appended messages can now be assigned all at once. Maildir now Timo Sirainen <tss@iki.fi> parents: 2123 diff changeset	180 int maildir_uidlist_lock(struct maildir_uidlist *uidlist)
15d959d262a4 UIDs for appended messages can now be assigned all at once. Maildir now Timo Sirainen <tss@iki.fi> parents: 2123 diff changeset	181 {
3106 9c4aa309dbac Changed dotlocking API. Timo Sirainen <tss@iki.fi> parents: 2892 diff changeset	182 return maildir_uidlist_lock_timeout(uidlist, FALSE);
2271 15d959d262a4 UIDs for appended messages can now be assigned all at once. Maildir now Timo Sirainen <tss@iki.fi> parents: 2123 diff changeset	183 }
15d959d262a4 UIDs for appended messages can now be assigned all at once. Maildir now Timo Sirainen <tss@iki.fi> parents: 2123 diff changeset	184
15d959d262a4 UIDs for appended messages can now be assigned all at once. Maildir now Timo Sirainen <tss@iki.fi> parents: 2123 diff changeset	185 int maildir_uidlist_try_lock(struct maildir_uidlist *uidlist)
15d959d262a4 UIDs for appended messages can now be assigned all at once. Maildir now Timo Sirainen <tss@iki.fi> parents: 2123 diff changeset	186 {
3106 9c4aa309dbac Changed dotlocking API. Timo Sirainen <tss@iki.fi> parents: 2892 diff changeset	187 return maildir_uidlist_lock_timeout(uidlist, TRUE);
2271 15d959d262a4 UIDs for appended messages can now be assigned all at once. Maildir now Timo Sirainen <tss@iki.fi> parents: 2123 diff changeset	188 }
15d959d262a4 UIDs for appended messages can now be assigned all at once. Maildir now Timo Sirainen <tss@iki.fi> parents: 2123 diff changeset	189
5176 db7b28a3ffbd If pop3_lock_session=yes, update maildir/dbox uidlist lock file every 10 Timo Sirainen <tss@iki.fi> parents: 5064 diff changeset	190 int maildir_uidlist_lock_touch(struct maildir_uidlist *uidlist)
db7b28a3ffbd If pop3_lock_session=yes, update maildir/dbox uidlist lock file every 10 Timo Sirainen <tss@iki.fi> parents: 5064 diff changeset	191 {
db7b28a3ffbd If pop3_lock_session=yes, update maildir/dbox uidlist lock file every 10 Timo Sirainen <tss@iki.fi> parents: 5064 diff changeset	192 i_assert(UIDLIST_IS_LOCKED(uidlist));
db7b28a3ffbd If pop3_lock_session=yes, update maildir/dbox uidlist lock file every 10 Timo Sirainen <tss@iki.fi> parents: 5064 diff changeset	193
db7b28a3ffbd If pop3_lock_session=yes, update maildir/dbox uidlist lock file every 10 Timo Sirainen <tss@iki.fi> parents: 5064 diff changeset	194 return file_dotlock_touch(uidlist->dotlock);
db7b28a3ffbd If pop3_lock_session=yes, update maildir/dbox uidlist lock file every 10 Timo Sirainen <tss@iki.fi> parents: 5064 diff changeset	195 }
db7b28a3ffbd If pop3_lock_session=yes, update maildir/dbox uidlist lock file every 10 Timo Sirainen <tss@iki.fi> parents: 5064 diff changeset	196
3863 55df57c028d4 Added "bool" type and changed all ints that were used as booleans to bool. Timo Sirainen <tss@iki.fi> parents: 3534 diff changeset	197 bool maildir_uidlist_is_locked(struct maildir_uidlist *uidlist)
3447 96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3280 diff changeset	198 {
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3280 diff changeset	199 return UIDLIST_IS_LOCKED(uidlist);
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3280 diff changeset	200 }
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3280 diff changeset	201
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	202 void maildir_uidlist_unlock(struct maildir_uidlist *uidlist)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	203 {
3447 96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3280 diff changeset	204 i_assert(uidlist->lock_count > 0);
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3280 diff changeset	205
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3280 diff changeset	206 if (--uidlist->lock_count > 0)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	207 return;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	208
5917 525b449313a7 Don't use uidlist lock to update uidlist. Have a separate real dotlock for Timo Sirainen <tss@iki.fi> parents: 5915 diff changeset	209 (void)file_dotlock_delete(&uidlist->dotlock);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	210 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	211
6599 a6f51026b969 Added maildir_uidlist_init_readonly() and maildir_uidlist_get_uid(). Timo Sirainen <tss@iki.fi> parents: 6598 diff changeset	212 struct maildir_uidlist *
a6f51026b969 Added maildir_uidlist_init_readonly() and maildir_uidlist_get_uid(). Timo Sirainen <tss@iki.fi> parents: 6598 diff changeset	213 maildir_uidlist_init_readonly(struct index_mailbox *ibox)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	214 {
6599 a6f51026b969 Added maildir_uidlist_init_readonly() and maildir_uidlist_get_uid(). Timo Sirainen <tss@iki.fi> parents: 6598 diff changeset	215 struct mailbox *box = &ibox->box;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	216 struct maildir_uidlist *uidlist;
6599 a6f51026b969 Added maildir_uidlist_init_readonly() and maildir_uidlist_get_uid(). Timo Sirainen <tss@iki.fi> parents: 6598 diff changeset	217 const char *control_dir;
a6f51026b969 Added maildir_uidlist_init_readonly() and maildir_uidlist_get_uid(). Timo Sirainen <tss@iki.fi> parents: 6598 diff changeset	218
a6f51026b969 Added maildir_uidlist_init_readonly() and maildir_uidlist_get_uid(). Timo Sirainen <tss@iki.fi> parents: 6598 diff changeset	219 control_dir = mailbox_list_get_path(box->storage->list, box->name,
a6f51026b969 Added maildir_uidlist_init_readonly() and maildir_uidlist_get_uid(). Timo Sirainen <tss@iki.fi> parents: 6598 diff changeset	220 MAILBOX_LIST_PATH_TYPE_CONTROL);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	221
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	222 uidlist = i_new(struct maildir_uidlist, 1);
5891 db959f0d94e3 Leave uidlist fd open so we can check if uidlist has been rewritten from Timo Sirainen <tss@iki.fi> parents: 5890 diff changeset	223 uidlist->fd = -1;
6599 a6f51026b969 Added maildir_uidlist_init_readonly() and maildir_uidlist_get_uid(). Timo Sirainen <tss@iki.fi> parents: 6598 diff changeset	224 uidlist->ibox = ibox;
a6f51026b969 Added maildir_uidlist_init_readonly() and maildir_uidlist_get_uid(). Timo Sirainen <tss@iki.fi> parents: 6598 diff changeset	225 uidlist->path = i_strconcat(control_dir, "/"MAILDIR_UIDLIST_NAME, NULL);
5890 989abd5017a4 Changed record buffers to arrays. Timo Sirainen <tss@iki.fi> parents: 5740 diff changeset	226 i_array_init(&uidlist->records, 128);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	227 uidlist->files = hash_create(default_pool, default_pool, 4096,
5902 0cef1d7a0ce3 Renamed maildir_cmp/hash to maildir_filename_base_cmp/hash. Timo Sirainen <tss@iki.fi> parents: 5899 diff changeset	228 maildir_filename_base_hash,
0cef1d7a0ce3 Renamed maildir_cmp/hash to maildir_filename_base_cmp/hash. Timo Sirainen <tss@iki.fi> parents: 5899 diff changeset	229 maildir_filename_base_cmp);
3106 9c4aa309dbac Changed dotlocking API. Timo Sirainen <tss@iki.fi> parents: 2892 diff changeset	230 uidlist->next_uid = 1;
5944 760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	231 uidlist->hdr_extensions = str_new(default_pool, 128);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	232
5938 805f2527a982 If use_io_notify dotlock setting is enabled, use I/O notify loop for waiting Timo Sirainen <tss@iki.fi> parents: 5933 diff changeset	233 uidlist->dotlock_settings.use_io_notify = TRUE;
4969 5c18ac362f65 More dotlock_use_excl uses. Timo Sirainen <tss@iki.fi> parents: 4960 diff changeset	234 uidlist->dotlock_settings.use_excl_lock =
6599 a6f51026b969 Added maildir_uidlist_init_readonly() and maildir_uidlist_get_uid(). Timo Sirainen <tss@iki.fi> parents: 6598 diff changeset	235 (box->storage->flags & MAIL_STORAGE_FLAG_DOTLOCK_USE_EXCL) != 0;
6592 b6631550cf84 dotlocks: Flush NFS attribute cache only if nfs_flush setting is TRUE. Timo Sirainen <tss@iki.fi> parents: 6442 diff changeset	236 uidlist->dotlock_settings.nfs_flush =
6599 a6f51026b969 Added maildir_uidlist_init_readonly() and maildir_uidlist_get_uid(). Timo Sirainen <tss@iki.fi> parents: 6598 diff changeset	237 (box->storage->flags &
6592 b6631550cf84 dotlocks: Flush NFS attribute cache only if nfs_flush setting is TRUE. Timo Sirainen <tss@iki.fi> parents: 6442 diff changeset	238 MAIL_STORAGE_FLAG_NFS_FLUSH_STORAGE) != 0;
3981 0d64f8888dcd Removed immediate_stale_timeout and changed the stale_timeout behavior to Timo Sirainen <tss@iki.fi> parents: 3879 diff changeset	239 uidlist->dotlock_settings.timeout = UIDLIST_LOCK_STALE_TIMEOUT + 2;
3106 9c4aa309dbac Changed dotlocking API. Timo Sirainen <tss@iki.fi> parents: 2892 diff changeset	240 uidlist->dotlock_settings.stale_timeout = UIDLIST_LOCK_STALE_TIMEOUT;
6599 a6f51026b969 Added maildir_uidlist_init_readonly() and maildir_uidlist_get_uid(). Timo Sirainen <tss@iki.fi> parents: 6598 diff changeset	241
a6f51026b969 Added maildir_uidlist_init_readonly() and maildir_uidlist_get_uid(). Timo Sirainen <tss@iki.fi> parents: 6598 diff changeset	242 return uidlist;
a6f51026b969 Added maildir_uidlist_init_readonly() and maildir_uidlist_get_uid(). Timo Sirainen <tss@iki.fi> parents: 6598 diff changeset	243 }
a6f51026b969 Added maildir_uidlist_init_readonly() and maildir_uidlist_get_uid(). Timo Sirainen <tss@iki.fi> parents: 6598 diff changeset	244
a6f51026b969 Added maildir_uidlist_init_readonly() and maildir_uidlist_get_uid(). Timo Sirainen <tss@iki.fi> parents: 6598 diff changeset	245 struct maildir_uidlist maildir_uidlist_init(struct maildir_mailbox mbox)
a6f51026b969 Added maildir_uidlist_init_readonly() and maildir_uidlist_get_uid(). Timo Sirainen <tss@iki.fi> parents: 6598 diff changeset	246 {
a6f51026b969 Added maildir_uidlist_init_readonly() and maildir_uidlist_get_uid(). Timo Sirainen <tss@iki.fi> parents: 6598 diff changeset	247 struct maildir_uidlist *uidlist;
a6f51026b969 Added maildir_uidlist_init_readonly() and maildir_uidlist_get_uid(). Timo Sirainen <tss@iki.fi> parents: 6598 diff changeset	248
a6f51026b969 Added maildir_uidlist_init_readonly() and maildir_uidlist_get_uid(). Timo Sirainen <tss@iki.fi> parents: 6598 diff changeset	249 uidlist = maildir_uidlist_init_readonly(&mbox->ibox);
a6f51026b969 Added maildir_uidlist_init_readonly() and maildir_uidlist_get_uid(). Timo Sirainen <tss@iki.fi> parents: 6598 diff changeset	250 uidlist->mbox = mbox;
4808 93bc9770f938 Initial code for separation of mailbox accessing and directory layout Timo Sirainen <tss@iki.fi> parents: 4451 diff changeset	251 uidlist->dotlock_settings.temp_prefix = mbox->storage->temp_prefix;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	252 return uidlist;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	253 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	254
5915 171c3f89295c fd leak fix Timo Sirainen <tss@iki.fi> parents: 5913 diff changeset	255 static void maildir_uidlist_close(struct maildir_uidlist *uidlist)
171c3f89295c fd leak fix Timo Sirainen <tss@iki.fi> parents: 5913 diff changeset	256 {
171c3f89295c fd leak fix Timo Sirainen <tss@iki.fi> parents: 5913 diff changeset	257 if (uidlist->fd != -1) {
171c3f89295c fd leak fix Timo Sirainen <tss@iki.fi> parents: 5913 diff changeset	258 if (close(uidlist->fd) < 0)
5917 525b449313a7 Don't use uidlist lock to update uidlist. Have a separate real dotlock for Timo Sirainen <tss@iki.fi> parents: 5915 diff changeset	259 i_error("close(%s) failed: %m", uidlist->path);
5915 171c3f89295c fd leak fix Timo Sirainen <tss@iki.fi> parents: 5913 diff changeset	260 uidlist->fd = -1;
171c3f89295c fd leak fix Timo Sirainen <tss@iki.fi> parents: 5913 diff changeset	261 uidlist->fd_ino = 0;
171c3f89295c fd leak fix Timo Sirainen <tss@iki.fi> parents: 5913 diff changeset	262 }
5933 fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	263 uidlist->last_read_offset = 0;
5915 171c3f89295c fd leak fix Timo Sirainen <tss@iki.fi> parents: 5913 diff changeset	264 }
171c3f89295c fd leak fix Timo Sirainen <tss@iki.fi> parents: 5913 diff changeset	265
6409 7f733ba453bf Changed _deinit() APIs to take * pointer and set it to NULL. Timo Sirainen <tss@iki.fi> parents: 6344 diff changeset	266 void maildir_uidlist_deinit(struct maildir_uidlist **_uidlist)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	267 {
6409 7f733ba453bf Changed _deinit() APIs to take * pointer and set it to NULL. Timo Sirainen <tss@iki.fi> parents: 6344 diff changeset	268 struct maildir_uidlist uidlist = _uidlist;
7f733ba453bf Changed _deinit() APIs to take * pointer and set it to NULL. Timo Sirainen <tss@iki.fi> parents: 6344 diff changeset	269
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	270 i_assert(!UIDLIST_IS_LOCKED(uidlist));
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	271
6409 7f733ba453bf Changed _deinit() APIs to take * pointer and set it to NULL. Timo Sirainen <tss@iki.fi> parents: 6344 diff changeset	272 *_uidlist = NULL;
6657 5da6da5cea95 Make sure uidlist gets written if we only added some extension fields to it. Timo Sirainen <tss@iki.fi> parents: 6599 diff changeset	273 maildir_uidlist_update(uidlist);
5915 171c3f89295c fd leak fix Timo Sirainen <tss@iki.fi> parents: 5913 diff changeset	274 maildir_uidlist_close(uidlist);
5891 db959f0d94e3 Leave uidlist fd open so we can check if uidlist has been rewritten from Timo Sirainen <tss@iki.fi> parents: 5890 diff changeset	275
6417 047d0d8bbf0a hash_destroy() and hash_iterate_deinit() now take ** pointer. Timo Sirainen <tss@iki.fi> parents: 6409 diff changeset	276 hash_destroy(&uidlist->files);
1962 4775ffa73874 Syncing should be fixed now. Timo Sirainen <tss@iki.fi> parents: 1960 diff changeset	277 if (uidlist->record_pool != NULL)
6428 7cad076906eb pool_unref() now takes ** pointer. Timo Sirainen <tss@iki.fi> parents: 6417 diff changeset	278 pool_unref(&uidlist->record_pool);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	279
5890 989abd5017a4 Changed record buffers to arrays. Timo Sirainen <tss@iki.fi> parents: 5740 diff changeset	280 array_free(&uidlist->records);
5944 760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	281 str_free(&uidlist->hdr_extensions);
5917 525b449313a7 Don't use uidlist lock to update uidlist. Have a separate real dotlock for Timo Sirainen <tss@iki.fi> parents: 5915 diff changeset	282 i_free(uidlist->path);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	283 i_free(uidlist);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	284 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	285
6034 dcaa8d5296a9 Fixes to handling appends and unexpunges. Timo Sirainen <tss@iki.fi> parents: 6029 diff changeset	286 static int maildir_uid_cmp(const void p1, const void p2)
dcaa8d5296a9 Fixes to handling appends and unexpunges. Timo Sirainen <tss@iki.fi> parents: 6029 diff changeset	287 {
dcaa8d5296a9 Fixes to handling appends and unexpunges. Timo Sirainen <tss@iki.fi> parents: 6029 diff changeset	288 const struct maildir_uidlist_rec const rec1 = p1, const rec2 = p2;
dcaa8d5296a9 Fixes to handling appends and unexpunges. Timo Sirainen <tss@iki.fi> parents: 6029 diff changeset	289
dcaa8d5296a9 Fixes to handling appends and unexpunges. Timo Sirainen <tss@iki.fi> parents: 6029 diff changeset	290 return (rec1)->uid < (rec2)->uid ? -1 :
dcaa8d5296a9 Fixes to handling appends and unexpunges. Timo Sirainen <tss@iki.fi> parents: 6029 diff changeset	291 (rec1)->uid > (rec2)->uid ? 1 : 0;
dcaa8d5296a9 Fixes to handling appends and unexpunges. Timo Sirainen <tss@iki.fi> parents: 6029 diff changeset	292 }
dcaa8d5296a9 Fixes to handling appends and unexpunges. Timo Sirainen <tss@iki.fi> parents: 6029 diff changeset	293
dcaa8d5296a9 Fixes to handling appends and unexpunges. Timo Sirainen <tss@iki.fi> parents: 6029 diff changeset	294 static void
dcaa8d5296a9 Fixes to handling appends and unexpunges. Timo Sirainen <tss@iki.fi> parents: 6029 diff changeset	295 maildir_uidlist_records_array_delete(struct maildir_uidlist *uidlist,
dcaa8d5296a9 Fixes to handling appends and unexpunges. Timo Sirainen <tss@iki.fi> parents: 6029 diff changeset	296 struct maildir_uidlist_rec *rec)
dcaa8d5296a9 Fixes to handling appends and unexpunges. Timo Sirainen <tss@iki.fi> parents: 6029 diff changeset	297 {
dcaa8d5296a9 Fixes to handling appends and unexpunges. Timo Sirainen <tss@iki.fi> parents: 6029 diff changeset	298 struct maildir_uidlist_rec const recs, const pos;
dcaa8d5296a9 Fixes to handling appends and unexpunges. Timo Sirainen <tss@iki.fi> parents: 6029 diff changeset	299 unsigned int count;
dcaa8d5296a9 Fixes to handling appends and unexpunges. Timo Sirainen <tss@iki.fi> parents: 6029 diff changeset	300
dcaa8d5296a9 Fixes to handling appends and unexpunges. Timo Sirainen <tss@iki.fi> parents: 6029 diff changeset	301 recs = array_get(&uidlist->records, &count);
dcaa8d5296a9 Fixes to handling appends and unexpunges. Timo Sirainen <tss@iki.fi> parents: 6029 diff changeset	302 pos = bsearch(&rec, recs, count, sizeof(*recs), maildir_uid_cmp);
dcaa8d5296a9 Fixes to handling appends and unexpunges. Timo Sirainen <tss@iki.fi> parents: 6029 diff changeset	303 i_assert(pos != NULL);
dcaa8d5296a9 Fixes to handling appends and unexpunges. Timo Sirainen <tss@iki.fi> parents: 6029 diff changeset	304
dcaa8d5296a9 Fixes to handling appends and unexpunges. Timo Sirainen <tss@iki.fi> parents: 6029 diff changeset	305 array_delete(&uidlist->records, pos - recs, 1);
dcaa8d5296a9 Fixes to handling appends and unexpunges. Timo Sirainen <tss@iki.fi> parents: 6029 diff changeset	306 }
dcaa8d5296a9 Fixes to handling appends and unexpunges. Timo Sirainen <tss@iki.fi> parents: 6029 diff changeset	307
5944 760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	308 static bool
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	309 maildir_uidlist_read_extended(struct maildir_uidlist *uidlist,
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	310 const char **line_p,
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	311 struct maildir_uidlist_rec *rec)
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	312 {
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	313 const char start, line = *line_p;
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	314 buffer_t *buf;
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	315
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	316 t_push();
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	317 buf = buffer_create_dynamic(pool_datastack_create(), 128);
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	318 while (line != '\0' && line != ':') {
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	319 /* skip over an extension field */
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	320 start = line;
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	321 while (line != ' ' && line != '\0') line++;
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	322 buffer_append(buf, start, line - start);
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	323 buffer_append_c(buf, '\0');
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	324 while (*line == ' ') line++;
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	325 }
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	326
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	327 if (buf->used > 0) {
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	328 /* save the extensions */
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	329 buffer_append_c(buf, '\0');
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	330 rec->extensions = p_malloc(uidlist->record_pool, buf->used);
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	331 memcpy(rec->extensions, buf->data, buf->used);
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	332 }
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	333 t_pop();
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	334
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	335 if (*line == ':')
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	336 line++;
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	337 if (*line == '\0')
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	338 return FALSE;
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	339
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	340 *line_p = line;
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	341 return TRUE;
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	342 }
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	343
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	344 static int maildir_uidlist_next(struct maildir_uidlist *uidlist,
2565 e14c2a36a0a2 "UIDs not ordered in file" errors could have occured sometimes wrongly. Timo Sirainen <tss@iki.fi> parents: 2531 diff changeset	345 const char *line)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	346 {
6599 a6f51026b969 Added maildir_uidlist_init_readonly() and maildir_uidlist_get_uid(). Timo Sirainen <tss@iki.fi> parents: 6598 diff changeset	347 struct mail_storage *storage = uidlist->ibox->box.storage;
6034 dcaa8d5296a9 Fixes to handling appends and unexpunges. Timo Sirainen <tss@iki.fi> parents: 6029 diff changeset	348 struct maildir_uidlist_rec rec, old_rec;
5944 760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	349 uint32_t uid;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	350
5944 760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	351 uid = 0;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	352 while (line >= '0' && line <= '9') {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	353 uid = uid10 + (line - '0');
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	354 line++;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	355 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	356
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	357 if (uid == 0 \|\| *line != ' ') {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	358 /* invalid file */
5944 760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	359 mail_storage_set_critical(storage,
5917 525b449313a7 Don't use uidlist lock to update uidlist. Have a separate real dotlock for Timo Sirainen <tss@iki.fi> parents: 5915 diff changeset	360 "Invalid data in file %s", uidlist->path);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	361 return 0;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	362 }
2565 e14c2a36a0a2 "UIDs not ordered in file" errors could have occured sometimes wrongly. Timo Sirainen <tss@iki.fi> parents: 2531 diff changeset	363 if (uid <= uidlist->prev_read_uid) {
5944 760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	364 mail_storage_set_critical(storage,
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	365 "UIDs not ordered in file %s (%u > %u)",
5917 525b449313a7 Don't use uidlist lock to update uidlist. Have a separate real dotlock for Timo Sirainen <tss@iki.fi> parents: 5915 diff changeset	366 uidlist->path, uid, uidlist->prev_read_uid);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	367 return 0;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	368 }
2565 e14c2a36a0a2 "UIDs not ordered in file" errors could have occured sometimes wrongly. Timo Sirainen <tss@iki.fi> parents: 2531 diff changeset	369 uidlist->prev_read_uid = uid;
e14c2a36a0a2 "UIDs not ordered in file" errors could have occured sometimes wrongly. Timo Sirainen <tss@iki.fi> parents: 2531 diff changeset	370
e14c2a36a0a2 "UIDs not ordered in file" errors could have occured sometimes wrongly. Timo Sirainen <tss@iki.fi> parents: 2531 diff changeset	371 if (uid <= uidlist->last_seen_uid) {
e14c2a36a0a2 "UIDs not ordered in file" errors could have occured sometimes wrongly. Timo Sirainen <tss@iki.fi> parents: 2531 diff changeset	372 /* we already have this */
e14c2a36a0a2 "UIDs not ordered in file" errors could have occured sometimes wrongly. Timo Sirainen <tss@iki.fi> parents: 2531 diff changeset	373 return 1;
e14c2a36a0a2 "UIDs not ordered in file" errors could have occured sometimes wrongly. Timo Sirainen <tss@iki.fi> parents: 2531 diff changeset	374 }
e14c2a36a0a2 "UIDs not ordered in file" errors could have occured sometimes wrongly. Timo Sirainen <tss@iki.fi> parents: 2531 diff changeset	375 uidlist->last_seen_uid = uid;
1984 9c159272f721 syncing fixes Timo Sirainen <tss@iki.fi> parents: 1978 diff changeset	376
5958 5825961b9862 Make sure that uid < next_uid if uidlist version is 1. Timo Sirainen <tss@iki.fi> parents: 5957 diff changeset	377 if (uid >= uidlist->next_uid && uidlist->version == 1) {
5825961b9862 Make sure that uid < next_uid if uidlist version is 1. Timo Sirainen <tss@iki.fi> parents: 5957 diff changeset	378 mail_storage_set_critical(storage,
5825961b9862 Make sure that uid < next_uid if uidlist version is 1. Timo Sirainen <tss@iki.fi> parents: 5957 diff changeset	379 "UID larger than next_uid in file %s (%u >= %u)",
5825961b9862 Make sure that uid < next_uid if uidlist version is 1. Timo Sirainen <tss@iki.fi> parents: 5957 diff changeset	380 uidlist->path, uid, uidlist->next_uid);
5825961b9862 Make sure that uid < next_uid if uidlist version is 1. Timo Sirainen <tss@iki.fi> parents: 5957 diff changeset	381 return 0;
5825961b9862 Make sure that uid < next_uid if uidlist version is 1. Timo Sirainen <tss@iki.fi> parents: 5957 diff changeset	382 }
5825961b9862 Make sure that uid < next_uid if uidlist version is 1. Timo Sirainen <tss@iki.fi> parents: 5957 diff changeset	383
5944 760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	384 rec = p_new(uidlist->record_pool, struct maildir_uidlist_rec, 1);
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	385 rec->uid = uid;
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	386 rec->flags = MAILDIR_UIDLIST_REC_FLAG_NONSYNCED;
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	387
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	388 while (*line == ' ') line++;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	389
5944 760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	390 if (uidlist->version == 3) {
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	391 /* read extended fields */
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	392 if (!maildir_uidlist_read_extended(uidlist, &line, rec)) {
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	393 mail_storage_set_critical(storage,
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	394 "Invalid data in file %s", uidlist->path);
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	395 return 0;
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	396 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	397 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	398
6034 dcaa8d5296a9 Fixes to handling appends and unexpunges. Timo Sirainen <tss@iki.fi> parents: 6029 diff changeset	399 old_rec = hash_lookup(uidlist->files, line);
dcaa8d5296a9 Fixes to handling appends and unexpunges. Timo Sirainen <tss@iki.fi> parents: 6029 diff changeset	400 if (old_rec != NULL) {
dcaa8d5296a9 Fixes to handling appends and unexpunges. Timo Sirainen <tss@iki.fi> parents: 6029 diff changeset	401 /* This can happen if expunged file is moved back and the file
dcaa8d5296a9 Fixes to handling appends and unexpunges. Timo Sirainen <tss@iki.fi> parents: 6029 diff changeset	402 was appended to uidlist. */
dcaa8d5296a9 Fixes to handling appends and unexpunges. Timo Sirainen <tss@iki.fi> parents: 6029 diff changeset	403 i_warning("%s: Duplicate file entry: %s", uidlist->path, line);
dcaa8d5296a9 Fixes to handling appends and unexpunges. Timo Sirainen <tss@iki.fi> parents: 6029 diff changeset	404 /* Delete the old UID */
dcaa8d5296a9 Fixes to handling appends and unexpunges. Timo Sirainen <tss@iki.fi> parents: 6029 diff changeset	405 maildir_uidlist_records_array_delete(uidlist, old_rec);
dcaa8d5296a9 Fixes to handling appends and unexpunges. Timo Sirainen <tss@iki.fi> parents: 6029 diff changeset	406 /* Replace the old record with this new one */
dcaa8d5296a9 Fixes to handling appends and unexpunges. Timo Sirainen <tss@iki.fi> parents: 6029 diff changeset	407 old_rec = rec;
dcaa8d5296a9 Fixes to handling appends and unexpunges. Timo Sirainen <tss@iki.fi> parents: 6029 diff changeset	408 rec = old_rec;
dcaa8d5296a9 Fixes to handling appends and unexpunges. Timo Sirainen <tss@iki.fi> parents: 6029 diff changeset	409 uidlist->recreate = TRUE;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	410 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	411
1962 4775ffa73874 Syncing should be fixed now. Timo Sirainen <tss@iki.fi> parents: 1960 diff changeset	412 rec->filename = p_strdup(uidlist->record_pool, line);
4775ffa73874 Syncing should be fixed now. Timo Sirainen <tss@iki.fi> parents: 1960 diff changeset	413 hash_insert(uidlist->files, rec->filename, rec);
5890 989abd5017a4 Changed record buffers to arrays. Timo Sirainen <tss@iki.fi> parents: 5740 diff changeset	414 array_append(&uidlist->records, &rec, 1);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	415 return 1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	416 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	417
5933 fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	418 static int maildir_uidlist_read_header(struct maildir_uidlist *uidlist,
fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	419 struct istream *input)
fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	420 {
6599 a6f51026b969 Added maildir_uidlist_init_readonly() and maildir_uidlist_get_uid(). Timo Sirainen <tss@iki.fi> parents: 6598 diff changeset	421 struct mail_storage *storage = uidlist->ibox->box.storage;
5933 fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	422 unsigned int uid_validity, next_uid;
5944 760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	423 string_t *ext_hdr;
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	424 const char line, value;
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	425 char key;
5933 fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	426
fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	427 line = i_stream_read_next_line(input);
fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	428 if (line == NULL) {
fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	429 /* I/O error / empty file */
fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	430 return input->stream_errno == 0 ? 0 : -1;
fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	431 }
fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	432
5944 760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	433 if (line < '0' \|\| line > '9' \|\| line[1] != ' ') {
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	434 mail_storage_set_critical(storage,
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	435 "%s: Corrupted header (invalid version number)",
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	436 uidlist->path);
5933 fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	437 return 0;
fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	438 }
5944 760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	439
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	440 uidlist->version = *line - '0';
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	441 line += 2;
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	442
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	443 switch (uidlist->version) {
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	444 case 1:
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	445 if (sscanf(line, "%u %u", &uid_validity, &next_uid) != 2) {
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	446 mail_storage_set_critical(storage,
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	447 "%s: Corrupted header (version 1)",
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	448 uidlist->path);
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	449 return 0;
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	450 }
5958 5825961b9862 Make sure that uid < next_uid if uidlist version is 1. Timo Sirainen <tss@iki.fi> parents: 5957 diff changeset	451 if (uid_validity == uidlist->uid_validity &&
5825961b9862 Make sure that uid < next_uid if uidlist version is 1. Timo Sirainen <tss@iki.fi> parents: 5957 diff changeset	452 next_uid < uidlist->next_uid) {
5825961b9862 Make sure that uid < next_uid if uidlist version is 1. Timo Sirainen <tss@iki.fi> parents: 5957 diff changeset	453 mail_storage_set_critical(storage,
5825961b9862 Make sure that uid < next_uid if uidlist version is 1. Timo Sirainen <tss@iki.fi> parents: 5957 diff changeset	454 "%s: next_uid was lowered (v1, %u -> %u)",
5825961b9862 Make sure that uid < next_uid if uidlist version is 1. Timo Sirainen <tss@iki.fi> parents: 5957 diff changeset	455 uidlist->path, uidlist->next_uid, next_uid);
5825961b9862 Make sure that uid < next_uid if uidlist version is 1. Timo Sirainen <tss@iki.fi> parents: 5957 diff changeset	456 return 0;
5825961b9862 Make sure that uid < next_uid if uidlist version is 1. Timo Sirainen <tss@iki.fi> parents: 5957 diff changeset	457 }
5944 760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	458 break;
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	459 case 3:
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	460 ext_hdr = uidlist->hdr_extensions;
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	461 str_truncate(ext_hdr, 0);
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	462 while (*line != '\0') {
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	463 t_push();
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	464 key = *line;
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	465 value = ++line;
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	466 while (line != '\0' && line != ' ') line++;
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	467 value = t_strdup_until(value, line);
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	468
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	469 switch (key) {
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	470 case MAILDIR_UIDLIST_HDR_EXT_UID_VALIDITY:
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	471 uid_validity = strtoul(value, NULL, 10);
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	472 break;
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	473 case MAILDIR_UIDLIST_HDR_EXT_NEXT_UID:
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	474 next_uid = strtoul(value, NULL, 10);
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	475 break;
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	476 default:
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	477 if (str_len(ext_hdr) > 0)
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	478 str_append_c(ext_hdr, ' ');
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	479 str_printfa(ext_hdr, "%c%s", key, value);
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	480 break;
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	481 }
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	482
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	483 while (*line == ' ') line++;
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	484 t_pop();
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	485 }
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	486 break;
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	487 default:
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	488 mail_storage_set_critical(storage, "%s: Unsupported version %u",
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	489 uidlist->path, uidlist->version);
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	490 return 0;
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	491 }
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	492
5933 fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	493 if (uid_validity == 0 \|\| next_uid == 0) {
5944 760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	494 mail_storage_set_critical(storage,
5933 fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	495 "%s: Broken header (uidvalidity = %u, next_uid=%u)",
fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	496 uidlist->path, uid_validity, next_uid);
fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	497 return 0;
fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	498 }
fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	499
fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	500 uidlist->uid_validity = uid_validity;
fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	501 uidlist->next_uid = next_uid;
fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	502 return 1;
fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	503 }
fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	504
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	505 static int
4072 7e6acdd8d18d If we reach the NFS_ESTALE_RETRY_COUNT, give an ESTALE error message as Timo Sirainen <tss@iki.fi> parents: 4071 diff changeset	506 maildir_uidlist_update_read(struct maildir_uidlist *uidlist,
7e6acdd8d18d If we reach the NFS_ESTALE_RETRY_COUNT, give an ESTALE error message as Timo Sirainen <tss@iki.fi> parents: 4071 diff changeset	507 bool *retry_r, bool try_retry)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	508 {
6599 a6f51026b969 Added maildir_uidlist_init_readonly() and maildir_uidlist_get_uid(). Timo Sirainen <tss@iki.fi> parents: 6598 diff changeset	509 struct mail_storage *storage = uidlist->ibox->box.storage;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	510 const char *line;
5933 fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	511 unsigned int orig_next_uid;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	512 struct istream *input;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	513 struct stat st;
5933 fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	514 uoff_t last_read_offset;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	515 int fd, ret;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	516
5891 db959f0d94e3 Leave uidlist fd open so we can check if uidlist has been rewritten from Timo Sirainen <tss@iki.fi> parents: 5890 diff changeset	517 *retry_r = FALSE;
db959f0d94e3 Leave uidlist fd open so we can check if uidlist has been rewritten from Timo Sirainen <tss@iki.fi> parents: 5890 diff changeset	518
5933 fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	519 if (uidlist->fd == -1) {
fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	520 fd = nfs_safe_open(uidlist->path, O_RDWR);
fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	521 if (fd == -1) {
fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	522 if (errno != ENOENT) {
fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	523 mail_storage_set_critical(storage,
fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	524 "open(%s) failed: %m", uidlist->path);
fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	525 return -1;
fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	526 }
fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	527 return 0;
fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	528 }
fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	529 last_read_offset = 0;
fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	530 } else {
fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	531 /* the file was updated */
fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	532 fd = uidlist->fd;
fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	533 if (lseek(fd, 0, SEEK_SET) < 0) {
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	534 mail_storage_set_critical(storage,
5933 fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	535 "lseek(%s) failed: %m", uidlist->path);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	536 return -1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	537 }
5933 fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	538 uidlist->fd = -1;
fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	539 uidlist->fd_ino = 0;
fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	540 last_read_offset = uidlist->last_read_offset;
fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	541 uidlist->last_read_offset = 0;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	542 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	543
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	544 if (fstat(fd, &st) < 0) {
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	545 close_keep_errno(fd);
4072 7e6acdd8d18d If we reach the NFS_ESTALE_RETRY_COUNT, give an ESTALE error message as Timo Sirainen <tss@iki.fi> parents: 4071 diff changeset	546 if (errno == ESTALE && try_retry) {
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	547 *retry_r = TRUE;
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	548 return -1;
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	549 }
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	550 mail_storage_set_critical(storage,
5917 525b449313a7 Don't use uidlist lock to update uidlist. Have a separate real dotlock for Timo Sirainen <tss@iki.fi> parents: 5915 diff changeset	551 "fstat(%s) failed: %m", uidlist->path);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	552 return -1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	553 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	554
1962 4775ffa73874 Syncing should be fixed now. Timo Sirainen <tss@iki.fi> parents: 1960 diff changeset	555 if (uidlist->record_pool == NULL) {
4775ffa73874 Syncing should be fixed now. Timo Sirainen <tss@iki.fi> parents: 1960 diff changeset	556 uidlist->record_pool =
4980 8e88ecc64563 debug: MEMPOOL_GROWING prefix in alloconly pool names means that when Timo Sirainen <tss@iki.fi> parents: 4969 diff changeset	557 pool_alloconly_create(MEMPOOL_GROWING
8e88ecc64563 debug: MEMPOOL_GROWING prefix in alloconly pool names means that when Timo Sirainen <tss@iki.fi> parents: 4969 diff changeset	558 "uidlist record_pool",
1931 35a947481130 fixes Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	559 nearest_power(st.st_size -
35a947481130 fixes Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	560 st.st_size/8));
35a947481130 fixes Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	561 }
35a947481130 fixes Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	562
6162 896cc473c1f0 Renamed i_stream_create_file() to i_stream_create_fd(). Timo Sirainen <tss@iki.fi> parents: 6161 diff changeset	563 input = i_stream_create_fd(fd, 4096, FALSE);
5933 fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	564 i_stream_seek(input, uidlist->last_read_offset);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	565
5933 fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	566 orig_next_uid = uidlist->next_uid;
fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	567 ret = input->v_offset != 0 ? 1 :
fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	568 maildir_uidlist_read_header(uidlist, input);
fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	569 if (ret > 0) {
2565 e14c2a36a0a2 "UIDs not ordered in file" errors could have occured sometimes wrongly. Timo Sirainen <tss@iki.fi> parents: 2531 diff changeset	570 uidlist->prev_read_uid = 0;
5913 699356ffe549 If uidlist changes while it's being iterated, don't crash. Timo Sirainen <tss@iki.fi> parents: 5903 diff changeset	571 uidlist->change_counter++;
5933 fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	572 uidlist->read_records_count = 0;
2094 8f46cf3e9b5a small fixes Timo Sirainen <tss@iki.fi> parents: 2053 diff changeset	573
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	574 ret = 1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	575 while ((line = i_stream_read_next_line(input)) != NULL) {
5933 fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	576 uidlist->read_records_count++;
2565 e14c2a36a0a2 "UIDs not ordered in file" errors could have occured sometimes wrongly. Timo Sirainen <tss@iki.fi> parents: 2531 diff changeset	577 if (!maildir_uidlist_next(uidlist, line)) {
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	578 ret = 0;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	579 break;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	580 }
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	581 }
5933 fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	582 if (input->stream_errno != 0)
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	583 ret = -1;
5933 fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	584
fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	585 if (uidlist->next_uid <= uidlist->prev_read_uid)
fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	586 uidlist->next_uid = uidlist->prev_read_uid + 1;
fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	587 if (uidlist->next_uid < orig_next_uid) {
fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	588 mail_storage_set_critical(storage,
fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	589 "%s: next_uid was lowered (%u -> %u)",
fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	590 uidlist->path, orig_next_uid,
fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	591 uidlist->next_uid);
fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	592 uidlist->recreate = TRUE;
fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	593 uidlist->next_uid = orig_next_uid;
fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	594 }
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	595 }
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	596
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	597 if (ret == 0) {
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	598 /* file is broken */
5917 525b449313a7 Don't use uidlist lock to update uidlist. Have a separate real dotlock for Timo Sirainen <tss@iki.fi> parents: 5915 diff changeset	599 (void)unlink(uidlist->path);
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	600 } else if (ret > 0) {
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	601 /* success */
5891 db959f0d94e3 Leave uidlist fd open so we can check if uidlist has been rewritten from Timo Sirainen <tss@iki.fi> parents: 5890 diff changeset	602 uidlist->fd = fd;
db959f0d94e3 Leave uidlist fd open so we can check if uidlist has been rewritten from Timo Sirainen <tss@iki.fi> parents: 5890 diff changeset	603 uidlist->fd_dev = st.st_dev;
db959f0d94e3 Leave uidlist fd open so we can check if uidlist has been rewritten from Timo Sirainen <tss@iki.fi> parents: 5890 diff changeset	604 uidlist->fd_ino = st.st_ino;
5933 fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	605 uidlist->fd_size = st.st_size;
fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	606 uidlist->last_read_offset = input->v_offset;
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	607 } else {
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	608 /* I/O error */
4072 7e6acdd8d18d If we reach the NFS_ESTALE_RETRY_COUNT, give an ESTALE error message as Timo Sirainen <tss@iki.fi> parents: 4071 diff changeset	609 if (input->stream_errno == ESTALE && try_retry)
7e6acdd8d18d If we reach the NFS_ESTALE_RETRY_COUNT, give an ESTALE error message as Timo Sirainen <tss@iki.fi> parents: 4071 diff changeset	610 *retry_r = TRUE;
7e6acdd8d18d If we reach the NFS_ESTALE_RETRY_COUNT, give an ESTALE error message as Timo Sirainen <tss@iki.fi> parents: 4071 diff changeset	611 else {
7e6acdd8d18d If we reach the NFS_ESTALE_RETRY_COUNT, give an ESTALE error message as Timo Sirainen <tss@iki.fi> parents: 4071 diff changeset	612 errno = input->stream_errno;
7e6acdd8d18d If we reach the NFS_ESTALE_RETRY_COUNT, give an ESTALE error message as Timo Sirainen <tss@iki.fi> parents: 4071 diff changeset	613 mail_storage_set_critical(storage,
5917 525b449313a7 Don't use uidlist lock to update uidlist. Have a separate real dotlock for Timo Sirainen <tss@iki.fi> parents: 5915 diff changeset	614 "read(%s) failed: %m", uidlist->path);
4072 7e6acdd8d18d If we reach the NFS_ESTALE_RETRY_COUNT, give an ESTALE error message as Timo Sirainen <tss@iki.fi> parents: 4071 diff changeset	615 }
5891 db959f0d94e3 Leave uidlist fd open so we can check if uidlist has been rewritten from Timo Sirainen <tss@iki.fi> parents: 5890 diff changeset	616 }
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	617
4070 71b8faa84ec6 Added i_stream_destroy() and o_stream_destroy() and used them instead of Timo Sirainen <tss@iki.fi> parents: 4023 diff changeset	618 i_stream_destroy(&input);
5891 db959f0d94e3 Leave uidlist fd open so we can check if uidlist has been rewritten from Timo Sirainen <tss@iki.fi> parents: 5890 diff changeset	619 if (ret <= 0) {
db959f0d94e3 Leave uidlist fd open so we can check if uidlist has been rewritten from Timo Sirainen <tss@iki.fi> parents: 5890 diff changeset	620 if (close(fd) < 0)
5917 525b449313a7 Don't use uidlist lock to update uidlist. Have a separate real dotlock for Timo Sirainen <tss@iki.fi> parents: 5915 diff changeset	621 i_error("close(%s) failed: %m", uidlist->path);
5891 db959f0d94e3 Leave uidlist fd open so we can check if uidlist has been rewritten from Timo Sirainen <tss@iki.fi> parents: 5890 diff changeset	622 }
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	623 return ret;
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	624 }
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	625
5933 fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	626 static int
6801 6a5540e5a464 NFS attribute cache flushing optimization Timo Sirainen <tss@iki.fi> parents: 6800 diff changeset	627 maildir_uidlist_has_changed(struct maildir_uidlist *uidlist, bool nfs_flush,
6a5540e5a464 NFS attribute cache flushing optimization Timo Sirainen <tss@iki.fi> parents: 6800 diff changeset	628 bool *recreated_r)
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	629 {
6599 a6f51026b969 Added maildir_uidlist_init_readonly() and maildir_uidlist_get_uid(). Timo Sirainen <tss@iki.fi> parents: 6598 diff changeset	630 struct mail_storage *storage = uidlist->ibox->box.storage;
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	631 struct stat st;
5933 fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	632
fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	633 *recreated_r = FALSE;
fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	634
fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	635 if (nfs_safe_stat(uidlist->path, &st) < 0) {
fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	636 if (errno != ENOENT) {
fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	637 mail_storage_set_critical(storage,
fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	638 "stat(%s) failed: %m", uidlist->path);
fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	639 return -1;
fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	640 }
fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	641 return 0;
fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	642 }
fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	643
fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	644 if (st.st_ino != uidlist->fd_ino \|\|
fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	645 !CMP_DEV_T(st.st_dev, uidlist->fd_dev)) {
fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	646 /* file recreated */
fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	647 *recreated_r = TRUE;
fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	648 return 1;
6797 a59deefc552f NFS attribute cache flushing fixes. nfs_flush_attr_cache() takes now Timo Sirainen <tss@iki.fi> parents: 6777 diff changeset	649 }
a59deefc552f NFS attribute cache flushing fixes. nfs_flush_attr_cache() takes now Timo Sirainen <tss@iki.fi> parents: 6777 diff changeset	650
6798 e70686f19f67 Don't flush attribute cache unless we're locked and we really need to know Timo Sirainen <tss@iki.fi> parents: 6797 diff changeset	651 if ((storage->flags & MAIL_STORAGE_FLAG_NFS_FLUSH_STORAGE) != 0 &&
6801 6a5540e5a464 NFS attribute cache flushing optimization Timo Sirainen <tss@iki.fi> parents: 6800 diff changeset	652 nfs_flush) {
6798 e70686f19f67 Don't flush attribute cache unless we're locked and we really need to know Timo Sirainen <tss@iki.fi> parents: 6797 diff changeset	653 /* NFS: either the file hasn't been changed, or it has already
e70686f19f67 Don't flush attribute cache unless we're locked and we really need to know Timo Sirainen <tss@iki.fi> parents: 6797 diff changeset	654 been deleted and the inodes just happen to be the same.
e70686f19f67 Don't flush attribute cache unless we're locked and we really need to know Timo Sirainen <tss@iki.fi> parents: 6797 diff changeset	655 check if the fd is still valid. */
6829 dbab5e592577 NFS cache flushing APIs changed and backend implements it a bit differently. Timo Sirainen <tss@iki.fi> parents: 6819 diff changeset	656 if (fstat(uidlist->fd, &st) < 0) {
dbab5e592577 NFS cache flushing APIs changed and backend implements it a bit differently. Timo Sirainen <tss@iki.fi> parents: 6819 diff changeset	657 if (errno == ESTALE) {
dbab5e592577 NFS cache flushing APIs changed and backend implements it a bit differently. Timo Sirainen <tss@iki.fi> parents: 6819 diff changeset	658 *recreated_r = TRUE;
dbab5e592577 NFS cache flushing APIs changed and backend implements it a bit differently. Timo Sirainen <tss@iki.fi> parents: 6819 diff changeset	659 return 1;
dbab5e592577 NFS cache flushing APIs changed and backend implements it a bit differently. Timo Sirainen <tss@iki.fi> parents: 6819 diff changeset	660 }
dbab5e592577 NFS cache flushing APIs changed and backend implements it a bit differently. Timo Sirainen <tss@iki.fi> parents: 6819 diff changeset	661 mail_storage_set_critical(storage,
dbab5e592577 NFS cache flushing APIs changed and backend implements it a bit differently. Timo Sirainen <tss@iki.fi> parents: 6819 diff changeset	662 "fstat(%s) failed: %m", uidlist->path);
dbab5e592577 NFS cache flushing APIs changed and backend implements it a bit differently. Timo Sirainen <tss@iki.fi> parents: 6819 diff changeset	663 return -1;
6798 e70686f19f67 Don't flush attribute cache unless we're locked and we really need to know Timo Sirainen <tss@iki.fi> parents: 6797 diff changeset	664 }
6797 a59deefc552f NFS attribute cache flushing fixes. nfs_flush_attr_cache() takes now Timo Sirainen <tss@iki.fi> parents: 6777 diff changeset	665 }
a59deefc552f NFS attribute cache flushing fixes. nfs_flush_attr_cache() takes now Timo Sirainen <tss@iki.fi> parents: 6777 diff changeset	666
a59deefc552f NFS attribute cache flushing fixes. nfs_flush_attr_cache() takes now Timo Sirainen <tss@iki.fi> parents: 6777 diff changeset	667 if (st.st_size != uidlist->fd_size) {
5933 fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	668 /* file modified but not recreated */
fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	669 return 1;
fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	670 } else {
fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	671 /* unchanged */
fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	672 return 0;
fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	673 }
fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	674 }
fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	675
6800 ca849e28a062 NFS: Delay flushing dovecot-uidlist's attribute cache until we find an Timo Sirainen <tss@iki.fi> parents: 6798 diff changeset	676 int maildir_uidlist_refresh(struct maildir_uidlist *uidlist, bool nfs_flush)
5933 fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	677 {
6797 a59deefc552f NFS attribute cache flushing fixes. nfs_flush_attr_cache() takes now Timo Sirainen <tss@iki.fi> parents: 6777 diff changeset	678 struct mail_storage *storage = uidlist->ibox->box.storage;
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	679 unsigned int i;
5933 fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	680 bool retry, recreated;
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	681 int ret;
b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	682
6800 ca849e28a062 NFS: Delay flushing dovecot-uidlist's attribute cache until we find an Timo Sirainen <tss@iki.fi> parents: 6798 diff changeset	683 if ((storage->flags & MAIL_STORAGE_FLAG_NFS_FLUSH_STORAGE) != 0) {
ca849e28a062 NFS: Delay flushing dovecot-uidlist's attribute cache until we find an Timo Sirainen <tss@iki.fi> parents: 6798 diff changeset	684 /* delay flushing attribute cache until we find a file that
ca849e28a062 NFS: Delay flushing dovecot-uidlist's attribute cache until we find an Timo Sirainen <tss@iki.fi> parents: 6798 diff changeset	685 doesn't exist in the uidlist. only then we really must have
ca849e28a062 NFS: Delay flushing dovecot-uidlist's attribute cache until we find an Timo Sirainen <tss@iki.fi> parents: 6798 diff changeset	686 the latest uidlist. */
ca849e28a062 NFS: Delay flushing dovecot-uidlist's attribute cache until we find an Timo Sirainen <tss@iki.fi> parents: 6798 diff changeset	687 if (!nfs_flush)
ca849e28a062 NFS: Delay flushing dovecot-uidlist's attribute cache until we find an Timo Sirainen <tss@iki.fi> parents: 6798 diff changeset	688 uidlist->nfs_dirty_refresh = TRUE;
ca849e28a062 NFS: Delay flushing dovecot-uidlist's attribute cache until we find an Timo Sirainen <tss@iki.fi> parents: 6798 diff changeset	689 else {
6857 41911abe6fa7 NFS cache flushing updates. Timo Sirainen <tss@iki.fi> parents: 6829 diff changeset	690 nfs_flush_file_handle_cache(uidlist->path);
6829 dbab5e592577 NFS cache flushing APIs changed and backend implements it a bit differently. Timo Sirainen <tss@iki.fi> parents: 6819 diff changeset	691 nfs_flush_attr_cache_unlocked(uidlist->path);
6800 ca849e28a062 NFS: Delay flushing dovecot-uidlist's attribute cache until we find an Timo Sirainen <tss@iki.fi> parents: 6798 diff changeset	692 uidlist->nfs_dirty_refresh = FALSE;
ca849e28a062 NFS: Delay flushing dovecot-uidlist's attribute cache until we find an Timo Sirainen <tss@iki.fi> parents: 6798 diff changeset	693 }
ca849e28a062 NFS: Delay flushing dovecot-uidlist's attribute cache until we find an Timo Sirainen <tss@iki.fi> parents: 6798 diff changeset	694 }
ca849e28a062 NFS: Delay flushing dovecot-uidlist's attribute cache until we find an Timo Sirainen <tss@iki.fi> parents: 6798 diff changeset	695
5891 db959f0d94e3 Leave uidlist fd open so we can check if uidlist has been rewritten from Timo Sirainen <tss@iki.fi> parents: 5890 diff changeset	696 if (uidlist->fd != -1) {
6801 6a5540e5a464 NFS attribute cache flushing optimization Timo Sirainen <tss@iki.fi> parents: 6800 diff changeset	697 ret = maildir_uidlist_has_changed(uidlist, nfs_flush,
6a5540e5a464 NFS attribute cache flushing optimization Timo Sirainen <tss@iki.fi> parents: 6800 diff changeset	698 &recreated);
5933 fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	699 if (ret <= 0)
fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	700 return ret;
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	701
5933 fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	702 if (recreated)
fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	703 maildir_uidlist_close(uidlist);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	704 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	705
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	706 for (i = 0; ; i++) {
4072 7e6acdd8d18d If we reach the NFS_ESTALE_RETRY_COUNT, give an ESTALE error message as Timo Sirainen <tss@iki.fi> parents: 4071 diff changeset	707 ret = maildir_uidlist_update_read(uidlist, &retry,
7e6acdd8d18d If we reach the NFS_ESTALE_RETRY_COUNT, give an ESTALE error message as Timo Sirainen <tss@iki.fi> parents: 4071 diff changeset	708 i < UIDLIST_ESTALE_RETRY_COUNT);
5933 fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	709 if (!retry)
fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	710 break;
6797 a59deefc552f NFS attribute cache flushing fixes. nfs_flush_attr_cache() takes now Timo Sirainen <tss@iki.fi> parents: 6777 diff changeset	711 /* ESTALE - try reopening and rereading */
a59deefc552f NFS attribute cache flushing fixes. nfs_flush_attr_cache() takes now Timo Sirainen <tss@iki.fi> parents: 6777 diff changeset	712 maildir_uidlist_close(uidlist);
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	713 }
5933 fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	714 if (ret >= 0)
fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	715 uidlist->initial_read = TRUE;
4023 b19ccbaa3802 Try to handle ESTALE NFS errors the best way we can. Timo Sirainen <timo.sirainen@movial.fi> parents: 3981 diff changeset	716 return ret;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	717 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	718
5950 49aebb3028f7 Added support for updating uidlist record extensions. Timo Sirainen <tss@iki.fi> parents: 5944 diff changeset	719 static struct maildir_uidlist_rec *
1947 777da553d1d3 Recent-flag should work now Timo Sirainen <tss@iki.fi> parents: 1940 diff changeset	720 maildir_uidlist_lookup_rec(struct maildir_uidlist *uidlist, uint32_t uid,
777da553d1d3 Recent-flag should work now Timo Sirainen <tss@iki.fi> parents: 1940 diff changeset	721 unsigned int *idx_r)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	722 {
5890 989abd5017a4 Changed record buffers to arrays. Timo Sirainen <tss@iki.fi> parents: 5740 diff changeset	723 struct maildir_uidlist_rec const recs;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	724 unsigned int idx, left_idx, right_idx;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	725
1962 4775ffa73874 Syncing should be fixed now. Timo Sirainen <tss@iki.fi> parents: 1960 diff changeset	726 if (!uidlist->initial_read) {
1954 2f6e137cdc44 Syncing optimizations. Timo Sirainen <tss@iki.fi> parents: 1947 diff changeset	727 /* first time we need to read uidlist */
6800 ca849e28a062 NFS: Delay flushing dovecot-uidlist's attribute cache until we find an Timo Sirainen <tss@iki.fi> parents: 6798 diff changeset	728 if (maildir_uidlist_refresh(uidlist, FALSE) < 0)
1954 2f6e137cdc44 Syncing optimizations. Timo Sirainen <tss@iki.fi> parents: 1947 diff changeset	729 return NULL;
2f6e137cdc44 Syncing optimizations. Timo Sirainen <tss@iki.fi> parents: 1947 diff changeset	730 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	731
5890 989abd5017a4 Changed record buffers to arrays. Timo Sirainen <tss@iki.fi> parents: 5740 diff changeset	732 idx = left_idx = 0;
989abd5017a4 Changed record buffers to arrays. Timo Sirainen <tss@iki.fi> parents: 5740 diff changeset	733 recs = array_get(&uidlist->records, &right_idx);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	734 while (left_idx < right_idx) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	735 idx = (left_idx + right_idx) / 2;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	736
5890 989abd5017a4 Changed record buffers to arrays. Timo Sirainen <tss@iki.fi> parents: 5740 diff changeset	737 if (recs[idx]->uid < uid)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	738 left_idx = idx+1;
5890 989abd5017a4 Changed record buffers to arrays. Timo Sirainen <tss@iki.fi> parents: 5740 diff changeset	739 else if (recs[idx]->uid > uid)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	740 right_idx = idx;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	741 else {
1947 777da553d1d3 Recent-flag should work now Timo Sirainen <tss@iki.fi> parents: 1940 diff changeset	742 *idx_r = idx;
5890 989abd5017a4 Changed record buffers to arrays. Timo Sirainen <tss@iki.fi> parents: 5740 diff changeset	743 return recs[idx];
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	744 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	745 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	746
6800 ca849e28a062 NFS: Delay flushing dovecot-uidlist's attribute cache until we find an Timo Sirainen <tss@iki.fi> parents: 6798 diff changeset	747 if (uidlist->nfs_dirty_refresh) {
ca849e28a062 NFS: Delay flushing dovecot-uidlist's attribute cache until we find an Timo Sirainen <tss@iki.fi> parents: 6798 diff changeset	748 /* We haven't flushed NFS attribute cache yet, do that
ca849e28a062 NFS: Delay flushing dovecot-uidlist's attribute cache until we find an Timo Sirainen <tss@iki.fi> parents: 6798 diff changeset	749 to make sure the UID really doesn't exist */
ca849e28a062 NFS: Delay flushing dovecot-uidlist's attribute cache until we find an Timo Sirainen <tss@iki.fi> parents: 6798 diff changeset	750 if (maildir_uidlist_refresh(uidlist, TRUE) < 0)
ca849e28a062 NFS: Delay flushing dovecot-uidlist's attribute cache until we find an Timo Sirainen <tss@iki.fi> parents: 6798 diff changeset	751 return NULL;
ca849e28a062 NFS: Delay flushing dovecot-uidlist's attribute cache until we find an Timo Sirainen <tss@iki.fi> parents: 6798 diff changeset	752 return maildir_uidlist_lookup_rec(uidlist, uid, idx_r);
ca849e28a062 NFS: Delay flushing dovecot-uidlist's attribute cache until we find an Timo Sirainen <tss@iki.fi> parents: 6798 diff changeset	753 }
ca849e28a062 NFS: Delay flushing dovecot-uidlist's attribute cache until we find an Timo Sirainen <tss@iki.fi> parents: 6798 diff changeset	754
1947 777da553d1d3 Recent-flag should work now Timo Sirainen <tss@iki.fi> parents: 1940 diff changeset	755 if (idx > 0) idx--;
777da553d1d3 Recent-flag should work now Timo Sirainen <tss@iki.fi> parents: 1940 diff changeset	756 *idx_r = idx;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	757 return NULL;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	758 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	759
1947 777da553d1d3 Recent-flag should work now Timo Sirainen <tss@iki.fi> parents: 1940 diff changeset	760 const char *
777da553d1d3 Recent-flag should work now Timo Sirainen <tss@iki.fi> parents: 1940 diff changeset	761 maildir_uidlist_lookup(struct maildir_uidlist *uidlist, uint32_t uid,
777da553d1d3 Recent-flag should work now Timo Sirainen <tss@iki.fi> parents: 1940 diff changeset	762 enum maildir_uidlist_rec_flag *flags_r)
777da553d1d3 Recent-flag should work now Timo Sirainen <tss@iki.fi> parents: 1940 diff changeset	763 {
777da553d1d3 Recent-flag should work now Timo Sirainen <tss@iki.fi> parents: 1940 diff changeset	764 const struct maildir_uidlist_rec *rec;
777da553d1d3 Recent-flag should work now Timo Sirainen <tss@iki.fi> parents: 1940 diff changeset	765 unsigned int idx;
777da553d1d3 Recent-flag should work now Timo Sirainen <tss@iki.fi> parents: 1940 diff changeset	766
777da553d1d3 Recent-flag should work now Timo Sirainen <tss@iki.fi> parents: 1940 diff changeset	767 rec = maildir_uidlist_lookup_rec(uidlist, uid, &idx);
5365 35f18edd5f17 If uidlist file doesn't exist but index files do, resync the maildir to Timo Sirainen <tss@iki.fi> parents: 5176 diff changeset	768 if (rec == NULL) {
6599 a6f51026b969 Added maildir_uidlist_init_readonly() and maildir_uidlist_get_uid(). Timo Sirainen <tss@iki.fi> parents: 6598 diff changeset	769 if (uidlist->fd != -1 \|\| uidlist->mbox == NULL)
5365 35f18edd5f17 If uidlist file doesn't exist but index files do, resync the maildir to Timo Sirainen <tss@iki.fi> parents: 5176 diff changeset	770 return NULL;
35f18edd5f17 If uidlist file doesn't exist but index files do, resync the maildir to Timo Sirainen <tss@iki.fi> parents: 5176 diff changeset	771
35f18edd5f17 If uidlist file doesn't exist but index files do, resync the maildir to Timo Sirainen <tss@iki.fi> parents: 5176 diff changeset	772 /* the uidlist doesn't exist. */
35f18edd5f17 If uidlist file doesn't exist but index files do, resync the maildir to Timo Sirainen <tss@iki.fi> parents: 5176 diff changeset	773 if (maildir_storage_sync_force(uidlist->mbox) < 0)
35f18edd5f17 If uidlist file doesn't exist but index files do, resync the maildir to Timo Sirainen <tss@iki.fi> parents: 5176 diff changeset	774 return NULL;
35f18edd5f17 If uidlist file doesn't exist but index files do, resync the maildir to Timo Sirainen <tss@iki.fi> parents: 5176 diff changeset	775
35f18edd5f17 If uidlist file doesn't exist but index files do, resync the maildir to Timo Sirainen <tss@iki.fi> parents: 5176 diff changeset	776 /* try again */
35f18edd5f17 If uidlist file doesn't exist but index files do, resync the maildir to Timo Sirainen <tss@iki.fi> parents: 5176 diff changeset	777 rec = maildir_uidlist_lookup_rec(uidlist, uid, &idx);
35f18edd5f17 If uidlist file doesn't exist but index files do, resync the maildir to Timo Sirainen <tss@iki.fi> parents: 5176 diff changeset	778 if (rec == NULL)
35f18edd5f17 If uidlist file doesn't exist but index files do, resync the maildir to Timo Sirainen <tss@iki.fi> parents: 5176 diff changeset	779 return NULL;
35f18edd5f17 If uidlist file doesn't exist but index files do, resync the maildir to Timo Sirainen <tss@iki.fi> parents: 5176 diff changeset	780 }
1947 777da553d1d3 Recent-flag should work now Timo Sirainen <tss@iki.fi> parents: 1940 diff changeset	781
777da553d1d3 Recent-flag should work now Timo Sirainen <tss@iki.fi> parents: 1940 diff changeset	782 *flags_r = rec->flags;
777da553d1d3 Recent-flag should work now Timo Sirainen <tss@iki.fi> parents: 1940 diff changeset	783 return rec->filename;
777da553d1d3 Recent-flag should work now Timo Sirainen <tss@iki.fi> parents: 1940 diff changeset	784 }
777da553d1d3 Recent-flag should work now Timo Sirainen <tss@iki.fi> parents: 1940 diff changeset	785
5944 760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	786 const char *
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	787 maildir_uidlist_lookup_ext(struct maildir_uidlist *uidlist, uint32_t uid,
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	788 enum maildir_uidlist_rec_ext_key key)
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	789 {
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	790 const struct maildir_uidlist_rec *rec;
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	791 unsigned int idx;
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	792 const char p, value;
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	793
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	794 rec = maildir_uidlist_lookup_rec(uidlist, uid, &idx);
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	795 if (rec == NULL \|\| rec->extensions == NULL)
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	796 return NULL;
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	797
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	798 p = rec->extensions; value = NULL;
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	799 while (*p != '\0') {
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	800 /* <key><value>\0 */
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	801 if (*p == (char)key)
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	802 return p + 1;
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	803
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	804 p += strlen(p) + 1;
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	805 }
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	806 return NULL;
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	807 }
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	808
2050 ee1095ccfd23 Index header changes now go through transaction log. Removed the kludgy Timo Sirainen <tss@iki.fi> parents: 2039 diff changeset	809 uint32_t maildir_uidlist_get_uid_validity(struct maildir_uidlist *uidlist)
ee1095ccfd23 Index header changes now go through transaction log. Removed the kludgy Timo Sirainen <tss@iki.fi> parents: 2039 diff changeset	810 {
ee1095ccfd23 Index header changes now go through transaction log. Removed the kludgy Timo Sirainen <tss@iki.fi> parents: 2039 diff changeset	811 return uidlist->uid_validity;
ee1095ccfd23 Index header changes now go through transaction log. Removed the kludgy Timo Sirainen <tss@iki.fi> parents: 2039 diff changeset	812 }
ee1095ccfd23 Index header changes now go through transaction log. Removed the kludgy Timo Sirainen <tss@iki.fi> parents: 2039 diff changeset	813
ee1095ccfd23 Index header changes now go through transaction log. Removed the kludgy Timo Sirainen <tss@iki.fi> parents: 2039 diff changeset	814 uint32_t maildir_uidlist_get_next_uid(struct maildir_uidlist *uidlist)
ee1095ccfd23 Index header changes now go through transaction log. Removed the kludgy Timo Sirainen <tss@iki.fi> parents: 2039 diff changeset	815 {
ee1095ccfd23 Index header changes now go through transaction log. Removed the kludgy Timo Sirainen <tss@iki.fi> parents: 2039 diff changeset	816 return !uidlist->initial_read ? 0 : uidlist->next_uid;
ee1095ccfd23 Index header changes now go through transaction log. Removed the kludgy Timo Sirainen <tss@iki.fi> parents: 2039 diff changeset	817 }
ee1095ccfd23 Index header changes now go through transaction log. Removed the kludgy Timo Sirainen <tss@iki.fi> parents: 2039 diff changeset	818
5932 6ac8d6c93d34 Handle next_uid changes better. Timo Sirainen <tss@iki.fi> parents: 5923 diff changeset	819 void maildir_uidlist_set_uid_validity(struct maildir_uidlist *uidlist,
6ac8d6c93d34 Handle next_uid changes better. Timo Sirainen <tss@iki.fi> parents: 5923 diff changeset	820 uint32_t uid_validity)
6ac8d6c93d34 Handle next_uid changes better. Timo Sirainen <tss@iki.fi> parents: 5923 diff changeset	821 {
6344 e45ba2950722 Added asserts against uid_validity==0. Timo Sirainen <tss@iki.fi> parents: 6162 diff changeset	822 i_assert(uid_validity != 0);
e45ba2950722 Added asserts against uid_validity==0. Timo Sirainen <tss@iki.fi> parents: 6162 diff changeset	823
5932 6ac8d6c93d34 Handle next_uid changes better. Timo Sirainen <tss@iki.fi> parents: 5923 diff changeset	824 uidlist->uid_validity = uid_validity;
6ac8d6c93d34 Handle next_uid changes better. Timo Sirainen <tss@iki.fi> parents: 5923 diff changeset	825 }
6ac8d6c93d34 Handle next_uid changes better. Timo Sirainen <tss@iki.fi> parents: 5923 diff changeset	826
6ac8d6c93d34 Handle next_uid changes better. Timo Sirainen <tss@iki.fi> parents: 5923 diff changeset	827 void maildir_uidlist_set_next_uid(struct maildir_uidlist *uidlist,
6ac8d6c93d34 Handle next_uid changes better. Timo Sirainen <tss@iki.fi> parents: 5923 diff changeset	828 uint32_t next_uid, bool force)
6ac8d6c93d34 Handle next_uid changes better. Timo Sirainen <tss@iki.fi> parents: 5923 diff changeset	829 {
6ac8d6c93d34 Handle next_uid changes better. Timo Sirainen <tss@iki.fi> parents: 5923 diff changeset	830 if (uidlist->next_uid < next_uid \|\| force)
6ac8d6c93d34 Handle next_uid changes better. Timo Sirainen <tss@iki.fi> parents: 5923 diff changeset	831 uidlist->next_uid = next_uid;
6ac8d6c93d34 Handle next_uid changes better. Timo Sirainen <tss@iki.fi> parents: 5923 diff changeset	832 }
6ac8d6c93d34 Handle next_uid changes better. Timo Sirainen <tss@iki.fi> parents: 5923 diff changeset	833
5950 49aebb3028f7 Added support for updating uidlist record extensions. Timo Sirainen <tss@iki.fi> parents: 5944 diff changeset	834 void maildir_uidlist_set_ext(struct maildir_uidlist *uidlist, uint32_t uid,
49aebb3028f7 Added support for updating uidlist record extensions. Timo Sirainen <tss@iki.fi> parents: 5944 diff changeset	835 enum maildir_uidlist_rec_ext_key key,
49aebb3028f7 Added support for updating uidlist record extensions. Timo Sirainen <tss@iki.fi> parents: 5944 diff changeset	836 const char *value)
49aebb3028f7 Added support for updating uidlist record extensions. Timo Sirainen <tss@iki.fi> parents: 5944 diff changeset	837 {
49aebb3028f7 Added support for updating uidlist record extensions. Timo Sirainen <tss@iki.fi> parents: 5944 diff changeset	838 struct maildir_uidlist_rec *rec;
49aebb3028f7 Added support for updating uidlist record extensions. Timo Sirainen <tss@iki.fi> parents: 5944 diff changeset	839 unsigned int idx;
49aebb3028f7 Added support for updating uidlist record extensions. Timo Sirainen <tss@iki.fi> parents: 5944 diff changeset	840 const char *p;
49aebb3028f7 Added support for updating uidlist record extensions. Timo Sirainen <tss@iki.fi> parents: 5944 diff changeset	841 buffer_t *buf;
49aebb3028f7 Added support for updating uidlist record extensions. Timo Sirainen <tss@iki.fi> parents: 5944 diff changeset	842 unsigned int len;
49aebb3028f7 Added support for updating uidlist record extensions. Timo Sirainen <tss@iki.fi> parents: 5944 diff changeset	843
49aebb3028f7 Added support for updating uidlist record extensions. Timo Sirainen <tss@iki.fi> parents: 5944 diff changeset	844 rec = maildir_uidlist_lookup_rec(uidlist, uid, &idx);
49aebb3028f7 Added support for updating uidlist record extensions. Timo Sirainen <tss@iki.fi> parents: 5944 diff changeset	845 i_assert(rec != NULL);
49aebb3028f7 Added support for updating uidlist record extensions. Timo Sirainen <tss@iki.fi> parents: 5944 diff changeset	846
49aebb3028f7 Added support for updating uidlist record extensions. Timo Sirainen <tss@iki.fi> parents: 5944 diff changeset	847 t_push();
49aebb3028f7 Added support for updating uidlist record extensions. Timo Sirainen <tss@iki.fi> parents: 5944 diff changeset	848 buf = buffer_create_dynamic(pool_datastack_create(), 128);
49aebb3028f7 Added support for updating uidlist record extensions. Timo Sirainen <tss@iki.fi> parents: 5944 diff changeset	849
49aebb3028f7 Added support for updating uidlist record extensions. Timo Sirainen <tss@iki.fi> parents: 5944 diff changeset	850 /* copy existing extensions, except for the one we're updating */
49aebb3028f7 Added support for updating uidlist record extensions. Timo Sirainen <tss@iki.fi> parents: 5944 diff changeset	851 if (rec->extensions != NULL) {
49aebb3028f7 Added support for updating uidlist record extensions. Timo Sirainen <tss@iki.fi> parents: 5944 diff changeset	852 p = rec->extensions;
49aebb3028f7 Added support for updating uidlist record extensions. Timo Sirainen <tss@iki.fi> parents: 5944 diff changeset	853 while (*p != '\0') {
49aebb3028f7 Added support for updating uidlist record extensions. Timo Sirainen <tss@iki.fi> parents: 5944 diff changeset	854 /* <key><value>\0 */
49aebb3028f7 Added support for updating uidlist record extensions. Timo Sirainen <tss@iki.fi> parents: 5944 diff changeset	855 len = strlen(p) + 1;
49aebb3028f7 Added support for updating uidlist record extensions. Timo Sirainen <tss@iki.fi> parents: 5944 diff changeset	856 if (*p != (char)key)
49aebb3028f7 Added support for updating uidlist record extensions. Timo Sirainen <tss@iki.fi> parents: 5944 diff changeset	857 buffer_append(buf, p, len);
49aebb3028f7 Added support for updating uidlist record extensions. Timo Sirainen <tss@iki.fi> parents: 5944 diff changeset	858 p += len;
49aebb3028f7 Added support for updating uidlist record extensions. Timo Sirainen <tss@iki.fi> parents: 5944 diff changeset	859 }
49aebb3028f7 Added support for updating uidlist record extensions. Timo Sirainen <tss@iki.fi> parents: 5944 diff changeset	860 }
49aebb3028f7 Added support for updating uidlist record extensions. Timo Sirainen <tss@iki.fi> parents: 5944 diff changeset	861 buffer_append_c(buf, key);
49aebb3028f7 Added support for updating uidlist record extensions. Timo Sirainen <tss@iki.fi> parents: 5944 diff changeset	862 buffer_append(buf, value, strlen(value) + 1);
49aebb3028f7 Added support for updating uidlist record extensions. Timo Sirainen <tss@iki.fi> parents: 5944 diff changeset	863 buffer_append_c(buf, '\0');
49aebb3028f7 Added support for updating uidlist record extensions. Timo Sirainen <tss@iki.fi> parents: 5944 diff changeset	864
49aebb3028f7 Added support for updating uidlist record extensions. Timo Sirainen <tss@iki.fi> parents: 5944 diff changeset	865 rec->extensions = p_malloc(uidlist->record_pool, buf->used);
49aebb3028f7 Added support for updating uidlist record extensions. Timo Sirainen <tss@iki.fi> parents: 5944 diff changeset	866 memcpy(rec->extensions, buf->data, buf->used);
49aebb3028f7 Added support for updating uidlist record extensions. Timo Sirainen <tss@iki.fi> parents: 5944 diff changeset	867
49aebb3028f7 Added support for updating uidlist record extensions. Timo Sirainen <tss@iki.fi> parents: 5944 diff changeset	868 uidlist->recreate = TRUE;
49aebb3028f7 Added support for updating uidlist record extensions. Timo Sirainen <tss@iki.fi> parents: 5944 diff changeset	869 t_pop();
49aebb3028f7 Added support for updating uidlist record extensions. Timo Sirainen <tss@iki.fi> parents: 5944 diff changeset	870 }
49aebb3028f7 Added support for updating uidlist record extensions. Timo Sirainen <tss@iki.fi> parents: 5944 diff changeset	871
5933 fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	872 static int maildir_uidlist_write_fd(struct maildir_uidlist *uidlist, int fd,
fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	873 const char *path, unsigned int first_idx,
fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	874 uoff_t *file_size_r)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	875 {
6599 a6f51026b969 Added maildir_uidlist_init_readonly() and maildir_uidlist_get_uid(). Timo Sirainen <tss@iki.fi> parents: 6598 diff changeset	876 struct mail_storage *storage = uidlist->ibox->box.storage;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	877 struct maildir_uidlist_iter_ctx *iter;
5917 525b449313a7 Don't use uidlist lock to update uidlist. Have a separate real dotlock for Timo Sirainen <tss@iki.fi> parents: 5915 diff changeset	878 struct ostream *output;
5944 760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	879 struct maildir_uidlist_rec *rec;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	880 string_t *str;
5944 760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	881 const char *p;
5917 525b449313a7 Don't use uidlist lock to update uidlist. Have a separate real dotlock for Timo Sirainen <tss@iki.fi> parents: 5915 diff changeset	882 int ret;
4152 e2edd333c473 Added MAILBOX_OPEN_KEEP_LOCKED flag to mailbox opening and implemented it Timo Sirainen <tss@iki.fi> parents: 4072 diff changeset	883
5933 fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	884 i_assert(fd != -1);
fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	885
6161 c62f7ee79446 Split o_stream_create_file() to _create_fd() and _create_fd_file(). Timo Sirainen <tss@iki.fi> parents: 6142 diff changeset	886 output = o_stream_create_fd_file(fd, (uoff_t)-1, FALSE);
5933 fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	887 str = t_str_new(512);
2051 596267d8e2e2 Some more UIDVALIDITY issues fixed. Timo Sirainen <tss@iki.fi> parents: 2050 diff changeset	888
5933 fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	889 if (output->offset == 0) {
fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	890 i_assert(first_idx == 0);
5944 760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	891 uidlist->version = 3;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	892
6344 e45ba2950722 Added asserts against uid_validity==0. Timo Sirainen <tss@iki.fi> parents: 6162 diff changeset	893 i_assert(uidlist->uid_validity != 0);
e45ba2950722 Added asserts against uid_validity==0. Timo Sirainen <tss@iki.fi> parents: 6162 diff changeset	894 i_assert(uidlist->next_uid > 0);
5944 760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	895 str_printfa(str, "%u V%u N%u", uidlist->version,
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	896 uidlist->uid_validity, uidlist->next_uid);
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	897 if (str_len(uidlist->hdr_extensions) > 0) {
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	898 str_append_c(str, ' ');
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	899 str_append_str(str, uidlist->hdr_extensions);
5933 fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	900 }
5944 760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	901 str_append_c(str, '\n');
5933 fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	902 o_stream_send(output, str_data(str), str_len(str));
3472 db29cc6754d5 Store new/ directory's timestamp in sync_size header in index (kludgy..). Timo Sirainen <tss@iki.fi> parents: 3447 diff changeset	903 }
5917 525b449313a7 Don't use uidlist lock to update uidlist. Have a separate real dotlock for Timo Sirainen <tss@iki.fi> parents: 5915 diff changeset	904
6599 a6f51026b969 Added maildir_uidlist_init_readonly() and maildir_uidlist_get_uid(). Timo Sirainen <tss@iki.fi> parents: 6598 diff changeset	905 iter = maildir_uidlist_iter_init(uidlist);
5933 fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	906 iter->next += first_idx;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	907
5944 760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	908 while (maildir_uidlist_iter_next_rec(iter, &rec)) {
5917 525b449313a7 Don't use uidlist lock to update uidlist. Have a separate real dotlock for Timo Sirainen <tss@iki.fi> parents: 5915 diff changeset	909 str_truncate(str, 0);
5944 760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	910 str_printfa(str, "%u", rec->uid);
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	911 if (rec->extensions != NULL) {
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	912 for (p = rec->extensions; *p != '\0'; ) {
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	913 str_append_c(str, ' ');
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	914 str_append(str, p);
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	915 p += strlen(p) + 1;
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	916 }
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	917 }
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	918 str_printfa(str, " :%s\n", rec->filename);
5917 525b449313a7 Don't use uidlist lock to update uidlist. Have a separate real dotlock for Timo Sirainen <tss@iki.fi> parents: 5915 diff changeset	919 o_stream_send(output, str_data(str), str_len(str));
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	920 }
6409 7f733ba453bf Changed _deinit() APIs to take * pointer and set it to NULL. Timo Sirainen <tss@iki.fi> parents: 6344 diff changeset	921 maildir_uidlist_iter_deinit(&iter);
5933 fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	922 o_stream_flush(output);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	923
5917 525b449313a7 Don't use uidlist lock to update uidlist. Have a separate real dotlock for Timo Sirainen <tss@iki.fi> parents: 5915 diff changeset	924 ret = output->stream_errno == 0 ? 0 : -1;
5933 fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	925
fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	926 *file_size_r = output->offset;
5917 525b449313a7 Don't use uidlist lock to update uidlist. Have a separate real dotlock for Timo Sirainen <tss@iki.fi> parents: 5915 diff changeset	927 o_stream_unref(&output);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	928
5917 525b449313a7 Don't use uidlist lock to update uidlist. Have a separate real dotlock for Timo Sirainen <tss@iki.fi> parents: 5915 diff changeset	929 if (ret < 0) {
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	930 mail_storage_set_critical(storage,
5917 525b449313a7 Don't use uidlist lock to update uidlist. Have a separate real dotlock for Timo Sirainen <tss@iki.fi> parents: 5915 diff changeset	931 "o_stream_send(%s) failed: %m", path);
525b449313a7 Don't use uidlist lock to update uidlist. Have a separate real dotlock for Timo Sirainen <tss@iki.fi> parents: 5915 diff changeset	932 (void)close(fd);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	933 return -1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	934 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	935
6599 a6f51026b969 Added maildir_uidlist_init_readonly() and maildir_uidlist_get_uid(). Timo Sirainen <tss@iki.fi> parents: 6598 diff changeset	936 if (!uidlist->ibox->fsync_disable) {
5917 525b449313a7 Don't use uidlist lock to update uidlist. Have a separate real dotlock for Timo Sirainen <tss@iki.fi> parents: 5915 diff changeset	937 if (fdatasync(fd) < 0) {
5064 db3f662588a4 Made one more fsync() optional with fsync_disable. Timo Sirainen <tss@iki.fi> parents: 5033 diff changeset	938 mail_storage_set_critical(storage,
6738 c3e6d7d96f1a Error message fix: s/fsync/fdatasync/ Timo Sirainen <tss@iki.fi> parents: 6657 diff changeset	939 "fdatasync(%s) failed: %m", path);
5917 525b449313a7 Don't use uidlist lock to update uidlist. Have a separate real dotlock for Timo Sirainen <tss@iki.fi> parents: 5915 diff changeset	940 (void)close(fd);
5064 db3f662588a4 Made one more fsync() optional with fsync_disable. Timo Sirainen <tss@iki.fi> parents: 5033 diff changeset	941 return -1;
db3f662588a4 Made one more fsync() optional with fsync_disable. Timo Sirainen <tss@iki.fi> parents: 5033 diff changeset	942 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	943 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	944 return 0;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	945 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	946
5919 cc439e4e99cb cleanup Timo Sirainen <tss@iki.fi> parents: 5918 diff changeset	947 static int maildir_uidlist_recreate(struct maildir_uidlist *uidlist)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	948 {
6599 a6f51026b969 Added maildir_uidlist_init_readonly() and maildir_uidlist_get_uid(). Timo Sirainen <tss@iki.fi> parents: 6598 diff changeset	949 struct mailbox *box = &uidlist->ibox->box;
a6f51026b969 Added maildir_uidlist_init_readonly() and maildir_uidlist_get_uid(). Timo Sirainen <tss@iki.fi> parents: 6598 diff changeset	950 const char control_dir, temp_path;
5891 db959f0d94e3 Leave uidlist fd open so we can check if uidlist has been rewritten from Timo Sirainen <tss@iki.fi> parents: 5890 diff changeset	951 struct stat st;
5917 525b449313a7 Don't use uidlist lock to update uidlist. Have a separate real dotlock for Timo Sirainen <tss@iki.fi> parents: 5915 diff changeset	952 mode_t old_mask;
5933 fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	953 uoff_t file_size;
6777 186b164a9579 Delay creating directories until we really need them. Timo Sirainen <tss@iki.fi> parents: 6738 diff changeset	954 int i, fd, ret;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	955
6599 a6f51026b969 Added maildir_uidlist_init_readonly() and maildir_uidlist_get_uid(). Timo Sirainen <tss@iki.fi> parents: 6598 diff changeset	956 control_dir = mailbox_list_get_path(box->storage->list, box->name,
a6f51026b969 Added maildir_uidlist_init_readonly() and maildir_uidlist_get_uid(). Timo Sirainen <tss@iki.fi> parents: 6598 diff changeset	957 MAILBOX_LIST_PATH_TYPE_CONTROL);
a6f51026b969 Added maildir_uidlist_init_readonly() and maildir_uidlist_get_uid(). Timo Sirainen <tss@iki.fi> parents: 6598 diff changeset	958 temp_path = t_strconcat(control_dir,
5917 525b449313a7 Don't use uidlist lock to update uidlist. Have a separate real dotlock for Timo Sirainen <tss@iki.fi> parents: 5915 diff changeset	959 "/" MAILDIR_UIDLIST_NAME ".tmp", NULL);
525b449313a7 Don't use uidlist lock to update uidlist. Have a separate real dotlock for Timo Sirainen <tss@iki.fi> parents: 5915 diff changeset	960
6777 186b164a9579 Delay creating directories until we really need them. Timo Sirainen <tss@iki.fi> parents: 6738 diff changeset	961 for (i = 0;; i++) {
186b164a9579 Delay creating directories until we really need them. Timo Sirainen <tss@iki.fi> parents: 6738 diff changeset	962 old_mask = umask(0777 & ~box->file_create_mode);
186b164a9579 Delay creating directories until we really need them. Timo Sirainen <tss@iki.fi> parents: 6738 diff changeset	963 fd = open(temp_path, O_RDWR \| O_CREAT \| O_TRUNC, 0777);
186b164a9579 Delay creating directories until we really need them. Timo Sirainen <tss@iki.fi> parents: 6738 diff changeset	964 umask(old_mask);
186b164a9579 Delay creating directories until we really need them. Timo Sirainen <tss@iki.fi> parents: 6738 diff changeset	965 if (fd != -1)
186b164a9579 Delay creating directories until we really need them. Timo Sirainen <tss@iki.fi> parents: 6738 diff changeset	966 break;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	967
6777 186b164a9579 Delay creating directories until we really need them. Timo Sirainen <tss@iki.fi> parents: 6738 diff changeset	968 if (errno != ENOENT \|\| i == MAILDIR_DELETE_RETRY_COUNT \|\|
186b164a9579 Delay creating directories until we really need them. Timo Sirainen <tss@iki.fi> parents: 6738 diff changeset	969 uidlist->mbox == NULL) {
186b164a9579 Delay creating directories until we really need them. Timo Sirainen <tss@iki.fi> parents: 6738 diff changeset	970 mail_storage_set_critical(box->storage,
186b164a9579 Delay creating directories until we really need them. Timo Sirainen <tss@iki.fi> parents: 6738 diff changeset	971 "open(%s, O_CREAT) failed: %m", temp_path);
186b164a9579 Delay creating directories until we really need them. Timo Sirainen <tss@iki.fi> parents: 6738 diff changeset	972 return -1;
186b164a9579 Delay creating directories until we really need them. Timo Sirainen <tss@iki.fi> parents: 6738 diff changeset	973 }
186b164a9579 Delay creating directories until we really need them. Timo Sirainen <tss@iki.fi> parents: 6738 diff changeset	974 /* the control dir doesn't exist. create it unless the whole
186b164a9579 Delay creating directories until we really need them. Timo Sirainen <tss@iki.fi> parents: 6738 diff changeset	975 mailbox was just deleted. */
186b164a9579 Delay creating directories until we really need them. Timo Sirainen <tss@iki.fi> parents: 6738 diff changeset	976 if (maildir_set_deleted(uidlist->mbox))
186b164a9579 Delay creating directories until we really need them. Timo Sirainen <tss@iki.fi> parents: 6738 diff changeset	977 return -1;
5917 525b449313a7 Don't use uidlist lock to update uidlist. Have a separate real dotlock for Timo Sirainen <tss@iki.fi> parents: 5915 diff changeset	978 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	979
6599 a6f51026b969 Added maildir_uidlist_init_readonly() and maildir_uidlist_get_uid(). Timo Sirainen <tss@iki.fi> parents: 6598 diff changeset	980 if (box->file_create_gid != (gid_t)-1) {
a6f51026b969 Added maildir_uidlist_init_readonly() and maildir_uidlist_get_uid(). Timo Sirainen <tss@iki.fi> parents: 6598 diff changeset	981 if (fchown(fd, (uid_t)-1, box->file_create_gid) < 0) {
a6f51026b969 Added maildir_uidlist_init_readonly() and maildir_uidlist_get_uid(). Timo Sirainen <tss@iki.fi> parents: 6598 diff changeset	982 mail_storage_set_critical(box->storage,
5917 525b449313a7 Don't use uidlist lock to update uidlist. Have a separate real dotlock for Timo Sirainen <tss@iki.fi> parents: 5915 diff changeset	983 "fchown(%s) failed: %m", temp_path);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	984 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	985 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	986
5933 fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	987 ret = maildir_uidlist_write_fd(uidlist, fd, temp_path, 0, &file_size);
5917 525b449313a7 Don't use uidlist lock to update uidlist. Have a separate real dotlock for Timo Sirainen <tss@iki.fi> parents: 5915 diff changeset	988 if (ret == 0) {
525b449313a7 Don't use uidlist lock to update uidlist. Have a separate real dotlock for Timo Sirainen <tss@iki.fi> parents: 5915 diff changeset	989 if (rename(temp_path, uidlist->path) < 0) {
6599 a6f51026b969 Added maildir_uidlist_init_readonly() and maildir_uidlist_get_uid(). Timo Sirainen <tss@iki.fi> parents: 6598 diff changeset	990 mail_storage_set_critical(box->storage,
5917 525b449313a7 Don't use uidlist lock to update uidlist. Have a separate real dotlock for Timo Sirainen <tss@iki.fi> parents: 5915 diff changeset	991 "rename(%s, %s) failed: %m",
525b449313a7 Don't use uidlist lock to update uidlist. Have a separate real dotlock for Timo Sirainen <tss@iki.fi> parents: 5915 diff changeset	992 temp_path, uidlist->path);
525b449313a7 Don't use uidlist lock to update uidlist. Have a separate real dotlock for Timo Sirainen <tss@iki.fi> parents: 5915 diff changeset	993 ret = -1;
525b449313a7 Don't use uidlist lock to update uidlist. Have a separate real dotlock for Timo Sirainen <tss@iki.fi> parents: 5915 diff changeset	994 }
525b449313a7 Don't use uidlist lock to update uidlist. Have a separate real dotlock for Timo Sirainen <tss@iki.fi> parents: 5915 diff changeset	995 }
525b449313a7 Don't use uidlist lock to update uidlist. Have a separate real dotlock for Timo Sirainen <tss@iki.fi> parents: 5915 diff changeset	996
525b449313a7 Don't use uidlist lock to update uidlist. Have a separate real dotlock for Timo Sirainen <tss@iki.fi> parents: 5915 diff changeset	997 if (ret < 0) {
525b449313a7 Don't use uidlist lock to update uidlist. Have a separate real dotlock for Timo Sirainen <tss@iki.fi> parents: 5915 diff changeset	998 if (unlink(temp_path) < 0) {
6599 a6f51026b969 Added maildir_uidlist_init_readonly() and maildir_uidlist_get_uid(). Timo Sirainen <tss@iki.fi> parents: 6598 diff changeset	999 mail_storage_set_critical(box->storage,
5917 525b449313a7 Don't use uidlist lock to update uidlist. Have a separate real dotlock for Timo Sirainen <tss@iki.fi> parents: 5915 diff changeset	1000 "unlink(%s) failed: %m", temp_path);
525b449313a7 Don't use uidlist lock to update uidlist. Have a separate real dotlock for Timo Sirainen <tss@iki.fi> parents: 5915 diff changeset	1001 }
525b449313a7 Don't use uidlist lock to update uidlist. Have a separate real dotlock for Timo Sirainen <tss@iki.fi> parents: 5915 diff changeset	1002 } else if (fstat(fd, &st) < 0) {
525b449313a7 Don't use uidlist lock to update uidlist. Have a separate real dotlock for Timo Sirainen <tss@iki.fi> parents: 5915 diff changeset	1003 i_error("fstat(%s) failed: %m", temp_path);
525b449313a7 Don't use uidlist lock to update uidlist. Have a separate real dotlock for Timo Sirainen <tss@iki.fi> parents: 5915 diff changeset	1004 (void)close(fd);
525b449313a7 Don't use uidlist lock to update uidlist. Have a separate real dotlock for Timo Sirainen <tss@iki.fi> parents: 5915 diff changeset	1005 ret = -1;
525b449313a7 Don't use uidlist lock to update uidlist. Have a separate real dotlock for Timo Sirainen <tss@iki.fi> parents: 5915 diff changeset	1006 } else {
5933 fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	1007 i_assert(file_size == (uoff_t)st.st_size);
5956 fc75386a57ab fd leak fix Timo Sirainen <tss@iki.fi> parents: 5950 diff changeset	1008 maildir_uidlist_close(uidlist);
5917 525b449313a7 Don't use uidlist lock to update uidlist. Have a separate real dotlock for Timo Sirainen <tss@iki.fi> parents: 5915 diff changeset	1009 uidlist->fd = fd;
525b449313a7 Don't use uidlist lock to update uidlist. Have a separate real dotlock for Timo Sirainen <tss@iki.fi> parents: 5915 diff changeset	1010 uidlist->fd_dev = st.st_dev;
525b449313a7 Don't use uidlist lock to update uidlist. Have a separate real dotlock for Timo Sirainen <tss@iki.fi> parents: 5915 diff changeset	1011 uidlist->fd_ino = st.st_ino;
5933 fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	1012 uidlist->fd_size = st.st_size;
fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	1013 uidlist->last_read_offset = st.st_size;
6034 dcaa8d5296a9 Fixes to handling appends and unexpunges. Timo Sirainen <tss@iki.fi> parents: 6029 diff changeset	1014 uidlist->recreate = FALSE;
5917 525b449313a7 Don't use uidlist lock to update uidlist. Have a separate real dotlock for Timo Sirainen <tss@iki.fi> parents: 5915 diff changeset	1015 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1016 return ret;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1017 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1018
5950 49aebb3028f7 Added support for updating uidlist record extensions. Timo Sirainen <tss@iki.fi> parents: 5944 diff changeset	1019 int maildir_uidlist_update(struct maildir_uidlist *uidlist)
49aebb3028f7 Added support for updating uidlist record extensions. Timo Sirainen <tss@iki.fi> parents: 5944 diff changeset	1020 {
49aebb3028f7 Added support for updating uidlist record extensions. Timo Sirainen <tss@iki.fi> parents: 5944 diff changeset	1021 int ret;
49aebb3028f7 Added support for updating uidlist record extensions. Timo Sirainen <tss@iki.fi> parents: 5944 diff changeset	1022
49aebb3028f7 Added support for updating uidlist record extensions. Timo Sirainen <tss@iki.fi> parents: 5944 diff changeset	1023 if (!uidlist->recreate)
49aebb3028f7 Added support for updating uidlist record extensions. Timo Sirainen <tss@iki.fi> parents: 5944 diff changeset	1024 return 0;
49aebb3028f7 Added support for updating uidlist record extensions. Timo Sirainen <tss@iki.fi> parents: 5944 diff changeset	1025
49aebb3028f7 Added support for updating uidlist record extensions. Timo Sirainen <tss@iki.fi> parents: 5944 diff changeset	1026 if (maildir_uidlist_lock(uidlist) <= 0)
49aebb3028f7 Added support for updating uidlist record extensions. Timo Sirainen <tss@iki.fi> parents: 5944 diff changeset	1027 return -1;
49aebb3028f7 Added support for updating uidlist record extensions. Timo Sirainen <tss@iki.fi> parents: 5944 diff changeset	1028 ret = maildir_uidlist_recreate(uidlist);
49aebb3028f7 Added support for updating uidlist record extensions. Timo Sirainen <tss@iki.fi> parents: 5944 diff changeset	1029 maildir_uidlist_unlock(uidlist);
49aebb3028f7 Added support for updating uidlist record extensions. Timo Sirainen <tss@iki.fi> parents: 5944 diff changeset	1030 return ret;
49aebb3028f7 Added support for updating uidlist record extensions. Timo Sirainen <tss@iki.fi> parents: 5944 diff changeset	1031 }
49aebb3028f7 Added support for updating uidlist record extensions. Timo Sirainen <tss@iki.fi> parents: 5944 diff changeset	1032
49aebb3028f7 Added support for updating uidlist record extensions. Timo Sirainen <tss@iki.fi> parents: 5944 diff changeset	1033 static int maildir_uidlist_sync_update(struct maildir_uidlist_sync_ctx *ctx)
5933 fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	1034 {
fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	1035 struct maildir_uidlist *uidlist = ctx->uidlist;
fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	1036 uoff_t file_size;
fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	1037
5959 5513182aed67 One more UIDVALIDITY=0 fix Timo Sirainen <tss@iki.fi> parents: 5958 diff changeset	1038 if (uidlist->uid_validity == 0) {
5513182aed67 One more UIDVALIDITY=0 fix Timo Sirainen <tss@iki.fi> parents: 5958 diff changeset	1039 /* saving a message to a newly created maildir. */
5513182aed67 One more UIDVALIDITY=0 fix Timo Sirainen <tss@iki.fi> parents: 5958 diff changeset	1040 const struct mail_index_header *hdr;
5513182aed67 One more UIDVALIDITY=0 fix Timo Sirainen <tss@iki.fi> parents: 5958 diff changeset	1041
6599 a6f51026b969 Added maildir_uidlist_init_readonly() and maildir_uidlist_get_uid(). Timo Sirainen <tss@iki.fi> parents: 6598 diff changeset	1042 hdr = mail_index_get_header(uidlist->ibox->view);
5959 5513182aed67 One more UIDVALIDITY=0 fix Timo Sirainen <tss@iki.fi> parents: 5958 diff changeset	1043 uidlist->uid_validity = hdr->uid_validity != 0 ?
5983 272796cfee66 compiler warning fixes Timo Sirainen <tss@iki.fi> parents: 5974 diff changeset	1044 hdr->uid_validity : (uint32_t)ioloop_time;
5959 5513182aed67 One more UIDVALIDITY=0 fix Timo Sirainen <tss@iki.fi> parents: 5958 diff changeset	1045 }
5513182aed67 One more UIDVALIDITY=0 fix Timo Sirainen <tss@iki.fi> parents: 5958 diff changeset	1046
5933 fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	1047 if (ctx->uidlist->recreate \|\| uidlist->fd == -1 \|\|
5957 874428d05272 If uidlist isn't version 3, we have to recreate it instead of appending. Timo Sirainen <tss@iki.fi> parents: 5956 diff changeset	1048 uidlist->version != 3 \|\|
5933 fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	1049 (uidlist->read_records_count + ctx->new_files_count) *
fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	1050 UIDLIST_COMPRESS_PERCENTAGE / 100 >= array_count(&uidlist->records))
fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	1051 return maildir_uidlist_recreate(uidlist);
fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	1052
6034 dcaa8d5296a9 Fixes to handling appends and unexpunges. Timo Sirainen <tss@iki.fi> parents: 6029 diff changeset	1053 i_assert(ctx->first_unwritten_pos != (unsigned int)-1);
5933 fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	1054
fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	1055 if (lseek(uidlist->fd, 0, SEEK_END) < 0) {
6599 a6f51026b969 Added maildir_uidlist_init_readonly() and maildir_uidlist_get_uid(). Timo Sirainen <tss@iki.fi> parents: 6598 diff changeset	1056 mail_storage_set_critical(uidlist->ibox->box.storage,
5933 fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	1057 "lseek(%s) failed: %m", uidlist->path);
fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	1058 return -1;
fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	1059 }
fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	1060
fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	1061 if (maildir_uidlist_write_fd(uidlist, uidlist->fd, uidlist->path,
6034 dcaa8d5296a9 Fixes to handling appends and unexpunges. Timo Sirainen <tss@iki.fi> parents: 6029 diff changeset	1062 ctx->first_unwritten_pos, &file_size) < 0)
5933 fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	1063 return -1;
fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	1064
fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	1065 uidlist->last_read_offset = file_size;
fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	1066 return 0;
fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	1067 }
fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	1068
1954 2f6e137cdc44 Syncing optimizations. Timo Sirainen <tss@iki.fi> parents: 1947 diff changeset	1069 static void maildir_uidlist_mark_all(struct maildir_uidlist *uidlist,
3863 55df57c028d4 Added "bool" type and changed all ints that were used as booleans to bool. Timo Sirainen <tss@iki.fi> parents: 3534 diff changeset	1070 bool nonsynced)
1954 2f6e137cdc44 Syncing optimizations. Timo Sirainen <tss@iki.fi> parents: 1947 diff changeset	1071 {
5890 989abd5017a4 Changed record buffers to arrays. Timo Sirainen <tss@iki.fi> parents: 5740 diff changeset	1072 struct maildir_uidlist_rec **recs;
989abd5017a4 Changed record buffers to arrays. Timo Sirainen <tss@iki.fi> parents: 5740 diff changeset	1073 unsigned int i, count;
1954 2f6e137cdc44 Syncing optimizations. Timo Sirainen <tss@iki.fi> parents: 1947 diff changeset	1074
5890 989abd5017a4 Changed record buffers to arrays. Timo Sirainen <tss@iki.fi> parents: 5740 diff changeset	1075 recs = array_get_modifiable(&uidlist->records, &count);
1954 2f6e137cdc44 Syncing optimizations. Timo Sirainen <tss@iki.fi> parents: 1947 diff changeset	1076 if (nonsynced) {
5890 989abd5017a4 Changed record buffers to arrays. Timo Sirainen <tss@iki.fi> parents: 5740 diff changeset	1077 for (i = 0; i < count; i++)
989abd5017a4 Changed record buffers to arrays. Timo Sirainen <tss@iki.fi> parents: 5740 diff changeset	1078 recs[i]->flags \|= MAILDIR_UIDLIST_REC_FLAG_NONSYNCED;
1954 2f6e137cdc44 Syncing optimizations. Timo Sirainen <tss@iki.fi> parents: 1947 diff changeset	1079 } else {
5890 989abd5017a4 Changed record buffers to arrays. Timo Sirainen <tss@iki.fi> parents: 5740 diff changeset	1080 for (i = 0; i < count; i++)
989abd5017a4 Changed record buffers to arrays. Timo Sirainen <tss@iki.fi> parents: 5740 diff changeset	1081 recs[i]->flags &= ~MAILDIR_UIDLIST_REC_FLAG_NONSYNCED;
1954 2f6e137cdc44 Syncing optimizations. Timo Sirainen <tss@iki.fi> parents: 1947 diff changeset	1082 }
2f6e137cdc44 Syncing optimizations. Timo Sirainen <tss@iki.fi> parents: 1947 diff changeset	1083 }
2f6e137cdc44 Syncing optimizations. Timo Sirainen <tss@iki.fi> parents: 1947 diff changeset	1084
5393 434047e53586 When saving/copying mails, we must not do any sorting based on the filename Timo Sirainen <tss@iki.fi> parents: 5388 diff changeset	1085 int maildir_uidlist_sync_init(struct maildir_uidlist *uidlist,
434047e53586 When saving/copying mails, we must not do any sorting based on the filename Timo Sirainen <tss@iki.fi> parents: 5388 diff changeset	1086 enum maildir_uidlist_sync_flags sync_flags,
3530 e9695ec7925b Recursive maildir uidlist syncs caused assert crashes. Also did some Timo Sirainen <tss@iki.fi> parents: 3472 diff changeset	1087 struct maildir_uidlist_sync_ctx **sync_ctx_r)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1088 {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1089 struct maildir_uidlist_sync_ctx *ctx;
3530 e9695ec7925b Recursive maildir uidlist syncs caused assert crashes. Also did some Timo Sirainen <tss@iki.fi> parents: 3472 diff changeset	1090 int ret;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1091
3530 e9695ec7925b Recursive maildir uidlist syncs caused assert crashes. Also did some Timo Sirainen <tss@iki.fi> parents: 3472 diff changeset	1092 if ((ret = maildir_uidlist_lock(uidlist)) <= 0)
e9695ec7925b Recursive maildir uidlist syncs caused assert crashes. Also did some Timo Sirainen <tss@iki.fi> parents: 3472 diff changeset	1093 return ret;
e9695ec7925b Recursive maildir uidlist syncs caused assert crashes. Also did some Timo Sirainen <tss@iki.fi> parents: 3472 diff changeset	1094
e9695ec7925b Recursive maildir uidlist syncs caused assert crashes. Also did some Timo Sirainen <tss@iki.fi> parents: 3472 diff changeset	1095 *sync_ctx_r = ctx = i_new(struct maildir_uidlist_sync_ctx, 1);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1096 ctx->uidlist = uidlist;
5393 434047e53586 When saving/copying mails, we must not do any sorting based on the filename Timo Sirainen <tss@iki.fi> parents: 5388 diff changeset	1097 ctx->sync_flags = sync_flags;
434047e53586 When saving/copying mails, we must not do any sorting based on the filename Timo Sirainen <tss@iki.fi> parents: 5388 diff changeset	1098 ctx->partial = (sync_flags & MAILDIR_UIDLIST_SYNC_PARTIAL) != 0;
6034 dcaa8d5296a9 Fixes to handling appends and unexpunges. Timo Sirainen <tss@iki.fi> parents: 6029 diff changeset	1099 ctx->first_unwritten_pos = (unsigned int)-1;
dcaa8d5296a9 Fixes to handling appends and unexpunges. Timo Sirainen <tss@iki.fi> parents: 6029 diff changeset	1100 ctx->first_nouid_pos = (unsigned int)-1;
1954 2f6e137cdc44 Syncing optimizations. Timo Sirainen <tss@iki.fi> parents: 1947 diff changeset	1101
5393 434047e53586 When saving/copying mails, we must not do any sorting based on the filename Timo Sirainen <tss@iki.fi> parents: 5388 diff changeset	1102 if (ctx->partial) {
6028 68be663e79ec Fixed reassigning new UIDs to more than one message. Timo Sirainen <tss@iki.fi> parents: 5983 diff changeset	1103 if ((sync_flags & MAILDIR_UIDLIST_SYNC_KEEP_STATE) == 0) {
68be663e79ec Fixed reassigning new UIDs to more than one message. Timo Sirainen <tss@iki.fi> parents: 5983 diff changeset	1104 /* initially mark all nonsynced */
68be663e79ec Fixed reassigning new UIDs to more than one message. Timo Sirainen <tss@iki.fi> parents: 5983 diff changeset	1105 maildir_uidlist_mark_all(uidlist, TRUE);
68be663e79ec Fixed reassigning new UIDs to more than one message. Timo Sirainen <tss@iki.fi> parents: 5983 diff changeset	1106 }
3530 e9695ec7925b Recursive maildir uidlist syncs caused assert crashes. Also did some Timo Sirainen <tss@iki.fi> parents: 3472 diff changeset	1107 return 1;
1954 2f6e137cdc44 Syncing optimizations. Timo Sirainen <tss@iki.fi> parents: 1947 diff changeset	1108 }
2f6e137cdc44 Syncing optimizations. Timo Sirainen <tss@iki.fi> parents: 1947 diff changeset	1109
4980 8e88ecc64563 debug: MEMPOOL_GROWING prefix in alloconly pool names means that when Timo Sirainen <tss@iki.fi> parents: 4969 diff changeset	1110 ctx->record_pool = pool_alloconly_create(MEMPOOL_GROWING
8e88ecc64563 debug: MEMPOOL_GROWING prefix in alloconly pool names means that when Timo Sirainen <tss@iki.fi> parents: 4969 diff changeset	1111 "maildir_uidlist_sync", 16384);
1962 4775ffa73874 Syncing should be fixed now. Timo Sirainen <tss@iki.fi> parents: 1960 diff changeset	1112 ctx->files = hash_create(default_pool, ctx->record_pool, 4096,
5902 0cef1d7a0ce3 Renamed maildir_cmp/hash to maildir_filename_base_cmp/hash. Timo Sirainen <tss@iki.fi> parents: 5899 diff changeset	1113 maildir_filename_base_hash,
0cef1d7a0ce3 Renamed maildir_cmp/hash to maildir_filename_base_cmp/hash. Timo Sirainen <tss@iki.fi> parents: 5899 diff changeset	1114 maildir_filename_base_cmp);
1962 4775ffa73874 Syncing should be fixed now. Timo Sirainen <tss@iki.fi> parents: 1960 diff changeset	1115
5890 989abd5017a4 Changed record buffers to arrays. Timo Sirainen <tss@iki.fi> parents: 5740 diff changeset	1116 i_array_init(&ctx->records, array_count(&uidlist->records));
3530 e9695ec7925b Recursive maildir uidlist syncs caused assert crashes. Also did some Timo Sirainen <tss@iki.fi> parents: 3472 diff changeset	1117 return 1;
1954 2f6e137cdc44 Syncing optimizations. Timo Sirainen <tss@iki.fi> parents: 1947 diff changeset	1118 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1119
5921 00bdb1f546d3 minor cleanup Timo Sirainen <tss@iki.fi> parents: 5919 diff changeset	1120 static void
1954 2f6e137cdc44 Syncing optimizations. Timo Sirainen <tss@iki.fi> parents: 1947 diff changeset	1121 maildir_uidlist_sync_next_partial(struct maildir_uidlist_sync_ctx *ctx,
2f6e137cdc44 Syncing optimizations. Timo Sirainen <tss@iki.fi> parents: 1947 diff changeset	1122 const char *filename,
2f6e137cdc44 Syncing optimizations. Timo Sirainen <tss@iki.fi> parents: 1947 diff changeset	1123 enum maildir_uidlist_rec_flag flags)
2f6e137cdc44 Syncing optimizations. Timo Sirainen <tss@iki.fi> parents: 1947 diff changeset	1124 {
2f6e137cdc44 Syncing optimizations. Timo Sirainen <tss@iki.fi> parents: 1947 diff changeset	1125 struct maildir_uidlist *uidlist = ctx->uidlist;
2f6e137cdc44 Syncing optimizations. Timo Sirainen <tss@iki.fi> parents: 1947 diff changeset	1126 struct maildir_uidlist_rec *rec;
2f6e137cdc44 Syncing optimizations. Timo Sirainen <tss@iki.fi> parents: 1947 diff changeset	1127
2f6e137cdc44 Syncing optimizations. Timo Sirainen <tss@iki.fi> parents: 1947 diff changeset	1128 /* we'll update uidlist directly */
1962 4775ffa73874 Syncing should be fixed now. Timo Sirainen <tss@iki.fi> parents: 1960 diff changeset	1129 rec = hash_lookup(uidlist->files, filename);
2123 e01de478882f locking fixes Timo Sirainen <tss@iki.fi> parents: 2121 diff changeset	1130 i_assert(rec != NULL \|\| UIDLIST_IS_LOCKED(uidlist));
1954 2f6e137cdc44 Syncing optimizations. Timo Sirainen <tss@iki.fi> parents: 1947 diff changeset	1131
1962 4775ffa73874 Syncing should be fixed now. Timo Sirainen <tss@iki.fi> parents: 1960 diff changeset	1132 if (rec == NULL) {
6800 ca849e28a062 NFS: Delay flushing dovecot-uidlist's attribute cache until we find an Timo Sirainen <tss@iki.fi> parents: 6798 diff changeset	1133 /* doesn't exist in uidlist */
ca849e28a062 NFS: Delay flushing dovecot-uidlist's attribute cache until we find an Timo Sirainen <tss@iki.fi> parents: 6798 diff changeset	1134 if (uidlist->nfs_dirty_refresh) {
ca849e28a062 NFS: Delay flushing dovecot-uidlist's attribute cache until we find an Timo Sirainen <tss@iki.fi> parents: 6798 diff changeset	1135 /* We haven't flushed NFS attribute cache yet, do that
ca849e28a062 NFS: Delay flushing dovecot-uidlist's attribute cache until we find an Timo Sirainen <tss@iki.fi> parents: 6798 diff changeset	1136 to make sure the file doesn't exist */
ca849e28a062 NFS: Delay flushing dovecot-uidlist's attribute cache until we find an Timo Sirainen <tss@iki.fi> parents: 6798 diff changeset	1137 if (maildir_uidlist_refresh(uidlist, TRUE) < 0) {
ca849e28a062 NFS: Delay flushing dovecot-uidlist's attribute cache until we find an Timo Sirainen <tss@iki.fi> parents: 6798 diff changeset	1138 ctx->failed = TRUE;
ca849e28a062 NFS: Delay flushing dovecot-uidlist's attribute cache until we find an Timo Sirainen <tss@iki.fi> parents: 6798 diff changeset	1139 return;
ca849e28a062 NFS: Delay flushing dovecot-uidlist's attribute cache until we find an Timo Sirainen <tss@iki.fi> parents: 6798 diff changeset	1140 }
ca849e28a062 NFS: Delay flushing dovecot-uidlist's attribute cache until we find an Timo Sirainen <tss@iki.fi> parents: 6798 diff changeset	1141 i_assert(!uidlist->nfs_dirty_refresh);
ca849e28a062 NFS: Delay flushing dovecot-uidlist's attribute cache until we find an Timo Sirainen <tss@iki.fi> parents: 6798 diff changeset	1142 return maildir_uidlist_sync_next_partial(ctx, filename,
ca849e28a062 NFS: Delay flushing dovecot-uidlist's attribute cache until we find an Timo Sirainen <tss@iki.fi> parents: 6798 diff changeset	1143 flags);
ca849e28a062 NFS: Delay flushing dovecot-uidlist's attribute cache until we find an Timo Sirainen <tss@iki.fi> parents: 6798 diff changeset	1144 }
6034 dcaa8d5296a9 Fixes to handling appends and unexpunges. Timo Sirainen <tss@iki.fi> parents: 6029 diff changeset	1145 if (ctx->first_nouid_pos == (unsigned int)-1)
dcaa8d5296a9 Fixes to handling appends and unexpunges. Timo Sirainen <tss@iki.fi> parents: 6029 diff changeset	1146 ctx->first_nouid_pos = array_count(&uidlist->records);
1973 f6d2dba3b78b and yet more syncing fixes.. Timo Sirainen <tss@iki.fi> parents: 1971 diff changeset	1147 ctx->new_files_count++;
5923 239078f515a8 Added maildir_uidlist_sync_remove() Timo Sirainen <tss@iki.fi> parents: 5922 diff changeset	1148 ctx->changed = TRUE;
1960 b9ea4db3f8e0 crashfix Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	1149
2051 596267d8e2e2 Some more UIDVALIDITY issues fixed. Timo Sirainen <tss@iki.fi> parents: 2050 diff changeset	1150 if (uidlist->record_pool == NULL) {
596267d8e2e2 Some more UIDVALIDITY issues fixed. Timo Sirainen <tss@iki.fi> parents: 2050 diff changeset	1151 uidlist->record_pool =
4980 8e88ecc64563 debug: MEMPOOL_GROWING prefix in alloconly pool names means that when Timo Sirainen <tss@iki.fi> parents: 4969 diff changeset	1152 pool_alloconly_create(MEMPOOL_GROWING
8e88ecc64563 debug: MEMPOOL_GROWING prefix in alloconly pool names means that when Timo Sirainen <tss@iki.fi> parents: 4969 diff changeset	1153 "uidlist record_pool",
2051 596267d8e2e2 Some more UIDVALIDITY issues fixed. Timo Sirainen <tss@iki.fi> parents: 2050 diff changeset	1154 1024);
596267d8e2e2 Some more UIDVALIDITY issues fixed. Timo Sirainen <tss@iki.fi> parents: 2050 diff changeset	1155 }
596267d8e2e2 Some more UIDVALIDITY issues fixed. Timo Sirainen <tss@iki.fi> parents: 2050 diff changeset	1156
1962 4775ffa73874 Syncing should be fixed now. Timo Sirainen <tss@iki.fi> parents: 1960 diff changeset	1157 rec = p_new(uidlist->record_pool,
4775ffa73874 Syncing should be fixed now. Timo Sirainen <tss@iki.fi> parents: 1960 diff changeset	1158 struct maildir_uidlist_rec, 1);
1973 f6d2dba3b78b and yet more syncing fixes.. Timo Sirainen <tss@iki.fi> parents: 1971 diff changeset	1159 rec->uid = (uint32_t)-1;
5890 989abd5017a4 Changed record buffers to arrays. Timo Sirainen <tss@iki.fi> parents: 5740 diff changeset	1160 array_append(&uidlist->records, &rec, 1);
5913 699356ffe549 If uidlist changes while it's being iterated, don't crash. Timo Sirainen <tss@iki.fi> parents: 5903 diff changeset	1161 uidlist->change_counter++;
1954 2f6e137cdc44 Syncing optimizations. Timo Sirainen <tss@iki.fi> parents: 1947 diff changeset	1162 }
2f6e137cdc44 Syncing optimizations. Timo Sirainen <tss@iki.fi> parents: 1947 diff changeset	1163
2f6e137cdc44 Syncing optimizations. Timo Sirainen <tss@iki.fi> parents: 1947 diff changeset	1164 rec->flags = (rec->flags \| flags) & ~MAILDIR_UIDLIST_REC_FLAG_NONSYNCED;
1962 4775ffa73874 Syncing should be fixed now. Timo Sirainen <tss@iki.fi> parents: 1960 diff changeset	1165 rec->filename = p_strdup(uidlist->record_pool, filename);
4775ffa73874 Syncing should be fixed now. Timo Sirainen <tss@iki.fi> parents: 1960 diff changeset	1166 hash_insert(uidlist->files, rec->filename, rec);
6029 10042803d65e Crashfixes to handling new messages. Timo Sirainen <tss@iki.fi> parents: 6028 diff changeset	1167
10042803d65e Crashfixes to handling new messages. Timo Sirainen <tss@iki.fi> parents: 6028 diff changeset	1168 ctx->finished = FALSE;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1169 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1170
1978 6303ef092c5b mbox code compiles again, but syncing is only partially implemented so Timo Sirainen <tss@iki.fi> parents: 1975 diff changeset	1171 int maildir_uidlist_sync_next_pre(struct maildir_uidlist_sync_ctx *ctx,
6303ef092c5b mbox code compiles again, but syncing is only partially implemented so Timo Sirainen <tss@iki.fi> parents: 1975 diff changeset	1172 const char *filename)
6303ef092c5b mbox code compiles again, but syncing is only partially implemented so Timo Sirainen <tss@iki.fi> parents: 1975 diff changeset	1173 {
2123 e01de478882f locking fixes Timo Sirainen <tss@iki.fi> parents: 2121 diff changeset	1174 if (!UIDLIST_IS_LOCKED(ctx->uidlist) &&
1978 6303ef092c5b mbox code compiles again, but syncing is only partially implemented so Timo Sirainen <tss@iki.fi> parents: 1975 diff changeset	1175 hash_lookup(ctx->uidlist->files, filename) == NULL &&
6303ef092c5b mbox code compiles again, but syncing is only partially implemented so Timo Sirainen <tss@iki.fi> parents: 1975 diff changeset	1176 (ctx->partial \|\| hash_lookup(ctx->files, filename) == NULL)) {
2123 e01de478882f locking fixes Timo Sirainen <tss@iki.fi> parents: 2121 diff changeset	1177 if (!ctx->uidlist->initial_read) {
e01de478882f locking fixes Timo Sirainen <tss@iki.fi> parents: 2121 diff changeset	1178 /* first time reading the uidlist */
6800 ca849e28a062 NFS: Delay flushing dovecot-uidlist's attribute cache until we find an Timo Sirainen <tss@iki.fi> parents: 6798 diff changeset	1179 if (maildir_uidlist_refresh(ctx->uidlist, FALSE) < 0) {
2123 e01de478882f locking fixes Timo Sirainen <tss@iki.fi> parents: 2121 diff changeset	1180 ctx->failed = TRUE;
e01de478882f locking fixes Timo Sirainen <tss@iki.fi> parents: 2121 diff changeset	1181 return -1;
e01de478882f locking fixes Timo Sirainen <tss@iki.fi> parents: 2121 diff changeset	1182 }
e01de478882f locking fixes Timo Sirainen <tss@iki.fi> parents: 2121 diff changeset	1183 return maildir_uidlist_sync_next_pre(ctx, filename);
e01de478882f locking fixes Timo Sirainen <tss@iki.fi> parents: 2121 diff changeset	1184 }
1978 6303ef092c5b mbox code compiles again, but syncing is only partially implemented so Timo Sirainen <tss@iki.fi> parents: 1975 diff changeset	1185
2123 e01de478882f locking fixes Timo Sirainen <tss@iki.fi> parents: 2121 diff changeset	1186 return 0;
1978 6303ef092c5b mbox code compiles again, but syncing is only partially implemented so Timo Sirainen <tss@iki.fi> parents: 1975 diff changeset	1187 }
6303ef092c5b mbox code compiles again, but syncing is only partially implemented so Timo Sirainen <tss@iki.fi> parents: 1975 diff changeset	1188
6303ef092c5b mbox code compiles again, but syncing is only partially implemented so Timo Sirainen <tss@iki.fi> parents: 1975 diff changeset	1189 return 1;
6303ef092c5b mbox code compiles again, but syncing is only partially implemented so Timo Sirainen <tss@iki.fi> parents: 1975 diff changeset	1190 }
6303ef092c5b mbox code compiles again, but syncing is only partially implemented so Timo Sirainen <tss@iki.fi> parents: 1975 diff changeset	1191
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1192 int maildir_uidlist_sync_next(struct maildir_uidlist_sync_ctx *ctx,
1947 777da553d1d3 Recent-flag should work now Timo Sirainen <tss@iki.fi> parents: 1940 diff changeset	1193 const char *filename,
777da553d1d3 Recent-flag should work now Timo Sirainen <tss@iki.fi> parents: 1940 diff changeset	1194 enum maildir_uidlist_rec_flag flags)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1195 {
1960 b9ea4db3f8e0 crashfix Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	1196 struct maildir_uidlist *uidlist = ctx->uidlist;
1962 4775ffa73874 Syncing should be fixed now. Timo Sirainen <tss@iki.fi> parents: 1960 diff changeset	1197 struct maildir_uidlist_rec rec, old_rec;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1198
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1199 if (ctx->failed)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1200 return -1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1201
5921 00bdb1f546d3 minor cleanup Timo Sirainen <tss@iki.fi> parents: 5919 diff changeset	1202 if (ctx->partial) {
00bdb1f546d3 minor cleanup Timo Sirainen <tss@iki.fi> parents: 5919 diff changeset	1203 maildir_uidlist_sync_next_partial(ctx, filename, flags);
00bdb1f546d3 minor cleanup Timo Sirainen <tss@iki.fi> parents: 5919 diff changeset	1204 return 1;
00bdb1f546d3 minor cleanup Timo Sirainen <tss@iki.fi> parents: 5919 diff changeset	1205 }
1954 2f6e137cdc44 Syncing optimizations. Timo Sirainen <tss@iki.fi> parents: 1947 diff changeset	1206
1962 4775ffa73874 Syncing should be fixed now. Timo Sirainen <tss@iki.fi> parents: 1960 diff changeset	1207 rec = hash_lookup(ctx->files, filename);
4775ffa73874 Syncing should be fixed now. Timo Sirainen <tss@iki.fi> parents: 1960 diff changeset	1208 if (rec != NULL) {
1947 777da553d1d3 Recent-flag should work now Timo Sirainen <tss@iki.fi> parents: 1940 diff changeset	1209 if ((rec->flags & (MAILDIR_UIDLIST_REC_FLAG_NEW_DIR \|
777da553d1d3 Recent-flag should work now Timo Sirainen <tss@iki.fi> parents: 1940 diff changeset	1210 MAILDIR_UIDLIST_REC_FLAG_MOVED)) == 0) {
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1211 /* possibly duplicate */
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1212 return 0;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1213 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1214
6037 d911d943438e Recent flag handling rewrite. Still not perfect with maildir. Timo Sirainen <tss@iki.fi> parents: 6034 diff changeset	1215 /* probably was in new/ and now we're seeing it in cur/.
d911d943438e Recent flag handling rewrite. Still not perfect with maildir. Timo Sirainen <tss@iki.fi> parents: 6034 diff changeset	1216 remove new/moved flags so if this happens again we'll know
d911d943438e Recent flag handling rewrite. Still not perfect with maildir. Timo Sirainen <tss@iki.fi> parents: 6034 diff changeset	1217 to check for duplicates. */
1947 777da553d1d3 Recent-flag should work now Timo Sirainen <tss@iki.fi> parents: 1940 diff changeset	1218 rec->flags &= ~(MAILDIR_UIDLIST_REC_FLAG_NEW_DIR \|
777da553d1d3 Recent-flag should work now Timo Sirainen <tss@iki.fi> parents: 1940 diff changeset	1219 MAILDIR_UIDLIST_REC_FLAG_MOVED);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1220 } else {
1962 4775ffa73874 Syncing should be fixed now. Timo Sirainen <tss@iki.fi> parents: 1960 diff changeset	1221 old_rec = hash_lookup(uidlist->files, filename);
2123 e01de478882f locking fixes Timo Sirainen <tss@iki.fi> parents: 2121 diff changeset	1222 i_assert(old_rec != NULL \|\| UIDLIST_IS_LOCKED(uidlist));
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1223
6800 ca849e28a062 NFS: Delay flushing dovecot-uidlist's attribute cache until we find an Timo Sirainen <tss@iki.fi> parents: 6798 diff changeset	1224 if (old_rec == NULL && uidlist->nfs_dirty_refresh) {
ca849e28a062 NFS: Delay flushing dovecot-uidlist's attribute cache until we find an Timo Sirainen <tss@iki.fi> parents: 6798 diff changeset	1225 /* We haven't flushed NFS attribute cache yet, do that
ca849e28a062 NFS: Delay flushing dovecot-uidlist's attribute cache until we find an Timo Sirainen <tss@iki.fi> parents: 6798 diff changeset	1226 to make sure the file doesn't exist */
ca849e28a062 NFS: Delay flushing dovecot-uidlist's attribute cache until we find an Timo Sirainen <tss@iki.fi> parents: 6798 diff changeset	1227 if (maildir_uidlist_refresh(uidlist, TRUE) < 0) {
ca849e28a062 NFS: Delay flushing dovecot-uidlist's attribute cache until we find an Timo Sirainen <tss@iki.fi> parents: 6798 diff changeset	1228 ctx->failed = TRUE;
ca849e28a062 NFS: Delay flushing dovecot-uidlist's attribute cache until we find an Timo Sirainen <tss@iki.fi> parents: 6798 diff changeset	1229 return -1;
ca849e28a062 NFS: Delay flushing dovecot-uidlist's attribute cache until we find an Timo Sirainen <tss@iki.fi> parents: 6798 diff changeset	1230 }
ca849e28a062 NFS: Delay flushing dovecot-uidlist's attribute cache until we find an Timo Sirainen <tss@iki.fi> parents: 6798 diff changeset	1231 i_assert(!uidlist->nfs_dirty_refresh);
ca849e28a062 NFS: Delay flushing dovecot-uidlist's attribute cache until we find an Timo Sirainen <tss@iki.fi> parents: 6798 diff changeset	1232 return maildir_uidlist_sync_next(ctx, filename, flags);
ca849e28a062 NFS: Delay flushing dovecot-uidlist's attribute cache until we find an Timo Sirainen <tss@iki.fi> parents: 6798 diff changeset	1233 }
ca849e28a062 NFS: Delay flushing dovecot-uidlist's attribute cache until we find an Timo Sirainen <tss@iki.fi> parents: 6798 diff changeset	1234
1962 4775ffa73874 Syncing should be fixed now. Timo Sirainen <tss@iki.fi> parents: 1960 diff changeset	1235 rec = p_new(ctx->record_pool, struct maildir_uidlist_rec, 1);
1960 b9ea4db3f8e0 crashfix Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	1236
6442 6f12401a149e Crashfix Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	1237 if (old_rec != NULL) {
1962 4775ffa73874 Syncing should be fixed now. Timo Sirainen <tss@iki.fi> parents: 1960 diff changeset	1238 rec = old_rec;
6442 6f12401a149e Crashfix Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	1239 rec->extensions =
6f12401a149e Crashfix Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	1240 p_strdup(ctx->record_pool, rec->extensions);
6f12401a149e Crashfix Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	1241 } else {
1973 f6d2dba3b78b and yet more syncing fixes.. Timo Sirainen <tss@iki.fi> parents: 1971 diff changeset	1242 rec->uid = (uint32_t)-1;
f6d2dba3b78b and yet more syncing fixes.. Timo Sirainen <tss@iki.fi> parents: 1971 diff changeset	1243 ctx->new_files_count++;
5923 239078f515a8 Added maildir_uidlist_sync_remove() Timo Sirainen <tss@iki.fi> parents: 5922 diff changeset	1244 ctx->changed = TRUE;
6037 d911d943438e Recent flag handling rewrite. Still not perfect with maildir. Timo Sirainen <tss@iki.fi> parents: 6034 diff changeset	1245 /* didn't exist in uidlist, it's recent */
d911d943438e Recent flag handling rewrite. Still not perfect with maildir. Timo Sirainen <tss@iki.fi> parents: 6034 diff changeset	1246 flags \|= MAILDIR_UIDLIST_REC_FLAG_RECENT;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1247 }
1971 e7244b4dd03c sync fix Timo Sirainen <tss@iki.fi> parents: 1962 diff changeset	1248
5890 989abd5017a4 Changed record buffers to arrays. Timo Sirainen <tss@iki.fi> parents: 5740 diff changeset	1249 array_append(&ctx->records, &rec, 1);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1250 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1251
2003 d1a2c2ae7226 sync fix Timo Sirainen <tss@iki.fi> parents: 1988 diff changeset	1252 rec->flags = (rec->flags \| flags) & ~MAILDIR_UIDLIST_REC_FLAG_NONSYNCED;
1962 4775ffa73874 Syncing should be fixed now. Timo Sirainen <tss@iki.fi> parents: 1960 diff changeset	1253 rec->filename = p_strdup(ctx->record_pool, filename);
4775ffa73874 Syncing should be fixed now. Timo Sirainen <tss@iki.fi> parents: 1960 diff changeset	1254 hash_insert(ctx->files, rec->filename, rec);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1255 return 1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1256 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1257
5923 239078f515a8 Added maildir_uidlist_sync_remove() Timo Sirainen <tss@iki.fi> parents: 5922 diff changeset	1258 void maildir_uidlist_sync_remove(struct maildir_uidlist_sync_ctx *ctx,
239078f515a8 Added maildir_uidlist_sync_remove() Timo Sirainen <tss@iki.fi> parents: 5922 diff changeset	1259 const char *filename)
239078f515a8 Added maildir_uidlist_sync_remove() Timo Sirainen <tss@iki.fi> parents: 5922 diff changeset	1260 {
6034 dcaa8d5296a9 Fixes to handling appends and unexpunges. Timo Sirainen <tss@iki.fi> parents: 6029 diff changeset	1261 struct maildir_uidlist_rec *rec;
5923 239078f515a8 Added maildir_uidlist_sync_remove() Timo Sirainen <tss@iki.fi> parents: 5922 diff changeset	1262
239078f515a8 Added maildir_uidlist_sync_remove() Timo Sirainen <tss@iki.fi> parents: 5922 diff changeset	1263 i_assert(ctx->partial);
239078f515a8 Added maildir_uidlist_sync_remove() Timo Sirainen <tss@iki.fi> parents: 5922 diff changeset	1264
6034 dcaa8d5296a9 Fixes to handling appends and unexpunges. Timo Sirainen <tss@iki.fi> parents: 6029 diff changeset	1265 if (ctx->first_unwritten_pos != (unsigned int)-1) {
dcaa8d5296a9 Fixes to handling appends and unexpunges. Timo Sirainen <tss@iki.fi> parents: 6029 diff changeset	1266 i_assert(ctx->first_unwritten_pos > 0);
dcaa8d5296a9 Fixes to handling appends and unexpunges. Timo Sirainen <tss@iki.fi> parents: 6029 diff changeset	1267 ctx->first_unwritten_pos--;
dcaa8d5296a9 Fixes to handling appends and unexpunges. Timo Sirainen <tss@iki.fi> parents: 6029 diff changeset	1268 }
dcaa8d5296a9 Fixes to handling appends and unexpunges. Timo Sirainen <tss@iki.fi> parents: 6029 diff changeset	1269 if (ctx->first_nouid_pos != (unsigned int)-1) {
dcaa8d5296a9 Fixes to handling appends and unexpunges. Timo Sirainen <tss@iki.fi> parents: 6029 diff changeset	1270 i_assert(ctx->first_nouid_pos > 0);
dcaa8d5296a9 Fixes to handling appends and unexpunges. Timo Sirainen <tss@iki.fi> parents: 6029 diff changeset	1271 ctx->first_nouid_pos--;
dcaa8d5296a9 Fixes to handling appends and unexpunges. Timo Sirainen <tss@iki.fi> parents: 6029 diff changeset	1272 }
5933 fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	1273
5923 239078f515a8 Added maildir_uidlist_sync_remove() Timo Sirainen <tss@iki.fi> parents: 5922 diff changeset	1274 rec = hash_lookup(ctx->uidlist->files, filename);
239078f515a8 Added maildir_uidlist_sync_remove() Timo Sirainen <tss@iki.fi> parents: 5922 diff changeset	1275 i_assert(rec != NULL);
6029 10042803d65e Crashfixes to handling new messages. Timo Sirainen <tss@iki.fi> parents: 6028 diff changeset	1276 i_assert(rec->uid != (uint32_t)-1);
5923 239078f515a8 Added maildir_uidlist_sync_remove() Timo Sirainen <tss@iki.fi> parents: 5922 diff changeset	1277
239078f515a8 Added maildir_uidlist_sync_remove() Timo Sirainen <tss@iki.fi> parents: 5922 diff changeset	1278 hash_remove(ctx->uidlist->files, filename);
6034 dcaa8d5296a9 Fixes to handling appends and unexpunges. Timo Sirainen <tss@iki.fi> parents: 6029 diff changeset	1279 maildir_uidlist_records_array_delete(ctx->uidlist, rec);
5923 239078f515a8 Added maildir_uidlist_sync_remove() Timo Sirainen <tss@iki.fi> parents: 5922 diff changeset	1280
239078f515a8 Added maildir_uidlist_sync_remove() Timo Sirainen <tss@iki.fi> parents: 5922 diff changeset	1281 ctx->changed = TRUE;
5933 fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	1282 ctx->uidlist->recreate = TRUE;
5923 239078f515a8 Added maildir_uidlist_sync_remove() Timo Sirainen <tss@iki.fi> parents: 5922 diff changeset	1283 }
239078f515a8 Added maildir_uidlist_sync_remove() Timo Sirainen <tss@iki.fi> parents: 5922 diff changeset	1284
4397 5cbabd4ccd9c Don't go fixing duplicate maildir filenames without properly checking that Timo Sirainen <tss@iki.fi> parents: 4238 diff changeset	1285 const char *
5cbabd4ccd9c Don't go fixing duplicate maildir filenames without properly checking that Timo Sirainen <tss@iki.fi> parents: 4238 diff changeset	1286 maildir_uidlist_sync_get_full_filename(struct maildir_uidlist_sync_ctx *ctx,
5cbabd4ccd9c Don't go fixing duplicate maildir filenames without properly checking that Timo Sirainen <tss@iki.fi> parents: 4238 diff changeset	1287 const char *filename)
5cbabd4ccd9c Don't go fixing duplicate maildir filenames without properly checking that Timo Sirainen <tss@iki.fi> parents: 4238 diff changeset	1288 {
5cbabd4ccd9c Don't go fixing duplicate maildir filenames without properly checking that Timo Sirainen <tss@iki.fi> parents: 4238 diff changeset	1289 struct maildir_uidlist_rec *rec;
5cbabd4ccd9c Don't go fixing duplicate maildir filenames without properly checking that Timo Sirainen <tss@iki.fi> parents: 4238 diff changeset	1290
5cbabd4ccd9c Don't go fixing duplicate maildir filenames without properly checking that Timo Sirainen <tss@iki.fi> parents: 4238 diff changeset	1291 rec = hash_lookup(ctx->files, filename);
5cbabd4ccd9c Don't go fixing duplicate maildir filenames without properly checking that Timo Sirainen <tss@iki.fi> parents: 4238 diff changeset	1292 return rec == NULL ? NULL : rec->filename;
5cbabd4ccd9c Don't go fixing duplicate maildir filenames without properly checking that Timo Sirainen <tss@iki.fi> parents: 4238 diff changeset	1293 }
5cbabd4ccd9c Don't go fixing duplicate maildir filenames without properly checking that Timo Sirainen <tss@iki.fi> parents: 4238 diff changeset	1294
6599 a6f51026b969 Added maildir_uidlist_init_readonly() and maildir_uidlist_get_uid(). Timo Sirainen <tss@iki.fi> parents: 6598 diff changeset	1295 bool maildir_uidlist_get_uid(struct maildir_uidlist *uidlist,
a6f51026b969 Added maildir_uidlist_init_readonly() and maildir_uidlist_get_uid(). Timo Sirainen <tss@iki.fi> parents: 6598 diff changeset	1296 const char filename, uint32_t uid_r)
a6f51026b969 Added maildir_uidlist_init_readonly() and maildir_uidlist_get_uid(). Timo Sirainen <tss@iki.fi> parents: 6598 diff changeset	1297 {
a6f51026b969 Added maildir_uidlist_init_readonly() and maildir_uidlist_get_uid(). Timo Sirainen <tss@iki.fi> parents: 6598 diff changeset	1298 struct maildir_uidlist_rec *rec;
a6f51026b969 Added maildir_uidlist_init_readonly() and maildir_uidlist_get_uid(). Timo Sirainen <tss@iki.fi> parents: 6598 diff changeset	1299
a6f51026b969 Added maildir_uidlist_init_readonly() and maildir_uidlist_get_uid(). Timo Sirainen <tss@iki.fi> parents: 6598 diff changeset	1300 rec = hash_lookup(uidlist->files, filename);
a6f51026b969 Added maildir_uidlist_init_readonly() and maildir_uidlist_get_uid(). Timo Sirainen <tss@iki.fi> parents: 6598 diff changeset	1301 if (rec == NULL)
a6f51026b969 Added maildir_uidlist_init_readonly() and maildir_uidlist_get_uid(). Timo Sirainen <tss@iki.fi> parents: 6598 diff changeset	1302 return FALSE;
a6f51026b969 Added maildir_uidlist_init_readonly() and maildir_uidlist_get_uid(). Timo Sirainen <tss@iki.fi> parents: 6598 diff changeset	1303
a6f51026b969 Added maildir_uidlist_init_readonly() and maildir_uidlist_get_uid(). Timo Sirainen <tss@iki.fi> parents: 6598 diff changeset	1304 *uid_r = rec->uid;
a6f51026b969 Added maildir_uidlist_init_readonly() and maildir_uidlist_get_uid(). Timo Sirainen <tss@iki.fi> parents: 6598 diff changeset	1305 return TRUE;
a6f51026b969 Added maildir_uidlist_init_readonly() and maildir_uidlist_get_uid(). Timo Sirainen <tss@iki.fi> parents: 6598 diff changeset	1306 }
a6f51026b969 Added maildir_uidlist_init_readonly() and maildir_uidlist_get_uid(). Timo Sirainen <tss@iki.fi> parents: 6598 diff changeset	1307
4960 4714befe2604 Added maildir_copy_preserve_filename setting. Timo Sirainen <timo.sirainen@movial.fi> parents: 4808 diff changeset	1308 const char *
4714befe2604 Added maildir_copy_preserve_filename setting. Timo Sirainen <timo.sirainen@movial.fi> parents: 4808 diff changeset	1309 maildir_uidlist_get_full_filename(struct maildir_uidlist *uidlist,
4714befe2604 Added maildir_copy_preserve_filename setting. Timo Sirainen <timo.sirainen@movial.fi> parents: 4808 diff changeset	1310 const char *filename)
4714befe2604 Added maildir_copy_preserve_filename setting. Timo Sirainen <timo.sirainen@movial.fi> parents: 4808 diff changeset	1311 {
4714befe2604 Added maildir_copy_preserve_filename setting. Timo Sirainen <timo.sirainen@movial.fi> parents: 4808 diff changeset	1312 struct maildir_uidlist_rec *rec;
4714befe2604 Added maildir_copy_preserve_filename setting. Timo Sirainen <timo.sirainen@movial.fi> parents: 4808 diff changeset	1313
4714befe2604 Added maildir_copy_preserve_filename setting. Timo Sirainen <timo.sirainen@movial.fi> parents: 4808 diff changeset	1314 rec = hash_lookup(uidlist->files, filename);
4714befe2604 Added maildir_copy_preserve_filename setting. Timo Sirainen <timo.sirainen@movial.fi> parents: 4808 diff changeset	1315 return rec == NULL ? NULL : rec->filename;
4714befe2604 Added maildir_copy_preserve_filename setting. Timo Sirainen <timo.sirainen@movial.fi> parents: 4808 diff changeset	1316 }
4714befe2604 Added maildir_copy_preserve_filename setting. Timo Sirainen <timo.sirainen@movial.fi> parents: 4808 diff changeset	1317
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1318 static int maildir_time_cmp(const void p1, const void p2)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1319 {
1962 4775ffa73874 Syncing should be fixed now. Timo Sirainen <tss@iki.fi> parents: 1960 diff changeset	1320 const struct maildir_uidlist_rec const rec1 = p1, const rec2 = p2;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1321
5903 5d0827cd9839 Added maildir_filename_sort_cmp() which sorts maildir filenames by time and Timo Sirainen <tss@iki.fi> parents: 5902 diff changeset	1322 return maildir_filename_sort_cmp((rec1)->filename, (rec2)->filename);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1323 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1324
6029 10042803d65e Crashfixes to handling new messages. Timo Sirainen <tss@iki.fi> parents: 6028 diff changeset	1325 static void maildir_uidlist_assign_uids(struct maildir_uidlist_sync_ctx *ctx)
1954 2f6e137cdc44 Syncing optimizations. Timo Sirainen <tss@iki.fi> parents: 1947 diff changeset	1326 {
5890 989abd5017a4 Changed record buffers to arrays. Timo Sirainen <tss@iki.fi> parents: 5740 diff changeset	1327 struct maildir_uidlist_rec **recs;
989abd5017a4 Changed record buffers to arrays. Timo Sirainen <tss@iki.fi> parents: 5740 diff changeset	1328 unsigned int dest, count;
1954 2f6e137cdc44 Syncing optimizations. Timo Sirainen <tss@iki.fi> parents: 1947 diff changeset	1329
5393 434047e53586 When saving/copying mails, we must not do any sorting based on the filename Timo Sirainen <tss@iki.fi> parents: 5388 diff changeset	1330 i_assert(UIDLIST_IS_LOCKED(ctx->uidlist));
6034 dcaa8d5296a9 Fixes to handling appends and unexpunges. Timo Sirainen <tss@iki.fi> parents: 6029 diff changeset	1331 i_assert(ctx->first_nouid_pos != (unsigned int)-1);
2094 8f46cf3e9b5a small fixes Timo Sirainen <tss@iki.fi> parents: 2053 diff changeset	1332
6034 dcaa8d5296a9 Fixes to handling appends and unexpunges. Timo Sirainen <tss@iki.fi> parents: 6029 diff changeset	1333 if (ctx->first_unwritten_pos == (unsigned int)-1)
dcaa8d5296a9 Fixes to handling appends and unexpunges. Timo Sirainen <tss@iki.fi> parents: 6029 diff changeset	1334 ctx->first_unwritten_pos = ctx->first_nouid_pos;
1954 2f6e137cdc44 Syncing optimizations. Timo Sirainen <tss@iki.fi> parents: 1947 diff changeset	1335
2f6e137cdc44 Syncing optimizations. Timo Sirainen <tss@iki.fi> parents: 1947 diff changeset	1336 /* sort new files and assign UIDs for them */
6034 dcaa8d5296a9 Fixes to handling appends and unexpunges. Timo Sirainen <tss@iki.fi> parents: 6029 diff changeset	1337 recs = array_get_modifiable(&ctx->uidlist->records, &count);
dcaa8d5296a9 Fixes to handling appends and unexpunges. Timo Sirainen <tss@iki.fi> parents: 6029 diff changeset	1338 qsort(recs + ctx->first_nouid_pos, count - ctx->first_nouid_pos,
5892 36056f044f1f Removed unused MAILDIR_UIDLIST_SYNC_ORDERED flag. Timo Sirainen <tss@iki.fi> parents: 5891 diff changeset	1339 sizeof(*recs), maildir_time_cmp);
6034 dcaa8d5296a9 Fixes to handling appends and unexpunges. Timo Sirainen <tss@iki.fi> parents: 6029 diff changeset	1340
dcaa8d5296a9 Fixes to handling appends and unexpunges. Timo Sirainen <tss@iki.fi> parents: 6029 diff changeset	1341 for (dest = ctx->first_nouid_pos; dest < count; dest++) {
5890 989abd5017a4 Changed record buffers to arrays. Timo Sirainen <tss@iki.fi> parents: 5740 diff changeset	1342 i_assert(recs[dest]->uid == (uint32_t)-1);
989abd5017a4 Changed record buffers to arrays. Timo Sirainen <tss@iki.fi> parents: 5740 diff changeset	1343 recs[dest]->uid = ctx->uidlist->next_uid++;
989abd5017a4 Changed record buffers to arrays. Timo Sirainen <tss@iki.fi> parents: 5740 diff changeset	1344 recs[dest]->flags &= ~MAILDIR_UIDLIST_REC_FLAG_MOVED;
1954 2f6e137cdc44 Syncing optimizations. Timo Sirainen <tss@iki.fi> parents: 1947 diff changeset	1345 }
2567 6c802bcb957e fix Timo Sirainen <tss@iki.fi> parents: 2565 diff changeset	1346
6029 10042803d65e Crashfixes to handling new messages. Timo Sirainen <tss@iki.fi> parents: 6028 diff changeset	1347 ctx->new_files_count = 0;
6034 dcaa8d5296a9 Fixes to handling appends and unexpunges. Timo Sirainen <tss@iki.fi> parents: 6029 diff changeset	1348 ctx->first_nouid_pos = (unsigned int)-1;
6029 10042803d65e Crashfixes to handling new messages. Timo Sirainen <tss@iki.fi> parents: 6028 diff changeset	1349
5393 434047e53586 When saving/copying mails, we must not do any sorting based on the filename Timo Sirainen <tss@iki.fi> parents: 5388 diff changeset	1350 ctx->uidlist->last_seen_uid = ctx->uidlist->next_uid-1;
6029 10042803d65e Crashfixes to handling new messages. Timo Sirainen <tss@iki.fi> parents: 6028 diff changeset	1351 ctx->uidlist->change_counter++;
1954 2f6e137cdc44 Syncing optimizations. Timo Sirainen <tss@iki.fi> parents: 1947 diff changeset	1352 }
2f6e137cdc44 Syncing optimizations. Timo Sirainen <tss@iki.fi> parents: 1947 diff changeset	1353
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1354 static void maildir_uidlist_swap(struct maildir_uidlist_sync_ctx *ctx)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1355 {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1356 struct maildir_uidlist *uidlist = ctx->uidlist;
5890 989abd5017a4 Changed record buffers to arrays. Timo Sirainen <tss@iki.fi> parents: 5740 diff changeset	1357 struct maildir_uidlist_rec **recs;
989abd5017a4 Changed record buffers to arrays. Timo Sirainen <tss@iki.fi> parents: 5740 diff changeset	1358 unsigned int count;
1960 b9ea4db3f8e0 crashfix Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	1359
2094 8f46cf3e9b5a small fixes Timo Sirainen <tss@iki.fi> parents: 2053 diff changeset	1360 /* buffer is unsorted, sort it by UID */
5890 989abd5017a4 Changed record buffers to arrays. Timo Sirainen <tss@iki.fi> parents: 5740 diff changeset	1361 recs = array_get_modifiable(&ctx->records, &count);
989abd5017a4 Changed record buffers to arrays. Timo Sirainen <tss@iki.fi> parents: 5740 diff changeset	1362 qsort(recs, count, sizeof(*recs), maildir_uid_cmp);
1947 777da553d1d3 Recent-flag should work now Timo Sirainen <tss@iki.fi> parents: 1940 diff changeset	1363
5890 989abd5017a4 Changed record buffers to arrays. Timo Sirainen <tss@iki.fi> parents: 5740 diff changeset	1364 array_free(&uidlist->records);
989abd5017a4 Changed record buffers to arrays. Timo Sirainen <tss@iki.fi> parents: 5740 diff changeset	1365 uidlist->records = ctx->records;
989abd5017a4 Changed record buffers to arrays. Timo Sirainen <tss@iki.fi> parents: 5740 diff changeset	1366 ctx->records.arr.buffer = NULL;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1367
6417 047d0d8bbf0a hash_destroy() and hash_iterate_deinit() now take ** pointer. Timo Sirainen <tss@iki.fi> parents: 6409 diff changeset	1368 hash_destroy(&uidlist->files);
1931 35a947481130 fixes Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	1369 uidlist->files = ctx->files;
35a947481130 fixes Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	1370 ctx->files = NULL;
35a947481130 fixes Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	1371
1962 4775ffa73874 Syncing should be fixed now. Timo Sirainen <tss@iki.fi> parents: 1960 diff changeset	1372 if (uidlist->record_pool != NULL)
6428 7cad076906eb pool_unref() now takes ** pointer. Timo Sirainen <tss@iki.fi> parents: 6417 diff changeset	1373 pool_unref(&uidlist->record_pool);
1962 4775ffa73874 Syncing should be fixed now. Timo Sirainen <tss@iki.fi> parents: 1960 diff changeset	1374 uidlist->record_pool = ctx->record_pool;
4775ffa73874 Syncing should be fixed now. Timo Sirainen <tss@iki.fi> parents: 1960 diff changeset	1375 ctx->record_pool = NULL;
4775ffa73874 Syncing should be fixed now. Timo Sirainen <tss@iki.fi> parents: 1960 diff changeset	1376
5933 fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	1377 if (ctx->new_files_count != 0) {
6034 dcaa8d5296a9 Fixes to handling appends and unexpunges. Timo Sirainen <tss@iki.fi> parents: 6029 diff changeset	1378 ctx->first_nouid_pos = count - ctx->new_files_count;
6029 10042803d65e Crashfixes to handling new messages. Timo Sirainen <tss@iki.fi> parents: 6028 diff changeset	1379 maildir_uidlist_assign_uids(ctx);
5933 fdc7e47ccea3 dovecot-uidlist can now be updated by appending to it. It's recreated only Timo Sirainen <tss@iki.fi> parents: 5932 diff changeset	1380 }
5913 699356ffe549 If uidlist changes while it's being iterated, don't crash. Timo Sirainen <tss@iki.fi> parents: 5903 diff changeset	1381
699356ffe549 If uidlist changes while it's being iterated, don't crash. Timo Sirainen <tss@iki.fi> parents: 5903 diff changeset	1382 ctx->uidlist->change_counter++;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1383 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1384
2123 e01de478882f locking fixes Timo Sirainen <tss@iki.fi> parents: 2121 diff changeset	1385 void maildir_uidlist_sync_finish(struct maildir_uidlist_sync_ctx *ctx)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1386 {
1978 6303ef092c5b mbox code compiles again, but syncing is only partially implemented so Timo Sirainen <tss@iki.fi> parents: 1975 diff changeset	1387 if (!ctx->partial) {
2123 e01de478882f locking fixes Timo Sirainen <tss@iki.fi> parents: 2121 diff changeset	1388 if (!ctx->failed)
1978 6303ef092c5b mbox code compiles again, but syncing is only partially implemented so Timo Sirainen <tss@iki.fi> parents: 1975 diff changeset	1389 maildir_uidlist_swap(ctx);
6303ef092c5b mbox code compiles again, but syncing is only partially implemented so Timo Sirainen <tss@iki.fi> parents: 1975 diff changeset	1390 } else {
6034 dcaa8d5296a9 Fixes to handling appends and unexpunges. Timo Sirainen <tss@iki.fi> parents: 6029 diff changeset	1391 if (ctx->new_files_count != 0)
6029 10042803d65e Crashfixes to handling new messages. Timo Sirainen <tss@iki.fi> parents: 6028 diff changeset	1392 maildir_uidlist_assign_uids(ctx);
1962 4775ffa73874 Syncing should be fixed now. Timo Sirainen <tss@iki.fi> parents: 1960 diff changeset	1393 }
2094 8f46cf3e9b5a small fixes Timo Sirainen <tss@iki.fi> parents: 2053 diff changeset	1394
1984 9c159272f721 syncing fixes Timo Sirainen <tss@iki.fi> parents: 1978 diff changeset	1395 ctx->finished = TRUE;
2038 df504dad3aec Recent flag fixes. Should work perfectly now with maildir. Timo Sirainen <tss@iki.fi> parents: 2005 diff changeset	1396 ctx->uidlist->initial_sync = TRUE;
1984 9c159272f721 syncing fixes Timo Sirainen <tss@iki.fi> parents: 1978 diff changeset	1397 }
9c159272f721 syncing fixes Timo Sirainen <tss@iki.fi> parents: 1978 diff changeset	1398
4238 3c8b191b0019 Adding mail to index while saving it had a race condition. Fixing it Timo Sirainen <timo.sirainen@movial.fi> parents: 4225 diff changeset	1399 int maildir_uidlist_sync_deinit(struct maildir_uidlist_sync_ctx **_ctx)
1984 9c159272f721 syncing fixes Timo Sirainen <tss@iki.fi> parents: 1978 diff changeset	1400 {
4238 3c8b191b0019 Adding mail to index while saving it had a race condition. Fixing it Timo Sirainen <timo.sirainen@movial.fi> parents: 4225 diff changeset	1401 struct maildir_uidlist_sync_ctx ctx = _ctx;
1984 9c159272f721 syncing fixes Timo Sirainen <tss@iki.fi> parents: 1978 diff changeset	1402 int ret = ctx->failed ? -1 : 0;
9c159272f721 syncing fixes Timo Sirainen <tss@iki.fi> parents: 1978 diff changeset	1403
4238 3c8b191b0019 Adding mail to index while saving it had a race condition. Fixing it Timo Sirainen <timo.sirainen@movial.fi> parents: 4225 diff changeset	1404 *_ctx = NULL;
3c8b191b0019 Adding mail to index while saving it had a race condition. Fixing it Timo Sirainen <timo.sirainen@movial.fi> parents: 4225 diff changeset	1405
1984 9c159272f721 syncing fixes Timo Sirainen <tss@iki.fi> parents: 1978 diff changeset	1406 if (!ctx->finished)
9c159272f721 syncing fixes Timo Sirainen <tss@iki.fi> parents: 1978 diff changeset	1407 maildir_uidlist_sync_finish(ctx);
1954 2f6e137cdc44 Syncing optimizations. Timo Sirainen <tss@iki.fi> parents: 1947 diff changeset	1408
1988 739b469b99e2 syncing fixes Timo Sirainen <tss@iki.fi> parents: 1987 diff changeset	1409 if (ctx->partial)
739b469b99e2 syncing fixes Timo Sirainen <tss@iki.fi> parents: 1987 diff changeset	1410 maildir_uidlist_mark_all(ctx->uidlist, FALSE);
739b469b99e2 syncing fixes Timo Sirainen <tss@iki.fi> parents: 1987 diff changeset	1411
5950 49aebb3028f7 Added support for updating uidlist record extensions. Timo Sirainen <tss@iki.fi> parents: 5944 diff changeset	1412 if ((ctx->changed \|\| ctx->uidlist->recreate) && !ctx->failed) {
5917 525b449313a7 Don't use uidlist lock to update uidlist. Have a separate real dotlock for Timo Sirainen <tss@iki.fi> parents: 5915 diff changeset	1413 t_push();
5950 49aebb3028f7 Added support for updating uidlist record extensions. Timo Sirainen <tss@iki.fi> parents: 5944 diff changeset	1414 ret = maildir_uidlist_sync_update(ctx);
5917 525b449313a7 Don't use uidlist lock to update uidlist. Have a separate real dotlock for Timo Sirainen <tss@iki.fi> parents: 5915 diff changeset	1415 t_pop();
3530 e9695ec7925b Recursive maildir uidlist syncs caused assert crashes. Also did some Timo Sirainen <tss@iki.fi> parents: 3472 diff changeset	1416 }
e9695ec7925b Recursive maildir uidlist syncs caused assert crashes. Also did some Timo Sirainen <tss@iki.fi> parents: 3472 diff changeset	1417
5917 525b449313a7 Don't use uidlist lock to update uidlist. Have a separate real dotlock for Timo Sirainen <tss@iki.fi> parents: 5915 diff changeset	1418 maildir_uidlist_unlock(ctx->uidlist);
1940 876bd2849a00 fixes Timo Sirainen <tss@iki.fi> parents: 1934 diff changeset	1419
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1420 if (ctx->files != NULL)
6417 047d0d8bbf0a hash_destroy() and hash_iterate_deinit() now take ** pointer. Timo Sirainen <tss@iki.fi> parents: 6409 diff changeset	1421 hash_destroy(&ctx->files);
1962 4775ffa73874 Syncing should be fixed now. Timo Sirainen <tss@iki.fi> parents: 1960 diff changeset	1422 if (ctx->record_pool != NULL)
6428 7cad076906eb pool_unref() now takes ** pointer. Timo Sirainen <tss@iki.fi> parents: 6417 diff changeset	1423 pool_unref(&ctx->record_pool);
5890 989abd5017a4 Changed record buffers to arrays. Timo Sirainen <tss@iki.fi> parents: 5740 diff changeset	1424 if (array_is_created(&ctx->records))
989abd5017a4 Changed record buffers to arrays. Timo Sirainen <tss@iki.fi> parents: 5740 diff changeset	1425 array_free(&ctx->records);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1426 i_free(ctx);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1427 return ret;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1428 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1429
2053 66f5d28f9b27 race condition fixes Timo Sirainen <tss@iki.fi> parents: 2052 diff changeset	1430 void maildir_uidlist_add_flags(struct maildir_uidlist *uidlist,
66f5d28f9b27 race condition fixes Timo Sirainen <tss@iki.fi> parents: 2052 diff changeset	1431 const char *filename,
66f5d28f9b27 race condition fixes Timo Sirainen <tss@iki.fi> parents: 2052 diff changeset	1432 enum maildir_uidlist_rec_flag flags)
66f5d28f9b27 race condition fixes Timo Sirainen <tss@iki.fi> parents: 2052 diff changeset	1433 {
66f5d28f9b27 race condition fixes Timo Sirainen <tss@iki.fi> parents: 2052 diff changeset	1434 struct maildir_uidlist_rec *rec;
66f5d28f9b27 race condition fixes Timo Sirainen <tss@iki.fi> parents: 2052 diff changeset	1435
66f5d28f9b27 race condition fixes Timo Sirainen <tss@iki.fi> parents: 2052 diff changeset	1436 rec = hash_lookup(uidlist->files, filename);
66f5d28f9b27 race condition fixes Timo Sirainen <tss@iki.fi> parents: 2052 diff changeset	1437 i_assert(rec != NULL);
66f5d28f9b27 race condition fixes Timo Sirainen <tss@iki.fi> parents: 2052 diff changeset	1438
66f5d28f9b27 race condition fixes Timo Sirainen <tss@iki.fi> parents: 2052 diff changeset	1439 rec->flags \|= flags;
66f5d28f9b27 race condition fixes Timo Sirainen <tss@iki.fi> parents: 2052 diff changeset	1440 }
66f5d28f9b27 race condition fixes Timo Sirainen <tss@iki.fi> parents: 2052 diff changeset	1441
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1442 struct maildir_uidlist_iter_ctx *
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1443 maildir_uidlist_iter_init(struct maildir_uidlist *uidlist)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1444 {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1445 struct maildir_uidlist_iter_ctx *ctx;
5890 989abd5017a4 Changed record buffers to arrays. Timo Sirainen <tss@iki.fi> parents: 5740 diff changeset	1446 unsigned int count;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1447
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1448 ctx = i_new(struct maildir_uidlist_iter_ctx, 1);
5913 699356ffe549 If uidlist changes while it's being iterated, don't crash. Timo Sirainen <tss@iki.fi> parents: 5903 diff changeset	1449 ctx->uidlist = uidlist;
5890 989abd5017a4 Changed record buffers to arrays. Timo Sirainen <tss@iki.fi> parents: 5740 diff changeset	1450 ctx->next = array_get(&uidlist->records, &count);
989abd5017a4 Changed record buffers to arrays. Timo Sirainen <tss@iki.fi> parents: 5740 diff changeset	1451 ctx->end = ctx->next + count;
5913 699356ffe549 If uidlist changes while it's being iterated, don't crash. Timo Sirainen <tss@iki.fi> parents: 5903 diff changeset	1452 ctx->change_counter = ctx->uidlist->change_counter;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1453 return ctx;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1454 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1455
5913 699356ffe549 If uidlist changes while it's being iterated, don't crash. Timo Sirainen <tss@iki.fi> parents: 5903 diff changeset	1456 static void
699356ffe549 If uidlist changes while it's being iterated, don't crash. Timo Sirainen <tss@iki.fi> parents: 5903 diff changeset	1457 maildir_uidlist_iter_update_idx(struct maildir_uidlist_iter_ctx *ctx)
699356ffe549 If uidlist changes while it's being iterated, don't crash. Timo Sirainen <tss@iki.fi> parents: 5903 diff changeset	1458 {
699356ffe549 If uidlist changes while it's being iterated, don't crash. Timo Sirainen <tss@iki.fi> parents: 5903 diff changeset	1459 unsigned int old_rev_idx, idx, count;
699356ffe549 If uidlist changes while it's being iterated, don't crash. Timo Sirainen <tss@iki.fi> parents: 5903 diff changeset	1460
699356ffe549 If uidlist changes while it's being iterated, don't crash. Timo Sirainen <tss@iki.fi> parents: 5903 diff changeset	1461 old_rev_idx = ctx->end - ctx->next;
699356ffe549 If uidlist changes while it's being iterated, don't crash. Timo Sirainen <tss@iki.fi> parents: 5903 diff changeset	1462 ctx->next = array_get(&ctx->uidlist->records, &count);
699356ffe549 If uidlist changes while it's being iterated, don't crash. Timo Sirainen <tss@iki.fi> parents: 5903 diff changeset	1463 ctx->end = ctx->next + count;
699356ffe549 If uidlist changes while it's being iterated, don't crash. Timo Sirainen <tss@iki.fi> parents: 5903 diff changeset	1464
699356ffe549 If uidlist changes while it's being iterated, don't crash. Timo Sirainen <tss@iki.fi> parents: 5903 diff changeset	1465 idx = old_rev_idx >= count ? 0 :
699356ffe549 If uidlist changes while it's being iterated, don't crash. Timo Sirainen <tss@iki.fi> parents: 5903 diff changeset	1466 count - old_rev_idx;
699356ffe549 If uidlist changes while it's being iterated, don't crash. Timo Sirainen <tss@iki.fi> parents: 5903 diff changeset	1467 while (idx < count && ctx->next[idx]->uid <= ctx->prev_uid)
699356ffe549 If uidlist changes while it's being iterated, don't crash. Timo Sirainen <tss@iki.fi> parents: 5903 diff changeset	1468 idx++;
699356ffe549 If uidlist changes while it's being iterated, don't crash. Timo Sirainen <tss@iki.fi> parents: 5903 diff changeset	1469 while (idx > 0 && ctx->next[idx-1]->uid > ctx->prev_uid)
699356ffe549 If uidlist changes while it's being iterated, don't crash. Timo Sirainen <tss@iki.fi> parents: 5903 diff changeset	1470 idx--;
699356ffe549 If uidlist changes while it's being iterated, don't crash. Timo Sirainen <tss@iki.fi> parents: 5903 diff changeset	1471
699356ffe549 If uidlist changes while it's being iterated, don't crash. Timo Sirainen <tss@iki.fi> parents: 5903 diff changeset	1472 ctx->next += idx;
699356ffe549 If uidlist changes while it's being iterated, don't crash. Timo Sirainen <tss@iki.fi> parents: 5903 diff changeset	1473 }
699356ffe549 If uidlist changes while it's being iterated, don't crash. Timo Sirainen <tss@iki.fi> parents: 5903 diff changeset	1474
5944 760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	1475 static bool maildir_uidlist_iter_next_rec(struct maildir_uidlist_iter_ctx *ctx,
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	1476 struct maildir_uidlist_rec **rec_r)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1477 {
5922 79513299d42c code cleanup Timo Sirainen <tss@iki.fi> parents: 5921 diff changeset	1478 struct maildir_uidlist_rec *rec;
79513299d42c code cleanup Timo Sirainen <tss@iki.fi> parents: 5921 diff changeset	1479
5913 699356ffe549 If uidlist changes while it's being iterated, don't crash. Timo Sirainen <tss@iki.fi> parents: 5903 diff changeset	1480 if (ctx->change_counter != ctx->uidlist->change_counter)
699356ffe549 If uidlist changes while it's being iterated, don't crash. Timo Sirainen <tss@iki.fi> parents: 5903 diff changeset	1481 maildir_uidlist_iter_update_idx(ctx);
699356ffe549 If uidlist changes while it's being iterated, don't crash. Timo Sirainen <tss@iki.fi> parents: 5903 diff changeset	1482
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1483 if (ctx->next == ctx->end)
5942 bf536b8dbc51 maildir_uidlist_iter_next() should return bool, not int. Timo Sirainen <tss@iki.fi> parents: 5941 diff changeset	1484 return FALSE;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1485
5922 79513299d42c code cleanup Timo Sirainen <tss@iki.fi> parents: 5921 diff changeset	1486 rec = *ctx->next;
79513299d42c code cleanup Timo Sirainen <tss@iki.fi> parents: 5921 diff changeset	1487 i_assert(rec->uid != (uint32_t)-1);
79513299d42c code cleanup Timo Sirainen <tss@iki.fi> parents: 5921 diff changeset	1488
79513299d42c code cleanup Timo Sirainen <tss@iki.fi> parents: 5921 diff changeset	1489 ctx->prev_uid = rec->uid;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1490 ctx->next++;
5922 79513299d42c code cleanup Timo Sirainen <tss@iki.fi> parents: 5921 diff changeset	1491
5944 760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	1492 *rec_r = rec;
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	1493 return TRUE;
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	1494 }
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	1495
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	1496 bool maildir_uidlist_iter_next(struct maildir_uidlist_iter_ctx *ctx,
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	1497 uint32_t *uid_r,
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	1498 enum maildir_uidlist_rec_flag *flags_r,
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	1499 const char **filename_r)
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	1500 {
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	1501 struct maildir_uidlist_rec *rec;
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	1502
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	1503 if (!maildir_uidlist_iter_next_rec(ctx, &rec))
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	1504 return FALSE;
760f910004d9 Created a new extensible version 3 dovecot-uidlist file. Timo Sirainen <tss@iki.fi> parents: 5942 diff changeset	1505
5922 79513299d42c code cleanup Timo Sirainen <tss@iki.fi> parents: 5921 diff changeset	1506 *uid_r = rec->uid;
79513299d42c code cleanup Timo Sirainen <tss@iki.fi> parents: 5921 diff changeset	1507 *flags_r = rec->flags;
79513299d42c code cleanup Timo Sirainen <tss@iki.fi> parents: 5921 diff changeset	1508 *filename_r = rec->filename;
5942 bf536b8dbc51 maildir_uidlist_iter_next() should return bool, not int. Timo Sirainen <tss@iki.fi> parents: 5941 diff changeset	1509 return TRUE;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1510 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1511
6409 7f733ba453bf Changed _deinit() APIs to take * pointer and set it to NULL. Timo Sirainen <tss@iki.fi> parents: 6344 diff changeset	1512 void maildir_uidlist_iter_deinit(struct maildir_uidlist_iter_ctx **_ctx)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1513 {
6409 7f733ba453bf Changed _deinit() APIs to take * pointer and set it to NULL. Timo Sirainen <tss@iki.fi> parents: 6344 diff changeset	1514 i_free(*_ctx);
7f733ba453bf Changed _deinit() APIs to take * pointer and set it to NULL. Timo Sirainen <tss@iki.fi> parents: 6344 diff changeset	1515 *_ctx = NULL;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1516 }

Mercurial > dovecot > original-hg > dovecot-1.2

annotate src/lib-storage/index/maildir/maildir-uidlist.c @ 6857:41911abe6fa7 HEAD