dovecot/original-hg/dovecot-1.2: src/lib-storage/index/maildir/maildir-sync.c annotate

annotate src/lib-storage/index/maildir/maildir-sync.c @ 1984:9c159272f721 HEAD

syncing fixes

author	Timo Sirainen <tss@iki.fi>
date	Mon, 10 May 2004 02:06:27 +0300
parents	6303ef092c5b
children	739b469b99e2

rev	line source
1955 0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	1 /* Copyright (C) 2004 Timo Sirainen */
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	2
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	3 /*
1955 0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	4 Here's a description of how we handle Maildir synchronization and
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	5 it's problems:
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	6
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	7 We want to be as efficient as we can. The most efficient way to
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	8 check if changes have occured is to stat() the new/ and cur/
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	9 directories and uidlist file - if their mtimes haven't changed,
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	10 there's no changes and we don't need to do anything.
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	11
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	12 Problem 1: Multiple changes can happen within a single second -
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	13 nothing guarantees that once we synced it, someone else didn't just
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	14 then make a modification. Such modifications wouldn't get noticed
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	15 until a new modification occured later.
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	16
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	17 Problem 2: Syncing cur/ directory is much more costly than syncing
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	18 new/. Moving mails from new/ to cur/ will always change mtime of
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	19 cur/ causing us to sync it as well.
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	20
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	21 Problem 3: We may not be able to move mail from new/ to cur/
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	22 because we're out of quota, or simply because we're accessing a
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	23 read-only mailbox.
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	24
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	25
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	26 MAILDIR_SYNC_SECS
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	27 -----------------
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	28
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	29 Several checks below use MAILDIR_SYNC_SECS, which should be maximum
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	30 clock drift between all computers accessing the maildir (eg. via
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	31 NFS), rounded up to next second. Our default is 1 second, since
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	32 everyone should be using NTP.
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	33
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	34 Note that setting it to 0 works only if there's only one computer
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	35 accessing the maildir. It's practically impossible to make two
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	36 clocks _exactly_ synchronized.
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	37
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	38 It might be possible to only use file server's clock by looking at
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	39 the atime field, but I don't know how well that would actually work.
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	40
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	41 cur directory
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	42 -------------
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	43
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	44 We have dirty_cur_time variable which is set to cur/ directory's
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	45 mtime when it's >= time() - MAILDIR_SYNC_SECS and we _think_ we have
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	46 synchronized the directory.
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	47
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	48 When dirty_cur_time is non-zero, we don't synchronize the cur/
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	49 directory until
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	50
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	51 a) cur/'s mtime changes
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	52 b) opening a mail fails with ENOENT
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	53 c) time() > dirty_cur_time + MAILDIR_SYNC_SECS
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	54
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	55 This allows us to modify the maildir multiple times without having
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	56 to sync it at every change. The sync will eventually be done to
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	57 make sure we didn't miss any external changes.
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	58
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	59 The dirty_cur_time is set when:
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	60
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	61 - we change message flags
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	62 - we expunge messages
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	63 - we move mail from new/ to cur/
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	64 - we sync cur/ directory and it's mtime is >= time() - MAILDIR_SYNC_SECS
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	65
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	66 It's unset when we do the final syncing, ie. when mtime is
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	67 older than time() - MAILDIR_SYNC_SECS.
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	68
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	69 new directory
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	70 -------------
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	71
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	72 If new/'s mtime is >= time() - MAILDIR_SYNC_SECS, always synchronize
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	73 it. dirty_cur_time-like feature might save us a few syncs, but
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	74 that might break a client which saves a mail in one connection and
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	75 tries to fetch it in another one. new/ directory is almost always
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	76 empty, so syncing it should be very fast anyway. Actually this can
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	77 still happen if we sync only new/ dir while another client is also
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	78 moving mails from it to cur/ - it takes us a while to see them.
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	79 That's pretty unlikely to happen however, and only way to fix it
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	80 would be to always synchronize cur/ after new/.
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	81
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	82 Normally we move all mails from new/ to cur/ whenever we sync it. If
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	83 it's not possible for some reason, we mark the mail with "probably
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	84 exists in new/ directory" flag.
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	85
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	86 If rename() still fails because of ENOSPC or EDQUOT, we still save
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	87 the flag changes in index with dirty-flag on. When moving the mail
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	88 to cur/ directory, or when we notice it's already moved there, we
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	89 apply the flag changes to the filename, rename it and remove the
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	90 dirty flag. If there's dirty flags, this should be tried every time
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	91 after expunge or when closing the mailbox.
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	92
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	93 uidlist
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	94 -------
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	95
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	96 This file contains UID <-> filename mappings. It's updated only when
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	97 new mail arrives, so it may contain filenames that have already been
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	98 deleted. Updating is done by getting uidlist.lock file, writing the
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	99 whole uidlist into it and rename()ing it over the old uidlist. This
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	100 means there's no need to lock the file for reading.
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	101
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	102 Whenever uidlist is rewritten, it's mtime must be larger than the old
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	103 one's. Use utime() before rename() if needed. Note that inode checking
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	104 wouldn't have been sufficient as inode numbers can be reused.
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	105
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	106 This file is usually read the first time you need to know filename for
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	107 given UID. After that it's not re-read unless new mails come that we
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	108 don't know about.
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	109
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	110 broken clients
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	111 --------------
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	112
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	113 Originally the middle identifier in Maildir filename was specified
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	114 only as <process id>_<delivery counter>. That however created a
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	115 problem with randomized PIDs which made it possible that the same
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	116 PID was reused within one second.
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	117
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	118 So if within one second a mail was delivered, MUA moved it to cur/
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	119 and another mail was delivered by a new process using same PID as
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	120 the first one, we likely ended up overwriting the first mail when
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	121 the second mail was moved over it.
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	122
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	123 Nowadays everyone should be giving a bit more specific identifier,
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	124 for example include microseconds in it which Dovecot does.
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	125
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	126 There's a simple way to prevent this from happening in some cases:
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	127 Don't move the mail from new/ to cur/ if it's mtime is >= time() -
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	128 MAILDIR_SYNC_SECS. The second delivery's link() call then fails
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	129 because the file is already in new/, and it will then use a
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	130 different filename. There's a few problems with this however:
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	131
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	132 - it requires extra stat() call which is unneeded extra I/O
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	133 - another MUA might still move the mail to cur/
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	134 - if first file's flags are modified by either Dovecot or another
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	135 MUA, it's moved to cur/ (you _could_ just do the dirty-flagging
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	136 but that'd be ugly)
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	137
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	138 Because this is useful only for very few people and it requires
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	139 extra I/O, I decided not to implement this. It should be however
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	140 quite easy to do since we need to be able to deal with files in new/
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	141 in any case.
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	142
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	143 It's also possible to never accidentally overwrite a mail by using
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	144 link() + unlink() rather than rename(). This however isn't very
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	145 good idea as it introduces potential race conditions when multiple
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	146 clients are accessing the mailbox:
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	147
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	148 Trying to move the same mail from new/ to cur/ at the same time:
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	149
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	150 a) Client 1 uses slightly different filename than client 2,
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	151 for example one sets read-flag on but the other doesn't.
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	152 You have the same mail duplicated now.
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	153
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	154 b) Client 3 sees the mail between Client 1's and 2's link() calls
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	155 and changes it's flag. You have the same mail duplicated now.
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	156
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	157 And it gets worse when they're unlink()ing in cur/ directory:
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	158
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	159 c) Client 1 changes mails's flag and client 2 changes it back
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	160 between 1's link() and unlink(). The mail is now expunged.
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	161
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	162 d) If you try to deal with the duplicates by unlink()ing another
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	163 one of them, you might end up unlinking both of them.
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	164
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	165 So, what should we do then if we notice a duplicate? First of all,
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	166 it might not be a duplicate at all, readdir() might have just
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	167 returned it twice because it was just renamed. What we should do is
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	168 create a completely new base name for it and rename() it to that.
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	169 If the call fails with ENOENT, it only means that it wasn't a
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	170 duplicate after all.
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	171 */
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	172
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	173 #include "lib.h"
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	174 #include "ioloop.h"
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	175 #include "buffer.h"
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	176 #include "hash.h"
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	177 #include "str.h"
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	178 #include "maildir-storage.h"
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	179 #include "maildir-uidlist.h"
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	180
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	181 #include <stdio.h>
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	182 #include <unistd.h>
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	183 #include <dirent.h>
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	184 #include <sys/stat.h>
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	185
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	186 #define MAILDIR_SYNC_SECS 1
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	187
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	188 #define MAILDIR_FILENAME_FLAG_FOUND 128
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	189
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	190 struct maildir_sync_context {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	191 struct index_mailbox *ibox;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	192 const char new_dir, cur_dir;
1954 2f6e137cdc44 Syncing optimizations. Timo Sirainen <tss@iki.fi> parents: 1947 diff changeset	193 int partial;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	194
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	195 struct maildir_uidlist_sync_ctx *uidlist_sync_ctx;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	196 };
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	197
1956 d6941cd8afdc Added support for setting dirty flags for messages (TODO: undirty..) Timo Sirainen <tss@iki.fi> parents: 1955 diff changeset	198 struct maildir_index_sync_context {
d6941cd8afdc Added support for setting dirty flags for messages (TODO: undirty..) Timo Sirainen <tss@iki.fi> parents: 1955 diff changeset	199 struct index_mailbox *ibox;
d6941cd8afdc Added support for setting dirty flags for messages (TODO: undirty..) Timo Sirainen <tss@iki.fi> parents: 1955 diff changeset	200 struct mail_index_view *view;
d6941cd8afdc Added support for setting dirty flags for messages (TODO: undirty..) Timo Sirainen <tss@iki.fi> parents: 1955 diff changeset	201 struct mail_index_sync_ctx *sync_ctx;
d6941cd8afdc Added support for setting dirty flags for messages (TODO: undirty..) Timo Sirainen <tss@iki.fi> parents: 1955 diff changeset	202
d6941cd8afdc Added support for setting dirty flags for messages (TODO: undirty..) Timo Sirainen <tss@iki.fi> parents: 1955 diff changeset	203 struct mail_index_sync_rec sync_rec;
d6941cd8afdc Added support for setting dirty flags for messages (TODO: undirty..) Timo Sirainen <tss@iki.fi> parents: 1955 diff changeset	204 uint32_t seq;
d6941cd8afdc Added support for setting dirty flags for messages (TODO: undirty..) Timo Sirainen <tss@iki.fi> parents: 1955 diff changeset	205 };
d6941cd8afdc Added support for setting dirty flags for messages (TODO: undirty..) Timo Sirainen <tss@iki.fi> parents: 1955 diff changeset	206
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	207 static int maildir_expunge(struct index_mailbox ibox, const char path,
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	208 void *context __attr_unused__)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	209 {
1955 0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	210 if (unlink(path) == 0) {
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	211 ibox->dirty_cur_time = ioloop_time;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	212 return 1;
1955 0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	213 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	214 if (errno == ENOENT)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	215 return 0;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	216
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	217 mail_storage_set_critical(ibox->box.storage,
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	218 "unlink(%s) failed: %m", path);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	219 return -1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	220 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	221
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	222 static int maildir_sync_flags(struct index_mailbox ibox, const char path,
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	223 void *context)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	224 {
1956 d6941cd8afdc Added support for setting dirty flags for messages (TODO: undirty..) Timo Sirainen <tss@iki.fi> parents: 1955 diff changeset	225 struct maildir_index_sync_context *ctx = context;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	226 const char *newpath;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	227 enum mail_flags flags;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	228 uint8_t flags8;
1958 4dec6a3d79fd s/custom flags/keywords/ Timo Sirainen <tss@iki.fi> parents: 1956 diff changeset	229 keywords_mask_t keywords;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	230
1958 4dec6a3d79fd s/custom flags/keywords/ Timo Sirainen <tss@iki.fi> parents: 1956 diff changeset	231 (void)maildir_filename_get_flags(path, &flags, keywords);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	232
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	233 flags8 = flags;
1958 4dec6a3d79fd s/custom flags/keywords/ Timo Sirainen <tss@iki.fi> parents: 1956 diff changeset	234 mail_index_sync_flags_apply(&ctx->sync_rec, &flags8, keywords);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	235
1958 4dec6a3d79fd s/custom flags/keywords/ Timo Sirainen <tss@iki.fi> parents: 1956 diff changeset	236 newpath = maildir_filename_set_flags(path, flags8, keywords);
1955 0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	237 if (rename(path, newpath) == 0) {
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	238 ibox->dirty_cur_time = ioloop_time;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	239 return 1;
1955 0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	240 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	241 if (errno == ENOENT)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	242 return 0;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	243
1956 d6941cd8afdc Added support for setting dirty flags for messages (TODO: undirty..) Timo Sirainen <tss@iki.fi> parents: 1955 diff changeset	244 if (ENOSPACE(errno)) {
d6941cd8afdc Added support for setting dirty flags for messages (TODO: undirty..) Timo Sirainen <tss@iki.fi> parents: 1955 diff changeset	245 if (mail_index_sync_set_dirty(ctx->sync_ctx, ctx->seq) < 0)
d6941cd8afdc Added support for setting dirty flags for messages (TODO: undirty..) Timo Sirainen <tss@iki.fi> parents: 1955 diff changeset	246 return -1;
d6941cd8afdc Added support for setting dirty flags for messages (TODO: undirty..) Timo Sirainen <tss@iki.fi> parents: 1955 diff changeset	247 return 1;
d6941cd8afdc Added support for setting dirty flags for messages (TODO: undirty..) Timo Sirainen <tss@iki.fi> parents: 1955 diff changeset	248 }
d6941cd8afdc Added support for setting dirty flags for messages (TODO: undirty..) Timo Sirainen <tss@iki.fi> parents: 1955 diff changeset	249
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	250 mail_storage_set_critical(ibox->box.storage,
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	251 "rename(%s, %s) failed: %m", path, newpath);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	252 return -1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	253 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	254
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	255 static int maildir_sync_record(struct index_mailbox *ibox,
1956 d6941cd8afdc Added support for setting dirty flags for messages (TODO: undirty..) Timo Sirainen <tss@iki.fi> parents: 1955 diff changeset	256 struct maildir_index_sync_context *ctx)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	257 {
1956 d6941cd8afdc Added support for setting dirty flags for messages (TODO: undirty..) Timo Sirainen <tss@iki.fi> parents: 1955 diff changeset	258 struct mail_index_sync_rec *sync_rec = &ctx->sync_rec;
d6941cd8afdc Added support for setting dirty flags for messages (TODO: undirty..) Timo Sirainen <tss@iki.fi> parents: 1955 diff changeset	259 struct mail_index_view *view = ctx->view;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	260 uint32_t seq, uid;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	261
1956 d6941cd8afdc Added support for setting dirty flags for messages (TODO: undirty..) Timo Sirainen <tss@iki.fi> parents: 1955 diff changeset	262 switch (sync_rec->type) {
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	263 case MAIL_INDEX_SYNC_TYPE_APPEND:
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	264 break;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	265 case MAIL_INDEX_SYNC_TYPE_EXPUNGE:
1956 d6941cd8afdc Added support for setting dirty flags for messages (TODO: undirty..) Timo Sirainen <tss@iki.fi> parents: 1955 diff changeset	266 for (seq = sync_rec->seq1; seq <= sync_rec->seq2; seq++) {
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	267 if (mail_index_lookup_uid(view, seq, &uid) < 0)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	268 return -1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	269 if (maildir_file_do(ibox, uid, maildir_expunge,
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	270 NULL) < 0)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	271 return -1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	272 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	273 break;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	274 case MAIL_INDEX_SYNC_TYPE_FLAGS:
1956 d6941cd8afdc Added support for setting dirty flags for messages (TODO: undirty..) Timo Sirainen <tss@iki.fi> parents: 1955 diff changeset	275 ctx->seq = sync_rec->seq1;
d6941cd8afdc Added support for setting dirty flags for messages (TODO: undirty..) Timo Sirainen <tss@iki.fi> parents: 1955 diff changeset	276 for (; ctx->seq <= sync_rec->seq2; ctx->seq++) {
d6941cd8afdc Added support for setting dirty flags for messages (TODO: undirty..) Timo Sirainen <tss@iki.fi> parents: 1955 diff changeset	277 if (mail_index_lookup_uid(view, ctx->seq, &uid) < 0)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	278 return -1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	279 if (maildir_file_do(ibox, uid, maildir_sync_flags,
1956 d6941cd8afdc Added support for setting dirty flags for messages (TODO: undirty..) Timo Sirainen <tss@iki.fi> parents: 1955 diff changeset	280 ctx) < 0)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	281 return -1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	282 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	283 break;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	284 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	285
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	286 return 0;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	287 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	288
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	289 int maildir_sync_last_commit(struct index_mailbox *ibox)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	290 {
1956 d6941cd8afdc Added support for setting dirty flags for messages (TODO: undirty..) Timo Sirainen <tss@iki.fi> parents: 1955 diff changeset	291 struct maildir_index_sync_context ctx;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	292 int ret;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	293
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	294 if (ibox->commit_log_file_seq == 0)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	295 return 0;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	296
1956 d6941cd8afdc Added support for setting dirty flags for messages (TODO: undirty..) Timo Sirainen <tss@iki.fi> parents: 1955 diff changeset	297 memset(&ctx, 0, sizeof(ctx));
d6941cd8afdc Added support for setting dirty flags for messages (TODO: undirty..) Timo Sirainen <tss@iki.fi> parents: 1955 diff changeset	298 ctx.ibox = ibox;
d6941cd8afdc Added support for setting dirty flags for messages (TODO: undirty..) Timo Sirainen <tss@iki.fi> parents: 1955 diff changeset	299
d6941cd8afdc Added support for setting dirty flags for messages (TODO: undirty..) Timo Sirainen <tss@iki.fi> parents: 1955 diff changeset	300 ret = mail_index_sync_begin(ibox->index, &ctx.sync_ctx, &ctx.view,
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	301 ibox->commit_log_file_seq,
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	302 ibox->commit_log_file_offset);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	303 if (ret > 0) {
1956 d6941cd8afdc Added support for setting dirty flags for messages (TODO: undirty..) Timo Sirainen <tss@iki.fi> parents: 1955 diff changeset	304 while ((ret = mail_index_sync_next(ctx.sync_ctx,
d6941cd8afdc Added support for setting dirty flags for messages (TODO: undirty..) Timo Sirainen <tss@iki.fi> parents: 1955 diff changeset	305 &ctx.sync_rec)) > 0) {
d6941cd8afdc Added support for setting dirty flags for messages (TODO: undirty..) Timo Sirainen <tss@iki.fi> parents: 1955 diff changeset	306 if (maildir_sync_record(ibox, &ctx) < 0) {
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	307 ret = -1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	308 break;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	309 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	310 }
1956 d6941cd8afdc Added support for setting dirty flags for messages (TODO: undirty..) Timo Sirainen <tss@iki.fi> parents: 1955 diff changeset	311 if (mail_index_sync_end(ctx.sync_ctx, 0, 0) < 0)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	312 ret = -1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	313 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	314
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	315 if (ret == 0) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	316 ibox->commit_log_file_seq = 0;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	317 ibox->commit_log_file_offset = 0;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	318 } else {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	319 mail_storage_set_index_error(ibox);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	320 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	321 return ret;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	322 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	323
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	324 static struct maildir_sync_context *
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	325 maildir_sync_context_new(struct index_mailbox *ibox)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	326 {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	327 struct maildir_sync_context *ctx;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	328
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	329 ctx = t_new(struct maildir_sync_context, 1);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	330 ctx->ibox = ibox;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	331 ctx->new_dir = t_strconcat(ibox->path, "/new", NULL);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	332 ctx->cur_dir = t_strconcat(ibox->path, "/cur", NULL);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	333 return ctx;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	334 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	335
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	336 static void maildir_sync_deinit(struct maildir_sync_context *ctx)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	337 {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	338 if (ctx->uidlist_sync_ctx != NULL)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	339 (void)maildir_uidlist_sync_deinit(ctx->uidlist_sync_ctx);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	340 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	341
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	342 static int maildir_fix_duplicate(struct index_mailbox ibox, const char dir,
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	343 const char *old_fname)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	344 {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	345 const char new_fname, old_path, *new_path;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	346 int ret = 0;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	347
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	348 t_push();
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	349
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	350 old_path = t_strconcat(dir, "/", old_fname, NULL);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	351 new_fname = maildir_generate_tmp_filename(&ioloop_timeval);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	352 new_path = t_strconcat(ibox->path, "/new/", new_fname, NULL);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	353
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	354 if (rename(old_path, new_path) == 0) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	355 i_warning("Fixed duplicate in %s: %s -> %s",
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	356 ibox->path, old_fname, new_fname);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	357 } else if (errno != ENOENT) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	358 mail_storage_set_critical(ibox->box.storage,
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	359 "rename(%s, %s) failed: %m", old_path, new_path);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	360 ret = -1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	361 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	362 t_pop();
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	363
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	364 return ret;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	365 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	366
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	367 static int maildir_scan_dir(struct maildir_sync_context *ctx, int new_dir)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	368 {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	369 struct mail_storage *storage = ctx->ibox->box.storage;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	370 const char *dir;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	371 DIR *dirp;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	372 string_t src, dest;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	373 struct dirent *dp;
1947 777da553d1d3 Recent-flag should work now Timo Sirainen <tss@iki.fi> parents: 1944 diff changeset	374 enum maildir_uidlist_rec_flag flags;
777da553d1d3 Recent-flag should work now Timo Sirainen <tss@iki.fi> parents: 1944 diff changeset	375 int move_new, ret = 1;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	376
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	377 src = t_str_new(1024);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	378 dest = t_str_new(1024);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	379
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	380 dir = new_dir ? ctx->new_dir : ctx->cur_dir;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	381 dirp = opendir(dir);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	382 if (dirp == NULL) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	383 mail_storage_set_critical(storage,
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	384 "opendir(%s) failed: %m", dir);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	385 return -1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	386 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	387
1947 777da553d1d3 Recent-flag should work now Timo Sirainen <tss@iki.fi> parents: 1944 diff changeset	388 move_new = new_dir && !mailbox_is_readonly(&ctx->ibox->box);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	389 while ((dp = readdir(dirp)) != NULL) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	390 if (dp->d_name[0] == '.')
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	391 continue;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	392
1978 6303ef092c5b mbox code compiles again, but syncing is only partially implemented so Timo Sirainen <tss@iki.fi> parents: 1970 diff changeset	393 ret = maildir_uidlist_sync_next_pre(ctx->uidlist_sync_ctx,
6303ef092c5b mbox code compiles again, but syncing is only partially implemented so Timo Sirainen <tss@iki.fi> parents: 1970 diff changeset	394 dp->d_name);
1984 9c159272f721 syncing fixes Timo Sirainen <tss@iki.fi> parents: 1978 diff changeset	395 if (ret == 0) {
9c159272f721 syncing fixes Timo Sirainen <tss@iki.fi> parents: 1978 diff changeset	396 if (new_dir)
9c159272f721 syncing fixes Timo Sirainen <tss@iki.fi> parents: 1978 diff changeset	397 ctx->ibox->last_new_mtime = 0;
9c159272f721 syncing fixes Timo Sirainen <tss@iki.fi> parents: 1978 diff changeset	398 else
9c159272f721 syncing fixes Timo Sirainen <tss@iki.fi> parents: 1978 diff changeset	399 ctx->ibox->dirty_cur_time = ioloop_time;
1978 6303ef092c5b mbox code compiles again, but syncing is only partially implemented so Timo Sirainen <tss@iki.fi> parents: 1970 diff changeset	400 continue;
1984 9c159272f721 syncing fixes Timo Sirainen <tss@iki.fi> parents: 1978 diff changeset	401 }
1978 6303ef092c5b mbox code compiles again, but syncing is only partially implemented so Timo Sirainen <tss@iki.fi> parents: 1970 diff changeset	402 if (ret < 0)
6303ef092c5b mbox code compiles again, but syncing is only partially implemented so Timo Sirainen <tss@iki.fi> parents: 1970 diff changeset	403 break;
6303ef092c5b mbox code compiles again, but syncing is only partially implemented so Timo Sirainen <tss@iki.fi> parents: 1970 diff changeset	404
1947 777da553d1d3 Recent-flag should work now Timo Sirainen <tss@iki.fi> parents: 1944 diff changeset	405 flags = 0;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	406 if (move_new) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	407 str_truncate(src, 0);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	408 str_truncate(dest, 0);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	409 str_printfa(src, "%s/%s", ctx->new_dir, dp->d_name);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	410 str_printfa(dest, "%s/%s", ctx->cur_dir, dp->d_name);
1970 0f4c4a043be1 When moving mails to cur/, we must add the ":2," part to filename. Timo Sirainen <tss@iki.fi> parents: 1962 diff changeset	411 if (strchr(dp->d_name, ':') == NULL)
0f4c4a043be1 When moving mails to cur/, we must add the ":2," part to filename. Timo Sirainen <tss@iki.fi> parents: 1962 diff changeset	412 str_append(dest, ":2,");
1947 777da553d1d3 Recent-flag should work now Timo Sirainen <tss@iki.fi> parents: 1944 diff changeset	413 if (rename(str_c(src), str_c(dest)) == 0) {
1984 9c159272f721 syncing fixes Timo Sirainen <tss@iki.fi> parents: 1978 diff changeset	414 /* we moved it - it's \Recent for us */
1955 0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	415 ctx->ibox->dirty_cur_time = ioloop_time;
1947 777da553d1d3 Recent-flag should work now Timo Sirainen <tss@iki.fi> parents: 1944 diff changeset	416 flags \|= MAILDIR_UIDLIST_REC_FLAG_MOVED \|
777da553d1d3 Recent-flag should work now Timo Sirainen <tss@iki.fi> parents: 1944 diff changeset	417 MAILDIR_UIDLIST_REC_FLAG_RECENT;
777da553d1d3 Recent-flag should work now Timo Sirainen <tss@iki.fi> parents: 1944 diff changeset	418 } else if (ENOTFOUND(errno)) {
777da553d1d3 Recent-flag should work now Timo Sirainen <tss@iki.fi> parents: 1944 diff changeset	419 /* someone else moved it already */
777da553d1d3 Recent-flag should work now Timo Sirainen <tss@iki.fi> parents: 1944 diff changeset	420 flags \|= MAILDIR_UIDLIST_REC_FLAG_MOVED;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	421 } else if (ENOSPACE(errno)) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	422 /* not enough disk space, leave here */
1947 777da553d1d3 Recent-flag should work now Timo Sirainen <tss@iki.fi> parents: 1944 diff changeset	423 flags \|= MAILDIR_UIDLIST_REC_FLAG_NEW_DIR \|
777da553d1d3 Recent-flag should work now Timo Sirainen <tss@iki.fi> parents: 1944 diff changeset	424 MAILDIR_UIDLIST_REC_FLAG_RECENT;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	425 move_new = FALSE;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	426 } else {
1947 777da553d1d3 Recent-flag should work now Timo Sirainen <tss@iki.fi> parents: 1944 diff changeset	427 flags \|= MAILDIR_UIDLIST_REC_FLAG_NEW_DIR \|
777da553d1d3 Recent-flag should work now Timo Sirainen <tss@iki.fi> parents: 1944 diff changeset	428 MAILDIR_UIDLIST_REC_FLAG_RECENT;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	429 mail_storage_set_critical(storage,
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	430 "rename(%s, %s) failed: %m",
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	431 str_c(src), str_c(dest));
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	432 }
1947 777da553d1d3 Recent-flag should work now Timo Sirainen <tss@iki.fi> parents: 1944 diff changeset	433 } else if (new_dir) {
777da553d1d3 Recent-flag should work now Timo Sirainen <tss@iki.fi> parents: 1944 diff changeset	434 flags \|= MAILDIR_UIDLIST_REC_FLAG_NEW_DIR \|
777da553d1d3 Recent-flag should work now Timo Sirainen <tss@iki.fi> parents: 1944 diff changeset	435 MAILDIR_UIDLIST_REC_FLAG_RECENT;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	436 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	437
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	438 ret = maildir_uidlist_sync_next(ctx->uidlist_sync_ctx,
1947 777da553d1d3 Recent-flag should work now Timo Sirainen <tss@iki.fi> parents: 1944 diff changeset	439 dp->d_name, flags);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	440 if (ret <= 0) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	441 if (ret < 0)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	442 break;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	443
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	444 /* possibly duplicate - try fixing it */
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	445 if (maildir_fix_duplicate(ctx->ibox,
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	446 dir, dp->d_name) < 0) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	447 ret = -1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	448 break;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	449 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	450 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	451 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	452
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	453 if (closedir(dirp) < 0) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	454 mail_storage_set_critical(storage,
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	455 "closedir(%s) failed: %m", dir);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	456 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	457 return ret < 0 ? -1 : 0;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	458 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	459
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	460 static int maildir_sync_quick_check(struct maildir_sync_context *ctx,
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	461 int new_changed_r, int cur_changed_r)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	462 {
1954 2f6e137cdc44 Syncing optimizations. Timo Sirainen <tss@iki.fi> parents: 1947 diff changeset	463 const struct mail_index_header *hdr;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	464 struct index_mailbox *ibox = ctx->ibox;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	465 struct stat st;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	466 time_t new_mtime, cur_mtime;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	467
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	468 new_changed_r = cur_changed_r = FALSE;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	469
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	470 if (stat(ctx->new_dir, &st) < 0) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	471 mail_storage_set_critical(ibox->box.storage,
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	472 "stat(%s) failed: %m", ctx->new_dir);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	473 return -1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	474 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	475 new_mtime = st.st_mtime;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	476
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	477 if (stat(ctx->cur_dir, &st) < 0) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	478 mail_storage_set_critical(ibox->box.storage,
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	479 "stat(%s) failed: %m", ctx->cur_dir);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	480 return -1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	481 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	482 cur_mtime = st.st_mtime;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	483
1954 2f6e137cdc44 Syncing optimizations. Timo Sirainen <tss@iki.fi> parents: 1947 diff changeset	484 if (ibox->last_cur_mtime == 0) {
2f6e137cdc44 Syncing optimizations. Timo Sirainen <tss@iki.fi> parents: 1947 diff changeset	485 /* first sync in this session, get cur stamp from index */
2f6e137cdc44 Syncing optimizations. Timo Sirainen <tss@iki.fi> parents: 1947 diff changeset	486 if (mail_index_get_header(ibox->view, &hdr) == 0)
2f6e137cdc44 Syncing optimizations. Timo Sirainen <tss@iki.fi> parents: 1947 diff changeset	487 ibox->last_cur_mtime = hdr->sync_stamp;
2f6e137cdc44 Syncing optimizations. Timo Sirainen <tss@iki.fi> parents: 1947 diff changeset	488 }
2f6e137cdc44 Syncing optimizations. Timo Sirainen <tss@iki.fi> parents: 1947 diff changeset	489
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	490 if (new_mtime != ibox->last_new_mtime \|\|
1955 0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	491 new_mtime >= ibox->last_new_sync_time - MAILDIR_SYNC_SECS) {
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	492 *new_changed_r = TRUE;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	493 ibox->last_new_mtime = new_mtime;
1955 0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	494 ibox->last_new_sync_time = ioloop_time;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	495 }
1954 2f6e137cdc44 Syncing optimizations. Timo Sirainen <tss@iki.fi> parents: 1947 diff changeset	496
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	497 if (cur_mtime != ibox->last_cur_mtime \|\|
1962 4775ffa73874 Syncing should be fixed now. Timo Sirainen <tss@iki.fi> parents: 1958 diff changeset	498 (ibox->dirty_cur_time != 0 &&
4775ffa73874 Syncing should be fixed now. Timo Sirainen <tss@iki.fi> parents: 1958 diff changeset	499 ioloop_time - ibox->dirty_cur_time > MAILDIR_SYNC_SECS)) {
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	500 /* cur/ changed, or delayed cur/ check */
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	501 *cur_changed_r = TRUE;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	502 ibox->last_cur_mtime = cur_mtime;
1984 9c159272f721 syncing fixes Timo Sirainen <tss@iki.fi> parents: 1978 diff changeset	503
9c159272f721 syncing fixes Timo Sirainen <tss@iki.fi> parents: 1978 diff changeset	504 ibox->dirty_cur_time =
9c159272f721 syncing fixes Timo Sirainen <tss@iki.fi> parents: 1978 diff changeset	505 cur_mtime >= ioloop_time - MAILDIR_SYNC_SECS ?
9c159272f721 syncing fixes Timo Sirainen <tss@iki.fi> parents: 1978 diff changeset	506 cur_mtime : 0;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	507 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	508
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	509 return 0;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	510 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	511
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	512 static int maildir_sync_index(struct maildir_sync_context *ctx)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	513 {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	514 struct index_mailbox *ibox = ctx->ibox;
1956 d6941cd8afdc Added support for setting dirty flags for messages (TODO: undirty..) Timo Sirainen <tss@iki.fi> parents: 1955 diff changeset	515 struct maildir_index_sync_context sync_ctx;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	516 struct maildir_uidlist_iter_ctx *iter;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	517 struct mail_index_transaction *trans;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	518 struct mail_index_view *view;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	519 const struct mail_index_header *hdr;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	520 const struct mail_index_record *rec;
1954 2f6e137cdc44 Syncing optimizations. Timo Sirainen <tss@iki.fi> parents: 1947 diff changeset	521 uint32_t seq, uid;
2f6e137cdc44 Syncing optimizations. Timo Sirainen <tss@iki.fi> parents: 1947 diff changeset	522 enum maildir_uidlist_rec_flag uflags;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	523 const char *filename;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	524 enum mail_flags flags;
1958 4dec6a3d79fd s/custom flags/keywords/ Timo Sirainen <tss@iki.fi> parents: 1956 diff changeset	525 keywords_mask_t keywords;
1954 2f6e137cdc44 Syncing optimizations. Timo Sirainen <tss@iki.fi> parents: 1947 diff changeset	526 uint32_t sync_stamp;
1917 68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	527 int ret;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	528
1956 d6941cd8afdc Added support for setting dirty flags for messages (TODO: undirty..) Timo Sirainen <tss@iki.fi> parents: 1955 diff changeset	529 memset(&sync_ctx, 0, sizeof(sync_ctx));
d6941cd8afdc Added support for setting dirty flags for messages (TODO: undirty..) Timo Sirainen <tss@iki.fi> parents: 1955 diff changeset	530 sync_ctx.ibox = ibox;
d6941cd8afdc Added support for setting dirty flags for messages (TODO: undirty..) Timo Sirainen <tss@iki.fi> parents: 1955 diff changeset	531
d6941cd8afdc Added support for setting dirty flags for messages (TODO: undirty..) Timo Sirainen <tss@iki.fi> parents: 1955 diff changeset	532 if (mail_index_sync_begin(ibox->index, &sync_ctx.sync_ctx, &view,
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	533 (uint32_t)-1, (uoff_t)-1) <= 0) {
1940 876bd2849a00 fixes Timo Sirainen <tss@iki.fi> parents: 1917 diff changeset	534 mail_storage_set_index_error(ibox);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	535 return -1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	536 }
1956 d6941cd8afdc Added support for setting dirty flags for messages (TODO: undirty..) Timo Sirainen <tss@iki.fi> parents: 1955 diff changeset	537 sync_ctx.view = view;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	538
1917 68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	539 ret = mail_index_get_header(view, &hdr);
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	540 i_assert(ret == 0); /* view is locked, can't happen */
68938dccbc45 Forced locking to be right with mprotect()ing index file. Support for Timo Sirainen <tss@iki.fi> parents: 1915 diff changeset	541
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	542 trans = mail_index_transaction_begin(view, FALSE);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	543
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	544 seq = 0;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	545 iter = maildir_uidlist_iter_init(ibox->uidlist);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	546 while (maildir_uidlist_iter_next(iter, &uid, &uflags, &filename)) {
1958 4dec6a3d79fd s/custom flags/keywords/ Timo Sirainen <tss@iki.fi> parents: 1956 diff changeset	547 maildir_filename_get_flags(filename, &flags, keywords);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	548
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	549 __again:
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	550 seq++;
1954 2f6e137cdc44 Syncing optimizations. Timo Sirainen <tss@iki.fi> parents: 1947 diff changeset	551 if ((uflags & MAILDIR_UIDLIST_REC_FLAG_NONSYNCED) != 0) {
2f6e137cdc44 Syncing optimizations. Timo Sirainen <tss@iki.fi> parents: 1947 diff changeset	552 /* partial syncing */
2f6e137cdc44 Syncing optimizations. Timo Sirainen <tss@iki.fi> parents: 1947 diff changeset	553 continue;
2f6e137cdc44 Syncing optimizations. Timo Sirainen <tss@iki.fi> parents: 1947 diff changeset	554 }
2f6e137cdc44 Syncing optimizations. Timo Sirainen <tss@iki.fi> parents: 1947 diff changeset	555
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	556 if (seq > hdr->messages_count) {
1984 9c159272f721 syncing fixes Timo Sirainen <tss@iki.fi> parents: 1978 diff changeset	557 if (uid < hdr->next_uid) {
9c159272f721 syncing fixes Timo Sirainen <tss@iki.fi> parents: 1978 diff changeset	558 /* message not in index, but next_uid header
9c159272f721 syncing fixes Timo Sirainen <tss@iki.fi> parents: 1978 diff changeset	559 is updated? shouldn't really happen.. */
9c159272f721 syncing fixes Timo Sirainen <tss@iki.fi> parents: 1978 diff changeset	560 mail_storage_set_critical(ibox->box.storage,
9c159272f721 syncing fixes Timo Sirainen <tss@iki.fi> parents: 1978 diff changeset	561 "Maildir sync: UID < next_uid "
9c159272f721 syncing fixes Timo Sirainen <tss@iki.fi> parents: 1978 diff changeset	562 "(%u < %u)", uid, hdr->next_uid);
9c159272f721 syncing fixes Timo Sirainen <tss@iki.fi> parents: 1978 diff changeset	563 mail_index_mark_corrupted(ibox->index);
9c159272f721 syncing fixes Timo Sirainen <tss@iki.fi> parents: 1978 diff changeset	564 ret = -1;
9c159272f721 syncing fixes Timo Sirainen <tss@iki.fi> parents: 1978 diff changeset	565 break;
9c159272f721 syncing fixes Timo Sirainen <tss@iki.fi> parents: 1978 diff changeset	566 }
9c159272f721 syncing fixes Timo Sirainen <tss@iki.fi> parents: 1978 diff changeset	567
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	568 mail_index_append(trans, uid, &seq);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	569 mail_index_update_flags(trans, seq, MODIFY_REPLACE,
1958 4dec6a3d79fd s/custom flags/keywords/ Timo Sirainen <tss@iki.fi> parents: 1956 diff changeset	570 flags, keywords);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	571 continue;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	572 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	573
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	574 if (mail_index_lookup(view, seq, &rec) < 0) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	575 mail_storage_set_index_error(ibox);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	576 ret = -1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	577 break;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	578 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	579
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	580 if (rec->uid < uid) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	581 /* expunged */
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	582 mail_index_expunge(trans, seq);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	583 goto __again;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	584 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	585
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	586 if (rec->uid > uid) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	587 /* new UID in the middle of the mailbox -
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	588 shouldn't happen */
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	589 mail_storage_set_critical(ibox->box.storage,
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	590 "Maildir sync: UID inserted in the middle "
1984 9c159272f721 syncing fixes Timo Sirainen <tss@iki.fi> parents: 1978 diff changeset	591 "of mailbox (%u > %u, file = %s)",
9c159272f721 syncing fixes Timo Sirainen <tss@iki.fi> parents: 1978 diff changeset	592 rec->uid, uid, filename);
1944 5a3c3b4d4a02 remove mail_index_reset() completely Timo Sirainen <tss@iki.fi> parents: 1940 diff changeset	593 mail_index_mark_corrupted(ibox->index);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	594 ret = -1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	595 break;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	596 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	597
1956 d6941cd8afdc Added support for setting dirty flags for messages (TODO: undirty..) Timo Sirainen <tss@iki.fi> parents: 1955 diff changeset	598 if ((rec->flags & MAIL_INDEX_MAIL_FLAG_DIRTY) != 0) {
d6941cd8afdc Added support for setting dirty flags for messages (TODO: undirty..) Timo Sirainen <tss@iki.fi> parents: 1955 diff changeset	599 /* we haven't been able to update maildir with this
d6941cd8afdc Added support for setting dirty flags for messages (TODO: undirty..) Timo Sirainen <tss@iki.fi> parents: 1955 diff changeset	600 record's flag changes. don't sync them. */
d6941cd8afdc Added support for setting dirty flags for messages (TODO: undirty..) Timo Sirainen <tss@iki.fi> parents: 1955 diff changeset	601 continue;
d6941cd8afdc Added support for setting dirty flags for messages (TODO: undirty..) Timo Sirainen <tss@iki.fi> parents: 1955 diff changeset	602 }
d6941cd8afdc Added support for setting dirty flags for messages (TODO: undirty..) Timo Sirainen <tss@iki.fi> parents: 1955 diff changeset	603
1958 4dec6a3d79fd s/custom flags/keywords/ Timo Sirainen <tss@iki.fi> parents: 1956 diff changeset	604 maildir_filename_get_flags(filename, &flags, keywords);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	605 if ((uint8_t)flags != (rec->flags & MAIL_FLAGS_MASK) \|\|
1958 4dec6a3d79fd s/custom flags/keywords/ Timo Sirainen <tss@iki.fi> parents: 1956 diff changeset	606 memcmp(keywords, rec->keywords,
4dec6a3d79fd s/custom flags/keywords/ Timo Sirainen <tss@iki.fi> parents: 1956 diff changeset	607 INDEX_KEYWORDS_BYTE_COUNT) != 0) {
1984 9c159272f721 syncing fixes Timo Sirainen <tss@iki.fi> parents: 1978 diff changeset	608 /* FIXME: this is wrong if there's syncs later.
9c159272f721 syncing fixes Timo Sirainen <tss@iki.fi> parents: 1978 diff changeset	609 it gets fixed in next sync however.. */
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	610 mail_index_update_flags(trans, seq, MODIFY_REPLACE,
1958 4dec6a3d79fd s/custom flags/keywords/ Timo Sirainen <tss@iki.fi> parents: 1956 diff changeset	611 flags, keywords);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	612 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	613 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	614 maildir_uidlist_iter_deinit(iter);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	615
1954 2f6e137cdc44 Syncing optimizations. Timo Sirainen <tss@iki.fi> parents: 1947 diff changeset	616 if (!ctx->partial) {
2f6e137cdc44 Syncing optimizations. Timo Sirainen <tss@iki.fi> parents: 1947 diff changeset	617 /* expunge the rest */
2f6e137cdc44 Syncing optimizations. Timo Sirainen <tss@iki.fi> parents: 1947 diff changeset	618 for (seq++; seq <= hdr->messages_count; seq++)
2f6e137cdc44 Syncing optimizations. Timo Sirainen <tss@iki.fi> parents: 1947 diff changeset	619 mail_index_expunge(trans, seq);
2f6e137cdc44 Syncing optimizations. Timo Sirainen <tss@iki.fi> parents: 1947 diff changeset	620 }
2f6e137cdc44 Syncing optimizations. Timo Sirainen <tss@iki.fi> parents: 1947 diff changeset	621
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	622 if (ret < 0)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	623 mail_index_transaction_rollback(trans);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	624 else {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	625 uint32_t seq;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	626 uoff_t offset;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	627
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	628 if (mail_index_transaction_commit(trans, &seq, &offset) < 0)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	629 mail_storage_set_index_error(ibox);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	630 else {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	631 ibox->commit_log_file_seq = seq;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	632 ibox->commit_log_file_offset = offset;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	633 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	634 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	635
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	636 /* now, sync the index */
1956 d6941cd8afdc Added support for setting dirty flags for messages (TODO: undirty..) Timo Sirainen <tss@iki.fi> parents: 1955 diff changeset	637 while ((ret = mail_index_sync_next(sync_ctx.sync_ctx,
d6941cd8afdc Added support for setting dirty flags for messages (TODO: undirty..) Timo Sirainen <tss@iki.fi> parents: 1955 diff changeset	638 &sync_ctx.sync_rec)) > 0) {
d6941cd8afdc Added support for setting dirty flags for messages (TODO: undirty..) Timo Sirainen <tss@iki.fi> parents: 1955 diff changeset	639 if (maildir_sync_record(ibox, &sync_ctx) < 0) {
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	640 ret = -1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	641 break;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	642 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	643 }
1954 2f6e137cdc44 Syncing optimizations. Timo Sirainen <tss@iki.fi> parents: 1947 diff changeset	644
1955 0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	645 sync_stamp = ibox->dirty_cur_time != 0 ? 0 : ibox->last_cur_mtime;
1956 d6941cd8afdc Added support for setting dirty flags for messages (TODO: undirty..) Timo Sirainen <tss@iki.fi> parents: 1955 diff changeset	646 if (mail_index_sync_end(sync_ctx.sync_ctx, sync_stamp, 0) < 0)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	647 ret = -1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	648
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	649 if (ret == 0) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	650 ibox->commit_log_file_seq = 0;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	651 ibox->commit_log_file_offset = 0;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	652 } else {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	653 mail_storage_set_index_error(ibox);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	654 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	655
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	656 return ret;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	657 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	658
1947 777da553d1d3 Recent-flag should work now Timo Sirainen <tss@iki.fi> parents: 1944 diff changeset	659 static int maildir_sync_context(struct maildir_sync_context *ctx)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	660 {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	661 int ret, new_changed, cur_changed;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	662
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	663 if (maildir_sync_quick_check(ctx, &new_changed, &cur_changed) < 0)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	664 return -1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	665
1947 777da553d1d3 Recent-flag should work now Timo Sirainen <tss@iki.fi> parents: 1944 diff changeset	666 if (!new_changed && !cur_changed)
777da553d1d3 Recent-flag should work now Timo Sirainen <tss@iki.fi> parents: 1944 diff changeset	667 return 0;
777da553d1d3 Recent-flag should work now Timo Sirainen <tss@iki.fi> parents: 1944 diff changeset	668
1954 2f6e137cdc44 Syncing optimizations. Timo Sirainen <tss@iki.fi> parents: 1947 diff changeset	669 ctx->partial = !cur_changed;
2f6e137cdc44 Syncing optimizations. Timo Sirainen <tss@iki.fi> parents: 1947 diff changeset	670 ctx->uidlist_sync_ctx =
2f6e137cdc44 Syncing optimizations. Timo Sirainen <tss@iki.fi> parents: 1947 diff changeset	671 maildir_uidlist_sync_init(ctx->ibox->uidlist, ctx->partial);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	672
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	673 if (maildir_scan_dir(ctx, TRUE) < 0)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	674 return -1;
1954 2f6e137cdc44 Syncing optimizations. Timo Sirainen <tss@iki.fi> parents: 1947 diff changeset	675 if (cur_changed) {
2f6e137cdc44 Syncing optimizations. Timo Sirainen <tss@iki.fi> parents: 1947 diff changeset	676 if (maildir_scan_dir(ctx, FALSE) < 0)
2f6e137cdc44 Syncing optimizations. Timo Sirainen <tss@iki.fi> parents: 1947 diff changeset	677 return -1;
2f6e137cdc44 Syncing optimizations. Timo Sirainen <tss@iki.fi> parents: 1947 diff changeset	678 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	679
1984 9c159272f721 syncing fixes Timo Sirainen <tss@iki.fi> parents: 1978 diff changeset	680 /* finish uidlist syncing, but keep it still locked */
9c159272f721 syncing fixes Timo Sirainen <tss@iki.fi> parents: 1978 diff changeset	681 maildir_uidlist_sync_finish(ctx->uidlist_sync_ctx);
9c159272f721 syncing fixes Timo Sirainen <tss@iki.fi> parents: 1978 diff changeset	682 if (maildir_sync_index(ctx) < 0)
9c159272f721 syncing fixes Timo Sirainen <tss@iki.fi> parents: 1978 diff changeset	683 return -1;
9c159272f721 syncing fixes Timo Sirainen <tss@iki.fi> parents: 1978 diff changeset	684
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	685 ret = maildir_uidlist_sync_deinit(ctx->uidlist_sync_ctx);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	686 ctx->uidlist_sync_ctx = NULL;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	687
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	688 return ret;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	689 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	690
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	691 static int maildir_sync_context_readonly(struct maildir_sync_context *ctx)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	692 {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	693 int ret;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	694
1954 2f6e137cdc44 Syncing optimizations. Timo Sirainen <tss@iki.fi> parents: 1947 diff changeset	695 ctx->uidlist_sync_ctx =
2f6e137cdc44 Syncing optimizations. Timo Sirainen <tss@iki.fi> parents: 1947 diff changeset	696 maildir_uidlist_sync_init(ctx->ibox->uidlist, FALSE);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	697
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	698 if (maildir_scan_dir(ctx, TRUE) < 0)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	699 return -1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	700 if (maildir_scan_dir(ctx, FALSE) < 0)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	701 return -1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	702
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	703 ret = maildir_uidlist_sync_deinit(ctx->uidlist_sync_ctx);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	704 ctx->uidlist_sync_ctx = NULL;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	705
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	706 return ret;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	707 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	708
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	709 int maildir_storage_sync_readonly(struct index_mailbox *ibox)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	710 {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	711 struct maildir_sync_context *ctx;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	712 int ret;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	713
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	714 ctx = maildir_sync_context_new(ibox);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	715 ret = maildir_sync_context_readonly(ctx);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	716 maildir_sync_deinit(ctx);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	717 return ret;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	718 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	719
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	720 int maildir_storage_sync(struct mailbox *box, enum mailbox_sync_flags flags)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	721 {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	722 struct index_mailbox ibox = (struct index_mailbox )box;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	723 struct maildir_sync_context *ctx;
1947 777da553d1d3 Recent-flag should work now Timo Sirainen <tss@iki.fi> parents: 1944 diff changeset	724 int ret;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	725
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	726 if ((flags & MAILBOX_SYNC_FLAG_FAST) == 0 \|\|
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	727 ibox->sync_last_check + MAILBOX_FULL_SYNC_INTERVAL <= ioloop_time) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	728 ibox->sync_last_check = ioloop_time;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	729
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	730 ctx = maildir_sync_context_new(ibox);
1947 777da553d1d3 Recent-flag should work now Timo Sirainen <tss@iki.fi> parents: 1944 diff changeset	731 ret = maildir_sync_context(ctx);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	732 maildir_sync_deinit(ctx);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	733
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	734 if (ret < 0)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	735 return -1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	736 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	737
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	738 return index_storage_sync(box, flags);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	739 }

Mercurial > dovecot > original-hg > dovecot-1.2

annotate src/lib-storage/index/maildir/maildir-sync.c @ 1984:9c159272f721 HEAD