dovecot/original-hg/dovecot-1.2: src/lib-storage/index/maildir/maildir-sync.c annotate

annotate src/lib-storage/index/maildir/maildir-sync.c @ 9202:797474444551 HEAD

OSX+Maildir: Also don't log an error if readdir() fails with EINVAL.

author	Timo Sirainen <tss@iki.fi>
date	Tue, 07 Jul 2009 21:43:57 -0400
parents	4e74f4651f5b
children	e1685886c795

rev	line source
8590 b9faf4db2a9f Updated copyright notices to include year 2009. Timo Sirainen <tss@iki.fi> parents: 8554 diff changeset	1 /* Copyright (c) 2004-2009 Dovecot authors, see the included COPYING file */
1955 0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	2
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	3 /*
1955 0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	4 Here's a description of how we handle Maildir synchronization and
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	5 it's problems:
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	6
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	7 We want to be as efficient as we can. The most efficient way to
3520 e2fe8222449d s/occured/occurred/ Timo Sirainen <tss@iki.fi> parents: 3472 diff changeset	8 check if changes have occurred is to stat() the new/ and cur/
1955 0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	9 directories and uidlist file - if their mtimes haven't changed,
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	10 there's no changes and we don't need to do anything.
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	11
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	12 Problem 1: Multiple changes can happen within a single second -
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	13 nothing guarantees that once we synced it, someone else didn't just
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	14 then make a modification. Such modifications wouldn't get noticed
3520 e2fe8222449d s/occured/occurred/ Timo Sirainen <tss@iki.fi> parents: 3472 diff changeset	15 until a new modification occurred later.
1955 0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	16
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	17 Problem 2: Syncing cur/ directory is much more costly than syncing
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	18 new/. Moving mails from new/ to cur/ will always change mtime of
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	19 cur/ causing us to sync it as well.
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	20
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	21 Problem 3: We may not be able to move mail from new/ to cur/
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	22 because we're out of quota, or simply because we're accessing a
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	23 read-only mailbox.
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	24
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	25
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	26 MAILDIR_SYNC_SECS
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	27 -----------------
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	28
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	29 Several checks below use MAILDIR_SYNC_SECS, which should be maximum
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	30 clock drift between all computers accessing the maildir (eg. via
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	31 NFS), rounded up to next second. Our default is 1 second, since
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	32 everyone should be using NTP.
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	33
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	34 Note that setting it to 0 works only if there's only one computer
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	35 accessing the maildir. It's practically impossible to make two
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	36 clocks _exactly_ synchronized.
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	37
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	38 It might be possible to only use file server's clock by looking at
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	39 the atime field, but I don't know how well that would actually work.
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	40
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	41 cur directory
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	42 -------------
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	43
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	44 We have dirty_cur_time variable which is set to cur/ directory's
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	45 mtime when it's >= time() - MAILDIR_SYNC_SECS and we _think_ we have
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	46 synchronized the directory.
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	47
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	48 When dirty_cur_time is non-zero, we don't synchronize the cur/
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	49 directory until
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	50
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	51 a) cur/'s mtime changes
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	52 b) opening a mail fails with ENOENT
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	53 c) time() > dirty_cur_time + MAILDIR_SYNC_SECS
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	54
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	55 This allows us to modify the maildir multiple times without having
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	56 to sync it at every change. The sync will eventually be done to
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	57 make sure we didn't miss any external changes.
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	58
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	59 The dirty_cur_time is set when:
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	60
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	61 - we change message flags
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	62 - we expunge messages
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	63 - we move mail from new/ to cur/
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	64 - we sync cur/ directory and it's mtime is >= time() - MAILDIR_SYNC_SECS
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	65
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	66 It's unset when we do the final syncing, ie. when mtime is
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	67 older than time() - MAILDIR_SYNC_SECS.
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	68
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	69 new directory
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	70 -------------
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	71
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	72 If new/'s mtime is >= time() - MAILDIR_SYNC_SECS, always synchronize
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	73 it. dirty_cur_time-like feature might save us a few syncs, but
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	74 that might break a client which saves a mail in one connection and
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	75 tries to fetch it in another one. new/ directory is almost always
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	76 empty, so syncing it should be very fast anyway. Actually this can
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	77 still happen if we sync only new/ dir while another client is also
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	78 moving mails from it to cur/ - it takes us a while to see them.
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	79 That's pretty unlikely to happen however, and only way to fix it
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	80 would be to always synchronize cur/ after new/.
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	81
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	82 Normally we move all mails from new/ to cur/ whenever we sync it. If
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	83 it's not possible for some reason, we mark the mail with "probably
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	84 exists in new/ directory" flag.
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	85
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	86 If rename() still fails because of ENOSPC or EDQUOT, we still save
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	87 the flag changes in index with dirty-flag on. When moving the mail
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	88 to cur/ directory, or when we notice it's already moved there, we
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	89 apply the flag changes to the filename, rename it and remove the
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	90 dirty flag. If there's dirty flags, this should be tried every time
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	91 after expunge or when closing the mailbox.
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	92
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	93 uidlist
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	94 -------
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	95
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	96 This file contains UID <-> filename mappings. It's updated only when
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	97 new mail arrives, so it may contain filenames that have already been
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	98 deleted. Updating is done by getting uidlist.lock file, writing the
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	99 whole uidlist into it and rename()ing it over the old uidlist. This
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	100 means there's no need to lock the file for reading.
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	101
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	102 Whenever uidlist is rewritten, it's mtime must be larger than the old
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	103 one's. Use utime() before rename() if needed. Note that inode checking
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	104 wouldn't have been sufficient as inode numbers can be reused.
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	105
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	106 This file is usually read the first time you need to know filename for
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	107 given UID. After that it's not re-read unless new mails come that we
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	108 don't know about.
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	109
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	110 broken clients
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	111 --------------
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	112
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	113 Originally the middle identifier in Maildir filename was specified
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	114 only as <process id>_<delivery counter>. That however created a
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	115 problem with randomized PIDs which made it possible that the same
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	116 PID was reused within one second.
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	117
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	118 So if within one second a mail was delivered, MUA moved it to cur/
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	119 and another mail was delivered by a new process using same PID as
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	120 the first one, we likely ended up overwriting the first mail when
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	121 the second mail was moved over it.
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	122
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	123 Nowadays everyone should be giving a bit more specific identifier,
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	124 for example include microseconds in it which Dovecot does.
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	125
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	126 There's a simple way to prevent this from happening in some cases:
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	127 Don't move the mail from new/ to cur/ if it's mtime is >= time() -
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	128 MAILDIR_SYNC_SECS. The second delivery's link() call then fails
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	129 because the file is already in new/, and it will then use a
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	130 different filename. There's a few problems with this however:
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	131
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	132 - it requires extra stat() call which is unneeded extra I/O
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	133 - another MUA might still move the mail to cur/
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	134 - if first file's flags are modified by either Dovecot or another
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	135 MUA, it's moved to cur/ (you _could_ just do the dirty-flagging
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	136 but that'd be ugly)
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	137
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	138 Because this is useful only for very few people and it requires
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	139 extra I/O, I decided not to implement this. It should be however
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	140 quite easy to do since we need to be able to deal with files in new/
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	141 in any case.
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	142
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	143 It's also possible to never accidentally overwrite a mail by using
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	144 link() + unlink() rather than rename(). This however isn't very
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	145 good idea as it introduces potential race conditions when multiple
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	146 clients are accessing the mailbox:
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	147
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	148 Trying to move the same mail from new/ to cur/ at the same time:
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	149
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	150 a) Client 1 uses slightly different filename than client 2,
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	151 for example one sets read-flag on but the other doesn't.
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	152 You have the same mail duplicated now.
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	153
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	154 b) Client 3 sees the mail between Client 1's and 2's link() calls
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	155 and changes it's flag. You have the same mail duplicated now.
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	156
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	157 And it gets worse when they're unlink()ing in cur/ directory:
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	158
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	159 c) Client 1 changes mails's flag and client 2 changes it back
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	160 between 1's link() and unlink(). The mail is now expunged.
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	161
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	162 d) If you try to deal with the duplicates by unlink()ing another
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	163 one of them, you might end up unlinking both of them.
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	164
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	165 So, what should we do then if we notice a duplicate? First of all,
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	166 it might not be a duplicate at all, readdir() might have just
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	167 returned it twice because it was just renamed. What we should do is
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	168 create a completely new base name for it and rename() it to that.
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	169 If the call fails with ENOENT, it only means that it wasn't a
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	170 duplicate after all.
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	171 */
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	172
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	173 #include "lib.h"
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	174 #include "ioloop.h"
3470 346a494c2feb Moved array declaration to array-decl.h and include it in lib.h. So array.h Timo Sirainen <tss@iki.fi> parents: 3453 diff changeset	175 #include "array.h"
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	176 #include "buffer.h"
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	177 #include "hash.h"
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	178 #include "str.h"
8847 43a49b3d3b5e Maildir: Another logging improvement to EACCES error. Timo Sirainen <tss@iki.fi> parents: 8772 diff changeset	179 #include "eacces-error.h"
5926 095b3adc537b Handle mailbox deletions while syncing silently. Timo Sirainen <tss@iki.fi> parents: 5920 diff changeset	180 #include "nfs-workarounds.h"
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	181 #include "maildir-storage.h"
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	182 #include "maildir-uidlist.h"
5899 f29b93c0519c Moved maildir filename related functions to maildir-filename.c Timo Sirainen <tss@iki.fi> parents: 5850 diff changeset	183 #include "maildir-filename.h"
4848 967de900c73a Mailbox list indexing and related changes. Currently works only with Timo Sirainen <tss@iki.fi> parents: 4774 diff changeset	184 #include "maildir-sync.h"
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	185
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	186 #include <stdio.h>
2050 ee1095ccfd23 Index header changes now go through transaction log. Removed the kludgy Timo Sirainen <tss@iki.fi> parents: 2039 diff changeset	187 #include <stddef.h>
5588 6c89106dee21 Keyword characters weren't sorted in the maildir filename. Timo Sirainen <tss@iki.fi> parents: 5582 diff changeset	188 #include <stdlib.h>
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	189 #include <unistd.h>
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	190 #include <dirent.h>
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	191 #include <sys/stat.h>
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	192
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	193 #define MAILDIR_FILENAME_FLAG_FOUND 128
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	194
3435 6f7ce690358e If we have rename()d more than 5 files from new/ to cur/, rescan the Timo Sirainen <tss@iki.fi> parents: 3417 diff changeset	195 /* When rename()ing many files from new/ to cur/, it's possible that next
6f7ce690358e If we have rename()d more than 5 files from new/ to cur/, rescan the Timo Sirainen <tss@iki.fi> parents: 3417 diff changeset	196 readdir() skips some files. we don't of course wish to lose them, so we
6f7ce690358e If we have rename()d more than 5 files from new/ to cur/, rescan the Timo Sirainen <tss@iki.fi> parents: 3417 diff changeset	197 go and rescan the new/ directory again from beginning until no files are
6f7ce690358e If we have rename()d more than 5 files from new/ to cur/, rescan the Timo Sirainen <tss@iki.fi> parents: 3417 diff changeset	198 left. This value is just an optimization to avoid checking the directory
6f7ce690358e If we have rename()d more than 5 files from new/ to cur/, rescan the Timo Sirainen <tss@iki.fi> parents: 3417 diff changeset	199 twice unneededly. usually only NFS is the problem case. 1 is the safest
6f7ce690358e If we have rename()d more than 5 files from new/ to cur/, rescan the Timo Sirainen <tss@iki.fi> parents: 3417 diff changeset	200 bet here, but I guess 5 will do just fine too. */
6f7ce690358e If we have rename()d more than 5 files from new/ to cur/, rescan the Timo Sirainen <tss@iki.fi> parents: 3417 diff changeset	201 #define MAILDIR_RENAME_RESCAN_COUNT 5
6f7ce690358e If we have rename()d more than 5 files from new/ to cur/, rescan the Timo Sirainen <tss@iki.fi> parents: 3417 diff changeset	202
5397 3a0964ac3a5c comment/duplicate link handling fixes. Timo Sirainen <tss@iki.fi> parents: 5392 diff changeset	203 /* This is mostly to avoid infinite looping when rename() destination already
3a0964ac3a5c comment/duplicate link handling fixes. Timo Sirainen <tss@iki.fi> parents: 5392 diff changeset	204 exists as the hard link of the file itself. */
5391 0c8705aad54c Avoid infinite looping when buggy filesystems. Timo Sirainen <tss@iki.fi> parents: 5390 diff changeset	205 #define MAILDIR_SCAN_DIR_MAX_COUNT 5
0c8705aad54c Avoid infinite looping when buggy filesystems. Timo Sirainen <tss@iki.fi> parents: 5390 diff changeset	206
5443 293bc7d1062f If we find duplicate hard links which haven't changed for 30 secs, unlink() Timo Sirainen <tss@iki.fi> parents: 5397 diff changeset	207 #define DUPE_LINKS_DELETE_SECS 30
293bc7d1062f If we find duplicate hard links which haven't changed for 30 secs, unlink() Timo Sirainen <tss@iki.fi> parents: 5397 diff changeset	208
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	209 struct maildir_sync_context {
3279 b698ae839a18 Moved mbox/maildir-specific variables from struct index_mailbox to Timo Sirainen <tss@iki.fi> parents: 3203 diff changeset	210 struct maildir_mailbox *mbox;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	211 const char new_dir, cur_dir;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	212
6881 b1554dea8ee8 If we're doing a fast sync, try only once to lock uidlist. Timo Sirainen <tss@iki.fi> parents: 6880 diff changeset	213 enum mailbox_sync_flags flags;
5368 7d45edb81fe4 When copying/syncing a lot of mails, send "* OK Hang in there" replies to Timo Sirainen <tss@iki.fi> parents: 5307 diff changeset	214 time_t last_touch, last_notify;
7d45edb81fe4 When copying/syncing a lot of mails, send "* OK Hang in there" replies to Timo Sirainen <tss@iki.fi> parents: 5307 diff changeset	215
2818 a758a5b542bb Always protect maildir syncing with uidlist lock. Before we only tried to Timo Sirainen <tss@iki.fi> parents: 2816 diff changeset	216 struct maildir_uidlist_sync_ctx *uidlist_sync_ctx;
6880 724c8f12eed2 maildir_storage_sync_force(): Don't wait on trying to lock uidlist. Just try Timo Sirainen <tss@iki.fi> parents: 6878 diff changeset	217 struct maildir_index_sync_context *index_sync_ctx;
724c8f12eed2 maildir_storage_sync_force(): Don't wait on trying to lock uidlist. Just try Timo Sirainen <tss@iki.fi> parents: 6878 diff changeset	218
724c8f12eed2 maildir_storage_sync_force(): Don't wait on trying to lock uidlist. Just try Timo Sirainen <tss@iki.fi> parents: 6878 diff changeset	219 unsigned int partial:1;
724c8f12eed2 maildir_storage_sync_force(): Don't wait on trying to lock uidlist. Just try Timo Sirainen <tss@iki.fi> parents: 6878 diff changeset	220 unsigned int locked:1;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	221 };
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	222
5920 00c5e3cbeaf0 Moved index syncing code to its own file. Timo Sirainen <tss@iki.fi> parents: 5918 diff changeset	223 void maildir_sync_notify(struct maildir_sync_context *ctx)
5368 7d45edb81fe4 When copying/syncing a lot of mails, send "* OK Hang in there" replies to Timo Sirainen <tss@iki.fi> parents: 5307 diff changeset	224 {
7d45edb81fe4 When copying/syncing a lot of mails, send "* OK Hang in there" replies to Timo Sirainen <tss@iki.fi> parents: 5307 diff changeset	225 time_t now;
7d45edb81fe4 When copying/syncing a lot of mails, send "* OK Hang in there" replies to Timo Sirainen <tss@iki.fi> parents: 5307 diff changeset	226
5390 4eeec560df01 If saving to maildir causes flag/expunge syncs, don't crash. Timo Sirainen <tss@iki.fi> parents: 5388 diff changeset	227 if (ctx == NULL) {
4eeec560df01 If saving to maildir causes flag/expunge syncs, don't crash. Timo Sirainen <tss@iki.fi> parents: 5388 diff changeset	228 /* we got here from maildir-save.c. it has no
4eeec560df01 If saving to maildir causes flag/expunge syncs, don't crash. Timo Sirainen <tss@iki.fi> parents: 5388 diff changeset	229 maildir_sync_context, */
4eeec560df01 If saving to maildir causes flag/expunge syncs, don't crash. Timo Sirainen <tss@iki.fi> parents: 5388 diff changeset	230 return;
4eeec560df01 If saving to maildir causes flag/expunge syncs, don't crash. Timo Sirainen <tss@iki.fi> parents: 5388 diff changeset	231 }
4eeec560df01 If saving to maildir causes flag/expunge syncs, don't crash. Timo Sirainen <tss@iki.fi> parents: 5388 diff changeset	232
5368 7d45edb81fe4 When copying/syncing a lot of mails, send "* OK Hang in there" replies to Timo Sirainen <tss@iki.fi> parents: 5307 diff changeset	233 now = time(NULL);
7046 e74a1d1dca07 When doing a forced sync, we're not necessarily locked. If it takes a while, Timo Sirainen <tss@iki.fi> parents: 6940 diff changeset	234 if (now - ctx->last_touch > MAILDIR_LOCK_TOUCH_SECS && ctx->locked) {
5368 7d45edb81fe4 When copying/syncing a lot of mails, send "* OK Hang in there" replies to Timo Sirainen <tss@iki.fi> parents: 5307 diff changeset	235 (void)maildir_uidlist_lock_touch(ctx->mbox->uidlist);
7d45edb81fe4 When copying/syncing a lot of mails, send "* OK Hang in there" replies to Timo Sirainen <tss@iki.fi> parents: 5307 diff changeset	236 ctx->last_touch = now;
7d45edb81fe4 When copying/syncing a lot of mails, send "* OK Hang in there" replies to Timo Sirainen <tss@iki.fi> parents: 5307 diff changeset	237 }
7d45edb81fe4 When copying/syncing a lot of mails, send "* OK Hang in there" replies to Timo Sirainen <tss@iki.fi> parents: 5307 diff changeset	238 if (now - ctx->last_notify > MAIL_STORAGE_STAYALIVE_SECS) {
7d45edb81fe4 When copying/syncing a lot of mails, send "* OK Hang in there" replies to Timo Sirainen <tss@iki.fi> parents: 5307 diff changeset	239 struct mailbox *box = &ctx->mbox->ibox.box;
7d45edb81fe4 When copying/syncing a lot of mails, send "* OK Hang in there" replies to Timo Sirainen <tss@iki.fi> parents: 5307 diff changeset	240
7d45edb81fe4 When copying/syncing a lot of mails, send "* OK Hang in there" replies to Timo Sirainen <tss@iki.fi> parents: 5307 diff changeset	241 if (box->storage->callbacks->notify_ok != NULL) {
7d45edb81fe4 When copying/syncing a lot of mails, send "* OK Hang in there" replies to Timo Sirainen <tss@iki.fi> parents: 5307 diff changeset	242 box->storage->callbacks->
7d45edb81fe4 When copying/syncing a lot of mails, send "* OK Hang in there" replies to Timo Sirainen <tss@iki.fi> parents: 5307 diff changeset	243 notify_ok(box, "Hang in there..",
7d45edb81fe4 When copying/syncing a lot of mails, send "* OK Hang in there" replies to Timo Sirainen <tss@iki.fi> parents: 5307 diff changeset	244 box->storage->callback_context);
7d45edb81fe4 When copying/syncing a lot of mails, send "* OK Hang in there" replies to Timo Sirainen <tss@iki.fi> parents: 5307 diff changeset	245 }
7d45edb81fe4 When copying/syncing a lot of mails, send "* OK Hang in there" replies to Timo Sirainen <tss@iki.fi> parents: 5307 diff changeset	246 ctx->last_notify = now;
7d45edb81fe4 When copying/syncing a lot of mails, send "* OK Hang in there" replies to Timo Sirainen <tss@iki.fi> parents: 5307 diff changeset	247 }
7d45edb81fe4 When copying/syncing a lot of mails, send "* OK Hang in there" replies to Timo Sirainen <tss@iki.fi> parents: 5307 diff changeset	248 }
7d45edb81fe4 When copying/syncing a lot of mails, send "* OK Hang in there" replies to Timo Sirainen <tss@iki.fi> parents: 5307 diff changeset	249
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	250 static struct maildir_sync_context *
6881 b1554dea8ee8 If we're doing a fast sync, try only once to lock uidlist. Timo Sirainen <tss@iki.fi> parents: 6880 diff changeset	251 maildir_sync_context_new(struct maildir_mailbox *mbox,
b1554dea8ee8 If we're doing a fast sync, try only once to lock uidlist. Timo Sirainen <tss@iki.fi> parents: 6880 diff changeset	252 enum mailbox_sync_flags flags)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	253 {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	254 struct maildir_sync_context *ctx;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	255
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	256 ctx = t_new(struct maildir_sync_context, 1);
3279 b698ae839a18 Moved mbox/maildir-specific variables from struct index_mailbox to Timo Sirainen <tss@iki.fi> parents: 3203 diff changeset	257 ctx->mbox = mbox;
b698ae839a18 Moved mbox/maildir-specific variables from struct index_mailbox to Timo Sirainen <tss@iki.fi> parents: 3203 diff changeset	258 ctx->new_dir = t_strconcat(mbox->path, "/new", NULL);
b698ae839a18 Moved mbox/maildir-specific variables from struct index_mailbox to Timo Sirainen <tss@iki.fi> parents: 3203 diff changeset	259 ctx->cur_dir = t_strconcat(mbox->path, "/cur", NULL);
5368 7d45edb81fe4 When copying/syncing a lot of mails, send "* OK Hang in there" replies to Timo Sirainen <tss@iki.fi> parents: 5307 diff changeset	260 ctx->last_touch = ioloop_time;
7d45edb81fe4 When copying/syncing a lot of mails, send "* OK Hang in there" replies to Timo Sirainen <tss@iki.fi> parents: 5307 diff changeset	261 ctx->last_notify = ioloop_time;
6881 b1554dea8ee8 If we're doing a fast sync, try only once to lock uidlist. Timo Sirainen <tss@iki.fi> parents: 6880 diff changeset	262 ctx->flags = flags;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	263 return ctx;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	264 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	265
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	266 static void maildir_sync_deinit(struct maildir_sync_context *ctx)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	267 {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	268 if (ctx->uidlist_sync_ctx != NULL)
9018 0bb192fe0abd Maildir: More fixes to uidlist handling. Timo Sirainen <tss@iki.fi> parents: 9017 diff changeset	269 (void)maildir_uidlist_sync_deinit(&ctx->uidlist_sync_ctx, FALSE);
4774 615b7738a62f Saving mails could have skipped over transactions, which caused different Timo Sirainen <tss@iki.fi> parents: 4612 diff changeset	270 if (ctx->index_sync_ctx != NULL) {
615b7738a62f Saving mails could have skipped over transactions, which caused different Timo Sirainen <tss@iki.fi> parents: 4612 diff changeset	271 (void)maildir_sync_index_finish(&ctx->index_sync_ctx,
615b7738a62f Saving mails could have skipped over transactions, which caused different Timo Sirainen <tss@iki.fi> parents: 4612 diff changeset	272 TRUE, FALSE);
615b7738a62f Saving mails could have skipped over transactions, which caused different Timo Sirainen <tss@iki.fi> parents: 4612 diff changeset	273 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	274 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	275
4397 5cbabd4ccd9c Don't go fixing duplicate maildir filenames without properly checking that Timo Sirainen <tss@iki.fi> parents: 4238 diff changeset	276 static int maildir_fix_duplicate(struct maildir_sync_context *ctx,
5910 289f828591f7 Code cleanup Timo Sirainen <tss@iki.fi> parents: 5904 diff changeset	277 const char dir, const char fname2)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	278 {
5910 289f828591f7 Code cleanup Timo Sirainen <tss@iki.fi> parents: 5904 diff changeset	279 const char fname1, path1, *path2;
289f828591f7 Code cleanup Timo Sirainen <tss@iki.fi> parents: 5904 diff changeset	280 const char new_fname, new_path;
289f828591f7 Code cleanup Timo Sirainen <tss@iki.fi> parents: 5904 diff changeset	281 struct stat st1, st2;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	282
5910 289f828591f7 Code cleanup Timo Sirainen <tss@iki.fi> parents: 5904 diff changeset	283 fname1 = maildir_uidlist_sync_get_full_filename(ctx->uidlist_sync_ctx,
289f828591f7 Code cleanup Timo Sirainen <tss@iki.fi> parents: 5904 diff changeset	284 fname2);
289f828591f7 Code cleanup Timo Sirainen <tss@iki.fi> parents: 5904 diff changeset	285 i_assert(fname1 != NULL);
4397 5cbabd4ccd9c Don't go fixing duplicate maildir filenames without properly checking that Timo Sirainen <tss@iki.fi> parents: 4238 diff changeset	286
5910 289f828591f7 Code cleanup Timo Sirainen <tss@iki.fi> parents: 5904 diff changeset	287 path1 = t_strconcat(dir, "/", fname1, NULL);
289f828591f7 Code cleanup Timo Sirainen <tss@iki.fi> parents: 5904 diff changeset	288 path2 = t_strconcat(dir, "/", fname2, NULL);
4397 5cbabd4ccd9c Don't go fixing duplicate maildir filenames without properly checking that Timo Sirainen <tss@iki.fi> parents: 4238 diff changeset	289
5910 289f828591f7 Code cleanup Timo Sirainen <tss@iki.fi> parents: 5904 diff changeset	290 if (stat(path1, &st1) < 0 \|\| stat(path2, &st2) < 0) {
4397 5cbabd4ccd9c Don't go fixing duplicate maildir filenames without properly checking that Timo Sirainen <tss@iki.fi> parents: 4238 diff changeset	291 /* most likely the files just don't exist anymore.
5cbabd4ccd9c Don't go fixing duplicate maildir filenames without properly checking that Timo Sirainen <tss@iki.fi> parents: 4238 diff changeset	292 don't really care about other errors much. */
5cbabd4ccd9c Don't go fixing duplicate maildir filenames without properly checking that Timo Sirainen <tss@iki.fi> parents: 4238 diff changeset	293 return 0;
5cbabd4ccd9c Don't go fixing duplicate maildir filenames without properly checking that Timo Sirainen <tss@iki.fi> parents: 4238 diff changeset	294 }
5910 289f828591f7 Code cleanup Timo Sirainen <tss@iki.fi> parents: 5904 diff changeset	295 if (st1.st_ino == st2.st_ino &&
289f828591f7 Code cleanup Timo Sirainen <tss@iki.fi> parents: 5904 diff changeset	296 CMP_DEV_T(st1.st_dev, st2.st_dev)) {
5397 3a0964ac3a5c comment/duplicate link handling fixes. Timo Sirainen <tss@iki.fi> parents: 5392 diff changeset	297 /* Files are the same. this means either a race condition
5443 293bc7d1062f If we find duplicate hard links which haven't changed for 30 secs, unlink() Timo Sirainen <tss@iki.fi> parents: 5397 diff changeset	298 between stat() calls, or that the files were link()ed. */
5910 289f828591f7 Code cleanup Timo Sirainen <tss@iki.fi> parents: 5904 diff changeset	299 if (st1.st_nlink > 1 && st2.st_nlink == st1.st_nlink &&
289f828591f7 Code cleanup Timo Sirainen <tss@iki.fi> parents: 5904 diff changeset	300 st1.st_ctime == st2.st_ctime &&
289f828591f7 Code cleanup Timo Sirainen <tss@iki.fi> parents: 5904 diff changeset	301 st1.st_ctime < ioloop_time - DUPE_LINKS_DELETE_SECS) {
5443 293bc7d1062f If we find duplicate hard links which haven't changed for 30 secs, unlink() Timo Sirainen <tss@iki.fi> parents: 5397 diff changeset	302 /* The file has hard links and it hasn't had any
293bc7d1062f If we find duplicate hard links which haven't changed for 30 secs, unlink() Timo Sirainen <tss@iki.fi> parents: 5397 diff changeset	303 changes (such as renames) for a while, so this
293bc7d1062f If we find duplicate hard links which haven't changed for 30 secs, unlink() Timo Sirainen <tss@iki.fi> parents: 5397 diff changeset	304 isn't a race condition.
5392 9a8402768b42 If duplicate links are found, rename() one over the other to get rid of Timo Sirainen <tss@iki.fi> parents: 5391 diff changeset	305
5443 293bc7d1062f If we find duplicate hard links which haven't changed for 30 secs, unlink() Timo Sirainen <tss@iki.fi> parents: 5397 diff changeset	306 rename()ing one file on top of the other would fix
293bc7d1062f If we find duplicate hard links which haven't changed for 30 secs, unlink() Timo Sirainen <tss@iki.fi> parents: 5397 diff changeset	307 this safely, except POSIX decided that rename()
293bc7d1062f If we find duplicate hard links which haven't changed for 30 secs, unlink() Timo Sirainen <tss@iki.fi> parents: 5397 diff changeset	308 doesn't work that way. So we'll have unlink() one
293bc7d1062f If we find duplicate hard links which haven't changed for 30 secs, unlink() Timo Sirainen <tss@iki.fi> parents: 5397 diff changeset	309 and hope that another process didn't just decide to
293bc7d1062f If we find duplicate hard links which haven't changed for 30 secs, unlink() Timo Sirainen <tss@iki.fi> parents: 5397 diff changeset	310 unlink() the other (uidlist lock prevents this from
293bc7d1062f If we find duplicate hard links which haven't changed for 30 secs, unlink() Timo Sirainen <tss@iki.fi> parents: 5397 diff changeset	311 happening) */
5910 289f828591f7 Code cleanup Timo Sirainen <tss@iki.fi> parents: 5904 diff changeset	312 if (unlink(path2) == 0)
289f828591f7 Code cleanup Timo Sirainen <tss@iki.fi> parents: 5904 diff changeset	313 i_warning("Unlinked a duplicate: %s", path2);
289f828591f7 Code cleanup Timo Sirainen <tss@iki.fi> parents: 5904 diff changeset	314 else {
5443 293bc7d1062f If we find duplicate hard links which haven't changed for 30 secs, unlink() Timo Sirainen <tss@iki.fi> parents: 5397 diff changeset	315 mail_storage_set_critical(
5910 289f828591f7 Code cleanup Timo Sirainen <tss@iki.fi> parents: 5904 diff changeset	316 &ctx->mbox->storage->storage,
289f828591f7 Code cleanup Timo Sirainen <tss@iki.fi> parents: 5904 diff changeset	317 "unlink(%s) failed: %m", path2);
5443 293bc7d1062f If we find duplicate hard links which haven't changed for 30 secs, unlink() Timo Sirainen <tss@iki.fi> parents: 5397 diff changeset	318 }
293bc7d1062f If we find duplicate hard links which haven't changed for 30 secs, unlink() Timo Sirainen <tss@iki.fi> parents: 5397 diff changeset	319 }
4397 5cbabd4ccd9c Don't go fixing duplicate maildir filenames without properly checking that Timo Sirainen <tss@iki.fi> parents: 4238 diff changeset	320 return 0;
5cbabd4ccd9c Don't go fixing duplicate maildir filenames without properly checking that Timo Sirainen <tss@iki.fi> parents: 4238 diff changeset	321 }
5cbabd4ccd9c Don't go fixing duplicate maildir filenames without properly checking that Timo Sirainen <tss@iki.fi> parents: 4238 diff changeset	322
5904 62ceb6b2b20d Renamed maildir_generate_tmp_filename() to maildir_filename_generate(). Also Timo Sirainen <tss@iki.fi> parents: 5901 diff changeset	323 new_fname = maildir_filename_generate();
5910 289f828591f7 Code cleanup Timo Sirainen <tss@iki.fi> parents: 5904 diff changeset	324 new_path = t_strconcat(ctx->mbox->path, "/new/", new_fname, NULL);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	325
5910 289f828591f7 Code cleanup Timo Sirainen <tss@iki.fi> parents: 5904 diff changeset	326 if (rename(path2, new_path) == 0)
289f828591f7 Code cleanup Timo Sirainen <tss@iki.fi> parents: 5904 diff changeset	327 i_warning("Fixed a duplicate: %s -> %s", path2, new_fname);
5392 9a8402768b42 If duplicate links are found, rename() one over the other to get rid of Timo Sirainen <tss@iki.fi> parents: 5391 diff changeset	328 else if (errno != ENOENT) {
5910 289f828591f7 Code cleanup Timo Sirainen <tss@iki.fi> parents: 5904 diff changeset	329 mail_storage_set_critical(&ctx->mbox->storage->storage,
5582 1c4fd25893bd Updated error message. Timo Sirainen <tss@iki.fi> parents: 5564 diff changeset	330 "Couldn't fix a duplicate: rename(%s, %s) failed: %m",
5910 289f828591f7 Code cleanup Timo Sirainen <tss@iki.fi> parents: 5904 diff changeset	331 path2, new_path);
6940 414c9d631a81 Replaced t_push/t_pop calls with T_FRAME() macros. Timo Sirainen <tss@iki.fi>* parents: 6933 diff changeset	332 return -1;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	333 }
6940 414c9d631a81 Replaced t_push/t_pop calls with T_FRAME() macros. Timo Sirainen <tss@iki.fi>* parents: 6933 diff changeset	334 return 0;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	335 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	336
5926 095b3adc537b Handle mailbox deletions while syncing silently. Timo Sirainen <tss@iki.fi> parents: 5920 diff changeset	337 static int
095b3adc537b Handle mailbox deletions while syncing silently. Timo Sirainen <tss@iki.fi> parents: 5920 diff changeset	338 maildir_stat(struct maildir_mailbox mbox, const char path, struct stat *st_r)
095b3adc537b Handle mailbox deletions while syncing silently. Timo Sirainen <tss@iki.fi> parents: 5920 diff changeset	339 {
6777 186b164a9579 Delay creating directories until we really need them. Timo Sirainen <tss@iki.fi> parents: 6771 diff changeset	340 int i;
186b164a9579 Delay creating directories until we really need them. Timo Sirainen <tss@iki.fi> parents: 6771 diff changeset	341
186b164a9579 Delay creating directories until we really need them. Timo Sirainen <tss@iki.fi> parents: 6771 diff changeset	342 for (i = 0;; i++) {
186b164a9579 Delay creating directories until we really need them. Timo Sirainen <tss@iki.fi> parents: 6771 diff changeset	343 if (nfs_safe_stat(path, st_r) == 0)
186b164a9579 Delay creating directories until we really need them. Timo Sirainen <tss@iki.fi> parents: 6771 diff changeset	344 return 0;
186b164a9579 Delay creating directories until we really need them. Timo Sirainen <tss@iki.fi> parents: 6771 diff changeset	345 if (errno != ENOENT \|\| i == MAILDIR_DELETE_RETRY_COUNT)
186b164a9579 Delay creating directories until we really need them. Timo Sirainen <tss@iki.fi> parents: 6771 diff changeset	346 break;
186b164a9579 Delay creating directories until we really need them. Timo Sirainen <tss@iki.fi> parents: 6771 diff changeset	347
8772 dfcb8a6a4f5f maildir: We didn't properly handle missing control directory at startup. Timo Sirainen <tss@iki.fi> parents: 8590 diff changeset	348 if (!maildir_set_deleted(mbox))
6777 186b164a9579 Delay creating directories until we really need them. Timo Sirainen <tss@iki.fi> parents: 6771 diff changeset	349 return -1;
186b164a9579 Delay creating directories until we really need them. Timo Sirainen <tss@iki.fi> parents: 6771 diff changeset	350 /* try again */
5926 095b3adc537b Handle mailbox deletions while syncing silently. Timo Sirainen <tss@iki.fi> parents: 5920 diff changeset	351 }
095b3adc537b Handle mailbox deletions while syncing silently. Timo Sirainen <tss@iki.fi> parents: 5920 diff changeset	352
095b3adc537b Handle mailbox deletions while syncing silently. Timo Sirainen <tss@iki.fi> parents: 5920 diff changeset	353 mail_storage_set_critical(mbox->ibox.box.storage,
095b3adc537b Handle mailbox deletions while syncing silently. Timo Sirainen <tss@iki.fi> parents: 5920 diff changeset	354 "stat(%s) failed: %m", path);
095b3adc537b Handle mailbox deletions while syncing silently. Timo Sirainen <tss@iki.fi> parents: 5920 diff changeset	355 return -1;
095b3adc537b Handle mailbox deletions while syncing silently. Timo Sirainen <tss@iki.fi> parents: 5920 diff changeset	356 }
095b3adc537b Handle mailbox deletions while syncing silently. Timo Sirainen <tss@iki.fi> parents: 5920 diff changeset	357
9201 4e74f4651f5b OSX+Maildir: If readdir() fails with EINVAL after doing rename()s, retry. Fixes HFS+. Timo Sirainen <tss@iki.fi> parents: 9067 diff changeset	358 static int
4e74f4651f5b OSX+Maildir: If readdir() fails with EINVAL after doing rename()s, retry. Fixes HFS+. Timo Sirainen <tss@iki.fi> parents: 9067 diff changeset	359 maildir_scan_dir(struct maildir_sync_context *ctx, bool new_dir, bool final)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	360 {
5459 78eaf595359c Removed struct index_storage abstraction. It's pointless. Timo Sirainen <tss@iki.fi> parents: 5443 diff changeset	361 struct mail_storage *storage = &ctx->mbox->storage->storage;
5927 b9865213da42 Store syncing information to maildir extension header instead of kludging Timo Sirainen <tss@iki.fi> parents: 5926 diff changeset	362 const char *path;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	363 DIR *dirp;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	364 string_t src, dest;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	365 struct dirent *dp;
5927 b9865213da42 Store syncing information to maildir extension header instead of kludging Timo Sirainen <tss@iki.fi> parents: 5926 diff changeset	366 struct stat st;
3435 6f7ce690358e If we have rename()d more than 5 files from new/ to cur/, rescan the Timo Sirainen <tss@iki.fi> parents: 3417 diff changeset	367 enum maildir_uidlist_rec_flag flags;
5914 ae731dbf3a6f Sync index changes while iterating through uidlist entries. This avoids Timo Sirainen <tss@iki.fi> parents: 5912 diff changeset	368 unsigned int i = 0, move_count = 0;
5927 b9865213da42 Store syncing information to maildir extension header instead of kludging Timo Sirainen <tss@iki.fi> parents: 5926 diff changeset	369 time_t now;
3863 55df57c028d4 Added "bool" type and changed all ints that were used as booleans to bool. Timo Sirainen <tss@iki.fi> parents: 3821 diff changeset	370 int ret = 1;
5927 b9865213da42 Store syncing information to maildir extension header instead of kludging Timo Sirainen <tss@iki.fi> parents: 5926 diff changeset	371 bool move_new, check_touch, dir_changed = FALSE;
5214 4e9d345df846 When syncing huge maildirs check once in a while if we need to update Timo Sirainen <tss@iki.fi> parents: 5080 diff changeset	372
5927 b9865213da42 Store syncing information to maildir extension header instead of kludging Timo Sirainen <tss@iki.fi> parents: 5926 diff changeset	373 path = new_dir ? ctx->new_dir : ctx->cur_dir;
6777 186b164a9579 Delay creating directories until we really need them. Timo Sirainen <tss@iki.fi> parents: 6771 diff changeset	374 for (i = 0;; i++) {
186b164a9579 Delay creating directories until we really need them. Timo Sirainen <tss@iki.fi> parents: 6771 diff changeset	375 dirp = opendir(path);
186b164a9579 Delay creating directories until we really need them. Timo Sirainen <tss@iki.fi> parents: 6771 diff changeset	376 if (dirp != NULL)
186b164a9579 Delay creating directories until we really need them. Timo Sirainen <tss@iki.fi> parents: 6771 diff changeset	377 break;
186b164a9579 Delay creating directories until we really need them. Timo Sirainen <tss@iki.fi> parents: 6771 diff changeset	378
186b164a9579 Delay creating directories until we really need them. Timo Sirainen <tss@iki.fi> parents: 6771 diff changeset	379 if (errno != ENOENT \|\| i == MAILDIR_DELETE_RETRY_COUNT) {
8847 43a49b3d3b5e Maildir: Another logging improvement to EACCES error. Timo Sirainen <tss@iki.fi> parents: 8772 diff changeset	380 if (errno == EACCES) {
43a49b3d3b5e Maildir: Another logging improvement to EACCES error. Timo Sirainen <tss@iki.fi> parents: 8772 diff changeset	381 mail_storage_set_critical(storage, "%s",
43a49b3d3b5e Maildir: Another logging improvement to EACCES error. Timo Sirainen <tss@iki.fi> parents: 8772 diff changeset	382 eacces_error_get("opendir", path));
43a49b3d3b5e Maildir: Another logging improvement to EACCES error. Timo Sirainen <tss@iki.fi> parents: 8772 diff changeset	383 } else {
43a49b3d3b5e Maildir: Another logging improvement to EACCES error. Timo Sirainen <tss@iki.fi> parents: 8772 diff changeset	384 mail_storage_set_critical(storage,
43a49b3d3b5e Maildir: Another logging improvement to EACCES error. Timo Sirainen <tss@iki.fi> parents: 8772 diff changeset	385 "opendir(%s) failed: %m", path);
43a49b3d3b5e Maildir: Another logging improvement to EACCES error. Timo Sirainen <tss@iki.fi> parents: 8772 diff changeset	386 }
5926 095b3adc537b Handle mailbox deletions while syncing silently. Timo Sirainen <tss@iki.fi> parents: 5920 diff changeset	387 return -1;
095b3adc537b Handle mailbox deletions while syncing silently. Timo Sirainen <tss@iki.fi> parents: 5920 diff changeset	388 }
6777 186b164a9579 Delay creating directories until we really need them. Timo Sirainen <tss@iki.fi> parents: 6771 diff changeset	389
8772 dfcb8a6a4f5f maildir: We didn't properly handle missing control directory at startup. Timo Sirainen <tss@iki.fi> parents: 8590 diff changeset	390 if (!maildir_set_deleted(ctx->mbox))
6777 186b164a9579 Delay creating directories until we really need them. Timo Sirainen <tss@iki.fi> parents: 6771 diff changeset	391 return -1;
186b164a9579 Delay creating directories until we really need them. Timo Sirainen <tss@iki.fi> parents: 6771 diff changeset	392 /* try again */
5927 b9865213da42 Store syncing information to maildir extension header instead of kludging Timo Sirainen <tss@iki.fi> parents: 5926 diff changeset	393 }
b9865213da42 Store syncing information to maildir extension header instead of kludging Timo Sirainen <tss@iki.fi> parents: 5926 diff changeset	394
b9865213da42 Store syncing information to maildir extension header instead of kludging Timo Sirainen <tss@iki.fi> parents: 5926 diff changeset	395 #ifdef HAVE_DIRFD
b9865213da42 Store syncing information to maildir extension header instead of kludging Timo Sirainen <tss@iki.fi> parents: 5926 diff changeset	396 if (fstat(dirfd(dirp), &st) < 0) {
b9865213da42 Store syncing information to maildir extension header instead of kludging Timo Sirainen <tss@iki.fi> parents: 5926 diff changeset	397 mail_storage_set_critical(storage,
b9865213da42 Store syncing information to maildir extension header instead of kludging Timo Sirainen <tss@iki.fi> parents: 5926 diff changeset	398 "fstat(%s) failed: %m", path);
b9865213da42 Store syncing information to maildir extension header instead of kludging Timo Sirainen <tss@iki.fi> parents: 5926 diff changeset	399 (void)closedir(dirp);
b9865213da42 Store syncing information to maildir extension header instead of kludging Timo Sirainen <tss@iki.fi> parents: 5926 diff changeset	400 return -1;
b9865213da42 Store syncing information to maildir extension header instead of kludging Timo Sirainen <tss@iki.fi> parents: 5926 diff changeset	401 }
b9865213da42 Store syncing information to maildir extension header instead of kludging Timo Sirainen <tss@iki.fi> parents: 5926 diff changeset	402 #else
b9865213da42 Store syncing information to maildir extension header instead of kludging Timo Sirainen <tss@iki.fi> parents: 5926 diff changeset	403 if (maildir_stat(ctx->mbox, path, &st) < 0) {
b9865213da42 Store syncing information to maildir extension header instead of kludging Timo Sirainen <tss@iki.fi> parents: 5926 diff changeset	404 (void)closedir(dirp);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	405 return -1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	406 }
5927 b9865213da42 Store syncing information to maildir extension header instead of kludging Timo Sirainen <tss@iki.fi> parents: 5926 diff changeset	407 #endif
b9865213da42 Store syncing information to maildir extension header instead of kludging Timo Sirainen <tss@iki.fi> parents: 5926 diff changeset	408
b9865213da42 Store syncing information to maildir extension header instead of kludging Timo Sirainen <tss@iki.fi> parents: 5926 diff changeset	409 now = time(NULL);
b9865213da42 Store syncing information to maildir extension header instead of kludging Timo Sirainen <tss@iki.fi> parents: 5926 diff changeset	410 if (new_dir) {
b9865213da42 Store syncing information to maildir extension header instead of kludging Timo Sirainen <tss@iki.fi> parents: 5926 diff changeset	411 ctx->mbox->maildir_hdr.new_check_time = now;
b9865213da42 Store syncing information to maildir extension header instead of kludging Timo Sirainen <tss@iki.fi> parents: 5926 diff changeset	412 ctx->mbox->maildir_hdr.new_mtime = st.st_mtime;
7526 a957567706ec Replaced HAVE_ST_NSEC macro checks with more generic ST_?TIME_NSEC() macros Timo Sirainen <tss@iki.fi> parents: 7515 diff changeset	413 ctx->mbox->maildir_hdr.new_mtime_nsecs = ST_MTIME_NSEC(st);
5927 b9865213da42 Store syncing information to maildir extension header instead of kludging Timo Sirainen <tss@iki.fi> parents: 5926 diff changeset	414 } else {
b9865213da42 Store syncing information to maildir extension header instead of kludging Timo Sirainen <tss@iki.fi> parents: 5926 diff changeset	415 ctx->mbox->maildir_hdr.cur_check_time = now;
b9865213da42 Store syncing information to maildir extension header instead of kludging Timo Sirainen <tss@iki.fi> parents: 5926 diff changeset	416 ctx->mbox->maildir_hdr.cur_mtime = st.st_mtime;
7526 a957567706ec Replaced HAVE_ST_NSEC macro checks with more generic ST_?TIME_NSEC() macros Timo Sirainen <tss@iki.fi> parents: 7515 diff changeset	417 ctx->mbox->maildir_hdr.cur_mtime_nsecs = ST_MTIME_NSEC(st);
5927 b9865213da42 Store syncing information to maildir extension header instead of kludging Timo Sirainen <tss@iki.fi> parents: 5926 diff changeset	418 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	419
2121 0840edf34f37 Locking fixes. use less memory Timo Sirainen <tss@iki.fi> parents: 2067 diff changeset	420 src = t_str_new(1024);
0840edf34f37 Locking fixes. use less memory Timo Sirainen <tss@iki.fi> parents: 2067 diff changeset	421 dest = t_str_new(1024);
0840edf34f37 Locking fixes. use less memory Timo Sirainen <tss@iki.fi> parents: 2067 diff changeset	422
3279 b698ae839a18 Moved mbox/maildir-specific variables from struct index_mailbox to Timo Sirainen <tss@iki.fi> parents: 3203 diff changeset	423 move_new = new_dir && !mailbox_is_readonly(&ctx->mbox->ibox.box) &&
6880 724c8f12eed2 maildir_storage_sync_force(): Don't wait on trying to lock uidlist. Just try Timo Sirainen <tss@iki.fi> parents: 6878 diff changeset	424 !ctx->mbox->ibox.keep_recent && ctx->locked;
5911 8bb5c7a69405 If readdir() or closedir() fails, return failure. Timo Sirainen <tss@iki.fi> parents: 5910 diff changeset	425
8bb5c7a69405 If readdir() or closedir() fails, return failure. Timo Sirainen <tss@iki.fi> parents: 5910 diff changeset	426 errno = 0;
8bb5c7a69405 If readdir() or closedir() fails, return failure. Timo Sirainen <tss@iki.fi> parents: 5910 diff changeset	427 for (; (dp = readdir(dirp)) != NULL; errno = 0) {
2258 087a43e29492 No maildir filename checking after all. Timo Sirainen <tss@iki.fi> parents: 2256 diff changeset	428 if (dp->d_name[0] == '.')
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	429 continue;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	430
5214 4e9d345df846 When syncing huge maildirs check once in a while if we need to update Timo Sirainen <tss@iki.fi> parents: 5080 diff changeset	431 check_touch = FALSE;
1947 777da553d1d3 Recent-flag should work now Timo Sirainen <tss@iki.fi> parents: 1944 diff changeset	432 flags = 0;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	433 if (move_new) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	434 str_truncate(src, 0);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	435 str_truncate(dest, 0);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	436 str_printfa(src, "%s/%s", ctx->new_dir, dp->d_name);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	437 str_printfa(dest, "%s/%s", ctx->cur_dir, dp->d_name);
3417 b0bdf32564b7 Replaced ':' and ',' character usages with #defines, so they can be changed Timo Sirainen <tss@iki.fi> parents: 3340 diff changeset	438 if (strchr(dp->d_name, MAILDIR_INFO_SEP) == NULL) {
b0bdf32564b7 Replaced ':' and ',' character usages with #defines, so they can be changed Timo Sirainen <tss@iki.fi> parents: 3340 diff changeset	439 str_append(dest, MAILDIR_FLAGS_FULL_SEP);
b0bdf32564b7 Replaced ':' and ',' character usages with #defines, so they can be changed Timo Sirainen <tss@iki.fi> parents: 3340 diff changeset	440 }
1947 777da553d1d3 Recent-flag should work now Timo Sirainen <tss@iki.fi> parents: 1944 diff changeset	441 if (rename(str_c(src), str_c(dest)) == 0) {
1984 9c159272f721 syncing fixes Timo Sirainen <tss@iki.fi> parents: 1978 diff changeset	442 /* we moved it - it's \Recent for us */
5927 b9865213da42 Store syncing information to maildir extension header instead of kludging Timo Sirainen <tss@iki.fi> parents: 5926 diff changeset	443 dir_changed = TRUE;
5914 ae731dbf3a6f Sync index changes while iterating through uidlist entries. This avoids Timo Sirainen <tss@iki.fi> parents: 5912 diff changeset	444 move_count++;
1947 777da553d1d3 Recent-flag should work now Timo Sirainen <tss@iki.fi> parents: 1944 diff changeset	445 flags \|= MAILDIR_UIDLIST_REC_FLAG_MOVED \|
777da553d1d3 Recent-flag should work now Timo Sirainen <tss@iki.fi> parents: 1944 diff changeset	446 MAILDIR_UIDLIST_REC_FLAG_RECENT;
777da553d1d3 Recent-flag should work now Timo Sirainen <tss@iki.fi> parents: 1944 diff changeset	447 } else if (ENOTFOUND(errno)) {
777da553d1d3 Recent-flag should work now Timo Sirainen <tss@iki.fi> parents: 1944 diff changeset	448 /* someone else moved it already */
5927 b9865213da42 Store syncing information to maildir extension header instead of kludging Timo Sirainen <tss@iki.fi> parents: 5926 diff changeset	449 dir_changed = TRUE;
5914 ae731dbf3a6f Sync index changes while iterating through uidlist entries. This avoids Timo Sirainen <tss@iki.fi> parents: 5912 diff changeset	450 move_count++;
6037 d911d943438e Recent flag handling rewrite. Still not perfect with maildir. Timo Sirainen <tss@iki.fi> parents: 6027 diff changeset	451 flags \|= MAILDIR_UIDLIST_REC_FLAG_MOVED \|
d911d943438e Recent flag handling rewrite. Still not perfect with maildir. Timo Sirainen <tss@iki.fi> parents: 6027 diff changeset	452 MAILDIR_UIDLIST_REC_FLAG_RECENT;
4199 fa135b615b01 If maildir is readonly, don't complain about rename() failing to move mails Timo Sirainen <tss@iki.fi> parents: 4152 diff changeset	453 } else if (ENOSPACE(errno) \|\| errno == EACCES) {
fa135b615b01 If maildir is readonly, don't complain about rename() failing to move mails Timo Sirainen <tss@iki.fi> parents: 4152 diff changeset	454 /* not enough disk space / read-only maildir,
fa135b615b01 If maildir is readonly, don't complain about rename() failing to move mails Timo Sirainen <tss@iki.fi> parents: 4152 diff changeset	455 leave here */
6037 d911d943438e Recent flag handling rewrite. Still not perfect with maildir. Timo Sirainen <tss@iki.fi> parents: 6027 diff changeset	456 flags \|= MAILDIR_UIDLIST_REC_FLAG_NEW_DIR;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	457 move_new = FALSE;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	458 } else {
6037 d911d943438e Recent flag handling rewrite. Still not perfect with maildir. Timo Sirainen <tss@iki.fi> parents: 6027 diff changeset	459 flags \|= MAILDIR_UIDLIST_REC_FLAG_NEW_DIR;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	460 mail_storage_set_critical(storage,
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	461 "rename(%s, %s) failed: %m",
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	462 str_c(src), str_c(dest));
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	463 }
5914 ae731dbf3a6f Sync index changes while iterating through uidlist entries. This avoids Timo Sirainen <tss@iki.fi> parents: 5912 diff changeset	464 if ((move_count % MAILDIR_SLOW_MOVE_COUNT) == 0)
ae731dbf3a6f Sync index changes while iterating through uidlist entries. This avoids Timo Sirainen <tss@iki.fi> parents: 5912 diff changeset	465 maildir_sync_notify(ctx);
1947 777da553d1d3 Recent-flag should work now Timo Sirainen <tss@iki.fi> parents: 1944 diff changeset	466 } else if (new_dir) {
777da553d1d3 Recent-flag should work now Timo Sirainen <tss@iki.fi> parents: 1944 diff changeset	467 flags \|= MAILDIR_UIDLIST_REC_FLAG_NEW_DIR \|
777da553d1d3 Recent-flag should work now Timo Sirainen <tss@iki.fi> parents: 1944 diff changeset	468 MAILDIR_UIDLIST_REC_FLAG_RECENT;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	469 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	470
5914 ae731dbf3a6f Sync index changes while iterating through uidlist entries. This avoids Timo Sirainen <tss@iki.fi> parents: 5912 diff changeset	471 i++;
ae731dbf3a6f Sync index changes while iterating through uidlist entries. This avoids Timo Sirainen <tss@iki.fi> parents: 5912 diff changeset	472 if ((i % MAILDIR_SLOW_CHECK_COUNT) == 0)
ae731dbf3a6f Sync index changes while iterating through uidlist entries. This avoids Timo Sirainen <tss@iki.fi> parents: 5912 diff changeset	473 maildir_sync_notify(ctx);
5214 4e9d345df846 When syncing huge maildirs check once in a while if we need to update Timo Sirainen <tss@iki.fi> parents: 5080 diff changeset	474
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	475 ret = maildir_uidlist_sync_next(ctx->uidlist_sync_ctx,
1947 777da553d1d3 Recent-flag should work now Timo Sirainen <tss@iki.fi> parents: 1944 diff changeset	476 dp->d_name, flags);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	477 if (ret <= 0) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	478 if (ret < 0)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	479 break;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	480
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	481 /* possibly duplicate - try fixing it */
7226 e6693a0ec8e1 Renamed T_FRAME_BEGIN/END to T_BEGIN/END. Removed T_FRAME() macro and Timo Sirainen <tss@iki.fi> parents: 7086 diff changeset	482 T_BEGIN {
6940 414c9d631a81 Replaced t_push/t_pop calls with T_FRAME() macros. Timo Sirainen <tss@iki.fi>* parents: 6933 diff changeset	483 ret = maildir_fix_duplicate(ctx, path,
414c9d631a81 Replaced t_push/t_pop calls with T_FRAME() macros. Timo Sirainen <tss@iki.fi>* parents: 6933 diff changeset	484 dp->d_name);
7226 e6693a0ec8e1 Renamed T_FRAME_BEGIN/END to T_BEGIN/END. Removed T_FRAME() macro and Timo Sirainen <tss@iki.fi> parents: 7086 diff changeset	485 } T_END;
6940 414c9d631a81 Replaced t_push/t_pop calls with T_FRAME() macros. Timo Sirainen <tss@iki.fi>* parents: 6933 diff changeset	486 if (ret < 0)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	487 break;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	488 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	489 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	490
9201 4e74f4651f5b OSX+Maildir: If readdir() fails with EINVAL after doing rename()s, retry. Fixes HFS+. Timo Sirainen <tss@iki.fi> parents: 9067 diff changeset	491 #ifdef __APPLE__
4e74f4651f5b OSX+Maildir: If readdir() fails with EINVAL after doing rename()s, retry. Fixes HFS+. Timo Sirainen <tss@iki.fi> parents: 9067 diff changeset	492 if (errno == EINVAL && move_count > 0 && !final) {
4e74f4651f5b OSX+Maildir: If readdir() fails with EINVAL after doing rename()s, retry. Fixes HFS+. Timo Sirainen <tss@iki.fi> parents: 9067 diff changeset	493 /* OS X HFS+: readdir() fails sometimes when rename()
4e74f4651f5b OSX+Maildir: If readdir() fails with EINVAL after doing rename()s, retry. Fixes HFS+. Timo Sirainen <tss@iki.fi> parents: 9067 diff changeset	494 have been done. */
4e74f4651f5b OSX+Maildir: If readdir() fails with EINVAL after doing rename()s, retry. Fixes HFS+. Timo Sirainen <tss@iki.fi> parents: 9067 diff changeset	495 move_count = MAILDIR_RENAME_RESCAN_COUNT + 1;
9202 797474444551 OSX+Maildir: Also don't log an error if readdir() fails with EINVAL. Timo Sirainen <tss@iki.fi> parents: 9201 diff changeset	496 } else
9201 4e74f4651f5b OSX+Maildir: If readdir() fails with EINVAL after doing rename()s, retry. Fixes HFS+. Timo Sirainen <tss@iki.fi> parents: 9067 diff changeset	497 #endif
4e74f4651f5b OSX+Maildir: If readdir() fails with EINVAL after doing rename()s, retry. Fixes HFS+. Timo Sirainen <tss@iki.fi> parents: 9067 diff changeset	498
5911 8bb5c7a69405 If readdir() or closedir() fails, return failure. Timo Sirainen <tss@iki.fi> parents: 5910 diff changeset	499 if (errno != 0) {
8bb5c7a69405 If readdir() or closedir() fails, return failure. Timo Sirainen <tss@iki.fi> parents: 5910 diff changeset	500 mail_storage_set_critical(storage,
5927 b9865213da42 Store syncing information to maildir extension header instead of kludging Timo Sirainen <tss@iki.fi> parents: 5926 diff changeset	501 "readdir(%s) failed: %m", path);
5911 8bb5c7a69405 If readdir() or closedir() fails, return failure. Timo Sirainen <tss@iki.fi> parents: 5910 diff changeset	502 ret = -1;
8bb5c7a69405 If readdir() or closedir() fails, return failure. Timo Sirainen <tss@iki.fi> parents: 5910 diff changeset	503 }
8bb5c7a69405 If readdir() or closedir() fails, return failure. Timo Sirainen <tss@iki.fi> parents: 5910 diff changeset	504
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	505 if (closedir(dirp) < 0) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	506 mail_storage_set_critical(storage,
5927 b9865213da42 Store syncing information to maildir extension header instead of kludging Timo Sirainen <tss@iki.fi> parents: 5926 diff changeset	507 "closedir(%s) failed: %m", path);
5911 8bb5c7a69405 If readdir() or closedir() fails, return failure. Timo Sirainen <tss@iki.fi> parents: 5910 diff changeset	508 ret = -1;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	509 }
2121 0840edf34f37 Locking fixes. use less memory Timo Sirainen <tss@iki.fi> parents: 2067 diff changeset	510
5927 b9865213da42 Store syncing information to maildir extension header instead of kludging Timo Sirainen <tss@iki.fi> parents: 5926 diff changeset	511 if (dir_changed) {
8876 88aeadb32151 Added maildir_very_dirty_syncs setting. Timo Sirainen <tss@iki.fi> parents: 8847 diff changeset	512 /* save the exact new times. the new mtimes should be >=
88aeadb32151 Added maildir_very_dirty_syncs setting. Timo Sirainen <tss@iki.fi> parents: 8847 diff changeset	513 "now", but just in case something weird happens and mtime
88aeadb32151 Added maildir_very_dirty_syncs setting. Timo Sirainen <tss@iki.fi> parents: 8847 diff changeset	514 doesn't update, use "now". */
88aeadb32151 Added maildir_very_dirty_syncs setting. Timo Sirainen <tss@iki.fi> parents: 8847 diff changeset	515 if (stat(ctx->new_dir, &st) == 0) {
88aeadb32151 Added maildir_very_dirty_syncs setting. Timo Sirainen <tss@iki.fi> parents: 8847 diff changeset	516 ctx->mbox->maildir_hdr.new_check_time =
88aeadb32151 Added maildir_very_dirty_syncs setting. Timo Sirainen <tss@iki.fi> parents: 8847 diff changeset	517 I_MAX(st.st_mtime, now);
88aeadb32151 Added maildir_very_dirty_syncs setting. Timo Sirainen <tss@iki.fi> parents: 8847 diff changeset	518 ctx->mbox->maildir_hdr.new_mtime = st.st_mtime;
88aeadb32151 Added maildir_very_dirty_syncs setting. Timo Sirainen <tss@iki.fi> parents: 8847 diff changeset	519 ctx->mbox->maildir_hdr.new_mtime_nsecs =
88aeadb32151 Added maildir_very_dirty_syncs setting. Timo Sirainen <tss@iki.fi> parents: 8847 diff changeset	520 ST_MTIME_NSEC(st);
88aeadb32151 Added maildir_very_dirty_syncs setting. Timo Sirainen <tss@iki.fi> parents: 8847 diff changeset	521 }
88aeadb32151 Added maildir_very_dirty_syncs setting. Timo Sirainen <tss@iki.fi> parents: 8847 diff changeset	522 if (stat(ctx->cur_dir, &st) == 0) {
88aeadb32151 Added maildir_very_dirty_syncs setting. Timo Sirainen <tss@iki.fi> parents: 8847 diff changeset	523 ctx->mbox->maildir_hdr.new_check_time =
88aeadb32151 Added maildir_very_dirty_syncs setting. Timo Sirainen <tss@iki.fi> parents: 8847 diff changeset	524 I_MAX(st.st_mtime, now);
88aeadb32151 Added maildir_very_dirty_syncs setting. Timo Sirainen <tss@iki.fi> parents: 8847 diff changeset	525 ctx->mbox->maildir_hdr.cur_mtime = st.st_mtime;
88aeadb32151 Added maildir_very_dirty_syncs setting. Timo Sirainen <tss@iki.fi> parents: 8847 diff changeset	526 ctx->mbox->maildir_hdr.cur_mtime_nsecs =
88aeadb32151 Added maildir_very_dirty_syncs setting. Timo Sirainen <tss@iki.fi> parents: 8847 diff changeset	527 ST_MTIME_NSEC(st);
88aeadb32151 Added maildir_very_dirty_syncs setting. Timo Sirainen <tss@iki.fi> parents: 8847 diff changeset	528 }
5927 b9865213da42 Store syncing information to maildir extension header instead of kludging Timo Sirainen <tss@iki.fi> parents: 5926 diff changeset	529 }
b9865213da42 Store syncing information to maildir extension header instead of kludging Timo Sirainen <tss@iki.fi> parents: 5926 diff changeset	530
5368 7d45edb81fe4 When copying/syncing a lot of mails, send "* OK Hang in there" replies to Timo Sirainen <tss@iki.fi> parents: 5307 diff changeset	531 return ret < 0 ? -1 :
9201 4e74f4651f5b OSX+Maildir: If readdir() fails with EINVAL after doing rename()s, retry. Fixes HFS+. Timo Sirainen <tss@iki.fi> parents: 9067 diff changeset	532 (move_count <= MAILDIR_RENAME_RESCAN_COUNT \|\| final ? 0 : 1);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	533 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	534
7515 c14a2b0a3126 Keep track of dovecot-uidlist mtime+size in index file. If it's up-to-date Timo Sirainen <tss@iki.fi> parents: 7506 diff changeset	535 int maildir_sync_header_refresh(struct maildir_mailbox *mbox)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	536 {
5927 b9865213da42 Store syncing information to maildir extension header instead of kludging Timo Sirainen <tss@iki.fi> parents: 5926 diff changeset	537 const void *data;
b9865213da42 Store syncing information to maildir extension header instead of kludging Timo Sirainen <tss@iki.fi> parents: 5926 diff changeset	538 size_t data_size;
4848 967de900c73a Mailbox list indexing and related changes. Currently works only with Timo Sirainen <tss@iki.fi> parents: 4774 diff changeset	539
5927 b9865213da42 Store syncing information to maildir extension header instead of kludging Timo Sirainen <tss@iki.fi> parents: 5926 diff changeset	540 if (mail_index_refresh(mbox->ibox.index) < 0) {
b9865213da42 Store syncing information to maildir extension header instead of kludging Timo Sirainen <tss@iki.fi> parents: 5926 diff changeset	541 mail_storage_set_index_error(&mbox->ibox);
b9865213da42 Store syncing information to maildir extension header instead of kludging Timo Sirainen <tss@iki.fi> parents: 5926 diff changeset	542 return -1;
b9865213da42 Store syncing information to maildir extension header instead of kludging Timo Sirainen <tss@iki.fi> parents: 5926 diff changeset	543 }
3472 db29cc6754d5 Store new/ directory's timestamp in sync_size header in index (kludgy..). Timo Sirainen <tss@iki.fi> parents: 3470 diff changeset	544
6275 913b188f4dd4 Removed explicit locking from views and maps. They were already locked all Timo Sirainen <tss@iki.fi> parents: 6037 diff changeset	545 mail_index_get_header_ext(mbox->ibox.view, mbox->maildir_ext_id,
913b188f4dd4 Removed explicit locking from views and maps. They were already locked all Timo Sirainen <tss@iki.fi> parents: 6037 diff changeset	546 &data, &data_size);
5927 b9865213da42 Store syncing information to maildir extension header instead of kludging Timo Sirainen <tss@iki.fi> parents: 5926 diff changeset	547 if (data_size == 0) {
b9865213da42 Store syncing information to maildir extension header instead of kludging Timo Sirainen <tss@iki.fi> parents: 5926 diff changeset	548 /* doesn't exist */
b9865213da42 Store syncing information to maildir extension header instead of kludging Timo Sirainen <tss@iki.fi> parents: 5926 diff changeset	549 return 0;
b9865213da42 Store syncing information to maildir extension header instead of kludging Timo Sirainen <tss@iki.fi> parents: 5926 diff changeset	550 }
4848 967de900c73a Mailbox list indexing and related changes. Currently works only with Timo Sirainen <tss@iki.fi> parents: 4774 diff changeset	551
7515 c14a2b0a3126 Keep track of dovecot-uidlist mtime+size in index file. If it's up-to-date Timo Sirainen <tss@iki.fi> parents: 7506 diff changeset	552 memcpy(&mbox->maildir_hdr, data,
c14a2b0a3126 Keep track of dovecot-uidlist mtime+size in index file. If it's up-to-date Timo Sirainen <tss@iki.fi> parents: 7506 diff changeset	553 I_MIN(sizeof(mbox->maildir_hdr), data_size));
5927 b9865213da42 Store syncing information to maildir extension header instead of kludging Timo Sirainen <tss@iki.fi> parents: 5926 diff changeset	554 return 0;
3472 db29cc6754d5 Store new/ directory's timestamp in sync_size header in index (kludgy..). Timo Sirainen <tss@iki.fi> parents: 3470 diff changeset	555 }
db29cc6754d5 Store new/ directory's timestamp in sync_size header in index (kludgy..). Timo Sirainen <tss@iki.fi> parents: 3470 diff changeset	556
7066 048325df1ec4 If MAILBOX_SYNC_FLAG_FULL_READ flag is set (SELECT and CHECK IMAP commands), Timo Sirainen <tss@iki.fi> parents: 7046 diff changeset	557 static int maildir_sync_quick_check(struct maildir_mailbox *mbox, bool undirty,
5927 b9865213da42 Store syncing information to maildir extension header instead of kludging Timo Sirainen <tss@iki.fi> parents: 5926 diff changeset	558 const char new_dir, const char cur_dir,
b9865213da42 Store syncing information to maildir extension header instead of kludging Timo Sirainen <tss@iki.fi> parents: 5926 diff changeset	559 bool new_changed_r, bool cur_changed_r)
3472 db29cc6754d5 Store new/ directory's timestamp in sync_size header in index (kludgy..). Timo Sirainen <tss@iki.fi> parents: 3470 diff changeset	560 {
5927 b9865213da42 Store syncing information to maildir extension header instead of kludging Timo Sirainen <tss@iki.fi> parents: 5926 diff changeset	561 #define DIR_DELAYED_REFRESH(hdr, name) \
b9865213da42 Store syncing information to maildir extension header instead of kludging Timo Sirainen <tss@iki.fi> parents: 5926 diff changeset	562 ((hdr)->name ## _check_time <= \
b9865213da42 Store syncing information to maildir extension header instead of kludging Timo Sirainen <tss@iki.fi> parents: 5926 diff changeset	563 (hdr)->name ## _mtime + MAILDIR_SYNC_SECS && \
7066 048325df1ec4 If MAILBOX_SYNC_FLAG_FULL_READ flag is set (SELECT and CHECK IMAP commands), Timo Sirainen <tss@iki.fi> parents: 7046 diff changeset	564 (undirty \|\| \
048325df1ec4 If MAILBOX_SYNC_FLAG_FULL_READ flag is set (SELECT and CHECK IMAP commands), Timo Sirainen <tss@iki.fi> parents: 7046 diff changeset	565 (time_t)(hdr)->name ## _check_time < ioloop_time - MAILDIR_SYNC_SECS))
5927 b9865213da42 Store syncing information to maildir extension header instead of kludging Timo Sirainen <tss@iki.fi> parents: 5926 diff changeset	566
b9865213da42 Store syncing information to maildir extension header instead of kludging Timo Sirainen <tss@iki.fi> parents: 5926 diff changeset	567 #define DIR_MTIME_CHANGED(st, hdr, name) \
5983 272796cfee66 compiler warning fixes Timo Sirainen <tss@iki.fi> parents: 5950 diff changeset	568 ((st).st_mtime != (time_t)(hdr)->name ## _mtime \|\| \
7526 a957567706ec Replaced HAVE_ST_NSEC macro checks with more generic ST_?TIME_NSEC() macros Timo Sirainen <tss@iki.fi> parents: 7515 diff changeset	569 !ST_NTIMES_EQUAL(ST_MTIME_NSEC(st), (hdr)->name ## _mtime_nsecs))
5927 b9865213da42 Store syncing information to maildir extension header instead of kludging Timo Sirainen <tss@iki.fi> parents: 5926 diff changeset	570
b9865213da42 Store syncing information to maildir extension header instead of kludging Timo Sirainen <tss@iki.fi> parents: 5926 diff changeset	571 struct maildir_index_header *hdr = &mbox->maildir_hdr;
b9865213da42 Store syncing information to maildir extension header instead of kludging Timo Sirainen <tss@iki.fi> parents: 5926 diff changeset	572 struct stat new_st, cur_st;
b9865213da42 Store syncing information to maildir extension header instead of kludging Timo Sirainen <tss@iki.fi> parents: 5926 diff changeset	573 bool refreshed = FALSE, check_new = FALSE, check_cur = FALSE;
b9865213da42 Store syncing information to maildir extension header instead of kludging Timo Sirainen <tss@iki.fi> parents: 5926 diff changeset	574
b9865213da42 Store syncing information to maildir extension header instead of kludging Timo Sirainen <tss@iki.fi> parents: 5926 diff changeset	575 if (mbox->maildir_hdr.new_mtime == 0) {
7515 c14a2b0a3126 Keep track of dovecot-uidlist mtime+size in index file. If it's up-to-date Timo Sirainen <tss@iki.fi> parents: 7506 diff changeset	576 if (maildir_sync_header_refresh(mbox) < 0)
6328 29d249982282 Error handling fixes. Timo Sirainen <tss@iki.fi> parents: 6275 diff changeset	577 return -1;
5927 b9865213da42 Store syncing information to maildir extension header instead of kludging Timo Sirainen <tss@iki.fi> parents: 5926 diff changeset	578 if (mbox->maildir_hdr.new_mtime == 0) {
b9865213da42 Store syncing information to maildir extension header instead of kludging Timo Sirainen <tss@iki.fi> parents: 5926 diff changeset	579 /* first sync */
b9865213da42 Store syncing information to maildir extension header instead of kludging Timo Sirainen <tss@iki.fi> parents: 5926 diff changeset	580 new_changed_r = cur_changed_r = TRUE;
b9865213da42 Store syncing information to maildir extension header instead of kludging Timo Sirainen <tss@iki.fi> parents: 5926 diff changeset	581 return 0;
b9865213da42 Store syncing information to maildir extension header instead of kludging Timo Sirainen <tss@iki.fi> parents: 5926 diff changeset	582 }
b9865213da42 Store syncing information to maildir extension header instead of kludging Timo Sirainen <tss@iki.fi> parents: 5926 diff changeset	583 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	584
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	585 new_changed_r = cur_changed_r = FALSE;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	586
5927 b9865213da42 Store syncing information to maildir extension header instead of kludging Timo Sirainen <tss@iki.fi> parents: 5926 diff changeset	587 /* try to avoid stat()ing by first checking delayed changes */
b9865213da42 Store syncing information to maildir extension header instead of kludging Timo Sirainen <tss@iki.fi> parents: 5926 diff changeset	588 if (DIR_DELAYED_REFRESH(hdr, new) \|\|
8876 88aeadb32151 Added maildir_very_dirty_syncs setting. Timo Sirainen <tss@iki.fi> parents: 8847 diff changeset	589 (DIR_DELAYED_REFRESH(hdr, cur) && !mbox->very_dirty_syncs)) {
5927 b9865213da42 Store syncing information to maildir extension header instead of kludging Timo Sirainen <tss@iki.fi> parents: 5926 diff changeset	590 /* refresh index and try again */
7515 c14a2b0a3126 Keep track of dovecot-uidlist mtime+size in index file. If it's up-to-date Timo Sirainen <tss@iki.fi> parents: 7506 diff changeset	591 if (maildir_sync_header_refresh(mbox) < 0)
6328 29d249982282 Error handling fixes. Timo Sirainen <tss@iki.fi> parents: 6275 diff changeset	592 return -1;
5927 b9865213da42 Store syncing information to maildir extension header instead of kludging Timo Sirainen <tss@iki.fi> parents: 5926 diff changeset	593 refreshed = TRUE;
3472 db29cc6754d5 Store new/ directory's timestamp in sync_size header in index (kludgy..). Timo Sirainen <tss@iki.fi> parents: 3470 diff changeset	594
5927 b9865213da42 Store syncing information to maildir extension header instead of kludging Timo Sirainen <tss@iki.fi> parents: 5926 diff changeset	595 if (DIR_DELAYED_REFRESH(hdr, new))
b9865213da42 Store syncing information to maildir extension header instead of kludging Timo Sirainen <tss@iki.fi> parents: 5926 diff changeset	596 *new_changed_r = TRUE;
8876 88aeadb32151 Added maildir_very_dirty_syncs setting. Timo Sirainen <tss@iki.fi> parents: 8847 diff changeset	597 if (DIR_DELAYED_REFRESH(hdr, cur) && !mbox->very_dirty_syncs)
5927 b9865213da42 Store syncing information to maildir extension header instead of kludging Timo Sirainen <tss@iki.fi> parents: 5926 diff changeset	598 *cur_changed_r = TRUE;
b9865213da42 Store syncing information to maildir extension header instead of kludging Timo Sirainen <tss@iki.fi> parents: 5926 diff changeset	599 if (new_changed_r && cur_changed_r)
b9865213da42 Store syncing information to maildir extension header instead of kludging Timo Sirainen <tss@iki.fi> parents: 5926 diff changeset	600 return 0;
1954 2f6e137cdc44 Syncing optimizations. Timo Sirainen <tss@iki.fi> parents: 1947 diff changeset	601 }
2f6e137cdc44 Syncing optimizations. Timo Sirainen <tss@iki.fi> parents: 1947 diff changeset	602
5927 b9865213da42 Store syncing information to maildir extension header instead of kludging Timo Sirainen <tss@iki.fi> parents: 5926 diff changeset	603 if (!*new_changed_r) {
b9865213da42 Store syncing information to maildir extension header instead of kludging Timo Sirainen <tss@iki.fi> parents: 5926 diff changeset	604 if (maildir_stat(mbox, new_dir, &new_st) < 0)
b9865213da42 Store syncing information to maildir extension header instead of kludging Timo Sirainen <tss@iki.fi> parents: 5926 diff changeset	605 return -1;
b9865213da42 Store syncing information to maildir extension header instead of kludging Timo Sirainen <tss@iki.fi> parents: 5926 diff changeset	606 check_new = TRUE;
b9865213da42 Store syncing information to maildir extension header instead of kludging Timo Sirainen <tss@iki.fi> parents: 5926 diff changeset	607 }
b9865213da42 Store syncing information to maildir extension header instead of kludging Timo Sirainen <tss@iki.fi> parents: 5926 diff changeset	608 if (!*cur_changed_r) {
b9865213da42 Store syncing information to maildir extension header instead of kludging Timo Sirainen <tss@iki.fi> parents: 5926 diff changeset	609 if (maildir_stat(mbox, cur_dir, &cur_st) < 0)
b9865213da42 Store syncing information to maildir extension header instead of kludging Timo Sirainen <tss@iki.fi> parents: 5926 diff changeset	610 return -1;
b9865213da42 Store syncing information to maildir extension header instead of kludging Timo Sirainen <tss@iki.fi> parents: 5926 diff changeset	611 check_cur = TRUE;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	612 }
1954 2f6e137cdc44 Syncing optimizations. Timo Sirainen <tss@iki.fi> parents: 1947 diff changeset	613
5927 b9865213da42 Store syncing information to maildir extension header instead of kludging Timo Sirainen <tss@iki.fi> parents: 5926 diff changeset	614 for (;;) {
b9865213da42 Store syncing information to maildir extension header instead of kludging Timo Sirainen <tss@iki.fi> parents: 5926 diff changeset	615 if (check_new)
b9865213da42 Store syncing information to maildir extension header instead of kludging Timo Sirainen <tss@iki.fi> parents: 5926 diff changeset	616 *new_changed_r = DIR_MTIME_CHANGED(new_st, hdr, new);
b9865213da42 Store syncing information to maildir extension header instead of kludging Timo Sirainen <tss@iki.fi> parents: 5926 diff changeset	617 if (check_cur)
b9865213da42 Store syncing information to maildir extension header instead of kludging Timo Sirainen <tss@iki.fi> parents: 5926 diff changeset	618 *cur_changed_r = DIR_MTIME_CHANGED(cur_st, hdr, cur);
1984 9c159272f721 syncing fixes Timo Sirainen <tss@iki.fi> parents: 1978 diff changeset	619
5927 b9865213da42 Store syncing information to maildir extension header instead of kludging Timo Sirainen <tss@iki.fi> parents: 5926 diff changeset	620 if ((!new_changed_r && !cur_changed_r) \|\| refreshed)
b9865213da42 Store syncing information to maildir extension header instead of kludging Timo Sirainen <tss@iki.fi> parents: 5926 diff changeset	621 break;
b9865213da42 Store syncing information to maildir extension header instead of kludging Timo Sirainen <tss@iki.fi> parents: 5926 diff changeset	622
b9865213da42 Store syncing information to maildir extension header instead of kludging Timo Sirainen <tss@iki.fi> parents: 5926 diff changeset	623 /* refresh index and try again */
7515 c14a2b0a3126 Keep track of dovecot-uidlist mtime+size in index file. If it's up-to-date Timo Sirainen <tss@iki.fi> parents: 7506 diff changeset	624 if (maildir_sync_header_refresh(mbox) < 0)
6328 29d249982282 Error handling fixes. Timo Sirainen <tss@iki.fi> parents: 6275 diff changeset	625 return -1;
5927 b9865213da42 Store syncing information to maildir extension header instead of kludging Timo Sirainen <tss@iki.fi> parents: 5926 diff changeset	626 refreshed = TRUE;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	627 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	628
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	629 return 0;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	630 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	631
6027 3b260c6d9207 If uidlist gets deleted but indexes not, give UIDs to messages beginning Timo Sirainen <tss@iki.fi> parents: 5983 diff changeset	632 static void maildir_sync_update_next_uid(struct maildir_mailbox *mbox)
3b260c6d9207 If uidlist gets deleted but indexes not, give UIDs to messages beginning Timo Sirainen <tss@iki.fi> parents: 5983 diff changeset	633 {
3b260c6d9207 If uidlist gets deleted but indexes not, give UIDs to messages beginning Timo Sirainen <tss@iki.fi> parents: 5983 diff changeset	634 const struct mail_index_header *hdr;
3b260c6d9207 If uidlist gets deleted but indexes not, give UIDs to messages beginning Timo Sirainen <tss@iki.fi> parents: 5983 diff changeset	635 uint32_t uid_validity, next_uid;
3b260c6d9207 If uidlist gets deleted but indexes not, give UIDs to messages beginning Timo Sirainen <tss@iki.fi> parents: 5983 diff changeset	636
3b260c6d9207 If uidlist gets deleted but indexes not, give UIDs to messages beginning Timo Sirainen <tss@iki.fi> parents: 5983 diff changeset	637 hdr = mail_index_get_header(mbox->ibox.view);
3b260c6d9207 If uidlist gets deleted but indexes not, give UIDs to messages beginning Timo Sirainen <tss@iki.fi> parents: 5983 diff changeset	638 if (hdr->uid_validity == 0)
3b260c6d9207 If uidlist gets deleted but indexes not, give UIDs to messages beginning Timo Sirainen <tss@iki.fi> parents: 5983 diff changeset	639 return;
3b260c6d9207 If uidlist gets deleted but indexes not, give UIDs to messages beginning Timo Sirainen <tss@iki.fi> parents: 5983 diff changeset	640
3b260c6d9207 If uidlist gets deleted but indexes not, give UIDs to messages beginning Timo Sirainen <tss@iki.fi> parents: 5983 diff changeset	641 uid_validity = maildir_uidlist_get_uid_validity(mbox->uidlist);
3b260c6d9207 If uidlist gets deleted but indexes not, give UIDs to messages beginning Timo Sirainen <tss@iki.fi> parents: 5983 diff changeset	642 next_uid = maildir_uidlist_get_next_uid(mbox->uidlist);
3b260c6d9207 If uidlist gets deleted but indexes not, give UIDs to messages beginning Timo Sirainen <tss@iki.fi> parents: 5983 diff changeset	643
3b260c6d9207 If uidlist gets deleted but indexes not, give UIDs to messages beginning Timo Sirainen <tss@iki.fi> parents: 5983 diff changeset	644 if (uid_validity == hdr->uid_validity \|\| uid_validity == 0) {
3b260c6d9207 If uidlist gets deleted but indexes not, give UIDs to messages beginning Timo Sirainen <tss@iki.fi> parents: 5983 diff changeset	645 /* make sure uidlist's next_uid is at least as large as
3b260c6d9207 If uidlist gets deleted but indexes not, give UIDs to messages beginning Timo Sirainen <tss@iki.fi> parents: 5983 diff changeset	646 index file's. typically this happens only if uidlist gets
3b260c6d9207 If uidlist gets deleted but indexes not, give UIDs to messages beginning Timo Sirainen <tss@iki.fi> parents: 5983 diff changeset	647 deleted. */
3b260c6d9207 If uidlist gets deleted but indexes not, give UIDs to messages beginning Timo Sirainen <tss@iki.fi> parents: 5983 diff changeset	648 maildir_uidlist_set_uid_validity(mbox->uidlist,
3b260c6d9207 If uidlist gets deleted but indexes not, give UIDs to messages beginning Timo Sirainen <tss@iki.fi> parents: 5983 diff changeset	649 hdr->uid_validity);
3b260c6d9207 If uidlist gets deleted but indexes not, give UIDs to messages beginning Timo Sirainen <tss@iki.fi> parents: 5983 diff changeset	650 maildir_uidlist_set_next_uid(mbox->uidlist,
3b260c6d9207 If uidlist gets deleted but indexes not, give UIDs to messages beginning Timo Sirainen <tss@iki.fi> parents: 5983 diff changeset	651 hdr->next_uid, FALSE);
3b260c6d9207 If uidlist gets deleted but indexes not, give UIDs to messages beginning Timo Sirainen <tss@iki.fi> parents: 5983 diff changeset	652 }
3b260c6d9207 If uidlist gets deleted but indexes not, give UIDs to messages beginning Timo Sirainen <tss@iki.fi> parents: 5983 diff changeset	653 }
3b260c6d9207 If uidlist gets deleted but indexes not, give UIDs to messages beginning Timo Sirainen <tss@iki.fi> parents: 5983 diff changeset	654
9067 e7792f4509e6 Maildir: Make sure messages are removed from index if they are deleted externally from new/. Timo Sirainen <tss@iki.fi> parents: 9018 diff changeset	655 static bool have_recent_messages(struct maildir_sync_context *ctx)
6037 d911d943438e Recent flag handling rewrite. Still not perfect with maildir. Timo Sirainen <tss@iki.fi> parents: 6027 diff changeset	656 {
d911d943438e Recent flag handling rewrite. Still not perfect with maildir. Timo Sirainen <tss@iki.fi> parents: 6027 diff changeset	657 const struct mail_index_header *hdr;
d911d943438e Recent flag handling rewrite. Still not perfect with maildir. Timo Sirainen <tss@iki.fi> parents: 6027 diff changeset	658
6859 64e0f45e8dc3 Delayed NFS attribute cache flushing had some bug, so removed it for now. Timo Sirainen <tss@iki.fi> parents: 6800 diff changeset	659 (void)maildir_uidlist_refresh(ctx->mbox->uidlist);
6037 d911d943438e Recent flag handling rewrite. Still not perfect with maildir. Timo Sirainen <tss@iki.fi> parents: 6027 diff changeset	660
d911d943438e Recent flag handling rewrite. Still not perfect with maildir. Timo Sirainen <tss@iki.fi> parents: 6027 diff changeset	661 /* if there are files in new/, we'll need to move them. we'll check
d911d943438e Recent flag handling rewrite. Still not perfect with maildir. Timo Sirainen <tss@iki.fi> parents: 6027 diff changeset	662 this by checking if we have any recent messages */
d911d943438e Recent flag handling rewrite. Still not perfect with maildir. Timo Sirainen <tss@iki.fi> parents: 6027 diff changeset	663 hdr = mail_index_get_header(ctx->mbox->ibox.view);
d911d943438e Recent flag handling rewrite. Still not perfect with maildir. Timo Sirainen <tss@iki.fi> parents: 6027 diff changeset	664 return hdr->first_recent_uid <
d911d943438e Recent flag handling rewrite. Still not perfect with maildir. Timo Sirainen <tss@iki.fi> parents: 6027 diff changeset	665 maildir_uidlist_get_next_uid(ctx->mbox->uidlist);
d911d943438e Recent flag handling rewrite. Still not perfect with maildir. Timo Sirainen <tss@iki.fi> parents: 6027 diff changeset	666 }
d911d943438e Recent flag handling rewrite. Still not perfect with maildir. Timo Sirainen <tss@iki.fi> parents: 6027 diff changeset	667
6452 a62923d3c969 Don't sync mailbox immediately when committing transactions. When it's done Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	668 static int maildir_sync_get_changes(struct maildir_sync_context *ctx,
a62923d3c969 Don't sync mailbox immediately when committing transactions. When it's done Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	669 bool new_changed_r, bool cur_changed_r)
a62923d3c969 Don't sync mailbox immediately when committing transactions. When it's done Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	670 {
a62923d3c969 Don't sync mailbox immediately when committing transactions. When it's done Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	671 enum mail_index_sync_flags flags = 0;
7066 048325df1ec4 If MAILBOX_SYNC_FLAG_FULL_READ flag is set (SELECT and CHECK IMAP commands), Timo Sirainen <tss@iki.fi> parents: 7046 diff changeset	672 bool undirty = (ctx->flags & MAILBOX_SYNC_FLAG_FULL_READ) != 0;
6452 a62923d3c969 Don't sync mailbox immediately when committing transactions. When it's done Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	673
7066 048325df1ec4 If MAILBOX_SYNC_FLAG_FULL_READ flag is set (SELECT and CHECK IMAP commands), Timo Sirainen <tss@iki.fi> parents: 7046 diff changeset	674 if (maildir_sync_quick_check(ctx->mbox, undirty,
048325df1ec4 If MAILBOX_SYNC_FLAG_FULL_READ flag is set (SELECT and CHECK IMAP commands), Timo Sirainen <tss@iki.fi> parents: 7046 diff changeset	675 ctx->new_dir, ctx->cur_dir,
6452 a62923d3c969 Don't sync mailbox immediately when committing transactions. When it's done Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	676 new_changed_r, cur_changed_r) < 0)
a62923d3c969 Don't sync mailbox immediately when committing transactions. When it's done Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	677 return -1;
a62923d3c969 Don't sync mailbox immediately when committing transactions. When it's done Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	678
a62923d3c969 Don't sync mailbox immediately when committing transactions. When it's done Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	679 if (new_changed_r \|\| cur_changed_r)
a62923d3c969 Don't sync mailbox immediately when committing transactions. When it's done Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	680 return 1;
a62923d3c969 Don't sync mailbox immediately when committing transactions. When it's done Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	681
9067 e7792f4509e6 Maildir: Make sure messages are removed from index if they are deleted externally from new/. Timo Sirainen <tss@iki.fi> parents: 9018 diff changeset	682 if (have_recent_messages(ctx)) {
e7792f4509e6 Maildir: Make sure messages are removed from index if they are deleted externally from new/. Timo Sirainen <tss@iki.fi> parents: 9018 diff changeset	683 if (!ctx->mbox->ibox.keep_recent) {
e7792f4509e6 Maildir: Make sure messages are removed from index if they are deleted externally from new/. Timo Sirainen <tss@iki.fi> parents: 9018 diff changeset	684 *new_changed_r = TRUE;
e7792f4509e6 Maildir: Make sure messages are removed from index if they are deleted externally from new/. Timo Sirainen <tss@iki.fi> parents: 9018 diff changeset	685 return 1;
e7792f4509e6 Maildir: Make sure messages are removed from index if they are deleted externally from new/. Timo Sirainen <tss@iki.fi> parents: 9018 diff changeset	686 } else if (*new_changed_r) {
e7792f4509e6 Maildir: Make sure messages are removed from index if they are deleted externally from new/. Timo Sirainen <tss@iki.fi> parents: 9018 diff changeset	687 /* we have some recent messages and new/ has changed.
e7792f4509e6 Maildir: Make sure messages are removed from index if they are deleted externally from new/. Timo Sirainen <tss@iki.fi> parents: 9018 diff changeset	688 if messages had been externally deleted from new/,
e7792f4509e6 Maildir: Make sure messages are removed from index if they are deleted externally from new/. Timo Sirainen <tss@iki.fi> parents: 9018 diff changeset	689 we need to get them out of index. this requires that
e7792f4509e6 Maildir: Make sure messages are removed from index if they are deleted externally from new/. Timo Sirainen <tss@iki.fi> parents: 9018 diff changeset	690 we make sure they weren't just moved to cur/. */
e7792f4509e6 Maildir: Make sure messages are removed from index if they are deleted externally from new/. Timo Sirainen <tss@iki.fi> parents: 9018 diff changeset	691 *cur_changed_r = TRUE;
e7792f4509e6 Maildir: Make sure messages are removed from index if they are deleted externally from new/. Timo Sirainen <tss@iki.fi> parents: 9018 diff changeset	692 return 1;
e7792f4509e6 Maildir: Make sure messages are removed from index if they are deleted externally from new/. Timo Sirainen <tss@iki.fi> parents: 9018 diff changeset	693 }
6452 a62923d3c969 Don't sync mailbox immediately when committing transactions. When it's done Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	694 }
a62923d3c969 Don't sync mailbox immediately when committing transactions. When it's done Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	695
a62923d3c969 Don't sync mailbox immediately when committing transactions. When it's done Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	696 if (!ctx->mbox->ibox.keep_recent)
a62923d3c969 Don't sync mailbox immediately when committing transactions. When it's done Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	697 flags \|= MAIL_INDEX_SYNC_FLAG_DROP_RECENT;
a62923d3c969 Don't sync mailbox immediately when committing transactions. When it's done Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	698
a62923d3c969 Don't sync mailbox immediately when committing transactions. When it's done Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	699 return mail_index_sync_have_any(ctx->mbox->ibox.index, flags) ? 1 : 0;
a62923d3c969 Don't sync mailbox immediately when committing transactions. When it's done Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	700 }
a62923d3c969 Don't sync mailbox immediately when committing transactions. When it's done Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	701
3863 55df57c028d4 Added "bool" type and changed all ints that were used as booleans to bool. Timo Sirainen <tss@iki.fi> parents: 3821 diff changeset	702 static int maildir_sync_context(struct maildir_sync_context *ctx, bool forced,
6886 82420ed23379 If doing a lockless forced sync, track the UID we're trying to find. If we Timo Sirainen <tss@iki.fi> parents: 6884 diff changeset	703 uint32_t find_uid, bool lost_files_r)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	704 {
6878 0c3ab0aef33c maildir_uidlist_sync_init() was called wrong, although it just happened to Timo Sirainen <tss@iki.fi> parents: 6859 diff changeset	705 enum maildir_uidlist_sync_flags sync_flags;
6886 82420ed23379 If doing a lockless forced sync, track the UID we're trying to find. If we Timo Sirainen <tss@iki.fi> parents: 6884 diff changeset	706 enum maildir_uidlist_rec_flag flags;
8554 c438437b709a maildir: Allow opening mailbox and expunging messages when uidlist couldn't be locked (e.g. out of quota). Timo Sirainen <tss@iki.fi> parents: 7526 diff changeset	707 bool new_changed, cur_changed, lock_failure;
9017 0aa17f3e4a6d Maildir: Handle uidlist errors better. Timo Sirainen <tss@iki.fi> parents: 8997 diff changeset	708 const char *fname;
3863 55df57c028d4 Added "bool" type and changed all ints that were used as booleans to bool. Timo Sirainen <tss@iki.fi> parents: 3821 diff changeset	709 int ret;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	710
6452 a62923d3c969 Don't sync mailbox immediately when committing transactions. When it's done Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	711 *lost_files_r = FALSE;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	712
6452 a62923d3c969 Don't sync mailbox immediately when committing transactions. When it's done Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	713 if (forced)
2140 e2cd51b99359 "readonly sync" -> "forced sync" Timo Sirainen <tss@iki.fi> parents: 2123 diff changeset	714 new_changed = cur_changed = TRUE;
6452 a62923d3c969 Don't sync mailbox immediately when committing transactions. When it's done Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	715 else {
a62923d3c969 Don't sync mailbox immediately when committing transactions. When it's done Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	716 ret = maildir_sync_get_changes(ctx, &new_changed, &cur_changed);
a62923d3c969 Don't sync mailbox immediately when committing transactions. When it's done Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	717 if (ret <= 0)
a62923d3c969 Don't sync mailbox immediately when committing transactions. When it's done Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	718 return ret;
2140 e2cd51b99359 "readonly sync" -> "forced sync" Timo Sirainen <tss@iki.fi> parents: 2123 diff changeset	719 }
1947 777da553d1d3 Recent-flag should work now Timo Sirainen <tss@iki.fi> parents: 1944 diff changeset	720
2818 a758a5b542bb Always protect maildir syncing with uidlist lock. Before we only tried to Timo Sirainen <tss@iki.fi> parents: 2816 diff changeset	721 /*
a758a5b542bb Always protect maildir syncing with uidlist lock. Before we only tried to Timo Sirainen <tss@iki.fi> parents: 2816 diff changeset	722 Locking, locking, locking.. Wasn't maildir supposed to be lockless?
a758a5b542bb Always protect maildir syncing with uidlist lock. Before we only tried to Timo Sirainen <tss@iki.fi> parents: 2816 diff changeset	723
a758a5b542bb Always protect maildir syncing with uidlist lock. Before we only tried to Timo Sirainen <tss@iki.fi> parents: 2816 diff changeset	724 We can get here either as beginning a real maildir sync, or when
a758a5b542bb Always protect maildir syncing with uidlist lock. Before we only tried to Timo Sirainen <tss@iki.fi> parents: 2816 diff changeset	725 committing changes to maildir but a file was lost (maybe renamed).
a758a5b542bb Always protect maildir syncing with uidlist lock. Before we only tried to Timo Sirainen <tss@iki.fi> parents: 2816 diff changeset	726
a758a5b542bb Always protect maildir syncing with uidlist lock. Before we only tried to Timo Sirainen <tss@iki.fi> parents: 2816 diff changeset	727 So, we're going to need two locks. One for index and one for
4238 3c8b191b0019 Adding mail to index while saving it had a race condition. Fixing it Timo Sirainen <timo.sirainen@movial.fi> parents: 4199 diff changeset	728 uidlist. To avoid deadlocking do the uidlist lock always first.
2121 0840edf34f37 Locking fixes. use less memory Timo Sirainen <tss@iki.fi> parents: 2067 diff changeset	729
2818 a758a5b542bb Always protect maildir syncing with uidlist lock. Before we only tried to Timo Sirainen <tss@iki.fi> parents: 2816 diff changeset	730 uidlist is needed only for figuring out UIDs for newly seen files,
a758a5b542bb Always protect maildir syncing with uidlist lock. Before we only tried to Timo Sirainen <tss@iki.fi> parents: 2816 diff changeset	731 so theoretically we wouldn't need to lock it unless there are new
a758a5b542bb Always protect maildir syncing with uidlist lock. Before we only tried to Timo Sirainen <tss@iki.fi> parents: 2816 diff changeset	732 files. It has a few problems though, assuming the index lock didn't
a758a5b542bb Always protect maildir syncing with uidlist lock. Before we only tried to Timo Sirainen <tss@iki.fi> parents: 2816 diff changeset	733 already protect it (eg. in-memory indexes):
a758a5b542bb Always protect maildir syncing with uidlist lock. Before we only tried to Timo Sirainen <tss@iki.fi> parents: 2816 diff changeset	734
a758a5b542bb Always protect maildir syncing with uidlist lock. Before we only tried to Timo Sirainen <tss@iki.fi> parents: 2816 diff changeset	735 1. Just because you see a new file which doesn't exist in uidlist
a758a5b542bb Always protect maildir syncing with uidlist lock. Before we only tried to Timo Sirainen <tss@iki.fi> parents: 2816 diff changeset	736 file, doesn't mean that the file really exists anymore, or that
a758a5b542bb Always protect maildir syncing with uidlist lock. Before we only tried to Timo Sirainen <tss@iki.fi> parents: 2816 diff changeset	737 your readdir() lists all new files. Meaning that this is possible:
a758a5b542bb Always protect maildir syncing with uidlist lock. Before we only tried to Timo Sirainen <tss@iki.fi> parents: 2816 diff changeset	738
a758a5b542bb Always protect maildir syncing with uidlist lock. Before we only tried to Timo Sirainen <tss@iki.fi> parents: 2816 diff changeset	739 A: opendir(), readdir() -> new file ...
a758a5b542bb Always protect maildir syncing with uidlist lock. Before we only tried to Timo Sirainen <tss@iki.fi> parents: 2816 diff changeset	740 -- new files are written to the maildir --
a758a5b542bb Always protect maildir syncing with uidlist lock. Before we only tried to Timo Sirainen <tss@iki.fi> parents: 2816 diff changeset	741 B: opendir(), readdir() -> new file, lock uidlist,
a758a5b542bb Always protect maildir syncing with uidlist lock. Before we only tried to Timo Sirainen <tss@iki.fi> parents: 2816 diff changeset	742 readdir() -> another new file, rewrite uidlist, unlock
a758a5b542bb Always protect maildir syncing with uidlist lock. Before we only tried to Timo Sirainen <tss@iki.fi> parents: 2816 diff changeset	743 A: ... lock uidlist, readdir() -> nothing left, rewrite uidlist,
a758a5b542bb Always protect maildir syncing with uidlist lock. Before we only tried to Timo Sirainen <tss@iki.fi> parents: 2816 diff changeset	744 unlock
2173 9438951e243f Don't crash if we're syncing last commit to maildir, but some file was lost Timo Sirainen <tss@iki.fi> parents: 2140 diff changeset	745
2818 a758a5b542bb Always protect maildir syncing with uidlist lock. Before we only tried to Timo Sirainen <tss@iki.fi> parents: 2816 diff changeset	746 The second time running A didn't see the two new files. To handle
a758a5b542bb Always protect maildir syncing with uidlist lock. Before we only tried to Timo Sirainen <tss@iki.fi> parents: 2816 diff changeset	747 this correctly, it must not remove the new unseen files from
a758a5b542bb Always protect maildir syncing with uidlist lock. Before we only tried to Timo Sirainen <tss@iki.fi> parents: 2816 diff changeset	748 uidlist. This is possible to do, but adds extra complexity.
a758a5b542bb Always protect maildir syncing with uidlist lock. Before we only tried to Timo Sirainen <tss@iki.fi> parents: 2816 diff changeset	749
a758a5b542bb Always protect maildir syncing with uidlist lock. Before we only tried to Timo Sirainen <tss@iki.fi> parents: 2816 diff changeset	750 2. If another process is rename()ing files while we are
a758a5b542bb Always protect maildir syncing with uidlist lock. Before we only tried to Timo Sirainen <tss@iki.fi> parents: 2816 diff changeset	751 readdir()ing, it's possible that readdir() never lists some files,
a758a5b542bb Always protect maildir syncing with uidlist lock. Before we only tried to Timo Sirainen <tss@iki.fi> parents: 2816 diff changeset	752 causing Dovecot to assume they were expunged. In next sync they
a758a5b542bb Always protect maildir syncing with uidlist lock. Before we only tried to Timo Sirainen <tss@iki.fi> parents: 2816 diff changeset	753 would show up again, but client could have already been notified of
a758a5b542bb Always protect maildir syncing with uidlist lock. Before we only tried to Timo Sirainen <tss@iki.fi> parents: 2816 diff changeset	754 that and they would show up under new UIDs, so the damage is
a758a5b542bb Always protect maildir syncing with uidlist lock. Before we only tried to Timo Sirainen <tss@iki.fi> parents: 2816 diff changeset	755 already done.
a758a5b542bb Always protect maildir syncing with uidlist lock. Before we only tried to Timo Sirainen <tss@iki.fi> parents: 2816 diff changeset	756
a758a5b542bb Always protect maildir syncing with uidlist lock. Before we only tried to Timo Sirainen <tss@iki.fi> parents: 2816 diff changeset	757 Both of the problems can be avoided if we simply lock the uidlist
a758a5b542bb Always protect maildir syncing with uidlist lock. Before we only tried to Timo Sirainen <tss@iki.fi> parents: 2816 diff changeset	758 before syncing and keep it until sync is finished. Typically this
a758a5b542bb Always protect maildir syncing with uidlist lock. Before we only tried to Timo Sirainen <tss@iki.fi> parents: 2816 diff changeset	759 would happen in any case, as there is the index lock..
a758a5b542bb Always protect maildir syncing with uidlist lock. Before we only tried to Timo Sirainen <tss@iki.fi> parents: 2816 diff changeset	760
a758a5b542bb Always protect maildir syncing with uidlist lock. Before we only tried to Timo Sirainen <tss@iki.fi> parents: 2816 diff changeset	761 The second case is still a problem with external changes though,
a758a5b542bb Always protect maildir syncing with uidlist lock. Before we only tried to Timo Sirainen <tss@iki.fi> parents: 2816 diff changeset	762 because maildir doesn't require any kind of locking. Luckily this
a758a5b542bb Always protect maildir syncing with uidlist lock. Before we only tried to Timo Sirainen <tss@iki.fi> parents: 2816 diff changeset	763 problem rarely happens except under high amount of modifications.
a758a5b542bb Always protect maildir syncing with uidlist lock. Before we only tried to Timo Sirainen <tss@iki.fi> parents: 2816 diff changeset	764 */
a758a5b542bb Always protect maildir syncing with uidlist lock. Before we only tried to Timo Sirainen <tss@iki.fi> parents: 2816 diff changeset	765
6886 82420ed23379 If doing a lockless forced sync, track the UID we're trying to find. If we Timo Sirainen <tss@iki.fi> parents: 6884 diff changeset	766 if (!cur_changed) {
6880 724c8f12eed2 maildir_storage_sync_force(): Don't wait on trying to lock uidlist. Just try Timo Sirainen <tss@iki.fi> parents: 6878 diff changeset	767 ctx->partial = TRUE;
724c8f12eed2 maildir_storage_sync_force(): Don't wait on trying to lock uidlist. Just try Timo Sirainen <tss@iki.fi> parents: 6878 diff changeset	768 sync_flags = MAILDIR_UIDLIST_SYNC_PARTIAL;
724c8f12eed2 maildir_storage_sync_force(): Don't wait on trying to lock uidlist. Just try Timo Sirainen <tss@iki.fi> parents: 6878 diff changeset	769 } else {
724c8f12eed2 maildir_storage_sync_force(): Don't wait on trying to lock uidlist. Just try Timo Sirainen <tss@iki.fi> parents: 6878 diff changeset	770 ctx->partial = FALSE;
724c8f12eed2 maildir_storage_sync_force(): Don't wait on trying to lock uidlist. Just try Timo Sirainen <tss@iki.fi> parents: 6878 diff changeset	771 sync_flags = 0;
6886 82420ed23379 If doing a lockless forced sync, track the UID we're trying to find. If we Timo Sirainen <tss@iki.fi> parents: 6884 diff changeset	772 if (forced)
82420ed23379 If doing a lockless forced sync, track the UID we're trying to find. If we Timo Sirainen <tss@iki.fi> parents: 6884 diff changeset	773 sync_flags \|= MAILDIR_UIDLIST_SYNC_FORCE;
6881 b1554dea8ee8 If we're doing a fast sync, try only once to lock uidlist. Timo Sirainen <tss@iki.fi> parents: 6880 diff changeset	774 if ((ctx->flags & MAILBOX_SYNC_FLAG_FAST) != 0)
b1554dea8ee8 If we're doing a fast sync, try only once to lock uidlist. Timo Sirainen <tss@iki.fi> parents: 6880 diff changeset	775 sync_flags \|= MAILDIR_UIDLIST_SYNC_TRYLOCK;
6880 724c8f12eed2 maildir_storage_sync_force(): Don't wait on trying to lock uidlist. Just try Timo Sirainen <tss@iki.fi> parents: 6878 diff changeset	776 }
6878 0c3ab0aef33c maildir_uidlist_sync_init() was called wrong, although it just happened to Timo Sirainen <tss@iki.fi> parents: 6859 diff changeset	777 ret = maildir_uidlist_sync_init(ctx->mbox->uidlist, sync_flags,
3530 e9695ec7925b Recursive maildir uidlist syncs caused assert crashes. Also did some Timo Sirainen <tss@iki.fi> parents: 3520 diff changeset	778 &ctx->uidlist_sync_ctx);
8554 c438437b709a maildir: Allow opening mailbox and expunging messages when uidlist couldn't be locked (e.g. out of quota). Timo Sirainen <tss@iki.fi> parents: 7526 diff changeset	779 lock_failure = ret <= 0;
3530 e9695ec7925b Recursive maildir uidlist syncs caused assert crashes. Also did some Timo Sirainen <tss@iki.fi> parents: 3520 diff changeset	780 if (ret <= 0) {
8554 c438437b709a maildir: Allow opening mailbox and expunging messages when uidlist couldn't be locked (e.g. out of quota). Timo Sirainen <tss@iki.fi> parents: 7526 diff changeset	781 struct mail_storage *storage = ctx->mbox->ibox.box.storage;
c438437b709a maildir: Allow opening mailbox and expunging messages when uidlist couldn't be locked (e.g. out of quota). Timo Sirainen <tss@iki.fi> parents: 7526 diff changeset	782
c438437b709a maildir: Allow opening mailbox and expunging messages when uidlist couldn't be locked (e.g. out of quota). Timo Sirainen <tss@iki.fi> parents: 7526 diff changeset	783 if (ret == 0) {
c438437b709a maildir: Allow opening mailbox and expunging messages when uidlist couldn't be locked (e.g. out of quota). Timo Sirainen <tss@iki.fi> parents: 7526 diff changeset	784 /* timeout */
c438437b709a maildir: Allow opening mailbox and expunging messages when uidlist couldn't be locked (e.g. out of quota). Timo Sirainen <tss@iki.fi> parents: 7526 diff changeset	785 return 0;
c438437b709a maildir: Allow opening mailbox and expunging messages when uidlist couldn't be locked (e.g. out of quota). Timo Sirainen <tss@iki.fi> parents: 7526 diff changeset	786 }
c438437b709a maildir: Allow opening mailbox and expunging messages when uidlist couldn't be locked (e.g. out of quota). Timo Sirainen <tss@iki.fi> parents: 7526 diff changeset	787 /* locking failed. sync anyway without locking so that it's
c438437b709a maildir: Allow opening mailbox and expunging messages when uidlist couldn't be locked (e.g. out of quota). Timo Sirainen <tss@iki.fi> parents: 7526 diff changeset	788 possible to expunge messages when out of quota. */
c438437b709a maildir: Allow opening mailbox and expunging messages when uidlist couldn't be locked (e.g. out of quota). Timo Sirainen <tss@iki.fi> parents: 7526 diff changeset	789 if (forced) {
c438437b709a maildir: Allow opening mailbox and expunging messages when uidlist couldn't be locked (e.g. out of quota). Timo Sirainen <tss@iki.fi> parents: 7526 diff changeset	790 /* we're already forcing a sync, we're trying to find
c438437b709a maildir: Allow opening mailbox and expunging messages when uidlist couldn't be locked (e.g. out of quota). Timo Sirainen <tss@iki.fi> parents: 7526 diff changeset	791 a message that was probably already expunged, don't
c438437b709a maildir: Allow opening mailbox and expunging messages when uidlist couldn't be locked (e.g. out of quota). Timo Sirainen <tss@iki.fi> parents: 7526 diff changeset	792 loop for a long time trying to find it. */
c438437b709a maildir: Allow opening mailbox and expunging messages when uidlist couldn't be locked (e.g. out of quota). Timo Sirainen <tss@iki.fi> parents: 7526 diff changeset	793 return -1;
c438437b709a maildir: Allow opening mailbox and expunging messages when uidlist couldn't be locked (e.g. out of quota). Timo Sirainen <tss@iki.fi> parents: 7526 diff changeset	794 }
c438437b709a maildir: Allow opening mailbox and expunging messages when uidlist couldn't be locked (e.g. out of quota). Timo Sirainen <tss@iki.fi> parents: 7526 diff changeset	795 ret = maildir_uidlist_sync_init(ctx->mbox->uidlist, sync_flags \|
c438437b709a maildir: Allow opening mailbox and expunging messages when uidlist couldn't be locked (e.g. out of quota). Timo Sirainen <tss@iki.fi> parents: 7526 diff changeset	796 MAILDIR_UIDLIST_SYNC_NOLOCK,
c438437b709a maildir: Allow opening mailbox and expunging messages when uidlist couldn't be locked (e.g. out of quota). Timo Sirainen <tss@iki.fi> parents: 7526 diff changeset	797 &ctx->uidlist_sync_ctx);
8997 cceea391d04d maildir: Don't assert-crash if we have no read-access to the Maildir. Timo Sirainen <tss@iki.fi> parents: 8876 diff changeset	798 if (ret <= 0) {
cceea391d04d maildir: Don't assert-crash if we have no read-access to the Maildir. Timo Sirainen <tss@iki.fi> parents: 8876 diff changeset	799 i_assert(ret != 0);
cceea391d04d maildir: Don't assert-crash if we have no read-access to the Maildir. Timo Sirainen <tss@iki.fi> parents: 8876 diff changeset	800 return -1;
cceea391d04d maildir: Don't assert-crash if we have no read-access to the Maildir. Timo Sirainen <tss@iki.fi> parents: 8876 diff changeset	801 }
8554 c438437b709a maildir: Allow opening mailbox and expunging messages when uidlist couldn't be locked (e.g. out of quota). Timo Sirainen <tss@iki.fi> parents: 7526 diff changeset	802
c438437b709a maildir: Allow opening mailbox and expunging messages when uidlist couldn't be locked (e.g. out of quota). Timo Sirainen <tss@iki.fi> parents: 7526 diff changeset	803 if (storage->callbacks->notify_no != NULL) {
c438437b709a maildir: Allow opening mailbox and expunging messages when uidlist couldn't be locked (e.g. out of quota). Timo Sirainen <tss@iki.fi> parents: 7526 diff changeset	804 storage->callbacks->notify_no(&ctx->mbox->ibox.box,
c438437b709a maildir: Allow opening mailbox and expunging messages when uidlist couldn't be locked (e.g. out of quota). Timo Sirainen <tss@iki.fi> parents: 7526 diff changeset	805 "Internal mailbox synchronization failure, "
c438437b709a maildir: Allow opening mailbox and expunging messages when uidlist couldn't be locked (e.g. out of quota). Timo Sirainen <tss@iki.fi> parents: 7526 diff changeset	806 "showing only old mails.",
c438437b709a maildir: Allow opening mailbox and expunging messages when uidlist couldn't be locked (e.g. out of quota). Timo Sirainen <tss@iki.fi> parents: 7526 diff changeset	807 storage->callback_context);
c438437b709a maildir: Allow opening mailbox and expunging messages when uidlist couldn't be locked (e.g. out of quota). Timo Sirainen <tss@iki.fi> parents: 7526 diff changeset	808 }
2140 e2cd51b99359 "readonly sync" -> "forced sync" Timo Sirainen <tss@iki.fi> parents: 2123 diff changeset	809 }
6880 724c8f12eed2 maildir_storage_sync_force(): Don't wait on trying to lock uidlist. Just try Timo Sirainen <tss@iki.fi> parents: 6878 diff changeset	810 ctx->locked = maildir_uidlist_is_locked(ctx->mbox->uidlist);
6886 82420ed23379 If doing a lockless forced sync, track the UID we're trying to find. If we Timo Sirainen <tss@iki.fi> parents: 6884 diff changeset	811 if (!ctx->locked)
82420ed23379 If doing a lockless forced sync, track the UID we're trying to find. If we Timo Sirainen <tss@iki.fi> parents: 6884 diff changeset	812 ctx->partial = TRUE;
2123 e01de478882f locking fixes Timo Sirainen <tss@iki.fi> parents: 2121 diff changeset	813
8554 c438437b709a maildir: Allow opening mailbox and expunging messages when uidlist couldn't be locked (e.g. out of quota). Timo Sirainen <tss@iki.fi> parents: 7526 diff changeset	814 if (!ctx->mbox->syncing_commit && (ctx->locked \|\| lock_failure)) {
5920 00c5e3cbeaf0 Moved index syncing code to its own file. Timo Sirainen <tss@iki.fi> parents: 5918 diff changeset	815 if (maildir_sync_index_begin(ctx->mbox, ctx,
4238 3c8b191b0019 Adding mail to index while saving it had a race condition. Fixing it Timo Sirainen <timo.sirainen@movial.fi> parents: 4199 diff changeset	816 &ctx->index_sync_ctx) < 0)
3c8b191b0019 Adding mail to index while saving it had a race condition. Fixing it Timo Sirainen <timo.sirainen@movial.fi> parents: 4199 diff changeset	817 return -1;
3c8b191b0019 Adding mail to index while saving it had a race condition. Fixing it Timo Sirainen <timo.sirainen@movial.fi> parents: 4199 diff changeset	818 }
3c8b191b0019 Adding mail to index while saving it had a race condition. Fixing it Timo Sirainen <timo.sirainen@movial.fi> parents: 4199 diff changeset	819
3447 96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	820 if (new_changed \|\| cur_changed) {
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	821 /* if we're going to check cur/ dir our current logic requires
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	822 that new/ dir is checked as well. it's a good idea anyway. */
5391 0c8705aad54c Avoid infinite looping when buggy filesystems. Timo Sirainen <tss@iki.fi> parents: 5390 diff changeset	823 unsigned int count = 0;
9201 4e74f4651f5b OSX+Maildir: If readdir() fails with EINVAL after doing rename()s, retry. Fixes HFS+. Timo Sirainen <tss@iki.fi> parents: 9067 diff changeset	824 bool final = FALSE;
5391 0c8705aad54c Avoid infinite looping when buggy filesystems. Timo Sirainen <tss@iki.fi> parents: 5390 diff changeset	825
9201 4e74f4651f5b OSX+Maildir: If readdir() fails with EINVAL after doing rename()s, retry. Fixes HFS+. Timo Sirainen <tss@iki.fi> parents: 9067 diff changeset	826 while ((ret = maildir_scan_dir(ctx, TRUE, final)) > 0) {
3447 96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	827 /* rename()d at least some files, which might have
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	828 caused some other files to be missed. check again
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	829 (see MAILDIR_RENAME_RESCAN_COUNT). */
9201 4e74f4651f5b OSX+Maildir: If readdir() fails with EINVAL after doing rename()s, retry. Fixes HFS+. Timo Sirainen <tss@iki.fi> parents: 9067 diff changeset	830 if (++count >= MAILDIR_SCAN_DIR_MAX_COUNT)
4e74f4651f5b OSX+Maildir: If readdir() fails with EINVAL after doing rename()s, retry. Fixes HFS+. Timo Sirainen <tss@iki.fi> parents: 9067 diff changeset	831 final = TRUE;
3447 96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	832 }
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	833 if (ret < 0)
1954 2f6e137cdc44 Syncing optimizations. Timo Sirainen <tss@iki.fi> parents: 1947 diff changeset	834 return -1;
3447 96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	835
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	836 if (cur_changed) {
9201 4e74f4651f5b OSX+Maildir: If readdir() fails with EINVAL after doing rename()s, retry. Fixes HFS+. Timo Sirainen <tss@iki.fi> parents: 9067 diff changeset	837 if (maildir_scan_dir(ctx, FALSE, TRUE) < 0)
3447 96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	838 return -1;
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	839 }
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	840
6027 3b260c6d9207 If uidlist gets deleted but indexes not, give UIDs to messages beginning Timo Sirainen <tss@iki.fi> parents: 5983 diff changeset	841 maildir_sync_update_next_uid(ctx->mbox);
3b260c6d9207 If uidlist gets deleted but indexes not, give UIDs to messages beginning Timo Sirainen <tss@iki.fi> parents: 5983 diff changeset	842
3447 96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	843 /* finish uidlist syncing, but keep it still locked */
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	844 maildir_uidlist_sync_finish(ctx->uidlist_sync_ctx);
1954 2f6e137cdc44 Syncing optimizations. Timo Sirainen <tss@iki.fi> parents: 1947 diff changeset	845 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	846
6883 15df8bb27d7a If we did unlocked maildir sync, make sure we sync it later with locking. Timo Sirainen <tss@iki.fi> parents: 6881 diff changeset	847 if (!ctx->locked) {
15df8bb27d7a If we did unlocked maildir sync, make sure we sync it later with locking. Timo Sirainen <tss@iki.fi> parents: 6881 diff changeset	848 /* make sure we sync the maildir later */
15df8bb27d7a If we did unlocked maildir sync, make sure we sync it later with locking. Timo Sirainen <tss@iki.fi> parents: 6881 diff changeset	849 ctx->mbox->maildir_hdr.new_mtime = 0;
15df8bb27d7a If we did unlocked maildir sync, make sure we sync it later with locking. Timo Sirainen <tss@iki.fi> parents: 6881 diff changeset	850 ctx->mbox->maildir_hdr.cur_mtime = 0;
15df8bb27d7a If we did unlocked maildir sync, make sure we sync it later with locking. Timo Sirainen <tss@iki.fi> parents: 6881 diff changeset	851 }
15df8bb27d7a If we did unlocked maildir sync, make sure we sync it later with locking. Timo Sirainen <tss@iki.fi> parents: 6881 diff changeset	852
8554 c438437b709a maildir: Allow opening mailbox and expunging messages when uidlist couldn't be locked (e.g. out of quota). Timo Sirainen <tss@iki.fi> parents: 7526 diff changeset	853 if (ctx->index_sync_ctx != NULL) {
3530 e9695ec7925b Recursive maildir uidlist syncs caused assert crashes. Also did some Timo Sirainen <tss@iki.fi> parents: 3520 diff changeset	854 /* NOTE: index syncing here might cause a re-sync due to
e9695ec7925b Recursive maildir uidlist syncs caused assert crashes. Also did some Timo Sirainen <tss@iki.fi> parents: 3520 diff changeset	855 files getting lost, so this function might be called
5927 b9865213da42 Store syncing information to maildir extension header instead of kludging Timo Sirainen <tss@iki.fi> parents: 5926 diff changeset	856 re-entrantly. */
4238 3c8b191b0019 Adding mail to index while saving it had a race condition. Fixing it Timo Sirainen <timo.sirainen@movial.fi> parents: 4199 diff changeset	857 ret = maildir_sync_index(ctx->index_sync_ctx, ctx->partial);
3c8b191b0019 Adding mail to index while saving it had a race condition. Fixing it Timo Sirainen <timo.sirainen@movial.fi> parents: 4199 diff changeset	858 if (maildir_sync_index_finish(&ctx->index_sync_ctx,
4774 615b7738a62f Saving mails could have skipped over transactions, which caused different Timo Sirainen <tss@iki.fi> parents: 4612 diff changeset	859 ret < 0, FALSE) < 0)
2173 9438951e243f Don't crash if we're syncing last commit to maildir, but some file was lost Timo Sirainen <tss@iki.fi> parents: 2140 diff changeset	860 return -1;
4238 3c8b191b0019 Adding mail to index while saving it had a race condition. Fixing it Timo Sirainen <timo.sirainen@movial.fi> parents: 4199 diff changeset	861
3c8b191b0019 Adding mail to index while saving it had a race condition. Fixing it Timo Sirainen <timo.sirainen@movial.fi> parents: 4199 diff changeset	862 if (ret < 0)
3c8b191b0019 Adding mail to index while saving it had a race condition. Fixing it Timo Sirainen <timo.sirainen@movial.fi> parents: 4199 diff changeset	863 return -1;
3436 3c51658d6846 We didn't notice if messages were deleted directly from new/. Timo Sirainen <tss@iki.fi> parents: 3435 diff changeset	864 if (ret == 0)
6452 a62923d3c969 Don't sync mailbox immediately when committing transactions. When it's done Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	865 *lost_files_r = TRUE;
3447 96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	866
8554 c438437b709a maildir: Allow opening mailbox and expunging messages when uidlist couldn't be locked (e.g. out of quota). Timo Sirainen <tss@iki.fi> parents: 7526 diff changeset	867 i_assert(maildir_uidlist_is_locked(ctx->mbox->uidlist) \|\|
c438437b709a maildir: Allow opening mailbox and expunging messages when uidlist couldn't be locked (e.g. out of quota). Timo Sirainen <tss@iki.fi> parents: 7526 diff changeset	868 lock_failure);
2173 9438951e243f Don't crash if we're syncing last commit to maildir, but some file was lost Timo Sirainen <tss@iki.fi> parents: 2140 diff changeset	869 }
1984 9c159272f721 syncing fixes Timo Sirainen <tss@iki.fi> parents: 1978 diff changeset	870
6886 82420ed23379 If doing a lockless forced sync, track the UID we're trying to find. If we Timo Sirainen <tss@iki.fi> parents: 6884 diff changeset	871 if (find_uid != NULL && *find_uid != 0) {
9017 0aa17f3e4a6d Maildir: Handle uidlist errors better. Timo Sirainen <tss@iki.fi> parents: 8997 diff changeset	872 ret = maildir_uidlist_lookup_nosync(ctx->mbox->uidlist,
0aa17f3e4a6d Maildir: Handle uidlist errors better. Timo Sirainen <tss@iki.fi> parents: 8997 diff changeset	873 *find_uid, &flags, &fname);
0aa17f3e4a6d Maildir: Handle uidlist errors better. Timo Sirainen <tss@iki.fi> parents: 8997 diff changeset	874 if (ret < 0)
0aa17f3e4a6d Maildir: Handle uidlist errors better. Timo Sirainen <tss@iki.fi> parents: 8997 diff changeset	875 return -1;
0aa17f3e4a6d Maildir: Handle uidlist errors better. Timo Sirainen <tss@iki.fi> parents: 8997 diff changeset	876 if (ret == 0) {
6886 82420ed23379 If doing a lockless forced sync, track the UID we're trying to find. If we Timo Sirainen <tss@iki.fi> parents: 6884 diff changeset	877 /* UID is expunged */
82420ed23379 If doing a lockless forced sync, track the UID we're trying to find. If we Timo Sirainen <tss@iki.fi> parents: 6884 diff changeset	878 *find_uid = 0;
82420ed23379 If doing a lockless forced sync, track the UID we're trying to find. If we Timo Sirainen <tss@iki.fi> parents: 6884 diff changeset	879 } else if ((flags & MAILDIR_UIDLIST_REC_FLAG_NONSYNCED) == 0) {
82420ed23379 If doing a lockless forced sync, track the UID we're trying to find. If we Timo Sirainen <tss@iki.fi> parents: 6884 diff changeset	880 /* we didn't find it, possibly expunged? */
82420ed23379 If doing a lockless forced sync, track the UID we're trying to find. If we Timo Sirainen <tss@iki.fi> parents: 6884 diff changeset	881 *find_uid = 0;
82420ed23379 If doing a lockless forced sync, track the UID we're trying to find. If we Timo Sirainen <tss@iki.fi> parents: 6884 diff changeset	882 }
82420ed23379 If doing a lockless forced sync, track the UID we're trying to find. If we Timo Sirainen <tss@iki.fi> parents: 6884 diff changeset	883 }
82420ed23379 If doing a lockless forced sync, track the UID we're trying to find. If we Timo Sirainen <tss@iki.fi> parents: 6884 diff changeset	884
9018 0bb192fe0abd Maildir: More fixes to uidlist handling. Timo Sirainen <tss@iki.fi> parents: 9017 diff changeset	885 return maildir_uidlist_sync_deinit(&ctx->uidlist_sync_ctx, TRUE);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	886 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	887
6886 82420ed23379 If doing a lockless forced sync, track the UID we're trying to find. If we Timo Sirainen <tss@iki.fi> parents: 6884 diff changeset	888 int maildir_storage_sync_force(struct maildir_mailbox *mbox, uint32_t uid)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	889 {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	890 struct maildir_sync_context *ctx;
6452 a62923d3c969 Don't sync mailbox immediately when committing transactions. When it's done Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	891 bool lost_files;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	892 int ret;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	893
7226 e6693a0ec8e1 Renamed T_FRAME_BEGIN/END to T_BEGIN/END. Removed T_FRAME() macro and Timo Sirainen <tss@iki.fi> parents: 7086 diff changeset	894 T_BEGIN {
6940 414c9d631a81 Replaced t_push/t_pop calls with T_FRAME() macros. Timo Sirainen <tss@iki.fi>* parents: 6933 diff changeset	895 ctx = maildir_sync_context_new(mbox, MAILBOX_SYNC_FLAG_FAST);
414c9d631a81 Replaced t_push/t_pop calls with T_FRAME() macros. Timo Sirainen <tss@iki.fi>* parents: 6933 diff changeset	896 ret = maildir_sync_context(ctx, TRUE, &uid, &lost_files);
414c9d631a81 Replaced t_push/t_pop calls with T_FRAME() macros. Timo Sirainen <tss@iki.fi>* parents: 6933 diff changeset	897 maildir_sync_deinit(ctx);
7226 e6693a0ec8e1 Renamed T_FRAME_BEGIN/END to T_BEGIN/END. Removed T_FRAME() macro and Timo Sirainen <tss@iki.fi> parents: 7086 diff changeset	898 } T_END;
6886 82420ed23379 If doing a lockless forced sync, track the UID we're trying to find. If we Timo Sirainen <tss@iki.fi> parents: 6884 diff changeset	899
82420ed23379 If doing a lockless forced sync, track the UID we're trying to find. If we Timo Sirainen <tss@iki.fi> parents: 6884 diff changeset	900 if (uid != 0) {
82420ed23379 If doing a lockless forced sync, track the UID we're trying to find. If we Timo Sirainen <tss@iki.fi> parents: 6884 diff changeset	901 /* maybe it's expunged. check again. */
7226 e6693a0ec8e1 Renamed T_FRAME_BEGIN/END to T_BEGIN/END. Removed T_FRAME() macro and Timo Sirainen <tss@iki.fi> parents: 7086 diff changeset	902 T_BEGIN {
6940 414c9d631a81 Replaced t_push/t_pop calls with T_FRAME() macros. Timo Sirainen <tss@iki.fi>* parents: 6933 diff changeset	903 ctx = maildir_sync_context_new(mbox, 0);
414c9d631a81 Replaced t_push/t_pop calls with T_FRAME() macros. Timo Sirainen <tss@iki.fi>* parents: 6933 diff changeset	904 ret = maildir_sync_context(ctx, TRUE, NULL,
414c9d631a81 Replaced t_push/t_pop calls with T_FRAME() macros. Timo Sirainen <tss@iki.fi>* parents: 6933 diff changeset	905 &lost_files);
414c9d631a81 Replaced t_push/t_pop calls with T_FRAME() macros. Timo Sirainen <tss@iki.fi>* parents: 6933 diff changeset	906 maildir_sync_deinit(ctx);
7226 e6693a0ec8e1 Renamed T_FRAME_BEGIN/END to T_BEGIN/END. Removed T_FRAME() macro and Timo Sirainen <tss@iki.fi> parents: 7086 diff changeset	907 } T_END;
6886 82420ed23379 If doing a lockless forced sync, track the UID we're trying to find. If we Timo Sirainen <tss@iki.fi> parents: 6884 diff changeset	908 }
6452 a62923d3c969 Don't sync mailbox immediately when committing transactions. When it's done Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	909 return ret;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	910 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	911
2322 aae574ed7f4c Broke mailbox_sync() into iterator. Timo Sirainen <tss@iki.fi> parents: 2320 diff changeset	912 struct mailbox_sync_context *
aae574ed7f4c Broke mailbox_sync() into iterator. Timo Sirainen <tss@iki.fi> parents: 2320 diff changeset	913 maildir_storage_sync_init(struct mailbox *box, enum mailbox_sync_flags flags)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	914 {
3279 b698ae839a18 Moved mbox/maildir-specific variables from struct index_mailbox to Timo Sirainen <tss@iki.fi> parents: 3203 diff changeset	915 struct maildir_mailbox mbox = (struct maildir_mailbox )box;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	916 struct maildir_sync_context *ctx;
6452 a62923d3c969 Don't sync mailbox immediately when committing transactions. When it's done Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	917 bool lost_files;
2322 aae574ed7f4c Broke mailbox_sync() into iterator. Timo Sirainen <tss@iki.fi> parents: 2320 diff changeset	918 int ret = 0;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	919
4894 24afafbfe47b Make sure the mailbox is opened when transaction is started (fixes deliver). Timo Sirainen <tss@iki.fi> parents: 4848 diff changeset	920 if (!box->opened)
24afafbfe47b Make sure the mailbox is opened when transaction is started (fixes deliver). Timo Sirainen <tss@iki.fi> parents: 4848 diff changeset	921 index_storage_mailbox_open(&mbox->ibox);
4848 967de900c73a Mailbox list indexing and related changes. Currently works only with Timo Sirainen <tss@iki.fi> parents: 4774 diff changeset	922
7291 db65d921b0e1 Check and update sync_last_check in one common function. Timo Sirainen <tss@iki.fi> parents: 7226 diff changeset	923 if (index_mailbox_want_full_sync(&mbox->ibox, flags)) {
7226 e6693a0ec8e1 Renamed T_FRAME_BEGIN/END to T_BEGIN/END. Removed T_FRAME() macro and Timo Sirainen <tss@iki.fi> parents: 7086 diff changeset	924 T_BEGIN {
6940 414c9d631a81 Replaced t_push/t_pop calls with T_FRAME() macros. Timo Sirainen <tss@iki.fi>* parents: 6933 diff changeset	925 ctx = maildir_sync_context_new(mbox, flags);
414c9d631a81 Replaced t_push/t_pop calls with T_FRAME() macros. Timo Sirainen <tss@iki.fi>* parents: 6933 diff changeset	926 ret = maildir_sync_context(ctx, FALSE, NULL,
414c9d631a81 Replaced t_push/t_pop calls with T_FRAME() macros. Timo Sirainen <tss@iki.fi>* parents: 6933 diff changeset	927 &lost_files);
414c9d631a81 Replaced t_push/t_pop calls with T_FRAME() macros. Timo Sirainen <tss@iki.fi>* parents: 6933 diff changeset	928 maildir_sync_deinit(ctx);
7226 e6693a0ec8e1 Renamed T_FRAME_BEGIN/END to T_BEGIN/END. Removed T_FRAME() macro and Timo Sirainen <tss@iki.fi> parents: 7086 diff changeset	929 } T_END;
3436 3c51658d6846 We didn't notice if messages were deleted directly from new/. Timo Sirainen <tss@iki.fi> parents: 3435 diff changeset	930
4152 e2edd333c473 Added MAILBOX_OPEN_KEEP_LOCKED flag to mailbox opening and implemented it Timo Sirainen <tss@iki.fi> parents: 4126 diff changeset	931 i_assert(!maildir_uidlist_is_locked(mbox->uidlist) \|\|
e2edd333c473 Added MAILBOX_OPEN_KEEP_LOCKED flag to mailbox opening and implemented it Timo Sirainen <tss@iki.fi> parents: 4126 diff changeset	932 mbox->ibox.keep_locked);
3447 96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	933
6452 a62923d3c969 Don't sync mailbox immediately when committing transactions. When it's done Timo Sirainen <tss@iki.fi> parents: 6429 diff changeset	934 if (lost_files) {
3436 3c51658d6846 We didn't notice if messages were deleted directly from new/. Timo Sirainen <tss@iki.fi> parents: 3435 diff changeset	935 /* lost some files from new/, see if thery're in cur/ */
6886 82420ed23379 If doing a lockless forced sync, track the UID we're trying to find. If we Timo Sirainen <tss@iki.fi> parents: 6884 diff changeset	936 ret = maildir_storage_sync_force(mbox, 0);
3436 3c51658d6846 We didn't notice if messages were deleted directly from new/. Timo Sirainen <tss@iki.fi> parents: 3435 diff changeset	937 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	938 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	939
8876 88aeadb32151 Added maildir_very_dirty_syncs setting. Timo Sirainen <tss@iki.fi> parents: 8847 diff changeset	940 if (mbox->very_dirty_syncs) {
88aeadb32151 Added maildir_very_dirty_syncs setting. Timo Sirainen <tss@iki.fi> parents: 8847 diff changeset	941 struct mail_index_view_sync_ctx *sync_ctx;
88aeadb32151 Added maildir_very_dirty_syncs setting. Timo Sirainen <tss@iki.fi> parents: 8847 diff changeset	942 bool b;
88aeadb32151 Added maildir_very_dirty_syncs setting. Timo Sirainen <tss@iki.fi> parents: 8847 diff changeset	943
88aeadb32151 Added maildir_very_dirty_syncs setting. Timo Sirainen <tss@iki.fi> parents: 8847 diff changeset	944 if (mbox->flags_view == NULL) {
88aeadb32151 Added maildir_very_dirty_syncs setting. Timo Sirainen <tss@iki.fi> parents: 8847 diff changeset	945 mbox->flags_view =
88aeadb32151 Added maildir_very_dirty_syncs setting. Timo Sirainen <tss@iki.fi> parents: 8847 diff changeset	946 mail_index_view_open(mbox->ibox.index);
88aeadb32151 Added maildir_very_dirty_syncs setting. Timo Sirainen <tss@iki.fi> parents: 8847 diff changeset	947 }
88aeadb32151 Added maildir_very_dirty_syncs setting. Timo Sirainen <tss@iki.fi> parents: 8847 diff changeset	948 sync_ctx = mail_index_view_sync_begin(mbox->flags_view,
88aeadb32151 Added maildir_very_dirty_syncs setting. Timo Sirainen <tss@iki.fi> parents: 8847 diff changeset	949 MAIL_INDEX_VIEW_SYNC_FLAG_FIX_INCONSISTENT);
88aeadb32151 Added maildir_very_dirty_syncs setting. Timo Sirainen <tss@iki.fi> parents: 8847 diff changeset	950 if (mail_index_view_sync_commit(&sync_ctx, &b) < 0) {
88aeadb32151 Added maildir_very_dirty_syncs setting. Timo Sirainen <tss@iki.fi> parents: 8847 diff changeset	951 mail_storage_set_index_error(&mbox->ibox);
88aeadb32151 Added maildir_very_dirty_syncs setting. Timo Sirainen <tss@iki.fi> parents: 8847 diff changeset	952 ret = -1;
88aeadb32151 Added maildir_very_dirty_syncs setting. Timo Sirainen <tss@iki.fi> parents: 8847 diff changeset	953 }
88aeadb32151 Added maildir_very_dirty_syncs setting. Timo Sirainen <tss@iki.fi> parents: 8847 diff changeset	954 /* make sure the map stays in private memory */
88aeadb32151 Added maildir_very_dirty_syncs setting. Timo Sirainen <tss@iki.fi> parents: 8847 diff changeset	955 if (mbox->flags_view->map->refcount > 1) {
88aeadb32151 Added maildir_very_dirty_syncs setting. Timo Sirainen <tss@iki.fi> parents: 8847 diff changeset	956 struct mail_index_map *map;
88aeadb32151 Added maildir_very_dirty_syncs setting. Timo Sirainen <tss@iki.fi> parents: 8847 diff changeset	957
88aeadb32151 Added maildir_very_dirty_syncs setting. Timo Sirainen <tss@iki.fi> parents: 8847 diff changeset	958 map = mail_index_map_clone(mbox->flags_view->map);
88aeadb32151 Added maildir_very_dirty_syncs setting. Timo Sirainen <tss@iki.fi> parents: 8847 diff changeset	959 mail_index_unmap(&mbox->flags_view->map);
88aeadb32151 Added maildir_very_dirty_syncs setting. Timo Sirainen <tss@iki.fi> parents: 8847 diff changeset	960 mbox->flags_view->map = map;
88aeadb32151 Added maildir_very_dirty_syncs setting. Timo Sirainen <tss@iki.fi> parents: 8847 diff changeset	961 }
88aeadb32151 Added maildir_very_dirty_syncs setting. Timo Sirainen <tss@iki.fi> parents: 8847 diff changeset	962 mail_index_record_map_move_to_private(mbox->flags_view->map);
88aeadb32151 Added maildir_very_dirty_syncs setting. Timo Sirainen <tss@iki.fi> parents: 8847 diff changeset	963 mail_index_map_move_to_memory(mbox->flags_view->map);
88aeadb32151 Added maildir_very_dirty_syncs setting. Timo Sirainen <tss@iki.fi> parents: 8847 diff changeset	964 maildir_uidlist_set_all_nonsynced(mbox->uidlist);
88aeadb32151 Added maildir_very_dirty_syncs setting. Timo Sirainen <tss@iki.fi> parents: 8847 diff changeset	965 }
2322 aae574ed7f4c Broke mailbox_sync() into iterator. Timo Sirainen <tss@iki.fi> parents: 2320 diff changeset	966 return index_mailbox_sync_init(box, flags, ret < 0);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	967 }
3472 db29cc6754d5 Store new/ directory's timestamp in sync_size header in index (kludgy..). Timo Sirainen <tss@iki.fi> parents: 3470 diff changeset	968
db29cc6754d5 Store new/ directory's timestamp in sync_size header in index (kludgy..). Timo Sirainen <tss@iki.fi> parents: 3470 diff changeset	969 int maildir_sync_is_synced(struct maildir_mailbox *mbox)
db29cc6754d5 Store new/ directory's timestamp in sync_size header in index (kludgy..). Timo Sirainen <tss@iki.fi> parents: 3470 diff changeset	970 {
3863 55df57c028d4 Added "bool" type and changed all ints that were used as booleans to bool. Timo Sirainen <tss@iki.fi> parents: 3821 diff changeset	971 bool new_changed, cur_changed;
55df57c028d4 Added "bool" type and changed all ints that were used as booleans to bool. Timo Sirainen <tss@iki.fi> parents: 3821 diff changeset	972 int ret;
3472 db29cc6754d5 Store new/ directory's timestamp in sync_size header in index (kludgy..). Timo Sirainen <tss@iki.fi> parents: 3470 diff changeset	973
7226 e6693a0ec8e1 Renamed T_FRAME_BEGIN/END to T_BEGIN/END. Removed T_FRAME() macro and Timo Sirainen <tss@iki.fi> parents: 7086 diff changeset	974 T_BEGIN {
6940 414c9d631a81 Replaced t_push/t_pop calls with T_FRAME() macros. Timo Sirainen <tss@iki.fi>* parents: 6933 diff changeset	975 const char new_dir, cur_dir;
3472 db29cc6754d5 Store new/ directory's timestamp in sync_size header in index (kludgy..). Timo Sirainen <tss@iki.fi> parents: 3470 diff changeset	976
6940 414c9d631a81 Replaced t_push/t_pop calls with T_FRAME() macros. Timo Sirainen <tss@iki.fi>* parents: 6933 diff changeset	977 new_dir = t_strconcat(mbox->path, "/new", NULL);
414c9d631a81 Replaced t_push/t_pop calls with T_FRAME() macros. Timo Sirainen <tss@iki.fi>* parents: 6933 diff changeset	978 cur_dir = t_strconcat(mbox->path, "/cur", NULL);
414c9d631a81 Replaced t_push/t_pop calls with T_FRAME() macros. Timo Sirainen <tss@iki.fi>* parents: 6933 diff changeset	979
7066 048325df1ec4 If MAILBOX_SYNC_FLAG_FULL_READ flag is set (SELECT and CHECK IMAP commands), Timo Sirainen <tss@iki.fi> parents: 7046 diff changeset	980 ret = maildir_sync_quick_check(mbox, FALSE, new_dir, cur_dir,
6940 414c9d631a81 Replaced t_push/t_pop calls with T_FRAME() macros. Timo Sirainen <tss@iki.fi>* parents: 6933 diff changeset	981 &new_changed, &cur_changed);
7226 e6693a0ec8e1 Renamed T_FRAME_BEGIN/END to T_BEGIN/END. Removed T_FRAME() macro and Timo Sirainen <tss@iki.fi> parents: 7086 diff changeset	982 } T_END;
3472 db29cc6754d5 Store new/ directory's timestamp in sync_size header in index (kludgy..). Timo Sirainen <tss@iki.fi> parents: 3470 diff changeset	983 return ret < 0 ? -1 : (!new_changed && !cur_changed);
db29cc6754d5 Store new/ directory's timestamp in sync_size header in index (kludgy..). Timo Sirainen <tss@iki.fi> parents: 3470 diff changeset	984 }

Mercurial > dovecot > original-hg > dovecot-1.2

annotate src/lib-storage/index/maildir/maildir-sync.c @ 9202:797474444551 HEAD