dovecot/original-hg/dovecot-1.2: src/lib-storage/index/maildir/maildir-sync.c annotate

annotate src/lib-storage/index/maildir/maildir-sync.c @ 5588:6c89106dee21 HEAD

Keyword characters weren't sorted in the maildir filename.

author	Timo Sirainen <tss@iki.fi>
date	Fri, 11 May 2007 16:14:10 +0300
parents	1c4fd25893bd
children	453128e12b11

rev	line source
1955 0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	1 /* Copyright (C) 2004 Timo Sirainen */
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	2
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	3 /*
1955 0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	4 Here's a description of how we handle Maildir synchronization and
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	5 it's problems:
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	6
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	7 We want to be as efficient as we can. The most efficient way to
3520 e2fe8222449d s/occured/occurred/ Timo Sirainen <tss@iki.fi> parents: 3472 diff changeset	8 check if changes have occurred is to stat() the new/ and cur/
1955 0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	9 directories and uidlist file - if their mtimes haven't changed,
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	10 there's no changes and we don't need to do anything.
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	11
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	12 Problem 1: Multiple changes can happen within a single second -
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	13 nothing guarantees that once we synced it, someone else didn't just
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	14 then make a modification. Such modifications wouldn't get noticed
3520 e2fe8222449d s/occured/occurred/ Timo Sirainen <tss@iki.fi> parents: 3472 diff changeset	15 until a new modification occurred later.
1955 0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	16
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	17 Problem 2: Syncing cur/ directory is much more costly than syncing
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	18 new/. Moving mails from new/ to cur/ will always change mtime of
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	19 cur/ causing us to sync it as well.
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	20
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	21 Problem 3: We may not be able to move mail from new/ to cur/
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	22 because we're out of quota, or simply because we're accessing a
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	23 read-only mailbox.
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	24
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	25
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	26 MAILDIR_SYNC_SECS
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	27 -----------------
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	28
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	29 Several checks below use MAILDIR_SYNC_SECS, which should be maximum
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	30 clock drift between all computers accessing the maildir (eg. via
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	31 NFS), rounded up to next second. Our default is 1 second, since
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	32 everyone should be using NTP.
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	33
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	34 Note that setting it to 0 works only if there's only one computer
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	35 accessing the maildir. It's practically impossible to make two
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	36 clocks _exactly_ synchronized.
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	37
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	38 It might be possible to only use file server's clock by looking at
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	39 the atime field, but I don't know how well that would actually work.
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	40
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	41 cur directory
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	42 -------------
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	43
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	44 We have dirty_cur_time variable which is set to cur/ directory's
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	45 mtime when it's >= time() - MAILDIR_SYNC_SECS and we _think_ we have
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	46 synchronized the directory.
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	47
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	48 When dirty_cur_time is non-zero, we don't synchronize the cur/
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	49 directory until
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	50
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	51 a) cur/'s mtime changes
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	52 b) opening a mail fails with ENOENT
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	53 c) time() > dirty_cur_time + MAILDIR_SYNC_SECS
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	54
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	55 This allows us to modify the maildir multiple times without having
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	56 to sync it at every change. The sync will eventually be done to
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	57 make sure we didn't miss any external changes.
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	58
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	59 The dirty_cur_time is set when:
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	60
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	61 - we change message flags
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	62 - we expunge messages
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	63 - we move mail from new/ to cur/
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	64 - we sync cur/ directory and it's mtime is >= time() - MAILDIR_SYNC_SECS
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	65
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	66 It's unset when we do the final syncing, ie. when mtime is
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	67 older than time() - MAILDIR_SYNC_SECS.
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	68
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	69 new directory
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	70 -------------
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	71
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	72 If new/'s mtime is >= time() - MAILDIR_SYNC_SECS, always synchronize
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	73 it. dirty_cur_time-like feature might save us a few syncs, but
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	74 that might break a client which saves a mail in one connection and
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	75 tries to fetch it in another one. new/ directory is almost always
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	76 empty, so syncing it should be very fast anyway. Actually this can
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	77 still happen if we sync only new/ dir while another client is also
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	78 moving mails from it to cur/ - it takes us a while to see them.
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	79 That's pretty unlikely to happen however, and only way to fix it
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	80 would be to always synchronize cur/ after new/.
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	81
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	82 Normally we move all mails from new/ to cur/ whenever we sync it. If
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	83 it's not possible for some reason, we mark the mail with "probably
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	84 exists in new/ directory" flag.
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	85
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	86 If rename() still fails because of ENOSPC or EDQUOT, we still save
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	87 the flag changes in index with dirty-flag on. When moving the mail
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	88 to cur/ directory, or when we notice it's already moved there, we
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	89 apply the flag changes to the filename, rename it and remove the
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	90 dirty flag. If there's dirty flags, this should be tried every time
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	91 after expunge or when closing the mailbox.
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	92
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	93 uidlist
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	94 -------
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	95
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	96 This file contains UID <-> filename mappings. It's updated only when
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	97 new mail arrives, so it may contain filenames that have already been
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	98 deleted. Updating is done by getting uidlist.lock file, writing the
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	99 whole uidlist into it and rename()ing it over the old uidlist. This
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	100 means there's no need to lock the file for reading.
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	101
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	102 Whenever uidlist is rewritten, it's mtime must be larger than the old
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	103 one's. Use utime() before rename() if needed. Note that inode checking
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	104 wouldn't have been sufficient as inode numbers can be reused.
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	105
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	106 This file is usually read the first time you need to know filename for
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	107 given UID. After that it's not re-read unless new mails come that we
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	108 don't know about.
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	109
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	110 broken clients
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	111 --------------
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	112
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	113 Originally the middle identifier in Maildir filename was specified
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	114 only as <process id>_<delivery counter>. That however created a
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	115 problem with randomized PIDs which made it possible that the same
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	116 PID was reused within one second.
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	117
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	118 So if within one second a mail was delivered, MUA moved it to cur/
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	119 and another mail was delivered by a new process using same PID as
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	120 the first one, we likely ended up overwriting the first mail when
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	121 the second mail was moved over it.
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	122
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	123 Nowadays everyone should be giving a bit more specific identifier,
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	124 for example include microseconds in it which Dovecot does.
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	125
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	126 There's a simple way to prevent this from happening in some cases:
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	127 Don't move the mail from new/ to cur/ if it's mtime is >= time() -
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	128 MAILDIR_SYNC_SECS. The second delivery's link() call then fails
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	129 because the file is already in new/, and it will then use a
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	130 different filename. There's a few problems with this however:
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	131
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	132 - it requires extra stat() call which is unneeded extra I/O
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	133 - another MUA might still move the mail to cur/
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	134 - if first file's flags are modified by either Dovecot or another
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	135 MUA, it's moved to cur/ (you _could_ just do the dirty-flagging
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	136 but that'd be ugly)
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	137
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	138 Because this is useful only for very few people and it requires
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	139 extra I/O, I decided not to implement this. It should be however
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	140 quite easy to do since we need to be able to deal with files in new/
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	141 in any case.
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	142
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	143 It's also possible to never accidentally overwrite a mail by using
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	144 link() + unlink() rather than rename(). This however isn't very
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	145 good idea as it introduces potential race conditions when multiple
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	146 clients are accessing the mailbox:
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	147
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	148 Trying to move the same mail from new/ to cur/ at the same time:
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	149
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	150 a) Client 1 uses slightly different filename than client 2,
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	151 for example one sets read-flag on but the other doesn't.
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	152 You have the same mail duplicated now.
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	153
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	154 b) Client 3 sees the mail between Client 1's and 2's link() calls
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	155 and changes it's flag. You have the same mail duplicated now.
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	156
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	157 And it gets worse when they're unlink()ing in cur/ directory:
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	158
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	159 c) Client 1 changes mails's flag and client 2 changes it back
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	160 between 1's link() and unlink(). The mail is now expunged.
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	161
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	162 d) If you try to deal with the duplicates by unlink()ing another
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	163 one of them, you might end up unlinking both of them.
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	164
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	165 So, what should we do then if we notice a duplicate? First of all,
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	166 it might not be a duplicate at all, readdir() might have just
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	167 returned it twice because it was just renamed. What we should do is
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	168 create a completely new base name for it and rename() it to that.
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	169 If the call fails with ENOENT, it only means that it wasn't a
0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	170 duplicate after all.
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	171 */
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	172
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	173 #include "lib.h"
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	174 #include "ioloop.h"
3470 346a494c2feb Moved array declaration to array-decl.h and include it in lib.h. So array.h Timo Sirainen <tss@iki.fi> parents: 3453 diff changeset	175 #include "array.h"
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	176 #include "buffer.h"
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	177 #include "hash.h"
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	178 #include "str.h"
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	179 #include "maildir-storage.h"
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	180 #include "maildir-uidlist.h"
3447 96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	181 #include "maildir-keywords.h"
4848 967de900c73a Mailbox list indexing and related changes. Currently works only with Timo Sirainen <tss@iki.fi> parents: 4774 diff changeset	182 #include "maildir-sync.h"
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	183
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	184 #include <stdio.h>
2050 ee1095ccfd23 Index header changes now go through transaction log. Removed the kludgy Timo Sirainen <tss@iki.fi> parents: 2039 diff changeset	185 #include <stddef.h>
5588 6c89106dee21 Keyword characters weren't sorted in the maildir filename. Timo Sirainen <tss@iki.fi> parents: 5582 diff changeset	186 #include <stdlib.h>
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	187 #include <unistd.h>
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	188 #include <dirent.h>
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	189 #include <sys/stat.h>
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	190
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	191 #define MAILDIR_FILENAME_FLAG_FOUND 128
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	192
3435 6f7ce690358e If we have rename()d more than 5 files from new/ to cur/, rescan the Timo Sirainen <tss@iki.fi> parents: 3417 diff changeset	193 /* When rename()ing many files from new/ to cur/, it's possible that next
6f7ce690358e If we have rename()d more than 5 files from new/ to cur/, rescan the Timo Sirainen <tss@iki.fi> parents: 3417 diff changeset	194 readdir() skips some files. we don't of course wish to lose them, so we
6f7ce690358e If we have rename()d more than 5 files from new/ to cur/, rescan the Timo Sirainen <tss@iki.fi> parents: 3417 diff changeset	195 go and rescan the new/ directory again from beginning until no files are
6f7ce690358e If we have rename()d more than 5 files from new/ to cur/, rescan the Timo Sirainen <tss@iki.fi> parents: 3417 diff changeset	196 left. This value is just an optimization to avoid checking the directory
6f7ce690358e If we have rename()d more than 5 files from new/ to cur/, rescan the Timo Sirainen <tss@iki.fi> parents: 3417 diff changeset	197 twice unneededly. usually only NFS is the problem case. 1 is the safest
6f7ce690358e If we have rename()d more than 5 files from new/ to cur/, rescan the Timo Sirainen <tss@iki.fi> parents: 3417 diff changeset	198 bet here, but I guess 5 will do just fine too. */
6f7ce690358e If we have rename()d more than 5 files from new/ to cur/, rescan the Timo Sirainen <tss@iki.fi> parents: 3417 diff changeset	199 #define MAILDIR_RENAME_RESCAN_COUNT 5
6f7ce690358e If we have rename()d more than 5 files from new/ to cur/, rescan the Timo Sirainen <tss@iki.fi> parents: 3417 diff changeset	200
5214 4e9d345df846 When syncing huge maildirs check once in a while if we need to update Timo Sirainen <tss@iki.fi> parents: 5080 diff changeset	201 /* After moving 100 mails from new/ to cur/, check if we need to touch the
4e9d345df846 When syncing huge maildirs check once in a while if we need to update Timo Sirainen <tss@iki.fi> parents: 5080 diff changeset	202 uidlist lock. */
4e9d345df846 When syncing huge maildirs check once in a while if we need to update Timo Sirainen <tss@iki.fi> parents: 5080 diff changeset	203 #define MAILDIR_SLOW_MOVE_COUNT 100
4e9d345df846 When syncing huge maildirs check once in a while if we need to update Timo Sirainen <tss@iki.fi> parents: 5080 diff changeset	204 /* readdir() should be pretty fast to do, but check anyway every 10000 mails
4e9d345df846 When syncing huge maildirs check once in a while if we need to update Timo Sirainen <tss@iki.fi> parents: 5080 diff changeset	205 to see if we need to touch the uidlist lock. */
4e9d345df846 When syncing huge maildirs check once in a while if we need to update Timo Sirainen <tss@iki.fi> parents: 5080 diff changeset	206 #define MAILDIR_SLOW_CHECK_COUNT 10000
4e9d345df846 When syncing huge maildirs check once in a while if we need to update Timo Sirainen <tss@iki.fi> parents: 5080 diff changeset	207
5397 3a0964ac3a5c comment/duplicate link handling fixes. Timo Sirainen <tss@iki.fi> parents: 5392 diff changeset	208 /* This is mostly to avoid infinite looping when rename() destination already
3a0964ac3a5c comment/duplicate link handling fixes. Timo Sirainen <tss@iki.fi> parents: 5392 diff changeset	209 exists as the hard link of the file itself. */
5391 0c8705aad54c Avoid infinite looping when buggy filesystems. Timo Sirainen <tss@iki.fi> parents: 5390 diff changeset	210 #define MAILDIR_SCAN_DIR_MAX_COUNT 5
0c8705aad54c Avoid infinite looping when buggy filesystems. Timo Sirainen <tss@iki.fi> parents: 5390 diff changeset	211
5443 293bc7d1062f If we find duplicate hard links which haven't changed for 30 secs, unlink() Timo Sirainen <tss@iki.fi> parents: 5397 diff changeset	212 #define DUPE_LINKS_DELETE_SECS 30
293bc7d1062f If we find duplicate hard links which haven't changed for 30 secs, unlink() Timo Sirainen <tss@iki.fi> parents: 5397 diff changeset	213
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	214 struct maildir_sync_context {
3279 b698ae839a18 Moved mbox/maildir-specific variables from struct index_mailbox to Timo Sirainen <tss@iki.fi> parents: 3203 diff changeset	215 struct maildir_mailbox *mbox;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	216 const char new_dir, cur_dir;
3863 55df57c028d4 Added "bool" type and changed all ints that were used as booleans to bool. Timo Sirainen <tss@iki.fi> parents: 3821 diff changeset	217 bool partial;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	218
5368 7d45edb81fe4 When copying/syncing a lot of mails, send "* OK Hang in there" replies to Timo Sirainen <tss@iki.fi> parents: 5307 diff changeset	219 unsigned int move_count, check_count;
7d45edb81fe4 When copying/syncing a lot of mails, send "* OK Hang in there" replies to Timo Sirainen <tss@iki.fi> parents: 5307 diff changeset	220 time_t last_touch, last_notify;
7d45edb81fe4 When copying/syncing a lot of mails, send "* OK Hang in there" replies to Timo Sirainen <tss@iki.fi> parents: 5307 diff changeset	221
2818 a758a5b542bb Always protect maildir syncing with uidlist lock. Before we only tried to Timo Sirainen <tss@iki.fi> parents: 2816 diff changeset	222 struct maildir_uidlist_sync_ctx *uidlist_sync_ctx;
a758a5b542bb Always protect maildir syncing with uidlist lock. Before we only tried to Timo Sirainen <tss@iki.fi> parents: 2816 diff changeset	223 struct maildir_index_sync_context *index_sync_ctx;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	224 };
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	225
1956 d6941cd8afdc Added support for setting dirty flags for messages (TODO: undirty..) Timo Sirainen <tss@iki.fi> parents: 1955 diff changeset	226 struct maildir_index_sync_context {
3279 b698ae839a18 Moved mbox/maildir-specific variables from struct index_mailbox to Timo Sirainen <tss@iki.fi> parents: 3203 diff changeset	227 struct maildir_mailbox *mbox;
5368 7d45edb81fe4 When copying/syncing a lot of mails, send "* OK Hang in there" replies to Timo Sirainen <tss@iki.fi> parents: 5307 diff changeset	228 struct maildir_sync_context *maildir_sync_ctx;
7d45edb81fe4 When copying/syncing a lot of mails, send "* OK Hang in there" replies to Timo Sirainen <tss@iki.fi> parents: 5307 diff changeset	229
2037 8763032d31bd Set dirty flags through transaction log, not directly. Some other flag Timo Sirainen <tss@iki.fi> parents: 2034 diff changeset	230 struct mail_index_view *view;
1956 d6941cd8afdc Added support for setting dirty flags for messages (TODO: undirty..) Timo Sirainen <tss@iki.fi> parents: 1955 diff changeset	231 struct mail_index_sync_ctx *sync_ctx;
3447 96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	232 struct maildir_keywords_sync_ctx *keywords_sync_ctx;
2037 8763032d31bd Set dirty flags through transaction log, not directly. Some other flag Timo Sirainen <tss@iki.fi> parents: 2034 diff changeset	233 struct mail_index_transaction *trans;
1956 d6941cd8afdc Added support for setting dirty flags for messages (TODO: undirty..) Timo Sirainen <tss@iki.fi> parents: 1955 diff changeset	234
4451 1a35d53c18fc Array API redesigned to work using unions. It now provides type safety Timo Sirainen <tss@iki.fi> parents: 4450 diff changeset	235 ARRAY_DEFINE(sync_recs, struct mail_index_sync_rec);
5562 24b751bc0995 Added sync_notify() callback to struct mail_storage. It's now called for Timo Sirainen <tss@iki.fi> parents: 5459 diff changeset	236 uint32_t seq, uid;
2272 ced88553af0b mail_index_sync_sort_flags() now merges flag changes so mail storage Timo Sirainen <tss@iki.fi> parents: 2258 diff changeset	237 int dirty_state;
1956 d6941cd8afdc Added support for setting dirty flags for messages (TODO: undirty..) Timo Sirainen <tss@iki.fi> parents: 1955 diff changeset	238 };
d6941cd8afdc Added support for setting dirty flags for messages (TODO: undirty..) Timo Sirainen <tss@iki.fi> parents: 1955 diff changeset	239
3775 8321e6191275 maildir_copy_with_hardlinks works again. Timo Sirainen <tss@iki.fi> parents: 3583 diff changeset	240 struct maildir_keywords_sync_ctx *
8321e6191275 maildir_copy_with_hardlinks works again. Timo Sirainen <tss@iki.fi> parents: 3583 diff changeset	241 maildir_sync_get_keywords_sync_ctx(struct maildir_index_sync_context *ctx)
8321e6191275 maildir_copy_with_hardlinks works again. Timo Sirainen <tss@iki.fi> parents: 3583 diff changeset	242 {
8321e6191275 maildir_copy_with_hardlinks works again. Timo Sirainen <tss@iki.fi> parents: 3583 diff changeset	243 return ctx->keywords_sync_ctx;
8321e6191275 maildir_copy_with_hardlinks works again. Timo Sirainen <tss@iki.fi> parents: 3583 diff changeset	244 }
8321e6191275 maildir_copy_with_hardlinks works again. Timo Sirainen <tss@iki.fi> parents: 3583 diff changeset	245
8321e6191275 maildir_copy_with_hardlinks works again. Timo Sirainen <tss@iki.fi> parents: 3583 diff changeset	246 int maildir_filename_get_flags(struct maildir_keywords_sync_ctx *ctx,
8321e6191275 maildir_copy_with_hardlinks works again. Timo Sirainen <tss@iki.fi> parents: 3583 diff changeset	247 const char fname, enum mail_flags flags_r,
4451 1a35d53c18fc Array API redesigned to work using unions. It now provides type safety Timo Sirainen <tss@iki.fi> parents: 4450 diff changeset	248 ARRAY_TYPE(keyword_indexes) *keywords_r)
3447 96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	249 {
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	250 const char *info;
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	251
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	252 array_clear(keywords_r);
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	253 *flags_r = 0;
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	254
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	255 info = strchr(fname, MAILDIR_INFO_SEP);
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	256 if (info == NULL \|\| info[1] != '2' \|\| info[2] != MAILDIR_FLAGS_SEP)
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	257 return 0;
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	258
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	259 for (info += 3; info != '\0' && info != MAILDIR_FLAGS_SEP; info++) {
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	260 switch (*info) {
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	261 case 'R': /* replied */
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	262 *flags_r \|= MAIL_ANSWERED;
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	263 break;
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	264 case 'S': /* seen */
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	265 *flags_r \|= MAIL_SEEN;
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	266 break;
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	267 case 'T': /* trashed */
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	268 *flags_r \|= MAIL_DELETED;
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	269 break;
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	270 case 'D': /* draft */
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	271 *flags_r \|= MAIL_DRAFT;
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	272 break;
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	273 case 'F': /* flagged */
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	274 *flags_r \|= MAIL_FLAGGED;
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	275 break;
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	276 default:
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	277 if (*info >= MAILDIR_KEYWORD_FIRST &&
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	278 *info <= MAILDIR_KEYWORD_LAST) {
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	279 int idx;
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	280
3775 8321e6191275 maildir_copy_with_hardlinks works again. Timo Sirainen <tss@iki.fi> parents: 3583 diff changeset	281 idx = maildir_keywords_char_idx(ctx, *info);
3447 96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	282 if (idx < 0) {
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	283 /* unknown keyword. */
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	284 break;
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	285 }
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	286
3563 505a5f3f3857 Compiler warning fix Timo Sirainen <tss@iki.fi> parents: 3530 diff changeset	287 array_append(keywords_r,
505a5f3f3857 Compiler warning fix Timo Sirainen <tss@iki.fi> parents: 3530 diff changeset	288 (unsigned int *)&idx, 1);
3447 96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	289 break;
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	290 }
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	291
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	292 /* unknown flag - ignore */
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	293 break;
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	294 }
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	295 }
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	296
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	297 return 1;
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	298 }
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	299
5588 6c89106dee21 Keyword characters weren't sorted in the maildir filename. Timo Sirainen <tss@iki.fi> parents: 5582 diff changeset	300 static int char_cmp(const void p1, const void p2)
6c89106dee21 Keyword characters weren't sorted in the maildir filename. Timo Sirainen <tss@iki.fi> parents: 5582 diff changeset	301 {
6c89106dee21 Keyword characters weren't sorted in the maildir filename. Timo Sirainen <tss@iki.fi> parents: 5582 diff changeset	302 const unsigned char c1 = p1, c2 = p2;
6c89106dee21 Keyword characters weren't sorted in the maildir filename. Timo Sirainen <tss@iki.fi> parents: 5582 diff changeset	303
6c89106dee21 Keyword characters weren't sorted in the maildir filename. Timo Sirainen <tss@iki.fi> parents: 5582 diff changeset	304 return c1 - c2;
6c89106dee21 Keyword characters weren't sorted in the maildir filename. Timo Sirainen <tss@iki.fi> parents: 5582 diff changeset	305 }
6c89106dee21 Keyword characters weren't sorted in the maildir filename. Timo Sirainen <tss@iki.fi> parents: 5582 diff changeset	306
3447 96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	307 static void
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	308 maildir_filename_append_keywords(struct maildir_keywords_sync_ctx *ctx,
4451 1a35d53c18fc Array API redesigned to work using unions. It now provides type safety Timo Sirainen <tss@iki.fi> parents: 4450 diff changeset	309 ARRAY_TYPE(keyword_indexes) *keywords,
1a35d53c18fc Array API redesigned to work using unions. It now provides type safety Timo Sirainen <tss@iki.fi> parents: 4450 diff changeset	310 string_t *str)
3447 96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	311 {
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	312 const unsigned int *indexes;
5588 6c89106dee21 Keyword characters weren't sorted in the maildir filename. Timo Sirainen <tss@iki.fi> parents: 5582 diff changeset	313 unsigned int i, count, start = str_len(str);
3447 96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	314 char chr;
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	315
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	316 indexes = array_get(keywords, &count);
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	317 for (i = 0; i < count; i++) {
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	318 chr = maildir_keywords_idx_char(ctx, indexes[i]);
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	319 if (chr != '\0')
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	320 str_append_c(str, chr);
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	321 }
5588 6c89106dee21 Keyword characters weren't sorted in the maildir filename. Timo Sirainen <tss@iki.fi> parents: 5582 diff changeset	322
6c89106dee21 Keyword characters weren't sorted in the maildir filename. Timo Sirainen <tss@iki.fi> parents: 5582 diff changeset	323 qsort(str_c_modifiable(str) + start, str_len(str) - start, 1, char_cmp);
3447 96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	324 }
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	325
3775 8321e6191275 maildir_copy_with_hardlinks works again. Timo Sirainen <tss@iki.fi> parents: 3583 diff changeset	326 const char maildir_filename_set_flags(struct maildir_keywords_sync_ctx ctx,
3447 96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	327 const char *fname, enum mail_flags flags,
4451 1a35d53c18fc Array API redesigned to work using unions. It now provides type safety Timo Sirainen <tss@iki.fi> parents: 4450 diff changeset	328 ARRAY_TYPE(keyword_indexes) *keywords)
3447 96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	329 {
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	330 string_t *flags_str;
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	331 enum mail_flags flags_left;
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	332 const char info, oldflags;
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	333 int nextflag;
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	334
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	335 /* remove the old :info from file name, and get the old flags */
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	336 info = strrchr(fname, MAILDIR_INFO_SEP);
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	337 if (info != NULL && strrchr(fname, '/') > info)
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	338 info = NULL;
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	339
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	340 oldflags = "";
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	341 if (info != NULL) {
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	342 fname = t_strdup_until(fname, info);
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	343 if (info[1] == '2' && info[2] == MAILDIR_FLAGS_SEP)
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	344 oldflags = info+3;
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	345 }
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	346
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	347 /* insert the new flags between old flags. flags must be sorted by
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	348 their ASCII code. unknown flags are kept. */
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	349 flags_str = t_str_new(256);
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	350 str_append(flags_str, fname);
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	351 str_append(flags_str, MAILDIR_FLAGS_FULL_SEP);
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	352 flags_left = flags;
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	353 for (;;) {
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	354 /* skip all known flags */
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	355 while (oldflags == 'D' \|\| oldflags == 'F' \|\|
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	356 oldflags == 'R' \|\| oldflags == 'S' \|\|
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	357 *oldflags == 'T' \|\|
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	358 (*oldflags >= MAILDIR_KEYWORD_FIRST &&
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	359 *oldflags <= MAILDIR_KEYWORD_LAST))
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	360 oldflags++;
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	361
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	362 nextflag = oldflags == '\0' \|\| oldflags == MAILDIR_FLAGS_SEP ?
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	363 256 : (unsigned char) *oldflags;
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	364
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	365 if ((flags_left & MAIL_DRAFT) && nextflag > 'D') {
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	366 str_append_c(flags_str, 'D');
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	367 flags_left &= ~MAIL_DRAFT;
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	368 }
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	369 if ((flags_left & MAIL_FLAGGED) && nextflag > 'F') {
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	370 str_append_c(flags_str, 'F');
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	371 flags_left &= ~MAIL_FLAGGED;
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	372 }
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	373 if ((flags_left & MAIL_ANSWERED) && nextflag > 'R') {
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	374 str_append_c(flags_str, 'R');
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	375 flags_left &= ~MAIL_ANSWERED;
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	376 }
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	377 if ((flags_left & MAIL_SEEN) && nextflag > 'S') {
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	378 str_append_c(flags_str, 'S');
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	379 flags_left &= ~MAIL_SEEN;
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	380 }
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	381 if ((flags_left & MAIL_DELETED) && nextflag > 'T') {
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	382 str_append_c(flags_str, 'T');
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	383 flags_left &= ~MAIL_DELETED;
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	384 }
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	385
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	386 if (keywords != NULL && array_is_created(keywords) &&
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	387 nextflag > MAILDIR_KEYWORD_FIRST) {
3775 8321e6191275 maildir_copy_with_hardlinks works again. Timo Sirainen <tss@iki.fi> parents: 3583 diff changeset	388 maildir_filename_append_keywords(ctx, keywords,
8321e6191275 maildir_copy_with_hardlinks works again. Timo Sirainen <tss@iki.fi> parents: 3583 diff changeset	389 flags_str);
3447 96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	390 keywords = NULL;
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	391 }
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	392
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	393 if (oldflags == '\0' \|\| oldflags == MAILDIR_FLAGS_SEP)
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	394 break;
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	395
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	396 str_append_c(flags_str, *oldflags);
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	397 oldflags++;
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	398 }
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	399
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	400 if (*oldflags == MAILDIR_FLAGS_SEP) {
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	401 /* another flagset, we don't know about these, just keep them */
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	402 while (*oldflags != '\0')
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	403 str_append_c(flags_str, *oldflags++);
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	404 }
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	405
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	406 return str_c(flags_str);
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	407 }
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	408
3279 b698ae839a18 Moved mbox/maildir-specific variables from struct index_mailbox to Timo Sirainen <tss@iki.fi> parents: 3203 diff changeset	409 static int maildir_expunge(struct maildir_mailbox mbox, const char path,
5562 24b751bc0995 Added sync_notify() callback to struct mail_storage. It's now called for Timo Sirainen <tss@iki.fi> parents: 5459 diff changeset	410 struct maildir_index_sync_context *ctx)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	411 {
5562 24b751bc0995 Added sync_notify() callback to struct mail_storage. It's now called for Timo Sirainen <tss@iki.fi> parents: 5459 diff changeset	412 struct mailbox *box = &mbox->ibox.box;
24b751bc0995 Added sync_notify() callback to struct mail_storage. It's now called for Timo Sirainen <tss@iki.fi> parents: 5459 diff changeset	413
1955 0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	414 if (unlink(path) == 0) {
5562 24b751bc0995 Added sync_notify() callback to struct mail_storage. It's now called for Timo Sirainen <tss@iki.fi> parents: 5459 diff changeset	415 if (box->v.sync_notify != NULL) {
24b751bc0995 Added sync_notify() callback to struct mail_storage. It's now called for Timo Sirainen <tss@iki.fi> parents: 5459 diff changeset	416 box->v.sync_notify(box, ctx->uid,
24b751bc0995 Added sync_notify() callback to struct mail_storage. It's now called for Timo Sirainen <tss@iki.fi> parents: 5459 diff changeset	417 MAILBOX_SYNC_TYPE_EXPUNGE);
24b751bc0995 Added sync_notify() callback to struct mail_storage. It's now called for Timo Sirainen <tss@iki.fi> parents: 5459 diff changeset	418 }
3279 b698ae839a18 Moved mbox/maildir-specific variables from struct index_mailbox to Timo Sirainen <tss@iki.fi> parents: 3203 diff changeset	419 mbox->dirty_cur_time = ioloop_time;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	420 return 1;
1955 0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	421 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	422 if (errno == ENOENT)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	423 return 0;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	424
5459 78eaf595359c Removed struct index_storage abstraction. It's pointless. Timo Sirainen <tss@iki.fi> parents: 5443 diff changeset	425 mail_storage_set_critical(&mbox->storage->storage,
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	426 "unlink(%s) failed: %m", path);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	427 return -1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	428 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	429
3279 b698ae839a18 Moved mbox/maildir-specific variables from struct index_mailbox to Timo Sirainen <tss@iki.fi> parents: 3203 diff changeset	430 static int maildir_sync_flags(struct maildir_mailbox mbox, const char path,
4907 5b4c9b20eba0 Replaced void context from a lot of callbacks with the actual context Timo Sirainen <tss@iki.fi>* parents: 4894 diff changeset	431 struct maildir_index_sync_context *ctx)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	432 {
5562 24b751bc0995 Added sync_notify() callback to struct mail_storage. It's now called for Timo Sirainen <tss@iki.fi> parents: 5459 diff changeset	433 struct mailbox *box = &mbox->ibox.box;
3446 113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	434 const struct mail_index_sync_rec *recs;
4450 14b10f7ea70e Don't break if mailbox path contains ':' characters. Timo Sirainen <tss@iki.fi> parents: 4397 diff changeset	435 const char dir, fname, newfname, newpath;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	436 enum mail_flags flags;
5562 24b751bc0995 Added sync_notify() callback to struct mail_storage. It's now called for Timo Sirainen <tss@iki.fi> parents: 5459 diff changeset	437 enum mailbox_sync_type sync_type = 0;
4451 1a35d53c18fc Array API redesigned to work using unions. It now provides type safety Timo Sirainen <tss@iki.fi> parents: 4450 diff changeset	438 ARRAY_TYPE(keyword_indexes) keywords;
3446 113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	439 unsigned int i, count;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	440 uint8_t flags8;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	441
2272 ced88553af0b mail_index_sync_sort_flags() now merges flag changes so mail storage Timo Sirainen <tss@iki.fi> parents: 2258 diff changeset	442 ctx->dirty_state = 0;
2037 8763032d31bd Set dirty flags through transaction log, not directly. Some other flag Timo Sirainen <tss@iki.fi> parents: 2034 diff changeset	443
4450 14b10f7ea70e Don't break if mailbox path contains ':' characters. Timo Sirainen <tss@iki.fi> parents: 4397 diff changeset	444 fname = strrchr(path, '/');
14b10f7ea70e Don't break if mailbox path contains ':' characters. Timo Sirainen <tss@iki.fi> parents: 4397 diff changeset	445 i_assert(fname != NULL);
14b10f7ea70e Don't break if mailbox path contains ':' characters. Timo Sirainen <tss@iki.fi> parents: 4397 diff changeset	446 fname++;
14b10f7ea70e Don't break if mailbox path contains ':' characters. Timo Sirainen <tss@iki.fi> parents: 4397 diff changeset	447 dir = t_strdup_until(path, fname);
14b10f7ea70e Don't break if mailbox path contains ':' characters. Timo Sirainen <tss@iki.fi> parents: 4397 diff changeset	448
4596 bf4e98a0de3f Replaced ARRAY_CREATE() macro with [ipt]_array_init() macros. The macro Timo Sirainen <tss@iki.fi> parents: 4594 diff changeset	449 t_array_init(&keywords, 16);
3775 8321e6191275 maildir_copy_with_hardlinks works again. Timo Sirainen <tss@iki.fi> parents: 3583 diff changeset	450 (void)maildir_filename_get_flags(ctx->keywords_sync_ctx,
4450 14b10f7ea70e Don't break if mailbox path contains ':' characters. Timo Sirainen <tss@iki.fi> parents: 4397 diff changeset	451 fname, &flags, &keywords);
3446 113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	452 flags8 = flags;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	453
4451 1a35d53c18fc Array API redesigned to work using unions. It now provides type safety Timo Sirainen <tss@iki.fi> parents: 4450 diff changeset	454 recs = array_get_modifiable(&ctx->sync_recs, &count);
3446 113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	455 for (i = 0; i < count; i++) {
113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	456 if (recs[i].uid1 != ctx->seq)
113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	457 break;
113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	458
113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	459 switch (recs[i].type) {
113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	460 case MAIL_INDEX_SYNC_TYPE_FLAGS:
113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	461 mail_index_sync_flags_apply(&recs[i], &flags8);
5562 24b751bc0995 Added sync_notify() callback to struct mail_storage. It's now called for Timo Sirainen <tss@iki.fi> parents: 5459 diff changeset	462 sync_type \|= MAILBOX_SYNC_TYPE_FLAGS;
3446 113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	463 break;
113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	464 case MAIL_INDEX_SYNC_TYPE_KEYWORD_ADD:
113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	465 case MAIL_INDEX_SYNC_TYPE_KEYWORD_REMOVE:
113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	466 case MAIL_INDEX_SYNC_TYPE_KEYWORD_RESET:
3447 96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	467 mail_index_sync_keywords_apply(&recs[i], &keywords);
5562 24b751bc0995 Added sync_notify() callback to struct mail_storage. It's now called for Timo Sirainen <tss@iki.fi> parents: 5459 diff changeset	468 sync_type \|= MAILBOX_SYNC_TYPE_KEYWORDS;
3446 113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	469 break;
113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	470 case MAIL_INDEX_SYNC_TYPE_APPEND:
113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	471 case MAIL_INDEX_SYNC_TYPE_EXPUNGE:
113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	472 i_unreached();
113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	473 break;
113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	474 }
113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	475 }
5562 24b751bc0995 Added sync_notify() callback to struct mail_storage. It's now called for Timo Sirainen <tss@iki.fi> parents: 5459 diff changeset	476 i_assert(sync_type != 0);
4450 14b10f7ea70e Don't break if mailbox path contains ':' characters. Timo Sirainen <tss@iki.fi> parents: 4397 diff changeset	477
14b10f7ea70e Don't break if mailbox path contains ':' characters. Timo Sirainen <tss@iki.fi> parents: 4397 diff changeset	478 newfname = maildir_filename_set_flags(ctx->keywords_sync_ctx,
14b10f7ea70e Don't break if mailbox path contains ':' characters. Timo Sirainen <tss@iki.fi> parents: 4397 diff changeset	479 fname, flags8, &keywords);
14b10f7ea70e Don't break if mailbox path contains ':' characters. Timo Sirainen <tss@iki.fi> parents: 4397 diff changeset	480 newpath = t_strconcat(dir, newfname, NULL);
1955 0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	481 if (rename(path, newpath) == 0) {
5562 24b751bc0995 Added sync_notify() callback to struct mail_storage. It's now called for Timo Sirainen <tss@iki.fi> parents: 5459 diff changeset	482 if (box->v.sync_notify != NULL)
24b751bc0995 Added sync_notify() callback to struct mail_storage. It's now called for Timo Sirainen <tss@iki.fi> parents: 5459 diff changeset	483 box->v.sync_notify(box, ctx->uid, sync_type);
24b751bc0995 Added sync_notify() callback to struct mail_storage. It's now called for Timo Sirainen <tss@iki.fi> parents: 5459 diff changeset	484
2272 ced88553af0b mail_index_sync_sort_flags() now merges flag changes so mail storage Timo Sirainen <tss@iki.fi> parents: 2258 diff changeset	485 if ((flags8 & MAIL_INDEX_MAIL_FLAG_DIRTY) != 0)
ced88553af0b mail_index_sync_sort_flags() now merges flag changes so mail storage Timo Sirainen <tss@iki.fi> parents: 2258 diff changeset	486 ctx->dirty_state = -1;
3279 b698ae839a18 Moved mbox/maildir-specific variables from struct index_mailbox to Timo Sirainen <tss@iki.fi> parents: 3203 diff changeset	487 mbox->dirty_cur_time = ioloop_time;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	488 return 1;
1955 0f0128b4af5d More syncing changes Timo Sirainen <tss@iki.fi> parents: 1954 diff changeset	489 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	490 if (errno == ENOENT)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	491 return 0;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	492
2037 8763032d31bd Set dirty flags through transaction log, not directly. Some other flag Timo Sirainen <tss@iki.fi> parents: 2034 diff changeset	493 if (ENOSPACE(errno) \|\| errno == EACCES) {
8763032d31bd Set dirty flags through transaction log, not directly. Some other flag Timo Sirainen <tss@iki.fi> parents: 2034 diff changeset	494 mail_index_update_flags(ctx->trans, ctx->seq, MODIFY_ADD,
5307 1e2b2b3f18e3 Added casts to some enums to avoid compiler warnings. Timo Sirainen <tss@iki.fi> parents: 5229 diff changeset	495 (enum mail_flags)MAIL_INDEX_MAIL_FLAG_DIRTY);
2272 ced88553af0b mail_index_sync_sort_flags() now merges flag changes so mail storage Timo Sirainen <tss@iki.fi> parents: 2258 diff changeset	496 ctx->dirty_state = 1;
1956 d6941cd8afdc Added support for setting dirty flags for messages (TODO: undirty..) Timo Sirainen <tss@iki.fi> parents: 1955 diff changeset	497 return 1;
d6941cd8afdc Added support for setting dirty flags for messages (TODO: undirty..) Timo Sirainen <tss@iki.fi> parents: 1955 diff changeset	498 }
d6941cd8afdc Added support for setting dirty flags for messages (TODO: undirty..) Timo Sirainen <tss@iki.fi> parents: 1955 diff changeset	499
5459 78eaf595359c Removed struct index_storage abstraction. It's pointless. Timo Sirainen <tss@iki.fi> parents: 5443 diff changeset	500 mail_storage_set_critical(&mbox->storage->storage,
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	501 "rename(%s, %s) failed: %m", path, newpath);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	502 return -1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	503 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	504
5368 7d45edb81fe4 When copying/syncing a lot of mails, send "* OK Hang in there" replies to Timo Sirainen <tss@iki.fi> parents: 5307 diff changeset	505 static void
7d45edb81fe4 When copying/syncing a lot of mails, send "* OK Hang in there" replies to Timo Sirainen <tss@iki.fi> parents: 5307 diff changeset	506 maildir_sync_check_timeouts(struct maildir_sync_context *ctx, bool move)
7d45edb81fe4 When copying/syncing a lot of mails, send "* OK Hang in there" replies to Timo Sirainen <tss@iki.fi> parents: 5307 diff changeset	507 {
7d45edb81fe4 When copying/syncing a lot of mails, send "* OK Hang in there" replies to Timo Sirainen <tss@iki.fi> parents: 5307 diff changeset	508 time_t now;
7d45edb81fe4 When copying/syncing a lot of mails, send "* OK Hang in there" replies to Timo Sirainen <tss@iki.fi> parents: 5307 diff changeset	509
5390 4eeec560df01 If saving to maildir causes flag/expunge syncs, don't crash. Timo Sirainen <tss@iki.fi> parents: 5388 diff changeset	510 if (ctx == NULL) {
4eeec560df01 If saving to maildir causes flag/expunge syncs, don't crash. Timo Sirainen <tss@iki.fi> parents: 5388 diff changeset	511 /* we got here from maildir-save.c. it has no
4eeec560df01 If saving to maildir causes flag/expunge syncs, don't crash. Timo Sirainen <tss@iki.fi> parents: 5388 diff changeset	512 maildir_sync_context, */
4eeec560df01 If saving to maildir causes flag/expunge syncs, don't crash. Timo Sirainen <tss@iki.fi> parents: 5388 diff changeset	513 return;
4eeec560df01 If saving to maildir causes flag/expunge syncs, don't crash. Timo Sirainen <tss@iki.fi> parents: 5388 diff changeset	514 }
4eeec560df01 If saving to maildir causes flag/expunge syncs, don't crash. Timo Sirainen <tss@iki.fi> parents: 5388 diff changeset	515
5368 7d45edb81fe4 When copying/syncing a lot of mails, send "* OK Hang in there" replies to Timo Sirainen <tss@iki.fi> parents: 5307 diff changeset	516 if (move) {
7d45edb81fe4 When copying/syncing a lot of mails, send "* OK Hang in there" replies to Timo Sirainen <tss@iki.fi> parents: 5307 diff changeset	517 ctx->move_count++;
7d45edb81fe4 When copying/syncing a lot of mails, send "* OK Hang in there" replies to Timo Sirainen <tss@iki.fi> parents: 5307 diff changeset	518 if ((ctx->move_count % MAILDIR_SLOW_MOVE_COUNT) != 0)
7d45edb81fe4 When copying/syncing a lot of mails, send "* OK Hang in there" replies to Timo Sirainen <tss@iki.fi> parents: 5307 diff changeset	519 return;
7d45edb81fe4 When copying/syncing a lot of mails, send "* OK Hang in there" replies to Timo Sirainen <tss@iki.fi> parents: 5307 diff changeset	520 } else {
7d45edb81fe4 When copying/syncing a lot of mails, send "* OK Hang in there" replies to Timo Sirainen <tss@iki.fi> parents: 5307 diff changeset	521 ctx->check_count++;
7d45edb81fe4 When copying/syncing a lot of mails, send "* OK Hang in there" replies to Timo Sirainen <tss@iki.fi> parents: 5307 diff changeset	522 if ((ctx->check_count % MAILDIR_SLOW_CHECK_COUNT) != 0)
7d45edb81fe4 When copying/syncing a lot of mails, send "* OK Hang in there" replies to Timo Sirainen <tss@iki.fi> parents: 5307 diff changeset	523 return;
7d45edb81fe4 When copying/syncing a lot of mails, send "* OK Hang in there" replies to Timo Sirainen <tss@iki.fi> parents: 5307 diff changeset	524 }
7d45edb81fe4 When copying/syncing a lot of mails, send "* OK Hang in there" replies to Timo Sirainen <tss@iki.fi> parents: 5307 diff changeset	525
7d45edb81fe4 When copying/syncing a lot of mails, send "* OK Hang in there" replies to Timo Sirainen <tss@iki.fi> parents: 5307 diff changeset	526 now = time(NULL);
7d45edb81fe4 When copying/syncing a lot of mails, send "* OK Hang in there" replies to Timo Sirainen <tss@iki.fi> parents: 5307 diff changeset	527 if (now - ctx->last_touch > MAILDIR_LOCK_TOUCH_SECS) {
7d45edb81fe4 When copying/syncing a lot of mails, send "* OK Hang in there" replies to Timo Sirainen <tss@iki.fi> parents: 5307 diff changeset	528 (void)maildir_uidlist_lock_touch(ctx->mbox->uidlist);
7d45edb81fe4 When copying/syncing a lot of mails, send "* OK Hang in there" replies to Timo Sirainen <tss@iki.fi> parents: 5307 diff changeset	529 ctx->last_touch = now;
7d45edb81fe4 When copying/syncing a lot of mails, send "* OK Hang in there" replies to Timo Sirainen <tss@iki.fi> parents: 5307 diff changeset	530 }
7d45edb81fe4 When copying/syncing a lot of mails, send "* OK Hang in there" replies to Timo Sirainen <tss@iki.fi> parents: 5307 diff changeset	531 if (now - ctx->last_notify > MAIL_STORAGE_STAYALIVE_SECS) {
7d45edb81fe4 When copying/syncing a lot of mails, send "* OK Hang in there" replies to Timo Sirainen <tss@iki.fi> parents: 5307 diff changeset	532 struct mailbox *box = &ctx->mbox->ibox.box;
7d45edb81fe4 When copying/syncing a lot of mails, send "* OK Hang in there" replies to Timo Sirainen <tss@iki.fi> parents: 5307 diff changeset	533
7d45edb81fe4 When copying/syncing a lot of mails, send "* OK Hang in there" replies to Timo Sirainen <tss@iki.fi> parents: 5307 diff changeset	534 if (box->storage->callbacks->notify_ok != NULL) {
7d45edb81fe4 When copying/syncing a lot of mails, send "* OK Hang in there" replies to Timo Sirainen <tss@iki.fi> parents: 5307 diff changeset	535 box->storage->callbacks->
7d45edb81fe4 When copying/syncing a lot of mails, send "* OK Hang in there" replies to Timo Sirainen <tss@iki.fi> parents: 5307 diff changeset	536 notify_ok(box, "Hang in there..",
7d45edb81fe4 When copying/syncing a lot of mails, send "* OK Hang in there" replies to Timo Sirainen <tss@iki.fi> parents: 5307 diff changeset	537 box->storage->callback_context);
7d45edb81fe4 When copying/syncing a lot of mails, send "* OK Hang in there" replies to Timo Sirainen <tss@iki.fi> parents: 5307 diff changeset	538 }
7d45edb81fe4 When copying/syncing a lot of mails, send "* OK Hang in there" replies to Timo Sirainen <tss@iki.fi> parents: 5307 diff changeset	539 ctx->last_notify = now;
7d45edb81fe4 When copying/syncing a lot of mails, send "* OK Hang in there" replies to Timo Sirainen <tss@iki.fi> parents: 5307 diff changeset	540 }
7d45edb81fe4 When copying/syncing a lot of mails, send "* OK Hang in there" replies to Timo Sirainen <tss@iki.fi> parents: 5307 diff changeset	541 }
7d45edb81fe4 When copying/syncing a lot of mails, send "* OK Hang in there" replies to Timo Sirainen <tss@iki.fi> parents: 5307 diff changeset	542
3446 113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	543 static int
113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	544 maildir_sync_record_commit_until(struct maildir_index_sync_context *ctx,
113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	545 uint32_t last_seq)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	546 {
3446 113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	547 struct mail_index_sync_rec *recs;
113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	548 unsigned int i, count;
113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	549 uint32_t seq, uid;
3863 55df57c028d4 Added "bool" type and changed all ints that were used as booleans to bool. Timo Sirainen <tss@iki.fi> parents: 3821 diff changeset	550 bool expunged, flag_changed;
3446 113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	551
4451 1a35d53c18fc Array API redesigned to work using unions. It now provides type safety Timo Sirainen <tss@iki.fi> parents: 4450 diff changeset	552 recs = array_get_modifiable(&ctx->sync_recs, &count);
3446 113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	553 for (seq = recs[0].uid1; seq <= last_seq; seq++) {
113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	554 expunged = flag_changed = FALSE;
113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	555 for (i = 0; i < count; i++) {
113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	556 if (recs[i].uid1 > seq)
113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	557 break;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	558
3446 113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	559 i_assert(recs[i].uid1 == seq);
113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	560 switch (recs[i].type) {
113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	561 case MAIL_INDEX_SYNC_TYPE_EXPUNGE:
113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	562 expunged = TRUE;
113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	563 break;
113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	564 case MAIL_INDEX_SYNC_TYPE_FLAGS:
113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	565 case MAIL_INDEX_SYNC_TYPE_KEYWORD_ADD:
113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	566 case MAIL_INDEX_SYNC_TYPE_KEYWORD_REMOVE:
113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	567 case MAIL_INDEX_SYNC_TYPE_KEYWORD_RESET:
113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	568 flag_changed = TRUE;
113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	569 break;
113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	570 case MAIL_INDEX_SYNC_TYPE_APPEND:
113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	571 i_unreached();
113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	572 break;
113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	573 }
113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	574 }
113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	575
5080 e0b83da1e12f Error handling fixes Timo Sirainen <tss@iki.fi> parents: 4907 diff changeset	576 if (mail_index_lookup_uid(ctx->view, seq, &uid) < 0) {
e0b83da1e12f Error handling fixes Timo Sirainen <tss@iki.fi> parents: 4907 diff changeset	577 mail_storage_set_index_error(&ctx->mbox->ibox);
2033 4f6b1118a53d Transaction log contains only UIDs now, no more sequences which just mess up Timo Sirainen <tss@iki.fi> parents: 2009 diff changeset	578 return -1;
5080 e0b83da1e12f Error handling fixes Timo Sirainen <tss@iki.fi> parents: 4907 diff changeset	579 }
2033 4f6b1118a53d Transaction log contains only UIDs now, no more sequences which just mess up Timo Sirainen <tss@iki.fi> parents: 2009 diff changeset	580
3446 113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	581 ctx->seq = seq;
5562 24b751bc0995 Added sync_notify() callback to struct mail_storage. It's now called for Timo Sirainen <tss@iki.fi> parents: 5459 diff changeset	582 ctx->uid = uid;
3446 113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	583 if (expunged) {
5368 7d45edb81fe4 When copying/syncing a lot of mails, send "* OK Hang in there" replies to Timo Sirainen <tss@iki.fi> parents: 5307 diff changeset	584 maildir_sync_check_timeouts(ctx->maildir_sync_ctx,
7d45edb81fe4 When copying/syncing a lot of mails, send "* OK Hang in there" replies to Timo Sirainen <tss@iki.fi> parents: 5307 diff changeset	585 TRUE);
3446 113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	586 if (maildir_file_do(ctx->mbox, uid,
113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	587 maildir_expunge, ctx) < 0)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	588 return -1;
3446 113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	589 } else if (flag_changed) {
5368 7d45edb81fe4 When copying/syncing a lot of mails, send "* OK Hang in there" replies to Timo Sirainen <tss@iki.fi> parents: 5307 diff changeset	590 maildir_sync_check_timeouts(ctx->maildir_sync_ctx,
7d45edb81fe4 When copying/syncing a lot of mails, send "* OK Hang in there" replies to Timo Sirainen <tss@iki.fi> parents: 5307 diff changeset	591 TRUE);
3446 113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	592 if (maildir_file_do(ctx->mbox, uid,
113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	593 maildir_sync_flags, ctx) < 0)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	594 return -1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	595 }
3446 113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	596
113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	597 for (i = count; i > 0; i--) {
113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	598 if (++recs[i-1].uid1 > recs[i-1].uid2) {
113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	599 array_delete(&ctx->sync_recs, i-1, 1);
4451 1a35d53c18fc Array API redesigned to work using unions. It now provides type safety Timo Sirainen <tss@iki.fi> parents: 4450 diff changeset	600 recs = array_get_modifiable(&ctx->sync_recs,
3446 113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	601 &count);
113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	602 if (count == 0) {
113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	603 /* all sync_recs committed */
113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	604 return 0;
113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	605 }
113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	606 }
113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	607 }
113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	608 }
113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	609 return 0;
113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	610 }
113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	611
113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	612 static int maildir_sync_record(struct maildir_index_sync_context *ctx,
113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	613 const struct mail_index_sync_rec *sync_rec)
113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	614 {
113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	615 struct mail_index_view *view = ctx->view;
113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	616 struct mail_index_sync_rec sync_copy;
2033 4f6b1118a53d Transaction log contains only UIDs now, no more sequences which just mess up Timo Sirainen <tss@iki.fi> parents: 2009 diff changeset	617
3446 113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	618 if (sync_rec == NULL) {
113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	619 /* deinit */
113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	620 while (array_count(&ctx->sync_recs) > 0) {
113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	621 if (maildir_sync_record_commit_until(ctx,
113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	622 (uint32_t)-1) < 0)
113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	623 return -1;
113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	624 }
113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	625 return 0;
113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	626 }
113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	627
113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	628 if (sync_rec->type == MAIL_INDEX_SYNC_TYPE_APPEND)
113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	629 return 0; /* ignore */
113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	630
113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	631 /* convert to sequences to avoid looping through huge holes in
113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	632 UID range */
113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	633 sync_copy = *sync_rec;
113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	634 if (mail_index_lookup_uid_range(view, sync_rec->uid1,
113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	635 sync_rec->uid2,
113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	636 &sync_copy.uid1,
5080 e0b83da1e12f Error handling fixes Timo Sirainen <tss@iki.fi> parents: 4907 diff changeset	637 &sync_copy.uid2) < 0) {
e0b83da1e12f Error handling fixes Timo Sirainen <tss@iki.fi> parents: 4907 diff changeset	638 mail_storage_set_index_error(&ctx->mbox->ibox);
3446 113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	639 return -1;
5080 e0b83da1e12f Error handling fixes Timo Sirainen <tss@iki.fi> parents: 4907 diff changeset	640 }
3446 113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	641
3447 96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	642 if (sync_copy.uid1 == 0) {
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	643 /* UIDs were expunged */
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	644 return 0;
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	645 }
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	646
3446 113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	647 while (array_count(&ctx->sync_recs) > 0) {
113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	648 const struct mail_index_sync_rec *rec =
113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	649 array_idx(&ctx->sync_recs, 0);
113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	650
113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	651 i_assert(rec->uid1 <= sync_copy.uid1);
113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	652 if (rec->uid1 == sync_copy.uid1)
2033 4f6b1118a53d Transaction log contains only UIDs now, no more sequences which just mess up Timo Sirainen <tss@iki.fi> parents: 2009 diff changeset	653 break;
4f6b1118a53d Transaction log contains only UIDs now, no more sequences which just mess up Timo Sirainen <tss@iki.fi> parents: 2009 diff changeset	654
3446 113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	655 if (maildir_sync_record_commit_until(ctx, sync_copy.uid1-1) < 0)
113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	656 return -1;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	657 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	658
3446 113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	659 array_append(&ctx->sync_recs, &sync_copy, 1);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	660 return 0;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	661 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	662
3446 113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	663 static int maildir_sync_index_records(struct maildir_index_sync_context *ctx)
113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	664 {
113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	665 struct mail_index_sync_rec sync_rec;
113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	666 int ret;
113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	667
113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	668 ret = mail_index_sync_next(ctx->sync_ctx, &sync_rec);
5080 e0b83da1e12f Error handling fixes Timo Sirainen <tss@iki.fi> parents: 4907 diff changeset	669 if (ret <= 0) {
e0b83da1e12f Error handling fixes Timo Sirainen <tss@iki.fi> parents: 4907 diff changeset	670 if (ret < 0)
e0b83da1e12f Error handling fixes Timo Sirainen <tss@iki.fi> parents: 4907 diff changeset	671 mail_storage_set_index_error(&ctx->mbox->ibox);
3446 113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	672 return ret;
5080 e0b83da1e12f Error handling fixes Timo Sirainen <tss@iki.fi> parents: 4907 diff changeset	673 }
3446 113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	674
4596 bf4e98a0de3f Replaced ARRAY_CREATE() macro with [ipt]_array_init() macros. The macro Timo Sirainen <tss@iki.fi> parents: 4594 diff changeset	675 t_array_init(&ctx->sync_recs, 32);
3446 113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	676 do {
113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	677 if (maildir_sync_record(ctx, &sync_rec) < 0)
113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	678 return -1;
113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	679
113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	680 ret = mail_index_sync_next(ctx->sync_ctx, &sync_rec);
113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	681 } while (ret > 0);
113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	682
5080 e0b83da1e12f Error handling fixes Timo Sirainen <tss@iki.fi> parents: 4907 diff changeset	683 if (ret < 0)
e0b83da1e12f Error handling fixes Timo Sirainen <tss@iki.fi> parents: 4907 diff changeset	684 mail_storage_set_index_error(&ctx->mbox->ibox);
e0b83da1e12f Error handling fixes Timo Sirainen <tss@iki.fi> parents: 4907 diff changeset	685
3446 113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	686 if (maildir_sync_record(ctx, NULL) < 0)
113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	687 return -1;
113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	688 return ret;
113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	689 }
113c888cdca1 Merge changes from multiple index sync records into one before actually Timo Sirainen <tss@iki.fi> parents: 3436 diff changeset	690
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	691 static struct maildir_sync_context *
3279 b698ae839a18 Moved mbox/maildir-specific variables from struct index_mailbox to Timo Sirainen <tss@iki.fi> parents: 3203 diff changeset	692 maildir_sync_context_new(struct maildir_mailbox *mbox)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	693 {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	694 struct maildir_sync_context *ctx;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	695
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	696 ctx = t_new(struct maildir_sync_context, 1);
3279 b698ae839a18 Moved mbox/maildir-specific variables from struct index_mailbox to Timo Sirainen <tss@iki.fi> parents: 3203 diff changeset	697 ctx->mbox = mbox;
b698ae839a18 Moved mbox/maildir-specific variables from struct index_mailbox to Timo Sirainen <tss@iki.fi> parents: 3203 diff changeset	698 ctx->new_dir = t_strconcat(mbox->path, "/new", NULL);
b698ae839a18 Moved mbox/maildir-specific variables from struct index_mailbox to Timo Sirainen <tss@iki.fi> parents: 3203 diff changeset	699 ctx->cur_dir = t_strconcat(mbox->path, "/cur", NULL);
5368 7d45edb81fe4 When copying/syncing a lot of mails, send "* OK Hang in there" replies to Timo Sirainen <tss@iki.fi> parents: 5307 diff changeset	700 ctx->last_touch = ioloop_time;
7d45edb81fe4 When copying/syncing a lot of mails, send "* OK Hang in there" replies to Timo Sirainen <tss@iki.fi> parents: 5307 diff changeset	701 ctx->last_notify = ioloop_time;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	702 return ctx;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	703 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	704
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	705 static void maildir_sync_deinit(struct maildir_sync_context *ctx)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	706 {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	707 if (ctx->uidlist_sync_ctx != NULL)
4238 3c8b191b0019 Adding mail to index while saving it had a race condition. Fixing it Timo Sirainen <timo.sirainen@movial.fi> parents: 4199 diff changeset	708 (void)maildir_uidlist_sync_deinit(&ctx->uidlist_sync_ctx);
4774 615b7738a62f Saving mails could have skipped over transactions, which caused different Timo Sirainen <tss@iki.fi> parents: 4612 diff changeset	709 if (ctx->index_sync_ctx != NULL) {
615b7738a62f Saving mails could have skipped over transactions, which caused different Timo Sirainen <tss@iki.fi> parents: 4612 diff changeset	710 (void)maildir_sync_index_finish(&ctx->index_sync_ctx,
615b7738a62f Saving mails could have skipped over transactions, which caused different Timo Sirainen <tss@iki.fi> parents: 4612 diff changeset	711 TRUE, FALSE);
615b7738a62f Saving mails could have skipped over transactions, which caused different Timo Sirainen <tss@iki.fi> parents: 4612 diff changeset	712 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	713 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	714
4397 5cbabd4ccd9c Don't go fixing duplicate maildir filenames without properly checking that Timo Sirainen <tss@iki.fi> parents: 4238 diff changeset	715 static int maildir_fix_duplicate(struct maildir_sync_context *ctx,
5cbabd4ccd9c Don't go fixing duplicate maildir filenames without properly checking that Timo Sirainen <tss@iki.fi> parents: 4238 diff changeset	716 const char dir, const char old_fname)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	717 {
4397 5cbabd4ccd9c Don't go fixing duplicate maildir filenames without properly checking that Timo Sirainen <tss@iki.fi> parents: 4238 diff changeset	718 struct maildir_mailbox *mbox = ctx->mbox;
5cbabd4ccd9c Don't go fixing duplicate maildir filenames without properly checking that Timo Sirainen <tss@iki.fi> parents: 4238 diff changeset	719 const char existing_fname, existing_path;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	720 const char new_fname, old_path, *new_path;
5397 3a0964ac3a5c comment/duplicate link handling fixes. Timo Sirainen <tss@iki.fi> parents: 5392 diff changeset	721 struct stat ex_st, old_st;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	722 int ret = 0;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	723
4397 5cbabd4ccd9c Don't go fixing duplicate maildir filenames without properly checking that Timo Sirainen <tss@iki.fi> parents: 4238 diff changeset	724 existing_fname =
5cbabd4ccd9c Don't go fixing duplicate maildir filenames without properly checking that Timo Sirainen <tss@iki.fi> parents: 4238 diff changeset	725 maildir_uidlist_sync_get_full_filename(ctx->uidlist_sync_ctx,
5cbabd4ccd9c Don't go fixing duplicate maildir filenames without properly checking that Timo Sirainen <tss@iki.fi> parents: 4238 diff changeset	726 old_fname);
5cbabd4ccd9c Don't go fixing duplicate maildir filenames without properly checking that Timo Sirainen <tss@iki.fi> parents: 4238 diff changeset	727 i_assert(existing_fname != NULL);
5cbabd4ccd9c Don't go fixing duplicate maildir filenames without properly checking that Timo Sirainen <tss@iki.fi> parents: 4238 diff changeset	728
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	729 t_push();
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	730
4397 5cbabd4ccd9c Don't go fixing duplicate maildir filenames without properly checking that Timo Sirainen <tss@iki.fi> parents: 4238 diff changeset	731 existing_path = t_strconcat(dir, "/", existing_fname, NULL);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	732 old_path = t_strconcat(dir, "/", old_fname, NULL);
4397 5cbabd4ccd9c Don't go fixing duplicate maildir filenames without properly checking that Timo Sirainen <tss@iki.fi> parents: 4238 diff changeset	733
5392 9a8402768b42 If duplicate links are found, rename() one over the other to get rid of Timo Sirainen <tss@iki.fi> parents: 5391 diff changeset	734 if (stat(existing_path, &ex_st) < 0 \|\|
9a8402768b42 If duplicate links are found, rename() one over the other to get rid of Timo Sirainen <tss@iki.fi> parents: 5391 diff changeset	735 stat(old_path, &old_st) < 0) {
4397 5cbabd4ccd9c Don't go fixing duplicate maildir filenames without properly checking that Timo Sirainen <tss@iki.fi> parents: 4238 diff changeset	736 /* most likely the files just don't exist anymore.
5cbabd4ccd9c Don't go fixing duplicate maildir filenames without properly checking that Timo Sirainen <tss@iki.fi> parents: 4238 diff changeset	737 don't really care about other errors much. */
5cbabd4ccd9c Don't go fixing duplicate maildir filenames without properly checking that Timo Sirainen <tss@iki.fi> parents: 4238 diff changeset	738 t_pop();
5cbabd4ccd9c Don't go fixing duplicate maildir filenames without properly checking that Timo Sirainen <tss@iki.fi> parents: 4238 diff changeset	739 return 0;
5cbabd4ccd9c Don't go fixing duplicate maildir filenames without properly checking that Timo Sirainen <tss@iki.fi> parents: 4238 diff changeset	740 }
5392 9a8402768b42 If duplicate links are found, rename() one over the other to get rid of Timo Sirainen <tss@iki.fi> parents: 5391 diff changeset	741 if (ex_st.st_ino == old_st.st_ino &&
9a8402768b42 If duplicate links are found, rename() one over the other to get rid of Timo Sirainen <tss@iki.fi> parents: 5391 diff changeset	742 CMP_DEV_T(ex_st.st_dev, old_st.st_dev)) {
5397 3a0964ac3a5c comment/duplicate link handling fixes. Timo Sirainen <tss@iki.fi> parents: 5392 diff changeset	743 /* Files are the same. this means either a race condition
5443 293bc7d1062f If we find duplicate hard links which haven't changed for 30 secs, unlink() Timo Sirainen <tss@iki.fi> parents: 5397 diff changeset	744 between stat() calls, or that the files were link()ed. */
293bc7d1062f If we find duplicate hard links which haven't changed for 30 secs, unlink() Timo Sirainen <tss@iki.fi> parents: 5397 diff changeset	745 if (ex_st.st_nlink > 1 && old_st.st_nlink == ex_st.st_nlink &&
293bc7d1062f If we find duplicate hard links which haven't changed for 30 secs, unlink() Timo Sirainen <tss@iki.fi> parents: 5397 diff changeset	746 ex_st.st_ctime == old_st.st_ctime &&
293bc7d1062f If we find duplicate hard links which haven't changed for 30 secs, unlink() Timo Sirainen <tss@iki.fi> parents: 5397 diff changeset	747 ex_st.st_ctime < ioloop_time - DUPE_LINKS_DELETE_SECS) {
293bc7d1062f If we find duplicate hard links which haven't changed for 30 secs, unlink() Timo Sirainen <tss@iki.fi> parents: 5397 diff changeset	748 /* The file has hard links and it hasn't had any
293bc7d1062f If we find duplicate hard links which haven't changed for 30 secs, unlink() Timo Sirainen <tss@iki.fi> parents: 5397 diff changeset	749 changes (such as renames) for a while, so this
293bc7d1062f If we find duplicate hard links which haven't changed for 30 secs, unlink() Timo Sirainen <tss@iki.fi> parents: 5397 diff changeset	750 isn't a race condition.
5392 9a8402768b42 If duplicate links are found, rename() one over the other to get rid of Timo Sirainen <tss@iki.fi> parents: 5391 diff changeset	751
5443 293bc7d1062f If we find duplicate hard links which haven't changed for 30 secs, unlink() Timo Sirainen <tss@iki.fi> parents: 5397 diff changeset	752 rename()ing one file on top of the other would fix
293bc7d1062f If we find duplicate hard links which haven't changed for 30 secs, unlink() Timo Sirainen <tss@iki.fi> parents: 5397 diff changeset	753 this safely, except POSIX decided that rename()
293bc7d1062f If we find duplicate hard links which haven't changed for 30 secs, unlink() Timo Sirainen <tss@iki.fi> parents: 5397 diff changeset	754 doesn't work that way. So we'll have unlink() one
293bc7d1062f If we find duplicate hard links which haven't changed for 30 secs, unlink() Timo Sirainen <tss@iki.fi> parents: 5397 diff changeset	755 and hope that another process didn't just decide to
293bc7d1062f If we find duplicate hard links which haven't changed for 30 secs, unlink() Timo Sirainen <tss@iki.fi> parents: 5397 diff changeset	756 unlink() the other (uidlist lock prevents this from
293bc7d1062f If we find duplicate hard links which haven't changed for 30 secs, unlink() Timo Sirainen <tss@iki.fi> parents: 5397 diff changeset	757 happening) */
293bc7d1062f If we find duplicate hard links which haven't changed for 30 secs, unlink() Timo Sirainen <tss@iki.fi> parents: 5397 diff changeset	758 if (unlink(old_path) == 0) {
293bc7d1062f If we find duplicate hard links which haven't changed for 30 secs, unlink() Timo Sirainen <tss@iki.fi> parents: 5397 diff changeset	759 i_warning("Unlinked a duplicate: %s",
293bc7d1062f If we find duplicate hard links which haven't changed for 30 secs, unlink() Timo Sirainen <tss@iki.fi> parents: 5397 diff changeset	760 old_fname);
293bc7d1062f If we find duplicate hard links which haven't changed for 30 secs, unlink() Timo Sirainen <tss@iki.fi> parents: 5397 diff changeset	761 } else {
293bc7d1062f If we find duplicate hard links which haven't changed for 30 secs, unlink() Timo Sirainen <tss@iki.fi> parents: 5397 diff changeset	762 mail_storage_set_critical(
5459 78eaf595359c Removed struct index_storage abstraction. It's pointless. Timo Sirainen <tss@iki.fi> parents: 5443 diff changeset	763 &mbox->storage->storage,
5443 293bc7d1062f If we find duplicate hard links which haven't changed for 30 secs, unlink() Timo Sirainen <tss@iki.fi> parents: 5397 diff changeset	764 "unlink(%s) failed: %m", old_path);
293bc7d1062f If we find duplicate hard links which haven't changed for 30 secs, unlink() Timo Sirainen <tss@iki.fi> parents: 5397 diff changeset	765 }
293bc7d1062f If we find duplicate hard links which haven't changed for 30 secs, unlink() Timo Sirainen <tss@iki.fi> parents: 5397 diff changeset	766 }
4397 5cbabd4ccd9c Don't go fixing duplicate maildir filenames without properly checking that Timo Sirainen <tss@iki.fi> parents: 4238 diff changeset	767 t_pop();
5cbabd4ccd9c Don't go fixing duplicate maildir filenames without properly checking that Timo Sirainen <tss@iki.fi> parents: 4238 diff changeset	768 return 0;
5cbabd4ccd9c Don't go fixing duplicate maildir filenames without properly checking that Timo Sirainen <tss@iki.fi> parents: 4238 diff changeset	769 }
5cbabd4ccd9c Don't go fixing duplicate maildir filenames without properly checking that Timo Sirainen <tss@iki.fi> parents: 4238 diff changeset	770
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	771 new_fname = maildir_generate_tmp_filename(&ioloop_timeval);
3279 b698ae839a18 Moved mbox/maildir-specific variables from struct index_mailbox to Timo Sirainen <tss@iki.fi> parents: 3203 diff changeset	772 new_path = t_strconcat(mbox->path, "/new/", new_fname, NULL);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	773
5392 9a8402768b42 If duplicate links are found, rename() one over the other to get rid of Timo Sirainen <tss@iki.fi> parents: 5391 diff changeset	774 if (rename(old_path, new_path) == 0)
9a8402768b42 If duplicate links are found, rename() one over the other to get rid of Timo Sirainen <tss@iki.fi> parents: 5391 diff changeset	775 i_warning("Fixed a duplicate: %s -> %s", old_fname, new_fname);
9a8402768b42 If duplicate links are found, rename() one over the other to get rid of Timo Sirainen <tss@iki.fi> parents: 5391 diff changeset	776 else if (errno != ENOENT) {
5459 78eaf595359c Removed struct index_storage abstraction. It's pointless. Timo Sirainen <tss@iki.fi> parents: 5443 diff changeset	777 mail_storage_set_critical(&mbox->storage->storage,
5582 1c4fd25893bd Updated error message. Timo Sirainen <tss@iki.fi> parents: 5564 diff changeset	778 "Couldn't fix a duplicate: rename(%s, %s) failed: %m",
1c4fd25893bd Updated error message. Timo Sirainen <tss@iki.fi> parents: 5564 diff changeset	779 old_path, new_path);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	780 ret = -1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	781 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	782 t_pop();
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	783
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	784 return ret;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	785 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	786
3863 55df57c028d4 Added "bool" type and changed all ints that were used as booleans to bool. Timo Sirainen <tss@iki.fi> parents: 3821 diff changeset	787 static int maildir_scan_dir(struct maildir_sync_context *ctx, bool new_dir)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	788 {
5459 78eaf595359c Removed struct index_storage abstraction. It's pointless. Timo Sirainen <tss@iki.fi> parents: 5443 diff changeset	789 struct mail_storage *storage = &ctx->mbox->storage->storage;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	790 const char *dir;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	791 DIR *dirp;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	792 string_t src, dest;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	793 struct dirent *dp;
3435 6f7ce690358e If we have rename()d more than 5 files from new/ to cur/, rescan the Timo Sirainen <tss@iki.fi> parents: 3417 diff changeset	794 enum maildir_uidlist_rec_flag flags;
3863 55df57c028d4 Added "bool" type and changed all ints that were used as booleans to bool. Timo Sirainen <tss@iki.fi> parents: 3821 diff changeset	795 int ret = 1;
5214 4e9d345df846 When syncing huge maildirs check once in a while if we need to update Timo Sirainen <tss@iki.fi> parents: 5080 diff changeset	796 bool move_new, check_touch;
4e9d345df846 When syncing huge maildirs check once in a while if we need to update Timo Sirainen <tss@iki.fi> parents: 5080 diff changeset	797
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	798 dir = new_dir ? ctx->new_dir : ctx->cur_dir;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	799 dirp = opendir(dir);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	800 if (dirp == NULL) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	801 mail_storage_set_critical(storage,
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	802 "opendir(%s) failed: %m", dir);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	803 return -1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	804 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	805
2121 0840edf34f37 Locking fixes. use less memory Timo Sirainen <tss@iki.fi> parents: 2067 diff changeset	806 t_push();
0840edf34f37 Locking fixes. use less memory Timo Sirainen <tss@iki.fi> parents: 2067 diff changeset	807 src = t_str_new(1024);
0840edf34f37 Locking fixes. use less memory Timo Sirainen <tss@iki.fi> parents: 2067 diff changeset	808 dest = t_str_new(1024);
0840edf34f37 Locking fixes. use less memory Timo Sirainen <tss@iki.fi> parents: 2067 diff changeset	809
5368 7d45edb81fe4 When copying/syncing a lot of mails, send "* OK Hang in there" replies to Timo Sirainen <tss@iki.fi> parents: 5307 diff changeset	810 ctx->move_count = 0;
3279 b698ae839a18 Moved mbox/maildir-specific variables from struct index_mailbox to Timo Sirainen <tss@iki.fi> parents: 3203 diff changeset	811 move_new = new_dir && !mailbox_is_readonly(&ctx->mbox->ibox.box) &&
b698ae839a18 Moved mbox/maildir-specific variables from struct index_mailbox to Timo Sirainen <tss@iki.fi> parents: 3203 diff changeset	812 !ctx->mbox->ibox.keep_recent;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	813 while ((dp = readdir(dirp)) != NULL) {
2258 087a43e29492 No maildir filename checking after all. Timo Sirainen <tss@iki.fi> parents: 2256 diff changeset	814 if (dp->d_name[0] == '.')
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	815 continue;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	816
1978 6303ef092c5b mbox code compiles again, but syncing is only partially implemented so Timo Sirainen <tss@iki.fi> parents: 1970 diff changeset	817 ret = maildir_uidlist_sync_next_pre(ctx->uidlist_sync_ctx,
6303ef092c5b mbox code compiles again, but syncing is only partially implemented so Timo Sirainen <tss@iki.fi> parents: 1970 diff changeset	818 dp->d_name);
1984 9c159272f721 syncing fixes Timo Sirainen <tss@iki.fi> parents: 1978 diff changeset	819 if (ret == 0) {
2038 df504dad3aec Recent flag fixes. Should work perfectly now with maildir. Timo Sirainen <tss@iki.fi> parents: 2037 diff changeset	820 /* new file and we couldn't lock uidlist, check this
df504dad3aec Recent flag fixes. Should work perfectly now with maildir. Timo Sirainen <tss@iki.fi> parents: 2037 diff changeset	821 later in next sync. */
1984 9c159272f721 syncing fixes Timo Sirainen <tss@iki.fi> parents: 1978 diff changeset	822 if (new_dir)
3279 b698ae839a18 Moved mbox/maildir-specific variables from struct index_mailbox to Timo Sirainen <tss@iki.fi> parents: 3203 diff changeset	823 ctx->mbox->last_new_mtime = 0;
1984 9c159272f721 syncing fixes Timo Sirainen <tss@iki.fi> parents: 1978 diff changeset	824 else
3279 b698ae839a18 Moved mbox/maildir-specific variables from struct index_mailbox to Timo Sirainen <tss@iki.fi> parents: 3203 diff changeset	825 ctx->mbox->dirty_cur_time = ioloop_time;
1978 6303ef092c5b mbox code compiles again, but syncing is only partially implemented so Timo Sirainen <tss@iki.fi> parents: 1970 diff changeset	826 continue;
1984 9c159272f721 syncing fixes Timo Sirainen <tss@iki.fi> parents: 1978 diff changeset	827 }
1978 6303ef092c5b mbox code compiles again, but syncing is only partially implemented so Timo Sirainen <tss@iki.fi> parents: 1970 diff changeset	828 if (ret < 0)
6303ef092c5b mbox code compiles again, but syncing is only partially implemented so Timo Sirainen <tss@iki.fi> parents: 1970 diff changeset	829 break;
6303ef092c5b mbox code compiles again, but syncing is only partially implemented so Timo Sirainen <tss@iki.fi> parents: 1970 diff changeset	830
5214 4e9d345df846 When syncing huge maildirs check once in a while if we need to update Timo Sirainen <tss@iki.fi> parents: 5080 diff changeset	831 check_touch = FALSE;
1947 777da553d1d3 Recent-flag should work now Timo Sirainen <tss@iki.fi> parents: 1944 diff changeset	832 flags = 0;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	833 if (move_new) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	834 str_truncate(src, 0);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	835 str_truncate(dest, 0);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	836 str_printfa(src, "%s/%s", ctx->new_dir, dp->d_name);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	837 str_printfa(dest, "%s/%s", ctx->cur_dir, dp->d_name);
3417 b0bdf32564b7 Replaced ':' and ',' character usages with #defines, so they can be changed Timo Sirainen <tss@iki.fi> parents: 3340 diff changeset	838 if (strchr(dp->d_name, MAILDIR_INFO_SEP) == NULL) {
b0bdf32564b7 Replaced ':' and ',' character usages with #defines, so they can be changed Timo Sirainen <tss@iki.fi> parents: 3340 diff changeset	839 str_append(dest, MAILDIR_FLAGS_FULL_SEP);
b0bdf32564b7 Replaced ':' and ',' character usages with #defines, so they can be changed Timo Sirainen <tss@iki.fi> parents: 3340 diff changeset	840 }
1947 777da553d1d3 Recent-flag should work now Timo Sirainen <tss@iki.fi> parents: 1944 diff changeset	841 if (rename(str_c(src), str_c(dest)) == 0) {
1984 9c159272f721 syncing fixes Timo Sirainen <tss@iki.fi> parents: 1978 diff changeset	842 /* we moved it - it's \Recent for us */
5368 7d45edb81fe4 When copying/syncing a lot of mails, send "* OK Hang in there" replies to Timo Sirainen <tss@iki.fi> parents: 5307 diff changeset	843 maildir_sync_check_timeouts(ctx, TRUE);
3435 6f7ce690358e If we have rename()d more than 5 files from new/ to cur/, rescan the Timo Sirainen <tss@iki.fi> parents: 3417 diff changeset	844 ctx->mbox->dirty_cur_time = ioloop_time;
1947 777da553d1d3 Recent-flag should work now Timo Sirainen <tss@iki.fi> parents: 1944 diff changeset	845 flags \|= MAILDIR_UIDLIST_REC_FLAG_MOVED \|
777da553d1d3 Recent-flag should work now Timo Sirainen <tss@iki.fi> parents: 1944 diff changeset	846 MAILDIR_UIDLIST_REC_FLAG_RECENT;
777da553d1d3 Recent-flag should work now Timo Sirainen <tss@iki.fi> parents: 1944 diff changeset	847 } else if (ENOTFOUND(errno)) {
777da553d1d3 Recent-flag should work now Timo Sirainen <tss@iki.fi> parents: 1944 diff changeset	848 /* someone else moved it already */
5368 7d45edb81fe4 When copying/syncing a lot of mails, send "* OK Hang in there" replies to Timo Sirainen <tss@iki.fi> parents: 5307 diff changeset	849 maildir_sync_check_timeouts(ctx, TRUE);
1947 777da553d1d3 Recent-flag should work now Timo Sirainen <tss@iki.fi> parents: 1944 diff changeset	850 flags \|= MAILDIR_UIDLIST_REC_FLAG_MOVED;
4199 fa135b615b01 If maildir is readonly, don't complain about rename() failing to move mails Timo Sirainen <tss@iki.fi> parents: 4152 diff changeset	851 } else if (ENOSPACE(errno) \|\| errno == EACCES) {
fa135b615b01 If maildir is readonly, don't complain about rename() failing to move mails Timo Sirainen <tss@iki.fi> parents: 4152 diff changeset	852 /* not enough disk space / read-only maildir,
fa135b615b01 If maildir is readonly, don't complain about rename() failing to move mails Timo Sirainen <tss@iki.fi> parents: 4152 diff changeset	853 leave here */
1947 777da553d1d3 Recent-flag should work now Timo Sirainen <tss@iki.fi> parents: 1944 diff changeset	854 flags \|= MAILDIR_UIDLIST_REC_FLAG_NEW_DIR \|
777da553d1d3 Recent-flag should work now Timo Sirainen <tss@iki.fi> parents: 1944 diff changeset	855 MAILDIR_UIDLIST_REC_FLAG_RECENT;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	856 move_new = FALSE;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	857 } else {
1947 777da553d1d3 Recent-flag should work now Timo Sirainen <tss@iki.fi> parents: 1944 diff changeset	858 flags \|= MAILDIR_UIDLIST_REC_FLAG_NEW_DIR \|
777da553d1d3 Recent-flag should work now Timo Sirainen <tss@iki.fi> parents: 1944 diff changeset	859 MAILDIR_UIDLIST_REC_FLAG_RECENT;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	860 mail_storage_set_critical(storage,
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	861 "rename(%s, %s) failed: %m",
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	862 str_c(src), str_c(dest));
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	863 }
1947 777da553d1d3 Recent-flag should work now Timo Sirainen <tss@iki.fi> parents: 1944 diff changeset	864 } else if (new_dir) {
777da553d1d3 Recent-flag should work now Timo Sirainen <tss@iki.fi> parents: 1944 diff changeset	865 flags \|= MAILDIR_UIDLIST_REC_FLAG_NEW_DIR \|
777da553d1d3 Recent-flag should work now Timo Sirainen <tss@iki.fi> parents: 1944 diff changeset	866 MAILDIR_UIDLIST_REC_FLAG_RECENT;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	867 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	868
5368 7d45edb81fe4 When copying/syncing a lot of mails, send "* OK Hang in there" replies to Timo Sirainen <tss@iki.fi> parents: 5307 diff changeset	869 maildir_sync_check_timeouts(ctx, FALSE);
5214 4e9d345df846 When syncing huge maildirs check once in a while if we need to update Timo Sirainen <tss@iki.fi> parents: 5080 diff changeset	870
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	871 ret = maildir_uidlist_sync_next(ctx->uidlist_sync_ctx,
1947 777da553d1d3 Recent-flag should work now Timo Sirainen <tss@iki.fi> parents: 1944 diff changeset	872 dp->d_name, flags);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	873 if (ret <= 0) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	874 if (ret < 0)
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	875 break;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	876
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	877 /* possibly duplicate - try fixing it */
4397 5cbabd4ccd9c Don't go fixing duplicate maildir filenames without properly checking that Timo Sirainen <tss@iki.fi> parents: 4238 diff changeset	878 if (maildir_fix_duplicate(ctx, dir, dp->d_name) < 0) {
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	879 ret = -1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	880 break;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	881 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	882 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	883 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	884
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	885 if (closedir(dirp) < 0) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	886 mail_storage_set_critical(storage,
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	887 "closedir(%s) failed: %m", dir);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	888 }
2121 0840edf34f37 Locking fixes. use less memory Timo Sirainen <tss@iki.fi> parents: 2067 diff changeset	889
0840edf34f37 Locking fixes. use less memory Timo Sirainen <tss@iki.fi> parents: 2067 diff changeset	890 t_pop();
5368 7d45edb81fe4 When copying/syncing a lot of mails, send "* OK Hang in there" replies to Timo Sirainen <tss@iki.fi> parents: 5307 diff changeset	891 return ret < 0 ? -1 :
7d45edb81fe4 When copying/syncing a lot of mails, send "* OK Hang in there" replies to Timo Sirainen <tss@iki.fi> parents: 5307 diff changeset	892 (ctx->move_count <= MAILDIR_RENAME_RESCAN_COUNT ? 0 : 1);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	893 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	894
3472 db29cc6754d5 Store new/ directory's timestamp in sync_size header in index (kludgy..). Timo Sirainen <tss@iki.fi> parents: 3470 diff changeset	895 static void
4848 967de900c73a Mailbox list indexing and related changes. Currently works only with Timo Sirainen <tss@iki.fi> parents: 4774 diff changeset	896 maildir_sync_update_from_header(struct maildir_mailbox *mbox,
967de900c73a Mailbox list indexing and related changes. Currently works only with Timo Sirainen <tss@iki.fi> parents: 4774 diff changeset	897 struct mail_index_header *hdr_r)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	898 {
4848 967de900c73a Mailbox list indexing and related changes. Currently works only with Timo Sirainen <tss@iki.fi> parents: 4774 diff changeset	899 struct mail_index_view *view;
967de900c73a Mailbox list indexing and related changes. Currently works only with Timo Sirainen <tss@iki.fi> parents: 4774 diff changeset	900 const struct mail_index_header *hdr;
967de900c73a Mailbox list indexing and related changes. Currently works only with Timo Sirainen <tss@iki.fi> parents: 4774 diff changeset	901
967de900c73a Mailbox list indexing and related changes. Currently works only with Timo Sirainen <tss@iki.fi> parents: 4774 diff changeset	902 /* open a new view so we get the latest header */
967de900c73a Mailbox list indexing and related changes. Currently works only with Timo Sirainen <tss@iki.fi> parents: 4774 diff changeset	903 view = mail_index_view_open(mbox->ibox.index);
967de900c73a Mailbox list indexing and related changes. Currently works only with Timo Sirainen <tss@iki.fi> parents: 4774 diff changeset	904 hdr = mail_index_get_header(view);
3472 db29cc6754d5 Store new/ directory's timestamp in sync_size header in index (kludgy..). Timo Sirainen <tss@iki.fi> parents: 3470 diff changeset	905
db29cc6754d5 Store new/ directory's timestamp in sync_size header in index (kludgy..). Timo Sirainen <tss@iki.fi> parents: 3470 diff changeset	906 /* FIXME: ugly, replace with extension header */
4848 967de900c73a Mailbox list indexing and related changes. Currently works only with Timo Sirainen <tss@iki.fi> parents: 4774 diff changeset	907 mbox->last_new_mtime = hdr->sync_size & 0xffffffff;
967de900c73a Mailbox list indexing and related changes. Currently works only with Timo Sirainen <tss@iki.fi> parents: 4774 diff changeset	908 mbox->last_dirty_flags = (hdr->sync_size >> 32) &
967de900c73a Mailbox list indexing and related changes. Currently works only with Timo Sirainen <tss@iki.fi> parents: 4774 diff changeset	909 (MAILDIR_DIRTY_NEW \| MAILDIR_DIRTY_CUR);
967de900c73a Mailbox list indexing and related changes. Currently works only with Timo Sirainen <tss@iki.fi> parents: 4774 diff changeset	910
967de900c73a Mailbox list indexing and related changes. Currently works only with Timo Sirainen <tss@iki.fi> parents: 4774 diff changeset	911 mbox->last_cur_mtime = hdr->sync_stamp;
3472 db29cc6754d5 Store new/ directory's timestamp in sync_size header in index (kludgy..). Timo Sirainen <tss@iki.fi> parents: 3470 diff changeset	912
4848 967de900c73a Mailbox list indexing and related changes. Currently works only with Timo Sirainen <tss@iki.fi> parents: 4774 diff changeset	913 if ((mbox->last_dirty_flags & MAILDIR_DIRTY_CUR) != 0 &&
967de900c73a Mailbox list indexing and related changes. Currently works only with Timo Sirainen <tss@iki.fi> parents: 4774 diff changeset	914 mbox->dirty_cur_time < mbox->last_cur_mtime)
967de900c73a Mailbox list indexing and related changes. Currently works only with Timo Sirainen <tss@iki.fi> parents: 4774 diff changeset	915 mbox->dirty_cur_time = mbox->last_cur_mtime;
967de900c73a Mailbox list indexing and related changes. Currently works only with Timo Sirainen <tss@iki.fi> parents: 4774 diff changeset	916
967de900c73a Mailbox list indexing and related changes. Currently works only with Timo Sirainen <tss@iki.fi> parents: 4774 diff changeset	917 hdr_r = hdr;
967de900c73a Mailbox list indexing and related changes. Currently works only with Timo Sirainen <tss@iki.fi> parents: 4774 diff changeset	918 mail_index_view_close(&view);
3472 db29cc6754d5 Store new/ directory's timestamp in sync_size header in index (kludgy..). Timo Sirainen <tss@iki.fi> parents: 3470 diff changeset	919 }
db29cc6754d5 Store new/ directory's timestamp in sync_size header in index (kludgy..). Timo Sirainen <tss@iki.fi> parents: 3470 diff changeset	920
db29cc6754d5 Store new/ directory's timestamp in sync_size header in index (kludgy..). Timo Sirainen <tss@iki.fi> parents: 3470 diff changeset	921 static int
db29cc6754d5 Store new/ directory's timestamp in sync_size header in index (kludgy..). Timo Sirainen <tss@iki.fi> parents: 3470 diff changeset	922 maildir_sync_quick_check(struct maildir_mailbox *mbox,
db29cc6754d5 Store new/ directory's timestamp in sync_size header in index (kludgy..). Timo Sirainen <tss@iki.fi> parents: 3470 diff changeset	923 const char new_dir, const char cur_dir,
3863 55df57c028d4 Added "bool" type and changed all ints that were used as booleans to bool. Timo Sirainen <tss@iki.fi> parents: 3821 diff changeset	924 bool new_changed_r, bool cur_changed_r)
3472 db29cc6754d5 Store new/ directory's timestamp in sync_size header in index (kludgy..). Timo Sirainen <tss@iki.fi> parents: 3470 diff changeset	925 {
4126 63d6890803b3 If mailbox is opened in non-readonly state and there are mails in new/, make Timo Sirainen <tss@iki.fi> parents: 4107 diff changeset	926 struct index_mailbox *ibox = &mbox->ibox;
4848 967de900c73a Mailbox list indexing and related changes. Currently works only with Timo Sirainen <tss@iki.fi> parents: 4774 diff changeset	927 struct mail_index_header hdr;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	928 struct stat st;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	929 time_t new_mtime, cur_mtime;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	930
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	931 new_changed_r = cur_changed_r = FALSE;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	932
3472 db29cc6754d5 Store new/ directory's timestamp in sync_size header in index (kludgy..). Timo Sirainen <tss@iki.fi> parents: 3470 diff changeset	933 if (stat(new_dir, &st) < 0) {
5459 78eaf595359c Removed struct index_storage abstraction. It's pointless. Timo Sirainen <tss@iki.fi> parents: 5443 diff changeset	934 mail_storage_set_critical(&mbox->storage->storage,
3472 db29cc6754d5 Store new/ directory's timestamp in sync_size header in index (kludgy..). Timo Sirainen <tss@iki.fi> parents: 3470 diff changeset	935 "stat(%s) failed: %m", new_dir);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	936 return -1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	937 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	938 new_mtime = st.st_mtime;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	939
3472 db29cc6754d5 Store new/ directory's timestamp in sync_size header in index (kludgy..). Timo Sirainen <tss@iki.fi> parents: 3470 diff changeset	940 if (stat(cur_dir, &st) < 0) {
5459 78eaf595359c Removed struct index_storage abstraction. It's pointless. Timo Sirainen <tss@iki.fi> parents: 5443 diff changeset	941 mail_storage_set_critical(&mbox->storage->storage,
3472 db29cc6754d5 Store new/ directory's timestamp in sync_size header in index (kludgy..). Timo Sirainen <tss@iki.fi> parents: 3470 diff changeset	942 "stat(%s) failed: %m", cur_dir);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	943 return -1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	944 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	945 cur_mtime = st.st_mtime;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	946
2893 fd431866c674 mail_index_refresh() isn't public anymore, mail_index_view_open_locked() Timo Sirainen <tss@iki.fi> parents: 2892 diff changeset	947 /* cur stamp is kept in index, we don't have to sync if
3472 db29cc6754d5 Store new/ directory's timestamp in sync_size header in index (kludgy..). Timo Sirainen <tss@iki.fi> parents: 3470 diff changeset	948 someone else has done it and updated the index.
db29cc6754d5 Store new/ directory's timestamp in sync_size header in index (kludgy..). Timo Sirainen <tss@iki.fi> parents: 3470 diff changeset	949
db29cc6754d5 Store new/ directory's timestamp in sync_size header in index (kludgy..). Timo Sirainen <tss@iki.fi> parents: 3470 diff changeset	950 FIXME: For now we're using sync_size field as the new/ dir's stamp.
db29cc6754d5 Store new/ directory's timestamp in sync_size header in index (kludgy..). Timo Sirainen <tss@iki.fi> parents: 3470 diff changeset	951 Pretty ugly.. */
4848 967de900c73a Mailbox list indexing and related changes. Currently works only with Timo Sirainen <tss@iki.fi> parents: 4774 diff changeset	952 maildir_sync_update_from_header(mbox, &hdr);
3472 db29cc6754d5 Store new/ directory's timestamp in sync_size header in index (kludgy..). Timo Sirainen <tss@iki.fi> parents: 3470 diff changeset	953 if ((mbox->dirty_cur_time == 0 && cur_mtime != mbox->last_cur_mtime) \|\|
db29cc6754d5 Store new/ directory's timestamp in sync_size header in index (kludgy..). Timo Sirainen <tss@iki.fi> parents: 3470 diff changeset	954 (new_mtime != mbox->last_new_mtime)) {
2893 fd431866c674 mail_index_refresh() isn't public anymore, mail_index_view_open_locked() Timo Sirainen <tss@iki.fi> parents: 2892 diff changeset	955 /* check if the index has been updated.. */
4126 63d6890803b3 If mailbox is opened in non-readonly state and there are mails in new/, make Timo Sirainen <tss@iki.fi> parents: 4107 diff changeset	956 if (mail_index_refresh(ibox->index) < 0) {
63d6890803b3 If mailbox is opened in non-readonly state and there are mails in new/, make Timo Sirainen <tss@iki.fi> parents: 4107 diff changeset	957 mail_storage_set_index_error(ibox);
2064 5cfdc99fab69 Don't use internal last_cur_mtime stamp - it's index-specific so always get Timo Sirainen <tss@iki.fi> parents: 2053 diff changeset	958 return -1;
5cfdc99fab69 Don't use internal last_cur_mtime stamp - it's index-specific so always get Timo Sirainen <tss@iki.fi> parents: 2053 diff changeset	959 }
5cfdc99fab69 Don't use internal last_cur_mtime stamp - it's index-specific so always get Timo Sirainen <tss@iki.fi> parents: 2053 diff changeset	960
4848 967de900c73a Mailbox list indexing and related changes. Currently works only with Timo Sirainen <tss@iki.fi> parents: 4774 diff changeset	961 maildir_sync_update_from_header(mbox, &hdr);
1954 2f6e137cdc44 Syncing optimizations. Timo Sirainen <tss@iki.fi> parents: 1947 diff changeset	962 }
2f6e137cdc44 Syncing optimizations. Timo Sirainen <tss@iki.fi> parents: 1947 diff changeset	963
4126 63d6890803b3 If mailbox is opened in non-readonly state and there are mails in new/, make Timo Sirainen <tss@iki.fi> parents: 4107 diff changeset	964 /* If we're removing recent flags, always sync new/ directory if
63d6890803b3 If mailbox is opened in non-readonly state and there are mails in new/, make Timo Sirainen <tss@iki.fi> parents: 4107 diff changeset	965 it has mails. */
3279 b698ae839a18 Moved mbox/maildir-specific variables from struct index_mailbox to Timo Sirainen <tss@iki.fi> parents: 3203 diff changeset	966 if (new_mtime != mbox->last_new_mtime \|\|
4848 967de900c73a Mailbox list indexing and related changes. Currently works only with Timo Sirainen <tss@iki.fi> parents: 4774 diff changeset	967 ((mbox->last_dirty_flags & MAILDIR_DIRTY_NEW) != 0 &&
967de900c73a Mailbox list indexing and related changes. Currently works only with Timo Sirainen <tss@iki.fi> parents: 4774 diff changeset	968 new_mtime < ioloop_time - MAILDIR_SYNC_SECS) \|\|
967de900c73a Mailbox list indexing and related changes. Currently works only with Timo Sirainen <tss@iki.fi> parents: 4774 diff changeset	969 (!ibox->keep_recent && hdr.recent_messages_count > 0)) {
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	970 *new_changed_r = TRUE;
3279 b698ae839a18 Moved mbox/maildir-specific variables from struct index_mailbox to Timo Sirainen <tss@iki.fi> parents: 3203 diff changeset	971 mbox->last_new_mtime = new_mtime;
4848 967de900c73a Mailbox list indexing and related changes. Currently works only with Timo Sirainen <tss@iki.fi> parents: 4774 diff changeset	972
967de900c73a Mailbox list indexing and related changes. Currently works only with Timo Sirainen <tss@iki.fi> parents: 4774 diff changeset	973 if (new_mtime < ioloop_time - MAILDIR_SYNC_SECS)
967de900c73a Mailbox list indexing and related changes. Currently works only with Timo Sirainen <tss@iki.fi> parents: 4774 diff changeset	974 mbox->last_dirty_flags &= ~MAILDIR_DIRTY_NEW;
967de900c73a Mailbox list indexing and related changes. Currently works only with Timo Sirainen <tss@iki.fi> parents: 4774 diff changeset	975 else
967de900c73a Mailbox list indexing and related changes. Currently works only with Timo Sirainen <tss@iki.fi> parents: 4774 diff changeset	976 mbox->last_dirty_flags \|= MAILDIR_DIRTY_NEW;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	977 }
1954 2f6e137cdc44 Syncing optimizations. Timo Sirainen <tss@iki.fi> parents: 1947 diff changeset	978
3279 b698ae839a18 Moved mbox/maildir-specific variables from struct index_mailbox to Timo Sirainen <tss@iki.fi> parents: 3203 diff changeset	979 if (cur_mtime != mbox->last_cur_mtime \|\|
b698ae839a18 Moved mbox/maildir-specific variables from struct index_mailbox to Timo Sirainen <tss@iki.fi> parents: 3203 diff changeset	980 (mbox->dirty_cur_time != 0 &&
b698ae839a18 Moved mbox/maildir-specific variables from struct index_mailbox to Timo Sirainen <tss@iki.fi> parents: 3203 diff changeset	981 ioloop_time - mbox->dirty_cur_time > MAILDIR_SYNC_SECS)) {
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	982 /* cur/ changed, or delayed cur/ check */
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	983 *cur_changed_r = TRUE;
3279 b698ae839a18 Moved mbox/maildir-specific variables from struct index_mailbox to Timo Sirainen <tss@iki.fi> parents: 3203 diff changeset	984 mbox->last_cur_mtime = cur_mtime;
1984 9c159272f721 syncing fixes Timo Sirainen <tss@iki.fi> parents: 1978 diff changeset	985
4848 967de900c73a Mailbox list indexing and related changes. Currently works only with Timo Sirainen <tss@iki.fi> parents: 4774 diff changeset	986 if (cur_mtime < ioloop_time - MAILDIR_SYNC_SECS) {
967de900c73a Mailbox list indexing and related changes. Currently works only with Timo Sirainen <tss@iki.fi> parents: 4774 diff changeset	987 mbox->last_dirty_flags &= ~MAILDIR_DIRTY_CUR;
967de900c73a Mailbox list indexing and related changes. Currently works only with Timo Sirainen <tss@iki.fi> parents: 4774 diff changeset	988 mbox->dirty_cur_time = 0;
967de900c73a Mailbox list indexing and related changes. Currently works only with Timo Sirainen <tss@iki.fi> parents: 4774 diff changeset	989 } else {
967de900c73a Mailbox list indexing and related changes. Currently works only with Timo Sirainen <tss@iki.fi> parents: 4774 diff changeset	990 mbox->last_dirty_flags \|= MAILDIR_DIRTY_CUR;
967de900c73a Mailbox list indexing and related changes. Currently works only with Timo Sirainen <tss@iki.fi> parents: 4774 diff changeset	991 mbox->dirty_cur_time = cur_mtime;
967de900c73a Mailbox list indexing and related changes. Currently works only with Timo Sirainen <tss@iki.fi> parents: 4774 diff changeset	992 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	993 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	994
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	995 return 0;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	996 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	997
4238 3c8b191b0019 Adding mail to index while saving it had a race condition. Fixing it Timo Sirainen <timo.sirainen@movial.fi> parents: 4199 diff changeset	998 int maildir_sync_index_begin(struct maildir_mailbox *mbox,
3c8b191b0019 Adding mail to index while saving it had a race condition. Fixing it Timo Sirainen <timo.sirainen@movial.fi> parents: 4199 diff changeset	999 struct maildir_index_sync_context **ctx_r)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1000 {
4238 3c8b191b0019 Adding mail to index while saving it had a race condition. Fixing it Timo Sirainen <timo.sirainen@movial.fi> parents: 4199 diff changeset	1001 struct maildir_index_sync_context *ctx;
3c8b191b0019 Adding mail to index while saving it had a race condition. Fixing it Timo Sirainen <timo.sirainen@movial.fi> parents: 4199 diff changeset	1002 struct mail_index_sync_ctx *sync_ctx;
3c8b191b0019 Adding mail to index while saving it had a race condition. Fixing it Timo Sirainen <timo.sirainen@movial.fi> parents: 4199 diff changeset	1003 struct mail_index_view *view;
2818 a758a5b542bb Always protect maildir syncing with uidlist lock. Before we only tried to Timo Sirainen <tss@iki.fi> parents: 2816 diff changeset	1004
4238 3c8b191b0019 Adding mail to index while saving it had a race condition. Fixing it Timo Sirainen <timo.sirainen@movial.fi> parents: 4199 diff changeset	1005 if (mail_index_sync_begin(mbox->ibox.index, &sync_ctx, &view,
3c8b191b0019 Adding mail to index while saving it had a race condition. Fixing it Timo Sirainen <timo.sirainen@movial.fi> parents: 4199 diff changeset	1006 (uint32_t)-1, (uoff_t)-1,
2818 a758a5b542bb Always protect maildir syncing with uidlist lock. Before we only tried to Timo Sirainen <tss@iki.fi> parents: 2816 diff changeset	1007 FALSE, FALSE) <= 0) {
3279 b698ae839a18 Moved mbox/maildir-specific variables from struct index_mailbox to Timo Sirainen <tss@iki.fi> parents: 3203 diff changeset	1008 mail_storage_set_index_error(&mbox->ibox);
4238 3c8b191b0019 Adding mail to index while saving it had a race condition. Fixing it Timo Sirainen <timo.sirainen@movial.fi> parents: 4199 diff changeset	1009 return -1;
2818 a758a5b542bb Always protect maildir syncing with uidlist lock. Before we only tried to Timo Sirainen <tss@iki.fi> parents: 2816 diff changeset	1010 }
3447 96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	1011
4238 3c8b191b0019 Adding mail to index while saving it had a race condition. Fixing it Timo Sirainen <timo.sirainen@movial.fi> parents: 4199 diff changeset	1012 ctx = i_new(struct maildir_index_sync_context, 1);
3c8b191b0019 Adding mail to index while saving it had a race condition. Fixing it Timo Sirainen <timo.sirainen@movial.fi> parents: 4199 diff changeset	1013 ctx->mbox = mbox;
3c8b191b0019 Adding mail to index while saving it had a race condition. Fixing it Timo Sirainen <timo.sirainen@movial.fi> parents: 4199 diff changeset	1014 ctx->sync_ctx = sync_ctx;
3c8b191b0019 Adding mail to index while saving it had a race condition. Fixing it Timo Sirainen <timo.sirainen@movial.fi> parents: 4199 diff changeset	1015 ctx->view = view;
3c8b191b0019 Adding mail to index while saving it had a race condition. Fixing it Timo Sirainen <timo.sirainen@movial.fi> parents: 4199 diff changeset	1016 ctx->keywords_sync_ctx =
3447 96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	1017 maildir_keywords_sync_init(mbox->keywords, mbox->ibox.index);
4238 3c8b191b0019 Adding mail to index while saving it had a race condition. Fixing it Timo Sirainen <timo.sirainen@movial.fi> parents: 4199 diff changeset	1018 *ctx_r = ctx;
3c8b191b0019 Adding mail to index while saving it had a race condition. Fixing it Timo Sirainen <timo.sirainen@movial.fi> parents: 4199 diff changeset	1019 return 0;
2818 a758a5b542bb Always protect maildir syncing with uidlist lock. Before we only tried to Timo Sirainen <tss@iki.fi> parents: 2816 diff changeset	1020 }
a758a5b542bb Always protect maildir syncing with uidlist lock. Before we only tried to Timo Sirainen <tss@iki.fi> parents: 2816 diff changeset	1021
4238 3c8b191b0019 Adding mail to index while saving it had a race condition. Fixing it Timo Sirainen <timo.sirainen@movial.fi> parents: 4199 diff changeset	1022 int maildir_sync_index_finish(struct maildir_index_sync_context **_sync_ctx,
4774 615b7738a62f Saving mails could have skipped over transactions, which caused different Timo Sirainen <tss@iki.fi> parents: 4612 diff changeset	1023 bool failed, bool cancel)
2818 a758a5b542bb Always protect maildir syncing with uidlist lock. Before we only tried to Timo Sirainen <tss@iki.fi> parents: 2816 diff changeset	1024 {
4238 3c8b191b0019 Adding mail to index while saving it had a race condition. Fixing it Timo Sirainen <timo.sirainen@movial.fi> parents: 4199 diff changeset	1025 struct maildir_index_sync_context sync_ctx = _sync_ctx;
3c8b191b0019 Adding mail to index while saving it had a race condition. Fixing it Timo Sirainen <timo.sirainen@movial.fi> parents: 4199 diff changeset	1026 struct maildir_mailbox *mbox = sync_ctx->mbox;
3c8b191b0019 Adding mail to index while saving it had a race condition. Fixing it Timo Sirainen <timo.sirainen@movial.fi> parents: 4199 diff changeset	1027 uint32_t seq;
3c8b191b0019 Adding mail to index while saving it had a race condition. Fixing it Timo Sirainen <timo.sirainen@movial.fi> parents: 4199 diff changeset	1028 uoff_t offset;
3c8b191b0019 Adding mail to index while saving it had a race condition. Fixing it Timo Sirainen <timo.sirainen@movial.fi> parents: 4199 diff changeset	1029 int ret = failed ? -1 : 0;
3c8b191b0019 Adding mail to index while saving it had a race condition. Fixing it Timo Sirainen <timo.sirainen@movial.fi> parents: 4199 diff changeset	1030
3c8b191b0019 Adding mail to index while saving it had a race condition. Fixing it Timo Sirainen <timo.sirainen@movial.fi> parents: 4199 diff changeset	1031 *_sync_ctx = NULL;
3c8b191b0019 Adding mail to index while saving it had a race condition. Fixing it Timo Sirainen <timo.sirainen@movial.fi> parents: 4199 diff changeset	1032
3c8b191b0019 Adding mail to index while saving it had a race condition. Fixing it Timo Sirainen <timo.sirainen@movial.fi> parents: 4199 diff changeset	1033 if (sync_ctx->trans != NULL) {
4774 615b7738a62f Saving mails could have skipped over transactions, which caused different Timo Sirainen <tss@iki.fi> parents: 4612 diff changeset	1034 if (ret < 0 \|\| cancel)
4238 3c8b191b0019 Adding mail to index while saving it had a race condition. Fixing it Timo Sirainen <timo.sirainen@movial.fi> parents: 4199 diff changeset	1035 mail_index_transaction_rollback(&sync_ctx->trans);
3c8b191b0019 Adding mail to index while saving it had a race condition. Fixing it Timo Sirainen <timo.sirainen@movial.fi> parents: 4199 diff changeset	1036 else {
3c8b191b0019 Adding mail to index while saving it had a race condition. Fixing it Timo Sirainen <timo.sirainen@movial.fi> parents: 4199 diff changeset	1037 if (mail_index_transaction_commit(&sync_ctx->trans,
5080 e0b83da1e12f Error handling fixes Timo Sirainen <tss@iki.fi> parents: 4907 diff changeset	1038 &seq, &offset) < 0) {
e0b83da1e12f Error handling fixes Timo Sirainen <tss@iki.fi> parents: 4907 diff changeset	1039 mail_storage_set_index_error(&mbox->ibox);
4238 3c8b191b0019 Adding mail to index while saving it had a race condition. Fixing it Timo Sirainen <timo.sirainen@movial.fi> parents: 4199 diff changeset	1040 ret = -1;
5080 e0b83da1e12f Error handling fixes Timo Sirainen <tss@iki.fi> parents: 4907 diff changeset	1041 } else if (seq != 0) {
4238 3c8b191b0019 Adding mail to index while saving it had a race condition. Fixing it Timo Sirainen <timo.sirainen@movial.fi> parents: 4199 diff changeset	1042 mbox->ibox.commit_log_file_seq = seq;
3c8b191b0019 Adding mail to index while saving it had a race condition. Fixing it Timo Sirainen <timo.sirainen@movial.fi> parents: 4199 diff changeset	1043 mbox->ibox.commit_log_file_offset = offset;
3c8b191b0019 Adding mail to index while saving it had a race condition. Fixing it Timo Sirainen <timo.sirainen@movial.fi> parents: 4199 diff changeset	1044 }
3c8b191b0019 Adding mail to index while saving it had a race condition. Fixing it Timo Sirainen <timo.sirainen@movial.fi> parents: 4199 diff changeset	1045 }
3c8b191b0019 Adding mail to index while saving it had a race condition. Fixing it Timo Sirainen <timo.sirainen@movial.fi> parents: 4199 diff changeset	1046 }
4774 615b7738a62f Saving mails could have skipped over transactions, which caused different Timo Sirainen <tss@iki.fi> parents: 4612 diff changeset	1047 if (ret < 0 \|\| cancel)
4238 3c8b191b0019 Adding mail to index while saving it had a race condition. Fixing it Timo Sirainen <timo.sirainen@movial.fi> parents: 4199 diff changeset	1048 mail_index_sync_rollback(&sync_ctx->sync_ctx);
3c8b191b0019 Adding mail to index while saving it had a race condition. Fixing it Timo Sirainen <timo.sirainen@movial.fi> parents: 4199 diff changeset	1049 else {
4612 2c033ffc8f6f maildir_storage_sync_force() crashed if called from mail index sync/expunge Timo Sirainen <tss@iki.fi> parents: 4596 diff changeset	1050 /* Set syncing_commit=TRUE so that if any sync callbacks try
2c033ffc8f6f maildir_storage_sync_force() crashed if called from mail index sync/expunge Timo Sirainen <tss@iki.fi> parents: 4596 diff changeset	1051 to access mails which got lost (eg. expunge callback trying
2c033ffc8f6f maildir_storage_sync_force() crashed if called from mail index sync/expunge Timo Sirainen <tss@iki.fi> parents: 4596 diff changeset	1052 to open the file which was just unlinked) we don't try to
2c033ffc8f6f maildir_storage_sync_force() crashed if called from mail index sync/expunge Timo Sirainen <tss@iki.fi> parents: 4596 diff changeset	1053 start a second index sync and crash. */
2c033ffc8f6f maildir_storage_sync_force() crashed if called from mail index sync/expunge Timo Sirainen <tss@iki.fi> parents: 4596 diff changeset	1054 mbox->syncing_commit = TRUE;
5080 e0b83da1e12f Error handling fixes Timo Sirainen <tss@iki.fi> parents: 4907 diff changeset	1055 if (mail_index_sync_commit(&sync_ctx->sync_ctx) < 0) {
e0b83da1e12f Error handling fixes Timo Sirainen <tss@iki.fi> parents: 4907 diff changeset	1056 mail_storage_set_index_error(&mbox->ibox);
4238 3c8b191b0019 Adding mail to index while saving it had a race condition. Fixing it Timo Sirainen <timo.sirainen@movial.fi> parents: 4199 diff changeset	1057 ret = -1;
5080 e0b83da1e12f Error handling fixes Timo Sirainen <tss@iki.fi> parents: 4907 diff changeset	1058 } else {
4238 3c8b191b0019 Adding mail to index while saving it had a race condition. Fixing it Timo Sirainen <timo.sirainen@movial.fi> parents: 4199 diff changeset	1059 mbox->ibox.commit_log_file_seq = 0;
3c8b191b0019 Adding mail to index while saving it had a race condition. Fixing it Timo Sirainen <timo.sirainen@movial.fi> parents: 4199 diff changeset	1060 mbox->ibox.commit_log_file_offset = 0;
3c8b191b0019 Adding mail to index while saving it had a race condition. Fixing it Timo Sirainen <timo.sirainen@movial.fi> parents: 4199 diff changeset	1061 }
4612 2c033ffc8f6f maildir_storage_sync_force() crashed if called from mail index sync/expunge Timo Sirainen <tss@iki.fi> parents: 4596 diff changeset	1062 mbox->syncing_commit = FALSE;
4238 3c8b191b0019 Adding mail to index while saving it had a race condition. Fixing it Timo Sirainen <timo.sirainen@movial.fi> parents: 4199 diff changeset	1063 }
3c8b191b0019 Adding mail to index while saving it had a race condition. Fixing it Timo Sirainen <timo.sirainen@movial.fi> parents: 4199 diff changeset	1064
3447 96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	1065 maildir_keywords_sync_deinit(sync_ctx->keywords_sync_ctx);
4238 3c8b191b0019 Adding mail to index while saving it had a race condition. Fixing it Timo Sirainen <timo.sirainen@movial.fi> parents: 4199 diff changeset	1066 sync_ctx->keywords_sync_ctx = NULL;
3c8b191b0019 Adding mail to index while saving it had a race condition. Fixing it Timo Sirainen <timo.sirainen@movial.fi> parents: 4199 diff changeset	1067
2818 a758a5b542bb Always protect maildir syncing with uidlist lock. Before we only tried to Timo Sirainen <tss@iki.fi> parents: 2816 diff changeset	1068 i_free(sync_ctx);
4238 3c8b191b0019 Adding mail to index while saving it had a race condition. Fixing it Timo Sirainen <timo.sirainen@movial.fi> parents: 4199 diff changeset	1069 return ret;
2818 a758a5b542bb Always protect maildir syncing with uidlist lock. Before we only tried to Timo Sirainen <tss@iki.fi> parents: 2816 diff changeset	1070 }
a758a5b542bb Always protect maildir syncing with uidlist lock. Before we only tried to Timo Sirainen <tss@iki.fi> parents: 2816 diff changeset	1071
4238 3c8b191b0019 Adding mail to index while saving it had a race condition. Fixing it Timo Sirainen <timo.sirainen@movial.fi> parents: 4199 diff changeset	1072 int maildir_sync_index(struct maildir_index_sync_context *sync_ctx,
3c8b191b0019 Adding mail to index while saving it had a race condition. Fixing it Timo Sirainen <timo.sirainen@movial.fi> parents: 4199 diff changeset	1073 bool partial)
2818 a758a5b542bb Always protect maildir syncing with uidlist lock. Before we only tried to Timo Sirainen <tss@iki.fi> parents: 2816 diff changeset	1074 {
3279 b698ae839a18 Moved mbox/maildir-specific variables from struct index_mailbox to Timo Sirainen <tss@iki.fi> parents: 3203 diff changeset	1075 struct maildir_mailbox *mbox = sync_ctx->mbox;
2818 a758a5b542bb Always protect maildir syncing with uidlist lock. Before we only tried to Timo Sirainen <tss@iki.fi> parents: 2816 diff changeset	1076 struct mail_index_view *view = sync_ctx->view;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1077 struct maildir_uidlist_iter_ctx *iter;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1078 struct mail_index_transaction *trans;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1079 const struct mail_index_header *hdr;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1080 const struct mail_index_record *rec;
5229 1f737b6e911b Added assert. Timo Sirainen <tss@iki.fi> parents: 5214 diff changeset	1081 uint32_t seq, uid, prev_uid;
1954 2f6e137cdc44 Syncing optimizations. Timo Sirainen <tss@iki.fi> parents: 1947 diff changeset	1082 enum maildir_uidlist_rec_flag uflags;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1083 const char *filename;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1084 enum mail_flags flags;
4451 1a35d53c18fc Array API redesigned to work using unions. It now provides type safety Timo Sirainen <tss@iki.fi> parents: 4450 diff changeset	1085 ARRAY_TYPE(keyword_indexes) keywords;
1a35d53c18fc Array API redesigned to work using unions. It now provides type safety Timo Sirainen <tss@iki.fi> parents: 4450 diff changeset	1086 ARRAY_TYPE(keyword_indexes) idx_keywords;
2050 ee1095ccfd23 Index header changes now go through transaction log. Removed the kludgy Timo Sirainen <tss@iki.fi> parents: 2039 diff changeset	1087 uint32_t uid_validity, next_uid;
3472 db29cc6754d5 Store new/ directory's timestamp in sync_size header in index (kludgy..). Timo Sirainen <tss@iki.fi> parents: 3470 diff changeset	1088 uint64_t value;
3863 55df57c028d4 Added "bool" type and changed all ints that were used as booleans to bool. Timo Sirainen <tss@iki.fi> parents: 3821 diff changeset	1089 int ret = 0;
55df57c028d4 Added "bool" type and changed all ints that were used as booleans to bool. Timo Sirainen <tss@iki.fi> parents: 3821 diff changeset	1090 bool full_rescan = FALSE;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1091
3453 f140acbe46b4 Assert/cleanup Timo Sirainen <tss@iki.fi> parents: 3447 diff changeset	1092 i_assert(maildir_uidlist_is_locked(sync_ctx->mbox->uidlist));
f140acbe46b4 Assert/cleanup Timo Sirainen <tss@iki.fi> parents: 3447 diff changeset	1093
2892 62d53b49110d Changed mail_index_get_header() to return the header as return value because Timo Sirainen <tss@iki.fi> parents: 2877 diff changeset	1094 hdr = mail_index_get_header(view);
3279 b698ae839a18 Moved mbox/maildir-specific variables from struct index_mailbox to Timo Sirainen <tss@iki.fi> parents: 3203 diff changeset	1095 uid_validity = maildir_uidlist_get_uid_validity(mbox->uidlist);
2051 596267d8e2e2 Some more UIDVALIDITY issues fixed. Timo Sirainen <tss@iki.fi> parents: 2050 diff changeset	1096 if (uid_validity != hdr->uid_validity &&
596267d8e2e2 Some more UIDVALIDITY issues fixed. Timo Sirainen <tss@iki.fi> parents: 2050 diff changeset	1097 uid_validity != 0 && hdr->uid_validity != 0) {
2050 ee1095ccfd23 Index header changes now go through transaction log. Removed the kludgy Timo Sirainen <tss@iki.fi> parents: 2039 diff changeset	1098 /* uidvalidity changed and mailbox isn't being initialized,
3322 49071cc19102 If UIDVALIDITY changes, don't invalidate the whole index. Just expunge all Timo Sirainen <tss@iki.fi> parents: 3280 diff changeset	1099 reset mailbox so we can add all messages as new */
5459 78eaf595359c Removed struct index_storage abstraction. It's pointless. Timo Sirainen <tss@iki.fi> parents: 5443 diff changeset	1100 mail_storage_set_critical(&mbox->storage->storage,
2067 3f97439ba59e Path was missing from Maildir sync-errors. Timo Sirainen <tss@iki.fi> parents: 2064 diff changeset	1101 "Maildir %s sync: UIDVALIDITY changed (%u -> %u)",
3279 b698ae839a18 Moved mbox/maildir-specific variables from struct index_mailbox to Timo Sirainen <tss@iki.fi> parents: 3203 diff changeset	1102 mbox->path, hdr->uid_validity, uid_validity);
3322 49071cc19102 If UIDVALIDITY changes, don't invalidate the whole index. Just expunge all Timo Sirainen <tss@iki.fi> parents: 3280 diff changeset	1103
3340 9c8220dfde7c Don't try to handle UIDVALIDITY changes nicely after all. It causes Timo Sirainen <tss@iki.fi> parents: 3322 diff changeset	1104 mail_index_mark_corrupted(mbox->ibox.index);
9c8220dfde7c Don't try to handle UIDVALIDITY changes nicely after all. It causes Timo Sirainen <tss@iki.fi> parents: 3322 diff changeset	1105 return -1;
2050 ee1095ccfd23 Index header changes now go through transaction log. Removed the kludgy Timo Sirainen <tss@iki.fi> parents: 2039 diff changeset	1106 }
ee1095ccfd23 Index header changes now go through transaction log. Removed the kludgy Timo Sirainen <tss@iki.fi> parents: 2039 diff changeset	1107
4238 3c8b191b0019 Adding mail to index while saving it had a race condition. Fixing it Timo Sirainen <timo.sirainen@movial.fi> parents: 4199 diff changeset	1108 sync_ctx->trans = trans =
3c8b191b0019 Adding mail to index while saving it had a race condition. Fixing it Timo Sirainen <timo.sirainen@movial.fi> parents: 4199 diff changeset	1109 mail_index_transaction_begin(sync_ctx->view, FALSE, TRUE);
3c8b191b0019 Adding mail to index while saving it had a race condition. Fixing it Timo Sirainen <timo.sirainen@movial.fi> parents: 4199 diff changeset	1110
5229 1f737b6e911b Added assert. Timo Sirainen <tss@iki.fi> parents: 5214 diff changeset	1111 seq = prev_uid = 0;
4596 bf4e98a0de3f Replaced ARRAY_CREATE() macro with [ipt]_array_init() macros. The macro Timo Sirainen <tss@iki.fi> parents: 4594 diff changeset	1112 t_array_init(&keywords, MAILDIR_MAX_KEYWORDS);
bf4e98a0de3f Replaced ARRAY_CREATE() macro with [ipt]_array_init() macros. The macro Timo Sirainen <tss@iki.fi> parents: 4594 diff changeset	1113 t_array_init(&idx_keywords, MAILDIR_MAX_KEYWORDS);
3279 b698ae839a18 Moved mbox/maildir-specific variables from struct index_mailbox to Timo Sirainen <tss@iki.fi> parents: 3203 diff changeset	1114 iter = maildir_uidlist_iter_init(mbox->uidlist);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1115 while (maildir_uidlist_iter_next(iter, &uid, &uflags, &filename)) {
3775 8321e6191275 maildir_copy_with_hardlinks works again. Timo Sirainen <tss@iki.fi> parents: 3583 diff changeset	1116 maildir_filename_get_flags(sync_ctx->keywords_sync_ctx,
8321e6191275 maildir_copy_with_hardlinks works again. Timo Sirainen <tss@iki.fi> parents: 3583 diff changeset	1117 filename, &flags, &keywords);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1118
5229 1f737b6e911b Added assert. Timo Sirainen <tss@iki.fi> parents: 5214 diff changeset	1119 i_assert(uid > prev_uid);
1f737b6e911b Added assert. Timo Sirainen <tss@iki.fi> parents: 5214 diff changeset	1120 prev_uid = uid;
1f737b6e911b Added assert. Timo Sirainen <tss@iki.fi> parents: 5214 diff changeset	1121
4107 d29677c59dc5 Keep \Seen flags privately only in indexes with shared mailboxes. Timo Sirainen <tss@iki.fi> parents: 3879 diff changeset	1122 /* the private flags are kept only in indexes. don't use them
d29677c59dc5 Keep \Seen flags privately only in indexes with shared mailboxes. Timo Sirainen <tss@iki.fi> parents: 3879 diff changeset	1123 at all even for newly seen mails */
d29677c59dc5 Keep \Seen flags privately only in indexes with shared mailboxes. Timo Sirainen <tss@iki.fi> parents: 3879 diff changeset	1124 flags &= ~mbox->private_flags_mask;
d29677c59dc5 Keep \Seen flags privately only in indexes with shared mailboxes. Timo Sirainen <tss@iki.fi> parents: 3879 diff changeset	1125
2038 df504dad3aec Recent flag fixes. Should work perfectly now with maildir. Timo Sirainen <tss@iki.fi> parents: 2037 diff changeset	1126 if ((uflags & MAILDIR_UIDLIST_REC_FLAG_RECENT) != 0 &&
df504dad3aec Recent flag fixes. Should work perfectly now with maildir. Timo Sirainen <tss@iki.fi> parents: 2037 diff changeset	1127 (uflags & MAILDIR_UIDLIST_REC_FLAG_NEW_DIR) != 0 &&
df504dad3aec Recent flag fixes. Should work perfectly now with maildir. Timo Sirainen <tss@iki.fi> parents: 2037 diff changeset	1128 (uflags & MAILDIR_UIDLIST_REC_FLAG_MOVED) == 0) {
df504dad3aec Recent flag fixes. Should work perfectly now with maildir. Timo Sirainen <tss@iki.fi> parents: 2037 diff changeset	1129 /* mail is recent for next session as well */
df504dad3aec Recent flag fixes. Should work perfectly now with maildir. Timo Sirainen <tss@iki.fi> parents: 2037 diff changeset	1130 flags \|= MAIL_RECENT;
df504dad3aec Recent flag fixes. Should work perfectly now with maildir. Timo Sirainen <tss@iki.fi> parents: 2037 diff changeset	1131 }
df504dad3aec Recent flag fixes. Should work perfectly now with maildir. Timo Sirainen <tss@iki.fi> parents: 2037 diff changeset	1132
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1133 __again:
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1134 seq++;
1954 2f6e137cdc44 Syncing optimizations. Timo Sirainen <tss@iki.fi> parents: 1947 diff changeset	1135
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1136 if (seq > hdr->messages_count) {
1984 9c159272f721 syncing fixes Timo Sirainen <tss@iki.fi> parents: 1978 diff changeset	1137 if (uid < hdr->next_uid) {
2053 66f5d28f9b27 race condition fixes Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1138 /* most likely a race condition: we read the
66f5d28f9b27 race condition fixes Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1139 maildir, then someone else expunged messages
66f5d28f9b27 race condition fixes Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1140 and committed changes to index. so, this
66f5d28f9b27 race condition fixes Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1141 message shouldn't actually exist. mark it
66f5d28f9b27 race condition fixes Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1142 racy and check in next sync.
66f5d28f9b27 race condition fixes Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1143
66f5d28f9b27 race condition fixes Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1144 the difference between this and the later
66f5d28f9b27 race condition fixes Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1145 check is that this one happens when messages
66f5d28f9b27 race condition fixes Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1146 are expunged from the end */
66f5d28f9b27 race condition fixes Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1147 if ((uflags &
2228 d19ba01fb5cd partial syncing fixes Timo Sirainen <tss@iki.fi> parents: 2173 diff changeset	1148 MAILDIR_UIDLIST_REC_FLAG_NONSYNCED) != 0) {
d19ba01fb5cd partial syncing fixes Timo Sirainen <tss@iki.fi> parents: 2173 diff changeset	1149 /* partial syncing */
d19ba01fb5cd partial syncing fixes Timo Sirainen <tss@iki.fi> parents: 2173 diff changeset	1150 continue;
d19ba01fb5cd partial syncing fixes Timo Sirainen <tss@iki.fi> parents: 2173 diff changeset	1151 }
d19ba01fb5cd partial syncing fixes Timo Sirainen <tss@iki.fi> parents: 2173 diff changeset	1152 if ((uflags &
2053 66f5d28f9b27 race condition fixes Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1153 MAILDIR_UIDLIST_REC_FLAG_RACING) != 0) {
66f5d28f9b27 race condition fixes Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1154 mail_storage_set_critical(
5459 78eaf595359c Removed struct index_storage abstraction. It's pointless. Timo Sirainen <tss@iki.fi> parents: 5443 diff changeset	1155 &mbox->storage->storage,
2067 3f97439ba59e Path was missing from Maildir sync-errors. Timo Sirainen <tss@iki.fi> parents: 2064 diff changeset	1156 "Maildir %s sync: "
3f97439ba59e Path was missing from Maildir sync-errors. Timo Sirainen <tss@iki.fi> parents: 2064 diff changeset	1157 "UID < next_uid "
2053 66f5d28f9b27 race condition fixes Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1158 "(%u < %u, file = %s)",
3279 b698ae839a18 Moved mbox/maildir-specific variables from struct index_mailbox to Timo Sirainen <tss@iki.fi> parents: 3203 diff changeset	1159 mbox->path, uid, hdr->next_uid,
2067 3f97439ba59e Path was missing from Maildir sync-errors. Timo Sirainen <tss@iki.fi> parents: 2064 diff changeset	1160 filename);
3279 b698ae839a18 Moved mbox/maildir-specific variables from struct index_mailbox to Timo Sirainen <tss@iki.fi> parents: 3203 diff changeset	1161 mail_index_mark_corrupted(
b698ae839a18 Moved mbox/maildir-specific variables from struct index_mailbox to Timo Sirainen <tss@iki.fi> parents: 3203 diff changeset	1162 mbox->ibox.index);
2053 66f5d28f9b27 race condition fixes Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1163 ret = -1;
66f5d28f9b27 race condition fixes Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1164 break;
66f5d28f9b27 race condition fixes Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1165 }
3279 b698ae839a18 Moved mbox/maildir-specific variables from struct index_mailbox to Timo Sirainen <tss@iki.fi> parents: 3203 diff changeset	1166 mbox->dirty_cur_time = ioloop_time;
b698ae839a18 Moved mbox/maildir-specific variables from struct index_mailbox to Timo Sirainen <tss@iki.fi> parents: 3203 diff changeset	1167 maildir_uidlist_add_flags(mbox->uidlist,
2053 66f5d28f9b27 race condition fixes Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1168 filename,
66f5d28f9b27 race condition fixes Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1169 MAILDIR_UIDLIST_REC_FLAG_RACING);
66f5d28f9b27 race condition fixes Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1170
66f5d28f9b27 race condition fixes Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1171 seq--;
66f5d28f9b27 race condition fixes Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1172 continue;
1984 9c159272f721 syncing fixes Timo Sirainen <tss@iki.fi> parents: 1978 diff changeset	1173 }
9c159272f721 syncing fixes Timo Sirainen <tss@iki.fi> parents: 1978 diff changeset	1174
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1175 mail_index_append(trans, uid, &seq);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1176 mail_index_update_flags(trans, seq, MODIFY_REPLACE,
3016 61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2936 diff changeset	1177 flags);
3447 96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	1178
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	1179 if (array_count(&keywords) > 0) {
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	1180 struct mail_keywords *kw;
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	1181
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	1182 kw = mail_index_keywords_create_from_indexes(
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	1183 trans, &keywords);
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	1184 mail_index_update_keywords(trans, seq,
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	1185 MODIFY_REPLACE, kw);
3879 928229f8b3e6 deinit, unref, destroy, close, free, etc. functions now take a pointer to Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	1186 mail_index_keywords_free(&kw);
3447 96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	1187 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1188 continue;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1189 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1190
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1191 if (mail_index_lookup(view, seq, &rec) < 0) {
5080 e0b83da1e12f Error handling fixes Timo Sirainen <tss@iki.fi> parents: 4907 diff changeset	1192 mail_storage_set_index_error(&mbox->ibox);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1193 ret = -1;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1194 break;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1195 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1196
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1197 if (rec->uid < uid) {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1198 /* expunged */
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1199 mail_index_expunge(trans, seq);
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1200 goto __again;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1201 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1202
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1203 if (rec->uid > uid) {
2034 8078400fe561 Fix bogus "UID inserted in the middle of mailbox" errors Timo Sirainen <tss@iki.fi> parents: 2033 diff changeset	1204 /* most likely a race condition: we read the
8078400fe561 Fix bogus "UID inserted in the middle of mailbox" errors Timo Sirainen <tss@iki.fi> parents: 2033 diff changeset	1205 maildir, then someone else expunged messages and
8078400fe561 Fix bogus "UID inserted in the middle of mailbox" errors Timo Sirainen <tss@iki.fi> parents: 2033 diff changeset	1206 committed changes to index. so, this message
2053 66f5d28f9b27 race condition fixes Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1207 shouldn't actually exist. mark it racy and check
66f5d28f9b27 race condition fixes Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1208 in next sync. */
2228 d19ba01fb5cd partial syncing fixes Timo Sirainen <tss@iki.fi> parents: 2173 diff changeset	1209 if ((uflags &
d19ba01fb5cd partial syncing fixes Timo Sirainen <tss@iki.fi> parents: 2173 diff changeset	1210 MAILDIR_UIDLIST_REC_FLAG_NONSYNCED) != 0) {
d19ba01fb5cd partial syncing fixes Timo Sirainen <tss@iki.fi> parents: 2173 diff changeset	1211 /* partial syncing */
d19ba01fb5cd partial syncing fixes Timo Sirainen <tss@iki.fi> parents: 2173 diff changeset	1212 seq--;
d19ba01fb5cd partial syncing fixes Timo Sirainen <tss@iki.fi> parents: 2173 diff changeset	1213 continue;
d19ba01fb5cd partial syncing fixes Timo Sirainen <tss@iki.fi> parents: 2173 diff changeset	1214 }
2053 66f5d28f9b27 race condition fixes Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1215 if ((uflags & MAILDIR_UIDLIST_REC_FLAG_RACING) != 0) {
3279 b698ae839a18 Moved mbox/maildir-specific variables from struct index_mailbox to Timo Sirainen <tss@iki.fi> parents: 3203 diff changeset	1216 mail_storage_set_critical(
5459 78eaf595359c Removed struct index_storage abstraction. It's pointless. Timo Sirainen <tss@iki.fi> parents: 5443 diff changeset	1217 &mbox->storage->storage,
2570 372d4b90c076 cleanup Timo Sirainen <tss@iki.fi> parents: 2533 diff changeset	1218 "Maildir %s sync: "
372d4b90c076 cleanup Timo Sirainen <tss@iki.fi> parents: 2533 diff changeset	1219 "UID inserted in the middle of mailbox "
2034 8078400fe561 Fix bogus "UID inserted in the middle of mailbox" errors Timo Sirainen <tss@iki.fi> parents: 2033 diff changeset	1220 "(%u > %u, file = %s)",
3279 b698ae839a18 Moved mbox/maildir-specific variables from struct index_mailbox to Timo Sirainen <tss@iki.fi> parents: 3203 diff changeset	1221 mbox->path, rec->uid, uid, filename);
b698ae839a18 Moved mbox/maildir-specific variables from struct index_mailbox to Timo Sirainen <tss@iki.fi> parents: 3203 diff changeset	1222 mail_index_mark_corrupted(mbox->ibox.index);
2034 8078400fe561 Fix bogus "UID inserted in the middle of mailbox" errors Timo Sirainen <tss@iki.fi> parents: 2033 diff changeset	1223 ret = -1;
8078400fe561 Fix bogus "UID inserted in the middle of mailbox" errors Timo Sirainen <tss@iki.fi> parents: 2033 diff changeset	1224 break;
2064 5cfdc99fab69 Don't use internal last_cur_mtime stamp - it's index-specific so always get Timo Sirainen <tss@iki.fi> parents: 2053 diff changeset	1225 }
2053 66f5d28f9b27 race condition fixes Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1226
3279 b698ae839a18 Moved mbox/maildir-specific variables from struct index_mailbox to Timo Sirainen <tss@iki.fi> parents: 3203 diff changeset	1227 mbox->dirty_cur_time = ioloop_time;
b698ae839a18 Moved mbox/maildir-specific variables from struct index_mailbox to Timo Sirainen <tss@iki.fi> parents: 3203 diff changeset	1228 maildir_uidlist_add_flags(mbox->uidlist, filename,
2053 66f5d28f9b27 race condition fixes Timo Sirainen <tss@iki.fi> parents: 2051 diff changeset	1229 MAILDIR_UIDLIST_REC_FLAG_RACING);
2034 8078400fe561 Fix bogus "UID inserted in the middle of mailbox" errors Timo Sirainen <tss@iki.fi> parents: 2033 diff changeset	1230
8078400fe561 Fix bogus "UID inserted in the middle of mailbox" errors Timo Sirainen <tss@iki.fi> parents: 2033 diff changeset	1231 seq--;
8078400fe561 Fix bogus "UID inserted in the middle of mailbox" errors Timo Sirainen <tss@iki.fi> parents: 2033 diff changeset	1232 continue;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1233 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1234
4107 d29677c59dc5 Keep \Seen flags privately only in indexes with shared mailboxes. Timo Sirainen <tss@iki.fi> parents: 3879 diff changeset	1235 /* the private flags are stored only in indexes, keep them */
d29677c59dc5 Keep \Seen flags privately only in indexes with shared mailboxes. Timo Sirainen <tss@iki.fi> parents: 3879 diff changeset	1236 flags \|= rec->flags & mbox->private_flags_mask;
d29677c59dc5 Keep \Seen flags privately only in indexes with shared mailboxes. Timo Sirainen <tss@iki.fi> parents: 3879 diff changeset	1237
2320 8a6666a9ac98 Handle recent flags in index file correctly. Fixes recent flag losing when Timo Sirainen <tss@iki.fi> parents: 2272 diff changeset	1238 if ((rec->flags & MAIL_RECENT) != 0) {
3279 b698ae839a18 Moved mbox/maildir-specific variables from struct index_mailbox to Timo Sirainen <tss@iki.fi> parents: 3203 diff changeset	1239 index_mailbox_set_recent(&mbox->ibox, seq);
b698ae839a18 Moved mbox/maildir-specific variables from struct index_mailbox to Timo Sirainen <tss@iki.fi> parents: 3203 diff changeset	1240 if (mbox->ibox.keep_recent) {
2320 8a6666a9ac98 Handle recent flags in index file correctly. Fixes recent flag losing when Timo Sirainen <tss@iki.fi> parents: 2272 diff changeset	1241 flags \|= MAIL_RECENT;
8a6666a9ac98 Handle recent flags in index file correctly. Fixes recent flag losing when Timo Sirainen <tss@iki.fi> parents: 2272 diff changeset	1242 } else {
8a6666a9ac98 Handle recent flags in index file correctly. Fixes recent flag losing when Timo Sirainen <tss@iki.fi> parents: 2272 diff changeset	1243 mail_index_update_flags(trans, seq,
8a6666a9ac98 Handle recent flags in index file correctly. Fixes recent flag losing when Timo Sirainen <tss@iki.fi> parents: 2272 diff changeset	1244 MODIFY_REMOVE,
3016 61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2936 diff changeset	1245 MAIL_RECENT);
2320 8a6666a9ac98 Handle recent flags in index file correctly. Fixes recent flag losing when Timo Sirainen <tss@iki.fi> parents: 2272 diff changeset	1246 }
8a6666a9ac98 Handle recent flags in index file correctly. Fixes recent flag losing when Timo Sirainen <tss@iki.fi> parents: 2272 diff changeset	1247 }
8a6666a9ac98 Handle recent flags in index file correctly. Fixes recent flag losing when Timo Sirainen <tss@iki.fi> parents: 2272 diff changeset	1248
2228 d19ba01fb5cd partial syncing fixes Timo Sirainen <tss@iki.fi> parents: 2173 diff changeset	1249 if ((uflags & MAILDIR_UIDLIST_REC_FLAG_NONSYNCED) != 0) {
d19ba01fb5cd partial syncing fixes Timo Sirainen <tss@iki.fi> parents: 2173 diff changeset	1250 /* partial syncing */
3436 3c51658d6846 We didn't notice if messages were deleted directly from new/. Timo Sirainen <tss@iki.fi> parents: 3435 diff changeset	1251 if ((flags & MAIL_RECENT) != 0) {
3c51658d6846 We didn't notice if messages were deleted directly from new/. Timo Sirainen <tss@iki.fi> parents: 3435 diff changeset	1252 /* we last saw this mail in new/, but it's
3c51658d6846 We didn't notice if messages were deleted directly from new/. Timo Sirainen <tss@iki.fi> parents: 3435 diff changeset	1253 not there anymore. possibly expunged,
3c51658d6846 We didn't notice if messages were deleted directly from new/. Timo Sirainen <tss@iki.fi> parents: 3435 diff changeset	1254 make sure. */
3c51658d6846 We didn't notice if messages were deleted directly from new/. Timo Sirainen <tss@iki.fi> parents: 3435 diff changeset	1255 full_rescan = TRUE;
3c51658d6846 We didn't notice if messages were deleted directly from new/. Timo Sirainen <tss@iki.fi> parents: 3435 diff changeset	1256 }
2228 d19ba01fb5cd partial syncing fixes Timo Sirainen <tss@iki.fi> parents: 2173 diff changeset	1257 continue;
d19ba01fb5cd partial syncing fixes Timo Sirainen <tss@iki.fi> parents: 2173 diff changeset	1258 }
d19ba01fb5cd partial syncing fixes Timo Sirainen <tss@iki.fi> parents: 2173 diff changeset	1259
1956 d6941cd8afdc Added support for setting dirty flags for messages (TODO: undirty..) Timo Sirainen <tss@iki.fi> parents: 1955 diff changeset	1260 if ((rec->flags & MAIL_INDEX_MAIL_FLAG_DIRTY) != 0) {
d6941cd8afdc Added support for setting dirty flags for messages (TODO: undirty..) Timo Sirainen <tss@iki.fi> parents: 1955 diff changeset	1261 /* we haven't been able to update maildir with this
d6941cd8afdc Added support for setting dirty flags for messages (TODO: undirty..) Timo Sirainen <tss@iki.fi> parents: 1955 diff changeset	1262 record's flag changes. don't sync them. */
d6941cd8afdc Added support for setting dirty flags for messages (TODO: undirty..) Timo Sirainen <tss@iki.fi> parents: 1955 diff changeset	1263 continue;
d6941cd8afdc Added support for setting dirty flags for messages (TODO: undirty..) Timo Sirainen <tss@iki.fi> parents: 1955 diff changeset	1264 }
d6941cd8afdc Added support for setting dirty flags for messages (TODO: undirty..) Timo Sirainen <tss@iki.fi> parents: 1955 diff changeset	1265
2038 df504dad3aec Recent flag fixes. Should work perfectly now with maildir. Timo Sirainen <tss@iki.fi> parents: 2037 diff changeset	1266 if (((uint8_t)flags & ~MAIL_RECENT) !=
3016 61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2936 diff changeset	1267 (rec->flags & (MAIL_FLAGS_MASK^MAIL_RECENT))) {
2038 df504dad3aec Recent flag fixes. Should work perfectly now with maildir. Timo Sirainen <tss@iki.fi> parents: 2037 diff changeset	1268 /* FIXME: this is wrong if there's pending changes in
df504dad3aec Recent flag fixes. Should work perfectly now with maildir. Timo Sirainen <tss@iki.fi> parents: 2037 diff changeset	1269 transaction log already. it gets fixed in next sync
df504dad3aec Recent flag fixes. Should work perfectly now with maildir. Timo Sirainen <tss@iki.fi> parents: 2037 diff changeset	1270 however.. */
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1271 mail_index_update_flags(trans, seq, MODIFY_REPLACE,
3016 61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2936 diff changeset	1272 flags);
2038 df504dad3aec Recent flag fixes. Should work perfectly now with maildir. Timo Sirainen <tss@iki.fi> parents: 2037 diff changeset	1273 } else if ((flags & MAIL_RECENT) == 0 &&
df504dad3aec Recent flag fixes. Should work perfectly now with maildir. Timo Sirainen <tss@iki.fi> parents: 2037 diff changeset	1274 (rec->flags & MAIL_RECENT) != 0) {
df504dad3aec Recent flag fixes. Should work perfectly now with maildir. Timo Sirainen <tss@iki.fi> parents: 2037 diff changeset	1275 /* just remove recent flag */
df504dad3aec Recent flag fixes. Should work perfectly now with maildir. Timo Sirainen <tss@iki.fi> parents: 2037 diff changeset	1276 mail_index_update_flags(trans, seq, MODIFY_REMOVE,
3016 61c8d205d887 Initial support for keywords. Syncing to mbox/maildir doesn't work yet. Timo Sirainen <tss@iki.fi> parents: 2936 diff changeset	1277 MAIL_RECENT);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1278 }
3447 96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	1279
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	1280 /* update keywords if they have changed */
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	1281 if (mail_index_lookup_keywords(view, seq, &idx_keywords) < 0) {
5080 e0b83da1e12f Error handling fixes Timo Sirainen <tss@iki.fi> parents: 4907 diff changeset	1282 mail_storage_set_index_error(&mbox->ibox);
3447 96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	1283 ret = -1;
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	1284 break;
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	1285 }
3821 c8b2ed2c9961 We assumed that keyword index arrays were always sorted. This isn't always Timo Sirainen <tss@iki.fi> parents: 3776 diff changeset	1286 if (!index_keyword_array_cmp(&keywords, &idx_keywords)) {
3447 96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	1287 struct mail_keywords *kw;
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	1288
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	1289 kw = mail_index_keywords_create_from_indexes(
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	1290 trans, &keywords);
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	1291 mail_index_update_keywords(trans, seq,
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	1292 MODIFY_REPLACE, kw);
3879 928229f8b3e6 deinit, unref, destroy, close, free, etc. functions now take a pointer to Timo Sirainen <tss@iki.fi> parents: 3863 diff changeset	1293 mail_index_keywords_free(&kw);
3447 96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	1294 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1295 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1296 maildir_uidlist_iter_deinit(iter);
3447 96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	1297 array_free(&keywords);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1298
3776 0cae3268c8af Accidentally had changed !partial-check to partial-check. Because of this Timo Sirainen <tss@iki.fi> parents: 3775 diff changeset	1299 if (!partial) {
1954 2f6e137cdc44 Syncing optimizations. Timo Sirainen <tss@iki.fi> parents: 1947 diff changeset	1300 /* expunge the rest */
2f6e137cdc44 Syncing optimizations. Timo Sirainen <tss@iki.fi> parents: 1947 diff changeset	1301 for (seq++; seq <= hdr->messages_count; seq++)
2f6e137cdc44 Syncing optimizations. Timo Sirainen <tss@iki.fi> parents: 1947 diff changeset	1302 mail_index_expunge(trans, seq);
3583 a45208fe7c83 Partial syncs caused index's next_uid to be updated even if the new mails Timo Sirainen <tss@iki.fi> parents: 3579 diff changeset	1303
a45208fe7c83 Partial syncs caused index's next_uid to be updated even if the new mails Timo Sirainen <tss@iki.fi> parents: 3579 diff changeset	1304 /* next_uid must be updated only in non-partial syncs since
a45208fe7c83 Partial syncs caused index's next_uid to be updated even if the new mails Timo Sirainen <tss@iki.fi> parents: 3579 diff changeset	1305 partial syncs don't add the new mails to index. also we'll
a45208fe7c83 Partial syncs caused index's next_uid to be updated even if the new mails Timo Sirainen <tss@iki.fi> parents: 3579 diff changeset	1306 have to do it here before syncing index records, since after
a45208fe7c83 Partial syncs caused index's next_uid to be updated even if the new mails Timo Sirainen <tss@iki.fi> parents: 3579 diff changeset	1307 that the uidlist's next_uid value may have changed. */
a45208fe7c83 Partial syncs caused index's next_uid to be updated even if the new mails Timo Sirainen <tss@iki.fi> parents: 3579 diff changeset	1308 next_uid = maildir_uidlist_get_next_uid(mbox->uidlist);
5369 e897aaa24cdd If uidlist is deleted and we set the uidvalidity from the index file, set Timo Sirainen <tss@iki.fi> parents: 5368 diff changeset	1309 if (hdr->next_uid < next_uid) {
3583 a45208fe7c83 Partial syncs caused index's next_uid to be updated even if the new mails Timo Sirainen <tss@iki.fi> parents: 3579 diff changeset	1310 mail_index_update_header(trans,
a45208fe7c83 Partial syncs caused index's next_uid to be updated even if the new mails Timo Sirainen <tss@iki.fi> parents: 3579 diff changeset	1311 offsetof(struct mail_index_header, next_uid),
a45208fe7c83 Partial syncs caused index's next_uid to be updated even if the new mails Timo Sirainen <tss@iki.fi> parents: 3579 diff changeset	1312 &next_uid, sizeof(next_uid), FALSE);
a45208fe7c83 Partial syncs caused index's next_uid to be updated even if the new mails Timo Sirainen <tss@iki.fi> parents: 3579 diff changeset	1313 }
1954 2f6e137cdc44 Syncing optimizations. Timo Sirainen <tss@iki.fi> parents: 1947 diff changeset	1314 }
2f6e137cdc44 Syncing optimizations. Timo Sirainen <tss@iki.fi> parents: 1947 diff changeset	1315
3583 a45208fe7c83 Partial syncs caused index's next_uid to be updated even if the new mails Timo Sirainen <tss@iki.fi> parents: 3579 diff changeset	1316 if (!mbox->syncing_commit) {
a45208fe7c83 Partial syncs caused index's next_uid to be updated even if the new mails Timo Sirainen <tss@iki.fi> parents: 3579 diff changeset	1317 /* now, sync the index. NOTE: may recurse back to here with
a45208fe7c83 Partial syncs caused index's next_uid to be updated even if the new mails Timo Sirainen <tss@iki.fi> parents: 3579 diff changeset	1318 partial syncs */
a45208fe7c83 Partial syncs caused index's next_uid to be updated even if the new mails Timo Sirainen <tss@iki.fi> parents: 3579 diff changeset	1319 mbox->syncing_commit = TRUE;
a45208fe7c83 Partial syncs caused index's next_uid to be updated even if the new mails Timo Sirainen <tss@iki.fi> parents: 3579 diff changeset	1320 if (maildir_sync_index_records(sync_ctx) < 0)
a45208fe7c83 Partial syncs caused index's next_uid to be updated even if the new mails Timo Sirainen <tss@iki.fi> parents: 3579 diff changeset	1321 ret = -1;
a45208fe7c83 Partial syncs caused index's next_uid to be updated even if the new mails Timo Sirainen <tss@iki.fi> parents: 3579 diff changeset	1322 mbox->syncing_commit = FALSE;
5564 47274663a71c Call sync_notify() once at the end of syncing. Allow accessing mails even Timo Sirainen <tss@iki.fi> parents: 5562 diff changeset	1323
47274663a71c Call sync_notify() once at the end of syncing. Allow accessing mails even Timo Sirainen <tss@iki.fi> parents: 5562 diff changeset	1324 if (mbox->ibox.box.v.sync_notify != NULL)
47274663a71c Call sync_notify() once at the end of syncing. Allow accessing mails even Timo Sirainen <tss@iki.fi> parents: 5562 diff changeset	1325 mbox->ibox.box.v.sync_notify(&mbox->ibox.box, 0, 0);
3583 a45208fe7c83 Partial syncs caused index's next_uid to be updated even if the new mails Timo Sirainen <tss@iki.fi> parents: 3579 diff changeset	1326 }
2037 8763032d31bd Set dirty flags through transaction log, not directly. Some other flag Timo Sirainen <tss@iki.fi> parents: 2034 diff changeset	1327
4848 967de900c73a Mailbox list indexing and related changes. Currently works only with Timo Sirainen <tss@iki.fi> parents: 4774 diff changeset	1328 if (mbox->dirty_cur_time != 0)
967de900c73a Mailbox list indexing and related changes. Currently works only with Timo Sirainen <tss@iki.fi> parents: 4774 diff changeset	1329 mbox->last_dirty_flags \|= MAILDIR_DIRTY_CUR;
967de900c73a Mailbox list indexing and related changes. Currently works only with Timo Sirainen <tss@iki.fi> parents: 4774 diff changeset	1330
967de900c73a Mailbox list indexing and related changes. Currently works only with Timo Sirainen <tss@iki.fi> parents: 4774 diff changeset	1331 if (mbox->last_cur_mtime != (time_t)hdr->sync_stamp) {
3279 b698ae839a18 Moved mbox/maildir-specific variables from struct index_mailbox to Timo Sirainen <tss@iki.fi> parents: 3203 diff changeset	1332 uint32_t sync_stamp = mbox->last_cur_mtime;
2050 ee1095ccfd23 Index header changes now go through transaction log. Removed the kludgy Timo Sirainen <tss@iki.fi> parents: 2039 diff changeset	1333
ee1095ccfd23 Index header changes now go through transaction log. Removed the kludgy Timo Sirainen <tss@iki.fi> parents: 2039 diff changeset	1334 mail_index_update_header(trans,
ee1095ccfd23 Index header changes now go through transaction log. Removed the kludgy Timo Sirainen <tss@iki.fi> parents: 2039 diff changeset	1335 offsetof(struct mail_index_header, sync_stamp),
3322 49071cc19102 If UIDVALIDITY changes, don't invalidate the whole index. Just expunge all Timo Sirainen <tss@iki.fi> parents: 3280 diff changeset	1336 &sync_stamp, sizeof(sync_stamp), TRUE);
2050 ee1095ccfd23 Index header changes now go through transaction log. Removed the kludgy Timo Sirainen <tss@iki.fi> parents: 2039 diff changeset	1337 }
ee1095ccfd23 Index header changes now go through transaction log. Removed the kludgy Timo Sirainen <tss@iki.fi> parents: 2039 diff changeset	1338
3472 db29cc6754d5 Store new/ directory's timestamp in sync_size header in index (kludgy..). Timo Sirainen <tss@iki.fi> parents: 3470 diff changeset	1339 /* FIXME: use a header extension instead of sync_size.. */
4848 967de900c73a Mailbox list indexing and related changes. Currently works only with Timo Sirainen <tss@iki.fi> parents: 4774 diff changeset	1340 value = mbox->last_new_mtime \|
967de900c73a Mailbox list indexing and related changes. Currently works only with Timo Sirainen <tss@iki.fi> parents: 4774 diff changeset	1341 ((uint64_t)mbox->last_dirty_flags << 32);
3472 db29cc6754d5 Store new/ directory's timestamp in sync_size header in index (kludgy..). Timo Sirainen <tss@iki.fi> parents: 3470 diff changeset	1342 if (value != hdr->sync_size) {
db29cc6754d5 Store new/ directory's timestamp in sync_size header in index (kludgy..). Timo Sirainen <tss@iki.fi> parents: 3470 diff changeset	1343 mail_index_update_header(trans,
db29cc6754d5 Store new/ directory's timestamp in sync_size header in index (kludgy..). Timo Sirainen <tss@iki.fi> parents: 3470 diff changeset	1344 offsetof(struct mail_index_header, sync_size),
4848 967de900c73a Mailbox list indexing and related changes. Currently works only with Timo Sirainen <tss@iki.fi> parents: 4774 diff changeset	1345 &value, sizeof(value), TRUE);
3472 db29cc6754d5 Store new/ directory's timestamp in sync_size header in index (kludgy..). Timo Sirainen <tss@iki.fi> parents: 3470 diff changeset	1346 }
db29cc6754d5 Store new/ directory's timestamp in sync_size header in index (kludgy..). Timo Sirainen <tss@iki.fi> parents: 3470 diff changeset	1347
2051 596267d8e2e2 Some more UIDVALIDITY issues fixed. Timo Sirainen <tss@iki.fi> parents: 2050 diff changeset	1348 if (hdr->uid_validity == 0) {
596267d8e2e2 Some more UIDVALIDITY issues fixed. Timo Sirainen <tss@iki.fi> parents: 2050 diff changeset	1349 /* get the initial uidvalidity */
3279 b698ae839a18 Moved mbox/maildir-specific variables from struct index_mailbox to Timo Sirainen <tss@iki.fi> parents: 3203 diff changeset	1350 if (maildir_uidlist_update(mbox->uidlist) < 0)
2051 596267d8e2e2 Some more UIDVALIDITY issues fixed. Timo Sirainen <tss@iki.fi> parents: 2050 diff changeset	1351 ret = -1;
3279 b698ae839a18 Moved mbox/maildir-specific variables from struct index_mailbox to Timo Sirainen <tss@iki.fi> parents: 3203 diff changeset	1352 uid_validity = maildir_uidlist_get_uid_validity(mbox->uidlist);
2051 596267d8e2e2 Some more UIDVALIDITY issues fixed. Timo Sirainen <tss@iki.fi> parents: 2050 diff changeset	1353 if (uid_validity == 0) {
596267d8e2e2 Some more UIDVALIDITY issues fixed. Timo Sirainen <tss@iki.fi> parents: 2050 diff changeset	1354 uid_validity = ioloop_time;
3279 b698ae839a18 Moved mbox/maildir-specific variables from struct index_mailbox to Timo Sirainen <tss@iki.fi> parents: 3203 diff changeset	1355 maildir_uidlist_set_uid_validity(mbox->uidlist,
5388 ce1bfc98df29 Last fix broke creating new uidlists. Timo Sirainen <tss@iki.fi> parents: 5369 diff changeset	1356 uid_validity, 0);
2051 596267d8e2e2 Some more UIDVALIDITY issues fixed. Timo Sirainen <tss@iki.fi> parents: 2050 diff changeset	1357 }
596267d8e2e2 Some more UIDVALIDITY issues fixed. Timo Sirainen <tss@iki.fi> parents: 2050 diff changeset	1358 } else if (uid_validity == 0) {
3279 b698ae839a18 Moved mbox/maildir-specific variables from struct index_mailbox to Timo Sirainen <tss@iki.fi> parents: 3203 diff changeset	1359 maildir_uidlist_set_uid_validity(mbox->uidlist,
5369 e897aaa24cdd If uidlist is deleted and we set the uidvalidity from the index file, set Timo Sirainen <tss@iki.fi> parents: 5368 diff changeset	1360 hdr->uid_validity,
e897aaa24cdd If uidlist is deleted and we set the uidvalidity from the index file, set Timo Sirainen <tss@iki.fi> parents: 5368 diff changeset	1361 hdr->next_uid);
2051 596267d8e2e2 Some more UIDVALIDITY issues fixed. Timo Sirainen <tss@iki.fi> parents: 2050 diff changeset	1362 }
596267d8e2e2 Some more UIDVALIDITY issues fixed. Timo Sirainen <tss@iki.fi> parents: 2050 diff changeset	1363
596267d8e2e2 Some more UIDVALIDITY issues fixed. Timo Sirainen <tss@iki.fi> parents: 2050 diff changeset	1364 if (uid_validity != hdr->uid_validity && uid_validity != 0) {
2050 ee1095ccfd23 Index header changes now go through transaction log. Removed the kludgy Timo Sirainen <tss@iki.fi> parents: 2039 diff changeset	1365 mail_index_update_header(trans,
ee1095ccfd23 Index header changes now go through transaction log. Removed the kludgy Timo Sirainen <tss@iki.fi> parents: 2039 diff changeset	1366 offsetof(struct mail_index_header, uid_validity),
3322 49071cc19102 If UIDVALIDITY changes, don't invalidate the whole index. Just expunge all Timo Sirainen <tss@iki.fi> parents: 3280 diff changeset	1367 &uid_validity, sizeof(uid_validity), TRUE);
2050 ee1095ccfd23 Index header changes now go through transaction log. Removed the kludgy Timo Sirainen <tss@iki.fi> parents: 2039 diff changeset	1368 }
ee1095ccfd23 Index header changes now go through transaction log. Removed the kludgy Timo Sirainen <tss@iki.fi> parents: 2039 diff changeset	1369
3436 3c51658d6846 We didn't notice if messages were deleted directly from new/. Timo Sirainen <tss@iki.fi> parents: 3435 diff changeset	1370 return ret < 0 ? -1 : (full_rescan ? 0 : 1);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1371 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1372
3863 55df57c028d4 Added "bool" type and changed all ints that were used as booleans to bool. Timo Sirainen <tss@iki.fi> parents: 3821 diff changeset	1373 static int maildir_sync_context(struct maildir_sync_context *ctx, bool forced,
55df57c028d4 Added "bool" type and changed all ints that were used as booleans to bool. Timo Sirainen <tss@iki.fi> parents: 3821 diff changeset	1374 bool sync_last_commit)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1375 {
3863 55df57c028d4 Added "bool" type and changed all ints that were used as booleans to bool. Timo Sirainen <tss@iki.fi> parents: 3821 diff changeset	1376 bool new_changed, cur_changed, full_rescan = FALSE;
55df57c028d4 Added "bool" type and changed all ints that were used as booleans to bool. Timo Sirainen <tss@iki.fi> parents: 3821 diff changeset	1377 int ret;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1378
3447 96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	1379 if (sync_last_commit) {
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	1380 new_changed = cur_changed = FALSE;
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	1381 } else if (!forced) {
3472 db29cc6754d5 Store new/ directory's timestamp in sync_size header in index (kludgy..). Timo Sirainen <tss@iki.fi> parents: 3470 diff changeset	1382 if (maildir_sync_quick_check(ctx->mbox,
db29cc6754d5 Store new/ directory's timestamp in sync_size header in index (kludgy..). Timo Sirainen <tss@iki.fi> parents: 3470 diff changeset	1383 ctx->new_dir, ctx->cur_dir,
db29cc6754d5 Store new/ directory's timestamp in sync_size header in index (kludgy..). Timo Sirainen <tss@iki.fi> parents: 3470 diff changeset	1384 &new_changed, &cur_changed) < 0)
2140 e2cd51b99359 "readonly sync" -> "forced sync" Timo Sirainen <tss@iki.fi> parents: 2123 diff changeset	1385 return -1;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1386
2140 e2cd51b99359 "readonly sync" -> "forced sync" Timo Sirainen <tss@iki.fi> parents: 2123 diff changeset	1387 if (!new_changed && !cur_changed)
3579 09565508b8ba We were forcing a maildir resync when it hadn't changed at all. Timo Sirainen <tss@iki.fi> parents: 3563 diff changeset	1388 return 1;
2140 e2cd51b99359 "readonly sync" -> "forced sync" Timo Sirainen <tss@iki.fi> parents: 2123 diff changeset	1389 } else {
e2cd51b99359 "readonly sync" -> "forced sync" Timo Sirainen <tss@iki.fi> parents: 2123 diff changeset	1390 new_changed = cur_changed = TRUE;
e2cd51b99359 "readonly sync" -> "forced sync" Timo Sirainen <tss@iki.fi> parents: 2123 diff changeset	1391 }
1947 777da553d1d3 Recent-flag should work now Timo Sirainen <tss@iki.fi> parents: 1944 diff changeset	1392
2818 a758a5b542bb Always protect maildir syncing with uidlist lock. Before we only tried to Timo Sirainen <tss@iki.fi> parents: 2816 diff changeset	1393 /*
a758a5b542bb Always protect maildir syncing with uidlist lock. Before we only tried to Timo Sirainen <tss@iki.fi> parents: 2816 diff changeset	1394 Locking, locking, locking.. Wasn't maildir supposed to be lockless?
a758a5b542bb Always protect maildir syncing with uidlist lock. Before we only tried to Timo Sirainen <tss@iki.fi> parents: 2816 diff changeset	1395
a758a5b542bb Always protect maildir syncing with uidlist lock. Before we only tried to Timo Sirainen <tss@iki.fi> parents: 2816 diff changeset	1396 We can get here either as beginning a real maildir sync, or when
a758a5b542bb Always protect maildir syncing with uidlist lock. Before we only tried to Timo Sirainen <tss@iki.fi> parents: 2816 diff changeset	1397 committing changes to maildir but a file was lost (maybe renamed).
a758a5b542bb Always protect maildir syncing with uidlist lock. Before we only tried to Timo Sirainen <tss@iki.fi> parents: 2816 diff changeset	1398
a758a5b542bb Always protect maildir syncing with uidlist lock. Before we only tried to Timo Sirainen <tss@iki.fi> parents: 2816 diff changeset	1399 So, we're going to need two locks. One for index and one for
4238 3c8b191b0019 Adding mail to index while saving it had a race condition. Fixing it Timo Sirainen <timo.sirainen@movial.fi> parents: 4199 diff changeset	1400 uidlist. To avoid deadlocking do the uidlist lock always first.
2121 0840edf34f37 Locking fixes. use less memory Timo Sirainen <tss@iki.fi> parents: 2067 diff changeset	1401
2818 a758a5b542bb Always protect maildir syncing with uidlist lock. Before we only tried to Timo Sirainen <tss@iki.fi> parents: 2816 diff changeset	1402 uidlist is needed only for figuring out UIDs for newly seen files,
a758a5b542bb Always protect maildir syncing with uidlist lock. Before we only tried to Timo Sirainen <tss@iki.fi> parents: 2816 diff changeset	1403 so theoretically we wouldn't need to lock it unless there are new
a758a5b542bb Always protect maildir syncing with uidlist lock. Before we only tried to Timo Sirainen <tss@iki.fi> parents: 2816 diff changeset	1404 files. It has a few problems though, assuming the index lock didn't
a758a5b542bb Always protect maildir syncing with uidlist lock. Before we only tried to Timo Sirainen <tss@iki.fi> parents: 2816 diff changeset	1405 already protect it (eg. in-memory indexes):
a758a5b542bb Always protect maildir syncing with uidlist lock. Before we only tried to Timo Sirainen <tss@iki.fi> parents: 2816 diff changeset	1406
a758a5b542bb Always protect maildir syncing with uidlist lock. Before we only tried to Timo Sirainen <tss@iki.fi> parents: 2816 diff changeset	1407 1. Just because you see a new file which doesn't exist in uidlist
a758a5b542bb Always protect maildir syncing with uidlist lock. Before we only tried to Timo Sirainen <tss@iki.fi> parents: 2816 diff changeset	1408 file, doesn't mean that the file really exists anymore, or that
a758a5b542bb Always protect maildir syncing with uidlist lock. Before we only tried to Timo Sirainen <tss@iki.fi> parents: 2816 diff changeset	1409 your readdir() lists all new files. Meaning that this is possible:
a758a5b542bb Always protect maildir syncing with uidlist lock. Before we only tried to Timo Sirainen <tss@iki.fi> parents: 2816 diff changeset	1410
a758a5b542bb Always protect maildir syncing with uidlist lock. Before we only tried to Timo Sirainen <tss@iki.fi> parents: 2816 diff changeset	1411 A: opendir(), readdir() -> new file ...
a758a5b542bb Always protect maildir syncing with uidlist lock. Before we only tried to Timo Sirainen <tss@iki.fi> parents: 2816 diff changeset	1412 -- new files are written to the maildir --
a758a5b542bb Always protect maildir syncing with uidlist lock. Before we only tried to Timo Sirainen <tss@iki.fi> parents: 2816 diff changeset	1413 B: opendir(), readdir() -> new file, lock uidlist,
a758a5b542bb Always protect maildir syncing with uidlist lock. Before we only tried to Timo Sirainen <tss@iki.fi> parents: 2816 diff changeset	1414 readdir() -> another new file, rewrite uidlist, unlock
a758a5b542bb Always protect maildir syncing with uidlist lock. Before we only tried to Timo Sirainen <tss@iki.fi> parents: 2816 diff changeset	1415 A: ... lock uidlist, readdir() -> nothing left, rewrite uidlist,
a758a5b542bb Always protect maildir syncing with uidlist lock. Before we only tried to Timo Sirainen <tss@iki.fi> parents: 2816 diff changeset	1416 unlock
2173 9438951e243f Don't crash if we're syncing last commit to maildir, but some file was lost Timo Sirainen <tss@iki.fi> parents: 2140 diff changeset	1417
2818 a758a5b542bb Always protect maildir syncing with uidlist lock. Before we only tried to Timo Sirainen <tss@iki.fi> parents: 2816 diff changeset	1418 The second time running A didn't see the two new files. To handle
a758a5b542bb Always protect maildir syncing with uidlist lock. Before we only tried to Timo Sirainen <tss@iki.fi> parents: 2816 diff changeset	1419 this correctly, it must not remove the new unseen files from
a758a5b542bb Always protect maildir syncing with uidlist lock. Before we only tried to Timo Sirainen <tss@iki.fi> parents: 2816 diff changeset	1420 uidlist. This is possible to do, but adds extra complexity.
a758a5b542bb Always protect maildir syncing with uidlist lock. Before we only tried to Timo Sirainen <tss@iki.fi> parents: 2816 diff changeset	1421
a758a5b542bb Always protect maildir syncing with uidlist lock. Before we only tried to Timo Sirainen <tss@iki.fi> parents: 2816 diff changeset	1422 2. If another process is rename()ing files while we are
a758a5b542bb Always protect maildir syncing with uidlist lock. Before we only tried to Timo Sirainen <tss@iki.fi> parents: 2816 diff changeset	1423 readdir()ing, it's possible that readdir() never lists some files,
a758a5b542bb Always protect maildir syncing with uidlist lock. Before we only tried to Timo Sirainen <tss@iki.fi> parents: 2816 diff changeset	1424 causing Dovecot to assume they were expunged. In next sync they
a758a5b542bb Always protect maildir syncing with uidlist lock. Before we only tried to Timo Sirainen <tss@iki.fi> parents: 2816 diff changeset	1425 would show up again, but client could have already been notified of
a758a5b542bb Always protect maildir syncing with uidlist lock. Before we only tried to Timo Sirainen <tss@iki.fi> parents: 2816 diff changeset	1426 that and they would show up under new UIDs, so the damage is
a758a5b542bb Always protect maildir syncing with uidlist lock. Before we only tried to Timo Sirainen <tss@iki.fi> parents: 2816 diff changeset	1427 already done.
a758a5b542bb Always protect maildir syncing with uidlist lock. Before we only tried to Timo Sirainen <tss@iki.fi> parents: 2816 diff changeset	1428
a758a5b542bb Always protect maildir syncing with uidlist lock. Before we only tried to Timo Sirainen <tss@iki.fi> parents: 2816 diff changeset	1429 Both of the problems can be avoided if we simply lock the uidlist
a758a5b542bb Always protect maildir syncing with uidlist lock. Before we only tried to Timo Sirainen <tss@iki.fi> parents: 2816 diff changeset	1430 before syncing and keep it until sync is finished. Typically this
a758a5b542bb Always protect maildir syncing with uidlist lock. Before we only tried to Timo Sirainen <tss@iki.fi> parents: 2816 diff changeset	1431 would happen in any case, as there is the index lock..
a758a5b542bb Always protect maildir syncing with uidlist lock. Before we only tried to Timo Sirainen <tss@iki.fi> parents: 2816 diff changeset	1432
a758a5b542bb Always protect maildir syncing with uidlist lock. Before we only tried to Timo Sirainen <tss@iki.fi> parents: 2816 diff changeset	1433 The second case is still a problem with external changes though,
a758a5b542bb Always protect maildir syncing with uidlist lock. Before we only tried to Timo Sirainen <tss@iki.fi> parents: 2816 diff changeset	1434 because maildir doesn't require any kind of locking. Luckily this
a758a5b542bb Always protect maildir syncing with uidlist lock. Before we only tried to Timo Sirainen <tss@iki.fi> parents: 2816 diff changeset	1435 problem rarely happens except under high amount of modifications.
a758a5b542bb Always protect maildir syncing with uidlist lock. Before we only tried to Timo Sirainen <tss@iki.fi> parents: 2816 diff changeset	1436 */
a758a5b542bb Always protect maildir syncing with uidlist lock. Before we only tried to Timo Sirainen <tss@iki.fi> parents: 2816 diff changeset	1437
3530 e9695ec7925b Recursive maildir uidlist syncs caused assert crashes. Also did some Timo Sirainen <tss@iki.fi> parents: 3520 diff changeset	1438 ctx->partial = !cur_changed;
e9695ec7925b Recursive maildir uidlist syncs caused assert crashes. Also did some Timo Sirainen <tss@iki.fi> parents: 3520 diff changeset	1439 ret = maildir_uidlist_sync_init(ctx->mbox->uidlist, ctx->partial,
e9695ec7925b Recursive maildir uidlist syncs caused assert crashes. Also did some Timo Sirainen <tss@iki.fi> parents: 3520 diff changeset	1440 &ctx->uidlist_sync_ctx);
e9695ec7925b Recursive maildir uidlist syncs caused assert crashes. Also did some Timo Sirainen <tss@iki.fi> parents: 3520 diff changeset	1441 if (ret <= 0) {
2818 a758a5b542bb Always protect maildir syncing with uidlist lock. Before we only tried to Timo Sirainen <tss@iki.fi> parents: 2816 diff changeset	1442 /* failure / timeout. if forced is TRUE, we could still go
a758a5b542bb Always protect maildir syncing with uidlist lock. Before we only tried to Timo Sirainen <tss@iki.fi> parents: 2816 diff changeset	1443 forward and check only for renamed files, but is it worth
a758a5b542bb Always protect maildir syncing with uidlist lock. Before we only tried to Timo Sirainen <tss@iki.fi> parents: 2816 diff changeset	1444 the trouble? .. */
a758a5b542bb Always protect maildir syncing with uidlist lock. Before we only tried to Timo Sirainen <tss@iki.fi> parents: 2816 diff changeset	1445 return ret;
2140 e2cd51b99359 "readonly sync" -> "forced sync" Timo Sirainen <tss@iki.fi> parents: 2123 diff changeset	1446 }
2123 e01de478882f locking fixes Timo Sirainen <tss@iki.fi> parents: 2121 diff changeset	1447
4238 3c8b191b0019 Adding mail to index while saving it had a race condition. Fixing it Timo Sirainen <timo.sirainen@movial.fi> parents: 4199 diff changeset	1448 if (!ctx->mbox->syncing_commit) {
3c8b191b0019 Adding mail to index while saving it had a race condition. Fixing it Timo Sirainen <timo.sirainen@movial.fi> parents: 4199 diff changeset	1449 if (maildir_sync_index_begin(ctx->mbox,
3c8b191b0019 Adding mail to index while saving it had a race condition. Fixing it Timo Sirainen <timo.sirainen@movial.fi> parents: 4199 diff changeset	1450 &ctx->index_sync_ctx) < 0)
3c8b191b0019 Adding mail to index while saving it had a race condition. Fixing it Timo Sirainen <timo.sirainen@movial.fi> parents: 4199 diff changeset	1451 return -1;
5368 7d45edb81fe4 When copying/syncing a lot of mails, send "* OK Hang in there" replies to Timo Sirainen <tss@iki.fi> parents: 5307 diff changeset	1452 ctx->index_sync_ctx->maildir_sync_ctx = ctx;
4238 3c8b191b0019 Adding mail to index while saving it had a race condition. Fixing it Timo Sirainen <timo.sirainen@movial.fi> parents: 4199 diff changeset	1453 }
3c8b191b0019 Adding mail to index while saving it had a race condition. Fixing it Timo Sirainen <timo.sirainen@movial.fi> parents: 4199 diff changeset	1454
3447 96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	1455 if (new_changed \|\| cur_changed) {
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	1456 /* if we're going to check cur/ dir our current logic requires
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	1457 that new/ dir is checked as well. it's a good idea anyway. */
5391 0c8705aad54c Avoid infinite looping when buggy filesystems. Timo Sirainen <tss@iki.fi> parents: 5390 diff changeset	1458 unsigned int count = 0;
0c8705aad54c Avoid infinite looping when buggy filesystems. Timo Sirainen <tss@iki.fi> parents: 5390 diff changeset	1459
3447 96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	1460 while ((ret = maildir_scan_dir(ctx, TRUE)) > 0) {
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	1461 /* rename()d at least some files, which might have
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	1462 caused some other files to be missed. check again
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	1463 (see MAILDIR_RENAME_RESCAN_COUNT). */
5391 0c8705aad54c Avoid infinite looping when buggy filesystems. Timo Sirainen <tss@iki.fi> parents: 5390 diff changeset	1464 if (++count > MAILDIR_SCAN_DIR_MAX_COUNT)
0c8705aad54c Avoid infinite looping when buggy filesystems. Timo Sirainen <tss@iki.fi> parents: 5390 diff changeset	1465 break;
3447 96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	1466 }
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	1467 if (ret < 0)
1954 2f6e137cdc44 Syncing optimizations. Timo Sirainen <tss@iki.fi> parents: 1947 diff changeset	1468 return -1;
3447 96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	1469
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	1470 if (cur_changed) {
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	1471 if (maildir_scan_dir(ctx, FALSE) < 0)
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	1472 return -1;
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	1473 }
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	1474
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	1475 /* finish uidlist syncing, but keep it still locked */
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	1476 maildir_uidlist_sync_finish(ctx->uidlist_sync_ctx);
1954 2f6e137cdc44 Syncing optimizations. Timo Sirainen <tss@iki.fi> parents: 1947 diff changeset	1477 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1478
3279 b698ae839a18 Moved mbox/maildir-specific variables from struct index_mailbox to Timo Sirainen <tss@iki.fi> parents: 3203 diff changeset	1479 if (!ctx->mbox->syncing_commit) {
3530 e9695ec7925b Recursive maildir uidlist syncs caused assert crashes. Also did some Timo Sirainen <tss@iki.fi> parents: 3520 diff changeset	1480 /* NOTE: index syncing here might cause a re-sync due to
e9695ec7925b Recursive maildir uidlist syncs caused assert crashes. Also did some Timo Sirainen <tss@iki.fi> parents: 3520 diff changeset	1481 files getting lost, so this function might be called
e9695ec7925b Recursive maildir uidlist syncs caused assert crashes. Also did some Timo Sirainen <tss@iki.fi> parents: 3520 diff changeset	1482 re-entrantly. FIXME: and that breaks in
e9695ec7925b Recursive maildir uidlist syncs caused assert crashes. Also did some Timo Sirainen <tss@iki.fi> parents: 3520 diff changeset	1483 maildir_uidlist_sync_deinit() */
4238 3c8b191b0019 Adding mail to index while saving it had a race condition. Fixing it Timo Sirainen <timo.sirainen@movial.fi> parents: 4199 diff changeset	1484 ret = maildir_sync_index(ctx->index_sync_ctx, ctx->partial);
3c8b191b0019 Adding mail to index while saving it had a race condition. Fixing it Timo Sirainen <timo.sirainen@movial.fi> parents: 4199 diff changeset	1485 if (maildir_sync_index_finish(&ctx->index_sync_ctx,
4774 615b7738a62f Saving mails could have skipped over transactions, which caused different Timo Sirainen <tss@iki.fi> parents: 4612 diff changeset	1486 ret < 0, FALSE) < 0)
2173 9438951e243f Don't crash if we're syncing last commit to maildir, but some file was lost Timo Sirainen <tss@iki.fi> parents: 2140 diff changeset	1487 return -1;
4238 3c8b191b0019 Adding mail to index while saving it had a race condition. Fixing it Timo Sirainen <timo.sirainen@movial.fi> parents: 4199 diff changeset	1488
3c8b191b0019 Adding mail to index while saving it had a race condition. Fixing it Timo Sirainen <timo.sirainen@movial.fi> parents: 4199 diff changeset	1489 if (ret < 0)
3c8b191b0019 Adding mail to index while saving it had a race condition. Fixing it Timo Sirainen <timo.sirainen@movial.fi> parents: 4199 diff changeset	1490 return -1;
3436 3c51658d6846 We didn't notice if messages were deleted directly from new/. Timo Sirainen <tss@iki.fi> parents: 3435 diff changeset	1491 if (ret == 0)
3c51658d6846 We didn't notice if messages were deleted directly from new/. Timo Sirainen <tss@iki.fi> parents: 3435 diff changeset	1492 full_rescan = TRUE;
3447 96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	1493
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	1494 i_assert(maildir_uidlist_is_locked(ctx->mbox->uidlist));
2173 9438951e243f Don't crash if we're syncing last commit to maildir, but some file was lost Timo Sirainen <tss@iki.fi> parents: 2140 diff changeset	1495 }
1984 9c159272f721 syncing fixes Timo Sirainen <tss@iki.fi> parents: 1978 diff changeset	1496
4238 3c8b191b0019 Adding mail to index while saving it had a race condition. Fixing it Timo Sirainen <timo.sirainen@movial.fi> parents: 4199 diff changeset	1497 ret = maildir_uidlist_sync_deinit(&ctx->uidlist_sync_ctx);
3436 3c51658d6846 We didn't notice if messages were deleted directly from new/. Timo Sirainen <tss@iki.fi> parents: 3435 diff changeset	1498 return ret < 0 ? -1 : (full_rescan ? 0 : 1);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1499 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1500
3279 b698ae839a18 Moved mbox/maildir-specific variables from struct index_mailbox to Timo Sirainen <tss@iki.fi> parents: 3203 diff changeset	1501 int maildir_storage_sync_force(struct maildir_mailbox *mbox)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1502 {
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1503 struct maildir_sync_context *ctx;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1504 int ret;
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1505
3279 b698ae839a18 Moved mbox/maildir-specific variables from struct index_mailbox to Timo Sirainen <tss@iki.fi> parents: 3203 diff changeset	1506 ctx = maildir_sync_context_new(mbox);
3447 96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	1507 ret = maildir_sync_context(ctx, TRUE, FALSE);
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	1508 maildir_sync_deinit(ctx);
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	1509 return ret < 0 ? -1 : 0;
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	1510 }
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	1511
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	1512 int maildir_sync_last_commit(struct maildir_mailbox *mbox)
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	1513 {
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	1514 struct maildir_sync_context *ctx;
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	1515 int ret;
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	1516
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	1517 if (mbox->ibox.commit_log_file_seq == 0)
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	1518 return 0;
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	1519
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	1520 ctx = maildir_sync_context_new(mbox);
96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	1521 ret = maildir_sync_context(ctx, FALSE, TRUE);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1522 maildir_sync_deinit(ctx);
3436 3c51658d6846 We didn't notice if messages were deleted directly from new/. Timo Sirainen <tss@iki.fi> parents: 3435 diff changeset	1523 return ret < 0 ? -1 : 0;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1524 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1525
2322 aae574ed7f4c Broke mailbox_sync() into iterator. Timo Sirainen <tss@iki.fi> parents: 2320 diff changeset	1526 struct mailbox_sync_context *
aae574ed7f4c Broke mailbox_sync() into iterator. Timo Sirainen <tss@iki.fi> parents: 2320 diff changeset	1527 maildir_storage_sync_init(struct mailbox *box, enum mailbox_sync_flags flags)
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1528 {
3279 b698ae839a18 Moved mbox/maildir-specific variables from struct index_mailbox to Timo Sirainen <tss@iki.fi> parents: 3203 diff changeset	1529 struct maildir_mailbox mbox = (struct maildir_mailbox )box;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1530 struct maildir_sync_context *ctx;
2322 aae574ed7f4c Broke mailbox_sync() into iterator. Timo Sirainen <tss@iki.fi> parents: 2320 diff changeset	1531 int ret = 0;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1532
4894 24afafbfe47b Make sure the mailbox is opened when transaction is started (fixes deliver). Timo Sirainen <tss@iki.fi> parents: 4848 diff changeset	1533 if (!box->opened)
24afafbfe47b Make sure the mailbox is opened when transaction is started (fixes deliver). Timo Sirainen <tss@iki.fi> parents: 4848 diff changeset	1534 index_storage_mailbox_open(&mbox->ibox);
4848 967de900c73a Mailbox list indexing and related changes. Currently works only with Timo Sirainen <tss@iki.fi> parents: 4774 diff changeset	1535
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1536 if ((flags & MAILBOX_SYNC_FLAG_FAST) == 0 \|\|
3279 b698ae839a18 Moved mbox/maildir-specific variables from struct index_mailbox to Timo Sirainen <tss@iki.fi> parents: 3203 diff changeset	1537 mbox->ibox.sync_last_check + MAILBOX_FULL_SYNC_INTERVAL <=
b698ae839a18 Moved mbox/maildir-specific variables from struct index_mailbox to Timo Sirainen <tss@iki.fi> parents: 3203 diff changeset	1538 ioloop_time) {
b698ae839a18 Moved mbox/maildir-specific variables from struct index_mailbox to Timo Sirainen <tss@iki.fi> parents: 3203 diff changeset	1539 mbox->ibox.sync_last_check = ioloop_time;
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1540
3279 b698ae839a18 Moved mbox/maildir-specific variables from struct index_mailbox to Timo Sirainen <tss@iki.fi> parents: 3203 diff changeset	1541 ctx = maildir_sync_context_new(mbox);
3447 96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	1542 ret = maildir_sync_context(ctx, FALSE, FALSE);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1543 maildir_sync_deinit(ctx);
3436 3c51658d6846 We didn't notice if messages were deleted directly from new/. Timo Sirainen <tss@iki.fi> parents: 3435 diff changeset	1544
4152 e2edd333c473 Added MAILBOX_OPEN_KEEP_LOCKED flag to mailbox opening and implemented it Timo Sirainen <tss@iki.fi> parents: 4126 diff changeset	1545 i_assert(!maildir_uidlist_is_locked(mbox->uidlist) \|\|
e2edd333c473 Added MAILBOX_OPEN_KEEP_LOCKED flag to mailbox opening and implemented it Timo Sirainen <tss@iki.fi> parents: 4126 diff changeset	1546 mbox->ibox.keep_locked);
3447 96de02ea7482 Keywords are stored in maildir filename and maildir-keywords file Timo Sirainen <tss@iki.fi> parents: 3446 diff changeset	1547
3436 3c51658d6846 We didn't notice if messages were deleted directly from new/. Timo Sirainen <tss@iki.fi> parents: 3435 diff changeset	1548 if (ret == 0) {
3c51658d6846 We didn't notice if messages were deleted directly from new/. Timo Sirainen <tss@iki.fi> parents: 3435 diff changeset	1549 /* lost some files from new/, see if thery're in cur/ */
3c51658d6846 We didn't notice if messages were deleted directly from new/. Timo Sirainen <tss@iki.fi> parents: 3435 diff changeset	1550 ret = maildir_storage_sync_force(mbox);
3c51658d6846 We didn't notice if messages were deleted directly from new/. Timo Sirainen <tss@iki.fi> parents: 3435 diff changeset	1551 }
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1552 }
79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1553
2322 aae574ed7f4c Broke mailbox_sync() into iterator. Timo Sirainen <tss@iki.fi> parents: 2320 diff changeset	1554 return index_mailbox_sync_init(box, flags, ret < 0);
1915 79790750c349 importing new index code. mbox still broken. Timo Sirainen <tss@iki.fi> parents: diff changeset	1555 }
3472 db29cc6754d5 Store new/ directory's timestamp in sync_size header in index (kludgy..). Timo Sirainen <tss@iki.fi> parents: 3470 diff changeset	1556
db29cc6754d5 Store new/ directory's timestamp in sync_size header in index (kludgy..). Timo Sirainen <tss@iki.fi> parents: 3470 diff changeset	1557 int maildir_sync_is_synced(struct maildir_mailbox *mbox)
db29cc6754d5 Store new/ directory's timestamp in sync_size header in index (kludgy..). Timo Sirainen <tss@iki.fi> parents: 3470 diff changeset	1558 {
db29cc6754d5 Store new/ directory's timestamp in sync_size header in index (kludgy..). Timo Sirainen <tss@iki.fi> parents: 3470 diff changeset	1559 const char new_dir, cur_dir;
3863 55df57c028d4 Added "bool" type and changed all ints that were used as booleans to bool. Timo Sirainen <tss@iki.fi> parents: 3821 diff changeset	1560 bool new_changed, cur_changed;
55df57c028d4 Added "bool" type and changed all ints that were used as booleans to bool. Timo Sirainen <tss@iki.fi> parents: 3821 diff changeset	1561 int ret;
3472 db29cc6754d5 Store new/ directory's timestamp in sync_size header in index (kludgy..). Timo Sirainen <tss@iki.fi> parents: 3470 diff changeset	1562
db29cc6754d5 Store new/ directory's timestamp in sync_size header in index (kludgy..). Timo Sirainen <tss@iki.fi> parents: 3470 diff changeset	1563 t_push();
db29cc6754d5 Store new/ directory's timestamp in sync_size header in index (kludgy..). Timo Sirainen <tss@iki.fi> parents: 3470 diff changeset	1564 new_dir = t_strconcat(mbox->path, "/new", NULL);
db29cc6754d5 Store new/ directory's timestamp in sync_size header in index (kludgy..). Timo Sirainen <tss@iki.fi> parents: 3470 diff changeset	1565 cur_dir = t_strconcat(mbox->path, "/cur", NULL);
db29cc6754d5 Store new/ directory's timestamp in sync_size header in index (kludgy..). Timo Sirainen <tss@iki.fi> parents: 3470 diff changeset	1566
db29cc6754d5 Store new/ directory's timestamp in sync_size header in index (kludgy..). Timo Sirainen <tss@iki.fi> parents: 3470 diff changeset	1567 ret = maildir_sync_quick_check(mbox, new_dir, cur_dir,
db29cc6754d5 Store new/ directory's timestamp in sync_size header in index (kludgy..). Timo Sirainen <tss@iki.fi> parents: 3470 diff changeset	1568 &new_changed, &cur_changed);
db29cc6754d5 Store new/ directory's timestamp in sync_size header in index (kludgy..). Timo Sirainen <tss@iki.fi> parents: 3470 diff changeset	1569 t_pop();
db29cc6754d5 Store new/ directory's timestamp in sync_size header in index (kludgy..). Timo Sirainen <tss@iki.fi> parents: 3470 diff changeset	1570 return ret < 0 ? -1 : (!new_changed && !cur_changed);
db29cc6754d5 Store new/ directory's timestamp in sync_size header in index (kludgy..). Timo Sirainen <tss@iki.fi> parents: 3470 diff changeset	1571 }

Mercurial > dovecot > original-hg > dovecot-1.2

annotate src/lib-storage/index/maildir/maildir-sync.c @ 5588:6c89106dee21 HEAD